Skip to content

microbenchmarks_ios timing out ~40% of the time, white screen #76163

Closed

Description

@jmagman

https://luci-milo.appspot.com/raw/build/logs.chromium.org/flutter/led/godofredoc_google.com/15e28fdf5416728310250fe80d868d178b795977279a5341418a44dcbd936cb4/+/annotations

The runner is not progressing from:

[microbenchmarks_ios] [STDOUT] [  +58 ms] Successfully connected to service protocol: http://127.0.0.1:53924/
[microbenchmarks_ios] [STDOUT] [  +14 ms] Application running.
[microbenchmarks_ios] [STDOUT] [   +5 ms] Connected to iPhone
[microbenchmarks_ios] [STDOUT] [   +2 ms] Flutter run key commands.
[microbenchmarks_ios] [STDOUT] [   +1 ms] h Repeat this help message.
[microbenchmarks_ios] [STDOUT] [        ] c Clear the screen
[microbenchmarks_ios] [STDOUT] [        ] q Quit (terminate the application on the device).
[microbenchmarks_ios] [STDOUT] [        ] An Observatory debugger and profiler on iPhone is available at: http://127.0.0.1:53927/7vUpRFk2E18=/

Split off from #75851, which will track the task never being killed.

Activity

  1. added
    c: contributor-productivityTeam-specific productivity, code health, technical debt.
    platform-iosiOS applications specifically
    team: benchmarkPerformance issues found by inspecting benchmarks
    P2Important issues not at the top of the work list
    on Feb 17, 2021
  2. jmagman commented on Feb 17, 2021

    @jmagman
    MemberAuthor

    I tried to reproduce this locally and couldn't.

  3. godofredoc commented on Mar 2, 2021

    @godofredoc
    Contributor

    Seems to happenning very frequently ~40%. Are there any logs that we can collect to help triage?

    https://ci.chromium.org/p/flutter/builders/prod/Mac_ios%20microbenchmarks_ios

  4. changed the title [-]microbenchmarks_ios running for hours without timing out[/-] [+]microbenchmarks_ios timing out ~40% of the time[/+] on Mar 2, 2021
  5. jmagman commented on Mar 3, 2021

    @jmagman
    MemberAuthor

    A screenshot from the device while it is hanging would be helpful.
    I wish we could automatically collect that before the task times out.

  6. added and removed
    P2Important issues not at the top of the work list
    on Mar 3, 2021
  7. godofredoc commented on Mar 3, 2021

    @godofredoc
    Contributor

    Moving this one as P1 as it is the main contributor to Flutter tree closures.

  8. jmagman commented on Mar 4, 2021

    @jmagman
    MemberAuthor

    I tried this again and I can't get it to fail locally. Will probably need to ssh into a machine again.

  9. godofredoc commented on Mar 4, 2021

    @godofredoc
    Contributor

    Replicated the issue using led. The application starts opening but it gets up to the white screen and then it hangs there.

  10. jmagman commented on Mar 4, 2021

    @jmagman
    MemberAuthor

    @godofredoc Are there particular machines it always fails on?

  11. 8 remaining items

  12. added
    engineflutter/engine related. See also e: labels.
    and removed
    engineflutter/engine related. See also e: labels.
    on Mar 8, 2021
  13. chinmaygarde commented on Mar 8, 2021

    @chinmaygarde
    Contributor

    @jmagman From the trace attached, No frames were scheduled. Could it be one one of those benchmarks that starts paused and then starts its workload after a service protocol request? Not sure why things got kicked off when you touched the screen however. Can you attach a trace when the UI starts rendering?

  14. zanderso commented on Mar 10, 2021

    @zanderso
    Member

    @jmagman is it possible to get the logs during and after the touch that kickstarts the UI?

  15. jmagman commented on Mar 10, 2021

    @jmagman
    MemberAuthor

    @godofredoc said in chat:

    The issue can be easily replicated by running the test several times, usually it replicate in less than 10 iterations.

    I've been trying to reproduce this for an hour, my local setup doesn't seem to do it anymore.
    @zanderso I recommend getting someone on the engine team to work with @godofredoc to figure out which host machine is the best candidate, SSH in, and just reproduce it there if it's happening in less than 10 iterations. It won't be possible to touch the device, but at least you have a debugger attached and can poke at the app however is helpful, and collect traces.

  16. godofredoc commented on Mar 11, 2021

    @godofredoc
    Contributor

    @jmagman can you please share the command you are using to collect the traces?

  17. jmagman commented on Mar 11, 2021

    @jmagman
    MemberAuthor

    I went to the observatory service URL ("An Observatory debugger and profiler on <> is available at: http://127.0.0.1:60931/91n_cdDMTRI=/") , then timeline
    Screen Shot 2021-03-10 at 5 43 21 PM
    "Save" button to save the current results, or "Refresh" to get the results since the last run

    There's probably an easier way from the command line but I don't normally collect traces.

    I'm not sure what "Recorded Streams" @chinmaygarde or @zanderso would find helpful.

  18. chinmaygarde commented on Mar 15, 2021

    @chinmaygarde
    Contributor

    Per @jason-simmons, this is not occurring at 40%. The trace in a previous comment shows that the engine didn't get any request to render a frame. Since there isn't any actionable information to make progress on this issue, @zanderso is going to find an owner to track this down. Given the rate of occurrence, I am lowering the priority of this issue.

  19. added
    P1High-priority issues at the top of the work list
    and removed on Mar 15, 2021
  20. mdebbar commented on Mar 15, 2021

    @mdebbar
    Contributor

    I saw another recipe timeout but in Mac_android this time: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac_android%20home_scroll_perf__timeline_summary/529/overview (issue filed here). Not sure if it's the same cause or not. It passed after re-running.

    Edit: Here's another Mac_android recipe timeout: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac_android%20microbenchmarks/547/overview

  21. jmagman commented on Apr 21, 2023

    @jmagman
    MemberAuthor

    Haven't seen activity here for awhile, closing.

  22. github-actions commented on May 5, 2023

    @github-actions

    This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

  23. locked as resolved and limited conversation to collaborators on May 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High-priority issues at the top of the work listc: contributor-productivityTeam-specific productivity, code health, technical debt.engineflutter/engine related. See also e: labels.platform-iosiOS applications specificallyteam: benchmarkPerformance issues found by inspecting benchmarks

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions