Repository navigation
Momentary blank screen transitioning into a FlutterViewController for the first time with a prewarmed engine #32937
Description
Activity
cc @chinmaygarde, I'll probably come bug you with this at some point.
This has to do because the UIView is being presented before a vsync has triggered a presentRenderBuffer.
We need to somehow either:
- prepopulate the renderbuffer before the UIView is presented
- pause UIView presentation on the main thread until the contents of the renderbuffer are correct
Relevant stack overflow: https://stackoverflow.com/questions/5115408/iphone-opengl-first-frame-black
Is this a deterministically one frame latency issue or does it have an undeterministic delay?
If the former, I wonder if we can get the sequence right be triggering something on viewWillAppear. If the latter, we might need to create something more holistic for rendering latencies both with pre-warmed and cold launched engines #33494.
@xster It is one frame. I tried moving the code from viewDidAppear to viewWillAppear in the FlutterViewController. That code doesn't have direct correspondence to starting to render a frame.
I've been so sidetracked, I plan on getting back to this. Maybe making sure the display link starts at viewWillAppear is a good avenue. I'm not sure that will guarantee we will see the first frame though like synchronization between the display link and the main thread would.
ummm, it seems like we follow the same approach with the https://api.flutter.dev/flutter/scheduler/SchedulerBinding/scheduleWarmUpFrame.html Dart side API. It ultimately translates into Animator.RequestFrame. The viewDidAppear code seems to do the same thing via snooping the "AppLifecycleState.resumed" message on the lifecycle channel. Perhaps a trace is needed to figure out the timing.
It seems like a lot of our rendering initialization that has to happen happens as a result of layout subviews. We are waiting to get dimensions before setting up rendering. For some things that is a true dependency, but not for all of them. One alternative is potentially de-tangling that dependency and try to do more work at viewWillAppear.
Previously it has been discussed that our first frame is extra slow because of shader compilation. That is an example operation that could be done before we know the dimensions of the view.
I was able to eliminate the black frame by forcing the main thread to synchronize on the gpu when it is presenting the view:
https://gist.github.com/gaaclarke/c9c7d306315aa4755a3d81ac60bb9cbc
I have to clean it up and make a more compelling PR for it. This will be another controversial change from my read.
With the preceding change I measured the amount of time the main thread will pause to wait for rendering. On the flutter sample project it introduces a ~43ms pause on the first frame, ~17ms pause on subsequent instances with a release engine.
Blocking an app's main thread for 43ms intuitively makes me think we should just solve #33494 instead. Though another aspect could be #32946. We should take a look at what is the delta between having run FlutterEngine.runWithEntrypoint and FlutterViewController.initWithEngine + viewWillAppear and determine whether the list of conceptual actions taken in that delta is indeed only rendering related. i.e. did we prewarm as much as we could without consuming graphics memory at the FlutterEngine.runWithEntrypoint point.
I am against any sort of splash screen. That's amateur imo, you can't ask clients to put a splash screen in the middle of their app.
In addition to synchronizing the main thread with the renderer we need to make the rendering of the first frame faster. We already have outstanding work to do that. We should be able to get the first frame down to the same time as the second frame. I think most of that time is shader compilation. No matter how fast you make it though you'll still need to do thread synchronization.
Also, 43ms is acceptable for a one time event. Video game controllers have up to 20 ms lag which doesn't even include screen lag which for a tv will probably be 30ms at a minimum, and that's for continuous feedback.
Optimizing the first frame render time is definitely preferred. Though I think there will be circumstances where it can't be the only option. For instance, there will be times when an engine can't be pre-warmed. Our users' analytics is showing that a cold launch + first frame render could take 2s on mid-tier Androids. We can't avoid making a choice for what to show, whether it's the previous frame frozen or a blank/spinner/loading native screen.
Not saying this applies to this situation here since it's likely more solvable. Can we check what this time is on an iPhone 4s or some such?
I was running my tests on iPhone SE (low end device from 3 years ago). I'll keep an eye out for a 4s. Do we have statistics about what percentage of our user base is iPhone 4s?
Cool. Unfortunately we don't have any analytics on our users' users. Since China's our biggest present add-to-app market, we can likely use China's iPhone model market share as proxy.
I think a combination of flutter-team-archive/engine#9506 and flutter-team-archive/engine#9813 should give developers the tools to eliminate this. 9506 made it so the main thread waits for the first frame to render and 9813 makes it so people can correctly pre-warm the shaders in case the first frame is still taking over 100ms.
My hope is that it is not even a thought for most developers. We'll have to keep our ears open. Yuqian is still investigating making the first frame faster. We might be able to program in more levers for developers and make our default shader warmup better too.
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of
flutter doctor -vand a minimal reproduction of the issue.- locked as resolved and limited conversation to collaborators
on Aug 28, 2021
It seems like pre-warming the engine isn't enough and there's still something that loads only once the FlutterViewController is being shown.
Only reproducible on physical devices.
in https://github.com/xster/flutter-test/tree/master/t130_leaf_flutter_ios, press the go to flutter button