Repository navigation
Automatic/scalable shader warm-up #32170
Description
Current status: #32170 (comment)
Although Flutter's current shader warm-up process is theoretically able to eliminate all shader compilation jank, it requires too much expertise and time to find all draw operations for such process.
We shall find a more automatic and scalable way of finding those draw operations to compose a custom warm-up process. That should allow average developers without much Skia knowledge to eliminate the shader compilation on their own. That should also save Flutter engineers a lot of time as they'll no longer have to manually analyze the janky frame.
Note that this issue affects iOS devices more significantly than Android devices because Android has binary shader persistent cache while iOS doesn't.
We have a staged plan to roll this out, and we're actively working on it.
- SkSL-based shader warmup
- Test-based shader warmup
- In the long term, we'd like to create a solution that requires 0 extra effort from developers, and handles all cases that may have never been encountered during any tests or warmups. For example, use the CPU backend when GPU backend needs warm-up.
Activity
eseidelGoogle commentedon Aug 22, 2019
My understanding is that customer: dream is requesting a solution for their setup by end of November at least. I'm not sure if this is the right bug to track that or not?
liyuqian commentedon Aug 23, 2019
@eseidelGoogle : Yes, we just had a meeting this week with customer: dream and Skia. Here's the plan that we've discussed (please "goto" flutter-compile-janks internal doc for more details):
PaintingBinding.shaderWarmUp. However, we’ll not spend more sources on SKP based tools or solutions because the SKP capture itself is quite problematic:flutter screenshot --type=skiacan't record images in real devices #21237PaintingBinding.shaderWarmUp.eseidelGoogle commentedon Aug 26, 2019
Question: Do we need a 4 stage plan, or can we trade some short/medium term pain for doing fewer stages and skip one or more of these and earlier achieve the desired end result? FYI @Hixie
Or maybe all of these stages are just separate parts of the desired end result and not throw-away work?
liyuqian commentedon Aug 26, 2019
Quick update: stage 1 might be a past stage as we learnt that it may not be able to satisfy our clients next workload (at least 18 new shaders) in time. I'm now mainly investigating stage 2, which might not be re-usable in the long term but I really don't think stage 3 is possible before the deadline of customer: dream in September: I still don't see any estimation when stage 3's first blocker #38329 could be resolved.
liyuqian commentedon Aug 28, 2019
Skia has estimated that their work on stage 2 SkSL will take about 0.5 to 1 week. They're checking priorities to see when this could happen.
eseidelGoogle commentedon Sep 3, 2019
@liyuqian Do we have any updates here to share with customer: dream?
liyuqian commentedon Sep 4, 2019
The last update is on an email sent out last Friday (08/30) that CC'ed flutter-team and "customer: dream":
Brian Osman is going to do the Skia change this week. I'll follow on that to give a proof of concept probably by next week.
liyuqian commentedon Sep 9, 2019
Update: Skia has added the API (https://skia-review.googlesource.com/c/skia/+/239438). I'm going to prototype the Flutter change this week.
Implement Base32Decode
213 remaining items
esDotDev commentedon Jan 26, 2022
Has there been any consideration to doing this in more of a JIT way?
Historically, on platforms like Unity or Flash, we have solved this issue by showing some invisible components in frame 1, while our splash screen is showing, so their shaders/textures are uploaded to gpu. Then, when they appear in the app for the first time, they are already cached and ready to go, everything is 60fps butter. Users don't notice anything, except maybe a small stutter on the preload animation and an extra 100-500ms of startup time.
Seems like we could have a widget, that could take a stack of items, and cache all the shaders created from those items. Then write those down to disk. You could have a full 1000ms to do this, it would be no problem, and developers would happily make that tradeoff.
Then all of this complexity becomes something as simple as sticking a widget on your splash page, with some child widgets, and waiting until its cache is complete. You also would get device-specific shaders this way, which seems simpler than dealing with myriad edge cases. It also does not bloat the file size of the download.
Obviously developers would need to be careful with views that initiate network calls or other actions when mounted, but that is a much easier problem to solve, then all of this crazy build process and bundling complexity.
dnfield commentedon Jan 26, 2022
Yes, that has been considered, but it turns out that it's very difficult to capture just the right shaders that Skia will create for the specific things your app will draw. This gets very difficult to maintain, and it slows down startup time which our customers care about.
talksik commentedon Dec 29, 2022
no solution yet? I started building with flutter desktop and noticed the transition jank of animations way later.
been years with this issue right?
Timmmm commentedon Dec 29, 2022
They have tried various unsatisfying hacks like profile-guided precompilation, but it looks like they're finally started working on a real solution:
Impeller is a replacement for Skia that won't dynamically construct and compile shaders at runtime. Not at all ready yet but I'm very glad they've finally bit the bullet. The end is in sight at least!
zanderso commentedon Aug 11, 2023
SkSL caching and warm-up is no longer the approach the team is pursuing to solve early-onset jank.
ghenry commentedon Aug 11, 2023
Hixie commentedon Aug 11, 2023
Our new graphics backend, Impeller, doesn't compile shaders on startup (it doesn't compile them at runtime at all), so there's no need to warm up shaders any more once Impeller is available. So the plan is to get Impeller working on as many platforms as we can. It's on by default on iOS now, Android is next, then we'll look at the desktop platforms. macOS should be relatively easy. Windows might be tricky because of DirectX. I forget the state of Linux and web regarding Impeller.
Once Impeller is more widely deployed we'll remove the old shader warm-up logic (see #132418).
acoutts commentedon Aug 11, 2023
Can we continue to use sksl warmup in the meantime or does it not have any effect for iOS now?
chinmaygarde commentedon Aug 12, 2023
@acoutts:
Packaging SKSL has no effect on iOS. You can continue to do so but it will not affect rendering at all.
@m-j-g (in a comment that has since been deleted but I got a GitHub email notification for):
Assuming you are talking about #131319. This does not affect Impeller at ToT or in 3.13. It affects all versions of the legacy renderer. We are still trying to wrap our heads around how iOS behavior changed so drastically in a point release. But we know what fixes the issue (since Impeller already doesn't exhibit it at ToT) and are trying to figure out how/if we should backport it and investigate the same fix in the legacy renderer. If you are aware of an Impeller specific performance issue however, please file a bug and we'll investigate it immediately.
github-actions commentedon Aug 26, 2023
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of
flutter doctor -vand a minimal reproduction of the issue.