Add RasterCache option to RepaintBoundary widget

@liyuqian

Use case

Although RepaintBoundary is a key widget in improving Flutter’s performance on the UI thread, it’s not well-utilized on the GPU thread. For example, only the lowest-level RepaintBoundary, which corresponds to leaf-level engine PictureLayer, currently triggers the raster cache on the GPU thread. In certain scenarios, raster caching can make a big performance difference -- e.g., 20%-30% in our home_scroll_perf average_frame_rasterizer_time_millis benchmark.

With the recent introduction of retained EngineLayers, we now have the opportunity to use RepaintBoundary to trigger more raster caching on the higher level RepaintBoundary. Similar to the recent opacity and PhysicalShapeLayer optimizations, we may be able to improve our performance by multiple times.

Proposal

Add a RasterCache option to the RepaintBoundary widget, so the developer can decide when to engage the optimizations. This option is not only to override our current raster cache heuristics, which are not very smart or reliable, but also allows developers to elect to make the tradeoff between quality and speed themselves. For example, when there’s a rotation transformation above the RepaintBoundary, raster caching the whole RepaintBoundary means that we’re no longer drawing pixel-perfect graphics, but the performance gain may significantly outweigh the quality drop (see #24627).

Things to implement:

Write performance tests for the scenarios where we’d like to see improvements. This will guide our implementation and check our correctness and performance.
Add a RasterCache option to the RepaintBoundary widget and wire it up between the framework and the engine.
Enable the engine-side raster caching based on the option.
Document the feature, and update Github issues to reflect the performance improvement, to teach our developers how to do it themselves.

Proposed by @liyuqian

Thanks for polishing the proposal! Closing #27952 as a duplicate and add a reference to #27677

Allow RenderObject.layer to by any ContainerLayer

Paste Yegor's comment here for convenience: "It sounds a lot like will-change. Wondering if we could just map the proposed RasterCache to it."

Any progress on this?

@flar

@flar : I have the same question on the progress 😄

We just received some internal bug b/137395062 last week, and its current solution cl/260167188 is to manually take a snapshot of the blurred widget. Adding RasterCache option to RepaintBoundary widget would:

Make developer do much less work for these kind of issues. Potentially just one line of change on RepaintBoundary instead of 50 lines of code to snapshot manually.
Make CPU/GPU do much less work as our current snapshot requires GPU->CPU->GPU copies. And according to comments in cl/260167188, Flutter also strangely requires an unnecessary raw->PNG->raw encoding/decoding process. (I'll write a test and probably create a issue on that encoding issue).

@chinmaygarde

I'm looking into revisiting my RepaintBoundary caching work. Meanwhile I think that @chinmaygarde had some suggestions or feedback on faster ways to snapshot widgets that might help with the solution for b/137395062

Thanks for the proposal. Any ETA on the implementation?
While we can try to find workaround in the scenario like b/137395062, it is often counter intuitive. It is hard to be certain on what will be raster cached or not. Having an option in RepaintBoundary will be very helpful.

@liyuqian

@liyuqian @flar What is the status of this issue? What is the remaining work to do?

I've been working on various internal caching mechanisms that would work automatically behind the scenes and fixing up errors in our existing caching mechanisms. Since RepaintBoundaries aren't visible to the engine, a significant rework of our layer mechanism would be required to make them visible so that they can control caching.

@flar

@flar I just want to know the status of this issue, is it ready for us?

A related suggestion! Rather than a boolean RasterCache option, how about a cacheStrategy option, with CacheStrategy.default being the current behavior of "trying to figure out the best option itself", cacheStrategy.raster for the people wanting that previously in this bug, and for me, CacheStrategy.noCache, which paints the child the normal way on the existing buffer as if RepaintBoundary wasn't present.

The distinction between CacheStrategy.noCache and not having a RepaintBoundary is for the case of non-overlapping widgets, let's say animatingWidget and staticWidget[1-10]. With no RepaintBoundary, all ten staticWidgets repaint every frame because animatingWidget is updating. With a RepaintBoundary on animatingWidget there's a buffer allocated for animatingWidget; the buffer isn't providing anything useful because animatingWidget is repainting every frame anyway, but the RepaintBoundary is preventing the other widgets from repainting when they don't need it. So the RepaintBoundary serves a purpose, but the buffer does not.

The buffer, in this case, incurs a nontrivial memory cost and maybe also a performance cost.

@liyuqian

@liyuqian @flar Is there any update on this issue?

As a workaround, I created a widget that attaches cache hints (isComplex, willChange) to the current/parent RepaintBoundary like this:

import 'package:flutter/material.dart';
import 'package:flutter/rendering.dart';

/// set `isComplex`, `willChange` hints to current DisplayList
class RasterCacheHint extends SingleChildRenderObjectWidget {
  const RasterCacheHint({
    Key? key,
    this.isComplex,
    this.willChange,
  }) : super(key: key);

  final bool? isComplex;
  final bool? willChange;

  @override
  void updateRenderObject(
    BuildContext context,
    _RasterCacheHintRenderObject renderObject,
  ) {
    renderObject.isComplex = isComplex ?? false;
    renderObject.willChange = willChange ?? false;
  }

  @override
  RenderObject createRenderObject(BuildContext context) =>
      _RasterCacheHintRenderObject(isComplex, willChange);
}

class _RasterCacheHintRenderObject extends RenderProxyBox {
  _RasterCacheHintRenderObject(bool? _isComplex, bool? _willChange)
      : isComplex = _isComplex ?? false,
        willChange = _willChange ?? false;

  bool isComplex;
  bool willChange;

  @override
  void paint(PaintingContext context, Offset offset) {
    // we need this line to generate a PictureLayer to set the hints
    context.canvas;

    if (isComplex) {
      context.setIsComplexHint();
    }

    if (willChange) {
      context.setWillChangeHint();
    }

    super.paint(context, offset);
  }
}

Hi, I wonder whether the toPictureSync will solve this problem?

The raster cache is a legacy feature at this point (no plans on adding it to impeller) so I'm going to close this.

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

Navigation Menu

Repository navigation

Add RasterCache option to RepaintBoundary widget #32662

Use case

Proposal

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Add RasterCache option to RepaintBoundary widget #32662

Description

Use case

Proposal

Activity

liyuqian commented on May 14, 2019

liyuqian commented on May 28, 2019

arsen commented on Jul 28, 2019

liyuqian commented on Jul 29, 2019

flar commented on Jul 30, 2019

jason721017 commented on Jan 28, 2020

zanderso commented on Jun 18, 2020

flar commented on Jun 18, 2020

fredlee12345678 commented on Jul 29, 2020

ravenblackx commented on Sep 25, 2020

Ibtesam-Mahmood commented on Jun 29, 2022

mj-hd commented on Sep 29, 2022

fzyzcjy commented on Oct 1, 2022

Hixie commented on May 2, 2024

github-actions commented on May 16, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions