[Impeller] Move image upload scheduling waits to GPU disable by FelixMittermeier · Pull Request #190445 · flutter/flutter

FelixMittermeier · 2026-08-03T15:33:33Z

Metal image uploads currently wait synchronously for every upload command buffer to reach the scheduled state. That wait was added to protect an iOS backgrounding race, but paying it after every foreground upload unnecessarily blocks image decoding and can delay images becoming ready.

This PR moves that synchronization to the lifecycle boundary where it is required:

foreground image uploads commit their Metal command buffer, register a scheduled-or-terminal receipt, and return without waiting;
when GPU access changes to disabled, ContextMTL synchronously drains any still-pending image-upload receipts before the transition returns;
receipt registration remains inside the enabled side of the GPU SyncSwitch, so a concurrent disable transition cannot miss an upload that was just committed;
existing GLES tracking-fence and glFlush/glFinish behavior is unchanged.

This is a deeper, correctness-preserving follow-up to my deliberately oversimplified first experiment, #181601. That PR demonstrated the performance opportunity by only changing block_on_schedule to false, but it did not replace the iOS lifecycle barrier and did not include the required tests. This PR implements the receipt-based design proposed during the original review and verifies it with deterministic concurrency tests and repeated physical-device measurements.

Related lifecycle work includes the original magenta-image report #166668, the scheduling wait in #169378, the deferred-task race fix in #169596, their stable cherry-pick #170846, and the separate initial GPU-state fix #171785. Older Impeller synchronization discussions #120399, #120406, and #131490 are background context only and are not closed by this PR.

Why was every upload waiting?

The original report, #166668, described images becoming pink/magenta after an iOS application was awakened in the background and later foregrounded.

#169378 addressed a narrow commit-versus-background transition race:

A notification can wake an application while it is still in the background.
Image decoding starts while the engine still considers GPU access available.
The decoder commits a Metal upload command buffer.
The application can re-enter the background before Metal acknowledges the command buffer as scheduled.
The upload can then fail under iOS background GPU restrictions, leaving a persistent magenta image.

The fix submitted image uploads with block_on_schedule=true. Because the upload runs inside the shared side of the GPU SyncSwitch, waiting before that handler returns prevents the switch's exclusive disable transition from passing until the upload is scheduled.

That is correct, but expensive: every normal upload performs the native [MTLCommandBuffer waitUntilScheduled], even when no lifecycle transition is happening.

This cost was called out in the original review. In particular:

The tech-debt issue #170599 was then filed specifically to remove this extra foreground thread blocking. This PR implements that deferred design rather than removing synchronization.

Relationship to the previous attempt

#181601 only disabled the per-upload wait. It found a very large reduction in the pathological UploadTextureToPrivate span, proving that the effective Metal wait was in the submission path. However, the review correctly identified two blockers:

It was closed because there was no follow-up at that time.

An automated review also suggested removing a later generic command_buffer->WaitUntilScheduled() call. That suggestion does not account for backend behavior: Metal's generic WaitUntilScheduled() and WaitUntilCompleted() hooks are no-ops in this path, while the same block has real glFlush/glFinish behavior on GLES. This PR leaves that backend-specific visibility behavior intact.

Correctness invariant

When a transition to GPU-disabled returns, every tracked image-upload Metal command buffer committed while the GPU SyncSwitch was enabled must have reached either:

the scheduled state; or
a terminal completed/failed state from which it cannot later begin GPU work.

At the same time, ordinary foreground image uploads must not synchronously wait for scheduling or completion.

How the new ordering works

Foreground: while holding the enabled/shared SyncSwitch lock, install the Metal handlers, commit the upload, register its receipt, and return.
GPU disable: the exclusive switch transition first waits for those shared handlers to finish, then synchronously drains every pending receipt before returning.

The SyncSwitch lock ordering is the proof that registration cannot be missed: SetSwitch(true) cannot acquire its unique lock until an upload's shared handler has committed and registered its receipt. Observer notification is synchronous, so the disable transition cannot return before those receipts are safe. Receipt waiting happens outside both the switch lock and tracker mutex.

Implementation

CommandBufferSchedulingReceipt provides a thread-safe, idempotent scheduled-or-terminal signal. Scheduling, completion, and failure all wake waiters, and callbacks run outside its mutex.
CommandBufferMTL installs scheduled and completed handlers before commit and returns the receipt without waiting or creating a retain cycle.
PendingImageUploadScheduleTracker tracks every outstanding upload receipt, handles callback/registration races, uses weak callback state, and waits only after releasing its mutex.
ContextMTL drains the tracker when GPU access is disabled while preserving the existing deferred-task behavior when it is enabled again.
SubmitWithReceipt replaces the generic block_on_schedule Boolean. Metal returns a real receipt; Vulkan and generic fallbacks preserve their existing submission paths; GLES fence and flush/finish behavior is unchanged.
Focused traces cover foreground submission, lifecycle drains, pending counts, and individual waits.

Positive impact

This changes where synchronization is paid, rather than weakening it:

image-decode workers return after commit and receipt registration instead of stalling on Metal scheduling for every image;
batches become ready sooner, with the largest gains for generated pixels, camera buffers, and other upload-heavy sources;
ordinary JPEG and disk-cached image paths also show repeatable end-to-end gains;
the rare GPU-disable transition waits only for image uploads that are still unscheduled, and only until scheduled/terminal—not until GPU completion;
the original backgrounding guarantee is retained and GLES behavior is unchanged.

There is no Dart or public Flutter framework API change.

Performance

All measurements below used profile builds with Impeller on the same wired physical iPhone 14 Pro Max running iOS 26.5. Lower is better.

Existing `large_image_changer` macrobenchmark: 30-run A/B/A

The repository macrobenchmark was run 10 times with the baseline, 10 times with this implementation, and 10 more times after returning to the baseline. The same trace surrounded the submission block in both variants.

Metric	Combined baseline	This PR	Change
Foreground submit p50	2.098 ms	0.024 ms	−98.86%
Foreground submit p99	2.855 ms	0.056 ms	−98.04%
Complete `UploadTextureToPrivate` p50	2.654 ms	0.559 ms	−78.94%
Complete `UploadTextureToPrivate` p99	4.355 ms	0.925 ms	−78.76%

The direct trace confirms that the native scheduling wait leaves the foreground critical path.

Image-upload stress suite: repeated A/B/A

A standalone A/B/A suite used three runs per engine and 120 image uploads per run. End-to-end values are run medians; native p50 values pool 360 events.

Workload	Median time until 120 images are ready	Synchronous `UploadTextureToPrivate` p50
Bundled JPEG + `ResizeImage`	768.692 → 726.815 ms (−5.4%)	505.5 → 60.0 µs (−88.1%)
Preseeded `CachedNetworkImageProvider` + `ResizeImage`	847.037 → 799.973 ms (−5.6%)	477.0 → 56.0 µs (−88.3%)
Raw 512×512 RGBA + resize	111.705 → 70.902 ms (−36.5%)	547.5 → 141.0 µs (−74.2%)

Every changed run was faster than every baseline run. The raw-pixel result is an upload-heavy upper bound; the 5.4–5.6% JPEG results are more representative. Native percentages measure removed synchronous CPU blocking, not faster GPU copy throughput.

Standalone benchmark reproduction

This self-contained sample reproduces the upload-heavy raw-pixel workload. Run it in profile mode against baseline and PR engines on the same iOS device and compare elapsed_micros plus the UploadTextureToPrivate trace spans.

Expand standalone Flutter benchmark source and commands

Create a project with flutter create --platforms=ios impeller_upload_benchmark, then replace/add the following files.

`pubspec.yaml`

name: impeller_upload_benchmark
publish_to: none

environment:
  sdk: '>=3.4.0 <4.0.0'

dependencies:
  flutter:
    sdk: flutter

dev_dependencies:
  flutter_driver:
    sdk: flutter
  flutter_test:
    sdk: flutter
  test: any

flutter:
  uses-material-design: true

`lib/main.dart`

import 'dart:convert';
import 'dart:ui' as ui;

import 'package:flutter/foundation.dart';
import 'package:flutter/material.dart';

final GlobalKey<BenchmarkPageState> benchmarkKey =
    GlobalKey<BenchmarkPageState>();

void main() {
  WidgetsFlutterBinding.ensureInitialized();
  runApp(const BenchmarkApp());
}

Future<String> handleDriverMessage(String? message) async {
  switch (message) {
    case 'warmup':
      await benchmarkKey.currentState!.warmup();
      return 'ready';
    case 'run':
      return jsonEncode(await benchmarkKey.currentState!.runBenchmark());
    default:
      throw UnsupportedError('Unsupported driver message: $message');
  }
}

class BenchmarkApp extends StatelessWidget {
  const BenchmarkApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      debugShowCheckedModeBanner: false,
      home: Scaffold(
        body: SafeArea(child: BenchmarkPage(key: benchmarkKey)),
      ),
    );
  }
}

class BenchmarkPage extends StatefulWidget {
  const BenchmarkPage({super.key});

  @override
  State<BenchmarkPage> createState() => BenchmarkPageState();
}

class BenchmarkPageState extends State<BenchmarkPage> {
  static const int imageCount = 120;
  static const int sourceWidth = 512;
  static const int sourceHeight = 512;
  static const int targetWidth = 860;
  static const int targetHeight = 1864;

  late final Uint8List _pixels;
  late List<ImageProvider<Object>> _providers;
  int _generation = 0;

  @override
  void initState() {
    super.initState();
    _pixels = _makePixels();
    _providers = _makeProviders();
  }

  Uint8List _makePixels() {
    final Uint8List pixels = Uint8List(sourceWidth * sourceHeight * 4);
    for (var offset = 0; offset < pixels.length; offset += 4) {
      final int pixel = offset ~/ 4;
      pixels[offset] = pixel & 0xFF;
      pixels[offset + 1] = (pixel >> 8) & 0xFF;
      pixels[offset + 2] = 0x80;
      pixels[offset + 3] = 0xFF;
    }
    return pixels;
  }

  List<ImageProvider<Object>> _makeProviders() {
    return List<ImageProvider<Object>>.generate(
      imageCount,
      (int index) => RawPixelsImageProvider(
        id: _generation * imageCount + index,
        pixels: _pixels,
        width: sourceWidth,
        height: sourceHeight,
        targetWidth: targetWidth,
        targetHeight: targetHeight,
      ),
      growable: false,
    );
  }

  Future<void> _reset() async {
    PaintingBinding.instance.imageCache
      ..clear()
      ..clearLiveImages();
    _generation += 1;
    _providers = _makeProviders();
    setState(() {});
    await WidgetsBinding.instance.endOfFrame;
  }

  Future<void> warmup() async {
    for (final ImageProvider<Object> provider in _providers.take(40)) {
      await precacheImage(provider, context);
    }
    await _reset();
    await Future<void>.delayed(const Duration(milliseconds: 500));
  }

  Future<Map<String, Object>> runBenchmark() async {
    await _reset();
    if (!mounted) {
      throw StateError('Benchmark page was disposed');
    }
    final Stopwatch stopwatch = Stopwatch()..start();
    for (final ImageProvider<Object> provider in _providers) {
      await precacheImage(provider, context);
    }
    stopwatch.stop();
    return <String, Object>{
      'image_count': imageCount,
      'elapsed_micros': stopwatch.elapsedMicroseconds,
    };
  }

  @override
  Widget build(BuildContext context) {
    return Center(
      key: const ValueKey<String>('benchmark-ready'),
      child: Text('Ready (generation $_generation)'),
    );
  }
}

@immutable
class RawPixelsImageProvider extends ImageProvider<RawPixelsImageProvider> {
  const RawPixelsImageProvider({
    required this.id,
    required this.pixels,
    required this.width,
    required this.height,
    required this.targetWidth,
    required this.targetHeight,
  });

  final int id;
  final Uint8List pixels;
  final int width;
  final int height;
  final int targetWidth;
  final int targetHeight;

  @override
  Future<RawPixelsImageProvider> obtainKey(ImageConfiguration configuration) {
    return SynchronousFuture<RawPixelsImageProvider>(this);
  }

  @override
  ImageStreamCompleter loadImage(
    RawPixelsImageProvider key,
    ImageDecoderCallback decode,
  ) {
    return MultiFrameImageStreamCompleter(
      codec: _loadCodec(),
      scale: 1,
      debugLabel: 'RawPixelsImageProvider($id)',
    );
  }

  Future<ui.Codec> _loadCodec() async {
    final ui.ImmutableBuffer buffer = await ui.ImmutableBuffer.fromUint8List(
      pixels,
    );
    final ui.ImageDescriptor descriptor = ui.ImageDescriptor.raw(
      buffer,
      width: width,
      height: height,
      pixelFormat: ui.PixelFormat.rgba8888,
    );
    try {
      return await descriptor.instantiateCodec(
        targetWidth: targetWidth,
        targetHeight: targetHeight,
      );
    } finally {
      descriptor.dispose();
      buffer.dispose();
    }
  }
}

`test_driver/app.dart`

import 'package:flutter_driver/driver_extension.dart';
import 'package:impeller_upload_benchmark/main.dart' as app;

void main() {
  enableFlutterDriverExtension(handler: app.handleDriverMessage);
  app.main();
}

`test_driver/benchmark_test.dart`

import 'dart:convert';
import 'dart:io';

import 'package:flutter_driver/flutter_driver.dart';
import 'package:test/test.dart' hide TypeMatcher, isInstanceOf;

Future<void> main() async {
  late FlutterDriver driver;

  setUpAll(() async {
    driver = await FlutterDriver.connect();
    await driver.waitUntilFirstFrameRasterized();
    await driver.waitFor(find.byValueKey('benchmark-ready'));
  });

  tearDownAll(() async {
    await driver.close();
  });

  test('measure raw image upload scheduling', () async {
    expect(await driver.requestData('warmup'), 'ready');
    await driver.forceGC();

    late String response;
    final Timeline timeline = await driver.traceAction(() async {
      response = await driver.requestData(
        'run',
        timeout: const Duration(minutes: 5),
      );
    }, streams: const <TimelineStream>[TimelineStream.embedder]);

    final Map<String, dynamic> result =
        jsonDecode(response) as Map<String, dynamic>;
    expect(result['image_count'], 120);
    expect(result['elapsed_micros'], greaterThan(0));

    await Directory('build').create(recursive: true);
    await File('build/raw_upload_result.json')
        .writeAsString(const JsonEncoder.withIndent('  ').convert(result));
    await TimelineSummary.summarize(timeline)
        .writeTimelineToFile('raw_upload', pretty: true, includeSummary: false);
    // Also keep the value visible in CI/terminal logs.
    // ignore: avoid_print
    print(response);
  }, timeout: Timeout.none);
}

Run against each local engine

flutter pub get

flutter drive \
  --profile \
  -d <device-id> \
  --local-engine-src-path=<flutter-checkout>/engine/src \
  --local-engine=<baseline-ios-profile-output-name> \
  --local-engine-host=<host-profile-output-name> \
  -t test_driver/app.dart \
  --driver test_driver/benchmark_test.dart

# Repeat with only --local-engine changed to the PR engine output.

Each run writes:

build/raw_upload_result.json
build/raw_upload.timeline.json

Use at least three runs per engine and return to the baseline after the changed phase. In the timeline, pair B/E events named UploadTextureToPrivate by PID and thread ID before calculating percentiles.

Pre-launch Checklist

Metal image uploads currently wait for command-buffer scheduling after every submission. This prevents uploads from crossing an iOS GPU-disable transition, but it also blocks foreground image decoding. Return a scheduled-or-terminal receipt from Metal command-buffer submission and register image-upload receipts while holding the enabled GPU SyncSwitch lock. Drain outstanding receipts synchronously when GPU access becomes disabled. Remove the obsolete block_on_schedule plumbing while preserving GLES tracking-fence behavior and Vulkan's specialized queue submission path. Fixes flutter#170599

Add deterministic coverage for scheduling receipt state transitions, multiple waiters, terminal completion, callback races, and tracker teardown. Verify that GPU disable cannot miss receipt registration or return while protected image uploads remain unscheduled. Also cover real Metal receipt submission, image-decoder registration, and Vulkan's specialized queue path.

gemini-code-assist

Code Review

This pull request introduces CommandBufferSchedulingReceipt and PendingImageUploadScheduleTracker to track the scheduling state of submitted command buffers, particularly for image uploads in the Metal backend, allowing the engine to drain pending uploads when the GPU is disabled. The CommandBuffer and CommandQueue interfaces have been updated to support submitting commands with receipts. Feedback on the changes includes a high-severity use-after-free bug in context_mtl.mm where a temporary string's buffer is passed to TRACE_EVENT1, and a style guide recommendation to explicitly include <cstddef> in pending_image_upload_schedule_tracker.h for size_t.

gaaclarke

This is looking great, thanks for the well thought out PR. I have some mostly style notes.

FelixMittermeier · 2026-08-05T11:57:53Z

@gaaclarke Thanks for the fast review! I’ve addressed the suggestions in the latest commit.

gaaclarke · 2026-08-05T16:31:48Z

@FelixMittermeier in the future please make sure to press the "re-request review" button to make sure reviewers see you are ready for another look. We get so many messages, messages are easy to miss =)

Also, it's helpful if you respond to comments to show they are addressed.

gaaclarke

lgtm! thanks @FelixMittermeier, I'll find a second reviewer.

FelixMittermeier · 2026-08-05T19:00:40Z

@gaaclarke Alright, thanks for the hint. Will keep that in mind for the future to re-request reviews and write comment responses 👍

gaaclarke · 2026-08-05T19:59:48Z

fyi, from dashboard checks:

❌ Failures for clang-tidy on /b/s/w/ir/cache/builder/engine/src/flutter/impeller/renderer/command_buffer_scheduling_receipt.cc:
/b/s/w/ir/cache/builder/engine/src/flutter/impeller/renderer/command_buffer_scheduling_receipt.cc:42:7: error: 'callback' used after it was moved [bugprone-use-after-move,-warnings-as-errors]
   42 |       callback();
      |       ^
/b/s/w/ir/cache/builder/engine/src/flutter/impeller/renderer/command_buffer_scheduling_receipt.cc:36:20: note: move occurred here
   36 |         callbacks_.push_back(std::move(callback));
      |                    ^
Suppressed 65 warnings (65 in non-user code).
Use -header-filter=.* or leave it as default to display errors from all non-system headers. Use -system-headers to display errors from system headers as well.
1 warning treated as error

FelixMittermeier · 2026-08-07T12:42:25Z

Sorry for the re-approval churn. I pushed a small follow-up to fix the mac_clang_tidy failure.

FelixMittermeier · 2026-08-11T07:29:34Z

Looks like the Mac check just hit a 404 fetching sky_engine.zip during setup. Probably just needs a rerun — I don’t have the permissions to retrigger it though.

…#12453) Manual roll Flutter from 27b098811f3b to c2437523d308 (179 revisions) Manual roll requested by [email protected] flutter/flutter@27b0988...c243752 2026-08-12 [email protected] Enable Gradle CI cache on all test targets that require android_sdk (flutter/flutter#190723) 2026-08-12 [email protected] [analysis] Reland "Added initial implementation of the flutter_analyzer_plugin (#175679)" (flutter/flutter#191022) 2026-08-12 [email protected] Switch testing to gradle bin distribution type instead of all (flutter/flutter#190738) 2026-08-12 [email protected] Convert Mockito instances in Kotlin to Mockk (flutter/flutter#189884) 2026-08-12 [email protected] Report individual test results to LUCI ResultDB (flutter/flutter#190254) 2026-08-12 [email protected] Toggleable reaction duration respects overrides (flutter/flutter#190857) 2026-08-12 [email protected] Roll Skia from e00dbd7448c4 to fee7272f5bc2 (1 revision) (flutter/flutter#191007) 2026-08-12 [email protected] Started caching text shadows by content. (flutter/flutter#190681) 2026-08-12 [email protected] Adds agent skill for spawning led tasks. (flutter/flutter#190937) 2026-08-12 [email protected] Roll Packages from aaaf246 to 94485f1 (8 revisions) (flutter/flutter#191008) 2026-08-12 [email protected] flutter_tools: validate plugin identifiers before generating registrant code (flutter/flutter#190462) 2026-08-12 [email protected] Roll Skia from 112f36148949 to e00dbd7448c4 (3 revisions) (flutter/flutter#190993) 2026-08-12 [email protected] Roll Skia from 7d366c802307 to 112f36148949 (3 revisions) (flutter/flutter#190983) 2026-08-12 [email protected] remove bringup for flavors test (flutter/flutter#190940) 2026-08-12 [email protected] Roll Skia from 1f10a20bdd61 to 7d366c802307 (2 revisions) (flutter/flutter#190980) 2026-08-12 [email protected] Remove `--no-sim-use-hardfp` flag (flutter/flutter#190790) 2026-08-12 [email protected] Roll pub packages (flutter/flutter#190977) 2026-08-12 [email protected] Roll Skia from 339bedab6766 to 1f10a20bdd61 (1 revision) (flutter/flutter#190975) 2026-08-12 [email protected] RawTooltip respects AnimationStyle updates and reverseCurve (flutter/flutter#190889) 2026-08-12 [email protected] ci: Support --target_arch option in prepare_package.dart (flutter/flutter#190960) 2026-08-12 [email protected] Roll Fuchsia Linux SDK from SFq4FVodIOQAS26Lr... to -uHuSGv3wt7QAlDwa... (flutter/flutter#190973) 2026-08-12 [email protected] Removes building of ci/android_debug_x86 as nobody should be consuming it. (flutter/flutter#190951) 2026-08-12 [email protected] Adds error about wimp_heavy not being implemented. (flutter/flutter#189945) 2026-08-11 [email protected] Add clang, cmake, and ninja deps to Linux windowing_test (flutter/flutter#190119) 2026-08-11 [email protected] [AGP 9.1.0 Migration #1] Add Android Gradle Plugin Public API migration documentation (flutter/flutter#190842) 2026-08-11 [email protected] [flutter_tools] Fix deadlock in debug adapters when process exits early (flutter/flutter#190931) 2026-08-11 [email protected] [tool] Define modular dependency injection containers and bootstrapper (flutter/flutter#190724) 2026-08-11 [email protected] [web] Unify MaskFilter and ColorFilter primitives across CanvasKit and Skwasm (flutter/flutter#190314) 2026-08-11 [email protected] Roll pub packages (flutter/flutter#190958) 2026-08-11 [email protected] [Impeller] Move image upload scheduling waits to GPU disable (flutter/flutter#190445) 2026-08-11 [email protected] Started generating the windows platform for macrobenchmarks (flutter/flutter#190932) 2026-08-11 [email protected] [web] Unify ui.Vertices (flutter/flutter#190563) 2026-08-11 [email protected] Fix accessibility_inspector service extensions map mutability (flutter/flutter#190888) 2026-08-11 [email protected] Add batch3 a11y_assessment for vpat (flutter/flutter#189042) 2026-08-11 [email protected] Remove Xcode environment when building swift tools in Xcode pre-action (flutter/flutter#190848) 2026-08-11 [email protected] [tool] Add missing play element in web test index.html to fix warning (flutter/flutter#190675) 2026-08-11 [email protected] Offload blocking work in ProcessTextPlugin to the background (flutter/flutter#189823) 2026-08-11 [email protected] [flutter_tools] Replace usages of package:dds/dap.dart with package:dap_adapters/dap_adapters.dart (flutter/flutter#190667) 2026-08-11 [email protected] Always update swift package dependencies (flutter/flutter#190886) 2026-08-11 [email protected] [flutter_tools] Add --preset option to flutter test (flutter/flutter#190878) 2026-08-11 [email protected] Include the examples cross imports checker in the analzyer. (flutter/flutter#190674) 2026-08-11 [email protected] [devicelab] Remove orphaned screenshot test files (flutter/flutter#190879) 2026-08-11 [email protected] Remove the bringup flag from the linux_arm_host_desktop_engine builder (flutter/flutter#190935) 2026-08-11 [email protected] Reduce web_skwasm_tests subshards from 8 to 2 (flutter/flutter#190728) ...

…urce exhaustion Stability fixes for the Vulkan backend discovered through sustained stress testing on AMD RDNA 2 hardware and Mesa dzn (Vulkan on D3D12 in WSL2), first published as part of flutter#183382 and rebased onto the current FenceWaiterVK submission architecture. Device loss handling: - ContextVK gains an atomic device_lost_ flag. CreateCommandBuffer() and CommandQueueVK::Submit() short-circuit once the device is lost, since any Vulkan call on a driver that entered a corrupted OOM state can access-fault inside the ICD. - On vkQueueSubmit failure, tracked objects are abandoned without making Vulkan calls (AbandonForDriverCrash on command pools and descriptor pools), the device is marked lost, and per-heap memory budgets are logged for diagnosis. - FenceWaiterVK releases rather than destroys the fence when the submit callback fails; some drivers partially track the fence even though the submission failed, and vkDestroyFence then crashes (VUID-vkDestroyFence-fence-01120 observed on AMD). Submission backpressure: - CommandQueueVK caps concurrent in-flight submissions (3) with a condition variable, releasing the slot from the fence completion callback. Prevents unbounded host memory growth when the CPU outpaces the GPU. Only frame submissions through Submit() are gated; SubmitWithReceipt() bypasses the gate so image uploads from the IO thread stay non-blocking (flutter#190445). - ContextVK::FlushCommandBuffers() submits in chunks of 64 to keep per-submit driver allocations bounded on frames that accumulate very large command buffer counts. - GPUSurfaceVulkanImpeller (embedder-managed image path) adds a two-entry frame fence ring mirroring the KHR swapchain's backpressure, with a bounded wait that skips the frame under extreme GPU pressure. Image views for embedder images are now cached instead of being allocated (and leaked) per frame, and in-flight frames are drained before views are destroyed on resize or teardown. Command pool lifecycle: - BackgroundCommandPoolVK releases its handles without Vulkan calls when the recycler is gone; the device may already be destroyed at that point. This is the same class of teardown race as the CommandPoolVK destructor fix that landed in flutter#186749. - The recycled pool list is capped at 8 entries; once the cache is full, further reclaimed pools are dropped, and RAII destroys their recycled buffers before the pool. Evicting from the vector instead would move-assign RecycledData in declaration order, destroying the overwritten entry's pool before freeing its buffers - a use-after- free inside the driver that corrupted the heap under SwiftShader in the canvas_test dart suite. A regression test asserts the free- before-destroy ordering on the drop path. - unused_command_buffers_ is annotated IPLR_GUARDED_BY(pool_mutex_). Descriptor pools: - Reset via vkResetDescriptorPool on reclaim. Reclaimed pools previously stayed at capacity and produced VK_ERROR_OUT_OF_POOL_MEMORY on every reuse. Barrier and render pass correctness: - Six barrier fixes in the blit pass: pre-copy barriers use transfer stage and access flags, post-copy barriers flush transfer writes, and mipmap generation barriers match the actual current layout. Fixes corrupted glyph atlases and SYNC-HAZARD-WRITE-AFTER-WRITE reported by the synchronization validation layer on AMD RDNA 2. - RenderPassBuilderVK moves subpass dependencies to a counted builder, drops eByRegion from external (VK_SUBPASS_EXTERNAL) dependencies where it has no meaning per Vulkan spec section 7.1, and gains SetFramebufferFetchEnabled() so input attachment references and self-dependencies are omitted on drivers without framebuffer fetch support. The flag is wired through pipeline and render pass creation and the compat render pass used for PSO construction. Driver workarounds: - DriverInfoVK exposes GetDriverID() (Vulkan 1.2) and detects Mesa dzn. - New skip_sub_region_buffer_to_image_copy workaround: Mesa dzn reports minImageTransferGranularity of (0,0,0) and corrupts textures on sub-region buffer-to-image copies; the blit pass stages such copies through a full-size temporary texture. Memory management: - The VMA buffer pool is capped at 256 blocks (about 1 GB); the pool otherwise grows without bound when high-water-mark demand spikes and fragmentation prevents block reuse. - Image allocation failure paths no longer leak the VMA allocation when image view creation fails. - On Windows, the working set is periodically trimmed when Resizable BAR mappings inflate RSS far beyond actual VMA usage (gated by cooldown, RSS threshold, and RSS-to-VMA ratio). KHR swapchain robustness: - The synchronizer fence is re-signaled with an empty submit on every early return after a failed or unusable acquireNextImageKHR (including the surface-lost path added in flutter#183338). The fence was reset by WaitForFence, and without a new signal the next wrap-around to the same synchronizer deadlocks in waitForFences with an infinite timeout. - AcquireNextDrawable short-circuits when the device is marked lost. - Zero-extent surfaces (minimized windows) skip the frame instead of attempting swapchain recreation, preserving recovery on restore. Embedder mode: - CapabilitiesVK enumerates instance layers in embedder mode so that validation layers can be detected and enabled when requested.

FelixMittermeier added 2 commits August 3, 2026 14:25

github-actions Bot added engine flutter/engine related. See also e: labels. e: impeller Impeller rendering backend issues and features requests flutter-gpu team-fluttergpu Owned by Flutter GPU team labels Aug 3, 2026

github-project-automation Bot added this to Flutter GPU Aug 3, 2026

github-project-automation Bot moved this to 🤔 Needs Triage in Flutter GPU Aug 3, 2026

gemini-code-assist Bot reviewed Aug 3, 2026

View reviewed changes

Comment thread engine/src/flutter/impeller/renderer/backend/metal/context_mtl.mm Outdated

Comment thread engine/src/flutter/impeller/renderer/backend/metal/pending_image_upload_schedule_tracker.h Outdated

[Impeller] Address review feedback

c25509f

gaaclarke requested review from bdero and gaaclarke and removed request for bdero August 3, 2026 18:11

gaaclarke added the CICD Run CI/CD label Aug 4, 2026

Merge branch 'master' into impeller-metal-upload-scheduling

2244ae0

flutter-dashboard Bot removed the CICD Run CI/CD label Aug 4, 2026

gaaclarke added the CICD Run CI/CD label Aug 4, 2026

gaaclarke reviewed Aug 4, 2026

View reviewed changes

Review fixes

bfbd1b5

flutter-dashboard Bot removed the CICD Run CI/CD label Aug 5, 2026

gaaclarke added the CICD Run CI/CD label Aug 5, 2026

gaaclarke self-requested a review August 5, 2026 16:31

gaaclarke previously approved these changes Aug 5, 2026

View reviewed changes

jason-simmons reviewed Aug 5, 2026

View reviewed changes

Comment thread engine/src/flutter/impeller/renderer/backend/metal/pending_image_upload_schedule_tracker.cc

cbracken self-requested a review August 5, 2026 22:07

Address clang-tidy failure

8be3a56

FelixMittermeier dismissed stale reviews from jason-simmons, cbracken, and gaaclarke via 9f5683b August 7, 2026 12:37

flutter-dashboard Bot removed the CICD Run CI/CD label Aug 7, 2026

FelixMittermeier requested review from cbracken, gaaclarke and jason-simmons August 7, 2026 12:42

gaaclarke added the CICD Run CI/CD label Aug 8, 2026

Merge branch 'master' into impeller-metal-upload-scheduling

92ba599

flutter-dashboard Bot removed the CICD Run CI/CD label Aug 10, 2026

gaaclarke approved these changes Aug 10, 2026

View reviewed changes

gaaclarke added the CICD Run CI/CD label Aug 10, 2026

jason-simmons approved these changes Aug 10, 2026

View reviewed changes

jason-simmons added the autosubmit Merge PR when tree becomes green via auto submit App label Aug 11, 2026

auto-submit Bot added this pull request to the merge queue Aug 11, 2026

Merged via the queue into flutter:master with commit 63c2ac1 Aug 12, 2026
30 of 31 checks passed

github-project-automation Bot moved this from 🤔 Needs Triage to ✅ Done in Flutter GPU Aug 12, 2026

flutter-dashboard Bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Aug 12, 2026

sero583 mentioned this pull request Aug 17, 2026

[Impeller] Harden the Vulkan backend against driver failures and resource exhaustion #189580

Draft

10 tasks done

JonasGrunau mentioned this pull request Aug 18, 2026

[Impeller][iOS] Picture.toImageSync rasterizes into a disabled GPU context and yields solid magenta — toImage defers, toImageSync does not #191255

Open

Conversation

FelixMittermeier commented Aug 3, 2026

Why was every upload waiting?

Relationship to the previous attempt

Correctness invariant

How the new ordering works

Implementation

Positive impact

Performance

Existing large_image_changer macrobenchmark: 30-run A/B/A

Image-upload stress suite: repeated A/B/A

Standalone benchmark reproduction

pubspec.yaml

lib/main.dart

test_driver/app.dart

test_driver/benchmark_test.dart

Run against each local engine

Pre-launch Checklist

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

gaaclarke left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

FelixMittermeier commented Aug 5, 2026

Uh oh!

gaaclarke commented Aug 5, 2026

Uh oh!

gaaclarke left a comment

Choose a reason for hiding this comment

Uh oh!

FelixMittermeier commented Aug 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gaaclarke commented Aug 5, 2026

Uh oh!

Uh oh!

FelixMittermeier commented Aug 7, 2026

Uh oh!

FelixMittermeier commented Aug 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Existing `large_image_changer` macrobenchmark: 30-run A/B/A

`pubspec.yaml`

`lib/main.dart`

`test_driver/app.dart`

`test_driver/benchmark_test.dart`

FelixMittermeier commented Aug 5, 2026 •

edited

Loading