et: Fix compile_commands.json post-processing by cbracken · Pull Request #192395 · flutter/flutter

cbracken · 2026-09-07T22:25:19Z

Previously, _postGn ran immediately after GN and rewrote GN's compile_commands.json in-place with the output of ninja -t compdb in order to pick up additional commands not emitted by gn but needed by language servers, such as swiftc invocations.

However, ninja -t compdb includes every edge in the build graph, including copy and link commands, not just compiles. LSPs key off the file field, and the additional non-compile edges produced additional entries for some files that shadow the compile command that LSPs need. Among others, some of the public header copy rules ended up causing symbols in embedder.h and others to stop resolving correctly.

This reverts the post-build rewriting of compile-commands.json to what it was before #189761: just stripping RBE rewrapper prefxies from C++/Objective-C++ compile commands. The in-place rewriting is still required because clang_tidy and clangd_check use it directly and don't understand rewrapper prefixes.

Once the in-place cleanup is done, we now run ninja -t compdb swift to get just the Swift compilation commands required rather than the whole graph, and tack those onto a new output at lsp/compile-commands.json which can be used by LSPs. This has the additional benefit of sticking around in case any tools (run_test.py for example) issue gn invocations directly, which stomps the compilation database.

Followup to: #189761
Fixes: #185741

Pre-launch Checklist

If you need help, consider asking for advice on the #hackers-new channel on Discord.

If this change needs to override an active code freeze, provide a comment explaining why. The code freeze workflow can be overridden by code reviewers. See pinned issues for any active code freezes with guidance.

Note: The Flutter team is currently trialing the use of Gemini Code Assist for GitHub. Comments from the gemini-code-assist bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed.

gemini-code-assist

Code Review

This pull request introduces a separate compilation database for language servers (lsp/compile_commands.json) that includes Swift targets, keeping it isolated from GN's raw database. It updates the build runner to execute post-processing after Ninja runs to prevent Ninja's regeneration from overwriting the post-processed database, and updates documentation, CI configurations, and tooling to reference the new LSP database path. Feedback recommends running the post-processing step even when runGn is false if Ninja implicitly regenerates the build files during its run.

Previously, `_postGn` ran immediately after GN and rewrote GN's `compile_commands.json` in-place with the output of `ninja -t compdb` in order to pick up additional commands not emitted by `gn` but needed by language servers, such as `swiftc` invocations. However, `ninja -t compdb` includes every edge in the build graph, including copy and link commands, not just compiles. LSPs key off the `file` field, and the additional non-compile edges produced additional entries for some files that shadow the compile command that LSPs need. Among others, some of the public header copy rules ended up causing symbols in `embedder.h` and others to stop resolving correctly. This reverts the post-build rewriting of `compile-commands.json` to what it was before flutter#189761: just stripping RBE rewrapper prefxies from C++/Objective-C++ compile commands. The in-place rewriting is still required because `clang_tidy` and `clangd_check` use it directly and don't understand rewrapper prefixes. Once the in-place cleanup is done, we now run `ninja -t compdb swift` to get *just* the Swift compilation commands required rather than the whole graph, and tack those onto a new output at `lsp/compile-commands.json` which can be used by LSPs. This has the additional benefit of sticking around in case any tools (run_test.py for example) issue `gn` invocations directly, which stomps the compilation database. Followup to: flutter#189761 Fixes: flutter#185741

flar · 2026-09-08T17:52:21Z

I'll celebrate, but John's probably a better reviewer.

Reading through the unfamiliar references in the description it appears that you didn't just revert, but you switched to a new process that is language tool friendly?

And we should now change our compile_commands.json references to the new file? Or can we still use the old file if we don't care about Swift editting (strictly C++ work)?

cbracken · 2026-09-08T21:11:30Z

Reading through the unfamiliar references in the description it appears that you didn't just revert, but you switched to a new process that is language tool friendly?

This is basically a revert + a fix forward that writes 23 additional lines into lsp/compile-commands.json. I updated the references in the docs, but I'll send a message to a few chats as well.

If you don't work on the iOS embedder, feel free to continue using the old compile-commands.json (it's back to exactly how it was), but there are some small advantages to using the new one aside from Swift. Running run_tests.py previously rewrote the old compile-commands.json in certain cases, which re-injects all the rewrapped RBE commands that LSPs don't understand. It doesn't break the new one.

jtmcdole · 2026-09-08T21:44:20Z

+
+/// Appends [entries] to the JSON array in [database].
+String _appendEntries(String database, List<Object?> entries) {
+  final int close = database.lastIndexOf(']');


jtmcdole · 2026-09-08T21:45:28Z

+    head = head.substring(0, head.length - 1);
+  }
+  final String rendered = entries.map(_jsonEncoder.convert).join(',\n  ');
+  return '$head${head.endsWith('[') ? '' : ','}\n  $rendered\n]\n';


:squint really hard:

jtmcdole

I believe that you wrote tests, but the json-like-string parsing has some smell to it. -1 codefu-bux.

flar · 2026-09-08T21:50:17Z

I can confirm that using the top level out/<variant>/compile_commands.json file now completes successfully in VSCode...

cbracken · 2026-09-08T22:12:30Z

I believe that you wrote tests, but the json-like-string parsing has some smell to it. -1 codefu-bux.

IIRC you wrote the original JSON like string parsing here because parsing 20 MB of JSON was slow json.decode. -1 codefu-bux for both of us!

jtmcdole · 2026-09-08T22:49:52Z

I believe that you wrote tests, but the json-like-string parsing has some smell to it. -1 codefu-bux.

IIRC you wrote the original JSON like string parsing here because parsing 20 MB of JSON was slow json.decode. -1 codefu-bux for both of us!

This is correct. For finding this gem, I grant you 1 codefu-bux

💵 The CodeFu LGTM Note

The LGTM exchange rates are currently trading at:

1 LGTM = golden images match closely enough if I squint.
2 LGTMs = I checked that this won't break the devicelab benchmarks
3 LGTMs = I trust that this C++ pointer won't leak memory every frame.
4 LGTMs = CI is green on the first try without a single retry or shard failure

💵 Value officially backed by the full faith, credit, and compute budget of the Flutter EngProd Infrastructure.

flutter/flutter@b444e78...63b9518 2026-09-09 [email protected] Roll Packages from 9af9c60 to 36e088a (24 revisions) (flutter/flutter#192485) 2026-09-09 [email protected] Re-add Xcode and OpenJDK dependencies to macOS wasm local_engine builders (flutter/flutter#192238) 2026-09-09 [email protected] Roll Dart SDK from 77094991d1c1 to 4be1b711b4ba (5 revisions) (flutter/flutter#192453) 2026-09-09 [email protected] [material] Use super parameters in more places (flutter/flutter#192149) 2026-09-09 [email protected] [cupertino] Use super parameters in more places (flutter/flutter#192148) 2026-09-09 [email protected] [flutter_tools] Gracefully handle EROFS and write-protected file system errors (flutter/flutter#192431) 2026-09-09 [email protected] [tool] Migrate BuildIOSCommand and BuildIOSArchiveCommand to modular dependency injection (flutter/flutter#192250) 2026-09-09 [email protected] Replace more enum instances comparison to use `equality ==` instead of `identical` and update related tests (flutter/flutter#192229) 2026-09-09 [email protected] [web] Support local screenshot and golden testing in felt test (flutter/flutter#191318) 2026-09-09 [email protected] [tool] Migrate DaemonCommand and DebugAdapterCommand to modular dependency injection (flutter/flutter#190793) 2026-09-09 [email protected] [web] Support local golden testing in flutter test (flutter/flutter#191386) 2026-09-09 [email protected] [flutter_tools] Ignore widget previews outside lib/ during LSP code generation (flutter/flutter#192326) 2026-09-09 [email protected] [flutter_tools] Read package_config.json asynchronously in loadPackageConfigWithLogging (flutter/flutter#192443) 2026-09-08 [email protected] Add precacheTestImage to flutter_test and unskip web image tests (flutter/flutter#191435) 2026-09-08 [email protected] et: Fix compile_commands.json post-processing (flutter/flutter#192395) 2026-09-08 [email protected] Refactor createDebuggingOptions and option registrations to OptionDescriptor (flutter/flutter#192307) 2026-09-08 [email protected] [flutter_tools] Fix test separation between `VersionFreshnessValidator` and `FlutterVersion.checkFlutterVersionFreshness` (flutter/flutter#191593) 2026-09-08 [email protected] [tool] Migrate EmulatorsCommand to modular dependency injection (flutter/flutter#190769) 2026-09-08 [email protected] [tool] Migrate Desktop build subcommands and toolchain to modular dependency injection (flutter/flutter#190781) 2026-09-08 [email protected] [tool] Migrate Tooling and Analysis commands to modular dependency injection (flutter/flutter#190788) 2026-09-08 [email protected] Skip KGP version validation for AGP built-in Kotlin (flutter/flutter#192206) If this roll has caused a breakage, revert this CL and set the roller to dry run mode using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC [email protected],[email protected] on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

…r#12814) flutter/flutter@b444e78...63b9518 2026-09-09 [email protected] Roll Packages from 9af9c60 to 36e088a (24 revisions) (flutter/flutter#192485) 2026-09-09 [email protected] Re-add Xcode and OpenJDK dependencies to macOS wasm local_engine builders (flutter/flutter#192238) 2026-09-09 [email protected] Roll Dart SDK from 77094991d1c1 to 4be1b711b4ba (5 revisions) (flutter/flutter#192453) 2026-09-09 [email protected] [material] Use super parameters in more places (flutter/flutter#192149) 2026-09-09 [email protected] [cupertino] Use super parameters in more places (flutter/flutter#192148) 2026-09-09 [email protected] [flutter_tools] Gracefully handle EROFS and write-protected file system errors (flutter/flutter#192431) 2026-09-09 [email protected] [tool] Migrate BuildIOSCommand and BuildIOSArchiveCommand to modular dependency injection (flutter/flutter#192250) 2026-09-09 [email protected] Replace more enum instances comparison to use `equality ==` instead of `identical` and update related tests (flutter/flutter#192229) 2026-09-09 [email protected] [web] Support local screenshot and golden testing in felt test (flutter/flutter#191318) 2026-09-09 [email protected] [tool] Migrate DaemonCommand and DebugAdapterCommand to modular dependency injection (flutter/flutter#190793) 2026-09-09 [email protected] [web] Support local golden testing in flutter test (flutter/flutter#191386) 2026-09-09 [email protected] [flutter_tools] Ignore widget previews outside lib/ during LSP code generation (flutter/flutter#192326) 2026-09-09 [email protected] [flutter_tools] Read package_config.json asynchronously in loadPackageConfigWithLogging (flutter/flutter#192443) 2026-09-08 [email protected] Add precacheTestImage to flutter_test and unskip web image tests (flutter/flutter#191435) 2026-09-08 [email protected] et: Fix compile_commands.json post-processing (flutter/flutter#192395) 2026-09-08 [email protected] Refactor createDebuggingOptions and option registrations to OptionDescriptor (flutter/flutter#192307) 2026-09-08 [email protected] [flutter_tools] Fix test separation between `VersionFreshnessValidator` and `FlutterVersion.checkFlutterVersionFreshness` (flutter/flutter#191593) 2026-09-08 [email protected] [tool] Migrate EmulatorsCommand to modular dependency injection (flutter/flutter#190769) 2026-09-08 [email protected] [tool] Migrate Desktop build subcommands and toolchain to modular dependency injection (flutter/flutter#190781) 2026-09-08 [email protected] [tool] Migrate Tooling and Analysis commands to modular dependency injection (flutter/flutter#190788) 2026-09-08 [email protected] Skip KGP version validation for AGP built-in Kotlin (flutter/flutter#192206) If this roll has caused a breakage, revert this CL and set the roller to dry run mode using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC [email protected],[email protected] on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

cbracken requested review from flar and jtmcdole September 7, 2026 22:25

flutter-dashboard Bot added the CICD Run CI/CD label Sep 7, 2026

github-actions Bot added engine flutter/engine related. See also e: labels. team-engine Owned by Engine team d: docs/ flutter/flutter/docs, for contributors labels Sep 7, 2026

gemini-code-assist Bot reviewed Sep 7, 2026

View reviewed changes

Comment thread engine/src/flutter/tools/pkg/engine_build_configs/lib/src/build_config_runner.dart

cbracken force-pushed the fix-compile-commands branch from 92ddc9c to 213a19f Compare September 8, 2026 01:21

Merge branch 'master' into fix-compile-commands

cc89064

jtmcdole reviewed Sep 8, 2026

View reviewed changes

jtmcdole approved these changes Sep 8, 2026

View reviewed changes

cbracken added this pull request to the merge queue Sep 8, 2026

Merged via the queue into flutter:master with commit 56cbb21 Sep 8, 2026
18 checks passed

cbracken deleted the fix-compile-commands branch September 8, 2026 23:07

This was referenced Sep 9, 2026

Roll Flutter from b444e7897e24 to 0056d8b88d34 (19 revisions) flutter/packages#12813

Closed

Roll Flutter from b444e7897e24 to 63b9518f16bc (21 revisions) flutter/packages#12814

Merged

Navigation Menu

Repository navigation

et: Fix compile_commands.json post-processing - #192395

et: Fix compile_commands.json post-processing#192395
cbracken merged 2 commits into
flutter:masterfrom
cbracken:fix-compile-commands

cbracken commented Sep 7, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

flar commented Sep 8, 2026

Uh oh!

cbracken commented Sep 8, 2026 •

edited

Loading

Uh oh!

jtmcdole Sep 8, 2026

Uh oh!

jtmcdole Sep 8, 2026

Uh oh!

jtmcdole left a comment

Uh oh!

flar commented Sep 8, 2026

Uh oh!

cbracken commented Sep 8, 2026 •

edited by jtmcdole

Loading

Uh oh!

jtmcdole commented Sep 8, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

cbracken commented Sep 7, 2026

Pre-launch Checklist

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

flar commented Sep 8, 2026

Uh oh!

cbracken commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jtmcdole Sep 8, 2026

Choose a reason for hiding this comment

Uh oh!

jtmcdole Sep 8, 2026

Choose a reason for hiding this comment

Uh oh!

jtmcdole left a comment

Choose a reason for hiding this comment

Uh oh!

flar commented Sep 8, 2026

Uh oh!

cbracken commented Sep 8, 2026 • edited by jtmcdole Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jtmcdole commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💵 The CodeFu LGTM Note

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

cbracken commented Sep 8, 2026 •

edited

Loading

cbracken commented Sep 8, 2026 •

edited by jtmcdole

Loading

jtmcdole commented Sep 8, 2026 •

edited

Loading