Repository navigation
Report individual test results to LUCI ResultDB - #190254
Merged
Merged
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request integrates LUCI ResultDB reporting into the Flutter test runner, allowing individual test results to be uploaded directly. It introduces the ResultDbRecorder client, parses test cases from the JSON reporter, and updates runDartTest and runFlutterTest to report results. Unit tests are also added to verify the implementation. Feedback on the changes suggests handling potential exceptions during LUCI_CONTEXT reading and recorder initialization to prevent test runner crashes, as well as optimizing string truncation to avoid performance bottlenecks with very long test names.
jtmcdole
reviewed
Aug 10, 2026
jtmcdole
reviewed
Aug 11, 2026
Comment on lines
+51
to
+54
| for (final LuciTestResult( | ||
| testId: LuciStructuredTestId(:moduleName, :moduleScheme, :moduleVariant), | ||
| ) | ||
| in results) { |
jtmcdole
previously approved these changes
Aug 11, 2026
jtmcdole
approved these changes
Aug 12, 2026
jtmcdole
added
the
autosubmit
auto-submit
Bot
removed
the
autosubmit
Contributor
|
autosubmit label was removed for flutter/flutter/190254, because - The status or check suite Dashboard Checks has failed. Please fix the issues identified (or deflake) before re-applying this label. |
Framework test shards previously only surfaced a single pass/fail status at the shard level, so the "Test Results" tab for a build in LUCI was effectively empty. This reports each individual test case to the build's ResultDB invocation, so the tab lists every test that ran. Details: - Parse the per-shard test metrics (already emitted by the JSON reporter) into structured TestResult objects (tool_subsharding.dart). - Add a ResultDB Recorder client (luci_resultdb.dart) that uploads results via BatchCreateTestResults, authorized by the update_token from LUCI_CONTEXT["resultdb"]. This works for the Flutter recipe, which exposes an invocation but does not run `rdb stream`. It is a no-op off LUCI. - Convert each case to a structured test id (testIdStructured, `flat` scheme) so the test file (module) and the test name (case) show as separate columns. - Report module names relative to the repository root: relative suite paths are resolved against the shard's working directory and separators normalized, so ids are consistent regardless of where the shard ran. - Mark results from expect-failure runs (e.g. test_smoke_test) as expected failures so they are not surfaced as red regressions. - Model the ResultDB format with typed classes (LuciTestResult / LuciStructuredTestId) and add unit tests.
- readLuciContext: wrap read+decode in try/catch so an unreadable file or malformed JSON returns null instead of crashing the test run (it is called outside the reporting try/catch). - reportTestResultsToResultDb: initialize ResultDbRecorder.fromEnvironment() inside the try/catch so initialization failures are logged, not fatal. - _truncateToBytes: truncate to maxBytes code units before the byte-length loop to avoid O(N^2) behavior on very long names. - Add a regression test for malformed LUCI_CONTEXT.
chingjun
added
the
autosubmit
flutter-dashboard
Bot
removed
the
autosubmit
auto-submit Bot
pushed a commit
to flutter/packages
that referenced
this pull request
Aug 13, 2026
…#12453) Manual roll Flutter from 27b098811f3b to c2437523d308 (179 revisions) Manual roll requested by [email protected] flutter/flutter@27b0988...c243752 2026-08-12 [email protected] Enable Gradle CI cache on all test targets that require android_sdk (flutter/flutter#190723) 2026-08-12 [email protected] [analysis] Reland "Added initial implementation of the flutter_analyzer_plugin (#175679)" (flutter/flutter#191022) 2026-08-12 [email protected] Switch testing to gradle bin distribution type instead of all (flutter/flutter#190738) 2026-08-12 [email protected] Convert Mockito instances in Kotlin to Mockk (flutter/flutter#189884) 2026-08-12 [email protected] Report individual test results to LUCI ResultDB (flutter/flutter#190254) 2026-08-12 [email protected] Toggleable reaction duration respects overrides (flutter/flutter#190857) 2026-08-12 [email protected] Roll Skia from e00dbd7448c4 to fee7272f5bc2 (1 revision) (flutter/flutter#191007) 2026-08-12 [email protected] Started caching text shadows by content. (flutter/flutter#190681) 2026-08-12 [email protected] Adds agent skill for spawning led tasks. (flutter/flutter#190937) 2026-08-12 [email protected] Roll Packages from aaaf246 to 94485f1 (8 revisions) (flutter/flutter#191008) 2026-08-12 [email protected] flutter_tools: validate plugin identifiers before generating registrant code (flutter/flutter#190462) 2026-08-12 [email protected] Roll Skia from 112f36148949 to e00dbd7448c4 (3 revisions) (flutter/flutter#190993) 2026-08-12 [email protected] Roll Skia from 7d366c802307 to 112f36148949 (3 revisions) (flutter/flutter#190983) 2026-08-12 [email protected] remove bringup for flavors test (flutter/flutter#190940) 2026-08-12 [email protected] Roll Skia from 1f10a20bdd61 to 7d366c802307 (2 revisions) (flutter/flutter#190980) 2026-08-12 [email protected] Remove `--no-sim-use-hardfp` flag (flutter/flutter#190790) 2026-08-12 [email protected] Roll pub packages (flutter/flutter#190977) 2026-08-12 [email protected] Roll Skia from 339bedab6766 to 1f10a20bdd61 (1 revision) (flutter/flutter#190975) 2026-08-12 [email protected] RawTooltip respects AnimationStyle updates and reverseCurve (flutter/flutter#190889) 2026-08-12 [email protected] ci: Support --target_arch option in prepare_package.dart (flutter/flutter#190960) 2026-08-12 [email protected] Roll Fuchsia Linux SDK from SFq4FVodIOQAS26Lr... to -uHuSGv3wt7QAlDwa... (flutter/flutter#190973) 2026-08-12 [email protected] Removes building of ci/android_debug_x86 as nobody should be consuming it. (flutter/flutter#190951) 2026-08-12 [email protected] Adds error about wimp_heavy not being implemented. (flutter/flutter#189945) 2026-08-11 [email protected] Add clang, cmake, and ninja deps to Linux windowing_test (flutter/flutter#190119) 2026-08-11 [email protected] [AGP 9.1.0 Migration #1] Add Android Gradle Plugin Public API migration documentation (flutter/flutter#190842) 2026-08-11 [email protected] [flutter_tools] Fix deadlock in debug adapters when process exits early (flutter/flutter#190931) 2026-08-11 [email protected] [tool] Define modular dependency injection containers and bootstrapper (flutter/flutter#190724) 2026-08-11 [email protected] [web] Unify MaskFilter and ColorFilter primitives across CanvasKit and Skwasm (flutter/flutter#190314) 2026-08-11 [email protected] Roll pub packages (flutter/flutter#190958) 2026-08-11 [email protected] [Impeller] Move image upload scheduling waits to GPU disable (flutter/flutter#190445) 2026-08-11 [email protected] Started generating the windows platform for macrobenchmarks (flutter/flutter#190932) 2026-08-11 [email protected] [web] Unify ui.Vertices (flutter/flutter#190563) 2026-08-11 [email protected] Fix accessibility_inspector service extensions map mutability (flutter/flutter#190888) 2026-08-11 [email protected] Add batch3 a11y_assessment for vpat (flutter/flutter#189042) 2026-08-11 [email protected] Remove Xcode environment when building swift tools in Xcode pre-action (flutter/flutter#190848) 2026-08-11 [email protected] [tool] Add missing play element in web test index.html to fix warning (flutter/flutter#190675) 2026-08-11 [email protected] Offload blocking work in ProcessTextPlugin to the background (flutter/flutter#189823) 2026-08-11 [email protected] [flutter_tools] Replace usages of package:dds/dap.dart with package:dap_adapters/dap_adapters.dart (flutter/flutter#190667) 2026-08-11 [email protected] Always update swift package dependencies (flutter/flutter#190886) 2026-08-11 [email protected] [flutter_tools] Add --preset option to flutter test (flutter/flutter#190878) 2026-08-11 [email protected] Include the examples cross imports checker in the analzyer. (flutter/flutter#190674) 2026-08-11 [email protected] [devicelab] Remove orphaned screenshot test files (flutter/flutter#190879) 2026-08-11 [email protected] Remove the bringup flag from the linux_arm_host_desktop_engine builder (flutter/flutter#190935) 2026-08-11 [email protected] Reduce web_skwasm_tests subshards from 8 to 2 (flutter/flutter#190728) ...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Framework test shards previously only surfaced a single pass/fail status at the shard level, so the "Test Results" tab for a build in LUCI was effectively empty. This reports each individual test case to the build's ResultDB invocation, so the tab lists every test that ran.
This PR populates the "test results" tab with actual information about the tests. Example: https://ci.chromium.org/ui/p/flutter/builders/try/Linux%20framework_tests_widgets/109467/test-results