flutter/packages tree closed on web unit tests due to package:test update #192248

The Flutter into Packages AutoRoller has been failing for the last day.

package:test 1.32.0 introduced changes to how Chrome is launched for browser tests. This is causing failures for Linux_web web_dart_unit_test_shard_# stable test with errors like "Chrome exited before connecting".

Dashboard for recent autoroll PR: https://flutter-dashboard.appspot.com/#/presubmit?repo=packages&pr=12742&sha=9fcafa01645249ed2e3c1901ba1f6ebb151fdd74&job=Linux_web%20web_dart_unit_test_shard_1%20stable

Summary from Gemini:

Problem Description

The CI test step Run_package_tests / Dart_unit_tests_-_web fails for recent PRs (e.g., flutter/packages#12742).

Example Error Output

Running command: "dart run test --platform=chrome" in /b/s/w/ir/x/w/packages/packages/cross_file
00:00 +0: loading test/x_file_html_test.dart
00:00 +0 -1: loading test/x_file_html_test.dart [E]
  Failed to load "test/x_file_html_test.dart": Chrome exited before connecting.
  package:test/src/runner/browser/browser_manager.dart 125:18  BrowserManager._start.<fn>

Affected Packages & Builds

Every package tested via dart run test --platform=chrome fails with Chrome exited before connecting:

• packages/cross_file (test/x_file_html_test.dart)
• packages/cross_file/example (test/main_test.dart)
• packages/google_identity_services_web (test/js_loader_test.dart)
• third_party/packages/path_parsing (test/parse_path_test.dart)

Log References:

• Failing recent build: PR #12742 Linux_web web_dart_unit_test_shard_3 stable
• Passing older build: PR #12712 Linux_web web_dart_unit_test_shard_3 stable

Investigation

1. Flutter Packages vs. Pure Dart Packages:

   • Flutter packages execute tests via flutter test --platform=chrome. These all pass because the Flutter CLI manages its own Chrome process lifecycle and arguments.
   • Pure Dart packages execute tests via dart run test --platform=chrome using package:test. 100% of these packages fail.

2. Package Version Resolution (package:test 1.31.2 vs 1.32.0):
   None of these packages have checked-in pubspec.lock files, so dart pub get resolves the latest available version on pub.dev satisfying ^1.16.0:

   • Passing runs (PR Failed assertion in scrollable.dart ('_drag == null': is not true.) #12712): Resolved package:test 1.31.2 (test_api 0.7.13, test_core 0.6.19).
   • Failing runs (PR Add a custom semantics sample #12742 & recent builds): Resolved package:test 1.32.0 (test_api 0.7.14, test_core 0.6.20).

Root Cause

package:test 1.32.0 introduced changes to how Chrome is launched for browser tests:

1. Removal of --remote-debugging-port=0 in non-debug mode:
   In package:test <= 1.31.2, --remote-debugging-port=0 was passed unconditionally to prevent Chrome from exiting. The previous comment in package:test noted:

   "We don't actually connect to the remote debugger, but Chrome will close as soon as the page is loaded if we don't turn it on."
   In 1.32.0, --remote-debugging-port is omitted when running tests without --debug.

2. Introduction of redirect.html file:
   In 1.32.0, package:test writes a temporary local redirect.html file (<script>location = ...</script>) and passes that local path to Chrome rather than passing the HTTP URL directly via CLI args.

3. Premature Process Termination:
   Without an active remote debugging port or open tab connection in headless mode on Linux CI, Chrome exits immediately upon loading the local redirect file before establishing the test WebSocket connection.

Potential Fixes

1. Pin test version across affected packages in flutter/packages (Immediate Workaround):
   Constrain test to <1.32.0 in pubspec.yaml for cross_file, google_identity_services_web, and path_parsing:

   dev_dependencies:
     test: ">=1.16.0 <1.32.0"

2. Repository Tooling Workaround:
   Update script/tool/lib/src/dart_test_command.dart to pass --pause-after-load or --debug if appropriate, or temporarily override dependency resolution in CI.

3. Upstream Fix in dart-lang/test:
   File an issue / submit a patch in dart-lang/test to restore --remote-debugging-port=0 (or otherwise prevent Chrome from prematurely terminating) when launching headless Chrome in non-debug mode with redirect.html.