Repository navigation
Allow passing exitCode from dart test to flutter test #100467
Copy link
Copy link
Closed
Labels
P3Issues that are less important to the Flutter projectIssues that are less important to the Flutter projecta: tests"flutter test", flutter_test, or one of our tests"flutter test", flutter_test, or one of our testsc: proposalA detailed proposal for a change to FlutterA detailed proposal for a change to Flutterteam-toolOwned by Flutter Tool teamOwned by Flutter Tool teamtoolAffects the "flutter" command-line tool. See also t: labels.Affects the "flutter" command-line tool. See also t: labels.triaged-toolTriaged by Flutter Tool teamTriaged by Flutter Tool team
Description
Use case
For multi-package management, like melos or mono-repo, we might specify --tags and --exclude-tags or any other filter arguments... That would cause the test execution result to no test, but It's common on multi-packages, since those might mark some tests as heavy works to split out tests.
In dart test, we have a exitCode list for knowing If It's a noTestsRan. But according to
flutter/packages/flutter_tools/lib/src/commands/test.dart
Lines 434 to 475 in 6af40a7
| final int result = await testRunner.runTests( | |
| testWrapper, | |
| _testFiles, | |
| debuggingOptions: debuggingOptions, | |
| names: names, | |
| plainNames: plainNames, | |
| tags: tags, | |
| excludeTags: excludeTags, | |
| watcher: watcher, | |
| enableObservatory: collector != null || startPaused || boolArg('enable-vmservice'), | |
| ipv6: boolArg('ipv6'), | |
| machine: machine, | |
| updateGoldens: boolArg('update-goldens'), | |
| concurrency: jobs, | |
| buildTestAssets: buildTestAssets, | |
| flutterProject: flutterProject, | |
| web: stringArg('platform') == 'chrome', | |
| randomSeed: stringArg('test-randomize-ordering-seed'), | |
| reporter: stringArg('reporter'), | |
| timeout: stringArg('timeout'), | |
| runSkipped: boolArg('run-skipped'), | |
| shardIndex: shardIndex, | |
| totalShards: totalShards, | |
| integrationTestDevice: integrationTestDevice, | |
| integrationTestUserIdentifier: stringArg(FlutterOptions.kDeviceUser), | |
| ); | |
| if (collector != null) { | |
| final bool collectionResult = collector.collectCoverageData( | |
| stringArg('coverage-path'), | |
| mergeCoverageData: boolArg('merge-coverage'), | |
| ); | |
| if (!collectionResult) { | |
| throwToolExit(null); | |
| } | |
| } | |
| if (result != 0) { | |
| throwToolExit(null); | |
| } | |
| return FlutterCommandResult.success(); | |
| } |
flutter didn't expose the result(exitCode), even though throwToolExit support It.
flutter/packages/flutter_tools/lib/src/base/common.dart
Lines 9 to 11 in 7d46d43
| Never throwToolExit(String message, { int? exitCode }) { | |
| throw ToolExit(message, exitCode: exitCode); | |
| } |
Somehow I sure result is exitCode is according to the implementation.
flutter/packages/flutter_tools/lib/src/test/runner.dart
Lines 61 to 185 in 9e88fe3
| class _FlutterTestRunnerImpl implements FlutterTestRunner { | |
| const _FlutterTestRunnerImpl(); | |
| @override | |
| Future<int> runTests( | |
| TestWrapper testWrapper, | |
| List<String> testFiles, { | |
| @required DebuggingOptions debuggingOptions, | |
| List<String> names = const <String>[], | |
| List<String> plainNames = const <String>[], | |
| String tags, | |
| String excludeTags, | |
| bool enableObservatory = false, | |
| bool ipv6 = false, | |
| bool machine = false, | |
| String precompiledDillPath, | |
| Map<String, String> precompiledDillFiles, | |
| bool updateGoldens = false, | |
| TestWatcher watcher, | |
| @required int concurrency, | |
| bool buildTestAssets = false, | |
| FlutterProject flutterProject, | |
| String icudtlPath, | |
| Directory coverageDirectory, | |
| bool web = false, | |
| String randomSeed, | |
| String reporter, | |
| String timeout, | |
| bool runSkipped = false, | |
| int shardIndex, | |
| int totalShards, | |
| Device integrationTestDevice, | |
| String integrationTestUserIdentifier, | |
| }) async { | |
| // Configure package:test to use the Flutter engine for child processes. | |
| final String shellPath = globals.artifacts.getArtifactPath(Artifact.flutterTester); | |
| // Compute the command-line arguments for package:test. | |
| final List<String> testArgs = <String>[ | |
| if (!globals.terminal.supportsColor) | |
| '--no-color', | |
| if (debuggingOptions.startPaused) | |
| '--pause-after-load', | |
| if (machine) | |
| ...<String>['-r', 'json'] | |
| else | |
| ...<String>['-r', reporter ?? 'compact'], | |
| if (timeout != null) | |
| ...<String>['--timeout', timeout], | |
| '--concurrency=$concurrency', | |
| for (final String name in names) | |
| ...<String>['--name', name], | |
| for (final String plainName in plainNames) | |
| ...<String>['--plain-name', plainName], | |
| if (randomSeed != null) | |
| '--test-randomize-ordering-seed=$randomSeed', | |
| if (tags != null) | |
| ...<String>['--tags', tags], | |
| if (excludeTags != null) | |
| ...<String>['--exclude-tags', excludeTags], | |
| if (runSkipped) | |
| '--run-skipped', | |
| if (totalShards != null) | |
| '--total-shards=$totalShards', | |
| if (shardIndex != null) | |
| '--shard-index=$shardIndex', | |
| '--chain-stack-traces', | |
| ]; | |
| if (web) { | |
| final String tempBuildDir = globals.fs.systemTempDirectory | |
| .createTempSync('flutter_test.') | |
| .absolute | |
| .uri | |
| .toFilePath(); | |
| final WebMemoryFS result = await WebTestCompiler( | |
| logger: globals.logger, | |
| fileSystem: globals.fs, | |
| platform: globals.platform, | |
| artifacts: globals.artifacts, | |
| processManager: globals.processManager, | |
| config: globals.config, | |
| ).initialize( | |
| projectDirectory: flutterProject.directory, | |
| testOutputDir: tempBuildDir, | |
| testFiles: testFiles, | |
| buildInfo: debuggingOptions.buildInfo, | |
| ); | |
| if (result == null) { | |
| throwToolExit('Failed to compile tests'); | |
| } | |
| testArgs | |
| ..add('--platform=chrome') | |
| ..add('--') | |
| ..addAll(testFiles); | |
| testWrapper.registerPlatformPlugin( | |
| <Runtime>[Runtime.chrome], | |
| () { | |
| return FlutterWebPlatform.start( | |
| flutterProject.directory.path, | |
| updateGoldens: updateGoldens, | |
| shellPath: shellPath, | |
| flutterProject: flutterProject, | |
| pauseAfterLoad: debuggingOptions.startPaused, | |
| nullAssertions: debuggingOptions.nullAssertions, | |
| buildInfo: debuggingOptions.buildInfo, | |
| webMemoryFS: result, | |
| logger: globals.logger, | |
| fileSystem: globals.fs, | |
| artifacts: globals.artifacts, | |
| processManager: globals.processManager, | |
| chromiumLauncher: ChromiumLauncher( | |
| fileSystem: globals.fs, | |
| platform: globals.platform, | |
| processManager: globals.processManager, | |
| operatingSystemUtils: globals.os, | |
| browserFinder: findChromeExecutable, | |
| logger: globals.logger, | |
| ), | |
| cache: globals.cache, | |
| ); | |
| }, | |
| ); | |
| await testWrapper.main(testArgs); | |
| return exitCode; | |
| } |
Proposal
Just curious why not pass result to throwToolExit to allow more usefulness?
if (result != 0) {
throwToolExit(null, exitCode: result);
}
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
P3Issues that are less important to the Flutter projectIssues that are less important to the Flutter projecta: tests"flutter test", flutter_test, or one of our tests"flutter test", flutter_test, or one of our testsc: proposalA detailed proposal for a change to FlutterA detailed proposal for a change to Flutterteam-toolOwned by Flutter Tool teamOwned by Flutter Tool teamtoolAffects the "flutter" command-line tool. See also t: labels.Affects the "flutter" command-line tool. See also t: labels.triaged-toolTriaged by Flutter Tool teamTriaged by Flutter Tool team
Activity
[-]Allow pass exitCode from dart test to flutter test[/-][+]Allow passing exitCode from dart test to flutter test[/+]christopherfujino commentedon Mar 22, 2022
This sounds completely reasonable, I'd happily review a PR for it.
Forward exit code from dart test to flutter test (#168604)
Forward exit code from dart test to flutter test (flutter#168604)
github-actions commentedon Jun 9, 2025
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of
flutter doctor -vand a minimal reproduction of the issue.Forward exit code from dart test to flutter test (#168604)