Repository navigation
Support running integration tests with the Flutter Tool #66264
Description
Use case
Integration tests today are run by users using the following command:
flutter drive \
--driver=test_driver/<driver_file> \
--target=integration_test/<test file>There is some friction when doing things like globbing over and running all tests in the integration_test/ directory.
Also, most of the time the test driver file is just the following copy-pasted:
import 'package:integration_test/integration_test_driver.dart';
Future<void> main() => integrationDriver();Proposal
Provide some sort of executable as a thin wrapper around flutter drive.
This tool will have two main purposes:
- Deciding where / what integration tests to run
- Reducing user boilerplate, such as the placeholder driver script above.
The result is that users can use the following command to run all their tests in integration_test/, without the need to add the aforementioned placeholder script to test_driver/ as mentioned above.
cd <application directory>
pub run integration_testDeciding Where / What integration tests to run
Some rough ideas:
# Runs all integration tests
pub run integration_test
# Runs just `foo_test.dart`
pub run integration_test integration_test/foo_test
# Run on iOS
pub run integration_test --device-id iosWe can forward all other args (or maybe just some like --device-id ?) to the flutter drive command.
Reducing user boilerplate
Host-side boilerplate
This tool will will make it no longer necessary for users to copy paste the placeholder driver file into the test_driver/ directory. We will provide a --driver parameter for users to override the driver script, for use cases such as handling screenshots (as mentioned in flutter-team-archive/plugins#2986).
Application-side boilerplate
At the same time, it faciliates the change to use the reporter from package test (#50816). With the new API (currently in progress dart-lang/test#1332) from package:test, users will likely need to add boilerplate code to their tests that run on the device.
// In package:integration_test.
// Expose a new API so that [testMain] runs with the reporter from package:test.
Future<void> runIntegrationTest(FutureOr<void> Function() testMain) async {
// Import and use `directRunTests` from `package:test`.
}// In the users test code, in their integration_test/ directory.
import 'package:integration_test/integration_test.dart';
// Users need to wrap their tests with [run].
void main() => runIntegrationTest(testMain);
Future<void> testMain() {
testWidgets('A widget test', (tester) async {
await tester.pumpWidget(CounterApp());
expect(find.byType(Text), findsNWidgets(2));
});
test('a test', () {
expect(1 + 1, 2);
});
}With this proposed tool, we can eliminate the boilerplate that users need to use in their tests. They do not have to import runIntegrationTest to wrap their tests; they can write their tests just like they know how with package:test and package:flutter_test.
Activity
jiahaog commentedon Sep 22, 2020
cc @dnfield Any thoughts on this? I can help to drive this if it makes sense.
dnfield commentedon Sep 22, 2020
I think that sounds fine, although I wonder if we could just instead extend the
flutter drivecommand to handle this. WDYT?jiahaog commentedon Sep 23, 2020
That could work, but currently
flutter driveoperates at a lower level of abstraction than we want to expose to our users - it mainly looks for a driver and target script, and logic such as deciding what tests to run is probably out of place there. I'd prefer something likeflutter integration_testthat would match the abstraction better, but I recall some chatter about how there are already too many top level commands in the flutter tool.We could also do this as a separate "mode" for
flutter test, though the UX of running integration tests will need to be explored further to avoid confusion with normal widget tests. Would we need to move development of this package from flutter/plugins to flutter/flutter to do this?mccadi commentedon Dec 4, 2020
It would be amazing to be able to use the
--machineflag on this executable (if you guys do decide to go through with it) since driver doesn't support it for test results.dnfield commentedon Dec 4, 2020
That may be feasible, but we should probably file a separate bug to track it.
jiahaog commentedon Jan 9, 2021
Forgot to link this earlier – http://flutter.dev/go/tool-integration-test-support is the doc I started to dive deeper into this.
[-]Provide an executable to run package:integration_test tests[/-][+]Support running integration tests with the Flutter Tool[/+]lhimo commentedon Sep 7, 2021
Is there already a way to include/exclude specific test tags when building apk/ipa to launch on Firebase Test Lab?
jiahaog commentedon Sep 9, 2021
There isn't at the moment unfortunately. Please file an issue if that is desired 🙂
lhimo commentedon Sep 10, 2021
Opened #89826
JaredEzz commentedon Jan 24, 2023
This should also be able to attach to a debugger, no? At the moment it's not possible to debug flutter web integration tests using IntelliJ or other IDE debuggers
matanlurey commentedon Jul 9, 2025
This is not planned.
pedromassango commentedon Jul 9, 2025
@matanlurey it would be great to give reasoning on why the team decided to do so instead of a simple "not planned" message
github-actions commentedon Jul 23, 2025
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of
flutter doctor -vand a minimal reproduction of the issue.