Repository navigation
Support running/debugging Flutter Driver integration tests #2609
Description
I'm working through this guide trying to get to a point where I can step through driver scripts and/or driven apps in VSCode: https://medium.com/flutter-community/hot-reload-for-flutter-integration-tests-e0478b63bd54
Running the terminal commands works, but I haven't been able to get either side to work when launched from VSCode yet. There seem to be two things in the way:
- Setting a known
--observation-portvia"args"inlaunch.jsonseems to get overridden by the extension - There doesn't seem to be any way to run the driver program as a dart CLI command because the extension detects the project type from the workspace folder and offers no
launch.json-level override. It gets launched on the Android device I just launched the app on, and I can breakpoint through it but it kills the app on the device to launch the driver
Here's my best guess so far:
{
"name": "Integration Tests: Launch App",
"request": "launch",
"type": "dart",
"program": "test_driver/app.dart",
"args": [
"--dart-define=app.api=http://10.0.2.2:5378/api/v1", // special android IP for host, works
"--observatory-port 4567", // doesn't seem to work
"--disable-service-auth-codes"
],
"flutterMode": "debug", // don't know if needed
},
{
"name": "Integration Tests: Launch Driver",
"request": "launch",
"type": "dart",
"program": "test_driver/app_test.dart",
"deviceId": null, // this needs to run on the host computer, but it wants to run on the same device as the app
"env": {
"VM_SERVICE_URL": "http://127.0.0.1:4567"
},
},
Activity
[-]Interactively debugging integration tests[/-][+]Support running/debugging Flutter Driver integration tests[/+]Add support for running test_driver tests if an instrumented app is r…
DanTup commentedon Jul 7, 2020
Could you try this version out for me?
Download the.vsixfile from https://github.com/Dart-Code/Dart-Code/releases/tag/v3.13.0-alpha.2Run the Extensions: Install from VSIX command from the VS Code command paletteBrowse to the file you downloadedClick the Restart button when promptedlaunch.jsontest_driver/app.dartand pressF5- this should now launch this file (the instrumented version)test_driver.app_test.dartF5to run the whole file, or use the CodeLensRun/Debuglinks to run individual testsThis should give you full debugging of both the app and the test, and you shouldn't need to mess around with launch configs. If you do want to be able to launch these from launch configs, you should be able to just strip out all the
args/envyou added:Let me know if this seems to work ok for you (I've never used driver tests before, so I'm flying a little blind here). If you hit any issues, let me know (ether here, or once this issue is closed, in new issues). If it all works, you can remain on this test version and when the next stable release goes out you'll be automatically changed over to it (if you have auto updates for extensions enabled).
BTW - I made it automatically Hot Restart the app at the start of the test run, as it seemed weird to run with existing state.. if this feels weird, let me know (I did it because the sample tests here would fail if run multiple times due to the counter state not being reset).
themightychris commentedon Jul 7, 2020
@DanTup it works!!! this is brilliant
Add support for running test_driver tests if an instrumented app is r…
DanTup commentedon Jul 8, 2020
Great, thanks for confirming!
themightychris commentedon Jul 8, 2020
@DanTup how long does it usually take for new versions to show up at https://marketplace.visualstudio.com/items?itemName=Dart-Code.dart-code
DanTup commentedon Jul 8, 2020
@themightychris this feature isn't included in a release yet - it'll be in the version in the assigned milestone (v3.13.0) which is likely to ship around the end of the month (there was a release today, but it was a minor version to support using the Flutter SDK Snap). The issues are automatically closed when the work is merged to master.
For now, you should stick with the preview build from above - if you have auto-updates on, VS Code will automatically upgrade you to the stable v3.13.0 when it's published.
tijanirf commentedon Oct 16, 2020
Hi @themightychris, you mention here that you can use --dart-define inside args. I already tried but it's not working on the test driver, it is only working on my default app. Are there any gotchas that I need to carefully check?
themightychris commentedon Oct 16, 2020
@tijanirf it's all super easy to do now, and built into the vscode extension. You don't need to pass
--observatory-portor--disable-service-auth-codesinlaunch.json, you just need to launch the driver-wrapped version of your app fromlaunch.json, and then when you open up your test suite in vscode the editor will be decorated withRun | Debugbuttons at various scopes, just click them while you have the instrumented app running from vscode and it should all Just Work!tomasfly commentedon Sep 23, 2021
Any ideas how to debug integration tests using new integration_test package from SDK?
DanTup commentedon Sep 23, 2021
@tomasfly it should generally just work since v3.22 (see https://dartcode.org/releases/v3-22/#flutter-integration-testing). If you're having issues with it not doing as you'd expect, please file a new issue with detailed steps and I'll take a look. Thanks!
JaredEzz commentedon Jan 26, 2023
Does anyone have any tips for doing this in IntelliJ? Currently I just run
flutter drive -d chromewith the target and driver args from the terminal which doesn't allow for debugginggabrielchaves7 commentedon Feb 24, 2023
@DanTup Is it possible to debug integration test using chrome driver?. As @JaredEzz mentioned, right now I'm running on the terminal using
flutter -d chrome drive --driver=test_driver/integration_test.dart --target=integration_test/main_test.dart --web-port=8080but I would like to debug the tests and couldn't figure it out a wayDanTup commentedon Feb 27, 2023
@gabrielchaves7 unfortunately I don't think this is currently possible. The ideal way would be for
flutter test(which is now how we run non-web integration tests.. this issue is a little old and predates that) to support web, then things should just work from the editors. I thought there was an open issue in flutter/flutter about this, but I can't find one. It may be worth opening one (flutter testalready supports all the flags needed for debugging, starting paused, parsing the test output to populate the test tree etc.).I did wonder if in the meantime if you only need the debugging part (not test tree etc.) you could do this with a launch configuration that does an
attachand apreLaunchTaskto runflutter drive, but it seems likeflutter drivedoesn't support--start-paused, since running this:...does not appear to start paused (as would be required for an
attachconfiguration to be able to connect to it before the tests started running). Not sure if that's a bug or expected though.