Don't show "Starting debug session…" so aggressively when running many test files #5682

Activity

1. Solido

   added

   is bug

   on Sep 3, 2025
2. DanTup commented on Sep 4, 2025

   

   DanTup

   on Sep 4, 2025

   Member

   More actions

   Launching an intensive serie of tests, a pop-up says 'Launching Debug Session' between each
   file it seems, maybe group.

   How are you triggering the test run? With something like the "Run all Tests" command?

   This is new behavior that slowdown the process by at least 20 times making checks impossible from the IDE.

   The prompt has always been there, the only change recently was not to hide it so quickly because it would disappear during the build of integration tests. I don't think this should change the actual performance of anything - you can see the change here:

   18b44e5

   Previously we would hide it when we got any messages back from the debug adapter, whereas now it's only for Output events (things that write to the debug console) or other progress events from the debug adapter. I'll try to repro your issue and see if there are other messages that could hide it without re-introducing the original issue.
3. DanTup

   added this to the v3.120.0 milestone on Sep 4, 2025
4. DanTup

   changed the title [-]Debug Session cascade[/-] [+]"Starting debug session…" shows repeatedly when running many tests[/+] on Sep 4, 2025
5. DanTup

   added

   is enhancementAn enhancement or improvement that should be listed in release notes but is not a bug fix.An enhancement or improvement that should be listed in release notes but is not a bug fix.

   in testingRelates to test execution of Dart/Flutter tests for end usersRelates to test execution of Dart/Flutter tests for end users

   and removed

   is bug

   on Sep 4, 2025
6. DanTup

   changed the title [-]"Starting debug session…" shows repeatedly when running many tests[/-] [+]Don't show "Starting debug session…" so aggressively when running many test files[/+] on Sep 4, 2025
7. DanTup

   closed this as completedin a792010on Sep 4, 2025
8. Solido commented on Sep 4, 2025

   

   Solido

   on Sep 4, 2025

   Author

   More actions

   Thanks for the inputs Dan.

   Yes I use the tests tabs and filter prompt.

   Curious I just discovered that the slowdown occurs only when I negate tags if left blank it is as fast as 2 days ago.
9. DanTup commented on Sep 4, 2025

   

   DanTup

   on Sep 4, 2025

   Last edited by DanTup

   Member

   More actions

   I've pushed a change so that we wait for 1s before showing this notification. This means for things that start up quickly, you won't see it, but for things that start slowly, you will.

   Curious I just discovered that the slowdown occurs only when I negate tags if left blank it is as fast as 2 days ago.

   Ah... my guess is that you're hitting a limitation of when we can run a single flutter test / dart test process for multiple tests compared to when we have to run multiple processes. So your slowdown is probably just from flipping between these and not actually related to the recent change that I assumed.

   For example when running in debug mode, we can't run multiple tests for Flutter:

   PS D:\Dev\Test Projects\flutter_counter> flutter test --machine --start-paused .\test\f1\widget_test.dart .\test\f2\widget_test.dart
   When using --start-paused, you must specify a single test file to run.
   

   If we remove --start-paused, then it runs fine. So depending on your selection of tests, we might have to run many debug sessions instead of just one, and that is much slower. This is unfortunately a limitation of Flutter (although I'm not sure of the specific reasons for that limitation).

   Edit: This means that although my change has reduced the noise from these notifications, you will still have slower test runs when you run a selection of tests in that way, and I believe that's always been the case.
10. Solido commented on Sep 4, 2025

    

    Solido

    on Sep 4, 2025

    Author

    More actions

    Is this a Flutter or Dart related limitation? currently running 1K tests and more to come, it has to be fast for user feedback. Running it on the shell is really good in time so I understand we need to avoid using tags...
11. DanTup commented on Sep 4, 2025

    

    DanTup

    on Sep 4, 2025

    Member

    More actions

    @Solido if using tags, then I think it's a bit of a combination of VS Code and Flutter.

    We try to combine things into a single process when we can (for example using Run instead of Debug). However when you filter with tags, AFAIK VS Code doesn't give us those tags, it just gives us a set of tests to run. If those tests are just test files then we can run flutter run a.dart b.dart c.dart but then there is a subset of tests within a file, this doesn't work (and we don't know the tag to create an equivalent command line).

    I'd have to do some testing to be sure we can't figure this info out though - is it possible you could create a small repro project that has a few test files and tags as you're using them, and the exact steps/filters that trigger the issue for you?

    👍React with 👍1Reacted by Robert Felker
12. DanTup commented on Sep 4, 2025

    

    DanTup

    on Sep 4, 2025

    Member

    More actions

    (please file a new issue for it so it's not mixed up with this one, which I've used to reduce the "noise" of the notifications)