"Run All Tests" passes all individual suite names for all projects when only some projects have exclusions #4553

Activity

1. dmrickey

   added

   is bug

   on May 18, 2023
2. DanTup commented on May 18, 2023

   

   DanTup

   on May 18, 2023

   Member

   More actions

   This happens when we're not running "everything" (such as if there are some exclusions that prevent us just doing flutter run test test).

   Unfortunately, our check for whether we're running everything is for the whole run, and not per-project:

   Dart-Code/src/extension/test/vs_test_controller.ts

   Line 71 in f854d99

   const isRunningAll = !request.include?.length && !request.exclude?.length;

   This means if you exclude a test from ProjectA, then ProjectB will also be run by including all of the filenames.

   At a minimum, we should attempt to group the exclusions into their projects, so we can still run "all" for the projects without exclusions.

   Additionally:

   • We should use relative paths if possible, because these are currently absolute and exceeding the command line limit faster earlier necessary
   • Consider splitting up large runs into multiple debug sessions if we know they'll exceed the limit
3. DanTup

   added this to the v3.66.0 milestone on May 18, 2023
4. DanTup

   added

   in testingRelates to test execution of Dart/Flutter tests for end usersRelates to test execution of Dart/Flutter tests for end users

   on May 18, 2023
5. DanTup

   changed the title [-]Running tests on windows is passing each individual test file instead of a single folder glob[/-] [+]"Run All Tests" passes all individual suite names for all projects when only some projects have exclusions (plus uses long absolute paths)[/+] on May 18, 2023
6. DanTup commented on May 18, 2023

   

   DanTup

   on May 18, 2023

   Member

   More actions

   Note to self: ping @dmrickey on Discord when a fix is available as a pre-release extension.
7. DanTup

   changed the title [-]"Run All Tests" passes all individual suite names for all projects when only some projects have exclusions (plus uses long absolute paths)[/-] [+]"Run All Tests" passes all individual suite names for all projects when only some projects have exclusions[/+] on May 20, 2023
8. DanTup

   modified the milestones: v3.66.0, v3.68.0 on May 31, 2023
9. DanTup

   added a commit that references this issue on Jun 19, 2023

   Allow running tests by folder when there are exclusions in other proj…

   ...

   783e73b
10. DanTup

    closed this as completedin b4d501fon Jun 19, 2023
11. DanTup commented on Jun 19, 2023

    

    DanTup

    on Jun 19, 2023

    Member

    More actions

    @dmrickey I pinged you on Discord, but in case you miss it, this is available in the v3.67.20230619 pre-release I just pushed. Please let me know if it doesn't/doesn't resolve the issue.
12. rizkyhaksono

    added a commit that references this issue on Apr 6, 2024

    Allow running tests by folder when there are exclusions in other proj…

    ...

    b4d501f