Skip to content

Dart: Run All Tests - producing strange test hierarchy and results #3603

Closed

Description

@up-doc

Describe the bug
In a multi package workspace consisting of an app and some libraries, when running the extensions Dart: Run All Tests the resulting hierarchy in the the testing panel is strange or wrong. Some tests look like they've been skipped, the tests under the file name are wrong or don't exist. Causing lots of issues identifying and fixing failing tests in a larger project. Only started happening after upgrading extension. Same weird output whether you are having Use VS Code test runner enabled or not.

This also introduces issues where you would right click on what you would expect to be a test under that file and click Go To Test only to have it point to the bottom of a file. Or tap Run test only to have the console spit out something like (No tests match regular expression "^adds one( (variant: .*))?$".)

To Reproduce
Steps to reproduce the behavior:

  1. Open the repro_workspace folder in vs code
  2. Execute Dart: Run All Tests

Expected behavior
The test files should be the top of the collapsable section with the tests within that file actually shown underneath with their success or failure status.

Screenshots

Screenshot 2021-10-11 at 10 08 24

Screenshot 2021-10-11 at 10 11 49

Versions (please complete the following information):

  • VS Code version: Version: 1.61.0
    Commit: ee8c7def80afc00dd6e593ef12f37756d8f504ea

  • Dart extension version: v3.27.1

  • Dart/Flutter SDK version: Flutter 2.2.3

Workspace to reproduce issues in:
repro_workspace.zip

Activity

  1. added
    in testingRelates to test execution of Dart/Flutter tests for end users
    on Oct 11, 2021
  2. added this to the v3.27.2 milestone on Oct 11, 2021
  3. DanTup commented on Oct 11, 2021

    @DanTup
    Member

    I was able to trim the repro down to just a simple folder with two tests and still see this:

    Screenshot 2021-10-11 at 10 05 47

    It looks like we assume we'll get a Suite event before each suite runs, and there will be no overlap, but I'm no longer certain that's a good assumption - though shouldn't be too complicated to remove/fix.

  4. up-doc commented on Oct 11, 2021

    @up-doc
    Author

    @DanTup - so if this has been closed. Do we have a patch release ETA?

  5. DanTup commented on Oct 11, 2021

    @DanTup
    Member

    @moonytoes29 still working on it. The fix is in master which is why this closed, but I'm still testing it cherry-picked into a patch of 3.27. I'm hopeful it won't be long though :-)

  6. added a commit that references this issue on Oct 11, 2021
    b22056c
  7. DanTup commented on Oct 11, 2021

    @DanTup
    Member

    This should be fixed in v3.27.2. Please let me know if you still see issues (or confirm it's resolved). Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    in testingRelates to test execution of Dart/Flutter tests for end usersis bug

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions