Repository navigation
Warn if multiple tests run when trying to run a single test by name #2832
Description
Have a look at this screencast
https://www.screencast.com/t/j0utedFnzL
We found out the reason is that there is another test with the same name further up the file
Versions (please complete the following information):
- VS Code version: 1.49.1
- Dart extension version: v3.14.1
- Dart/Flutter SDK version:1.22.0
Activity
DanTup commentedon Sep 30, 2020
This is unfortunately a known limitation. I don't believe
pub run testhas any way to run a single instance of a test if it has the exact same name (including group) as another test.I don't think it's very common to have this, and they should both show up in the test tree (though trying to re-run either will re-run both), so hopefully it's easy to spot and correct.
escamoteur commentedon Sep 30, 2020
Don't you think it would make sense to give a warning in such a case? especially as tests are often copied something like this can easily happen and the result is conpletely confusing
DanTup commentedon Sep 30, 2020
Yep, I do - but unfortunately it's not very simple to do reliably, as tests can be dynamic (include variable names, etc.).
There's an open issue in
package:testdiscussing some possible improvements in this area:dart-lang/test#1328
And some related things here:
dart-lang/test#1332
Hopefully these will help improve some of these things here.
DanTup commentedon Dec 8, 2020
I've excluded when test names contain variables (since that's more tricky), but for the most common case we'll know show a warning at the end of a test run if we expected only a single test to run (that is, you're not running a group, there is exactly one test name, and it does not contain a
$) but multiple did:[-]if more than one test has the same name clicking on the debug or run link always executes the one further up[/-][+]Warn if multiple tests run when trying to run a single test by name[/+]escamoteur commentedon Dec 8, 2020
I think that should be enough in most cased. will close this now.
what happens if you click run on one of the tests that have a brother with the same name?
DanTup commentedon Dec 8, 2020
(Re-opening as I haven't landed the change yet, but it'll close when I do :-))
I'm not sure what you mean - isn't this the same case described above? Or do you mean a non-sibling test? That should work as expected, as the "test name" we construct when running from code lens is actually a full name of all of the groups prefixed to it. It's only if the complete name matches that this will trigger. This is similar to how if you have two tests named
"foo"in different groups, running one of them doesn't run the other today already.escamoteur commentedon Dec 8, 2020
ah, I reread your comment about. I wasn't sure if it were possible that just one but the wrong test is executed.
DanTup commentedon Dec 8, 2020
It shouldn't be - we should always run the test that you wanted, but if there is another test with the exact same name, that may run too. All we do now is count how many tests ran, and if it's
> 1but we had run in a way that we expected only one (eg. a single test name) we'll show the warning.