Repository navigation
Run tests using line/col to support groups/test with dynamic names #4021
Description
Added by @DanTup:
Some test cases to ensure are handled:
- Run tests using line/col to support groups/test with dynamic names #4021
- Multiple tests execution #4150
- Parens around test name prevents CodeLens links from working #4168
- "No tests match regular expression...", customize regex #4099
- Debugging flutter tests fail if the descrription string has an interpolated value and a is composed of two concatenated strings #4250
Describe the bug
I have some functions to give a specific format to the descriptions of the groups and tests, like indentation, in this way it is easier to read them in the console, but when I execute a specific test, giving run in that group or test it tells me:
No tests match regular expression "^groupDescriptionFormatted\('group 1'\)".
To Reproduce
this is a minimal example:
import 'package:test/test.dart';
String groupDescriptionFormatted(String description) {
return description;
}
String subGroupDescriptionFormatted(String description) {
return '\n $description';
}
void main() {
group(
groupDescriptionFormatted('group 1'),
() {
group(subGroupDescriptionFormatted('sub group 1'), () {
test('test', () {
expect(true, isTrue);
});
});
},
);
group(
groupDescriptionFormatted('group 2'),
() {
group(subGroupDescriptionFormatted('sub group 1'), () {
test('test', () {
expect(true, isTrue);
});
});
},
);
}
If you press run or debug on a specific group or test, you can see the error, an also in the testing tab you can see that the names are the function instead of the real test name.
If you press run or debug on main, or use the command run all tests, it runs normal but also in the testing tab keep the names are the function instead of the real test name, and now also the real tests names.
Thank you.
Please complete the following information:
Operating System and version: Windows 10
VS Code version: 1.68.0
Dart extension version: 3.42.0
Dart/Flutter SDK version: flutter 3.0.1
Activity
DanTup commentedon Jun 20, 2022
Currently there are some limitations because of how tests are run individually (by name). Without executing the tests, it's impossible for the extension to know what
groupDescriptionFormatted('group 1')returns, so it's not possible to build a dart test` command that has the correct test name to run.However, once you've run the test file, you should be able to run the tests from the explorer, or from the test icons in the gutter, because the previous runs names would have been captured.
package:testdoes now have some other ways we can run tests (such as by line number/column) which may allow us to handle this better, but it'll need some work to move over (and ensure we don't try to do it for older versions ofpackage:test).I'll keep this issue open to track that, as it looks like there isn't an existing issue tracking it right now.
[-]When run an specific group or test where the description is got from a function, It Fails and get "No tests match regular expression"[/-][+]Run tests using line/col to support groups/test with dynamic names[/+]jtdLab commentedon Oct 2, 2022
Will running single tests/groups with dynamic descriptions via vs code be possible with this?
DanTup commentedon Oct 3, 2022
@jtdLab I'm hoping so, but I haven't done much testing yet (and it may depend on the exact configuration of the test). If you have some specific tests in mind, you can try running them from the terminal using the line/col like:
Where line/col are where your test definitions are. If that runs them, then it should work here. If it doesn't work, please post an example and I can do some digging.
jtdLab commentedon Oct 3, 2022
@DanTup im thinking about a scenario like this
foo_test.dart
my_shared_testing_library.dart
Is this possible via line/col and not description?
DanTup commentedon Oct 3, 2022
@jtdLab I would hope/expect that to work. You can test it by running
dart test foo_test.dart?line=1(where1is the line it's on). I'm not sure how well we handle tests without names in the Test Runner tree though (when they are discovered from source).. we might need to fabricate a name (like "test on line 1") so there is some label for the tree node.jtdLab commentedon Oct 3, 2022
@DanTup When running via the clickable Run | Debug above the test on
v3.50.0i getNo tests match regular expression "^const MyWidget\(\)( \(variant: .*\))?$".DanTup commentedon Oct 3, 2022
@jtdLab that's expected right now, using line/col has not been implemented in the extension yet.
You can use
dart test foo_test.dart?line=1from the terminal to verify wherepackage:tests's support for line/cols works for this example. If it does, then once I update the extension to use this (implementing this issue), then it should also work from the Run/Debug CodeLens links.jtdLab commentedon Oct 3, 2022
@DanTup Perfect thank you a lot
jtdLab commentedon Oct 3, 2022
@DanTup Let
<LINE>be the line of the test. When runningdart test foo_test.dart?line=<LINE>i getno matches found: foo_test.dart?line=<LINE>ontest: ^1.21.6so looks like this case is not covered there6 remaining items
Add some basic support for running tests by line number instead of name
Add some basic support for running tests by line number instead of name
Add tests for several issues requiring running tests by line
Add some basic support for running tests by line number instead of name
Add tests for several issues requiring running tests by line
Add some basic support for running tests by line number instead of name
Add tests for several issues requiring running tests by line
Add some basic support for running tests by line number instead of name
Add tests for several issues requiring running tests by line
DanTup commentedon Apr 19, 2023
In the next release (and the pre-release version just pushed today - v3.63.20240419), you can choose to have tests executed using their line numbers rather than their names. This fixes a number of issues including this one, caused by not always being able to statically compute the exact name of a test.
Please let me know if you find any issues with this functionality!