Repository navigation
Multiple tests execution #4150
Description
Describe the bug
Once you're trying to run test, which description contains variable or type used in other test, multiple tests would be executed.
This could be annoying if you're using some kind of BLoC/Redux actions, which types may be located in multiple test descriptions.
Simular issue in /test repo: dart-lang/test#1760.
There we found out, that the plugin could interprete expressions like 'is $Object' incorrectly.
To Reproduce
Run 'is $Object' test.
import 'package:test/test.dart';
class MyObj {
final number = 10;
bool foo() => true;
}
void main() {
group('$MyObj', () {
late MyObj myObj;
setUp(() {
myObj = MyObj();
});
test('is $Object', () {
expect(myObj, isA<Object>());
});
test('foo returns true and is an $Object', () {
expect(myObj.foo(), isTrue);
expect(myObj, isA<Object>());
});
test('number equals 10', () {
expect(myObj.number, equals(10));
});
});
}Expected behavior
Running 'is $Object' test, via IDE would execute only selected test.
Screenshots
It seems like plugin runs test like this.
Where $Object is an empty env. Which means, that we try to run test with 'is ' description, which also suits for the second test.
Please complete the following information:
- Operating System and version: macOS 12.3.1 (21E258)
- VS Code version: 1.71.0
- Dart extension version: v3.48.2
- Dart/Flutter SDK version: 2.17.3
Activity
DanTup commentedon Sep 7, 2022
There are some known limitations in this area. When we run tests, we cannot use the literal name in the source if it contains variables:
We also cannot tell what
$MyObjor$Objectare to substitute their values, since they could be dynamic:So right now, any variable is simple substitute with the regular expression pattern
.*so it will match any value at runtime. This results in the behaviour that you see.There are plans to improve this though, there were some changes made to
package:testthat will allow us to run tests by their line number. If we switch to this, we will be able to more accurately pinpoint a test to be run without relying on names that we can't compute the exact values for. That's being tracked in #4021, so I'll close this as a dupe of that. You can subscribe there for updates.Thanks!
DanTup commentedon Apr 18, 2023
Re-opening this for tracking to ensure this issue is fixed by the upcoming support for running tests by lines.
Add tests for several issues requiring running tests by line
Add tests for several issues requiring running tests by line
Add tests for several issues requiring running tests by line
Add tests for several issues requiring running tests by line
DanTup commentedon Apr 19, 2023
In the next release (and the pre-release version just pushed today - v3.63.20240419), you can choose to have tests executed using their line numbers rather than their names. This fixes a number of issues including this one, caused by not always being able to statically compute the exact name of a test.
Please let me know if you find any issues with this functionality!