Preamble
It would be nice if flutter test was a true superset for dart test. It's frustrating that I can only find useful and relevant CI/CD features with detailed documentation on dart test, but when I run flutter test the same behaviors are wildly different!
It would probably add a lot of benefits for you folks, too. These CI/CD features could be used in-house for proper test configurations and dev environments. Presets for example is how I'd have liked to configure my test runner's behavior locally and on the pipeline without tags or weird regex-ing of (brittle) filenames.
While I've titled this specifically regarding --preset, perhaps the task should be scoped to simply ensure some level of parity with dart test, depending on what else is missing that is!
Steps to reproduce
- Write some tests
- Create
dart_test.yaml and add a preset
- Attempt
flutter test -P some_preset
Expected results
dart_test.yaml is respected
Actual results
no flag for --preset
Code sample
N/A
Screenshots or Video
No response
Logs
N/A
Flutter Doctor output
N/A imo, but Flutter 3.44 on Windows 11 (x64) if you're curious.
Other Notes
Due to the implementation of flutter test, which itself is a wrapper around dart test, there are cases where the usage info for the dart command is surfaced even when you're running the flutter command.
If the initial flutter command is valid, but for some reason breaks once passed to the internal dart command (see #189193) the returned usage refers to the dart API which has a different feature-set and (at least for me) can lead to some rabbit holes trying to figure out why X feature doesn't work when "I just saw it in the CLI usage!"
Preamble
It would be nice if flutter test was a true superset for dart test. It's frustrating that I can only find useful and relevant CI/CD features with detailed documentation on dart test, but when I run flutter test the same behaviors are wildly different!
It would probably add a lot of benefits for you folks, too. These CI/CD features could be used in-house for proper test configurations and dev environments. Presets for example is how I'd have liked to configure my test runner's behavior locally and on the pipeline without tags or weird regex-ing of (brittle) filenames.
While I've titled this specifically regarding --preset, perhaps the task should be scoped to simply ensure some level of parity with dart test, depending on what else is missing that is!
Steps to reproduce
dart_test.yamland add a presetflutter test -P some_presetExpected results
dart_test.yaml is respected
Actual results
no flag for --preset
Code sample
N/A
Screenshots or Video
No response
Logs
N/A
Flutter Doctor output
N/A imo, but Flutter 3.44 on Windows 11 (x64) if you're curious.
Other Notes
Due to the implementation of
flutter test, which itself is a wrapper arounddart test, there are cases where the usage info for the dart command is surfaced even when you're running the flutter command.If the initial flutter command is valid, but for some reason breaks once passed to the internal dart command (see #189193) the returned usage refers to the dart API which has a different feature-set and (at least for me) can lead to some rabbit holes trying to figure out why X feature doesn't work when "I just saw it in the CLI usage!"