Repository navigation
The flutter test command should require a flutter_test or test_api dependency (and check for it). #91107
Description
Use case
The flutter test command itself is built with a certain version of the test package (whatever is pinned in the repo at the time the SDK was published). If a user is only using the regular test package, then there is nothing to ensure that the version is compatible with that of the flutter test command.
See #91059 as an example failure.
Proposal
The flutter test command should check for a dependency on flutter_test, and fail (or at least warn) if one does not exist. This package pins the test_api package to a compatible version, solving the issue.
Activity
Hixie commentedon Oct 1, 2021
Would it make sense to test for the right version of test_api, rather than flutter_test? (In the message we should suggest depending on flutter_test, certainly, but I feel like if someone is doing the dance to point to the right test_api version that should work too; they don't have to use flutter_test with
flutter test.)jakemac53 commentedon Oct 1, 2021
I would probably just depend on flutter_test, as it will take into account all the possible deps and make sure they are compatible. Being an sdk dependency it is already downloaded and essentially free (assuming it's a dev dependency).
In practice it's likely that just enforcing test_api is compatible would fix most issues though.
Hixie commentedon Oct 1, 2021
I guess asking for it to be in the pubspec.yaml as a dev dependency is probably ok. It doesn't require actually importing it.
jakemac53 commentedon Mar 23, 2023
Note dart-lang/test#1977, this is another good example of why we should be requiring this dependency even if people don't use the package as an import.
natebosch commentedon Mar 23, 2023
This is also what
webdevdoes - it checks for appropriate versions of packages likebuild_web_compilers.If any tool is going to operate with packages from the pub solve, but not be installed with that same pub solve, it should validate any assumptions it makes about how it can operate with those packages.
From the user standpoint a
dev_dependencyon the SDK vendored version offlutter_testis a far superior UX to a pinned dependency ontest_api. With the former they can be compatible across Flutter SDK releases.polina-c commentedon Mar 23, 2023
DevTools just faced the issue
guyluz11 commentedon Mar 24, 2023
Not sure if it was mentiond already but wanted to point out that in the documentation of flutter it is written that
https://docs.flutter.dev/cookbook/testing/unit/introduction
jakemac53 commentedon Mar 24, 2023
That documentation should also be updated as well to just always tell you to add the
flutter_testdependency I suppose?It feels a bit weird, but it does make sense, based on how the
flutter testcommand ships. If you rundart testwithout atestdependency it will make you add that dependency, FWIW. So it also makes sense forflutter testto require aflutter_testdependency, even if it technically can work without it, it isn't safe to run it without it.Use dart test for attributed_text
[-]The `flutter test` command should require a `flutter_test` dependency.[/-][+]The `flutter test` command should require a `flutter_test` or `test_api` dependency (and check for it).[/+]7 remaining items
flutter-triage-bot commentedon Nov 25, 2023
This issue is assigned to @christopherfujino but has had no recent status updates. Please consider unassigning this issue if it is not going to be addressed in the near future. This allows people to have a clearer picture of what work is actually planned. Thanks!
jakemac53 commentedon Dec 1, 2023
@christopherfujino this is about to come up again when we change the wire format for the
Runtimeclass here dart-lang/test#2144. This type of change should not be breaking but it ends up breaking because there is no forced constraint on package:test_api when usingflutter test, so the wire format between the tool and the package don't line up.I think we can probably make it backwards compatible, but it would be great if we didn't have to, otherwise we are stuck with this tech debt until we decide to knowingly break users on some older version of flutter.
christopherfujino commentedon Dec 1, 2023
@jakemac53 thanks for the heads up! Assigning @eliasyishak to fix.
christopherfujino commentedon Apr 1, 2024
@eliasyishak is this still on your radar?
eliasyishak commentedon Apr 2, 2024
Still tracking on my todos but haven't got around to it yet as I figure out how to handle the breaking change that comes with it
andrewkolos commentedon Jul 9, 2024
Unassigning since I am not confident I will find time to work on this within the next 2-4 weeks.