Repository navigation
customer_testing failing: type 'Null' is not a subtype of type 'Object' in type cast #123385
Description
>> .\flutter_test_registry\flutter_test_repo_test.bat
|
| C:\b\s\w\ir\x\t\flutter_customer_testing.super_editor.3c8f01bf\tests>PUSHD attributed_text
|
| C:\b\s\w\ir\x\t\flutter_customer_testing.super_editor.3c8f01bf\tests\attributed_text>CALL flutter test || GOTO :END
| 00:00 +0 -1: loading C:/b/s/w/ir/x/t/flutter_customer_testing.super_editor.3c8f01bf/tests/attributed_text/test/attributed_spans_test.dart [E]
| Failed to load "C:/b/s/w/ir/x/t/flutter_customer_testing.super_editor.3c8f01bf/tests/attributed_text/test/attributed_spans_test.dart": type 'Null' is not a subtype of type 'Object' in type cast
| package:test_api RemoteListener.start.<fn>.<fn>
|
| 00:00 +0 -2: loading C:/b/s/w/ir/x/t/flutter_customer_testing.super_editor.3c8f01bf/tests/attributed_text/test/attributed_text_test.dart [E]
| Failed to load "C:/b/s/w/ir/x/t/flutter_customer_testing.super_editor.3c8f01bf/tests/attributed_text/test/attributed_text_test.dart": type 'Null' is not a subtype of type 'Object' in type cast
| package:test_api RemoteListener.start.<fn>.<fn>
|
| 00:00 +0 -2: Some tests failed.
|
|
| Testing failed.
see: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20customer_testing/12080/overview
logs: https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8785773537030190081/+/u/Customer_testing/customer_testing/test_stdout
First appeared with roll commit 077cccd. Appears to be flaky, with 0-3 of the *_customer_testing shards failing each commit. I would have expected all three shards to consistent fail given that the error message seems to indicate a compilation issue.
Activity
zanderso commentedon Mar 24, 2023
A new version of mockito was published 12 hours ago, and that aligns with when these tests started to fail. In passing runs super_editor has mockito at
5.3.2, and in failing runs super_editor has mockito at5.4.0.The constraint for the package looks like this https://github.com/superlistapp/super_editor/blob/main/attributed_text/pubspec.yaml#L18.
@natebosch @matthew-carroll.
zanderso commentedon Mar 24, 2023
Bumping to P0 because the tree is closed and rollers are stopped.
zanderso commentedon Mar 24, 2023
It looks like I have permissions to retract
mockitoat 5.4.0, so I'm going to do that to get the tree back open. If that is a mistake, it looks like it is straightforward to undo.natebosch commentedon Mar 24, 2023
This error is not related directly to
mockito. The reason you are seeing this is that the new version allowed a different version oftest_apiwhich is incompatible withflutter test.You should not use the
flutter testcommand to run tests unless you have a dependency on theflutter_testpackage.See #91107
Please unretract
mockitosince that version is needed to rolltest_apiinto the flutter SDK. To get unblocked for these failures you can either switch the command todart testfromflutter test, or add a dependency onflutter_test.zanderso commentedon Mar 24, 2023
@natebosch Flutter's breaking change policy is that any breakage to a
customer_testtest is a breaking change that requires us to migrate customer code. The Flutter/Dart team will need to submit PRs to https://github.com/superlistapp/super_editor to perform the migration for any breaking changes that need to land.zanderso commentedon Mar 24, 2023
To be more clear about what the policy is, it is the responsibility of the person landing the breaking change to migrate the customer repo.
zanderso commentedon Mar 24, 2023
The tree is back open, so I will close this issue. @natebosch feel free to file a new issue to track any follow-up that's needed.
natebosch commentedon Mar 24, 2023
The breaking change I landed was modeled correctly with semantic versioning.
This is "breaking" because of a bug in
flutter test, and it has a workaround for end users. There is nothing I can do on the package publishing side to makeflutter testnot be broken. We should not wait for a new release of flutter with a fixedflutter testbefore we can make any progress with improvements to thetestrunner.natebosch commentedon Mar 24, 2023
For expediency I opened https://github.com/superlistapp/super_editor/pull/1071 to fix the customer test. As a general practice I do not expect it to be the responsibility of a pub package publisher to fix client breakages like this.
Hixie commentedon Mar 24, 2023
Is there an issue filed on this bug?
It's not our intent to have that limitation.
flutter testshould work fine for any kind of test.Hixie commentedon Mar 24, 2023
Looks like #91107 is the issue.
natebosch commentedon Mar 24, 2023
It won't work for node tests today, and AFAIK it won't work for non-flutter tests on the browser (it might, but flutter has a custom browser platform that likely only works for flutter tests). It does work for VM tests, and a dependency on
flutter_testis sufficient to guarantee that compatibility.matthew-carroll commentedon Mar 24, 2023
I just saw this thread. I see that there's a desire to change Super Editor from
flutter testtodart testforattributed_text.@Hixie it looks like you're saying that shouldn't be necessary.
@natebosch it looks like you're saying that it is.
Am I expected to accept the change to Super Editor, or should I expect this thread to work itself out such that Super Editor's change is unnecessary?
natebosch commentedon Mar 24, 2023
I do not expect this to work itself out.
There are two possible changes for the Super Editor repo.
dart test- this is already what it is doing for CI and is what my PR does. This is the change I would recommend.flutter_testfromattributed_text.My expectation is that the eventual fix in flutter will be to give you an earlier warning that you need to make one of those 2 fixes, so that you won't be broken by situations like this.
Hixie commentedon Mar 24, 2023
I misunderstood the
have a dependency on the flutter_test packagebit above. I thought this meant a dependency on that package from the tests. You definitely need to have the (dev) dependency on the package (or test_api) from your pubspec.yaml, though.github-actions commentedon Apr 10, 2023
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of
flutter doctor -vand a minimal reproduction of the issue.