Repository navigation
Flutter framework customer_testing tests cocoon with latest Flutter SDK, while cocoon CI uses stable SDK #65045
Description
In #65041, a change to dartfmt caused the customer_testing shard cocoon test to fail the formatter test. When trying to update cocoon with the new formatting, Cocoon CI failed because it is based on Flutter stable SDK.
If cocoon's test suite is included in customer_testing, then Cocoon's CI should use the latest Flutter SDK, to minimize the time Cocoon's CI is broken. If Cocoon's CI needs to use stable Flutter SDK, then it should be removed from customer_testing (which usually runs latest SDK).
The version of the Flutter SDK used in Cocoon CI is defined in https://flutter.googlesource.com/recipes/+/refs/heads/master/recipes/cocoon.py#36.
Activity
keyonghan commentedon Sep 1, 2020
Shall we update cocoon CI to use latest SDK, or the other way around?
christopherfujino commentedon Sep 1, 2020
I fleshed out the description.
christopherfujino commentedon Sep 1, 2020
I'm not sure which version of Flutter we use for prod, but I'm guessing CI should use the same.
CaseyHillers commentedon Sep 2, 2020
Would it be okay to remove the dartfmt checks from flutter/tests? The more important part is the tests.
christopherfujino commentedon Sep 2, 2020
What do we do if a non-formatting breaking change happens that breaks Cocoon tests? I disabled the flutter/tests cocoon tests after hacking around the formatting failure but updating Cocoon to latest in the registry picked up an API change that failed.
christopherfujino commentedon Sep 2, 2020
I believe the failure was related to this 4d8934af485
CaseyHillers commentedon Sep 2, 2020
In this case, I think a breaking change to Cocoon's tests should then follow Flutter's breaking change policy.
Since customer tests run in presubmit, developers have been able to update Cocoon to conform to updates without needing to go through the breaking change policy.
christopherfujino commentedon Sep 2, 2020
But then to land the change, they'd have to land on red (since tests run on stable). And cocoon ci would stay red until the Flutter breaking change is published to stable. Also, does Cocoon run in prod using Flutter master? If not, then applying a breaking change could break Cocoon.
Per @godofredoc , I think he said CI used to run on master, but it would frequently be broken.
CaseyHillers commentedon Sep 2, 2020
In the case of flutter/tests#34 (IIRC Cocoon was running on stable) Cocoon was able to be rolled forward. No need to land on red.
I think these cases where it causes deadlock require a soft transition. With test code that should be possible, right?
christopherfujino commentedon Sep 3, 2020
In this case, methods were renamed 4d8934af485
CaseyHillers commentedon Dec 2, 2020
There's some issues with Cocoon when running tests on Windows. These have crept in as Cocoon's CI is only a Linux bot. There's two paths forward:
@godofredoc what do you think? #2 is a few hours of work to get the builder running and Cocoon fixed.
github-actions commentedon Aug 9, 2021
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of
flutter doctor -vand a minimal reproduction of the issue.