You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Upgrade Chrome on CI builders, "Mac web_tool_tests" timing out on macOS 12 #98278
@zanderso I wonder if the priority is correct on this - we normally get P2 if the tests cause more than 2% flakes. Do we have data on flakiness for those tests yet?
Locally Chrome 96 runs fine on the latest macOS version. That's the only evidence I have that updating Chrome may resolve this issue. If not, we'll have to investigate further. macOS frequently makes changes that require you to upgrade your apps, so trying to make an ancient version of Chrome to run on the new macOS doesn't seem useful.
@zanderso no, I think the gallery update can happen in parallel to this issue. The update to the framework to recognize the new metric names from Chrome 89+ landed here: #98629
@jmagman I think @yjbanov had a PR where he attempted to update the version of Chrome via a new CIPD tag, but I have no idea how to recreate that PR. Is it a generated change or does this need to be done manually?
changed the title [-]Mac web_tool_tests frequent flaky timeouts on CI[/-][+]Upgrade Chrome on CI builders, "Mac web_tool_tests" timing out on macOS 12[/+]on Feb 18, 2022
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.
Activity
annagrin commentedon Feb 11, 2022
@zanderso I wonder if the priority is correct on this - we normally get P2 if the tests cause more than 2% flakes. Do we have data on flakiness for those tests yet?
annagrin commentedon Feb 11, 2022
@yjbanov this would require investigation to detect a component that caused the flake.
@christopherfujino appreciate your help this.
zanderso commentedon Feb 11, 2022
Yes, P0 is correct. 7 of the last 10 runs have failed on the tree since yesterday evening, and most framework PRs including rolls are blocked on this. https://ci.chromium.org/p/flutter/builders/prod/Mac%20web_tool_tests
zanderso commentedon Feb 11, 2022
@godofredoc Aside from the Xcode change, were there any other changes in infra yesterday that might have caused this test to become extremely flaky?
zanderso commentedon Feb 11, 2022
All pre- and post- submit builds that have picked up #98263 have been passing. /cc @yusuf-goog
godofredoc commentedon Feb 11, 2022
No we haven't changed anything that will affect this test. The only changes that have been submitted to recipes are engine related changes.
godofredoc commentedon Feb 11, 2022
Can we try running the test with the old version of xcode in a mac 12 machine?
christopherfujino commentedon Feb 11, 2022
Taking a look now
christopherfujino commentedon Feb 11, 2022
Actually, it looks to me like that last failure from https://ci.chromium.org/p/flutter/builders/prod/Mac%20web_tool_tests used Xcode 13a233:
christopherfujino commentedon Feb 11, 2022
The two newer builds that have run are green (and use the previous xcode version): https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20web_tool_tests/4899/overview and https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20web_tool_tests/4900/overview
yusuf-goog commentedon Feb 11, 2022
This one passed and used 13a233: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20web_tool_tests/4897/overview
9 remaining items
yjbanov commentedon Feb 16, 2022
Locally Chrome 96 runs fine on the latest macOS version. That's the only evidence I have that updating Chrome may resolve this issue. If not, we'll have to investigate further. macOS frequently makes changes that require you to upgrade your apps, so trying to make an ancient version of Chrome to run on the new macOS doesn't seem useful.
zanderso commentedon Feb 17, 2022
What is the current status of this? In chat I see that this might be currently waiting on https://github.com/flutter/gallery/pull/604 to land? Is that correct?
ditman commentedon Feb 17, 2022
@zanderso no, I think the gallery update can happen in parallel to this issue. The update to the framework to recognize the new metric names from Chrome 89+ landed here: #98629
jmagman commentedon Feb 17, 2022
Thanks @ditman. What are next steps then, do the cipd packages have the tags they need for us to flip the version in
.ci.yaml?ditman commentedon Feb 17, 2022
@jmagman I think @yjbanov had a PR where he attempted to update the version of Chrome via a new CIPD tag, but I have no idea how to recreate that PR. Is it a generated change or does this need to be done manually?
This was the PR: #93289
jmagman commentedon Feb 17, 2022
Manually. Sounds like the infra team prefers using
verson:cipd tags #93289 (comment).flutter/.ci.yaml
Line 85 in 0eedf42
@yjbanov were you working with @hterkelsen on getting adding the tags to https://chrome-infra-packages.appspot.com/p/flutter_internal/browsers/chrome ?
[-]Mac web_tool_tests frequent flaky timeouts on CI[/-][+]Upgrade Chrome on CI builders, "Mac web_tool_tests" timing out on macOS 12[/+]ditman commentedon Mar 1, 2022
This migration happened here: #99318 (to Chromium 98)
jmagman commentedon Mar 1, 2022
https://ci.chromium.org/p/flutter/builders/prod/Mac%20web_tool_tests/5087 succeeded after the upgrade to macOS 12 and Chromium 98. Thanks for working on this @yjbanov @ditman @godofredoc!
Closing.
zanderso commentedon Mar 2, 2022
The Engine roll here #99362 is failing on Mac web_tool_tests: https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20web_tool_tests/12828/overview. Does it need to be rebased?
jmagman commentedon Mar 2, 2022
I think that machine has the malformed cipd package cached, filed #99372
github-actions commentedon Mar 16, 2022
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of
flutter doctor -vand a minimal reproduction of the issue.