Repository navigation
[CP-beta]Increase number of 'Mac tool_integration_tests' subshards from 5 to 10 - #189743
Conversation
flutter#189040) Rename builders from `tool_integration_tests_1_5` to `tool_integration_tests_1` so the total number isn't encoded in the builder name. Because it currently _is_ encoded in the builder name (which isn't parsed, it's just informational) the original builders need to become `bringup:true` unnecessarily, since adding shards creates new builders. Queue times are ~4 seconds, which means we seem to have enough capacity, so throwing more shards at this problem shouldn't be an issue. Will keep an eye on queue times after this change, though. Fixes flutter#189039 ### 7 shards Shard | Time -- | -- | 1 | [23 minutes](https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20tool_integration_tests_1_5/26349) 2 | [14 minutes](https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20tool_integration_tests_2_5/29560) 3 | [14 minutes](https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20tool_integration_tests_3_5/29422) 4 | [16 minutes](https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20tool_integration_tests_4_5/29435) 5 | [25 minutes](https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20tool_integration_tests_5_5/29533 ) ### 9 shards Shard | Time -- | -- | 1|[19+ minutes](https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20tool_integration_tests_1_5/26351/overview) 2|[13 minutes](https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20tool_integration_tests_2_5/29562/overview) 3|[11 minutes](https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20tool_integration_tests_3_5/29424/overview) 4|[15 minutes](https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20tool_integration_tests_4_5/29437/overview) 5|[13 minutes](https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20tool_integration_tests_5_5/29535/overview) ### 10 shards Shard | Time -- | -- | 1|[13 minutes](https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20tool_integration_tests_1_5/26352/overview) 2|[22 minutes](https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20tool_integration_tests_2_5/29563?) 3|[9 minutes](https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20tool_integration_tests_3_5/29425/overview) 4|[12 minutes](https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20tool_integration_tests_4_5/29438/overview) 5|[9.5 minutes](https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20tool_integration_tests_5_5/29536/overview) ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [AI contribution guidelines] and understand my responsibilities, or I am not using AI tools. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant documentation (doc comments with `///`). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. If this change needs to override an active code freeze, provide a comment explaining why. The code freeze workflow can be overridden by code reviewers. See pinned issues for any active code freezes with guidance. **Note**: The Flutter team is currently trialing the use of [Gemini Code Assist for GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code). Comments from the `gemini-code-assist` bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [AI contribution guidelines]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#ai-contribution-guidelines [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
|
@camsim99 please fill out the PR description above, afterwards the release team will review this request. |
|
This pull request was opened from and to a release candidate branch. This should only be done as part of the official Flutter release process. If you are attempting to make a regular contribution to the Flutter project, please close this PR and follow the instructions at Tree Hygiene for detailed instructions on contributing to Flutter. Reviewers: Use caution before merging pull requests to release branches. Ensure the proper procedure has been followed. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the CI configuration in .ci.yaml by expanding the Mac tool_integration_tests from 5 subshards to 10 subshards, enabling bringup: true, and removing the test_timeout_secs property. The review feedback identifies configuration inconsistencies across the subshards, specifically regarding the runIf trigger paths and the presence of the add_recipes_cq property.
| runIf: | ||
| - dev/** | ||
| - packages/flutter_tools/** | ||
| - bin/** | ||
| - .ci.yaml | ||
| - engine/** | ||
| - DEPS |
There was a problem hiding this comment.
There is an inconsistency in the runIf triggers between the different subshards of Mac tool_integration_tests.
Specifically, shards 5 through 9 have been configured to run on changes to bin/**, .ci.yaml, engine/**, and DEPS in addition to dev/** and packages/flutter_tools/**:
runIf:
- dev/**
- packages/flutter_tools/**
- bin/**
- .ci.yaml
- engine/**
- DEPSHowever, shards 1, 2, 3, 4, and 10 are only configured to run on:
runIf:
- dev/**
- packages/flutter_tools/**This means that if a PR only modifies files under bin/**, .ci.yaml, engine/**, or DEPS, only shards 5-9 will be triggered, while shards 1-4 and 10 will not run. This will result in incomplete test coverage and could allow regressions to slip through.
Please align the runIf triggers across all 10 subshards so they are completely consistent.
| timeout: 60 | ||
| properties: | ||
| cpu: arm64 | ||
| add_recipes_cq: "true" |
There was a problem hiding this comment.
There is an inconsistency in the properties configuration across the subshards.
Shards 6 through 10 have add_recipes_cq: "true" specified under properties, whereas shards 1 through 5 do not.
To ensure consistent behavior (especially when testing recipe changes in the recipes CQ), please either add add_recipes_cq: "true" to shards 1-5 or remove it from shards 6-10 if it is not required.
camsim99
added
the
autosubmit
c9bf3f3
into
flutter:flutter-3.47-candidate.0
3 participants
Note
This was re-created from #189299, which pointed to the old beta branch (3.46).
This pull request is created by automatic cherry pick workflow
Please fill in the form below, and a flutter domain expert will evaluate this cherry pick request.
Issue Link:
What is the link to the issue this cherry-pick is addressing?
#189039
Impact Description:
What is the impact (ex. visual jank on Samsung phones, app crash, cannot ship an iOS app)?
Does it impact development (ex. flutter doctor crashes when Android Studio is installed),
or the shipping of production apps (the app crashes on launch).
This information is for domain experts and release engineers to understand the consequences of saying yes or no to the cherry pick.
Release builders should match main. These builders were renamed in #189040 to increase the number of shards. Without this change, the beta release branch will try to run the old builder names and fail because the infra config is missing.
There will be a follow-up cherry-pick to bring it out of bringup when they pass in post-submit #189368
Changelog Description:
Explain this cherry pick:
See best practices for examples.
< Replace with changelog description here >
[flutter/189039] Improve speed of Mac Flutter tool integration tests in CI
Workaround:
Is there a workaround for this issue?
No
Risk:
What is the risk level of this cherry-pick?
Test Coverage:
Are you confident that your fix is well-tested by automated tests?
Validation Steps:
What are the steps to validate that this fix works?
This PR correctly skips
Mac tool_integration_testsin presubmit.