Repository navigation
[CP-stable]Move iOS compile benchmarks to Mac_benchmark and pin Mac_benchmark to M1 - #193581
Conversation
… M1 (flutter#192472) Part of flutter#186414 This batch moves the 4 iOS compile benchmarks off physical-device bots and onto the pinned benchmark pool: * `basic_material_app_ios__compile` * `flutter_gallery_ios__compile` * `hello_world_ios__compile` * `imitation_game_flutter` (task: `imitation_game_flutter__compile`) These tasks only run `flutter build ios` and never launch an app on the attached iPhone (details in [flutter#186414 (comment)](flutter#186414 (comment))). On `Mac_arm64_ios` they tie up physical devices and, because that pool mixes M1 and M4 Pro hosts, every metric is split into two disconnected Skia Perf series (`device_type=iPhone_11` vs `iPhone_16_Pro`) with a ~40–60% timing gap. Per [flutter#186414 (comment)](flutter#186414 (comment)), instead of moving them to the unpinned `Mac_arm64` pool: **Changes** - `platform_properties.mac_benchmark`: `mac_model` `"Macmini8,1"` (Intel, being turned down per [flutter.dev/go/macos-intel-deprecation](https://flutter.dev/go/macos-intel-deprecation)) → `"Macmini9,1"` (Apple Silicon M1 Mac mini), plus `cpu: arm64` and an `arm64` tag. - Moved the 4 targets from `Mac_arm64_ios` to `Mac_benchmark` with `bringup: true`. They inherit `device_type: none` and `["devicelab", "hostonly", "mac", "arm64"]` from the platform; dropping the `ios` tag also stops the devicelab recipe from running `ideviceinfo` (which would fail on a host-only bot). `flutter_gallery_ios__compile` keeps the `ruby` dependency for CocoaPods, matching `flutter_gallery_macos__compile`. - `imitation_game_swiftui` stays on `Mac_arm64_ios` — it runs `xcodebuild test` against a physical device. **Heads-up for the metrics sheriff** - All 13 existing `Mac_benchmark` macOS targets (`*_macos__compile`, `*_macos__start_up`, `complex_layout_scroll_perf_macos*`, `new_gallery_macos_impeller__transition_perf`, `flutter_tool_startup__macos`, …) move from Intel to M1 hardware with this change, so their baselines will shift (and the `arch` benchmark tag changes from `intel` to `arm`). Expect one-time step changes on Skia Perf rather than regressions. - Host-only `Macmini9,1` capacity in `luci.flutter.prod` is ~56 bots (vs ~37 Intel `Macmini8,1`), shared with `Mac_arm64`; today that pool is all M1, so the pin mostly future-proofs against M4 Pros being added.
|
@eyebrowsoffire please fill out the PR description above, afterwards the release team will review this request. |
|
This pull request was opened from and to a release candidate branch. This should only be done as part of the official Flutter release process. If you are attempting to make a regular contribution to the Flutter project, please close this PR and follow the instructions at Tree Hygiene for detailed instructions on contributing to Flutter. Reviewers: Use caution before merging pull requests to release branches. Ensure the proper procedure has been followed. |
There was a problem hiding this comment.
Code Review
This pull request updates the .ci.yaml configuration to transition benchmark bots from Intel Mac minis (Macmini8,1) to Apple Silicon M1 Mac minis (Macmini9,1), adding the arm64 CPU and tag. It also renames several Mac_arm64_ios compile targets to Mac_benchmark, enables bringup: true for them, and removes or updates their tags and dependencies. I have no feedback to provide.
|
autosubmit label was removed for flutter/flutter/193581, because - The status or check suite Dashboard Checks has failed. Please fix the issues identified (or deflake) before re-applying this label. |
a9f5569
into
flutter:flutter-3.47-candidate.0
This pull request is created by automatic cherry pick workflow
Please fill in the form below, and a flutter domain expert will evaluate this cherry pick request.
Issue Link:
What is the link to the issue this cherry-pick is addressing?
https://github.com/flutter/flutter/issues/< Replace with issue link here >
Impact Description:
What is the impact (ex. visual jank on Samsung phones, app crash, cannot ship an iOS app)?
Does it impact development (ex. flutter doctor crashes when Android Studio is installed),
or the shipping of production apps (the app crashes on launch).
This information is for domain experts and release engineers to understand the consequences of saying yes or no to the cherry pick.
< Replace with impact description here >
Changelog Description:
Explain this cherry pick:
See best practices for examples.
< Replace with changelog description here >
[flutter/<ISSUE_NUMBER>] When [on <PLAFORMS(S)>],
Workaround:
Is there a workaround for this issue?
< Replace with workaround here >
Risk:
What is the risk level of this cherry-pick?
Test Coverage:
Are you confident that your fix is well-tested by automated tests?
Validation Steps:
What are the steps to validate that this fix works?
< Replace with validation steps here >