Pin `customer_testing` to the SHA specified in `tests.version` by matanlurey · Pull Request #162048 · flutter/flutter

matanlurey · 2025-01-23T00:04:26Z

Closes #162041.

I also converted the .sh and .bat files (largely) to Dart, mostly because I don't know enough Windows XP command prompt in order to make the change safely. They should still run the same from the CI bootstrapping, but most of the logic is now Dart code.

Will send a separate PR to update the flutter/tests PR template.

/cc @johnmccutchan @Piinks

flutter-dashboard · 2025-01-23T00:04:29Z

It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption, contact "@test-exemption-reviewer" in the #hackers channel in Discord (don't just cc them here, they won't see it!).

If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix?

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing. The test exemption team is a small volunteer group, so all reviewers should feel empowered to ask for tests, without delegating that responsibility entirely to the test exemption group.

Part of flutter/flutter#162048.

zanderso

lgtm, and probably needs comms to flutter-team, test owners, and on Discord before landing. @Piinks has offered to help with that.

Piinks · 2025-01-23T18:02:17Z

Touched base with @matanlurey, comms to go out in just a bit.

auto-submit · 2025-01-23T18:07:33Z

auto label is removed for flutter/flutter/162048, due to - The status or check suite Merge Queue Guard has failed. Please fix the issues identified (or deflake) before re-applying this label.

Piinks · 2025-01-23T18:08:30Z

Comms have been sent!

gnprice · 2025-01-23T23:16:39Z

-# Next we need to update the flutter/tests checkout.
-#
-# We use find_commit.dart so that we pull the version of flutter/tests
-# that was contemporary when the branch we are on was created. That
-# way, we can still run the tests on long-lived branches without being
-# affected by breaking changes on trunk causing changes to the tests
-# that wouldn't work on the long-lived branch.
-#
-# (This also prevents trunk from suddenly failing when tests are
-# revved on flutter/tests -- if you rerun a passing customer_tests
-# shard, it should still pass, even if we rolled one of the tests.)


It looks like this find_commit.dart script has no other callers besides the two deleted in this PR. So it'd be good to delete at the same time.

matanlurey · 2025-01-23T23:56:17Z

Just desk reviewed the Windows changes with @jtmcdole; tl;dr don't use package:glob if you can future selves.

jtmcdole

Glob should have provided some /\ conversion support. That was "fun".

harri35 · 2025-01-24T11:53:34Z

@@ -0,0 +1,15 @@
+# customer_testing
+
+This tool checks out <https://github.com/flutter/tests> at the SHA specified in


Maybe easier to read if the sentence says explicitly it is talking about a Git commit SHA?

Suggested change

This tool checks out <https://github.com/flutter/tests> at the SHA specified in

This tool checks out <https://github.com/flutter/tests> at the commit SHA specified in

Maybe it is just me, feel free to ignore it.

harri35 · 2025-01-24T12:07:40Z

+/// Or:
+///
+/// ```sh
+/// ./dev/customer_testing/ci.sh [sha]


Shouldn't this example be just:

Suggested change

/// ./dev/customer_testing/ci.sh [sha]

/// ./dev/customer_testing/ci.sh

as it actually does not take (or forward to ci.dart) any parameters?

harri35 · 2025-01-24T12:09:51Z

+
+import 'package:path/path.dart' as p;
+
+/// To run this script, either:


Just a suggestion, feel free to ignore - maybe it makes sense to mention the ci.bat here also, so it is explicitly clear both exist (and do the same)?

flutter/flutter#162048)

Roll Flutter from c1ffaa9 to b007899 (43 revisions) flutter/flutter@c1ffaa9...b007899 2025-01-29 [email protected] Fix `Tab` linear and elastic animation blink (flutter/flutter#162315) 2025-01-29 [email protected] Pass-through `textInputAction` in `DropdownMenu` (flutter/flutter#162309) 2025-01-29 [email protected] Fix scrollUntilVisible in WidgetTester (flutter/flutter#159582) 2025-01-29 [email protected] Pass-through `maxLines` in `DropdownMenu` (flutter/flutter#161903) 2025-01-29 [email protected] fix: appbar leading width is not square for custom toolbar height (flutter/flutter#161880) 2025-01-29 [email protected] [DisplayList] Don't call Skia Ganesh methods when its not available. (flutter/flutter#162345) 2025-01-29 [email protected] Update README.md to include googler post verification steps (flutter/flutter#162272) 2025-01-29 [email protected] [engine, web] return switch expressions in many places (flutter/flutter#162336) 2025-01-29 [email protected] Update README.md to not have engine link for android (flutter/flutter#162330) 2025-01-29 [email protected] Reland "[ Widget Previews ] Add support for detecting previews and generating code (#161911)"" (flutter/flutter#162337) 2025-01-29 [email protected] Add instructions to download the Gradle wrapper to FGP readme, and add to gitignore (flutter/flutter#162332) 2025-01-29 [email protected] Fix tests to prepare for `--explicit-package-dependencies` and a bug. (flutter/flutter#162289) 2025-01-29 [email protected] Add a currently unused `runs_in_merge_queue` property to `Linux analyze`. (flutter/flutter#162335) 2025-01-28 98614782+auto-submit[bot]@users.noreply.github.com Reverts "[ Widget Previews ] Add support for detecting previews and generating code (#161911)" (flutter/flutter#162327) 2025-01-28 [email protected] Support hot restart for DDC library bundle format (flutter/flutter#162123) 2025-01-28 [email protected] Started adjusting uvs to match pixel snapping. (flutter/flutter#162049) 2025-01-28 [email protected] Refactor code inside flutter.groovy (flutter/flutter#160250) 2025-01-28 [email protected] Table implements redepth (flutter/flutter#162282) 2025-01-28 [email protected] [ Widget Previews ] Add support for detecting previews and generating code (flutter/flutter#161911) 2025-01-28 [email protected] remove dependency on `Usage` from `Pub` class (flutter/flutter#162279) 2025-01-28 [email protected] Roll Packages from 258f6dc to 02c6fef (6 revisions) (flutter/flutter#162313) 2025-01-28 [email protected] Remove `scenario_app/android` and rename to `ios_scenario_app`. (flutter/flutter#160992) 2025-01-28 [email protected] Apparently it is illegal to use `stderr` in this project. (flutter/flutter#162294) 2025-01-28 [email protected] Fix update_engine_version_test in presence of FLUTTER_PREBUILT_ENGINE_VERSION env vars. (flutter/flutter#162270) 2025-01-28 [email protected] Add missing `properties: ...` and move to presubmit. (flutter/flutter#162170) 2025-01-27 [email protected] [Impeller] make swapchain related external fence/semaphore extensions optional. (flutter/flutter#162205) 2025-01-27 49699333+dependabot[bot]@users.noreply.github.com Bump the all-github-actions group with 2 updates (flutter/flutter#162277) 2025-01-27 [email protected] Don't depend on Dart from FML. (flutter/flutter#162271) 2025-01-27 [email protected] [DisplayList] Move nested canvas enums into their own TU. (flutter/flutter#162037) 2025-01-27 [email protected] Avoid iOS text selection crash by returning nil range (flutter/flutter#161996) 2025-01-27 [email protected] fix `felt` link to point to flutter repo instead of the engine repo (flutter/flutter#161423) 2025-01-27 [email protected] Enable the Android Engine OpenGLES/Vulkan suites. (flutter/flutter#162258) 2025-01-27 [email protected] [canvaskit] Fix debug build for CanvasKit (flutter/flutter#162198) 2025-01-27 [email protected] Roll Packages from 3d3ab7b to 258f6dc (19 revisions) (flutter/flutter#162254) 2025-01-25 [email protected] Pin `customer_testing` to the SHA specified in `tests.version` (flutter/flutter#162048) 2025-01-25 [email protected] Formalize `update_engine_version.{sh|ps1}`. (flutter/flutter#162118) 2025-01-25 [email protected] Rename 'SelectionChangedCause.scribble' to 'SelectionChangedCause.stylusHandwriting' (flutter/flutter#161518) 2025-01-25 [email protected] Don't install xcode when doing `local_engine` web builds on mac. (flutter/flutter#162164) 2025-01-25 [email protected] Force Impeller backend for `android_engine_test`, and test both OpenGLES and Vulkan (flutter/flutter#162089) 2025-01-24 [email protected] [Impeller] when a command pool has many unused buffers, reset with release resources flag. (flutter/flutter#162171) 2025-01-24 [email protected] [web] Remove HTML renderer from framework tests (flutter/flutter#162038) 2025-01-24 [email protected] [Impeller] Skip clip entity replay that cannot impact current clip. (flutter/flutter#162113) 2025-01-24 [email protected] Update Android integration test package for newer AGP (flutter/flutter#161856) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: ...

Part of flutter/flutter#162048. This can be of course tweaked and improved.

Looks like this is the first roll since this pin was introduced in flutter#162041 / flutter#162048. For fun and the possible convenience of the next person doing one of these, here's the command I used to generate the log below: $ git log --oneline --no-decorate --format='%cs %aE %s' b4cc097.. \ | perl -pe 's,#,flutter/tests#,g' flutter/tests@b4cc097...6a6d096 2025-02-05 [email protected] Re-land "Update Zulip tests and re-enable (flutter/tests#441)" (flutter/tests#446) 2025-01-30 [email protected] Update contact for pkg:animations (flutter/tests#447) 2025-01-24 [email protected] Add a PR template that explains updating `tests.version` (flutter/tests#445)

Looks like this is the first roll since this pin was introduced in #162041 / #162048. For fun and the possible convenience of the next person doing one of these, here's the command I used to generate the log below: $ git log --oneline --no-decorate --format='%cs %aE %s' b4cc097.. \ | perl -pe 's,#,flutter/tests#,g' flutter/tests@b4cc097...6a6d096 2025-02-05 [email protected] Re-land "Update Zulip tests and re-enable (flutter/tests#441)" (flutter/tests#446) 2025-01-30 [email protected] Update contact for pkg:animations (flutter/tests#447) 2025-01-24 [email protected] Add a PR template that explains updating `tests.version` (flutter/tests#445) ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [ ] I listed at least one issue that this PR fixes in the description above. - [ ] I updated/added relevant documentation (doc comments with `///`). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord].  [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md

This updates to the following change in Zulip: zulip/zulip-flutter#1480 zulip/zulip-flutter@88e7bcc in order to unblock this PR: flutter/flutter#164818 > [!IMPORTANT] > This repository is a testing suite that contains references to tests (in the `registry` directory) that are run with every commit to Flutter > to verify that no breaking changes have been introduced (in the "customer_testing" shards). Your merged PR is _not_ automatically run in the > Flutter CI tree. > > After merging this PR, you must send another PR to flutter/flutter updating `dev/customer_testing/tests.version` in that repo with the latest git commit SHA of the flutter/test repo. > > See <flutter/flutter#162048> for details.

flutter/flutter#162048)

Roll Flutter from c1ffaa9 to b007899 (43 revisions) flutter/flutter@c1ffaa9...b007899 2025-01-29 [email protected] Fix `Tab` linear and elastic animation blink (flutter/flutter#162315) 2025-01-29 [email protected] Pass-through `textInputAction` in `DropdownMenu` (flutter/flutter#162309) 2025-01-29 [email protected] Fix scrollUntilVisible in WidgetTester (flutter/flutter#159582) 2025-01-29 [email protected] Pass-through `maxLines` in `DropdownMenu` (flutter/flutter#161903) 2025-01-29 [email protected] fix: appbar leading width is not square for custom toolbar height (flutter/flutter#161880) 2025-01-29 [email protected] [DisplayList] Don't call Skia Ganesh methods when its not available. (flutter/flutter#162345) 2025-01-29 [email protected] Update README.md to include googler post verification steps (flutter/flutter#162272) 2025-01-29 [email protected] [engine, web] return switch expressions in many places (flutter/flutter#162336) 2025-01-29 [email protected] Update README.md to not have engine link for android (flutter/flutter#162330) 2025-01-29 [email protected] Reland "[ Widget Previews ] Add support for detecting previews and generating code (#161911)"" (flutter/flutter#162337) 2025-01-29 [email protected] Add instructions to download the Gradle wrapper to FGP readme, and add to gitignore (flutter/flutter#162332) 2025-01-29 [email protected] Fix tests to prepare for `--explicit-package-dependencies` and a bug. (flutter/flutter#162289) 2025-01-29 [email protected] Add a currently unused `runs_in_merge_queue` property to `Linux analyze`. (flutter/flutter#162335) 2025-01-28 98614782+auto-submit[bot]@users.noreply.github.com Reverts "[ Widget Previews ] Add support for detecting previews and generating code (#161911)" (flutter/flutter#162327) 2025-01-28 [email protected] Support hot restart for DDC library bundle format (flutter/flutter#162123) 2025-01-28 [email protected] Started adjusting uvs to match pixel snapping. (flutter/flutter#162049) 2025-01-28 [email protected] Refactor code inside flutter.groovy (flutter/flutter#160250) 2025-01-28 [email protected] Table implements redepth (flutter/flutter#162282) 2025-01-28 [email protected] [ Widget Previews ] Add support for detecting previews and generating code (flutter/flutter#161911) 2025-01-28 [email protected] remove dependency on `Usage` from `Pub` class (flutter/flutter#162279) 2025-01-28 [email protected] Roll Packages from 258f6dc to 02c6fef (6 revisions) (flutter/flutter#162313) 2025-01-28 [email protected] Remove `scenario_app/android` and rename to `ios_scenario_app`. (flutter/flutter#160992) 2025-01-28 [email protected] Apparently it is illegal to use `stderr` in this project. (flutter/flutter#162294) 2025-01-28 [email protected] Fix update_engine_version_test in presence of FLUTTER_PREBUILT_ENGINE_VERSION env vars. (flutter/flutter#162270) 2025-01-28 [email protected] Add missing `properties: ...` and move to presubmit. (flutter/flutter#162170) 2025-01-27 [email protected] [Impeller] make swapchain related external fence/semaphore extensions optional. (flutter/flutter#162205) 2025-01-27 49699333+dependabot[bot]@users.noreply.github.com Bump the all-github-actions group with 2 updates (flutter/flutter#162277) 2025-01-27 [email protected] Don't depend on Dart from FML. (flutter/flutter#162271) 2025-01-27 [email protected] [DisplayList] Move nested canvas enums into their own TU. (flutter/flutter#162037) 2025-01-27 [email protected] Avoid iOS text selection crash by returning nil range (flutter/flutter#161996) 2025-01-27 [email protected] fix `felt` link to point to flutter repo instead of the engine repo (flutter/flutter#161423) 2025-01-27 [email protected] Enable the Android Engine OpenGLES/Vulkan suites. (flutter/flutter#162258) 2025-01-27 [email protected] [canvaskit] Fix debug build for CanvasKit (flutter/flutter#162198) 2025-01-27 [email protected] Roll Packages from 3d3ab7b to 258f6dc (19 revisions) (flutter/flutter#162254) 2025-01-25 [email protected] Pin `customer_testing` to the SHA specified in `tests.version` (flutter/flutter#162048) 2025-01-25 [email protected] Formalize `update_engine_version.{sh|ps1}`. (flutter/flutter#162118) 2025-01-25 [email protected] Rename 'SelectionChangedCause.scribble' to 'SelectionChangedCause.stylusHandwriting' (flutter/flutter#161518) 2025-01-25 [email protected] Don't install xcode when doing `local_engine` web builds on mac. (flutter/flutter#162164) 2025-01-25 [email protected] Force Impeller backend for `android_engine_test`, and test both OpenGLES and Vulkan (flutter/flutter#162089) 2025-01-24 [email protected] [Impeller] when a command pool has many unused buffers, reset with release resources flag. (flutter/flutter#162171) 2025-01-24 [email protected] [web] Remove HTML renderer from framework tests (flutter/flutter#162038) 2025-01-24 [email protected] [Impeller] Skip clip entity replay that cannot impact current clip. (flutter/flutter#162113) 2025-01-24 [email protected] Update Android integration test package for newer AGP (flutter/flutter#161856) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: ...

…ration (#460) This updates to the following change in Zulip: zulip/zulip-flutter@834834b zulip/zulip-flutter#1547 in order to unblock this PR: flutter/flutter#165832 > [!IMPORTANT] > This repository is a testing suite that contains references to tests (in the `registry` directory) that are run with every commit to Flutter > to verify that no breaking changes have been introduced (in the "customer_testing" shards). Your merged PR is _not_ automatically run in the > Flutter CI tree. > > After merging this PR, you must send another PR to flutter/flutter updating `dev/customer_testing/tests.version` in that repo with the latest git commit SHA of the flutter/test repo. > > See <flutter/flutter#162048> for details.

Updates DevTools in customer testing to roll: flutter/devtools#9459 This unblocks: flutter/flutter#173862 > [!IMPORTANT] > This repository is a testing suite that contains references to tests (in the `registry` directory) that are run with every commit to Flutter > to verify that no breaking changes have been introduced (in the "customer_testing" shards). Your merged PR is _not_ automatically run in the > Flutter CI tree. > > After merging this PR, you must send another PR to flutter/flutter updating `dev/customer_testing/tests.version` in that repo with the latest git commit SHA of the flutter/test repo. > > See <flutter/flutter#162048> for details.

This is merely an `analyzer` change to avoid the bug explained at: - dart-lang/sdk#62555 > [!IMPORTANT] > This repository is a testing suite that contains references to tests (in the `registry` directory) that are run with every commit to Flutter > to verify that no breaking changes have been introduced (in the "customer_testing" shards). Your merged PR is _not_ automatically run in the > Flutter CI tree. > > After merging this PR, you must send another PR to flutter/flutter updating `dev/customer_testing/tests.version` in that repo with the latest git commit SHA of the flutter/test repo. > > See <flutter/flutter#162048> for details.

Hi, Flutter Team, as Patrol team, we want to reanimate patrol tests that were disabled. > [!IMPORTANT] > This repository is a testing suite that contains references to tests (in the `registry` directory) that are run with every commit to Flutter > to verify that no breaking changes have been introduced (in the "customer_testing" shards). Your merged PR is _not_ automatically run in the > Flutter CI tree. > > After merging this PR, you must send another PR to flutter/flutter updating `dev/customer_testing/tests.version` in that repo with the latest git commit SHA of the flutter/test repo. > > See <flutter/flutter#162048> for details.

Roll Flutter from c1ffaa9 to b007899 (43 revisions) flutter/flutter@c1ffaa9...b007899 2025-01-29 [email protected] Fix `Tab` linear and elastic animation blink (flutter/flutter#162315) 2025-01-29 [email protected] Pass-through `textInputAction` in `DropdownMenu` (flutter/flutter#162309) 2025-01-29 [email protected] Fix scrollUntilVisible in WidgetTester (flutter/flutter#159582) 2025-01-29 [email protected] Pass-through `maxLines` in `DropdownMenu` (flutter/flutter#161903) 2025-01-29 [email protected] fix: appbar leading width is not square for custom toolbar height (flutter/flutter#161880) 2025-01-29 [email protected] [DisplayList] Don't call Skia Ganesh methods when its not available. (flutter/flutter#162345) 2025-01-29 [email protected] Update README.md to include googler post verification steps (flutter/flutter#162272) 2025-01-29 [email protected] [engine, web] return switch expressions in many places (flutter/flutter#162336) 2025-01-29 [email protected] Update README.md to not have engine link for android (flutter/flutter#162330) 2025-01-29 [email protected] Reland "[ Widget Previews ] Add support for detecting previews and generating code (#161911)"" (flutter/flutter#162337) 2025-01-29 [email protected] Add instructions to download the Gradle wrapper to FGP readme, and add to gitignore (flutter/flutter#162332) 2025-01-29 [email protected] Fix tests to prepare for `--explicit-package-dependencies` and a bug. (flutter/flutter#162289) 2025-01-29 [email protected] Add a currently unused `runs_in_merge_queue` property to `Linux analyze`. (flutter/flutter#162335) 2025-01-28 98614782+auto-submit[bot]@users.noreply.github.com Reverts "[ Widget Previews ] Add support for detecting previews and generating code (#161911)" (flutter/flutter#162327) 2025-01-28 [email protected] Support hot restart for DDC library bundle format (flutter/flutter#162123) 2025-01-28 [email protected] Started adjusting uvs to match pixel snapping. (flutter/flutter#162049) 2025-01-28 [email protected] Refactor code inside flutter.groovy (flutter/flutter#160250) 2025-01-28 [email protected] Table implements redepth (flutter/flutter#162282) 2025-01-28 [email protected] [ Widget Previews ] Add support for detecting previews and generating code (flutter/flutter#161911) 2025-01-28 [email protected] remove dependency on `Usage` from `Pub` class (flutter/flutter#162279) 2025-01-28 [email protected] Roll Packages from 258f6dc to 02c6fef (6 revisions) (flutter/flutter#162313) 2025-01-28 [email protected] Remove `scenario_app/android` and rename to `ios_scenario_app`. (flutter/flutter#160992) 2025-01-28 [email protected] Apparently it is illegal to use `stderr` in this project. (flutter/flutter#162294) 2025-01-28 [email protected] Fix update_engine_version_test in presence of FLUTTER_PREBUILT_ENGINE_VERSION env vars. (flutter/flutter#162270) 2025-01-28 [email protected] Add missing `properties: ...` and move to presubmit. (flutter/flutter#162170) 2025-01-27 [email protected] [Impeller] make swapchain related external fence/semaphore extensions optional. (flutter/flutter#162205) 2025-01-27 49699333+dependabot[bot]@users.noreply.github.com Bump the all-github-actions group with 2 updates (flutter/flutter#162277) 2025-01-27 [email protected] Don't depend on Dart from FML. (flutter/flutter#162271) 2025-01-27 [email protected] [DisplayList] Move nested canvas enums into their own TU. (flutter/flutter#162037) 2025-01-27 [email protected] Avoid iOS text selection crash by returning nil range (flutter/flutter#161996) 2025-01-27 [email protected] fix `felt` link to point to flutter repo instead of the engine repo (flutter/flutter#161423) 2025-01-27 [email protected] Enable the Android Engine OpenGLES/Vulkan suites. (flutter/flutter#162258) 2025-01-27 [email protected] [canvaskit] Fix debug build for CanvasKit (flutter/flutter#162198) 2025-01-27 [email protected] Roll Packages from 3d3ab7b to 258f6dc (19 revisions) (flutter/flutter#162254) 2025-01-25 [email protected] Pin `customer_testing` to the SHA specified in `tests.version` (flutter/flutter#162048) 2025-01-25 [email protected] Formalize `update_engine_version.{sh|ps1}`. (flutter/flutter#162118) 2025-01-25 [email protected] Rename 'SelectionChangedCause.scribble' to 'SelectionChangedCause.stylusHandwriting' (flutter/flutter#161518) 2025-01-25 [email protected] Don't install xcode when doing `local_engine` web builds on mac. (flutter/flutter#162164) 2025-01-25 [email protected] Force Impeller backend for `android_engine_test`, and test both OpenGLES and Vulkan (flutter/flutter#162089) 2025-01-24 [email protected] [Impeller] when a command pool has many unused buffers, reset with release resources flag. (flutter/flutter#162171) 2025-01-24 [email protected] [web] Remove HTML renderer from framework tests (flutter/flutter#162038) 2025-01-24 [email protected] [Impeller] Skip clip entity replay that cannot impact current clip. (flutter/flutter#162113) 2025-01-24 [email protected] Update Android integration test package for newer AGP (flutter/flutter#161856) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: ...

Roll Flutter from c1ffaa9 to b007899 (43 revisions) flutter/flutter@c1ffaa9...b007899 2025-01-29 [email protected] Fix `Tab` linear and elastic animation blink (flutter/flutter#162315) 2025-01-29 [email protected] Pass-through `textInputAction` in `DropdownMenu` (flutter/flutter#162309) 2025-01-29 [email protected] Fix scrollUntilVisible in WidgetTester (flutter/flutter#159582) 2025-01-29 [email protected] Pass-through `maxLines` in `DropdownMenu` (flutter/flutter#161903) 2025-01-29 [email protected] fix: appbar leading width is not square for custom toolbar height (flutter/flutter#161880) 2025-01-29 [email protected] [DisplayList] Don't call Skia Ganesh methods when its not available. (flutter/flutter#162345) 2025-01-29 [email protected] Update README.md to include googler post verification steps (flutter/flutter#162272) 2025-01-29 [email protected] [engine, web] return switch expressions in many places (flutter/flutter#162336) 2025-01-29 [email protected] Update README.md to not have engine link for android (flutter/flutter#162330) 2025-01-29 [email protected] Reland "[ Widget Previews ] Add support for detecting previews and generating code (#161911)"" (flutter/flutter#162337) 2025-01-29 [email protected] Add instructions to download the Gradle wrapper to FGP readme, and add to gitignore (flutter/flutter#162332) 2025-01-29 [email protected] Fix tests to prepare for `--explicit-package-dependencies` and a bug. (flutter/flutter#162289) 2025-01-29 [email protected] Add a currently unused `runs_in_merge_queue` property to `Linux analyze`. (flutter/flutter#162335) 2025-01-28 98614782+auto-submit[bot]@users.noreply.github.com Reverts "[ Widget Previews ] Add support for detecting previews and generating code (#161911)" (flutter/flutter#162327) 2025-01-28 [email protected] Support hot restart for DDC library bundle format (flutter/flutter#162123) 2025-01-28 [email protected] Started adjusting uvs to match pixel snapping. (flutter/flutter#162049) 2025-01-28 [email protected] Refactor code inside flutter.groovy (flutter/flutter#160250) 2025-01-28 [email protected] Table implements redepth (flutter/flutter#162282) 2025-01-28 [email protected] [ Widget Previews ] Add support for detecting previews and generating code (flutter/flutter#161911) 2025-01-28 [email protected] remove dependency on `Usage` from `Pub` class (flutter/flutter#162279) 2025-01-28 [email protected] Roll Packages from 4eb5fd5 to 786f099 (6 revisions) (flutter/flutter#162313) 2025-01-28 [email protected] Remove `scenario_app/android` and rename to `ios_scenario_app`. (flutter/flutter#160992) 2025-01-28 [email protected] Apparently it is illegal to use `stderr` in this project. (flutter/flutter#162294) 2025-01-28 [email protected] Fix update_engine_version_test in presence of FLUTTER_PREBUILT_ENGINE_VERSION env vars. (flutter/flutter#162270) 2025-01-28 [email protected] Add missing `properties: ...` and move to presubmit. (flutter/flutter#162170) 2025-01-27 [email protected] [Impeller] make swapchain related external fence/semaphore extensions optional. (flutter/flutter#162205) 2025-01-27 49699333+dependabot[bot]@users.noreply.github.com Bump the all-github-actions group with 2 updates (flutter/flutter#162277) 2025-01-27 [email protected] Don't depend on Dart from FML. (flutter/flutter#162271) 2025-01-27 [email protected] [DisplayList] Move nested canvas enums into their own TU. (flutter/flutter#162037) 2025-01-27 [email protected] Avoid iOS text selection crash by returning nil range (flutter/flutter#161996) 2025-01-27 [email protected] fix `felt` link to point to flutter repo instead of the engine repo (flutter/flutter#161423) 2025-01-27 [email protected] Enable the Android Engine OpenGLES/Vulkan suites. (flutter/flutter#162258) 2025-01-27 [email protected] [canvaskit] Fix debug build for CanvasKit (flutter/flutter#162198) 2025-01-27 [email protected] Roll Packages from eee89d8 to 4eb5fd5 (19 revisions) (flutter/flutter#162254) 2025-01-25 [email protected] Pin `customer_testing` to the SHA specified in `tests.version` (flutter/flutter#162048) 2025-01-25 [email protected] Formalize `update_engine_version.{sh|ps1}`. (flutter/flutter#162118) 2025-01-25 [email protected] Rename 'SelectionChangedCause.scribble' to 'SelectionChangedCause.stylusHandwriting' (flutter/flutter#161518) 2025-01-25 [email protected] Don't install xcode when doing `local_engine` web builds on mac. (flutter/flutter#162164) 2025-01-25 [email protected] Force Impeller backend for `android_engine_test`, and test both OpenGLES and Vulkan (flutter/flutter#162089) 2025-01-24 [email protected] [Impeller] when a command pool has many unused buffers, reset with release resources flag. (flutter/flutter#162171) 2025-01-24 [email protected] [web] Remove HTML renderer from framework tests (flutter/flutter#162038) 2025-01-24 [email protected] [Impeller] Skip clip entity replay that cannot impact current clip. (flutter/flutter#162113) 2025-01-24 [email protected] Update Android integration test package for newer AGP (flutter/flutter#161856) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: ...

matanlurey added 3 commits January 22, 2025 15:31

Use manually pinned flutter/tests SHA pin.

1209a24

Pin flutter/tests with a tests.version file.

eb20cbf

Add a README.

e1aee0a

matanlurey requested review from jtmcdole and zanderso January 23, 2025 00:04

matanlurey added a commit to flutter/tests that referenced this pull request Jan 23, 2025

Add a PR template that explains updating tests.version

f25b4bc

Part of flutter/flutter#162048.

matanlurey mentioned this pull request Jan 23, 2025

Add a PR template that explains updating tests.version flutter/tests#445

Merged

zanderso approved these changes Jan 23, 2025

View reviewed changes

Comment thread dev/customer_testing/ci.bat Outdated

jtmcdole approved these changes Jan 23, 2025

View reviewed changes

Use bin/dart.

7a6d066

matanlurey requested a review from Piinks January 23, 2025 00:20

matanlurey added 3 commits January 22, 2025 19:21

Quote the *.test.

e77f461

Merge remote-tracking branch 'upstream/main' into pin-customer_testing

bbe7b44

Tweak quotes.

2a4f61e

matanlurey added the autosubmit Merge PR when tree becomes green via auto submit App label Jan 23, 2025

auto-submit Bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Jan 23, 2025

gnprice mentioned this pull request Jan 23, 2025

Re-land "Update Zulip tests and re-enable (#441)" flutter/tests#446

Merged

5 of 8 tasks

gnprice reviewed Jan 23, 2025

View reviewed changes

Fix cursed script.

665495f

jtmcdole approved these changes Jan 24, 2025

View reviewed changes

Merge branch 'master' into pin-customer_testing

d414c85

harri35 reviewed Jan 24, 2025

View reviewed changes

Merge remote-tracking branch 'upstream/main' into pin-customer_testing

58c9b81

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jan 29, 2025

a16de73 Pin customer_testing to the SHA specified in tests.version (

03734f6

flutter/flutter#162048)

victoreronmosele pushed a commit to victoreronmosele/tests that referenced this pull request Feb 5, 2025

Add a PR template that explains updating tests.version (flutter#445)

c5d0e14

Part of flutter/flutter#162048. This can be of course tweaked and improved.

gnprice mentioned this pull request Feb 5, 2025

Roll customer_testing from b4cc09721 to 6a6d0963c (3 commits) #162763

Merged

7 of 9 tasks

LongCatIsLooong mentioned this pull request Mar 20, 2025

Bump zulip.test to2117fe176aa740d00ff4e6be5ac38bd98eab8a3d flutter/tests#449

Merged

8 tasks

gnprice mentioned this pull request Mar 27, 2025

MenuItemButton and SubmenuButton should be used in a menu or menu bar zulip/zulip-flutter#1443

Merged

aam mentioned this pull request Apr 15, 2025

Try again flutter/tests#454

Closed

8 tasks

gnprice mentioned this pull request Apr 23, 2025

Bump Zulip pin to update sliver z-order tests flutter/tests#456

Merged

5 of 8 tasks

guspan-tanadi mentioned this pull request Apr 30, 2025

Update comments Dependabot reference link flutter/tests#457

Merged

3 of 8 tasks

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 20, 2025

a16de73 Pin customer_testing to the SHA specified in tests.version (

94570d9

flutter/flutter#162048)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 20, 2025

a16de73 Pin customer_testing to the SHA specified in tests.version (

fb30302

flutter/flutter#162048)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 21, 2025

a16de73 Pin customer_testing to the SHA specified in tests.version (

137664e

flutter/flutter#162048)

chrisbobbe mentioned this pull request Jun 4, 2025

Bump Zulip pin to update some tests that hard-code route animation duration flutter/tests#460

Merged

stereotype441 mentioned this pull request Oct 2, 2025

Bump Zulip customer tests. flutter/tests#472

Merged

4 of 8 tasks

loic-sharma mentioned this pull request Oct 21, 2025

Roll DevTools flutter/tests#473

Merged

7 of 8 tasks

FMorschel mentioned this pull request Mar 3, 2026

Update commit hash for devtools tests flutter/tests#481

Merged

6 of 8 tasks

Kendru98 mentioned this pull request Mar 25, 2026

Fix/reanimate patrol tests flutter/tests#484

Merged

8 tasks done

IvoneDjaja mentioned this pull request Aug 24, 2026

Add assets for api docs to test registry flutter/tests#492

Closed

8 tasks done

chrisbobbe mentioned this pull request Oct 8, 2026

Re-enable Zulip tests flutter/tests#507

Open

4 of 8 tasks

Navigation Menu

Repository navigation

Pin `customer_testing` to the SHA specified in `tests.version` - #162048

Pin `customer_testing` to the SHA specified in `tests.version`#162048
auto-submit[bot] merged 15 commits into
flutter:masterfrom
matanlurey:pin-customer_testing

matanlurey commented Jan 23, 2025

Uh oh!

flutter-dashboard Bot commented Jan 23, 2025

Uh oh!

zanderso left a comment

Uh oh!

Uh oh!

Piinks commented Jan 23, 2025

Uh oh!

auto-submit Bot commented Jan 23, 2025

Uh oh!

Piinks commented Jan 23, 2025

Uh oh!

gnprice Jan 23, 2025

Uh oh!

matanlurey Jan 23, 2025

Uh oh!

matanlurey commented Jan 23, 2025

Uh oh!

jtmcdole left a comment

Uh oh!

harri35 Jan 24, 2025

Uh oh!

harri35 Jan 24, 2025

Uh oh!

harri35 Jan 24, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

		@@ -0,0 +1,15 @@
		# customer_testing

		This tool checks out <https://github.com/flutter/tests> at the SHA specified in

	/// ./dev/customer_testing/ci.sh [sha]
	/// ./dev/customer_testing/ci.sh


		import 'package:path/path.dart' as p;

		/// To run this script, either:

Conversation

matanlurey commented Jan 23, 2025

Uh oh!

flutter-dashboard Bot commented Jan 23, 2025

Uh oh!

zanderso left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Piinks commented Jan 23, 2025

Uh oh!

auto-submit Bot commented Jan 23, 2025

Uh oh!

Piinks commented Jan 23, 2025

Uh oh!

gnprice Jan 23, 2025

Choose a reason for hiding this comment

Uh oh!

matanlurey Jan 23, 2025

Choose a reason for hiding this comment

Uh oh!

matanlurey commented Jan 23, 2025

Uh oh!

jtmcdole left a comment

Choose a reason for hiding this comment

Uh oh!

harri35 Jan 24, 2025

Choose a reason for hiding this comment

Uh oh!

harri35 Jan 24, 2025

Choose a reason for hiding this comment

Uh oh!

harri35 Jan 24, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants