Skip to content

[flaky-ci] Retry transient ADB install failures - #12711

Merged
simonrozsival merged 4 commits into
mainfrom
simonrozsival-adb-install-retry
Sep 10, 2026
Merged

simonrozsival merged 4 commits into
mainfrom
simonrozsival-adb-install-retry

Conversation

@simonrozsival

@simonrozsival simonrozsival commented Sep 8, 2026 •

Copy link
Copy Markdown
Member

Why

Build 1576002 failed while installing the Debug test APK after adb install reached Android package-manager streamed install:

Performing Streamed Install
adb: failed to install ...:
cmd: Failure calling service package: Broken pipe (32)

This was not a host transport-setup failure, a device disconnect, or an app-specific INSTALL_FAILED_* result. The Release and NoAab configurations passed immediately afterward on the same emulator.

#11831 added Azure task retries around the separate install step tracked by #11830 and #11833. #12267 later moved install inside dotnet test, removing that retry boundary. #11966 handles a related package-service failure in device-test setup by restarting the emulator, but product deployment must not restart a user device.

What changed

  • Classify only package-service Broken pipe output as transient, after semantic install-result classification.
  • Add shared AdbRunner.WaitForPackageManagerAsync() readiness polling for the selected device; it builds on the existing WaitForDeviceAsync() transport wait.
  • Wait at most 10 seconds, then retry the identical install once without restarting ADB, the device, or the emulator.
  • Preserve the original install output, readiness diagnostics, and retry output if recovery fails.
  • Keep the existing uninstall/reinstall behavior and prevent more than one transient retry across it.

Validation

  • 119 AdbRunnerTests passed.

  • 19 FastDeploy2-focused host tests passed.

  • Xamarin.Android.Build.Debugging.Tasks.csproj builds successfully.

  • Useful description of why the change is necessary

  • Links to issues fixed

  • Unit tests

Fixes #12704

Build 1576002 reached Android package-manager streamed install but lost the package service pipe before receiving an install result.  The prior pipeline task retry no longer covers this path because install now runs inside dotnet test.

Classify only device-offline and package-service broken-pipe output as transient.  Wait up to 10 seconds for the selected device and package manager through shared AdbRunner infrastructure, then retry the identical install once while preserving all attempt and recovery diagnostics.

Semantic INSTALL_FAILED results continue to fail immediately without a transport retry.

Fixes: #12704

Co-authored-by: Copilot App <[email protected]>
@simonrozsival simonrozsival added the flaky-ci Intermittent CI failures and work to improve CI reliability label Sep 8, 2026
@simonrozsival

Copy link
Copy Markdown
Member Author

/review

@github-actions

github-actions Bot commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

✅ Android PR Reviewer completed successfully!

Generated by Android PR Reviewer for #12711

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Needs Changes

Code findings: 0 errors · 0 warnings · 1 suggestion

The retry is narrowly classified, preserves diagnostics, respects cancellation, and shares retry state across the uninstall path. I left one inline suggestion to cover that cross-path state with a regression test.

CI build #1586927 is currently red in MSBuild Emulator Tests macOS > Tests > MSBuild+Emulator 8; all other reported lanes passed. The Azure timeline was unavailable from this runner, so the failure could not be classified as related or flaky. Please resolve or re-run that lane before merging.

Generated by Android PR Reviewer for #12711 · gpt56 · 279.4 AIC · ⌖ 8.85 AIC · ⊞ 27.7K
Comment /review to run again

The transport retry state is shared with the existing uninstall/reinstall path, but the original tests exercised those paths independently.

Cover both orderings.  An already-installed result can still recover from one later transient failure, while a transient failure before uninstall prevents a second transport retry afterward and preserves all three install diagnostics.

Co-authored-by: Copilot App <[email protected]>
@simonrozsival

Copy link
Copy Markdown
Member Author

/review

@github-actions

github-actions Bot commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

✅ Android PR Reviewer completed successfully!

Generated by Android PR Reviewer for #12711

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Needs Changes

Findings: 0 errors · 1 warning · 0 suggestions

The retry state is correctly shared across the install/uninstall path, semantic install failures remain higher priority than transient-text detection, and the accumulated diagnostics preserve each attempt. The package-manager polling also preserves caller cancellation and bounds the combined device/readiness wait.

One new timeout test uses a load-sensitive 500 ms process budget; please make that test deterministic as noted inline.

CI: currently red: the Linux build and macOS build checks failed, while the Windows build was still running; the focused Android Tools checks passed. The Azure timeline details were unavailable from this runner, so I could not determine whether those two build failures are caused by this patch.

Generated by Android PR Reviewer for #12711 · gpt56 · 199.5 AIC · ⌖ 13.7 AIC · ⊞ 25.7K
Comment /review to run again

Comment thread tests/Xamarin.Android.Tools.AndroidSdk-Tests/AdbRunnerTests.cs Outdated
@simonrozsival
simonrozsival marked this pull request as ready for review September 9, 2026 15:05
Copilot AI lite review requested due to automatic review settings September 9, 2026 15:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The new package-manager polling loop logs at Warning level on every poll iteration, which can create excessively noisy warning output during normal transient recovery.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Lite
Findings: 1 Medium severity

New issues introduced by this change (1)
Severity Finding
Medium severity src/​Xamarin.Android.Tools.AndroidSdk/​Runners/​AdbRunner.cs — ⚠️ Logging this poll-loop message at TraceLevel.Warning can generate a large volume of warnings (up…
What changed in this PR

Improves resiliency of .NET for Android Debug/FastDeploy2 APK installation by retrying a narrowly-scoped set of transient adb install failures (device transport/package-service “Broken pipe”), including a short readiness poll for the package manager before the retry.

Changes:

  • Added AdbRunner.WaitForPackageManagerAsync() to poll pm responsiveness after wait-for-device.
  • Updated FastDeploy2 install flow to retry exactly once on classified transient install failures, preserving diagnostic output across attempts.
  • Added unit tests covering transient-failure classification, retry boundaries (including across uninstall), and timeout/cancellation behavior.
File Description
tests/​Xamarin.Android.Tools.AndroidSdk-Tests/​AdbRunnerTests.cs Adds tests for the new package-manager readiness wait (retry/timeout/cancel).
src/​Xamarin.Android.Tools.AndroidSdk/​Runners/​AdbRunner.cs Introduces WaitForPackageManagerAsync() and helper command execution/diagnostics.
src/​Xamarin.Android.Tools.AndroidSdk/​PublicAPI/​netstandard2.0/​PublicAPI.Unshipped.txt Records new public API surface for netstandard2.0.
src/​Xamarin.Android.Tools.AndroidSdk/​PublicAPI/​net10.0/​PublicAPI.Unshipped.txt Records new public API surface for net10.0.
src/​Xamarin.Android.Tools.AndroidSdk/​PublicAPI/​net11.0/​PublicAPI.Unshipped.txt Records new public API surface for net11.0.
src/​Xamarin.Android.Build.Tasks/​Tests/​Xamarin.Android.Build.Tests/​DebuggingTasksTests.cs Adds regression tests for FastDeploy2 transient classification and retry behavior.
src/​Xamarin.Android.Build.Debugging.Tasks/​Tasks/​FastDeploy2.cs Makes AdbCommandResult internal to enable test harnessing.
src/​Xamarin.Android.Build.Debugging.Tasks/​Tasks/​FastDeploy2.Adb.cs Implements single transport/package-manager recovery retry with preserved attempt diagnostics.

Comment thread src/Xamarin.Android.Tools.AndroidSdk/Runners/AdbRunner.cs
Comment thread src/Xamarin.Android.Build.Debugging.Tasks/Tasks/FastDeploy2.Adb.cs Outdated
Comment thread src/Xamarin.Android.Tools.AndroidSdk/Runners/AdbRunner.cs
Comment thread src/Xamarin.Android.Build.Debugging.Tasks/Tasks/FastDeploy2.Adb.cs
Retry only the package-service broken-pipe failure observed in build 1576002.  A device-offline result is no longer considered recoverable because waiting for package-manager readiness cannot make a disconnected device return.

Log intermediate readiness probes at verbose level and give the timeout diagnostic test a load-tolerant process budget.

Co-authored-by: Copilot App <[email protected]>
@simonrozsival
simonrozsival merged commit b03157a into main Sep 10, 2026
44 checks passed
@simonrozsival
simonrozsival deleted the simonrozsival-adb-install-retry branch September 10, 2026 20:10
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 11, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

flaky-ci Intermittent CI failures and work to improve CI reliability

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI flakiness inventory: Aug 11-Sep 8, 2026

3 participants