Skip to content

[flaky-ci] Recover corrupt cached Android SDK archives - #12886

Merged
simonrozsival merged 1 commit into
mainfrom
simonrozsival-investigate-sdk-unzip-failure
Sep 23, 2026
Merged

simonrozsival merged 1 commit into
mainfrom
simonrozsival-investigate-sdk-unzip-failure

Conversation

@simonrozsival

Copy link
Copy Markdown
Member

Why

Build 1607097 repeatedly failed extracting docs-24_r01.zip because the cached archive had a bad CRC. Its cached SHA-256 validation stamp let the download target skip rechecking the archive, so each retry extracted the same damaged file.

What changed

  • If SDK extraction fails, hash the cached ZIP against its pinned SHA-256. Remove the ZIP and validation stamp only on a mismatch, then fail that attempt so the existing CI task retry downloads it again. Preserve the ZIP for failures unrelated to corruption, such as disk-space or permission errors.
  • Move to a new Azure Pipelines cache namespace to replace the immutable poisoned cache, with a v1 restore fallback so unaffected downloads can be reused and promoted to v2 after a successful run.

Validation

  • Exercised a deliberately CRC-corrupted ZIP alongside a valid but stale SHA-256 stamp in a local MSBuild fixture: the first attempt removed both files and failed; the second downloaded and extracted the valid ZIP.
  • Confirmed a matching SHA-256 with an extraction error fails without deleting the archive, and validated the targets XML and cache YAML.

No issue filed; this addresses the linked CI failure.

A cached Android SDK ZIP can fail CRC validation after its SHA-256 stamp
has been saved.  This left every retry extracting the same damaged file.

On extraction failure, compare the archive with its pinned SHA-256.
Discard a mismatched ZIP and its stamp so the existing CI task retry
fetches a clean copy.  Leave archives intact for non-corruption errors.

Start a new cache namespace to replace the poisoned immutable cache,
while retaining a v1 restore fallback for unaffected downloads.

Co-authored-by: Copilot App <[email protected]>
Copilot AI lite review requested due to automatic review settings September 23, 2026 05:51
@simonrozsival simonrozsival added the flaky-ci Intermittent CI failures and work to improve CI reliability label Sep 23, 2026
@simonrozsival simonrozsival changed the title [ci] Recover corrupt cached Android SDK archives [flaky-ci] Recover corrupt cached Android SDK archives Sep 23, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved moderate issues affect corruption recovery coverage and retry behavior.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 Low severity

Open (1)
What changed in this PR

This PR improves recovery from corrupt cached Android SDK archives and rotates the Azure Pipelines cache namespace.

Changes:

  • Validates and removes corrupt archives and stamps after failed extraction.
  • Moves caching to v2 with v1 fallback restoration.
  • Updates cache-safety documentation.
File Summary Final comments
src/​androidsdk/​androidsdk.targets Adds archive hash validation and cleanup after extraction failures. Moderate: Cleanup is skipped when extraction succeeds but final source.properties validation fails.
build-tools/​automation/​yaml-templates/​cache-android-archives.yaml Updates cache keys and recovery documentation. Moderate: Recovery does not cover other cached consumers. Nit: Clarify archive verification and sidecar usage.

Comment thread build-tools/automation/yaml-templates/cache-android-archives.yaml
@simonrozsival

Copy link
Copy Markdown
Member Author

/review

@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

✅ Android PR Reviewer completed successfully!

Generated by Android PR Reviewer for #12886

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM

Findings: 0 errors · 0 warnings · 1 suggestion

The extraction failure is captured without masking the original tool output, checksum mismatches remove both the poisoned archive and its validation stamp, and matching archives are preserved for unrelated failures. The cache namespace transition retains reuse while allowing corrected contents to be promoted. CI build 1608350 and all 45 reported checks completed successfully.

The remaining suggestion is to preserve the manually validated corruption scenarios as automated regression coverage.

Generated by Android PR Reviewer for #12886 · copilot · gpt56 · 65.8 AIC · ⌖ 11.1 AIC · ⊞ 25.9K
Comment /review to run again

Comment thread src/androidsdk/androidsdk.targets
@simonrozsival simonrozsival added the ready-to-review This PR is ready to review/merge, I think any CI failures are just flaky (ignorable). label Sep 23, 2026
@simonrozsival
simonrozsival enabled auto-merge (squash) September 23, 2026 12:49
@simonrozsival
simonrozsival merged commit e936eb4 into main Sep 23, 2026
45 checks passed
@simonrozsival
simonrozsival deleted the simonrozsival-investigate-sdk-unzip-failure branch September 23, 2026 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

flaky-ci Intermittent CI failures and work to improve CI reliability ready-to-review This PR is ready to review/merge, I think any CI failures are just flaky (ignorable).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants