Repository navigation
[skills] Improve CI flake triage and targeted retries - #12926
Merged
Merged
Conversation
Add evidence-based failure classification, flaky issue matching guidance, targeted Azure stage retry recommendations, machine-readable analysis, and offline fixtures. Co-authored-by: Copilot App <[email protected]>
Remove the fixture suite and fixture-only analyzer input paths to keep the skill focused on live CI analysis. Co-authored-by: Copilot App <[email protected]>
5 tasks
Co-authored-by: Copilot App <[email protected]>
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The analyzer can conflate or misclassify test failures and inadvertently exclude valid targeted retries.
Review effort: Balanced
Findings: 3
Open (3)
What changed in this PR
Improves CI failure classification, flaky-issue tracking, and confirmation-gated Azure stage retries.
Changes:
- Adds structured failure analysis and retry plans.
- Documents classification and issue-matching rules.
- Adds targeted stage-retry guidance.
| File | Description |
|---|---|
SKILL.md |
Defines the updated CI triage workflow. |
scripts/ci_failures.cs |
Implements classification and retry planning. |
references/flaky-issues.md |
Adds flaky-issue matching guidance. |
references/error-patterns.md |
Defines failure categories and confidence rules. |
references/azdo-queries.md |
Documents targeted Azure retries. |
…idence Suppress synthetic test gates only with same-stage/attempt test evidence, retain assembly and normalized root error when collecting outcomes and fingerprints, and keep sibling-only passes out of retry recommendations. Preserve each failing stage and hydrate missing summary errors only from an exact result ID. Co-authored-by: Copilot App <[email protected]>
Member
Author
|
@dalexsoto stamp |
dalexsoto
approved these changes
Oct 6, 2026
dalexsoto
left a comment
dalexsoto
left a comment
Member
There was a problem hiding this comment.
Stamping as requested by @simonrozsival; no code review performed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
dotnet-androidbuilds as likely PR regressions, known flaky tests, transient infrastructure failures, timeout/crashes, or unknowns with evidence and confidenceforceRetryAllJobs: falseover full/azp runretriesci_failures.cs, including previous attempts, paginated test results, cross-run Auto-Retry correlation, stage-safe retry plans, and fail-closed error handlingValidation