Repository navigation
ci(tests): remove redundant Release E2E workflow - #2978
Conversation
The Release E2E workflow (`release-e2e.yaml`) ran the full bats suite on every release tag as a non-blocking alarm — its own header notes "a failure here is an alarm, not a release blocker." It duplicates coverage the release-cut PR already provides: the "Prepare release" commit bakes image digests into `packages/core/` (and other packages), which TIA escalates to the full suite, so the release PR exercises the whole suite against the commit the tag will point at. It has also never been green since it was added (2026-05-02): every tag run failed at the build step. Even after #2973 fixed the registry login today, the next run still failed (Build Talos image). Removing a perpetually-red, non-gating duplicate removes noise without losing coverage. Removes the workflow, drops `release-e2e` from `select-e2e.sh`'s full-suite pattern, repoints its unit test to `pull-requests.yaml`, and updates the e2e-testing doc. Refs #2397 Assisted-By: Claude <[email protected]> Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Signed-off-by: Myasnikov Daniil <[email protected]>
github-actions
Bot
added
area/testing
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request removes the 'Release E2E' workflow, which served as a non-blocking, redundant alarm that frequently failed. The full E2E suite coverage is already adequately provided by the release-cut PR process, making this workflow unnecessary. The changes include cleaning up the CI configuration, updating test coverage, and refining documentation to reflect the current testing strategy. Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Ignored Files
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on Gemini (@gemini-code-assist) comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
dosubot
Bot
added
area/ci
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (4)
💤 Files with no reviewable changes (1)
📝 WalkthroughWalkthroughThe ChangesRemove release-e2e workflow and update trigger references
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~3 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request removes references to the deprecated release-e2e.yaml workflow from the E2E testing documentation, the select-e2e.sh script, and its corresponding BATS tests. The coverage gap is now closed at release-cut time when image digests are baked into the packages, which naturally triggers the full test suite. No review comments were provided, and there is no additional feedback to provide.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Aleksei Sviridkin (lexfrei)
left a comment
Aleksei Sviridkin (lexfrei)
left a comment
There was a problem hiding this comment.
LGTM — the removed Release E2E workflow is genuinely redundant: the full E2E suite already runs on the release-cut PR, and the deleted workflow was non-blocking and had never been green, so it provided no real signal. Removing it drops noise without losing coverage.
Business context: deletes a tag-triggered full-suite E2E workflow whose coverage is already delivered by the release-cut PR.
Coverage equivalence (the central question) — verified end to end:
- The
Prepare release vX.Y.Zcommit pins image digests intopackages/core/{installer,platform,testing}/values.yaml, and the release-cut PR carries thereleaselabel.- Evidence:
docs/release.md:193-195(digest pins land inpackages/core/...);.github/workflows/tags.yaml:138(thePrepare releasecommit);.github/workflows/tags.yaml:239(labels: ['release']).
- Evidence:
- On that PR the selector runs (it is skipped only under the
full-e2elabel), and anypackages/core/path matches the full-suite pattern, so the whole bats suite runs against the commit the tag will point at.- Evidence:
hack/select-e2e.sh:22(full-suite pattern matchespackages/core/);.github/workflows/pull-requests.yaml:298-300(selector gated only byfull-e2e; E2E runs onreleasePRs via the draft-release asset path). Reproduced against the realPrepare release v1.5.1commit: the selector escalates to the full suite (33-39 apps).
- Evidence:
No dangling references: a whole-tree search finds zero remaining mentions of the deleted workflow; nothing references it via workflow_run or a uses: pointer.
- Evidence:
grep -rni 'release-e2e' .on the PR head returns nothing.
Tests: select-e2e_test.bats is repointed to pull-requests.yaml, preserving "workflow edit -> full suite" coverage; all 11 cases pass locally.
- Evidence:
hack/select-e2e_test.bats:114-118; local run reports1..11 ... ok 11.
Blockers: none.
Non-blocking:
- Timing nuance: the deleted workflow ran the full suite against the exact published tag commit (after the force-move), whereas the release-cut PR runs it against the release-branch head before merge. If
maindiverges during the release-PR window, the final force-moved tag (merge) commit is not itself independently E2E'd. Net signal still improves, since the deleted workflow was non-blocking and perpetually red. Evidence:docs/release.md:540-542(the tag is force-moved onto the merge commit bypull-requests-release.yaml); deletedrelease-e2e.yamlheader — "a failure here is an alarm, not a release blocker." - The deleted workflow triggered only on
push: tags, so it could not have been a PR-merge required status check; I cannot read branch protection, but the trigger type rules out a PR gate. Evidence: deleted.github/workflows/release-e2e.yamlon: push: tags.
IvanHunters
left a comment
IvanHunters
left a comment
There was a problem hiding this comment.
Verdict
LGTM.
Removing a perpetually-red, non-gating workflow whose coverage is provably already delivered by the release-cut PR via TIA escalation. Wiring is consistent, tests pass, docs match the new reality, and no stale references remain.
Red-team concerns
- Release PR vs tag-commit SHA mismatch (pre-existing, not a regression).
tags.yaml:138adds a "Prepare release" commit aftermake buildruns against the tagged commit. The published images/charts are built from the tag commit, but the release PR's E2E runs against tag commit + prepare-release commit. So ifpackages/core/only changes in the prepare-release commit (typical case), the full suite is exercised against a tree that is one commit ahead of what the tag actually points at. In practice this is fine — the only delta is digest rewrites invalues.yamlpointing at images that were just pushed — but it is worth being explicit about in the doc that "the full suite runs against the release PR head", not "against the tag commit". The oldrelease-e2e.yamldid run on the tag itself (alarm semantics). The new doc text (docs/agents/e2e-testing.md:62) says "the commit the tag will point at" — technically true only if no merge-cleanup commits land on the release branch between tag and release-PR merge. Minor wording polish, not blocking. prepare-releasecommit empty path.tags.yaml:138ends with|| echo "No changes to commit". If a retag (or a re-run of the publish workflow against an already-published tag) produces nogit add .delta, no "Prepare release" commit lands, the release PR's diff against base would not includepackages/core/digest churn, and TIA would not escalate. Low-probability (retag scenario), not introduced by this PR, but the safety-net behaviourrelease-e2e.yamlpreviously provided on the tag itself is gone — there is now no independent verification that the tagged commit's artifacts build and install end-to-end. Acceptable given the workflow never produced a green signal anyway.
Caveats
- Phase 5b (existing-customer upgrade impact / fresh-install impact) does not apply: this PR touches only
.github/workflows/,hack/, anddocs/. No runtime impact on running clusters and no install-path changes. - shellcheck reports 3 issues in
hack/select-e2e.sh(SC2011 at line 25, SC2016 at lines 42 and 47) butgit blameconfirms all three predate this PR — out of scope here. - Verified locally:
bats hack/select-e2e_test.bats→ 11/11 passing, including the renamedpull-requests workflow change triggers full suite. - Verified locally: simulating a release-cut diff (
packages/core/{platform,installer,testing,talos}/values.yaml) throughhack/select-e2e.shselects all 18 apps (full suite). - Verified locally:
grep -r release-e2eover the working tree finds zero references outside.git/. No dangling docs, badges,needs:,workflow_run, orworkflow_callreferences.retest.yamlalready targetspull-requests.yaml.tags.yamldoes not depend onrelease-e2e.
Recommended follow-ups
- Consider tightening the new doc wording in
docs/agents/e2e-testing.md:62to say "against the release-cut PR head (tag commit + prepare-release commit)" so future readers do not assume the suite runs on the exact tagged SHA. - Pre-existing shellcheck findings on
hack/select-e2e.sh(SC2011, SC2016) can be cleaned up in a follow-up; not a blocker here.
## What this PR does Cleans up four GitHub Actions workflows that are redundant, broken, or noisier than they need to be. Each change is its own commit so they can be reviewed (or reverted) independently. - **`retest.yaml` — removed.** The `/retest` command re-ran a job named `Prepare environment`, but the TIA / job-split refactor of `pull-requests.yaml` collapsed that into a *step* inside the `E2E Tests` job, so the job lookup can never match. Every real `/retest` invocation since fails with `Could not find "Prepare environment" job` — **10 failed runs spanning 2026-01-08 → 2026-06-24, none that ever completed a retest**. (The 41 runs GitHub records as `success` are `issue_comment` events where the job-level `if`-guard was false, so no steps ran.) `gh run rerun --failed` covers the need. - **`codeql.yml` — switch Go to manual build; keep the per-PR run.** The Go `autobuild` ran `make`, whose included `hack/common-envs.mk` does a `$(shell git fetch upstream --tags)` at parse time — a network call to a non-origin remote that has nothing to do with the analysis and is a flake vector. Replace it with `build-mode: manual` that compiles each first-party module explicitly (the root module, `api/apps/v1alpha1`, and the three nested image modules; the gitignored `_repos/*` upstream clones drop out for free on a clean checkout). Same coverage, no upstream fetch. The `pull_request` trigger is **kept on purpose**: per-PR analysis is the only mode that diffs *new* alerts against the base branch and annotates them on the changed lines — the basis for promoting CodeQL to a required pre-merge check (follow-up: add `CodeQL / Analyze (go)` to branch protection and set the code-scanning failure threshold to errors-only). No `paths` filter — a required check that is path-skipped never reports and would block PRs that touch no Go. - **`scorecard.yml` — badge-only; stop polluting code-scanning.** Scorecard uploads one code-scanning alert per check, which floods Security → Code scanning and buries CodeQL's first-party findings. Drop the `upload-sarif` step and the now-unneeded `security-events: write`; keep `publish_results` (the public scorecard.dev badge is the canonical view). Also drop the redundant `push:[main]` trigger — the weekly cron + `branch_protection_rule` already cover it. - **`update-releasenotes.yaml` — path-gate to changelog changes + manual hatch.** The release body is synced from `docs/changelogs/v*.md`, which only lands on main when the changelog PR merges or a maintainer edits an existing changelog. Gate the `push` trigger to `docs/changelogs/v*.md` (the `v*` glob skips `template.md` / `patch-template.md`) so the self-hosted job stops firing on every unrelated main push, and add `workflow_dispatch` as a manual re-sync hatch. - **`SECURITY.md` — document the security scanners.** Add an "Automated security analysis" subsection describing the CodeQL (SAST) and OpenSSF Scorecard (supply-chain) setup, including the manual Go build, the new-alert-on-diff behaviour, the intent to make CodeQL a required pre-merge check, and how findings are dismissed — so the posture is documented, not just encoded in YAML. Not touched: `release-e2e.yaml` — already removed by #2978. ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Updated automation workflows for security checks and release-note generation. * Removed an unused retry workflow and tightened when some checks run. * **Documentation** * Expanded security guidance to better explain automated analysis, alert handling, and reporting. * Clarified when release notes updates are generated. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
3 participants
What this PR does
Removes the
Release E2Eworkflow (.github/workflows/release-e2e.yaml), which ran the full bats suite on every release tag.Why it's redundant.
release-e2e.yamlwas a non-blocking alarm — its own header states "a failure here is an alarm, not a release blocker." The coverage it provided is already delivered by the release-cut PR: the "Prepare release" commit bakes image digests intopackages/core/(installer, platform, testing) and other packages, andselect-e2e.sh's full-suite pattern escalates anypackages/core/change to the full suite. So the release PR already runs the whole suite against the commit the tag will point at.Why now. The workflow has never been green since it was added (2026-05-02) — every tag run (
v1.5.0-rc.2,-rc.3,v1.5.0) failed at the build step. Even after #2973 fixed the registry login earlier today, the next run still failed at Build Talos image. A perpetually-red, non-gating duplicate is pure noise.Changes
.github/workflows/release-e2e.yaml.release-e2efrom the full-suite pattern inhack/select-e2e.sh.hack/select-e2e_test.batstopull-requests.yaml(the remaining E2E workflow), preserving "workflow edit → full suite" coverage. All 11select-e2e_test.batscases pass.docs/agents/e2e-testing.mdto describe the release-cut PR as where the full suite runs.Refs #2397.
Release note
Summary by CodeRabbit
Chores
Documentation