You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
e2e: no remediation-cycle guard covers the tenant addon HelmReleases, and the existing helper cannot tell a retry from a teardown #3553
helmrelease_has_remediation_cycle in hack/e2e-chainsaw/_lib/remediation-guard.sh catches a HelmRelease that fell into an install or upgrade remediation cycle. It reads .status.history and flags a failed or uninstalled Snapshot, deliberately rather than .status.installFailures / .status.upgradeFailures, since ClearFailures() zeroes those on every successful reconcile and a guard reading them after the release is Ready is vacuous.
The helper is wired, and it is wired exactly as documented. hack/e2e-chainsaw/_lib/run-kubernetes.sh sources it at line 3 and calls it at line 1039 against the parent kubernetes-${test_name} HelmRelease, and both places the documentation mentions it scope it to that same object: docs/agents/e2e-testing.md:60 heads the section "Assert the parent HelmRelease did not remediate", and :125 asks for the guard when a change "touches parent-HR behavior". So this is not an instrument that was left unconnected or pointed the wrong way.
The gap is that nothing covers the tenant addon HelmReleases the parent installs, and no guidance asks for it. A remediation cycle on one of those leaves no mark that the suite inspects, so a run surfaces only the downstream symptom, such as a node-readiness deadline, with nothing identifying the teardown that preceded it. The CNI release is the sharpest case, because it is the one whose teardown removes a cluster-wide precondition rather than churning a single component, but the same blindness applies to every addon.
Extending the coverage needs more than another release name, because the helper's signal is ambiguous under the newer action strategy. Treating failed and uninstalled as one signal is correct for a release under the default RemediateOnFailure, where a failed attempt is followed by a teardown. It is wrong for a release under RetryOnFailure, where a failed attempt leaves the applied manifests in place and is retried as an upgrade: there a failed Snapshot is the normal non-destructive path and only uninstalled indicates a teardown. Pointed at such a release as it stands, the guard would report a remediation cycle on the first failed attempt even though nothing was removed.
The two parts are separable and the second gates the first: the helper has to distinguish a teardown from a plain failure before its coverage can reach releases that no longer remediate by uninstalling.
The second part of this issue doesn't hold, so #3594 does not split the reading. The issue assumes that under RetryOnFailure a failed Snapshot is harmless and only uninstalled means a teardown. In helm-controller v1.5.0 a release with an active retry strategy never remediates: internal/reconcile/atomic_release.go returns an upgrade for a failed release before it reaches the remediation switch. The other two uninstall paths, a changed release target and deletion, clear history afterwards. So uninstalled can't appear on such a release, and failed is the only footprint a failed action leaves.
What #3594 does instead is extend the guard to the tenant addon releases. A failed or uninstalled Snapshot fails the run on the parent, cilium and csi, which retry in place, and is printed as a note on the default-strategy addons. The same classification also runs on the failure paths that exit before the guard.
Activity
lexfrei commentedon Sep 28, 2026
The second part of this issue doesn't hold, so #3594 does not split the reading. The issue assumes that under
RetryOnFailureafailedSnapshot is harmless and onlyuninstalledmeans a teardown. In helm-controller v1.5.0 a release with an active retry strategy never remediates:internal/reconcile/atomic_release.goreturns an upgrade for a failed release before it reaches the remediation switch. The other two uninstall paths, a changed release target and deletion, clear history afterwards. Souninstalledcan't appear on such a release, andfailedis the only footprint a failed action leaves.What #3594 does instead is extend the guard to the tenant addon releases. A
failedoruninstalledSnapshot fails the run on the parent,ciliumandcsi, which retry in place, and is printed as a note on the default-strategy addons. The same classification also runs on the failure paths that exit before the guard.fix(e2e): fail on a failed release of cilium and csi as on the parent…