Repository navigation
[Backport release-1.5] fix(cert-manager): move the webhook off the kubelet's port - #3366
Merged
Merged
Conversation
Author
|
Please cherry-pick the changes locally and resolve any conflicts. git fetch origin backport-3359-to-release-1.5
git worktree add --checkout .worktree/backport-3359-to-release-1.5 backport-3359-to-release-1.5
cd .worktree/backport-3359-to-release-1.5
git reset --hard HEAD^
git cherry-pick -x df85632c8909e775580a7c8732dffae8b8e473ca
git push --force-with-lease |
…injection cainjector loads all ValidatingWebhookConfigurations, APIServices and CRDs into its informer caches at startup. On a full cozystack install that working set exceeds the 128Mi limit, so the single leader-elected cainjector Pod is OOMKilled during cache population and CrashLoops before it injects any caBundle. Webhook ValidatingWebhookConfigurations then keep an empty caBundle and every admission call fails with "x509: certificate signed by unknown authority" (failurePolicy: Fail), intermittently blocking Ingress creation and stalling tenant installs. Raise the cainjector memory limit to 512Mi (matching the controller) and the request to 128Mi, and add a helm-unittest that pins the sizing so a blanket resource change or a `make update` cannot silently lower it again. Signed-off-by: Ivan Okhotnikov <[email protected]> (cherry picked from commit c3afc63) (cherry picked from commit 9f312d7a216c34724848142415d2de7c086a711e)
myasnikovdaniil
force-pushed
the
backport-3359-to-release-1.5
branch
from
July 31, 2026 05:28
1e7474e to
ea15a20
Compare
Backport of #3359 onto release-1.5. The cert-manager webhook listened on 10250, the port the kubelet also serves. When a connection to the webhook Service resolved to a node IP rather than a Pod IP it reached the kubelet, which completed the TLS handshake with its own node serving certificate, and the API server then rejected every cert-manager admission call cluster-wide: failed calling webhook "webhook.cert-manager.io": ... x509: certificate is valid for srv3, not cert-manager-webhook.cozy-cert-manager.svc The certificate is valid — it is simply the wrong server's — so the error reads as a cert-manager fault and hides the misroute that caused it. Moving the webhook to 10260 is upstream's own remedy for this signature. It does not fix the misroute; it removes the collision that turns a rare transient one into a cluster-wide outage attributed to the wrong component. The override lives in the package-root values file, which `make update` leaves alone, so it survives re-vendoring. Stacked on the #3199 backport (#3202): release-1.5 ships an EMPTY packages/system/cert-manager/values.yaml, so every resource override in that file arrived after v1.5.2. The bot's cherry-pick of this fix conflicted for that reason alone — nothing to merge into — and applies cleanly once the cainjector backport is in place. Upgrade safety is unchanged from the original: one replica, no explicit strategy, so maxUnavailable 0 / maxSurge 1 apply and the replacement Pod must be Ready before the old one goes. The Service targets a NAMED port, so during the overlap each Pod is reached on the port it actually serves and there is no window with zero ready endpoints. Closes #3355 (cherry picked from commit 3affec9) Assisted-By: Claude <[email protected]> Signed-off-by: Myasnikov Daniil <[email protected]>
myasnikovdaniil
force-pushed
the
backport-3359-to-release-1.5
branch
from
July 31, 2026 07:05
ea15a20 to
982d6d4
Compare
myasnikovdaniil
marked this pull request as ready for review
August 4, 2026 08:27
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Backport of #3359 to
release-1.5.