Skip to content

tenant Gateway: a failed route-status write can freeze the shared Gateway for every tenant on it #4195

Closed

Description

@kvaps

Split out of the review of #3342, which was approved under release pressure with the remaining items carried as follow-ups. This is the one I would do first, because it is a regression that PR introduces rather than a pre-existing gap.

updateRouteStatuses and reconcileWholeApexRouteStatuses now collect their route-status write errors and return them (internal/controller/tenantgateway/conflict.go:718, :730, :733), and runReconcileSteps returns on both (internal/controller/tenantgateway/reconciler.go:667-672). On the merge base these were logged and the pass carried on. Failing rather than silently swallowing a failed write is the right instinct and is not what this issue asks to undo.

The gap is that a retryable conflict and a write that can never succeed are treated the same. RouteStatus.Parents carries MaxItems=32 and parentRefs likewise (gateway-api v1.4.1, apis/v1/shared_types.go:597 and :245), and both this controller and Cilium write one entry per parentRef. A single route with seventeen or more parentRefs to the shared Gateway therefore produces a status object the apiserver rejects as Invalid on every pass, forever.

What is lost while that holds: reconcileHTTPToHTTPSRedirect (reconciler.go:672) and reconcileStatus (:677) never run again, so the TenantGateway is pinned at Ready=False and, on a fresh install, the http-to-https redirect route is never created at all — which is the object that guard exists to provide. The Gateway is shared across the inheriting tenant subtree and the attached cozy-* namespaces, so the route that triggers this need not belong to whoever notices.

Suggested shape, all small: treat IsConflict and IsNotFound on a route status as skip-and-requeue rather than fatal; move the status pass after reconcileHTTPToHTTPSRedirect so a status problem cannot withhold the redirect; and batch a route's parentRefs into one read-modify-write, since updateRouteParentStatus (conflict.go:915-942) currently re-reads a cached object once per attaching ref, which is what makes conflicts routine in the first place.

Unit tests cannot see any of this today — the fake client has no cache lag and no second writer.

Activity

  1. kvaps commented on Sep 10, 2026

    @kvaps
    MemberAuthor

    Fixed in #3342 at 8af986398. retryableRouteWrite classifies IsConflict and IsNotFound as survivable and everything else as fatal, recursing into a joined error so one fatal leaf fails the whole batch rather than being retried into silence; routeStatusWriteError tags the failure so Reconcile can tell it from a desired-state failure, and a retryable one requeues instead of pinning Ready=False. reconcileHTTPToHTTPSRedirect now runs ahead of the route-status pass and reconcileStatus before the collected errors are returned, so a status the apiserver refuses can no longer withhold the redirect or freeze the TenantGateway's own status.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    triage/needs-triageIndicates an issue needs triage by a maintainer

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions