Skip to content

fix(gateway): pin identical allowedRoutes.kinds on port 443 listeners (cilium#45559) - #3084

Merged
Aleksei Sviridkin (lexfrei) merged 2 commits into
cozystack:mainfrom
shreyaabaranwal:fix/gateway-cilium-45559-divergent-kinds
Jun 26, 2026
Merged

Aleksei Sviridkin (lexfrei) merged 2 commits into
cozystack:mainfrom
shreyaabaranwal:fix/gateway-cilium-45559-divergent-kinds

Conversation

@shreyaabaranwal

@shreyaabaranwal Shreya Baranwal (shreyaabaranwal) commented Jun 26, 2026 •

Copy link
Copy Markdown
Contributor

What this PR does

Cilium collapses all listeners that share a port when their allowedRoutes.kinds sets differ (cilium#45559). In the default TLSPassthroughServices configuration both HTTPS-terminate listeners (kinds: [HTTPRoute]) and TLS-passthrough listeners (kinds: [TLSRoute]) land on port 443. The divergent kinds trigger the collapse, which then drops the HTTPRoutes that were accepted by the HTTPS listener.

Fix (Option B — security intact): give every port-443 listener the same kinds set [HTTPRoute, TLSRoute].

  • Introduces a shared port443Kinds slice in renderGateway and assigns it to both httpsAllowedRoutes.Kinds and passthroughAllowed.Kinds.
  • GRPCRoute / TCPRoute / UDPRoute are still excluded, so the Layer 7 VAP (cozystack-route-hostname-policy) continues to gate exactly the two expected route types. Gateway API itself rejects any HTTPRoute that references a Passthrough sectionName at attach time, so the expanded kinds on passthrough listeners add no actual attach surface.
  • Adds regression test TestReconcile_Port443ListenersShareKinds that reconciles a TenantGateway with both an attached HTTPRoute and a TLS-passthrough service, then asserts every port-443 listener carries a non-empty, identical {HTTPRoute, TLSRoute} kinds set.
  • Updates two pre-existing tests (TestReconcile_TLSPassthroughListenersRendered, TestReconcile_HTTPSListenersRestrictRouteKindsToHTTPRoute) that pinned the old single-kind behavior.

Release note

fix(gateway): all port-443 listeners now carry identical allowedRoutes.kinds=[HTTPRoute,TLSRoute] to prevent Cilium (cilium#45559) from collapsing HTTPS-terminate and TLS-passthrough listeners and silently dropping HTTPRoutes

Summary by CodeRabbit

  • Bug Fixes
    • Updated port 443 gateway listeners to use a consistent set of allowed route types.
    • HTTPS (TLS-terminate) and TLS-passthrough listeners now both allow HTTPRoute and TLSRoute, avoiding configuration discrepancies.
  • Tests
    • Added regression coverage to verify all port 443 listeners share the same allowed route-kind set and exclude unsupported route types.

Closes #3070

@github-actions github-actions Bot added size/L This PR changes 100-499 lines, ignoring generated files area/networking Issues or PRs related to networking (ingress, gateway, vpn, metallb, kube-ovn) kind/bug Categorizes issue or PR as related to a bug labels Jun 26, 2026
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a listener merging issue in Cilium where divergent allowedRoutes.kinds on port 443 caused the unintended dropping of HTTPRoutes. By standardizing the allowed kinds across both HTTPS-terminate and TLS-passthrough listeners to include both HTTPRoute and TLSRoute, the fix ensures listener stability while maintaining the existing security posture, as Gateway API naturally prevents invalid route attachments.

Highlights

  • Listener Configuration Fix: Updated the TenantGateway reconciler to ensure all port-443 listeners (HTTPS-terminate and TLS-passthrough) share an identical allowedRoutes.kinds set containing both HTTPRoute and TLSRoute.
  • Cilium Compatibility: Prevented Cilium from collapsing port-443 listeners, which previously caused HTTPRoutes to be dropped when divergent kinds were detected on the same port.
  • Regression Testing: Added a new regression test TestReconcile_Port443ListenersShareKinds and updated existing tests to verify that port-443 listeners maintain consistent kind configurations.
New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment Gemini (@gemini-code-assist) Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on Gemini (@gemini-code-assist) comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

@coderabbitai

coderabbitai Bot commented Jun 26, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 27de344e-f739-408b-a53e-f15ba5c1f06a

📥 Commits

Reviewing files that changed from the base of the PR and between 786c098 and 6acf34a.

📒 Files selected for processing (2)
  • internal/controller/tenantgateway/reconciler.go
  • internal/controller/tenantgateway/reconciler_test.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • internal/controller/tenantgateway/reconciler.go
  • internal/controller/tenantgateway/reconciler_test.go

📝 Walkthrough

Walkthrough

TenantGateway now assigns the same AllowedRoutes.Kinds set, containing HTTPRoute and TLSRoute, to port-443 HTTPS-terminate and TLS-passthrough listeners. Tests were updated and expanded to verify the shared kinds set.

Changes

TenantGateway port-443 route kinds

Layer / File(s) Summary
Shared port 443 kinds
internal/controller/tenantgateway/reconciler.go
renderGateway builds a shared port-443 kinds slice and applies it to HTTPS-terminate and TLS-passthrough listeners.
Updated listener assertions
internal/controller/tenantgateway/reconciler_test.go
Existing tests assert both route kinds on port 443 and compare the expected API group values.
Shared kinds regression
internal/controller/tenantgateway/reconciler_test.go
A new test reconciles both listener types on port 443 and checks that their AllowedRoutes.Kinds match.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related issues

Poem

🐰 I hopped by the gateway gate tonight,
With HTTPRoute and TLSRoute all bright.
Two kinds now share the same fluffy set,
So every port-443 listener behaves just right.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: making port 443 listeners share identical allowedRoutes.kinds.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request addresses an issue (cilium#45559) where divergent allowedRoutes.kinds on the same port causes Cilium to collapse port-443 listeners and drop HTTPRoutes. To resolve this, both HTTPS-terminate and TLS-passthrough listeners on port 443 now share the same kinds set containing both HTTPRoute and TLSRoute. The corresponding unit tests have been updated, and a new regression test TestReconcile_Port443ListenersShareKinds has been added to ensure port-443 listeners share the same kinds. There are no review comments to assess, and the changes look correct and well-tested.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
internal/controller/tenantgateway/reconciler.go (1)

870-883: 🩺 Stability & Availability | 🟡 Minor

TLS-passthrough listeners must advertise TLSRoute, not HTTPRoute.

The current configuration assigns port443Kinds (HTTPRoute) to a listener with Protocol: TLS and Mode: Passthrough. Per the Gateway API specification, HTTPRoute is incompatible with TLS-passthrough listeners. Cilium enforces this contract and will set the listener status condition to InvalidRouteKinds, preventing route attachment and marking the listener as not ready. The code comment claiming this behavior is "harmless" is incorrect.

If the intent is TLS passthrough, update AllowedRoutes.Kinds to TLSRoute. If the intent is HTTP/HTTPS routing with termination, change the Protocol to HTTPS and retain HTTPRoute.

Relevant Diff
- passthroughAllowed.Kinds = port443Kinds // HTTPRoute
- listeners = append(listeners, gatewayv1.Listener{
-     Name:     gatewayv1.SectionName("tls-" + svc),
-     Port:     443,
-     Protocol: gatewayv1.TLSProtocolType,
-     Hostname: &host,
-     TLS: &gatewayv1.ListenerTLSConfig{
-         Mode: ptrTLSMode(gatewayv1.TLSModePassthrough),
-     },
-     AllowedRoutes: &passthroughAllowed,
- })
+ passthroughAllowed.Kinds = []gatewayv1.RouteGroupKind{{Group: &v1.Group("gateway.networking.k8s.io"), Kind: gatewayv1.Kind("TLSRoute")}}
+ listeners = append(listeners, gatewayv1.Listener{
+     Name:     gatewayv1.SectionName("tls-" + svc),
+     Port:     443,
+     Protocol: gatewayv1.TLSProtocolType,
+     Hostname: &host,
+     TLS: &gatewayv1.ListenerTLSConfig{
+         Mode: ptrTLSMode(gatewayv1.TLSModePassthrough),
+     },
+     AllowedRoutes: &passthroughAllowed,
+ })
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/controller/tenantgateway/reconciler.go` around lines 870 - 883, The
TLS-passthrough listener setup in the tenant gateway reconciler is advertising
the wrong route kind: the listener created for `TLSPassthroughServices` uses
`Protocol: TLS` and `TLSModePassthrough` but still assigns `port443Kinds` to
`AllowedRoutes.Kinds`. Update this block so the `Listener` built in the
`TLSPassthroughServices` loop advertises `TLSRoute` when passthrough is
intended, or switch the listener to `HTTPS` with `HTTPRoute` only if you mean
terminated TLS routing. Keep the change aligned with the existing
`allowedRoutes`, `port443Kinds`, and `gatewayv1.Listener` construction.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@internal/controller/tenantgateway/reconciler.go`:
- Around line 870-883: The TLS-passthrough listener setup in the tenant gateway
reconciler is advertising the wrong route kind: the listener created for
`TLSPassthroughServices` uses `Protocol: TLS` and `TLSModePassthrough` but still
assigns `port443Kinds` to `AllowedRoutes.Kinds`. Update this block so the
`Listener` built in the `TLSPassthroughServices` loop advertises `TLSRoute` when
passthrough is intended, or switch the listener to `HTTPS` with `HTTPRoute` only
if you mean terminated TLS routing. Keep the change aligned with the existing
`allowedRoutes`, `port443Kinds`, and `gatewayv1.Listener` construction.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 563cef8d-9314-425c-a4e0-17b970805a78

📥 Commits

Reviewing files that changed from the base of the PR and between 7ff0bf8 and 786c098.

📒 Files selected for processing (2)
  • internal/controller/tenantgateway/reconciler.go
  • internal/controller/tenantgateway/reconciler_test.go

@shreyaabaranwal

Shreya Baranwal (shreyaabaranwal) commented Jun 26, 2026 •

Copy link
Copy Markdown
Contributor Author

This should fix #3070.

A quick note on the approach: the issue suggested just dropping the kinds from the HTTPS listeners, but I went a slightly different route , keeping both HTTPRoute and TLSRoute on every port-443 listener instead. That way the divergence (and the cilium#45559 collapse) goes away, but GRPCRoute/TCPRoute/UDPRoute are still kept out, so the existing security behaviour doesn't change.

That said, this felt like a design call, so I'm totally open to switching to the simpler "omit kinds" version, or moving it to a design proposal first if you'd rather discuss it before merging. Let me know what you prefer!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NOT LGTM — the fix is correct and security-preserving (verified against the shipped Cilium 1.19.5 source), but the test suite is incomplete: it pins only positive/happy-path behavior in one cert mode and has no negative tests documenting what must not be rendered.

Business context: default TenantGateways on Cilium 1.19.x silently drop HTTPRoutes because divergent per-listener allowedRoutes.kinds on port 443 trigger cilium#45559 (issue #3070).

Blockers

B1: Tests are positive-only — missing negative tests and the bug's real cert mode

Treat the tests as the executable spec for this contract: they must document both how it should work and how it must not. Right now every assertion is happy-path ("the rendered listener carries {HTTPRoute, TLSRoute}"), and the one test that collects all port-443 listeners and asserts they're identical (TestReconcile_Port443ListenersShareKinds) runs in HTTP-01 only. Three things are unpinned:

  1. Negative — forbidden kinds (security boundary). No test asserts that GRPCRoute / TCPRoute / UDPRoute never appear on any port-443 listener. This is the whole reason the kinds set is restricted rather than dropped: those kinds aren't covered by the cozystack-route-hostname-policy VAP, so allowing them would let a route attach by hostname and bypass the apex check. Today this is only implied by len(kinds)==2; it should be asserted by name so the contract is explicit.
  2. Negative — no empty/default set. No test asserts that no port-443 listener has empty or nil AllowedRoutes.Kinds. An empty set means the Gateway API default (all kinds for the protocol) applies — the same VAP-bypass hole. The "restricted, never default" property is the security invariant and isn't pinned.
  3. Anti-collapse not covered in the bug's actual mode. The reported shape (#3070) is the DNS-01 / existingSecret root gateway: wildcard + apex + child-apex terminate listeners rendered together with TLS-passthrough on :443. The DNS-01 subtest of TestReconcile_HTTPSListenersRestrictRouteKindsToHTTPRoute has no passthrough services, and TestReconcile_ExistingSecretModeKeepsHTTPRedirectAndPassthrough renders the mix but asserts only that the passthrough listener exists — not its kinds, not the cross-listener identity. So the fix could regress in exactly the mode the bug manifests in and the suite would stay green.

How to close it (guidance, not a prescription):

  • Make the "collect every port-443 listener" assertion table-driven over all three cert modes (HTTP-01, DNS-01, existingSecret); each case seeds ≥1 terminate listener and TLSPassthroughServices, and the DNS-01/existingSecret cases also seed an inheriting child namespace so the *.<child-apex> listener is included in the sweep.
  • In that sweep assert the full contract per listener: (a) kinds equal the canonical set {HTTPRoute, TLSRoute} including the API group; (b) the set is identical across all :443 listeners; (c) it is non-empty; (d) it contains none of GRPCRoute / TCPRoute / UDPRoute.
  • Add at least one explicitly negative case whose name and comment state the failure mode it guards (e.g. "forbidden route kinds never reach a :443 listener — they would bypass the route-hostname VAP" and "divergent kinds across :443 listeners — would re-trigger cilium#45559"). The test name should read as the requirement.

Non-blocking follow-ups

  1. The code comment ("listing HTTPRoute here is harmless") understates one effect: on Cilium 1.19.x every port-443 listener surfaces ResolvedRefs=False / InvalidRouteKinds on the raw Gateway. It's cosmetic (Accepted/Programmed, traffic, security, and TenantGateway readiness are all unaffected — readiness checks only Accepted+Programmed), but worth a one-line note so operators inspecting kubectl get gateway -o yaml aren't surprised. Removable once Cilium 1.20 (with #45693) ships.
  2. Add Closes #3070.
  3. A gateway e2e run on a live 1.19.x cluster would turn the static proof that routing/passthrough still works into empirical confirmation.

@shreyaabaranwal

Copy link
Copy Markdown
Contributor Author

Aleksei Sviridkin (@lexfrei)
Thanks for the detailed review — really helpful framing on treating the tests as the spec.

Addressed all three blockers:

  • Forbidden kinds (GRPCRoute / TCPRoute / UDPRoute) are now asserted by name on every :443 listener, with a comment explaining they'd bypass the route-hostname VAP.
  • Added an explicit non-empty / non-nil check so the "default = all kinds" case can't slip through silently.
  • TestReconcile_Port443ListenersShareKinds is now table-driven across HTTP-01, DNS-01, and existingSecret. The DNS-01 and existingSecret cases seed an inheriting child namespace so the *. listener gets swept too — covering the mode the bug actually shows up in.

Also added the one-line note about the cosmetic ResolvedRefs=False on Cilium 1.19.x near the passthrough loop.

Re-requested your review — let me know if the contract coverage looks right now.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — the test-completeness blocker from the previous review is resolved.

The new assertPort443Contract sweep pins all three parts of the contract on every port-443 listener: (1) identical kinds across all listeners, (2) the forbidden kinds GRPCRoute/TCPRoute/UDPRoute never appear by name (with the route-hostname VAP-bypass rationale spelled out), and (3) the set is non-empty/non-nil. It runs table-driven across all three cert modes — HTTP-01, DNS-01, and existingSecret — each with a TLS-passthrough service so a terminate + passthrough mix is actually rendered on port 443. The DNS-01 and existingSecret cases seed an inheriting child namespace (namespace.cozystack.io/gateway + host labels) so collectInheritingChildApexes renders the per-child-apex listener and it is included in the sweep — covering the production root-gateway shape the bug manifests in, not just HTTP-01. I ran the package locally and all tests pass.

The reconciler.go comment now also documents the cosmetic ResolvedRefs=False/InvalidRouteKinds condition every port-443 listener surfaces on Cilium 1.19.x (Accepted/Programmed/traffic/TenantGateway-readiness all unaffected; removable once Cilium 1.20 ships), which addresses the earlier non-blocking note.

Approving.

@lexfrei
Aleksei Sviridkin (lexfrei) merged commit f4a000e into cozystack:main Jun 26, 2026
11 of 12 checks passed
@shreyaabaranwal
Shreya Baranwal (shreyaabaranwal) deleted the fix/gateway-cilium-45559-divergent-kinds branch June 26, 2026 16:05
@shreyaabaranwal

Shreya Baranwal (shreyaabaranwal) commented Jun 26, 2026 •

Copy link
Copy Markdown
Contributor Author

Thanks for the thorough reviews, Aleksei Sviridkin (@lexfrei) , the "tests as executable spec" framing was a great learning point for me. Happy to pick up the revert in #3087 once Cilium 1.20 lands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/networking Issues or PRs related to networking (ingress, gateway, vpn, metallb, kube-ovn) kind/bug Categorizes issue or PR as related to a bug size/L This PR changes 100-499 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(gateway): default TenantGateway drops HTTPRoutes on cilium 1.19.x via cilium#45559 (TLS-passthrough listeners)

2 participants