Repository navigation
gateway: TLS passthrough to a hostNetwork backend on the same node as the serving Envoy never connects #4823
Description
TLS passthrough through the tenant Gateway never connects when the backend is a hostNetwork pod on the same node as the cilium-envoy that handles the connection. The same backend on any other node connects every time.
I hit it on a three-node test cluster (isp-full, Kube-OVN chained with Cilium) and isolated it with a TLSRoute to a selectorless Service that has a single kube-apiserver endpoint:
| Client pod on | apiserver on node A | apiserver on node B | apiserver on node C |
|---|---|---|---|
| node A | 0/10 | 10/10 | 10/10 |
| node C | 10/10 | 10/10 | 0/10 |
The client times out in the TLS handshake (curl exit 28). cilium-dbg monitor --type drop shows nothing. Cilium 1.19.5 and 1.20.2 (the #3343 and #4638 stack) behave the same, so this is not a 1.20 regression.
This hits the platform's own api.<host> route. It passes through to the kubernetes Service, and its endpoints are the control-plane apiservers on host network. In the test about 10 of 30 requests to api.<host> hung, which matches one of three endpoints being on the client's node. External clients probably see the same on the node that announces the Gateway address.
I don't have the mechanism yet. My guess is that Envoy's upstream connection to a local host address takes a path where the reply skips the proxy, but nothing proves it. The next step is a full cilium-dbg monitor trace and conntrack for one failing connection.
Related: #3913.
Activity