Repository navigation
Nested tenants (depth > 1) cannot reach root tenant ingress due to incomplete egress policy #2171
Description
Problem
Tenants nested more than one level deep cannot access services in the root tenant (e.g., the root ingress controller). Traffic to the root ingress LoadBalancer IP times out because the egress policy only allows access to the immediate parent's services, not the full ancestry chain.
Example Hierarchy
tenant-root (has ingress at 192.168.50.100)
└── tenant-prod (can reach tenant-root ingress ✅)
└── tenant-prod-argocd (cannot reach tenant-root ingress ❌)
Symptoms
Pods in tenant-prod-argocd trying to curl api.enigma.vgijssel.nl (which resolves to tenant-root's ingress IP) timeout:
* Host api.enigma.vgijssel.nl:443 was resolved.
* IPv4: 192.168.50.100
* Trying 192.168.50.100:443...
[timeout]
Meanwhile, the same curl from tenant-prod or tenant-root succeeds.
Root Cause
The CiliumClusterwideNetworkPolicy template in packages/apps/tenant/templates/networkpolicy.yaml (lines ~56-68) generates egress rules by splitting .Release.Namespace:
{{- $parts := splitList "-" .Release.Namespace }}
{{- range $i, $v := $parts }}
{{- if ne $i 0 }}
- matchLabels:
cozystack.io/service: ingress
"k8s:io.kubernetes.pod.namespace": {{ join "-" (slice $parts 0 (add $i 1)) }}
{{- end }}
{{- end }}For tenant-prod-argocd (deployed in namespace tenant-prod):
.Release.Namespace="tenant-prod"$parts=["tenant", "prod"]- Generated egress: only
tenant-prod - Missing:
tenant-root
The template assumes the namespace name encodes the full hierarchy, but tenant-prod doesn't contain information about its parent tenant-root.
Additional Evidence
The namespace labels also only include immediate parent, not full ancestry:
# tenant-prod labels (correct - has both)
tenant.cozystack.io/tenant-prod: ""
tenant.cozystack.io/tenant-root: ""
# tenant-prod-argocd labels (missing tenant-root)
tenant.cozystack.io/tenant-prod: ""
tenant.cozystack.io/tenant-prod-argocd: ""
# ❌ Missing: tenant.cozystack.io/tenant-root: ""Related Issue
This is the same class of bug as #1970 (nested Fluent Bit cannot send logs to parent VLogs), where the egress policy is missing rules for parent services.
Suggested Fix
The namespace labels already track ancestry via tenant.cozystack.io/* labels. The egress policy template could iterate through all tenant.cozystack.io/* labels on the parent namespace instead of trying to parse the namespace name.
Alternatively, ensure child namespaces inherit all tenant.cozystack.io/* labels from their entire ancestry chain, then use those labels to generate egress rules.
Workaround
Manually add a CiliumNetworkPolicy in the affected namespace:
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: allow-to-root-ingress
namespace: tenant-prod-argocd
spec:
endpointSelector: {}
egress:
- toEndpoints:
- matchLabels:
cozystack.io/service: ingress
k8s:io.kubernetes.pod.namespace: tenant-rootEnvironment
- Cozystack version: 1.0.2
Activity
fix(tenant): inherit full ancestor label chain from parent namespace (c…
fix(platform): backfill tenant ancestor labels (backport #2912)
fix(platform): backfill tenant ancestor labels (backport #2912)