Skip to content

tenant chart: nested namespace labels miss ancestor tenant.cozystack.io/* labels beyond direct parent #2810

Closed

Description

@IvanHunters

Summary

When a tenant B is created under another tenant A (which itself is under tenant-root), the namespace tenant-B receives tenant.cozystack.io/tenant-A and tenant.cozystack.io/tenant-B labels, but not tenant.cozystack.io/tenant-root. This breaks the chain that egress CiliumClusterwideNetworkPolicy objects rely on and prevents the root ingress controller from proxying traffic into nested tenants.

Impact

  • addons.ingressNginx.hosts on a nested Kubernetes (k8s-tenant) CR creates an Ingress on the root ingress class, but the root ingress controller cannot reach the proxy Service in the nested tenant namespace because the tenant-root-egress CiliumClusterwideNetworkPolicy restricts egress to namespaces labeled tenant.cozystack.io/tenant-root.
  • External traffic to the host gets a TCP accept on the root ingress IP, the HTTP request is sent successfully, but the upstream connect times out. Root ingress controller logs:
    upstream timed out (110: Operation timed out) while connecting to upstream,
    upstream: "http://10.244.x.y:30000/", host: "<configured-host>"
    

Repro

  1. Create apps.cozystack.io/v1alpha1/Tenant A in tenant-root namespace. Resulting tenant-A namespace correctly has both tenant.cozystack.io/tenant-root and tenant.cozystack.io/tenant-A labels.
  2. Create apps.cozystack.io/v1alpha1/Tenant B in tenant-A namespace. Resulting tenant-B namespace has only tenant.cozystack.io/tenant-A and tenant.cozystack.io/tenant-B. Missing: tenant.cozystack.io/tenant-root.
  3. Create apps.cozystack.io/v1alpha1/Kubernetes in tenant-B namespace with addons.ingressNginx.enabled: true and addons.ingressNginx.hosts: [foo.example.com].
  4. Point DNS foo.example.com to the root ingress public IP.
  5. curl http://foo.example.com/ from the internet — TCP connects, HTTP request is sent, no response.

Root cause

packages/apps/tenant/templates/namespace.yaml derives ancestor labels by splitting the parent namespace name on -:

```yaml
{{- $parts := splitList "-" .Release.Namespace }}
{{- range $i, $v := $parts }}
{{- if ne $i 0 }}
tenant.cozystack.io/{{ join "-" (slice $parts 0 (add $i 1)) }}: ""
{{- end }}
{{- end }}
```

This assumes tenant names encode the full hierarchy chain in their name (e.g. tenant-root-a-b), but the actual naming convention uses short names (tenant-a, tenant-b). So for Release.Namespace = \"tenant-a\", the loop yields only tenant.cozystack.io/tenant-a, missing tenant.cozystack.io/tenant-root (and any deeper ancestor in a 3+ level hierarchy).

Expected

Each tenant namespace should carry tenant.cozystack.io/<ancestor> labels for all ancestors up to tenant-root, so the egress/ingress policies derived from those labels grant cross-tenant traversal as the platform model intends.

Suggested fix

Read the parent namespace's existing tenant.cozystack.io/* labels via `lookup` and propagate them all, plus add the namespace's own label. Roughly:

```yaml
{{- $parentNS := lookup "v1" "Namespace" "" .Release.Namespace }}
{{- range $k, $_ := (default dict $parentNS.metadata.labels) }}
{{- if hasPrefix "tenant.cozystack.io/" $k }}
{{ $k }}: ""
{{- end }}
{{- end }}
tenant.cozystack.io/{{ $tenantName }}: ""
```

Workaround

Manually label the affected nested namespace until the chart is fixed:

```bash
kubectl label namespace tenant-b tenant.cozystack.io/tenant-root=""
```

Activity

  1. dosubot commented on Jun 3, 2026

    @dosubot

    This is a duplicate of #2171, which reports the same root cause — the hyphen-splitting logic in packages/apps/tenant/templates/namespace.yaml cannot reconstruct the full ancestry chain when tenant names don't encode the hierarchy [1].

    The same problem affects the egress rules in networkpolicy.yaml, which uses the identical splitting pattern to generate parent ingress access rules [2].

    Your suggested lookup-based fix is essentially the same approach proposed in #2171 — read the parent namespace's existing tenant.cozystack.io/* labels and propagate them forward. That's the cleanest path since labels already encode ancestry correctly at depth 1 and the fix makes each level self-consistent by inheriting from its parent.

    Worth noting: PR #2470 (merged May 26, 2026) introduced a controller-based label inheritance pattern for Gateway API support (namespace.cozystack.io/gateway labels) that patches descendant namespaces with ancestor information [3]. That same pattern could potentially be adapted for the tenant.cozystack.io/* labels as a more robust solution than lookup (which only evaluates at Helm template render time and wouldn't self-heal if a parent is relabeled later).

    To reply, just mention @dosu.


    Share context across your team and agents. Try Dosu.

    Leave Feedback Learn about cozystack with Dosu Add Dosu to your team

  2. added
    kind/bugCategorizes issue or PR as related to a bug
    area/platformIssues or PRs related to platform infrastructure (bundle, flux, talos, installer)
    on Jun 4, 2026
  3. lexfrei commented on Jul 1, 2026

    @lexfrei
    Contributor

    Duplicate of #2171 — same root cause: the hyphen-split of .Release.Namespace in packages/apps/tenant/templates/namespace.yaml (still present on main). #2171 is broader (Cilium egress netpol + ancestor labels); this is the labels-only subset, and #2171 already documents this exact missing-label symptom. Consolidating on #2171.

  4. added a commit that references this issue on Jul 3, 2026
    1baf126
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/platformIssues or PRs related to platform infrastructure (bundle, flux, talos, installer)kind/bugCategorizes issue or PR as related to a bugtriage/duplicateIndicates an issue is a duplicate of another issue

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions