Skip to content

feat(kubernetes): expose only etcd.replicas to users, keep per-cluster etcd sizing internal #3179

Open

Description

@scooby87

Context

Design proposal cozystack/community#25 folds etcd into the kubernetes app (one etcd per cluster). As part of that work, the app will gain a new etcd values block. The proposal's draft exposes the full block to users:

etcd:
  replicas: 3
  size: 4Gi
  storageClass: ""
  resources:
    cpu: 1000m
    memory: 512Mi

This issue tracks a scoped design decision raised in review by Sergei Makarov (@SerjioTT) (comment): keep the user-facing etcd surface minimal — expose only replicas — and treat everything else as a control-plane implementation detail.

Problem

Per-cluster etcd is cluster infrastructure, not a user surface. The kubernetes chart already has a clear convention for this split:

  • Control plane (apiserver / cm / scheduler / konnectivity) — tunable via controlPlane.*.resourcesPreset.
  • Infrastructure satellites (kccm, cluster-autoscaler, kcsi-controller) — resources are hardcoded, not exposed through values, and there are no field reports of anyone needing to tune them.

Exposing a full {replicas, size, storageClass, resources} block for etcd would break this convention and add UX complexity for knobs users don't need to turn. resourcePresets for the 4 CP components are already hard enough to reason about.

Proposal

Expose a single knob:

## @param {Etcd} etcd - Control-plane etcd datastore for this cluster.
etcd:
  ## @field {int} replicas=3 - etcd replicas. 3 = HA (Raft quorum). 1 = non-HA/dev.
  replicas: 3

Everything else becomes chart-internal:

  • resources — hardcoded to the current packages/extra/etcd default (cpu: 1000m, memory: 512Mi); VPA handles growth (same pattern already used for the satellites).
  • size — hardcoded to 4Gi (matches current behaviour; ample headroom for a Kamaji datastore).
  • storageClass — derived from replicas, not set by the user:
    • replicas: 3 → local-style SC (etcd replicates via Raft),
    • replicas: 1 → replicated/DRBD-style SC (single member gets storage-layer HA as a safety net).
    • The SC mapping should be platform-level config, not app-level — e.g. via cozystack-values (default-etcd-sc-local / default-etcd-sc-replicated), with the kubernetes app just picking between them by etcd.replicas.

Why not the full block

  • Matches the existing satellite convention (hardcoded infra resources + VPA).
  • Keeps the Kubernetes CR simple — no new tuning surface users didn't ask for.
  • Forward-compatible: the minimal API is a strict subset of the full block. If a real production need to override resources/size/storageClass shows up later, the API can be widened with no breaking change.

Notes

  • Default replicas: 3 (safe-by-default HA); dev/throwaway clusters flip to 1 explicitly.
  • Surface the etcd footprint in NOTES.txt so consumers see the full per-cluster cost (~1000m/512Mi × replicas).

Dependencies

Activity

  1. added
    kind/featureCategorizes issue or PR as related to a new feature
    triage/needs-triageIndicates an issue needs triage by a maintainer
    area/kubernetesIssues or PRs related to the tenant Kubernetes app
    priority/important-longtermImportant over the long term, but may not be staffed and/or may need multiple releases to complete
    on Jul 2, 2026
  2. added
    triage/needs-proposalIndicates an issue needs a design proposal in cozystack/community before work starts here
    and removed
    triage/needs-triageIndicates an issue needs triage by a maintainer
    on Sep 23, 2026
  3. lexfrei commented on Sep 23, 2026

    @lexfrei
    Contributor

    The design question is already in cozystack/community#25, so the decision should happen there. This issue stays open to track the implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/kubernetesIssues or PRs related to the tenant Kubernetes appkind/featureCategorizes issue or PR as related to a new featurepriority/important-longtermImportant over the long term, but may not be staffed and/or may need multiple releases to completetriage/needs-proposalIndicates an issue needs a design proposal in cozystack/community before work starts here

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions