Skip to content

etcd v1alpha2 adoption (migration 50): etcd-migrate has no kubeconfig in-cluster → adoption aborts #3255

Description

What happened

After getting past the cert-SAN wait (#3243), migration 50's etcd-migrate invocations abort immediately — both the dry-run and --apply:

=== etcd-migrate dry-run ===
Error: error building kubeconfig: stat /root/.kube/config: no such file or directory

etcd-migrate only reads a kubeconfig file (-k/--kubeconfig, default /root/.kube/config). Unlike kubectl, it does not fall back to the in-cluster ServiceAccount (KUBERNETES_SERVICE_HOST + the mounted SA token). The pre-upgrade hook Job (packages/core/platform/templates/migration-hook.yaml) and migrations/50 set no KUBECONFIG and pass no --kubeconfig, so /root/.kube/config does not exist in the Job pod and etcd-migrate can't build a client.

kubectl in the same pod works (it uses in-cluster auth automatically), which masks this until execution reaches the etcd-migrate calls — i.e. only after the cert step is cleared.

Impact

Every in-cluster 1.5→1.6 etcd adoption, once #3243's cert step is passed. The whole platform pre-upgrade hook fails.

Suggested fix

Synthesize an in-cluster kubeconfig from the mounted ServiceAccount and pass it to both etcd-migrate calls (dry-run + --apply). Minimal patch in migrations/50:

ETCD_MIGRATE_KUBECONFIG=/tmp/in-cluster.kubeconfig
sa=/var/run/secrets/kubernetes.io/serviceaccount
cat > "$ETCD_MIGRATE_KUBECONFIG" <<EOF
apiVersion: v1
kind: Config
clusters: [{name: in, cluster: {certificate-authority: $sa/ca.crt, server: https://$KUBERNETES_SERVICE_HOST:${KUBERNETES_SERVICE_PORT:-443}}}]
contexts: [{name: in, context: {cluster: in, user: sa, namespace: $(cat $sa/namespace)}}]
current-context: in
users: [{name: sa, user: {tokenFile: $sa/token}}]
EOF
etcd-migrate --kubeconfig="$ETCD_MIGRATE_KUBECONFIG" ...

Verified working on a 1.5.2 → 1.6.0-rc.1 adoption (dry-run + --apply both succeed with the kubeconfig).

Found during a 1.5.2 → 1.6.0-rc.1 upgrade / etcd-adoption test on a 3-node cluster.

Activity

  1. myasnikovdaniil commented on Jul 21, 2026

    @myasnikovdaniil
    Contributor

    Fixed on main by #3270 (merged 2026-07-16). migrations/50 now synthesizes an in-cluster kubeconfig from the mounted ServiceAccount before invoking etcd-migrate, which is the fix suggested here.

    It differs from the suggestion in one respect: the server URL is https://kubernetes.default.svc rather than https://$KUBERNETES_SERVICE_HOST:$PORT, because on an IPv6-only cluster that env var is a bare IPv6 address that would need bracketing to form a valid URL.

    Closing as resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bug

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions