You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[backupstrategy-controller] Replace platform-migrations with a dedicated ch-backup-client image #3420
The Altinity ClickHouse backup strategy runs its Pod on platform-migrations — the platform migrations image. It should get a dedicated, minimal image owned by packages/system/backupstrategy-controller.
#3405 made the current reuse safe (the two pins now move in lockstep) so the v1.6.0 promote could proceed. It deliberately did not change the image, because that means a new image, a new build target and a new digest introduced during a release cut. This issue is the follow-up it points to; the TODO lives in packages/system/backupstrategy-controller/values.yaml.
What the Pod actually needs
The strategy script in templates/strategy-altinity-default.yaml drives clickhouse-backup's HTTP API on :7171. Measured against the rendered script, it uses exactly four binaries: curl, jq, sleep, date.
Surface. A tenant-adjacent backup Pod carries cluster tooling it has no use for. kubectl and helm are on the PATH of a Pod whose entire job is to call an HTTP API and parse JSON.
A load-bearing single line. Because the image's own ENTRYPOINT is the migration runner, the command: ["/bin/sh","-c"] override in the strategy template is the only thing standing between this Pod and running platform migrations. That is safe today and one careless edit from not being.
Coupling. Bumping ETCD_OPERATOR_VERSION or editing any migration script re-digests platform-migrations, which forces the backup client's pin to move for reasons that have nothing to do with backups. This is the coupling that produced the release blocker #3405 fixed.
Why "one more tag to re-digest" is not a reason to keep it
That objection is recorded in the tree as the justification for the reuse, and it does not survive contact with the actual defect. Every other first-party image already pays that cost, automatically, on every cut. What broke was not an extra tag — it was a pin with no producer. chBackupClientImage had no build step writing it, so it froze at v1.4.0-rc.2 while .migrations.image advanced to v1.5.0, and promotion then tried to retag two digests to one stable tag.
An image owned by its own package cannot reproduce that failure, because the producer and the pin live together.
Proposed shape
packages/system/backupstrategy-controller/images/ch-backup-client/Dockerfile — alpine + curl + jq + ca-certificates, no apk add at run time so the Pod stays air-gap-safe (the property the current reuse was protecting).
An image-ch-backup-client target in that package's Makefile stamping .backupStrategyController.chBackupClientImage in its ownvalues.yaml.
hack/image-pin-consistency.bats should keep passing throughout; it asserts the general invariant rather than this specific pair.
Alternatives considered
Two third-party images already pinned in this repo were evaluated as drop-in replacements. Neither works:
Candidate
curl
jq
Size
Verdict
clastix/kubectl:v1.32
yes
no
95 MB
unusable — the script uses jq five times
alpine/k8s:1.36.2
yes
yes
1.25 GB
3.6× worse than the status quo
clastix/kubectl is otherwise the best-established third-party pattern here (digest-pinned, cozy-lib.image-wrapped, Renovate-managed via a regex manager, covered by helm-unittest). If a small, well-maintained public curl+jq image exists with comparable provenance, pinning it that way would be preferable to building one — it would also sit outside the promote path entirely. I did not find one I would trust.
Not urgent
The current state is correct and guarded. This is cleanup and hardening, not a bug.
Heads-up that this now has a second call site. PR #3385 makes examples/backups/clickhouse/01-create-strategy.sh read the client image off the shipped cozy-default-altinity strategy instead of hardcoding alpine:3.19 + apk add curl jq — the demo (and the e2e backup round-trip that drives it) needed a pre-baked curl+jq image so a backup does not depend on the Alpine CDN and works air-gapped.
Consequence for this issue: the command: ["/bin/sh","-c"] override that keeps platform-migrations from running run-migrations.sh is now load-bearing in two places rather than one — the strategy template and the example. When the dedicated ch-backup-client image lands, the example inherits it automatically (it reads the image from the strategy at apply time, no second pin to update), but the override in the example's own pod spec should be revisited at the same time.
Summary
The Altinity ClickHouse backup strategy runs its Pod on
platform-migrations— the platform migrations image. It should get a dedicated, minimal image owned bypackages/system/backupstrategy-controller.#3405 made the current reuse safe (the two pins now move in lockstep) so the v1.6.0 promote could proceed. It deliberately did not change the image, because that means a new image, a new build target and a new digest introduced during a release cut. This issue is the follow-up it points to; the
TODOlives inpackages/system/backupstrategy-controller/values.yaml.What the Pod actually needs
The strategy script in
templates/strategy-altinity-default.yamldrives clickhouse-backup's HTTP API on:7171. Measured against the rendered script, it uses exactly four binaries:curl,jq,sleep,date.What it gets instead
ghcr.io/cozystack/cozystack/platform-migrations)kubectl,helm,git,cozyhr, theetcd-migratebinary, every migration script, vendored etcd v1alpha2 CRDsENTRYPOINTrun-migrations.shA minimal
alpine:3.24+curl+jq+ca-certificatesimage measures 23 MB — 15× smaller.Why this is worth fixing
Surface. A tenant-adjacent backup Pod carries cluster tooling it has no use for.
kubectlandhelmare on the PATH of a Pod whose entire job is to call an HTTP API and parse JSON.A load-bearing single line. Because the image's own
ENTRYPOINTis the migration runner, thecommand: ["/bin/sh","-c"]override in the strategy template is the only thing standing between this Pod and running platform migrations. That is safe today and one careless edit from not being.Coupling. Bumping
ETCD_OPERATOR_VERSIONor editing any migration script re-digestsplatform-migrations, which forces the backup client's pin to move for reasons that have nothing to do with backups. This is the coupling that produced the release blocker #3405 fixed.Why "one more tag to re-digest" is not a reason to keep it
That objection is recorded in the tree as the justification for the reuse, and it does not survive contact with the actual defect. Every other first-party image already pays that cost, automatically, on every cut. What broke was not an extra tag — it was a pin with no producer.
chBackupClientImagehad no build step writing it, so it froze atv1.4.0-rc.2while.migrations.imageadvanced tov1.5.0, and promotion then tried to retag two digests to one stable tag.An image owned by its own package cannot reproduce that failure, because the producer and the pin live together.
Proposed shape
packages/system/backupstrategy-controller/images/ch-backup-client/Dockerfile—alpine+curl+jq+ca-certificates, noapk addat run time so the Pod stays air-gap-safe (the property the current reuse was protecting).image-ch-backup-clienttarget in that package's Makefile stamping.backupStrategyController.chBackupClientImagein its ownvalues.yaml.packages/core/platform/Makefile, and theTODOblock invalues.yaml.hack/image-pin-consistency.batsshould keep passing throughout; it asserts the general invariant rather than this specific pair.Alternatives considered
Two third-party images already pinned in this repo were evaluated as drop-in replacements. Neither works:
clastix/kubectl:v1.32jqfive timesalpine/k8s:1.36.2clastix/kubectlis otherwise the best-established third-party pattern here (digest-pinned,cozy-lib.image-wrapped, Renovate-managed via a regex manager, covered by helm-unittest). If a small, well-maintained public curl+jq image exists with comparable provenance, pinning it that way would be preferable to building one — it would also sit outside the promote path entirely. I did not find one I would trust.Not urgent
The current state is correct and guarded. This is cleanup and hardening, not a bug.