Skip to content

CNPG backups broken on 1.6: standard image variant lacks barman-cloud but backups use native barmanObjectStore #3300

Description

Summary

CNPG backups of system PostgreSQL databases are non-functional on 1.6: the backup config uses the native spec.backup.barmanObjectStore, but the pinned PostgreSQL image variant no longer ships the barman-cloud-* binaries, so backups fail with:

LastBackupFailed: exec: "barman-cloud-backup": executable file not found in $PATH

WAL archiving is affected too (no barman-cloud-wal-archive either) — beyond failing base backups, a failing archive_command blocks WAL recycling and risks filling the instance disk.

Versions

  • cozystack v1.6.0-rc.x
  • CNPG operator 1.27.1 (chart cloudnative-pg 0.26.1; observed 1.27.3 on cluster)
  • PostgreSQL image ghcr.io/cloudnative-pg/postgresql:17.7-standard-trixie (PostgreSQL 17.7, standard variant)

Root cause

#2342 moved system Postgres to the CNPG standard image variant (via ClusterImageCatalog). Per that PR's own variant table:

  • standard: minimal + pgaudit + extensions ✅ "We use this"
  • system: standard + Barman Cloud binaries (deprecated) ❌

So the chosen standard variant intentionally excludes the barman-cloud-* tools (native Barman Cloud is deprecated in CNPG 1.27, removed in 1.29 — the replacement is the Barman Cloud Plugin). But cozystack still configures native spec.backup.barmanObjectStore, which requires those binaries in the image, in at least:

Migration 37 pins all system DBs (keycloak-db, grafana-db, alerta-db, seaweedfs-db, harbor DB) to 17.7-standard-trixie, so the whole system-Postgres backup surface is affected; the cozy-default-cnpg strategy extends the breakage to tenant Postgres apps that run on the barman-less image.

Reproduction

On a 1.6 cluster, enable a barman backup on any system PG DB (e.g. keycloak backup.enabled=true + destinationPath/creds), then trigger a Backup:

$ kubectl -n cozy-keycloak get backup keycloak-db-test -o jsonpath='{.status.error}'
instance manager was restarted during backup on pod keycloak-db-2
$ kubectl -n cozy-keycloak get cluster keycloak-db -o jsonpath='{.status.conditions[?(@.type=="LastBackupSucceeded")].message}'
exec: "barman-cloud-backup": executable file not found in $PATH

Confirmed the image has no barman tools:

$ kubectl -n cozy-keycloak exec keycloak-db-1 -c postgres -- sh -c 'ls /usr/bin/barman* /usr/local/bin/barman* 2>&1; which barman-cloud-backup'
# (nothing)

Suggested fix

Adopt the CNPG Barman Cloud Plugin across the CNPG-backup surface: install the plugin (operator + ObjectStore CRD) and convert spec.backup.barmanObjectStore → spec.plugins: [{name: barman-cloud.cloudnative-pg.io, ...}] + an ObjectStore CR in system/keycloak, system/backupstrategy-controller (cozy-default-cnpg), and apps/postgres. (Alternatively ship an image variant that still bundles barman-cloud, but that runs counter to CloudNativePG's direction.)

Related: #2342 (image variant change, the cause), #3174 (keycloak backup, affected).

Found while testing #3174 keycloak backups on a freedom-portal cluster (cozystack 1.6).

Activity

  1. added a commit that references this issue on Jul 21, 2026
    e73ad68
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bug

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions