Repository navigation
[Bug]: ScheduledBackup stuck on "already exists" error and fails to dispatch new backups #10562
Description
Is there an existing issue already for this bug?
- I have searched for an existing issue, and could not find anything. I believe this is a new bug.
I have read the troubleshooting guide
- I have read the troubleshooting guide and I think this is a new bug.
I am running a supported version of CloudNativePG
- I have read the troubleshooting guide and I think this is a new bug.
Contact Details
via Github
Version
1.28
What version of Kubernetes are you using?
1.32
What is your Kubernetes environment?
Other
How did you install the operator?
Helm
What happened?
We've observed a bug in the ScheduledBackups on three of our cloudnative-pg clusters.
For a while the ScheduledBackup is fine. At some point the operator starts logging "backups.postgresql.cnpg.io "<scheduled backup name>-<timestamp>" already exists And after that, the ScheduledBackup just completely stops running. The operator continues logging this error indefinitely. The only workaround seems to be to delete the Backup resource with the timestamp that's being logged: kubectl -n <namespace> delete backup/<scheduled backup name>-<timestamp>
I understand that a backup might occasionally fail for some reason. But the operator should be able to get past that. In this case it seems that the operator is repeatedly trying to create the
Cluster resource
I don't think the full cluster is relevant here. Omitting bc it would take a lot of redacting. Can include if it's actually needed.
Here's the ScheduledBackup definition.
apiVersion: postgresql.cnpg.io/v1
kind: ScheduledBackup
metadata:
name: cnpg-cluster-name-scheduled-backup
namespace: namespace
labels:
app.kubernetes.io/name: cnpg-cluster-name
app.kubernetes.io/instance: release-name
app.kubernetes.io/managed-by: Helm
environment: eng
kubernetes.cluster: k8s-cluster-name
spec:
schedule: "42 35 22 * * *"
backupOwnerReference: self
immediate: true
method: plugin
pluginConfiguration:
name: barman-cloud.cloudnative-pg.io
cluster:
name: cnpg-cluster-name
And here is the live resource via get -o json:
{
"apiVersion": "postgresql.cnpg.io/v1",
"kind": "ScheduledBackup",
"metadata": {
"creationTimestamp": "2026-01-12T19:07:22Z",
"generation": 1,
"labels": {
"app.kubernetes.io/instance": "database-name",
"app.kubernetes.io/managed-by": "Helm",
"app.kubernetes.io/name": "cnpg-cluster-name",
"kubernetes.cluster": "kubernetes-cluster-name"
},
"name": "cnpg-cluster-name-scheduled-backup",
"namespace": "namespace",
"resourceVersion": "2787404118",
"uid": "e5c7831a-0f53-46c8-98ea-d4f9c4bbcf14"
},
"spec": {
"backupOwnerReference": "self",
"cluster": {
"name": "cnpg-cluster-name"
},
"immediate": true,
"method": "plugin",
"pluginConfiguration": {
"name": "barman-cloud.cloudnative-pg.io"
},
"schedule": "42 35 22 * * *"
},
"status": {
"lastCheckTime": "2026-04-16T22:35:42Z",
"lastScheduleTime": "2026-04-16T22:35:42Z",
"nextScheduleTime": "2026-04-17T22:35:42Z"
}
}
Note that this is taken on April 27 and the nextScheduleTime is still April 17.
Relevant log output
The full error message looks like this:
{
"time": "2026-04-27T21:39:13.141628998Z",
"level": "error",
"ts": "2026-04-27T21:39:13.141570358Z",
"msg": "Reconciler error",
"controller": "scheduled-backup",
"controllerGroup": "postgresql.cnpg.io",
"controllerKind": "ScheduledBackup",
"ScheduledBackup": {
"name": "cnpg-cluster-name-scheduled-backup",
"namespace": "user-application"
},
"namespace": "user-application",
"name": "cnpg-cluster-name-scheduled-backup",
"reconcileID": "d207ef72-18e1-4f7f-be93-ecab28fb4790",
"error": "backups.postgresql.cnpg.io \"cnpg-cluster-name-scheduled-backup-20260417223542\" already exists",
"stacktrace": "sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\tpkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:474\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\tpkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:421\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1\n\tpkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:296",
"kubernetes": {
"pod_name": "cloudnative-pg-operator-5456498967-0iewp",
"labels": {
"app.kubernetes.io/instance": "cloudnative-pg-operator",
"app.kubernetes.io/name": "cloudnative-pg",
"pod-template-hash": "5456498967"
},
"pod_ip": "redacted"
}
}It repeats ~8 times / hour for each cluster.
Code of Conduct
- I agree to follow this project's Code of Conduct
Activity
dosubot commentedon Apr 27, 2026
This looks like it's related to a known bug that was fixed in v1.28.1 — the ScheduledBackup controller had a broken field indexer that couldn't detect existing Backup resources, causing it to repeatedly try to
Createa Backup that already exists and then get stuck in a retry loop [1].Specifically, the
GetChildBackups()method was checking for owner references with kindBackupinstead ofScheduledBackup, so it always returned an empty slice. This was fixed in PR #9489 by replacing the owner reference-based indexer with a label-based indexer using thecnpg.io/scheduled-backuplabel [2].You mentioned you're on 1.28 (latest patch) — can you confirm the exact patch version? The fix shipped in v1.28.1 (Feb 5, 2026) and is also in v1.28.2 (Apr 1, 2026). If you're still on v1.28.0, upgrading should resolve the "already exists" loop.
There's also a separate, still-open issue (#10385) where backups stuck in the
walArchivingFailingphase can permanently block scheduled backups [3]. The reconciler also only checks forIsConflict(HTTP 409) errors on creation but notIsAlreadyExists, which means an existing Backup with the same deterministic name causes an unhandled error that prevents status updates [4].Immediate workaround: Delete the stuck Backup resource as you've already discovered:
Permanent fix: Upgrade to v1.28.1 or later if you haven't already.
To reply, just mention @dosu.
Share context across your team and agents. Try Dosu.
alexklibisz commentedon Apr 27, 2026
#9485 and #9489 seem related but from reading the description and PR I'm not convinced it's the same thing.
armru commentedon May 4, 2026
Hello @alexklibisz thanks for the report!
I've opened a PR
fix(scheduledbackup): treat AlreadyExists from Create as no-op
refactor(scheduledbackup): observe before acting on the next iteration
fix(scheduledbackup): adopt existing Backup instead of looping on Alr…
fix(scheduledbackup): adopt existing Backup instead of looping on Alr…
fix(scheduledbackup): adopt existing Backup instead of looping on Alr…
fix(scheduledbackup): adopt existing Backup instead of looping on Alr…