Describe the bug
The post-delete cleanup hooks of harbor and clickhouse remove the release's PVCs with --wait, but the hook Role grants only get, list and delete on persistentvolumeclaims. Waiting opens a watch, so every run logs a forbidden watch error and then reports success. The PVCs are removed, but the hook cannot actually wait for them, and the job log carries an error on every uninstall.
Environment
- Cozystack v1.6.1 (platform 54);
packages/apps/harbor/templates/hooks/cleanup.yaml on main still lists verbs: ["get", "list", "delete"]
- on-prem, Talos
To Reproduce
- Create and then remove a ClickHouse (or Harbor) application in a tenant namespace.
- Read the log of the
<release>-cleanup Job.
Expected behaviour
The hook waits for the PVCs to disappear and exits without RBAC errors.
Actual behaviour
Deleting orphaned PVCs for clickhouse-clickhouse-backup...
persistentvolumeclaim "data-volume-template-chi-clickhouse-clickhouse-backup-clickhouse-0-0-0" deleted
persistentvolumeclaim "log-volume-template-chi-clickhouse-clickhouse-backup-clickhouse-0-0-0" deleted
E0904 12:34:06.605355 7 reflector.go:166] "Unhandled Error" err="k8s.io/client-go/tools/watch/informerwatcher.go:146: Failed to watch *unstructured.Unstructured: persistentvolumeclaims \"log-volume-template-chi-clickhouse-clickhouse-backup-clickhouse-0-0-0\" is forbidden: User \"system:serviceaccount:tenant-rup6ev1r6z:clickhouse-clickhouse-backup-cleanup\" cannot watch resource \"persistentvolumeclaims\" in API group \"\" in the namespace \"tenant-rup6ev1r6z\""
PVC cleanup complete.
Adding watch to the persistentvolumeclaims rule of the hook Roles (harbor, clickhouse, and any sibling chart that reuses the same hook) is the whole fix. Separately, on Harbor the hook pod sometimes fails to mount its token because the hook ServiceAccount is already gone (failed to fetch token: serviceaccount "<release>-cleanup" not found), which points at the hook weights of the ServiceAccount versus the Job; I have the events for that as well if useful.
Describe the bug
The post-delete cleanup hooks of
harborandclickhouseremove the release's PVCs with--wait, but the hookRolegrants onlyget,listanddeleteonpersistentvolumeclaims. Waiting opens a watch, so every run logs a forbidden watch error and then reports success. The PVCs are removed, but the hook cannot actually wait for them, and the job log carries an error on every uninstall.Environment
packages/apps/harbor/templates/hooks/cleanup.yamlonmainstill listsverbs: ["get", "list", "delete"]To Reproduce
<release>-cleanupJob.Expected behaviour
The hook waits for the PVCs to disappear and exits without RBAC errors.
Actual behaviour
Adding
watchto thepersistentvolumeclaimsrule of the hook Roles (harbor, clickhouse, and any sibling chart that reuses the same hook) is the whole fix. Separately, on Harbor the hook pod sometimes fails to mount its token because the hook ServiceAccount is already gone (failed to fetch token: serviceaccount "<release>-cleanup" not found), which points at the hook weights of the ServiceAccount versus the Job; I have the events for that as well if useful.