Repository navigation
Add helm post-uninstall hook to delete PVs/PVCs for deleted tenant k8s clusters #1608
Description
Kubevirt-csi provisioned datavolumes and PVCs in the management cluster are not removed if the tenant k8s cluster is deleted without cleaning up the PVCs inside it.
To repro:
kubectl -n tenant-root create -f- <<EOF
apiVersion: apps.cozystack.io/v1alpha1
kind: Kubernetes
metadata:
name: trigger
spec: {}
EOF
k -n tenant-root get secret kubernetes-trigger-admin-kubeconfig -ojsonpath='{.data.super-admin\.conf}' | base64 -d > /tmp/trigger
kubectl --kubeconfig=/tmp/trigger create -f- <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: myclaim
spec:
accessModes:
- ReadWriteOnce
volumeMode: Filesystem
resources:
requests:
storage: 8Gi
EOF
kubectl -n tenant-root delete kuberneteses.apps.cozystack.io trigger
Observe that the related volumes in the management cluster remain.
This could be fixed by a post-delete hook that removes any lingering datavolumes.
Activity
dosubot commentedon Nov 6, 2025
To clean up lingering PVCs and KubeVirt DataVolumes in the management cluster after deleting a tenant Kubernetes cluster, you can add a Helm post-delete hook to the tenant cluster Helm chart. This is typically done by creating a Kubernetes Job annotated with
helm.sh/hook: post-deletethat runs after the Helm release is uninstalled.A recent example in the Cozystack repo shows how this is done for the MariaDB Operator: a Job runs
kubectl delete pvcwith a label selector matching the Helm release, and is granted the necessary RBAC permissions via a ServiceAccount, Role, and RoleBinding—all annotated as Helm hooks so they are created and cleaned up with the hook execution. You can adapt this pattern for tenant clusters by targeting PVCs and DataVolumes associated with the deleted cluster, using labels or naming conventions that link them to the tenant cluster. See the example implementation and details in this PR.Make sure your cleanup Job has the right permissions and label selectors to only delete resources belonging to the deleted tenant cluster. This approach ensures that orphaned storage resources are removed automatically when a tenant cluster is deleted via Helm.
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
lllamnyp commentedon Nov 6, 2025
WIP in #1606