vm-disk from golden image stays unbound forever when target StorageClass differs from the image's #2908

Summary

Creating a VM disk from a golden image into a StorageClass different from the one the golden images live in never completes: the temporary clone PVC stays unbound forever and no error is surfaced to the user. Reported in the community chat on v1.4.2 with LINSTOR storage (golden images in a replicated StorageClass, target disk in a node-local one); the relevant code is unchanged on main.

Symptom

1. Golden images (vm-default-images-* DataVolumes in cozy-public) live in a replicated StorageClass.
2. The user creates a vm-disk with source.image and a local target StorageClass.
3. CDI attempts a CSI clone: it creates a temporary PVC tmp-pvc-… in cozy-public with the target StorageClass (WaitForFirstConsumer) plus a prep-… pod.
4. The PVC never binds; the prep pod stays stuck for hours with Unable to attach or mount volumes: PVC cozy-public/tmp-pvc-… is not bound, and LINSTOR CSI logs context deadline exceeded.

Mechanism

1. packages/apps/vm-disk/templates/dv.yaml renders a DataVolume whose source.pvc points at cozy-public/vm-default-images-* while storage.storageClassName is the user-selected target class, so picking any class other than the images' one produces a cross-StorageClass clone request.
2. packages/system/kubevirt-cdi/templates/cdi-cr.yaml sets a global clone strategy override on the CDI CR and packages/system/kubevirt-cdi/values.yaml defaults it to csi-clone, so every install that has not changed the value forces csi-clone for every clone. GetGlobalCloneStrategyOverride is consulted before the target StorageClass's StorageProfile in pkg/controller/clone/planner.go, so the override replaces per-profile selection entirely.
3. Neither of CDI's advanced clone paths guards against a cross-StorageClass clone. At v1.66.1 validateAdvancedClonePVC runs for snapshot and csi-clone alike and checks for a common CSI driver, matching volume modes, the target class existing, and the requested size against allowVolumeExpansion. The snapshot-only extra check GetCompatibleVolumeSnapshotClass requires a common driver and a common snapshot class. Nothing compares StorageClass names, so with a LINSTOR source and a LINSTOR target the driver matches and no fallback to host-assisted copy fires on either path.
4. A local StorageClass (placement count 1, no remote volume access) can only materialize a clone on a node that already holds the source data. With WaitForFirstConsumer the prep pod is scheduled without regard to source-data placement, so LINSTOR CreateVolume cannot satisfy the request and times out.

Root cause

The csi-clone override is deliberate. 42778cf4b, merged in #2133, switched the strategy from snapshot to csi-clone in order to use LINSTOR's native rd-clone rather than pod-based copying. Its commit body mentions the relocateAfterClone parameter added in the same release, which reads like a prerequisite and is not one: per the parameter's author it is a workaround for replica concentration, keeping a hundred VMs cloned from one image from all ending up on the host that held the source PVC, and it stops being relevant in topologies where storage nodes are separate from compute. It has no bearing on whether a clone into a different StorageClass can complete.

So the root cause is narrower than a missing prerequisite. Forcing csi-clone globally commits every clone to a path with no handling for a target in a different StorageClass, and CDI offers no fallback there because neither advanced path compares class names. The cross-class case was never covered by the design, and the override removes the one input that could have varied the outcome, which is per-StorageProfile selection.

The override cannot currently be switched off

c47bb6edb made the value configurable, but the knob cannot express "no override". The CRD enum is copy, snapshot and csi-clone, with no unset value, and the template emits the field unconditionally. Rendering with an empty value produces cloneStrategyOverride: "", which the enum rejects; rendering with null produces a bare key. So an operator cannot reach CDI's per-profile selection through values at all, and the value they would naturally try to disable the override makes the CDI resource invalid.

Fix directions

1. Stop emitting cloneStrategyOverride when the configured value is empty, so per-StorageProfile selection becomes reachable at all. Today it is not, for the reason in the section above.
2. Decide whether csi-clone stays the default once it can be turned off. Same-class clones are the common case and csi-clone is the fast path for them, so the default is a separate question from whether the mechanics allow changing it.
3. Have vm-disk refuse or warn when the requested StorageClass differs from the golden images' one. No per-class attribute makes a cross-class clone safe, so the mismatch itself is the predicate. Today the failure is a PVC that stays unbound with nothing surfaced to the user, which is the worst part of this for whoever hits it.

Workaround

Create the disk in the same StorageClass the golden images live in (replicated in the default layout).