You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
linstor-scheduler-admission (v0.3.2) strips unknown pod fields cluster-wide → breaks image volumes / KubeVirt containerDisk and native sidecars #3198
Cozystack ships linstor-scheduler-extender v0.3.2 (packages/system/linstor-scheduler, appVersion: v0.3.2). Its linstor-scheduler-admission mutating webhook is installed with an empty namespaceSelector and objectSelector, so it intercepts every pod CREATE cluster-wide. The v0.3.2 binary vendors k8s.io/api v0.25.6 (Kubernetes 1.25) and round-trips each pod through the typed corev1.Pod, so any pod field newer than 1.25 is silently dropped at admission — even for pods with no LINSTOR volumes and even when no schedulerName change is made.
Full mechanism and reproducer are in the upstream issue linked below.
Impact on Cozystack
Breaks KubeVirt containerDisk where the ImageVolume path is used (non-Talos / k3s bundles). The image VolumeSource (added in k8s.io/api v0.31.0 / Kubernetes 1.31) is rewritten to emptyDir: {} at admission, so the KubeVirt launcher's image-volume mount is empty and the VM fails to init. This is the actual root cause behind Bumping platform KubeVirt to ≥1.9 will break containerDisk VMs on non-Talos (k3s) bundles (default-on ImageVolume) #3190: the strip happens at the admission layer, before containerd, and persists even on containerd >= 2.3.2 — so it is not (only) a container-runtime CRI-support gap. On affected clusters the practical workaround today is disabledFeatureGates: [ImageVolume] in the KubeVirt CR, but that only sidesteps the symptom.
Strips restartPolicy: Always from init containers cluster-wide, breaking Kubernetes native sidecars (GA in 1.33) for any tenant workload — not just KubeVirt.
Root cause / upstream fix
The root cause is upstream in linstor-scheduler-extender. The fix is upstream PR piraeusdatastore/linstor-scheduler-extender#19 (switches the webhook to targeted RFC 6902 JSON patches so unknown fields are never touched); it is approved but stalled on a DCO sign-off. Upstream tracking issue with the image VolumeSource repro: piraeusdatastore/piraeus-operator#1025.
Defense-in-depth: add a namespaceSelector/objectSelector to the chart's mutatingwebhookconfiguration.yaml to shrink the blast radius. This does not fully fix it (LINSTOR-backed pods still need the mutation), so it is a mitigation, not a replacement for the upstream fix.
What happened
Cozystack ships
linstor-scheduler-extenderv0.3.2 (packages/system/linstor-scheduler,appVersion: v0.3.2). Itslinstor-scheduler-admissionmutating webhook is installed with an emptynamespaceSelectorandobjectSelector, so it intercepts every pod CREATE cluster-wide. The v0.3.2 binary vendorsk8s.io/api v0.25.6(Kubernetes 1.25) and round-trips each pod through the typedcorev1.Pod, so any pod field newer than 1.25 is silently dropped at admission — even for pods with no LINSTOR volumes and even when noschedulerNamechange is made.Full mechanism and reproducer are in the upstream issue linked below.
Impact on Cozystack
Breaks KubeVirt
containerDiskwhere theImageVolumepath is used (non-Talos / k3s bundles). TheimageVolumeSource (added ink8s.io/api v0.31.0/ Kubernetes 1.31) is rewritten toemptyDir: {}at admission, so the KubeVirt launcher's image-volume mount is empty and the VM fails to init. This is the actual root cause behind Bumping platform KubeVirt to ≥1.9 will break containerDisk VMs on non-Talos (k3s) bundles (default-on ImageVolume) #3190: the strip happens at the admission layer, before containerd, and persists even on containerd >= 2.3.2 — so it is not (only) a container-runtime CRI-support gap. On affected clusters the practical workaround today isdisabledFeatureGates: [ImageVolume]in the KubeVirt CR, but that only sidesteps the symptom.Strips
restartPolicy: Alwaysfrom init containers cluster-wide, breaking Kubernetes native sidecars (GA in 1.33) for any tenant workload — not just KubeVirt.Root cause / upstream fix
The root cause is upstream in
linstor-scheduler-extender. The fix is upstream PR piraeusdatastore/linstor-scheduler-extender#19 (switches the webhook to targeted RFC 6902 JSON patches so unknown fields are never touched); it is approved but stalled on a DCO sign-off. Upstream tracking issue with theimageVolumeSource repro: piraeusdatastore/piraeus-operator#1025.Options for Cozystack
appVersiononce upstream releases the fix. Note: the current latest upstream tag (v0.3.4) still vendorsk8s.io/api v0.25.6, so a plain tag bump does not fix this — it must be a release that contains the fix: use JSON patches to preserve unknown pod fields piraeusdatastore/linstor-scheduler-extender#19 fix.linstor-scheduler-extenderimage (Cozystack already rebuilds several piraeus images) cherry-picking fix: use JSON patches to preserve unknown pod fields piraeusdatastore/linstor-scheduler-extender#19.namespaceSelector/objectSelectorto the chart'smutatingwebhookconfiguration.yamlto shrink the blast radius. This does not fully fix it (LINSTOR-backed pods still need the mutation), so it is a mitigation, not a replacement for the upstream fix.Related: #3190.