Repository navigation
[Backport release-1.4] fix(csi): verify VMI Ready after kubevirt-csi Publish to surface stuck-PVC failures - #2748
Merged
Conversation
Upstream ControllerPublishVolume has a fast path that returns success
based only on the volume's presence in VM.spec.template.spec.volumes
(via EnsureVolumeAvailableVM). If a prior Publish wrote that entry but
the volume never became Ready in VMI status - e.g. the infra PVC is
stuck ClaimPending because LINSTOR cannot autoplace - the retry hits
the fast path and reports success. external-attacher marks the
VolumeAttachment Attached=true, but QEMU was never sent device_add, so
the tenant kubelet fails NodeStageVolume with:
rpc error: code = Unknown desc = couldn't find device by serial id
Wrap upstream Publish for hotplug volumes (non-NFS) with a post-call
check that the volume reached VolumeReady in VMI.Status.VolumeStatus.
When it has not, the RPC returns codes.Unavailable carrying the actual
reason (e.g. "PVC is in phase ClaimPending"), so external-attacher
keeps retrying and the underlying provisioning failure is visible at
the CSI layer rather than as a confusing "device not found by serial
id" on the tenant kubelet.
This is a wrapper-side workaround for an upstream defect in
kubevirt.io/csi-driver; the structural fix belongs upstream (tighten
the fast path to require VMI Ready) and will be filed separately.
Assisted-By: Claude <[email protected]>
Signed-off-by: Myasnikov Daniil <[email protected]>
(cherry picked from commit 5bb5e95)
…nnot run The fallback branches in publishHotplugVolume silently returned the upstream success response when the node ID failed to parse or the VMI get call returned an error. external-attacher then set Attached=true without VolumeReady proof, which recreates the false-positive attach this wrapper exists to prevent. Fail loudly instead: - parseErr -> codes.Internal: external-attacher is expected to provide a well-formed "namespace/name" node ID per the CSI contract; if it does not, surface the contract violation rather than mask it. - getErr -> codes.Unavailable: this is the same code returned for the not-Ready and not-present cases just below, so the retry behavior on the attacher side stays consistent. By the time we reach this branch the upstream Publish call already succeeded, which means it just looked the VMI up; a transient get error here is a race or API hiccup and a retry is safe. Also add a one-line comment on GetVirtualMachine: in the upstream kubevirt.io/csi-driver client this method returns a VirtualMachineInstance despite its name, so accessing vmi.Status.VolumeStatus on the result is type-correct. Assisted-By: Claude <[email protected]> Signed-off-by: Myasnikov Daniil <[email protected]> (cherry picked from commit cbe4b86)
dosubot
Bot
added
area/storage
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Backport of #2659 to
release-1.4.