feat(platform): auto-wire KubeVirt permittedHostDevices + HostDevices feature gate when gpu-operator is enabled #2765

Summary

After enabling cozystack.gpu-operator with the default (passthrough) variant, the registered GPU resource (e.g. nvidia.com/GH100_H200_SXM_141GB) shows up in kubectl get node -o yaml | grep allocatable — but a KubeVirt VM that references it as a hostDevices.deviceName is rejected by the admission webhook.

Two pieces have to be wired into the KubeVirt CR manually for VFIO passthrough to actually be usable, and neither is automated by Cozystack today:

1. spec.configuration.developerConfiguration.featureGates must include HostDevices. The default Cozystack KubeVirt CR ships GPU but not HostDevices — and they are separate gates in current KubeVirt; the GPU gate alone does not unlock spec.template.spec.domain.devices.hostDevices.
2. spec.configuration.permittedHostDevices.pciHostDevices must list the GPU's PCI vendor:device and the resourceName published by nvidia-sandbox-device-plugin. Without this entry the admission webhook errors with "PCI host device not permitted".

Observed

Patch needed before any VM with the GPU can be created:

spec:
  configuration:
    developerConfiguration:
      featureGates:
      - HotplugVolumes
      - ExpandDisks
      - CPUManager
      - VMExport
      - HostDevices        # <-- added
    permittedHostDevices:
      pciHostDevices:
      - pciVendorSelector: "10DE:2335"
        resourceName: nvidia.com/GH100_H200_SXM_141GB
        externalResourceProvider: true

externalResourceProvider: true is required because the resource is published by nvidia-sandbox-device-plugin DaemonSet (deployed by gpu-operator), not by KubeVirt itself.

gpu-operator's ServiceAccount has no RBAC to patch the KubeVirt CR, so this cannot be done by the operator. Cozystack's cozystack-engine overwrites Secret cozystack-values entirely, so trying to inject this through the existing user-overrides path is also a dead end.

Proposal

When cozystack.gpu-operator is enabled, the platform should auto-populate the KubeVirt CR with:

• HostDevices added to developerConfiguration.featureGates
• permittedHostDevices.pciHostDevices entries built from the PCI device IDs the sandbox-device-plugin publishes (vendor:device discovered from the host's lspci -nn, resourceName matching the device-plugin's registered name)

Implementation options:

• A small controller in cozystack-engine that watches Node Allocatable for nvidia.com/* resources and reconciles the corresponding KubeVirt CR entries.
• Or: extend the system/kubevirt-operator chart with a values block that takes a list of (vendor:device, resourceName) pairs and templates them into the KubeVirt CR, then wire that from the Platform Package when gpu-operator is enabled.
• Or: a one-shot Job included in the gpu-operator package that patches the KubeVirt CR on install and on upgrade.

The first option is the cleanest but the heaviest. The third is the smallest delta and matches the way other Cozystack post-install adjustments are done.

Related

• The need for a container variant of cozystack.gpu-operator is a separate issue; this issue is specifically about the passthrough path. For the container variant the KubeVirt wiring would not be needed.
• Without this wiring the passthrough variant ends up "halfway working": the gpu-operator stack is green, the GPU resource is visible in kubectl describe node, but no VM can actually use it. That is the most confusing failure mode for a new user — every component reports healthy and the VM simply refuses to schedule with an opaque message.