Repository navigation
fix(kubernetes): move kubevirt-csi-driver base off quay.io/centos to almalinux - #3400
Conversation
…almalinux quay.io serves only currently-tagged manifests: when a rolling tag moves, the superseded digest immediately returns MANIFEST_UNKNOWN. The pinned `centos:stream9` digest has died twice in six days (#3303, #3386), each time breaking `make build` on main and on every PR that triggers a full rebuild. Repinning is not a fix -- the newly pinned digest is always the live tag, so it is next in line to be retired. Move the runtime stage to a registry that provably retains superseded manifests instead, which makes the digest pin durable rather than decorative. Pinning policy is unchanged. Verified equivalent: after `dnf install -y e2fsprogs xfsprogs nfs-utils`, both bases provide mkfs.ext4, mkfs.xfs, blkid, mount, umount, fsck.ext4, fsck.xfs and mount.nfs at identical paths. e2fsprogs and xfsprogs are identical NVRs. The driver is built CGO_ENABLED=0, so the base libc is not linked. The pinned digest is the OCI index covering amd64, arm64, ppc64le and s390x -- the same architecture set as the previous base. Assisted-By: Claude <[email protected]> Signed-off-by: Myasnikov Daniil <[email protected]>
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request resolves persistent build failures in the kubevirt-csi-driver by switching the base image to a registry that retains historical manifests. The previous dependency on Quay.io caused frequent build breaks due to the rapid rotation of rolling tags and the immediate removal of superseded digests. AlmaLinux 9 provides an equivalent RHEL 9-based environment while ensuring that pinned image digests remain valid over time. Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on Gemini (@gemini-code-assist) comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThe kubevirt CSI driver Dockerfile now uses a pinned AlmaLinux 9 image for its final runtime stage. Package installation, binary copying, and the entrypoint remain unchanged. ChangesKubevirt CSI driver runtime
Estimated code review effort: 1 (Trivial) | ~2 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 Checkov (3.3.8)packages/apps/kubernetes/images/kubevirt-csi-driver/DockerfileTraceback (most recent call last): Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
dosubot
Bot
added
area/build
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the base image for the kubevirt-csi-driver Dockerfile, switching from CentOS Stream 9 to AlmaLinux 9. There are no review comments to assess, and I have no feedback to provide.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
github-actions
Bot
added
the
size/XS
Andrey Kolkov (androndo)
left a comment
Andrey Kolkov (androndo)
left a comment
There was a problem hiding this comment.
LGTM
This is a one-line runtime-base swap in packages/apps/kubernetes/images/kubevirt-csi-driver/Dockerfile, from quay.io/centos/centos:stream9@sha256:3714c89f… to docker.io/library/almalinux:9@sha256:d2515c76…. The PR is unusually well-argued, and I was able to independently confirm every load-bearing claim.
Verified against the live registry and repo:
- Digest validity + architecture parity. The pinned digest
d2515c76…resolves to anapplication/vnd.oci.image.index.v1+jsonindex covering exactlylinux/amd64,linux/arm64,linux/ppc64le,linux/s390x— the same architecture set as the CentOS base it replaces. Claim confirmed. - The pin is the current live tag.
HEADondocker.io/library/almalinux:9returnsdocker-content-digest: sha256:d2515c76…, matching the pinned digest exactly. This is expected and correct: the whole premise is that Docker Hub retains superseded digests when the tag rolls, unlike Quay, so the pin stays fetchable after the tag moves. - Renovate policy genuinely unchanged.
.github/renovate.jsonpins dockerfile bases viapinDigests: truewith the human-readable tag kept in front of the digest (rule at lines 63–67), and batches digest refreshes into the weekly group (69–74). The newalmalinux:9@sha256:…line matches that format precisely, so it will be tracked and refreshed exactly like the existinggolang/alpine/debianbases. No policy change smuggled in. - Registry consistency. The builder stage in this same Dockerfile already pulls
docker.io/library/golang, so moving the runtime stage todocker.io/libraryaligns it with existing practice rather than introducing a new registry dependency. - No documentation drift. The only
centos/stream9references anywhere in the tree outside this Dockerfile are indocs/changelogs/v1.3.0*.md, and those are about VM images (vm-default-images), entirely unrelated to the CSI driver base. Nothing stale to fix.
Functional equivalence rests on the PR's local verification tables (mkfs/fsck/mount/nfs binary paths and e2fsprogs/xfsprogs NVRs identical; driver built CGO_ENABLED=0 so the base libc is never linked). I could not re-run dnf install here, but the reasoning is sound: almalinux:9 is a full RHEL-9 userspace base (BaseOS+AppStream, dnf present), the install line and RUN/COPY/ENTRYPOINT are byte-for-byte unchanged, and the static-binary + shell-out-to-mount-utils design means only those tool paths matter.
On tests: the real contract here (the image builds and ships working mkfs/fsck/mount/mount.nfs) is exercised by make build and the kubernetes-app e2e path, not by any unit test — and this repo has no per-Dockerfile test to extend. A bespoke test asserting the base-image string would just grep the Dockerfile, which verifies nothing about the running driver. So I am not blocking on a missing test; the existing build/e2e flow is the correct guard.
One consideration, non-blocking: this shifts another base pull onto Docker Hub, which enforces anonymous pull rate limits that Quay did not. In practice this is not a new regression — CI already pulls golang/alpine/debian from docker.io/library, the digest is BuildKit-cached, and if anything Quay was the outlier. Worth being aware of only if CI later starts hitting Hub 429s across the board.
The PR's own "Scope" section is honest that this does not solve the broader #3305 problem (any retired digest still aborts the serial platform build with no warning); that is correctly left out of scope.
2 participants
What this PR does
packages/apps/kubernetes/images/kubevirt-csi-driver/Dockerfilepinned its runtime stage toquay.io/centos/centos:stream9by digest. Quay serves only currently-tagged manifests — when a rolling tag moves, the superseded digest immediately returnsMANIFEST_UNKNOWN, with no grace period.stream9rolls roughly every six days, so the pin has already died twice:0c9db821…→a93e3316…0c9db821gonea93e3316…→3714c89f…a93e3316goneEach death broke
make buildonmainand on every PR triggering a full rebuild. Because the rootbuildtarget is serial, a failure inpackages/apps/kubernetes(4th of 29) aborts the remaining packages and leaves the shared build cache unwritten.Repinning cannot fix this: the freshly pinned digest is always the current live tag, so it is next in line to be retired. This is not specific to CentOS — the same probe against
quay.io/fedora/fedora:latestandquay.io/centos/centos:stream10found every historical digest unfetchable.This PR keeps the digest pin and changes the source to a registry that provably retains superseded manifests, making the pin durable rather than decorative. Renovate policy is unchanged —
pinDigestsstays on, andalmalinux:9is an ordinary rolling tag on a retaining registry exactly like thegolang,alpineanddebianbases already in the grouped digest PR.Verification
Both bases were run locally with the Dockerfile's exact install line (
dnf install -y e2fsprogs xfsprogs nfs-utils):mkfs.ext4/usr/sbin/mkfs.ext4/usr/sbin/mkfs.ext4mkfs.xfs/usr/sbin/mkfs.xfs/usr/sbin/mkfs.xfsblkid/usr/sbin/blkid/usr/sbin/blkidmount/usr/bin/mount/usr/bin/mountumount/usr/bin/umount/usr/bin/umountfsck.ext4/usr/sbin/fsck.ext4/usr/sbin/fsck.ext4fsck.xfs/usr/sbin/fsck.xfs/usr/sbin/fsck.xfsmount.nfs/usr/sbin/mount.nfs/usr/sbin/mount.nfsEvery path
k8s.io/mount-utilsshells out to is identical. Package versions:e2fsprogsxfsprogsnfs-utilsutil-linuxglibcSame upstream versions throughout; the el9 release-number lag is expected since AlmaLinux is downstream of RHEL 9 while CentOS Stream 9 is upstream.
e2fsprogsandxfsprogs— the two that governmkfs/fsckbehaviour — are identical NVRs.glibcis not linked at all: the driver is builtCGO_ENABLED=0.The pinned digest is the OCI image index covering
amd64,arm64,ppc64leands390x— the same architecture set as the previous base.Retention was verified empirically rather than assumed: four superseded
docker.io/librarydigests taken from this repo's own git history (golang:1.26,golang:1.26-alpine,alpine:3.24,alpine:3.22) all still resolve after their tags moved.Scope
This addresses the immediate recurring breakage only. The broader concern in #3305 — that any retired digest blocks the entire platform build, with no advance warning and no fast path to repin — is not solved here and remains open.
Release note
Summary by CodeRabbit