Repository navigation
ci(build): route PR base-image pulls through mirror.gcr.io - #3042
Conversation
The Pull Request Build job authenticates only to the push registry (OCIR), so every base image (golang, ubuntu, alpine, nginx, ...) is pulled from Docker Hub anonymously. Those pulls intermittently fail at auth.docker.io/token under Docker Hub's anonymous-pull rate limit, leaving the base image unresolved and failing make build. Build on a docker-container buildx builder whose buildkitd routes docker.io through mirror.gcr.io, Google's public pull-through cache for Docker Hub: no auth, far higher limits, transparent fall-through to Docker Hub on a miss. This reuses the builder pattern already proven in build-main.yaml (same make build, same DOCKER_CONFIG auth forwarding); only the committed hack/buildkitd.toml mirror config is new. Fail-safe: if mirror.gcr.io is unreachable, BuildKit falls back to registry-1.docker.io and the build still succeeds, so this strictly reduces build failures without adding a new hard dependency. Assisted-By: Claude <[email protected]> Signed-off-by: Aleksei Sviridkin <[email protected]>
github-actions
Bot
added
area/build
📝 WalkthroughWalkthroughThe pull-request workflow now creates a docker-container buildx builder using ChangesCI buildx and BuildKit mirror
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
dosubot
Bot
added
the
area/ci
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request improves CI build stability by routing base image pulls through Google's mirror.gcr.io. This mitigates intermittent build failures caused by Docker Hub's anonymous pull rate limits without introducing new hard dependencies, as the configuration includes a fail-safe fallback to the default registry. Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Ignored Files
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on Gemini (@gemini-code-assist) comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds a BuildKit daemon configuration (hack/buildkitd.toml) to route Docker Hub base-image pulls through Google's public mirror (mirror.gcr.io) to avoid anonymous-pull rate limits in CI. The review feedback suggests updating the configuration comments to clarify that only official images are cached by this mirror, with others falling back to Docker Hub. Additionally, the commit scope in the PR title should be updated to comply with the repository's style guide.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| # Route docker.io base-image pulls through mirror.gcr.io, Google's public | ||
| # pull-through cache for Docker Hub. The Build job authenticates only to the | ||
| # push registry (OCIR), so every base image (golang, alpine, ubuntu, nginx, ...) | ||
| # is otherwise pulled from Docker Hub ANONYMOUSLY and is subject to Docker Hub's | ||
| # anonymous-pull rate limit -- which intermittently fails the build at | ||
| # auth.docker.io/token with the base image left unresolved. |
There was a problem hiding this comment.
Note that mirror.gcr.io only mirrors official Docker Hub images (those in the library/ namespace, such as ubuntu, golang, alpine, nginx, etc.). Non-official/third-party images (such as grafana/grafana, nginxinc/nginx-unprivileged, clickhouse/*, and altinity/*) are not cached by mirror.gcr.io. For these images, BuildKit will transparently fall back to registry-1.docker.io, meaning they will still be pulled anonymously and remain subject to Docker Hub's rate limits.
It is recommended to update the comment in this file to reflect this limitation so future maintainers aren't confused if rate limit issues persist for non-official images.
| # Route docker.io base-image pulls through mirror.gcr.io, Google's public | |
| # pull-through cache for Docker Hub. The Build job authenticates only to the | |
| # push registry (OCIR), so every base image (golang, alpine, ubuntu, nginx, ...) | |
| # is otherwise pulled from Docker Hub ANONYMOUSLY and is subject to Docker Hub's | |
| # anonymous-pull rate limit -- which intermittently fails the build at | |
| # auth.docker.io/token with the base image left unresolved. | |
| # Route docker.io base-image pulls through mirror.gcr.io, Google's public | |
| # pull-through cache for Docker Hub. Note that mirror.gcr.io only caches | |
| # official images (the library/ namespace, e.g., golang, alpine, ubuntu, nginx). | |
| # Non-official images (e.g., grafana/grafana, clickhouse/*) are not cached and | |
| # will fall back to registry-1.docker.io, remaining subject to rate limits. |
| @@ -0,0 +1,21 @@ | |||
| # BuildKit daemon config for CI builds. | |||
There was a problem hiding this comment.
According to the repository style guide, the commit scope build in the PR title ci(build): ... is not a valid scope. Valid scopes under Meta are api, hack, tests, ci, and docs. Please update the PR title and commit message to use a valid scope, such as ci(ci) or ci(hack).
References
- Each commit must follow Conventional Commits format with a valid scope from the predefined list (e.g., ci, hack). (link)
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (1)
hack/buildkitd.toml (1)
20-21: 🚀 Performance & Scalability | 🔵 TrivialOptional: add
debug = trueto verify mirror usage in logs.Setting
debug = trueinbuildkitd.tomlexposes BuildKit daemon logs that explicitly confirm whenmirror.gcr.iois used (e.g., via specific response headers). This is helpful for troubleshooting to ensure the registry fallback isn't being triggered unexpectedly.🔍 Optional observability tweak
+debug = true + [registry."docker.io"] mirrors = ["mirror.gcr.io"]🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@hack/buildkitd.toml` around lines 20 - 21, Add the optional BuildKit daemon debug setting to the BuildKit config so mirror usage can be confirmed in logs; update the existing buildkitd.toml registry configuration near the [registry."docker.io"] block to include debug = true alongside the mirror setting, keeping the change scoped to this BuildKit daemon config.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@hack/buildkitd.toml`:
- Around line 20-21: Add the optional BuildKit daemon debug setting to the
BuildKit config so mirror usage can be confirmed in logs; update the existing
buildkitd.toml registry configuration near the [registry."docker.io"] block to
include debug = true alongside the mirror setting, keeping the change scoped to
this BuildKit daemon config.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: b9876708-2b1e-451c-ae41-d0e6912887e3
📒 Files selected for processing (2)
.github/workflows/pull-requests.yamlhack/buildkitd.toml
IvanHunters
left a comment
IvanHunters
left a comment
There was a problem hiding this comment.
Reviewed the diff. Scope cleanly limited to pull-requests.yaml + hack/buildkitd.toml, mirror only for docker.io, BuildKit falls back to upstream on miss. Final image digests unaffected (content-addressed, mirror serves bit-identical layers), so downstream consumers see identical manifests. Other registries (quay.io, ghcr.io, gcr.io) and private images are not touched. LGTM.
Resolve conflict in .github/workflows/pull-requests.yaml: keep the new per-package build matrix and fold in main's docker.io->mirror.gcr.io base-image routing (#3042). Apply buildkitd-config: hack/buildkitd.toml to all three new build legs (build, build-talos, finalize) so the mirror fix covers the whole decomposed pipeline, not just the conflicted leg. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Signed-off-by: Myasnikov Daniil <[email protected]>
2 participants
Problem
The Pull Request Build job authenticates only to the push registry (OCIR), so every base image (golang, ubuntu, alpine, nginx, …) is pulled from Docker Hub anonymously. Those pulls intermittently fail at
auth.docker.io/tokenunder Docker Hub's anonymous-pull rate limit, leaving the base image unresolved and failingmake build— observed across several maintainer-branch runs in a short window.Fix
Build the PR job on a docker-container buildx builder whose buildkitd routes
docker.iothroughmirror.gcr.io(Google's public pull-through cache for Docker Hub: no auth, far higher limits, transparent fall-through to Docker Hub on a miss). This reuses the builder pattern already proven inbuild-main.yaml(samemake build, sameDOCKER_CONFIGauth forwarding); only the committedhack/buildkitd.tomlmirror config is new. No base image versions or tags change.Risk
Switches the PR build from the default embedded
dockerdriver to a docker-container builder.build-main.yamlalready runs the identicalmake buildon that driver, so the path is exercised. The "Build Talos image" step usesdocker run/skopeo(not buildx) and is unaffected.Fallback
If
mirror.gcr.iois unreachable, BuildKit logstrying next hostand falls back toregistry-1.docker.io, so the build still succeeds. Verified locally: a build pointed at a non-resolving mirror fell back to Docker Hub and exited 0 (at a small latency cost from the failed lookups). This strictly reduces build failures without adding a new hard dependency.Coverage
Covered (the actual failure class — Docker Hub bases):
ubuntu,golang*,alpine*,debian*,grafana/grafana,nginxinc/nginx-unprivileged,clickhouse/*,altinity/*, and explicitdocker.io/library/*. Not covered, and never the problem:quay.io/*,gcr.io/distroless/static,ghcr.io/siderolabs/imager— the mirror is keyed todocker.ioonly, by design, and these were never subject to Docker Hub limits.Optional follow-up (not in this PR)
The
moby/buildkitbuilder image is still pulled from Docker Hub by the daemon to bootstrap the container builder (the mirror lives inside buildkit, so it cannot mirror its own bootstrap). That is one small pinned image versus dozens of base images, so this change is a strict net reduction in anonymous Docker Hub traffic. Pinning the bootstrap image through a mirror is deliberately left out: a no-fallback mirror on the bootstrap path would add risk for little gain.Summary by CodeRabbit