Repository navigation
fix(ci): resolve nightly packages digest without image-specific skopeo ops - #3332
Conversation
…o ops
The Nightly workflow has never completed a run. Every scheduled run since
NIGHTLY_ENABLED was set has failed in `mirror` at "Resolve source commit
and version", ~80s in:
level=fatal msg="unsupported image-specific operation on artifact with
type \"application/vnd.cncf.flux.config.v1+json\""
cozystack-packages is a Flux OCI artifact, not a container image: its
config mediaType is application/vnd.cncf.flux.config.v1+json. Plain
`skopeo inspect` parses the config blob as an image config and fatals on
any non-image artifact, so resolving the digest with
`inspect --format '{{.Digest}}'` could never work.
Fetch the :main manifest once with `inspect --raw` and derive both the
digest (`skopeo manifest-digest`, which just hashes the manifest bytes —
no image-specific parsing) and the revision annotation from those exact
bytes. This also tightens the race the original comment describes: the
digest and the annotation now come from a single fetch, so there is no
window at all between pinning the tag and reading its revision, and one
registry round-trip is saved.
Because the workflow never ran past this step, the rest was unproven.
Also fix both `flux pull artifact` calls, which passed an --output
directory that does not exist and would have failed with "invalid output
path": create it first, matching the established idiom in
pull-requests.yaml ("Pull current-main packages tree").
Verified against the live OCIR artifact: the fixed step resolves
packages_ref to the same digest the registry reports and yields main's
current HEAD sha; the fixed pull extracts the tree. skopeo copy on Flux
artifacts, the flux push digest awk parse, the cozy-installer chart name
and nightly-mirror.sh's per-image digest verification were each checked
and are correct as-is.
Signed-off-by: Myasnikov Daniil <[email protected]>
|
Note Gemini is unable to generate a summary for this pull request due to the file types involved not being currently supported. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThe nightly workflow now reads OCI package metadata from a single manifest and creates required output directories before pulling baked and E2E package trees. ChangesNightly artifact flow
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
A second internal pass found that the previous commit's disclosure argument did not hold together and that its support test could not be evaluated by a reader. The document had nominated the monthly aggregate summaries as safe-to-publish evidence on the grounds that they hold no vulnerability detail. They do: they name accepted-risk findings individually by CVE ID, severity and package, some of them critical, next to a count of confirmed findings awaiting a fix. State the line where it actually falls — aggregate counts by severity are publishable, naming an individual unfixed finding is not, because a CVE ID plus a package plus this project's public pinned component list locates unpatched exposure — and note that the monthly summary needs reducing to counts before it can be published. Present the resulting posture as a proposal rather than as a position the maintainers have taken. The scanner SLA paragraph said a triage backlog was running against the clock. The pipeline's own output says something more specific and worse: triage is keeping up, remediation is not, with the last monthly period closing on no confirmed findings fixed. Say that instead. The supported-versions table keyed support to having an active release branch. Release branches are never deleted, so every line back to the earliest has one and the test tells a reader nothing; read literally it also committed the project to every 1.x line. Key it to patch activity within a stated window instead, restore an end-of-life category the previous commit had removed entirely, and put the v0.41 cutoff back rather than extending discretionary backports across all of 0.x. Also: drop an unevidenced claim that the per-PR checks are healthy, correct the nightly attribution — the current red run is a regression after #3332 and is not tracked by an issue, unlike the older failure in #3278 — and qualify how zizmor gates, through the required pre-commit hook rather than through its own path-filtered workflow. Assisted-By: Claude <[email protected]> Signed-off-by: Timur Tukaev <[email protected]>
…timeline The previous commit narrowed a true statement into a false one. It said the scanner gap was remediation throughput and that triage ran on schedule; triage is also behind, and the backlog awaiting a triage decision includes findings at critical severity well past the one-business-day target. Both tracks are now stated as behind. Figures stay out of this repository for the disclosure reason the document already gives and belong in the private channel, so the admission is qualitative rather than rounded down. The nightly characterisation was also wrong. It read as fixed-then-regressed, which understates it: after the fix in #3332 the scheduled run passed for four days, went red for five, passed for three, and has failed every run since 2026-08-02 — seven of the last twenty-four passed. It is flapping, not newly broken, and the current failure is untriaged rather than provably unfiled. Carry the target-attainment caveat into SECURITY.md, which stated the triage SLAs with no indication they are not being met, and resync the self-assessment on the monthly report: it names findings individually and so is not publishable until reduced to counts by severity. Also note in SECURITY.md what the activity test actually classifies today — every 1.x line from 1.0 to 1.6 — and how backport effort is prioritised across that set, since supporting seven lines in parallel otherwise invites the question it does not answer. Assisted-By: Claude <[email protected]> Signed-off-by: Timur Tukaev <[email protected]>
A second internal pass found that the previous commit's disclosure argument did not hold together and that its support test could not be evaluated by a reader. The document had nominated the monthly aggregate summaries as safe-to-publish evidence on the grounds that they hold no vulnerability detail. They do: they name accepted-risk findings individually by CVE ID, severity and package, some of them critical, next to a count of confirmed findings awaiting a fix. State the line where it actually falls — aggregate counts by severity are publishable, naming an individual unfixed finding is not, because a CVE ID plus a package plus this project's public pinned component list locates unpatched exposure — and note that the monthly summary needs reducing to counts before it can be published. Present the resulting posture as a proposal rather than as a position the maintainers have taken. The scanner SLA paragraph said a triage backlog was running against the clock. The pipeline's own output says something more specific and worse: triage is keeping up, remediation is not, with the last monthly period closing on no confirmed findings fixed. Say that instead. The supported-versions table keyed support to having an active release branch. Release branches are never deleted, so every line back to the earliest has one and the test tells a reader nothing; read literally it also committed the project to every 1.x line. Key it to patch activity within a stated window instead, restore an end-of-life category the previous commit had removed entirely, and put the v0.41 cutoff back rather than extending discretionary backports across all of 0.x. Also: drop an unevidenced claim that the per-PR checks are healthy, correct the nightly attribution — the current red run is a regression after #3332 and is not tracked by an issue, unlike the older failure in #3278 — and qualify how zizmor gates, through the required pre-commit hook rather than through its own path-filtered workflow. Assisted-by: LLM Signed-off-by: Timur Tukaev <[email protected]>
…timeline The previous commit narrowed a true statement into a false one. It said the scanner gap was remediation throughput and that triage ran on schedule; triage is also behind, and the backlog awaiting a triage decision includes findings at critical severity well past the one-business-day target. Both tracks are now stated as behind. Figures stay out of this repository for the disclosure reason the document already gives and belong in the private channel, so the admission is qualitative rather than rounded down. The nightly characterisation was also wrong. It read as fixed-then-regressed, which understates it: after the fix in #3332 the scheduled run passed for four days, went red for five, passed for three, and has failed every run since 2026-08-02 — seven of the last twenty-four passed. It is flapping, not newly broken, and the current failure is untriaged rather than provably unfiled. Carry the target-attainment caveat into SECURITY.md, which stated the triage SLAs with no indication they are not being met, and resync the self-assessment on the monthly report: it names findings individually and so is not publishable until reduced to counts by severity. Also note in SECURITY.md what the activity test actually classifies today — every 1.x line from 1.0 to 1.6 — and how backport effort is prioritised across that set, since supporting seven lines in parallel otherwise invites the question it does not answer. Assisted-by: LLM Signed-off-by: Timur Tukaev <[email protected]>
What this PR does
The Nightly workflow has never completed a run. Every scheduled run since
NIGHTLY_ENABLEDwas set has failed inmirrorat "Resolve source commit and version", ~80s in, with:Nothing has ever been published —
cozy-installer:nightlyandcozystack-packages:nightlyboth 404 on GHCR, andcozystack-noclouddoes not exist.This fixes that failure and the next two behind it.
1. The digest lookup could never have worked
cozystack-packagesis a Flux OCI artifact (config mediaTypeapplication/vnd.cncf.flux.config.v1+json), not a container image. A plainskopeo inspectparses the config blob as an image config and fatals on it, soskopeo inspect --format '{{.Digest}}'was never viable against this ref — the--rawcall on the very next line already acknowledged this.Replaced with
skopeo inspect --raw+skopeo manifest-digest, which hashes the manifest bytes — exactly what the registry addresses the artifact by, with no image-specific parsing.This also collapses two registry fetches into one: the digest and the
org.opencontainers.image.revisionannotation now come from the same bytes, which strengthens the race the original comment describes — there is no longer any window between pinning the tag and reading it.2.
flux pull artifactrejects a non-existent output path (×2)flux pull artifact --output _bakedfails withinvalid output path "_baked": no such file or directory._bakedis neither tracked nor created by anything.pull-requests.yamldoesrm -rf+mkdir -pfirst;nightly.yamlomitted it. Since the workflow always died at step 1, this was never reached.Same bug for
_rwin the e2e staging step. Both now create the directory first.Verification
shathat matches the currentmainHEAD._bakedfailure and confirmed the fix extracts the tree.actionlintclean;zizmor --offlineclean;shellcheckclean on changed lines.Also checked and not changed, each disproven rather than assumed:
skopeo copy --multi-arch allon a Flux artifact — works;copyis mediaType-agnostic.awk -F@ '/artifact successfully pushed/'digest capture — matches current flux CLI output, and is the same idiompackages/core/installer/Makefilealready uses in production.nightly-mirror.sh'sinspect --format Digest— all mirrored refs there are genuine image manifests, so it is correct as written.make -C packages/core/talos talos-nocloud— exists and is CI-proven viatags.yaml.Honest scope
This fixes the
mirrorjob's blockers. It does not promise a green nightly on the first run: every step aftermirroris still unproven, having never executed. The most likely next wall isbuild-disk—cozystack-nocloudis a brand-new package, so the firstoras pushcreates it, and the follow-uporas tagcan hit403 permission_denieduntil the package is linked (the risk the workflow already documents in-line). The first-ever full e2e run may well surface its own issues.Reviewing that follow-on separately seems better than growing this PR.
Summary by CodeRabbit