You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
ci(nightly): Mirror images to GHCR fails on ghcr write_package, so the nightly e2e has not run since 2026-09-15 #4423
The nightly workflow has been red since 2026-09-15, and since then it stops at the "Mirror images to GHCR" job, so the full e2e suite has not run on main for nine nights. Before that, 2026-09-04 to 2026-09-14 failed in "E2E (full suite)".
The mirror step fails on a ghcr.io write, not on the source side. On 2026-09-15 to 2026-09-18 it was opensearch-operator, and from 2026-09-20 it is cluster-api-provider-kubevirt:
level=fatal msg="writing blob: initiating layer upload to /v2/cozystack/cozystack/cluster-api-provider-kubevirt/blobs/uploads/ in ghcr.io: denied: permission_denied: write_package"
2026-09-19 failed differently, with authenticating creds for ..., so that night has its own cause.
The package changing from one night to the next fits a per-package permission problem: the workflow token has no write access to a package that hack/nightly-mirror.sh pushes to, and once one package is fixed, the next one in the list fails. I have not checked the package settings on ghcr.io, so this is the likely explanation, not a confirmed one. If it holds, the fix is to give the repository's Actions write access on each first-party package under ghcr.io/cozystack/cozystack, and to make the mirror step list every package it can't write in one run instead of stopping at the first.
"Mirror images to GHCR" has passed three nights in a row: run 36659663201 (2026-09-30), 36806044588 (2026-10-01) and 36955308949 (2026-10-02). Nightly e2e runs again, so I'm closing this. The suite reds since then are tracked separately (#4653 for the LINSTOR baseline, #4412 for opensearch).
Activity
lexfrei commentedon Oct 2, 2026
"Mirror images to GHCR" has passed three nights in a row: run 36659663201 (2026-09-30), 36806044588 (2026-10-01) and 36955308949 (2026-10-02). Nightly e2e runs again, so I'm closing this. The suite reds since then are tracked separately (#4653 for the LINSTOR baseline, #4412 for opensearch).