Repository navigation
[Epic] Cozystack on ARM #903
Description
Cozystack should be able to run on ARM hardware.
Steps:
- ARM images available, Cross arch build for images #519
- CI runs on dedicated ARM runners #906
Metadata
Metadata
Assignees
Labels
Type
Projects
- Status
Activity
[-]Cozystack on ARM[/-][+][Epic] Cozystack on ARM[/+]dosubot commentedon Oct 18, 2025
Hi, Nikita (@nbykov0). I'm Dosu, and I'm helping the cozystack team manage their backlog and am marking this issue as stale.
Issue Summary:
Next Steps:
Thank you for your understanding and contribution!
nbykov0 commentedon Oct 20, 2025
Work is currently on hold but this feature is not abandoned.
dosubot commentedon Jan 24, 2026
Hi, Nikita (@nbykov0). I'm Dosu, and I'm helping the cozystack team manage their backlog and am marking this issue as stale.
Issue Summary:
Next Steps:
Thank you for your understanding and contribution!
kvaps commentedon May 1, 2026
From my point of view the only limitation we have right now is that we need to solve multi-arch builds and tests, which might make our already complex pipeline even more complex and fragile.
The solutions:
Yes, we just need to find the solution for the issues mentioned above.
We have everything prepared, ARM workers sponsored by CNCF. All we need to do is find answers to the questions of how to build and test it without slowing down our existing development for amd64.
ajaykumarmandapati commentedon Sep 18, 2026
Adding a data point from the local-development angle, which I think is the one part of this epic that #1961 doesn't already cover.
I built and ran Cozystack v1.6.2 (
bfde94f,isp-full-generic) on a single Apple Silicon machine — k3d in a Colima VM, M4, 8 CPU / 20 GB. It converged: 98/98 HelmReleases, 74/74 Packages, 109 pods, and a SeaweedFS bucket created through the dashboard provisioned end to end via COSI with a verified S3 PUT/LIST/GET round trip. LINSTOR was excluded (DRBD needs an out-of-tree module the VM doesn't have), andcore/taloswas skipped.This mostly corroborates Aleksei Sviridkin (@lexfrei)'s result in #1961 on a smaller and less representative host, so I'm not claiming it as new. Three things did come out of it that I haven't seen recorded elsewhere.
1.
ciliumshould be rebuilt, not switched to upstreamThe table in #1961 lists cilium under "previously republished without patches". In v1.6.2 that isn't the case —
packages/system/cilium/images/cilium/DockerfilebuildsFROM quay.io/cilium/cilium:v1.19.5and then appliespatches/pass-ipv6-to-stack-when-ipv6-datapath-disabled.difftobpf/bpf_host.c.With
ENABLE_HOST_FIREWALLset butENABLE_IPV6not compiled in, the protocol-switch default drops all IPv6 asDROP_UNKNOWN_L3before policy evaluation, in both directions. That breaks ICMPv6 Neighbor Discovery and BGP unnumbered over link-local addresses on IPv6/L3 fabrics, and noCiliumClusterwideNetworkPolicycan allow the traffic back. Upstream report is cilium/cilium#33155 — closed stale, not fixed.The build already guards it (
patch --fuzz=0, then asserts exactly 3cozystack-ipv6-passthroughmarkers), so upstream drift fails loudly. But a swap to the upstream image bypasses the Dockerfile entirely and would drop the patch silently. It rebuilds for arm64 in ~35s with the patch intact, so rebuilding costs little here. Flagging in case that list was drawn against an earlier version.2. A class of image that no build-pipeline change can fix
Enabling ingress on a tenant surfaced this on arm64:
The controller image beside it (
ghcr.io/cozystack/ingress-nginx-with-protobuf-exporter/controller) has arm64 and ran fine. Theprotobuf-exportersidecar, referenced atpackages/system/ingress-nginx/values.yaml:19, is amd64-only at both the tag and the digest, lives in a different namespace from the rest of the platform images, and has no in-tree Dockerfile.So it can't be fixed by
PLATFORM, by a separate ARM pipeline, or by moving apps out-of-tree — it needs republishing multi-arch, vendoring into the repo, or dropping. It's a metrics sidecar and not in the request path, so dropping it was viable for my purposes, at the cost of ingress protobuf metrics.Might be worth an audit pass for other image references outside
ghcr.io/cozystack/*before committing to a pipeline shape, since this class is invisible to any amount of build-system work.3. Three hardcoded
--platform=linux/amd64that bypassBUILDX_ARGSpackages/system/dashboard/Makefile:30and:48packages/system/metallb/Makefile:20packages/system/kubeovn-plunger/Makefile:8callsdocker buildx buildwithoutBUILDX_ARGSat all.These are bespoke buildx invocations that don't inherit the
PLATFORMplumbing from #519, so they silently produce amd64 even when the rest of the build is told otherwise.On the pipeline question in your 1 May comment — I don't have an answer, but one observation from doing this by hand: the packages OCI artifact is assembled by
core/installeras the last build unit, so any image reference fixed after that point doesn't reach the cluster at all. That ordering constraint seems relevant to whichever shape you pick; it bit me twice before I noticed.Also flagging in case it affects planning: the CNCF Community Infrastructure Lab is no longer accepting new applications, and Equinix Metal sunset on 30 June 2026. If the CNCF-sponsored ARM workers mentioned above were on that infrastructure, it may be worth confirming they're still available before planning around them.
Happy to share the scripts that reproduce the macOS/Apple Silicon build if that's useful for the local-dev angle in #519 — they're rough, but they go from a clean machine to a converged cluster.
feat(build): build package images for amd64 and arm64 (#4498)