Repository navigation
e2e: node-join diagnostics still describe the QEMU sandbox that no CI lane boots #4825
Description
The node-join diagnostics in hack/e2e-chainsaw/_lib/run-kubernetes.sh were written for the QEMU sandbox, where the three management nodes are QEMU guests started by hack/e2e-prepare-cluster.bats. Since #4020 the PR lanes run the nodes as Talos containers, and once nightly and the release-tag lane move too, no CI lane boots that sandbox. The diagnostics still run on every failed tenant node-join and still explain their output in QEMU terms.
Two of the legends now state something false in the report. The node CPU-time legend says a zero steal is meaningful because "the sandbox VMs are started with accel=kvm and -cpu host by hack/e2e-prepare-cluster.bats". On the container lane the node's /proc/stat is the runner kernel's, so that reasoning does not apply. The NO-KVM-DIR line says the same thing about accel=kvm and reads a missing kvm debugfs directory as evidence about how the sandbox VMs were run, while the only KVM guests left on this lane are the tenant workers.
The sandbox QEMU thread capture (cozy_capture_sandbox_qemu_thread_cpu) looks for srv<N>/qemu.pid files that the container lane never writes, so it can only report that it found nothing. It still runs on every tenant Kubernetes suite, once before the node-join wait and once after it, green runs included.
hack/run-kubernetes-kvm-exits_test.bats and hack/run-kubernetes-runner-cpu_test.bats pin parts of this wording, so changing it means changing the tests too. I think each capture should either be re-targeted at what the container lane actually has (tenant worker VMs under KubeVirt on the runner kernel) or removed together with the QEMU sandbox when it is retired (#4642).
Activity
lexfrei commentedon Oct 9, 2026
Same thing in the header of
hack/sandbox-runner-headroom.bats: it describes every lane as booting the QEMU guests fromhack/e2e-prepare-cluster.bats. The note inside the test already explains that container lanes read the same figures, so only the header text is out of date.lexfrei commentedon Oct 9, 2026
Two more places still name the QEMU entry point as the way every lane gets its sandbox:
hack/e2e-runner-identity.shsays every lane goes throughmake prepare-env, anddocs/agents/overview.mdpresentsmake prepare-envas the e2e environment. Both still work by accident, sinceprepare-env-containeralso runsapply, but no CI lane callsprepare-envonce nightly and the release-tag lane move.lexfrei commentedon Oct 9, 2026
One more for whoever removes the QEMU target:
packages/core/testing/Makefilejustifiesoverride PLATFORM := linux/amd64with the QEMU lane booting the amd64 disk, anddocs/agents/overview.mdsays the same. Still true whileprepare-clusterexists, but that text has to be rewritten in the same change that drops it.lexfrei commentedon Oct 9, 2026
The same "every e2e lane reaches the sandbox through
make prepare-env" sentence is also indocs/agents/e2e-testing.mdand inpackages/core/testing/Makefilenext to the runner-identity call, and the Makefile comment there saysapplyboots three QEMU guests, which it does not. The mechanism still works becauseprepare-env-containerrunsapplytoo, so only the wording is off.ci(e2e): run nightly and release-tag e2e on the container lane (#4828)