Repository navigation
e2e: node-join diagnostics still describe the QEMU sandbox that no CI lane boots #4825
Description
Activity
- addedpriority/backlogGeneral backlog priority. Lower than priority/important-longtermGeneral backlog priority. Lower than priority/important-longtermkind/cleanupCategorizes issue or PR as related to cleanup of code, process, or technical debtCategorizes issue or PR as related to cleanup of code, process, or technical debtarea/testingIssues or PRs related to testing (e2e, bats, unit tests)Issues or PRs related to testing (e2e, bats, unit tests)
on Oct 9, 2026 - addedtriage/needs-triageIndicates an issue needs triage by a maintainerIndicates an issue needs triage by a maintainer
on Oct 9, 2026 lexfrei commented
on Oct 9, 2026 ContributorAuthorMore actionsSame thing in the header of
hack/sandbox-runner-headroom.bats: it describes every lane as booting the QEMU guests fromhack/e2e-prepare-cluster.bats. The note inside the test already explains that container lanes read the same figures, so only the header text is out of date.lexfrei commented
on Oct 9, 2026 ContributorAuthorMore actionsTwo more places still name the QEMU entry point as the way every lane gets its sandbox:
hack/e2e-runner-identity.shsays every lane goes throughmake prepare-env, anddocs/agents/overview.mdpresentsmake prepare-envas the e2e environment. Both still work by accident, sinceprepare-env-containeralso runsapply, but no CI lane callsprepare-envonce nightly and the release-tag lane move.lexfrei commented
on Oct 9, 2026 ContributorAuthorMore actionsOne more for whoever removes the QEMU target:
packages/core/testing/Makefilejustifiesoverride PLATFORM := linux/amd64with the QEMU lane booting the amd64 disk, anddocs/agents/overview.mdsays the same. Still true whileprepare-clusterexists, but that text has to be rewritten in the same change that drops it.lexfrei commented
on Oct 9, 2026 ContributorAuthorMore actionsThe same "every e2e lane reaches the sandbox through
make prepare-env" sentence is also indocs/agents/e2e-testing.mdand inpackages/core/testing/Makefilenext to the runner-identity call, and the Makefile comment there saysapplyboots three QEMU guests, which it does not. The mechanism still works becauseprepare-env-containerrunsapplytoo, so only the wording is off.- added a commit that references this issue
on Oct 10, 2026
The node-join diagnostics in
hack/e2e-chainsaw/_lib/run-kubernetes.shwere written for the QEMU sandbox, where the three management nodes are QEMU guests started byhack/e2e-prepare-cluster.bats. Since #4020 the PR lanes run the nodes as Talos containers, and once nightly and the release-tag lane move too, no CI lane boots that sandbox. The diagnostics still run on every failed tenant node-join and still explain their output in QEMU terms.Two of the legends now state something false in the report. The node CPU-time legend says a zero steal is meaningful because "the sandbox VMs are started with accel=kvm and -cpu host by hack/e2e-prepare-cluster.bats". On the container lane the node's
/proc/statis the runner kernel's, so that reasoning does not apply. TheNO-KVM-DIRline says the same thing aboutaccel=kvmand reads a missing kvm debugfs directory as evidence about how the sandbox VMs were run, while the only KVM guests left on this lane are the tenant workers.The sandbox QEMU thread capture (
cozy_capture_sandbox_qemu_thread_cpu) looks forsrv<N>/qemu.pidfiles that the container lane never writes, so it can only report that it found nothing. It still runs on every tenant Kubernetes suite, once before the node-join wait and once after it, green runs included.hack/run-kubernetes-kvm-exits_test.batsandhack/run-kubernetes-runner-cpu_test.batspin parts of this wording, so changing it means changing the tests too. I think each capture should either be re-targeted at what the container lane actually has (tenant worker VMs under KubeVirt on the runner kernel) or removed together with the QEMU sandbox when it is retired (#4642).