e2e: an unbounded read in cozyreport.sh loses the whole diagnostic tarball instead of truncating it #3568

Describe the bug

A single hung read inside hack/cozyreport.sh does not truncate cozyreport.tgz — it loses the artifact entirely, on exactly the runs where it is the only evidence left.

Mechanism

The Collect report step carries no timeout-minutes of its own, in all three workflows that run it — pull-requests.yaml (step at :764, e2e job at :548 capped by the timeout-minutes: 180 on :566), nightly.yaml (:337, job :221, cap :225) and e2e-tag.yaml (:260, job :130, cap :137). In each of them Upload cozyreport.tgz is the step immediately after. When a step burns the remaining job budget, the job is cancelled and the steps after it never run — so the upload never happens. make ... collect-report || true protects against a non-zero exit, not against a read that does not return.

Most of hack/cozyreport.sh reads with no wall-clock ceiling. Four sections are bounded — not-Ready pods, VMs, VMIs and pending-address services — and the file header names those four as the complete list. Everything else is unbounded: nodes, namespaces, events, HelmReleases, packages and packagesources, PVCs, COSI, kamaji, LINSTOR and Talos.

The sharpest of them is the LINSTOR section, because it is not a get against the apiserver but a shell inside a pod:

kubectl -n cozy-linstor exec deploy/linstor-controller --container=linstor-controller -- sh -c '
  cd /var/log/linstor-controller 2>/dev/null || exit 0
  tar -czf - ErrorReport-*.log 2>/dev/null || true
' > "$DIR/controller-error-reports.tgz" 2>/dev/null || true

kubectl exec streams until the remote command closes the stream. A wedged controller pod, a satellite that stops answering, or a broken SPDY upgrade holds that stream open with no clock anywhere in the path — not on the step, not on the command. The LINSTOR module makes six such calls in total: the three linstor n l / sp l / r l listings and the error-reports list immediately above this one, this controller bundle, and the matching per-satellite bundle below it. No line numbers here on purpose — the surrounding file is under active change, and every one of the six is findable by grepping the module for kubectl exec.

Expected behaviour

A collector that cannot finish gives up and ships what it has. Every read is bounded, and a bound that fires is recorded in the artifact so a truncated report is distinguishable from a healthy cluster.

Actual behaviour

An unbounded read on a degraded cluster consumes the rest of the job budget and the job is cancelled before the upload step. There is no tarball, no partial tree, and nothing in the job log saying which read did not return — so the run that most needed the diagnostics is the run that produces none.

Suggested fix

hack/cozyreport.sh already resolves a wall-clock wrapper once, as COZYREPORT_BOUND, and falls back to running unbounded with a note when timeout is absent. Extending it over the remaining reads is mechanical; the exec calls are the ones worth doing first, since they are the only reads whose duration is controlled by a workload rather than by the apiserver. A timeout-minutes on the Collect report step is worth having regardless, as the backstop for whatever is added later without a bound.

Note on scope

kubectl exec propagates the remote command's exit status, so it is the one construct where timeout's 124 is ambiguous. The existing classification helper in that file assumes no bounded read is an exec and a unit test pins that assumption, so bounding these calls means either relaxing that assumption deliberately or reporting the exec cut-offs through a separate path.