Repository navigation
e2e: cozy_diag_read has no per-call bound, so a read that runs inside the resource it measures cannot use it #3687
Description
cozy_diag_read takes its bound from the global COZY_DIAG_READ_TIMEOUT (default 20s) with no per-call override. That works while every read in the block is comparable, and breaks for a read whose own process runs inside the resource it measures.
The concrete case is a collector that reads cgroup CPU-throttling counters out of a virt-launcher container. The sh that kubectl exec starts lives inside the very cgroup whose throttling is being measured, so on a throttled worker, the only case the collector exists for, that read is slower than the same read on a healthy one. A bound calibrated on the healthy case expires precisely where it matters and records the result as unknown.
Such a collector therefore cannot route through the shared helper without either lowering everyone else's bound or accepting a bound that defeats its purpose, and ends up open-coding timeout instead. That is a second spelling of the idiom the helper exists to unify, and the next reader is likely to "finish" the unification and reintroduce the defect.
Suggested shape: an optional per-call bound, cozy_diag_read_for <seconds> <label> <cmd...>, falling back to COZY_DIAG_READ_TIMEOUT when not given, and routing the CPU-throttling collector through it. Whatever the spelling, the reason the bound differs has to stay next to the call, since the deviation looks like an oversight without it.
Update: the motivating consumer is gone. The collector that needed a longer bound now reads the same counters from the kubelet's cadvisor endpoint instead of exec-ing into the container. Nothing about that read runs inside the cgroup being measured, so it takes the block's bound as-is and the deviation that motivated this issue no longer exists.
The gap itself is still real: cozy_diag_read has no per-call bound, and the next read that needs one will hit the same wall. But there is no blocked work behind it today, so this is a design note rather than something to schedule.
One thing worth separating, since it looks like the same issue and is not. That collector still spells timeout by hand, for a different reason: cozy_diag_read returns 0 by design, and the caller needs the status to tell "the kubelet was not read" from "it was read and held no worker container". That is a request for a status, not for a budget, and it would survive a per-call bound being added here.
Activity