Skip to content

e2e: cozy_diag_read has no per-call bound, so a read that runs inside the resource it measures cannot use it #3687

Open

Description

cozy_diag_read takes its bound from the global COZY_DIAG_READ_TIMEOUT (default 20s) with no per-call override. That works while every read in the block is comparable, and breaks for a read whose own process runs inside the resource it measures.

The concrete case is a collector that reads cgroup CPU-throttling counters out of a virt-launcher container. The sh that kubectl exec starts lives inside the very cgroup whose throttling is being measured, so on a throttled worker, the only case the collector exists for, that read is slower than the same read on a healthy one. A bound calibrated on the healthy case expires precisely where it matters and records the result as unknown.

Such a collector therefore cannot route through the shared helper without either lowering everyone else's bound or accepting a bound that defeats its purpose, and ends up open-coding timeout instead. That is a second spelling of the idiom the helper exists to unify, and the next reader is likely to "finish" the unification and reintroduce the defect.

Suggested shape: an optional per-call bound, cozy_diag_read_for <seconds> <label> <cmd...>, falling back to COZY_DIAG_READ_TIMEOUT when not given, and routing the CPU-throttling collector through it. Whatever the spelling, the reason the bound differs has to stay next to the call, since the deviation looks like an oversight without it.

Update: the motivating consumer is gone. The collector that needed a longer bound now reads the same counters from the kubelet's cadvisor endpoint instead of exec-ing into the container. Nothing about that read runs inside the cgroup being measured, so it takes the block's bound as-is and the deviation that motivated this issue no longer exists.

The gap itself is still real: cozy_diag_read has no per-call bound, and the next read that needs one will hit the same wall. But there is no blocked work behind it today, so this is a design note rather than something to schedule.

One thing worth separating, since it looks like the same issue and is not. That collector still spells timeout by hand, for a different reason: cozy_diag_read returns 0 by design, and the caller needs the status to tell "the kubelet was not read" from "it was read and held no worker container". That is a request for a status, not for a budget, and it would survive a per-call bound being added here.

Activity

  1. added
    kind/featureCategorizes issue or PR as related to a new feature
    area/testingIssues or PRs related to testing (e2e, bats, unit tests)
    triage/acceptedIndicates an issue is ready to be actively worked on
    priority/backlogGeneral backlog priority. Lower than priority/important-longterm
    and removed
    triage/needs-triageIndicates an issue needs triage by a maintainer
    on Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/testingIssues or PRs related to testing (e2e, bats, unit tests)kind/featureCategorizes issue or PR as related to a new featurepriority/backlogGeneral backlog priority. Lower than priority/important-longtermtriage/acceptedIndicates an issue is ready to be actively worked on

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions