Skip to content

e2e(gateway, redis-tls): remaining unbounded cleanup and swallowed errors #4644

Open

Description

After #4643, the gateway and redis-tls e2e suites still have a few waits that can run past their op and errors that get replaced by a fixed guess. Each has the same shape as #4114 or #4246, in a step that PR didn't touch.

  • Gateway, the VAP-probe cleanup (around line 239 of hack/e2e-chainsaw/gateway/chainsaw-test.yaml): kubectl delete tenants.apps.cozystack.io vaphostprobe has no --timeout and no --wait=false, so a stuck finalizer holds the trap until the op kills it.
  • Gateway, the listener-set polls: a poll that hits its timeout N sh -ec exits 124 under set -e without naming which wait expired.
  • redis-tls, the TLS PONG fallback: || pong='(no reply: redis-cli failed or hit the 15s cap)' throws away the real exit status and output.
  • redis-tls, the sentinel lookup: -o jsonpath='{.items[0].metadata.name}' fails hard on an empty list, and the step exits with no message of its own.

Fix shape: a --timeout on the probe delete with the op sized to include it, a named failure line on each poll, capture rc and stderr in the PONG check the way the plaintext check does, and read {.items[*]...} with a named failure when the list is empty.

Activity

  1. added
    kind/bugCategorizes issue or PR as related to a bug
    area/testingIssues or PRs related to testing (e2e, bats, unit tests)
    on Oct 1, 2026
  2. lexfrei commented on Oct 2, 2026

    @lexfrei
    ContributorAuthor

    Two more of the same shape, found while reworking #4643.

    The listener-set step in the gateway suite runs three kubectl apply and one kubectl get ns ... -o jsonpath label read with no time limit. Its polls and cleanup deletes all run under timeout, and the budget guard in hack/chainsaw-op-budgets.bats only keeps a flat 60s of headroom for these four calls. That 60s is an assumption, not a bound. A stalled request runs until the op ceiling, Chainsaw kills the shell, and the cleanup trap never runs. The fix is to run them under timeout -k like the deletes and let the guard sum them.

    The PONG fallback with "or hit the 15s cap" is in two places in hack/e2e-chainsaw/redis/chainsaw-test.yaml, not one: the TLS check and the redis-tls-flip-rejected check further down. Both need the rc and stderr capture.

  3. lexfrei commented on Oct 2, 2026

    @lexfrei
    ContributorAuthor

    One more misreported failure in redis-tls, in the CA-projection step after #4643. The check that redis-secure.ca-tls still carries tls.key pipes the key list into grep -x 'tls.key', and any failure of that pipeline prints "redis-secure.ca-tls no longer carries tls.key: this check has stopped proving anything". A failed read of the Secret (an API error, or the step budget already spent) ends up with that message too. kubectl's own error is still printed above it, but the last line points at the Secret's content instead of the read. The step runs under pipefail now, so the read's exit status is there to tell the two apart.

  4. lexfrei commented on Oct 5, 2026

    @lexfrei
    ContributorAuthor

    One more from the review of #4643, a comment this time. The cleanup comment in the gateway listener-set step (line 1000 at 2ddd55c) says 124 is the bound. The deletes run under timeout -k 5, so a kubectl that ignores SIGTERM gets the SIGKILL and exits 137. The comment should name both codes (comment).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/testingIssues or PRs related to testing (e2e, bats, unit tests)kind/bugCategorizes issue or PR as related to a bugtriage/acceptedIndicates an issue is ready to be actively worked on

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions