test(e2e): redis-tls reports the 15s cap for any exec failure

Describe the bug

Two checks in the redis-tls test of hack/e2e-chainsaw/redis/chainsaw-test.yaml print a timeout message for failures that are not timeouts. Both wrap a kubectl exec and name the inner 15-second cap whenever the exec exits non-zero.

The replication read, line 441:

if ! info=$(k -n "$NAMESPACE" exec "rfr-redis-secure-$i" -c redis -- \
  sh -c 'REDISCLI_AUTH="$REDIS_PASSWORD" timeout 15 redis-cli --tls --cacert /tls/ca.crt -h localhost -p 6379 info replication'); then
  echo "rfr-redis-secure-$i did not answer INFO replication within the 15s cap" >&2

The sentinel read, line 466:

if ! addr=$(k -n "$NAMESPACE" exec "$sentinel" -c sentinel -- \
  timeout 15 redis-cli --tls --cacert /tls/ca.crt -h localhost -p 26379 \
  sentinel get-master-addr-by-name mymaster); then
  echo "the sentinel did not answer get-master-addr-by-name within the 15s cap" >&2

k() is timeout 30 kubectl "$@". The exec exits non-zero when the inner timeout 15 fires, when the outer 30-second cap kills kubectl, when the pod is not there, on a 403 on pods/exec, and on a container name that does not match. Only the first is what the message says. The rest send a reader looking for a slow redis when the problem is RBAC, a missing pod, or a typo.

Environment

Cozystack: main
In-tree e2e lane

To Reproduce

Reading the two blocks shows it. To see it fire, break the exec some other way: rename the container in either call, or drop RBAC on pods/exec, then look at what the step prints.

Expected behaviour

The message carries what the dial printed and the exit status, and names the cap only when the cap fired.

Actual behaviour

Every failure of these two execs reads as a 15-second timeout.

Additional context

The same file already does this correctly forty lines above, in the plaintext-listener check:

The verdict is the listener's refusal, not the absence of PONG. A missing pod, a 403 on exec, a wrong container name and the k() cap firing print no PONG either, and a check that only looked for it would pass on every one of them.

Same reasoning applies to these two. Capture stderr and the exit status, print both, drop the claim about which cap fired.

Found while reading this step for #4238. Both messages predate it.

Checklist

I have checked the documentation
I have searched for similar issues
I have included all required information
I have provided clear steps to reproduce
I have included relevant logs

One more for whoever picks this up: the convergence wait going in for #4238 turns the replication read into a poll, so that branch runs once per pod on every poll rather than once per run. With a 60s deadline at 5s intervals that is a dozen or so reads on a run where the link never settles, each printing the same message about a cap that may not have fired.

Second shape of the same thing in this script, wider than the two exec guards. The script runs under set -eu with no pipefail, so every read piped into something takes its status from the tail of the pipe:

src=$(k -n "$NAMESPACE" get secret redis-secure.ca-tls -o jsonpath='{.data.ca\.crt}' | base64 -d)
if [ -z "$src" ]; then
  echo "redis-secure.ca-tls has no ca.crt: the declared source name or key is wrong" >&2

A kubectl that fails leaves src empty, base64 -d exits 0 on empty input, and the check below reports a wrong source name for what was an API error. Reproduced with a stub:

$ printf 'set -eu\nk() { return 1; }\nsrc=$(k get secret x -o jsonpath=y | base64 -d)\necho "survived, src=[$src]"\n' | sh
survived, src=[]

Same for proj, and for the two keys reads that pipe through sort | tr. The emptiness checks do catch the state, so nothing passes that should fail; only the reported cause is wrong. Worth fixing in the same pass as the messages above, since it is the same question of whether a check has earned the cause it names.

A file in this same tree already avoids that trap and writes down why. hack/e2e-chainsaw/_lib/backup-access-preflight.sh, above the BucketInfo read:

Keep the producer and decoder as separate commands. This helper is sourced by cozytest.sh under /bin/sh, where pipefail is unavailable; a pipeline would otherwise let a failed Secret read look successful when base64 exits zero on empty input.

Same mechanism and the same base64. It splits the read into two guarded commands with a message on each, failed to read BucketInfo Secret ... for the kubectl half and ... is not valid base64 for the decode half, which is also the shape a fix here would take.

What makes it worth fixing is that the failure is a wrong diagnosis rather than a crash. An API error gets reported as the declared source name or key is wrong, so whoever is reading a red run goes to the manifest when the problem was the cluster. That is the same reason the two messages at the top of this issue are worth fixing, which is why both live here.

fix(e2e): wait for the redis replica link instead of sampling it once (…

test(e2e): keep gateway and redis-tls op ceilings above their own wai…

Navigation Menu

Repository navigation

test(e2e): redis-tls reports the 15s cap for any exec failure #4246

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

test(e2e): redis-tls reports the 15s cap for any exec failure #4246

Description

Activity

lexfrei commented on Sep 14, 2026

lexfrei commented on Sep 14, 2026

lexfrei commented on Sep 14, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions