Repository navigation
test(e2e): data-plane capture still overwrites some files and hides failed reads #4629
Description
The e2e data-plane capture (hack/e2e-capture-dataplane.sh) still has a few output and input defects that the budget and filename work did not touch. Each is small; together they decide whether a second capture run leaves a readable artifact.
Files that lose or blur an earlier run:
ovn-lflows.txtis overwritten on every run.node-<node>.txtis appended to without the run separator the other per-run files use.lb-speaker-logs.txtis truncated on every run.
Reads that hide what happened:
- The speaker-log read discards stderr and writes no note when it fails.
ovs_iface_forfalls back to the geneve interface without saying so.COZY_DATAPLANE_READ_TIMEOUTandCOZY_DATAPLANE_LIST_TIMEOUTare not validated. A non-numeric value surfaces as "kubectl exited 125" although kubectl never ran.
Two comments narrate history instead of the reason: the "merge-base behaviour" note in lb_capture_decision and "Five of them used to" in capture_lb_datapath.
Fix shape: append with the shared run separator everywhere, write a note on every failed or fallen-back read, validate the timeout knobs the way the budget knobs are validated, and restate the two comments as reasons.
Activity
lexfrei commentedon Oct 1, 2026
Update after #4640. That PR fixed the items whose lines it already changed:
node-<node>.txtnow gets the run separator, a failed speaker-log read writes a note,ovs_iface_fornotes its geneve fallback, andCOZY_DATAPLANE_READ_TIMEOUTandCOZY_DATAPLANE_LIST_TIMEOUTare validated. Still open here:ovn-lflows.txtandlb-speaker-logs.txtare overwritten or truncated each run, and the two history comments.Review of #4640 turned up four more of the same kind, all on main before that PR:
COZY_DATAPLANE_MAX_PODSandCOZY_DATAPLANE_MAX_LBSare not validated. A non-numeric value makes the-gttest fail, so the pod cap never fires andlb_budget_okreports "cap reached" on the first broken LoadBalancer.capture_pod_dataplaneignores its lookup statuses. When a failed lookup is served from the memo, the conntrack and OVS blocks are skipped with no note.dp_read_outcomecuts kubectl stderr to 300 characters incapture-notes.txt. On a throttled apiserver the throttling lines come first and the real reason is cut off.docs/agents/e2e-testing.mdsays why a read produced nothing goes tocapture-notes.txt. Exec-block errors and the Ready-block reads report inside the block instead.lexfrei commentedon Oct 2, 2026
One more of the same kind, found during the last #4640 review: at around lines 1699 and 1710 of
hack/e2e-capture-dataplane.sh, when the cni-server lookup failure is served from the lookup cache, the announcer or endpoint tcpdump is skipped with no note and no file, the same silent skip the pod captures have after a cached failed lookup.fix(e2e): bound the dataplane capture collector by wall clock (#4640)
lexfrei commentedon Oct 5, 2026
Three more from the review of #4640, checked on main at 2ddd55c.
COZY_DATAPLANE_BUDGETaborts withIllegal number, and under bash a 19-digit one wrapsDP_DEADLINEnegative, so every call is refused (comment).dp_note_spentchecks the notes file and then appends. The two background tcpdump execs incapture_lb_datapathrundp_runat the same time, so if the deadline passes in that window the section note can be written twice. Only cosmetic,dp_walk_endstill matches it (comment).kubectl logsper MetalLB speaker, up to 15s each and with no cap on the count, before the first LB probe. On a slow apiserver with many nodes that can use up the 90s reserve before any LB is probed. A cap on the walk would bound it (comment).