test(tests): run unit suites with Bats and retain POSIX coverage - #3849

NOT LGTM. The migration is well built and I could not fault the mechanism, but it leaves a dozen file headers asserting the opposite of what it just enabled, in the same files it edits.

Business context: move the hack unit suite off the home-grown runner onto bats(1), and replace the remembered set -u helper convention with an audit that walks the tree.

What I verified rather than assumed:

• set -u set in setup() really does reach the test body under Bats 1.14.0. Wrote a throwaway file that reads an unset variable in the body and it aborts with unbound variable, so hack/test_helper.bash does what it claims.
• bats runs a test with the working directory it was invoked from, not the test file's directory. That matters because a large share of these files resolve repo-relative paths against the cwd, and the Makefile invokes bats from the repo root, so they keep working.
• All 66 unit files carry load test_helper and none defines its own setup(), so hack/bats-strict-setup.bats has nothing to catch today.
• BATS_JOBS degrades correctly. On a box without GNU parallel it resolves to 1 instead of handing bats a flag it cannot honour.
• Full suite under the new target: 1161 tests planned, 1146 ok, 15 not ok, zero bats warnings, and all 66 files executed. Every one of those 15 fails identically at the base commit (14 in hack/migration-seaweedfs-db-adopt.bats, plus test 25 of hack/seaweedfs-naming-audit.bats), so this PR introduces no new failures. Both look platform-specific to macOS.

Worth stating because the description does not: the old target ran hack/cozytest.sh "$$f" || exit 1 inside a single recipe loop, so it stopped at the first red file and everything sorting after it never ran. On this machine that hid 23 files. Handing the whole list to one bats invocation is what fixes that, and it is a bigger win than the runner swap itself.

Blockers

B1: twelve headers now say setup() is never invoked, directly above the line that installs one

hack/build-matrix_test.bats:6 reads "A bats setup() hook would be dead here, cozytest never invokes it, so the repo-root cwd is supplied by the runner rather than a setup() cd." Line 18 of the same file is the load test_helper this PR adds, and that helper exists to define a setup() which the new runner does invoke.

hack/select-e2e_test.bats:8 is the same shape ("setup()/teardown() are not honored"), and so are hack/cozystack-version-stamp.bats, hack/etcd-probe_test.bats, hack/ghcr-mirror_test.bats, hack/helm-unit-tests.bats, hack/pod-label-census_test.bats, hack/remediation-guard.bats, hack/run-kubernetes-drain_test.bats, hack/run-kubernetes-schedulable_test.bats, hack/select-install_test.bats and hack/talos-image-cache_test.bats. Several of the same headers also say "there is no bats run or $status", which stops being true for the runner CI now uses.

This is the failure mode hack/bats-strict-setup.bats was written against, in its own words: a requirement whose only record is the diff that introduced it. A reader who opens one of these files to add a test is told not to use the hook the file three lines later depends on.

B2: cozytest.sh is the CI path stops being true, and the convention doc still argues from it

hack/cozyreport.bats:33 says "cozytest.sh is the CI path." This PR edits that file (the load goes in at line 36) and makes the sentence false.

docs/agents/e2e-testing.md:48 is the bigger one. It reads "the CI runner hack/cozytest.sh, which is not the bats binary, reports the same failure correctly, so the two disagree exactly when it matters." That sentence is the doc's reason the vanished-TAP-line hazard was survivable: the runner CI used reported the failure. After this PR the unit suite runs under bats in CI, so for those 66 files the hazard is the default rather than the theoretical case, and the paragraph still describes the old arrangement. The PR touches no markdown at all.

Non-blocking follow-ups

1. The pre-commit filter does not cover what its own comment says it covers. The comment calls it "the union of what the suite reads" and argues that narrowing it "would let exactly the regressions they exist to catch through", but ^(hack/|Makefile$|\.github/workflows/|packages/|\.claude/hooks/) at .pre-commit-config.yaml:50 matches neither .github/labels.yml, which hack/issue-triage-contract.bats:37 asserts over, nor docs/agents/contributing.md, which hack/downstream-trigger-map.bats:40 asserts over. Commits touching only those skip the hook. CI still runs the suite unconditionally so nothing reaches main, which is why this is not a blocker, but the claim is wider than the regex.
2. The 151s figure in the same comment is the eight-core GNU parallel case. Without GNU parallel BATS_JOBS resolves to 1, and parallel is not present by default on macOS. Serial here took 9m52s. The Makefile documents the fallback; the hook comment quotes only the fast number, which is the one a developer picking whether to keep the hook will read.

E2E is red, same install-step failure as the PR underneath. Nothing on that path runs a hack/*.bats unit file and the unit job is green, so I do not read it as this diff.

NOT LGTM

Two structural gaps in the coverage guarantee this PR is built to provide. Neither is a live break at this head. Both are the same silent-decay mechanism the PR exists to remove, left sitting one layer up.

[MAJOR] hack/test_helper.bash:42-48: the audit pins the presence of the mechanism, nothing pins its effect

hack/bats-strict-setup.bats proves that all 85 unit files carry a column-zero load test_helper and that no file-local setup() drops the call. It does not prove that set -u is actually in force inside a test body. The file admits this at hack/bats-strict-setup.bats:57-61, where it points at "the mutation check in #3453", a manual check that ships with nothing.

So the property the whole PR is about has no in-tree guard. Grep the unit lane for an assertion on the effect and you get three files, all of them asserting that a script under test did not print unbound variable (hack/capture-dataplane.bats:1357, hack/run-kubernetes-node-join_test.bats:1990), not that the harness has nounset on.

Concrete scenario: someone refactors strict_setup (renames the option, wraps it in a conditional, replaces the body while keeping the name), or a future bats release changes when setup()'s shell options reach the test function. All 16 audit tests stay green, CI stays green, and 1566 tests quietly go back to reading unset variables as empty strings. That is the "test asserting nothing while reporting ok" state described at hack/bats-strict-setup.bats:6-10, restored by a change nobody can see.

Verified by mutation in a throwaway copy of the checkout:

$ sed -i '' 's/^  set -u$/  : # set -u REMOVED BY MUTATION/' hack/test_helper.bash
$ bats hack/bats-strict-setup.bats    -> rc=0, 0 failures
$ bats hack/bats-runner-contract.bats -> rc=0, 0 failures
$ bats hack/md-no-hardwrap.bats       -> rc=0, 0 failures

The counter-mutations do work, which is what makes this gap specific rather than general:

# drop the load line from one real file
$ grep -v '^load test_helper$' hack/helm-unit-tests.bats > /tmp/x && mv /tmp/x hack/helm-unit-tests.bats
$ bats hack/bats-strict-setup.bats
not ok 4 every hack/*.bats unit file restores set -u through the shared helper
# FAIL: strict-mode setup missing or overridden:
# helm-unit-tests.bats: does not load the strict-mode helper; ...

# narrow BATS_UNIT_FILES without narrowing the audit
$ bats hack/bats-strict-setup.bats
not ok 3 the audited set is exactly the set the Makefile hands the runner

A five-line canary closes it, and it does discriminate. Measured on Bats 1.13.0 here:

# hack/strict-mode-canary.bats
load test_helper
@test "reading an unset variable aborts the test body" {
  run bash -c 'set -u; echo "${DEFINITELY_UNSET_VAR}"'
  [ "$status" -ne 0 ]
}

Written against the real helper it goes red the moment strict_setup stops setting -u. A direct in-body read (echo "${UNSET}" with no run) works too: it fails with unbound variable with the load in place, and passes vacuously without it.

[MAJOR] Makefile:210-224: the POSIX-compat lane is discovered for one class of file and hand-listed for the other, and only the hand-listed half can rot

The Makefile comment at Makefile:205-209 states the principle: discover the files "instead of maintaining a list that goes stale as main adds tests". The discovery at Makefile:210 implements that for exactly one path shape:

BATS_SOURCED_CHAINSAW_FILES := $(shell grep -El '^[[:space:]]*\.[[:space:]]+.*e2e-chainsaw/_lib/.*\.sh' $(BATS_UNIT_FILES))

Everything that dot-sources a POSIX helper living anywhere else is covered only by the literal names at Makefile:212-223. Today that is five files, all of them listed, so the lane is complete at this head:

Nothing audits completeness. hack/bats-runner-contract.bats:80-99 pins the list's contents (every one of the 11 literals must appear in the recipe) and asserts discovery is non-empty at line 84, but nothing asserts in either direction that every unit file which dot-sources a .sh is in BATS_POSIX_COMPAT_FILES.

Concrete scenario: a contributor adds hack/new-mirror_test.bats with . hack/lib/image-refs.sh. The regex at Makefile:210 does not match it, the literal list does not contain it, and hack/bats-strict-setup.bats only checks its load line. The file runs under bash and passes. A local, a [[, or a ${x,,} then lands in hack/lib/image-refs.sh, unit CI stays green, and the break surfaces in the sandbox's dash at release or e2e time. That is the exact failure the comment this PR rewrites at hack/seaweedfs-naming-audit.sh:57-63 used to warn about ("anything bash-only here is untested in CI and unavailable at run time").

Two ways out, either is fine. Broaden the discovery to any dot-sourced .sh and drop the literals that become redundant, or keep the list and add one audit test asserting the two sets agree, in the same shape hack/bats-strict-setup.bats:225-247 already uses to hold the audit and $(BATS_UNIT_FILES) together.

[MINOR] .pre-commit-config.yaml:43-48: an unconditional commit gate that hard-fails on a newly introduced dependency

- id: bats-unit-tests
  name: Run hack/*.bats unit tests
  entry: make bats-unit-tests bats-posix-compat-tests
  language: system
  always_run: true

make bats-unit-tests exits 1 with an install pointer when bats is absent (Makefile:197-200). Correct as an error. The consequence is that every git commit fails for anyone who has run pre-commit install and has not yet installed bats-core, on a repo whose previous runner was pure bash and awk with no external dependency. And on a machine without GNU parallel BATS_JOBS resolves to 1 (Makefile:181-183), so the hook runs 1566 tests serially plus the compat lane on top of that, on every commit. SKIP=bats-unit-tests git commit is documented in docs/agents/overview.md, but that is a per-commit opt-out for a gate CI already runs unconditionally after the workflow change at .github/workflows/pull-requests.yaml:282-284. Worth scoping to files: or making it opt-in.

Claim mismatches

[PARTIAL] "the 11 unit files that source production helpers with a POSIX sh contract retain a narrow secondary pass through cozytest" (PR body). The lane resolves to 30 files, not 11: 11 literals at Makefile:212-223 plus 20 discovered at Makefile:210, one overlapping. Those 30 files hold 964 of the suite's 1566 declared tests, so make unit-tests (Makefile:109) re-runs 62% of the suite a second time, serially, under the runner this PR is retiring. Measured from the Unit & controller tests job log on run 33162401350:

10:13:02  bats -j 4 --report-formatter junit -o "_out/test-reports" <85 files>
10:13:16  1..1566
10:17:58  --- running POSIX compatibility: hack/capture-dataplane.bats ---   (bats lane: ~4m56s)
10:23:58  --- running POSIX compatibility: hack/talos-image-cache_test.bats ---
10:23:59  go test                                                            (compat lane: ~6m01s)

The serial compat re-run costs more wall time than the entire parallel primary lane. Defensible as a trade for dash fidelity, but it is not "narrow", and the .pre-commit-config.yaml:29-35 comment attributing the cost mainly to sleeps in hack/run-kubernetes-node-join_test.bats understates where it actually goes.

Caveats

• Phase 5b in its chart form is N/A: the diff touches no packages/, no api/, no chart, no migration, no RBAC, no image reference. The upgrade-equivalent surface here is CI and the local commit gate, which is what the findings above cover. envelope.cozy_invariants and repo_invariants are both empty, consistent with that classification.
• Phase 5c equivalent, executed rather than reasoned. Corners rendered: GNU parallel absent (make print-bats-jobs returns 1, bats -j 1 runs and passes without parallel); moreutils parallel present (returns 1, asserted by hack/bats-runner-contract.bats:50-64, green here); GNU parallel present (CI ran bats -j 4, plan 1..1566, no bats warning: Executed N instead of expected M); BATS_JOBS forced above 1 with no GNU parallel (bats -j 2 exits 1 loudly rather than reporting zero executed tests as success, so the override is fail-closed); bats absent (Makefile:197-200); code PR versus docs-only PR (.github/workflows/pull-requests.yaml:296-303).
• Not executed: the full 85-file suite on this workstation. hack/select-e2e_test.bats test 45 fails here both before and after every mutation, a local BSD/ugrep environment difference rather than a PR effect. Ubuntu CI runs the lane green.
• The #3453 item-5 shellcheck gate that the second commit names as the static replacement for the lost dash coverage does not exist yet. Until it does, Makefile:210-224 is the only thing standing between a POSIX helper and bash-only CI, which is why the completeness gap above matters more than it otherwise would.

NOT LGTM. The audit work holds up and both standing reviews are answered, but the branch cannot land on main as it stands, and one new comment claims coverage that is not there.

What I checked by mutation rather than by reading, under Bats 1.14.0 on this head:

• Deleting load test_helper from one real unit file turns hack/bats-strict-setup.bats red naming that file.
• Adding a second name to the Makefile's filter-out turns "the audited set is exactly the set the Makefile hands the runner" red with the diff.
• Replacing set -u in hack/test_helper.bash with : turns "default and direct canonical setup enable nounset in real nested Bats tests" red. That closes the first major finding of the 2026-09-01 review, where the same mutation left every audit green.
• Emptying BATS_SOURCED_SH_FILES turns two of the seven runner contracts red, which closes its second one.
• The pinned bats commit is right. The annotated tag v1.14.0 dereferences to eb7f42f8d608ac693d7a4b67474f6714ea68cfc5.
• The stale headers from my 2026-08-19 review are gone. Grepping the unit lane for the sentences I quoted ("cozytest.sh is the CI path", "setup()/teardown() are not honored", "canonical runner") returns nothing, and the one surviving "on the CI runner" line describes the compatibility lane, where it is true.

Blockers

B1: the branch sits two weeks behind main and the rebase is not mechanical

Three things, all measured against origin/main at e3ab40eb8.

The merge base with main is c8e4d97df, 2026-08-27. hack/ghcr-mirror_test.bats and hack/node-join-soft-red_test.bats are edited here and were deleted from main in 186547d83, which landed 2026-09-05. Both come back as modify/delete conflicts.

Main has since answered the same problem a different way. It replaced the serial for f in ...; do hack/cozytest.sh ... loop with one Make target per file (BATS_UNIT_TARGETS := $(patsubst hack/%.bats,bats-unit-%,$(BATS_UNIT_FILES))), and CI runs make unit-tests test-controllers -j4 --output-sync=target. hack/unit-test-parallelism.bats pins that. It counts lines matching ^COZYTEST_TRACE=[^ ]* hack/cozytest.sh "hack/.*\.bats"$ in the output of make -n -j4 --output-sync=target bats-unit-tests and requires the count to equal the number of unit files. make -n bats-unit-tests on this head emits one bats -j 1 --report-formatter junit ... line and zero matches, so that test fails after the rebase with make scheduled 0 BATS files independently, expected N. Nothing warns you at rebase time, because the file does not exist on this branch's base: the rebase applies cleanly and the suite goes red on the next run.

Main's unit lane holds 104 files against this branch's 85. 23 exist only on main and all 23 postdate the merge base. hack/test_helper.bash does not exist on main, so none of them carries a load line, and hack/bats-strict-setup.bats will name all 23 the first time it runs after the rebase.

None of that is a defect in this design. It does mean the landing order is now a decision rather than a formality. Main answered "parallelise the unit lane and stop it dying at the first red file" with make's fan-out and no new dependency; this answers it with bats(1), JUnit, --filter-status, and the deletion of the awk parser. Both hold up. They cannot both be the recipe, and whichever lands second has to adapt or retire the other's contract test.

Separately: the description says "The stack was rebased onto main immediately before push". The last push is dated 2026-09-10 and the merge base is 2026-08-27, so that sentence does not describe this head.

B2: a new comment claims coverage that is not there

hack/bats-strict-setup.bats:81-83 reads "hack/e2e-apps/ holds live-cluster suites the Makefile's wildcard never reaches and that packages/core/testing runs against a real cluster". The first half is right. The second is wrong: packages/core/testing/Makefile runs exactly three BATS files (hack/e2e-prepare-cluster.bats, hack/e2e-install-cozystack.bats, hack/e2e-test-openapi.bats) and never mentions e2e-apps. Across the tree the only references to that directory outside its own contents are comments and the "Chainsaw port of hack/e2e-apps/..." headers on the suites that replaced most of it. The two files still sitting there, monitoring-oidc-customconfig.bats and monitoring-oidc-system.bats, are executed by nothing.

That sentence costs more here than it would in another file. A reader who notices the audit's glob stopping at the directory boundary asks exactly this question and gets a false reassurance, in the one file whose whole subject is coverage that does not decay. Say they are run by nothing today, or drop the clause.

Non-blocking

1. Two numbers in the description. make print-bats-posix-compat-files prints 30 files, not the "11 unit files" still described there as a narrow secondary pass. That was raised on 2026-09-01 and is the only item from either standing review still open. bats --count over the lane reports 1570 tests, not 1,564.
2. The runner move leaves most of the !-negated assertions where they were. 101 of them across the unit lane hand their status to nothing on the right, and errexit skips a pipeline that starts with !, so bats reads one only when it is the last thing the body runs. A pair in hack/seaweedfs-guard-parity.bats shows both sides: falsifying line 112, the last command of the for loop that ends the body, fails the test, and falsifying line 109 inside the same loop does not. That class belongs to #3453 and this PR does not claim to close it, but "make coverage structural" in the title invites the opposite reading.
3. The compatibility lane's discovery is pinned in one direction only. Emptying BATS_SOURCED_SH_FILES reddens two contracts, while replacing it with $(BATS_UNIT_FILES) leaves all seven green and would grow the lane from 30 files to 85 under the serial runner with nothing to say so. Since the 30-file lane already costs more wall time than the whole parallel primary lane, an upper bound is worth about as much as the lower one.

myasnikovdaniil NOT LGTM, and only because of the PR description. The code at b6c204f is fine by me. This verdict covers the one commit on top of test/bats-exit-handlers and says nothing about #3848, which still has its own open review.

What I checked on this head:

• CI runs the whole lane under real Bats. The "Unit & controller tests" job ran bats -j 4 over the full list and printed 1..1943 with 1943 ok, zero not ok and zero skips. So GNU parallel is on the runner, and 1.14.0 is installed from the tag with a commit check.
• Locally the lane is 139 files and 1943 tests under Bats 1.14.0. The 25 red cases are all in the five docker-backed migration files and fail on /work/cmdlog: Permission denied from a bind mount. hack/cozytest.sh fails the same way on the same files on my machine, so that is my host, not the runner change.
• Mutations. Adding a filter-out of one file to BATS_UNIT_FILES turns "the audited set is exactly the set the Makefile hands the runner" red. Replacing set -u in hack/test_helper.bash with : turns the nested nounset canary red. Replacing the grep in BATS_SOURCED_SH_FILES with $(BATS_UNIT_FILES) grows the POSIX lane from 39 to 139 files and turns "compatibility discovery includes new shell dependencies and excludes unrelated files" red. That closes the widening point from my last review.
• The branch now merges into main at 8b86166 with no conflicts, and the false e2e-apps comment is gone. Both blockers from 2026-09-11 are closed.
• The commit message is fine: it gives the reason, has the sign-off and Assisted-by: LLM.

Blocker: the description states things the code does not do

The description becomes the merge commit message, so its facts land in history as written. Four of them are wrong on this head, and one of them was already raised on 2026-09-01 and 2026-09-11.

1. "The unit lane now contains 85 files and 1,564 declared tests." It is 139 files and 1943 tests; the CI log prints 1..1943. The Evidence section repeats the old numbers (1,564 and 1,561).
2. "CI installs a commit-addressed, checksum-verified Bats 1.14.0 archive." There is no archive and no checksum. The workflow does a git clone --depth=1 --branch v1.14.0 and compares rev-parse HEAD with BATS_COMMIT, and hack/bats-runner-contract.bats fails if the installer ever uses /archive/.
3. "The pre-commit hook runs for every commit, including deletion-only changes." The bats-unit-tests hook in .pre-commit-config.yaml has stages: [manual], so it runs only when invoked by hand. The comment above it says so too.
4. "the 11 unit files that source production helpers with a POSIX sh contract retain a narrow secondary pass through cozytest." make print-bats-posix-compat-files prints 39.

The fix is a description edit, no push needed. Numbers that grow with the tree could also just be dropped.

When this is retargeted to main

Main added hack/collect-images_test.bats, hack/stitch-multiarch_test.bats and hack/verify-multiarch_test.bats after your base. None of them loads test_helper, so on the merged tree hack/bats-strict-setup.bats goes red and names all three. That is the audit doing its job. With a load test_helper line added to each, the audit and all three files pass under Bats with set -u on the merged tree (45/45). The header of hack/collect-images_test.bats also says "run by hack/cozytest.sh, not real bats", which will be stale in the same way as the headers you already cleaned up.

myasnikovdaniil NOT LGTM. The description and the design are fine now, but the unit job on this head is red, and one of its two causes is in the diff. Everything from my 2026-09-11 and 2026-09-29 reviews is closed, and so is everything from the 2026-09-01 review.

Blockers

1. hack/bats-unit-keep-going.bats fails on Linux

Run 36977262336, job "Unit & controller tests", reports not ok 97 and not ok 98 from this file:

/usr/local/libexec/bats-core/bats: line 120: bats_readlinkf: command not found
/usr/local/libexec/bats-core/bats-exec-file: line 236: /tmp/tmp.670oBMNT96/bats-exec-test: No such file or directory
# bats warning: Executed 0 instead of expected 2 tests

Inside a test, Bats has put its own libexec directory first on PATH, and that directory has a second bats script. That script needs bats_readlinkf, which bin/bats exports as a bash function. kg_run reaches the inner bats through make, and make runs the recipe with /bin/sh. On Ubuntu that is dash, and dash drops exported bash functions from the environment. The recipe then picks up the libexec bats without its function. On macOS /bin/sh is bash, the function survives, and the file passes, which matches it being green on my machine.

I reproduced it on macOS by putting SHELL := /bin/dash at the top of the Makefile: same two failures, same error lines. Removing the libexec directory from PATH in kg_run makes both pass with that mutation in place:

  env -u MAKEFLAGS -u MFLAGS -u MAKELEVEL PATH="${PATH#"$BATS_LIBEXEC":}" \
    make --file="$REPO_ROOT/Makefile" ...

The previous green CI run did not cover this, because the file came from main after my last review and was rewritten during the rebase.

2. Two unit files from main do not load test_helper

Main added hack/kafka-suite-pvc-cleanup_test.bats and hack/tracing-central-credentials.bats after this branch's merge base. On the merge ref, "every hack/*.bats unit file restores set -u through the shared helper" fails and names both (not ok 83 in the same job). The audit is doing its job here. With a load test_helper line added to each, the audit and both files pass under Bats with set -u on the merged tree (22/22). The header of hack/tracing-central-credentials.bats also says "Run via hack/cozytest.sh from the repo root (make bats-unit-tests)", which stops being true once this lands.

E2E Tests is red only because the in-tree e2e job was skipped behind the unit job.

What I checked on a711b7c

• Both rebase conflicts are resolved on substance. pull-requests.yaml keeps main's -k invocation and its comment. Main's trigger_map output goes away, which is correct because the checks job now runs for every non-release PR and docs-only PRs run make bats-unit-tests. hack/unit-test-parallelism.bats keeps the -k contract and checks with a stub bats that every discovered file reaches one call exactly once.
• The description no longer carries the four wrong statements from 2026-09-29. The three files main added before (collect-images, stitch-multiarch, verify-multiarch) load the helper, and their headers no longer say cozytest is the runner.
• The changes asked for on 2026-09-01 are in: the nested nounset canary, POSIX discovery for any dot-sourced .sh, and the hook moved to the manual stage.
• Mutations, Bats 1.14.0. Filtering hack/md-no-hardwrap.bats out of BATS_UNIT_FILES turns two tests red: "the audited set is exactly the set the Makefile hands the runner" and "the modelled runner selection still matches the Makefile". Adding export SHELLOPTS to strict_setup turns "the helper leaves subprocess shell options unchanged" red. Removing hack/capture-dataplane.bats from the reviewed list (discovery does not pick it up) turns "the POSIX compatibility lane retains reviewed and sourced shell-facing files" red.
• Locally make bats-unit-tests runs 2072 tests. 2047 pass. The 25 failures are all in the docker-backed migration files and fail on /work/cmdlog: Permission denied, the same host problem as last time, and CI runs them green. make bats-posix-compat-tests BATS_POSIX_SHELL=dash passes all 42 files. I have no GNU parallel here, so both ran serially.
• The commit has the sign-off and only Assisted-by: LLM.

myasnikovdaniil LGTM. Both blockers from my last review are fixed, and the unit job on this head is green.

The keep-going test now drops the Bats libexec directory from PATH before calling make, so the recipe finds the real bats under dash too. I checked it on macOS with SHELL := /bin/dash at the top of the Makefile: both tests pass. With the PATH change removed and the same mutation in place, both fail again with bats_readlinkf: command not found. In CI (run 36994882194) they are ok 97 and ok 98.

hack/kafka-suite-pvc-cleanup_test.bats and hack/tracing-central-credentials.bats now load test_helper, and the stale cozytest line in the tracing header is gone. The 7 commits main gained since your rebase touch only one Bats file, hack/mariadb-first-boot-waits.bats, which already loads the helper. On the trial merge with current main, every non-e2e hack/*.bats loads it, and the audit is ok 83 in CI.

What I ran on the trial merge, Bats 1.14.0, GNU make 3.81, no GNU parallel, so serially:

• make bats-unit-tests bats-posix-compat-tests -k BATS_POSIX_SHELL=dash: 2063 of 2088 pass. The 25 failures are the docker-backed migration tests hitting /work/cmdlog: Permission denied, the same host problem as last time, and all 25 pass in CI. The POSIX lane passes all 42 files.
• Filtering hack/md-no-hardwrap.bats out of BATS_UNIT_FILES turns "the modelled runner selection still matches the Makefile" and "the audited set is exactly the set the Makefile hands the runner" red.
• export SHELLOPTS in strict_setup turns "the helper leaves subprocess shell options unchanged" red.
• Removing hack/capture-dataplane.bats from the reviewed POSIX list turns "the POSIX compatibility lane retains reviewed and sourced shell-facing files" red.

CI on f924cb8: the unit job ran 2088 Bats tests with no failure and 42 POSIX files, pre-commit and the builds are green. E2E Tests is still running as I write this. The commit has the sign-off and only Assisted-by: LLM, and the description matches the diff.

myasnikovdaniil LGTM. Since my approval on f924cb8 the branch was rebased, and the only change beyond the rebase is in pull-requests.yaml: the Go module steps and the helm-unittest install now run only when code is true, so a docs-only PR runs Bats alone. Main's HELM_UNITTEST_VERSION pin survived the rebase inside that branch. No Bats file needs Go modules or the plugin: hack/helm-unit-tests.bats stubs make, and the files that mention go only grep Dockerfile fixtures.

The unit job on this head is red because main added a unit file without the helper. Run 37470335591 fails not ok 93 on hack/check-postgres-postgis-versions.bats. On a trial merge with current main (bcda5c8) the audit also names hack/computeplane-kcp-hold_test.bats and hack/migration-59-vmi-hotplug-volumes.bats, so the rebase needs load test_helper in all three. With it added, the audit passes and all three files pass under Bats with set -u, the migration-59 one through Docker. Their headers still say cozytest is the runner, worth fixing in the same pass like the tracing file last time.

The points from IvanHunters' 2026-09-01 review are in on this head: the nested nounset canary in hack/bats-strict-setup.bats, POSIX discovery of any dot-sourced .sh, and the pre-commit hook on the manual stage.

pre-commit is green, and E2E Tests is red only because the in-tree e2e job is skipped behind the unit job. The commit has the sign-off and only Assisted-by: LLM.

myasnikovdaniil I rebased this onto current main. Main gained three unit bats files since, so I added load test_helper to each of them to keep the strict-setup audit green.

LGTM. Earlier findings closed. The migration is mostly a one-line load test_helper across existing suites; the three substantive test changes check out, negated assertions and set -- $var splitting do not appear in the added lines, a conversion mutation reddens correctly, and the dash lane still catches what bats cannot. CI at the head runs 2289 tests with none silently dropped.

Non-blocking: nothing pins that unit-tests keeps the bats-posix-compat-tests prerequisite, and the "unconditional lane" test compares only run: strings, not if: conditions. Operational note: bats -j falls back to single-threaded without GNU parallel on the runner, which could slow the job a lot. Merge still needs the API-owner review.

myasnikovdaniil I rebased this onto current main. Main had added the test-backport-audit target and new helpers in hack/select-install_test.bats, so I kept both next to the Bats conversion and the load test_helper line.