Repository navigation
CI must be a strict superset of the local checks #670
Description
Ruling (USER, 2026-08-25): CI is a strict superset of the local checks by default, and a developer can opt into running everything locally — including the cross-platform checks, whose setup can be postponed.
Today neither gate contains the other. Both directions have gaps, and one of them is serious.
CI runs, make check does not
| Check | |
|---|---|
make build-all |
compile every package under every supported GOOS |
make vet-all |
vet under every GOOS |
make lint-all |
lint under every GOOS |
./build/star lint go ./... |
a different tool from make lint — see below |
./build/star lint shell . |
vs make shell-lint, which runs .github/scripts/shell-lint.sh |
make test-race |
vs make test, which has no race detector |
make test-scenario |
end-to-end, the install path's only coverage |
make check runs, CI does not
| Check | Why it matters |
|---|---|
make complexity |
cyclomatic complexity ceiling of 20 — unenforced in CI |
make verify-ldflags |
unenforced in CI, and see below |
make lint |
bare golangci-lint run — a different invocation from CI's star lint go |
The serious one
verify-ldflags exists because unstamped releases shipped for months, unnoticed — a -X flag naming a symbol that does not exist is silently ignored by the linker, and every release before 2026-08-16 went out that way. The check asserts that every build definition in the repository stamps the same package, because agreement between definitions is the only available proof.
No workflow runs it. It fires only when a developer types make check. The gate written in response to a defect that shipped silently is itself only enforced by habit.
complexity is in the same position, with lower stakes.
The lint mismatch, concretely
make lint is golangci-lint run. CI runs ./build/star lint go ./.... They are different invocations and they find different things.
This is not theoretical: on #668, make vet and make test passed locally while CI's star lint go failed on four unused symbols left behind by a refactor. Worse, make check's lint stage cannot run at all on a Go 1.27 toolchain — golangci-lint dies typechecking crypto/internal/randutil in the standard library — so the local gate had been silently degraded to "everything except lint" for anyone on that toolchain, while still reporting itself as make check.
And in the other direction: with those four fixed, the local star lint go reports four staticcheck SA4023 findings in cmd/writ that CI does not (#669).
So "green locally" and "green in CI" are currently two different claims, and a developer cannot tell which one they have.
The shape to build
- CI runs everything the local gate runs, plus the cross-platform matrix.
complexityandverify-ldflagsjoin a workflow. - The two gates share one definition rather than two lists that drift — CI invoking a make target rather than restating the commands, so a check added locally is automatically in CI.
- One lint tool. Either
make lintbecomesstar lint go, or CI stops runningstar lint go. Two tools reporting different sets is the root of both failures above. - A local opt-in for the full matrix —
make check-all, or similar — accepting that the cross-platform pieces need setup and can be postponed.
Item 3 is the one worth doing first: it is small, and it is what actually bit.
Related
- staticcheck flags a possible typed-nil interface guard in writ — locally, but not in CI #669 — the staticcheck divergence, a symptom of item 3
- devlore-test run fails on a quarter of the fixtures the Go harness passes #664 —
devlore-test runfails on a quarter of the fixture suite the Go harness passes, another instance of two runners disagreeing about what "passing" means
Activity
David-Noble-at-work commentedon Aug 25, 2026
Correction: item 3 was misdiagnosed, and so was the cause of #668
I filed this saying
make lint(baregolangci-lint run) and CI'sstar lint goare "different invocations that find different things," and that the local one "cannot run at all on a Go 1.27 toolchain."Both claims are wrong. Measured just now:
The two tools agree. The stdlib typecheck failure I built the diagnosis on happened once, early, and I never re-ran it — many
make cleancycles later it does not reproduce.What actually caused #668's dead code to reach CI
Not a tool mismatch. I stopped running lint after one failure and did not try again, then treated "lint is unavailable locally" as established and verified with
vet+testfor the rest of the session — neither of which can see unused code. The gap was in the habit, not the tooling.What the real divergence is
Local reports 8 issues: 4 in
mem(now fixed) and 4 incmd/writ(#669). CI reported only the 4 inmem.So the same tools disagree across environments — a toolchain difference, not a tool difference. Likely the golangci-lint version, its staticcheck vintage, or the Go version driving type inference.
Restated items
complexityandverify-ldflagsjoin CI. Unchanged, and still the most serious gap —verify-ldflagsexists because unstamped releases shipped for months, and no workflow runs it.One lint tool→ Pin the linter. The tools agree; the environments do not. Pin the golangci-lint version in CI and locally so both report the same set, and decide which vintage is authoritative. Until then "green in CI" is weaker than "green locally" by an unknown margin.make check-allfor the local opt-in. Scope corrected below.Scope correction on item 4
Most of the matrix needs no setup at all. Measured on darwin:
make build-allmake vet-allmake lint-allCross-compilation is pure Go, so all three static sweeps already run locally. What needs real machines is cross-execution —
test,test-race, andtest-scenarioon linux and windows — which is the "build once, test many" thread in #598.So
make check-allcan land now covering the static matrix, with the execution matrix deferred behind the connection points.Also noted:
lint-allinvokes baregolangci-lintper GOOS whilelintnow invokesstar lint go. If item 3 settles on a pinned linter, both should route through the same entry point.David-Noble-at-work commentedon Aug 26, 2026
The divergence, pinpointed (from #669, closed 2026-08-25)
#669 produced a worked example of the two gates disagreeing, and measurement identifies the cause. It is
version skew, not tool choice.
ci.yaml:32,:130,:168)ci.yaml:73)CI already runs both linters:
./build/star lint go ./...—ci.yaml:89make lint-all—ci.yaml:95, golangci-lint across linux, darwin, and windowsLocal
make checkruns neither of those: itslinttarget is a baregolangci-lint run, host GOOS only.The gaps run in both directions
SA4023findings that v2.13.1 on Go 1.27reports. Analysis on staticcheck flags a possible typed-nil interface guard in writ — locally, but not in CI #669 confirmed all four were real dead code, not false positives — so CI let genuine
defects through, and will begin failing on them at the next toolchain bump.
make checkis weaker on GOOS. It runslint(host only) where CI runslint-all(three platforms).A developer can be green locally and red in CI on a platform-conditional finding.
What that implies for this issue's requirement
"CI is a strict superset of local checks by default" needs two things, not one:
make checkmust call what CI calls —star lint gopluslint-all, not the bare host-onlylintside only guarantees they drift, and drift in the direction observed here means CI is the weaker gate — the
opposite of the stated requirement.
The second point is the one this issue did not previously name: pinning CI alone cannot make CI a superset.
Whichever side runs the newer analyzer finds more, so the pin has to be shared or the comparison is undefined.
chore: sandbox vocabulary, lint-gate parity, and the access-directive…
fix(goast): two declarations named LoadSourceFile, one of which had t…
David-Noble-at-work commentedon Aug 26, 2026
Second concrete instance of this gap, worth recording because it cost a red CI run.
make checkrunstest, nottest-race, and no scenarios. On #697 that meant every green reported locally was green on a strict subset of what CI runs — and the thing it did not run is exactly what failed:pkg/op/claimchecktimed out under-raceon darwin-arm64 and windows-amd64 while passingmake checkcleanly every time.Measured just now on this machine, same test:
make test(no race)make test-racemake test-raceA ~25x race multiplier on a type-checking load is not something
make checkcan hint at, because it never builds with-raceat all. Any developer following the documented gate would have shipped this.Suggests
checkshould either includetest-race, or say out loud that it is a subset and name what CI adds.David-Noble-at-work commentedon Aug 26, 2026
Related and now filed separately: #702.
make checkbeing a subset of CI is one failure mode; the other is that CI itself does not run deterministically — codegen is mtime-gated, so generator validations fire on some legs and not others. Both produce a green that means nothing, from opposite directions.Providers claim what they guarantee, and a type-checked pass holds th…
build: the shell gate follows Declare-BashScript, and discovers throu…