Repository navigation
Codegen runs every time, and its output must match what is committed - #705
Merged
Merged
Conversation
added 4 commits
August 26, 2026 17:44
…at was removed Narrowing the test to one platform per CI leg left the doc comment on CheckGOOS making the case for the four-platform sweep it replaced: Verifying only the developer's platform would leave the others to CI, and CI runs one of them. That is now both stale and an argument against the design around it -- a reader looking for why the sweep is gone finds a paragraph explaining why it should not be. It also misdescribes CI, which runs five legs spanning three platforms rather than one of them. Rewritten to say where the coverage actually comes from: each leg checks its own host, the legs span darwin, linux, and windows between them, and an explicit goos is for sweeping the rest from one machine.
Codegen is mtime-gated, and a fresh clone stamps every file at checkout -- so whether a provider looks stale is decided by write order, not by its source. Measured on CI run 33024633546, ONE commit across six jobs: test (darwin-arm64) 159 files 30 providers quality-gate 9 files 1 provider test (linux-amd64) 9 files 1 provider test (linux-arm64) 0 files 0 providers test (windows-amd64) 0 files 0 providers test (windows-arm64) 0 files 0 providers Nothing varied but which runner took the job. Half of them ran no provider generator at all, and every generator validation rides along with that -- so which defects get caught was luck. #697 has the receipt: darwin regenerated goast, caught a duplicate declaration through validate_action_name_consts, and failed; linux skipped it and passed. Neither declaration was build-tagged. Touching sources, not deleting outputs The obvious lever is `rm -f $(NEW_OP_INVENTORY)` -- one output per provider, each the head of a grouped rule, list already maintained. It cannot work. The generated files hold the op.Announce*() call sites devlore-inventory scans, inventory is a prerequisite of the star binary, and `build/star: FORCE` rebuilds star every run. THE OUTPUTS ARE ALSO INPUTS, so deleting them destroys the bootstrap that would regenerate them. Tried it: inventory found 8 packages instead of 19, then failed with `no op.Announce*() call sites found`. Deletion would be safe with an LKG present -- STAR resolves to build/star.lkg when it exists and build/star: FORCE is never consulted. Not taken: CI has no LKG, and a check that depends on an artifact CI lacks is not a check. `make -B generate` also forces the rules and forces far more than the question needs, propagating into the build/star sub-make. Verified make regenerate 159 files, 30 providers -- matches the darwin leg exactly make generate 0 files -- the mtime gate still skips unchanged providers check / test-race / test-scenario all pass
…duces Generated files are committed, CI never ran `generate` explicitly, and no step asserted that generation produced no diff. Two consequences, both live until now: A stale committed .gen.go passed CI. Edit a provider.go, forget to regenerate, commit -- if the mtimes fell the wrong way, nothing noticed. When codegen DID run, CI tested the regenerated output rather than the committed output. The artifact under test was not the artifact in the repository, and the rewrite happened inside the build where nobody saw it. The fix already existed in this file, applied to go.mod: run the tool, then assert nothing changed. `make regenerate` forces every rule; `git diff --exit-code` holds the committed tree to it. One leg is the right scope Generated output is platform-independent -- it has to be, since the repository holds one copy. Confirmed rather than assumed: regenerating all 159 files on darwin reproduced the committed tree byte for byte. Forcing regeneration on quality-gate makes the generator validations fire deterministically there, and the diff makes every other leg's mtime-gated build safe, because the committed files are then known current. A deletion the generator no longer emits also fails this check -- the #572 class of bug, caught for free. Proved it can fail before trusting it A check that has only ever passed is not a check. Removing one +devlore:claim directive from ui.Note changed pkg/op/provider/ui/gen/provider.gen.go, which the diff catches. Restored byte-identical afterward. Worth recording: the first probe edited a method doc comment and changed NOTHING -- doc comments do not reach the generated artifacts. Stopping there would have concluded the check was broken.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #702.
Codegen is mtime-gated, and a fresh clone stamps every file at checkout — so whether a provider looks
stale is decided by write order rather than by its source. Measured on run
33024633546, one commit across
six jobs:
test (darwin-arm64)quality-gatetest (linux-amd64)test (linux-arm64)test (windows-amd64)test (windows-arm64)Nothing varied but which runner took the job. Half ran no provider generator at all, and every
generator validation rides along with that — so which defects get caught was luck.
#697 has the receipt: darwin regenerated
goast, caught a duplicate declaration throughvalidate_action_name_consts, and failed. Linux skipped it and passed. Neither declaration wasbuild-tagged.
Underneath it, a second hole
Generated files are committed, CI never ran
generateexplicitly, and nothing asserted thatgeneration produced no diff. So a stale committed
.gen.gopassed outright — and when codegen didrun, CI tested the regenerated output rather than the committed output. The artifact under test was not
the artifact in the repository, and the rewrite happened inside the build where nobody saw it.
The fix already existed in the same workflow file
go.modis guarded the way generated files were not: run the tool, assert nothing changed.make regenerateforces every rule; the diff holds the committed tree to it.Touching sources, not deleting outputs
The obvious lever was
rm -f $(NEW_OP_INVENTORY)— one output per provider, each the head of a groupedrule, list already maintained. It cannot work. The generated files hold the
op.Announce*()callsites
devlore-inventoryscans,inventoryis a prerequisite of the star binary, andbuild/star: FORCErebuilds star every run. The outputs are also inputs, so deleting them destroys thebootstrap that would regenerate them. Confirmed by doing it: inventory found 8 packages instead of 19,
then failed with
no op.Announce*() call sites found.Deletion would be safe with an LKG present —
STARresolves tobuild/star.lkgwhen it exists andbuild/star: FORCEis never consulted. Not taken: CI has no LKG, and a check that depends on anartifact CI lacks is not a check.
Proved it can fail before trusting it
A check that has only ever passed is not a check. Removing one
+devlore:claimdirective fromui.Notechanged
pkg/op/provider/ui/gen/provider.gen.go, which the diff catches. Restored byte-identical after.Worth recording: the first probe edited a method doc comment and changed nothing — doc comments do
not reach the generated artifacts. Stopping there would have concluded the check was broken.
One leg is the right scope
Generated output is platform-independent — it has to be, since the repository holds one copy. Confirmed
rather than assumed: regenerating all 159 files on darwin reproduced the committed tree byte for byte.
A deletion the generator no longer emits also fails this check — the #572 class of bug, caught for free.
Verified
make regeneratemake generatemake checkmake test-racemake test-scenarioAlso included
One unrelated commit that rode along from a branch switch: the
CheckGOOSdoc comment still argued forthe four-platform sweep #697 removed, and misdescribed CI as running one platform.