Skip to content

build: one host builds every platform; PLATFORM selects, dist ships all - #654

Merged
David-Noble-at-work merged 2 commits into
developfrom
cross-compile-matrix
Aug 25, 2026
Merged

David-Noble-at-work merged 2 commits into
developfrom
cross-compile-matrix

Conversation

@David-Noble-at-work

Copy link
Copy Markdown
Contributor

make build now produces every platform's binaries from one host. The codebase is pure Go, so the whole
matrix cross-compiles with GOOS/GOARCH alone — no per-platform toolchains, no build VMs.

Closes #598. Unblocked by #639 (merged), which stopped the ambient GOOS from cross-compiling the
generators themselves.

Layout

build/darwin-arm64/{lore,star,writ,devlore-test}
build/linux-amd64/{lore,star,writ,devlore-test}
build/windows-arm64/{lore,star,writ,devlore-test}.exe        ... 6 platforms
build/{devlore-docs,devlore-index,devlore-inventory}         host-only tools
build/star                                                    host tool, $(STAR)

Products ship, so they are built once per $(PLATFORMS) entry. Tools act on this checkout and
nowhere else, so they are built once for the host and stay flat. star is in both: the product copy
ships beside the others, while build/star remains what $(STAR) drives codegen with.

Keeping the host star where it was means CI's five ./build/star call sites (ci.yaml,
knowledge-extract.yaml) are untouched — and it retires the path collision left standing in #653, since
nothing else writes build/star any more.

Matrix

PLATFORMS (Makefile:71, six entries) drives both build and dist-all — ruled: one list, one meaning
of "supported platform"
. #598's body named four; the existing six won because build and dist disagreeing
about what is supported is exactly the drift that bites later. make build PLATFORMS=darwin/arm64
restricts the inner loop.

The version-stamp guard

It executes writ to prove -X bound to real symbols — the failure that shipped unnoticed until
2026-08-16. It now runs against the host's own product, and announces a skip when a restricting
override leaves the host out of the matrix:

darwin/arm64 is not in PLATFORMS: skipping the version-stamp check (nothing built here can run here)

Two consumers moved with the layout

cmd/scenario/selfinstall_scenario_test.go and cmd/writ/scenario_integration_test.go resolved binaries
at ../../build/<tool>; both now resolve ../../build/<goos>-<goarch>/<tool> via runtime.GOOS/GOARCH.

Two targets narrowed to the host

test-scenario and docs requested build as a prerequisite, which would now cross-compile 20 binaries
neither can execute — and CI runs test-scenario on three platforms. Both now invoke the documented
relief valve, $(MAKE) build PLATFORMS=$(HOST_GOOS)/$(HOST_GOARCH). Verified: make test-scenario builds
one platform, not six.

Verification

Result
make build from clean 0 — 6 platforms x 4 products, plus 3 host tools
Architectures spot-checked build/star Mach-O arm64; linux-amd64/writ ELF x86-64; windows-arm64/lore.exe PE32+ Aarch64
Version-stamp check ran and passed (not silently skipped)
make build PLATFORMS=darwin/arm64 0 — one platform only
make build PLATFORMS=linux/amd64 0 — skip announced, host excluded
Second consecutive build zero codegen invocations (#572's property, preserved)
make test-scenario 0 — and builds one platform, not six
make dist 0 — all six archives plus checksums
make test 103 ok / 0 fail
make vet darwin / linux / windows clean
make docs, complexity, shell-lint, verify-ldflags 0
gofmt -l clean

Not done here

dist-all: build still holds, so make dist builds the matrix and then cross-compiles writ/lore
again for its archives. The dependency exists for the stamp proof, not the binaries, and dist could now
archive from build/<goos>-<goarch>/ instead. That is a change to the release path, so it wants its own
PR rather than riding along with this one.

@David-Noble-at-work David-Noble-at-work self-assigned this Aug 24, 2026
@David-Noble-at-work David-Noble-at-work changed the title build: one host builds every platform, products under build/<goos>-<goarch> build: one host builds every platform; PLATFORM selects, dist ships all Aug 25, 2026
@David-Noble-at-work

Copy link
Copy Markdown
Contributor Author

Four follow-ups from review, all in one commit.

1. PLATFORM selects; the defaults differ by target

make build                                   # this machine
make build PLATFORM=linux/amd64
make build PLATFORM=all                      # every supported platform
make dist                                    # every supported platform
make dist  PLATFORM="linux/amd64 darwin/arm64"

ALL_PLATFORMS names what the project supports. PLATFORM is the caller's selection and stays empty
unless named, because you build what you are about to run, and you ship everything — opposite
defaults, so the selection resolves per target through $(call select,<default>) rather than through a
variable with one default.

Measured, warm: 16.5s for the host against 47.5s for the full matrix. Full-matrix-by-default was
a ~3x tax on every local build, and the relief valve only ever helped developers who remembered it.
Cross-compilation is now one word away instead.

override turned out to be required, not stylistic: PLATFORM=all arrives on the command line, and a
command-line variable wins over any plain makefile assignment. The first attempt reassigned it inside an
ifeq, which make silently ignored — producing build/all-all/ and a failed build. Caught by running
it; recorded in the comment so the next person does not rediscover it.

2. devlore-test is not a shipping product

Moved from PRODUCTS to TOOLS. It is the graph test harness and it does not ship today, so
cross-compiling it produced binaries nobody can run and nothing distributes. build/<platform>/ now
holds exactly lore, star, writ.

Checked before moving it: the self-install scenario resolves lore, star, writ — not
devlore-test.

3. Flat host copies for lore and writ

New HOST_COPIES. Products that also get run from this checkout keep a flat build/<name> beside their
per-platform copy, so "the binary that runs on this machine" has one path that does not shift with the
host triple — the guarantee $(STAR) has always relied on for build/star.

star is deliberately absent: build/star comes from the star: target via the build/star: FORCE
bootstrap, which must run before codegen and therefore before this list is reached. Adding it would
build identical bytes twice and put two recipes in charge of one path.

Verified byte-identical to their host-platform products.

4. One make run, one stamp; host tools are static

  • CGO_ENABLED=0 folded into HOST_GO. The matrix built products with it while the star: target
    inherited the ambient default — cgo on. build/star and build/<host>/star were genuinely
    different binaries, one linking the cgo implementations of net and os/user and one the pure-Go
    ones. That was introduced by this PR's first commit; folding the flag into HOST_GO fixes it for
    every host tool at once.

  • export VERSION COMMIT BUILD_DATE. build/star: FORCE shells out to $(MAKE) star, and a
    sub-make re-parses this file — a second date, a second git describe. The two binaries agreed
    locally only because a warm Go cache kept both inside one tick of date; a cold cache or a loaded
    runner separates them. Exporting puts the frozen values in the sub-make's environment, where the ?=
    guards find them already defined.

    This extends the simply-expanded fix at Makefile:44-51 rather than reversing it. That fix stopped
    re-evaluation within a run — the Windows runner comparing 8a62b17-dirty against a binary reporting
    8a62b17. The sub-make is the one boundary it did not reach, and the freeze comment already describes
    the mechanism used here: "?= above still honors an override from the environment."

    Verified by mechanism, not coincidence: with BUILD_DATE=2000-01-01T00:00:00Z in the environment, the
    sub-make's build/star and the cross-compiled products all report that value, read out of the linker
    flags with go version -m.

5. Two recursive-make hacks removed

test-scenario and docs had been narrowed with $(MAKE) build PLATFORM=<host>. Host-only is now the
default, so both are plain : build prerequisites again.

Verification

make build default / PLATFORM=all / PLATFORM=linux/amd64 / PLATFORM="linux/amd64 darwin/arm64" —
all 0, each producing exactly the selected directories, with the stamp-check skip announced when the host
is excluded. make dist 0 (six archives), make dist PLATFORM=linux/amd64 0 (one). make test 103 ok ·
make vet darwin/linux/windows · test-scenario · docs · complexity · shell-lint ·
verify-ldflags · gofmt -l — all clean.

@David-Noble-at-work

Copy link
Copy Markdown
Contributor Author

Correction to the follow-up comment above: it says "override turned out to be required, not stylistic." There is no override in the merged design, and none is needed.

That sentence describes a discarded intermediate. The first attempt made PLATFORM a variable with a default and tried to expand all by reassigning it inside an ifeq — which make silently ignores for a command-line variable, producing build/all-all/ and a failed build. override would have fixed that design.

The design that landed does not reassign PLATFORM at all. select reads it at expansion time and filters all there:

PLATFORM ?=
select = $(if $(PLATFORM),$(if $(filter all,$(PLATFORM)),$(ALL_PLATFORMS),$(PLATFORM)),$(1))

A function was needed anyway, because build and dist want opposite defaults and one variable cannot hold two. Sidestepping the command-line-precedence problem is a consequence of that, not a separate fix. The Makefile comment at the select definition states this correctly; only the PR comment was wrong.

@David-Noble-at-work
David-Noble-at-work merged commit 728d28e into develop Aug 25, 2026
8 checks passed
@David-Noble-at-work
David-Noble-at-work deleted the cross-compile-matrix branch August 25, 2026 00:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

make build cross-compiles for the selected platforms; dist ships every supported one

1 participant