Skip to content

Add cross-platform compiler distribution - #103

Merged
itsfuad merged 7 commits into
mainfrom
feature/cross-platform-distribution
Aug 29, 2026
Merged

Add cross-platform compiler distribution#103
itsfuad merged 7 commits into
mainfrom
feature/cross-platform-distribution

Conversation

@itsfuad

@itsfuad itsfuad commented Aug 29, 2026

Copy link
Copy Markdown
Member

Summary

  • add native compiler distributions for Linux, macOS, and Windows on amd64 and arm64
  • introduce canonical managed toolchain profiles, object/link stages, and private peeper_rt_v1_* runtime ABI
  • add deterministic compiler/toolchain packs, signed manifests, secure installer activation, and peeper doctor
  • add six-host CI plus signed draft-release workflow with SBOMs, checksums, and provenance attestations
  • normalize compiler-owned source paths and native executable naming across supported hosts

Validation

  • bash scripts/build.sh
  • go test -count=1 ./...
  • go vet ./...
  • PEEPER_BIN=$PWD/build/bin/peeper go test -count=1 ./x_test
  • git diff --check
  • GitHub CI run 33259245296: race detector and all six native host jobs passed

Release gates

  • merge does not publish a release
  • first v* tag requires configured Ed25519, Apple signing/notarization, and Windows Authenticode credentials
  • release workflow creates a draft only; installers, signatures, checksums, SBOMs, attestations, and native smoke tests must be reviewed before publication
  • v1 guarantees native compilation only; foreign cross-linking remains out of scope

Tracks #102.

itsfuad added 7 commits August 29, 2026 20:13
@itsfuad
Add cross-platform compiler distribution
704f0db 
Add native six-host CI and signed release packaging for Linux, macOS, and Windows on amd64 and arm64.

Introduce canonical managed toolchain profiles, reproducible packs, secure installation, private runtime ABI, release manifests, doctor diagnostics, and distribution documentation.

Distribution and profile helpers centralize archive safety, compatibility, release completeness, and linker policy invariants. Validation passed with full tests, vet, race tests, scripts/build.sh, bundled fixtures, package smoke, actionlint, and shell checks.
@itsfuad
Fix native CI path handling
e8691bd 
Normalize linker response-file paths for Windows, avoid unsupported Windows directory fsync, and compare installation roots using platform canonical paths.

Focused package tests, full go test ./..., and scripts/build.sh pass on Linux.
@itsfuad
Document directory sync portability
2f596e7 
Explain Unix rename durability and Windows directory fsync limitations in syncDirectory.
@itsfuad
Normalize Windows compiler paths
c7008c2 
Canonicalize LSP cache keys, normalize LLVM debug paths, and remove Unix-only filesystem assumptions from cross-platform tests.

Full go test ./... and scripts/build.sh pass on Linux.
@itsfuad
Remove redundant LSP cache path normalization
7aa377d
@itsfuad
Canonicalize compiler source paths at ownership boundaries
6aa7cf2
@itsfuad
Use native executable suffix in source fixtures
0687a21
@itsfuad itsfuad moved this from Todo to In Progress in Peeper Roadmap Aug 29, 2026
@itsfuad
itsfuad requested a lite review from Copilot August 29, 2026 15:14
Copilot AI reviewed Aug 29, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Large, security-sensitive change spanning installer, packaging, runtime ABI, and CI/release workflows needs careful human verification beyond automated review.

Pull request overview

Add cross-platform, signed, deterministic Peeper compiler distribution pipeline: native runtime ABI (peeper_rt_v1_*), managed toolchain profiles, secure installer, and multi-host CI/release workflows.

Changes:

  • Add managed toolchain profile resolution + native object/link stages, plus peeper doctor installation diagnostics.
  • Add deterministic pack/manifest format, Ed25519-signed release manifest, secure installer activation, and toolchain lockfile.
  • Add 6-host CI plus tag-triggered draft-release workflow producing packs, SBOMs, checksums, and attestations.
File summaries
File Description
x_test/negative_scalar_shrink_bad_free/src/hijack.peep Update negative fixture to new runtime ABI symbol.
x_test/negative_print_receiver_printf_symbol/src/main.peep Update negative fixture to new runtime ABI symbol.
x_test/negative_print_printf_symbol/src/main.peep Update negative fixture to new runtime ABI symbol + naming.
x_test/negative_invalid_free_abi/src/main.peep Update negative fixture externs to runtime ABI.
x_test/negative_imported_printf_symbol/src/hijack.peep Update negative fixture to new runtime ABI symbol.
x_test/fixtures_test.go Make fixture executable naming OS-aware.
scripts/package-release.sh Package compiler/target/toolchain packs + bootstrap installer.
scripts/bundle.go Bundle runtime archive and OS-specific compiler binary name.
runtime/peeper_rt.h Define runtime ABI C header for bundled runtime.
runtime/peeper_rt.c Implement runtime ABI functions across platforms.
README.md Document binary install flow and new build entrypoint.
pkg/peeper/install.go Compute installation root from compiler executable path.
pkg/peeper/install_test.go Tests for installation-root resolution behavior.
pkg/manifest/write.go Make atomic directory fsync Windows-tolerant.
pkg/manifest/write_test.go Adjust mode assertions for Windows semantics.
pkg/manifest/manifest_test.go Adjust mode assertions for Windows semantics.
internal/toolchain/profile.go Add managed/system toolchain profile model + args/response file.
internal/toolchain/profile_test.go Test profile parsing, resolution, and argument construction.
internal/target/wordsize_test.go Update expected Linux amd64 triple to musl default.
internal/target/llvm_triple.go Switch canonical triples (musl + mingw) + add SystemLLVMTriple.
internal/target/llvm_triple_test.go Update triple tests; add SystemLLVMTriple coverage.
internal/project/modules.go Canonicalize module file paths once for stable indexing.
internal/project/modules_test.go Test AddModule canonicalizes FilePath.
internal/project/imports.go Canonicalize resolved import absolute paths.
internal/project/imports_test.go Update expected resolved paths to canonical form.
internal/project/context.go Resolve packaged libs relative to install root.
internal/project/context_test.go Update tests for new install-root based layout.
internal/pipeline/pipeline_test.go Update pipeline tests to new runtime ABI symbols.
internal/lsp/workspace.go Use canonical paths consistently for workspace indexing.
internal/lsp/workspace_test.go Update tests to use applyDocumentSnapshot overlay path flow.
internal/lsp/state.go Align cache path comparisons with canonical cache keys.
internal/lsp/server_test.go Update tests to use applyDocumentSnapshot for overlays.
internal/lsp/completion.go Canonicalize current file once for overlay filtering.
internal/lsp/completion_test.go Update completion tests to use applyDocumentSnapshot.
internal/lsp/benchmark_test.go Update benchmark overlays to use applyDocumentSnapshot.
internal/installer/install.go Secure installer: verify signed manifest, download, hash, extract, activate.
internal/installer/install_test.go Installer tests for success, hash failure rollback, truncation rejection.
internal/distribution/toolchains_lock_test.go Validate toolchain lockfile schema and required assets.
internal/distribution/release.go Build/sign/verify release manifest and host install sets.
internal/distribution/release_test.go Tests for manifest signing/verification and host set selection.
internal/distribution/pack.go Deterministic pack writer with embedded manifest and hash inventory.
internal/distribution/pack_test.go Tests for deterministic pack bytes and symlink constraints.
internal/distribution/extract.go Safe pack extraction with manifest-verified paths, sizes, hashes.
internal/distribution/extract_test.go Extraction tests: verification, traversal rejection, symlink-parent rejection.
internal/backend/llvm/instruction_emit.go Emit print calls via runtime ABI printf symbol.
internal/backend/llvm/emitter.go Reserve/declare runtime ABI symbols; update alloc/free handling.
internal/backend/llvm/emitter_test.go Update tests for runtime ABI symbols, triples, and debug paths.
internal/backend/llvm/debug_info.go Use slash-separated path ops for canonicalized source paths.
docs/distribution.md Document distribution model, security boundaries, CI/release gates.
docs/diagrams/cli-flow-detailed.d2 Update diagram labels to new staging/link flow.
distribution/toolchains.lock.json Add pinned toolchain assets with size/SHA metadata.
cmd/sign-release/main.go CLI to sign release manifest using Ed25519 private key env var.
cmd/release-profile/main.go Generate and validate managed toolchain profile.json for staged install.
cmd/release-index/main.go Assemble release manifest from pack result JSON files.
cmd/peeper-installer/main.go Bootstrap installer CLI embedding manifest URL + public key.
cmd/peeper-installer/main_test.go Test default install root is user-scoped.
cmd/init_subprocess_test.go Build CLI via build.sh and use OS-aware executable suffix.
cmd/dump.go Rename artifact staging helper to replacePath.
cmd/doctor.go Add peeper doctor command and installation inspection logic.
cmd/doctor_test.go Tests for doctor inspection of managed toolchain install.
cmd/distpack/main.go CLI wrapper to produce deterministic packs + manifest JSON.
cmd/dispatch.go Register doctor command in CLI dispatcher.
cmd/cli/init_test.go Adjust mode assertions for Windows semantics.
cmd/build.go Switch build to object/link via toolchain profile + response file + staging.
cmd/build_test.go Replace clang-args tests with replacePath behavior test.
.github/workflows/release.yml Add tag workflow: stage, sign, pack, SBOMs, attest, draft release publish.
.github/workflows/ci.yml Add 6-host native CI, bundle build, doctor, fixtures; keep race gate on linux.
.github/workflows/build-release-host.yml Reusable workflow to stage host install + pinned toolchain + runtime build.
_builtin_library/core/src/global.peep Route core IO/exit externs through runtime ABI symbols.
_builtin_library/core/src/allocator.peep Route allocator externs through runtime ABI symbols.
Review details
  • Files reviewed: 70/70 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread internal/toolchain/profile.go
Comment on lines +284 to +303
func (profile Profile) WriteResponseFile(path string, objectPaths []string) error {
entries := make([]string, 0, len(objectPaths)+1)
entries = append(entries, objectPaths...)
if profile.RuntimeArchive != "" {
entries = append(entries, profile.RuntimeArchive)
}
var response strings.Builder
for _, entry := range entries {
argument, err := responseArgument(entry)
if err != nil {
return err
}
response.WriteString(argument)
response.WriteByte('\n')
}
if err := os.WriteFile(path, []byte(response.String()), 0o600); err != nil {
return fmt.Errorf("write linker response file: %w", err)
}
return nil
}
@itsfuad
itsfuad merged commit acb405a into main Aug 29, 2026
15 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in Peeper Roadmap Aug 29, 2026
@itsfuad itsfuad mentioned this pull request Aug 29, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments

Assignees

No one assigned

Labels

None yet

Projects

Peeper Roadmap
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@itsfuad