Skip to content

ci(release): shorten the release pipeline without dropping a check - #499

Merged
bomly-guy merged 5 commits into
mainfrom
ci/faster-release-pipeline
Oct 4, 2026
Merged

bomly-guy merged 5 commits into
mainfrom
ci/faster-release-pipeline

Conversation

@bomly-guy

@bomly-guy bomly-guy commented Oct 4, 2026 •

Copy link
Copy Markdown
Collaborator

Why

Measured on v0.28.1, the pipeline took about 51 minutes from starting Auto Version to the report being committed: 16 for the prerequisites stage, 29 for Release, 6 for the assessment. Most of the wait was work done twice, or work queued behind something it did not need.

What changes

Release prerequisites (was 16.3 min, bounded by the Windows unit suite at 15.1)

  • The unit suite still runs on Linux, macOS and Windows. It runs once per platform; the repeat is removed, since the same suite also runs on every pull request, on every push to main and in the release workflow. The public wording of the unit-portable claim is updated to match. Modules are downloaded, with retries, before the suite, after a macOS runner timed out reaching the module proxy mid-test.
  • The 12 cross-builds run as three jobs, one per target operating system, instead of sequentially inside the Java stability job (11.2 min). The cross-build check now has three instances. Builds use -trimpath, the flag a release is built with.
  • The two fuzz targets that compile most of the module graph (engine, plugin) start first. In the last run the other 28 targets were done after 5.5 minutes and the job then spent 6 more compiling those two alone. Those two now run as their own job beside the 28 parser targets; the fuzz check reports two instances.

Release (was 28.7 min)

  • validate (the unit suite and vet) is unchanged: it still runs before the build on every release. Two ways of not repeating it were tried in review and removed; the reason is recorded on the job.
  • verify-draft skips the Go cache. Those jobs only go run the assurance tool, which has one small dependency; restoring the cache took 5.1 of the Windows job's 5.9 minutes.

Release assessment: the install-script jobs skip the Go cache for the same reason (3.7 of 4.2 minutes on Windows).

Not changed

No check is removed. Smoke, SBOM interoperability and draft verification are untouched.

Verification

  • make test and make lint pass locally.
  • The prerequisites stage was dispatched on this branch; its timings are in a comment below.
  • The release.yml and assurance-assessment.yml changes cannot run before a release. The assurance tool was run from an empty module and build cache locally to confirm it needs no restored cache.

🤖 Generated with Claude Code

Measured on v0.28.1, tagging to report took about 51 minutes. Most of the
wait was work done twice or work waiting behind something it did not need.

Release prerequisites (16 min, bounded by the Windows unit suite):
- The unit suite still runs on all three platforms, but the second run now
  happens on Linux only. The repeat catches a test that passes once and not
  twice, which is a property of the test rather than of the platform.
- The twelve cross-builds run as three jobs, one per target operating
  system, instead of one after another in the Java stability job. They now
  use -trimpath, the flag a release is built with.
- The two fuzz targets that compile most of the module graph start first,
  so the other 28 fuzz while those compile instead of the job ending on six
  minutes of compiling alone.

Release (29 min):
- The unit tests run beside the build instead of before it. publish still
  needs them to pass.
- The draft-verification jobs skip the Go cache. They only run the
  assurance tool; restoring the cache for it took five minutes on Windows.

Release assessment: the install-script jobs skip the Go cache for the same
reason.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
@coderabbitai

coderabbitai Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 21 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Repository: bomly-dev/bomly-cli/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: dd0ae8e0-3665-47bb-b3c9-b1e4b0fe40cb
📥 Commits

Reviewing files that changed from the base of the PR and between 04c68f3 and 9a4fc7a.

📒 Files selected for processing (11)
  • .github/workflows/assurance-assessment.yml
  • .github/workflows/assurance-prerequisites.yml
  • .github/workflows/fuzz.yml
  • .github/workflows/portable-assurance.yml
  • .github/workflows/release.yml
  • dev-docs/CI.md
  • dev-docs/RELEASE_ASSURANCE.md
  • docs/ASSURANCE.md
  • docs/assurance/catalog.json
  • internal/assurance/aggregate.go
  • scripts/run-fuzz.sh
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Bomly Diff Summary

Compared 04c68f3c2a0d2990101277defd8703636028f945 to 9a4fc7abbcd65fd8fafef1b22ec80995d5fd38c2.

Overview

Status Manifests Dependencies Findings Duration
✅ Pass +0 / ~0 / -0 0 added / 0 version changed / 0 detail changes / 0 removed 0 introduced / 0 persisted / 0 resolved 1m 48s

Dependency Changes

✅ No dependency changes.

Vulnerabilities

✅ No vulnerability changes.

License Changes

✅ No license changes.

Project Posture

✅ No project posture changes (--matchers +scorecard was not selected).

Policy Findings

✅ No policy differences were identified.

@bomly-guy

Copy link
Copy Markdown
Collaborator Author

Measured: the prerequisites stage on this branch (run) passed in 10.3 minutes, against 16.3 for v0.28.1.

Job v0.28.1 This branch
Unit suite, Windows 15.1 min 6.8 min
Unit suite, macOS 12.1 min 7.0 min
Unit suite, Linux (still run twice) 9.1 min 9.2 min
Cross-builds 11.2 min, sequential 3.5 / 3.7 / 4.4 min, in parallel
Fuzzing 11.8 min 9.7 min

Fuzzing and the Linux unit suite now set the total. Fuzzing is bounded by compiling the engine and plugin targets, which finish about two and a half minutes after everything else.

The release and assessment changes are not measured: they cannot run before a release.

🤖 Generated with Claude Code

The unit suite no longer repeats in the release prerequisites stage. The
repeat guarded against a single lucky pass, but the same suite already runs
on every pull request, on every push to main and in the release workflow,
so an intermittent test surfaces there. The unit-portable claim now says
what is done: the suite passes on Linux, macOS and Windows.

Fuzzing was the slowest remaining job, and more targets at once would not
help: it is bounded by compiling the two targets whose packages pull in
most of the module graph. Those two now run as their own job, beside the
28 parser targets, so neither waits on the other. The fuzz check reports
two instances, parsers and engine-and-plugin.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2fd0931d09

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/release.yml Outdated
Comment thread .github/workflows/fuzz.yml
bomly-guy and others added 2 commits October 4, 2026 12:45
…onfirmed

Running the unit suite beside the GoReleaser build let the build finish
first when the suite failed, and GoReleaser opens the Homebrew, Scoop and
WinGet pull requests as it runs. The build waits on validate again.

What made the wait pointless is that preflight has already confirmed the
release prerequisites stage for this commit, which runs the same suite on
three operating systems. validate now skips its steps in that case and runs
them, ahead of the build, when preflight could not confirm it.

Also sum targets_failed across fuzz instances, now that the fuzz check
reports two; the report's top-level figure had dropped out.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
The macOS leg of the prerequisites stage failed on a TLS handshake timeout
to the module proxy: TestPluginDoctorJSON builds a plugin in a temporary
module and had to download the SDK inside the test. It is the second time
a macOS runner has failed this way. The modules are now fetched in their
own step, retried up to three times, so the tests find them in the cache.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e139c923db

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/release.yml Outdated
Comment thread .github/workflows/release.yml Outdated
Two attempts to avoid re-running the suite in the release workflow were
each found unsafe in review, so the job is back to what it was:

- beside the build, GoReleaser could open package-manager pull requests
  for a release whose tests then failed;
- skipped when preflight confirmed the prerequisites stage, a hand-made tag
  on an untested child of a verified commit would be built without tests,
  and vet would not run at all.

The decision is recorded on the job so it is not tried a third time.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
@bomly-guy
bomly-guy merged commit abab0e3 into main Oct 4, 2026
20 checks passed
@bomly-guy
bomly-guy deleted the ci/faster-release-pipeline branch October 4, 2026 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant