Skip to content

Speed up CI and make its builds visible - #16

Merged
lhotari merged 1 commit into
mainfrom
ci-improvements
Sep 23, 2026
Merged

lhotari merged 1 commit into
mainfrom
ci-improvements

Conversation

@lhotari

@lhotari lhotari commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator

CI improvements: Gradle build visibility and caching, Docker layer caching, a dependency-submission workflow, and per-fixture test progress in the console.

Gradle

  • setup-gradle (gradle/actions v6.3.0, the latest release, SHA-pinned) switches to its default Enhanced Caching provider, which is free for public repositories, instead of cache-provider: basic. Per Caching build state between Jobs, the default is kept: cache entries are written from main only and every other run restores them read-only. PR-scoped entries could only be reused by re-runs of the same PR and would evict main's.
  • Gradle build cache is enabled (org.gradle.caching=true), so its entries travel in the Gradle User Home the action caches. Only cacheable task types (JavaCompile, Javadoc, and similar) use it. The Docker builds (Exec) and the fixtures (JavaExec) always run.
  • Build visibility: every Gradle build publishes a Build Scan to scans.gradle.com (terms of use accepted in the workflow). The job summary lists each build with its scan link, and a failing build is also posted as a PR comment (add-job-summary-as-pr-comment: on-failure). The comment needs pull-requests: write, which ci.yml and the release workflow's build-and-verify job now grant to the reusable workflow.
  • Dependency submission: the new dependency-submission.yml runs gradle/actions/dependency-submission on pushes to main (and manually). It submits the resolved Gradle dependencies to GitHub's dependency graph, which Dependabot alerts and security updates for them use. ⚠️ Dependabot alerts are currently disabled in the repository settings (the API returns 404), so enable them under Settings → Code security for the alerts to appear.

Docker layer caching

  • The native bundle builds (Gradle Exec → docker buildx build) take -PdockerCache=gha to restore all stages from the GitHub Actions cache (--cache-from type=gha,scope=native-bundle-<platform>). With -PdockerCacheWrite=true they also export them (mode=max, ignore-error=true). CI writes only on main, for the same reason as the Gradle cache.
  • docker/setup-buildx-action (v4.4.1, use: false) creates a docker-container builder, which the builds select through BUILDX_BUILDER; the default builder stays in use for everything else. crazy-max/ghaction-github-runtime (v4.0.0) exposes the Actions cache runtime that type=gha needs.
  • The packaged smoke's runtime image uses the same cache (docker buildx build --load) when JONOFFCPU_DOCKER_CACHE=gha.
  • Dockerfile change: both native bundle Dockerfiles now compile the collector's Cargo dependencies in their own stage from a placeholder crate. These include the vendored libbpf/elfutils/zlib, the slowest part of the build. The placeholder has an empty build.rs, so the build dependencies compile there too, and the stage uses the same RUSTFLAGS as the real build; the musl flags now live in one file used by both. The placeholder's fingerprints are removed so the real crate always rebuilds. Measured locally (x86-64 glibc + musl, collector-only change): 102 s → 32 s, with libbpf-sys no longer recompiled.

Test progress in the console

The fixtures are main-based JavaExec tasks, not Test tasks, so Gradle's testLogging doesn't apply. Instead:

  • every fixture task logs <class> STARTED and <class> PASSED (<s> s) (root build script);
  • fixtures made of scenario methods report each scenario as <Fixture> > <scenario> STARTED / PASSED (<ms> ms) / FAILED (<ms> ms): <error> through a small FixtureSteps helper. That covers 62 scenarios in SignalCaptureControllerTest, StackProfileTest, StreamingCorrelatorTest, PartialCorrelatorTest, PrimitiveStructuresTest, StackTransformsTest, CommandLineTest and FixtureAcceptanceTest. Fixtures with a single inline main report at class level.

Verification

  • ./gradlew spotlessCheck :jonoffcpu-agent:check :jonoffcpu-correlator:check -PnativeArchitectures=x86_64 -PnativeLibcs=all passes locally with the new Dockerfiles and the build cache enabled, with the progress lines in the output.
  • The packaged smoke passes on x86-64 with both new bundles: glibc (1,480 matched samples) and musl (1,499).
  • -PdockerCache=bogus fails with Unsupported dockerCache value 'bogus'; expected none or gha.
  • actionlint reports only the ubuntu-26.04 runner labels it doesn't know yet.
  • Not verifiable before merge: this PR's CI run exercises the cache restore path, which finds nothing on the first run, plus Build Scans, job summaries and the buildx builder. The cache export path, and the dependency-submission workflow, first run on main after merge.

Gradle in GitHub Actions:
- setup-gradle (gradle/actions v6.3.0, the latest release) uses its
  default enhanced cache provider, which is free for public repositories,
  instead of the basic one. It saves the Gradle User Home, the local build
  cache included, from main only, and every other run restores it
  read-only, as the action recommends.
- Gradle's build cache is on (org.gradle.caching), so compiled classes,
  generated code and javadoc come from that cache. Only cacheable task
  types use it; the Docker builds and the fixtures always run.
- Every Gradle build publishes a Build Scan to scans.gradle.com, the job
  summary lists the builds with their scans, and a failing build is also
  summarized as a pull request comment (pull-requests: write, granted by
  both callers of the reusable workflow).
- A new dependency-submission workflow submits the resolved Gradle
  dependencies to GitHub's dependency graph on every push to main, which
  Dependabot alerts and security updates for them are based on.

Docker layer caching:
- The native bundle builds, which Gradle runs with docker buildx, restore
  every stage from the GitHub Actions cache with -PdockerCache=gha, and
  export them with mode=max when -PdockerCacheWrite=true. CI writes only
  on main, like the Gradle cache, so pull requests read main's layers
  without evicting them. A docker-container buildx builder, selected
  through BUILDX_BUILDER, and ghaction-github-runtime provide what the
  gha cache needs; the default builder stays in use for everything else.
  The packaged smoke's runtime image uses the same cache.
- Both native bundle Dockerfiles now compile the collector's Cargo
  dependencies, the vendored libbpf, elfutils and zlib among them, in a
  stage of their own from a placeholder crate, with the same RUSTFLAGS as
  the real build. A collector change then rebuilds only the collector:
  locally, the x86-64 glibc and musl bundles rebuild in 32 s instead of
  102 s after a collector-only change.

Test progress: the fixtures are main-based JavaExec tasks, so Gradle's
test logging does not apply. Each fixture task now reports its class as
STARTED and PASSED with its duration, and fixtures made of scenario
methods report each scenario as STARTED, PASSED or FAILED with its time
(FixtureSteps).
@lhotari
lhotari added this pull request to stack #18 September 23, 2026 21:52
@lhotari
lhotari merged commit 261ab6e into main Sep 23, 2026
5 checks passed
@lhotari
lhotari deleted the ci-improvements branch September 24, 2026 13:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant