Repository navigation
perf(ci): drop the full dev shell from the test job - #1269
Conversation
The test job spent ~73s on "Warm Nix dev shell" realizing the complete development shell — every linter, formatter, renovate, and the pre-commit hook closure — even though pnpm install itself was only ~7s. Now that coverage is a `nix build` derivation, Vitest is the sole step that needs a JS toolchain. Replace `nix develop` with `nix profile install pnpm_11 bun just`, mirroring the perf-comment and pkg-pr-new jobs, then run `just test-vitest` directly. This skips the heavy shell closure while keeping the exact tools Vitest needs.
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes — drops the full dev shell from the test CI job, replacing it with nix profile install for just the three tools Vitest needs (pnpm/bun/just). Follows the same pattern already used by the perf-comment and pkg-pr-new jobs.
- Replace
nix developwithnix profile install— installs onlypnpm_11,bun, andjustinstead of realizing the full dev shell (~73s → near-zero overhead). - Add explicit
pnpm install --frozen-lockfile— previously handled by the dev shell's implicit hook; now an explicit step after tooling install. - Drop
nix develop --commandwrapper fromjust test-vitest— runsjustdirectly since it's on$PATHvia the Nix profile.
Big Pickle (free via Pullfrog for OSS) ｜ 𝕏
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
ccusage-guide | 0a3391e | Commit Preview URL Branch Preview URL |
Jun 11 2026, 03:09 PM |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Code Coverage OverviewLanguages: Rust Rust / code-coverage/cargo-llvm-covThe overall coverage remains at 77%, unchanged from the Code Coverage is in Public Preview. Learn more and provide us with your feedback. |
ccusage performance comparisonPR SHA: This compares the Rust PR release binary against the configured base package on the same CI runner. Package runner startupExecution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one
Cached bunx execution performanceRuns the same large fixture through Fixtures: Claude
Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
ccusage performance comparisonPR SHA: This compares the PR package against the configured base package on the same CI runner. Package runner startupExecution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one
Cached bunx execution performanceRuns the same large fixture through Fixtures: Claude
Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
* perf(ci): run CI once per change instead of twice
`on: { push, pull_request }` with no branch filter meant every push to a PR
branch triggered the full workflow twice — once for `push` and once for
`pull_request` — duplicating the check, test, and native-build matrix on every
commit.
Restrict `push` to `main` so branch commits are validated once via the
`pull_request` event, while main keeps its post-merge run. Rust tests stay
covered: the `Run test suite` step only fired on pushes to non-default
branches (now gone), and the coverage step already runs `cargo llvm-cov` over
the workspace on both remaining triggers. Drop that dead step and the
now-always-true event guards on the Vitest and coverage steps.
* perf(ci): drop the full dev shell from the test job (#1269)
The test job spent ~73s on "Warm Nix dev shell" realizing the complete
development shell — every linter, formatter, renovate, and the pre-commit hook
closure — even though pnpm install itself was only ~7s. Now that coverage is a
`nix build` derivation, Vitest is the sole step that needs a JS toolchain.
Replace `nix develop` with `nix profile install pnpm_11 bun just`, mirroring the
perf-comment and pkg-pr-new jobs, then run `just test-vitest` directly. This
skips the heavy shell closure while keeping the exact tools Vitest needs.
1 participant
Summary
The
testjob spent ~73s on "Warm Nix dev shell" realizing the completedevelopment shell (every linter, formatter, renovate, and the pre-commit hook
closure) while
pnpm installitself was only ~7s.Now that coverage is a
nix buildderivation (#1266), Vitest is the only stepthat needs a JS toolchain. This replaces
nix developwithnix profile install pnpm_11 bun just— the same pattern already used by theperf-comment and pkg-pr-new jobs — and runs
just test-vitestdirectly.Why
The full dev shell pulls in tooling the test job never uses. Installing only
pnpm/bun/just skips that closure entirely.
Notes
mainonce that merges.nix build .#ccusage-coverage).~73s "Warm Nix dev shell" step.
Need help on this PR? Tag
/codesmithwith what you need. Autofix is disabled.Summary by cubic
Drop the full Nix dev shell from the test job to cut ~73s of shell warmup. Install only
pnpm_11,bun, andjust, runVitestdirectly, and keep coverage vianix build.nix developwithnix profile install nixpkgs#pnpm_11 nixpkgs#bun nixpkgs#just.pnpm install --frozen-lockfile; run tests withjust test-vitest.nix build .#ccusage-coverage).Written for commit 0a3391e. Summary will update on new commits.