Skip to content

fix(ci): repair the failing update-pricing workflow - #1262

Merged
ryoppippi merged 2 commits into
mainfrom
fix/update-pricing-workflow
Jun 11, 2026
Merged

ryoppippi merged 2 commits into
mainfrom
fix/update-pricing-workflow

Conversation

@ryoppippi

@ryoppippi ryoppippi commented Jun 11, 2026 •

Copy link
Copy Markdown
Member

Summary

Both jobs of the scheduled update pricing workflow have been failing on every run. This PR fixes the two independent root causes.

What changed

  • nix/models-dev-pricing.nix — also copy models-dev-compact.ts into the writable Bun workspace. perf(pricing): shrink embedded snapshots 58% and binary 2.9% #1244 split the generator so models-dev-gen.ts imports its sibling ./models-dev-compact.ts, but the derivation only copied gen.ts, so the sandboxed Bun run failed with Cannot find module './models-dev-compact.ts'. This broke the update-models-dev-pricing job and any manual just gen-models-dev-pricing.
  • .github/workflows/update-pricing.yaml — push the automation branches with git push --force instead of --force-with-lease. The job checks out main at fetch-depth: 1 and never fetches the bot branch, so there is no remote-tracking ref and the lease check rejects the push with stale info. These branches are written only by this workflow, so a plain force-push is safe.

Why

The hourly cron has been red since the #1244 refactor (models.dev job) and on every run that already had an open automation branch (force-with-lease push). Restoring both keeps the LiteLLM and models.dev pricing snapshots updating automatically.

Testing

  • nix build .#models-dev-pricing now succeeds locally, and the regenerated snapshot is byte-identical to the committed rust/crates/ccusage/src/models-dev-pricing.json.

View with Codesmith Autofix with Codesmith
Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.


Summary by cubic

Fixes the scheduled update-pricing workflow by addressing failures in the Nix build and CI push step. Restores automatic LiteLLM and models.dev pricing snapshot updates.

  • Bug Fixes
    • In nix/models-dev-pricing.nix, copy models-dev-compact.ts alongside gen.ts so Bun can resolve the sibling import.
    • In .github/workflows/update-pricing.yaml, push automation branches with git push --force instead of --force-with-lease to avoid lease errors on shallow checkouts.

Written for commit 94a62cb. Summary will update on new commits.

Review in cubic

Summary by CodeRabbit

  • Chores
    • Refined internal automation workflows for pricing update operations.
    • Improved build system configuration to ensure reliable dependency resolution during deployment processes.

The models.dev pricing generator was split in #1244 so that
nix/models-dev-gen.ts imports its sibling ./models-dev-compact.ts.
The Nix derivation only copied gen.ts into the writable workspace,
so Bun could not resolve the new import inside the sandbox:

    error: Cannot find module './models-dev-compact.ts' from '/build/work/gen.ts'

This broke the scheduled "update pricing" workflow on the
update-models-dev-pricing job and every manual gen-models-dev-pricing run.

Copy models-dev-compact.ts alongside gen.ts under the same relative
name the import expects. Verified locally: the regenerated snapshot is
byte-identical to the committed rust/crates/ccusage/src/models-dev-pricing.json.
The "Create pull request" steps pushed with --force-with-lease, but the
job checks out main at fetch-depth 1 and never fetches the bot branch.
Without a remote-tracking ref, git refuses the lease:

    ! [rejected]  automation/litellm-pricing -> automation/litellm-pricing (stale info)
    error: failed to push some refs

These branches are written only by this workflow, so a plain --force is
safe and is the standard pattern for a bot-owned automation branch.
@coderabbitai

coderabbitai Bot commented Jun 11, 2026 •

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Two small, independent fixes address pricing automation safety and build setup: GitHub Actions workflow push commands switch from --force-with-lease to --force for branch pushes in both LiteLLM and models.dev pricing jobs, and the Nix build script adds a missing file copy to resolve the generator's import path.

Changes

Git Push Flag Updates in Pricing Workflows

Layer / File(s) Summary
Workflow push command flags
.github/workflows/update-pricing.yaml
Both LiteLLM pricing (line 65) and models.dev pricing (line 129) PR creation steps now use git push --force instead of git push --force-with-lease.

Models.dev Pricing Generator Build Fix

Layer / File(s) Summary
Nix build workspace file copy
nix/models-dev-pricing.nix
Build workspace setup now copies models-dev-compact.ts into work/ alongside gen.ts, enabling the generator's relative import to resolve during bun run gen.ts execution.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Poem

A rabbit hops through automation,
Flags and files in sweet formation,
--force replaces --force-with-lease,
And compact models find their peace,
Build scripts dance in harmony! 🐰

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main purpose of the PR—repairing the failing update-pricing workflow by fixing both root causes (Nix derivation and Git push options).
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/update-pricing-workflow

Comment @coderabbitai help to get the list of available commands and usage tips.

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
ccusage-guide 94a62cb Commit Preview URL

Branch Preview URL
Jun 11 2026, 09:31 AM

@ryoppippi
ryoppippi merged commit b722690 into main Jun 11, 2026
24 of 25 checks passed
@ryoppippi
ryoppippi deleted the fix/update-pricing-workflow branch June 11, 2026 09:31
@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: 94a62cbdc997
Base SHA: b7311555b47b

This compares the Rust PR release binary against the configured base package on the same CI runner.

Package runner startup

Execution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one bunx -p <url> ccusage --version run with an empty Bun install cache. Warm reuses that cache and reports the median of repeated runs.

Package SHA Execution setup Bunx temp cache Bunx warm median Warm samples
Base pkg.pr.new b7311555b47b 1.128s 747.5ms 52.4ms 3
PR pkg.pr.new 94a62cb 780.0ms 1.089s 51.6ms 3

Cached bunx execution performance

Runs the same large fixture through bunx -p <pkg.pr.new URL> ccusage after the Bun install cache has already been populated by the startup measurement. This separates cached package-runner execution from first-fetch package materialization.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base package: b7311555b47b; PR package: 94a62cb. Both run through bunx -p <pkg.pr.new URL> ccusage using the warmed Bun install cache from package runner startup, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
bunx -p <pkg> ccusage claude --offline --json 1.01 GiB 918.8ms 995.0ms 0.92x 730.25 MiB 726.00 MiB 0.99x 1.10 GiB/s 1.01 GiB/s
bunx -p <pkg> ccusage codex --offline --json 1.01 GiB 191.1ms 187.8ms 1.02x 89.50 MiB 91.00 MiB 1.02x 5.27 GiB/s 5.36 GiB/s

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 996.4ms 1.01 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 848.2ms 1.19 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 177.9ms 5.66 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 130.2ms 7.74 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs rust/target/release/ccusage directly. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 47.1ms 6.0ms 7.80x 43.25 MiB 2.75 MiB 0.06x 0.03 MiB/s 0.26 MiB/s
claude session --offline --json 0.00 MiB 48.1ms 6.6ms 7.24x 43.25 MiB 2.75 MiB 0.06x 0.03 MiB/s 0.23 MiB/s
codex daily --offline --json 0.00 MiB 48.4ms 6.2ms 7.83x 43.00 MiB 2.75 MiB 0.06x 0.02 MiB/s 0.14 MiB/s
codex session --offline --json 0.00 MiB 47.0ms 5.7ms 8.20x 43.25 MiB 2.75 MiB 0.06x 0.02 MiB/s 0.15 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs rust/target/release/ccusage directly. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 871.8ms 860.5ms 1.01x 731.50 MiB 729.75 MiB 1.00x 1.15 GiB/s 1.17 GiB/s
codex --offline --json 1.01 GiB 174.2ms 141.9ms 1.23x 90.25 MiB 91.75 MiB 1.02x 5.78 GiB/s 7.09 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 17.32 KiB 17.32 KiB +0.00 KiB 1.00x
installed native package binary 3324.77 KiB 3324.77 KiB +0.00 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: 94a62cbdc997
Base SHA: b7311555b47b

This compares the PR package against the configured base package on the same CI runner.

Package runner startup

Execution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one bunx -p <url> ccusage --version run with an empty Bun install cache. Warm reuses that cache and reports the median of repeated runs.

Package SHA Execution setup Bunx temp cache Bunx warm median Warm samples
Base pkg.pr.new b7311555b47b 966.8ms 812.1ms 61.9ms 3
PR pkg.pr.new 94a62cb 1.005s 859.2ms 67.6ms 3

Cached bunx execution performance

Runs the same large fixture through bunx -p <pkg.pr.new URL> ccusage after the Bun install cache has already been populated by the startup measurement. This separates cached package-runner execution from first-fetch package materialization.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base package: b7311555b47b; PR package: 94a62cb. Both run through bunx -p <pkg.pr.new URL> ccusage using the warmed Bun install cache from package runner startup, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
bunx -p <pkg> ccusage claude --offline --json 1.01 GiB 928.9ms 872.7ms 1.06x 740.25 MiB 729.25 MiB 0.99x 1.08 GiB/s 1.15 GiB/s
bunx -p <pkg> ccusage codex --offline --json 1.01 GiB 256.3ms 242.3ms 1.06x 92.75 MiB 92.00 MiB 0.99x 3.93 GiB/s 4.15 GiB/s

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 1.040s 991.02 MiB/s 1
claude --offline --json Installed native binary 1.01 GiB 884.8ms 1.14 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 230.0ms 4.38 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 202.6ms 4.97 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 69.6ms 48.0ms 1.45x 43.00 MiB 43.00 MiB 1.00x 0.02 MiB/s 0.03 MiB/s
claude session --offline --json 0.00 MiB 55.6ms 51.3ms 1.08x 43.25 MiB 43.25 MiB 1.00x 0.03 MiB/s 0.03 MiB/s
codex daily --offline --json 0.00 MiB 57.0ms 52.5ms 1.08x 43.25 MiB 43.00 MiB 0.99x 0.02 MiB/s 0.02 MiB/s
codex session --offline --json 0.00 MiB 51.0ms 50.4ms 1.01x 42.75 MiB 43.00 MiB 1.01x 0.02 MiB/s 0.02 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 817.7ms 803.8ms 1.02x 727.50 MiB 728.25 MiB 1.00x 1.23 GiB/s 1.25 GiB/s
codex --offline --json 1.01 GiB 246.5ms 225.1ms 1.10x 91.75 MiB 91.75 MiB 1.00x 4.08 GiB/s 4.47 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 17.32 KiB 17.32 KiB +0.00 KiB 1.00x
installed native package binary 3324.77 KiB 3324.77 KiB +0.00 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant