Repository navigation
ci(pricing): land pricing updates on main instead of opening PRs - #1509
Conversation
The hourly pricing refresh opened a PR with github.token, which cannot trigger ci.yaml, so the PR arrived with no checks and nothing merged it. Pricing data was therefore only as fresh as the last time someone merged that PR by hand, even though the workflow already accepts a workflow_dispatch. Both jobs now run the checks that a pull request would have run - flake check, Rust tests and the Node tests, in a parallel step - and push the result straight to main. `just gen-models-dev-pricing` already runs before validation, so the committed snapshots never lag the locked revision. The commit and rebase-onto-main dance is shared by both jobs, so it lives in a composite action. A concurrency group keeps a manual dispatch from racing the hourly run's push.
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
ccusage-guide | fbdc937 | Commit Preview URL Branch Preview URL |
Jul 28 2026, 01:03 AM |
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAdds a reusable action that commits and rebases changes onto ChangesPricing automation
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant PricingWorkflow
participant UpdateScripts
participant ValidationChecks
participant CommitAndPushMain
participant MainBranch
PricingWorkflow->>UpdateScripts: update pins and detect artifact changes
UpdateScripts-->>PricingWorkflow: changed output
PricingWorkflow->>ValidationChecks: run checks when git changes exist
ValidationChecks-->>PricingWorkflow: validation result
PricingWorkflow->>CommitAndPushMain: provide commit message and paths
CommitAndPushMain->>MainBranch: fetch, rebase, and push validated changes
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
ccusage
@ccusage/ccusage-darwin-arm64
@ccusage/ccusage-darwin-x64
@ccusage/ccusage-linux-arm64
@ccusage/ccusage-linux-x64
@ccusage/ccusage-win32-x64
commit: |
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
Important
The codex-auto-review-fallbacks.json path in the models-dev job is wrong — the tracked file is at rust/adapters/codex/src/codex-auto-review-fallbacks.json, not rust/crates/ccusage-adapter-codex/src/codex-auto-review-fallbacks.json. This would cause the skip step's git checkout -- to fail when the snapshots are unchanged, and the push step's git commit to fail when they changed — the models-dev update job can't succeed with either outcome.
Reviewed changes — replaces the PR-creation workflow with inline validation (nix flake check, Rust tests, JS tests in parallel) and a direct push to main via a new composite action. Adds concurrency guarding, step outputs for change detection, and timeouts.
- New
commit-and-push-maincomposite action — commits given paths asgithub-actions[bot], rebases ontoorigin/main, and pushes. Fails on rebase conflicts so the next run retries. - Inline validation — both jobs run
nix flake check,nix build .#ccusage-tests, andjust test-nodein parallel before pushing, replacing the old single-stepjust check. Thegit diff --quiet && exit 0no-op in each leg skips validation when unchanged. - Step outputs for change detection — skip steps emit
changed=true/falseviaGITHUB_OUTPUT, gating the push action. - Concurrency —
update-pricinggroup withcancel-in-progress: falseprevents hourly/manual dispatch races. - Permissions and config — dropped
pull-requests: write, addedtimeout-minutes: 60andfetch-depth: 0.
@v0 or keep the SHA fresh with Dependabot ｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using DeepSeek Pro (free via Pullfrog for OSS) ｜ 𝕏
| paths: >- | ||
| flake.lock | ||
| rust/crates/ccusage-core/src/models-dev-pricing.json | ||
| rust/crates/ccusage-adapter-codex/src/codex-auto-review-fallbacks.json |
There was a problem hiding this comment.
Wrong path for codex-auto-review-fallbacks.json. The tracked file is at rust/adapters/codex/src/codex-auto-review-fallbacks.json (matching just gen-models-dev-pricing). The path used here (rust/crates/ccusage-adapter-codex/...) doesn't exist and would cause git commit to fail because the actual changed file at the correct path never gets committed.
Technical details
# Wrong file path for codex-auto-review-fallbacks.json
## Affected sites
- `.github/workflows/update-pricing.yaml:141` — `paths:` input to `commit-and-push-main` (new code)
- `.github/workflows/update-pricing.yaml:108` — `git checkout --` in the skip step's unchanged branch (pre-existing; would fail, killing the step)
- `.github/workflows/update-pricing.yaml:106` — `git diff --quiet --` check in the skip step (harmless — nonexistent pathspec returns 0, so the check silently ignores this file)
## Required outcome
- Replace `rust/crates/ccusage-adapter-codex/src/codex-auto-review-fallbacks.json` with `rust/adapters/codex/src/codex-auto-review-fallbacks.json` in all three locations above
## Suggested approach
Lines 106 and 108 are pre-existing, unchanged context lines; this PR's new code at line 141 is the trigger for fixing all three. The `git ls-files` output and `gen-models-dev-pricing` just recipe both confirm the correct path is `rust/adapters/codex/src/codex-auto-review-fallbacks.json`.
ccusage performance comparisonPR SHA: This compares the Rust PR release binary against the configured base package on the same CI runner. Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
ccusage performance comparisonPR SHA: This compares the PR package against the configured base package on the same CI runner. Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/update-pricing.yaml:
- Around line 82-93: Change the if condition for update-models-dev-pricing from
always() to !cancelled() so the dependent job still runs after success or
failure but does not start after manual workflow cancellation.
- Around line 56-75: Replace the repeated git diff --quiet guards in the
parallel checks at .github/workflows/update-pricing.yaml lines 56-75 and 117-133
with the existing changed-state outputs from steps.litellm.outputs.changed or
steps.models-dev.outputs.changed; if step-level if conditions are unavailable,
use git --no-optional-locks diff --quiet instead. Apply the same gating approach
to every affected check leg.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 627289ff-42f4-4325-9517-c7ab552c510a
📒 Files selected for processing (2)
.github/actions/commit-and-push-main/action.yaml.github/workflows/update-pricing.yaml
| update-models-dev-pricing: | ||
| needs: update-pricing | ||
| if: ${{ always() }} | ||
| runs-on: blacksmith-32vcpu-ubuntu-2404 | ||
| timeout-minutes: 60 | ||
| permissions: | ||
| contents: write | ||
| pull-requests: write | ||
| steps: | ||
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| fetch-depth: 0 | ||
| persist-credentials: false |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== locate workflow =="
fd -a 'update-pricing\.yaml' .github/workflows || true
echo "== relevant workflow excerpt =="
if [ -f .github/workflows/update-pricing.yaml ]; then
nl -ba .github/workflows/update-pricing.yaml | sed -n '1,140p'
fi
echo "== changed context if available =="
git diff -- .github/workflows/update-pricing.yaml | sed -n '1,220p' || true
echo "== search relevant expressions in workflows =="
rg -n "update-models-dev-pricing|needs: update-pricing|if:\s*\$\{\{\s*always|if:\s*\$\{\{\s*!cancelled|push|checkout@" .github/workflows || trueRepository: ccusage/ccusage
Length of output: 304
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== relevant workflow excerpt =="
sed -n '1,140p' .github/workflows/update-pricing.yaml
echo "== changed context if available =="
git diff -- .github/workflows/update-pricing.yaml | sed -n '1,220p' || true
echo "== search relevant expressions in workflows =="
rg -n "update-models-dev-pricing|needs: update-pricing|if:\s*\$\{\{\s*always|if:\s*\$\{\{\s*!cancelled|push|checkout@" .github/workflows || trueRepository: ccusage/ccusage
Length of output: 9593
🌐 Web query:
GitHub Actions always() function cancelled workflow run needs context documentation
💡 Result:
The GitHub Actions always() function causes a job or step to execute even when the workflow run is canceled [1][2]. Because always() evaluates to true even during a cancellation event, it can effectively prevent a job from being canceled when included in a job-level or step-level if condition [3][4]. Official GitHub documentation explicitly warns against using always() for tasks that could suffer from critical failures (such as fetching sources), as this may cause the workflow to hang until it times out [1][2]. For scenarios where you want a job or step to run regardless of success or failure, but you still want it to be skipped if the workflow is canceled, the recommended alternative is to use the !cancelled() expression [1][2]: if: ${{!cancelled }} This approach ensures the task executes after successful or failed steps, but respects the cancellation state of the workflow [5].
Citations:
- 1: https://github.com/github/docs/blob/main/content/actions/reference/workflows-and-actions/expressions.md
- 2: https://docs.github.com/en/enterprise-cloud@latest/actions/reference/workflows-and-actions/expressions
- 3: https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-cancellation
- 4: Provide a way to force-cancel actions and improve cancellation feedback actions/runner#1846
- 5: https://stackoverflow.com/questions/58858429/how-to-run-a-github-actions-step-even-if-the-previous-step-fails-while-still-f
Use !cancelled() for the dependent pricing job.
always() forces update-models-dev-pricing to start even when the workflow run is manually cancelled, then it can still push changes to main before the cancellation takes effect. Use !cancelled() to retain “run after success/failure” while honoring cancellations.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/update-pricing.yaml around lines 82 - 93, Change the if
condition for update-models-dev-pricing from always() to !cancelled() so the
dependent job still runs after success or failure but does not start after
manual workflow cancellation.
The two update steps were shell pipelines around jq, curl and cmp writing
JSON to /tmp. Nushell handles the same work as structured data: the lock
file is read with `open`, the pricing document is fetched with `http get`
and compared as a record, so no temporary files or byte-level formatting
assumptions are involved.
The scripts carry the repository's Nix shebang, so they need no extra
nushell setup step in the workflow and stay runnable by hand.
models.dev now decides through a `match` over {snapshots, lock}, which
names the three states the step can end in - snapshots moved, only the lock
moved, nothing moved - instead of inferring them from consecutive `git
diff` calls.
The validation legs only covered flake check, the Rust tests and the Node tests, so the Babashka harness test from ci.yaml's test job was missing. Now that the push lands on main without any pull request checks behind it, the workflow has to run what a PR would have run: `nix flake check` already covers the preflight job's treefmt and gitleaks checks, and the remaining legs match the test job leg for leg. The two sources were separate jobs with a duplicated leg list, which is how the Babashka leg went missing in the first place. They are a matrix now, so the checks are written once and both sources cannot drift apart. `max-parallel: 1` preserves the sequential push the previous `needs:` ordering gave.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
2 issues found and verified against the latest diff
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name=".github/actions/commit-and-push-main/action.yaml">
<violation number="1" location=".github/actions/commit-and-push-main/action.yaml:38">
P1: A manual run started from a non-main branch can publish that branch's existing commits to `main`, not just the generated pricing snapshot. Because checkout is not pinned to `main` and the action rebases/pushes the whole `HEAD`, the direct push can replay every branch commit that is not already reachable from `main`. Restricting this workflow to `refs/heads/main` or explicitly checking out `main` before generating the update would prevent that history leak.</violation>
</file>
<file name=".github/workflows/update-pricing.yaml">
<violation number="1" location=".github/workflows/update-pricing.yaml:20">
P2: Since this job now pushes directly to `main` via `commit-and-push-main` instead of opening a PR, `if: ${{ always() }}` lets the job start and push even after a manual cancellation of the workflow run, since `always()` ignores cancellation status. Consider using `!cancelled()` instead to preserve run-after-success/failure semantics while still respecting cancellation.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| echo 'main moved with a conflicting change; the next run will retry.' | ||
| exit 1 | ||
| fi | ||
| git push origin HEAD:main |
There was a problem hiding this comment.
P1: A manual run started from a non-main branch can publish that branch's existing commits to main, not just the generated pricing snapshot. Because checkout is not pinned to main and the action rebases/pushes the whole HEAD, the direct push can replay every branch commit that is not already reachable from main. Restricting this workflow to refs/heads/main or explicitly checking out main before generating the update would prevent that history leak.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At .github/actions/commit-and-push-main/action.yaml, line 38:
<comment>A manual run started from a non-main branch can publish that branch's existing commits to `main`, not just the generated pricing snapshot. Because checkout is not pinned to `main` and the action rebases/pushes the whole `HEAD`, the direct push can replay every branch commit that is not already reachable from `main`. Restricting this workflow to `refs/heads/main` or explicitly checking out `main` before generating the update would prevent that history leak.</comment>
<file context>
@@ -0,0 +1,38 @@
+ echo 'main moved with a conflicting change; the next run will retry.'
+ exit 1
+ fi
+ git push origin HEAD:main
</file context>
| update-pricing: | ||
| name: ${{ matrix.name }} | ||
| runs-on: blacksmith-32vcpu-ubuntu-2404 | ||
| timeout-minutes: 60 |
There was a problem hiding this comment.
P2: Since this job now pushes directly to main via commit-and-push-main instead of opening a PR, if: ${{ always() }} lets the job start and push even after a manual cancellation of the workflow run, since always() ignores cancellation status. Consider using !cancelled() instead to preserve run-after-success/failure semantics while still respecting cancellation.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At .github/workflows/update-pricing.yaml, line 19:
<comment>Since this job now pushes directly to `main` via `commit-and-push-main` instead of opening a PR, `if: ${{ always() }}` lets the job start and push even after a manual cancellation of the workflow run, since `always()` ignores cancellation status. Consider using `!cancelled()` instead to preserve run-after-success/failure semantics while still respecting cancellation.</comment>
<file context>
@@ -7,15 +7,22 @@ on:
jobs:
update-pricing:
runs-on: blacksmith-32vcpu-ubuntu-2404
+ timeout-minutes: 60
permissions:
contents: write
</file context>
There was a problem hiding this comment.
The matrix refactor in fbdc937 merged the two jobs into one and removed the if: ${{ always() }} this targeted, so the condition no longer exists on main.
There was a problem hiding this comment.
Important
The path bug for codex-auto-review-fallbacks.json flagged in the prior review was not addressed — it was carried forward into the new update-models-dev-lock.nu script's SNAPSHOTS constant. The models-dev update job will fail on every run with either outcome.
Reviewed changes — moved the lock-update logic for both LiteLLM and models.dev from inline shell steps into two Nushell scripts, each using a Nix shebang so the workflow can call them directly without a separate Nushell setup step.
- New
update-litellm-lock.nu— fetches the pricing JSON before and afternix flake update litellm, compares the records in Nushell (replacing thecurl/jq/cmpdance), and emitschanged=true/falseviaGITHUB_OUTPUT. - New
update-models-dev-lock.nu— runsnix flake update models-devandjust gen-models-dev-pricing, checks snapshot and lock dirtiness with adirtyhelper, and emitschanged=true/falsewith a three-waymatchover{snapshots, lock}. - Workflow call-site simplification — both jobs replace inline shell capture/update/skip/create-PR steps with single
run:calls to the Nushell scripts plus the existingparallel:validation legs andcommit-and-push-main.
ℹ️ Path bug carried into new update-models-dev-lock.nu SNAPSHOTS
The prior review identified that codex-auto-review-fallbacks.json lives at rust/adapters/codex/src/codex-auto-review-fallbacks.json, not rust/crates/ccusage-adapter-codex/src/codex-auto-review-fallbacks.json. This commit relocated the lock-update logic into a Nushell script without fixing the path — the SNAPSHOTS constant on line 6 uses the same wrong path, and the workflow's paths: input on line 97 was left unchanged. The models-dev pricing update job cannot succeed: git diff --quiet -- silently ignores the nonexistent path, git checkout -- fails when the snapshots haven't changed, and commit-and-push-main fails to commit the actual changed file at the correct path.
Technical details
# Path bug carried into new Nushell script
## Affected sites
- `.github/scripts/update-models-dev-lock.nu:6` — `SNAPSHOTS` constant (new in this commit)
- `.github/workflows/update-pricing.yaml:97` — `paths:` input to `commit-and-push-main` (pre-existing)
## Required outcome
- Replace `rust/crates/ccusage-adapter-codex/src/codex-auto-review-fallbacks.json` with `rust/adapters/codex/src/codex-auto-review-fallbacks.json` in both locations
- Verify `just gen-models-dev-pricing` still writes to the corrected path
## Suggested approach
`git ls-files` confirms the tracked file is at `rust/adapters/codex/src/codex-auto-review-fallbacks.json`. Fix line 6 in the script and line 97 in the workflow.@v0 or keep the SHA fresh with Dependabot ｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using DeepSeek Pro (free via Pullfrog for OSS) ｜ 𝕏
|
|
||
| const SNAPSHOTS = [ | ||
| 'rust/crates/ccusage-core/src/models-dev-pricing.json' | ||
| 'rust/crates/ccusage-adapter-codex/src/codex-auto-review-fallbacks.json' |
There was a problem hiding this comment.
rust/adapters/codex/src/codex-auto-review-fallbacks.json (confirmed by git ls-files and just gen-models-dev-pricing). This path doesn't exist, so git diff --quiet -- silently ignores the file and git checkout -- fails when the snapshots haven't changed.
Technical details
# Wrong path in SNAPSHOTS constant
## Affected sites
- `.github/scripts/update-models-dev-lock.nu:6` — this line
- `.github/workflows/update-pricing.yaml:97` — `paths:` input to `commit-and-push-main`
## Required outcome
- Replace `rust/crates/ccusage-adapter-codex/src/codex-auto-review-fallbacks.json` with `rust/adapters/codex/src/codex-auto-review-fallbacks.json` in both locations
## Suggested approach
`git ls-files` confirms the correct path; the two fix sites are a single string replacement each.| paths: >- | ||
| flake.lock | ||
| rust/crates/ccusage-core/src/models-dev-pricing.json | ||
| rust/crates/ccusage-adapter-codex/src/codex-auto-review-fallbacks.json |
There was a problem hiding this comment.
rust/adapters/codex/src/codex-auto-review-fallbacks.json. When the snapshots changed, git commit will succeed for flake.lock and models-dev-pricing.json but the actually-changed file at the correct path will be left out of the commit.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/scripts/update-models-dev-lock.nu:
- Around line 15-27: Update the snapshot change detection around state and
SNAPSHOTS so it captures the pre-regeneration snapshots and compares their open
--raw ... | from json results rather than raw git bytes. Keep git diff-based
detection only for flake.lock, and preserve the existing lock-only rollback and
validation behavior when parsed snapshot content is unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 55762c46-7bf9-4639-bf12-aa6a038d5e81
📒 Files selected for processing (3)
.github/scripts/update-litellm-lock.nu.github/scripts/update-models-dev-lock.nu.github/workflows/update-pricing.yaml
🚧 Files skipped from review as they are similar to previous changes (1)
- .github/workflows/update-pricing.yaml
| let state = { | ||
| snapshots: (dirty ...$SNAPSHOTS) | ||
| lock: (dirty flake.lock) | ||
| } | ||
| let changed = match $state { | ||
| {snapshots: true} => true | ||
| {snapshots: false, lock: true} => { | ||
| print 'models.dev pricing snapshots are unchanged; dropping the lock-only bump.' | ||
| ^git checkout -- flake.lock ...$SNAPSHOTS | ||
| false | ||
| } | ||
| _ => false | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Compare generated snapshots as JSON, not raw Git bytes.
dirty ...$SNAPSHOTS treats whitespace or formatting-only rewrites as pricing changes, causing unnecessary validation and commits to main. Capture the snapshots before regeneration and compare open --raw ... | from json results; keep git diff only for flake.lock.
Proposed fix
def main [] {
+ let before = ($SNAPSHOTS | each {|path| open --raw $path | from json })
^nix flake update models-dev
^nix develop --command just gen-models-dev-pricing
let state = {
- snapshots: (dirty ...$SNAPSHOTS)
+ snapshots: (($SNAPSHOTS
+ | each {|path| open --raw $path | from json }) != $before)
lock: (dirty flake.lock)
}Also applies to: 32-35
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/scripts/update-models-dev-lock.nu around lines 15 - 27, Update the
snapshot change detection around state and SNAPSHOTS so it captures the
pre-regeneration snapshots and compares their open --raw ... | from json results
rather than raw git bytes. Keep git diff-based detection only for flake.lock,
and preserve the existing lock-only rollback and validation behavior when parsed
snapshot content is unchanged.
ccusage performance comparisonPR SHA: This compares the Rust PR release binary against the configured base package on the same CI runner. Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
ccusage performance comparisonPR SHA: This compares the PR package against the configured base package on the same CI runner. Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
ccusage performance comparisonPR SHA: This compares the Rust PR release binary against the configured base package on the same CI runner. Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
ccusage performance comparisonPR SHA: This compares the PR package against the configured base package on the same CI runner. Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
main has no required status checks, so GitHub auto-merge has nothing to wait for: `gh pr merge --auto` merged #1509 while its CI was still running, and renovate's platformAutomerge would merge dependency PRs the same way. Requiring the existing checks directly does not work, because everything behind `changes` is skipped on docs-only pull requests and a skipped check never satisfies a requirement. This adds the usual aggregator instead: it needs every job action-timeline already waits for, treats skipped as fine, and fails only when a job failed or was cancelled. That single "ci gate" check is what main's ruleset can require.
main has no required status checks, so GitHub's auto-merge merges the moment it is enabled - that is how #1509 landed mid-CI. platformAutomerge hands renovate's automerge to that same mechanism, which would merge dependency PRs before their checks report. Requiring the new `ci gate` check on main would fix it for every merge path at once, but a repository ruleset cannot list the built-in GitHub Actions integration as a bypass actor: Actor GitHub Actions integration must be part of the ruleset source or owner organization and required status checks reject direct pushes too, so the rule would break update-pricing.yaml's push to main. Bypassing by write role does not help either, because renovate holds write and would bypass the gate it is meant to obey. With platformAutomerge off, renovate merges the branch itself once it has seen the branch checks pass, which needs no repository rules at all.
main has no required status checks, so GitHub's auto-merge merges the moment it is enabled - that is how #1509 landed mid-CI. platformAutomerge hands renovate's automerge to that same mechanism, which would merge dependency PRs before their checks report. Requiring the new `ci gate` check on main would fix it for every merge path at once, but a repository ruleset cannot list the built-in GitHub Actions integration as a bypass actor: Actor GitHub Actions integration must be part of the ruleset source or owner organization and required status checks reject direct pushes too, so the rule would break update-pricing.yaml's push to main. Bypassing by write role does not help either, because renovate holds write and would bypass the gate it is meant to obey. With platformAutomerge off, renovate merges the branch itself once it has seen the branch checks pass, which needs no repository rules at all.
Summary
update-pricing.yamlalready runs hourly and already acceptsworkflow_dispatch, but it opened a pull request withgithub.token. Such a PR cannot triggerci.yaml, so it arrived with no checks and nothing merged it — pricing data was only as fresh as the last time someone merged that PR by hand (#1355 for the flake lock sat open for a month for the same reason). Both jobs now validate in place and push straight tomain, so a dispatch lands the update immediately.What Changed
.github/workflows/update-pricing.yamlnix flake check,nix build .#ccusage-tests,just test-node— in aparallel:step, then pushes tomain.git diff.concurrencygroup: a manual dispatch would otherwise race the hourly run's push.pull-requests: write; the jobs only needcontents: writenow..github/actions/commit-and-push-main/action.yaml: new composite action shared by both jobs. It commits the given paths asgithub-actions[bot], rebases onto whatevermainhas become during validation, and fails loudly (so the next run retries) if that rebase conflicts..github/scripts/update-litellm-lock.nuand.github/scripts/update-models-dev-lock.nu: the two update steps moved out of shell into Nushell. The lock file is read withopen, the LiteLLM pricing document is fetched withhttp getand compared as a record, so the jq/curl/cmp dance and its/tmpJSON files are gone. models.dev decides through amatchover{snapshots, lock}, which names the three states the step can end in — snapshots moved, only the lock moved, nothing moved. Both carry the repository's Nix shebang, so no nushell setup step is needed and they stay runnable by hand.Why the checks run inline
A push made with
github.tokencannot triggerci.yamleither, somainwould otherwise receive the pricing bump with no validation at all. Running the PR's checks inside the workflow keeps the same gate without needing a PR that nobody merges.just gen-models-dev-pricingstill runs before validation, so the committedmodels-dev-pricing.jsonandcodex-auto-review-fallbacks.jsonsnapshots never lag the lockedmodels-devrevision.Testing
34561482tof2cda740, saw the pricing JSON change and reportedchanged=true; models.dev regeneratedmodels-dev-pricing.jsonand reportedchanged=true. The lock and snapshot were restored afterwards.nu-checkpasses on both scripts.just fmtpasses, which runs actionlint over the workflow, zizmor over both the workflow and the new composite action, and nufmt over the scripts.parallel:step syntax already ignored repo-wide forci.yaml.Note
The
parallel:legs each start withgit diff --quiet && exit 0so they no-op when the snapshot was already current.if:is used for the ordinary steps, but its behaviour on aparallel:step is not something this repo exercises yet, so the guard stays inside the legs.Summary by CodeRabbit