Repository navigation
fix(usage): correct replay and pricing accounting - #1438
Conversation
Fast pricing previously doubled usage whenever a model lacked a known multiplier, which turned missing rate data into an unsupported estimate. Apply only explicit multipliers, keep unknown rates at standard pricing, refresh the report snapshot, and remove the obsolete timing-inference scratch plan.
A global package-file rule can classify Rust golden snapshots as binary, hiding reviewable output changes. Override that classification only under snapshot directories while leaving actual package archives binary.
The unsuffixed family alias matched the longest sibling key and selected the balanced tier rather than the documented flagship tier. Preserve exact user overrides, resolve canonical aliases before fuzzy fallback, and apply the same mapping to context limits and request-tier thresholds. Partial overrides now inherit unspecified canonical rates.
Current subagent rollouts can persist copied parent snapshots before the child's first turn. Treating those cumulative records as fresh usage compounds totals across nested sessions. Use the durable turn boundary for current logs, retain the timestamp fallback for older formats, and suppress unchanged cumulative snapshots while preserving child deltas.
|
This PR was auto-closed. Only contributors approved with Maintainers review auto-closed issues and reopen worthwhile ones. Issues that do not meet the quality bar in CONTRIBUTING.md may not be reopened or receive a reply. If a maintainer replies See CONTRIBUTING.md. |
|
Caution Review failedThe pull request is closed. Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughCodex parsing now records service tiers and replay metadata. Tier-aware deduplication and aggregation preserve standard, fast, and long-context usage for cost reporting. Pricing aliases, multipliers, documentation, CLI help, and snapshot attributes were updated. ChangesCodex and pricing updates
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant SessionLog
participant CodexParser
participant CodexAggregator
participant CostReporter
SessionLog->>CodexParser: Read thread settings and token events
CodexParser->>CodexAggregator: Emit tier-tagged usage events
CodexAggregator->>CostReporter: Provide deduplicated standard/fast buckets
CostReporter->>CostReporter: Apply speed policy and pricing
Possibly related issues
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Recent rollouts persist explicit Fast and Standard settings. Preserve those transitions during parsing so mixed-mode reports apply published rates only to matching requests. Reconcile copied records before attributing tier buckets so file order and worker scheduling cannot change cost estimates. Command-line overrides remain authoritative, with config used only for unclassified usage.
There was a problem hiding this comment.
Pull request overview
Fixes Codex usage/cost accounting regressions by (1) preventing replayed parent-history from being double-counted in MultiAgent V2 subagent sessions, and (2) tightening pricing resolution so speed-tier multipliers and model-family aliases are applied only when explicitly defined—while preserving exact and partial overrides.
Changes:
- Correct Codex replay/subagent accounting by establishing a child-session baseline, skipping non-advancing cumulative snapshots, and counting only the child’s advancing usage (including around the durable turn boundary).
- Introduce recorded per-event Codex service tier tracking (Standard/Fast) and apply mixed-tier pricing in
--speed auto, with explicit fast multipliers required before adjusting costs. - Improve pricing lookups by resolving canonical family aliases (notably
gpt-5.6 → gpt-5.6-sol) before fuzzy matching, and ensuring overrides merge onto canonical base entries.
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| rust/crates/ccusage/src/types.rs | Adds Codex service-tier and bucket types plus deterministic tier-merge logic for deduped events. |
| rust/crates/ccusage/src/pricing.rs | Adjusts alias/exact/fuzzy resolution order; canonicalizes alias use for context limits and long-context thresholds; adds tests for alias + override precedence. |
| rust/crates/ccusage/src/main.rs | Updates test fixtures to include the new service_tier field. |
| rust/crates/ccusage/src/fast-multiplier-overrides.json | Adds explicit fast multipliers for GPT‑5.6 Sol/Terra/Luna. |
| rust/crates/ccusage/src/adapter/codex/types.rs | Extends decoded payload schema to capture thread settings / service tier markers and related metadata. |
| rust/crates/ccusage/src/adapter/codex/speed.rs | Introduces CodexSpeedPolicy (auto vs forced) and maps CLI speed selections into tier policy. |
| rust/crates/ccusage/src/adapter/codex/snapshots/ccusage__adapter__codex__tests__snapshots_codex_reports_for_periods_sessions_costs_and_fallback_models.snap | Updates golden outputs to reflect corrected cost calculations. |
| rust/crates/ccusage/src/adapter/codex/report.rs | Implements mixed-tier pricing by splitting recorded standard/fast usage and applying multipliers only where defined. |
| rust/crates/ccusage/src/adapter/codex/README.md | Documents the new thread_settings_applied service-tier markers and how they influence auto pricing. |
| rust/crates/ccusage/src/adapter/codex/parser.rs | Reworks replay handling (subagent marker vs timestamp fallback), skips non-advancing totals, and records service tier transitions. |
| rust/crates/ccusage/src/adapter/codex/mod.rs | Uses resolved speed policy for production runs; keeps test helper wiring updated. |
| rust/crates/ccusage/src/adapter/codex/loader.rs | Dedupes events while preserving/merging service-tier metadata; adds regression tests around tier transitions and replay behavior. |
| rust/crates/ccusage/src/adapter/codex/aggregate.rs | Stores service-tier metadata alongside dedupe keys and reapplies recorded-tier usage into aggregated model buckets deterministically. |
| rust/crates/ccusage/src/adapter/all/loader.rs | Ensures unified “all agents” rows compute Codex costs consistently with mixed recorded tiers. |
| rust/crates/ccusage-cli/src/snapshots/ccusage_cli__tests__codex_daily_help.snap | Updates CLI help snapshot to reflect new --speed auto behavior description. |
| rust/crates/ccusage-cli/src/cli-help.json | Updates generated CLI help JSON for --speed description. |
| docs/guide/codex/index.md | Updates user docs to describe replay baselining, recorded tier tracking, and conservative unknown-fast pricing behavior. |
| codex-speed-infer-plan.md | Removes obsolete inference-plan notes now superseded by recorded-tier support. |
| .gitattributes | Treats Insta .snap files as text with LF endings for consistent diffs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (1)
rust/crates/ccusage/src/adapter/codex/aggregate.rs (1)
414-475: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚖️ Poor tradeoffPeriod/date-filter logic is duplicated between
apply_recorded_usage_entriesandadd_deduped_event_to_groups.The
since/untilcompaction check and thematch kind { Daily/Weekly/Monthly/Session }period selection here are a copy of the logic inadd_deduped_event_to_groups(Lines 309-323). Since recorded buckets are attributed independently of the totals accumulation, any future divergence between these two blocks would silently misattributerecorded_standard_usage/recorded_fast_usageto a period the totals don't share, without a compile error. Consider extracting a sharedcodex_period_for(timestamp, session_id, kind, shared, timezone) -> Option<String>helper used by both paths.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@rust/crates/ccusage/src/adapter/codex/aggregate.rs` around lines 414 - 475, Extract the duplicated date filtering and period selection from add_deduped_event_to_groups and apply_recorded_usage_entries into a shared codex_period_for helper returning Option<String>. Pass the event timestamp, optional session ID, report kind, shared date bounds, and timezone; preserve the existing Daily, Weekly, Monthly, and Session behavior, including returning None for out-of-range dates or missing session IDs, and use the helper in both paths.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@rust/crates/ccusage/src/adapter/codex/aggregate.rs`:
- Around line 414-475: Extract the duplicated date filtering and period
selection from add_deduped_event_to_groups and apply_recorded_usage_entries into
a shared codex_period_for helper returning Option<String>. Pass the event
timestamp, optional session ID, report kind, shared date bounds, and timezone;
preserve the existing Daily, Weekly, Monthly, and Session behavior, including
returning None for out-of-range dates or missing session IDs, and use the helper
in both paths.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 019f9d96-f3d6-42f6-be32-17aaf13039d2
⛔ Files ignored due to path filters (2)
rust/crates/ccusage-cli/src/snapshots/ccusage_cli__tests__codex_daily_help.snapis excluded by!**/*.snaprust/crates/ccusage/src/adapter/codex/snapshots/ccusage__adapter__codex__tests__snapshots_codex_reports_for_periods_sessions_costs_and_fallback_models.snapis excluded by!**/*.snap
📒 Files selected for processing (17)
.gitattributescodex-speed-infer-plan.mddocs/guide/codex/index.mdrust/crates/ccusage-cli/src/cli-help.jsonrust/crates/ccusage/src/adapter/all/loader.rsrust/crates/ccusage/src/adapter/codex/README.mdrust/crates/ccusage/src/adapter/codex/aggregate.rsrust/crates/ccusage/src/adapter/codex/loader.rsrust/crates/ccusage/src/adapter/codex/mod.rsrust/crates/ccusage/src/adapter/codex/parser.rsrust/crates/ccusage/src/adapter/codex/report.rsrust/crates/ccusage/src/adapter/codex/speed.rsrust/crates/ccusage/src/adapter/codex/types.rsrust/crates/ccusage/src/fast-multiplier-overrides.jsonrust/crates/ccusage/src/main.rsrust/crates/ccusage/src/pricing.rsrust/crates/ccusage/src/types.rs
💤 Files with no reviewable changes (1)
- codex-speed-infer-plan.md
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
All reported issues were addressed across 19 files
Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.
Re-trigger cubic
Totals and recorded-tier buckets previously repeated date filtering and period grouping independently. Route both paths through one helper so their daily, weekly, monthly, and session keys cannot drift.
ccusage
@ccusage/ccusage-darwin-arm64
@ccusage/ccusage-darwin-x64
@ccusage/ccusage-linux-arm64
@ccusage/ccusage-linux-x64
@ccusage/ccusage-win32-x64
commit: |
Recorded speed tiers are written as either "default" or "standard", and only the first was mapped. The other value was treated as an unknown tier, which both dropped the classification and cleared the tier that preceding events had established. The two spellings are not a version split. They appear in the same Codex release on the same day, and which one is written depends on the client rather than the CLI version, so this is a plain value mapping. Across recorded rollouts only "default", "standard", and "priority" occur; no other spelling exists to accommodate. Recognizing "standard" leaves almost all recorded usage classified instead of falling through to the config-based estimate.
Every thread_settings_applied event overwrote the current tier, even when it carried no service_tier key at all. Codex emits such events for auto-review threads, so a rollout that had already recorded a tier lost it as soon as one of those arrived, and the usage that followed fell back to the config-based estimate. An absent key says nothing about the tier, so the previous value now stands. A tier that is present but unrecognized keeps clearing it, since that signals a change to something unknown and a stale Fast value must not be inherited.
Replay handling picked its strategy from session_meta: a subagent rollout was routed to the turn-marker boundary only when the recorded thread source, multi-agent version, and CLI version all matched. That metadata is not trustworthy. Resuming a session re-records the original session's cli_version, so resumed rollouts are misattributed, and the gate silently reverts them to the timestamp heuristic. Worse, the marker path had no floor. Once selected, a rollout whose marker never appeared buffered every request and discarded the lot at end of file, reporting zero usage with no warning. The marker was also renamed upstream, and only the newer spelling was matched, so older rollouts would have hit exactly that case. Select the boundary by scanning the rollout instead. A file that records a turn marker uses it, which still delimits a replay spanning several seconds; anything else falls back to the repeated-second heuristic. Both marker spellings are accepted, and trigger_turn is checked because the event is emitted with either value. The marker boundary is now only chosen after a marker has actually been seen, so detection failure can no longer erase a rollout. Costs are unchanged on real rollouts, including ones that move from the timestamp path to the marker path, and the added scan is within measurement noise.
#1435 landed on main and solves replay deduplication the same problem this branch addressed, but from a better angle: it matches a forked session's leading events against the immutable parent stream instead of inferring the boundary from the child log alone. Resolution takes main's replay implementation wholesale and keeps only this branch's speed-tier work, which main does not have: - parser: main's CodexReplayState and replay_prefix plumbing, with recorded service tiers threaded back through the session entry visitor - aggregate: main's CodexAggregateRun and CodexReplayPlan wiring, with the recorded-tier dedupe records and bucket attribution kept - loader: main's replay plan threading, with tier reconciliation kept The branch's own replay work is dropped as superseded. That also retires the subagent turn-marker heuristic, its CLI-version gate, and the marker tests, which is a real improvement: the gate read session_meta cli_version, a field that resume rewrites from the original session, and a missing marker discarded a whole rollout. Matching against the parent stream depends on neither. main's replay tests are carried over intact. Codex totals are byte-identical to main on real rollouts, and recorded tiers still classify usage that config fallback would otherwise cover.
|
Merged Conflict resolution#1435 landed on main and addresses the same replay double-counting this branch did, but from a better angle: it matches a forked session's leading events against the immutable parent stream rather than inferring the boundary from the child log alone. The resolution takes main's implementation wholesale and keeps only this branch's speed-tier work, which main does not have. This branch's own replay work is dropped as superseded, which also retires the subagent turn-marker heuristic and its CLI-version gate. That is a real improvement rather than a loss:
Matching against the parent stream depends on neither. All of main's replay tests are carried over intact. Follow-up fixes
VerificationAgainst real rollouts, Codex totals are byte-identical to main (844,010,276 input tokens, $12,927.03), including the 15 MultiAgent V2 subagent rollouts (5,007,243 tokens) that exercise the replay path. The tier fix is what makes the feature actually work. On rollouts that record a tier, with
( Also worth flagging for a follow-up, not changed here: |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
rust/crates/ccusage/src/adapter/codex/loader.rs (1)
199-259: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winCover omitted-tier preservation.
Line 199 tests explicit tier transitions, but not the contract where a later
thread_settings_appliedlacksservice_tierand must retain the prior Fast/Standard value. Add a fixture sequence with a recognized tier, an omitted tier field, and a following token event.As per coding guidelines, “prefer fixture-backed parser/loader tests.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@rust/crates/ccusage/src/adapter/codex/loader.rs` around lines 199 - 259, Add a fixture-backed test case alongside records_service_tier_transitions_for_following_usage that emits a recognized service tier, then a thread_settings_applied event without service_tier, followed by token usage. Assert the token event retains the previously recognized Fast or Standard CodexServiceTier value, covering omitted-tier preservation through load_codex_events_from_directory.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@rust/crates/ccusage/src/adapter/codex/loader.rs`:
- Around line 199-259: Add a fixture-backed test case alongside
records_service_tier_transitions_for_following_usage that emits a recognized
service tier, then a thread_settings_applied event without service_tier,
followed by token usage. Assert the token event retains the previously
recognized Fast or Standard CodexServiceTier value, covering omitted-tier
preservation through load_codex_events_from_directory.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 1a9d3749-6517-4df2-befe-5b86dd592203
📒 Files selected for processing (7)
rust/crates/ccusage/src/adapter/codex/README.mdrust/crates/ccusage/src/adapter/codex/aggregate.rsrust/crates/ccusage/src/adapter/codex/loader.rsrust/crates/ccusage/src/adapter/codex/mod.rsrust/crates/ccusage/src/adapter/codex/parser.rsrust/crates/ccusage/src/adapter/codex/types.rsrust/crates/ccusage/src/pricing.rs
💤 Files with no reviewable changes (1)
- rust/crates/ccusage/src/adapter/codex/types.rs
🚧 Files skipped from review as they are similar to previous changes (4)
- rust/crates/ccusage/src/adapter/codex/README.md
- rust/crates/ccusage/src/adapter/codex/mod.rs
- rust/crates/ccusage/src/adapter/codex/aggregate.rs
- rust/crates/ccusage/src/pricing.rs
LiteLLM now ships gpt-5.6 as its own entry. Exact matches win over the alias, so the bare family name stopped inheriting the Sol variant's long-context rates and Fast multiplier, and fell back to flat pricing at a 200K boundary. Resolve the alias inside the built-in long-context lookup and the Fast multiplier lookup instead of at individual call sites, so an alias bills at the variant it points to no matter where upstream data happens to cover it. That also removes the duplicate alias resolution long_context_split_threshold was doing.
67db3f4 to
3f35031
Compare
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
ccusage-guide | 3f35031 | Commit Preview URL Branch Preview URL |
Jul 26 2026, 07:48 PM |
|
Correction to my earlier comment: the OpenAI's API pricing lists Priority tiers as:
So One observation rather than a defect: the embedded LiteLLM snapshot carries no |
|
@coderabbitai review it |
|
✅ Action performedReview finished.
|
The loader covered explicit tier transitions and unclassified rollouts, but not the contract that a thread_settings_applied event carrying no service_tier key leaves the recorded tier in place. Codex emits exactly that shape for auto-review threads, so the gap sat on the case the behaviour exists for. The fixture also pins the opposite case in the same sequence: a tier that is present but unrecognized still clears the recorded value.
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
Historical audit: this pull request was auto-closed by the legacy contributor gate. That closure did not assess technical importance. Audit result: resolved. A later merged change or the current |
ryoppippi
added
the
triage:resolved
Summary
Correct usage reports that can count inherited child-session history more than once, and fix pricing paths involving recorded speed tiers, default family aliases, and partial overrides.
Recent rollouts now preserve explicit Fast and Standard settings per request. Auto mode honors those records, falls back to configuration only for unclassified usage, and explicit speed flags override the complete report.
What Changed
Notes
Verification
Related: #1434, #1436
Summary by CodeRabbit
--speed autobehavior over time.