fix(codex): correct legacy token total fallback by MaxGhenis · Pull Request #1458 · ccusage/ccusage

MaxGhenis · 2026-07-19T12:23:13Z

Summary

derive a missing Codex total_tokens value as input plus output
avoid adding reasoning a second time, since Codex reports reasoning as a subset of output
keep the existing explicit-zero compatibility behavior unchanged
add regression coverage with nonzero reasoning and no source total

The expected invariant and fixture/oracle method follow the Logpile accounting review. The exact 2026-07-18 matrix behind this change is published as the 2026-07-18 ccusage verification.

Testing

cargo +1.96.0 test --manifest-path rust/Cargo.toml --workspace --offline (445 passed)
focused absent-total and explicit-zero regressions pass
all six legacy rows across the patched harness/full-CLI matrix now report the oracle totalTokens

Prepared with AI assistance; I reviewed the implementation and test results.

^{Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.}

Summary by cubic

Fixes Codex legacy total_tokens fallback to compute correct totals and avoid double-counting reasoning tokens. Preserves explicit-zero behavior and adds regression tests for missing totals and explicit zero.

Bug Fixes
- If total_tokens is missing: derive as input + output (exclude reasoning).
- If total_tokens is 0: derive as input + output + reasoning (keep legacy semantics).
- Added focused tests for both cases.

^{Written for commit 705c85f. Summary will update on new commits.}

Summary by CodeRabbit

Bug Fixes
- Corrected token usage totals when usage data omits a total, preventing reasoning tokens from being counted twice.
- Improved handling of explicitly reported zero totals by calculating usage from the available input, output, and reasoning values.
- Preserved explicitly provided nonzero totals without modification.

Prepared with AI assistance (audit + implementation: gpt-5.6-sol; review: Claude Fable 5); reviewed and tested before filing. Co-Authored-By: gpt-5.6-sol <[email protected]> Co-Authored-By: Claude Fable 5 <[email protected]>

github-actions · 2026-07-19T12:23:23Z

This PR was auto-closed. Only contributors approved with lgtm can open PRs. Open an issue first.

Maintainers review auto-closed issues and reopen worthwhile ones. Issues that do not meet the quality bar in CONTRIBUTING.md may not be reopened or receive a reply.

If a maintainer replies lgtmi, your future issues will stay open. If a maintainer replies lgtm, your future issues and PRs will stay open.

See CONTRIBUTING.md.

coderabbitai · 2026-07-19T12:23:45Z

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: fdb00f8f-f870-45cd-a55f-37e56845364b

📥 Commits

Reviewing files that changed from the base of the PR and between 7acee6c and 705c85f.

📒 Files selected for processing (1)

rust/crates/ccusage/src/adapter/codex/types.rs

📝 Walkthrough

Walkthrough

Codex usage deserialization now excludes reasoning tokens when deriving an omitted total, while an explicit zero derives the total from input, output, and reasoning tokens. Unit tests cover both cases.

Changes

Codex token derivation

Layer / File(s)	Summary
Total token derivation and validation `rust/crates/ccusage/src/adapter/codex/types.rs`	`CodexRawUsage` derives omitted totals from input and output, derives explicit zero totals from all components, preserves nonzero totals, and tests the omitted and zero behaviors.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

ccusage/ccusage#1122: Both changes update CodexRawUsage total token deserialization and test missing versus explicit zero totals.
ccusage/ccusage#1137: Both changes modify CodexRawUsage deserialization logic in types.rs.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

ryoppippi · 2026-07-27T13:18:10Z

Thanks — the bug is real and this is now landing in #1499 with your commit kept intact.

I could not update this branch in place: MaxGhenis/ccusage is a fork of jackcpku/ccusage rather than a direct fork of this repository, so GitHub reports maintainer_can_modify: true but rejects the push. Hence the new PR rather than a rebase here.

Two notes on what changed on top of your commit:

Rebased onto main. The conflict was purely positional — perf(rust): split the workspace into independently cached crates #1428 moved the adapter to rust/adapters/codex/src/types.rs.
The Some(0) branch is wrong for the same reason as the absent case, so it now takes the same path. A recorded total_tokens of zero alongside nonzero components means the field is unusable, not that the turn spent nothing; keeping reasoning there left the inflated sum for exactly the records that cannot be trusted to report a total. The loader.rs assertion that encoded the old value moved from 14 to 13.

For the record, the invariant was confirmed against real logs rather than fixtures. Across 170,429 token_count records in a local ~/.codex/sessions, 21,815 carry nonzero reasoning_output_tokens, and total_tokens == input_tokens + output_tokens holds for 21,815 of 21,815 while input_tokens + output_tokens + reasoning_output_tokens holds for 0. docs/guide/codex/index.md already documented the same rule.

Closing in favor of #1499.

…1499) Codex reports `reasoning_output_tokens` as a subset of `output_tokens`, so `total_tokens` is `input_tokens + output_tokens`. The fallback used when a record omits `total_tokens` added reasoning on top, inflating the reported total: a saved `codex exec --json` turn with 100 input, 50 output and 20 reasoning tokens reported 170 instead of 150. Derive the total as input plus output, and treat a recorded zero the same way. A zero alongside nonzero components means the field is unusable, not that the turn spent nothing. Defer and saturate the sum so a corrupt log carrying near-`u64` components cannot panic a debug build or wrap in release. Verified against real logs rather than fixtures. Across 170,429 `token_count` records, the 21,815 carrying nonzero reasoning satisfy `total_tokens == input_tokens + output_tokens` in every case and `input_tokens + output_tokens + reasoning_output_tokens` in none. `docs/guide/codex/index.md` already documented the same rule, noting that reasoning tokens are part of the output charge rather than billed separately, so the code now matches the guide. Rollout session logs always record a usable total, so this only affects saved exec JSON usage, where OpenAI-style payloads may omit the field. Costs were never affected because pricing reads input, cached input and output directly. Supersedes #1458, whose branch could not be updated in place because `MaxGhenis/ccusage` is a fork of `jackcpku/ccusage` rather than a direct fork of this repository. Co-authored-by: Max Ghenis <[email protected]>

ryoppippi · 2026-08-31T14:44:36Z

Historical audit: this pull request was auto-closed by the legacy contributor gate. That closure did not assess technical importance.

Audit result: needs review. The current state does not prove resolution, but a fresh technical or product check is required before deciding whether the underlying request is still relevant. This closed PR will not be revived as-is; create a new PR only after reviewing the related issue and current main.

fix(codex): correct legacy token total fallback

705c85f

Prepared with AI assistance (audit + implementation: gpt-5.6-sol; review: Claude Fable 5); reviewed and tested before filing. Co-Authored-By: gpt-5.6-sol <[email protected]> Co-Authored-By: Claude Fable 5 <[email protected]>

github-actions Bot closed this Jul 19, 2026

MaxGhenis mentioned this pull request Jul 19, 2026

Fix Codex token delta accounting jackcpku/ccusage#1

Closed

MaxGhenis mentioned this pull request Jul 19, 2026

Codex fork replays spanning multiple seconds are counted as native usage #1460

Closed

ryoppippi reopened this Jul 23, 2026

This was referenced Jul 25, 2026

fix(codex): dedupe replayed usage #1435

Merged

fix(codex): stop double-counting reasoning in derived token totals #1499

Merged

ryoppippi closed this Jul 27, 2026

ryoppippi mentioned this pull request Jul 27, 2026

fix(codex): detect structural fork replay boundaries #1457

Closed

ryoppippi added the triage:needs-review Requires a fresh technical or product decision. label Aug 31, 2026

Navigation Menu

Repository navigation

fix(codex): correct legacy token total fallback - #1458

fix(codex): correct legacy token total fallback#1458
MaxGhenis wants to merge 1 commit into
ccusage:mainfrom
MaxGhenis:codex-legacy-total-fallback

MaxGhenis commented Jul 19, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

github-actions Bot commented Jul 19, 2026

Uh oh!

coderabbitai Bot commented Jul 19, 2026 •

edited

Loading

Review failed

Walkthrough

Changes

Uh oh!

ryoppippi commented Jul 27, 2026

Uh oh!

ryoppippi commented Aug 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

MaxGhenis commented Jul 19, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Testing

Summary by cubic

Summary by CodeRabbit

Uh oh!

github-actions Bot commented Jul 19, 2026

Uh oh!

coderabbitai Bot commented Jul 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review failed

Walkthrough

Changes

Uh oh!

ryoppippi commented Jul 27, 2026

Uh oh!

ryoppippi commented Aug 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

MaxGhenis commented Jul 19, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jul 19, 2026 •

edited

Loading