Skip to content

codex: re-emitted token_count events are double-counted (usage overstated ~30-67% on real data) #1288

Description

@maxmoneycash

Summary

The codex adapter sums last_token_usage per token_count event, deduping on a key that includes the millisecond timestamp. But Codex CLI re-emits token_count events with an unchanged total_token_usage (UI/rate-limit refreshes, not new API calls), and each re-emit has a fresh timestamp — so every re-emit is counted as new usage. Forked/subagent sessions amplify this: at spawn they replay the parent thread's history as a sub-second burst of token_count events that are pure duplicates of values already counted in the parent file.

On my real ~/.codex/sessions (229 files, ~141k token_count events, 5,357 of 9,157 events in the largest file are pure re-emits): ccusage reports 8.58B codex tokens; counting deltas of the cumulative total_token_usage per session (replay baselines removed) gives 5.15B — a ~67% overstatement. Verified in v20.0.9 and v20.0.11.

Repro fixture

One session, two real turns (cumulative ends at 3,300 tokens), with two re-emits of the first turn's counter — the kind Codex writes constantly:

{"timestamp":"2026-05-01T10:00:00.000Z","type":"session_meta","payload":{"id":"01970000-0000-7000-8000-000000000001","timestamp":"2026-05-01T10:00:00.000Z","cwd":"/tmp","originator":"codex_cli_rs","cli_version":"0.73.0","source":"cli","model_provider":"openai"}}
{"timestamp":"2026-05-01T10:00:01.000Z","type":"turn_context","payload":{"model":"gpt-5"}}
{"timestamp":"2026-05-01T10:00:02.000Z","type":"event_msg","payload":{"type":"token_count","info":{"total_token_usage":{"input_tokens":1000,"cached_input_tokens":800,"output_tokens":100,"reasoning_output_tokens":50,"total_tokens":1100},"last_token_usage":{"input_tokens":1000,"cached_input_tokens":800,"output_tokens":100,"reasoning_output_tokens":50,"total_tokens":1100},"model_context_window":258400},"rate_limits":{}}}
{"timestamp":"2026-05-01T10:00:02.500Z","type":"event_msg","payload":{"type":"token_count","info":{"total_token_usage":{"input_tokens":1000,"cached_input_tokens":800,"output_tokens":100,"reasoning_output_tokens":50,"total_tokens":1100},"last_token_usage":{"input_tokens":1000,"cached_input_tokens":800,"output_tokens":100,"reasoning_output_tokens":50,"total_tokens":1100},"model_context_window":258400},"rate_limits":{}}}
{"timestamp":"2026-05-01T10:00:03.000Z","type":"event_msg","payload":{"type":"token_count","info":{"total_token_usage":{"input_tokens":1000,"cached_input_tokens":800,"output_tokens":100,"reasoning_output_tokens":50,"total_tokens":1100},"last_token_usage":{"input_tokens":1000,"cached_input_tokens":800,"output_tokens":100,"reasoning_output_tokens":50,"total_tokens":1100},"model_context_window":258400},"rate_limits":{}}}
{"timestamp":"2026-05-01T10:00:10.000Z","type":"event_msg","payload":{"type":"token_count","info":{"total_token_usage":{"input_tokens":3000,"cached_input_tokens":2300,"output_tokens":300,"reasoning_output_tokens":130,"total_tokens":3300},"last_token_usage":{"input_tokens":2000,"cached_input_tokens":1500,"output_tokens":200,"reasoning_output_tokens":80,"total_tokens":2200},"model_context_window":258400},"rate_limits":{}}}

Save as $CODEX_HOME/sessions/2026/05/01/rollout-2026-05-01T10-00-00-01970000-0000-7000-8000-000000000001.jsonl, then:

CODEX_HOME=... ccusage codex monthly --json --offline

Expected: totalTokens = 3,300 (the final cumulative).
Actual (v20.0.9–v20.0.11): totalTokens = 5,500 (the two re-emits of the 1,100 counter are added again).

Suggested fix

Count per-session deltas of total_token_usage (only when the cumulative moves), or dedupe last_token_usage events on the usage payload alone (excluding timestamp) and skip events whose cumulative is unchanged. For forked sessions, the replayed burst at spawn re-states cumulative values already present in the parent file and needs a baseline subtraction.

Thanks for ccusage — happy to test a fix against my real data.

Activity

  1. pullfrog commented on Jun 11, 2026

    @pullfrog
    Contributor
  2. github-actions commented on Jun 11, 2026

    @github-actions
    Contributor

    This issue was auto-closed. Issues from new contributors are auto-closed by default.

    Maintainers review auto-closed issues and reopen worthwhile ones. Issues that do not meet the quality bar in CONTRIBUTING.md may not be reopened or receive a reply.

    Keep the issue short, concrete, and written in your own voice.

    If a maintainer replies lgtmi, your future issues will stay open. If a maintainer replies lgtm, your future issues and PRs will stay open.

    See CONTRIBUTING.md.

  3. ryoppippi commented on Aug 31, 2026

    @ryoppippi
    Member

    Historical audit: this discussion was auto-closed by the legacy contributor gate. That closure did not assess technical importance.

    Audit result: resolved. A later merged change or the current main implementation covers this request. This item is kept for history and does not need to be reopened.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingtriage:resolvedResolved by a later change or current implementation.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions