Skip to content

Codex loader file ordering is nondeterministic, causing Rust test failure #1105

Description

@haru0416-dev

What happened?

The Rust test suite fails in tests::loads_codex_token_count_events_in_parallel because Codex session files are loaded in a nondeterministic filesystem order.

Observed with:

cargo test --manifest-path rust/Cargo.toml --workspace

Failure:

---- tests::loads_codex_token_count_events_in_parallel stdout ----
thread 'tests::loads_codex_token_count_events_in_parallel' panicked at crates/ccusage/src/main.rs:649:9:
assertion `left == right` failed
  left: "session-b"
 right: "session-a"

This is distinct from the existing Codex overcounting issues (#950, #988). This report is about deterministic file/event ordering and test reliability, not token totals or cost inflation.

Steps to reproduce

  1. Check out the current repository.

  2. Run the targeted test:

    cargo test --manifest-path rust/Cargo.toml -p ccusage --bin ccusage tests::loads_codex_token_count_events_in_parallel -- --exact
  3. Or run the full Rust workspace suite:

    cargo test --manifest-path rust/Cargo.toml --workspace

Expected behavior

Codex event loading should be deterministic for a fixed set of session files, or tests should not assume an ordering that the loader does not guarantee.

At minimum:

  • load_codex_events_from_directory(..., true) and (..., false) should remain equivalent.
  • The test should not depend on filesystem read_dir ordering.
  • Any fix should preserve Codex dedupe and token/cost aggregation semantics.

Evidence-gated analysis

The failing test creates two files:

  • session-a.jsonl
  • session-b.jsonl

It then compares single-threaded and parallel Codex loading. The equality assertion passes before the failing order assertion:

assert_eq!(parallel_events, single_thread_events);
assert_eq!(parallel_events[0].session_id, "session-a");
assert_eq!(parallel_events[1].session_id, "session-b");

So the evidence does not support “parallel loading corrupts the order relative to single-thread loading.” Both modes agree with each other. The failure is that the shared baseline order is session-b, then session-a, while the test expects session-a, then session-b.

Source context:

  • rust/crates/ccusage/src/codex_loader.rs collects files with collect_usage_files(sessions_dir, &mut files) and does not sort them before loading.
  • The Claude loader sorts discovered usage files with files.sort_by_cached_key(|path| path.to_string_lossy().into_owned()).

Local probes run before a minimal patch:

  • Targeted exact test: 3/3 failed with session-b before session-a.
  • Targeted filtered test: 10/10 failed with the same assertion.
  • Full Rust suite: failed only this test in the ccusage binary suite.

A local one-line patch that sorts Codex files after collection made the targeted test and full Rust suite pass:

collect_usage_files(sessions_dir, &mut files);
files.sort_by_cached_key(|path| path.to_string_lossy().into_owned());

Post-patch verification run locally:

cargo test --manifest-path rust/Cargo.toml -p ccusage --bin ccusage tests::loads_codex_token_count_events_in_parallel -- --exact
cargo test --manifest-path rust/Cargo.toml -p ccusage --bin ccusage tests::builds_codex_daily_json_report -- --exact
cargo test --manifest-path rust/Cargo.toml --workspace

All passed after the sort patch.

Suggested fix

Sort collected Codex usage files before single-thread or parallel loading, mirroring Claude loader behavior:

collect_usage_files(sessions_dir, &mut files);
files.sort_by_cached_key(|path| path.to_string_lossy().into_owned());

Likely patch target:

rust/crates/ccusage/src/codex_loader.rs

Non-goals / caveats

Activity

  1. github-actions commented on May 20, 2026

    @github-actions
    Contributor

    This issue was auto-closed. Issues from new contributors are auto-closed by default.

    Maintainers review auto-closed issues and reopen worthwhile ones. Issues that do not meet the quality bar in CONTRIBUTING.md may not be reopened or receive a reply.

    Keep the issue short, concrete, and written in your own voice.

    If a maintainer replies lgtmi, your future issues will stay open. If a maintainer replies lgtm, your future issues and PRs will stay open.

    See CONTRIBUTING.md.

  2. added a commit that references this issue on May 25, 2026
    63cf17e
  3. ryoppippi commented on Aug 31, 2026

    @ryoppippi
    Member

    Historical audit: this discussion was auto-closed by the legacy contributor gate. That closure did not assess technical importance.

    Audit result: resolved. A later merged change or the current main implementation covers this request. This item is kept for history and does not need to be reopened.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    triage:resolvedResolved by a later change or current implementation.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions