perf: cache PricingMap::find() results and skip redundant opencode pricing checks by turtton · Pull Request #1358 · ccusage/ccusage

turtton · 2026-06-22T13:09:08Z

Thank you for your interest in my PR!

Summary

bunx ccusage@latest opencode was taking over two minutes on my machine with a large OpenCode history (~3.6 GB, 87k messages). Profiling showed that PricingMap::find() accounted for 91% of execution time — despite only 47 unique model names being present, each of the 87k messages independently repeated the full fuzzy lookup through ~2,200 pricing entries.

This PR addresses the issue with two independent optimizations:

A result cache for PricingMap::find() so repeated lookups for the same model complete in O(1)
Skipping the redundant missing-pricing check in the OpenCode adapter when cost is already known

Changes

`rust/crates/ccusage/src/pricing.rs`

Added find_cache: OnceLock<Mutex<FxHashMap<String, Option<Pricing>>>> to PricingMap
find() now checks the cache first; on cache miss the full lookup runs and the result (including None for models not in pricing) is stored for subsequent calls
Added clear_find_cache() called from load_json_with_overrides(), load_models_dev_models(), and apply_overrides() to maintain consistency when the pricing table is mutated
Changed from derived Default to manual implementation to accommodate the OnceLock field
Fixed Mutex lock poisoning recovery to use unwrap_or_else(|error| error.into_inner()) consistently across all cache operations

`rust/crates/ccusage/src/adapter/opencode/parser.rs`

Both calculate_open_code_cost and missing_open_code_pricing independently iterated the same model candidates. When cost calculation already returned a positive cost (cost > 0.0), the missing-pricing check is now skipped entirely.

Benchmark

Measured on the same machine with the same 3.6 GB OpenCode history:

$ hyperfine --warmup 1 --runs 3 \
    "ccusage opencode" (before) \
    "ccusage opencode" (after)

  Before: 122.2s
  After:   2.3s  (54× faster)

The cache is in the shared PricingMap layer, so other adapters (Claude Code, Codex, Amp, etc.) may benefit from it as well. Memory overhead is proportional to the number of unique model names looked up, typically a few kilobytes.

Checklist

direnv exec . just fmt
direnv exec . just test (368/368 passed)
cargo build --release succeeds
Verified output parity (NO_COLOR=1 TZ=UTC) between cached and uncached paths

^{Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.}

Summary by cubic

Cache results of PricingMap::find() and skip redundant OpenCode pricing checks to massively speed up large runs (e.g., ccusage opencode from ~122s to ~2.3s). The cache lives in the shared pricing layer, so other adapters benefit too.

Refactors
- Memoize find() results (including misses) in an OnceLock<Mutex<FxHashMap>> keyed by model.
- Invalidate the cache when pricing data changes (JSON loads, dev models, overrides).
- In OpenCode, skip the missing-pricing check when a positive cost is already known.

^{Written for commit 82c5d84. Summary will update on new commits.}

Summary by CodeRabbit

Refactor
- Optimized pricing lookups with caching for improved resolution speed
- Enhanced computational efficiency by eliminating redundant calculations in cost determination

…zy matching PricingMap::find() does an exact HashMap lookup followed by expensive fuzzy matching through all ~2,200 pricing entries when the exact model name is not in the map. When adapters repeatedly query the same model names, a large fraction of lookups miss the HashMap and trigger a full scan of the pricing table for every call. Add a OnceLock<Mutex<FxHashMap>> cache that memoizes find() results by model name (including None for models not found in pricing). Once a model name has been resolved, future lookups complete in O(1) instead of O(n) over the pricing table. Also add clear_find_cache() called from load_json_with_overrides(), load_models_dev_models(), and apply_overrides() so the cache stays consistent when the pricing table is mutated.

calculate_open_code_cost and missing_open_code_pricing independently iterate through the same model candidates. When the cost calculation already found a valid positive cost (either from a stored cost_usd field or from pricing lookup), skip the missing-pricing check entirely since pricing was already resolved.

pullfrog · 2026-06-22T13:09:12Z

no API key found — this repo is configured to use deepseek/deepseek-v4-pro, which needs DEEPSEEK_API_KEY, but the runner has no key for it.

To fix: add the key as a GitHub Actions secret (referenced from your workflow's env: block) or as a Pullfrog secret in the console — or switch this repo to a different model (free models need no key).

Open repo secrets → · Configure model → · Setup docs → · Ask in Discord →

^{｜ Rerun failed job ➔ ｜ View workflow run ｜ via Pullfrog ｜ Using DeepSeek Pro (free via Pullfrog for OSS) ｜ 𝕏}

coderabbitai · 2026-06-22T13:09:25Z

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 50885666-2115-41c6-aa8d-fb052b33badb

📥 Commits

Reviewing files that changed from the base of the PR and between a98e374 and 82c5d84.

📒 Files selected for processing (2)

rust/crates/ccusage/src/adapter/opencode/parser.rs
rust/crates/ccusage/src/pricing.rs

📝 Walkthrough

Walkthrough

Two performance optimizations targeting slow pricing lookups: PricingMap::find() gains a thread-safe memoization cache (OnceLock<Mutex<FxHashMap>>) that stores results including misses and is invalidated on data load or override; the OpenCode parser skips a redundant missing_open_code_pricing call when cost is already positive.

Changes

Pricing Lookup Performance

Layer / File(s)	Summary
PricingMap cache field and Default impl `rust/crates/ccusage/src/pricing.rs`	`PricingMap` drops `#[derive(Default)]`, adds `find_cache: OnceLock<Mutex<FxHashMap<String, Option<Pricing>>>>`, and gains an explicit `Default` impl to initialize the new field alongside existing maps.
`find()` cache fast-path and write-back `rust/crates/ccusage/src/pricing.rs`	`find()` locks `find_cache` and returns immediately on a cache hit; on a miss it runs the full alias/fuzzy/fallback resolution without holding the lock, then re-acquires and writes the result (including `None`) back into the cache.
Cache invalidation on data load and override `rust/crates/ccusage/src/pricing.rs`	`clear_find_cache()` is added to lock and clear the memoized entries (handling poisoned locks), and is called after LiteLLM JSON load, models.dev merge, and override application.
OpenCode parser missing-pricing short-circuit `rust/crates/ccusage/src/adapter/opencode/parser.rs`	`message_value_to_entry` now sets `missing_pricing_model = None` directly when `cost > 0.0`, skipping the call to `missing_open_code_pricing()`.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐇 Hoppy news from the warren today,
The pricing lookups zoom right away!
A cache now remembers each model's price,
No more fuzzy scanning—isn't that nice?
When cost is found, we skip the rest,
This speedy rabbit approves: 10/10 best! ✨

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

github-actions · 2026-06-22T13:09:33Z

This PR was auto-closed. Only contributors approved with lgtm can open PRs. Open an issue first.

Maintainers review auto-closed issues and reopen worthwhile ones. Issues that do not meet the quality bar in CONTRIBUTING.md may not be reopened or receive a reply.

If a maintainer replies lgtmi, your future issues will stay open. If a maintainer replies lgtm, your future issues and PRs will stay open.

See CONTRIBUTING.md.

ryoppippi · 2026-08-31T14:52:59Z

Historical audit: this pull request was auto-closed by the legacy contributor gate. That closure did not assess technical importance.

Audit result: needs review. The current state does not prove resolution, but a fresh technical or product check is required before deciding whether the underlying request is still relevant. This closed PR will not be revived as-is; create a new PR only after reviewing the related issue and current main.

turtton added 2 commits June 22, 2026 21:35

github-actions Bot closed this Jun 22, 2026

ryoppippi mentioned this pull request Jul 8, 2026

perf: cache PricingMap::find() results and skip redundant opencode pricing checks #1407

Merged

ryoppippi added the triage:needs-review Requires a fresh technical or product decision. label Aug 31, 2026

Navigation Menu

Repository navigation

perf: cache PricingMap::find() results and skip redundant opencode pricing checks - #1358

perf: cache PricingMap::find() results and skip redundant opencode pricing checks#1358
turtton wants to merge 2 commits into
ccusage:mainfrom
turtton:perf/pricing-cache-opencode-skip

turtton commented Jun 22, 2026 •

edited

Loading

Uh oh!

pullfrog Bot commented Jun 22, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Jun 22, 2026 •

edited

Loading

Review failed

Walkthrough

Changes

Estimated code review effort

Poem

Uh oh!

github-actions Bot commented Jun 22, 2026

Uh oh!

ryoppippi commented Aug 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

turtton commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

rust/crates/ccusage/src/pricing.rs

rust/crates/ccusage/src/adapter/opencode/parser.rs

Benchmark

Checklist

Summary by cubic

Summary by CodeRabbit

Uh oh!

pullfrog Bot commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review failed

Walkthrough

Changes

Estimated code review effort

Poem

Uh oh!

github-actions Bot commented Jun 22, 2026

Uh oh!

ryoppippi commented Aug 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

turtton commented Jun 22, 2026 •

edited

Loading

`rust/crates/ccusage/src/pricing.rs`

`rust/crates/ccusage/src/adapter/opencode/parser.rs`

pullfrog Bot commented Jun 22, 2026 •

edited

Loading

coderabbitai Bot commented Jun 22, 2026 •

edited

Loading