Skip to content

fix(utils): stop double-counting cumulative cache invocation counters - #7400

Closed
rioyu123 wants to merge 1 commit into
google:mainfrom
rioyu123:fix/cache-invocation-counts
Closed

rioyu123 wants to merge 1 commit into
google:mainfrom
rioyu123:fix/cache-invocation-counts

Conversation

@rioyu123

@rioyu123 rioyu123 commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

Link to Issue or Description of Change

No existing issue found; the bug and reproduction are described below.

Problem:

CachePerformanceAnalyzer adds invocations_used from every event, but that field is a cumulative counter for a cache instance. A cache used across three invocations produces snapshots 1, 2, 3; the analyzer reports six invocations and an average of two per cache instead of three for both. Multiple model turns in one invocation repeat the same counter and inflate the result further.

To reproduce:

  1. Create an InMemorySessionService session.
  2. Append three Event records for the same agent and cache resource, with CacheMetadata.invocations_used set to 1, 2, and 3.
  3. Call analyze_agent_cache_performance() for that session and agent.

Expected: total_invocations=3, avg_invocations_used=3.0.
Actual: total_invocations=6, avg_invocations_used=2.0.

Reproduced on ADK main 86a47f6, on Windows and Linux. No model, LiteLLM, cloud credentials, or paid API calls are needed.

Solution:

Keep the highest cumulative counter for each cache resource name, then calculate the existing total and mean from those per-cache values. Refreshed caches remain distinct even when their fingerprints match.

This fixes the reported reuse statistics without changing cache creation, reuse, invalidation, request/token metrics, or public signatures and result keys.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally in the clean environments below.

Eight added cases cover successive invocations, repeated turns, cache refresh, out-of-order snapshots, single-snapshot controls, fingerprint-only records, and zero counters. Existing tests are unchanged.

  • Exact final analyzer tests against the original source: 4 failed, 18 passed.
  • Analyzer and cache metadata tests after the fix: 36 passed.
  • Full tox suites pass on Python 3.10, 3.11, 3.12, 3.13, and 3.14: 17,427 passed on 3.11 and 17,418 passed on each other version. Existing skips and xfail/xpass results remain.
  • Changed-file pre-commit hooks and source-file Mypy pass; uv build succeeds.

Manual End-to-End (E2E) Tests:

Ran an offline integration using the real ContextCacheRequestProcessor, session service, events, and analyzer. Cache creation was represented by initial metadata at the cloud boundary. Three invocations, including an extra turn in the second invocation, produce snapshots [1, 2, 2, 3]:

                         before    after
total_invocations             8        3
avg_invocations_used        2.0      3.0
total_requests                4        4
total_prompt_tokens         400      400
total_cached_tokens         320      320

Also installed the built wheel in a clean environment and repeated the offline integration. This does not exercise a live model or cache-provider API.

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes in the clean environments above.
  • I have manually tested my changes end-to-end (offline scope described above).
  • Any dependent changes have been merged and published in downstream modules (none required).

Additional context

The existing docstring already describes per-cache statistics; this restores that behavior without adding a new metric or changing documentation contracts.

The first Python 3.12 run used the host's system interpreter and failed two import-allowlist tests because it automatically loads sitecustomize. Both also fail on unmodified main in that environment. The full 3.12 suite was rerun successfully using an isolated, uv-managed Python 3.12.13 interpreter, without changing project code or the allowlist.

copybara-service Bot pushed a commit that referenced this pull request Oct 8, 2026
@adk-bot

adk-bot commented Oct 8, 2026

Copy link
Copy Markdown
Collaborator

Thank you @rioyu123 for your contribution! 🎉

Your changes have been successfully imported and merged via Copybara in commit 1492ff7.

Closing this PR as the changes are now in the main branch.

@adk-bot adk-bot added the merged [Status] This PR is merged label Oct 8, 2026
@adk-bot adk-bot closed this Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merged [Status] This PR is merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants