Skip to content

Lead the digest with busy time and take its idle patterns in correlate - #30

Merged
lhotari merged 1 commit into
mainfrom
busy-first-digest
Sep 24, 2026
Merged

lhotari merged 1 commit into
mainfrom
busy-first-digest

Conversation

@lhotari

@lhotari lhotari commented Sep 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

The digest (jonoffcpu-summary.md / .json) wasn't useful on its own:

  • Busy and idle were ranked side by side, and the idle table listed waits nobody needs.
  • Idle waits were known only through preset:jvm-idle, so an application's own idle waits landed in the busy tables. On an Apache Pulsar broker capture, 87 % of the digest's "busy" time was BookKeeper workers waiting for a task (GrowableBatchedArrayBlockingQueue.takeAll).

This PR makes the digest the idle-filtered summary.

  • correlate --idle REGEX / --idle-from FILE (default preset:jvm-idle) set the idle patterns for the digest alone.
    • The collapsed stacks, the profile and the report keep every interval.
    • The patterns are resolved before correlating, so a bad pattern fails at once.
    • Two combinations are rejected: these options with --summary-output false, and these options in partial mode.
  • The digest is busy-first:
    • "Where the time went" leads with Busy, then "Idle, left out below" and All selected, and keeps the over-exclusion check.
    • A note under that table says the tables and stacks below cover busy time only, names the idle patterns, and gives the top command that lists the idle waits.
    • The busy tables and the heaviest stacks (now titled "Heaviest busy stacks") leave idle intervals out, as before.
    • The idle table is gone.
  • summarize writes the same layout, and its help and the shared --idle description say what idle now means for the digest.

On that broker capture, with --idle-from pointing at the harness's idle list (the preset plus BookKeeper's …BlockingQueue.take(All)), busy time comes to 43.371 s. That matches the harness's own idle-filtered flame graph, and the busy table now ranks thread start, monitor and lock contention.

Compatibility

  • jonoffcpu-summary.json: Digest.idle (field 10) is removed and reserved. The idle totals remain in whereTheTimeWent.
  • Markdown: "Where the time went" rows are reordered. There's no idle section, and the heaviest stacks section is renamed.

Verification

  • ./gradlew spotlessCheck :jonoffcpu-agent:check :jonoffcpu-correlator:check :jonoffcpu-jfr-converter:check :jonoffcpu-capture-codec:check passes: 302 tests, 0 failures.
  • New assertions check four things:
    • correlate --idle-from with a pattern matching every frame leaves no busy time in the digest.
    • It leaves the collapsed stacks and the profile byte-identical.
    • The digest names the pattern file.
    • --idle is rejected with --summary-output false and in partial mode.
  • The digest test checks the busy-first layout, and that the default preset's idle interval stays out of the busy table.
  • Help snapshots for correlate, summarize, top and the root command are regenerated. The README option table is checked by readmeTest.

The digest ranked busy and idle time side by side, and it knew idle
waits only through preset:jvm-idle. An application's own idle waits,
such as BookKeeper's task queues in a Pulsar broker, therefore filled
its busy tables: 87 % of one broker's "busy" time was a worker waiting
for a task, and the idle table beside it listed waits nobody needs.

correlate now takes --idle and --idle-from (default preset:jvm-idle)
for the digest alone; the collapsed stacks, the profile and the report
keep every interval. The digest is about busy time: where the time went
leads with it, idle intervals are counted there and left out of every
table and stack, and the Markdown names the idle patterns and the top
command that lists the waits it left out. The idle table is removed
from the Digest message, its field reserved. summarize writes the same
layout.
@lhotari lhotari added the enhancement New feature or request label Sep 24, 2026
@lhotari
lhotari merged commit 598fbdf into main Sep 24, 2026
7 checks passed
@lhotari
lhotari deleted the busy-first-digest branch September 24, 2026 19:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant