Repository navigation
Root the digest at the application and make it readable - #31
Merged
Merged
Conversation
The digest never used an application pattern: correlate built it without one, so it ranked stack leaves, which name the wait mechanism (AbstractQueuedSynchronizer.acquireSharedInterruptibly, a C2 monitor) rather than the code that waited. Busy time without an application frame dominated its tables (40.27 of 40.57 busy seconds on a Pulsar broker), and --root-at often landed on an executor or a lambda bridge instead of the work. stacks and top gain --root-at-unmatched hide, the one transform that removes weight: unmatched entries leave the lines and rows and are reported as a total of their own (rootAtUnmatchedHidden in the stacks summary, a totals row and the pool table in top). preset:jvm-dispatch hides lambda bridges and JDK/Guava executor adapters so the root is the task's code. top gains --by root (the flame graph's root, with the heaviest line under it) and --by app-method (application methods across all stacks, inclusive, call chains grouped, with self time); with hiding, boundary shares are of the busy time with an application frame. correlate and summarize take --app and --hide/--hide-from (default preset:jvm-dispatch). With --app the digest is computed from the transforms of the application-rooted flame graph and leads with the application method that waited, then the roots, the heaviest application stacks and the application methods, followed by where the time went and the unattributed busy time by pool, with a note when it is more than half of the busy time. The Digest message gains schemaVersion 2 and the new tables. Without --app the Markdown is unchanged, which a golden file generated from the previous code checks.
Nothing before the 1.0.0 release needs to read old files or accept old spellings, so the compatibility kept so far is removed rather than carried: the digest's schemaVersion and its golden file of the layout without --app, the reserved field declarations of the analysis, profile and report messages, the deprecated `--dump` alias (the signal pressure tool now runs `dump`), the schema 1 fallback wording of the package-name rule and merge, a SourceColumns.add overload without a switch-out reason, and the digest's 16 KB size bound. export no longer keeps its pre-0.5.0 columns first: CSV and JSON Lines share one order, the run and the entry's identity, its counters, then its stacks, and ExportRow is renumbered to match. --reason no longer accepts unspecified, which no capture records; all selects blocked, runnable and preempted. AGENTS.md now says the compatibility rule applies from 1.0.0 on, and that until then formats and options change outright.
The digest hid its tables behind prose, printed 300-character commands inline, carried a table of unreadable stack strings, and called off-CPU time "busy", which readers take for CPU time. It also did not say when or on what the capture was recorded. Busy and idle become blocked and waiting everywhere: the Markdown of both digest layouts and of top, the analysis messages, --waiting and --waiting-from, and preset:jvm-waiting; the old spellings are unknown. With runnable or preempted intervals selected the blocked slice is "Blocked or in the run queue". The digest opens with when the capture was recorded, the window analysed, the process and the system, then one headline line, then the tables with one sentence each. Where the time went gains shares of the blocked time and of the selection. Definitions and caveats move to About this digest, which starts with the terms, and every command is a bash block wrapped one option per line (a shared Top.shellBlock, also used by top --format md). The heaviest-stacks sections are removed; the flame graph's stacks command stays under How to reproduce. The JFR pass keeps a snapshot of the recording in the report: the chunk bounds, async-profiler's version, and the first jdk.JVMInformation, jdk.OSInformation, jdk.CPUInformation and jdk.ContainerConfiguration events whole, as protobuf Structs, with the initial system properties and environment variables. The command line, system properties and environment can hold secrets: the report keeps them, and the digest shows them, only with --process-details true.
A root row's heaviest transformed line is hundreds of characters long, which made the root table unreadable in the digest and in top --format md. The flame graph shows what is under a root better; the column, TopRow.heaviest_stack and the CSV heaviest_stack column are removed.
A consumer that adds its own patterns to an option with a default preset had to name the preset to keep it, tying it to today's preset names: a preset split or added for the same purpose would be missed silently. Each preset's header now says which options it is meant for (# options: hide), and preset:* in a -from option stands for every such preset, in --list-presets order, so --hide-from 'preset:*' --hide-from FILE keeps jonoffcpu's patterns and adds the consumer's. An option's default is exactly its preset:*. An option without presets (--include-from, --root-at-from, --leaf-at-from, --app-from) refuses preset:* with exit status 64 rather than expanding to nothing, as an unknown preset now is too. Summaries and the digest's notes record the presets it expanded to; the digest's reproduce commands repeat 'preset:*' as given, quoted. --list-presets prints each preset's options.
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The digest (
jonoffcpu-summary.md/.json) is the first thing a reader opens, but it didn't say where to look. On an Apache Pulsar broker capture:AbstractQueuedSynchronizer.acquireSharedInterruptibly, a C2 monitor) rather than the code that waited.ReservedThreadExecutorwaiting for work, ZGC's driver), which pushed the 0.3 s of application waits to the tail.This PR makes three changes, one commit each (plus a follow-up that removes the root table's heaviest-stack column, which made that table unreadable):
1. Root the digest at the application (
--app)--root-at-unmatched hide(stacks,top): stacks without a--root-atmatch are left out, and reported asrootAtUnmatchedHiddenand by pool. It is the one transform that removes weight, and it's always accounted for.preset:jvm-dispatch: lambda bridges and JDK/Guava executor adapters, for--hide, so that--root-atlands on the work instead of the executor that ran it.top --by root: the flame graph's root boxes, where threads entered the application.top --by app-method: application methods ranked across all stacks, inclusive. Methods always found in the same stacks are grouped into one call-chain row, with self time.--by boundaryshares are of the blocked time that has an application frame.correlateandsummarizetake--appand--hide/--hide-from(defaultpreset:jvm-dispatch). With--app, every digest table comes from the transforms of the application-rooted flame graph.2. Drop backward compatibility until 1.0.0
AGENTS.md now says the compatibility rule applies from the 1.0.0 release on. Removed:
reservedproto declarations--dumpalias--reason unspecifiedexportcolumns now follow one logical order in CSV and JSON Lines.3. A readable digest
--waiting/--waiting-fromandpreset:jvm-waiting. The old spellings are unknown options.bashblock, wrapped one option per line past 100 characters.top --format mduses the same helper.stackscommand stays under How to reproduce.summarizeshows it too. It comes from the JFR passcorrelatealready makes:jdk.ActiveRecordingjdk.JVMInformation,jdk.OSInformation,jdk.CPUInformationandjdk.ContainerConfiguration, each kept whole as a protobufStruct--process-details true|false(defaultfalse) oncorrelateandsummarize. The command line, system properties and environment variables can hold secrets. Without the flag they're removed from both the report and the digest.On the key-shared broker fixture, the first table is led by
MessageDeduplication.isDuplicateNormalon a C2 monitor (0.087 s, 29.1 % of the time with an application frame). Where the time went reads:4.
preset:*selects every bundled preset meant for an option# options:headers. Each preset's header names the options it is meant for, such as# options: hide.preset:*in a-fromoption stands for every such preset, in--list-presetsorder.--hide-from 'preset:*' --hide-from FILEkeeps jonoffcpu's patterns and adds the consumer's. It also picks up a preset added later for the same purpose.preset:*.--include-from,--root-at-from,--leaf-at-from,--app-from) refusepreset:*with exit status 64. An unknown preset is now a usage error (64) too.'preset:*', quoted.--list-presetsprints each preset's options.Compatibility
None is kept: this is before 1.0.0, and AGENTS.md now says so.
--idle/--idle-from→--waiting/--waiting-frompreset:jvm-idle→preset:jvm-waiting--dump→dump--reason unspecifiedis removedExportRowis renumbered, and the export columns are reordered--waiting-frominstead of--idle-fromtocorrelate, and can pass--hide-from 'preset:*'instead of namingpreset:jvm-dispatch.Verification
./gradlew spotlessCheck :jonoffcpu-agent:check :jonoffcpu-correlator:check :jonoffcpu-jfr-converter:check :jonoffcpu-capture-codec:check :jonoffcpu-correlator:readmeTestpasses. This includes the agent's privileged end-to-end integration tests (none skipped).--process-detailsgatinglanguage-bashblock per command, no long inline code, and every in-page link resolves--process-details true-PjonoffcpuFixtures=…, key-shared fixture):maintoo, so they aren't run here.