Repository navigation
[improve][broker] Optimize consumer selection for shared subscriptions - #26593
Merged
merlimat merged 3 commits intoSep 15, 2026
Merged
Conversation
Preserve selection and availability-check order while isolating list traversal for benchmarking. Keep the dispatcher cursor and lifecycle checks in place. Assisted-by: Codex
Measure ready, sparse, lower-priority, blocked and changing-permit consumers with persistent round-robin state. This revision is the unoptimized baseline. Assisted-by: Codex
Track MutableInt priority counts with synchronized membership updates, removing entries when their count reaches zero. Use availability-only round-robin traversal for one priority and reuse group boundaries for mixed priorities. Preserve the bulk-removal repair path for inconsistent dispatcher membership. Validate with JMH comparisons against the benchmark-only parent, including 50 consumers at one priority, selection-order and membership tests, dispatcher recovery tests, explicitly enabled priority tests, and quickCheck. Assisted-by: Codex
lhotari
requested review from
Technoboy-,
dao-jun,
david-streamlio,
merlimat and
nodece
September 15, 2026 13:49
merlimat
approved these changes
Sep 15, 2026
dao-jun
pushed a commit
to ascentstream/pulsar
that referenced
this pull request
Sep 20, 2026
apache#26593) (cherry picked from commit e7cbb33)
dao-jun
pushed a commit
to ascentstream/pulsar
that referenced
this pull request
Sep 20, 2026
apache#26593) (cherry picked from commit e7cbb33)
dao-jun
pushed a commit
to ascentstream/pulsar
that referenced
this pull request
Sep 20, 2026
apache#26593) (cherry picked from commit e7cbb33)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Shared subscriptions perform priority checks during consumer selection even when every consumer has the same priority. With mixed priorities, round-robin wraparound can also rescan higher-priority consumers to locate the current priority group.
Reducing this work lowers selection overhead while preserving priority ordering and round-robin behavior.
Modifications
ConsumerPrioritySelector, allowing isolated unit tests and JMH benchmarks.Int2ObjectMap<MutableInt>. Remove entries when their counts reach zero and skip priority checks when only one priority remains.Verifying this change
Added tests cover:
Local validation passed:
ConsumerPrioritySelectorTest./gradlew quickCheck./gradlew :microbench:shadowJarJMH comparison
Compared baseline
2d540fa0853with optimization9c88617c944using identical benchmark sources and settings across 26 parameter combinations.Environment: Apple M3 Max, macOS arm64, Corretto 25.0.4, JMH 1.37; two forks, three 500-ms warmup iterations, five 500-ms measurement iterations, one thread, and the GC profiler. Builds and tests did not overlap measurements.
Values below are mean nanoseconds per selection. Ranges cover priority offsets zero and three; speedups compare matching cases.
No GC occurred, and measured allocation was approximately zero. The 256-consumer mixed-priority
FLOWcase at offset zero had a wide confidence interval and remains inconclusive.These measurements isolate selection cost and exclude dispatcher locking, membership updates, and message processing. Linux x86_64 confirmation is needed before extrapolating to broker throughput.
Does this pull request potentially affect one of the following parts:
Selection and membership helpers explicitly synchronize on the existing dispatcher monitor so the consumer list and priority counts remain consistent. Existing dispatch call sites already hold that monitor.