Skip to content

[fix][broker] Bound classic Key_Shared dispatcher replay queue look-ahead - #26677

Merged
poorbarcode merged 1 commit into
apache:masterfrom
poorbarcode:pause_if_rediliver_queue_full
Sep 22, 2026
Merged

poorbarcode merged 1 commit into
apache:masterfrom
poorbarcode:pause_if_rediliver_queue_full

Conversation

@poorbarcode

Copy link
Copy Markdown
Contributor

Motivation

In the classic Key_Shared dispatcher, messages are assigned to consumers according to their sticky-key hash. If a consumer becomes slow and exhausts its available permits, messages mapped to that consumer cannot be dispatched. To keep other consumers making progress, the dispatcher continues normal reads and places these undispatchable entries into the in-memory replay queue.
This can cause broker OOM when all of the following conditions hold:

  • There were 12 consumers
  • consumer#xzqg9 was slow; other consumers have consumed all messages that they can.
  • The dispatcher keeps reading entries and pushing messages(which should be consumed by consumer#xzqg9) into the redeliver queue since consumer#xzqg9's permits is 0.
  • Eventually, the queue is too large, causing broker OOM.

Modifications

Let classic Key_Shared dispatcher support the configuration keySharedLookAheadMsgInReplayThresholdPerSubscription

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

@poorbarcode
poorbarcode force-pushed the pause_if_rediliver_queue_full branch from 417677b to e2f55c7 Compare September 22, 2026 08:53

@Technoboy- Technoboy- left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@poorbarcode
poorbarcode merged commit 3d5fbc3 into apache:master Sep 22, 2026
81 of 83 checks passed
poorbarcode added a commit that referenced this pull request Sep 22, 2026
poorbarcode added a commit that referenced this pull request Sep 22, 2026
@lhotari

lhotari commented Sep 22, 2026

Copy link
Copy Markdown
Member

@poorbarcode I'm just wondering why wouldn't users just use the current dispatcher instead of the legacy one. The current PIP-379 has been available for almost 2 years and it has been battle tested by the majority of all Pulsar users in the community. Most users don't disable PIP-379. After 4.0.x there has been a very critical fixes for PIP-379 implementation which have been quickly fixed. At the same time, the legacy dispatcher has a lot of known issues that remain unfixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants