Skip to content

[fix][broker] Avoid blocking the bundle-throughput lookup on per-bundle metadata reads - #26054

Merged
lhotari merged 1 commit into
apache:masterfrom
merlimat:mmerli/fix-bundle-data-or-default-blocking-io
Jun 18, 2026
Merged

lhotari merged 1 commit into
apache:masterfrom
merlimat:mmerli/fix-bundle-data-or-default-blocking-io

Conversation

@merlimat

Copy link
Copy Markdown
Contributor

Motivation

ModularLoadManagerImpl.getBundleDataOrDefault(String) reads bundle data (with a resource-quota fallback) from the metadata store, blocking on two .join() calls behind a plain BundleData return. Its clean signature hid that it blocks.

NamespaceBundleFactory.getBundleWithHighestThroughputAsync() called it once per bundle inside a getBundlesAsync(...).thenApply(...) continuation — so the continuation thread (which completes getBundlesAsync, a metadata-cache callback) blocked on a metadata read for every bundle in the namespace. This is reachable from the admin path (NamespacesBase.findHotBundleAsync).

Modifications

  • Add ModularLoadManager.getBundleDataOrDefaultAsync(String), composing the two metadata reads asynchronously and returning the default BundleData on miss/error (preserving the exact logic of the blocking method). To stay backward compatible for custom ModularLoadManager implementations, it is a default method delegating to the existing sync method; ModularLoadManagerImpl overrides it with the real async implementation.
  • Deprecate the blocking getBundleDataOrDefault(String); it now delegates to getBundleDataOrDefaultAsync(...).join(). All internal callers (the load-data update loop, preallocateBundle, selectBroker) and the tests now use the async variant, so the deprecated method has no remaining callers except the interface's compat bridge.
  • NamespaceBundleFactory.getBundleWithHighestThroughputAsync() now composes getBundleDataOrDefaultAsync() for all bundles (thenCompose + waitForAll) and selects the max-throughput bundle off the already-complete futures — no blocking on the continuation thread.

Verifying this change

This change is covered by existing tests, which pass: ModularLoadManagerImplTest.testBundleDataDefaultValue and NamespaceServiceTest.testSplitBundleWithHighestThroughput.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

…le metadata reads

ModularLoadManagerImpl.getBundleDataOrDefault(String) reads bundle data (with a
resource-quota fallback) from the metadata store, blocking on two .join() calls
behind a plain BundleData return. NamespaceBundleFactory
.getBundleWithHighestThroughputAsync() called it once per bundle inside a
getBundlesAsync(...).thenApply(...) continuation, so the continuation thread
(which completes getBundlesAsync, a metadata-cache callback) blocked on a
metadata read for every bundle in the namespace.

- Add ModularLoadManager.getBundleDataOrDefaultAsync(String), composing the two
  metadata reads asynchronously and returning the default BundleData on
  miss/error (preserving the exact logic). To stay backward compatible for
  custom ModularLoadManager implementations it is a default method delegating to
  the existing sync method; ModularLoadManagerImpl overrides it with the real
  async implementation.
- Deprecate the blocking getBundleDataOrDefault(String); it now delegates to
  getBundleDataOrDefaultAsync(...).join(). All internal callers (the load-data
  update loop, preallocateBundle, selectBroker) and the tests use the async
  variant.
- NamespaceBundleFactory.getBundleWithHighestThroughputAsync() composes
  getBundleDataOrDefaultAsync() for all bundles (thenCompose + waitForAll) and
  selects the max-throughput bundle off the already-complete futures, with no
  blocking on the continuation thread.
@lhotari
lhotari merged commit 561f373 into apache:master Jun 18, 2026
44 checks passed
@lhotari lhotari added this to the 5.0.0-M2 milestone Jun 18, 2026
lhotari pushed a commit that referenced this pull request Jun 22, 2026
lhotari pushed a commit that referenced this pull request Jun 22, 2026
sandeep-ctds pushed a commit to datastax/pulsar that referenced this pull request Jul 31, 2026
nodece pushed a commit to ascentstream/pulsar that referenced this pull request Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants