Skip to content

[improve][broker] Skip unused replicated subscription timestamp updates - #26655

Merged
lhotari merged 2 commits into
masterfrom
lh-improve-repl-timestamp
Sep 19, 2026
Merged

lhotari merged 2 commits into
masterfrom
lh-improve-repl-timestamp

Conversation

@lhotari

@lhotari lhotari commented Sep 19, 2026 •

Copy link
Copy Markdown
Member

Motivation

Every successful normal publish advances the transaction buffer's maximum readable position. That path currently reads the wall clock and writes lastMaxReadPositionMovedForwardTimestamp, even when replicated subscriptions are not enabled for the topic.

ReplicatedSubscriptionsController is the only production consumer of this timestamp. On ordinary topics, the clock read and volatile write therefore add work to every publish without affecting broker behavior.

Modifications

  • Skip the timestamp update when the topic has no replicated-subscription controller.
  • Seed the timestamp for topics with stored entries immediately before controller construction and again after publishing the controller reference. Empty topics remain idle, while the initial snapshot covers earlier messages and publishes racing with controller activation.
  • Preserve marker handling, maximum-readable-position updates, transaction recovery, and snapshot scheduling behavior.
  • Add lifecycle, transactional, recovery, and callback microbenchmark coverage, including empty-topic activation and a publish during controller construction.

Performance

An actual-callback JMH benchmark measured the controller-absent path:

Threads Before After Change
1 19.360 ns/op 1.497 ns/op -92.3%
4 20.690 ns/op 1.604 ns/op -92.2%

Allocation remains effectively zero. When a replicated-subscription controller is active, the added presence check measured 20.090 ns/op versus 19.360 ns/op for the unconditional baseline (+0.730 ns/op).

A five-million-message broker scenario with 500 producers and five Key_Shared subscriptions produced two positive matched comparisons:

Pair Before After Change
1 61,423 msg/s 75,164 msg/s +22.4%
2 58,497 msg/s 62,952 msg/s +7.6%

The spread indicates substantial whole-host variance, so these runs establish repeatability and absence of a throughput regression rather than a precise effect size. Every run delivered all messages to every subscription with zero duplicates, ordering violations, or invalid messages.

Verifying this change

This change added tests and can be verified as follows:

  • Reproduced ReplicatedSubscriptionTest.testReplicationSnapshotStopWhenNoTraffic and the empty-topic lifecycle regression on the original code.
  • After the fix, all 15 tests in ReplicatedSubscriptionTest and ReplicatedSubscriptionsControllerTest passed with -PtestRetryCount=0.
  • Focused persistent-topic and replicated-subscription controller lifecycle tests
  • Transaction commit and transaction-buffer recovery tests
  • ./gradlew :microbench:shadowJar quickCheck
  • JMH with two forks, three 1-second warmups, five 1-second measurements, and GC profiling

Does this pull request potentially affect one of the following parts:

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

@lhotari
lhotari added this pull request to stack #26656 September 19, 2026 12:50
@lhotari
lhotari force-pushed the lh-improve-repl-timestamp branch from d4804f6 to 47eb231 Compare September 19, 2026 18:10
@lhotari
lhotari force-pushed the lh-improve-repl-timestamp branch from 47eb231 to 2beeeb9 Compare September 19, 2026 19:41
Base automatically changed from lh-improve-auto-split-select to master September 19, 2026 21:16
Only seed snapshot activity when the managed ledger contains entries. Preserve the post-construction check for publishes racing with controller activation, and cover both paths in the lifecycle test.

Assisted-by: Codex
@lhotari
lhotari force-pushed the lh-improve-repl-timestamp branch from 90b1c58 to 04ed5eb Compare September 19, 2026 21:16
@lhotari
lhotari merged commit 3947c95 into master Sep 19, 2026
43 checks passed
@lhotari
lhotari deleted the lh-improve-repl-timestamp branch September 19, 2026 22:41
@lhotari lhotari added this to the 5.0.0 milestone Oct 1, 2026
ascentstream-bot pushed a commit to ascentstream/pulsar that referenced this pull request Oct 1, 2026
ascentstream-bot pushed a commit to ascentstream/pulsar that referenced this pull request Oct 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants