[improve][broker] Disable subscription-thread dispatch by default - #26578

Motivation

As discussed in #26576 (comment), enabling dispatcherDispatchMessagesInSubscriptionThread by default slows dispatching and increases resource consumption. Queuing entries for dispatch on a subscription thread prolongs their retention and can contribute to memory pressure and OOMs. The discussion also relates this to #24926.

The additional thread handoff also increases CPU usage, context switches, and contention on mutexes.

Modifications

Change the default of dispatcherDispatchMessagesInSubscriptionThread from true to false in both ServiceConfiguration and conf/broker.conf. This uses the existing dispatch path without the additional subscription-thread handoff. Operators can explicitly set the option to true to retain the previous behavior, including offloading expensive broker-side filters.

Verifying this change

The existing ServiceConfigurationTest.testConfigFileDefaults checks that the Java defaults and conf/broker.conf agree.

Passed locally:

• ./gradlew quickCheck (including Spotless and main/test Checkstyle checks)
• ./gradlew :pulsar-broker:test --tests org.apache.pulsar.common.naming.ServiceConfigurationTest.testConfigFileDefaults -PtestRetryCount=0

No new performance measurements were collected for this default-only change.

Does this pull request potentially affect one of the following parts:

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• The metrics
• Anything that affects deployment

Brokers that do not explicitly configure this option will use false. Existing configurations explicitly setting it to true retain their behavior.