[improve][broker] Reduce cache retention after reads and apply retention settings at startup - #26627

Motivation

Recently accessed managed-ledger cache entries currently receive an extra TTL extension by default, even after their expected reads have completed. For consumers keeping up with a topic, this retains already-read entries longer and increases cache occupancy without necessarily serving additional reads.

Disable this recent-access extension by default to reduce that retention. Expected-read retention remains enabled, with up to five TTL extensions for entries with remaining expected reads.

A comparison with 500 producers on separate connections and 20 consumers on one Key_Shared subscription showed the following results. Both configurations used the same broker code, one persistent topic, random keys, unbatched 128-byte messages, unrestricted production with at most 40 outstanding sends per producer, and a 12-million-message receive target. Only the recent-access extension setting differed.

Steady throughput uses topic counters after 20 seconds of warmup, includes only intervals with all 500 producers connected, and ends before producers disconnect. Cache size is the cache's accounting of stored data, not total heap usage. Allocation reductions were not consistent. These short runs demonstrate the benefit for this workload; they do not establish the same improvement for multiple subscriptions or lagging consumers.

The broker also accepts two cache retention settings without passing them to the managed-ledger factory during initialization. Dynamic update listeners do not apply initial values, so startup configuration can silently leave the factory's defaults in effect. Correct startup propagation is necessary for the new broker default and for explicit operator settings to take effect.

Modifications

• Default managedLedgerCacheEvictionExtendTTLOfRecentlyAccessed to false in ServiceConfiguration, broker.conf, and standalone.conf, and update their configuration documentation.
• Pass managedLedgerCacheEvictionExtendTTLOfRecentlyAccessed and managedLedgerCacheEvictionExtendTTLOfEntriesWithRemainingExpectedReadsMaxTimes to ManagedLedgerFactoryConfig during initialization.
• Preserve the existing expected-read retention limit of five extensions and dynamic configuration updates. The standalone managed-ledger factory default is unchanged; the broker supplies its configured policy.

Deployments that benefit from retaining recently read entries, such as repeated reads across subscriptions or redelivery, can explicitly set managedLedgerCacheEvictionExtendTTLOfRecentlyAccessed=true to retain the previous policy. Existing configuration files that explicitly set true continue to do so.

Verifying this change

Parameterized tests capture the factory configuration during initialization and verify the new broker default, explicit enable/disable overrides, and expected-read extension limits. The startup propagation regression test fails without the factory assignments.

• ./gradlew :pulsar-broker:test --tests '*ManagedLedgerClientFactoryTest' -PtestRetryCount=0
• ./gradlew spotlessCheck checkstyleMain checkstyleTest

Does this pull request potentially affect one of the following parts:

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• The metrics
• Anything that affects deployment

Broker startup honors both retention settings, and recent-access TTL extension is disabled by default.