Skip to content

[improve][broker] Add configuration to enable shadow topics - #26739

Merged
lhotari merged 1 commit into
apache:masterfrom
lhotari:lh-improve-enable-shadow-topics
Sep 29, 2026
Merged

lhotari merged 1 commit into
apache:masterfrom
lhotari:lh-improve-enable-shadow-topics

Conversation

@lhotari

@lhotari lhotari commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

Motivation

Brokers already provide enablePersistentTopics and enableNonPersistentTopics to control which topic types they support. Add an equivalent setting for shadow topics so operators can explicitly choose whether to enable the feature.

Modifications

  • Add enableShadowTopics=false to the broker configuration and the broker/standalone configuration files. Changing the setting requires a broker restart.
  • Apply the setting to shadow topic creation, shadow-source property updates, topic loading, and replication. Policy inspection and removal remain available when disabled.
  • Use the existing tenant and superuser access rules when validating shadow-topic configuration, with asynchronous checks.
  • Update existing tests to enable the feature explicitly and cover both enabled and disabled behavior for partitioned and non-partitioned topics.

Deployments using shadow topics must set enableShadowTopics=true on their brokers to retain that behavior.

Verifying this change

  • ./gradlew quickCheck and ./gradlew sanityCheck passed.
  • 26 focused tests passed, covering topic configuration, loading, replication, and access rules.
  • Configuration-loader and service-configuration test suites passed.
  • Make sure that the change passes the CI checks.

Does this pull request potentially affect one of the following parts:

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations — shadow topics require explicit enablement.
  • The threading model
  • The binary protocol
  • The REST endpoints — shadow-topic configuration follows the feature setting and access rules.
  • The admin CLI options
  • The metrics
  • Anything that affects deployment — existing shadow-topic deployments must enable the feature and restart brokers.

Add enableShadowTopics alongside the existing topic-type controls, with
shadow topics disabled by default. Apply the setting to topic creation,
loading and replication, and validate configuration asynchronously using
the existing tenant and superuser access rules.

Document the setting in broker and standalone configurations and cover
both disabled and explicitly enabled behavior in tests.

Assisted-by: Codex
@lhotari
lhotari merged commit 9f827de into apache:master Sep 29, 2026
45 checks passed
ascentstream-bot pushed a commit to ascentstream/pulsar that referenced this pull request Oct 1, 2026
ascentstream-bot pushed a commit to ascentstream/pulsar that referenced this pull request Oct 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants