[fix][broker] Apply subscription policies to namespace and topic subscription operations - #26767

Motivation

Namespace subscription policies (the subscription roles allowlist and the Prefix subscription auth mode) are applied by topic level operations that name a subscription, such as skip, reset cursor and expire. Several other admin operations that act on subscriptions did not apply them:

• The namespace and bundle level clearBacklog/{subscription} and unsubscribe/{subscription} operations only checked the namespace level consume permission, and the subscription name was not passed to the authorization provider.
• The namespace and bundle level clearBacklog without a subscription, and the topic level all_subscription/expireMessages, act on every subscription of every affected topic, including subscriptions that the caller could not act on one at a time. The namespace level clear backlog also cleared replicator and shadow replicator cursors for any role with consume permission.
• Topic level operations that name a subscription set the subscription on the request authentication data. Authentication data types that don't store a subscription, such as the anonymous role's, dropped it, so the subscription policies were not applied for those callers.
• The namespace level clear backlog for a subscription falls back from a subscription to a replicator cursor and then to a shadow replicator cursor with the same name, for any caller.

Modifications

Subscription operations on namespaces, bundles and topics now apply the namespace subscription policies consistently for roles that are not super users or tenant admins. Super users and tenant admins keep the existing behaviour.

• Operations that name a subscription:
  • Namespace and bundle clearBacklog/{subscription} and unsubscribe/{subscription} pass the subscription name to the authorization provider, the same way as topic level subscription operations. PulsarAuthorizationProvider applies the subscription roles and the Prefix auth mode to them.
  • The request authentication data is wrapped in AuthenticationDataSubscription for both namespace and topic level operations, so the subscription reaches the provider for every authentication data type, including the anonymous role's. For proxied requests, the original principal's own authentication data is wrapped separately. AuthorizationService has a new allowNamespaceOperationAsync overload that takes separate authentication data for the original principal, as the topic operation overload already does.
  • For namespace clearBacklog/{subscription}, other roles have the name resolved as a subscription only. Super users and tenant admins keep the fallback to replicator and shadow replicator cursors. The broker passes this restriction to the bundles with an internal, hidden subscriptionOnly query parameter, and a direct bundle request from other roles whose name resolves to a replicator or shadow replicator cursor is rejected.
• Operations on all subscriptions, for other roles:
  • Namespace and bundle clearBacklog without a subscription clear only the subscriptions that the caller may clear, using the same check as clearBacklog/{subscription}. Replicator and shadow replicator cursors are left alone. When no subscription policy restricts the caller, every subscription is still cleared.
  • Topic all_subscription/expireMessages expires only the subscriptions that the caller may expire, using the same check as expireMessages for one subscription, on topics, partitioned topics and single partitions.
  • The broker's internal admin client uses the broker's own identity, so for these callers the namespace clear backlog and the partitioned topic expire no longer fan out per bundle or per partition. The broker that receives the request lists the topics or partitions and their subscriptions, checks each subscription name for the caller, and sends one request per allowed subscription. The namespace clear backlog still redirects to the owning peer cluster for global namespaces.
• Super users and tenant admins are recognized with AuthorizationService#isSuperUserOrTenantAdmin. For proxied requests, both the proxy role and the original principal must be a super user or a tenant admin, and the original principal is checked with its own authentication data.
• Add NamespacesImpl#clearNamespaceBundleBacklogForSubscriptionAsync(namespace, bundle, subscription, subscriptionOnly), which the broker uses internally. It is not part of the public Namespaces interface.

Verifying this change

• Make sure that the change passes the CI checks.

This change added tests and can be verified as follows:

• NamespaceAuthZTest:
  • Namespace and bundle clear backlog and unsubscribe for a subscription under subscription roles and Prefix auth mode, for regular roles, tenant admins and super users, including namespace and topic level replication, shadow topics (with short shadow topic names) and subscriptions named after a cluster.
  • Namespace and bundle clear backlog without a subscription under subscription policies, without policies, with replicators, and for tenant admins.
  • Topic expire for all subscriptions on non-partitioned and partitioned topics and on a single partition.
• NamespaceSubscriptionAnonymousRoleTest: namespace and topic level subscription operations apply the subscription policies to the anonymous role.
• NamespaceSubscriptionProxyRoleTest: proxied requests resolving a replicator cursor name are checked for the original principal.
• MultiRolesTokenAuthorizationProviderTest: namespace clear backlog with a secondary tenant admin role, and namespace and all-subscription operations through a proxy whose token carries a tenant admin role.
• Updated NamespacesTest for the new bundle endpoint parameter.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• The metrics
• Anything that affects deployment

REST endpoints: the bundle level clearBacklog/{subscription} endpoint accepts a new hidden, internal subscriptionOnly query parameter. Namespace clear backlog (with or without a subscription), namespace unsubscribe and topic expire for all subscriptions now apply subscription policies for roles that are not super users or tenant admins, and these roles no longer clear replicator or shadow replicator cursors through the namespace operations. For these roles, the namespace clear backlog and the partitioned topic expire for all subscriptions send one internal request per allowed subscription instead of one per bundle or partition. Brokers that don't support the subscriptionOnly parameter ignore it, so the restriction on named namespace operations applies once all brokers are upgraded. Admin CLI: the namespace clear-backlog and unsubscribe commands and the topic expire-messages-all-subscriptions command are affected by the same authorization change. Their options are unchanged.