[improve][cli] Pick the v4 or V5 client from the topic domain in pulsar-perf and pulsar-client - #26693

Motivation

#26480 restored the v4 client in pulsar-perf and pulsar-client, but as separate commands:
produce-v4, consume-v4, read-v4 and transaction-v4. So anyone upgrading from 4.x has to rename
every script and benchmark invocation to get the behaviour they already had. The -v4 commands have
only shipped in the 5.0.0-M2 milestone, so it is still possible to fix this before 5.0.

The topic name already carries the information needed. Scalable topics always use the topic://
domain. Everything else — a name without a domain, persistent:// or non-persistent:// — is a v4
topic that the v4 client can address.

Modifications

The externally visible commands are merged back into produce, consume, read (both tools) and
transaction (pulsar-perf). The -v4 commands are removed.

• Client selection. The topic domain picks the client: topic:// uses the V5 client, and any
  other topic uses the v4 client. A new --client-api V4|V5 option overrides the choice, for example
  to drive a persistent:// topic with the V5 client as the V5 commands could before. The following
  are rejected as usage errors:

  • mixing topic:// and other topics in one invocation;
  • --client-api V4 with a topic:// topic;
  • --client-api V5 with a non-persistent:// topic;
  • segment:// topic names.

  Each run logs one line saying which client was chosen.

• Option groups in --help. Options that only one client supports are declared in picocli
  @ArgGroups with headings, so --help shows a Common options section, a v4 client options
  section and, where there are any, a V5 client options section. gen-doc and
  generate_documentation produce the same sections.

• Wrong-client options are rejected. An option from the other client's section, typed on the
  command line, is a picocli usage error. Previously the V5 commands warned about or silently
  ignored many v4-only options. The check follows the resolved client, so
  consume --client-api V5 -aq persistent://… is rejected, while
  consume --client-api V5 -sct Stream persistent://… is accepted. Values that come from the
  configuration file never trigger the check.
  -rs/--replicated is in the v4 group of consume (both tools) and pulsar-perf transaction, because
  [fix][broker] Do not enable replicated subscriptions on scalable topic segments #26679 removed replicated subscriptions from the V5 consumers.

• Shared helpers. The routing and the group check live in ClientApi and ClientApiOptionGroups
  in pulsar-cli-utils, which both tools use. EnumNameConverter accepts both enum spellings, so
  for example -am ExclusiveWithFencing (4.x) and -am EXCLUSIVE_WITH_FENCING (5.0-M2) both work.

• Parsing is decoupled from the implementations. Each command is a picocli class that only parses
  and validates. The benchmark or client work runs in per-client runners that take the parsed
  command:

  • pulsar-perf: PerformanceProducerV5 / PerformanceProducerV4, and the same pairs for the
    consumer, reader and transaction.
  • pulsar-client: ProduceV5 / ProduceV4, and so on.

  The pulsar-perf runners log under the command's class name, so the report lines
  (PerformanceProducer - Aggregated throughput stats, …) are the same for both clients and match
  4.x.

• pulsar-client specifics:

  • topic:// is rejected with a ws:// service URL, because the WebSocket proxy only serves
    persistent and non-persistent topics.
  • A non-file: --encryption-key-value is rejected with the V5 client, which only reads key files.
  • pulsar-shell's client commands pick all of this up unchanged.

• Harness and docs. PerfToolTest, the profiling harness and the docs now use the merged
  commands. The harness derives the client from the topic domain and passes the v4-only -o and
  --isolated-clients options only to the v4 client.

Verifying this change

• Make sure that the change passes the CI checks.

This change added tests and can be verified as follows:

• ClientApiTest (pulsar-cli-utils): routing for every topic kind and override, the rejected
  combinations, the wrong-group check, conf-file defaults, the help section layout and
  EnumNameConverter.
• PulsarPerfTestToolTest (pulsar-perf, parsed through PulsarPerfTestTool with a conf file):
  • every option of both former commands is still present;
  • --help section order and membership;
  • routing, rejection of other-client options, and the usage-error exit code;
  • value-based rules such as read -m lid:eid, and --scalable together with --partitions.
• CmdClientApiRoutingTest (pulsar-client) replaces CmdV4CommandsTest with the same kind of
  coverage.
• The existing broker-backed tests run on the merged commands, and V5-specific assertions use
  --client-api V5:
  • pulsar-perf: PerformanceV4CommandsTest, PerformanceProducerTest,
    PerformanceTransactionV4Test and PerformanceTransactionTest;
  • pulsar-client: PulsarClientToolTest and PulsarClientToolWsTest.
• Mutation check: turning the wrong-group check into a no-op makes tests fail in all three modules.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
  • pulsar-perf and pulsar-client: the -v4 commands are removed and --client-api is added.
  • A persistent:// or unprefixed topic now uses the v4 client by default.
  • An option for the client that is not in use is rejected instead of ignored.
• Anything that affects deployment