[fix][broker] Preserve compaction state on ledger close failure - #26665

Motivation

StrategicTwoPhaseCompactor acknowledges the compaction subscription with the new ledger ID before closing that ledger. If closing fails after the broker has processed the acknowledgment, the failure cleanup deletes a ledger that the subscription already references.

This follows up on #26576 by fixing the publication order.

Modifications

• Close the compacted ledger before acknowledging the compaction subscription, matching AbstractTwoPhaseCompactor.
• If closing fails, skip the acknowledgment, attempt to delete the new ledger, and fail the compaction. The previously published ledger and cursor position remain unchanged.
• Add a regression test to the existing CompactionConcurrencyTest. It uses a real broker and BookKeeper, persists conflicting ledger metadata to trigger a real close failure, and verifies preservation of the previous compaction state, cleanup of the failed ledger, and successful retry.
• Use AssertJ future assertions for bounded waits and failure diagnostics, and define the compaction strategy locally in the test.

Verifying this change

• Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

• Added integration tests for end-to-end deployment with large payloads (10MB)
• Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• The metrics
• Anything that affects deployment