Skip to content

[fix][ml] Fix batch ACK index loss when recovering cursor from MetadataStore - #26474

Merged
merlimat merged 1 commit into
apache:masterfrom
void-ptr974:fix/metadata-store-batch-ack-recovery
Sep 9, 2026
Merged

merlimat merged 1 commit into
apache:masterfrom
void-ptr974:fix/metadata-store-batch-ack-recovery

Conversation

@void-ptr974

Copy link
Copy Markdown
Contributor

Motivation

When a cursor is cleanly closed using the MetadataStore persistence path, partial batch acknowledgment indexes are written to ManagedCursorInfo. The cursorsLedgerId == -1 recovery path restored the mark-delete position, individual deleted-message ranges, and properties, but skipped batchedEntryDeletionIndexInfos. After topic unload/reload or another cursor reconstruction, already acknowledged messages within a batch could therefore be redelivered.

Modifications

  • Restore persisted batch deletion indexes from ManagedCursorInfo in the MetadataStore recovery path.
  • Keep recovery gated by batch-index acknowledgment being enabled and by metadata containing batch records, preserving compatibility with old metadata and disabled configurations.
  • Add ManagedCursor recovery coverage for MetadataStore and BookKeeper paths, persistence limits, cross-word bitmaps, missing metadata, disabled batch-index ACK, and combined cursor state.
  • Add a Shared consumer end-to-end test using real producer batching, topic unload, a new client, and an explicitly rebuilt cursor.

Verifying this change

  • ./gradlew :managed-ledger:test --tests '*ManagedCursorBatchAckRecoveryTest' -PtestRetryCount=0 -PtestFailFast=false
  • ./gradlew :pulsar-broker:test --tests '*ManagedCursorBatchAckMetadataStoreRecoveryTest' -PtestRetryCount=0 -PtestFailFast=false
  • ./gradlew quickCheck

@merlimat
merlimat merged commit 5a47023 into apache:master Sep 9, 2026
44 checks passed
@lhotari lhotari added this to the 5.0.0-M2 milestone Sep 12, 2026
lhotari pushed a commit that referenced this pull request Sep 23, 2026
lhotari pushed a commit that referenced this pull request Sep 23, 2026
Radiancebobo pushed a commit to Radiancebobo/pulsar that referenced this pull request Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants