Skip to content

Make outgoing pending count getters non-blocking - #1632

Merged
scottf merged 2 commits into
mainfrom
non-blocking-pending-counts
Sep 28, 2026
Merged

scottf merged 2 commits into
mainfrom
non-blocking-pending-counts

Conversation

@scottf

@scottf scottf commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

Resolves #1631

Connection.outgoingPendingMessageCount() and outgoingPendingBytes() took closeSocketLock in NatsConnection and writerLock in NatsConnectionWriter to return two AtomicLong values. Neither lock is held by any code that changes those counters (push, accumulate, filter), so they added no consistency. What they did add: writerLock parked the caller behind an in-flight sendMessageBatch, bounded only by the socket write timeout (default 1 minute), and closeSocketLock parked the caller for the entire reconnect, since closeSocket holds it through reconnectImpl(). publishInternal takes neither lock, so an application that reads the getter once per publish was blocked for the whole reconnect while plain publishes would have kept buffering.

The closeSocketLock was added in #1416 (2.23.0) because the writer was replaced on every reconnect at that time. That swap was removed in 2.25.2 (62f6325); the writer is now created once in the constructor and reused, so the lock has guarded nothing since. The writerLock and the null checks in the writer never guarded anything: the queue field has always been private final. The analysis is in the issue.

Changes

  • NatsConnection: both getters return the writer's value directly, no closeSocketLock.
  • NatsConnectionWriter: both getters return normalOutgoing.length() / sizeInBytes() directly, no writerLock, no null checks. This restores the Expose connection outgoingPendingMessageCount and outgoingPendingBytes #1412 bodies.
  • NatsConnectionImplTests: new testOutgoingPendingGettersDoNotBlockOnCloseSocketLock holds closeSocketLock on the test thread and reads both getters from another thread under a timeout. Trimmed the now-stale comment in testOutgoingPendingCountCoverage.

Verification

  • NatsConnectionImplTests: 5 tests, 0 failures, 0 errors.
  • Negative control: the new test run against unmodified main in a separate worktree fails on all 5 retry attempts with java.util.concurrent.TimeoutException.

🤖 Generated with Claude Code

Remove closeSocketLock and writerLock from outgoingPendingMessageCount and outgoingPendingBytes. The counters are atomics, and neither lock serializes with the code that changes them, so the locks added nothing except parking callers behind an in-flight socket write and, on the closeSocket path, behind the entire reconnect. The closeSocketLock guarded a writer swap on reconnect that was removed in 2.25.2. Adds a regression test that reads both getters while another thread holds closeSocketLock.

Resolves #1631
@synadia-claude-reviewer

synadia-claude-reviewer Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Claude finished @scottf's task in 43s —— View job


Review complete

  • Fetch PR diff and description
  • Examine changed files (NatsConnection, NatsConnectionWriter, tests)
  • Check lock/thread-safety history for regressions
  • Review new test correctness
  • Post final review summary

Verified the core claim. WriterMessageQueue (MessageQueueBase) backs length/sizeInBytes with AtomicLong, and push/accumulate/filter/clear mutate them under editLock, not writerLock. writerLock only guards sendMessageBatch (the in-flight socket write loop) and flushBuffer, so it never provided any consistency for these getters — only added blocking. Same story for closeSocketLock: writer (NatsConnection.java:191) and normalOutgoing (NatsConnectionWriter.java:77) are each assigned exactly once, in their constructors, and never reassigned, so the removed null checks were dead code and the lock added no synchronization value, just reconnect-duration blocking.

Correctness/thread-safety: No issues. The getters now do a plain volatile-backed AtomicLong.get(), which is safe for concurrent reads against the atomic writers elsewhere. No new race is introduced — this removes locks that were never actually providing exclusion relevant to the read.

Test: testOutgoingPendingGettersDoNotBlockOnCloseSocketLock correctly reproduces the bug by holding closeSocketLock on the test thread and asserting the getters return promptly from another thread — this is a real regression test, not just a smoke test, and matches the PR's negative-control claim (fails on unmodified main).

API compatibility: Return values change from -1 (never actually reachable given final, once-assigned fields) to the real counts in that same dead branch — not an observable behavior change for any real caller.

Minor, non-blocking observation: the comment on NatsConnection.java:2672 says "the writer is created once in the constructor so the reference cannot change," which is accurate, but note writer is protected (not final) — harmless today since nothing reassigns it, just flagging that the safety relies on that invariant holding, same as before this PR.

No blocking issues found.

@scottf

scottf commented Sep 25, 2026

Copy link
Copy Markdown
Contributor Author

On the protected, non-final writer observation: the field was made protected in 62f6325 for custom Connection implementations, so making it final is a separate decision about that extension point and is not part of this PR. Nothing in src/main or src/test reassigns it after the constructor, including the three test subclasses of NatsConnection. The comment at NatsConnection.java:2672 overstated this as "the reference cannot change". It now states the actual invariant: assigned once in the constructor, nothing reassigns it, protected rather than final for custom implementations. That wording change goes up with the next push.

State the assign-once invariant on writer precisely: the field is protected rather than final for custom Connection implementations, and nothing reassigns it after the constructor. Raised by the PR review.

@Jarema Jarema left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@scottf
scottf merged commit bb11198 into main Sep 28, 2026
5 checks passed
@scottf
scottf deleted the non-blocking-pending-counts branch September 28, 2026 17:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

outgoingPendingMessageCount / outgoingPendingBytes block on closeSocketLock and writerLock

2 participants