Repository navigation
fs: do not close FileHandle on stream destroy with autoClose: false - #66669
Open
SahinurDEV wants to merge 1 commit into
Open
SahinurDEV wants to merge 1 commit into
SahinurDEV wants to merge 1 commit into
Conversation
When a ReadStream or WriteStream is created from a FileHandle, the stream takes a reference on the handle. Destroying the stream (for example by breaking out of a `for await` loop) dropped that reference and then unconditionally called `filehandle.close()`, which also dropped the caller's reference and closed the handle, even when the stream was created with `autoClose: false`. Only release the stream's own reference when `autoClose` is false, so that the FileHandle stays usable and remains owned by the caller, as documented. Also remove the stream's 'close' listener from the FileHandle when the stream closes, so that creating many streams from a long-lived FileHandle does not leak listeners and streams. Fixes: nodejs#45721 Refs: nodejs#45909 Signed-off-by: Sahinur <[email protected]>
Contributor
|
Welcome to Node.js, and thank you for your first contribution! Before review, please take a moment to read:
Please make sure every commit is signed off. For a first pull request, GitHub Actions require collaborator approval and Jenkins CI must be started by a collaborator or triager, so an initial wait is normal. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes: #45721
Refs: #45909, #45917
Problem
Root cause
Each
FileHandlekeeps a reference count (kRefs), which starts at 1 for the owner. A stream created from the handle takes its own reference inimportFd(). When the stream is destroyed,_destroy()always callsclose(), which ends up inFileHandleOperations.close(). That function did two things:handle[kUnref](), which releases the stream's reference. This part is correct.handle.close(), which releases the owner's reference and closes the fd.Step 2 runs whatever
autoCloseis set to.autoCloseonly maps to the stream'sautoDestroy, so it controls whether the stream destroys itself on'end'/'error'. It has no effect on what destroying the stream does. Breaking out offor await,stream.destroy(),stream.close()andpipeline()teardown all destroy the stream, so all of them closed a handle the user said they own. The docs forfilehandle.createReadStream()andfilehandle.createWriteStream()say that withautoClose: false"the file descriptor won't be closed … It is the application's responsibility to close it". Newer FileHandle APIs already work that way (readableWebStream({ autoClose }),writer({ autoClose }),pull({ autoClose })).There was also a second leak.
importFd()added a'close'listener on the FileHandle for every stream and never removed it. A long-lived handle that serves many streams therefore kept all of them alive and eventually hitMaxListenersExceededWarning. The issue reporter mentions this case.Fix
In
FileHandleOperations.close()(shared by ReadStream and WriteStream):'close'listener from the handle;handle.close()only whenstream.autoCloseis true. Otherwise, call back on the next tick.Streams opened from a raw numeric fd or a path are unchanged. So is the default
autoClose: truebehaviour, and so is closing the FileHandle while a stream is still using it: the handle's'close'event still destroys the stream, and releasing the stream's reference still finishes the pending close.Why this differs from #45909
#45909 (ronag) had the same goal. It decided whether to close by checking
handle[kFd] !== -1after the unref ("someone else has a ref"). The owner always holds a ref, so that condition is also true withautoClose: true. The handle would then never be closed, which breaks the documented default and existing tests ("LGTM except the test failures"). This PR keys on the option the user actually passed (autoClose) instead. It also leaves out #45909's unrelated change that movedFileHandle's'close'emit after the native close, because that changes event timing for every FileHandle user. The listener cleanup from #45909 is kept. #45917 was closed by its author in favour of #45909.Tests
New
test/parallel/test-fs-stream-file-handle-autoclose-false.js:for await … breakon a ReadStream 15 times on one handle: the handle stays open, the'close'listener count stays flat, and noMaxListenersExceededWarningis emitted;destroy()on a ReadStream and a WriteStream withautoClose: falseleaves the handle usable;autoClose: falsestream holds a ref still closes both;autoClose: truestill closes the handle on destroy and removes the listener.Existing
test/parallel/test-fs-*stream*andtest-filehandle*tests pass. [fill in actual results]Semver
I consider this a bug fix (semver-patch): it makes the implementation match the documented behaviour of
autoClose: falsefor FileHandle streams. Code that relied ondestroy()closing a FileHandle it created withautoClose: falsewould now have to callfilehandle.close()itself, which the docs already require. Unclosed FileHandles are still closed on GC, with a warning. If reviewers think this should be semver-major, I'm fine with that label.AI-assistance disclosure: I used an AI coding assistant to help investigate the issue and draft the code, tests and this description. I reviewed, tested and understand all of the changes.