Fix blocking / non-blocking stdio woes

Currently process.stdin / stdout / stderr is blocking, except when it is a pipe on windows. Weird and surprising. Very unpractical in cases where stdio is used as an IPC mechanism between node processes.

Discussion:http://logs.nodejs.org/libuv/2012-06-29#00:40:38.256

Preliminary results: have process.stdout.setBlocking(true|false) or process.stdout.writeSync.

Feel free to post thoughts here.

I've experienced a similar issue with stdio in Windows, and have somewhat worked around it in the meantime with this horrible, horrible hack.

While my mini exit lib works well in place of process.exit in some cases, it completely fails to help in this case:

var exitHack = require('./exit').exit;

// If there are pending stdout / stderr writes when "condition" is met...
if (condition) {
  exitHack();
}

// ...this code will still do something. Whoops!
doSomething();

I haven't seen this problem in OS X or Linux. It would be great if Node.js behaved the same cross-OS.

Any update? Will this land in 0.10?

This issue has been stagnant for months. In order for this to happen, it will need a champion.

Are you that champion? Would you like to see stdio have a more consistently blocking/nonblocking interface, which works the same on Windows and Unix?

Heed the call. Build the bits that need to go into libuv. Sketch out an API in node for it.

It seems that it won't make the 0.10 cut-off, I'm afraid. Maybe next pass.

I have started to look into this issue. Here is what I have found so far.
If stdout is not piped, then the output goes through libuv tty code which is blocking as expected.
If stdout is piped, the output goes through libuv pipe code.
The pipe code checks to see if the pipe supports overlapped IO.
In the case of piped stdout it does not - NtQueryInformationFile returns a mode of FILE_SYNCHRONOUS_IO_NONALERT, which means:
All operations on the file are performed synchronously.
Wait requests in the system that must synchronize I/O queuing and completion are not subject to alerts
In this case the pipe code uses the thread pool to perform all IO.
So when the process exits, the write requests are queued up for the thread pool to execute, but they are never executed.

I made a quick hack to simply let the Write operations execute synchronously when the pipe is in this mode.
In the tests that I have, this fixes the problem of data being lost when the process exits.

This approach means that stdout would have blocking IO behavior even when piped, without requiring any new API.

Other pipes that have mode FILE_SYNCHRONOUS_IO_NONALERT or FILE_SYNCHRONOUS_IO_ALERT would also end up being blocking.
These modes may be set on a pipe or file when creating it, but libuv never sets it.

Does this seem like an acceptable way of fixing this issue?

Excellent! Any chance of getting this into node 0.10.x? This fix will allow me to get rid of ugly workarounds I need to use currently to not loose process output.

@piscisaureus

@piscisaureus and/or @sblom: request for comment, please.

Second pull request fixes commit message problems.

@HenryRawas

@HenryRawas works with me. I'll look at thiS.

Updated the proposed fix. Added API uv_pipe_setblocking() to libuv and changed windows pipe to use synchronous writes if this is set. Updated node code to call uv_pipe_setblocking() for stdio over pipe.

@HenryRawas

@HenryRawas is it realistic to get this fix in for 0.10.x?

I am still trying to get the change accepted. I don’t know where it will land.

From: Benjamin Pasero [mailto:[email protected]]
Sent: Friday, June 14, 2013 2:07 AM
To: joyent/node
Cc: Henry Rawas
Subject: Re: [node] Fix blocking / non-blocking stdio woes (#3584)

@HenryRawashttps://github.com/HenryRawas is it realistic to get this fix in for 0.10.x?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/3584#issuecomment-19446612.

Just wanted to 👍 this issue. Was just bit by it in Windows 8 with Node v0.10.12. I'm still looking for a workaround atm.

@jdalton

@jdalton Did you try the pipe to file workaround?

@dcherman

@dcherman No I haven't. Did I miss it already covered?

Here is what I'm trying to do. I noticed that the stdout output was truncated:

var foo = cp.spawn(fooPath, fooOptions);

var output = '';
foo.stdout.on('data', function(data) {
  output += data;
});

foo.on('exit', function() {
  fs.writeFileSync(outputPath, output, 'utf8');
});

No matter what, don't do string concat like that. Use Buffer instances and .concat them.

Workaround for nodejs/node-v0.x-archive#3584.

Avoid node.js 0.10.x exit code bug for MS Windows

cli: replace calls to process.exit with exit as Windows workaround for …

Use exit on Windows to ensure stdio is flushed

Flush buffer on `process.exit` in Windows

Use exit module instead of process.exit for Windows workaround

bug: Fix for Windows' drain error on process.exit (Issue 3584)

Add error-handling for rj-build gulp task.

Hi, is this considered resolved? Is the use of https://github.com/cowboy/node-exit still recommended for some versions of node?

Yes, it is resolved. It was fixed with 20176a9.
You shouldn't need the workaround anymore.

Great, thanks!

Issue still exists for me in 0.12.4, Windows 8.1:

var child = require('child_process').spawn('php',
['-S', 'localhost:8007']);
child.stdout.on('data', function(data) {
console.log("child: "+data.toString());
});

Not sure if this is a bug, but if the process is infinite - then the child.stdout's callback is never triggered.

I wrapped php process in another child script, now it works fine.

Navigation Menu

Repository navigation

Fix blocking / non-blocking stdio woes #3584

33 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Fix blocking / non-blocking stdio woes #3584

Description

Activity

cowboy commented on Aug 26, 2012

bpasero commented on Jan 17, 2013

isaacs commented on Mar 6, 2013

HenryRawas commented on May 24, 2013

bpasero commented on May 29, 2013

bnoordhuis commented on May 29, 2013

HenryRawas commented on May 29, 2013

sblom commented on May 30, 2013

HenryRawas commented on Jun 10, 2013

bpasero commented on Jun 14, 2013

HenryRawas commented on Jun 14, 2013

jdalton commented on Jun 27, 2013

dcherman commented on Jun 27, 2013

jdalton commented on Jun 27, 2013

cowboy commented on Jun 27, 2013

33 remaining items

pgilad commented on May 26, 2015

orangemocha commented on May 26, 2015

pgilad commented on May 26, 2015

gigapromoters commented on Jun 1, 2015

gigapromoters commented on Jun 1, 2015

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions