- Version: v10.0.0-nightly20180318d74184c2fa
- Platform: Ubuntu Linux
- Subsystem: fs/promises
When dealing with big files, the beginning of the file is cut. I was not able to reproduce this with small files.
$ wget -O emojis.json https://api.github.com/emojis
$ node -p "require('fs').readFile('./emojis.json',(err,buffer)=>console.log(buffer.toString('utf8', 0, 100)))"
undefined
{
"+1": "https://assets-cdn.github.com/images/icons/emoji/unicode/1f44d.png?v8",
"-1": "https://
$ node -p "require('fs/promises').readFile('./emojis.json').then(buffer=>console.log(buffer.toString('utf8', 0, 100)))"
Promise { <pending> }
(node:13241) ExperimentalWarning: The fs/promises API is experimental
arge_square": "https://assets-cdn.github.com/images/icons/emoji/unicode/2b1c.png?v8",
"white_mediu
I made a script to compare different approches, and fs/promises seems to be the only one impacted by thus bug.
const { promisify } = require("util");
const fs = require("fs");
const fsPromises = require("fs/promises");
const promisifiedRead = promisify(fs.readFile);
const JSON_FILE = process.argv[2];
fs.readFile(JSON_FILE, (err, buffer) => {
try {
JSON.parse(buffer);
console.log("FS OK");
} catch (e) {
console.error("FS Failed");
}
});
fsPromises.readFile(JSON_FILE).then(buffer => {
try {
JSON.parse(buffer);
console.log("FS/promises OK");
} catch (e) {
console.error("FS/promises Failed");
}
});
promisifiedRead(JSON_FILE).then(buffer => {
try {
JSON.parse(buffer);
console.log("FS promisified OK");
} catch (e) {
console.error("FS promisified Failed");
}
});I tried with the following:
$ wget -O emojis.json https://api.github.com/emojis
$ node json.js ./emoji.json
(node:11797) ExperimentalWarning: The fs/promises API is experimental
FS/promises Failed
FS OK
FS promisified OK
I used the following script to find the thershold where the promises API starts to work:
const fs = require("fs");
const fsPromises = require("fs/promises");
const JSON_FILE = process.argv[2];
const readAndRemoveOneLine = () =>
fs.readFile(JSON_FILE, (err, buffer) => {
const data = JSON.parse(buffer);
delete data[Object.keys(data)[0]];
fs.writeFile(
JSON_FILE,
JSON.stringify(data),
err =>
err
? console.error(err)
: fsPromises
.readFile(JSON_FILE)
.then(
buffer =>
JSON.parse(buffer) &&
console.log("API works for that size:", buffer.length)
)
.catch(readAndRemoveOneLine)
);
});
readAndRemoveOneLine();And the API seems to work fine for files smaller than 16 kiB.
$ wget -O emojis.json https://api.github.com/emojis
$ node threshold.js emojis.json
(node:12609) ExperimentalWarning: The fs/promises API is experimental
API works for that size: 16368
When dealing with big files, the beginning of the file is cut. I was not able to reproduce this with small files.
I made a script to compare different approches, and
fs/promisesseems to be the only one impacted by thus bug.I tried with the following:
I used the following script to find the thershold where the promises API starts to work:
And the API seems to work fine for files smaller than 16 kiB.