How to read an entire text stream in node.js?
Solution 1
The key is to use these two Stream events:
Event: 'data'
Event: 'end'
For stream.on('data', ...)
you should collect your data data into either a Buffer (if it is binary) or into a string.
For on('end', ...)
you should call a callback with you completed buffer, or if you can inline it and use return using a Promises library.
Solution 2
As node.js is event and stream oriented there is no API to wait until end of stdin and buffer result, but it's easy to do manually
var content = '';
process.stdin.resume();
process.stdin.on('data', function(buf) { content += buf.toString(); });
process.stdin.on('end', function() {
// your code here
console.log(content.split('').reverse().join(''));
});
In most cases it's better not to buffer data and process incoming chunks as they arrive (using chain of already available stream parsers like xml or zlib or your own FSM parser)
Solution 3
Let me illustrate StreetStrider's answer.
Here is how to do it with concat-stream
var concat = require('concat-stream');
yourStream.pipe(concat(function(buf){
// buf is a Node Buffer instance which contains the entire data in stream
// if your stream sends textual data, use buf.toString() to get entire stream as string
var streamContent = buf.toString();
doSomething(streamContent);
}));
// error handling is still on stream
yourStream.on('error',function(err){
console.error(err);
});
Please note that process.stdin
is a stream.
Solution 4
There is a module for that particular task, called concat-stream.
Solution 5
If you are in async
context and have a recent version of Node.js, here is a quick suggestion:
const chunks = []
for await (let chunk of readable) {
chunks.push(chunk)
}
console.log(Buffer.concat(chunks))
Related videos on Youtube
Aadit M Shah
Computer science engineer with over ten years of experience in software development. Implemented web development, web analytics, and information technology solutions for both Cloud 100 and startup companies. Contributed to several open source software projects, with over 1.2 million aggregate downloads.
Updated on July 28, 2022Comments
-
Aadit M Shah almost 2 years
In RingoJS there's a function called
read
which allows you to read an entire stream until the end is reached. This is useful when you're making a command line application. For example you may write atac
program as follows:#!/usr/bin/env ringo var string = system.stdin.read(); // read the entire input stream var lines = string.split("\n"); // split the lines lines.reverse(); // reverse the lines var reversed = lines.join("\n"); // join the reversed lines system.stdout.write(reversed); // write the reversed lines
This allows you to fire up a shell and run the
tac
command. Then you type in as many lines as you wish to and after you're done you can press Ctrl+D (or Ctrl+Z on Windows) to signal the end of transmission.I want to do the same thing in node.js but I can't find any function which would do so. I thought of using the
readSync
function from thefs
library to simulate as follows, but to no avail:fs.readSync(0, buffer, 0, buffer.length, null);
The file descriptor for stdin (the first argument) is
0
. So it should read the data from the keyboard. Instead it gives me the following error:Error: ESPIPE, invalid seek at Object.fs.readSync (fs.js:381:19) at repl:1:4 at REPLServer.self.eval (repl.js:109:21) at rli.on.self.bufferedCmd (repl.js:258:20) at REPLServer.self.eval (repl.js:116:5) at Interface.<anonymous> (repl.js:248:12) at Interface.EventEmitter.emit (events.js:96:17) at Interface._onLine (readline.js:200:10) at Interface._line (readline.js:518:8) at Interface._ttyWrite (readline.js:736:14)
How would you synchronously collect all the data in an input text stream and return it as a string in node.js? A code example would be very helpful.
-
beatgammit over 11 yearsYou can't synchronously read in an asynchronous stream. Why would you want to anyway?
-
ton over 9 yearsI'm trying to do the same thing. The reason is to create a interactive option in my program, useful for a lot of reasons. A async reader do not help too much.
-
ton over 9 years
-
-
Mitar about 11 yearsYou can do
process.stdin.setEncoding('utf-8');
after resume andbug
in callback will already be string. -
Raja over 9 yearsSimilar, but using
Buffer.concat()
: stackoverflow.com/questions/10686617/… -
Raja over 9 yearsThis module allows you to intersperse the chunks with another string. Probably only useful for debugging: npmjs.org/package/join-stream
-
evandrix about 8 years@Mitar: it's
buf
, notbug
. -
Paul about 7 yearsWhy reverse the string?
-
Andrey Sidorov about 7 yearsthat was just an example of doing something with data
-
Brad Parks over 6 yearsNote: If nothing is passed to stdin, your program will hang. You can detect if your script is being called in a pipe using
process.stdin.isTTY
, as detailed in this other answer on stackoverflow