How to pipe one readable stream into two writable streams at once in Node.js?
Solution 1
I found that zlib returns a readable stream which can be later piped into multiple other streams. So I did the following to solve the above problem:
var sourceFileStream = fs.createReadStream(sourceFile);
// Even though we could chain like
// sourceFileStream.pipe(zlib.createGzip()).pipe(response);
// we need a stream with a gzipped data to pipe to two
// other streams.
var gzip = sourceFileStream.pipe(zlib.createGzip());
// This will pipe the gzipped data to response object
// and automatically close the response object.
gzip.pipe(response);
// Then I can pipe the gzipped data to a file.
gzip.pipe(fs.createWriteStream(targetFilePath));
Solution 2
Pipe chaining/splitting doesn't work like you're trying to do here, sending the first to two different subsequent steps:
sourceFileStream.pipe(gzip).pipe(response);
However, you can pipe the same readable stream into two writeable streams, eg:
var fs = require('fs');
var source = fs.createReadStream('source.txt');
var dest1 = fs.createWriteStream('dest1.txt');
var dest2 = fs.createWriteStream('dest2.txt');
source.pipe(dest1);
source.pipe(dest2);
esengineer
Eye'm a Soft? Reindeer at the River. Previously worked at Korean Google, as a project manager of not MySQL open source database; then a full time Node.js developer. At Korean Google I developed a real-time distributed log search system which used ElasticSearch, Kafka, Storm, Spark, CUBRID, Redis, MongoDB, and Node.js. Glossary Eye = I ? = where Reindeer = Engineer River = Amazon Korean Google = NAVER Corp. not MySQL = CUBRID
Updated on June 17, 2022Comments
-
esengineer almost 2 years
The goal is to:
- Create a file read stream.
- Pipe it to gzip (
zlib.createGzip()
) Then pipe the read stream of zlib output to:
1) HTTP
response
object2) and writable file stream to save the gzipped output.
Now I can do down to 3.1:
var gzip = zlib.createGzip(), sourceFileStream = fs.createReadStream(sourceFilePath), targetFileStream = fs.createWriteStream(targetFilePath); response.setHeader('Content-Encoding', 'gzip'); sourceFileStream.pipe(gzip).pipe(response);
... which works fine, but I need to also save the gzipped data to a file so that I don't need to regzip every time and be able to directly stream the gzipped data as a response.
So how do I pipe one readable stream into two writable streams at once in Node?
Would
sourceFileStream.pipe(gzip).pipe(response).pipe(targetFileStream);
work in Node 0.8.x?