How to chain write stream, immediately with a read stream in Node.js 0.10?

Question

The following line will download an image file from a specified url variable:

var filename = path.join(__dirname, url.replace(/^.*[\\\/]/, ''));
request(url).pipe(fs.createWriteStream(filename));

And these lines will take that image and save to MongoDB GridFS:

 var gfs = Grid(mongoose.connection.db, mongoose.mongo);
 var writestream = gfs.createWriteStream({ filename: filename });
 fs.createReadStream(filename).pipe(writestream);

Chaining pipe like this throws Error: 500 Cannot Pipe. Not Pipeable.

request(url).pipe(fs.createWriteStream(filename)).pipe(writestream);

This happens because the image file is not ready to be read yet, right? What should I do to get around this problem?Error: 500 Cannot Pipe. Not Pipeable.

Using the following: Node.js 0.10.10, mongoose, request and gridfs-stream libraries.

Solution 1

request(url).pipe(fs.createWriteStream(filename)).pipe(writestream);

is the same as this:

var fileStream = fs.createWriteStream(filename);
request(url).pipe(fileStream);
fileStream.pipe(writestream);

So the issue is that you are attempting to .pipe one WriteStream into another WriteStream.

Solution 2

// create 'fs' module variable
var fs = require("fs");

// open the streams
var readerStream = fs.createReadStream('inputfile.txt');
var writerStream = fs.createWriteStream('outputfile.txt');

// pipe the read and write operations
// read input file and write data to output file
readerStream.pipe(writerStream);

Solution 3

I think the confusion in chaining the pipes is caused by the fact that the pipe method implicitly "makes choices" on it's own on what to return. That is:

readableStream.pipe(writableStream) // Returns writable stream
readableStream.pipe(duplexStream) // Returns readable stream

But the general rule says that "You can only pipe a Writable Stream to a Readable Stream." In other words only Readable Streams have the pipe() method.

Comments

[user568109]

Yes, you can only read from readable or duplex streams not writable. See github.com/joyent/node/pull/4843. To chain pipes the stream needs to be duplex i.e. both readable and writable

[loretoparisi]

It was actually the opposite: pipe a readable stream to a writeable stream.

[Ammar Bayg]

Then whats going on here? In npm axios documentation it says; response.data.pipe(fs.createWriteStream('path-to-file)) As we know that request is readable stream and response is writable stream, and you said that only Readable streams have pipe() method but here we are pipping writable stream to new writestream directly using pipe no read stream involve. This may suggest that pipe method is available to writable streams also.

[mechanicious]

@AmmarBayg I have written this post 2 years ago relating to npm version 0.10. Have you checked it on that version also?