NodeJs: binary fs.createReadStream streamed as UTF-8

Question

My task at hand was to read a jpeg file in NodeJs and send it as http-response to answer a server request. Seemed to be trivial. However, my first solution failed. The browser did receive some binary gibrish, that was about 30% larger than the original file.

My code was (simplified; res is of type SeverResponse):

...
var fs = require('fs');
var stream = fs.createReadStream(pathToJPEG, {encoding: 'binary'});
res.setHeader('Content-Type', "image/jpeg");
stream.pipe(res);
...

As it turned out, what arrived at the browser was the UTF-8 encoded version of my source data. I also was able to exlude the response object to be the culprit. When I gave it an alternative stream (from Buffer, not file) it worked just fine.

Turns out the solution to my problem was to drop the option {encoding: 'binary'}. With that my browser received the right picture:

...
var fs = require('fs');
var stream = fs.createReadStream(pathToJPEG);
res.setHeader('Content-Type', "image/jpeg");
stream.pipe(res);
...

My question is: Why?

It seems intuitive that the first non-working version should be the correct one since it explicitly declares how to read the file.

Solution 1

This is because the binary encoding is not really binary. createReadStream uses the same encoding parameters that accepted by Buffer. From the Node Buffer Docs:

'binary' - A way of encoding the buffer into a one-byte (i.e. latin-1) encoded string. The string 'latin-1' is not supported. Instead simply pass 'binary' to use 'latin-1' encoding.

Just set encoding to null to get the raw stream or buffer, or don't specify anything at all, as you did in your second example.

Solution 2

Ixe is correct, changing encoding to null worked, but only after upgrading to a newer node/express package. Here is my code that correctly uploads a tar file:

    fs.exists(filePath, function(exists){
    if (exists) {
        var stat = fs.statSync(filePath);
        console.log('sending file, size %d', stat.size);

        res.writeHead(200, {
            "Content-Type": "application/x-tar",
            "Content-Disposition": "attachment; filename=" + filePath,
            "Content-Length": stat.size,
            "Content-Transfer-Encoding": "binary"
        });
        fs.createReadStream(filePath, { encoding: null }).pipe(res);    //must set encoding to null, as binary is not treated correctly

    } else {
        console.log('file not exist.');
        res.writeHead(400, {"Content-Type": "text/plain"});
        res.end("ERROR File does not exist");
    }
});

Comments

[James Watkins]

This didn't work for me. I'm still getting the file re-encoded and corrupt.