Getting binary content in Node.js using request

Question

I was trying to GET a binary data using request, and had something like:

var requestSettings = {
    method: 'GET',
    url: url,
};
request(requestSettings, function(error, response, body) {
    // Use body as a binary Buffer
}

But body was always a few bytes different from expected. After further investigation I found out that request assumed body is string and replaced all non-unicode bytes.

I tried to add

encoding: 'binary'

to requestSettings but it didn't help.

How can I get the binary data?

Solution

OK, after a lot of digging, I found out that requestSettings should have:

encoding: null

And then body will be of type Buffer, instead of the default, which is string.

Comments

[Dan Nissenbaum]

What an absurd nightmare. Took me 12 hours to hunt this down. It seems that the Node Request module, by default, treats incoming data in the content of the response as UTF-8, and automatically converts any non-UTF-8 byte sequences to junk (but valid UTF-8) characters. No amount of setting 'mimetype", etc. works (not that it's supposed to for response data). The encoding: null is the only option that works. And - very poorly documented. There ought to be an obvious warning in the Node Request documentation about how to retrieve pure binary data. Thanks!

[GilZ]

@StoyanBerov, I'm glad you found this answer helpful, but in the 5 years since I wrote this answer, the package readme was corrected to highlight this solution in several places. In addition, I highly recommend using a package that supports Promises instead of this package.

[Stoyan Berov]

@Gilz, thanks for update! I was actually under the impression that encoding is set to null by default. Also, the issue came up at a legacy project, set to a super old node version and callbacks-only everywhere.