Writing binary data using node.js fs.writeFile to create an image file

Question

I'm trying to write a canvas data with node.js fs.writeFile as a binary. JPEG file, but after the file is written I can see that the file is stored as plain text, not binary data.

This is an example of the data sent from the client to my node, representing the JPEG image data (just a few first characters):

/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAFA3PEY8MlBGQUZaVVBfeM...

I'm getting this data on the client side by performing:

canvas.toDataURL('image/jpeg', 0.5).replace('data:image/jpeg;base64,', '')

Here is the function usage in my node.js server:

fs.writeFile('../some.jpeg', data, 'binary', function(err){});

Instead of the file being written as binary (״״ JFIF ...), it writes exactly the data it received from the client.

What am I doing wrong here?

Solution 1

JavaScript language had no mechanism for reading or manipulating streams of binary data. The Buffer class was introduced as part of the Node.js API to make it possible to interact with octet streams in the context of things like TCP streams and file system operations.

Pure JavaScript, while great with Unicode encoded strings, does not handle straight binary data very well.

When writing large amounts of data to a socket it's much more efficient to have that data in binary format vs having to convert from Unicode.

var fs = require('fs');
// string generated by canvas.toDataURL()
var img = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAUCAYAAACNiR0"
    + "NAAAAKElEQVQ4jWNgYGD4Twzu6FhFFGYYNXDUwGFpIAk2E4dHDRw1cDgaCAASFOffhEIO"
    + "3gAAAABJRU5ErkJggg==";
// strip off the data: url prefix to get just the base64-encoded bytes
var data = img.replace(/^data:image\/\w+;base64,/, "");
var buf = Buffer.from(data, 'base64');
fs.writeFile('image.png', buf, /* callback will go here */);

Reference

Solution 2

I have had the question in question. I solved the problem when I made the default value null of "encoding" in the "request" library

var request = require("request").defaults({ encoding: null });
var fs = require("fs");

fs.writeFile("./image.png", body, function(err) {
    if (err) throw err;
});

Solution 3

Use Buffer.from, as Buffer is deprecated, will get the following warning

(node:15707) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.

var fs = require('fs');
// string generated by canvas.toDataURL()
var img = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAUCAYAAACNiR0"
    + "NAAAAKElEQVQ4jWNgYGD4Twzu6FhFFGYYNXDUwGFpIAk2E4dHDRw1cDgaCAASFOffhEIO"
    + "3gAAAABJRU5ErkJggg==";
// strip off the data: url prefix to get just the base64-encoded bytes
var data = img.replace(/^data:image\/\w+;base64,/, "");
var buf = Buffer.from(data, 'base64');
fs.writeFile('image.png', buf);

Solution 4

Instead of writing the file directly to your client, first, ask the server to send images in binary format.

   let request= {
        headers: {
            'Content-Type': 'image/jpeg',
            'Authorization': "your token"
        },
        encoding:'binary'
    };
     request.get(url,request,(error, response, body)=>{
        if(error){
            console.log('error in get photo',error)
            return "default image to server";  
        }else{
            if(response.statusCode == 200){ 

      Fs.writeFile('path',body,'binary',function(err){
                    if(err){
                        return "your message";   
                    }else{
                        return "success";
                    }
                })
            }else{
                console.log('error in get photo 3')
                return "your message";  
            }
        }
    })

View more solutions

Comments

[Rayon]

stackoverflow.com/a/6933413/1746830

[Rayon]

gist.github.com/madhums/e749dca107e26d72b64d Another approach...

[Koby Douek]

@Rayon Thanks, I'm looking through your 1st comment

[Koby Douek]

@Rayon I changed it to fs.writeFile('../some.jpeg', data, 'base64', function(err){}); but still it writes it as the same string it received.

[Koby Douek]

@Rayon var buf = new Buffer(data, 'base64'); fs.writeFile('image.png', buf); works ! Please post this as an answer so I can rep you.

[Rayon]

I was just about to suggest you later option... Glad it helped...

[Rayon]

stackoverflow.com/a/6926804/1746830 suggests the same approach.. I think it should be marked as a dupe...

[Koby Douek]

Rayon, This works perfectly. Thanks for all your help, peace and love from Israel.

[Rayon]

– I'm glad it helped! Happy Coding

[Mörre]

"Pure javascript... does not handle straight binary data very well." -- I don't understand this answer, given that this is a recent Q&A. Did you completely miss TypedArrays and ArrayBuffer? Binary data is not an issue in Javascript, and while it it's "newish" it's not exactly that new. .developer.mozilla.org/en-US/docs/Web/JavaScript/Typed_array‌​s

[Rayon]

@Mörre - I referred that statement from here

[Mörre]

@Rayon Which is from 2011. I refer to my comment above.

[Ricardo G Saraiva]

Work like a charm! Thx

[Rehmat]

I was downloading an image file using a link and require("request").defaults({ encoding: null }); solved my day. But, the answer can be improved a bit more.

[Salamit]

You sir, are a time saver! ^_^

[Crashalot]

thanks for this, but according to this answer, fs.write requires a callback function: stackoverflow.com/a/51155728/144088. how come this works for you?

[Rayon]

@Crashalot - I seriously did not pay attention to that callback argument. I have updated my answer. Thanks for pointing that out.