Uploading binary file on Node.js

Question

I am using Flash to record and upload audio to a node server. The Flash client is a variation of jrecorder. When the user is done recording the audio is uploaded using a POST request (not a form because Flash cannot create files) with the audio ByteArray as the data of the POST request (see more here).

I am able to receive the file correctly on Node-land using the code below but the audio that comes out is mangled and you cannot hear anything. With that said, the content of the file can be played by VLC and other players + Sox is able to encode it as an mp3.

Here is my code when using Node:

var express = require('express');
var app = express();

app.use (function(req, res, next) {
    req.rawBody = '';
    req.setEncoding('utf8');

    if(req.method.toLowerCase() == "post")
    {
        req.on('data', function(chunk) { req.rawBody += chunk });
        req.on('end', function() { done(req, res); });
    }

    next();
});

function done(req, res)
{
    fs.writeFile('abc.wav', req.rawBody, 'binary', function(err){
        if (err) throw err;

        // Save file to S3
    }   
}

Now if I use the same Flash client and make the POST request to a Rails server and use the code below, the file is saved perfectly.

def record
    file = request.raw_post

    # Save file to S3
end

Note that I am not a Node expert so please if you have any suggestions on what should I use instead of saving the chunks please post code examples. My main purpose right now is to get this to a working state before exploring other way of accomplishing more efficiently in Node (buffers, streams, etc)

Solution

Take out the following line

req.setEncoding('utf8');

You're not receiving utf8 data, you're receiving binary data.

You would be better off using a buffer instead of a string

app.use(function(req, res, next) {
  var data = new Buffer('');
  req.on('data', function(chunk) {
      data = Buffer.concat([data, chunk]);
  });
  req.on('end', function() {
    req.rawBody = data;
    next();
  });
});

Comments

[Gabriel Cebrian]

Dear sir you are AWESOME! That worked, thanks! I can't believe I missed that line. It was one of the many iterations I'v tried today. Thanks again!

[Johan S]

Thanks for the great answer! I had so much problems with this and I don't understand how this couldn't be default implemented by express, but nvm. Thanks!

[sakib11]

So when I use form Data as my payload, doesn't it convert to binary?

[Aijaz]

@sakib11 were you able to crack this? I am having trouble to upload parquet files. I believe it's encoding changes when I upload it.