Generating HTTP multipart body for file upload in JavaScript

Question

I'm trying to build HTTP multipart form data in JavaScript (on the server for Meteor.js HTTP request).

Here is the Meteor code that sends POST request.

var res = HTTP.post(url, {
  headers: formatted.headers,
  content: formatted.content
});

I'm preparing headers and content using this code.

function MultipartFormData(parts) {
  var boundary = (new Date()).getTime();
  var bodyParts = [];

  _.each(parts, function (value, key) {
    value.data = (new Buffer(value.data)).toString('binary');

    bodyParts.push(
      '---------------------------' + boundary,
      'Content-Disposition: form-data; name="' + key + '"; filename="' + value.filename + '"',
      'Content-Type: ' + value.contentType,
      '',
      value.data);
  });

  bodyParts.push('---------------------------' + boundary + '--', '');

  var bodyString = bodyParts.join('\r\n');

  return {
    content: bodyString,
    headers: {
      'Content-Type': 'multipart/form-data; boundary=' + '---------------------------' + boundary,
      'Content-Length': bodyString.length
    }
  }
}

Details about file:

key = 'file'
value.filename = 'file.png'
value.contentType = 'image/png'
value.data is an Uint8Array

Server is unable to process this request. When I'm using standard Node.js request object and FormBuilder using the same data everything work properly. I'm just requesting http connection between two servers. Could anyone tell me what is wrong with my code? I'm not the expert in HTTP protocol and I have only shreds of information about generating HTTP request's content.

And one more thing. I've tried converting Uint8Array to Buffer, ArrayBuffer, String and it also didn't work.

EDIT:

I've made test what is the http body when sending the same file in Firefox and in my app:

Firefox:

Content-Type: multipart/form-data; boundary=---------------------------19039834425958857471335503868
Content-Length: 299

-----------------------------19039834425958857471335503868
Content-Disposition: form-data; name="file"; filename="test.png"
Content-Type: image/png

PNG


IHDR

·ü]þPLTEÿâ  7
IDAT×cÀÚqÅIEND®B`
-----------------------------19039834425958857471335503868--

My App:

Content-Type: multipart/form-data; boundary=---------------------------1408816255735
Content-Length: 263

---------------------------1408816255735
Content-Disposition: form-data; name="file"; filename="test.png"
Content-Type: image/png

PNG


IHDR

·ü]þPLTEÿâ      7
IDA×cÀ
                                 ÚqÅIEND®B`
---------------------------1408816255735--

It differs a little bit but I don't know what is the source of this difference.

EDIT 2

And the server response is: Error: failed [400] Invalid multipart request with 0 mime parts.

EDIT 3

When generating body like this:

Content-Type: multipart/form-data; boundary=1408827490794
Content-Length: 213

--1408827490794
Content-Disposition: form-data; name="file"; filename="test.png"
Content-Type: image/png

PNG


IHDR

·ü]þPLTEÿâ  7
IDA×cÀ
      ÚqÅIEND®B`
--1408827490794--

I get the error: Error: failed [400] Missing end boundary in multipart body.

Solution 1

Fix the boundary

1. If you define boundary as: BOUNDARY,
2. then the beginning of each subpart is marked with: --BOUNDARY
3. and the end of message is marked with: --BOUNDARY--

Your boundary is defined as

Content-Type: multipart/form-data; boundary=---------------------------1408816255735

So the boundary string is: ---------------------------1408816255735, which already has 28 - (If I counted properly), So the divider should be -- + the boundary string, (not "just the boundary string").

I would get rid of overhead --------------------------- as it obscures if this preceding -- is added or not.

Content-Type: multipart/form-data; boundary=1408816255735

...

--1408816255735
Content-Disposition: form-data; name="file"; filename="test.png"

...

--1408816255735--

The "Missing end boundary in multipart body" Error

As you say the problem after fixing boundary is: Missing end boundary in multipart body.

You probably also need '\r\n' after ending bondary mark ;). As var bodyString = bodyParts.join('\r\n'); does not put \r\n the after the last element, which I suppose is required. (Our discussion in comments showed that the this is not that case, so...)

...so I suspect that Content-Length is set incorectly (it should be the byte count not character's count in string). As Content-Length is not required - maybe try without it. (And this happened to be the right guess in this case).

Binary data

Ok until now we fixed mistakes in given code. As you say now the multipart request is parsed successfully but You get server error when it tries to interpret sent data as PNG image.

I do not know how you get binary data - there is no code for that. Assuming that in your loop over partd value.data is nodejs Buffer with image bytes I would try with:

_.each(parts, function (value, key) {
    bodyParts.push(
        '--' + boundary,
        'Content-Disposition: form-data; name="' + key + '"; filename="' + 
                value.filename + '"',
        'Content-Transfer-Encoding: base64',
        'Content-Type: ' + value.contentType,
        '',
        value.data.toString('base64'));
});

Solution 2

To reply @Jared Martin question I haven't been calculating content length. This solution only works with base64. If you would like to send bigger amount of data you would need to use binary format and compose response as a buffer.

FormData = function () {
    this._parts = {};
};

FormData.prototype.append = function (name, part) {
    this._parts[name] = part;
};

FormData.prototype.generate = function () {
    var boundary = Date.now();
  var bodyParts = [];

  _.each(this._parts, function (part, name) {
    part.data = (new Buffer(part.data)).toString('base64');

    bodyParts.push(
      '--' + boundary,
      'Content-Disposition: form-data; name="' + name + '"; filename="' + part.filename + '"',
      'Content-Type: ' + part.contentType,
      'Content-Transfer-Encoding: base64',
      '',
      part.data);
  });

  bodyParts.push('--' + boundary + '--', '');

  return {
    headers: {
      'Content-Type': 'multipart/form-data; boundary=' + boundary,
    },
    body: bodyParts.join('\r\n')
  }
};