JSON Zip Response in node.js

Question

I'm pretty new to node.js and I'm trying to send back a zip file containing JSON results. I've been trying to figure it out how to do it, but haven't had the expected results.

I'm using NodeJS, ExpressJS, LocomotiveJS, Mongoose and MongoDB.

Since we're building a mobile oriented application, I'm trying to save as many as bandwith as I can.

The daily initial load for the mobile app could be a big JSON document, so I want to zip it before sending it to the device. If possible I'd like to do it everything in memory in order to avoid disk I/O.

I tried with 3 libraries so far:

• adm-zip
• node-zip
• zipstream

The best result I achieved is using node-zip. Here's my code:

  return Queue.find({'owners': this.param('id')}).select('name extra_info cycle qtype purge purge_time tasks').exec(function (err, docs) {
    if (!err) {
      zip.file('queue.json', docs);
      var data = zip.generate({base64:false,compression:'DEFLATE'});

      res.set('Content-Type', 'application/zip');
      return res.send(data);
    }
    else {
      console.log(err);
      return res.send(err);
    }
  });

The result is a downloaded zip file but the content is unreadable.

I'm pretty sure I'm mixing up things, but to this point I'm not sure how to proceed.

Any advise?

Thanks in advace

Solution 1

You can compress output in express 3 with this:

app.configure(function(){
  //....
  app.use(express.compress());
});


app.get('/foo', function(req, res, next){
  res.send(json_data);
});

If the user agent supports gzip it will gzip it for you automatically.

Solution 2

For Express 4+, compress does not come bundled with Express and needs to be installed separately.

$ npm install compression

Then to use the library:

var compression = require('compression');
app.use(compression());

There are a bunch of options you can tune, see here for the list.

Solution 3

I think you mean how do I send Gzip content with node?

Node version 0.6 and above have a built in zlip module, so there is no need to require external modules.

You can send Gzip content like this.

 response.writeHead(200, { 'content-encoding': 'gzip' });
    json.pipe(zlib.createGzip()).pipe(response);

Obviously you will need to first check weather the client accepts the Gzip encoding and also remember gzip is a expensive operation so you should cache the results.

Here is full example taking from the docs

// server example
// Running a gzip operation on every request is quite expensive.
// It would be much more efficient to cache the compressed buffer.
var zlib = require('zlib');
var http = require('http');
var fs = require('fs');
http.createServer(function(request, response) {
  var raw = fs.createReadStream('index.html');
  var acceptEncoding = request.headers['accept-encoding'];
  if (!acceptEncoding) {
    acceptEncoding = '';
  }

  // Note: this is not a conformant accept-encoding parser.
  // See http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.3
  if (acceptEncoding.match(/\bdeflate\b/)) {
    response.writeHead(200, { 'content-encoding': 'deflate' });
    raw.pipe(zlib.createDeflate()).pipe(response);
  } else if (acceptEncoding.match(/\bgzip\b/)) {
    response.writeHead(200, { 'content-encoding': 'gzip' });
    raw.pipe(zlib.createGzip()).pipe(response);
  } else {
    response.writeHead(200, {});
    raw.pipe(response);
  }
}).listen(1337);