Compressing base64 data uri images

Question

Problem

I'm creating multiple charts that are then sent to the server to create a PDF. This data can get large.

Question

What is the best way to compress and send the image data to the server

Background

The charts I'm creating are fairly complex, to save myself the headache all of this is converted to a canvas where the base64 data uri is generated. Currently the data uri(s) are posted to the server to handle the processing. Posted info can get fairly large at around 400-600kb each for a small chart and 12mb for the largest chart.

The charts are org charts that can be manipulated/reordered.

Is there a better method of compressing these strings before sending it back up to the server?

Research

Some things I've check out:

https://github.com/brunobar79/J-I-C/blob/master/src/JIC.js (does not look like an option just resizes)

http://pastebin.com/MhJZBsqW (looks interesting)

But references external libraries that I cannot find: C_H_Image Lib_MinifyJpeg

https://code.google.com/p/jslzjb/source/browse/trunk/Iuppiter.js?r=2 (looks like this could work but relies on decompression on server side)

Solution 1

Maybe string compression is the solution for you. This converts the data to byte arrays.

There are multiple implementations and algorithms around, for instance

• LZMA-JS A standalone JavaScript implementation of the Lempel-Ziv-Markov chain (LZMA) compression algorithm.

  my_lzma = new LZMA("./lzma_worker.js");
  my_lzma.compress("This is my compression test.", 1, function on_compress_complete(result) {
      console.log("Compressed: " + result);
      my_lzma.decompress(result, function on_decompress_complete(result) {
          console.log("Decompressed: " + result);
      }, function on_decompress_progress_update(percent) {
          console.log("Decompressing: " + (percent * 100) + "%");
      });
  }, function on_compress_progress_update(percent) {
      console.log("Compressing: " + (percent * 100) + "%");
  });
  

• lz-string: JavaScript compression, fast!

  var str = "This is my compression test.";
  console.log("Size of sample is: " + str.length);
  var compressed = LZString.compress(str);
  console.log("Size of compressed sample is: " + compressed.length);
  str = LZString.decompress(compressed);
  console.log("Sample is: " + str);
  

Solution 2

I needed to generate thumbnails from larger pictures. I decided to solve my version of this problem with the HTML5 Canvas technique. I am using GWT so my code is:

//Scale to size
public static String scaleImage(Image image, int width, int height) {

    Canvas canvasTmp = Canvas.createIfSupported();
    Context2d context = canvasTmp.getContext2d();

    canvasTmp.setCoordinateSpaceWidth(width);
    canvasTmp.setCoordinateSpaceHeight(height);

    ImageElement imageElement = ImageElement.as(image.getElement());

    context.drawImage(imageElement, 0, 0, width, height);

    //Most browsers support an extra option for: toDataURL(mime type, quality) where quality = 0-1 double.
    //This is not available through this java library but might work with elemental version?
    String tempStr = canvasTmp.toDataUrl("image/jpeg");

    return tempStr;
}

If you are using JS you can probably get the idea:

• Make a canvas the size of the desired output image
• draw the input image on the canvas in the new size
• call canvas.toDataURL("mime-type", quality)

You can use any mime-type I think, but for me the jpeg one was the smallest and was comparable to results of my desktop image program.

My GWT would not let me do the quality parameter (and I'm not 100% sure how widely supported it is in which browsers), but that was OK because the resulting images were quite small. If you leave the mime-type blank it defaults to png which in my case was 3-4x larger than jpeg.

Solution 3

I finally got this working with fairly large files.

To do it, I used lz-string, as shown above.

There is one issue though, which is that sending UTF-16 data via ajax is deprecated, so.. it doesn't work. The fix is to convert to and from UTF-16 and UTF-8.

Here's the conversion code I'm using. I'm not sure what the standard endian is, btw:

var UTF = {};

UTF.U16to8 = function(convertMe) {
var out = "";
  for(var i = 0; i < convertMe.length; i++) {
      var charCode = convertMe.charCodeAt(i);
      out += String.fromCharCode(~~(charCode/256));
      out += String.fromCharCode(charCode%256);
    }
  return out;
}

UTF.U8to16 = function(convertMe) {
  var out = ""
  for(var i = 0; i < convertMe.length; i += 2) {
    var charCode = convertMe.charCodeAt(i)*256;
    charCode += convertMe.charCodeAt(i+1);
    out += String.fromCharCode(charCode)
  }
  return out;
}

If you're using Node.js, this code works on both sides. If not, you'll have to write your own version of these converters for the server side.

Phew! What a day this was.

Comments

[David Nguyen]

I think compression is the way to go, I'll look into this. Thanks for the answer.

[David Nguyen]

What worries be is that I'll have to compress client side and decompress server side. Hopefully the differently libraries will work together.

[David Nguyen]

Thanks for the suggestion, I don't think it would work out because the input was truncating the data.

[Seph Reed]

I tried lz-string, and it appears to not work for images larger than 100x100

[Seph Reed]

I figured it out, it was an issue with UTF-16 being deprecated. See my answer below on how to get around this.

[Akashii]

I'm compress then convert to UTF-8 before sending to server. In server I convert to UTF-16 then decompress but I get null

[KolonUK]

UTF-16 is not DEPRECATED, just that it has issues... Plus, even if you use it on both ends, enabling compression on the transfer doesn't really solve the amount of data transferred...

[Dennis Hackethal]

This is cool but for me it loses most of the compression UTF-16 affords.