Rapidly updating image with Data URI causes caching, memory leak

Question

I have a webpage that rapidly streams JSON from the server and displays bits of it, about 10 times/second. One part is a base64-encoded PNG image. I've found a few different ways to display the image, but all of them cause unbounded memory usage. It rises from 50mb to 2gb within minutes. Happens with Chrome, Safari, and Firefox. Haven't tried IE.

I discovered the memory usage first by looking at Activity Monitor.app -- the Google Chrome Renderer process continuously eats memory. Then, I looked at Chrome's Resource inspector (View > Developer > Developer Tools, Resources), and I saw that it was caching the images. Every time I changed the img src, or created a new Image() and set its src, Chrome cached it. I can only imagine the other browsers are doing the same.

Is there any way to control this caching? Can I turn it off, or do something sneaky so it never happens?

Edit: I'd like to be able to use the technique in Safari/Mobile Safari. Also, I'm open to other methods of rapidly refreshing an image if anyone has any ideas.

Here are the methods I've tried. Each one resides in a function that gets called on AJAX completion.

Method 1 - Directly set the src attribute on an img tag

Fast. Displays nicely. Leaks like crazy.

$('#placeholder_img').attr('src', 'data:image/png;base64,' + imgString);

Method 2 - Replace img with a canvas, and use drawImage

Displays fine, but still leaks.

var canvas = document.getElementById("placeholder_canvas");
var ctx = canvas.getContext("2d");
var img = new Image();
img.onload = function() {
    ctx.drawImage(img, 0, 0); 
}   
img.src = "data:image/png;base64," + imgString;

Method 3 - Convert to binary and replace canvas contents

I'm doing something wrong here -- the images display small and look like random noise. This method uses a controlled amount of memory (grows to 100mb and stops), but it is slow, especially in Safari (~50% CPU usage there, 17% in Chrome). The idea came from this similar SO question: Data URI leak in Safari (was: Memory Leak with HTML5 canvas)

var img = atob(imgString);
var binimg = [];
for(var i = 0; i < img.length; i++) {
    binimg.push(img.charCodeAt(i));
}
var bytearray = new Uint8Array(binimg);

// Grab the existing image from canvas
var ctx = document.getElementById("placeholder_canvas").getContext("2d");
var width = ctx.canvas.width, 
    height = ctx.canvas.height;
var imgdata = ctx.getImageData(0, 0, width, height);

// Overwrite it with new data
for(var i = 8, len = imgdata.data.length; i < len; i++) {
    imgdata.data[i-8] = bytearray[i];
}

// Write it back
ctx.putImageData(imgdata, 0, 0);

Solution 1

I know it's been years since this issue was posted, but the problem still exists in recent versions of Safari Browser. So I have a definitive solution that works in all browsers, and I think this could save jobs or lives!.

Copy the following code somewhere in your html page:

// Methods to address the memory leaks problems in Safari
var BASE64_MARKER = ';base64,';
var temporaryImage;
var objectURL = window.URL || window.webkitURL;

function convertDataURIToBlob(dataURI) {
    // Validate input data
    if(!dataURI) return;

    // Convert image (in base64) to binary data
    var base64Index = dataURI.indexOf(BASE64_MARKER) + BASE64_MARKER.length;
    var base64 = dataURI.substring(base64Index);
    var raw = window.atob(base64);
    var rawLength = raw.length;
    var array = new Uint8Array(new ArrayBuffer(rawLength));

    for(i = 0; i < rawLength; i++) {
        array[i] = raw.charCodeAt(i);
    }

    // Create and return a new blob object using binary data
    return new Blob([array], {type: "image/jpeg"});
}

Then when you receive a new frame/image base64Image in base64 format (e.g. data:image/jpeg;base64, LzlqLzRBQ...) and you want to update a html <img /> object imageElement, then use this code:

// Destroy old image
if(temporaryImage) objectURL.revokeObjectURL(temporaryImage);

// Create a new image from binary data
var imageDataBlob = convertDataURIToBlob(base64Image);

// Create a new object URL object
temporaryImage = objectURL.createObjectURL(imageDataBlob);

// Set the new image
imageElement.src = temporaryImage;

Repeat this last code as much as needed and no memory leaks will appear. This solution doesn't require the use of the canvas element, but you can adapt the code to make it work.

Solution 2

I don't think there are any guarantees given about the memory usage of data URLs. If you can figure out a way to get them to behave in one browser, it guarantees little if not nothing about other browsers or versions.

If you put your image data into a blob and then create a blob URL, you can then deallocate that data.

Here's an example which turns a data URI into a blob URL; you may need to change / drop the webkit- & WebKit- prefixes on browsers other than Chrome and possibly future versions of Chrome.

var parts = dataURL.match(/data:([^;]*)(;base64)?,([0-9A-Za-z+/]+)/);

//assume base64 encoding
var binStr = atob(parts[3]);

//might be able to replace the following lines with just
// var view = new Uint8Array(binStr);
//haven't tested.

//convert to binary in ArrayBuffer
var buf = new ArrayBuffer(binStr.length);
var view = new Uint8Array(buf);
for(var i = 0; i < view.length; i++)
  view[i] = binStr.charCodeAt(i);
//end of the possibly unnecessary lines

var builder = new WebKitBlobBuilder();
builder.append(buf);

//create blob with mime type, create URL for it
var URL = webkitURL.createObjectURL(builder.getBlob(parts[1]))
return URL;

Deallocating is as easy as :

webkitURL.revokeObjectURL(URL);

And you can use your blob URL as your img's src.

Unfortunately, blob URLs do not appear to be supported in IE prior to v10.

API reference:

http://www.w3.org/TR/FileAPI/#dfn-createObjectURL

http://www.w3.org/TR/FileAPI/#dfn-revokeObjectURL

Compatibility reference:

http://caniuse.com/#search=blob%20url

Solution 3

Try setting image.src = "" after drawing.

var canvas = document.getElementById("placeholder_canvas");
var ctx = canvas.getContext("2d");
var img = new Image();
img.onload = function() {
    ctx.drawImage(img, 0, 0); 
    //after drawing set src empty
    img.src = "";
}   
img.src = "data:image/png;base64," + imgString;

This might be helped

Solution 4

I had a very similar issue.

Setting img.src to dataUrl Leaks Memory

Long story short, I simply worked around the Image element. I use a javascript decoder to decode and display the image data onto a canvas. Unless the user tries to download the image, they'll never know the difference either. The other downside is that you're going to be limited to modern browsers. The up side is that this method doesn't leak like a sieve :)

Solution 5

patching up ellisbben's answer, since BlobBuilder is obsoleted and https://developer.mozilla.org/en-US/Add-ons/Code_snippets/StringView provides what appears to be a nice quick conversion from base64 to UInt8Array:

in html:

<script src='js/stringview.js'></script>

in js:

window.URL =    window.URL ||
                window.webkitURL;
function blobify_dataurl(dataURL){
    var parts = dataURL.match(/data:([^;]*)(;base64)?,([0-9A-Za-z+/]+)/);

    //assume base64 encoding
    var binStr = atob(parts[3]);

    //convert to binary in StringView
    var view = StringView.base64ToBytes(parts[3]);

    var blob = new Blob([view], {type: parts[1]}); // pass a useful mime type here

    //create blob with mime type, create URL for it
    var outURL = URL.createObjectURL(blob);
    return outURL;
}

I still don't see it actually updating the image in Safari mobile, but chrome can receive dataurls rapid-fire over websocket and keep up with them far better than having to manually iterate over the string. And if you know you'll always have the same type of dataurl, you could even swap the regex out for a substring (likely quicker...?)

Running some quick memory profiles, it looks like Chrome is even able to keep up with deallocations (if you remember to do them...):

URL.revokeObjectURL(outURL);

View more solutions