Javascript string size limit: 256 MB for me - is it the same for all browsers?

Question

Curious about what was the maximum string length I could get in Javascript, I tested it myself, today, on my Firefox 43.0.1, running in Windows 7. I was able to construct a string with length 2^28 - 1, but when I tried to create a string with one more char, Firebug showed me the "allocation size overflow" error, meaning the string must be less than 256 MB.

Is this the same thing for all browsers, all computers, all operational systems, or it depends?

I created the following snippet to find out the limit:

(function() {
    strings = ["z"];
    try {
        while(true) {
            strings.push(strings[strings.length - 1] + strings[strings.length - 1]);
        }
    } catch(err) {
        var k = strings.length - 2;
        while(k >= 0) {
            try {
                strings.push(strings[strings.length - 1] + strings[k]);
                k--;
            } catch(err) {}
        }
        console.log("The maximum string length is " + strings[strings.length - 1].length);
    }
})();

If you are running a different browser/OS, I would like to see your results. My result was The maximum string length is 268435455.

P.S.: I searched around for an answer, but the most recent topic I found was from 2011, so I am looking for a more up-to-date information.

Solution 1

Characters are stored on 16 bits

When you see that 256*2**20 characters are in a string, that does not mean that 256 megabytes of memory is allocated. JavaScript stores every character on two bytes (as it is utf16 encoded by the specification).

A word about ropes

Todays browsers (even IE) store strings in an advanced way, most often using a rope datastructure.

• Ropes do not need a coherent memory region to be allocated
• Can even deduplicate substrings, that means s+s does not necessarily use twice the memory as s
• Concatenation is very fast
• Element access is a bit slower

By examining some runs in IE and Chrome, I would say that both of them use some lazy evaluation for strings, and will try to expand them occasionally. After running the following snippet, none of the browsers used more memory than before. But if I tried to manipulate the stored window.LONGEST_STRING in the console, IE throw an out of memory error, and chrome froze for a short time, and consumed a lot of memory (>2 GB).

ps: On my laptop IE11 had a maximum string size of 4 GB, Chrome had 512 MB

Browser behaviour

IE11

Chrome47

A faster algorithm to determine max string size

var real_console_log = console.log;
console.log = function(x) {
  real_console_log.apply(console, arguments);
  var d = document,b=d.body,p=d.createElement('pre');
  p.style.margin = "0";
  p.appendChild(d.createTextNode(''+x));
  b.appendChild(p);
  window.scrollTo(0, b.scrollHeight);
};


function alloc(x) {
    if (x < 1) return '';
    var halfi = Math.floor(x/2);
    var half = alloc(halfi);
    return 2*halfi < x ? half + half + 'a' : half + half;
}

function test(x) {
    try {
        return alloc(x);
    } catch (e) {
        return null;
    }
}

function binsearch(predicateGreaterThan, min, max) {
    while (max > min) {
        var mid = Math.floor((max + min) / 2);
        var val = predicateGreaterThan(mid);
        if (val) {
            min = mid + 1;
        } else {
            max = mid;
        }
    }
    return max;
}

var maxStrLen = binsearch(test, 10, Math.pow(2, 52)) - 1;
console.log('Max string length is:');
console.log(maxStrLen + ' characters');
console.log(2*maxStrLen + ' bytes');
console.log(2*maxStrLen/1024/1024 + ' megabytes');
console.log('');
console.log('Store longest string');
window.LONGEST_STRING = alloc(maxStrLen);

console.log('Try to read first char');
console.log(window.LONGEST_STRING.charAt(0));
console.log('Try to read last char');
console.log(window.LONGEST_STRING.charAt(maxStrLen - 1));
console.log('Try to read length');
console.log(window.LONGEST_STRING.length);

Solution 2

Internal implementations can use either UCS2 or UTF16. As @hege_hegedus suggested, atleast Firefox uses the Rope structure (https://dxr.mozilla.org/mozilla-central/search?q=%2Btype-ref%3ARopeBuilder). The codes give me below results:

CHROME VERSION 39.0.2171.95 OS VERSION Linux: 3.13.0-43-generic

Firefox 34.0

Chrome output (from @@hege_hegedus code): Max string length is: 268435440 characters 536870880 bytes 511.9999694824219 megabytes Store longest string Try to read first char a Try to read last char a Try to read length 268435440

Firefox output (from OP code): "The maximum string length is 268435455"

Archived at http://gpupowered.org/string_js.txt

Solution 3

A bug report for the chromium tracker has this comment:

... When allocation fails, we create a 
Failure pointer encoding the amount requested, as well as some tag and 
type bits. This puts a limit on the maximally possible allocation 
request in 32-bit versions of 2^27-1. The maximal flat string length is 
~2^28 (512MB space), and the maximal string length is 2^29-1...

Note that this is from 2009, so I imagine that this still has consequences in current versions of V8 as the previous link is in regard to a NodeJS tool running into limits of toString().

Comments

[epascarello]

Or are you just testing the fact a browser errors out when doing a while loop....

[Clay]

Looks like an array not a string...

[Sebastian Simon]

I get “RangeError: repeat count must be less than infinity and not overflow maximum string size” as soon as I try "a".repeat(268435456) in Firefox 45.0a2 on Ubuntu 15.10.

[epascarello]

ecma-international.org/ecma-262/6.0/…

[Tamas Hegedus]

I know you didn't ask, but you can use Blobs to store more data. Blobs will reliably handle 500MB of data (the last time I checked, chrome just cut off the data behind 500MB)

[Admin]

Quoting Douglas Crockford from here. The ECMAScript Programming Language Specification does not specify a maximum length. The maximum length with will be implementation-specific. For some implementations, it will be a function of available memory, but you should not rely on that if portability is important.

[traktor]

Since strings are 16 bits per character, a string of 250 MB characters is using 500 MB of memory. On the surface, limitations of string size could be the same as a limitation on blob size @hege_hegedus

[Barmar]

@noob The ES6 spec linked to above says the max is 2^53-1 characters. That Crockford response is over 10 years old.

[Tamas Hegedus]

Just a sidenote: 2^53-1 is the largest number below what every positive integer can be represented in an IEEE double precision floating point number. That is a theoretical limitation. In practice, much smaller maximum string lengths are expected.

[check_ca]

Note that since Firefox 65, the max size of a String is approximately 1GB, see developer.mozilla.org/en-US/docs/Mozilla/Firefox/Releases/…

[Pedro A]

It is curious that your maximum length on Chrome47 is only 15 characters less than my limit, I was expecting differences to be related to powers of 2... Thank you very much!

[Tamas Hegedus]

@Hamsteriffic I wouldn't be surprised if the maximum size depended on the size of the concatenated substrings.