Calculating usage of localStorage space

Question

I am creating an app using the Bespin editor and HTML5's localStorage. It stores all files locally and helps with grammar, uses JSLint and some other parsers for CSS and HTML to aid the user.

I want to calculate how much of the localStorage limit has been used and how much there actually is. Is this possible today? I was thinking for not to simply calculate the bits that are stored. But then again I'm not sure what more is there that I can't measure myself.

Solution 1

I didn't find a universal way to get the remaining limit on the browsers I needed, but I did find out that when you do reach the limit there is an error message that pops up. This is of-course different in each browser.

To max it out I used this little script:

for (var i = 0, data = "m"; i < 40; i++) {
    try { 
        localStorage.setItem("DATA", data);
        data = data + data;
    } catch(e) {
        var storageSize = Math.round(JSON.stringify(localStorage).length / 1024);
        console.log("LIMIT REACHED: (" + i + ") " + storageSize + "K");
        console.log(e);
        break;
    }
}
localStorage.removeItem("DATA");

From that I got this information:

Google Chrome

• DOMException:
  • code: 22
  • message: "Failed to execute 'setItem' on 'Storage': Setting the value of 'data' exceeded the quota."
  • name: "QuotaExceededError"

Mozilla Firefox

• DOMException:
  • code: 1014
  • message: "Persistent storage maximum size reached"
  • name: "NS_ERROR_DOM_QUOTA_REACHED"

Safari

• DOMException:
  • code: 22
  • message: "QuotaExceededError: DOM Exception 22"
  • name: "QuotaExceededError"

Internet Explorer, Edge (community)

• DOMException:
  • code: 22
  • message: "QuotaExceededError"
  • name: "QuotaExceededError"

---

My solution

So far my solution is to add an extra call each time the user would save anything. And if the exception is caught then I would tell them that they are running out of storage capacity.

---

Edit: Delete the added data

I forgot to mention that for this to actually work you would need to delete the DATA item that was set originally. The change is reflected above by using the removeItem() function.

Solution 2

You may be able to get an approximate idea by using the JSON methods to turn the whole localStorage object to a JSON string:

JSON.stringify(localStorage).length

I don't know how byte-accurate it would be, especially with the few bytes of added markup if you're using additional objects - but I figure it's better than thinking you're only pushing 28K and instead doing 280K (or vice-versa).

Solution 3

IE8 implements the remainingSpace property for this purpose:

alert(window.localStorage.remainingSpace);  // should return 5000000 when empty

Unfortunately it seems that this is not available in the other browsers. However I am not sure if they implement something similar.

Solution 4

You can use the below line to accurately calculate this value and here is a jsfiddle for illustration of its use

alert(1024 * 1024 * 5 - escape(encodeURIComponent(JSON.stringify(localStorage))).length);

Solution 5

Ran into this today while testing (exceeding storage quota) and whipped up a solution. IMO, knowing what the limit is and where we are in relation is far less valuable than implementing a functional way to continue storing beyond the quota.

Thus, rather than trying to do size comparisons and capacity checks, lets react when we've hit the quota, reduce our current storage by a third, and resume storing. If said reduction fails, stop storing.

set: function( param, val ) { 
    try{
        localStorage.setItem( param, typeof value == 'object' ? JSON.stringify(value) : value )
        localStorage.setItem( 'lastStore', new Date().getTime() )
    }
    catch(e){
      if( e.code === 22 ){
        // we've hit our local storage limit! lets remove 1/3rd of the entries (hopefully chronologically)
        // and try again... If we fail to remove entries, lets silently give up
        console.log('Local storage capacity reached.')

        var maxLength = localStorage.length
          , reduceBy = ~~(maxLength / 3);

        for( var i = 0; i < reduceBy; i++ ){
          if( localStorage.key(0) ){
            localStorage.removeItem( localStorage.key(0) );
          }
          else break;
        }

        if( localStorage.length < maxLength ){
          console.log('Cache data reduced to fit new entries. (' + maxLength + ' => ' + localStorage.length + ')');
          public.set( param, value );
        }
        else {
          console.log('Could not reduce cache size. Removing session cache setting from this instance.');
          public.set = function(){}
        }
      }
    }
}

This function lives within a wrapper object, so public.set simply calls itself. Now we can add to storage and not worry what the quota is or how close we are too it. If a single store is exceeding 1/3rd the quota size is where this function will stop culling and quit storing, and at that point, you shouldn't be caching anyways, right?

View more solutions

Comments

[brasofilo]

Related: What is the max size of localStorage values?

[CoderPi]

JavaScript / HTML5 localStorage usefull Functions: stackoverflow.com/q/34245593/4339170

[tripleee]

Possible duplicate of How to find the size of localStorage

[Daniel Vassallo]

@WmasterJ: I'm seeing that the IE Team actually proposed for this (or something similar) to be included in the standard (back in 2008): markmail.org/thread/ugzdcamtyftcyk6y.

[JeroenEijkhof]

This time IE was right. It would seem obvious that this is needed.

[artlung]

Very nice, what was the value of i when this was reached in each case?

[JeroenEijkhof]

20-21 i think. You can run the test yourself if you want to.

[Christopher]

This is actually accurate enough to work with. With localStorage maxed out on Chrome 14, JSON.stringify(localStorage).length === 2636625 or 2.51448 MB, which is close enough (dev-test.nemikor.com/web-storage/support-test). Used in tandem with the try{} catch{}, and you've got enough to build a helper class.

[Rui Lima]

IE: ABORT_ERR: 20 code: 22 message: "QuotaExceededError" name: "QuotaExceededError"

[Rui Lima]

in IE your code raises exception before space end: window.localStorage.remainingSpace : 805692 | window.localStorage.getItem("DATA").length : 4194304 | JSON.stringify(localStorage).length : 4194315 | i: 22

[Rui Lima]

I don't understand this, Microsoft says: "The localStorage attribute provides persistent storage areas for domains. It allows Web applications to store nearly 10 MB of user data, such as entire documents or a user's mailbox, on the client for performance reasons." but at the end remainingSpace returns 5000000 (?!?!) why?!

[Mortimer]

note that, as pointed out in the test you link, this is a size in characters and not in Bytes: "Strings in JavaScript are UTF-16, so each character requires two bytes of memory. This means that while many browsers have a 5 MB limit, you can only store 2.5 M characters."

[Adam Tuttle]

I'm using this algorithm to determine used space for a chrome extension, but I remember reading another SO question (forget which one, now) that JS uses UTF-16 strings instead of UTF-8, and as such, you have to double the number of bytes you get by doing string.length. Do you know if that's true? If so, I'll have to update my code...

[Justin]

your jsfiddle doesn't load in my browser (mac/chrome).

[Ishmael Smyrnow]

@RuiLima, note that because characters in javascript use up two bytes instead of one, 5000000 characters actually takes up 10MB of space. Seems like IE may be reporting characters remaining, instead of bytes, or they changed their minds about the storage limit.

[c24w]

It looks like you broke JSFiddle, although you can still see the output here!

[Andy]

This remainingSpace property is important if you're expecting your overflowing of localStorage to throw an exception: IE won't throw one. Thus you need to check this property before and after your attempted save into localStorage. If the size hasn't changed you know you've exceeded your limit.

[Abhishek]

Don't you have to multiply by 8 for 8 bytes per character or something like that?

[jas-]

George, that example assumes a UTF-8 character set as defined by the 'Content-Type' meta tag on the majority of US based HTML sites. If you wish to account for unicode support take a look here mathiasbynens.be/notes/javascript-unicode Or for the quick fix here is a function to account for unicode support in JS github.com/craniumslows/Countable/commit/…

[brasofilo]

I wrapped the unescape function in parenthesis so it worked 1024 * 1024 * 5 - ( unescape(/*etc*/).length )

[cdmckay]

Doesn't this assume that you have 5MB max?

[jas-]

Which according to the RFC's regarding the HTML5 localStorage, sessionStorage & globalStorage browser options is the maximum per Same Origin policy.

[brasofilo]

jas, there gotta be something to adjust in your formula, even after passed 1024*1024*5, there's still quite some space (in Chrome). The problem with @cdmckay code is that it takes a while to calculate if there's too much space. I'm building a web app to manage Stack favorites and would like to add the "space remaining" feature. If you guys could test-drive, I'd be honoured. Thanks!

[jas-]

You are going to have to provide a test case for me to review. I have tested several browsers with the above method without problems.

[NiRUS]

good then i can wrap the local storage code in try catch block and alert user in catch block on size exceed.

[Jules]

Has the spec changed since this? All this appears to be doing, for my chrome browser, is subtracting 15 from 1024*1024*5. 15 = the length of the following: {"size": "5000"}

[jas-]

You are right. I misread the encode() function API believing it was operating on the bytes of ALL characters, not just those non-utf8. The best method would be to encode as a byte array and get the size of that.

[Juribiyan]

You brobably meant 5120k char

[Morteza Tourani]

@Juribiyan Yes I do.

[Thien Ly]

Okay, now those DOMException are still marked as experimental and shouldn't be used in production. I'm not sure I should catch those by checking DOMException name. i.e. QuotaExceededError