Uint8Array to string in Javascript

Question

I have some UTF-8 encoded data living in a range of Uint8Array elements in Javascript. Is there an efficient way to decode these out to a regular javascript string (I believe Javascript uses 16 bit Unicode)? I dont want to add one character at the time as the string concaternation would become to CPU intensive.

Solution 1

TextEncoder and TextDecoder from the Encoding standard, which is polyfilled by the stringencoding library, converts between strings and ArrayBuffers:

var uint8array = new TextEncoder().encode("¢");
var string = new TextDecoder().decode(uint8array);

Solution 2

This should work:

// http://www.onicos.com/staff/iz/amuse/javascript/expert/utf.txt
/* utf.js - UTF-8 <=> UTF-16 convertion
 *
 * Copyright (C) 1999 Masanao Izumo <[email protected]>
 * Version: 1.0
 * LastModified: Dec 25 1999
 * This library is free.  You can redistribute it and/or modify it.
 */
function Utf8ArrayToStr(array) {
    var out, i, len, c;
    var char2, char3;
    out = "";
    len = array.length;
    i = 0;
    while(i < len) {
    c = array[i++];
    switch(c >> 4)
    { 
      case 0: case 1: case 2: case 3: case 4: case 5: case 6: case 7:
        // 0xxxxxxx
        out += String.fromCharCode(c);
        break;
      case 12: case 13:
        // 110x xxxx   10xx xxxx
        char2 = array[i++];
        out += String.fromCharCode(((c & 0x1F) << 6) | (char2 & 0x3F));
        break;
      case 14:
        // 1110 xxxx  10xx xxxx  10xx xxxx
        char2 = array[i++];
        char3 = array[i++];
        out += String.fromCharCode(((c & 0x0F) << 12) |
                       ((char2 & 0x3F) << 6) |
                       ((char3 & 0x3F) << 0));
        break;
    }
    }
    return out;
}

It's somewhat cleaner as the other solutions because it doesn't use any hacks nor depends on Browser JS functions, e.g. works also in other JS environments.

Check out the JSFiddle demo.

Also see the related questions: here and here

Solution 3

Here's what I use:

var str = String.fromCharCode.apply(null, uint8Arr);

Solution 4

In Node "Buffer instances are also Uint8Array instances", so buf.toString() works in this case.

Solution 5

Found in one of the Chrome sample applications, although this is meant for larger blocks of data where you're okay with an asynchronous conversion.

/**
 * Converts an array buffer to a string
 *
 * @private
 * @param {ArrayBuffer} buf The buffer to convert
 * @param {Function} callback The function to call when conversion is complete
 */
function _arrayBufferToString(buf, callback) {
  var bb = new Blob([new Uint8Array(buf)]);
  var f = new FileReader();
  f.onload = function(e) {
    callback(e.target.result);
  };
  f.readAsText(bb);
}

View more solutions

Comments

[Scarlet]

@jcubic for me, toString on Uint8Array returns comma-separated numbers such as "91,50,48,49,57,45" (Chrome 79)

[Jack Wester]

As you said, this would perform terribly unless the buffer to convert is really really huge. The synchronous UTF-8 to wchar converstion of a simple string (say 10-40 bytes) implemented in, say, V8 should be much less than a microsecond whereas I would guess that your code would require a hundreds times that. Thanks all the same.

[Albert]

From the doc, this does not seem to decode UTF8.

[Albert]

This does not do UTF8 decoding.

[Redsandro]

This will throw RangeError on bigger texts. "Maximum call stack size exceeded"

[Redsandro]

This seems kinda slow. But the only snippet in the universe I found that works. Good find+adoption!

[yonran]

If you are converting large Uint8Arrays to binary strings and are getting RangeError, see the Uint8ToString function from stackoverflow.com/a/12713326/471341.

[DanHorner]

I don't understand why this doesn't have more upvotes. It seems eminently sensible to crank through the UTF-8 convention for small snippets. Async Blob + Filereader works great for big texts as others have indicated.

[Vincent Scheib]

This does not produce the correct result from the example unicode characters on en.wikipedia.org/wiki/UTF-8. e.g. String.fromCharCode.apply(null, new Uint8Array([0xc2, 0xa2])) does not produce ¢.

[Jack Wester]

The question was how to do this without string concatenation

[unhammer]

Works great, except it doesn't handle 4+ byte sequences, e.g. fromUTF8Array([240,159,154,133]) turns out empty (while fromUTF8Array([226,152,131])→"☃")

[Evan Hu]

For anyone lazy like me, npm install text-encoding, var textEncoding = require('text-encoding'); var TextDecoder = textEncoding.TextDecoder;. No thanks.

[Rohit Poudel]

Add some description to answer. @terran

[doom]

Works great for me. And so simple ! But in fact Uint8Array have toString() method.

[wayofthefuture]

beware the npm text-encoding library, webpack bundle analyzer shows the library is HUGE

[Ten Bitcomb]

Even shorter: String.fromCharCode.apply(null, unitArr);. As mentioned, it doesn't handle UTF8 encoding, but sometimes this is simple enough if you only need ASCII support but don't have access to TextEncoder/TextDecoder.

[LeOn - Han Li]

Simple and elegant, was not aware Buffer is also Uint8Array. Thanks!

[fiffy]

Best solution here, as it also handles 4-byte-characters (e.g. emojis) Thank you!

[simbo1905]

and what is the inverse of this?

[Joachim Lous]

@doom On the browser side, Uint8Array.toString() will not compile a utf-8 string, it will list the numeric values in the array. So if what you have is a Uint8Array from another source that does not happen to also be a Buffer, you will need to create one to do the magic: Buffer.from(uint8array).toString('utf-8')

[Loilo]

Note that Node.js added the TextEncoder/TextDecoder APIs in v11, so no need to install any extra packages if you only target current Node versions.

[RaR]

Why cases 8, 9, 10 and 11 are excluded? Any particular reason? And case 15 is also possible, right? 15(1111) will denote 4 bytes are used, isn't it?

[Rosberg Linhares]

I think that nowadays the best polyfill is FastestSmallestTextEncoderDecoder, as recommended by the MDN website.

[Marc J. Schmidt]

String.fromCharCode.apply(null, chars) will error if chars is too big.

[simbo1905]

is that everywhere or just some browsers and is it documented at all?

[Marc J. Schmidt]

e.g. here developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/… But beware: by using apply this way, you run the risk of exceeding the JavaScript engine's argument length limit. The consequences of applying a function with too many arguments (that is, more than tens of thousands of arguments) varies across engines. (The JavaScriptCore engine has hard-coded argument limit of 65536.

[simbo1905]

thanks. in my case i was doing crypto over smallish strings so not a problem. do you have a fix for long strings? :-)

[Marc J. Schmidt]

solution is to batch in 64k chars.

[John Henckel]

This does not work in Chrome. Buffer is nodejs only.