Unzipping files

Question

I want to display OpenOffice files, .odt and .odp at client side using a web browser.

These files are zipped files. Using Ajax, I can get these files from server but these are zipped files. I have to unzip them using JavaScript, I have tried using inflate.js, http://www.onicos.com/staff/iz/amuse/javascript/expert/inflate.txt, but without success.

How can I do this?

Solution 1

I wrote an unzipper in Javascript. It works.

It relies on Andy G.P. Na's binary file reader and some RFC1951 inflate logic from notmasteryet. I added the ZipFile class.

working example:
http://cheeso.members.winisp.net/Unzip-Example.htm (dead link)

The source:
http://cheeso.members.winisp.net/srcview.aspx?dir=js-unzip (dead link)

NB: the links are dead; I'll find a new host soon.

Included in the source is a ZipFile.htm demonstration page, and 3 distinct scripts, one for the zipfile class, one for the inflate class, and one for a binary file reader class. The demo also depends on jQuery and jQuery UI. If you just download the js-zip.zip file, all of the necessary source is there.

---

Here's what the application code looks like in Javascript:

// In my demo, this gets attached to a click event.
// it instantiates a ZipFile, and provides a callback that is
// invoked when the zip is read.  This can take a few seconds on a
// large zip file, so it's asynchronous. 
var readFile = function(){
    $("#status").html("<br/>");
    var url= $("#urlToLoad").val();
    var doneReading = function(zip){
        extractEntries(zip);
    };

    var zipFile = new ZipFile(url, doneReading);
};


// this function extracts the entries from an instantiated zip
function extractEntries(zip){
    $('#report').accordion('destroy');

    // clear
    $("#report").html('');

    var extractCb = function(id) {
        // this callback is invoked with the entry name, and entry text
        // in my demo, the text is just injected into an accordion panel.
        return (function(entryName, entryText){
            var content = entryText.replace(new RegExp( "\\n", "g" ), "<br/>");
            $("#"+id).html(content);
            $("#status").append("extract cb, entry(" + entryName + ")  id(" + id + ")<br/>");
            $('#report').accordion('destroy');
            $('#report').accordion({collapsible:true, active:false});
        });
    }

    // for each entry in the zip, extract it. 
    for (var i=0; i<zip.entries.length;  i++) {
        var entry = zip.entries[i];

        var entryInfo = "<h4><a>" + entry.name + "</a></h4>\n<div>";

        // contrive an id for the entry, make it unique
        var randomId = "id-"+ Math.floor((Math.random() * 1000000000));

        entryInfo += "<span class='inputDiv'><h4>Content:</h4><span id='" + randomId +
            "'></span></span></div>\n";

        // insert the info for one entry as the last child within the report div
        $("#report").append(entryInfo);

        // extract asynchronously
        entry.extract(extractCb(randomId));
    }
}

---

The demo works in a couple of steps: The readFile fn is triggered by a click, and instantiates a ZipFile object, which reads the zip file. There's an asynchronous callback for when the read completes (usually happens in less than a second for reasonably sized zips) - in this demo the callback is held in the doneReading local variable, which simply calls extractEntries, which just blindly unzips all the content of the provided zip file. In a real app you would probably choose some of the entries to extract (allow the user to select, or choose one or more entries programmatically, etc).

The extractEntries fn iterates over all entries, and calls extract() on each one, passing a callback. Decompression of an entry takes time, maybe 1s or more for each entry in the zipfile, which means asynchrony is appropriate. The extract callback simply adds the extracted content to an jQuery accordion on the page. If the content is binary, then it gets formatted as such (not shown).

---

It works, but I think that the utility is somewhat limited.

For one thing: It's very slow. Takes ~4 seconds to unzip the 140k AppNote.txt file from PKWare. The same uncompress can be done in less than .5s in a .NET program. EDIT: The Javascript ZipFile unpacks considerably faster than this now, in IE9 and in Chrome. It is still slower than a compiled program, but it is plenty fast for normal browser usage.

For another: it does not do streaming. It basically slurps in the entire contents of the zipfile into memory. In a "real" programming environment you could read in only the metadata of a zip file (say, 64 bytes per entry) and then read and decompress the other data as desired. There's no way to do IO like that in javascript, as far as I know, therefore the only option is to read the entire zip into memory and do random access in it. This means it will place unreasonable demands on system memory for large zip files. Not so much a problem for a smaller zip file.

Also: It doesn't handle the "general case" zip file - there are lots of zip options that I didn't bother to implement in the unzipper - like ZIP encryption, WinZip encryption, zip64, UTF-8 encoded filenames, and so on. (EDIT - it handles UTF-8 encoded filenames now). The ZipFile class handles the basics, though. Some of these things would not be hard to implement. I have an AES encryption class in Javascript; that could be integrated to support encryption. Supporting Zip64 would probably useless for most users of Javascript, as it is intended to support >4gb zipfiles - don't need to extract those in a browser.

I also did not test the case for unzipping binary content. Right now it unzips text. If you have a zipped binary file, you'd need to edit the ZipFile class to handle it properly. I didn't figure out how to do that cleanly. It does binary files now, too.

---

EDIT - I updated the JS unzip library and demo. It now does binary files, in addition to text. I've made it more resilient and more general - you can now specify the encoding to use when reading text files. Also the demo is expanded - it shows unzipping an XLSX file in the browser, among other things.

So, while I think it is of limited utility and interest, it works. I guess it would work in Node.js.

Solution 2

I'm using zip.js and it seems to be quite useful. It's worth a look!

Check the Unzip demo, for example.

Solution 3

I found jszip quite useful. I've used so far only for reading, but they have create/edit capabilities as well.

Code wise it looks something like this

var new_zip = new JSZip();
new_zip.load(file);
new_zip.files["doc.xml"].asText() // this give you the text in the file

One thing I noticed is that it seems the file has to be in binary stream format (read using the .readAsArrayBuffer of FileReader(), otherwise I was getting errors saying I might have a corrupt zip file

Edit: Note from the 2.x to 3.0.0 upgrade guide:

The load() method and the constructor with data (new JSZip(data)) have been replaced by loadAsync().

Thanks user2677034

Solution 4

If you need to support other formats as well or just need good performance, you can use this WebAssembly library

it's promised based, it uses WebWorkers for threading and API is actually simple ES module

How to use

Install with npm i libarchive.js and use it as a ES module.

The library consists of two parts: ES module and webworker bundle, ES module part is your interface to talk to library, use it like any other module. The webworker bundle lives in the libarchive.js/dist folder so you need to make sure that it is available in your public folder since it will not get bundled if you're using bundler (it's all bundled up already) and specify correct path to Archive.init() method.

import {Archive} from 'libarchive.js/main.js';

Archive.init({
    workerUrl: 'libarchive.js/dist/worker-bundle.js'
});

document.getElementById('file').addEventListener('change', async (e) => {
    const file = e.currentTarget.files[0];

    const archive = await Archive.open(file);
    let obj = await archive.extractFiles();
    
    console.log(obj);
});

// outputs
{
    ".gitignore": {File},
    "addon": {
        "addon.py": {File},
        "addon.xml": {File}
    },
    "README.md": {File}
}

Solution 5

I wrote "Binary Tools for JavaScript", an open source project that includes the ability to unzip, unrar and untar: https://github.com/codedread/bitjs

Used in my comic book reader: https://github.com/codedread/kthoom (also open source).

HTH!

View more solutions

Comments

[OcuS]

"no success" please be more specific, show us some code, show us some errors... we're here to help, not to guess.

[user69260]

Basically I just called inflate function - data = zip_inflate(src); But I think this is meant for single file. If a zip files contain multiple files in a directory structure then what will be the content of "data". I do not know how to use this library.

[Dave Jarvis]

@techfandu: (1) Click on your name. (2) Click on a previous question you have asked. (3) Accept the answer that helped you the most. (4) Repeat until all your questions have an accepted answer.

[Giulio Prisco]

This looks great but I got this error message: This zipfile uses UTF8, which is not supported by ZipFile.js. Any quick workaround that you can recommend?

[Cheeso]

@Giulio - ok, I modified the ZipFile class to support decoding of UTF8-encoded filenames. It should just work now.

[SineSwiper]

Awesome! Can you add KMZ (Binary) and KML (XML) support to JSIO.guessFileType?

[Sachin J]

@Cheeso Where from u get ZipFile.js file??

[J punto Marcos]

@Cheeso All js-unzip links are dead. Is there any valid link to js-unzip?

[Cheeso]

ah yes, Sorry. I have the code, need to find a new home for it. thanks for the reminder.

[James P. Wright]

I would like to see your code too. Could you just put it somewhere like github?

[user969275]

i am using the zip.js the same your using but in safari i get the filereader is not defined.pls help me to work with safari.

[Dani bISHOP]

I don't have experience with Safari. You should ask to the zip.js developers. There is an email address at the bottom of the project page: gildas-lormeau.github.com/zip.js . Maybe it is a bug, so they will thank you for notifying.

[user969275]

Thanks for replying ,i have posted an issue .

[geocodezip]

I have an old version of one of the demos online, but I came here looking for updates. @Cheeso Would be interested in updated links when you have time.

[Danny Beckett]

You can get a free SVN repo at assembla.com and host portions publicly (like these demos).

[Cheeso]

@DannyBeckett - ok, thanks for the reminder and suggestion. I'll put the demo up somewhere soon.

[enigment]

I have JSON files with a base64 encoded JSON string in zip format inside them. I need that inner JSON object. Java's InflatorInputStream can unpack it on the server, so it actually is in zip format. However, when I pass the decoded base64 data from atob() to zip.js using the BlobReader, I get "Error while reading zip file." error. Visually the output from atob() is binary, so the BlobReader seems right, tried the TextReader anyway, it gives "File format is not recognized.". Any ideas?

[enigment]

Solved my problem in one line of code with pako pako.inflate(binaryData, { to: 'string' })

[Christian Gawron]

There is now also an npm package unzip-stream which works well with the Node.js streams API.

[user2677034]

This method has been removed in JSZip 3.0, please check the upgrade guide.

[Cody Gray]

Just got a flag claiming that the links are dead. Can you check this again, and put the code in your SO answer? You get 30,000 characters. If that's not enough, please post a second answer. It doesn't do any good for these links to keep going down.

[Peter Moore]

Why cant the code be in github instead of the dead links? Seems easier.

[Ajit Hogade]

most of the links are dead links.