download file client-side chunk by chunk

Question

I'm using WebRTC to send a file to a connected peer, and I'm sending the file in chunks. However, I'm having trouble figuring out how to get the peer to save/download the file as it is streaming in, chunk by chunk.

All the examples I've found online recommend doing something like this:

// sender
dataConnection.send({
   'file': file
});

// receiver
dataConnection.on('data', function(fileData) {

    var dataView = new Uint8Array(fileData);
    var dataBlob = new Blob([dataView]);
    var url = window.URL.createObjectURL(dataBlob);

    // create <a>
    var link = document.createElement('a');
    link.href = url;
    link.download = fileName;
    document.body.appendChild(link);

    // trigger the download file dialog
    link.click();
}

This approach, however, doesn't support getting chunks of the file and writing each chunk as it comes... it must wait for the entire file to be read on the sender's side and sent to the receiver.

What I'm trying to do is something like this:

// sender
for (var i = 0; i < fileSize; i += chunkSize) {

    var fileReader = new FileReader();

    // read next chunk
    var blob = file.slice(start, end);
    ...
    fileReader.onload = function(e) {
        ...
        dataConnection.send({ 'blob': blob });
    }
    fileReader.readAsArrayBuffer(blob);
}

This way, I'm reading the file chunk by chunk and sending each chunk to the receiver as I read it. Now, the only way I know how to actually save a file that is being sent in this way is by doing what is described in this blog post:

http://bloggeek.me/send-file-webrtc-data-api

... described in "Step 6: Downloading to regular FS". However, this approach takes all the chunks as they come in, stores them in memory, then builds a large UInt8Array in memory and then lets the receiver download the file. This is really heavy on memory and is realistically limited to maybe a couple hundred MB, so it doesn't scale.

Is there a way to open the file download dialog after the first chunk comes in, and keep writing the chunks as they come in so that the download is a "streaming" download?

Solution

UPDATE

Streams API: https://streams.spec.whatwg.org

https://jakearchibald.com/2016/streams-ftw

---

Unfortunately, from what I've researched, there is no way to open the file save / file download dialog and save/download a file in a "streaming" fashion.

The approach I will take is to use the FileSystem API. Unfortunately, this isn't fully supported by every browser:

• http://caniuse.com/#feat=filesystem

... and it doesn't seem likely that many browsers will adopt this API :(

• http://www.w3.org/TR/file-system-api/

  Work on this document has been discontinued and it should not be referenced or used as a basis for implementation

Comments

[guest271314]

See JavaScript: Writing to download stream

[guest271314]

See stackoverflow.com/questions/39959467/…