How to handle file upload chunks on the server (Plupload/Spring MVC)?

Question

On our client, it is possible to be uploading large files. We want to use chunking to reduce the size of the requests. We are using Plupload, so it's easy to send up the files in chunks. However, I'm not sure how to handle the chunks. We are using Spring MVC on the server, and I current have the following for my controller method: uploadMedia(@RequestBody MultipartFile file) where MultipartFile is org.springframework.web.multipart.MultipartFile. This works just fine when not doing chunking. When I turn on the chunking on the client side, it still gets into this method just fine, but I don't see anything on the file that gets passed in that identifies what file the chunk is a part of. Hopefully I'm just missing something.

It seems like this is a common workflow, but I can't seem to find any good examples of how this is done on the server side. A solution with just Spring would be great, but if another library is needed, that is fine too. I looked some at Apache Commons FileUpload, but I couldn't find anything about chunking on there. Any help with this would be great. Thanks.

Solution

I debugged through this some more and added a HttpServletRequest to my controller method to see if there was anything available in there. I found that the request I received was a org.springframework.web.multipart.support.DefaultMultipartHttpServletRequest which has a field called multipartParameters which is just a HashMap. I found that this map had keys of name, chunks, and chunk. So, I decided to try the following as my signature:

uploadMedia(@RequestBody MultipartFile file, @RequestParam String name, @RequestParam int chunks, @RequestParam int chunk)

Sure enough, those @RequestParam parameters got populated with the name of the file, the number of chunks, and the chunk number (zero-based) respectively. Having this information with each request will make it pretty easy to assemble the chunks into the final file.

One thing to know also, is that if the file does not need to be chunked (the file size is less than the chunk size), the chunks and chunk parameters are not sent up. So, my final signature ended up looking like this:

uploadMedia(@RequestBody MultipartFile file, @RequestParam String name, @RequestParam(required=false, defaultValue="-1") int chunks, @RequestParam(required=false, defaultValue="-1") int chunk)

Then I can check for -1 to see if I need to worry about chunking at all.

So, in the controller method itself I have something like this:

Media media = new Media();
//set stuff on my Media object for storing info about the file in the DB
//....
if (chunks > 0 && chunk > 0)
{
    //Need to append the bytes in this chunk
    mediaRepository.appendBytes(media, file.getBytes());
    if (chunk == chunks - 1)
    {
        //last chunk, upload is done
        onUploadFinished(media);
    }
}
else
{
    //Write out the first set of bytes
    mediaRepository.saveBytes(media, file.getBytes());
    if (chunks <= 0)
    {
        //no chunks were needed, all the bytes have been written out, upload is done
        onUploadFinished(media);
    }
}

Comments

[zmanc]

is it possible to see more of the controller? I am trying to implement the same functionality and am struggling the the rest of the controller.

[dnc253]

I've updated the answer to include more of the controller code.

[zmanc]

if you don't mind can you share a portion of your mediaRepository?

[dnc253]

It's just using the java.nio.file.Files package in Java 7 (docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html)‌​. So, the methods above are pretty much one-liners. Files.write(getFilePath(media), bytes, StandardOpenOption.CREATE); and Files.write(getFilePath(media), bytes, StandardOpenOption.APPEND);

[ammianus]

Are you writing the chunks of content to a temporary file before sending to the DB?

[dnc253]

I'm not saving the bytes in the DB, just writing them to disk. There's no reason the saveBytes and appendBytes couldn't write to a DB though. What those methods do is outside the scope of this question. This question and answer shows how to get the necessary info from Plupload, and then you can do whatever you need to with that information.