Creating a zip on the fly with csv files inside

Question

I'm trying to create a zip file on the fly containing a bunch of csv files to return from a servlet and its very confusing. A little guidance would be great. Here are chunks of code I have that somehow need to work together:

// output stream coming from httpResponse, thats all fine
ZipOutputStream zip = new ZipOutputStream(outputStream);


// using the openCSV library to create the csv file
CSVWriter writer = new CSVWriter(Writer?); 
// what writer do I use? I want to write to memory, not a file

writer.writeNext(entries); 
writer.close();

// at this point should I have the csv file in memory somewhere? 
//and then try to copy it into the zip file?

int length;
byte[] buffer = new byte[1024 * 32];    
zip.putNextEntry(new ZipEntry(getClass() + ".csv"));

// the 'in' doesn't exist yet - where am I getting the input stream from?
while((length = in.read(buffer)) != -1)
    zip.write(buffer, 0, length);

zip.closeEntry();
zip.flush();

Solution

You can stream the ZIP file containing your CSVs as follows :

try {
    OutputStream servletOutputStream = httpServletResponse.getOutputStream(); // retrieve OutputStream from HttpServletResponse
    ZipOutputStream zos = new ZipOutputStream(servletOutputStream); // create a ZipOutputStream from servletOutputStream

    List<String[]> csvFileContents  = getContentToZIP(); // get the list of csv contents. I am assuming the CSV content is generated programmatically
    int count = 0;
    for (String[] entries : csvFileContents) {
        String filename = "file-" + ++count  + ".csv";
        ZipEntry entry = new ZipEntry(filename); // create a zip entry and add it to ZipOutputStream
        zos.putNextEntry(entry);

        CSVWriter writer = new CSVWriter(new OutputStreamWriter(zos));  // There is no need for staging the CSV on filesystem or reading bytes into memory. Directly write bytes to the output stream.
        writer.writeNext(entries);  // write the contents
        writer.flush(); // flush the writer. Very important!
        zos.closeEntry(); // close the entry. Note : we are not closing the zos just yet as we need to add more files to our ZIP
    }

    zos.close(); // finally closing the ZipOutputStream to mark completion of ZIP file
} catch (Exception e) {
    log.error(e); // handle error
}

Comments

[ant-depalma]

Sure but is it ok that I am never calling writer.close anywhere?

[nadirsaghar]

Yes it is. The writer is writing bytes directly to the outputstream of the servlet. If you notice you are working with the same stream throughout the code. You only need to close it once when you are done writing. We are doing that later in the code just before the catch block

[Tan]

@nadirsaghar Could you tell me from where did you get ` getContentToZIP();` method? I couldn't find it online.

[NobesInd]

How much time did it took to generate a zip of say 30mb?( having multiple csv inside)

[ahiijny]

In case anyone runs into the same problem I did: I was using the javacsv library instead of opencsv for this one, and unlike opencsv, the CsvWriter class in javacsv includes a call to close() in its finalize() method, so when CsvWriter gets garbage collected, it can result in it closing the ZipOutputStream prematurely, which would cause a "Stream closed" error.