How can I create a zip file without temporary files in ruby?

Question

I create a zip file with some generated content (in other words: the files in the archive don't exist, the content is build in my script).

I use a script similar to this:

#~ gem 'rubyzip', '=1.1.0'
require 'zip/zip'

zipname = 'test.zip'
File.delete(zipname) if File.exists?(zipname) #delete previous version

Zip::ZipFile.open(zipname, Zip::ZipFile::CREATE) do |zipfile|
  1.upto(100) do |i| #Just some testfiles with content
    zipfile.get_output_stream("%08i.txt" % i) do |output_entry_stream| #Filename
          output_entry_stream.write("Testcontent %08i" % i)            #generated content
        end
      sleep 1  #sleep some time to see the temporary files
  end   #testdocs
end #ZipFile.open(zipname)

This works fine, I get my zip with the correct data inside.

But during the zip creation I have a lot of temporary files. They are deleted when the zip is finished, but the files disturb me during the creation. And if the process raises an exception, then I have to delete them manual.

I have this behaviour with Zip::VERSION 2.0.2 and 1.1.0 (using the gem rubyzip)

• Can I avoid this temporary files?
• If not: Can I determine a (temporary) folder for them?

Solution 1

See the answer at "How can I generate zip file without saving to the disk with Ruby?"

I adapted your example to demonstrate it works.

require 'zip/zip'

zipname = 'test.zip'
File.delete(zipname) if File.exists?(zipname) #delete previous version

stringio = Zip::ZipOutputStream::write_buffer do |zio|
  1.upto(5) do |i| #Just some testfiles with content
    zio.put_next_entry("test#{i}.txt") #Filename
    zio.write("Testcontent %08i" % i)  #generated content
    sleep 1 #sleep some time to see the temporary files
  end
end
stringio.rewind #reposition buffer pointer to the beginning
File.new("test.zip","wb").write(stringio.sysread) #write buffer to zipfile

Solution 2

Do you need zip compatibility, or is gzip sufficient? Ruby comes with the Zlib module, which makes it easy to read/write gzip compressed files using Zlib::GzipWriter and Zlib::GzipWriter respectively:

require 'zlib'
Zlib::GzipWriter.open('foobar.gz') do |fo|
  fo.write "how now brown cow\n"
end

At the command-line:

$file foobar.gz
foobar.gz: gzip compressed data, from Unix, last modified: Fri May 30 08:56:58 2014
$gunzip foobar.gz
$cat foobar
how now brown cow

If you need to create archives containing multiple files, Ruby's Archive::Tar class can help. tar.gz files are smaller than the equivalent .zip files according to Wikipedia due to differences in how Zip and tar -> gZip work.

Be very careful trying to buffer in memory when tarring or compressing files. It's easy to test in development because we rarely use full-size files, but once code hits production, and the file sizes balloon to astronomical sizes, all of a sudden your system can run into a memory crunch. We routinely deal with files well into the GB-range, and trying to compress those in memory would make an app a bad citizen on a production server. I don't like getting calls late at night just to hear something I wrote was causing problems, so be conservative and sure that your application can handle its task without failures.