How can I generate zip file without saving to the disk with Ruby?

Question

I have generated many PDF files in memory and I want to compress them into one zip file before sending it as a email attachment. I have looked at Rubyzip and it does not allows me to create a zip file without saving it to disk (maybe I am wrong).

Is there any way I can compress those file without creating a temp file?

Solution 1

I had a similar problem which I solved using the rubyzip gem and the stringio object. It turns out that rubyzip provides a method that returns a stringio object: ZipOutputStream.write_buffer.

You can create the zip file structure as you like using put_next_entry and write and once you are finished you can rewind the stringio and read the binary data using sysread.

See the following simple example (works for rubyzip 0.9.X)

require 'zip/zip'
stringio = Zip::OutputStream.write_buffer do |zio|
  zio.put_next_entry("test.txt")
  zio.write "Hello world!"
end
stringio.rewind
binary_data = stringio.sysread

Tested on jruby 1.6.5.1 (ruby-1.9.2-p136) (2011-12-27 1bf37c2) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_29) [Windows Server 2008-amd64-java])

The following example works for rubyzip >= 1.0.0

require 'rubygems'    
require 'zip'
stringio = Zip::OutputStream.write_buffer do |zio|
  zio.put_next_entry("test.txt")
  zio.write "Hello world!"
end
binary_data = stringio.string

Tested on jruby 1.7.22 (1.9.3p551) 2015-08-20 c28f492 on OpenJDK 64-Bit Server VM 1.7.0_79-b14 +jit [linux-amd64] and rubyzip gem 1.1.7

Solution 2

Ruby comes with a very convenient StringIO library - this can be used for using a String as output IO object or faking reading a file backed by a String.

The challenge here is that RubyZip does not support directly taking an IO object when creating a Zip::ZipOutputStream, but if you look at the implementation of the initialize, and depending on your willingness to experiment, you may be able to extend the class and allow it to take either an IO object or a file name in the constructor.

Solution 3

There are two RubyZip libraries that I was able to find.

1. Chilkat's Ruby Zip Library
2. rubyzip on Sourceforge

Chilkat's library definitely allows one to create a zip file in memory instead of writing it to disk automatically as seen in these links: Zip to Memory, Zip from in memory data

The one on SourceForge, on the other hand, may provide an option of zipping a file in memory but I'm not entirely certain since I'm very new to ruby. The SourceForge rubyzip is based on java.util.zip which has led to it having a class called ZipOutputStream. I don't know how good the rubyzip implementation is, but with java.util.zip implementation the OutputStream can be set to ByteArrayOutputStream, FileOutputStream, FilterOutputStream, ObjectOutputStream, OutputStream, PipedOutputStream....

If that holds true for the rubyzip implementation then it should be a matter of using ZipOutputStream to pass in a ByteArrayOutputStream of sorts which would result in it being output to memory.

If it doesn't exist in rubyzip, then I'm sure you could always write your own implementation and submit it for inclusion in rubyzip seeing as it is opensource.

Comments

[maxam]

The one I was referring was the gem: rubygems.org/gems/rubyzip I don't want to use the Chilkat library since there it's not open source.

[dimitarvp]

I won't downrate this, but I didn't find anywhere a comment like "no, the standard Ruby ZIP classes can NOT do this". I believe you should have started with it.

[maxam]

In fact the ZipOutputStream new method takes a file name as argument, I didn't find any way to pass an IO object.

[maxam]

dimitko, there are no standard ruby library for zip. I have checked rubyzip but it seems that there is nothing for doing what I want.

[user3953201]

@flutedemetan, like I said above, if it doesn't exist then you'll either need to use another library or make your own addition to the rubyzip open source project.

[maxam]

Thanks, it works. However at the time that I wrote the question, the method was not existing yet. It was added in 2011-01-07. Thanks for your answer and next time that I need to zip in memory, I will know how.

[Arnold Roa]

LoadError cannot load such file -- zip

[vas]

Above works with older versions of rubyzip gem. Version 1.0 changed the interface. Please see: github.com/rubyzip/rubyzip#important-note

[rkabir]

write_buffer is now Zip::OutputStream::write_buffer

[3coins]

Also, noticed that with the newer version require 'zip/zip' does not work. require 'zip' is the correct statement.

[Paul Danelli]

OMG stringio.string was exactly what i was missing. Thank you for this.

[K M Rakibul Islam]

This example works! Tested withe Ruby 2.6.6! +1 :-)

[El-Cortez]

Hello, can anyone help, I'm unable to make this code work :( I'm trying to put a file from S3 inside the zip, but it does not work ` s3_file = s3_client.get_object(bucket: bucket_name, key: s3_filename) file = s3_file.body stringio = Zip::OutputStream.write_buffer do |zio| zio.put_next_entry("test1.zip") zf = Zip::File.new(file, create: true, buffer: true) zio.write(file) end ` I get the following error message TypeError (no implicit conversion of StringIO into String)