java.util.zip - Recreating directory structure

Question

While trying to zip an archive using the java.util.zip I ran into a lot of problems most of which I solved. Now that I finally get some output I struggle with getting the "right" output. I have an extracted ODT file (directory would be more fitting a description) to which I did some modifications. Now I want to compress that directory as to recreate the ODT file structure. Zipping the directory and renaming it to end with .odt works fine so there should be no problem.

The main problem is that I lose the internal structure of the directory. Everything becomes "flat" and I do not seem to find a way to preserve the original multi-layered structure. I would appreciate some help on this as I can not seem to find the problem.

Here are the relevant code snippets:

ZipOutputStream out = new ZipOutputStream(new FileOutputStream(
    FILEPATH.substring(0, FILEPATH.lastIndexOf(SEPARATOR) + 1).concat("test.zip")));
    compressDirectory(TEMPARCH, out);

The SEPARATOR is the system file separator and the FILEPATH is the filepath of the original ODT which I will override but have not done here for testing purposes. I simply write to a test.zip file in the same directory.

private void compressDirectory(String directory, ZipOutputStream out) throws IOException
{
    File fileToCompress = new File(directory);
    // list contents.
    String[] contents = fileToCompress.list();
    // iterate through directory and compress files.
    for(int i = 0; i < contents.length; i++)
    {
        File f = new File(directory, contents[i]);
        // testing type. directories and files have to be treated separately.
        if(f.isDirectory())
        {
            // add empty directory
            out.putNextEntry(new ZipEntry(f.getName() + SEPARATOR));
            // initiate recursive call
            compressDirectory(f.getPath(), out);
            // continue the iteration
            continue;
        }else{
             // prepare stream to read file.
             FileInputStream in = new FileInputStream(f);
             // create ZipEntry and add to outputting stream.
             out.putNextEntry(new ZipEntry(f.getName()));
             // write the data.
             int len;
             while((len = in.read(data)) > 0)
             {
                 out.write(data, 0, len);
             }
             out.flush();
             out.closeEntry();
             in.close();
         }
     }
 }

The directory that contains the files to zip is somewhere in the user space and not in the same directory as the resulting file. I assume this could be trouble but I can not really see how. Also I figured that the problem could be in using the same stream for outputting but again I can not see how. I saw in some examples and tutorials that they use getPath() instead of getName() but changing that gives me an empty zip file.

Solution 1

The URI class is useful for working with relative paths.

File mydir = new File("C:\\mydir");
File myfile = new File("C:\\mydir\\path\\myfile.txt");
System.out.println(mydir.toURI().relativize(myfile.toURI()).getPath());

The above code will emit the string path/myfile.txt.

For completeness, here is a zip method for archiving a directory:

  public static void zip(File directory, File zipfile) throws IOException {
    URI base = directory.toURI();
    Deque<File> queue = new LinkedList<File>();
    queue.push(directory);
    OutputStream out = new FileOutputStream(zipfile);
    Closeable res = out;
    try {
      ZipOutputStream zout = new ZipOutputStream(out);
      res = zout;
      while (!queue.isEmpty()) {
        directory = queue.pop();
        for (File kid : directory.listFiles()) {
          String name = base.relativize(kid.toURI()).getPath();
          if (kid.isDirectory()) {
            queue.push(kid);
            name = name.endsWith("/") ? name : name + "/";
            zout.putNextEntry(new ZipEntry(name));
          } else {
            zout.putNextEntry(new ZipEntry(name));
            copy(kid, zout);
            zout.closeEntry();
          }
        }
      }
    } finally {
      res.close();
    }
  }

This code makes doesn't preserve dates and I'm not sure how it would react to stuff like symlinks. No attempt is made to add directory entries, so empty directories would not be included.

The corresponding unzip command:

  public static void unzip(File zipfile, File directory) throws IOException {
    ZipFile zfile = new ZipFile(zipfile);
    Enumeration<? extends ZipEntry> entries = zfile.entries();
    while (entries.hasMoreElements()) {
      ZipEntry entry = entries.nextElement();
      File file = new File(directory, entry.getName());
      if (entry.isDirectory()) {
        file.mkdirs();
      } else {
        file.getParentFile().mkdirs();
        InputStream in = zfile.getInputStream(entry);
        try {
          copy(in, file);
        } finally {
          in.close();
        }
      }
    }
  }

Utility methods on which they rely:

  private static void copy(InputStream in, OutputStream out) throws IOException {
    byte[] buffer = new byte[1024];
    while (true) {
      int readCount = in.read(buffer);
      if (readCount < 0) {
        break;
      }
      out.write(buffer, 0, readCount);
    }
  }

  private static void copy(File file, OutputStream out) throws IOException {
    InputStream in = new FileInputStream(file);
    try {
      copy(in, out);
    } finally {
      in.close();
    }
  }

  private static void copy(InputStream in, File file) throws IOException {
    OutputStream out = new FileOutputStream(file);
    try {
      copy(in, out);
    } finally {
      out.close();
    }
  }

The buffer size is entirely arbitrary.

Solution 2

I see 2 problems in your code,

1. You don't save the directory path so there is no way to get it back.
2. On Windows, you need to use "/" as path separator. Some unzip program doesn't like \.

I include my own version for your reference. We use this one to zip up photos to download so it works with various unzip programs. It preserves the directory structure and timestamps.

  public static void createZipFile(File srcDir, OutputStream out,
   boolean verbose) throws IOException {

  List<String> fileList = listDirectory(srcDir);
  ZipOutputStream zout = new ZipOutputStream(out);

  zout.setLevel(9);
  zout.setComment("Zipper v1.2");

  for (String fileName : fileList) {
   File file = new File(srcDir.getParent(), fileName);
   if (verbose)
    System.out.println("  adding: " + fileName);

   // Zip always use / as separator
   String zipName = fileName;
   if (File.separatorChar != '/')
    zipName = fileName.replace(File.separatorChar, '/');
   ZipEntry ze;
   if (file.isFile()) {
    ze = new ZipEntry(zipName);
    ze.setTime(file.lastModified());
    zout.putNextEntry(ze);
    FileInputStream fin = new FileInputStream(file);
    byte[] buffer = new byte[4096];
    for (int n; (n = fin.read(buffer)) > 0;)
     zout.write(buffer, 0, n);
    fin.close();
   } else {
    ze = new ZipEntry(zipName + '/');
    ze.setTime(file.lastModified());
    zout.putNextEntry(ze);
   }
  }
  zout.close();
 }

 public static List<String> listDirectory(File directory)
   throws IOException {

  Stack<String> stack = new Stack<String>();
  List<String> list = new ArrayList<String>();

  // If it's a file, just return itself
  if (directory.isFile()) {
   if (directory.canRead())
    list.add(directory.getName());
   return list;
  }

  // Traverse the directory in width-first manner, no-recursively
  String root = directory.getParent();
  stack.push(directory.getName());
  while (!stack.empty()) {
   String current = (String) stack.pop();
   File curDir = new File(root, current);
   String[] fileList = curDir.list();
   if (fileList != null) {
    for (String entry : fileList) {
     File f = new File(curDir, entry);
     if (f.isFile()) {
      if (f.canRead()) {
       list.add(current + File.separator + entry);
      } else {
       System.err.println("File " + f.getPath()
         + " is unreadable");
       throw new IOException("Can't read file: "
         + f.getPath());
      }
     } else if (f.isDirectory()) {
      list.add(current + File.separator + entry);
      stack.push(current + File.separator + f.getName());
     } else {
      throw new IOException("Unknown entry: " + f.getPath());
     }
    }
   }
  }
  return list;
 }
}

Solution 3

Just go through the source of java.util.zip.ZipEntry. It treats a ZipEntry as directory if its name ends with "/" characters. Just suffix the directory name with "/". Also you need to remove the drive prefix to make it relative.

Check this example for zipping just the empty directories,

http://bethecoder.com/applications/tutorials/showTutorials.action?tutorialId=Java_ZipUtilities_ZipEmptyDirectory

As long as you are able to create both empty & non-empty directories in ZIP file, your directory structure is intact.

Good luck.

Solution 4

Here is another example (recursive) which also lets you include/exclude the containing folder form the zip:

import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;

public class ZipUtil {

  private static final int DEFAULT_BUFFER_SIZE = 1024 * 4;

  public static void main(String[] args) throws Exception {
    zipFile("C:/tmp/demo", "C:/tmp/demo.zip", true);
  }

  public static void zipFile(String fileToZip, String zipFile, boolean excludeContainingFolder)
    throws IOException {        
    ZipOutputStream zipOut = new ZipOutputStream(new FileOutputStream(zipFile));    

    File srcFile = new File(fileToZip);
    if(excludeContainingFolder && srcFile.isDirectory()) {
      for(String fileName : srcFile.list()) {
        addToZip("", fileToZip + "/" + fileName, zipOut);
      }
    } else {
      addToZip("", fileToZip, zipOut);
    }

    zipOut.flush();
    zipOut.close();

    System.out.println("Successfully created " + zipFile);
  }

  private static void addToZip(String path, String srcFile, ZipOutputStream zipOut)
    throws IOException {        
    File file = new File(srcFile);
    String filePath = "".equals(path) ? file.getName() : path + "/" + file.getName();
    if (file.isDirectory()) {
      for (String fileName : file.list()) {             
        addToZip(filePath, srcFile + "/" + fileName, zipOut);
      }
    } else {
      zipOut.putNextEntry(new ZipEntry(filePath));
      FileInputStream in = new FileInputStream(srcFile);

      byte[] buffer = new byte[DEFAULT_BUFFER_SIZE];
      int len;
      while ((len = in.read(buffer)) != -1) {
        zipOut.write(buffer, 0, len);
      }

      in.close();
    }
  }
}

Solution 5

If you don't want to bother dealing with byte input streams, buffer sizes, and other low level details. You can use Ant's Zip libraries from your java code (maven dependencies can be found here). Here's now I make a zip consisting a list of files & directories:

public static void createZip(File zipFile, List<String> fileList) {

    Project project = new Project();
    project.init();

    Zip zip = new Zip();
    zip.setDestFile(zipFile);
    zip.setProject(project);

    for(String relativePath : fileList) {

        //noramalize the path (using commons-io, might want to null-check)
        String normalizedPath = FilenameUtils.normalize(relativePath);

        //create the file that will be used
        File fileToZip = new File(normalizedPath);
        if(fileToZip.isDirectory()) {
            ZipFileSet fileSet = new ZipFileSet();
            fileSet.setDir(fileToZip);
            fileSet.setPrefix(fileToZip.getPath());
            zip.addFileset(fileSet);
        } else {
            FileSet fileSet = new FileSet();
            fileSet.setDir(new File("."));
            fileSet.setIncludes(normalizedPath);
            zip.addFileset(fileSet);
        }
    }

    Target target = new Target();
    target.setName("ziptarget");
    target.addTask(zip);
    project.addTarget(target);
    project.executeTarget("ziptarget");
}

View more solutions

Comments

[Eric Tobias]

Thank you very much! Alas I can not see how the original directory format is kept with your compression code. In the ODT I use there are empty directories. As far as I understand your code, those directories will never be created. Am I perhaps missing something?

[McDowell]

Directories are empty entries with a name that ends with /. I've altered the code.

[Eric Tobias]

Thank you for your contribution. I am grateful for your code example but I am not sure how it will help me find the error in my code as; the initial call to the function is with the full directory path which will then be handed down in the function and secondly; my SEPARATOR constant is initialised with the System.getProperty("file.separator") which will give me the OS default file separator. I would never hardcode a separator since that assumes that your code will only be deployed on a given OS.

[Eric Tobias]

I adapted the structure of your code and abandoned the recursive calls. I think it was the wrong way to look at this. The code runs smoothly with one exception; it adds empty folders to most of the child directories even if they are not empty. I found that removing the following line solves the problem: name = name.endsWith("/") ? name : name + "/"; I suspect that when adding a directory by appending the "\" one also creates an empty folder inside. By simply letting the ZipEntries take care of the structure building, everything seems fine. Thank you all for your help!

[ZZ Coder]

Don't use File.separator in ZIP. The separator must be "/" according to the spec. If you are on Windows, you must open file as "D:\dir\subdir\file" but ZIP entry must be "dir/subdir/file".

[Eric Tobias]

I see. Thank you for pointing this out. Did not know that ZIP was so picky! :)

[Axel Fontaine]

Potential NPE: directory.listFiles() will return null if the directory is empty!

[McDowell]

@AxelFontaine It'll return an empty directory if the directory is empty - doc - but there are conditions under which null can be returned so it would probably be best to throw an IOException in this case.

[ojblass]

Thank you so much for saving me time!