Recursively list files in Java

Question

How do I recursively list all files under a directory in Java? Does the framework provide any utility?

I saw a lot of hacky implementations. But none from the framework or nio

Solution 1

Java 8 provides a nice stream to process all files in a tree.

Files.walk(Paths.get(path))
        .filter(Files::isRegularFile)
        .forEach(System.out::println);

This provides a natural way to traverse files. Since it's a stream you can do all nice stream operations on the result such as limit, grouping, mapping, exit early etc.

UPDATE: I might point out there is also Files.find which takes a BiPredicate that could be more efficient if you need to check file attributes.

Files.find(Paths.get(path),
           Integer.MAX_VALUE,
           (filePath, fileAttr) -> fileAttr.isRegularFile())
        .forEach(System.out::println);

Note that while the JavaDoc eludes that this method could be more efficient than Files.walk it is effectively identical, the difference in performance can be observed if you are also retrieving file attributes within your filter. In the end, if you need to filter on attributes use Files.find, otherwise use Files.walk, mostly because there are overloads and it's more convenient.

TESTS: As requested I've provided a performance comparison of many of the answers. Check out the Github project which contains results and a test case.

Solution 2

FileUtils have iterateFiles and listFiles methods. Give them a try. (from commons-io)

Edit: You can check here for a benchmark of different approaches. It seems that the commons-io approach is slow, so pick some of the faster ones from here (if it matters)

Solution 3

// Ready to run

import java.io.File;

public class Filewalker {

    public void walk( String path ) {

        File root = new File( path );
        File[] list = root.listFiles();

        if (list == null) return;

        for ( File f : list ) {
            if ( f.isDirectory() ) {
                walk( f.getAbsolutePath() );
                System.out.println( "Dir:" + f.getAbsoluteFile() );
            }
            else {
                System.out.println( "File:" + f.getAbsoluteFile() );
            }
        }
    }

    public static void main(String[] args) {
        Filewalker fw = new Filewalker();
        fw.walk("c:\\" );
    }

}

Solution 4

Java 7 will have has Files.walkFileTree:

If you provide a starting point and a file visitor, it will invoke various methods on the file visitor as it walks through the file in the file tree. We expect people to use this if they are developing a recursive copy, a recursive move, a recursive delete, or a recursive operation that sets permissions or performs another operation on each of the files.

There is now an entire Oracle tutorial on this question.

Solution 5

No external libraries needed.
Returns a Collection so you can do whatever you want with it after the call.

public static Collection<File> listFileTree(File dir) {
    Set<File> fileTree = new HashSet<File>();
    if(dir==null||dir.listFiles()==null){
        return fileTree;
    }
    for (File entry : dir.listFiles()) {
        if (entry.isFile()) fileTree.add(entry);
        else fileTree.addAll(listFileTree(entry));
    }
    return fileTree;
}

View more solutions

Comments

[helios]

Please! let the caller initialize the files list so it hasn't have to check its nullity each time. If you want create a second (public) method that creates the list, calls this internal method and returns the complete list.

[pstanton]

whatever. a null check isn't very expensive, convenience + personal preference aside i think he'll get the point.

[hfs]

From the documentation of listFiles(): “If this abstract pathname does not denote a directory, then this method returns null.”

[andronikus]

FYI/TLDR: if you just want to list all files recursively with no filtering, do FileUtils.listFiles(dir, TrueFileFilter.INSTANCE, TrueFileFilter.INSTANCE), where dir is a File object that points to the base directory.

[uday]

Can you explain little bit more verbosely?

[Wei]

But your algorithm cannot print with indented output. Dirs and files are messed. Any solution?

[Ben]

Improved variant public static Collection<File> listFileTree(File dir) { if (null == dir || !dir.isDirectory()) { return Collections.emptyList(); } final Set<File> fileTree = new HashSet<File>(); for (File entry : dir.listFiles()) { if (entry.isFile()) { fileTree.add(entry); } else { fileTree.addAll(listFileTree(entry)); } } return fileTree; }

[schnatterer]

You might want to consider using listFilesAndDirs(), as listFiles() does not return empty folders.

[ocramot]

@MikeFHay Looking at the FileUtils code, I think that vould be FileUtils.listFiles(dir, true, true). using FileUtils.listFiles(dir, null, true) will throw an Exception, while FileUtils.listFiles(dir, true, null) will list all files without looking into subdirectories.

[Christian Bongiorno]

How about a JDK native library? I can implement this easy but I would simply be C&P from other places

[Johnny]

One of those examples that can show the magic of functional programming even for the beginners.

[Samuel Kerrien]

While it probably works, the question is about file browsing, not rendering of browsed files. Better expose your algorithm as such, it is not a recommended practice to embed business logic inside a JSP.

[Nux]

That depends what you are doing. In an enterprise-size application you are absolutely right. If you just need this as a drop-in to a simple, standalone listing, then this is perfectly fine.

[Sridhar Sarnobat]

How does the performance of this compare with pre-java 8 methods? My current directory traversal is too slow and I'm looking for something that will speed it up.

[Brett Ryan]

Just beware that for symbolic links that point to a path higher in the path hierarchy will cause the method to never end. Consider a path with a symlink that points to -> ..

[Brett Ryan]

I'm writing up some tests containing most of the variants in the answers provided. So far, it seems that using Files.walk with a parallel stream is the best, followed closely by Files.walkFileTree which is only marginally slower. The accepted answer using commons-io is by far the slowest by my tests to be 4 times slower.

[Brett Ryan]

I'm putting some tests together, but so far this seems to be performing 4 times slower than that of using JDK8 or JDK7 alternatives. Symlinks also prove to be problematic with this approach, especially where they link to directories higher in the tree, this causes the method to never return, this can be avoided by handling the filter, but unfortunately the symlinks themselves don't get visited even as a file.

[Brett Ryan]

Tests are complete and a project added to Github, check it out for the results. The short answer is use any NIO flavour of the answers provided.

[Kachna]

@BrettRyan, I tried your solution but I get an exception Exception in thread "main" java.io.UncheckedIOException: java.nio.file.AccessDeniedException. How could I correct it

[thouliha]

How do I get an actual list of files from this?

[Brett Ryan]

@thouliha instead of forEach use collect with a set or list collector such as .collect(Collectors.toList()). If you can avoid a collector in your logic the better as you will be using less objects and processing can be faster.

[jFrenetic]

Wow, can't get my eyes off of this beautiful code. Who would've thought that one day Java would allow to implement such operations in one line.

[Matt Passell]

@BrettRyan am I correct that by default Files.walk() doesn't follow symbolic links? If so and one wanted to follow symlinks, it looks like the first chunk of code could instead be Files.walk(Paths.get(path), FileVisitOption.FOLLOW_LINKS)

[Brett Ryan]

Apologies for the delayed response @MattPassell, you are correct. There is a caveat to using this when a circular link is found, you won't end up with a recursive loop though you unfortunately will get a FileSystemLoopException being thrown which is wrapped in an UncheckedIOException

[Matt Passell]

@BrettRyan thanks, good point. In the particular spot I'm using it, the starting path might be a symlink, but from that point down, it isn't. That allowed me to do something like this Path basePath = Files.isSymbolicLink(basePath) ? Files.readSymbolicLink(basePath) : basePath; and then walk the tree without following links. :)

[Alkanshel]

@BrettRyan "instead of forEach use collect" Sorry but I'm still unclear. Calling .collect(stagingFileList::add) on the resulting stream from that code gets me Collector<Path,A,R> is not a functional interface on the collect parameter. This stream stuff is cool but I'm not completely sure what I'm looking at anymore to fix this.

[Alkanshel]

This is what I'm attempting-- Files.find( Paths.get(path), Integer.MAX_VALUE, (filePath, fileAttr) -> fileAttr.isRegularFile()) .collect(list::add);

[Brett Ryan]

@Amalgovinus what you're looking for is a collector, what you've given is a method reference. Replace list::add with Collectors.toList()

[Jonathan Hult]

Watch out for this issue: bugs.openjdk.java.net/browse/JDK-8039910 More info here: stackoverflow.com/questions/22867286/…

[WillieT]

To me this is the most concise answer that is recursive.

[Tyler Nichols]

This is just essentially a bad implementation of Files.walkFileTree. I would reccomend that people look at FIles.walkFileTree instead of trying to roll it yourself... It has handling for the exact problem @BrettRyan pointed it.

[razor]

Files.walk looks nice, but quite often is useless.... it throws exception if you don't have an access to even one file... and you got nothing...

[barrypicker]

Thank you for including import java.io.File;. So many examples forget to include the namespace stuff or even datatype stuff making the example a starting point on a voyage of discovery. Here this example is ready-to-run. Thanks.

[Vahid Pazirandeh]

@BrettRyan - What does it mean for a dir to be "open", thus requiring it to be closed? The javadoc says to put walk() and find() into a try-with-resources block. I've never heard of the concept of an "open dir"

[Moses Kirathe]

Path may vary depending on where the Filewalker file is. Use "/","./" or "../" for root directory, current working directory, and parent directory, respectively

[Farid]

And it never notifies end of walk.

[Ginger McMurray]

"Use recursion" isn't helpful. There should be an example of how.

[joseph]

You must put Files.walk in a try-with-resources block since it's backed by DirectoryStreams (docs.oracle.com/javase/8/docs/api/java/nio/file/… ) which says: "Failure to close the stream may result in a resource leak. The try-with-resources statement provides a useful construct to ensure that the stream is closed"

[joseph]

Please see stackoverflow.com/a/69489309/1810962 for a more complete answer.