Python list directory, subdirectory, and files

Question

I'm trying to make a script to list all directory, subdirectory, and files in a given directory.
I tried this:

import sys, os

root = "/home/patate/directory/"
path = os.path.join(root, "targetdirectory")

for r, d, f in os.walk(path):
    for file in f:
        print(os.path.join(root, file))

Unfortunatly it doesn't work properly.
I get all the files, but not their complete paths.

For example if the dir struct would be:

/home/patate/directory/targetdirectory/123/456/789/file.txt

It would print:

/home/patate/directory/targetdirectory/file.txt

What I need is the first result. Any help would be greatly appreciated! Thanks.

Solution 1

Use os.path.join to concatenate the directory and file name:

for path, subdirs, files in os.walk(root):
    for name in files:
        print(os.path.join(path, name))

Note the usage of path and not root in the concatenation, since using root would be incorrect.

---

In Python 3.4, the pathlib module was added for easier path manipulations. So the equivalent to os.path.join would be:

pathlib.PurePath(path, name)

The advantage of pathlib is that you can use a variety of useful methods on paths. If you use the concrete Path variant you can also do actual OS calls through them, like changing into a directory, deleting the path, opening the file it points to and much more.

Solution 2

Just in case... Getting all files in the directory and subdirectories matching some pattern (*.py for example):

import os
from fnmatch import fnmatch

root = '/some/directory'
pattern = "*.py"

for path, subdirs, files in os.walk(root):
    for name in files:
        if fnmatch(name, pattern):
            print(os.path.join(path, name))

Solution 3

Couldn't comment so writing answer here. This is the clearest one-line I have seen:

import os
[os.path.join(path, name) for path, subdirs, files in os.walk(root) for name in files]

Solution 4

Here is a one-liner:

import os

[val for sublist in [[os.path.join(i[0], j) for j in i[2]] for i in os.walk('./')] for val in sublist]
# Meta comment to ease selecting text

The outer most val for sublist in ... loop flattens the list to be one dimensional. The j loop collects a list of every file basename and joins it to the current path. Finally, the i loop iterates over all directories and sub directories.

This example uses the hard-coded path ./ in the os.walk(...) call, you can supplement any path string you like.

Note: os.path.expanduser and/or os.path.expandvars can be used for paths strings like ~/

Extending this example:

Its easy to add in file basename tests and directoryname tests.

For Example, testing for *.jpg files:

... for j in i[2] if j.endswith('.jpg')] ...

Additionally, excluding the .git directory:

... for i in os.walk('./') if '.git' not in i[0].split('/')]

Solution 5

A bit simpler one-liner:

import os
from itertools import product, chain

chain.from_iterable([[os.sep.join(w) for w in product([i[0]], i[2])] for i in os.walk(dir)])

View more solutions

Comments

[Roman Rdgz]

It does work, but to excluve .git directoy you need to check if '.git' is NOT into the path.

[Roman Rdgz]

Yep. Should be if '.git' not in i[0].split('/')]

[ThorSummoner]

I would recommend os.walk over a manual dirlisting loop, generators are great, go use them.

[harrisonfooord]

this is the one and only useful answer for the many questions that have been asked concerning "how to get all files recursively in python".

[Nir]

comprehension list: all_files = [os.path.join(path, name) for name in files for path, subdirs, files in os.walk(folder)]

[Ehsan]

In Python3 use parenthesis for print function print(os.path.join(path, name))

[Ahmad Ismail]

In Python3 use parenthesis for print function print(os.path.join(path, name)). You can also use print(pathlib.PurePath(path, name)).

[Aakash Gupta]

how do I list each file ?

[Matt]

this is the answer for all you googlers