Listing of all files in directory?

Question

Can anybody help me create a function which will create a list of all files under a certain directory by using pathlib library?

Here, I have a:

I have

• c:\desktop\test\A\A.txt

• c:\desktop\test\B\B_1\B.txt

• c:\desktop\test\123.txt

I expected to have a single list which would have the paths above, but my code returns a nested list.

Here is my code:

from pathlib import Path

def searching_all_files(directory: Path):   
    file_list = [] # A list for storing files existing in directories

    for x in directory.iterdir():
        if x.is_file():

           file_list.append(x)
        else:

           file_list.append(searching_all_files(directory/x))

    return file_list


p = Path('C:\\Users\\akrio\\Desktop\\Test')

print(searching_all_files(p))

Hope anybody could correct me.

Solution 1

Use Path.glob() to list all files and directories. And then filter it in a List Comprehensions.

p = Path(r'C:\Users\akrio\Desktop\Test').glob('**/*')
files = [x for x in p if x.is_file()]

More from the pathlib module:

• pathlib, part of the standard library.
• Python 3's pathlib Module: Taming the File System

Solution 2

from pathlib import Path
from pprint import pprint

def searching_all_files(directory):
    dirpath = Path(directory)
    assert dirpath.is_dir()
    file_list = []
    for x in dirpath.iterdir():
        if x.is_file():
            file_list.append(x)
        elif x.is_dir():
            file_list.extend(searching_all_files(x))
    return file_list

pprint(searching_all_files('.'))

Solution 3

With pathlib, it is as simple as the below comand.

path = Path('C:\\Users\\akrio\\Desktop\\Test')    
list(path.iterdir())

Solution 4

If you can assume that only file objects have a . in the name (i.e., .txt, .png, etc.) you can do a glob or recursive glob search...

from pathlib import Path

# Search the directory
list(Path('testDir').glob('*.*'))

# Search directories and subdirectories, recursively
list(Path('testDir').rglob('*.*'))

But that's not always the case. Sometimes there are hidden directories like .ipynb_checkpoints and files that do not have extensions. In that case, use list comprehension or a filter to sort out the Path objects that are files.

# Search Single Directory
list(filter(lambda x: x.is_file(), Path('testDir').iterdir()))

# Search Directories Recursively
list(filter(lambda x: x.is_file(), Path('testDir').rglob('*')))

# Search Single Directory
[x for x in Path('testDir').iterdir() if x.is_file()]

# Search Directories Recursively
[x for x in Path('testDir').rglob('*') if x.is_file()]

Solution 5

A similar, more functional-oriented solution to @prasastoadi's one can be achieved by using the built-in filter function of Python:

from pathlib import Path

my_path = Path(r'C:\Users\akrio\Desktop\Test')
list(filter(Path.is_file, my_path.glob('**/*')))

View more solutions

Comments

[Julien]

use extend instead of append in the second case?

[Akrios]

The requirment is to use pathlib. Thank you anyway

[PirateNinjas]

Your answer is correct, but you're making use of both OS and pathlib. I think it would be better if you used just pathlib

[Vineet Sharma]

@PirateNinjas Thanks for the suggestion. I have made the change.

[PatrickT]

assert is a statement, not a function, so I think you want assert dirpath.is_dir() with no parenthesis. In Python 2 and 3. Or simply assert dirpath.exists()

[Charlie Parker]

what if I want to list all directories in a directory?

[Jonas]

To list all directories simply replace "x.is_file()" with "x.is_dir()" as described in the docs

[Nico Schlömer]

pathlib2 is deprecated.

[xmcp]

I didn't see how pathlib2 is related to this question. It seems that pathlib2 is only a backport of pathlib (to Python 2.x) and therefore path.iterdir() in pathlib2 cannot recursively walk the directory.

[Povilas]

Need only list of files (not dirs)? one liner: [f for f in Path(path_to_dir).iterdir() if f.is_file()]