How to open every file in a folder

Question

I have a python script parse.py, which in the script open a file, say file1, and then do something maybe print out the total number of characters.

filename = 'file1'
f = open(filename, 'r')
content = f.read()
print filename, len(content)

Right now, I am using stdout to direct the result to my output file - output

python parse.py >> output

However, I don't want to do this file by file manually, is there a way to take care of every single file automatically? Like

ls | awk '{print}' | python parse.py >> output 

Then the problem is how could I read the file name from standardin? or there are already some built-in functions to do the ls and those kind of work easily?

Thanks!

Solution 1

Os

You can list all files in the current directory using os.listdir:

import os
for filename in os.listdir(os.getcwd()):
   with open(os.path.join(os.getcwd(), filename), 'r') as f: # open in readonly mode
      # do your stuff

Glob

Or you can list only some files, depending on the file pattern using the glob module:

import os, glob
for filename in glob.glob('*.txt'):
   with open(os.path.join(os.getcwd(), filename), 'r') as f: # open in readonly mode
      # do your stuff

It doesn't have to be the current directory you can list them in any path you want:

import os, glob
path = '/some/path/to/file'
for filename in glob.glob(os.path.join(path, '*.txt')):
   with open(os.path.join(os.getcwd(), filename), 'r') as f: # open in readonly mode
      # do your stuff

Pipe

Or you can even use the pipe as you specified using fileinput

import fileinput
for line in fileinput.input():
    # do your stuff

And you can then use it with piping:

ls -1 | python parse.py

Solution 2

You should try using os.walk.

import os

yourpath = 'path'

for root, dirs, files in os.walk(yourpath, topdown=False):
    for name in files:
        print(os.path.join(root, name))
        stuff
    for name in dirs:
        print(os.path.join(root, name))
        stuff

Solution 3

I was looking for this answer:

import os,glob
folder_path = '/some/path/to/file'
for filename in glob.glob(os.path.join(folder_path, '*.htm')):
  with open(filename, 'r') as f:
    text = f.read()
    print (filename)
    print (len(text))

you can choose as well '*.txt' or other ends of your filename

Solution 4

You can actually just use os module to do both:

1. list all files in a folder
2. sort files by file type, file name etc.

Here's a simple example:

import os #os module imported here
location = os.getcwd() # get present working directory location here
counter = 0 #keep a count of all files found
csvfiles = [] #list to store all csv files found at location
filebeginwithhello = [] # list to keep all files that begin with 'hello'
otherfiles = [] #list to keep any other file that do not match the criteria

for file in os.listdir(location):
    try:
        if file.endswith(".csv"):
            print "csv file found:\t", file
            csvfiles.append(str(file))
            counter = counter+1

        elif file.startswith("hello") and file.endswith(".csv"): #because some files may start with hello and also be a csv file
            print "csv file found:\t", file
            csvfiles.append(str(file))
            counter = counter+1

        elif file.startswith("hello"):
            print "hello files found: \t", file
            filebeginwithhello.append(file)
            counter = counter+1

        else:
            otherfiles.append(file)
            counter = counter+1
    except Exception as e:
        raise e
        print "No files found here!"

print "Total files found:\t", counter

---

Now you have not only listed all the files in a folder but also have them (optionally) sorted by starting name, file type and others. Just now iterate over each list and do your stuff.

Solution 5

import pyautogui
import keyboard
import time
import os
import pyperclip

os.chdir("target directory")

# get the current directory
cwd=os.getcwd()

files=[]

for i in os.walk(cwd):
    for j in i[2]:
        files.append(os.path.abspath(j))

os.startfile("C:\Program Files (x86)\Adobe\Acrobat 11.0\Acrobat\Acrobat.exe")
time.sleep(1)


for i in files:
    print(i)
    pyperclip.copy(i)
    keyboard.press('ctrl')
    keyboard.press_and_release('o')
    keyboard.release('ctrl')
    time.sleep(1)

    keyboard.press('ctrl')
    keyboard.press_and_release('v')
    keyboard.release('ctrl')
    time.sleep(1)
    keyboard.press_and_release('enter')
    keyboard.press('ctrl')
    keyboard.press_and_release('p')
    keyboard.release('ctrl')
    keyboard.press_and_release('enter')
    time.sleep(3)
    keyboard.press('ctrl')
    keyboard.press_and_release('w')
    keyboard.release('ctrl')
    pyperclip.copy('')

View more solutions

Comments

[Charlie Parker]

does this handle the file opening and closing automatically too? I'm surprised ur not using with ... as ...: statements. Could you clarify?

[David R]

Charlie, glob.glob and os.listdir return the filenames. You would then open those one by one within the loop.

[RockwellS]

This opens, prints, closes every PDF in a directory using PyPerClip and PyAutoGui. Hope others find this helpful.

[Khan]

this is the answer because you are reading all files in a directory ;D