Passing arguments with wildcards to a Python script

Question

I want to do something like this:

c:\data\> python myscript.py *.csv

and pass all of the .csv files in the directory to my python script (such that sys.argv contains ["file1.csv", "file2.csv"], etc.)

But sys.argv just receives ["*.csv"] indicating that the wildcard was not expanded, so this doesn't work.

I feel like there is a simple way to do this, but can't find it on Google. Any ideas?

Solution 1

You can use the glob module, that way you won't depend on the behavior of a particular shell (well, you still depend on the shell not expanding the arguments, but at least you can get this to happen in Unix by escaping the wildcards :-) ).

from glob import glob
filelist = glob('*.csv') #You can pass the sys.argv argument

Solution 2

In Unix, the shell expands wildcards, so programs get the expanded list of filenames. Windows doesn't do this: the shell passes the wildcards directly to the program, which has to expand them itself.

Vinko is right: the glob module does the job:

import glob, sys

for arg in glob.glob(sys.argv[1]):
    print "Arg:", arg

Comments

[user1066101]

It's not "can"; for windows, it's "must".

[Vinko Vrsalovic]

Well, you can also use os.walk so it's not strictly a must :P

[user1066101]

@Vinko Vrsalovic: true. os.walk seems more cumbersome than glob. Glob's not required, but it's so perfect a fit for this problem.

[Kiv]

Thanks, exactly what I was after.

[1.01pm]

Just what I was looking for :)

[Ryan Thames]

Does anyone know why Windows shells don't handle this for you?

[Vinko Vrsalovic]

@Ryan: Because DOS utilities were built by people who didn't know UNIX and decided to let each utility to handle the expansion instead of making the shell smarter. Awful design, but that's just one among many :-)

[Florian Brucker]

+1. Note that this also works perfectly fine if sys.argv[1] is a fully specified filename instead of a wildcard. In that case, glob.glob just returns a list containing that very filename.

[Stefatronik]

also works if you forward a parameter to your Python script script.py c:\path\*\subdir glob generates a list that I can use later on. Great module!