Trying to use open(filename, 'w' ) gives IOError: [Errno 2] No such file or directory if directory doesn't exist

Question

I am trying to create and write to a text file using Python. I have searched and cannot find a solution/reason for this error.

Here's the code that doesn't work:

afile = 'D:\\temp\\test.txt'
outFile = open(afile, 'w' )
outFile.write('Test.')
outFile.close()

# Error: 2
# Traceback (most recent call last):
#   File "<maya console>", line 1, in <module>
# IOError: [Errno 2] No such file or directory: 'D:\\temp\\test.txt' #

Most answers I found related to the slashes in the path, so...

I tried 'D:/temp/test.txt' and got an error.
I tried r'D:\temp\test.txt' and got an error.

When I try to create a file at the root of D:/ I have success.

'D:/test.txt' works.
'D:\\test.txt' works.
r'D:\test.txt' works.

It seems that I can't create the directory path I would like while trying to create the file. What is the correct method for creating files at a specific path with Python on Windows(7)? Am I misunderstanding what open() can do? Does it create directories if they don't exist or do I need to explicitly create the directory path before I use open() in 'write' mode to create a file?

Solution 1

You are correct in surmising that the parent directory for the file must exist in order for open to succeed. The simple way to deal with this is to make a call to os.makedirs.

From the documentation:

os.makedirs(path[, mode])

Recursive directory creation function. Like mkdir(), but makes all intermediate-level directories needed to contain the leaf directory.

So your code might run something like this:

filename = ...
dirname = os.path.dirname(filename)
if not os.path.exists(dirname):
    os.makedirs(dirname)
with open(filename, 'w'):
    ...

Solution 2

If you try to create a file in a directory that doesn't exist, you will get that error.

You need to ensure the directory exists first. You can do that with os.makedirs() as per this answer.

Comments

[abarnert]

When you're dealing with the filesystem, EAFP is always better than LBYL, because otherwise there are huge openings for race conditions. Just try the makedirs, and handle the EEXIST by doing nothing (or, if you prefer, you can even try the open, and handle the ENOENT by trying makedirs…).

[Burhan Khalid]

If you are really looking for a temporary file, tempfile might be more appropriate.

[abarnert]

@BurhanKhalid: Definitely… but if you want a specific directory instead of the default (e.g., you need to guarantee a directory with no spaces in it), tempfile.NamedTemporaryFile (and mkstemp and so on) doesn't create directories either, so he could have the same problem anyway.

[abarnert]

By the way, an even better solution is to update to a more modern Python. In 3.2, you can just call os.makedirs(dir, exist_ok=True). In 3.1, you can at least catch FileExistsError instead of catching OSError and checking the errno.

[David Heffernan]

@abarnert Using EAFP rather than LBYL changes nothing regarding race conditions. There's a race no matter what between the call to makedirs and that to open. And there's always a race inside makedirs. I agree that testing for existence is crappy. But so is catching the exception. Python 3.2 and exist_ok is the right approach, but question says 2.7.

[davejagoda]

Using dir as a variable name is not ideal - this has the side effect of overwriting the dir() builtin.