python: Change the scripts working directory to the script's own directory

Question

I run a python shell from crontab every minute:

* * * * * /home/udi/foo/bar.py

/home/udi/foo has some necessary subdirectories, like /home/udi/foo/log and /home/udi/foo/config, which /home/udi/foo/bar.py refers to.

The problem is that crontab runs the script from a different working directory, so trying to open ./log/bar.log fails.

Is there a nice way to tell the script to change the working directory to the script's own directory? I would fancy a solution that would work for any script location, rather than explicitly telling the script where it is.

EDIT:

os.chdir(os.path.dirname(sys.argv[0]))

Was the most compact elegant solution. Thanks for your answers and explanations!

Solution 1

This will change your current working directory to so that opening relative paths will work:

import os
os.chdir("/home/udi/foo")

However, you asked how to change into whatever directory your Python script is located, even if you don't know what directory that will be when you're writing your script. To do this, you can use the os.path functions:

import os

abspath = os.path.abspath(__file__)
dname = os.path.dirname(abspath)
os.chdir(dname)

This takes the filename of your script, converts it to an absolute path, then extracts the directory of that path, then changes into that directory.

Solution 2

You can get a shorter version by using sys.path[0].

os.chdir(sys.path[0])

From http://docs.python.org/library/sys.html#sys.path

As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter

Solution 3

Don't do this.

Your scripts and your data should not be mashed into one big directory. Put your code in some known location (site-packages or /var/opt/udi or something) separate from your data. Use good version control on your code to be sure that you have current and previous versions separated from each other so you can fall back to previous versions and test future versions.

Bottom line: Do not mingle code and data.

Data is precious. Code comes and goes.

Provide the working directory as a command-line argument value. You can provide a default as an environment variable. Don't deduce it (or guess at it)

Make it a required argument value and do this.

import sys
import os
working= os.environ.get("WORKING_DIRECTORY","/some/default")
if len(sys.argv) > 1: working = sys.argv[1]
os.chdir( working )

Do not "assume" a directory based on the location of your software. It will not work out well in the long run.

Solution 4

Change your crontab command to

* * * * * (cd /home/udi/foo/ || exit 1; ./bar.py)

The (...) starts a sub-shell that your crond executes as a single command. The || exit 1 causes your cronjob to fail in case that the directory is unavailable.

Though the other solutions may be more elegant in the long run for your specific scripts, my example could still be useful in cases where you can't modify the program or command that you want to execute.

View more solutions

Comments

[jfs]

unrelated to crontab use-case: both sys.argv[0] and __file__ fail if script is run using execfile(); inspect-based solution could be used instead.

[Ikke]

Equals hardcoding the directory.

[Eli Courtwright]

I added info on how to change into the script's directory; I posted a partial answer while I looked up the os.path functions, which I couldn't remember off the top of my head.

[Stefano Borini]

S. Lott is right. Always keep data and code separated, unless data is not transitory. For example, if you have icons, that is data, but it's not transitory, and it makes sense to consider it relative to the software bundle (whatever that means)

[user1066101]

@Udi Pasmon: Not far-fetched at all. It's the "small maintenance scripts" that get organizations into deep trouble. Years from now, this "small maintenance script" and it's children and derivations and data files will be a nightmare to disentangle and reimplement. Keep data as far from code as possible -- pass parameters for everything -- assume nothing.

[Chris Down]

If you are running it from a symlink, this will not work. Use __file__ instead of sys.argv[0].

[Iain Samuel McLean Elder]

+1 I thought I wanted to do as the OP, but after reading your advice I instead modified my script. Now it requires a parameter to specify the location of a log file.

[Colonel Panic]

Why the abspath step? Why not simply os.chdir(os.path.dirname(__file__))?

[Eli Courtwright]

@ColonelPanic: In most cases you won't need an absolute path, but it will help in some edge cases such as if some other code already changed the working directory, or if your script is being run without a working directory at all like in a cron job or something.

[jfs]

__file__ fails in "frozen" programs (created using py2exe, PyInstaller, cx_Freeze). sys.argv[0] works. @ChrisDown: If you want to follow symlinks; os.path.realpath() could be used.

[jfs]

+1. It is easier to create a package (rpm) for a python script if data directories can be customized easily.

[user313114]

I'm using this to pull in configuration files that are shared with a different script in a nearby directory.

[Bruno Bronosky]

@user313114 should be using an entrypoint. github.com/RichardBronosky/entrypoint_demo

[Bruno Bronosky]

This is an extremely sound solution. I usually find myself editing other peoples answers to add things like the || exit 1. It's refreshing to see this. Although I do have to wonder why you wouldn't just do cd /home/udi/foo/ && ./bar.py

[Ruud Althuizen]

@BrunoBronosky With the explicit exit 1 your crond will be notified of an error, and in most cases will send an email notification of the failure.

[Arthur Tacca]

@EliCourtwright If __file__ is not already an absolute path, and the user has changed the working directory, then os.path.abspath will fail anyway.

[fx-kirin]

One liner is better in this case.os.chdir(os.path.dirname(os.path.abspath(__file__)))

[Basj]

Ready-to-copy-paste line that helps for some scripts: import os; os.chdir(os.path.dirname(os.path.abspath(__file__)))

[marco]

Be aware that this does not work if you import git. The first path will point to <pythondir>/lib/site-packages/git/ext/gitdb. At least this happens to me

[xverges]

Thanks, @marco. I just created a PR for this github.com/gitpython-developers/GitPython/pull/1068

[Kukuster]

@ChrisDown dang! I've actullay given up this solution because file doesn't work for symlinks for me.

[amit kumar]

@RuudAlthuizen cd /home/udi/foo/ && ./bar.py will do the same.