How do I import a module given the full path?

Question

How do I load a Python module given its full path?

Note that the file can be anywhere in the filesystem.

Solution 1

For Python 3.5+ use (docs):

import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/path/to/file.py")
foo = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = foo
spec.loader.exec_module(foo)
foo.MyClass()

For Python 3.3 and 3.4 use:

from importlib.machinery import SourceFileLoader

foo = SourceFileLoader("module.name", "/path/to/file.py").load_module()
foo.MyClass()

(Although this has been deprecated in Python 3.4.)

For Python 2 use:

import imp

foo = imp.load_source('module.name', '/path/to/file.py')
foo.MyClass()

There are equivalent convenience functions for compiled Python files and DLLs.

See also http://bugs.python.org/issue21436.

Solution 2

The advantage of adding a path to sys.path (over using imp) is that it simplifies things when importing more than one module from a single package. For example:

import sys
# the mock-0.3.1 dir contains testcase.py, testutils.py & mock.py
sys.path.append('/foo/bar/mock-0.3.1')

from testcase import TestCase
from testutils import RunTests
from mock import Mock, sentinel, patch

Solution 3

To import your module, you need to add its directory to the environment variable, either temporarily or permanently.

Temporarily

import sys
sys.path.append("/path/to/my/modules/")
import my_module

Permanently

Adding the following line to your .bashrc (or alternative) file in Linux and excecute source ~/.bashrc (or alternative) in the terminal:

export PYTHONPATH="${PYTHONPATH}:/path/to/my/modules/"

Credit/Source: saarrrr, another Stack Exchange question

Solution 4

If your top-level module is not a file but is packaged as a directory with __init__.py, then the accepted solution almost works, but not quite. In Python 3.5+ the following code is needed (note the added line that begins with 'sys.modules'):

MODULE_PATH = "/path/to/your/module/__init__.py"
MODULE_NAME = "mymodule"
import importlib
import sys
spec = importlib.util.spec_from_file_location(MODULE_NAME, MODULE_PATH)
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module 
spec.loader.exec_module(module)

Without this line, when exec_module is executed, it tries to bind relative imports in your top level __init__.py to the top level module name -- in this case "mymodule". But "mymodule" isn't loaded yet so you'll get the error "SystemError: Parent module 'mymodule' not loaded, cannot perform relative import". So you need to bind the name before you load it. The reason for this is the fundamental invariant of the relative import system: "The invariant holding is that if you have sys.modules['spam'] and sys.modules['spam.foo'] (as you would after the above import), the latter must appear as the foo attribute of the former" as discussed here.

Solution 5

It sounds like you don't want to specifically import the configuration file (which has a whole lot of side effects and additional complications involved). You just want to run it, and be able to access the resulting namespace. The standard library provides an API specifically for that in the form of runpy.run_path:

from runpy import run_path
settings = run_path("/path/to/file.py")

That interface is available in Python 2.7 and Python 3.2+.

View more solutions

Comments

[inger]

Nice and simple question - and useful answers but they make me wonder what happened with the python mantra "There is one obvious way" to do it.. It doesn't seem like anything like a single or a simple and obvious answer to it.. Seems ridiculously hacky and version-dependent for such a fundamental operation (and it looks and more bloated in newer versions..).

[John Frazer]

@inger what happened with the python mantra "There is one obvious way" to do it [...] [not] a single or a simple and obvious answer to it [...] ridiculously hacky[...] more bloated in newer versions Welcome to the terrible world of python package management. Python's import, virtualenv, pip, setuptools whatnot should all be thrown out and replaced with working code. I just tried to grok virtualenv or was it pipenv and had to work thru the equivalent of a Jumbo Jet manual. How that contrivance is paraded as The Solution to dealing with deps totally escapes me.

[John Frazer]

relevant XKCD xkcd.com/1987

[Błażej Michalik]

@JohnFrazer it's been made worse by constant nagging of people who couldn't be bothered to read 2 paragraphs of documentation. Your XKCD isn't really relevant, as it shows what these kinds of people can achieve when trying things until something works. Also, just because there's a new way doesn't mean there's now "two obvious ways". The old way is obvious for some cases, the new way introduces ease of use to other. That's what happens when you actually care about DevX.

[Alex]

And think that Java or even PHP (these days) have clear and simple way of splitting things in packages/namespaces and reuse it. It's a shock to see such pain in Python which adopted simplicity in every other aspect.

[user202729]

There's parsing - What's the best practice using a settings file in Python? - Stack Overflow -- but I don't find a good way to use Python configuration file.

[Mark K Cowan]

How I miss the obvious and intuitive semantics JS "require" and C/C++ "#include" when working with Python.

[Sridhar Ratnakumar]

If I knew the namespace - 'module.name' - I would already use __import__.

[Dan D.]

@SridharRatnakumar the value of the first argument of imp.load_source only sets the .__name__ of the returned module. it doesn't effect loading.

[user618677]

@Wheat Why sys.path[0:0] instead of sys.path[0]?

[ychaouche]

os.chdir ? (minimal characters to approve comment).

[dom0]

B/c sys.path[0] = xy overwrites the first path item while path[0:0] =xy is equivalent to path.insert(0, xy)

[ReneSac]

A more general way to cut the extension out is: name, ext = os.path.splitext(os.path.basename(infile)). Your method works because the previous restriction to .py extension. Also, you should probably import the module to some variable/dictionary entry.

[Brandon Rhodes]

@DanD. — the first argument of imp.load_source() determines the key of the new entry created in the sys.modules dictionary, so the first argument does indeed affect loading.

[Chris Johnson]

Why write 14 lines of buggy code when this is already addressed by the standard library? You haven't done error checking on format or content of full_path_to_module or the os.whatever operations; and using a catch-all except: clause is rarely a good idea.

[jsh]

hm the path.insert worked for me but the [0:0] trick did not.

[Phani]

How do we use sys.path.append to point to a single python file instead of a directory?

[Srivatsan]

@Daryl Spitzer: How do we do it for just one python file

[kay]

You should use more "try-finally"s in here. E.g. save_cwd = os.getcwd() try: … finally: os.chdir(save_cwd)

[capybaralet]

Is the idea that this is replicating: ""import module.name as foo"", if your working directory was /path/to ?? Else, what is foo here??

[JenNic]

How do we use this for a single python file? And why have you been ignoring us for 7 years?

[Daryl Spitzer]

:-) Perhaps your question would be better suited as a StackOverflow question, not a comment on an answer.

[Kevin Edwards]

sys.path[0:0] = ['/foo']

[m33k]

To all people who were trying to include a file to their path... by definition "the shell path is a colon delimited list of directories". I'm relatively new to python, but the python path also follows the unix design principle from what I have seen. Please correct me if I am wrong.

[alexis]

The python path can contain zip archives, "eggs" (a complex kind of zip archives), etc. Modules can be imported out of them. So the path elements are indeed containers of files, but they are not necessarily directories.

[Sushi271]

@ChrisJohnson this is already addressed by the standard library yeah, but python has nasty habit of not being backward-compatible... as the checked answer says there're 2 different ways before and after 3.3. In that case I'd rather like to write my own universal function than check version on the fly. And yes, maybe this code isn't too well error-protected, but it shows an idea (which is os.chdir(), I haven't though about it), basing on which I can write a better code. Hence +1.

[Gripp]

This is the hack I used, and it has caused me some problems. One of the more painful ones was that IDEA has an issue where it doesn't pickup altered code from within the link, but yet attempts to save what it thinks is there. A race condition where the last to save is what sticks... I lost a decent amount of work because of this.

[ComFreek]

Beware of the fact that Python caches import statements. In the rare case that you have two different folders sharing a single class name (classX), the approach of adding a path to sys.path, importing classX, removing the path and repeating for the reamaining paths won't work. Python will always load the class from the first path from its cache. In my case I aimed at creating a plugin system where all plugins implement a specific classX. I ended up using SourceFileLoader, note that its deprecation is controversial.

[d9k]

See new (for december 2015) message in bug discussion (bugs.python.org/issue21436#msg255901): there a third new three-lines-long way to load module in python 3.5!

[HEKTO]

... and imp.load_dynamic(module_name, path_to_file) for DLLs

[Ryne Everett]

I'm not seeing that SourceFileLoader is deprecated in the 3.4+ docs.

[Sebastian Rittau]

SourceFileLoader is not deprecated, but SourceFileLoader.load_module() is.

[t1m0]

heads up that imp is deprecated now.

[ncoghlan]

Despite the use of "import" in the question, you probably want to use runpy.run_path for execution of Python code as a configuration format, rather than dynamically manipulating the import system.

[rocky]

@AXO and more to the point one wonders why something as simple and basic as this has to be so complicated. It isn't in many many other languages.

[sorin]

If you want a version of the code that works in all versions of Python check stackoverflow.com/a/37611448/99834

[Paolo Celati]

I'm in Python 3.5.2 and I've found it only works if the extension of the file is .py

[ZigZag]

@Paolo Celati In Python 3.5+ You should use importlib.import_module(module_name). Some like this. sys,path.append(path_to_file) module = importlib.import_module(module_name)

[Mahesha999]

may I know why importlib.import_module is not mentioned here?

[Sebastian Rittau]

@Mahesha999 Because importlib.import_module() does not allow you to import modules by filename, which is what the original question was about.

[Pedro Cattori]

For Python 3.5+, if /path/to/file.py imports a sibling implicitly (e.g. import bar to import /path/to/bar.py), the solution yields ModuleNotFoundError: No module named 'bar'. Any way to fix this?

[Pedro Cattori]

^I ended up asking this in a separate StackOverflow Question: stackoverflow.com/q/41861427/1490091

[AlexLordThorsen]

importlib.util.spec_from_file_location won't import files that don't end in .py =(

[user5359531]

@Gripp not sure if I am understanding your issue, but I frequently (almost exclusively) edit my scripts on a remote server from my desktop via SFTP with a client like CyberDuck, and in that case as well it is a bad idea to try and edit the symlinked file, instead its much safer to edit the original file. You can catch some of these issues by using git and checking your git status to verify that your changes to the script are actually making it back to the source document and not getting lost in the ether.

[Klik]

This answer worked for me where load_source did not because it imports the script and provides the script access to the modules and globals at the time of importing.

[Cole Robertson]

I'm running python 3.6.3 on Sierra 10.12.6 and using option 1 (for python 3.5+). The code works, but when I run the line foo.MyClass() I get the error AttributeError: module 'myFileName' has no attribute 'MyClass', where myFileName is the name of the python file I pass to the first arg of importlib.util.spec_from_file_location("module.name", "/path/to/file.py"). Yet when I comment out the line foo.MyClass() the script executes the imported script without issue. Would someone please explain what foo.MyClass() is doing in the suggested code?

[Sebastian Rittau]

@ColeRobertson That line is just an example to show that you need to prefix any access of the module with foo. (or however you call that variable).

[JacksonHaenchen]

It's mentioned that for Python 2, importlib should be used instead of lib, yet I see no example of using importlib to import a module at a path. Anyone have such an example?

[setholopolus]

Is that top one safe? It seems a bit too close to just "exec"ing whatever the user tells you too. Wouldn't it be safer to add a directory to sys.path, and then ask for the specific module that you want?

[Sebastian Rittau]

@setholopolus None of those is safe, if you use untrusted user input.

[setholopolus]

@SebastianRittau You're right I guess, because even if you imported a specific module by name they could have replaced it by their own module with the same name.

[ybull]

I prefer this solution to the accepted one because it still works on Python 3.6 and because it only requires 2 simple lines of code to allow any number of modules to be found within another site directory. Our specific situation was on a web server where only a few core packages are installed centrally and this way CGI scripts created by users can import from their user-site directory. Thanks!

[Micah Smith]

This solution does not allow you to provide the path, which is what the question asks for.

[Nathan]

@SebastianRittau Then what is currently the best way for python 3.4?

[Sam Grondahl]

This almost worked for me but when I'm importing a module structured as a directory with an _init_.py I needed an additional line. See my answer below -- hope it helps somebody!

[mforbes]

As mentioned by @SamGrondahl, these fail if the module has relative imports. His answer provides a solution for python 3.5+: Is there any similar solution for python 2.7?

[bgusach]

This way you are modifying the sys.path, which does not fit every use case.

[Ataxias]

@bgusach This may be true, but it is also desirable in some cases (adding a path to sys.path simplifies things when importing more than one module from a single package). At any rate, if this not desirable, one can immediately afterwards do sys.path.pop()

[Stephen Ellwood]

I like this method but when I get the result of run_path its a dictionary which I cannot seem to access?

[ncoghlan]

What do you mean by "cannot access"? You can't import from it (that's why this is only a good option when import-style access isn't actually required), but the contents should be available via the regular dict API (result[name], result.get('name', default_value), etc)

[fordy]

This "temp" solution is a great answer if you want to prod a project around in a jupyter notebook elsewhere.

[frakman1]

I've spent all day troubleshooting an import bug in a pyinstaller generated exe. In the end this is the only thing that worked for me. Thank you so much for making this!

[Janaka Bandara]

In my case the imported file had other (transitive) relative-path imports; combining the accepted answer (importlib) with this worked for me.

[Eli_B]

Note this approach allows the imported module to import other modules from the same dir, which modules often do, while the accepted answer's approach does not (at least on 3.7). importlib.import_module(mod_name) can be used instead of the explicit import here if the module name isn't known at runtime I would add a sys.path.pop() in the end, though, assuming the imported code doesn't try to import more modules as it is used.

[winklerrr]

Explicit is better than implicit. So why not sys.path.insert(0, ...) instead of sys.path[0:0]?

[Lena]

Should spec.loader.exec_module(foo) also ensure that the script is completely run? I have a set-up where I import a list of a variables (and purposely not a class), that all need to be valid and active within the script from which I'm calling it. In contrast to the regular import statement, however, my variables cannot be further recalled.

[Sebastian Rittau]

@Lena It should and in my short test it did.

[smerllo]

This one should be on top !

[Andry]

Beware of periods in a file name, because the latest version of the python importlib module still could not handle module file names with periods: bugs.python.org/issue38000

[Marcin Wojnarski]

Interesting how the code gets LONGER and longer with every new version of Python. Not sure if this is really the "pythonic way" of developing a programming language :/ (same for other things, like print() etc.)

[tebanep]

Thanks a lot! This method enables relative imports between submodules. Great!

[Shai Alon]

Python 2.7: TypeError: load_module() takes exactly 4 arguments

[Shai Alon]

But... it's dangerous tampering with the path

[Shai Alon]

That is not dynamically.

[Shai Alon]

I tried: config_file = 'setup-for-chats', setup_file = get_setup_file(config_file + ".py"), sys.path.append(os.path.dirname(os.path.expanduser(setup_fil‌​e))), import config_file >> "ImportError: No module named config_file"

[Miladiouss]

@ShaiAlon You are adding paths, so no danger other than when you transfer codes from one computer to another, paths might get messed up. So, for package development, I only import local packages. Also, package names should be unique. If you are worried, use the temporary solution.

[Guimoute]

@dom0 Just go with sys.path.append(...) then. It's clearer.

[Marco Castanho]

If the imported file contains any relative import these solutions fail ImportError: attempted relative import with no known parent package

[Tim Ludwinski]

This answer matches the documentation here: docs.python.org/3/library/….

[Gulzar]

but what is mymodule?

[Gulzar]

module.name is the module from which I run?

[Idodo]

@Gulzar, it is whatever name you'd like to give your module, such that you can later do: "from mymodule import myclass"

[Gulzar]

So... /path/to/your/module/ is actually /path/to/your/PACKAGE/? and by mymodule you mean myfile.py?

[drootang]

Using the v3.5+ method described leads to pickling errors. An answer linked above by @mforbes here adds an additional step that appears to fix this: sys.modules[spec.name] = foo

[RuRo]

This answer is way underrated. It's very short and simple! Even better, if you need a proper module namespace, you can do something like from runpy import run_path; from argparse import Namespace; mod = Namespace(**run_path('path/to/file.py'))

[IAbstract]

What is mymodule in relation to /path/to/your/module/__init__.py?

[Ardhi]

the simplest one IMO

[Christoph90]

Are they out of their minds establishing 3 different ways for the same thing in these different language versions???

[Pithikos]

It would be cool if this actually returned the module.

[Myridium]

What is module.name?? I'm importing a file, or something from a file. It does not have a namespace.

[Maggyero]

Hi Nick. In PEP 338, you introduced only the function runpy.run_module and said that python -m module_name now delegates to it. 1. Did you write a similar PEP for the function runpy.run_path? 2. Does python file_path.py now delegate to runpy.run_path?

[Maggyero]

Also, I have just opened a public question here about your PEP 366. I am very interested by your thought on this.

[ncoghlan]

@Maggyero The command line never goes through runpy.run_path, but if a given path is a directory or zipfile, then it ends up delegating to runpy.run_module for the __main__ execution. The duplicated logic for "Is it a script, directory, or zipfile?" isn't complicated enough to be worth delegating to Python code.

[Maggyero]

Thanks, I was not aware of this. In CPython, does the delegation to runpy.run_module for a given module name (python -m name) happen exactly here and for a given directory or zip file path (python path) happen exactly here?

[Maggyero]

Also by looking at the implementation of the C function pymain_run_module, it seems that CPython delegates to the Python function runpy._run_module_as_main instead of runpy.run_module—though if I understood correctly the only difference is that the first function executes the code in the built-in __main__ environment (cf. here) while the second function executes it in a new environment?

[ncoghlan]

@Maggyero Yep, that's the only difference. Originally it used the public function, but that turned out to interact badly with the interpreter's -i option (which drops you into an interactive shell in the original __main__ module, so -m running in a new module was inconvenient)

[Azmisov]

Though unconventional, if your package entry point is something other than __init__.py, you can still import it as a package. Include spec.submodule_search_locations = [os.path.dirname(MODULE_PATH)] after creating the spec. You can also treat a __init__.py as a non-package (e.g. single module) by setting this value to None

[Chris A]

Isn't module.name always going to be the same as the name of the .py file?

[mikey]

This is an excellent solution. Note that you can use a relative path in your sys.path.append, so with this answer if your working directory was /foo/bar/, you could use sys.path.append('./mock-0.3.1')

[YoussefDir]

I just wanted a quick solution. So I changed the directory cd "folder" and then I was fine. Hope this helps someone.

[ジョージ]

Since no one had mentioned pydoc yet -- here are my two cents: pydoc::importfile()

[user202729]

Note that the behavior of this answer is different from importing a module, as for a module (imported the normal way or not) the "global" scope of the code is the module object, while for this answer it's the globals scope of the called object. (although this answer can be modified to change the scope too, any dictionary can be passed in as globals and locals)

[Nam G VU]

This simply work for me. Thank you!

[djvg]

The example in the docs is also quite clear: importing-a-source-file-directly

[Shihab]

does the upper code means the same as this import module i mean it is can compile variables, functions and classes ?

[kevinarpe]

If you need to import class MyClass into the current namespace (like I did during a dynamic Py2 vs Py3 import), try this: MyClass = getattr(foo, "MyClass") Discovered here: stackoverflow.com/a/41678146/257299