How to import a module from a different folder?

Question

I have a project which I want to structure like this:

myproject
├── api
│   ├── __init__.py
│   └── api.py
├── backend
│   ├── __init__.py
│   └── backend.py
├── models
│   ├── __init__.py
│   └── some_model.py
└── __init__.py

Now, I want to import the module some_model.py in both api.py and backend.py. How do I properly do this?

I tried:

from models import some_model

but that fails with ModuleNotFoundError: No module named 'models'.

I also tried:

from ..models import some_model

which gave me ValueError: attempted relative import beyond top-level package.

What am I doing wrong here? How can I import a file from a different directory, which is not a subdirectory?

Solution 1

Firstly, this import statement:

from models import some_model

should be namespaced:

# in myproject/backend/backend.py or myproject/api/api.py
from myproject.models import some_model

Then you will need to get the directory which contains myproject, let's call this /path/to/parent, into the sys.path list. You can do this temporarily by setting an environment variable:

export PYTHONPATH=/path/to/parent

Or, preferably, you can do it by writing a setup.py file and installing your package. Follow the PyPA packaging guide. After you have written your setup.py file, from within the same directory, execute this to setup the correct entries in sys.path:

pip install --editable .

Solution 2

Unfortunately, Python will only find your file if your file is in the systems path. But fear not! There is a way around this!

Using python's sys module, we can add a directory to the path just while Python is running, and once Python stops running, it will remove it from the path.

You can do this by:

import sys
sys.path.insert(0, '/path/to/application/app/folder')
import [file]

It is important to import sys and set the directory path before you import the file however.

Good luck!

Jordan.

Comments

[Phydeaux]

This will work, but you should also be able to do a relative import from within the myproject package, which OP has tried. What would cause that not to work?

[wim]

Changing the path from within code, though possible, is messy and bad - it makes the test setup more difficult and makes the import ordering become significant.

[wim]

That has been covered a billion times over at Relative imports for the billionth time, so I won't go into it again.

[Phydeaux]

you think OP is running api.py as a script? that would explain it, yes

[Gasp0de]

I guess I'll dive into the PyPA packaging guide then =)

[Gasp0de]

I created a package and installed it using pip install --editable as suggested. Works like a charm, thank you!

[claudio]

This answer really helped me. Although I’ve seen similar answers on many other similar questions, I couldn’t believe you have to install the package you want to develop itself. That is – at least for me – highly contra intuitive. Your answer clarified it.

[alper]

After doing pip install --editable . if we made a change on any file under the imported folder, should we do pip install --editable . again to fetch the new updates?

[wim]

@alper it should not be necessary. The whole point of the —editable installation is that you are able to edit the installed code directly.