Import modules from subfolders

Question

I have following arrangement of files:

python
|---- main.py
|---- files
      |---- folder1
            |---- a.py, a1.py, ...
      |---- folder2
            |---- b.py, b1.py, ...

I wanted to import my modules a.py and b.py to main.py. For this I used the following commands in main.py:

a = 'C:/python/files/folder1'
sys.path.insert(0, a)
from files.folder1 import *

However, I am unable to import modules from folder1 (similarly for folder2).

I get an error:

No module named files.folder1

I cannot use import command as there are many Python files in folder1, folder2, ...

What am I missing here?

Solution 1

When I do this in Python 2.7 I use:

import sys
sys.path.append('C:/python/files/folder1')

import a
import a1

Here's a hack I built to import all modules in a directory into a dictionary:

import os
import sys

dir_of_interest = 'C:/python/files/folder1'
modules = {}

sys.path.append(dir_of_interest)
for module in os.listdir(dir_of_interest):
    if '.py' in module and '.pyc' not in module:
        current = module.replace('.py', '')
        modules[current] = __import__(current)

I just built it and it's extremely hacky but it might be more like something you want. So, to access the module you want, instead of saying module_name.thing you would say modules["module_name"].thing

Solution 2

Add a file __init__.py (can be blank) to folders files, folder1 and folder2. Then you got a package files with sub-packages folder1 and folder2. After this you can import from the main.py like this:

from files.folder1 import *

Solution 3

I cannot use import command as there are many Python files in folder1, folder2, ...

What am I missing here?

I believe the part you are missing is the __init__.py file in each of the folders. That file should include an __all__ variable that lists all the submodules that will imported by: from somepackage.subpackage import *.

This is all elegantly explained in the Python Tutorial section on Packages.

Solution 4

If you add folder1 to the path, that doesn't mean you can import folder1 as a module. It means you can import the files inside folder1. So you could do:

import a
import a1

If you want to have folder1 be a package of which a and a1 are modules, you need to put an __init__.py in folder1 and then do import folder1. If you further want to be able to do from folder1 import * and have that import a and a1, you need to put code in your __init__.py that imports a and a1.

If you have a lot of files in a folder that you want to be able to import in a structured way, you should make that folder into a package.

View more solutions

Comments

[user741592]

I agree, however, isn't it cumbersome to write many import statements if you have large number of files?

[BrenBarn]

@user741592: Why are you splitting your code up between many files? You should group relevant stuff together into a module.

[user741592]

Unfortunately, there is a requirement to split up the code. I would have been happy to group the chunk into a single file.

[mjgpy3]

See the above edit/hack... It might be something more like what you're looking for.

[Sahil Sareen]

Nice straightforward answer!

[tiru]

suppose if my folder structure is like dataaccess.service/services.py, now how to import this path?

[Dipankar]

The reference link provided really explain it very well.