Import error on installed package using setup.py

Question

I have a problem with using setup.py to setup a python package. First, I have the following directory setup:

maindir
   |- setup.py
   |-mymodule
         |- __init__.py
         |- mainmodule.py
         |-subdir
             |- __init__.py
             |- submodule.py

i.e. the project directory contains the setup.py and a directory mymodule, which in itself contains two python modules in two directories. The file submodule.py contains just

teststring = "hello world"

mainmodule.py contains:

from .subdir import submodule
mainstring = "42"

and setup.py contains:

import os
from setuptools import setup
setup(
    name = "mytestmodule",
    version = "0.0.1",
    description = ("A simple module."),
    packages=['mymodule'],
)

When I do from mymodule import mainmodule with ipython from within sourceTest the behaviour works as expected and I can reference e.g. mainmodule.submodule.teststring which gives me the string hello world.

On the other side, when I install this 'package' using python setup.py install and try to do the same (from within some other directory), I get an import error:

In [1]: from mymodule import mainmodule
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
/home/alexander/<ipython-input-1-cf4c9bafa487> in <module>()
----> 1 from mymodule import mainmodule

/home/alexander/build/bdist.linux-i686/egg/mymodule/mainmodule.py in <module>()

ImportError: No module named subdir

I do not see what I have done wrong, as I followed a Getting started tutorial and rules for importing intra-packages. I suppose my mistake is a really tiny one, but I cannot spot it and help is appreciated.

Solution 1

You have to list all packages in setup, including subpackages:

setup(
    name = "mytestmodule",
    version = "0.0.1",
    description = ("A simple module."),
    packages=['mymodule', 'mymodule.subdir'],
)

Or you can use setuptools's magic function find_packages:

from setuptools import setup, find_packages
setup(
    name = "mytestmodule",
    version = "0.0.1",
    description = ("A simple module."),
    packages=find_packages(),
)

This is mentioned here:

If you have sub-packages, they must be explicitly listed in packages, but any entries in package_dir automatically extend to sub-packages. (In other words, the Distutils does not scan your source tree, trying to figure out which directories correspond to Python packages by looking for __init__.py files.)

Solution 2

You need to specify your each modules explicitly. Instead of maintaining the complexity of adding module to setup.py everytime, you may use the find_packages method from setuptools.

find_packages takes two optional arguments:

1. where which is default to '.' i.e your curdir.
2. exclude list of stuff to exclude

I usually have tests in my repo, so I use:

from setuptools import find_packages

packages=find_packages(exclude=["*.tests", "*.tests.*", "tests.*", "tests"]),

Comments

[Alex]

Is this described somewhere? Is there some documentation on the content of setup on distutils? It does not seem easy to find representative documentation.

[0 _]

For further reference that elaborates on how package_dir is used recursively to find sub-packages, provided that those sub-packages are listed in packages, read this.