Python "ImportError: No module named" Problem

Question

I'm running Python 2.6.1 on Windows XP SP3. My IDE is PyCharm 1.0-Beta 2 build PY-96.1055.

I'm storing my .py files in a directory named "src"; it has an __init__.py file that's empty except for an "__author__" attribute at the top.

One of them is called Matrix.py:

#!/usr/bin/env python
"""
"Core Python Programming" chapter 6.
A simple Matrix class that allows addition and multiplication
"""
__author__ = 'Michael'
__credits__ = []
__version__ = "1.0"
__maintainer__ = "Michael"
__status__ = "Development"

class Matrix(object):
    """
    exercise 6.16: MxN matrix addition and multiplication
    """
    def __init__(self, rows, cols, values = []):
        self.rows = rows
        self.cols = cols
        self.matrix = values

    def show(self):
        """ display matrix"""
        print '['
        for i in range(0, self.rows):
            print '(',
            for j in range(0, self.cols-1):
                print self.matrix[i][j], ',',
            print self.matrix[i][self.cols-1], ')'
        print ']'

    def get(self, row, col):
        return self.matrix[row][col]

    def set(self, row, col, value):
        self.matrix[row][col] = value

    def rows(self):
        return self.rows

    def cols(self):
        return self.cols

    def add(self, other):
        result = []
        for i in range(0, self.rows):
            row = []
            for j in range(0, self.cols):
                row.append(self.matrix[i][j] + other.get(i, j))
            result.append(row)
        return Matrix(self.rows, self.cols, result)

    def mul(self, other):
        result = []
        for i in range(0, self.rows):
            row = []
            for j in range(0, other.cols):
                sum = 0
                for k in range(0, self.cols):
                    sum += self.matrix[i][k]*other.get(k,j)
                row.append(sum)
            result.append(row)
        return Matrix(self.rows, other.cols, result)

    def __cmp__(self, other):
        """
        deep equals between two matricies
        first check rows, then cols, then values
        """
        if self.rows != other.rows:
            return self.rows.cmp(other.rows)
        if self.cols != other.cols:
            return self.cols.cmp(other.cols)
        for i in range(0, self.rows):
            for j in range(0, self.cols):
                if self.matrix[i][j] != other.get(i,j):
                    return self.matrix[i][j] == (other.get(i,j))
        return True # if you get here, it means size and values are equal



if __name__ == '__main__':
    a = Matrix(3, 3, [[1, 2, 3], [4, 5, 6], [7, 8, 9]])
    b = Matrix(3, 3, [[6, 5, 4], [1, 1, 1], [2, 1, 0]])
    c = Matrix(3, 3, [[2, 0, 0], [0, 2, 0], [0, 0, 2]])
    a.show()
    b.show()
    c.show()
    a.add(b).show()
    a.mul(c).show()

I've created a new directory named "test" that also has an __init__.py file that's empty except for an "__author__" attribute at the top. I've created a MatrixTest.py to unit my Matrix class:

#!/usr/bin/env python
"""
Unit test case for Matrix class
See http://jaynes.colorado.edu/PythonGuidelines.html#module_formatting for Python coding guidelines
"""

import unittest #use my unittestfp instead for floating point
from src import Matrix # Matrix class to be tested

__author__ = 'Michael'
__credits__ = []
__license__ = "GPL"
__version__ = "1.0"
__maintainer__ = "Michael"
__status__ = "Development"

class MatrixTest(unittest.TestCase):
    """Unit tests for Matrix class"""
    def setUp(self):
        self.a = Matrix.Matrix(3, 3, [[1, 2, 3], [4, 5, 6], [7, 8, 9]])
        self.b = Matrix.Matrix(3, 3, [[6, 5, 4], [1, 1, 1], [2, 1, 0]])
        self.c = Matrix.Matrix(3, 3, [[2, 0, 0], [0, 2, 0], [0, 0, 2]])

    def testAdd(self):
        expected = Matrix.Matrix(3, 3, [[7, 7, 7], [5, 6, 7], [9, 9, 9]])    # need to learn how to write equals for Matrix
        self.a.add(self.b)
        assert self.a == expected

if __name__ == '__main__':    #run tests if called from command-line
    suite = unittest.TestLoader().loadTestsFromTestCase(TestSequenceFunctions)
    unittest.TextTestRunner(verbosity=2).run(suite)

Yet when I try to run my MatrixTest I get this error:

C:\Tools\Python-2.6.1\python.exe "C:/Documents and Settings/Michael/My Documents/Projects/Python/learning/core/test/MatrixTest.py"
Traceback (most recent call last):
  File "C:/Documents and Settings/Michael/My Documents/Projects/Python/learning/core/test/MatrixTest.py", line 8, in <module>
    from src import Matrix # Matrix class to be tested
ImportError: No module named src

Process finished with exit code 1

Everything I've read tells me that having the init.py in all my directories should take care of this.

If someone could point out what I've missed I'd greatly appreciate it.

I'd also like advice on the best way to develop and maintain source and unit test classes. I'm thinking about this the way I usually do when I write Java: /src and /test directories, with identical package structures underneath. Is this "Pythonic" thinking, or should I consider another organization scheme?

UPDATE:

Thanks to those who have answered, here's the solution that worked for me:

1. Change import to from src import Matrix # Matrix class to be tested
2. Add sys.path as an environment variable to my unittest configuration, with ./src and ./test directories separated by semi-colon.
3. Change declarations in MatrixTest.py as shown.

Solution 1

This is a bit of a guess, but I think you need to change your PYTHONPATH environment variable to include the src and test directories.

Running programs in the src directory may have been working, because Python automatically inserts the directory of the script it is currently running into sys.path. So importing modules in src would have worked as long as you are also executing a script that resides in src.

But now that you are running a script from test, the test directory is automatically added to sys.path, while src is not.

All directories listed in PYTHONPATH get added to sys.path, and Python searches sys.path to find modules.

Also, if you say

from src import Matrix

then Matrix would refer to the package, and you'd need to say Matrix.Matrix to access the class.

Solution 2

Regarding best practices, PycURL uses a tests directory at the same level as the main source code. On the other hand projects like Twisted or sorl-thumbnail use a test(s) subdirectory under the main source code.

The other half of the question has been already answered by ~unutbu.

Comments

[unutbu]

@Cristian: I put import sys; print(sys.path) in a file called test.py. Then I ran cd /some/other/dir; python /path/to/test.py. The first path listed is /path/to, not /some/other/dir. So it appears the directory of the script, not the CWD that is added to sys.path.

[Cristian Ciupitu]

@~unutbu: you're right! The reason for my comment was this snippet from the tutorial: "When a module named spam is imported, the interpreter searches for a file named spam.py in the current directory, and then in the list of directories specified by the environment variable PYTHONPATH.". I guess someone should rephrase it in less ambiguous way.

[unutbu]

@Cristian: Hmm, that certainly is confusing. (Though, it tries to clarify in the second paragraph). Here's another link to the docs; this one I think is clearer: docs.python.org/library/sys.html#sys.path

[duffymo]

Hi Christian, Thank you for the example. I prefer the way you do it: /src and /test at the same level.

[duffymo]

Thank you, ~unutbu. The solution you and Christian Ciupitu led me to is added to my question.

[Cristian Ciupitu]

@~unutbu: yes the second paragraph clarifies things, although when I read the tutorial a long time ago, it didn't exist or I skipped it somehow. Anyway, I reported this to the documentation team and maybe they'll rephrase that section to make it clearer. Also, thanks for the sys.path link!