Python imports for tests using nose - what is best practice for imports of modules above current package

Question

This is a question which is asked frequently in different forms, and often obtains "lol you're not doing it properly" responses. Pretty sure that's because there's a common sense scenario people (including me) are trying to use as an implementation, and the solution is not obvious (if you've not done it before).

Would accept an answer which "lets the fly out of the bottle".

Given

project/
    __init__.py
    /code
        __init__.py
        sut.py
    /tests
        __init__.py
        test_sut.py

Where tests_sut.py starts:

import code.sut

Running nosetests in the root dir leads to:

ImportError: No module named code.sut

Avenues traveled:

a) do a relative using

from ..code import sut

b) add root of project to PYTHONPATH

c) use the

sys.path.append

to add the .. path before the imports at the start of each test module.

d) just remember to do a

setup.py 

on the project to install the modules into the site-packages before running tests.

---

So the requirement is to have tests located beneath the test package root which have access to the project. Each of the above don't feel "natural" to me, have proved problematic or seem like too much hard work!

In java this works, but basically by dint of your build tool / IDE placing all your classes on the classpath. Perhaps the issue is I'm expecting "magic" from Python? Have noted in the Flask webframework tests, option d) seems to be preferred.

In any case, statements below recommending a preferred solution would remove the feeling of "unnaturalness" in my own.

Solution 1

You have answered your question pretty well already.. D (install to system location) is preferred for distributable code. I usually use C (modify sys.path) because I don't want system-wide installs of my hundreds of custom libs. In theory A (relative import) seems nicer, but there are cases where it fails. B (PYTHONPATH) is right out, really only for testing purposes in my opinion.

That pretty much sums up all of the options. The option you prefer (Python magically knows where to look) is really not a workable solution because it can lead to unpredictable results, such as automagically finding libraries from unrelated projects.

In my opinion, the best thing to do is put this at the entry point(s) to your program:

import sys, os
sys.path = [os.path.abspath(os.path.dirname(__file__))] + sys.path

Solution 2

I had the same problem and found an answer in a related question work for me.

Just remove the __init__.py in the project root.

Solution 3

I know there is a answer checked and I still think it's a good reason to share other alternatives :)

There is a nose-pathmunge giving you a control to set sys.path while invoking nosestests.

Comments

[leonigmig]

but i guess my entry point(s!) is the nosetests command! i considered going down the route of writing a nosetests wrapper calling nose.run after sys.path append.. but again it felt unnatural. thanks for this, it will help me press on.

[Luke]

If you are running through nosetests, then the entry points are the individual modules that are imported, and this will be very awkward. In this case, you might find PYTHONPATH to be the least awkward option (as I said, it's ok for testing purposes). I feel your pain here--I'm also hoping more people chime in to make it seem ok (or to suggest a better solution).

[frabcus]

Great answer! I had an old __init__.pyc left over causing me this problem.

[user541905]

This did not help me, how is this supposed to work out?

[furins]

@user541905: because nose intelligently try to populate your sys.path. A good explanation from django-nose itself: github.com/django-nose/…

[neurotempest]

Surprisingly this did work for me - Although I feel that nose should actually add the current working directory to sys.path (just as the python unitest module does) as well as the directory containing the top of the current package tree

[Zeinab Abbasimazar]

I think it's better to use append instead of +.