Running python script from inside virtualenv bin is not working

Question

I have a script I want to be available globally. I've started it with the standard hashbang:

#! /usr/bin/env python

And linked it into the bin directory of my virtualenv:

~/environments/project/env/bin/myscript

And added that directory to my path. When I run the command:

myscript

I get an import error with one of the libraries. However, if I activate the virtual environment and run the script, it works as expected.

I've ruled out a problem with the symlink (I've also tried just moving the script inside the bin folder). I've also tried running the script with python

python ~/environments/project/env/bin/myscript

Previously I was using a script that activated the environment and then ran my script, but I was under the impression that script run from this folder should run with the virtualenv's interpretor and site-packages. Any ideas of why this might not be working or some ways I could debug this?

Solution 1

Putting the script into the bin of your virtualenv, and then adding that bin location to your global PATH will not automatically source your virtualenv. You do need to source it first to make it active.

All that your system knows is to check that extra path for the executable and run it. There isn't anything in that script indicating a virtualenv.

You could, however, hardcode the she-bang line to your virtualenv python, in which case the site-packages will end up on the path:

#!/Users/foo/environments/project/env/bin/python

Or another option is to simply create a tiny bash wrapper that calls your original pythons script, which will allow you to leave your original script with a generic she-bang..

So if myscript.py is: #!/usr/bin/env python ...

Then you can make a myscript :

#!/bin/bash

/Users/foo/environments/project/env/bin/python myscript.py

When you do myscript, it will explicitly call your python script with the interpreter you set up.

Solution 2

I think you are confused as to how virtualenv works.

In a nutshell, virtualenv modifies your shell environment so that Python will look in different areas to find modules you want to import. There really isn't any relation between where you store your virtual environment and where you store your source files that you run in the virtualenv. If you wanted to, you could store your virtualenv in a directory called ~/environments/my_env, and all the source that you code while using your virtualenv in ~/projects/my_proj.

You can read more about what virtulenv does in the docs.

Really, the only thing that tells python where to find the modules are completely based on python (see the docs on how it works). Activating a virtualenv changes the way that python works.

You can go back to having a shell script activate the virtualenv for you, or you can follow this recipe to activate it directly from your script.

activate_this = '/path/to/env/bin/activate_this.py'
execfile(activate_this, dict(__file__=activate_this))

If you choose this route, keep the info the docs give in mind:

This will change sys.path and even change sys.prefix, but also allow you to use an existing interpreter. Items in your environment will show up first on sys.path, before global items. However, global items will always be accessible (as if the --system-site-packages flag had been used in creating the environment, whether it was or not). Also, this cannot undo the activation of other environments, or modules that have been imported. You shouldn’t try to, for instance, activate an environment before a web request; you should activate one environment as early as possible, and not do it again in that process.

Solution 3

Couldn't you just add a relative path instead? This worked for me:

#!./env/bin/python

Comments

[justin]

Thanks that's what my original script did. I had a memory of seeing something that I thought said that folder would activate the environment, but you're right on reflection, I see that would be impossible.

[Josh Usre]

Wouldn't you have to source myscript, instead of simply calling it by name?

[jdi]

@JoshUsre no, myscript in this case is a bootstrap that launches myscript.py in a specific way. Sourcing scripts is what you do when you want the effects of a script to be evaluated in the current shell.