setting an environment variable in virtualenv

Question

I have a Heroku project that uses environment variables to get its configuration, but I use virtualenv to test my app locally first.

Is there a way to set the environment variables defined on the remote machine inside virtualenv?

Solution 1

Update

As of 17th May 2017 the README of autoenv states that direnv is probably the better option and implies autoenv is no longer maintained.

Old answer

I wrote autoenv to do exactly this:

https://github.com/kennethreitz/autoenv

Solution 2

In case you're using virtualenvwrapper (I highly recommend doing so), you can define different hooks (preactivate, postactivate, predeactivate, postdeactivate) using the scripts with the same names in $VIRTUAL_ENV/bin/. You need the postactivate hook.

$ workon myvenv

$ cat $VIRTUAL_ENV/bin/postactivate
#!/bin/bash
# This hook is run after this virtualenv is activated.
export DJANGO_DEBUG=True
export S3_KEY=mykey
export S3_SECRET=mysecret

$ echo $DJANGO_DEBUG
True

If you want to keep this configuration in your project directory, simply create a symlink from your project directory to $VIRTUAL_ENV/bin/postactivate.

$ rm $VIRTUAL_ENV/bin/postactivate
$ ln -s .env/postactivate $VIRTUAL_ENV/bin/postactivate

You could even automate the creation of the symlinks each time you use mkvirtualenv.

Cleaning up on deactivate

Remember that this wont clean up after itself. When you deactivate the virtualenv, the environment variable will persist. To clean up symmetrically you can add to $VIRTUAL_ENV/bin/predeactivate.

$ cat $VIRTUAL_ENV/bin/predeactivate
#!/bin/bash
# This hook is run before this virtualenv is deactivated.
unset DJANGO_DEBUG

$ deactivate

$ echo $DJANGO_DEBUG

Remember that if using this for environment variables that might already be set in your environment then the unset will result in them being completely unset on leaving the virtualenv. So if that is at all probable you could record the previous value somewhere temporary then read it back in on deactivate.

Setup:

$ cat $VIRTUAL_ENV/bin/postactivate
#!/bin/bash
# This hook is run after this virtualenv is activated.
if [[ -n $SOME_VAR ]]
then
    export SOME_VAR_BACKUP=$SOME_VAR
fi
export SOME_VAR=apple

$ cat $VIRTUAL_ENV/bin/predeactivate
#!/bin/bash
# This hook is run before this virtualenv is deactivated.
if [[ -n $SOME_VAR_BACKUP ]]
then
    export SOME_VAR=$SOME_VAR_BACKUP
    unset SOME_VAR_BACKUP
else
    unset SOME_VAR
fi

Test:

$ echo $SOME_VAR
banana

$ workon myenv

$ echo $SOME_VAR
apple

$ deactivate

$ echo $SOME_VAR
banana

Solution 3

You could try:

export ENVVAR=value

in virtualenv_root/bin/activate. Basically the activate script is what is executed when you start using the virtualenv so you can put all your customization in there.

Solution 4

Using only virtualenv (without virtualenvwrapper), setting environment variables is easy through the activate script you're sourcing in order to activate the virtualenv.

On unix, run:

nano YOUR_ENV/bin/activate

or if you're on windows:

nano YOUR_ENV/Scripts/activate.bat

Then, add the environment variables to the end of the file. If you're on unix:

export KEY=VALUE

or if you're on windows:

set KEY=VALUE

You can also set a similar hook to unset the environment variable as suggested by Danilo Bargen in his excellent answer above.

Solution 5

While there are a lot of nice answers here, I didn't see a solution posted that both includes unsetting environment variables on deactivate and doesn't require additional libraries beyond virtualenv, so here's my solution that just involves editing /bin/activate, using the variables MY_SERVER_NAME and MY_DATABASE_URL as examples:

There should be a definition for deactivate in the activate script, and you want to unset your variables at the end of it:

deactivate () {
    ...

    # Unset My Server's variables
    unset MY_SERVER_NAME
    unset MY_DATABASE_URL
}

Then at the end of the activate script, set the variables:

# Set My Server's variables
export MY_SERVER_NAME="<domain for My Server>"
export MY_DATABASE_URL="<url for database>"

This way you don't have to install anything else to get it working, and you don't end up with the variables being left over when you deactivate the virtualenv.

View more solutions

Comments

[Zoneur]

Just a precision: doing ln -s .env/postactivate $VIRTUAL_ENV/bin/postactivate did not work for me. ln wants a full path, so I had to do ln -s `pwd`/.env/postactivate $VIRTUAL_ENV/bin/postactivate

[Danilo Bargen]

@Zoneur What OS are you on? Under Linux relative paths work for ln.

[Zoneur]

@DaniloBargen I use LinuxMint 3.2.0. This answer said that ln likes full paths so I tried that and it worked. When I tried to cat the symlink with relative path it said No such file or directory.

[Kent Fredric]

@dpwrussel, that almost didn't make it through review, its a good addition, but its so significant it could have been made as its own post ( which would have gotten you some rep ). Lots of Good answers are good :)

[Michel Müller]

a much more sane approach IMO. overriding cd just to have environment variables? shudder

[chachan]

Not sure if that's clean enough but definitively works!

[Michael Scheper]

Yeah, it's cheap and nasty, but occasionally that's what you need.

[galarant]

Just FYI it seems that .env files bork Heroku builds, at least in my experience. So don't include it in your repo. Long time user / huge fan of autoenv btw. Hi Kenneth, you da man!

[VStoykov]

I like this approach because I don't want external libs or apps but the problem with this is that if you rebuild the environment you will loose all your settings.

[Anthony Manning-Franklin]

The advantage to this approach is the speed of setup and lack of magic. Keeping environ variables out of source control will always lead you back to the problem of potentially destroying your secrets/settings when rebuilding environments.

[CpILL]

And source control? How does this translate to other people cloning and setting up a project that needs the env. var.s?

[fraxture]

Does the virtualenv directory end up being checked into the repository for this to work? What if the variables hold secrets that you don't want in the repo? How would you handle this?

[TheLetterN]

I don't really see why it would be a good idea to include a virtualenv in your repository, as they are not very portable, but I imagine you could put your exports in a separate file instead of the activate script and source the file if it's present, and don't add that file to your repository.

[freezed]

Is this answer still relevant after edit? What is your opinion about solution suggested by Nagasaki45 & TheLetterN

[guettli]

It would be very nice, if I could enable the environment variables without the need for a shell.

[William]

I don't recommend this, I did it and sometime later all the activate scripts (activate, activate.csh, activate.fish) were overwritten automatically so I lost my change. Use postactivate and predeactivate.

[Rik Schoonbeek]

do not use spaces around the =

[Lou Zell]

Could also add 'unset ENVVAR' in the deactivate function defined virtualenv_root/bin/activate to balance setting and unsetting

[Daniil Mashkin]

save to aliases with echo 'alias e=". env/bin/activate && set -a; source .env; set +a"' >> ~/.bash_aliases

[buncis]

how about the cleanup after deactive?

[Manuel Lazo]

I worked with this method and works like a charm!!, it was a perfect method for a tool called pytlint that everytime that was called it required the variable "DJANGO_SETTINGS_MODULE" to be set with the settings file. Greetingss

[LittleEaster]

I like this most other because 1: reuse present code, 2: do not add extra overstructure tools (like autoenv, env, env...) that add logics and commands... only thing that the sourced vars remain in the user space (aka: are effective also after deactivate) but nice and (quite) clean solution for most of dedicated systems

[niid]

The last line will auto-activate the virtual env? I don't think that was asked in the original question. Also I think autoenv is maybe a security risk if you download repos or code from the internet. Are we sure no code from the .env is executed?

[niid]

Can you explain set +a?

[Mitalee Rao]

@niid - this may help - unix.stackexchange.com/a/79084

[waykiki]

I like this approach a lot, it's way cleaner than any of the other answers. However, my concern is whether there is any possibility/action that could lead to the "activate" script being overwritten/reset. Some of the other answers contain such discussions. A guarantee that the "activate" script cannot be overwritten without the user doing so manually would make this the best and simplest answer IMO.

[Rivers Cuomo]

This works but for me on Windows it was YOUR_ENV/Scripts/activate.bat and set KEY=VALUE.

[Nagasaki45]

Thanks for the windows specific info @RiversCuomo! Added this to the answer.

[Stephen Bosch]

This is the only answer that uses the modern venv, and should be the correct one. The other answers are hopelessly outdated. The postactivate and predeactivate hooks do not exist in venv-created virtual environments.