Why can't Python see environment variables?

python bash environment-variables

31,909

You need to export environment variables for child processes to see them:

export SECRET_KEY

Demo:

$ SECRET_KEY='foobar'
$ bin/python -c "import os; print os.environ.get('SECRET_KEY', 'Nonesuch')"
Nonesuch
$ export SECRET_KEY
$ bin/python -c "import os; print os.environ.get('SECRET_KEY', 'Nonesuch')"
foobar

You can combine the setting and exporting in one step:

export SECRET_KEY=xxx-xxx-xxxx

Note that new variables in /etc/environment do not show up in your existing shells automatically, not until you have a new login. For a GUI desktop, you'll have to log out and log in again, for SSH sessions you'll have to create a new SSH login. Only then will you get a new tree of processes with the changes present. Using source /etc/environment only sets 'local' variables (the file is not a script). See How to reload /etc/environment without rebooting? over on Super User.

31,909

Author by

Richard

Updated on August 14, 2020

Comments

Richard almost 4 years
I'm working on Debian Jessie with Python 2. Why can't Python's environ see environment variables that are visible in bash?
```
# echo $SECRET_KEY
xxx-xxx-xxxx
# python
>>> from os import environ
>>> environ["SECRET_KEY"]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/.virtualenvs/prescribing/lib/python2.7/UserDict.py", line 23, in __getitem__
    raise KeyError(key)
KeyError: 'SECRET_KEY'
```
I set these environment variables using /etc/environment - not sure if that's relevant:
```
SECRET_KEY=xxx-xxx-xxx
```
I had to run source /etc/environment to get bash to see them, which I thought was strange.

UPDATE: printenv SECRET_KEY produces nothing, so I guess SECRET_KEY is a shell not an environment variable.
Richard almost 9 years

I was trying to follow the instructions in help.ubuntu.com/community/… - should I use export foo-bar instead of foo=bar in /etc/environment?
amza almost 9 years

It's worth noting that os.environ is only loaded the first time the os module is imported. So if you call a subprocess to execute a bash script, when it returns to the python script it won't be able to see the new environment variables that were possibly exported in your bash script.
Martijn Pieters almost 9 years

@Richard: in your question you named it /env/environment, not /etc/environment.. I'm not that familiar with Ubuntu, I'll have a look as to how the OS is using those. Did you restart at all after making changes to the file?
Martijn Pieters almost 9 years

@amza: a subprocess won't be able to set variables for the parent process anyway. This has nothing to do with os.environ being loaded when first imported (the mapping is actually built when Python starts). The process environment can only be altered by the current process or a parent process, and subprocess never creates such a process, so the distinction is entirely moot.
Martijn Pieters almost 9 years

@Richard: a quick experiment shows that changes in /etc/environment only apply to new logins; after editing I didn't see the change in new subshells, but use ssh localhost and those new login sessions do see the new variable. So I suspect that this is used by PAM to set up the new user environment.
Ariel over 5 years

Didn't work for me. I exported it but still get the error.
Martijn Pieters over 5 years

@Ariel: that's not something that we can help you with, though. I can't see how you exported your shell variables, and what software is not seeing that variable set or how you run your software. Exported shell variables are visible to child processes only, not to processes started from a different shell or parent process, for example.
Ariel over 5 years

@MartijnPieters it was an array variable and it turns out those cannot be exported at all. I was trying to read it from a Python shell, like the OP.