How to unset the 'http_proxy' environment variable in Python

Question

I am using below python code to reset the environment variable http_proxy in Linux CentOS 6, but it is not unsetting the variable for the rest of the Python script.

Code:

 import os 
 print "Unsetting http..." 
 os.system("unset http_proxy") 
 os.system("echo $http_proxy") 
 print "http is reset"

Output:

Unsetting http...
http://web-proxy.xxxx.xxxxxxx.net:8080
http is reset
Process finished with exit code 0

Solution 1

Each invocation of os.system() runs in its own subshell, with its own fresh environment:

>>> import os
>>> os.system("echo $$")
97678
0
>>> os.system("echo $$")
97679
0

You are unsetting the http_proxy variable, but then your subshell has completed executing the command (to wit: unset), and terminates. You then start a new subshell with a new environment in which to run echo.

I believe what you are trying to do is del os.environ['http_proxy'], or os.environ.pop('http_proxy') if you want to ensure there is no http_proxy environment variable whether or not one previously existed:

$ export foo=bar
$ python2
Python 2.7.10 (default, Jul 15 2017, 17:16:57)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.environ['foo']
'bar'
>>> del os.environ['foo']
>>> os.system('echo $foo')

0

Solution 2

Each call to os.system() is creating a separate process. Do echo $$ inside them both to see that the PID changes with each.

So your first os.system() spawns a new process which inside of itself unsets one variable. This doesn't effect the variables values in the process of the python script. Then the first spawned process exits and you spawn a new process. The second os.system() spawns this new process which inherits all of the environment variables from the python script - which haven't been touched. So of course the second spawned process sees the variable, because it was never unset in the process of the python script so it won't missing from any new child processes.

See https://stackoverflow.com/questions/3575165/what-is-the-correct-way-to-unset-a-linux-environment-variable-in-python for how to do this in python and http://docs.python.org/library/os.html#os.environ for the official docs.

Comments

[user2357112]

The putenv/unsetenv API is basically broken in Python, because getenv goes through os.environ, but putenv and unsetenv don't update os.environ. putenv and unsetenv update the environment variables, but there's no way to examine their effects through either os.environ or getenv. It's much better to delete the key from os.environ.

[user2357112]

A related issue has been languishing on the tracker for over a decade.

[Pankaj Goyal]

To clarify: does deleting the key from os.environ actually DTRT and remove the environment variable?

[user2357112]

Yes: "If the platform supports the unsetenv() function, you can delete items in this mapping to unset environment variables. unsetenv() will be called automatically when an item is deleted from os.environ, and when one of the pop() or clear() methods is called." Same with putenv: if the platform has putenv, setting keys in os.environ will automatically perform the corresponding putenv calls. I'm not sure what platforms don't have those functions, but the docs say "Availability: most flavors of Unix, Windows."

[user2357112]

That does bring up a caveat I should have thought to mention earlier, which is that if the platform doesn't have putenv/unsetenv, attempting to call those functions directly will throw an error, while modifying os.environ will just have no effect on environment variables. On such platforms (whichever ones they are), you have to launch subprocesses with a function that accepts an explicit environment dict instead of os.system if you want to set environment variables for subprocesses. You can check if you're on such a platform with hasattr(os, 'putenv') and hasattr(os, 'unsetenv').

[Sum]

Now if I want to set new http_proxy manually for the CentOS 6.6 , then what is the correct way to do it permanently. I am using CentOS 6.6 image inside Vmware Workstation.