How to get environment from a subprocess?
Solution 1
Since you're apparently in Windows, you need a Windows answer.
Create a wrapper batch file, eg. "run_program.bat", and run both programs:
@echo off
call proc1.bat
proc2
The script will run and set its environment variables. Both scripts run in the same interpreter (cmd.exe instance), so the variables prog1.bat sets will be set when prog2 is executed.
Not terribly pretty, but it'll work.
(Unix people, you can do the same thing in a bash script: "source file.sh".)
Solution 2
Here's an example of how you can extract environment variables from a batch or cmd file without creating a wrapper script. Enjoy.
from __future__ import print_function
import sys
import subprocess
import itertools
def validate_pair(ob):
try:
if not (len(ob) == 2):
print("Unexpected result:", ob, file=sys.stderr)
raise ValueError
except:
return False
return True
def consume(iter):
try:
while True: next(iter)
except StopIteration:
pass
def get_environment_from_batch_command(env_cmd, initial=None):
"""
Take a command (either a single command or list of arguments)
and return the environment created after running that command.
Note that if the command must be a batch file or .cmd file, or the
changes to the environment will not be captured.
If initial is supplied, it is used as the initial environment passed
to the child process.
"""
if not isinstance(env_cmd, (list, tuple)):
env_cmd = [env_cmd]
# construct the command that will alter the environment
env_cmd = subprocess.list2cmdline(env_cmd)
# create a tag so we can tell in the output when the proc is done
tag = 'Done running command'
# construct a cmd.exe command to do accomplish this
cmd = 'cmd.exe /s /c "{env_cmd} && echo "{tag}" && set"'.format(**vars())
# launch the process
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, env=initial)
# parse the output sent to stdout
lines = proc.stdout
# consume whatever output occurs until the tag is reached
consume(itertools.takewhile(lambda l: tag not in l, lines))
# define a way to handle each KEY=VALUE line
handle_line = lambda l: l.rstrip().split('=',1)
# parse key/values into pairs
pairs = map(handle_line, lines)
# make sure the pairs are valid
valid_pairs = filter(validate_pair, pairs)
# construct a dictionary of the pairs
result = dict(valid_pairs)
# let the process finish
proc.communicate()
return result
So to answer your question, you would create a .py file that does the following:
env = get_environment_from_batch_command('proc1')
subprocess.Popen('proc2', env=env)
Solution 3
As you say, processes don't share the environment - so what you literally ask is not possible, not only in Python, but with any programming language.
What you can do is to put the environment variables in a file, or in a pipe, and either
- have the parent process read them, and pass them to proc2 before proc2 is created, or
- have proc2 read them, and set them locally
The latter would require cooperation from proc2; the former requires that the variables become known before proc2 is started.
Solution 4
The Python standard module multiprocessing have a Queues system that allow you to pass pickle-able object to be passed through processes. Also processes can exchange messages (a pickled object) using os.pipe. Remember that resources (e.g : database connection) and handle (e.g : file handles) can't be pickled.
You may find this link interesting : Communication between processes with multiprocessing
Also the PyMOTw about multiprocessing worth mentioning : multiprocessing Basics
sorry for my spelling
Solution 5
You can use Process
in psutil to get the environment variables for that Process.
If you want to implement it yourself, you can refer to the internal implementation of psutil. It adapts to some operating system.
Currently supported operating systems are:
- AIX
- FreeBSD, OpenBSD, NetBSD
- Linux
- macOS
- Sun Solaris
- Windows
Eg: In Linux platform, you can find one pid 7877 environment variables in file /proc/7877/environ
, just open with rt
mode to read it.
Of course the best way to do this is to:
import os
from typing import Dict
from psutil import Process
process = Process(pid=os.getpid())
process_env: Dict = process.environ()
print(process_env)
You can find other platform implementation in source code
Hope I can help you.
Related videos on Youtube
Admin
Updated on April 17, 2022Comments
-
Admin about 2 years
I want to call a process via a python program, however, this process need some specific environment variables that are set by another process. How can I get the first process environment variables to pass them to the second?
This is what the program look like:
import subprocess subprocess.call(['proc1']) # this set env. variables for proc2 subprocess.call(['proc2']) # this must have env. variables set by proc1 to work
but the to process don't share the same environment. Note that these programs aren't mine (the first is big and ugly .bat file and the second a proprietary soft) so I can't modify them (ok, I can extract all that I need from the .bat but it's very combersome).
N.B.: I am using Windows, but I prefer a cross-platform solution (but my problem wouldn't happen on a Unix-like ...)
-
0x6adb015 almost 15 yearsUnder Linux, if you are root, you can check /proc/<pid>/environ and parse it... not "impossible".
-
Martin v. Löwis almost 15 years@0x6abd015: however, this is not what the OP asks: he doesn't want proc2 find out what proc1's environment is (what you can do with the proc file system), but he wants proc1 to set environment for proc2 - I still claim that this impossible.
-
Jason R. Coombs over 9 yearsAn edit suggested that the implementation as written doesn't support non-ASCII code pages. In that case, the recommended fix was to place
chcp 65001 > NULL &&
before{env_cmd}
in the cmd construction. -
Anton Kochkov over 8 yearsHere is more updated version of this function github.com/PySide/pyside2-setup/blob/master/utils.py#L379
-
nmz787 over 7 yearsI am specifically trying to work around using
sourch blah; now_do_second_thing
-
nmz787 over 7 yearsThis doesn't answer the question. There are good reasons against doing this too, if you have programs that need conflicting ENV vars set, for example.
-
nmz787 over 7 yearsThis is not cross-platform, as OP requested.
-
Jason R. Coombs over 7 yearsThat's right, it's not generally possible to extract the environment from a running process, as seen in Martin v. Löwis' answer. This answer focuses on the OP's problem at hand. Presumably, this would work cross platform if you had
cmd.exe
on that platform, which you would if you're working with a batch file for proc1. -
drodri almost 4 yearsAccordingly to psutil.readthedocs.io/en/latest/#processes, the
environ()
method might not contain the changes of the environment after it started, so this would render this approach invalid