Using Python in a Bash Script

Question

If I try to start python in a bash script, the script will stop running and no commands will execute after "Python" is called. In this simple example, "TESTPRINT" will not be printed. It seems like the script just stops.

#!/bin/bash

python

print("TESTPRINT")

Echo

How do I make the script continue running after going into Python? I believe I had the same problem a few years ago after writing a script that first needed to shell into an Android Phone. I can't remember how I fixed it that time.

Solution 1

To run a set of Python commands from a bash script, you must give the Python interpreter the commands to run, either from a file (Python script) that you create in the script, as in

#!/bin/bash -e

# Create script as "script.py"
cat >script.py <<'END_SCRIPT'
print("TESTPRINT")
END_SCRIPT

# Run script.py
python script.py

rm script.py

(this creates a new file called script.py or overwrites that file if it already exists, and then instructs Python to run it; it is then deleted)

... or directly via some form of redirection, for example a here-document:

#!/bin/bash

python - <<'END_SCRIPT'
print("TESTPRINT")
END_SCRIPT

What this does is running python - which instructs the Python interpreter to read the script from standard input. The shell then sends the text of the Python script (delimited by END_SCRIPT in the shell script) to the Python process' standard input stream.

Note that the two bits of code above are subtly different in that the second script's Python process has its standard input connected to the script that it's reading, while the first script's Python process is free to read data other than the script from standard input. This matters if your Python code reads from standard input.

Python can also take a set of commands from the command line directly with its -c option:

#!/bin/bash

python -c 'print("TESTPRINT")'

What you can't do is to "switch to Python" in the middle of a bash script.

The commands in a script is executed by bash one after the other, and while a command is executing, the script itself waits for it to terminate (if it's not a background job).

This means that your original script would start Python in interactive mode, temporarily suspending the execution of the bash script until the Python process terminates. The script would then try to execute print("TESTPRINT") as a shell command.

It's a similar issue with using ssh like this in a script:

ssh user@server
cd /tmp
ls

(which may possibly be similar to what you say you tried a few years ago).

This would not connect to the remote system and run the cd and ls commands there. It would start an interactive shell on the remote system, and once that shell has terminated (giving control back to the script), cd and ls would be run locally.

Instead, to execute the commands on a remote machine, use

ssh user@server "cd /tmp; ls"

(This is a lame example, but you may get the point).

---

The below example shows how you may actually do what you propose. It comes with several warning label and caveats though, and you should never ever write code like this (because it's obfuscated and therefore unmaintainable and, dare I say it, downright bad).

python -
print("TESTPRINT")

Running it:

$ sh -s <script.sh
TESTPRINT

What happens here is that the script is being run by sh -s. The -s option to sh (and to bash) tells the shell to execute the shell script arriving over the standard input stream.

The script then starts python -, which tells Python to run whatever comes in over the standard input stream. The next thing on that stream, since it's inherited from sh -s by Python (and therefore connected to our script text file), is the Python command print("TESTPRINT").

The Python interpreter would then continue reading and executing commands from the script file until it runs out or executes the Python command exit().

Solution 2

In addition to Kusalananda's answer, if you want the entire script to be run by python you can just change the first line to #!/usr/bin/env python3 and run it like any normal shell script. That way you don't have to remember what script you have to run with which interpreter.

Solution 3

You may also try the <<< (Here Strings) operator to save on lines:

$ python <<< 'print("MyTest")'
MyTest

Solution 4

You can also set python path while calling the python file from bash script:

export PYTHONPATH=/tmp/python_dep.zip && python test_my.py

View more solutions

Comments

[Admin]

Note, python may call python2 by default on some distributions. It is often best to be safe and explicitly call python3.

[RonJohn]

This is why languages like JCL (on IBM mainframes) and DCL (OpenVMS and some PDP-11 OSs) had each command start with a specific character. In DCL, for example, it's $. Every line which does not start with a $ is presumed to be program input, aka stdin. This is a legacy of batch jobs and card decks.

[dipi evil]

Even better, use #!/usr/bin/env python3, so that you use the system version of python3. See this answer for why.

[Skaperen]

IBM JCL still needs an explicit directive to read the next card as input. //SYSIN DD * see ibm.com/support/knowledgecenter/SSLTBW_2.1.0/…

[Akko]

@Jonathon Neat, I didn't know about that. Very useful!

[Shadur]

-1 for that first option, which opens up a big honking race condition exploit vulnerability in the delay between writing script.py and subsequently running it.

[Kusalananda]

@Shadur Sure, but it would be easier for the attacker to just change the shell script, or run their own Python code, as they have already compromised the account.

[Shadur]

Depending on where the script is written to - another big no-no in that example, because you don't specify a directory and don't test whether $CWD is writeable in the first place - they only need to be able to overwrite script.py which may not require account level access.

[Kusalananda]

@Shadur Fixed it.

[kmiklas]

You should probably clean up and delete script.py after it runs.

[Kusalananda]

@kmiklas That is reasonable.

[Akko]

@Paradox true, but a script with that shebang can be run in bash so I don't see how it would not apply?

[Paradox]

@Akko Since OP is asking "within a Bash script", I might be missing something here, but I do not see how to put two shebang in a single script and then how it would apply.

[Akko]

@Paradox ok but the example in the question seems to run only python code so it's still possible OP or somebody else might find this information helpful