How do I call a sed command in a python script?

Question

Through python script, I am trying to replace a string by a string in a file using sed command. I do that through subprocess.call as it in the script.

When I run the command in the shell script or command, it runs fine, but in python I get a result saying "No input file". Any idea how to fix that error?

#!/usr/bin/python
import subprocess
subprocess.call(["sed -i -e 's/hello/helloworld/g'","www.txt"], shell=True)

Output

No input file

Solution 1

You should avoid subprocess and implement the functionality of sed with Python instead, e.g. with the fileinput module:

#! /usr/bin/python
import fileinput
for line in fileinput.input("www.txt", inplace=True):
    # inside this loop the STDOUT will be redirected to the file
    # the comma after each print statement is needed to avoid double line breaks
    print line.replace("hello", "helloworld"),

Solution 2

With subprocess.call, either every argument to the command should be a separate item in the list (and shell should not be set to True):

subprocess.call(["sed", "-i", "-e",  's/hello/helloworld/g', "www.txt"])

Or, the entire command should one string, with shell=True:

subprocess.call(["sed -i -e 's/hello/helloworld/g' www.txt"], shell=True)

The arguments are treated similarly for subprocess.call and Popen, and as the documentation for subprocess.Popen says:

On Unix with shell=True, the shell defaults to /bin/sh. … If args is a sequence, the first item specifies the command string, and any additional items will be treated as additional arguments to the shell itself. That is to say, Popen does the equivalent of:

Popen(['/bin/sh', '-c', args[0], args[1], ...])

Comments

[muru]

Next time, ask generic programming questions on Stack Overflow.

[Jacob Vlijm]

Why use sed anyway in a python script? python has built-in tools.

[Adam]

I like this solution, but it messes up the file, it gives spaces between the lines everytime i do replace !. any idea how to fix that ?

[Byte Commander]

It shouldn't do that unless you omitted the comma in the end of the print statement. Do you have that in the script? Check it again.

[Maximilian]

Won't this be significantly slower than using sed?

[Paula Livingstone]

Hi I have been trying this but its obviously been written for Python 2.X however when I put parentheses around the arguments to the print command on the last line it seems to work but inserts a blank line between every line of the file to be edited. How would this be modified to cope with Python 3.

[Byte Commander]

@PaulaLivingstone print(line.replace("hello", "helloworld"), end="") should do it. The trailing comma at the end of the Python 2 print statement suppressed the line break, which is what the keyword parameter end="" does in Python 3.

[Paula Livingstone]

Aah great that works. I was just in the process of munging some text locally to see if I could make it work for Python3 but you've saved me the trouble. Thanks :) +1 from me..

[kRazzy R]

how do you do the same if you wanted to pass variables instead of values to the sed?