Python Subprocess Error in using "cp"

Question

I was trying to use subprocess calls to perform a copy operation (code below):

import subprocess
pr1 = subprocess.call(['cp','-r','./testdir1/*','./testdir2/'], shell = True)

and I got an error saying:

cp: missing file operand

Try `cp --help' for more information.

When I try with shell=False , I get

cp: cannot stat `./testdir1/*': No such file or directory

How do I get around this problem?

I'm using RedHat Linux GNOME Deskop version 2.16.0 and bash shell and Python 2.6

P.S. I read the question posted in Problems with issuing cp command with Popen in Python, and it suggested using shell = True option, which is not working for me as I mentioned :(

Solution 1

When using shell=True, pass a string, not a list to subprocess.call:

subprocess.call('cp -r ./testdir1/* ./testdir2/', shell=True)

The docs say:

On Unix with shell=True, the shell defaults to /bin/sh. If args is a string, the string specifies the command to execute through the shell. This means that the string must be formatted exactly as it would be when typed at the shell prompt. This includes, for example, quoting or backslash escaping filenames with spaces in them. If args is a sequence, the first item specifies the command string, and any additional items will be treated as additional arguments to the shell itself.

So (on Unix), when a list is passed to subprocess.Popen (or subprocess.call), the first element of the list is interpreted as the command, all the other elements in the list are interpreted as arguments for the shell. Since in your case you do not need to pass arguments to the shell, you can just pass a string as the first argument.

Solution 2

This is an old thread now, but I was just having the same problem.

The problem you were having with this call:

subprocess.call(['cp','-r','./testdir1/*','./testdir2/'], shell = False)

was that each of the parameters after the first one are quoted. So to the shell sees the command like this:

cp '-r' './testdir1/*' './testdir2/'

The problem with that is the wildcard character (*). The filesystem looks for a file with the literal name '*' in the testdir1 directory, which of course, is not there.

The solution is to make the call like the selected answer using the shell = True option and none of the parameters quoted.

Solution 3

I know that the option of shell=True may be tempting but it's always inadvisable due to security issues. Instead, you can use a combination of the subprocess and glob modules.

For Python 3.5 or higher:

import subprocess
import glob

subprocess.run(['cp', '-r'] + glob.glob('./testdir1/*') + ['./testdir2/'])

For Python 3.4 or lower:

import subprocess
import glob

subprocess.call(['cp', '-r'] + glob.glob('./testdir1/*') + ['./testdir2/'])

Comments

[desmond13]

What If I want to do a call like this: subprocess.call(["cp","-r", "/home/user/logs", directory_logs]) where directory_logs is a string in which I have an existing directory. How can I use the shell=True option in this case?

[Antonio Serrano]

The option shell=True is strongly discouraged due to security concerns. I posted a safer alternative below.

[tripleee]

@desmond13 You have no wildcards or other shell features in that command, so absolutely no reason to use shell=True.