What's the difference between `curl | sh` and `sh -c "$(curl)"`?

shell pipe curl docker command-substitution

25,903

Solution 1

There is a practical difference.

curl -sSL https://get.docker.com/ | sh starts curl and sh at the same time, connecting the output of curl with the input of sh. curl will carry out with the download (roughly) as fast as sh can run the script. The server can detect the irregularities in the timing and inject malicious code not visible when simply downloading the resource into a file or buffer or when viewing it in a browser.

In sh -c "$(curl -sSL https://get.docker.com/)", curl is run strictly before the sh is run. The whole contents of the resource are downloaded and passed to your shell before the sh is started. Your shell only starts sh when curl has exited, and passes the text of the resource to it. The server cannot detect the sh call; it is only started after the connection ends. It is similar to downloading the script into a file first.

(This may not relevant in the docker case, but it may be a problem in general and highlights a practical difference between the two commands.)

Solution 2

I believe that they are practically identical. However, there are rare cases where they are different.

$(cmd) gets substituted with the results of cmd. Should the length of that result command exceeds the maximum argument length value returned by getconf ARG_MAX, it will truncate the result, which may result in unpredictable results.

The pipe option does not have this limitation. Each line of output from the curl command will be executed by bash as it arrives from the pipe.

But ARG_MAX is usually in the 256,000 character range. For a docker install, I'd be confident using either method. :-)

Solution 3

In curl -sSL https://get.docker.com/ | sh:

Both commands, curl and sh, will start at the same time, in respective subshells
The STDOUT from curl will be passed as the STDIN to sh (this is what pipe, |, does)

Whereas in sh -c "$(curl -sSL https://get.docker.com/)":

The command substitution, $(), will be executed first i.e. curl will be run first in a subshell
The command substitution, $(), will be replaced by the STDOUT from curl
sh -c (non-interactive, non-login shell) will execute the STDOUT from curl

Solution 4

With pipe version you loose script interactivity, as you pipe curl stdout to sh stdin. So no keyboard input and possible script crash when awaiting input.

Solution 5

One difference between the two (taken from other answers across the web) is that if you don't download the entire script at once, it could cut off half way through the script at an unknown point and change the meaning of the command to be executed. So it seems first downloading the whole file and then evaluating it would be better.

View more solutions

25,903

Sarke

Updated on September 18, 2022

Comments

Sarke almost 2 years
One easy install method for Docker (for example) is this:
```
curl -sSL https://get.docker.com/ | sh
```
However, I have also seen some that look like this (using the Docker example):
```
sh -c "$(curl -sSL https://get.docker.com/)"
```
They appear to be functionally the same, but is there a reason to use one over the other? Or is it just a preference/aesthetic thing?

(Of note, be very careful when running script from unknown origins.)
- Wyck over 3 years
  
  Is it weird that I trust docker to install an entire software execution platform on my computer, but I don't trust the website not to put junk into my | sh command? This makes me uneasy. Like maybe I need to spend some time considering what would happen if I entered <html>internal server error</html> into sh.
Sarke over 7 years

Interesting, so there is a difference. Thanks
Sarke over 7 years

Thanks, this seems like the most important difference between the two.
hvd over 7 years

"it will truncate the result" -- The shell should issue an error message for that, not silently truncate. When testing, I even get an error from the shell far below ARG_MAX, bash limits an individual argument to 131072 bytes on my system, when getconf ARG_MAX prints 2097152. But either way, error or truncation, it wouldn't work.
AndyB over 7 years

But old implementations of Bourne shell had much lower limits. In 4.2BSD the limit was 10240 characters, and on earlier systems it was even lower.. Of course that was 30 years ago, so you're unlikely to encounter such low limits today. If I recall correctly, some of these early shells did just silently truncate.
Alfred Armstrong over 7 years

Could you cite a resource backing up this claim please? I'd be very interested in knowing how the server can detect the 'sh' call.
Jonas Schäfer over 7 years

@AlfredArmstrong Uhm, I put a link on the vulnerable to server-side detection phrase. It leads to a blog post which explains in great detail how they achieve it. TL;DR: put a sleep in your script and observe the delay in reception on the server.
Alfred Armstrong over 7 years

@JonasWielicki thanks - the link wasn't very clear - not your fault, down to SE's CSS I think. People are brilliantly sneaky, aren't they? :)
ilkkachu almost 6 years

The 128 kB limit for a single argument is a Linux thing, it's not about Bash.
ilkkachu almost 6 years

This is actually rather interesting. It might be worth noting here that to detect the pipe, the server must output a rather large amount of data (just the pipe buffer is something like 64 kB on Linux). That should be rather obvious if one saves the script to a file, or looks at it with something like less (which shows control characters and NUL bytes). Plus, if the server outputs a script of more than 128 kB, then bash -c "$(curl ...)" won't work since the kernel limits the size of a single command line argument.
ilkkachu almost 6 years

All of which leads me to wonder if anyone has ever tried to do that trick in reality without getting immediately caught. Not that it probably matters much since you could probably just have the script do something malicious even without such tricks, and anyone who doesn't read it in full would be vulnerable.
hasufell over 5 years

@JanusTroelsen what do you mean with not downloaded completely? Why would that happen except for a server error in which case there won't be anything piped to sh.
Janus Troelsen over 5 years

Or connection interruption... there are so many ways a transfer can fail