Apply SHA256 and Base64 to string in script

Question

I'm trying to apply SHA256 and then Base64 encode a string inside a shell script. Got it working with PHP: php -r 'echo base64_encode(hash("sha256", "asdasd", false));'. But I'm trying to get rid of the PHP dependency.

Got this line that works well in the terminal (using the fish shell):

$ echo -n "asdasd" | shasum -a 256 | cut -d " " -f 1 | xxd -r -p | base64
X9kkYl9qsWoZzJgHx8UGrhgTSQ5LpnX4Q9WhDguqzbg=

But when I put it inside a shell script, the result differs:

$ cat foo.sh
#!/bin/sh
echo -n "asdasd" | shasum -a 256 | cut -d " " -f 1 | xxd -r -p | base64
$ ./foo.sh
IzoDcfWvzNTZi62OfVm7DBfYrU9WiSdNyZIQhb7vZ0w=

How can I make it produce expected result? My guess is that it's because of how binary strings are handled?

Solution

The problem is that you are using different shells. The echo command is a shell builtin for most shells and each implementation behaves differently. Now, you said your default shell is fish. So, when you run this command:

~> echo -n "asdasd" | shasum -a 256 | cut -d " " -f 1 | xxd -r -p | base64
X9kkYl9qsWoZzJgHx8UGrhgTSQ5LpnX4Q9WhDguqzbg=

you will get the output shown above. This is because the echo of fish supports -n. Apparently, on your system, /bin/sh is a shell whose echo doesn't support -n. If the echo doesn't understand -n, what is actually being printed is -n asdasd\n. To illustrate, lets use printf to print exactly that:

$ printf -- "-n asdasd\n" 
-n asdasd

Now, if we pass that through your pipeline:

$ printf -- "-n asdasd\n" | shasum -a 256 | cut -d " " -f 1 | xxd -r -p | base64
IzoDcfWvzNTZi62OfVm7DBfYrU9WiSdNyZIQhb7vZ0w=

Thats the output you get from your script. So, what happens is that echo -n "asdasd" is actually printing the -n and a trailing newline. A simple solution is to use printf instead of echo:

$ printf "asdasd" | shasum -a 256 | cut -d " " -f 1 | xxd -r -p | base64
X9kkYl9qsWoZzJgHx8UGrhgTSQ5LpnX4Q9WhDguqzbg=

The above will work the same on the commandline and in your script and should do so with any shell you care to try. Yet another reason why printf is better than echo.

Comments

[Niklas Berglund]

@terdon I'm using fish shell. If I launch bash($ bash) and run the script the output is the same though.

[terdon]

OK, but what's the output of readlink -f /bin/sh? What shell is your sh?

[Niklas Berglund]

@terdon I'm on OS X so can't use readlink's -f flag. But greadlink -f /bin/sh is saying /bin/sh

[terdon]

OK, in that case, the sh implementation that OSX ships with has an echo that doesn't understand -n. Unlike your default fish shell's.

[Niklas Berglund]

You're right. Using printf instead of echo -n solves my issue. Thank you!

[Niklas Berglund]

Moving away from echo and instead using printf does seem like the way to go. Two other solutions could be to make the script use Bash #!/bin/bash or invoke echo with the absolute path /bin/echo: apple.stackexchange.com/a/173837/50639

[terdon]

@NiklasBerglund yes, but even /bin/echo can be different on different machines. Always use printf for maximum portability. Read the link at the end of my answer for more than you ever wanted to know about why echo should be avoided.