How to store an image file in bash variable?

Question

After I use the following command,

pngString="$(cat example.png)"    
echo -n "$pngString" > tmp.png

I can't open the tmp.png as a PNG file. Maybe some information is lost when I use $pngString to store the image file.

So the question is: How can I store the complete image information using a variable in bash script?

Solution 1

You are right in that echo & company don't seem to handle binary that well. I suspect that the null characters break the stream all too early.

You can convert picture information in some ASCII based format. For instance, this is with base64:

$ pic=`base64 pic.jpeg`
$ echo $pic | base64 --decode > pic2.jpeg
$ diff pic*
$ echo $?
0

Solution 2

The problem is that null bytes cannot be passed through command line arguments as they are internally used as argument terminators. All other bytes seem to be fine. So a somewhat more space-efficient (typically) alternative to using base64 would be to escape the null bytes and then use printf to convert the data to original form:

pngString="$(sed 's/\\/\\\\/g;s/%/%%/g;s/\x00/\\x00/g' <example.png)"
printf "$pngString" >tmp.png

The \ and % characters are special to printf so they need to be escaped too.

Also note that if the input data ends with a newline, it will get stripped by command substitution. That should not be an issue specifically for PNGs as the last byte in a valid PNG should be 0x82, the least significant byte in the empty IEND chunk's CRC sum.

Comments

[tuzion]

The cat command works fine (to store the image data in the variable). Just remove the double quotes. The question is what you want to do with this data after saving them in the variable?

[Shadur]

cat and echo and all its ilk are at heart text utilities. Letting them loose unsuspectingly on binary files will have unpredictable results. That's why things like base64 were invented.

[Gilles 'SO- stop being evil']

Why do you want to have it as a variable? Shell variables are typically intended to store small amount of text, not large amounts of binary data. None of the usual shell can cope with binary data in variables (they choke on null bytes) except zsh.

[Wildcard]

@Shadur, cat isn't really a text utility. It's the command substitution (which strips trailing newlines), the variable assignment (which can't contain null bytes) and the echo command (which may interpret backslash sequences) which will mangle the binary, not cat. But I agree with your overall point.

[SiBrit]

In my bash system I had to enclose the variable in quotes, probably because base64 can contain white space: echo "$pic" | base64 --decode > pic2.jpeg

[Karel Vlk]

If you want even more backslashes, use: sed s/\\\\/\\\\\\\\/g\;s/%/%%/g\;s/\\x00/\\\\x00/g