How do I decode a list of base64-encoded file names?

Question

I have a list of base64-encoded file names in the pattern of {base64-encoded part here}_2015-11-12.pdf. I'm trying to decode that list of files and return it on the command line as another list, separated by newlines. Here's what I'm trying now:

find . -name "*_*" -printf "%f\0" | sed 's/_....-..-..\.pdf//g' | xargs -0 -i echo "{}" | base64 -d

I think what I'm doing here is . . .

1. finding the files, printing out only the file's name (i.e., stripping off the "./" prefix) separated by a null character
2. using sed to preserve only the base64-encoded part (i.e., removing the _2015-11-12.pdf part of the file's name)
3. using xargs to ostensibly pass each file name to echo
4. then decoding the value returned by echo.

The result of that is apparently a big string of all of the base64-decoded file names, each name separated by a null character, with the entire string followed by a newline. The desired result would be each individual decoded file name on a line by itself.

I've tried all kinds of tricks to try and fix this but I haven't found anything that works. I've tried ... | base64 -d | echo, ... | base64 -d && echo, etc., trying to insert a newline at various points along the way. It seems like by the time the values end up at | base64 -d, they are all processed at once, as a single string. I'm trying to find a way to send each value to base64 -d one at a time, NOT as a monolithic list of file names.

Solution 1

Just add the base64 encoding of newline (Cg==) after each file name and pipe the whole thing to base64 -d:

find . -name "*_*" -printf "%f\n" |
  sed -n 's/_....-..-..\.pdf$/Cg==/p' |
  base64 -d

With your approach, that would have to be something like:

find . -name "*_*" -printf "%f\0" |
  sed -zn 's/_....-..-..\.pdf$//p' |
  xargs -r0 sh -c '
    for i do
      echo "$i" | base64 -d
    done' sh

as you need a shell to create those pipelines. But that would mean running several commands per file which would be quite inefficient.

Solution 2

One trick is to encode \n in base64 ... so it becomes Cg== this you can append to the printf-command. A '\' cannot be in a filename. So in the end you can sed it back

find . -name "*_*" -printf "%f\0Cg==" | sed 's/_....-..-..\.pdf//g' | xargs -0 -i echo "{}" | base64 -d | sed 's/\\n/\n/g'

Solution 3

Bash has a nifty here-document <<<, that will let you pipe multiple lines using xargs.

❯ echo -e "T0sK\nT0sK" | xargs -n1 bash -c 'base64 -d <<< $1' _
OK
OK

Comments

[yorkshiredev]

XG4= encodes \n as two separate characters, which is \\n once escaped. Cg== is the right base64 encoding for a newline character (ASCII 0x0A).

[Caner Bacaksız]

Nice. Nearly same idea, but you did it better than me!

[Stéphane Chazelas]

Never embed {} in the code argument of the shell (or any interpreter for that matters), that introduces a command injection vulnerability.

[Stéphane Chazelas]

You're using \0 for find and xargs, but sed processes \n delimited records.

[Ryan]

-i is equivalent to -I{}?

[Ryan]

The null delimiter was from the original example. I've removed it because it's not necessary to illustrate the example.