Convert binary mode to text mode and the reverse option

Question

I converted a simple binary file into a text file with:

od –t x1 Check.tar | cut –c8- > Check.txt

Which gives a content similar to:

 64 65 76 2f 6e 75 6c 6c 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 [...]

What is the opposite way -- to convert Check.txt to Check.tar as the original file?

Solution 1

od -An -vtx1 Check.tar > Check.txt

You need -v or od will condense sequences of identical bytes.

For the reverse:

LC_ALL=C tr -cd 0-9a-fA-F < Check.txt | xxd -r -p > Check.tar

Or:

perl -ape '$_=pack "(H2)*", @F' Check.txt > Check.tar

If your purpose is to transfer files over a channel that only supports ASCII text, then there are dedicated tools for that like uuencode:

tar cf - myfiles.* | xz | uuencode myfiles.tar.xz | that-channel 

And to recover those files on the other end:

uudecode < file.uu

would recreate myfiles.tar.xz.

Or:

uudecode -o - < file.uu | xz -d | tar xf -

To extract the files.

Solution 2

Answering the X part of this XY problem, I would recommend you investigate the reason your binary file transfers don't transfer properly.

If it turns out the reason is because you don't have an 8-bit clean datapath you could then use existing tools that were created to handle this situation, such as base64 or even uuencode. Old but still very effective.

tar czvf - /etc/h* | base64 >/tmp/tar.tgz.b64
ls -l /tmp/tar.tgz.b64
-rw-r--r-- 1 root root 7364 May 26 11:52 /tmp/tar.tgz.b64
...
base64 -d /tmp/tar.tgz.b64 | tar tzvf -

or

tar czvf - /etc/h* | uuencode - >/tmp/tar.tgz.uue
ls -l /tmp/tar.tgz.uue
-rw-r--r-- 1 root root 7530 May 26 11:51 /tmp/tar.tgz.uue
...
uudecode /tmp/tar.tgz.uue | tar xzvf -

Solution 3

In my case I didn't have xxd or uudecode on the remote device but I did have bash. I ended up with the following:

Convert from binary to txt with:

od -An -vtx1 myfile.bin > myfile.txt

Then convert back from txt to binary with:

while read p; do
    IFS=' ' read -r -a array <<< "$p" 
    for index in "${!array[@]}" 
    do
        echo -en "\x${array[index]}" 
    done
done < myfile.txt > myfile.bin

Comments

[cuonglm]

Maybe xxd -r -p Check.txt > Check.tar

[maihabunash]

its still text file , isnt work

[Stephen Kitt]

What leads you to believe it's still a text file? xxd -r -p is the exact reverse of the od conversion you did; the output of cuonglm's command should be strictly identical to the original tarball.

[maihabunash]

xxd -r -p Check.txt > Check.tar ..... file Check.tar Check.tar: ascii text

[maihabunash]

I also try this - tar xvf Check.tar tar: directory checksum error

[ctrl-alt-delor]

for me this test echo hello world | od -t x1 | xxd -r -p produces garbage. But echo hello world | od -t x1 | xxd -r works.

[Janis]

@richard; for me both od/xxd versions create garbage (the latter creates spurious characters after the correctly decoded data).

[Janis]

@maihabunash; Note that it's not good to use different tools mixed for encoding and decoding. There were already answers suggesting base64 and uuencode, and of course if you use xxd for encoding (instead of od | cut) then xxd will correctly work also for decoding.

[maihabunash]

first thanx for the answer , but I need to create file.tar file , how to do it?

[Stéphane Chazelas]

@maihabunash If you created file.txt without -v and with removing the address, then you can't reliably recover file.tar if there were condensed sequences (do a grep '[*]' file.txt to check) as you've lost the information of how long those condensed sequences were by removing the address.

[maihabunash]

or maybe there are more option for example to compress the 30 scripts but as ascii or text mode , I need that because when I copy from one machine to another machine from unclear reason the binary files are changes ( chksum ) and not ascii files

[Stéphane Chazelas]

@maihabunash, you're looking for uuencode or base64 encoding. Note that my answer covers your question. I give the code to convert back to binary from od output provided you don't forget the -v option. If you're transferring files over FTP, don't forget to set the mode to "binary" (TYPE I FTP command, something like binary in your client)

[Stéphane Chazelas]

@richard, tar will add NUL characters which make them non-text. In any case, I suspect the mangling in transfer problem is with characters like LF/CR, more than with bytes with the 8th bit set (all of which are taken care of by using uuencode).

[maihabunash]

Upvote +1 for you thx

[maihabunash]

Upvote +1 for you thx

[mwfearnley]

+1 for suggesting base64, which works on my relatively fresh cygwin install.