How do you write specific bytes to a file?

Question

Given a file myfile with the following contents:

$ cat myfile
foos

A hexdump of the file gives us the contents:

$ hexdump myfile
6f66 736f 000a

Currently, I can create the file by specifying the contents in ascii like this:

$ echo foos > myfile

Is it possible to create the file by giving it the exact bytes in hexadecimal rather than ascii?

$ # How can I make this work?
$ echo --fake-hex-option "6f66 736f 000a" > myfile
$ cat myfile
foos

Update: For clarity's sake, I worded the question to ask how to write a small number of bytes to a file. In reality, I need a way to pipe a large amount of hexadecimal numbers directly into a file rather than just 3 bytes:

$ cat hexfile
6f66 736f 6f66 736f ...
$ some_utility hexfile > myfile
$ cat myfile
foosfoosfoosfoos...

Solution 1

This is the hexundump script from my personal collection:

#!/usr/bin/env perl
$^W = 1;
$c = undef;
while (<>) {
    tr/0-9A-Fa-f//cd;
    if (defined $c) { warn "Consuming $c"; $_ = $c . $_; $c = undef; }
    if (length($_) & 1) { s/(.)$//; $c = $1; }
    print pack "H*", $_;
}
if (!eof) { die "$!"; }
if (defined $c) { warn "Odd number of hexadecimal digits"; }

Solution 2

You can use echo -e:

echo -e "\x66\x6f\x6f"

Do note that hexdump -C is what you want to dump the contents of the file in byte order instead of being interpreted as 4-byte words in network byte order.

Solution 3

Simulate a byte train:

echo 41 42 43 44 | 

Change spaces into newlines so the while/read can easily parse them on by one

tr ' ' '\n' | 

Parse byte by byte

while read hex; do

Convert hex to ascii:

  printf \\x$hex

until end of input

done

If the files to parse are seriously big, you probably don't want to use bash because it is slow. PERL for example would be a better choice.

Solution 4

The *some_utility* you're finding is dd (manual). You can copy any valid bytes from any valid position of a file to another file by specify bs(block size), count, and skip options.

Example

Copy the first 1024 bytes of a file to another file.

$ dd if=liveusb-creator-3.11.7-setup.exe of=test.ex_ bs=1 count=1024
1024+0 records in
1024+0 records out
1024 bytes (1.0 kB) copied, 0.03922 s, 26.1 kB/s

Solution 5

To write arbitrary hex data to binary file:

echo -n 666f6f | xxd -r -p - file.bin

For hex (input) data stored in some file to be written to binary file:

xxd -r -p file.hex file.bin

To read binary data:
hd file.bin or xxd file.bin

To read data only (without offsets):

xxd -p file.bin

View more solutions

Comments

[Admin]

I think you might like the -C flag for hexdump. hexdump -C displays a byte by byte output instead of 16 bit words obfuscated up by endianess.

[Cory Klein]

Although this does answer my question, it doesn't help in my situation. I have a very large input of hexadecimal numbers that I would like to pipe into a file. I will update the question with this.

[Cory Klein]

That's a pretty impressive 3-liner. I'd be interested if you cared to elaborate on that regex!

[mikeserv]

@CoryKlein - the regex isn't really what matters - sed just inserts a P after every two not-space chars and appends the string []p to every line. It is dc that is interesting. First echo sets its input radix to 16 (so it will interpret the hex char pairs as numbers) then for every byte it reads in it Prints its (UCHAR_MAX+1) value to stdout and pops it off the stack. Try echo 8i141P12PAi97P10P16i61P0AP |dc and see what you get.

[Cory Klein]

With that, I get a\na\na, is that supposed to be what I see?

[mikeserv]

@CoryKlein - yeah. It sets the input radix to octal then prints the ascii char for the octal bytes 141 and 12 - then sets it to A (10 - decimal) and prints the ascii chars for 97 and 10, then sets it to hex and prints the ascii chars for 61 and 0A. All of those char pairs are a\n.