Why can't sed replace 0x24 byte in binary file?

Question

I'm want to replace some bytes in binary file to another. Created sample (6 bytes long) file via

echo -ne '\x8f\x15\x42\x02\x24\xc2' > test

Then tried to replace bytes \x15\x42\x02 to \x12\x12\x02 via sed:

sed 's \x15\x42\x02 \x12\x12\x02 g' test > test1

sed replaced bytes:

cat test test1 | xxd -c 6
0000000: 8f15 4202 24c2  ..B.$.
0000006: 8f12 1202 24c2  ....$.
           ^^ ^^^^

Tried then replace bytes \x42\x02\x24 to \x12\x02\x24:

sed 's \x42\x02\x24 \x12\x02\x24 g' test > test2

sed NOT replaced bytes:

cat test test2 | xxd -c 6
0000000: 8f15 4202 24c2  ..B.$.
0000006: 8f15 4202 24c2  ..B.$.
              ^^^^ ^^

What's wrong? I have sed (GNU sed) 4.2.2 (Kubuntu 13.10)

Thank You.

Solution 1

It is because 0x24 is the hex code for the $ character. Note that ^ and $ have special meaning in regex (which sed uses for matching), and literally means "at the beginning of the line", and "at the end of the line", respectively. As such, those characters are ignored unless escaped. Therefore,

echo "this and that" | sed 's/this$/cat/'

will leave the string unchanged because it you're telling sed to look for 'this' at the end of the line -- since 'that' is at the end, it doesn't match. However,

echo "It costs $50.00." | sed 's/\$50\.00/47.00 Swiss Francs/'

Will change the line as expected because the $ was escaped. For binary data, just include the hex code for the \ character (\x5c) just prior to \x24 in the regex section, and it should work just fine.

Cheers.

Solution 2

You can try this,

hexdump -ve '1/1 "%.2X"' file1 | sed 's/420224/121224/g' | xxd -r -p > new_updated

Test:

sat:~# xxd -c 6  file1
0000000: 8f15 4202 24c2  ..B.$.
sat:~# hexdump -ve '1/1 "%.2X"' file1 | sed 's/420224/121224/g' | xxd -r -p > new_updated
sat:~# xxd -c 6 new_updated
0000000: 8f15 1212 24c2  ....$.

Solution 3

the sed statement should be like below

sed 's/\x15\x42\x02/\x12\x12\x02/g' test > test1

their should be / between the search pattern and the pattern that should replace it

and the result should be like below

[root@localhost ~]# cat test test1 | xxd -c 6       
0000000: 8f15 4202 24c2  ..B.$.
0000006: 8f12 1202 24c2  ....$.

Comments

[Admin]

It works with / just like witn space as command/regex delimiter - still replacing \x15\x42\x02, but not replacing \x42\x02\x24

[Admin]

Thank You, but I'm interested, why sed can replace ceritain sequence of bytes bytes, and can't replace another sequence of bytes definetely presented in file. There is no place to mistake - I'd directly copy-pasted bytes from echo command to sed command.

[Saddam Abu Ghaida]

it works but you need to pass 8 instead of 6 to xdd check it out and confirm

[Admin]

sed 's/\x42\x02\x24/\x12\x02\x24/g' test > test4 ; md5sum test test4 d1c66101d794687bf5d61b5ae4b7da03 test d1c66101d794687bf5d61b5ae4b7da03 test4 - no luck, xxd is just to visualize changes

[Admin]

What OS do You using? I'm using Kubuntu 13.10 AMD64, md5sum of /bin/sed is 081c50c87d36bd90dadfb4df9483cacd. pastebin.com/wAMji5g7