How to remove this symbol "^@" with vim?

Question

I have some files that are corrupted with this symbol:

^@

It's not part of the string; it's not searchable. How do I substitute this symbol with nothing, or how do I delete this symbol?

Here is an example line from one file:

^@F^@i^@l^@e^@n^@a^@m^@e^@ ^@ ^@ ^@ ^@ ^@ ^@ ^@ ^@ ^@ ^@:^@ ^@^M^@

Solution 1

You could try:

• %s/<CTRL-2>//g (on regular PCs)

• %s/<CTRL-SHIFT-2>//g (on Mac PCs)

where <CTRL-2> means first press down the CTRL on regular PCs, keeping it as pressed down, hit 2, release CTRL.

and <CTRL-SHIFT-2> means first press down the control on Mac PCs, keeping it as pressed down, press down shift on Mac PCs, keeping it as pressed down, hit 2, release control and shift.

Finally, both of the two commands should result in %s/^@//g on screen. ^@ means a single character (a NULL byte, which otherwise couldn’t be displayed), not ^ followed by @, so you can't just type ^ and @ in a row in the above command.

This command removes all the ^@.

Solution 2

I don't think your files are corrupted. Your example line looks like it contains regular text with null bytes between each character. This suggests it's a text file that's been encoded in UTF-16 but the byte-order mark is missing from the start of the file. See http://en.wikipedia.org/wiki/Byte-order_mark

Suppose I open Notepad, type the word 'filename', and save as Unicode Big-endian. A hex dump of this file looks like this:

fe ff 00 66 00 69 00 6c 00 65 00 6e 00 61 00 6d 00 65

If I open this file in Vim it looks fine - the 'fe ff' bytes tell Vim how the file is encoded. Now suppose I create a file containing the exact same sequence of bytes, but without the leading 'fe ff'. Vim inserts ^@ (or <00>, depending on your config), in place of the null bytes; Notepad inserts spaces.

So rather than remove the nulls, you should really be looking to get Vim to interpret the file correctly. You can get Vim to reload the file with the correct encoding with the command:

:e ++enc=utf16

Solution 3

This actually worked for me within vim:

:%s/\%x00//g

Solution 4

That 'symbol' represents a NULL character, with ASCII value 000.

It's difficult to remove with vim, try

tr -d '\000' < file1 > file2

Solution 5

As others have noted, those are null bytes (ASCII 00). On Linux, the way to enter ASCII values into vim is to press Ctrl-V followed by the 3-digit octal value of any character. To replace all null bytes, use:

    :%s/Ctrl-V000//g

(with no spaces).

Likewise, you can search for nulls with:

    /Ctrl-V000

In both cases, it won't show the zeros as you're typing them, but after entering all three, it will display ^@. On color terminals it will show that in blue to indicate that it's a control character.

View more solutions

Comments

[mrt181]

Yes, the last command made vim interpret the file correctly but does not remove the nullbytes.

[scy]

To remove them, choose another encoding and save the file again: :set fenc=utf-8

[arayinfree]

this works with substitute(), but Ctl-VCtl-Shift-2 does not.

[Jeff B]

Same problem for me, I couldn't get <Ctrl-V><Ctrl-2> (as well as the one with <Ctrl-Shift-2>) to work either, but this worked.

[George]

@Mario could you tell us more about the encoding change? Is it something related to jrb's answer below?

[Mario]

See rpyzh's answer further down below. Shows loading the file using the proper encoding as well as saving it with a different one (although the answer could need some more explanation). Jrb's last note is enough if you just want to read it, but not if you want to have it saved without the null bytes using another encoding.

[Andre Albuquerque]

Precious info. In my case it was the endianness of the BOM byte.

[H. Ferrer]

This works for me linux. '00' is the ASCII hex value, which you can find for any character in vim by placing the cursor over it and typing 'ga' (think "get ascii) in command mode or :as / :ascii on the command line. vim.wikia.com/wiki/…

[Edward Falk]

^Vx00 also works. You can also enter 16-bit unicode with ^VuXXXX. I tried \%uXXXX in a search and that also worked.

[Gonzalo Cao]

You will be my beloved man up to the end of time. From the deep of my heart...thank you!

[L29Ah]

Why do you think it's a newline character?

[XoXo]

for a large file (~3GB), this is much faster than the vim solution of :%s/\%x00//g

[StevenWernerCS]

doesnt work on windows. using cygwin use jriggins's answer below instead