<0xEF,0xBB,0xBF> character showing up in files. How to remove them?

Question

I am doing compressing of JavaScript files and the compressor is complaining that my files have  character in them.

How can I search for these characters and remove them?

Solution 1

perl -pi~ -CSD -e 's/^\x{fffe}//' file1.js path/to/file2.js

I would assume the tool will break if you have other utf-8 in your files, but if not, perhaps this workaround can help you. (Untested ...)

Edit: added the -CSD option, as per tchrist's comment.

Solution 2

You can easily remove them using vim, here are the steps:

1) In your terminal, open the file using vim:

vim file_name

2) Remove all BOM characters:

:set nobomb

3) Save the file:

:wq

Solution 3

Another method to remove those characters - using Vim:

vim -b fileName

Now those "hidden" characters are visible (<feff>) and can be removed.

Solution 4

Thanks for the previous answers, here's a sed(1) variant just in case:

sed '1s/^\xEF\xBB\xBF//'

Solution 5

On Unix/Linux:

sed 's/\xEF\xBB\xBF//' < inputfile > outputfile

On MacOSX

sed $'s/\xEF\xBB\xBF//' < inputfile > outputfile

Notice the $ after sed for mac.

On Windows

There is Super Sed an enhanced version of sed. For Windows this is a standalone .exe, intended for running from the command line.

View more solutions