How can I use vim to convert my file to utf8?

Question

I have a text file. I've been told to make it UTF8. How can I do that with Vim?

Solution 1

If you are editing a file encoded as latin1, you will find that 'fileencoding' for that buffer is set to latin1. So you will need to manually set the fileencoding before saving the file.

:set fileencoding=utf8
:w myfilename

Also note that UTF8 files often begin with a Byte Order Mark (BOM) which indicates endianness. The BOM is optional but some programs use it exclusively to determine the file encoding. Under certain conditions Vim will write the BOM but sometimes it won't. To explicitly set the BOM do this:

:set bomb

For more information :help mbyte-options and :help utf8 and :help bomb.

Solution 2

:w ++enc=utf-8 %

to write the file in utf-8 encoding to disk.

Comments

[Benoit]

Although this is correct, your answer shouldn't have collected so many upvotes compared to Eric Johnson's. The reason is that if you don't set fileencoding, the :w ++enc=utf-8 is valid one time, but next time you run :w, the value of 'fileencoding' will be used, and if you haven't changed it (explicitly while editing, or by reloading the file, hoping that 'fencs' is set appropriately and the actual encoding is well-detected), the old encoding will come back.

[Michael Krelin - hacker]

@Benoit, I do not change fileencoding, because I am answering the question, not just telling everything I know about encodings and how vim works with them. It may take a while.

[Benoit]

@MichaelKrelin-hacker, changing fileencoding is also a valid answer to the question, which does not IMO lead to taking bad habits. But OK, that's just a matter of mood I suppose.

[Michael Krelin - hacker]

@Benoit, I didn't imply Eric's answer is not valid! If I find anything strange about his answer, it's not the content, but rather why did he post the question and the answer to his own question in rapid succession :)

[Benoit]

Oh, I hadn't seen that he also was the OP.

[0xC0000022L]

@Michael Krelin - hacker: check the FAQ. It is actually encouraged to answer your own question. It's the whole point of a Q&A page like this one. It doesn't matter who answers.

[Michael Krelin - hacker]

@0xC0000022L, last time I checked (haven't checked now), there was something about coming up with a solution after doing the research, not about posting question and answer in a minute. That said, I was only talking about what is strange about his answer, not what makes it invalid or something.

[Stein]

The instructions turn out to work fine, but: 1) utf-8 doesn't have endianness (like vim help explains, the utf-8 BOM merely indicates that the file is utf-8); 2) When you open the file again later, you need to have set fileencoding=utf-8 again up front. Vim doesn't even notice the BOM if you told it to write one (and it does indeed write it). To make utf-8 work, you need that or set encoding=utf-8 in your startup settings. At least, that's the story on my system.

[HoldOffHunger]

Great answer, upvoted! If you want to know that it worked, check with file -bi filename.txt before and after, and you'll see "text/plain; charset=utf-8" after.