Java properties UTF-8 encoding in Eclipse

Question

I've recently had to switch encoding of webapp I'm working on from ISO-xx to utf8. Everything went smooth, except properties files. I added -Dfile.encoding=UTF-8 in eclipse.ini and normal files work fine. Properties however show some strange behaviour.

If I copy utf8 encoded properties from Notepad++ and paste them in Eclipse, they show and work fine. When I reopen properties file, I see some Unicode characters instead of proper ones, like:

Zur\u00EF\u00BF\u00BDck instead of Zurück

but app still works fine. If I start to edit properties, add some special characters and save, they display correctly, however they don't work and all previously working special characters don't work any more.

When I compare local version with CVS I can see special characters correctly on remote file and after update I'm at start again: app works, but Eclipse displays Unicode chars.

I tried changing file encoding by right clicking it and selecting „Other: UTF8” but it didn't help. It also said: „determined from content: ISO-8859-1”

I'm using Java 6 and Jboss Developer based on Eclipse 3.3

I can live with it by editing properties in Notepad++ and pasting them in Eclipse, but I would be grateful if someone could help me with fixing this in Eclipse.

Solution 1

Don't waste your time, you can use Resource Bundle plugin in Eclipse

Old Sourceforge page

Solution 2

Answer for "pre-Java-9" is below. As of Java 9, properties files are saved and loaded in UTF-8 by default, but falling back to ISO-8859-1 if an invalid UTF-8 byte sequence is detected. See the Java 9 release notes for details.

---

Properties files are ISO-8859-1 by definition - see the docs for the Properties class.

Spring has a replacement which can load with a specified encoding, using PropertiesFactoryBean.

EDIT: As Laurence noted in the comments, Java 1.6 introduced overloads for load and store which take a Reader/Writer. This means you can create a reader for the file with whatever encoding you want, and pass it to load. Unfortunately FileReader still doesn't let you specify the encoding in the constructor (aargh) so you'll be stuck with chaining FileInputStream and InputStreamReader together. However, it'll work.

For example, to read a file using UTF-8:

Properties properties = new Properties();
InputStream inputStream = new FileInputStream("path/to/file");
try {
    Reader reader = new InputStreamReader(inputStream, "UTF-8");
    try {
        properties.load(reader);
    } finally {
        reader.close();
    }
} finally {
   inputStream.close();
}

Solution 3

It is not a problem with Eclipse. If you are using the Properties class to read and store the properties file, the class will escape all special characters.

From the class documentation:

When saving properties to a stream or loading them from a stream, the ISO 8859-1 character encoding is used. For characters that cannot be directly represented in this encoding, Unicode escapes are used; however, only a single 'u' character is allowed in an escape sequence. The native2ascii tool can be used to convert property files to and from other character encodings.

From the API, store() method:

Characters less than \u0020 and characters greater than \u007E are written as \uxxxx for the appropriate hexadecimal value xxxx.

Solution 4

Properties props = new Properties();
URL resource = getClass().getClassLoader().getResource("data.properties");         
props.load(new InputStreamReader(resource.openStream(), "UTF8"));

Works like a charm

:-)

Solution 5

Properties props = new Properties();
URL resource = getClass().getClassLoader().getResource("data.properties");         
props.load(new InputStreamReader(resource.openStream(), "UTF8"));

this works well in java 1.6. How can i do this in 1.5, Since Properties class does not have a method to pars InputStreamReader.

View more solutions

Comments

[Laurence Gonsalves]

In Java 1.6 you can use other encodings by using the methods that use Reader/Writer instead of InputStream/OutputStream.

[chesterbr]

This is a hugely useful plugin, thanks for the tip! So bad it does not have an install URL, but just dropping it on the plugins folder does the trick.

[Admin]

Unfortunately props.load, in 1.6, requires an InputStream and specifically indicates it expects old-school ISO-8859-1

[Ivaylo Slavov]

Does this use the native2ascii tool for non-unicode characters internally or I should manually escape unicode symbols?

[Radu Murzea]

A more general solution than the one in the accepted answer, therefore better :)

[Chrissi]

NetBeans nicely displays properties files that have \uXXXX escapes and lets you edit them with UTF characters properly displayed. Why doesn't Eclipse? In my opinion this is a problem with Eclipse.

[Sefran2]

@baybora.oren: I expanded the .zip in the eclipse folder plugin, but I don't see any change in the ide (I shutdown and restart eclipse). Any hint?

[baybora.oren]

@cricket check eclipse version maybe it is not compatible with that eclipse you use

[rexford]

+1: the so quantum of solace for having been skeet'd

[Fering]

@JonSkeet I have been working on this problem for weeks. Thank you so much!

[Rune Aamodt]

As of Java 9+, properties files should be encoded in UTF-8: docs.oracle.com/javase/9/intl/…

[Jon Skeet]

@RuneAamodt: Although I'm glad to see ISO-8859-1 is still supported when reading. Will update my answer.