Windows-1252 encoding - incorrect characters displayed

Question

I have a buffer with chars encoded in Windows-1252. However when I create a new String with appropriate encoding, instead of expected result I've get quite often interrogation marks, ex.

byte[] tmps = new byte[] {(byte) 0xfb};
System.out.println (new String (tmps,0,1,"Windows-1252" ));

As result the system should display "u" char with "^" above it. Instead it displays "?".

Any idea?

Solution 1

First of all Windows-1252 is a supported encoding:

• If it wasn't you'd get an UnsupportedEncodingException in new String (...,"Windows-1252"). (That's what the javadoc says!)

• The Oracle Java documentation say Windows-1252 is in the "Basic Encoding Set" - http://docs.oracle.com/javase/7/docs/technotes/guides/intl/encoding.doc.html, http://docs.oracle.com/javase/6/docs/technotes/guides/intl/encoding.doc.html, etcetera.

I think that the most likely problem here is on the output side. Specifically, Java may think that your locale's default charset is ASCII or something that doesn't support that codepoint.

One way to eliminate Windows-1252 as the cause of the problem is to write the equivalent string using a Unicode escape; e.g.

    System.out.println("\u00fb");

Solution 2

I've already found this.

Menu Run/Run configurations/ next Java Application and your own app name/tab common/ next encoding set to UTF-8

And since now both windows 1250 and 1252 chars seems to be displayed ok.

Comments

[user2707175]

After a bit deeper investigation, it's clear that the problem is with displaying chars in console (eclipse) as the string converted to integer values char by char gives the correct values. Moreover when debugging and displaying the value of String variable, the "u" with "^" is displayed correctly. Thus it's for sure the problem with console, but how to fix it?

[user2707175]

And java run directly from command line (without Eclipse) have the same problem with displaying. :( Surprisingly Windows-1250 chars are displayed correctly. I tried to change the font used for cmd window but it's the same story. How to solve this?

[user2707175]

What helps is "java -Dfile.encoding=cp1252 MyClassNameWithMain" but it's not the solution as I need to have full Unicode - two different languages displayed in the console.

[Stephen C]

If your console is UTF-8 capable, you should change the relevant system settings to force the console to use UTF-8. Then set the Java default encoding to UTF-8.

[Wienczny]

You should have a look at docs.oracle.com/javase/8/docs/technotes/guides/intl/… the name may be different