Java InputStream encoding/charset

Question

Running the following (example) code

import java.io.*;

public class test {
    public static void main(String[] args) throws Exception {
        byte[] buf = {-27};
        InputStream is = new ByteArrayInputStream(buf);
        BufferedReader r = new BufferedReader(
                new InputStreamReader(is, "ISO-8859-1"));
        String s = r.readLine();
        System.out.println("test.java:9 [byte] (char)" + (char)s.getBytes()[0] + 
                " (int)" + (int)s.getBytes()[0]);
        System.out.println("test.java:10 [char] (char)" + (char)s.charAt(0) + 
                " (int)" + (int)s.charAt(0));
        System.out.println("test.java:11 string below");
        System.out.println(s);
        System.out.println("test.java:13 string above");
    }
}

gives me this output

test.java:9 [byte] (char)? (int)63
test.java:10 [char] (char)? (int)229
test.java:11 string below
?
test.java:13 string above

How do I retain the correct byte value (-27) in the line-9 printout? And consequently receive the expected output of the System.out.println(s) command (å).

Solution 1

If you want to retain byte values, don't use a Reader at all, ideally. To represent arbitrary binary data in text and convert it back to binary data later, you should use base16 or base64 encoding.

However, to explain what's going on, when you call s.getBytes() that's using the default character encoding, which apparently doesn't include Unicode character U+00E5.

If you call s.getBytes("ISO-8859-1") everywhere instead of s.getBytes() I suspect you'll get back the right byte value... but relying on ISO-8859-1 for this is kinda dirty IMO.

Solution 2

As noted, getBytes() (no-arguments) uses the Java platform default encoding, which may not be ISO-8859-1. Simply printing it should work, provided your terminal and the default encoding match and support the character. For instance, on my system, the terminal and default Java encoding are both UTF-8. The fact that you're seeing a '?' indicates that yours don't match or å is not supported.

If you want to manually encode to UTF-8 on your system, do:

String s = r.readLine();
byte[] utf8Bytes = s.getBytes("UTF-8");

It should give a byte array with {-61, -91}.