UTF-8 encoding in Volley Requests

Question

In my Android app I am loading json data with a Volley JsonArrayRequest. The data were created by myself and I saved them with Sublime with UTF-8 encoding. When I get the Response and fill my ListView, the texts are not displayed correctly (umlauts). This is what my Request looks like:

JsonArrayRequest request = new JsonArrayRequest(targetUrl,
                new Response.Listener<JSONArray>() {
                    @Override
                    public void onResponse(final JSONArray response) {
                        try {
                            fillList(response);
                        } catch (JSONException e) {
                            e.printStackTrace();
                        }
                    }
                },
                new Response.ErrorListener() {
            @Override
            public void onErrorResponse(VolleyError error) {

            }
        });
        requestQueue.add(request);

When I load the exact same data with this method, all texts are displayed correctly:

final StringBuilder builder = new StringBuilder();
        final HttpClient client = new DefaultHttpClient();
        final HttpGet httpGet = new HttpGet(request);
        try {
            final HttpResponse response = client.execute(httpGet);
            final StatusLine statusLine = response.getStatusLine();
            int statusCode = statusLine.getStatusCode();
            if (statusCode == 200) {
                final HttpEntity entity = response.getEntity();
                final InputStream content = entity.getContent();
                final BufferedReader reader = new BufferedReader(new InputStreamReader(content));
                String line;
                while ((line = reader.readLine()) != null) {
                    builder.append(line);
                }
            } else {

            }
        } catch (ClientProtocolException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }

So to me it seems like there is no problem with the encoding of my json file. How can I fix this encoding problem in Volley?

Solution 1

If you know that absolutely all of the files you are requesting will be in the UTF-8 format, which it sounds like you do, then you might consider forcing your Volley request to return UTF-8 formatted strings. You could accomplish this by subclassing the standard JSON request. Something like this:

public class Utf8JsonRequest extends JsonRequest<JSONObject> {
    ...
    @Override
    protected Response<JSONObject> parseNetworkResponse (NetworkResponse response) {
        try {
            String utf8String = new String(response.data, "UTF-8");
            return Response.success(new JSONObject(utf8String), HttpHeaderParser.parseCacheHeaders(response));
        } catch (UnsupportedEncodingException e) {
            // log error
            return Response.error(new ParseError(e));
        } catch (JSONException e) {
            // log error
            return Response.error(new ParseError(e));
        }
    }
}

Solution 2

I have same problem like this and i solve it using UTF-8 charset.

String str = "";
try {
     str = new String(strFromService.getBytes("ISO-8859-1"), "UTF-8");
} catch (UnsupportedEncodingException e) {

 e.printStackTrace();
}

String decodedStr = Html.fromHtml(str).toString();

I hope this will work for you

Solution 3

donot use try{} catch in the onResponse block, that is giving some problem in my code , rather than that you can implement like this .

@Override 
onResponse(String s) {

s= fixEncoding(s);
Toast.makeToast(this,s,Toast.LENGTH_LONG).show();

}

and i think you will get the required result

 public static String fixEncoding(String response) {
            try {
                byte[] u = response.toString().getBytes(
                        "ISO-8859-1");
                response = new String(u, "UTF-8");
            } catch (UnsupportedEncodingException e) {
                e.printStackTrace();
                return null;
            }
            return response;
        }

Solution 4

This is worked as a charm for me,, I just created a static method from @Muhammad Naeem's answer,, Thanks Muhammed..

public static String fixEncodingUnicode(String response) {
    String str = "";
    try {
        str = new String(response.getBytes("ISO-8859-1"), "UTF-8");
    } catch (UnsupportedEncodingException e) {

        e.printStackTrace();
    }

    String decodedStr = Html.fromHtml(str).toString();
    return  decodedStr;
}

Solution 5

another approach: you can encode your whole response by this line of code.

newStr = URLDecoder.decode(URLEncoder.encode(oldStr, "iso8859-1"),"UTF-8");

I have encoded whole response string into UTF-8 as I know volley default encode method is iso8859-1

View more solutions

Comments

[Matthew Pape]

I see now that you are using a JsonArrayRequest. You could subclass that instead. I don't have the syntax handy, but it would be pretty similar.

[JensJensen]

Thanks, I will give this a try!

[JensJensen]

It works. And overriding parseNetworkResponse in my StringRequest also solved the problems I had with some XML Feeds I was loading. Thanks @xaevinx

[iBobb]

Hey @JensJensen , how did this fix your problem? I tried extending both JsonArrayRequest and JsonRequest<JSONArray> and none worked. I still get "?????" as the response. When I go to the php service in Chrome json is displayed properly, however when debugging, when reading the received json it appears to be ???? so it should be Volley's fault again. It's strange that we have to override Volley's encoding as in JsonRequest class, it's using UTF8 - protected static final String PROTOCOL_CHARSET = "utf-8";

[user924]

why HttpHeaderParser.parseCharset(response.headers) isn't working correctly?