Android JSON CharSet UTF-8 problems

Question

I am currently working on an Android application, which gets its data from an API which provides JSON data. I am storing items for the JSON data as Strings which is resulting in some odd character appearing (e.g. 'Â'). I understand this it to do with the character set, hence I have set the InputStreamReader to "UTF-8" but it does not seem to have solved the problem.

       URL hukd = new URL(requestUrl);
        URLConnection tc = hukd.openConnection();
        BufferedReader in = new BufferedReader(new InputStreamReader(tc.getInputStream(), "UTF-8"));
        String line = in.readLine();
        Log.d("line :", line);
        JSONObject obj = new JSONObject(line);
        JSONArray ja = obj.getJSONObject("deals").getJSONArray("items");
        for (int i = 0; i < ja.length(); i++) { // each deal
            JSONObject jo = (JSONObject) ja.get(i);
            // make the deal
            Deal d = new Deal(jo.getString("title"),
                    jo.getString("description"),
                    jo.getString("price"),jo.getString("temperature"), 
                    jo.getJSONObject("merchant").getString("name"),
                    jo.getString("deal_image"),
                    jo.getString("deal_image_highres"));
            listItems.add(d);
            Log.d("Deal:", d.toString());
        }

Log.d "line" gives

01-30 19:56:01.909: D/line :(610): {"deals":{"items":[{"title":"Absolute steal ** Harmony one remote in store only Comet \u00c2\u00a349.98 **","deal_link":"http:\/\/www.hotukdeals.com\/deals\/absolute-steal-harmony-one-remote-store-only-comet-49-98-1131779?aui=465","mobile_deal_link":"http:\/\/m.hotukdeals.com\/deals\/absolute-steal-harmony-one-remote-store-only-comet-49-98-1131779?aui=465","deal_image":"http:\/\/www.hotukdeals.com\/images\/threads\/1131779_1.jpg","description":"Was just in comet getting two new washing machines when I stumbled upon a load of Harmony ones on the end of an isle \u00e2\u0080\u00a6. I nearly blew a fuse when I saw they had been reduced from \u00c2\u00a380.00 to under \u00c2\u00a350 as I just bought 3 of these before xmas at \u00c2\u00a368.00 ea which I thought was a true bargain.\r\n\r\nI asked they guy and he said they were stopping doing them as far as he knew and that the deal should be nation wide this must be the  cheapest I've seen for the UK version ever  even though this is the Harmony one and not the one + is it still a great remote which does just about everything you could want it to. \r\n\r\n** note I don\u00e2\u0080\u0099t have a picture of the price tag but I have these at my home and are 100% that it\u00e2\u0080\u0099s the harmony one on sale hope this helps some of you guys  and girls out  **\r\n\r\nThe Logitech Harmony One remote control lets you replace 15 remotes with one, easy to use device. With its clear, touch screen display this Harmony learning remote control makes controlling the most complex of systems easy. A graphic interface allows you, for example, to watch a DVD at the touch of a button.\r\n","submit_time":"48 minutes ago","hot_time":"1 minute ago","poster_name":"no1son","temperature":106.88999939,"price":49.98,"timestamp":1327950390,"expired":"false","forum":{"name":"Deals","url_name":"deals"},"category":{"name":"Audiovisual","url_name":"audiovisual"},"merchant":{"name":"Comet","url_name":"comet.co.uk"},"tags":{"items":[{"name":"leeds"}]},"deal_image_highres":"http:\/\/www.hotukdeals.com\/images\/threads\/high-res\/1131779_1.jpg"},{"title":"Youth Brazil Shirt \u00c2\u00a34 at Very\/Littlewoods","deal_link":"http:\/\/www.hotukdeals.com\/deals\/youth-brazil-shirt-4-very-littlewoods-1131765?aui=465","mobile_deal_link":"http:\/\/m.hotukdeals.com\/deals\/youth-brazil-shirt-4-very-littlewoods-1131765?aui=465","deal_image":"http:\/\/www.hotukdeals.com\/images\/threads\/1131765_1.jpg","description":"2010 Brazil shirt by Nike\r\nAvailable in all youth sizes.\r\nFree delivery through collect plus","submit_time":"1 hour, 4 minutes ago","hot_time":"14 minutes ago","poster_name":"ericagradus","temperature":161.479995728,"price":4,"timestamp":1327949447,"expired":"false","forum":{"name":"Deals","url_name":"deals"},"category":{"name":"Fashion","url_name":"fashion"},"merchant":{"name":"Very","url_name":"very.co.uk"},"tags":{"items":[{"name":"brazil shirt"},{"name":"very"},{"name":"littlewoods"}]},"deal_image_highres":"http:\/\/www.hotukdeals.com\/images\/threads\/high-res\/1131765_1.jpg"},{"title":"Milk Chocolate Mikado 29p at Home Bargains","deal_link":"http:\/\/www.hotukdeals.com\/deals\/milk-chocolate-mikado-29p-home-bargains-1130742?aui=465","mobile_deal_link":"http:\/\/m.hotukdeals.com\/deals\/milk-chocolate-mikado-29p-home-bargains-1130742?aui=465","deal_image":"http:\/\/www.hotukdeals.com\/images\/threads\/1130742_1.jpg","description":"Was in Home Bargains in Lancaster and they had the MIlk Chocolate Mikado for 29p. The white and hazelnut were 59p. Didn't check the sell by day as it took all my will power to resist haha!\r\n\r\nSorry if this is a duplicate I did look for it but couldn't find it.","submit_time":"1 day, 3 hours ago","hot_time":"14 minutes ago","poster_name":"cazi77","temperature":146.61000061,"price":0.29,"timestamp":1327853963,"expired":"false","forum":{"name":"Deals","url_name":"deals"},"category":{"name":"Groceries","url_name":"groceries"},"merchant":{"name":"Home Bargains","url_name":"homebargains.co.uk"},"tags":{"items":[]},"deal_image_highres":"http:\/

Any ideas?

Solution

Try the following snippet code which helped me when I was having a similar problem:

new String(jo.getString("name").getBytes("ISO-8859-1"), "UTF-8");   

Comments

[Bilbo Baggins]

why do we have to use such a twisted function? It has to be enclosed in a try catch block.

[vm204]

I personally like to enclose all of my json parsing inside of try catch blocks just in case.