How to get a UTF-8 JSON

Question

I'm using LitJSON library but things gets a little bit odd.

Do you know any JSON library that keeps the accents when converting ?

Here's the test :

test.json

[{"id":"CS_001","name":"L'élément","type":"Tôt"},{"id":"CS_002","name":"L'outrage","type":"Tôt"},{"id":"CS_003","name":"Test","type":"Tôt"}]

test.cs

public class test : MonoBehaviour {
    private string jsonString;
    private JsonData cardData;
    JsonData database;

    void Start () {
        jsonString = File.ReadAllText (Application.dataPath + "/test.json");
        cardData = JsonMapper.ToObject (jsonString);
        database = JsonMapper.ToJson (cardData);
        Debug.Log (database.ToString ());
    }
}

And the Debug.Log turns to :

[{"id":"CS_001","name":"L'\u00E9l\u00E9ment","type":"T\u00F4t"},{"id":"CS_002","name":"L'outrage","type":"T\u00F4t"},{"id":"CS_003","name":"Test","type":"T\u00F4t"}]

Any idea how to get a proper Json ? Even if it's with another JSON library.

Thank you very much.

Solution 1

Here's an example using Json.Net to deserialize the string:

using System;
using Newtonsoft.Json;
using System.Collections.Generic;

public class Program
{
    public static void Main()
    {
        // Deserialize the JSON into a list of CardData
        var ob = JsonConvert.DeserializeObject<List<CardData>>("[{\"id\":\"CS_001\",\"name\":\"L'élément\",\"type\":\"Tôt\"},{\"id\":\"CS_002\",\"name\":\"L'outrage\",\"type\":\"Tôt\"},{\"id\":\"CS_003\",\"name\":\"Test\",\"type\":\"Tôt\"}]" );

        /*
          The output will be:
            id: CS_001, name: L'élément, type: Tôt
            id: CS_002, name: L'outrage, type: Tôt
            id: CS_003, name: Test, type: Tôt
        */
        foreach(var i in ob){
            Console.WriteLine(i);  
        }
    }
}

// Class that will hold the deserialized data
// For demo puposes
public class CardData
{
    public string id { get; set; }
    public string name { get; set; }
    public string type { get; set; }

    public override string ToString(){
        return String.Format("id: {0}, name: {1}, type: {2}",id, name, type);   
    }
}

Live demo available here

Solution 2

Content-type: application/json; charset=utf-8 designates the content to be in JSON format, encoded in the UTF-8 character encoding. The default encoding for JSON is UTF-8. In this case the receiving server apparently does not know that it's dealing with JSON in the UTF-8 encoding and you may need to convert it manually:

byte[] encodedBytes = Encoding.UTF8.GetBytes(jsonString);
Encoding.Convert(Encoding.UTF8, Encoding.Unicode, encodedBytes);

or just try to specify the content type on your request:

content-type: application/json; charset=utf-8

Comments

[shraysalvi]

Try using Json.net.

[MedianP]

Hi, thank you for you answer, I tried the byte[] but that's not working. I'm on mono/unity and didn't found where to specify the content type, thank you

[MedianP]

Hi, thank you for your answer, I tried : File.WriteAllText (Application.dataPath + "/database.json", database.ToString(), Encoding.UTF8); but it stays the same.

[Felype]

Just a comment here, if you're getting a json string from a stream and using a stream reader to "ReadToEnd" then .net will default to read as Unicode instead of the UTF8 the data is in, which will very often cause a catastrophic and silent bug that may be quite hard to figure out. To work around, one can either set the Encoding parameter on the StreamReader constructor or read the stream into a byte[] and decode with Encoding.UTF8.GetString