JavaScript Fetch: characters with encoding issues

Question

I'm trying to use Fetch to bring some data into the screen, however some of the characters ares showing a weird � sign which I believe has something to do with converting special chars.

When debugging on the server side or if I call the servlet on my browser, the problem doesn't happen, so I believe the issue is with my JavaScript. See the code below:

var myHeaders = new Headers();
myHeaders.append('Content-Type','text/plain; charset=UTF-8');

fetch('getrastreiojadlog?cod=10082551688295', myHeaders)
        .then(function (response) {
            return response.text();
        })
        .then(function (resp) {
            console.log(resp);
        });

I think it is probably some detail, but I haven't managed to find out what is happening. So any tips are welcome Thx

Solution

The response's text() function always decodes the payload as utf-8.

If you want the text in other charset you may use TextDecoder to convert the response buffer (NOT the text) into a decoded text with chosen charset.

Using your example it should be:

var myHeaders = new Headers();
myHeaders.append('Content-Type','text/plain; charset=UTF-8');

fetch('getrastreiojadlog?cod=10082551688295', myHeaders)
        .then(function (response) {
            return response.arrayBuffer();
        })
        .then(function (buffer) {
            const decoder = new TextDecoder('iso-8859-1');
            const text = decoder.decode(buffer);
            console.log(text);
        });

Notice that I'm using iso-8859-1 as decoder.

Credits: Schneide Blog

Comments

[Mister Jojo]

look there.. and other github.com/polygonplanet/encoding.js

[Paulo Künzel]

Hi Nil, Thx for the reply. I'm running it locally in my machine and the POM file is set with <project.build.sourceEncoding>UTF-8</project.build.sourceEnc‌​oding>. So that means it should be working with UTF enconding right?

[Nil]

POM file? That's java / maven back-end right? I'm not familiar with these environments but I'm pretty sure they use ISO-8859-1 as default encoding. The project.build.Source is more like to be the input (source) encoding, and I've no id how to manage output encoding for your backend. But maybe you can try the linkTextDecoder Class in javascript :

[Paulo Künzel]

Yes, you are correct on both statments. That's why we need to explicitly say that it is utf-8. Since it doesn't happen by openning the URL in the browser and it does when using fetch(), could it be something on the fetch header or something like it?

[Nil]

have you tried "<project.reporting.outputEncoding>UTF-8</project.reporting.‌​outputEncoding>" in your POM?

[Paulo Künzel]

Just did, unfortunatly it didn't work... Shame, I was quite hopeful with that one

[allez l'OM]

I had a similar problem with my question stackoverflow.com/questions/71195466/…. The code works also in node.js using the node-fetch API.