What character encoding is used by StreamReader.ReadToEnd()?
The standard encoding used by StreamReader
is Encoding.Default, which will vary from machine to machine depending on your version of Windows and the locale that you have set. Encoding.UTF8
.
I have trouble remembering what the defaults are, so I prefer to use the StreamReader
constructor that lets me specify the encoding. For example:
Using reader As StreamReader = New StreamReader(Response.GetResponseStream(), Encoding.UTF8)
See the constructor documentation for more info.
If you use that constructor in your example a, the results will be the same as for your example b.
Should you use UTF-8? That depends on the page you're downloading. If the page you're downloading was encoded with UTF-8 then, yes, you should use UTF-8. UTF-8 is supposed to be the default if no character set is defined in the HTTP headers. But you need to check the Content-Type
header to determine if the page uses some other encoding. For example, the Content-Type
header might read:
application/xml; charset=ISO-8859-2
You would have to examine the ContentType property of the HttpWebResponse
, check to see if there is a charset
field, and set the encoding properly based on that.
Or, just use UTF-8 and hope for the best.
CJ7
Updated on July 03, 2022Comments
-
CJ7 almost 2 years
- What character encoding is used by
StreamReader.ReadToEnd()
? - What would be the reason to use (b) instead of (a) below?
- Is there a risk of their being a character encoding problem if (a) is used instead of (b)?
- Is there another method that is better than (a) and (b)?
(a)
Dim strWebResponse As String Dim Request As HttpWebRequest = WebRequest.Create(Url) Using Response As WebResponse = smsRequest.GetResponse() Using reader As StreamReader = New StreamReader(Response.GetResponseStream()) strWebResponse = reader.ReadToEnd() End Using End Using
(b)
Dim encoding As New UTF8Encoding() Dim strWebResponse As String Dim Request As HttpWebRequest = WebRequest.Create(Url) Using Response As WebResponse = Request.GetResponse() Dim responseBuffer(Response.ContentLength - 1) As Byte Response.GetResponseStream().Read(responseBuffer, 0, Response.ContentLength - 1) strWebResponse = encoding.GetString(responseBuffer) End Using
- What character encoding is used by