utf8_encode function purpose

Question

Supposed that im encoding my files with UTF-8.

Within PHP script, a string will be compared:

$string="ぁ";
$string = utf8_encode($string); //Do i need this step?
if(preg_match('/ぁ/u',$string))
//Do if match...

Its that string really UTF-8 without the utf8_encode() function? If you encode your files with UTF-8 dont need this function?

Solution 1

If you read the manual entry for utf8_encode, it converts an ISO-8859-1 encoded string to UTF-8. The function name is a horrible misnomer, as it suggests some sort of automagic encoding that is necessary. That is not the case. If your source code is saved as UTF-8 and you assign "あ" to $string, then $string holds the character "あ" encoded in UTF-8. No further action is necessary. In fact, trying to convert the UTF-8 string (incorrectly) from ISO-8859-1 to UTF-8 will garble it.

To elaborate a little more, your source code is read as a byte sequence. PHP interprets the stuff that is important to it (all the keywords and operators and so on) in ASCII. UTF-8 is backwards compatible to ASCII. That means, all the "normal" ASCII characters are represented using the same byte in both ASCII and UTF-8. So a " is interpreted as a " by PHP regardless of whether it's supposed to be saved in ASCII or UTF-8. Anything between quotes, PHP simply takes as the literal bit sequence. So PHP sees your "あ" as "11100011 10000001 10000010". It doesn't care what exactly is between the quotes, it'll just use it as-is.

Solution 2

PHP does not care about string encoding generally, strings are binary data within PHP. So you must know the encoding of data inside the string if you need encoding. The question is: does encoding matter in your case?

If you set a string variables content to something like you did:

$string="ぁ";

It will not contain UTF-8. Instead it contains a binary sequence that is not a valid UTF-8 character. That's why the browser or editor displays a questionmark or similar. So before you go on, you already see that something might not be as intended. (Turned out it was a missing font on my end)

This also shows that your file in the editor is supporting UTF-8 or some other flavor of unicode encoding. Just keep the following in mind: One file - one encoding. If you store the string inside the file, it's in the encoding of that file. Check your editor in which encoding you save the file. Then you know the encoding of the string.

Let's just assume it is some valid UTF-8 like so (support for my font):

$string="ä";

You can then do a binary comparison of the string later on:

if ( 'ä' === $string )
  # do your stuff

Because it's in the same file and PHP strings are binary data, this works with every encoding. So normally you don't need to re-encode (change the encoding) the data if you use functions that are binary safe - which means that the encoding of the data is not changed.

For regular expressions encoding does play a role. That's why there is the u modifier to signal you want to make the expression work on and with unicode encoded data. However, if the data is already unicode encoded, you don't need to change it into unicode before you use preg_match. However with your code example, regular expressions are not necessary at all and a simple string comparison does the job.

Summary:

$string="ä";
if ( 'ä' === $string )
  # do your stuff

Solution 3

Your string is not a utf-8 character so it can't preg match it, hence why you need to utf8_encode it. Try encoding the PHP file as utf-8 (use something like Notepad++) and it may work without it.

Comments

[nEAnnam]

I encoded all my file with UTF-8, is still neccesary that function?

[fire]

Well have you tried removing the utf8_encode function and seen if it matches?

[nEAnnam]

So its really UTF-8 string if i encode the file with UTF-8? so is not neccesary that function.

[fire]

No, not unless the original string is coming from user input that might not be a UTF-8 encoded string. Even then utf8_encode won't solve your problem as it only converts from ISO-8859-1.

[nEAnnam]

Why you say that $string="ぁ"; will not contain UTF-8. Instead it contains a binary sequence that is not a valid UTF-8 character if i encode my file as UTF-8

[hakre]

Because of the question-mark like character. Not all binary sequences that exist are valid UTF-8 chars (or unicode code-points to be precise). That's why your editor shows that question mark, the binary sequence can not be "read" as a valid character.

[Gromski]

$string="ぁ"; displays perfectly fine as a Japanese あ to me. That is a valid UTF-8 character. Not sure what you're seeing there...

[nEAnnam]

@hakre There are some things that i cant understand from your answer but thanks so much for your answer and time.

[hakre]

@deceze, @nEAnnam: Okay on my computer it displays a character like [?], from what you both write I'm now sure that I'm missing a font. So just ignore that part that is saying it's not valid, a fault by me (:D). As long it's valid it's like with the ä so that part should still be valid. Edited the answer. hope that's better now.