Working with UTF-8 encoding in Python source

Question

Consider:

$ cat bla.py 
u = unicode('d…')
s = u.encode('utf-8')
print s
$ python bla.py 
  File "bla.py", line 1
SyntaxError: Non-ASCII character '\xe2' in file bla.py on line 1, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details

How can I declare UTF-8 strings in source code?

Solution 1

In Python 3, UTF-8 is the default source encoding (see PEP 3120), so unicode characters can be used anywhere.

In Python 2, you can declare in the source code header:

# -*- coding: utf-8 -*-
....

It is described in the PEP 0263:

Then you can use UTF-8 in strings:

# -*- coding: utf-8 -*-

u = 'idzie wąż wąską dróżką'
uu = u.decode('utf8')
s = uu.encode('cp1250')
print(s)

In addition, it may be worth verifying that your text editor properly encodes your code in UTF-8. Otherwise, you may have invisible characters that are not interpreted as UTF-8.

Solution 2

Do not forget to verify if your text editor encodes properly your code in UTF-8.

Otherwise, you may have invisible characters that are not interpreted as UTF-8.