Why can't Python print Unicode symbols?

Question

Possible Duplicate:
Python UnicodeDecodeError - Am I misunderstanding encode?

I am having trouble printing some unicode symbols in Python like this:

# encoding: utf-8
print u'ęėįųšįšū'

When I try to run this on my VPS Ubuntu 12 server with Python 2.7, I get an error:

UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-7: ordinal not in range(128)

Why does it try to encode them in ASCII?

The commands run correctly on my local machines.

The file is correctly encoded in utf-8.

Solution

Printing unicode objects requires Python to guess the output encoding and encoding the Unicode codepoints to that encoding.

On your VPS server, the output encoding appears to be ASCII, which is the default when no encoding could be detected (such as when using a pipe). If you run the same code on a terminal, the terminal encoding is usually detected and the encoding succeeds.

The solution is to encode explicitly depending on your script requirements.

Please do read the Python Unicode HOWTO to understand how Python does this detection and why it needs to encode for you.

Comments

[Andreas Jung]

Please make a comment for such questions asked every day. You are repeating only FAQ answers.

[Martijn Pieters]

@RashanGandi: If you have a good dupe target for the question, feel free to vote to close as such. You'll probably get my vote too. In the meantime, I'll help question askers as best I can.

[Taha Jahangir]

Output encoding sys.stdout.encoding, which is probably not utf-8 in this case.