UnicodeDecodeError : 'ascii' codec can't decode byte 0xe0 in position 0: ordinal not in range(128)

44,749

Solution 1

This is a bug in mimetypes, triggered by bad data in the registry. (рєфшю/AMR is not at all a valid MIME media type.)

ctype is a registry key name returned by _winreg.EnumKey, which mimetypes is expecting to be a Unicode string, but it isn't. Unlike _winreg.QueryValueEx, EnumKey returns a byte string (direct from the ANSI version of the Windows API; _winreg in Python 2 doesn't use the Unicode interfaces even though it returns Unicode strings, so it'll never read non-ANSI characters correctly).

So the attempt to .encode it fails with a Unicode​Decode​Error trying to get a Unicode string before encoding it back to ASCII!

try:
    ctype = ctype.encode(default_encoding) # omit in 3.x!
except UnicodeEncodeError:
    pass

These lines in mimetypes should simply be removed.

ETA: added to bug tracker.

Solution 2

By the way, the main culpit of the problem is QuickTime which adds non-ascii mime types to the windows registry. The easiest way to fix it is to manually find and remove from the registry the subsections of the HKCR/Mime/Database/ContentType/ starting with аудио/ and видео/.

Solution 3

There is a patch:

http://bugs.python.org/file18143/9291.patch

Works great for me.

Just replace UnicodeEncodeError to be UnicodeError

Solution 4

An alternative solution from python issue9291 by Alexandr Zarubkin (me21):

add file named sitecustomize.py in Lib\site-packages folder.

import sys
sys.setdefaultencoding("cp1251")

Solution 5

its a python bug with latin MIME tipes in registry start regedit and inspect "HKEY_CLASSES_ROOT\MIME\Database\Content Type" for non-latin names.

Share:
44,749
ceth
Author by

ceth

I'm a programmer, occasional SRE, Unix automator. I'm currently working primarily in Go, and prior to that my weapon of choice was Python, but I'm also familiar with Java, C#, JavaScript, and bash. I dabble in Clojure and F# but I've never thrown a big problem at it. I run Linux at home and at work but that doesn't mean I'm ignorant of other systems :)

Updated on February 10, 2020

Comments

  • ceth
    ceth over 4 years

    On one of my machines I have the error when I am working with google apps engine or django.

    For example:

    • app.yaml

      application: demas1252c
      version: 1
      runtime: python
      api_version: 1
      
      
      handlers:
         - url: /images
      static_dir: images
         - url: /css
      static_dir: css
         - url: /js
      static_dir: js
         - url: /.*
      script: demas1252c.py
      
    • demas1252c.py

      import cgi
      import wsgiref.handlers
      
      
      from google.appengine.ext.webapp import template
      from google.appengine.ext import webapp
      
      
      class MainPage(webapp.RequestHandler): 
      def get(self):
      values = {'id' : 10}
      
      
      self.response.out.write(template.render('foto.html', values))
      
      
      application = webapp.WSGIApplication([('/', MainPage)], debug = True)
      wsgiref.handlers.CGIHandler().run(application)
      
    • foto.html

      <!DOCTYPE html>
      <html lang="en">
          <head></head>
      <body>some</body>
      </html>
      

    error message:

    C:\artefacts\dev\project>"c:\Program Files\Google\google_appengine\dev_appserver.py" foto-hosting
    Traceback (most recent call last):
      File "c:\Program Files\Google\google_appengine\dev_appserver.py", line 69, in <module>
        run_file(__file__, globals())
      File "c:\Program Files\Google\google_appengine\dev_appserver.py", line 65, in run_file
        execfile(script_path, globals_)
      File "c:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_main.py", line 92, in <module>
        from google.appengine.tools import dev_appserver
      File "c:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py", line 140, in <module>
        mimetypes.add_type(mime_type, '.' + ext)
      File "C:\Python27\lib\mimetypes.py", line 344, in add_type
        init()
      File "C:\Python27\lib\mimetypes.py", line 355, in init
        db.read_windows_registry()
      File "C:\Python27\lib\mimetypes.py", line 260, in read_windows_registry
        for ctype in enum_types(mimedb):
      File "C:\Python27\lib\mimetypes.py", line 250, in enum_types
        ctype = ctype.encode(default_encoding) # omit in 3.x!
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xe0 in position 0: ordinal not in range(128)
    

    When I am working with static files in django (without gae) I have very similar error (with different stack).

    I tried to find the reason of error and added code to mimetypes.py:

    print '====='
    print ctype
    ctype = ctype.encode(default_encoding) # omit in 3.x!
    

    Then I get next messages in my console:

    =====
    video/x-ms-wvx
    =====
    video/x-msvideo
    =====
    рєфшю/AMR
    Traceback (most recent call last):
    

    In the registry HKCR/Mime/Database/ContentType/ I have five keys with russian (cyrilic) letters. But how can I fix this error?