ruby 1.9 + sinatra incompatible character encodings: ASCII-8BIT and UTF-8

Question

I'm trying to migrate a sinatra application to ruby 1.9

I'm using sinatra 1.0, rack 1.2.0 and erb templates

when I start sinatra it works but when I request the web page from the browser I get this error:

Encoding::CompatibilityError at /
incompatible character encodings: ASCII-8BIT and UTF-8

all .rb files has this header:

#!/usr/bin/env ruby
# encoding: utf-8

I think the problem is in the erb files even if it shows that it's UTF-8 encoded

[user@localhost views]$ file home.erb
home.erb: UTF-8 Unicode text

any one had this problem before? is sinatra not fully compatible with ruby 1.9?

Solution

I'm not familiar with the specifics of your situation, but this kind of error has come up in Ruby 1.9 when there's an attempt to concatenate a string in the source code (typically encoded in UTF-8) with a string from outside of the system, e.g., input from an HTML form or data from a database.

ASCII-8BIT is basically a synonym for binary. It suggests that the input string was not tagged with the actual encoding that has been used (for example, UTF-8 or ISO-8859-1).

My understanding is that exception messages are not seen in Ruby 1.8 because it treats strings as binary and silently concatenates strings of different encodings. For subtle reasons, this often isn't a problem.

I ran into a similar error yesterday and found this excellent overview.

• http://yehudakatz.com/2010/05/05/ruby-1-9-encodings-a-primer-and-the-solution-for-rails/

One option to get your error message to go away is to use force_encoding('UTF-8') (or some other encoding) on the string coming from the external source. This is not to be done lightly, and you'll want to have a sense of the implications.

Comments

[Adrian]

Try temporarily changing the files to ascii only.

[John]

the problem is I need to use utf-8 charterers in the templates.

[Adrian]

If you are using HTML, you should replace them with entities. Otherwise, you might want to try temporarily taking them out just to see if they are the problem.