Converting UTF8 to ANSI with Ruby
Solution 1
ascii_str = yourUTF8text.unpack("U*").map{|c|c.chr}.join
assuming that your text really does fit in the ascii character set.
Solution 2
I finally managed to do it using iconv, I was just messing up the parameters. So, this is how you do it:
require 'iconv'
utf8_csv = File.open("utf8file.csv").read
# gotta be careful with the weird parameters order: TO, FROM !
ansi_csv = Iconv.iconv("LATIN1", "UTF-8", utf8_csv).join
File.open("ansifile.csv", "w") { |f| f.puts ansi_csv }
That's it!
Solution 3
Since ruby 1.9 there is an easier way:
yourstring.encode('ASCII')
To avoid problems with invalid (non-ASCII) characters you can ignore the problems:
yourstring.encode('ASCII', invalid: :replace, undef: :replace, replace: "_")
Solution 4
I had a similar issue trying to generate CSV files from user-generated content on the server. I found the unidecoder gem which does a nice job of transliterating unicode characters into ascii.
Example:
"olá, mundo!".to_ascii #=> "ola, mundo!"
"你好".to_ascii #=> "Ni Hao "
"Jürgen Müller".to_ascii #=> "Jurgen Muller"
"Jürgen Müller".to_ascii("ü" => "ue") #=> "Juergen Mueller"
For our simple use case, this worked well.
Pivotal Labs has a great blog post on unicode transliteration to ascii discussing this in more detail.
Dema
Updated on August 19, 2020Comments
-
Dema over 3 years
I have a Ruby script that generates a UTF8 CSV file remotely in a Linux machine and then transfers the file to a Windows machine thru SFTP.
I then need to open this file with Excel, but Excel doesn't get UTF8, so I always need to open the file in a text editor that has the capability to convert UTF8 to ANSI.
I would love to do this programmatically using Ruby and avoid the manual conversion step. What's the easiest way to do it?
PS: I tried using iconv but had no success.
-
Dema over 15 yearsThat did it as well and it didn't need to use iconv at all. Thanks!
-
Sam Soffes over 13 yearsNote, if
c
is greater than 255, it will fail (since it is out of the ASCII range). -
metavida about 13 yearsIn order to fix the non-ASCII error that Sam ran into you can use the following: yourUTF8text.unpack("U*").map{|c|c.chr rescue '_' }.join