Error importing large MySQL dump file which includes binary BLOBs in Windows

Question

I'm trying to import a MySQL dump file, which I got from my hosting company, into my Windows dev machine, and i'm running into problems.

I'm importing this from the command line, and i'm getting a very weird error:

ERROR 2005 (HY000) at line 3118: Unknown MySQL server host '╖?*á±dÆ╦N╪Æ·h^ye"π╩i╪ Z+-$▼₧╬Y.∞┌|↕╘l∞/l╞⌂î7æ▌X█XE.ºΓ[ ;╦ï♣éµ♂º╜┤║].♂┐φ9dë╟█'╕ÿG∟═0à¡úè♦╥↑ù♣♦¥'╔NÑ' (11004)

I'm attaching the screenshot because i'm assuming the binary data will get lost...

I'm not exactly sure what the problem is, but two potential issues are the size of the file (2 Gb) which is not insanely large, but it's not trivially small either, and the other is the fact that many of these tables have JPG images in them (which is why the file is 2Gb large, for the most part).
Also, the dump was taken in a Linux machine and I'm importing this into Windows, not sure if that could add to the problems (I understand it shouldn't)

Now, that binary garbage is why I think the images in the file might be a problem, but i've been able to import similar dumps from the same hosting company in the past, so i'm not sure what might be the issue.

Also, trying to look into this file (and line 3118 in particular) is kind of impossible given its size (i'm not really handy with Linux command line tools like grep, sed, etc).

The file might be corrupted, but i'm not exactly sure how to check it. What I downloaded was a .gz file, which I "tested" with WinRar and it says it looks OK (i'm assuming gz has some kind of CRC). If you can think of a better way to test it, I'd love to try that.

Any ideas what could be going on / how to get past this error?

I'm not very attached to the data in particular, since I just want this as a copy for dev, so if I have to lose a few records, i'm fine with that, as long as the schema remains perfectly sound.

Thanks!
Daniel

Solution 1

For this reason I always use mysqldump --hex-blob.

Re-dump the database encoding the blobs using this switch and it will work.

You can try to import it using a windows mysql client IDE like sqlyog or mysql administrator. It worked for me once.

Solution 2

You do not necessarily need to use the --hex-blob option. I have just resolved this problem my self and the issue was that I needed the --max_allowed_packet to be set to a large enough value to accommodate the largest data blob I would be loading. Your restore command should look something like:

mysql -u user -h hostname --max_allowed_packet=32M dbname < dumpfile.sql

If you use the --hex-blob option you will significantly increase the size of your backup - by a factor of 2 or more. NOTE: to restore the same data that I restored with the above command required setting the --max_allowed_packet=64M in my.ini(cnf) and restarting the server AS WELL AS setting it to 64M on the command line to restore a dump created with the --hex-blob option.

Solution 3

There could still be a problem because of large file size so make sure you set max-allowed-packet to some high value (param for mysql command).

Solution 4

Ok, I had this issue today. But my problem was that the database was already dropped when I realized that the backup was broken. So, no --hex-blob for me! To be able to fix it I made a little script in PHP that converts the "binary string" into the hex representation where the values are represented like "_binary '!@{#!@{#'"...

It's using a REGEX for parsing the SQL, which is not completely safe, but it did the job for me.

<?php
function convertEncoding($str)
{
    $r = '';
    for ($i = 0; $i < mb_strlen($str); $i++) {
        $r .= sprintf('%02X', mb_ord(mb_substr($str, $i, 1, 'UTF-8'), 'UTF-8'));
    }

    return '0x' . $r;
}


$str = file_get_contents('data.sql');

$newStr = preg_replace_callback('/_binary \'(.+?)\'(,|\))/im', function ($str) {
    $s = convertEncoding(stripcslashes($str[1]));
    echo 'Translated: ' . $str[1] . ' => ' . $s . PHP_EOL;
    echo 'Ending char was: ' . $str[2] . PHP_EOL;
    return $s . $str[2];
}, $str);

file_put_contents('fixed.sql', $newStr) ;

I hope it saves someone the headache I got!

View more solutions

Comments

[Daniel Magliola]

I'll try asking the hosting guys for that, let's see if that does it. That may take a couple of days, though :-( - What confuses me is that i've been able to import other binary dumps from them in the past. Any idea what it might be?

[Pablo]

If you can import on linux, and you can import on windows after exporting with --hex-blob, you could temporarily import into a linux, and export it from there with --hex-blob. Let me know if you need help (a.k.a.: linux box) with that.

[T. Brian Jones]

This works great and should probably be the accepted answer.

[T. Brian Jones]

See @BobC's answer below for a solution that does not require a special export.