MySQL crash on startup

10,502

Solution 1

It was probably a corruption of the InnoDB data. I added

innodb_force_recovery = 2

to my.ini, restarted the DB and I was able to dump all the data and recover it.

Be aware of using this, read the documentation before: http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html

Solution 2

I had similar problem on a CentOS VPS and Stefano Giacone's answer is basically what I did so it worked after hours of researching and a lot of stress...

Well the steps were:

1) Find my.cnf file (mine was located in /etc/my.cnf) and add the line:

innodb_force_recovery = X

replacing X with a integer from 1 to 6, starting from 1 and then incrementing if MySQL won't start. Setting to 4, 5 or 6 can delete your data so be carefull and read http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html before.

2) Restart MySQL service. Only SELECT will run and that's normal at this point.

3) Dump all your databases/schemas with mysqldump one by one, do not compress the dumps because you'd have to uncompress them later anyway in 6).

4) Move (or delete!) only the bd's directories inside /var/lib/mysql, preserving the individual files in the root.

5) Stop MySQL and then uncomment the line added in 1). Start MySQL.

6) Recover all bd's dumped in 3).

That worked for me, good luck!

Share:
10,502
Stefano Giacone
Author by

Stefano Giacone

I'm a Mathematician, a developer and a creative. I love to think about new ideas, new projects and new experiences. I have always so many ideas that I never have the time to develop every single one. I'm passionate about technology, web and finance. I have strong problem solving skills, everytime I get involved in something I love to think about new ideas to solve problems or improving the existing solutions. Of course I'm really interested in innovation and startup environment. Interests in: Artificial Neural Networks & Machine Learning Open Innovation Customer experience I'd like to dedicate part of my spare time in creative works, such as photography and software development, I always have a side project running.

Updated on June 04, 2022

Comments

  • Stefano Giacone
    Stefano Giacone almost 2 years

    I'm developing on my laptop with wamp and mysql is running fine from weeks. Today after 60 seconds after boot mysql crashes I and find the following error inside the log:

    2016-03-17T20:34:37.662021Z 0 [ERROR] InnoDB: Cannot allocate 4294956804 bytes of memory after 60 retries over 60 seconds. OS error: Not enough space (12). Check if you should increase the swap file or ulimits of your operating system. Note that on most 32-bit computers the process memory space is limited to 2 GB or 4 GB.
    2016-03-17 21:34:37 0x2b74  InnoDB: Assertion failure in thread 11124 in file ut0ut.cc line 938
    InnoDB: Failing assertion: !m_fatal
    InnoDB: We intentionally generate a memory trap.
    InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
    InnoDB: If you get repeated assertion failures or crashes, even
    InnoDB: immediately after the mysqld startup, there may be
    InnoDB: corruption in the InnoDB tablespace. Please refer to
    InnoDB: http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html
    InnoDB: about forcing recovery.
    20:34:37 UTC - mysqld got exception 0x80000003 ;
    This could be because you hit a bug. It is also possible that this binary
    or one of the libraries it was linked against is corrupt, improperly built,
    or misconfigured. This error can also be caused by malfunctioning hardware.
    Attempting to collect some information that could help diagnose the problem.
    As this is a crash and something is definitely wrong, the information
    collection process might fail.
    
    key_buffer_size=67108864
    read_buffer_size=2097152
    max_used_connections=0
    max_threads=151
    thread_count=0
    connection_count=0
    It is possible that mysqld could use up to 
    key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 685380 K  bytes of memory
    Hope that's ok; if not, decrease some variables in the equation.
    
    Thread pointer: 0x103288c0
    Attempting backtrace. You can use the following information to find out
    where mysqld died. If you see no messages after this, something went
    terribly wrong...
    2016-03-17T20:34:37.682021Z 0 [ERROR] InnoDB: Cannot allocate 4294954524 bytes of memory after 60 retries over 60 seconds. OS error: Not enough space (12). Check if you should increase the swap file or ulimits of your operating system. Note that on most 32-bit computers the process memory space is limited to 2 GB or 4 GB.
    2016-03-17 21:34:37 0x2b58  InnoDB: Assertion failure in thread 11096 in file ut0ut.cc line 938
    InnoDB: Failing assertion: !m_fatal
    InnoDB: We intentionally generate a memory trap.
    InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
    InnoDB: If you get repeated assertion failures or crashes, even
    InnoDB: immediately after the mysqld startup, there may be
    InnoDB: corruption in the InnoDB tablespace. Please refer to
    InnoDB: http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html
    InnoDB: about forcing recovery.
    

    To me it seems that there is a memory issue:

    [ERROR] InnoDB: Cannot allocate 4294956804 bytes of memory after 60 retries over 60 seconds. OS error: Not enough space (12).

    Could it be that the crash is due to this error? It's weird that mysql try to allocate 4Gb of ram, normally it uses more or less 500Mb.

    this is my.ini:

    innodb_buffer_pool_size = 16M
    # Set .._log_file_size to 25 % of buffer pool size
    innodb_log_file_size = 5M
    innodb_log_buffer_size = 8M
    innodb_flush_log_at_trx_commit = 1
    innodb_lock_wait_timeout = 50
    

    I'm running mysql 32 bit on a win7 64bit.

    Is this something I can solve changing some variable?

    Thanks very much for your help