Linux mysql server will not start, or restart

Question

Intro

Im running MariaDB and Apache on my linux machine and today I ran into some problems.

I connect to the server via SSH and here are some of the issues Im facing. I have absolutely no idea what Im dooing here.

What I am doing

Step 1: mysql -u USERNAME -p

ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2 "No such file or directory")

Ok, so I figured, lets try and stop the mysql service

---

Step 2: sudo service mysql stop

Ok, no errors, so restart the service

---

Step 3: sudo service mysql restart

Job for mysql.service failed. See "systemctl status mysql.service" and "journalctl -xe" for details.

Ok so I better run those two commands and se what I get

---

Step 4: status mysql.service

Hmm.. ok lets try the other one

---

Step 5: journalctl -xe

No journal files were found.

Ok, Im lost here..

MySQL error log

150824 14:42:58 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
150824 14:42:58 [Note] /usr/sbin/mysqld (mysqld 10.0.20-MariaDB-0ubuntu0.15.04.1) starting as process 22637 ...
150824 14:42:58 [ERROR] mysqld: File '/var/lib/mysql/aria_log_control' not found (Errcode: 13 "Permission denied")
150824 14:42:58 [ERROR] mysqld: Got error 'Can't open file' when trying to use aria control file '/var/lib/mysql/aria_log_control'
150824 14:42:58 [ERROR] Plugin 'Aria' init function returned error.
150824 14:42:58 [ERROR] Plugin 'Aria' registration as a STORAGE ENGINE failed.
150824 14:42:58 [Note] InnoDB: Using mutexes to ref count buffer pool pages
150824 14:42:58 [Note] InnoDB: The InnoDB memory heap is disabled
150824 14:42:58 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
150824 14:42:58 [Note] InnoDB: Memory barrier is not used
150824 14:42:58 [Note] InnoDB: Compressed tables use zlib 1.2.8
150824 14:42:58 [Note] InnoDB: Using Linux native AIO
150824 14:42:58 [Note] InnoDB: Using CPU crc32 instructions
150824 14:42:58 [Note] InnoDB: Initializing buffer pool, size = 128.0M
150824 14:42:58 [Note] InnoDB: Completed initialization of buffer pool
150824 14:42:58 [ERROR] InnoDB: ./ibdata1 can't be opened in read-write mode
150824 14:42:58 [ERROR] InnoDB: The system tablespace must be writable!
150824 14:42:58 [ERROR] Plugin 'InnoDB' init function returned error.
150824 14:42:58 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
150824 14:42:58 [Note] Plugin 'FEEDBACK' is disabled.
150824 14:42:58 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it.
150824 14:42:58 [ERROR] Unknown/unsupported storage engine: InnoDB
150824 14:42:58 [ERROR] Aborting

150824 14:42:58 [Note] /usr/sbin/mysqld: Shutdown complete

150824 14:42:58 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended

The issue of diskspace

It does not look like diskspace is an issue

Solution

This is the reason it isn't starting:

150824 14:42:58 [ERROR] mysqld: File '/var/lib/mysql/aria_log_control' not found (Errcode: 13 "Permission denied")
150824 14:42:58 [ERROR] mysqld: Got error 'Can't open file' when trying to use aria control file '/var/lib/mysql/aria_log_control'

Double-check permissions on /var/lib/mysql. Also, check your system log to see if SE-Linux or Apparmor are interfering.

Comments

[Jeroen]

Is there still disk space available in the volume where ibdata is stored?

[Oldenborg]

@Jeroen Look at my update, diskspace does not seem to be an issue

[MadHatter]

ls -al /var/lib/mysql?

[Navern]

It's permission issues. As suggested check owner of files /var/lib/mysql.

[Michael Hampton]

Check your permissions and fix your SELinux contexts.

[Oldenborg]

Just to close this one down, I ended up spinning up a new server and migrated all the data from backups.. problem solved never encountered the issue again.