MariaDB won't start after upgrade to Ubuntu 16.04

Question

Upgraded to Ubuntu 16.04 from 14.04 x64

If I try to start mysqld it fails and in the syslog I have

kernel: [ 2336.792423] audit: type=1400 audit(1470265086.730:518): apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/systemd/notify" pid=11850 comm="mysqld" requested_mask="w" denied_mask="w" fsuid=117 ouid=0

I added /run/systemd/notify/ to apparmor, but I cannot add run/systemd/notify - "/" is missing

How can I fix this? I need MariaDB to run on a separate partition

I am using MariaDB with data folder on /home/db/mysql I had a symlink configured to point to it from

lrwxrwxrwx 1 mysql mysql 14 iun 22 20:58 /var/lib/mysql -> /home/db/mysql

and also had app armor profile configured:

# vim:syntax=apparmor
# Last Modified: Tue Jun 19 17:37:30 2007
#include <tunables/global>

/usr/sbin/mysqld {
#include <abstractions/base>
#include <abstractions/nameservice>
#include <abstractions/user-tmp>
#include <abstractions/mysql>
#include <abstractions/winbind>

capability dac_override,
capability sys_resource,
capability setgid,
capability setuid,

network tcp,

/etc/hosts.allow r,
/etc/hosts.deny r,

/etc/mysql/*.pem r,
/etc/mysql/conf.d/ r,
/etc/mysql/conf.d/* r,
/etc/mysql/*.cnf r,
/usr/lib/mysql/plugin/ r,
/usr/lib/mysql/plugin/*.so* mr,
/usr/sbin/mysqld mr,
/usr/share/mysql/** r,
/var/log/mysql.log rw,
/var/log/mysql.err rw,
/var/lib/mysql/ r,
/var/lib/mysql/** rwk,
/var/log/mysql/ r,
/var/log/mysql/* rw,
/var/run/mysqld/mysqld.pid rw,
/var/run/mysqld/mysqld.sock w,
/run/mysqld/mysqld.pid rw,
/run/mysqld/mysqld.sock w,
/run/systemd/notify w,
/home/db/** rwk,
/home/db/mysql/** rwk,

/sys/devices/system/cpu/ r,

# Site-specific additions and overrides. See local/README for details.
#include <local/usr.sbin.mysqld>
}

It previously failed on cd to /home/db/mysql, but fixed it by editing the /lib/systemd/system/mariadb.service and setting ProtectHome=false

Solution

After several failed tries involving apparmor configuration, editing /lib/systemd/system/mariadb.service and /etc/mysql/my.cnf to set the datadir, which failed, I went on a different approach.

1. Stopped the service (if started): sudo service mysqld stop

2. Purged the installation apt-get purge mysqld*

3. I removed everything involving mysql and mariadb (searched /etc, /var for them). You may want to back them up if you think you may need them for reference in the future.

   • mysql and mariadb configuration in /etc/systemd/system

   • mysql profile in /etc/apparmor.d

   • dir /etc/mysql

   • the symlink I had for mysql in /var/lib (my data is safe on /home/db/mysql). Skip this if your data is in /var/lib/mysql

4. Reinstalled sudo apt-get install mariadb-server

5. Renamed the /var/lib/mysql to /var/lib/mysql.bk and created symlink for /home/db/mysql to /var/lib/mysql
6. Set ownership to the symlink and /home/db/mysql for mysql:mysql

All works on reboot and it doesn't die anymore.

I can start and stop the service with sudo /etc/init.d/mysql start sudo /etc/init.d/mysql stop

I guess the migration to systemd caused all this trouble. Maybe using the old init.d scripts is not the best way to do things, but it's the only way I found to keep MariaDB running after reboot, otherwise it kept crashing because of

kernel: [ 2336.792423] audit: type=1400 audit(1470265086.730:518): apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/systemd/notify" pid=11850 comm="mysqld" requested_mask="w" denied_mask="w" fsuid=117 ouid=0 

Since some of the clients I was using gave Failed to connect to database: Error: Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2), I had to create a symlink sudo ln -s /var/run/mysqld/mysqld.sock /tmp/mysql.sock. The link got removed on reboot though...

Therefore, I had to edit the /etc/mysql/my.cnf and /etc/mysql/debian.cnf files to set socket = /tmp/mysql.sock. The debian.cnf says DO NOT TOUCH!, but I took the chances, since my.cnf says Remember to edit /etc/mysql/debian.cnf when changing the socket location. Note: a reboot was required to see the changes applied. Only restarting the service didn't apply them. Maybe some daemon should have been reloaded, but didn't know which one.