Fix permissions of server after accidental chmod debian

Question

in my panic i posted to ubuntu forum by mistake, im reposting in the correct place (i think)

While trying to debug a mail server, I typed:

chmod -R 777 /

instead of:

chmod -R 777 .

and the icing on the cake, due to me forgetting I had changed a script I use to sign in to fix something, I did all this as root. I do not have backups of most of the system (poor choice I know).

Unlike in the questions "Recovering from chmod -R -777 /" and "What to do after 'chmod -R 777 /'?" i am still signed in as root, and not the whole system was changed, so I do have some control over the system. I also ^C d out of the command within one second to minimize damage. Since then I have physically disconnected the server from the internet.

I believe a script could fix it if it restored the permissions of the filesystem based on data from the package manager, but i do not know how I would go about doing this. If this is not possible, how would I save the data from the server to reinstall the OS?

I am aware of the potential risks of missing a file, but would prefer recovery to reinstall despite them.

this is the current output of ls -la /:

drwxrwxrwx  22 root root  4096 Sep  7  2016 .
drwxrwxrwx  22 root root  4096 Sep  7  2016 ..
drwxr-xr-x   2 root root  4096 May 18 07:55 bin
drwxr-xr-x   3 root root  4096 Sep 21 07:53 boot
drwxr-xr-x  19 root root  3180 Sep 11 20:54 dev
drwxrwxrwx  92 root root  4096 Aug 23 07:50 etc
drwxr-xr-x   4 root root  4096 May 23  2016 home
lrwxrwxrwx   1 root root    31 Feb 24  2016 initrd.img -> /boot/initrd.img-3.16.0-4-amd64
drwxrwxrwx  18 root root  4096 Feb 24  2016 lib
drwxr-xr-x   2 root root  4096 Jun 20 07:00 lib64
drwx------   2 root root 16384 May 19  2016 lost+found
drwxrwxrwx   2 root root  4096 May  5  2015 media
drwxr-xr-x   2 root root  4096 May  5  2015 mnt
drwxr-xr-x   3 root root  4096 May 28  2016 opt
dr-xr-xr-x 148 root root     0 Sep  3 21:55 proc
drwxrwxrwx  10 root root  4096 Aug 19 17:58 root
drwxr-xr-x  22 root root   800 Sep 21 17:09 run
drwxrwxrwx   3 root root  4096 Jun 20 07:00 sbin
drwxr-xr-x   4 root root  4096 Sep 20 23:18 sftp
dr-xr-xr-x  13 root root     0 Sep  3 21:55 sys
drwxrwxrwx   8 root root  4096 Sep 21 17:17 tmp
drwxrwxrwx  11 root root  4096 Feb 24  2016 usr
drwxr-xr-x  14 root root  4096 Jun 25 06:21 var
lrwxrwxrwx   1 root root    27 Feb 24  2016 vmlinuz -> boot/vmlinuz-3.16.0-4-amd64

im aware thats not how you fix a mail server. it was a hacky sloppy fix to see what broke. trust me, im not going to do that again

Solution 1

comment from LasVegasCoder from the other forum i posted to by mistake:

Fixing Permission Error

How to restore root directory permission to default?

Rule #1: If you are not comfortable with command lines, do not run any command as root.

Running chmod -R 777 / as root will break your system.

Running rm -rf / as root will result in a disaster!.

If you've ran chmod -R 777 / as root, follow these steps to restore it back:

Step 1:

Copy the following script, paste it on your console to generate fixpermission script

echo '
chmod -R 755 /bin /boot /dev /etc/ /home /lib /lib64 \
/media /mnt /opt /run /sbin /srv /usr /var

chmod -R 777 /initrd.img /vmlinuz
chmod -R 1777 /tmp
chmod -R 555 /sys
chmod -R 555 /proc
chmod -R 700 /root

' > fixpermission

chmod +x fixpermission

./fixpermission

The above will create a script named fixpermission and run it by ./fixpermission if not already invoked.

Step 2:

Run stat -c '%A %a %n' /* to show your proper directory and their permission as restored.

Example: Your directory permission structure should look similar to the following:

root@plab:~# stat -c '%A %a %n' /*
drwxr-xr-x 755 /bin
drwxr-xr-x 755 /boot
drwxr-xr-x 755 /dev
drwxr-xr-x 755 /etc
drwxr-xr-x 755 /home
lrwxrwxrwx 777 /initrd.img
lrwxrwxrwx 777 /initrd.img.old
drwxr-xr-x 755 /lib
drwxr-xr-x 755 /lib64
drwx------ 700 /lost+found
drwxr-xr-x 755 /media
drwxr-xr-x 755 /mnt
drwxr-xr-x 755 /opt
dr-xr-xr-x 555 /proc
drwx------ 700 /root
drwxr-xr-x 755 /run
drwxr-xr-x 755 /sbin
drwxr-xr-x 755 /srv
dr-xr-xr-x 555 /sys
drwxrwxrwt 1777 /tmp
drwxr-xr-x 755 /usr
drwxr-xr-x 755 /var
lrwxrwxrwx 777 /vmlinuz
lrwxrwxrwx 777 /vmlinuz.old

Step 3:

Reboot your system!

Hope this helps.

Solution 2

If this is a system you have physical access to, you could just mount its drive on to another running, working system and at the very least copy any data you want to save off of it. This is probably the safest and most reliable means of recovery of just the data itself.

You could also find the permissions of a working system (ideally of the exact same OS version and with the exact same packages installed on it) and change the permissions of the broken system to match it -- again, this is most easily done if you can mount the broken system's drive on to a working system, but it might also be possible to do if the system is remote and you can still ssh in to it, become root and chmod.

Note: Before doing anything, I recommend doing a full backup of the broken system (again, this is most easily done if you have physical access to it). If anything goes wrong, make a fresh copy from your backup and try again on the copy, keeping your original backup untouched so it can be restored from again if there's another mistake.

On the working system, you should be able to create a null-separated list of files and working permissions like this:

# find / -name '*' -printf '%m %p\0' > working-permissions.txt

Sidenote: It's null-separated because it's most likely you don't have any filenames with nulls in them, so it's safer to separate them like this rather than using newlines (you still aren't likely to have filenames with newlines in them, but better safe than sorry, and null-separation is safer).

Take a look at the output of xargs --null -n1 --arg-file=working-permissions.txt echo | less to make sure the permissions and filenames look sane. If they do, you now have a choice of ways to proceed:

Option 1:

If you can still log in to the broken system and can become root, you could try copying the working-permissions.txt file to the broken system and when logged in to the broken system run:

# perl -0ne '$_ =~ m{^(\d*\d\d\d) (.*)\0$} ; print "chmod $1 $2\n" ; chmod oct($1), $2 ;' working-permissions.txt

Option 2:

You could mount the disk from the broken system on to the working system. Now you will need to change the perl command above to reflect where you've mounted it. For example, if you've mounted it on /mnt/broken, you'd do:

# perl -0ne '$_ =~ m{^(\d*\d\d\d) (.*)\0$} ; print "chmod $1 $2\n" ; chmod oct($1), "/mnt/broken/$2" ;' working-permissions.txt

The perl script will now change permissions in /mnt/broken.

Note: Option 2 is actually more dangerous to the working system (in case there's a mistake in the script). So when you're ready to run the perl script to change permissions, I suggest you use a LiveDVD or USB drive to first boot the working system, and don't have any drives except the broken system's drive connected to it.

Finally, once you've got your broken system working again, make frequent backups so that if any catastrophe hits it again, you'll be able to just restore from backup and move on.

Solution 3

Really, if you backed up the settings & data files and then did a fresh install & restore files (with correct permissions, should be easy for just those files, matching the default directory may be good enough), that's probably the easiest. But you said the server's broken anyway, so even after fixing the permissions it'll still be broken, so why bother? Just re-install.

I don't know how your server's set up or where the data is, but that should be easy for you to search the web for yourself, and it really should be step one in making a backup anyway.

If you really want a list of file permissions, you can do a separate fresh install (even to a VM) and compare it to your current broken install, then adjust as necessary.

Comments

[Giacomo1968]

“I believe a script could fix it if it restored the permissions of the filesystem…” How is your system bad? Your ls -la output seems normal. If it was 777 everything would be some version of drwxrwxrwx.

[kaioker2]

i ctrl + c d out quickly, but not fast enough as nothing seems to work anymore. /etc /lib /media /root /sbin /tmp and /usr are all drwxrwxrwx and i dont know what they should have been

[Zanna]

This is a bad idea. See my comments on the original answer on Ask Ubuntu

[Ravindra Bawane]

I seem to recall we've dealt with this question before, and this was the answer then. Unfortunately, I can't find the previous question to confirm.

[Blairg23]

This bricked my OS :)

[BeanBagKing]

Option 2 just saved me from rebuilding a server. Thank you. One thing I noticed though was that this doesn't capture/restore things like setuid. I found this out after rebooting and getting "sudo: /usr/bin/sudo must be owned by uid 0 and have the setuid bit set". Mounting offline again and doing chmod +s ... fixed this, but I'm wondering how many other files there are? I'm not quite sure how your bash script or perl script work. Is there any chance of making it fix setuid and other, similar, permissions as well?

[izkon]

Thank you. It looks like you found a bug in my perl scripts. They're fixed now.

[BeanBagKing]

Still appears to be some kind of issue. the print output is correct (chmod 4755), however, permissions that are written to the file seem to be 0777 (in the case of /usr/bin/sudo, not sure about every other). There's no PM function on SU is there?

[BeanBagKing]

i.imgur.com/4r9ZdmU.jpg Screenshot showing output, everything appears correct (working-permissions.txt shows correct permissions, etc.). Let me know if it looks like I'm doing anything wrong on my end.

[BeanBagKing]

Wish you could edit comments more than 5 minutes old. Found the answer in python here, maybe you can do the same in perl? - reddit.com/r/learnpython/comments/8mk8ao/…

[izkon]

Hmm. That's strange. The perl scripts are working for me as they are. I need to look at this more carefully and see if there's something I'm missing. At this point, they should just work for you and I'm not sure why they're not.

[izkon]

Ok. It looks like there was a typo in the 2nd perl script (the one you were using). I fixed it. Could you please give it another try?

[FlyingZebra1]

+1 one on bricking system. do not follow this copy paste response.