noexec on /tmp and /var/tmp directory

Question

Our client servers are running old version of jboss (Jboss 4.0) which has lots of known vulnerabilities. Result is that the server is hacked and we are seeing some unknown executable running from /tmp, /var/tmp and /dev/shm directories. The server is being used as a zombie machine from which other machines are targeted.

We modified jboss configuration to remove the flaw with Nessus confirming that the vulnerability is no more there. Apart from that, I executed the following

Mounted /tmp with noexec option
Mounted /dev/shm with noexec,nosuid option 

I had no issue in executing the above two steps as /tmp is on a separate partition for me. I am facing issues with /var/tmp location as I cannot use noexec on /var partition. So, planning to mount /var/tmp on the same partition that I am using for /tmp location. I hoped that will help resolve the issue for me but after executing the changes I noticed that files and dirs in /tmp are also available in /var/tmp.

# ls -l /var/tmp/
total 32
drwx------ 2 ssdg  ssdg   4096 Dec 17 12:10 gconfd-ssdg
drwxr-xr-x 3 root  root   4096 Feb 21  2013 hp_sum
drwxr-xr-x 2 jboss jboss  4096 Dec 18 11:56 hsperfdata_jboss
drwx------ 2 root  root  16384 May 18  2011 lost+found
srwxr-xr-x 1 root  root      0 Aug 29  2011 mapping-root

# ls -l /tmp/
total 32
drwx------ 2 ssdg  ssdg   4096 Dec 17 12:10 gconfd-ssdg
drwxr-xr-x 3 root  root   4096 Feb 21  2013 hp_sum
drwxr-xr-x 2 jboss jboss  4096 Dec 18 11:56 hsperfdata_jboss
drwx------ 2 root  root  16384 May 18  2011 lost+found
srwxr-xr-x 1 root  root      0 Aug 29  2011 mapping-root

So , probably the idea of using the same partition for /tmp and /var/tmp is not going to work. Need suggestions how can I implement noexec for /var/tmp directory.

Solution 1

You have no guarantee that programs were only on stored on /tmp and /var/tmp. I would put them there, as it is less likely that a directory timestamp is detected there, but that doesn't preclude putting executables somewhere else.

Reinstalling the system from scratch—including the version of jboss without the vulnarabilities before going online—is in my experience—little fortunately—the only way to handle intrusions.

Solution 2

You can use mandatory access control (MAC) to restrict if and under which location the JBoss processes may execute other processes.

On Red Hat/Fedora etc. there is SELinux and under Ubuntu there is AppArmor for MAC.

Comments

[clerksx]

If your server was compromised, start again from scratch. I do not recommend any other way.

[user]

In addition to what @ChrisDown wrote, take the server off the Internet until you have resolved the issue completely. Leaving the server online is irresponsible even if you believe you have fixed the originally exploited vulnerability.

[TheBeginner]

Micheal is right, because if it is a zombie machine it might attack someone else's...

[user]

/tmp and /var/tmp have different semantics. /var/tmp is persisted across reboots, which /tmp isn't. Filesystem Hierarchy Standard: /tmp and /var/tmp.

[vonbrand]

Change all passwords, audit databases for unauthorized changes. Search for funny directories, files named ... or with spaces, ...

[vonbrand]

@MichaelKjörling, additionally, /tmp is often very limited in size.

[Zelda]

The list to search for is without end, that is why I recommend starting from scratch