httpd dead but subsys locked

Question

I modified today max_execution_time in php.ini, when I restarted the server, I get this error :

Stopping httpd:                                            [FAILED]
Starting httpd: (98)Address already in use: make_sock: could not bind to address [::]:80
(98)Address already in use: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
Unable to open logs

I killed httpd proc : killall httpd, and started it fine, but the I can't open any web site on the server.

service httpd status OUTPUT :

httpd dead but subsys locked

I removed httpd file from /var/lock/subsys/

:S Same problem.

Please Help!

http -S Output :

VirtualHost configuration:
188.165.241.194:80     is a NameVirtualHost
         default server default (/etc/httpd/conf.d/zz010_psa_httpd.conf:47)
         port 80 namevhost default (/etc/httpd/conf.d/zz010_psa_httpd.conf:47)
         port 80 namevhost lists (/etc/httpd/conf.d/zz010_psa_httpd.conf:101)
         port 80 namevhost apps-srv.com (/var/www/vhosts/apps-srv.com/conf/httpd.include:10)
         port 80 namevhost billiard.apps-srv.com (/var/www/vhosts/apps-srv.com/conf/httpd.include:78)
         port 80 namevhost curve.apps-srv.com (/var/www/vhosts/apps-srv.com/conf/httpd.include:115)
         port 80 namevhost games-world.apps-srv.com (/var/www/vhosts/apps-srv.com/conf/httpd.include:152)
         port 80 namevhost gifts.apps-srv.com (/var/www/vhosts/apps-srv.com/conf/httpd.include:189)
         port 80 namevhost horoscopes.apps-srv.com (/var/www/vhosts/apps-srv.com/conf/httpd.include:226)
         port 80 namevhost questions.apps-srv.com (/var/www/vhosts/apps-srv.com/conf/httpd.include:263)
         port 80 namevhost raja.apps-srv.com (/var/www/vhosts/apps-srv.com/conf/httpd.include:300)
         port 80 namevhost roses.apps-srv.com (/var/www/vhosts/apps-srv.com/conf/httpd.include:337)
         port 80 namevhost search.apps-srv.com (/var/www/vhosts/apps-srv.com/conf/httpd.include:374)
         port 80 namevhost social.apps-srv.com (/var/www/vhosts/apps-srv.com/conf/httpd.include:411)
         port 80 namevhost wikileaks.apps-srv.com (/var/www/vhosts/apps-srv.com/conf/httpd.include:448)
         port 80 namevhost amine.apps-srv.com (/var/www/vhosts/amine.apps-srv.com/conf/httpd.include:68)
         port 80 namevhost streamlikehell.co.cc (/var/www/vhosts/streamlikehell.co.cc/conf/httpd.include:10)
         port 80 namevhost pageweb.uni.cc (/var/www/vhosts/pageweb.uni.cc/conf/httpd.include:68)
         port 80 namevhost findati.com (/var/www/vhosts/findati.com/conf/httpd.include:10)
         port 80 namevhost floupie.com (/var/www/vhosts/floupie.com/conf/httpd.include:10)
         port 80 namevhost rl.floupie.com (/var/www/vhosts/floupie.com/conf/httpd.include:64)
         port 80 namevhost srv1.floupie.com (/var/www/vhosts/floupie.com/conf/httpd.include:88)
         port 80 namevhost test.floupie.com (/var/www/vhosts/floupie.com/conf/httpd.include:119)
         port 80 namevhost torrent.floupie.com (/var/www/vhosts/floupie.com/conf/httpd.include:143)
         port 80 namevhost zoudani.apps-srv.com (/var/www/vhosts/zoudani.apps-srv.com/conf/httpd.include:10)
         port 80 namevhost horde.webmail (/etc/httpd/conf.d/zzz_horde_vhost.conf:1)
188.165.241.194:443    is a NameVirtualHost
         default server default-188-165-241-194 (/etc/httpd/conf.d/zz010_psa_httpd.conf:78)
         port 443 namevhost default-188-165-241-194 (/etc/httpd/conf.d/zz010_psa_httpd.conf:78)
         port 443 namevhost lists (/etc/httpd/conf.d/zz010_psa_httpd.conf:121)
         port 443 namevhost amine.apps-srv.com (/var/www/vhosts/amine.apps-srv.com/conf/httpd.include:13)
         port 443 namevhost pageweb.uni.cc (/var/www/vhosts/pageweb.uni.cc/conf/httpd.include:13)
         port 443 namevhost horde.webmail (/etc/httpd/conf.d/zzz_horde_vhost.conf:41)
wildcard NameVirtualHosts and _default_ servers:
_default_:443          ks380062.kimsufi.com (/etc/httpd/conf.d/ssl.conf:81)
Syntax OK

Solution 1

You've got an errant lockfile in either /var/subsys or /var/run, or both. The general form of this question has been answered before on serverfault.

Solution 2

Run httpd -S and post the output.

[Tue Mar 15 19:21:34 2011] [warn] Init: You should not use name-based virtual hosts in conjunction with SSL!!

This error could be causing the issue too. Try disabling all https (port 443) configurations. Update the configs with chmod 000 in /etc/httpd/conf.d/ that are doing so and restart httpd (/etc/init.d/httpd restart)

Solution 3

This can actually be a bad sign. It can indicate you have a rogue process that has been previously ran on the server which inherited the binding to port 80.

Run the following to find out what pid it is.

fuser 80/tcp

From there you'll need to get the start time of the process at least before killing it.

ps -o lstart $pid

You can use the start time by cross referencing around the same time in the logs to try to determine how that process got to be there in the first place.

You can kill the process in question after doing this to restart apache but if it does turn out to be that you should consider your system compromised and requiring further investigation.

Solution 4

Have you checked the logs to see what is going on yet? I would expect to see something in syslog and httpd.

You can find the logs in /var/log on most systems.

Also have you looked to see if there are any existing file locks as a result of hung processes?

psgrep httpd should show you if there are any zombie processes.

Solution 5

For me, the solution to this issue was to update the php apc config.

For apc 3.1.4 and higher you should specify the values using a suffix, ie:

apc.shm_size=512M

For pre 3.1.4, this format seems to crash apache. apc version can be seen via phpinfo();. Solution: Remove "M" suffix for apc.shm_size and apc.max_file_size. Then delete the lock file and restart httpd.

View more solutions

Comments

[user9517]

Did you /sbin/service httpd restart after removing the lock file ?

[checksum]

Mine happen because stackoverflow.com/questions/9143964/…. Make sure there is no Listen <port> twice. I used grep -ril Listen 443 ./ to check.

[Hamza]

Hi Mike, I checked httpd log : [Tue Mar 15 19:21:34 2011] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec) [Tue Mar 15 19:21:34 2011] [warn] RSA server certificate CommonName (CN) plesk' does NOT match server name!? [Tue Mar 15 19:21:34 2011] [warn] RSA server certificate CommonName (CN) plesk' does NOT match server name!? [Tue Mar 15 19:21:34 2011] [warn] Init: SSL server IP/port conflict: pageweb.uni.cc:443 (/var/www/vhosts/pageweb.uni.cc/conf/httpd.include:13) vs. horde.webmail:443 (/etc/httpd/conf.d/zzz_horde_vhost.conf:41)

[Hamza]

[Tue Mar 15 19:21:34 2011] [warn] Init: SSL server IP/port conflict: amine.apps-srv.com:443 (/var/www/vhosts/amine.apps-srv.com/conf/httpd.include:13) vs. horde.webmail:443 (/etc/httpd/conf.d/zzz_horde_vhost.conf:41) [Tue Mar 15 19:21:34 2011] [warn] Init: SSL server IP/port conflict: default-188-165-241-194:443 (/etc/httpd/conf.d/zz010_psa_httpd.conf:78) vs. horde.webmail:443 (/etc/httpd/conf.d/zzz_horde_vhost.conf:41) [Tue Mar 15 19:21:34 2011] [warn] Init: You should not use name-based virtual hosts in conjunction with SSL!!

[Mike]

While those do appear to be issues, they aren't related to a file lock as is suggested by your initial post. SSL is definitely having an issue though; these are all just warnings though.

[Hamza]

Hi wrimine, I updated the question with : httpd -S output

[wrmine]

Check for extra 'Listen 80' or 'Listen 443' in your configuration files with grep. The error makes it sound like another process has the address ::::80.

[Hamza]

I already removed these files, same problem

[Jeff Albert]

I see from your updated answer that you found a file in /var/lock/subsys; did you check /var/run?