Restarting the httpd service in RHEL returns "Stopping httpd: [FAILED]" - How do I fix this?

Question

This is my first indication of an issue:

$ sudo /sbin/service httpd restart
Stopping httpd:                                            [FAILED]
Starting httpd: no listening sockets available, shutting down
Unable to open logs
                                                           [FAILED]     

I know httpd is running

$ ps -ef | grep httpd | grep -v grep
apache    9619 20181  0 07:08 ?        00:00:03 /usr/sbin/httpd
apache   10092 20181  0 Jan24 ?        00:00:07 /usr/sbin/httpd
apache   13086 20181  0 06:09 ?        00:00:00 /usr/sbin/httpd
apache   13717 20181  0 Jan25 ?        00:00:01 /usr/sbin/httpd
apache   14730 20181  0 07:13 ?        00:00:01 /usr/sbin/httpd
apache   16359 20181  0 09:54 ?        00:00:00 /usr/sbin/httpd
root     20181     1  0  2011 ?        00:00:01 /usr/sbin/httpd
apache   21450 20181  0 09:55 ?        00:00:00 /usr/sbin/httpd

and it is using ports 80 and 443

$ sudo netstat -lnp | grep :80
tcp        0      0 :::80                       :::*                        LISTEN      9619/httpd
$ sudo netstat -lnp | grep :443
tcp        0      0 :::443                      :::*                        LISTEN      9619/httpd 

so I assume that I get the message "no listening sockets available" because httpd cannot stop to release ports 80 and 443.

I am using RHEL version 5.7:

$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.7 (Tikanga)

I can see a bunch of processes running for httpd:

$ pgrep httpd
9619
10092
13086
13717
14730
16359
20181
21450

What could prevent httpd from stopping? If I kill the processes for httpd, will I be able to start httpd without a problem?

Solution 1

The stop function in /etc/init.d/httpd uses the pidfile:

killproc -p ${pidfile} -d ${STOP_TIMEOUT} $httpd

Possibly, the pidfile /var/run/httpd.pid is out of date or missing (could you have started httpd in the past without using the /etc/init.d script or service?). You can check that file (and its contents) with your ps -ef |grep http.

Meh, just sudo kill 20181. Then start the service up as you normally would. Then try the restart after that.

If it happens again, you probably should investigate why the pid file is getting out of whack with the process table.

Solution 2

Ran into similar on centos6 with ispconfig3 and Apache. cjc's post helped me find the answer to mine. (thanks!) It turns out the PID path in /etc/httpd/conf/httpd.conf was different than in /etc/init.d/httpd for some reason. Once I made them the same (I chose to edit httpd.conf leaving init.d script alone) then killed httpd using PID shown in ps -ef then started httpd again with /etc/init.d/httpd start & voila it works like it should. service httpd status/stop/restart/start all work as expected now too.

I suspect the http.conf file was possibly replaced from a backup of centos5 or perhaps it was edited at some point unless default was wrong although that seems unlikely.

/etc/httpd/conf/httpd.conf

Before:
PidFile run/httpd.pid

After:
PidFile run/httpd/httpd.pid

/etc/init.d/httpd gave the hint of what it was supposed to be:

# pidfile: /var/run/httpd/httpd.pid

On mine status showed Stopped even though httpd was clearly shown running in ps. stop gave an error, start gave ports some already in use error. In hindsight it makes sense now but wanted to post this up if anyone else runs into the same. stop was also giving httpd dead but subsys locked sometimes.

Example error I was getting:

Stopping httpd:                                            [FAILED]
Starting httpd: (98)Address already in use: make_sock: could not bind to address [::]:80
(98)Address already in use: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
Unable to open logs

Solution 3

You should be able to kill the HTTP processes - current connections will be disrupted, of course, and if anyone is currently sending you data (say, submitting a form) that might get lost. But generally, yeah, this will be ok. The same thing would happen with service httpd restart. You will probably have to clean up the lock file at /var/lock/subsys/httpd (delete that file).

I would then run service httpd configtest to make sure your apache config is at least loadable. Then try to start it with service httpd start

You're right that the errors about ports being in use are due to the fact that httpd is still running, and so are using those ports. Likewise, the log error too.

One other thing - there's no need to run sudo to use ps or netstat. Get into the habit of only using sudo for things that require it. It will save you trouble later, when you avoid running some destructive command with root privileges when you don't mean to.

Comments

[Tim]

FYI manually killing a process started by a service, usually will not effect the service, you should still be able to start/stop if fine. (in this case service = init scripts)

[LeXxyIT]

This might help: stackoverflow.com/a/24308634/386579

[malcolmpdx]

Yep, you're right about the "-p", I missed that.

[ghbarratt]

Thank you! I had a look in /var/run/ and discovered that although there were other pid files in there, there was no httpd.pid file in there. I had a look at the other pid file in there and saw that they had nothing more that a process id in plain text. I decided instead of killing all the processes myself, to just create a httpd.pid file, carefully placing the correct process id in the file and nothing more. After that I was able to issue a service httpd restart as many times as I wanted. THANKS AGAIN!

[ghbarratt]

+1 Thanks for a good answer. Yes, I should have not put the sudo on my ps command (will fix the question now). I have never used configtest, I will be sure to check that out.