httpd dead but pid file exist | httpd dead but sudsys locked. Centos 6

web-server http centos6 httpd httpd.conf

6,061

Sent a support request to the hosting provider and after 2 days they fixed the issue. The ticket had been closed with this message: "Hello, the problem was on the node, we had to do a change on the node kernel."

Now with the same configuration (and more packages) Apache is working perfect. Thanks everybody anyway!

exit

6,061

cospps

Updated on September 18, 2022

Comments

cospps over 1 year

Centos 6 64bit clean install on VPS, any kind of package installed just trying to use the already installed httpd.

my case:    
[root@bucch ~]# service httpd status    
httpd is stopped    
[root@bucch ~]# service httpd start    
Starting httpd:                                           [  OK  ]    
[root@bucch ~]# service httpd status    
httpd dead but pid file exists     
[root@bucch ~]# rm -f /var/run/httpd/httpd.pid    
[root@bucch ~]# service httpd status    
httpd dead but subsys locked    
[root@bucch ~]# rm -f /var/lock/subsys/httpd    
[root@bucch ~]# service httpd status    
httpd is stopped    
[root@bucch ~]# tail -n 10 /var/log/httpd/error_log    
[Fri Oct 07 11:38:47 2016] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)    
[Fri Oct 07 11:38:47 2016] [notice] Digest: generating secret for digest authentication ...    
[Fri Oct 07 11:38:47 2016] [notice] Digest: done
[Fri Oct 07 11:38:47 2016] [info] APR LDAP: Built with OpenLDAP LDAP SDK
[Fri Oct 07 11:38:47 2016] [info] LDAP: SSL support available
[Fri Oct 07 11:38:47 2016] [notice] Apache/2.2.15 (Unix) DAV/2 configured -- resuming normal operations    
[Fri Oct 07 11:38:47 2016] [info] Server built: Jul 18 2016 15:24:00
[Fri Oct 07 11:38:47 2016] [debug] prefork.c(1028): AcceptMutex: sysvsem (default: sysvsem)    
[Fri Oct 07 11:38:47 2016] [debug] proxy_util.c(1909): proxy: grabbed scoreboard slot 0 in child 4040 for worker proxy:reverse    
[Fri Oct 07 11:38:47 2016] [debug] proxy_util.c(2025): proxy: initialized single connection worker 0 in child 4040 for (*)

in the /etc/httpd/conf/httpd.conf have this directive:

PidFile /var/run/httpd/httpd.pid

and in the /etc/sysconfig/httpd have this:

PIDFILE=/var/run/httpd/httpd.pid

in the /etc/sysconfig/iptables have:

-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT    
-A INPUT -p tcp -m tcp --dport 443 -j ACCEPT

the directory /var/run/httpd have this kind of permission:

chmod a+rx

this is the output of netstat -plant that is the same before and after starting the httpd service

[root@bucch ~]# netstat -plant    
Active Internet connections (servers and established)    
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name    
tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      693/sshd    
tcp        0      0 127.0.0.1:25                0.0.0.0:*                   LISTEN      1875/sendmail    
tcp        0     64 212.8.250.64:22             x.x.x.x:49817               ESTABLISHED 3737/sshd    
tcp        0      0 212.8.250.64:22             x.x.x.x:49167               ESTABLISHED 3143/sshd    
tcp        0      0 212.8.250.64:22             x.x.x.x:46407               ESTABLISHED 4225/sshd    
tcp        0      0 :::22                       :::*                        LISTEN      693/sshd    
[root@bucch ~]#

i am going crazy! any suggest?

over there i got some info from strace -f -o trace2.txt /etc/rc.d/init.d/httpd start

[root@bucch ~]# tail -n 100 trace2.txt
6699  mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdefce27000
6699  arch_prctl(ARCH_SET_FS, 0x7fdefce28700) = 0
6699  mprotect(0x7fdefc670000, 4096, PROT_READ) = 0
6699  mprotect(0x7fdefca00000, 16384, PROT_READ) = 0
6699  mprotect(0x7fdefcc10000, 4096, PROT_READ) = 0
6699  mprotect(0x7fdefce31000, 4096, PROT_READ) = 0
6699  munmap(0x7fdefce2a000, 21145)     = 0
6699  set_tid_address(0x7fdefce289d0)   = 6699
6699  set_robust_list(0x7fdefce289e0, 24) = 0
6699  futex(0x7ffe5442baac, FUTEX_WAKE_PRIVATE, 1) = 0
6699  futex(0x7ffe5442baac, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, NULL, 7fdefce28700) = -1 EAGAIN (Resource temporarily unavailable)
6699  rt_sigaction(SIGRTMIN, {0x7fdefc45ecb0, [], SA_RESTORER|SA_SIGINFO, 0x7fdefc4687e0}, NULL, 8) = 0
6699  rt_sigaction(SIGRT_1, {0x7fdefc45ed40, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7fdefc4687e0}, NULL, 8) = 0
6699  rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
6699  getrlimit(RLIMIT_STACK, {rlim_cur=10240*1024, rlim_max=RLIM64_INFINITY}) = 0
6699  brk(0)                            = 0x12db000
6699  brk(0x12fc000)                    = 0x12fc000
6699  open("/var/lock/subsys/httpd", O_WRONLY|O_CREAT|O_NOCTTY|O_NONBLOCK, 0666 <unfinished ...>
6698  <... open resumed> )              = 8
6698  fcntl(8, F_GETFD)                 = 0x1 (flags FD_CLOEXEC)
6698  fcntl(8, F_SETFD, FD_CLOEXEC)     = 0
6698  write(8, "6698\n", 5)             = 5
6698  close(8)                          = 0
6698  semget(IPC_PRIVATE, 1, IPC_CREAT|0600) = 688130
6698  semctl(688130, 0, SETVAL, 0x1)    = 0
6698  geteuid()                         = 0
6698  semctl(688130, 0, IPC_SET, 0x7ffe08648870) = 0
6698  mmap(NULL, 19224, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 0x7fc5ac649000
6698  rt_sigaction(SIGSEGV, {0x7fc5ac687050, [], SA_RESTORER|SA_RESETHAND, 0x7fc5aaf2f7e0}, NULL, 8) = 0
6698  rt_sigaction(SIGBUS, {0x7fc5ac687050, [], SA_RESTORER|SA_RESETHAND, 0x7fc5aaf2f7e0}, NULL, 8) = 0
6698  rt_sigaction(SIGABRT, {0x7fc5ac687050, [], SA_RESTORER|SA_RESETHAND, 0x7fc5aaf2f7e0}, NULL, 8) = 0
6698  rt_sigaction(SIGILL, {0x7fc5ac687050, [], SA_RESTORER|SA_RESETHAND, 0x7fc5aaf2f7e0}, NULL, 8) = 0
6698  rt_sigaction(SIGFPE, {0x7fc5ac687050, [], SA_RESTORER|SA_RESETHAND, 0x7fc5aaf2f7e0}, NULL, 8) = 0
6698  rt_sigaction(SIGTERM, {0x7fc5ac691240, [], SA_RESTORER, 0x7fc5aaf2f7e0}, NULL, 8) = 0
6698  rt_sigaction(SIGWINCH, {0x7fc5ac691240, [], SA_RESTORER, 0x7fc5aaf2f7e0}, NULL, 8) = 0
6698  rt_sigaction(SIGINT, {0x7fc5ac691240, [], SA_RESTORER, 0x7fc5aaf2f7e0}, NULL, 8) = 0
6698  rt_sigaction(SIGXCPU, {SIG_DFL, [], SA_RESTORER, 0x7fc5aaf2f7e0}, NULL, 8) = 0
6698  rt_sigaction(SIGXFSZ, {SIG_IGN, [], SA_RESTORER, 0x7fc5aaf2f7e0}, NULL, 8) = 0
6698  rt_sigaction(SIGPIPE, {SIG_IGN, [], SA_RESTORER, 0x7fc5aaf2f7e0}, NULL, 8) = 0
6698  rt_sigaction(SIGHUP, {0x7fc5ac691270, [HUP USR1], SA_RESTORER, 0x7fc5aaf2f7e0}, NULL, 8) = 0
6698  rt_sigaction(SIGUSR1, {0x7fc5ac691270, [HUP USR1], SA_RESTORER, 0x7fc5aaf2f7e0}, NULL, 8) = 0
6698  clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7fc5ac641ab0) = 6700
6698  write(2, "[Fri Oct 07 20:31:44 2016] [noti"..., 104) = 104
6698  write(2, "[Fri Oct 07 20:31:44 2016] [info"..., 69) = 69
6698  write(2, "[Fri Oct 07 20:31:44 2016] [debu"..., 92) = 92
6698  wait4(-1, 0x7ffe086488ac, WNOHANG|WSTOPPED, NULL) = 0
6698  select(0, NULL, NULL, NULL, {1, 0} <unfinished ...>
6700  set_robust_list(0x7fc5ac641ac0, 24) = 0
6700  rt_sigaction(SIGHUP, {0x7fc5ac6920f0, [], SA_RESTORER|SA_INTERRUPT, 0x7fc5aaf2f7e0}, {0x7fc5ac691270, [HUP USR1], SA_RESTORER, 0x7fc5aaf2f7e0}, 8) = 0
6700  rt_sigaction(SIGTERM, {0x7fc5ac6920f0, [], SA_RESTORER|SA_INTERRUPT, 0x7fc5aaf2f7e0}, {0x7fc5ac691240, [], SA_RESTORER, 0x7fc5aaf2f7e0}, 8) = 0
6700  rt_sigaction(SIGINT, {SIG_IGN, [], SA_RESTORER|SA_INTERRUPT, 0x7fc5aaf2f7e0}, {0x7fc5ac691240, [], SA_RESTORER, 0x7fc5aaf2f7e0}, 8) = 0
6700  rt_sigaction(SIGUSR1, {0x7fc5ac691900, [], SA_RESTORER|SA_INTERRUPT, 0x7fc5aaf2f7e0}, {0x7fc5ac691270, [HUP USR1], SA_RESTORER, 0x7fc5aaf2f7e0}, 8) = 0
6700  geteuid()                         = 0
6700  setgid(48)                        = 0
6700  open("/proc/sys/kernel/ngroups_max", O_RDONLY) = 8
6700  read(8, "65536\n", 31)            = 6
6700  close(8)                          = 0
6700  open("/etc/group", O_RDONLY|O_CLOEXEC) = 8
6700  fstat(8, {st_mode=S_IFREG|0644, st_size=532, ...}) = 0
6700  mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc5ac648000
6700  lseek(8, 0, SEEK_CUR)             = 0
6700  read(8, "root:x:0:\nbin:x:1:bin,daemon\ndae"..., 4096) = 532
6700  read(8, "", 4096)                 = 0
6700  close(8)                          = 0
6700  munmap(0x7fc5ac648000, 4096)      = 0
6700  setgroups(1, [48])                = 0
6700  geteuid()                         = 0
6700  setuid(48)                        = 0
6700  write(2, "[Fri Oct 07 20:31:44 2016] [debu"..., 127) = 127
6700  write(2, "[Fri Oct 07 20:31:45 2016] [debu"..., 123) = 123
6700  epoll_create1(EPOLL_CLOEXEC)      = 8
6700  epoll_ctl(8, EPOLL_CTL_ADD, 3, {EPOLLIN, {u32=2915366168, u64=140487000651032}}) = 0
6700  accept4(3,  <unfinished ...>
6699  <... open resumed> )              = 3
6699  dup2(3, 0)                        = 0
6699  close(3)                          = 0
6699  utimensat(0, NULL, NULL, 0)       = 0
6699  close(0)                          = 0
6699  close(1)                          = 0
6699  close(2)                          = 0
6699  exit_group(0)                     = ?
6699  +++ exited with 0 +++
6690  <... wait4 resumed> [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 6699
6690  rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
6690  --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=6699, si_status=0, si_utime=0, si_stime=0} ---
6690  wait4(-1, 0x7ffd4819c31c, WNOHANG, NULL) = -1 ECHILD (No child processes)
6690  rt_sigreturn()                    = 0
6690  rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7f8437653660}, {0x43d610, [], SA_RESTORER, 0x7f8437653660}, 8) = 0
6690  rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
6690  rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
6690  read(255, "\nexit $RETVAL\n", 3488) = 14
6690  rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
6690  rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
6690  exit_group(0)                     = ?
6690  +++ exited with 0 +++
6698  <... select resumed> )            = 0 (Timeout)
6698  wait4(-1, 0x7ffe086488ac, WNOHANG|WSTOPPED, NULL) = 0
6698  select(0, NULL, NULL, NULL, {1, 0} <unfinished ...>
6698  +++ killed by SIGKILL +++
6700  +++ killed by SIGKILL +++

Admin over 7 years

What do your logs show?
Admin over 7 years

Change the log level to LogLevel debug. Clear the error log. run service httpd start and then add the contents of the error log to your question.
Admin over 7 years

it is already set to debug and on the top of the request can see the output of tail -n 10 /var/log/httpd/error_log
Admin over 7 years

What does "any kind of package installed just trying to use the already installed httpd." mean exactly?
Admin over 7 years

Sorry, wanna mean: I just rebuild the VPS from the control panel than at fisrt login i added the iptables rule and after: service httpd start And it doesn't work..
Admin over 7 years

iptables has nothing to do with your problem at this moment. That will only affect whether anyone can get to your webserver once it's running correctly.
Admin over 7 years

Port your whole error log after clearing it and restarting httpd like @Unbeliever suggested. 10 lines may not be enough to see the problem.
Admin over 7 years

made, but i got exactly the 10 linees posted before..
Admin over 7 years

I have more important info about the error in the output of strace -f -o trace.txt /etc/rc.d/init.d/httpd start but i am not able to porst by phone now.
Admin over 7 years

You are getting SIGKILL signals. Check for security software (such as SELinux) settings that prevent Apache from starting. Also, just to verify, try starting Apache with apachectl start. If anything is shown on the console add it to your question.
Admin over 7 years

[root@bucch selinux]# sestatus SELinux status: disabled [root@bucch selinux]# apachectl start [root@bucch selinux]# service httpd status httpd dead but pid file exists