Nagios misinterpreting Http response!

Question

I've installed Nagios 4.0.1 on my CentOS 6.5 server. Since I'd set up Node.js to use port 80, I configured the Apache (httpd) service to listen to port 3000 instead.

I can open Nagios console and everything looks fine using HTTP on port 3000. The problem is in the Nagios services status that shows my HTTP as CRITICAL with the description:

Connection refused HTTP CRITICAL - Unable to open TCP socket

Any ideas how I can fix this issue?

Solution

The plugins within Nagios are really the secret sauce of what makes Nagios so powerful. The check that performs the HTTP check is this one:

$ /usr/lib64/nagios/plugins/check_http
check_http: Could not parse arguments
Usage:
 check_http -H <vhost> | -I <IP-address> [-u <uri>] [-p <port>]
       [-w <warn time>] [-c <critical time>] [-t <timeout>] [-L] [-a auth]
       [-b proxy_auth] [-f <ok|warning|critcal|follow|sticky|stickyport>]
       [-e <expect>] [-s string] [-l] [-r <regex> | -R <case-insensitive regex>]
       [-P string] [-m <min_pg_size>:<max_pg_size>] [-4|-6] [-N] [-M <age>]
       [-A string] [-k string] [-S] [--sni] [-C <age>] [-T <content-type>]
       [-j method]

Notice that there's a switch, -p <port> which you can override the default which is port 80, if it isn't specified.

Example

$ /usr/lib64/nagios/plugins/check_http -H fan
HTTP OK: HTTP/1.1 200 OK - 5233 bytes in 0.025 second response time |time=0.024596s;;;0.000000 size=5233B;;;0

To change the port, or to be explicit:

$ /usr/lib64/nagios/plugins/check_http -H fan -p 80
HTTP OK: HTTP/1.1 200 OK - 5233 bytes in 0.003 second response time |time=0.002890s;;;0.000000 size=5233B;;;0

If you're using a default check within Nagios, you'll need to likely create your own custom service check that calls out port 3000 instead of the default port 80.

Customizing a service check

You can create and/or modify your HTTP service check so that it'll take an additional argument, in this case the TCP port to use.

Details are further discussed in this article titled: Working with check_http and check_bigip_pool to monitor web sites.

Example

excerpt

Here is how I defined the commands. For separation purposes, I prefer to the definitions within a distinct configuration file and give each command name a distinct definition. This way if one port is not accessible, you can address the one port issue and not assume the entire web application is down.

Command Definition > (nagios_host:/path_to_nagios/etc/objects/commands/cmd_http.cfg):

   define command{
           command_name    check_webstore
           command_line    $USER1$/check_http  -H $ARG1$ -u $ARG2$ -p $ARG3$
           }

Comments

[cuonglm]

What nagios command you use to check service status?

[artajalli]

i use the web-interface ...

[cuonglm]

You should change the check command in nagios to check for port 3000 instead of 80 by default.

[artajalli]

I had suspected that I souhld change something in Nagios ... but how can i change the check command to listen for port 3000?!

[cuonglm]

You should give us more details about your nagios config, then we can give you some hint.

[artajalli]

i did not touch the default config ... just the httpd.conf file and set the 3000 port for http access.

[artajalli]

thanks for the guidance .. but how can i change the default port to 3000?! I ran the check and got the bellow result HTTP WARNING: HTTP/1.1 403 Forbidden - 5237 bytes in 0.041 second response time |time=0.041062s;;;0.000000 size=5237B;;;0

[slm]

@art2014 - does the sever respond on http://hostname/? That looks like it responded just fine but the URL isn't quite right. Also are you setting Nagios up directly using the .cfg files under /etc/nagios/?

[slm]

@art2014 - see updates.

[artajalli]

thanks for your answer ... Since I am new to nagios and haven't worked with plugins yet ( I suppose your answer is based on plugins syntax, or if I am wrong correct me ), I presumed that it would be much more straightforward to change the default HTTP port from 80 to 3000 in nagios. By the way, Nagios web interface works fine; just the HTTP CRITICAL notification. Meanwhile, I'll try to figure out your answer.

[slm]

@art2014 - The .cfg files for Nagios are installed somewhere on your system. I'd start with the directory /etc/nagios. Are there files there?

[artajalli]

it is under /usr/local/nagios; I looked upon a couple of files but didnt find anything relating to this issue ... ( I searched the nagios.cfg and localhost.cfg files).

[slm]

@art2014 - where did this Nagios install come from?