script to automatically test if a web site is available

Question

I'm a lone web developer with my own Centos VPS hosting a few small web sites for my clients. Today I discovered my httpd service had stopped (for no apparent reason - but that's another thread). I restarted it but now I need to find a way that I can be notified by email and/or SMS if it happens again - I don't like it when my client rings me to tell me their web site doesn't work!

I know there are probably many different possibilities, including server monitoring software. I think all I really need is a script that I can run as a cron job from my dev host (which is permanently running in my office) that attempts to load a page from my production server and if it doesn't load within say 30 seconds then it sends me an email or SMS. I'm pretty rubbish at shell scripting, hence this question.

Any suggestions would be gratefully appreciated.

Solution 1

Well... The most simple script, I cam write:

/usr/bin/wget "www.example.com" --timeout 30 -O - 2>/dev/null | grep "Normal operation string" || echo "The site is down" | /usr/bin/mail -v -s "Site is down" [email protected]

Add it to cron as:

* * * * * /usr/bin/wget "www.example.com" --timeout 30 -O - 2>/dev/null  | grep "Normal operation string" || echo "The site is down" | /usr/bin/mail -v -s "Site is down" [email protected]

But it is too simple to tell you what the problem is if it exists.

UPD: Now this one-liner checks for a specific string on the page ("Normal operation string"), which should appear only on normal operation.

UPD2: A simple way to send the error page in the e-mail:

/usr/bin/wget "www.example.com" --timeout 30 -O - 2>/dev/null | grep "Normal operation string" || /usr/bin/wget "www.example.com" --timeout 30 -O - 2>/dev/null | /usr/bin/mail -v -s "Site is down" [email protected]

It's minus is that the page is re-requested in case of first test failure. This time the request may be successful and you won't see the error. Of course, it is possible to store the output and send it as an attachment, but it will make the script more complex.

Solution 2

Take a look at this script:

• http://answers.google.com/answers/threadview/id/276934.html

curl is a command-line utility to fetch a URL. The script checks the exit code ($? refers to the exit code of the most recent command in a shell script) and if it was anything other than 0, reports an error (an exit code of 0 generally refers to success). As mentioned in HUB's answer, you can also just || on the command-line to run a second command when the first one fails.

Once you have the status figured out, you just have to send yourself some mail. Here is an example that uses the mail command to send mail from a shell script, assuming the box you're testing from has SMTP setup:

• http://www.crazysquirrel.com/computing/debian/scripts/email-via-script.jspx

BTW: if you're not good at shell scripting, don't limit yourself to a shell script. You could use a ruby script, a php script, any kind of script your server can run! Just add the #!/path/to/executable line at the beginning of the script - for instance:

#!/usr/bin/php

Solution 3

Check this script. it's checking against a list of websites and sends email (to list of emails) whenever something wrong (http response different from 200). The script creates a .temp file to "remember" which website(s) failed at last check so you won't get multiple emails. the .temp file is deleted when the website is working again.

#!/bin/bash
# list of websites. each website in new line. leave an empty line in the end.
LISTFILE=/scripts/isOnline/websites.lst
# Send mail in case of failure to. leave an empty line in the end.
EMAILLISTFILE=/scripts/isOnline/emails.lst

# `Quiet` is true when in crontab; show output when it's run manually from shell.
# Set THIS_IS_CRON=1 in the beginning of your crontab -e.
# else you will get the output to your email every time
if [ -n "$THIS_IS_CRON" ]; then QUIET=true; else QUIET=false; fi

function test {
  response=$(curl --write-out %{http_code} --silent --output /dev/null $1)
  filename=$( echo $1 | cut -f1 -d"/" )
  if [ "$QUIET" = false ] ; then echo -n "$p "; fi

  if [ $response -eq 200 ] ; then
    # website working
    if [ "$QUIET" = false ] ; then
      echo -n "$response "; echo -e "\e[32m[ok]\e[0m"
    fi
    # remove .temp file if exist.
    if [ -f cache/$filename ]; then rm -f cache/$filename; fi
  else
    # website down
    if [ "$QUIET" = false ] ; then echo -n "$response "; echo -e "\e[31m[DOWN]\e[0m"; fi
    if [ ! -f cache/$filename ]; then
        while read e; do
            # using mailx command
            echo "$p WEBSITE DOWN" | mailx -s "$1 WEBSITE DOWN" $e
            # using mail command
            #mail -s "$p WEBSITE DOWN" "$EMAIL"
        done < $EMAILLISTFILE
        echo > cache/$filename
    fi
  fi
}

# main loop
while read p; do
  test $p
done < $LISTFILE

Add the following lines to crontab config ($ crontab -e)

THIS_IS_CRON=1
*/30 * * * * /path/to/isOnline/checker.sh

Available on Github

Solution 4

I know that all the above scripts are exactly what you've asked, but I would suggest looking at monit because it will send you an email if apache is down but it will also restart it (if it's down).

Solution 5

I would recommend pingdom for this. Their free service allows you to check 1 site, but that's all you need to check 1 server. If you have an iPhone they push-message you for free, so no need to buy SMS credits from them, and they have multiple settings you can use. Mine is set to notify me after 2 retries (10min) and every 10min downtime after that. It's awesome, since it also checks for HTTP 500 messages indicating a site is down. If it fails, it immediately checks your site again from a different server in a different location. If that one fails, well, that triggers your preference in how/when you'd like to get notified.

View more solutions

Comments

[Toby Mao]

Have you looked into Nagios or Pingdom? They have that functionality built in (Well, Pingdom has SMS built in, with Nagios it requires a bit of tweeking but it is possible)

[Xoundboy]

No, I'm going to take a look now, thanks for the advice.

[Xoundboy]

Looking interesting....so if the page doesn't load within 30 seconds an email is sent right? What if the page loads but an error message it displayed - how could this be adapted to test for a specific output?

[Xoundboy]

I really like this Pingdom service - just set up the free account and tested the check and the SMS to my Czech mobile number - both work - will try this out for the time being and see how it goes.

[HUB]

Updated the answer. But the script still doesn't send you the error.

[Xoundboy]

- hi, thanks for the update - I'm struggling to get this to work on my machine - I don't think the mail part of the command is doing its thing properly - haven't got any time now to troubleshoot but will try again tomorrow or asap.

[Xoundboy]

This also looks interesting - I'll try to find time to check it out soon and p[ost back my findings.

[HUB]

Well... Try to run mail program manually and check the output. Also see "/var/log/mail" for results Check the firewall settings (access to remote port 25 should be permitted).

[John Gardeniers]

I like the simplicity but that won't test if the web site is available over the Internet. The service could be running but not actually serving clients. I've fallen for that one before.

[frogstarr78]

Agreed, although, if you want to ensure that it's accessible from the Internet, you'd have to test it over the Internet. Even running wget on the server to it's external network interface wouldn't be an accurate test that the Internet can access it.

[Ben Johnson]

Sadly, Pingdom no longer appears to offer the Free tier. The least expensive plan is $14.95/month.

[Riz]

You can use cronitor.io - they offer one free site for checking and can push notifications to email, slack or any other webhook.

[trolologuy]

Another alternative to pingdom would be uptimerobot, 5 minutes monitoring intervals and 50 monitored website in the free plan. It integrates easily with slack and telegram too (besides push, SMS or email notification, ...).

[Tobias Gaertner]

If you just whant to break the script, e.g. in a deployment chain, you can do "wget "www.example.com" --timeout 30 -O - 2>/dev/null | grep "Normal operation string" || echo "The site is down" | exit 1". With this you can also make as many tests after each other as you like.

[Chaminda Bandara]

Who done this is please ? @shahid shaik

[shahid shaik]

@ChamindaBandara I have created this script for myself to monitor few websites