How do I get (only) the http status of a site in a shell script?

Question

I figure curl would do the job. I wrote in a script:

#!/bin/sh

function test {
    res=`curl -I $1 | grep HTTP/1.1 | awk {'print $2'}`
    if [ $res -ne 200 ]
    then
        echo "Error $res on $1"
    fi
}  

test mysite.com
test google.com

The problem here is no matter what I do I can't get it to stop printing the below to stdout:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

I want a cronjob to run this script and if it writes such a message then every time I run it I'll get an email because something has been printed to stdout in cron, even though the site may be fine.

How do I get the status code without getting junk into stdout? This code works except the bonus junk to the stdout preventing me from using it.

Solution 1

   -s/--silent
          Silent or quiet mode. Don't show progress meter 
          or error messages.  Makes Curl mute.

So your res should look like

res=`curl -s -I $1 | grep HTTP/1.1 | awk {'print $2'}`

Result is Error 301 on google.com, for example.

Solution 2

you want

...
res=$( curl -w %{http_code} -s --output /dev/null $1)
...

That will give you the HTTP_STATUS code as the only output.

Solution 3

Use this as your condition...

res=`curl -s --head <URL> | head -n 1 | grep -c HTTP/1.1 200 OK`

if [ $res -eq 1 ]
then
MSG = " OKAY"
EXIT_CODE = 0
else
MSG = " NOT OKAY"
EXIT_CODE = 2
fi

Solution 4

I'd do it like this to ensure i get redirected to the final host and get the response from it:

res=($(curl -sLI "${1}" | grep '^HTTP/1' | tail -1))
((res[1] == 200)) || echo ${res[2]}

Solution 5

as pointed out above curl can natively provide the http response:

#!/bin/bash
#example1.sh
function test {
  RESPONSE=$(curl -so /dev/null -w "%{http_code}\n" ${1})
  if [[ $RESPONSE != 200 ]]; then
    echo "Error ${RESPONSE} on ${1}"
  fi
}    
test mysite.com
test google.com

Example 1 with -s we silence the progress & -o /dev/null lets us trash the response but -w is your ally here:

$ ./example1.sh 
Error 000 on mysite.com
Error 301 on google.com

This can be further simplified natively in curl by asking for the url_effective and redirect_url:

#!/bin/bash
#example2.sh
function test {
  curl -so /dev/null -w "%{http_code}\t%{url_effective}\t%{redirect_url}\n" ${1}
}  
test mysite.com
test google.com

Example 2 we see our initial http response, original requested domains, and redirect the server responded with:

$ ./example2.sh 
000 HTTP://mysite.com/  
301 HTTP://google.com/  http://www.google.com/

However if you're ultimately looking to establish a 200 response even after a 301 or 302 redirect: Then you can and drop the aforementioned 'redirect_url'

#!/bin/bash
#example3.sh
function test {
  curl -sLo /dev/null -w "%{http_code}\t%{url_effective}\n" ${1}
}  
test mysite.com
test google.com

Example 3 we add -L which instructs curl to chase the redirect(s):

$ ./example3.sh 
000 HTTP://mysite.com/
200 http://www.google.com/

View more solutions

Comments

[Admin]

Thank you! this irratated me so much. (and they should mention this under the "progress meter" in the man page)

[Anthon]

Welcome to U&L. Your first sentence states "as pointed out above" which is confusing as the position of your answer is depending on votes and user settings. It is better to use some line "As @some_user pointed out above" (with some_user being an actual name) or "As others have pointed out"