Curl. Check redirect

Question

Lets suppose that we have 3 links: link1, link2, link3. link1 redirects to link2 and link2 redirects to link3. So how to see that with curl?

Solution 1

You can see the HTML headers using -I. If the redirect is a meta-refresh it would should up in this way as a header.

curl -I http://google.com

HTTP/1.1 301 Moved Permanently
Location: http://www.google.com/
Content-Type: text/html; charset=UTF-8
Date: Thu, 21 Nov 2013 14:59:13 GMT
Expires: Sat, 21 Dec 2013 14:59:13 GMT
Cache-Control: public, max-age=2592000
Server: gws
Content-Length: 219
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Alternate-Protocol: 80:quic 

If the redirect is happening via PHP, you could detect this by comparing where the browser is going vs. where it's actually going. There are a bunch of ways to do this with Python, JS, etc. One project which may be interesting to you is phantomjs, a scriptable headless browser.

Solution 2

Try this :

for link in link1 link2 link3; do
    curl -Is "$link" | awk '/Location/{print $2}'
done

Or using netcat :

for link in link1 link2 link3; do
    printf '%s\n%s\n\n%s\n' 'HEAD / HTTP/1.1' "Host: $link" 'Connexion:close' |
    netcat $link 80 | awk '/Location/{print $2}'
done

Solution 3

From man curl:

   -w, --write-out <format>
          Defines what to display on stdout after a completed and
          successful operation.

          <...>

          redirect_url   When an HTTP request was made without -L to
                         follow redirects, this variable will show the 
                         actual URL a redirect would take you to.
                         (Added in 7.18.2)

So probably curl -w "%{redirect_url}" link1 will give you the first redirection url.

Maybe something like this works for you:

URL="http://google.com"
while [ -n "${URL}" ]
do
    echo $URL
    URL=$(curl -sw "\n\n%{redirect_url}" "${URL}" | tail -n 1)
done