Checking URLs for HTTP code 200

Question

I have a text file containing a list of URLs like this:

http://somesite.com/some-article/
https://anothersite.fr/somepage.php?page=something

What I'd like to do is check which URLs return HTTP code 200 (OK) and put them in a list in another file.

How could I accomplish this using terminal? Curl?

Solution

Try doing this using bash :

while read url ; do
    ret=$(curl -I -s "$url" -o /dev/null -w "%{http_code}\n")
    ((ret==200)) && echo "$url" >> new_list_URL
done < list_URL

or POSIX mode :

while read url ; do
    ret=$(curl -I -s "$url" -o /dev/null -w "%{http_code}\n")
    [ $ret -eq 200 ] && echo "$url" >> new_list_URL
done < list_URL

Check man curl

Comments

[Eddie M. Patrick]

Thank you sputnick, but I can't seem to make it work. For some reason it doesn't output to new_list_URL. What could be the problem?