Resolve filename from a remote URL without downloading a file

Question

I am creating a script, which should download latest version of an application from repository and deploy the app.

The main issue: there are several repositories and I need to check, which of them has most recent version.

E.g.

http://repo1/xyz/LATEST -> (redirects to) -> http://repo1/xyz/app-1.0.0.0.zip
http://repo2/xyz/LATEST -> (redirects to) -> http://repo1/xyz/app-1.1.0.0.zip

So I need to iterate over available repositories and get only a filename - no need to download obsolette versions of software.

Solution 1

Something like this could work for you:

curl -sIkL http://repo1/xyz/LATEST | sed -r '/filename=/!d;s/.*filename=(.*)$/\1/'

Take a look at man page curl(1) for the options. The interesting one is -I, --head.

Explanation as requested per comments:

The idea is to request the HTTP response header only.

Therefore the -I options is used. -s silents curl to not print anything else than the header. -k allows "insecure" SSL connections (curl would reject self-signed certs otherwise). And -L to follow HTTP(S) location redirects.

Then sed(1) is used to get the file name from response header. We are searching for the filename= field, so the /filename=/!d part removes anything without that field from output. Finally the s/.*filename=(.*)$/\1/ part prints the file name only if the field is found.

Solution 2

I came out with this solution, quite similar to the one by @FloHimself.

curl -L --head http://repo1/xyz/LATEST 2>/dev/null | grep Location: | tail -n1 | cut -d' ' -f2

• -L lets curl follows the redirection
• --head makes it fetch only the headers and not the pages' content.
• grep Location: looks for the Location: header in 30x HTTP responses by the server
• tail -n1 selects the last one
• cut -d' ' -f2 selects the second field (the URL)

The same, but letting curl do all the work:

curl -L --head -w '%{url_effective}' http://repo1/xyz/LATEST  2>/dev/null | tail -n1

This solution use the -w, --write-out option to ask curl for a specific output. man curl gives the available variables.

Comments

[Bernhard]

Your answer would be more useful if you shortly explain -I here. Also, what is you sed command supposed to do?

[user1065145]

CooL! Needs few upgrades, but general idea is very helpful.

[EightBitTony]

I echo Bernhard's comment - by explaining what the different elements in your response do, you will help people who want to do something similar to the OP but not exactly the same.