wget is returning an html page instead of original file

Question

In my case the problem is with SourceForge.net. Actually I was trying to download these files:

http://sourceforge.net/projects/wxpython/files/wxPython/2.9.4.0/wxPython-src-2.9.4.0.tar.bz2/download
http://sourceforge.net/projects/wxpython/files/wxPython/2.9.4.0/wxPython-docs-2.9.4.0.tar.bz2/download

Something did not went good, so I logged wget, and the log file was:

--2012-08-21 09:52:17--  http://sourceforge.net/projects/wxpython/files/wxPython/2.9.4.0/wxPython-src-2.9.4.0.tar.bz2/download
Resolving sourceforge.net... 216.34.181.60
Connecting to sourceforge.net|216.34.181.60|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 18846 (18K) [text/html]
Saving to: `download'

     0K .......... ........                                   100% 6.30K=2.9s

2012-08-21 09:52:25 (6.30 KB/s) - `download' saved [18846/18846]

--2012-08-21 09:52:25--  http://sourceforge.net/projects/wxpython/files/wxPython/2.9.4.0/wxPython-docs-2.9.4.0.tar.bz2/download
Connecting to sourceforge.net|216.34.181.60|:80... connected.
HTTP request sent, awaiting response... 200 OK

    The file is already fully retrieved; nothing to do.

FINISHED --2012-08-21 09:52:28--
Downloaded: 1 files, 18K in 2.9s (6.30 KB/s)

As you can see in this log file, I end up in getting a single HTML file :(

---

This shows that wget doesn't waits on the page to let webpage to redirect it to another location, and it downloads the page itself.

Question:

I want those two files, with wget, how can I?

Solution

It seems redirection is disabled at your wget, try specifying --max-redirect=2 or a higher number. You probably also want to specify --trust-server-names to allow wget to update the file name after redirection, otherwise the downloaded files will still be called download.

Comments

[Santosh Kumar]

I set those two options, but still getting an HTML file.

[hayath786]

@Santosh Interesting, it works here. Which wget version do you use?

[Santosh Kumar]

GNU Wget 1.12 built on linux-gnu.

[Santosh Kumar]

Do you find any error in my .wgetrc file?

[hayath786]

@Santosh No, but have you tried it without?

[Santosh Kumar]

Hey, It is working! But I don't like passing those much argument everytime I download anything. What is wrong with my wget?

[Santosh Kumar]

I'm quite sure something is wrong with my .wgetrc file. I need some research on it.

[hayath786]

Try commenting out all configurations in your .wgetrc file. Then comment in one after the other and call the wget command from above. You should see which of your configurations leads to the strange behaviour.

[Santosh Kumar]

I found that --user-agent was creating the problem. I don't know why.

[Freddy]

No, you don't need any cookies. Try wget http://sourceforge.net/projects/wxpython/files/wxPython/2.9.‌​4.0/wxPython-src-2.9‌​.4.0.tar.bz2.

[krishnakeshan]

That's the solution for this particular website. I was mentioning the more common one.