How to download big file with chrome on Mac OSX?

Question

If I try to download a big file on unstable connection/server (XCode 4) Google chrome simply "stops" downloading on first network error so I have a first 1-2-3 gigabytes of file and chrome thinks that download is finished.

Unfortunately, I need to download an entire file, so I need a more advanced download tool like a wget. But there comes a problem: most URL's currently on the web is not a direct URL but multiple "redicrect" pages that utilize complex javascript in order to generate next url and redirect browser to it. Chrome handles such things ok, but if I try to supply such URL to wget it will download some "intermediate" page as a file - not a file itself but an HTML page with complex redirect javascript.

is it any way to get a direct URL from chrome or to somehow discover it so I can use it with wget? Maybe it's some avanced download manager integrated in chrome that I just need to install? I use MacOS X 10.6.6 and latest Google chrome.

Solution 1

The actual path to the downloaded file is stored in the file's metadata when downloading using Safari. Open the file's Get Info dialog to see it.

I'm not sure if it also applies to Chrome.

Solution 2

Using the chrome Feature: Copy as cURL. For details, check this post

Thanks for comment. To make this post self-contained, I quote the main procedure from the blog.

turn on the developer tools in Chrome, make the request, and check the "Network" tab. There I see requests for the page itself and also each of the assets. Right-click on any of the entries there, and you'll see the option to copy the request as curl:

When you paste the contents of your clipboard onto the command line, you'll see something like this:

curl 'http://www.lornajane.net/' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Connection: keep-alive' -H 'Accept-Encoding: gzip,deflate,sdch' -H 'Host: www.lornajane.net' -H 'Accept-Language: en-GB,en-US;q=0.8,en;q=0.6' -H 'User-Agent: Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1535.3 Safari/537.36' --compressed

And don't forget to use the -o train.zip options, which switch to save it to a file. Otherwise, it'd print the entire content on the console.

Solution 3

This answer just doesn't apply to Chrome but to big downloads in general.

If the download is very large, I try to download it via lynx or curl to a server I have elsewhere (in my case, I have a VPS account with Dreamhost). The transfer speed is quite fast and then I just just whatever tool I want to download it to my local machine.

If you need to download it in pieces (because the large file size is cumbersome), you can always use the unix 'split' command; assuming the files are in binary, you may have to convert them to a suitable 'text' format, but the command itself is very straightforward, as is reassembling the pieces.

Comments

[Darokthar]

Just a shot into the blue: Google for "download manager mac os" and try some of the programs?

[tobylane]

Try another browser. Even though Safari and Chrome have the same render, the download managers are different.

[mmmonk]

Yes, the path is here but trying to download it with wget raises a "403 forbidden" error :(. It seems that chrome supplies some additional info to HTTP GET request :(

[HikeMike]

@Eye It's probably a dynamic download location where the web site somehow authenticates you, e.g. through a login. In that case, there's little (and nothing I know) you can do about that.

[mmmonk]

I have tried to download XCode 4 using Lynx, but have no success :(. Apple developers website navigation is composed of subtle javascript code and redirections so lynx simply can't see a beautiful "download now" button that is, actually, is just an image with lots of javascript "on click" code :(

[HikeMike]

@EyeofHell Start the download in Safari, abort instantly, then right-click and select Copy Address. Try using that in wget/curl. The URL might change between restarts, try to configure your downloader to pick up where the previous download aborted.

[Mokubai]

Whilst this may theoretically answer the question, it would be preferable to include the essential parts of the answer here, and provide the link for reference.