wget - Many URL's in .txt file - download and save as

Question

I have 2000 URLs in excel file. The URLs are in the first column and in the second column there are names for the files downloaded from URL in the first column. I can copy that and paste to .txt file if it's needed, no problem. File names contain spaces. I need to do this on Windows 7. Could you help me?

@Edit: Well, sorry If my problem is unclear. I'm not english native speaker. I have URL in first column and and I want to save the file downloaded from this URL with name from the second column. I want that spaces to be there. I want to download all the files with one command or batch file using "wget" tool.

Solution

Steps

1. Open your worksheet in Excel and click File → Save As.

2. Close Excel to unlock the file.

3. Choose CSV (comma separated values) as type and same your file as urls.csv.

4. Open a command prompt, execute

   type urls.csv
   

   and identify the value separator (character placed between URL and file name.

   If it's, e.g., a semicolon, execute the following command:

   for /f "delims=; tokens=1,2" %a in (urls.csv) do @wget -O "%b" "%a"
   

How it works

• Excel saves the URLs and corresponding names as comma (or semicolon) separated values.

  Example:

  http://foo;bar
  http://foo bar;foobar
  

• for /f ... %a (urls.csv) goes through all lines and saves the first value in %a and the second in %b.

  Here, delims=; specifies the semicolon as value separator and token=1,2 specifies that there will be two tokens.

• wget -O "%b" "%a" saves %a in %b. Since the URL is quoted, Wget will automatically take care of spaces and other special characters.

• The @ in front of @wget prevents the commands from being printed.

See also: For /f - Loop through text | SS64.com

Comments

[Rob]

wget -i will read a list of URLs from a file, I'm not sure how you'd get it to rename the files as it download them, though.

[user194380]

Well, sorry If my problem is unclear. I'm not english native speaker. I have URL in first column and and I want to save the file downloaded from this URL with name from the second column. I want that spaces to be there. I want to download all the files with one command or batch file using "wget" tool.

[user194380]

Here is the example of my URL, there is a redirection but wget has no problems with downloading. There are no spaces in the URLs. Will it work? dominiopublico.gov.br/pesquisa/…

[Karan]

I can (and did) try it, but would you care to try it yourself on some sample URLs and confirm?

[Karan]

Excel 2010 saves CSVs with commas for me.

[Dennis]

Excel 2003 seems to use semicolons.

[Karan]

Why call it "C"SV then and not SSV?!

[Dennis]

Slightly misleading, yes. Other applications use tabs. Semicolons are usually a better choice, since it's less likely that they will naturally occur in the cells. Now that I think of it, it's probably because my Office is in Spanish. We use the comma as a decimal separator, so actual CSVs would be a poor choice...

[Karan]

I don't think it's because of your Office language. Excel simply uses whatever character you've set as your OS' preferred List separator under Control Panel's Region and Language / Additional settings / Customize Format.

[Dennis]

I checked and it's set to ,, but I found the responsible setting: You can adjust the decimal separator in Excel (in 2003, it's in Tools, Options, International). If it's set to ., Excel uses actual comma separated values. If it's ,, it uses semicolons instead.

[user194380]

Thank you very much! It works. And thanks for the explanation! (LibreOffice Calc saves CVS with commas)

[Mike]

could you give an example for a csv with urls only please?

[Mike]

could you give an example for a csv with urls only please?

[Mike]

could you give an example for a csv with urls only please?

[Dennis]

@mugur: Would the URLs be separated by commas or newlines?

[Mike]

newlines ... I got stuck in this answer and I did search for other options as wget has an option for this in the form of wget -i file.csv. Sorry for bothering.