How do I download all the files in a directory with cURL?

windows bash shell curl

15,204

Curl does not support recursive download. Use wget --mirror --no-parent [URL]

EDIT: For SSH, from the man page of curl:

Get a file from an SSH server using SFTP:

    curl -u username sftp://shell.example.com/etc/issue

Get a file from an SSH server using SCP using a private key to authenticate:

    curl -u username: --key ~/.ssh/id_dsa --pubkey ~/.ssh/id_dsa.pub \
            scp://shell.example.com/~/personal.txt

15,204

Alex Gordon

Check out my YouTube channel with videos on Azure development.

Updated on September 18, 2022

Comments

Alex Gordon over 1 year
I am using cURL to try to download all files in a certain directory.

Here's what my list of files looks like:

I have tried to do in bash script: iiumlabs.[].csv.pgp and iiumlabs* and I guess cURL is not big on wildcards.
```
curl -u login:pass ftp.myftpsite.com/iiumlabs* -O
```
Question: How do I download this directory of files using cURL?
- cjc almost 12 years
  
  Why not use wget with -r and maybe some other options instead? curl can be made to do it with some scripting around it, but there are other tools that are better suited.
- Philip almost 12 years
  
  Why not use good old ftp with a script file to mget all the files?
- Alex Gordon almost 12 years
  
  @ChrisS thank you, can i do sFTP with mget?
- Alex Gordon almost 12 years
  
  @cjc thank you! how would i do sftp with wget?
- Zoredache almost 12 years
  
  Is the server SFTP only, or can you login with SSH? If the server isn't limited to SFTP only, you would probably be better off using scp/rsync.
- Alex Gordon almost 12 years
  
  im sorry i dont know the difference. in filezilla to connect to it i connect to sftp.thesite.com
- Alex Gordon almost 12 years
  
  @Zoredache does what i said make any sense?
- Alex Gordon almost 12 years
  
  @Zoredache let me know if i can clarify something
Alex Gordon almost 12 years

thank yuou very much!!!!!! what is SCP? scp://..... if the key is stored on my machine can i just point to c:/.....
Mircea Vutcovici almost 12 years

scp is an old way to use ssh to transfer files. The latest version of ssh support SFTP. I do not know the syntax for local files in curl. You can enable some debugging options in curl. Besides curl you can use pscp and psftp to transfer files. See: chiark.greenend.org.uk/~sgtatham/putty/download.html.