How do I recursively copy/download a whole webdav directory?

Question

When I attempt to copy a folder from a webdav server to a local disk using Nautilus, it copies what appeas to be a manifest file (xml with the directory listing etc..). With cadaver I get an empty file.

I would like to be able to recursively copy a whole directory tree. Does anyone know how I can do this?

ps: I'm using Ubuntu 11.04 with Nautilus 2.32.2.1 and Cadaver 0.23.3

Solution 1

This answer summarises suggestions given in comments by @Ocaso and @Rinzwind.

I used this:

wget -r -nH -np --cut-dirs=1 --no-check-certificate -U Mozilla --user={uname} 
    --password={pwd} https://my-host/my-webdav-dir/my-dir-in-webdav

Not perfect (downloaded lots of 'index.html?C=M;O=D' and the like) but otherwise worked ok.

The "-r" downloads recursively, following links.

The "-np" prevents ascending to parent directories (else you download the whole website!).

The "-nH" prevents creating a directory called "my-host" (which I didn't want).

The "--cut-dirs=1" prevents creating a directory called "my-webdav-dir".

The "--no-check-certificate" is because I'm using a self-signed certificate on the webdav server (I'm also forcing https).

The "-U Mozilla" sets the user agent in the http request to "Mozilla" - my webdav server didn't actually need this, but I've included it anyway.

Solution 2

Alternatively you can mount it as a path to be accessed as part of your own file system.

sudo mount -t davfs https://your.remote/path /your/local/mount/point

Note: /your/local/mount/point has to be a real existing directory for this to work.

As far as I know you only need to run the following to get the command to work: sudo apt-get install davfs2 (If more configuration is required I apologise, it was a long time ago that I did this.)

(I added this as an answer because I feel Liam's answer didn't give enough info.)

Solution 3

I realize that this is an old question, but I was wanting to do this and I found rclone which is like rsync for the cloud and supports a lot of different protocols including WebDAV. You use rclone config to configure a remote and then just do rclone copy remote:path . and int will download everything. You can also use rclone sync remote:path . and it will do checking of the files and only copy updates. It is a golang program so it is very portable.

Solution 4

Actually with Cadaver you can cd to directory from which you want to download files and mget *.

Solution 5

You can use dav2fs to mount the webdav server, and then you can access it as you would a local directory.

View more solutions

Comments

[Jason Southwell]

For this to be a useful answer, you need to tell the user how to do this.

[bobbyrne01]

also use this method, easy access to files and folders

[alexanderadam]

Hm, mget * . in a folder or mget foldername results in 501 Not Implemented. Did it really work for you on box.com?

[alexanderadam]

And this method is the slowest in comparison with others and won't always work (bugs related to filename-squirks).

[JasoonS]

@Wurstsalat You mustn't put the extra '.' ; also it takes filenames as arguments, so you will need to do `mget foldername/filename'.

[Equidamoid]

using sudo to download a file doesn't really sound right.

[Radon Rosborough]

That doesn't work for me when there's subdirectories (the recursive part).

[Radon Rosborough]

@Equidamoid Mounting filesystems requires root access because it can allow you to bypass local UNIX permissions in some cases.

[Micromegas]

Using this to download a file from nextcloud it does download a file with the content: This is the WebDAV interface. It can only be accessed by WebDAV clients such as the Nextcloud desktop sync client.. Any idea why that happens?

[drevicko]

My guess is that nextcloud expects a user agent that looks like a WebDAV client (as opposed to Mozilla, which is a browser). If you have their client and a packet sniffer (eg: wireshark) you could find out what user agent it uses in it's requests.

[drevicko]

Alternatively find out that it's user agent could be "Mozilla/5.0 (Android) Nextcloud-android/3.8.0" from the web.

[Micromegas]

Thank you very much drevicko!! I'll do this

[drevicko]

Also here. Good luck!