Download ALL files from Website with directory listing

Question

I know that it's possible but I'm having problem when I want to enter the folder with 'index.html / index.php', it automatically redirects me to the site, instead of opening the folder. You can test it here.

http://svamservices.com.au/truck/ (archived version) And try entering the main folder.

I would like to get the contest of the main folder. Any idea? I tried with wGet but it just skips the main folder.

Solution

When you running following script (creating mirror of a site using wget):

#!/bin/sh

wget --mirror           \
     --convert-links    \
     --adjust-extension \
     --page-requisites  \
     --no-parent        \
http://svamservices.com.au/truck/

it will give you content that was produced by a web server. If web server allows directory listing then it will return your list of files and directories, otherwise it will execute index.php on its side and return you result of this execution (logic inside of index.php).

Comments

[Seth]

It's by design. Otherwise wget http://<bigcompany/ would mean you could read a lot of data you're not supposed to read.

[Seth]

If you give that link a shot you'll see that the "redirection" is merly the delivery of whatever index file is configured. As long as there is no strange configuration to not deliver the index for wget this isn't really going to help in this case.

[Alex]

Ohh, you right, I thought that OP wanted content produced by index.php, but as I can see it now he wants listing of files

[Alex]

@Seth I edited my answer to reflect OP question. Tnx for catching my error

[Ramhound]

Can you edit your answer, so you specifically call out the configuration file, and specifcally indicate what edits should be done? People like myself might know what to do, but many, do not.

[Andy]

I note that the site is running on LightSpeed webserver. I would not say I was sufficiently adept with this server to comment on the perfect method to make this change. I note that it is supposedly Apache compatible but I'd rather not add information I cant verify. I can add for Apache and Nginx if you feel this is still of value?