wget ftp connection failing after PASV command

Question

In attempting to transfer all files from one web server ("source") to another ("destination"), the wget command is connecting via FTP, but cannnot proceed beyond the PASV command.

I'm using an SSH connection to the "destination" server (a Linux box on shared hosting) to run the wget command.

The "source" server is a Microsoft server, and the FTP client on my desktop has no problem with it.

Here's the command I'm using to initiate the transfer:

wget -m ftp://username:'password'@sourceserver.com

The login is successful, then these commands are issued:

==> SYST ... done.      ==> PWD ... done.
==> TYPE I ... done.    ==> CWD not needed.
==> ... couldn't connect to xxx.xxx.xxx.xxx port 1128: Connection timed out
Retrying.

With the "couldn't connect" error, on each retry, it attempts a different port number (not 21, which it has already successfully connected to). The first time I made a note of the error, it tried ports in the 487X range.

I can't tell if the issue is on the Microsoft ("source") server side or on the Linux ("client") side.

Thoughts?

Solution 1

For file transfers or directory listings FTP opens additional TCP connections on dynamic ports. In active mode the client creates a local listener and let the server know about its IP:Port using the PORT command and the server then connects to the clients port (usually from port 20 on the server side). In passive mode the server opens the port and let the client know where it listens in response to the clients PASV command.

Both modes need

• an IP reachable by the other side, e.g. active mode with a client behind a simple NAT router will not work
• none or a wide open firewall, because the ports on the listener side will be different for each connection.

If you don't have any problem to reach it from your desktop client it might be, that your desktop client is using active mode, while wget uses passive mode, or that there is no firewall/NAT router between your desktop and the server, but between your shared hosting and the server there is one.

Without getting more details about your setup its hard to speculate more.

Solution 2

Another way is to avoid the passive mode, add --no-passive argument in your wget command can do it.

wget -r --no-passive --no-parent ftp://account:<password>@<ip address>/folder/ -P /root

Comments

[Admin]

FTP always uses 2 connections. Port 21 is simply for control/commands. PASV mode is the client instructing the server 'hey, tell me where I can grab the data' instead of the standard way of the client telling the server 'hey send me the data here'. You've either got a firewall blocking access to that second port on the windows side, the desktop side, or somewhere in between; or you have a poorly set up NAT on either end.

[TheDavidJohnson]

Thanks, @yoonix. Since my FTP client from my desktop has no problems with the "source" server, I'm guessing the problem may be on the "destination" server (where I'm issuing the commands via FTP). Might there be a way to specify a proxy or otherwise bypass any firewall at the destination host?

[TheDavidJohnson]

Appreciate the input here. You've helped me narrow this down. I'll be reaching out to the hosting provider for the destination server and see what can be done with their firewall. Incidentally, my desktop client is using passive mode... just to settle any curiosity. Thanks!

[HBruijn]

In active mode the client opens the command connection to the server and sends the IP address port number that the client will use for the data connection, and the server opens a connection back to the client IP/port. Typically this will fail when the client is behind a firewall and/or a NAT gateway, which is the reason why passive FTP was derived in the first place. So in practice this is hardly a good solution.

[Arek]

For me, it was the best solution, thank you ytll21.