wget doesn't take variable http_proxy

Question

If I run:

echo $http_proxy

It output:

http://user:pass@host:port

But when I run:

wget www.google.com

It output:

Resolviendo www.google.com... 209.85.195.104
Caching www.google.com => 209.85.195.104
Conectando a www.google.com|209.85.195.104|:80... Closed fd 3
falló: Expiró el tiempo de conexión.
Releasing 0x088293a8 (new refcount 1).
Reintentando.

--2010-07-07 19:40:00--  (intento: 2)  http://www.google.com/
Found www.google.com in host_name_addresses_map (0x88293a8)
Conectando a www.google.com|209.85.195.104|:80...

In English:

Resolving www.google.com... 209.85.195.104
Caching www.google.com => 209.85.195.104
conecting to www.google.com|209.85.195.104|:80... Closed fd 3
fail: conection time has expired.
Releasing 0x088293a8 (new refcount 1).
retrying.
...

What I am doing wrong?

NOTE: the https_proxy and ftp_proxy have the same value. And I already try to modify /etc/wgetrc and reinit without success.

Solution 1

Thanks for the debug logs.

So you're clearly not using up the env proxy settings you expect based on that output.

See http://www.gnu.org/software/wget/manual/wget.html#Proxies for some alternate methods of configuring the proxy settings.

It's not clear if you're in some kind of controlled environment, but it's also possible that the wget you're calling has been modified in some how -- perhaps there's a wrapper script that's stripping out the environment or disabling the proxy settings from being used (--no-proxy). Does file $(which wget) show it's an executable?

Solution 2

In order for variables defined in your shell to be passed to subprocesses, they have to be exported. Failing to export them can cause the issue you describe. Here's an example from my own terminal:

First, I try to download the page, and discover it's not using my proxy.

sh-3.2$ wget serverfault.com
--2016-11-03 18:46:15--  http://serverfault.com/
Resolving serverfault.com... 151.101.65.69, 151.101.129.69, 151.101.193.69, ...
Connecting to serverfault.com|151.101.65.69|:80... ^C

I discover the error of my ways, and define the environment variable.

sh-3.2$ http_proxy="http://proxy.example.com:80/"
sh-3.2$ echo $http_proxy
http://proxy.example.com:80/
sh-3.2$ wget serverfault.com
--2016-11-03 18:46:53--  http://serverfault.com/
Resolving serverfault.com... 151.101.65.69, 151.101.129.69, 151.101.193.69, ...
Connecting to serverfault.com|151.101.65.69|:80... ^C

Unfortunately, it still doesn't work. The problem is that I didn't export the variable after I defined it, so subprocesses can't see it.

sh-3.2$ env | grep http_proxy
sh-3.2$ export http_proxy
sh-3.2$ env | grep http_proxy
http_proxy=http://proxy.example.com:80/

wget should now attempt to use my proxy.

sh-3.2$ wget serverfault.com
--2016-11-03 18:48:30--  http://serverfault.com/
Resolving proxy.example.com... failed: nodename nor servname provided, or not known.
wget: unable to resolve host address 'proxy.example.com'

Comments

[medina]

Can you elucidate what's in the [...] error there? Are you seeing [proxyhost|ip_address|:port], as expected with $http_proxy set correctly? Could you share an obfuscated version of wget --debug www.google.com?

[damian]

ok, but it's Spanish

[damian]

file $(which wget) outputs: /usr/bin/wget: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.15, stripped