How do I ignore a proxy if not available?

Question

Following the instructions in Best way to cache apt downloads on a LAN?, I've set up a caching proxy in my local network. Since that machine is not always up, I'd like to be able to refresh the sources list and install packages without using that proxy if unavailable.

I've already read the Acquire group section in the manual page of apt.conf(5), but I couldn't find an option like "Silent-Fail".

At the moment, sudo apt-get update and related commands fail because no connection could be established. So how do I configure the client so that the proxy is ignored if it's not available?

Solution 1

There is an undocumented setting, Acquire::http::ProxyAutoDetect. This setting should contains the full path to the binary and cannot have arguments. The command should output the proxy to be used (example: http://10.0.0.1:8000).

Given the above information, a script could be created that tries a proxy before setting it. If no proxy is available, a direct connection should be used.

Below is such a proxy detection script which tries the http://10.0.0.1:8000/ and http://10.0.0.2:8000 proxies.

Put the code in /etc/apt/detect-http-proxy:

#!/bin/bash
# detect-http-proxy - Returns a HTTP proxy which is available for use

# Author: Lekensteyn <[email protected]>

# Supported since APT 0.7.25.3ubuntu1 (Lucid) and 0.7.26~exp1 (Debian Squeeze)
# Unsupported: Ubuntu Karmic and before, Debian Lenny and before

# Put this file in /etc/apt/detect-http-proxy and create and add the below
# configuration in /etc/apt/apt.conf.d/30detectproxy
#    Acquire::http::ProxyAutoDetect "/etc/apt/detect-http-proxy";

# APT calls this script for each host that should be connected to. Therefore
# you may see the proxy messages multiple times (LP 814130). If you find this
# annoying and wish to disable these messages, set show_proxy_messages to 0
show_proxy_messages=1

# on or more proxies can be specified. Note that each will introduce a routing
# delay and therefore its recommended to put the proxy which is most likely to
# be available on the top. If no proxy is available, a direct connection will
# be used
try_proxies=(
10.0.0.1:8000
10.0.0.2:8000
)

print_msg() {
    # \x0d clears the line so [Working] is hidden
    [ "$show_proxy_messages" = 1 ] && printf '\x0d%s\n' "$1" >&2
}

for proxy in "${try_proxies[@]}"; do
    # if the host machine / proxy is reachable...
    if nc -z ${proxy/:/ }; then
        proxy=http://$proxy
        print_msg "Proxy that will be used: $proxy"
        echo "$proxy"
        exit
    fi
done
print_msg "No proxy will be used"

# Workaround for Launchpad bug 654393 so it works with Debian Squeeze (<0.8.11)
echo DIRECT

Now, APT must be configured to use the above proxy detection script, so put the following code in /etc/apt/apt.conf.d/30detectproxy:

# Fail immediately if a file could not be retrieved. Comment if you have a bad
# Internet connection
Acquire::Retries 0;

# undocumented feature which was found in the source. It should be an absolute
# path to the program, no arguments are allowed. stdout contains the proxy
# server, stderr is shown (in stderr) but ignored by APT
Acquire::http::ProxyAutoDetect "/etc/apt/detect-http-proxy";

I've also put the next code to the file to prevent some host from being proxified.

# Override the default proxy, DIRECT causes a direct connection to be used
Acquire::http::Proxy {
    deb.opera.com DIRECT;
    dl.google.com DIRECT;
};

By default, the script outputs whether a proxy is used or not. To disable that, edit /etc/apt/detect-http-proxy and change show_proxy_messages=1 to show_proxy_messages=0.

Solution 2

There's now an officially supported way to do this - using the option - Acquire::http::Proxy-Auto-Detect (see apt.conf man page). Behaviour is similar to the old undocumented Acquire::http::ProxyAutoDetect (note presence/absence of hyphens in new/old config options), it's largely backwardly compatible, but has been extended...

I'm in the process of submitted a patch to the apt maintainers to improve the documentation, but since this is unlikely to make it into a version of apt which ships with a distro release for quite a while, I'll include the text of the proposed patch here:

Acquire::http::Proxy-Auto-Detect can be used to specify an external command to discover the http proxy to use. APT may invoke the command multiple times, and will pass a URI to the command as its first and only parameter. APT expects the command to output the proxy which is to be used to contact the URI in question on its stdout as a single line in the style http://proxy:port/, or the word DIRECT if no proxy should be used. No output indicates that the generic proxy settings should be used.

Note that auto-detection will not be used for a host if a host-specific proxy configuration is already set via Acquire::http::Proxy::HOST.

To diagnose interactions with the external command, set Debug::Acquire::http=yes and/or Debug::Acquire::https=yes e.g using the -o command line parameter.

Note that is using a pre-release version of apt, versions 1.3~exp2 to 1.3 then there is a bug (likely fixed by 1.3.1) which causes apt to parse the stderr of the external command along with the stdout.

Solution 3

/etc/apt/apt.conf.d/02proxy:

Acquire::http::Proxy-Auto-Detect "/usr/local/bin/apt-proxy-detect.sh";

/usr/local/bin/apt-proxy-detect.sh:

#!/bin/bash
IP=192.168.88.1
PORT=3142
if nc -w1 -z $IP $PORT; then
    echo -n "http://${IP}:${PORT}"
else
    echo -n "DIRECT"
fi

Command Line

• It needs nc to work (sudo apt-get install netcat) if missing.
• Make sure you chmod +x /usr/local/bin/apt-proxy-detect.sh
• Use the full path when specifying the script.

How it works

If it can connect to a proxy, it prints the proxy out APT uses that. If it can't, it prints out DIRECT and APT chugs along normally.

sauce

Comments

[Garth Kidd]

Lekensteyn, could I have your permission to paste that into puppet-apt-cacher-ng as an option to apt-cacher-ng::client?

[Lekensteyn]

Permission granted, do whatever you want with it :)

[geirha]

I'm very happy I found this answer when I set up apt-cacher-ng a while back. I have apt-cacher-ng installed on one of my home-computers (192.168.0.2). The script felt a bit overkill for my case, so I wrote a simplified detect-http-proxy: if nc -w1 -z 192.168.0.2 3142; then printf http://192.168.0.2:3142; else printf DIRECT; fi. Let's just hope the undocumented feature gets removed :)

[Lekensteyn]

I documented the script extensively for your benefit, if you care about size you can make it even shorter with i=192.168.0.2;nc -zw1 $i 3142&&echo http://$i:3142/||echo DIRECT :p

[Lekensteyn]

See the first link in the question where Jorge suggests the squid-deb-proxy-client package. This may work with desktop computers that do not move, but not so much with laptops that move between networks. Hence this question :-)

[leggewie]

Sorry, but you are utterly mistaken. It works very well with laptops that move a lot. That is my primary use case and where squid-deb-proxy-client really shines. As well, Jorge does not mention squid-deb-proxy-client. The currently accepted answer is unnecessarily cumbersome as well as outdated (it uses the obsolete and never-published Acquire::http::ProxyAutoDetect config). You are reinventing the wheel when there is supported software right in the Ubuntu repositories. If you feel that works better for you, no problem ;-)

[Lekensteyn]

9 years ago, when I posted the question and answer, squid-deb-proxy-client did not work and would still try to connect to unavailable hosts. It appears that this issue finally got fixed 7 years ago (bug, patch).

[Lekensteyn]

Additionally, while the ProxyAutoDetect option has been renamed, the mechanism is not obsolete and the legacy name is still in use by squid-deb-proxy-client (rev 150). If you have Avahi enabled, I guess that your suggested client is the easiest to set up, but it might be interesting to keep in mind how things work behind the scenes.

[Greg]

For those looking for documentation on it, in more recent versions of apt the documentation has moved from apt.conf manpage to apt-transport-http.

[Luke Mlsna]

This feature is no longer undocumented.