Configure apt to use proxy in vagrant box

Question

I'm using Vagrant to setup a virtual machine for testing. I would like apt inside the virtual machine to use a proxy on the host machine to allow caching of all the downloads to persist between instances of the virtual machine running, not only for improved speed but to allow me to start the vagrant instance when I have no internet connection.

I've setup the proxy which is running, and I thought I'd told apt to use it by setting this in the Vagrant script:

config.vm.provision :shell, :inline => 'echo \'Acquire { Retries "0"; HTTP { Proxy "http://10.0.2.2:3128"; }; };\' >> /etc/apt/apt.conf'
config.vm.provision :shell, :inline => 'echo \'Acquire { Retries "0"; FTP { Proxy "ftp://10.0.2.2:3128"; }; };\' >> /etc/apt/apt.conf'
config.vm.provision :shell, :inline => 'echo \'Acquire { Retries "0"; HTTPS { Proxy "https://10.0.2.2:3128"; }; };\' >> /etc/apt/apt.conf'

And it's working partially i.e. the initial requests for sources hit the cache and work when my wifi connection is disabled but Squid is running and has some entries in it's cache. I can also see some requests hitting Squid from Squids log file:

1367492453.816     34 127.0.0.1 TCP_REFRESH_MODIFIED/200 592 GET http://security.ubuntu.com/ubuntu/dists/precise-security/Release.gpg - DIRECT/91.189.92.181 -
1367492453.987    168 127.0.0.1 TCP_REFRESH_MODIFIED/200 49973 GET http://security.ubuntu.com/ubuntu/dists/precise-security/Release - DIRECT/91.189.92.181 -
1367492453.999    325 127.0.0.1 TCP_MISS/404 588 GET http://us.archive.ubuntu.com/ubuntu/dists/precise/InRelease - DIRECT/91.189.91.13 text/html
1367492454.113    114 127.0.0.1 TCP_MISS/404 596 GET http://us.archive.ubuntu.com/ubuntu/dists/precise-updates/InRelease - DIRECT/91.189.91.13 text/html

However, the bulk of the download of packages are not being cached and doesn't appear to go through Squid at all i.e. I can see large network usage when they're downloaded and they don't appear in the Squid access log.

So my question how do I configure Apt to use a proxy for all it's requests?

btw I also tried configuring by setting the environment variable http_proxy:

 config.vm.provision :shell, :inline => "echo 'export http_proxy=http://10.0.2.2:3128' >> /etc/profile.d/proxy.sh"

This has the same effect - some requests appear to be hitting Squid but not all of them, and particularly not the actual packages.

Solution 1

Its possible that apt is using squid for all its requests, but squid isn't caching the results. You could verify this by sniffing traffic, or shutting down squid while apt is downloading packages and seeing if they fail. What is the maximum_object_size in your squid configuration?

Solution 2

Even though your problem might be in the squid config, to answer to the topic, there is now vagrant-proxyconf plugin. You can install it via:

vagrant plugin install vagrant-proxyconf

With it you can specify the Apt proxy globally in $HOME/.vagrant.d/Vagrantfile without the need to use shell provisioners in all project specific Vagrantfiles.

Example:

Vagrant.configure("2") do |config|
  config.apt_proxy.http  = "http://10.0.2.2:3128"
  config.apt_proxy.https = "http://10.0.2.2:3128"
end

or default proxy configuration:

Vagrant.configure("2") do |config|
  if Vagrant.has_plugin?("vagrant-proxyconf")
    config.proxy.http     = "http://192.168.0.2:3128"
    config.proxy.https    = "http://192.168.0.2:3128"
    config.proxy.no_proxy = "localhost,127.0.0.1,.example.com"
  end
  # ... other stuff
end

Comments

[Danack]

Hi Michael - the accepted answer to that question is to set "Acquire::http { Proxy "proxy:port"; };" which I'm already doing. I think the error may be on the squid side so will update it with relevant info.

[Danack]

It was set to 128MB which I though was enough. However changing it to 256MB makes the actual packages appear in the squid log. It looks like Squids still not caching them, but now they are appearing and I'm investigating with your suggestion of inspecting the packets.

[Danack]

I'm still not sure why changing it to 256 made all the difference. None of the objects are anywhere near that size, but all the packages are showing up in the access log now. There's still some issues but they're Squid config issues.

[Marius Gedminas]

Would it be possible to autodetect a local apt proxy via Zeroconf, like squid-deb-proxy-client does?