Tunneling apt-get Through SSH Works... Mostly

Question

I am running a Linux container under Proxmox 4.4. The container is based on Debian Jessie. Here are the version specifics:

root@myHostName:/home/myUserName# uname -a
Linux myHostName 4.4.35-1-pve #1 SMP Fri Dec 9 11:09:55 CET 2016 x86_64 GNU/Linux
root@myHostName:/home/myUserName# cat /etc/debian_version
8.6
root@myHostName:/home/myUserName# cat /etc/issue
Debian GNU/Linux 8 \n \l

This container is isolated. Its only way out is via an SSH tunnel. And so, I'm using an SSH tunnel for package management (apt-get, etc.). This seems to mostly be working, but I do get some occasional warning / errors.

Here is the original /etc/apt/sources.list:

deb http://ftp.debian.org/debian jessie main contrib
deb http://ftp.debian.org/debian jessie-updates main contrib
deb http://security.debian.org jessie/updates main contrib

Here is what the updated /etc/apt/sources.list:

deb http://localhost:39860/debian jessie main contrib
deb http://localhost:39860/debian jessie-updates main contrib
deb http://localhost:39861 jessie/updates main contrib

Here's how I create the SSH tunnel:

ssh -N -L39860:ftp.debian.org:80 -L39861:security.debian.org:80 myRemoteUserName@myRemoteHostname

Here is what happens when I execute apt-get update:

root@myHostName:/home/myUserName# apt-get update
Ign http://localhost:39860 jessie InRelease
Hit http://localhost:39860 jessie-updates InRelease
Ign http://localhost:39861 jessie/updates InRelease
Hit http://localhost:39860 jessie Release.gpg
Get:1 http://localhost:39860 jessie-updates/main amd64 Packages/DiffIndex [8392 B]
Hit http://localhost:39860 jessie-updates/contrib amd64 Packages
Hit http://localhost:39860 jessie-updates/contrib Translation-en
Get:2 http://localhost:39860 jessie-updates/main Translation-en/DiffIndex [3196 B]
Ign http://localhost:39861 jessie/updates Release.gpg
Hit http://localhost:39860 jessie Release
Ign http://localhost:39861 jessie/updates Release
Hit http://localhost:39860 jessie/main amd64 Packages
Hit http://localhost:39860 jessie/contrib amd64 Packages
Hit http://localhost:39860 jessie/contrib Translation-en
Hit http://localhost:39860 jessie/main Translation-en
Err http://localhost:39861 jessie/updates/main amd64 Packages
  404  Not Found [IP: ::1 39861]
Err http://localhost:39861 jessie/updates/contrib amd64 Packages
  404  Not Found [IP: ::1 39861]
Ign http://localhost:39861 jessie/updates/contrib Translation-en
Ign http://localhost:39861 jessie/updates/main Translation-en
Fetched 11.6 kB in 9s (1279 B/s)
W: Failed to fetch http://localhost:39861/dists/jessie/updates/main/binary-amd64/Packages  404  Not Found [IP: ::1 39861]

W: Failed to fetch http://localhost:39861/dists/jessie/updates/contrib/binary-amd64/Packages  404  Not Found [IP: ::1 39861]

E: Some index files failed to download. They have been ignored, or old ones used instead.

How can I best get around the warnings and errors shown at the end?

Solution

Your approach doesn't work because of the HTTP protocol's Host: header.

Under normal circumstances, the HTTP client (in this case, apt-get) sends the header Host: ftp.debian.org, which lets the server know which website to load, in case the server is hosting multiple domains.

What you have done is make apt-get send Host: localhost:39860, which likely ends up not being anything on the remote server. That explains your HTTP 404 Not Found errors.

---

Proposed Resolution

Put an HTTP proxy on your host and make it available to your container via SSH port forwarding.

1. Install Squid on your Internet-connected machine. Ubuntu/Debian: sudo apt install squid

2. Write this configuration into /etc/squid/squid.conf:

   http_port 8080
   
   acl container src XXX.XXX.XXX.XXX/32
   http_access allow container
   

   Replace XXX.XXX.XXX.XXX with your container's IP address.

3. Start or restart Squid with service squid restart.

4. Connect to your container from your Internet-connected machine with this command:

   ssh -R8080:localhost:8080 myRemoteUserName@myRemoteHostname
   

   Replace myRemoteUserName@myRemoteHostname with the login credentials to your container.

5. Inside the container, create a new file called /etc/apt/apt.conf.d/80proxy with the following contents:

   Acquire::http::Proxy "http://localhost:8080/";
   

6. As long as the connection from step #4 is up, your container's apt package manager can now make remote requests.

If you need to add site restrictions to your proxy, check out Restrict Squid access to only one site.

Comments

[Dave]

Yes, the host is Proxmox (version 4.4). Please note that the Proxmox host is isolated in the exact same manner as the Linux containers it encapsulates. The Proxmox host has outbound access only via an SSH tunnel to the machine I referred to as myRemoteHostname. I do have the ability to install squid on the Proxmox host, but I don't have the ability to install it on the Internet-connected machine through which I am tunneling (i.e. myRemoteHostname). What ramifications does this have on your proposed resolution?

[Deltik]

@Dave: In that case, put Squid on the Internet-connected machine rather than the host. I'll update my answer to reflect this new information.

[Dave]

I do not have the ability to put squid on the Internet-connected machine. I am not the administrator of that machine, and policy prevents me from doing so. :(

[Deltik]

@Dave: Sounds like you need to review requirements with your organization. It sounds like you wouldn't be allowed to open a SOCKS5 proxy (ssh -D) from the container to your Internet machine. Another possibility: Create a Squid proxy on another Internet-connected machine (other-internet-machine) where you are allowed to do so and forward traffic with ssh -L8080:other-internet-machine:8080 myRemoteUserName@myRemoteHostname.

[Dave]

I do actually have the ability to open a SOCKS5 proxy via ssh -D. In that case, I tried the following entries in the container's /etc/apt/sources.list: deb socks://localhost:39860/debian jessie main contrib deb socks://localhost:23990/debian jessie-updates main contrib deb socks://localhost:23990 jessie/updates main contrib This was some time ago, so I don't clearly remember the outcome, but I believe it was something to the effect of "socks" being an unknown scheme. Perhaps I need to install a socks package on the containers???

[Deltik]

@Dave: You would have to upload tsocks to your container, configure /etc/tsocks.conf to use the SSH SOCKS5 proxy, and then run sudo tsocks apt-get update.

[Omid]

This works. But when creating ssh tunnel from Internet-connected machine to container I encountered address already in use error so I had to create the tunnel from container to Internet-connected machine which works perfectly

[Deltik]

@Omid: I had a typo in my answer. -L should have been -R to forward the port from the Internet machine to the container. This is now corrected, so you can now use the updated command to connect from the Internet machine to the container.

[Omid]

@Deltik Thanks! It worked! Great answer!