How does reverse SSH tunneling work?

Question

As I understand this, firewalls (assuming default settings) deny all incoming traffic that has no prior corresponding outgoing traffic.

Based on Reversing an ssh connection and SSH Tunneling Made Easy, reverse SSH tunneling can be used to get around pesky firewall restrictions.

I would like to execute shell commands on a remote machine. The remote machine has its own firewall and is behind an additional firewall (router). It has an IP address like 192.168.1.126 (or something similar). I am not behind a firewall and I know the remote machine's IP address as seen from the Internet (not the 192.168.1.126 address). Additionally, I can ask someone to execute ssh (something) as root on the remote machine first.

Could anyone explain me, step by step, how reverse SSH tunneling works to get around the firewalls (local and remote machines' firewalls and the additional firewall between them)?

What is the role of the switches (-R, -f, -L, -N)?

Solution 1

I love explaining this kind of thing through visualization. :-)

Think of your SSH connections as tubes. Big tubes. Normally, you'll reach through these tubes to run a shell on a remote computer. The shell runs in a virtual terminal (tty). But you know this part already.

Think of your tunnel as a tube within a tube. You still have the big SSH connection, but the -L or -R option lets you set up a smaller tube inside it.

Every tube has a beginning and an end. The big tube, your SSH connection, started with your SSH client and ends up at the SSH server you connected to. All the smaller tubes have the same endpoints, except that the role of "start" or "end" is determined by whether you used -L or -R (respectively) to create them.

(You haven't said, but I'm going to assume that the "remote" machine you've mentioned, the one behind the firewall, can access the Internet using Network Address Translation (NAT). This is kind of important, so please correct this assumption if it is false.)

When you create a tunnel, you specify an address and port on which it will answer, and an address and port to which it will be delivered. The -L option tells the tunnel to answer on the local side of the tunnel (the host running your client). The -R option tells the tunnel to answer on the remote side (the SSH server).

So... To be able to SSH from the Internet into a machine behind a firewall, you need the machine in question to open an SSH connection to the outside world and include a -R tunnel whose "entry" point is the "remote" side of his connection.

Of the two models shown above, you want the one on the right.

From the firewalled host:

ssh -f -N -T -R22222:localhost:22 yourpublichost.example.com

This tells your client to establish a tunnel with a -Remote entry point. Anything that attaches to port 22222 on the far end of the tunnel will actually reach "localhost port 22", where "localhost" is from the perspective of the exit point of the tunnel (i.e. your ssh client).

The other options are:

• -f tells ssh to background itself after it authenticates, so you don't have to sit around running something on the remote server for the tunnel to remain alive.
• -N says that you want an SSH connection, but you don't actually want to run any remote commands. If all you're creating is a tunnel, then including this option saves resources.
• -T disables pseudo-tty allocation, which is appropriate because you're not trying to create an interactive shell.

There will be a password challenge unless you have set up DSA or RSA keys for a passwordless login.

Note that it is STRONGLY recommended that you use a throw-away account (not your own login) that you set up for just this tunnel/customer/server.

Now, from your shell on yourpublichost, establish a connection to the firewalled host through the tunnel:

ssh -p 22222 username@localhost

You'll get a host key challenge, as you've probably never hit this host before. Then you'll get a password challenge for the username account (unless you've set up keys for passwordless login).

If you're going to be accessing this host on a regular basis, you can also simplify access by adding a few lines to your ~/.ssh/config file:

host remotehostname
    User remoteusername
    Hostname localhost
    Port 22222

Adjust remotehostname and remoteusername to suit. The remoteusername field must match your username on the remote server, but remotehostname can be any hostname that suits you, it doesn't have to match anything resolvable.

See also:

• Expose the reverse endpoint on a non-localhost IP
• Tips on using ControlMaster to maintain your tunnel

Solution 2

I have drawn some sketches

The machine, where the ssh tunnel command is typed is called »your host«.

---

Introduction

1. local: -L Specifies that the given port on the local (client) host is to be forwarded to the given host and port on the remote side.

   ssh -L sourcePort:forwardToHost:onPort connectToHost means: connect with ssh to connectToHost, and forward all connection attempts to the local sourcePort to port onPort on the machine called forwardToHost, which can be reached from the connectToHost machine.

2. remote: -R Specifies that the given port on the remote (server) host is to be forwarded to the given host and port on the local side.

   ssh -R sourcePort:forwardToHost:onPort connectToHost means: connect with ssh to connectToHost, and forward all connection attempts to the remote sourcePort to port onPort on the machine called forwardToHost, which can be reached from your local machine.

Additional options

• -f tells ssh to background itself after it authenticates, so you don't have to sit around running something on the remote server for the tunnel to remain alive.
• -N says that you want an SSH connection, but you don't actually want to run any remote commands. If all you're creating is a tunnel, then including this option saves resources.
• -T disables pseudo-tty allocation, which is appropriate because you're not trying to create an interactive shell.

Your example

The third image represents this tunnel. But the blue computer called »your host« represents the computer where someone starts the ssh tunnel, in this case the firewalled machine.

So, ask someone to start a ssh tunnel connection to your machine. The command should basically look like

ssh -R 12345:localhost:22 YOURIP

Now the tunnel is opened. You can now connect via ssh to the firewalled machine through the tunnel with the command

ssh -p 12345 localhost

which will connect to your own localhost (your machine) on port 12345, but port 12345 is forwarded through the tunnel to port 22 of the localhost of the firewalled computer (i.e. the firewalled computer itself).

Solution 3

ssh tunneling works by using the already established ssh connection for sending additional traffic.

When you connect to a remote server, you usually just have 1 channel for the normal user interaction (or 3 channels if you consider STDIN/STDOUT/STDERR separate). At any time, the local or remote ssh process can open additional channels on the existing connection. These channels then send/receive the tunnel traffic. When sending or receiving any traffic, the ssh process simply says "this traffic is for channel foobar".

It essentially works like this:

1. You tell ssh to start listening on port XXXX and that any traffic received should be tunneled, and then set to Y.Y.Y.Y on port ZZZZ.
2. The local ssh starts listening on port XXXX (generally on 127.0.0.1, but can be changed).
3. Some application opens a connection to port XXXX on the local machine.
4. The local ssh opens a channel to the remote ssh and says "any traffic on this channel goes to Y.Y.Y.Y:ZZZZ
5. The remote ssh connects to Y.Y.Y.Y:ZZZZ and sends back the "OK, channel is open"
6. Now any traffic sent along the connection to port XXXX on the local machine is proxied by ssh to Y.Y.Y.Y:ZZZZ.

This process is the exact same for both forward and reverse tunneling (just swap the words 'local' and 'remote' in the above procedure). Either side can start the tunnel. It doesn't even have to be when you first start ssh. You can open tunnels while ssh is already running (see ESCAPE CHARACTERS, specifically ~C).

For the role of -R, -f, -L, and -N, you really should just consult the man page, it gives you the best possible explanation. But I'll mention -R and -L.
-R tells the remote ssh to listen for connections, and that the local ssh should connect to the real destination. -L tells the local ssh to listen for connections, and that remote ssh should connect to the real destination.

Note, this is a very crude description, but it should give you enough info to know what's going on

Solution 4

This is explained in SSH manual, especially the differences between -L (local) and -R (remote).

---

-L

-L [bind_address:]port:host:hostport

Specifies that the given port on the local (client) host is to be forwarded to the given host and port on the remote side.

This works by allocating a socket to listen to port on the local side, optionally bound to the specified bind_address.

Whenever a connection is made to this port, the connection is forwarded over the secure channel, and a connection is made to host port hostport from the remote machine.

The following example tunnels an IRC session from client machine 127.0.0.1 (localhost) using port 1234 to remote server server.example.com:

$ ssh -f -L 1234:localhost:6667 server.example.com sleep 10

^{Note: The -f option backgrounds ssh and the remote command sleep 10 is specified to allow an amount of time to start the service which is to be tunnelled.}

Example:

ssh `-N` -L 22000:localhost:11000 remote.server.com

• -N After you connect just hang there (you won't get a shell prompt)

  Do not execute a remote command.

• -L 22000 The connection will originate on port 22000 of your personal, Local machine

• localhost:11000 - remote.server.com will make sure that the other end of the tunnel is localhost, port 11000

^{Source: An illustrated guide, tutorial, how-to, on ssh tunneling.}

---

-R

-R [bind_address:]port:host:hostport

Specifies that the given port on the remote (server) host is to be forwarded to the given host and port on the local side.

This works by allocating a socket to listen to port on the remote side, and whenever a connection is made to this port, the connection is forwarded over the secure channel, and a connection is made to host port hostport from the local machine.

Example:

ssh -N -R 22000:localhost:11000 remote.server.com

• -N After you connect just hang there (you won't get a shell prompt)

  Do not execute a remote command.

• -R 22000 The connection will originate on port 22000 of the Remote computer (in this case, remote.server.com)

• localhost:11000 your personal, local computer will make sure that the other end of the tunnel is localhost, port 11000

^{Source: An illustrated guide, tutorial, how-to, on ssh tunneling.}

View more solutions

Comments

[Gilles 'SO- stop being evil']

Related (how-to): How can I configure a reverse SSH connection to the connecting computer?

[BigSack]

what about ssh -D . Please explain using same method.

[Pacerier]

@erik, How did you draw these images?

[erik]

I used the open source vector drawing tool called Inkscape and my mice (well, actually it is a trackball: Marble FX from Logitech).

[Nathan Basanese]

// , In this explanation, what does YOURIP in ssh -R 12345:localhost:22 YOURIP mean? Does YOURIP mean the IP address of the firewalled server, or does YOURIP mean the IP address of the host trying to connect to the firewalled server?

[erik]

@basanese: YOURIP is the address of the host trying to connect to the firewalled server. I called it YOURIP because I was referring to the computer the questioner called his computer.

[ghoti]

Hey, Erik, I was just about to start recycling your images for a presentation I'm preparing, but ... something about your -R option seems wrong. [-R [bind_address:]port:host:hostport] is what's on the openssh man page. To create what's in your drawing, wouldn't you need -R remotehost:456:nearhost:123 ? In fact, your -L options look as if they're reversed as well (man page says -L [bind_address:]port:host:hostport). Am I missing something?

[ghoti]

@erik, ah, the confusing part is because my eyes are bad -- I wasn't seeing the space before remotehost in your commands. Upon closer inspection, I see that your diagrams don't actually address the optional bind_address at all. Sorry for the confusion.

[Rucent88]

I'm having a very difficult time following this explanation due to the loose usage of terms: server, client, local machine, remote machine, host, yourpublichost, localhost, remotehostname. With all these loose and undefined terms, it could be assumed that someone would need as much as 8 computers to set this up. I state this because in all other aspects it seems a very good explanation. Please reduce and define terms.

[ghoti]

@Rucent88, I'm not sure what reduction would be possible. A client establishes a connection to a server. That's common terminology all across the world of networking. Local and remote machines seem pretty self evident. If you're confused about SSH or general unix terminology after reading the documentation, I'm sure you'll have no trouble finding people here very willing to answer any questions you may have.

[marcotama]

How does the -p parameter come into play in your diagrams? Say I run ssh -R 12345:localhost:22 YOURIP -p 6789.. what does it change?

[erik]

@tamzord The -p option is the port where your ssh connects to the remotehost. By default ssh uses port 22 (changeable in the server’s /etc/ssh/sshd_config). Then, following your example ssh -p 6789 -R 12345:localhost:22 remotehost, all connections made to remotehost to port 12345 are redirected through the tunnel to yourhost to port 22. If at yourhost you have the default sshd-configuration, someone could connect with ssh to remotehost to port 12345, which would turn out in a ssh-connection to your host in reality (through your established and active tunnel).

[briankip]

Hi does this mean that I need sshd on the device behind the firewall?

[erik]

@briankip Yes, of course. At least if you want to make a ssh connection to the computer behind the firewall (through the tunnel or directly, if the firewall permits that). I.e.: The ssh-command you write on the firewalled computer only opens a tunnel, but does not start or work as a ssh-daemon/-server. Your next question should be, how to start a sshd without root privileges (if that is your problem). :-)

[briankip]

I thought that the tunnel would somehow deliver my tty to the other side. Now the idea is sinking in. I guess the other thing made me even more confused was the -D option.

[CivFan]

So could you open another reverse tunnel through the first tunnel connection, back to remote host? Y'know, for reasons.

[erik]

@CivFan Yes you can. No problem. You can do this thousands of times, back and forth. But you need different port numbers every time, of course.

[Dionysian]

@ghoti It's indeed confusing, because local and remote machines are relative. When I sit at the workplace my home computer is the remote, when I sit at home, the workplace computer is remote. Server and client is also confusing when speaking of tunneling. For example If I connect to home from work with ssh -R. I connect to my work from the home computer by connecting localhost. So from sockets perspective the server is home computer itself. But logically I connected to my work computer. That's confusing.

[Dionysian]

In the -L and -R options which host performs the name resolution? For example if I connect from home to the company gateway via SSH. From that gateway I can access the company network. For example if I use host.internalcompanydomain in the -L option from the home computer, which computer will resolve it?

[ghoti]

@Calmarius, a client initiates a connection, a server receives that connection. A tunnel is a product of an SSH connection. Make a tunnel with -R, and you provide something through which some other client (on the remote side from the perspective of the SSH client) might establish a connection to a server that is "local" to the SSH client. You're right that things are relative; terminology is necessarily dependent on perspective and scope. It becomes easier the more you work with it.

[paulhauner]

I just love these diagrams. They're as equally 'lol' as they are informative. Great work.

[Daniel F]

This answer is absolutely great, but it's even possible to bind that port to a specific interface, ie ssh -R 0.0.0.0:12345:localhost:22 YOURIP to make that port publicly accessible on the remote server. It would be great if this answer also mentiones that fact. See superuser.com/a/591963/100853

[yukashima huksay]

I'm getting ssh_exchange_identification: Connection closed by remote host on one side

[Thufir]

the ear and mouth make it crystal clear who's talking and who's listening!

[cjaphe]

Usage of localhost and username@localhost is ambiguous. It's unclear if localhost is always meant literally or not, and if not, then what exactly it means. It's also unclear what username means.

[yildizabdullah]

What should I do if both machines are behind NAT?

[mcantsin]

I love your illustrations!

[ghoti]

@yildizabdullah, you'll need a common location external to both NATted sites for one site to establish a public place for the other to access, in order to connect two tunnels. But your best bet is probably to get your network admins on-side, and have them punch a hole for you in one firewall or the other.

[GP Singh]

Hi @Erik, my remotehost does not have internet connection. So as per your 4th image I rely on nearhost for internet access. I get 403 forbidden error when yum is executed. Where can I mention the credentials of nearhost server?

[girorme]

Wow, nice sketches! Helped me a lot! Thanks!

[John Smith Optional]

Awesome. Thank you!

[Jorge Cornejo Bellido]

For what I understand, it is possible to forward a port to another local LAN PC instead of localhost only? So remote server running ssh server, can connect not just to the PC running the SSH client, but also another PC in the network?

[Mykola]

Diagrams from this answer are my favourite; after examining them I started to understand the port forwarding data flow.

[dbernard]

This should be the accepted answer. :) The drawings are simply amazing!

[Franz Wong]

@erik I read some pages saying if remotehost is behind a firewall which doesn't accept any incoming connection, then we need to use reverse ssh tunnel. My question is if it is the case, then how can we create a ssh connection at the very beginning?

[erik]

@franziga The ssh-tunnel must be created from inside the firewall to the outside. Then from outside you can again ssh through this tunnel inside the firewalled network/computer. That is the very reason for a tunnel.

[HamedH]

if I start ssh with -f how can I easily stop that tunnel later?

[ghoti]

@HamedH .. There's some documentation on ssh session (and tunnel) control here.

[Lucas Aimaretto]

I liked your explanation. Only doubt though, on point 6: Y.Y.Y.Y receives traffic on port ZZZZ. In other words, dest-IP is Y.Y.Y.Y and dest-port is ``ZZZZ. Which are the source IP address and source port of the receiving packet?

[masonCherry]

nice explanation one thing I don't understand is why you need a reverse tunnel in the first place why can't the remothost just ssh directly to the blue computer.

[erik]

@MrBrN197 Blue computer is often behind a firewall, behinda NAT, behind an unknown (or often changing) IP, … Example: My friend (not a computer expert) is in Europe with his computer. I want to access his computer, to help him with a problem. He connects to my computer with ssh opening a tunnel (I tell him the IP of my computer). I then connect to his computer (I don’t know his IP and he is behind a company’s firewall) via ssh and help fix the problem.

[cregox]

i've got all id_rsa in place, and regular passwordless ssh works fine. but when connecting through the localhost tunnel, it always ask for a password. any idea what might be happening?

[information_interchange]

"which can be reached from the connectToHost machine" I'm trying to understand this part better for local SSH tunnelling. From my understanding, this would mean that whatever connectToHost is mapped to on the local machine, the traffic would get routed there. Are there other usecases?

[lashgar]

"The -L option tells the tunnel to answer on the local side of the tunnel (the host running your client). The -R option tells the tunnel to answer on the remote side (the SSH server)." That's all needed for answer.

[John Jiang]

Really helpful drawings. May I suggesting adding some dotted lines between the machines that can be connected directly?

[InvisibleWolf]

Thank you for this!

[parttimeturtle]

This rocks. Love the whimsical children's book vibes of the drawings.