How do I forward/NAT all traffic to one interface/IP to a remote IP?

Question

I have one "server A" that has multiple IPs attached to it, like so:

eth0:0 1.1.1.1
eth0:1 1.1.1.2
eth0:2 1.1.1.3

I have another "server B" that also has multiple IPs attached to it, like so:

eth0:0 2.2.2.1
eth0:1 2.2.2.2
eth0:2 2.2.2.3

Now, I want to setup iptables on "Server A" to forward/NAT all incomming traffic on "eth0:2" to IP 2.2.2.3 on "Server B".

I have verified that "Server A" is able to "talk" to "Server B" on IP 2.2.2.3. Ping and telnet to open ports works just fine and I have the forward-flag turned on (net.ipv4.ip_forward=1)

I have tried multiple different ways, DNAT, SNAT, MASQUERADE etc, but I cannot get anything to work.

This line works fine if I try to forward traffic between IPs on the SAME server:

iptables -t nat -A PREROUTING -d 1.1.1.3 -j DNAT --to-destination 1.1.1.2

But if I switch out the "1.1.1.2" for "2.2.2.3", it does not work.

I assume that I need a second iptable rule to solve it. I have tried with the following POSTROUTING rules without any luck (not at the same time):

iptables -t nat -A POSTROUTING -d 2.2.2.3 -j MASQUERADE
iptables -t nat -A POSTROUTING -d 2.2.2.3 -j SNAT --to 1.1.1.3
iptables -t nat -A POSTROUTING -j MASQUERADE

What am I missing?

EDIT 1:

I finally got it to work by using the following:

net.bridge.bridge-nf-call-iptables=0

iptables -t nat -A PREROUTING -d 1.1.1.3 -j DNAT --to-destination 2.2.2.3
iptables -t nat -A POSTROUTING -d 2.2.2.3 -j SNAT --to 1.1.1.3

However, now another problem appeared. All logs etc on server 2.2.2.3 shows that ALL traffic now comes from 1.1.1.3, like apache logs, mail logs etc. I assume this is the nature of NAT.

However, when I do standard port forwarding on my home-router to my laptop that is running apache, I see the original "requester IP" in the logs. So, how does the router do this? And how can I do the same on my server setup?

Bottom line, I want to forward all traffic from Server A (1.1.1.3) to Server B (2.2.2.3), BUT I also want to be able to see where the traffic came from on Server B (2.2.2.3), i.e the apache logs should show the original IP of the requester.

I assume I should use some other way than NAT to make this happen, and it should be possible, as even my simple home-router is able to do this.

One extra thing, the IPs attached to Server A and Server B is LOCKED to each respective server. Thus, Server A is NOT able to send out traffic FROM IP 2.2.2.3. It is locked by my provider in the router.

Solution

Short answer to your revised question is that there are two ways to do it; both require you remove the second NAT step (which destroys the info you're looking for). Your options after doing that are:

1) Make Server A the next hop for Server B for the traffic in question, which is why it works for your router as mentioned. This could be accomplished, in order of cludgeyness, by making server A the default route for Server B, or using policy routing, or using some fancy iptables, or using a tunnel of some sort.

2) "Manually" reversing server A's NAT on server B, resulting in asymmetric traffic flow (generally discouraged). Something like iptables -t nat -I POSTROUTING -j SNAT -s 2.2.2.3 --to 1.1.1.3

I am 100% confident in option (1). I'm about 90% confident in (2).

To understand this, you need to understand the traffic flow.

1. Client X sends a packet to 1.1.1.3.
2. Server A NATs the destination of that packet to 2.2.2.3 pre-routing, then routes the traffic to 2.2.2.3, then NATs the source of that packet to 1.1.1.3 post-routing, then sends the packet to Server B.
3. Server B receives the packet on 2.2.2.3 and sees a source address of 1.1.1.3, per the second NAT step. It processes the packet and sends the reply back to its source (1.1.1.3).
4. Server A receives the packet on 1.1.1.3, reverses the source NAT, routes the packet, reverses the destination NAT, and sends the packet back to client X.
5. Client X receives the response from 1.1.1.3

Now let's imagine what would happen if you didn't have the second NAT:

1. Client X sends a packet to 1.1.1.3.
2. Server A NATs the destination of that packet to 2.2.2.3 pre-routing, then routes the traffic to 2.2.2.3, but leaves the source address as X when it sends the packet to Server B.
3. Server B receives the packet on 2.2.2.3 and sees a source address of X. It processes the packet and sends the reply back to its source X.
4. Client X receives the response from 2.2.2.3 and discards it because it doesn't know 2.2.2.3 from Adam!

In order for Client X to understand the packet, it needs to arrive at Client X with the same source address as was the destination of the original packet.

The normal way for this to happen is for Server B to have an opportunity to reverse the pre-routing NAT. For that to happen you need the packet to go back through it at a later date. Currently you do that by changing the source address of the packet, but that destroys the information you're asking for in your revised question.

So the first step of your answer is, you can't do the second NAT step (post-routing SNAT): on server A run iptables -t nat -D POSTROUTING -j SNAT --to 1.1.1.3.

Now you're left with the challenge of reversing the first NAT step.

If Server B is going to do it, you need Server B to receive the packets.

• This is relatively easy if server A has an address C on the same LAN as server B. On server B: ip route replace default via C, OR ip route add default via C table a; ip rule add from 2.2.2.3 table a.
• Otherwise you have to do something fancy with tunnels.

If, however, the router at Server B's location is not particularly sophisticated (statefully inspecting packets and rejecting ones that are not in correct sequence for a known traffic flow), you have a somewhat simpler, if super-ugly, option: reverse the NAT at server B based on your knowledge of what was done at server A: on server B iptables -t nat -I POSTROUTING -j SNAT -s 2.2.2.3 --to 1.1.1.3 should do it the proposed example. This will leave the Linux connection tracking system at A and B somewhat befuddled (servers will not be able to associate return traffic with incoming traffic so their connection tracking will leave connections in UNREPLIED state) but it should work fine for most traffic into the hundreds of megabits.

Walking through the traffic flow one last time in this case:

1. Client X sends a packet to 1.1.1.3.
2. Server A NATs the destination of that packet to 2.2.2.3 pre-routing, then routes the traffic to 2.2.2.3, but leaves the source address as X when it sends the packet to Server B.
3. Server B receives the packet on 2.2.2.3 and sees a source address of X. It processes the packet and routes the reply back to its source X, but before sending it out does the post-routing NAT of the source adderess to 1.1.1.3.
4. Client X receives the response claiming to be from 1.1.1.3 and is happy.

Comments

[Falcon Momot]

Can you ping 2.2.2.3 with a source address of 1.1.1.3? This looks like a connectivity issue; your rules are right. Also, it's better to use example subnets when redacting addresses: 192.0.2.0/24, 198.51.100.0/24, and 203.0.113.0/24.

[Daniele Testa]

I have updated my question. Could you please check it again?

[Zatarra]

net.bridge.bridge-nf-call-iptables=0 - this saved me after 4 days of searches and tries. Thanks a lot!

[Daniele Testa]

As I mentioned in my post, I already have ip_forward=1. My FORWARD chain has a default-rule set to ACCEPT.

[ALex_hha]

As far as I know, if the packets was generated by the host himself - there is no need to enable ip forwarding at all. And you will need to use OUTPUT chain instead of PREROUTING

[Daniele Testa]

I have updated my question. Could you please check it again?

[ALex_hha]

I think it is not possible, because you are using SNAT and there is no way to get original source ip on the 2.2.2.3 without additional software. On your home router you have used only DNAT, that's why you saw original ip. You could try to use reverse proxy for example nginx to still able get original ip address. What type of application is running on 2.2.2.3?

[Daniele Testa]

Why can I use only DNAT on my home router, but have to use SNAT in the server setup? I don't really see why the setup would differ. I cannot use reserve proxy, the server should forward ALL traffic, not only certain ports. The traffic is forwarded from Server A to a VPS running on Server B

[ALex_hha]

As I understand, on home router you have used only port forwarding (DNAT), this is not the same as the problem you have described. It would be tcp or udp?