Linux Netfilter: How does connection tracking track connections changed by NAT?

Question

At the moment I am diving into the details of the Netfilter Architecture of Linux. I am already familiar with Netfilter hooks, tables, chains, the different kernel modules, etc.
But there is one detail of NAT in combination with connection tracking that I don't understand.

I try to explain my problem with a little SNAT example. In this example I will ignore the transport layer because I think it is not necessary to understand my problem. Please correct me if I am wrong!

There is a client computer on the local network with the ip address 192.168.2.55 and a NAT-gateway with the external ip address 193.157.56.3. Now the Client wants to communicate with a server and the ip adress 217.254.1.76 on the Internet.
So the client sends a packet with src = 192.168.2.55/dst = 217.254.1.76 to the NAT-gateway which is also the default gateway. Connection tracking tracks this new connection and creates two new tuples:

IP_CT_DIR_ORIGINAL: src = 192.168.2.55, dst = 217.254.1.76
IP_CT_DIR_REPLY: src = 217.254.1.76, dst = 192.168.2.55

IP_CT_DIR_ORIGINAL and IP_CT_DIR_REPLY are macros to access an array of two tuples.

At the POSTROUTING hook NAT asks connection tracking for a existing connection and, if this is successful, changes the source address in the header of the packet. It also silently creates a DNAT rule to revert the destination address of replies.
Now I get to the point of my poblem. NAT changes the destination address in IP_CT_DIR_REPLY to its external ip address 193.157.56.3. So the tuples look like this:

IP_CT_DIR_ORIGINAL: src = 192.168.2.55, dst = 217.254.1.76
IP_CT_DIR_REPLY: src = 217.254.1.76, dst = 193.157.56.3

That is why connection tracking can track the replies in the PREROUTING hook because there is an existing tuple for the reply. But after tracking the packet NAT changes the destination address to the address of the client, 192.168.2.55.
Now my question: How can connection tracking track this packet in the POSTROUTING hook? There is no reply tuple with src = 217.254.1.76/dst = 192.168.2.55 because NAT has changed it.
Is there something that I have missed?

Solution

You should install the conntrack command usually packaged as conntrack or conntrack-tools, from http://conntrack-tools.netfilter.org/ . It will mostly display the same content as /proc/net/nf_conntrack but can do more.

Run conntrack in event mode on the NAT gateway: conntrack -E (or you can choose conntrack -E --proto tcp --orig-port-dst 443 to limit to HTTPS). Now your previous example with an HTTPS request would give something similar to this:

    [NEW] tcp      6 120 SYN_SENT src=192.168.2.55 dst=217.254.1.76 sport=50798 dport=443 [UNREPLIED] src=217.254.1.76 dst=193.157.56.3 sport=443 dport=50798
 [UPDATE] tcp      6 60 SYN_RECV src=192.168.2.55 dst=217.254.1.76 sport=50798 dport=443 src=217.254.1.76 dst=193.157.56.3 sport=443 dport=50798
 [UPDATE] tcp      6 432000 ESTABLISHED src=192.168.2.55 dst=217.254.1.76 sport=50798 dport=443 src=217.254.1.76 dst=193.157.56.3 sport=443 dport=50798 [ASSURED]
 [UPDATE] tcp      6 120 FIN_WAIT src=192.168.2.55 dst=217.254.1.76 sport=50798 dport=443 src=217.254.1.76 dst=193.157.56.3 sport=443 dport=50798 [ASSURED]
 [UPDATE] tcp      6 60 CLOSE_WAIT src=192.168.2.55 dst=217.254.1.76 sport=50798 dport=443 src=217.254.1.76 dst=193.157.56.3 sport=443 dport=50798 [ASSURED]
 [UPDATE] tcp      6 30 LAST_ACK src=192.168.2.55 dst=217.254.1.76 sport=50798 dport=443 src=217.254.1.76 dst=193.157.56.3 sport=443 dport=50798 [ASSURED]
 [UPDATE] tcp      6 120 TIME_WAIT src=192.168.2.55 dst=217.254.1.76 sport=50798 dport=443 src=217.254.1.76 dst=193.157.56.3 sport=443 dport=50798 [ASSURED]
[DESTROY] tcp      6 src=192.168.2.55 dst=217.254.1.76 sport=50798 dport=443 src=217.254.1.76 dst=193.157.56.3 sport=443 dport=50798 [ASSURED]

The nat table is special, in that it is used only once per flow*, when the [NEW] state is created. Everything else is short-circuited by the conntrack entry found. The SNAT rule in POSTROUTING didn't "silently create a DNAT rule" for the reply. It altered the conntrack entry to have reply dst=193.157.56.3, and told netfilter "I changed something", that's most of it. Everything else (including the source ip alteration) is handled by conntrack (modules nf_conntrack, nf_conntrack_ipv4) and nat (modules nf_nat, nf_nat_ipv4 and maybe a few more here), not by iptables. Consider the entries are a connections state database.

When a reply packet is received, the packet, having no association stored, is looked through the conntrack entries (actually looking an association both ways, either in the original part, or in the reply part, this doesn't matter), and a match is found because the entry was created before. Then the packed information is updated with this connection association. There's no need to look up this packet though the entries anymore later, even if some of its properties (source or destination...) are changed the information is directly available. Some of the macro/inline functions handling this are defined in skbuff.h . Look for _nfct and nfct. The packet (aka the skbuff), after the lookup, receives this value in skb->_nfct.

So the few things you might have missed:

• iptables is not netfilter. It's a user of netfilter. nftables is an other user of netfilter. conntrack and nat (eg module nf_nat) are part of netfilter.
• the POSTROUTING hook will never see the reply packet because the connection is not NEW: the nat table is not called anymore for this flow and the packets identified as part of this flow.
• most of the handling of conntrack and nat is done... by conntrack and nat, not by iptables. iptables can use the ressources of conntrack (eg: -m conntrack --ctstate ESTABLISHED) or nat (anything in the nat table has to). For the example above, The conntrack entry alone has the information to "de-SNAT" the packet, and indeed it looks similar to a DNAT with a connection originally inbound.
• There's usually no need to look up the conntrack entries more than once for a packet part of an existing connection, the "index" is attached to the packet after the lookup.

You can be convinced that the nat table doesn't see other packets after the first by running a few times iptables-save -c and looking at how the counters increment for the rules in the nat table: only for the first packet.

*look at the NAT part:

This table is slightly different from the `filter' table, in that only the first packet of a new connection will traverse the table: the result of this traversal is then applied to all future packets in the same connection.

Comments

[idlmn89]

Thank you very much for your good explanation. That makes it all very clear.