How --set-mark option works on Netfilter (IPTABLES)?

Question

On Netfilter, you have the option --set-mark for packets that pass through the mangle table.

The majority of tutorials and examples over the Internet, say that this just adds a mark on the packet, like this, but there's no additional detail of what mark is set and where it resides on the packet:

iptables -A PREROUTING -t mangle -i eth0 -p tcp --dport 80 -j MARK --set-mark 1

My question is:

• What kind of mark is set and exactly where in the packet this mark resides?

Solution

The mark is a 32 bits integer value attached to a network packet. Some network parts interacting with it (see below) can do bitwise operations on this value, it can then be interpreted between one single 32 bits value up to a collection of 32 flags, or a mix of flags and smaller values, depending on how one chooses to organise its use (tc can't do this). Of course this mark exists only as long as it's handled by the Linux kernel. It's only purely virtual and internal, as it can have no existence on the wire. Depending on where's it's used, it may be called firewall mark, fwmark or simply mark.

Each network packet processed by the kernel, is handled by a structure called sk_buff, defined in linux/include/linux/skbuff.h. This structure includes various meta-data related to the packet when applicable, like IPsec information if any, related conntrack entry once looked up, ... and also its mark.

Various parts of the network stack can read this mark, change behaviour based on its value or (re)write it, eg:

• tc,
• the routing stack can have special rules set with ip rule (eg ip rule add fwmark 1 lookup 42), to alter its routing decisions with this fwmark (eg to use a routing table sending those packets to an other interface than default),
• of course iptables,
• its candidate successor nftables,

and a few other places...

The main goal of this mark is to have all these network parts interact with each other by using it as a kind of message. The Packet flow in Netfilter and General Networking can help see in what order those elements will receive handling of the packet and thus its mark.

There are other related marks beside fwmark:

• connmark, which isn't stored with a packet's sk_buff, but in a conntrack entry tracking packet flows. Its connmark can of course be used by iptables with its connmark match and CONNMARK target, with an usage example there: Netfilter Connmark To Linux and beyond !. It allows the decision made based on one single packet to be memorized and then applied to all the packets of the same connection.
• secmark and likewise its associated connsecmark which are intended to interact with Linux Security Modules such as SELinux.

Comments

[nelaaro]

Thank you for your wonderful explanation and important documentation of netfilter and its limitations this mark exists only as long as it's handled by the Linux kernel First place I found information about this limitation