tc don't see marked with -j MARK packets

Question

Server has 2 network interfaces:

• eth1 with address 13.0.0.254/24
• eth0 with address 172.20.203.4/24.

It's routing traffic between this two networks. Task is to limit bandwidth between this two networks to 1Vbit/sec, but not to limit bandwidth between server and network hosts(i. e. limit all packets going though FORWARD)

iptables -t mangle -A POSTROUTING -s 13.0.0.0/24 -d 172.20.203.0/24 -j MARK --set-mark 0x0001
iptables -t mangle -A POSTROUTING -s 172.20.203.0/24 -d 13.0.0.0/24 -j MARK --set-mark 0x0002

# eth1
tc qdisc add dev eth1 root handle 1:0 htb default 2

tc class add dev eth1 parent 1:0 classid 1:1 htb rate 1000mbps ceil 1000mbps
tc class add dev eth1 parent 1:1 classid 1:2 htb rate 999mbps ceil 1000mbps
tc class add dev eth1 parent 1:1 classid 1:3 htb rate 1mbps

tc qdisc add dev eth1 parent 1:2 handle 2:0 sfq perturb 10
tc qdisc add dev eth1 parent 1:3 handle 3:0 sfq perturb 10

tc filter add dev eth1 parent 1:0 handle 1 fw flowid 1:3
tc filter add dev eth1 parent 1:0 handle 2 fw flowid 1:3

# eth0
tc qdisc add dev eth0 root handle 1:0 htb default 2

tc class add dev eth0 parent 1:0 classid 1:1 htb rate 1000mbps ceil 1000mbps
tc class add dev eth0 parent 1:1 classid 1:2 htb rate 999mbps ceil 1000mbps
tc class add dev eth0 parent 1:1 classid 1:3 htb rate 1mbps

tc qdisc add dev eth0 parent 1:2 handle 2:0 sfq perturb 10
tc qdisc add dev eth0 parent 1:3 handle 3:0 sfq perturb 10

tc filter add dev eth0 parent 1:0 handle 2 fw flowid 1:3
tc filter add dev eth0 parent 1:0 handle 1 fw flowid 1:3

This doesn't work. If I use this at the beginning:

tc qdisc add dev eth1 root handle 1:0 htb default 3
tc qdisc add dev eth0 root handle 1:0 htb default 3

it works. So problem is in filter settings.

iptables -L -v -n -t mangle

shows, that packets are going though MARK rules. I tried to mark packets not in POSTROUTING, but in FORWARD or PREROUTING - this does not work too. What am I doing wrong?

Here is some diagnostics:

# tc -s -d -r filter show dev eth0
filter parent 1: protocol [768] pref 49151 fw
filter parent 1: protocol [768] pref 49151 fw handle 0x1 classid 1:3
filter parent 1: protocol [768] pref 49152 fw
filter parent 1: protocol [768] pref 49152 fw handle 0x2 classid 1:3
# tc -s -d -r filter show dev eth1
filter parent 1: protocol [768] pref 49151 fw
filter parent 1: protocol [768] pref 49151 fw handle 0x2 classid 1:3
filter parent 1: protocol [768] pref 49152 fw
filter parent 1: protocol [768] pref 49152 fw handle 0x1 classid 1:3

Kernel config:

/boot # uname -a
Linux armada-sc-02 2.6.32-5-amd64 #1 SMP Sun May 6 04:00:17 UTC 2012 x86_64 GNU/Linux
/boot # grep CONFIG_IP_MULTIPLE_TABLES config-2.6.32-5-amd64
CONFIG_IP_MULTIPLE_TABLES=y
/boot # grep CONFIG_IP_ADVANCED_ROUTER config-2.6.32-5-amd64
CONFIG_IP_ADVANCED_ROUTER=y
/boot # grep CONFIG_IP_ROUTE_FWMARK config-2.6.32-5-amd64

Solution

The problem you're experiencing with the MARK IPTABLES target not working as expected was caused by a missing kernel module which enables that specific Netfilter functionality. In order to use the MARK target, you need to load the XT_MARK module which must be compiled with the Linux kernel.

Check your kernel config for CONFIG_NETFILTER_... items and ensure that ...XT_MARK and its prerequisites are compiled. If the XT_MARK item was compiled as a module, you'll need to load it with modprobe xt_mark.

Comments

[Sean C.]

Is the Linux kernel configured with fwmark support? Per tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.netfilter.html you need CONFIG_IP_ADVANCED_ROUTER, CONFIG_IP_MULTIPLE_TABLES and CONFIG_IP_ROUTE_FWMARK.

[Sean C.]

Try grep IP_NF /boot/config-2.6.32-5-amd64, grep NETFILTER /boot/config-2.6.32-5-amd64, grep CONNMARK /boot/config-2.6.32-5-amd64. I'm not sure what the exact option would be, but from reading it appears that things have changed since 2.6.20. In 3.5 it's CONFIG_NETFILTER_XT_MARK.

[Yusr Safour]

modprobe xt_mark helped. Thank you very much! Write an answer and I'll accept it

[Sean C.]

I wrote up an answer. Feel free to add any edits you like and I'm glad I was able to help.