Making iptables recognize youtube streams and large files

Question

I use Ubuntu server 10.04 with the stock kernel as a NAT router. I use iptables and tc to do port based traffic prioritization. It is working fine. I have given http higher priority. So my network gets swamped with youtube and bigger http downloads. My question is: Is it possible to make iptables recognize and mark packets that belong to youtube streams or larger files? my goal is to make webpage browsing faster.

Solution 1

You can try using -m connbytes. Have it match something large. Then mark the packet, and use tc's fwmark matching mechanism.

However, I tend to agree with the others that for this purpose, you should use something that works at Layer 7.

---

Example usage of -m connbytes

iptables -A FORWARD -i $WAN_IFACE -o $LAN_IFACE -m connbytes --connbytes $TRIGGER_SIZE: --connbytes-dir reply --connbytes-mode bytes -j CONNMARK --set-mark $BULK_MARKER
iptables -t mangle -A PREROUTING -m connmark --mark $BULK_MARKER -j CONNMARK --restore-mark

The first rule will match against connections whose total bytes transferred is greater than $TRIGGER_SIZE (careful; that colon must exist!)

The second rule will transfer the CONNMARK to packet MARK (fwmark) so that tc can do some throttling/policing.

Note that I specify -i, -o, and -dir reply in the 1st rule; this will match against packets entering WAN interface (and exiting LAN interface) that are replies to an outgoing connection initiated by someone in the LAN. If you do not specify -i and -o, and/or use -dir both, you will also match against outgoing packets (e.g., mails from the mail server).

Solution 2

As already posted by Caleb, this is not a task for iptables as you are lacking "proper" content inspection features there. You might want to take a look into a HTTP proxy like Squid. Recent versions come with sophisticated bandwidth regulation mechanisms called "delay pools" which can be combined with tagging mechanisms called "ACLs" for complex bandwidth management scenarios.

We have set up squid installations at customer sites with requirements pretty similar to yours.

Solution 3

This from the perspective of iptables for a sec. When you handle packets one by one it is not possible to identify a packet as coming from the transfer of a large or a small file, you just pass them along. If you do deep packet inspection you can find a little bit more about the content, but you still don't really know how big a file is going to be or even that it's part of a file and not a web page.

You could tag packets routed to and from Youtube and filter those separately, but I find the last thing you want to do with streaming media is slow it down. The faster it happens the happier the users are and the sooner the traffic is out of the way. You can use TC to put everything coming from Youtube servers (by IP block) into a lower priority queue, but if your users are still watching videos, whether they take a long time or a short time to download the same number of bytes are going to move over your network.

I suggest you consider this at least partially an HR problem and make and enforce (using human communication not digital detection) some acceptable use policies. Then if you still need network capacity, grow your infrastructure a little.

Comments

[the-wabbit]

In some scenarios it is sensible to slow down bulk transfers - whenever you are expecting data with the need for a low-latency link (VoIP, Terminal Services or remote shells) for example.

[clemens utschig]

My eth0 is WAN, and eth1 is LAN interface. So what would be the iptables connbytes syntax to mark packets if data download goes above 512 kB.

[pepoluan]

@nixnotwin I've edited my answer

[clemens utschig]

I tried it, the rules got applied, but when I did iptables -v -L, it doesn't show any bytes or packets that are transferred. Here are my entire qos rules:pastebin.com/2vkmptbA

[pepoluan]

@nixnotwin unfortunately I'm not (yet) familiar with tc. can you replace the -j on both rule with -j LOG --log-prefix somestring, and see in dmesg if the rules 'fired'?