How can I programmatically manage iptables rules on the fly?

Question

I need to query existing rules, as well as being able to easily add and delete rules. I haven't found any API's for doing this. Is there something that I'm missing?

The closest I've come to a solution is using iptables-save | iptables-xml for querying and manually calling the iptables command itself to add/delete rules. Another solution I've considered is simply regenerating the entire ruleset out of my application's database and flushing the whole chain, then applying it again. But I want to avoid this as I don't want to drop any packets -- unless there's a way to atomically do this. I'm wondering if there's a better way.

An API in C would be great; however, as I'm planning to build this into a stand-alone suid program, libraries that do this in ANY language are fine too.

Solution 1

From the netfilter FAQ:

The answer unfortunately is: No.

Now you might think 'but what about libiptc?'. As has been pointed out numerous times on the mailinglist(s), libiptc was NEVER meant to be used as a public interface. We don't guarantee a stable interface, and it is planned to remove it in the next incarnation of linux packet filtering. libiptc is way too low-layer to be used reasonably anyway.

We are well aware that there is a fundamental lack for such an API, and we are working on improving that situation. Until then, it is recommended to either use system() or open a pipe into stdin of iptables-restore. The latter will give you a way better performance.

Solution 2

Using iptables-save and iptables-restore to query and regenerate rules is easily the most efficient way of doing it. These used to, once, be shell scripts, but now they are C programs that work very efficiently.

However, I should point out that there is a tool that you can use which will make maintaining iptables much easier. Most dynamic rulesets are really the same rule repeated many times, such as:

iptables -A INPUT -s 1.1.1.1 -p tcp -m --dport 22 -j ACCEPT
iptables -A INPUT -s 2.2.2.0/24 -p tcp -m --dport 22 -j ACCEPT
iptables -A INPUT -p tcp -m tcp --dport 22 -j REJECT

Instead of replacing those rules every time you want to change what ports can access port 22 (useful for say, port knocking), you can use ipsets. Viz:

ipset -N ssh_allowed nethash
iptables -A ssh_allowed -m set --set ssh_allowed src -p tcp -m --dport 22 -j ACCEPT
ipset -A ssh_allowed 1.1.1.1
ipset -A ssh_allowed 2.2.2.0/24

Sets can hold ip addresses, networks, ports, mac addresses, and have timeouts on their records. (Ever wanted to add something for just an hour?).

There is even an atomic way of swapping one set with another, so a refresh means creating a new temporary set, then swapping it in as the name of the existing set.

Solution 3

You may consider using rfw which is the REST API for iptables. It is serializing iptables commands from various potentially concurrent sources and remotely executes iptables on the fly.

rfw is designed for distributed systems that try to update firewall rules on multiple boxes but it can be run also on a single machine on localhost interface. Then it allows avoiding the SSL and authentication overhead as it can be run on plain HTTP in this case.

Sample command:

PUT /drop/input/eth0/11.22.33.44

which corresponds to:

iptables -I INPUT -i eth0 -s 11.22.33.44 -j DROP

You can insert and delete rules as well as query for current status to get the existing rules in JSON format:

GET /list/input

Disclaimer: I started that project. It's open source under the MIT license.

Solution 4

As far as I understand (although no reference seems to mention it), iptables-restore is atomic. At the end, when the COMMIT line is read, iptables calls iptc_commit in libiptc (which in an internal interface you aren't supposed to use), which then calls setsockopt(SO_SET_REPLACE) with your new rulesets.

That sounds about as atomic as you can get: with one kernel call. However, more knowledgeable parties are invited to dispute this. :-)

Edit: I can confirm that your description is correct. iptables-restore is done as an atomic operation in the kernel.

To be even more specific the operation "only" is atomic on a per CPU basis. As we store the entire ruleset blob per CPU (due to cache optimizations).

Solution 5

There is deliberately no API to manage these rules. You're not supposed to want to do so. Or something.

If you need rules which are sufficiently dynamic you care about the performance of executing /sbin/iptables, there are other ways to do it:

• Using something like the "recent" match or ip set matching, you can add/remove IP addresses from black/white lists without changing the rule set.
• You can pass packets into userspace for filtering using NFQUEUE

View more solutions

Comments

[CMCDragonkai]

Apparently it's possible to go from XML to iptables-restore xsltproc iptables.xslt my-iptables.xml | iptables-restore. See manpage of iptables-xml.

[forgiven24]

It seems silly to me that there's no API for this. I don't really care about the performance as such, but calling iptables feels like a horribly hacky way of doing things.

[forgiven24]

Hmm, I could use ipsets - and from a performance perspective it's a really good idea. Unfortunately I'd have to roll my own kernel, and this isn't possible as some of the places this software will run is on VMs where I can't mess with the kernel easily. And they still don't provide a nice API.

[forgiven24]

I can accept that using internal APIs is bad, but why is this so bad that they would deliberately not include a public API? I might go with doing a restore if I can do a restore of a single chain - I'll have to do some testing.

[C. K. Young]

I wonder why the FAQ doesn't address the issue of atomicity. It should; I've gone to the trouble of looking at the implementation of iptables-restore just to make sure it's atomic. It's important to the OP here, and I've had a project that required it too.

[C. K. Young]

Yes, you can restore a single chain. :-)

[user1839201]

This netfilter mailing list post says iptables-restore is atomic: mail-archive.com/[email protected]/msg00456.ht‌​ml

[forgiven24]

Aye, the top answer mentioned ipsets, but as I said in a comment there - it requires a kernel module which isn't in Ubuntu by default, and it's not something I can install on any of the VMs that I use.

[MarkR]

ipt_recent is a standard iptables match target which allows you to dynamically add/remove IP addresses from a set by writing to a file in /proc without changing the rules. On the other hand, it's not intended for large sets of IPs and seems to have a fixed maximum limit.

[Jerub]

Yep, I reported that bug in January '07. bugs.launchpad.net/ubuntu/+source/ipset/+bug/79182

[Roman]

For the very brave that still want to do this, there's some info here: netfilter.org/documentation/HOWTO/…

[Jerub]

Ubuntu supports ipset adequately now.

[Grzegorz Luczywo]

Programmatic use of netfilter/iptables suffers from two things: lack of API and lack of atomicity in concurrent access. These are obstacles to changing rules on the fly. To overcome this problem we developed rfw which serves as a system wide iptables guard with REST API. See the full answer: stackoverflow.com/a/22647960/2184341

[tripleee]

What's the significance of the number 123? If you want to check that you have a valid IP address by examining the value of the first octet, it could be anything up to 223 I believe.

[Andrew]

This is not a test of any octets. If when parsing /var/log/secure you end up with an empty or corrupt field, you test the value to not run the iptables command. The '123' value was quite arbitrary. In testing this a bit more, I found you probably should replace '123' with '1' to include a complete IP range, though. When testing an IP less than '123', it will be true for IP addresses in the range 2.0.0.0 to 255.255.255.255, so it will not block IPs in the 1.x.x.x range. When testing IP is less than '1', it matches 0.0.0.1 to 255.255.255.255.

[Alexis Wilke]

If it's over HTTP, doesn't that make it dangerous for a firewall?

[Grzegorz Luczywo]

As stated above, default is HTTPS. There is an option to disable SSL and use plain HTTP, useful when running on localhost in single user environment.

[Dwight Spencer]

Anything that manipulates the firewall "programmatically"/remotely and/or automatically is dangerous. However, proper security in place can mitigate this, ie sso tokens/2fa behind an api gateway(like kong) would go a long way to meet ones security needs.

[Cold]

Good job!!! I think it could be "core" tech for other platforms. Some extensions missing, but it's ok.

[Cold]

Your answer to de OP is, I think, "NO! it's not possible, unless you use subprocess (or direct OS invocation)"

[Victor]

This link may answer the question, but it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes.

[Oliver Gaida]

Have a look at fail2ban. It should the same.