Development of l7-filter has moved to the Clear Foundation. These pages are out of date, but will remain as a historical record.

Back to l7-filter main page

L7-filter Userspace Version HOWTO

Last update 10 July 2008

If you have not already, please read the README.

Please note that this version of l7-filter is still in the early stages of development. This documentation may be out-of-date. You should read the documentation included in the package itself also, particularly the man page and the BUGS, Changelog and TODO files.

This version of l7-filter has a number of compatibility problems with newer kernels. We hope to eventually get it all sorted out, but for now the kernel version is the development priority. If you can improve this version, please send your patches to our mailing list!

Table of Contents

Important links on this page:

What you need to get

Download the "l7-filter userspace version" and "Protocol definitions" packages from our Sourceforge project page.

Installation

Compile the program:

./configure will probably find that you need to install some libraries (libnetfilter_conntrack and libnetfilter_queue) from ftp.netfilter.org. Do so if prompted.

Now set up the protocol definitions (a.k.a. pattern files). These files tell l7-filter how protocol names correspond to regular expressions, for example "ftp" means "^220[\x09-\x0d -~]*ftp". Uncompress the "Protocol definitions" package and make the resulting directory /etc/l7-protocols.

Getting it going

First, a reminder: Just because you're using l7-filter, you don't need to do all of your packet classification using it. It's likely that what you want to accomplish can be at least partially done with less demanding classifiers, such as port matching. For instance, you can probably assume that traffic on TCP port 80 that isn't matched by any P2P patterns is HTTP; you don't need to actually use the HTTP pattern.

Kernel

For Linux 2.6.14 through 2.6.19.7, you simply need to have connection tracking and the connection tracking netlink interface enabled. I think that this is the default in most cases. Older versions of Linux don't have this interface and so will not work.

For Linux 2.6.20 and newer, Netfilter has new "Layer 3 Independent Connection tracking" which l7-filter is not yet compatible with (mostly due to lack of library support from libnetfilter_conntrack). While the old layer 3 dependent connection tracking is still available, it is not selected by default, so you will probably need to recompile your kernel with it. In the Linux kernel config, go to:

• Networking → Networking options → Network packet filtering framework (Netfilter) → Core Netfilter Configuration

Under "Netfilter connection tracking support", select "Layer 3 Dependent Connection tracking (OBSOLETE)". Then go to:

• Networking → Networking options → Network packet filtering framework → IP: Netfilter Configuration

and enable "Connection tracking netlink interface" (and probably most of the rest of the stuff on that page). Warning: the kernel guys like to rename these configuration options frequently, so they may not be called exactly what we show here.

Either way, you need either the module ip_conntrack_netlink or nf_conntrack_netlink or the same code compiled into your kernel.

l7-filter

Set up your l7-filter configuration file. It consists of pairs of protocol names and Netfilter mark numbers. l7-filter will mark packets that match each given protocol with the corresponding mark number. There is a sample in the l7-filter-userspace source tarball. For a list of valid protocols, see the protocols page. You can also add your own protocols. Marks are 32 bit integers, but l7-filter assigns special meaning to 0, 1 and 2, so yours must be at least 3.

Load the ip_conntrack_netlink module with modprobe ip_conntrack_netlink, or make sure it is compiled into your kernel.

Send traffic to l7-filter using one of:

iptables [specify table and chain] -j QUEUE
iptables [specify table and chain] -j NFQUEUE --queue-num [queue number]

NFQUEUE defaults to queue number 0 and so does l7-filter. So to send all traffic passing through your machine to l7-filter on queue number 0:

iptables -A FORWARD -j (NF)QUEUE

Now run l7-filter:

l7-filter -f [configuration file] -q [queue number]

For example, if you're using queue number 0 and your configuration file is l7-filter.conf:

l7-filter -f l7-filter.conf

In order to do its classification, l7-filter must be able to see all of the relevant traffic, so make sure you do not use a chain like OUTPUT which only gets traffic going in one direction. See this packet flow diagram for details. In some cases, l7-filter can sucessfully match even if it can only see one side of the connection, but in general, this won't work.

See man l7-filter for various ways of tweaking l7-filter's behavior.

What it does

l7-filter examines the application data in connections to determine what protocol is being used and sets Netfilter marks accordingly. It expects that packets it receives have no mark yet (that is, their mark is 0). It generally takes a few packets before l7-filter can identify what protocol is being used, so it gives packets from new, unidentified connections the special mark of 1. Eventually if l7-filter can't identify a connection, it will give up. In this case, it marks them with the special value 2. Otherwise, it marks packets with the marks specified in its configuration file.

l7-filter ACCEPTs all packets sent to it. This means that an iptables rule that sends packets to l7-filter always terminates the chain and sends the packet on. (This is not how we would like it to work, but the Netfilter libraries currently make any other approach very difficult.) So in order to use the marks that l7-filter sets, you need to use a chain that is downstream of it. See the next section of this document.

Making it useful

Just marking packets isn't very interesting. What you actually want is probably one of these three things: (1) accounting (2) controlling bandwidth use (3) blocking. To do these, you must set up rules to act on the marks that l7-filter applies to the packets. These can either be iptables (firewall) or tc (QoS) rules.

1. Accouting

If you just want to keep track of what's in use on your network, simply match on the mark in some chain that is downstream of where you are QUEUEing to l7-filter and don't use any -j option. For instance, if l7-filter acts in FORWARD, you can do, for instance:

iptables -t mangle -A POSTROUTING -m mark --mark 3

and then get statistics by using iptables -L. (See man iptables for details.)

2. Bandwidth Restriction

If l7-filter has marked, say, IMAP packets with 3, you can use tc ("traffic control", the userspace tool for Linux QoS, part of the iproute2 package) to control the bandwidth used by IMAP using commands like this:

tc filter add dev eth0 protocol ip parent 1:0 prio 1 handle 3 fw flowid 1:3

Did you understand that? You can try reading The Linux Advanced Routing and Traffic Control HOWTO for enlightenment. You should do this so that you have some idea what you're doing, but unfortunately, tc is incredibly obtuse and you're likely to wish you just had a canned script such as this one to work from. This will need to be modified if your setup is significantly different than mine, but it should provide a good starting point.

Be prudent when choosing the amount of bandwidth you allow each protocol. Restricting a protocol to an unusably low bandwidth can have similar consequences to blocking it (see next section).

3. Blocking

Don't. Here's why:

Instead of dropping packets you don't like, we recommend using Linux QoS to restrict their bandwidth usage. If you insist on using l7-filter to drop packets, make sure you have investigated other options first, such as the features of your HTTP proxy (useful for worms).

More information

Dealing with FTP, IRC, etc.

Some protocols open child connections to transfer data. This version of l7-filter currently cannot handle this by itself. It will only correctly match the parent connection.

Other things that use Netfilter marks

If a packet already has a non-zero Netfilter mark when l7-filter gets it, it will pass it on with that same mark without trying to identify it. You can modify this behaviour using the -m and -c options. See man l7-filter.

Upgrading the protocol definitions

The protocol definitions are simple text files with a format described in the Pattern-HOWTO. They can be updated as a package or individually.

L7-filter only reads the definitions when it starts up, so if you update the protocol definitions, restart it.

Other things to know

Please see the FAQ for more information.