FreeBSD. Fighting HTTP Floods with IPFW

HTTP flooding is a massive attack on a web server involving an enormous number of HTTP requests.

Three steps to fighting HTTP floods

1. Enable IPFW
2. Write our firewall
3. Write a log parser and add IPs to the blacklist

Enable IPFW

Add this to /etc/rc.conf

firewall_enable="YES"
firewall_script="/etc/fw.sh"
firewall_logging="YES"

Warning! Don’t reboot right away, because you’ll get blocked from the network and won’t be able to get in over SSH. Naturally, this doesn’t apply to those with physical access! :)

Write our firewall

Create our firewall in the file /etc/fw.sh

#!/bin/sh

FwCMD="/sbin/ipfw"

${FwCMD} -f flush
${FwCMD} 10 add check-state

${FwCMD} 40 add allow all from "table(1)" to me
cat /etc/fw_white_list | awk '{ if ($1!="") {system("ipfw table 1 add "$1"")} }' > /dev/null

${FwCMD} 50 add deny all from "table(2)" to me
cat /etc/fw_black_list | awk '{ if ($1!="") {system("ipfw table 2 add "$1"")} }' > /dev/null

${FwCMD} 100 add allow ip from any to any

I hope you can figure out the file permissions yourselves…

table(1) will contain a list of IPs that are allowed in no matter what (for admins :).
table(2) will contain a list of IPs that you block from accessing your server.

Now let’s test our rules!

# ipfw show

You’ll get something like this:

00040     0      0 allow ip from table(1) to me
00050     0      0 deny ip from table(2) to me

If table(1) and table(2) are there, everything is fine. If not, try creating them manually:

# ipfw 40 add allow all from "table(1)" to me
# ipfw 50 add deny all from "table(2)" to me

Let’s test the contents of our tables

# ipfw table 1 list
# ipfw table 2 list

There should be lists like these:

111.111.111.111/32 0
222.222.222.222/32 0

If ipfw won’t run, use the full path, /sbin/ipfw.
Now you can reboot the server to apply the settings.

Write a log parser and add IPs to the blacklist

I use nginx as the frontend, so I’ll be analyzing its access logs, but you can write your own regular expression to parse Apache logs, for example…

For nginx, you need to create an additional access log file, “/var/log/nginx/access-ddos.log”. It’s just more convenient for me that way, but you don’t have to bother—you can use a single log. I simply find it convenient to have two: one contains the full log, while I clear out the other from time to time.

Let’s create the file /etc/banip.sh, which will collect IPs and ban them!

#!/bin/sh

# зачищаем fw_black_list
echo '' > /etc/fw_black_list

# выберем с помощью регулярок IP, у которых нет Useg-Agent'а, отсортируем с уникальным ключом
cat /var/log/nginx/access-ddos.log | awk ' match($0, /(\d*.\d*.\d*.\d*).*"-" "-"/) {print $1}' | sort | uniq >> /etc/fw_black_list

# чистим лог
# echo '' > /var/log/nginx/access-ddos.log

# сбрасываем таблицу с забаненными IP
/sbin/ipfw table 2 flush

# добавляем в таблицу список забанненых IP
cat /etc/fw_black_list | awk '{ if ($1!="") {system("/sbin/ipfw table 2 add "$1"")} }' > /dev/null

Here’s an example of a log with an HTTP flood plus normal user requests (162.115.149.232)

111.111.111.111 - - [31/Mar/2009:14:00:00 +0000] "GET / HTTP/1.1" 304 0 "-" "-"
162.115.149.232 - - [31/Mar/2009:14:01:00 +0000] "GET / HTTP/1.1" 200 8960 "http://example.com./" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322)"
111.111.111.111 - - [31/Mar/2009:14:02:00 +0000] "GET / HTTP/1.1" 304 0 "-" "-"
222.222.222.222 - - [31/Mar/2009:14:03:00 +0000] "GET / HTTP/1.1" 304 0 "-" "-"

Using the regular expression

/(\d*.\d*.\d*.\d*).*"-" "-"/

you can also do it without a regular expression: see the comment

we get

111.111.111.111
111.111.111.111
222.222.222.222

then we’ll pipe the output along and sort the unique IPs in the list

sort | uniq

and the result will be:

111.111.111.111
222.222.222.222

this result is what goes onto the blacklist!

Add a cron job

Set up a cron job; let it do the checking and ban IPs automatically from now on.
I think you can figure out the timing yourselves, because it depends on your particular situation.

0 * * * * /etc/banip.sh| mail -s"`uname -n` backup report" root

Cron cheat sheet

P.S. I wrote this post as simply as possible so that anyone could easily deal with this nuisance and solve the problem. I recommend studying, or at least familiarizing yourself with, AWK before changing anything. You also won’t get anywhere here without regular expressions…

UPD: I forgot to mention this—I think you may be wondering why we need “/etc/fw.sh”. It’s there so we can reload the firewall and change the rules.