le blog » Linux

Adding Cloudflare support to fail2ban

norm — Thu, 02 Feb 2012 23:05:28 +0000

I use a few open source packages to help automate some defensive responses against hackers to my server. One is fail2ban. fail2ban can be configured to monitor pretty much any system daemon that produces a log file.

I also use CloudFlare, which is a CDN and DNS service that amongst its many features is automatic protection against malicious attacks (DDoS, SQL injections, comment spamming, etc). In addition, if there are IPs that slip through their detection, you can add them manually to your CloudFlare blacklist. Automatically detecting these additional IPs, and adding them to CloudFlare, is where fail2ban comes in.

So on to my scenario: I noticed some very specific, recurring HTTP requests on my server that served no purpose other than to probe for a vulnerability or to attack my server, a perfect scenario for creating a fail2ban “jail.” A fail2ban jail is a combination of a fail2ban filter and action. Typically you have a jail configured to block an IP at the iptables level, but I wanted it to be blocked at the CloudFlare level as well.

Setting up a fail2ban jail is pretty straightforward. The jail contains all the information that fail2ban needs to detect and act upon a matched condition. In a typical fail2ban install, you’ll find the jail.conf file, where all the jails are defined, at /etc/fail2ban/jail.conf

Creating a new fail2ban jail

Creating a new jail is as simple as adding something like so to jail.conf:

[some-jail-name]
enabled  = true
filter   = my-filter-name
action   = iptables-allports[name=some-jail-name, protocol=tcp]
           cloudflare-blacklist
           sendmail-whois[name=some-jail-name, dest=me@example.com]
logpath  = /var/log/some/log
maxretry = 0
bantime  = 604800

This creates a fail2ban filter called “some-jail-name”, scanning the log “/var/log/some/log” defined by “logpath,” applying the filter “my-filter-name” defined by “filter” on said log, and acting upon a match with the action(s) defined by “action” by doing the following: 1) block the IP in iptables, 2) block the IP at CloudFlare, and 3) send me a notification email. (Consult the fail2ban manual on jail options to see what they all mean)

Creating the fail2ban filter

The filter “my-filter-name” would then be defined at /etc/fail2ban/filter.d/my-filter-name.conf. Make sure to modify the failregex regex string to match what you are looking for in the log file. Be sure to consult the section on testing your filters in the fail2ban manual to confirm your filter works correctly.

[Definition]

# Option:  failregex
# Notes.:  regex to match the suspicious attemps in the logfile. The
#          host must be matched by a group named "host". The tag "<HOST>" can
#          be used for standard IP/hostname matching and is only an alias for
#          (?:::f{4,6}:)?(?P<host>\S+)
# Values:  TEXT
#
failregex = ^<HOST> -.*POST /tmp.*$ 

# Option:  ignoreregex
# Notes.:  regex to ignore. If this regex matches, the line is ignored.
# Values:  TEXT
#
ignoreregex =

Creating the fail2ban action

And finally, the action “cloudflare-blacklist” is defined at /etc/fail2ban/action.d/cloudflare-blacklist.conf. It will call CloudFlare’s client API to add/remove an IP from your blacklist.

In order to configure your action properly:

Go to your CloudFlare account settings page to get your API key (it should be near the bottom of the page)
In the actionban and actionunban sections, replace CLOUDFLARE_API_TOKEN with your API key
In the actionban and actionunban sections,replace CLOUDFLARE_LOGIN with your CloudFlare login email

# Fail2Ban configuration file
#
# Author: Norman Yee
#
# $Revision$
#

[Definition]

# Option:  actionstart
# Notes.:  command executed once at the start of Fail2Ban.
# Values:  CMD
#
actionstart =

# Option:  actionstop
# Notes.:  command executed once at the end of Fail2Ban
# Values:  CMD
#
actionstop =

# Option:  actioncheck
# Notes.:  command executed once before each actionban command
# Values:  CMD
#
actioncheck =

# Option:  actionban
# Notes.:  command executed when banning an IP. Take care that the
#          command is executed with Fail2Ban user rights.
# Tags:    <ip>  IP address
#          <failures>  number of failures
#          <time>  unix timestamp of the ban time
# Values:  CMD
#
actionban = curl -s "https://www.cloudflare.com/api.html?a=ban&key=<ip>&u=CLOUDFLARE_LOGIN&tkn=CLOUDFLARE_API_TOKEN"

# Option:  actionunban
# Notes.:  command executed when unbanning an IP. Take care that the
#          command is executed with Fail2Ban user rights.
# Tags:    <ip>  IP address
#          <failures>  number of failures
#          <time>  unix timestamp of the ban time
# Values:  CMD
#
actionunban = curl -s "https://www.cloudflare.com/api.html?a=nul&key=<ip>&u=CLOUDFLARE_LOGIN&tkn=CLOUDFLARE_API_TOKEN"

Once you have the jail, filter, and action set up and configured in fail2ban, you should be all set, with an additional layer of security courtesy of CloudFlare!

CloudFlare, Apache, WordPress and IP address logging

norm — Fri, 07 Oct 2011 06:53:40 +0000

If like me, you use the very useful CloudFlare service to speed up & protect your site(s), you may have noticed that since using CloudFlare, your access logs may seem to have a ton of visits from a very narrow range of IP addresses. This is because CloudFlare acts as a reverse proxy and the IPs you are seeing are from CloudFlare’s network.

This is a bit sucky for analytics since those IPs are not of the actual visitors to your site(s). The original IP is still in the HTTP request headers when CloudFlare is enabled, though, and looks something like this sample request header:

GET /blog/feed/ HTTP/1.0
Host: www.normyee.net
Accept-Encoding: gzip
CF-Connecting-IP: 66.249.71.111
CF-IPCountry: US
X-Forwarded-For: 66.249.71.111
Connection: close
Set-Keepalive: 0
Accept: */*
From: googlebot(at)googlebot.com
User-Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

CloudFlare inserts a CF-Connecting-IP header containing the original requester’s IP. In this case, the IP 66.249.71.111 is google’s web crawler paying me a visit, although the request was logged as coming from 199.27.128.71 — one of CloudFlare’s IPs. We of course want the original IP logged, and not CloudFlare’s. Fortunately there are quick solutions for both Apache and WordPress.

For Apache, CloudFlare has an Apache module, mod_cloudflare, which you’ll need to compile from source for your system. You can get more info and instructions here & view the source on github here (it’s linked to from the previous link as well). It’s pretty straightforward, assuming you have shell access and the ability to run apxs (the APache eXtenSion tool).

For WordPress, you can just simply download the CloudFlare WordPress plugin at wordpress.org to get the correct IPs back in WordPress. CloudFlare has a wiki page for the plugin as well, but the WordPress.org plugin page has all the info you need.

Using DNS whitelists in Postfix

norm — Fri, 08 Jul 2011 20:15:22 +0000

Update 7/21/2011: I’ve tweaked my main.cf config more recently that provided better results. I will have a follow-up post with the changes and additions.

In an earlier post several years back, I wrote about using DNS blacklists (DNSBLs) in postfix to block unwanted spam from hitting my inbox, and in the last week I tweaked it a bit to include DNS whitelists (DNSWLs). I was discovering that some of the DNSBLs were blocking a small portion of legitimate email from coming through (eg. blocking entire IP ranges for webmail providers).

You need Postfix 2.8 or higher (I was on 2.3.3 so I had to download and compile the latest, 2.8.3), which has a new configuration parameter which added support for querying a DNSWL like a DNSBL:

permit_dnswl_client dnswl_domain=d.d.d.d

The relevant portion of my updated Postfix main.cf is below (note permit_dnswl_client is inserted before any of the reject_rbl* parameters). The DNSWL queries are processed first before the DNSBLs, so it should allow legitimate IPs through and reduce the amount of false positives vs just using the DNSBLs alone.

smtpd_recipient_restrictions =
        permit_mynetworks,
        reject_invalid_hostname,
        reject_non_fqdn_sender,
        reject_non_fqdn_recipient,
        reject_unknown_sender_domain,
        reject_unknown_recipient_domain,
        reject_unauth_destination,
        permit_dnswl_client list.dnswl.org,
        reject_rbl_client b.barracudacentral.org,
        reject_rbl_client zen.spamhaus.org,
        reject_rbl_client bl.spameatingmonkey.net,
        reject_rbl_client bl.spamcop.net,
        reject_rbl_client hostkarma.junkemailfilter.com=127.0.0.2,
        reject_rbl_client dnsbl.njabl.org,
        reject_rbl_client bl.tiopan.com,
        reject_rbl_client spamsources.fabel.dk,
        reject_rbl_client truncate.gbudb.net,
        reject_rbl_client ubl.unsubscore.com,
        reject_rbl_client aspews.ext.sorbs.net,
        reject_rbl_client dnsbl.sorbs.net,
        reject_rbl_client backscatter.spameatingmonkey.net,
        reject_rbl_client bl.spameatingmonkey.net,
        reject_rhsbl_sender fresh15.spameatingmonkey.net,
        reject_rhsbl_client fresh15.spameatingmonkey.net,
        reject_rhsbl_sender uribl.spameatingmonkey.net,
        reject_rhsbl_client uribl.spameatingmonkey.net,
        reject_rhsbl_sender urired.spameatingmonkey.net,
        reject_rhsbl_client urired.spameatingmonkey.net,
        reject_rbl_client dnsbl.inps.de,
        reject_rbl_client DDNSBL.InternetDefenseSystems.com,
        permit

After watching the logs for the last couple days, this setup seems to work quite well…only 1 piece of spam slipped through, and more importantly, no false positives! I should also note that I stopped using spamassassin a while back and just these settings in postfix seems to block almost all legitimate spam coming into my server while not generating false positives.

BTW, if you want to tail your maillog from the command line in realtime and see the rejects in a more readable format, this short one liner will work:

$ tail -f /var/log/maillog | sed -r -n 's/^.* blocked using ([^;]+).*; from=]+)>\s*to=]+)>.*$/\1 - \2 - \3/p'

it will output in the format DNSBL – FROM ADDRESS – TO ADDRESS

Updating WordPress via SSH instead of FTP

norm — Tue, 01 Dec 2009 19:09:54 +0000

I’ve shut off FTP access to my server a while back, since FTP passwords are passed in plaintext over the net which is A Bad Thing™. For the occasions that I would need to update WordPress via its web interface, I would start up the FTP daemon so that I could use WordPress’ auto-update feature, and then shut it back down after I was done. Not that big of a hassle, but it required me to login into my server before doing anything from the web interface which was one step too many.

I discovered that WordPress has built-in support for updating via SSH2 if PHP has the PECL SSH2 library installed. Following is a quick summary of how to get it running.

The PECL ssh2 library requires libssh2 which I downloaded and then compiled painlessly with a

$ ./configure
$ make
$ make install

Once libssh2 was compiled and installed, I installed the PECL ssh2 library via

$ pecl install -f ssh2

The -f flag is to force the install since ssh2 is still in beta & you’d otherwise get a warning like

Failed to download pecl/ssh2 within preferred state "stable", latest release is version 0.11.0, stability "beta", use "channel://pecl.php.net/ssh2-0.11.0" to install

After installing ssh2 via pecl, i edited the php.ini file (located at /etc/php.ini for me) to tell PHP to load this extension

extension=ssh2.so

and then restart apache afterwards (via apachctl, service httpd restart, or whatever is appropriate for your system) so that the library would be available to PHP.

After apache comes back up, you should see a new option (SSH) in the wordpress upgrade page (which applies to plugins as well).

BTW, if you’re on a Red Hat Fedora system and/or use yum, you might be able to just pull the libssh2 rpm from yum instead of compiling it:

$ yum install libssh2
$ yum install libssh2-devel
$ yum install libssh2-docs

le blog » Linux

Adding Cloudflare support to fail2ban

CloudFlare, Apache, WordPress and IP address logging

More robust Postfix anti-spam configuration

Using DNS whitelists in Postfix

Updating WordPress via SSH instead of FTP