Graphing temperatures from 1-Wire sensors and the Nest Thermostat in rrdtool

0

Background, and adding temperature graphing

A couple of years back I added additional attic insulation into my home, bring the insulation up from R-19 to R-38. I also added a radiant barrier in the attic as well, in an attempt to reduce the heat rise in the attic during the summer months especially since the air conditioning ductwork is located up there.

Since I was curious how hot it actually gets in the attic, I installed a ControlByWeb temperature module to monitor the temps. The module provides 4 temperature inputs, using 1-Wire sensors, and uses its built-in web server to output data on a web page or in XML format. It came with 1 temperature sensor, but it was easy to fabricate 3 more using Cat-6 cable and some shrink wrap tubing.

I put two of the temperature sensors in the attic: one near the center, and another near the rear of the attic right above the master bedroom. The remaining two sensors were placed in the garage and outside of the the eaves on the shaded (north-facing) side of the home.

I wrote a short PHP script that parses the temperature module’s XML page and set up a cron job to run every 5 minutes to poll it + pipe it to rrdupdate. A separate bash script calls rrdtool graph and updates the actual rrd graph every couple of minutes via cron and voila, graphs:

Sending temperatures to Weather Underground

Since I am tracking exterior temperature 24/7, I figured it would be cool to create a Weather Underground Personal Weather Station. That same PHP script that polls the temperature also sends the exterior temp to Weather Underground, via their simple upload protocol.

Adding the Nest Thermostat to the mix

I also installed a Nest Thermostat into the mix, and since it was network-enabled like the ControlByWeb temperature module, it would be great to add its temperature reading to my graphs as well. I could have just repurposed one of the 1-Wire sensors to track indoor temps, but that would have required potentially drilling through some drywall and poking out a sensor somewhere in my house, which would have been unsightly. What better than to just use the reading from the thermostat?

Sadly, Nest doesn’t provide a public API to access its data (at least not yet, until the Nest Developer Program is publicly available), but some enterprising folks have figure out the API that the Nest uses to communicate with the mothership. One such package is nest-api. The package is easy to use, and after a few minutes configuring it, I was able to get the temperature reading of my thermostat. It was more work adding the new data source to the rrd than get nest-api integrated.

The Nest’s temperature graphed in my temperature graph:

Download the files

Go to my project on github to get the source files.

TODOs

1) Since nest-api basically gives me all the data on the Nest, like if the A/C or furnace are on, it would be nice add to the graph a visualization of that.

2) Also, once I get a Nest Protect, and assuming that it has a built-in temperature sensor, it would be nice to integrate its temperature reading into the graphs as well.

Postfix anti-spam configuration, December 2012

15

I’ve written a few posts on my Postfix anti-spam config, but since it’s been a while and I tweak it from time to time, here’s the config as of late December 2012:

smtpd_helo_restrictions =
        reject_non_fqdn_helo_hostname,
        reject_invalid_helo_hostname,
        reject_rhsbl_helo hostkarma.junkemailfilter.com=127.0.0.2,
        reject_rhsbl_helo zen.spamhaus.org

smtpd_data_restrictions =
        reject_unauth_pipelining

smtpd_client_restrictions =
        reject_rbl_client b.barracudacentral.org,
        reject_rbl_client hostkarma.junkemailfilter.com=127.0.0.2,
        reject_unknown_client_hostname

smtpd_sender_restrictions =
        reject_unknown_sender_domain,
        reject_unknown_address,
        reject_rhsbl_reverse_client dbl.spamhaus.org,
        reject_rbl_client b.barracudacentral.org

smtpd_recipient_restrictions =
        permit_mynetworks,
        reject_invalid_hostname,
        reject_non_fqdn_sender,
        reject_non_fqdn_recipient,
        reject_unknown_sender_domain,
        reject_unknown_recipient_domain,
        reject_unauth_destination,

        permit_dnswl_client list.dnswl.org,

        check_policy_service inet:127.0.0.1:10023,

        reject_rhsbl_reverse_client dbl.spamhaus.org,
        reject_rhsbl_sender dbl.spamhaus.org,
        reject_rhsbl_client dbl.spamhaus.org,
        reject_rhsbl_sender fresh15.spameatingmonkey.net,
        reject_rhsbl_client fresh15.spameatingmonkey.net,
        reject_rhsbl_sender uribl.spameatingmonkey.net,
        reject_rhsbl_client uribl.spameatingmonkey.net,
        reject_rhsbl_sender urired.spameatingmonkey.net,
        reject_rhsbl_client urired.spameatingmonkey.net,
        reject_rhsbl_client hostkarma.junkemailfilter.com=127.0.0.2,

        reject_rbl_client b.barracudacentral.org,
        reject_rbl_client zen.spamhaus.org,
        reject_rbl_client bl.spameatingmonkey.net,
        reject_rbl_client bl.spamcop.net,
        reject_rbl_client hostkarma.junkemailfilter.com=127.0.0.2,
        reject_rbl_client dnsbl.njabl.org,
        reject_rbl_client spamsources.fabel.dk,
        reject_rbl_client truncate.gbudb.net,
        reject_rbl_client ubl.unsubscore.com,
        reject_rbl_client aspews.ext.sorbs.net,
        reject_rbl_client dnsbl.sorbs.net,
        reject_rbl_client backscatter.spameatingmonkey.net,
        reject_rbl_client bl.spameatingmonkey.net,
        reject_rbl_client psbl.surriel.com,
        reject_rbl_client cidr.bl.mcafee.com,
        reject_rbl_client bl.mailspike.net,
        reject_rbl_client ix.dnsbl.manitu.net,
        reject_rbl_client black.uribl.com,
        reject_rbl_client spam.spamrats.com,

        permit

I had a string of spam making it into my mailbox recently which is why I added a few new RBLs to the config, but based on dnsblcount‘s report on the number of DNSBL rejections since the beginning of the month, I can probably trim it down a little (note that the DNSBLs are queried in the order listed in your config).

b.barracudacentral.org            6457
hostkarma.junkemailfilter.com     1513
dbl.spamhaus.org                  1110
fresh15.spameatingmonkey.net        35
zen.spamhaus.org                    17
dnsbl.webequipped.com                3
ubl.unsubscore.com                   3
spam.spamrats.com                    2
truncate.gbudb.net                   2
uribl.spameatingmonkey.net           1
psbl.surriel.com                     1
dnsbl.sorbs.net                      1
bl.mailspike.net                     1
=======================================
Total DNSBL rejections:            9146

Note one additional new item I added to my postfix config is the postgrey greylisting policy service (via the check_policy_service inet:127.0.0.1:10023).

Adding Cloudflare support to fail2ban

0

I use a few open source packages to help automate some defensive responses against hackers to my server. One is fail2ban. fail2ban can be configured to monitor pretty much any system daemon that produces a log file.

I also use CloudFlare, which is a CDN and DNS service that amongst its many features is automatic protection against malicious attacks (DDoS, SQL injections, comment spamming, etc). In addition, if there are IPs that slip through their detection, you can add them manually to your CloudFlare blacklist. Automatically detecting these additional IPs, and adding them to CloudFlare, is where fail2ban comes in.

So on to my scenario: I noticed some very specific, recurring HTTP requests on my server that served no purpose other than to probe for a vulnerability or to attack my server, a perfect scenario for creating a fail2ban “jail.” A fail2ban jail is a combination of a fail2ban filter and action. Typically you have a jail configured to block an IP at the iptables level, but I wanted it to be blocked at the CloudFlare level as well.

Setting up a fail2ban jail is pretty straightforward. The jail contains all the information that fail2ban needs to detect and act upon a matched condition. In a typical fail2ban install, you’ll find the jail.conf file, where all the jails are defined, at /etc/fail2ban/jail.conf

Creating a new fail2ban jail

Creating a new jail is as simple as adding something like so to jail.conf:

[some-jail-name]
enabled  = true
filter   = my-filter-name
action   = iptables-allports[name=some-jail-name, protocol=tcp]
           cloudflare-blacklist
           sendmail-whois[name=some-jail-name, dest=me@example.com]
logpath  = /var/log/some/log
maxretry = 0
bantime  = 604800

This creates a fail2ban filter called “some-jail-name”, scanning the log “/var/log/some/log” defined by “logpath,” applying the filter “my-filter-name” defined by “filter” on said log, and acting upon a match with the action(s) defined by “action” by doing the following: 1) block the IP in iptables, 2) block the IP at CloudFlare, and 3) send me a notification email. (Consult the fail2ban manual on jail options to see what they all mean)

Creating the fail2ban filter

The filter “my-filter-name” would then be defined at /etc/fail2ban/filter.d/my-filter-name.conf. Make sure to modify the failregex regex string to match what you are looking for in the log file. Be sure to consult the section on testing your filters in the fail2ban manual to confirm your filter works correctly.

[Definition]

# Option:  failregex
# Notes.:  regex to match the password failure messages in the logfile. The
#          host must be matched by a group named "host". The tag "<HOST>" can
#          be used for standard IP/hostname matching and is only an alias for
#          (?:::f{4,6}:)?(?P<host>\S+)
# Values:  TEXT
#
failregex = ^<HOST> -.*POST /tmp.*$ 

# Option:  ignoreregex
# Notes.:  regex to ignore. If this regex matches, the line is ignored.
# Values:  TEXT
#
ignoreregex =

Creating the fail2ban action

And finally, the action “cloudflare-blacklist” is defined at /etc/fail2ban/action.d/cloudflare-blacklist.conf. It will call CloudFlare’s client API to add/remove an IP from your blacklist. 

In order to configure your action properly:

  1. Go to your CloudFlare account settings page to get your API key (it should be near the bottom of the page)
  2. In the actionban and actionunban sections, replace CLOUDFLARE_API_TOKEN with your API key
  3. In the actionban and actionunban sections,replace CLOUDFLARE_LOGIN with your CloudFlare login email
# Fail2Ban configuration file
#
# Author: Norman Yee
#
# $Revision$
#

[Definition]

# Option:  actionstart
# Notes.:  command executed once at the start of Fail2Ban.
# Values:  CMD
#
actionstart =

# Option:  actionstop
# Notes.:  command executed once at the end of Fail2Ban
# Values:  CMD
#
actionstop =

# Option:  actioncheck
# Notes.:  command executed once before each actionban command
# Values:  CMD
#
actioncheck =

# Option:  actionban
# Notes.:  command executed when banning an IP. Take care that the
#          command is executed with Fail2Ban user rights.
# Tags:    &lt;ip&gt;  IP address
#          &lt;failures&gt;  number of failures
#          &lt;time&gt;  unix timestamp of the ban time
# Values:  CMD
#
actionban = curl -s "https://www.cloudflare.com/api.html?a=ban&key=<ip>&u=CLOUDFLARE_LOGIN&tkn=CLOUDFLARE_API_TOKEN"

# Option:  actionunban
# Notes.:  command executed when unbanning an IP. Take care that the
#          command is executed with Fail2Ban user rights.
# Tags:    &lt;ip&gt;  IP address
#          &lt;failures&gt;  number of failures
#          &lt;time&gt;  unix timestamp of the ban time
# Values:  CMD
#
actionban = curl -s "https://www.cloudflare.com/api.html?a=nul&key=<ip>&u=CLOUDFLARE_LOGIN&tkn=CLOUDFLARE_API_TOKEN"

Once you have the jail, filter, and action set up and configured in fail2ban, you should be all set, with an additional layer of security courtesy of CloudFlare!

Xcode freezing up or hanging, using 100% CPU for no apparent reason?

0

If you have an iOS 5 device that is configured to use iOS 5′s new WiFi sync feature, and you’re experiencing high CPU when using Xcode or it becomes non-responsive (eg. Xcode gets pegged at 100% or higher for not apparent reason on an otherwise idle machine), disable WiFi sync. Once I disabled WiFi sync, the high CPU usage went away. I’m guessing that perhaps my older version of Xcode (3.2.6) is trying to sync with the device over WiFi but doesn’t quite know how to do that.

CloudFlare, Apache, WordPress and IP address logging

0

If like me, you use the very useful CloudFlare service to speed up & protect your site(s), you may have noticed that since using CloudFlare, your access logs may seem to have a ton of visits from a very narrow range of IP addresses. This is because CloudFlare acts as a reverse proxy and the IPs you are seeing are from CloudFlare’s network.

This is a bit sucky for analytics since those IPs are not of the actual visitors to your site(s). The original IP is still in the HTTP request headers when CloudFlare is enabled, though, and looks something like this sample request header:

GET /blog/feed/ HTTP/1.0
Host: www.normyee.net
Accept-Encoding: gzip
CF-Connecting-IP: 66.249.71.111
CF-IPCountry: US
X-Forwarded-For: 66.249.71.111
Connection: close
Set-Keepalive: 0
Accept: */*
From: googlebot(at)googlebot.com
User-Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

CloudFlare inserts a CF-Connecting-IP header containing the original requester’s IP. In this case, the IP 66.249.71.111 is google’s web crawler paying me a visit, although the request was logged as coming from 199.27.128.71 — one of CloudFlare’s IPs. We of course want the original IP logged, and not CloudFlare’s. Fortunately there are quick solutions for both Apache and WordPress.

For Apache, CloudFlare has an Apache module, mod_cloudflare, which you’ll need to compile from source for your system. You can get more info and instructions here & view the source on github here (it’s linked to from the previous link as well). It’s pretty straightforward, assuming you have shell access and the ability to run apxs (the APache eXtenSion tool).

For WordPress, you can just simply download the CloudFlare WordPress plugin at wordpress.org to get the correct IPs back in WordPress. CloudFlare has a wiki page for the plugin as well, but the WordPress.org plugin page has all the info you need.

Page 1 of 1612345...10...Last »
Go to Top