Background, and adding temperature graphing
A couple of years back I added additional attic insulation into my home, bring the insulation up from R-19 to R-38. I also added a radiant barrier in the attic as well, in an attempt to reduce the heat rise in the attic during the summer months especially since the air conditioning ductwork is located up there.
Since I was curious how hot it actually gets in the attic, I installed a ControlByWeb temperature module to monitor the temps. The module provides 4 temperature inputs, using 1-Wire sensors, and uses its built-in web server to output data on a web page or in XML format. It came with 1 temperature sensor, but it was easy to fabricate 3 more using Cat-6 cable and some shrink wrap tubing.
I put two of the temperature sensors in the attic: one near the center, and another near the rear of the attic right above the master bedroom. The remaining two sensors were placed in the garage and outside of the the eaves on the shaded (north-facing) side of the home.
I wrote a short PHP script that parses the temperature module’s XML page and set up a cron job to run every 5 minutes to poll it + pipe it to rrdupdate. A separate bash script calls
rrdtool graph and updates the actual rrd graph every couple of minutes via cron and voila, graphs:
Sending temperatures to Weather Underground
Since I am tracking exterior temperature 24/7, I figured it would be cool to create a Weather Underground Personal Weather Station. That same PHP script that polls the temperature also sends the exterior temp to Weather Underground, via their simple upload protocol.
Adding the Nest Thermostat to the mix
I also installed a Nest Thermostat into the mix, and since it was network-enabled like the ControlByWeb temperature module, it would be great to add its temperature reading to my graphs as well. I could have just repurposed one of the 1-Wire sensors to track indoor temps, but that would have required potentially drilling through some drywall and poking out a sensor somewhere in my house, which would have been unsightly. What better than to just use the reading from the thermostat?
Sadly, Nest doesn’t provide a public API to access its data (at least not yet, until the Nest Developer Program is publicly available), but some enterprising folks have figure out the API that the Nest uses to communicate with the mothership. One such package is nest-api. The package is easy to use, and after a few minutes configuring it, I was able to get the temperature reading of my thermostat. It was more work adding the new data source to the rrd than get nest-api integrated.
The Nest’s temperature graphed in my temperature graph:
Download the files
Go to my project on github to get the source files.
1) Since nest-api basically gives me all the data on the Nest, like if the A/C or furnace are on, it would be nice add to the graph a visualization of that.
2) Also, once I get a Nest Protect, and assuming that it has a built-in temperature sensor, it would be nice to integrate its temperature reading into the graphs as well.
If like me, you use the very useful CloudFlare service to speed up & protect your site(s), you may have noticed that since using CloudFlare, your access logs may seem to have a ton of visits from a very narrow range of IP addresses. This is because CloudFlare acts as a reverse proxy and the IPs you are seeing are from CloudFlare’s network.
This is a bit sucky for analytics since those IPs are not of the actual visitors to your site(s). The original IP is still in the HTTP request headers when CloudFlare is enabled, though, and looks something like this sample request header:
GET /blog/feed/ HTTP/1.0 Host: www.normyee.net Accept-Encoding: gzip CF-Connecting-IP: 188.8.131.52 CF-IPCountry: US X-Forwarded-For: 184.108.40.206 Connection: close Set-Keepalive: 0 Accept: */* From: googlebot(at)googlebot.com User-Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
CloudFlare inserts a
CF-Connecting-IP header containing the original requester’s IP. In this case, the IP 220.127.116.11 is google’s web crawler paying me a visit, although the request was logged as coming from 18.104.22.168 — one of CloudFlare’s IPs. We of course want the original IP logged, and not CloudFlare’s. Fortunately there are quick solutions for both Apache and WordPress.
For Apache, CloudFlare has an Apache module, mod_cloudflare, which you’ll need to compile from source for your system. You can get more info and instructions here & view the source on github here (it’s linked to from the previous link as well). It’s pretty straightforward, assuming you have shell access and the ability to run apxs (the APache eXtenSion tool).
For WordPress, you can just simply download the CloudFlare WordPress plugin at wordpress.org to get the correct IPs back in WordPress. CloudFlare has a wiki page for the plugin as well, but the WordPress.org plugin page has all the info you need.
I’ve shut off FTP access to my server a while back, since FTP passwords are passed in plaintext over the net which is A Bad Thing™. For the occasions that I would need to update WordPress via its web interface, I would start up the FTP daemon so that I could use WordPress’ auto-update feature, and then shut it back down after I was done. Not that big of a hassle, but it required me to login into my server before doing anything from the web interface which was one step too many.
The PECL ssh2 library requires libssh2 which I downloaded and then compiled painlessly with a
$ ./configure $ make $ make install
Once libssh2 was compiled and installed, I installed the PECL ssh2 library via
$ pecl install -f ssh2
The -f flag is to force the install since ssh2 is still in beta & you’d otherwise get a warning like
Failed to download pecl/ssh2 within preferred state "stable", latest release is version 0.11.0, stability "beta", use "channel://pecl.php.net/ssh2-0.11.0" to install
After installing ssh2 via pecl, i edited the php.ini file (located at /etc/php.ini for me) to tell PHP to load this extension
and then restart apache afterwards (via
service httpd restart, or whatever is appropriate for your system) so that the library would be available to PHP.
After apache comes back up, you should see a new option (SSH) in the wordpress upgrade page (which applies to plugins as well).
BTW, if you’re on a Red Hat Fedora system and/or use yum, you might be able to just pull the libssh2 rpm from yum instead of compiling it:
$ yum install libssh2 $ yum install libssh2-devel $ yum install libssh2-docs
One of my Facebook apps hits Amazon’s Ecommerce Service (ECS) for item information via REST queries. I needed to process 19 separate queries searching for a title (basically searching 1 of 19 “browse nodes”), and return that data onto the Facebook canvas.
Simple enough, I could just do a foreach loop to make each REST request. Only problem was, say, each loop took 500 milliseconds total. That’s not hard when you consider all the steps: DNS, performing the REST query, waiting for the response, receiving the response, and then parsing the XML (in this case, using the SimpleXML extension). At 19 requests that’s 9.5 seconds which is way too slow, not to mention Facebook times out the request as well and returns a lovely error page.
For the test REST query I was benchmarking, it was averaging 9.65 seconds for the entire PHP script to complete (performing all 19 REST queries and then formatting the output). Simply switching from file_get_contents() to PHP’s cURL functions dropped the average to 7.32 seconds, a roughly 24% improvement. That was a great improvement, but still too slow and Facebook was still timing out the pages.
The root of the problem was that the 19 REST queries were sequential and as a result too slow. I needed to make those requests concurrently. Fortunately, PHP supported multi curl (basically making multiple cURL requests concurrently) via curl_multi_* functions.
BAM. Using curl_multi dropped the page generation time down to 1.6 seconds — much more reasonable and a 83.4% improvement. w00t.