Keeping tabs on my standby generator with Raspberry Pi (Part I)

Background: We live in a rural area of southern Ontario, Canada. As beautiful an area as it is, we are not exactly sheltered from the effects of wild and nasty weather. Electric power failures can occur at any time during snow storms, ice storms, thunder storms or just plain wind storms. We need power badly though, to keep the sump pump running in order to keep the basement dry, for one thing. So, after a flood some years ago, we decided to have a contractor install a backup generator. This generator, brand name Carrier, powered by propane, will kick in automatically 30 seconds after a power loss and provide power to most of the house, especially crucial elements such as freezer, fridge and sump pump. Now, I must say, this generator has worked flawlessly since the day it was installed 4 years ago, but... it really doesn't provide any feedback as to whether or not it is performing at all.

In order to keep parts lubricated and battery charged, it performs a self-test every Monday morning. It starts up, runs for 12 minutes and then shuts down. If we happen to be home, you might actually hear it running. If you are away, well, you have no idea what it did. Never mind if you are away and an actually hydro power loss occurs: you are in the dark as to this event happening.

Now they say that ignorance is bliss, but only up to a point. When you are away on vacation, there is always this nagging feeling in the back of your head: how are things back home?
Rapsberry Pi to the rescue. In short, this is what I did in order to have it monitor the generator:

- I hooked up three DS18B20 1-wire temperature sensors, one for inside the house, one outside and one in the generator cabinet
- I wrote a Python program to take temperature readings every minute and log these to a SQLite database
- In this Python application, I pay special attention to the generator cabinet sensor. If the temperature difference in this cabinet exceeds 5 degrees in a five minute span, that means in all likelihood that the generator has started
- If that happens, the Python application will send me an SMS (text message), as well as an email.

In addition, I also set up the Pi as a web server, to allow me to follow the temperatures. For that I use Bottle.py, a micro web-framework for Python, running in a separate application. This allows me to keep tabs on the generator and other temperatures no matter where I may be chasing my dreams in the world. Lastly, I decided to add some LEDs to the Pi, to allow some feedback when it is running headless. When the red LED is on, it is processing the current set of temperatures. When the yellow LED is on, that means somebody somewhere is accessing the Pi via the web. It all sounds pretty simple and straightforward, doesn't it? Here's how it is done:
Note: The Raspberry Pi community is fabulous. If you search long and hard enough, you will find the answer to your Pi question somewhere. Without all the work done by others, I still would not have left the starting gate. Specifically, I want to mention:

http://www.trainelectronics.com/RaspberryPi/Graph_Temperature/index.html

whose ideas and code I used as a basis for mine, though mine deviates in significant ways, especially when it comes to storing data and how this data is made available to the end user (FTP versus web page).

Design parameters:
1. cost has to be low. Not because I am cheap, I simply believe that more money thrown at something doesn't necessarily make for a better product.
2. reliability. You can't have something like this and have it quit on you two days in.
3. simplicity. I don't care for hard to configure software.
4. low power. Sure you can do this with a desktop machine, but that will likely consume 100 Watts. The Pi? 3 Watts.
5. low skill set for DIY.

I received by Model B 512 Mb in mid October 2012. I installed the Raspbian Wheezy 2012-09-18 distro of Linux with a minimum amount of trouble. I was truly amazed that after installing omxplayer, it would actually play H.264 .MOV video files from my Canon T2i camera unconverted, something my 3-year-old MSI notebook running Windows cannot do! But I digress...
Note that I am new to Linux and Python, having spent 30 years with DOS, Windows, dBase, FoxPro and Visual FoxPro. Be forewarned: old habits break hard...
Having previously experimented with 1 wire sensors, I decided to research this feature on the Pi. It turns out you can use basically one of two approaches: either use a bus master and hook this device up to a USB port on the Pi to control the sensors, or, forego the bus master and hook up the sensors straight to the GPIO pins of the Pi. Design parameter 1 (low cost) made me look first at using the GPIO pins. So, I ordered some DS18B20 sensors from element-14 (cost: about $ 4.00 each), waited a few days and started experimenting. Surprisingly enough, support for 1 wire sensors is built in to the Linux kernel. A good, basic, tutorial can be found here: http://www.cl.cam.ac.uk/freshers/raspberrypi/tutorials/temperature/index.html on the University of Cambridge website. If you have Raspbian version 2012-09-18 or later installed, all you need to do is hook up the sensor this way:

and then issue at the prompt
sudo modprobe w1-gpio
sudo modprobe w1-therm
cd /sys/bus/w1/devices/
ls

This should give you a listing of the attached 1 wire sensors the operating system found. The series of letters/numbers at the end signify the serial number of the sensor(s). Next, you can issue a change directory command followed by the serial number of a sensor, e.g. cd /sys/bus/w1/devices/A1-00000000000123. Then issue: cat w1_slave. The response you should get should be 2 lines. The first line will end in either YES or NO. If it is YES, then that signifies that the Cyclical Redundancy Check (CRC) that the Pi (through the operating system) performed was correct and the temperature reading that ends the second line as e.g 21125 is correct. Now, you need to divide the temperature reading by 1000 in order to find the correct temperature, in this case 21.125 degrees Celsius. If you get a NO response, the temperature reading is unreliable and you should try again. In my experience, the CRC check fails about 5% of the time, but is virtually always successful the second time around. If your reading is 85000, then more than likely you have reversed the leads of the DS18B20. Simply flip around and try again. Note that we are working on all this outside of Python, straight at the Raspbian prompt.

Background: Just to step back and discuss this whole sensor reading affair, it is sometimes said that everything in Linux is a file. So are the temperature readings here. When we issue a 'cat w1_slave’ command, realistically we are asking the operating system to list the output of the file contained in that folder. What really happens though it that Linux says, 'Oh, they want the temperature reading' and then goes out and gets it. That's why there is a slight delay when you issue the 'cat' command. The actual reading is done using a concept called "bit-banging", which is not foolproof, hence the need to check to see if the CRC was correct. Temperature readings of the DS18B20 are also somewhat slow, you cannot use this to monitor millisecond by millisecond temperature changes for metallurgical processes for example.

I set up the sensor on a breadboard I had, along with the pullup 470k Ohm resistor. This resistor ensures that when there is no activity on the line, the DATA pin (Pin 2) goes high, i.e. to 3V3. Without that, ambiguous readings might be had. With the Raspberry Pi powered off, I hooked up the DATA pin to Pin 4 on the GPIO and power to Pin 1, ground to Pin 3. Then, turned on the Pi and allowed the boot process to complete.  Once I was successful in obtaining the first temperature reading, I added another sensor at the end of a 25 foot cable and tried to obtained a reading for 2 sensors. Success! With this behind me, I knew that the likelihood of 3 sensors working was greatly increased.

Next, I started working on the Python application. A word about my software development environment first. After the Raspberry Pi’s initial setup, I switched to using it remotely using Remote Desktop. The instructions in this blog posting are easy to follow. For Python editing, HTML editing etc I use Geany, which is an extremely capable editor. So, to sum up, I use my (Windows 7) laptop, plunk my ass on the couch, start Remote Desktop (which automatically puts you in the XDE environment (‘startx’)), then start Geany, then open up the various Python applications and HTML\text files I need to have open.

It quickly became evident that I wanted to record my temperatures, not just read and discard them. So I added the SQLite database into the mix. Fortunately, Python has built in support for this database.
The database has 3 main tables: sensordesc, sensors and events.

- The table sensordesc stores the serial number of the sensor, the short name of the sensor e.g. "6", used to minimize space used in the database, and the long description of the sensor, usually its location, e.g. "Generator Housing". When you physically connect another sensor and start the application again, the app will automatically register the new sensor in this table. It is then up to you to add the long description.
- The table sensors stores the actual readings and has the fields dataread, sensor and temperature along with a field called topofhour in which I store an 'X' if the temperature reading is at the top of the hour. This makes for faster query retrievals later on.
- The table events stores any SMS messages, date/time sent, along with the contents of the message. Its primary use is to prevent duplicate messages from being sent out.

Background: SQLite is the little guy when it comes to SQL databases. The big guns are Oracle, Microsoft SQL Server and others. Then there are a host of databases holding the middle ground. Lastly, there are the smaller versions such as MySQL and SmallSql. Like most of the small guys, SQLite is free. Essentially a flat file, SQLite doesn't employ a server type application to serve up results requested by another application. Instead, the requesting application queries the database itself through a module that has to be built into that application. SQLite's footprint is very small, but is is surprising sophisticated in what it can do. By planning your database before you create it and by creating the right indices, you can significantly speed up query results. To query and work with the database, I use the SQLite Manager, which is an add-on to the Firefox browser. It allows you to manage any SQLite database on your PC or network. After you install it you can find it under the FireFox Tools menu.

Now, as I stated earlier, I am by no means a seasoned Python programmer. My programming learning style is that of the lazy variety: do what you have to do to make it work and move on. I do my last name proud by hacking to my heart's content, sometimes I think that hacking is the only thing I am really good at.

Here's the core of the Python program: surprisingly, it is less than 200 lines of code. First off, it establishes the environment by loading add-on modules and the Linux kernel 1 wire drivers. Then it queries the operating system for the number of sensors that are attached. It checks the serial numbers found against its own database and add any new sensors to it. Then it goes into a do while loop, turning on the red LED, reading the sensors every minute and logging the results. At the end of each loop it looks for the sensor "Generator Housing". It notes its current temperature and does a query in the database to see what it was 5 minutes ago. If the difference is less than 5 degrees Celsius, nothing happens. If it is over 5 degrees celsius, it checks to see if it has already sent out an SMS/email combination in the last 2 hours, telling me the generator has started. If it hasn’t, the SMS/email combination will be sent out.

   1: #!/usr/bin/env python

   2:  

   3: import datetime

   4: import os

   5: import time

   6: import wiringpi

   7: import sqlite3

   8: import sendsms2

   9: #from datetime import datetime

  10: #from datetime import timedelta

  11:  

  12: conn = sqlite3.connect('templog.db')

  13: conn.text_factory = str

  14: cur = conn.cursor()

  15:  

  16: def loadDrivers():

  17:     os.system("sudo modprobe w1-gpio")

  18:     os.system("sudo modprobe w1-therm")

  19:  

  20: def findNumberOfSensors():

  21:     #Find the number of sensors currently hooked up, that the system actually found.

  22:     tSlaveCountFile = open("/sys/bus/w1/devices/w1_bus_master1/w1_master_slave_count")

  23:     nSlaveCount = tSlaveCountFile.read()

  24:     tSlaveCountFile.close()

  25:     nSlaveCount = nSlaveCount.lstrip()

  26:     nSlaveCount = int(nSlaveCount.rstrip())

  27:     print "Number of sensors found: " + str(nSlaveCount)

  28:     return nSlaveCount

  29:     

  30: def findSlaves(nSlaveCount):    

  31:     tSlaveNameFile = open("/sys/bus/w1/devices/w1_bus_master1/w1_master_slaves")

  32:     cSlaveNames = tSlaveNameFile.read()

  33:     tSlaveNameFile.close()

  34:     #List apparently start with 0

  35:     lSlave = list()

  36:     count = 0

  37:     while (count < nSlaveCount ):

  38:         lSlave.append(cSlaveNames[:16].rstrip('\n'))

  39:         cSlaveNames = cSlaveNames[16:]

  40:         print "Sensor " + str(count + 1) + ": " + lSlave[count]

  41:         #Check to see if the sensor is already present in the sensorcodes table.

  42:         #If not, insert it.

  43:         cFindSensorSQL = "select * from sensorcodes where sensor_long = '" + lSlave[count] + "'"

  44:         cur.execute(cFindSensorSQL)

  45:         data = cur.fetchall()

  46:         if (len(data) > 1):

  47:             print "Error, more than 1 sensor with the same name encountered"

  48:             return

  49:         if (len(data) == 0):

  50:             #New sensor encountered, needs to be entered in the database

  51:             #Sensor_short is self incrementing

  52:             cur.execute('''INSERT INTO Sensorcodes(sensor_long) VALUES(?)''',[lSlave[count]])

  53:             conn.commit()

  54:             

  55:         cSql = cur.execute("SELECT sensor_short,sensor_desc from Sensorcodes where sensor_long = (?)",[lSlave[count]])

  56:         data = cur.fetchone()

  57:         cSensorShort = str(data[0])

  58:         lSlave[count] = lSlave[count] + "~" + cSensorShort + "~" + str(data[1])

  59:             

  60:         count = count + 1

  61:         

  62:     return lSlave

  63:     

  64: def checkTemps(nSlaveCount,lSlave):    

  65:     count = 0

  66:     cTime = datetime.datetime.now()

  67:     #replace whatever seconds we have with 0, to make database queries easier

  68:     cCurrentTime = str(datetime.datetime(cTime.year,cTime.month,cTime.day,cTime.hour,cTime.minute,0))[:19]

  69:     while (count < nSlaveCount ):

  70:         cCurrentSensor = lSlave[count]

  71:         cCurrentSensorLong = cCurrentSensor[:15]

  72:         nRightMostPoundSign = cCurrentSensor.rfind("~")

  73:         cCurrentSensorShort = cCurrentSensor[16:nRightMostPoundSign]

  74:         cCurrentSensorDesc = cCurrentSensor[nRightMostPoundSign + 1:]

  75:         tSlaveFile = "/sys/bus/w1/devices/" + cCurrentSensorLong + "/w1_slave"

  76:         nAttemptsToRead = 0

  77:         while True:

  78:             cTopOfHour = ""

  79:             tfile = open(tSlaveFile)

  80:             # Read all of the text in the file.

  81:             text = tfile.read()

  82:             # Close the file now that the text has been read.

  83:             tfile.close()

  84:             # Split the text with new lines (\n).

  85:             firstline = text.split("\n")[0]

  86:             cCyclicalRedudancyCheck = firstline.split(" ")[11]

  87:             if (cCyclicalRedudancyCheck == "NO"):

  88:                 nAttemptsToRead = nAttemptsToRead + 1

  89:                 if (nAttemptsToRead > 10):

  90:                     temperature = 99.999

  91:                     print "More than 10 attempts to re-read, abandoning until next cycle..."

  92:                     break

  93:                 print "CRC of file incorrect for sensor " + cCurrentSensorShort + " " + cCurrentSensorDesc + ", re-reading"

  94:             else:    

  95:                 secondline = text.split("\n")[1]

  96:                 # Split the line into words, referring to the spaces, and select the 10th word (counting from 0).

  97:                 temperaturedata = secondline.split(" ")[9]

  98:                 # The first two characters are "t=", so get rid of those and convert the temperature from a string to a number.

  99:                 temperature = float(temperaturedata[2:])

 100:                 # Put the decimal point in the right place and display it.

 101:                 temperature = temperature / 1000

 102:                 if (temperature == 85):

 103:                     print "Incorrect temperature reading of 85.0 degrees, re-reading"

 104:                 else:    

 105:                     break

 106:  

 107:         print cCurrentTime + " Temperature for sensor " + cCurrentSensorShort + " " + cCurrentSensorDesc + " = " + str(temperature)

 108:         if (cCurrentSensorDesc.upper() == "AMBIENT"):

 109:             cAmbientTemperature = str(temperature)

 110:             if (cCurrentTime[13:] == ":00:00"): #top of the hour

 111:                 cTopOfHour = "X"

 112:             

 113:         cur.execute('''INSERT INTO Sensors(dateread,sensorshort,sensorvalue,topofhour) VALUES(?,?,?,?)''',(cCurrentTime,cCurrentSensorShort,temperature,cTopOfHour))

 114:         if (cCurrentSensorDesc.upper() == "GENERATOR HOUSING"):

 115:             checkGeneratorStatus(cCurrentSensorShort,temperature,cCurrentTime,cAmbientTemperature)

 116:         count = count + 1

 117:         

 118:     while True:

 119:         try:

 120:             conn.commit()

 121:             break

 122:         except sqlite3.OperationalError:

 123:             print "Database is locked, retrying commit."

 124:             time.sleep(2)

 125:             

 126: def checkGeneratorStatus(cSensor,temperature,cDtTime,cAmbient):

 127:     cSql = cur.execute("SELECT dateread,sensorshort,sensorvalue from Sensors where sensorshort = (?) order by dateread desc limit 5",cSensor)

 128:     data = cur.fetchall()

 129:     nOldGeneratorHousingTemperature = float(data[4][2])

 130:     nTempDiff = temperature - nOldGeneratorHousingTemperature

 131:     if (nTempDiff > 5):

 132:         cur.execute("select datetm,eventtype from events order by datetm desc limit 1")

 133:         data = cur.fetchone()

 134:         dSMSLastSent = datetime.datetime.strptime(data[0],"%Y-%m-%d %H:%M:%S")

 135:         dOneHourAgo = datetime.datetime.now() - datetime.timedelta(hours=1)

 136:         if (dOneHourAgo < dSMSLastSent):

 137:             return

 138:         cMessage = "Temp diff detected. Current Gen Temp: " + str(temperature) + "\r\n. Prev Gen Temp: " + str(nOldGeneratorHousingTemperature) + ".\r\n Ambient Temp: " + cAmbient + ".\r\n Dt time: " + cDtTime

 139:         cRecipient = "5195552345"

 140:         carrier = "yourisp.com"

 141:         cSMSSent = sendsms2.sms("SMS",cMessage,cRecipient,carrier)     

 142:         if (cSMSSent[:7] != "Problem"): #if sms has recently been sent, or there was another problem, don't send email.

 143:             cur.execute('''INSERT INTO events(datetm, eventtype,description,recipient) VALUES(?,?,?,?)''',(cDtTime,"SMS",cMessage,cRecipient))

 144:             conn.commit()

 145:             sendsms2.sms("EMAIL",cMessage,"","")

 146:             cur.execute('''INSERT INTO events(datetm, eventtype,description,recipient) VALUES(?,?,?,?)''',(cDtTime,"EMAIL",cMessage,"yourname@gmail.com"))

 147:             conn.commit()

 148:         else:

 149:             cMessage = "Unable to send message. The sendsms module reports: " + cSMSSent

 150:             cur.execute('''INSERT INTO events(datetm, eventtype,description,recipient) VALUES(?,?,?,?)''',(cDtTime,"SMS",cMessage,"Unable to send"))

 151:             conn.commit()

 152:     

 153:  

 154:  

 155:  

 156:             

 157: def main():

 158:     os.system("gpio export 18 out")

 159:     os.system("gpio export 24 out")

 160:     wiringpi.wiringPiSetupSys()

 161:     wiringpi.GPIO(wiringpi.GPIO.WPI_MODE_SYS)

 162:     wiringpi.pinMode(18,wiringpi.OUTPUT)

 163:     wiringpi.pinMode(24,wiringpi.OUTPUT)

 164:     loadDrivers()

 165:     nSlaveCount = findNumberOfSensors()

 166:     lSlave = findSlaves(nSlaveCount)

 167:     cAmbientTemperature = "N/A"

 168:     while True:

 169:         wiringpi.digitalWrite(18,1)

 170:         checkTemps(nSlaveCount,lSlave)

 171:         wiringpi.digitalWrite(18,0)

 172:         nCurrentSecond = datetime.datetime.now().second

 173:         time.sleep(60-nCurrentSecond)

 174: main()

to be continued…