Friday, January 11, 2013

Keeping tabs on my standby generator with Raspberry Pi (Part III)

This part will describe the web server portion of the project. As I stated in a previous post, I am using bottle.py as my web serving agent.

Officially, according to http://bottlepy.org, bottle.py is known as a fast, simple and lightweight WSGI micro web-framework for Python. It is distributed as a single file module and has no dependencies other than the Python Standard Library. WSGI stands for Web Server Gateway Interface. According to Wikipedia, it defines a simple and universal interface between web servers and web applications or frameworks for the Python programming language.

In other words, you don’t really need anything beyond Python to take advantage of publishing things to the web. Another thing I really like about it is that its footprint is extremely small: 47kB, yes, that’s kilo not mega.

Perhaps it isn’t quite robust enough to be able to handle hundreds of requests per second, but for my purposes, it is perfect.

The simplest way to get started is download and install bottle.py, create the smallest of Python programs:

   1: from bottle import route, run, template
   2:  
   3: @route('/hello/:name')
   4: def index(name='World'):
   5:     return template('<b>Hello {{name}}</b>!', name=name)
   6:  
   7: run(host='localhost', port=8080)


Save it, call it whatever you like and run it. It shouldn’t do anything. Now open up a browser window and type ‘http://localhost:8080/hello/world’. That’s it. A running web application.


I did the above, got it working and then started expanding on the application. I created a method for retrieving the temperatures for the last half hour of logging. I also wrote a method that creates a graph of this data using gnuplot. Gnuplot is a somewhat quirky, but very powerful and fast graphing application. Did I mention it is free? Finally, the last method I wrote gathers some relevant system data such as CPU temperature, CPU utilization etc.


Then I created a template web page to host this data. This template contains place holders which will be replaced by the relevant data once a web request comes in. Nothing more than an old fashioned mail merge really.


This template is actually fairly advanced in that it uses jquery. Not wanting to overload my little Pi with download requests every time the page was requested, I decided to use Google’s library storage facility (ajax.googleapis.com/ajax/libs/jquery/1.6.4/jquery.min.js), which will easy the burden on the Pi. Same with the graphics: they are stored on Google Picasa, for lack of a better place.


Data retrieval is done using SQLite. I optimized the database by adding several indices, which makes it fairly snappy. Overall response time is about 2 seconds, from the time I issue the request (provided I have a fairly speedy internet connection, wherever I am in the world), to the time the screen is populated. Not bad.


Here’s a screen. Notice that this screen captured occurred shortly after a generator startup, hence the curve.


temperatures20121217am


Here’s a shot of the overall web page:


mainpage


Here’s the bottle code. In an earlier post, I stated that I added some LEDs as indicator lights to the Pi to indicate when it was running headless that a request from the web was coming in. Hence I added the library wiringpi to allow this to happen.



   1: from bottle import route, run, template, get, post, request, static_file
   2: import sqlite3
   3: import os
   4: import wiringpi
   5: import sendsms2
   6: import datetime
   7: from datetime import timedelta
   8:  
   9: @route('/hello/<name>')
  10: def index(name='World'):
  11:     return gettemps()
  12:  
  13: @route('/main')
  14: def test(name='World'):
  15:     wiringpi.digitalWrite(24,1)
  16:     cOutput = createOutput("")
  17:     wiringpi.digitalWrite(24,0)
  18:     return cOutput
  19:     
  20:     
  21: def getTemps():
  22:     now = datetime.datetime.now()
  23:     cStartTime = now - timedelta(hours=1)
  24:     #cur.execute('select sensors.*,sensorcodes.sensor_short,sensor_desc from sensors,sensorcodes where sensors.sensorshort = sensorcodes.sensor_short order by dateread desc LIMIT 30')
  25:     rows = cur.execute('select dateread,sum(case when sensorshort="5" then sensorvalue end) sensor_5,sum(case when sensorshort="6" then sensorvalue end) sensor_6,sum(case when sensorshort="7" then sensorvalue end) sensor_7 from sensors where dateread > "' + str(cStartTime)[:19] + '" group by dateread order by dateread desc limit 30')
  26:     data = cur.fetchall()
  27:     cOutput = ""
  28:     for x in range(0,len(data)):
  29:         #Returning dateread, temperature and sensor location/description
  30:         cOutput = cOutput + '#tr##td#' + ''.join(str(data[x][0])) + '#/td##td#' + ''.join(str(data[x][1])) +  '#/td##td#' + ''.join(str(data[x][2])) +  '#/td##td#' + ''.join(str(data[x][3])) + '#/td##/tr#'
  31:     return cOutput    
  32:  
  33: @get('/login') # or @route('/login')
  34: def login_form():
  35:     return '''<form method="POST" action="/login">
  36:                 <input name="name"     type="text" />
  37:                 <input name="password" type="password" />
  38:                 <input type="submit" />
  39:               </form>'''
  40:     
  41: @post('/manage') # or @route('/manage', method='POST')
  42: def manage_submit():
  43:     smstext = request.forms.get('smstxt') 
  44:     smstext = smstext + '\nSent from my Raspberry Pi'
  45:     number = request.forms.get('smsnumber')
  46:     carrier = request.forms.get('carrier')
  47:     if (len(carrier) == 0):
  48:         carrier = "txt.bellmobility.ca"
  49:     
  50:     cSMSSent = sendsms2.sms("SMS",smstext,number,carrier)     
  51:     cOutput = createOutput(cSMSSent)
  52:     return cOutput
  53:     
  54:     
  55: @get('/<filename:re:.*\.(jpg|png|gif|ico)>')
  56: def server_static(filename):
  57:     return static_file(filename, root="./images")
  58:     
  59: @get('/<filename:re:.*\.(js)>')
  60: def server_static(filename):
  61:     return static_file(filename, root="./js")    
  62:     
  63: @get('favicon.ico')
  64: def fav_icon():
  65:     return static_file("favicon.ico", root="./images")
  66:  
  67:     
  68: # Return CPU temperature as a character string                                      
  69: def getCPUtemperature():
  70:     res = os.popen('vcgencmd measure_temp').readline()
  71:     return(res.replace("temp=","").replace("'C\n",""))
  72:  
  73: # Return RAM information (unit=kb) in a list                                        
  74: # Index 0: total RAM                                                                
  75: # Index 1: used RAM                                                                 
  76: # Index 2: free RAM                                                                 
  77: def getRAMinfo():
  78:     p = os.popen('free')
  79:     i = 0
  80:     while 1:
  81:         i = i + 1
  82:         line = p.readline()
  83:         if i==2:
  84:             return(line.split()[1:4])
  85:  
  86: # Return % of CPU used by user as a character string                                
  87: def getCPUuse():
  88:     return(str(os.popen("top -n1 | awk '/Cpu\(s\):/ {print $2}'").readline().strip(\
  89: )))
  90:  
  91: # Return information about disk space as a list (unit included)                     
  92: # Index 0: total disk space                                                         
  93: # Index 1: used disk space                                                          
  94: # Index 2: remaining disk space                                                     
  95: # Index 3: percentage of disk used                                                  
  96: def getDiskSpace():
  97:     p = os.popen("df -h /")
  98:     i = 0
  99:     while 1:
 100:         i = i +1
 101:         line = p.readline()
 102:         if i==2:
 103:             return(line.split()[1:5])
 104:             
 105: def gnuplot():
 106:     now = datetime.datetime.now()
 107:     cStartTime = now - timedelta(hours=1)
 108:     rows = cur.execute('select dateread,sum(case when sensorshort="5" then sensorvalue end) sensor_5,sum(case when sensorshort="6" then sensorvalue end) sensor_6,sum(case when sensorshort="7" then sensorvalue end) sensor_7 from sensors where dateread > "' + str(cStartTime)[:19] + '" group by dateread order by dateread desc limit 30')
 109:     data = cur.fetchall()
 110:     cOutput = ""
 111:     for x in range(0,len(data)):
 112:         #Returning dateread, temperature and sensor location/description
 113:         cOutput = cOutput +  ' ' + ''.join(str(data[x][0])) + ' ' + ''.join(str(data[x][1])) +  '  ' + ''.join(str(data[x][2])) + '  ' + ''.join(str(data[x][3])) + "\n"    
 114:     cDatafile = open("images/graphdata.txt","w")
 115:     cDatafile.write(cOutput)
 116:     cDatafile.close()
 117:     cPlot= os.system("gnuplot gnuplotconfig.txt")
 118:     return True
 119:     
 120: def createOutput(cSMSSent):
 121:     # CPU informatiom
 122:     CPU_temp = getCPUtemperature()
 123:     CPU_usage = getCPUuse()
 124:     # RAM information
 125:     # Output is in kb, here I convert it in Mb for readability
 126:     RAM_stats = getRAMinfo()
 127:     RAM_total = round(int(RAM_stats[0]) / 1000,1)
 128:     RAM_used = round(int(RAM_stats[1]) / 1000,1)
 129:     RAM_free = round(int(RAM_stats[2]) / 1000,1)
 130:  
 131:     # Disk information
 132:     DISK_stats = getDiskSpace()
 133:     DISK_total = DISK_stats[0]
 134:     DISK_used = DISK_stats[1]
 135:     DISK_perc = DISK_stats[3]
 136:     cSystemInfo = "CPU Temperature = " + CPU_temp + "#BR#" + "CPU Usage = " + CPU_usage + "#BR#" + "RAM Total = " + str(RAM_total) + "#BR#"
 137:     cSystemInfo = cSystemInfo + "RAM Used = " + str(RAM_used) + "#BR#" + "RAM Free = " + str(RAM_free) + "#BR#" + "Disk Total = " + DISK_total + "#BR#"
 138:     cSystemInfo = cSystemInfo + "Disk Used = " + str(DISK_used) + "#BR#" + "Disk Percentage Used = " + str(DISK_perc) + "#BR#"
 139:     
 140:     cOutput=template('index1',cData=getTemps(),cSystemInfo=cSystemInfo,cSMSSent=cSMSSent)
 141:     cOutput = cOutput.replace("#BR#","<BR>")
 142:     cOutput = cOutput.replace("#td#","<td>")
 143:     cOutput = cOutput.replace("#/td#","</td>")
 144:     cOutput = cOutput.replace("#tr#","<tr>")
 145:     cOutput = cOutput.replace("#/tr#","</tr>")
 146:     
 147:     gnuplot()
 148:     return cOutput
 149:             
 150:             
 151:                 
 152: wiringpi.wiringPiSetupSys()
 153: wiringpi.GPIO(wiringpi.GPIO.WPI_MODE_SYS)
 154: wiringpi.pinMode(24,wiringpi.OUTPUT)
 155: conn = sqlite3.connect('templog.db')
 156: conn.text_factory = str
 157: cur = conn.cursor()
 158:  
 159:  
 160:  
 161: run(host='192.168.0.150', port=8080)

 

No comments: