This suite of programs takes data from a sensor and saves it to a text file on a network. Each sensor reading is time stamped. The suite illustrates the basic principles involved in sending sensor data to a networked file or database.
\nThe first program involved is a microcontroller program, written in PicBasic Pro, tested on a PIC18F258. It waits for serial input from an external program. Then it reads its analog sensor, and sends the result out in two bytes.
\nThe second program is the same microcontroller code in Wiring\/Arduino, thanks to Jamie Allen for the cleanup.
\nThe third program involved is a desktop computer program, written in Processing. It requests data via its serial port from the microprocessor and sends that data to a CGI program on a web server. It passes the data into the CGI using an HTTP GET request. This program only sends every three seconds, so as not to overwhelm the server with hits.
\nThe fourth program is a CGI (common gareway interface) program, written in PHP. It takes in data from an HTTP GET request and appends it to a text file, along with the time the request was received. Note that this program does not check to see how big the file is, or whether the incoming data is properly formatted, so it isn’t terribly secure.
\nThe fifth program is another PHP script that logs the data to a mySQL database. Running this doesn’t require any change in the microcontroller code, but it does require a slight change in the Processing code. The change is in the sentToNet() method, and is noted below.<\/p>\n
<\/p>\n
Technorati Tags: networked objects<\/a>, networks<\/a><\/p>\n Wiring Arduino code, written and tested on an Arduino and Arduino NG:<\/p>\n Desktop computer code, written in Processing, tested on mac OSX:<\/p>\n CGI program, written in PHP, tested using an Apache web server running on a Redhat Linux machine:<\/p>\n This PHP script saves the sensor value to a mySQL database and timestamps it. It requires a change to the Processing code, which follows after the script:<\/p>\n Here’s the modified sendToNet() method for the Processing code above, to work with the mySQL PHP script:<\/p>\n This suite of programs takes data from a sensor and saves it to a text file on a network. Each sensor reading is time stamped. The suite illustrates the basic principles involved in sending sensor data to a networked file or database.<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9,3,10,7],"tags":[],"class_list":["post-36","post","type-post","status-publish","format-standard","hentry","category-arduinowiring","category-PHP","category-picbasic-pro","category-Processing"],"_links":{"self":[{"href":"https:\/\/www.tigoe.com\/pcomp\/code\/wp-json\/wp\/v2\/posts\/36","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.tigoe.com\/pcomp\/code\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.tigoe.com\/pcomp\/code\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.tigoe.com\/pcomp\/code\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.tigoe.com\/pcomp\/code\/wp-json\/wp\/v2\/comments?post=36"}],"version-history":[{"count":2,"href":"https:\/\/www.tigoe.com\/pcomp\/code\/wp-json\/wp\/v2\/posts\/36\/revisions"}],"predecessor-version":[{"id":614,"href":"https:\/\/www.tigoe.com\/pcomp\/code\/wp-json\/wp\/v2\/posts\/36\/revisions\/614"}],"wp:attachment":[{"href":"https:\/\/www.tigoe.com\/pcomp\/code\/wp-json\/wp\/v2\/media?parent=36"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.tigoe.com\/pcomp\/code\/wp-json\/wp\/v2\/categories?post=36"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.tigoe.com\/pcomp\/code\/wp-json\/wp\/v2\/tags?post=36"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\n
\nMicrocontroller code, written in PicBasic Pro, tested on a PIC18F258:<\/p>\n\r\n' call and response serial example for picBasic Pro.\r\n' By Tom Igoe, 2003\r\n' updated 25 Oct. 2005\r\n\r\n' This example waits for a byte on the incoming serial connection,\r\n' It then sends the value of a sensor on RA0, as a two-byte value.\r\n\r\n' serial RX is on pin RC7\r\n' serial TX is on pin RC6\r\n\r\n ' Define ADCIN parameters\r\nDEFINE ADC_BITS 10 ' Set number of bits in result\r\nDEFINE ADC_CLOCK 3 ' Set clock source (3=rc)\r\nDEFINE ADC_SAMPLEUS 50 ' Set sampling time in uS\r\n\r\n' constant to set the baud rate:\r\ninv9600 con 16468\r\n\r\n' define variables:\r\nadcVar var word\r\ninByte var byte\r\n\r\nTRISA = %11111111 ' Set PORTA to all input\r\nADCON1 = %10000010 ' Set PORTA analog and right justify result\r\n main:\r\n\r\n ' read sensors, convert to bytes:\r\n adcin 0, adcVar\r\n \r\n ' read serial data in:\r\n serin2 portc.7, inv9600, [inByte]\r\n \r\n ' if you got the message from the remote device, send out data:\r\n serout2 portc.6, inv9600, [adcVar.lowByte, adcVar.highByte]\r\ngoto main\r\n<\/pre>\n
\r\n\r\n\/*\r\nAnalog Call-and-Response for 10 bit sensor readings\r\nby Tom Igoe\r\n\r\nCreated 26 Sept. 2005\r\nUpdated 30 May 2006\r\nCleaned up for use with the Network Data Logging Suite\r\nby Jamie Allen, Feb 2007\r\n\r\nWaits for serial input. If the incoming value\r\nis a valid byte (i.e. \"A\"), the program then\r\nreads two analog inputs and one digital input.\r\nIt slices up the data into Most and Least Significant\r\nBYTES and sends them out sequentially to the\r\nserial port\r\n\r\nArduino hardware connections:\r\nA0: analog sensor on analog in 0\r\n*\/\r\n\r\nint firstSensor = 0; \/\/ first analog sensor\r\nint inByte = 0; \/\/ incoming serial byte\r\nbyte firstSensorLSB; \/\/LSB for the analog sensor value\r\nbyte firstSensorMSB; \/\/MSB for the analog sensor value\r\n\r\nvoid setup()\r\n{\r\n \/\/ start serial port at 9600 bps:\r\n Serial.begin(9600);\r\n}\r\n\r\nvoid loop()\r\n{\r\n \/\/ if we get a valid byte, read analog ins:\r\n if (Serial.available() > 0) {\r\n \/\/ get incoming byte:\r\n inByte = Serial.read();\r\n\r\n if (inByte == 65) \/\/i.e.: it's an ASCII \"A\"\r\n {\r\n firstSensor = analogRead(0);\r\n firstSensorLSB = firstSensor;\r\n firstSensorMSB = firstSensor >> 8;\r\n\r\n Serial.print(firstSensorLSB, BYTE);\r\n Serial.print(firstSensorMSB, BYTE);\r\n }\r\n }\r\n}\r\n<\/pre>\n\r\n\/* datalogging client\r\n by Tom Igoe\r\n \r\n Communicates between a microcontroller reading a 10-bit analog sensor and\r\n a CGI script that timestamps the reading and writes it to a file.\r\n \r\n The program starts by sending to the sensor for an initial value.\r\n When a good sensor value is obtained, the program checks to see if there's \r\n a net send in progress. If there's not, it sends the sensor reading to the net\r\n by starting a network client that connects to a server on port 80,\r\n sends an HTTP 1.1 GET request, and prints the results. \r\n \r\n Once the sensor request is done, the client waits 3 seconds before the next send\r\n so as not to overwhelm the server.\r\n\r\n created 18 March 2005\r\n updated 24 Oct. 2005\r\n *\/\r\n \r\nimport processing.net.*;\r\nimport processing.serial.*;\r\n\r\nSerial port; \/\/ The serial port\r\nClient client; \/\/ the net client\r\nint[] serialInArray = new int[2]; \/\/ Where we'll put what we receive\r\nint serialCount = 0; \/\/ A count of how many bytes we receive\r\nint sensorValue = 0; \/\/ value of the sensor\r\nboolean firstContact = false; \/\/ whether we've heard from the microcontroller\r\nboolean netSendInProgress = false; \/\/ whether or not the last net request is finished\r\nboolean newSensorData = false; \/\/ whether or not we have new sensor data\r\n\r\n\r\n\r\nvoid setup()\r\n{\r\n size(200, 200);\r\n \/\/ Print a list of the serial ports, for debugging purposes:\r\n println(Serial.list());\r\n\r\n \/\/ I know that the first port in the serial list on my mac\r\n \/\/ is always my Keyspan adaptor, so I open Serial.list()[0].\r\n \/\/ On Windows machines, this generally opens COM1.\r\n \/\/ Open whatever port is the one you're using.\r\n port = new Serial(this, Serial.list()[0], 9600);\r\n port.write(65); \/\/ Send a capital A to start the microcontroller sending\r\n}\r\n\r\nvoid draw()\r\n{\r\n background(0);\r\n\r\n\/\/ if there's any serial data available, get it:\r\n if (port.available() > 0) {\r\n serialEvent();\r\n \/\/ Note that we heard from the microntroller at least once:\r\n firstContact = true;\r\n }\r\n \/\/ If there's no serial data, send again until we get some incoming data.\r\n \/\/ (in case you tend to start Processing before you start your \r\n \/\/ external device):\r\n if (firstContact == false) {\r\n delay(300);\r\n port.write(65);\r\n }\r\n\r\n \/\/ if we have new sensor data, check to see that there's no open\r\n \/\/ net connections. If there aren't, send the data.\r\n if (newSensorData) {\r\n if (!netSendInProgress) {\r\n sendToNet(sensorValue);\r\n }\r\n }\r\n\r\n\r\n \/\/ print the results of the net send:\r\n if (netSendInProgress) {\r\n if (client.available() > 0) {\r\n int inByte = client.read();\r\n print((char)inByte);\r\n \/\/ when we get a byte of value 0, it's the end of the response\r\n \/\/ from the server. Stop listening and get some more data:\r\n if (inByte == 0) {\r\n netSendInProgress = false;\r\n \/\/ don't overwhelm the server:\r\n delay(3000);\r\n \/\/ Send a capital A to request new sensor readings:\r\n port.write(65);\r\n }\r\n }\r\n }\r\n}\r\n\r\n\r\nvoid serialEvent() {\r\n \/\/ Add the latest byte from the serial port to array:\r\n serialInArray[serialCount] = port.read();\r\n serialCount++;\r\n \/\/ If we have 2 bytes, combine them into one value:\r\n if (serialCount > 1 ) {\r\n sensorValue = serialInArray[1] * 256 + serialInArray[0];\r\n newSensorData = true;\r\n \r\n \/\/ Reset serialCount:\r\n serialCount = 0;\r\n } \r\n else {\r\n \/\/ if we have only one byte, don't let the main loop \r\n \/\/ send out yet:\r\n newSensorData = false;\r\n }\r\n}\r\n\r\nvoid sendToNet(int sensorValue) {\r\n \/\/ open a TCP socket to the host:\r\n client = new Client(this, \"myserver.com\", 80);\r\n\r\n \/\/print the IP address of the host:\r\n println(client.ip());\r\n \/\/ send the HTTP GET request:\r\n client.write(\"GET \/~someaccount\/logger.php?tempValue=\" + sensorValue + \" HTTP\/1.1\\n\");\r\n client.write(\"HOST: myserver.com\\n\\n\");\r\n netSendInProgress = true;\r\n}\r\n<\/pre>\n\r\n<?php\r\n$filename = 'datalog.txt';\r\n\/\/ compose the data string from the date and the incoming value:\r\n $dataString = date(\"Y-m-d h:i:s\\t\");\r\n $dataString = $dataString.$_REQUEST['tempValue'];\r\n \/\/ add a linefeed and carriage return\r\n $dataString = $dataString.\"\\r\\n\";\r\n\r\n\/\/ make sure the file exists and is writable first:\r\nif (is_writable($filename)) {\r\n\r\n \/\/ Open $filename in append mode. anything you add \r\n \/\/ will be appended to the end of the file:\r\n if (!$handle = fopen($filename, 'a')) {\r\n echo \"Can't open file $filename\";\r\n exit;\r\n }\r\n\r\n \/\/ Write $dataString to the opened file.\r\n if (fwrite($handle, $dataString) == FALSE) {\r\n echo \"Can't write to file $filename\";\r\n exit;\r\n }\r\n \/\/ data successfully written:\r\n echo \"Wrote $dataString to file $filename\";\r\n \/\/ send a 0 to tell the remote side we're done:\r\n echo \"\\0\";\r\n \/\/ close the file\r\n fclose($handle);\r\n\r\n} else {\r\n echo \"The file $filename is not write-enabled.\";\r\n}\r\nend;\r\n?>\r\n<\/pre>\n\r\n<?php\r\n\/\/ get username & pwd info:\r\ninclude \"secret.php\";\r\n\r\n\/\/ initialize variables:\r\n$sensorValue = -1; \/\/ value from the sensor\r\n$date = -1; \/\/ date string: YYYY-MM-DD\r\n$time = -1; \/\/ time string: HH:mm:ss in 24-hour clock\r\n$recordNumber = -1; \/\/ which record to delete\r\n$list = 0; \/\/ whether or not to list results in HTML format\r\n$databaseName = 'mydatabaseName';\r\n$tableName = 'myTableName';\r\n\r\n\/\/ open the database:\r\n$link = open_database('localhost', $databaseName, $username, $password);\r\n\r\n\/\/ process all the HTTP Request variables:\r\nforeach ($_REQUEST as $key => $value)\r\n{ \r\n \/\/ action is the SQL action: insert, delete, select, etc.\r\n if ($key == \"action\") {\r\n $action = $value;\r\n }\r\n \/\/ sensorValue is the result from the remote sensor system\r\n if ($key == \"sensorValue\") {\r\n $sensorValue = $value;\r\n }\r\n \/\/ date that the sensor reading was taken\r\n if ($key == \"date\") {\r\n $date = $value; \r\n }\r\n \/\/ time that the sensor reading was taken\r\n if ($key == \"time\") { \r\n $time = $value;\r\n }\r\n \/\/ database record number (for deleting only):\r\n if ($key == \"recNum\") { \r\n $recordNumber = $value;\r\n }\r\n \/\/ whether or not to print out results in HTML:\r\n if ($key == \"list\") { \r\n $list = $value;\r\n }\r\n\r\n}\r\n\r\n\/\/ insert a new record in the database:\r\nif ($action == \"insert\") {\r\n \/\/ make sure date and time have values:\r\n if ($date == -1 || $time == -1) {\r\n \/\/ if not values, generate them from the server time\r\n \/\/ (I should probably properly check for valid date and time strings here):\r\n list($date, $time) = split(\" \", date(\"Y-m-d H:i:s\"));\r\n }\r\n\r\n \/\/ Only insert if we got a sensor value from the GET:\r\n if (sensorValue != -1) {\r\n insert_record($tableName, $sensorValue, $date, $time);\r\n }\r\n}\r\n\r\n\/\/ if we're supposed to delete, delete:\r\nif ($action == \"delete\") {\r\n \/\/ only delete if we got a record number from the GET:\r\n if ($recordNumber != -1) {\r\n delete_record($tableName, $recordNumber);\r\n }\r\n}\r\n\r\n\/\/ if we should list in HTML format, list the whole table:\r\nif ($list == 1) {\r\n echo \"<html><head><\/head><body>\";\r\n \/\/ browse the whole table:\r\n browse_table($tableName);\r\n echo \"<\/body><\/html>\";\r\n}\r\n\r\n\/\/ close the database:\r\nclose_database($link);\r\n\r\n\/\/ end with a 0 to close the session to the client:\r\necho \"\\0\";\r\nend;\r\n\r\n\/\/ Functions -------------------------------\r\n\r\n\/\/ Connect to a server and open a database:\r\nfunction open_database($myServer, $myDatabase, $myUser, $myPwd) {\r\n $myLink = mysql_connect($myServer, $myUser, $myPwd)\r\n or die('Could not connect: ' . mysql_error());\r\n if ($list == 1) {\r\n echo 'Connected successfully';\r\n }\r\n mysql_select_db($myDatabase) or die('Could not select database');\r\n return $myLink;\r\n}\r\n\r\n\/\/ close an open database:\r\nfunction close_database($myLink) {\r\n mysql_close($myLink);\r\n}\r\n\r\n\/\/ select all from a table:\r\nfunction browse_table($myTable) {\r\n $query = \"SELECT * FROM `$myTable`\";\r\n $result = mysql_query($query) or die('Query failed: ' . mysql_error());\r\n\r\n \/\/ Printing results in HTML\r\n echo \"<table>\\n\";\r\n while ($line = mysql_fetch_array($result, MYSQL_ASSOC)) {\r\n echo \"\\t<tr>\\n\";\r\n foreach ($line as $col_value) {\r\n echo \"\\t\\t<td>$col_value<\/td>\\n\";\r\n }\r\n echo \"\\t<\/tr>\\n\";\r\n }\r\n echo \"<\/table>\\n\";\r\n \/\/ Free resultset\r\n mysql_free_result($result);\r\n}\r\n\r\n\/\/ insert a new record in the table:\r\nfunction insert_record($myTable, $recValue, $recDate, $recTime) {\r\n $query = \"INSERT INTO `$myTable` (`Value`, `Date`, `Timestamp`) VALUES ('$recValue', '$recDate','$recTime')\";\r\n $result = mysql_query($query) or die('Query failed: ' . mysql_error());\r\n \/\/ Free resultset\r\n mysql_free_result($result);\r\n\r\n}\r\n\r\n\/\/ delete a record from the table:\r\nfunction delete_record($myTable, $recNum) {\r\n $query = \"DELETE FROM `$myTable` WHERE `ID` = $recNum LIMIT 1\";\r\n $result = mysql_query($query) or die('Query failed: ' . mysql_error());\r\n \/\/ Free resultset\r\n mysql_free_result($result);\r\n}\r\n\r\n?>\r\n<\/pre>\n\r\nvoid sendToNet(int sensorValue) {\r\n \/\/ open a TCP socket to the host:\r\n client = new Client(this, \"itp.nyu.edu\", 80);\r\n\r\n \/\/print the IP address of the host:\r\n println(client.ip());\r\n \/\/ send the HTTP GET request:\r\n String sensorString = Integer.toString(sensorValue);\r\n client.write(\"GET \/~myAccount\/sql_datalog.php?action=insert&sensorValue=\" + sensorValue + \" HTTP\/1.1\\n\");\r\n client.write(\"HOST: myserver.com\\n\\n\");\r\n netSendInProgress = true;\r\n}\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"