Datalogging with Arduino

There are several ways to save data from a sensor attached to an Arduino. If you’re connected to a personal computer, you can simply send the data from the Arduino to the personal computer serially, and save it to a file. If you’ve got an SD card attached to the microcontroller, you can save the data to the card. Or, if you have access to the internet and a device that can connect to a server, you can save the data to a server. In the tutorial below, you’ll read a DHT11 temperature and humidity sensor and log data in three ways:

  • Serial transmission to a personal computer, and serial capture to a file.
  • Saving data to an SD card mounted on the Arduino
  • HTTP upload to pachube.com via an Ethernet shield or Ethernet Arduino.

Hardware you need:

  • personal computer
  • Ethernet Arduino and USB-to-serial connector, or Arduino Uno and Ethernet shield (or equivalents)
  • SD Micro card
  • DHT11 temperature and humidity sensor
  • 10-kilohm resistor

Software you need:

Concepts you should know:

  • Basic understanding of Arduino code
  • Basic electrical concepts
  • Serial communication concepts
  • How to install libraries in Arduino
  • HTTP request concepts

For more on the DHT sensors, see Adafruit’s tutorial.

To get started, install the Arduino IDE and download the DHT library. Unzip the library and change the directory name to DHT, then copy it to the libraries/ directory of your Arduino sketch directory. If this is your first time using Arduino, the default location is in your user directory, called Arduino/ . You might have to create the libraries directory inside the sketch directory. Then download the example sketches from my gitHub repository, unzip them and save them to your sketch directory as well. Then start the Arduino IDE.

Connecting the sensor

Connect the DHT11 sensor to the Arduino as follows:

  • Vcc (pin 1) – Digital pin 8
  • Output (pin 2) – Digital pin 7
  • Ground (pin 4) – Digital pin 5

Since the DHT11 sensor uses very little current, you can use the output pins of the microcontoller to power the sensor, using pins 5 and 8. Then attach the output pin of the sensor to pin 7. The output pin will also need a pullup resistor so that it goes high when no data is transmitted. To do this, connect the 10-kilohm resistor from pin 8 to pin 7, or to the +5V pin.

Serial Data Capture to a Graph

The simplest way to get data off the sensor is to print the results out serially and capture them directly on your computer. To make this happen, you’ll need your Arduino attached via a USB-to-serial connection. The sketch SerialTempHumidityReader uses the Adafruit DHT library to read the sensor and sends the results back to the computer serially. Download it and copy it into a new window in the Arduino IDE.

To upload the sketch to your board, connect the board to the computer via USB-to-serial, and look for the board type in the Tools > Board menu. Assuming you’re using an Arduino Ethernet, choose that:

To find your serial port, check the Tools >Serial Port menu with the USB-to-serial adapter NOT connected. Then plug the adapter into your computer again and check the menu again.  The new port that shows up is your USB-to-serial adapter’s port.

Once you’ve selected the board type and the port, upload the sketch by clicking the upload button in the toolbar, or ctrl-U:

When you’ve got the sketch loaded, open the serial monitor by clicking the Serial Monitor button on the right hand side of the toolbar.

You should see readings like this:

The sketch is outputting tab-separated data, and prints out column headers at the beginning.

When you’ve got several readings, click in the serial monitor window, select all (ctrl-A), and copy (ctrl-C). Then open a blank spreadsheet in OpenOffice and paste the results into the sheet. When the Text Import dialog comes up, make sure you click Separated by Tab in the Separator options:

Select the two columns of numbers, and click the graph icon. In the Graph dialog box, choose Line type, the Lines Only option, then click Finish. Your graph will be inserted in your spreadsheet. Voila, graphing made simple!

Serial Data Capture to a File

If you have a serial terminal program like CoolTerm on your computer, you can also capture to a file. To do so, open CoolTerm, and choose your serial port in the Options menu. Click the Connect icon, then from the Connection Menu, choose Capture to TextFile… and Start. Give your file a name and save.

Note: if you want the file to start from the beginning of your sketch, hold down the reset until you’ve started capture.

To stop capture, choose Connect > Capture to TextFile… > Stop. Then you can use your file in any application you want. Change the file extension to .csv and you can open it in a spreadsheet and graph as above.

Saving Data to an SD Card

Sometimes you want to save data when you’re not connected to a personal computer. Attaching an SD card to an Arduino is fairly straightforward. There are several different shields that have SD cards on board, and the Arduino Ethernet has an SD card right on the main board. The SD card library makes it simple to save files to your SD card.

Format a microSD card as FAT16 or FAT32 on your personal computer, and load it onto your Arduino. Then upload the SDCardDataLogger sketch. This sketch reads the sensor using the Adafuit DHT library, and if there is an SD card present and initialized, it saves the results to a file called “DATALOG.CSV”. Transfer this file to your computer and use it as you wish. You can open it in a spreadsheet and graph it as you did above, or anything else you wish.

Posting data to Pachube.com

If your Arduino is connected to the internet via Ethernet, you can connect to pachube.com and program it to post data there. Then you can use Pachube’s data storage and graphing tools to save and visualize your data.

Set up a pachube.com account. Click Create a Feed. Give it a name, add any descriptive tags you want, and add a location if you wish. Add two datastreams called rH (relative humidity) and temp (temperature) as follows:

Write down the feed number and fill it into the feed variable in the sketch (PachubeDataLogger). You’ll also need your API key, which is a long string that identifies you to pachube.com. Click My Keys from the pachube menu to get your key. Copy it into the apiKey variable in the sketch below.

Upload this sketch to your Arduino, and connect the board to the net via Ethernet. Open the serial monitor. When the sketch starts, it will look for an IP address using DHCP. Then every ten seconds, it will read the sensor and try to upload to Pachube. A successful upload will print out something like this:

192.168.2.6
rH,26.00
temp,30.00
connecting...
data uploaded
HTTP/1.1 200 OK
Date: Sat, 16 Feb 2012 10:03:14 GMT
Content-Type: text/plain; charset=utf-8
Connection: close
X-Pachube-Logging-Key: logging.G5DkQtnxEdwyKbk0KSKI
X-PachubeRequestId: 0b4efe30282b715276cf1f849f7123ed3eaba234
Cache-Control: max-age=0
Content-Length: 1
Age: 0
Vary: Accept-Encoding

When you look at your feed, you’ll see a graph of your temperature and humidity.

Now you’ve got several methods to log data from an Arduino. You can change the sensor to any sensor you wish, and revise the sketches shown here to read that sensor instead of the DHT11. Enjoy!

Update

I’ve added code to the repository for this tutorial that shows how to use the Pachube code with a BMP085 barometric pressure sensor. Thanks to Adafruit for the excellent library for this sensor.  I’ve also added an example showing how to use the BMP085 with the SD card, and with a Realtime Clock. Again, Adafruit’s library for the RTC makes this easy.

 

Posted in arduino/wiring, circuits, code, electronics, Pachube | 2 Comments

Glif + DSO Nano: two great tastes…

I was recently given a Glif, and even though I don’t own an iPhone 4, the Glif has been very useful. It it’s a perfect tripod mount for my DSO Nano oscilloscope, and makes it easier to prop the scope up on my workbench like so:

Thanks, Studio Neat!

Posted in circuits, construction, electronics, iPhone, misc | Leave a comment

SQL RESTian example in PHP

Here’s a PHP script that reads and writes from a SQL database. This example assumes you’ve got a SQL database account on the same machine that the script is running on. It also assumes you’re using a .htaccess file that looks something like this:

RewriteEngine On
RewriteBase /directoryname/
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^.*$ sqlRestDatalog.php

Continue reading

Posted in code, PHP | Tagged | Leave a comment

Mr. Data Converter

Mr. Data Converter is your friend. It’s a conversion tool to take tab or comma delimited data and convert it to a variety of formats: XML, JSON, ActionScript, PHP, you name it. Mighty handy if you have to take stuff from one place and move it to another.

Thanks to James Tu for the link.

Posted in code, Flash/ActionScript, java, PHP, Processing | Leave a comment

Networked Bed Presence sensor

John Schimmel and I had a meeting with a friend the other day who has chronic condition, and is looking for tools to notify her family and friends of her activity at home, so she can live more independently. A lot of good ideas came up, making for a potentially exciting project that’d be useful someone we care about: a win-win situation.

One of the ideas she had was a bed sensor, which would let her family know whether she’s in bed or not, so they can call to either wake her up, or tell her to get some rest if she’s up too late. Inspired by Mustafa Bagdatli and Diego Rioja’s BedData project, and assisted by them, we thought we’d see if we could put something together quickly, connecting force sensing resistors to an Arduino and using an Ethernet shield to upload data to Pachube.  The project has not been tested in the field yet, but it’s working in the lab, and was simple to set up.  It may be a useful example for other sensor projects.

Continue reading

Posted in arduino/wiring, circuits, electronics | Tagged , , | Leave a comment

Serial ports on an iPhone?

hc gilje posted some useful notes on accessing the serial ports of an iPhone. Bottom line:

Apple has not made it easy to let the iphone communicate with external devices. Basically, you need a jailbroken phone to do anything.

That said, his notes are useful for anyone who’s really really got to get at the serial port.

Other ways of getting sensors into the iPhone include Tellart’s NADAMobile, which uses the microphone input as an analog input, and Brian Jepson’s PhoneGap solution, which uses ethernet and an ethernet shield.

Posted in arduino/wiring, iPhone | Tagged , | Leave a comment

Using PepperMill to turn a motor into a sensor

Nicolas Villar sent me a sample of the PepperMill, a new sensor board he and Steve Hodges designed at Microsoft Research in Cambridge, UK.  It’s a nifty little board.  You attach a DC motor and the board can an output voltage when the motor is turned,  and analog signals telling you the direction and speed of the motor.  It turns a DC motor into a rotary encoder, of sorts.

Continue reading

Posted in arduino/wiring, circuits, electronics | 3 Comments

Rob Faludi just turned me on to CoolTerm, a serial terminal app for OSX and Windows. It looks like a nice addition, and is cross platform, which is handy for teaching; no need to reference different apps on different platforms. I’ve only tried it out a little, but so far, it’s nice. You can have multiple ports open in multiple windows, and you can see bytes in ASCII or hex. It’d be nice to see them in decimal too, but that’s just a wish.

Posted in electronics, OSX | Tagged , , | Leave a comment

Bluetooth Mate with Lilypad

Coming late to the game, I’m trying out a Bluetooth Mate from SparkFun with an Arduino Lilypad.Overall, it’s a nice Bluetooth radio to use with microcontrollers.  A few thoughts on getting it working:

Continue reading

Posted in arduino/wiring, circuits, electronics | Tagged , , | 1 Comment

Intervalometer and motion trigger for Nikon D90

Matt Mets has written a nice Nikon remote control library for Arduino. I used it to make a motion trigger and intervalometer. It looks like this:

Continue reading

Posted in arduino/wiring, circuits, electronics | Leave a comment