{"id":1109,"date":"2012-11-07T21:42:50","date_gmt":"2012-11-08T02:42:50","guid":{"rendered":"http:\/\/www.tigoe.com\/pcomp\/code\/?p=1109"},"modified":"2012-11-07T21:42:50","modified_gmt":"2012-11-08T02:42:50","slug":"serial-to-json-in-node-js-and-arduino","status":"publish","type":"post","link":"https:\/\/www.tigoe.com\/pcomp\/code\/arduinowiring\/1109\/","title":{"rendered":"Serial to JSON in Node.js and Arduino"},"content":{"rendered":"<p><a href=\"http:\/\/nodejs.org\/\">Node.js<\/a> is great for making web services, and node-serialport makes it very easy to connect to serial devices on your computer. In my last post, I showed how to connect an Arduino microcontroller application to a web page using Node. \u00a0This post expands on that, introducing how to use <a href=\"http:\/\/en.wikipedia.org\/wiki\/JSON\">JavaScript Object Notation (JSON)<\/a> from Arduino all the way through to your HTML page.<\/p>\n<p>The beauty of node is that it&#8217;s JavaScript, so you get to use everything Javacript gives you, including its wonderful lightweight data format, JSON, or JavaScript Object Notation. JSON describes data objects using arrays of key-value pairs separated by colons. If you want to add more properties to an object, just add another array element. the value of an element can be a JSON object itself. When you&#8217;ve got an application that uses JavaScript on the server side and on the client side, it makes sense to use JSON to describe your data all the way through, so you can just pass it around without a lot of conversion.<\/p>\n<p>Arduino doesn&#8217;t speak JSON natively. There are a couple JSON parser libraries out there for Arduino, but I haven&#8217;t seen one that I like yet. All of the ones I&#8217;ve seen expect more pointer knowledge from the user than I&#8217;d like. \u00a0So for this example you&#8217;ll assemble your own JSON string using Arduino&#8217;s String class, and letting node.js turn it into a JSON object.<\/p>\n<p>To make this happen you&#8217;ll need:<\/p>\n<ul>\n<li>An\u00a0<a href=\"http:\/\/www.arduino.cc\">Arduino<\/a>, and the Arduino IDE<\/li>\n<li>An accelerometer. I used the <a href=\"http:\/\/www.adafruit.com\/products\/1018\">ADXL326<\/a> from <a href=\"http:\/\/www.adafruit.com\">Adafruit<\/a><\/li>\n<li>an HTML5-capable browser. I used Chrome, but Opera or Firefox or Safari will work too)<\/li>\n<li><a href=\"http:\/\/nodejs.org\">node.js<\/a>\u00a0and the following libraries for node:\n<ul>\n<li><a href=\"http:\/\/expressjs.com\/\">express.js<\/a><\/li>\n<li><a href=\"http:\/\/socket.io\/#how-to-use\">socket.io<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/voodootikigod\/node-serialport\">node-serialport<\/a><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>This post assumes you understand:<\/p>\n<div>\n<ul>\n<li>How to use\u00a0<a href=\"http:\/www.arduino.cc\">Arduino<\/a>, including how to send serial data<\/li>\n<li>How to write a\u00a0<a href=\"http:\/\/www.codecademy.com\/tracks\/web\">web page<\/a>, and what a server is<\/li>\n<li>A little about\u00a0<a href=\"http:\/\/www.codecademy.com\/tracks\/javascript\">JavaScript<\/a><\/li>\n<li>A little about the\u00a0<a href=\"http:\/\/etutorials.org\/Mac+OS\/mac+os+x+power+tools\/Part+IV+Mastering+Your+Mac-Security+Maintenance+and+Unix\/Chapter+15+Utilizing+Unix\/Getting+Started+with+Terminal+and+a+Shell\/\">command line<\/a><\/li>\n<li>Beginning experience with node.js. \u00a0For my brief intro, see <a href=\"http:\/\/www.tigoe.com\/pcomp\/code\/arduinowiring\/1096\/\">this post<\/a>.<\/li>\n<\/ul>\n<\/div>\n<p><!--more--><\/p>\n<p>The example works like this:<\/p>\n<ul>\n<li>Arduino reads accelerometer<\/li>\n<li>Arduino sends JSON string of X,Y, and Z values to node.js<\/li>\n<li>node.js listens for clients<\/li>\n<li>If node.js gets a client, it reads serial string and generates a serialEvent, sending along the JSON object read from the Arduino<\/li>\n<li>Web client (also written in JavaScript) opens a websocket to node.js<\/li>\n<li>Web client listens for serialEvents, extracts object, uses it to set properties of a paragraph tag in the HTML document.<\/li>\n<\/ul>\n<div><span style=\"font-size: medium;\"><span style=\"line-height: 24px;\">First, <a href=\"https:\/\/github.com\/tigoe\/NetworkExamples\/tree\/master\/serialToJSON\">get the project from my GitHub repository<\/a>. If you&#8217;re familiar with git, just clone the whole NetworkExamples repository. If you&#8217;re not familiar with git, then download the .zip file and copy the serialToJSON directory to your user directory or desktop or wherever you like to work.<\/span><\/span><\/div>\n<div><\/div>\n<div><\/div>\n<h2>The Arduino Sketch<\/h2>\n<p>You can find this sketch in the ADXL326 directory of the project.<\/p>\n<p>The accelerometer used for this project is an <a href=\"http:\/\/www.adafruit.com\/products\/1018\">ADXL326 breakout board<\/a> from Adafruit. \u00a0It&#8217;s conveniently arranged so that you can plug it directly into pins A0 through A5 of an Arduino Uno, leaving the Test pin of the accelerometer hanging loose. \u00a0The pins line up as follows:<\/p>\n<ul>\n<li>Vin &#8211; A0<\/li>\n<li>3Vo &#8211; A1<\/li>\n<li>Zout &#8211; A2<\/li>\n<li>Yout &#8211; A3<\/li>\n<li>XOut &#8211; A4<\/li>\n<li>Gnd &#8211; A5<\/li>\n<\/ul>\n<div><span style=\"font-size: medium;\"><span style=\"line-height: 24px;\">The accelerometer consumes very little amperage, so it can run off the output of an I\/O pin easily. To make it work, set the pinMode of pins A0 and A5 to OUTPUT and take them HIGH and LOW respectively, like so:<\/span><\/span><\/div>\n<pre>  \/\/ initialize power and ground pins for accelerometer:\r\n  pinMode(A0, OUTPUT);\r\n  pinMode(A5, OUTPUT);\r\n  digitalWrite(A0, HIGH);\r\n  digitalWrite(A5, LOW);<\/pre>\n<p>Reading the accelerometer on the Arduino is pretty standard, but the formatting of the serial output is special. \u00a0The X, Y, and Z pins of the accelerometer outpu an analog voltage that varies with the acceleration on that axis. You read them using analogRead(). \u00a0To format the data as a JSON string, though, it should look like this:<\/p>\n<pre>{\"x\":\"379\",\"y\":\"364\",\"z\":\"389\"}<\/pre>\n<p>One easy way to format it is to assemble the values and the text around it as a String:<\/p>\n<pre> \/\/ form a JSON-formatted string:\r\n    String jsonString = \"{\\\"x\\\":\\\"\";\r\n    jsonString += x;\r\n    jsonString +=\"\\\",\\\"y\\\":\\\"\";\r\n    jsonString += y;\r\n    jsonString +=\"\\\",\\\"z\\\":\\\"\";\r\n    jsonString += z;\r\n    jsonString +=\"\\\"}\";\r\n\r\n    \/\/ print it:\r\n    Serial.println(jsonString);<\/pre>\n<p>In order to get the quotation marks in the string (which are needed for proper JSON formatting), \u00a0you need to escape them with a backslash like so: \\&#8221;<\/p>\n<p>There&#8217;s a handshaking setup between node.js and the Arduino so that the Arduino sends data only when there&#8217;s a web client connected to node.js. Arduino starts sending when it sees the letter c come in the serial port, and stops when it sees the letter x:<\/p>\n<pre> \/\/ read for incoming messages. c = send, x = don't send:\r\n  char inChar = Serial.read();\r\n  switch (inChar) {\r\n  case 'c':    \/\/ connection open\r\n    sending = true;\r\n    break;\r\n  case 'x':    \/\/ connection closed\r\n    sending = false;\r\n    break;\r\n  }\r\n\r\n  if (sending) {\r\n     \/\/ read sensors and send serial data here\r\n  }<\/pre>\n<p>That&#8217;s it on the Arduino side. You can test the code by opening the Serial Monitor and sending a c, then watching the data, then sending an x. You should see something like the string shown above.<\/p>\n<h2>\u00a0The node.js Server Script<\/h2>\n<p>To use node.js, you need to be familiar with the\u00a0\u00a0<a href=\"http:\/\/etutorials.org\/Mac+OS\/mac+os+x+power+tools\/Part+IV+Mastering+Your+Mac-Security+Maintenance+and+Unix\/Chapter+15+Utilizing+Unix\/Getting+Started+with+Terminal+and+a+Shell\/\">command line<\/a>\u00a0interface on your computer. \u00a0Open a new terminal window and change directories to the serialToJSON project directory you downloaded. If you list the files, you should see something like the following:<\/p>\n<ul>\n<li>ADXL326 &#8211; a directory containing the Arduino sketch<\/li>\n<li>SerialToJSON.js &#8211; the node.js server script<\/li>\n<li>README.md &#8211; the readme file<\/li>\n<li>index.html &#8211; the index page to be served to the web client<\/li>\n<li>package.json &#8211; the package description for the project.<\/li>\n<\/ul>\n<p>The node.js server script relies on several node packages:<\/p>\n<ul>\n<li><a href=\"http:\/\/expressjs.com\/\">express.js<\/a><\/li>\n<li><a href=\"http:\/\/socket.io\/#how-to-use\">socket.io<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/voodootikigod\/node-serialport\">node-serialport<\/a><\/li>\n<\/ul>\n<p>There&#8217;s a package.json file included in the\u00a0\u00a0<a href=\"https:\/\/github.com\/tigoe\/NetworkExamples\/tree\/master\/nodeSerialServer\">gitHub repository for this project<\/a>, so once you&#8217;ve downloaded this project from gitHub, you can install all the packages by typing:<\/p>\n<pre>npm install<\/pre>\n<div>\n<p>\u00a0This command will download and install all the necessary packages for this project from the net. \u00a0It&#8217;ll create a new directory for them in the project directory called node_modules. For more on this, see the npm<a href=\"https:\/\/npmjs.org\/doc\/json.html\">\u00a0guide to making packages<\/a>.<\/p>\n<p>Once you&#8217;ve got the packages installed, it&#8217;s time to launch the server. \u00a0You&#8217;ll need to know the name of your serial port. \u00a0It&#8217;lll be the same name you used to choose your serial port in Arduino. For example, on my mac, it&#8217;s \/dev\/tty.usb241331. On my PC, it&#8217;s COM4. To launch the server, type<\/p>\n<pre>node SerialToJSON.js portname<\/pre>\n<p>For example, for me it&#8217;s<\/p>\n<pre>node SerialToJSON.js \/dev\/tty.usbmodem241331<\/pre>\n<p>You&#8217;ll get a message like this:<\/p>\n<pre>info - socket.io started\r\nopening serial port: \/dev\/tty.usbmodem241331\r\nListening for new clients on port 8080<\/pre>\n<\/div>\n<p>Now you can open a connection to the server in a browser. Open a browser and enter this address:<\/p>\n<address>http:\/\/localhost:8080<\/address>\n<p>The server will deliver index.html to your browser and start listening to the serial port. \u00a0The index page will listen for serial events from the server, and update the DIV with the new data. \u00a0Voila! You&#8217;re serving serial data to a web page. You should see the accelerometer values in your browser window, and if you shake the accelerometer you&#8217;ll see the values jump around in the window.<\/p>\n<p>To stop the server, hold down the control key then type C on the command line.<\/p>\n<p>There are three main parts to the server script. The top part is where you set up all the initial variables and start listening on the serial port and on TCP port 8080 for web clients. The second part is where you listen for HTTP GET requests from the client. The only thing you&#8217;ll do here is to serve the index.html file, like so:<\/p>\n<pre>\/\/ respond to web GET requests with the index.html page:\r\napp.get('\/', function (request, response) {\r\n  response.sendfile(__dirname + '\/index.html');\r\n});<\/pre>\n<p>The third and most complex part is the section that responds to a websocket connection event. This occurs when a client connects, so the first thing you want to do is keep track of whether you&#8217;re connected or not, and send a message to the Arduino to start sending data, like so:<\/p>\n<pre>\/\/ if the client connects:\r\nif (!connected) {\r\n\t\/\/ clear out any old data from the serial bufffer:\r\n\tmyPort.flush();\r\n\t\/\/ send a byte to the serial port to ask for data:\r\n\tmyPort.write('c');\r\n        console.log('user connected');\r\n    \tconnected = true;\r\n}<\/pre>\n<p>You&#8217;ll also listen for two other events: a websocket disconnect event, where you&#8217;ll send a serial message to the Arduino to disconnect, like so:<\/p>\n<pre> \/\/ if the client disconnects:\r\nsocket.on('disconnect', function () {\r\n\tmyPort.write('x');\r\n        console.log('user disconnected');\r\n    \tconnected = false;\r\n  });<\/pre>\n<p>The other event to listen for is a new serial data event. \u00a0When this happens, you&#8217;ll convert the JSON string that came in serially to a JSON object, and send it to the client via the websocket:<\/p>\n<pre>        \r\n\/\/ listen for new serial data:  \r\nmyPort.on('data', function (data) {\r\n\t\/\/ Convert the string into a JSON object:\r\n\tvar serialData = JSON.parse(data);\r\n\t\/\/ for debugging, you should see this in the terminal window:\r\n\tconsole.log(data);\r\n\t\/\/ send a serial event to the web client with the data:\r\n\tsocket.emit('serialEvent', serialData);\r\n});<\/pre>\n<p>That&#8217;s it for the server script. The rest is handled by the web client page.<\/p>\n<h2>The Web Client page<\/h2>\n<p>Printing the serial data to the web page is done using JavaScript and CSS. \u00a0When the page loads, it makes a websocket connection back to the server, then waits for serialEvents.<\/p>\n<p>There&#8217;s a paragraph tag in the body of the HTML page. \u00a0When the client gets a new serialEvent, it uses the JSON object it gets from the server to set the CSS properties of the paragraph tag, like so:<\/p>\n<pre> \/\/ when you get a serialdata event, do this:\r\nsocket.on('serialEvent', function (data) {\r\n\t\/\/ you're going to move the div in the HTML:\r\n\tvar element = document.getElementById('textDisplay');\r\n\telement.style.position = \"absolute\";\r\n        \/\/ set the left edge of the text\t\r\n\telement.style.left = data.x+ \"px\";\r\n        \/\/ set the top edge of the text\t\r\n\telement.style.top = data.y+ \"px\";\t\r\n        \/\/ set the size of the text\r\n\telement.style.fontSize = data.z + \"%\";\t\r\n        \/\/ set the text itself\t\r\n\telement.innerHTML = data.x + \",\" + data.y + \",\"+ data.z;\r\n});<\/pre>\n<p>Notice how easy it is to pull the data out of the message thanks to JSON. You just refer to<br \/>\nThat&#8217;s it! When the page loads, the client starts the JavaScript, which starts the websocket, and the rest is history.<\/p>\n<p>Using a combination of websockets and JSON like this, you can pass any data from a serially connected device to a web page via node. Enjoy!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Node.js is great for making web services, and node-serialport makes it very easy to connect to serial devices on your computer. In my last post, I showed how to connect an Arduino microcontroller application to a web page using Node. \u00a0This post expands on that, introducing how to use JavaScript Object Notation (JSON) from Arduino &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/www.tigoe.com\/pcomp\/code\/arduinowiring\/1109\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Serial to JSON in Node.js and Arduino&#8221;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9,100,103,104],"tags":[],"class_list":["post-1109","post","type-post","status-publish","format-standard","hentry","category-arduinowiring","category-code","category-javascript","category-node-js"],"_links":{"self":[{"href":"https:\/\/www.tigoe.com\/pcomp\/code\/wp-json\/wp\/v2\/posts\/1109","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.tigoe.com\/pcomp\/code\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.tigoe.com\/pcomp\/code\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.tigoe.com\/pcomp\/code\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.tigoe.com\/pcomp\/code\/wp-json\/wp\/v2\/comments?post=1109"}],"version-history":[{"count":10,"href":"https:\/\/www.tigoe.com\/pcomp\/code\/wp-json\/wp\/v2\/posts\/1109\/revisions"}],"predecessor-version":[{"id":1120,"href":"https:\/\/www.tigoe.com\/pcomp\/code\/wp-json\/wp\/v2\/posts\/1109\/revisions\/1120"}],"wp:attachment":[{"href":"https:\/\/www.tigoe.com\/pcomp\/code\/wp-json\/wp\/v2\/media?parent=1109"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.tigoe.com\/pcomp\/code\/wp-json\/wp\/v2\/categories?post=1109"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.tigoe.com\/pcomp\/code\/wp-json\/wp\/v2\/tags?post=1109"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}