Node.js and the Propeller.

Heater.

Some common requirements come up on the forums from time to time one of which is:

We want a cross platform GUI to monitor and control our Propeller projects.
It should be usable on at least Windows, Mac, Linux.
It should involve minimal installation work and a language common to all platforms.
Let's say the Prop(s) is connected via a serial port for now.

Here is the plan:

1) One good way to make a cross platform GUI to is to make it web based. Then your GUI is usable anywhere you can run a browser. With recent HTML 5 developments like websockets and webgl your web pages get to feel very much like traditional desktop apps.

2) Problem is a browser can only talk to a web server, not directly to your Prop. So we need a web server. Enter Node.js. Node.js runs JavaScript outside the browser, just like a command line interpreter for Python or Perl or PHP. With node.js you don't need a web server like Apache or all those complicated PHP scripts and stuff. Just write one in JavaScript in ten lines of code. Also, as we will see later, one can stream data, bidirectionally, straight from Prop to browser very easily with Node.js.

3) Node.js also has modules available to talk over serial ports, so there is the link between Propeller and browser.

4) At some point there may be JavaScript on the Propeller so we have JS all the way through the chain from sensors to browser. But for now we use Spin or C/C++ or Forth or BASIC or whatever on the Prop.

Given the above one only has to run a node.js script on your PC and point a browser at it.

How to start?

First install node.js. You can find it here: http://nodejs.org/. Windows and Mac users get a ready to run installer, Linux users get to compile from sources (But, hey. that's how we like it).

When installed you should be able to just type "node" at the command prompt and get into the node.js interactive execution mode were you can just type JavaScript and have it execute immediately.

Write yourself a "Hello World!" program and save it to a file hello.js in a directory hello-node for example.

console.log("Hello World!");

Now you can run that hello world program with:

$ cd hello-node
$ node  hello.js
Hello World!

So far so good. First thing we need is a means of talking via serial to the Propeller. Luckily Node has a brilliant modules system with lots of good modules available. It's called "Node Package Manager" or "npm". There is a serial port module we can install with npm like so:

$ npm install serialport  

This will create a directory called node_modules in your hello-node directory where it puts all installed modules for your project.

Now to test the serial connection we need to write a little script like so:

"use strict";

var serialport = require("serialport");
var SerialPort = serialport.SerialPort;

var propellerPort = "/dev/ttyS0";

var serialOptions = {
    baudrate: 115200,
    databits: 8,
    stopbits: 1,
    parity: "none",
    buffersize: 256,
    parser: serialport.parsers.readline("\n")
};

var sp = new SerialPort(propellerPort, serialOptions);

sp.on("open", function () {
    console.log('Serial open.');

    sp.on('data', function (data) {
        console.log('Serial rx:', data);
    });

    sp.write("Some propeller command.\n");
});

Notes;
1) Always include "use strict"; as it ensure compliance with latest JS standards and helps prevent a lot of annoying errors.
2) The "require" imports the SerialPort object from the serialport module into our program.
3) We set up an event handler on the port open event. Everything in Node.js is asynchronous there is no "open" call that blocks until it's done. No, the serial port constructor returns immediately and notifies you when the port is open with the open event.
4) Similarly when the port is open we set up an event handler to receive the data.
5) We only write a single command message to the Prop.
6) The options include setting a line parser on the input data so you don't have to worry about where your data blocks start and end. This could be customized with your own parsing functon.

All this event handling should be familiar to JS developers were all browsers use an event loop for button pushes and such.

You can read more about the serialport modues here: https://github.com/voodootikigod/node-serialport

Assuming you have a Prop connected and sending text up the line and have adjusted the serial port parameters in the script you can run it as:

$node serial_test.js

and see data on the screen.

Next we need a webserver....


Edit: Rearranged things a bit and added line parsing of input.
Edit: The web server comes in post #18 http://forums.parallax.com/showthread.php/147253-Node.js-and-the-Propeller.?p=1175515&viewfull=1#post1175515

Edit: A little enhancement of the web server to serve html files comes in post #42 http://forums.parallax.com/showthread.php/147253-Node.js-and-the-Propeller.?p=1175952&viewfull=1#post1175952

The code discussed here is now up on GitHub https://github.com/ZiCog/propeller-node

KC_Rob

Heater. wrote: »

Some common requirements come up on the forums from time to time one of which is:

We want a cross platform GUI to monitor and control our Propeller projects.
It should be usable on at least Windows, Mac, Linux.
It should involve minimal installation work and a language common to all platforms.
Let's say the Prop(s) is connected via a serial port for now. ...

Now this is some cool stuff, Heater! Thank you; I'm installing Node.js now.

KC_Rob

Node.js installed and working on Windoz. Slick.

jazzed

Thanks for starting this thread Heater.

Some kind of a super simple web serial terminal might be a good next step.
There are many ideas that can be implemented of course. Just need time.

I'll be watching and picking up a few tidbits along the way.

--Steve

mindrobots

I started playing with the example on my Mac talking to a C3 board running PropForth (I thought that would give me a good interactive target to talk to). The node script work to the point of openning the port and responding to the open event or getting an error when you forget to release it from the serial terminal.

On the Mac, it appears to reset the Prop when it opens the port. I need to try this on Win and/or Linux to verify.

I write simple commands to the serial port from node and never see any responses. A blank line should at least give me a PropForth prompt. I'm going to try more with a Prop that's just spewing output every so often to make things as simple as possible. I'm sure I'll be shooting myself in the foot with my JavaScript skills soon enough!

The testing will continue.

Thanks for getting the ball rolling, Heater!!

Heater.

I updated the code a bit to add a line by line parser of the incoming bytes. Your "data" event should only fire now when a new line is received.

I was just considering this business of reset. Seems things can get reset when we don't want and there is no way to wiggle RTS ot DTR to reset when we do want. I'm just looking at what it might take to add that to the serialport module.

doggiedoc

I'm in.

Got node.js and npm installed and a working "hello world". I do hit a snag on the test script that I haven't resolved on my own yet. I'm not sure what mac os calls the serial connection. I used

$ ls /dev|grep cu.usbserial

to get cu.usbserial-A601FBHZ and replaced /dev/ttyS0 in the script but that didn't work. I still get:

events.js:2817: Uncaught Error: Cannot open cu.usbserial-A601FBHZ

.

I'll keep trying.


@Rick what did you use?

Paul

Heater.

I have no idea how this works on a Mac but on Linux one always has to be sure one has permissions to use the serial port.

Heater.

I presume you mean you used "/dev/cu.usbserial-A601FBHZ"?

doggiedoc

fixed... forgot the path info with cu.usbserial.....

Dr_Acula

Wow, this looks clever!

I've got a bit of a wish list. Hope that is ok?

Could you put a textbox on the screen and a button and read in a text file (eg a file that happens to end with the extension .spin You can see where this might be heading!)

Could you put a button on the screen that can run a batch file with some command line parameters? eg BSTC?

If so, that is the beginning of a new IDE.

doggiedoc

Heater. wrote: »

I presume you mean you used "/dev/cu.usbserial-A601FBHZ"?

Not at first. But I got there..

Heater.

Dr_A,

Indeed I do see where you are heading. Text edit box->enter Spin code->send to sever->compile->send to Prop->listen for results->send to browser.

Of course node.js has all the facilities to do that. It can read and write files on your server. It can run programs, like BSTC. It can send and receive files from your browser....

Let's do the baby steps first:)

doggiedoc

I get a response now, no errors. "Serial open."

Nice work Heater!

Bob Lawrence (VE1RLL)

Hers's links to a few inteeesting YouTube Video's about Node.js that I watch a few a few weeks ago .




Node.js: JavaScript on the Server (Ryan Dahl the creator of Node.JS)

https://www.youtube.com/watch?v=F6k8lTrAE2g

================================================================

Ryan Dahl - Node Summit 2012 - theCUBE( interview - Why he created Node.js)

https://www.youtube.com/watch?v=Fc26auhSLqM

==================================================================
Node.js Explained

https://www.youtube.com/watch?v=L0pjVcIsU6A

===================================================================

jmg

Interesting stuff.
However, I see it has a sizable pre- requisite list, that is also quite version-constrained. Early days ?



["To Install

npm install serialport

This assumes you have everything on your system necessary to compile ANY native module for Node.js. This may not be the case, though, so please ensure the following are true for your system before filing an issue about "Does not install". For all operatings systems, please ensure you have Python 2.x installed AND not 3.0, node-gyp (what we use to compile) requires Python 2.x.
Windows:

Ensure you have Visual Studio 2010 installed. If you have any version OTHER THAN VS 2010, please read this: https://github.com/TooTallNate/node-gyp/issues/44
Mac OS X:

Ensure that you have at a minimum the xCode Command Line Tools installed appropriate for your system configuration. If you recently upgrade OS, it probably removed your installation of Command Line Tools, please verify before submitting a ticket."]

mindrobots

Paul,

I used the complete device name as created by OS X "/dev/I forget-8004IHX" - I believe I got this error when the terminal program still had the port assigned. You could also get it if you are trying to use the wrong port, a port something else is using.

You can always plug in the prop, list the ports, then unplug the prop, list them again and see what disappears. This helps with my confusion at times.

EDIT: never mind - took too long to type while watching TV! Guilty!!

Heater.

OK before I call it a day let's turn this into a web server and display what comes up the line from your Prop. All be it crudely.

We can do this by adding 9 lines of code to the serial port test example like so:

"use strict";

var serialport = require("serialport");
var SerialPort = serialport.SerialPort;

var propellerPort = "/dev/ttyS0";

var SERVER_PORT = 8080;

var propRxBuffer = "Propeller Port not open yet.";

var serialOptions = {
    baudrate: 115200,
    databits: 8,
    stopbits: 1,
    parity: "none",
    buffersize: 256,
    parser: serialport.parsers.readline("\n")
};

var sp = new SerialPort(propellerPort, serialOptions);

sp.on("open", function () {
    console.log('Serial open.');
    propRxBuffer = "Propeller Port open. Nothing recieved yet...";

    sp.on('data', function (data) {
        console.log('Serial rx:', data);
        propRxBuffer = data;
    });

    sp.write("Some propeller command.\n");
});

sp.on("error", function (err) {
   console.log(err);
   propRxBuffer = "Error: " + err.message;
});


// Load the http module to create an http server.
var http = require('http');

// Configure our HTTP server to respond with our serial port data to all requests.
var server = http.createServer(function (request, response) {
    response.writeHead(200, {"Content-Type": "text/plain"});
    response.write(propRxBuffer);
    response.end("\n");
});

// Listen on port 8080, IP defaults to 127.0.0.1
server.listen(SERVER_PORT);

// Put a friendly message on the terminal
console.log("Server running at http://127.0.0.1:" + SERVER_PORT + "/");

Basically we drop every line received from the Prop into a string buffer "propRxBuffer".
The http server should be self explanatory. It just sends the last line received to any browser that connects.
Also I added an error handler for the serial port. Those errors should also show in the browser.

With that in place you should be able to point your browser at http://localhost:8080 and see your Props messages.

Of course this is very crude and requires a page reload to get new messages, but hey we have that hot line from Prop to browser already.

jmg

Heater. wrote: »

I was just considering this business of reset. Seems things can get reset when we don't want and there is no way to wiggle RTS ot DTR to reset when we do want. I'm just looking at what it might take to add that to the serialport module.

Control of ALL handshake lines would be important.
If direct access to RTS.DTR.CTS etc is locked out, then there may be other pathways.,

I notice node says this

List Ports

You can also list the ports along with some metadata as well.

serialport.list(function (err, ports) {
ports.forEach(function(port) {
console.log(port.comName);
console.log(port.pnpId);
console.log(port.manufacturer);
});
});


ie it seems to have no problems with many serial ports...

The SiLabs CP2105 has two serial ports, so you could open one for Data, and use one for Prop Reset, and if you wanted to expand control lines, then you could go as simple as send/echo to the control port. The CP2105 is under $2, so is similar to a FTDI part.

Heater.

jmg,

Things are not quite so bad now. npm is included with node. It used to be a separate install.
Windows support is the new kid on the block but that has come along very fast. In fact node.js is supported by MS itself in it's latest dev tools and they support it in their Azure cloud services.

Bob,

Great vids. My favorite is Ryan's first introduction of Node. He seems so nervous and excited and well, brilliant.
https://www.youtube.com/watch?v=ztspvPYybIY

Heater.

jmg,

My evil plan is to run all this on a Raspberry Pi. In that case node.js also has control of the GPIO pins which can be used for Prop reset.

jmg

Heater. wrote: »

jmg,

My evil plan is to run all this on a Raspberry Pi. In that case node.js also has control of the GPIO pins which can be used for Prop reset.

Yes, but it needs to be widely portable - the ideal is of course, proper Serial handshake controls, but as an interim, the idea of using a second serial port gives a work-around. (certainly reset is easy to do )

mindrobots

jmg wrote: »

Yes, but it needs to be widely portable - the ideal is of course, proper Serial handshake controls, but as an interim, the idea of using a second serial port gives a work-around. (certainly reset is easy to do )

A Cp2105 dual port interface doesn't seem that widely portable in a land of FTDI single port boards.

jmg

mindrobots wrote: »

A Cp2105 dual port interface doesn't seem that widely portable in a land of FTDI single port boards.

If you are awash with FTDI parts, then just apply two
The CP2105 was just an example of a part with this in one package.

Dr_Acula

re post #22 I agree with jmg. And, is the propeller reset that important?

I mean, if you are programming a propeller, you can use the proptool (works on a wide variety of PC platforms) or BSTC (ditto).

Once you can get a tiny bootloader program onto the propeller then you can do a software reset. So you can invent a comms protocol using two serial lines, and send a message to do a reset, if you wanted to do that.

I've just spent an hour reading about node.js. It does look a nifty solution. Can it do clever things like a richtextbox with syntax highlighting?

Bob Lawrence (VE1RLL)

For the client side you can use java script, JQuery, HTML5, etc I made a teaser client side app . I can deploy it in Google Chrome. It doesn't actually do anything put display the buttons etc in a web browser. Just to put a few ideas out there.

Phil Pilgrim (PhiPi)

Awhile back, I started an HTML/JS-based Spin editor. You can read about it here:

http://forums.parallax.com/showthread.php/132690

I did not pursue it further, though.

-Phil

Heater.

Dr_A, Phil and co.

If you want to make a web based editor/IDE for Spin, or any other language, I suggest you start from this one : https://c9.io/site/features/

The Cloud9 IDE already does editing, syntax highlighting, code completion, search etc for JavaScript. The commercial version does all that for dozens of other languages. The source code is avaihttps://github.com/ajaxorg/cloud9/

Or perhaps just start from the ACE editor, which Cloud9 uses, http://ajaxorg.github.io/ace/#nav=about

mindrobots

Anyone trying this on Windows?

I just ran into this bug report on node-gyp from 10 days ago:

"Additionally I just found out that serialport currently does not work with the Express version of VS due to serialport's use of MFC/ATL which is only available in the retail version. :-\"

Guess it's back to the Mac or get out my Linux laptop....

If anyone is having any luck with the Express versions of VS2010 on this project, let us know!

Tor

Just throwing something into the mix here.. is there anyway that this 'terminal in a web server' project could fit in somehow? https://code.google.com/p/shellinabox/
I don't know much about it but I've seen it used as a vt100-terminal-in-a-browser for a javascript-implemented 8080 emulator.

-Tor

Heater.

Tor,

Yes, I have seen shellinabox before and the 8080 emulator. Immediately I thought "I want that" to talk to Props and Raspberry Pis over the net. I'm sure it can be done.