Self-Hosted Prop2 Development Over The Web
cgracey
Posts: 14,134
It seems that people are quite comfortable with their tablets and such, and they are used to making things happen from them.
Do you think people would be interested in Prop2 software development over their tablet devices (and PCs, for that matter)?
Imagine if they could monitor their Prop2 operations remotely, and even pull up, modify, compile, and store their Prop2 applications, all on the target system. Do you think people would like this?
I picture a setup where the tools are located on a Prop2 that is 'live' on the web. It acts as a web site, of sorts, and via some simple HTML interface, it lets you work on the Prop2 as if you were right there. The system might even target a 2nd Prop2 to run the final application, and let the first Prop2 be the interface, since it wouldn't be subject to bugs introduced by the user.
It seems that, at least, for industrial control, this could be useful. For agriculture, too. All those geek types might enjoy it, as their expectations seem to be in this realm, already.
Do you think people would be interested in Prop2 software development over their tablet devices (and PCs, for that matter)?
Imagine if they could monitor their Prop2 operations remotely, and even pull up, modify, compile, and store their Prop2 applications, all on the target system. Do you think people would like this?
I picture a setup where the tools are located on a Prop2 that is 'live' on the web. It acts as a web site, of sorts, and via some simple HTML interface, it lets you work on the Prop2 as if you were right there. The system might even target a 2nd Prop2 to run the final application, and let the first Prop2 be the interface, since it wouldn't be subject to bugs introduced by the user.
It seems that, at least, for industrial control, this could be useful. For agriculture, too. All those geek types might enjoy it, as their expectations seem to be in this realm, already.
Comments
We have done it already for the Prop I. See this thread: http://forums.parallax.com/showthread.php/152711-A-Propeller-WebTool-Framework-for-Compiled-Languages
Short story is that I managed to "transpile" Roy's openspin compiler to JavaScript, using Emscripten, and get it running in the browser. https://github.com/kripken/emscripten
Yes, that sounds nuts but it works very well and runs nearly as fast as the C++ openspin compiled to native code.
msrobots and others have an in browser editor with Spin syntax highlighing working.
With the compiler and editor we have an IDE in a web page!!
Files can be loaded and save from you local PC or uploaded and downloaded to a server.
But what if the server is a Propeller? Then the Propeller can serve up the IDE web page and be programmed from files compiled in that web page.
David Betz and others have been doing exactly that with Prop serving up web pages via some wireless gizmos (No idea what myself), not sure if they got as far as getting the Prop reprogramming working, I think so though.
All this sort of works as a proof of concept. Needs a lot of polishing.
You can try out msrobot's web Spin editor and JS Spin compiler here http://parallax.msrobots.net/Editor16.htm
Available for all OS's, tablets etc., essentially equivalent of serial over tcp/ip with a vt100 equivalent terminal.
There is also VNC and RDP, both require more work, both available for every os.
That's all pretty interesting. I wasn't even aware that all this had been going.
The last link to MSRobots IDE seems like a pretty complete solution. I'm way behind the curve on how things work today. My notions are quite anachronistic, it seems.
I am playing around with the 'WebIDE' plan for a while now. I even have syntax high lighting for PASM2 now.
But there are some problems with it.
First of all it is not really on-chip development. Its on-browser development. I am able to fetch all of this from a spinneret and I have a version of your spin propeller loader modified by Mike G. to load another Propeller from a file save on the SD card of the Spinneret.
So you can Edit a project, compile it, save to SD of the Spinneret and program another Propeller. Sort of a PropPlug with Webserver and IDE on it. This is still work in progress
But the all File systems available on the Spinneret just support 8.3 filenames. So FullDuplexSerial.spin and FullDuplexSerial4port.spin are not easy to keep apart.
And besides using a Parallax Product to do this, it is more cost effective to buy a RasPi then a Spinneret.
The second way to use the Editor by now is what David is doing. He uses a resident program in the EEPROM of a Activity board and the new WIFI module Parallax sells.
The Editor can send a binary to the WIFI module and the propeller resident loader overwrites the upper EEPROM. This is already working.
I guess that as soon as you have a working SPIN2/PASM2 compiler in some stable form Roy will update OpenSpin. Then @Heater can do his magic and we can run it in a webbrowser.
You are thinking quite seriously wrong there. The Webbrowser environment is FULL of contracts, security permissions and differences between different OS and Browser combinations.
By now we have that worked out, but what about the next OS and Browser versions? This is all in Flux. All the time.
IMHO the best way to go there is development on the P2 itself. You do not need a browser/pc/tablet at all. just P2, Keyboard, Monitor and software running on the P2.
This can also run over the Internet, connecting two P2s if one wishes, but does not have to.
Mpark was right on the point with sphinx. It took weeks to get that stupid smile off my face after I had it running on my first Propeller board.
Enjoy!
Mike
Problem is, everyone wants the nice GUI Editor, and that takes resources. And that is what slows the display with key inputs going out over the web and GUI coming back (not just text).
I think the Dev system on a P2 makes the most sense - a P2 (+ xtal & flash) + microSD + KBD+ Display (composite video/vga/etc). SDRAM would be an option. But just a simple minimum system. If that system had wifi or Bluetooth, perhaps the tablet could be the kbd + display but it could have the same delays too. BTW for real editing, you definitely need a keyboard (available for most tablets anyway).
BTW The WR703N router with Ethernet, WiFi and USB for $20-25 running Linux makes a nice device. It has been hacked to add a serial port. There is quite a bit of sw for this device. It could make a nice addon for the P2. I have one I want to add to the P1.
WOW, I wonder how I did not know about this !!
I tried the online Editor16 and it seems to work great !
Now, with all of my source files on Dropbox and with this online Spin Editor, I have ALL of my Prop Development in the Cloud, and with easy access from all of my devices. - Thanks !
(note: I still need to try out the "download to Prop" functionality)
Yes. Me too.
My Spinneret loader is not working yet and the solution from David needs a S6B WIFI module...
But we are getting there.
Enjoy!
Mike
Well, that works best for me, as we don't need to involve lots of protocols to get something done.
And the idea of Prop-to-Prop communication over the web is about as clean as it could be, right?
Yes. Exactly. Since we are on both ends we 'own' the communication.
And besides the move to IPV6 which is a ongoing process now in the web the basic TCP/IP protocols are pretty stable.
Enjoy!
Mike
You can do that right now.
In menu FILE select 'Upload local files'
Select (multiple) files to upload. click OK. All files are uploaded and opened in a Tab each. Select the one to compile. hit compile.
Download result binary to your computer and do whatever you want to do with it.
Enjoy!
Mike
Is this running on a Prop Spinneret or another webserver?
But now that you come up for air you might see that the browser is the operating system today.
If you want to make anything cross-platform forget about cross-platform tool kits like Qt or Lasarus that work on Windows or Mac or Linux just make it for the universal GUI environment, the browser.
Ah, you say, the browser cannot interact with the local machine, read files and so on. So what? just put a simple server on the local machine. Written in Python or JavaScript or whatever you like.
Does this local server have to be online to the internet for the browser to access the files on the local server?
Doesn't the local server make your machine more vulnerable to security attacks?
Thing is that compiling in the browser on a PC is perhaps half or quarter the speed of the native compiler. Not bad. Not sure about the editor but it works decently fast enough.
I suspect that if you had a natively compiled openspin for the zoom it would also be slow.
If anyone wants to do that it would be an interesting comparison.
When I say simple web server I mean a little 100 line program you write for yourself in Python or whatever language you like.
Well, no what I mean is a web server created in JavaScript using Node.js.
This for example is the web server I have been using to play with the openspin compiler on the web:
// Import required modules and wire them up var express = require('express'); var app = express(); var server = require('http').createServer(app) var io = require('socket.io').listen(server); var fs = require('fs'); // Log http requests app.use(express.logger()); // We want to parse bodies when uploading files. app.use(express.bodyParser()); // EXAMPLE OF MULTIPLE PARAMETERS: app.get('/fruit/:fruitName/:fruitColor', function(request, response) { /* return the response JSON data as above using request.params.fruitName and request.params.fruitColor to fetch the fruit apple and update its color to red*/ }); // Handle GET requests to files in the "project" area. app.get("/project/:file", function(req, res) { // Fun fact: this has security issues // Open a read stream from the target file var fileName = './project/' + req.params.file; var readStream = fs.createReadStream(fileName); // Pipe data from file into response readStream.pipe(res); // After all the data is saved, respond with a 200 readStream.on('end', function () { //res.writeHead(200); //res.end(); }); // Handle any errors on the stream readStream.on('error', function (err) { console.log(err); res.send(404); }); }); // Handle GET requests to files in the "library" area app.get("/library/:file", function(req, res) { // Fun fact: this has security issues // Open a read stream from the target file var fileName = './library/' + req.params.file; var readStream = fs.createReadStream(fileName); // Pipe data from file into response readStream.pipe(res); // After all the data is saved, respond with a 200 readStream.on('end', function () { //res.writeHead(200); //res.end(); }); // Handle any errors on the stream readStream.on('error', function (err) { console.log(err); res.send(404); }); }); // Serve static files from "public" dnd "js" directories app.use(express.static(__dirname + '/public')); app.use(express.static(__dirname + '/js')); // Serve an index page app.get('/', function (req, res) { res.sendfile(__dirname + '/public/index.html'); }); /** * HTTP PUT /{fNNilename}.spin * Param: Name of file to save. (No :xxx params) * Body: Contents of file to save. * Returns: 200 HTTP code * Error: 404 HTTP code if the task doesn't exists */ app.put("/project/:file", function (req, res) { var fileName = './project/' + req.params.file; // Open a stream into the target file var writeStream = fs.createWriteStream(fileName); // Pipe data from request into file req.pipe(writeStream); // After all the data is saved, respond with a 200 req.on('end', function () { res.writeHead(200); res.end(); }); // Handle any errors on the stream writeStream.on('error', function (err) { console.log(err); res.send(403); }); }); // Any other request gets this app.use(function(req, res){ console.log(req.url); console.log("WTF"); res.send(404); }); // Handle web socket connections io.sockets.on('connection', function (socket) { // Send hello message to all new socket connections socket.emit('hello', { hello: 'world' }); // Just log socket client messages socket.on('message', function (data) { console.log(data); }); }); // Start listening server.listen(3000); console.log('Listening on port 3000');Simple enough to follow I think. It serves up files from here and there, it allows the browser to PUT files here and there. It has the beginnings of two way communication between browser and, well, whatever you like via a websocket. Where "whatever you like" could be a serial line connection to your Propeller.If it bothers you the a couple of extra lines of code can be added to reject connections from outside "localhost" or the local LAN or whatever. A do it yourself firewall if you like.
If it really bothers you the a couple more extra lines of code gets you secure communication with HTTPS. Then you can safely commune with your Propeller at home from the other side of the planet.
Requiring a Spinneret is probably a non-starter (cost prohibitive plus needs Ethernet/Wifi adapter). Requiring an RPi works for some people (cheap, and flexible plus needs USB Wifi adapter).
A Propeller ActivityBoard with a 64KB EEPROM (32KB loader + 32KB user space) and a Digi XBee S6B is a simple and low cost solution.
Of course if you have a captive audience, it doesn't matter what you do ... to a point.
If P2 can do full-speed USB host and have code room for a USB-Wifi driver as a program store/boot-loader, then that would be really neat. I wouldn't want to see two more years added to the schedule though.
Err,...was that graphical enough?
I would be happy if we could program a PI or PII from a HTML5 based web IDE running in Chrome via a serial connection. This is already shown to be possible by the Espruino project.
If you happen to not want to use Chrome then a simple server is required on the local machine to make the serial connection for you. It's not such a big deal.
I think I meant to say "the internet". What do you say about that?
Words like upload and download are quite confusing with this online editor. I need to explain this a little bit.
This is no webserver-application in any form. It runs locally in your browser on your own computer you are using to open editorxx.htm
Depending on OS/Browser you need to start the Editor from a Webserver or can open it from your local File System. But then it lives on your local System in your Browser.
So when you 'Upload local file(s)' you just load them from your local file system into your local browser. Nothing is going over the internet or is done by any server.
if you compile, you compile in your local browser. Except for fetching files out of the Propeller Library nothing is going over the internet or is done by any server.
It is just html,css and javascript running in the webbrowser you are using to open Editorxx.htm.
By now the Editor support two different 'File Systems'
A) your own hard drive. This is referred to as 'Upload from local files system' or 'download to local file system'.
In case of
The funny thing is that you can compile a local file using a file out of a webserver library and saving the result to Heaters. server in where ever.
I am COBOL programmer by education, C# programmer by trade and Heater talked me into JavaScript a while ago. Thus I stumbled into Html5 also.
I haven't had as much fun programming for a while as now. This Html5 stuff is different.
Enjoy!
Mike
Enjoy!
Mike
Terms like Load and Save should refer to local resources. Upload and Download should refer to remote resources. No?
I am running this from a spinneret. Editor and files. for the Editor (and me) the File System of the SD is sort of local. I save and load just by dir/filename.
No Upload needed. no http needed, no server/ip needed. so I load and save there.
To get a resource from your -browser-local-file system I need to Upload it. input type file opens a Upload File Dialog box. That is what the Browser does. Not me.
Same with download to local file system. If you click the link the Browser ask you if you want to download or open the file. That is what the Browser does. Not me.
As I said. I am unhappy also. I tried the other way around a while ago and it was way more confusing to click on load then getting a Upload Dialog and clicking on upload to simply load a file without dialog...
we just need to find the right wording for this.
Enjoy!
Mike
.
What is your load target? That is, where does the spin code run?
Edit ....
Ok, all these questions were answered already in that other thread :-)
Maybe someone will come in with a beam of light and clear up the point of view confusion.
I tried editing a small few line file on my Xoom with your v16 (IIRC). It was horribly slow so I presumed it was because al the keystrokes were going out over the internet and the GUI commands came back. If the actual editing can be done on a local file from within the browser, then I will give it another try using my PC and see if its the Xoom. I have an iPad Mini Retina which is likely much faster and I have a keypad for it too.
heater,
I don't mind if the compile takes a minute or two. It's just editing where you have to wait for the characters to appear after you type them, and can get dropped keystrokes, means that it is not realistically usable.
You have both at least enlightened me on how this works (sort of anyway). If the little WR703N box could be recoded to take the WiFi and interface it to the P1 or P2 via the added serial port then this could be impressive. I believe we could do it with a Propplug into the USB port, and some simple recoded sw. The *nix hacks apparently are pretty powerful and lots of these boxes are being used that way. They are a nice little ~2"x2"x0.6" plastic box.
Whatever we do with on chip development, can we at least make sure we've got a plain old text / serial one? It might not always make sense in a browser, nor with two props.
That said, having a two prop IP capable environment would be very nice. As would one that just works with a browser and a single propeller, if possible.
The only external action happens when our read and write source or binary files either on your local machine or a remote server.
As such I don't know why this is so slow on your Xoom. I'll have to give it a go on some Android pad if I can find one around.
Now that I read your post again I'm not sure I know what you meant by "Prop-to-Prop communication over the web[/internet]". What did you have in mind?
Are we talking adding a gadget to the system that does IP on be half of the Propeller? Say a wifi dongle or serial to ethernet device, or what?
Are we talking putting a IP stack in the Propeller and giving it a ethernet physical interface somehow?
If I wanted to have two Propellers communicating of the internet I would have them both connected to Raspberry Pis or, something similar, and let that take care of all the networking stuff. Simple, cheap, flexible.
Which brings us to that idea of using two Propeller's, one to act as the network node, serving up the web IDE and other files, and programming a second Propeller that is the "target" for the user application.
That's a fine idea as it means your communication node cannot be messed up by faulty user programs. But I cannot see a Propeller system fulfilling that role as cost effectively or capably as a cheap ARM board. The Raspberry Pi is one example there are increasingly more on the market.
P.S. I was going to edit away my Ganges comment as I started to think it was in bad taste. But now i have been quoted so there is no point I guess.