HTML-based Propeller IDE -- now with auto-completion
Phil Pilgrim (PhiPi)
Posts: 23,514
In another thread, I broached the possibility of an HTML-based Propeller IDE. Among other things, there was an open question about whether HTML(5?)/Javascript/Ajax could provide a rich-enough environment for such an endeavor. In an effort to find out, I've spent the last two weeks investigating both Ajax-related topics and HTML-based editors. I found a rather nice editor written in Javascript called CodeMirror 2, which provides hooks for doing text highlighting and other things a code editor might need. I wrote a Spin "mode" program for it in Javascript to do the highlighting, and modified the CodeMirror.js code to accommodate Spin's rather unusual tabbing and indentation functions.
You can try out the result here: http://www.phipi.com/editspin It's preloaded with Float32Full.spin, but you can delete that and type or paste your own code if you like.
I chose highlighting colors that I liked and elected not to do any section-dependent background shading. Such shading could easily be accommodated: it would just require a more comprehensive spin.css file. I also added syntax checking to catch the easier-to-detect, low-context errors, which show up in red as typing/editing occurs. In addition, I changed the way a shift-tab is handled when no text is selected. In the Propeller Tool, such an action just moves the cursor. In this editor, everything to the right of the cursor shifts left until it hits the previous tab stop or a non-blank character. Finally, I added bold-facing to document comment words that begin with a backtick, in line with the autodocumentation markup proposed for "gold standard" objects.
Here's the Javascript function I wrote for the Spin highlighting:
This editor only scratches the surface of the features supported by CodeMirror, which can also include autocompletion. I've not made any effort at dynamic window resizing, although that could be done, too.
Anyway, an editor is a necessary first step to any IDE framework. For the Ajax part, I've been investigating jQuery and have read reviews for other such frameworks. Since I'm very new to this stuff, jQuery seems to be the most approachable and still has a rich feature set.
At this point, it's unclear how much more effort I want to expend in this feasibility study, even though the results are very encouraging so far. If jQuery seems easy enough to grok, I may take it one step further to produce a demo IDE with simple open, save, compile, and load functionality.
-Phil
You can try out the result here: http://www.phipi.com/editspin It's preloaded with Float32Full.spin, but you can delete that and type or paste your own code if you like.
I chose highlighting colors that I liked and elected not to do any section-dependent background shading. Such shading could easily be accommodated: it would just require a more comprehensive spin.css file. I also added syntax checking to catch the easier-to-detect, low-context errors, which show up in red as typing/editing occurs. In addition, I changed the way a shift-tab is handled when no text is selected. In the Propeller Tool, such an action just moves the cursor. In this editor, everything to the right of the cursor shifts left until it hits the previous tab stop or a non-blank character. Finally, I added bold-facing to document comment words that begin with a backtick, in line with the autodocumentation markup proposed for "gold standard" objects.
Here's the Javascript function I wrote for the Spin highlighting:
function tokenLexer(stream, state) {
var matched;
if (stream.match(/^\{\{/)) {
state.superComment++;
return 'doccomment';
}
if (stream.match(/^\}\}/)) {
if (state.superComment) {
state.superComment--;
return 'doccomment';
} else {
return 'error';
}
}
if (state.superComment) {
if (state.bold) {
if (stream.match(/^\W/)) {
state.bold = false;
return 'doccomment'
} else {
stream.next();
return 'boldcomment'
}
}
if (stream.match(/^`/)) {
state.bold = true;
} else {
stream.next();
}
return 'doccomment';
}
if (stream.match(/^\{/)) {
state.nestedComment++;
return 'codecomment';
}
if (stream.match(/^\}/)) {
if (state.nestedComment) {
state.nestedComment--;
return 'codecomment';
} else {
return 'error';
}
}
if (state.nestedComment) {
stream.next();
return 'codecomment';
}
if (stream.match(/^"/)) {
if (!(stream.skipTo('"') && stream.next())) {
stream.skipToEnd();
}
return 'string';
}
if (matched = stream.match(/^''/)) {
stream.skipToEnd();
return 'doccomment';
}
if (matched = stream.match(/^'/)) {
stream.skipToEnd();
return 'codecomment';
}
if (stream.match(sectionHeaders, false)) {
var sol = stream.sol();
matched = stream.match(sectionHeaders);
if (!sol) {
state.section = 'none';
return 'error';
}
state.section = matched[0].toLowerCase();
return 'keyword';
}
if (stream.match(wordOperators)) {
return 'operator';
}
if (state.section === 'dat') {
if (stream.match(/^\$(?=(\W|$))/i)) return 'keyword';
keywords = pasmKeywords; errors = pasmErrors;
} else {
keywords = spinKeywords; errors = spinErrors;
}
if (stream.match(keywords)) {
return 'keyword';
}
if (stream.match(errors)) {
return 'error';
}
if (matched = stream.match(/^[+-\.]*[$%\d]/, false)) {
stream.match(/^[+-]?\d[\d_]*(\.\d[\d_]*(e[\+\-]?\d[\d_]*)?)?/i)
|| stream.match(/^[+-]?\$[\da-f][\da-f_]*/i)
|| stream.match(/^[+-]?%[01][01_]*/)
|| stream.match(/^[+-]?%%[0-3][0-3]*/)
if (stream.match(/^[\da-f$%\._]+/)) {
return 'error';
} else {
return 'number';
}
}
if (stream.match(tripleOperators) || stream.match(doubleOperators) || stream.match(singleOperators)) {
return 'operator';
}
stream.match(identifiers) || stream.next(); return 'normal';
}
This editor only scratches the surface of the features supported by CodeMirror, which can also include autocompletion. I've not made any effort at dynamic window resizing, although that could be done, too.
Anyway, an editor is a necessary first step to any IDE framework. For the Ajax part, I've been investigating jQuery and have read reviews for other such frameworks. Since I'm very new to this stuff, jQuery seems to be the most approachable and still has a rich feature set.
At this point, it's unclear how much more effort I want to expend in this feasibility study, even though the results are very encouraging so far. If jQuery seems easy enough to grok, I may take it one step further to produce a demo IDE with simple open, save, compile, and load functionality.
-Phil
Comments
-Tor
OBC
-Phil
My first thought was using Propellent.dll to generate a binary file on the server, then send the binary file via HTTP the to the Spinneret. I believe Kye's SD fat driver can load the binary into EEPROM (have not tried it yet). Could be a little sketchy if the program caused an unresponsive Spinneret/Propeller.
EDIT: My mind is reeling...consume the Propellent.dll in a service on the host.
-Phil
With all this Javascript flying around next thing you know Phil will be losing interest in Perl especially having all those RegEx abilities.
-Phil
However, for someone with an iPad, android tablet, or even one of the new mobile phones, who just needs to modify or extend some programs and compile and download, this would be great. That way we wouldn't need to take the laptop. So a way is needed to upload and download the source files.
This could be a neat app for Parallax Semiconductor to have on their server so that users could check out a virtual prop, or program and download some simple apps to the Quick Start board.
For those of us with web servers or paid web servers, this could be a nice touch
I really think there could be a lot of publicity mileage for the prop if this worked sufficiently.
-Phil
That was with the google Chromium browser, on a 64-bit quad-core 2.66GHz Linux box. So now I tried it again, but in Firefox 3.5 on the same machine.. there's a definite delay there, when entering characters. It must be noted though that this particular instance of Firefox has about 50 windows open right now and is connected to two dozen sites.
Conclusion: Chromium on my particular machine works just fine with that editor. Firefox must be re-tested with less load. Haven't tried Opera or any of the other browsers yet.
-Tor
It's snappy for me. Firefox on a Intel Quad 2.4G Windows box and Firefox on an old AMD box running Linux. The underline characters that decorate Revision History has an encoding issue on my Firefox/Linux system.
I think this is a splendid idea, just the cross platform aspect alone is killer.
It would be just as easy to send a binary (any binary) down for boot-up programming.
I just wish we could make some of those cheap SDCARD dongles work with Propeller.
I use Chrome and rarely have performance issues with that - Internet Exploder? No Thanks!
I don't have any problems typing into the forum software, but I did have problems with your program keeping up with my typing.
Just tried with firefox and it was fine. I retried with IE8 and it was a little jerky in displaying the characters I typed. Not really surprising
So I am interested to see comparisons by anyone who can try both on the same pc.
I could definately work with firefox
IE8's JS interpreter is extremely sluggish and somewhat buggy. Google has an Chrome V8 add-in for it that helps to speed things up a bit. I've been experimenting with it and will post a link later today on the editor page so IE users can install it.
-Phil
-Phil
OBC,
Ever since Phil first brought this up, I've been thinking about the Spinneret thing too. My first thought was using it to host an HTML/JavaScript-based IDE and compiler which would run entirely on the user's machine. Any important data (compiled/non-compiled code, settings, etc) could be PUT/POST on the Spinneret's SD card. Of course, it should also have the ability to act as a program/debug interface. Now, I don't know much about JavaScript, so I couldn't say for sure if you can write an entire compiler out of it, but I imagine it should be possible (Any thoughts?). The Sphinx idea is a cool too, and probably the least far fetched. Either way is cool since they both have their merits. Just the thought of having everything you need to program a Prop aside from ANY computer (phone/PDA even?) with a web browser and an ethernet cable, in the form factor of a Spinneret sounds really awesome.
EDIT: IMO Phil's approach is very powerful. Besides cross platform, you also have one server end point that deals with compiling the source code. Updates are done directly on the centralized server (end point) and available to all clients. No need to install a new client app. The only sticky part is the target Propeller must be attached to a system that knows how to transmit the binary to the Prop.
So, internet cafeteria, vacation, caffe latte, quickstart board and usb stub cable in pocket, there are maybe no astronomical hurdles in the way for connecting everything together?
-Tor
Obviously, you need the editor/compiler loaded on your jungle box. If you don't have internet access, well, an Internet based app would not make much sense. For those of us that have Internet access, I think it makes sense.
@Tor, cool
On the one hand I need an internet connection all the time. I need to work in a browser all the time which has to be clunkier than having a native app.
On the other hand I can run my own server but then I've still got the clunky web browser interface but now with the added complication of running my own web server. I've also lost the automatic update of the tools that a network would provide.
Still, this guy http://bellard.org/jslinux/ has written a whole x86 emulator in java script that runs and boots up Linux in a web page!!! So I guess writing the entire Spin tool, for example, in javascript is not out of the question.
-Phil
JavaScript is cool because it can be severed up.
Anyway, I've got this Spinneret thing I'm doing so when I get that done I'd like to take a closer look at Phil's HTML-based Propeller IDE.
A network-connected "Prop Plug" would would avoid such limitations, as you wouldn't need to use any USB port. It would also certainly be possible to up/download files to/from the device through the browser, given that it's configured to act as a server.
A full-featured back end written in JS probably isn't too practical. My thought was more along the line of something lightweight that could be hosted by something like a Spinneret, while delegating the comiling duties to the web browser's JS engine. I know this isn't exactly what Phil was talking about, so I apologize if I created some confusion.
What I'm really interested in, and mentioned it elsewhere, is that if a web browser can in fact provide a suitable UI, a Prop2 based web server/compiler/programmer could offer the benefit of native multi-platform support, and the convienience of having all the development tools you need all in a little box connected to the network. Someone has to think this is at least a *little* compelling
Auto-completion priority is alphabetical: e.g. rdbyte is displayed for "rd", not rdlong. A better strategy would be to base the priority on frequency of usage. In any event, it's important that, once a strategy is settled upon, not to change it, because fast typists will become accustomed to typing "rd[rt-arrow]" for rdbyte and would not be amused if it suddenly produced rdword instead due to a software change.
-Phil