Advanced Spin Editor - Project Proposal
macca
Posts: 770
Hi,
Since I'm not very comfortable with the current development tools available for P2 (and P1 for that matter) for various reasons, I started to develop my own tool and want to share what I would like to do, and see if there is interest in such thing.
The base idea for this tool is to have a more advanced source code editor with context sensitive syntax highlight, code assist and continuous real-time compile and error checking (i.e. no need to push a button to compile and see the result).
This is the current tool layout, with basic syntax highlight (provisional name):
It is possible to note how local and parameter variables are differentiated, as well as pri and pub methods (pri are italic).
Moving the mouse over a function or pasm instruction shows a popup description:
If the parser/compiler detects an error, it is highlighted and a popup description linked to it:
Code assist shows a list of candidates with the associated description:
The tool is written in Java with Eclipse SWT (Standard Widget Toolkit) for the native look and feel and can be built for Linux, Windows and Mac-OS for a truly multi-platform tool. I don't have a ready-to-test example to share but if you are interested I can build something in the next days.
Initially I would like to provide a front-end interface for the existing tools (mainly openspin for P1 and flexspin for P1/P2 because those are the only multi-platform tools I know of), then have an integrated compiler to support all functions. I have successfully implemented a P2/PASM compiler that, while not yet fully tested, can compile existing sources (tested with my own code and the Spin2 interpreter source) generating binary-exact files. This is somewhat easy to do, compiling spin is another story.
It would also be nice to have an integrated debugger for P2 able to step through code while running.
At the current stage, the tool is just a bit more of a proof of concept rather than an usable tool. The scope of this post is first to check if there is a general interest in having something like that, and, maybe more important, to see what kind of collaboration is possible to develop the things I have in mind. Collaboration that can span from active code contribution to financial support because I won't hide that, with all good will, I can't develop such a thing just for pride and glory. I would like to make it at least a free-to-use product, possibly open source, but I can't dedicate months to it for free.
So, to recap the planned features:
- Support for P1 and P2
- Context sensitive syntax highlight
- Context sensitive code assist
- Integrated compiler with realtime error checking
- Seamless serial terminal emulation
- Support for openspin and flexspin external compilers
- Integrated help
What do you think ?
Comments
I personally wouldn’t want to switch to a new editor. You will never catch up with the amount of efforts spent on the existing ones such as Visual Studio Code or Emacs in my case. I would lose way too much productivity. Your tool won’t support git, not Python, both which I use extensively in my P2 projects.
And both those (and others) already integrate flexspin and pnut etc.
However there is a way for huge improvements without doing that: implementing the language server protocol. This way the features you aspire to - code completion, context sensitive help, code suggestions and error messages - can be implemented agnostic of the editor. Which is honestly one of the biggest and best things MS has helped create in ages for me.
WRT to the debugger: I don’t see how that’s possible without the hardware supporting it. Which AFAIK it doesn’t. It would require JTAG or some such.
Cross-platform feature parity with PNut's debug() would be nice. Would make it a real contender for Parallax to pick it up and have an official cross-platform replacement of Proptool. Since Proptool comms still doesn't work on Wine.
A very good project. If you can, please add indentation tracking as in the Propeller Tool. In longer Spin programs the indentation is a PITA to the point where I want to quit Spin and go to C.
There are no good universal editor tool for Spin2/Pasm2. The original Propeller Tool has a good editor, but strange language limitations (they don't exist in FlexProp). The FlexProp has much weaker editor features, the syntax highlighting not always works properly and there is no indentation tracing. The Visual Studio Code Editor is an inflated and overcomplicated beast. I tried this several times, always with the same result: $%^&$!@, then uninstall.
One of the best code editing tools for me is the code editor used in Lazarus. There is a syntax highlighting editor component in there, but with very weak documentation (or I don't know where to find it) so I still don't know how to configure the component for Spin highlighting.
What I would like in a Spin code editor is:
and that's all.
IDEs seem to be an intensely personal choice, and we end up with almost as many as there are programmers. Existing Spin IDEs that I know of include:
I'm sure there are more that I've forgotten / don't know about. The first 5, at least, support P2 development, and all but Propeller Tool and SpinEdit are cross platform (SpinEdit works under Wine, IIRC). I'm not trying to pour cold water on your idea, just pointing out that for a niche language like Spin there are already a surprising number of IDEs available. Everyone seems to have their own preferences for IDEs, so if none of these meet your needs by all means write your own. Just recognize that there may not be a huge demand out there for new Spin IDEs...
Yeah, it seems like a nice idea with a lot of great features but I'd hate to see you suffer the same burnout and see the IDE suffer the same eventual disuse that those previous efforts mentioned above did. It's a shame that some of those people aren't with the community any longer and/or their hard work has been abandoned. I also wouldn't use the "verbal support" (the X number of people who say something to the effect, "yes, I'd get behind that") expressed by forum regulars to gauge interest, because the actual volume of interest may be much lower. I agree with the sentiment that if you can, it would be best to develop tools that can be integrated with whatever people are already using.
Cheers
The reason there are so many Spin IDEs is that they all suck, haha.
Propeller Tool sucks because windows-only, no external compiler suppport, kinda buggy. Great because assembly-friendly editor, super fast compile (but slow loads...), DEBUG, good compiler integration (can show constant values, labels, etc).
VSC is great because it's like, an actual IDE, but I haven't find any mode/extension that makes it nice for PASM editing (tab to some particular columns, "Align" mode, etc)
The other ones are all either unmaintained or a bit undercooked.
My first instinct would be to put effort into improving the VSC experience, rather than starting again from scratch. I'm sure the current maintainer would not only appreciate, but eagerly welcome your support / partnership!
Reading back, a few others seem to think the same. Focus on the actual parts that are missing from VSC, rather than burn all your time on repeating what that already does.
Of course, you might just want to learn writing an IDE, in which case it all looks really good !!
My second instinct also tends to VSC
I would also lean in the direction of VSCode. Part of the reason for that is that I have a strong affinity for the VIM editor, and there is now a neovim plugin for VSCode that's actually good and feels like it works well. By leveraging VSCode, there are more root editors available for various peoples' muscle memories to work with while still allowing support for the features you're proposing.
No, sorry, I'm not going to improve support for VSC. If I have to choose an existing IDE I would choose Eclipse because it is my main development environment. Supporting an existing IDE is not magic there is still a considerable amount of work to do, based on past experience, it is almost the same, if not much more, work needed to write an editor from scratch (you need to learn the API which may not be very clear, understand how things are presented to them, how the editor understands the code structure, and a zillion other things). In addition you have to keep up with the IDE releases, deal with API changes, internal changes, etc. It is easy to have something that works today but stops working tomorrow because some developer decided that an API is no longer needed or a feature must be implemented in another way. And (divinity of your choice) helps you, if you find a bug in the IDE and nobody cares to fix. No thanks!
Well, this may lead to a long discussion, but it is not a surprise that projects made by one person in the free time gets abandoned when the interest in them fades away. A different thing would be if there is a comunity around it or the project is backed with enough contributions to keep one or more developers on it.
Of course I don't make any promise on that matter, one reason for this post is to check if there is enough interest and tune the development efforts, if there is not enough interest I'll keep it for myself, fine with that, it isn't the first time, I have a number of tools written for myself that nobody else uses.
Yes, I'll certainly add it, I was surprised to see it in Propeller Tool (never used it before P2) and find it very useful too.
Well, I don't know what my tool will support or not support. I'm using git myself, altough I prefer the command line, so adding support for it would be a good thing. Certainly I don't plan to make it a generic source code editor so maybe Python will not be supported, but who knows ? flexspin can compile C and BASIC, maybe adding support for them would be a future feature.
There is a debug interrupt that can be enabled on a cog-by-cog basis, I absolutely have no idea (yet) how it can work but there is a reason if it was implemented.
You kindof have a point, haha. Because I just mentioned it, I thought I'd try to implement the aforementioned Align mode in VSC aaaand... OH MY GOD THE API SUCKS
There apparently isn't a function to get the length of a range. Or if there is, I can't find it because the documentation is just one enormous page with stuff on it.
That makes sense to me. In general, there are few reasons to mess with byte positions of an active editor document, and byte position length difference makes even less sense. Depending on what you mean by "Align" mode, there may be better ways to handle it.
Some time has passed since I started this project and I would like to show you some progress I made.
The editor can now compile both Spin1 and Spin2 programs producing the same binary files produced by Propeller Tool, with some minor exceptions. Still not perfect and by far not 100% complete but it starts to be usable. A number of examples from the Propeller Tool library works perfectly.
I have added an informations dialog that not only shows the binary program dump:
but also the bytecode listing:
Code assist is now complete with object inspection and function document:
There is also an integrated terminal window that doesn't drive me nuts because it gets disabled anytime I switch away from it.
Still a lot of things to implement but it is taking shape, and best of all I can now develop with Linux.
Looking good.
) was a website editor display that had windows for Source and Listing, but was smart enough to track-scroll the listing as the source scrolled.
A suggestion :
One feature I stumbled across a few weeks back (and I cannot find again
I found it was a great way to quickly scan the created assembler, and they must have used some debug features that tracked the C lines thru to the listing file.
The listing you show above, may be well behaved enough that an extracted sequential-string-match is enough to sync-up the Source and Listing windows ?
Maybe, the listing shows the source statement before the corresponding bytecode, there are some quirks like that dat and methods are reordered in memory and the linear correspondence may not be respected. Anyway this won't be necessary, since the compiler is integrated in the editor it could annotate the source with the translated bytecode, in realtime.
Did you utter madlad just implement a new Spin2 compiler? That sounds really good, want to see that available.
Yes, still not ready for prime time, but soon.
Wow, that looks really great. Thanks for your efforts.
Relatedly, Flexspin is actually really good at compiling Spin1 (and some subset of C) to optimized bytecode now, so I think you'd want to integrate it, too. Since process init is too expensive for real-time, that'd probably require figuring out how to run flexspin as a library without leaking memory (as-is it just forgets about objects that it doesn't need anymore. Would probably implement a custom allocator that can free everything after a compiler run).
Well yes, but it isn't necessary, maybe it wasnì't clear in my post but I have implemented the spin1 compiler too, all integrated into the editor.
It was clear, but flexspin has decently good bytecode optimizations and useful language extensions (object pointers in particular are pretty handy) that your new compiler probably doesn't have (yet).
If you want to implement some of the optimizations yourself, here's the gist (though they're all fairly obvious, IMO):
- Prefer 1-byte constants over bit-encoded constants, they're faster.
- Replace top-level postclear/postset with assignment
- Omit TJZ op at the top of REPEAT N if the repeat count is constant and non-zero
- There are a lot of IR-level control flow optimizations, you can see them all here
I see. Well, no, I won't add support for external compilers. I'll work on the optimizations, some of which I already noticed, when I'm confident enough that the compiler works. For now I need a 1:1 correspondence with openspin (or Propeller Tool).
Why not ? Flexspin also supports many languages.
External compiler/batch etc support is relatively easy to do, and it can actually save you time as it improves the quality of field bug reports, and it encourages more users too, as they know they have a back-up should they hit any issues.
It is great tho, to have a long term goal to not need any external compilers.
my guess is that he need his own compiler for code completion and stuff like that.
Mike
I downloaded and installed 0.19.3 today. How do I tell it where my library is? I added the path to my library in using the Preferences dialog, but it doesn't seem to use it -- but then, it doesn't flag my serial object that is in the same folder.
The test program compiles and runs fine under Propeller Tool and FlexProp.
Thanks for your help.
A few minutes later....
It looks like you're using libraries under the path of the executable. I hope you'll consider allowing the user to specify this path in the Preferences dialog.
I copied my .spin2 library files into the SpinTools library/Spin2 folder and the IDE is happy now. The next issue is with the terminal. Would it be possible for the terminal to remember it's baud rate between usage? ATM it defaults to 115200, but I use 230400. My normal routine of clearing the rx buffer then waiting for a key doesn't seem to work when I open the terminal and change the baud rate. If I change the baud down to 115200 and get rid of the "wait for a key" routine, it does work.
Yes, currently the library path includes only the library/spin1 (or library/spin2) folders under the spin-tools folder. I started to add the library path in the preferences but found some issues and put that aside.
The file browser paths are only used in the file browser, they aren't added to the libraries path, but this gave me the idea to add the current library paths to the file browser...
Yes, I'll add that.
I'll do some tests because usually I comment the wait_for_terminal call since it should not be necessary with my IDE, however sometimes seems it needs a bit of delay before starting to send data to the serial, I think it has to do with switching between the upload baud rate and the runtime baud rate.
I almost forgot this thread... I think it would be better to keep the discussion on the other thread:
https://forums.parallax.com/discussion/174436/spin-tools-ide
Maybe these last posts could be moved there ?