[FYI] PropWare IDE: A web IDE for PropGCC
DavidZemon
Posts: 2,973
There have been numerous discussions of web IDEs for the Propeller, and one with an in-browser, JavaScript, Spin compiler even! A pretty excellent application I think. Unfortunately, C and C++ were never supported by that application, nor any other.
I've finally begun work on just such an application, and I call it PropWare IDE. This thread will house all discussion related to PropWare IDE as well as its current status.
Current Status: In development
Implemented features:
Technical Details
Client Source Code: https://github.com/DavidZemon/PropWareIDE-Client
The web client is written in a RESTful fashion using the AngularJS framework and Ace text editor.
Server Source Code: https://github.com/DavidZemon/PropWare-Server
The server uses Java 8 with the Spring MVC framework and invokes PropWare as the build system.
Screenshots
I've finally begun work on just such an application, and I call it PropWare IDE. This thread will house all discussion related to PropWare IDE as well as its current status.
Current Status: In development
Implemented features:
- Unsecure user accounts (no password authentication)
- Open, close project
- Open, close, save, create, rename, delete files
- Delete, rename projects
- Build projects (this will expand into numerous sub points soon)
- Add some pretty CSS
- Change tab theme to dark when the editor theme switches
- Show alert when errors occur
- Support version control (perhaps easier to just provide a terminal, though at least "import" would need to be a gui)
- Import
- Update
- Commit
- Allow downloading the CMakeLists.txt file and commiting to git (only a "feature" if the file is hidden from the user, otherwise it's just a file like any other and this feature is irrelavent)
Technical Details
Client Source Code: https://github.com/DavidZemon/PropWareIDE-Client
The web client is written in a RESTful fashion using the AngularJS framework and Ace text editor.
Server Source Code: https://github.com/DavidZemon/PropWare-Server
The server uses Java 8 with the Spring MVC framework and invokes PropWare as the build system.
Screenshots
836 x 405 - 38K
Comments
I will be using PropWare as the build system, which is based on CMake. CMake supports doing all kinds of things that I do not want users doing to my server. If I gave unrestricted access to execute whatever CMake script they pleased, they could easily wipe out my entire server. Obviously, bad.
One option is, I don't allow them to create their own CMakeLists.txt files, but rather provide checkboxes and input boxes for various options and custom compiler flags. This would be along the same lines as SimpleIDE, where the specifics of the build system are completely hidden from the user. I'm sure many users will like this option a lot because of its simplicity... but it severely restricts those that know what they're doing and want to do something just a little outside of the norm. I don't like restricting people, so if possible, I'd like to provide a better solution.
Perhaps docker instances could be of use here??? Any other ideas? I thought about creating a user account on the server for every user that logs in, but haven't thought that all the way through. It's also a lot more work than mimicking SimpleIDE's solution.
I would have gone for react.js in the browser rather than angular, and node.js on the server rather than Java. But that's just me.
I would not want to be creating actual accounts on the server for ever user, that would become unmanageable wouldn't it? And I'd worry about the security of it. Rather just create user accounts for the web service, user data stored in a database, like any other web service.
Off the top of my head I'm thinking:
1) Have the actual compile "engine" separate from the web server. It can be background service with it's own RESTful API. Jobs are sent to the compile engine as they come in. The compile engine need not know anything about users and so on, just source and build file in, binaries out.
2) That compile engine could be another virtual machine, or it could be running inside a Docker container. I like the Docker idea.
3) Don't worry about people screwing up the system with evil CMake files. Just start up a fresh Docker compile engine instance for a user when he logs in, kill and delete it when he logs out. Or perhaps better still use a fresh instance for every build, that way a user can never get himself "stuck".
Thanks. I've wanted to get this written for a long time.
I've certainly heard lots about react - but Angular is what we use at work, so I'm already very well familiar with it.
As for Java vs Node: I don't like JavaScript :P. I much prefer languages with types that I can declare, and a compiler to double-check my work for me. I thought about doing it in Python (since I know the ins and outs of it, along with PyCharm, and can get proper type-checking), but again, we use Spring MVC at work so I already knew how to code it up.
Yea I didn't really like the idea of a linux user account for every user. It just floated to top of my head as an idea.
Eventually, when I figure out how it works, I'll be using Google and Github for user authentication. I want to offload as much of the security as possible - I don't want that kind of responsibility on my shoulders.
Once logged in, I'll store user metadata in a database. Specifically: what project and file they last had open as well as their theme preference. I could store file contents in the database too... not sure yet. For now, a local file system works fine.
This is a very interesting idea... I might do that. I like the idea of separating out the compile engine from the rest of the server a lot.
These two points are definitely the holy grail goal. Getting there could take some time, since I am only familiar enough with Docker to know what it is and that its overhead is low enough to be useful for this.
I don't mean to insult your work, but what is the point of a web IDE?
Because it's there. Because we can. Because everything is moving to the cloud.
More seriously, using the browser and not having to install anything locally has some advantages:
With a web IDE I could be hacking Prop code at the moment using this public PC. Or on my friends PC's or work PC's where I'm not allowed to install anything.
With a web IDE I could be hacking Prop code from my Chrome book or any other machine.
If my Prop(s) happened to be connected to the internet and in some remote location I don't need any tools installed locally. Just hack and build the code in the web IDE and have the server send it to my Prop(s)
Actually storing files and running the compiler locally is not possible from a browser. Unless of course you have SwimDude0614's build server running locally as well. Might as well use SimpleIDE in that case.
That must have been a typo. I'm sure you meant to say:
Heater mentioned some good points above - mostly, being away from your main machine where you have everything stored locally.
And if you want to do it locally PropWare, the build system, is readily available for you to download and install.
I forgot to mention two important features which will be integral to the IDE:
1) Import project from git
2) Allow downloading the CMakeLists.txt file and commiting to git (only a "feature" if the file is hidden from the user, otherwise it's just a file like any other and this feature is irrelavent)
At least github support and perhaps bitbucket?
Specific websites are questionable. I'm more likely to support just a URL, and that URL can come from wherever you please.
I'd certainly accept pull requests for code that pings GitHub and BitBucket for a user's repos and then returns the link!
Certainly github has it's own API, I have no idea if it is any way compatible with bitbuckets.
But, who cares? Git is distributed. One should be able to do "git pull", "git push" etc from anywhere without an REST API.
It might get horribly messy when the user encounters conflicts though...
Yes... that scares me more than any other part of this project lol. I'm quite undecided what to do about it. One idea is to just give the user access to a terminal, vim, and emacs and let them deal with it.
c9 has github integration. I played with it a long while ago. No idea what it does with conflicts.
Updates
Spent this evening refactoring some things and enabling the rename and delete file buttons.
Next up
Tomorrow is the "new project" button, which hopefully won't take long. Once that's done, I think I'll be ready to start on the real build server.
Build Server Configuration (& security)
I've put a bit more thought into it. Version 1.0 will not use docker, nor will it support modifiable CMakeLists.txt files. I will implement options with checkboxes or dropdown menus, much like SimpleIDE, and I think that will satisfy 95% of users. Unlike SimpleIDE, I will not provide any user input text boxes for custom arguments. This decision does limit the flexibility, but it takes makes it impossible for someone to hack my server (via the CMake scripts at least :P ). None of these features will be hard, and it will get a usable IDE into the hands of users reasonably quickly. As time allows, I will release v1.1 which will add one key feature: the ability to download your project as a compressed archive (aka, a zip file with your source code and the CMakeLists.txt file).
Version 1.2 will be a complete change to the build server, implementing Docker and fully customizable CMake configurations. No idea when that will happen. v1.2 also depends heavily on how much disk space each docker instance requires, how much time a new docker instance needs to start up, how many users sign up, and how much money I feel like paying to store everyone's docker instances (disk space might be very cheap, but very cheap is a far stretch from free).
The most basic features are now implemented! I can create files, save them to backend, tell the backend to build, and if successful, the binary is then downloaded!
Lots more work is needed before I can call it v1.0, not the least of which is the ability to create a new user :P (and log in securely, through Google/GitHub/Bitbucket/etc)
BUT, for those willing to bear with lacking features, I'm ready to release the URL: http://david.zemon.name/PropWareIDE. Right now, only one user exists: "davidz". Feel free to log in as that user and poke around.
Seems to work as expected here. Chrome on Linux.