A call to arms, to ARM or not to ARM
Peter Jakacki
Posts: 10,193
Here's a bit of a call to arms for anyone for all those who have an interest not just in Forth, or an SDFS and/or network empowered Prop without having to have memory expansion "cards" or an RPi etc with all that minimal hardware implies for general embedded use.
As you know I have been steadily adding functionality to the Prop even though it seems underpowered from the ARM world or even from a C standpoint. At this particular point I have pretty much everything that would make it a stand-alone development system as well as an embedded target, I even have an assembler that with a tweak could do multi-pass assembly from file so it's possible to assemble and run objects at runtime even, a different slant on my "embedded runtime OBEX". Perhaps I could compile the kernel this way too (yes). The system doesn't just rely on SD cards either as this works with tiny SPI Flash too so a bare minimum Prop system only needs a cheap SPI Flash to be empowered. It doesn't matter what language an OS is written in does it? Or does it? Does it do the job I thought was a far more pertinent question and for which I do know the answer.
Of course the 32K memory limitation affects video display but I even have a VGA text object that only requires 2K RAM for the image which is reused as the video buffer after the cog is loaded. So with a PS/2 or serial keyboard/mouse, VGA display, SDFS FAT32, Ethernet servers and clients we have an embedded system that is also a development system in what is essentially an unexpanded Prop.
But there are lots of little things needed to tie this all together now but I don't know if it is worth it. Putting aside the promise of "P2" I wonder if there is any real further interest in P1 itself, it seems as if the Prop community has fallen flat, there does not seem to have been any real developments and the forum seems to be much ado about nothing in particular. Like the Constantinople charioteers the focus is on the "colors" while the empire crumbles and decays around them, any seeming victory is illusory and actually a symptom of the decay.
I can see busy bees off in their little corners trying to make honey but for the lack of a hive it won't thrive. Even in the fragmented Forth community it's just bits and pieces without any general direction or application, and yet there is so much talent here. So I've been battling my own problems that one or two of you forumistas may know a tiny bit about but all the while I keep trying to contribute and document as much as possible if not practically all, back to the community, which then takes up even more time and effort. I don't even hear crickets somtimes. Is it worth it? Is anyone really interested? If we are then can we work in together more openly?
The alluring tug at my arm as been ARM itself but this is the call to arms I'm talking about, work together and thrive or limp along under our own colors. The Propeller chip needs focus now for both P1 and eventually P2 to grow into more than just hobby projects for Parallax to have a future as a chip company for us to have a future with the Propeller. We, that is meaning "us" as the Prop's ardent ambassadors need to be contributing in a meaningful way and if we all bring something yummy to the picnic we are all going to be sure and certain that we come away full and satisfied and that is something I am very much looking forward to!
As you know I have been steadily adding functionality to the Prop even though it seems underpowered from the ARM world or even from a C standpoint. At this particular point I have pretty much everything that would make it a stand-alone development system as well as an embedded target, I even have an assembler that with a tweak could do multi-pass assembly from file so it's possible to assemble and run objects at runtime even, a different slant on my "embedded runtime OBEX". Perhaps I could compile the kernel this way too (yes). The system doesn't just rely on SD cards either as this works with tiny SPI Flash too so a bare minimum Prop system only needs a cheap SPI Flash to be empowered. It doesn't matter what language an OS is written in does it? Or does it? Does it do the job I thought was a far more pertinent question and for which I do know the answer.
Of course the 32K memory limitation affects video display but I even have a VGA text object that only requires 2K RAM for the image which is reused as the video buffer after the cog is loaded. So with a PS/2 or serial keyboard/mouse, VGA display, SDFS FAT32, Ethernet servers and clients we have an embedded system that is also a development system in what is essentially an unexpanded Prop.
But there are lots of little things needed to tie this all together now but I don't know if it is worth it. Putting aside the promise of "P2" I wonder if there is any real further interest in P1 itself, it seems as if the Prop community has fallen flat, there does not seem to have been any real developments and the forum seems to be much ado about nothing in particular. Like the Constantinople charioteers the focus is on the "colors" while the empire crumbles and decays around them, any seeming victory is illusory and actually a symptom of the decay.
I can see busy bees off in their little corners trying to make honey but for the lack of a hive it won't thrive. Even in the fragmented Forth community it's just bits and pieces without any general direction or application, and yet there is so much talent here. So I've been battling my own problems that one or two of you forumistas may know a tiny bit about but all the while I keep trying to contribute and document as much as possible if not practically all, back to the community, which then takes up even more time and effort. I don't even hear crickets somtimes. Is it worth it? Is anyone really interested? If we are then can we work in together more openly?
The alluring tug at my arm as been ARM itself but this is the call to arms I'm talking about, work together and thrive or limp along under our own colors. The Propeller chip needs focus now for both P1 and eventually P2 to grow into more than just hobby projects for Parallax to have a future as a chip company for us to have a future with the Propeller. We, that is meaning "us" as the Prop's ardent ambassadors need to be contributing in a meaningful way and if we all bring something yummy to the picnic we are all going to be sure and certain that we come away full and satisfied and that is something I am very much looking forward to!
Comments
http://www.xmos.com/products/silicon/xa-series
You then get the best of both worlds.
You have presented a lot of thoughts there to digest. Can I distil the main point down to:
"Anyone interested in a stand alone, self hosting, Propeller development system?"
and hence:
"Anyone actually interested in helping with that?"
With that as the focus a few comments:
You have made super human efforts in that direction. Believe it or not, despite my views of your chosen language, I do follow you developments with admiration.
As an interesting exercise and challenge, creating a self hosting system is a great idea. It's certainly one that Chip himself has expressed a desire to achieve on occasion. That was sort of part of the motivation behind us getting Z80 emulations and CP/M running on the Prop all those years ago. With a simple OS and file system one could imagine creating assemblers or even higher level language compilers for the Propeller itself.
Over in Germany the guys building the Hive project have done it already. They don't get much coverage here, perhaps because all their web pages and documentation are in German.
But what about practicalities?
The Prop is small and slow. So any such self hosting system is necessarily limited.
To use such a self hosting Propeller you will need at least a keyboard and a screen. At that point you may as well hook it up to your PC, laptop, pad whatever. If you do that you have no need for the Prop to be self hosting, you already have a machine that will run any tools you need.
As such the whole concept has limited appeal to the general user. It's just adding complication. I really don't want to go there:) I agree however that it should not matter as long as it works AND as long as it does not dictate language choice to the user. Seems to me a lot of the high fliers have flown off into hacking on the P1 verilog code for their FPGA's
Us others are still very much into the P1 itself. In recent months we have seen a web based IDE for the P1 created. We have seen Props being programmed from cheap off the shelf WIFI routers. Etc. Perhaps these are "fluff" around the edges rather than actual projects running on the Prop but still they are Prop at heart. For me it's not the ARM itself that is the tug. Rather it's that I can run Linux on something very small and cheap. Be it an ARM based board like a Raspberry Pi or a tiny $20 MIPS based WIFI router.
On a more general note, I also am worried that activity on the forums seems to have slowed significantly. One of the reasons I have stuck with the Propeller for so long is the active community I found on these forums. I would hate to see that go away.
I can read all the source code, I can step through interactively and experiment with the part to gain understanding.
And if I spend enough effort I can understand and then even change the thing to my liking.
Of course this is true in general for a linux insider as well.
You can read all the source code, you can study the compiler and you can change it.
But this seems a quite complex endeavour -
while with Tachyon-FORTH by the very nature of the system this is quite easy -
or how could you have all the functionality in Tachyon running with just a 64k EEPROM in 32k RAM.
Regarding the 'Standalone Development System' ...
I do not see a need for it. Even if it could be done, I would in general prefer to just plug a Prop-Plug in from my
Laptop - or use a cheap Bluetooth module wit the tablet, or dial in via Telnet ...
Having a KBD & mouse & display might be interresting for application purposes in it's own right -
but not because I need a standalone development system.
The COMPILER thing is an other area. I think this is mostly useful, because there is no
PC-based compiler for the full tachyon system available. And this makes it a little tricky to use different tools
to create the PASM modules to be loaded into the Tachyon system to make even more effektive use of the
multiple COGs.
The idea of having a kind of OBEX on SD card with modules that can be loaded at runtime is a great way to expand the system.
imo this should be acompanied by a Tachyon OBEX on the web, where the community members can share their code pieces and modules.
A few minutes ago I was looking how to interface Tachyon to a Nokia 5110 LCD display - I had a faint memory of it having been discussed - and searching the Forums I found a few discussion fragments - even for Tachyon -
but there is no place to look for the results of those exchanges.
Should we put Tachyon pieces on the regular OBEX ?? or somewhere else?
so much for today ...
I know I could spend more time with Forth and get more proficient but I can't see that knowledge being transferable to any other system. I admire what Peter has done with Tacyhon but I don't want to adopt it as my primary development language.
For sure this is not possible with Linux. Nobody has the time or capacity to understand all of it. That's just the kernel never mind the user land programs never mind the compilers etc. Although it is possible to write simple Linux drivers without a life time of experience and study.
Thinking about it, when I said "Rather it's that I can run Linux" even that is not quite correct. Linux is not the main point here. Could probably as well be BSD or Minix or whatever else. Point is it has all the facilities for running familiar tools like editors, IDEs, git etc. It has all the netwoking and file system facilities you need. And so on.
David Betz, I have noticed that as well. What is going on? It cannot be that everything a Prop can do has been done!
Be aware that the XMOS device linked to is not actually an 8 core device.
It is an 8 "logical core" device. That is to say a single core that can run 8 hardware scheduled threads. It's built such that the speed of each thread is constant as you use from 1 to 4 threads, after that thread performance drops with the addition of each new thread. Mind you, given the speed of the thing for a lot of the time, up to a point, that is indistinguishable from 8 cores.
Also note that the XMOS cores only have access to 64K RAM. That's for all of them.
Also note that chip comes in a 256 pin BGA package! Do you really want to deal with that?
I'm hoping that the P2 will stack up quite well against the XMOS devices of similar price.
Having a little ARM core on chip is fine and all but if its not got the chops to run a full up Linux then there is little point from my point of view.
Real time cores plus and ARM may well be the "best of both worlds" but I think many would see it as just stacking up the complication of both.
of course you COULD have a high level language compiler that compiles to Tachyon codes ...
but practically I think you are right
to work with Peter's Tachyon you have to adopt this as your development language.
This also means you are based on the very small Tachyon community for new device drivers and code snippets ...
or you have to do it yourself - as Peter is doing it.
This will propably not be the approach that very many will take - what a pitty for them ...
it took me quite some time to adapt to Tachyon - I still read BASIC and even C faster ...
You know LISP, so you can value the flexibility that a FORTH gives, to change everything,
that you do not get in any classic procedural language.
Tachyon + PASM:
I think this is a great combination with Peter's new Tachyon embedded Assembler.
Where you can assemble new PASM code modules INSIDE a running Tachyon system.
Save them and load them to a COG.
A flavor of STANDALONE - which can easily be used with a device, that provides Display and Keyboard -
whatever this will be - connected via serial / LAN / WLAN / Bluetooth ... whaever you like.
I rather have a Laptop or Tablet with me than a spare Keyboard and a VGA-Display ...
e
Like many others I've been caught up in the P1 verilog stuff.
While it's been a "BIG" distraction it has reinforced my appreciation of Propeller 1
Starting a new project tomorrow and Propeller will be specification #1.
I like Propeller's because they make me appear smarter than I actually am!
Quite so. The P1 is not really comparable to the XMOS. Much slower, less memory and so on.
I'm hoping the PII is a lot more of an XMOS peer.
Linux on embedded systems running from FLASH, like WIFI routers, should not be taking an age to boot.
I agree that Linux is not required on all embedded systems. Especially when handling real-time events. But given that it now runs on things the sized of an SD card or an ethernet socket it is finding it's way into all sorts of places.
http://mbed.org/
It can be used with your own servers apparently but if you are going to do that why use mbed in the first place?
Peter: What are the languages you plan to support for developers using your system?
while Peter is asleep, let's think about it:
- the OS is TACHYON.
- there is an embedded PASM like assembler with some syntax tweeks to make it easy for Tachyon to parse it.
- the assembler can be used to convert PASM source from file to loadable modules to be loaded to free COGs.
- the main COG is running TACHYON
- other COGs might be running Tachyon as well to execute parallel tasks, handle drivers ...
- some COGs can be dynamically loaded with prepared PASM modules to run timecritical / driver etc. code
the HUB RAM is basically managed by TACHYON, some sections can be allocated to be used by the PASM COGs.
so how would other languages fit in here?
could they coexist in this framework - and if so how?
would it make sense to do so?
for having a really self contained system you would need a compiler for the other Language, that would need to run in
Tachyon - hmmm
so write a C-Compiler in Tachyon ?
would it compile to Tachyon byte codes?
or directly to PASM ?
---- I don't think you would get this from Peter ...
how would other highlevel languages coexist with Tachyon?
would they be able to 'call' the OS-functions provided in Tachyon ?
actually I can't see a practical model for another high level language on top of TACHYON.
Memory is practically full now with SDFS/FTP/TELNET/WebServer/Email
of course if you had a PASM COG module, created from any language, that will accept the
conventions of the Tachyon system, I see no reason why they should not be loadable to a COG.
... and still ... would this make any sense ??
just my thinking ...
When the prop is running, the average user could care less if the program was written in C, TACHYON, or Klingon!
I for one have no desire to create something which can be ported into an inferior, IMO, chip like the R-Dweeb-No, or some other Microcontroller.
I would not say other MCU's are inferior. It all depends what problem you are trying to solve. Often having portable code is actually part of the solution for various reasons above and beyond running on the target device. For example, ease of development on a host machine, testing, simulation etc.
Except for the Arduinos of course, that goes without saying
All in all I agree with you. The Prop, it's architecture, it's instruction set, it's assembler language, Spin and the PropTool IDE are a coherent whole. All designed together by one guy with a clear vision of what he wants in a system. Nice and simple to use, and works very well. With the simplicity and speed of turn around of a tool like the PropTool I feel no need for an interactive environment on the Prop itself.
Is the Klingon user input error message: "I should kill you where you stand! "?
Its a little wordy for a 32k byte system, but puts a whole new spin on "user friendly".
What was the topic again?
Yes, "I should kill you where you stand! " is a bit wordy. I have no idea about Klingon but supposedly they say such things quite a lot. So we might guess they have condensed these error messages down to single words that summarize how you are going to die and perhaps why.
The topic? I'm not sure. We are still waiting on Peter to come back to us on that.
I'm waiting a bit to see if Chip gets an FPGA image. I'm very eager to jam on P2.
The small amount of P1 activity I've got going on is in tandem with an Apple ][ computer. Nothing I want to share yet, though I may soon as I may have some questions.
Put simply, now is a good time to take a little break, get my head wrapped around a new, and pretty great job. When Chip is ready, I'm ready.
As for an ARM + P1 kind of system. Maybe! Really depends on if there is a development board I find appealing. For those people wanting to do bigger things, doing this makes sense. Go for it. Maybe a lot of us see the appeal and jump in.
I know Chip is deep into it, maximizing what we all learned so far. I'm feeling good about it all and am perfectly willing to wait it out. No worries here.
However the capability to load in runtime objects helps overcome some of the memory limitations of what is otherwise a "microcontroller", albeit a very powerful and unique one at that. Be that as it may, neither is the implementation of software or hardware the focus here either although it has been demonstrated that essentially "we have the technology".
I suppose my lament which is what it is in effect is about the P1 ending up a bit like a much loved but "deprecated" 6502/Z80 etc. Even to this day there are groups writing code for these devices and designing hardware but they are just a hobbyist's pet and who would really use them? The P1 is being used to essentially "blink LEDs" in many projects, mundane things that any micro could do and no real reason to use the Prop. Given that other micros are cheaper and self-contained with oscillators and Flash etc you might rightly wonder why anyone would use a Prop chip.
So then, why use a Prop? What is it good for?
To which I might reply:
Why might I not use a Prop, especially the P1?
Okay, some of these points can be argued back and forth but if I had to blurt it out on the spot then that is basically what I would say.
Now however I am inclined to use the P1 with the further hope that someday P2 will alleviate many of these restrictions to some extent but by that time it will have to compete for attention with even more advances in technology. So back to P1, if there is a nexus where a vast plethora of embedded skills can come together and focus on polishing the rough edges, in filling in the gaps, and testing and documenting, then I think that the Prop would hold a lot more appeal to many many more who at present rely greatly on libraries and examples as with other unmentionable chips. More appeal, more zeal, more zeal, more appeal and so on.
Though not my original intent such a nexus could be the Tachyon VM itself, the cog that executes fast bytecodes rather than Spin or CMM or LMM. Why? because it works and it works well even if I say so myself. The Forth that is built upon this Tachyon VM can be viewed as an OS if you like but the reality is that any compiler can be written to eventually compile down to these bytecodes. LMM sounds great but it takes four memory hogging bytes for each simple machine instruction and can only run at hub speed at best (if not cached). Contrast this with the Tachyon bytecodes which are mostly one byte, take from 400ns to execute but also include more powerful instructions as well as efficient operations for SPI etc.
Where's this heading? Exactly, let's choose a direction and head in that direction with a purpose, each doing our little bit, contributing in a meaningful and useful way and have a lot of fun while we are at it.
@David Betz: If some source code is compiled to Tachyon VM bytecodes then C, or D or E but what does it matter as long as it can all be integrated seamlessly if possible. I would think that an FFT might just compile to PASM to run in it's own cog but applications themselves as well as many drivers would be better off as more compact bytecode. BTW, it takes only a couple of milliseconds to read in sectors from an SD card (or SPI Flash) straight into a cog so cogs could even be preloaded with a "loader" which may even stay resident, just pass the sector number to it and it's up and running almost immediately so this lends itself to overlays. This is something I may play with a bit more now that I mention it.
Tachyon VM bytecodes are not fixed and could change because each code is actually the 8-bit address of the first 256 longs in the VM cog. So there is no lookup as such, it is simply a matter of reading in a bytecode and jumping to that address, simple and fast. The return is a vector that normally points right back to the doNEXT bytecode loader which uses an IP instruction pointer that is directly analogous to the cog's PC.
To access some less used codes in the second "page" there is the XOP which will read in the next bytecode as the 8-bit address offset by $100 and this is useful for some less used and slower but necessary instructions such as CMOVE and EMIT for instance.
Now you mention register based rather than stack based addressing but really the VM uses a hybrid approach as it always expects operands to be in a fixed position simply because the Prop's architecture can't efficiently address base+offset and we need it efficient and compact. So originally there was only a 12 operand data stack which was fixed in cog memory but later on I introduced the hybrid stack where I kept the top four operands in fixed cog memory and the overflow in hub if necessary. So therefore an ADD operation reads in two registers in fixed positions and writes the result to the second register while "dropping" or discarding the first.
As you can see just from doNEXT and PLUS itself that this is both fast and efficient but the stack pushing and popping takes time too which is why I try to avoid this in some repetitive operations and simply use other fixed registers referred to as COGREGs.
12 34 LAP + LAP . .LAP 46 1.000us ok ( just time the PLUS operation which includes a DROP)
LAP 12 34 + LAP . .LAP 46 4.200us ok ( both 12 and 34 compile as 2 bytes each so we are timing 5 bytecodes)
LAP 4 16 + LAP . .LAP 20 3.600us ok ( use some fast constants of 1 byte each for 3 bytecodes total)
LAP 16 2/ LAP . .LAP 8 1.800us ok ( push a fast constant and shift right)
16 LAP 2/ LAP . .LAP 8 400ns ok ( a single non-stacking operation takes 8 cycles or 400ns)
Anyway, all the information is online in various Google docs and Dropbox folders as well as the voluminous Tachyon thread from which anyone can more than just glean from. I'm just throwing this idea out there although as I stated previously that was not my original intention. I tell you, the countless times I see "problems" on the forum that aren't problems at all, but the problem is in the insistence on a certain approach such as I want to do this in Spin and it's not fast enough
Now, I know when to shut my mouth up, at least most of the time I hope, but I feel like waving my hand in the air so many times and saying "come over here and I will show you how easy it really is". Instead the forum becomes clogged in "problems" that seem to go on unresolved. Imagine what that looks like to newcomers, a forum full of unresolved problems over rather trivial things? Doesn't paint a pretty picture for the Prop.
My way might not be the perceived norm but it really works and it's fun but I'm only one person, imagine what can happen if we choose a direction and work together?!
The underlying language for this is totally irrelevant unless you want to add OS features.
Many times a few of us have tried unsuccessfully to get collaboration on a standard propeller OS.
IMHO, a propeller OS that could also compile would be a great feature to have for the hobbyists. Would we really compile on the prop? Well mostly no. But being possible makes the OS much more usable for the hobbyists. This is not going to help commercial prop uses, but it might get more using the prop and make it more noticeable to the wide world where that could result in more commercial users. Besides, a large hobby group could use the prop in decent quantities.
There is absolutely nothing to lose in joining forces and creating a propeller OS with Peter. It could just be the turning point to get all the various languages running under the same propeller OS. From a hobbyist point of view, imagine an OS with the ability to run C, C++, Forth, Tachyon, Basic, Spin, PASM, Spin+, etc, etc.
Here are the reins and the saddle, I've come this far but surely you are not asking me to do it by myself all over again too? Putting aside "language" we should always consider Forth both a language and an environment or O/S even and just like I can make Forth look like an assembler I can also make it look like a Spin/Basic/C compiler if I wished. But Forth code maps one for one essentially so I have tight control over the code and for me it's a fun interactive and fast language to develop in. Spin/Basic/C are just languages with their very strict syntax and typing without the flexibility I myself need to have to stay creative. Yes, I can write in any language and like many of us we have had to code in many but I prefer the non-swearing kind that sweet talks back to me interactively and one that I can teach new tricks. I would like to think we have gone far beyond submitting our code in one great big "punch-card" blob and crossing our fingers.