Multitasking token threaded Forth
Peter Jakacki
Posts: 10,193
As an embedded system designer I frequently write a version of Forth for the chips that I work with. Number one for the reason that I can investigate the chip and system interactively and make sense of the datasheet and user guides (or lack thereof) and secondly to have the ability to write code that is both fast and compact and easy to debug. I wrote a complete Forth kernel, SD FAST16 file-system, software VGA generator etc in 32K on an ARM7 in 32-bit ARM code rather than the more compact 16-bit THUMB. I am planning to do the same for the Propeller but in a slightly different fashion of course.
At first glance I wasn't sure how I would tackle the job or if I would bother as I was quite happy coming to grips with SPIN. But lately I have come to a bit of a wall with code size and speed and lack of debugging capability. Now I can see how I can implement a stand-alone Forth system that can run without a PC babysitting it. With the addition of SD memory, VGA, keyboard, mouse, and a Forth O/S I can completely develop and debug on the Propeller and break free from the PC (not quite yet though).
It will probably be a few more weeks before I release any code plus I'll be busy moving house but I'm throwing this in to stir the pot and see who might be interested and perhaps more importantly who might like to contribute to this project. I'll leave it at that for now and include the header documentation block from my Propeller Forth (plus some screen interaction from my ARM system) that I have started to write.
At first glance I wasn't sure how I would tackle the job or if I would bother as I was quite happy coming to grips with SPIN. But lately I have come to a bit of a wall with code size and speed and lack of debugging capability. Now I can see how I can implement a stand-alone Forth system that can run without a PC babysitting it. With the addition of SD memory, VGA, keyboard, mouse, and a Forth O/S I can completely develop and debug on the Propeller and break free from the PC (not quite yet though).
It will probably be a few more weeks before I release any code plus I'll be busy moving house but I'm throwing this in to stir the pot and see who might be interested and perhaps more importantly who might like to contribute to this project. I'll leave it at that for now and include the header documentation block from my Propeller Forth (plus some screen interaction from my ARM system) that I have started to write.
**************************************************************************************
8X32A Assembly language Forth Kernel
**************************************************************************************
Forth is both a high-level stack based language and an Operating System.
It is interactive like Basic yet fast, compact, and extensible.
This language provides a high-level interface that gets very close to bare metal and also ideal for debugging.
A cog can run the 32-bit Forth kernel as one task and read 8-bit instructions tokens from main memory.
Either the token points to a code function in the cog or another high-level function
is called in main memory with additional bytes indicating the address if necessary.
IP points to the next 8-bit instruction in main memory
The 8-bit instruction addresses kernel routines aligned on 8-byte boundary in the 512 word cog ram
The inner interpreter:-
:NEXT rdbyte token,IP 'read next instruction
add IP,#1
shl token,#1 'translate to address 512 longs
jmp token
No token lookup table is employed but rather it is assumed that the token points to
one of 256 code locations on a 2-longs boundary. This may waste a long word between some functions
but it simplifies the translation. Anyway, the waste can be used as a general register.
Obviously not all 256 tokens will be used but maybe 64 as each definition takes around 4 assembly instructions.
A second optional method uses a jump table in the cog memory so that any further "opcodes" can be utilized
to call common high-level functions in main memory.
The data and return stack is held in local cog memory.
Heads (names and links) can consume a lot of memory and are not required for run-time operation
and so are delegated to serial memory, either I2C,SPI,or SD.
Text buffers are based in main memory.
Source code files are held and edited in serial memory as well.
Such a Forth system allows the whole Propeller development process to be completely PC free as
the Propeller can handle the screen, keyboard, mouse, and file system.
Many of the 8-bit Forth instructions are only short code snippets and I expect the performance to
be around 2 MIPS per cog. Each cog runs a single task but it is possible to create further tasks on each
cog by loading the multitasking Forth interpreter into the cog and setting the IP (instruction pointer).
Optimizations may include reading a long from main rather than tokens if this can speed things up a bit.
Further enhancements may include automatic overlay into main memory from virtual memory in SD.
Sample disassembly showing memory usage for a high-level file function vs a more basic prtbyte function
SEE FLOAD- .... ( adr cnt|0 -- )
--------------------------------------------------
x0000_1402: LFA FFE2
x0000_1404: ATR F7
x0000_1405: NFA 46 4C 4F 41 44 2D 00
--------------------------------------------------
m0000_27C2: CFA 003E
m0000_27C4: 54 DUP
m0000_27C5: 21 0=
m0000_27C6: F3 14 (IF) >0000_27DB
m0000_27C8: 5B DROP
m0000_27C9: 6E 13F0 DIR@
m0000_27CC: 6E 099F +START
m0000_27CF: 6E 10CE X@
m0000_27D2: 6E 13F9 DIR@
m0000_27D5: 6E 0990 +CNT
m0000_27D8: 6E 10D7 X@
m0000_27DB: 6E 142A FILE@
m0000_27DE: 55 ?DUP
m0000_27DF: 63 06 (IF) >0000_27E6
m0000_27E1: 5A ROT
m0000_27E2: 5A ROT
m0000_27E3: 6E 0E72 XLOAD
m0000_27E6: 7F EXIT
ok
: .BYTE BASE C@ >R HEX FF AND 2 U.N R> BASE C! ; ok
DECIMAL 6789 .BYTE 85 ok
HEX 1234 8 SHR .BYTE 12 ok
SEE .BYTE .... ( byte -- ) Print as hex byte
--------------------------------------------------
x0000_1BCA: LFA FFE4
x0000_1BCC: ATR FF
x0000_1BCD: NFA 2E 42 59 54 45 00 00
--------------------------------------------------
m0000_0
m0000_0BD8: 16 BASE
m0000_0BD9: 3A C@
m0000_0BDA: 4C >R
m0000_0BDB: 35 HEX
m0000_0BDC: 12 FF
m0000_0BDD: 2A AND
m0000_0BDE: 02 2
m0000_0BDF: C6 U.N
m0000_0BE0: 4D R>
m0000_0BE1: 16 BASE
m0000_0BE2: 39 C!
m0000_0BE3: 7F EXIT
ok
Comments
COG 0 Forth
COG 1 Keyboard, Mouse, SD Card, I2C
COG 2 hires VGA
COG 3 hires VGA
COG 4 RS-232 Serial I/O
COG 5 Audio I/O
COG 6 Spare
COG 7 Spare
The keyboard and mouse do not have to be run in real-time but can be polled by the cog when other services such as SD card access is not required.
Obviously some cogs could be released if hires VGA was not required or even RS-232. It is tempting to place some of these more common functions onto another micro such as an LPC2101 seeing they are only around a couple of dollars apiece. But I want to see what I can do we just the one chip.
*Peter*
Has there been any work on this?
Thanks,
Doug
The TCP/IP stack will rely on a Forth implementation as well, plus I have lots of goodies that can run under Forth, including the SD card and file handlers, audio, I2C, RS485 networking, communications etc.
I am trying to stick with the Propeller IDE to generate the kernel but I have to work with the way Spin operates in order that contributed objects can be added to the kernel easily. Once the kernel is compiled with the objects it can be considered stand-alone in that it does not need the PC in order to edit and compile Forth code. So I haven't done as much on the kernel as I would like but all I need is a week or two of burning candles to have it up and running.
*Peter*
We have estimated that the speed-up wrt to equivalent SPIN programs will be at least 2 in a slightly optimized version. I personally think that 4 will be the limit due to the same overhead issues that any interpreter on the Propeller is bitten by.
However there are many more benefits other than speed, as already expressed in above postings.
I welcome this competition!
Though he hasn't posted since November of last year so I wouldn't expect any updates.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Paul Baker
Propeller Applications Engineer
Parallax, Inc.
*Peter*
Do keep us posted on progress ...I like this ..
Peter Said:
"The TCP/IP stack will rely on a Forth implementation as well, plus I have lots of goodies that can run under Forth, including the SD card and file handlers, audio, I2C, RS485 networking, communications etc. "
Regards,
QuattroRS4
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
'Necessity is the mother of invention'
Maybe a third, Cliff Biffle's nose, which is out of joint.
I look forward to alternatives, even as I muddle through making my own.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
I forgot he "binary-ized" his Forth, which is counter to what we were trying to accomplish to begin with·(had he released the·code others could use it as a stepping stone), I think Peter will be much more successful in his approach.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Paul Baker
Propeller Applications Engineer
Parallax, Inc.
Post Edited (Paul Baker (Parallax)) : 7/23/2007 9:17:02 PM GMT
Just to set the record straight, I have not yet released a "PropForth", that was Cliff's, although I did write some fundamental & necessary extension words such as "words" and "dump" by analyzing the output of his Forth. I wonder why he didn't include them in his release as he must have had them written for debugging his own code.
Now if you tried one of my non-public ARM versions I think you would be very impressed, but I'd rather work with Propellers, they are so much more interesting and configurable without all the hundreds of pages and errata that are typical of conventional one-chippers.
I also look forward to the interactive development and debugging that comes with using Forth. I guess I am so spoilt with using Forth that when I use a conventional compiler etc it feels just like when you try and use windows without a mouse, so so awkward. My usually development cycle includes a lot of "what ifs" that are very easy to test out with one liners or even just tickling the right bits from the "command line" (hey, it's much more than a command line).
*Peter*
I am interested in the OS and interpreter side of things. I do not know Forth, but hey, it's just another language.
I will be in Qld later this week, so I'll give you a buz and we can catch up for a Qld Propfest.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:
· Home of the MultiBladeProps: TriBlade,·RamBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: CPUs Z80 etc; Micros Altair etc;· Terminals·VT100 etc; (Index) ZiCog (Z80) , MoCog (6809)·
· Prop OS: SphinxOS·, PropDos , PropCmd··· Search the Propeller forums·(uses advanced Google search)
My cruising website is: ·www.bluemagic.biz·· MultiBlade Props: www.cluso.bluemagic.biz
http://forums.parallax.com/showthread.php?p=882046
I would really love to run a Forth (interactive and fast!) on the Prop but as I had mentioned, it feels like a kludge. The ARM is so much better in this respect for the reasons that it has plenty of memory, optimal instructions, and it's fast. What to do? There are issues with interfacing to standard objects, there are issues with memory and cog usage, and speed. Yet I remember Forth running very nicely in an 8K ROM on a 2MHz 6502 so surely we should be able to do it (well) for the Prop, even on one cog. Maybe.
Your bump makes me think that maybe I should just try for a solid functional Forth first, one that can work well on serial coms at least and then possibly one that uses VGA and SD as a stand-alone. In fact, any kind of Flash or EEPROM memory would be advantageous anyway just to hold the Forth headers. That would be a good start. As to how to interface to other objects I don't know yet. But boy, I sure could use that interactivity and extensibility that has always proved to be so conducive to development and debugging and exploring hardware, I really miss that.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
*Peter*
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:
· Home of the MultiBladeProps: TriBlade,·RamBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: CPUs Z80 etc; Micros Altair etc;· Terminals·VT100 etc; (Index) ZiCog (Z80) , MoCog (6809)·
· Prop OS: SphinxOS·, PropDos , PropCmd··· Search the Propeller forums·(uses advanced Google search)
My cruising website is: ·www.bluemagic.biz·· MultiBlade Props: www.cluso.bluemagic.biz
I missed your post before but we will catch up though and see if we can plan a Propfest! However, Forth is not just another computer language. If you have never used a "proper" version then you won't understand, even if you read all about it. When we catch up I will demonstrate my ARM version, I am very sure you will be impressed.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
*Peter*