JDForth: Forth-to-Spin compiler
Carl Jacobs
Posts: 41
What is it?
What does it do?
Who's it for?
Anybody who wants a little more performance out of their propller without having to resort to propeller assembly language. It's also great for writing bits of assembler that get included dynamically.
Included is the DeSilva Fibonacci demo. Fib(28) takes 28.7 seconds in pure Spin, 6.5 seconds in pure Forth and 0.86 seconds in "in-line" assembly in JDForth. See http://www.jacobsdesign.com.au/software/jdforth/fibo.php for more details.
Can I try before I buy?
Yes, the attached version will run in a demo mode until the end of November 2008, but only for new users!
What resources does it require?
1 cog per Forth object invoked.
What libraries does it have? (Alphabetically)
These libraries are in addition to the base libraries.
Anything else?
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Carl Jacobs
JDForth - Forth to Spin Compiler http://www.jacobsdesign.com.au/software/jdforth/jdforth.php
Includes: FAT16 support for SD cards. Bit-bash Serial at 2M baud. 32-bit floating point maths.·Fib(28) in 0.86 seconds. ~3x faster than spin, ~40% larger than spin.
Post Edited (Carl Jacobs) : 11/2/2008 10:43:18 PM GMT
- A Windows Forth to Spin compiler. No Mac/Linux version.
- Most of the demos provided use only 1 cog! Eg: High level code + FAT16 code + SD card driver + 2M Baud Simple Serial Comms = 1 cog!
- Support for breakpoints and single-stepping.
- Ability to compile an interactive forth environment.
What does it do?
- JDForth takes Forth source files,
- combines them with a forth kernel (similar but different to the spin interpreter),
- optimises the result to within an inch of its life, and finally
- generates a Spin object that may be used like any other Spin object.
Who's it for?
Anybody who wants a little more performance out of their propller without having to resort to propeller assembly language. It's also great for writing bits of assembler that get included dynamically.
Included is the DeSilva Fibonacci demo. Fib(28) takes 28.7 seconds in pure Spin, 6.5 seconds in pure Forth and 0.86 seconds in "in-line" assembly in JDForth. See http://www.jacobsdesign.com.au/software/jdforth/fibo.php for more details.
Can I try before I buy?
Yes, the attached version will run in a demo mode until the end of November 2008, but only for new users!
What resources does it require?
1 cog per Forth object invoked.
What libraries does it have? (Alphabetically)
These libraries are in addition to the base libraries.
- NEW Compilers.jd4 - words required to generate an interactive image.
- Debug.jd4 - Support for single-step debugging with breakpoints.
- EEPROM.jd4 - Read/Write the boot EEPROM, as well as other I2C EEPROMs.
- FAT16.jd4 - Thomas Rokickis fsrw.spin translated to JDForth.
- I2C.jd4 - Low level I2C routines.
- FloatMath.jd4 - A translation of FloatMath.spin - 32bit floating point maths.
- FloatString.jd4 - A translation of FLoatString.spin to forth.
- SDDriver.jd4 - Low level words to communicate with an SD card.
- SimpleSerial.jd4 - A translation of SimpleSerial.spin to forth that supports 57600 baud.
- SimplexSerial.jd4 - Uses inline assembler to do bit-bash serial at 2M baud.
- Throw.jd4 - Words to do exception protected code.
Anything else?
- JDForth uses it's own assembler - Propellent.dll is not used. The syntax is the same as the Parallax syntax, but several helper opcodes have been added. The fibonacci demo shows most of the extensions.
- JDForth has been designed to be productive and fun to use! It looks a little different to your run-of-the-mill IDE.
- JDForth has "Quick Help" which results in less gymnastics to get help on forth words, and Ctrl-D which jumps to the definition of the word under the cursor.
- Most of the libraries have a little example project to get one started.
- For everything else you need to know, go to http://www.jacobsdesign.com.au/software/jdforth/jdforth.php.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Carl Jacobs
JDForth - Forth to Spin Compiler http://www.jacobsdesign.com.au/software/jdforth/jdforth.php
Includes: FAT16 support for SD cards. Bit-bash Serial at 2M baud. 32-bit floating point maths.·Fib(28) in 0.86 seconds. ~3x faster than spin, ~40% larger than spin.
Post Edited (Carl Jacobs) : 11/2/2008 10:43:18 PM GMT
Comments
I'm intrigued by the hint on your website that an interactive Forth environment is possible. Can you expand on what you mean? Not a self-hosted JDForth, I presume...
Can you tell us more about the push, r-push, and related·"opcodes" in the Fibonacci example?
I'll be downloading the trial version as soon as I get a couple of things off my plate.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Michael Park
PS, BTW, and FYI:
To search the forum, use search.parallax.com (do not use the Search button).
Check out the Propeller Wiki: propeller.wikispaces.com/
To answer your questions:
JDForth does not invoke Propellent. I tried that for compiling the assembly language sections, but it proved too confusing to work out how to get the error messages back to JDForth. JDForth allows you to double click on error messages to get to the line of source (as you'd possibly expect), and mapping from the Propellent DLL back to my own·source code was more difficult than writing an assembler!
You don't actually have to open the spin file that's generated by JDForth using the Proptool. Using Proptool, make a test harness program that includes the generated spin file as one of the objects. My development cycle for testing has been: (Make sure the JDForth spin file is NOT open in the Proptool).
Interactive Forth Environment: Traditionally, forths are interactive. There have been two forths previously announced/published on this forum (that I know of), and they have both been interactive. BUT, they've consumed the entire·Propeller in the process - there's no space for spin objects. What I'm planning to introduce later this year is the ability to compile the·forth source in such a way that all the word headers are left in place. With·that, you can implement a traditional forth interpreter that is·still a spin object.
Extra opcodes: The following opcodes have been added to JDForth PASM (Propeller Assembly):
So in reality - no new opcodes have been added to the assembler, but these macro expansion make assembly language much easier to write in JDForth. The assembler also gives hints if you miss out the # in a jmp or call instruction.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Carl Jacobs
·
Looking forward to giving JDForth a try.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Michael Park
PS, BTW, and FYI:
To search the forum, use search.parallax.com (do not use the Search button).
Check out the Propeller Wiki: propeller.wikispaces.com/
A few comments ( V08.08.08 Pro ) ...
* Can you either force the Editor background to be white or preferably allow colouring of that and syntax high-lighting to be selected by the user - doesn't work with my Windows Colour Scheme of white-on-black.
* Probably related to colour scheme - Typing in the Editor window causes horrendous flickering / black-white redraw on every character typed.
* Can there be a mechanism to hide "Hint : Do you require a # for destination ? Line 51 {JDForth}\lib\kernel.pasm", especially when it is the default kernel. Maybe allow 'jmp *reg' or similar to mean "I know there's no #, it is jump indirect/via a register".
* Editor Line/Col tracking seems a bit hit-and-miss. Not sure exactly what makes it change or track but just clicking on a line doesn't.
* [noparse][[/noparse]X] close all files and it leaves an "untitled.pasm". "Create a new JD4 file" seems to do nothing at times ( esp after [noparse][[/noparse]X] close ).
* Open a .JD4 file and its tab / source isn't automatically selected for editing.
* Create a new file / open an old one, click "Compile the current file" and it compiles a previous file selected.
* No "Save As" option.
* Open an old file, edit it, open the same file again and it loses the first edits. "Create new JD4 file" and that loses the first "untitled.jd4".
That's all pretty minor stuff, what it does is amazing. One suggestion for the future might be to put the generated kernel code/longs in a separate sub-object which would save wasted code space with multiple Forth objects, leave PropTool to optimise away the duplicate.
You may be pleased to know the IDE works under Windows 98SE.
Wow! I'm truly thankful for the time you've taken to write about your experience, and the bugs found. I have a couple of comments. If I don't comment on something it doesn't mean I'm ignoring it - it probably just means I'm going to go and fix/change it.
* I'd like to introduce fully configurable syntax colouring, but not yet. I'll force the editor background to white - for now.
* Since I wrote the assembler in JDForth, I think that using * as a mechanism to avoid the kernel hint is probably a good idea that I can and will implement. I just like the idea of having a warning when the # is missing.
* The file to compile is selected by using the right menu to "Set the Top File" - which is what's compiled from then on - using F9. I went back and forth between how it worked, and in the end this felt better to me. I kept compiling system.jd4, which of course is a senseless operation.
* Right click for Save-As.
The kernel is optimised during the compile, so wont necessarly work with a separately compiled forth file. So I'd recommend doing multiple forth objects as follows:
* Set a single top forth file which includes the other forth files and has the various spin entry points.
* Create your multiple objects - which all get their own stacks and user variables.
* The Forth and Kernel code are in the DAT section, so will only be included once.
* If you know what you're doing, then you can even supply your own kernel which could be quite tiny in size.
If you're really curious, then the kernel that gets generated during the compile is saved as {top file name}.kernel - which may be opened (with syntax highlighting) in JDForth.
Hippy, I had a suspicion that you're on a Windows 98 machine from other posts of yours that I've seen. Could I please know what sort of spec machine you're using. It has the greatest bearing on the syntax highlighter - which is my first attempt, and begging for some more optimization.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Carl Jacobs
JDForth - Forth to Spin Compiler http://www.jacobsdesign.com.au/software/jdforth/jdforth.php·
Good point about the kernel being optimised so likely different each cog - I'd wondered what the *.kernel files were
I agree the Hint for missing # is an excellent idea, just a pain that it warns about the kernel which one assumes you got right
As to the PC from the Ark ... 1GHz Celeron, 384MB RAM, seems comparable in benchmarks with a 500MHz Pentium.
I also use more "elderly" equipment. My current PC is ex-government, bought from a second-hand PC shop. Even so it's a 1.8GHz P4 with a not-quite-bottom-of-the-range nVidia graphics card.
PC spec is most going to hurt the editor. I'm sure you'll find it still compiles in a blaze!
Only one other performance question - does the quick-help feel "quick", or does it feel like it's sucking the life out of your system? I could have an option to switch it off, but my guess is that it should feel fine.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Carl Jacobs
JDForth - Forth to Spin Compiler http://www.jacobsdesign.com.au/software/jdforth/jdforth.php·
Am I correct in assuming you are using the Spin Interpreter?
I have never looked at forth, but as soon as I have finished my Fast Spin Interpreter (Ram version) I will give it a try. In fact you probably have some test suites which could give my Interpreter a work out. Unfortunately, I have been away most of the last 6 weeks and haven't progressed much.
I guess you and I are simply on different paths to the same goal - More Performance. The output from JDForth is a spin file which has two components:
1) A Forth kernel - which performs a similar function to the Spin Interpreter, and
2) The forth bytes codes - which are like the spin bytecodes - except that they're forth!
What JDForth has - the big performance advantage - is the ability to have PASM words written which either get combined with the kernel at compile time, or compiled with the kernel at run-time. When combined at compile time you get a great performance advantage, when combined at run time you have a slight performance hit, but you can overlay many PASM words into the same COG memory space. I think the cog has about 120 longs free if all the kernel words are required, which is enough to implement lots and lots of good stuff. I've even been thinking of moving the largest of the words */MOD out of the kernel and into the system file. That way you can define it as CORE: if you're going to use it a lot, or define it as PASM: if you're only going to use it occasionaly.
I've got a couple more optimisations planned for JDForth, and one of them is to automatically promote all PASM: words to CORE: if there's sufficient space in the kernel for all of them to fit!
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Carl Jacobs
JDForth - Forth to Spin Compiler http://www.jacobsdesign.com.au/software/jdforth/jdforth.php·
@ Carl : Syntax highlighting and quick help seems quick enough to me.
The use of right-click for menu is perhaps a little less intuitive than you think, even with the "Right click for menu" hint in the status bar. Good idea when used to it, but I'd still add traditional toolbar buttons or top of window menu text if I were you.
Renaming the "key" tool-tip from "compile the current file", to "compile selected top file" would probably get round most confusion there.
Now I could set up a counter to help me do the copy as follows:
:kC1· rdlong :kASM, CTRA
Or I could load the assembly code in reverse order. But,·what is presented is nice and simple and still allows for HUGE performance gains. eg: Fib(28) takes 28 seconds in spin, and 0.86 seconds using the presented code overlay mechanism - where the specific·cost of the overlay is about 0.0017% of the execution time.
Where a quicker overlay mechanism would benefit is if */mod is taken out of the kernel and placed into the system file. The presented overlay mechanism adds about a 40% penalty in that case. However you still have the option of including the word·at compile time in which case there would be no penalty at all.
The finbonacci example that I've·given is interesting in that if·you change the definition of (fibo2) from CORE: to PASM: then it runs at about the same speed as the pure forth version! That's quite a dramatic demonstration of the speed (or lack thereof) of the overlay mechanism - as (fibo2) gets called several hundred-thousand times (off the top of my head)·during the course of calculating Fib(28).
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Carl Jacobs
JDForth - Forth to Spin Compiler http://www.jacobsdesign.com.au/software/jdforth/jdforth.php·
You might also be interested in the way I have decoded the spin interpreter to speed it up (and save longs). I started yesterday putting the pieces of code back together. I calculated that just to test the maths opcodes with all variables would take 16 days, so have opted for testing just a few cases for now (done). What I need is a big spin program to test the variants and I will log the real Interpreter results for each bytecode and compare with my Interpreter. I have a 6 long debugger controlled by LMM within both interpreters.
It includes FloatMath and FloatString libraries - a reasonably direct translation to forth of the Propeller Tool library files. These functions run a little over 3x faster as forth than their spin equivalents. There are two PASM: words included in FloatMath - more as an example of how its done versus performance enhancement, although the latter is a small bonus that you get for free.
A compiler bug (for begin ... while ... repeat) and several other bugs were fixed that were discovered as part of writing FloatMath and FloatString. eg: I had written the forth word min to use the assembly opcode min, but should have used mins! I've added umin and umax if you really *have* to do an unsigned 32-bit comparison.
Several of the user interface issues were addressed - thanks to those who gave feedback, but especially hippy! (BTW. hippy, could you please send me a PM / email).
Another 50 entries were added to the help file - including help for all the new floating point words.
The new version may be downloaded from the first post in this thread.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Carl Jacobs
JDForth - Forth to Spin Compiler http://www.jacobsdesign.com.au/software/jdforth/jdforth.php
Post Edited (Carl Jacobs) : 8/14/2008 4:00:37 AM GMT
New libraries include:
- FAT16·support on SD Cards.
- I2C routines for general use and reading/writing the boot EEPROM.
- Bit-bash serial to 2Mbaud. No extra cogs required.
- Rudimentary debugging support.
----Question: The FAT16 library is derived from Thomas Rokicki's fsrw.spin (with his permission) and with the SD driver supplied get·read/write access to the SD card of around 220k bytes per second. The SD driver does not·use the counters as it stands. If the counters were used then the SD card routines could be made quite a bit faster. So the question is: In JDForth, would you prefer the counters to be left as an avilable resource, or for the counters to be used to enhance the speed of JDForth where possible?
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Carl Jacobs
JDForth - Forth to Spin Compiler http://www.jacobsdesign.com.au/software/jdforth/jdforth.php
Includes: FAT16 support for SD cards. Bit-bash Serial at 2M baud. 32-bit floating point maths.·Fib(28) in 0.86 seconds. ~3x faster than spin, ~40% larger than spin.
Ideally, I'd say both. If they aren't used by the user put them to good use in the libraries. That may be easier said than done.
If you have to make a decision I'd say keep them as a user resource or you'll have problems if someone does want them later - Unless you could make it a user selected option.
Yes, well you have been busy.
·I got a chance to have a quick look ·and read the updates, but won't get a really chance to play around for a week or so.
The Quick Help really is really something. You might consider making the Help File a printable document in the registered version, but can that you might not wish to do so.
Counters ?· Keep them available to users ATM.· I·am· prepared to have slightly slower·access to the SD card and have access to the counters.
Ron
Post Edited (Ron Sutcliffe) : 9/2/2008 12:32:04 AM GMT
I'm tossing around with the idea of having two drivers for the SD card. It's a little painful, but I think that with the faster driver using the counters I might get just above 500k bytes-per-second transfer rates.
Part of the question is that I'm not sure how counters would be utilised in the JDForth cog. You could set them up as a free running oscillator, but any of the more complex tasks eg PWM and servo-control are probably going to need a dedicated cog in any case. JDForth is fantastic glue (much better than spin in my opinion) for tying things together, but it's hopeless at anything too hardware timing critical. Forget about video drivers and full-duplex serial, those jobs should be left to a cog that can handle the·exacting requirements. So, that still leaves the question of how the timers would be utilised. If they are going to be just oscillators, then any of the other cog's timers could just as easily be used for the same purpose. I imaging that the way timers will be used in JDForth is "here-and-now". In other words the timers will be used to perform whetever small task is immediately at hand, and then be promptly forgotton about. An example of this kind of usage·would be Beau Schwabe's Memsic 2125 dual accelerometer demo. If that is how they get used then the timers could be free to enhance whatever part of JDForth was currently running. Hence the original question...
@ Ron
No, there is no plan for a printed version of the document, we need to look after the environment these days. But I do find most help systems cumbersome. The design of the JDForth Quick Help is like the voice of a prompter who is sitting just off the stage, telling the actor what his next lines are.
There is a second·reason for not having a printed document. The JDForth help file is a living document. What·is provided is just the·start. If there's something specific that could enhance the help file system, then I'm interested in adding that. If I feel myself basking in vast amounts of time, I might even starting writing some tutorials.
Have you discovered Ctrl-D yet. It jumps to the source code·for the definition of the word under the cursor, if it exists.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Carl Jacobs
JDForth - Forth to Spin Compiler http://www.jacobsdesign.com.au/software/jdforth/jdforth.php
Includes: FAT16 support for SD cards. Bit-bash Serial at 2M baud. 32-bit floating point maths.·Fib(28) in 0.86 seconds. ~3x faster than spin, ~40% larger than spin.
Carl
In tv_Text.spin the color attributes are added to the character like so
Var
long color
word screen_adr
byte c
Screen_adr := (color << 1 + c & 1) << 10 + $200 + c & $FE
In Forth we might create a word ‘convert’· to add the color attributes like so
VAR16 cx
VAR32 color
: convert color@ 1·<< cx·h@ 1·and·+ 10·<< cx·h@ $fe·and $200·+·+ cx·H! ;
I am not sure that I need to use cx h@ and cx h! in this situation.
Would not cx@ and cx! Suffice, because cx has been defined.
Cx would be truncated when cx! is executed.
The situtation would not arise in Standard Forth
Ron
Post Edited (Ron Sutcliffe) : 9/2/2008 12:42:45 AM GMT
cx@ and cx! will compile to cx h@ and cx h!.
There's only 1 caveat - cx is treated as signed 16 bit. Normally this isn't a problem, but when bit twiddling you need to know if cx will ever be in the range 32768 to 65535. If it is, then you need uh@ to do an unsigned fetch. Either that, or define cx as VAR32 and just use the post-fix notation.
There is no magic, the postfix notation is just a shorthand!·You could redefine cx as a VAR16, VAR32, CON16, CON32 or USER16 and not have to change your code at all - as long as there are no number overflow issues.
eg: if you have
0 CON16 cx
: MyWord 10 cx! ;
Then in this instance cx! will compile to [noparse][[/noparse]'] cx rel>abs 2+ H! which I'm sure you don't want to know. The reason for using·constants is that they fetch more efficiently than a variable, and they can also be defined with a default value at compile time.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Carl Jacobs
JDForth - Forth to Spin Compiler http://www.jacobsdesign.com.au/software/jdforth/jdforth.php
Includes: FAT16 support for SD cards. Bit-bash Serial at 2M baud. 32-bit floating point maths.·Fib(28) in 0.86 seconds. ~3x faster than spin, ~40% larger than spin.
thanks, that makes sense
Ron
It seems that flavour of the week is single-stepping and debugging.
Well, JDForth now does single-stepping of forth words.
It's as simple as:
NEEDS DEBUG
and then
BREAK
wherever you want a breakpoint.
JDForth operates in dumb-teminal mode between 'BREAK's.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Carl Jacobs
JDForth - Forth to Spin Compiler http://www.jacobsdesign.com.au/software/jdforth/jdforth.php
Includes: FAT16 support for SD cards. Bit-bash Serial at 2M baud. 32-bit floating point maths.·Fib(28) in 0.86 seconds. ~3x faster than spin, ~40% larger than spin.
To make an interactive forth environment:
1) Open and compile Interactive.jdp in JDForth.
2) Open and compile DEMO_Interactive.spin in the Propeller Tool.
See top posting or JDForth web site for latest version.
For all those not interested in Forth:
JDForth includes an alternative FullDuplexSerial driver. NOT ANOTHER ONE!!!
WHY?
1)· The FIFO buffers are stored entirely within the cog. There are just over 1300 bytes of buffer space available that may be assigned to either the transmit or the receive FIFO buffers. Each instance of JDCogSerial requires only 6 longs of extra VAR space.
2)· The interface for reading and writing characters has been dramatically simplified, allowing easy access from spin and assembly (and forth) processes. Eg:
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Carl Jacobs
JDForth - Forth to Spin Compiler http://www.jacobsdesign.com.au/software/jdforth/jdforth.php
Includes: FAT16 support for SD cards. Bit-bash Serial at 2M baud. 32-bit floating point maths.·Fib(28) in 0.86 seconds. ~3x faster than spin, ~40% larger than spin.