pfth - An ANS Forth Interpreter for the Prop
Dave Hein
Posts: 6,347
pfth is an ANS Forth compliant interpreter written in PASM and Forth. pfth was originally written in C, but the C version is no longer supported. The original post to this thread follows below. The latest version of pfth is attached to this post, and also in a post near the end of this thread.
pfth is an ANS Forth compliant interpreter written in C. It can be compiled to run on almost any platform that has a C compiler. I have run it under Cygwin on a PC and on a Prop C3 board using the XMMC model under PropGCC.
pfth currently implements 90 of the ANS Forth core words in the C kernel, and 43 core words in the Forth source file ansi.fth. The design goal for pfth is to allow easy porting of ANS Forth programs. When run on the Prop, pfth also supports a small number of words to read and write the OUTA and DIRA registers. More words will be added in the future to support other features of the Prop.
I also plan on moving more of the dictionary to a Forth source file so that the interpreter will be small enough to run in CMM or LMM mode. This will increase the number of Prop platforms that pfth can run on, and also increase the speed.
The Prop version of pfth is built and loaded using the SimpleIDE. The target platform must have a board configuration file so that the XMMC image can be loaded, and the SD driver can be initialized by the loader. This will change in future versions that will run using CMM or LMM mode.
The current version of pfth is attached below.
pfth is an ANS Forth compliant interpreter written in C. It can be compiled to run on almost any platform that has a C compiler. I have run it under Cygwin on a PC and on a Prop C3 board using the XMMC model under PropGCC.
pfth currently implements 90 of the ANS Forth core words in the C kernel, and 43 core words in the Forth source file ansi.fth. The design goal for pfth is to allow easy porting of ANS Forth programs. When run on the Prop, pfth also supports a small number of words to read and write the OUTA and DIRA registers. More words will be added in the future to support other features of the Prop.
I also plan on moving more of the dictionary to a Forth source file so that the interpreter will be small enough to run in CMM or LMM mode. This will increase the number of Prop platforms that pfth can run on, and also increase the speed.
The Prop version of pfth is built and loaded using the SimpleIDE. The target platform must have a board configuration file so that the XMMC image can be loaded, and the SD driver can be initialized by the loader. This will change in future versions that will run using CMM or LMM mode.
The current version of pfth is attached below.
Comments
How big is the memory footprint?
@localroger - I believe the goal is to be ANS compliant, not to be fastest. Fastest might be Tachyon, if you are interested in bit banging. All around crazy fast would be propforth. pfth is only a few hours old, give it a change to find a niche.
Not only is it ANS compliant, it's multi-platform from PC to Propeller! That's an impressive feather in its hat!!
Yes! And perhaps other platforms with GCC?
localroger is correct that anything other than assembly is slow, but hey, it works! Yes, it has "dismal performance", and it may be a "clever novelty". I'm not sure what it is useful for, but it does run various programs that I've grabbed from the internet without having to rewrite them. So it may be useful for programs where speed is not an issue.
Raspberry Pi and Beagle Bone are on my list.....my PandaBoard if I ever uncrate it.
Well, then there's Mac and Bigger Linux boxes, too!
If you want comments from the peanut gallery we could give you suggestions...
That sounds like a valid use case right there. Well done!
I've added pfth to the giant list of 240 Propeller languages and versions.
No doubt such a case can be made for high speed professional apps. Though not all applications require languages of high speed. Some can be useful for academics, testing, exampling, experimenting, hobby pursuit, and retro computing. The Quantum Reflection Project uses the high speed of four BASIC Stamp boards which are perfectly suitable. Another example is a Propeller based COSMAC ELF simulator project in the works that will run a version of TINY BASIC similar to Tom Pittman's original programming language that ran on the 1970s NETRONICS COSMAC ELF II microcomputer. Some have suggested actually slowing down the language on the Prop to make it more authentic.
I did a bit more work on the XMMC version, and I was able to speed it up by a factor of two by forcing portions of the interpreter to reside in hub RAM. It will run a tight "DO LOOP" at about 13,500 loops per second. The nice feature of XMMC programs is that the code can be megabytes in size as long as the data requirements fit the 32K RAM space. This will allow me to add more ANS Forth words without too much concern about memory limitations.
That's the issue with writing a forth in C, and trying to be ANSII compliant. Both those options tend to demand a lot. OF course you could try to write a kernel in assembler, but that would take forever unless you were an assembler ninja, in which case you won't need forth. Since you are using extra memory, you don't have a problem.
I've never had the opportunity to use extra memory (besides SD and EEPROM), how do you boot it?
I never had faith that an ANS Forth for the Prop was practical, especially one written in C. That's not to say that it shouldn't be done because it offers another option for choosing a language which might be based upon using existing ANS code.
As for speed this is how Tachyon performs with an empty DO LOOP in 1 second:
#1,666,666 0 NEWCNT DO LOOP .LAP 1,000ms ok
So 1,666,666 empty DO LOOPs every second or one every 600ns.
THE ON LINE VERSION OF 'Starting Forth' has been rewritten to comply with ANS Forth 94. So new users have a solid entry tutorial.
http://www.forth.com/starting-forth/index.html
http://www.forth.com/starting-forth/index.html
13,500 Loops per second is quite enough to deal with lights, sound, and even motors. So, it is quite valid for a micro-controller - - especially one with 8 processors.
The thing is that ANS Forth is a good entry point for the new user, especially with Leo Brodie's text. Once they have a bit of confidence, they will enjoy TachyonForth more.
My Starting Forth Lexicon.PDF had some formating gliches in it. I just reposted it. Attachment 96656 is now a broken link.
On a Propeller? Bit Banging soft peripherals in a COG? Creating things like high speed I2C channels, running fast SPI or multi-channel PWM controllers. Sorting and searching would need fast loops across a large address space, but fast loops work dandy with small memory and I/O space. Time you trade in your loop overhead can be used in your HUB access or other places where you can't help how long it takes. Your loop has a finite time to complete a task - if you minimize the loop overhead, you maximize "user" time in the loop. Speed can sometimes make the impossible, possible!
I like the Starting Forth Lexicon, thank you!
I'm speechless, like I don't know what I'm writing a Forth for and why I chose to use a Propeller chip? Of course I know and I can assure you it certainly isn't sorting and searching normally. Looping means repetition and repetition means one small delay is amplified greatly. Even on small loops too much time can be spent in looping overheads which if it's manipulating I/O can result in uneven I/O timing for instance. Response time is important in control systems and some of those responses can only be issued from within a super loop after all the information has been crunched each and every time, thousands of times a second. I'm not talking about a blinking LED tutorial.
However admirable your efforts are in learning a new way of thinking and programming you are still wasting too much time posting opinions whereas the time would be much better spent learning a lot more then coming to an opinion through that experience. An apprentice may like to think he knows of a better way of doing things but it's not until he learns his trade properly and has applied himself that he understands the wisdom behind the way things are or were done (plus his own foolishness at the time). He may come up with a better way then and others will have no problem agreeing with him either. You are an old hand, certainly you must understand this.
Good news is I think I get it on a basic level. Really struggled with the difference between a language that has a solid definition and the idea that one starts with a small core, defining what is needed to get stuff done. Expressions are the hardest for me, because I have to parse it once to understand it, then parse it again attempting to do it in a stack based form. A Forth that would take standard expressions would be very interesting, though likely horribly broken... That's my issue, but maybe one that's informative to you guys.
(Heh, I was thinking of a word: "SPRESSION" that would take a standard expression, solve it and leave the result on the stack... Of course, for it to work, one would need memory allocated to hold values, or stack relative references, which I understand to be somewhat taboo. Again, don't know enough yet.)
I find switching case a lot annoying, FWIW. Do you guys just write with the caps lock on, or what? LOL!!
The smaller sets of words found in Prop Forth and Tachyon are actually a bonus to me! There is less to wrap one's mind around and it seems to work a lot like assembly language does. Assembly doesn't do all that much either, though we are kind of spoiled on the Prop with a lot of great instructions. Honestly, that smaller lexicon kind of focuses a person onto building words more than it does seeking to find words already done. Not sure whether or not that is a good thing, but it is a more lean thing. Probably that impacts program reuse as definitions vary at a more basic level than they would with a wider lexicon.
I think somebody who wants the book to just work would get some utility out of an ANSI Forth. From there, adapting to the more lean and fast Forths might be easy, or might not be, depending on how that somebody thinks. There are other people out there building little tiny Forths on all sorts of things, including the old TI-99A computer. I found I could follow most of that discussion with what I learned off of Tachyon so far. That seemed encouraging to me.
Generally, I am in the speed camp. Propellers are interesting because we are always facing that speed / size / robustness trade-off. I'm not sure I would use a Forth that was slow, but I am sure I could learn from it. Again, I don't know what the damage is having gotten acclimated to the ANSI lexicon, then having to work on a limited set from there. Not experienced enough to know. I see people using smaller Forths to get stuff done, and they really understand the process well enough to build out what they need. An analogy would be somebody with good manual skills building kit buildings easily and quickly simply choosing the kit that makes the most sense compared to the experienced builder who can take raw materials and build exactly the right building...
I really like how interactive Tachyon is. Every single character seems to matter, and I get caught a lot doing the wrong thing, and interestingly, I'm almost always caught right then, the moment I type it! This both confused me and helped me, depending on what it was. If I were to get some greater skill, I totally see how having it all interactive really is powerful and fun, though I don't know whether or not I'll get there. I explored Forth to broaden my perspective more than I did to really apply it at this moment in time.
Anyway, there you go. I don't think either of you are wrong, just unclear on boundaries of things. We all learn how we learn.
You know what would be interesting --and IMHO, more interesting than the Starting Forth book, is one that takes a person through building a forth and then extending it. There are bound to be some good habits there, and implications to doing things that would be good to know. And that path could start small, and would seem to align much better to what I see as the dominant Forth use case; namely, building an environment that is well suited to a particular hardware and or task. That would get right at the guts of how things work and from there a person would be well equipped to deal with whatever Forth they encounter. Perhaps that exists. Do tell.
Smile, this is getting long...
One other observation:
When learning a new language or environment, I've always defaulted to a minimum set of things. From there, write some programs, see success, then having that test bed foundation done, explore the goodies / unique elements present and work toward some mastery of it all.
It's hard to see where the core is with Forth. At least it is for me. Heater got at that with, "What is the real Forth?" and the answer varies depending on what you've got, what it's capable of, and the kernel you happen to be running. That's another way to get at the book I think would be extremely useful.
To tell
it's a bit OT but seeing that you asked.
JonesForth is a good start for a Tutorial re ASM Forth compiler. It's ix386 asm but you will have no trouble to following,
Try a Google search. onesforth.S - Eighty-twenty.org
Ron
SPRESSION is a cool idea. You mean, evaluate a regular infix expression? This is doable. We should take this to a separate thread or something. You don't necessarily need to allocate memory. And allocating memory is not "taboo" per se, the rule of thumb is don't waste it. I allocate variable for memory all the time, and sometime leave it and sometimes factor it all out so things only run on the stack. per issue.
This is agood idea. I have plans for this. V5.3 is optimized for ease of generting custom kernels, there's no reason why folks can easily create custom kernels for any possible combination of drivers and functions. We need to get a little momentum going on the docs, but this is in the plan.
heater may never find and answer, because he's looking for a C language context, maybe like simpleIDE and a compiler. Those might be hard to find in forth context.
Forth is the command line. Its the dictionary threading, the interpreter, the executioner. These are the advanced topics, you just have to know these exist for now, you don't need to know how to write them to be able to use them.
Hi Ron
Could you pipe in with progress on your forth editor? These guys might want to use it!
EDIT: Here's the word that I used to time the "do loop" loop. I ran it on PropForth (with some modifications for cnt) and got about 385,000 cycles/second.
Worry less about low level optimization until after high level design optimization is complete.
Its cool that you got it to go faster, but its only a week old. The niche for prth is ANSII compatibility, its is the only forth on the prop that even attempts to claim this. Tachyon will always be fastest at bit banging, and Propforth might be the the example of "many prop chips" support.
I for one would much more impressed with well implemented ANS compatibility over execution benchmarks at this stage. Optimize for benchmarks last.
Just my $0.02
Dave, is there any particular reason why the low level operations and stack handling can't be implemented with inline assembly (or other gcc specific accelleration techniques)?
Now while it could be difficult to define the boundary between what's "core" and what's built upon it, there are indeed examples where that boundary had to be defined, if not for other reasons, because of the implementation requirement for the smallest possible interpreter.
I'm referring to this in particular, which you're probably already aware of: http://www.ioccc.org/1992/buzzard.2.design
Anyway I concur with prof_braino, at this stage of developement the goal of compliance is impressive enough, optimization can be done in later stages.
@Antoine, the webpage on First and Third is very interesting. I hadn't seen it before. It contains a lot of useful information on building a Forth interpreter with a minimal kernel. I'll look into that more in the future for educational purposes. Personally, I think a faster Forth kernel could be written in assembly instead of boostrapping it from a minimal kernel.
I've also integrated pfth with the PropGCC filetest demo program. The "bye" command will exit to the filetest program, which provides basic Linux commands, such as ls, cat, chdir, mkdir, and a few others. pfth can be re-entered by typing "pfth" from the command line. You can also include a list of Forth programs to load from the command line, such as "pfth ansi.fth".
I've looked at porting a few more Forth programs from the internet, and I've come to the conclusion that Forth is not an easy language for porting large programs. Most of the programs that I've tried are just not well commented or documented.
I've been thinking about ANS Forth 94 and it just may be too much for the average person to get started with. If you eliminated Chapter 3 of the text, the rest of the "Starting Forth Lexicon" might be very easy to include in pfth. This would give you a good tutorial reference for new users - just tell them to ignore Chapter 3 as you don't provide that kind of storage and OS.
There are a few words that may seem rather odd to you, but they still are important to introducing Forth, so try to include all of them so that the text examples can be tested and explored.
This will work for EVERYONE -- pfth, PropForth, and TachyonForth if you all are willing to just standardize this limited subset.
This is good. There is a way to do this. But consider ANS Forth is just a yardstick, and not the item we are trying to measure. Not all words make the same sense on the prop.
FALSE and TRUE are artifacts from before a numberic value was established for the logical value. FALSE is pretty much always been 0 (zero), but TRUE used to be 1, then we changed to -1 at the time when "fuzzy logic" was the rage, now TRUE is anything NON-zero. Turns out is we know what we are doing, we can skip these and just use the value directly, if we understand it, its much clearer.
-TRAILING is for removing trailing blanks, only useful if you do lots of text processing. On the prop, this shouldonly ever be in an extension, until you write a word processor.
! is the cell store word. On the prop, there are at least two sizes for the default cell. If you use ! in the context of one, you neglect the other; if you get fancy and use ! for both you completely confuse cog and hub ram. We based this for weeks and decided the giving each type an explicite store word was the easiest solution. It allows us to treat the defaults cell sizes equivalently for COG longs and HUB words, as well as character and what ever else we want. The ! as the store word is an artifact from when everything was a byte. This word should be retired, it is not helpful.
Looking at the first six entries, at least three of them raise contentious issue that need to be resolved before we could get any "standard". How important is it to adhere to the examples in some ancient tome? (That sounds like a loaded question, somehow).
One could make a set of extensions to to transform any forth into a "standard" forth, but then it would not behave as the original, which is what we purportedly are trying to learn. Is this effort we want to expend? Life it too short (at least mine is). Perhaps it would be more productive to acknowledge that pfth is suitable for use with Starting Forth; the tutorials are suitable for use with PropForth, and seat belt + fire extinguisher are suitable for use with Tachyon.
Personally, I think "right tool for the right job" should be applied here. The first question to ask is "what do we really want to accomplish". If you wish to implement a set of extensions to get propforth to behave as an ANSII forth (code compatible) this would be an interesting project, and I'll help. Is this really what you want to do?