pfth - An ANS Forth Interpreter for the Prop

15681011

Comments

  • David BetzDavid Betz Posts: 14,020
    edited 2012-12-06 - 09:28:11
    mindrobots wrote: »
    Not bad considering this thread started on 10/31 as a try at a C based ANS Forth...and now it's a viable tool to start developing on the P2.
    Yes, Dave did a nice job with pfth! It was easy to port to the P2 because it's entirely in PASM now. Mostly, I just deleted the Spin startup code, modified the serial code to use the new getp and setpc instructions instead of accessing ina and outa and rearranged things a little to match the expectations of my p2load loader.
  • Ron SutcliffeRon Sutcliffe Posts: 420
    edited 2012-12-06 - 09:38:02
    @David B Ok David
    My post was directed to David Hein. There is a need to be able to edit input at the command line before committing the text to Forth by CR.
    Forth is or should be a rapid development environment and interactive.
    Code should be able to be debugged at the console, word by word. ( : myword...........; )

    All Forth invirons provide for this.

    Ron
  • David BetzDavid Betz Posts: 14,020
    edited 2012-12-06 - 09:40:07
    @David B Ok David
    My post was directed to David Hein. There is a need to be able to edit input at the command line before committing the text to Forth by CR.
    Forth is or should be a rapid development environment and interactive.
    Code should be able to be debugged at the console, word by word. ( : myword...........; )

    All Forth invirons provide for this.

    Ron
    I agree with that 100%! The first surprise I had when I got pfth up and running was that I couldn't correct typing errors while entering a line of code. I'm sure Dave will fix that soon though! :-)
  • Dave HeinDave Hein Posts: 6,049
    edited 2012-12-06 - 09:44:55
    Excellent work, David. Thanks for porting pfth to the P2 emulator. You can fix the CR/LF problem by adding a "cr" after the "refill" command in the "interpret" word. It's the 9th line from the bottom in init.fth.

    I think Peter got tachyon running on the P2 emulator a few days ago, so I think he has claims to porting the first high level language. I was going to comment at the time that I got Spin running on the P2 a year ago -- running under spinsim in the P2 mode. I don't think I ever mentioned this on a public forum. I only commented on it in the PropGCC developer's forum. I only had to change a few things in the Spin interpreter to get it to run under P2. I assumed that Chip would have already done this. David are you interested in porting the Spin interpreter?

    EDIT: Yes, I need to add support for editing using the backspace key. I haven't gotten around to it yet.
  • David BetzDavid Betz Posts: 14,020
    edited 2012-12-06 - 09:49:16
    Dave Hein wrote: »
    I think Peter got tachyon running on the P2 emulator a few days ago, so I think he has claims to porting the first high level language.
    Well, he didn't announce it so how do we know for sure? :-)
    David are you interested in porting the Spin interpreter?
    Not right now. My next task is to get a C program running on P2!
  • David BetzDavid Betz Posts: 14,020
    edited 2012-12-06 - 09:54:41
    Dave Hein wrote: »
    You can fix the CR/LF problem by adding a "cr" after the "refill" command in the "interpret" word. It's the 9th line from the bottom in init.fth.
    That did the trick. It is now working well except for the lack of line editing.
    david-betzs-macbook-pro:pfth-p2 dbetz$ p2load -v -s pfth.obj -t
    Trying /dev/cu.usbserial-A700fKXl                    
    Found propeller version 32
    Loading 'pfth.obj' on port /dev/cu.usbserial-A700fKXl
    .....................
    [ Entering terminal mode. Type ESC or Control-C to exit. ]
    PASM pfth 0.59
     ok
    : double dup + ;
     ok
    3 double .
    6  ok
    
  • David BetzDavid Betz Posts: 14,020
    edited 2012-12-06 - 11:25:41
    Dave,

    Have you thought of how you'd like to extend pfth for use on the Propeller II? If you want to write some new P2-specific words, I'd be happy to test them for you.

    Thanks,
    David
  • Dave HeinDave Hein Posts: 6,049
    edited 2012-12-06 - 12:01:43
    I haven't given it much thought. Currently there are three words that provide Prop-specific functionality. They are cog@, cog! and cogx1. cog@ and cog! provide read/write access to cog RAM and memory mapped registers. cogx1 will execute a single Prop instruction that is fetched from the TOS. It uses the value below the TOS for the destination data, and returns the result on the stack. I need to add a cogx2 word so I can specify a source value also. The PASM code for these instructions would need to add extra NOPs to account for P2's deeper instruction pipeline.

    propwords.fth would need to be modified also. It defines various Prop-specific words based on the 3 primitive words. Once I finish up a few more items for P1 I'll start looking at P2.
  • Brian RileyBrian Riley Posts: 626
    edited 2012-12-06 - 23:21:57
    I have 059 running ... sorta ... to make CR display right I had enable AUTOLF in ZTerm's preferences. I never had to do that before!!!!

    I have Parkinson's. I make a lot of typos. Typing is a drag for me ... but much more so in 059 since backspace to edit/correct doesn't seem to work right .
  • Dave HeinDave Hein Posts: 6,049
    edited 2012-12-07 - 04:57:00
    I got backspace editing working yesterday evening. I should have put it in from the beginning, but I never got around to it. It will be in the next update in a day or so.
  • Brian RileyBrian Riley Posts: 626
    edited 2012-12-07 - 23:35:29
    Dave Hein wrote: »
    I got backspace editing working yesterday evening. I should have put it in from the beginning, but I never got around to it. It will be in the next update in a day or so.

    My fingers and my frayed patience thank you profusely.

    p.s. do I understand correctly that If I create a file "xxxx.fth" to get it processed I need to do a "file xxxx.fth" in the spin file and rerun the IDE? how soon will we be able to do a load "xxxx.fth" from SD ???
  • Dave HeinDave Hein Posts: 6,049
    edited 2012-12-08 - 04:38:04
    Yes, that is the current way to include a Forth source file. SD support should be available in about a week.
  • Dave HeinDave Hein Posts: 6,049
    edited 2012-12-09 - 20:34:56
    Here's version 0.67. This version adds backspace editing to the command line. I also added some basic SD support. However, it's just a beginning, and more work is needed to be able to load programs from an SD card.

    There are two main Spin files -- pfth.spin and sdpfth.spin. sdpfth.spin starts up an SPI driver cog and mounts the SD card. This is all done in Spin/PASM code prior to starting up the Forth cog. The Forth SD driver code is located in sd.fth. There are currently two main commands implemented. file-list will list the files in the root directory of the SD card. file-type will type the contents of a text file. It is executed by typing "file-type filename".

    I also ported an ANS Forth chess program I found on the internet. This is currently compiled into pfth.spin. After it starts up type "chess" to run the chess program. Level 2 runs fairly fast, and is pretty easy to beat. Level 3 takes a lot longer to compute it's moves, but it plays a much better game. However, it takes some patience to play it at level 3, and I haven't played a complete game against it at that level.

    The moves are entered by typing the column and row starting point and ending point for a piece, such as E2-E4. You have to use upper case letters. The chess program is exited by typing "Q". The program assumes it's displaying on the Prop terminal since it sends a NULL to clear the screen.
  • David BetzDavid Betz Posts: 14,020
    edited 2012-12-09 - 20:45:36
    Dave Hein wrote: »
    Here's version 0.67. This version adds backspace editing to the command line. I also added some basic SD support. However, it's just a beginning, and more work is needed to be able to load programs from an SD card.

    There are two main Spin files -- pfth.spin and sdpfth.spin. sdpfth.spin starts up an SPI driver cog and mounts the SD card. This is all done in Spin/PASM code prior to starting up the Forth cog. The Forth SD driver code is located in sd.fth. There are currently two main commands implemented. file-list will list the files in the root directory of the SD card. file-type will type the contents of a text file. It is executed by typing "file-type filename".

    I also ported an ANS Forth chess program I found on the internet. This is currently compiled into pfth.spin. After it starts up type "chess" to run the chess program. Level 2 runs fairly fast, and is pretty easy to beat. Level 3 takes a lot longer to compute it's moves, but it plays a much better game. However, it takes some patience to play it at level 3, and I haven't played a complete game against it at that level.

    The moves are entered by typing the column and row starting point and ending point for a piece, such as E2-E4. You have to use upper case letters. The chess program is exited by typing "Q". The program assumes it's displaying on the Prop terminal since it sends a NULL to clear the screen.
    Thanks Dave! Is it possible to run pfth.spin by itself without first running sdpfth.spin? There is no Spin compiler yet for P2 and I'd like to try your backspace code on the P2.

    Also, if I send you a diff between your P1 version and my P2 version would you consider adding ifdefs in your code to support P2?

    Thanks!
    David
  • Dave HeinDave Hein Posts: 6,049
    edited 2012-12-10 - 05:50:37
    pfth.spin is almost the same as sdpfth.spin except that I commented out a few lines that start the SPI driver and call the mount Method. The other difference is that pfth.spin includes the chess*.fth Forth code at the end, and sdpfth.spin includes the sd.fth file. My plan for the next update is that pfth.spin will just provide the ANS Forth words, and can be used by the programmer to build a stand-alone application. That's pretty much what it is right now, except that I included the chess*.fth files.

    In the next update sdpfth.spin will provide the standard ANS FILE words and programs can be loaded from the SD card using INCLUDE. INCLUDE isn't in ANS Forth '94, but the 2012 version of ANS Forth does contain INCLUDE plus a few other useful words, such as DEFER and IS. DEFER and IS are words that allow a word's function to be redefined after it has been declared. I had to implement them for the chess program.

    David, send me your P2 version and I'll add the changes to the latest pfth.spin code.
  • Ron SutcliffeRon Sutcliffe Posts: 420
    edited 2012-12-10 - 13:29:50
    Looking good.
    Unfortunately I only have the upper 32k of eeprom for storage ATM, but I can live with that until next month. Nice work David.
  • Dave HeinDave Hein Posts: 6,049
    edited 2012-12-16 - 17:30:18
    Here's a new pfth update that supports the INCLUDE command, which is used to load Forth files from an SD card. It also supports the ANS Forth words open-file, read-line, file-size, file-position and repostion-file. The non-standard words list-files and type-file can be used to list the files in the root directory and to type a file to the console.
  • Dave HeinDave Hein Posts: 6,049
    edited 2013-07-16 - 20:41:15
    The latest version of pfth is attached below. This version has the following changes:

    1. Changed SEE in see.fth so that it doesn't print "_lit" for numbers

    2. Added bufser.fth that contains a buffered serial driver

    3. Added a BUFFERIT word in bufser.fth that reads the serial input into memory until it encounters an "ESC" character.

    4. Added an EVALBUF word in bufser.fth that will evaluate multiple lines in a memory buffer

    5. Updated the readme.txt file with text contributed by Loopy

    6. Updated the readme.txt file with a description of the dictionary word format

    7. Added a SAVE word to i2c.fth that will write the current hub RAM image to EEPROM. Note: The small Spin program at the end of memory must have been preserved for this to boot properly.

    8. Changed the CR word so that it emits both a carriage return and a line feed.

    9. Added CASE, ENDCASE, OF and ENDOF to init.fth

    10. Improved WAITCNT and MS in propwords.fth. Removed INA!

    11. Fixed LIST-FILES in sd.fth to skip over deleted files in the directory

    12. Changed the execution token so that it points to the code pointer instead of the beginning of the word in the dictionary

    13. Made KEY a deferred word so that it can be re-assigned to the serial input after reading the FILEs in memory
  • LoopyBytelooseLoopyByteloose Posts: 12,537
    edited 2013-07-17 - 12:11:05
    @ Dave
    Happy day... a lot of nice improvements.

    Are you aware that SEE also displays other low level code beside _lit?
    When I decompile some words, such as a Begin... Until loop, I get unexpected results as it provides what it compiled for storage, not the original.
    Sorry, but I fear that SEE may still need more work.

    The improvements to WAITCNT and MS are excellent. I have to write something about how I am getting small delays in the range of 25 us increments that have no timing error at all. It is a bit tricky to remove the system calls overhead to Hubram, but very easy to do ... once you understand.
  • Dave HeinDave Hein Posts: 6,049
    edited 2013-07-17 - 13:21:45
    Yes, SEE needs more work. I noticed that things like ABORT" aren't handled correctly. And then there's the issue of displaying high level words like BEGIN and UNTIL. Once a word is compiled it loses it's high level content. The word BEGIN actually doesn't even generate any code, it just provides an address for UNTIL to jump back to. And UNTIL is implemented as a _JZ (jump on zero) primitive. I'm not sure how other Forths handle this. Some Forths compile words directly into machine code, so it would be even harder for SEE to disassemble this back into high level words.

    One approach would be to define different versions of _JZ that would indicate where it came from. All the versions would do the same thing, but SEE could display UNTIL or AGAIN or IF depending on with version was used. That's something to think about later on.
  • LoopyBytelooseLoopyByteloose Posts: 12,537
    edited 2013-07-18 - 06:44:54
    Hi again,
    If fear that fixing SEE might make it hog all the HubRAM space. There is a need for a tiny, compact decompiler and it may be the original SEE what the right one.

    I can and do look up the actual HIGH level definition in the related .fth listings to compare.

    If the process really needs to be provided and will take up a lot of space, maybe another word.. such as DECOMPILE should represent the High Level decode, and SEE should not be 'cleaned up'.

    After all, SEE actually is honest about presenting what it sees. And users might learn something (I know I have) from SEE's reports of exactly what is.

    And I did notice that Begin indeed doesn't generate any code... it just appears to mark for the compiler where the backward branch is suppose to go in relative distance. Until seems to indicate how far back to go. Interesting and informative once the user begins to catch on.

    +++++++++++++++
    My personal feelings are to NOT junk up pfth with features that are both complex to build and actually make it easier for a new learner to become lost in the complexity. Forth is inherently small, simple, and honest -- but it can become a huge maze that is hard to penetrate.

    The more complexity.. the more there is to document and explain... thus the paper chase begins to bury the software and daunt the new user.

    For example, you have a DUMP for HubRam, but not CogDump. For me, that is fine. I can create my won Cog Dump easily with a DO..LOOP and a COG@ if I need to see something that is going on in the COG.

    Some people will ask for everything and never begin to code their own tools.
  • Dave HeinDave Hein Posts: 6,049
    edited 2013-07-22 - 13:18:31
    I have a new update to pfth, which is attached below. I decided to revert SEE back to the previous version because it was useful to see _lit in the output. This version also contains substantial improvements in the size of compiled code, and the speed of the inner loop. I changed the word header and execution list to use 16-bit pointers instead of 32-bit pointers. This makes the compiled code almost twice as compact. A large pad is no longer needed before the FILE section in the Spin code. A smaller pad of 100 longs, or even less can be used without having the compiled code overrun the source code.

    The more compact code allows the SD version of the interpreter to load the chess program from the SD card, and with plenty of room to spare for other programs as well. The I2C code can be loaded, which allows saving the image to EEPROM. I changed the word that saves to EEPROM from SAVE to EESAVE.

    I also moved the code pointer and does pointer in the word header to be just before the body of the word. This makes the inner loop tighter, and it runs about 25 percent faster. Calls can now be made to compiled words without incurring any hub stalls. I ran the following word in pfth, PropForth and Tachyon to measure the time it takes to run the following loop.
    : TIME CNT@ 100000 BEGIN 1 - DUP 0 = UNTIL DROP CNT@ SWAP - 8000 / . ;
    
    PropForth took 17.6 usec per loop, while Tachyon was 7.6 usec and pfth was 6.8 usec. So, at least for this test, pfth was more than twice as fast as PropForth, and even slightly faster than Tachyon.

    I also compared the size of compiled words, and the sizes were similar to those of PropForth and Tachyon.
  • Martin_HMartin_H Posts: 4,050
    edited 2013-07-22 - 13:35:39
    An ANS standards compliant, memory compact, and fast FORTH for the Propeller. This sounds great! I'll download and try this tonight.
  • Peter JakackiPeter Jakacki Posts: 9,226
    edited 2013-07-22 - 15:36:38
    Dave Hein wrote: »
    I have a new update to pfth, which is attached below. I decided to revert SEE back to the previous version because it was useful to see _lit in the output. This version also contains substantial improvements in the size of compiled code, and the speed of the inner loop. I changed the word header and execution list to use 16-bit pointers instead of 32-bit pointers. This makes the compiled code almost twice as compact. A large pad is no longer needed before the FILE section in the Spin code. A smaller pad of 100 longs, or even less can be used without having the compiled code overrun the source code.

    The more compact code allows the SD version of the interpreter to load the chess program from the SD card, and with plenty of room to spare for other programs as well. The I2C code can be loaded, which allows saving the image to EEPROM. I changed the word that saves to EEPROM from SAVE to EESAVE.

    I also moved the code pointer and does pointer in the word header to be just before the body of the word. This makes the inner loop tighter, and it runs about 25 percent faster. Calls can now be made to compiled words without incurring any hub stalls. I ran the following word in pfth, PropForth and Tachyon to measure the time it takes to run the following loop.
    : TIME CNT@ 100000 BEGIN 1 - DUP 0 = UNTIL DROP CNT@ SWAP - 8000 / . ;
    
    PropForth took 17.6 usec per loop, while Tachyon was 7.6 usec and pfth was 6.8 usec. So, at least for this test, pfth was more than twice as fast as PropForth, and even slightly faster than Tachyon.

    I also compared the size of compiled words, and the sizes were similar to those of PropForth and Tachyon.

    Whatcha talkin bout Dave, I looked at the code and it's an empty loop constructed with BEGIN UNTIL instead of FOR NEXT or DO LOOP but that's not all. No, my eyes weren't playing tricks with me, instead of the 1- and 0= words which are standard Forth words you have constructed a loop that simply times stacking and unstacking by writing 1- as 1 - and 0= as 0 =. Who would do that and why? Yes, the one thing that Tachyon is not as efficient at is stacking/unstacking and this loop emphasizes that without doing anything else.

    That's not how you write in Forth, so here are my loops adjusted as it would normally be written except I am using 1,000,000 instead so you can interpret the readings a little easier:
    NEWCNT 1000000 BEGIN 1- DUP 0= UNTIL DROP .LAP 3,600ms ok
    So that's easy then to convert that back to 3.6us/loop, however a FOR NEXT loop:
    NEWCNT 1000000 FOR NEXT .LAP 400,005us ok
    So this takes less than half a microsecond per loop.

    EDIT: try timing the NORMALIZE routine and report back the execution time and code size.
  • Dave HeinDave Hein Posts: 6,049
    edited 2013-07-22 - 16:13:49
    Peter, I figured I would get a rise out of you if I posted code that pfth can run faster than Tachyon. :) If I use 1- and 0= I get 8.8 usec for pfth. That's because 1- and 0= are compiled words in pfth, so that runs slower than "1 -" and "0 =". So 8.8 versus 3.6 is more like what I expected comparing pfth to Tachyon. Actually, I was expecting a factor of 10:1, so I'm pretty happy with 2.4:1.

    I have FOR...NEXT in the comus.fth file, and I tried it. However, I found that I had forgotten to convert it from 32-bit XTs to 16-bit. I'll have to fix that when I have a chance. If I understand correctly, FOR...NEXT is basically the same as "0 DO...LOOP", so I gave that a try in pfth. I get 3.6 usec per loop with pfth, which is 9 times slower than Tachyon. Now that's more like what I was expecting.

    Now if you could just port the chess program into Tachyon we could compare something useful. :)
  • Peter JakackiPeter Jakacki Posts: 9,226
    edited 2013-07-22 - 16:32:32
    Dave Hein wrote: »
    Peter, I figured I would get a rise out of you if I posted code that pfth can run faster than Tachyon. :) If I use 1- and 0= I get 8.8 usec for pfth. That's because 1- and 0= are compiled words in pfth, so that runs slower than "1 -" and "0 =". So 8.8 versus 3.6 is more like what I expected comparing pfth to Tachyon. Actually, I was expecting a factor of 10:1, so I'm pretty happy with 2.4:1.

    I have FOR...NEXT in the comus.fth file, and I tried it. However, I found that I had forgotten to convert it from 32-bit XTs to 16-bit. I'll have to fix that when I have a chance. If I understand correctly, FOR...NEXT is basically the same as "0 DO...LOOP", so I gave that a try in pfth. I get 3.6 usec per loop with pfth, which is 9 times slower than Tachyon. Now that's more like what I was expecting.

    Now if you could just port the chess program into Tachyon we could compare something useful. :)

    Don't worry, I was having a chuckle myself, bring it on :)
    As for doing something useful I would hardly think a chess program on a microcontroller is something useful !!! :)
    I will look at it though but I am far more motivated when it is something useful. It's a pity you couldn't work on kernel extensions for Tachyon as I'm very sure that the work that you have been doing would prove to be very useful, yes.
  • LoopyBytelooseLoopyByteloose Posts: 12,537
    edited 2013-07-23 - 11:12:45
    pfth running loops faster than Tackyon! At least Peter is a good sport about this.

    I just went over to pfth because I wanted something that might be easier to learn. But the truth is that the more I do learn, the more interesting the contrasts between the various version... a very large playground.

    Peter had mentioned elsewhere that Forth is optimal for programing at a distance. I've gotten a nice little PC to RS422 and RS422 to Propeller setup working now... so one can put a Propeller at the end of a few thousand feet of wire and program via Forth.

    The fellow I was helping with a Propeller QuickStart has completed his first project using Forth and loves it. And I learned quite about about what needs to be introduced early to Forth on the Propeller that Starting Forth and ANS Forth might never cover.... mostly Pin i/o and precise delay creation.
  • Dave HeinDave Hein Posts: 6,049
    edited 2013-07-23 - 11:47:09
    Loopy, good to hear that things are going well with Forth. I would be interested in learning more about your experience in teaching Forth on the Prop to someone else. Maybe you could develop a tutorial about how to do pin I/O and delays on the Prop.

    To be fair to Tachyon, it is still the king of speed on the Prop. I just happen to pick an example that I knew would do OK under pfth. I didn't know until I timed it under Tachyon that it would actually run slightly faster under pfth.

    Programming the Prop at a distance is an interesting concept. It would simulate reprogramming a processor that's sitting in a satellite orbiting the earth, or located at some inaccessible location on the Earth. You would need some way for it to reboot if it got hung -- maybe a watchdog reset of some kind. I suppose with RS422 there may be a control line that could be used to reboot it.
  • D.PD.P Posts: 790
    edited 2013-07-23 - 13:14:01
    Don't worry, I was having a chuckle myself, bring it on :)
    As for doing something useful I would hardly think a chess program on a microcontroller is something useful !!! :)
    I will look at it though but I am far more motivated when it is something useful. It's a pity you couldn't work on kernel extensions for Tachyon as I'm very sure that the work that you have been doing would prove to be very useful, yes.


    Yup, to think of the cycles used on chess as opposed to WizNet 5200 support, Filesystem support, general string handling/parsing support and low level chip driver support just makes me sad.

    To each their own.
  • Dave HeinDave Hein Posts: 6,049
    edited 2013-07-23 - 13:40:51
    I hear ya D.P. I would like to see more work being done with pfth on real-world applications. I have ... a just a minute.
    ...
    Sorry, I'm in the middle of playing chess against my Prop, and I'm getting close to putting it in Check-mate. It plays an OK game at level 3, but it takes a few minutes between moves.

    pfth does have some file system support. Try the sdpfth.spin version. Type the LIST-FILES words to see the files in the root directory, and use INCLUDE to compile it. You can use WORDS to see the other file systems words that are implemented, and check the ANS Forth standard for documentation on how they work. I haven't implemented any SD write words at this point. I might add something to write back to files that already exist so I don't have to mess with the FAT tables.
Sign In or Register to comment.