TACHYON O/S V3.0 JUNO - Furiously Fast Forth, FAT32+LAN+VGA+RS485+OBEX ROMS+FP+LMM+++

Comments

• ericball Posts: 772

  2012-07-03 - 11:25:39 edited 2012-07-03 - 11:25:39

  Hi Peter,
  
  Sounds cool. Did you read localroger's Windmill blogs? It's a Forth-like bytecode interpreter designed to run out of SPI EEPROM. I'd recommend checking out http://forums.parallax.com/entry.php?39-Windmill-Byte-Code-Interpreter where he talks about mapping bytecodes to PASM instructions.

  Pay for your free software - let the developers know how much you appreciate their work!
  
  Links to Propeller stuff I've done (mostly composite video)
• mindrobots Posts: 6,506

  2012-07-03 - 11:34:44 edited 2012-07-03 - 11:34:44

  Peter, Excellent!!
  
  Tachyon is a perfect name for it since Forth is often considered a hypothetical language that may or may not exist.
  
  I look forward to taking this out for a cruise!

  MOV OUTA, PEACE <div>Rick </div><div>"I've stopped using programming languages with Garbage Collection, they keep deleting my source code!!"</div>
• richaj45 Posts: 179

  2012-07-03 - 17:21:30 edited 2012-07-03 - 17:21:30

  Hello:
  
  I am curious as to what the advantage of developing another Forth when PropForth looks very complete?
  
  That is besides the need to learn by doing, which i understand completely.
  
  cheers,
  rich
• kuroneko Posts: 3,623

  2012-07-03 - 17:33:48 edited 2012-07-03 - 17:33:48

  Peter Jakacki wrote: »

  doNEXT                  rdbyte  token,IP                'read byte code instruction
                          add     IP,#1                   'advance IP to next byte token
                          shl     token,#1                'expand to 9-bits - all byte codes point to code on double-long boundary
                         [COLOR="orange"] test    token,#$180 wz[/COLOR]          'tokens $C0..$FF are calls to kernel byte code via kbctbl
                 [COLOR="orange"]if_nz    jmp     token[/COLOR]                   'directly execute PASM byte codes without further ado
  

  If you want to separate between $00-BF and $C0-$FF shouldn't the test/jmp pair be a cmp/jmp (less than^A)? Currently anything above $3F produces !Z. Or are your tokens inverted somewhere? Do I need more coffee?
  
  ^A or greater equal depending on when you do the jump

  video driver collection, odds & ends
• Peter Jakacki Posts: 8,197

  2012-07-03 - 17:58:01 edited 2012-07-03 - 17:58:01

  kuroneko wrote: »

  If you want to separate between $00-BF and $C0-$FF shouldn't the test/jmp pair be a cmp/jmp (less than)? Currently anything above $3F produces !Z. Or are your tokens inverted somewhere? Do I need more coffee?

  
  We all need more coffee and we don't need an excuse. The code was all mish-mashed by some major upheavals when I posted this about 3:30AM in the morning (or at night from my pont of view) and I spotted it this morning and changed it to if_z instead but still wrong Oh look! time for a coffee! The original just tested the msb so the reason for "test" instead of "cmp". Anyway I thought I would leave the post without corrections as it serves it's purpose and I will find out if anyone is analyzing it (which you did 30 minutes later).
  
  I had changed the point at which byte tokens are either used as a direct 9-bit address into the cog's PASM code or as an index into a jump table to byte code. Somehow it looked right when I did it. Sometimes I aim for "close enough is good enough" as I know I will come back and if it's still there in that I haven't changed it all again then I will make it right (or just crumple it up and toss it into the trash).

  ---

  Tachyon Forth - compact, fast, forthwright and interactive
  
  --->CLICK THE LOGO for more links<---
  P2 +++++ TAQOZ INTRO & LINKS +++++ P2 SHORTFORM DATASHEET
  P1 +++++ Latest binary V5.4 includes EASYFILE +++++ Tachyon Forth News Blog
  Brisbane, Australia
  
• prof_braino Posts: 4,312

  2012-07-04 - 18:47:31 edited 2012-07-04 - 18:47:31

  WOW. You rock! A new forth on the prop. Next thing you know Cliffe Biffle will release his source. Are there now more versions of forth than C for the prop? We make catch up to basic yet!
  
  I look forward to (attempting) assimilating all your best ideas! Especially those that yield more run time speed without assembler. You got some neat stuff going on. Can you borrow from localroger's work?
  
  When your design starts to get stable, consider using the test automation so the workstation runs a regression test suite after every development change is "done". Sal says there is no reason the automation would not work with any language, I would like to find evidence one way or the other. I'd like to help set it up.
  
  @richaj45: The biggest advantage is getting the perspective of a different approach. Sal's way is deemed the best by Sal to do what Sal wants to do, and does not necessarily lend itself to what Peter wants to do or the way Peter wants to do it. "Right tool for the right job". Its almost guaranteed that Peter will find a better way to do what Peter wants to do, since he does not have the same constraints. Often, we will find a new perspective in the way one does something that improves the other. This effect might not be limited to forth kernel development . I heard of some guy Darwin that made some notes about exploiting niches, but he has not posted anything in the OBEX.
• Cluso99 Posts: 14,808

  2012-07-04 - 19:42:48 edited 2012-07-04 - 19:42:48

  Nice work Peter. I don't have time to analyse what you've done and I have never looked at Forth.
  
  As Prof says, another version is great, and ideas can be shared, making each version better than before. Good ideas will find their way into other code too.
  
  Prof & Sal: I am sure the test automation suite will be great once things settle. Hopefully it should even find its way into normal code too.

  My Prop boards: P8XBlade2, RamBlade, CpuBlade, TriBlade
  Prop OS (also see Sphinx, PropDos, PropCmd, Spinix)
  Website: www.clusos.com
  Prop Tools (Index) , Emulators (Index) , ZiCog (Z80)
• Peter Jakacki Posts: 8,197

  2012-07-04 - 21:44:11 edited 2012-07-04 - 21:44:11

  While I really like PropForth and the work that's been put into it the problem is that it runs too slow at the Forth level since very little is actually coded in directly executed PASM. I don't know if my Forth will measure up to the work that Sal and Prof Braino have put into theirs but it definitely will suit my needs. Plus it should be a lot easier and faster to recompile a new kernel with other objects. I'm a simple guy and I like to keep things simple. If I can't knock this TACHYON Forth over in a week or two then it's probably too complicated for me
  
  One of my aims is to also have enough resources left over that I can hook-up a monitor, keyboard and SD card and run the whole system stand-alone if I have to. But mainly I find that I interface to a great variety of chips but I need more speed and I would like to stay within the normal Forth environment when doing that. Forth was after all designed to get at the bare metal in a transparent and interactive manner having first been employed on radio telescopes in the 70's. At the very most if necessary having to patch in a Spin file, recompile the kernel and have it up and running just as quick. Well, at least, that's my aim. Having efficient byte code means I can pack a large application program in and still have memory left over for video etc.
  
  I hadn't looked at localroger's Windmill before but that's the basic idea I had before for running larger programs in that I would use those small 4M byte serial Flash chips I have on some boards or else SD but it's still a lot more cumbersome than running from hub RAM. Running interpreted code from serial memory is an old idea, I remember the TSS400 for one. It's interesting to see that he is thinking of a scheme to encode PASM. But what has happened to Windmill since? Has he charged at it a bit too quixotically?

  ---

  Tachyon Forth - compact, fast, forthwright and interactive
  
  --->CLICK THE LOGO for more links<---
  P2 +++++ TAQOZ INTRO & LINKS +++++ P2 SHORTFORM DATASHEET
  P1 +++++ Latest binary V5.4 includes EASYFILE +++++ Tachyon Forth News Blog
  Brisbane, Australia
  
• prof_braino Posts: 4,312

  2012-07-05 - 11:29:12 edited 2012-07-05 - 11:29:12

  Peter Jakacki wrote: »

  .. PropForth ... runs too slow at the Forth level ... PASM. ... lot easier and faster to recompile a new kernel with other objects.
  ... hook-up a monitor, keyboard and SD card and run the whole system stand-alone .
  ... patch in a Spin file, recompile the kernel and have it up and running just as quick.
  ... running larger programs ... 4M byte serial Flash chips or SD

  
  I'm looking forward to your experimenting along these lines, your perspective and results will be interesting. We do have a way to "lock" the forth prompt so only the user level application words are available, and to eliminate the development extensions from the final application to save space, but we haven't worked on trimming down the kernel further, there is a lot of unexplored custom kernel development.
  
  In the meantime, maybe look at the JupiterACE code from v3.6; this runs on the Prop Demo Board and is a stand alone forth with VGA and keyboard, 80 column text in hires, 40 column text in low res. This might be towards what you are looking for, it will be brought up to 5.0 kernel when the test automation is complete. You might be able to build on this, but Sal's version is still a couple weeks out. The JupiteACE was actually my goal for joining the project. V3.6 was the teaser, v5.3 may be the final result. Running VGA takes most of the resources of a prop chip, which lead us to the idea of just adding more props, which lead us to MCS and Go-channels. So the propforth development has been getting "bigger", if you find ways to get it "smaller" again, that will be really helpful.
  
  Sal plans to simplify the process for optimizing in assembler, he will add new words that start and end the assembly process, and the assembler code can be compiled right into the dictionary. This could help in creating tachyon, but it may not ready until 5.4.
  
  We looked at linking in arbitrary SPIN files, but that seemed to require the SPIN be written to support a "standard" interface, and we couldn't find anything to use as the standard; (every spin program seems to be too different) so we stopped that investigation. Maybe you can provide some insight or example of a specific spin file you want linked in, we can use that as a starting point for a "standard".
  
  Sal's model supports adding more hub and cog memory in the form of more units of prop + SD, rather than adding more dedicated memory parts etc. For Sal's purposes, its easier and cheaper to just grab a couple more props out of the bin. In the case of the JupiteACE, it allows the full resource of oa prop to run the VGA, and permits a a full prop or more to be available to an application. But we have kept in the 32K memory mind set, it will be interesting to see what we gain will large external memory configurations.
  
  I have a pile of HIVE boards (hive-project.de) that accept 1 meg x 8 bit SRAM. I was thinking of circuit bending these toward a propforth rig, but that is way down the road. In fact, you might want to check out the "m" language Ingo is working on over there. It appears to be a version of forth for the Hive hardware, and works with the hive OS running on the other chips (which might get you the "link in spin programs" function you seek). I don't know the details, but a bunch of it seems similar to your goals. Google translate does a fair job with the German, and Ingo and the Borg drones can do English.
• Peter Jakacki Posts: 8,197

  2012-07-08 - 06:58:40 edited 2012-07-08 - 06:58:40

  Alright, after a few delays away from the project here and there due do the dear wife and her holiday projects I have reached a milestone with TACHYON. I am now able to run precompiled byte code successfully and have run some tests on various byte code routines. Afterwards I wrote an SPI output routine that manipulated the SPI bits individually and this ran at a speed of 82kHz with a clock high time of 1us. I compared this to the equivalent Spin routine which ran at 26.88kHz with a clock high time of 7.8us. Great, that's 3 times faster already for this kind of code. Next step was to code a simple primitive (6 PASM instructions) that clocked a single bit at a time and place that within a DO..LOOP to keep the code flexible and ran the tests. This time it ran at 625kHz with a clock high time of 50ns so that the CLKBIT primitive and the looping amounted to 1.6us per bit and 11 bytes for the SPI routine, that's pretty good I think. So the average execution speed of most byte codes is around 1us which is mainly limited by the RDBYTE access times.
  
  I have a little tidying up to do (if I am not seconded to the garden project in the meantime) and I will build in the dictionary and high-level words that form the text interpreter (vs the byte code interpreter) so that I can work with this interactively in a terminal. At this stage I may release the source for the alpha which should be in the next week of so.
  
  Another change I made was to the serial I/O and include the serial transmit code into the Forth cog and leave the serial receive to a dedicated cog. The transmit code is very small and doesn't have to worry about multi-tasking with any receive code etc. This way the receive timing can be very precise and run at 1,382,400 baud for the maximum speed of my Bluetooth modules. Also since the transmit speed is very high there is no need to buffer or waste time writing to hub RAM as each character completes transmission in 7.23us.

  ---

  Tachyon Forth - compact, fast, forthwright and interactive
  
  --->CLICK THE LOGO for more links<---
  P2 +++++ TAQOZ INTRO & LINKS +++++ P2 SHORTFORM DATASHEET
  P1 +++++ Latest binary V5.4 includes EASYFILE +++++ Tachyon Forth News Blog
  Brisbane, Australia
  
• Cluso99 Posts: 14,808

  2012-07-08 - 18:39:35 edited 2012-07-08 - 18:39:35

  Nice job Peter. May it rain a little more so you don't need to garden until finished

  My Prop boards: P8XBlade2, RamBlade, CpuBlade, TriBlade
  Prop OS (also see Sphinx, PropDos, PropCmd, Spinix)
  Website: www.clusos.com
  Prop Tools (Index) , Emulators (Index) , ZiCog (Z80)
• Peter Jakacki Posts: 8,197

  2012-07-09 - 07:28:04 edited 2012-07-09 - 07:28:04

  Just a progress report now as I streamline the code. I have decided to revert back to using all 256 byte-code values for direct long access to the first 256 longs in the cog as this also simplifies the precompiled byte-code in that I don't need to specify a "/2" after each word. Plus not all byte-code values are used though as I try to cram the code for each into 2 longs if possible rather than just wasting code by jumping to the actual code. Some code primitives are 6 longs long but are still in the first 256 longs of the cog because it's a balance between wasting byte-code values and wasting cog memory (extra jmps). Having some byte-codes that use external tables for vectoring to more byte-code is too awkward to maintain and at this point seems redundant so it's been removed. The byte-code interpreter is reduced to:

  doNEXT
                          rdbyte  token,IP                'read byte code instruction
                          add     IP,#1                   'advance IP to next byte token
                          jmp     token
  

  
  The SPI primitives now include a flexible transmit routine which clocks data out at 2.85MHz (without doing anything fancy that is) so this handles a lot of SPI and I2C style protocols very efficiently. To slow it down just requires accessing the CLKBIT primitive at your leisure.
  
  I have also found that it is far more efficient to store my inline literals and constants in big endian form to facilitate shifting and accumulating. So there is only one routine that reads in bytes to form these numbers and depending upon the entry point is what decides how many bytes are read. Constants are also coded as a standard definition with an exit (return from call) as there is not much advantage in having a special operation just for this. So the structure of say a 24-bit constant is [PUSH3] [$A5] [$00] [$C1] [EXIT] where PUSH3 is the byte-code for reading in 3 bytes and pushing the result onto the data stack after which the definition EXITs. Too easy.
  
  This is the simple PASM code that effects pushing 1 to 4 inline bytes onto the datastack.

  PUSH4                   call    #ACCBYTE                ' read the next byte @IP++ and shift accumulate
  PUSH3                   call    #ACCBYTE
  PUSH2                   call    #ACCBYTE
  PUSH1                   call    #ACCBYTE
                          call    #_PUSHACC               ' Push the accumulator onto the stack then zero it
                          jmp     #doNEXT
  
  ' This code here is located in the second half of the cog RAM to leave the first half free for byte-code access.
  
  ACCBYTE                 call    #GETBYTE
                          shl     ACC,#8
                          or      ACC,X
  ACCBYTE_ret             ret
  
  GETBYTE                 rdbyte  X,IP
                          add     IP,#1
  GETBYTE_ret             ret
  

  
  And this is how a separate constant is coded (similar to inline literals without the EXIT):

  06DD(0023) 74          | MYCON   byte  PUSH4,$A5,$00,$FC,$01,EXIT
  06DE(0023) A5          | 
  06DF(0023) 00          | 
  06E0(0024) FC          | 
  06E1(0024) 01          | 
  06E2(0024) 03          |
  

  Because all values are non-aligned there is no wasted space aligning them to word or long boundaries. Also relative addressing gets around any offsets and allows for relocating code easily.
  
  So my test routine which prints a start-up message, sends out 32-bits via SPI, and does a hex dump of hub RAM looks like this in byte-code listing form (courtesy BST):

  06F8(002A) 77          | TXSPI   byte  PUSH1,32,CLKBITS,EXIT                  ' Send the 32-bits
  06F9(002A) 20          | 
  06FA(002A) 8D          | 
  06FB(002A) 03          | 
  06FC(002B) 74          | MYCON   byte  PUSH4,$A5,$00,$FC,$01,EXIT
  06FD(002B) A5          | 
  06FE(002B) 00          | 
  06FF(002B) FC          | 
  0700(002C) 01          | 
  0701(002C) 03          | 
  0702(002C)             | DEMO
  0702(002C) B9          |         byte  PRTSTR,$0D,$0A,"TACHYON Forth V1.0 ",0
  0703(002C) 0D          | 
  0704(002D) 0A          | 
  0705(002D) 54          | 
  0706(002D) 41          | 
  0707(002D) 43          | 
  0708(002E) 48          | 
  0709(002E) 59          | 
  070A(002E) 4F          | 
  070B(002E) 4E          | 
  070C(002F) 20          | 
  070D(002F) 46          | 
  070E(002F) 6F          | 
  070F(002F) 72          | 
  0710(0030) 74          | 
  0711(0030) 68          | 
  0712(0030) 20          | 
  0713(0030) 56          | 
  0714(0031) 31          | 
  0715(0031) 2E          | 
  0716(0031) 30          | 
  0717(0031) 20          | 
  0718(0032) 00          | 
  0719(0032) 77          |         byte  PUSH1,15,MASK,_8,MASK
  071A(0032) 0F          | 
  071B(0032) 4F          | 
  071C(0033) 35          | 
  071D(0033) 4F          | 
  071E(0033) 96          |         byte  RCALL,@DO7-@MYCON
  071F(0033) 24          | 
  0720(0034) 09          | DO7     byte  DUP,RCALL,@DO8-@PRTLONG
  0721(0034) 96          | 
  0722(0034) 53          | 
  0723(0034) 77          | DO8     byte  PUSH1,32,_REV
  0724(0035) 20          | 
  0725(0035) 4D          | 
  0726(0035) 96          |         byte  RCALL,@DO5-@TXSPI
  0727(0035) 30          | 
  0728(0036) 2B          | DO5     byte  _0,PUSH2,$02,00,RCALL,@DO9-@DUMP
  0729(0036) 76          | 
  072A(0036) 02          | 
  072B(0036) 00          | 
  072C(0037) 96          | 
  072D(0037) 55          | 
  072E(0037) A0          | DO9     byte  _AGAIN,@DO6-@DEMO
  072F(0037) 2E          | 
  0730(0038)             | DO6
  

  
  Please note too that Forth is only coded this way in the Spin compiler to form the kernel after which the Forth itself would handle normal text input for compiling which would look like this:

  : TXSPI       32 CLKBITS ;
  : MYCON       $A500FC01 ;       \ Could also be coded as: $A500FC01 CONSTANT MYCON
  : DEMO
          begin
            cr ." TACHYON Forth V1.0 "            \
            15 MASK 8 MASK MYCON                  \ Assemble clockmask outmask and data for SPI
            DUP PRTLONG                           \ Let's see what is going to be sent
            32 REV                                \ Make it MSB first
            TXSPI                                 \ Off it goes
            0 $200 DUMP                            \ Hex dump of first 512 bytes of RAM
          again ;                                 \ Continue forever
  

  
  So I'll try not to bore you any further with any details suffice to say that there are a lot of very neat things going on and planned. With the dictionary (names of functions and pointers etc) in external memory such as EEPROM and SD there will be a lot of program code that will be able to be squeezed into just a few k of RAM, count the bytes that are used in the demo! Some code will be available very soon now. (Hey Cluso, I hear it's going to rain all week )

  ---

  Tachyon Forth - compact, fast, forthwright and interactive
  
  --->CLICK THE LOGO for more links<---
  P2 +++++ TAQOZ INTRO & LINKS +++++ P2 SHORTFORM DATASHEET
  P1 +++++ Latest binary V5.4 includes EASYFILE +++++ Tachyon Forth News Blog
  Brisbane, Australia
  
• mindrobots Posts: 6,506

  2012-07-09 - 07:58:46 edited 2012-07-09 - 07:58:46

  Excellent work Peter! This is exciting!
  
  I'm doing a rain dance on this side of the world because we need the rain.....I'll add some "remote location" rain dancing for purely selfish reasons!

  MOV OUTA, PEACE <div>Rick </div><div>"I've stopped using programming languages with Garbage Collection, they keep deleting my source code!!"</div>
• Dave Hein Posts: 5,897

  2012-07-09 - 08:02:13 edited 2012-07-09 - 08:02:13

  Peter,
  
  Do you think your interpreter could be applied to Spin to improve the speed? One approach would be to compile Spin to your bytecodes. Another approach may be to use your instruction-decoding technique to decode the existing Spin bytescodes.
  
  Also, maybe it would be possible to compile C to your bytecodes. Can you describe your VM in more detail? It appears to be stack-based, but also includes an accumulator. Are there any other registers in the VM?
  
  Dave
• Peter Jakacki Posts: 8,197

  2012-07-09 - 09:04:10 edited 2012-07-09 - 09:04:10

  Dave Hein wrote: »

  Peter,
  
  Do you think your interpreter could be applied to Spin to improve the speed? One approach would be to compile Spin to your bytecodes. Another approach may be to use your instruction-decoding technique to decode the existing Spin bytescodes.
  
  Also, maybe it would be possible to compile C to your bytecodes. Can you describe your VM in more detail? It appears to be stack-based, but also includes an accumulator. Are there any other registers in the VM?
  
  Dave

  
  I just had a look at the Spin bytecodes and it ain't fun, there's no way you could code all that and still cram it into just the cog. Tachyon Forth bytecode operations are fairly simple, just like PASM, but they are very flexible and implement a simple virtual stack based processor. The reference to an accumulator is really nominal as this is just a location to shift and accumulate literal values. The so named "accumulator" is cleared for next use after every push onto the datastack. There are other temporary registers also named for convenience such as R,X,Y,Z,R0..R3 as well as the IP which is equivalent to the PC in a real processor. The "X" register is used a lot for passing a value without upsetting the tos (top of stack) value as you can see in the GETBYTE and ACCBYTE routines. Creating virtual registers is not a problem though.
  
  Stack manipulation can be very easy and transparent on some processors but the Prop isn't one of them, there are no auto increment/decrement indexed instructions. I push and pop my stacks by physically moving values which sounds kind of brute force'ish but I worked out that this is still far more efficient as the PASM routines can access all stack items (not just tos) directly without any extra overhead so rotating and swapping etc is very fast and compact. The push and pop operations only take a tiny bit longer than a conventional stack implemented with a pointer anyway. Upon detecting non-zero values "falling" off the bottom of the stack I jump to an error processing routine but who cares if zero values "fall off" as I also pump zero values back into the bottom of the stack when it's popped.
  
  As for compiling from C to these bytecodes there shouldn't be any problem at all as it would only be the PASM code that fits in the VM cog that would be required to run them. It's a bit like compiling from C to Java bytecodes and using the JVM but a lot simpler of course and without all the overhead that would normally be involved. At present the Tachyon bytecodes are not fixed in value as the VM is in a state of flux but even so I don't think that there would be any requirement for portable bytecode normally. The symbol address of the bytecode function is the same as the bytecode which is why I can just reference them directly with the "byte" directive in a DAT section.
  
  Hope that sort of answers your questions and when i release some source soon you will be able to have a good look yourself to see if Tachyon bytecode is suitable for your task.

  ---

  Tachyon Forth - compact, fast, forthwright and interactive
  
  --->CLICK THE LOGO for more links<---
  P2 +++++ TAQOZ INTRO & LINKS +++++ P2 SHORTFORM DATASHEET
  P1 +++++ Latest binary V5.4 includes EASYFILE +++++ Tachyon Forth News Blog
  Brisbane, Australia
  
• Dave Hein Posts: 5,897

  2012-07-09 - 09:23:44 edited 2012-07-09 - 09:23:44

  Can you publish a complete list of the Tachyon bytecodes? It seems that Spin could be compiled to Tachyon bytecodes with about the same code size, but faster speed. Most of the Spin bytecodes are used to support various addressing modes, but this takes up a lot of space in the Spin interpreter. It may be more efficient to use a simpler VM, and do some operations with multiple bytecodes. This way the interpreter could be streamlined to a smaller set of intructions, and could run faster. It's sort of a RISC approach. Maybe a common streamlined bytecode VM could be developed that could be used by Spin, C, Forth and any other high level languages.
• Phil Pilgrim (PhiPi) Posts: 22,254

  2012-07-09 - 09:50:08 edited 2012-07-09 - 09:50:08

  Peter,
  
  I like your stack philosophy. There's no reason to have a huge stack, except for deep recursion. One annoyance of Forth, of course, is stack maintenance. It would be handy to add two more kernel instructions, pushmark and poptomark. These allows one to punctuate the stack in such a way that post-operative cleanup requires only a poptomark without having to know how much garbage remains. The "mark" deosn't really have to exist in the stack itself, but can be tracked either via a rotating bitmask that's synchronous with the stack, or a separate mark stack whose top element keeps track of the relative position of the next mark (i.e. increments on a push, decrements on a pop, and gets popped off when it goes negative).
  
  -Phil

  “Perfection is achieved not when there is nothing more to add, but when there is nothing left to take away. -Antoine de Saint-Exupery
• Peter Jakacki Posts: 8,197

  2012-07-10 - 21:30:35 edited 2012-07-10 - 21:30:35

  This bytecode is proving to be very compact as I have tediously coded in the high-level words that are needed to handle the text input, parsing, interpreting/compiling and number conversion etc. How compact? The bytecode for all this with the bells and whistles is under 1k.
  
  Some changes:
  1) Added a jump table for kernel bytecodes as I originally planned. This simplifies calling them from the Spin compiler and allows these functions to be anywhere.
  The table allows for up to 256 vectors which so that a call looks like:..... BCALL,xPARSE ..... where PARSE is referred to by the byte label xPARSE. Now I don't have to do a relative call with all that awkward setup for the Spin tool as in:...... RCALL,@L1-@PARSE .....where I also have to create a label such as L1 that follows everytime. The DATA section has it's advantages as I just set the ORG to 0 before the table so that each entry has consecutive values from 0 up to the maximum reference of 255.
  
  2) Added a local register bank which is great for storing temporary values and settings etc. This makes the kernel bytecode a lot easier too as it doesn't have to create special variables for number bases and interpreter flags etc. The registers used by the kernel are referred to in the CON section where they are created with a simple:
  

  ' REGISTER byte OFFSETS - 64 bytes
          #32,NTIB,IN,NUMFLG,FLAGS,BASE
  

  While the reference in the Spin tool to the register is in the form:

  COMMENT byte  REG,NTIB,CFETCH,REG,IN,CSTORE,EXIT
  

  and compiled:

  091E(002A) DF          | COMMENT byte  REG,NTIB,CFETCH,REG,IN,CSTORE,EXIT   ' ignore the rest of the text line
  091F(002A) 20          | 
  0920(002B) 62          | 
  0921(002B) DF          | 
  0922(002B) 21          | 
  0923(002B) 6A          | 
  0924(002C) 05          |
  

  
  3) Enhanced number processing so that sensible prefixes and suffixes can be used:

  Prefixs:
      -    negative number
      $    hex number ($0F00)
      %    binary number 
      '    ASCII character ('H') used in the form..... '*' EMIT
      ^    ASCII control character (^G) used in the form..... KEY? ^C = IF .....
   Suffixes:
      h    hex number (0F00h)
      d    decimal number (1000d)
      b    binary number (11000101b)
  
   Others:
       .    decimal point - decimal place stored in DPL
      _  standard Spin digit separator (ignored)
  

  
  
  4) 5) 6) lots more stuff
  
  I will probably allow direct addressing of stack items so that .... 3 STACK ... will return the address of the third stack item which can be manipulated just like any other variable.
  
  @Phil:
  I have tinkered before with shadow stacks that hold information about the stack items, a bit like a type identifier. But although I understand what you are saying and ways to implement it I'm not sure how useful this would be, at least to a Forth programmer. You see, although the stack can be a nuisance especially when it is misused and layers deep, part of coding any Forth is the art of factoring things into small, clear, and manageable chunks of code. My Tachyon Forth is being driven by need, by actual embedded control use so I tweak it to optimize what I really need. But could you please give me an example of where your suggestions would really excel? Thanks.
  

  ---

  Tachyon Forth - compact, fast, forthwright and interactive
  
  --->CLICK THE LOGO for more links<---
  P2 +++++ TAQOZ INTRO & LINKS +++++ P2 SHORTFORM DATASHEET
  P1 +++++ Latest binary V5.4 includes EASYFILE +++++ Tachyon Forth News Blog
  Brisbane, Australia
  
• Phil Pilgrim (PhiPi) Posts: 22,254

  2012-07-10 - 22:01:22 edited 2012-07-10 - 22:01:22

  Peter,
  
  My suggestion arises from comparative experience programming in both Forth and Postscript. The latter has mark and cleartomark words that make stack maintenance a breeze compared to Forth. One simple example of their utility can be seen when a subroutine needs to abort, returning the stack to a known state upon exit. Depending upon how much stuff got pushed on before the fault condition was encountered, this could be a real chore without a way simply to wipe out the garbage down to a known point in the stack.
  
  -Phil

  “Perfection is achieved not when there is nothing more to add, but when there is nothing left to take away. -Antoine de Saint-Exupery
• Peter Jakacki Posts: 8,197

  2012-07-10 - 22:14:45 edited 2012-07-10 - 22:14:45

  Phil,
  Although Postscript may be based on Forth it is also very non-Forth'ish because of it's huge complexity. I understand the Postscript requirements but how useful would that really be in the simpler embedded Forth model with lots of "small" words? But I will see if this can be implemented without too much fuss, especially since I am looking at being able to create local variables straight from the stack description which is normally only a comment such as: ( pin channel -- flg ) ..... and referring directly to pin and channel etc. However EXIT or it's cousin will then have to clean-up the stack and place whatever results there are onto the stack.

  ---

  Tachyon Forth - compact, fast, forthwright and interactive
  
  --->CLICK THE LOGO for more links<---
  P2 +++++ TAQOZ INTRO & LINKS +++++ P2 SHORTFORM DATASHEET
  P1 +++++ Latest binary V5.4 includes EASYFILE +++++ Tachyon Forth News Blog
  Brisbane, Australia
  
• Peter Jakacki Posts: 8,197

  2012-07-12 - 22:04:53 edited 2012-07-12 - 22:04:53

  I'm not really ready yet to release the source for this but in a way I am doing so. I've been experimenting with formatting the source code in Google Docs (<-- click here) and just copying and pasting the formatted text into BST to make sure that it runs. Well, guess what?, it does!. So this is more an exercise into posting code in fancy format as a shared document just to see how it will work. This is also a good opportunity for those who want to get their hands on to the source for Tachyon just to play with until I manage to get the text interpreter all sorted out. BTW, I copied the higher-level source from an old Forth I wrote for the ARM chip several years ago and although it worked there it is ugly and I am in the process of re-doing it all. So just a little while longer but in the meantime have a good look.
  
  The Google Docs format makes it easy to view and even download in various formats. Of course if you want to try out the code then just select all and copy&paste into your Spin IDE. The settings are so that anyone with the link can view and comment and if anyone wants to be able to edit this document as part of a collaborative effort then please just email me. There's also the chat box you can use when you have the document opened.

  ---

  Tachyon Forth - compact, fast, forthwright and interactive
  
  --->CLICK THE LOGO for more links<---
  P2 +++++ TAQOZ INTRO & LINKS +++++ P2 SHORTFORM DATASHEET
  P1 +++++ Latest binary V5.4 includes EASYFILE +++++ Tachyon Forth News Blog
  Brisbane, Australia
  
• msrobots Posts: 2,469

  2012-07-12 - 23:10:41 edited 2012-07-12 - 23:10:41

  @Peter.
  
  now I need to start thinking backwards.This is nice. Even I might understand FORTH now - Will study your pasm ...
  
  Thank you
  
  Mike

  I am just another Code Monkey.
  A determined coder can write COBOL programs in any language. -- Author unknown.
  Press any key to continue, any other key to quit
  
  The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this post are to be interpreted as described in RFC 2119.
• Peter Jakacki Posts: 8,197

  2012-07-12 - 23:15:17 edited 2012-07-12 - 23:15:17

  Yes, the fancy formatting makes it a lot easier to read, even for me. It's certainly a lot clearer to see where the flow goes and I can insert images etc without upsetting the text itself. So you can just copy and paste or download as text to compile.
  
  Another great feature is that this "source" is live, you may even see things change before your eyes as I am formatting it or making code changes !!! Maybe this way should be the way we format source for the gold standard?
  
  Fancy formatted source code in Google Docs

  ---

  Tachyon Forth - compact, fast, forthwright and interactive
  
  --->CLICK THE LOGO for more links<---
  P2 +++++ TAQOZ INTRO & LINKS +++++ P2 SHORTFORM DATASHEET
  P1 +++++ Latest binary V5.4 includes EASYFILE +++++ Tachyon Forth News Blog
  Brisbane, Australia
  
• Humanoido Posts: 5,770

  2012-07-14 - 04:07:01 edited 2012-07-14 - 04:07:01

  Peter, congrats on this spectacular project! You mentioned CogForth and how you reworked it to create Tachyon Forth. I'm interested in a copy of the original CogForth, in any condition, for its great historical value.

  Humanoido
• prof_braino Posts: 4,312

  2012-07-14 - 13:54:06 edited 2012-07-14 - 13:54:06

  That looks pretty cool. How do you do the formatting? Is there a link to how-to instructions?
• Peter Jakacki Posts: 8,197

  2012-07-14 - 17:25:09 edited 2012-07-14 - 17:25:09

  Hi prof, this is a Google cloud document so all the normal editing tools are there online when you are signed in and you have permission to edit. For some reason some may not be able to access this document although I had no problems on many different systems as guest and only one person has complained so far that their browser didn't render it properly..grumble grumble. I've tried accessing this on several completely clean systems with different browsers etc and it all seems to work fine. What I like about this type of formatted document is that it is a very good way to document as you go and it makes important parts of the source "pop out" while subduing others. Images are a bonus too. I've probably gone a little wild with some of the colors on this document but I am experimenting at present to see what does work. It's actually faster to edit in this document normally and when it comes time to compile it takes less than 3 seconds to "select all - copy& paste into BST and F10.
  
  The other advantage is that this is a live document, what you see is what I see and what I am changing.

  ---

  Tachyon Forth - compact, fast, forthwright and interactive
  
  --->CLICK THE LOGO for more links<---
  P2 +++++ TAQOZ INTRO & LINKS +++++ P2 SHORTFORM DATASHEET
  P1 +++++ Latest binary V5.4 includes EASYFILE +++++ Tachyon Forth News Blog
  Brisbane, Australia
  
• prof_braino Posts: 4,312

  2012-07-14 - 17:36:33 edited 2012-07-14 - 17:36:33

  I think I want to try this for propforth. So how do you get the context highlighting for forth? In the PSPad text editor, we can set up the set of forth control words, and it automagically does the context highlights. Do we have to set up up something to define the set of forth words, or do you have to do the color for every word as you type it in?
• Peter Jakacki Posts: 8,197

  2012-07-14 - 20:34:47 edited 2012-07-14 - 20:34:47

  Even with automagic converters you still have to have it formatted correctly plus there are some things that you don't want it to do automagically anyway. Well that's my argument against it No but the Google document is manually formatted just like any word processor and if you are formatting as you are entering text (vs pasting an existing text file) then you make it stand out just as you would like etc. My suggestion is to give it a try, it can't hurt and you can play with it to see how well it works plus the source code is always intact.

  ---

  Tachyon Forth - compact, fast, forthwright and interactive
  
  --->CLICK THE LOGO for more links<---
  P2 +++++ TAQOZ INTRO & LINKS +++++ P2 SHORTFORM DATASHEET
  P1 +++++ Latest binary V5.4 includes EASYFILE +++++ Tachyon Forth News Blog
  Brisbane, Australia
  
• Peter Jakacki Posts: 8,197

  2012-07-18 - 08:59:32 edited 2012-07-18 - 08:59:32

  Well I'm starting to get somewhere with this beyond the VM part of it that is. I am staring off simple again after scrubbing out some ugly code I imported and tried to use from another Forth project. Now I have a simple text interpreter running that executes immediately on each word for the moment. The idea is that even in interpretive mode text will be compiled into a temporary location word by word (as it is entered) and then that complied code executed on an CR. The dictionary is running out of RAM at present but I will be copying this over the 2K image of the Tachyon's VM kernel that lies unused after boot rather than worrying about accessing it in EEPROM. Once the dictionary is copied over the VM image there is still about 29K of RAM left at present. The structure of the dictionary is settling down now as I have this running nicely with non-counted string names terminated by an attribute byte and 2 bytecodes so this structure is very compact and easy to scan plus there are no link fields.
  
  I have also tested the serial transmit and receive to at least 2M baud at present and I will do some further tests at higher speeds later on. I'm also overlaying the serial receive cog's image as well with the receive buffer so there is no wasted memory.
  
  So far...so good.

  ---

  Tachyon Forth - compact, fast, forthwright and interactive
  
  --->CLICK THE LOGO for more links<---
  P2 +++++ TAQOZ INTRO & LINKS +++++ P2 SHORTFORM DATASHEET
  P1 +++++ Latest binary V5.4 includes EASYFILE +++++ Tachyon Forth News Blog
  Brisbane, Australia
  
• richaj45 Posts: 179

  2012-07-18 - 09:36:10 edited 2012-07-18 - 09:36:10

  Hello:
  
  So how is your coding doing 2M baud?
  Why is it so much faster than the standard Full Duplex 4 port object?
  Is the serial driver full duplex?
  When do you think the whole of the Tachyon code will be posted?
  
  Thanks in advance for your patience with all these questions.
  
  cheers,
  rich