Planned Tachyon V6 ZOOM - requesting input (and output)

In case you are wondering where I am going with Tachyon, and if I am going, then yes, Tachyon is an active project. So here are some plans and ideas for a new version to round out 2020.

There are a few idiosyncrasies I'd like to clean up in V5 NEON and also other improvements I'd make including making it more compatible with TAQOZ etc. For instance I have some weird stuff that happens on user input where you need to hit the enter twice but that's more to do with the experimental input method that was used. Then there are the different types of constants, and there is the INTERCOM that needs to be checked and upgraded. As I look at how I use the H and L I/O words using a pin mask from the stack I think that this should be more like TAQOZ where the mask/pin is specified by PIN etc. There are probably dozens of things I have forgotten but I'm sure you remember them so how about we work together on a version that has as few quirks as possible, is more compatible with TAQOZ, has proper signed/unsigned operations etc. Perhaps we can find "the most useless function" and show it the door while ushering some fresh faces in. This is everyone's chance to have some input, and hopefully some output. I might even github it if necessary.

I can't do much with the dictionary and in the past I moved most of it into EEPROM but even with the smart caching methods it still slowed down compilation a bit much for my liking. The way it is now it takes up about 7k with EXTEND and EASYFILE which isn't too bad. Moving it out of hub makes it nigh impossible to distribute an easy to use single binary. Logically, if we were using EASYFILE then at this point it is a larger more complex system and we would want to optimize RAM as much as possible. This is where the bulk of the dictionary could end up in a file and SD is so much faster to access than EEPROM. I could cache dictionary entries in a small buffer in RAM that is searched first and indeed new entries would be added there and the entry marked dirty for processing at some point, and certainly when a BACKUP is executed.

Bear in mind too that Tachyon was not developed for Forth's sake, but for using the Prop effectively, which is why it is not so standard as some would like. But if it were then it wouldn't be as compact and fast as it is and while you could play with standard Forth on it, it wouldn't be as useful in real projects. So don't think like an ANSI committee, think like hackers and come up with the tools that get the job done. So it helps if you have a real project you are working on and you can see how it can work better which is how I have developed Tachyon so far.

The upgrade scope includes not just the kernel, but also EXTEND and EASYFILE as well. Current V5 code name is NEON. Since V3 I normally pick a 4-letter NASA mission but I'm not tied to that so let's see what we can call V6. Maybe ZOOM seeing it is an aspect of the year of covid19, and Tachyon is fast, although I am busy enough and in no hurry to upgrade it myself. I will let users set the pace and help document it at the same time.

Tachyon is not just a language, it is a whole development and runtime system that maximizes the usefulness of the Prop beyond what is normally possible.
982 x 632 - 74K

Comments

  • Some ideas:
    - Don't require sdcard storage. My projects work w/o this up to now. I don't want to add a socket to my boards.
    - I also don't need standards compatibility.
    - Make input case insensitive again. On a usual German keyboard there really is no caps lock function, it acts as shift lock. So I have to press shift too often.
  • @rbehm - SD card is only ever an option and not required but many of my projects use it extensively. If it were added then the extended dictionary could be stored on file with the kernel words in RAM.

    I had autocase before where lowercase would be translated to uppercase when it wasn't found the first time. This gives the best of both worlds. I didn't know that about the German keyboard though, that would be a PITA, or should I say pita :)
  • I'm too new to Tachyon to dare making suggestions at this point but if you could just estimate a time frame for this possibility to be open that would be great. In 2-3 weeks I think I'll be brave enough to try :)
  • We should stay with case sensitivity as CmlCse make appriviations readable and there should no ambiguity allowed. I hate to have two ways to reach a goal ;-)
  • Peter JakackiPeter Jakacki Posts: 9,597
    edited 2020-09-08 - 00:01:26
    ErNa wrote: »
    We should stay with case sensitivity as CmlCse make appriviations readable and there should no ambiguity allowed. I hate to have two ways to reach a goal ;-)

    That's why I forced it into case sensitive mode because I got tired of finding dumb case bugs. Autocase helps when we are interacting at the console but is disabled when we load source. These days I try to avoid having a lower and upper case version of the same word as in "trap" vs "TRAP" which I just noticed in EXTEND. What I try to do now is use an underscore prefix for the variable perhaps.

    So if I abolished case sensitivity then trap and TRAP and Trap are all the same word. I would have to have warning messages that a new definition is not unique which might slow down compilation since it has to search the entire dictionary to find out.

    In general I prefer uppercase, why? Well UPPERCASE characters use the screen pixels more effectively instead of "x", we have "X". Why bother with hires fonts if we only use half of them? But coders have lowercase ingrained in them as the "proper" method and seem to think that there is something wrong with uppercase!*&%^$#!

    Uppercase makes each character more discernible without making the font larger, just simply using the font fully. Imagine if we wrote numbers in lower case too!
    abcdefghijklmnopqrstuvwxyz
    ABCDEFGHIJKLMNOPQRSTUVWXYZ
    
  • MJBMJB Posts: 1,192
    edited 2020-09-08 - 15:03:33
    ErNa wrote: »
    We should stay with case sensitivity as CmlCse make appriviations readable and there should no ambiguity allowed. I hate to have two ways to reach a goal ;-)

    That's why I forced it into case sensitive mode because I got tired of finding dumb case bugs. Autocase helps when we are interacting at the console but is disabled when we load source. These days I try to avoid having a lower and upper case version of the same word as in "trap" vs "TRAP" which I just noticed in EXTEND. What I try to do now is use an underscore prefix for the variable perhaps.

    So if I abolished case sensitivity then trap and TRAP and Trap are all the same word. I would have to have warning messages that a new definition is not unique which might slow down compilation since it has to search the entire dictionary to find out.

    In general I prefer uppercase, why? Well UPPERCASE characters use the screen pixels more effectively instead of "x", we have "X". Why bother with hires fonts if we only use half of them? But coders have lowercase ingrained in them as the "proper" method and seem to think that there is something wrong with uppercase!*&%^$#!

    Uppercase makes each character more discernible without making the font larger, just simply using the font fully. Imagine if we wrote numbers in lower case too!
    abcdefghijklmnopqrstuvwxyz
    ABCDEFGHIJKLMNOPQRSTUVWXYZ
    

    Case or not CASE
    actually I like exactly what we have.
    CaseSensitivity and on Input the optional AUTO UPPERCASE since by convention ACTION words are in UPPER anyhow
    and lower case myvariable things are not what you usually type at the command prompt interactively anyhow.
    And as you say - at fileIn the AUTO UPPERCASE is disabled to not mask any typos.

    Why change it?
    And if you change it - how many old code will not run any more ?

    ADDIT:
    What I try to do now is use an underscore prefix for the variable perhaps.
    for internals might be ok,
    but anything the user sees or types ... better not.
    Would start the slippery path to what we know from MS-C++ coding conventions - better not start.
    Having code that reads almost like human language is such a great plus in Forth imho.

  • Peter JakackiPeter Jakacki Posts: 9,597
    edited 2020-09-09 - 02:36:13
    @MJB - I try to use constants rather than variables for readability but changing a constant is not always clear and simple :)

    Perhaps I can create two types of constants then, one that is immutable in that the value would be embedded when referenced at compile time such as the 15-bit defined constants are now (not 15-bit literals). The other type would be the mutable constant which has its own code definition and is normally a 32-bit long, although it could be a 64-bit double.

    By way of suggesting (but not implementing)
    1000 is my1k
    $42 := device
    

    So when "1k" is referenced it compiles the literal value of 1000, but when "device" is referenced it compiles the address to that constant. Changing the value of the constant instantly changes it to everything to references it.

    Reading "device IO@" rather than "device C@ IO@" is a lot cleaner and normally device would only change when there is a change to the configuration.

    Of course constants can be created as a normal Forth colon definition too which is sometimes useful when multiple mutable constants are involved such as clock frequency etc. But in this case it is not directly mutable.

    Compiled size of constants:
    15-bit literal = 2 bytes encoded as $8000..$FFFF
    16-bit literal = 4 bytes with 2 bytes for code pointer to _WORD which reads the next 2 byte literal
    32-bit literal = 6 bytes with 2 bytes for code pointer to _LONG which read the next 2 bytes as upper 16-bits and then the lower 16-bits
    64-bit literal = not supported yet but is simply two lots of 32-bit literal, or 12 bytes + EXIT

    All references to constants only take 2 bytes.
  • Peter

    With the advent of the P2, it appears to me that Tachyon will be the main player. You have devoted a lot of time and effort into it.
  • Peter JakackiPeter Jakacki Posts: 9,597
    edited 2020-09-11 - 04:25:31
    idbruce wrote: »
    Peter

    With the advent of the P2, it appears to me that Tachyon will be the main player. You have devoted a lot of time and effort into it.

    Spin and C are the main players for P2 and Chip's built-in debug looks good too. I will look to see if I can leverage off it and allow TAQOZ to talk to PNut the same way.

    However, for the humble P1, I think Tachyon is the only way to use the P1 effectively and to it's max. But for small projects Spin has always been very nimble too.
  • Peter

    There is a trend. Even the serious players are turning to Tachyon :wink:
  • Peter Jakacki is certainly a force to be rewckoned with :wink:
  • bob_g4bbybob_g4bby Posts: 65
    edited 2020-09-14 - 09:51:06
    1. I've been looking at converting RS485.fth to Tachyon 5 whilst on holiday. Because of the change to HSUART ROM's parameter record, it'd now be useful to define an array of records (up to 5, one parameter record per HSUART loaded). org and ds seem to be OK for single records, but not arrays of records. I have some untested words for that. Same thing would be useful for the data buffers.
    2. Loading EXTEND.fth is essential for anyone making a standalone program, for the word BACKUP. But, I would split out all the 'drivers' from EXTEND.fth as separate files - they're all useful, but only if you need them, otherwise they're just bloat. I was looking to make separate files of ANSI terminal support, Simple Serial, Loudspeaker noises, maybe PCF8574 style I2C, Realtime clock, Ultrasound ping, DH22 humidity/temp driver, NEC IR transmit, WS2812 driver, Garden lights, PBASIC style shifts, PWM, Servos, F32 support. That would liberate a lot of memory for a user program and/or more data.
    3. I volunteer to help do a glossary when the dust has settled
  • bob_g4bby wrote: »
    ...
    2. Loading EXTEND.fth is essential for anyone making a standalone program, for the word BACKUP. But, I would split out all the 'drivers' from EXTEND.fth as separate files - they're all useful, but only if you need them, otherwise they're just bloat. I was looking to make separate files of ANSI terminal support, Simple Serial, Loudspeaker noises, maybe PCF8574 style I2C, Realtime clock, Ultrasound ping, DH22 humidity/temp driver, NEC IR transmit, WS2812 driver, Garden lights, PBASIC style shifts, PWM, Servos, F32 support. That would liberate a lot of memory for a user program and/or more data.
    ...
    Good points here but it may not be that easy depending on how the words are interconnected. I think Peter proceeded with the development in Tachyon as the particular need arose so sorting these interconnections might not be trivial. It might even be easier (but almost certainly not faster) to write the necessary parts anew. But in general the concept of modular build appeals to me.
  • All these "drivers" were eventually integrated into EXTEND for several reasons, one being the fact that they could easily be. Many newbies would ask "how can I interface this" and so it was easier to have all there in the first place. Many of these "drivers" were nothing more than a few lines of code anyway. The other thing is that I wanted to emphasize how comprehensive Tachyon is by having an "obex" all on-board at your fingertips.

    In the early days of Android phones there were apps that would kill tasks to "free up memory" which resulted in having all this free memory. What was the purpose of memory in the first place you may ask? :)

    Same too with Tachyon and there was more than enough memory to start with but if anyone was actually getting serious with code that took up more than a few k then by that stage they knew what they could do if they needed more memory. Although I don't ever recall anyone ever needing more Tachyon memory, just wanting to have more memory free :)

    Despite the sheer compactness and speed of Tachyon, it has attracted relatively few takers with some more "comfortable" with what they already know, although that proves to be a long and frustrating path that often comes to a dead-end. I suppose that there is this mentality that pervades that "programming" is done "this way" and not "that way" and so they end up trying to do "that way" "this way" which obviously doesn't work that way.

    As I always say, just apply the same amount of effort and patience to Tachyon, and reap the rewards of your hard work in a fraction of the time and have fun doing it.
  • All these "drivers" were eventually integrated into EXTEND for several reasons, one being the fact that they could easily be. Many newbies would ask "how can I interface this" and so it was easier to have all there in the first place. Many of these "drivers" were nothing more than a few lines of code anyway. The other thing is that I wanted to emphasize how comprehensive Tachyon is by having an "obex" all on-board at your fingertips.
    ...
    ...just apply the same amount of effort and patience to Tachyon, and reap the rewards of your hard work in a fraction of the time and have fun doing it.

    Sounds sensible to me. Let's just leave it at that. For now. There is never a solution that suits all at once. Nothing wrong with that.
  • MJBMJB Posts: 1,192
    Maciek wrote: »
    bob_g4bby wrote: »
    ...
    2. Loading EXTEND.fth is essential for anyone making a standalone program, for the word BACKUP. But, I would split out all the 'drivers' from EXTEND.fth as separate files - they're all useful, but only if you need them, otherwise they're just bloat. I was looking to make separate files of ANSI terminal support, Simple Serial, Loudspeaker noises, maybe PCF8574 style I2C, Realtime clock, Ultrasound ping, DH22 humidity/temp driver, NEC IR transmit, WS2812 driver, Garden lights, PBASIC style shifts, PWM, Servos, F32 support. That would liberate a lot of memory for a user program and/or more data.
    ...
    Good points here but it may not be that easy depending on how the words are interconnected. I think Peter proceeded with the development in Tachyon as the particular need arose so sorting these interconnections might not be trivial. It might even be easier (but almost certainly not faster) to write the necessary parts anew. But in general the concept of modular build appeals to me.

    In the early days of Tachyon 2.x I was working my way through EXTEND to identify the modules and their dependencies. So at build time you could specify what modules you wanted.
    But with the ongoing improvements in speed and compactness of Tachyon
    And the FORGET and COMPACT words there was never really a need for it.
    You could always comment out some sections of EXTEND at build time.
    So at a next mayor revision @Peter Jakacki removed my module system again since it was not really needed and not used by anybody besides me anyhow ;-)
  • Peter JakackiPeter Jakacki Posts: 9,597
    edited 2020-09-15 - 00:20:02
    @MJB - that's right, but it was awkward keeping track of the bit fields used to specify what was being loaded etc. I also had a complicated caching dictionary scheme in upper EEPROM in V3 I think to make more RAM available too. As good as V3 was it needed a breakthrough so that Ethernet could fit it into 32k without having to weed the garden everytime, and that breakthrough came from using "more memory", wordcode instead of bytecode :) The problem had been that a lot of the bytecodes were taking 3 bytes for literals and calls and of course there was the vector table to translate calls that took memory and needed maintaining etc. These features had been great in smaller builds but hampered Tachyon when it grew too big for its bytecode boots.

    At present I have TOOLS loaded onto the end of EXTEND which can be trimmed off simply with a FORGET since it is fast and easier to load it first, then trim. RECLAIM finds all those private words and removes them from the dictionary, compacting it up as it goes. There are a lot of PASM operations that take two values and return one that are encoded for free. At V5.7 I broke up the kernel source into sections to be preprocessed so each section could be included from its own source file. This generated a conventional .spin file and I pass that to bstc to compile and load with clock options specified on the command line.

    What's wrong with V5? Not much really but one of the things I want to do is overhaul the dictionary and also the console routines and make it more like TAQOZ and pass improvements on to TAQOZ as well.

    Since we like to type interactively at the console input and that has a lot to do with productivity, I'm thinking of breaking up the console input into a couple of levels where in the first level you can input text and backspace etc and when it encounters a space or control character other than BS it will pass the pointer to the text to the console where text input is parsed (I think this is where a lot of people get confused thinking Forth is "interpreted"). This isolates the text parser from the input so it is more streamlined and able to handle blocking and non-blocking input. A very short console history buffer would be useful here, even one line's worth and you could also backspace past the current word which you can't at present since the previous has already been compiled (editing like this would just reset the current compiled code for that line).

    None of this makes Tachyon faster or more compact though, and I'm not sure I can either, but I want to.

    I'm starting to think that anything different makes a system "unfriendly" to many who are afraid to put the effort in to learn. Some differences are its strength, but some could be shaped a bit more. Even those familiar with some kind of Forth want Tachyon to be more like that kind of Forth they are used to (that isn't suitable anyway).

    V5 NEON is a very optimized Tachyon as it is. Could it be better? To make it so requires more than rearranging the furniture, so think deep, think laterally, and think practically as if you were using it in a real project.

  • I'm a big Tachyon fan as you know, but also constructively critical

    "Despite the sheer compactness and speed of Tachyon, it has attracted relatively few takers " The nature of Forth probably limits numbers that can stand writing in a language littered with mnemonics. But, like I've said before there are additional barriers to the newcomer becoming effective with tachyon which are presently higher than they could be. What can we (not just Peter) do to reduce these without making it a life's work?
    1. Despite the reams written, the documentation is only average, scattered, some out of date. (Look at the docs for Spin in comparison) Hence the suggested glossary update as a start. The present one is incomplete for 5v7.
    2. Small examples of each tachyon specific word in <wordname(s).fth included in the forth folder. Needs be 3 lines only but what a difference for the beginner. Suggest we all aim to write one a week else put a question on the forum - 'hey how does this work, I can't find a description?' Go and find a tachyon word or construct that isn't described properly ;-) It would take less than a year to knock em all down.
    3. Tachyon is a moderately fat forth. This makes for a large word set that the newcomer may find daunting. That's another reason I suggest breaking it back out into language and add-ons. More byte size chunks which can be more easily digested before moving on to explore more
    4. 5 min videos showing the application of a tachyon feature
    5. The dropbox is a very confusing place full of debris from past experiments. If something turns out it doesn't work, that turns newcomers right off. Prune it back hard - people can ask for it if they really want it. The really old stuff is only useful to Peter, I suggest.

    I would suspect that there is little processing speed increase to be had, since the system has been revised once and thoroughly thought over for so long. It would be dominated by the inner interpreter speed, I guess?

    The tachyon specific constructs don't bother me as long as I know how to use them and I can see the advantage they bring. I couldn't care less about ansi standards. As soon as you start building your own defining words, you've gone non-standard anyway. Tachyon isn't that far off standard anyway - it's till a recognizable forth.

    I find myself typing in the wrong case very often, swapping between text editor and terminal - but don't mind it.

    Lovely sunny day today - me and the missis out in our Lomax three-wheeler for lunch to meet with eldest daughter - being careful of course.
  • Good arguments from both Peter and bob_g4bby.

    Although I lack your experience in Tachyon as I am fairly late adopter I do, however, have a newbie perspective. And maybe I'll just share my view instead of suggesting or criticizing things.

    From what I gather the main thing with Tachyon and all other things associated with it is that Peter created it and improved it to suit his needs along the way and was generous enough to just share his work with the community. It wasn't meant for, or it seems to me that way, general public to suit everybody. And it shows and this is perfectly fine.
    The words dictionary and the docs could and should be improved but I wouldn't waste Peter's time on that one. The same goes for the examples but I just hate the generic type of examples as they almost always do not work in my particular setup. I prefer an example that shows and explains how a particular goal is achieved and indicates how to configure whatever parameters are there to configure. No Forth will ever be mass adopted. It is very efficient and flexible and useful but a niche. Let us not make a goal out of making Tachyon No.1 choice and a perfect tool because in this we will, in my view, fail miserably.
    And one last thing - I've learned that the hard work and determination will not make anybody faster and smarter at the beginning but will help achieve whatever sensible goal there is to achieve with confidence and ease once all the sweat and tears dry out. One just doesn't appreciate anything that came too easy or free. Forth, any forth, is not for the lazy people and nothing can change that ever.
    In my view, provided one has a friendly and more experienced person to ask, the " I need to do this xxx and I am trying this yyy solution and it doesn't work and how do I proceed ? " is the fastest approach (save a few hopeless cases :) ).
  • Hey Bob, glad you enjoyed riding your trike mate! :)

    I've been migrating selected source over to SF rather than Dropbox which is more like my working repository. Should I disable shares to the Dropbox to encourage SF? I had already moved hardware out of the Tachyon folders too.

    Faster is better, but while I don't expect to gain much, if I do then that is "better". From what I've been saying it looks like I'm focusing on the user interface mostly though and gaining more memory too.

    As for the word set as long as the kernel words are described adequately then all other post compiled words can have that information in with the source code itself. This is where words can be categorized too and if they are using syntax highlighting then these documentation blocks will stand out more.
    However, that brings back to mind the fact that I was also creating help files at one stage which sat inside a HELP folder on the target if it was equipped with an SD or a serial Flash. Some of my boards have serial Flash as well as SD.

    There are so many unique features in Tachyon that are so unlike interactive Forths in general, yet Forths in general shouldn't be so traditional and standard and boring as they are. More than two stacks makes a lot of sense, as does compiling text input then executing rather than interpreting etc.

    @Maciek
    The best questions are questions that arise from trying to figure stuff out, so that you not only know and understand enough to know what to ask for, but that you are then able to understand the answer.
    Little boy asks why the sky is blue but any right answer he gets he is not able to understand, so to him it's a bad answer. So you just tell him the dumb answer that "they used up all the green on the grass and the red on the sunsets, and blue was all that was left" or something like that.
    So when someone asks why isn't Tachyon like abc Forth or C, or how do I do this (when they haven't even given it a go yet) then I am inclinded to tell them the dumb answer, although I don't, but maybe I should.
    Moral of this piece is: Ask yourself lots of dumb questions and discover the answers for yourself because you will really grok them then, but if it still eludes you, then ask someone and then you and they will understand what you are asking and so there won't be any dumb answers :)


    Excuse my ramblings, I will try to be more focused when I get to sit down and play with the code again. These new pcbs are keeping me busy and making me crazy!
  • bob_g4bbybob_g4bby Posts: 65
    edited 2020-09-15 - 17:44:49
    Hi Peter, hope you're well down there in lovely Brisbane,
    I see no problem in a single location for Tachyon code - and it'd be less work for you. Also, more descriptive names would help first timers e.g. 'Tachyon 5v7 115200 baud 80MHz.binary'. I downloaded a few before I got one that ran on my board. I can compile the source with BST now, so am over that 'hump'. Anyway, it's good to see that Tachyon is still an active project and with some useful goals in mind. I'm quite happy to write supporting documentation along the way - and the colder months lie ahead so will be more indoors.
  • If the P2 was in a smaller package and as simple to connect as the P1 then maybe there wouldn't be as much need for Tachyon on the P1. But no, the P1 is still going to be used where we don't need all the resources of the P2, and with the UB3 USB serial chip I think we have a minimal solution available teamed up with it as it can replace the EEPROM and clock which jmg has been doing some work on. Remember too that the UB3 is also the power monitor and watchdog as well as the USB serial. At some point I will do a new P1 board with just the P1 and the UB3 in a 50mil form factor for smd or headers, much like the P2D2 but much smaller.

    At a later stage again I may get some time to try out a P1V on some of the newer Lattice FPGAs in small packages and in this way we can have a very fast P1 with more memory, just as we have always wanted. However, if P2 is eventually made available in smaller packages then that is a good option too.
  • If the P2 was in a smaller package and as simple to connect as the P1 then maybe there wouldn't be as much need for Tachyon on the P1. But no, the P1 is still going to be used where we don't need all the resources of the P2, and with the UB3 USB serial chip I think we have a minimal solution available teamed up with it as it can replace the EEPROM and clock which jmg has been doing some work on. Remember too that the UB3 is also the power monitor and watchdog as well as the USB serial. At some point I will do a new P1 board with just the P1 and the UB3 in a 50mil form factor for smd or headers, much like the P2D2 but much smaller.

    At a later stage again I may get some time to try out a P1V on some of the newer Lattice FPGAs in small packages and in this way we can have a very fast P1 with more memory, just as we have always wanted. However, if P2 is eventually made available in smaller packages then that is a good option too.

    A perfect minimal P1 board/module. Perfect. I need (much more than) one.
  • bob_g4bbybob_g4bby Posts: 65
    edited 2020-09-16 - 22:06:24
    What increase in MIPS comes with the fpga P1?
Sign In or Register to comment.