Planned Tachyon V6 ZOOM - requesting input (and output)
Peter Jakacki
Posts: 10,193
in Propeller 1
In case you are wondering where I am going with Tachyon, and if I am going, then yes, Tachyon is an active project. So here are some plans and ideas for a new version to round out 2020.
There are a few idiosyncrasies I'd like to clean up in V5 NEON and also other improvements I'd make including making it more compatible with TAQOZ etc. For instance I have some weird stuff that happens on user input where you need to hit the enter twice but that's more to do with the experimental input method that was used. Then there are the different types of constants, and there is the INTERCOM that needs to be checked and upgraded. As I look at how I use the H and L I/O words using a pin mask from the stack I think that this should be more like TAQOZ where the mask/pin is specified by PIN etc. There are probably dozens of things I have forgotten but I'm sure you remember them so how about we work together on a version that has as few quirks as possible, is more compatible with TAQOZ, has proper signed/unsigned operations etc. Perhaps we can find "the most useless function" and show it the door while ushering some fresh faces in. This is everyone's chance to have some input, and hopefully some output. I might even github it if necessary.
I can't do much with the dictionary and in the past I moved most of it into EEPROM but even with the smart caching methods it still slowed down compilation a bit much for my liking. The way it is now it takes up about 7k with EXTEND and EASYFILE which isn't too bad. Moving it out of hub makes it nigh impossible to distribute an easy to use single binary. Logically, if we were using EASYFILE then at this point it is a larger more complex system and we would want to optimize RAM as much as possible. This is where the bulk of the dictionary could end up in a file and SD is so much faster to access than EEPROM. I could cache dictionary entries in a small buffer in RAM that is searched first and indeed new entries would be added there and the entry marked dirty for processing at some point, and certainly when a BACKUP is executed.
Bear in mind too that Tachyon was not developed for Forth's sake, but for using the Prop effectively, which is why it is not so standard as some would like. But if it were then it wouldn't be as compact and fast as it is and while you could play with standard Forth on it, it wouldn't be as useful in real projects. So don't think like an ANSI committee, think like hackers and come up with the tools that get the job done. So it helps if you have a real project you are working on and you can see how it can work better which is how I have developed Tachyon so far.
The upgrade scope includes not just the kernel, but also EXTEND and EASYFILE as well. Current V5 code name is NEON. Since V3 I normally pick a 4-letter NASA mission but I'm not tied to that so let's see what we can call V6. Maybe ZOOM seeing it is an aspect of the year of covid19, and Tachyon is fast, although I am busy enough and in no hurry to upgrade it myself. I will let users set the pace and help document it at the same time.
Tachyon is not just a language, it is a whole development and runtime system that maximizes the usefulness of the Prop beyond what is normally possible.
There are a few idiosyncrasies I'd like to clean up in V5 NEON and also other improvements I'd make including making it more compatible with TAQOZ etc. For instance I have some weird stuff that happens on user input where you need to hit the enter twice but that's more to do with the experimental input method that was used. Then there are the different types of constants, and there is the INTERCOM that needs to be checked and upgraded. As I look at how I use the H and L I/O words using a pin mask from the stack I think that this should be more like TAQOZ where the mask/pin is specified by PIN etc. There are probably dozens of things I have forgotten but I'm sure you remember them so how about we work together on a version that has as few quirks as possible, is more compatible with TAQOZ, has proper signed/unsigned operations etc. Perhaps we can find "the most useless function" and show it the door while ushering some fresh faces in. This is everyone's chance to have some input, and hopefully some output. I might even github it if necessary.
I can't do much with the dictionary and in the past I moved most of it into EEPROM but even with the smart caching methods it still slowed down compilation a bit much for my liking. The way it is now it takes up about 7k with EXTEND and EASYFILE which isn't too bad. Moving it out of hub makes it nigh impossible to distribute an easy to use single binary. Logically, if we were using EASYFILE then at this point it is a larger more complex system and we would want to optimize RAM as much as possible. This is where the bulk of the dictionary could end up in a file and SD is so much faster to access than EEPROM. I could cache dictionary entries in a small buffer in RAM that is searched first and indeed new entries would be added there and the entry marked dirty for processing at some point, and certainly when a BACKUP is executed.
Bear in mind too that Tachyon was not developed for Forth's sake, but for using the Prop effectively, which is why it is not so standard as some would like. But if it were then it wouldn't be as compact and fast as it is and while you could play with standard Forth on it, it wouldn't be as useful in real projects. So don't think like an ANSI committee, think like hackers and come up with the tools that get the job done. So it helps if you have a real project you are working on and you can see how it can work better which is how I have developed Tachyon so far.
The upgrade scope includes not just the kernel, but also EXTEND and EASYFILE as well. Current V5 code name is NEON. Since V3 I normally pick a 4-letter NASA mission but I'm not tied to that so let's see what we can call V6. Maybe ZOOM seeing it is an aspect of the year of covid19, and Tachyon is fast, although I am busy enough and in no hurry to upgrade it myself. I will let users set the pace and help document it at the same time.
Tachyon is not just a language, it is a whole development and runtime system that maximizes the usefulness of the Prop beyond what is normally possible.
982 x 632 - 74K
Comments
- Don't require sdcard storage. My projects work w/o this up to now. I don't want to add a socket to my boards.
- I also don't need standards compatibility.
- Make input case insensitive again. On a usual German keyboard there really is no caps lock function, it acts as shift lock. So I have to press shift too often.
I had autocase before where lowercase would be translated to uppercase when it wasn't found the first time. This gives the best of both worlds. I didn't know that about the German keyboard though, that would be a PITA, or should I say pita
That's why I forced it into case sensitive mode because I got tired of finding dumb case bugs. Autocase helps when we are interacting at the console but is disabled when we load source. These days I try to avoid having a lower and upper case version of the same word as in "trap" vs "TRAP" which I just noticed in EXTEND. What I try to do now is use an underscore prefix for the variable perhaps.
So if I abolished case sensitivity then trap and TRAP and Trap are all the same word. I would have to have warning messages that a new definition is not unique which might slow down compilation since it has to search the entire dictionary to find out.
In general I prefer uppercase, why? Well UPPERCASE characters use the screen pixels more effectively instead of "x", we have "X". Why bother with hires fonts if we only use half of them? But coders have lowercase ingrained in them as the "proper" method and seem to think that there is something wrong with uppercase!*&%^$#!
Uppercase makes each character more discernible without making the font larger, just simply using the font fully. Imagine if we wrote numbers in lower case too!
Case or not CASE
actually I like exactly what we have.
CaseSensitivity and on Input the optional AUTO UPPERCASE since by convention ACTION words are in UPPER anyhow
and lower case myvariable things are not what you usually type at the command prompt interactively anyhow.
And as you say - at fileIn the AUTO UPPERCASE is disabled to not mask any typos.
Why change it?
And if you change it - how many old code will not run any more ?
ADDIT: for internals might be ok,
but anything the user sees or types ... better not.
Would start the slippery path to what we know from MS-C++ coding conventions - better not start.
Having code that reads almost like human language is such a great plus in Forth imho.
Perhaps I can create two types of constants then, one that is immutable in that the value would be embedded when referenced at compile time such as the 15-bit defined constants are now (not 15-bit literals). The other type would be the mutable constant which has its own code definition and is normally a 32-bit long, although it could be a 64-bit double.
By way of suggesting (but not implementing)
So when "1k" is referenced it compiles the literal value of 1000, but when "device" is referenced it compiles the address to that constant. Changing the value of the constant instantly changes it to everything to references it.
Reading "device IO@" rather than "device C@ IO@" is a lot cleaner and normally device would only change when there is a change to the configuration.
Of course constants can be created as a normal Forth colon definition too which is sometimes useful when multiple mutable constants are involved such as clock frequency etc. But in this case it is not directly mutable.
Compiled size of constants:
15-bit literal = 2 bytes encoded as $8000..$FFFF
16-bit literal = 4 bytes with 2 bytes for code pointer to _WORD which reads the next 2 byte literal
32-bit literal = 6 bytes with 2 bytes for code pointer to _LONG which read the next 2 bytes as upper 16-bits and then the lower 16-bits
64-bit literal = not supported yet but is simply two lots of 32-bit literal, or 12 bytes + EXIT
All references to constants only take 2 bytes.
With the advent of the P2, it appears to me that Tachyon will be the main player. You have devoted a lot of time and effort into it.
Spin and C are the main players for P2 and Chip's built-in debug looks good too. I will look to see if I can leverage off it and allow TAQOZ to talk to PNut the same way.
However, for the humble P1, I think Tachyon is the only way to use the P1 effectively and to it's max. But for small projects Spin has always been very nimble too.
There is a trend. Even the serious players are turning to Tachyon
2. Loading EXTEND.fth is essential for anyone making a standalone program, for the word BACKUP. But, I would split out all the 'drivers' from EXTEND.fth as separate files - they're all useful, but only if you need them, otherwise they're just bloat. I was looking to make separate files of ANSI terminal support, Simple Serial, Loudspeaker noises, maybe PCF8574 style I2C, Realtime clock, Ultrasound ping, DH22 humidity/temp driver, NEC IR transmit, WS2812 driver, Garden lights, PBASIC style shifts, PWM, Servos, F32 support. That would liberate a lot of memory for a user program and/or more data.
3. I volunteer to help do a glossary when the dust has settled
In the early days of Android phones there were apps that would kill tasks to "free up memory" which resulted in having all this free memory. What was the purpose of memory in the first place you may ask?
Same too with Tachyon and there was more than enough memory to start with but if anyone was actually getting serious with code that took up more than a few k then by that stage they knew what they could do if they needed more memory. Although I don't ever recall anyone ever needing more Tachyon memory, just wanting to have more memory free
Despite the sheer compactness and speed of Tachyon, it has attracted relatively few takers with some more "comfortable" with what they already know, although that proves to be a long and frustrating path that often comes to a dead-end. I suppose that there is this mentality that pervades that "programming" is done "this way" and not "that way" and so they end up trying to do "that way" "this way" which obviously doesn't work that way.
As I always say, just apply the same amount of effort and patience to Tachyon, and reap the rewards of your hard work in a fraction of the time and have fun doing it.
Sounds sensible to me. Let's just leave it at that. For now. There is never a solution that suits all at once. Nothing wrong with that.
In the early days of Tachyon 2.x I was working my way through EXTEND to identify the modules and their dependencies. So at build time you could specify what modules you wanted.
But with the ongoing improvements in speed and compactness of Tachyon
And the FORGET and COMPACT words there was never really a need for it.
You could always comment out some sections of EXTEND at build time.
So at a next mayor revision @"Peter Jakacki" removed my module system again since it was not really needed and not used by anybody besides me anyhow ;-)
At present I have TOOLS loaded onto the end of EXTEND which can be trimmed off simply with a FORGET since it is fast and easier to load it first, then trim. RECLAIM finds all those private words and removes them from the dictionary, compacting it up as it goes. There are a lot of PASM operations that take two values and return one that are encoded for free. At V5.7 I broke up the kernel source into sections to be preprocessed so each section could be included from its own source file. This generated a conventional .spin file and I pass that to bstc to compile and load with clock options specified on the command line.
What's wrong with V5? Not much really but one of the things I want to do is overhaul the dictionary and also the console routines and make it more like TAQOZ and pass improvements on to TAQOZ as well.
Since we like to type interactively at the console input and that has a lot to do with productivity, I'm thinking of breaking up the console input into a couple of levels where in the first level you can input text and backspace etc and when it encounters a space or control character other than BS it will pass the pointer to the text to the console where text input is parsed (I think this is where a lot of people get confused thinking Forth is "interpreted"). This isolates the text parser from the input so it is more streamlined and able to handle blocking and non-blocking input. A very short console history buffer would be useful here, even one line's worth and you could also backspace past the current word which you can't at present since the previous has already been compiled (editing like this would just reset the current compiled code for that line).
None of this makes Tachyon faster or more compact though, and I'm not sure I can either, but I want to.
I'm starting to think that anything different makes a system "unfriendly" to many who are afraid to put the effort in to learn. Some differences are its strength, but some could be shaped a bit more. Even those familiar with some kind of Forth want Tachyon to be more like that kind of Forth they are used to (that isn't suitable anyway).
V5 NEON is a very optimized Tachyon as it is. Could it be better? To make it so requires more than rearranging the furniture, so think deep, think laterally, and think practically as if you were using it in a real project.
"Despite the sheer compactness and speed of Tachyon, it has attracted relatively few takers " The nature of Forth probably limits numbers that can stand writing in a language littered with mnemonics. But, like I've said before there are additional barriers to the newcomer becoming effective with tachyon which are presently higher than they could be. What can we (not just Peter) do to reduce these without making it a life's work?
1. Despite the reams written, the documentation is only average, scattered, some out of date. (Look at the docs for Spin in comparison) Hence the suggested glossary update as a start. The present one is incomplete for 5v7.
2. Small examples of each tachyon specific word in <wordname(s).fth included in the forth folder. Needs be 3 lines only but what a difference for the beginner. Suggest we all aim to write one a week else put a question on the forum - 'hey how does this work, I can't find a description?' Go and find a tachyon word or construct that isn't described properly ;-) It would take less than a year to knock em all down.
3. Tachyon is a moderately fat forth. This makes for a large word set that the newcomer may find daunting. That's another reason I suggest breaking it back out into language and add-ons. More byte size chunks which can be more easily digested before moving on to explore more
4. 5 min videos showing the application of a tachyon feature
5. The dropbox is a very confusing place full of debris from past experiments. If something turns out it doesn't work, that turns newcomers right off. Prune it back hard - people can ask for it if they really want it. The really old stuff is only useful to Peter, I suggest.
I would suspect that there is little processing speed increase to be had, since the system has been revised once and thoroughly thought over for so long. It would be dominated by the inner interpreter speed, I guess?
The tachyon specific constructs don't bother me as long as I know how to use them and I can see the advantage they bring. I couldn't care less about ansi standards. As soon as you start building your own defining words, you've gone non-standard anyway. Tachyon isn't that far off standard anyway - it's till a recognizable forth.
I find myself typing in the wrong case very often, swapping between text editor and terminal - but don't mind it.
Lovely sunny day today - me and the missis out in our Lomax three-wheeler for lunch to meet with eldest daughter - being careful of course.
Although I lack your experience in Tachyon as I am fairly late adopter I do, however, have a newbie perspective. And maybe I'll just share my view instead of suggesting or criticizing things.
From what I gather the main thing with Tachyon and all other things associated with it is that Peter created it and improved it to suit his needs along the way and was generous enough to just share his work with the community. It wasn't meant for, or it seems to me that way, general public to suit everybody. And it shows and this is perfectly fine.
The words dictionary and the docs could and should be improved but I wouldn't waste Peter's time on that one. The same goes for the examples but I just hate the generic type of examples as they almost always do not work in my particular setup. I prefer an example that shows and explains how a particular goal is achieved and indicates how to configure whatever parameters are there to configure. No Forth will ever be mass adopted. It is very efficient and flexible and useful but a niche. Let us not make a goal out of making Tachyon No.1 choice and a perfect tool because in this we will, in my view, fail miserably.
And one last thing - I've learned that the hard work and determination will not make anybody faster and smarter at the beginning but will help achieve whatever sensible goal there is to achieve with confidence and ease once all the sweat and tears dry out. One just doesn't appreciate anything that came too easy or free. Forth, any forth, is not for the lazy people and nothing can change that ever.
In my view, provided one has a friendly and more experienced person to ask, the " I need to do this xxx and I am trying this yyy solution and it doesn't work and how do I proceed ? " is the fastest approach (save a few hopeless cases
I've been migrating selected source over to SF rather than Dropbox which is more like my working repository. Should I disable shares to the Dropbox to encourage SF? I had already moved hardware out of the Tachyon folders too.
Faster is better, but while I don't expect to gain much, if I do then that is "better". From what I've been saying it looks like I'm focusing on the user interface mostly though and gaining more memory too.
As for the word set as long as the kernel words are described adequately then all other post compiled words can have that information in with the source code itself. This is where words can be categorized too and if they are using syntax highlighting then these documentation blocks will stand out more.
However, that brings back to mind the fact that I was also creating help files at one stage which sat inside a HELP folder on the target if it was equipped with an SD or a serial Flash. Some of my boards have serial Flash as well as SD.
There are so many unique features in Tachyon that are so unlike interactive Forths in general, yet Forths in general shouldn't be so traditional and standard and boring as they are. More than two stacks makes a lot of sense, as does compiling text input then executing rather than interpreting etc.
@Maciek
The best questions are questions that arise from trying to figure stuff out, so that you not only know and understand enough to know what to ask for, but that you are then able to understand the answer.
Little boy asks why the sky is blue but any right answer he gets he is not able to understand, so to him it's a bad answer. So you just tell him the dumb answer that "they used up all the green on the grass and the red on the sunsets, and blue was all that was left" or something like that.
So when someone asks why isn't Tachyon like abc Forth or C, or how do I do this (when they haven't even given it a go yet) then I am inclinded to tell them the dumb answer, although I don't, but maybe I should.
Moral of this piece is: Ask yourself lots of dumb questions and discover the answers for yourself because you will really grok them then, but if it still eludes you, then ask someone and then you and they will understand what you are asking and so there won't be any dumb answers
Excuse my ramblings, I will try to be more focused when I get to sit down and play with the code again. These new pcbs are keeping me busy and making me crazy!
I see no problem in a single location for Tachyon code - and it'd be less work for you. Also, more descriptive names would help first timers e.g. 'Tachyon 5v7 115200 baud 80MHz.binary'. I downloaded a few before I got one that ran on my board. I can compile the source with BST now, so am over that 'hump'. Anyway, it's good to see that Tachyon is still an active project and with some useful goals in mind. I'm quite happy to write supporting documentation along the way - and the colder months lie ahead so will be more indoors.
At a later stage again I may get some time to try out a P1V on some of the newer Lattice FPGAs in small packages and in this way we can have a very fast P1 with more memory, just as we have always wanted. However, if P2 is eventually made available in smaller packages then that is a good option too.
A perfect minimal P1 board/module. Perfect. I need (much more than) one.
At the moment, if the user needs more workspace for data than is available, he does the following, probably in order of 'easyness':-
1. Assess whether SD card is needed, if not don't include Easyfile
2. Culls what isn't required from EXTEND - but that can lead to broken words that depend on what you've just deleted, so several goes until it compiles without error
3. Fires up BSTC and sees what can be chucked out of the kernel. Iterate around until it assembles without error. Mm - but does Extend.fth still compile, so you go around that again
What about only-compile-what you-really-need, although it does rely on there being an SD card fitted (A huge underused memory at the moment):-
1. You type in a word that doesn't yet exist in the dictionary
2. Tachyon looks for a file of the same name on SD card and switches the input stream to it - this feature exists already
3. The file contains forth source code which is compiled
4. We come across an unknown word in the source code, so that word fails to compile
5. We roll back to before the failed word in hub ram and on SD card and stack our present bookmark in the SD card
6. We search the SD card for the unknown word and switch the input stream to the new file
7. This mechanism will stack as many files as is necessary to completely compile our original word with all it's dependencies automatically
8. There is also IFDEF and IFNDEF to help us
If the top word is for debug, words at the end of the file may forget the word after it ran, and other tidy up. This is acceptable to the user as long as the whole process takes less than a second (say). Words like WORDS, WWORDS, lsio, lsi2c, .TASKS , .TIMERS , .TASKS etc would all be candidates for being only temporarily in RAM.
If the top word is an application, then the same mechanism applies, only the code remains resident, no FORGET command at the end of the file.
The above has three effects:-
1. The code in hub ram is reduced in size - especially if the kernel / Extend.fth / Easyfile.fth is limited to the bare minimum required for the above
2. Fragmenting the code base allows folks to take in smaller sets of words at a time, so they're faced with a six page glossary with extensions glossaries instead of a 20 page glossary
3. The system takes care of all dependencies for the user and encourages the use of libraries of cohesive words. The kernel and Extend are a mish-mash of system and applications that might be useful at the moment, which is wasteful of ram
4. The new user who likes to quickly try out the applications like PING, humidity/thermometers, real time clock etc. can still do so, they're waiting for him on SD card as little libraries, he knows no different when he types the word, it's just a tad slower to run first time.
I think the main advantage is maximising the working area for data in a product made with Tachyon. 32k isn't much these days, so it would increase the family of applications that Tachyon could handle. it would also appear better organised?
The big question - would the response time be acceptable?
I speak from great ignorance, having never made a product with Tachyon, so i could also be talking tosh!
I've also taken an interest in umbilical forths, essential when a microcontroller is so small that a complete forth would never fit. So the majority of the Forth system is on the PC, tablet, phone and as the user writes code, only those words are transferred to the target. The target starts off with two words as a bootloader: (1) Accept an address and data byte from the terminal and store it (b) Accept an address from the terminal and jump to it. Examples are Holonforth and MPE forth. An article - pages.cs.wisc.edu/~bolo/shipyard/3ins4th.html
What you lose - the ability to interact with the product with a dumb terminal - the Forth host has to be fired up to talk to the product.
Have you ever had a go at an umbilical (tethered) forth? I used one for a while on a very small pic16f84 and it worked very well. The developer experience wasn't much different to a resident system
No, I just don't see the sense in it. If the mcu is that low on resources then it is easy enough to program it in assembler since the program will have to be very small. However, there are plenty of these really tiny and really cheap mcus that have enough to support a resident Forth. I'm talking about something for around $1 and in TSSOP20 or a small QFN and most of these are ARM based. However I am getting some RISC-V devices in from tiny chips to dual 64-bit SOCs. I'd like to develop a Forth that runs on these so in future when I need a tiny mcu I can drop one of these in. The dual 64-bit SOC devices are cheap and I'd like to see what I can do with these too.
Here's an ARM Cortex-M4 108MHz 64k Flash 8k RAM USB-OTG in QFN28 for under a $1.
Bob,
I tend to agree with Peter here.
If you really need small but very useful forth on a limited resources mcu, maybe take a look at this project https://github.com/TG9541/stm8ef.
I'm following it from day one and admire both the author and the project.
Peter,
that RISC-V option looks promising. Maybe a SF repository ?
I threw the umbilical thing in as another way of reducing code space to provide data space, but of course it would be a shed-load of work starting from scratch again, on a processor that is well along it's lifetime. I would think it does offer a benefit over hand written assembly language - the code size is likely smaller due to higher level of factoring if forth is well written. I doubt non-forthers would reuse code fragments so heavily.
I mucked about this afternoon with reducing the size of Extend.fth by taking out the little applications and the rtc code, which gains you a modest 2kbyte space.
There's no getting away from the size of the Tachyon code for even a simple SD card driver - it's a fair size compared to the rest of the Tachyon system and the 32k of ram available. We are used to adding sub-processors to Propeller systems - like bluetooth, usb and wifi modules, how if the SD card interface were offloaded to a small module, such that the tachyon file interface was much smaller? It might also be more simple for users to program? I haven't seen any such modules on the market, but also haven't looked yet. Such a module could be used on P2 as well as others. A full fat32 'filer' worked out once and reused over and over, like so many of these chinese modules.
There's maybe some modest code savings to be made in the kernel where groups of similar function words have grown over many projects. The family of 'print number' words, the <# #> set and the .as" word do similar things to one another.
The org and DS words are useful in minimising data space over traditional forth. However, they don't seem useful when an array of records is needed - I may be mistaken though.
Across the board, if you compare Tachyon with say Forth-79, Tachyon has more basic functions, swelling the number of words to learn. I wonder what an analysis of the number of times each word is used in Extend.fth and Easyfile.fth would reveal. If words haven't been used, then removing them to a library file e.g. 'Extra logic functions' etc would slim the system down a bit. A modest gain?
The SPI instructions and the like could be split out into small libraries ofsource code, so that a project that doesn't use SPI hasn't that code in hub ram. It would need trying to be sure of how many bytes it might save - it might only be another modest gain.
One has to consider the present users of Tachyon, in that any change can impact on prior code. I've seen various comments on that in the past. Has that affected the numbers of Tachyon users out there, I wonder?
Given the potential offered by P2 and Taqoz where 512kb ram changes everything, it's quite possible that Tachyon development is best limited to just a tidy-up, nothing more radical. Ensuring all the functions and as many examples as possible are current and working may be appreciated by the (mostly silent) tachyon community e.g. SAVETEXT works for me sometimes but not always. We can add information on how to use these more obscure words into the glossary, so that users (like me) don't accidentally missuse them. I got the HSUART demo from v4 going again the other day, but there are other useful demos that have not been carried forward to v5. Given Peter's commitments with P2D2, we shouldn't expect him to do much on tachyon for some time, either.