Tachyon & C++?
DavidZemon
Posts: 2,973
One of the best parts of Tachyon (IMO) is the fact that it is interactive. I'd like to have something interactive in the C/C++ world too, but the language isn't built for it.
So... Tachyon... how big is the interpreter? It fits in a single cog right? If it's less than 64 instructions, it could be theoretically implemented in fcache... though it probably relies on some extra space for in-cog memory which fcache would not provide. So fcache probably isn't an option - but surely it could run in a dedicated cog, as a "driver" of sorts.
Does Tachyon compile into byte codes and then execute those bytecodes, or does the interpreter execute the source code directly... or is it capable of both like a Python interpreter?
I wonder what it would take to invoke a C/C++ function from Tachyon...
With that said, I really don't care if it's Tachyon or something else. But I'd love to have an interactive, Turing-complete programming language on a Propeller which is running an application with PropWare components, and Tachyon seems like a good option. Basic is not interactive right?
So... Tachyon... how big is the interpreter? It fits in a single cog right? If it's less than 64 instructions, it could be theoretically implemented in fcache... though it probably relies on some extra space for in-cog memory which fcache would not provide. So fcache probably isn't an option - but surely it could run in a dedicated cog, as a "driver" of sorts.
Does Tachyon compile into byte codes and then execute those bytecodes, or does the interpreter execute the source code directly... or is it capable of both like a Python interpreter?
I wonder what it would take to invoke a C/C++ function from Tachyon...
With that said, I really don't care if it's Tachyon or something else. But I'd love to have an interactive, Turing-complete programming language on a Propeller which is running an application with PropWare components, and Tachyon seems like a good option. Basic is not interactive right?
Comments
https://github.com/dbetz/ebasic
Of course, if you're willing to add a SPI flash chip then everything changes and much more is possible.
However, Tachyon will be much faster if you can find a way to merge it with your library.
It may be possible to preprocess the source to make it easier to compile but there is still a lot of lexing and parsing to do. In the future when P2 becomes a reality (?) then it might be possible to do this
I seem to recall a figure of 10k of hub memory at the time but that must be before I add a lot of other networking stuff to it. I basically dedicate about 24k of the upper EEPROM and store them in blocks and hashing the word to decide which block it will be saved in. When it wants to find a word it hashes it to get the index into the block which is effectively cached with any newer words in hub ram. I don't know if I have explained it all that well but this method is very flexible and almost as fast as searching the dictionary in hub memory especially considering that EEPROM is so very slow.
Each word in the dictionary is stored thus:
Count[1]
Name[?]
Attribute[1]
bytecodes[2]
There is no link field as all the headers are bunched up together in their own area. The attribute byte has the 8th bit set which is also a valid terminator for the string so it doesn't really need the count but this does make searching a bit faster.
Hm... That is quite large. But not unreasonable if in upper eeprom. How big is the cache in hub? Is it variable?
Yes, I want to call C/C++ from Tachyon. At this point, that still seems very hard. I'm wondering though if compiling with debug symbols would help though. I don't actually know what that does at the binary level, but I'm thinking maybe then I'd be able to look for the function name rather than having to code up a big tree with a lookup for every public function in the app. I'm sure having to search hub for debug symbols would be slow, but at least it would be a lot more flexible
In this scheme the EEPROM is allocated in 64 blocks of 384 bytes so there is always a buffer for reading in a single block much like reading a sector from an SD card. I then have a cache of the same size just before it in memory and just before all this is the dictionary in hub memory for any new additions afterwards. The hub dictionary falls through search-wise into the cache and the cache is kept fresh as well. Here is a terminal grab just after I compact the 8k of dictionary into EEPROM and I then define a new dummy word which stays in hub memory. The dump shows the new word, the contents of the cache (inc some other stuff I did), and also the EEPROM buffer. So the dictionary becomes a hybrid with what has been compacted plus what has been added which goes in hub memory. Of course that can be compacted again and the main thing is that runtime code does not have to deal with slow EEPROM as the hub memory has been freed up for code and buffers.
What does an applicatiotn with Propware components buy you over components in the native interactive language? Speed? Flexibility? Reusability? Portability?
Tachyon is pretty darn quick. It's quite flexible (it rather amazing the features that Peter has built into it and consolidated under one roof compared to the rest of the Propeller languages and libraries). The BASICs are slower but certainly usable, especially in teaching. The Spin/PASM gang repeatedly states tehy have no issues with the compile/load/test cycle of that language choice.
So what is being able to run an application with Propware components in an interactive environment going to do for someone?
Are you looking at the abilitiy to load a group of C++ objects and interact with them from a command line like you can import a python module and interact with it from the REPL? To me this is one of the truly amazing and fun things of working with Python, Javascript and other similar languages (yes, even Forth). I can see where you would want that with C++ but it's not even a mainstream reality on BIG computers, so putting it on a Propeller seems quite the challenge.
I'm not being contrary, I'm truly curious what the requirements are to see how things fit together and better understand what the answer might be. I've done very little work with C and even less than that with C++ but I'm a huge fan of interactive languages so I may be missing some connection opportunity here.
As you surmised, I want to be able to interact with my C++ objects (preferably both public and private methods, but at least public). That is the one and only requirement, but then I have a few other goals as well. Not reinventing the wheel is always one of my goals, which is why I'm looking at PropWare (and Basic?) rather than writing my own interpreter.
In fact, you bringing up the idea of Python makes me think of the "import" statement. It'd be great if I could load standard library archives (.a files, with debug symbols) onto an SD card and then execute some command like "import PropWare::GPIO" to load that module into HUB RAM.
I've been thinking the same, but wanted to understand the whole picture before I start trimming down.
@Peter,
So the majority of memory consists of a dictionary for Forth words. Like David suggested, a subset of these will be needed to interact with functions written in another language. So now we have a small dictionary permanently located in HUB probably. Great. Then we have the Tachyon interpreter running in a cog by itself, interpreting the user's commands which are typed in at the terminal. This means Tachyon needs to either have all the words necessary to read and write to the terminal, or it needs access to TX/RX buffers that are handled by another cog (running Tachyon or something else - doesn't matter). Which of those two methods does Tachyon currently do, and is it capable of doing both? Is there another option I didn't consider?
Glad Basic is interactive. I've never actually worked with it so I wasn't sure. You mention a compile step... does that mean the PASM VM also contains the necessary logic to compile ebasic source into bytecodes and then execute them?
Essentially, same questions to you as I've posed to Peter. How much memory is required and where is that memory located, and how does it interact with the terminal?
no the dictionary can be moved to a great extend to upper EEPROM,
but the actual bytecode, that makes up the words resides in HUB,
as well as buffers and 'variables / registers' for special Tachyon use.
yes - you could strip down Tachyon you could, but to make space for your C++ code still better dictionary in EEPROM and stripped down Tachyon bytecode in HUB.
yes - plus Tachyon uses some space in the HUB for bytecodes and 'variables/registers/buffers/ Yes third option ;-)
at present Tachyon directly writes from the Tachyon COG to the terminal without special TX buffer handling
and a separate Serial Receive COG handles the input to the RX buffer, from which the Tachyon COG reads.
Now you need a COG to run the C++ (LMM?) interpreter and a way to interact with it ...
And when you want to have it interactive, you sure need the C++ symbol table so Tachyon can use introspection to deal with it. Quite a stretch ...
And what do you gain ? A C++ debugger ?
Of course if it's COG code, that's quite easy isn't it?
Okay, that's a bit what I thought. So in its current form, Tachyon needs enough Forth words in its dictionary to transmit via UART and read from a buffer in HUB.
To limit the amount of Forth code, the RX buffer could be fed by a C++ object. Perhaps Tachyon could even be tweaked a bit to transmit via a TX buffer (handled by another C++ object) which would really limit the number of Forth words needed... but that's a small optimization for much later down the road.
A C++ debugger, for sure. Some fun as well. A hardware debugger that doesn't require intimate knowledge of Tachyon's HAL (though basic knowledge the Forth language will certainly be required).
We have GDB available for the Prop... but raw GDB is fugly. And though its great at inspecting variable states and such, it's not so great at being an interactive shell.
This is already a running system, but without the comfort we have ;-) this is just a PASM COG doing the job, no Forth involved this will not gain you a lot probably it is even more effort that the straight bitbanging done now.
A C++ debugger, for sure. Some fun as well. A hardware debugger that doesn't require intimate knowledge of Tachyon's HAL (though basic knowledge the Forth language will certainly be required).
We have GDB available for the Prop... but raw GDB is fugly. And though its great at inspecting variable states and such, it's not so great at being an interactive shell.[/quote]
for a HW debugger just load up Peter's EXPLORER.bin image and you have a huge repertoire of functionality to debug your HW.
That is what it is made for.
Can't imagine C++ can make it any better - just WAY more compliceted.
Just learn a little FORTH / TACHYON - even I managed it ;-)
it is really worth it.
so enough of evangelising - back to Tachyon fun
Markus
What this gains is more code C++ code re-use. I want as much of the system to be C/C++ as possible - and naturally, that means PropWare objects. Since I already have UART capabilities in PropWare, I'd rather see Tachyon communicate via two buffers that are handled by PropWare's UART classes.
Peter's EXPLORER.bin is great if you 1) want to learn about the functionality provided by Tachyon and/or 2) want to learn about the Propeller architecture and other bits of hardware that are connected to it. It won't do squat for helping you learn about PropWare's libraries though. PropWare also has a vast suite of libraries available to the user - though they are likely more complex than Tachyon's, what with inheritance and polymorphism and all that. It'd be great if someone could practice creating a Printer class and then test out the different methods available without having to wait for the compile -> load -> run process.
fully dynamic interactive embedded with almost full introspection - great thing if you work in Java.
Don't know if such a thing exists for C++
and if then it is propably WAY to big for the Propeller to be useful.
You played with / tried Tachyon did you?
First off, is there any type of REPL type environment for the c++ programmer? (I'm happily c++ ignorant, so please keep that in mind as you read further)
In the Java world, which is as fully mired in objectification as c++, there is groovy and the Java-esque Clojure and Scala which give you Java with an interactive interpretive flair. Does anything like these exist in the c++ world? That at least would give you an idea of what interpreted c++ is like and what may be given up in implementing the same idea on Propeller.
Then there is the question of what does it actually take to instantiate a c++ object both from source entered in the interpreter environment and from including/importing a precompiled object from a library.
Then there is the question of how you have the Tachyon environment for its minimal interpreter coexist with the runtime environment of a lmm c++ runtime environment. I assumed lmm because I wasn't sure if a complete, self sufficient c++ object could exist in a cog. Just the Mashup of these two worlds should be interesting.
When you consider all the above and compare it to the Propeller based BASIC interpreters that exist and the imitations they have in implementing a simple non-modern BASIC interpreter*, you have to wonder how satisfying and useful your c++ interpreter will be in the end.
*I don't want to take anything away from the Propeller hosted BASICs that have been given to the Prop community. They are great implementations and provide useful implementations of BASIC. ColorBASIC, from Mike Green's FemtoBASIC, gives a very rewarding retro-BASIC experience on one of OBC's PMC platforms. It is about as much BASIC as you can stuff in 32k and still leaves the programmer with 4k of program space.
Courtesy the smart folks at CERN. https://root.cern.ch/cling
Never going to be small enough to run on a Prop.
C++ is not "fully mired in objectification". It's there if you want it. Unlike Java it's not compulsory.
Basically all the non-static functions of your C application were globally available. There is no memory protection in VxWorks.
I wouldn't consider Closure and Scala quite the same as Java. They do run on the JVM, so any library you've written in Java should be invokable via Closure or Scala... but I don't know exactly how, or whether or not some kind of interface needs to be written to make that call. On that note though, I was considering what features would be possible this morning. I don't think it would be worth attempting to write your own C++ function in the interpreter. That would require something entirely more complex. What I think might be possible is invoking a function from Forth, and the return value of that function always gets placed onto the top of Tachyon's stack. This should work reasonably well I think. Anyone who has trouble grasping pointers is not going to have fun with this at all lol... but it would at least work. And then you could invoke C++ functions and see the output. Any variable declaration would have to be done via the "new" function though, but that's okay.
Loops and if-statements would not be runnable in C/C++ syntax. A basic for-loop with an index could be written easily enough in Forth syntax, with Forth running the loop and the body of the loop being a C++ function call. If-statements probably wouldn't be too feasible, since any useful if-statement is going to depend on the value of a C++ variable and I don't think I'd want to implement anything in Forth capable of reading the C++ variables.
Your Forth engine is your REPL, the command line interface, and somehow it is built together with the C functions of your application. Now you can invoke C functions from Forth in the REPL or from a Forth program.
You are not about to be compiling or interpreting C or C++ on the Prop itself though.
Unless....so far there is one C compiler that actually runs and compiles stuff on on the Propeller itself, BDSC. For that you need to be running CP/M under my ZiCog Z80 emulator or PullMolls qZ80 emulator.
struct Invocation { char *functionName; unsigned int *parameters; }If I understand GCC/C++ correctly, every parameter on the Propeller (excluding 64-bit values) eventually comes down to a 32-bit (or less) value right? For bool/char/int, it's just a register. For classes and structs, its a pointer. Even pass-by-value is really a pointer getting passed into the parameter, and then some magic code inserted at the start of the function copies the values from that class instance to a new instance on the stack right?
So if I'm understanding this right, I need a few things:
1) A Tachyon kernel, and the basic Forth code to read from/write to the terminal
2) Forth code capable of taking user input and dropping it onto the above-mentioned FIFO buffer
3) C++ code capable of reading from the FIFO buffer, searching the symbol table by the function name and then invoking that function
4) Some way to pass the return value back to Tachyon... possibly another FIFO? Maybe the same FIFO with a third value for the return?
5) Infinite HUB RAM
mayb offline preprocess your symbol table so it can be loaded into the Tachyon dictionary, so you wont need to store the strings in the C++ code.
And then just put the invokation parameters on the stack like
par1 par2 MyFunction <CR> ok
and the results are put on the stack
so you can view and do as you like with them
your MyFunction just has to 'know' how many parameters to take, and where to put them, then put the function pointer and your C++ code can do it's work
given this sounds not too complicated on the Tachyon side.
you could even use Tachyon's SD/Filesystem to page in code from SD to feed your C++-Cog
If you have your C++/LMM/CMM COG image it could even be loaded with Tachyon from SD and started in a COG
Tachyon will be the master.
There is a multi tasking mechanism in Tachyon already to sync multiple COGs