A Lua Luau
RossH
Posts: 5,462
Aloha!
Catalina's support for Lua has been significantly enhanced in release 8.1, but as well as announcing it in the Catalina thread, I thought I'd start a separate thread with a summary of the current Lua "state of play" ... well, ok, really I just wanted to use A Lua Luau as the title of a post 
Lua is a remarkable language, and the more I learn about it, the more I think one of Catalina's primary ongoing roles is as a "Lua enabler" for the Propeller.
Lua would be ideal as a teaching language - it is clean, simple, logical, orthogonal, immensely expressive, and supports multiple programming paradigms. It offers support for procedural, object-oriented, functional and data-driven programming, all "out of the box" - to which Catalina (naturally) adds multi-processing.
Lua is also an ideal language for self-hosted Propeller development.
Catalina's Lua support started because of a need for a simple scripting language to add automated testing capabilities. That was pretty much all Lua could offer the Propeller 1, because the Propeller 1 does not have enough Hub RAM to make it practical to use Lua for anything other than very small programs. On the Propeller 1, Lua is little more than a novelty.
But on the Propeller 2, Lua is a practical reality. In fact, on the Propeller 2 Catalina, Catalyst, and Lua are now irrevocably intertwined - Lua is partly implemented using Catalina and Catalyst, Catalyst is partly implemented using Catalina and Lua, and Catalina itself is partly implemented using Catalyst and Lua.
To put it another way - if the Worm Ouroboros were to have an illicit affair with the Gordian Knot, then Catalina, Catalyst and Lua could be their love children.
As well as allowing the embedding of Lua in C programs and vice-versa, the Catalina/Catalyst/Catapult L'universe offers the usual stand-alone Lua components:
- lua - an interactive version of Lua
- luax - an execution engine for compiled Lua
- luac - the Lua compiler
Those are described in the standard Lua documentation.
Catalina release 5.0 added multi-processing support to Lua, allowing Lua programs to execute in multiple threads and on multiple cogs:
- mlua - an interactive version of Lua with multi-processing
- mluax - an execution engine for compiled Lua with multi-processing
Those are described in the document 'Lua on the Propeller 2 with Catalina'
Catalina 8.0 added:
- ilua - an enhanced Read-Eval-Print Loop (can be used in place of lua, to significantly enhance self-hosted Lua development)
This is described in the README.TXT and README.md in the demos/catalyst/ilua folder.
Catalina 8.1 adds client/server support to Lua. This allows Lua programs to be easily distributed across multiple cogs without the need for using complex multi-processing primitives (although of course these remain available):
- elua - an interactive version of Lua supporting client/server programming
- eluax - an execution engine for compiled Lua supporting client/server programming
These are described in the document Aloa from Lua, which is also attached to this post.
But the L'universe doesn't end there - eLua client/server programs can also be run on multiple propellers connected via a simple 2 pin serial connection. Several complete demos using two Propellers are provided. Up to nine propellers can easily be 'clustered' in a simple star configuration, and more complex configurations are also possible. A fast parallel connection option is under development to replace the serial connection, but is not yet ready for release.
In summary, eLua significantly enhances Lua's capabilities on the Propeller 2. An eLua program can execute some parts of the program at 'native' propeller speeds while other parts of the program execute from XMM RAM and have access to megabytes of XMM RAM. If that's not enough, parts of the program can also execute on up to 8 other Propellers with no additional programming effort.
I'm not aware of anyone actually developing a multi-propeller board for the Propeller 2 yet - mine is still on the drawing board, and likely to remain that way for some time - but these were popular 'back in the day' on the Propeller 1, and I am fairly confident someone soon will. And when they do, Catalina will be ready for it.
Who would have thought that a vintage propeller-powered aircraft could end up taking you not just to the moon, but to the stars as well?
Ross.
UPDATE: Catalina 8.1.2 has improved eLua/ALOHA support, and the** Aloha from Lua** document attached to this post has been updated accordingly.
Comments
That cool @RossH! I’ve been wanting to learn more a bout Lua. Thanks for the incentive!
A bit of a breakthrough today with Lua garbage collection. The Lua documentation is a little sparse on this issue. It also seems to be incorrect in places. However, I've finally figured out how to configure the garbage collector to better suit the Propeller.
Languages that rely on dynamic memory allocation and garbage collection tend to be an issue in embedded environments where memory is tight (as Hub RAM tends to be on the Propeller). But the full range of multi-processing capabilities are only available to programs (or parts of the program) that execute from Hub RAM.
So I'm going to update both Lua and eLua with settings that are more aggressive than the defaults, but which should ensure the dynamic memory allocation stays within reasonable bounds without the garbage collector slowing the program to a crawl.
Once a program is working, the settings then can be relaxed until the program starts to run out of memory. This will be a whole lot better than the way things have to be done at present, which is that the program may not run at all if it does not specify sufficiently aggressive garbage collection.
I'll post again when the updated release is ready.
Ross.
Catalina release 8.1.1 is out, which includes better garbage collection for all the Lua applications.
See the main Catalina thread (here) for details.
Boy, did I miss a trick! I have been using XMM RAM (i.e. PSRAM) to execute Lua programs for so long that that I missed a really obvious benefit of using ALOHA with eLua - which is that in many cases Lua won't need to use XMM RAM at all.
It dawned on me last night that since the combination of eLua and ALOHA allows a Lua program to be be distributed across multiple Propellers, both the Client and the Server should be able to execute from fast Hub RAM. Lua can't do that on a single Propeller because there simply isn't enough Hub RAM, so one of the two (in eLua it is always the Server) normally executes from XMM RAM.
But if both the Client and Server can execute from Hub RAM, it should speed things up considerably. And so it does.
The demo I used is Conway's 'Game of Life' described in Appendix B of the ALOHA from Lua document (see the first post in this thread) because it is very easy to convert the original Lua program to a Client/Server program. The times are per iteration of the Life universe:
Original Life (i.e. not using either eLua or ALOHA):
That's not so good ... either too small or slow to be much use.
Client/Server Life (using eLua but not ALOHA):
That's a little better, but still painfully slow.
Client/Server Life (using both eLua and ALOHA):
Now that's much better!
It's that last possibility I had missed - by using both eLua and ALOHA, Life can be executed on two Propellers with both the Client and the Server executed from Hub RAM. No changes are required to the Life program, and the changes required to eLua are very simple. These changes have now been released in Catalina 8.1.2
Ross.
** i.e. NMM Client and XMM Server.
Catalina 8.1.2 has now been released (see here). This release includes the changes to eLua and ALOHA mentioned in the previous post. The Aloha from Lua document in the first post in this thread has been updated accordingly.
Ross.
Just as a sanity check, I recompiled Life using Catalina 8.1.2 with a universe of 80x30 cells. Here are the results (seconds per iteration):
Original Life (i.e. not using either eLua or ALOHA):
=> useless - too large to execute or too slow to use
Client/Server Life (using eLua but not ALOHA):
=> better - but still painfully slow
Client/Server Life (using both eLua and ALOHA):
=> ok!