Catalina - ANSI C and Lua for the Propeller 1 & 2
I guess I should start a new thread for Catalina, instead of continuing the separate ones for the P1 and P2. I will add new updates here.
Catalina is an open source ANSI C compiler. It support the Propeller 1 and Propeller 2.
Catalina also supports the Lua scripting language (currently version 5.4.4), both embedded in C programs, and in stand-alone mode.
Catalina is available on GitHub and on SourceForge . GitHub is recommended if you are using Linux or want the "bleeding edge", but SourceForge is recommended if you want the most recent stable release and/or a simple Windows installer that includes pre-compiled binaries.
The current version is 8.1.2 on both Windows and Linux.
EDIT: Updated the version number, to mention GitHub, and also to mention Lua.
Comments
I am releasing Catalina 4.9.5. The main thing this release adds is enhanced Lua support and a couple of bug fixes. See the README extract below.
Lua is lots of fun, and well suited to the Propeller (especially the Propeller 2). For those unfamiliar with it, here is a description of Lua from lua.org:
In addition to adding Lua 5.4.3 (the latest version) I have upgraded both Catalina and Catalyst to use Lua 5.1.5, because this version adds the ability to not only run an interactive Lua interpreter and a Lua compiler, but also to execute pre-compiled Lua programs using less memory, which enables the execution of larger Lua programs on the Propeller.
Here is the relevant section of the README.TXT file.
RELEASE 4.9.5 New Functionality ----------------- 1. Updated Lua to version 5.1.5 in both payload (where it is embedded for scripting) and Catalyst (where it is used as a stand-alone demo program). Version 5.1.5 is the last release of Lua 5.1. On a Propeller 1 it must be compiled in LARGE mode. On a Propeller 2 it can be compiled in either NATIVE or COMPACT mode. Catalyst will continue to incorporate Lua 5.1 because it is small enough to be useful on a micro controller like the Propeller even when compiled in NATIVE mode (for speed). Also, Lua 5.1.5 incorporates a new option to execute compiled Lua programs using a Lua execution engine that does not load the Lua parser (luax). This saves even more Hub RAM when all the Lua code to be executed is pre-compiled using the Lua compiler (luac). For example, to execute the "life.lua" demo program, you can now either do: lua life.lua or luac -o life.out life.lua luax life.out Lua version 5.4.3 is now also included as a Catalyst demo, but is not compiled by default. Version 5.4.3 is the current release of Lua 5.4. On the Propeller 2 it should be compiled with the COMPACT option, or it may be too large to be much use. For example: build_all P2_EVAL SIMPLE VT100 COMPACT On the Propeller 2, the code sizes of Lua 5.1 (without the Lua parser), Lua 5.1 (with the Lua parser) and Lua 5.4 are: COMPACT NATIVE Lua 5.1 (no parser) 138k 250k Lua 5.1 (with parser) 166k 297k Lua 5.4 (with parser) 227k 412k Hence Lua 5.1 is quite usable in both NATIVE and COMPACT modes, and with or without loading the parser, whereas Lua 5.4 is really only usable in COMPACT mode. This is because Lua 5.4 itself takes so much Hub RAM that there is not much left to run anything except small Lua programs (the above sizes are the code sizes only - there is also stack and heap space required to actually execute a Lua program). Note that there is currently no option to execute Lua 5.4 programs without loading the Lua parser. This option may be added in a later Catalina release. Note also that on memory constrained systems such as a Propeller, the Lua garbage collector sometimes requires a "tweak" to ensure it runs often enough to stop Lua programs running out of memory. The "life.lua" demo program is an example - it does many string concatenations per generation, and if the garbage collection does not run often enough, the program can run out of memory. To prevent this, the following line has been added to the program: collectgarbage("setstepmul", 500) Refer to the Lua documentation for more details. 2. Updated MAX_SYMBOLS in p2asm to be 20,000 (was 10,000) - this is required to allow Lua 5.4.3 to be compiled with optimization enabled. 3. Added decoding of Catalina symbols MHZ_300 and OPTIMIZE. This allows these options to be specified on the command line of the build_all scripts, or via CATALINA_DEFINE: MHZ_300 - set the default Propeller 2 clock to 300Mhz. Note that this may not work on all Propeller 2 platforms. USE THIS OPTION WITH CAUTION, BECAUSE IT MAY NOT BE IMMEDIATELY APPARENT THAT A PROGRAM IS NOT EXECUTING CORRECTLY! OPTIMIZE - set the optimizer level to 5. For example, you can now say: build_all P2_EVAL MHZ_300 OPTIMIZE or set CATALINA_DEFINE=P2_EVAL MHZ_300 OPTIMIZE build_all These symbols are not intended to be used on the command line, but they can be used via the -C option. For example: catalina hello_world.c -p2 -lci -C MHZ_300 -C OPTIMIZE would be the same as: catalina hello_world.c -p2 -lci -f300Mhz -O5 Other Changes ------------- 1. Reformatted some Catalyst text files (e.g. Catalyst.txt) to be Unix format instead of DOS. This makes them easier to view in the Catalyst text editor (vi). 2. Fixed a bug in the Catalina Optimizer when optimizing COMPACT programs. This may have resulted in programs that would not execute correctly. Affected only Propeller 2 COMPACT programs. 3. Fixed a bug when compiling Catalyst - it would not compile for the Propeller 2 if the file sio_array.h was not present in the core folder, even though that file is only required when compiling for the Propeller 1.Both Windows and Linux versions are now available.
Ross.
I have had a question about integrating C with Lua. This is very easy to do, and there is a minimalist example (min.c) that is included with Lua-5.1.5 in Catalina 4.9.5. However, this example is not so easy to execute using the payload terminal emulator because when you run the program, payload terminates before it prints the output the program produces.
The min.c program reads stdin and expects the input to be terminated with an EOT (i.e. Ctrl-D) before it produces output to stdout and/or stderr. But Ctrl-D is also the signal to payload to terminate, so you never see the output. To fix this, I have added the ability for payload to send an EOT to the application without terminating. Just press Ctrl-D once to send EOT. To terminate payload you now have to enter TWO CONSECUTIVE Ctrl-D characters.
Copy the new payload executable over the existing payload.exe in your Catalina\bin directory. The amended source (payload.c) is also included, and will also be included in the next release of Catalina.
The program min.c is in the Lua-5.1.5\etc folder, but the easiest way to compile it is to copy it to the Lua src folder, then build the Lua library and then compile the min program. For example, you could use commands like:
To load and run the program, use the new payload in interactive mode:
payload -i -b230400 min.binThen in the payload window, type the following:
a=1.1 b=2.2 print("a+b=",a+b)Make sure to press ENTER after the final line. Then press Ctrl-D. You should see output like:
a+b= 3.300000191Press Ctrl-D again to exit payload. The min.c program interprets any Lua statements entered via stdin, but it loads no Lua libraries - the "print" function you just executed is not the normal Lua print function - it is the C function in min.c. This minimalist example shows how easy it is to call Lua from C, and also C from Lua!
Here is the entire min.c program:
/* * min.c -- a minimal Lua interpreter * loads stdin only with minimal error handling. * no interaction, and no standard library, only a "print" function. */ #include <stdio.h> #include "lua.h" #include "lauxlib.h" static int print(lua_State *L) { int n=lua_gettop(L); int i; for (i=1; i<=n; i++) { if (i>1) printf("\t"); if (lua_isstring(L,i)) printf("%s",lua_tostring(L,i)); else if (lua_isnil(L,i)) printf("%s","nil"); else if (lua_isboolean(L,i)) printf("%s",lua_toboolean(L,i) ? "true" : "false"); else printf("%s:%p",luaL_typename(L,i),lua_topointer(L,i)); } printf("\n"); return 0; } int main(void) { lua_State *L=lua_open(); lua_register(L,"print",print); if (luaL_dofile(L,NULL)!=0) fprintf(stderr,"%s\n",lua_tostring(L,-1)); lua_close(L); return 0; }You can perhaps see why I find Lua so interesting!
I have just issued a small patch for Catalina - release 4.9.6. It is available on SourceForge. Please note that this is NOT a full release. It must be installed over an existing installation of Catalina 4.9.5. It includes all new sources and Windows binaries, but for Linux you will need to recompile Catalina from source after you install the patch.
The main purpose of release 4.9.6 is to tidy up a few outstanding issues before the next full release of Catalina, which may be a while away because it will introduce some significant changes. For example, the next full release will include a thread-safe version of malloc, and multithreading support for Lua.
Here is an extract from the README.TXT:
RELEASE 4.9.6 New Functionality ----------------- 1. Updated the payload loader to allow EOT (Ctrl-D) to be passed to the application program in interactive mode without also closing the terminal emulator. Now pressing Ctrl-D ONCE passes an EOT to the application but does not terminate payload. To terminate payload you must now enter TWO SUCCESSIVE Ctrl-D characters. 2. Updated the minimalist Lua demo program (min.c) so that it would compile under both Lua 5.1.5 and Lua 5.4.3 (which changed the names of some Lua library functions). 3. Updated the Lua 5.1.5 and 5.4.3 Makefiles to include pattern templates. This simplifies the compilation of Lua programs like min.c - you can now use just the Catalina Makefile to build Lua programs. For example, to build the minimalist demo: make -f Makefile.Catalina clean min.bin 4. Added the Lua execution engine (luax) that executes only compiled Lua programs to Lua-5.4.3. This saves Hub RAM by not loading the Lua parser if all the Lua programs to be executed are pre-compiled. 5. Added "clean_all" scripts to the Catalyst demo folders, to make it easier to remove compiled versions of all the Catalyst programs. 6. Added "clean_all" scripts to the Catalina source folders, to make it easier to remove compiled versions of all the Catalina programs. 7. Removed the -C NATIVE from the Catalyst build_all and Makefiles, and some of the -O5 options on individual demo programs. This means that you can now specify OPTIMIZE on the command line if you want Catalyst programs to be optimized, and on the Propeller 2 you can now choose to compile all the demo programs as COMPACT (instead of NATIVE). Note that this applies to the Propeller 2 only. Do not specify a memory model on the Propeller 1 - the core programs must always compiled as COMPACT on the Propeller 1 because otherwise they would be too large for a Propeller 1 without XMM RAM, and the other programs must usually be compiled in LARGE mode. Other Changes ------------- 1. Fixed some errors in Dumbo Basic that stopped it compiling in COMPACT mode. Affected the Propeller 2 only. 2. Release 4.9.5 fixed one bug in the COMPACT mode optimizer, but introduced another, which has been fixed in this release. Affected the Propeller 2 only (and the only program known to be affected was the vi text editor, which would not compile in COMPACT mode).Just a quick progress report. I have just finished making Catalina's memory management functions (malloc, calloc, realloc, free etc) thread-safe, which also makes them safe to execute from multiple cogs and also from multiple memory models.
As a test I thought I'd try executing multiple parallel Lua interpreters. Turns out to be quite trivial.
Here is an example program which executes four Lua interpreters, each one running on a separate cog. Currently the Lua interpreters are pretty much independent, and can only communicate via C shared functions and global variables (such as is demonstrated by the "print" function in this example):
/* * multi_cog.c -- a minimal multi-cog Lua interpreter. * * executes Lua interpreters on multiple cogs. * * no user interaction, no standard Lua library, only a "print" function. * */ /* * include Catalina definitions: */ #include <catalina_cog.h> //#include <catalina_threads.h> /* * include Lua definitions: */ #include "lua.h" #include "lauxlib.h" /* * define a stack size for the Lua interpreters (the * exact value depends on the Lua code to be executed): */ #define STACK_SIZE 1000 // longs /* * define a global flag to synchronize starting the Lua interpreters * (so we can easily see that they do execute in parallel): */ static int start; /* * define a global lock to protect memory functions: */ static int memory_lock; /* * define a global lock to protect our HMI functions * (so the output is not garbled): */ static int hmi_lock; /* * define a C function for Lua to call (just as an example): */ static int print(lua_State *L) { int n=lua_gettop(L); int i; ACQUIRE(hmi_lock); for (i=1; i<=n; i++) { if (i>1) printf("\t"); if (lua_isstring(L,i)) printf("%s",lua_tostring(L,i)); else if (lua_isnil(L,i)) printf("%s","nil"); else if (lua_isboolean(L,i)) printf("%s",lua_toboolean(L,i) ? "true" : "false"); else printf("%s:%p",luaL_typename(L,i),lua_topointer(L,i)); } printf("\n"); RELEASE(hmi_lock); return 0; } /* * define a function that can be started on a cog to * execute a lua interpreter on a string (the string * can contain any arbirary lua code): */ void lua_on_a_cog(char *code) { int error; lua_State *L; L=luaL_newstate(); lua_register(L,"print",print); while (!start) { }; // wait for start signal error = luaL_loadstring(L, code) || lua_pcall(L, 0, 0, 0); if (error) { ACQUIRE(hmi_lock); printf("%s\n", lua_tostring(L, -1)); RELEASE(hmi_lock); lua_pop(L, 1); // pop error message from the stack } lua_close(L); while(1); } /* * one cog will execute this Lua code: */ void lua_interpreter_1(void) { lua_on_a_cog("for a = 1, 10 do print(\"a= \"..a) end"); } /* * one cog will execute this Lua code: */ void lua_interpreter_2(void) { lua_on_a_cog("for b = 10, 1, -1 do print(\"b= \"..b) end"); } /* * one cog will execute this Lua code: */ void lua_interpreter_3(void) { lua_on_a_cog("for c = 20, 30 do print(\"c= \"..c) end"); } /* * one cog will execute this Lua code: */ void lua_interpreter_4(void) { lua_on_a_cog("for d = 30, 20, -1 do print(\"d= \"..d) end"); } /* * the main function just starts the cog functions: */ void main(void) { // allocate stack space for each Lua interpreter: unsigned long lua_stack_1[STACK_SIZE]; unsigned long lua_stack_2[STACK_SIZE]; unsigned long lua_stack_3[STACK_SIZE]; unsigned long lua_stack_4[STACK_SIZE]; // even though we do not use threading, multi-kernel programs // must assign a memory lock to protect the memory management // functions from multi-cog access: memory_lock = _locknew(); _memory_set_lock(memory_lock); // because multiple cogs may access the services provided by plugins, // we must assign one or more locks to protect the services during // multi-cog access. In previous versions of Catalina, this had to be // done by specifying -C PROTECT_PLUGINS during compilation, which is // still supported. In that case the following function does nothing: _set_service_lock(-1); // allocate a lock to protect HMI functions: hmi_lock = _locknew(); // start multiple cogs, each one executing a Lua interpreter // (they won't start executing their Lua code until we tell them): _coginit_C(&lua_interpreter_1, &lua_stack_1[STACK_SIZE]); _coginit_C(&lua_interpreter_2, &lua_stack_2[STACK_SIZE]); _coginit_C(&lua_interpreter_3, &lua_stack_3[STACK_SIZE]); _coginit_C(&lua_interpreter_4, &lua_stack_4[STACK_SIZE]); // start all the Lua interpreters: start = 1; // just let the cogs do their thing: while (1); }Here is the actual output produced when the above program is run (on a Propeller 2). It is pretty much what you would expect if all the Lua interpreters are in fact executing in parallel:
Of course, this program requires Catalina 5.0, which is not released yet!
The next step is to add multi-threading support to Lua (i.e. in addition to just multi-cog and multi-model support). With Catalina's multi-threading, this should be trivial, and should allow the execution of dozens of concurrent Lua interpreters, not bounded by the number of available cogs (as the example above is!).
However, I am not sure yet that this will be particularly useful. Ideally I would prefer to be able to run individual Lua functions as threads - i.e. to offer a much finer granularity of threading than just instances of the Lua interpreter.
Time will tell.
I don't suppose there is any chance for a Mac release?
Hi David
I never did Mac releases, but I used to use a virtual Mac to make sure Catalina would compile and basically run. So the Linux version of Catalina used to be able to be compiled on a Mac, but we are going back quite a few years, and I doubt very much that it would do so now.
Ross.
You're probably right that it won't compile out of the box if you haven't done anything to make that happen. I remember trying a while back and there were some problems that I didn't have time to resolve.
As expected, the multi-threaded version proved to be trivial, but worth doing to verify (before I dig deeper into making Lua multi-threaded) that once the memory management functions are made thread-safe, so is Lua.
Here is a very similar program to the multi-cog version above, but this one is not limited by the available cogs - it executes 20 Lua interpreters in separate threads:
/* * multi_thread.c -- a minimal multi-threaded Lua interpreter. * * executes Lua interpreters on multiple threads. * * no user interaction, no standard Lua library, only a "print" function. * */ /* * include Catalina definitions: */ #include <catalina_cog.h> #include <catalina_threads.h> #include <thread_utilities.h> /* * include Lua definitions: */ #include "lua.h" #include "lauxlib.h" /* * define how many Lua interpreters (i.e. threads) we want: */ #define NUM_THREADS 20 /* * define a stack size for the Lua interpreters (the * exact value depends on the Lua code to be executed): */ #define STACK_SIZE (MIN_THREAD_STACK_SIZE + 1000) // longs /* * define the number of thread locks we need (we only need one * to protect our HMI functions): */ #define NUM_LOCKS 1 /* * a pool of thread locks - note that the pool must be 5 bytes larger than * the actual number of locks required (MIN_THREAD_POOL_SIZE = 5): */ static char pool[MIN_THREAD_POOL_SIZE + NUM_LOCKS]; /* * a lock allocated from the pool - required to protect our HMI functions: */ static int hmi_lock; /* * a global flag to synchronize starting the Lua interpreters * (so we can easily see that they do execute in parallel): */ static int start; /* * a global lock to protect kernels and memory functions: */ static int kernel_lock; /* * a C function for Lua to call (just as an example): */ static int print(lua_State *L) { int n=lua_gettop(L); int i; for (i=1; i<=n; i++) { if (i>1) _thread_printf(pool, hmi_lock, "\t"); if (lua_isstring(L,i)) _thread_printf(pool, hmi_lock, "%s",lua_tostring(L,i)); else if (lua_isnil(L,i)) _thread_printf(pool, hmi_lock, "%s","nil"); else if (lua_isboolean(L,i)) _thread_printf(pool, hmi_lock, "%s",lua_toboolean(L,i) ? "true" : "false"); else _thread_printf(pool, hmi_lock, "%s:%p",luaL_typename(L,i),lua_topointer(L,i)); } _thread_printf(pool, hmi_lock, "\n"); _thread_yield(); return 0; } /* * a Lua interpreter that can be executed as a thread - it will * execute whatever Lua code passed to it in code[0]: */ int lua_thread(int me, char *code[]) { int error; lua_State *L; L=luaL_newstate(); lua_register(L,"print",print); while (!start) { }; // wait for start signal error = luaL_loadstring(L, code[0]) || lua_pcall(L, 0, 0, 0); if (error) { _thread_printf(pool, hmi_lock, "%s\n", lua_tostring(L, -1)); lua_pop(L, 1); // pop error message from the stack } lua_close(L); return error; } /* * the main function just starts the thread functions: */ void main(void) { int i; // the lua code to be executed - one string will be executed // by each Lua interpreter running as a separate thread: char *code[] = { "for a = 1, 10 do print(\"a= \"..a) end", "for b = 11, 20 do print(\"b= \"..b) end", "for c = 21, 30 do print(\"c= \"..c) end", "for d = 31, 40 do print(\"d= \"..d) end", "for e = 41, 50 do print(\"e= \"..e) end", "for f = 51, 60 do print(\"f= \"..f) end", "for g = 61, 70 do print(\"g= \"..g) end", "for h = 71, 80 do print(\"h= \"..h) end", "for i = 81, 90 do print(\"i= \"..i) end", "for j = 91, 100 do print(\"j= \"..j) end", "for k = 101, 110 do print(\"k= \"..k) end", "for l = 111, 120 do print(\"l= \"..l) end", "for m = 121, 130 do print(\"m= \"..m) end", "for n = 131, 140 do print(\"n= \"..n) end", "for o = 141, 150 do print(\"o= \"..o) end", "for p = 151, 160 do print(\"p= \"..p) end", "for q = 161, 170 do print(\"q= \"..q) end", "for r = 171, 180 do print(\"r= \"..r) end", "for s = 181, 190 do print(\"s= \"..s) end", "for t = 191, 200 do print(\"t= \"..t) end", }; // allocate stack space for each Lua interpreter: unsigned long stack[STACK_SIZE * NUM_THREADS]; // assign a lock to protect kernels // (and memory management functions): kernel_lock = _locknew(); _thread_set_lock(kernel_lock); // initialize a pool of thread locks (we need only 1 lock) _thread_init_lock_pool (pool, NUM_LOCKS, _locknew()); // assign a thread lock to avoid HMI plugin contention: hmi_lock = _thread_locknew(pool); // start multiple threads, each one executing a Lua interpreter // (they won't start executing their Lua code until we tell them): for (i = 0; i < NUM_THREADS; i++) { _thread_start(&lua_thread, &stack[STACK_SIZE*(i+1)], i, &code[i]); } // start all the Lua interpreters: start = 1; // just let the threads do their thing: while (1) { _thread_yield(); }; }The output is exactly what you would expect. Here are just the last few lines:
Now comes the hard part ...
A quick update on Catalina 5, and a note about a strange Windows 10 bug that affects Catalina's payload loader ...
First, the bug ...
The Windows 10 Command-Line interpreter (cmd.exe) crashes and burns if you execute payload in a command window with the "Wrap Text Output on Resize" option unchecked. This option is on the Layout tab of the cmd.exe Properties dialog box. Not only does cmd.exe crash, it leaves payload executing in the background, hanging onto the executable it was loading (which then cannot be updated) and it also takes out the Windows 10 USB subsystem. I have sometimes had to reboot Windows to get it all working again. Took me a while to track this one down, and I am still not sure exactly what the problem is, but the workaround is easy - just make sure this option is selected in cmd.exe if you use the payload loader.
Next, a quick update on Catalina 5 ...
All looking very promising. I now have Lua fully multithreaded. It turned out the easiest way to test out various existing Lua multi-threading extensions was to implement the Posix thread library (pthread). So now I have a pthread library implemented using Catalina's existing thread subsystem. Compared to Catalina's lean and mean threads, Posix threads are a bloated abomination designed by a committee to be impossible to use - but they are a common way of implementing platform independent multi-threading in C, so I gave in and implemented a thin binding.
Catalina's pthread.h is attached, which summarizes Catalina's level of support (which is fairly complete).
Here is a simple example of a producer/consumer problem implemented with pthreads:
#include <pthread.h> #include <stdio.h> #define NUM_CARS 10 #define NUM_TANKERS 4 pthread_mutex_t mutexFuel; pthread_cond_t condFuel; int fuel = 0; void* tanker(void* arg) { int i; for (i = 0; i < 5; i++) { pthread_mutex_lock(&mutexFuel); fuel += 30; pthread_printf("Tanker %d arrived. Station now has %d\n", (int)arg, fuel); pthread_mutex_unlock(&mutexFuel); pthread_cond_broadcast(&condFuel); } return NULL; } void* car(void* arg) { pthread_mutex_lock(&mutexFuel); while (fuel < 40) { pthread_printf("Station does not have enough fuel. Car %d waiting.\n", (int)arg); pthread_cond_wait(&condFuel, &mutexFuel); } fuel -= 40; pthread_printf("Car %d got fuel. Station now has %d\n", (int)arg, fuel); pthread_mutex_unlock(&mutexFuel); return NULL; } int main(int argc, char* argv[]) { pthread_t th[NUM_CARS + NUM_TANKERS]; int i; pthread_mutex_init(&mutexFuel, NULL); pthread_cond_init(&condFuel, NULL); for (i = 0; i < NUM_CARS + NUM_TANKERS; i++) { if (i > NUM_CARS) { if (pthread_create(&th[i], NULL, &tanker, (void *)(i-NUM_CARS)) != 0) { pthread_printf("Failed to create tanker thread"); } } else { if (pthread_create(&th[i], NULL, &car, (void *)i) != 0) { pthread_printf("Failed to create car thread"); } } } for (i = 0; i < NUM_CARS + NUM_TANKERS; i++) { if (pthread_join(th[i], NULL) != 0) { pthread_printf("Failed to join thread"); } } pthread_printf("All Done\n"); pthread_mutex_destroy(&mutexFuel); pthread_cond_destroy(&condFuel); return 0; }and here is the output when compiled and executed with Catalina 5.0:
Catalina's threading subsystem is so efficient that Catalina can support pthreads on any Propeller 1 platform, as well as on the Propeller 2. When compiled in COMPACT mode, the above program takes less than 7k of Hub RAM on a Propeller 1 or 2.
I expect to release a first version of Catalina 5 as soon as I have finished documenting all the new features.
Not 100% happy with this yet, but here is an example of using the new "threads" package in Lua:
threads.msg("ball") function ping() for i = 1,21 do threads.get("ball") threads.print("ping ") threads.put("ball") end end function pong() for i = 1,21 do threads.put("ball") threads.print("pong\n") threads.get("ball") end end threads.new(pong) threads.new(ping)That's the complete program. One thing I like about Lua is that it is even more concise than C. There is very little unnecessary or obscure syntax.
You can probably guess the output this program produces, but here it is anyway:
Clearly, there is a lot more of the underlying pthreads capabilities that could be exposed. But I'm not yet sure how much is either necessary or desirable. This is probably too simple as it stands - it only has message passing as the mechanism for interaction between threads - but on the other hand pthreads is way too complex.
I'm open to suggestions as to what would constitute a good set of minimal "primitives" for Lua.
to me this looks like JMPRET on the P1.
It does not run in parallel but just passes the 'ball' but easy on the eyes, the source.
Mike
I agree it is hard to spot the difference with such simple examples. But not impossible. Here is another complete Lua program (this one is even simpler, and correspondingly less interesting - but it does exhibit true multi-threading):
threads.new(' for i = 1,1000 do threads.print("A") end ') threads.new(' for j = 1,1000 do threads.print("B") end ')And here is the actual output:
The formal name is cooperative multi-tasking.
Both pthreads and Catalina are pre-emptive. But native Lua supports only co-routines, which are cooperative. This is one reason adding multi-threading to Lua is non-trivial.
I have done some more investigation and more thinking and have decided on the following set of primitives:
The main reasons for this are practical - (1) is already implemented in the package (luaproc) I am basing my threads package on, and (2) and (3) are logical extensions of this message passing paradigm. There is already an implicit "mutex" in receiving a message (only one receiver can get each message), and an implicit "condition" in sending a message (sending blocks the sender until there is a receiver ready).
Nice. Sorry, I hadn't been reading and just reacted to Mike's observation.
@RossH I just got the 3Color Eink program working with Catalina (P1), but have some questions, if you don't mind.
Most of the work was moving variable definitions to the top of the functions, not such a big deal.
Rest of changes documented in the attached.
There are some things I don't understand...
Is "propeller.h" automatically included? If I try to include it, I get errors, looks like it's including propeller2.h for some reason...
Does #warning work? Should it?
There are some things in flexprop C's propeller.h that I don't think are here, right? Like:
getpin(), togglepin(), setpin()
I tried using Geany, but I seem to have forgotten the trick for making it work.
Get "Process failed (The system cannot find the file specified)" when trying to make or build.
Vaguely remember something about needing a .geany file for this to work, right?
Sounds like you're working on making Geany faster. That's good.
Hello Rayman
The errors you found are perfectly legitimate, and are due to differences between ANSI C and other C variants. Here are some explanations:
Yes, ANSI C requires all declarations to precede all code in the block. This rule was relaxed in C99, but with some consequences. If you want to see why this was not such a brilliant idea, try guessing what the following program produces before running it (it compiles and runs with gcc but will not compile with ANSI C):
#include <stdio.h> void main(void) { int i = 1; goto error; int j = 2; error: printf("i = %d\n", i); printf("j = %d\n", j); }The only place this feature ever made sense was in declaring the control variable for for loops in the loop (as in "for (int i =1; i < 100; i++)". Which, to be fair, is where it mostly gets used.
Yes, only #error is supported by ANSI C. AFAIK #warning (and #info) were added by specific compiler vendors and are not part of any C standard.
Problem: #include <propeller.h> -->C:\Program Files (x86)\Catalina_4.9\include/propeller2.h:100: redeclaration of cogstop previously declared at C:\Program Files (x86)\Catalina_4.9\include/catalina_cog.h:80You should not include "propeller2.h" if compiling for the Propeller 1!
This is actually required by C. But some compilers are more relaxed about it than others.
Yes, Catalina offer many more library options than C - mostly to do with whether or not floating point and full streams support is included.
Try using the optimizer - "-O5"
The remaining errors are due to differences in the propeller-specific libraries and header files - we never standardized these on the P1 as we did on the P2.
@RossH Thanks.
I wasn't including propeller2.h... I got that error when including propeller.h. No idea why...
I find the functions: getpin(), togglepin(), setpin()
to be very useful. It was a bit of a chore to set particular pins without it.
Also, I don't seem to have direct access to, for example, DIRA. Am I missing something there?
That is odd!
I'll look at adding these to the next release.
Again, odd. These are defined in "propeller.h".
Can you post your source?
Ross.
@RossH
Code is posted already in top post of the seeed studio 3 color eink thread in this category.
I’m curious as to why including propeller.h doesn’t work…
The include was in platform.h …
Maybe has to to in main file?
https://forums.parallax.com/discussion/174323/seeed-studios-3-color-e-ink-display-176x264-pixels-with-flexprop-c-catalina-on-p2-and-p1/p1
The problem seems to be with "SPI.c", which includes "propeller2.h". If you remove that, you can include "propeller.h".
Ross.
Thanks!
No idea why I did that.
Guess that explains it.
Deleted because there is a new version of Multi-threaded Lua available. See later in this thread.
Deleted because there is a new version of Multi-threaded Lua available. See later in this thread.
Catalina 5 is nearly ready for release, but the final packaging process always takes much longer than I expect, especially as there have been many more changes and improvements in Catalina than I originally planned. While I am getting it ready, attached is the final release candidate of a Multi-threaded version of Lua, compiled for the Propeller 2 Evaluation board.
It is a self-contained Lua development environment for the Propeller 2, and now includes a comprehensive tutorial and reference manual. You do not need Catalina at all to install or run it. The sources are not included - but they will be in the release.
If anyone has any suggestions for improvement of the tutorial, I may have time to incorporate them in the final release of Catalina 5.
Ross.
EDIT: The attached file has been updated to reflect Catalina 5.0.2
Blast!
I was just about to release Catalina 5 on SourceForge when I noticed that SourceForge is now flagging some previous Catalina releases as malware affected. I have sent the same files off to various online virus scanners and also virus checked my PC with multiple anti-virus programs (including the same ones SourceForge uses!) and they are all clean.
I am fairly sure this is a false positive by SourceForge. However, I'll spend some time trying to sort this out before I upload the new version
Ross.
Ok. I seem to have SourceForge sorted, and have now uploaded a release candidate of Catalina 5.0 for Windows. A corresponding Linux release will follow when I get more time (very busy with flooding here in Australia at present!).
Also, I have updated the compiled version of multi-threaded Lua (see https://forums.parallax.com/discussion/comment/1536442/#Comment_1536442) to match the released version (which saves you having to compile it yourself).
Apart from bug fixes, this is about as far as I can go with multi-threaded Lua without making internal changes to Lua itself (which is something I have managed to avoid doing so far). But for a high-level language like Lua (it is a much higher level language than C, supporting both object-oriented and functional programming) the Propeller is quite memory constrained - so much so that code size and garbage collection can become significant issues for non-trivial programs.
Instead of frigging about trying to improve Lua's memory management, I may reconsider adding support for extended memory on the Propeller 2 - this would allow Lua to become much more useful without needing internal changes.
Here is the "New Functionality" section of the README file for Catalina 5.0:
1. The standard C memory management functions (i.e. malloc, realloc, calloc, free) and a new system break function (sbrk) are now all thread safe. For details, see the "Multi-processing, Locks and Memory Management" section in the Catalina Reference Manual. 2. Catalina's dynamic memory management has been modified to make it less prone to running out of memory when lots of small randomly sized chunks of memory are allocated, freed or re-allocated. For details, see the "Multi-processing, Locks and Memory Management" section in the Catalina Reference Manual. 3. A new function has been added that can be used to assign one or more locks to protect services offered by plugins in multi-cog or multi-thread programs. For details, see the "Multi-processing, Locks and Memory Management" section in the Catalina Reference Manual. 4. An example of a public domain itoa function has been added to the folder demos\examples - it is called ex_itoa.c. The itoa function is also used in the new demo program test_thread_malloc.c (as an alternative to using sprintf, which can make the program too large for the Propeller 1). 5. Catalina used to use 4 bits for the lock in the service registry. The Propeller 1 only had 8 locks, so this was sufficient to represent all possible locks plus a bit to indicate "no lock". But the Propeller 2 has 16 locks, so Catalina must now use 5 bits for the lock. To make space for the extra bit, the number of bits used to hold the cog-specific service code has been reduced from 8 to 7. This still allows up to 127 services, but less than 40 such services are implemented even by the most complex plugins (the HMI plugins), so this is unlikely to be a problem. If it becomes so, the size of each service entry can simply be increased from a 16 bit word to a 32 bit long. Although this change is only required for the Propeller 2, to keep the library functions consistent, it has been made on the Propeller 1 as well. 6. The speed of the p2asm assembler has been improved - it is now approximately twice as fast. I am still not sure why it is so slow on Windows compared to Linux, where exactly the same code executes about 10 times faster on the same machine. 7. If a baud rate is not specified and a file with a ".bin" extension is loaded, payload now assumes it is loading a Propeller 2 and sets the baud rate to 230400. This can be overridden by specifying a baud rate either on the command line or via the PAYLOAD_BAUD environment variable. 8. Windows 10 has a bug in the Command-Line interpreter. When loading a program using payload, it crashes (taking payload down with it) if the Wndows option "Wrap Text Output on Resize" option is not set in the Layout Options (in the Properties dialog box). This can leave a payload task executing in the background which needs to be terminated manually (e.g. using Task Manager) in order to release the USB port. 9. The translation of outgoing CR to LF has been disabled in the payload interactive terminal mode by default, but mode 16 has been added to restore the original behaviour if needed. The translation was preventing the use of payload to interact with TAQOZ and the P2 Monitor - the correct mode to use for these is now -q1. For instance, if you have a Propeller 2 connected to port X, you can communicate with TAQOZ by entering a command like: payload -i -q1 -pX Then reset the Propeller 2 and enter the usual TAQOZ entry sequence (i.e. '>' SPACE ESC) or P2 monitor entry sequence ('>' SPACE CTRL-D) 10. Catalina now supports a "thin" binding to Posix threads. See the "Posix threads (pthreads) support" section in the Catalina Reference Manual. 11. Lua module "threads" adds multi-threading capabilities to Lua (for the Propeller 2 only). The document "Lua on the Propeller 2 with Catalina" describes this new functionality in detail. 12. The interactive versions of Lua (and Multi-threaded Lua) now support trivial line editing (e.g. backspace is now recognized). 13. Catalina now supports setpin(), getpin() and togglepin() for both the Propeller 1 and 2. These are defined in propeller.h: setpin(pin, value) - sets pin to output and sets value setpin(pin) - sets pin to input and gets value togglepin(pin) - sets pin to input and toggles value These functions are also provided in the Lua propeller module, described in The document "Lua on the Propeller 2 with Catalina".Oops - just noticed a typo in the previous post, which is also in the README.TXT for Catalina 5.0 - togglepin(pin) sets pin to output (not input) and toggles its value. The software is correct, it is the README.TXT that is wrong.