I've just updated my earlier post (here) with a slightly improved version of payload. This new version is also on GitHub, and will be included in the next Catalina release.
When using payload in 'dumb' mode on Windows, payload now maps the keyboard so that programs see the same input that they would see under a terminal emulator operating in vt100 mode.
This makes programs such as Catalyst and the vi text editor (which are designed to work with a vt100 terminal emulator) work better under the Windows Console Host and the Windows Terminal app, which interpret some vt100 escape codes, but do not generate vt100 keyboard output.
There are no changes to 'dumb' mode on Linux, since most Linux terminals already do this.
I just discovered that to use ANSI escape codes in a Console Host window (i.e. what used to be plain old cmd.exe) now requires a registry setting. If you use the Terminal App instead of the Console Host, you don't need to worry about this. but I tend to still use the Console Host because it is both simpler and and faster than the Terminal App.
Attached is a simple batch file to set or clear the appropriate registry setting. It will be included in the next Catalina release. In the meantime, just unzip it into Catalina's bin directory, and then to enable ANSI escape code processing enter: ansi_mode on
or to disable it enter: ansi mode off
To just check the current state enter: ansi_mode
Note that because this is a registry setting, you only have to do it once. I made it possible to turn it on or off in just case it upsets other programs that use the Console Host window.
Turning ANSI mode on makes the new version of payload (which adds 'dumb' terminal support) work properly in a Console Host window. Otherwise ANSI escape codes (e.g. used by Catalyst) will work when using the Terminal App, but not Console Host.
I had a need to rebuild my Windows MinGW installation recently, which upgraded gcc to 15.2.0 (the most recent version). I found a few things in Catalina needed tweaking because this version of gcc raises errors in cases where previous versions did not. This does not affect any existing binaries, and there are no functional changes, so those who use Windows but do not need to rebuild Catalina don't need to do anything.
I have uploaded the tweaks to GitHub, and will include them in the next release.
@RossH said:
I had a need to rebuild my Windows MinGW installation recently, which upgraded gcc to 15.2.0 (the most recent version). I found a few things in Catalina needed tweaking because this version of gcc raises errors in cases where previous versions did not.
An update: The problem is not with Catalina. It is with either the latest version of gcc or MinGW, or with Windows. More investigation required. I hope the problem is with gcc or MinGW, because I can get around those simply by reverting to a version that works. If the problem with with Windows, then it will be a lot more difficult to find, and may be impossible to fix.
Just to repeat - the currently compiled Windows binaries (both on SourceForge or GitHub) are fine.
The problem is with the expansion of wildcards on the Windows command line - a process known as 'globbing'. On Linux, the command processor always does this for you. On Windows you have to do it yourself, but gcc can also do it for you.
What has happened is that in the current release of gcc/MinGW/Windows (not 100% sure which) globbing is apparently now turned off by default, whereas it used to be on by default.
All I had to do was add the line:
extern int _CRT_glob = 1; /* 0 turns off globbing; 1 turns it on */
I have added this line near the top of the following files:
source\catalina\catalina.c
source\catalina\catbind.c
source\catalina\bcc.c
source\lcc\etc\lcc.c
source\lcc\cpp\cpp.c
source\lcc\src\main.c
I will fix this on GitHub and also include it in the next release.
This is a full release. It adds a 'dumb' terminal mode to payload, updates Cake and the VT100 test program (vttest), and a has few minor non-functional changes that allow Catalina to compile correctly with the latest version of gcc.
Here is the relevant extract from the README.TXT:
New Functionality
-----------------
1. Payload now has a 'dumb' mode, which is enabled by using the -I option
but specifying either "dumb" or "none" as the terminal name (e.g. -Idumb
or -I none). In dumb mode (after download, if a file is also specified)
payload just echoes all characters received on the serial port to stdout,
and all characters received on stdin to the serial port. Dumb mode can be
terminated by entering TWO consecutive CTRL-D characters.
As the name implies, in dumb mode, payload does not do any character
processing on either input or output characters (except for detecting
two successive CTRL-D characters) - If the Propeller program does any
terminal-dependent stuff (e.g. cursor movement, screen manipulation,
applying colors etc) this will work if that capability is implemented
by the terminal window in which payload is executing. In general, this
means many things will work quite smoothly (in Windows) using cmd.exe,
conhost or the terminal app, or (in Linux) in any window type derived
from an xterm.
On Windows, the keyboard is assumed to be an extended ASCII keyboard,
and the keyboard input is mapped to that which would be generated by a
terminal emulator operating in vt100 mode. This is done so that programs
see the same keyboard input when -I dumb is specified as they would see if
either -i or -I vt100 was specified. This makes both the Windows Console
Host and Terminal App behave more like a real terminal emulator in vt100
mode when used with programs like Catalyst or the vi text editor.
2. A new batch file (ansi_mode.bat) is included to set or clear the registry
setting that enables processing ANSI escape codes in the Console Host
window (this is enabled by default in the Terminal App). To enable ANSI
escape code processing in Console Host enter:
ansi_mode on
or to disable it enter:
ansi mode off
Note that the setting will only affect NEW Console Host Windows, not
the current window. To just check the current state enter:
ansi_mode
Because this setting is in the registry, this only has to do be done
once, not on each login, or each time a Console Host window is opened.
It can be turned off again if it upsets other programs that use the
Console Host window.
Enabling ANSI mode makes the new 'dumb' terminal option offered by payload
work correctly in a Console Host window. Otherwise ANSI escape codes (e.g.
used by Catalyst) work in the Terminal App, but not Console Host.
Other Changes
-------------
1. The vttest program, which is used to test vt100 compatibility, has been
updated to the latest version. Note that this program is NOT required to
use Catalina - it is a utility that can be used to test compatibility
of various vt100 terminal emulators, including Catalina's terminal emulator
(Comms.exe) as well as the Windows Console Host and Terminal App. See the
README file in /source/comms/vttest for more details.
2. There were two problems in the build scripts in /source/geany_bundle:
1. The build_all script used the pacman cache. This only worked if the
packages had already been downloaded. Now they are downloaded on each
build.
2. The option to unpack compressed packages is now --zstd, not -xz
Affected Windows only.
3. In the blackbox debugger, calls to the function blackbox_close() should have
no parameters. This caused recent versions of gcc to throw an error. No
functional changes. Affected Windows only.
4. In awka, the function awka_init_parachute() was being defined incorrectly
in error.c to accept parameters. This causes the latest versions of gcc to
throw an error. No functional changes.
Affected Windows and Linux.
5. When compiling awka, the Catalina parallelizer and the Catalina optimizer
with the latest version of gcc, the option -std=c99 must be added for
the programs to compile correctly. No functional changes.
Affected Windows and Linux.
6. When compiling lcc with the latest version of gcc, the definition of
_spawnvp() must be changed to match that in stdio.h for the program to
compile correctly. No functional changes.
Affected Windows and Linux.
7. There is an issue with the current version of gcc/MinGW on Windows, which
means that 'globbing' (expansion of wildcard characters on the command line
was disabled by default. It can be manually re-enabled for programs that
need it by including the following line near the top of the main program
file:
extern int _CRT_glob = 1; /* 0 turns off globbing; 1 turns it on */
No functional changes.
Affected Windows only.
8. There is an issue with versions of gcc which reserve the keyword
"constexpr", which was used by LCC as an identifier. This has now been
changed in LCC to "cnstexpr". No functional changes.
Affected Windows and Linux.
9. Non-C89 compatible comments in LCC (i.e. introduced using //) have now
been replaced with C89-compatible comments (i.e. /* ... */). No functional
changes.
Affected Windows and Linux.
10. There is an issue in MinGW32 that means some programs may fail to compile
with gcc when an old version of libwinpthread-1.dll exists in the PATH.
An error message that "clock_gettime64" was not found is generated.
The current fix is that "libwinpthread-1.dll" in Catalina's bin and
catalina_geany/bin directories have been replaced with the latest version.
However, in case this fix causes other problems, the old version are still
included, but have been renamed to "libwinpthread-1.dll.old".
Affected Windows only.
This is a full release. The impetus for this release was the recent release of both Lua 5.4.8 and Lua 5.5.0. Lua 5.4.8 is the latest and greatest stable Lua release, while Lua 5.5.0 is the first of the next generation of Lua releases - interesting, but not mature or stable enough just yet to make it the default. However, while testing Lua 5.4.8 I found some issues that needed fixing, and also did some preparation work for the changes that will be coming in Lua 5.5.X.
Here is the relevant extract from the README.TXT:
New Functionality
-----------------
1. Lua 5.5.0 has now been included in this release. However, Lua 5.5.X is not
yet mature or stable enough to replace Catalina's fully integrated
version, which remains at 5.4.X. However, Lua 5.5.0 is fully functional
when used stand-alone, apart from the capabilities enabled by defining the
ENABLE_PSRAM or ENABLE HYPER Catalina symbols when compiling Lua. Those
capabilities are deeply embedded in the internals of Lua 5.4.X, were
developed before the more general capability of using PSRAM as XMM RAM
was added, and are deprecated in this release. They are likely to be
dropped altogether in the next Catalina release.
To build Lua 5.5.0, go to the folder demos\catalyst\lua-5.5.0 and use the
build_all script in the same way as used to build Lua 5.4.X. For example:
build_all P2_EDGE SIMPLE VT100 MHZ_200 OPTIMIZE
To copy all the executables to an SD card, use the copy_all script. Note
that this script will supersede any existing Lua binaries and programs on
the SD Card. For example, if drive the SD Card is detected as drive H, the
command to enter would be:
copy_all h:
2. The version of Lua fully integrated into Catalina has been updated from
version 5.4.4 to 5.4.8. There is no new Lua functionality in this update,
just a few Lua bug fixes. However, there have been some minor tweaks to
some example programs as a result, mainly the multi-processing with Lua
examples. This is because the changes may not affect Lua's functionality,
but they do sometimes affect Lua's memory utilization. The updated example
programs now work with both Lua 5.4.X and Lua 5.5.X.
Note that the compiled (i.e. binary) format of Lua programs has changed
between versions 5.4.X and 5.5.X. This means that the corresponding version
of the lua compiler (luac) must be used. If the wrong version is used, the
following error message will be displayed:
bad binary format (version mismatch)
3. There are now only two copies of the Lua 5.4.X source files in this release
of Catalina (in previous releases there were three copies). The Lua source
files that used to be in demos/catalyst/lua/5.4.X are now copied from
source/catalina/lua-5.4.X when Catalyst is compiled using the build_all
scripts, and deleted again by the clean_all scripts (similar to the way
the self-hosted version of Catalina itself is built). Note that when the
Windows installer is used to install Catalina, this means that to compile
Catalyst from source, the Catalina sources MUST be installed. This option
is now the default.
A copy of the Lua 5.4.X source files still exists in the library source
tree. This duplication is likely to remain so that the stand-alone version
of Lua can be updated independently from the integrated version.
Note that when Lua is built as part of Catalina (it is used for scripting
in both payload and blackbox) the standard Lua Makefiles are used. When it
is built as part of Catalyst, the Makefiles in the src folder are used
(Makefile.Catalina on Linux, or Makefile.Catalina.mgw on Windows).
4. Partly in anticipation of Lua 5.5.X becoming Catalina's default integrated
version of Lua, the multiple different versions of the Lua initialization
file (variously called linit.c, iinit.c or xinit.c) which enabled different
Lua libraries in previous releases have all been replaced with a common
version (linit.c). In this release, this file must still be present and
compiled in the same folder as the Lua application, but this will change
in Lua 5.5.X, which uses a different initialization process).
Specific libraries to be enabled or disabled are now specified on the
Catalina command line (or in a catapult pragma) as follows:
propeller always enabled
hmi always enabled
threads enabled by -lthreads
serial enabled by -lserial2 or -lserial8
wifi enabled by -lwifi
service enabled by -C LUA_SERVICE
linenoise enabled by -C LUA_LINENOISE
If memory is very tight, the optional debug, utf8 and os libraries can
be disabled by -C LUA_NO_OPT, and if only integer math is required then
the math library can be disabled by -C LUA_NO_MATH - but neither of
these are used in any of the demo programs, and doing so is deprecated
because these options will not be available when Lua 5.5.X becomes the
default version of Lua.
Other Changes
-------------
1. There was a bug in the Multiprocessing versions of Lua (i.e. mlua and
mluax) that may have led to a program lock-up when multiple factories were
in use (for example, it may have occurred when executing the example
programs ex9.lua or ex12.lua). Affected the Propeller 2 only.
2. There was a bug in Catalyst that limited commands to 8 characters when
no extension was specified. So (for example) the program "bin/dbasic.bin"
would be correctly found and executed if the command was entered as:
dbasic
dbasic.bin
bin/dbasic.bin
but NOT if entered as:
bin/dbasic
Instead, "cannot open file BIN/DBASIC" would be displayed.
Affected the Propeller 1 and Propeller 2.
3. Some Lua programs have been updated to allow for the use of either Lua 5.4
or Lua 5.5. In particular, Lua 5.5 requires different parameters to the
collectgarbage() function. The following code takes the lua version into
account, and can be used set a specific garbage collection mode and/or
parameters in a way that is backwards compatible:
lua_version = tonumber(string.sub(_VERSION, 5, 7));
if lua_version and lua_version < 5.4 then
-- default mode is incremental, so just set parameters ...
collectgarbage("setpause", 150);
collectgarbage("setstepmul", 500);
elseif lua_version and lua_version < 5.5 then
-- incremental mode is optional, so set it with parameters ...
collectgarbage("incremental", 150, 500);
else
-- mode and parameter setting now separate in Lua 5.5 ...
collectgarbage("incremental");
collectgarbage("param", "pause", 150);
collectgarbage("param", "stepmul", 500);
end
Affected the Propeller 1 and Propeller 2.
4. To minimize Lua's use of memory, the threads functions wait() and workers()
also used to do garbage collection in previous releases - this was done to
collect garbage after workers and threads had been initiated. However, a
better solution is to do this explicitly if needed, and also for threads
themselves to be able to perform garbage collection if required, so a new
"thread safe" function for invoking the Lua garbage collector has been
added to the threads module. The threads.gc() function can combine the
functionality of collectgarbage() and threads.sbrk() (which are often used
together to maximize free Lua memory and also defragment the C heap) into
one function. See the updated "Lua on the Propeller 2 with Catalina"
document for more details. The example program ex9.lua demonstrates the
new function. If backwards compatibility is necessary, Lua code such as
the following can be used to detect if the threads.gc() function is
available or not:
if threads.gc then
threads.gc(1) -- also invokes sbrk(true)
else
-- note: not guaranteed thread safe ...
collectgarbage()
thread.sbrk(true)
end
Affected the Propeller 2 only.
5. Upgrading to Lua 5.4.8 has made some programs that used the integrated
version of Lua (i.e. the version in the Catalina library) slightly larger,
which may cause the program to run out of Hub RAM. For example, the Aloha
WiFi RPC demo program sluarfx.c is now too large to execute correctly if
compiled in NATIVE mode, and so has been replaced by the functionally
identical program slaurcx.c which instead uses COMPACT mode.
This change may affect other programs. Affected the Propeller 2 only.
6. The 'Aloha from Lua' document mentioned that the rpc_network table could
be in the common.lua module, but in the RPC demo programs the table was
instead duplicated in the server.lua, remote.lua and serverbg.lua modules.
While this is technically ok, it complicates the process of configuring
the demo programs. Now, the document and all the demo programs have been
made consistent, with only one copy of the rpc_service table in the
common.lua module. Affected the Propeller 2 only.
7. There were a couple of issues with the WiFi version of ALOHA:
(a) There was an undocumented limitation on the size of an WiFi RPC
request. The code currently limits such requests to a maximum of 512
bytes (the RPC response can be larger). This limit caused one of the
demo programs (i.e. the 'complete' demo) to work with the serial
version, but not the WiFi version because the string used was too
long. The example has now been corrected to work with both, and the
'ALOHA from Lua' document also has been updated. This limit may be
increased in a future release.
(b) The Lua dispatcher could fail to retrieve all data returned by an
RPC call when the response consisted of multiple data packets.
Affected the Propeller 2 only.
Comments
See the document BUILD.TXT for instructions on building Catalina on both Windows and Linux. Linux is pretty straightforward.
I've just updated my earlier post (here) with a slightly improved version of payload. This new version is also on GitHub, and will be included in the next Catalina release.
When using payload in 'dumb' mode on Windows, payload now maps the keyboard so that programs see the same input that they would see under a terminal emulator operating in vt100 mode.
This makes programs such as Catalyst and the vi text editor (which are designed to work with a vt100 terminal emulator) work better under the Windows Console Host and the Windows Terminal app, which interpret some vt100 escape codes, but do not generate vt100 keyboard output.
There are no changes to 'dumb' mode on Linux, since most Linux terminals already do this.
Ross.
sigh - Windows gets you every time
I just discovered that to use ANSI escape codes in a Console Host window (i.e. what used to be plain old cmd.exe) now requires a registry setting. If you use the Terminal App instead of the Console Host, you don't need to worry about this. but I tend to still use the Console Host because it is both simpler and and faster than the Terminal App.
Attached is a simple batch file to set or clear the appropriate registry setting. It will be included in the next Catalina release. In the meantime, just unzip it into Catalina's bin directory, and then to enable ANSI escape code processing enter:
ansi_mode onor to disable it enter:
ansi mode offTo just check the current state enter:
ansi_modeNote that because this is a registry setting, you only have to do it once. I made it possible to turn it on or off in just case it upsets other programs that use the Console Host window.
Turning ANSI mode on makes the new version of payload (which adds 'dumb' terminal support) work properly in a Console Host window. Otherwise ANSI escape codes (e.g. used by Catalyst) will work when using the Terminal App, but not Console Host.
Ross.
I've put up a new Patreon video about Catalina's new smart "dumb" mode - see here.
I had a need to rebuild my Windows MinGW installation recently, which upgraded gcc to 15.2.0 (the most recent version). I found a few things in Catalina needed tweaking because this version of gcc raises errors in cases where previous versions did not. This does not affect any existing binaries, and there are no functional changes, so those who use Windows but do not need to rebuild Catalina don't need to do anything.
I have uploaded the tweaks to GitHub, and will include them in the next release.
Ross.
An update: The problem is not with Catalina. It is with either the latest version of gcc or MinGW, or with Windows. More investigation required. I hope the problem is with gcc or MinGW, because I can get around those simply by reverting to a version that works. If the problem with with Windows, then it will be a lot more difficult to find, and may be impossible to fix.
Just to repeat - the currently compiled Windows binaries (both on SourceForge or GitHub) are fine.
Ross.
Ha! Found it!
The problem is with the expansion of wildcards on the Windows command line - a process known as 'globbing'. On Linux, the command processor always does this for you. On Windows you have to do it yourself, but gcc can also do it for you.
What has happened is that in the current release of gcc/MinGW/Windows (not 100% sure which) globbing is apparently now turned off by default, whereas it used to be on by default.
All I had to do was add the line:
I have added this line near the top of the following files:
I will fix this on GitHub and also include it in the next release.
Ross.
Catalina 8.8.4 has been released on GitHub and SourceForge.
This is a full release. It adds a 'dumb' terminal mode to payload, updates Cake and the VT100 test program (vttest), and a has few minor non-functional changes that allow Catalina to compile correctly with the latest version of gcc.
Here is the relevant extract from the README.TXT:
New Functionality ----------------- 1. Payload now has a 'dumb' mode, which is enabled by using the -I option but specifying either "dumb" or "none" as the terminal name (e.g. -Idumb or -I none). In dumb mode (after download, if a file is also specified) payload just echoes all characters received on the serial port to stdout, and all characters received on stdin to the serial port. Dumb mode can be terminated by entering TWO consecutive CTRL-D characters. As the name implies, in dumb mode, payload does not do any character processing on either input or output characters (except for detecting two successive CTRL-D characters) - If the Propeller program does any terminal-dependent stuff (e.g. cursor movement, screen manipulation, applying colors etc) this will work if that capability is implemented by the terminal window in which payload is executing. In general, this means many things will work quite smoothly (in Windows) using cmd.exe, conhost or the terminal app, or (in Linux) in any window type derived from an xterm. On Windows, the keyboard is assumed to be an extended ASCII keyboard, and the keyboard input is mapped to that which would be generated by a terminal emulator operating in vt100 mode. This is done so that programs see the same keyboard input when -I dumb is specified as they would see if either -i or -I vt100 was specified. This makes both the Windows Console Host and Terminal App behave more like a real terminal emulator in vt100 mode when used with programs like Catalyst or the vi text editor. 2. A new batch file (ansi_mode.bat) is included to set or clear the registry setting that enables processing ANSI escape codes in the Console Host window (this is enabled by default in the Terminal App). To enable ANSI escape code processing in Console Host enter: ansi_mode on or to disable it enter: ansi mode off Note that the setting will only affect NEW Console Host Windows, not the current window. To just check the current state enter: ansi_mode Because this setting is in the registry, this only has to do be done once, not on each login, or each time a Console Host window is opened. It can be turned off again if it upsets other programs that use the Console Host window. Enabling ANSI mode makes the new 'dumb' terminal option offered by payload work correctly in a Console Host window. Otherwise ANSI escape codes (e.g. used by Catalyst) work in the Terminal App, but not Console Host. Other Changes ------------- 1. The vttest program, which is used to test vt100 compatibility, has been updated to the latest version. Note that this program is NOT required to use Catalina - it is a utility that can be used to test compatibility of various vt100 terminal emulators, including Catalina's terminal emulator (Comms.exe) as well as the Windows Console Host and Terminal App. See the README file in /source/comms/vttest for more details. 2. There were two problems in the build scripts in /source/geany_bundle: 1. The build_all script used the pacman cache. This only worked if the packages had already been downloaded. Now they are downloaded on each build. 2. The option to unpack compressed packages is now --zstd, not -xz Affected Windows only. 3. In the blackbox debugger, calls to the function blackbox_close() should have no parameters. This caused recent versions of gcc to throw an error. No functional changes. Affected Windows only. 4. In awka, the function awka_init_parachute() was being defined incorrectly in error.c to accept parameters. This causes the latest versions of gcc to throw an error. No functional changes. Affected Windows and Linux. 5. When compiling awka, the Catalina parallelizer and the Catalina optimizer with the latest version of gcc, the option -std=c99 must be added for the programs to compile correctly. No functional changes. Affected Windows and Linux. 6. When compiling lcc with the latest version of gcc, the definition of _spawnvp() must be changed to match that in stdio.h for the program to compile correctly. No functional changes. Affected Windows and Linux. 7. There is an issue with the current version of gcc/MinGW on Windows, which means that 'globbing' (expansion of wildcard characters on the command line was disabled by default. It can be manually re-enabled for programs that need it by including the following line near the top of the main program file: extern int _CRT_glob = 1; /* 0 turns off globbing; 1 turns it on */ No functional changes. Affected Windows only. 8. There is an issue with versions of gcc which reserve the keyword "constexpr", which was used by LCC as an identifier. This has now been changed in LCC to "cnstexpr". No functional changes. Affected Windows and Linux. 9. Non-C89 compatible comments in LCC (i.e. introduced using //) have now been replaced with C89-compatible comments (i.e. /* ... */). No functional changes. Affected Windows and Linux. 10. There is an issue in MinGW32 that means some programs may fail to compile with gcc when an old version of libwinpthread-1.dll exists in the PATH. An error message that "clock_gettime64" was not found is generated. The current fix is that "libwinpthread-1.dll" in Catalina's bin and catalina_geany/bin directories have been replaced with the latest version. However, in case this fix causes other problems, the old version are still included, but have been renamed to "libwinpthread-1.dll.old". Affected Windows only.Catalina 8.8.5 has been released on GitHub and SourceForge.
This is a full release. The impetus for this release was the recent release of both Lua 5.4.8 and Lua 5.5.0. Lua 5.4.8 is the latest and greatest stable Lua release, while Lua 5.5.0 is the first of the next generation of Lua releases - interesting, but not mature or stable enough just yet to make it the default. However, while testing Lua 5.4.8 I found some issues that needed fixing, and also did some preparation work for the changes that will be coming in Lua 5.5.X.
Here is the relevant extract from the README.TXT:
New Functionality ----------------- 1. Lua 5.5.0 has now been included in this release. However, Lua 5.5.X is not yet mature or stable enough to replace Catalina's fully integrated version, which remains at 5.4.X. However, Lua 5.5.0 is fully functional when used stand-alone, apart from the capabilities enabled by defining the ENABLE_PSRAM or ENABLE HYPER Catalina symbols when compiling Lua. Those capabilities are deeply embedded in the internals of Lua 5.4.X, were developed before the more general capability of using PSRAM as XMM RAM was added, and are deprecated in this release. They are likely to be dropped altogether in the next Catalina release. To build Lua 5.5.0, go to the folder demos\catalyst\lua-5.5.0 and use the build_all script in the same way as used to build Lua 5.4.X. For example: build_all P2_EDGE SIMPLE VT100 MHZ_200 OPTIMIZE To copy all the executables to an SD card, use the copy_all script. Note that this script will supersede any existing Lua binaries and programs on the SD Card. For example, if drive the SD Card is detected as drive H, the command to enter would be: copy_all h: 2. The version of Lua fully integrated into Catalina has been updated from version 5.4.4 to 5.4.8. There is no new Lua functionality in this update, just a few Lua bug fixes. However, there have been some minor tweaks to some example programs as a result, mainly the multi-processing with Lua examples. This is because the changes may not affect Lua's functionality, but they do sometimes affect Lua's memory utilization. The updated example programs now work with both Lua 5.4.X and Lua 5.5.X. Note that the compiled (i.e. binary) format of Lua programs has changed between versions 5.4.X and 5.5.X. This means that the corresponding version of the lua compiler (luac) must be used. If the wrong version is used, the following error message will be displayed: bad binary format (version mismatch) 3. There are now only two copies of the Lua 5.4.X source files in this release of Catalina (in previous releases there were three copies). The Lua source files that used to be in demos/catalyst/lua/5.4.X are now copied from source/catalina/lua-5.4.X when Catalyst is compiled using the build_all scripts, and deleted again by the clean_all scripts (similar to the way the self-hosted version of Catalina itself is built). Note that when the Windows installer is used to install Catalina, this means that to compile Catalyst from source, the Catalina sources MUST be installed. This option is now the default. A copy of the Lua 5.4.X source files still exists in the library source tree. This duplication is likely to remain so that the stand-alone version of Lua can be updated independently from the integrated version. Note that when Lua is built as part of Catalina (it is used for scripting in both payload and blackbox) the standard Lua Makefiles are used. When it is built as part of Catalyst, the Makefiles in the src folder are used (Makefile.Catalina on Linux, or Makefile.Catalina.mgw on Windows). 4. Partly in anticipation of Lua 5.5.X becoming Catalina's default integrated version of Lua, the multiple different versions of the Lua initialization file (variously called linit.c, iinit.c or xinit.c) which enabled different Lua libraries in previous releases have all been replaced with a common version (linit.c). In this release, this file must still be present and compiled in the same folder as the Lua application, but this will change in Lua 5.5.X, which uses a different initialization process). Specific libraries to be enabled or disabled are now specified on the Catalina command line (or in a catapult pragma) as follows: propeller always enabled hmi always enabled threads enabled by -lthreads serial enabled by -lserial2 or -lserial8 wifi enabled by -lwifi service enabled by -C LUA_SERVICE linenoise enabled by -C LUA_LINENOISE If memory is very tight, the optional debug, utf8 and os libraries can be disabled by -C LUA_NO_OPT, and if only integer math is required then the math library can be disabled by -C LUA_NO_MATH - but neither of these are used in any of the demo programs, and doing so is deprecated because these options will not be available when Lua 5.5.X becomes the default version of Lua. Other Changes ------------- 1. There was a bug in the Multiprocessing versions of Lua (i.e. mlua and mluax) that may have led to a program lock-up when multiple factories were in use (for example, it may have occurred when executing the example programs ex9.lua or ex12.lua). Affected the Propeller 2 only. 2. There was a bug in Catalyst that limited commands to 8 characters when no extension was specified. So (for example) the program "bin/dbasic.bin" would be correctly found and executed if the command was entered as: dbasic dbasic.bin bin/dbasic.bin but NOT if entered as: bin/dbasic Instead, "cannot open file BIN/DBASIC" would be displayed. Affected the Propeller 1 and Propeller 2. 3. Some Lua programs have been updated to allow for the use of either Lua 5.4 or Lua 5.5. In particular, Lua 5.5 requires different parameters to the collectgarbage() function. The following code takes the lua version into account, and can be used set a specific garbage collection mode and/or parameters in a way that is backwards compatible: lua_version = tonumber(string.sub(_VERSION, 5, 7)); if lua_version and lua_version < 5.4 then -- default mode is incremental, so just set parameters ... collectgarbage("setpause", 150); collectgarbage("setstepmul", 500); elseif lua_version and lua_version < 5.5 then -- incremental mode is optional, so set it with parameters ... collectgarbage("incremental", 150, 500); else -- mode and parameter setting now separate in Lua 5.5 ... collectgarbage("incremental"); collectgarbage("param", "pause", 150); collectgarbage("param", "stepmul", 500); end Affected the Propeller 1 and Propeller 2. 4. To minimize Lua's use of memory, the threads functions wait() and workers() also used to do garbage collection in previous releases - this was done to collect garbage after workers and threads had been initiated. However, a better solution is to do this explicitly if needed, and also for threads themselves to be able to perform garbage collection if required, so a new "thread safe" function for invoking the Lua garbage collector has been added to the threads module. The threads.gc() function can combine the functionality of collectgarbage() and threads.sbrk() (which are often used together to maximize free Lua memory and also defragment the C heap) into one function. See the updated "Lua on the Propeller 2 with Catalina" document for more details. The example program ex9.lua demonstrates the new function. If backwards compatibility is necessary, Lua code such as the following can be used to detect if the threads.gc() function is available or not: if threads.gc then threads.gc(1) -- also invokes sbrk(true) else -- note: not guaranteed thread safe ... collectgarbage() thread.sbrk(true) end Affected the Propeller 2 only. 5. Upgrading to Lua 5.4.8 has made some programs that used the integrated version of Lua (i.e. the version in the Catalina library) slightly larger, which may cause the program to run out of Hub RAM. For example, the Aloha WiFi RPC demo program sluarfx.c is now too large to execute correctly if compiled in NATIVE mode, and so has been replaced by the functionally identical program slaurcx.c which instead uses COMPACT mode. This change may affect other programs. Affected the Propeller 2 only. 6. The 'Aloha from Lua' document mentioned that the rpc_network table could be in the common.lua module, but in the RPC demo programs the table was instead duplicated in the server.lua, remote.lua and serverbg.lua modules. While this is technically ok, it complicates the process of configuring the demo programs. Now, the document and all the demo programs have been made consistent, with only one copy of the rpc_service table in the common.lua module. Affected the Propeller 2 only. 7. There were a couple of issues with the WiFi version of ALOHA: (a) There was an undocumented limitation on the size of an WiFi RPC request. The code currently limits such requests to a maximum of 512 bytes (the RPC response can be larger). This limit caused one of the demo programs (i.e. the 'complete' demo) to work with the serial version, but not the WiFi version because the string used was too long. The example has now been corrected to work with both, and the 'ALOHA from Lua' document also has been updated. This limit may be increased in a future release. (b) The Lua dispatcher could fail to retrieve all data returned by an RPC call when the response consisted of multiple data packets. Affected the Propeller 2 only.