Catalina - ANSI C and Lua for the Propeller 1 & 2

Wingineer19 · 2023-09-30 03:40

@RossH said:
A MAC address is a type with 48 bits, not a 48 bit integer value. You don't generally need integer operations on a MAC address, and it is probably better represented not as an integer at all, but as a type with 6 octets and with any operations you need specifically defined for it.

@Wingineer19 said:
That's pretty much how I do it now. In my code, the MAC address is defined as a union of six bytes with three unsigned shorts.

Comparing each of the three unsigned shorts works fine, but having the 48-bit MAC address stored within a union containing a 64-bit long might allow a faster and more efficient way, at least from a coding perspective, to perform the lookup. But this would come at the cost of an increased size of the union.

Yeah, or I can just use the memcmp() function to compare the six bytes of the MAC address within the union of each WiFi station the rover hears with those listed within the Table in order to extract the station coordinates.

Something along these lines:

 //program is mactest.c
//last revision on 29sep23

//catalina mactest.c -lc -lma -C TTY -C FLIP -C COMPACT -y


#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdarg.h>
#include <math.h>
#include <time.h>
#include <propeller.h>

struct Tenrehte         
{
 union
  {
   unsigned char  Byte[6];
   unsigned short Word[3];
  }MAC;
};

struct Tenrehte One;
struct Tenrehte Two;
struct Tenrehte Three;

void main(void)
{
 One.MAC.Word[2]=0x1234;  // 0x123456789abc
 One.MAC.Word[1]=0x5678;
 One.MAC.Word[0]=0x9abc;

 Two.MAC.Word[2]=0x1234;  // 0x123456789abc
 Two.MAC.Word[1]=0x5678;
 Two.MAC.Word[0]=0x9abc;

 Three.MAC.Word[2]=0x2345;  // 0x23456789abcd
 Three.MAC.Word[1]=0x6789;
 Three.MAC.Word[0]=0xabcd;

 if(memcmp(&One.MAC,&Two.MAC,sizeof(One.MAC)) == 0) puts("One and Two Are Equal\r\n");
 else puts("One And Two Are Different\r\n");

 if(memcmp(&One.MAC,&Three.MAC,sizeof(One.MAC)) == 0) puts("One and Three Are Equal\r\n");
 else puts("One And Three Are Different\r\n");

 if(memcmp(&Two.MAC,&Three.MAC,sizeof(One.MAC)) == 0) puts("Two and Three Are Equal\r\n");
 else puts("Two And Three Are Different\r\n");

 while(1);
}

RossH · 2023-10-01 00:07

Yes, being able to define your own operations is one of the few advantages C++ has over C. But having conceded that C++ has any advantages over C, I must also point out that Ada is far better at it than C++, since you can intuitively define your own operations even in non-OO programs

RossH · 2023-10-01 05:37

Looking for advice from any Windows/Linux/POSIX gurus here. Anyone not at least vaguely familiar with the internals of all of these should probably look away - things can get very ugly very fast!

Although Catalina runs on Windows, it is not a Windows program, and it does not use any Windows-specific features. It could be be most accurately described as a suite of POSIX programs plus a few batch scripts. This is intentional, because I want it to run natively on Linux and other operating systems as well as on Windows.

However, compiling POSIX programs to run on Windows can be a painful process. Windows itself is not POSIX compliant (there used to be a Microsoft POSIX Subsystem and then a Windows Subsystem for Unix, but both were discontinued some time ago) so there seems to be no perfect Windows POSIX solution.

For a long time I was able to use just MinGW with MSYS1, but MinGW now appears to be deprecated if not already defunct, and it has been replaced by MinGW-32 or MinGW-64 with MSYS2. Sadly, these are not fully compatible with the original MinGW, and are also heading in the wrong direction from my perspective (i.e. they have decided they need to be more Windows compatible rather than more POSIX compatible).

For the last few releases of Catalina I have been compiling the Windows versions using a combination of MinGW and MinGW-32, but now I find there is at least one component that will no longer compile under this regime, and so I now need to use something else as well. Currently, that is Cygwin. Clearly, this is not a great solution, and one that it likely to get more complicated over time. I really want a single, stable long-term solution.

I have tried the current Microsoft offering, which is Windows Subsystem for Linux (WSL). WSL is quite easy - you essentially just install Linux and run Linux programs on Linux under Windows. In many ways WSL is similar to running Linux in a Virtual Machine, and so it has the same drawbacks - for example, you cannot use the Windows command-line. Instead you have to use a Linux shell (which many Windows users would be unfamiliar with). Also, it involves installing a completely new operating system, so it has a substantial footprint in size, performance and ongoing maintenance. Also, Microsoft has a track record of discontinuing such products.

So, the only candidate seems to be Cygwin64 - which as the name implies is the 64 bit version of Cygwin (the original was 32 bits only). Cygwin64 provides what is perhaps the most complete POSIX functionality for Windows. Certainly, it is sufficiently POSIX compliant to compile and run the whole suite of Catalina components. However, unlike WSL, Cygwin applications can be run seamlessly from a Windows command line or Terminal window just as easily as from a Linux shell. In fact, if it were not for a couple of minor issues (described below), you would be unable to tell whether Catalina was a Cygwin64 application or not. Also, Cygwin has been around longer than any of the Microsoft POSIX solutions and looks like it is here for the long term.

Note that Cygwin64 itself is not required by programs built using Catalina, and so it imposes no additional license restrictions on such programs. Also note that Catalina does not require the installation of Cygwin64 itself unless you want to rebuild Catalina - to just use it you only need the Cygwin DLL (i.e. cygwin1.dll).

Here are the minor issues I have found (so far) with using Cygwin64 to build Catalina:

Cygwin does not like having multiple versions of its DLL executing even if they are the same version - so Catalina users need to be aware of this if they already have some Cygwin software installed and then install Catalina (or vice versa). This is a known Cygwin issue with no completely fool-proof solution. The solution suggested by Cygwin in their FAQ is to never just install the Cygwin DLL alone, but to always install the full version of Cygwin if it is not already installed.
Cygwin really does not like spaces in path names, so if Catalina is built with Cygwin64 but installed in the usual location for Windows programs (e.g. "C:\Program Files\Catalina" or C:\Program Files (x86)\Catalina" then it will not work, and no amount of quoting or using escape characters seems able to make it work in all cases. However, there is a solution - just create a symbolic link to Catalina for Cygwin to use that has no spaces in it. Here is what I do on on my machine to make it work for version 6.2 (yet to be released):

mklink /D "C:\catalina\6.2\" "C:\Program Files (x86)\Catalina_6.2"

Then if I set LCCDIR to C:\catalina\6.2 rather than the more usual C:\Program Files (x86)\Catalina_6.2 everything works as expected. Again, this can be done during installation, but again it is not 100% seamless since it means the installer now requires Administrator permissions.

So, here are a few questions before I commit to this course for the next release:

Am I missing an easier option?
Are there other issues associated with Cygwin64 that I am yet to discover?
Is the need to install Catalina with Administrator permissions likely to be a problem?
Should Catalina install the full version of Cygwin, or just the Cygwin DLL? Neither approach is guaranteed to be problem free.

Thanks in advance!

Ross.

ersmith · 2023-10-01 13:20

@RossH : What problems are you having with MSYS2? As you've noted, Cygwin is very fragile -- I found the same thing. Any attempt to mix versions or to do anything differently from what they expected tended to break things. I used to use it, but gave up on it in favor of Mingw, which so far at least just works for me. If you have access to a Linux box you can cross-compile your tools for Windows using mingw64, which is an extra bonus. For Windows users MSYS2, on the whole, seems to work. Yes, they may be more intested in Windows compatibility than POSIX compatibility, but even so the libraries do track the C and C++ standards, and have a lot of compatibility with various Unix extensions as well.

RossH · 2023-10-01 23:29

@ersmith said:

@RossH : What problems are you having with MSYS2?

My main problem at the moment is compiling 'awka'. I used to be able to compile it but I cannot do so with the current versions of MinGW/MSYS. Now, 'awka' is a very old program, and it has not changed since I began using it years ago. I don't even routinely re-compile it, so I didn't notice exactly when it stopped being able to be compiled by the tools I use. I have tried to reproduce the version (or combination of versions) of MinGW & MSYS that can compile it, but have been unable to do do.

But Cygwin can compile it (along with everything else I use). My reading of various articles about the differences between Cygwin and MinGW is that this is probably because Cygwin is more POSIX compliant. Cygwin starts with the POSIX standards and implements them fairly rigorously on Windows - this is its primary goal. On the other hand MinGW starts with the Windows API and includes some POSIX compliance on top of that, but this is not its primary goal. There are known to be Unix applications that MinGW cannot compile which Cygwin can.

If anyone can get awka to compile for Windows using tools other than Cygwin it would at least give me more options. The source is available here - https://awka.sourceforge.net/download.html or here - https://gnuwin32.sourceforge.net/packages/awka.htm (note that I don't use the binaries available in that second link - I need to be able to compile and use my own, because I would like to make some changes to awka).

Ross.

EDIT: Ok, something strange is going on - I just decided to try recompiling 'awka' with MinGW one last time, and this time it apparently worked - and it hasn't been working all week!

I am wondering if my MinGW installation was corrupted. Or possibly having both MinGW and Cygwin installed is bad. I will try cleaning them both out and re-installing.

ANOTHER EDIT: Ok, so not 100% success - 'awka' now compiles under MSYS2 MINGW64 but 'geany' now doesn't, whereas previously it was the other way around. Yet everything always compiles under Cygwin64. I think I've been around this loop several times before

ANOTHER EDIT: Oooh, it just keeps getting worse with MSYS2 and MINGW ... it turns out the 'curl' command doesn't work properly ... more hours wasted ...

RossH · 2023-10-03 03:05

Just to finish off this little detour - it seems that MSYS2 is a bit too unstable just yet. With enough frigging about with different versions I can build all the Catalina components, but there doesn't seem to be any single version of MSYS2 that can be used to build all of them. And mixing versions seems to do very bad things. Sometimes basic MSYS2 components just stop working, and removing and re-installing just those components doesn't fix things. Checking the integrity of all installed packages doesn't identify any problems either. The only solution I have found is to completely delete and re-install the entire MSYS2 installation. I have had to do this half a dozen times already, and this makes finding problems really difficult. I suspect there are some broken shared libraries being installed somewhere along the way which breaks things that were already installed and working. Clearly, this should not happen if the package manager was doing its job properly.

So it looks like Cygwin64 is it - at least for the immediate future. Catalina users should not notice much (if any) difference - it only becomes an issue for those who want to rebuild Catalina itself, which I doubt applies to many people other than me (at least not on Windows). For instance, just adding a new platform, plugin, XMM driver or C library does not require rebuilding any Catalina components.

I will revisit MSYS2 again at a later date.

Ross.

RossH · 2023-10-05 23:23

Yet another final update on the Windows/POSIX issue

I have struck another minor issue - Cygwin works but it does not use native Windows graphics for GUI applications - it uses X-Windows (i.e. Cygwin-X). Whereas MSYS implements GUI applications natively. This does not affect the functionality, but it does affect the look and feel of components like the Catalina Geany GUI.

I have found I can just do a quick update of MSYS1 (now defunct) to MSYS2, so I may persist with having MSYS in the mix for a while longer. It makes building Catalina on Windows more complex, but I am probably the only one that actually needs to do this.

Ross.

RossH · 2023-10-15 04:41

Catalina 6.2 has been released here.

The main purpose of this release is to bring Catalina up to date in terms of the tools required to build it on Windows. It is now built with the currents version of MINGW, MSYS2 and Cygwin. I am deferring the decision as to whether or not to move Catalina entirely to Cygwin until I get more familiarity with it. I have also added more detailed build instructions for both Windows and Linux, tidied up the licensing details, and fixed a few small issues. There are no major functional additions.

Here are the relevant extracts from the README.TXT since Catalina 6.1:

RELEASE 6.2

New Functionality
-----------------

1. The lua execution engine can now be built to use either PSRAM or HYPER RAM
   for code - just as you could add ENABLE_PRSAM to to the normal Catalyst
   or Lua build_all script, you can now add ENABLE_HYPER. The resulting
   executable will be called src/luaxp.bin. Refer to the Catalyst Reference
   Manual, and the section in it called Propeller 2 Platform-specific Notes
   for more details.

2. Two new Multi-processing demos have been added - diners.lua models the
   classic "dining philosopher" problem, and solved.lua offers one possible 
   solution to it.

3. All versions of Lua now accept a new optional parameter to the version 
   function in the propeller module - the string "hardware". For example:

      propeller.version("hardware")

   will now return: 

      0 on Windows or Linux
      1 on a Propeller 1
      2 on a Propeller 2

4. A new SPI demo program (demos\spi\dump_eeprom.c) has been added. See the 
   demos\spi folder for details.

5. New PASM test programs have been added to test the PASM _PSTR macro.
   See the demos\pasm_pstr folder for details.

6. A new demo program has been added (demos\examples\ex_str_print.c) to 
   print values to strings.

Other Changes
-------------

1. The Lua Windows Makefile has been updated to build the Multi-processor 
   version of Lua (i.e. mlua and mluax) correctly for Windows using mingw,
   or Linux using gcc. Note that the Propeller version was being built 
   correctly - it uses a different Makefile (Makefile.Catalina).

2. The Lua build_all script and Catalina Makefile (Makefile.Catalina) now 
   accept any memory model on the command line, whereas previously they would
   only work for SMALL and LARGE. If no memory model is specified they still
   default to LARGE for the Propeller 1 or COMPACT for the Propeller 2, but 
   it is now possible to override this (for instance) to build Lua in NATIVE 
   mode on the Propeller 2, which is much faster than COMPACT mode for small
   Lua programs.

3. The Windows and Linux versions of Multi-processing Lua (mlua) and the 
   corresponding Multi-processing Lua Execution Engine (mluax) have been 
   updated to accept requests for more than one factory without generating
   an error message. This was for consistency with the Propeller versions,
   which already did so, and requests for more factories than there were
   available cogs would simply result in all available cogs being used. 
   However, on both Windows and Linux only one factory will actually be
   used.

4. Geany has been updated to version 1.37.1. This was primarily a bug fix
   release, with no significant functional changes.

5. Details build instructions have been added for Windows and Linux - see
   the file BUILD.TXT.

6. Some Windows components now need to be built using Cygwin, and the
   remainder are now built with the current version of MINGW and MSYS2. 
   The build scripts have been updated (see the file BUILD.TXT for more
   details) and the installer now installs the necessary Cygwin DLL 
   (as bin\cygwin1.dll).

7. Various bugs have been fixed in the following scripts. Affected Linux only:
      build_utilities
      catalina_geany 
      Set_Linux_Permissions
      flash_payload

8. The maximum number of symbols the Spinnaker (OpenSpin) compiler can have
   in its symbol table has been increased from 8192 to 16384 to allow the
   compilation of very large programs. In particular, the Catalyst "vi" text
   editor would not compile if the Catalina Optimizer was used, which causes
   the generation of more symbols. Affected only the Propeller 1.

9. Details on the licensing of Catalina and programs built with Catalina
   have been clarified and updated. There have been no changes to the actual
   license conditions which are essentially GPL for Catalina itself and 
   Lesser GPL (aka LGPL) for programs built USING Catalina (see the files 
   COPYING and COPYING.LESSER in the Catalina directory, and also the file 
   Copyright in the sources/lib directory for the C89 library, which is 
   based largely on the Amsterdam Compiler Kit). The only time you have 
   to impose additional restrictions on a program BUILT with Catalina is if 
   you distribute the Catalina Target Package or C89 library (e.g. because 
   you have modified them and want to be able to distribute sources to your
   program). In that case, you must also distribute the modified components 
   according to the terms of the appropriate license.

10. On Windows, payload now uses ncurses rather than pdcurses by default. 
    This means that the window size cannot be set from within the program. 
    The -g command line option sets the terminal geometry parameters, but 
    no longer resizes the terminal window, which must be done manually.
    This was always the case on Linux, so this change makes both platforms 
    now behave the same way. 
    To ease the transition, a new batch file (payloadi on Unix, payloadi.bat 
    on Windows) is included with Catalina to do this for you when the -g 
    parameter is specified. Use payloadi just as you would use payload.

11. A delay between characters has been added to payload's interactive 
    terminal emulator, since the Propeller 1 TTY interface has such a small 
    character buffer (16 chars) it is very easy to overwhelm it by holding a 
    key down. This was a particular problem for programs such as the vi 
    editor when the cursor movement keys are held down.

RELEASE 6.1.1

New Functionality
-----------------

1. A _PSTR() macro has been defined, which can be used within calls to the
   PASM() function to interpret C escape sequences in strings. See the 
   Catalina Reference Manual for details.

Other Changes
-------------

None.

Ross.

RossH · 2023-10-16 05:42

Attached is a slightly updated Catalina Target Package (CTP) for the Propeller 1. It can be used with either Catalina 6.1 or 6.2, on either Windows or Linux.

This CTP just adds one new Propeller 1 HMI option - TTY256 - which specifies the program should use the Full Duplex Serial Plugin with 256 byte send and receive buffers. There are no changes to the Propeller 2 CTP.

The TTY256 HMI option applies to all supported P1 platforms, and can generally be used in place of the existing TTY option. However, note that using TTY256 consumes an additional 500 or so bytes of Hub RAM which some programs may be unable to accommodate, so TTY remains the default serial HMI option.

Using TTY256 in place of TTY can speed up serial output from P1 programs, and it also makes programs such as the vi full-screen text editor less prone to losing keystrokes if (for example) too many cursor keys are entered too fast. Since vi uses DEC VT100 escape sequences, every single cursor key press (such as cursor left, right, up or down) is actually 4 bytes long, so a 16 character buffer can fill up very quickly - and if the program loses any bytes because of this then the screen can end up garbled.

There are several ways to use this new CTP:

Just unzip it anywhere, then copy the target_6.2.1\p1 folder over the existing target\p1 folder in the main Catalina 6.2 directory. Then specify -C TTY256 on the Catalina command line (or in the Catalina Options in Geany) - e.g:

cd demos
catalina hello_world.c -lci -C TTY256

Unzip it in the main Catalina 6.2 directory. Then specify the new target package using the -T option (note that quotes may be required when using this option) as well as specifying -C TTY356 - e.g:

cd demos
catalina hello_world.c -lci -T "C:\Program Files (x86)\Catalina_6.2\target_6.2.1"  -C TTY256

Unzip it in the main Catalina directory, and specify the new target package using the CATALINA_TARGET environment variable (note that no quotes should be used when setting the environment variable) as well as specifying -C TTY256 - e.g:

cd demos
set CATALINA_TARGET=C:\Program Files (x86)\Catalina_6.2\target_6.2.1
catalina hello_world.c -lci -C TTY256

If you are ever in doubt about which target package is being used and which plugins are being loaded, add the -v option to the catalina command.

Note that just as the -C TTY HMI option is separate to the -ltty library option, the -C TTY256 HMI option is separate to the -ltty256 library option. You can only use one of the four options without needing additional configuration work.

This CTP will become standard Propeller 1 target package from the next release onwards.

Ross.

RossH · 2023-10-26 07:16

Well, it seems I broke payload's internal terminal emulator in the 6.2 release (the external VT100 compatible terminal emulator was fine), so I thought I'd better bring forward the next release.

Catalina 6.3 has been released here.

Apart from fixing payload and formally including the new Propeller 1 TTY256 HMI option, the main addition in this release is support for the Parallax hardware RTC add-on. This is supported on the Propeller 2 only.

The RTC support includes:

C versions of the Spin 2 I2C and RTC drivers;
a C version of the Spin 2 RTC example program;
a new Catalyst time utility that can set or display the time, date and other parameters;
an updated clock plugin that supports both the old software-only CLOCK as well as a new hardware RTC option;
all Catalyst utilities and the stdio SD card file system functions now set the file creation and modification dates and times correctly if either the hardware RTC is installed, or the software clock is used and the time is set.

The pre-built Propeller 2 demos included in the release assume the hardware RTC is installed on base pin 24 - if it is not detected, then the software clock will be used. If the hardware RTC is installed on a different base pin, you will need to recompile Catalyst. To enable the hardware RTC in a C program you can just use the Catalina symbol RTC wherever you previously used the symbol CLOCK (e.g. on the catalina command line). So to compile the old ex_time.c program to use the hardware RTC you might use commands like:

cd demos\examples
catalina -p2 ex_time.c -lci -C VT100 -C RTC
payload -i ex_time

Here is the relevant extract from the README.TXT:

RELEASE 6.3

New Functionality
-----------------

1. A new Catalina HMI option has been implemented on the Propeller 1, which 
   can be selected by defining the Catalina symbol TTY256 (e.g. by including
   -C TTY556 on the Catalina command line). This HMI option is applicable to 
   all supported Propeller 1 platforms. It can be specified wherever TTY 
   could be specified. It tells Catalina to use the serial interface plugin 
   with 256 byte transmit and receive buffers instead of the normal one, 
   which uses 16 byte buffers. Note that using this option instead of TTY
   can improve serial performance and reduce the chance of lost bytes, but 
   it also increases program sizes by around 500 bytes.

2. Catalina now supports the Parallax Real-Time Clock module on the Propeller
   2. This support is implemented in the existing software clock plugin, and 
   is enabled by specifying the Catalina symbol RTC instead of CLOCK. Note 
   that if RTC is specified, the clock plugin will always use a separate cog,
   whereas previously if both the clock and the SD card plugin were in use 
   they could share the same cog. This means that programs that use the RTC 
   hardware clock may require one more cog than programs that use only the 
   software clock.

   While the clock plugin accepts a new time being set, the new time is 
   still only set in the plugin - i.e. it only applies until the Propeller 
   is reset. To actually change the RTC hardware time, separate software 
   must be used. This is both for backwards compatibility, and because the
   time setting software is too large to fit in the clock plugin. Instead,
   stand-alone RTC/I2C drivers (in files demos\catalyst\time\rtc_driver.c and 
   demos\catalyst\time\i2c_driver.c) are provided to do this, and a Catalyst 
   utility program called "time" is also provided. This utility allows setting
   the RTC hardware time as well as various associated RTC parameters from 
   the Catalyst command line. A C version of the Parallax RTC example 
   program is also provided (in file demos\catalyst\time\rtc_example.c).

3. Catalina's stdio file functions now set the file creation and modification 
   date and time if either the RTC hardware clock is in use, or the software
   clock is in use and the time has been set (to the year 2000 or later). 
   If not, the default DOSFS time of 1/1/2006 01:01:00 is used. Note that 
   the file access date is never modified after file creation, because 
   modifying this on every file access would slow down the file system 
   too much.

4. Catalyst's self-hosted version of Catalina now prints the time the 
   compilation starts and finishes if very verbose mode is specified (by 
   adding the command line options -v -v).

5. The inter-character delay introduced in the payload terminal emulator
   in version 6.2 was set at a fixed 100 ms, but this value caused problems 
   in some cases, so it has been reduced to 25 ms by default, and a new 
   command-line option (-K) has been introduced to set it. 

   A better solution for Propeller 1 programs that have trouble keeping up 
   with the shorter inter-character delay is to use the TTY256 HMI option 
   instead of the TTY HMI option.

Other Changes
-------------

1. The Catalyst ls utility was not printing the file times correctly when
   the very long output option was specified (by adding the command line
   options -ll or -l -l) . Affected both the Propeller 1 and 2.

2. The stdio file system functions were not setting the file dates correctly
   when creating files. Affected both the Propeller 1 and 2.

3. When building Catalyst on the Propeller 1, there are more cases when the
   option EEPROM_CATALYST will need to be included, such as if the new
   TTY256 HMI option is used. See the README.TXT file in demos\catalyst 
   folder for more details.

4. The _pinclear() function was not working correctly. Affected the Propeller
   2 only.

5. When the VGA HMI option was in use, the self-hosted version of Catalina 
   did not have enough cogs to run the rcc component. Also, the catalina 
   command would prompt to continue at several points during the compilation.
   Affected the Propeller 2 only.

EDIT: Files now updated.

RossH · 2023-10-26 10:52

D'oh! That was stupid of me! I've just realized that I've put the RTC driver on the same default pin as the VGA driver in release 6.3 (i.e. pin 0)!

That means that the pre-compiled VGA demos of Catalyst (i.e. P2_EVAL_VGA.ZIP and P2_EDGE_VGA.ZIP) won't work.

The pre-compiled serial versions of Catalyst (i.e. P2_DEMO.ZIP, P2_EVAL.ZIP and P2_EDGE.ZIP) are ok.

Too late to fix tonight - I'll recompile and repackage release 6.3 tomorrow!

Ross.

EDIT: Files now updated.

RossH · 2023-10-28 05:19

Apologies for the delay in re-releasing 6.3. Fixing the pin conflict was straightforward - the RTC now uses pins 24 & 25 by default. However, I immediately ran into another issue - when the VGA HMI options are used (which use more cogs than the serial HMI options) then there is often not enough cogs to also use the RTC plugin, which requires an extra cog. Also, I clearly only ever tested the self-hosted version of Catalina using the serial HMI, because it had multiple issues when VGA was used instead - now fixed.

I have done a bit of re-thinking about using the hardware RTC. Instead of just replacing the software CLOCK plugin with the new RTC plugin (which also supports the software clock), programs should continue to use the CLOCK plugin wherever possible - if they need real-time (most programs do not) then they can set the software clock once (typically on startup) from the hardware RTC using the stand-alone I2C/RTC drivers. In most cases, this will provide a sufficiently accurate real-time clock. It makes the programs larger but it means it does not require an extra cog.

I have updated one of the RTC demo programs (demos\catalyst\time\ex_time.c) to provide an example of doing this.

If you downloaded the earlier version of release 6.3, please replace it with the new one.

Ross.

RossH · 2023-10-28 07:30

Now, here's a thing ...

Recompiling all the Catalyst demo programs (which of course includes Catalina itself) from source takes about 3 minutes on Linux on my machine.

But it now takes nearly 20 minutes to do the same job on Windows 10 on the same machine.

But before you say "Wow! how bad is Windows?" things are actually much worse than that - my Linux compiles are done in a Linux virtual machine running under Windows - yet they are still many, many times faster than doing the same job under Windows.

So I have a question - has anyone recompiled Catalina from source under Windows 11? If so, is Windows 11 any faster than Windows 10? I always resist upgrading because so many things break with each new upgrade (and yes, I do get the irony here !), but this is getting downright ridiculous.

RossH · 2023-11-02 06:57

I've had a bug report on Catalina. If you use the _register_plugin() or _unregister_plugin() functions but no other registry functions, you get an error that the symbol C__registry is undefined.

The reason for the error is that these functions are missing the appropriate "import" statement (because they were hand written in PASM). In most cases this doesn't matter because the _registry() function ends up being imported by some other function. But just in case there are others I have also missed, a simple solution is to explicitly reference the necessary function yourself, which will cause it to be imported.

For instance, here is a demo program that exhibits the problem, and also contains a simple solution:

#include <plugin.h>
void main() {
   _registry(); // this program will not compile without this line!
   _register_plugin(0, 0);
}

This is just a temporary workaround - it will be fixed properly in the next release.

The next release will also add environment variable support to both Catalina and Catalyst. You will be able to set variables on the Catalyst command line, just as you can in Linux or Windows, and they will be permanently remembered and automatically made available to all programs started by Catalyst, where they can be accessed from C using the function getenv() or from Lua using os.getenv(). This proved to be the easiest way to complete the Real-Time Clock support, because it allows you to specify the local time zone via the most common C mechanism (i.e. specifying it in the TZ environment variable). Prior to this, the time zone would always just default to UTC.

Environment variables will also improve the self-hosted version of Catalina, because it allows common platform defaults to be permanently stored in environment variables (such as CATALINA_DEFINE) just as you currently can under Linux or Windows. This means you no longer have to specify them all in every Catalina command. It also reduces the problem of Catalyst only supporting a limited number of command line arguments.

Like the the Real-Time Clock itself and the self-hosting capability, environment variables will be supported on the Propeller 2 only.

RossH · 2023-11-10 12:06

Catalina 6.4 has been released here.

The main purpose of this release is to add support for environment variables to both Catalina & Catalyst. Currently, these are supported on the Propeller 2 only, since I needed them to finish off some outstanding Propeller 2 functionality. This functionality was:

Adding time zone support for the RTC clock (which requires the ability to set the TZ environment variable).
Adding support for the CATALINA_DEFINE environment variable to the self-hosted version of Catalina, so that all the Catalina options no longer need to be passed on the command line (which quite quickly ran up against the Catalyst limit of 24 command line arguments).
Adding error detection to the self-hosted version of Catalina (so that errors detected during a long compilation process can terminate the compilation). This is done by having the various sub-processes set an _EXIT_CODE environment variable, and being able to detect this in the Catalyst script.
In the process of implementing the above, I also added the ability to pass parameters to Catalyst scripts, and these scripts also now support a very primitive type of looping using environment variables (a word of warning - such loops execute reeeaaaally sloooowly! - you would almost always be far better using off using a Lua script to do such things, but since I essentially got these capabilities for nothing, why not? ** )

** As the proverb says: "The marvel is not that the bear dances well, but that the bear dances at all."

Here is the relevant extract from the README.TXT:

RELEASE 6.4

New Functionality
-----------------

1. Catalina and Catalyst now both support environment variables on the 
   Propeller 2. Previously, the C "getenv()" function would only return 
   hardcoded values for specific environment variables that needed to be 
   present to meet the ANSI C standard, and this is still the case on the
   Propeller 1. But proper support has been added for setting and fetching
   environment variables from C programs, Lua programs and on the Catalyst
   command line when using the Propeller 2 and the Catalyst program loader.

   There are two new functions provided (defined in stdlib.h) that affect
   environment variables:

      int setenv(char *name, char *value, int overwrite);
      int unsetenv(char *name);

   These work as described in Linux (e.g. see "man setenv"), but with the 
   limitations described in the notes below.

   Two corresponding functions have been added to the Lua "propeller"
   module:

      propeller.setenv(name, value, overwrite)
      propeller.unsetenv(name)

   The corresponding "getenv(name)" function already existed in Lua, but 
   it is in the "os" module:

      os.getenv(name)

   See also the description of the new Catalyst "set" command, below.

   Notes: 

   - Environment variables are stored in the file CATALYST.ENV in the
     root directory of the SD Card. This is a text file that can be edited
     with any text editor (such as vi), but unless the format of the file 
     is correct, the results of getenv() and setenv() are not defined. Only
     the first 2048 characters of this file are used. If the definition of
     an environment variable goes beyond that limit, it will be truncated.

   - On the Propeller 2, the CogStore plugin (used by the Catalyst program 
     loader to implement command-line arguments) now also supports storing 
     up to 2048 characters in the LUT. This is used to pass the environment
     to programs started by Catalyst, and this is why environment variables 
     are limited to a total of 2048 characters.

   - To be able to retrieve the value of an environment variable, a C program 
     must be started using the Catalyst program loader. However, the program
     does not need to be compiled with the extended C library, since the 
     environment is passed using CogStore. However, to SET an environment 
     variable, a C program must be compiled with the extended C library (i.e. 
     -lcx or -lcix).

   - Setting an environment variable does not affect the current environment.
     The variable is set by writing to the CATALYST.ENV file, and will be 
     available to C or Lua programs subsequently loaded by Catalyst.

   Environment variables are currently supported on the Propeller 2 only.

2. A new Catalyst "set" command has been added. This command is supported on
   the Propeller 2 only. It allows environment variables to be displayed or
   set from the Catalyst command line, or in Catalyst scripts. The syntax
   of the command is:

   set [options] [ NAME [ = [ VALUE ] ] ... ]
      to display all variables, do not specify a name
      to display specific variables, specify names
      to unset specific variable, specify -u or name=

   options:
      -? or -h  print a help message
      -d        print diagnostics
      -u        unset the specified variable

   Values that contain spaces must be quoted. More than one value can be
   set on the command line. The names of environment variables must start
   with a letter, number or underscore, and cannot contain the character '='.

   For example:
      set
      set A = 1 B = 2
      set C="a string value"
      set A B C
      set -u _EXIT_CODE
      set _0=

3. A new Catalyst "if" command has been added. This command is supported on
   the Propeller 2 only. It allows testing the value of environment variables
   and performing actions based on the result. It is primarily intended to be
   used in Catalyst scripts. 

    The syntax of the command is:

      if <name> [ <action> [ <param> ] ]
    or
      if <name> <unary_op> [ <action> [ <param> ] ]
    or
      if <name> <binary_op> <value> [ <action> [ <param> ] ]

   where:

     <name>      is the name of an environment variable. If only a name is
                 specified, the script exits if the variable does not exist

     <unary_op>  can be "exists" or "missing" (if no operator is specified,
                 "exists" is assumed).

     <binary_op> can be "equals", =, ==, !=, <>, ~=, <, <=, =<, >, >=. =>
                 (note Lua promotes string values to integers if used in an
                 arithmetic expression - so the string "1" is the same as 1)

     <value>     can be a string or an integer (note that strings that 
                 contain spaces must be quoted).

     <action>    can be "continue", "exit", "skip", "echo", "add", "sub", 
                 "set", "assign" or "prompt" (if no action is specified, 
                 "continue" is assumed). 

     <param>     for "skip" this is the number of lines to skip when true 
                 (if no parameter is specified then 1 is assumed).

                 for "echo" this is a value to echo (if no parameter is 
                  specified then the value of the variable is echoed).

                 for "add" and "sub" this is the amount to add or subtract
                 from the variable (a variable that does not exist or which 
                 has a value that cannot be interpreted as an integer has 
                 value 0).

                 for "set" this is the value to set in the variable (if no
                 value is specified the variable is deleted).

                 for "assign" it is the name of another environment variable
                 to assign to the variable.

                 for "prompt" it is the string prompt to display (if a value 
                 is entered it is stored in the variable, otherwise the
                 variable is deleted). If no string is specified the name
                 of the environment variable is used.

   For example:

      if A                    -- exits if variable A doesn't exist
      if A continue           --   ditto
      if A exists             --   ditto
      if A missing exit       --   ditto
      if A missing            -- exits if variable A exists
      if A exists exit        --   ditto
      if A skip               -- skips one line if variable A exists
      if A missing skip 2     -- skips two line if variable A doesn't exist
      if A = 1                -- exits if value of variable A is not 1
      if A = 1 exit           -- exits if value of A is 1
      if A = "2" skip 3       -- skips three lines if value of A is 2
      if A != "a string" exit -- exits if value of A is not "a string"
      if A echo               -- echo the value of A if A exists
      if A missing echo "No!" -- echo "No" if A does not exist
      if A > 0 sub 1          -- subtract 1 to A if value of A is > 0
      if A add 1              -- add 1 to A if it exists
      if A missing prompt ">" -- prompt for value for A with ">" if A missing
      if A exists prompt      -- prompt for value for A with "A=" if A exists
      if A missing set "ok"   -- set A to "ok" if A does not exist
      if A set                -- set A to null (i.e. unset it) if A exists
      if A missing assign _1  -- set A to the value of _1 if A is missing

   Note that at least one space must separate each argument, so:

      if A = 1                -- will continue if A equals 1
      if A=1                  -- will generate an error and exit
      if A= 1                 --   ditto
      if A =1                 --   ditto

   Note that case is significant in string values but not in the variable 
   names, operations or actions.

   Note that although only one condition can be specified in each "if" 
   command, logical AND or OR of multiple conditions can be achieved 
   using multiple "if" commands and the "skip" action.

   For instance to exit if (A = 1) OR (B = 2) OR (C = 3):
      if A = 1 exit
      if B = 2 exit
      if C = 3 exit

   To exit if (A = 1) AND (B = 2) AND (C = 3):
      if A != 1 skip 2
      if B != 2 skip 1
      if C == 3 exit

   Note that loops can be created using the "if" command with the "skip" 
   and "add" or "sub" actions. For instance, put the following in a file 
   called "loop" (this file is provided as part of the pre-build demos):

      @# use argument 1 if provided, otherwise prompt
      @if _LOOP missing assign _1
      @if _LOOP missing prompt "Number of times to execute? "
      @# do loopy stuff here ...
      @if _LOOP echo
      @# ... end loopy stuff
      @if _LOOP sub 1
      @if _LOOP = 0 skip 2
      @# re-execute this script to loop (note use of "_0"!)
      @exec _0
      @if _LOOP > 0 skip 2
      @# clean up after ourselves
      @set _LOOP=

   Then execute the script using the "exec" command. For example:

      exec loop
   or
      exec loop 10

   See the description of the "exec" command (below) for more details.


4. A new Catalyst "exec" command has been added. This command is supported on
   the Propeller 2 only. It allows executing a Catalyst script from the
   command line or from a Catalyst script and passing parameters to it.

   The format of the command is:

      exec script [ parameters ... ]

   The "exec" command works by passing the parameters to the script using 
   environment variables. The Catalyst script name is passed in "_0", and 
   up to 22 parameters are passed in "_1" to "_22".

   For example, the following command executes the Catalyst script in 
   the file "script" and passes it the value "script" in variable "_0",
   the value 3 in variable "_1" and the value "my string" in variable "_2":

      exec script 3 "my string"

   Note that the "exec" command does PARAMETER SUBSTITUTION - any arguments
   passed to it of the form "_0" .. "_22" are substituted with the current
   value of the relevant environment variable. Note this is done BEFORE the 
   new values are set in these variables. So an example of using exec in a 
   Catalysts script might be:

      exec _0 _1 "a different string"

   This would re-execute the currently executing script (whose name is in
   variable "_0"), passing it the same first parameter as it was passed,
   but passing "a different string" in place of the second parameter.

   Note that the exec command passes control to the new script - i.e. it
   is not a "call" statement, has no stack, and never returns.

   For a working example, see the loop example in the description of the
   new "if" statement.

5. The Catalyst "time" command has been updated to know about time zones. 
   Previously the time displayed was always the time in the Real-time clock,
   and the time zone (as identified in the TZ environment variable) was
   hardcoded to "GMT", which meant no offsets were applied to that time.
   This can still be done if (for example) the time set in the RTC is
   the correct local time. However, now the RTC can be set to the correct
   UTC time and the TZ environment variable can be used to adjust this to 
   the correct local time. The format and use of the TZ environment variable
   is described in:

      https://www.gnu.org/software/libc/manual/html_node/TZ-Variable.html

   To set the time to one of the time zones hardcoded in the library file 
   "source/lib/time/misc.c" just specify the 3 or more character time zone 
   name. For example:

      set TZ=AEST

   To set the time zone to any time zone but WITHOUT corrections for daylight
   savings, give the time zone name and the offset required to convert that 
   time zone to UTC. For example:

      set TZ=AEST-10
   or
      set TZ=AEST-10:00

   To set the time zone to any time zone WITH corrections for daylight 
   savings, specify a fully qualified TZ. For example:

      set TZ=AEST-10AEDT,M10.1.0/2,M4.1.0/2

   The "time" command with no options, now prints local time in unix format. 
   New options have been added to the "time" command to display the UTC time 
   in either the 12 or 24 hour format used by the Parallax RTC driver code.

      time [options] [ DD/MM/YY ] [ HH:MM:SS [ AM | PM ] ]

    options:
              -? or -h  print this helpful message and exit
              -b mode   set battery mode (0 or 1) and report
              -c        calibrate on RTC start
              -r        software reset the RTC
              -d        prompt for new date
              -t        prompt for new time
              -u        print UTC time in 12 hour format
              -w        print UTC time in 24 hour format
              -v        verbose mode

   The "time" command is supported on the Propeller 2 only.

6. The precompiled demo versions of Catalyst for the P2_EVAL and P2_EDGE 
   (i.e. P2_EVAL.ZIP, P2_EVAL_VGA.ZIP, P2_EDGE.ZIP and P2_EDGE_VGA.ZIP)
   now all assume a Parallax RTC add-on card is installed on pin 24.
   However, if one is not, then apart from the "time" command, which will
   report that an RTC is not detected, everything should work correctly.

   Note that the pre-build version in P2_DEMO.ZIP does NOT assume an
   RTC is present, and still uses only the software clock.

7. The self-hosted version of Catalina (supported on the Propeller 2 only)
   now uses environment variables, such as CATALINA_DEFINE, just as it does
   on Windows or Linux. This effectively removes the limitations of Catalyst
   only allowing 24 command line arguments. If the catalina command detects
   that this would happen, it now prints a message suggesting the use of
   the CATALINA_DEFINE environment variable instead of passing parameters 
   on the command line.

   The pre-built versions of Catalyst now set the necessary environment
   variables to the correct defaults. 

   Catalyst now includes a new "cat_env" command that does the same job
   as the "catalyst_env" command does on Windows or Linux. For example, 
   in the case of the pre-built version in P2_EDGE.ZIP it would display:

      CATALINA_DEFINE   = P2_EDGE SIMPLE VT100 CR_ON_LF USE_COLOR
      CATALINA_INCLUDE  = /include
      CATALINA_LIBRARY  = /target
      CATALINA_TARGET   = /lib
      CATALINA_TEMPDIR  = /tmp
      LCCDIR            = [Default]

8. The Catalina sub-processes in the self-hosted version of Catalina 
   (i.e. cpp, rcc, bcc, spp, pstrip, p2asm) now set the environment variable 
   _EXEC_CODE to 0 on success, or 1 on failure. This allows errors in the
   compilation to be detected and the compilation process to be terminated
   without waiting for it to complete (which can take a long time!).

   To do this, the "catalina" command now adds the following line to the
   script it generates that invokes each sub-process:

      if _EXIT_CODE != 0 exit

   Refer to the description of the new "if" command for details.

Other Changes
-------------

1. A problem in the "_register_plugin()" and "_unregister_plugin()" library
   functions which could result in the error message that the symbol
   "C__registry" was not defined has been fixed. Affected the Propeller 1
   and Propeller 2.

2. The library file misc_ytab.c had a name incompatible with the DOS 8.3
   character file names, and has been renamed to misc_yt.c. Affected the
   self-hosted version of Catalina on the Propeller 2 only.

3. The Lua functions propeller.getpin() and propeller.setpin() were not
   validating their arguments correctly. Affected the Propeller 1 and
   Propeller 2.

4. An error in the RTC driver "rtc_time_24()" function has been fixed.
   Affected the Propeller 2 only.

RossH · 2023-11-13 07:49

This was supposed to be in the 6.4 release, but because of a silly oversight of mine it didn't make it. It is not worth a new release, being entirely cosmetic.

Now that you can set environment variables in Catalyst (on the Propeller 2 only) you were supposed to be able to set the Catalyst prompt from the command-line. Overwrite the Catalyst source code (in demos\catalyst\core) with the version attached and recompile Catalyst from source. Then you can set the prompt (up to 20 characters) from the Catalyst command line.

For example:

set prompt="Catalyst> "

This will be formally included in the next release.

Ross.

RossH · 2023-11-15 22:21

Another bug report - Catalina's version of the Geany IDE crashes if there is no current project and you open a file (e.g. using the File->Open ... menu entry or pressing the Open button)

The workaround is simple - if there is no current project (e.g. left open from the previous Geany session) then open one using Project->Open ... or create one using Project->New ... before opening any files.

I will fix this in the next release.

Ross.

RossH · 2023-11-15 23:52

Blast! And another one ...

The Linux version of Catalina has some script files that are in DOS format instead of Unix format. The bash shell doesn't like that. Before building Catalina under Linux, execute the following commands:

cd catalina/source
find . -name build_all -exec dos2unix {} \;

This will be fixed in the next release.

RossH · 2023-11-16 23:41

Catalina 6.5 has been released here.

Apologies for the recent spate of releases with bugs, and thanks to those who reported them. This release addresses all the ones reported, and a few others I found myself while investigating. Many of these bugs are the consequences of the recent migration on Windows from the old MINGW gcc compiler to the current MSYS2 gcc compiler, and affected Windows only. Fingers crossed that this release fixes the last of them, because re-building Catalina on Windows is quite a complex and time-consuming process (Catalina users don't have to worry about that, but I do, and it's not a lot of fun!).

Here is the relevant extract from the README.TXT:

RELEASE 6.5

New Functionality
-----------------

1.  Catalyst now uses the value of the PROMPT environment variable as the 
    prompt if it exists (Propeller 2 only). If not, the default prompt of "> "
    is used. The prompt can be up to 20 characters, and can be set from the 
    Catalyst command line using the "set" command. For example:

       set prompt="Catalyst> "

    Environment variables are supported on the Propeller 2 only.

2.  Catalyst now uses the LUA and LUAX environment variables to determine
    the executable to use to execute normal and compiled Lua scripts on the 
    Propeller 2. These environment variables can be set on the command line
    using the "set" command. If not set, then "lua" and "luax" are used by 
    default to execute files with a .lua or .lux extension (respectively).

    For example, to execute ".lua" files using the XMM LARGE version of Lua,
    and ".lux" files using the Multi-threading version of Lua:

       set LUA=xl_lua
       set LUAX=mluax

    Environment variables are supported on the Propeller 2 only.

3.  The value of the LUA_INIT environment variable is now used by Lua on the
    Propeller 2 to execute Lua commands before entering interactive mode. 
    This environment variable can contain either a Lua statement to execute, 
    or the value "@filename" to indicate that the statements should be read 
    from a file with name "filename". This environment variable can be set on 
    the command line using the "set" command. For example:

       set LUA_INIT="_PROMPT='Lua> '"
    or
       set LUA_INIT="@LUA_INIT"

    Environment variables are supported on the Propeller 2 only.

4.  The lua scripts list.lua, find.lua, freq.lua and wild.lua have been
    updated to be more consistent with other Catalyst commands. 
    Primarily minor cosmetic improvements but note that the order of 
    parameters of the find script has been reversed, and that a command
    can now be specified for the wild script to use. The various syntaxes 
    are now:

       list [ file_spec ]
       freq [ pattern ] file_spec
       find pattern [ file_spec ] 
       wild [ command ] file_spec

5.  Payload now accepts '_' as well as ',' to separate the row and columns
    specified in the -g command-line option. For example, both the following
    will specify the same window geometry when an interactive terminal window
    is requested (i.e.80 columns and 40 rows):

       -g 80,40
       -g 80_40

   This change allows the -g option to be specified in the Loader field of
   the Project Properties in a Geany project file - previously it could not 
   because the comma caused Geany to misinterpret the options. To facilitate 
   this, the default Geany "Download and Interact" build command is now 
   "payloadi" instead of "payload". This means that the terminal window this 
   command opens will be sized correctly when geometry is specified using 
   an option of the form -gXXX_YYY.

Other Changes
-------------

1. Catalina's version of the Geany IDE crashed if there was no current project
   and a file was opened (e.g. using the File->Open ... menu entry or pressing
   the Open button). Affected Windows and Linux.

2. The Linux version of Catalina had some script files that were in DOS format
   instead of Unix format, which meant they could not be executed by the bash
   shell. Affected Linux only.

3. Some required DLLs were missing, which meant some executables (such as
   payload) may have reported missing DLLs if they could not be found on the
   user's path (e.g. if MSYS2 was not installed). Affected Windows only.

4. The script that starts the Windows Geany IDE now uses the Windows START
   command, which means the Geany window always starts visible - previously
   it would sometimes start the Geany window minimized. Affected Windows only.

5. The Basic demo ut-trek.bas did not execute correctly because Dumbo Basic
   was not correctly initializing array variables to zero. Also, there are
   some minor cosmetic changes in the program to improve the format of the
   program output. Affected the Propeller 1 and 2 on Windows and Linux.

RossH · 2023-11-18 03:09

Someone suggested some time ago that I make a video to showcase Catalina. At the time I didn't have the capability to do so, but I just found a neat-o (and free!) screen recording utility (the one that is built into Windows has never worked for me) so here is a first attempt:

An Introduction to the Catalina Geany IDE

This video was recorded at a resolution of 1024x768, and probably needs to be viewed full screen to read the text.

Happy to receive any comments or suggestions. Too long? Too short? Too dull?

If there is a general consensus that such videos are worthwhile, I will do a few to demonstrate various aspects of Catalina.

Ross.

RossH · 2023-11-19 01:18

Another short video, this time to showcase Catalyst ...

An introduction to Catalina Catalyst

That's probably enough for the moment

Ross.

RossH · 2023-11-19 07:41

Catalina C is now on Patreon since this seems like a good place to host videos, which I plan to make a few more of.

Also, as my signature now says - feel free to make a donation!

Ross.

RossH · 2023-11-20 04:25

Another short video - this time to showcase Catalina's Blackbox source level debugger ...

Introduction to the Catalina Blackbox debugger

Note that you need to install the Catalina 6.5.1 patch to reproduce the session in this video.

To set up for the Catalina Blackbox debugger session shown in the Blackbox 
video:

1. The AUX USB port on the P2 evaluation board is for power only. To add a 
   second USB port, you need to install a Prop Plug. For example. on pins 
   16 - 20 (i.e. Gnd, 16, 18, 20) of a P2 Evaluation board. You can use other
   pins, but make sure you also modify the pins specified in step 2 below.

2. Edit target/p2/P2EVAL.inc and find the following lines:

      #define _BLACKCAT_RX_PIN  63
      #define _BLACKCAT_TX_PIN  62

   Replace those lines with the following (or the pins you installed the 
   Prop Plug on in step 1):

      #define _BLACKCAT_RX_PIN  18 
      #define _BLACKCAT_TX_PIN  20

3. Connect BOTH serial ports (i.e. the standard P2 Evaluation board port on 
   pins 62 &63, and the one using the newly installed Prop Plug) to the PC.

Ross.

RossH · 2023-11-21 07:33

Well, some people are watching the videos

Just got a bug report about the Blackbox debugger. A memory alignment issue with structures containing short values. I see where the problem is and am testing a fix now. It won't need a full release - I will release a patch shortly.

Ross.

RossH · 2023-11-22 07:12

A patch for Catalina - 6.5.1 - has been released. It is available here. This patch release must be installed over Catalina 6.5.

This is a small patch which contains only the Blackbox debugger - if you don't use that then you don't need it. It is primarily a bug fix release, but while doing that I added a few minor but useful enhancements.

Here is the relevant extract from the README file:

RELEASE 6.5.1

New Functionality
-----------------

1. Added support for reading and writing XMM RAM on the Propeller 2, and for 
   specifying Propeller 2 specific memory models (e.g. on the command line 
   or using the xmm command).

2. Added support for (limited) indexing into arrays. The index must be the 
   last element specified, so 
      array[index]           is supported
      array[index].field     is NOT supported

3. Added support for (limited) use of the '->' operator on pointers. 
   The operator is only supported when the first element specified is a 
   pointer. This operator and can only be followed by a single field 
   (including optional index), so
      p->elt          is supported 
      p->elt[index]   is supported 
      p->elt1.elt2    is NOT supported.

4. Added support for (limited) use of the '*' operator when the first element
   specified is a pointer, so if p is a pointer then 
      *p              is supported               
   Also, note that '*' can be combined with '->' so if p and q are BOTH 
   pointers then
      *p->q           is supported 
   and means *(p->q)

5. Added support for variable names, &variable and &function being used to 
   specify the address in read and write commands.

Other Changes
-------------

1. The Blackbox debug cog was always resetting pins 62 & 63 even if it was 
   configured to use a different serial port. Affected Windows and Linux, 
   Propeller 2 only.

2. Fixed an issue that meant word-sized values (e.g. short or unsigned short) 
   in structures were not displayed correctly. Affected Windows and Linux, 
   Propeller 1 and Propeller 2.

3. Fixed an issue with determining the offset of a field within a nested 
   structure or union. Affected Windows and Linux, Propeller 1 and Propeller 2.

4. Fixed an issue that meant negative values could not be written using the
   write command. Affected Windows and Linux, Propeller 1 and Propeller 2.

5. Fixed an issue with the type search, which was aborting the search at the 
   first instance of "<array>" (indicating an anonymous array) instead of 
   reporting "not found" when searching for such types. This led to Blackbox
   being unable to determine the type of variables in some instances. 
   Affected Windows and Linux, Propeller 1 and Propeller 2.

While testing the changes, I struck a more significant issue, which is that LCC is not always generating the necessary stabs (symbol table information) needed by any debugger. Blackbox uses all the stabs LCC generates, but there is not much it can do when there are none. It tends to only affect complex cases such as nested structures that contain anonymous arrays, so it won't affect most programs. I decided not to hold up the patch while investigating further, since it is not a Blackbox issue, and digging deep into the innards of LCC could take me some time.

Anyway - real programmers know that debuggers are for wimps!

Ross.

RossH · 2023-11-26 07:00

Well, that was a bit of a marathon!

Took me only a few minutes to figure out where it was that the symbol table information (aka stabs) were not being emitted by the compiler, and not much more time to figure out how to make it emit them.

But Blackbox wouldn't process them, and I eventually figured out why they were not being emitted!

Basically, just laziness. Not mine - the original LCC authors. Emitting stabs that might contain forward references (e.g. structures that contain pointers to themselves, or to other types that were forward defined) had been turned off. Why? Because processing stabs guaranteed not to contain forward references is a doddle. But processing stabs that contain forward references is very much not a doddle. The former can be done in a single pass over the stabs. The latter requires two passes, plus a bit of of sleight-of-hand.

Still, now done. Catalina and Blackbox now both generate and process stabs with forward references. I've also added some other usability enhancements to Blackbox.

I'm in the middle of a few other things, and in any case I was supposed to be taking a break from Catalina because we are rapidly approaching our "busy" season, but I will issue another patch release as soon as I have done a bit more testing.

Ross.

RossH · 2023-11-27 11:14

Another patch for Catalina - 6.5.2 - has been released. It is available here. This patch release must be installed over Catalina 6.5 or 6.5.1.

This patch contains usability enhancements for the Blackbox debugger - if you don't use that then you don't need it.

Here is the relevant extract from the README file:

RELEASE 6.5.2

New Functionality
-----------------

1. Catalina now generates debugger information (i.e. .stabs) for structures 
   that contain pointers to themselves or to types that are forward defined,
   and Catdbgfilegen now knows how to process such .stabs.

2. Blackbox now has the following usability enhancements:

   - The printing of the hex values for 1 and 2 byte variables is now more 
     consistent - previously a value like -1 would have been printed as 
     "-1 (0xFFFFFFFF)" even if the type was a 1 or 2 byte variable. Now, 
     these will be printed as "-1 (0xFF)" or "-1 (0xFFFF)" respectively.

   - Values specified with a size that is not 4 bytes, such as "byte", "char",
     "word" or "short" now mask the value to the correct size, so "byte -1" 
     is 0xFF and not 0xFFFFFFFF, and "short -1" is now 0xFFFF and not
     0xFFFFFFFF

   - The printing of character values now always includes the decimal value 
     and the character value (if it is printable). For example, if variable 
     c is a char that contains the value 'x' then it will be printed as 
     "120 ('x' 0x78)". If it contains the non-printable character ESC then 
     it will be printed as "27 (0x1B)".

   - If the verbose mode is "on", then variables will be printed with type 
     and location information as well as their current value. For example, 
     if verbose mode is "off" the command "print i" might print:
        = 1 (0x00000001) 
     but if verbose mode is "on" it might print:
        LOCAL i int <size=4,align=4> @ register 23
        int @ register 23 = 1 (0x00000001)

   - Values used in commands such as read, write and update can now include 
     simple scalar variables or the address of a variable, so it is now 
     possible to say things like:
        update i = j
        update k = &i
        read i
        read xmm &i
        write hub &i = j

     However, note that the read and write commands ALWAYS read and write 
     4 bytes, even if the expression is a 1 or 2 byte value, and even if 
     the address being written is the address of a 1 or 2 byte variable.

   - It is now possible to use size specifiers to convert values, so the 
     following are all valid no matter what the type of variable x:
        update x = char 'a'
        update x = short 65535
        update x = float 1.234

   - The type of a variable was being shown as "<anon>" in some cases,
     but now either the correct type is printed or no type is printed.

Other Changes
-------------

1. Programs that use the new Real-Time Clock (RTC) plugin would not compile 
   if the -g -or -g3 options were also used (to use the Blackbox debugger)
   and the program was compiled in NATIVE or COMPACT mode.
   Affected both the Propeller 1 and Propeller 2 on Windows or Linux.

2. Payload previously detected if an extension was present by looking for "." 
   in the file name but this causes problems in Geany, which uses the full path 
   name (which may contain a "."). Also, payload now tries the base file name 
   BEFORE trying any extensions, which means it works correctly in Geany if 
   there are both .bin and .binary files in the project folder.

Ross.

RossH · 2023-11-27 23:50

Just a suggestion for those who use Catalina Geany ...

On Linux, when you open a Command Line window from within Geany it opens it in the current project directory, which is quite useful. But Windows opens it in your home directory, which is not as useful.

However, this is configurable - either just for the current project (via the Build->Set Build Commands menu item), or for all projects (via the filedefs.c file, which is in "%LCCDIR%\catalina_geany\data\filedefs" , and which may also appear in "%HOMEPATH%\AppData\Roaming\geany\filedefs" ).

Find the Catalina Command Line command (near the end of the file), and change it from:
cmd.exe /k "%LCCDIR%/use_catalina"
to:
cmd.exe /k "call "%LCCDIR%"\use_catalina && cd %p"

Ross.

currentc · 2023-12-07 20:37

I'm confused why I'm receiving the following errors. All interrupt code is written in C. There is zero pasm2 in my source files. Is this a compiler issue? Do I need to select different options?

bcc done, result = 0
assembling main
preprocess assemble src = /opt/catalina/target/p2/nmmdef.t
preprocess assemble dst = main
defined libc
defined libm
defined libint
defined P2PASM__
defined NATIVE
defined P2
preprocess command = spp -D libc -D libm -D libint -D P2PASM__ -D NATIVE -D P2  -I /opt/catalina/target/p2  -I . /opt/catalina/target/p2/nmmdef.t main.s
assemble command = p2asm -v33 main.s 
5877: ERROR: Invalid opcode "1"
   iret1
5902: ERROR: Invalid opcode "2"
   iret2
5927: ERROR: Invalid opcode "3"
   iret3

RossH · 2023-12-07 23:06

Hello @currentc

Many of Catalina's library functions are implemented in PASM. You have found a bug in Catalina's _clr_int_X() library functions (where X = 1,2, 3) which are trying to use iret1 (or 2, 3) as an instruction instead of reti1 (or 2, 3).

I will post a proper patch soon. In the meantime you can fix your Catalina installation yourself by editing the relevant files as follows (they are simple text files):

$LCCDIR/lib/p2/nmm/int/clr_int1.s     <- - change iret1 to reti1 on line 38
$LCCDIR/lib/p2/nmm/int/clr_int2.s     <- - change iret2 to reti2 on line 38
$LCCDIR/lib/p2/nmm/int/clr_int3.s     <- - change iret3 to reti3 on line 38

Ross.

Catalina - ANSI C and Lua for the Propeller 1 & 2

Comments