Catalina 3.5

RossH

All,

Catalina 3.6 has now been released. See here.

There is a new release of Catalina available (here). Catalina 3.5 offers significant improvements over previous releases of Catalina.It implements several LMM kernel enhancements, and also the new "service-oriented" registry structure, which simplifies the internals and makes it much easier to use the same objects from both C and Spin (examples are provided). There is also new functionality in the areas of file system support, stdio support, graphics support and serial device support.

Here's a summary of the major new features in Catalina 3.5:

• The default platform is now the CUSTOM platform, which comes preconfigured to be suitable for just about any Propeller with a 5Mhz clock. The default HMI option for this platform is serial communications (i.e. the PC HMI option), so the default platform is now suitable for nearly any propeller including the Parallax QuickStart board and various Gadget Gangster boards. The default platform is now preconfigured in all the target packages provided, and no longer requires any further editing in most cases.
• Payload now has an interactive mode (enabled via a new -i switch). When enabled, the Payload loader starts an interactive terminal emulator immediately after the program has been loaded. Programs compiled with the PC HMI option are therefore much easier to load and use, and no output will be lost while the user starts or enables their terminal emulator after downloading their program. The interactive mode implements a simple VT100 compatible terminal, so some ANSI escape code can be used in C programs (e.g. the vi text editor runs quite happily in the new terminal emulator). The Code::Blocks Tools menu has been updated to add a new corresponding tool entry called Download to Hub RAM and Interact which enables the new mode. Along with the previous change, this should help make the "out of the box" experience with Catalina much simpler for new users.
• The registry has been extended to become "service-oriented". Catalina reserves the first 64 "services" but the registry allows for 96, so that users can easily extend it to add their own services. A full set of registry manipulation functions provided for use from either Spin programs or C programs has been added. This has also allowed significant tidying up and simplification of various target files, and makes it much simpler to write Spin objects that can be used from any language. The new registry also incorporates automatic access management for plugins in multithread or multicog C programs.
• Added a new command line option (-C) for defining 'Catalina' symbols (i.e. symbols primarily intended to affect the configuration of the Spin target files). Previously this was done using -D, but this was confusing for some users, so -D has now reverted to being used for defining C language symbols - this usage of -D will be more familiar to many C programmers.
• Several minor enhancements have been made to the Catalina LMM Kernel which reduce the code size and improves the performance of all programs slightly. For programs that do not require floating point, the speed improvements in the ALTERNATE kernel are quite significant.
• A new function has been added to the SD Card file system to create directories. The Catalina Reference manual has been updated with more information about the various file system support options offered by Catalina.
• Added new library functions to access all of the Propeller special registers (frqa, frqb, ctra, ctrb, phsa, phsb, vscl, vcfg) and also fixed some of the existing functions (e.g. outa, outb, dira, dirb) to prevent toggling outputs while updating their value - thanks to Ted Stefanik for these contributions.
• Added a new library libtiny, based on Ted Stefanik's library. To use this library add -ltiny to your command line (in addition to the standard C library included by -lc).

For example:

catalina hello_world.c -lc -ltiny

The tiny library provides smaller versions of the stdio functions printf, scanf, sprintf and sscanf, which are generally compatible with their stdio equivalents but significantly smaller. Using these functions, the traditional "Hello, World" C program can now take as little as 1300 bytes of code space.

Note that libtiny also fixes an outstanding problem in the existing implementation of libc and libci - i.e. that they do not support sscanf and related functions - previously, for that you had to use the extended C library (which implements full streams support, but often makes programs too large to fit into the available Hub RAM). I may incorporate libtiny into the standard library in a future release, so that even the additon of -ltiny is not required.

• Added the 4 Port Serial plugin, which supports up to 4 serial ports using only a single cog, and a new library (libserial4) to access them. To use this new plugin, simply add -lserial4 to your command line.

For example:

catalina test_serial4.c -lc -lserial4

A section on serial support has been added to the Catalina Reference manual. To configure the serial ports, edit the Extras.spin file. The serial ports can be used in addition to any of the existing HMI options (including the serial PC HMI option).

• Added full mouse driver support to the C version of the Parallax graphics demo. This makes Catalina C graphics support 100% compatible with the Parallax Spin graphics.
• Many other small improvements and bug fixes - see the file README.WhatsNew (attached) for more details.

Note that I have decided not to release the new XMM Kernel just yet, as I still have a lot of testing yet to do on it. However, instead of forcing everyone to wait, I thought it was about time I released some of the recent improvements to the basic LMM kernel, and also to the Spin support infrastructure and C libraries.

Ross.

ERRATA:

1. It has just been pointed out to me that when you first install Catalina using the Windows "one touch" installer, the demo workspaces shown in the Catalina program group (in the Windows Start menu) do not start Code::Blocks - this is because until you have explicitly started Code::Blocks at least once (e.g. using the Catalina CodeBlocks menu entry) then the Code::Blocks file types are not associated with the Code::Blocks application. If you follow the QuickStart guide, you will do this as the first step - but if you instead start by trying to open one of the demo workspaces provided, they won't work. The solution is to just start Code::Blocks manually at least once - after that, everything will work as expected.
2. Payload's new interactive terminal mode does not work when an LMM program is loaded using the mouse port loader on the Hydra or Hybrid - the terminal is opened on the mouse port instead of the standard serial port. In this case, simply load the programs normally - i.e. without using the mouse port loader. The mouse port loader is only necessary for loading XMM programs, and these programs cannot use the standard serial port anyway because the pins conflict with the HX512 XMM board.

RossH

A small update to Catalina 3.5:

• One of the demo programs (test_plugin_names.c) won't compile because I changed the names of the plugin types in C to match the names I use in Spin, and I forgot to update this program which uses them. A new version is attached.

Sapieha

Hi RossH.

Nice work as usually.

prof_braino

How the heck do you manage to do all the testing on these releases? It must take days?

Do we have to remove or uninstall the old version, or does it automagically install over the the old one?

Rayman

Thanks Ross!

Cluso99

Nice work again Ross.
Where do you find the time??? I am always trying to find more time

RossH

prof_braino wrote: »

Do we have to remove or uninstall the old version, or does it automagically install over the the old one?

Good question!. In theory, with Catalina you never need to uninstall the old version - if you install the new version to the same directory the installer will correctly overwrite the old version, only preserving any changes you have made to the CUSTOM platform configuration (so you don't lose them just because you installed a new version).

However, in this release I have made changes to the CUSTOM platform support files that would probably make it necessary to modify them anyway - so I think there are two answers:

• If you have not made any changes to the CUSTOM platform, then I recommend uninstalling the old version first.
• If you have made changes to the CUSTOM platform, then I would recommend installing this new release of Catalina to a different directory (I install it to C:\Program Files\Catalina_3.5 and name the Start menu group as Catalina_3.5) until you have time to understand the changes, and add your CUSTOM target into the new version.

Ross.

RossH

Cluso99 wrote: »

Nice work again Ross.
Where do you find the time??? I am always trying to find more time

Yes, finding time is a real problem. It is also the main reason I decided not to release any significant XMM changes just yet.

In Catalina, the LMM kernel is the same for all platforms, but the XMM kernel is customized for each one - which makes testing a real nightmare. I originally did this to save space, but I have now decided that was probably a bad decision, so I'm working on a new basic XMM Kernel that is identical for every platform - but without going down the GCC path of using another cog. Unfortunately (and as usual!) I'm just a little short of space to make this work - so I seem to spend most of my time poring over the existing code trying to save an instruction here or there.

Ross.

prof_braino

RossH wrote: »

Yes, finding time is a real problem.

I don't know if you are interested in a possibly tangential project, but this might be of interest later on this year.

If you implement synchronous channels, (in the threads I called them CSP channels, this was incorrect, a better name would be GO-channels for now) in C, you might be able to take advantage of automated testing. That is, the PC runs a bunch of scripts that runs your test suite on the prop, and flags where something doesn't match expected output. You have to structure the tests for this, but the investment is usually worth it.

There's not much useful at present, just the first experiments, but we should have something interesting this summer. Just a heads up.

Retrobits

Thanks very much for the serial port library! I'll be using this right away.

- Earl

RossH

prof_braino wrote: »

I don't know if you are interested in a possibly tangential project, but this might be of interest later on this year.

If you implement synchronous channels, (in the threads I called them CSP channels, this was incorrect, a better name would be GO-channels for now) in C, you might be able to take advantage of automated testing. That is, the PC runs a bunch of scripts that runs your test suite on the prop, and flags where something doesn't match expected output. You have to structure the tests for this, but the investment is usually worth it.

There's not much useful at present, just the first experiments, but we should have something interesting this summer. Just a heads up.

Thanks, prof_braino. I have thought about automated testing, and I will certainly be going down this path in future - I have a lot of the basic stuff in place now that I should have had much earlier - but Catalina started out as a hobbyist project and I never realized just how big and complex it was going to get. The complexity blew out because I kept "bolting on" new stuff. Now that Catalina seems to have found its niche, my goal is to simplify the internals significantly with each new release - this will continue to reduce the code size and improve performance, and also help with the testing.

I will certainly have a look at Go. If anyone could take CSP and make it usable, it would have to be Ken Thompson!

Ross.

Cluso99

I installed Catalina 3.5 into C:\Catalina35
My previous vesrion was installed into C:\Catalina34

Of course 3.5 changed my %LCCDIR%. I spent a little time trying to recall the syntax for getting LCCDIR to be set to use C:\Catalina34
So here is my updated...
C:\Catalina\USE_CATALINA.BAT

 @echo off
echo.
echo ===================
echo SETTING UP CATALINA 3.4
echo ===================
echo.
SET LCCDIR=C:\Catalina34
if "%LCCDIR%" == "" goto default_lccdir
PATH=%LCCDIR%\bin;%PATH%
goto done
:default_lccdir
PATH=C:\Program Files\Catalina\bin;%PATH%
:done
call catalina_env.bat

BTW I renamed it U.BAT because its simpler to type. I have the same C.BAT and P.BAT for compile and payload when I am developing a c program.

I don't like command lines, so I started using CodeBlocks in Catalina 3.3 but somehow started using the command line for compiles and payload (downloading) in 3.4 and using PropTool for editting the .spin and .c programs. Only trap is to ensure you save the files you edit before you compile. While payload is running I switch to PropTerminal and as soon as payload completes I enable PropTerminal. So I am definately going to look at Ross's Terminal program... I am assuming it runs just like bst.

RossH

Cluso99 wrote: »

I installed Catalina 3.5 into C:\Catalina35
...

Hi Cluso,

Thanks for this - I think a lot of people will first install Catalina 3.5 alongside 3.4, and this technique works well for command line use - I use much the same technique myself, but I'm a confirmed "command-line" user.

For those who prefer an IDE, note that if you start Code::Blocks from the Start menu, it will use the setting of LCCDIR in your system environment variables, which (unless you told the installer otherwise) will be set to be whichever version of Catalina you installed last.

To swap between versions of Catalina when using Code::Blocks, you can either always start it from the command line, or create a little batch file that sets LCCDIR before starting it and make the Start menu entry execute that, or else manually set your default LCCDIR whenever you need to switch.

Ross.

RossH

All,

Just added an errata (to the first post in the thread) about the necessity for starting Code::Blocks manually at least once after a new Catalina install. The QuickStart guide takes you through the process, but if you do not follow the guide then you may not be able open the demo workspaces just by selecting the appropriate Start menu entries.

I'll see if I can fix this for the next release.

Ross.

RossH

All,

Just added another errata:

• Payload's new interactive terminal mode does not work when an LMM program is loaded using the mouse port program loader on the Hydra or Hybrid - the terminal is opened on the mouse port instead of the standard serial port. In this case, simply load the programs normally - i.e. without using the mouse port loader. The mouse port program loader is only necessary for loading XMM programs, and these programs cannot use the standard serial port anyway because the pins conflict with the HX512 XMM board.

I have a fix for this which I will include in the next release. If anyone has a Hydra or Hybrid with a HX512 (or uses the mouse port program loader on another platform) and would like the fixed version sooner, contact me by PM or email.

Ross.

RossH

All,

I just updated the Windows installer on SourceForge - there are no functional changes, but I fixed the Start Menu entries for the various Catalina and Catalyst documents. There is no need to reinstall, just create a folder called Catalina_Documents in your main Catalina folder, and move (or copy) all the PDF files to it - then the start menu entries will work correctly.

From now on, the documents will not appear in the main Catalina folder, but will have a folder of their own.

Ross.

jorozco

Hi Ross.
First of all thanks for the serial4 library addition on this release, very useful.
On the test_serial4 demo why does this work as expected:
> catalina -CNO_HMI test_serial4.c -lc -lserial4
But this does not:
> catalina -DNO_HMI test_serial4.c -lc -lserial4
The problem is that Code::Blocks uses the -D option but in this second case the compiler is ignoring it.
By the way, after some head scratching I was able to compile and run this demo successfully with Code::Blocks by adding -C NO_HMI to "other linker options ...", but I guess this workaround should not be necessary when the -DNO_HMI flag is already selected.

Juan Carlos

RossH

jorozco wrote: »

Hi Ross.
First of all thanks for the serial4 library addition on this release, very useful.
On the test_serial4 demo why does this work as expected:
> catalina -CNO_HMI test_serial4.c -lc -lserial4
But this does not:
> catalina -DNO_HMI test_serial4.c -lc -lserial4
The problem is that Code::Blocks uses the -D option but in this second case the compiler is ignoring it.
By the way, after some head scratching I was able to compile and run this demo successfully with Code::Blocks by adding -C NO_HMI to "other linker options ...", but I guess this workaround should not be necessary when the -DNO_HMI flag is already selected.

Juan Carlos

Hi Juan,

The latest version of Catalina (and Code::Blocks) no longer use -D for defining the Spin symbols that Catalina uses to configure targets. The -D option is now used for defining C symbols. This means using -D will no longer affect the compilation process as it did in prior Catalina releases. In Catalina 3.5 you use -C for this, so in this case you have to say -C NO_HMI, not -D NO_HMI.

Whether or not you have to specify NO_HMI at all depends on your Propeller platform. If the default HMI option for your propeller is PC (or PROPTERMINAL), then the pins used by the serial port will probably conflict with the pins used by the HMI option (i.e. both use pins 30 & 31 by default) - so you have to either say NO_HMI. or reconfigure one or the other (e.g. to reconfigure the pins used for the serial ports, edit Extras.spin). But if the default HMI is TV or VGA, you don't need to do anything.

As to the Code::Blocks problem, are you using the version of Code::Blocks that came with 3.5? If so, it should correctly add -C NO_HMI when compiling the test_serial4.c program. You should not need to add this manually.

Ross.

jorozco

Excellent Ross.
The problem was that I was using a separately installed Code::Blocks.
I also found that you already have a serial4 demo for codeblocks on Catalina: C:\Program Files (x86)\Catalina\codeblocks\serial4_demo
I agree that NO_HMI is not always required to use serial4 library as you explained.
Thanks.
Juan Carlos

RossH

All,

A very minor update to Catalina 3.5 - I forgot to update one of the demo programs (test_plugin_names.c). The version included with the release no longer compiles because I changed the names of the plugin types in C (to match the names I use in Spin). An updated version is attached to the second post in this thread - just copy it to the demos directory over the existing program.

Now for the good news ...

I have been working on a revamped binary image format for Catalina. The new image format reduces the memory footprint of all programs by a few hundred bytes, and also dramatically reduces the size of binary files on platforms that use Flash RAM. It is also much simpler to load, so simultaneously I have been rewriting and simplifying all the various load support modules, and also doing a few other things to enhance the speed of the Payload serial loader.

The outome of all this is that Payload now loads both Catalina programs and ordinary Spin programs faster (in some cases even faster than the standard Parallax loader). And on platforms with XMM, the load times for XMM programs are now about the same as for LMM programs - even on platforms that use Flash RAM for their XMM (such as the C3).

Another enhancement is that all XMM platforms now use the same debugger overlay - previously, each platform had to have its own overlay specially built in order to be able to use the Blackbox debugger. Given how many different platforms Catalina supports, this made my job of building and testing each release a real chore. Now it is much simpler to add new XMM types, and all new types will automatically get the full benefit of Catalina's Blackbox source level C debugger. As a result, I will start adding new XMM types to Catalina again (something I put on hold because it just became too onerous!)

Another enhancement is that Catalina now compiles all programs in the user's current directory rather than in its own directory. This makes it much easier to support Windows Vista and Windows 7 - e.g. it means you no longer need to battle with Microsoft's "VirualStore" nonsense, or give users administrator privileges just so they could compile C programs. It also means several users can be compiling programs at the same time, or the same user can be performing multiple compilations.

I'm in the final throes of testing all these changes, and the results are looking so good that I think I will release them soonish (probably as Catalina 3.6) rather than delaying them till the next round of Kernel improvments are ready.

Ross.

RossH

All,

Just another quick update on Catalina 3.6. While I was going back through the list of things people had requested in the past, I found a request for direct access to the Propeller special cog registers (similar to what the ICC C compiler did, and which I suppose the GCC one also does - I haven't checked).

Initially I resisted this since I thought it would break ANSI compliance, but I was wrong (as I often am!) - it turns out it is trivially easy to implement while maintaining full ANSI compliance should you need it. And I can see its advantages for people migrating to C from Spin, so I decided to just go ahead and do it.

In Catalina 3.6, all you will need to do to directly acces the special cog registers is declare them as "extern volatile" - e.g.:

extern volatile unsigned CNT;
extern volatile unsigned INA;
extern volatile unsigned OUTA;
extern volatile unsigned DIRA;

If these variable names are declared any other way (e.g. if INA were declared as a local) then they simply represent ordinary C variables and not the special cog register. This maintains full ANSI C compatibility should you need it. I wrap up the definitions of all 16 special registers in a new header file (<propeller.h>).

Here is an example Catalina 3.6 program that uses this method ...

#include <propeller.h>

#define BIT 0 // bit number to toggle

#define BITMASK (1<<BIT)

void main(void) {

   unsigned count  = CNT+CLKFREQ;

    DIRA |= BITMASK;
    OUTA &= BITMASK;

    while (1) {
      OUTA ^= BITMASK;
      WAITCNT(count, CLKFREQ);
    }
}

This C function currently compiles to 13 longs, whereas a PASM version takes around 8 longs, and a Spin version takes around 28 bytes (or 7 longs). A little tweaking of the Catalina code generator and/or optimizer will eventually reduce this gap further, but I'm keen to get 3.6 out, so this will probably wait for a later release.

Of course, the existing Catalina method of accessing the special cog registers (i.e. via functions like _dira() or _ina()) will continue to be supported.

Ross.

Rayman

That looks great Ross. Should make transition from Spin to C much more straight foreward for people...

RossH

Catalina 3.6 has now been released. See here.