Catalina 3.0
RossH
Posts: 5,519
UPDATE: Catalina 3.1 has now been released. See here for details.
Release 3.0 of Catalina is now available on SourceForge (here). Linux and Windows versions are both available.
Release 3.0 is a full release of Catalina. It contains new features, improvements and bug fixes. See the file README.WhatsNew (attached) for more details.
Here is a summary of the more significant changes since the previous release:
If you have a C3, I have just added a mini-tutorial at post #14 in this thread (here) which will help you get up and running quickly. The tutorial would be similar on Morpheus - just substitute MORPHEUS CPU_1 where you see C3 (or -D MORPHEUS -D CPU_1 where you see -D C3).
UPDATE: Catalina 3.1 has now been released. See here for details.
Here are the details on the last 3.0.X release ...
A new patch release (3.0.4) is now available (on SourceForge). This patch release must be installed over an existing Catalina 3.0 installation. This is a cumulative patch that contains all the bug fixes from previous patches. Here are the main changes:
Patch Release 3.0.4:
Release 3.0 of Catalina is now available on SourceForge (here). Linux and Windows versions are both available.
Release 3.0 is a full release of Catalina. It contains new features, improvements and bug fixes. See the file README.WhatsNew (attached) for more details.
Here is a summary of the more significant changes since the previous release:
- Full SPI Flash support on both the C3 and Morpheus (CPU #1) - this gives those platforms up to 1Mb of additional space for program code, with seamless integration of Flash support in both Catalyst and Payload (i.e. you use the Flash just like any other type of RAM).
- Full caching XMM RAM driver support on all platforms. The caching RAM driver can improve program performance by an order of magnitude when using serial memory as XMM RAM (such as on the C3). The caching driver can also improve program performance significantly on other platforms with slow parallel memory. Enabling the cache always costs an extra cog, but the Hub RAM it uses can be configured - from 1k to 8k.
- Improved command-line handling. All memory model, kernel and loader configuration is now done using only the symbols TINY, SMALL or LARGE (for selecting the memory model and kernel) and EEPROM, SDCARD or FLASH (for selecting the loader option). No more confusing options and magic numbers! However, the previous method is still supported if you have batch files or makefiles that depend on it.
- All HMI plugins now default to the behaviour defined in the ANSI C standard for character I/O. However, the previous behaviour can be selected at compile time if you have programs that depend on it. Also, it is now possible to specify either Linux or Windows line termination handling (i.e. LF vs CRLF). Previously, it could be difficult to run Catalina programs in a terminal emulator which was not itself configurable.
- Improved cog registration and registry handling. Free cogs can be identified at either load or run time, so targets can now make decisions about what plugins to load based on how many cogs are available. Also, all cogs used by Catalina are now correctly registered and can be identified at run time. This simplifies the process of dynamically loading and unloading plugins.
- Massive internal cleanup, simplification and streamlining of all target code. This should make it easier to add new plugins and drivers, and to support new platforms.
- Many documentation updates and documentation consistency improvements.
- Significant bug fixes:
- The SD Card driver initialization could fail on some platforms, causing strange behaviour.
- The Catalyst program loader could fail to pass command line arguments correctly when starting programs.
- The C functions setjmp and longjmp failed on some XMM programs.
- The Proxy keyboard driver was not being initialized correctly, which could lead to spurious characters.
- The Proxy HMI drivers were broken on the Morpheus platform.
UPDATE: Catalina 3.1 has now been released. See here for details.
Here are the details on the last 3.0.X release ...
A new patch release (3.0.4) is now available (on SourceForge). This patch release must be installed over an existing Catalina 3.0 installation. This is a cumulative patch that contains all the bug fixes from previous patches. Here are the main changes:
Patch Release 3.0.4:
- Added an enchanced spinc utility which allows the execution of Spin programs from within Catalina C programs.
- Various other small fixes and enhancements - see the attached README.WhatsNew for more details.
Comments
Want to know why Catalina takes so long to test? Just look closely at the command options - I have a script that recompiles all the various platform, memory model, kernel, loader and HMI options (and now the cache and flash options) just to make sure I haven't broken something. This script now compiles "Hello, World" over 10,000 times just to test all the supported variations. Of course, I can't possibly run all of these, or even a tiny fraction of them - but I try to actually run a representative sample. And that's without testing any of the really time-consuming things like the different library variants, the floating point options, the debuggger support or the multi-cpu features - not to mention that the tests have to be run on both Linux and Windows!
Sometimes I suspect I may be completely insane
Ross.
P.S. By popular demand (well, one demand!), here is a Payload Command Summary as well (click on the image for a larger version, or download the attached pdf):
Ross.
You're in the "champagne" zone, congratulations...
I hope you're right - after all this work I could certainly do with a drink or two!
I would characterize the 1.x releases as "prototypes", and the 2.x releases as "explorations".
I intend for the 3.x releases to be "consolidations". There will be little or no new functionality added for a while - I intend to concentrate solely on making what is already there work faster and better.
Ross.
A warning about Catalina 3.0 - this release has re-arranged the use of upper Hub RAM and moved the registry to a new location. I think you had the registry address hardcoded for some of your cogjects programs, so you will need to update your programs to work with this release.
Ross.
Your work is appreciated :-)
1MB code support on Morpheus CPU#1... yumm... that opens many possibilities...
PS to Bill -- yes I know I have a Morpheus, but Ross's example was for the C3.
@localroger,
No worries - I realize C is not the solution for everything. But if you do try it, let me know how you get on.
@tubular,
Yes, it's amazing what a bit of insomnia can do. But if I could, I'd much rather just sleep!
For those who download Catalina 3.0 specifically for playing around with the C3, here is a quick tutorial to get you up and running ...
First, I suggest you compile Catalyst, as this is the easiest way to try out a few things. Note that under Windows you need both MinGW and MSYS installed to build Catalyst and all the Catalyst demo programs. If you don't want to do this yet, skip straight to building the utilities folder and you can instead load a program serially (using Payload).
To build Catalyst, open a command shell, go to the Catalina main directory (e.g. C:\Program Files\Catalina) and enter the commands: This will build all the Catalyst binaries (to use the C3's TV display and keyboard by default) and put them all in catalyst\bin.
Note the parameters - C3 (obviously) then CACHED_4K to enable the cache and set its size to 4K, and FLASH to enable use of the SPI Flash chip as XMM RAM. All these parameters are required to run large XMM programs. You could run small ones without either cache or Flash support - but they would be slower, and also only use the 64kb SPI RAM, not the 1Mb SPI FLASH.
Then do two things:
1. Program Catalyst into the EEPROM of the C3. You can do this using Payload:
You should be presented with the Catalyst 3.0 prompt on the C3. Lets try a command that uses SPI Flash - what about the vi text editor?
Enter the following command on the C3: The screen will go blank for a few seconds while Catalyst programs the FLASH. Then vi should start and display the file CATALYST.TXT (the -s fmt=msdos parameter just tells vi the file uses MSDOS line termination).
Next, you should compile the utilities folder specifically for the C3 (we will need these utilities to load XMM programs with Payload). For simplicity, we will use the same options we used when compiling Catalyst - that way, we will end up with binaries that can either be loaded serially, or copied to the SD Card and loaded with Catalyst).
Enter the following commands (on your PC, not on the C3!): Next, you can go to the demos folder and compile a program. Let's try startrek.c. Again, we will (for convenience) use the same parameters (but note with catalina itself, you must specify -D before each one): Then you can load the result using Payload (for Catalina 3.0): Or (for Catalina 3.0.1): On the C3 you should see the opening screen of Super Star Trek. Note the first parameter must be XMM to first load the XMM program loader (i.e. XMM.binary, which we just built in the utilities folder). The -t 4000 parameter is required only if the 3.0.1 patch has not been applied - this tells payload to wait a bit longer than usual when loading, as the xmm loader needs to first erase the SPI Flash (which can take up to 14 seconds!). Catalina 3.0.1 does not erase the whole flash, it instead erases each 4k block just prior to programming it, so the longer timeout is not required.
Enjoy!
Ross.
P.S. Wanna see something really clever? Straight after loading startrek, go back to the utilities folder and recompile it using the following commands: Now you can load the Flash_Boot utility program. Now you can play the same game - but this time using the VGA display on the C3! You don't need to recompile or even reload startrek because it is already stored in the Flash. We can do this because of Catalina's Hardware Abstraction Layer. All the drivers (like the TV or VGA driver) are loaded separately to the program. If you wanted to, you can now program the flash_boot utility into EEPROM, and execute startrek from FLASH automatically each time you boot the C3.
Your quick-start is great and so is the Catalina options cheat-sheet. Can you do a payload cheat-sheet too?
These things are all in your manuals I'm sure, but the short/simple presentations are much more effective.
Thanks,
--Steve
Done! I have added it to the post with the Catalina Command Summary (here).
Ross.
Congratulations on the release of Catalina 3.0! I just installed it and will try using it later tonight. I have a couple of minor problems to report though. First, and this may be just because of the Cygwin version of unzip that I'm using, the files in the bin directory can't be executed even under the DOS command prompt until I run a "chmod +x" on them from the Cygwin shell. I'm not sure what the problem is with their protection bits but the chmod command seems to fix them. I couldn't figure out how to do the equivilent fix with the DOS attrib command.
Secondly, and this is *really* minor, typing "build_all" gives help information for the targets that can be built but you failed to add C3 and CACHED to that help text.
Anyway, as I said earlier, these are all minor issues. It looks like Catalina 3.0 will be a great product! Will there be an update to the optimizer to go with it?
Thanks,
David
Ross.
__attribute__ ((section(".hub"))) char buf[1024];Is there some equivilent in Catalina?thanks for this new release.
Every version is more feature rich yet more simplified in the internals, this will help supporting more hardware variants.
A few wishes (not requests!) for the forthcoming versions:
- switch to the single COG F32 object for the floating point support
- even in absence of RTC chip drivers, define the proxy protocol for them (thinking of a PC proxy server here...)
- (if appropriate) a target consisting of RamBlade+ProtoBoard seen as a dual CPU system
I was able to fool 2.9 to do the latter, by creating two modified multi-CPU sections for both RamBlade and ProtoBoard.
But I was wondering if some more clean solution is preferable.
Alessandro
Hi David,
No, there is no equivalent in Catalina. To guarantee a block of RAM is located in the Hub in all memory models, declare it as a local variable. For example, allocate the buffer locally to "main" and pass the address of the buffer to wherever it is needed. Or use a global variable and store the address of the buffer there.
For example:
// declare the size of the buffers #define BUFFER_SIZE 1000 // create global variables to point to the buffers we need static char *buff_addr_1 = 0; static char *buff_addr_2 = 0; // define a function that accepts the buffer addresses as parameters void function_1 (char *buffer_1, char *buffer_2) { // do something with the buffers here } // define a function that uses the globals to get the buffer addresses void function_2 () { char *buffer_1 = buff_addr_1; char *buffer_2 = buff_addr_2; // do something with the buffers here } void main() { // declare a buffer - because it is local to "main" it will be in hub RAM, // and it will persist until the whole program exits: char buffer_1[BUFFER_SIZE]; // For buffers at a fixed Hub RAM location, we can instead just set up // the address manually char *buffer_2 = (char *)0x7000; // set up the globals with the addresses of the buffers buff_addr_1 = buffer_1; buff_addr_2 = buffer_2; // call functions that use the buffers, using two different mechanisms: function_1(buffer_1, buffer_2); function_2(); }Hi Alessandro,
The last time I looked, the single cog F32 object was still under test. If it now gets the same answers as the existing Float32 objects in all cases, then I'll look at integrating it.
Catalina does not actually have an RTC chip driver - the RTC (CLOCK) object uses the system clock, not an RTC chip. This means it will work on all Propellers. An RTC chip driver is something I intended to add, but haven't yet. If I added that, I suppose it would make sense to add a proxy version of it as well.
If you send me your modifications to 2.9 to support your multi-CPU setup, I'll have a look at how to do the same thing in Catalina 3.0. But multi-CPU systems are very complex and time consuming to test, so I probably wouldn't add it to the standard target package unless it proved to be a popular combination. I would instead encourage you to develop an alternate target package for your platform. An alternate target package can be much simpler than the standard target package, because it only has to support one platform - not the nine or ten different ones supported in the standard target package. Also, an alternate target package wouldn't have to include all the different driver variants - it would only need to include those actually supported by the platform concerned. Note that I originally did have a different target package for each propeller platform - while it simplified each one, it made my job (of testing and updating all the target packages for each new release) an absolute nightmare! That's the main reason I decided in the end to provide Catalina with a consolidated target package - it was not intended to discourage others from generating them for new platforms!
Ross.
Okay, thanks for the suggestion. That will work for now. I think it would be better to allow the declaration decorations though since your approach depends on the fact that Catalina currently puts the stack and hence local variables in hub memory. If a new memory layout were to be defined that placed the stack in external memory this technique wouldn't work any more. Seems like it would be better to be explicit about where variables are to be allocated.
Hi David,
Yes, I considered that - I just don't yet have a good answer yet.
I really loathe the way that with some languages (C# is probably the worst offender) the code has to be littered with wierd attributes that really should be part of the language but aren't (I also don't know why C# calls them "decorations", since (a) they're not decorative at all, they're butt-ugly; and (b) in many cases they're not even optional - they're essential for the program to compile and run correctly!).
Ross.
I certainly agree that the GCC attribute syntax is ugly. I have no idea why it requires so many parens. Maybe it was designed by a Lisp lover (like me!). :-)
Here is how I solved it in ZOG. Pretty simple but it helps a little:
// in propeller.h #define HUB __attribute__ ((section(".hub"))) // in user code HUB char buf[1024];Jonathan
You have just thrown the whole enterprise into doubt:)
Did I say Excel? I meant....MAT(h)CAD-lab-cel-matica-tave-kipedia...or something.
Jonathan
Excellent:)
I'd have used python myself.
I was thinking of a standardized configuration, like DemoBoard+RamBlade or ProtoBoard in strict "DemoBoard wannabe mode" i.e. with VGA/PS2 kit and RamBlade, using 3 fixed pins from P0..P7 range.
But I think you're right, it's better to keep clutter out of distribution, cause there are just too many minor variations around. And after all it's a very simple hack to do, can be done in a few minutes if anyone like to try.
I should really try a custom target with separate dir... and I tried, but not hard enough: starting with the empty dir, and selecting stuff from the main target got me scared at some point.
Anyway...
This was my first attempt, and requires specifying DEMO for one cpu and RAMBLADE for the other during compilation:
{{excerpt from Catalina_Common.spin}} {...} #elseifdef DEMO '=============================================================================== ' ' Demo Board General definitions: ' '=============================================================================== KBD_PIN = 26 ' BASE PIN (Proto) MOUSE_PIN = 24 ' BASE PIN (Proto) TV_PIN = -1 ' BASE PIN (Proto) VGA_PIN = 16 ' BASE PIN (Proto) SD_DO_PIN = 12 ' PIN (Proto) SD_CLK_PIN = 13 ' PIN (Proto) SD_DI_PIN = 14 ' PIN (Proto) SD_CS_PIN = 15 ' PIN (Proto) I2C_PIN = 28 ' I2C Boot EEPROM SCL Pin I2C_DEV = $A0 ' I2C Boot EEPROM Device Address SI_PIN = 31 ' PIN (Proto) SO_PIN = 30 ' PIN (Proto) ' ' Demo Board Clock definitions: ' CLOCKMODE = xtal1 + pll16x ' (Proto) XTALFREQ = 6_250_000 ' (Proto) CLOCKFREQ = 100_000_000 ' (Proto) Nominal clock frequency (required by some drivers) ' ' Demo Board Loader and InterProp Comms definitions: ' SIO_BAUD = 115200 ' Baud rate to use for all interprop comms SIO_LOAD_MODE = %0000 ' SIO Mode used by targets during load (loader uses SIO_COMM_MODE) SIO_COMM_MODE = %0000 ' SIO Mode used by clients and servers for InteProp Comms #ifdef CPU_1 TX_PIN = 0 ' Server Tx Pin on CPU #1 (to CPU #2 Pin 23) RX_PIN = 1 ' Server Rx Pin on CPU #1 (to CPU #2 Pin 22) #else TX_PIN = 22 ' Server Tx Pin on CPU #2 (to CPU #1 Pin 0) RX_PIN = 23 ' Server Rx Pin on CPU #2 (to CPU #1 Pin 1) #endif ' ' Demo Board XMM Base Address: ' XMM_BASE_ADDRESS = -1 ' Demo board has no XMM ' ' Demo Board Payload pins (SI/SO pins) ' PAYLOAD_RXPIN = SI_PIN ' Rx pin to use for Payload comms PAYLOAD_TXPIN = SO_PIN ' Tx pin to use for Payload comms ' ' Demo Board BlackCat debug pins (SI/SO pins) ' BLACKCAT_RXPIN = SI_PIN ' Rx pin to use for BlackCat comms BLACKCAT_TXPIN = SO_PIN ' Tx pin to use for BlackCat comms BLACKCAT_MODE = %0000 ' SIO mode to use for BlackCat comms ' ' ' Demo Board Last Cog. The value of LASTCOG specifies the last cog ' that will be restarted during the load process. Since CogStore (if used) ' will always be in Cog 7, this value should always be 6 or less. Set this ' value lower than 6 to reserve another cog for other purposes. Note that ' it is still the users responsibility to ensure that the combination of ' loaded drivers and plugins does not exceed this value ' #ifdef RESERVE_COG LASTCOG = 5 ' Last restarted cog is 5 (i.e. reserve cog 6). #else LASTCOG = 6 ' Last restarted cog is 6 (7 is for CogStore) #endif {...} #elseifdef RAMBLADE '=============================================================================== ' ' RamBlade General definitions: ' '=============================================================================== KBD_PIN = 23 ' BASE PIN (RamBlade - for "1 pin" KBD driver only) MOUSE_PIN = -1 ' BASE PIN (RamBlade) TV_PIN = 22 ' BASE PIN (RamBlade - for "1 pin" TV driver only) VGA_PIN = -1 ' BASE PIN (RamBlade) SD_DO_PIN = 24 ' PIN (RamBlade) SD_CLK_PIN = 26 ' PIN (RamBlade) SD_DI_PIN = 25 ' PIN (RamBlade) SD_CS_PIN = 19 ' PIN (RamBlade) I2C_PIN = 28 ' I2C Boot EEPROM SCL Pin I2C_DEV = $A0 ' I2C Boot EEPROM Device Address SI_PIN = 23 ' PIN (RamBlade) SO_PIN = 22 ' PIN (RamBlade) ' ' RamBlade Clock definitions: ' 'CLOCKMODE = xtal1 + pll8x ' (RamBlade) 'XTALFREQ = 10_000_000 ' (RamBlade) 'CLOCKFREQ = 80_000_000 ' (RamBlade) Nominal clock frequency (required by some drivers) ' CLOCKMODE = xtal1 + pll16x ' (RamBlade) XTALFREQ = 6_250_000 ' (RamBlade) CLOCKFREQ = 100_000_000 ' (RamBlade) Nominal clock frequency (required by some drivers) ' 'CLOCKMODE = xtal1 + pll16x ' (RamBlade) 'XTALFREQ = 6_500_000 ' (RamBlade) 'CLOCKFREQ = 104_000_000 ' (RamBlade) Nominal clock frequency (required by some drivers) ' ' RamBlade Loader and InterProp Comms definitions: ' SIO_BAUD = 115200 ' Baud rate to use for all interprop comms and serial loading SIO_LOAD_MODE = %0000 ' RamBlade has only one CPU - this is for serial loading SIO_COMM_MODE = %0000 ' RamBlade has only one CPU - this is for serial loading #ifdef CPU_1 TX_PIN = 0 ' Server Tx Pin on CPU #1 (to CPU #2 Pin 23) RX_PIN = 1 ' Server Rx Pin on CPU #1 (to CPU #2 Pin 22) #else TX_PIN = 22 ' Server Tx Pin on CPU #2 (to CPU #1 Pin 0) RX_PIN = 23 ' Server Rx Pin on CPU #2 (to CPU #1 Pin 1) #endif ' ' RamBlade XMM Base Address: ' XMM_BASE_ADDRESS = $0000_0000 ' XMM adddressing from 0 ' ' RamBlade Payload pins (SI/SO pins) ' PAYLOAD_RXPIN = SI_PIN ' Rx pin to use for Payload comms PAYLOAD_TXPIN = SO_PIN ' Tx pin to use for Payload comms ' ' RamBlade BlackCat debug pins (SI/SO pins) ' BLACKCAT_RXPIN = 31 ' Rx pin to use for BlackCat comms BLACKCAT_TXPIN = 30 ' Tx pin to use for BlackCat comms BLACKCAT_MODE = %0000 ' SIO mode to use for BlackCat comms ' ' ' RamBlade Last Cog. The value of LASTCOG specifies the last cog ' that will be restarted during the load process. Since CogStore (if used) ' will always be in Cog 7, this value should always be 6 or less. Set this ' value lower than 6 to reserve another cog for other purposes. Note that ' it is still the users responsibility to ensure that the combination of ' loaded drivers and plugins does not exceed this value ' #ifdef RESERVE_COG LASTCOG = 5 ' Last restarted cog is 5 (i.e. reserve cog 6). #else LASTCOG = 6 ' Last restarted cog is 6 (7 is for CogStore) #endif {...}Then a different attempt, single target / dual cpu compilation, but this time assuming Demo/Proto is going to supply SD card services to the combo:
{{excerpt from Catalina_Common.spin}} {...} #elseifdef HACKBLADE ' ' Comment out the following line when you have configured the Custom platform: ' 'ERROR : CUSTOM PLATFORM HAS NOT BEEN CONFIGURED! '=============================================================================== ' ' Custom platform General definitions: ' '=============================================================================== SD_DO_PIN = 12 ' PIN (Custom) SD_CLK_PIN = 13 ' PIN (Custom) SD_DI_PIN = 14 ' PIN (Custom) SD_CS_PIN = 15 ' PIN (Custom) TV_PIN = 16 ' TV has same pin as VGA, need ext adapter VGA_PIN = 16 ' BASE PIN (Custom) MOUSE_PIN = 24 ' BASE PIN (Custom) KBD_PIN = 26 ' BASE PIN (Custom) I2C_PIN = 28 ' I2C Boot EEPROM SCL Pin I2C_DEV = $A0 ' I2C Boot EEPROM Device Address SO_PIN = 30 ' PIN (Custom) SI_PIN = 31 ' PIN (Custom) ' ' Custom platform Clock definitions: ' CLOCKMODE = xtal1 + pll16x ' (Custom) XTALFREQ = 6_250_000 ' (Custom) CLOCKFREQ = 100_000_000 ' (Custom) Nominal clock frequency (required by some drivers) ' ' Custom platform Loader and InterProp Comms definitions: ' SIO_BAUD = 115200 ' Baud rate to use for all interprop comms SIO_LOAD_MODE = %0000 ' Custom has only one CPU - this is for serial loading SIO_COMM_MODE = %0000 ' Custom has only one CPU - this is for serial loading ' ' CPU #1 is RamBlade ' CPU #2 is ProtoBoard ' #ifdef CPU_1 TX_PIN = 22 ' Client Tx Pin on CPU #1 (to CPU #2 Pin 1) RX_PIN = 23 ' Client Rx Pin on CPU #1 (to CPU #2 Pin 0) #else TX_PIN = 0 ' Server Tx Pin on CPU #2 (to CPU #1 Pin 23) RX_PIN = 1 ' Server Rx Pin on CPU #2 (to CPU #1 Pin 22) #endif ' ' Custom platform XMM Base Address: ' XMM_BASE_ADDRESS = $0000_0000 ' XMM adddressing from 0 ' ' Custom platform Payload pins (SI/SO pins) ' PAYLOAD_RXPIN = SI_PIN ' Rx pin to use for Payload comms PAYLOAD_TXPIN = SO_PIN ' Tx pin to use for Payload comms ' ' Custom platform BlackCat debug pins (SI/SO pins) ' BLACKCAT_RXPIN = SI_PIN ' Rx pin to use for BlackCat comms BLACKCAT_TXPIN = SO_PIN ' Tx pin to use for BlackCat comms BLACKCAT_MODE = %0000 ' SIO mode to use for BlackCat comms ' ' ' Custom platform Last Cog. The value of LASTCOG specifies the last cog ' that will be restarted during the load process. Since CogStore (if used) ' will always be in Cog 7, this value should always be 6 or less. Set this ' value lower than 6 to reserve another cog for other purposes. Note that ' it is still the users responsibility to ensure that the combination of ' loaded drivers and plugins does not exceed this value ' #ifdef RESERVE_COG LASTCOG = 5 ' Last restarted cog is 5 (i.e. reserve cog 6). #else LASTCOG = 6 ' Last restarted cog is 6 (7 is for CogStore) #endif {...}Regards
Alessandro
P.S. maybe this really belong to the 2.9 thread, it's untested on 3.0.