Catalina 4.0 is now available
RossH
Posts: 5,484
in Propeller 1
I just posted this in the Propeller 2 forums:
http://forums.parallax.com/discussion/170697/catalina-4-0-is-now-available
However, Catalina 4.0 supports both the Propeller 1 and the Propeller 2, so I am posting this here as well!
http://forums.parallax.com/discussion/170697/catalina-4-0-is-now-available
However, Catalina 4.0 supports both the Propeller 1 and the Propeller 2, so I am posting this here as well!
Comments
I was originally intending to add a specific platform symbol to Catalina for the FLiP module, but when I came to do so I realized that there is actually nothing on the FLiP that requires special configuration - the default platform (aka CUSTOM) is suitable already.
However, it now occurs to me (too late for this release!) that it might have been better to add the FLiP symbol anyway, just in case anyone has both the FLiP module and some other platform that requires a different CUSTOM configuration.
I will add this symbol to the next release. In the meantime, if you have a FLiP module, just use the default platform.
Ross.
No doubt you've already anticipated this question already, so I'm going to go ahead and ask it:
Do you plan on adding a 2-Port serial library to the Propeller1 as well as a 4-Port serial library to the Propeller2?
I'm just curious about this.
Please note that I do not need a 2-Port serial library for the Propeller1 code I'm working on, but if I decide to migrate to the Propeller2 (P2D2), I will need a 4-Port serial library for it.
At this point I don't anticipate migrating to the Propeller2 as the tweaks you've done to the 4-Port serial library and 8K XMM cache are working great. All of the Menus are working fine, and I'm able to communicate with the GPS receiver via both the 4-Port serial library as well as my own I2C functions. I'm really happy with the results I'm getting.
The only reason I would contemplate moving to the Propeller2 as this point would be if I need to perform message unpacking/formatting and serial port traffic management using another cog in the background. Without having to use XMM on the Propeller2 I would be able to run multiple C functions on the other cogs.
Anyway, just wondering what your plans were to have common serial port library functions available for the two Propeller platforms...
No, there seems very little point. Apart from the fact that the P1 supports 4 ports, and the P2 supports higher baud rates (neither of which I can do much about), the existing 4 port serial library on the P1 is almost identical to the 2 port library on the P2. I did that deliberately. All the functions have very similar names and profiles.
A few #define statements would be all that is necessary to port a program that required only 2 ports and lower baud rates from the P1 to the P2 or vice-versa. I suppose I could provide a common "catalina_serial.h" file for the P1 and P2 which did that for you - i.e.:
Ross.
Just realized I didn't answer this bit ...
I am not sure there will ever be a 4-port serial driver for the P2, but it would be fairly easy to load two copies of the current 2-port serial drivers. It would cost an additional cog **.
Ross.
** EDIT: To facilitate this, I have just reserved another plugin name and number for a second 2-port serial plugin. This will make it trivial for me to add 4-port support in future if no-one has developed a 4-port serial plugin for the P2 by the time you need it!
The smartpins should allow a single cog to do many serial ports...
And yes more ports should be possible but my Goal was to have the LUT as buffer, so it is two ports with buffer in the LUT. It might be extensible, but with less available buffer for rx/tx of the ports.
Mike
I'm having some issues with structures and unions in Catalina and wanted to make sure I've got a proper grasp on variable sizes to ensure they aren't causing some alignment issues.
First, I want to verify that under Catalina, a short int is a 16-bit value?
Second, under Catalina, an int is a 32-bit value?
If so, is a long int a 32-bit or 64-bit value?
OK, with that being said, I want to read in a serial stream of data from a GPS receiver (ublox ZOE-M8Q). I'm doing this using one of the Ports on the 4-Port serial driver.
The data is stored in a character string while being read in. This string is "unioned" to a structure so I can grab, unpack, and print the values.
This test structure is a truncated version of the actual UBX-NAV-SOL message for the GPS receiver, but it will be enough to illustrate the problem I'm having.
Here's the test structure and union:
struct UBX_NAV_TEST { unsigned char Header1; // 0xb5 unsigned char Header2; // 0x62 unsigned char MsgClass; // 0x01 unsigned char MsgId; // 0x06 unsigned short MsgLen; // 0x34 unsigned int GpsTime; signed int GpsFrac; unsigned short GpsWeek; unsigned char GpsFixType; unsigned char GpsFlags; }; union Tsetvanxbu { unsigned char Chr[40]; struct UBX_NAV_TEST Info; }; union Tsetvanxbu UbxNavTest;Placing all of this inside my program, along with the code that retrieves the various Structure values and prints them out, yields no compiling problems at all.
Upon execution, the code appears to work fine.
The string is properly populated.
When I unpack and display the Structure variables as Hex values, the data up to and including the MsgLen variable is correct:
However, for some reason, retrieving the Structure variables after MsgLen there appears to be some strange offset that causes the GpsTime to be retrieved starting 2-bytes to the right of where it should be.
And because of this offset, all the other Structure variables after GpsTime are also skewed and incorrect as well.
Any idea what is going on here? I'm completely puzzled by this.
Is there a Packed issue with the Structure that I'm not accounting for?
I think all C compilers for the P1 default to the ILP32 programming model, where "int", "long", and pointers are all 32 bits. The P1 hardware only allows accesses on properly aligned boundaries: that is, a 16 bit quantity must start on a 16 bit boundary, and a 32 bit quantity must start on a 32 bit boundary in memory. So the struct:
struct foo { unsigned short s; unsigned long i; } f;will actually be arranged in memory as if it were: Some compilers support a #pragma or __attribute__ to force structures to be packed (ignoring the hardware alignment restrictions). In this case 32 bit loads have to be emulated by loading bytes, shifting, and or'ing. I don't think Catalina supports packed structs, although @RossH will know for sure. I tried the following test program with Catalina and it printed offsets of 0 and 4:#include <stdio.h> #include <stddef.h> #pragma pack(1) struct foo { unsigned short S; unsigned long L; } x; int main() { printf("offsets: S=%d L=%d\n", offsetof(struct foo, S), offsetof(struct foo, L)); for(;;) ; }The P2 hardware allows unaligned accesses, so theoretically it should be straightforward to support packed structs on P2. This doesn't seem to be implemented yet in Catalina (or in fastspin, it's something I should look into).
I believe Eric is quite correct. Catalina will pad structures according to the alignment of their types (e.g. 1 byte for char, 2 bytes for short, 4 for int). I could allow unaligned access on the P2, but on the P1 it would be horrendously inefficient.
Thanks for confirming it was an alignment issue within the structure that required padding to the next boundary.
I suspected that was the case and was hoping for a compiler directive like Packed to get around it, but unfortunately it's not available.
However, I was able to work around this problem by breaking up the int values into short ones then reassembling them back into int after reading them.
With this workaround in place, the union of the struct with the char string worked great!
I'm now able to read the GPS Time, GPS Week, ECEF position and velocity, and various Status information from the receiver as desired.
Thanks again for your feedback!
And I do have a P2 Eval Board on order
With the structure issue resolved, I do have a couple more questions to ask:
It seems that Catalina only allows Bitfield variables to be constructed using int but not short?
It also appears that the string functions like strcpy() and strcat(), for example, only work with char strings and not unsigned char strings?
So this will work: But this one will throw a compiler error:
Which isn't a big deal, just something to keep in mind.
I guess I still haven't completely recovered from my OpenWatcom C/C++ days when I wrote code for the AMD Geode processor boards at work and took advantage of the Packed directive for Structures, created Bitfield variables using short instead of int, and carelessly tossed unsigned char variables at the string functions without pushback
When keeping these very, very minor limitations in mind, one can see that Catalina is quite an impressive compiler!
I've had a lot of fun working with it while making the Propeller do the same types of GPS data collection and display functions that the Geode board did
Your work on propeller2.h was instrumental in this -- on P1 the header file situation is a mess, and producing output that's compatible with all of the P1 C compilers is quite a headache. I think P2 should be much cleaner. Thanks for your help!
Yes, according to the C standard a bit field can only be an integer (signed or unsigned). Some compilers may allow other types.
Yes, again, that's according to the C standard. Just use a cast - i.e. pass "(char *)str" instead of just "str".
I'll have a look at it. I have been waiting to update Catalina's Spin support until there is an "official" Spin compiler/interpreter available. Or is fastspin "official"? I have not been keeping up with Spin developments on the P2 - no time!
No worries!
As I recall Catalina's Spin support involves invoking the Spin interpreter and interpreting bytecodes. spin2cpp is an alternate path, since it converts the Spin source into C source. The two can co-exist quite nicely, I think
I have a couple more questions to toss your way:
1. What is the RX buffer size for the s4 library? 16 bytes? 32? I assume the TX buffer is the same size as the RX...
2. Several months ago we had a brief discussion about XMM and Threads. Your initial impression was that it might be possible, perhaps for 2 to 3 Threads, but were concerned about cache misses and jitter. Have you had a chance to give this any more thought?
Here's why I ask:
My code reads a 19.2K serial stream from a GPS receiver, populates various GPS message structures (and sets some flags when ready), executes some motor control functions based upon the fresh GPS data as commanded by a radio link, then provides this information to the operator via the Console Port (i.e. Pins 30 & 31).
I'm somewhat able to do this now without Threads using a pseudo time slicing scheme within an infinite loop. The purpose of the time slicing scheme is to attempt to avoid RX buffer overrun issues on the GPS serial stream, while performing the housekeeping, motor control, radio transceiver, and operator display functions mentioned previously. This scheme works fair with the s4 library, and even better with the tty256 library, but it's still very sloppy, choppy, and clumsy.
If XMM Multithreading was possible I could allocate one Thread to handle housekeeping, motor control, radio transceiver, and operator interface chores, while another Thread could be dedicated to reading the serial stream from the GPS receiver, populate the various message structures needed, then set flags when ready.
A couple of Threads with more deterministic time slicing, even with the jitter problems, might be a cleaner approach than what I'm doing now...
To answer your questions:
1. 64 bytes each for tx and rx buffer
2. Possible? Yes. Likely in the short term? No.
Have you tried compiling your program using the Compact Memory model? If you could get your code & data (don't worry about the plugins like the 4 port serial driver) to under 32kb by doing so, then you could use multithreading and it would also be faster than XMM.
Ross.
Indeed, CMM would make these problems go away. Instead of Multithreading, I would just assign the GPS parsing and structure packing functions to a separate Cog and be done with it.
Unfortunately all of the things I'm trying to get the Prop1 to do vastly exceed the 32KB limit of HubRam. My code is presently pushing 90KB, and likely will be around 128KB by the time I add in all of the motor control and radio transceiver code.
Hence I have to go with XMM. There doesn't appear to be a big problem with speed, as the code seems to execute at a decent rate using the Rampage2 configuration and 8K of cache.
I see at least two possible solutions to solving the problem: Multithreading or new cog assignment.
Multithreading isn't possible yet on the Prop1 while in XMM mode, so this option is out.
That leaves the new cog option, but assigning C code to a new cog won't work when the main C code is running in the XMM kernel.
However, Catalina will allow a new cog to execute PASM or Spin while C code is running in XMM.
Which brings me to the next question: How can Spin running in a separate cog have access to the same block of data as that used by a C program in the main XMM code? I'm using the Small Memory Model so these variables are contained within HubRam.
The thought is that I could have the GPS parsing and structure packing functions be performed by a Spin program on a separate cog, but it would need access to the same block of HubRam memory that the C program reserved for the data structures.
The Catalina manual makes reference to the possibility of running Spin and PASM programs on separate cogs while C is executing in the XMM one. But it doesn't get into details of how it can be done.
Do you have hints, tips, or suggestions on how the C program can have access to the same variables as the Spin program?
Actually, interesting you should raise this - I have given some thought to another option - essentially "partitioning" the Prop - i.e. having one cog (actually two if you need the cache) that runs C using XMM, while other cogs runs C using CMM or LMM. Not "multithreading", but just executing in parallel. This is actually much easier than multithreaded XMM. It would even be possible for the cogs running CMM or LMM to multithread, but that would complicate things - I probably won't bother with that in the short term.
Essentially, you would just be able to have multiple independent C partitions executing, of which only one can be XMM.
The independent C partitions would have to communicate either via the registry or via fixed memory addresses (whereas with multithreading they can just use global C variables) - but it might still be quite handy.
Sadly, I am not getting much time for Prop stuff just at the moment, so this would still be a "medium term" thing. I would just have to amend Catalina's Hub memory management code a bit.
Yes, you can certainly execute PASM or SPIN programs in conjunction with C. See the "demos\spinc" folder for examples. These demos should work in XMM SMALL mode (you may need to modify the "build_all" script, since I just noticed some of the demos assume the platform has a TV HMI option!).
As I mentioned above, the SPIN and C programs can potentially communicate via the registry, but the simplest way would be via dedicated Hub RAM buffers - all that is required is that both programs know where the buffers are, and don't use that RAM for anything else. To do this in C, the easiest way is to modify the file "Catalina_Common.spin" (in the "target" directory) and add your own buffers by amending the RUNTIME_ALLOC value (make sure you use the right one - there are two - one is for cached access, and the other is for non-cached access).
For example, assuming you use the cache, it currently says something like:
You would need to modify this to something like:
Then compile this, find out where MY_BUFFER points to, then hardcode that address in both your C and SPIN program.
Ross.
That capability would definitely get me where I want to go. The XMM kernel would continue to handle the operator interface (Menus, etc) like it does now, while a separate Cog running C code via a CMM kernel could handle the GPS serial parsing and structure populating functions.
The GPS parsing and structure populating code isn't all that large. I could probably have the CMM kernel also handle the radio message traffic management and maybe even motor control too.
From the C programmer point of view, being able to access common HubRam memory needed by both the XMM and CMM kernels would probably be easiest via global variables as you mentioned, but passing pointers to the data using the registry or fixed mailboxes would also work. I don't know how many mailboxes would be available or needed to do that, but it certainly seems like a practical solution.
The XMM and CMM kernels would also need to have access to plugins like the 4-Port Serial driver for example. I assume this wouldn't be a problem. In my case, the XMM kernel would use Port0 as a Console to interface with the operator, while Port1 and Port2 would be used by the CMM kernel to communicate with the GPS receiver and radio, respectively.
Enhancing Catalina to simultaneously support a single XMM kernel and multiple CMM kernels would certainly allow users to fully exploit the multicore capability of the Prop1 while running large programs, would be highly attractive to users like myself who aren't quite ready to migrate to the Prop2 (even though I do have a P2 Eval Board
I think your idea is excellent!
No problem. All access to plugins is already done via the registry, using locks where necessary to prevent contention. This already works in multithreaded environments, and would work exactly the same in a partitioned environment.
Ross.
I've been thinking about this and wondering what approaches could be used to allow the XMM cog and CMM cogs to have access to common variables within HubRam.
Months ago, when I was doing some preliminary work on my Project, I was using CMM exclusively. One cog was running the main() C code, while another cog was running C code handling console I/O functions using pins P31 and P30.
Now, if memory serves correct, and since code, constants, variables, and stack were contained within HubRam, both cogs had direct access to common variables since they had been declared as Global in scope.
Expanding this to an XMM cog with additional CMM cogs, a Global variable approach would also work, but only if the XMM cog was using the Small Memory Model, where all variables are contained within HubRam.
Unfortunately, this fails if the XMM cog is running the Large Memory Model (which is what I'm using right now for testing) since this Model has the variables contained within the External Memory and not HubRam.
So a Global variable scheme won't work here, unless you introduce a new variable definition that specifies a variable is contained within HubRam as opposed to External Memory. I think PropGCC offers something like this but I haven't looked into it.
The Registry is one approach that would work for both Small and Large Memory Models, but how many variables would the Registry permit access to?
The Shared HubRam Memory technique would also work, but the same question applies as to how many memory Locations would be reserved?
I suppose I could bundle a group of common variables into a Structure, then have a Shared Memory location (i.e. Mailbox) contain a Pointer to this Structure to permit access by both the XMM and CMM cogs.
The Registry, Shared Memory (Mailbox), or HubRam defined Global variable scheme would allow an XMM and CMM cogs to have access to common variables within HubRam, and it wouldn't matter if XMM was using the Small or Large Memory Model.
However, none of these schemes would permit a CMM cog to have direct access to an XMM Large Memory Model variable. Of course there are ways to get around that limitation, but no need to get into it here.
Anyway, I'm just curious about your thoughts on the most practical memory access scheme you're considering for common XMM and CMM variables...
Sorry not to answer sooner. We are busy fighting bushfires here - have been for weeks now. I will get back to this stuff as soon as I get some spare time
Ross.
Not a problem. I figured you were preoccupied with the fires around Sydney.
From what I've seen on the news it's horrific. I saw a video of an airtanker conducting low altitude sorties dropping fire retardant. And watching the Firenados was unsettling.
Then there's New Zealand with the White Island volcano catastrophe. I saw a news clip about an entire American family perishing in that one...
Stay safe over there!!
Been logging in occasionally to see what's happening, but I haven't had time to do any more work on Catalina recently.
Our entire region got burned out by the Australian bushfires a couple of months ago, then flooded out soon after that, and we are still in recovery mode, and will be for a while yet. We lost our entire holiday season trade (50% of our annual income), and are busy trying to get our business back on track in time for Easter.
I expect to get some time to do more work on Catalina in a month or two.
Ross.
I was thinking of you during the fires and then the big wet. Great to hear that you survived but I feel your pain with the economic hit. It’s a big problem for all the businesses in such a large area.
Many don’t realise that over a couple of days we received 12” which is 1/3 of our annual rainfall and this added 20% to Sydney’s dam level, or to put it another way, 2 years of drinking water.
And now we have the Coronavirus to contend with. I’m just waiting with hope for China to get back on track as I need parts and pcbs to go with the P2s I’ve ordered.
All the best,
Ray
I'm sorry to hear about the huge revenue loss but glad that you survived the bushfires.
I guess they're pretty much out now, at least in your area?
I've been checking your Facebook posts periodically, and it looks like the foliage is coming back to life. That's very good news. Now maybe the phone company can get your service restored soon?
I do have some questions about the Catalina EEPROM loader, but those can wait for the appropriate time.
I didn't expect to see you resuming work on Catalina until at least June given the current circumstances.
Plus, as we head into Spring here in the States, I will have to shift my focus to outside chores and away from the propeller coding on my computer anyway, so no biggie.
But still looking forward to discussing some Catalina issues/ideas with you when the time is right.
Cheers,
Jeff
Yes, the fires are now out, and the floods have receded. Our biggest problems now are the telephone (not expected to be restored till Easter), the access roads (3 out of 4 closed due to burnt-out bridges - one was fixed after the bushfires and then got washed away in the floods 4 days later!), and the damage to the local attractions (all the National Parks near us - our biggest draw - may remain closed for another year).
But we soldier on!
@Wingineer19, you can email me your questions, or post them here. I can at least be thinking about them even if I don't have any time to do anything yet.
Ross
I hope all is well with you, and your recovery from the fires and floods is progressing nicely. Maybe even your phone has or will be restored soon?
Considering the global crisis with the pandemic and lockdowns, as well as a health emergency with my 91 year old Dad, I've been away from this forum for a while. When trying to log back in tonight, I had a Senior Moment and forgot my password, prompting a reset request to Parallax.
My last post was entitled "Catalina EEPROM Loader Issues And Resolution" and centered around an issue with the EEPROM Loader having a problem writing across multiple EEPROMs on a carrier board I built. Interestingly, once the code was already written across these multiple EEPROMs, your bootup Loader had no problem reading from them and executing the code.
The fact the bootup Loader could read code > 128KB across multiple EEPROMs and execute it, while the programming Loader couldn't, puzzled me. Anyway, I was able to work around the problem by writing my own programming Loader to burn the code on the EEPROMs. I think I posted the code on that forum thread. I've since damaged the EEPROM carrier board, so the issue is moot for now until I build up a new board.
I guess my other question has to do with your idea of allowing Catalina to execute XMM code on one cog, while allowing LMM or CMM code to execute on other cogs, along with your library plugins. This idea is intriguing, and would solve many of the programming issues I was having last year on my Project. I was just wondering if you've had any time to revisit this particular idea.
Thanks.
We are as good as can be expected here. Hope you are the same. We had partially recovered from our various natural disasters and had just re-started our business when along came the Coronavirus crisis and shut it all down again. You might think that this would give me more time to work on Catalina, but it just hasn't worked out that way. We had to spend a lot of time and effort getting our retreat ready for Easter - our second busiest period of the year - only to see all our bookings get cancelled. After previously losing all our Christmas bookings - our busiest period of the year - this was very disheartening.
We still have no phone - the fires and floods wiped out a lot of infrastructure in our region, and much of it is yet to be rebuilt. Roads, bridges, telephones, power lines etc etc ... we will probably not have some of these fully repaired for another year or more. Luckily, our power is all solar and our internet connection is via satellite, so at least we still have that!
One thing we had to do during the fires was pack up our valuables in case we needed to evacuate (we didn't, thank goodness). But we have still not even unpacked all those boxes - and in one of them is all my Propeller stuff. Naturally, that went in the first box I packed, so it will probably also be the last one to be unpacked!
I did see your last post, but haven't had time to investigate the loader yet. It didn't seem urgent because you also posted your own work-around!
As for the multiple memory model option - I had forgotten all about it. I do remember now that I had thought of a simple way to accommodate it, but at the moment I can't remember what it was. It will probably come back to me once I start looking into it again.
Still, I had better not promise anything - every time I do, some new disaster crops up to make a liar of me! But as the days get shorter and the weather gets colder, I hope to have less to do outside around the property, and more time inside.
Ross.