When using Catalina's version of Geany, using Cake as the C preprocessor produces messy error and warning messages. This is because Cake uses ANSI escape codes to set the message color - which works in a command-line or terminal window, but not in Geany's compiler output window.
It is not very intuitive, but you can disable Cake's use of color by adding -W-Wp-fdiagnostics-format=msvc to the Catalina command line (see note below as to what this means) - and the easiest way to do that in Geany is using the Project Properties dialog box.
For example, to enable the use of Cake but also disable Cake's use of color, you might add the following line to the Catalina Options:
-C99 -W-Wp-fdiagnostics-format=msvc
In the next release I will make disabling color the default, and instead add an option to enable the use of color.
There is also a very minor tweak to Cake's error message formatting that I have pushed to GitHub.
Ross
A note about -W-Wp-fdiagnostics-format=msvc : the first -W tells Catalina to pass the rest of the option to LCC . Then the -Wp tells LCC to pass the rest of the option to the preprocessor (i.e. Cake). Then the -fpdiagnostics-format=msvc tells Cake to use Microsoft format for messages, which (in this version of Cake) is the only format that disables the use of color. I have asked the Cake author to make enabling/disabling color an independent option, which will make things simpler in the next version!
@RossH said:
... But someone else is welcome to do so, since it involves rewriting just a single and entirely self-contained plugin which is just a few hundred lines of PASM (see target/p2/cogsd.t - ignore the clock support which is implemented in the same cog just because there is spare space).
Oh, I see, ignore the RTC functions. And these forum outages are getting annoying too. I come back to check on a posting and, bam, an error page is all I get.
Well, I've hacked at that file a bit and got results. Albeit nowhere near where it should be as yet. Doubled the bit rate (For 300 MHz sysclock) of the low level R/W bit-bashed routine - Now locked at sysclock/12 clock divider, producing a few percent gain on reads and nothing at all on writes. Then took a look at the block write routine to find an unexpected 10 ms WAITX instruction with a comment asking why its needed ... wiped that out and all its related conditions, nearly halving the code size of that routine. Result is sequential writes are now 5x faster: 50 kB/s.
Can't see any quick improvements for block reads though. I can see the driver runs in its own cog so I guess a block of prefetching will help but I'd like to see what can be done about enhancing the mailbox features first. There is no multi-block request! A multi-block request would remove large amounts of mailbox handshake. The block read duration is currently at 350 us but each has a 1.4 ms gap of inactivity from completion of prior block to request for a contiguous block!
What file do I look at to mangle the mailbox interface as I see fit?
While the forums are up (and who knows how long they will stay!), I'm posting this same message in a couple of threads:
I will no longer be monitoring these forums until Parallax gets it sh*t together. If anyone needs Catalina support, my email address is in the Catalina documentation. Email me direct for support.
@evanh said:
Then took a look at the block write routine to find an unexpected 10 ms WAITX instruction with a comment asking why its needed ... wiped that out and all its related conditions, nearly halving the code size of that routine. Result is sequential writes are now 5x faster: 50 kB/s.
That 10ms delay was required to support some SD cards. I never could figure out why it was needed, or exactly what value it should be, so I just settled for 10ms which worked with all the cards I tested. Removing or reducing it might work with some cards but not others. There seems no way to tell whether or not it is actually required. However, I could make it a compile-time option.
Can't see any quick improvements for block reads though. I can see the driver runs in its own cog so I guess a block of prefetching will help but I'd like to see what can be done about enhancing the mailbox features first. There is no multi-block request! A multi-block request would remove large amounts of mailbox handshake. The block read duration is currently at 350 us but each has a 1.4 ms gap of inactivity from completion of prior block to request for a contiguous block!
There is no multi-block support in DOSFS. It was specifically designed to minimize its RAM footprint, so (for example) it only ever use a single sector buffer. This is what made it suitable for the Propeller 1, where Hub RAM is at a premium - but on the Propeller 2 it is indeed a bit of a bottleneck. However, having a different file system on the Propeller 1 and Propeller 2 is also not a very good option.
What file do I look at to mangle the mailbox interface as I see fit?
You can add new services easily enough, but you can't modify the mailbox interface itself - the Registry (which is what Catalina calls it) underpins too many aspects of Catalina's functionality, and if you change it the amount of rework required would be huge. If you find the Registry is not suitable, use it to establish a shared memory buffer on startup and then use that as your interface instead. However, you make a good point - I retired the document that described Catalina's Registry in detail some time ago and never replaced it. I will add more detail on the Registry to the Catalina reference manual.
I've not seen fixed delays in other drivers. They just require a card busy/ready check at the right points ... and Catalina's driver looks to be doing those checks already. I didn't try to evaluate it too hard though.
On a related note, the low level bit-bashing is a lot cleaner the way I've got now without the extraneous extra clocks. It is possible that SPI mode could get sensitive to mismatched byte lengths during a transaction the way it was. The spec talks about SPI mode working in exact bytes.
EDIT: Err, the original code doesn't actually do that. It just looks slightly odd with the terminating OUTL #_SD_CK is all.
What might be of significance though is the version notes says:
Version 4.0 - Added 8 extra clocks after each operation. Added a new "ready" check before each operation.
The 10 ms block write delay was likely compensating for the missing ready check. With that fixed I doubt you'll find any cards that require the 10 ms any longer.
@RossH said:
You can add new services easily enough, but you can't modify the mailbox interface itself - the Registry (which is what Catalina calls it) underpins too many aspects of Catalina's functionality, and if you change it the amount of rework required would be huge. If you find the Registry is not suitable, use it to establish a shared memory buffer on startup and then use that as your interface instead. However, you make a good point - I retired the document that described Catalina's Registry in detail some time ago and never replaced it. I will add more detail on the Registry to the Catalina reference manual.
I retired the document that described Catalina's Registry in detail some time ago and never replaced it. I will add more detail on the Registry to the Catalina reference manual.
Yes please. No doubt I'll regret it.
Attached is a description of the Registry extracted (and updated) from an old document. I will add this to the Catalina Reference Manual in the next release.
The 10 ms block write delay was likely compensating for the missing ready check. With that fixed I doubt you'll find any cards that require the 10 ms any longer.
Thanks. I'll give it a try with all the SD cards I have. But I may no longer have the cards that actually needed it. There were some SD cards that were so dodgy I ended up finally just throwing them out.
I can't get this code to work. Writing to the SD card locks it up and I have to power cycle it to recover. I am using Catalina's DOSFS test program in the demos\dosfs folder. Perhaps you could test your code with that, or else post your own working test program?
Oh, that wasn't everything I edited. There is other single lines to support it. Ah, namely the longcnt replaces bytecnt and all the presets longcnt are quartered in value compared to bytecnt. And I did the same to _readBLOCK() too.
Umm, you could revert that part and just remove the 10ms delay.
EDIT: Or here's the whole file (Looks to be from July 2025 of Catalina)
EDIT: Or here's the whole file (Looks to be from July 2025 of Catalina)
Yes, this works. Thanks.
I've only tested one SD card so far, and your driver runs about the same speed as the version I just posted to GitHub, which allows the plugin delay to be set at compile time (using -C SD_DELAY=n, where n is the value to use in the waitx statements).
For example, to set the delay to zero:
cd demos/file_systems
catalina -p2 -lcx test_stdio_fs.c -C SD_DELAY=0
Both your plugin and mine give a noticeable improvement in the speed of this simple test program, but (as I had expected) in a real-world case (e.g. using Catalina to compile "hello.c" on the Propeller 2) they both give only a modest improvement - about 5%. This is probably because I use a sector cache in Catalina.
Both your plugin and mine give a noticeable improvement in the speed of this simple test program
Ok, I may have spoken too soon. I just had a card lockup on a "known good" SD card using your driver. Perhaps there is a problem with it, or perhaps that delay really is necessary.
I will revert to my driver but with SD_DELAY set to zero for a while, and see if I get similar occasional lockups.
@RossH said:
I will revert to my driver but with SD_DELAY set to zero for a while, and see if I get similar occasional lockups.
Just had an SD card lockup with my driver as well when the delay is set to zero. Haven't had a problem with that particular card for months (if not years). It is an 8Gb card that that I use all the time in various different Propellers.
So some delay seems to be required in the driver to make SD card writes reliable, but I still don't know why
Is it easily/quickly repeated? It sounds like you have been doing long run testing to catch any problem.
Was it the same type of failure on the same card when testing with the driver I edited?
@evanh said:
Is it easily/quickly repeated? It sounds like you have been doing long run testing to catch any problem.
Was it the same type of failure on the same card when testing with the driver I edited?
It is not very predictable, but I have only seen it happen after starting some intense SD card writing activity, such as a C compilation or saving a file in the vi text editor. While I can't guarantee it is the same problem, the symptoms seem to be the same in both your driver and mine - i.e. the SD card locks up and a reset won't work - the SD card must be power cycled before it will respond again.
Hmm, I'm not seeing any problems using my speed tester and the above edited driver. Done quite a lot of repeated tests. Written to all seven of my uSD cards.
Ahhh, I'm getting something now ... It's funny, just mulling over the fact I wasn't able to reproduce the issue made me consider what I could change. And it dawned on me that I should use the default boot pins in my testing. A while back I'd setup Catalina/target/p2/P2EVAL.inc for using pins config of Roger Loh's add-on and hadn't changed it back. Guess what? Changing back to the boot pins now nets a failed write when overwriting an existing file, but only if that is the first file to be opened, and also the SD card has stayed powered up. If any other file operation is performed first then the fault doesn't occur.
I've not looked at what's happening in the driver as yet.
What options is there for debugging plug-ins? They don't look particularly conducive to printf().
Err, umm, not quite correct. It seems all file ops fail without a double reboot. I thought it was the writes only because that's what is first op in the tester. Oddly, that first write doesn't actually produce an error. It reports the file written normally even though nothing is changed on the SD card. It's the read op following that throws the error.
So, card init has failed I guess. Except then wouldn't that first write also abort with an error?
@evanh said:
Err, umm, not quite correct. It seems all file ops fail without a double reboot. I thought it was the writes only because that's what is first op in the tester. Oddly, that first write doesn't actually produce an error. It reports the file written normally even though nothing is changed on the SD card. It's the read op following that throws the error.
So, card init has failed I guess. Except then wouldn't that first write also abort with an error?
I'm not sure what you mean by "double reboot". When the SD card locks up, it needs a power cycle - no number of reboots will get it going again.
@evanh said:
Changing back to the boot pins now nets a failed write when overwriting an existing file, but only if that is the first file to be opened, and also the SD card has stayed powered up. If any other file operation is performed first then the fault doesn't occur.
I've never seen anything like that. Is this with your driver or mine? Also, this could be due to a corrupt file system. I do all testing with a freshly formatted SD card, to which I always copy the same files for testing. That way, any failures are more repeatable.
@evanh said:
What options is there for debugging plug-ins? They don't look particularly conducive to printf().
I find that just using the LEDs is usually enough.
I need a lot more than that, I'll be getting creative I guess. I want to be able to report parameters from the mailbox as well as SD card interactions and internal state of the driver.
@RossH said:
@evanh said:
Changing back to the boot pins now nets a failed write when overwriting an existing file, but only if that is the first file to be opened, and also the SD card has stayed powered up. If any other file operation is performed first then the fault doesn't occur.
I've never seen anything like that. Is this with your driver or mine? Also, this could be due to a corrupt file system. I do all testing with a freshly formatted SD card, to which I always copy the same files for testing. That way, any failures are more repeatable.
It's my test program as posted, and the edited driver as posted. Correct, not the same behaviour as you've seen. As mentioned it works flawlessly when not using the boot pins. I can move off the boot pins and there's no issue again. This was the only way I could get any issue at all.
SD cards aren't being corrupted. They aren't being accessed at all when the fault occurs. Which is why I suspect it's a card init fail.
But then I'd expect the first file open for writing to fail with an error as well, but it doesn't. It just silently does nothing instead.
@evanh said:
I need a lot more than that, I'll be getting creative I guess. I want to be able to report parameters from the mailbox as well as SD card interactions and internal state of the driver.
Ross,
I see the driver is arbitrarily writing to 32 bytes at hubRAM $20 through $3f. How much room is likely available directly beyond that?
Comments
When using Catalina's version of Geany, using Cake as the C preprocessor produces messy error and warning messages. This is because Cake uses ANSI escape codes to set the message color - which works in a command-line or terminal window, but not in Geany's compiler output window.
It is not very intuitive, but you can disable Cake's use of color by adding -W-Wp-fdiagnostics-format=msvc to the Catalina command line (see note below as to what this means) - and the easiest way to do that in Geany is using the Project Properties dialog box.
For example, to enable the use of Cake but also disable Cake's use of color, you might add the following line to the Catalina Options:
-C99 -W-Wp-fdiagnostics-format=msvcIn the next release I will make disabling color the default, and instead add an option to enable the use of color.
There is also a very minor tweak to Cake's error message formatting that I have pushed to GitHub.
Ross
A note about -W-Wp-fdiagnostics-format=msvc : the first -W tells Catalina to pass the rest of the option to LCC . Then the -Wp tells LCC to pass the rest of the option to the preprocessor (i.e. Cake). Then the -fpdiagnostics-format=msvc tells Cake to use Microsoft format for messages, which (in this version of Cake) is the only format that disables the use of color. I have asked the Cake author to make enabling/disabling color an independent option, which will make things simpler in the next version!
Oh, I see, ignore the RTC functions. And these forum outages are getting annoying too. I come back to check on a posting and, bam, an error page is all I get.
Well, I've hacked at that file a bit and got results. Albeit nowhere near where it should be as yet. Doubled the bit rate (For 300 MHz sysclock) of the low level R/W bit-bashed routine - Now locked at sysclock/12 clock divider, producing a few percent gain on reads and nothing at all on writes. Then took a look at the block write routine to find an unexpected 10 ms WAITX instruction with a comment asking why its needed ... wiped that out and all its related conditions, nearly halving the code size of that routine. Result is sequential writes are now 5x faster: 50 kB/s.
Can't see any quick improvements for block reads though. I can see the driver runs in its own cog so I guess a block of prefetching will help but I'd like to see what can be done about enhancing the mailbox features first. There is no multi-block request! A multi-block request would remove large amounts of mailbox handshake. The block read duration is currently at 350 us but each has a 1.4 ms gap of inactivity from completion of prior block to request for a contiguous block!
What file do I look at to mangle the mailbox interface as I see fit?
Here's the main edits so far:
'+-----------------------------------------------------------------------------+ '+ SD SPI Send/Recv Routines... (write/read byte/long simultaneously) + '+ Bit-bashed at sysclock/12 clock divider, ignoring any SD spec + '+-----------------------------------------------------------------------------+ _recvlong neg dataout, #1 ' call here to Recv a Long (+send 1's) _sendlong mov bitscnt, #31 ' call here to Send a Long (long=32bits) jmp #_sendrecv _sendFF ' call here to Send $FF Byte _recvbyte neg dataout, #1 ' call here to Recv a Byte (+send 1's) _sendbyte rol dataout, #24 ' call here to Send a Byte (msbit first) mov bitscnt, #7 ' (byte=8bits) _sendrecv rol dataout, #1 wc ' prepare output bit (DI=0/1).. outl #_SD_CK ' CLK=0 outc #_SD_DI ' write output bit: output on CLK falling edge mov reply, #0 ' clear reply outh #_SD_CK ' CLK=1 rep @.rend, bitscnt rol dataout, #1 wc ' prepare output bit (DI=0/1).. outl #_SD_CK ' CLK=0 outc #_SD_DI ' write output bit: output on CLK falling edge testp #_SD_DO wc ' read input bit: SD Standard Speed output on CLK falling edge outh #_SD_CK ' CLK=1 rcl reply, #1 ' accum DO input bits .rend waitx #2 ' safe lag timing of 16 ticks from CLK falling to IN sampling outh #_SD_DI ' idle high testp #_SD_DO wc ' read input bit: SD Standard Speed output on CLK falling edge _RET_ rcl reply, #1 ' accum DO input bits'+-----------------------------------------------------------------------------+ '+ Write Block/Sector: (512 bytes) + '+ CMD24, PAR=blocknr + '+-----------------------------------------------------------------------------+ _writeBLOCK ' CMD24: PAR=sector, 512 bytes getct ini_time '\ set timeout for cmd24 mov time_out, ##delay2s '/ '+-----------------------------------------------------------------------------+ call #_cmdRZtoken ' /CS=0, send cmd, recv R1, /CS=0(ena) if_nz jmp #_fail '24 ' <>$00(good): error '+-----------------------------------------------------------------------------+ mov dataout, #$FE ' start block token call #_sendbyte ' write data byte '+-----------------------------------------------------------------------------+ .writebyte rdlong dataout, bufaddr ' load byte movbyts dataout, #%00_01_10_11 call #_sendlong ' write data byte add bufaddr, #4 ' bufaddr++ djnz longscnt, #.writebyte ' byte-- ' NOTE: CRC16 is ignored in SPI mode call #_getreply ' and reply, #$1f cmp reply, #5 wz drvl #56 ' diag if_z ret or reply, #$100 ' ensure reply is not zero jmp #_failI've also tweaked
_readBLOCKto use_recvlonginstead of_recvbyteAnd SD clock pin now idles high too. This suits better for not clashing with the SPI CS pin of the Flash EEPROM.
While the forums are up (and who knows how long they will stay!), I'm posting this same message in a couple of threads:
I will no longer be monitoring these forums until Parallax gets it sh*t together. If anyone needs Catalina support, my email address is in the Catalina documentation. Email me direct for support.
Ross.
That 10ms delay was required to support some SD cards. I never could figure out why it was needed, or exactly what value it should be, so I just settled for 10ms which worked with all the cards I tested. Removing or reducing it might work with some cards but not others. There seems no way to tell whether or not it is actually required. However, I could make it a compile-time option.
There is no multi-block support in DOSFS. It was specifically designed to minimize its RAM footprint, so (for example) it only ever use a single sector buffer. This is what made it suitable for the Propeller 1, where Hub RAM is at a premium - but on the Propeller 2 it is indeed a bit of a bottleneck. However, having a different file system on the Propeller 1 and Propeller 2 is also not a very good option.
You can add new services easily enough, but you can't modify the mailbox interface itself - the Registry (which is what Catalina calls it) underpins too many aspects of Catalina's functionality, and if you change it the amount of rework required would be huge. If you find the Registry is not suitable, use it to establish a shared memory buffer on startup and then use that as your interface instead. However, you make a good point - I retired the document that described Catalina's Registry in detail some time ago and never replaced it. I will add more detail on the Registry to the Catalina reference manual.
Thanks for the reply.
I've not seen fixed delays in other drivers. They just require a card busy/ready check at the right points ... and Catalina's driver looks to be doing those checks already. I didn't try to evaluate it too hard though.
On a related note, the low level bit-bashing is a lot cleaner the way I've got now without the extraneous extra clocks. It is possible that SPI mode could get sensitive to mismatched byte lengths during a transaction the way it was. The spec talks about SPI mode working in exact bytes.
EDIT: Err, the original code doesn't actually do that. It just looks slightly odd with the terminating
OUTL #_SD_CKis all.What might be of significance though is the version notes says:
The 10 ms block write delay was likely compensating for the missing ready check. With that fixed I doubt you'll find any cards that require the 10 ms any longer.
Yes please. No doubt I'll regret it.
Attached is a description of the Registry extracted (and updated) from an old document. I will add this to the Catalina Reference Manual in the next release.
Thanks. I'll give it a try with all the SD cards I have. But I may no longer have the cards that actually needed it. There were some SD cards that were so dodgy I ended up finally just throwing them out.
I can't get this code to work. Writing to the SD card locks it up and I have to power cycle it to recover. I am using Catalina's DOSFS test program in the demos\dosfs folder. Perhaps you could test your code with that, or else post your own working test program?
Oh, that wasn't everything I edited. There is other single lines to support it. Ah, namely the
longcntreplacesbytecntand all the presets longcnt are quartered in value compared to bytecnt. And I did the same to _readBLOCK() too.Umm, you could revert that part and just remove the 10ms delay.
EDIT: Or here's the whole file (Looks to be from July 2025 of Catalina)
Yes, this works. Thanks.
I've only tested one SD card so far, and your driver runs about the same speed as the version I just posted to GitHub, which allows the plugin delay to be set at compile time (using -C SD_DELAY=n, where n is the value to use in the waitx statements).
For example, to set the delay to zero:
Both your plugin and mine give a noticeable improvement in the speed of this simple test program, but (as I had expected) in a real-world case (e.g. using Catalina to compile "hello.c" on the Propeller 2) they both give only a modest improvement - about 5%. This is probably because I use a sector cache in Catalina.
Still, every little bit helps!
Ross.
Ok, I may have spoken too soon. I just had a card lockup on a "known good" SD card using your driver. Perhaps there is a problem with it, or perhaps that delay really is necessary.
I will revert to my driver but with SD_DELAY set to zero for a while, and see if I get similar occasional lockups.
Just had an SD card lockup with my driver as well when the delay is set to zero. Haven't had a problem with that particular card for months (if not years). It is an 8Gb card that that I use all the time in various different Propellers.
So some delay seems to be required in the driver to make SD card writes reliable, but I still don't know why
Ross.
Is it easily/quickly repeated? It sounds like you have been doing long run testing to catch any problem.
Was it the same type of failure on the same card when testing with the driver I edited?
It is not very predictable, but I have only seen it happen after starting some intense SD card writing activity, such as a C compilation or saving a file in the vi text editor. While I can't guarantee it is the same problem, the symptoms seem to be the same in both your driver and mine - i.e. the SD card locks up and a reset won't work - the SD card must be power cycled before it will respond again.
Ross.
Hmm, I'm not seeing any problems using my speed tester and the above edited driver. Done quite a lot of repeated tests. Written to all seven of my uSD cards.
Example Sandisk Extreme 64GB (2021)
I'm suspicious that that added 10ms delay is hiding the real problem outside the driver, which is still present.
Need a clear way to reproduce the fault.
Ahhh, I'm getting something now ... It's funny, just mulling over the fact I wasn't able to reproduce the issue made me consider what I could change. And it dawned on me that I should use the default boot pins in my testing. A while back I'd setup Catalina/target/p2/P2EVAL.inc for using pins config of Roger Loh's add-on and hadn't changed it back. Guess what? Changing back to the boot pins now nets a failed write when overwriting an existing file, but only if that is the first file to be opened, and also the SD card has stayed powered up. If any other file operation is performed first then the fault doesn't occur.
I've not looked at what's happening in the driver as yet.
What options is there for debugging plug-ins? They don't look particularly conducive to printf().
Err, umm, not quite correct. It seems all file ops fail without a double reboot. I thought it was the writes only because that's what is first op in the tester. Oddly, that first write doesn't actually produce an error. It reports the file written normally even though nothing is changed on the SD card. It's the read op following that throws the error.
So, card init has failed I guess. Except then wouldn't that first write also abort with an error?
I find that just using the LEDs is usually enough.
I'm not sure what you mean by "double reboot". When the SD card locks up, it needs a power cycle - no number of reboots will get it going again.
I've never seen anything like that. Is this with your driver or mine? Also, this could be due to a corrupt file system. I do all testing with a freshly formatted SD card, to which I always copy the same files for testing. That way, any failures are more repeatable.
I need a lot more than that, I'll be getting creative I guess. I want to be able to report parameters from the mailbox as well as SD card interactions and internal state of the driver.
It's my test program as posted, and the edited driver as posted. Correct, not the same behaviour as you've seen. As mentioned it works flawlessly when not using the boot pins. I can move off the boot pins and there's no issue again. This was the only way I could get any issue at all.
SD cards aren't being corrupted. They aren't being accessed at all when the fault occurs. Which is why I suspect it's a card init fail.
But then I'd expect the first file open for writing to fail with an error as well, but it doesn't. It just silently does nothing instead.
Ross,
I see the driver is arbitrarily writing to 32 bytes at hubRAM $20 through $3f. How much room is likely available directly beyond that?
[solved]
[solved]