SD mode is a little different. Only the block read/write commands need a card-busy check before being issued. Other commands can be issued while the card is still busy.
EDIT: There is a separate CMD-ready check for all commands though. Can't issue a start bit if the CMD pin is already low before starting.
@Wuerfel_21 said:
EDIT: So what you're saying is, the busy check should not be at the end of a write, but before issuing a command? Because the audio driver does have a ready check after a block write.
I didn't analyse if that was a vital config but pretty much yes. I've copied how sdmm.cc command sequence does it. That is the more efficient solution anyway since it exits from the driver while the card is known to be in a busy state. Only checking in if more action is requested.
There is a second busy check after CMD24, but before any block data is sent, as well.
The point I was asking about is wether doing it that way is actually a bug? (and given I don't have a lot of time to re-verify right now, what the least invasive fix would be)
@Wuerfel_21 said:
The point I was asking about is wether doing it that way is actually a bug? (and given I don't have a lot of time to re-verify right now, what the least invasive fix would be)
sdmm.cc has always been that way. I would imagine it has a proven track record well beyond the Propellers. I only sped up the bit rate when I modified sdmm.cc.
PS: And why I mention the SD mode driver, sdsd.cc, is because I've implemented it in a similar fashion as well - Only check the card status when the driver really needs to know.
@Wuerfel_21 said:
The point I was asking about is wether doing it that way is actually a bug? (and given I don't have a lot of time to re-verify right now, what the least invasive fix would be)
sdmm.cc has always been that way. I would imagine it has a proven track record well beyond the Propellers. I only sped up the bit rate when I modified sdmm.cc.
No, I mean wether the audio driver needs to be fixed
Yep. You should be able to remove the leading call #read_byte in do_sdcmd routine. The busy check now performs that duty.
And also should be able to remove the post-write call #sd_busy as well since every command is now effectively checking for that.
@evanh said:
Yep. You should be able to remove the leading call #read_byte in do_sdcmd routine. The busy check now performs that duty.
And also should be able to remove the post-write call #sd_busy as well since every command is now effectively checking for that.
Yeah, but I don't want to change it too much, don't have time to test it on many configurations right now
Oh, yep, you've done better than I thought too. I had imagined a card-busy check was done before each raw block read as well as written but that's not so in sdmm.cc. All good.
EDIT: In hindsight, duh. It is looking for the start token rather than a busy check.
@"Stephen Moraco" I have a special use case for a potential feature request (if such a feature does not already exist).
The Ask: Is it possible to have an option where a file that is written to/created by this object, is defragmented after it is closed?
The Why: The P2s in my project boot from SD. They upgrade themselves via ethernet. They boot using the "_BOOT_P2.BIX" filesystem method. If this file is fragmented, the P2 will not boot. I have validated that the driver will fragment these files as it writes them. Defragmenting the _BOOT_P2.BIX file allows it to boot.
I hope this is not too much of an ask. If there is defrag functionality that already exists, that would perfect.
I'd convert to booting from the Flash EEPROM instead. And have it loaded with a full FAT32 filesystem that handles fragmented files correctly. Also allows better error reporting when the SD card isn't working.
@Rayman said:
@ke4pjw wondering if copying the file would do that automatically…
Well, I am doing a sd.moveFile. I suspect it just updates the dir entry without actually copying the file. You may be on to something there. This is the only file that I care about being contiguous. Another work around is 512K cluster sizes, LOL! But that's ridiculous.
@evanh I want people to just be able to place files on the SD card and it boot. Another workaround, would be to use the MBR and build a 2nd stage bootloader that can deal with fragmented boot files. I would much rather do that, as the end user would just need to run a program to make the SD card bootable by the P2, rather than having to get propplug hardware and figure out how to load the bootloader.
@ke4pjw said:
I want people to just be able to place files on the SD card and it boot. Another workaround, would be to use the MBR and build a 2nd stage bootloader that can deal with fragmented boot files.
Make a small _BOOT_P2.BIX file and make it the second stage loader. for anything large.
@ke4pjw said:
I want people to just be able to place files on the SD card and it boot. Another workaround, would be to use the MBR and build a 2nd stage bootloader that can deal with fragmented boot files.
Make a small _BOOT_P2.BIX file and make it the second stage loader. for anything large.
Use Catalyst. It can auto-boot another program from the SD card, even when Catalyst itself is being booted from FLASH. You just put the command in a file called AUTOEXEC.TXT on the SD card.
The ENABLE_AUTO capability is all you need to execute the command in AUTOEXEC.TXT on boot. The other stuff is fancier stuff you can disable.
Then (after the initial compile using build_all, which builds some necessary stuff) manually recompile just catalyst, removing the -llinenoise and -lhyper libraries (they are RAM hogs, and are not needed for this).
ke4pjw,
Here's what should be a usable second stage loader. Now just got to dig up some instructions...
Obviously first rename it to _BOOT_P2.BIX and place that in the SD card.
And I'm guessing also make a text file named AUTOEXEC.TXT and place one line in it with the name of the main executable.
I read it. It doesn't apply here, since we no longer ever have to re-write _BOOT_P2.BIX. Just write it - once - to a freshly formatted SD card, and it will never get fragmented even if you rewrite all the other files on the card multiple times.
That's another advantage of using Catalyst's AUTOEXEC.TXT capability.
He'd just use the EEPROM if that's what he was up to. I think he's doing maker demos or something where lots of online users are duplicating his instructions using their own kits. So he wants it as simple and foolproof as possible.
Not reformatting just makes the whole process less scary for the users I suspect. Which also means there could be hundreds of other files on the target SD card.
@evanh said:
He'd just use the EEPROM if that's what he was up to. I think he's doing maker demos or something where lots of online users are duplicating his instructions using their own kits. So he wants it as simple and foolproof as possible.
Not reformatting just makes the whole process less scary for the users I suspect. Which also means there could be hundreds of other files on the target SD card.
Fair enough. But there are so many issues with the P2 and SD cards that I would always recommend using a freshly formatted SD card. Especially for novices.
Also, I always use the SD Association's SD card format program, because the Windows one has been dodgy (I don't know if it still is, because I no longer ever use it).
Ross,
How close is Catalyst's Prop2 start up environment to a cold boot? Assuming a Spin2 program that won't return - Compiled by Pnut, SpinTools, or FlexSpin.
@evanh said:
Ross,
How close is Catalyst's Prop2 start up environment to a cold boot? Assuming a Spin2 program that won't return - Compiled by Pnut, SpinTools, or FlexSpin.
Catalyst should start any Propeller program, however created. Ultimately, the Catalyst loader loads the cluster list of the program to be loaded into upper Hub RAM, then loads a cog program that loads those clusters into Hub RAM and then uses a simple coginit instruction to restart itself as the program it has just loaded.
So there are some size restrictions due to the fact that this loader cog expects the cluster list to be in upper Hub RAM (in order that the SD card plugin can be used to load the clusters themselves into Hub RAM) and also that the Catalina Registry itself is intact (so it can communicate with the SD card plugin) - but in most cases (especially on the P2 - it was more of an issue on the P1, which had very limited Hub RAM) these are not significant limitations.
So the differences are:
1. that the loaded program must not be so large that loading it would overwrite upper Hub RAM;
2. that the cog that the loaded program starts in may differ than if it was loaded directly (e.g. from BOOT_P2.BIX); and
3. that other cogs may still be executing when the program starts (e.g. one cog will still be executing the SD card plugin).
I could probably add an option to "cogstop" all the other cogs before executing the "coginit" if the last one was a problem, but Catalina programs already do this - it is not quite as trivial as it sounds, because some programs need to leave some cogs running to work (for example, a cog executing an XMM driver, or a cog used to transfer arguments and/or environment variables to the new program).
Ross.
EDIT: Added details about cogs also being used to transfer arguments and environment variables.
Thanks for all the suggestions @rossh and @evanh - @"Stephen Moraco" added some functionality to support defrag and tuned the FS to help prevent it. I implemented it and it seems to be working. It is in the SD_INCLUDE_DEFRAG feature flag. Spin-tools choked on it, but I had Claude create a spin2 preprocessor in .NET core and that fixed that problem for me. I will share that app.
Comments
SD mode is a little different. Only the block read/write commands need a card-busy check before being issued. Other commands can be issued while the card is still busy.
EDIT: There is a separate CMD-ready check for all commands though. Can't issue a start bit if the CMD pin is already low before starting.
The point I was asking about is wether doing it that way is actually a bug? (and given I don't have a lot of time to re-verify right now, what the least invasive fix would be)
sdmm.cc has always been that way. I would imagine it has a proven track record well beyond the Propellers. I only sped up the bit rate when I modified sdmm.cc.
PS: And why I mention the SD mode driver, sdsd.cc, is because I've implemented it in a similar fashion as well - Only check the card status when the driver really needs to know.
No, I mean wether the audio driver needs to be fixed
Ya, the double card-busy check on block transfers seems to be important. One before command issue and another before each raw data block.
Like this?
(Also flexsplorp broke my game
, so had to fix that, too)
Yep. You should be able to remove the leading
call #read_byteindo_sdcmdroutine. The busy check now performs that duty.And also should be able to remove the post-write
call #sd_busyas well since every command is now effectively checking for that.Yeah, but I don't want to change it too much, don't have time to test it on many configurations right now
Oh, yep, you've done better than I thought too. I had imagined a card-busy check was done before each raw block read as well as written but that's not so in sdmm.cc. All good.
EDIT: In hindsight, duh. It is looking for the start token rather than a busy check.
NEWS
I just released v1.4.0
Here's what's new/fixed:
Visit the P2 microSD FAT32 Filesystem repository for full documentation, including a driver tutorial, theory of operations, and card catalog.
The release package can be downloaded from the Releases page. Download the sd-card-driver-{version}.zip
file.
If you find issues, please file them at the Issues page.
What's next? Quickly address any reported issues. Likely conditional compile for LFN support.
Enjoy!
Stephen
@"Stephen Moraco" I have a special use case for a potential feature request (if such a feature does not already exist).
The Ask: Is it possible to have an option where a file that is written to/created by this object, is defragmented after it is closed?
The Why: The P2s in my project boot from SD. They upgrade themselves via ethernet. They boot using the "_BOOT_P2.BIX" filesystem method. If this file is fragmented, the P2 will not boot. I have validated that the driver will fragment these files as it writes them. Defragmenting the _BOOT_P2.BIX file allows it to boot.
I hope this is not too much of an ask. If there is defrag functionality that already exists, that would perfect.
Thank you for a wonderful driver!
--Terry
@ke4pjw wondering if copying the file would do that automatically…
I'd convert to booting from the Flash EEPROM instead. And have it loaded with a full FAT32 filesystem that handles fragmented files correctly. Also allows better error reporting when the SD card isn't working.
Well, I am doing a sd.moveFile. I suspect it just updates the dir entry without actually copying the file. You may be on to something there. This is the only file that I care about being contiguous. Another work around is 512K cluster sizes, LOL! But that's ridiculous.
@evanh I want people to just be able to place files on the SD card and it boot. Another workaround, would be to use the MBR and build a 2nd stage bootloader that can deal with fragmented boot files. I would much rather do that, as the end user would just need to run a program to make the SD card bootable by the P2, rather than having to get propplug hardware and figure out how to load the bootloader.
If you copy to different filename might work?
Make a small _BOOT_P2.BIX file and make it the second stage loader. for anything large.
Use Catalyst. It can auto-boot another program from the SD card, even when Catalyst itself is being booted from FLASH. You just put the command in a file called AUTOEXEC.TXT on the SD card.
He just finished saying he didn't want to load anything into the Flash EEPROM.
Even easier - just copy catalyst.bin onto an SD card as _BOOT_P2.BIX
Ross.
Okay, found it in Catalina/demos/catalyst/
so Catalyst is a Prop based modular loader system. Fancy.
Trying it out I'm not getting any joy:
EDIT: [Doh! PATH was screwed]
EDIT2: Right, got it compiling with the
build_all. Now how to trim it down from the 72 kB? Atleast below 32 kB to fit in a common FAT cluster size.Why would you want to do that? It's not necessary.
However, if you feel you have to do so for some reason, edit the catalyst.h file as follows:
The ENABLE_AUTO capability is all you need to execute the command in AUTOEXEC.TXT on boot. The other stuff is fancier stuff you can disable.
Then (after the initial compile using build_all, which builds some necessary stuff) manually recompile just catalyst, removing the -llinenoise and -lhyper libraries (they are RAM hogs, and are not needed for this).
I used this command for my P2 EDGE ...
Final binary file size = 30080 bytes.
Oh, the 32 kB, you obviously haven't read the above conversation - https://forums.parallax.com/discussion/comment/1572524/#Comment_1572524
Cool, done.
ke4pjw,
Here's what should be a usable second stage loader. Now just got to dig up some instructions...
Obviously first rename it to
_BOOT_P2.BIXand place that in the SD card.And I'm guessing also make a text file named
AUTOEXEC.TXTand place one line in it with the name of the main executable.Attached the Catalyst manual too.
I read it. It doesn't apply here, since we no longer ever have to re-write _BOOT_P2.BIX. Just write it - once - to a freshly formatted SD card, and it will never get fragmented even if you rewrite all the other files on the card multiple times.
That's another advantage of using Catalyst's AUTOEXEC.TXT capability.
He'd just use the EEPROM if that's what he was up to. I think he's doing maker demos or something where lots of online users are duplicating his instructions using their own kits. So he wants it as simple and foolproof as possible.
Not reformatting just makes the whole process less scary for the users I suspect. Which also means there could be hundreds of other files on the target SD card.
Fair enough. But there are so many issues with the P2 and SD cards that I would always recommend using a freshly formatted SD card. Especially for novices.
Also, I always use the SD Association's SD card format program, because the Windows one has been dodgy (I don't know if it still is, because I no longer ever use it).
Well, if Terry decides this is a good solution then Catalyst will presumably get a broad workout on lots of SD cards as a result.
Ross,
How close is Catalyst's Prop2 start up environment to a cold boot? Assuming a Spin2 program that won't return - Compiled by Pnut, SpinTools, or FlexSpin.
Catalyst should start any Propeller program, however created. Ultimately, the Catalyst loader loads the cluster list of the program to be loaded into upper Hub RAM, then loads a cog program that loads those clusters into Hub RAM and then uses a simple coginit instruction to restart itself as the program it has just loaded.
So there are some size restrictions due to the fact that this loader cog expects the cluster list to be in upper Hub RAM (in order that the SD card plugin can be used to load the clusters themselves into Hub RAM) and also that the Catalina Registry itself is intact (so it can communicate with the SD card plugin) - but in most cases (especially on the P2 - it was more of an issue on the P1, which had very limited Hub RAM) these are not significant limitations.
So the differences are:
1. that the loaded program must not be so large that loading it would overwrite upper Hub RAM;
2. that the cog that the loaded program starts in may differ than if it was loaded directly (e.g. from BOOT_P2.BIX); and
3. that other cogs may still be executing when the program starts (e.g. one cog will still be executing the SD card plugin).
I could probably add an option to "cogstop" all the other cogs before executing the "coginit" if the last one was a problem, but Catalina programs already do this - it is not quite as trivial as it sounds, because some programs need to leave some cogs running to work (for example, a cog executing an XMM driver, or a cog used to transfer arguments and/or environment variables to the new program).
Ross.
EDIT: Added details about cogs also being used to transfer arguments and environment variables.
Thanks for all the suggestions @rossh and @evanh - @"Stephen Moraco" added some functionality to support defrag and tuned the FS to help prevent it. I implemented it and it seems to be working. It is in the SD_INCLUDE_DEFRAG feature flag. Spin-tools choked on it, but I had Claude create a spin2 preprocessor in .NET core and that fixed that problem for me. I will share that app.