The actually functional Spin2 SD Driver (I hope) (KyeFAT+ sdspi_with_audio)
Wuerfel_21
Posts: 5,052
So this is the simple but functioning P2 SD driver. It was cobbled together from a couple different things, notably the original KyeFAT, @Cluso99 's Spin2 version thereof, the sd_asm_mb that was pulled from some version of FSRW and the audio bits from TinySDDA.
I'm just releasing this as-is as a stopgap, this should be cleaned up a lot.
Features include:
- Actually works (I hope)
- Actually supports all SD cards
- Actually reports errors
- Almost all features from P1 KyeFAT work
- Should work with multiple files (not tested)
- optional Audio streaming abilities semi-compatible with TinySDDA (including pseudo-subcode (used in Spin Hexagon for the pulsing playfield))
- decently fast
Misfeatures include:
- cobbled together
- needs a cog
- Audio features barely documented and hassle to use
- File booting is not yet implemented (but should be easy to add)
Comments
@Wuefel_21,
Thanks for taking the time to port Kye's code (and the other enhancements)!
I am interested in this object because I need to have several files open at the same time. During testing of having 2 files open at the same time (which seems to work, at least for reading), I ran into the following issue:
When reading via readString(stringPointer, maximumStringLength), a line consisting of only a new-line character causes the first character in the returned string to be a null (or unchanged) and not the new-line character. This triggers eof on the file.
@Wuerfel_21
The latest version of Kye's FAT driver I posted works with multiple files, and the SD PASM Driver works too. So DIR/COPY etc all work.
This should be a help to you.
I've ben hung up on updating the serial driver and then putting this all into separate binaries for the full P2 OS - so little time
I don't think I've ever actually used that function in either the P1 version and definitely not this P2 version, so IDK if that's a new bug, an old bug, or working-as-intended. Presumably not the latter, because the code suggests that any newline/null is included in the readback. So checking if the string returns empty is not the way to check for EoF. I think you'd check that
fileSize() == fileTell()?@Weufel_21, that works, thanks.
@Cluso99, I'll take a look at your version. I didn't see it when I did a search for "sd card" in the forum.
Thanks to both of you,
Walter
I can't find where this is posted... can you post the link?
Thanks,
Walter
Cluso's Propeller OS (v2.43) - includes SD Driver and FAT32 Object plus Serial Driver
https://forums.parallax.com/discussion/173395/clusos-propeller-os-v2-38-includes-sd-driver-and-fat32-object-plus-serial-driver
Thanks for creating and posting this. I'm hoping to try it out soon and just now downloaded it.
Looking at the comments for the start method, it says to use values 0..31 (Port A) for the four I/O pins. However, in the v201 update note towards the top, it says "chipSelectPin, clockPin, dataInPin, dataOutPin are now 0-63 instead of bit-masks." So does that mean that we can also pass in Port B pin numbers? I'm guessing/hoping so.
IIRC correctly yes.
Thanks, Cluso99. And thanks for your hard work in porting Kye's driver. And thanks to Wuerfel_21 for "clobbering" "cobbling" (Wuerfel_21's word) this particular version together. I haven't yet compared this version to the one that Cluso99 posted, so I'm not sure how much they differ.
Anyway, based on that "...are now 0-63 instead of bit-masks" comment, it sounds like the pins are now being referenced individually after being set to anything in the range of 0..63.
And in the do_init method of the SPI object that this FAT32 object now uses, I see this code using getbyte:
getbyte pinDO, arg1_in, #0 getbyte pinCLK, arg1_in, #1 getbyte pinDI, arg1_in, #2 getbyte pinCS, arg1_in, #3And I didn't see any subsequent alterations being made to, for example, the pinDI. So hopeful the driver works with anything in the range of 0..63.
Of course it works with any pin, otherwise it wouldn't be able to use the SD socket on the EVAL board
Thanks for the confirmation. Although I wasn't aware that this had been used with the EVAL board, it did occur to me that (something like) that had already been done. But I also believed that there is at least one or two other code options out there for accessing an SD card (though not as "user friendly" as this), so any such access could have been achieved with them. And I figured it wouldn't hurt to ask, as it might help someone else who saw the comment about using 0..31 (even though I sort of guessed that that was just a legacy comment). Anyway, generally, the forum is pretty friendly when posters are making at least some effort to try to understand things (apologies if I missed something obvious or wasted anyone's time). After I solder on a connector onto my board for an SD card module, I'll give this a whirl. Thanks again.
Update:, in my previous post, I just now discovered (much to my chagrin) that I typed "clobbering" instead of "cobbling." That was totally a typo, and I was trying to use Wuerfel_21's own humble word in the description in the first post. It was neither a joke nor a Freudian slip, and my expression of thanks was sincere (as is hopefully clear from the context). But I apologize that my careless typo may have given the wrong idea. I was glad when Wuerfel_21 announced this version of the driver and was surprised that no one commented on it for quite a while (I had held off on downloading it until I got a board made and preliminarily tested). I think that the P2 really needs a user-friendly SD card driver like this (a top-five priority, I'd say), one that will be readily accessible to folks that haven't been following along closely or who are new to the P2. So the easier we can make it for folks to understand/use, the better.
Yeah, the other SD driver options basically are
That's why I threw together this one, because I needed something to put in Spin Hexagon that, well, actually worked. Hence why the block driver has audio streaming features (BTW, there's 2K of RAM being wasted if you don't use the audio. You can just set their sizes to zero near the bottom of the block driver file and it shouldn't explode)
Thanks for the additional info. When I skimmed through the SDSPI file earlier, I noticed the reserved longs for left and right audio channel pins, which caught me off guard (though the full name of that file does end with the words "_with_audio." And on quick inspection, I didn't figure out where those pins were being passed in or otherwise set. But just as long as I didn't have to pass them in to FAT32 first, I sort of hoped/assumed that I could ignore them (famous last words).
As for saving memory in the "block driver file" when not using audio, I presume that you're talking about the following 2K of buffers (the word "music" is a pretty obvious clue, and I suppose that "sfx" could stand for "special (audio) effects"):
musicHubBuffer1 byte 0[512]
musicHubBuffer2 byte 0[512]
sfxHubBuffer1 byte 0[512]
sfxHubBuffer2 byte 0[512]
So I could zero out those "512's" assuming that I don't make any audio calls, as a "minimally invasive" change. Yes, that still compiles (wasn't sure if "byte 0[0]" would). Oh, and I just noticed that the audio pins are apparently passed in using the ConfigureDAC() method (which is also code that could probably be yanked if not using audio). Of course, if I do yank it, then I'll probably decide that I need it soon afterwards.
Thanks for for mentioning that 2K, as every K of memory can count. I remember going through Kye's driver (the SPIN part) and weeding out as much as I could to save memory (of course, that was on the 32KB P1 where things were much tighter). Anyway, my weeding skills are pretty minimal, but fortunately, I didn't seem to break anything (that I needed, anyway). Still, I was kind of swinging blindly and can imagine doing that again for this version. So thanks for the tip.
Also, thanks for the rundown of the currently available SD card options with status info and other comments. That's a helpful summary.
Update: Not that you mentioned it, but my apology for typing Wuerfel_21 as Wuerful_21. Hopefully, I corrected all of those misspellings in my posts up above.
Have you heard that me and Eric have been working on adding Spin bytecode support to flexspin? Currently only on P1, but aside from automatic removal of unused code, it can squeeze out hundreds of bytes and a little extra speed using clever code generation and flow optimizations. Maybe it'll be available for P2 soon-ish.
Yes, that's exactly what I mean, it only uses the buffers when it actually plays. You can also remove only one buffer and leave the other.
Also, if you do need the audio component, you can see how it's used in Spin Hexagon for P2 in
hexagon_sdda.spin2.A rough overview:
- include the block driver object into your code
- call
ConfigureDAC- use the FAT driver to find the address and length of your files (which must be raw 16 bit stereo PCM for music files and raw mono A-law for SFX. File length should ideally be multiple of 512 bytes.) and verify that they are not fragmented.
- issue the relevant commands with the gathered values to start playing sound.
I have no idea why that particular misspelling is so common, haha.
I just quickly skimmed my SD_DRIV.spin v270 and it is currently restricted to the portB pins as I do an OUTB which cannot be avoided to get the speed in an unrolled loop. This would require a global source change to OUTA if pins in the range 0-31 were to be used.
Currently I don;t pass the pins from a higher level so the source requires a change.
From the latest SD_DRIV V275 (released as part of my P2 OS v243)
https://forums.parallax.com/discussion/173395/clusos-propeller-os-v2-38-includes-sd-driver-and-fat32-object-plus-serial-driver
'' RR20210130 v270 third release (spin2 demo version) '' RR20210228 v271 try FLTH #DO {I} and Init twice (changes discarded) '' RR20210513 v272 use _PINDEF2.spin2 '' fix: in _releaseDO: bith sdx_bitout, #_pin.SD_CS-32 ' prepare /CS=1(disable) '' RR20210516 v273 fix: in startSD move ptr_mailbox := sd_mailbox before calling stopSD '' RR20210518 v274 fix: in stopSD only set cmd=status=$FF if actually stopping the cog (v273 consequence) '' RR20210518 v275 change: -> startSD(sd_mailbox, sd_buffer, sd_pins) & set mailbox (note sd_pins not used yet!!) '' wait until mbox_command = $A5 on start '' add 'Q' command to stop cog from driver & quitSD() '' WARNING: SD pins valid only for 32-63 (not 0-31) because DIRB/OUTB/INB are used. '' WARNING: _CLOCKFREQ is set to 200MHz for delay calcs!@Clusso99: Regarding SD_DRIV.spin v270 being "currently restricted to the portB," I think that's the better port of the two for SD. I look at Port B as the "system port" and Port A as the "application port." I think that many of us could live with that restriction just fine.
@Wuerfel_21: Thanks for the additional info. And no, I didn't know that you and Eric were working to add Spin bytecode support to Flexspin. That's cool, and partnering on that big goal seems like the way to go. That would be especially useful in Spin1, obviously. The auto dead code removal would be welcome, naturally, and being able to do all those other optimizations (both for code size and speed) is amazing. Also, thanks for the tips for audio usage (maybe I can spare a pin for mono). BTW, I haven't played Hexagon but I watched the video shortly after it first came out and it looked cool (though I might not have sufficient "fast-twitch" muscles for it).
Okay, this is working as "advertised" and I'm rapidly reading through a large text file just fine. BTW, I'm using P32-35 {Port B} for the SPI communication. Anyway, this seems quite user-friendly. Thanks, folks.
I wonder what is involved to add this. It's probably over my head, but what would one do, load in longs sequentially from a file and store them in the Hub starting at some specific location and then call some P2 function to reboot?
Hmm, I just looked and saw that there is a bootPartition() method in both Wuerfel_21 and Clusso99's versions of the FAT32 driver. But they are the same, I believe, and Wuerfel_21 states that "file booting is not yet implemented." So I wonder if that's the case with Ray's driver, too, though the secret sauce is likely in the called methods, which I haven't checked. So perhaps Ray's version does implement it, as it would seem that an OS could use that (though the OS may run all the time, in which case it might not need that, unless there were a way to get back to/reload the OS). Anyway, looking at the aforementioned method, I see something about a checksum, so perhaps booting is more involved than I speculated about above (one wouldn't want to run potentially garbled code from the transmission from the SD card).
Anyhow, having boot functionality in the FAT driver on the P1 was useful and convenient, so it'd be great to have it on the P2 (though having it in a separate object would be great, too).
Booting mostly has to do with the block driver. It basically has to stop all the other cogs, then load the file into RAM, init cog 0 and then stop itself.
On P2 there's the added issue that the file being booted may be fragmented (this is mostly irrelevant to P1 since the cluster size on SD cards is usually 32k)
Thanks, Wuerfel_21, for that reply with the main steps. Sounds even more complicated than I imagined. Wonder if, when Cog 0 is "initialized," its data is automatically read in (such as the SPIN2 interpreter) from the first part of the Hub. I suppose that it would be (though prior to your comment, I had speculated that, for booting from an SD card, maybe another cog would load up Cog 0 and then start it (before possibly killing itself). But it sounds like Cog 0 gets loaded automatically (by the firmware, I presume) based on how I'm reading your comment (though I'm foggy on what "init" entails). I know that such details have been discussed in the forum on occasion over the years, but I've totally forgotten the details (or never paid sufficient attention).
Also, I've not studied SD cards, but I assume that there's a "pointer" of some kind or link that's associated with each cluster (or block or whatever the actual data units are called) that points to the next logical cluster, such that a user can follow the "chain" of clusters when the clusters are non-sequential (fragmented). I wonder whether that "pointer" is stored in the clusters themselves or stored separately (assuming that the SD card controller doesn't manage this). Hmm, but it would seem that the existing driver that you guys have already provided would already be taking care of this with it's regular read methods (whether in the block driver, as you refer to it, or the FAT object, or both, I know not) since any file could be fragmented, not just a file to be booted from.
Anyway, thanks very much for the overview of the steps involved that you provided.
The SD-Card itself has no notion of blockchains (pun intended). That is what a filesystem does, in this case FAT32. It has blocks at known positions from which it builds the data structure to encode the file tree and it’s contents.
So within this context one could write a firmware flat into the SD-Card from a known start Block (probably 0) and just read it sequentially. Or use FAT32 and then load the firmware from a file that can be spread all over the place.
The way Kye's driver boots on the P1 is to read the FAT to get the sector number chain into the block driver cog. This COG then kills all other ones, loads the file into HUB, sets the clock to RCFAST and starts itself with the SPIN interpreter.
On the P2 things are a bit more complex because you need the existing clock settings to switch the PLL off, sadly Chip does not use the commonly agreed upon memory location ALL other compilers use.
Enjoy!
Mike
@deets: That makes sense, as there are different kinds of files systems (at the high level). Plus, the SD card controller takes care of low-level black magic like wear leveling. Thanks for your comments.
@msrobots: Thanks for the analysis and summary of how Kye's booter works, as well as the comments regarding conversion to the P2. Much appreciated.
Example code of mounting the FS, opening a file and reading a line. No error handling or anything like that, just a quick and dirty example.
https://forums.parallax.com/uploads/editor/7d/6alo16txn1xt.zip
{ SD Card Example Terry E. Trapp, ke4pjw Place test.txt on the SD card you wish to use. Be sure to enable debug to see the contents of the first line of the file. } CON _xtlfreq = 20_000_000 _clkfreq = 270_000_000 '270Mhz DOPin = 58 CLKPin = 61 DIPin = 59 CSPin = 60 VAR BYTE results[255] BYTE RDATA[255] OBJ FS : "FAT32" PUB Main() FS.FATEngineStart(DOPin, CLKPin, DIPin, CSPin, 0) FS.mountPartition(0) results := FS.openfile(string("test.txt"),string("R")) FS.readString(@RDATA, 20) FS.closeFile() FS.unmountPartition() debug(zstr(@RDATA)) DAT {{ /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// // TERMS OF USE: MIT License /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// // Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation // files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, // modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the // Software is furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in all copies or substantial portions of the // Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE // WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR // COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, // ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// }}Belated comment on this: In Spin source code, use the
clkfreqsymbol. You don't need to know the actual hubRAM location. FlexC uses_clockfreq()for the same.So I found that this driver just hangs when an invalid filename, or filename that does not exist is passed to openfile or listEntry. It never returns from those methods.
Not sure where the problem is. Is there a more current version of this driver where the error handing works? Otherwise I will have to roll my own method to enumerate the FS before attempting to open the file. I haven't checked to see if that works either.
Thanks,
Terry
OK, it appears instead of a return value, it is using abort. I need to learn how to properly handle an abort.
Your topmost method call will be prefaced with \ to tell the compiler to abort back to that point.
OK, got it! Had to add the ABORT trap
IF \FS.Method(yadda,yadda,yadda)when attempting to open the file. Woot! I have a little webserver on the P2 now!What sent me down that rabbit hole was trying to figure out why it would just stop. I think I had to go 4 or 5 methods deep before I saw something I wasn't sure how it would behave, which was abort.
Not sure why I have not come across it before, but now that I know, it will come in handy!
One of the few places I've seen ABORT used is in FSRW (P1 SD driver).