Rayman,
Drop this in as updated driver. It's not any bug fixes or added features, although it does now support the new block devices interface, it's mostly just a tidy up and a little less trigger happy on recalibrating. I think it should work fine with the older Flexprop versions.
EDIT: Err, there is one line you'll need to change: Replace #include "../fatfs/diskio.h" with #include "diskio.h"
EDIT2: I've removed one diagnostic line that wasn't still meant to be there.
EDIT3: Correction to above comment: Huh, there is a new feature. I managed to forget about the adding of pin registration toggle to runtime calibration function. This makes it more rugged if using sysclock/2 for speed when CRC processing is disabled in the block read routine.
EDIT4: UPDATED driver! v0.4 has a bug. Oopsie, I hadn't completed the removal of SELECT/DESELECT for block read/write calls. I haven't had any read/write errors of late so it never triggered the bug. I'd missed out adjusting the recalibration step which requires the card deselected to perform. Removal of select/deselect is a small optimisation with SD operating mode, as opposed to SPI mode. In SD mode the card doesn't much care about select/deselect for normal block read/write. SD cards are assumed a point-to-point exclusive link, whereas SPI/MM modes have provision for a shared bus to multiple devices.
Ouch! Nowhere does FSRW deal with a read/write error. It just ploughs on as if no error occurred! That's gotta be prone to corruption of the stored filesystem as well as the files themselves.
I mean when the low level block read, or write, reports back that a CRC failed the check FSRW then just ignores the error and continues as if all is fine. No attempt at even one retry nor any further accommodating like attempting to adjust the file metadata and FAT tables.
The low level block functions don't do retries themselves.
LOL, nothing happened ... I found a tester that uses Kye's FAT32 and patched my driver to it and it compiled first try surprisingly. Running the tester, said everything happened but the activity LED did nothing, which shouldn't possible, and upon inspecting the card's content I see no file created. I guess I need to see how that tester works now ... or add in the debug() messages to the spin driver. I didn't port those over to save some mental effort at the time.
Starting FAT32 Engine...
Attempting to Mount uSD partition a few times...
SD Partition Mounted.
SD Root Directory:
End of SD Root Directory.
Writing partition profile info to Profile.txt on uSD.
Writing large file to test write speed. This may take a few seconds...
Writing 1MB took: 0 milliseconds.
Reading that large file to test read speed.
Reading 1MB took: 0 milliseconds.
Closing Partition.
Oh-no! TRUE is exclusively -1 in Spin. That's going to be some work to fix up. Hang on I'm using the wrong operator ... 43 operators later ...
Bah! More operator problems. This time bit-shift precedence is way higher in Spin than C.
Got all the way to start of block reading ... now it's gone into a black hole in the pasm code. Going to have to reorganise the top level code to use debug() before I can dig deeper. It's currently using SmartSerial.spin2 ...
Hmm, the program crashes when compiled with Flexspin, but not when compiled with Pnut ...
Yay, Pnut compiled edition is completing its run and looks okay at least. It clearly isn't verifying anything though.
Attempting to Mount uSD partition a few times...
SD Partition Mounted.
SD Root Directory:
SPEED10.BIN
SPEED2.BIN
SPEED3.BIN
SPEED4.BIN
SPEED5.BIN
SPEED6.BIN
SPEED7.BIN
SPEED8.BIN
SPEED1.BIN
TEST.TXT
PROFILE.TXT
SPEED.TXT
End of SD Root Directory.
Writing partition profile info to Profile.txt on uSD.
Writing large file to test write speed. This may take a few seconds...
Writing 1MB took: 2105 milliseconds.
Reading that large file to test read speed.
Reading 1MB took: 600 milliseconds.
Closing Partition.
Done.
I've avoided using the differences so it should compile as is in both environments. After all, the same Pasm mechanisms are used in the original C version of the driver.
EDIT: I've worked out that, when compiled with Flexspin, the "inline" Pasm2 code isn't getting stuck/crashing directly. It successfully exits back to the compiled Spin2 code of its function, rx_datablocks() but the function itself never returns.
EDIT2: Oh, duh! The new compile line had lost the --fcache=256. Man, there needs to be a way for a program to specify that. Here's the Flexspin compiled edition:
Attempting to Mount uSD partition a few times...
SD Partition Mounted.
SD Root Directory:
SPEED10.BIN
SPEED2.BIN
SPEED3.BIN
SPEED4.BIN
SPEED5.BIN
SPEED6.BIN
SPEED7.BIN
SPEED8.BIN
SPEED1.BIN
TEST.TXT
PROFILE.TXT
SPEED.TXT
End of SD Root Directory.
Writing partition profile info to Profile.txt on uSD.
Writing large file to test write speed. This may take a few seconds...
Writing 1MB took: 2080 milliseconds.
Reading that large file to test read speed.
Reading 1MB took: 517 milliseconds.
Closing Partition.
Done.
Don't think so. I'll see what Eric says to my pleas.
PS: Here it is as it stands. I've not rewritten the tester program yet so no verifying is done. I've modified init steps to take more pin parameters but otherwise, excepting the new driver, it's mostly unchanged. I've also altered use of SmartSerial to include a line-feed with the carriage-returns. This fits better with debug() when pairing them under Flexspin.
PPS: The Spin edition of the driver doesn't attempt to select High Speed access mode. It was one of a number of items initially dropped in the name of getting it going. I've only just remembered it's still missing. However, I'm not sure it's worth adding though. It never benefited the C edition speed wise and had really only become a case of completeness since it was already done there.
@evanh said:
Don't think so. I'll see what Eric says to my pleas.
PS: Here it is as it stands. I've not rewritten the tester program yet so no verifying is done. I've modified init steps to take more pin parameters but otherwise, excepting the new driver, it's mostly unchanged. I've also altered use of SmartSerial to include a line-feed with the carriage-returns. This fits better with debug() when pairing them under Flexspin.
What's with the CRs as new lines in two of these source files in the zipfile? Usually you have either LF, or CRLF. This is like some old Mac text format. Need to do some file cleanup. Is this generated from PNut/PropellerTool or something else?
LOL, yeah, good point. I'm using Pnut's new "structures" that Chip finalised in the last couple of weeks. Needs both Pnut v45 and spanking new build of Flexspin.
I only did it like that for labelling convenience. Same reason I'd used C structures too. I could've just used some arrays of longwords.
Pnut is awkward by itself too. It's debug terminal is useless for anything other than debug. I guess I should get in the habit of using it from the shell instead of GUI ... ah, that's right, Wine doesn't search the PATH. One has to explicitly specify where Pnut lives. Although, in earlier incarnations of Ubuntu, I do vaguely remember Bash used to use Wine if a Windoze exec was specified. I wouldn't have a clue how that was achieved.
@evanh said:
ah, that's right, Wine doesn't search the PATH. One has to explicitly specify where Pnut lives.
Wine has it's own %PATH%. So you can do wine explorer and it will load explorer.exe from C:\windows\system32 (inside the virtual FS). Might need regedit to add something to it. EDIT: Should be at HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Environment\PATH
@evanh said:
ah, that's right, Wine doesn't search the PATH. One has to explicitly specify where Pnut lives.
Wine has it's own %PATH%. So you can do wine explorer and it will load explorer.exe from C:\windows\system32 (inside the virtual FS). Might need regedit to add something to it.
Fair enough. I've got it compiling from the shell now anyway. I'll try to keep the history alive.
@Rayman said:
Ok, downloaded PNut v46 and it works. Thanks.
Speed seems slower than the C version?
That's all down to using Kye's FAT32 filesystem layer. I expect FSRW to be somewhat faster but probably still won't be a patch on Eric's code.
One issue is it only reads and writes a single block at a time. There's no performance difference between a buffer size of 512 bytes and 64 kB. I imagine it wouldn't be too hard to remedy but that's how "FAT32" object is currently written. Or not. It uses an intermediate buffer for everything. It does a lot of unnecessary shuttling for block transfers.
First thing I'd do is trash the readWriteBlock() extraneous wrapper around the driver interface. That's just bloat. Oh, ah, I see, that's there to handle errors so the bulk of the code doesn't need any checks. Huh, I've not read enough code like that before.
Well, I've cleaned out all of the DAT section variables. Moved the few that were actually in use into VAR section. A ton of them simply weren't used, I don't know why. So "FAT32" object should be instantiable now. ie: Usable for copying between two mounted volumes.
Oh, wow, I'd clean forgotten the control logic details in rx_datablocks(). I was starting to think I had a lingering bug that wasn't causing any issue other than some excess overhead. I'd not clearly noted one detail and it took a while of reading over the code to recognise and then remember how I'd managed to squeeze it all into the C/Z flag reuse.
This is written for my posterity really.
Requirements:
One routine to handle 1 to N consecutive blocks. It's big, don't want to duplicate for single-block receive for example.
Handles busy timeout (card not ready)
CRC processing in parallel with streamer DMA to hubRAM. Critical performance feature.
Final block has no space for CRC nibbles in the designated buffer, CRC nibbles have to be diverted to scratch memory instead
Things are muchly complicated by the fact that CRC processing is lagged by one block. It can only be processed after a block is already received. This lag has to be tracked to (A) skip the CRC processing step during first block transfer, and (B) process the final CRC after completion of streamer ops.
It would have been easier rereading my source code if I'd been accepting to use extra state variables and branching instructions. But I found, with effort, I could just squeeze it all into the C/Z flags. So I did.
Now I've added a couple more notes for later perusal too. eg:
crc_check
' check CRC of prior read SD block
' Z is clear upon first block, returns quickly with Z set
...
It was this CRC processing during first block transfer that I thought wasn't been skipped. The call to the subroutine still happens after all. What I hadn't noted anywhere is that it also returns quickly because of the initial clear state of Z.
Z stays set after this point unless there a subsequent CRC error.
@Rayman said:
Have you compared speeds with the 1 bit version?
It probably makes more difference than I thought it would. "FAT32" is very slow. Using "FAT32" for both, sdsd_dma.spin2 is around twice as fast as sdspi_bashed2.spin2.
Comments
Rayman,
Drop this in as updated driver. It's not any bug fixes or added features, although it does now support the new block devices interface, it's mostly just a tidy up and a little less trigger happy on recalibrating. I think it should work fine with the older Flexprop versions.
EDIT: Err, there is one line you'll need to change: Replace
#include "../fatfs/diskio.h"with#include "diskio.h"EDIT2: I've removed one diagnostic line that wasn't still meant to be there.
EDIT3: Correction to above comment: Huh, there is a new feature. I managed to forget about the adding of pin registration toggle to runtime calibration function. This makes it more rugged if using sysclock/2 for speed when CRC processing is disabled in the block read routine.
EDIT4: UPDATED driver! v0.4 has a bug. Oopsie, I hadn't completed the removal of SELECT/DESELECT for block read/write calls. I haven't had any read/write errors of late so it never triggered the bug. I'd missed out adjusting the recalibration step which requires the card deselected to perform. Removal of select/deselect is a small optimisation with SD operating mode, as opposed to SPI mode. In SD mode the card doesn't much care about select/deselect for normal block read/write. SD cards are assumed a point-to-point exclusive link, whereas SPI/MM modes have provision for a shared bus to multiple devices.
v0.5 replacement attached
Ouch! Nowhere does FSRW deal with a read/write error. It just ploughs on as if no error occurred! That's gotta be prone to corruption of the stored filesystem as well as the files themselves.
Do you mean FSRW doesn’t do crc check ? That could be.
If your code has it, a definite advantage then…
Does FatFs do crc checks?
Do you check writes for errors somehow?
I mean when the low level block read, or write, reports back that a CRC failed the check FSRW then just ignores the error and continues as if all is fine. No attempt at even one retry nor any further accommodating like attempting to adjust the file metadata and FAT tables.
The low level block functions don't do retries themselves.
Maybe Kye’s fat32 driver is better? It uses the same low level block driver…
LOL, nothing happened ... I found a tester that uses Kye's FAT32 and patched my driver to it and it compiled first try surprisingly. Running the tester, said everything happened but the activity LED did nothing, which shouldn't possible, and upon inspecting the card's content I see no file created. I guess I need to see how that tester works now ... or add in the debug() messages to the spin driver. I didn't port those over to save some mental effort at the time.
Oh-no! TRUE is exclusively -1 in Spin. That's going to be some work to fix up. Hang on I'm using the wrong operator ... 43 operators later ...
Going to have to reorganise the top level code to use debug() before I can dig deeper. It's currently using SmartSerial.spin2 ...
Bah! More operator problems. This time bit-shift precedence is way higher in Spin than C.
Got all the way to start of block reading ... now it's gone into a black hole in the pasm code.
Hmm, the program crashes when compiled with Flexspin, but not when compiled with Pnut ...
Yay, Pnut compiled edition is completing its run and looks okay at least. It clearly isn't verifying anything though.
Sounds like progress!
Think inline assembly is vastly different between the two…
Yes! Progress for sure.
I've avoided using the differences so it should compile as is in both environments. After all, the same Pasm mechanisms are used in the original C version of the driver.
EDIT: I've worked out that, when compiled with Flexspin, the "inline" Pasm2 code isn't getting stuck/crashing directly. It successfully exits back to the compiled Spin2 code of its function,
rx_datablocks()but the function itself never returns.EDIT2: Oh, duh! The new compile line had lost the
--fcache=256. Man, there needs to be a way for a program to specify that. Here's the Flexspin compiled edition:Can you specify fcache with enum?
Be nice if was a way in code…
Don't think so. I'll see what Eric says to my pleas.
PS: Here it is as it stands. I've not rewritten the tester program yet so no verifying is done. I've modified init steps to take more pin parameters but otherwise, excepting the new driver, it's mostly unchanged. I've also altered use of SmartSerial to include a line-feed with the carriage-returns. This fits better with debug() when pairing them under Flexspin.
PPS: The Spin edition of the driver doesn't attempt to select High Speed access mode. It was one of a number of items initially dropped in the name of getting it going. I've only just remembered it's still missing. However, I'm not sure it's worth adding though. It never benefited the C edition speed wise and had really only become a case of completeness since it was already done there.
What's with the CRs as new lines in two of these source files in the zipfile? Usually you have either LF, or CRLF. This is like some old Mac text format. Need to do some file cleanup. Is this generated from PNut/PropellerTool or something else?
That's remnants of Chip's Propeller Serial Terminal. I missed a couple I guess. Yes, people using PropTool often use PST.
Updated tester program:
@evanh Doesn't compile with PropTool or FlexProp? What does this work with?
Ok, looks like need to update FlexProp...
Hmm.. Seems have the latest release... Does one need to build from source to use this?
LOL, yeah, good point. I'm using Pnut's new "structures" that Chip finalised in the last couple of weeks. Needs both Pnut v45 and spanking new build of Flexspin.
I only did it like that for labelling convenience. Same reason I'd used C structures too. I could've just used some arrays of longwords.
Pnut is awkward by itself too. It's debug terminal is useless for anything other than debug. I guess I should get in the habit of using it from the shell instead of GUI ... ah, that's right, Wine doesn't search the PATH. One has to explicitly specify where Pnut lives. Although, in earlier incarnations of Ubuntu, I do vaguely remember Bash used to use Wine if a Windoze exec was specified. I wouldn't have a clue how that was achieved.
EDIT: Okay, not too hard. Now changed to using arrays. And done enough documenting to make it readable later I think.
EDIT2: Updated again below - https://forums.parallax.com/discussion/comment/1563777/#Comment_1563777
Ok, downloaded PNut v46 and it works. Thanks.
Speed seems slower than the C version?
Wine has it's own %PATH%. So you can do
wine explorerand it will load explorer.exe from C:\windows\system32 (inside the virtual FS). Might need regedit to add something to it. EDIT: Should be at HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Environment\PATHFair enough. I've got it compiling from the shell now anyway. I'll try to keep the history alive.
That's all down to using Kye's FAT32 filesystem layer. I expect FSRW to be somewhat faster but probably still won't be a patch on Eric's code.
BTW, you can go back to using the older FlexProp and PropTool now.
@evanh Getting this with FlexProp and similar in PropTool:
C:/Prop2/SDIO/sdsd_dma.spin2:435: error: unknown identifier bytecomp used in function call
Oh, I guess that's newer too ... yep, bytecomp() was added with Pnut v44 ... I'll fix that too ...
Right, checked with Pnut v39 now.
One issue is it only reads and writes a single block at a time. There's no performance difference between a buffer size of 512 bytes and 64 kB. I imagine it wouldn't be too hard to remedy but that's how "FAT32" object is currently written. Or not. It uses an intermediate buffer for everything. It does a lot of unnecessary shuttling for block transfers.
First thing I'd do is trash the
readWriteBlock()extraneous wrapper around the driver interface. That's just bloat. Oh, ah, I see, that's there to handle errors so the bulk of the code doesn't need any checks. Huh, I've not read enough code like that before.Well, I've cleaned out all of the DAT section variables. Moved the few that were actually in use into VAR section. A ton of them simply weren't used, I don't know why. So "FAT32" object should be instantiable now. ie: Usable for copying between two mounted volumes.
Oh, wow, I'd clean forgotten the control logic details in
rx_datablocks(). I was starting to think I had a lingering bug that wasn't causing any issue other than some excess overhead. I'd not clearly noted one detail and it took a while of reading over the code to recognise and then remember how I'd managed to squeeze it all into the C/Z flag reuse.This is written for my posterity really.
Requirements:
Things are muchly complicated by the fact that CRC processing is lagged by one block. It can only be processed after a block is already received. This lag has to be tracked to (A) skip the CRC processing step during first block transfer, and (B) process the final CRC after completion of streamer ops.
It would have been easier rereading my source code if I'd been accepting to use extra state variables and branching instructions. But I found, with effort, I could just squeeze it all into the C/Z flags. So I did.
Now I've added a couple more notes for later perusal too. eg:
It was this CRC processing during first block transfer that I thought wasn't been skipped. The call to the subroutine still happens after all. What I hadn't noted anywhere is that it also returns quickly because of the initial clear state of Z.
Z stays set after this point unless there a subsequent CRC error.
Thanks @evanh will test in a sec.
Have you compared speeds with the 1 bit version?
It probably makes more difference than I thought it would. "FAT32" is very slow. Using "FAT32" for both, sdsd_dma.spin2 is around twice as fast as sdspi_bashed2.spin2.
Here's the new tester with SPI pinout:
Huh, some cards don't mount easily using sdspi_bashed2. Looks like that driver might need some work in the init area.
Works in PropTool now! Thanks @evanh
Guess should make some of these boards soon...
What was the speed before?