Secure digital read/write code released
rokicki
Posts: 1,000
{{
fsrw 1.0 by Radical Eye Software
This is secure digital and FAT16 read/write code.
For connectivity, see "sdspifasm.spin". You only need four
Propeller lines and six pullup resistors and some sort of SD or
microSD or miniSD socket.
We only support the root directory at the moment, and only one
file open at a time. The file can be open for read, write, or
append. We also support delete (open a file in mode "d"), as well
as a traversal of the root directory.
No long file name support. No (real) date support yet. No
formatting utility yet.
I've tested it on 10 different secure digital cards, and it works
on all of them. There is some variance in the speed. For reads,
I see speeds of between 271 to 304 KBytes/sec. For writes, I see
speeds of between 114 to 228 KBytes/sec, depending on the card.
This is with full file system overhead, and the file system code
is completely SPIN. If you use only the block I/O layer you can do
better than this; if you write small files or flush frequently you
will see worse than this.
I am supplying three different block level implementations. You
should start with the "sdspi" code which is slow and completely in
spin. If that works, move up to "sdspiasm" and then "sdspifasm"
which increases the frequencies the card is driven at. All cards
should support the fastest mode, but you have to make sure your
wires are not terribly long and stuff like that. You change the
block layer used in the object section of the "fsrw.spin" source.
I've tested this code on cards in their virgin (out of the bubble
pack) state, and when formatted with XP, and when formatted with
Win2K, and when formatted with the Canon XTi camera, and when
formatted with the Canon SD400 camera. No problems in any case.
This code will *only* read FAT16, not FAT12 and not FAT32. This means
the number of clusters on your volume must be greater than 4096 and
less than 65530 (or thereabouts). Clusters can range from 512 bytes to
32K (they must be a power of two in size), so this means we can support
cards from 4M up to 2G.
When you format your card, you can use XP to do it, but you have to pick
a cluster size that will make the card end up in FAT16 (which means the
cluster count must satisfy the constraints above). For performance, the
larger the cluster size, the better (since fewer metadata writes and
reads are required). This does introduce some additional space loss due
to fragmentation, but with the large cards available now, this should not
be a problem.
The following table shows the range of cluster sizes you can select in
XP when formatting in order to guarantee a FAT16 volume. The number
of clusters will be the size of the card divided by the cluster size
(approximately).
min max
4M 512 512
8M 512 1K
16M 512 2K
32M 512 4K
64M 1K 8K
128M 2K 16K
256M 4K 32K
512M 8K 32K
1G 16K 32K
2G 32K 32K
If you do not select the right cluster size, your volume may be formatted
in FAT12 or FAT32, neither of which is supported by this code.
FAT16 does not support volumes greater than 4G. If you must use a 4G
card, you will have to ensure the first volume is 2G or less, and this
code will only mount that first volume. In reality this is not much of
an issue, because this code only supports the SD protocol, and cards
of 4G or larger require the new SDHC protocol. (I have no way at this
time to know if an SDHC card will default to the SD protocol for the
first portion of the card.)
}}
OBJ
fsrw: "fsrw" ' the main FAT16 reading and writing code
sdspi: "sdspi" ' low-level SD block I/O in spin (slow)
sdspiasm: "sdspiasm" ' low-level SD block I/O in simple asm
sdspifasm: "sdspifasm" ' low-level SD block I/O in faster asm
fsrw_speed: "fsrw_speed" ' a test of speed
sdrw_test: "sdrw_test" ' a simple example of use
I have removed the file attachment, and instead am pointing everyone
to use this URL:
http://tomas.rokicki.com/fat16work/
which will always have the latest and greatest as well as status.
Post Edited (rokicki) : 12/29/2006 6:40:36 PM GMT
fsrw 1.0 by Radical Eye Software
This is secure digital and FAT16 read/write code.
For connectivity, see "sdspifasm.spin". You only need four
Propeller lines and six pullup resistors and some sort of SD or
microSD or miniSD socket.
We only support the root directory at the moment, and only one
file open at a time. The file can be open for read, write, or
append. We also support delete (open a file in mode "d"), as well
as a traversal of the root directory.
No long file name support. No (real) date support yet. No
formatting utility yet.
I've tested it on 10 different secure digital cards, and it works
on all of them. There is some variance in the speed. For reads,
I see speeds of between 271 to 304 KBytes/sec. For writes, I see
speeds of between 114 to 228 KBytes/sec, depending on the card.
This is with full file system overhead, and the file system code
is completely SPIN. If you use only the block I/O layer you can do
better than this; if you write small files or flush frequently you
will see worse than this.
I am supplying three different block level implementations. You
should start with the "sdspi" code which is slow and completely in
spin. If that works, move up to "sdspiasm" and then "sdspifasm"
which increases the frequencies the card is driven at. All cards
should support the fastest mode, but you have to make sure your
wires are not terribly long and stuff like that. You change the
block layer used in the object section of the "fsrw.spin" source.
I've tested this code on cards in their virgin (out of the bubble
pack) state, and when formatted with XP, and when formatted with
Win2K, and when formatted with the Canon XTi camera, and when
formatted with the Canon SD400 camera. No problems in any case.
This code will *only* read FAT16, not FAT12 and not FAT32. This means
the number of clusters on your volume must be greater than 4096 and
less than 65530 (or thereabouts). Clusters can range from 512 bytes to
32K (they must be a power of two in size), so this means we can support
cards from 4M up to 2G.
When you format your card, you can use XP to do it, but you have to pick
a cluster size that will make the card end up in FAT16 (which means the
cluster count must satisfy the constraints above). For performance, the
larger the cluster size, the better (since fewer metadata writes and
reads are required). This does introduce some additional space loss due
to fragmentation, but with the large cards available now, this should not
be a problem.
The following table shows the range of cluster sizes you can select in
XP when formatting in order to guarantee a FAT16 volume. The number
of clusters will be the size of the card divided by the cluster size
(approximately).
min max
4M 512 512
8M 512 1K
16M 512 2K
32M 512 4K
64M 1K 8K
128M 2K 16K
256M 4K 32K
512M 8K 32K
1G 16K 32K
2G 32K 32K
If you do not select the right cluster size, your volume may be formatted
in FAT12 or FAT32, neither of which is supported by this code.
FAT16 does not support volumes greater than 4G. If you must use a 4G
card, you will have to ensure the first volume is 2G or less, and this
code will only mount that first volume. In reality this is not much of
an issue, because this code only supports the SD protocol, and cards
of 4G or larger require the new SDHC protocol. (I have no way at this
time to know if an SDHC card will default to the SD protocol for the
first portion of the card.)
}}
OBJ
fsrw: "fsrw" ' the main FAT16 reading and writing code
sdspi: "sdspi" ' low-level SD block I/O in spin (slow)
sdspiasm: "sdspiasm" ' low-level SD block I/O in simple asm
sdspifasm: "sdspifasm" ' low-level SD block I/O in faster asm
fsrw_speed: "fsrw_speed" ' a test of speed
sdrw_test: "sdrw_test" ' a simple example of use
I have removed the file attachment, and instead am pointing everyone
to use this URL:
http://tomas.rokicki.com/fat16work/
which will always have the latest and greatest as well as status.
Post Edited (rokicki) : 12/29/2006 6:40:36 PM GMT
Comments
I believe Rokicki used this SD card adapter from www.sparkfun.com:
Breakout Board for SD-MMC Cards (SKU#: PCB-SDMMC) = $14.95
If you like to roll you own, I used this SD socket from www.mouser.com
http://www.mouser.com/search/refine.aspx?Ntt=688-SCDA1A0901
Mounted this socket on a little perf board with an 11 pin header so it will plug into a bread board, attached is a couple of photos and the data sheet for the SD socket from mouser.
GREAT WORK ROKICKI!!
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Mike
Post Edited (Mike Cook) : 12/27/2006 11:36:12 AM GMT
suggestions for its improvement (not all of which I've implemented yet).
There are some sneaky counter tricks in the assembly code if anyone is interested. (I use PHSB as a
shift register to shift data out with FRQB=0 in NCO mode, and I use counter A to generate the clock
in several different ways.)
I can make the assembly code faster if anyone needs more speed; I have some preliminary versions
but I'm not yet happy with them so I'm holding them back for the moment.
Here's a picture of the secure digital cards I've tested:
I tend to buy the cheapest cards; it's quite possible more expensive cards would also be faster.
I will also note that I have been trying to wear out a card (so far unsuccessfully) by doing millions of
random writes (random block writes should be the worst case for one of these flash cards).
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Who says you have to have knowledge to use it?
I've killed a fly with my bare mind.
BTY: The pin spacing on the SD socket is .1" & it's a surface mount part. Real easy to make an adapter like the photo in my previous post.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Mike
Post Edited (Mike Cook) : 12/27/2006 6:27:40 PM GMT
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Who says you have to have knowledge to use it?
I've killed a fly with my bare mind.
to pull up all lines that might be in an input state with a resistor between 10K and 100K. Floating inputs on CMOS chips can
cause all sorts of nastiness.
(The propeller itself seems to deal with unconnected inputs reasonably well; anyone know how? Are there tiny pullups or
pulldowns we don't know about? Or is it possible we need to pull up all the unused propeller lines too, or set them to output?)
I would strongly recommend spending the $15 and getting the breakout board from SparkFun; the time it will save you will
more than pay for itself. (I just soldered on a male header so I can plug the breakout board directly into a breadboard.)
For my main project, using the Propeller protoboard which isn't out yet, I will probably go with a microSD socket breakout
board since it is physically smaller and microSD cards are pretty cheap.
SD cards are so commoditized now; the smallest I can find is 512MB, and they go for about $10 to $15 a pop. I could add
MMC support, but I don't want to make the code any larger when SD cards are all available so cheaply. This is the same
reason I only support FAT16; since we can format a card in FAT16, there's no sense wasting code space on FAT12 or FAT32.
Have you tried the 4 bit mode IO? I'm wondering if that's another way to get more speed out of these cards.
David
In Rokicki's archive there is a simple example, you might want to start with that.
This is what I've been doing with this and·I'm not proud of it, but I'll post it anyway. This archive is a 'work in progress' for testing several ideas, with the SD Routines.
What it is, is a data logger, it uses:
1. DS1307 to get time
2. DS1820 to get temp
3. Then writes this mess to a file
Right now it's configured to run for 10 hours. WARNING I have slightly modified TV_TEXT.spin to work similar to the FullDuplexSerial.spin, this way I can easily switch back and forth from the TV and a PC running a terminal program.
If it's too confusing, please delete it!·
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Mike
Post Edited (Mike Cook) : 12/27/2006 8:43:14 PM GMT
Some ideas I've had were:
Back in the olden day's before there was an IBM Home PC, I had a Commodore 64 and a TRS-80 Model 12. A buddy and I wrote a BBS (Bulletin Board Software) kind of like the old WildCat BBS software, but this was before WildCat was written. We used the data storage to feed menus back to the BBS user. Since our software was written in BASIC we did not want to hard code all the User Menus for the BBS, so we made all menus a *.txt file, and just passed them to the user.
Actually If Mike Green can integrate this into his Propeller OS that would be pretty awesome. All sorts of stuff can be done with these routines, let your mind run wild!
<EDIT>
originator: Another Idea! I did a quick search on some of your posts, it looks like your a CNC junkie. I am too, still going through CNC-101 on my home built machine (www.allsurplus.net/CNC). Granted a Full blown PC running a CNC is THE way to go, however you could use these SD routines to read in G-Code to drive a small CNC run from a propeller chip.
</EDIT>
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Mike
Post Edited (Mike Cook) : 12/27/2006 9:49:24 PM GMT
I think this is possible, but it may involve licensing fees. I'm not entirely clear on this.
There are other good ways to get the speed up; moving a bit more into assembly should be
able to double the speeds I posted above, and then using multiblock mode would probably
get you even higher. On the other hand, the speeds I posted above are pretty good. I think
you should have no trouble playing back 44KHz 16-bit audio (assuming you use sufficient
buffering; I suspect as little as 2K buffering would be enough). Recording 44KHz 16-bit
audio would be much trickier and would require some attention to detail (and some cards
give significant delays---I've seen up to 150ms---on some block writes) so more buffering
would be needed, but I think it's possible.
What I want to build with the propeller is a fully self-contained microcomputer with a built-in
BASIC-like interpreter. I want it to drive a TV, have built-in mass storage (the SD card),
use a cheap PC keyboard and mouse, and be buildable for next to nothing. So far the
propellor and the protoboard, plus a few odds and ends, seems to be the best solution.
The goal here is accessible introduction to programming for kids, no PC required. In a
perfect world I'd like to build a few hundred of these and give them away for free. I'm not
sure it's a perfect world, but I'm gonna see how close I can get.
A SD card has so much storage I can have all the documentation right on the device itself
and even provide built-in help. Also, I can swap out the interpreter/compiler (if I choose to
separate the two) fairly quickly, if needed; I can even swap chunks of code out and in as
needed. And I can package the thing with hundreds of small sample programs.
Other than this, though, clearly the SD card can be used for a lot of different things, like
sound storage, image storage, video storage, code storage. You can even put the boot
code in the EEPROM that boots from the SD card, and bingo, you swap out SD cards to
change the functionality of your embedded device without requiring its physical removal
or attachment to a PC. SD cards are cheap enough where you can just send them out by
mail as "software updates" for people using whatever product you have.
I remember when the TRS-80 color computer floppies were only something like 150K, yet
we stored a lot of programs on them. These days a $10 512MB SD card has more than
3,000 times the storage space. Surely we can figure out more things to do with this
than we did with the old 150K storage on the 895KHz Coco.
-tom
Can the card be read/written by the PC and Propeller at the same or separate times?
I like the idea of writing a Gcode interpolator code for the Prop but that is down the road. The PC with Mach3 is my first choice of systems for now. I have an more immediate need to build a proof of concept prototype to read a fingerprint into a PC(30kbytes per print), then transfer that file to any number of Propeller based systems that can store the file for it's own use. Ideally, a method that would allow the PC to capture the print, then send it to each separate system for storage and future use. In theis case the Prop only reads the file, never writes. The PC only writes the file. The fingerprint system is on the way, and communicates serially with any micro or PC. What I am looking for is the method to store the files.
Man I loved that old TRS80. I think I threw it in the trash at some point, got tired of moving it out of the way, had the printer, drive, monitor too.
Post Edited (originator) : 12/27/2006 11:02:06 PM GMT
Note that the secure digital card can only physically be attached to one thing at a time---either the PC or the propeller. Either one can
read or write at any time (you can move the card back and forth) arbitrarily. So you can format the card on the PC, add files,
move it to the propeller, the propeller can read those files, add its own, and back and forth and so on. It's just like a huge floppy disk.
Excellent timing too, I just made a simple mini-SD adapter for myself out of a nine pin Molex right angle header and a mini-SD to SD converter that came with one of my mini-SD cards.
I took a look at the code; interesting use of the timers [noparse]:)[/noparse] I considered trying something like that, but I was worried about getting the timing right - that is, knowing how many cycles I had to start sending/receiving data so that there would be no extra clocks before/after the bits.
Best,
Bill
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
---
www.mikronauts.com - a new blog about microcontrollers
counters when you modify the various registers, and also spent some time checking for glitches. It's very important
to avoid the glitches. Luckily I was able to drive everything from just the counters which simplified some things.
I've got a new version of the block driver coming along, and I am thinking of using the memory space that the
assembly occupies in the object as the sector buffers too, so I can make the overall memory consumption smaller at
the same time I make the whole thing much faster. But doing those sorts of stupid memory tricks can get a bit uglier
than I'd like.
http://mikronauts.com/2006/12/28/cheap-sd-socket/
He just buys a miniSD or microSD which comes with an SD adapter card, and solders directly to the pads on
the adapter card. That's a pretty cool way to go! I don't know how well the plastic will hold up, and I
think at least the microSD isn't *guaranteed* to support SPI (but I bet they all do), but it's a neat idea.
Bill, do share how it goes!
In Tom's fswr.spin, function: pri pdate
I added the following comments and assignments: (just to make it more clear, on how to assign values)·
Next job is to get the date/time from the DS1307 in there!
Having a ball playing with this!·This is like going from cassette to a floppy disk back in the late 70's!
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Mike
Post Edited (Mike Cook) : 12/28/2006 3:44:38 PM GMT
Sean
This has all the formatting and partitioning tools you need. You can right-click on the "removable disk" and ask it to
reformat, and when you reformat, pick the cluster size in order to guarantee it's FAT16 as documented in the first
message in this thread.
A 64MB card can be formatted as FAT12, FAT16, or FAT32 depending on the sectors per cluster. I suspect your camera
may have formatted it to FAT12 with 32K clusters (since this gives the best write performance, and this is what cameras
care most about). If we want it to be FAT16, we need to use a maximum cluster size of 8K (which is still pretty reasonable
and will give pretty good performance). So make sure in the disk management tool that you pick this cluster size.
Otherwise it will default to some value that may not be what we want.
Note that frequently windows will format a card without a partition table (that is, the entire card is a filesystem partition
and there is no MBR with partition information). The code I posted above handles that case too.
If you can get me the first dozen sectors on the card, I can let you know how it's organized.
creation time was not being set correctly.
FYI - Just some notes on using the file DATE/TIME stamp in the SD routines: (See Tom's link for an updated fsrw.spin file)
The ONLY reason the time stamp was implemented was to eliminate the following message from a Windows 'DOS' box:
"If you open a command prompt in Windows (W2K or XP) and do a directory (DIR<ENTER>) you will get "The Parameter is incorrect", before the 'DIR' command displays the file name of each file. Windows is wining about the file date/time not being present and or all ZEROs. If you are only accessing the files written from the Windows GUI then you'll never see this message."
Some tips on using 'time stamps' on your files. Note this assumes that you have access to the current date and time via a DS1307, GPS receiver, or whatever other device that can provide time:
1. To time stamp a file, you will need to assign variables to the pri pdate routine in fsrw.spin BEFORE you open a file for write. By doing this, when reading the directory of the SD on a Windows box you will see the date and time the file was created.
2. If you are appending to a file, the time stamp will be when the file was originally created. The time stamp on the file will not be updated when you are writing in append mode.
I'm speaking for Tom here, so Tom come in and 'slap me' if I'm 'off base' here. I believe Tom's original intent with the SD routines was to be able to add mass storage capability to the propeller with the smallest/fastest code possible.
In all reality if your logging Date/Time stamped data to a file, this data (Date/Time) should be part of each 'record' that is written to the contents of the log file.
Go out and get some SD cards and a socket! I guarantee you'll have fun with this!
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Mike
Today got a USB card reader and a microSD card.
Is seems the Micro SD cards come packaged with a SD adapter, get two cards and you have a spare adapter, so you could solder one up.
Got a long weekend to have fun with this, well done Rokicki.
Gavin
I also posted it to my blog.
Best,
Bill
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
www.mikronauts.com - a new blog about microcontrollers
Are the hardware hookups for this version the same as your read-only version you posted earlier?
Regards
will work fine.
If you want to go faster, you must eliminate the current limiting resistors (I believe they are 150 ohms
in the diagram I showed) because these resistors degrade the signals too much to run very fast.
The file "sdspiasm.spin" contains the hookup for high-speed operation. Again, the only distinction is
that the current limiting resistors are gone.
I doubt you could hurt either the propeller or an SD card no matter how you drive the lines; I put the
current limiters in there just so I could experiment without worrying about it, but now that it seems to
work well I'm pretty comfortable omitting them.
I am pleased this code is proving useful to many and I want to thank all of you who have taken the
time to write.
Tom,
Tried your new:
"News flash: I'm testing out some faster assembly language code. How much faster it is than the old code depends greatly on the card, but for reading it's probably about 40% faster, and for writing, it is probably about 20% faster. "
located in your web link in your first post.
And it does appear to be card dependent. First tried it on one of my old 64Meg SD cards and got random -2xx errors trying to mount the SD card. Though it might be my 5" wiring between my SD socket and the Propeller (I will be shortening that soon!). However just tried it (same setup) with my 1G SD card and all appears to be good, can mount the card, read and write!
Just an FYI, keep up the Good Work!
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Mike
I'd recommend using the sdspifasm stuff for now.
Do you have a scope? (A real, analog scope, not one of those funky logic analyzers some people
call scopes.) If you do, and it's got reasonable resolution, you might take a peek at the signals
at the card. The fastest signals I'm creating are about 10MHz.
The only difference between fasm and qasm should be that qasm has a lot more stuff in
assembly---but the signals themselves should be generated the same way. So a failure is
somewhat troublesome and I'll have to get down to the bottom of it.
-tom
I've got a scope :
2120B Dual Trace BK Precision 30MHz, can borrow a TEK 100MHz digital storage,·from work if need be.
I want to first try shorting the length of the wiring between my SD socket and the Propeller Chip. Will be later this evening or weekend before I can do this.
However just received 6 different SD cards in the mail today (eBay). Will try those in the next hour or so. Right now running a 3600 second test with the "sdspiqasm.spin" low level driver on my existing setup with the 1G "micro-center" card.
<EDIT>
Just wanted to add........
Since 12/23/2006 I've been using: "sdspifasm.spin" as the low level driver, and it's BEEN ROCK SOLID. Keep posting the experimentals!
</EDIT>
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Mike
Post Edited (Mike Cook) : 1/3/2007 7:42:55 PM GMT
More info:
1. The "SanDisk - Shoot&Store Card" 64Meg, the original 64 meg card that I reported problems with, before you released your SD R/W drivers and the problem reported today still fails no mater how I format it & when used with the low level driver "sdspiqasm.spin" will always work with "sdspifasm.spin"
2. The "Micro-Center" 1G SD card works with all versions of low level drivers.
3. The "eBay - KODAK 64Meg SD Cards" once formatted to FAT16 via command prompt in XPSP2 work with the "sdspiqasm.spin" low level driver.
Please remember that the platform I'm testing on is probably the "worst case" scenario, see photo #3 in the second post of this thread. Ken needs to get those "P8X32A-Q44 ProtoBoards" out to us PRONTO!! Could use several right now for various test platforms.
At this point I don't know if you should pull the link to the "sdspiqasm.spin" low level driver biased on my testing. What I would like is for you to email me your address and I will send you this "SanDisk - Shoot&Store Card" 64Meg so you can see what's up with it. Right now I'm thinking that this card is just flaky.
I will continue to use the "eBay - KODAK 64Meg SD Cards" with the "sdspiqasm.spin" low level driver for further testing.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Mike
Just got it working after buying a real SD card socket, made up a bit of vero board with sil pins, soldering socket and 10k pull resistors and tested·it in the demo board.
Next step is to get a DS1307 and try the time stamping code.
I2C pins·same as the boot eeeprom pins, after booting they are free for other I2C chips.
Once that is working I can change my CNC controller design again.
Hmm where's that LCD code, might stick socket on for LCD as alternative option over VGA.
Gavin
http://groups.yahoo.com/group/MicroCNC/
·