FSRW: She's So Unusual
rokicki
Posts: 1,000
[noparse][[/noparse]fsrw22 replaced by fsrw23; readahead and writebehind gives better speed;
multiple file support.]
Another test release of fsrw23; this one supports multiple files (see serial_terminal.spin
for an example), and it's substantially faster when using pread/pwrite.
On my 100MHz demo board, I now get 1.3MB/sec writes, 1.6MB/sec reads when using
pread/pwrite; the raw reads and writes are above 2MB/sec.
We are approaching a blessed release. Give this one a spin and see if you can break it.
Post Edited (rokicki) : 7/25/2009 4:52:29 AM GMT
multiple file support.]
Another test release of fsrw23; this one supports multiple files (see serial_terminal.spin
for an example), and it's substantially faster when using pread/pwrite.
On my 100MHz demo board, I now get 1.3MB/sec writes, 1.6MB/sec reads when using
pread/pwrite; the raw reads and writes are above 2MB/sec.
We are approaching a blessed release. Give this one a spin and see if you can break it.
Post Edited (rokicki) : 7/25/2009 4:52:29 AM GMT
zip
48K
Comments
Here's my results with a brand new Kinsgston 2GB card with my PSM (Propeller System Module):
(Note to PSM users...· You need to change SD pins from 0,1,2,3 to 3,2,1,0)
Cr*p!· PST (Propeller Serial Terminal)·won't let me copy to clipboard!
Here's a screen shot:
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
My Prop Info&Apps: ·http://www.rayslogic.com/propeller/propeller.htm
Post Edited (Rayman) : 7/16/2009 5:36:37 PM GMT
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
My Prop Info&Apps: ·http://www.rayslogic.com/propeller/propeller.htm
Yes, this new version supports files up to 2GB, and (on FAT32) up to nearly 1024
files. It will not currently read or extend the root directory past one cluster, so
that's why you can only use slightly less than 1K files. I plan to remove this
limitation in the next release.
I don't know how much effort I plan to put in to make it support files larger than
2GB; does anyone really need this? FAT itself only supports up to 4GB, so you
only gain a factor of two. While it should not be terribly difficult to make it work,
the problem is the testing; it takes a while to read or write more than 2GB, and
as we add features (seeking for instance) that all needs to be tested again.
Anyway, great results on the PST!
I switched to PropViewer specifically so I could Copy the results to the clipboard (though it seems to grab the entire window, instead of just the text I highlighted). And yes, this version lets you mount MMC or SD or SDHC cards larger than 2GB, FAT16 or FAT32. Individual files larger than 2GB are still not supported (see the caveats.html file in the zip).
My results on my 80MHz system (I'll report results from my 100MHz system later for the 2 uSD cards)
Kodak 2GB SD
Clock: 80000000 ClusterSize: 32768 ClusterCount: 62409
Raw write 3968 kB in 2626 ms at 1511 kB/s
Raw read 1920 kB in 2053 ms at 934 kB/s
fsrw pwrite 2016 kB in 2875 ms at 701 kB/s
fsrw pread 2016 kB in 3250 ms at 620 kB/s
FSRW pputc 63 kB in 2134 ms at 29 kB/s
FSRW pgetc 63 kB in 1896 ms at 33 kB/s
Canon 32MB MMC-Plus
Clock: 80000000 ClusterSize: 4096 ClusterCount: 7830
Raw write 3968 kB in 2832 ms at 1401 kB/s
Raw read 1920 kB in 2048 ms at 937 kB/s
fsrw pwrite 480 kB in 2260 ms at 212 kB/s
fsrw pread 480 kB in 793 ms at 605 kB/s
FSRW pputc 63 kB in 2477 ms at 25 kB/s
FSRW pgetc 63 kB in 1892 ms at 33 kB/s
SanDisk (Ultra II) 2GB SD
Clock: 80000000 ClusterSize: 32768 ClusterCount: 62022
Raw write 3968 kB in 2621 ms at 1513 kB/s
Raw read 1920 kB in 2050 ms at 936 kB/s
fsrw pwrite 2016 kB in 3190 ms at 632 kB/s
fsrw pread 2016 kB in 3246 ms at 621 kB/s
FSRW pputc 63 kB in 2149 ms at 29 kB/s
FSRW pgetc 63 kB in 1891 ms at 33 kB/s
SanDisk 256MB SD
Clock: 80000000 ClusterSize: 32768 ClusterCount: 7718
Raw write 3968 kB in 2846 ms at 1394 kB/s
Raw read 1920 kB in 2217 ms at 866 kB/s
fsrw pwrite 2016 kB in 2816 ms at 715 kB/s
fsrw pread 2016 kB in 3419 ms at 589 kB/s
FSRW pputc 63 kB in 2143 ms at 29 kB/s
FSRW pgetc 63 kB in 1895 ms at 33 kB/s
PNY 4GB uSDHC
Clock: 80000000 ClusterSize: 32768 ClusterCount: 120804
Raw write 3968 kB in 2869 ms at 1383 kB/s
Raw read 1920 kB in 2049 ms at 937 kB/s
fsrw pwrite 2016 kB in 3457 ms at 583 kB/s
fsrw pread 2016 kB in 3295 ms at 611 kB/s
FSRW pputc 63 kB in 2572 ms at 24 kB/s
FSRW pgetc 63 kB in 1938 ms at 32 kB/s
SanDisk 2GB uSD
Clock: 80000000 ClusterSize: 32768 ClusterCount: 61981
Raw write 3968 kB in 2656 ms at 1494 kB/s
Raw read 1920 kB in 2049 ms at 936 kB/s
fsrw pwrite 2016 kB in 2782 ms at 724 kB/s
fsrw pread 2016 kB in 3245 ms at 621 kB/s
FSRW pputc 63 kB in 2131 ms at 29 kB/s
FSRW pgetc 63 kB in 1890 ms at 33 kB/s
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
lonesock
Piranha are people too.
Keep up the good work!
Jim
Firstly, thanks guys, for without FSRW there wouldn't be sooooo much stuff going on with Prop, in a million and one directions now.
Here's my results, on an Integral 1GB SD running on a Hybrid @ 96Mhz.
Clock: 96000000 ClusterSize: 16384 ClusterCount: 62089
Raw write 3968 kB in 2385 ms at 1663 kB/s
Raw read 3968 kB in 3532 ms at 1123 kB/s
fsrw pwrite 2016 kB in 3782 ms at 533 kB/s
fsrw pread 2016 kB in 2714 ms at 742 kB/s
FSRW pputc 127 kB in 3666 ms at 34 kB/s
FSRW pgetc 127 kB in 3174 ms at 40 kB/s
Cheers,
Baggers.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
http://www.propgfx.co.uk/forum/·home of the PropGFX Lite
·
Do you require pullups on the lines?
I was able to run the old speed test, but not the new one.
I write to my Proto board all the time as a GPS Data logger, basepin 0. This test using PST as I/O.
Thanks, Jim
Pull-up resistors are recommended by the SD (and MMC) spec, but not required. This timeout tells me that during the initialization process the card never left BUSY mode for 4 seconds, which is a huge timeout. Could you power cycle your board, then run test.spin using PropViewer, and copy & paste the full log into a PM to me? I'll try to figure out what's going on.
Thanks for your help!
Jonathan
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
lonesock
Piranha are people too.
Just downloaded PropView, so I could capure a log, ·which required .NET 3.5 upgrade·which·just screwed up my laptop. Well I've got some work to do before I get back to this.
Question is: I am running a plain Demo board with uSD at Base Pin 0 and PST for I/O. Same for my Proto board.
Do I need to change anything in sysdep.spin other than the base pin setup? There is mention of "If you do not have an input mechanism, make the rx method return " " the first time it is called and make it sit in an infinite loop all subsequent times (so the test is run exactly once)."
Also my "CD" pin is floating right now. Do I need to tie high/low? Have not had to up to now.
I'm·dealing with·the flu right now so these problems may be virus related, ie not thinking straight.
Jim
Just found a loose socket on the demo board (VDD to SD Socket.. Gee THAT wont' affect anything [noparse]:)[/noparse] Test results will follow.
Jim
Post Edited (hover1) : 7/16/2009 11:55:36 PM GMT
It didn't work at first... Was hanging on "Mount_Explicit"... Then tried the old code, then back to the new code and now it works!
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
My Prop Info&Apps: ·http://www.rayslogic.com/propeller/propeller.htm
Anyway, I'm delighted with a 65% speed bump in a "real" application.
on failures, so if anyone sees any sort of failure, that screen capture could be very useful.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Visit the home of pPropQL: propeller.wikispaces.com/pPropQL
pPropQL020: propeller.wikispaces.com/pPropQL
OMU for the pPropQL/020 propeller.wikispaces.com/OMU
I'm using pread ('cause I've been to lazy to figure that block read stuff out!)
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
My Prop Info&Apps: ·http://www.rayslogic.com/propeller/propeller.htm
Jonathan
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
lonesock
Piranha are people too.
Jonathan
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
lonesock
Piranha are people too.
what is going on.
And yes, longword-aligned buffers and read requests do help performance. Maybe in the PropII we can get a
memmove() command that takes care of the needed shifting (and RMW at the ends) so we can get good
performance even on bytemove (note that bytemove itself is not very fast).
Feel free to use additions I added to updated version (v1.7) as per this
obex.parallax.com/objects/402/
Added SDstr,SDbin,SDhex and SDdec to fsrw to allow simple writing of Strings, Decimal, Binary and Hexadecimal values to SDcard.(Submitted to Obex Jan 2009)
if additions are deemed worthy of course ...
Rgds,
John
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
'Necessity is the mother of invention'
'Those who can, do.Those who can’t, teach.'
'Convince a man against his will, he's of the same opinion still.'
·
(Note that the size issue is especially important as the multiblock access code is necessarily larger, and the initialization is more complex (both for getting a card out of multiread or multiwrite mode, and for handling MMC and SD and SDHC cards). I do plan to add a tiny version of the block layer driver, without unrolled loops, and perhaps using only single block mode.)
thanks,
Jonathan
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
lonesock
Piranha are people too.
Can you please plan for having a boot function in the assembly code using a digit from "0" to "7" as the command code? Before the read starts, it would do a COGSTOP on the cog # provided as the command code, then do the read just like the "r" command, then drop through to a Spin boot routine like in sdspiFemto.spin (starting at "nowBootSpin", line 730). Note that sdspiFemto.spin has the ability to act like it's loading a program, but without changing memory. This was used for testing purposes and isn't really needed.
I'm not sure that I fully understand, but I'd be happy to discuss it. Send me a PM? I think it might be easiest to have the loading from SD to RAM buffer controlled via Spin, then simply have the reboot code by itself inside the PASM.
Jonathan
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
lonesock
Piranha are people too.
Rokicki:· I'm not quite sure I'd call the video player a "real world" app, but I do appreciate the increased speed!
I think now that I can comfortably do film speed at 24 fps with decent audio...
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
My Prop Info&Apps: ·http://www.rayslogic.com/propeller/propeller.htm
(There is also one thing I'm working on: 1-instruction per bit reads. So far I have 2 failed schemes. 1 fails on all systems at PLL16X, but works for all other cases, and 2 works on one system at PLL16X (at 100MHz no less), but fails on other systems (even at the lower 80 MHz). On the system where it works, I get raw reads and writes at the same speed, just over 2MB/s at 100 MHz, but I do not want to distribute code that works on 1 out of 4 tests systems.)
Thanks for testing!
Jonathan
P.S. Anyone else have some throughput numbers to post?
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
lonesock
Piranha are people too.
If you followed my other post I've just almost completed a FAT32/16 file system driver for everyone to use.
My block driver is in spin however so its very slow right now. And for some reason the first block write fails always after powerup.... Odd but after that it seems to work while being slow.
I've got every feature implemented except for long file names. That reminds me, I need to ask how format should work. I'll go update that post right now.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Nyamekye,
No problems at all - everything seemed to run great on a professional dev board with a homemade SD socket plugged into the breadboard.
1G Sandisk SD
Clock: 80000000 ClusterSize: 16384 ClusterCount: 61970
Raw write 3968 kB in 2739 ms at 1448 kB/s
Raw read 1920 kB in 2049 ms at 936 kB/s
fsrw pwrite 2016 kB in 3708 ms at 543 kB/s
fsrw pread 2016 kB in 3257 ms at 619 kB/s
FSRW pputc 63 kB in 2136 ms at 29 kB/s
FSRW pgetc 63 kB in 1891 ms at 33 kB/s
256 MB Sandisk SD
Clock: 80000000 ClusterSize: 16384 ClusterCount: 15434
Raw write 3968 kB in 2604 ms at 1523 kB/s
Raw read 1920 kB in 2048 ms at 937 kB/s
fsrw pwrite 2016 kB in 3568 ms at 565 kB/s
fsrw pread 2016 kB in 3259 ms at 618 kB/s
FSRW pputc 63 kB in 2256 ms at 27 kB/s
FSRW pgetc 63 kB in 1890 ms at 33 kB/s
128 MB Panasonic SD
Clock: 80000000 ClusterSize: 2048 ClusterCount: 62591
Raw write 3968 kB in 3107 ms at 1277 kB/s
Raw read 1920 kB in 2058 ms at 932 kB/s
fsrw pwrite 224 kB in 2954 ms at 75 kB/s
fsrw pread 224 kB in 395 ms at 566 kB/s
FSRW pputc 63 kB in 2765 ms at 22 kB/s
FSRW pgetc 63 kB in 1907 ms at 33 kB/s
I have a request. What about allowing users to supply a date when creating a file? You could have FSRW initialize a long as the default date, but provide a function to give callers the option of overriding it with their own date.
I made a GPS logger that did this - since GPS provide the date and time, I built a long and passed it to a modified FSRW so the logged files would have a reasonable timestamp, using code like this. The first part was run every second to generate a date for the logged data; the second part ran when the logfile was first opened, using the built-up datetimestring to generate a long in the MS-DOS timestamp style.
David
Post Edited (David : 7/17/2009 1:28:18 PM GMT
Fantastic work guys.
Ross has my working TriBlade as I have not had the time to build one for him · Hopefully Ross will test this on a TriBlade for you.
BTW: I posted a fix for sdspiFemto so that the SD card would release D0 to tristate after each access. It was only a·simple fix, but probably not the best/fastest method. This is necessary as I share the pins with the sram pins, and this is the same for my RamBlade when I have time to finish the pcb. This method is required for the fastest ram access without latches.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:
· Home of the MultiBladeProps: TriBladeProp, RamBlade, TwinBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: Micros eg Altair, and Terminals eg VT100 (Index) ZiCog (Z80), MoCog (6809)
· Search the Propeller forums (via Google)
My cruising website is: ·www.bluemagic.biz·· MultiBladeProp is: www.bluemagic.biz/cluso.htm
Yes, I'll try testing this on Cluso's TriBlade over the weekend. I already have fsrw working on the TriBladeProp (I use it in my generic program loader) so this should be quite simple.
I'll post figures in a day or so.
Ross.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Catalina - a FREE C compiler for the Propeller - see Catalina
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:
· Home of the MultiBladeProps: TriBladeProp, RamBlade, TwinBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: Micros eg Altair, and Terminals eg VT100 (Index) ZiCog (Z80), MoCog (6809)
· Search the Propeller forums (via Google)
My cruising website is: ·www.bluemagic.biz·· MultiBladeProp is: www.bluemagic.biz/cluso.htm
I have never had a problem copying text from PST, I do it all the time, I often will copy values into excell for graphs. You just can't right click....CTRL+C instead.
Also..dont highlight past what your readout is..if that makes any sense.
Thanks Again for all the effort being put into media reading/writing!
Rick
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
1GB Sandisk SD
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Mount tests first
First mount.
Succeeded; stopping cog.
Second mount.
Succeeded.
Reading block 0 (should be a boot block)
Read finished; checking for boot block signature
Boot block checks out; unmounting
Third mount.
Succeeded.
Reading block 0 again (should still be a boot block)
Read finished; checking for boot block signature
Boot block checks out; writing it back
Write finished; unmounting
Fourth mount.
Succeeded.
Reading block 0 again (should still be a boot block)
Read finished; checking for boot block signature
Block layer seems to check out
Now speed tests
How fast can we write, sequentially?
Raw write 3968 kB in 2608 ms at 1521 kB/s
Do a single non-sequential write...Done
How fast can we read, sequentially?
Raw read 1920 kB in 2049 ms at 937 kB/s
Now the filesystem tests
Trying to mount
Mounted.
How fast can we write using pwrite?
fsrw pwrite 992 kB in 2168 ms at 457 kB/s
How fast can we read using pread?
fsrw pread 992 kB in 1606 ms at 617 kB/s
How fast can we write using pputc?
FSRW pputc 63 kB in 2229 ms at 28 kB/s
How fast can we read using pgetc?
FSRW pgetc 63 kB in 1891 ms at 33 kB/s
Repeating all the speed results:
Clock: 80000000 ClusterSize: 16384 ClusterCount: 61977
Raw write 3968 kB in 2608 ms at 1521 kB/s
Raw read 1920 kB in 2049 ms at 937 kB/s
fsrw pwrite 992 kB in 2168 ms at 457 kB/s
fsrw pread 992 kB in 1606 ms at 617 kB/s
FSRW pputc 63 kB in 2229 ms at 28 kB/s
FSRW pgetc 63 kB in 1891 ms at 33 kB/s
All done!
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
PQI 256MB SD
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Mount tests first
First mount.
Succeeded; stopping cog.
Second mount.
Succeeded.
Reading block 0 (should be a boot block)
Read finished; checking for boot block signature
Boot block checks out; unmounting
Third mount.
Succeeded.
Reading block 0 again (should still be a boot block)
Read finished; checking for boot block signature
Boot block checks out; writing it back
Write finished; unmounting
Fourth mount.
Succeeded.
Reading block 0 again (should still be a boot block)
Read finished; checking for boot block signature
Block layer seems to check out
Now speed tests
How fast can we write, sequentially?
Raw write 3968 kB in 2706 ms at 1466 kB/s
Do a single non-sequential write...Done
How fast can we read, sequentially?
Raw read 1920 kB in 2048 ms at 937 kB/s
Now the filesystem tests
Trying to mount
Mounted.
How fast can we write using pwrite?
fsrw pwrite 32 kB in 2325 ms at 13 kB/s
How fast can we read using pread?
fsrw pread 32 kB in 53 ms at 606 kB/s
How fast can we write using pputc?
FSRW pputc 1 kB in 2151 ms at 0 kB/s
How fast can we read using pgetc?
FSRW pgetc 1 kB in 30 ms at 34 kB/s
Repeating all the speed results:
Clock: 80000000 ClusterSize: 4096 ClusterCount: 62387
Raw write 3968 kB in 2706 ms at 1466 kB/s
Raw read 1920 kB in 2048 ms at 937 kB/s
fsrw pwrite 32 kB in 2325 ms at 13 kB/s
fsrw pread 32 kB in 53 ms at 606 kB/s
FSRW pputc 1 kB in 2151 ms at 0 kB/s
FSRW pgetc 1 kB in 30 ms at 34 kB/s
All done!
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Kingston 4GB microSD in SD Adapter
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Mount tests first
First mount.
Succeeded; stopping cog.
Second mount.
Succeeded.
Reading block 0 (should be a boot block)
Read finished; checking for boot block signature
Boot block checks out; unmounting
Third mount.
Succeeded.
Reading block 0 again (should still be a boot block)
Read finished; checking for boot block signature
Boot block checks out; writing it back
Write finished; unmounting
Fourth mount.
Succeeded.
Reading block 0 again (should still be a boot block)
Read finished; checking for boot block signature
Block layer seems to check out
Now speed tests
How fast can we write, sequentially?
Raw write 3968 kB in 2641 ms at 1502 kB/s
Do a single non-sequential write...Done
How fast can we read, sequentially?
Raw read 1920 kB in 2054 ms at 934 kB/s
Now the filesystem tests
Trying to mount
Mounted.
How fast can we write using pwrite?
fsrw pwrite 2016 kB in 3506 ms at 575 kB/s
How fast can we read using pread?
fsrw pread 2016 kB in 3252 ms at 619 kB/s
How fast can we write using pputc?
FSRW pputc 63 kB in 2335 ms at 26 kB/s
How fast can we read using pgetc?
FSRW pgetc 63 kB in 1894 ms at 33 kB/s
Repeating all the speed results:
Clock: 80000000 ClusterSize: 32768 ClusterCount: 120770
Raw write 3968 kB in 2641 ms at 1502 kB/s
Raw read 1920 kB in 2054 ms at 934 kB/s
fsrw pwrite 2016 kB in 3506 ms at 575 kB/s
fsrw pread 2016 kB in 3252 ms at 619 kB/s
FSRW pputc 63 kB in 2335 ms at 26 kB/s
FSRW pgetc 63 kB in 1894 ms at 33 kB/s
All done!
Waiting for key press to start tests
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
NYC Area Prop Club
Prop Forum Search (Via Google)
Post Edited (CassLan) : 7/17/2009 11:10:16 AM GMT