Shop OBEX P1 Docs P2 Docs Learn Events
FSRW: She's So Unusual - Page 5 — Parallax Forums

FSRW: She's So Unusual

1235710

Comments

  • lonesocklonesock Posts: 917
    edited 2009-07-28 00:26
    Baggers said...
    lonesock, I put a release command after the pclose and it works now with mp_rawb_spi [noparse]:D[/noparse]

    Thanks,
    Baggers.
    Great! Thanks for testing.

    OK, I just committed 2 changes to the fsrw repository dealing with this issue.
    1 - release is called from pclose
    2 - release is automatically called from the block layer after 125ms of inactivity (you can edit this in the start_explicit routine if you wish)

    Also, if you are using pullup resistors, you can uncomment the line:
    mov dira,#0
    


    in the "release_DO" PASM routine. In this case, after exiting the current multiblock mode, the prop will deselect the card and send out 16 clocks, (hopefully) letting the card tristate it's DO line. (This is for you, Cluso99!)

    So, basically the routines are now fairly bulletproof. If you have a reset within 125ms of a read or write that was _NOT_ closed with pclose, and your card is one of the select few that has issues with this, you could require a power cycle to re-initialize the card.

    I'm attaching two Spin files for the eager among you, and as always you can find the latest code here: fsrw.hg.sourceforge.net/hgweb/fsrw/.

    We will do some more testing, then push out another test release zip.

    Thanks again, everybody.
    Jonathan

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    lonesock
    Piranha are people too.
  • BaggersBaggers Posts: 3,019
    edited 2009-07-28 08:49
    No worries, glad to help out, and thanks for the update! [noparse]:D[/noparse]

    Baggers.

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    http://www.propgfx.co.uk/forum/·home of the PropGFX Lite

    ·
  • Cluso99Cluso99 Posts: 18,069
    edited 2009-07-28 09:24
  • YorkBWYorkBW Posts: 6
    edited 2009-07-29 01:04
    Just got my microSD card hooked up today, and first tried FSRW17 (from OBEX) and got very slow read/write...so I found this thread and tried FSRW23 -- much better! I'm having a weird problem though: the card will only mount every *other* power cycle on the board. Doesn't matter what I do, "odd" power-ups mount, "even" ones don't ("fails spectacularly"). Anyone have any idea what might cause this? (No pullups, just the 4 wires to the Propeller [noparse][[/noparse]I'm using pins 24-27] and Vcc / GND connections.)

    EDIT: Uncommenting the debug code dumps "0 => 1, 8 => 1" followed by a slew of "55 => 1, 41 => 1" messages on the failed mount, but I don't know if that's helpful. Also, even falling back to FSRW16 gives the same behavior (i.e., every other power-up has mount failures).

    Post Edited (YorkBW) : 7/29/2009 1:25:01 AM GMT
  • rokickirokicki Posts: 1,000
    edited 2009-07-29 02:08
    Howdy!

    Are you sure you mean *power cycle* and not *prop program download* or some such?

    Unless something very odd is going on, behavior after a *power cycle* (where you actually
    switch off, then switch on, the power) should be pretty deterministic.

    That said, can you tell us about your card? Brand name, size? Would you be willing to try
    one of the other block drivers? Are you using pullups?

    We really, really want this to work every single time. I'll upload another version soon that
    has some enhancements by lonesock that should help remedy this, but it's important to
    understand what's going on.
  • lonesocklonesock Posts: 917
    edited 2009-07-29 13:45
    @Baggers & Cluso99: You guys are welcome, of course!
    YorkBW said...
    Just got my microSD card hooked up today, and first tried FSRW17 (from OBEX) and got very slow read/write...so I found this thread and tried FSRW23 -- much better! I'm having a weird problem though: the card will only mount every *other* power cycle on the board. Doesn't matter what I do, "odd" power-ups mount, "even" ones don't ("fails spectacularly"). Anyone have any idea what might cause this? (No pullups, just the 4 wires to the Propeller [noparse][[/noparse]I'm using pins 24-27] and Vcc / GND connections.)

    EDIT: Uncommenting the debug code dumps "0 => 1, 8 => 1" followed by a slew of "55 => 1, 41 => 1" messages on the failed mount, but I don't know if that's helpful. Also, even falling back to FSRW16 gives the same behavior (i.e., every other power-up has mount failures).
    Hi. Thanks for the error report, and for going the extra mile to enable the Command=>Response logging! The log shows the following:

    Reset: worked
    Are you a SD card, type 2: yes
    Initialize while checking for the High Capacity bit (i.e. are you a SDHC card?): stays busy for 4 seconds then times out (assuming it returned -1000?)

    About 5 posts back I posted an updated fsrw and mb_rawb_spi. I would recommend starting by using those files and repeating your tests. Then if the problem persists there are 2 things I would try:

    1 - this card may just need an obscene amount of time for the reset...at the bottom of the "PRI read_slow" routine, change the (clkfreq << 2) to a (clkfreq << 4) and see if it just needed more time.

    2 - According to the spec, a SDHC card can return busy (by returning a 1) if ACMD41 came before a CMD8 (which is not the case here) or if ACMD41 is sent without the high capacity bit (also not the case). It may be possible that if your card is _not_ a SDHC card (i.e. the size is 2GB or less), it may not be following the spec, and will not initialize if the HC bit _is_ set. We can bypass the SDHC init section by changing the following:
    ' Is this a SD type 2 card?
      if send_cmd_slow( CMD8, $1AA, $87 ) == 1
    


    to
    ' Is this a SD type 2 card?
      if false
    


    I am not sure what to expect in this case, as it goes against the spec.

    I would definitely try the timeout fix before the initialization hack. Partly because if #2 works for you, the code would not be able to mount SDHC cards. It would help to know the card type and size. I use uSD and uSDHC cards on a custom PCB using the exact same pins [noparse][[/noparse]24..27], running the prop at 100MHz, so I don't think the setup is a problem.

    Jonathan

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    lonesock
    Piranha are people too.
  • YorkBWYorkBW Posts: 6
    edited 2009-07-29 19:21
    @rokicki, Yes, I meant *power cycle* (remove power from board, re-apply power to board). I thought it was very strange, too. Particularly that it's *deterministic* in that it fails every *other* time. BTW: yes, the EEPROM is "empty" (DoNothing.spin). The card's a SanDisk 2GB microSD (not SDHC) in SparkFun's microSD breakout board. As I stated above, no pullups; the only slightly unusual thing is I'm using P24-P27 on the Propeller. Note the FSRW16 code also failed to mount the card every other boot, so I'm pretty sure the problem is mine, not FSRW's. I was posting this issue to see if anyone else has encountered it before or has any idea why I'm getting this weird behavior.

    @lonesock, I'd already updated to your latest fsrw/mb_rawb_spi before posting, but I'll try the fixes you suggest tonight. Again, the frustrating thing is that it works as-is...exactly 50% of the time.

    Thanks for the quick feedback!
  • YorkBWYorkBW Posts: 6
    edited 2009-07-29 23:53
    OK, I'm sure it's not a code problem at all.

    If the mount / tests work, and then I turn off the power on my breadboard (from the PE Kit), then the power LED still glows faintly from the power supplied through the USB connection. If I turn it off after a failed mount, the LED does *not* glow. Unplugging the microSD card from the +3.3V rail and re-plugging gets the LED glow back, and makes the mount operation work. So I guess power off isn't *really* power off, with the USB Prop Plug always plugged in. Something about the residual (low) voltage on the rails must be messing up the uSD card. (Power off, disconnect Prop Plug, reconnect, power on also works every time.)

    I thought I could get away with connecting the card's VCC to +3.3V, GND to ground, and the 4 control pins directly to the Propeller, with no pull-up/down resistors anywhere. Do I have the wiring wrong? I haven't had luck finding a schematic anywhere...
  • rokickirokicki Posts: 1,000
    edited 2009-07-30 16:37
    In *theory* you don't need pullups (a 10K pullup on each of the four control lines if you want them). I
    always run with pullups; I hate having CMOS inputs floating.

    On the residual power---it is possible that is sufficient to keep the card from resetting. And based on
    what you are reporting with the LED, it sounds like the SD card is pulling power after a failed mount.

    I can envision a scenario where feeding a card undervoltage for a bit and then bringing the voltage
    back up to 3.3 might put the card into a state where it will not respond (properly); essentially, you may
    be confusing the card's internal power-on reset circuitry.

    So for me, I'd put *this* one down as a problem caused by parasitic power from the USB.

    If you don't power cycle, but just re-download (say) or reset the prop, do mounts work reliably?

    Thanks; I think we are making some progress on this!
  • YorkBWYorkBW Posts: 6
    edited 2009-07-30 17:44
    @rokicki, Yes, re-download or reset without any power cycle works, too. It's clearly just a problem with keeping the Prop Plug attached; I know better now. (Voltage across the uSD card was about 1.7V from the Prop Plug alone...probably confusing it!)

    Anyway...the latest FSRW is working flawlessly for me, now I know how to treat the hardware!!!!
  • lonesocklonesock Posts: 917
    edited 2009-07-30 18:03
    YorkBW said...
    @rokicki, Yes, re-download or reset without any power cycle works, too. It's clearly just a problem with keeping the Prop Plug attached; I know better now. (Voltage across the uSD card was about 1.7V from the Prop Plug alone...probably confusing it!)

    Anyway...the latest FSRW is working flawlessly for me, now I know how to treat the hardware!!!!
    Excellent!

    Jonathan

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    lonesock
    Piranha are people too.
  • lonesocklonesock Posts: 917
    edited 2009-07-31 06:34
    Hi, everybody!

    We're getting close to finalizing the block layer for the official release, and I have 2 favors to ask.

    1 - Please report any errors or instabilities you get with the absolute latest mb_rawb_spi.spin driver. We want this release to be rock solid. (I wouldn't mind seeing some of the pwrite/pread speed numbers you are getting either [noparse][[/noparse]8^)

    2 - A quick poll. I want to support as only those block layers which are necessary, and I was thinking of 5 possibilities:

    MB_fast: multiblock readahead writebehind
    MB_tiny: multiblock smaller no readahead or writebehind, slower
    SB_fast: singleblock readahead writebehind
    SB_tiny: singleblock smaller, no readahead or writebehind, slower
    SB_spin: Spin only, singleblock only, really slow

    Things to keep in mind:
    * singleblock writes are glacially slow on SDHC cards
    * I really don't want to have to support that many block layers.

    The MB_fast version will definitely be included, but please vote on the additional block layers if you think you would actually use it in a product. Just post with a list of which ones you'd like. In the absence of any votes I'll probably just do MB_fast and MB_tiny.

    thanks,
    Jonathan

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    lonesock
    Piranha are people too.
  • Peter JakackiPeter Jakacki Posts: 10,193
    edited 2009-07-31 07:13
    Well Jonathan & Tom, I must say that I have been greatly impressed and heartened by your efforts. Previously I didn't have a lot of faith in the SD side of things with the Propeller as it seemed rather slow and buggy. It seemed safer to stick to my ARM processors for this stuff rather than investing time in trying to make the Prop perform.

    Having used your recent code (but not the most recent) it seems that I could actually seriously use this stuff. Take for instance a wave player I knocked together with which I was happy to just to play back files but now I can contemplate incorporating more advanced features that will allow me not only to record but also clone cards without waiting a month of Sundays for the thing to finish.

    I'll try to do my bit and provide as much info as I can but I also want to build in multiple file access. Is this something you have been considering at all? Would save me the time and I know you will do a thoroughly good job [noparse]:)[/noparse]

    What's amazing is that we all have to eat yet for the better part we are seeing all kinds of voluntary efforts being put forth, with many at far above the amateur level resulting in some real quality stuff. This has got to be the best microcontroller forum in the world! The guys at Parallax just seem to foster this response simply by their open and enthusiastic attitude toward all things Prop'er (thanks a million guys).

    *Peter*

    Post Edited (Peter Jakacki) : 7/31/2009 7:21:01 AM GMT
  • Cluso99Cluso99 Posts: 18,069
    edited 2009-07-31 07:29
    Well said Peter smile.gif

    Jonathan,
    I presume MB_tiny will be capable of just doing 1 block anyway. If correct, then MB_fast & MB_tiny would be fine. However, if not the case, then please include SB_tiny as well. Sorry, I have not had time to look at your code - maybe next week with a bit of luck.

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    Links to other interesting threads:

    · Home of the MultiBladeProps: TriBladeProp, RamBlade, TwinBlade,·SixBlade, website
    · Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
    · Prop Tools under Development or Completed (Index)
    · Emulators: Micros eg Altair, and Terminals eg VT100 (Index) ZiCog (Z80), MoCog (6809)
    · Search the Propeller forums (via Google)
    My cruising website is: ·www.bluemagic.biz·· MultiBladeProp is: www.bluemagic.biz/cluso.htm
  • max72max72 Posts: 1,155
    edited 2009-07-31 09:00
    Test report from my two bigger cards:

    Demo board with SD card reader from ucontroller.
    fsrw2.3 with fsrw and mb_rawb_spi overwritten with versions posted Posted 7/27/2009.

    Sandisk SDHC ultra II 16 Gb

    Clock: 80000000 ClusterSize: 32768 ClusterCount: 499410
    Raw write 3968 kB in 2586 ms at 1534 kB/s
    Raw read 3968 kB in 2348 ms at 1690 kB/s
    fsrw pwrite 4064 kB in 3866 ms at 1051 kB/s
    fsrw pread 4064 kB in 3212 ms at 1265 kB/s
    FSRW pputc 63 kB in 2096 ms at 30 kB/s
    FSRW pgetc 63 kB in 1838 ms at 34 kB/s
    All done!

    Sandisk SDHC 4 Gb

    Clock: 80000000 ClusterSize: 32768 ClusterCount: 120754
    Raw write 3968 kB in 2534 ms at 1565 kB/s
    Raw read 3968 kB in 2347 ms at 1690 kB/s
    fsrw pwrite 4064 kB in 3912 ms at 1038 kB/s
    fsrw pread 4064 kB in 3212 ms at 1264 kB/s
    FSRW pputc 63 kB in 2263 ms at 27 kB/s
    FSRW pgetc 63 kB in 1838 ms at 34 kB/s
    All done!
  • lonesocklonesock Posts: 917
    edited 2009-07-31 10:06
    Peter Jakacki said...
    Well Jonathan & Tom, I must say that I have been greatly impressed and heartened by your efforts. Previously I didn't have a lot of faith in the SD side of things with the Propeller as it seemed rather slow and buggy. It seemed safer to stick to my ARM processors for this stuff rather than investing time in trying to make the Prop perform.

    Having used your recent code (but not the most recent) it seems that I could actually seriously use this stuff. Take for instance a wave player I knocked together with which I was happy to just to play back files but now I can contemplate incorporating more advanced features that will allow me not only to record but also clone cards without waiting a month of Sundays for the thing to finish.

    I'll try to do my bit and provide as much info as I can but I also want to build in multiple file access. Is this something you have been considering at all? Would save me the time and I know you will do a thoroughly good job [noparse]:)[/noparse]

    What's amazing is that we all have to eat yet for the better part we are seeing all kinds of voluntary efforts being put forth, with many at far above the amateur level resulting in some real quality stuff. This has got to be the best microcontroller forum in the world! The guys at Parallax just seem to foster this response simply by their open and enthusiastic attitude toward all things Prop'er (thanks a million guys).

    *Peter*
    Thanks, Peter!

    Not only are we considering it, multiple file access is already working! The new block layer code is a singleton, and fsrw has been adapted to have a minimal VAR footprint, but enough to keep separate files working. Here's an example:
    CON
      ' file handling, using a 8KB buffer
      buf_len = 8 * 1024
      ' SD pins
      sd_base = 24
      sd_DO = 0+sd_base
      sd_CLK = 1+sd_base
      sd_DI = 2+sd_base
      sd_CS = 3+sd_base
    
    OBJ
      ' declare however many files you need open at once
      sdfat: "fsrw"
    
    VAR
      long file_buf[noparse][[/noparse]buf_len >> 2] ' force this to be long aligned, and #longs = #bytes/4
    
    PUB start | tmp, i
      ' mount the first sdfat only
      sdfat.mount_explicit( sd_DO, sd_CLK, sd_DI, sd_CS )
    
      ' do an example file copy
      tmp := sdfat.popen( string( "copyfrom.txt" ), "r" )
      if tmp => 0
         tmp := sdfat[noparse][[/noparse] 1].popen( string( "copyto.txt" ), "w" )
      repeat while tmp > -1
        i := @file_buf
        tmp := sdfat.pread( i, buf_len )
        if tmp > 0
           sdfat[noparse][[/noparse] 1].pwrite( i, tmp )
    
      ' close both files
      sdfat.pclose
      sdfat[noparse][[/noparse] 1].pclose
    
      ' unmount the first sdfat only
      sdfat.unmount
    
    


    That code was hacked out of the "serial_terminal.spin" file up in the repo, so I apologize for any errors during the ensamplification [noparse][[/noparse]8^) Also, this is a nifty way to do something with every file in the directory (since the .opendir and the .nextfile commands tie up a single fsrw object the same as opening a file).

    Thanks for all you kind words!
    Jonathan

    <edit> Eegad! bracket 1 bracket in the code section...how hard can that be?

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    lonesock
    Piranha are people too.

    Post Edited (lonesock) : 7/31/2009 10:40:47 AM GMT
  • lonesocklonesock Posts: 917
    edited 2009-07-31 10:28
    Cluso99 said...
    Well said Peter smile.gif

    Jonathan,
    I presume MB_tiny will be capable of just doing 1 block anyway. If correct, then MB_fast & MB_tiny would be fine. However, if not the case, then please include SB_tiny as well. Sorry, I have not had time to look at your code - maybe next week with a bit of luck.
    All versions of the block layer will have exactly the same API and functionality. The multiblock vs. singleblock only refers to the data transfer method to the card, as defined in the SD and MMC specs.

    In singleblock mode, the card does not presume to know the address of the next block to be written or read. You simple say "Read X", "Write Y", and it does for that block only. The trick is that flash memory has to be erased before it can be written, and the erase block size is typically much larger than the write block size (e.g. 32kB and 512 bytes, respectively). So, especially on SDHC cards, when you write a single block, the card has to flush an entire erase block, copy in all the other blocks, except the one section of 512 bytes you specified, then finally it can write the 512 bytes you just handed it.

    To get around this issue, the MMC and SD and SDHC cards have what is called multiblock mode. You specify that you want to read or write starting at address X, then begin reading or writing as many blocks as you want, then terminate the transaction with a "all done" type message. This allows the card to start pre-fetching the next block during reads, and to hold off on the erasure and copying during writes, and so is much faster (especially on SDHC cards where the erase blocks are huge, comparatively).

    Typically you would write all the blocks at once, but the prop doesn't have RAM to waste, so I trick the card by leaving it in multiblock mode, but transferring only one block at a time. Between blocks I give the card a few clock pulses to let it complete I/O handshaking, then shut down the clock. This is allowed by the spec, and has worked on all the cards we tried it on. The downside is there is some added complexity to manage the multiblock state (the read or write mode must be exited and entered anytime the fsrw layer changes from read to write (or "release"), or anytime the next block to be read/written is not immediately following the previous block. This is all handled transparently by the driver.

    One last pair of tricks I do for speed: Read-ahead asks for the block that was just requested, passes it up to the Hub RAM and says "done" to fsrw. However, instead of going to sleep, it asks the card for the next block in sequence and transfers it to a Cog buffer. If the next request happens to be a block read for that next address, great! We just copy up the data we already stored in the cog buffer, say "done" again, then ask for the _next_ block from the card. If the request is not a "read X+1" request, we flush the buffer and do the requested operation.

    Write-behind is similar, and simpler. When you ask the block layer to write a block, it first copies the data down into the same cog buffer, then says "done" to fsrw. Then the cog goes about doing the actual card write. Both these tricks leverage the parallel nature of the propeller to hide some card latency and communication overhead, all without increasig the Hub RAM requirements.

    Jonathan

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    lonesock
    Piranha are people too.

    Post Edited (lonesock) : 7/31/2009 10:37:57 AM GMT
  • lonesocklonesock Posts: 917
    edited 2009-07-31 10:33
    max72 said...
    Test report from my two bigger cards:

    Demo board with SD card reader from ucontroller.
    fsrw2.3 with fsrw and mb_rawb_spi overwritten with versions posted Posted 7/27/2009.

    Sandisk SDHC ultra II 16 Gb
    ...
    fsrw pwrite 4064 kB in 3866 ms at 1051 kB/s
    fsrw pread 4064 kB in 3212 ms at 1265 kB/s
    ...
    Sandisk SDHC 4 Gb
    ...
    fsrw pwrite 4064 kB in 3912 ms at 1038 kB/s
    fsrw pread 4064 kB in 3212 ms at 1264 kB/s
    ...
    Thanks for sharing! Using fsrw, better than 1 MB/s transfers at 80MHz..that's what I like to see! [noparse][[/noparse]8^)

    Jonathan

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    lonesock
    Piranha are people too.
  • Cluso99Cluso99 Posts: 18,069
    edited 2009-07-31 11:10
    Jonathan, thanks for the explanation.
    What you have done is really neat, but unfortunately the gains will not work with my hardware designs because I share the pins with sram :-( Oh for a few more i/o pins !

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    Links to other interesting threads:

    · Home of the MultiBladeProps: TriBladeProp, RamBlade, TwinBlade,·SixBlade, website
    · Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
    · Prop Tools under Development or Completed (Index)
    · Emulators: Micros eg Altair, and Terminals eg VT100 (Index) ZiCog (Z80), MoCog (6809)
    · Search the Propeller forums (via Google)
    My cruising website is: ·www.bluemagic.biz·· MultiBladeProp is: www.bluemagic.biz/cluso.htm
  • KyeKye Posts: 2,200
    edited 2009-07-31 14:11
    Eghaz!!!

    I'm gonna have a fun time porting your block driver. Seems like it will be challenge. Luckiliy I'll only need to take the fastest one.

    ...

    What do you think about having the block driver re-intialize the SD card however? My simple spin bock driver does this now anyways and it works quite nicely. It seems like an auto intialize feature in the block driver saves headaches.

    ...

    Also, is the actuall bandwith of the block driver fully untilized? Can spin actually produce 1MB of data to be read back and forth from the SD card? I'm asking this because I may slow down the driver so that it works just fast enough for the spin code to use it at full speed and nothing more.

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    Nyamekye,
  • lonesocklonesock Posts: 917
    edited 2009-07-31 15:58
    Cluso99 said...
    Jonathan, thanks for the explanation.
    What you have done is really neat, but unfortunately the gains will not work with my hardware designs because I share the pins with sram :-( Oh for a few more i/o pins !
    When you call pclose, the driver releases the pins and has the card tristate the DO pin. So it should work for you (I intentionally added in this feature thinking of your design). Just uncomment the "mov dira,#0" line in the "release_DO" routine in the block driver. I do not tristate the pins by default because not everyone uses pullups. Please try out this functionality for me?

    Jonathan

    <edit>You can also call the block layer's "release" routine manually, if for some reason you do not wish to pclose the files you are working on before switching the pin usage. The very next read or write will grab control of the pins again and resume where it left off.

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    lonesock
    Piranha are people too.

    Post Edited (lonesock) : 7/31/2009 4:34:26 PM GMT
  • lonesocklonesock Posts: 917
    edited 2009-07-31 16:29
    Kye said...
    Eghaz!!!

    I'm gonna have a fun time porting your block driver. Seems like it will be challenge. Luckiliy I'll only need to take the fastest one.

    ...

    What do you think about having the block driver re-intialize the SD card however? My simple spin bock driver does this now anyways and it works quite nicely. It seems like an auto intialize feature in the block driver saves headaches.

    ...

    Also, is the actuall bandwith of the block driver fully untilized? Can spin actually produce 1MB of data to be read back and forth from the SD card? I'm asking this because I may slow down the driver so that it works just fast enough for the spin code to use it at full speed and nothing more.
    You mean initialize the card before every block read or written? Seems overkill to me. And if you want fast you'll have to port the init code to PASM, which will take up much more space than in Spin, obviously.

    Regarding the transfer rate, I have a hobby project where I will definitely need full bandwidth, in fact I may be forced to use the raw block driver to write 2MB/s contiguously, and only copy over the file to a FAT32 partition at the end. Of course all data processing is performed in PASM. I'm saving local data plus data coming in over an ENC28J60 chip, so there's quite enough to keep the fsrw routines busy.

    Jonathan

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    lonesock
    Piranha are people too.
  • ericballericball Posts: 774
    edited 2009-07-31 17:51
    2 - A quick poll. I want to support as only those block layers which are necessary, and I was thinking of 5 possibilities:

    MB_fast: multiblock readahead writebehind
    MB_tiny: multiblock smaller no readahead or writebehind, slower
    SB_fast: singleblock readahead writebehind
    SB_tiny: singleblock smaller, no readahead or writebehind, slower
    SB_spin: Spin only, singleblock only, really slow

    Things to keep in mind:
    * singleblock writes are glacially slow on SDHC cards
    * I really don't want to have to support that many block layers.

    The MB_fast version will definitely be included, but please vote on the additional block layers if you think you would actually use it in a product. Just post with a list of which ones you'd like. In the absence of any votes I'll probably just do MB_fast and MB_tiny.
    Are there cases (i.e. random read/write block) where the singleblock (or tiny) version is noticably better than MB_fast?· How much difference in HUB RAM consumption is there between the versions?· Each requires one cog in addition to the filesystem code, right?· Or does SB_spin execute inline?

    For my Z machine interpreter project I will be balancing HUB RAM consumption versus speed of random block read/writes.

    Oh, and I assume the readahead/writebehind code will handle the case where a random read/write happens before the readahead/writebehind has completed.


    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    Composite NTSC sprite driver: Forum
    NTSC & PAL driver templates: ObEx Forum
    OnePinTVText driver: ObEx Forum
  • lonesocklonesock Posts: 917
    edited 2009-08-01 00:12
    ericball said...
    Are there cases (i.e. random read/write block) where the singleblock (or tiny) version is noticably better than MB_fast? How much difference in HUB RAM consumption is there between the versions? Each requires one cog in addition to the filesystem code, right? Or does SB_spin execute inline?

    For my Z machine interpreter project I will be balancing HUB RAM consumption versus speed of random block read/writes.

    Oh, and I assume the readahead/writebehind code will handle the case where a random read/write happens before the readahead/writebehind has completed.
    Hi!

    * There is a tiny bit more overhead for multiblock compared to singleblock because you have to send the stop command/token. There is also bit more overhead in the multiblock state handler. So in the case of random 512-byte reads, singleblock will be a tiny bit faster (say 10%). But everything else will be slower, especially on SDHC cards. Now the readahead optimization is a bad idea for random reads, as I will keep wasting time requesting the next block from the card. Whether this impact performance or not depends on the overhead of your Spin routines, as the pre-fetch is happening in the driver cog without any supervision needed from fsrw (or the calling process in general).

    * I'm betting the MB_tiny will be about 25% slower, but that is a wild guess. It should save approx 60 longs. Again a guess, but not so wild.

    * Yes it does. The driver cog will not handle a new request until it's done processing whatever is currently going on. The calling Spin code will just spin waiting for the command to be acknowledged.

    thanks,
    Jonathan

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    lonesock
    Piranha are people too.
  • ericballericball Posts: 774
    edited 2009-08-01 01:17
    10% for SB vs MB isn't enough to worry about, so I'd just stick with just the MB version.· Put in a way for the caller to turn off readahead and that problem is handled too.· (That might even be good for fsrw, I bet there's code which "knowns" it's only going to need the one block.)· 60 longs?· Hrmm.. 240 bytes, half a block... that's almost enough to make it desirable.


    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    Composite NTSC sprite driver: Forum
    NTSC & PAL driver templates: ObEx Forum
    OnePinTVText driver: ObEx Forum
  • hinvhinv Posts: 1,255
    edited 2009-08-09 18:39
    I know I am a bit late to the party, but I was excited to get the results I did with the new 100_000_000 clock:
    1GB used pqi uSD:
    Clock: 80000000 ClusterSize: 16384 ClusterCount: 60942
    Raw write 3968 kB in 2334 ms at 1700 kB/s
    Raw read 3968 kB in 2373 ms at 1672 kB/s
    fsrw pwrite 2016 kB in 2102 ms at 958 kB/s
    fsrw pread 2016 kB in 1599 ms at 1260 kB/s
    FSRW pputc 63 kB in 2097 ms at 30 kB/s
    FSRW pgetc 63 kB in 1839 ms at 34 kB/s
    
    Clock: 100000000 ClusterSize: 16384 ClusterCount: 60942
    Raw write 8064 kB in 3765 ms at 2142 kB/s
    Raw read 8064 kB in 3857 ms at 2091 kB/s
    fsrw pwrite 4064 kB in 3902 ms at 1041 kB/s
    fsrw pread 4064 kB in 2579 ms at 1575 kB/s
    FSRW pputc 127 kB in 3514 ms at 36 kB/s
    FSRW pgetc 127 kB in 2965 ms at 42 kB/s
    
    2GB new SanDisk uSD:
    Clock: 80000000 ClusterSize: 32768 ClusterCount: 61982
    Raw write 3968 kB in 2317 ms at 1712 kB/s
    Raw read 3968 kB in 2351 ms at 1687 kB/s
    fsrw pwrite 4064 kB in 3899 ms at 1042 kB/s
    fsrw pread 4064 kB in 3225 ms at 1259 kB/s
    FSRW pputc 63 kB in 2099 ms at 30 kB/s
    FSRW pgetc 63 kB in 1839 ms at 34 kB/s
    
    Clock: 100000000 ClusterSize: 32768 ClusterCount: 61982
    Raw write 8064 kB in 3805 ms at 2118 kB/s
    Raw read 8064 kB in 3824 ms at 2109 kB/s
    fsrw pwrite 4064 kB in 3105 ms at 1308 kB/s
    fsrw pread 4064 kB in 2556 ms at 1589 kB/s
    FSRW pputc 127 kB in 3378 ms at 37 kB/s
    FSRW pgetc 127 kB in 2964 ms at 42 kB/s
    
    



    Great work guys! I am really impressed.
    BTW, this is way better performance than my 624MHz Del Axim X51v does on a typical file copy.

    Thanks,
    Doug
  • Bill HenningBill Henning Posts: 6,445
    edited 2009-08-09 19:17
    WOW!

    Great results doug...

    Bill
    hinv said...
    I know I am a bit late to the party, but I was excited to get the results I did with the new 100_000_000 clock:
    1GB used pqi uSD:
    Clock: 80000000 ClusterSize: 16384 ClusterCount: 60942
    Raw write 3968 kB in 2334 ms at 1700 kB/s
    Raw read 3968 kB in 2373 ms at 1672 kB/s
    fsrw pwrite 2016 kB in 2102 ms at 958 kB/s
    fsrw pread 2016 kB in 1599 ms at 1260 kB/s
    FSRW pputc 63 kB in 2097 ms at 30 kB/s
    FSRW pgetc 63 kB in 1839 ms at 34 kB/s
    
    Clock: 100000000 ClusterSize: 16384 ClusterCount: 60942
    Raw write 8064 kB in 3765 ms at 2142 kB/s
    Raw read 8064 kB in 3857 ms at 2091 kB/s
    fsrw pwrite 4064 kB in 3902 ms at 1041 kB/s
    fsrw pread 4064 kB in 2579 ms at 1575 kB/s
    FSRW pputc 127 kB in 3514 ms at 36 kB/s
    FSRW pgetc 127 kB in 2965 ms at 42 kB/s
    
    2GB new SanDisk uSD:
    Clock: 80000000 ClusterSize: 32768 ClusterCount: 61982
    Raw write 3968 kB in 2317 ms at 1712 kB/s
    Raw read 3968 kB in 2351 ms at 1687 kB/s
    fsrw pwrite 4064 kB in 3899 ms at 1042 kB/s
    fsrw pread 4064 kB in 3225 ms at 1259 kB/s
    FSRW pputc 63 kB in 2099 ms at 30 kB/s
    FSRW pgetc 63 kB in 1839 ms at 34 kB/s
    
    Clock: 100000000 ClusterSize: 32768 ClusterCount: 61982
    Raw write 8064 kB in 3805 ms at 2118 kB/s
    Raw read 8064 kB in 3824 ms at 2109 kB/s
    fsrw pwrite 4064 kB in 3105 ms at 1308 kB/s
    fsrw pread 4064 kB in 2556 ms at 1589 kB/s
    FSRW pputc 127 kB in 3378 ms at 37 kB/s
    FSRW pgetc 127 kB in 2964 ms at 42 kB/s
    
    



    Great work guys! I am really impressed.
    BTW, this is way better performance than my 624MHz Del Axim X51v does on a typical file copy.

    Thanks,
    Doug
    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    Please use mikronauts _at_ gmail _dot_ com to contact me off-forum, my PM is almost totally full
    Morpheus & Mem+ Advanced dual Propeller SBC with XMM and 256 Color VGA - PCB, kit, A&T available NOW!
    www.mikronauts.com - my site 6.250MHz custom Crystals for running Propellers at 100MHz
    Las - Large model assembler for the Propeller Largos - a feature full nano operating system for the Propeller
  • hinvhinv Posts: 1,255
    edited 2009-08-09 19:50
    Yeah, I was wondering about that. Lonesock had slightly better results on a 80MHz clock than I did, but our results are far and away better than the others.
    What could account for this large difference in the results? I know the 6.25 crystals make a difference, but other than that, there is still a large difference.

    I am using a demo board with the sparkfun microSD card adapter. I soldered some surface mount pullup resisters on the underside to make a nice neat package and get rid of the jungle of 10k resisters on my breadboard.
    I should have tried it before this to see if there was any difference.
    I haven't yet used the updates from page5 of this thread, just fsrw23.zip

    Thanks,
    Doug
  • hinvhinv Posts: 1,255
    edited 2009-08-09 20:35
    New better result on pqi 1GB microSD (after formatting):

    Clock: 100000000 ClusterSize: 16384 ClusterCount: 60941
    Raw write 8064 kB in 3726 ms at 2163 kB/s
    Raw read 8064 kB in 3857 ms at 2090 kB/s
    fsrw pwrite 4064 kB in 3823 ms at 1062 kB/s
    fsrw pread 4064 kB in 2577 ms at 1577 kB/s
    FSRW pputc 127 kB in 3376 ms at 37 kB/s
    FSRW pgetc 127 kB in 2964 ms at 42 kB/s
  • Cluso99Cluso99 Posts: 18,069
    edited 2009-08-27 06:39
    Jonathan,
    Finally here are my results on the TriBlade using a SanDisk 2GB microSD card...
    Clock: 80000000 ClusterSize: 32768 ClusterCount: 60282
    Raw write 3968 kB in 2288 ms at 1734 kB/s
    Raw read 3968 kB in 2353 ms at 1685 kB/s
    fsrw pwrite 2016 kB in 2574 ms at 783 kB/s
    fsrw pread 2016 kB in 1594 ms at 1264 kB/s
    FSRW pputc 63 kB in 2464 ms at 25 kB/s
    FSRW pgetc 63 kB in 1841 ms at 34 kB/s
    All done!
    
    

    I am curious that the pwrite is slower than other reports and also that my tests are using 2016 kb instead of 4096 kb. Just in case, I downloaded the latest code today.

    Do you have any comments?? (PS I changed the code to write to 800 * 2048 so as not to destroy data on the card)

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    Links to other interesting threads:

    · Home of the MultiBladeProps: TriBladeProp, RamBlade, TwinBlade,·SixBlade, website
    · Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
    · Prop Tools under Development or Completed (Index)
    · Emulators: Micros eg Altair, and Terminals eg VT100 (Index) ZiCog (Z80) , MoCog (6809)
    · Search the Propeller forums·(uses advanced Google search)
    My cruising website is: ·www.bluemagic.biz·· MultiBladeProp is: www.bluemagic.biz/cluso.htm
Sign In or Register to comment.