FSRW: Walk Right Back (seek, and ye shall find)

rokicki · 2009-09-29 22:35

Read seek is now supported! So now we have very fast I/O including read-ahead and write-behind,
multiple file support, FAT32/FAT16, SD and SDHC, and seek. What more could anyone want.

This is a release candidate. If no one finds any showstopper problems, we will release this code to
the object exchange next week.

Oldbitcollector (Jeff) · 2009-09-29 22:45

rokicki said...

What more could anyone want.

Directory support? [noparse];)[/noparse]

Thanks Rokicki for all your efforts on this! This is a great release!

OBC

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
New to the Propeller?

Visit the: The Propeller Pages @ Warranty Void.

Rayman · 2009-09-29 23:42

Great work!!!!!!!!

But, since you did ask... The only feature I can think of is a disk free space function (i.e., how many bytes are free on the card).
Is that hard?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
My Prop Info&Apps: ·http://www.rayslogic.com/propeller/propeller.htm

James Long · 2009-09-30 00:00

Rockicki,

Not to say that you are the only person to give total praise too.....but dang.....i am very impressed at the FSRW object.

Other than long file names (I understand the issue)......you have just about covered anything (and actually more) than I can ever see needing at this point. I know at some point I will want more......typical human response.

You and the helpers (sorry don't know everyone) deserve a huge pat on the back.

James L

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
James L
Partner/Designer

Lil Brother SMT Assembly Services

Please note: Due to economic conditions the light at the end of the tunnel will be turned off until further notice. Thanks for your understanding.

BTX · 2009-09-30 00:58

Tomas.
It's simply... you're the best !!
I vote for you, to have a special place & gift, in the new prop contest.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Regards.

Alberto.

Rayman · 2009-09-30 01:53

Hmm... I guess as OBC said, directory support would be nice...
Would it be easier to navigate to a fixed directory name? (Like \BIN for example)
Maybe that's easier?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
My Prop Info&Apps: ·http://www.rayslogic.com/propeller/propeller.htm

ericball · 2009-09-30 02:44

There's no reason you can't add subdirectory support yourself. rokicki and lonesock have done the hardest part - the low-level read & write routines. The power is in the block level routines, everything above that is normal filesystem logic.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Composite NTSC sprite driver: Forum
NTSC & PAL driver templates: ObEx Forum
OnePinTVText driver: ObEx Forum

Cluso99 · 2009-09-30 02:54

rokicki / lonesock: I have a problem when trying to use the block driver directly which I posted under the older thread.

Perhaps it is something I have/have not done. What I am wanting to do, is after I locate the files I am interested in - I locate the first sector of each file - I use the block driver to read each sector directly using the low level sector address. However, it fails. I posted a simplified example of this.

Any help or suggestions would be appreciated.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:

· Home of the MultiBladeProps: TriBlade,·RamBlade, RetroBlade,·TwinBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: Micros eg Altair, and Terminals eg VT100 (Index) ZiCog (Z80) , MoCog (6809)
· Search the Propeller forums·(uses advanced Google search)
My cruising website is: ·www.bluemagic.biz·· MultiBladeProp is: www.bluemagic.biz/cluso.htm

Kye · 2009-09-30 03:09

I'll be posting my version with all that extra support you guys were asking about it using the ultra nice block driver. However, I have no time anymore so that will be done.... later, school's really throwing down the tests on me. Its like I have one or two every week.

Maybe I'll get some time during midsemester break somewhere in october.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Nyamekye,

lonesock · 2009-09-30 16:55

Cluso99 said...
rokicki / lonesock: I have a problem when trying to use the block driver directly which I posted under the older thread.

Perhaps it is something I have/have not done. What I am wanting to do, is after I locate the files I am interested in - I locate the first sector of each file - I use the block driver to read each sector directly using the low level sector address. However, it fails. I posted a simplified example of this.

Any help or suggestions would be appreciated.

Sorry for the slow response, any block driver issues are my responsibility. I'll try to take a look at the issue over lunch...thanks for the simplified test case.

Have I posted a picture recently of why I'm busy? Well, this is as good an excuse as any [noparse][[/noparse]8^)

Jonathan

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
lonesock
Piranha are people too.

Baggers · 2009-09-30 17:06

Congrats lonesock [noparse]:)[/noparse] and yes, it's clear now to see why you've been busy lol, they take up pretty much all your time! [noparse]:D[/noparse]

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
http://www.propgfx.co.uk/forum/·home of the PropGFX Lite

·

potatohead · 2009-09-30 18:29

Congrats!!

They do take a lot of time, but it's quality time.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Propeller Wiki: Share the coolness!
Chat in real time with other Propellerheads on IRC #propeller @ freenode.net
Safety Tip: Life is as good as YOU think it is!

David B · 2009-09-30 18:57

Great work! Thank you very much!

I do have one suggestion - why not give users access to set the file date? This can be really handy for managing files when a whole bunch of files have been generated on the SD card.

A minimal change to allow this is trivial -

Instead of having a fixed method "pdate" that returns a default date, define a variable "pdate" initialized to the generated default date:

DAT

pdate long (((2009-1980) << 25) + (1 << 21) + (27 << 16) + (7 << 11))

and provide a public method users can call to override the default:

pub setdate(newdate)
pdate := newdate

I'm using the older FSRW in my GPS data logger where I made a mod like this, and it's really handy, especially as GPS provides the current date and time.

This does require users to know how to generate a FAT16 timestamp.

If someone wanted to write a method to accept YYYY-MM-DD HH:MM:SS or something and convert that into a FAT16 timestamp, that would be even better. But this change would allow immediate access to the date.

rokicki · 2009-09-30 23:55

I like this; simple. We can also provide the utility function you describe (although just
variables as int, so we're not parsing a string); anyone not using it can comment it out.

Cluso99 · 2009-10-01 09:57

Lonesock: Yes I can understand. We have a 6 month old boy - grandson so we can hand back when the going gets rough

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:

· Home of the MultiBladeProps: TriBlade,·RamBlade, RetroBlade,·TwinBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: Micros eg Altair, and Terminals eg VT100 (Index) ZiCog (Z80) , MoCog (6809)
· Search the Propeller forums·(uses advanced Google search)
My cruising website is: ·www.bluemagic.biz·· MultiBladeProp is: www.bluemagic.biz/cluso.htm

lonesock · 2009-10-01 16:28

Cluso99 said...
rokicki / lonesock: I have a problem when trying to use the block driver directly which I posted under the older thread.

Perhaps it is something I have/have not done. What I am wanting to do, is after I locate the files I am interested in - I locate the first sector of each file - I use the block driver to read each sector directly using the low level sector address. However, it fails. I posted a simplified example of this.

Any help or suggestions would be appreciated.

I just checked out the code...I think the issue is that your buffer is not long aligned. In the VAR section, I see a word, then a bunch of longs, then the 512-byte buffer. If I understand correctly, the SPIN compiler will rearrange to longs, words, bytes, so the buffer is in fact word aligned, but not long aligned. You can adjust by either declaring buff as long buff[noparse][[/noparse]512/4], but that has complications, depending on how you address the buffer's data, or add an extra dummy word (e.g. declare word err, should be transparent).

Jonathan

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
lonesock
Piranha are people too.

Cluso99 · 2009-10-03 04:08

Jonathan, I just tried adding another dummy word but unfortunately that did not solve the problem :-(
BTW In case you try the femto version, I removed the initialise line so it does not work.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:

· Home of the MultiBladeProps: TriBlade,·RamBlade, RetroBlade,·TwinBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: Micros eg Altair, and Terminals eg VT100 (Index) ZiCog (Z80) , MoCog (6809)
· Search the Propeller forums·(uses advanced Google search)
My cruising website is: ·www.bluemagic.biz·· MultiBladeProp is: www.bluemagic.biz/cluso.htm

Cluso99 · 2009-10-03 04:35

Problem Solved

However, it may be a bug. The file I am trying to read is only 256 bytes long. Is this legal ???

The problem is that once I open the file (which works) I try to find out it's first sector address (which fails) and also I try to find out it's length (which fails). I do not try to read the file, although this also fails. On each call (tried individually) the code (driver) fails to return and hence locks.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:

· Home of the MultiBladeProps: TriBlade,·RamBlade, RetroBlade,·TwinBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: Micros eg Altair, and Terminals eg VT100 (Index) ZiCog (Z80) , MoCog (6809)
· Search the Propeller forums·(uses advanced Google search)
My cruising website is: ·www.bluemagic.biz·· MultiBladeProp is: www.bluemagic.biz/cluso.htm

Cluso99 · 2009-10-03 08:38

A little more info. There has to be 2 512B sectors. 1 512B sector still fails.

Now, I still do not have read/write block working - this still locks (does not return). Still investigating.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:

· Home of the MultiBladeProps: TriBlade,·RamBlade, RetroBlade,·TwinBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: Micros eg Altair, and Terminals eg VT100 (Index) ZiCog (Z80) , MoCog (6809)
· Search the Propeller forums·(uses advanced Google search)
My cruising website is: ·www.bluemagic.biz·· MultiBladeProp is: www.bluemagic.biz/cluso.htm

Cluso99 · 2009-10-03 13:10

fsrw24 fixes the read/write block after I disable tristate the SD pins. However, I still have a problem and am investigating as it may be my problem. The test code now works. I haven't tried fsrw24 to see if it works with <1024 byte files.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:

· Home of the MultiBladeProps: TriBlade,·RamBlade, RetroBlade,·TwinBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: Micros eg Altair, and Terminals eg VT100 (Index) ZiCog (Z80) , MoCog (6809)
· Search the Propeller forums·(uses advanced Google search)
My cruising website is: ·www.bluemagic.biz·· MultiBladeProp is: www.bluemagic.biz/cluso.htm

lonesock · 2009-10-03 18:01

Thanks for keeping us updated. Let me know if I can be of assistance.

Jonathan

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
lonesock
Piranha are people too.

Kye · 2009-10-04 19:14

Hey lonesock. Can you post the maunals you used for understanding the sd card protocol. The ones I've been able to find online are really not very helpful for figuring out how to do SPI stuff perfectly.

Thanks,

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Nyamekye,

BTX · 2009-10-04 19:29

Yes Kye, I'm agree with you !!, that would be great !!

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Regards.

Alberto.

Rayman · 2009-10-06 00:55

Just got FSRW2.4 working with my PSB prototype!

One question for you guys:

Are the units of the test results kBytes/second or kBits/second?
As I understand it kB is supposed to be kBytes and kb is kbits...
Just want to make sure... Thanks.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
My Prop Info&Apps: ·http://www.rayslogic.com/propeller/propeller.htm

BTX · 2009-10-06 01:06

I've used the old FSRW (first one) at 400 Kbytes/sec, so I suppose this should be 1.2Mbytes.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Regards.

Alberto.

HollyMinkowski · 2009-10-06 01:17

@lonesock

She's just adorable!
Sitting there grinning in her cute little romper

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
"Where am I? Where am I going? Why am I in a handbasket?"

rokicki · 2009-10-06 01:37

kB==kiloBytes, for sure. I'm a stickler for things like that.

Good for you to get it working!

Rayman said...
Just got FSRW2.4 working with my PSB prototype!

One question for you guys:

Are the units of the test results kBytes/second or kBits/second?
As I understand it kB is supposed to be kBytes and kb is kbits...
Just want to make sure... Thanks.

Rayman · 2009-10-06 01:45

Ok. Wow, I haven't looked closely in a while... Those are some pretty impressive throughput speeds!

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
My Prop Info&Apps: ·http://www.rayslogic.com/propeller/propeller.htm

lonesock · 2009-10-06 17:07

Kye said...
Hey lonesock. Can you post the maunals you used for understanding the sd card protocol. The ones I've been able to find online are really not very helpful for figuring out how to do SPI stuff perfectly.

Thanks,

OK, here goes [noparse][[/noparse]8^)

www.sdcard.org/developers/tech/sdcard/pls/ - the Simplified Physical Layer Spec. This one is as official as you can get, but leaves out lots of interesting information. However this one should be the final word, if you find discrepancies with other specs.

Next, Google "ProdManRS-MMCv1.3.pdf". This is the spec from SanDisk for the older MMC cards. There are some differences between SD and MMC, but for SPI mode they are very similar. And, most importantly, this document shows waveform timing diagrams for the SPI mode. SanDisk seems to have pulled it from their server, so I'm attaching a copy.

elm-chan.org/docs/mmc/mmc_e.html is a very good concise overview. There is also some code in the FATFS source that controls the SD card in SPI mode, but I think it's platform dependent (look for a PIC version, IIRC).

Unfortunately, there is no example code for using multiblock mode, but only sending one block at a time. All the multiblock examples (and there are fewer than single block mode examples) open the multiblock transfer...send down all the data at once, then close the multiblock mode. So the latest code from FSRW may be the only example of how to use multiblock mode to increase speed, but still have a very small buffer for doing the data transfer. A portion of the 30-odd cards we tested, however, do not re-initialize gracefully if they were suspended in a multiblock transfer. For this reason we auto release the card after a short timeout. Also, closing a file will explicitly release the card. Calling umount will also release the card. You can also manually call release. The release call is blocking, so after you call release you can power down or eject the card. The block driver is a state machine, so it will automatically re-establish the proper multiblock read or write mode as needed.

We also added the read-ahead and write-behind code to increase throughout. When you write a data block, the data is copied into a cog buffer, then written to the card in the driver cog, freeing the calling cog as soon as the 512-byte transfer to the cog buffer is done. We use rdlong for speed, so the buffers must be long aligned. Another write request will wait until any previous write is done before copying in the next data block. Since we are using multiblock mode, sequential writes automatically gain speed. Read-ahead is similar, in that if you ask for a block to be read, we will read the block into a cog buffer, then pass it back out to you. We then immediately request the next sequential block from the SD card and read it into the cog buffer. If the next block from the calling layer happens to be a read request for the next sequential block (and it typically is), the data is already in the buffer, so it gets copied out (using wrlong for speed), and the process repeats.

I think that's about it. Let me know if you have any questions or if the code is unclear.

Jonathan

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
lonesock
Piranha are people too.

lonesock · 2009-10-06 17:09

HollyMinkowski said...
@lonesock

She's just adorable!
Sitting there grinning in her cute little romper

Thanks! We agree! [noparse][[/noparse]8^)

Jonathan

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
lonesock
Piranha are people too.

lonesock · 2009-10-06 17:14

Rayman said...
Ok. Wow, I haven't looked closely in a while... Those are some pretty impressive throughput speeds!

Thanks! Rokicki's had some great ideas (the read-ahead / write-behind stuff, for example), and the multiblock mode seems to be critical for SDHC cards. I'm pretty happy with the speed now. I think I'm to the point where I can't get it any faster without adding a bunch more code. (e.g. I could use the video hardware for writing the data out.) I believe most of the overhead now is from the SPIN-only fsrw code. I think without porting that to PASM, we're pretty near the limit for now.

Jonathan

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
lonesock
Piranha are people too.

FSRW: Walk Right Back (seek, and ye shall find)

Comments