Speeding up file access in FSRW
CarlosFandango
Posts: 67
Well I have been programming the prop for about three weeks now, and thanks to the help of the knowledgeable users of this forum and the brilliant OBEX, things are going well. I'm using Rockiki's excellent FSRW + sdspiqasm to interface to the SD card on the uOLED-96, and so far (with very little requirement for modification) this has solved all of my SD issues. However I have run into a problem, specifically the time it takes to find a file during popen(). The application I'm developing requires a large number of small files - hundreds in fact - on the card, and if the file I'm looking for is further up the FAT table it takes progressively longer to locate. The time delay thus incurred is now significant.
So I wondered if anyone has encountered the same problem and developed an asm version? That's a little unlikely I guess, but it's worth asking! Otherwise, I will have a go at reverse engineering the code and implementing an asm version of my own. To that end, I thought I saw a post somewhere on this forum with a link to some very useful information on SD card interfacing, but now I can't find it. If anyone knows where this link is, or has a good (that's to say, readily understandable) source of information on this subject, I would be very grateful.
-CF
So I wondered if anyone has encountered the same problem and developed an asm version? That's a little unlikely I guess, but it's worth asking! Otherwise, I will have a go at reverse engineering the code and implementing an asm version of my own. To that end, I thought I saw a post somewhere on this forum with a link to some very useful information on SD card interfacing, but now I can't find it. If anyone knows where this link is, or has a good (that's to say, readily understandable) source of information on this subject, I would be very grateful.
-CF
Comments
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
http://www.propgfx.co.uk/forum/·home of the PropGFX Lite
·
I've not got one, but I believe this is something that might help you.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
http://www.propgfx.co.uk/forum/·home of the PropGFX Lite
·
Also, I note in this thread (to which you responded)
http://forums.parallax.com/showthread.php?p=711374
there is a discussion on file seeking which I found very interesting. I think I need a fundemental change of direction; my small files are all individual characters making up a 65K color font for the OLED - you can't store these in memory, they're too big - so there are a lot of them for different font sizes. But implementing the file seek as discussed in this thread, I could compound all of my fonts into one large file (per font), store the block address of the file, and access it directly (and in a fairly random way). After making sure the font files are contiguous, anyway. That would make things blisteringly quick I believe - but I do have more windows programming to do now, which isn't as much fun as programming the prop
-CF
The best technique here might be a contiguous file approach and using lower-layer block I/O; if you
put it in one file, and it's contiguous, then you can just do a simple block read to get random access.
But I'd need to see more of the big picture before I can make any solid recommendations.
Hi Rockicki... Well, at the moment there are about 120 files but if I were to continue down that path, I would end up with several hundred at least. They are font characters - images that are blitted to the OLED, so that I can do 65K color fonts, and are around 800-100 bytes each. W x H x 2 + 4-byte header actually. 0-9, A-Z, plus other characters, and of course multiple font sizes.
Yep, I've come to that conclusion already, so I'm in the middle of rewriting my VB.NET image encoder to stuff all of the characters for a given font in a single file with a lookup table in block 0. Then I'll reformat the card, and put all of the font files on the card FIRST....! Thanks to the help supplied by everyone on this forum and the various informative threads here, that will in fact be the first time I've had to remove the card from the OLED to put files on it - everything so far has been transferred over the comms link.
Hi Mike. I have looked at these, and do have them downloaded here. I think that this would be a good solution, but the logic in these routines is quite complex (thereby leading to a more compact and undoubtedly more elegant solution). I don't really understand it, at least not yet. So if I needed to change anything for any reason I don't think I would be able to, not at the moment anyway. I'm a bit confused generally by the various flavors of basic on offer (although I understand the routines you mention would be usable without the basic interpreter). So I'm going to modify Rockiki's routines to provide a seek, which looks easy, and will revisit the sdspiFemto code when I have learned enough to cope with it! The app I'm working on has to function in less than two weeks
I have to say though that after lots of PIC and Atmel programming (am I allowed to say that?!) this little prop chip is amazing. I love it.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Paul Baker
Propeller Applications Engineer
Parallax, Inc.
That sounds interesting, but somewhat awkward! Anyway, I seem to have solved this problem just by doing what others have already done. My screen updates are now so fast, I have to put delays between them to make it readable. I'm becoming increasingly surprised at how flexible this system is; I now have file support, comms, I2C and screen drivers all in place (largely pre-existing in the OBEX) and still have more than half of the HUB left for the actual application code. Good job! Can't wait to see how good the propII will be. And despite obvious shortcomings, the development environment is in my opinion more than adequate. I've also got a windows app built using the propellent dll that allows field firmware updates. Overall I think you guys (parallax AND the user base) have made a brilliant effort. Try getting this level of support from other manufacturers and see how far you get!