fsrw
Twim
Posts: 10
Hello,
·····would someone be kind enough to provide some direction.
I would like to add seek functionality to fsrw.spin I need to randomly
access·bytes in an open file the function should modify "pgetc"·so that:
··· pSeekbyte ( long_byte_location_in_file)·
would return the character located at "long_byte_location_in_file"
thank you
·····would someone be kind enough to provide some direction.
I would like to add seek functionality to fsrw.spin I need to randomly
access·bytes in an open file the function should modify "pgetc"·so that:
··· pSeekbyte ( long_byte_location_in_file)·
would return the character located at "long_byte_location_in_file"
thank you
Comments
1) Random read and write access
2) Multiple instances of the FSRW object can co-exist and reference the same memory card so you can have more than one file open at the same time, each requiring a 512 byte buffer with one other 512 buffer shared among all the open files.· Each instance requires only about 580 bytes including the buffer.
It's much easier to add random read access.· It's harder to provide for random write access for updating in place and appending to an existing file.
I'm in the process of debugging it.· It's not in any shape yet to pass on to others since it sometimes fails to correctly look up files and occasionally corrupts the root directory.· I don't have a lot of time to work on it currently, but I'd be happy to keep people informed.
Oh yeah, it also includes read-only subdirectories.· You can travel the subdirectory tree and create and delete files in subdirectories, but you can't change the subdirectories.
Post Edited (Mike Green) : 2/26/2008 2:26:43 AM GMT
on linux (using a loopback filesystem). I've got a Perl script that converts the C to
Spin. This strategy may make testing and debugging much easier. It's a bit
idiosyncratic (I used the same strategy for femtoBASIC by the way) and I do not
support it as a general programming paradigm, but it let me test the *heck* out of
it, comparing operations side by side for many combinations of things that gave
me enough confidence to release it (without worrying too much about messing up
people's cards.) I'd be happy to dump the whole thing into your lap if you are
interested.
(Indeed, I could actually release a family of fsrw implementations with the different
featuers through the use of either m4 or the C preprocessor. Too bad I've not
had much time for the Prop lately.)
if that makes sense to you. There is no parsing, or symbol table, or anything like that; just a set of very simple regular expressions.
I just wanted the bare minimum so I could program in something the C compiler would compile, and that could be turned trivially into a
legitimate equivalent Spin program. No structures. No variable declarations other than at the top. Etc.
··· am I correct in assuming that I can move to a specific location within a file by setting "floc" to the desired location, then calling· pfillbuf().· Does floc· always need to· be aligned with the first byte of the sector/block
thanks
has a seek function been developed yet for Rokicki's Sd object? I am in the same boat as Twim in that I need to access particular characters within a large (>16MB) file.
Paul
1) when the program starts it will read a location from the end (or start) of the file
2) then it will go to that location and retrieve a block of about 8 or 10 characters
3) send these characters to remote device serially
4) read the next block and and repeat from step 3 until end of program
end of program:
save the location of the next block of characters to the end (or start) of the file.
I have achieved this using the Vinculum VDrive but the fact the the USB thumb drive projects from the enclosure is a problem, so I'm hoping to reproduce using the SD card.
I would appreciate any assistance on this.
Paul
I've been struggling with this all day.
Does anyone know if it is possible to go to a specific location in a file. I am in a position where I have to make a decision to use SD or stick with USB. As mentioned in last post, SD is much more preferable (if not essential) for my project.
Thanks again for any help you may be able to offer.
Paul
I still don't have the FAT file system changes working.· I'll take a look at it in the next day or two to see if I can post just a read-only seek routine.
Post Edited (Mike Green) : 9/23/2008 10:38:55 PM GMT
Just an idea, and not testet:
You can change the pread method in fsrw a little bit:
pub pread(ubuf, count) | r, t {{ ' Read count bytes into the buffer ubuf. Returns the number of bytes ' successfully read, or a negative number if there is an error. ' The buffer may be as large as you want. ' * if ubuf is set to zero, count bytes are only read, without ' writing to the buffer. }} r := 0 repeat while (count > 0) if (bufat => bufend) t := pfillbuf if (t =< 0) if (r > 0) return r return t t := bufend - bufat if (t > count) t := count ' bytemove(ubuf, @buf+bufat, t) <-- replaced by the next 2 lines if ubuf > 0 bytemove(ubuf, @buf+bufat, t) bufat += t r += t ubuf += t count -= t return rThis allows to overread bytes on the SD card, without storing anywhere, if you set the buffer address to 0. This is a kind of relativ seek.
Then you can do something like that:
... VAR long position byte buffer[noparse][[/noparse]8] ... fsrw.popen(...) fsrw.pread(@position,4) 'read the position at start fsrw.pread(0,position-4) 'relativ seek fsrw.pread(@buffer,8) 'read 8 bytes in buffer repeat i from 0 to 7 send(buffer[noparse][[/noparse] i ]) 'and send ...Andy
Ariba, I'll try that method this evening when I get back to base. Thanks
Actually Ariba, that looks very elegant and it seems to me that it will work wonderfully. Thanks again.
One last thing, and excuse me if this is in the documentation for fsrw, but can I open a file for write and write to the first 4 bytes of the file without affecting the remainder of the file?
Paul
Post Edited (RivTec) : 9/24/2008 10:57:18 AM GMT
if (t > count) t := count ' bytemove(ubuf, @buf+bufat, t) <-- replaced by the next 2 lines if ubuf > 0 bytemove(ubuf, @buf+bufat, t) ubuf += t ' moved this line to here because it doesn't need to increase ubuf if it's 0 bufat += t r += t ' ubuf += t ' removed this line, and put it with the if ubuf > 0 above▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
http://www.propgfx.co.uk/forum/·home of the PropGFX Lite
·
I have overlooked that line
Andy
I just done a few small tests and it takes approx. 5 seconds to seek to the 250,000th block of 11 characters (not 8 as indicated earlier). So I imagine if I have 5 million blocks of 11 chars it will take 100 seconds approx. This may not be that much of an issue as it is only when the program starts but if there was a quicker mechanism it would be a Godsend!
If the original file was written to a clean newly formatted card (on a PC), do you think it might be posible to read the 'position' at the start of the file as above and then calculate which cluster it points to and finally the location within the last cluster. I say all this but my experience with FATs of any sort is next to nothing.
As per my earlier post, can I open a file for write and write to the first 4 bytes of the file (incrementing the position of the next block of characters) without affecting the remainder of the file?. I tried putc but this deletes everything after that.
Regards, Paul
Just make pfillbuf take a boolean on whether to do the actual block read or not, and add logic to compute if this is the last block
of the read or not (you have to read the last block, so that subsequent reads *with* buffers work).
Essentially, if we have a buffer, or if the count of bytes to go is <= 512 chars, we *must* read that
data block, else we do not need to read the data block.
You still need to read the FAT blocks, but generally this will be negligible overhead (there are 256
FAT entries in a single metadata block, and that's enough for 16MB of data, assuming the file is
contiguous).
What I meant to say was thanks Rokicki, that worked a treat. I am able to go to the location within 1 second or less.
Regarding R+, I have moved the position pointer to the last 4 bytes of the file and retrieving this by getting the FileSize - 4. Is it possible for me to overwrite these in with putc or any other command whilst not affecting the contents of the file up to the last 4 bytes?
Regards and thanks.
Paul
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Paul Baker
Propeller Applications Engineer
Parallax, Inc.
Another idea, I used such a trick before in one of my programs.
- Write a file with the needed size to the SD card. It must have contiguous sectors. Normally you get this if you write it with Windows and a Cardreader. Or with fsrw and a fresh formatted card.
- Get the Startsector for that file on the SD card (see code below)
- Handle all the read and write accesses with sector reads and writes by yourself. Because the sectors are contiguous, it's easy to calculate the position.
Your datablocks shoud have a size of 16 bytes,instead of 11 bytes, so 32 of them fit exactly in one sector. You still can use only the first 11 bytes of the 16.
Add this routines to the fsrw object:
PUB getStartSector(strptr) : s s~ ifnot popen(strptr,"r") s := fclust << clustershift + dataregion pclose PUB readblock(n, b) 'wrapper routines for read and write a sector sdspi.readblock(n, b) PUB writeblock(n, b) sdspi.writeblock(n, b)And in the Main object use the routines like that:
VAR long fsector byte sbuff[noparse][[/noparse]512] ... fsector = fsrw.getStartSector("filename") ... fsrw.readblock(fsector, @sbuff) 'read 1.sector of file long[noparse][[/noparse]@sbuff] := position 'overwrite first long with new position fsrw.writeblock(fsector, @sbuff) 'write modified 1. sector ... sect := position / 512 + fsector 'fast seek offset := position // 512 fsrw.readblock(sect, @sbuff) 'read whole sector repeat i from 0 to 11 send(sbuff[noparse][[/noparse]offset+i]) 'send the 11 bytesYou don't need to open or close the file, for reading or writing sectors.
I hope this is clear enough.
Also this code is not tested, and can have some bugs [noparse];)[/noparse]
Andy
There is no r+ or w+ or a+ option at all or support for switching between reading and writing on the same file (without
opening it again from scratch).
Andy's trick is a useful one, but care must be taken that the file is indeed contiguous. Then you can just do block I/O,
and using read-block/modify-block/write-block, you can modify any of the file bits you want retaining all the rest.
-tom