Double buffering for SD card

Graham Stabler

I'm using the propeller for some data capture and have problems with the delay caused by writing, I'm using rokicki's code and from what I can tell fsrw has a buffer called buf, you fill it using pwrite and once full it is flushed. The flushing is done by the cog running the assembly however I want to be able to double buffer so that while writing I can carry on filling the buffer. What I did do is create a 1024 byte buffer and once half full I would pwrite the first half and when completely full I would pwrite the second half. but pwriting 512bytes first copies it in to buf and this is a considerable delay.

I really want my double buffer to live in fsrw, I think I can code this but wanted to check I have not got something horribly wrong.

Cheers,

Graham

lonesock

The latest version of FSRW (2.6) uses write-behind. Once the FSRW buffer is full, it gets copied to a buffer inside the block-driver's cog, and from there it is streamed out. The fastest way to use the FSRW code is to have your own long-aligned, multiple of 512-bytes buffer. Fill that with your own data. Once that is full just call pwrite.

Jonathan

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
lonesock
Piranha are people too.
(geek humor escalation => "There are 100 types of people: those who understand binary, and those who understand bit masks")

Graham Stabler

I've looked at the source forge page but I am unable to get the files, if I download then I also get a load of html formatting.

But looking at the code it does not look as if it will work for me, I need to carry on filling my buffer while the data is being written, the whole reason for having the card is because I can't allocate sufficient memory and I want to be able to fill with card with data if I need to.

When I call pwrite it moves one buffer in hub ram to another in hub ram before writing so in that time I cannot fill my buffer. If however buf was twice as long I could use that as my buffer and pwrite would just call the flush program. Even then the code waits for the write to be complete from what I can tell in sdspiqasm and that occasionally seems to take longer than other times.

As I said I may have missed the point completely but it seems I can'd find a way to keep filling my buffer while the card is being written to.

Graham

MagIO2

Of course writing sometimes takes longer because then a lot of other stuff has to be done. Find the next free cluster and reserve it. So, if you want to be fast you should create a big file with spaces or whatever and copy that to a fresh formatted SD card. Then you can be sure that the file uses continuous sectors and you simply use the sector write instruction (I think it's writeblock).

You can find out the start sector of the file by an easy modification of popen. You only have to return the datasector as far as I remember.
Of course you then have to keep track of the file-pointer by yourself (how many bytes have been written). Maybe you could use a background COG wich writes the file-pointer to EEPROM? Or your can make sure that the data is always different from $0000_0000 and then you know whenever you find that in the file you look at the end of the data.

Graham Stabler

I think that there is plenty of time to write a 512 byte block to the card in the time it takes me to fill the other 512 byte buffer the problem is I can't wait for it, I need it to happen in parallel.

Graham

lonesock

Version 2.6 does do this for you in parallel, at least 512 bytes at a time. The 512-byte block is copied into the cog ram, then control is returned to you. Then the data is sent out to the SD card in parallel. Do not call flush too often, though, as it syncs up everything, including FAT tables. This takes extra time. Here's a copy of the latest version.

Jonathan

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
lonesock
Piranha are people too.
(geek humor escalation => "There are 100 types of people: those who understand binary, and those who understand bit masks")

Graham Stabler

I'm confused:

pub pwrite(ubuf, count) : r | t
{{
'   Write count bytes from the buffer ubuf.  Returns the number of bytes
'   successfully written, or a negative number if there is an error.
'   The buffer may be as large as you want.
}}
   repeat while (count > 0)
      if (bufat => bufend)
         pflushbuf(bufat, 0)
      t := (bufend - bufat) <# (count)
      if ((t | (ubuf) | bufat) & 3)
         bytemove(@buf+bufat, ubuf, t)
      else
         longmove(@buf+bufat, ubuf, t>>2)
      r += t
      bufat += t
      ubuf += t
      count -= t

It would seem that if I call this function with the address to my 512byte buffer it will copy it into buf and that is in hub ram. Only once buf is full will it call pflushbuf which in turn calls writeblock and I assume the assembly code reads the 512b from hub ram.

Graham

lonesock

You are correct. There is another 512B buffer inside FSRW. This is to allow FSRW to reuse the same underlying mechanism no matter if the user want to write a byte, or add a string, or write a block. When you pwrite your buffer into FSRW, it copies as much as possible into its internal long-aligned 512B buffer. When that buffer is full it triggers the copy into cog ram, and the block driver takes over from there. (Note that I think it will fill the FSRW buffer, and not trigger a write until the next write is requested, or a pflush is fired...I will look into this).

If you don't care about the FAT32/16 file system, you can just hijack the block driver for the fastest possible writes, as long as you only need to write 512 bytes at a time. (You can get almost 2MB/s write speeds this way.) You could even pre-create a contiguous "null-file" of all zeros, then use the block layer to sequentially overwrite the data in the null-file.

Jonathan

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
lonesock
Piranha are people too.
(geek humor escalation => "There are 100 types of people: those who understand binary, and those who understand bit masks")

Graham Stabler

Well I have implemented double buffering of buf which works for my needs, I have a helper function to let me fill buf (now 1024 long) and another helper function that calls a modified version of pflushbuf where sdspi.writeblock has an offset option so I can make it write the upper half of the 1024 byte buf.

This gets rid of the delay I got every 512bytes written but there is still an occasional delay of 0.25 seconds which seems to take place in sdspiqasm so I assume it is something to do with the card or the file system.

I like the idea of a zeroed file but I don't really get the way everything works so I'm not sure how to do it and like usual I'm trying to get something working at the last minute.

Graham

Miner_with_a_PIC

I have also experienced these occasional (I don't see any pattern) delays which range from a few milliseconds all the way up to almost 0.6 seconds.

I haven't started to track down the cause but if I don't find the source it may be a serious limiter for my project...if anyone on this thread knows or has a theory regarding what might be the cause of these sporadic delays please chime in...My first guess is that the SD card needs to erase its bits (set to all 1s?) before writing to them and so on occasion needs to do some house cleaning on a block of memory...this seems to be a sketchy theory but then again SD card inner workings are unfamiliar to me.

lonesock

Graham Stabler said...
Well I have implemented double buffering of buf which works for my needs, I have a helper function to let me fill buf (now 1024 long) and another helper function that calls a modified version of pflushbuf where sdspi.writeblock has an offset option so I can make it write the upper half of the 1024 byte buf.

This gets rid of the delay I got every 512bytes written but there is still an occasional delay of 0.25 seconds which seems to take place in sdspiqasm so I assume it is something to do with the card or the file system.

I like the idea of a zeroed file but I don't really get the way everything works so I'm not sure how to do it and like usual I'm trying to get something working at the last minute.

Graham

sdspiqasm is from a much older version of FSRW...there is no write-behind implemented there. Please try the update FSRW 2.6 I attached a few posts up.

Jonathan

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
lonesock
Piranha are people too.
(geek humor escalation => "There are 100 types of people: those who understand binary, and those who understand bit masks")

pgbpsu

Graham-

At what rate are you filling your buffer?

I've done a lot of work with FSRW and datalogging. I've found to get maximum speed without dropping consecutive samples, or having larger but occasional gaps in the data I had to setup a buffer that was nearly .5 seconds long. I can give you more specifics tomorrow, but if you give a ballpark of what rate you are trying to capture I may be able to help.

A long time ago, Rokicki told me that when using FSRW (with file system layer) you need a buffer that can absorb ~.33 seconds of data. Every once in a while the filesystem layer does something that can take this long. If that's really not acceptable you'll need to move to the block layer...

I was able to do 5000 bytes/second continuous by creating a circular buffer with 1 second worth of space. If you have the space available, I think it's a straight forward way to gain speed.

p

Graham Stabler

A buffer is only useful if the code filling it does not have to wait for the write so my double buffering of buf has got rid one one delay there is now just the delay in waiting for write block to finish (there is a repeat loop where it waits for the assembly to clear the command variable). Changing to 2.6 has improved this, now around 0.12 second delay down from 0.6 seconds. In theory I should be able to get rid of the wait for the command to be cleared as long as I am sure I will give it long enough to write before trying again, perhaps I can remove it if I call a special version of it from my already special version of pflushbuf, I'll try that.

Graham

Graham Stabler

OK that works. Modifications made:

1. Double the size of buf
2. Add a function to put data directly in to buf (in my case longs)
3. Add a function to call a modified version of pflushbuf with an offset (equal to 0 or 512bytes) so I can write first or second half of buf
4. Modified version of pflushbuf does block write to @buf + offset and calls modified version of blockwrite
5 modified blockwrite (only used when I am writing my data) has the wait for command to be clear removed.

So I think this will work as long as you do not try to write more than 512bytes every 0.12 ish seconds. This is OK for this application but I will definitely investigate having empty files to fill for the future to boost the speed further.

Cheers,

Graham