Understanding FSRW flush

DavidGreg · 2010-02-05 18:13

I'm working on a simple data logger using two MCP3208s. My ADC seem to occur ever 1.5 ms or so. Which is plenty fast. But, it seems to take about 0.4 seconds to do pwrite and pflush, independent of how much data I'm writing. Is this normal?


CON
   
  _clkmode = xtal1 + pll16x                     ' Crystal and PLL settings.
  _xinfreq = 5_000_000                          ' 5 MHz crystal.
  
  cpin1  =  20
  dpin1  =  21
  spin1  =  22

  cpin2  = 27
  dpin2  = 24
  spin2  = 26

OBJ

  sd     : "fsrw"
  mcp1   : "MCP3208"                             ' ADC Object
  mcp2   : "MCP3208"


DAT
   FileName  byte "data.bin",0

VAR   
   long Buff[noparse][[/noparse]5000]
   
PUB go | counter, i, j, k                      

  sd.mount(0)
  sd.popen(@FileName, "a" )
  mcp1.start(dpin1,cpin1,spin1,%11111111)
  mcp2.start(dpin2,cpin2,spin2,%11111111)


   repeat
     repeat j from 0 to 99
      k := j*13
      buff[noparse][[/noparse]0+k] := cnt 
      repeat i from 0 to 7 
       buff[noparse][[/noparse]i+1+k] := mcp1.in(i) 
      repeat i from 0 to 3       
       buff[noparse][[/noparse]i+8+1+k] := mcp2.in(i)
    sd.pwrite(@buff,4*13*100)
    sd.pflush

  sd.pclose
  sd.unmount

rokicki · 2010-02-05 19:31

A 0.4s delay for pflush is not normal, but it could occur every so often. Essentially, sequential writes tend to be
fast, but the SD card can have a significant delay at various times.

I would not be shocked to see a 0.4s delay for a flush every, say, 100 to 1000 writes, but most of the time
it should be *significantly* faster than this.

Note that the flush updates the data block, possibly a FAT block, and the directory block, so there's a fair
amount of writing going on, and the wear leveling code may need to scramble to find the appropriate
erase blocks to work with.

I would drop the flush unless you *really* need it and let pclose() finish things off. If it's a perpetual loop,
then you need to trade off buffering required vs how much data you lose to evaluate how frequently you want
to flush.

What kind of card are you using, and how do you have it formatted?

DavidGreg · 2010-02-05 19:48

The card is a kingston 4gb microSDHC. I didn't format it [noparse]:)[/noparse]

The 0.4 seconds is very repeatable with the above code - every write/flush takes about 0.4s per the cnt output to the file. I can upload an example ifyou're interested.

I guess i'm greedy because I want all my data at a fixed sample rate [noparse]:)[/noparse] So re-wrote this a little with two alternating buffers and a dedicated cog for acquiring the samples.

It works if the repeat until loops are commented out, but of course there is no timing between the writing and the reading to the buffer. If i uncomment the repeat until loops, nothing is ever written to the file.


CON
   
  _clkmode = xtal1 + pll16x                     ' Crystal and PLL settings.
  _xinfreq = 5_000_000                          ' 5 MHz crystal.

OBJ

  sd     : "fsrw"
  log: "buffLog"


DAT
   FileName  byte "data.bin",0
   Buff1 long 1 [noparse][[/noparse]2500]
   Buff2 long 2 [noparse][[/noparse]2500]
   flag1 byte True
   flag2 byte True   
   
PUB go | counter, i, j, k                      

  sd.mount(0)
  sd.popen(@FileName, "a" )
  log.start(@Buff1,@Buff2,@flag1,@flag2)
 
   repeat
     ' repeat until (flag1 == True)
     '  Next
        
      sd.pwrite(@buff1,4*13*150)
      sd.pflush

     ' repeat until (flag2 == True)
     '   Next

      sd.pwrite(@buff2,4*13*150)
      sd.pflush

OBJ
  mcp1   : "MCP3208"                             ' ADC Object
  mcp2   : "MCP3208"
CON
  cpin1  =  20
  dpin1  =  21
  spin1  =  22

  cpin2  = 27
  dpin2  = 24
  spin2  = 26
  
VAR
  long  Stack[noparse][[/noparse]120]                      'Stack space for new cog
  byte  Cog                            'Hold ID of cog in use, if any   
 
PUB Start(bufAdd1,bufAdd2,fAdd1,fAdd2): Success
{{Start new servo input process.  Return True if successful.}}
  Stop
  Success := (Cog := cognew(log(bufAdd1,bufAdd2,fAdd1,fAdd2), @Stack) + 1)

PUB Stop
{{Stop toggling process, if any.}}
  if Cog
    cogstop(Cog~ - 1)
 
PUB Active: YesNo
{{Return TRUE if process is active, FALSE otherwise.}}
  YesNo := Cog > 0
      
PUB log (buffAdd1,buffAdd2,flAdd1,flAdd2) | i,j,k
  mcp1.start(dpin1,cpin1,spin1,%11111111)
  mcp2.start(dpin2,cpin2,spin2,%11111111)

  repeat
    Byte[noparse][[/noparse]flAdd1] := False
    repeat j from 0 to 149
       k := j*13
       LONG[noparse][[/noparse]buffAdd1][noparse][[/noparse]0+k] := cnt 
       repeat i from 0 to 7 
          LONG[noparse][[/noparse]buffAdd1][noparse][[/noparse]i+1+k] := mcp1.in(i) 
       repeat i from 0 to 3       
           LONG[noparse][[/noparse]buffAdd1][noparse][[/noparse]i+8+1+k] := mcp2.in(i)
       waitcnt(cnt + 218_560)
    Byte[noparse][[/noparse]flAdd1] := True
        

    Byte[noparse][[/noparse]flAdd2]:= False
    repeat j from 0 to 149
       k := j*13
       LONG[noparse][[/noparse]buffAdd2][noparse][[/noparse]0+k] := cnt 
       repeat i from 0 to 7 
          LONG[noparse][[/noparse]buffAdd2][noparse][[/noparse]i+1+k] := mcp1.in(i) 
       repeat i from 0 to 3       
           LONG[noparse][[/noparse]buffAdd2][noparse][[/noparse]i+8+1+k] := mcp2.in(i)
       waitcnt(cnt + 218_560)
    Byte[noparse][[/noparse]flAdd2] := True

rokicki · 2010-02-05 21:23

Why don't you mock up the code with a fake sample generator (so I can run it without your
MCP3208 hardware) that writes a file that includes the cnt samples (in ASCII would be nice)
and include the file it generates in the zip. I'll run it on my hardware here and my cards and
compare.

Understanding FSRW flush

Comments