Atmel DataFlash; basil- do you still read the forums?

pgbpsu

I was wondering if anyone out there has code for working with Atmel DataFlash they are willing to share. I know Basil was using it in his rocket datalogger. I checked the OBEX and didn't see anything. Basil did you ever post your code? If so where, if not, would you be willing to share it?

I'm (still ) working on a seismic datalogger that records 5 24-bit channels at 10Khz each. When we set off an explosion we want to capture as much as 30 seconds of data. I haven't yet found a storage solution that's fast enough to keep up with this and retain the Prop as the brains of the operation. I've tried the Vinculum/USB and straight to SD card no luck on either count. So it looks like I need to buffer the data and once the shot is over I can write the data to disk. At that point I have all the time in the world. At first I gave up on the DataFlash because, although it could be clocked faster than I needed, that was only getting data into the onboard buffer. Once the on board buffers were full there was a significant wait time before the data were committed to memory. Too much for my application. After doing a bit of figuring I think if I use one DataFlash for each channel I can get this to work.

All that is just long-winded way to ask if anyone has DataFlash code they'd be willing to let me have a look at. Other suggestions for buffering memory are welcome.

Thanks,
pgb

Cluso99

Corrected - thanks Baggers (though it looked a lot)

Let me know if my maths are wrong.
30 secs @ (5 x 10KHz) x 24 bits
= 30 @ 50KHz x 24 bits
=1.5M x 24 bits
= 4.5MBytes

So you have to store off-chip somewhere 4.5MBytes in 30 seconds.

I am not sure of the write times for the Flash without looking them up so maybe somene can say if this can be done.

Alternately, you may be able to use a static ram as temorary storage before offloading to wherever.

Post Edited (Cluso99) : 10/16/2008 11:09:13 AM GMT

Mike Green

How about using strings of Ramtron FRAM. This is much faster than flash EEPROM like the DataFlash. The largest I2C memory is 64K bytes (FM24C512), but they have SPI memory with 256K bytes of storage (FM25H20). You could have 8 of these in parallel using 8 I/O pins for data, 1 for a clock and 1 for the chip enable. If you put another 8 or 16 in parallel, you double or triple the amount of storage with only one or two more I/O pins for chip enables. That gives you up to 6MBytes of very fast storage.

Tracy Allen

The programming time of a previously erased page for the AT45DB161 is between 3 and 6 milliseconds, and at 512 bytes per page, that amounts to ~100 kilobytes per second. The key is that the chip has two buffers, and for constant throughput, the program ping-pongs between them so that one is accepting data while the other is busy writing to a page. Five 24 bit channels at 10kHz each--wow! Maybe one chip per channel, but they are relatively cheap.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Tracy Allen
www.emesystems.com

Baggers

Cluso99, it's not 48MB
30 * 5 * 10K * 24bit

so for each second you need 5 * 10K * 24bit
= 5 * 10_000 * 3
= 150_000 per second ( 146.484375 KB )

= 30 * 150_000
= 4_500_000 bytes ( = 4.291534423828125 MB )

So taking into account you need 150_000 bytes per second.
At 60fps you only need 2500 bytes.
now what you should do is make the buffer slightly bigger, as 15 bytes ( 5 channels 24bit ) doesn't go evenly into sector sizes [noparse];)[/noparse]
and write the sector, and move the remainding bytes to the beginning and continue from there.

pgbpsu, A quick question to you about your SD approach, did you write each entry at a time?
or did you buffer it into 512, 1024, 2048 etc. byte buffers to write sectors at a time, instead of making it read, modify write sectors?
plus it will also make it easier for you to retreive the data [noparse]:)[/noparse]

Hope this helps
Baggers.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
http://www.propgfx.co.uk/forum/·home of the PropGFX Lite

·

pgbpsu

Baggers is right about the data volume 150,000bytes/sec * 30 sec = ~4.5Mbytes.

I didn't try anything fancy when testing the SD card, i.e. low-level block writes, pre-defining sectors, etc. I did however use the write X bytes to disk where X was 512, 1024, and 2048 (although once I get all the rest of the programming done for this system, I doubt I'll have 2048 left as a data buffer in the prop). None of these configurations were fast enough. rockiki (sp?) actually pm me several times showing a great willingness to help. In the end he and my tests convinced me that there was too much jitter in the SD write for me to be safe. I can't afford any dropped samples during acq. I still plan to use an SD card as the final resting place for these data. If I can shove them out to memory during the acquisition, I can read them back in a channel at a time, format them into standard seismic format and write proper SEGY files to the SD card. As far as we're concerned that's a perfect solution.

Mike, thanks for the tip about Ramtron FRAM. I haven't heard of it but I'll go look it up. I'm still optimistic that the DataFlash can work as Tracy has suggested. By bouncing back and fourth between the buffers I think we can do this. One of the nice things about this choice is, as Tracy points out they are cheap. And they come in an 8-pin package which is still hand solderable, but even with one for each channel, they don't take up much space. Running these chips in parallel will require 4 propeller lines each which is getting a bit pin expensive. Maybe there's something clever that we can do to reduce that- shared clock, using a single line for SDI/SDO. You'll probably see me back here asking for help on that one.

Tracy, thanks for clarifying that one buffer can be written to while the other is being moved to main memory. That's what I figured, but I know you have some experience with them. I remember some threads a while back where basil was working with them with your help. That's where the idea came from. These forums are invaluable for ideas! Unfortunately, I can't find some of those posts now. Searching leads me to believe that basil is switching from DataFlash to SD.

Thanks all.
pgb

Mike Green

Regarding FRAM... When you run the SPI versions in parallel, you need one pin for each data bit. All of the FRAMs share a clock bit and each bank uses a chip enable bit. For 3 banks of 8 FRAMs, that's 8 + 1 + 3 = 12 I/O pins.

lonesock

Just out of curiosity, is the data per channel compressible (lossless)? E.g. if you switched to deltas between adjacent values, could you use 16-bits per sample, instead of 24?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
lonesock
Piranha are people too.

Tracy Allen

@pgb, this thread may be the one you are looking for:
http://forums.parallax.com/showthread.php?p=646332
I don't see that Basil posted code there for his recording altimeter. His data rate requirement was not as demanding as yours. Nevertheless, one main market for the dataFlash memories is audio recording at a sustained high data rate. If you split your memory across 5 chips for 5 channels, that is less than one megabyte per chip, which fits in 5 * AT45DB081, 8 pin soic. One cog per chip? Wiith regard to pins, yes, all of them can share one clock, and the SO and SI pins on each one can be muxed. So total 11 Prop pins: 5*(SO+SI) + 5*CS\ + 1 * CK.

I'm curious, what kind of ADC you are using to capture 24 bits at 10kHz? Also following up on Lonesock's compression question.. analog logarithmic preamp?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Tracy Allen
www.emesystems.com

pgbpsu

@lonesock - we've looked at compression algorithms for part of our project. We'd like quality control data from each of the devices. The available bandwidth on the radio links between one of these devices and our base station will demand some kind of compression. There we were looking at lossy compression because we'll have a full compliment of data stored on each device and really only need to know if the device is working. The oscillatory nature of seismic data may not lend itself to first differences. But it's worth a look. I can run a couple of test seismograms through Matlab to see if the results are promising. Thanks for the suggestion.

@Tracy Allen - thanks for the link. That's one of the threads I remember reading but I thought there was another with at least pseudo-code in it. Although basil's code would be educational to look at, I'm not sure I'd be able to use it. I'll need to code this in PASM. I think he was able to get things done with SPIN. The datasheet doesn't look too bad. I'll get to work building some PASM routines. It seems one can tie SDI and SDO together with the proper resistors.

We are actually using an ADC aimed at the audio market: AD1871. I've gotten lots of help through the forums with that part of the project, but now I can't find those posts either. OT: How does one find threads more than 1 year old? Here's the datasheet for the chips we're using.

www.analog.com/static/imported-files/data_sheets/AD1871.pdf

They are meant to run at 96Khz. We've tuned them down to 80Khz which the prop reads and averages down to 10Khz. The 11 pins needed to implement a DataFlash solution is probably doable. Right now I'm reading and averaging 2 channels per cog then writing that to HUB memory. I'll have to look at the code I've got to see if there's a way to remove the write to HUB and replace it with write to DataFlash. We're running 2 of these ADC chips for 4 channels total and one timer channel which is basically the cnt stored every time we take a sample. We are running tight on resources so, finding a way to get all 5 channels read and stored with 3 cogs would be the best possible solution. I'll have to see how many instructions it takes to manage the address locations as well as do the writing to DataFlash and see what kind of time I have left over in my acq/average cogs to see if that's reasonable.

I've never used a logarithmic pre-amp before, but we really want to preserve the dynamic range of these devices. We need to be able to see very small variations between small samples and not saturate on big ones.

I've ordered some DataFlash chips which should give me something to play with early next week. AT45DB321D which is 32-Mbit=4Mbyte which should be more than enough for one channel. Using the entire device should allow us to hit the 100,000 write cycle barrier a bit later in life.

Basil

Every now and then hehe Ill digest all that has been said and update this post :P

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
-Alec

My our page

Basil

Right. Here is my rather bloated code. I haven't gone through and made it generic so I hope it makes sense [noparse]:([/noparse]

Note: DF_Logging is started in a new COG.

PUB DF_Logging | page, buffer_number, buffer_byte, DF_First_Page, temp, i
'**********************************************************************
'Init ports
'**********************************************************************
  dira[noparse][[/noparse]LED1..LED3]~~ 
  dira[noparse][[/noparse]DF_CS]~~
  outa[noparse][[/noparse]DF_CS]~~
  dira[noparse][[/noparse]DF_SCK]~~
  outa[noparse][[/noparse]DF_SCK]~
'**********************************************************************
'Get First DF_First_Page from the point saved after the last flight. 
'**********************************************************************
  DF_Read_Page_Bytes(0, 4)               'Setup to recieve
  DF_First_Page := DF_Recieve(32)         'Read first page # (end of first flight) 
  DF_End                                 'End recieve
    
  if DF_First_Page == -1                   'If no first page recorded then start recording from page 0
    DF_First_Page := 0 
  elseif DF_First_Page > 0               'Else keep jumping from the end of one flight to the next until a free page is found.
    repeat while DF_First_Page > 0 
      page := DF_First_Page              'Current page = the first page
      DF_Read_Page_Bytes(DF_First_Page, 4) 'This long is stored @ byte 4 in page
      DF_First_Page := DF_Recieve(32)
      DF_End
      if DF_First_Page > 4093
        repeat 'Memory full!!!
    DF_First_Page := page
  Serial.tx($0D)
  Serial.str(string("Starting at page "))
  Serial.dec(DF_First_Page)  
  Serial.tx($0D)         
'**********************************************************************
'Erase pre-flight buffer pages ready for record.
'**********************************************************************
  repeat page from 1 to 2
    temp := $81 << 24
    temp := temp | ((DF_First_Page + page) << 10)
    DF_Send(temp, 32)
    DF_End
    waitcnt(clkfreq/1000*30 + cnt)
  Serial.str(string("First 2 pages erased for buffers to go into."))
  Serial.tx($0D)
'**********************************************************************
'Pre-flight circular buffer in 2 buffers
  repeat until Flight_Status > 0 'Wait until rocket is armed.
'**********************************************************************
  buffer_byte := 0
  buffer_number := 1
  repeat while Flight_Status =< 1
    if buffer_byte == 0                                 'Setup to log into buffer 
      DF_End
      buffer_number := (buffer_number // 2) + 1   
      DF_Buf(buffer_number,0)
    repeat while SAMPLES_READY == 0                     'Wait for next sample to be ready
    SAMPLES_READY := 0                                  'Reset flag                                                                                  
    DF_Send(bADC, 12)                                   'Write bADC
    DF_Send(iADC, 12)                                   'Write iADC
    buffer_byte := (buffer_byte + 3) // 528             'Increase buffer_byte by 3, looping back to 0 when 528 is reached.
  DF_End                                                'End last buffer write.
  Circ_Buf_Start_byte := buffer_byte
'**********************************************************************
'Launch detected, copy the 2 pre-flight buffers into main page memory
'page Start_Page +1 & +2. Keep circlar buffer start addr to save later. 
'**********************************************************************   
  page := DF_First_Page + 1
  DF_Buf_to_Main(buffer_number, page, 0) 'Write current buffer
  waitcnt(clkfreq/1000*6 + cnt)          'Wait a bit
  buffer_number := (buffer_number // 2) + 1   
  page++
  DF_Buf_to_Main(buffer_number, page, 0) 'Write other buffer
'**********************************************************************
'In flight loop. Starts recording into Start_Page +3
'**********************************************************************
  repeat until Flight_Status == 5 ' while Flight_Status == 2 or Flight_Status == 3 or Flight_Status == 4
    page++                                'Increment page for next time around
    if page => 4095                                 'Stop logging if end of memory
      quit
    buffer_number := (buffer_number // 2) + 1   
    DF_Buf(buffer_number,0)
    repeat 176 '(528/3)
      if Flight_Status <> 5
        repeat while SAMPLES_READY == 0                     'Wait for next sample to be ready
      SAMPLES_READY := 0                                  'Reset flag                                                                                  
      DF_Send(bADC, 12)                                   'Write bADC
      DF_Send(iADC, 12)                                   'Write iADC      
    DF_End
    DF_Buf_to_Main(buffer_number, page, 1) 'Write current buffer
  DF_End

  Serial.str(string("Flight ended on page "))
  Serial.dec(page)
  Serial.tx($0D) 
'**********************************************************************
'Post flight loop. Save other variables into Start_Page +0
'**********************************************************************
  DF_Next_Flight := page + 1
  buffer_number := (buffer_number // 2) + 1   

  DF_Buf(buffer_number,0)
  
  i := @Circ_Buf_Start_Byte
  repeat while i =< (@Pyro_Test)
    DF_Send(long[i],32)
    i := i + 4  
  DF_End
  DF_Buf_to_Main(buffer_number, DF_First_Page, 1) 'Write current buffer
  waitcnt(clkfreq/10 + cnt)

  'Erase first page of next flight incase of data from previous set of flights
  temp := $81 << 24
  temp := temp | (DF_next_flight << 10)
  DF_Send(temp, 32)
  DF_End
  waitcnt(clkfreq/1000*30 + cnt)

  Serial.str(string("Stopping logging!"))
  Serial.tx($0D)

[/i]

And here are the red/write bits...

PRI DF_Erase | temp, p
'**********************************************************************
'Init ports
'**********************************************************************
  dira[noparse][[/noparse]DF_CS]~~
  outa[noparse][[/noparse]DF_CS]~~
  dira[noparse][[/noparse]DF_SCK]~~
  outa[noparse][[/noparse]DF_SCK]~
  
  repeat p from 0 to 1' 511 Only erase first page for testing
    temp := $50<< 24
    temp := temp | (p << 13) 

    DF_Send(temp, 32)
    DF_End
    waitcnt(clkfreq/1000*100 + cnt)

PRI DF_Buf(buffer, buf_byte) | data

  'Prepare the command
  if buffer == 1
    data := $84
  elseif buffer == 2
    data := $87                   
    
  data <<= 24

  data := data | buf_byte 'Combine command with data

  DF_Send(data, 32)
    
PRI DF_Buf_to_Main(buffer, main_page, erase) | data

  'Prepare the data
  if erase == 1
    if buffer == 1
      data := $83
    elseif buffer == 2
      data := $86
  elseif erase == 0
    if buffer == 1
      data := $88
    elseif buffer == 2
      data := $89

  data <<= 24
  
  data := data | (main_page << 10) 'Combine command with data
  
  DF_Send(data, 32)

  outa[noparse][[/noparse]DF_CS]~~
  
PRI DF_Read_Page_Bytes(main_Page, page_byte) | data

  data := $D2

  data <<= 24
  data := data | (main_page << 10)
  data := data | page_byte

  DF_Send(data, 32)
  DF_Send($0000,32)  '4 dont care bytes

PRI DF_Send(data, bits)

  dira[noparse][[/noparse]DF_DATA]~~
  outa[noparse][[/noparse]DF_SCK]~ 'SCK low for mode 0 
  repeat 1    
  outa[noparse][[/noparse]DF_CS]~ 'CS low

  data <<= (32-bits)

  repeat bits
    outa[noparse][[/noparse]DF_DATA] := (Data <-= 1) & 1
    outa[noparse][[/noparse]DF_SCK]~~ 
    outa[noparse][[/noparse]DF_SCK]~
    
PRI DF_Recieve(bits) : Temp

  dira[noparse][[/noparse]DF_DATA]~ 'set to input
    
  if bits > 32
    bits := 32

  Temp := 0

  repeat (bits)
    outa[noparse][[/noparse]DF_SCK]~~ 
    temp <<= 1   
    temp := temp | ina[noparse][[/noparse]DF_DATA]                                    
    outa[noparse][[/noparse]DF_SCK]~

  Return Temp
  
PRI DF_End

  outa[noparse][[/noparse]DF_CS]~~
  dira[noparse][[/noparse]DF_DATA]~~ 'set to output again 

Hope this helps a little.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
-Alec

My our page

Bill Henning

Hi,

No code I am afraid, but I did figure out a one chip solution for you.

Assumption: you will erase the chip before beginning the data logging

Chip I recommend: AT25DF641, 64Mbit flash, 1ms program time per 256 bytes (> 200Kb/sec write speed, exceeds your requirements)

It supports using SI/SO for dual-bit SPI-based mode, propeller should be able to bitbang that at around 900k2bps, for 1.8mbps data rate, which is > 180KB/sec, which is greater than your 150KB/sec requirement.

Only four pins needed.

Best Regards,

Bill

pgbpsu said...
I was wondering if anyone out there has code for working with Atmel DataFlash they are willing to share. I know Basil was using it in his rocket datalogger. I checked the OBEX and didn't see anything. Basil did you ever post your code? If so where, if not, would you be willing to share it?

I'm (still ) working on a seismic datalogger that records 5 24-bit channels at 10Khz each. When we set off an explosion we want to capture as much as 30 seconds of data. I haven't yet found a storage solution that's fast enough to keep up with this and retain the Prop as the brains of the operation. I've tried the Vinculum/USB and straight to SD card no luck on either count. So it looks like I need to buffer the data and once the shot is over I can write the data to disk. At that point I have all the time in the world. At first I gave up on the DataFlash because, although it could be clocked faster than I needed, that was only getting data into the onboard buffer. Once the on board buffers were full there was a significant wait time before the data were committed to memory. Too much for my application. After doing a bit of figuring I think if I use one DataFlash for each channel I can get this to work.

All that is just long-winded way to ask if anyone has DataFlash code they'd be willing to let me have a look at. Other suggestions for buffering memory are welcome.

Thanks,
pgb

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
www.mikronauts.com - a new blog about microcontrollers

pgbpsu

@basil- Many, many thanks for posting your code. I'll have a look through it this weekend in anticipation of my hardware arriving next week. From my reading of the datasheet and some testing in spin, I don't think I'll SPIN will be fast enough for what I need to do. Nonetheless, writing something in PASM will be easier if I can see what needs to be done and verify that the hardware is working in SPIN. Again, thanks for so generously sharing your code.

How do you feel about the robustness of the DataFlash cards?

@Bill Henning- I didn't see that chip when I was looking over the site and I've already ordered a few samples of the AT45DB321Ds. Although the pin count will be higher I think (using Terry's trick of tying SDI and SDO together with a resistor) I can manage the 11 pins required and still get all the other things connected that I need: 2 com ports for GPS, 6 lines for the 2 ADC chips, 1 com port for Zigbee radio, lines for the SD card. I'm sure I forgot something. However, if I do run out of pins, I'll try that chip. Thanks for the suggestion.

Hopefully, Monday or Tuesday I'll have some hardware to play with and I can report back on my progress.

Thanks again to everyone for their suggestions and contributions.
pgb

rjo_

on the compression issue... if you integrate the sum of p(i)log2(pi)... where pi is the frequency (percent occurance) of a particular value... over the full range, ·you will get a number representing the·minimum· bits per sample required for lossless compression using optimum compression...for the data sample you test.· It doesn't indicate how to compress ... just what the best compression will achieve.· This is without any filtering... just compression.

I like real data... so the path that you are on seems the best to me...but in a crunch you can filter filter filter and then compress.

Post Edited (rjo_) : 10/17/2008 2:51:03 PM GMT

Dennis Ferron

Incidently the Atmel 45DBxx series of flash is used to store data in a popular brand of digital label maker that is on my list of items to hack. [noparse]:)[/noparse] It'll be a while before I can get to that project, but if I write a Propeller driver for it, I'll surely put it on the Obex.

Basil

pgbpsu said...

How do you feel about the robustness of the DataFlash cards?

Well, i had a few issues whe power was lost during a read or write.
It had something to do with the state of the pins when the power was restored and cause alot of grief!

If you are carefull about the order in whhich you change the pin states you should be ok though.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
-Alec

My our page

pgbpsu

@rjo_ - I agree that real data is the best data. I think the design we've laid out will allow us to preserve the original 24-bits of each sample so until I need to do otherwise, we'll go with that. If, next week when I actually get my hands on these things, I find out I don't have enough space/speed, I'll return to the compression mentioned here. Our seismic source is pretty band limited (<300Hz) relative to our sample rate so I maybe these compression suggestions (even first differences) will be better than I suspect. There's now way around using dataflash so I've got to write up that code. Once that's done having 3 - 5 of those chips shouldn't be too much worse that the first one. If (and that's a big IF) that works than there'll be no reason to worry about compression during storage.

@Basil - I'll keep your comments in mind. I hope to tie SDO and SDI together to save on pins so I'll pay special attention to the order of changing states. Thanks for the warning.

@Dennis - I hope the chips I ordered will be in on Monday so maybe I'll have something worked out for you before you get to hacking the label maker. If/when I get something worked out, I'll post it back here.

pgb

Tracy Allen

pgb,
I've used several hundred of those AT45 chips in systems that are deployed long term in rough environments, in terms of temperature, but not as rough as Basil's in terms of vibration. I have no complaints about the durability or reliability of the dataFlash. (Granted mine is a Stamp-based system, and a very slow data rate.).

-- Be sure to include the bypass capacitor right next to the chip.
-- I have SO is connected to a 1k resistor and that to SI, and that to the CPU data i/o pin. The clock line is shared with other SPI peripherals.
-- If your experiment will not fill the whole memory, it is probably a good idea to do some wear leveling. At each new run, the origin pointer is moved up to the previous end pointer, and data is stored from that point modulo the memory length.
-- You have to decide what to do if the memory fills, either stop, or else continue on keeping the most recent data instead of the old.

I had a question about seismic data in general and the compression issue. I've always heard about it on a logarithmic amplitude scale, Richter of course, but I'm no expert. When we see a seismograph chart as we often do here in California, the wiggles on paper are much larger for a major quake, but not orders of magnitude larger than for a small one. There is some compression there. That is why I was wondering if an analog front end with logarithmic compression might help? Resolution concentrates at the low end and favors ratios over absolute values. Maybe the data is already in that form from the sensor. It comes down to the question, if there is a magnitude .01 quake on top of a magnitude 7 quake, does it matter?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Tracy Allen
www.emesystems.com

Dennis Ferron

Thanks pgbpsu! That would help a lot. Good luck on your project!

pgbpsu

After a couple of weeks working with our new hardware, I've got something working. However I'm getting some strange results which I wonder if anyone else who's used DataFlash with the prop has seen. I hope this isn't too far from what the forums are for. I'm just hoping someone with more experience might recognize these symptoms.

In all the figures below were looking at Counts/Amplitude vs sample number/time. Sample rate is 10Khz which means theres a new sample every 100uS.
Figure 1 is the response of a microphone for ~5 seconds. It looks just like it should.
Figure 3 is the Propeller Counter value vs sample number. Every second it gets reset by a 1PPS from a GPS unit. This looks just like it should. Although the systems starts recording a wee bit early.
Figure 5 is the microphone channel; captured 5 minutes after Fig1.
Figure 8 is the mic again, another 5 minutes later.
Figure 11 is the mic for the last time; again 5 minutes after the last acquisition.

What baffles me is that not that things started alright and then turned to garbage, but that is seemed to recover (a bit) between fig 8 and Figure 11.
The proximity of the problems in figures 5 and 11 make me think that it's something to do with the DF cards. Since the total number of acquisitions will be low, I'm erasing the entire chip every time so I don't think there should be old data still left behind. But how can I be sure? It seems like I'm not fully erasing the chips, but sometimes things actually turn out just fine again.

Could this simply be interference?

Any crazy ideas?
I've been getting a lot of help from this forum on this project. For that I'm truly thankful.

Regards,
pgb

jazzed

Just noticed this thread - I don't hang out here much anymore.

Here is an AT45DB161D PASM version that allows 3 or 4 pin interface and
allows burst read access at 350KB/s, flash program time is not empirically
known although it should finish in specified time.

spiasm-flash.spin:
Basic features are get mfg id, buffer read, buffer write (one buffer at a time);
no chip erase, or advanced "recording" features.

- added -

spiasm-flash2.spin:
Basic features are get mfg id, buffer read, double buffer write and program,
chip erase, and page erase. This driver requires erase before write and should
allow up to 85KB/s or more continuous double buffer flash write operation.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔


Post Edited (jazzed) : 11/23/2008 8:29:58 PM GMT

Tracy Allen

Hi pgb,

It's not clear to me what the graphs in your last post are supposed to look like. If fig8 is the correct one, why? and how does that relate to the wrong? ones fig5 and fig8, which look to me more like the microphone output in the middle sections. Is the number of samples in the bad sections a multiple of the AT45xxx page length?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Tracy Allen
www.emesystems.com

pgbpsu

@jazzed
Thanks for sharing your code. I worked up some assembly code similar to what you posted. I'm happy to post mine, but at the moment, I'm still not sure it works correctly. I've got a few spin front-ends built in to do stuff like erase the chip, set the page size from 528 to 512, etc. But given the results I'm getting I'm not sure it's ready for release. You're code looks ready to use. Dennis can use it for hacking that label maker- better than starting from scratch or waiting till I get the bugs worked out of mine.

@Terry
Thanks for taking the time to look over the plots. I was hoping you might check this thread as you seem to have considerable experience with the Atmel DF.
Regarding the plots...
The plots should fall into 1 of 2 categories. Fig1 and Fig3 are the two modes. Figure1 is actual recorded data. Here it's noise, but it has a reasonable DC shift and is what I'd expect the mic to produce in a "quiet" environment. Fig3 is the time channel. It's simply the system counter since the last 1 Pulse per second from our GPS. It also looks like it should. We're recording this so we can reconstruct our microphone and be sure that we aren't dropping samples. igures 1 and 3 were captured at exactly the same time. I have the mic going into one DF chip and the Counter going into another. When I capture a sample from the mic, I write it to HUB. I also grab the system counter as near to the sample capture as possible and write that to the HUB as well. In another cog, I have an assembly routine waiting for these samples. When they arrive in hub, this cog grabs them and writes them to DF. I was really only including Fig3 to show that my code is capable of working correctly. At least that what how I interpreted it.

Figures 5,8, and 11 should look like figure 1 as they are all mic channels.

Fig 5 does look correct in the middle section. It looks to me like something wasn't working, then worked, then stopped working again. In Fig 8 there's just a lot of garbage, but it looks to me like under it, at least in the middle section, the mic signal is still there. However, this type of record is unusable. Fig 11 resembles fig5 where things don't start out well, but then seem okay in the middle, then go bad again.

Exactly the same code for all the captures. In the case of Figs 1 and 3 captured at exactly the same time to two different DF chips. You're question about the number of samples in bad sections being a multiple of page size is a good one. I don't have matlab on this machine so I'll have to get back to you on this. Are you suggesting corrupted pages in the DF? Maybe this is a naive/stupid question but is this possible? Does one manage these with code, or simply replace the chip? If that's the case, how does one explain Fig11 where things are work (at least during the middle section) where they didn't in an earlier capture (Fig8). All these data were recorded on the same unit. And Figs 1,5,8, and 11 all used the EXACT same DF chip (here I don't mean make and model, I mean exact same serial number). Fig 1 was recorded first, then 5, 8, and 11. Since all these recording start at page 0 and are the same number of samples long they occupy the same exact part of the same exact DF chip. The chip was erased between uses. At least I issued an erase command and waited for the status register to read ready.

Again, thanks for looking over this and for all your help.
pgb

Thanks for looking at these. I'll have a look at the page size issue.
pgb

jazzed

I added double buffer write version of my posted driver.

The S08 device I'm using seems to have come up in 512 power of 2 mode as
default ... this is curious since spec says it should be 528 mode. According to the
spec, this mode is one time programmable so I may never know the truth.

Good luck with your projects.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔

pgbpsu

@jazzed-
Indeed the power of 2 register is only one time writable. It's possible that you purchased at 512-byte version. I think I recall reading in the datasheet that it's possible to purchase them already set to 512. I didn't realize that until I'd already ordered a number of the 528. I simply burned the fuse on my units as I install them.


Have you ever seen them go bad? Or return corrupt data?

p

jazzed

I've never seen a "correctly accessed" flash device of any kind return bad data since the technology was introduced except perhaps in the early days with Data-IO programmers (probably just buggy code though). It is theoretically possible for a write to be wrong yielding a "bad read" because of exceeding the number of writable cycles. It is quite easy to make flash return bad data with an incorrect access of course in terms of not cooperating with the internal state machine ... i.e. while the device is busy.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔

pgbpsu

I was unable to work on this project for some time, but I've recently returned to it and solved my DF corruption problem. Turns out, if one reads the datasheet, Atmel states the the chip erase function does not work. There's a small, tiny, nearly microscopic 1 in the datasheet where they proudly explain how to use the chip erase function. That footnote at the bottom of the page points to the 2nd to last page of the entire datasheet where it says, I'm paraphrasing here, "Despite all that good stuff we told you about the chip erase function and how it's simpler to use than the other erase modes? Well, don't use it. It doesn't work"

So there you have it. Had I read the datasheet carefully, I would have avoided a lot of head scratching, recoding and self-doubt.

Two good things learned here.
1. Use block erase as the errata states. Now my code does exactly what I expect every time.
2. READ THE MANUAL!

I wanted to post this so any future users of Atmel DataFlash might find this post before they find this problem themselves.

jazzed

Getting new device drivers working takes lots of patience and expectation of such gotcha's. I've found many hardware types will add a hack in verilog or documentation (which is somehow equivalent lolz) to get something out the door by hook or crook and leave it to you to sweat ... then they complain because you had a hard time uggh. Did you read the comments in the eraseChip method of my posted driver?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
--Steve

Post Edited (jazzed) : 2/23/2009 4:02:18 PM GMT

pgbpsu

Steve-

Unfortunately, no. I had code that did darn near everything I needed- which became complicated because I was programming 5 different DF chips in parallel. Although I didn't need the entire chip to be wiped each time, it seemed the simplest thing to do at first: use chip erase and implement wear leveling and all the associated book keeping after I got it working. At least that's what I though would be easiest. I did look over your code, but when I compared how long it would take me to understand and re-work your code, I thought I'd be just as well off to find the bug in mine. So, honestly, I never read the code carefully (until now) and the comment you have in your eraseChip method and the comment in the datasheet were lost on me because I was sure I had a programming error. Normally the devil is in the details of my code, but every once in a while the devil is in the datasheet.

Thanks for your help, if only I'd accepted it....

All's well that ends well and I'm quite happy with what I've got. Although it's not in any form for distribution, I've got something that uses the 2 buffers on the Atmel DF to support sustained write speeds of ~85Kbytes/sec. I've got that running in parallel on 5 DF chips so with one asm cog, I can stream data to DF at just over 400Kbytes/sec until each of my chips is full (4Mbytes). More than adequate for the application I'm working on: 40Kbytes/sec for 30 seconds to each DF chip.

Thanks again and next time you offer me help, I'll pay more attention.
p