Where to start with "properly" reading logged data from SDcards? aka more help

WBA Consulting

So I am finalizing my plungelogger that records temp to an SD card for this Saturday's Polar Bear Plunge and I would like to also make a "plotter" for the log file using a uOLED-128 display. I have figured out all the code to plot how I need on the display, but I am getting confused on how to use FSRW to read the values that have been put onto the SD card by the logging program.

I am currently working with my program from this thread. My data on the SD card is currently in this format as it logs temp and humidity from two SHT11 sensors:

Internal, 78.9, 42.4
External, 77.7, 42.2
Internal, 78.6, 42.6
External, 77.8, 42.1
Internal, 78.6, 43.4

Since I only need to log a single whole-number temp value for the plunge, I am thinking of changing my format to:

1, 78
2, 77
3, 74
4, 42 brrrr
etc

I would like to read the data back from the SD card and use the temperature as my offset for the Y of the pixel plotted on the uOLED to make a line chart. For example, the first internal temp reading is 78.9, so I want to plot the pixel at a Y position of 78 (and an X position equal to the logged number, IE: 1st value:x=1, 2nd value, x=2, etc so it plots to the right as a line chart).

I am currently reading through all of the FSRW files/demos to try to better understand how to use read commands. I need to understand how to control which bits of data from the data on the SD card is read and how to use that as my variable for the X value for my plotting. (I can make my uOLED plot values easily, it's getting the correct data from the SD that's got my scratching my head).

Anyone know of any simple examples of writing/reading a set of data values to/from an SD card and properly stepping through them? How do I manage where on the SD card I am pulling data (I see pointers and buffers in my future?)

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Andrew Williams
WBA Consulting
WBA-TH1M Sensirion SHT11 Module
Special Olympics Polar Bear Plunge, Mar 20, 2010

rjo_

Your temperature range is within the byte range of values... nothing less than zero and certainly nothing more than 255. So you just need to do the calls in sequence to append to a particular file...There is a complete demo in the OBEX. If you do an open fie for writing... you erase everything. My guess is that this is where you are having a problem. You don't need to write the number of the reading... you can keep track of this on the fly. Just store the temperatures as a series of bytes.

By the way, I am also heavily involved in Special Olympics... but I think you guys are completely nuts for jumping into cold water in the middle of the winter. How about "brakeless Thursday?" I love the idea of getting everyone off the rode... so we can drive around without brakes and earn money for Special Olympics by not killing anyone[noparse]:)[/noparse]

It you would move over to video... I could shake this out for you in a couple of minutes.

Good work... thank you.

Rich

WBA Consulting

I'm not having any issues with writing the data, just how to properly read them back to use as variables for my plotting on the display. I was looking into the code for FSRW and it does appear that pointers are used to keep track of the position in the file that's being read. I am going to try to read data back in a stream to see if FSRW will just handle what I need automatically by advancing the point on it's own. Here's pseudo code:

Startup
Mount SD
Open File
repeat x times (where x is the number of readings, probably 120)
__Read first value
__Plot pixel in position X, value
__next value
quit

Not sure what you mean by moving to video, unless you mean displaying the plot line using TV out instead of the uOLED. My plan is to set things up for TV out, then migrate to the uOLED. If I run out of time to get the uOLED code working, I will use my small composite display or as a last resort, just take my laptop and use excel.

Special Olympics is dear to me as my employer is directly involved with people with disabilities. I haven't been able to get anyone else here to join my plunge though, as they all think I am nuts as well.

My goal for next year is to add a heart rate monitor; that should be interesting to chart....

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Andrew Williams
WBA Consulting
WBA-TH1M Sensirion SHT11 Module
Special Olympics Polar Bear Plunge, Mar 20, 2010

rjo_

I understand... will look at it now.

They won't join you because they are cowards... like me[noparse]:)[/noparse]

Rich

rjo_

Ok

It depends on how you are writing to the sd.

If you wrote with pputc

to convert the ascii·value you read with r := sdfat.pgetc·· you just subtract 48...

so for instance
r := sdfat.pgetc
x:=r-48

if you wrote the value 2 to your file.... with pputc

x now has the value of 2.·

To get the number you actually stored you have to read in two characters and combine them into a final value.· so you would multiply 2 by 10... get the next digit and add it in.

If you use a pwrite to store the value then when you do sdfat.pgetc... your value pops out directly.


my code for writing looks like this:

·r:=sdfat.popen(string("camfile2.txt"),"a")
·sdfat.pwrite(@iod,512)
·sdfat.pclose

where 512 is the number of bytes to write from the iod array to the card

Not the clearest explanation... is it?
I think that will do it
Rich


·

rjo_

I have to apologize... I'm running around in between posts... the first part of my post is good. The second part might be complete trash[noparse]:)[/noparse]... more around dinner[noparse]:)[/noparse]


after dinner:
The way the human brain works can be absolutely charming... so, I like to· leave it out there....
In this case I was between botchiball, the dentist and the forum...

Anyway, the above post is actually correct.· I have double checked it and it works out just the way I describe[noparse]:)[/noparse]



Post Edited (rjo_) : 3/18/2010 12:37:33 AM GMT

rokicki

Right, fsrw in no way helps with parsing the ascii data and turning it into numbers or vice versa.

You'll need to put a layer on top to do that.

Note that this parsing will probably be *much* slower than anything else you do. But the parsing
is pretty easy to do.

I'd start by writing a pgets() routine that gets a "line" to parse into a fixed-sized buffer.
It can initially do this by calling pgetc() until it sees a newline or runs out of buffer space.

Then write some code to parse that line into your data. You can do this separately, using a
constant buffer, until it works, and then integrate it into the rest of the program.

-tom

rjo_

Please check my second post... my original was correct. I write my data consecutively to the sd card, using the second method. The only caveat that I can see is that in my application, the word data gets taken out of an array, gets sent over a serial line and arrives a byte at a time at my second prop and then gets put into another word array... which is then written in byte order to the sd card, using the second method above. Normally, I open it up as a raw image in ImageJ. The images have always been fine, so I wasn't worried bout the write or serial routines. Until today, I had actually have never tried reading the data back into the Prop. I graph it on the fly, send it to another prop, which stores it to the SD and then forgets about it. When I tried reading the data back in from the SDcard tonight with the prop as described above... it is fine.

It is easy to test... you just put some numbers into an array.... write them to the SDcard as described... then read them back in... add a number to each as you read them in and then display the result on the screen using tv.dec(your variable) to the screen. If you see what you want to see, you are done. If not... somewhere in the ether is a little confusion that we need help with.

rjo_

Once you have convinced yourself that this works... then you will just be writing a stream of byte sized numbers out to your sd card... how can you be certain that every number was written and that every other number will be one kind of data point and the next will be the other kind of data point?

The quick and dirty way is just to encode one kind of data by adding 100 to it. You have plenty of room in you byte to do this. Then when you read it back in... if the value is above one hundred... you subtract one hundred and log it in as that particular type of data[noparse]:)[/noparse]

You mentioned graphing... I like your idea about graphing the change from base line. You only have 200 or so vertical graph points to work with and the differences in your data should be fairly small. If you stick to graphing changes from baseline, you can stretch the changes over the entire vertical extent of the display for a more dramatic appearance , that will be easier to look at... but a little more difficult for people to interpret. But if you catch their eye, their brain won't be far behind.

By the way... we have some guys in our area... mostly cops, but a few insurance salesmen, etc. who are doing a "super" pluge, which involves them plunging into freezing water·evey hour for 24 hours.· If you would hook up an LED to their core body temperature measurement and make them wear it on their heads...· when the LEDs goes out... they need help[noparse]:)[/noparse]

I am thinking about going and taking a pole... "would you rather plunge into freezing water every hour for twenty four hours... or follow me around on "brakeless thursday?"· Why would I need a pole?· Something for them to grab onto... silly.

Post Edited (rjo_) : 3/18/2010 2:04:15 AM GMT

WBA Consulting

Well, I tried for over an hour to figure out how to pull data from the SD card that would be usable for my needs, but without luck. I am still to new and SPIN to figure out how to handle the different formats of data/bytes/etc. I am just going to focus on my logger for now and just take my laptop with me Saturday. Essentially, I need to write a list of numbers to the SD card and then pull that same list back off. Somehow I am missing the formats of the data and can't get it to convert properly. I have gotten quite good and getting gibberish in PST though!

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Andrew Williams
WBA Consulting
WBA-TH1M Sensirion SHT11 Module
Special Olympics Polar Bear Plunge, Mar 20, 2010