buffer 10 seconds of data until trigger then record to SD?
Dwayne Dibbley
Posts: 63
Is it possible for the propeller to have a 10 sec buffer of data, that when a trigger is tripped this 10 sec + live data is recorded to a sd card?
idea is for a gps data logger, that records your track data only when moving over a certain speed. then when you stop the log is closed
eg
gps incoming data ,add this live data to a 10 sec buffer purging old data out when new is stored then when speed > 30 mph record the 10sec buffer + all live data to a file then when speed drops to 0 for x secs close log file?
Thanks
idea is for a gps data logger, that records your track data only when moving over a certain speed. then when you stop the log is closed
eg
gps incoming data ,add this live data to a 10 sec buffer purging old data out when new is stored then when speed > 30 mph record the 10sec buffer + all live data to a file then when speed drops to 0 for x secs close log file?
Thanks
Comments
LOG04.txt
Thanks
http://forums.parallax.com/showthread.php?104049
I built this on a Proto Board with all Parallax parts.
Since most GPS's will output once a second, you have plenty of time to write info to an SD card once a second. No need to buffer. I have used this program to log 300+ miles at a time.
You just have to do a little logic to extract the speed out of the $GPRMC sentence to start the logging. The program puts the header and footer for Google Earth when it opens and closes the SD file.
i need at least 10hz output rate, so i can overlay my videos with the data like my example here:
The version of FSRW (V2.6 the SD card object) I checked uses just over 4Kbytes of memory. 32k - 1k (for FSRW) - 15k (buffer) you'll need to fit you stuff into remaining ~16Kbytes. Depending on what else you need to do that should be doable.
Thanks
index := 1
repeat
add array(index) "text(index)" to array ' at index 1
index := index + 1
if index := 4 then index := 1
only guesses the code to try an explain
then when trigger is triggered dump the array from the current index + 1 to current index to sd card?
or maybe another spin way ?
Thanks
Without knowing what else you needed to fit, I thought you'd have enough space. What about cutting down the NMEA messages. Do you really use the checksum? Couldn't you use the position, speed, etc as numbers rather than strings? Do you need the time twice? I think you could probably get all the information you need from the messages into half the current size. That sounds much simpler than adding additional memory.
What speed is your GPS serial port? Writing to the SD card will almost certainly be faster, however you have the overhead of opening and closing files and there is some jitter in the time it takes to dump a buffer to SD card.
I would probably start by setting up as large an array as possible. Cog A fills the buffer. Cog B dumps the buffer. When cog A has filled at least 1/2 the buffer it alerts cog B. It continues filling the second half of the array. Cog B dumps the 1st half, then waits for the second half to fill. You continue ping-ponging back and forth. If you want to squeeze a bit more performance from this setup you could use a circular buffer. It's a bit more complicated but give you more time to write to disk.
Where did the 10 second buffer requirement come from? I suspect two 2-3 second buffers would be more than enough to keep up with things.
Or you could go Mike G's route and skip the buffer.
pb
Yes that's right so I get all the data from a standing start automagicly recorded, instead of recording all the time and starting stopping manually. so i would need some sort of circular logging for that 10 sec buffer to loop round.
And yes I could simplify the NMEA strings but the current video overlay software imports NMEA strings, and I would need to additionally ( later ) create an import template for the new format.
Thanks
I understand the problem better now. One could cut down the size of the NMEA strings before storing and then "grow" them back to just before writing them, although that's starting to sound cumbersome. Redoing my math from earlier. FSRW needs about 4Kbytes of memory. FloatMath and FloatString need an additional 1.6Kbytes. A 10 second circular buffer would need (80chars*2messages*10hz*10seconds) = 16Kbytes. 32Kbytes-16Kbytes-4Kbytes-1.6Kbytes= 10.4Kbytes.
How large is your LCD driver? And what else do you use the gps for? Do you make use of it in the program, or do you just need to grab GPS serial data and spit it to the sd card? If the latter you don't really need a GPS driver. One of the serial port drivers would do it.
I still believe there's a way to do this, if I understand all this program has to do, without resorting to external memory.
p
I think it was Phil who told someone to "Just learn to do it the right way" or something like that when it came to a circular buffer. I had been avoiding the subject myself but I took Phil's advice (though not directed to me) and learned to use a circular buffer. This is really the only "right" alternative for your need.
I think it was Phil who suggested the Wikipedia article on circular buffers. I learned a lot from it.
I also agree with pb. You should be able to reduce the data needed during those 10 minutes by changing the way the data is stored.
If you decide to use external memory, you'll want a kind that doesn't "wear out." SRAM is a good choice.
Here's my attempt at a DIY SRAM module.
not got as far as the lcd driver yet . gps will also be used for a trip meter ( currently just using math with the speed in knots from the gps ) this will hopefully displayed on the lcd when i get that far. apart from that in the future i hope to add accelerometers to the log.
im currently breadboarding with a quickstart board is that suppose to have a 64k eeprom?
Thanks again
Most of the QuickStart boards have 64K EEPROMs. The first batch made have 32K EEPROMs.
Here's a program you can use to test your board.
If it's 64K every other LED will flash on and off every four seconds.
If it's 32K all the LEDs will flash on and off every two seconds.
This is not the purpose of this program, but it should to the trick for you.
Duane (Now that flows from the fingers.)
Edit: I was correct about the number of LEDs but not the druation of the flash.
I don't think the EEPROM space will be much use. The read/write nature of this will kill it off too quickly in my opinion.
I would actually suggest using the quickstart board to wire up the SD card and GPS. Use the 4-serial ports in one object to talk to the GPS and send debug messages to the terminal/PC. Get that code working and see if you can log data the way you want to. When that works you can go back and wire up the LCD.
Have you picked an LCD? If you haven't I'd suggest a unit that accepts serial commands. You can then use the 4-serial-ports in one object, to read the GPS and write to the LCD. Doing this should still get you under the code limit.
Duane is right that learning to do a circular buffer correctly is worth the effort. If you have one cog feeding it and one cog emptying it you need to be careful that you don't overwrite data that haven't been dumped yet, handle buffer overrun, etc. It can be done. I believe the FSRW object writes 512 bytes at a time to the SD card so you may want to use a circular buffer and anytime it contains more than 512bytes you write that chunk. Most of the time your buffer will be essentially empty. At start-up when you are trying to collect the 10 seconds of stationary data and when the SD card is opening and close files, you'll get a small amount of filling, but generally the SD card is more than fast enough to dump serial port data.
In fact, I would probably go with a 12 second circular buffer. You'll need a little bit of time (storage) to handle incoming data while you are writing the 10 seconds of stationary data. Once you start writing, you'll quickly drain the buffer. But that's alright.
Since pb mentioned the four port serial object, I thought I'd mention my "improved" (in my opinion) versions.
I doubled the RX buffer from 64 bytes to 128 bytes without increasing the program size more than a few longs. I just rearranged the "circular buffers" so they reused space in the cog image. It was my attempt to understand Tim Moore's driver when I finally learned to use a circular buffer.
The four port driver (Tim deserves the credit for it) has really help me in a lot of my projects. You might want to look through the thread with my versions. Besides the 128 byte RX buffer version there is a 512 byte RX buffer version.
Duane
64K quickstart is what i have but as pgbpsu says eeprom would not be the best place from the buffer.
of to learn some more on the circular buffer
Thanks
Yes, that thread does discuss a circular buffer.
I think it would be good to figure out if you'll need some external RAM. External RAM isn't so hard (especially since I know how to do it now). I think one 23K256 SRAM would be plenty of memory (32KB).
The driver I wrote can read and write to the external chip faster than 100KB/s. There are faster drivers than the one I wrote (I wrote mine as a learning exercise). It looks like you need to write at about 1.5KB/s so the SRAM would more than meet you needs.
The SRAM I used cost $1.14 each (in quantity of 25).
15KB is lot of hub RAM to give up for a buffer.
If you think you can get by with hub RAM, that would probably be an easier solution. But hey, wouldn't it be cool to have the option of using external memory if you needed it?
Duane
still haven't a clue how a circular buffer works yet
DD
With a circular buffer you have variable "head" keeping track of where new data should be written. You have a second variable "tail" keeping track of where the data should be removed.
Once the head reaches the last position in the buffer it jumps to the first position (0).
In FullDuplexSerial, if the head catches the tail the method waits for room in the buffer before proceeding. In your case you'll want the tail to move to make room for the head.
If you use external memory and need about 15,000KB of data in the buffer, you might want to use 16,384 byte buffer. 16,384 is two to the power of 14. There are tricks you can do with power of 2 numbers that make them useful for buffers.
if easier to write than
BTW 16,383 = $3FFF.
I like to use the hex notation when using the bitwise AND "&" (I can go from hex to binary a lot easier than decimal to binary in my brain).
DD
speed data looks like this for example ( each digit is 1 sec and using a 3 sec buffer 123 )
so the above show the 3 sec buffer looping 6 times till speed =1 then record to uSD the 6th buffer (011) + all the live data and then stop logging / close file when speed trigger = 0 for 3 secs then back to buffering again
Does that make things clearer
DD