GPS data logger
Here is my data logger project. It is a portable, battery-powered unit that writes several hours of GPS data to RAM, then feeds it into a PC display program.
A Scenix SX28 runs four VP UARTs. One receives 4800 baud data from a Garmin GPS sensor. One sends 9600 baud data to a 2-line Seetron LCD display. The last two are the 19200 baud PC interface: the PC sends request codes to the logger, and the logger returns all saved data.
The logger is powered by 4 AA NiMH cells, which run it for about 8 hours. It has 512K of CMOS RAM for data storage, enough to log almost 7 hours of data when logging every second. A board-mounted switch lets the user select that data is to be saved every 1, 2 or 5 seconds, further extending the logging time if you want.
The logger saves the date, time, latitude, longitude, altitude and speed to RAM every second (if the GPS receiver is synced to satellites). A small amount of data condensing is done: the date, the full time, latitude and longitude full degrees and minutes are saved only on the minute. Each second, only the seconds, the longitude and latitude fractions of minutes, speed and altitude are saved, but enough is saved each second to be able to fully reconstruct all data.
I actually had an MMC/SD version of this logger working, but those cards were so much additional trouble that I abandoned that approach; CMOS RAM provides enough storage for my needs and is much easier to deal with.
The LCD displays the current latitude and longitude on the top line and altitude, speed, time, amount of RAM saved, etc. on the second LCD line.
Data readout is simple: when the logger receives the character "A", it returns all data in ASCII-readable form. HyperTerminal can be used to extract the data, but I'm in the middle of developing a program that will read in the serial data, save the raw data to file, decompress the data to its full form, add a descriptive comment, and save to a "processed" file, ready for display.
I'm also in the middle of developing a display program. In its current state, the program lets individual logfiles be selected, assigned a plot color, then displayed. The included screenshot shows a couple of logs of bicycle rides around my town. You can add logs, remove them, set the color, zoom in and out, and I'm working on the display of altitude vs. distance and speed vs. distance. But right now the program has lots of bugs - it's got a long way to go before I'm comfortable publishing it.
The project uses wire-wrap construction and is mounted in a cardboard box (a Parallax shipping box) which also holds the GPS sensor and the batteries. It's just the right size to fit into a backpack or a bicycle saddlebag and be able to hold the GPS sensor positioned vertically.
I'm hoping to be able to provide a more detailed schematic, photos and the display code when I get the time, and if anyone expresses interest.
David
A Scenix SX28 runs four VP UARTs. One receives 4800 baud data from a Garmin GPS sensor. One sends 9600 baud data to a 2-line Seetron LCD display. The last two are the 19200 baud PC interface: the PC sends request codes to the logger, and the logger returns all saved data.
The logger is powered by 4 AA NiMH cells, which run it for about 8 hours. It has 512K of CMOS RAM for data storage, enough to log almost 7 hours of data when logging every second. A board-mounted switch lets the user select that data is to be saved every 1, 2 or 5 seconds, further extending the logging time if you want.
The logger saves the date, time, latitude, longitude, altitude and speed to RAM every second (if the GPS receiver is synced to satellites). A small amount of data condensing is done: the date, the full time, latitude and longitude full degrees and minutes are saved only on the minute. Each second, only the seconds, the longitude and latitude fractions of minutes, speed and altitude are saved, but enough is saved each second to be able to fully reconstruct all data.
I actually had an MMC/SD version of this logger working, but those cards were so much additional trouble that I abandoned that approach; CMOS RAM provides enough storage for my needs and is much easier to deal with.
The LCD displays the current latitude and longitude on the top line and altitude, speed, time, amount of RAM saved, etc. on the second LCD line.
Data readout is simple: when the logger receives the character "A", it returns all data in ASCII-readable form. HyperTerminal can be used to extract the data, but I'm in the middle of developing a program that will read in the serial data, save the raw data to file, decompress the data to its full form, add a descriptive comment, and save to a "processed" file, ready for display.
I'm also in the middle of developing a display program. In its current state, the program lets individual logfiles be selected, assigned a plot color, then displayed. The included screenshot shows a couple of logs of bicycle rides around my town. You can add logs, remove them, set the color, zoom in and out, and I'm working on the display of altitude vs. distance and speed vs. distance. But right now the program has lots of bugs - it's got a long way to go before I'm comfortable publishing it.
The project uses wire-wrap construction and is mounted in a cardboard box (a Parallax shipping box) which also holds the GPS sensor and the batteries. It's just the right size to fit into a backpack or a bicycle saddlebag and be able to hold the GPS sensor positioned vertically.
I'm hoping to be able to provide a more detailed schematic, photos and the display code when I get the time, and if anyone expresses interest.
David
Comments
Very nice project. I'm impressed.
Bean.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
"SX-Video·Module" Now available from Parallax for only $28.95
http://www.parallax.com/detail.asp?product_id=30012
Product web site: www.sxvm.com
"If you keep doing what you always did, you'll keep getting what you always got."
·
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
·
Steve
http://ca.geocities.com/steve.brady@rogers.com/index.html
"Inside each and every one of us is our one, true authentic swing. Something we was born with. Something that's ours and ours alone. Something that can't be learned... something that's got to be remembered."
I've already been asked to provide a version for auto datalogging (accelerometer, RPM, speed, various sensors) and one for agricultural sensing (soil moisture, solar radiation, rain, temperature, humidity, and wind in several places), but I want to get a more basic version working first
Which Garmin unit are you using?
-dave
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
This is not a sig. This is a duck. Quack.
I'd considered breaking this project into two parts - a generic data logger and a GPS message receiver/parser unit. I've also thought about adding other sensors - a bike wheel rotation sensor, a heartrate sensor, maybe a pedometer sensor, for logging. Ahh, maybe someday...
Any general pointers?
Tom
PS Do you know anyone who has hacked the Polar heart rate pickup?
Gradient would be tougher, because you'd probably want to separate bike acceleration and deceleration from the road angle. I suppose a long-period filter would do it. Maybe a two-axis accelerometer like the Memsic, that Parallax sells, could work?
GPS does elevation OK, but with lots of fluctuation; much more than for longitude/latitude. Maybe atmospheric pressure might work better? I don't know.
A few years ago, I put on my Polar heartrate sensor and held a small coil up to it while my oscilloscope displayed the coil's pickup. As I remember, the transmitted signal was 4 or 5 cycles of 455 kHz pulses per heartbeat, with a pretty strong output. So I don't think it would be too hard to turn that into pulses that a CPU could process, but I never did it.
I've never used the Javelin, but all the above features are probably doeable on any platform, because they would be changing at such a low data rate.
This project would involve a fair amount of hardware hacking, but would be a great way to get some experience - you could start with maybe a single wheel rotation sensor, then add sensors and functionality as you gained practice.
Good luck!
David
Looks like a good device to track the movement of a teenage driver ....
A Perl script expands the data so each row becomes a full time-latitude-longitude-altitude data entry. The processed data is saved to a file named by the date and time and still contains the text description in the first line.
A Java program reads a directory of these files and builds a list of the file descriptions. I can select one or more descriptions, and the Java then plots the lat and lon, properly scaled. Each route can be set to a different color. Once plotted, I can zoom in to the scale of feet or zoom out to the scale of miles. The data for time and altitude are logged, but currently I don't do anything with them.
In my latest burst of energy, I modified the Java to store all data points in an embedded Postgres relational database, rather than in Java arrays, and the display issues SQL queries to retrieve the selected points. It's not real advanced but is working.
Now this all sounds great, and it works pretty well, but I have to confess I am not a programmer. Anyone who feels strongly about programming style will probably puke over my code design, but hey, it's just a hobby. I'll be happy to send along my code to anyone who wants it.
David
Some time ago I posted a link to a Hitachi whitepaper on embedding a microdrive in a project. It's written primarily for the 8051 uC, but it has some good background info that would be applicable to any system.
Here's the thread:
http://forums.parallax.com/showthread.php?p=553269
Post Edited (Kevin Wood) : 12/3/2005 5:24:12 AM GMT
Hello from Gregg C Levine
Isn't a shofar a ram's horn used as a device to call the people to prayer during one of the many Jewish holidays?
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Buck Rogers
www.gregg.levine.name
The cause could have been me. During my learning how to access the card, my code could have accidently looped and performed hundreds of thousands of writes to those sectors and caused the damage myself. I was quite careful with the voltage - I don't think I ever accidently allowed other than the 3.3 volts onto it.
A minor problem was the sector write time. It takes about 23 milliseconds to write a sector. I would flow incoming GPS characters straight into the MMC, and as soon as the count reached 512, the card would perform the write. So during the write period, incoming characters had to be managed or else they'd be lost.
My GPS had been configured to send at 9600 baud, but that rate could produce 23 characters during a write, requiring a complex data buffer that would span multiple SX pages.
At first I reconfigured my GPS to send at 1200 baud, and made an 8 character SX buffer, but at 1200 baud, the GPS didn't have time to send as many messages as I wanted - position, including altitude, time, date, velocity in 3 dimensions. Eventually I compromised by upping the GPS send rate to 4800 baud and making a 16 character SX buffer.
But after the errors showed up, I reconsidered the entire project. The huge storage capacity of the card was nice but not required for my purposes. The ability to store data during power-off also was nice but not critical.
For my purposes, conventional RAM, either static or dynamic, was just as suitable and would avoid the error problem, the need for buffering data during writes, would need no additional 3.3 volt supply and had a much simpler interface.
So I went for the conventional RAM and quit MMC/SD card development.
It was kind of fun learning about the cards and the FAT filesystem. I had even developed a plan where I would format the FAT with generic files that spanned the entire card. My logger would sequentially write to the file data area, and so a conventional card reader could, by reading the files, retrieve the logged data. Through a PC interface I was able to reformat a card and have it recognized by a card reader!
David
have a look at www.spring.com/seaclear
The software enables you to scan a (marine) chart, or a scaled map, calibrate your scan, then plot coarses, waypoints and a whole bunch of other stuff, then upload to a GPS.
We used it at the beginning of last month to bring a boat about 1,000 nm down the coast, and it was very impressive.
For your use, I think you will find that you will be able to upload your logged data back into a scanned map, thereby bring a whole new graphical dimension to your efforts.
Scott me up, Beamy
bongo
The shofar is indeed the ram's horn.
Great project. I'm working on a data logger also, but I'm having trouble with the reduced data rate of the GPS. One NMEA string per second is not enough resolution when the other sensors (ping sensors, MEMSIC accelerometer, temp sensors and Hall Effect sensors) are providing data at up to 20Hz.
This data logger is for my racing motorcycle and things happen fast at 160mph so a capture rate of 1Hz isnt enough. I'm still looking for alternative methods to capture more GPS data.
I've been using STAMP Plot to display the data from the other sensors while looking into methods of plotting the GPS data. I'd love to see the code from your JAVA app.
As you say, my GPS sensor and many other sensors update only at a maximum of once per second. There are faster systems - the Garmin GPS16A can send 5 readings per second. The data sheet says that their lat and lon are at a different precision at those faster rates, but doesn't supply any more information than that.
But you might want to do some of your own tests or research before relying on this too much, because I've found that my GPS sensor does a fair amount of internal averaging before delivering results to the display. Once I was speeding down the road on my bicycle, watching my GPS-reported velocity, when I had to stop fast for a red light. I sat there stopped, and watched my GPS velocity display slowly ramp down over about 10 seconds to zero, at practically a classic linear averaging ramp pattern. So just because numbers may be reported at a faster rate is no guarantee that there will be any more or any better information in the data.
I had a helpful response once from a Garmin engineer when I emailed him a technical question about how the GPS velocity is derived. You might want to try the same - try to contact an engineer to verify that your specific needs will be met by any particular GPS sensor.
I'll package up my java code and post it here within the next few days.
David
A friend just showed me what a commercial GPS data logger can do. It uses a 5Hz GPS receiver and displays speeds to 0.1mph: http://dpcars.aprsworld.com/900/recent.htm.
A system like this would be nice but its an awful lot of money.
I'd like to hook up the garmin 16a in conjunction with my digital compass and speed sensor, but at $400 for that receiver I might stick with the sensors and abandon the idea of using GPS.
Shaun.
Unzip the zipfile into its own folder, run the "make.bat", then run the "run.bat", and you should be able to show the half-dozen demo paths I've included. Run "clean.bat" before recompiling.
You will need Java installed, but I think that's about all. I've tested this on WinNT and Win98, and I've run it on Linux in the past, although Linux operation may need some datapath editing to get the correct filesystem directory separators.
Have fun!
David
I'm going to run a comparison between a gps sensor and a wheel speed/compass around a closed circuit to see which is more accurate.
Where did you learn the procedure for writing to them?
I have been looking for a while and can not seem to find it.
I’m researching a up and coming project
A good first step is to gather as many documents as you can find. Here is a good place to start:
http://www.sandisk.com/Assets/File/OEM/Manuals/SD_SDIO_specsv1.pdf
There are many other documents at this same site, as well. But you really have to pick and choose; many of the documents you find are just high-level sales blurbs.
It may be helpful to look at code that others have written to do this same thing.
The next step I would suggest would be to use a PC and its parallel port to start probing a test card. It's hard enough figuring out something as complicated as this card without being hobbled by the delays of programming a microcontroller to test every code variation. I like to use Borland's Delphi, because it compiles really fast, has a nice GUI and lets me drive the PC parallel port.
You need to make a 3.3 Volt card supply. You need to also either make signal voltage translation stages, or you could run the card signals directly from a Scenix at 3.3 volts and skip the signal translations, but with that method you'd need to power up your SX-Key itself with 5 volts.
You can use a section of PC card edge socket to connect to an SD or MMC card, but it is much nicer to buy some sockets. Mouser sells some pretty inexpensively.
You need to initialize the card in several stages. You power it up with certain pins set for SPI mode, if that is what you want to use. You also need to send like 80 initialization clocks to the card at power-up. Then there will be some code initializations.
Code - you need to develop a byte transfer routine that reads bits at the same time as you clock out bits. This is not too hard, but it is easy to get the order wrong of whether you write and read bits before clocking or after, and it is easy to get the bit polarity wrong, and it is easy to mess up the order in which you disassemble the sending byte and assemble the received byte.
Then you need to combine a series of byte transfer routines to make complete operations. Many operations take time, and you sometimes have to extract the correct status bit from one of the reads to figure out what status the card is in.
At different stages during sending a command you have to sometimes send a dummy byte in order to read a valid byte, or read a dummy byte in order to send a valid command - the docs don't make this clear but that's how the card works.
There are several software command initializations you need to start with. These will distinguish an SD card from an MMC, which you need to know, because they finish their initialization differently.
There are several layers of errors - there are errors due to sending a command with illegal parameters, and there may be separate errors due to the card attempting to execute a valid command but failing. So you need to be careful to try to trap all the different points of possible failure.
Once the card is initialized, a good test is to read the hardcoded configuration registers. If you can identify the card this way, consistently, then you're doing just about everything right, and you're ready to do some data transfers.
If you want, at this stage, if you started with a formatted card, you can dissassemble the FAT filesystem sectors and trace the directory and data storage. But that's a whole nother layer of technical information in itself.
The technical specs were not easy for me to follow, but in hindsight, I have to admit that everything you need to know is indeed in that document. It is hard reading, but there really is a lot of detail needed to use these cards.
Good luck,
David
My name is Jerome from Quebec, I have a few simple questions.
Presently I have a Garmin Etrex and would like to know how it works.
I'll tell you first what project I want to do, I want to read GPS data and put them in a buffer so I could build a robotics guided robot. I know there is some projects out there, I have seen them but I need to know how the GPS works. I might not use a basicstamp for the final design.
I am refering this link www.parallax.com/dl/docs/article/rambal-gps.pdf It is the RC plane with GPS data logger.
Fisrt of all, the GPS only use two pins ( ground and dataout ).
question #1 - Does this mean that the GPS always send data via dataout?
SERIN Rx, Baud, TimeOut, No_GPS, [noparse][[/noparse]WAIT ("@ " ), SKIP 6, DEC2 hr, DEC2 mn, DEC2 sc, latHemi, DEC2 latD, DEC2 latM, DEC2 latF, SKIP 1, lonHemi, DEC3 lonD, DEC2 lonM, DEC2 lonF, SKIP 1, SKIP 4, altSign, DEC5 alt, ewDir, DEC4 ewVel, nsDir, DEC4 nsVelserin 8, 16572, [noparse][[/noparse]wait.("GPRMC,"),wait(","),wait(","),wait(","),wait(","),wait(","), wait(","),wait(","),dec tr1] 'puts serin data in RAM, it is the current true headingSERIN GPSpin,N4800,[noparse][[/noparse]WAIT.("$GPRMC"),SPSTR 65]I am not really familiarised with basic.
question #2 - Why people are waiting for wait("GPRMC,") and WAIT("@")
Why is that?
question #3 - Is GPRMC the command that tell's you when it is the beginning of the communication or frame? Sorry I can't find the english definition. the beginning of a Frame??
question #4 - Is this what I receive in my buffer?
The only thing I need to do, is set a baud rate and a buffer. I just always read the GPS and put the data in the buffer?
I did not try to hook up the GPS with a basic stamp because I need to make the cable
In other words I just want to know how does the GPS serial communication works? If you respond to my question it might help.
I want to use it first with a basic stamp then with a PIC. With a PIC I can receive by interrupts
Thank you very much for your time!
Jerome
google is your friend.
If you do a google on GPRMC you'll find out what it breaks down to mean. Also, there are other strings output from the GPS that may be of use. Google the string prefix to determine what they are and if you can use them.
The WAIT modifier is in the PBASIC help file within serin. It basically leaves the stamp input waiting until it sees the specified character. When it does, it'll continue with the code.
Take your GPS and connect it to your computer and use a terminal program (hyperterminal) to open a com port and view the data coming from the GPS.
You'll see many lines starting with various GP codes. Each one means something for that string. (google to find out).
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
·
Steve
"Inside each and every one of us is our one, true authentic swing. Something we was born with. Something that's ours and ours alone. Something that can't be learned... something that's got to be remembered."
These are all good questions, but I have no "SERIN" experience.
May I suggest that you start a new topic in the "Sandbox" section, or the "BASIC Stamp" section with these questions, where you will probably reach more people who would be able to answer.
David