Multi-Accelerometer data to Memory Stick Datalogger

mica

I apologize in advance for this post. I’m a mechanical engineer jumping into electrical stuff. So you’ll probably have to use small words[noparse]:)[/noparse] After reading a few posts on this subject I get the feeling I’m in over my head.

What I want to do is write XYZ data from 3 accelerometers as fast as I can to the Parallax Memory Stick Datalogger. (By ‘fast’ I’m looking for at least 8 reads per second but I’ll take what I can get to start)

I have (1)BS2, (6)SX (not very familiar with these), (2) 3-axis and (1) 2-axis accelerometers and a number of other goodies from the Parallax candy store (oscilloscope, compass, etc.).

With BS2 stamp, Memory Stick Datalogger and accelerometers I’ve used the test programs to verify all are working and are wired correctly. My problem is the output to the Datalogger. From “DataloggerDemoV1.0.bs2” and “DataloggerTestV1.0.bs2” test programs, with lines such as:

SEROUT TX\CTS, Baud, [noparse][[/noparse]$08, $20, $00, $00, $00, $0D, CR,
DEC5 counter, ",", DEC5 result, CR, LF, CR]

I’m stuck with 10 or less character or numbers per write. So I can get the following out correctly:

12345,12345
12345,12345

But what I really want is something like the following all at one time:

Accel1,1.00,1.00,1.00,accel2,1.00,1.00,1.00,accel3,1.00,1.00,1.00

Is this possible or is there some limitation due to Baud rate or something like that? If this is a limitation is there a common compression scheme?

I was able to use two SEROUT commands to get one accelerometer output in the format I show above. But with the delay between the SEROUTs I’m loosing the accuracy I wanted. So I would like to stay away from that.

And I’m open to other output methods and even other chips if you think it is appropriate. Maybe the spin chips with one CPU per accelerometer then another CPU to write?

Unsoundcode

Hi mica, the 3rd 4th 5th and 6th hex values of you serout string represent the number of bytes you will be writing which in you current example is $0D (13). This requires you know ahead of time how many bytes you intend to transmit. This is fine in the example because the variables are fixed at 5 bytes (DEC5 counter and DEC5 result) that is 10 of your bytes, the other 3 are the comma, CR and LF. All you have to do is increase the 6th hex value to write additional characters.

Heres an example

SEROUT TX\CTS, Baud, [noparse][[/noparse]$08, $20, $00, $00, $00, $22, CR, "Acel_1 ",DEC3 A1,",","Acel_2 ",DEC3 A2,",","Acel_3 ",DEC3 A3,CR, LF, CR]

I changed the $0D to $22 (34) because thats how many bytes I will be sending (including the spaces·at the end of·the strings)

Hope I counted right but it gives you the idea

Jeff T.

EDIT One thought that will help increase the speed of you data collection. If you write the headers Acel1 , Acel2 and Acel3 just once on initialisation and format so the data falls below each respective header you will more than double your write speed

Post Edited (Unsoundcode) : 9/9/2007 5:36:19 PM GMT

mica

Hi Jeff, Thanks. I was thinking it was that simple but the combinations I tried where not getting it. With your confirmation I'll keep pounding on it.

Have a good weekend.

Mica

John Bond

Hi Mica

Your post is interesting - I was playing with a two axis accelerometer yesterday. I was writing to the internal EEprom on a BS2sw at 10 readings a second. Reading 3 axis to a memory stick should be do-able

The reading of the accelerometer isn't the problem because you're looking for 3 pulses. Lets say the worst case is the microcontroller misses the 1st pulse of each read and therefore you need 9 milliseconds to read G-force in every 100 milliseconds (assuming 10 read/writes a second). You have 91 milliseconds to write these 3 words (6 bytes) to your memory stick. There’s no information of how fast the Memory Stick Logger device writes to the stick but I would think it’s going to be at least one order of magnitude faster than writing to the slow EEprom on the chip.

You did say only 8 times a second… didn’t you?

Good luck and Kind Regards from Kwa Dukuza in Darkest Africa.

John Bond

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔

Unsoundcode

There was a post here a few month ago where we discussed an inconsistancy with the write time to a Vdrive. I don't remember the exact details but if data was written to the drive above a certain frequency the drive would stall for as long as 0.5 sec while it did some internal processing. I am thinking this was in the region of every 30 bytes or so. It would be interesting to know if the Parallax Datalogger exhibits the same characteristics. If I can find the link I will add it to the post.

Jeff T.

mica

With Jeff T's help I was able to write more to the datalogger but I get some fun stuff like:

a1x,y,z,a2x,y,z,a3x,y,z,ruEBeBa1x,y,z,a2x,y,z,a3x,y,z,run

with the "B" being blocks when I look at it in notepad. I think they are carriage returns. It should have written:

a1x,y,z,a2x,y,z,a3x,y,z,run

So I'm not sure what the problem is but I'll keep in mind the possible 30 byte issue. I'm just past that amount.

Thanks guys, good information. I'll get back to it this weekend. The problem could be in the in the Get_Data: shown below but I'm guessing based on my limited understanding. Full code as it is, and still a work in progress, is attached "accel_2c_USB.bs2" if anyone wants to look.

Get_Data:
index = 0 ' Reset Index Pointer
DO ' Receive Data
SERIN RX\RTS, Baud, 100, Timeout, [noparse][[/noparse]ioByte]
buffer(index) = ioByte ' Add Received Byte To Buffer
index = index + 1 ' Increment Index Pointer
IF index > 14 THEN Timeout ' Check For Overflow
LOOP

Timeout:
RETURN


Mica

Unsoundcode

Hi Mica, here is the link I mentioned. The circumstances here though are different to yours. At that time Chris Savage's·code wasn't available and we were coding without CTS RTS flow control. It worked well but the code with flow control probably overcomes some of the problems we were having.

http://forums.parallax.com/showthread.php?p=644902

I noticed you had 1E (30) in part of your code which, if I counted right, should be 1D (29). The final CR of the SEROUT is part of the command and not data to write.

hope this helps

Jeff T.

mica

Hi Jeff, good catch on the 1E and 1D. I was desperate and was just playing with values to try and get something to work. Possible reason for my odd results.

I'll take a look at the link tomorrow.

Mica

John Bond

Hi Jeff

Thanks for that "heads up" on flow control. I may use the same device.

John

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔

mica

Hi Jeff and All, I wanted to post a quick update in case it helps others. In the 'standard' programs for the USB recorder I finally found the one bit the kept messing things up. In the Get_Data the Timeout was too short. Originally it was 100. Once you get the program going this is probably okay but to start it should be more. Or add a DEBUG to know when you've tripped this.

Get_Data:
index = 0 ' Reset Index Pointer
DO ' Receive Data
SERIN RX\RTS, Baud, 500, Timeout, [noparse][[/noparse]ioByte]
buffer(index) = ioByte ' Add Received Byte To Buffer
index = index + 1 ' Increment Index Pointer
IF index > 14 THEN overrun ' Check For Overflow was > 14

Another interesting bit was voltage or power issues. I was using a 2 cell LiPo going through a voltage regulator. As the voltage dropped off I got more and more timeouts from the above SERIN. On my voltmeter, going through the BS2 regulator it reads 4.98 volts with a good source like a 9V battery. When the Vdd to Vss dropped below 4.71 the USB recorder got flaky. Something to keep in mind.

I am now able to get one accelerometer output to a USB but I'm only getting 2 writes per second due to timeouts. As I get the efficiency of he code worked out I'll post it for those that are interested.

mica

Oh yeah, I did run into the problem with length of data sent. I could get 25 bytes (if that is the right term) through but not much more. I didn't dwell on it to find the exact amount but 30 sounds about right.

Unsoundcode

Hi Mica, I said earlier that I wrote a few programs that had no handshake connection and no time outs in the serial commands. I relied on the fact that if the drive received a good command it would return a prompt ">".

I don't think·that is particularly·good practice but it was ok for tests and easy to·put the error checks·in at a later date.

I modified one of my test programs to write the data string in your example. I didn't include variables just the test string. I was able to write the data in excess of 10 times a second.

You are welcome to try it if you want. I used Pin 0 as the Stamp receive line and Pin 1 as the transmit, CTS was connected to VSS and RTS was left not connected.

When you have all the wires hooked up power up and download the program. At the prompt for a file name insert the memory stick and wait for the logger to finish examining·it. Type a file name and it should go. The program is configured to write at 19200 (which it will), to change this to 9600 comment out GOSUB changebaud and change the constant baud CON 32 to baud CON 84. The debug screen is best run full screen.

You will see I split the CR,LF from the main string into its own write sequence, I was getting errors sending as one, not sure why.

Jeff T.

EDIT Ok I did find the reason for some of the errors I was getting. I was flip flopping between two memory sticks a PNY 1G and a Lexar 64Mb. The PNY would give me errors at times where the Lexar ran fine.

Post Edited (Unsoundcode) : 9/18/2007 3:20:30 AM GMT

mica

Hi Jeff,

Thanks for your efforts. In using your program I found the main USB stick I was using would not work at all. So I used another one and had some luck. This new stick seems to work most of the time. So I cleaned it off and re-formated it. Then the results got very spotty. I tried both FAT and FAT32. No real help.

Is there maybe another format that works better for this. Maybe a hidden file? I know I'm reaching.

Is it possible the USB logger has some type of buffer we need to clear. In the test files like "DataloggerTestV1.0.bs2" in the SERIN, is pulling the information out clearing this some how:

Get_Serial_Bytes:
timeout = 1 ' Set Timeout Indicator Flag
index = 0 ' Initialize Index
DO WHILE (timeout > 0) ' While Timeout Has Not Occurred
ioByte = 0 ' Clear Temporary Storage
SERIN RX\RTS, Baud, 100, No_Data, [noparse][[/noparse]ioByte]
buffer(index) = ioByte ' Save Byte Received To Array
index = index + 1 ' Increment Index
IF (index > 14) THEN ' Check For Overflow
index = 14 ' Prevent Overflow
ENDIF
LOOP
RETURN

Using bits of these test files works with all my USB sticks but the byte array for the ioByte is taking too much room.

Another kind of funny thing I found was when watching the voltage when I unplug the system it takes a long time to settle. Maybe 30 seconds or so to drop below 0.1 volts with a very smooth exponential decay. Like it has a real big capacitor. Probably means nothing but I thought it worth mentioning.

In case someone else follows along this path, to get any real write speed from the test program I disable the GOSUB Get_Serial_Bytes: above for writing data. This speeds things up considerably. It is fairly reliable the first few times through but falls apart if you stop the program and start again. Again, I think its a buffer issue.

Thanks again Jeff,

Mica

Unsoundcode

Hi Mica, my memory is formatted Fat32, with the Lexar I get 100% write success with the example program. Things do get iffy when I mix a string with numeric data.

With the example program I am writing over 5000-5500 bytes in around 14 seconds at 9600. I guess it could be a memory stick issue, that is the only problem I have had.

sorry the example didn't work for you

Jeff

mica

Hey Jeff,

Little different tactic on the same subject. The ability to send data to a USB stick is a little to valuable for me to let go of. I see two paths:
1) Can I pay someone to look into this? They would either need to solve the 'buffer' problem (I think I'm having) or find a good combination of code and USB stick that would get me what I want, namely a minimum of 10 writes per second of the size I'm using. Even when I got your code working with a Baud of 19200 maybe it was 10 writes per second but I doubt it. Seemed a lot slower (i.e. I don't think I was seeing what you were seeing or maybe I drink too much coffee). So a shorter question, is there an avenue to pay for someone to do this type of work? Someplace in the $100 or $200 doesn't bother me but much more and I would have to think more about it.

2) The other path is to have a dedicated chip to act as a buffer to write to the USB using the test type code. So one chip to get the accelerometer data or whatever, then send out the data via hard line, IR or RF to another chip that would only send data to the USB data logger. Given I already have a number of SX chips this is probably the cheapest route but adds more complexity in both hardware and code.

Mica

Unsoundcode

To be honest Mica I think with a little perseverance it will eventually come together for you. I know the frustrations when things are not working right but it only takes perhaps one small element to say "Eureka".

I will certainly persevere and see if I can get some more acurate timing on the data. Like I said earlier the speed is going to be improved if you only send the string header once and place the data in order below it. Which if you are using Excel is really what you want anyway with a CSV file. Is this an option??

Actually yesterday I was playing with the Acceleration code adding in the error checking I mentioned, keep plugging away

Jeff

mica

Yes, I've been hoping for the Eureka bit but with the flakiness of getting data to the device I keep thinking I'm missing something. The SERIN in the demo code baffles me why it makes the code so much more functional. With the oscilloscope I've been watching the different TX, RX, RTS, etc trying to get a handle where the problem is. Maybe find some indication I can feed back off of to make your simpler code more robust. Still don't have a handle on that info to say anything yet.

And yes, I have been writing just a header once then sending the data in a separate SEROUT. With this, and bits from the demo code it does fairly well. Both speed and repeatability are good. Problem is it takes so much memory I can't do much more. Ideally I want two accelerometers a clock and a compass. Probably just too much to ask for.

Again, I appreciate your efforts. I need to put what I had working before back together for a demo today. So I will not get back to anything original until late today. Don't spend too much time on it if you have other fun stuff to do.

Mica

mica

Hi Jeff,

Don't want to bug you but this might help... or hurt. Attached is a zip file (hopefully) with pictures of my setup, debug window, resulting files and my program. The program is a mess and hard to follow with lots of debug statement. I also left in the slower Get_Data for the accelerometer write. And I'm also not getting the reference voltage for the accelerometer so the results are pretty much junk. The program has been very reliable though.

Also, I use a switch to in PIN 5 to tell the program when to write and when to stop. So you'll see a ..1..2..3 in the debug. This is a pause mode before sleeping if I want to make a new file and write some more.

Again, a mess but might help.

Mica

Unsoundcode

Hi Mica, I did a few tests today using variations of the logging program. I am convinced the type of USB stick used is crucial in collecting reliable data. Unfortunately I don't have a large collection here at home, just the one Lexar and several PNY. The Lexar is an old stick I have had 2 or 3 years, its a 64Mb Jumpdrive, this memory stick never missed a beat. The PNY are all 1Gb Attache, I never could get a full set of data without errors with any of them·( I have 3 ).

So the results of the tests are focused around the Lexar using a BS2. I repeatedly ran 200 samples of data each sample had 17 bytes including commas CR and LF. I repeated tests at 4800,9600,19200 and 38400. With the best that I got the difference in write speed between 9600 and 38400 was only approximately 1.5 samples per second. At 38400 the average was 5.5 samples per second. For test purposes I was reading the data from EEPROM, obviously the time it takes to read your data source is going to have an effect on the write times.

You may be able to improve on this, I think finding the right memory stick is certainly the place to start.

Jeff T.

mica

Hi Jeff, I agree it sounds like finding the right USB stick might make this functional. Now to figure out which one. I have another experiment to try. My one stick that seemed to work with your code was almost full when it worked. After formating it reliably dropped off. So I'm going to load it up and try to mix it around and see if that helps. Slim chance but maybe the way the logger accesses the data it'll work better that way. I'll let you know my findings.

Failing that I'm going to give the 433MHz RF transmitter/receiver a try.

Mica

Unsoundcode

Hi Mica, I rearranged my test program a little to optimize.

My last post wasn't worded right, the way I gathered the samples from EEPROM was·to read·3 axis from 1 "virtual" accelerometer at a time and then·write.·So it was not 200 samples that I wrote it was 3x200 writes plus a CR LF every 3 writes. Which totals 800 writes. The reason I did it that way was to conserve variable space by using the same 3 variables for each axis.

I optimized by removing the string array I was using for the text file name and re-allocating·the variable space·to the three accelerometers ( a total of 9 word values ).Now each accelerometer axis has its own dedicated variable.· Writing is reduced to a total of 200 times·x 46 bytes (data + commas + CR + LF).

I am now back to 10 writes a second at 9600 without a single error. You are going to lose some of that·when you read the data from the accelerometers and a little more if you use a time stamp but it might still fall within your expectations.

Even then I think you ought to be able to get over 5 or 6 writes a second. This figure would improve with a Propellor but if your memory stick is flaky it's not going to help whichever processor you use.

Jeff T.

mica

Hi Jeff, I tried my experiment with loading up the USB stick and mixing around files. With your program I got it to work two times. Never again after that. Funny thing. It seemed to hang both times around 71 writes then kept going. No idea what that means.

So next experiment was to get another flash drive. The smallest I could find and not be PNY was SONY at 1GB and $25. This never worked. It did write the name of the file I used twice, not in order of attempt, but never any information.

I suggest you keep your lucky USB stick in a safe place. I'll see if I can round up some old ones from some friends and try those. But I've spend too much time on this so I'm going to try the RF route.

Wile testing both of these sticks I tried some experiments with waiting before giving the name and leaving the unit powered off for a few minutes before attempts. See if that voltage drop off was an issue. Didn't seem to make a difference.

If I learning anything more I'll post it but otherwise I'm done for now. If I get up to having a dedicated chip to use the 'long' program I'll post that too.

Thanks again Jeff. You were a great help in co-testing.

Mica

Unsoundcode

Hi Mica, I think I may have found the reason for the errors on my PNY Flash Drives, what is even more pleasing is that I believe I have found a remedy. I reformatted the Flash drives and instead of leaving them at the·default allocation size I entered a value of 512.

To alter the allocation size you have to go to Control Panel---Administrive Tools---Computer Management. Right click your Flash Drive and select Format, from the options you see select File System=·Fat32 and Allocation Unit Size=512

I know you are thinking of taking another path now but if this works for your flash drives it is something to bear in mind for future reference. Since I have formatted in this way my errors have stopped.

Jeff T.

mica

Hi Jeff, I'll give that a try later this week. But my one USB stick that worked a few times is only 256MB. Maybe I'll cut that down to 64MB and see how that goes.

Another friend found a number of other USB options from http://www.hobbyengineering.com/SectionIN.html. As I do want this capability, I just don't have time keep playing with the one I have, I'll look into these later (after I test your idea above).

BTW - I got the my BS2 to ping my SX via the 433MHz transmitter/receiver last night. Hopefully today I can do a data transfer test. Very exciting.

Mica

Unsoundcode

Hi Mica, I don't know what the default allocaion is but the 512 allocation is 512 bytes so there is nothing to stop you from formatting your 256Mb that way.

I also brought the test program more in line with your data so I have attached the version 2



Jeff T.

Unsoundcode

Hi Mica,·I can't be sure but I would say that say that the USB devices at·Hobby Engineering·are not going to perform any differently to the Parallax logger. You may not have had the time yet to try with a flash drive formatted to 512 allocation units but for me and the setup I have it was the eureka I have been looking for. I extended the number of writes to 47 bytes x 500·, the average speed increased greater·than 10 per second writing constant values. I wrote 5 consecutive logs all without error.

Having a little free time today I took a look at how the variable space might be utilized for your program. I don't have a clue about accelerometers but I tried to assess the method you were using and came up with a subroutine that iterated the axis, assigned the values to a 9 word array and writes them to file. If the data needs some kind of conversion and you needed additional variables I think it is possible to gain another 3 to 5 bytes. You would gain these bytes by making the file name a constant string value or limiting the file name to a 2 character variable string.

I attached version 3 with a couple of ideas that might help

Jeff T.