Easy to use datalogger

cessnapilot

Hi,

A new datalogger driver object is placed in OBEX (http://obex.parallax.com/objects/542/)

The object uses the standard Parallax Serial Terminal for UART and supports 3 types of datalogging:

1. Comma Separated string format Values into lines of CSV textfiles

2. Data bytes written into FAT files in small (32-64 byte) packages·····························

3. Collected data (bunched 512 bytes) written directly to disk sectors
······························
The slowest supported method is printing data into CSV textfiles. It works only for about a 150 sensor values per second. However, this yields the most convenient result to read by a human or for further analysis with MS Excel, for example.

Appending packages of raw data bytes to FAT files can be a good compromise between rate of measurements and ease of post processing on a PC. About 500 sensor readouts(=<16-bit) per second can be stored to FAT files with the driver sending 64 byte data packages to Vinculum at 57 Kbaud.

For direct sector writes I use good quality, empty formatted drives of size 1-2G. The driver is able to write NED or ECEF coordinates, velocities and yaw, pitch, roll attitudes (altogether 40 bytes with CNTtime) 100 times per second directly as they stream out from a 6DOF IMU accelerometer array. The data rate is about 4.5 KBytes/sec and a 2G thumb drive can record it for days.

The demo exercises each of these datalogging methods with speed tests. The rate of data transfer to USB disks depends on many factors.

- First of all there is a large variability of the quality of· USB flash disks. Some makes and models are definitely faster and more stable than others.

- Data transfer speed, of course, depends on the baud rate of the monitor firmware. The driver uses standard baud rates from 9600 to 115200.

- Transfer speed depends on the fragmentation state and the amount of free space on the drive. Because of this, response time of the Vinculum monitor to a FAT filesystem commands is somewhat unpredictable. Empty or not fragmented drives can be filled up seemingly faster than fragmented or nearly full drives.··

The data transfer measurements, that are summarized in the following table, were made with an acceptably good quality, empty formatted 2G drive. Peak transfer speed was measured with continuously repeated writes in 3 logging modes while the driver was running at 80 MHz.

Vinculum Monitor··· Sector write····· Bytes to file··· · ·CSV data to file
·· baud rate······· · bytes/second···· bytes/second··· ··· values/second

---

··
·· 115_200········ ···· 9_730······· ··· · · 520· 970········ ··· ·· ~120
······
··· 57_600··········· ·· 4_980········ ·· 1_170· 890········ ···· ·· ~140
··
··· 38_400············ · 3_340······· ·· ·1_030· 800··········· ·· · ~100

··· 18_200·········· ··· 1_690············ ·· 890· 630················ ·~80
···· 9_600··········· ···· · 850············ · ·600· 420··················~50

---

················································· ·· ( 64·· 45······· ·· (3-6 Qval, FLOAT
········································· ····· · byte packages)· ·· · and LONG / line)

The last digit of the actual experiental figures is cleared, not rounded. These figures are typical. The drop of speed of the 'Bytes to file' method at the highes baud rate with the 64 byte packages is systematic.

Beside the standard functions (File_Open, etc...) the DataLogger_USB_driver v1.0 contains additional procedures for each of the supported datalogging methods.

For CSV textfiles:
- Line_New
- Line_Append_Int
- Line_Append_Qval
- Line_Append_Float········ ·· (this one would go with FPU or with FloatSting object)
- Line_WriteCRLF
- Line_Seek(Line No.)
- Line_Read······················· (Next read will read next line without seek, of course)····
- Value_Read(Index, Type)··· (But, you have to know which type is at what index)

For byte packages:
- File_Random_Access_Read
- File_Random_Access_Write
- File_Random_Access_Insert
- File_Split
- File_Merge
- File_Copy
·····
For direct sector storage, beside the standard read/write:
- Sector_Peek(DataBlock)
- Sector_Poke(DataBlock)

The driver uses the RESET line of the Vinculum and has full· Standby/WakeUp control.

A PASM UART capable of 230 Kbaud with full hardware handshaking and with a 512 byte buffer in COG seems to be neccessary for version 2.0. It would be great to have a ready made one from somewhere. With that driver the data transfer speeds of all three methods would increase noticably. Preliminary experiments (with PST at 230 Kbaud) indicates a minimum 50% increase of speed of each methods, but the handshaking with SPIN does not work at that baud rate.

Until that, enjoy v1.0


Cheers

Istvan

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔


Intentionally Left Blank

Don M

Istvan- I was messing around with your object. In particular I was tryig the Demo that you have included. I connected the Memory Stick Datalogger just as shown in your documentation and downloaded the files but what am I supposed to see? Is the Parallax Serial Terminal supposed to spawn on its own and provide some output messages? If not am I supposed to open Serial Terminal and connect to see the messages? Either way I see nothing. The files all compile ok and download into my propeller.

Please excuse my ignorance here as I am new to the propeller platform. Just trying to understand what is supposed to happen here and what I should be seeing.

Don

cessnapilot

Hi Don,


During the actual development I used different pins then published. It was a larger breadboard containing, or connected to some additional circuitry. After everything worked properly (at least with the Logger) I rebuilt the Datalogger circuit in the prototyping area of a Propeller Demo Board, as shown on the schematics. This was good for the development of the demo code in a 'clean' situation.

I wrote the demo code to send proof messages to the PST PC application, while·exercising·most of the·features of the Parallax memory Stick Datalogger. During the development of the comprehensive demo code,·some bugs·of the driver appeared and were smashed.

The final check went as follows:

The Demo Board was connected to a PC with a USB cable.

A thumb drive (empty, formatted) was plugged into the datalogger, power was on· on the Demoboard.

On this PC I used the IDE for the Propeller and the PST application, but not simultaneously.

I opened the IDE, loaded the 'DataLogger_USB_Demo'

Complied the application into EEPROM.

I put IDE on the tray and in the next 6 seconds I started PST.exe (preconfigured to 57Kbaud, using same COM port as IDE) on the PC.

(The demo program uses the same pins for communication with the PST applications as the IDE uses for program download.)

So messages of the demo program appeared on the PC in the text window of the PST.

Using the driver the demo application created, copied, splitted, merged, appended· files, wrote and read CSV files. All these files were listed on PST to verify operations. Sectors were read&write and their content were dumped to screen of the PC, too.

You do not need the demo code and PST ·for your application, just the driver. PST is a·great debugging tool and the demo code contains examples of how to use the driver.·Both (PST, demo code) ·might be useful in the development of your application.


Welcome on the platform!


Istvan.·

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔


Intentionally Left Blank