Some simple Data Logging/Retrieval Questions

crgwbr

I'm looking at using the memory stick data logger for an industrial application; I just have a few questions:

1. What's the maximum usable size of the USB flash drive?

2. I want to use a Computer app to write a text (.txt) file (composed of 6 decimal numbers and 6 letters) to the drive. When the user removes the drive, and places it in the data logger, can the propeller read the text file and extract the numbers ad letters?

3. When logging data to the flash drive, what format is the file in? (example .txt, .rtf, etc)

4. How much data per second can I put on the drive; if there is a bottleneck, is it at the Vinculum or the flask drive?

Thanks,
Craig

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
"... one of the main causes of the fall of the Roman Empire was that, lacking zero, they had no way to indicate successful termination of their C programs." -

"If Python is executable pseudocode, then perl is executable line noise."

"The best accelerator available for a Mac is one that causes it to go at 9.81 m/s2."

"My software never has bugs. It just develops random features."

"Windows isn't a virus, viruses do something."

"Programmers are tools for converting caffeine into code."

"Enter any 11-digit prime number to continue."

Mike Green

1) I think it's 4GB. The Vinculum (memory stick data logger's chip) currently uses FAT16 and I believe that's limited to 4GB.

2 and 3) The Vinculum does not know about the format of the file. As far as it's concerned, it's just a sequence of bytes. It's up to the Propeller or Stamp program to handle that. The Propeller can certainly read the text information and interpret it. The Vinculum doesn't know about text lines. You ask it to supply a certain number of bytes and it provides them so it's easiest if the data in the text file is fixed format (a certain specific number of characters per line including the end of line delimiter(s)).

4) It's variable. If you're using serial I/O to communicate with the Vinculum, the Baud of the serial I/O sets an upper limit to the transfer rate. If you're using SPI, it depends on the Propeller's SPI routines. The Vinculum has a certain amount of buffer space. Once that's full, the bottleneck is the flash drive which has its own buffer space which varies from one flash drive model to another. The Vinculum also has to handle space allocation on the flash drive's file system. If the file is preallocated and contiguous, the Vinculum doesn't have to do extra reads and writes to access it. Remember that the flash drive is still flash memory and sector writes take at least 5ms once the buffers fill up.

Default Baud is 9600. This can be changed to 230KB and above. The FullDuplexSerial driver only goes up to about 230KB.

Internal Vinculum buffer space is less than 4K. There's 4K of SRAM and it's used for multiple purposes.

Post Edited (Mike Green) : 9/18/2007 3:10:59 PM GMT

Fred Hawkins

I have seen warnings in the documentation on SD cards about how the fat is handled. They warn about major slowdowns if the fat is updated for every little write to the card. The fat update forces a copy, an erase and a write of at least a page and maybe the entire fat. They recommend writing as big a chuck as is possible and then updating the fat.

If your flash drive looks and works like an SD card, that's where I would expect a bottleneck.

Mike Green

Fred,
The handling of the file system (FAT) is a function of the operating system (or data logger in this case). It's not handled by the flash drive or flash card. The Vinculum chip handles the FAT in this case and we don't know how optimally it does this. Preallocating the file eliminates this concern since the FAT doesn't need to be updated at all.

crgwbr

Thanks for the replies. So, a text file would show up as a sequence of bytes. How would I open a specific file (say, "Depth.txt")?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
"... one of the main causes of the fall of the Roman Empire was that, lacking zero, they had no way to indicate successful termination of their C programs." -

"If Python is executable pseudocode, then perl is executable line noise."

"The best accelerator available for a Mac is one that causes it to go at 9.81 m/s2."

"My software never has bugs. It just develops random features."

"Windows isn't a virus, viruses do something."

"Programmers are tools for converting caffeine into code."

"Enter any 11-digit prime number to continue."

Mike Green

Read the Vinculum firmware manual (see here: www.parallax.com/detail.asp?product_id=27937 for VDAP Firmware Spec). It has plenty of examples.

crgwbr

Thanks again; just have a couple final questions. In the example code for the bs2, it uses CTS/RTS lines in addition to RX and TX. How can I accomplish this with the Full duplex Serial driver?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
"... one of the main causes of the fall of the Roman Empire was that, lacking zero, they had no way to indicate successful termination of their C programs." -

"If Python is executable pseudocode, then perl is executable line noise."

"The best accelerator available for a Mac is one that causes it to go at 9.81 m/s2."

"My software never has bugs. It just develops random features."

"Windows isn't a virus, viruses do something."

"Programmers are tools for converting caffeine into code."

"Enter any 11-digit prime number to continue."

Mike Green

The Stamp needs the RTS/CTS lines because its serial I/O isn't buffered. For the Propeller, you can just ignore CTS and tie RTS through a 10K resistor to ground. The FullDuplexSerial routines have a 16 byte buffer. You might consider increasing the size of the receive buffer to 64 or 128 bytes. I believe someone posted a modified FullDuplexSerial driver that included this. The buffer size has to be a power of 2 and there are two or three places in the code where a bitmask is used to wraparound the buffer pointer. There's also one place in the assembly initialization where the code computes the start of the transmit buffer given the address of the receive buffer and uses a constant of 16 for this that'd have to be changed.