Datalogger - writing requirement question

xanatos

I am writing a set of numbers to a datalogger and I want to save variables in doing so. The data length is several words in length, consisting of several individual hex bytes. The output file is a comma delimited text file.

The question is - can I open the file, and use a for/next loop to write each var, terminating the last entry with a lf and a cr, or do I need to write each line as as a complete line? An example of the desired file line as viewed later in something like notepad would look like:

76,09,07,02,16,54,09,30

I want to open once, write many, and close once, then sleep the logger until I issue another WKD and repeat the whole process. Am I thinking correctly? I'd just try this - but I am on my iPod Touch about 200 miles away from the hardware I could test this on! Thanks for the info!!!

Dave

CounterRotatingProps

Dave,

since you're going to read this on Windows, you'd need to delimit each line with CRLF

when you say 'write many' how many is that - you said 'several words in length' earlier, but that's not many. Normally, if the write is in batches and is huge chunks (as in 100k bytes plus) you open, write, flush buffer, write, flush, etc. close. Or open, write, close, pause, open write close.

If it's just a few lines like 76,09...30, just open, write, and close

> I open the file, and use a for/next loop to write each var, terminating the last entry with a lf and a cr,
> or do I need to write each line as as a complete line?

Depends on the language and the write [noparse]:)[/noparse]
I'd do it like this:

open file
for data_rec_cnt 1 to number_of_records   //(or number_of_records - 1 )
   for data_byte_cnt 1 to record_size     //(or record_size - 1 )
      write hex_btye to file
      write "," to file
   end for
   write "CRLF" to file
end for
close file
Have lunch

hth
- Howard

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔


Post Edited (CounterRotatingProps) : 7/5/2009 10:37:39 PM GMT

xanatos

OK, I'm at least on a full size laptop now so I can type & copy stuff from my BS2 code...

Here's an example of a line I am writing in full:

SEROUT TX, Baud, [noparse][[/noparse]$8,$20,$0,$0,$0,$2C,CR,"08,06,18,09,19,31,09,00",CR,$0A]

I know the 6th value in that package needs to be teh exact byte count in hex (in this example, it is the $2C element, which is probably NOT correct for the example, but I digress...). What I need to be able to do, and it sounds like you have confirmed this as correct above, is something like (loose approximation of code...):

open file
for x = 1 to TotalBytesToWrite
X = first var
SEROUT TX, Baud, [noparse][[/noparse]$8,$20,$0,$0,$0,$01 + X + ","]
X = second var
SEROUT TX, Baud, [noparse][[/noparse]$8,$20,$0,$0,$0,$01 + X + ","]
X = third var
SEROUT TX, Baud, [noparse][[/noparse]$8,$20,$0,$0,$0,$01 + X + ","]

etc... until all the vars have been written into the file dataline, then

SEROUT TX, Baud, [noparse][[/noparse]$8,$20,$0,$0,$0,$01 + ,CR,$0A]

Then a line to put the datalogger back into sleep mode until the next time the need comes up to write data.

Why am I doing this? I have more data to write than I have variables left for my stamp. I want to use one variable to pick up a byte, drop it into the file, go back, get the next byte (different var in a different location), drop IT into the datalogger, and so on until all the requisite vars are read in, then just close the file and - have lunch. Or dinner as will be the case in about 10 minutes... It sounds like I can do this, I just wanted to be as specific as I can be without having the hardware here to test it with, as I am writing a large code block relatively blind here and when I get back to the hardware I want to hit the ground running as it were.

Thanks very much for your help.

Dave

PS., Just to clarify, normally I'd just make a single string variable that would contain the whole thing by setting someVar = var1 + ", " + var2 + ", " + var3... etc., but I don't have enough space left for that fat string variable with everything else I am doing...· I have only 3 bytes left! :-)



Post Edited (xanatos) : 7/5/2009 11:05:37 PM GMT

phil kenny

Dave,

I'd encourage you verify that the Data Logger sleep mode works
properly.

When I ran the Parallax demo code a while back, the Data Logger
never woke up. That might have been fixed with the latest firmware
update.

Something else I found useful is the addition of some sort of visual
indicator showing when the file is open or closed. Shutting down
power with an open file leads to loss of data.

phil

xanatos

Hi phil! Yes, I'll definitely check the datalogger sleep/wake functions! Thanks! As for powering off while file open, the operator will not have power control to the system, and programattically I want to ensure that file is closed after the write routine is done. The write duty cycle is going to be something along the lines of 1 set of writes as shown above every few hours.. so the file should be open for a few milliseconds, and closed for hours. Unfortunately this item needs to run reliably on its own - there will be no one technically competent to service or understand what a file open indication would mean - I, as the programmer, need to be sure all housekeeping happens no matter what, unfailingly, in a technical-service vacuum... or else I'll be doing a lot of driving!

Thanks for the heads-up about the sleep issue. If that turns out to be a problem, this thing is on mains power, with battery backup, so I'll just leave it awake and bored!

Dave

Post Edited (xanatos) : 7/5/2009 11:58:16 PM GMT

Tracy Allen

Dave,


It is not as you showed in your message just above where you opened the file and then had separate SEROUT statements each with the command to send one byte. After opening for write, there needs to be one statement telling it how many bytes in total to expect. Subsequently the values can be spread out over many SEROUT commands either in line or in a loop, followed by the command to close the file.

' open file for write, then...
SEROUT TX, Baud, [noparse][[/noparse] $8,$20,$0,$0,$0,Nvalues*3+2,CR]
FOR idx=1 to Nvalues
    SEROUT TX, Baud, [noparse][[/noparse] HEX2 X(idx), ","]  ' 3 bytes per value
NEXT
  SEROUT TX, Baud, [noparse][[/noparse] CR,LF]   ' 2 bytes for CRLF
' ...close file

That could be extended over several lines of data, but they all need to be covered by the one $8 WRF command with its count of Nvalues plus punctuation.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Tracy Allen
www.emesystems.com

xanatos

This is awesome! Thank you. I am home now and getting close to being able to try it out, I'll let you know!

Thanks again,

Dave

xanatos

Well, I'm not doing SOMETHING right.... Here's the scoop.

First, here's the vars/IOs I have set up:

DLRX CON 1 ' P1 assigned TO SERIN on Datalogger
DLTX CON 0 ' P0 assigned TO SEROUT on Datalogger

Baud CON 240 ' 9600 bps (BS2sx)
DLBaud CON 500 ' Datalogger 4800 baud (BS2sx)


Now - I need to have this thing run with the USB drive inserted for hours at a time - probably 24 hours at a time. During that time, there will be the intermittant writes as described above in previous posts. At some point, someone will pull the USB drive, copy the file to their PC, and re-insert the USB drive (I intend to have the USB Drive.

I have taken the code from the SimpleLog.bs2 demo code from the product page, and combined a little from the Vinculum manual to see how the code works, and what the shortened command set allows, etc. I have placed the following code in the "Initializations" section of my code:

DEBUG "Initializing USB Logger... "
SEROUT DLTX, Baud, [noparse][[/noparse]"IPA", CR]
PAUSE 200
SEROUT DLTX, Baud, [noparse][[/noparse]"SCS", CR]
PAUSE 200
SEROUT DLTX, Baud, [noparse][[/noparse]$14,$20,$71,$02,$00,$0D] 'SET BAUD TO 4800
SERIN DLRX, DLBaud, [noparse][[/noparse]WAIT(">")]
DEBUG "Initialized ",CR
PAUSE 200

It never goes beyond the debug of the "Initializing USB Logger... ". Just sits there.

Doe sthat code look correct, or is there something previous to that that needs to happen?

I have tried this code with the USB Drive both inserted and removed.

The SimpleLog.bs2 demo application requires that you remove the drive and reinsert every time you want to write something - this can't work for my application. I need to insert the drive once, and have it accessed multiple times without removal and reinsertion, and likewise, I need the system to be able to initialize and recognize the USB drive on power-up with the drive inserted.

I simply do not understand enough about the workings of this item to recognize what I am doing wrong, and again, I humbly submit my ignorant self before your good graces in the hopes that it will be obvious to you what my code is lacking.

Thanks again, very much!

Dave

PS., the LED on the Datalogger just goes to solid red.

PPS., Here is an edited version of the SimpleLog.bs2 that I have used and that I know works.· It has been edited to simply, in this case, write the same data line over & over again in the USB Drive, as I was planning on replacing that dataline with the variables.· Here's the code that worded on the BS2 (and I have adjusted it for the BS2sx Baudmode settings, etc):

' {$STAMP BS2}
' {$PBASIC 2.5}
TX         CON        0              'P0 assigned TO SEROUT
RX         CON        1              'P1 assigned TO SERIN
Baud       CON        188            'A baud rate of 4800
GP_WORD    VAR        Word           'A general-purpose word variable
idx        VAR        Byte
 
'***********[noparse][[/noparse]INITIALISE LOGGER & DETECT MEMORY STICK]*********
LOW TX
DEBUG HOME," Remove Memory and Press 1 ",CR
DO WHILE idx=0
GP_WORD=GP_WORD+1
DEBUGIN DEC1 idx
LOOP
DEBUG CLS
SEROUT TX, 84, [noparse][[/noparse]"IPA", CR]
PAUSE 200
SEROUT TX, 84, [noparse][[/noparse]"SCS", CR]
PAUSE 200
SEROUT TX,84, [noparse][[/noparse]$14,$20,$71,$02,$00,$0D]  'SET BAUD TO 4800
SERIN RX,baud,[noparse][[/noparse]WAIT(">")]
DEBUG "Initialized ",CR
PAUSE 200
GOSUB DETECT

'**********************************************************
Main:
GOSUB OPEN_WRITE_CLOSE
DEBUG "Pausing 5 seconds", CR
PAUSE 5000        ' Will return control to primary program
DEBUG "Resuming...",CR
GOTO main
 
'****************[noparse][[/noparse]DETECT MEMORY STICK]*********************
DETECT:
  DEBUG CLS, HOME, "It is safe to Insert or Remove Memory", CR,"(Remove and re-insert to go again)"
  SERIN RX, Baud,[noparse][[/noparse]WAIT ("DD")]
  DEBUG CLS, "Device Detected ...", CR
  PAUSE 3000
  DEBUG "Logger is ready.", CR
  PAUSE 500
RETURN
 
'****************[noparse][[/noparse]OPEN FOR WRITE - ORIGINAL]*****************
OPEN_WRITE_CLOSE:
  SEROUT TX, Baud,[noparse][[/noparse]$9, $20, "test.csv", CR]  ' Open file
  SERIN RX, Baud,[noparse][[/noparse]WAIT (">")]
  DEBUG "Writing...", CR
  SEROUT TX, Baud, [noparse][[/noparse]$8,$20,$0,$0,$0,$2C,CR,"08,06/18/2009,19:31:28,06/21/2009,09:00:00",CR,$0A]  ' 6th code is size
  SERIN RX, Baud,[noparse][[/noparse]WAIT (">")]
  SEROUT TX, Baud,[noparse][[/noparse]$0A, $20, "test.csv", CR] ' Close file
  SERIN RX, Baud,[noparse][[/noparse]WAIT (">")]
RETURN
 

Post Edited (xanatos) : 7/7/2009 12:30:43 AM GMT

Unsoundcode

Hi Dave ,
to use Simplelog.bs2 this initialization has to be followed , the flash drive should be removed from the datalogger then power applied to both the BS2 and the datalogger simultaneously. The debug screen displays a message once everything has boooted asking for keyboard input , pressing a key is when the datalogger is initialized.

It is not true that the flash drive be removed after each write , once the drive is inserted and verified you can leave it in there as long as you want writing and reading at will.

See if this gets you past the initialization

LOW DLTX
DEBUG HOME," Remove Memory and Press 1 ",CR
DO WHILE idx=0
DEBUGIN DEC1 idx
LOOP
DEBUG CLS
SEROUT DLTX, Baud, [noparse][[/noparse]"IPA", CR]
PAUSE 200
SEROUT DLTX, Baud, [noparse][[/noparse]"SCS", CR]
PAUSE 200
SEROUT DLTX,Baud, [noparse][[/noparse]$14,$20,$71,$02,$00,$0D]  'SET BAUD TO 4800
SERIN DLRX,DLbaud,[noparse][[/noparse]WAIT(">")]
DEBUG "Initialized ",CR

Jeff T.

Edit: BTW the WRF instruction must end with a CR ($0D) and you must write the exact number that you state in the WRF instruction. I mention this because your example states $2C (44·bytes) and I counted 43 so you are 2 bytes short (the final CR is not counted but needed)

Edit: I miscounted , but it still goes that the final character should be CR so you are in fact 1 byte short

Post Edited (Unsoundcode) : 7/7/2009 2:15:27 AM GMT

xanatos

Hi Jeff,

Yes, having the memory removed works, but the apparent requirement to do so will make this item unusable in the application.

Is there no way to have the drive initialize without the required removal and reinsertion?

Thanks very much for your efforts on my behalf,

Dave

Tracy Allen

Dave,
I always include means for PBASIC to cycle the power to the drive in all my applications. I know what you mean about having to operate unattended, reliably. The power cycling can be done via a single p-channel mosfet, or via a low-power, low dropout voltage regulator with on-off capabilities. Having this capability has three advantages.
1) The initialization becomes super easy. The drive starts in a known state, and PBASIC needs only wait for the prompt.
2) Nearly zero power operation is possible, especially if there will be periods of minutes between writes to the disk. The SUD and SUM commands can get the current drain down to as low as 2 mA, but some disks don't like to be suspended.
3) Power cycling admits an easy route to recover from errors. The drive is very touchy about receiving the correct number of characters and the correct command sequences. If there is a "glitch", or should I say, "when"! the drive has no reset mechanism except the mysterious long series of "E"s for resynchronization. Power cycling recovers operation immediately.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Tracy Allen
www.emesystems.com

xanatos

I like this idea. It would only add one more pin to my I/O count and a small subroutine to cycle the power. I was under the impression that the BS2 *AND* the Datalogger needed to both be cycled at the same time - which flat-out precluded that idea, as the BS2 CANNOT be power cycled in this application. But if I can cycle just the datalogger, and WAIT(">") and be ready to go - that could work well.

It's too late for this project, but I'll be considering it for the next one!

Thanks again, very very much. Your information is immensely helpful and appreciated.

Dave X

Unsoundcode

Hi Dave, Simplelog.bs2 and Two_Wire.doc is intended to give examples and descriptions of one method to communicate with the Datalogger. The program is flexible enough to be suited to most applications.

The drawbacks with the program as it stands are the initialization process and running at 4800 baud. The lower baud rate is more than compensated for by the seamless method of communication.

Initialization is neccessary because the default bootup paramaters require changing via a software command.

Looking at your requirements there are changes I would make that would significantly improve system operation as a whole, in particular the suggestion of using a pin as a cycle power switch.

This brings us back to Initialization , the bootup paramaters of the datalogger can be changed. So instead of the default 9600 , IPH and ECS bootup can be customized to 4800 , IPA and SCS.

Doing the above would eliminate the initialization code completely and cycling power would immediately give us a Datalogger that was ready to go ( as Tracy said "The drive starts in a known state" )

The utilities to customize the firmware are located at the Viniculum web site , don't go rushing into these changes it's simplicity make it dangerous and entering the wrong paramater can render the Datalogger useless.

Jeff T.

xanatos

Now THIS may open some possibilities. I'll research this - THANKS!

Dave

xanatos

Jeff T: I believe you've made my week! Thank you, this appears to be just what I'm looking for. I've got some FETs that would be very happy to do the power cycling work for me, so all I'll need to do is figure the best way to tell the system when a drive is inserted to cycle the datalogger's power, and I should be able to start writing... if I am doing it all correctly! I'm going to go order 2 dataloggers this instant!

---Update:· I have downloaded from Vinculum the Vinculum Firmware Customiser

Is this the proper item you have referred to?

Thanks again!

Dave X

Post Edited (xanatos) : 7/11/2009 10:57:40 PM GMT

Unsoundcode

Hi Dave , if you configure a logger at 4800 , SCS ,IPA (optional dependant on data ) and finally remove the option to display the version number on bootup the logger will recover from power cycling real quick and be ready in 2 or 3 seconds.

Do not remove the option that checks for firmware upgrades or your stuck .

With the above configuration every succesful command will be answered by the logger with a prompt ">" , that includes cycling power or inserting a thumbdrive so detection of either of those events is a simple

SERIN rx,baud,[noparse][[/noparse]WAIT(">")]

DEBUG "Logger detected"

The docs say to send the two echo commands E and e to synch or you might just send a carriage return which reponds with the prompt ">" disk present or ND no disk present ·but you can probably run quite happily without having to do that.

Jeff T.

EDIT: yes that is the utility , you also need the latest VDAP firmware version from the Viniculum site , it's version 3.68·. You will need the Reflash version which has the FTD extension not the ROM version . The final stage of customization requires you to save and rename the upgrade , which I forget right now what it should be. If you can't find the rename info let me know I have it on a computer somewhere.

Post Edited (Unsoundcode) : 7/11/2009 11:17:48 PM GMT

xanatos

Thanks very much! The only question I have left is - and I didn't see it in the firmware changer's user guide PDF unless I just missed it - is how to get the firmware change into the datalogger. Do I just put it on a USB drive and stick it in, or...??? And if I understand one part correctly, when I do the firmware upgrade, it must be configured for UART mode, not SPI. Am I correct on these counts?

Thank you greatly!!!

Dave X

PS - I'm stepping away for about an hour - but I'll be back on later this evening... I've got a lot to do!

phil kenny

Dave,

You're right. Just put the new ROM code (with the right name) on the USB module
and plug it into the Data Logger. When the Data Logger first reads the USB device
it looks for a ROM update file. If present, it upgrades the firmware.

As far as UART vs SPI mode, I don't know if it makes any difference, but I've
always used the SPI mode.

phil

Unsoundcode

Dave , there are three ways to upgrade the firmware , the only one open to us with the Datalogger is a file on the thumbdrive. Thats why it's not advisable to remove that option with the customization tool.

Every time the logger has a drive inserted it searches the drive for a firmware upgrade.

I have a utility I use to monitor the upgrade but that is not neccessary . If you run the Simplelog example (with the flash stick removed ) and then when it says it is safe to insert the memory ,· it will automatically upgrade when you put the memory stick in. After the upgrade the logger will reboot (takes less than a minute)·,·at that point I would power down and set the Stamp up to try the logger with code that has no initialization.

Heres the sequence
Customize Firmware
Rename and save
Copy new file to a known good memory stick
Start a modified Simplelog (remove all the subroutines and references except initialize and detect)
Insert memory stick
Wait for any disk activity to stop
Power down and try with non initializing code

Jeff T.

xanatos

I'm back... I have downloaded the VDAP 3.68 now as well. I have thre dataloggers on order (!) just in case I mess one up! But I expect that thanks to all of your extraordinatily useful and helpful information I will encounter a success I didn't think was possible with these items! Thanks VERY much for this. I seem to be one of those experiementers that always needs to make the hardware work in some non-standard completely custom configuration!

Now I just need to patiently wait until the delivery arrives! I'll certainly let you know how it goes.

Thanks again. I hope I can return the favor somehow.

Dave X

Tracy Allen

The file on the USB drive has to be named "ftrfb.ftd", and placed in the root directory. A caveat; I found to my surprise that the process will also happily downgrade your firmware; it just looks for "different" not necessarily "new", so watch out for old versions hanging around on old USB drives.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Tracy Allen
www.emesystems.com

Chris Savage

Tracy, nice information to know. Thanks!

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Chris Savage
Parallax Engineering

xanatos

Thanks also for the naming requirement - the firmware customizer only seemed to require that I append a three character code to the end as a personal identifier. This place is so awesome!

Dave

xanatos

One more question on this:· I've got some p channel enhancement mode FETs - DMP2012SN - and these are what I'm considering using to do the power cycling.· Do you have another part number that you like to use that I can look up and compare datasheets?· I've attached the datasheet for the DMP2012SN in case you want to take a look at that one.· Nice SMT package.

Thanks again for everything!

Dave