Data logging and downloading suggestions wanted

Archiver

[noparse][[/noparse]Sorry if this seems long. I think an explanation of what I'm trying to do
is in order... :]

Okay! I'm almost done with my project, but I need some input from you more
experienced folks.

My application is a temperature logger. I'm using a BS2, an LTC1298 ADC with
a couple thermistors, matching the measured voltages with a lookup table to
get my actual readings. I have a DS1302 RTC for timekeeping purposes. I have
a 24LC16B 16kbit EEPROM for storage. I also have a 74F675A serial-in
parallel-out shift register to talk to a Maxim ICM7218A 8 digit 7 segment
LED display to show a "bank clock" type display.

All this works. Currently, I set the time on initial startup using DEBUGIN,
but I will have some buttons to use to set the time. I will have a serial
port on the device, which I would like to be able to connect and disconnect
at will, while the device is on, to download the data to the PC. It will be
powered from a regular power supply, with a battery backup for the RTC.

Now, here are my questions:

1) I would like to be able to recover from a loss of power. I want to be
able to tell whether the RTC's time is set. I suppose I could use a portion
of the RTC's RAM to check this - but since the contents are undefined on
power up, this could (rarely) give a false indication.

2) Since I'm only using a 2kbyte EEPROM, space is limited. Data will be
logged every 30 minutes. Upon initial startup, I want to write $FF, followed
by the date/time, followed by the sampled data. Each new-day would write $FF
and date/time, followed by more samples ($FF should never appear for any
date/time/sample). If the device is powered off, upon startup, simply act
like it's a new day and continue (assuming clock is set - if not, then after
setting the clock).

3) I would like to connect a PC at any time to download the stored
information, clear the EEPROM, and start fresh. What is the best way to
signal the stamp that I want to download? SHould the program send a signal,
or should the process be started with a button on the device?

4) How do I keep track of "current EEPROM location" for my sequential
writes? I though of using the first two bytes of the EEPROM for this, but
they would be written to every time a sample is written. At 30
minutes/sample, this shouldn't be a problem, but is there another solution?
I thought of the RTC RAM, but if the battery fails (or isn't present), I
lose this. If the EEPROM is halfway full, power is lost, and the clock must
be reset, I still want to continue where I left off.

Thanks for any and all input!
Steve

Archiver

The 1302 takes a 3-volt 'button cell' for a
battery backup. It keeps time on battery
backup, also. I'd highly recommend you use
it's battery backup feature to keep time.
Once you have that, you can write to its
RAM also (both for a power-up value, and to
insure the battery/time are still good).

Also, you'll want a 'pull-down' on the
BS2 pin 3. Otherwise, when you plug in
the PC, you have a very good chance of
'pinging' the pin 3 'ATN' or 'RESET'
line high.

I'd be tempted to have a:

DataByte = 0
SERIN P16, I9600, 200, TimeOut, [noparse][[/noparse]DataByte]
Timeout:
IF Databyte <> 0 THEN ListenToPC
...
ListenToPC:
SEROUT P16, I9600, [noparse][[/noparse]"BS2>"]
...
so the PC can 'break in' to trigger some
download code. The '200' will wait for
200 mSec for the PC to break in, then
continue on.

Now, space used in EEPROM.
I'd be tempted to use the RTC to store
the current write location. I'd also
tend to 'clear' the EEPROM to all
zero's, once data is downloaded, so
if disaster DID strike, the BS2 could
still scan through the EEPROM, and know
end of data was when it hit a bunch of
zeros.


--- In basicstamps@yahoogroups.com, "Steve Ziuchkovski"
<zman97211@y...> wrote:
> [noparse][[/noparse]Sorry if this seems long. I think an explanation of what I'm
trying to do
> is in order... :]
>
> Okay! I'm almost done with my project, but I need some input from
you more
> experienced folks.
>
> My application is a temperature logger. I'm using a BS2, an LTC1298
ADC with
> a couple thermistors, matching the measured voltages with a lookup
table to
> get my actual readings. I have a DS1302 RTC for timekeeping
purposes. I have
> a 24LC16B 16kbit EEPROM for storage. I also have a 74F675A serial-in
> parallel-out shift register to talk to a Maxim ICM7218A 8 digit 7
segment
> LED display to show a "bank clock" type display.
>
> All this works. Currently, I set the time on initial startup using
DEBUGIN,
> but I will have some buttons to use to set the time. I will have a
serial
> port on the device, which I would like to be able to connect and
disconnect
> at will, while the device is on, to download the data to the PC. It
will be
> powered from a regular power supply, with a battery backup for the
RTC.
>
> Now, here are my questions:
>
> 1) I would like to be able to recover from a loss of power. I want
to be
> able to tell whether the RTC's time is set. I suppose I could use a
portion
> of the RTC's RAM to check this - but since the contents are
undefined on
> power up, this could (rarely) give a false indication.
>
> 2) Since I'm only using a 2kbyte EEPROM, space is limited. Data
will be
> logged every 30 minutes. Upon initial startup, I want to write $FF,
followed
> by the date/time, followed by the sampled data. Each new-day would
write $FF
> and date/time, followed by more samples ($FF should never appear
for any
> date/time/sample). If the device is powered off, upon startup,
simply act
> like it's a new day and continue (assuming clock is set - if not,
then after
> setting the clock).
>
> 3) I would like to connect a PC at any time to download the stored
> information, clear the EEPROM, and start fresh. What is the best
way to
> signal the stamp that I want to download? SHould the program send a
signal,
> or should the process be started with a button on the device?
>
> 4) How do I keep track of "current EEPROM location" for my
sequential
> writes? I though of using the first two bytes of the EEPROM for
this, but
> they would be written to every time a sample is written. At 30
> minutes/sample, this shouldn't be a problem, but is there another
solution?
> I thought of the RTC RAM, but if the battery fails (or isn't
present), I
> lose this. If the EEPROM is halfway full, power is lost, and the
clock must
> be reset, I still want to continue where I left off.
>
> Thanks for any and all input!
> Steve

Archiver

well, at 100,000 write life, it'd take 50,000 hours to wear out your BS2
eeprom... Thats a mere 5.7 years of writes every 30 seconds. If thats a
worry, keep a second counter (in eeprom as well) and every X writes, change
to a different BS2 eeprom location for both the address and the write
counters. If you assume 4 bytes for the two values and set aside 512 bytes
for it, that gives you an eeprom life something like 17,000 years.

You can always tell when the power has completely failed, the RTC time goes
back to the first second of the first day of 2000 (or was it 2001? cant
remember). If its ever that date, you've lost all power.

---

Original Message

---

From: "Steve Ziuchkovski" <zman97211@y...>
To: <basicstamps@yahoogroups.com>
Sent: Saturday, July 12, 2003 8:58 AM
Subject: [noparse][[/noparse]basicstamps] Data logging and downloading suggestions wanted


> [noparse][[/noparse]Sorry if this seems long. I think an explanation of what I'm trying to do
> is in order... :]
>
> Okay! I'm almost done with my project, but I need some input from you more
> experienced folks.
>
> My application is a temperature logger. I'm using a BS2, an LTC1298 ADC
with
> a couple thermistors, matching the measured voltages with a lookup table
to
> get my actual readings. I have a DS1302 RTC for timekeeping purposes. I
have
> a 24LC16B 16kbit EEPROM for storage. I also have a 74F675A serial-in
> parallel-out shift register to talk to a Maxim ICM7218A 8 digit 7 segment
> LED display to show a "bank clock" type display.
>
> All this works. Currently, I set the time on initial startup using
DEBUGIN,
> but I will have some buttons to use to set the time. I will have a serial
> port on the device, which I would like to be able to connect and
disconnect
> at will, while the device is on, to download the data to the PC. It will
be
> powered from a regular power supply, with a battery backup for the RTC.
>
> Now, here are my questions:
>
> 1) I would like to be able to recover from a loss of power. I want to be
> able to tell whether the RTC's time is set. I suppose I could use a
portion
> of the RTC's RAM to check this - but since the contents are undefined on
> power up, this could (rarely) give a false indication.
>
> 2) Since I'm only using a 2kbyte EEPROM, space is limited. Data will be
> logged every 30 minutes. Upon initial startup, I want to write $FF,
followed
> by the date/time, followed by the sampled data. Each new-day would write
$FF
> and date/time, followed by more samples ($FF should never appear for any
> date/time/sample). If the device is powered off, upon startup, simply act
> like it's a new day and continue (assuming clock is set - if not, then
after
> setting the clock).
>
> 3) I would like to connect a PC at any time to download the stored
> information, clear the EEPROM, and start fresh. What is the best way to
> signal the stamp that I want to download? SHould the program send a
signal,
> or should the process be started with a button on the device?
>
> 4) How do I keep track of "current EEPROM location" for my sequential
> writes? I though of using the first two bytes of the EEPROM for this, but
> they would be written to every time a sample is written. At 30
> minutes/sample, this shouldn't be a problem, but is there another
solution?
> I thought of the RTC RAM, but if the battery fails (or isn't present), I
> lose this. If the EEPROM is halfway full, power is lost, and the clock
must
> be reset, I still want to continue where I left off.
>
> Thanks for any and all input!
> Steve
>
>
>
> To UNSUBSCRIBE, just send mail to:
> basicstamps-unsubscribe@yahoogroups.com
> from the same email address that you subscribed. Text in the Subject and
Body of the message will be ignored.
>
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>

Archiver

>I suppose I could use a portion
>of the RTC's RAM to check this - but since the contents are undefined on
>power up, this could (rarely) give a false indication.

Are you sure they come up undefined? Every clock chip I have used
comes up with some definite initial state in the clock registers.


>
>3) I would like to connect a PC at any time to download the stored
>information, clear the EEPROM, and start fresh. What is the best way to
>signal the stamp that I want to download? SHould the program send a signal,
>or should the process be started with a button on the device?

That's up to your UI planning! I think in general it will be best to
keep all the interaction on the PC end. So the PC will present a
menu to the user, with an option to offload the data. Then the PC
sends the command to the logger, and finishes up with an option to
either erase the old data or to continue to add data. There might be
reasons for doing it some other way.

>
>4) How do I keep track of "current EEPROM location" for my sequential
>writes? I though of using the first two bytes of the EEPROM for this, but
>they would be written to every time a sample is written. At 30
>minutes/sample, this shouldn't be a problem, but is there another solution?
>I thought of the RTC RAM, but if the battery fails (or isn't present), I
>lose this. If the EEPROM is halfway full, power is lost, and the clock must
>be reset, I still want to continue where I left off.

The RTC memory is by far the best option in my opinion. If it is
backed with a lithium cell, it can run for 10 or more years. I use
the DS1307.

There is the option of completely erasing the eeprom to some state,
such as $ff in every location. This batch erase can usually be done
quite rapidly using block erase commands. Then, as you use the
memory, those locations are changed to some non-$ff value. Every
time the program starts, it scans the memory to find the first $ff,
and that is the first free location. With fixed field data, you can
have it scan only the marker byte at the start of each record, so
that the data itself can hold $ff values. The scan itself can use a
binary search algorithm like the one I provided for your table
lookup. Using that, you can find the first free byte in a 64kbyte
memory in 16 or fewer iterations.

Another way to do the pointer in eeprom, is to change the pointer
location every time you start a new run. That way, one location
won't become so worn out. There has to be a pointer to a circular
pointer buffer, a bit of a PITA.





>
>Thanks for any and all input!
>Steve
>
>
>
>To UNSUBSCRIBE, just send mail to:
> basicstamps-unsubscribe@yahoogroups.com
>from the same email address that you subscribed. Text in the
>Subject and Body of the message will be ignored.
>
>
>Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

Archiver

Thanks for all the input Tracy.

I looked again at the data sheet for the 1302, and although I don't see
anything that says a definate value upon power up, I did see a note that the
write protect bit of one of the registers is set upon power up, and needs to
be cleared before setting the time. I could simply read that register and
make a decision based upon the results.

I also like your idea of initially filling the EEPROM with $FF - Since this
never occurs in the data, and since I'll only use one byte of $ff to "start
a new day", I can simply search for two consecutive bytes if the RTC is
lost, otherwise use the value stored in the RTC's RAM.

Seems like I have a solution. Thanks again for your and others help [noparse]:)[/noparse]

Steve

---

Original Message

---

From: "Tracy Allen" <tracy@e...>
To: <basicstamps@yahoogroups.com>
Sent: Saturday, July 12, 2003 9:39 PM
Subject: Re: [noparse][[/noparse]basicstamps] Data logging and downloading suggestions wanted


> >I suppose I could use a portion
> >of the RTC's RAM to check this - but since the contents are undefined on
> >power up, this could (rarely) give a false indication.
>
> Are you sure they come up undefined? Every clock chip I have used
> comes up with some definite initial state in the clock registers.
>
>
> >
> >3) I would like to connect a PC at any time to download the stored
> >information, clear the EEPROM, and start fresh. What is the best way to
> >signal the stamp that I want to download? SHould the program send a
signal,
> >or should the process be started with a button on the device?
>
> That's up to your UI planning! I think in general it will be best to
> keep all the interaction on the PC end. So the PC will present a
> menu to the user, with an option to offload the data. Then the PC
> sends the command to the logger, and finishes up with an option to
> either erase the old data or to continue to add data. There might be
> reasons for doing it some other way.
>
> >
> >4) How do I keep track of "current EEPROM location" for my sequential
> >writes? I though of using the first two bytes of the EEPROM for this, but
> >they would be written to every time a sample is written. At 30
> >minutes/sample, this shouldn't be a problem, but is there another
solution?
> >I thought of the RTC RAM, but if the battery fails (or isn't present), I
> >lose this. If the EEPROM is halfway full, power is lost, and the clock
must
> >be reset, I still want to continue where I left off.
>
> The RTC memory is by far the best option in my opinion. If it is
> backed with a lithium cell, it can run for 10 or more years. I use
> the DS1307.
>
> There is the option of completely erasing the eeprom to some state,
> such as $ff in every location. This batch erase can usually be done
> quite rapidly using block erase commands. Then, as you use the
> memory, those locations are changed to some non-$ff value. Every
> time the program starts, it scans the memory to find the first $ff,
> and that is the first free location. With fixed field data, you can
> have it scan only the marker byte at the start of each record, so
> that the data itself can hold $ff values. The scan itself can use a
> binary search algorithm like the one I provided for your table
> lookup. Using that, you can find the first free byte in a 64kbyte
> memory in 16 or fewer iterations.
>
> Another way to do the pointer in eeprom, is to change the pointer
> location every time you start a new run. That way, one location
> won't become so worn out. There has to be a pointer to a circular
> pointer buffer, a bit of a PITA.
>
>
>
>
>
> >
> >Thanks for any and all input!
> >Steve
> >
> >
> >
> >To UNSUBSCRIBE, just send mail to:
> > basicstamps-unsubscribe@yahoogroups.com
> >from the same email address that you subscribed. Text in the
> >Subject and Body of the message will be ignored.
> >
> >
> >Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>
> To UNSUBSCRIBE, just send mail to:
> basicstamps-unsubscribe@yahoogroups.com
> from the same email address that you subscribed. Text in the Subject and
Body of the message will be ignored.
>
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>