Best solution for a Propeller Database: FSRW->SD, SettingsOBJ->EEPROM, ROMEngin

WBA Consulting

I am working on a project for a component baking oven that will require a simple database to store data dynamically for use by itself (not for export to a PC for Access or other database software). The data to be stored is somewhat simple, but I am still uncertain as to which route I should take to properly implement a SPIN database. My setup already has SD, so it can use SD, the upper portion of of the 64K boot EEPROM, or a secondary EEPROM. The data to be stored consists of the following chunks:

Tag# (2 digit number, 1-140))
MSL# (single digit, 1-6)
Initials (two alphabetical characters, IE: "AW")
Date (in four digit format, IE: "0427")
Time (in four digit 24 hour format, IE: "1700" (5pm))
Item# (alphanumeric string up to 24 characters, IE: "EIN-044-1944-001")

The user interface will prompt for a tag# to start a record which will be entered by scanning a barcoded tag with a Symbol LS1220 serial scanner. The operator will then be prompted to enter the MSL#, their initials, and the item#. The date and time will be added by the Prop using a RTC. This group of data needs to be stored in a manner that can be pulled back using the tag# (and if possible the item#) by the propeller to display status information as requested on a screen (analog TV or Ray's 4.3" touchscreen).

The data will not be used with a PC, only on the propeller setup, so I don't need to make the data PC friendly however it is stored.

Anyhow, I have been digging through EEPROM options (ROMEngine Object and Settings Object) but as I dig, I am concerned that I may be heading the wrong direction. I already use a few SD objects, but the data handling seems a little harder than the EEPROM objects (I am still unsuccessful at reading logged data properly back from SD).
It appears that I could use the Settings object easily, but I think I would have to make a key for all 140 tags in the CON section to handle them. Seems like a mess of work for something simple.
The ROMEngine seems easy to use by utilizing the Tag# as an EEPROM offset for the location in EEPROM and just store the data in chunks at that offset. (am I understanding those two objects correctly?)

Any suggestions on where to look deeper to make my SPIN Database?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Andrew Williams
WBA Consulting
WBA-TH1M Sensirion SHT11 Module
Special Olympics Polar Bear Plunge, Mar 20, 2010

Post Edited (WBA Consulting) : 4/28/2010 1:35:05 PM GMT

bill190

Let's see...

I recall seeing something about a FAT file system object for SD cards? Perhaps this would be at the object exchange...

http://obex.parallax.com

Then so far as storing the data, there is a thing called a "flat file database". You can just list your data in a text file with a text editor like "notebook". More on that here...

http://en.wikipedia.org/wiki/Flat_file_database

Then if you have "access" to Microsoft Access (db program), with this you can import or export data... OK.... So what you do is create a database with the fields you want above (with Microsoft Access or something similar like Open Office perhaps?), then fill in maybe 10 records of the information you want to store (as you said above)...

Then simply export·that data·using the different flat file formats listed in Access. Such as comma delimited, rich text, etc., then see what·that data looks like. See which flat file format you would prefer to use.

That would be data stored in a format a regular database could import and use.

Then you could use one of these same simple formats on your SD card. And/or store this in a regular text file using the above FAT file system object? (I have not looked at that and have no idea what it does???).

Anyway if you could store that data on the SD card in such a format, then in theory you should be able to stick that sd card in a PC, import the data with a regular database like Microsoft Access, then use that data.

So then the data could be read / shared. And then the only thing to do would be to write code so you could write/read this data with the Propeller.

But a flat file database has its data stored in the most simple format possible. It is just text! Yet also a "database".



Post Edited (bill190) : 4/28/2010 8:11:34 AM GMT

bill190

PS... You can examine a text file on a PC with a "hex editor". And these text files have a certain format of just text and an end of line character and end of file character.

The hex editors I have used show hex values on most of the screen and ASCII characters in the right column (same data).
·

WBA Consulting

Thanks for the response, but I guess I should have mentioned that this database will probably never be used on a PC, only the Propeller setup. The structure I have in my first post would be exactly what you mention as a flat file so if I did need to read it with a PC, it could be done how you mention. However, only the system itself needs to be able to pull the data back for use. I will update my original post so it doesn't appear that I mean to use it with a PC database program like Access. thanks!

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Andrew Williams
WBA Consulting
WBA-TH1M Sensirion SHT11 Module
Special Olympics Polar Bear Plunge, Mar 20, 2010

Mike Green

Why not just store the data in the 2nd 32K of the EEPROM? You'd have 140 x 32 byte entries. The MSL# will fit in a byte and the date and time will each fit in a word. You wouldn't need the Tag# since the address would be computed from the Tag#. You can store the Item# in a fixed 24+a zero byte field padded with zeroes for a total of 32 bytes per entry.

Use the Basic_I2C_Driver object to read and write this. If you keep the entries aligned on a 32 byte address boundary, you can use paged writes to write the whole thing in one 10ms write cycle when you need to update it.

For testing purposes, you might write a program that would dump the EEPROM database to an SD card file for analysis on a PC. This program would always be loaded into RAM and run when needed, never to EEPROM. Similarly, you could have an EEPROM database initialization program that would be only loaded and run from RAM when needed.

Post Edited (Mike Green) : 4/28/2010 2:28:10 PM GMT

KPR

This reminds me of doing fixed length data records on my Apple ][noparse][[/noparse] ... hmmm project anyone??

Apple DOS 3.3 Text File Commands

OPEN name Opens a text file for reading.
CLOSE name Closes a text file.
READ name Sets the file 'name' to read mode.
WRITE name Sets the file 'name' to write mode.
APPEND name Sets write mode for 'name' at the end of the current file.
POSITION name,x Moves the record pointer for 'name' to the position specified by x.
EXEC name Reads text from file 'name' entering it as though it were typed from the keyboard.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
I always have someone watching my back.

bill190

Even if you're never going to use the database on a PC, it is a good idea to use the same format as a PC database would use. (pick one) These are the way they are for a reason, and that is so programs can be written to write, retrieve, and sort·the data.

There are all sorts of problems which can occur when you try to read that stored data and display *just* the data. So if you use an existing format, you would know that it would work in all situations. (You can avoid potential problems.)

There are always exceptions/changes and human input error. (Be sure to only allow·valid characters to be input - you wouldn't want someone to be able to accidentally enter a comma in a comma delimited format database!)

And people are people (management). Frequently a database will be designed and "set in stone", then the customer/management will come along and say "Can we add an additional field to the database so we can do this and that?"

If it is designed to allow for future changes, then this would be much easier to do.


Post Edited (bill190) : 4/28/2010 3:13:15 PM GMT

Ravenkallen

I would use the i2c EEPROM. It is easy to use, cheap and pretty easy to expand(if you need more room). That is what i use for all my logging projects. I got a plant monitor that takes in soil moisture levels along with air temp and it saves the data to the EEPROM every hour or so. I don't have any problems with it.

WBA Consulting

Sounds like EEPROM is getting the most votes as the storage method.....

@Mike: Are you looking at my notes? You describe exactly what I derived after spending about an hour between the Propeller manual, PEKit book, and the EEPROM datalogging PE Kit lab. My notes are in pencil on graph paper so I can't scan them in to share, but attached is the storage template I was brainstorming.
I don't really understand page writes (I believe that means I will have to use some type of buffering to hold the different pieces of the 32 byte page?) Your idea of dumping to SD sounds good and I think I may have the end of my main loop check for the presence of an SD card and if it exists, prompt to dump the EEPROM data onto it.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Andrew Williams
WBA Consulting
WBA-TH1M Sensirion SHT11 Module
Special Olympics Polar Bear Plunge, Mar 20, 2010

Dave Hein

I'm developing an EEProm file system if your interested.· It allows for multiple read and write files, and will·have subdirectories as well.· However, it seems like your application could just read and write the EEProm directly.· You mention a 32-byte page size, but I believe the page size is 64 bytes for the 32K and 64K EEProms.

Dave