Large dynamic array needs
HomeSchoolDad
Posts: 32
Greetings,
I am building an application that requires far more memory than my BS2pe can handle.
The array will have to hold at least 1,000 entries of at least 128 bytes each (mostly text).· I remember seeing a sample program someplace that had a 1MB memory chip (NVRAM?·EEPROM?· FRAM?·- I can't remember) and after days of reading and searching I can't seem to fnd it here.· The chip was able to withstand billions of writes as I remember and would have been perfect as it would be easy to write out a record per page on initial load, update each record as required and then offload the whole table.
Does anyone have any idea of (1) what chip that was, (2) what chip could be substituted to work most easily with the BS2p series, (3) how to hook it up with the right voltages so I don't fry the chip, and (4) where I can find some example code?
Yes and fries with that.· [noparse]:)[/noparse]· I realize that I am asking for the world here, but any help would be MOST appeciated!
Thanks guys!
····· Jeff
·
I am building an application that requires far more memory than my BS2pe can handle.
The array will have to hold at least 1,000 entries of at least 128 bytes each (mostly text).· I remember seeing a sample program someplace that had a 1MB memory chip (NVRAM?·EEPROM?· FRAM?·- I can't remember) and after days of reading and searching I can't seem to fnd it here.· The chip was able to withstand billions of writes as I remember and would have been perfect as it would be easy to write out a record per page on initial load, update each record as required and then offload the whole table.
Does anyone have any idea of (1) what chip that was, (2) what chip could be substituted to work most easily with the BS2p series, (3) how to hook it up with the right voltages so I don't fry the chip, and (4) where I can find some example code?
Yes and fries with that.· [noparse]:)[/noparse]· I realize that I am asking for the world here, but any help would be MOST appeciated!
Thanks guys!
····· Jeff
·
Comments
You were quite close in what you remembered. Check out the FRAM memory by Ramtron here: http://www.ramtron.com/ . Use one of the serial access units of an appropriate size, with the SHIFTIN/SHIFTOUT commands, and you should be good to go. Nearly unlimited endurance NVRAM, and no wait state on writes!
Regards,
Bruce Bates
You are probably thinking of Ramtron's FRAM product. Check their website at : http://www.ramtron.com/doc/Products/serial.asp for datasheets on the many serial memory offerings that they have. The FRAM products are drop-in compatible (pin-wise and electrically) with conventional eeprom memory. Here is a link to a Paralax document that can help get you started with accessing an SPI serial eeprom (it isn't the FRAM product, but should work)[noparse]:http:[/noparse]//www.parallax.com/dl/docs/prod/oem/8keeprom.pdf
Good luck!
peter
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Peter C. Charles
Director, Research and Technology
CyberBiota, Incorporated
Peter.charles@cyberbiota.com
http://www.cyberbiota.com
I might be able to·crunch down·my needs to 1000 entries of 128 bytes if I eat my spinach and take lots of brain boosting vitamins, but that still puts me at 128K bytes so I would still need a bank of 4 of their 256K bit chips, right?
Which one would you guys recommend?
·
Ramtron does sell a 1M Fram, but I am not familiar with the "JEDEC 128Kx8 SRAM pinout" that they claim the part conforms to. Check out :
http://www.ramtron.com/doc/Products/Products_detail.asp?ID=66&FamID=3
I suspect that the 1M chip is not a serial part, and is intended more for PC bus kind of applications. I have used the 256K parts, and found them quite easy to implement. Your calculations look correct (but it has been a long day already, and my math skills are shot...), so I would suspect you will need 4 of them. It is not supposed to be difficult to daisy chain parts on an SPI bus, but I have never done it, so you will need help from one of the Stamp gurus! Please post your results when you get it working, as I am sure this is an issue I will be faced with eventually.
Good luck!
peter
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Peter C. Charles
Director, Research and Technology
CyberBiota, Incorporated
Peter.charles@cyberbiota.com
http://www.cyberbiota.com
You know I saw that chip and figured it was parallel and not a good choice.· I do have lots of extra pins since I have a Matrix Orbital display that does the 4x4 keyboard matrix for me all through the same I2C interface I use to talk to the LCD, so it is not out of the question I guess - if I could figure out how to make it work.
Dagnab it.· I just went back to look at it and it is 1M bit so it looks like I have to noodle a way to get my records to fit in 128 bytes.· The documentation says this:
So.... 128 x 1024 = 131,072 which makes this a 128K byte chip.· That appears to be a 16K byte row that is broken into·8·blocks of·2k bytes.· Each of those 2K rows are divided into 8 columns·of 256·bytes.· So (check me on this) I have 512 pages of 256 bytes on this chip I think.· If I can reduce my record size to 128 bytes I have my 1024 records (in a buffer size of 2 records per buffer - or page) all on one device!
Now I have two problems left.· It does not look lik it will plug into the prototype board and I have no idea how to connect up and program a parallel FRAM memory chip.
·· Jeff
·
I just skimmed through the datasheet for the FM20L08T (1Mb FRAM). I think your two biggest problems with this part are liable to be that 1) it is a surface mount part, and thus will need to be soldered down onto a PCB and 2) It is a parallel device and requires 17 lines for addressing, 8 lines for the data bus and 3 control lines. The first problem is not insurmountable- soldering down fine-pitch SMT parts is tricky, but entirely doable, and there are adapter boards available that will allow you to fit the mounted SMT part to a standard socket or proto board. The second issue is more problematic. I don't think you have 26 free pins on the BSIIpe! This can also be worked around with shift registers and some programming finesse, but I wouldn't attemp it. It is easier to plug 4 256Kb FM24256 FRAMS into your protoboard and connect them all up on the same SPI bus, which will set you back 2 lines for the clock and data IO, plus one line per chip for the chip select line. For 4 chips, thats 6 pins total. I know there are many people on this list who can help with the code for writing to multiple memory chips on an SPI bus. I would go that route.
Best regards,
peter
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Peter C. Charles
Director, Research and Technology
CyberBiota, Incorporated
Peter.charles@cyberbiota.com
http://www.cyberbiota.com
It may be time to go back to the definition of the problem for a moment. What is the content of the data that's being stored? Perhaps some data reduction is possible, to reduce the amount of EEPROM storage required?
So too there are devices other than simple EEPROMs for storing data externally. What about using CF or SD memory cards, as used in digital cameras, and devices like that.
Regards,
Bruce Bates
Post Edited (Bruce Bates) : 8/16/2005 3:06:32 AM GMT
Peter:· I was in the other room and thinking about the chip not being made in a DIP package when it hit me that all those pins had to be for something, and I was not going to have enough pins to cover them all.· You hit that one perfectly on the head.· I guess I could buy a BS2P40 and make it do, but I think the multi-chip bank makes more sense since I want to extend the battery power as far as possible and the BS2pe does that well.· By the way, I looked at the alternatives for attaching the chip to the breadboard and there were two, the one you mentioned (I hate to solder) and an adapter that sells for between $95 and $249 (for just the adapter!).· The other·memory chips don't come in·DIP packages either, but they are·looking better and better.· I have no idea what they charge for the chips (the web site insists on letting someone get back with you in their own sweet time).
Bruce: I considered that and found an add on·module for about $100 that will do an SD card.· I have been looking at the total cost of the system and when I get this done I am expecting to add a wireless adapter to it and they seem to be about $100.· The LCD was about $100 and the stamp and board was about $100.· Am I seeing a pattern here?· Anyway, I REALLY like the idea of using an SD card for this - for lots of reasons - but the cost of this is getting quite high - but then, I don't know what the cost of those 4 chips are yet, it may be comperable.· I always worry when they won't tell you the price on a website.· Do you have any idea where I can go SD without spending $100?· Of course, I hope that I get enough orders that I can get a discount someplace, but I have to build the prototypes out of my own pocket and this unit is getting to $400 kinda quick.
Thanks guys you are being very helpful, and I truly appreciate your help.
···· Jeff
About the LAST things I'd be trying to do at this juncture, is to try to price out a product based on a prototype that isn't even completely designed, much less operational yet. That phase needs to be a ways down the road. Some prototypes cost 10X the eventual selling price of the final product. That's just the nature of prototyping.
Just as a matter of information, the prices you're indicating for the LCD and "Stamp and board" are probably 2 times reality. Just by way of example, take a look at the BS-2 OEM modules. They are designed and priced for quantity purchase. You can probably purchase ordinary LCD's that are GOLD PLATED for less than $100.00 in quantity, unless there is something special or unique about them, that you're not mentioning.
Here is a $60.00 (one off) solution to interfacing with an SD card:
http://www.ghielectronics.com/ALFAT-SD.htm
As you will note, in production quantities, the price drops to $45.00 (25-->100 units), which makes it an attractive alternative, especially if you're not prepared to solder SMT chips. Additionally there is great portabilty advantage with the SD type solution, which is not as easily seen with EEPROM solutions, as well as the possibility of add-on sales if you're providing the SD cards.
If I can't get the price of an item as seen on a web site in one email, I start looking elsewhere, as the provider isn't a serious business person or company. Having owned my own businesses for over 30 years I never denied a customer a price, although the FINAL price might well be negotiable, depending on many factors. Some web sites or companies want to identify YOU (generic) as a "real" customer before opening their price list to you. This keeps mere browsers, price-only shoppers, and the competiton at bay a bit.
Your answer above doesn't address the thought of data reduction, so I guess that's of no interest to you. Reducing the amount of data, is probably going to be the single biggest key to reducing the final cost, whether you go with SD, EEPROM, FRAM, or anything else. The larger the data "container" the more you're going to pay for it. You've given no indication that speed was of any major concern. If that is indeed the case there is a trade-off that can be realized between throughput (overall processing speed) and data compression or reduction (storage size). Simply put, if you can afford the time, you can reduce the size external storage requirements.
Regards,
Bruce Bates
Here is a URL for stamp application of dataflash: <http://emesys.com/BS2flash.htm>
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Tracy Allen
www.emesystems.com
I have actually reduced the size of the data by almost half without resorting to compression by doing an indexed match-back when I syncronize the unit to a computer at start-up and shut-down.· That·need was the attraction to the SD card.· Since the data will need to be indexed on the handheld·and looked up dynamically during use, I am loath to use even space compresion (even though I know that I will be storing lots of spaces).· I have considered creating a page index that contains a key and a page number, doing a binary search for the key·and then unpacking the page where the record exists to find the data I need·-·I could put any inserted or modified·records at the end and check for those each table·lookup.· As the application would by nature have inserted records from time to time, I need to be able to add them to the table dynamically (or do a reload and keep the modified data separate from the base record).· As you surmise, I don't have a huge need for speed - but I do have to insure that the system will run all day on a 9v battery.
Peter, Tracy and Bruce:· May I please ask for advice about my choice?· Here is my dillema that I was trying to handle with the extra memory:· The database·will be 128 bytes per record and should contain at least·1,000·records.· There will be a second (very similer)·handheld unit that will·hold the same database.· The·first unit will not·need to have speed, but should be responsive and·will have to announce a hit on the database reliably to the second unit and the base computer wirelessly.· The second unit will have·every record modified and it will need to be faster,·employ a real time clock and update the base computer of every change it makes.· Modifications can come at any time from the base computer to the database on the handhelds which must remain in sync with each other and the base computer.
The entire database on each unit·would need to be loaded to the handheld, updated continuously sending the computer the result of each update·as a transaction·over·the next several hours and then off-loaded back to the computer for additional processing at the end of each use (could easily be used 5 times a day).· The distance between the handlends and the computer would vary greatly and could·be as far as 300 feet - I was considering the bluetooth modules but I worry if that would be secure and if they would burn up the battery too quickly.
Soooo...··My thinking was·that I need the databases to live on the handheld in order to process the lookups locally and make the thing responsive - without the long lag times·that I envision from·scanning up and down a database one record at a time wirelessly from a PC.· I will be working·in a metal building with lots of people, computers, wireless lans, walkie-talkies, other·wireless equipment attached to stamps, cell phones with bluetooth·and God only knows what all else·-·so I have a limited trust in the speed and accuracy of the·wireless link in data that HAS TO BE PERFECT (I envision using CRC on the packets somehow?).
I am wondering if·my thinking·was just stupid.
Should I just trust the wireless link to be fast and accurate and forget the database on the handhelds?
Should I just trust EEPROMs, SD or·flash memory chips to not fail prematurely (read that: at the worst time possible)?
Should I continue with this strategy and what components·wold you recommend?
Thank you for the reality check here.· I·VERY MUCH appreciate·any comments you guys would be willing to make.
···· Jeff
·
A more 'primitive' hand-held, merely to give you a keypad, display, and blue-tooth access to the 'main' computer (where the 'main' database would reside) might be a more practical approach. I know you're trying to minimize communication -- but an "INSERT" into a database shouldn't take much communication.
Perhaps a Palm-Pilot based solution might be more practical, though. With the cost of LCD, keypad, memory, and battery, you're starting to enter the Palm-Pilot cost area. And the Palm runs quite fast, runs for a month or so on 2 AAA batteries, and is hot-dockable at the end of the day.
After having read what you are going to require for the total solution, I have to second Allan's suggestion of looking into a Palm device. Wireless links in noisy environments are fraught with problems, and I don't think you are going to get the performance you want from a Stamp-managed database of the size you are envisioning. I am all about the challenge of designing your own solutions to vexing problems, but I think this time it might be best to realize that a good hardware platform is already available and resign yourself to creatively programming the database and GUI for the Palm or Embedded Windows OS. Take a look at www.appforge.com for an interesting VB programming environment for mobile devices.
peter
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Peter C. Charles
Director, Research and Technology
CyberBiota, Incorporated
Peter.charles@cyberbiota.com
http://www.cyberbiota.com
In the words of Super Chicken, "You knew the job was dangerous when you took it, Fred."
Yes, I actually started out·thinking about a Palm PC - then I realised that it would never work because I need the results updated real time at a distance of up to 300 feet.· I needed it dedicated to task so it was not possible to screw up - even with a first time user, and it could not be intimidating to use - even by a 90 year old blue haired lady with cateracts.· The accuracy has to be perfect so tactile switches are preferred over touch screen with fat fingers.· The units could be used outdoors so they would have to be tough and water resistant.· Lastly, I wanted something proprietary that could not be beamed to another person to use in the inconcievable case of a person that actually understands technology.· Don't want much, do I?·
Yes, this is a scary first project to take on, but I think it is do-able.· I just need to come up with the most rock solid design on the front end.
I can add batteries and directional antennas on the base computer if that helps.
So, now·what do you think?
··· Jeff
·
I haven't seen direct evidence in your posts supporting the need for FRAM over EEPROM, databases are typically read many/write few. Reading a EEPROM doesn't affect the life of the cell, only writing. If you can give an estimate of how frequently the most changed record will be updated and your expected lifetime for your product is, we can give you a better indication if FRAM is nessesary. Cheap interfaces for SD/MMC are availible, but the interfacing gruntwork will be on your shoulders. Alot of members taught the ALFAT-SD, but I see the product as overkill unless you need to insert the SD card into a PC computer and have the files readable by both the Stamp and the PC or you just don't want to deal with the complexity of the SD/MMC interfacing (not the easiest but not impossible).
If you settle on FRAM, get a few of the 32KB (256 Kbit) serial memories. If you have a Stamp that supports I2C, you can get that variety and be able to set the address of each one to support upto 8 of the chips on the same I2C bus (for a total of 256MB). Or you can purchase some of the SPI memories and include a 3 to 8 or a 2 to 4 decoder to provide the chip select signal so that you can share the clock, data in and data out lines between the memories.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
·1+1=10
Post Edited (Paul Baker) : 8/16/2005 5:57:08 PM GMT
Just be aware you are limited to 26 bytes of useable variables, and see what you can do. BTW, there does exist the SX platform -- the SX52 board has more resources, and is definitely faster, and is cheaper at $10.00. You'll need one $100 or so SX programming package to use the SX platforms. That way you can build a proprietary platform, yet still get the speed and memory space of the raw SX.
I bought a BS2Pe for the battery performance, larger memory and extra instruction set. I got the Matrix Orbital LK204-25 blue screen LCD because it had the keypad matrix built in, ran the LCD and the keypad thru the same I2C, and looked really cool. I ordered a 4 x 4 keypad from Jameco that I really liked but I had to build my own cable and software translate for it.
I have the LCD doing everything I need as a separate subroutine for the difficult stuff and the keypad translation works as a separate subroutine too (using some great advice that others got from this really wonderful forum). I am hoping that a gosub more appropriate than creating it in another slot - I am trying to make good educated guesses, but I feel like a beginner again. I have been programming for 30 years so I am picking that up OK - but this hardware is a long cry from the water-cooled mainframes I started with.
I was under the misconception that any access to a memory location wore an EEPROM down. As a result of all of this conversation, I have reduced the requirements dramitically. I still need 128 byte records, but I think I can get by with 4 initial writes and 8 rewrites per day for every record on the database.
I liked the idea of using SD for the database delivery and implied automatic backup when that was removed, but it still did not satisfy the requirement to include new records that crop up from the main computer or the requirement to deliver status changes in real time as they occur. I think I still need the handheld to PC link in real time, and if that is a solid link, do I even need the extra memory or the SD card at all? I guess it comes down to how much can I trust the wireless link given the conditions I described above and how responsive would the handheld be if every lookup had to be routed through the PC and back (including scrolls up and down the list).
I think I am stuck with both a large database and a wireless link, aren't I?
Jeff