I need to think about how the timing will work. The nice thing about SPI flash is that it can be read in a known amount of time, which works well with the serial window.
That's true once is comes out of reset.
However, newer SPI parts are now specifying varying reset-exit times, with Erase/pgm operations taking longer time-to-ready, than read.
This would mostly affect a WATCHDOG reset, but the BOOT code will need modification to tolerate this.
eg SD can be used for logging and updates, but mission critical code comes from SPI. Key difference : The system is still operational with removed/faulty SD.
Not being catered for here. The system builder will have to sort that out for themselves.
Eg: The SD option is not likely to be sharing with another Flash SPI part, it would be just the SD card only. If that SD card is removed then the equipment will not boot until the SD card is reinserted.
I'm not certain about that 'not likely' part...
Hobbyists might tolerate SD card insert, and having a dead-duck system with removed/faulty SD, but more serious industrial users are likely to want SD and SPI Flash.
eg SD can be used for logging and updates, but mission critical code comes from SPI. Key difference : The system is still operational with removed/faulty SD.
I need to think about how the timing will work. The nice thing about SPI flash is that it can be read in a known amount of time, which works well with the serial window.
That's true once is comes out of reset.
However, newer SPI parts are now specifying varying reset-exit times, with Erase/pgm operations taking longer time-to-ready, than read.
This would mostly affect a WATCHDOG reset, but the BOOT code will need modification to tolerate this.
Yes, we will need to review our current strategy. Right now, I just wait for the BUSY flag to go inactive, however long that takes. If it preempts serial, so be it.
I need to think about how the timing will work. The nice thing about SPI flash is that it can be read in a known amount of time, which works well with the serial window.
That's true once is comes out of reset.
However, newer SPI parts are now specifying varying reset-exit times, with Erase/pgm operations taking longer time-to-ready, than read.
This would mostly affect a WATCHDOG reset, but the BOOT code will need modification to tolerate this.
Yes, we will need to review our current strategy. Right now, I just wait for the BUSY flag to go inactive, however long that takes. If it preempts serial, so be it.
The data is annoyingly vague around reset exit. It says "During this period, no command will be accepted" - sounds like read status pauses, along with everything else. (BUSY floats)
That means this
send write-disable command to clear WEL
send read-status command
get status
if WEL high, no SPI memory
could falsely exit, if done too soon after reset ?
Better for SPI present test might be a repeated RDID(9FH) with timeout, following the reset command ?
Then, BUSY can be checked as usual.
A light pullup, would then also work on code that does
if BUSY high, wait for erase/write to finish
Some example times :
tREADY2 (Macronix MX25LM51245G)
Reset Recovery time (During instruction decoding) 40 us
Reset Recovery time (for read operation) 40 us
Reset Recovery time (for program operation) 310 us
Reset Recovery time(for SE4KB operation) 12 ms
Reset Recovery time (for BE64K operation) 25 ms
Reset Recovery time (for Chip Erase operation) 100 ms
Reset Recovery time (for WRSR operation) 40 ms
This time does reference from the reset leading edge, so an (ideal) external watchdog can generate a 100ms reset pulse.
For software resets, that has less control.
Eg: The SD option is not likely to be sharing with another Flash SPI part, it would be just the SD card only. If that SD card is removed then the equipment will not boot until the SD card is reinserted.
I'm not certain about that 'not likely' part...
Hobbyists might tolerate SD card insert, and having a dead-duck system with removed/faulty SD, but more serious industrial users are likely to want SD and SPI Flash.
eg SD can be used for logging and updates, but mission critical code comes from SPI. Key difference : The system is still operational with removed/faulty SD.
For that configuration, the designer will have the SD on different pins, and it will be his code to boot the SD card. So its up to the designer to figure what happens if the SD card fails.
In my boot code, I expect that if an SD card is present, and it fails to boot for one of a number of reasons, then fallback will be to serial.
Chip, timing is NOT predictable with SD cards. It depends on the controllers and they vary greatly between manufacturers and various card versions within manufacturers.
Just want to chime in here. What we are doing is very basic, and hopefully won't map over to what I'm seeing with SD cards.
Basically, all but industrially rated SD cards, which cost a lot of money, are flaky. I've got a product I'm working with right now. It used SD cards in two contexts, one being the SD card runs a Raspi and the other an 8 bit Atmel board.
A fairly significant percentage, say 10 to 20 percent, get corrupt or just die over a period of months.
I'm having real trouble getting to root causes. In some cases, it's users removing the cards from the 8 bit board without a unmount type operation happening first so the pins are inactive. That one is known. User needs to respect the card.
Some are getting corrupted other ways. Not sure what those are. It seems to be related to activity on the cards, which is mostly read on the 8 bit board.
On the Raspi:
The Raspi 1 rarely corrupts cards. The 2 does it a lot, and the 3 appears somewhere in between, though it will be a while before I have that data. Lol, it can make more sense to phase out a Raspi 2 than it does to deal with the SD cards.
These are industrial systems. I'm not at all confident in SD card as primary boot device. It's attractive, but not as solid as it should be. Seems worse than it was a while ago, but that could just be perception bias happening from more machines in general out there now.
My gut says whatever we do should be ultra conservative. Known, basic, touch the card, mode change, etc... as little as possible, use very permissive timings. These things appear to vary more than it may seem.
More advanced, faster, push the card, configure the card, type cases may well be best left to post P2 init and boot processes.
I wonder where the flakiness is coming from? You'd think a memory card wouldn't be able to die so easily. Maybe because SD cards are such consumer commodities, their quality-control is no better than the $1 phone charger at the Dollar Store, that likely doesn't even work. I think SPI flash chip manufacturers couldn't skate as easily on poor quality, as they'd be detected and there'd be market consequences. Not so for the untraceable junk with meaningless brand names.
I think QC has a lot to do with it. In particular, the quality of controller on the card appears to vary quite a lot. I just don't think those are getting validated across a wide range of possible cases.
Like I said, our use case here is really basic. Maybe just keeping it that way is wise, that's all.
I don't intend to Smile on SD cards in general. But, I am just not seeing as robust of operation as I have prior too.
While I have not experienced the problem some people have, I mostly buy genuine SanDisk cards from a reputable supplier. However, having said that, I have an eBay cheapie 8GB C10 and a 32GB C10 and both have performed flawlessly. Both get removed and plugged regularly in my P1 and occasionally in my P2 boards.
Often the cheapies have Smile controllers! Also often the cheapies are not the capacity they are being sold as.
Perhaps handling also has something to do with it too. Easy to get a static charge to hit the card.
I'd put operational data corruption separately from card reliability.
I know I've had many SD cards misbehave/die because of solder breaks on the internal PCB. This is simply a case of the very thin and flimsy board not handling the mechanical treatment I put them through. Same thing happens to protruding USB connectors. I've learnt to protect them in transport now.
One answer is to use the write protect tab of the SD card, since we only need to read to boot
I've been involved with a rasp pi 1 NFC door access system that had a few card corruptions early on. The solution (that's worked well for about 4 years now) was to write-protect the SD card and use a flash drive for output logs etc.
That may sound limiting and not ideal, but at least it works. Despite this I still think its worth having SD boot if possible.
When it checks for the two prenamed/hardcoded FAT files (currently "_P2_BOOT.BIX" and "_P2_BOOT.BIY") I now check for the first one to be in the first 16 directory sectors ie the first 256 entries, and if not found, check again for the second file name.
If found, then I load the files data into Hub starting at $0_0000 onwards. The length will be the length of the file rounded up to the next 512 byte sector if necessary.
This means for a 32KB file, I will load 64 sectors of 512 bytes. For a file of say 10 bytes, I will load 1 sector = 512 bytes. I figure this is good enough.
I don't check for file fragmentation. This is the users responsibility. The usual is 32KB clusters. I don't check for a maximum file size, so you can load the whole 1MB but I am not sure if the top 16KB will be protected, nor wraparound if there is less than 1MB of Hub ram. Again the users responsibility.
This is working.
BTW I have two files that I load. One is ~96 bytes, and the other is 32KB. They both flash RGB LEDs that I have connected to the P2 pins.
When it comes to SD cards, I refuse to take chances. To date, brand-name cards purchased from major brick-and-mortar stores have not let me down. The absence of flaky behavior and lost data has been more than worth the few extra bucks.
I wouldn't even be upset if Parallax published an approved SD card list that consisted of only one card, one brand, one size, or one vendor.
When it comes to SD cards, I refuse to take chances. To date, brand-name cards purchased from major brick-and-mortar stores have not let me down. The absence of flaky behavior and lost data has been more than worth the few extra bucks.
I wouldn't even be upset if Parallax published an approved SD card list that consisted of only one card, one brand, one size, or one vendor.
Of course I may be alone in this.
The only problem with this is that SD card sizes are a moving target. Even within the one size, technologies change.
Having said that, I have never had a problem with "real" SanDisk" cards. That's what I, and many others, recommend.
Yup. None of my RPi boards have ever had anything but a real SanDisk card installed. Same with our cell phones.
But the only way I know to be sure I'm getting a real SanDisk card is to buy it from a real store. In the wild, it's a bit like buying an Ultrafire Li-ion cell - almost sure to be a fraud.
You are absolutely right about card capacities! Do SanDisk make a 4GB or 8GB card anymore? For my recent installation of Stretch, 32GB was the smallest card I could find. But it was at the price of the 8GB card I purchased for Wheezy.
I'm hoping I will get some time now to check all this stuff including your boot code. But I don't have a problem with specifying "works with SanDisk" and if somebody wants to use their 10 year old eBay "genuine" 2GB ol faithful then that is their problem. I also don't have a problem with depending solely upon the card because if it is removed then it is the same as an RPi and last time i checked nobody had stopped buying those for that reason.
Yup. None of my RPi boards have ever had anything but a real SanDisk card installed. Same with our cell phones.
But the only way I know to be sure I'm getting a real SanDisk card is to buy it from a real store. In the wild, it's a bit like buying an Ultrafire Li-ion cell - almost sure to be a fraud.
You are absolutely right about card capacities! Do SanDisk make a 4GB or 8GB card anymore? For my recent installation of Stretch, 32GB was the smallest card I could find. But it was at the price of the 8GB card I purchased for Wheezy.
16GB can be bought in Oz, but at 2.5x the price of the old 8GB
Yup. None of my RPi boards have ever had anything but a real SanDisk card installed. Same with our cell phones.
But the only way I know to be sure I'm getting a real SanDisk card is to buy it from a real store. In the wild, it's a bit like buying an Ultrafire Li-ion cell - almost sure to be a fraud.
You are absolutely right about card capacities! Do SanDisk make a 4GB or 8GB card anymore? For my recent installation of Stretch, 32GB was the smallest card I could find. But it was at the price of the 8GB card I purchased for Wheezy.
16GB can be bought in Oz, but at 2.5x the price of the old 8GB
this is very good news. So your 'directory' loader, looks for two possible files to load, loads up to 1MB and starts COG 0 in HUBexec at address 0? Or does it load something into the COG and starts in COGexec?
As far as I understand Flash loads a COG and goes from there, your MBR code loads into HUB copies to COG and starts there, and the serial commands Prop_Txt and Prop_Hex load the complete HUB ram and start in HUBexec at Address 0.
If so, the Flash loader and your MBR loader will need to call some second stage loader to load a complete HUB image, but your directory loader does not.
This is a perfectly fine solution. Thank you!
and the MBR loader MAY work even with other file systems as FAT32. Wonderful.
this is very good news. So your 'directory' loader, looks for two possible files to load, loads up to 1MB and starts COG 0 in HUBexec at address 0? Or does it load something into the COG and starts in COGexec?
After loading into hub $0 upwards, the bottom 2KB (496 longs) are copied to Cog 0 at $000-$1EF and then a JMP $000 (Cog) so its cog code executing.
FWIW hubexec cannot run from hub $00000-$00FFF (think thats the limit ie bottom 4KB) due to mapping the program counter for cog and lut with hub.
As far as I understand Flash loads a COG and goes from there, your MBR code loads into HUB copies to COG and starts there, and the serial commands Prop_Txt and Prop_Hex load the complete HUB ram and start in HUBexec at Address 0.
No. 1KB Flash is loaded into hub $00000 upwards by code running in Cog 0, then the code jumps to lut and that then copies that 1KB from hub $00000+ to Cog $000+ and then JMP $000 (cog 0).
If so, the Flash loader and your MBR loader will need to call some second stage loader to load a complete HUB image, but your directory loader does not.
This is a perfectly fine solution. Thank you!
and the MBR loader MAY work even with other file systems as FAT32. Wonderful.
One answer is to use the write protect tab of the SD card, since we only need to read to boot
Unfortunately that won't protect against corruption. The write protect tab on SD cards isn't a physical write protect, it's just setting one of the bits that the software can read, and then decide (and most actually don't) that 'Ok, user doesn't want me to write.'
So this would do very little for accidental corruption.
One answer is to use the write protect tab of the SD card, since we only need to read to boot
Unfortunately that won't protect against corruption. The write protect tab on SD cards isn't a physical write protect, it's just setting one of the bits that the software can read, and then decide (and most actually don't) that 'Ok, user doesn't want me to write.'
So this would do very little for accidental corruption.
I did a really bad job with by previous post, it was posted in response to Potatohead's experiences with the Pi. Let me put it differently
- I think SD card booting can be reliable if the SD card is only read by the P2, never written.
- This is based on experience with a Rasp Pi 1 project that has been running continuously since early 2013.
- We had a lot of corruptions during the first few months of that project, before switching the card to read only, adding a usb flash drive for file output, and had no SD issues since
- I believe we can use the SD card the same way on the P2 reliably. ie strictly read commands only.
- This is still very useful as it lets field upgrades be achieved by end users relatively easily
Those feeling more adventurous could try writing to the SD card from the P2, and see how bad it really is. Perhaps fixed length write commands (single byte etc), or power supply shutdown management might help limit the damage
Potatohead's experiences with the Pi. Let me put it differently
- I think SD card booting can be reliable if the SD card is only read by the P2, never written.
- This is based on experience with a Rasp Pi 1 project that has been running continuously since early 2013.
- We had a lot of corruptions during the first few months of that project, before switching the card to read only, adding a usb flash drive for file output, and had no SD issues since
Thanks for that. Appreciated, and something to explore in my troublesome scenario. It's not too many cards, but enough to really bother me.
Re: real SanDisk
Yeah, I've no failures on those that aren't "user touched card." That one does not concern me. Happens. Just work through it, find out why, and then get them somewhere they don't have to mess with the card.
Interesting. It would stand to reason that a card treated as read-only in an embedded system would not fail, but those that were being modified and maybe powered down at the wrong time suffered corruption that disabled the system.
So, the SD used for booting should remain unmodified, while a secondary SD could be used for data.
Yep, it's file corruption from lagging buffer writes that is being addressed. Not electrical/mechanical damage. They only have to be a separate volumes. Even the rescue tools can be on the boot volume, ready to fix up a corrupted data volume.
Comments
That's true once is comes out of reset.
However, newer SPI parts are now specifying varying reset-exit times, with Erase/pgm operations taking longer time-to-ready, than read.
This would mostly affect a WATCHDOG reset, but the BOOT code will need modification to tolerate this.
Then the SD card should be run on other pins.
Yes, we will need to review our current strategy. Right now, I just wait for the BUSY flag to go inactive, however long that takes. If it preempts serial, so be it.
The data is annoyingly vague around reset exit. It says "During this period, no command will be accepted" - sounds like read status pauses, along with everything else. (BUSY floats)
That means this
send write-disable command to clear WEL
send read-status command
get status
if WEL high, no SPI memory
could falsely exit, if done too soon after reset ?
Better for SPI present test might be a repeated RDID(9FH) with timeout, following the reset command ?
Then, BUSY can be checked as usual.
A light pullup, would then also work on code that does
if BUSY high, wait for erase/write to finish
Some example times :
This time does reference from the reset leading edge, so an (ideal) external watchdog can generate a 100ms reset pulse.
For software resets, that has less control.
In my boot code, I expect that if an SD card is present, and it fails to boot for one of a number of reasons, then fallback will be to serial.
Chip, timing is NOT predictable with SD cards. It depends on the controllers and they vary greatly between manufacturers and various card versions within manufacturers.
Basically, all but industrially rated SD cards, which cost a lot of money, are flaky. I've got a product I'm working with right now. It used SD cards in two contexts, one being the SD card runs a Raspi and the other an 8 bit Atmel board.
A fairly significant percentage, say 10 to 20 percent, get corrupt or just die over a period of months.
I'm having real trouble getting to root causes. In some cases, it's users removing the cards from the 8 bit board without a unmount type operation happening first so the pins are inactive. That one is known. User needs to respect the card.
Some are getting corrupted other ways. Not sure what those are. It seems to be related to activity on the cards, which is mostly read on the 8 bit board.
On the Raspi:
The Raspi 1 rarely corrupts cards. The 2 does it a lot, and the 3 appears somewhere in between, though it will be a while before I have that data. Lol, it can make more sense to phase out a Raspi 2 than it does to deal with the SD cards.
These are industrial systems. I'm not at all confident in SD card as primary boot device. It's attractive, but not as solid as it should be. Seems worse than it was a while ago, but that could just be perception bias happening from more machines in general out there now.
My gut says whatever we do should be ultra conservative. Known, basic, touch the card, mode change, etc... as little as possible, use very permissive timings. These things appear to vary more than it may seem.
More advanced, faster, push the card, configure the card, type cases may well be best left to post P2 init and boot processes.
I wonder where the flakiness is coming from? You'd think a memory card wouldn't be able to die so easily. Maybe because SD cards are such consumer commodities, their quality-control is no better than the $1 phone charger at the Dollar Store, that likely doesn't even work. I think SPI flash chip manufacturers couldn't skate as easily on poor quality, as they'd be detected and there'd be market consequences. Not so for the untraceable junk with meaningless brand names.
Like I said, our use case here is really basic. Maybe just keeping it that way is wise, that's all.
I don't intend to Smile on SD cards in general. But, I am just not seeing as robust of operation as I have prior too.
Often the cheapies have Smile controllers! Also often the cheapies are not the capacity they are being sold as.
Perhaps handling also has something to do with it too. Easy to get a static charge to hit the card.
I know I've had many SD cards misbehave/die because of solder breaks on the internal PCB. This is simply a case of the very thin and flimsy board not handling the mechanical treatment I put them through. Same thing happens to protruding USB connectors. I've learnt to protect them in transport now.
I've been involved with a rasp pi 1 NFC door access system that had a few card corruptions early on. The solution (that's worked well for about 4 years now) was to write-protect the SD card and use a flash drive for output logs etc.
That may sound limiting and not ideal, but at least it works. Despite this I still think its worth having SD boot if possible.
Where is the write protect tab on a microSD?
When it checks for the two prenamed/hardcoded FAT files (currently "_P2_BOOT.BIX" and "_P2_BOOT.BIY") I now check for the first one to be in the first 16 directory sectors ie the first 256 entries, and if not found, check again for the second file name.
If found, then I load the files data into Hub starting at $0_0000 onwards. The length will be the length of the file rounded up to the next 512 byte sector if necessary.
This means for a 32KB file, I will load 64 sectors of 512 bytes. For a file of say 10 bytes, I will load 1 sector = 512 bytes. I figure this is good enough.
I don't check for file fragmentation. This is the users responsibility. The usual is 32KB clusters. I don't check for a maximum file size, so you can load the whole 1MB but I am not sure if the top 16KB will be protected, nor wraparound if there is less than 1MB of Hub ram. Again the users responsibility.
This is working.
BTW I have two files that I load. One is ~96 bytes, and the other is 32KB. They both flash RGB LEDs that I have connected to the P2 pins.
I wouldn't even be upset if Parallax published an approved SD card list that consisted of only one card, one brand, one size, or one vendor.
Of course I may be alone in this.
The only problem with this is that SD card sizes are a moving target. Even within the one size, technologies change.
Having said that, I have never had a problem with "real" SanDisk" cards. That's what I, and many others, recommend.
But the only way I know to be sure I'm getting a real SanDisk card is to buy it from a real store. In the wild, it's a bit like buying an Ultrafire Li-ion cell - almost sure to be a fraud.
You are absolutely right about card capacities! Do SanDisk make a 4GB or 8GB card anymore? For my recent installation of Stretch, 32GB was the smallest card I could find. But it was at the price of the 8GB card I purchased for Wheezy.
Somebody ripping you off there for sure.
this is very good news. So your 'directory' loader, looks for two possible files to load, loads up to 1MB and starts COG 0 in HUBexec at address 0? Or does it load something into the COG and starts in COGexec?
As far as I understand Flash loads a COG and goes from there, your MBR code loads into HUB copies to COG and starts there, and the serial commands Prop_Txt and Prop_Hex load the complete HUB ram and start in HUBexec at Address 0.
If so, the Flash loader and your MBR loader will need to call some second stage loader to load a complete HUB image, but your directory loader does not.
This is a perfectly fine solution. Thank you!
and the MBR loader MAY work even with other file systems as FAT32. Wonderful.
Enjoy!
Mike
FWIW hubexec cannot run from hub $00000-$00FFF (think thats the limit ie bottom 4KB) due to mapping the program counter for cog and lut with hub.
No. 1KB Flash is loaded into hub $00000 upwards by code running in Cog 0, then the code jumps to lut and that then copies that 1KB from hub $00000+ to Cog $000+ and then JMP $000 (cog 0).
So this would do very little for accidental corruption.
I did a really bad job with by previous post, it was posted in response to Potatohead's experiences with the Pi. Let me put it differently
- I think SD card booting can be reliable if the SD card is only read by the P2, never written.
- This is based on experience with a Rasp Pi 1 project that has been running continuously since early 2013.
- We had a lot of corruptions during the first few months of that project, before switching the card to read only, adding a usb flash drive for file output, and had no SD issues since
- I believe we can use the SD card the same way on the P2 reliably. ie strictly read commands only.
- This is still very useful as it lets field upgrades be achieved by end users relatively easily
Those feeling more adventurous could try writing to the SD card from the P2, and see how bad it really is. Perhaps fixed length write commands (single byte etc), or power supply shutdown management might help limit the damage
Thanks for that. Appreciated, and something to explore in my troublesome scenario. It's not too many cards, but enough to really bother me.
Re: real SanDisk
Yeah, I've no failures on those that aren't "user touched card." That one does not concern me. Happens. Just work through it, find out why, and then get them somewhere they don't have to mess with the card.
So, the SD used for booting should remain unmodified, while a secondary SD could be used for data.
Are you sure that would be sufficient isolation?