@rogloh said:
Of course you can always unbrick flash with serial if you can boot to that. The issue is that it'd be nice for people using some device to not have to buy prop-plugs, use a PC and tools to reprogram a failed device in the field (particularly end user consumers) and just have/download a SD card file in order to automatically restore things for them if they can put the card into the device and reset it, then take it out again once it's fixed. I have some products at home that work like that and it's convenient. If the P2 cannot do this type of thing inherently especially if it was originally designed with this capability, that's not good.
I would never use an SD card for booting. It's just not reliable enough. You can use SD cards for data storage in data-loggers or similar applications. You can also run user applications from SD card on retro computers, game stations or whatever. But booting should also be done from flash ROM. In all my boards that might require updates in the flield I have a multi-page flash ROM (bigger than 32k for P1 or >512kB for P2) and a boot loader in page 0 that is nerver changed. The actual application software is in page 1. So if something goes wrong the bootloader can still communicate with a PC and restore a valid software image.
If your application requires excternal data storage or more than several MB of data or code I'd connect an SD card to some extra pins that are not shared with the boot ROM.
@ManAtWork said:
If your application requires external data storage or more than several MB of data or code I'd connect an SD card to some extra pins that are not shared with the boot ROM.
It's looking more and more like this is the best way forward. Treat SD Cards as non-native booting - needs a boot loader in the EEPROM. And at the same time can then support the much faster 4-bit SD mode of operation. The 240 ohm resistor on the data line had sucked the life out of performance as it was anyway.
@ManAtWork said:
If your application requires external data storage or more than several MB of data or code I'd connect an SD card to some extra pins that are not shared with the boot ROM.
For any actual product where the SD card is accessible, the ROM SD loader was never an option. It has no way of indicating failure to the user. Also, it will boot any _BOOT_P2.BIX, which may cause undesirable outcomes if the P2 is connected to something spicy (motors etc) and gets loaded with an application for a totally different hardware setup.
For something like the P2 stamp here where the card is sortof internal, I don't really think there's a problem with using it. Just get a name brand SD card, it will be perfectly reliable. RasPis all boot off SD cards and they're fine. The main issue is when the P2 resets during a transfer, which only really happens if you externally reset it or the BOD is triggered (the card also has one, so in most cases both should reset).
Around 2003 I built a CNC controller with a full custom Coldfire (Motorola 68k based CPU) board. It ran a multitasking OS with GUI and filesystem I have written all from scratch. It had a USB device port but no host (for memory sticks) and no network. Around 2008 everybody asked fro network connectivity. RasPis weren't available at that time so we decided to buy really expensive ARM CPU boards running Linux and supporting ethernet and USB host. We had ~100 in the field. Once every month one came back with a broken SD card. We replaced the card and it worked again.
A few dozens of those machines are still running today, 20 years later. We made some experiments with BananaPis which have a true SSD with SATA instead of the SD card. They are much more reliable. But we decided to replace them with NUCs.
Ok, off topic. But I wanted to point out that IMHO SD cards are OK for student projects or hardware hacks in the lab. But I'd never use them in an industrial application, at least not for something critical like to boot the OS from.
The conversation is mainly about using the SD Card to perform system updates rather than being the system. With the idea that it could also manage restoring a corrupted EEPROM that is no longer booting. But such an option isn't proving worth the pain of getting them sharing on the same pins.
@evanh said:
The conversation is mainly about using the SD Card to perform system updates rather than being the system. With the idea that it could also manage restoring a corrupted EEPROM that is no longer booting. But such an option isn't proving worth the pain of getting them sharing on the same pins.
So @evanh have you actually tried this out (ie. boot a P2 off SD with the flash present and then try to access the SD card and flash after this at runtime) and you personally already found it to not work, or are you presuming it will fail based on your understanding of the SPI signals being sent out from the P2 during the boot phase? Even if it is a bit flaky and only works once or something and the SD later locks up at the next reset, maybe it could still help restore a flash in a pinch as a one off operation. I'll think I'll have to try this at some point to prove it for myself. I do know they cannot both drive the shared MISO pin at the same time because the flash would swamp the SD output with that resistor, but if flash doesn't drive its output until it recognizes a valid instruction clocked in and aborts each failed access with each rising CS (SD CLK), perhaps it could still work....?
EDIT: I just read this in the ISSI data sheet for the flash fitted to the later rev of the P2-stamp.
"Generally, CE# remains low for 8-bit transfer multiples to transfer byte granularity information. All commands will not be accepted if CE# is returned high not at an 8-bit boundary."
If the SD clock is the CE# pin for the flash, then each time it rises with a new SD data bit it will reject the flash operation. The flash should just sit there and ignore SD accesses, at least that's what I would have thought. Isn't that how it was meant to work when originally designed?
I've not tested it. There have been reported incidents of both SD lockups and SD corruption. Yes, I'm presuming they were attempting to enable both devices at those times - which is as simple as leaving the FLASH switch on while booting an SD Card.
Ah, I do remember a more recent one where someone was trying to use both devices from application code and was having to keep reformatting. It was finally resolved by changing all the low-level routines from CPOL=0 to CPOL=1. Which was as simple as setting the pin mode of the clock pin to be inverted output.
Ok well we need to check if there is a way for it to work, not go from hearsay. I think at runtime the restoration code could jump through whatever hoops are needed to keep both devices working. It's just got to be able to initially boot off SD once, and then we get full control of the pins.
SD boot might work if the EEPROM doesn't drive it's MISO pin as soon as CS goes low. If if it does drive immediately then it'll overpower any SD Card because of the 240 ohm resistor on the SD side.
The clock problem definitely means EEPROM booting is excluded while the SD Card is present. Which means the Card can't be left in place for long term storage. It has to be removed before next reboot.
Probably the best way to make EEPROM booting work would be to somehow gate the SD CLK so that the SD Card doesn't see any clocking while the EEPROM is repeatedly selected. Without any clocking, it won't drive either CMD or DAT0.
EDIT: Here's just the minimum for EEPROM exclusive booting while still being able to read/write an SD Card post-boot.
EDIT2: Err, maybe the top 1G32 isn't needed ... Throwing this out as something to build on at least.
EDIT3: Lol, it's not really minimum since P57 could be used much more effectively as SD CLK pin. No need for any added gates then.
More I look at it, more I think all that is not necessary.
If the SD is repeatedly selected but not receiving clock, it won't drive its MOSI (or even if it does, that can be taken care of by adding a suitable value resistor in series with SD pin 3).
That would be the flash case. The opposite is the same - the flash will not produce anything on its MOSI line if there is no driving clock.
So, I think if the P2 executes the boot sequence properly, there should not be any problem booting from either flash or SD.
The current rev.D places the flash as the preferred source, while the previous revisions were opposite. I think this will do fine as it is now.
Will update the github repo in the coming day/days
@knivd said:
So, I think if the P2 executes the boot sequence properly, ...
It doesn't.
The first thing the ROM code does is drive the CLK low, then it drives the CS low too, both devices are now selected ... and critically, this repeats for every command. Whatever bit level was last on MOSI from prior command gets clocked into the other device at the beginning of each command.
To be fair, it probably only impacts the SD Card because it is not operating in SPI mode until switched to it. SD mode doesn't have a CS signal, or equivalent, therefore won't reset its command buffer when CS returns high.
But just that alone can lock the data pins and thereby mess with EEPROM command issuing - A 240 ohm resistor on CMD also would've helped here - But has been capable of corrupting stored SD data as well.
None of these designs have a way to power cycle the SD card with P2 reset. That is going to remain a problem for SD boot (without a change in P2 boot code). In order to perform a soft reset, I had to perform a CMD0 before the soft reset to place the card in a state where to the P2 will boot from SD.
However, if you punch the reset button (or attempt to program the flash)..........
@evanh said:
The first thing the ROM code does is drive the CLK low, then it drives the CS low too, both devices are now selected ...
Aaahhh, the dreadful SPI mode 0. Those from 0 to 2 must be outlawed.
Only mode 3 works properly in actual shared bus.
If someone from Parallax is reading here, given the already known problems (and not even linked to the Stamp itself) with this otherwise great idea of sharing the flash and the SD, how hard would it be to change that boot code and intialise the bus in mode 3 ?
@evanh said:
Terry,
I gather that was the ROM booting straight to the SD Card. Not booting the EEPROM, right? Did you figure a workaround at all?
There is no workaround, except for soft booting. If you know you are going to soft boot, you can CMD0 and it will allow it to reboot. While the SD card is mounted, you are out of luck. Power cycling it is the only way to ensure the state of the SD card.
@ke4pjw did you look at supplying extra SDXC clock pulses early in your program, in order to have the SDXC card release its SO line? I only found one SDXC card so far, that didn't respond to that fix.
I think this can mostly fix the boot/reboot issue, not so sure if it can help where the SDXC card is in active use beyond boot though...
Actually, on a further thought it seems to me that there shouldn't be a problem with SPI 0 as it is now, and even if both the flash and the SD are getting selected simultaneously when the CLK/CS and CS/CLK and both low. The devices should still behave as expected if not getting clock, and only problematic part there is the MISO which I think could be put through an OR gate, and both the gate inputs weakly pulled up because as far as I know the SD card initialised with its MISO in OD mode. The other one at the flash's side probably isn't, but just in case.
I am currently on my summer travels, but will play more with the idea as soon as I get back, possibly for a new reivision later.
@knivd said:
Actually, on a further thought it seems to me that there shouldn't be a problem with SPI 0 as it is now,
Evidence is this matters when the SD Card is operating in SPI interface mode. I haven't sat down and examined what goes wrong with cmode0 but it really does mess up sharing of the SPI bus. Others have verified that just by flipping the clock polarities their existing routines went from corrupting storage to working brilliant.
EEPROM booting is different though since the SD Card is in the default SD interface mode at that stage. Each EEPROM CS cycle is a clock cycle to the SD Card. If a SD command becomes valid then the Card will respond back on CMD (MOSI). This would fight horribly because the Prop2 will also be driving the same line. What the EEPROM then sees is anyone's guess.
That's my best understanding so far. It may be that booting isn't as bad as I'm imagining. SD booting probably is okay. The EEPROM won't get upset or cause problems as long its MISO pin stays floating while the CS pin is rapidly cycling - Which seems to be true in the datasheets I've read.
Eek, reading all that, I may have tipped too much cold water on the project.
More recent events have demonstrated shared booting may not be as bad as I was portraying there. The biggest issue to get right, aside from also using cmode3, looks to be always ensuring the releasing of the SD card back to "ready" state during run-time use. This then allows, after a reset, the ROM code to correctly detect the presence of the SD card and boot up as desired.
Or, alternatively, power cycling the SD card when the P2 is reset. Problem here is there is no clear way to signal a reset externally from software. The HUBSET for resetting the Prop2 doesn't affect the RESN pin.
Comments
I would never use an SD card for booting. It's just not reliable enough. You can use SD cards for data storage in data-loggers or similar applications. You can also run user applications from SD card on retro computers, game stations or whatever. But booting should also be done from flash ROM. In all my boards that might require updates in the flield I have a multi-page flash ROM (bigger than 32k for P1 or >512kB for P2) and a boot loader in page 0 that is nerver changed. The actual application software is in page 1. So if something goes wrong the bootloader can still communicate with a PC and restore a valid software image.
If your application requires excternal data storage or more than several MB of data or code I'd connect an SD card to some extra pins that are not shared with the boot ROM.
It's looking more and more like this is the best way forward. Treat SD Cards as non-native booting - needs a boot loader in the EEPROM. And at the same time can then support the much faster 4-bit SD mode of operation. The 240 ohm resistor on the data line had sucked the life out of performance as it was anyway.
For any actual product where the SD card is accessible, the ROM SD loader was never an option. It has no way of indicating failure to the user. Also, it will boot any _BOOT_P2.BIX, which may cause undesirable outcomes if the P2 is connected to something spicy (motors etc) and gets loaded with an application for a totally different hardware setup.
For something like the P2 stamp here where the card is sortof internal, I don't really think there's a problem with using it. Just get a name brand SD card, it will be perfectly reliable. RasPis all boot off SD cards and they're fine. The main issue is when the P2 resets during a transfer, which only really happens if you externally reset it or the BOD is triggered (the card also has one, so in most cases both should reset).
I bet the RPi is loading a big-Smile firmware from a flash chip first. The SD boot will be from that.
Yes, they have some onboard firmware that does the actual booting, but it still in the end loads the full fat OS from the card.
And, in case of RPi4, this firmware downloads and inits itself in something about 5-7 seconds so even bare metal code boot time is at least 7 seconds.
It takes my home router 60 seconds to boot, then a further 8 seconds for the fibre WAN connection and 2 more seconds for the Wifi.
Around 2003 I built a CNC controller with a full custom Coldfire (Motorola 68k based CPU) board. It ran a multitasking OS with GUI and filesystem I have written all from scratch. It had a USB device port but no host (for memory sticks) and no network. Around 2008 everybody asked fro network connectivity. RasPis weren't available at that time so we decided to buy really expensive ARM CPU boards running Linux and supporting ethernet and USB host. We had ~100 in the field. Once every month one came back with a broken SD card. We replaced the card and it worked again.
A few dozens of those machines are still running today, 20 years later. We made some experiments with BananaPis which have a true SSD with SATA instead of the SD card. They are much more reliable. But we decided to replace them with NUCs.
Ok, off topic. But I wanted to point out that IMHO SD cards are OK for student projects or hardware hacks in the lab. But I'd never use them in an industrial application, at least not for something critical like to boot the OS from.
The conversation is mainly about using the SD Card to perform system updates rather than being the system. With the idea that it could also manage restoring a corrupted EEPROM that is no longer booting. But such an option isn't proving worth the pain of getting them sharing on the same pins.
So @evanh have you actually tried this out (ie. boot a P2 off SD with the flash present and then try to access the SD card and flash after this at runtime) and you personally already found it to not work, or are you presuming it will fail based on your understanding of the SPI signals being sent out from the P2 during the boot phase? Even if it is a bit flaky and only works once or something and the SD later locks up at the next reset, maybe it could still help restore a flash in a pinch as a one off operation. I'll think I'll have to try this at some point to prove it for myself. I do know they cannot both drive the shared MISO pin at the same time because the flash would swamp the SD output with that resistor, but if flash doesn't drive its output until it recognizes a valid instruction clocked in and aborts each failed access with each rising CS (SD CLK), perhaps it could still work....?
EDIT: I just read this in the ISSI data sheet for the flash fitted to the later rev of the P2-stamp.
"Generally, CE# remains low for 8-bit transfer multiples to transfer byte granularity information. All commands will not be accepted if CE# is returned high not at an 8-bit boundary."
If the SD clock is the CE# pin for the flash, then each time it rises with a new SD data bit it will reject the flash operation. The flash should just sit there and ignore SD accesses, at least that's what I would have thought. Isn't that how it was meant to work when originally designed?
I've not tested it. There have been reported incidents of both SD lockups and SD corruption. Yes, I'm presuming they were attempting to enable both devices at those times - which is as simple as leaving the FLASH switch on while booting an SD Card.
Ah, I do remember a more recent one where someone was trying to use both devices from application code and was having to keep reformatting. It was finally resolved by changing all the low-level routines from CPOL=0 to CPOL=1. Which was as simple as setting the pin mode of the clock pin to be inverted output.
Ok well we need to check if there is a way for it to work, not go from hearsay. I think at runtime the restoration code could jump through whatever hoops are needed to keep both devices working. It's just got to be able to initially boot off SD once, and then we get full control of the pins.
SD boot might work if the EEPROM doesn't drive it's MISO pin as soon as CS goes low. If if it does drive immediately then it'll overpower any SD Card because of the 240 ohm resistor on the SD side.
The clock problem definitely means EEPROM booting is excluded while the SD Card is present. Which means the Card can't be left in place for long term storage. It has to be removed before next reboot.
Probably the best way to make EEPROM booting work would be to somehow gate the SD CLK so that the SD Card doesn't see any clocking while the EEPROM is repeatedly selected. Without any clocking, it won't drive either CMD or DAT0.
EDIT: Here's just the minimum for EEPROM exclusive booting while still being able to read/write an SD Card post-boot.
EDIT2: Err, maybe the top 1G32 isn't needed ... Throwing this out as something to build on at least.
EDIT3: Lol, it's not really minimum since P57 could be used much more effectively as SD CLK pin. No need for any added gates then.
More I look at it, more I think all that is not necessary.
If the SD is repeatedly selected but not receiving clock, it won't drive its MOSI (or even if it does, that can be taken care of by adding a suitable value resistor in series with SD pin 3).
That would be the flash case. The opposite is the same - the flash will not produce anything on its MOSI line if there is no driving clock.
So, I think if the P2 executes the boot sequence properly, there should not be any problem booting from either flash or SD.
The current rev.D places the flash as the preferred source, while the previous revisions were opposite. I think this will do fine as it is now.
Will update the github repo in the coming day/days
It doesn't.
The first thing the ROM code does is drive the CLK low, then it drives the CS low too, both devices are now selected ... and critically, this repeats for every command. Whatever bit level was last on MOSI from prior command gets clocked into the other device at the beginning of each command.
To be fair, it probably only impacts the SD Card because it is not operating in SPI mode until switched to it. SD mode doesn't have a CS signal, or equivalent, therefore won't reset its command buffer when CS returns high.
But just that alone can lock the data pins and thereby mess with EEPROM command issuing - A 240 ohm resistor on CMD also would've helped here - But has been capable of corrupting stored SD data as well.
None of these designs have a way to power cycle the SD card with P2 reset. That is going to remain a problem for SD boot (without a change in P2 boot code). In order to perform a soft reset, I had to perform a CMD0 before the soft reset to place the card in a state where to the P2 will boot from SD.
However, if you punch the reset button (or attempt to program the flash)..........
Yep, diagrams are definitely not complete. They are missing the RTC chip as well.
Terry,
I gather that was the ROM booting straight to the SD Card. Not booting the EEPROM, right? Did you figure a workaround at all?
Aaahhh, the dreadful SPI mode 0. Those from 0 to 2 must be outlawed.
Only mode 3 works properly in actual shared bus.
If someone from Parallax is reading here, given the already known problems (and not even linked to the Stamp itself) with this otherwise great idea of sharing the flash and the SD, how hard would it be to change that boot code and intialise the bus in mode 3 ?
technically just a change in the rom mask, practically a rerun to TMC so $120,000 or even more.
Mike
There is no workaround, except for soft booting. If you know you are going to soft boot, you can CMD0 and it will allow it to reboot. While the SD card is mounted, you are out of luck. Power cycling it is the only way to ensure the state of the SD card.
@ke4pjw did you look at supplying extra SDXC clock pulses early in your program, in order to have the SDXC card release its SO line? I only found one SDXC card so far, that didn't respond to that fix.
I think this can mostly fix the boot/reboot issue, not so sure if it can help where the SDXC card is in active use beyond boot though...
I remembered a $5k figure for the rom mask area but wasn't sure what else was needed. Chip mentions it here.
https://forums.parallax.com/discussion/comment/1447243/#Comment_1447243
I wouldn't be surprised if these things are more expensive nowadays though
Actually, on a further thought it seems to me that there shouldn't be a problem with SPI 0 as it is now, and even if both the flash and the SD are getting selected simultaneously when the CLK/CS and CS/CLK and both low. The devices should still behave as expected if not getting clock, and only problematic part there is the MISO which I think could be put through an OR gate, and both the gate inputs weakly pulled up because as far as I know the SD card initialised with its MISO in OD mode. The other one at the flash's side probably isn't, but just in case.
I am currently on my summer travels, but will play more with the idea as soon as I get back, possibly for a new reivision later.
Evidence is this matters when the SD Card is operating in SPI interface mode. I haven't sat down and examined what goes wrong with cmode0 but it really does mess up sharing of the SPI bus. Others have verified that just by flipping the clock polarities their existing routines went from corrupting storage to working brilliant.
EEPROM booting is different though since the SD Card is in the default SD interface mode at that stage. Each EEPROM CS cycle is a clock cycle to the SD Card. If a SD command becomes valid then the Card will respond back on CMD (MOSI). This would fight horribly because the Prop2 will also be driving the same line. What the EEPROM then sees is anyone's guess.
That's my best understanding so far. It may be that booting isn't as bad as I'm imagining. SD booting probably is okay. The EEPROM won't get upset or cause problems as long its MISO pin stays floating while the CS pin is rapidly cycling - Which seems to be true in the datasheets I've read.
Has there been any further progress on this board? Long time quiet.....
I keep watching for an update. Just got excited when I saw the thread pop up
Craig
Hi
me too
Dave
Eek, reading all that, I may have tipped too much cold water on the project.
More recent events have demonstrated shared booting may not be as bad as I was portraying there. The biggest issue to get right, aside from also using cmode3, looks to be always ensuring the releasing of the SD card back to "ready" state during run-time use. This then allows, after a reset, the ROM code to correctly detect the presence of the SD card and boot up as desired.
Or, alternatively, power cycling the SD card when the P2 is reset. Problem here is there is no clear way to signal a reset externally from software. The HUBSET for resetting the Prop2 doesn't affect the RESN pin.
Further reading - https://forums.parallax.com/discussion/comment/1556517/#Comment_1556517