P2 SD Boot Code (v32+)

135

Comments

  • jmgjmg Posts: 13,531
    cgracey wrote: »
    I need to think about how the timing will work. The nice thing about SPI flash is that it can be read in a known amount of time, which works well with the serial window.

    That's true once is comes out of reset.
    However, newer SPI parts are now specifying varying reset-exit times, with Erase/pgm operations taking longer time-to-ready, than read.
    This would mostly affect a WATCHDOG reset, but the BOOT code will need modification to tolerate this.

  • jmg wrote: »
    eg SD can be used for logging and updates, but mission critical code comes from SPI.
    Key difference : The system is still operational with removed/faulty SD.
    Not being catered for here. The system builder will have to sort that out for themselves.

    "... peers into the actual workings of a quantum jump for the first time. The results
    reveal a surprising finding that contradicts Danish physicist Niels Bohr's established view
    —the jumps are neither abrupt nor as random as previously thought."
  • jmg wrote: »
    evanh wrote: »
    Eg: The SD option is not likely to be sharing with another Flash SPI part, it would be just the SD card only. If that SD card is removed then the equipment will not boot until the SD card is reinserted.
    I'm not certain about that 'not likely' part...
    Hobbyists might tolerate SD card insert, and having a dead-duck system with removed/faulty SD, but more serious industrial users are likely to want SD and SPI Flash.
    eg SD can be used for logging and updates, but mission critical code comes from SPI.
    Key difference : The system is still operational with removed/faulty SD.


    Then the SD card should be run on other pins.
  • jmg wrote: »
    cgracey wrote: »
    I need to think about how the timing will work. The nice thing about SPI flash is that it can be read in a known amount of time, which works well with the serial window.

    That's true once is comes out of reset.
    However, newer SPI parts are now specifying varying reset-exit times, with Erase/pgm operations taking longer time-to-ready, than read.
    This would mostly affect a WATCHDOG reset, but the BOOT code will need modification to tolerate this.

    Yes, we will need to review our current strategy. Right now, I just wait for the BUSY flag to go inactive, however long that takes. If it preempts serial, so be it.
  • jmgjmg Posts: 13,531
    cgracey wrote: »
    jmg wrote: »
    cgracey wrote: »
    I need to think about how the timing will work. The nice thing about SPI flash is that it can be read in a known amount of time, which works well with the serial window.

    That's true once is comes out of reset.
    However, newer SPI parts are now specifying varying reset-exit times, with Erase/pgm operations taking longer time-to-ready, than read.
    This would mostly affect a WATCHDOG reset, but the BOOT code will need modification to tolerate this.

    Yes, we will need to review our current strategy. Right now, I just wait for the BUSY flag to go inactive, however long that takes. If it preempts serial, so be it.

    The data is annoyingly vague around reset exit. It says "During this period, no command will be accepted" - sounds like read status pauses, along with everything else. (BUSY floats)

    That means this
    send write-disable command to clear WEL
    send read-status command
    get status
    if WEL high, no SPI memory
    could falsely exit, if done too soon after reset ?

    Better for SPI present test might be a repeated RDID(9FH) with timeout, following the reset command ?
    Then, BUSY can be checked as usual.
    A light pullup, would then also work on code that does
    if BUSY high, wait for erase/write to finish

    Some example times :
    tREADY2  (Macronix MX25LM51245G)
    Reset Recovery time (During instruction decoding) 40 us
    Reset Recovery time (for read operation)          40 us
    Reset Recovery time (for program operation)      310 us
    Reset Recovery time(for SE4KB operation)          12 ms
    Reset Recovery time (for BE64K operation)         25 ms
    Reset Recovery time (for Chip Erase operation)   100 ms
    Reset Recovery time (for WRSR operation)          40 ms
    

    This time does reference from the reset leading edge, so an (ideal) external watchdog can generate a 100ms reset pulse.
    For software resets, that has less control.
  • jmg wrote: »
    evanh wrote: »
    Eg: The SD option is not likely to be sharing with another Flash SPI part, it would be just the SD card only. If that SD card is removed then the equipment will not boot until the SD card is reinserted.
    I'm not certain about that 'not likely' part...
    Hobbyists might tolerate SD card insert, and having a dead-duck system with removed/faulty SD, but more serious industrial users are likely to want SD and SPI Flash.
    eg SD can be used for logging and updates, but mission critical code comes from SPI.
    Key difference : The system is still operational with removed/faulty SD.

    For that configuration, the designer will have the SD on different pins, and it will be his code to boot the SD card. So its up to the designer to figure what happens if the SD card fails.

    In my boot code, I expect that if an SD card is present, and it fails to boot for one of a number of reasons, then fallback will be to serial.

    Chip, timing is NOT predictable with SD cards. It depends on the controllers and they vary greatly between manufacturers and various card versions within manufacturers.
    My Prop boards: P8XBlade2, RamBlade, CpuBlade, TriBlade
    Prop OS (also see Sphinx, PropDos, PropCmd, Spinix)
    Website: www.clusos.com
    Prop Tools (Index) , Emulators (Index) , ZiCog (Z80)
  • Just want to chime in here. What we are doing is very basic, and hopefully won't map over to what I'm seeing with SD cards.

    Basically, all but industrially rated SD cards, which cost a lot of money, are flaky. I've got a product I'm working with right now. It used SD cards in two contexts, one being the SD card runs a Raspi and the other an 8 bit Atmel board.

    A fairly significant percentage, say 10 to 20 percent, get corrupt or just die over a period of months.

    I'm having real trouble getting to root causes. In some cases, it's users removing the cards from the 8 bit board without a unmount type operation happening first so the pins are inactive. That one is known. User needs to respect the card.

    Some are getting corrupted other ways. Not sure what those are. It seems to be related to activity on the cards, which is mostly read on the 8 bit board.

    On the Raspi:

    The Raspi 1 rarely corrupts cards. The 2 does it a lot, and the 3 appears somewhere in between, though it will be a while before I have that data. Lol, it can make more sense to phase out a Raspi 2 than it does to deal with the SD cards.

    These are industrial systems. I'm not at all confident in SD card as primary boot device. It's attractive, but not as solid as it should be. Seems worse than it was a while ago, but that could just be perception bias happening from more machines in general out there now.

    My gut says whatever we do should be ultra conservative. Known, basic, touch the card, mode change, etc... as little as possible, use very permissive timings. These things appear to vary more than it may seem.

    More advanced, faster, push the card, configure the card, type cases may well be best left to post P2 init and boot processes.


    Do not taunt Happy Fun Ball! @opengeekorg ---> Be Excellent To One Another SKYPE = acuity_doug
    Parallax colors simplified: https://forums.parallax.com/discussion/123709/commented-graphics-demo-spin<br>
  • cgraceycgracey Posts: 11,132
    edited 2018-04-11 - 07:26:59
    Interesting observations, Potatohead.

    I wonder where the flakiness is coming from? You'd think a memory card wouldn't be able to die so easily. Maybe because SD cards are such consumer commodities, their quality-control is no better than the $1 phone charger at the Dollar Store, that likely doesn't even work. I think SPI flash chip manufacturers couldn't skate as easily on poor quality, as they'd be detected and there'd be market consequences. Not so for the untraceable junk with meaningless brand names.
  • potatoheadpotatohead Posts: 9,715
    edited 2018-04-11 - 07:49:47
    I think QC has a lot to do with it. In particular, the quality of controller on the card appears to vary quite a lot. I just don't think those are getting validated across a wide range of possible cases.

    Like I said, our use case here is really basic. Maybe just keeping it that way is wise, that's all.

    I don't intend to crap on SD cards in general. But, I am just not seeing as robust of operation as I have prior too.







    Do not taunt Happy Fun Ball! @opengeekorg ---> Be Excellent To One Another SKYPE = acuity_doug
    Parallax colors simplified: https://forums.parallax.com/discussion/123709/commented-graphics-demo-spin<br>
  • While I have not experienced the problem some people have, I mostly buy genuine SanDisk cards from a reputable supplier. However, having said that, I have an eBay cheapie 8GB C10 and a 32GB C10 and both have performed flawlessly. Both get removed and plugged regularly in my P1 and occasionally in my P2 boards.

    Often the cheapies have crap controllers! Also often the cheapies are not the capacity they are being sold as.

    Perhaps handling also has something to do with it too. Easy to get a static charge to hit the card.
    My Prop boards: P8XBlade2, RamBlade, CpuBlade, TriBlade
    Prop OS (also see Sphinx, PropDos, PropCmd, Spinix)
    Website: www.clusos.com
    Prop Tools (Index) , Emulators (Index) , ZiCog (Z80)
  • I'd put operational data corruption separately from card reliability.

    I know I've had many SD cards misbehave/die because of solder breaks on the internal PCB. This is simply a case of the very thin and flimsy board not handling the mechanical treatment I put them through. Same thing happens to protruding USB connectors. I've learnt to protect them in transport now.

    "... peers into the actual workings of a quantum jump for the first time. The results
    reveal a surprising finding that contradicts Danish physicist Niels Bohr's established view
    —the jumps are neither abrupt nor as random as previously thought."
  • TubularTubular Posts: 3,579
    edited 2018-04-11 - 21:36:14
    One answer is to use the write protect tab of the SD card, since we only need to read to boot

    I've been involved with a rasp pi 1 NFC door access system that had a few card corruptions early on. The solution (that's worked well for about 4 years now) was to write-protect the SD card and use a flash drive for output logs etc.

    That may sound limiting and not ideal, but at least it works. Despite this I still think its worth having SD boot if possible.
  • tubular,
    Where is the write protect tab on a microSD?
    My Prop boards: P8XBlade2, RamBlade, CpuBlade, TriBlade
    Prop OS (also see Sphinx, PropDos, PropCmd, Spinix)
    Website: www.clusos.com
    Prop Tools (Index) , Emulators (Index) , ZiCog (Z80)
  • Cluso99Cluso99 Posts: 14,957
    edited 2018-04-12 - 05:45:46
    Just updated my SD Boot code.

    When it checks for the two prenamed/hardcoded FAT files (currently "_P2_BOOT.BIX" and "_P2_BOOT.BIY") I now check for the first one to be in the first 16 directory sectors ie the first 256 entries, and if not found, check again for the second file name.

    If found, then I load the files data into Hub starting at $0_0000 onwards. The length will be the length of the file rounded up to the next 512 byte sector if necessary.

    This means for a 32KB file, I will load 64 sectors of 512 bytes. For a file of say 10 bytes, I will load 1 sector = 512 bytes. I figure this is good enough.

    I don't check for file fragmentation. This is the users responsibility. The usual is 32KB clusters. I don't check for a maximum file size, so you can load the whole 1MB but I am not sure if the top 16KB will be protected, nor wraparound if there is less than 1MB of Hub ram. Again the users responsibility.

    This is working. :)

    BTW I have two files that I load. One is ~96 bytes, and the other is 32KB. They both flash RGB LEDs that I have connected to the P2 pins.
    My Prop boards: P8XBlade2, RamBlade, CpuBlade, TriBlade
    Prop OS (also see Sphinx, PropDos, PropCmd, Spinix)
    Website: www.clusos.com
    Prop Tools (Index) , Emulators (Index) , ZiCog (Z80)
  • When it comes to SD cards, I refuse to take chances. To date, brand-name cards purchased from major brick-and-mortar stores have not let me down. The absence of flaky behavior and lost data has been more than worth the few extra bucks.

    I wouldn't even be upset if Parallax published an approved SD card list that consisted of only one card, one brand, one size, or one vendor.

    Of course I may be alone in this. :)
  • K2 wrote: »
    When it comes to SD cards, I refuse to take chances. To date, brand-name cards purchased from major brick-and-mortar stores have not let me down. The absence of flaky behavior and lost data has been more than worth the few extra bucks.

    I wouldn't even be upset if Parallax published an approved SD card list that consisted of only one card, one brand, one size, or one vendor.

    Of course I may be alone in this. :)

    The only problem with this is that SD card sizes are a moving target. Even within the one size, technologies change.

    Having said that, I have never had a problem with "real" SanDisk" cards. That's what I, and many others, recommend.
    My Prop boards: P8XBlade2, RamBlade, CpuBlade, TriBlade
    Prop OS (also see Sphinx, PropDos, PropCmd, Spinix)
    Website: www.clusos.com
    Prop Tools (Index) , Emulators (Index) , ZiCog (Z80)
  • Yup. None of my RPi boards have ever had anything but a real SanDisk card installed. Same with our cell phones.

    But the only way I know to be sure I'm getting a real SanDisk card is to buy it from a real store. In the wild, it's a bit like buying an Ultrafire Li-ion cell - almost sure to be a fraud.

    You are absolutely right about card capacities! Do SanDisk make a 4GB or 8GB card anymore? For my recent installation of Stretch, 32GB was the smallest card I could find. But it was at the price of the 8GB card I purchased for Wheezy. :)

  • I'm hoping I will get some time now to check all this stuff including your boot code. But I don't have a problem with specifying "works with SanDisk" and if somebody wants to use their 10 year old eBay "genuine" 2GB ol faithful then that is their problem. I also don't have a problem with depending solely upon the card because if it is removed then it is the same as an RPi and last time i checked nobody had stopped buying those for that reason.

    Tachyon Forth - compact, fast, forthwright and interactive
    useforthlogo-s.png
    --->CLICK THE LOGO for more links<---
    P2 +++++ TAQOZ INTRO & LINKS +++++ P2 SHORTFORM DATASHEET
    P1 +++++ Latest binary V5.4 includes EASYFILE +++++ Tachyon Forth News Blog
    Brisbane, Australia
  • Cluso99Cluso99 Posts: 14,957
    edited 2018-04-12 - 08:21:07
    K2 wrote: »
    Yup. None of my RPi boards have ever had anything but a real SanDisk card installed. Same with our cell phones.

    But the only way I know to be sure I'm getting a real SanDisk card is to buy it from a real store. In the wild, it's a bit like buying an Ultrafire Li-ion cell - almost sure to be a fraud.

    You are absolutely right about card capacities! Do SanDisk make a 4GB or 8GB card anymore? For my recent installation of Stretch, 32GB was the smallest card I could find. But it was at the price of the 8GB card I purchased for Wheezy. :)
    16GB can be bought in Oz, but at 2.5x the price of the old 8GB :(
    My Prop boards: P8XBlade2, RamBlade, CpuBlade, TriBlade
    Prop OS (also see Sphinx, PropDos, PropCmd, Spinix)
    Website: www.clusos.com
    Prop Tools (Index) , Emulators (Index) , ZiCog (Z80)
  • Cluso99 wrote: »
    K2 wrote: »
    Yup. None of my RPi boards have ever had anything but a real SanDisk card installed. Same with our cell phones.

    But the only way I know to be sure I'm getting a real SanDisk card is to buy it from a real store. In the wild, it's a bit like buying an Ultrafire Li-ion cell - almost sure to be a fraud.

    You are absolutely right about card capacities! Do SanDisk make a 4GB or 8GB card anymore? For my recent installation of Stretch, 32GB was the smallest card I could find. But it was at the price of the 8GB card I purchased for Wheezy. :)
    16GB can be bought in Oz, but at 2.5x the price of the old 8GB :(

    Somebody ripping you off there for sure.
    In science there is no authority. There is only experiment.
    Life is unpredictable. Eat dessert first.
  • msrobotsmsrobots Posts: 2,540
    edited 2018-04-13 - 04:47:55
    @Cluso99,

    this is very good news. So your 'directory' loader, looks for two possible files to load, loads up to 1MB and starts COG 0 in HUBexec at address 0? Or does it load something into the COG and starts in COGexec?

    As far as I understand Flash loads a COG and goes from there, your MBR code loads into HUB copies to COG and starts there, and the serial commands Prop_Txt and Prop_Hex load the complete HUB ram and start in HUBexec at Address 0.

    If so, the Flash loader and your MBR loader will need to call some second stage loader to load a complete HUB image, but your directory loader does not.

    This is a perfectly fine solution. Thank you!

    and the MBR loader MAY work even with other file systems as FAT32. Wonderful.

    Enjoy!

    Mike
    I am just another Code Monkey.
    A determined coder can write COBOL programs in any language. -- Author unknown.
    Press any key to continue, any other key to quit

    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this post are to be interpreted as described in RFC 2119.
  • msrobots wrote: »
    @Cluso99,

    this is very good news. So your 'directory' loader, looks for two possible files to load, loads up to 1MB and starts COG 0 in HUBexec at address 0? Or does it load something into the COG and starts in COGexec?
    After loading into hub $0 upwards, the bottom 2KB (496 longs) are copied to Cog 0 at $000-$1EF and then a JMP $000 (Cog) so its cog code executing.
    FWIW hubexec cannot run from hub $00000-$00FFF (think thats the limit ie bottom 4KB) due to mapping the program counter for cog and lut with hub.
    As far as I understand Flash loads a COG and goes from there, your MBR code loads into HUB copies to COG and starts there, and the serial commands Prop_Txt and Prop_Hex load the complete HUB ram and start in HUBexec at Address 0.
    No. 1KB Flash is loaded into hub $00000 upwards by code running in Cog 0, then the code jumps to lut and that then copies that 1KB from hub $00000+ to Cog $000+ and then JMP $000 (cog 0).
    If so, the Flash loader and your MBR loader will need to call some second stage loader to load a complete HUB image, but your directory loader does not.

    This is a perfectly fine solution. Thank you!

    and the MBR loader MAY work even with other file systems as FAT32. Wonderful.

    Enjoy!

    Mike

    My Prop boards: P8XBlade2, RamBlade, CpuBlade, TriBlade
    Prop OS (also see Sphinx, PropDos, PropCmd, Spinix)
    Website: www.clusos.com
    Prop Tools (Index) , Emulators (Index) , ZiCog (Z80)
  • TorTor Posts: 1,975
    Tubular wrote: »
    One answer is to use the write protect tab of the SD card, since we only need to read to boot
    Unfortunately that won't protect against corruption. The write protect tab on SD cards isn't a physical write protect, it's just setting one of the bits that the software can read, and then decide (and most actually don't) that 'Ok, user doesn't want me to write.'
    So this would do very little for accidental corruption.
  • Cluso99 wrote: »
    FWIW hubexec cannot run from hub $00000-$00FFF (think thats the limit ie bottom 4KB) due to mapping the program counter for cog and lut with hub.
    Hubexec can operate from $400+.

    Melbourne, Australia
  • Tor wrote: »
    Tubular wrote: »
    One answer is to use the write protect tab of the SD card, since we only need to read to boot
    Unfortunately that won't protect against corruption. The write protect tab on SD cards isn't a physical write protect, it's just setting one of the bits that the software can read, and then decide (and most actually don't) that 'Ok, user doesn't want me to write.'
    So this would do very little for accidental corruption.

    I did a really bad job with by previous post, it was posted in response to Potatohead's experiences with the Pi. Let me put it differently

    - I think SD card booting can be reliable if the SD card is only read by the P2, never written.
    - This is based on experience with a Rasp Pi 1 project that has been running continuously since early 2013.
    - We had a lot of corruptions during the first few months of that project, before switching the card to read only, adding a usb flash drive for file output, and had no SD issues since
    - I believe we can use the SD card the same way on the P2 reliably. ie strictly read commands only.
    - This is still very useful as it lets field upgrades be achieved by end users relatively easily

    Those feeling more adventurous could try writing to the SD card from the P2, and see how bad it really is. Perhaps fixed length write commands (single byte etc), or power supply shutdown management might help limit the damage

  • Potatohead's experiences with the Pi. Let me put it differently

    - I think SD card booting can be reliable if the SD card is only read by the P2, never written.
    - This is based on experience with a Rasp Pi 1 project that has been running continuously since early 2013.
    - We had a lot of corruptions during the first few months of that project, before switching the card to read only, adding a usb flash drive for file output, and had no SD issues since

    Thanks for that. Appreciated, and something to explore in my troublesome scenario. It's not too many cards, but enough to really bother me.

    Re: real SanDisk

    Yeah, I've no failures on those that aren't "user touched card." That one does not concern me. Happens. Just work through it, find out why, and then get them somewhere they don't have to mess with the card.



    Do not taunt Happy Fun Ball! @opengeekorg ---> Be Excellent To One Another SKYPE = acuity_doug
    Parallax colors simplified: https://forums.parallax.com/discussion/123709/commented-graphics-demo-spin<br>
  • cgraceycgracey Posts: 11,132
    edited 2018-04-13 - 15:10:27
    Interesting. It would stand to reason that a card treated as read-only in an embedded system would not fail, but those that were being modified and maybe powered down at the wrong time suffered corruption that disabled the system.

    So, the SD used for booting should remain unmodified, while a secondary SD could be used for data.
  • Or just a second partition would do the same job.
    "... peers into the actual workings of a quantum jump for the first time. The results
    reveal a surprising finding that contradicts Danish physicist Niels Bohr's established view
    —the jumps are neither abrupt nor as random as previously thought."
  • evanh wrote: »
    Or just a second partition would do the same job.

    Are you sure that would be sufficient isolation?
  • cgracey wrote: »
    Are you sure that would be sufficient isolation?
    Yep, it's file corruption from lagging buffer writes that is being addressed. Not electrical/mechanical damage. They only have to be a separate volumes. Even the rescue tools can be on the boot volume, ready to fix up a corrupted data volume.

    "... peers into the actual workings of a quantum jump for the first time. The results
    reveal a surprising finding that contradicts Danish physicist Niels Bohr's established view
    —the jumps are neither abrupt nor as random as previously thought."
Sign In or Register to comment.