Shop OBEX P1 Docs P2 Docs Learn Events
Propeller II - Page 16 — Parallax Forums

Propeller II

1131416181945

Comments

  • pedwardpedward Posts: 1,642
    edited 2012-08-13 15:26
    Few points:

    *An SD boot is of no practical use if it isn't dead simple to use.
    *Not all SD cards behave the same, right now we have 3 SD interfaces and at least 1 won't read from Lexar cards on boot. I still need to stick this on the Logic analyzer to inspect.
    *512 bytes of a boot sector is only 128 longs, will a 128 long overlay get you FAT support?
    *DOS had problems with loading io.sys and msdos.sys if you started moving them around. They had to be in the first 20 file entries IIRC. Also, fragmentation plays a big role.

    The chief advantages of having SD are to do simple field upgrades of firmware (goes back to point 1) and extra storage space.

    SPI flash is very inexpensive, in qty 1 it's 44 cents for a 64KB, in large quantities like was used in the strawman argument, it would approach 10 cents for a million units.

    I guess at this point I would say this: Would you like the existing 126KB of memory or 124KB of memory as a tradeoff to support SD boot?

    Kye said very clearly that just the code to TALK to an SD device is half a COG, there isn't that much space left. So, you are at a point where more ROM code space is
    necessary to support SD boot, and that can either come from external SPI flash or from Hub RAM.

    Most of the people making the fervent arguments for SD boot are comparing it to the RPi, which is another strawman argument. I also see hobbyists that will probably stick to the P1 because the P2 is too expensive and too powerful for their needs.

    Now we are left with professional developers who will create embedded devices, many without an SD card present, and some with optional SD support.

    An 8Mbit flash chip is really cheap and can store a ton of code, is faster than SD, and you don't need to worry about a filesystem getting corrupted or the highly variable quality of the storage medium.

    ARM consumer devices use eMMC and SD because it's easy to source and because all of their binaries are giant pigs and need cheap huge storage. Since the Prop2 actually shrinks code size by 30% because of additional instructions, it's actually much less dependent on external program storage and is much more efficient.
  • SapiehaSapieha Posts: 2,964
    edited 2012-08-13 15:53
    Hi Chip.

    MBR can't be used as program storage ---->
    READ it to CLUT
    Test its checksum give info if SD present and valid
    NEXT from it find Partition table position from its Drive parameter Table.
    READ it to CLUT and find starting point of Partition You will BOOT from (Usualy first one)--- Then by skipping FAT table find position of first DIR sector
    READ it ti CLUT and find High level loader/OS ---- Parallax.prg <---name need be in 8+3 name format
    LOAD it to HUB and start
    ALL done

    Parallax.prg this ----> This part is comunity and profesional users mostly writ by them selves

    Only criteria as It can READ that file <
    This is NOT Yours work
    cgracey wrote: »
    Okay. Let's do this, then.

    Does anybody have any quick pointers to what I need to learn?

    How much memory could we expect to consistently be available to us in the MBR?
  • SapiehaSapieha Posts: 2,964
    edited 2012-08-13 15:54
    Hi Chip.

    Sequence looks very good !!
    cgracey wrote: »
    On power up, the Prop II would do the following:

    1) Attempt to boot from serial RX/TX
    2) Attempt to boot from SPI flash
    3) Attempt to boot from SD card
    4) If none of the above validated, shut down and wait for another reset
  • cgraceycgracey Posts: 14,133
    edited 2012-08-13 16:09
    Pedward,

    The main thing I see in favor of supporting direct SD booting is that there would be NO intermediate memory to introduce any variability into how the SD card would work. It gets rid of a layer of complexity and uncertainty. Of course, this only makes sense if it is possible to do easily from ROM.

    It would be nice to be able to completely change the personality of a Prop II device by just changing an SD card and never worrying about what version of firmware is in an SPI flash. Systems deployed this way would never even need RX/TX serial to (re)configure the flash! That gets rid of TWO sets of parts.
  • Roy ElthamRoy Eltham Posts: 3,000
    edited 2012-08-13 16:13
    Doing something funky with the MBR is going to require a special program to put the boot code onto the SD card. If you're going to require that, then why not just make up a simple file system format? Then booting can be made trivial, and code to read files from the SD card will be easy also. The downside is that you need a special program to read/write the sd card on a PC (win/mac/linux), but you need that already if you do anything non-standard (which is what all these solutions here are).

    Roy
  • cgraceycgracey Posts: 14,133
    edited 2012-08-13 16:16
    So, it would take a couple of KB of ROM to do this all the way correctly? It might be worth doing.
  • markaericmarkaeric Posts: 282
    edited 2012-08-13 16:24
    @Chip

    Like pedward said, there is <512 Bytes in the MBR. If you try to utilize this space you would either have to fit a FAT engine (probably unlikely) or a map to the boot file location, which would require a special utility to write this information to the SD. Both these options seem unnecessarily complex and probably prone to more issues than the method I previously suggested. Please consider it as I really believe that it's overall the simplest, and the most likely to fit in the tiny bit of ROM space you have available. If you missed my previous post, here's the gist:

    1) Propeller binary contiguously loaded at the very end of SD card memory - it shouldn't matter what file system card it formatted with (Requires special utility to do this - all practical methods will)
    2) On P2 boot up, if no serial comm, or SPI flash located, initialize SD card
    3) Read SD card special registers to determine memory capacity
    4) Jump to an offset from the last memory address and copy P2 binary into hub ram
    5) Presto!

    :)
  • jmgjmg Posts: 15,157
    edited 2012-08-13 16:26
    cgracey wrote: »
    On power up, the Prop II would do the following:

    1) Attempt to boot from serial RX/TX
    2) Attempt to boot from SPI flash
    3) Attempt to boot from SD card
    4) If none of the above validated, shut down and wait for another reset


    How long do these decisions take ?
    Reset-to-alive time will matter in some applications.
  • average joeaverage joe Posts: 795
    edited 2012-08-13 16:36
    IF it's done, I would like to see it as a simple file system. Roy has made very good points. I'm not an expert on SD cards but I have had issues that would frustrate most hobbyists. I've had cards that work with FSRW and not Kye's code, and the other way around.
    My BIGGEST concern is: IF the SD is the only boot device and there is "something" wrong with the SD card... How do we troubleshoot? Kye's driver aborts with errors which is very nice. FSRW has a similar mechanism. But these require code to be running to trap the abort. So, it seems like SD booting either works, or it doesn't and there's no way to diagnose.
  • Roy ElthamRoy Eltham Posts: 3,000
    edited 2012-08-13 16:37
    markaeric,
    Your solution requires a special program be written for Windows/Mac/Linux/etc. to be able to write the boot data to the end of the SD card. That's less than ideal. In my opinion, anything that requires special programs to put the Ppop2 boot data onto the card is just not a good solution.
  • David BetzDavid Betz Posts: 14,516
    edited 2012-08-13 16:41
    cgracey wrote: »
    So, it would take a couple of KB of ROM to do this all the way correctly? It might be worth doing.
    I'd hate to reduce hub memory to 124k just to get SD card booting. Is this really a good tradeoff?
  • KyeKye Posts: 2,200
    edited 2012-08-13 16:48
    Does anyone bother reading the FSRW or FATEngine ASM code for talking to an SD card?

    The protocol is VERY complex. It is not simple at all.

    ...

    I understand the desire to have boot capability from the SD card. But, the end result is that even more pins will need to be used by the boot loader statically. Right now the boot loader takes 6 pins. You'll need 7 for SD card boot. You can't share the CS pin between a SPI flash chip and an optional SD card. This will cause problems.

    Additionally, for anyone who wants real performance out of the P2 and wants to make a competitive product you'll not want to share the SPI bus between the SPI flash chip and the SD card. If you want to use external flash to store code you'll most likely want to have either quadSPI or octalSPI action going on attached to the P2.

    If you don't you will make cache misses very expensive for the C LMM machine (or flexible SPIN interpreter).

    SPI mode for SD cards do not support clock rates faster than 25 MHz. This is on purpose by the SD card association to make the SPI bus not suitable for commercial products. With quadSPI or octalSPI parts you can drive the clock up to 100 MHz.

    ...

    It sounds good to have SD card boot. But, the devil is in the details. Ignoring the fact that the SD card boot loader will not likely work with every SD card is not acceptable to build into the hardware. Building SD card boot into the hardware should only be done if Parallax plans to license the SD card protocol from the SD card association and do it right.

    And again... most people who want the P2 for speed and performance will not share the SD card with the Flash bus. The performance you can get out of Octal SPI will run circles around an SD card.

    ...

    Did you know that an over-clocked to 100 MHz FSRW driver using the fastest block driver on the P1 hits about the maximum write speed in RAW mode that you can achieve on an SD card. Do you want to be stuck with sub 2 MBs speed limits? With 100 MHz octalSPI flash you can hit 100 MBs a second.

    The only way to hit the really fast read and write speeds on an SD card is to have sequential reads and writes (not random reads and writes... which is what is needed for a execute in place setup), use 1.8V signaling and license the SD card spec form the SD card association.

    SPI flash memory is actually designed for execute in place (XIP) performance. This means random reads on SPI flash memory are designed to be fast.

    ...

    *An octal SPI flash setup is where you have 2 quadSPI chips hooked up. This setup cost a $1 for about 16 MB of space. An SD card will likely cost you more money. Why pay for a 8 GB SD card when you do not need one and it will not be as fast?*

    ...

    Sorry, for sounding angry in this post. I'm just a bit upset that many people seems to ignore the elephant in the room. Please look at the SD card init code in either the FAT Engine or FSRW and see how complex it is. Then understand that that neither the FATEngine or FSRW is perfect and the P2 needs to be.

    This means Chip will have to devote a large portion of his time testing every SD card he can find under the sun for compatibility.
  • cgraceycgracey Posts: 14,133
    edited 2012-08-13 16:48
    jmg wrote: »
    How long do these decisions take ?
    Reset-to-alive time will matter in some applications.

    Serial takes about 150ms to determine no-go.

    SPI flash takes about 5ms to determine no-go.

    So, at ~155ms from reset, we could start checking SD. There's nothing after that - just shutdown, which nobody will ever perceive, anyway.
  • RaymanRayman Posts: 14,319
    edited 2012-08-13 16:50
    I've seen products that use just raw block mode of SD. That should be easy enough to integrate into bootloader.
    This seems a whole lot easier than these other approaches...

    Also, I've seen some android tablets that somehow integrate an internal SD card... Still wondering how they do that, exactly...
  • markaericmarkaeric Posts: 282
    edited 2012-08-13 16:56
    Roy Eltham wrote: »
    markaeric,
    Your solution requires a special program be written for Windows/Mac/Linux/etc. to be able to write the boot data to the end of the SD card. That's less than ideal. In my opinion, anything that requires special programs to put the Ppop2 boot data onto the card is just not a good solution.

    Unless you can implement a robust file system support (and you'll probably only be able to support one) then there really is no other option. Placing any kind of code in the MBR will also require a special utility. At any rate, there's a good chance such utilities exist. For example, there is a utility for Windows which places a file in the first file location - almost everyone that has an Android device that can boot from SD uses this as this is where the Linux kernel is expected to reside. I'm not sure if there are similar open source utilities, but I would be surprised if there weren't. Even if there are not, I think creating such a utility would be quite trivial for many of the brainiacs around here. I'm going to search for such a utility and report back.
  • jmgjmg Posts: 15,157
    edited 2012-08-13 17:04
    Kye wrote: »
    With quadSPI or octalSPI parts you can drive the clock up to 100 MHz.

    On this topic, I see Microchip has just released QuadSPI SRAM, in 512k and 1024k bits.

    Two of the Microchip parts, 2 will give 256K bytes of OctalSPI, and they have what they call NVSRAM variant, which is really A VBat pin. I think that drops from Quad SPI, to double SPI to fit into 8pins.

    There is a minor caveat, which is the SPI FLASH parts can go to over 100MHz, but these SRAMs have a 20MHz CLK max.
    I guess a de-selected SRAM will be OK with 100MHz CLKs ?


    This again raises the issue of native silicon support for QuadIO, which is rapidly becoming a standard, and easy to implement.
  • SapiehaSapieha Posts: 2,964
    edited 2012-08-13 17:07
    Other alternative are to use --- Unused space between BOOT sector and first FAT that is at least 30 sectors of RAW data area.
    Ad is simple to find ---> first byte after BOOT sectors Checksum. about 30x2KB space.

    That don't infer Any OS system -- simple to find to. And big enough to have High level language system to BOOT- RUN from

    BUT that needs special loader for it.
  • SapiehaSapieha Posts: 2,964
    edited 2012-08-13 17:10
    Hi Kye.

    IF Flash boot shall only support 4 bits flach IC -- very BAD.

    Kye wrote: »
    Does anyone bother reading the FSRW or FATEngine ASM code for talking to an SD card?

    The protocol is VERY complex. It is not simple at all.

    ...

    I understand the desire to have boot capability from the SD card. But, the end result is that even more pins will need to be used by the boot loader statically. Right now the boot loader takes 6 pins. You'll need 7 for SD card boot. You can't share the CS pin between a SPI flash chip and an optional SD card. This will cause problems.

    Additionally, for anyone who wants real performance out of the P2 and wants to make a competitive product you'll not want to share the SPI bus between the SPI flash chip and the SD card. If you want to use external flash to store code you'll most likely want to have either quadSPI or octalSPI action going on attached to the P2.

    If you don't you will make cache misses very expensive for the C LMM machine (or flexible SPIN interpreter).

    SPI mode for SD cards do not support clock rates faster than 25 MHz. This is on purpose by the SD card association to make the SPI bus not suitable for commercial products. With quadSPI or octalSPI parts you can drive the clock up to 100 MHz.

    ...

    It sounds good to have SD card boot. But, the devil is in the details. Ignoring the fact that the SD card boot loader will not likely work with every SD card is not acceptable to build into the hardware. Building SD card boot into the hardware should only be done if Parallax plans to license the SD card protocol from the SD card association and do it right.

    And again... most people who want the P2 for speed and performance will not share the SD card with the Flash bus. The performance you can get out of Octal SPI will run circles around an SD card.

    ...

    Did you know that an over-clocked to 100 MHz FSRW driver using the fastest block driver on the P1 hits about the maximum write speed in RAW mode that you can achieve on an SD card. Do you want to be stuck with sub 2 MBs speed limits? With 100 MHz octalSPI flash you can hit 100 MBs a second.

    The only way to hit the really fast read and write speeds on an SD card is to have sequential reads and writes (not random reads and writes... which is what is needed for a execute in place setup), use 1.8V signaling and license the SD card spec form the SD card association.

    SPI flash memory is actually designed for execute in place (XIP) performance. This means random reads on SPI flash memory are designed to be fast.

    ...

    *An octal SPI flash setup is where you have 2 quadSPI chips hooked up. This setup cost a $1 for about 16 MB of space. An SD card will likely cost you more money. Why pay for a 8 GB SD card when you do not need one and it will not be as fast?*

    ...

    Sorry, for sounding angry in this post. I'm just a bit upset that many people seems to ignore the elephant in the room. Please look at the SD card init code in either the FAT Engine or FSRW and see how complex it is. Then understand that that neither the FATEngine or FSRW is perfect and the P2 needs to be.

    This means Chip will have to devote a large portion of his time testing every SD card he can find under the sun for compatibility.
  • markaericmarkaeric Posts: 282
    edited 2012-08-13 17:15
    I do not doubt that as far as speed is concerned, the SPI flash chips have a significant advantage over SD cards, but that doesn't negate the value of being able to boot from SD. In no way am I suggesting that SPI flash support be dropped in favor of SD, though. All I'm proposing is that Parallax give it real consideration, especially if it can be squeezed into the remaining ROM space. Is that so unfair to ask?
  • cgraceycgracey Posts: 14,133
    edited 2012-08-13 17:15
    Kye wrote: »
    Does anyone bother reading the FSRW or FATEngine ASM code for talking to an SD card?

    The protocol is VERY complex. It is not simple at all.

    ...

    I understand the desire to have boot capability from the SD card. But, the end result is that even more pins will need to be used by the boot loader statically. Right now the boot loader takes 6 pins. You'll need 7 for SD card boot. You can't share the CS pin between a SPI flash chip and an optional SD card. This will cause problems.

    Additionally, for anyone who wants real performance out of the P2 and wants to make a competitive product you'll not want to share the SPI bus between the SPI flash chip and the SD card. If you want to use external flash to store code you'll most likely want to have either quadSPI or octalSPI action going on attached to the P2.

    If you don't you will make cache misses very expensive for the C LMM machine (or flexible SPIN interpreter).

    SPI mode for SD cards do not support clock rates faster than 25 MHz. This is on purpose by the SD card association to make the SPI bus not suitable for commercial products. With quadSPI or octalSPI parts you can drive the clock up to 100 MHz.

    ...

    It sounds good to have SD card boot. But, the devil is in the details. Ignoring the fact that the SD card boot loader will not likely work with every SD card is not acceptable to build into the hardware. Building SD card boot into the hardware should only be done if Parallax plans to license the SD card protocol from the SD card association and do it right.

    And again... most people who want the P2 for speed and performance will not share the SD card with the Flash bus. The performance you can get out of Octal SPI will run circles around an SD card.

    ...

    Did you know that an over-clocked to 100 MHz FSRW driver using the fastest block driver on the P1 hits about the maximum write speed in RAW mode that you can achieve on an SD card. Do you want to be stuck with sub 2 MBs speed limits? With 100 MHz octalSPI flash you can hit 100 MBs a second.

    The only way to hit the really fast read and write speeds on an SD card is to have sequential reads and writes (not random reads and writes... which is what is needed for a execute in place setup), use 1.8V signaling and license the SD card spec form the SD card association.

    SPI flash memory is actually designed for execute in place (XIP) performance. This means random reads on SPI flash memory are designed to be fast.

    ...

    *An octal SPI flash setup is where you have 2 quadSPI chips hooked up. This setup cost a $1 for about 16 MB of space. An SD card will likely cost you more money. Why pay for a 8 GB SD card when you do not need one and it will not be as fast?*

    ...

    Sorry, for sounding angry in this post. I'm just a bit upset that many people seems to ignore the elephant in the room. Please look at the SD card init code in either the FAT Engine or FSRW and see how complex it is. Then understand that that neither the FATEngine or FSRW is perfect and the P2 needs to be.

    This means Chip will have to devote a large portion of his time testing every SD card he can find under the sun for compatibility.

    It sounds like the SD card manufacturers don't want us playing unless we pay. And we can't signal that fast at 1.8V, anyway, with those 3.3V transistors.

    I understand that for performance, Quad-bit SPI chips soldered onto the PCB are the way to go.

    The attractive thing about SD cards is their removability and interoperability with other computers. It sounds unrealistic for us to do properly, though.
  • SapiehaSapieha Posts: 2,964
    edited 2012-08-13 17:18
    Hi markaeric.

    Same from my side --- More to ad is that as it give Professional system builders more alternatives.
    markaeric wrote: »
    I do not doubt that as far as speed is concerned, the SPI flash chips have a significant advantage over SD cards, but that doesn't negate the value of being able to boot from SD. In no way am I suggesting that SPI flash support be dropped in favor of SD, though. All I'm proposing is that Parallax give it real consideration, especially if it can be squeezed into the remaining ROM space. Is that so unfair to ask?
  • cgraceycgracey Posts: 14,133
    edited 2012-08-13 17:22
    Sapieha wrote: »
    Hi Kye.

    IF Flash boot shall only support 4 bits flach IC -- very BAD.

    The Flash booter supports 1-bit devices, but after booting, the chip can be reconfigured for 4-bit operations.
  • SapiehaSapieha Posts: 2,964
    edited 2012-08-13 17:25
    Hi Chip.

    That NICE

    One more question --- How about length of data to load
    > Is it flexible ?

    cgracey wrote: »
    The Flash booter supports 1-bit devices, but after booting, the chips can be reconfigured for 4-bit operations.
  • cgraceycgracey Posts: 14,133
    edited 2012-08-13 17:33
    Sapieha wrote: »
    Hi Chip.

    That NICE

    One more question --- How about length of data to load
    > Is it flexible ?

    Right now, it just loads one cog's worth of data. This way, it's quick to load AND authenticate. Once running, the user's loader can turn on the crystal and PLL, switch over to 160MHz, and then load and decrypt the rest of the code very quickly.
  • SapiehaSapieha Posts: 2,964
    edited 2012-08-13 17:38
    Hi Chip.

    That I can live with ---> BUT if it was me .
    I have be used other way ----> first Long counter of Bytes to load

    cgracey wrote: »
    Right now, it just loads one cog's worth of data. This way, it's quick to load AND authenticate. Once running, the user's loader can turn on the crystal and PLL, switch over to 160MHz, and then load and decrypt the rest of the code very quickly.
  • AntoineDoinelAntoineDoinel Posts: 312
    edited 2012-08-13 17:39
    Somebody already suggested linear scanning the SD card in RAW mode (Cluso?).
    After thinking about it, it seems to me that the simplest method could somewhat encompass all the others. And without bothering with FAT at all.

    Given code to read SPI bytes already in place, it should take only a few SD commands:
    1. Perform initialization (send wakeup clocks, set up SPI mode, wait for card response).
    2. Linearly read in one RAW sector at a time, until we find a specific signature, let's say one long equal to 0x43484950 :lol:
    3. From now on, we read in up to the whole HUB image.

    The important bit is to put the signature in a place compatible with both the MBR structure and a plain file (as data, at fixed offset, and jumping over it in the code).

    So let's see how different cases would be handled:

    Case 1: we found the signature immediately (in the first sector)
    * 1.1: it's a RAW card, so reading 128KB gives a full HUB image.
    * 1.2: it's the MBR of a FAT card, in this case the "minimalist" users can try to squezee some code in the 110 longs available, supposed to do further steps to locate the real boot image. The sector following are read in, but ignored since that small code never references it.

    Case 2: we found the signature after the MBR, but before the start of the file area
    * 2.1: There should be up to ~60 reserved sectors are available there, right? That's where the second stage of GRUB usually lives, I guess.
    * 2.2: Maybe we just hit the boot sector of the 1st partition, and again this is exactly like the MBR case.

    Case 3: we found the signature in the file area (but we're not concerned with "files" at all now, only when creating the card!)
    * 3.1: a 32KB per sector FAT32 formatting guarantees at least 32KB contiguous, so if we use a boot file <32KB tha's enough for a 2nd stage bootloader, even if the rest of what we read in is garbage from the files following our one.
    * 3.2: with the only requirement that we wrote a boot image on a freshly formatted FAT card, the whole 128KB image is valid, and it's all good.

    So the load sequence is unique, but how to use it is left to firmware programmers or final users.
    When to give up? Obviously if the card does not responds to initialization, and also after a reasonable number of sectors have been scanned (512? more?).

    Obviously the startup time will be a bit slower and less predictable than a flash ROM, so users needing fast start will be required to use the SPI flash anyway.

    For the SD boot, it would be convenient to give some indication of failure, without touching pins that aren't already part of the boot process. Since the SD is is the last resort, so we are not delaying anything else, maybe CS could be flashed a few times to signal boot failure, before entering sleep?

    About card differences (and customer support):
    Parallax would simply test the thing with two major and well behaving brands (i.e. 1. SanDisk, 2. Lexar, Trascend or whatever), certify only those specific brands and sizes, and leave to the community the task of building a big list of "unofficially" working cards.
  • cgraceycgracey Posts: 14,133
    edited 2012-08-13 17:43
    Sapieha wrote: »
    Hi Chip.

    That I can live with ---> BUT if it was me .
    I have be used other way ----> first Long counter of Bytes to load

    By just loading $1F8 longs, then an 8-long HMAC signature, you are dealing with a known quantity that will take an exact amount of time to process, and will provide enough code to start anything else (except SD card FAT).
  • RaymanRayman Posts: 14,319
    edited 2012-08-13 17:51
    cgracey wrote: »
    The Flash booter supports 1-bit devices, but after booting, the chip can be reconfigured for 4-bit operations.

    The SQI chips I know about don't support both 1-bit and 4-bit data transfer, just 4-bit data transfer....
  • David BetzDavid Betz Posts: 14,516
    edited 2012-08-13 17:56
    Rayman wrote: »
    The SQI chips I know about don't support both 1-bit and 4-bit data transfer, just 4-bit data transfer....
    I think you'll find that they support 1 bit reads but sometimes not writes. We don't care about writes for booting.
  • KyeKye Posts: 2,200
    edited 2012-08-13 17:59
    All quadSPI chips support 1-bit and 4-bit transfer (maybe 2-bit). These chips startup natively in 1-bit mode and require a write to the status register to change to 4-bit mode.

    I spent a day researching every SPI flash chip on the market to confirm this.

    ...

    So many datasheets...
Sign In or Register to comment.