For boot, I would like to support SPI SD over 4 pins, total. If someone wants to use 6-pin mode, they'd need to wire up additional pins. So, for now, can we get this 4-pin thing going? What do we do with CS, CLK, DI, and DO to read in boot code, reliably? I think you said you could do this in 1KB. Does that mean only 256 instructions? If so, let's get it into the ROM!
For boot, I would like to support SPI SD over 4 pins, total. If someone wants to use 6-pin mode, they'd need to wire up additional pins. So, for now, can we get this 4-pin thing going? What do we do with CS, CLK, DI, and DO to read in boot code, reliably? I think you said you could do this in 1KB. Does that mean only 256 instructions? If so, let's get it into the ROM!
I will get to it after the w/e.
Yes, I expect <256 instructions if my calls are correct. Plus of course the sector buffer of 512 bytes, but we will likely load the sectors into cog or hub for execution.
On the MBR, there are 4 Partition blocks of 16 bytes each, starting $1BE, $1CE, $1DE and $1EE.
Below this is usable for booting, so I intend to use the 16 bytes immediately below these, at $1AE(16).
I will checksum the 16 bytes to be "Prop" as you have done with the downloaded.
One long will store the "raw sector address" of the first block to download, followed by a long for the "number of sectors to read". There is space for two of these if we want.
You might get away with reverse order data on Flash, although as Andy said, you cannot mix 1-bit and 4-bit reading and writing or you will get incorrect data.
If you are pre-programming the flash, then there is one more thing to go wrong.
Not a good choice IMHO. Just use a REV instruction.
However, with SD Card, you must have the data correctly stored, or otherwise you will not be able to read/write the card in another system, such as a pc.
But I can live with using REV to reverse the bit order.
I really want to persue the faster SD 4-bit mode. Thing is, self hosting (ie compiling) needs faster SD access if we can get it. It's a simple matter to permit both 4 & 6 pin SD boot options. I can write this in the time required.
So what boot options in my diagram above are out ???
How will we handle FLASH that passes checksum but has a bug preventing operation ?
I am presuming that we bypassed Serial download to make bootup quicker.
A header and jumper on the trace to nCS?
If FLASH gets corrupted, pull the jumper, boot from serial, install the jumper and re-program?
My boards are too small for standard jumpers. It's also a height issue.
I have used 0.5" jumpers but they are expensive.
Anyway, I don't expect to use FLASH as I will be using just an SD CARD.
I'm tending to shy away from all this massive clutter of "got to haves" but I will say this. I just know what's practical and what I would actually use in real product vs what I see are armchair academic points. SPI 1-bit mode is all I need to boot, either from Flash or SD, with the simple shared 4 I/O mode that Chip proposes. I can't see a product that somehow "needs" quad SPI because if speed were important than so would memory requirements, and in which case I would be using SD. As for 4-bit SD mode that is still a licensed protected mode ("Secure" Digital) and although hackable, it is unexplored territory and unproven for our use. Having 1-bit SPI SD mode for boot will not preclude somehow using 4-bit SD mode for second stage boot or applications when it is later proven.
On the MBR, there are 4 Partition blocks of 16 bytes each, starting $1BE, $1CE, $1DE and $1EE.
Below this is usable for booting, so I intend to use the 16 bytes immediately below these, at $1AE(16).
I will checksum the 16 bytes to be "Prop" as you have done with the downloaded.
One long will store the "raw sector address" of the first block to download, followed by a long for the "number of sectors to read". There is space for two of these if we want.
Cluso99,
The partition entries already have those two fields, "raw sector address" and "number of sectors to read", in the last two longs. For the first partition entry those fields are $1C6, and $1CA. They are called "Relative Sectors", and "Total sectors".
Here go my two cents (valued at $0.01 each proposal):
Why just not check in the first partition entry if it has a Propeller Partition ID (byte $xx at $1C2), and then use those already-assigned-and-standard fields $1C6 and $1CA for boot? As a COG only have 512 longs, two Partition IDs can be defined: one to load sectors into COG, and another partition ID can be used to load sectors both on HUB and COG0 (here we can only copy up to 4 sectors). Or only one partition ID, if the number of sectors is higher than 4 then load sectors both in COG and HUB.
Second proposal: (because maybe we don't want to mess with partition IDs) Define a new bootable signature. Instead of $55, $AA (at the last two bytes of the MBR) we can use the ascii string 'P2' or '2P'. As before, there will be two options: with the 'P2' boot signature, the boot proccess will load into COG0 the number of sectors (up to 4) defined in $1CA starting at "Relative Sectors" ($1C6). And with the boot signature '2P' it will load into HUB the number of sectors (up to max HUB size, now 512KB?) defined in $1CA starting at "Relative Sectors" ($1C6), and also the first sectors in COG0. Or just use one boot signature 'P2' that loads the first 4 sectors into COG0, and if the number of sectors is higher than 4 then load sectors both in COG and HUB.
I think this two options are OS/MBR bomb-proof. And much better than using 16 bytes at $1AE (those bytes are reserved or used for other purposes in many different OS).
EDIT: actually, if the SD card is not fixed or soldered into the PCB, the two mechanisms must be used: a custom boot signature to avoid any attemp from a external device to boot from the SD card, and also a reseved partition to protect the boot sectors in the SD card.
I'm tending to shy away from all this massive clutter of "got to haves" but I will say this. I just know what's practical and what I would actually use in real product vs what I see are armchair academic points. SPI 1-bit mode is all I need to boot, either from Flash or SD, with the simple shared 4 I/O mode that Chip proposes. I can't see a product that somehow "needs" quad SPI because if speed were important than so would memory requirements, and in which case I would be using SD. As for 4-bit SD mode that is still a licensed protected mode ("Secure" Digital) and although hackable, it is unexplored territory and unproven for our use. Having 1-bit SPI SD mode for boot will not preclude somehow using 4-bit SD mode for second stage boot or applications when it is later proven.
Peter, the SD pinout for using 4-bit mode has the boot pins arranged differently when using 1-bit SPI. There is a 2 pin gap. See my diagram above for differences. Otherwise we will have to re-arrange the nibble bits and the reverse them.
On the MBR, there are 4 Partition blocks of 16 bytes each, starting $1BE, $1CE, $1DE and $1EE.
Below this is usable for booting, so I intend to use the 16 bytes immediately below these, at $1AE(16).
I will checksum the 16 bytes to be "Prop" as you have done with the downloaded.
One long will store the "raw sector address" of the first block to download, followed by a long for the "number of sectors to read". There is space for two of these if we want.
Cluso99,
The partition entries already have those two fields, "raw sector address" and "number of sectors to read", in the last two longs. For the first partition entry those fields are $1C6, and $1CA. They are called "Relative Sectors", and "Total sectors".
Here go my two cents (valued at $0.01 each proposal):
Why just not check in the first partition entry if it has a Propeller Partition ID (byte $xx at $1C2), and then use those already-assigned-and-standard fields $1C6 and $1CA for boot? As a COG only have 512 longs, two Partition IDs can be defined: one to load sectors into COG, and another partition ID can be used to load sectors both on HUB and COG0 (here we can only copy up to 4 sectors). Or only one partition ID, if the number of sectors is higher than 4 then load sectors both in COG and HUB.
Second proposal: (because maybe we don't want to mess with partition IDs) Define a new bootable signature. Instead of $55, $AA (at the last two bytes of the MBR) we can use the ascii string 'P2' or '2P'. As before, there will be two options: with the 'P2' boot signature, the boot proccess will load into COG0 the number of sectors (up to 4) defined in $1CA starting at "Relative Sectors" ($1C6). And with the boot signature '2P' it will load into HUB the number of sectors (up to max HUB size, now 512KB?) defined in $1CA starting at "Relative Sectors" ($1C6), and also the first sectors in COG0. Or just use one boot signature 'P2' that loads the first 4 sectors into COG0, and if the number of sectors is higher than 4 then load sectors both in COG and HUB.
I think this two options are OS/MBR bomb-proof. And much better than using 16 bytes at $1AE (those bytes are reserved or used for other purposes in many different OS).
EDIT: actually, if the SD card is not fixed or soldered into the PCB, the two mechanisms must be used: a custom boot signature to avoid any attemp from a external device to boot from the SD card, and also a reseved partition to protect the boot sectors in the SD card.
I think it's dangerous to use a Partition Table to store our info. Other devices are almost certainly going to use these SD cards (pc is most likely as I already do this).
Secondly, changing the last two signature bytes should render the card as invalid for use in other devices like the pc.
The area I propose to use is precisely for boot purposes. That's it's function. So there is nothing wrong with us using that area. As for other devices storing anything there, such as boot code, I haven't found any yet. I had s number of people also check their cards too. Phones don't use it, and pcs don't use it. They are likely to be the main devices we may share the SD card with. It's important to ensure those other devices will work with our SD cards.
This is how I use some of the SD Cards on my Prop boards. The EEPROM contains a short boot program that searches the FAT16/32 formatted SD card for the existence of one of two files in fixed order. If either is found, then that program is loaded into hub and re-booted.
The proposed P2 ROM BOOT CODE is far simpler than this. It doesn't know or care about the format of the SD card, nor the Partition tables. It just uses 16 bytes reserved for booting in the MBR (Master Boot Record which is sector 0). We are using the MBR in a similar way that it's meant to be used.
I use both FAT16 (older smaller SD cards) and FAT32 (newer SD cards up to 64GB). The 64GB card is a cheapie of eBay!!!
I have 8 x 32MB contiguous files that I use for CPM2.2. Each of these represent one 8MB HDD for CPM. The internal format of each of these files is in CPM format, so they contain their own file structure with CPM files contained within.
I have written a prop program which can transfer the CPM files between the CPM HDD File Structure and the FAT16/32 File Structure.
On my RamBlade boards I can run my Prop OS, and from there load and run CPM using ZiCog (a Propeller Z80 emulation written by heater). When you exit from CPM/ZiCog, the Prop OS is rebooted. I can also transfer files to/from the PC over serial/USB.
About SD filesystem support, why not put a Tachyon SD block driver in ROM and have it load the first cluster of the SD card and run it as Tachyon bytecode? Could a simple Tachyon FAT driver smart enough to locate and read a specific file fit in the MBR? This would allow booting off of other filesystems and partition tables formats, by just loading the appropriate Tachyon loader for you parition table format and file system.
After all other boot methods fail, the P2 would run the Tachyon monitor, which would look for an SD card and, if present, would load the first cluster, see if it looked like a Tachyon bootloader, and then run it.
I think it's dangerous to use a Partition Table to store our info. Other devices are almost certainly going to use these SD cards (pc is most likely as I already do this).
OK, let's start with the second sentence: yes, I would expect that if anyone is booting from SD, he just will no only use the card for P2 boot but also to store some data (for datalogging, store data, images ... whatever) but also he most likely will remove that card and plug it to a PC or any other device to retrieve and process that data.
Now the first sentence: technically the partition table entry doesn't contain 'our info'. The partition table (entries) is just a pointer (four entries) were the information is located (in our case, the P2 sectors we want to load into HUB or COG space). And partition table entries are precisely the way in which the card (the MBR) define an area that contains partitions (our info).
I wonder, if you 'think it's dangerous to use a Partition Table to store our info' ... then where are you going to write the data you want to load into P2?
- In the MBR? No, it is not possible as it is only 512 bytes. (Unless you just want to load 512 bytes)
- In the 'gap' between the MBR and the first partition? No, highly dangerous. You don't know if any SD card will create copies of the MBR there.
- In unnused 'gaps' between partition entries? No, also highly dangerous as any OS can use that free space to create a partition.
(...)
- In a defined partition? Right, that's the place. A reserved or hidden partition just for P2 boot, or even inside any FAT16/32 partition. (Actually you already said that on you prop boards your 'EEPROM contains a short boot program that searches the FAT16/32 formatted SD card for the existence of one of two files in fixed order.' But note that maybe any user, the own OS, or any computer virus can overwrite those files.)
The way you propose is also dangerous. You are using 16 bytes at $1AE, when this area belongs to the 'Master Boot Code' area. And the function of those 16 bytes are to be the pointer were the P2 sectors are located (same function as a partition entry). But then you will not be able to reserve (protect) that area from write if you do not define it in any of the four MBR-defined partition entries. It might be working for you on the PCs/OS you have tried, but can you prove that it will work for others? I don't think so.
No matter in what place you store the data: reserved/hidden partition, public FAT16/32 partition, or unused gaps in the card; if the SD card is removable, then you cannot prevent any user or application to erase the P2 boot code. It is just too easy that by any error (intended or not) your device will not boot next day. So if you ask me, I would not use just only a SD removable card as primary boot for any P2 application that is not for personal use. Too risky that any user could brick the boot.
So I will need to edit my previous post: There is no OS/MBR bomb-proof way unless some manufacturer makes a SD card with One-Time-Programable boot sector(s).
Secondly, changing the last two signature bytes should render the card as invalid for use in other devices like the pc.
I don't think so. The card is still completely valid for use as data storage (for both PC and P2) and also for P2 boot.
Maybe you wanted to say this:
- 'Changing the last two signature bytes should render the MBR sector (first 512 bytes) as invalid for PC boot code.'.
- or 'Changing the last two signature bytes should render the card as invalid for boot purposes on a PC'
The card will still be valid for P2 boot code, and also valid for data storage (on *both* P2 and PC, the most likely case). I will not expect anyone to use the SD card for *both* PC boot code and P2 boot code. And I don't think that any OS would not let you use your partitions just because you do not have those two bytes in the MBR boot signature.
The area I propose to use is precisely for boot purposes. That's it's function. So there is nothing wrong with us using that area. As for other devices storing anything there, such as boot code, I haven't found any yet. I had s number of people also check their cards too. Phones don't use it, and pcs don't use it. They are likely to be the main devices we may share the SD card with. It's important to ensure those other devices will work with our SD cards.
You confuse the MBR sector area (that is only one sector wide, or 512 bytes size) with the (many) sectors that contain the P2 bytes we want to transfer to P2 COG(s) or HUB.
Technically the area your propose to use ($1AE + 16) is precisely reserved to store the opcode bytes (not all 512 bytes are used, but just only the first 446 bytes) for boot purposes. And if you wanted to load those 446 bytes into the P2 as opcode bytes, then there is nothing wrong with that. Nothing wrong, as far as you are not using Windows 2000. Because W2K writes data in offset $1B8 (just over the same place were you want to write) as 'disk signature' to identify the disk. But let's forget that case as nobody is now using W2K anymore, right? But what happens with the many any other OS that you have never used? ...
Anyway you do not want to load only those 446 bytes into the P2. You want only 16 bytes as a pointer to another area in the card that contains several sectors, and load those sectors into a cog, several cogs, or the full HUB. I remember this has been already discussed, to death, one or two years ago. And no one was able to agree on which was better.
For those that do not know how MBR works, here are some links to help them get their own opinion:
This is how I use some of the SD Cards on my Prop boards. The EEPROM contains a short boot program that searches the FAT16/32 formatted SD card for the existence of one of two files in fixed order. If either is found, then that program is loaded into hub and re-booted.
Then you don't need to write any 16 bytes into the MBR, you just define two read-only and system-attribute files that can be randomly called MSDOS.SYS or IO.SYS, ... wait, sorry, much better call them P2COG0.sys and P2HUB.SYS and that's all you need.
The proposed P2 ROM BOOT CODE is far simpler than this. It doesn't know or care about the format of the SD card, nor the Partition tables. It just uses 16 bytes reserved for booting in the MBR (Master Boot Record which is sector 0). We are using the MBR in a similar way that it's meant to be used.
No, please read my previos reply. Those 16 bytes are at an offset that is intended for PC boot opcodes. You want to use them as a partition entry pointer, but the information will not be protected by any actual partition entry. Sorry, but I think this is a very flaky implementation.
I use both FAT16 (older smaller SD cards) and FAT32 (newer SD cards up to 64GB). The 64GB card is a cheapie of eBay!!!
I have 8 x 32MB contiguous files that I use for CPM2.2. Each of these represent one 8MB HDD for CPM. The internal format of each of these files is in CPM format, so they contain their own file structure with CPM files contained within.
I have written a prop program which can transfer the CPM files between the CPM HDD File Structure and the FAT16/32 File Structure.
On my RamBlade boards I can run my Prop OS, and from there load and run CPM using ZiCog (a Propeller Z80 emulation written by heater). When you exit from CPM/ZiCog, the Prop OS is rebooted. I can also transfer files to/from the PC over serial/USB.
Well, I am sure it will work for you in your computers with your Operating Systems (BTW, long life to CP/M!) . But is far away from a reliable and standard way to deal with MBR and many different OS/devices that any other P2 user might use.
I can't see a product that somehow "needs" quad SPI because if speed were important than so would memory requirements, and in which case I would be using SD.
There will be plenty of SMD placed, higher volume uses where Quad or Octal SPI parts are the correct single chip memory solution.
SD is a module and usually also requires a socket, so is a quite different usage/market footprint, and more aligned with a RaspPi usage than a Microcontroller.
Having 1-bit SPI SD mode for boot will not preclude somehow using 4-bit SD mode for second stage boot or applications when it is later proven.
yup, but you missed the vital provided the Boot pin out allows that.
Any manner of pinouts are 'possible', but it is important to chose a smart-subset, so that Quad and octal SPI memory can be used and that the Streamer can be used, with a common single PCB layout.
I've not seen anyone confirm (Chip?) what is a Streamer Compatible pin-out yet ?
Here is "THE PINOUT" for memory booting. And that's "the" pronounced "thee". This is it. If anyone wants more, they can replicate data pins, as desired, elsewhere. This is what we are going to boot with! Note that this does allow for dual-data-pin SPI flash with properly-oriented pins that are streamer-compatible.
All I'm missing to finish the ROM is the code that works these pins to pull boot code out of an SD card.
First, I'll try to extract SPI flash data. If that doesn't work, I'll try SD card data.
- If neither works, we'll allow 60 seconds for a serial connection.
- If either flash or SD worked, and there's no pull-up on P60, we'll allow 100ms for serial before running the SD/Flash code.
Here is "THE PINOUT" for memory booting. And that's "the" pronounced "thee". This is it. If anyone wants more, they can replicate data pins, as desired, elsewhere. This is what we are going to boot with! Note that this does allow for dual-data-pin SPI flash with properly-oriented pins that are streamer-compatible.
Sounds good.
Just to clarify, is that also compatible with Quad and Octal Streamer use, by merely 'adding more pins' ?
I mean, if we allow this quad SPI setup of CS, CLK, D3, D2, D1, D0, then we'd need to drive all those pins on boot-up or the user would have to put pull-ups on what were HOLDn and WPn. It just looks like a sprawling mess to me with lots of ugly contingencies. I DO LIKE SD, though.
@cgracey
you do not need to drive this pins at boot-up. You assume 1 pin SPI. the 4bit mode will be than used by the user.
If the designer uses 1bit SPI, by no driving the other pins will make them usable for anything else. Hold and WP will be hw defined by PCB in this case.
If the designer chooses a QuadSPI connection he will add also the 2 pull-ups for the scope.
The supported SD boot have to be in SPI mode only, thus again 4 pins.
The most future proof boot is to use a raw (un-formatted, without file-system partition). In this way you read the MBR and from the partition table you take its start and then start reading a given number of bytes in.
In the MBR there is "safe" and standard places to have also a small signature/checksum and also clock configuration and mode (cog/hub-exec).
The best thing is to implement the same MBR also in the flash. I would recommend to foresee for double-image so that first a new image can be written and than pointed to in the MBR. If the pointed image fails the system could try the other one. This will give reliability for field updates even through the air/internet as even a big image can be written directly to flash/SD and then verified prior to switch the pointer to it.
The ROM booter will in this way read the MBR, being it from flash or SD and live-extract from the flow the image start/length, clock-setup and execution-mode to a few COG registers and finally stream into the hub the image. Then cog or hub exec it based upon the mode setup.
This will also make most of the read/streaming code reusable between the two media reducing the ROM footprint.
Also IMHO the serial boot should be available only if flash/SD checks fails or are not detected. That means that for serial-to-ram download and or flash/sd rewrite, in case of working/good image the flash/SD firmware should foresee this option. Serial boot will be available only with damaged/missing/empty flash/SD. Should the user want always the serial-boot even with programmed flash/SD: that means the flash/SD will not be on boot pins but elsewhere.
In this way serial path can be used with full speed from the beginning because the flash/SD boot is already completed. The needed flash/SD function (pasm/spin) can be added to the image by the IDE on user request (flag) and called(hub/cog-executed) from the user program at user's conditions. It could be a standard/library function. This gives also a minimal level of code/image safety/protection.
The IDE will support empty-flash/SD and programmed flash/SD image-write processes. And will support double image writes. This by downloading its sw-writer to the P2 RAM in case of empty/failed/missing flash/SD or will comunicate to the firmware built-in image-update function.
dMajo, I see what you are saying, but I think that serial needs to be able to get in more easily, as it's the means to update the flash/SD, plus it's the fast path for development. I made a proposal in the other thread about having a static 4-pin layout for both flash and SD. Could you please say if you think it would work to do that? It seems by far the cleanest approach to me, but maybe it's a little unorthodox.
I'll just replicate the post here:
So, could we make a static 4-pin pinout that works with both SPI flash and SD card, assuming one or the other is present (or neither is present)?
If we had this:
P61 = CS (pull-up indicates presence of either SPI flash or SD card)
P60 = CLK (pull-up indicates 'skip serial wait if boot data okay')
P59 = DI (could be used as SPI flash DQ0)
P58 = DO (could be used as SPI flash DQ1)
P57 = (could be used as SPI flash DQ2)
P56 = (could be used as SPI flash DQ3)
Chip, what about this?
P61 = CS/SDA (pull-up indicates presence of either SPI flash or SD card or I2C Eeprom)
P60 = CLK/SCL (pull-up indicates I2C device)
P59 = not used (could be used as SPI flash DQ3)
P58 = not used (could be used as SPI flash DQ2)
P57 = DO (could be used as SPI flash DQ1, not used with I2C)
P56 = DI (could be used as SPI flash DQ0, not used with I2C) (pull-up indicates 'skip serial wait if boot data okay')
1) still 4 pin interface, not necessarily the pins have to be consecutive. This do not require the reversal of the 4 bits in case of QSPI use
2) Consider also I2C device, eg AT24CM02 (2Mb; 256KBx8). This is half the hub memory,
In the ROM booter now, I do some initial command/response sleuthing to determine if a SPI chip is present. That takes umpteen microseconds. If it seems present, it takes ~4ms to read in $100 longs and determine if they're legit.
If you consider MBR also in flash $80longs (512 bytes) are enough. They can validate the device, retrive which image (start/lenght) to use, retrive the clock setup and image execution mode (hub/cog)
If either of those checks fail, could we then treat those 4 pins as if an SD card is connected, and see what we get? This would all be predicated on seeing a pull-up on P61, indicating some kind of memory is present. This keeps our initial "is-memory-present" pin checking down to ONE pin. Also, by keeping DI and DO separate for both SPI flash and SD card, we could hold DI high while reading SPI flash (CS is low during reading) to avoid some command being interpreted by the SD card. Also, keeping those pins separate allows for quad SPI flash usage.
What do you guys think about this? Is it possible to have a pin-out scheme for booting that supports both SPI flash and SD card, with static CS, CLK, DI, and DO pins?
If you test immediately both P61&P60 you can proceed securely to I2C boot or, as you described, first attempt a flash boot and on fail go with SD.
Flash are bigger and can store many images. Perhaps with EEprom, it will be better, in case of failure (wrong MBR of IC failure), to try a second attempt with A2 high in the address to boot from a second EEprom allowing thus a redundant boot eeprom.
In case of double-eeprom the starting address in the first eeprom's MBR can of course point also to the 2nd eeprom data, depending on its size.
...
First, I'll try to extract SPI flash data. If that doesn't work, I'll try SD card data.
- If neither work, we'll allow 60 seconds for a serial connection.
- If either flash or SD worked, and there's no pull-up on P60, we'll allow 100ms for serial before running the SD/Flash code.
To clarify, that allows both of these ?
* fastest SPI boot (SPI memory connected)
* Option to skip SPI, and do a fastest serial load (MCU host as memory)
If someone wants remote steering of 'done first', there are SOT6 package dual pre-biased transistors, that can manage both
RST pin, and do UART/SPI choice via the second UART handshake line. That would be a useful Eval board default.
...
First, I'll try to extract SPI flash data. If that doesn't work, I'll try SD card data.
- If neither work, we'll allow 60 seconds for a serial connection.
- If either flash or SD worked, and there's no pull-up on P60, we'll allow 100ms for serial before running the SD/Flash code.
To clarify, that allows both of these ?
* fastest SPI boot (SPI memory connected)
* Option to skip SPI, and do a fastest serial load (MCU host as memory)
If someone wants remote steering of 'done first', there are SOT6 package dual pre-biased transistors, that can manage both
RST pin, and do UART/SPI choice via the second UART handshake line. That would be a useful Eval board default.
How is pull-up-presence detected ?
There's no option to skip SPI, but it's only a 4ms excursion to see if we've got a valid SPI booter.
Pull-up is detected by pulsing the pin low, waiting 5us, and checking if it's high.
DMajo, I want to keep things ultra-simple, like in that diagram. I don't want to mess with I2C stuff - just SPI flash and SD card.
A question for you and Cluso99:
Is it realistic to load a 256-long program from SD which, in turn, loads in the main application from the SD card, turning on the crystal and PLL, to get max speed? Is a 256-long program enough?
From SPI flash, I only load a 256-long program and that's plenty to switch the clock and load the app from SPI flash. Are these same metrics suitable for SD card?
There's no option to skip SPI, but it's only a 4ms excursion to see if we've got a valid SPI booter.
?
4ms is a long time to add to the 'do not have control of the pins' watchdog reset recovery
Not only that, but someone who used 1-pin UART mode to really save pins, now finds the SPI pins they hoped to use themselves, become highly active during boot with CLK and DATA.
Isn't that a quite significant step backwards, from where P2 was ?
There's no option to skip SPI, but it's only a 4ms excursion to see if we've got a valid SPI booter.
?
4ms is a long time to add to the 'do not have control of the pins' watchdog reset recovery
Not only that, but someone who used 1-pin UART mode to really save pins, now finds the SPI pins they hoped to use themselves, become highly active during boot with CLK and DATA.
Isn't that a quite significant step backwards, from where P2 was ?
They only become active if there is a pull up on P61.
By using smart pins, I could probably knock that four milliseconds down to less than one. I'm just bit banging it in software right now.
...
First, I'll try to extract SPI flash data. If that doesn't work, I'll try SD card data.
- If neither work, we'll allow 60 seconds for a serial connection.
- If either flash or SD worked, and there's no pull-up on P60, we'll allow 100ms for serial before running the SD/Flash code.
To clarify, that allows both of these ?
* fastest SPI boot (SPI memory connected)
* Option to skip SPI, and do a fastest serial load (MCU host as memory)
If someone wants remote steering of 'done first', there are SOT6 package dual pre-biased transistors, that can manage both
RST pin, and do UART/SPI choice via the second UART handshake line. That would be a useful Eval board default.
How is pull-up-presence detected ?
There's no option to skip SPI, but it's only a 4ms excursion to see if we've got a valid SPI booter.
Pull-up is detected by pulsing the pin low, waiting 5us, and checking if it's high.
A 4ms delay before attempting to boot an SD Card is fine.
The SD Card boot process is not a fast one. It requires at least a 1ms delay after the power is stable. Then it requires >74 Clocks at <400KHz. Next, the reset and initialisation of different cards varies, but is significant.
It's only once we have booted where we gain the speed from SD Cards.
DMajo, I want to keep things ultra-simple, like in that diagram. I don't want to mess with I2C stuff - just SPI flash and SD card.
A question for you and Cluso99:
Is it realistic to load a 256-long program from SD which, in turn, loads in the main application from the SD card, turning on the crystal and PLL, to get max speed? Is a 256-long program enough?
From SPI flash, I only load a 256-long program and that's plenty to switch the clock and load the app from SPI flash. Are these same metrics suitable for SD card?
That would be good enough to have capability to load more SD data, but not enough to do full support for a file system like FAT16/32. However, it might be close to be able to transverse the file system to find and load a named file.
If I can fit it into the MBR space which is from memory 512-(4*16)-2 = 446bytes = 111 longs, while using some of the ROM boot routines too, it would solve a lot of arguments! Will have to see
..
They only become active if there is a pull up on P61.
If that's the case, why does it not skip the attempt, and go straight to UART ?
Or, does it also need to try SD (which adds even more time?)..
It checks and if the data is good it only allows 100ms for serial. Otherwise, it allows a whole minute. If the data is good and P60 (Clk) is pulled high, it skips serial.
DMajo, I want to keep things ultra-simple, like in that diagram. I don't want to mess with I2C stuff - just SPI flash and SD card.
A question for you and Cluso99:
Is it realistic to load a 256-long program from SD which, in turn, loads in the main application from the SD card, turning on the crystal and PLL, to get max speed? Is a 256-long program enough?
From SPI flash, I only load a 256-long program and that's plenty to switch the clock and load the app from SPI flash. Are these same metrics suitable for SD card?
Chip,
I can't follow you.
If you load anything from flash or SD that means that your boot-rom code handles spi communication with the memory device.
Why you need to load a program to turn on crystal? This is something I think should pertain to rom code.
In my vision this is how it should work (rom-code):
1) both (for simplicity) flash device and SD should have the MBR (Master Boot Record) where up to 4 partitions can be declared (16 bytes each starting at offset 446 in the MBR). The MBR is at absolute 0 of the media, the first 512 bytes.
- on flash (eg 16MB) device there will be no file system. The PartitionEntry will define which image to load and where it starts. I foresee max 2 images (to ensure safe field upgrades), but up to 4 can be implemented.
- on SD we can use 2 PartitionEntries for boot images and a 3rd one for data exchange. That means the 3rd will have a filesystem and it will be available only after completed boot, when the image (containing the filesystem drivers) will be fully loaded and executed by the P2. By moving the SD from P2 to PC only the files and folders of the 3rd partition will be seen. However any disk imaging software will be able to update also the boot images in the other 2 partitons.
- Since no PC will ever boot from the P2 SD card also the addresses from 0 to 445 of the MBR are free to any use. For example an additional validation key can stored.
2) the rom-code needs to:
- read the MBR,
- validate among a key stored somewhere in the first 446 bytes
- look at the partiton entries to see which is the active/bootable one (1st byte of each partition entry)
- ensure that the bootable partiton is of the right type (5th byte of bootable partition entry)
- retrive image start and lenght (4 + 4 bytes, starting at offset 8 in partition entry)
- the old CHS first/last (3 + 3 bytes of partition entry) can store crystal and execution mode settings
- the rom cade setup clock-source and start reading/streaming the image(app) to hub from the previously retrived start and lenght.
To me load a second-stage booter, now that fuses has gone, is a nonsense. If the user needs it it will be in the image. The image will have a shorter length parameter and once loaded it will do whatever the user needs.
IMHO the rom-boot code should read a validation key somewhere in the first 446 bytes and 16 longs starting from the 446th byte, then start loading from the pointer retrived by examination of the partition table.
But to reply to your question the maximum available area in the MBR is up to the partition table. This is 446 bytes or 111 longs (Generic MBR - Bootstrap code area).
This "second stage boot loader" will find the SD already initialized from the rom-boot code so it have only to read the partition table, elaborate its parameters and start loading the image after clock-source adjustment. You need at least 16 registers to buffer internally the partition table. The 111 then needs to have the SD-SPI routines and handle the logic.
Ideally your rom-code, regardless of the boot source, will have a 512 byte buffer in cog or lut, load the full MBR (128 longs, first 512 bytes) to that buffer and then jump to its start. In this way the first 111 longs of the buffer (executable code) will have only to evaluate the buffered MBR and then start loading the image(app) into the hub memory (re)using your rom-spi-routines. Due to long alignment the MBR's first 2 bytes are moved to the end of buffer.
Can you expand that pinout, to include the streamer-compatible pins for QuadSPI and OctaSPI ?
Can you also give the tests applied, Pin/jumper straps used, shortest possible times taken, & timeouts, for each of
* Serial UART boot, one pin
* Serial UART boot, two pin
* SPI memory boot
* SD Card boot
* Whatever applies after all 3 were tried..
The ideal for users, is very fast skip of undesired stages, and minimal pin impact, on any unused/skipped pins.
Comments
For boot, I would like to support SPI SD over 4 pins, total. If someone wants to use 6-pin mode, they'd need to wire up additional pins. So, for now, can we get this 4-pin thing going? What do we do with CS, CLK, DI, and DO to read in boot code, reliably? I think you said you could do this in 1KB. Does that mean only 256 instructions? If so, let's get it into the ROM!
Yes, I expect <256 instructions if my calls are correct. Plus of course the sector buffer of 512 bytes, but we will likely load the sectors into cog or hub for execution.
On the MBR, there are 4 Partition blocks of 16 bytes each, starting $1BE, $1CE, $1DE and $1EE.
Below this is usable for booting, so I intend to use the 16 bytes immediately below these, at $1AE(16).
I will checksum the 16 bytes to be "Prop" as you have done with the downloaded.
One long will store the "raw sector address" of the first block to download, followed by a long for the "number of sectors to read". There is space for two of these if we want.
I have used 0.5" jumpers but they are expensive.
Anyway, I don't expect to use FLASH as I will be using just an SD CARD.
Cluso99,
The partition entries already have those two fields, "raw sector address" and "number of sectors to read", in the last two longs. For the first partition entry those fields are $1C6, and $1CA. They are called "Relative Sectors", and "Total sectors".
Here go my two cents (valued at $0.01 each proposal):
Why just not check in the first partition entry if it has a Propeller Partition ID (byte $xx at $1C2), and then use those already-assigned-and-standard fields $1C6 and $1CA for boot? As a COG only have 512 longs, two Partition IDs can be defined: one to load sectors into COG, and another partition ID can be used to load sectors both on HUB and COG0 (here we can only copy up to 4 sectors). Or only one partition ID, if the number of sectors is higher than 4 then load sectors both in COG and HUB.
Second proposal: (because maybe we don't want to mess with partition IDs) Define a new bootable signature. Instead of $55, $AA (at the last two bytes of the MBR) we can use the ascii string 'P2' or '2P'. As before, there will be two options: with the 'P2' boot signature, the boot proccess will load into COG0 the number of sectors (up to 4) defined in $1CA starting at "Relative Sectors" ($1C6). And with the boot signature '2P' it will load into HUB the number of sectors (up to max HUB size, now 512KB?) defined in $1CA starting at "Relative Sectors" ($1C6), and also the first sectors in COG0. Or just use one boot signature 'P2' that loads the first 4 sectors into COG0, and if the number of sectors is higher than 4 then load sectors both in COG and HUB.
I think this two options are OS/MBR bomb-proof. And much better than using 16 bytes at $1AE (those bytes are reserved or used for other purposes in many different OS).
EDIT: actually, if the SD card is not fixed or soldered into the PCB, the two mechanisms must be used: a custom boot signature to avoid any attemp from a external device to boot from the SD card, and also a reseved partition to protect the boot sectors in the SD card.
I think it's dangerous to use a Partition Table to store our info. Other devices are almost certainly going to use these SD cards (pc is most likely as I already do this).
Secondly, changing the last two signature bytes should render the card as invalid for use in other devices like the pc.
The area I propose to use is precisely for boot purposes. That's it's function. So there is nothing wrong with us using that area. As for other devices storing anything there, such as boot code, I haven't found any yet. I had s number of people also check their cards too. Phones don't use it, and pcs don't use it. They are likely to be the main devices we may share the SD card with. It's important to ensure those other devices will work with our SD cards.
The proposed P2 ROM BOOT CODE is far simpler than this. It doesn't know or care about the format of the SD card, nor the Partition tables. It just uses 16 bytes reserved for booting in the MBR (Master Boot Record which is sector 0). We are using the MBR in a similar way that it's meant to be used.
I use both FAT16 (older smaller SD cards) and FAT32 (newer SD cards up to 64GB). The 64GB card is a cheapie of eBay!!!
I have 8 x 32MB contiguous files that I use for CPM2.2. Each of these represent one 8MB HDD for CPM. The internal format of each of these files is in CPM format, so they contain their own file structure with CPM files contained within.
I have written a prop program which can transfer the CPM files between the CPM HDD File Structure and the FAT16/32 File Structure.
On my RamBlade boards I can run my Prop OS, and from there load and run CPM using ZiCog (a Propeller Z80 emulation written by heater). When you exit from CPM/ZiCog, the Prop OS is rebooted. I can also transfer files to/from the PC over serial/USB.
After all other boot methods fail, the P2 would run the Tachyon monitor, which would look for an SD card and, if present, would load the first cluster, see if it looked like a Tachyon bootloader, and then run it.
Now the first sentence: technically the partition table entry doesn't contain 'our info'. The partition table (entries) is just a pointer (four entries) were the information is located (in our case, the P2 sectors we want to load into HUB or COG space). And partition table entries are precisely the way in which the card (the MBR) define an area that contains partitions (our info).
I wonder, if you 'think it's dangerous to use a Partition Table to store our info' ... then where are you going to write the data you want to load into P2?
- In the MBR? No, it is not possible as it is only 512 bytes. (Unless you just want to load 512 bytes)
- In the 'gap' between the MBR and the first partition? No, highly dangerous. You don't know if any SD card will create copies of the MBR there.
- In unnused 'gaps' between partition entries? No, also highly dangerous as any OS can use that free space to create a partition.
(...)
- In a defined partition? Right, that's the place. A reserved or hidden partition just for P2 boot, or even inside any FAT16/32 partition. (Actually you already said that on you prop boards your 'EEPROM contains a short boot program that searches the FAT16/32 formatted SD card for the existence of one of two files in fixed order.' But note that maybe any user, the own OS, or any computer virus can overwrite those files.)
The way you propose is also dangerous. You are using 16 bytes at $1AE, when this area belongs to the 'Master Boot Code' area. And the function of those 16 bytes are to be the pointer were the P2 sectors are located (same function as a partition entry). But then you will not be able to reserve (protect) that area from write if you do not define it in any of the four MBR-defined partition entries. It might be working for you on the PCs/OS you have tried, but can you prove that it will work for others? I don't think so.
No matter in what place you store the data: reserved/hidden partition, public FAT16/32 partition, or unused gaps in the card; if the SD card is removable, then you cannot prevent any user or application to erase the P2 boot code. It is just too easy that by any error (intended or not) your device will not boot next day. So if you ask me, I would not use just only a SD removable card as primary boot for any P2 application that is not for personal use. Too risky that any user could brick the boot.
So I will need to edit my previous post: There is no OS/MBR bomb-proof way unless some manufacturer makes a SD card with One-Time-Programable boot sector(s).
I don't think so. The card is still completely valid for use as data storage (for both PC and P2) and also for P2 boot.
Maybe you wanted to say this:
- 'Changing the last two signature bytes should render the MBR sector (first 512 bytes) as invalid for PC boot code.'.
- or 'Changing the last two signature bytes should render the card as invalid for boot purposes on a PC'
The card will still be valid for P2 boot code, and also valid for data storage (on *both* P2 and PC, the most likely case). I will not expect anyone to use the SD card for *both* PC boot code and P2 boot code. And I don't think that any OS would not let you use your partitions just because you do not have those two bytes in the MBR boot signature.
You confuse the MBR sector area (that is only one sector wide, or 512 bytes size) with the (many) sectors that contain the P2 bytes we want to transfer to P2 COG(s) or HUB.
Technically the area your propose to use ($1AE + 16) is precisely reserved to store the opcode bytes (not all 512 bytes are used, but just only the first 446 bytes) for boot purposes. And if you wanted to load those 446 bytes into the P2 as opcode bytes, then there is nothing wrong with that. Nothing wrong, as far as you are not using Windows 2000. Because W2K writes data in offset $1B8 (just over the same place were you want to write) as 'disk signature' to identify the disk. But let's forget that case as nobody is now using W2K anymore, right? But what happens with the many any other OS that you have never used? ...
Anyway you do not want to load only those 446 bytes into the P2. You want only 16 bytes as a pointer to another area in the card that contains several sectors, and load those sectors into a cog, several cogs, or the full HUB. I remember this has been already discussed, to death, one or two years ago. And no one was able to agree on which was better.
For those that do not know how MBR works, here are some links to help them get their own opinion:
[1] https://technet.microsoft.com/en-us/library/cc976786.aspx
The reference. Current MBR as defined by its creators: Microsoft/IBM PC DOS
[2] http://thestarman.pcministry.com/asm/mbr/PartTables.htm
Brief and visually descriptive summary of MBR partition tables records.
[3] http://wiki.osdev.org/Partition_Table
Partition table from an PC OS boot perspective
[4] https://en.wikipedia.org/wiki/Master_boot_record
Wikipedia. (No introduction required.)
Agreed.
There will be plenty of SMD placed, higher volume uses where Quad or Octal SPI parts are the correct single chip memory solution.
SD is a module and usually also requires a socket, so is a quite different usage/market footprint, and more aligned with a RaspPi usage than a Microcontroller.
yup, but you missed the vital provided the Boot pin out allows that.
Any manner of pinouts are 'possible', but it is important to chose a smart-subset, so that Quad and octal SPI memory can be used and that the Streamer can be used, with a common single PCB layout.
I've not seen anyone confirm (Chip?) what is a Streamer Compatible pin-out yet ?
All I'm missing to finish the ROM is the code that works these pins to pull boot code out of an SD card.
First, I'll try to extract SPI flash data. If that doesn't work, I'll try SD card data.
- If neither works, we'll allow 60 seconds for a serial connection.
- If either flash or SD worked, and there's no pull-up on P60, we'll allow 100ms for serial before running the SD/Flash code.
That's the plan.
Sounds good.
Just to clarify, is that also compatible with Quad and Octal Streamer use, by merely 'adding more pins' ?
P61 = CS/SDA (pull-up indicates presence of either SPI flash or SD card or I2C Eeprom)
P60 = CLK/SCL (pull-up indicates I2C device)
P59 = not used (could be used as SPI flash DQ3)
P58 = not used (could be used as SPI flash DQ2)
P57 = DO (could be used as SPI flash DQ1, not used with I2C)
P56 = DI (could be used as SPI flash DQ0, not used with I2C) (pull-up indicates 'skip serial wait if boot data okay')
1) still 4 pin interface, not necessarily the pins have to be consecutive. This do not require the reversal of the 4 bits in case of QSPI use
2) Consider also I2C device, eg AT24CM02 (2Mb; 256KBx8). This is half the hub memory,
If you consider MBR also in flash $80longs (512 bytes) are enough. They can validate the device, retrive which image (start/lenght) to use, retrive the clock setup and image execution mode (hub/cog)
If you test immediately both P61&P60 you can proceed securely to I2C boot or, as you described, first attempt a flash boot and on fail go with SD.
Flash are bigger and can store many images. Perhaps with EEprom, it will be better, in case of failure (wrong MBR of IC failure), to try a second attempt with A2 high in the address to boot from a second EEprom allowing thus a redundant boot eeprom.
In case of double-eeprom the starting address in the first eeprom's MBR can of course point also to the 2nd eeprom data, depending on its size.
* fastest SPI boot (SPI memory connected)
* Option to skip SPI, and do a fastest serial load (MCU host as memory)
If someone wants remote steering of 'done first', there are SOT6 package dual pre-biased transistors, that can manage both
RST pin, and do UART/SPI choice via the second UART handshake line. That would be a useful Eval board default.
How is pull-up-presence detected ?
There's no option to skip SPI, but it's only a 4ms excursion to see if we've got a valid SPI booter.
Pull-up is detected by pulsing the pin low, waiting 5us, and checking if it's high.
A question for you and Cluso99:
Is it realistic to load a 256-long program from SD which, in turn, loads in the main application from the SD card, turning on the crystal and PLL, to get max speed? Is a 256-long program enough?
From SPI flash, I only load a 256-long program and that's plenty to switch the clock and load the app from SPI flash. Are these same metrics suitable for SD card?
?
4ms is a long time to add to the 'do not have control of the pins' watchdog reset recovery
Not only that, but someone who used 1-pin UART mode to really save pins, now finds the SPI pins they hoped to use themselves, become highly active during boot with CLK and DATA.
Isn't that a quite significant step backwards, from where P2 was ?
They only become active if there is a pull up on P61.
By using smart pins, I could probably knock that four milliseconds down to less than one. I'm just bit banging it in software right now.
A 4ms delay before attempting to boot an SD Card is fine.
The SD Card boot process is not a fast one. It requires at least a 1ms delay after the power is stable. Then it requires >74 Clocks at <400KHz. Next, the reset and initialisation of different cards varies, but is significant.
It's only once we have booted where we gain the speed from SD Cards.
That would be good enough to have capability to load more SD data, but not enough to do full support for a file system like FAT16/32. However, it might be close to be able to transverse the file system to find and load a named file.
If I can fit it into the MBR space which is from memory 512-(4*16)-2 = 446bytes = 111 longs, while using some of the ROM boot routines too, it would solve a lot of arguments! Will have to see
Can we use the SmartPins in a really basic way?
I ask because I wonder if we can use the smartpins to reorder the data pins?
They have a reach of +/-3 pins.
Or, does it also need to try SD (which adds even more time?)..
Whoa! Yes, you can use them to reorder inputs, but not outputs.
It checks and if the data is good it only allows 100ms for serial. Otherwise, it allows a whole minute. If the data is good and P60 (Clk) is pulled high, it skips serial.
Chip,
I can't follow you.
If you load anything from flash or SD that means that your boot-rom code handles spi communication with the memory device.
Why you need to load a program to turn on crystal? This is something I think should pertain to rom code.
In my vision this is how it should work (rom-code):
1) both (for simplicity) flash device and SD should have the MBR (Master Boot Record) where up to 4 partitions can be declared (16 bytes each starting at offset 446 in the MBR). The MBR is at absolute 0 of the media, the first 512 bytes.
- on flash (eg 16MB) device there will be no file system. The PartitionEntry will define which image to load and where it starts. I foresee max 2 images (to ensure safe field upgrades), but up to 4 can be implemented.
- on SD we can use 2 PartitionEntries for boot images and a 3rd one for data exchange. That means the 3rd will have a filesystem and it will be available only after completed boot, when the image (containing the filesystem drivers) will be fully loaded and executed by the P2. By moving the SD from P2 to PC only the files and folders of the 3rd partition will be seen. However any disk imaging software will be able to update also the boot images in the other 2 partitons.
- Since no PC will ever boot from the P2 SD card also the addresses from 0 to 445 of the MBR are free to any use. For example an additional validation key can stored.
2) the rom-code needs to:
- read the MBR,
- validate among a key stored somewhere in the first 446 bytes
- look at the partiton entries to see which is the active/bootable one (1st byte of each partition entry)
- ensure that the bootable partiton is of the right type (5th byte of bootable partition entry)
- retrive image start and lenght (4 + 4 bytes, starting at offset 8 in partition entry)
- the old CHS first/last (3 + 3 bytes of partition entry) can store crystal and execution mode settings
- the rom cade setup clock-source and start reading/streaming the image(app) to hub from the previously retrived start and lenght.
To me load a second-stage booter, now that fuses has gone, is a nonsense. If the user needs it it will be in the image. The image will have a shorter length parameter and once loaded it will do whatever the user needs.
IMHO the rom-boot code should read a validation key somewhere in the first 446 bytes and 16 longs starting from the 446th byte, then start loading from the pointer retrived by examination of the partition table.
But to reply to your question the maximum available area in the MBR is up to the partition table. This is 446 bytes or 111 longs (Generic MBR - Bootstrap code area).
This "second stage boot loader" will find the SD already initialized from the rom-boot code so it have only to read the partition table, elaborate its parameters and start loading the image after clock-source adjustment. You need at least 16 registers to buffer internally the partition table. The 111 then needs to have the SD-SPI routines and handle the logic.
Ideally your rom-code, regardless of the boot source, will have a 512 byte buffer in cog or lut, load the full MBR (128 longs, first 512 bytes) to that buffer and then jump to its start. In this way the first 111 longs of the buffer (executable code) will have only to evaluate the buffered MBR and then start loading the image(app) into the hub memory (re)using your rom-spi-routines. Due to long alignment the MBR's first 2 bytes are moved to the end of buffer.
Chip, pls have a look at the attached excel sheet
Can you expand that pinout, to include the streamer-compatible pins for QuadSPI and OctaSPI ?
Can you also give the tests applied, Pin/jumper straps used, shortest possible times taken, & timeouts, for each of
* Serial UART boot, one pin
* Serial UART boot, two pin
* SPI memory boot
* SD Card boot
* Whatever applies after all 3 were tried..
The ideal for users, is very fast skip of undesired stages, and minimal pin impact, on any unused/skipped pins.