I agree with this. A small, we'll defined thing can be debugged and be robust. Easy to document and use. Everyone makes tradeoffs to suit their requirements from there.
And....
By cutting the loader down to 256 longs and halving the load/verify time to 14ms, the serial running in the background can admit 161 bytes at 115.2k baud into its 256-byte buffer, so that they can be processed later, in case the flash loader doesn't validate - so no host data is lost and both purposes are served.
The lingering issues are how long to initially wait for a host (0 or 100ms), and then how long to wait in case the flash loader doesn't verify (1 minute or infinity).
The RC fast starts immediately. The crystal takes 10ms to stabilize The PLL takes 10us.
When exactly does the Crystal Start/stop ?
As the user controls the Crystal Config, they can add delays as needed.
I'm still unclear on if the First Flash load sizecan be User-Defined ?
Does this first flash partial load have to use the same security levels as later, higher level loads ?
With a Pin-Skip, a 10us hard reset delay, & user Size choice, the important time-to-pins-defined could be slashed from the present 20ms+
Another question is, can the smart pins be used for the SPI load, to push up the MHz there ?
( Times given indicate the SPI SW clock is below 2MHz, ~ 1.5MHz ? )
In contrast, 20MHz can stream 256 longs in 409us
SPI_CS pull-up?
No - Wait for serial.
Yes - SPI_CK pull-up?
No - Wait 100ms for serial, attempt flash boot, if fail then wait for serial.
Yes - Attempt flash boot, if fail then wait for serial.
This way, a jumper on the SPI_CK pull-up can be used to reload a system that normally (jumper on) boots from flash immediately.
At this point, I've decided against a time-out on waiting for serial. Any objections?
This way, a jumper on the SPI_CK pull-up can be used to reload a system that normally (jumper on) boots from flash immediately.
Sounds good - what are the expected times now, from reset-release ?
Not sure if this is what you meant, by 'pull-up?', but it should be possible to support a (NOT Pull-Up) test that catches either Pull down, or floating, to mean PCBs can be simpler, just one resistor, with a series jumper.
Some rough numbers give
10k pullup, and highish 15pF total pin C == 10k*15p = 150ns
Leakage and lowish C == 1M*8p = 8 us
That indicates an impulse and then test after (say) 250~300ns would catch both cases.
Not sure what FPGA boards give, but should be easy to check. Higher nett C just needs a lower pull-up.
At this point, I've decided against a time-out on waiting for serial. Any objections?
The wait for serial will be lowish power ?
How does this tree interact with the encryption option, and possible attacks ?
FWIR, Encrypt is an OTP fuse and once on, everything coming into the chip is then treated Encrypted ?
Default is Encrypt Off, which means non encrypt systems may have faster boot times ?
The flash booter must always be signed. The only time an unsigned program is permitted is when the chip's first 128 fuses are 0's (no key) and "~" ends a serial loading transmission. You can always sign a serial transmission using a key of zero, if you want to be sure the data gets there okay. For the case of any non-zero key, the serial loading transmission must always be signed.
About how much power it would take to wait around indefinitely for a serial load, it's one cog running at 20MHz. It will probably take a few mA. Leakage is going to be ~1mA, anyway.
For casual development, it's not necessary to provide a signature if the chip's key is zero. This makes it easy to get started making tools. Asking people to do SHA-256/HMAC right away is a tall order.
SPI_CS pull-up?
No - Wait for serial.
Yes - SPI_CK pull-up?
No - Wait 100ms for serial, attempt flash boot, if fail then wait for serial.
Yes - Attempt flash boot, if fail then wait for serial.
This way, a jumper on the SPI_CK pull-up can be used to reload a system that normally (jumper on) boots from flash immediately.
At this point, I've decided against a time-out on waiting for serial. Any objections?
IMHO this is EXCELLENT.
If the user doesn't want to waste power while waiting for serial, then his option is to have a bootloader in flash that does a waitpne/waitpeq for a serial edge.
Only suggestion is that if SPI Flash fails, then try SPI SD (simple boot, no FAT knowledge).
P63 RXD (SI = serial in)
P62 TXD (SO = serial out)
Flash... | SD...
P61 #CE | DI / CMD
P60 CLK | CLK
P59 (#HOLD) / SIO3 | #CS / D3-CD
P58 (#WP) / SIO2 | x / D2
P57 SO / SIO1 | x / D1
P56 SI / SIO0 | DO / D0
Note: The akward use of the CD pinout to permit possible use of quad mode CD at a later date.
1. This permits the later use of QUAD SPI without the need to swap pins/bits around when using QUAD SPI.
2. This also permits SD to be used on the same pins instead of (ie without) SPI.
3. When accessing Flash normally (not Quad mode),#HOLD=1. This is compatible with the SD Card being deselected. Only if Flash was determined not to be present would the SD Card be tried.
Only suggestion is that if SPI Flash fails, then try SPI SD (simple boot, no FAT knowledge).
That may be dictated by spare code size ?
As a check, I see SPI Flash parts are now down to 10.4c/4k at Digikey, for 104MHz 4Mbit, SO8, and 14c for 120MHz TSSOP8.
SPI_CS pull-up?
No - Wait for serial.
Yes - SPI_CK pull-up?
No - Wait 100ms for serial, attempt flash boot, if fail then wait for serial.
Yes - Attempt flash boot, if fail then wait for serial.
This way, a jumper on the SPI_CK pull-up can be used to reload a system that normally (jumper on) boots from flash immediately.
At this point, I've decided against a time-out on waiting for serial. Any objections?
IMHO this is EXCELLENT.
If the user doesn't want to waste power while waiting for serial, then his option is to have a bootloader in flash that does a waitpne/waitpeq for a serial edge.
Only suggestion is that if SPI Flash fails, then try SPI SD (simple boot, no FAT knowledge).
P63 RXD (SI = serial in)
P62 TXD (SO = serial out)
Flash... | SD...
P61 #CE | DI / CMD
P60 CLK | CLK
P59 (#HOLD) / SIO3 | #CS / D3-CD
P58 (#WP) / SIO2 | x / D2
P57 SO / SIO1 | x / D1
P56 SI / SIO0 | DO / D0
Note: The akward use of the CD pinout to permit possible use of quad mode CD at a later date.
1. This permits the later use of QUAD SPI without the need to swap pins/bits around when using QUAD SPI.
2. This also permits SD to be used on the same pins instead of (ie without) SPI.
3. When accessing Flash normally (not Quad mode),#HOLD=1. This is compatible with the SD Card being deselected. Only if Flash was determined not to be present would the SD Card be tried.
I'm sorry to ask this, but what is a simple SD Card boot process? I know we've been over this ten times, but nothing I recall is very simple.
My version of simple is to read a signature from sector 0, then the starting sector and number of sectors to load. I do this in Tachyon where I save a file that always has contiguous sectors and write the boot info to sector 0. All the boot loader has to do after initializing the SD is to read sector 0 for this info and then transfer the specified sectors to memory.
SPI_CS pull-up?
No - Wait for serial.
Yes - SPI_CK pull-up?
No - Wait 100ms for serial, attempt flash boot, if fail then wait for serial.
Yes - Attempt flash boot, if fail then wait for serial.
This way, a jumper on the SPI_CK pull-up can be used to reload a system that normally (jumper on) boots from flash immediately.
At this point, I've decided against a time-out on waiting for serial. Any objections?
IMHO this is EXCELLENT.
If the user doesn't want to waste power while waiting for serial, then his option is to have a bootloader in flash that does a waitpne/waitpeq for a serial edge.
Only suggestion is that if SPI Flash fails, then try SPI SD (simple boot, no FAT knowledge).
P63 RXD (SI = serial in)
P62 TXD (SO = serial out)
Flash... | SD...
P61 #CE | DI / CMD
P60 CLK | CLK
P59 (#HOLD) / SIO3 | #CS / D3-CD
P58 (#WP) / SIO2 | x / D2
P57 SO / SIO1 | x / D1
P56 SI / SIO0 | DO / D0
Note: The akward use of the CD pinout to permit possible use of quad mode CD at a later date.
1. This permits the later use of QUAD SPI without the need to swap pins/bits around when using QUAD SPI.
2. This also permits SD to be used on the same pins instead of (ie without) SPI.
3. When accessing Flash normally (not Quad mode),#HOLD=1. This is compatible with the SD Card being deselected. Only if Flash was determined not to be present would the SD Card be tried.
I'm sorry to ask this, but what is a simple SD Card boot process? I know we've been over this ten times, but nothing I recall is very simple.
Assumptions:
- in the block 0 (the MBR) you expect to find the partition table at 1BEh, 1CEh, 1DEh, 1EEh respectively for partition 1,2,3,4
- the partition entry you have inside at offsets 1,2 and 3 the absolute starting sector in 3 bytes and the partition type at offset 4
so eg for partition 1:
1) read block 0 (MBR)
2) check at offset 1C2h for partition type value (eg look for value A2h, 12h, 27h, ...)
3) if partition type matches take bytes 1BFh,1C0h and 1C1h and
4) start reading sequentially from the pointed sector in step 3. Eventually first bytes of data can contain how many longs (bytes/sectors) to read.
This means that partition 1 is P2 firmware and eventually partition2 will be a FAT32 that will be accessed later when the firmware is loaded and the prop will have proper drivers. ... or also partition 2 wil be used as raw space, without any FS up to the programmer needs.
No file system, no searches for anything, no filenames, just raw data, compatible with every OS, simple.
Edit: If you apply the same logic for spi flash than only the low level access changes between flash and SD, the reading code remain the same
Chip,
Precisely what Peter said. This method does not require any FAT16/32 or any other code. All normal formatting for FAT now defaults to 32KB sector blocks, so the boot sector can be set to a 32KB contiguous block. So nothing requires any knowledge of the format used on the SD Card so it becomes forward compatible.
The only thing to work out is
* the use of sector 0
* the offset in sector 0
* the identification string
* the sector number
I have used a number of different versions of SD software... Femto, a multiple version (need to lookup the names) and the latest is Kye's, which is what my current Prop OS is based upon. Key's handles SD and SDHC with FAT16 and FAT32. We don't care about the FAT, but handling the SD and SDHC cards allows for older and newer cards - the initialisation is slightly different. IIRC Peter only handles the newer SDHC cards.
To place the boot code file on the card is a simple matter of copying a file onto a formatted card. Then run a simple program (on a Prop or whatever) to copy the identification, start sector, and length, onto sector 0. It's a simple method with minimal risk.
Fwiw I have a Prop P8XBlade2 that does this with my OS. I sent Ken a board earlier this year. It only requires a micro SD card and the software copied to it. If/when you want to give it a try just drop me a line. Ken has my email or we can Skype/FaceTime/Viber. The boot code in EEPROM is preloaded and write protected with a link.
Please don't let others scare you off including SD booting. It is quite simple and works reliably - I have proved it!!!
My version of simple is to read a signature from sector 0, then the starting sector and number of sectors to load. I do this in Tachyon where I save a file that always has contiguous sectors and write the boot info to sector 0. All the boot loader has to do after initializing the SD is to read sector 0 for this info and then transfer the specified sectors to memory.
This is exactly what I explained:
the signature is the partition type (look this values in the table, is unlikely that you can confuse them with some other system) while the starting sector is contained in the partition entry (using 3 bytes) of the MBR. All contained in block 0 and by using standard offsets compatible with everything.
The MBR do not involve any File System hassle or knowledge. MBR has nothing to do with file systems, but helps to isolate the firmware (raw, not file system partition) from a second regular partition that can be used with file system (or not) later for application data storage. And makes the things transparent to any PC using any OS. And no OS will by mistake delete your firmware partition.
And to claryfy, the firmware partition is only unused, unformatted space reserved through the partition entry of the MBR, you simply read (or write, to update firmware) sequentially its blocks.
This do not require to write the file in newly formatted card, and its transparent to any sector size, fragmentation is not a problem: the firmware partition becomes exactly the same as the flash.
I haven't really followed the SD booting discussion, but I do hope that whatever is used is compatible with PCs and their OSes. I would like to be able to read and write files from my computer, and also to be able to boot the P2 from the same SD card. It seems like the way to do this is to use the standard MBR.
Yes, the MBR gives you the possibility to have up to 4 primary partitions. In this case the first is the firmware and the second the data storage that can be both or under file system (any) or again raw like a flash. And what is best is that everything you need (signature, firmware start sector) have already a defined/standard place in the MBR, you only need to use the right offsets of the first block.
This isolates your raw firmware partition from being used by other systems like PCs, it remains hidden. The other partition, the user/app data storage if used with file system recognized by PC can be then used from both the P2 and PC for data exchange.
Perhaps, if there is a potential need to exchange data with other MCU systems, the partition entry 4 can be used for firmware and leave the first to the filesystem because some SD frameworks (eg on many PIC development tools) honor the MBR but can access only the first partition under fat16/32. Again, this is only required if someone plan to eg store data from prop and then place the SD in a pic system to be read after.
If you use a single partition the only easier thing is to copy the firmware on because at this point it is a normal file. But you then still need to write its starting block somewhere, in block 0 thus making the MBR potentially non standard and requiring the same tool that can write the firmware image in the raw partition if used. At the same time you expose the firmware to being deleted on the PC because is one of the many files in the root. And you also don't have the benefit of easy upgrade because if you overwrite the firmware the new will not use the same blocks and you will still need the tool to update its starting point in the MBR.
I'm still not sure what direction to go. Am I right that Peter's and Cluso's method differs from dMajo's?
Do these cards use SPI command $03 to begin reading? I don't know how to access sectors, exactly, as opposed to addresses.
Today I will push the loader I'm working on further along and try to get a release out soon.
I would love for the chip to be able to boot straight off an SD card. That would be fantastic, in itself. That it could provide a big file system is another good thing. Perhaps we could have a 3- or 4-way Skype chat about this and get things ironed out, sometime in the next few days (or hours). My Skype name is "potkuripaa". I think two of you are in Australia and one of you may be in Europe. If we could all talk, this issue could probably move forward quickly.
I'm still not sure what direction to go. Am I right that Peter's and Cluso's method differs from dMajo's?
Chip, they are nearly the same in practice, but yes they are slightly different.
Peter and Cluso are for a single partition. In this case you copy the firmware file (image) on the formatted card (with file system) like any other file.
Then you need a "special" tool that, aware of filesystem, reads the file (firmware) starting point and write this information to block 0 so that the P2, unaware of file system, can read the starting point (by accessing block0) and than start reading blocks from the one pointed from block0.
Now while this facilitates the firmware transfer (simple PC copy to SD card) it still needs than a tool to write the required info to the block0.
But this method relies on the 32KB sectors for shorter firmware and on newly formatted SD for larger ones. Because otherwise, with larger firmware, you still risk file fragmentation and and this will require the file system awareness to follow the fragments.
Where Peter/Cluso's solution is meant to store the firmware start-point info (block0) is what is called MBR. During the years also the MBR has evolved to accomodate the various needs but the partition table has always remained the same and always it will be.
The partition table holds 4 records for up to four primary partitions info. Each record (partition entry) have the same structure and contains its starting and ending point together with its type information and few others.
Again where Peter/Cluso are saying to use a tool to write the firmware starting point I am of opinion to define a regular partition (eg n°4) but instead of formatting it just write sequentially the firmware in its space. The partition starting point will be the firmware beginning, no file system in this partition.
Before start booting, only for insurance check the partition type, if matches. Like Potatohead said you can use any tool (fdisk) to create the partitions. The tool that will write the image to partition 4 will change also its type value. Potatohead was for 02h, I am more for A2h, 12h, 27h, B0h or some other
A2h: Hard Processor System (HPS) ARM preloader
12h: configuration partition / diagnostics and firmware partition
27h: RooterBOOT kernel partition (no file system)
B0h: Boot-Star dummy partition
but any value that is uncommon can be used.
If this is not enough and you want other media/boot signatures other 8 bytes in the MBR can be used to store it, but I think that after matching the partition type (of partition 4) the system can safely start booting its code.
If you want dynamic firmware size you need to have its dimension (and perhaps its checksum) written in the first bytes because you can not determine it from the partition boundaries because it will be certainly greater.
Having the first stage loader in the rom the partition can contain the second stage loader and(or) the whole 512K image, this is up to the developer.
So for both (Peter/Cluso ans my) solutions the P2 have to read the block 0, retrive from it the firmware starting point and start booting from there. The difference is that in one case the firmware is a file between the others in the root of the only existing partition while in the other case it is just data in its dedicated partition isolated from the user's/application's one.
That BTW also means that a programmer while using or developing a FAT16/32 framework/driver (Kye's one for example) is unlikely that will by mistake destroy the firmware image/partition.
The only think to consider is that by booting from SD anyone will have easy access to firmware code. It is easier to put the SD into any PC rather than sniff/desolder a flash chip. But I am not expert in encryption/validation so I leave this concerns to others even if this is only an option and if to use an SD or a flash is again a developer consideration.
Today I will push the loader I'm working on further along and try to get a release out soon.
I would love for the chip to be able to boot straight off an SD card. That would be fantastic, in itself. That it could provide a big file system is another good thing.
Important detail : How much spare ROM Code Size do you have available, to fit this in ?
... All normal formatting for FAT now defaults to 32KB sector blocks, so the boot sector can be set to a 32KB contiguous block.
Does this mean SD boot is best managed as another 2 step process (like SPI Flash )
First step is to load a Single-Cog image that contains the real loader.
This avoids needing to know sizes, and placements, but a more useful loader (the stage 2 one) is likely to want to manage a choice of images, plus sizes from 2k to 512k, and that one can be FAT aware etc, as users decide.
The SD ROM loader should focus on loading that first, very compact loader, and I think that can fit within MBR ?
Does this mean SD boot is best managed as another 2 step process (like SPI Flash )
First step is to load a Single-Cog image that contains the real loader.
This avoids needing to know sizes, and placements, but a more useful loader (the stage 2 one) is likely to want to manage a choice of images, plus sizes from 2k to 512k, and that one can be FAT aware etc, as users decide.
The SD ROM loader should focus on loading that first, very compact loader, and I think that can fit within MBR ?
Yes and no.
If you use 32KB sectors that means that in the first one the first 512 bytes still must contains the MBR, even if GPT is used. In this case is a protective MBR while GPT starts in the second sector.
The unused space in the first sector (after the first 512 bytes can contain the prop firmware. But its size can be "sector dimension"-512bytes.
At this point if you are reading the sector 0 (the first 512 bytes) than you can use the LBA address of partition4 offset 1F6h(502), a long (4 bytes), that contain the absolute sector number where the partition starts. You start booting from there and this can be the whole 512KB image or more for future props. Or it can be a second stage loader. I think that dynamic image size is mandatory to allow this choice.
If in the partition table you use the status byte (at offset 1EEh(494) in the MBR) given eg a 100MB firmware partition size you can even have more firmware images and use the status byte as a pointer to the bootable one.
This allows for safe field upgrades over wireless or other kind of media. Some developers can prefer to use an non externally accessible uSD in place of a flash in their products so the fw upgrade will still be over the wire(less) rather than by removing the SD.
@Chip - if you want some sample SD boot timings and command/response flow charts I should be able to cobble something together as that would cut through all the SD access fluff. It is quite easy really though.
@Everyone - the boot process needs to be kept simple yet flexible and even if it were in a protected partition or is a normal file it can still be accessed as a raw sector surely? (the card is not aware of any filesystem) I have many years of experience with writing code for and writing to SD cards and I know it doesn't have to be complicated. A simple pointer and sector count is all we need to worry about in the bootloader. It doesn't know or care that FAT32 might have 32kB clusters or that the sector is in a protected partition etc.
In regards to SD FAT32 I have never found any fragmentation on any of my cards although this is common for hard drives running Windows. I use my own Tachyon code to scan cards in this regard even reporting back the SD card ID, CSD, and OCR etc.
Comments
And....
By cutting the loader down to 256 longs and halving the load/verify time to 14ms, the serial running in the background can admit 161 bytes at 115.2k baud into its 256-byte buffer, so that they can be processed later, in case the flash loader doesn't validate - so no host data is lost and both purposes are served.
The lingering issues are how long to initially wait for a host (0 or 100ms), and then how long to wait in case the flash loader doesn't verify (1 minute or infinity).
Maybe default to the most robust thing possible, with an initial fast boot path in there for those who need or want it.
This behavior can be circumvented or managed with outside circuits.
Other MCUs use some microseconds as the reset-exit delay, so 10us seems a good target for Fixed Hardware delays.
When exactly does the Crystal Start/stop ?
As the user controls the Crystal Config, they can add delays as needed.
I'm still unclear on if the First Flash load size can be User-Defined ?
Does this first flash partial load have to use the same security levels as later, higher level loads ?
With a Pin-Skip, a 10us hard reset delay, & user Size choice, the important time-to-pins-defined could be slashed from the present 20ms+
Another question is, can the smart pins be used for the SPI load, to push up the MHz there ?
( Times given indicate the SPI SW clock is below 2MHz, ~ 1.5MHz ? )
In contrast, 20MHz can stream 256 longs in 409us
SPI_CS pull-up? No - Wait for serial. Yes - SPI_CK pull-up? No - Wait 100ms for serial, attempt flash boot, if fail then wait for serial. Yes - Attempt flash boot, if fail then wait for serial.This way, a jumper on the SPI_CK pull-up can be used to reload a system that normally (jumper on) boots from flash immediately.
At this point, I've decided against a time-out on waiting for serial. Any objections?
Sounds good - what are the expected times now, from reset-release ?
Not sure if this is what you meant, by 'pull-up?', but it should be possible to support a (NOT Pull-Up) test that catches either Pull down, or floating, to mean PCBs can be simpler, just one resistor, with a series jumper.
Some rough numbers give
10k pullup, and highish 15pF total pin C == 10k*15p = 150ns
Leakage and lowish C == 1M*8p = 8 us
That indicates an impulse and then test after (say) 250~300ns would catch both cases.
Not sure what FPGA boards give, but should be easy to check. Higher nett C just needs a lower pull-up.
The wait for serial will be lowish power ?
How does this tree interact with the encryption option, and possible attacks ?
FWIR, Encrypt is an OTP fuse and once on, everything coming into the chip is then treated Encrypted ?
Default is Encrypt Off, which means non encrypt systems may have faster boot times ?
last time I checked the goal is to have it always encrypted, so no fuse is encrypted by key 0000000...
Mike
About how much power it would take to wait around indefinitely for a serial load, it's one cog running at 20MHz. It will probably take a few mA. Leakage is going to be ~1mA, anyway.
Chip's example here
http://forums.parallax.com/discussion/comment/1384233/#Comment_1384233
made me think there was a simple loader being included too ?
IMHO this is EXCELLENT.
If the user doesn't want to waste power while waiting for serial, then his option is to have a bootloader in flash that does a waitpne/waitpeq for a serial edge.
Only suggestion is that if SPI Flash fails, then try SPI SD (simple boot, no FAT knowledge).
P63 RXD (SI = serial in) P62 TXD (SO = serial out) Flash... | SD... P61 #CE | DI / CMD P60 CLK | CLK P59 (#HOLD) / SIO3 | #CS / D3-CD P58 (#WP) / SIO2 | x / D2 P57 SO / SIO1 | x / D1 P56 SI / SIO0 | DO / D0Note: The akward use of the CD pinout to permit possible use of quad mode CD at a later date.1. This permits the later use of QUAD SPI without the need to swap pins/bits around when using QUAD SPI.
2. This also permits SD to be used on the same pins instead of (ie without) SPI.
3. When accessing Flash normally (not Quad mode),#HOLD=1. This is compatible with the SD Card being deselected. Only if Flash was determined not to be present would the SD Card be tried.
Does this case then boot faster, because it can skip the De-Crypt ?
As a check, I see SPI Flash parts are now down to 10.4c/4k at Digikey, for 104MHz 4Mbit, SO8, and 14c for 120MHz TSSOP8.
That's right.
I'm sorry to ask this, but what is a simple SD Card boot process? I know we've been over this ten times, but nothing I recall is very simple.
Assumptions:
- in the block 0 (the MBR) you expect to find the partition table at 1BEh, 1CEh, 1DEh, 1EEh respectively for partition 1,2,3,4
- the partition entry you have inside at offsets 1,2 and 3 the absolute starting sector in 3 bytes and the partition type at offset 4
so eg for partition 1:
1) read block 0 (MBR)
2) check at offset 1C2h for partition type value (eg look for value A2h, 12h, 27h, ...)
3) if partition type matches take bytes 1BFh,1C0h and 1C1h and
4) start reading sequentially from the pointed sector in step 3. Eventually first bytes of data can contain how many longs (bytes/sectors) to read.
This means that partition 1 is P2 firmware and eventually partition2 will be a FAT32 that will be accessed later when the firmware is loaded and the prop will have proper drivers. ... or also partition 2 wil be used as raw space, without any FS up to the programmer needs.
No file system, no searches for anything, no filenames, just raw data, compatible with every OS, simple.
Edit: If you apply the same logic for spi flash than only the low level access changes between flash and SD, the reading code remain the same
Precisely what Peter said. This method does not require any FAT16/32 or any other code. All normal formatting for FAT now defaults to 32KB sector blocks, so the boot sector can be set to a 32KB contiguous block. So nothing requires any knowledge of the format used on the SD Card so it becomes forward compatible.
The only thing to work out is
* the use of sector 0
* the offset in sector 0
* the identification string
* the sector number
I have used a number of different versions of SD software... Femto, a multiple version (need to lookup the names) and the latest is Kye's, which is what my current Prop OS is based upon. Key's handles SD and SDHC with FAT16 and FAT32. We don't care about the FAT, but handling the SD and SDHC cards allows for older and newer cards - the initialisation is slightly different. IIRC Peter only handles the newer SDHC cards.
To place the boot code file on the card is a simple matter of copying a file onto a formatted card. Then run a simple program (on a Prop or whatever) to copy the identification, start sector, and length, onto sector 0. It's a simple method with minimal risk.
Fwiw I have a Prop P8XBlade2 that does this with my OS. I sent Ken a board earlier this year. It only requires a micro SD card and the software copied to it. If/when you want to give it a try just drop me a line. Ken has my email or we can Skype/FaceTime/Viber. The boot code in EEPROM is preloaded and write protected with a link.
Please don't let others scare you off including SD booting. It is quite simple and works reliably - I have proved it!!!
This is exactly what I explained:
the signature is the partition type (look this values in the table, is unlikely that you can confuse them with some other system) while the starting sector is contained in the partition entry (using 3 bytes) of the MBR. All contained in block 0 and by using standard offsets compatible with everything.
The MBR do not involve any File System hassle or knowledge. MBR has nothing to do with file systems, but helps to isolate the firmware (raw, not file system partition) from a second regular partition that can be used with file system (or not) later for application data storage. And makes the things transparent to any PC using any OS. And no OS will by mistake delete your firmware partition.
And to claryfy, the firmware partition is only unused, unformatted space reserved through the partition entry of the MBR, you simply read (or write, to update firmware) sequentially its blocks.
This do not require to write the file in newly formatted card, and its transparent to any sector size, fragmentation is not a problem: the firmware partition becomes exactly the same as the flash.
This isolates your raw firmware partition from being used by other systems like PCs, it remains hidden. The other partition, the user/app data storage if used with file system recognized by PC can be then used from both the P2 and PC for data exchange.
Perhaps, if there is a potential need to exchange data with other MCU systems, the partition entry 4 can be used for firmware and leave the first to the filesystem because some SD frameworks (eg on many PIC development tools) honor the MBR but can access only the first partition under fat16/32. Again, this is only required if someone plan to eg store data from prop and then place the SD in a pic system to be read after.
If you use a single partition the only easier thing is to copy the firmware on because at this point it is a normal file. But you then still need to write its starting block somewhere, in block 0 thus making the MBR potentially non standard and requiring the same tool that can write the firmware image in the raw partition if used. At the same time you expose the firmware to being deleted on the PC because is one of the many files in the root. And you also don't have the benefit of easy upgrade because if you overwrite the firmware the new will not use the same blocks and you will still need the tool to update its starting point in the MBR.
Linux and Mac OS have all we need on the command line. Windows fdisk can set a card up, leaving only a copy to the partition to do.
We can make simple 2GB images that users can apply to most any card with any number of image tools out there.
If it's a big card, two partitions will remain free.
Card image would have P2 partition and a large fat one. That works on anything.
I agree with 4th partition too, though it's easy to check by type. There are a few types free for us to use with no real conflicts.
My .02
Do these cards use SPI command $03 to begin reading? I don't know how to access sectors, exactly, as opposed to addresses.
Today I will push the loader I'm working on further along and try to get a release out soon.
I would love for the chip to be able to boot straight off an SD card. That would be fantastic, in itself. That it could provide a big file system is another good thing. Perhaps we could have a 3- or 4-way Skype chat about this and get things ironed out, sometime in the next few days (or hours). My Skype name is "potkuripaa". I think two of you are in Australia and one of you may be in Europe. If we could all talk, this issue could probably move forward quickly.
Ditto
I'll add you later today Chip.
Chip, they are nearly the same in practice, but yes they are slightly different.
Peter and Cluso are for a single partition. In this case you copy the firmware file (image) on the formatted card (with file system) like any other file.
Then you need a "special" tool that, aware of filesystem, reads the file (firmware) starting point and write this information to block 0 so that the P2, unaware of file system, can read the starting point (by accessing block0) and than start reading blocks from the one pointed from block0.
Now while this facilitates the firmware transfer (simple PC copy to SD card) it still needs than a tool to write the required info to the block0.
But this method relies on the 32KB sectors for shorter firmware and on newly formatted SD for larger ones. Because otherwise, with larger firmware, you still risk file fragmentation and and this will require the file system awareness to follow the fragments.
Where Peter/Cluso's solution is meant to store the firmware start-point info (block0) is what is called MBR. During the years also the MBR has evolved to accomodate the various needs but the partition table has always remained the same and always it will be.
The partition table holds 4 records for up to four primary partitions info. Each record (partition entry) have the same structure and contains its starting and ending point together with its type information and few others.
Again where Peter/Cluso are saying to use a tool to write the firmware starting point I am of opinion to define a regular partition (eg n°4) but instead of formatting it just write sequentially the firmware in its space. The partition starting point will be the firmware beginning, no file system in this partition.
Before start booting, only for insurance check the partition type, if matches. Like Potatohead said you can use any tool (fdisk) to create the partitions. The tool that will write the image to partition 4 will change also its type value. Potatohead was for 02h, I am more for A2h, 12h, 27h, B0h or some other
A2h: Hard Processor System (HPS) ARM preloader
12h: configuration partition / diagnostics and firmware partition
27h: RooterBOOT kernel partition (no file system)
B0h: Boot-Star dummy partition
but any value that is uncommon can be used.
If this is not enough and you want other media/boot signatures other 8 bytes in the MBR can be used to store it, but I think that after matching the partition type (of partition 4) the system can safely start booting its code.
If you want dynamic firmware size you need to have its dimension (and perhaps its checksum) written in the first bytes because you can not determine it from the partition boundaries because it will be certainly greater.
Having the first stage loader in the rom the partition can contain the second stage loader and(or) the whole 512K image, this is up to the developer.
So for both (Peter/Cluso ans my) solutions the P2 have to read the block 0, retrive from it the firmware starting point and start booting from there. The difference is that in one case the firmware is a file between the others in the root of the only existing partition while in the other case it is just data in its dedicated partition isolated from the user's/application's one.
That BTW also means that a programmer while using or developing a FAT16/32 framework/driver (Kye's one for example) is unlikely that will by mistake destroy the firmware image/partition.
The only think to consider is that by booting from SD anyone will have easy access to firmware code. It is easier to put the SD into any PC rather than sniff/desolder a flash chip. But I am not expert in encryption/validation so I leave this concerns to others even if this is only an option and if to use an SD or a flash is again a developer consideration.
Important detail : How much spare ROM Code Size do you have available, to fit this in ?
Does this mean SD boot is best managed as another 2 step process (like SPI Flash )
First step is to load a Single-Cog image that contains the real loader.
This avoids needing to know sizes, and placements, but a more useful loader (the stage 2 one) is likely to want to manage a choice of images, plus sizes from 2k to 512k, and that one can be FAT aware etc, as users decide.
The SD ROM loader should focus on loading that first, very compact loader, and I think that can fit within MBR ?
If you use 32KB sectors that means that in the first one the first 512 bytes still must contains the MBR, even if GPT is used. In this case is a protective MBR while GPT starts in the second sector.
The unused space in the first sector (after the first 512 bytes can contain the prop firmware. But its size can be "sector dimension"-512bytes.
At this point if you are reading the sector 0 (the first 512 bytes) than you can use the LBA address of partition4 offset 1F6h(502), a long (4 bytes), that contain the absolute sector number where the partition starts. You start booting from there and this can be the whole 512KB image or more for future props. Or it can be a second stage loader. I think that dynamic image size is mandatory to allow this choice.
If in the partition table you use the status byte (at offset 1EEh(494) in the MBR) given eg a 100MB firmware partition size you can even have more firmware images and use the status byte as a pointer to the bootable one.
This allows for safe field upgrades over wireless or other kind of media. Some developers can prefer to use an non externally accessible uSD in place of a flash in their products so the fw upgrade will still be over the wire(less) rather than by removing the SD.
@Everyone - the boot process needs to be kept simple yet flexible and even if it were in a protected partition or is a normal file it can still be accessed as a raw sector surely? (the card is not aware of any filesystem) I have many years of experience with writing code for and writing to SD cards and I know it doesn't have to be complicated. A simple pointer and sector count is all we need to worry about in the bootloader. It doesn't know or care that FAT32 might have 32kB clusters or that the sector is in a protected partition etc.
In regards to SD FAT32 I have never found any fragmentation on any of my cards although this is common for hard drives running Windows. I use my own Tachyon code to scan cards in this regard even reporting back the SD card ID, CSD, and OCR etc.