The code I posted uses FAT (like all the other solutions I've seen).
Right now it is just under 512 instructions.
But, a lot of that is checks and some is messaging.
I bet it could be whittled down to 256 instructions.
BTW: I'd say that if you want it to be a stand alone computer, then you really want to use FAT.
...And, I'm really glad you guys are concerned about this. Being able to boot from an SD card would be really clean. Hello, stand-alone bench computer!
Couldn't that also be done with a two-stage boot? I'm not necessarily opposed to baking in SD support (I've stayed out of the debate), but is it really necessary? The very fact that agreement hasn't been reached means to me that there is more than one way customers will desire to use an SD card. Why pigeonhole everyone into just one way do SD boot?
If you wanna boot from SD, chain load it from the in-SPI-flash loader.
The design was intended to permit chain loading with full authentication down the road. The ROM authenticates the chain loader in flash, which then loads user code from either flash or SD card to execute in the chip.
The design was intended to mimic the ring 0, 1, 2, 3 concept. ROM is ring 0, boot loader is ring 1, user code is ring 2. At each step of the way there are permissions dropped (keys are erased from RAM). By the time you get to ring 2, there are no crypto keys in RAM and the user code has no access to protected data.
...And, I'm really glad you guys are concerned about this. Being able to boot from an SD card would be really clean. Hello, stand-alone bench computer!
Couldn't that also be done with a two-stage boot? I'm not necessarily opposed to baking in SD support (I've stayed out of the debate), but is it really necessary? The very fact that agreement hasn't been reached means to me that there is more than one way customers will desire to use an SD card. Why pigeonhole everyone into just one way do SD boot?
Some people want to avoid having to add a SPI flash part to their BOM.
FAT16 for regular SD cards
FAT32 for SDHC
FAT64 ("exFAT") for SDXC
So why exactly do you want ROM code that reads SD cards to not know about FAT? What SD card is not FAT formatted? What devices use SD cards without them being FAT formatted (and thus not following the SD standard)?
I think it would be a HUGE mistake to have ROM code that boots from SD and have it NOT use FAT.
FAT16 for regular SD cards
FAT32 for SDHC
FAT64 ("exFAT") for SDXC
So why exactly do you not want ROM code that reads SD cards to know about FAT? What SD card is not FAT formatted? What devices used SD cards without them being FAT formatted (and thus not following the SD standard)?
I think it would be a HUGE mistake to have ROM code that boots from SD and have it NOT use FAT.
The idea of using a boot partition means you can have a FAT filesystem on the SD card or not. There are other filesystems people may use. I think Tachyon uses something other than FAT. SD cards under Linux or the Mac sometimes have different filesystems. I'm not sure if any of those would be usable on a P2 system though. It just seems silly to be tied to a particular filesystem when there is already a provision for supporting multiple partitions allowing a boot partition and a filesystem partition if you want.
FSRW and the code I posted handles both FAT16 and FAT32. Also does SD and SDHC.
I'm not sure about that differentiation between SD and SDHC, though.
Think you can put FAT32 on regular SD cards and maybe FAT16 on SDHC.
Just wouldn't make a lot of sense...
exFAT may be a problem though. Seems to be proprietary...
No, partition table starts at 1BE and ends at 1FD.
1AE is in the boot code area of very old systems (ms-dos). The last 6 bytes of the 16 Cluso proposed falls in the disk signature area that modern systems use and could be risky.
As opposite the 4 bytes starting at 1F6h corresponds exactly to the LBA address (start of partition) of partition entry n°4 in the partition table and thus the same pointer(location) can be used as partition boundary and pointer to the fw image. Otherwise the ones willing to use a hidden raw partition must maintain two locations.
The bootcode area can be used but not past address 1B7.
FAT16 for regular SD cards
FAT32 for SDHC
FAT64 ("exFAT") for SDXC
So why exactly do you want ROM code that reads SD cards to not know about FAT? What SD card is not FAT formatted? What devices use SD cards without them being FAT formatted (and thus not following the SD standard)?
I think it would be a HUGE mistake to have ROM code that boots from SD and have it NOT use FAT.
ROM code do not have to know FAT to load the firmware. The firmware if needed can have FAT drivers to access the SD
No, partition table starts at 1BE and ends at 1FD.
1AE is in the boot code area of very old systems (ms-dos). The last 6 bytes of the 16 Cluso proposed falls in the disk signature area that modern systems use and could be risky.
As opposite the 4 bytes starting at 1F6h corresponds exactly to the LBA address (start of partition) of partition entry n°4 in the partition table and thus the same pointer(location) can be used as partition boundary and pointer to the fw image. Otherwise the ones willing to use a hidden raw partition must maintain two locations.
The bootcode area can be used but not past address 1B7.
That is true of the "classic MBR" but not for the "modern MBR". Shouldn't P2 support the modern version? What is the objection to using a partition table entry?
Could we define the pointer location being a little-endian long (4 bytes) at absolute address 1F6h (502 dec) of SD's block 0.
Could we define an ID byte being at abbsolute address 1F2h (498 dec) with a defined value from one of this: B0h, A2h, 12h, 27h
** Could we optionally define absolute address 1EEh (494dec) as fw image pointer (for multi image cases, safe filed fw upgrades).
Could we define the following SD boot process:
1) read block 0 form SD
2) evaluate if its 1F2h address contains B0h value
3) start sequentially reading reading blocks from the one pointed by address 1F6h
3**) with image pointer: start sequentially reading blocks from the one pointed by: value(1F6h)+(value(1EEh)*4096)
4) first fw image data will contain the size to read.
I like the low level simplicity of this approach. Call it SD-Raw.
This is likely to have the smallest code footprint.
However, this also has use-cases, call this SD-FAT
...
I'm still with Roy, that a normally formatted SD card with a specific filename to boot from is the simplest approach. That's what the code I posted does.
This is also how most other devices with SD storage, work. Just with files, not MBR or partition tricks...
I also expect there to be other resources on the SD card besides just the boot code, BTW.
...And, I'm really glad you guys are concerned about this. Being able to boot from an SD card would be really clean. Hello, stand-alone bench computer!
Given the BOOT is already multi-tiered, what about doing both, SD_Raw then SD_Fat ?
ie it first tries absolute-address SD_Raw boot, and then tries FAT ?
How about this: What can be achieved in 64 instructions? In 128? In 256?
A Key question seems to be, how much space is there to do this ?
With 2k used and 16kB ROM, I though HubExec could be used > 2k, but Chip's comments suggest other limits below that 16kB ?
No, partition table starts at 1BE and ends at 1FD.
1AE is in the boot code area of very old systems (ms-dos). The last 6 bytes of the 16 Cluso proposed falls in the disk signature area that modern systems use and could be risky.
As opposite the 4 bytes starting at 1F6h corresponds exactly to the LBA address (start of partition) of partition entry n°4 in the partition table and thus the same pointer(location) can be used as partition boundary and pointer to the fw image. Otherwise the ones willing to use a hidden raw partition must maintain two locations.
The bootcode area can be used but not past address 1B7.
That is true of the "classic MBR" but not for the "modern MBR". Shouldn't P2 support the modern version? What is the objection to using a partition table entry?
Every MBR, old/present/future will support partition table.
And the partition table always start at 1BE and ends at 1FD. In this area there is space for 4 partition entries of 16 bytes each.
BTW even the new GPT (guid partition table) which starts in sector1 have the sector 0 filled as per MBR specification with partition entries of type EE.
My proposal to store the pointer of the fw image at 1F6 uses 4 of the 16 bytes of partition entry n°4 the LBA address precisely.
No, partition table starts at 1BE and ends at 1FD.
1AE is in the boot code area of very old systems (ms-dos). The last 6 bytes of the 16 Cluso proposed falls in the disk signature area that modern systems use and could be risky.
As opposite the 4 bytes starting at 1F6h corresponds exactly to the LBA address (start of partition) of partition entry n°4 in the partition table and thus the same pointer(location) can be used as partition boundary and pointer to the fw image. Otherwise the ones willing to use a hidden raw partition must maintain two locations.
The bootcode area can be used but not past address 1B7.
That is true of the "classic MBR" but not for the "modern MBR". Shouldn't P2 support the modern version? What is the objection to using a partition table entry?
Every MBR, old/present/future will support partition table.
And the partition table always start at 1BE and ends at 1FD. In this area there is space for 4 partition entries of 16 bytes each.
BTW even the new GPT (guid partition table) which starts in sector1 have the sector 0 filled as per MBR specification with partition entries of type EE.
My proposal to store the pointer of the fw image at 1F6 uses 4 of the 16 bytes of partition entry n°4 the LBA address precisely.
Again, what is the objection to using a partition table entry?
What is the objection to using a partition table entry?
Great question. Been wondering myself.
None. I am in favour to use it and store there all the pointers and identification codes.
I have just not called it with its name in my latest explanation because I do not want that people think that this requires partition and/or filesystem.
You can also have a single partition with fat and fw as file in the root and use the area pertaining to partition entry n°4 to store its pointer to allow the firmware to be loaded just with raw block access. Of course this approach involves the capability/warranty to write the fw contiguously, which involves a freshly initialized SD.
Given the BOOT is already multi-tiered, what about doing both, SD_Raw then SD_Fat ?
ie it first tries absolute-address SD_Raw boot, and then tries FAT ?
Given the reality to write the fw image on SD in contiguous way even if it reside in an filesystem area (eg root of single partition with FAT file system) and read it with raw block access using its starting block pointer why bother with filesystem aware loaders?
Once the fw loads if the user/application needs filesystem support it will be available through the drivers contained in the firmware.
We have a lot more than 256 instructions that could be used, but here's my concern:
A program bigger than 256 instructions is complex enough that we may not be able to know for sure that it is RELIABLE, given our time frame. We need something simple and provable SOON.
Given the reality to write the fw image on SD in contiguous way even if it reside in an filesystem area (eg root of single partition with FAT file system) and read it with raw block access using its starting block pointer why bother with filesystem aware loaders?
Once the fw loads if the user/application needs filesystem support it will be available through the drivers contained in the firmware.
Well, yes, I agree, and for most volume uses this would be fine.
(I have seen notes on reflow of SD cards, direct to PCBs)
However, I can see the appeal of the casual/student usage of 'Grab any SD', which was why I suggested supporting both.
Seems there is more than enough raw ROM space, but Chip's comments suggest maybe some lower ceiling too ?
No, partition table starts at 1BE and ends at 1FD.
1AE is in the boot code area of very old systems (ms-dos). The last 6 bytes of the 16 Cluso proposed falls in the disk signature area that modern systems use and could be risky.
As opposite the 4 bytes starting at 1F6h corresponds exactly to the LBA address (start of partition) of partition entry n°4 in the partition table and thus the same pointer(location) can be used as partition boundary and pointer to the fw image. Otherwise the ones willing to use a hidden raw partition must maintain two locations.
The bootcode area can be used but not past address 1B7.
That is true of the "classic MBR" but not for the "modern MBR". Shouldn't P2 support the modern version? What is the objection to using a partition table entry?
Every MBR, old/present/future will support partition table.
And the partition table always start at 1BE and ends at 1FD. In this area there is space for 4 partition entries of 16 bytes each.
BTW even the new GPT (guid partition table) which starts in sector1 have the sector 0 filled as per MBR specification with partition entries of type EE.
My proposal to store the pointer of the fw image at 1F6 uses 4 of the 16 bytes of partition entry n°4 the LBA address precisely.
Again, what is the objection to using a partition table entry?
Again, there is no objection in using partition entry. This is what I am advocating from the very early SD discussion.
Could we define the pointer location being a little-endian long (4 bytes) at absolute address 1F6h (502 dec) of SD's block 0.
Could we define an ID byte being at abbsolute address 1F2h (498 dec) with a defined value from one of this: B0h, A2h, 12h, 27h
** Could we optionally define absolute address 1EEh (494dec) as fw image pointer (for multi image cases, safe filed fw upgrades).
Could we define the following SD boot process:
1) read block 0 form SD
2) evaluate if its 1F2h address contains B0h value
3) start sequentially reading reading blocks from the one pointed by address 1F6h
3**) with image pointer: start sequentially reading blocks from the one pointed by: value(1F6h)+(value(1EEh)*4096)
4) first fw image data will contain the size to read.
And in all this I purposely never used any definition like MBR, partition, file, file-system, ...
Anyone is free tho call the above locations/process however he wants.
My pointer location and all the other used addresses are all within partition entry N°4 area, which is the last entry of the partition table, which is part of every MBR, which at the end correspond to sector (block) 0 of the SD.
No, partition table starts at 1BE and ends at 1FD.
1AE is in the boot code area of very old systems (ms-dos). The last 6 bytes of the 16 Cluso proposed falls in the disk signature area that modern systems use and could be risky.
As opposite the 4 bytes starting at 1F6h corresponds exactly to the LBA address (start of partition) of partition entry n°4 in the partition table and thus the same pointer(location) can be used as partition boundary and pointer to the fw image. Otherwise the ones willing to use a hidden raw partition must maintain two locations.
The bootcode area can be used but not past address 1B7.
That is true of the "classic MBR" but not for the "modern MBR". Shouldn't P2 support the modern version? What is the objection to using a partition table entry?
Every MBR, old/present/future will support partition table.
And the partition table always start at 1BE and ends at 1FD. In this area there is space for 4 partition entries of 16 bytes each.
BTW even the new GPT (guid partition table) which starts in sector1 have the sector 0 filled as per MBR specification with partition entries of type EE.
My proposal to store the pointer of the fw image at 1F6 uses 4 of the 16 bytes of partition entry n°4 the LBA address precisely.
Again, what is the objection to using a partition table entry?
Again, there is no objection in using partition entry. This is what I am advocating from the very early SD discussion.
Could we define the pointer location being a little-endian long (4 bytes) at absolute address 1F6h (502 dec) of SD's block 0.
Could we define an ID byte being at abbsolute address 1F2h (498 dec) with a defined value from one of this: B0h, A2h, 12h, 27h
** Could we optionally define absolute address 1EEh (494dec) as fw image pointer (for multi image cases, safe filed fw upgrades).
Could we define the following SD boot process:
1) read block 0 form SD
2) evaluate if its 1F2h address contains B0h value
3) start sequentially reading reading blocks from the one pointed by address 1F6h
3**) with image pointer: start sequentially reading blocks from the one pointed by: value(1F6h)+(value(1EEh)*4096)
4) first fw image data will contain the size to read.
And in all this I purposely never used any definition like MBR, partition, file, file-system, ...
Anyone is free tho call the above locations/process however he wants.
My pointer location and all the other used addresses are all within partition entry N°4 area, which is the last entry of the partition table, which is part of every MBR, which at the end correspond to sector (block) 0 of the SD.
Sounds good but is little-endian the standard way of specifying values in the MBR / partition table? I think the entry should be a valid partition table entry to avoid confusing other software that knows the MBR / partition table format.
Given the reality to write the fw image on SD in contiguous way even if it reside in an filesystem area (eg root of single partition with FAT file system) and read it with raw block access using its starting block pointer why bother with filesystem aware loaders?
Once the fw loads if the user/application needs filesystem support it will be available through the drivers contained in the firmware.
Well, yes, I agree, and for most volume uses this would be fine.
(I have seen notes on reflow of SD cards, direct to PCBs)
However, I can see the appeal of the casual/student usage of 'Grab any SD', which was why I suggested supporting both.
Seems there is more than enough raw ROM space, but Chip's comments suggest maybe some lower ceiling too ?
What you mean by 'Grab any SD'? Isn't already like this?
You can take a new SD from the store, transfer the FW on it as a file. The only additional step is to write the starting block of the fw file into block 0. And then boot.
BTW IMHO I will use also the first 512 bytes of the flash in the same way. Having a MBR also there. This will not change anything compared to like is now, but will add the opportunity to partition th flash and use a file system on it.
and the same decision-making procedure:
1) read block 0 form SD
2) evaluate if its 1F2h address contains B0h value
3) start sequentially reading reading blocks from the one pointed by address 1F6h
3**) with image pointer: start sequentially reading blocks from the one pointed by: value(1F6h)+(value(1EEh)*4096)
4) first fw image data will contain the size to read.
can be used for both the cases.
In this way, when using both, part of the flash can become internal storage while SD can be removable storage.
Sounds good but is little-endian the standard way of specifying values in the MBR / partition table? I think the entry should be a valid partition table entry to avoid confusing other software that knows the MBR / partition table format.
Yes, all the multibyte values in the MBR are little-endian.
Sounds good but is little-endian the standard way of specifying values in the MBR / partition table? I think the entry should be a valid partition table entry to avoid confusing other software that knows the MBR / partition table format.
Yes, all the multibyte values in the MBR are little-endian.
BTW: The FSRW port code I posted is fairly well commented (by my standards anyway).
It's also very linear. So, you can use the comments as a flow chart about how to boot using FAT on SD.
It's really not that complicated to use FAT. It doesn't add that much to the total code size considering how much simpler it makes things.
Also, I've tested it with every uSD card I own (about 6 or so) and it worked for all of them.
Comments
Right now it is just under 512 instructions.
But, a lot of that is checks and some is messaging.
I bet it could be whittled down to 256 instructions.
BTW: I'd say that if you want it to be a stand alone computer, then you really want to use FAT.
I can foresee this thread getting a lot longer...
Couldn't that also be done with a two-stage boot? I'm not necessarily opposed to baking in SD support (I've stayed out of the debate), but is it really necessary? The very fact that agreement hasn't been reached means to me that there is more than one way customers will desire to use an SD card. Why pigeonhole everyone into just one way do SD boot?
The design was intended to permit chain loading with full authentication down the road. The ROM authenticates the chain loader in flash, which then loads user code from either flash or SD card to execute in the chip.
The design was intended to mimic the ring 0, 1, 2, 3 concept. ROM is ring 0, boot loader is ring 1, user code is ring 2. At each step of the way there are permissions dropped (keys are erased from RAM). By the time you get to ring 2, there are no crypto keys in RAM and the user code has no access to protected data.
FAT16 for regular SD cards
FAT32 for SDHC
FAT64 ("exFAT") for SDXC
So why exactly do you want ROM code that reads SD cards to not know about FAT? What SD card is not FAT formatted? What devices use SD cards without them being FAT formatted (and thus not following the SD standard)?
I think it would be a HUGE mistake to have ROM code that boots from SD and have it NOT use FAT.
https://www.sdcard.org/developers/overview/capacity/index.html
I'm not sure about that differentiation between SD and SDHC, though.
Think you can put FAT32 on regular SD cards and maybe FAT16 on SDHC.
Just wouldn't make a lot of sense...
exFAT may be a problem though. Seems to be proprietary...
1AE is in the boot code area of very old systems (ms-dos). The last 6 bytes of the 16 Cluso proposed falls in the disk signature area that modern systems use and could be risky.
As opposite the 4 bytes starting at 1F6h corresponds exactly to the LBA address (start of partition) of partition entry n°4 in the partition table and thus the same pointer(location) can be used as partition boundary and pointer to the fw image. Otherwise the ones willing to use a hidden raw partition must maintain two locations.
The bootcode area can be used but not past address 1B7.
If you want to have a stand alone computer you need to boot a firmware which knows fat/16/32/... but the loader do not need this to boot the firmware.
Great question. Been wondering myself.
I like the low level simplicity of this approach. Call it SD-Raw.
This is likely to have the smallest code footprint.
However, this also has use-cases, call this SD-FAT
&
Given the BOOT is already multi-tiered, what about doing both, SD_Raw then SD_Fat ?
ie it first tries absolute-address SD_Raw boot, and then tries FAT ?
That brings us to
A Key question seems to be, how much space is there to do this ?
With 2k used and 16kB ROM, I though HubExec could be used > 2k, but Chip's comments suggest other limits below that 16kB ?
And the partition table always start at 1BE and ends at 1FD. In this area there is space for 4 partition entries of 16 bytes each.
BTW even the new GPT (guid partition table) which starts in sector1 have the sector 0 filled as per MBR specification with partition entries of type EE.
My proposal to store the pointer of the fw image at 1F6 uses 4 of the 16 bytes of partition entry n°4 the LBA address precisely.
I have just not called it with its name in my latest explanation because I do not want that people think that this requires partition and/or filesystem.
You can also have a single partition with fat and fw as file in the root and use the area pertaining to partition entry n°4 to store its pointer to allow the firmware to be loaded just with raw block access. Of course this approach involves the capability/warranty to write the fw contiguously, which involves a freshly initialized SD.
Once the fw loads if the user/application needs filesystem support it will be available through the drivers contained in the firmware.
A program bigger than 256 instructions is complex enough that we may not be able to know for sure that it is RELIABLE, given our time frame. We need something simple and provable SOON.
(I have seen notes on reflow of SD cards, direct to PCBs)
However, I can see the appeal of the casual/student usage of 'Grab any SD', which was why I suggested supporting both.
Seems there is more than enough raw ROM space, but Chip's comments suggest maybe some lower ceiling too ?
You can take a new SD from the store, transfer the FW on it as a file. The only additional step is to write the starting block of the fw file into block 0. And then boot.
BTW IMHO I will use also the first 512 bytes of the flash in the same way. Having a MBR also there. This will not change anything compared to like is now, but will add the opportunity to partition th flash and use a file system on it.
and the same decision-making procedure: can be used for both the cases.
In this way, when using both, part of the flash can become internal storage while SD can be removable storage.
Most people will just want to copy the boot file from PC to SD and not have to think too much about it...
It's also very linear. So, you can use the comments as a flow chart about how to boot using FAT on SD.
It's really not that complicated to use FAT. It doesn't add that much to the total code size considering how much simpler it makes things.
Also, I've tested it with every uSD card I own (about 6 or so) and it worked for all of them.