Andy: Yes, I found that the last 4 or 6 bytes plus 2 bytes of $00 are sometimes used immediately prior to the partition table for an identifier. So $1BE - 10bytes = $1B4. Therefore I thought that using an even 16byte block at $1A0 now seems the best allocation.
On most of the MBR/VBRs posted that contain some form of boot data, they almost never seem to run this long. Hence, make it near the end of the boot code. The start of the boot code varies depending of whether it is an MBR or VBR sector.
I realise this of course does not matter for our testing.
We seem to now have consensus that we should use a block within the MBR/VBR boot code section, rather than format our own partition, even if the end-user wants it to be in a partition (because it is just a matter of setting our pointer up in the bot section. 16 bytes seems to be the best allocation...4 bytes for some identifier (you used "BOOT"), 4 bytes for the sector pointer.
Currently you have a further 4 bytes for length. Lets discuss this over the next few days? We need Chip's input so we can make this like the flash boot if possible.
I agree that putting this boot code into a file is the simplest to maintain and change using a PC. Just remember, this is being done for commercial users. They will no doubt be preparing lots of identical SD cards. The main boot code is unlikely tochange, but rather load a filesystem that will in turn point to a filename.
2KB is currently loaded from flash ($1F8 cog longs, plus 8 longs of SHA-256/HMAC signature).
For SD boot, we'd need to load much more than that because it must be FAT-aware.
Chip hasn't picked up on the split loader approach yet. From his comments in that post and the previous one I'm certain there isn't any particular size that matters. The 1kB load idea would work fine.
Cluso, have a read of his previous post to catch what Chips thoughts are.
There is one problem Kye has repeatedly mentioned but it hasn't truly been discussed. It's one of those oh **** problems.
The support required for switching a SD card into a high speed transfer mode is a bit complicated and will cost ROM space. But the real humdinger is when the Prop is reset without dropping the power ... The SD card is still operating in a high speed mode and there is no specification on initialising cards from such a state (I think I'm getting this right). The various specs have always assumed the card is freshly powered up.
There is two options as I see it (keeping it mildly legit):
- Either stay in the initial low speed SPI mode (and this may still have init problems on a reset) saving some ROM.
- Or have the Prop also control the power pin of the SD card (costing another I/O pin).
I've been thinking about the partition thing a lot.
The core idea is we just need a pointer to a known sequential 4K block on the SD. That block contains the code that has filesystem support, or just the ability to fetch the remainder of the code through additional pointers. Something like that anyway.
How about we make the pointer look like a partition table entry anyway? Just make it 16 bytes long. To the P2, the only significant bytes are the start block addresses. It doesn't care past that.
From there, one could do filesystem tricks, or just put raw block data on the card. Won't matter, right? On the other hand, if somebody does want to use the partition method to insure compliance in many cases, put the pointer in the partition table, where it will be seen as a partition table entry, which then is compliant across multiple uses of the SD card. And the bonus is file system tools that create partitions will create the pointer for us! That means being able to build a bootable SD on UNIX with nothing more than the standard commands and tools available.
No matter what we do, there is a prep stage performed either with a Propeller, or some short program that does the work to write the SD data properly. So there will be a program, and or a Propeller involved in the process, at least once.
Edit: Scratch that. On a UNIX, no program would ever be required.
From there, it's either partition or not.
With no partition, one could just hack in a file, and then just copy to it over and over.
With a partition, there is UNIX and not UNIX. On UNIX, one just writes the boot file to the partition with standard command line tools. On windows, the same program that sets the SD card up, would also do this to update the boot code from the boot file. In both cases, the boot code originates from the boot file produced by the assembler / compiler.
Others have mentioned that a partition could be used or not, it being optional. I totally agree, and realized my objection to this was compliant use of the SD card, not whether or not there is a partition.
My question is: How much bigger would the SD ROM have to be, if it fetched the MBR like being planned now, but instead of looking at one specific byte address for the boot block pointer, it would either look at a few of them, or just scan, until it finds one?
There would be a few use cases:
1. Non-compliant use. Just put the pointer somewhere "empty" and be done with it. It points to a file block, or "unused space", or the start of a partition, what ever makes sense.
2. Compliant use, first partition. Pointer is partition table entry proper, recognized by other operating systems.
3. Compliant use, other partition.
Seems to me, if we look for more than a byte pointer, we really do have all the options on the table! Everybody is happy, and we get the best of all worlds for a few longs.
The program could then:
1. Just look in two places. One is the specific address for the pointer, the other being the first partition table entry.
2. Look in multiple places. Specific address, and all four primary partition table entries.
3. Scan for it in the MBR, meaning there is no specific address, and the developer could put the pointer anywhere they think makes sense, including the partition table.
Whichever takes the fewest longs.
One disadvantage of #3 is the potential to have to go and look to see what somebody actually did with their pointer, because the scan would mean they could put it anywhere in the MBR.
A pointer then would be 16 bytes. I'll try and run the test suite on my SD cards.
Seems Chip has put this out there as a simple cost. Is SD card worth the RAM it will cost? Whatever gets done will cost RAM above $7FF in the P2. The majority cost is just doing it, which means:
We've got three basic camps:
1. Don't do it at all. RAM is worth more than SD.
2. Do it with minimal support, non-compliant. SD is worth more than RAM.
3. Do it compliant.
Camps two and three don't want to be mandated to doing it the "other way", so let's factor that out, resolving to "just do it"
FWIW, technically I'm in camp 1, as I don't see SD as necessary, and I'm worried about the SPI mode issue myself.
But, the door is open for it to happen if you guys all get that sorted out! Maybe you can.
If that all succeeds, then I really want to be able to build a compliant SD card boot using a partition, etc... so that it's recognizable across OSes and other devices proper, and don't want to force others into same. I understand all the reasons by just doing it in other ways.
I'm home again. Will try to run some SD cards this weekend, just to get you guys the data points. If it does happen, it needs to be robust.
Once the prop gets a hold of a small spot in mbr space, jump to full prop ram code (or even very small, just enough to load format/structure code to then look for prop full code) can be made, from any format/structure/style.
It all comes back to a simple, known first accessed, slow location, not deep into any data space. In that space is a pretty large amount of data space to do format recognition code, and boot like that also.
No matter what way you choose, a predetermined structure or not, it still starts like this.
Cluso, as far as i can tell, that location looks good, even that can redirect to a lower location in mbr to run more code (for you partition/formaters)...
Programs can do buildups, and if you don't power cycle the sdcard at the re-program stage of the prop, the sdcard will stay in previous state. Means you stay high speed. Like a hiccup in the matrix?
If we get SD at all, I just want to make a partition and boot with standard media / filesystem tools. A quick mkfs followed by a dd of the boot file and it's done! I also don't want to mandate that, because it doesn't have to be that way, and there are lots of people who either don't have those tools (windows), or who don't understand them, or don't care to use them. (programs and such to make bootable)
Can't we either have two places for the pointer, or scan for a pointer, or something so that fully compliant boot is possible?
This is a good last ditch effort for AFTER the initial jump to pointer location of cluso's. But then I think a more complicated unique identifier should be used, other than BOOT.
Like deadbeef
I suppose a complete volume scan for identifier could be used. From mbr 0 and on, make it scan assuming infinite size (max index)
Can't we either have two places for the pointer, or scan for a pointer, or something so that fully compliant boot is possible?
We'd have to agree on a partition type, and have the ROM code look for it, to solely rely on generic disc tools. The other issue is then the bootloader can't be inside a common partition type. So, two boot methods would be required. Not my first choice.
I don't see a big problem making a tool for writing the pointers and/or bootloader. Keeps the boot process simpler then.
As for relying on common disk tools, I don't see that. Just don't put the pointer in the partition table.
If there is a scan, it's gonna hit the lower one first, meaning people can do what they want with it. If they don't do that, then it's in the partition table, and the common tools are on the table. That's even true if two locations are defined, or multiple ones. Boot from lowest one.
Only needs to look like a partition table entry for that to work. Everybody is happy.
In the case of "boot from file", just put the pointer in low MBR. In the case of boot from partition, put it in the partition table. In the case of raw card, put it anywhere in MBR. The "boot from file" method also means "boot from common partition", so put that pointer in the lower MBR as well.
If the pointer looks like a partition table entry, it can serve as one, or just as a pointer, depending on whether or not it's actually in the partition table.
If there is a scan, it's gonna hit the lower one first, meaning people can do what they want with it. If they don't do that, then it's in the partition table, and the common tools are on the table. That's even true if two locations are defined, or multiple ones. Boot from lowest one.
If the pointer is formatted like a partition table entry, and we either:
Define a non partition table entry location in the MBR for it
, OR
A partition table entry location
, THEN
It's possible to create a compliant, bootable SD card, as well as the various non-compliant schemes mentioned, including boot from file.
This only requires the booter look in more than one location. At a minimum, it can look at the non-partition table location, and one partition table location. Or, it could just scan the MBR, until it finds it.
Adding an addtional test for a partition in the MBR needs about 18 instructions more, if you want to scan all 4 partition entries add another 3..4 instructions. The code makes a lot of pointer manipulation, and reads unaligned longs, so it can be shorter in P2 PASM.
Here is a possible code (snippet):
...
' check signature and startsector in sector 0 (FAT boot record or MBR)
begin mov blocknr,#0
mov ptra,hubbuf
call #rdblock 'read sector 0
mov ptra,hubbuf
add ptra,#400
rdlong data,ptra 'boot signature at 400 ?
cmp data,sigBOOT wz
if_nz jmp #mbrboot
add ptra,#4
rdlong blocknr,ptra 'get boot startsector at 404
add ptra,#4
rdlong x,ptra 'get number of contiguous boot blocks at 408
found mov ptra,hubbuf 'load sectors from file into hubram
:load call #rdblock
add blocknr,#1
djnz x,#:load
' the P2 code would boot cog0 here! (mybe from offset 16 to give some space for a sigature?)
' we just return the sector number to Spin on success in this test code
mov data,blocknr 'okay
fback sub hubbuf,#4 'feedback to Spin
wrlong data,hubbuf
halt jmp #halt
' check MBR partition table for a partition type 39
mbrboot add ptra,#50 'to MBR partition table
rdbyte data,ptra 'volume type of partition 1
cmp data,#39 wz 'a P2 boot partition?
if_nz neg data,#10
if_nz jmp #fback 'no signature or partition = err -10
add ptra,#4
call #getLong 'sector
mov blocknr,x
call #getLong 'size
jmp #found
getLong rdword x,ptra 'read a an unaligned long
add ptra,#2 '(will be shorter in P2 PASM)
rdword data,ptra
add ptra,#2
shl data,#16
or x,data
getLong_ret ret
It uses partition type 39. This is a hidden partition for Windows and is used also by others for raw boot code.
One problem is that you don't know if sector 0 is an MBR (there is no simple check for that), and we test only one byte (the partition type) to decide that bootcode exists. The signature of the shortcut is more safe, because a whole long is checked.
A partition will have the shortcut already in the MBR table, so you don't need to write it, but formating an SD card with partitions and store a file into a FATless partition is complicated. And this must be done everytime you change the bootcode.
A bootfile in the root directory and a shortcut in sector 0 is simpler for the user. If the shortcut is once there you just can replace the bootfile with new code. But you should not delete the bootfile without removing te shortcut, for example.
I've been thinking about the partition thing a lot.
The core idea is we just need a pointer to a known sequential 4K block on the SD. That block contains the code that has filesystem support, or just the ability to fetch the remainder of the code through additional pointers. Something like that anyway.
How about we make the pointer look like a partition table entry anyway? Just make it 16 bytes long. To the P2, the only significant bytes are the start block addresses. It doesn't care past that.
From there, one could do filesystem tricks, or just put raw block data on the card. Won't matter, right? On the other hand, if somebody does want to use the partition method to insure compliance in many cases, put the pointer in the partition table, where it will be seen as a partition table entry, which then is compliant across multiple uses of the SD card. And the bonus is file system tools that create partitions will create the pointer for us! That means being able to build a bootable SD on UNIX with nothing more than the standard commands and tools available.
No matter what we do, there is a prep stage performed either with a Propeller, or some short program that does the work to write the SD data properly. So there will be a program, and or a Propeller involved in the process, at least once.
It uses partition type 39. This is a hidden partition for Windows and is used also by others for raw boot code.
One problem is that you don't know if sector 0 is an MBR (there is no simple check for that), and we test only one byte (the partition type) to decide that bootcode exists. The signature of the shortcut is more safe, because a whole long is checked.
Andy
Because the partition entry in the partition table is 16 bytes as follows : STATUS, CHS Start, TYPE, CHS End, LBA Start, N°SECTORS
I am with Potatohead: it seems safer to use this area as pointer thus complying with the standards.
Ariba's code can symply use LBA information as Startsector and N°SECTORS as NumberOfSectors to boot. It doesn't matter if these paramenters point to a really hidden reserved partition (where the raw data resides) or to one file in the filesystem of another partition
For the signature you can verify:
- the MBR 55AAh @+1FEh
- the 32bit disk signature @+1B8h (May be one of your choice or part of the P2 encryption key: making the SD bootable only on a specific system)
- the partition STATUS @+1BEh (80h) & TYPE info @+1BEh+4h: I agree with 39 (27h) or perhaps better 127 (7Fh) "Alternative OS Development Partition Standard[3] - reserved for individual or local use and temporary or experimental projects"
In the early beginning every Prop can be used as tool to write the needed data on the SD (I mean the pointer informations and even the boot code in a file or fileless partition)
@Ariba: How much will the code increase by doing the same but only taking the pointers information from different area in the MBR?
That completely fails for all forms of partitionless booting, it also fails for volume reserved space booting, or mapped file booting. There isn't any boot time benefit for the ROM to look at the partition table at all.
Putting the pointer in the boot code space is perfectly valid, only one bootable machine is expected to be supported. Put simply, the pointer method is both more flexible and just as robust.
The only benefit that can be gleaned is the use of generic disc tools, plural, instead of having a special Prop installer tool. If the pointers are pointing into a custom partition then also filling that out in the partition table is sensible, otherwise it's not a partition at all.
I'm not keen on having two methods when one covers both cases.
Because the partition entry in the partition table is 16 bytes as follows : STATUS, CHS Start,TYPE, CHS End, LBA Start, N°SECTORS
I am with Potatohead: it seems safer to use this area as pointer thus complying with the standards.
Ariba's code can symply use LBA information as Startsector and N°SECTORS as NumberOfSectors to boot. It doesn't matter if these paramenters point to a really hidden reserved partition (where the raw data resides) or to one file in the filesystem of another partition
For the signature you can verify:
- the MBR 55AAh @+1FEh
- the 32bit disk signature @+1B8h (May be one of your choice or part of the P2 encryption key: making the SD bootable only on a specific system)
- the partition STATUS @+1BEh (80h) & TYPE info @+1BEh+4h: I agree with 39 (27h) or perhaps better 127 (7Fh) "Alternative OS Development Partition Standard[3] - reserved for individual or local use and temporary or experimental projects"
In the early beginning every Prop can be used as tool to write the needed data on the SD (I mean the pointer informations and even the boot code in a file or fileless partition)
@Ariba: How much will the code increase by doing the same but only taking the pointers information from different area in the MBR?
Actually evanh, the one method doesn't cover the "wants compliant boot media" case at all. Having at a minimum two pointer locations means not taking ANY option off the table. P2's would boot under a wide variety of SD card format scenarios. Given we really don't know how the feature will end up being used, it seems foolish to ship a device that cannot boot in a standards compliant way. Anyone who actually understands how disks work at that level is going to ask, "why?"
On the other hand, two pointer locations makes perfect sense! All kinds of hacks are possible, and those that want to employ them can. I'm in favor of that AND compliant operation. I wouldn't personally hack the file system to get a boot file, but that doesn't mean it's not a perfectly fine thing to do. Why deny others from actually using the media in a standards based way, when there really isn't a solid reason for doing that?
Let's make it possible for compliant formatting to also work.
I've looked at a number of completed products that will boot from SD, and a partition is there pretty often. Some will boot from filesystem too. I don't know the reasons behind those choices, but I do know that a whole lot of stuff went with a boot partition, and where they did that, they did it in a standards compliant way.
Can't really do that, unless the code actually looks in the partition table. It only needs to look once at a minimum too. Personally, I don't care if it has to be a first partition, as resizing existing ones isn't a required feature, nor is there a lot of benefit to it, and some low level program could just shuffle the partition table entries around anyway. All of that is above the visibility of the P2, and up to whoever is prepping SD cards.
Is there some real problem with having a compliant boot setup? Seriously? Some anti Microsoft thing, or turf war deal? I have to wonder at this point.
Anyway, at a minimum, wouldn't it just be:
[fetch MBR]
1. Read non-partition table pointer data
2. Is it bootable?
3. Yes, prep block address, goto boot
4. Read first partition table entry pointer data
5. Is is bootable?
6. Yes, prep block address, goto boot
7. Fail.
Boot!
In terms of cost, it's not much. And there is the issue of whether or not it's even worth it at all, trading RAM for SD. If it is, the small cost of actually booting in a compliant way isn't significant. And all of that is assuming the mode issue gets sorted out.
Re: Two methods.
Well, if one pointer is used, and it's just somewhere in the MBR people think will be safe, then there still is the issue of sorting out what was done. Did the pointer point to a partition, some "let's hope it stays where we think it will" location in the file system, or maybe just "space we think won't get written", etc... There won't be one standard way of doing it. People will just do stuff, and that's OK. They get to do stuff, and we want them to do stuff.
Let's say there is a problem with those things. Could happen right? What's the standard, reference answer for booting? Do we pick a specific hack, or do we tell people if all else fails, use a standards compliant format for booting? Think of the data sheet.
"Now to boot your P2 from SD, you need to put this pointer here, insure your filesystem has a file that's written this way, then....." What reference boot method do we put in that datasheet? A FAT hack, or "just pick some space on the card, and we recommend...?" Maybe to be safe, we recommend a partition to hold the data, just to be safe and sure. Of course, one would ask, "why not read the partition table?", LOL!!
By contrast:
P2 offers two methods for booting SD card. One is the MBR pointer method, which can potentially be used in the following ways, which are entirely up to the user.
The other is a standard boot partition method, and here's the script for formatting the SD card, using standard tools, etc...?
Place BOOTSTRAP program in MBR and if its signature OK then RUN it.-->
If that BOOTSTRAP links by ROM code with already existing SPI driver in ROM code --- Then it can be much done in that STRAP program placed in MBR !!!!!!!!
Need only read MBR once and transfer to its program if Boot-able !!
Putting the pointer(s) in the code region is compliant. That area, 440? bytes of the MBR, is allocated for exactly this sort of thing. So, the ROM loads the bootloader, pointed to by the pointers, which in turn can scan the full range of tables if you like.
Let's say there is a problem with those things. Could happen right? What's the standard, reference answer for booting? Do we pick a specific hack, or do we tell people if all else fails, use a standards compliant format for booting? Think of the data sheet.
The datasheet will just be the ID and pointers at the specified bytes in the first block. The rest is up to the ready-made tools that insert the ready-made bootloaders. For most people it'll all be done as reference kits.
None of the methods I've listed are hacks, they are all compliant. The boot process up to bringing up the filesystem has no requirement to understand partitioning unless you are wanting to support chain booting of foreign equipment.
EDIT: The ID and pointers could be searchable in the first block I guess. Gives a little more flexibility for differing first block structures of partitionless volume formats.
Maybe it's best that SD just isn't a boot option at all. I'll bow out of this thing, and advocate we just do it with flash, so that there is enough support to boot with standard tools and filesystems, with no worries about the SD card state, etc...
That additional clean page of RAM is probably worth more anyway.
Comments
On most of the MBR/VBRs posted that contain some form of boot data, they almost never seem to run this long. Hence, make it near the end of the boot code. The start of the boot code varies depending of whether it is an MBR or VBR sector.
I realise this of course does not matter for our testing.
We seem to now have consensus that we should use a block within the MBR/VBR boot code section, rather than format our own partition, even if the end-user wants it to be in a partition (because it is just a matter of setting our pointer up in the bot section. 16 bytes seems to be the best allocation...4 bytes for some identifier (you used "BOOT"), 4 bytes for the sector pointer.
Currently you have a further 4 bytes for length. Lets discuss this over the next few days? We need Chip's input so we can make this like the flash boot if possible.
I agree that putting this boot code into a file is the simplest to maintain and change using a PC. Just remember, this is being done for commercial users. They will no doubt be preparing lots of identical SD cards. The main boot code is unlikely tochange, but rather load a filesystem that will in turn point to a filename.
http://forums.parallax.com/showthread.php?141706-Propeller-II&p=1120425&viewfull=1#post1120425
Chip hasn't picked up on the split loader approach yet. From his comments in that post and the previous one I'm certain there isn't any particular size that matters. The 1kB load idea would work fine.
Cluso, have a read of his previous post to catch what Chips thoughts are.
If 8 blocks are to be loaded then 8 pointers should be used.
The core idea is we just need a pointer to a known sequential 4K block on the SD. That block contains the code that has filesystem support, or just the ability to fetch the remainder of the code through additional pointers. Something like that anyway.
How about we make the pointer look like a partition table entry anyway? Just make it 16 bytes long. To the P2, the only significant bytes are the start block addresses. It doesn't care past that.
From there, one could do filesystem tricks, or just put raw block data on the card. Won't matter, right? On the other hand, if somebody does want to use the partition method to insure compliance in many cases, put the pointer in the partition table, where it will be seen as a partition table entry, which then is compliant across multiple uses of the SD card. And the bonus is file system tools that create partitions will create the pointer for us! That means being able to build a bootable SD on UNIX with nothing more than the standard commands and tools available.
No matter what we do, there is a prep stage performed either with a Propeller, or some short program that does the work to write the SD data properly. So there will be a program, and or a Propeller involved in the process, at least once.
Edit: Scratch that. On a UNIX, no program would ever be required.
From there, it's either partition or not.
With no partition, one could just hack in a file, and then just copy to it over and over.
With a partition, there is UNIX and not UNIX. On UNIX, one just writes the boot file to the partition with standard command line tools. On windows, the same program that sets the SD card up, would also do this to update the boot code from the boot file. In both cases, the boot code originates from the boot file produced by the assembler / compiler.
Others have mentioned that a partition could be used or not, it being optional. I totally agree, and realized my objection to this was compliant use of the SD card, not whether or not there is a partition.
My question is: How much bigger would the SD ROM have to be, if it fetched the MBR like being planned now, but instead of looking at one specific byte address for the boot block pointer, it would either look at a few of them, or just scan, until it finds one?
There would be a few use cases:
1. Non-compliant use. Just put the pointer somewhere "empty" and be done with it. It points to a file block, or "unused space", or the start of a partition, what ever makes sense.
2. Compliant use, first partition. Pointer is partition table entry proper, recognized by other operating systems.
3. Compliant use, other partition.
Seems to me, if we look for more than a byte pointer, we really do have all the options on the table! Everybody is happy, and we get the best of all worlds for a few longs.
The program could then:
1. Just look in two places. One is the specific address for the pointer, the other being the first partition table entry.
2. Look in multiple places. Specific address, and all four primary partition table entries.
3. Scan for it in the MBR, meaning there is no specific address, and the developer could put the pointer anywhere they think makes sense, including the partition table.
Whichever takes the fewest longs.
One disadvantage of #3 is the potential to have to go and look to see what somebody actually did with their pointer, because the scan would mean they could put it anywhere in the MBR.
A pointer then would be 16 bytes. I'll try and run the test suite on my SD cards.
Seems Chip has put this out there as a simple cost. Is SD card worth the RAM it will cost? Whatever gets done will cost RAM above $7FF in the P2. The majority cost is just doing it, which means:
We've got three basic camps:
1. Don't do it at all. RAM is worth more than SD.
2. Do it with minimal support, non-compliant. SD is worth more than RAM.
3. Do it compliant.
Camps two and three don't want to be mandated to doing it the "other way", so let's factor that out, resolving to "just do it"
FWIW, technically I'm in camp 1, as I don't see SD as necessary, and I'm worried about the SPI mode issue myself.
But, the door is open for it to happen if you guys all get that sorted out! Maybe you can.
If that all succeeds, then I really want to be able to build a compliant SD card boot using a partition, etc... so that it's recognizable across OSes and other devices proper, and don't want to force others into same. I understand all the reasons by just doing it in other ways.
I'm home again. Will try to run some SD cards this weekend, just to get you guys the data points. If it does happen, it needs to be robust.
On UNIX one would just mount the partition and write out all the blocks from the boot file on the OS. The partition is created only once.
On Windows, a program is needed. Just have the prep-program accept a file name, and it can also just put the boot code into the partition.
In any case, creating a boot partition is only done once, and only when / if a boot partition is employed.
Given You can write to any uninitiated partition from any OS system --- NO problems
It all comes back to a simple, known first accessed, slow location, not deep into any data space. In that space is a pretty large amount of data space to do format recognition code, and boot like that also.
No matter what way you choose, a predetermined structure or not, it still starts like this.
Cluso, as far as i can tell, that location looks good, even that can redirect to a lower location in mbr to run more code (for you partition/formaters)...
Programs can do buildups, and if you don't power cycle the sdcard at the re-program stage of the prop, the sdcard will stay in previous state. Means you stay high speed. Like a hiccup in the matrix?
Can't we either have two places for the pointer, or scan for a pointer, or something so that fully compliant boot is possible?
Like deadbeef
I suppose a complete volume scan for identifier could be used. From mbr 0 and on, make it scan assuming infinite size (max index)
https://en.wikipedia.org/wiki/Hexspeak
We'd have to agree on a partition type, and have the ROM code look for it, to solely rely on generic disc tools. The other issue is then the bootloader can't be inside a common partition type. So, two boot methods would be required. Not my first choice.
I don't see a big problem making a tool for writing the pointers and/or bootloader. Keeps the boot process simpler then.
As for relying on common disk tools, I don't see that. Just don't put the pointer in the partition table.
If there is a scan, it's gonna hit the lower one first, meaning people can do what they want with it. If they don't do that, then it's in the partition table, and the common tools are on the table. That's even true if two locations are defined, or multiple ones. Boot from lowest one.
Only needs to look like a partition table entry for that to work. Everybody is happy.
In the case of "boot from file", just put the pointer in low MBR. In the case of boot from partition, put it in the partition table. In the case of raw card, put it anywhere in MBR. The "boot from file" method also means "boot from common partition", so put that pointer in the lower MBR as well.
If the pointer looks like a partition table entry, it can serve as one, or just as a pointer, depending on whether or not it's actually in the partition table.
EDIT: Sure, yep, it can be done. Isn't too much more code.
Ah, lowest one what?
If the pointer is formatted like a partition table entry, and we either:
Define a non partition table entry location in the MBR for it
, OR
A partition table entry location
, THEN
It's possible to create a compliant, bootable SD card, as well as the various non-compliant schemes mentioned, including boot from file.
This only requires the booter look in more than one location. At a minimum, it can look at the non-partition table location, and one partition table location. Or, it could just scan the MBR, until it finds it.
Here is a possible code (snippet):
... ' check signature and startsector in sector 0 (FAT boot record or MBR) begin mov blocknr,#0 mov ptra,hubbuf call #rdblock 'read sector 0 mov ptra,hubbuf add ptra,#400 rdlong data,ptra 'boot signature at 400 ? cmp data,sigBOOT wz if_nz jmp #mbrboot add ptra,#4 rdlong blocknr,ptra 'get boot startsector at 404 add ptra,#4 rdlong x,ptra 'get number of contiguous boot blocks at 408 found mov ptra,hubbuf 'load sectors from file into hubram :load call #rdblock add blocknr,#1 djnz x,#:load ' the P2 code would boot cog0 here! (mybe from offset 16 to give some space for a sigature?) ' we just return the sector number to Spin on success in this test code mov data,blocknr 'okay fback sub hubbuf,#4 'feedback to Spin wrlong data,hubbuf halt jmp #halt ' check MBR partition table for a partition type 39 mbrboot add ptra,#50 'to MBR partition table rdbyte data,ptra 'volume type of partition 1 cmp data,#39 wz 'a P2 boot partition? if_nz neg data,#10 if_nz jmp #fback 'no signature or partition = err -10 add ptra,#4 call #getLong 'sector mov blocknr,x call #getLong 'size jmp #found getLong rdword x,ptra 'read a an unaligned long add ptra,#2 '(will be shorter in P2 PASM) rdword data,ptra add ptra,#2 shl data,#16 or x,data getLong_ret retIt uses partition type 39. This is a hidden partition for Windows and is used also by others for raw boot code.One problem is that you don't know if sector 0 is an MBR (there is no simple check for that), and we test only one byte (the partition type) to decide that bootcode exists. The signature of the shortcut is more safe, because a whole long is checked.
Andy
Because the partition entry in the partition table is 16 bytes as follows : STATUS, CHS Start, TYPE, CHS End, LBA Start, N°SECTORS
I am with Potatohead: it seems safer to use this area as pointer thus complying with the standards.
Ariba's code can symply use LBA information as Startsector and N°SECTORS as NumberOfSectors to boot. It doesn't matter if these paramenters point to a really hidden reserved partition (where the raw data resides) or to one file in the filesystem of another partition
For the signature you can verify:
- the MBR 55AAh @+1FEh
- the 32bit disk signature @+1B8h (May be one of your choice or part of the P2 encryption key: making the SD bootable only on a specific system)
- the partition STATUS @+1BEh (80h) & TYPE info @+1BEh+4h: I agree with 39 (27h) or perhaps better 127 (7Fh) "Alternative OS Development Partition Standard[3] - reserved for individual or local use and temporary or experimental projects"
In the early beginning every Prop can be used as tool to write the needed data on the SD (I mean the pointer informations and even the boot code in a file or fileless partition)
@Ariba: How much will the code increase by doing the same but only taking the pointers information from different area in the MBR?
Putting the pointer in the boot code space is perfectly valid, only one bootable machine is expected to be supported. Put simply, the pointer method is both more flexible and just as robust.
The only benefit that can be gleaned is the use of generic disc tools, plural, instead of having a special Prop installer tool. If the pointers are pointing into a custom partition then also filling that out in the partition table is sensible, otherwise it's not a partition at all.
I'm not keen on having two methods when one covers both cases.
That is valid as long You use TYPE number. that don't interfere with any other BOOT system.
On the other hand, two pointer locations makes perfect sense! All kinds of hacks are possible, and those that want to employ them can. I'm in favor of that AND compliant operation. I wouldn't personally hack the file system to get a boot file, but that doesn't mean it's not a perfectly fine thing to do. Why deny others from actually using the media in a standards based way, when there really isn't a solid reason for doing that?
Let's make it possible for compliant formatting to also work.
I've looked at a number of completed products that will boot from SD, and a partition is there pretty often. Some will boot from filesystem too. I don't know the reasons behind those choices, but I do know that a whole lot of stuff went with a boot partition, and where they did that, they did it in a standards compliant way.
Can't really do that, unless the code actually looks in the partition table. It only needs to look once at a minimum too. Personally, I don't care if it has to be a first partition, as resizing existing ones isn't a required feature, nor is there a lot of benefit to it, and some low level program could just shuffle the partition table entries around anyway. All of that is above the visibility of the P2, and up to whoever is prepping SD cards.
Is there some real problem with having a compliant boot setup? Seriously? Some anti Microsoft thing, or turf war deal? I have to wonder at this point.
Anyway, at a minimum, wouldn't it just be:
[fetch MBR]
1. Read non-partition table pointer data
2. Is it bootable?
3. Yes, prep block address, goto boot
4. Read first partition table entry pointer data
5. Is is bootable?
6. Yes, prep block address, goto boot
7. Fail.
Boot!
In terms of cost, it's not much. And there is the issue of whether or not it's even worth it at all, trading RAM for SD. If it is, the small cost of actually booting in a compliant way isn't significant. And all of that is assuming the mode issue gets sorted out.
Re: Two methods.
Well, if one pointer is used, and it's just somewhere in the MBR people think will be safe, then there still is the issue of sorting out what was done. Did the pointer point to a partition, some "let's hope it stays where we think it will" location in the file system, or maybe just "space we think won't get written", etc... There won't be one standard way of doing it. People will just do stuff, and that's OK. They get to do stuff, and we want them to do stuff.
Let's say there is a problem with those things. Could happen right? What's the standard, reference answer for booting? Do we pick a specific hack, or do we tell people if all else fails, use a standards compliant format for booting? Think of the data sheet.
"Now to boot your P2 from SD, you need to put this pointer here, insure your filesystem has a file that's written this way, then....." What reference boot method do we put in that datasheet? A FAT hack, or "just pick some space on the card, and we recommend...?" Maybe to be safe, we recommend a partition to hold the data, just to be safe and sure. Of course, one would ask, "why not read the partition table?", LOL!!
By contrast:
P2 offers two methods for booting SD card. One is the MBR pointer method, which can potentially be used in the following ways, which are entirely up to the user.
The other is a standard boot partition method, and here's the script for formatting the SD card, using standard tools, etc...?
And most reliable -- Same as all systems use
>
Place BOOTSTRAP program in MBR and if its signature OK then RUN it.-->
If that BOOTSTRAP links by ROM code with already existing SPI driver in ROM code --- Then it can be much done in that STRAP program placed in MBR !!!!!!!!
Need only read MBR once and transfer to its program if Boot-able !!
All other handling of BOOT are USERS problem !!
Just the pointers is all that's needed in there.
If it will be compatible with all others systems AND all SD formats --- My way is only one that are correct.
The datasheet will just be the ID and pointers at the specified bytes in the first block. The rest is up to the ready-made tools that insert the ready-made bootloaders. For most people it'll all be done as reference kits.
None of the methods I've listed are hacks, they are all compliant. The boot process up to bringing up the filesystem has no requirement to understand partitioning unless you are wanting to support chain booting of foreign equipment.
EDIT: The ID and pointers could be searchable in the first block I guess. Gives a little more flexibility for differing first block structures of partitionless volume formats.
Doesn't work for partitionless booting and we are wanting more than the 440 bytes available in the MBR.
That additional clean page of RAM is probably worth more anyway.