My previous post assumes a regular FAT32 formatted SD card, ie: how they are purchased. No disc imaging, no reformatting, not even any kludging aside from that one assumption.
So, not much checking at all. It does have to find the first partition from the MBR but if that is too much then about the only thing less is boot the MBR itself.
That's interesting. So the RaspberryPi can't parse the FAT filesystem from it's boot ROM? It just loads a second stage bootloader from some known location on the SD card? That certainly simplifies things!
Well, no, I wouldn't call it simple at all. The R-Pi has a full disc image that includes a preconfigured set of partitions that have a feely complex boot up process. You have to dedicate the whole card to it.
Well, no, the R-Pi has a full disc image that includes a preconfigured set of partitions that have a feely complex boot up process. You have to dedicate the whole card to it.
Yes but if they require a specially configured card it is almost certainly because at least part of the bootloader is located at a place on the card where they can find it without parsing the FAT filesystem structures. That first part that they load probably has FAT filesystem code in it that it uses to find all of the actual files required to boot Linux. We could do the same thing by loading a second-stage loader from a fixed location like several other people have suggested and have that load the actual user program. Then the user program could be in a FAT file anywhere in the filesystem since the second-stage loader would have FAT support.
Read what I've written in post 812, that one is simple. It gives a solid 14.5kB of easy block loading. From there the booting could do whatever it likes.
>solid 14.5kB of easy block loading
I like this SD-Boot option best.
Prop-Tool will have an option to save compiled code (that most likely will be a fat32 loader) to this area.
And after that the user can drag and drop any files it want to the SD card.
>solid 14.5kB of easy block loading
I like this SD-Boot option best.
Prop-Tool will have an option to save compiled code (that most likely will be a fat32 loader) to this area.
And after that the user can drag and drop any files it want to the SD card.
This would work but I think it might be even better to use this space for a second-stage loader that would find a user file to load in the FAT filesystem. 14.5k should be big enough to load a robust FAT filesystem loader. Then the user would only have to write this special loader into the reserved area once and after that would just have to change a file in the FAT filesystem to load their program, say "PROPBOOT.IMG" or something like that.
>even better to use this space for a second-stage loader that would find a user file to load in the FAT filesystem
That is what is said will happened most of the time, you will put a FAT loader there.
If this loader looks for a specific file to load more code, it's up to the programmer.
Unless you Format the SD card, this boot program will always be there.,
And there will be a demo in obex that do load PROPBOOT.IMG etc
But keep it open for any option, the Prop-II will load 14k of Spincode and run it.
That can be any code you want, maybe the 14k is all the space your full code uses and no more loading after that.
>even better to use this space for a second-stage loader that would find a user file to load in the FAT filesystem
That is what is said will happened most of the time, you will put a FAT loader there.
If this loader looks for a specific file to load more code, it's up to the programmer.
Unless you Format the SD card, this boot program will always be there.,
But keep it open for any option, the Prop-II will load 14k of Spincode and run it.
That can be any code you want, maybe the 14k is all the space your full code uses and no more loading after that.
I agree with everything you said except "spin code". I hope what will be loaded will be a generic binary image that could contain Spin code or Forth or C or Snobol or whatever the programmer desires. :-)
Complied Spin code is generic binary, Prop uses a bytecode interpreter and it not actually seeing Spin text
It can include Asm that is copied to a cog then Prop sees the bytecode for cognew.
But I'm pretty sure they are still gone keep the interpreter in ROM that copies itself to cog1 and then only run compiled Spin code.
And that SD boot loader will not be used to load the first cog. (right?)
complied Spin code is generic binary, Prop uses a bytecode interpreter and it not actually seeing Spin text
It can include Asm that is copied to a cog then Prop sees the bytecode for cognew.
I guess there isn't much choice on P2 since there is no Spin interpreter in ROM like on P1. Anyway, you're absolutely correct that the second-stage loader can be skipped for applications where the code doesn't change very often and will fit in the reserved space. That decision doesn't even have to be made now. The only thing we have to agree on is how the first-stage loader works. That's the only part that will actually be in ROM on the chip. There is lots of time to find a clever way to use the reserved space.
I don't see how a separate ROM couldn't easily be resized/remapped. Ie: I'm not seeing any advantage here.
Depends on how you define 'easily'.
a) To resize a separate ROM, you need to recompile the whole chip, an all-layer change. Not going to happen.
b) To resize a RAM patched as ROM, you just (usually) need the top-metal change, to only those cells affected. Mapping is already all done, you are just swapping how the cell reads.
Many consider b) easily/relatively cheap (in ROM contexts), and a) difficult and expensive and more risky.
SECURITY
I was thinking that it might be a possible weakness to allow downloading ability permitting brute force attacks.
I wonder if it might be prudent (since we have a number of user fuses) to identify 3 fuses, one each for Serial, Flash and SD. If the individual fuse was blown then that boot ability would be blocked. i.e. if the Serial fuse bit was blown, then serial downloading would not be possible. If all 3 fuses were blown then perhaps the Flash only should be allowed.
Any thoughts???
Yes, good idea, and I suggested almost exactly this somewhere above.
Such decisions need to be 'fuse safe' so a faulty fuse block, does not kill the whole device.
Besides security from attacks (and I'm thinking code replacement, as much as code-snooping) , you save the time and pin-action of the skipped boots.
b) To resize a RAM patched as ROM, you just (usually) need the top-metal change, to only those cells affected. Mapping is already all done, you are just swapping how the cell reads.
Does this mean that only one or two masks need to be changed? IIRC, that is the expensive part of the IC fabrication (making the 30(?) or so masks).
Many consider b) easily/relatively cheap (in ROM contexts), and a) difficult and expensive and more risky.
Heh, fair comment.
I know I hadn't use this reasoning but if a block of ROM is pre-existing, like the RAM is in this case, then as long as no more is required than what was fitted it will be very easy to re-purpose the code. Gotta compare apples with apples after all. :P
Very small delays, completely asynchronous. And remember this is all happening inside the one piece of silicon where there is no massive driver stages for hauling on a pin. The biggest delays are likely the trace lengths getting from one side of the chip to the other.
To emphasise my earlier point on it happens everywhere, an independent RAM block sees the same delays, albeit through a different decoder, as a ROM block would see.
Not quite. Chip has said separate ROM is slower, as it must be, as another MUX and address-decision stage is needed.
Every fractional ns counts - if you add ~400ps, you drop ~10MHz in clock speed.
Does this mean that only one or two masks need to be changed? IIRC, that is the expensive part of the IC fabrication (making the 30(?) or so masks).
Yep, layout is done. Only tiny little tweaks are allowed now. Like changing a 1 for a 0. No removing or adding of new cells. Well, almost none at least.
if a block of ROM is pre-existing, like the RAM is in this case, then as long as no more is required than what was fitted it will be very easy to re-purpose the code.
True, but you need to decide way-back in the design flow, just how much ROM you want, and you need to live with the speed-cost.
Given a tiny ROM is used only for first level boot, in Prop 2, this 'using fixed RAM' way makes sense.
Does this mean that only one or two masks need to be changed? IIRC, that is the expensive part of the IC fabrication (making the 30(?) or so masks).
All the ROMs I've worked with, are single-layer changes. So I'm guessing that is true in Prop 2.
Lower level masks tend to have tighter tolerances, and impact the overall specs, so all-layer changes are costly and much higher risk. ie a single upper layer change, is less costly than (Mask Cost/Divided by layers).
True, but you need to decide way-back in the design flow, just how much ROM you want, and you need to live with the speed-cost.
Given a tiny ROM is used only for first level boot, in Prop 2, this 'using fixed RAM' way makes sense.
I really ddo not understand the discussion about coustomer-provided ROMs.
Is not the ability to have a encrypted Flash to load exactly what a userRom could do? Each long you add to the size of the rom gets subtracted from the ram - so what gain you have with a larger rom? And does a user-rom NOT need the boot-code already there?
And what customer will pay for a custom-development 120.000++ setupcosts+ production-run to save 0.xx cents for a flash?
how many customers you think will be willing to do that and for what reason please?
If Chip decides to make the SD card boot loader he will just make it so that it looks for a special partition in the partition table that has the entire image in it. This is the only method that is feasable and that will work.
Yep, layout is done. Only tiny little tweaks are allowed now. Like changing a 1 for a 0. No removing or adding of new cells. Well, almost none at least.
But the masks aren't made yet, and I doubt are even designed . Those are only done when the chip design is finalized and it's time to manufacture the chip. That's what I was questioning: if you could take two chips that are identical except for their ROM, how many masks would be different?
If Chip decides to make the SD card boot loader he will just make it so that it looks for a special partition in the partition table that has the entire image in it. This is the only method that is feasable and that will work.
So are you saying that if Chip decides to add SD boot support that he will do it himself and that there is no point in any of us working on code to do it?
I think he is advising CHIP to do the partition version.
I do not know how you can create partitions on SDs in windows. It will allway just see the first partition. So changing that will require quite some amount, and you will not be able to save some propboot.bin simply on the SD to change your boot-program.
why not use the boot-function already there and supported by any PC/ARM/whatever provided in the FIRST partition. Do we need to boot something else from a propSD as a prop-program?
Ariba's Versions are workable and are correct to the standards of FAT32. and you allmost don't get SDs smaller then 8GB just look at walmart. So usually we will have 32K cluster. Good for any second stage loader.
or evanh's version ... You may need a prop-program to write the boot-loader on sd. ... but without prop you do not nedd to do that, so you will have one... Multi-OS support...
or skip all that and put a nice serial HUBMONITOR there...
You could always just keep a standard FAT boot loader in the boot partition instead of having to re-load your program into the boot partition every time you have an update.
I had a long 3 hour conversation with Chip about all the things needed to support SD/MMC cards. There isn't space for FAT support. Justing mounting the SD card takes up about all the space. This leaves just enough room to read through the partition table and find a special propeller image partition and load it.
Everyone seems to keep ignoring me when I say that they have to supply code to mount the SD card. It's more than 100 longs on the P1 and will not likely be shrink with P2 opcodes. I have written this code before...
I advised Chip that the best solution for this problem of limited space would be to just create a propeller image partition on the SD card and store the image there. Most SD cards have large tracks of unused space before and after partition 0 on the SD card. So, a small 128K image can be tucked in this unused space. If there was no space then FAT partition can just be shrunken to make space.
The Propeller 2 would then just scan the master boot record for the the partition identifier character and then, if found, load up all the sectors on the partition. It's an easy solution to implement in ASM and will work on every card that supports a MBR.
Unfortunately, not all SD cards have an MBR. Such cards will not be supported as they are no longer manufactured. All SD cards from the SD2.0 standard released many years ago must have an MBR. This allows support for every SD card sold since SDHC cards came out.
---
Getting file level access is a lot of work. There are too many things to do to fit all that within 150 longs. Shortcuts can be taken... but, then compatibility will be lost. Please remember that files are not contiguous. Reading a file requires interrogation of the FAT as the file is read. Additionally, FAT32 and FAT16 auto switching is not easy.
A separate partition is the way to go. If you Google "MBR" and look at the byte layout you will see how trivially simple it is to support booting from a partition.
I find this whole SD-card-boot discussion extremely off-putting. First of all, which high-roller volume purchasers would even use such a capability? My guess is very few. Moreover it won't be a selling point that will attract volume purchases. I see nothing at all wrong with booting from the lowest-common-denominator programmable device with the longest anticipated lifespan and lowest pin-count requirements, whether that be EEPROM or flash, and letting the users be responsible, from there, for more advanced booting. It would be a mistake, IMO, to let a vocal hobbyist minority dictate something as indelible as advanced ROM code. Parallax needs to recoup their investment from volume purchases. This discussion is not helping them meet that objective.
-Phil
Correct me if I am wrong, but volume purchasers already balk at having to purchase an eeprom for the propeller because it doesn't have internal eeprom? If a project is being designed with an SDCard anyway(and many are), not having to spec/buy/stock an spi flash would would be a definite plus to the bottom line if said SDCard booting is reliable.
Chip works this way. At each stage in the engineering, Chip has reached inflection points. At each of those points, he opened it up for discussion. Notice at each stage, he then goes off and does what he does, consulting where it makes sense, decisions made. He even highlighted how it's kind of ugly when it's out in public. It's ugly because of all the meta-discussion that goes on, it's valuable because of the ideas mixed in with that meta-discussion. Chip wants the ideas so he's got a broad base to pull from and factors to consider. Nobody dictates anything. There is advocacy, and he hears it, or not, considers the idea or not, and that's it. Anything else seen here is just ugly meta, and it happens, and it happens just because of how people work more than not, no worries.
Re: Partition.
This makes a lot of sense to me personally. A utility can be written that deals with the SD card, and one either is supported or not, and that's the end of it. One could very easily use Propellers to prep the SD cards too.
Having a few robust boot options makes sense to me. Lots of design options, lots of trade-offs and there just is no one size fits all, and who knows how things shift and change?
In any case, we are at one of the last inflection points. Settle the nature, location, size of the ROM. I am not willing to believe that somebody capable of designing such a fine CPU would be so easily swayed into some fatal position. Not going to happen folks. Whatever decision is made, it will be solid and well reasoned, well executed and workable. From there, it's software and product engineering problems, just like it was for P1. And like P1 all sorts of stuff is going to get thought up that was not even on the table during design and the robustness of the design will play out in great ways too. Have some faith people. Engage Chip as he asked, knowing he will do his thing as we know he can do very well.
Edit: You know, when he built P1, he more or less worked alone on many aspects of it as he didn't have a community of people who "get it" back then. Sometimes it would get shown off to a mere, "meh" which made things difficult, but he went ahead and just did it anyway. Something I find amazing frankly.
Now that we are here, and we've proven the design out, he's essentially got a pool of people to actually discuss Propeller stuff with! That's a good thing for everybody, Chip included, which is why it's happened as it has so far.
Comments
So, not much checking at all. It does have to find the first partition from the MBR but if that is too much then about the only thing less is boot the MBR itself.
Well, no, I wouldn't call it simple at all. The R-Pi has a full disc image that includes a preconfigured set of partitions that have a feely complex boot up process. You have to dedicate the whole card to it.
I like this SD-Boot option best.
Prop-Tool will have an option to save compiled code (that most likely will be a fat32 loader) to this area.
And after that the user can drag and drop any files it want to the SD card.
That is what is said will happened most of the time, you will put a FAT loader there.
If this loader looks for a specific file to load more code, it's up to the programmer.
Unless you Format the SD card, this boot program will always be there.,
And there will be a demo in obex that do load PROPBOOT.IMG etc
But keep it open for any option, the Prop-II will load 14k of Spincode and run it.
That can be any code you want, maybe the 14k is all the space your full code uses and no more loading after that.
It can include Asm that is copied to a cog then Prop sees the bytecode for cognew.
But I'm pretty sure they are still gone keep the interpreter in ROM that copies itself to cog1 and then only run compiled Spin code.
And that SD boot loader will not be used to load the first cog. (right?)
Depends on how you define 'easily'.
a) To resize a separate ROM, you need to recompile the whole chip, an all-layer change. Not going to happen.
b) To resize a RAM patched as ROM, you just (usually) need the top-metal change, to only those cells affected. Mapping is already all done, you are just swapping how the cell reads.
Many consider b) easily/relatively cheap (in ROM contexts), and a) difficult and expensive and more risky.
Yes, good idea, and I suggested almost exactly this somewhere above.
Such decisions need to be 'fuse safe' so a faulty fuse block, does not kill the whole device.
Besides security from attacks (and I'm thinking code replacement, as much as code-snooping) , you save the time and pin-action of the skipped boots.
Does this mean that only one or two masks need to be changed? IIRC, that is the expensive part of the IC fabrication (making the 30(?) or so masks).
I know I hadn't use this reasoning but if a block of ROM is pre-existing, like the RAM is in this case, then as long as no more is required than what was fitted it will be very easy to re-purpose the code. Gotta compare apples with apples after all. :P
Not quite. Chip has said separate ROM is slower, as it must be, as another MUX and address-decision stage is needed.
Every fractional ns counts - if you add ~400ps, you drop ~10MHz in clock speed.
Yep, layout is done. Only tiny little tweaks are allowed now. Like changing a 1 for a 0. No removing or adding of new cells. Well, almost none at least.
Okay, we're looping now.
True, but you need to decide way-back in the design flow, just how much ROM you want, and you need to live with the speed-cost.
Given a tiny ROM is used only for first level boot, in Prop 2, this 'using fixed RAM' way makes sense.
All the ROMs I've worked with, are single-layer changes. So I'm guessing that is true in Prop 2.
Lower level masks tend to have tighter tolerances, and impact the overall specs, so all-layer changes are costly and much higher risk. ie a single upper layer change, is less costly than (Mask Cost/Divided by layers).
That wasn't a Prop2 topic.
Is not the ability to have a encrypted Flash to load exactly what a userRom could do? Each long you add to the size of the rom gets subtracted from the ram - so what gain you have with a larger rom? And does a user-rom NOT need the boot-code already there?
And what customer will pay for a custom-development 120.000++ setupcosts+ production-run to save 0.xx cents for a flash?
how many customers you think will be willing to do that and for what reason please?
Did anybody ever asked for a custom rom on Prop1?
confused
Mike
But the masks aren't made yet, and I doubt are even designed . Those are only done when the chip design is finalized and it's time to manufacture the chip. That's what I was questioning: if you could take two chips that are identical except for their ROM, how many masks would be different?
I found this an interesting read: http://www.nxp.com/about/corporate-social-responsibility/sustainability/semiconductor-manufacturing.html
I do not know how you can create partitions on SDs in windows. It will allway just see the first partition. So changing that will require quite some amount, and you will not be able to save some propboot.bin simply on the SD to change your boot-program.
why not use the boot-function already there and supported by any PC/ARM/whatever provided in the FIRST partition. Do we need to boot something else from a propSD as a prop-program?
Ariba's Versions are workable and are correct to the standards of FAT32. and you allmost don't get SDs smaller then 8GB just look at walmart. So usually we will have 32K cluster. Good for any second stage loader.
or evanh's version ... You may need a prop-program to write the boot-loader on sd. ... but without prop you do not nedd to do that, so you will have one... Multi-OS support...
or skip all that and put a nice serial HUBMONITOR there...
Enjoy!
Mike
Everyone seems to keep ignoring me when I say that they have to supply code to mount the SD card. It's more than 100 longs on the P1 and will not likely be shrink with P2 opcodes. I have written this code before...
I advised Chip that the best solution for this problem of limited space would be to just create a propeller image partition on the SD card and store the image there. Most SD cards have large tracks of unused space before and after partition 0 on the SD card. So, a small 128K image can be tucked in this unused space. If there was no space then FAT partition can just be shrunken to make space.
The Propeller 2 would then just scan the master boot record for the the partition identifier character and then, if found, load up all the sectors on the partition. It's an easy solution to implement in ASM and will work on every card that supports a MBR.
Unfortunately, not all SD cards have an MBR. Such cards will not be supported as they are no longer manufactured. All SD cards from the SD2.0 standard released many years ago must have an MBR. This allows support for every SD card sold since SDHC cards came out.
---
Getting file level access is a lot of work. There are too many things to do to fit all that within 150 longs. Shortcuts can be taken... but, then compatibility will be lost. Please remember that files are not contiguous. Reading a file requires interrogation of the FAT as the file is read. Additionally, FAT32 and FAT16 auto switching is not easy.
A separate partition is the way to go. If you Google "MBR" and look at the byte layout you will see how trivially simple it is to support booting from a partition.
Thanks,
Correct me if I am wrong, but volume purchasers already balk at having to purchase an eeprom for the propeller because it doesn't have internal eeprom? If a project is being designed with an SDCard anyway(and many are), not having to spec/buy/stock an spi flash would would be a definite plus to the bottom line if said SDCard booting is reliable.
Chip works this way. At each stage in the engineering, Chip has reached inflection points. At each of those points, he opened it up for discussion. Notice at each stage, he then goes off and does what he does, consulting where it makes sense, decisions made. He even highlighted how it's kind of ugly when it's out in public. It's ugly because of all the meta-discussion that goes on, it's valuable because of the ideas mixed in with that meta-discussion. Chip wants the ideas so he's got a broad base to pull from and factors to consider. Nobody dictates anything. There is advocacy, and he hears it, or not, considers the idea or not, and that's it. Anything else seen here is just ugly meta, and it happens, and it happens just because of how people work more than not, no worries.
Re: Partition.
This makes a lot of sense to me personally. A utility can be written that deals with the SD card, and one either is supported or not, and that's the end of it. One could very easily use Propellers to prep the SD cards too.
Having a few robust boot options makes sense to me. Lots of design options, lots of trade-offs and there just is no one size fits all, and who knows how things shift and change?
In any case, we are at one of the last inflection points. Settle the nature, location, size of the ROM. I am not willing to believe that somebody capable of designing such a fine CPU would be so easily swayed into some fatal position. Not going to happen folks. Whatever decision is made, it will be solid and well reasoned, well executed and workable. From there, it's software and product engineering problems, just like it was for P1. And like P1 all sorts of stuff is going to get thought up that was not even on the table during design and the robustness of the design will play out in great ways too. Have some faith people. Engage Chip as he asked, knowing he will do his thing as we know he can do very well.
Edit: You know, when he built P1, he more or less worked alone on many aspects of it as he didn't have a community of people who "get it" back then. Sometimes it would get shown off to a mere, "meh" which made things difficult, but he went ahead and just did it anyway. Something I find amazing frankly.
Now that we are here, and we've proven the design out, he's essentially got a pool of people to actually discuss Propeller stuff with! That's a good thing for everybody, Chip included, which is why it's happened as it has so far.