I think you don't need to reformate the SD card everytime to change the boot file. Just overwrite it with the new one, as long as it has the same name and the same size (or is smaller) the same sectors get overwritten. It may depend on the OS but I have done tests for this on Windows (I think it was XT then).
Files change size, files get deleted. We can't be sure without crafting the block writes. That is why a separate raw partition is the correct solution for consecutive blocks.
If fragmentation is unacceptable then we shouldn't be using files.
Existing checks (serial, eeprom/flash) then...
1. SD MBR boot as suggested by Peter (ID,pointer,length,CRCC,description)
2. SD FAT16/32 file: Search for filename "P2Boot.bin" in primary partition
3. Give up.
This has the advantage that if a new FAT system comes into play, the simplest method of using a pointer/length will work with almost anything including no file system!
If it is not there, then the FAT file system will be used to search for a file "P2Boot.bin" to boot the propeller.
Remember, in both these cases, the binary boot data can be a short stub as a bootloader for another file type.
Here we have the best solution of all. The existing system (whatever that entails) and if not found, the subsequent SD system of simplest first (becomes a failsafe for later if the SD formats change) of pointer/length, and finally a search for a specific filename to boot.
Note that in the simplest case, it really only needs to point to a short block of sectors. Those sectors can contain boot code which know about whatever file system is used on the SD. Both systems could point to the same file "P2Boot.bin" and this file could be supplied as a standard in a zip file just like you can format a complete Raspberry Pi Debian release.
IMHO we should be aiming to be able to dispense with the eeprom/flash part, only if desired.ie the eeprom/flash is an optional part in the BOM. Why include it if you have an SD card!!!
Please go with the Propeller Partition method. It's the simplest way possible (from a P2 ROM standpoint) that is still Windows compatible, and Prop Tool can be made to deal with partitioning and dd'ing for you. Filesystem reading, etc. is too complicated for something in ROM that can't have any bugs.
Another bonus of a Propeller Partition (that others have probably mentioned, sorry, didn't have time to read whole thread) is that the second stage boot loader isn't necessary - the entire program can fit in it - but is still possible.
If you want to load a program from a FAT filesystem, you can have a second stage boot loader in the Propeller Partition (that you only ever need to load once!) that understands FAT and knows how to find and load your boot file, even if it's fragmented, not the first file entry, or whatever.
MMC/SD boot support is nice, but is it necessary? Reading through this thread, it seems to me that there is considerable risk in adding this. I'd much rather the boot ROM be kept as simple as possible. Also, does the new A9 board even have an SD card slot to test this with?
Seairth,
I understand and agree in general with KISS.
The problem is that CF/SD has been a standard for a while now, and not having that as a first-class citizen seems like a Marketing 101 blunder in this day and age.
Being able to boot from EEPROM/Hyperflash because someone wants to manf. cards that will never have mass storage and they can source them cheaper?
Sure, thats fine.
However, Parallax should be looking to reduce BOM count wherever possible, as that cuts their cost and customer's as well.
P16 Proto-boards and P16 BOE's are undoubtedly going to have SD, as are most anything else thats not a simple LED toy. That being so, Parallax should expend some effort into making this as dead stupid simple for their users as possible.
Whether thats a Prop partition, or FAT, it doesn't much matter. As long as an expert here, or some new user in Timbuktu or a class of 8th graders can toss the SD into a card reader and read/manipulate it without yet another 'special' program required.
In the interim FPGA testing phase, Peter's workaround sounds like a good way to get SD booting going.
I just think going that direction as "THE" SD booting process of record is unorthodox, and messing about outside of its own sandbox so to speak. Seems far more risky than find and loading a P2BOOT.bin from a Prop or FAT partition.
Not sure why SD boot seems to be some sort of red-headed stepchild here.
Forget the BOM savings, and focus on how much time and grief Parallax can save all of its potential P16 customers by simply having it work 'out of the box'.
No arcane disk geometry knowledge needed, or SD pre-conditioning app/step that need be remembered?
Final 4) is chosen once the ROM resource is defines, and some actual field tests of candidates are done.
jmg,
The only problem with your suggestion is that SD keeps getting kicked down the ladder.
SD needs to work out of the box, because that is what that majority of people using have easy access to and will undoubtedly want or require in any board they buy or build.
I agree that an EEPROM option needs to be available for the commercial type of customer who wants an even lower BOM.
If space is such a problem that something like SD booting is not assured, then I think Parallax has misplaced priorities.
What in ROM is actually Necessary vs Nice To Have?
Nice To Have could be on the EEPROM, for those that need it, right?
So there is a P2BOOT.bin, and a P2ROM2.bin now.
Booting from SD is Necessary.
Having tables of Sin/Cos/Tan, thats Nice To Have. (If thats even in the P16 ROM).
From there, it's just fdisk and the dd command for the partition method.
Fdisk it to create the Propeller partition. Then copy the boot image into that partition with dd.
Of course, a running P1 or P2 system can prep SD cards pretty easy too. We could provide a number of options for people without a lot of hassles. They can pick.
How many KB of code would it take to locate a file, then load it? Is it really that difficult? We do have a 16KB ROM. Of course, big code is more likely be buggy than small code.
Some of us were concerned about locking FAT in as the boot option. Is that really a concern, or maybe just an option?
I like simple, of course, but maybe making it most simple for the customer is worth some trouble now. At least, I would like to have an idea of what it would take.
Honestly, if we've got the ROM, I'm all for "find the boot file and go" type capability. In "hot" we were concerned as it ate into RAM space. Maybe it's a lot different now.
I also think we really should do at least one low level block only option, just in case.
I have already outlined the simplest SD boot approach - as a minimum requirement - that does not require any special format or any format at all in fact, just the first 16 bytes of sector 0 to point to a sector to start loading from. Having the bootrom scan for a FAT32 file name is a bonus but also a time-consuming and possibly buggy one that I would have happen after the first MBR method.
How many KB of code would it take to locate a file, then load it? Is it really that difficult? We do have a 16KB ROM. Of course, big code is more likely be buggy than small code.
It's hard to say without writing the code. I guess it takes about 1k longs (= 4 kByte) PASM code to find the first sector of the file. If you also want to be able to read fragmented files with the clustor list in the FAT it will be more. The code for this is not written yet, and needs quite some effort to do it (mpark's Sphinx code may be a big help here, it's the only FAT code in PASM that I now, but it's not up to date with FAT32 and SDHC).
The minimal methode with a pointer in sector 0 or a search through the sectors takes under 200 longs.
This methode also allows to have the raw bootdata in a separate partition instead of a file, some people trust that more.
And the code is already written as P1-PASM code, with a Spin test code to try it: forums.parallax.com/discussion/comment/1122184/#Comment_1122184
(it's only the third time this link gets posted in this thread )
My P1 OS which is based on Kye's FATdriver takes
From PropTool: 3147 longs + 474 longs of VAR (4567 longs free)
This is a fully blown OS boot using a FAT file search, so its way more than it needs to be.
My older fsrw based I2C/SPI driver takes
From PropTool: 1262 longs + 410 longs of VAR (6516 longs free)
This OS is a simple FAT file search. Simplification is possible by removing the I2C support.
It also searches for AUTOEXEC.BAT to pick a new bin file.
I have attached the zip file for this version for reference.
Both versions are in the field and work successfully.
First use Peter's MBR method...
It's the safest, foolproof, shortest and fastest method. If SD cards change, it should always still work!
Next, use FAT16/FAT32 to locate a "P2BOOT.BIN" file...
It's the easiest and simplest for the user. No need to create a special partition, no need to write the MBR, just copy files to the SD card.
However, if a new FATxx is developed (very likely) that is incompatible, we have the fallback to Peter's MBR method.
I started converting my SD to a minimal PASM version although I haven't had much time to spend on it yet. But as a compromise to a full FAT32 aware bootloader in addition to the simple MBR pointer method how about a simple scan for a bootsig at the start of each cluster perhaps even aborting a full sector read (if possible) so that it can zip through. To assist this scan process I would always put the P2BOOT.BIN file as the first file on the card and always overwrite this from the Prop directly or if it was going to be updated from the PC then I believe that deleting the file before an update should make it end up in the same spot since it uses the FAT which will have these clusters available first. No need to scan for a directory entry, just rely upon the bootsig. The image would always have 512k +sig allocated for it even if the sig indicated that only 32k needed to be loaded for instance.
just my $0.02's worth, but IMHO, I think a FAT file searching/loading boot loader would be better for attracting newbies to P2, as having to mess around with flashing ROMs etc can be a show stopper for some, instead of just put this file on an SD card and plug it in, also means people have SD access from day 1 of coding P2.
just my $0.02's worth, but IMHO, I think a FAT file searching/loading boot loader would be better for attracting newbies to P2, as having to mess around with flashing ROMs etc can be a show stopper for some, instead of just put this file on an SD card and plug it in, also means people have SD access from day 1 of coding P2.
Agreed, this is the "marketing" angle I mentioned in another post plus it just makes good sense. But since I didn't know how much of the bootrom Chip would devote to it and I am mainly interested in just getting something simple during FPGA testing so therefore the first stage of reading the MBR for a start sector to the image also just makes good sense, it's a no-brainer and the absolute minimum that does not prevent higher level methods from being used. I use my SD with the network servers so it is also possible to drag and drop new firmware over FTP! Wonder if Chip would put that in the bootrom?
IIRC the RPi can have a file on the SD card with a filename kernel.??? (can't remember name ATM) that allows users to write "bare metal" code and bypass the OS altogether.
Being able to plug in a SD with a .spin file would be luxurious.
The Raspberry Pi appears to have enough FAT filesystem support internal to the chipset to at least scan the root directory of the first partition, find the filename bootcode.bin and load it's associated clusters. And apparently this binary is not ARM code. There is a couple more config files in that FAT volume but all the ARM binary code resides in the second partition I think.
Even with FAT support baked in, the Pi has still ended up being a multi-stage boot process. Whatever is done here, I doubt we'll be following the Pi's example.
Whether skilled or not, messing with the MBR of SD cards even for development testing is still more a pain than just being able to drop a file on the SD card.
But that's only my view on the matter, you other guys may be happy with jumping through those hoops to do this.
I mean I know once the boot is sorted then you can add files to SD to do a more complete FAT handler, it just seems an unnecessary step.
Comments
Files change size, files get deleted. We can't be sure without crafting the block writes. That is why a separate raw partition is the correct solution for consecutive blocks.
If fragmentation is unacceptable then we shouldn't be using files.
Isn't now the time to put something in there that could be fraught with peril?
Nope - which is why is not the first in the chain.
I think SD can be "added as proven".
Agreed, which is why the first boot items are the simplest, and lowest BOM cost, ones.
Existing checks (serial, eeprom/flash) then...
1. SD MBR boot as suggested by Peter (ID,pointer,length,CRCC,description)
2. SD FAT16/32 file: Search for filename "P2Boot.bin" in primary partition
3. Give up.
This has the advantage that if a new FAT system comes into play, the simplest method of using a pointer/length will work with almost anything including no file system!
If it is not there, then the FAT file system will be used to search for a file "P2Boot.bin" to boot the propeller.
Remember, in both these cases, the binary boot data can be a short stub as a bootloader for another file type.
Here we have the best solution of all. The existing system (whatever that entails) and if not found, the subsequent SD system of simplest first (becomes a failsafe for later if the SD formats change) of pointer/length, and finally a search for a specific filename to boot.
Note that in the simplest case, it really only needs to point to a short block of sectors. Those sectors can contain boot code which know about whatever file system is used on the SD. Both systems could point to the same file "P2Boot.bin" and this file could be supplied as a standard in a zip file just like you can format a complete Raspberry Pi Debian release.
IMHO we should be aiming to be able to dispense with the eeprom/flash part, only if desired.ie the eeprom/flash is an optional part in the BOM. Why include it if you have an SD card!!!
Another bonus of a Propeller Partition (that others have probably mentioned, sorry, didn't have time to read whole thread) is that the second stage boot loader isn't necessary - the entire program can fit in it - but is still possible.
If you want to load a program from a FAT filesystem, you can have a second stage boot loader in the Propeller Partition (that you only ever need to load once!) that understands FAT and knows how to find and load your boot file, even if it's fragmented, not the first file entry, or whatever.
Seairth,
I understand and agree in general with KISS.
The problem is that CF/SD has been a standard for a while now, and not having that as a first-class citizen seems like a Marketing 101 blunder in this day and age.
Being able to boot from EEPROM/Hyperflash because someone wants to manf. cards that will never have mass storage and they can source them cheaper?
Sure, thats fine.
However, Parallax should be looking to reduce BOM count wherever possible, as that cuts their cost and customer's as well.
P16 Proto-boards and P16 BOE's are undoubtedly going to have SD, as are most anything else thats not a simple LED toy. That being so, Parallax should expend some effort into making this as dead stupid simple for their users as possible.
Whether thats a Prop partition, or FAT, it doesn't much matter. As long as an expert here, or some new user in Timbuktu or a class of 8th graders can toss the SD into a card reader and read/manipulate it without yet another 'special' program required.
In the interim FPGA testing phase, Peter's workaround sounds like a good way to get SD booting going.
I just think going that direction as "THE" SD booting process of record is unorthodox, and messing about outside of its own sandbox so to speak. Seems far more risky than find and loading a P2BOOT.bin from a Prop or FAT partition.
Not sure why SD boot seems to be some sort of red-headed stepchild here.
Forget the BOM savings, and focus on how much time and grief Parallax can save all of its potential P16 customers by simply having it work 'out of the box'.
No arcane disk geometry knowledge needed, or SD pre-conditioning app/step that need be remembered?
Just my $0.02 worth.
jmg,
The only problem with your suggestion is that SD keeps getting kicked down the ladder.
SD needs to work out of the box, because that is what that majority of people using have easy access to and will undoubtedly want or require in any board they buy or build.
I agree that an EEPROM option needs to be available for the commercial type of customer who wants an even lower BOM.
If space is such a problem that something like SD booting is not assured, then I think Parallax has misplaced priorities.
What in ROM is actually Necessary vs Nice To Have?
Nice To Have could be on the EEPROM, for those that need it, right?
So there is a P2BOOT.bin, and a P2ROM2.bin now.
Booting from SD is Necessary.
Having tables of Sin/Cos/Tan, thats Nice To Have. (If thats even in the P16 ROM).
Again, just my $0.02 worth
From there, it's just fdisk and the dd command for the partition method.
Fdisk it to create the Propeller partition. Then copy the boot image into that partition with dd.
Of course, a running P1 or P2 system can prep SD cards pretty easy too. We could provide a number of options for people without a lot of hassles. They can pick.
If ROM space is limited, then I think the idea of writing the boot file, right after formatting the SD card is next best.
BTW: I think I'd write out the full 512kB to SD, even if not all to be loaded so that the size of boot code could increase and still have it work...
I like simple, of course, but maybe making it most simple for the customer is worth some trouble now. At least, I would like to have an idea of what it would take.
Now, it's FAT golf!
Honestly, if we've got the ROM, I'm all for "find the boot file and go" type capability. In "hot" we were concerned as it ate into RAM space. Maybe it's a lot different now.
I also think we really should do at least one low level block only option, just in case.
It probably does that already
Just need to port fsrw from p1 to p2
I think it could take less than 2kB of ROM
Also boot process could be fully explained in one sentence.
It's hard to say without writing the code. I guess it takes about 1k longs (= 4 kByte) PASM code to find the first sector of the file. If you also want to be able to read fragmented files with the clustor list in the FAT it will be more. The code for this is not written yet, and needs quite some effort to do it (mpark's Sphinx code may be a big help here, it's the only FAT code in PASM that I now, but it's not up to date with FAT32 and SDHC).
The minimal methode with a pointer in sector 0 or a search through the sectors takes under 200 longs.
This methode also allows to have the raw bootdata in a separate partition instead of a file, some people trust that more.
And the code is already written as P1-PASM code, with a Spin test code to try it:
forums.parallax.com/discussion/comment/1122184/#Comment_1122184
(it's only the third time this link gets posted in this thread
Andy
My P1 OS which is based on Kye's FATdriver takes
From PropTool: 3147 longs + 474 longs of VAR (4567 longs free)
This is a fully blown OS boot using a FAT file search, so its way more than it needs to be.
My older fsrw based I2C/SPI driver takes
From PropTool: 1262 longs + 410 longs of VAR (6516 longs free)
This OS is a simple FAT file search. Simplification is possible by removing the I2C support.
It also searches for AUTOEXEC.BAT to pick a new bin file.
I have attached the zip file for this version for reference.
Both versions are in the field and work successfully.
First use Peter's MBR method...
It's the safest, foolproof, shortest and fastest method. If SD cards change, it should always still work!
Next, use FAT16/FAT32 to locate a "P2BOOT.BIN" file...
It's the easiest and simplest for the user. No need to create a special partition, no need to write the MBR, just copy files to the SD card.
However, if a new FATxx is developed (very likely) that is incompatible, we have the fallback to Peter's MBR method.
What is all the fuss about ???
Searched the forum and found this:
http://forums.parallax.com/discussion/123866/minimal-sd-card-reader-code-if-possible
Looks like one can remove write and seek functions and get it down to 900+ longs...
But, I remember one that was only 2 kB... Have to search some more...
http://forums.parallax.com/discussion/140056/running-a-presentation-from-a-propeller-putting-a-bmp-in-vga/p5
Looks like he got it to 3.3 kB.
That may still have writing code in it, so maybe further reductions are simple.
Agreed, this is the "marketing" angle I mentioned in another post plus it just makes good sense. But since I didn't know how much of the bootrom Chip would devote to it and I am mainly interested in just getting something simple during FPGA testing so therefore the first stage of reading the MBR for a start sector to the image also just makes good sense, it's a no-brainer and the absolute minimum that does not prevent higher level methods from being used. I use my SD with the network servers so it is also possible to drag and drop new firmware over FTP! Wonder if Chip would put that in the bootrom?
Being able to plug in a SD with a .spin file would be luxurious.
Even with FAT support baked in, the Pi has still ended up being a multi-stage boot process. Whatever is done here, I doubt we'll be following the Pi's example.
But that's only my view on the matter, you other guys may be happy with jumping through those hoops to do this.
I mean I know once the boot is sorted then you can add files to SD to do a more complete FAT handler, it just seems an unnecessary step.