I see Winbond & Microchip have some 'gotcha' modes..
eg Sending cmd 0xAB on either W25Q or SST25WF does Release from Deep PowerDown
Also W25Q has a continuous read mode reset of 0ffffH
Looks like a couple of housekeeping commands are needed, thereafter the 0x03,00,00,00 should start read, no matter what state the FLASH was in, at P2 RESET.
Some designs use a header & tail tag longs, & read clocks until it reads the expected preamble, then extracts data.
Expanding on this set of necessary housekeeping commands, I see the Microchip SPI SRAM* uses 0x03 to read, and it has a
RSTIO 0xFF = Reset Dual and Quad I/O access
That is common with a mode-reset above.
* Some battery backed/tamper proof systems may want to run QuadSPI RAM. This also allows faster writes than Flash.
Would the 1) serial-boot stage need a Pin-skip option to best support this (& other modes) ?
1 - Either load and execute just block 0 as a boot block. This could then proceed to perform option 2 or 3 as a second stage boot.
2 - Or, read the partition table at block 0 to locate a Prop boot partition then load and execute a full boot image from that partition.
3 - Or, read block 0 as a block list of boot image distribution on the drive - Load and execute them as a full boot image.
For option 3, an uncompressed block list in the MBR, 440 bytes, can support up to 55kB boot image so I guess we'll be wanting run-length compression on that as a minimum. 440 bytes provides 88 run-length segments of up to 128kB per segment using 512-byte blocks, 32-bit LBA and 8-bit lengths. The build tool will have to handle potentially excessive fragmentation conditions somehow.
I'm more than happy with option 3 ... and it seems most people here are wanting a single stage loader. So, we can agree on this boot path then?
EDIT: Grrr, just reading up on LBA and noticed it's not 32-bit addressing, it's 48-bit! If we were to fully support LBA's address range that cuts down the number of run-length segments from 88 to 63 ish. Again, this is for SD cards greater than 2TB.
EDIT2: Ah, MBR only stores LBA addresses as 32-bit. That explains my confusion there. So, Option 2 and 3 are equal on the basis that support stops at 2TB limit. Option 3 gains back it's 88 segments again.
Option 2 should be able to find a big enough empty space without shifting/resizing an existing partition. There is potential for legacy CHS upsets here but that's only a problem for the build tool.
Option 3 can leave the partition table untouched. Some will be happy about this.
With option 3, that 88 segments will need to be reduced to accommodate the Prop boot signature within that first 440 bytes of block 0. 87 run-length segments gives us five bytes spare, 86 segments for ten bytes spare, and so on.
Pertaining to the Raspberry Pi boot process; So far I've not been able to identify exactly how it manages to find bootcode.bin other than the file must be placed in the first primary partition (FAT) as part of a fresh custom build of the whole SD card. I'll presume the Pi firmware does in fact scan the root, and only the root, directory to find the FAT clusters used by bootcode.bin.
I think this process is too complicated, personally.
This is what I would like to see, just a standard MBR that has boot info written to the first 16 byte:
SECTOR 0 MBR:
0000: P2BL (P2 bootloader signature)
0004: IMGSECT (start sector of P2 image)
0008: IMGSIZE (size of image in bytes )
000C: CRC (crc or checksum of image and opt header)
0010: METADATA (optional name/date/version of image etc)
Anything in addition to this is fine, but this would be the minimum requirement and this is easy for my system to save an image to a file and modify the MBR accordingly. On next boot it finds the header, loads the sectors and it's away.
If I now take that same SD card and put it in another system then away it boots, great!. I think if we want to it is easy enough to make a little command PC/linux utility where we can load an image to the card and then tell it the filename that we want bootable for those who may want to do it that way. Either way the image is a normal file that is assumed to be non-fragmented contiguous sectors which always seems to be the case and keeps things simple. Could we have a consensus for this minimum and reasonable requirement.
Peter,
The start and size can go in a generic partition entry of the partition table along with the Prop's reserved partition type. The rest goes in the custom Prop partition itself. This Prop partition will likely fit in the empty space between block 0 and the regular FAT32 partition without shifting anything. This would be option 2 from my list. This arrangement is what Potato has raised.
PS: It cannot be a normal file if only a start and length is given - only good for raw partitions - otherwise you're trying to override the filesystem management of the volume! Option 3 is for semi-normal files in that it still needs an install tool to build the file's booting block list.
EDIT: Hmm, I'll admit it's not impossible to place a file at consecutive blocks via special edits of the volume but it's highly frowned on when it's ordinarily mounted for write access. Option 3 covers this with very little issue.
Evan, I can't help but fell frustrated that you keep interfering and insisting upon restrictions because of "compliancies" and "filesystem managment". I know what I'm talking about and you can ask for your way if you want but don't twist and naysay this simple, practical, and efficient approach. If you are ever in doubt about finding a file in contiguous blocks that don't require "special edits" just try it yourself. Especially so if you format a card, and create a 512K P2BOOT.IMG file on it, you won't find it going hither and thither at all. Just fixed and contiguous. KISS.
When a memory card is 4GB or much more the PC filesystems do not seem to try to fragment a file by eating up every loose cluster but that is not the point though. We are not talking about running a PC O/S and system files off the card which would really mess it up with constant size updates and rewrites (plus cause an early demise).
Please just leave this workable proposal alone, I've done very similar before and it works.
Well, the beauty of using a partition, on *nix at least, is making a Propeller bootable SD card can be done with standard operating system tools. (Windows users can get this: http://www.chrysocome.net/dd and do the same thing, using fdisk)
That means we can publish some basic instructions and be done with it. No executable to provide or maintain, etc... It also means our stuff just works with the rest of the world out there, and it will just work because we will have used best practices, not shortcuts and or hacks too.
And all we need to do is read sector 0, and read two longs from each entry in the partition table, located at $1BE.
A Propeller Partition would point right to the boot image thus:
$0 = $00 --> Partition Status
$1 - $3 --> Legacy Cylinder, Head, Sector address, not needed to read
$4 = $20 -->Partition Type, Propeller
$5 - $7 = Legacy Cylinder, Head, Sector Address, not needed to read
$8 - $A --> Start Block Address (LBA) of Sector in little endian format
$C = $F = Size of Partition in blocks in little endian format
Really, we only need two values out of that!
Partition type ($20) and the block address.
That's it! Could be coded very simply, a few PASM instructions.
Entry 1, is it type $20? If so, get Start Block Address and boot.
Entry 2, is it type $20? etc... Just a little loop that reads, compares, adds, etc...
Everything else, including that nice header Peter has put here, goes in the partition, starting at that block address, and our stuff works in a simple, robust, best practice way.
I see Evan's post, and agree. Just write a partition entry pointing at whatever "free space" makes sense and go. Anything that can write some bytes, can write that partition table entry easy enough.
The legacy head, cylinder, sector addressing can be ignored with SD cards. (and most all media these days)
Could we have a consensus for this minimum and reasonable requirement?
I personally dislike suggesting filesystem support by referencing files when the boot process doesn't actually offer any filesystem support. All we have are blocks, and that is what partitions are for.
At the, "chip hooked to SD card for booting" level, blocks are it. That really should be a block specification, not a file based one. We don't support files in the P2 ROM. Secondly, unlike dedicated storage, like EEPROM and friends, an SD card has an existing, well defined way to specify blocks that works across many devices and systems.
Later stage boot code can and probably will offer real filesystem support, and of course file specifications and procedures make good sense. At this level, there isn't even a need to identify an SD card as bootable, just include the file, and if the system sees it, then the system loads it.
Remember, we are actually targeting "Pro" or "Commercial" users with stuff like this, and are including encryption, etc... too. If we want to appear serious, then we benefit by actually dealing with the SD card in a serious, best practice way.
For contiguous blocks, Potato's solution using a dedicated partition is the way.
For files - which simply cannot be counted on to be contiguous when written as ordinary files, bear in mind why files are used at all - then my option 3 is the way.
Option 3
- The boot image file's block list placed in the MBR, 440 bytes of space for this.
- With 5-byte run-length compression: Provides up to 88 run-length segments of up to 128kB per segment using 512-byte blocks, 32-bit LBA and 8-bit lengths.
- A segment entry in block 0 can be traded for a Prop boot signature or check code to indicate the block list is valid.
IMHO Peters suggestion is the simplest and most reliable way as it caters for future file systems which will by necessity follow as SD cards get bigger and bigger. Peters way is almost as I suggested a few years ago.
It does not require a separate partition, nor does it require FAT16, FAT32, or indeed any other file system, BUT IT ABSOLUTELY DOES NOT PRECLUDE ANY OF THESE !!!
So, you format an SD Card with your favourite program. Then if you want, make a special partition and make it active, or not. Then copy your boot file (which can be any contiguous size you require). I will address the contiguous info later.
Now you just run a simple program on your windows PC or *nix (may be blocked from direct writes on a Mac), or even a prop program, locate the first data sector of your file data using whatever format your SD card is in, and get its length, and write those details plus the extra info into the MBR sector 0. It can be a single load of 512KB (or 1MB for future) provided contiguous sectors are used - the special program can verify this.
On a 2GB FAT32 card, the default block size is 32KB. Therefore only 32KB can be guaranteed to be contiguous unless the card is newly formatted.
FWIW, the instructions I supply for my RamBlade creates a FAT32 formatted card with at least 32KB blocks. I then have them copy my 10 supplied 32MB CPM Disk files. These are guaranteed to then be contiguous. ZiCog accesses these files as disk drives, using the first sector of each file as the base for an 8MB contiguous block of sectors for each CPM disk file. CPM has its own file format within this standard FAT file. It's simple and effective.
Not only does it run CPM, but it also runs my PropOS using normal FAT files, and Catalina C programs and files. All can be maintained and run from the same SD card.
And I am sure I could add Peters Tachyon Forth System too!
My P1 contains a simple FAT boot loader in eeprom. But the boot could be simplified to use Peters Method. However I have never been able to get any consensus, only the flock saying it's not possible! I have been doing it for more than 4 years!
And a prop plug is not required to update any of my programs.
Here is a very quick session with SD cards and FAT32 in Tachyon. There's about 7GB worth of files on the card (all contiguous without anything special) but I open a file and manually enter some commands to create a P2BL boot signature and a sector pointer to the file. Of course I wouldn't make a source file the boot file but it demonstrates the method and all those commands would become just a single word: MAKEBOOT <myfile>
I would certainly expect any initial files written to be contiguous.
So, how do we get to the data of interest?
We check first if there is a pointer in sector 0 of the SD card at a certain position, otherwise we do a search for the first sector of the file that starts with some signature bytes.
The pointer allows faster boot times if needed (and is optional).
The tough guys can then just copy a bootfile to a fresh formated SD card.
The axious guys can use their custom made tools to make a safe place on the SD card and copy the bootfile there.
These are all simple methods. I'm struggling to see why some think otherwise. The advantage of staying compliant means we don't have to re-initialise the whole drive every time there is a change to the boot image. Doesn't that have some value? I certainly value it.
This seems to be one of those potentially defining moments where you find yourself at a crossroads. One direction takes you to the Wizard of Oz, the other, a house falls on your head.
In that spirit, I will be that guy, and go out on a limb and possibly fall flat on my face.
Honestly, this thread has gone from progressing nicely, to nerd fight at dawn with handbags at 10 paces.
I have to ask, if Chip et al really want to make the Prop2 FUN, as it was in the old days apparently, how much memory would it really take to have the P2 find the bootcode.bin file on the first FAT partition? Using unused space on Sector 0 or other 'hidden' spaces just seems like a hack, and throws me back 20 years.
I mean seriously, whether its the routers I work on or the RPi, I can take their SD card and throw it into any Win/Mac/Linux computer and just SEE the darn boot file!
Unless I am totally off my rocker, and this very explicitly limited FAT capability will just blow out the ROM, I think going ahead with some one-off, NIH-type boot process via co-opting the MBR is going to end up in the Prop2 seeing Hack-A-Day and embedded forums jump on the bandwagon and dismiss the P2 yet again.
Prioritize SD booting to that which is expected in this current millenium, and get rid of other stuff if you have to.
Who in their right mind doesn't get the fact that no one wants to have to include an EEPROM in the BOM If they don't have to?
The 'hype' so far about the P2 has been 16 32b Cores, massive bandwidth, etc, etc.
For it to be useful outside of simple LED blinkers, you are going to need mass storage for Real Programs and Real Data. And knowing that, most are going to be adding SD by default, no?
As we used to say, Duhhh!
Having to use special tools to access or manipulate a boot file is decidedly not-FUN.
Aside from the fact that Parallax now has to put up another Challenge and beg someone to build multiple sets of tools across the 3 main OS.
Some people here are totally seeing the bark and not the forests.
I mean, making it easy should help Parallax sell more, right. And thats what we all want, not just something that we personally like or prefer as expert users... ?
Yeah, let's do that, but don't forget the little header, or it won't work.
Seriously, I'm good either way, as I said above and in my initial post. There is a difference between a discussion and "no", and I never said no to anything this time around, save for the file blocks. That just doesn't seem to make sense to me personally.
Both the header and a partition table entry do make a lot of sense. Both, by the way, do require the user do something besides format 'n copy too. Both can be refreshed after a successful bootable format and header addition by merely copying a file too.
Maybe not, just pick one. That's good too. Fine by me.
Locate on FAT?
Anyone want to write that up and make sure it works always? The two methods above will work always. I think that's important.
I think Chip is looking for a procedure he can put in the ROM. Want "first file on FAT?"
The right thing to do is write up the procedure. Just as Peter and I did with the methods we thought made sense, and both of them do. Only a handful of instructions are needed.
As for Hack a Day, I've a prediction:
When they do get their hands on a P2, they will say a lot of stuff! Most of it will be good. They will mention that it boots from SD, and they will mention the BOM savings over the P1, or just as a savings, and they will compare it to flash and other things on chip... like they tend to do.
Later, no matter what we do, there will be an article on that SD, and it will have a nice set of instructions, just like the other articles involving media do. And those instructions will work too. One of us will probably submit it. I'm game and may do just that. Because the P2 is going to be pretty great.
People will read that, follow them, and boot from their SD.
Anyone want to boil that down to simple "read this block, operate on this value, read this block", etc...? And then check it?
IMHO, that is what it would take. Right now, Chip can take any of the low level ideas here from Andy, Peter, or myself and get it done in a handful of instructions. All of those ideas translate into a few basic things a user does to make the SD work.
Here is the work done last time, tested by a few of us:
I think you don't need to reformate the SD card everytime to change the boot file. Just overwrite it with the new one, as long as it has the same name and the same size (or is smaller) the same sectors get overwritten. It may depend on the OS but I have done tests for this on Windows (I think it was XT then).
I mean seriously, whether its the routers I work on or the RPi, I can take their SD card and throw it into any Win/Mac/Linux computer and just SEE the darn boot file!
I agree that making this RPi compatible ( which means similar in process and not in conflict with ) is in important aspiration. I think the ROM will allow a 'good' SD Boot.
Chip does not have to do all boot choices in one pass so it seems simplest to do the small, simple ones first, like
1) Serial (what Baud choices & MHz specs ? Skip choices ?)
2) i2c
3) SPI
and tbf
{4) SD}
Final 4) is chosen once the ROM resource is defines, and some actual field tests of candidates are done.
Who in their right mind doesn't get the fact that no one wants to have to include an EEPROM in the BOM If they don't have to?
Agreed, but users do need to be able to include an EEPROM if they want to.
I added i2c to the above sequence tree for BOM reasons - i2c is the smallest and cheapest configurable memory, and it allows more choice on higher level Boot & future proofing.
eg Suppose any issues are found later with the SD ROM code, that needs patching ? - best way to do that, is a smallest i2c memory.
Suppose any issues are found later with the SD ROM code, that needs patching ?
This is precisely why several of us are focused on basic, block level processes. There won't be trouble with those at the "find the boot file" level. We could experience trouble with SD cards, modes, types, and that's why Peter is asking for a simple process right now.
I agree with that request, however it gets done.
We need to test this idea out and do so on live SD cards, switching modes, powering down, warm booting, etc...
IMHO, that priority trumps higher level options, should someone go through and propose detail on how those might be done.
Right now, we have no way to boot from any persistent media. That is hindering testing, and certainly a roadblock for testing any secondary boot scenarios since we don't even have a primary boot path.
If Peter's itch can be scratched with a handful of instructions, then he is one step (and nearly the final step) to having a self hosting P2 running. That's pretty cool for a chip that is still in an FPGA. I also think it would be a big help toward testing many other things.
Andy has a solution that was tested out on P2 hot for a bit. If it is a small amount of ROM space for the first stage support, then let's get it out there to test.
If the primary options are really just handfuls of instructions for I2C and SPI (and therefore SD basic block boots) then it makes sense if Chip could spend some time getting this functionality into an FPGA image. That opens up a bunch of doors.
It's not going into an unchangeable mask. It can be tweaked and adjusted based on real experience, not just debate. We should at least have the same level of support for persistent booting as a P1 does at this point.
Let's not worry about support for hyperion-super-octo-quad-mega-TB devices...nobody has any of those and nobody is stalled in their testing because the P2 can't support it.
I'd say more but we have the children to think about!
MMC/SD boot support is nice, but is it necessary? Reading through this thread, it seems to me that there is considerable risk in adding this. I'd much rather the boot ROM be kept as simple as possible. Also, does the new A9 board even have an SD card slot to test this with?
Comments
Expanding on this set of necessary housekeeping commands, I see the Microchip SPI SRAM* uses 0x03 to read, and it has a
RSTIO 0xFF = Reset Dual and Quad I/O access
That is common with a mode-reset above.
* Some battery backed/tamper proof systems may want to run QuadSPI RAM. This also allows faster writes than Flash.
Would the 1) serial-boot stage need a Pin-skip option to best support this (& other modes) ?
For option 3, an uncompressed block list in the MBR, 440 bytes, can support up to 55kB boot image so I guess we'll be wanting run-length compression on that as a minimum. 440 bytes provides 88 run-length segments of up to 128kB per segment using 512-byte blocks, 32-bit LBA and 8-bit lengths. The build tool will have to handle potentially excessive fragmentation conditions somehow.
EDIT: Grrr, just reading up on LBA and noticed it's not 32-bit addressing, it's 48-bit! If we were to fully support LBA's address range that cuts down the number of run-length segments from 88 to 63 ish. Again, this is for SD cards greater than 2TB.
EDIT2: Ah, MBR only stores LBA addresses as 32-bit. That explains my confusion there. So, Option 2 and 3 are equal on the basis that support stops at 2TB limit. Option 3 gains back it's 88 segments again.
Option 2 should be able to find a big enough empty space without shifting/resizing an existing partition. There is potential for legacy CHS upsets here but that's only a problem for the build tool.
Option 3 can leave the partition table untouched. Some will be happy about this.
I think this process is too complicated, personally.
PS: Best detail I've found to date is https://github.com/raspberrypi/noobs/wiki/Standalone-partitioning-explained and https://www.raspberrypi.org/forums/viewtopic.php?f=50&t=13163
SECTOR 0 MBR:
0000: P2BL (P2 bootloader signature)
0004: IMGSECT (start sector of P2 image)
0008: IMGSIZE (size of image in bytes )
000C: CRC (crc or checksum of image and opt header)
0010: METADATA (optional name/date/version of image etc)
Anything in addition to this is fine, but this would be the minimum requirement and this is easy for my system to save an image to a file and modify the MBR accordingly. On next boot it finds the header, loads the sectors and it's away.
If I now take that same SD card and put it in another system then away it boots, great!. I think if we want to it is easy enough to make a little command PC/linux utility where we can load an image to the card and then tell it the filename that we want bootable for those who may want to do it that way. Either way the image is a normal file that is assumed to be non-fragmented contiguous sectors which always seems to be the case and keeps things simple. Could we have a consensus for this minimum and reasonable requirement.
The start and size can go in a generic partition entry of the partition table along with the Prop's reserved partition type. The rest goes in the custom Prop partition itself. This Prop partition will likely fit in the empty space between block 0 and the regular FAT32 partition without shifting anything. This would be option 2 from my list. This arrangement is what Potato has raised.
EDIT: Hmm, I'll admit it's not impossible to place a file at consecutive blocks via special edits of the volume but it's highly frowned on when it's ordinarily mounted for write access. Option 3 covers this with very little issue.
When a memory card is 4GB or much more the PC filesystems do not seem to try to fragment a file by eating up every loose cluster but that is not the point though. We are not talking about running a PC O/S and system files off the card which would really mess it up with constant size updates and rewrites (plus cause an early demise).
Please just leave this workable proposal alone, I've done very similar before and it works.
So, how do we get to the data of interest?
That means we can publish some basic instructions and be done with it. No executable to provide or maintain, etc... It also means our stuff just works with the rest of the world out there, and it will just work because we will have used best practices, not shortcuts and or hacks too.
And all we need to do is read sector 0, and read two longs from each entry in the partition table, located at $1BE.
A Propeller Partition would point right to the boot image thus:
$1 - $3 --> Legacy Cylinder, Head, Sector address, not needed to read
$4 = $20 -->Partition Type, Propeller
$5 - $7 = Legacy Cylinder, Head, Sector Address, not needed to read
$8 - $A --> Start Block Address (LBA) of Sector in little endian format
$C = $F = Size of Partition in blocks in little endian format
Really, we only need two values out of that!
Partition type ($20) and the block address.
That's it! Could be coded very simply, a few PASM instructions.
Entry 1, is it type $20? If so, get Start Block Address and boot.
Entry 2, is it type $20? etc... Just a little loop that reads, compares, adds, etc...
Everything else, including that nice header Peter has put here, goes in the partition, starting at that block address, and our stuff works in a simple, robust, best practice way.
I see Evan's post, and agree. Just write a partition entry pointing at whatever "free space" makes sense and go. Anything that can write some bytes, can write that partition table entry easy enough.
The legacy head, cylinder, sector addressing can be ignored with SD cards. (and most all media these days)
I personally dislike suggesting filesystem support by referencing files when the boot process doesn't actually offer any filesystem support. All we have are blocks, and that is what partitions are for.
At the, "chip hooked to SD card for booting" level, blocks are it. That really should be a block specification, not a file based one. We don't support files in the P2 ROM. Secondly, unlike dedicated storage, like EEPROM and friends, an SD card has an existing, well defined way to specify blocks that works across many devices and systems.
Later stage boot code can and probably will offer real filesystem support, and of course file specifications and procedures make good sense. At this level, there isn't even a need to identify an SD card as bootable, just include the file, and if the system sees it, then the system loads it.
Remember, we are actually targeting "Pro" or "Commercial" users with stuff like this, and are including encryption, etc... too. If we want to appear serious, then we benefit by actually dealing with the SD card in a serious, best practice way.
And that's all I'll say on it.
For files - which simply cannot be counted on to be contiguous when written as ordinary files, bear in mind why files are used at all - then my option 3 is the way.
Option 3
- The boot image file's block list placed in the MBR, 440 bytes of space for this.
- With 5-byte run-length compression: Provides up to 88 run-length segments of up to 128kB per segment using 512-byte blocks, 32-bit LBA and 8-bit lengths.
- A segment entry in block 0 can be traded for a Prop boot signature or check code to indicate the block list is valid.
I'm happy with either solution.
1. I really don't feel good about the "file block table" at all. (Option #3 above.)
2. Fetching boot image from a partition table entry is literally a handful of instructions. Read a few longs, get address, go!
3. Fetching from the header Peter put here is literally a handful of instructions. Read a few longs and go!
Let's not do #1.
I've said all I'm gonna say about #2 & #3. Not up to me. What happens, happens.
It does not require a separate partition, nor does it require FAT16, FAT32, or indeed any other file system, BUT IT ABSOLUTELY DOES NOT PRECLUDE ANY OF THESE !!!
So, you format an SD Card with your favourite program. Then if you want, make a special partition and make it active, or not. Then copy your boot file (which can be any contiguous size you require). I will address the contiguous info later.
Now you just run a simple program on your windows PC or *nix (may be blocked from direct writes on a Mac), or even a prop program, locate the first data sector of your file data using whatever format your SD card is in, and get its length, and write those details plus the extra info into the MBR sector 0. It can be a single load of 512KB (or 1MB for future) provided contiguous sectors are used - the special program can verify this.
On a 2GB FAT32 card, the default block size is 32KB. Therefore only 32KB can be guaranteed to be contiguous unless the card is newly formatted.
FWIW, the instructions I supply for my RamBlade creates a FAT32 formatted card with at least 32KB blocks. I then have them copy my 10 supplied 32MB CPM Disk files. These are guaranteed to then be contiguous. ZiCog accesses these files as disk drives, using the first sector of each file as the base for an 8MB contiguous block of sectors for each CPM disk file. CPM has its own file format within this standard FAT file. It's simple and effective.
Not only does it run CPM, but it also runs my PropOS using normal FAT files, and Catalina C programs and files. All can be maintained and run from the same SD card.
And I am sure I could add Peters Tachyon Forth System too!
My P1 contains a simple FAT boot loader in eeprom. But the boot could be simplified to use Peters Method. However I have never been able to get any consensus, only the flock saying it's not possible! I have been doing it for more than 4 years!
And a prop plug is not required to update any of my programs.
We check first if there is a pointer in sector 0 of the SD card at a certain position, otherwise we do a search for the first sector of the file that starts with some signature bytes.
The pointer allows faster boot times if needed (and is optional).
The tough guys can then just copy a bootfile to a fresh formated SD card.
The axious guys can use their custom made tools to make a safe place on the SD card and copy the bootfile there.
Andy
I like Ariba's idea because it's very simple for me and doesn't need a whole lot of explanation or special tools...
Just reformat SD card and copy over boot file...
In that spirit, I will be that guy, and go out on a limb and possibly fall flat on my face.
Honestly, this thread has gone from progressing nicely, to nerd fight at dawn with handbags at 10 paces.
I have to ask, if Chip et al really want to make the Prop2 FUN, as it was in the old days apparently, how much memory would it really take to have the P2 find the bootcode.bin file on the first FAT partition? Using unused space on Sector 0 or other 'hidden' spaces just seems like a hack, and throws me back 20 years.
I mean seriously, whether its the routers I work on or the RPi, I can take their SD card and throw it into any Win/Mac/Linux computer and just SEE the darn boot file!
Unless I am totally off my rocker, and this very explicitly limited FAT capability will just blow out the ROM, I think going ahead with some one-off, NIH-type boot process via co-opting the MBR is going to end up in the Prop2 seeing Hack-A-Day and embedded forums jump on the bandwagon and dismiss the P2 yet again.
Prioritize SD booting to that which is expected in this current millenium, and get rid of other stuff if you have to.
Who in their right mind doesn't get the fact that no one wants to have to include an EEPROM in the BOM If they don't have to?
The 'hype' so far about the P2 has been 16 32b Cores, massive bandwidth, etc, etc.
For it to be useful outside of simple LED blinkers, you are going to need mass storage for Real Programs and Real Data. And knowing that, most are going to be adding SD by default, no?
As we used to say, Duhhh!
Having to use special tools to access or manipulate a boot file is decidedly not-FUN.
Aside from the fact that Parallax now has to put up another Challenge and beg someone to build multiple sets of tools across the 3 main OS.
Some people here are totally seeing the bark and not the forests.
I mean, making it easy should help Parallax sell more, right. And thats what we all want, not just something that we personally like or prefer as expert users... ?
/rant
Yeah, let's do that, but don't forget the little header, or it won't work.
Seriously, I'm good either way, as I said above and in my initial post. There is a difference between a discussion and "no", and I never said no to anything this time around, save for the file blocks. That just doesn't seem to make sense to me personally.
Both the header and a partition table entry do make a lot of sense. Both, by the way, do require the user do something besides format 'n copy too. Both can be refreshed after a successful bootable format and header addition by merely copying a file too.
Maybe not, just pick one. That's good too. Fine by me.
Locate on FAT?
Anyone want to write that up and make sure it works always? The two methods above will work always. I think that's important.
I think Chip is looking for a procedure he can put in the ROM. Want "first file on FAT?"
The right thing to do is write up the procedure. Just as Peter and I did with the methods we thought made sense, and both of them do. Only a handful of instructions are needed.
As for Hack a Day, I've a prediction:
When they do get their hands on a P2, they will say a lot of stuff! Most of it will be good. They will mention that it boots from SD, and they will mention the BOM savings over the P1, or just as a savings, and they will compare it to flash and other things on chip... like they tend to do.
Later, no matter what we do, there will be an article on that SD, and it will have a nice set of instructions, just like the other articles involving media do. And those instructions will work too. One of us will probably submit it. I'm game and may do just that. Because the P2 is going to be pretty great.
People will read that, follow them, and boot from their SD.
Here is a tutorial on FAT from a "use it on a micro controller" point of view:
http://codeandlife.com/2012/04/02/simple-fat-and-sd-tutorial-part-1/
Anyone want to boil that down to simple "read this block, operate on this value, read this block", etc...? And then check it?
IMHO, that is what it would take. Right now, Chip can take any of the low level ideas here from Andy, Peter, or myself and get it done in a handful of instructions. All of those ideas translate into a few basic things a user does to make the SD work.
Here is the work done last time, tested by a few of us:
http://forums.parallax.com/discussion/comment/1122184/#Comment_1122184
All are reasonable, IMHO. Not hard. I don't care which gets done. It's worth discussing the best, or potential issues, what the user has to do, etc...
That's all I'm doing.
I think you don't need to reformate the SD card everytime to change the boot file. Just overwrite it with the new one, as long as it has the same name and the same size (or is smaller) the same sectors get overwritten. It may depend on the OS but I have done tests for this on Windows (I think it was XT then).
Andy
I agree that making this RPi compatible ( which means similar in process and not in conflict with ) is in important aspiration. I think the ROM will allow a 'good' SD Boot.
Chip does not have to do all boot choices in one pass so it seems simplest to do the small, simple ones first, like
1) Serial (what Baud choices & MHz specs ? Skip choices ?)
2) i2c
3) SPI
and tbf
{4) SD}
Final 4) is chosen once the ROM resource is defines, and some actual field tests of candidates are done.
Agreed, but users do need to be able to include an EEPROM if they want to.
I added i2c to the above sequence tree for BOM reasons - i2c is the smallest and cheapest configurable memory, and it allows more choice on higher level Boot & future proofing.
eg Suppose any issues are found later with the SD ROM code, that needs patching ? - best way to do that, is a smallest i2c memory.
This is precisely why several of us are focused on basic, block level processes. There won't be trouble with those at the "find the boot file" level. We could experience trouble with SD cards, modes, types, and that's why Peter is asking for a simple process right now.
I agree with that request, however it gets done.
We need to test this idea out and do so on live SD cards, switching modes, powering down, warm booting, etc...
IMHO, that priority trumps higher level options, should someone go through and propose detail on how those might be done.
Right now, we have no way to boot from any persistent media. That is hindering testing, and certainly a roadblock for testing any secondary boot scenarios since we don't even have a primary boot path.
If Peter's itch can be scratched with a handful of instructions, then he is one step (and nearly the final step) to having a self hosting P2 running. That's pretty cool for a chip that is still in an FPGA. I also think it would be a big help toward testing many other things.
Andy has a solution that was tested out on P2 hot for a bit. If it is a small amount of ROM space for the first stage support, then let's get it out there to test.
If the primary options are really just handfuls of instructions for I2C and SPI (and therefore SD basic block boots) then it makes sense if Chip could spend some time getting this functionality into an FPGA image. That opens up a bunch of doors.
It's not going into an unchangeable mask. It can be tweaked and adjusted based on real experience, not just debate. We should at least have the same level of support for persistent booting as a P1 does at this point.
Let's not worry about support for hyperion-super-octo-quad-mega-TB devices...nobody has any of those and nobody is stalled in their testing because the P2 can't support it.
I'd say more but we have the children to think about!