@Chip - if you want some sample SD boot timings and command/response flow charts I should be able to cobble something together as that would cut through all the SD access fluff. It is quite easy really though.
Do you have some code-sizes for this, ideally in P2 ASM ?
'Will it fit' is going to be an important question re ROM.
Yes, it will fit easily enough, as long as we don't concern it with any kind of partition or file system, just tell it the raw sector to start loading from. I will try to get back later today with some timings and flow which is derived from using Tachyon on the P2 although I will clean up and simplify the init process.
The point being missed by some is that the format of the SD card is totally irrelevant. You can have a single FAT Partition or Multiple Partitions in any format whatsoever. The only part required is the MBR sector 0 has a small block that includes pointers to the first sector used and its length, plus of course the "identifier". Nothing could be simpler and everyone's ideas are covered.
Chip,
I suggest you get a ROM released (without SD). Once that is done, Peter and I can get you a P2 working SD boot code with comments so you can understand it simply. We are both in Oz and can phone each other when/if necessary. And we can skype or whatever at then.
Ray.
Chip,
I suggest you get a ROM released (without SD). Once that is done, Peter and I can get you a P2 working SD boot code with comments so you can understand it simply. We are both in Oz and can phone each other when/if necessary. And we can skype or whatever at then.
Ray.
...
Do these cards use SPI command $03 to begin reading? I don't know how to access sectors, exactly, as opposed to addresses.
...
I would love for the chip to be able to boot straight off an SD card. That would be fantastic, in itself. That it could provide a big file system is another good thing. Perhaps we could have a 3- or 4-way Skype chat about this ...
The SPI protocol for SD cards has not much to do with the way you access an SPI-Flash. So no $03 commands and such.
The commands are a bit more complicated and expect responses with timeouts. And SD cards need an initalization process which is different for SD and SDHC cards.
The low level stuff, like initialization and block-read is necessary for every of the above boot methodes.
The discussion about the right way to handle the higher level, the filesystem part, is very tedious, everyone has a different view, and advocate it in a religion like way. Not sure why this is so.
The main question is: Should it be simple to handle for the User or simple to handle for the Booter code. This is somewhat mutually exclusive, especially on Windows.
I agree on having a pointer to a set of blocks. And one gets the whole sector on a read anyway.
Seems a shame to not put those where the partitions go. If people want to just treat the SD as a block device, do it!
Others, who would employ partitions would benefit as well.
In a way, this is a lot like the interrupt discussion. If we build it to read the partition table, which is just a few longs, all the possible OS, mixed use options remain possible.
Oh well, like last time. It goes how it goes. I'll just make a booter that loads from something else, then handles the SD card partition aware, worst case.
Carry on guys... I'm out. (Not on P2, just this discussion)
Aim to make it easy for the booter code, along the line of what is being recently described. Don't underestimate getting that up and running and documented.
After that is done, it'll be easier to see what might potentially be required to make it easier for the user.
Regarding that information - please don't even bother with SD cards, they are obsolete, just stick with SDHC. They require a slightly different initialization plus the memory is addressed in blocks where block size default is 512.
We have a 16KB ROM in the Prop2 which takes 17ms to read into the hub after reset is done. Right now, the current booter only takes 2KB. I will get this current incarnation done and make some FPGA images, for now.
I'm glad you guys know what to do regarding SD cards. It's true that the ROM booter needs a simple approach to booting from them, free of FAT concerns.
We have a 16KB ROM in the Prop2 which takes 17ms to read into the hub after reset is done. Right now, the current booter only takes 2KB. I will get this current incarnation done and make some FPGA images, for now.
I'm glad you guys know what to do regarding SD cards. It's true that the ROM booter needs a simple approach to booting from them, free of FAT concerns.
We have a 16KB ROM in the Prop2 which takes 17ms to read into the hub after reset is done. Right now, the current booter only takes 2KB. I will get this current incarnation done and make some FPGA images, for now.
I'm glad you guys know what to do regarding SD cards. It's true that the ROM booter needs a simple approach to booting from them, free of FAT concerns.
What are your plans for the remaining space?
@David Betz Why a mini Tachyon Kernel of course, don't be silly
The point being missed by some is that the format of the SD card is totally irrelevant. You can have a single FAT Partition or Multiple Partitions in any format whatsoever. The only part required is the MBR sector 0 has a small block that includes pointers to the first sector used and its length, plus of course the "identifier". Nothing could be simpler and everyone's ideas are covered.
Everyone has understood that FAT have noting to do with SD boot. If you do not need to read it in other systems you can use raw block access.
But as you point also single partition this means that the fw will be copied as a file eg in the root of that partition. You will have always your pointer in block 0 and P2 will perform a raw block access to read it. But the fw file will be visible from other systems. This will not help to update it since you can't just overwrite it because it will go in other sectors during the rewrite. This will only expose the fw file to be accidentally deleted or damaged from other systems.
What I am saying, like you, is to use the pointer to the fw in block 0 and then do a raw block read from that point. But the pointer have to be written in a place where it conforms to the MBR architecture. IMHO this place is the partition entry and more precisely its LBA starting address.
More over if you use the right partition types every system will refuse to manipulate the MBR or at least it will ask you several times if you are sure you wan't to do that.
This do not differ from your point of view and/or the boot process is needed to do that. This only fixes the place where the pointer have to be and that, regardless of how many partitions you do (if any at all), the fw image will never appear as a file in any regular (user) partition.
Moreover if you fix max fw size eg 2MB (to cover possible future hub memory increases) using partition entry your pointer can become:
LBAaddress+(Status x 2MB)
allowing for the use of more than one image for field fw upgrades. No OS will change this details in your "partition entry" without asking you permission several times.
SPI_CS pull-up?
No - Wait for serial.
Yes - SPI_CK pull-up?
No - Wait 100ms for serial, attempt flash boot, if fail then wait for serial.
Yes - Attempt flash boot, if fail then wait for serial.
This way, a jumper on the SPI_CK pull-up can be used to reload a system that normally (jumper on) boots from flash immediately.
At this point, I've decided against a time-out on waiting for serial. Any objections?
We have a 16KB ROM in the Prop2 which takes 17ms to read into the hub after reset is done. Right now, the current booter only takes 2KB. I will get this current incarnation done and make some FPGA images, for now.
I'm glad you guys know what to do regarding SD cards. It's true that the ROM booter needs a simple approach to booting from them, free of FAT concerns.
What are your plans for the remaining space?
@David Betz Why a mini Tachyon Kernel of course, don't be silly
I actually wrote that as part of my original post but thought maybe I'd better not try to put words in Chip's mouth. :-)
SPI_CS pull-up?
No - Wait for serial.
Yes - SPI_CK pull-up?
No - Wait 100ms for serial, attempt flash boot, if fail then wait for serial.
Yes - Attempt flash boot, if fail then wait for serial.
This way, a jumper on the SPI_CK pull-up can be used to reload a system that normally (jumper on) boots from flash immediately.
At this point, I've decided against a time-out on waiting for serial. Any objections?
If an SPI flash chip is present, there needs to be a pull-up on SPI_CS to inhibit unintended toggling. It wouldn't be wise to not have that pull-up, don't you think?
SPI_CS pull-up?
No - Wait for serial.
Yes - SPI_CK pull-up?
No - Wait 100ms for serial, attempt flash boot, if fail then wait for serial.
Yes - Attempt flash boot, if fail then wait for serial.
This way, a jumper on the SPI_CK pull-up can be used to reload a system that normally (jumper on) boots from flash immediately.
At this point, I've decided against a time-out on waiting for serial. Any objections?
If an SPI flash chip is present, there needs to be a pull-up on SPI_CS to inhibit unintended toggling. It wouldn't be wise to not have that pull-up, don't you think?
Maybe it's better to keep SD out of the ROM boot process. Perhaps it's better to keep it simple and fast.
But then, you may want to boot from SD card.
Here's a P2V V10 code that boots from SD or SDHC. Code fits in one cog, even with some serial messages. A second cog does the serial I/O.
The code is more or less a porting of the read only parts of FSRW.
Maybe it's better to keep SD out of the ROM boot process. Perhaps it's better to keep it simple and fast.
But then, you may want to boot from SD card.
Chip says above "We have a 16KB ROM in the Prop2 which takes 17ms to read into the hub after reset is done. Right now, the current booter only takes 2KB."
So there is ROM room, but one COG is full already - maybe it can overflow into HubEXEC for SD ?
There is also a reset-exit-time-penalty for larger code, so adding stuff is not quite a free lunch.
I was involved in the SD boot discussions the first time it came up. But in my opinion dMajo's scheme is the best. No tools, no file concerns, just fdisk with a specific partition type, and from my Linux box I can simply 'dd' or 'cat' a file directly to e.g. /dev/sdb2, with no worries about where it ends if I rewrite it, and the card will be 100% standard for anything else that accesses it.
I was involved in the SD boot discussions the first time it came up. But in my opinion dMajo's scheme is the best. No tools, no file concerns, just fdisk with a specific partition type, and from my Linux box I can simply 'dd' or 'cat' a file directly to e.g. /dev/sdb2, with no worries about where it ends if I rewrite it, and the card will be 100% standard for anything else that accesses it.
That sounds fine as long as there is some Windows equivalent to fdisk and dd.
I was involved in the SD boot discussions the first time it came up. But in my opinion dMajo's scheme is the best. No tools, no file concerns, just fdisk with a specific partition type, and from my Linux box I can simply 'dd' or 'cat' a file directly to e.g. /dev/sdb2, with no worries about where it ends if I rewrite it, and the card will be 100% standard for anything else that accesses it.
That sounds fine as long as there is some Windows equivalent to fdisk and dd.
There is many tools also for windows. And anyway you already have the most universal tool: prop itself.
Well, it seems there are a lot of different opinions about how to handle SD boot.
Maybe that's another reason to leave that to a second stage booter...
Having different opinions is called life and in spite of these opinions we still get on and do "something", not nothing. The simple raw booter that is totally file system unaware is both fast and lean. The overhead to get that image onto a card be it by a Prop itself or a PC is easily handled outside of the boot process where it should be rather than trying to slow down and complicate the P2 booter. I understand the rationale presented by having partitions but nothing beats the simplicity of raw sectors and besides who says we need FAT32 anyway?
I envisage using microSD cards simply as local storage with its own secure and Flash friendly file system without ever being removed or placed into a PC. The initial boot image burner and image would be loaded via serial.
I've been meaning to test some P2 SD boot code but I've also been waiting on a new P2 image with a boot ROM for serial Flash.
If we locate the pointer to the data in the partition table, both methods work. Like with interrupts, it's there should one need or want it, but nobody is required to do it.
There is no speed penalty at all. True either way.
Those who want a partition will make one, those who will point to a file can do that too.
In both cases first block gets read, and in both cases a pointer gets fetched, and from there other blocks get read.
Yeah, I said I'm out. And I am as far as do this or not, and which way makes sense. No matter the outcome of this, I will just boot with something else, and make a SD booter that gets its pointer from the partition table so SD card use is standard and interoperable with other SD card uses.
My SD stuff will be partitions. Got plans
At the block level, the partition is the best practice way to identify block data on the media and insure higher order access, which is filesystem access, does not cause a problem.
They are portable media, after all.
The differences, should we want to do the minimum and just examine the 4 partition table entries, is a few longs tops. We should do this, but if that actually is an exception, just picking one will do.
BTW, doing that will insure boot methods will remain relevant and coexist with other card uses for a very long time.
Both ideas are only block methods too. This all literally boils down to where we put the pointer.
Comments
'Will it fit' is going to be an important question re ROM.
I suggest you get a ROM released (without SD). Once that is done, Peter and I can get you a P2 working SD boot code with comments so you can understand it simply. We are both in Oz and can phone each other when/if necessary. And we can skype or whatever at then.
Ray.
+1
The last reported ROM loader size was a few pages back, but it did not seem to have much spare space then, for extra features.
The SPI protocol for SD cards has not much to do with the way you access an SPI-Flash. So no $03 commands and such.
The commands are a bit more complicated and expect responses with timeouts. And SD cards need an initalization process which is different for SD and SDHC cards.
The low level stuff, like initialization and block-read is necessary for every of the above boot methodes.
The discussion about the right way to handle the higher level, the filesystem part, is very tedious, everyone has a different view, and advocate it in a religion like way. Not sure why this is so.
The main question is: Should it be simple to handle for the User or simple to handle for the Booter code. This is somewhat mutually exclusive, especially on Windows.
Andy
Seems a shame to not put those where the partitions go. If people want to just treat the SD as a block device, do it!
Others, who would employ partitions would benefit as well.
In a way, this is a lot like the interrupt discussion. If we build it to read the partition table, which is just a few longs, all the possible OS, mixed use options remain possible.
Oh well, like last time. It goes how it goes. I'll just make a booter that loads from something else, then handles the SD card partition aware, worst case.
Carry on guys... I'm out. (Not on P2, just this discussion)
Aim to make it easy for the booter code, along the line of what is being recently described. Don't underestimate getting that up and running and documented.
After that is done, it'll be easier to see what might potentially be required to make it easier for the user.
http://forums.parallax.com/discussion/141952/sd-card-test-please-test-may-be-used-for-boot-code-for-the-prop2/p1
http://forums.parallax.com/discussion/162732/p2-bootload-from-flash-and-optional-sd/p1
Here is a flow chart for SDC/MMC Initialisation Flow (SPI Mode)
http://elm-chan.org/docs/mmc/pic/sdinit.png
And a good writeup including the commands
http://elm-chan.org/docs/mmc/mmc_e.html
And here is a writeup. I have no read it so not sure how good it will be.
http://codeandlife.com/2012/04/02/simple-fat-and-sd-tutorial-part-1/
And here is my Prop1 code to test any SD cards
I'm glad you guys know what to do regarding SD cards. It's true that the ROM booter needs a simple approach to booting from them, free of FAT concerns.
@David Betz Why a mini Tachyon Kernel of course, don't be silly
But as you point also single partition this means that the fw will be copied as a file eg in the root of that partition. You will have always your pointer in block 0 and P2 will perform a raw block access to read it. But the fw file will be visible from other systems. This will not help to update it since you can't just overwrite it because it will go in other sectors during the rewrite. This will only expose the fw file to be accidentally deleted or damaged from other systems.
What I am saying, like you, is to use the pointer to the fw in block 0 and then do a raw block read from that point. But the pointer have to be written in a place where it conforms to the MBR architecture. IMHO this place is the partition entry and more precisely its LBA starting address.
More over if you use the right partition types every system will refuse to manipulate the MBR or at least it will ask you several times if you are sure you wan't to do that.
This do not differ from your point of view and/or the boot process is needed to do that. This only fixes the place where the pointer have to be and that, regardless of how many partitions you do (if any at all), the fw image will never appear as a file in any regular (user) partition.
Moreover if you fix max fw size eg 2MB (to cover possible future hub memory increases) using partition entry your pointer can become:
LBAaddress+(Status x 2MB)
allowing for the use of more than one image for field fw upgrades. No OS will change this details in your "partition entry" without asking you permission several times.
I agree with Peter that SDHC is enough.
I caught up on the thread, so I have one suggestion:
CS-PULLUP, NO-CK-PULLUP: Wait 100ms for serial, attempt flash boot, if fail attempt SD boot, if fail then wait for serial
Windows prefers FAT partitions in the first partition, so I suggest SD based boot image be stored in partition 2/3/4, but not 1.
If an SPI flash chip is present, there needs to be a pull-up on SPI_CS to inhibit unintended toggling. It wouldn't be wise to not have that pull-up, don't you think?
I really like your mapping, this just changes one state slightly.
But then, you may want to boot from SD card.
Here's a P2V V10 code that boots from SD or SDHC. Code fits in one cog, even with some serial messages. A second cog does the serial I/O.
The code is more or less a porting of the read only parts of FSRW.
Chip says above
"We have a 16KB ROM in the Prop2 which takes 17ms to read into the hub after reset is done. Right now, the current booter only takes 2KB."
So there is ROM room, but one COG is full already - maybe it can overflow into HubEXEC for SD ?
There is also a reset-exit-time-penalty for larger code, so adding stuff is not quite a free lunch.
A dd command is sadly not there. But as dMajo said why not simply use the propeller itself?
Put the firmware in the root file system and run a program on the propeller to locate the file and write its size and starting sector into the MBR.
Works for Windows, Linux, Mac ...
Enjoy!
Mike
Maybe that's another reason to leave that to a second stage booter...
I envisage using microSD cards simply as local storage with its own secure and Flash friendly file system without ever being removed or placed into a PC. The initial boot image burner and image would be loaded via serial.
I've been meaning to test some P2 SD boot code but I've also been waiting on a new P2 image with a boot ROM for serial Flash.
There is no speed penalty at all. True either way.
Those who want a partition will make one, those who will point to a file can do that too.
In both cases first block gets read, and in both cases a pointer gets fetched, and from there other blocks get read.
Yeah, I said I'm out. And I am as far as do this or not, and which way makes sense. No matter the outcome of this, I will just boot with something else, and make a SD booter that gets its pointer from the partition table so SD card use is standard and interoperable with other SD card uses.
My SD stuff will be partitions. Got plans
At the block level, the partition is the best practice way to identify block data on the media and insure higher order access, which is filesystem access, does not cause a problem.
They are portable media, after all.
The differences, should we want to do the minimum and just examine the 4 partition table entries, is a few longs tops. We should do this, but if that actually is an exception, just picking one will do.
BTW, doing that will insure boot methods will remain relevant and coexist with other card uses for a very long time.
Both ideas are only block methods too. This all literally boils down to where we put the pointer.