My main input here is really that we could wait on the SD card booting code given the other priorities in front of Parallax. Once Chip engages in certain engineering efforts they can go on for a long time. I'd rather have that time be used for providing some input on our toolchain plan, data sheets, and the rest of the list somewhere else in this thread.
I would like to formally withdraw my request to support booting with a QuadSPI (Winbond) or SQI (Microchip SST) device. I think there is too much risk. The most common denominator seems to be single bit SPI or other similar solutions for power on reset.
You have the risk backwards. You trade off a short term perceived risk, against a long term real risk, that QuadSPI becomes the only game in town.
Have you checked the prices/stocks even now, on moderately sized SPI Flash ?
( a Winbond 8M W25Q80BLZPIGTR is already cheaper than a 512k AT25F512B )
If you have deliberately designed (by omission) a ROM that fails to properly handle these QuadSPI parts, even in single bit mode, you have created a chip that faces early obsolescence.
Chip has it100% right when he says : "We need to be sure that we can reset it properly, so that we can be assured that it is in one-bit mode when we boot from it. We need to be sure that the ROM code can reset chips that are wired in all the useful ways."
It needs to do this with the cheapest/most available parts in 2012, and in 2020.
Ken,
If there are issues with SD card brands (and I think this is just a software problem that has never been properly resolved, and is not at issue with what I am proposing) then the professional user will solve this. They will buy SD cards from reputable sources. Commercially it makes no sense to do otherwise.
As I said, I will code the P1 for this tonight. And even if in the end, there is a bug in ROM and it does not work, nothing will be lost. The Flash will still boot. My concept specifically avoids the format of the SD card, be it FAT or something else.
It would be such a pitty to miss the opportunity to be able to boot from SD without requiring Flash for such a simple task. You can be assured you will receive lots of complaints and criticism from the professional circle if an external chip is required just to boot from SD card. Remember, the P2 has no internal flashlike other chips, so this can be a real bugbear. I don't care about the hobbyist for they won't care about having Flash as well. But they are not going to be the salvation of the P2 for Parallax.
You have the risk backwards. You trade off a short term perceived risk, against a long term real risk, that QuadSPI becomes the only game in town.
Have you checked the prices/stocks even now, on moderately sized SPI Flash ?
( a Winbond 8M W25Q80BLZPIGTR is already cheaper than a 512k AT25F512B )
If you have deliberately designed (by omission) a ROM that fails to properly handle these QuadSPI parts, even in single bit mode, you have created a chip that faces early obsolescence.
Chip has it100% right when he says : "We need to be sure that we can reset it properly, so that we can be assured that it is in one-bit mode when we boot from it. We need to be sure that the ROM code can reset chips that are wired in all the useful ways."
It needs to do this with the cheapest/most available parts in 2012, and in 2020.
QSPI isn't needed to load 512 longs of bootloader. Once the bootloader is loaded, you use whatever mode/media you want to run the user program from.
Just been thinking about the comport part, what's the chances of reusing the TxD pin for MOSI and RxD pin for MISO? Shouldn't affect the SPI flash chip at all. The download/debug software should be able to handle any SPI bootup noise.
Keeps the preallocated pin count down to four pins, same as the Prop1.
EDIT: Or make it RxD reused as SCLK so that if tristating doesn't work on the SPI flash it's only it's inputs that are tied to the comport pins.
Just been thinking about the comport part, what's the chances of reusing the TxD pin for MOSI and RxD pin for MISO? Shouldn't affect the SPI flash chip at all. The download/debug software should be able to handle any SPI bootup noise.
That is OK, but what about the user who wants to get FONTS from the QuadSPI and run the COM port, in their Run-time code ?
It displays the MBR record in 16 byte blocks per line.
It locates a file (called "_VER.CMD") in this example (because that is how I boot my Prop OS). From the file the code determines and displays the first data sector of the file and the files size. This will not be done in the boot ROM code as it will be read from the MBR.
I then force the values of the first data sector and size into fbase and fsize. In the ROM the first data sector will come from the MBR and the size will most likely be 1024 (whatever is used in the ROM for booting from FLASH).
The sector# and size is then used to read from the SD card and boot the prop.
Just a few notes...
This is not the final condensed code.
I used a RamBlade3 for testing - it uses 104MHz and the SD is on pins 20-23 and serial on 30-31 at 115200 baud. Use PropTool to recompile with changes.
1) Sounds like it is going take RAM space for the required ROM code space. RAM is sacred and I don't want to lose a single byte.
2) Sounds like it is going to take some dedicated GPIO pins above and beyond those for serial and FLASH loading. This is bad because:
2.a) I/O Pins are sacred I don't want to lose a single pin.
2.b) It dictates for all time which pins the SD cards will be on, this goes against the Prop philosophy of any pin for any job (all pins are equal) Who would use different pins in this scenario.
3) In my experience SD is damn unreliable. If you can get on that works in the first place. Use one for logging and it will fail and then you won't even boot to find out what happened.
4) Contrary to some statements here, I can't see booting from SD as something professional builders of embedded systems with a Prop are going to want. It seems to be more use for hobbyists who hopefully will be the minority of purchasers for the Prop II,
5) The cost of a FLASH boot device hardly makes it a show stopper in size or cost for a hobbyist who wants their main boot from SD,
6) This thing about ARM booting from SD is irrelevant. Those things are System On Chip, loaded with peripheral hardware modules, the ones I have seen that boot from SD have a separate processor to read the SD and boot the ARM. It makes sense there as they are booting a huge OS like Andorid or other Linux. No so for the Propeller.
For those who reply to 2.a) above "No pins are lost because you don't have to use it". This is not quite so. When I have my FLASH and SD free Prop II permanently attached to an ARM that programs and boots it via normal serial I do not want the Prop flailing pins around trying to find an SD to boot from when the ARM perhaps was not yet ready to provide reset and boot to the Prop. Those pins may be connected to something important.
How has the Prop II serial programming protocol changed?
I ask because many times I come across discussion of programming a Prop over Bluetooth, Xbee, WIFI or whatever and the answers always seem to be that it wont't work because of the timing difficulties.
4) Contrary to some statements here, I can't see booting from SD as something professional builders of embedded systems with a Prop are going to want. It seems to be more use for hobbyists who hopefully will be the minority of purchasers for the Prop II,
Yeah, SD booting is just a current fad that "professionals" targeting the hobby sector are all excited about for the moment. The market is targetting ready made boards than don't require any soldering. A tiny boot chip along side the processor is no more than a talking point.
I don't have a problem with SD booting as such. I do, however, have a problem with it subtracting from the 128KB of hub RAM. I didn't realise that would be the case originally.
Can some of the commercial developers answer my question?
With proposed boot order: tx/RX, flash, SD, wait for reset - you release a product with only SD card and the SD boot fails. You have a product in the users hand that is dead until they call you for support. Repeated resets see same failure.
What to do to make this product fail more gracefully? Nothing I can see.
If boot order was SD, RX/tx, flash, you could put a small flash with code to fail more gracefully and help with debug/ user recovery. Pop the SD out at any time and reset would boot from other sources as in the original proposed order.
Here is code that will locate a file and update the MBR record with a string "PropellerII"+$00 followed by the sector# where booting will start. This is at offset $1E0 (+480)
Just to answer those critics who don't see the professional requirements...
I don't care if a hobby board requires Flash or not. What I do care about is the professional user. I have many professional designs under my belt (not hoby use) so I am stating this from my professional background.
Heater: The moment you provide a SPI Flash chip you do not require any pins to be wasted, so to speak, for SD access.
Would this mean that we need to drive all 4 QuadSPI data pins high and give clock pulses? It looks like we might as well commit to QuadSPI and wire up all four bits, if so.
That's the way I look at it... I guess that's why I forgot you can do SPI read: It doesn't make much sense to support SPI read when your driver has to do SQI to ensure that you are in SPI mode...
Anyway, my SQI driver initializes this:
First, check to see if already in SQI mode by SQI JEDEC read and if so, you're done.
Next, read JEDEC in SPI mode.
Next, set SQI mode.
Finally, read SQI JEDEC to make sure in SQI mode...
What I'm not sure about is what would happen if you tried this process on a SPI only chip... Hopefully, trying a SQI read on it wouldn't mess it up...
Now that I think about it some more... My current vote would be to drop SD card boot support and add SQI boot support.
SQI gives faster read speed than SPI and SD (regular mode SD).
Heater: The moment you provide a SPI Flash chip you do not require any pins to be wasted, so to speak, for SD access.
Yes, but what about a design with no FLASH either, dependent on a permanent serial connection to larger "host" processor that loads the Prop RAM at start up.? If for some reason the host is not ready to load the Prop straight after power up the Prop starts waving it's SD pins. Not good.
I don't doubt your designs, but I do question if the mass market of embedded "things" that the Prop II aspires too demands an SD boot.
In my opinion making the first partition hidden, reserved for boot (call it parallax), like it is on all the todays PCs (at least the Windows ones) where in it is stored the factory pre-installed/deployed OS (the system-recovery partition) is the easest method.
This way you know where the MBR resides (always in the first sector 0x0000), you know where in the MBR is the partition table (0x01BC) and you know where in the first partition is located its starting point bytes from 0x01 to 0x03 so given the SD is correct you can start a raw read directly at sector absolute pointed by bytes 0x01BD to 0x01BF. You do not need FAT support, after the SD has been properly initialised you have to start the raw SPI read of contigous data (only 2K if just one cog image is enough or the whole hub image: perhaps it can be determined by one of the excess bits of the encryption key)
If you want a degree more of security use "Partition type" details (0x0000+0x1BC+0x0004 thus absolute offset 0x01C0) to verify the SD is properly formatted with the right partition structure (at least the first, the one of our interest); search for 0x27, 0x78 or 07F data (or whatever you want as long as they were being properly written during the formatting). In this way you test the partition and at the same time you prevent windows accessing it so corrupting it: with normal OS tools the user can only delete the partition.
The second advantage is that you keep the firmware separated from the application data stored in the FAT32 partition and so avoiding end-user (and/or developer) accidental corruptions.
@mindrobots is right with his concerns about the boot order.
a pc usually boots from hd, but in case of NEED can boot from a removable (and replacable/exchangable) other boot-media (floppy, cd, usb, even sd sometimes see your bios)
so if you want to use sd for firmwareupdates it has to boot before flash? Or do I see something wrong there? Else it will NEVER boot having flash soldered in?
As of now you can replace a eeprom on p1 (if you have it in a socket). You can mail that down to your technican/serviceguy like you can with a sd-card. He/she then can replace it and restart the device. Can you do this with Flash?
Lets say you have devices on telephone poles/trees or other not easy to reach places. If there would be internet or power they would use a pc. But it is not so they use a embedded device. How to update? Climbing up the pole and replacing sd/eeprom? possible.
Taking your laptop up there to reprogram flash? well - i do not think so.
If sd is to complicated PLEASE at least give us EEPROM. But we need SOME replaceable boot-media besides new flashing with a pc.
I think Clusos solution is pretty solid. Using the mbr for what it is supposed to be used for... as a boot-record.
one cluster is 32k - good for the p1 to load all of it - good for the p2 to load/encrypt a more serious bootloader.
ALL pc are doing this since dos 1.0. reading the boot-sector and getting the info needed to proceed. arm/x86/pc/linux does not matter. they all boot from floppy/hd/sd/usb-stick/dvd you name it. And they do all the same.
reading the bootsector and proceed from there. RAW access.
And we do not even need a Program running on all of our Osses to create a Prop-boot-sd - we need a prop with a sd-card. And the prop can create the boot-disk/make it Prop-bootable.
If you do not have a prop with a sd-card, you will have no need to create a bootable sd. If you have one - well then you can.
But you also need a PC/XT to create a sytem floppy for dos 3.2 or whatever. you can not do this in Win7 or linux. So yes you need a prop to make your sd prop-bootable. So what?
It is all in the standards, since 20?+ years. Nothing what Cluso is doing is against any rules of file-systems or 'using undocumented stuff'.
Its just plain and simple correct to do it this way. And easy to implement.
Read bootsector - look for signature - get sectoradress of first sector in cluster and read next 32k.
where is the problem here besides that sd should boot before flash?
1) Sounds like it is going take RAM space for the required ROM code space. RAM is sacred and I don't want to lose a single byte.
2) Sounds like it is going to take some dedicated GPIO pins above and beyond those for serial and FLASH loading. This is bad because:
2.a) I/O Pins are sacred I don't want to lose a single pin.
2.b) It dictates for all time which pins the SD cards will be on, this goes against the Prop philosophy of any pin for any job (all pins are equal) Who would use different pins in this scenario.
3) In my experience SD is damn unreliable. If you can get on that works in the first place. Use one for logging and it will fail and then you won't even boot to find out what happened.
4) Contrary to some statements here, I can't see booting from SD as something professional builders of embedded systems with a Prop are going to want. It seems to be more use for hobbyists who hopefully will be the minority of purchasers for the Prop II,
5) The cost of a FLASH boot device hardly makes it a show stopper in size or cost for a hobbyist who wants their main boot from SD,
6) This thing about ARM booting from SD is irrelevant. Those things are System On Chip, loaded with peripheral hardware modules, the ones I have seen that boot from SD have a separate processor to read the SD and boot the ARM. It makes sense there as they are booting a huge OS like Andorid or other Linux. No so for the Propeller.
For those who reply to 2.a) above "No pins are lost because you don't have to use it". This is not quite so. When I have my FLASH and SD free Prop II permanently attached to an ARM that programs and boots it via normal serial I do not want the Prop flailing pins around trying to find an SD to boot from when the ARM perhaps was not yet ready to provide reset and boot to the Prop. Those pins may be connected to something important.
2a You loose only one pin. You need one cs for the flash and another for the SD, the clock and data can share the SPI pins. The users that will have both the devices can also assign them to different pins because only one will be the boot device.
Probably the MISO MOSI pins can be shared also with tx and rx because when the CS is not asserted they are HiZ on the devices.
Can some of the commercial developers answer my question?
With proposed boot order: tx/RX, flash, SD, wait for reset - you release a product with only SD card and the SD boot fails. You have a product in the users hand that is dead until they call you for support. Repeated resets see same failure.
What to do to make this product fail more gracefully? Nothing I can see.
If boot order was SD, RX/tx, flash, you could put a small flash with code to fail more gracefully and help with debug/ user recovery. Pop the SD out at any time and reset would boot from other sources as in the original proposed order.
Regarding the boot order I prefer the serial first because in case of partially corrupted firmware you will not boot anymore. Otherwise you can download to ram only your diag software and test the hardware around the prop. As a second device I will choose the fastest device for the time critical apps.
For the ones that is against the SD boot consider for example the environmental monitoring stations, weather stations, data loggers, electronic advertisement panels and other applications in the field, perhaps running on alternative power (wind, solar) that can be located in strange places where it is not always possible to reach them with the NB to flash a new firmware. Now with one operation (exchange of SD) you can have your data and also the firmware changed/updated if the case.
@dMajo, my new order is essentially serial first if the SD is removed or is not an option in your hardware design.
1) boot from SD if it is part of hardware design, inserted and a viable boot image and functional
2) boot from TX/RX (allows intercept during development/debugging of Flash boot)
3) Flash boot (or EEPROM?) allows graceful, partial recovery from failed SD, allows debug monitor, allows more complex boot from a "non-standard" SD image
If your application has a need to boot/run from SD but can't conform to whatever standard is implemented, you put in Flash and code it to bootstrap to YOUR SD image standard. YOUR SD would fail the standard boot, and you would end up running YOUR flash bootstrap.
TX/RX boot is always an option just by connecting it and popping out any SD cards.
I earlier wrote a posting about the way the Linux LiLo loader works - multi-stage booting with only minimal code in the MBR. However, that was for info - I'm _not_ taking sides in SD vs. not SD, I'm a consumer at this point and not a board designer. But _if_ there's additional boot options then the boot sequence is important:
With proposed boot order: tx/RX, flash, SD, wait for reset - you release a product with only SD card and the SD boot fails. You have a product in the users hand that is dead until they call you for support. Repeated resets see same failure.
[..]If boot order was SD, RX/tx, flash, you could put a small flash with code to fail more gracefully and help with debug/ user recovery. Pop the SD out at any time and reset would boot from other sources as in the original proposed order.
This is very important. It does not make sense to, in particular, let flash boot before SD. That would mean that SD boot would only work for boards without flash. The normal way to do this is to use SD as primary boot and flash as fallback boot. Then you'll have your standard boot setup in flash and your new/testing/alternate boot setup on SD. Think of it as a PC with harddisk boot and CD (or SD) boot: The equivalent scenario would be that if there's HD then you won't ever be able to boot from a removable medium.. not good.
So, if there's SD then SD should be first, then if there's serial put it second (or if those two are switched it's not a big deal), then flash, and if that fails, reset.
And again, I'm not arguing for SD as such. I'm not in a position where I _should_ argue. I'm sure I, as a consumer, would be getting SD support indirectly anyway by loading something into flash which would then let me choose SD.
I have only skimmed a couple of posts. Thanks dMajo. Seems to be better to place our 16 bytes of data at ofset $180 in the MBR sector#0.
We are better to have the SD initial boot code residing as a data file that does not move. That way, we only have to modify the MBR once. WIndows/*nix/Mac can then update our second stage file with ease (no special program).
Here is code that boots my SD (using the older offset of $1E0 for the 16 bytes). It is working. I used the previously posted code to write the boot data location in the MBR.
The files fsrwFemto & sdspiFemto have been combined. There are still some major shrinking of code to be done to remove write calls and their support routines, and some FAT support. Also, the pasm driver requires the removal of eeprom support although much of this support may exist in the ROM to support the SPI Flash.
It's late so I will have a go tomorrow, time permitting.
Chip, thanks for posting the ROM code so far. I will be able to see what SPI routines we can use.
Just to answer those critics who don't see the professional requirements...
I don't care if a hobby board requires Flash or not. What I do care about is the professional user. I have many professional designs under my belt (not hoby use) so I am stating this from my professional background.
Hehe, I'm pretty sure you were pointing out the Raspberry Pi .. ah, was that only once, nah, more like ten times ... :P
Lol, no, keep up the good work. I'm all just talk. If you can put together a solution that works for most SD cards and fit it in the remaining ROM space I'm all for it.
Re your P2 pinouts...
rx_pin = 91
tx_pin = 90
spi_cs = 89
spi_ck = 88
spi_di = 87
spi_do = 86 Postedit.. changed recommendation after thinking it through further - possibility of adding second SPI Quad Flash chip
From the W25Q80 the Quad access uses these pins... (I am not sure about other Quad SPi chips)
rx_pin = 91
tx_pin = 90
spi_cs = 89
spi_ck = 88
spi_io3 = 87 = /Hold (not used in boot process)
spi_io2 = 86 = /WP (not used in boot process)
spi_io1 = 85
spi_io0 = 84
This way, the nibble(s) would be aligned to a byte boundary and be in the correct bit order. I know it seems a waste to define these pins, but if the P2 is to support Quad mode efficiently, I would think this would be a better arrangement.
evanh: No I was not referring to the R-Pi, or any other product for that matter.
I saw some confusion above for what Flash is. It is similar to EEPROM in that it is programmed in a fairly similar fashion. Some Flash chips are available in DIP8 packages. They are now much cheaper than eeproms and larger and easier to buy. The choice of Flash is the right one.
I saw some discussion about booting the SD before Flash. The flash can still contain a simple stub loader to load from SD if required. No problems here. Checking the Flash before the SD is fine by me.
In reference to Heater's (and others) comments about the P2 "wiggling" pins while finding a place to boot from, is there a way to select, specify, DEMAND a boot ONLY from TX/RX that will not wiggle any other pins? I could see this being useful to developers who wanted something completely different as far as pin assignments and is also in keeping with the Propeller underlying philosophy of soft peripherals and no dedicated pins. It may also be useful to boot device developers who may want to get some code running before "wiggling" any pins they are working with.
TX/RX boot is always an option just by connecting it and popping out any SD cards.
Ick! Not my preferred arrangement at all. I can understand the desire to have a bootable SD card override the hard-wired firmware, I'd also like to keep the debugging override as well. Ie: Use development/debugging software to control bootup execution when it's connected. Including control of SD card bootup, which also needs the SD card in place.
Comments
Yes, that's reasonable.
My main input here is really that we could wait on the SD card booting code given the other priorities in front of Parallax. Once Chip engages in certain engineering efforts they can go on for a long time. I'd rather have that time be used for providing some input on our toolchain plan, data sheets, and the rest of the list somewhere else in this thread.
You have the risk backwards. You trade off a short term perceived risk, against a long term real risk, that QuadSPI becomes the only game in town.
Have you checked the prices/stocks even now, on moderately sized SPI Flash ?
( a Winbond 8M W25Q80BLZPIGTR is already cheaper than a 512k AT25F512B )
If you have deliberately designed (by omission) a ROM that fails to properly handle these QuadSPI parts, even in single bit mode, you have created a chip that faces early obsolescence.
Chip has it100% right when he says : "We need to be sure that we can reset it properly, so that we can be assured that it is in one-bit mode when we boot from it. We need to be sure that the ROM code can reset chips that are wired in all the useful ways."
It needs to do this with the cheapest/most available parts in 2012, and in 2020.
If there are issues with SD card brands (and I think this is just a software problem that has never been properly resolved, and is not at issue with what I am proposing) then the professional user will solve this. They will buy SD cards from reputable sources. Commercially it makes no sense to do otherwise.
As I said, I will code the P1 for this tonight. And even if in the end, there is a bug in ROM and it does not work, nothing will be lost. The Flash will still boot. My concept specifically avoids the format of the SD card, be it FAT or something else.
It would be such a pitty to miss the opportunity to be able to boot from SD without requiring Flash for such a simple task. You can be assured you will receive lots of complaints and criticism from the professional circle if an external chip is required just to boot from SD card. Remember, the P2 has no internal flashlike other chips, so this can be a real bugbear. I don't care about the hobbyist for they won't care about having Flash as well. But they are not going to be the salvation of the P2 for Parallax.
QSPI isn't needed to load 512 longs of bootloader. Once the bootloader is loaded, you use whatever mode/media you want to run the user program from.
Just been thinking about the comport part, what's the chances of reusing the TxD pin for MOSI and RxD pin for MISO? Shouldn't affect the SPI flash chip at all. The download/debug software should be able to handle any SPI bootup noise.
Keeps the preallocated pin count down to four pins, same as the Prop1.
EDIT: Or make it RxD reused as SCLK so that if tristating doesn't work on the SPI flash it's only it's inputs that are tied to the comport pins.
Correct, QSPI isn't needed to load 512 longs, nor will Quad mode be used - notice Chip says
"it is in one-bit mode when we boot from it",
but the important bit, is here :
"We need to be sure that we can reset it properly, so that we can be assured that it is in one-bit mode"
With QuadSPI parts, you need to ENSURE it IS in one bit mode. That needs some deliberate action.
That is what the discussion is about, not about booting in Quad Mode.
It is a reset housekeeping issue, to make sure not loader-code "can reset chips that are wired in all the useful ways".
That is OK, but what about the user who wants to get FONTS from the QuadSPI and run the COM port, in their Run-time code ?
- It displays the MBR record in 16 byte blocks per line.
- It locates a file (called "_VER.CMD") in this example (because that is how I boot my Prop OS). From the file the code determines and displays the first data sector of the file and the files size. This will not be done in the boot ROM code as it will be read from the MBR.
- I then force the values of the first data sector and size into fbase and fsize. In the ROM the first data sector will come from the MBR and the size will most likely be 1024 (whatever is used in the ROM for booting from FLASH).
- The sector# and size is then used to read from the SD card and boot the prop.
Just a few notes...- This is not the final condensed code.
- I used a RamBlade3 for testing - it uses 104MHz and the SD is on pins 20-23 and serial on 30-31 at 115200 baud. Use PropTool to recompile with changes.
- I use my PropOS on my SD card. You can find it here http://forums.parallax.com/showthread.php?138251-A-Propeller-OS-that-can-run-on-multiple-hardware.../page2&highlight=prop+os+kyedos (post #45 contains source? and object compiled for 5MHz, Serial on P30-31 at 115200 baud, and SD on P0-3)
I will post a minimised version shortly.PropBootSD_V1_10_RB3_P2 - Archive [Date 2012.08.14 Time 18.50].zip
1) Sounds like it is going take RAM space for the required ROM code space. RAM is sacred and I don't want to lose a single byte.
2) Sounds like it is going to take some dedicated GPIO pins above and beyond those for serial and FLASH loading. This is bad because:
2.a) I/O Pins are sacred I don't want to lose a single pin.
2.b) It dictates for all time which pins the SD cards will be on, this goes against the Prop philosophy of any pin for any job (all pins are equal) Who would use different pins in this scenario.
3) In my experience SD is damn unreliable. If you can get on that works in the first place. Use one for logging and it will fail and then you won't even boot to find out what happened.
4) Contrary to some statements here, I can't see booting from SD as something professional builders of embedded systems with a Prop are going to want. It seems to be more use for hobbyists who hopefully will be the minority of purchasers for the Prop II,
5) The cost of a FLASH boot device hardly makes it a show stopper in size or cost for a hobbyist who wants their main boot from SD,
6) This thing about ARM booting from SD is irrelevant. Those things are System On Chip, loaded with peripheral hardware modules, the ones I have seen that boot from SD have a separate processor to read the SD and boot the ARM. It makes sense there as they are booting a huge OS like Andorid or other Linux. No so for the Propeller.
For those who reply to 2.a) above "No pins are lost because you don't have to use it". This is not quite so. When I have my FLASH and SD free Prop II permanently attached to an ARM that programs and boots it via normal serial I do not want the Prop flailing pins around trying to find an SD to boot from when the ARM perhaps was not yet ready to provide reset and boot to the Prop. Those pins may be connected to something important.
I ask because many times I come across discussion of programming a Prop over Bluetooth, Xbee, WIFI or whatever and the answers always seem to be that it wont't work because of the timing difficulties.
Doh! /me slaps himself.
Yeah, SD booting is just a current fad that "professionals" targeting the hobby sector are all excited about for the moment. The market is targetting ready made boards than don't require any soldering. A tiny boot chip along side the processor is no more than a talking point.
I don't have a problem with SD booting as such. I do, however, have a problem with it subtracting from the 128KB of hub RAM. I didn't realise that would be the case originally.
With proposed boot order: tx/RX, flash, SD, wait for reset - you release a product with only SD card and the SD boot fails. You have a product in the users hand that is dead until they call you for support. Repeated resets see same failure.
What to do to make this product fail more gracefully? Nothing I can see.
If boot order was SD, RX/tx, flash, you could put a small flash with code to fail more gracefully and help with debug/ user recovery. Pop the SD out at any time and reset would boot from other sources as in the original proposed order.
Shortly I will post the minimised code.
PropBootSD_V1_10_RB3_P2 - Archive [Date 2012.08.14 Time 20.29].zip
Just to answer those critics who don't see the professional requirements...
I don't care if a hobby board requires Flash or not. What I do care about is the professional user. I have many professional designs under my belt (not hoby use) so I am stating this from my professional background.
Heater: The moment you provide a SPI Flash chip you do not require any pins to be wasted, so to speak, for SD access.
That's the way I look at it... I guess that's why I forgot you can do SPI read: It doesn't make much sense to support SPI read when your driver has to do SQI to ensure that you are in SPI mode...
Anyway, my SQI driver initializes this:
First, check to see if already in SQI mode by SQI JEDEC read and if so, you're done.
Next, read JEDEC in SPI mode.
Next, set SQI mode.
Finally, read SQI JEDEC to make sure in SQI mode...
What I'm not sure about is what would happen if you tried this process on a SPI only chip... Hopefully, trying a SQI read on it wouldn't mess it up...
Now that I think about it some more... My current vote would be to drop SD card boot support and add SQI boot support.
SQI gives faster read speed than SPI and SD (regular mode SD).
Yes, but what about a design with no FLASH either, dependent on a permanent serial connection to larger "host" processor that loads the Prop RAM at start up.? If for some reason the host is not ready to load the Prop straight after power up the Prop starts waving it's SD pins. Not good.
I don't doubt your designs, but I do question if the mass market of embedded "things" that the Prop II aspires too demands an SD boot.
Here are some infos on the MBR: http://en.wikipedia.org/wiki/Master_Boot_Record (all you need)
In my opinion making the first partition hidden, reserved for boot (call it parallax), like it is on all the todays PCs (at least the Windows ones) where in it is stored the factory pre-installed/deployed OS (the system-recovery partition) is the easest method.
This way you know where the MBR resides (always in the first sector 0x0000), you know where in the MBR is the partition table (0x01BC) and you know where in the first partition is located its starting point bytes from 0x01 to 0x03 so given the SD is correct you can start a raw read directly at sector absolute pointed by bytes 0x01BD to 0x01BF. You do not need FAT support, after the SD has been properly initialised you have to start the raw SPI read of contigous data (only 2K if just one cog image is enough or the whole hub image: perhaps it can be determined by one of the excess bits of the encryption key)
If you want a degree more of security use "Partition type" details (0x0000+0x1BC+0x0004 thus absolute offset 0x01C0) to verify the SD is properly formatted with the right partition structure (at least the first, the one of our interest); search for 0x27, 0x78 or 07F data (or whatever you want as long as they were being properly written during the formatting). In this way you test the partition and at the same time you prevent windows accessing it so corrupting it: with normal OS tools the user can only delete the partition.
The second advantage is that you keep the firmware separated from the application data stored in the FAT32 partition and so avoiding end-user (and/or developer) accidental corruptions.
Booter ($78 longs)
SHA-256 with HMAC ($E3 longs)
'************************ '* SHA-256 + HMAC * '* (byte-level) * '************************ org sha_256 setf #%0_1111_0000 'configure movf for sbyte0 -> {dbyte3,dbyte2,dbyte1,dbyte0,dbyte3,...} call #init_hash 'init hash, clear hmac mode, reset byte count ' ' ' Command Loop ' command rdlong x,ptra 'wait for command (%cc_nnnnnnnnnnnnn_ppppppppppppppppp) tjz x,#command setptrb x 'get pointer (%ppppppppppppppppp) mov count,x 'get count (%nnnnnnnnnnnnn) shl count,#2 shr count,#2+17 add count,#1 '+1 for 1..8192 range shr x,#32-2 'get command (%cc) cachex 'invalidate cache for fresh rdbytec's djz x,#begin_hmac '1 = begin hmac, pointer @key (count+1 bytes, 1..64) djz x,#hash_bytes '2 = hash bytes, pointer @message (count+1 bytes, 1..8192) djz x,#read_hash '3 = read hash, pointer @hashbuffer (32 bytes) done wrlong zero,ptra 'clear command to signal done jmp #command 'get next command ' ' ' Begin HMAC ' begin_hmac call #end_hash 'end any hash in progress :ipad mov x,#$00 'get and hash ipad key (full block) cmp count,bytes wz,wc if_a rdbytec x,ptrb++ 'after key bytes, hash $00's to fill block xor x,#$36 'xor bytes with ipad ($36) call #hash_byte '(last iteration triggers hash_block, z=1) if_nz jmp #:ipad reps #16,#2 'save opad key setinds opad_key,w mov indb,inda++ xor indb++,opad 'xor bytes with opad ($5C) mov hmac,#1 'set hmac mode jmp #done ' ' ' Hash Bytes ' hash_bytes rdbytec x,ptrb++ 'hash bytes call #hash_byte djnz count,#hash_bytes jmp #done ' ' ' Read Hash ' read_hash tjz hmac,#:not 'if not hmac, output hash call #end_hash 'hmac, end current hash reps #16,#1 'get opad key into w[0..15] (full block) setinds w,opad_key mov indb++,inda++ call #hash_block 'hash opad key reps #8,#1 'get hashx[0..7] into w[0..7] setinds w,hashx mov indb++,inda++ movd hash_byte,#w+8 'account for opad key and hashx bytes mov bytes,#64+32 '(1-1/2 blocks, 1/2 block needs end_hash) :not call #end_hash 'end current hash setinda hashx 'store hashx[0..7] at pointer, big-endian mov count,#8 :out reps #4,#2 mov x,inda++ rol x,#8 wrbyte x,ptrb++ djnz count,#:out jmp #done ' ' ' End Hash - hash $80, any $00's needed to get to offset $38, then 8-byte length ' end_hash mov length,bytes 'get message length in bits shl length,#3 mov x,#$80 'hash end-of-message byte ($80) :fill call #hash_byte '(may trigger hash_block) mov x,bytes 'hash any $00's needed to get to offset $38 and x,#$3F cmp x,#$38 wz mov x,#$00 if_nz jmp #:fill :len test bytes,#$04 wc 'hash 8-byte length, big-endian if_c rol length,#8 '(hash four $00's, then four length bytes) if_c mov x,length call #hash_byte '(last iteration triggers hash_block) if_nz jmp #:len reps #8,#1 'save hash[0..7] into hashx[0..7] setinds hashx,hash mov indb++,inda++ init_hash reps #8,#1 'copy hash_init[0..7] into hash[0..7] setinds hash,hash_init mov indb++,inda++ mov hmac,#0 'clear hmac mode mov bytes,#0 'reset byte count init_hash_ret end_hash_ret ret ' ' ' Hash Byte - add byte to w[0..15] and hash block if full (z=1) ' hash_byte movf w,x 'store byte into w[0..15], big-endian add bytes,#1 'increment byte count test bytes,#$03 wz 'every 4th byte, increment w pointer if_z add hash_byte,d0 test bytes,#$3F wz 'every 64th byte, reset w pointer if_z movd hash_byte,#w if_z call #hash_block 'every 64th byte, hash block hash_byte_ret ret ' ' ' Hash Block - first extend w[0..15] into w[16..63] to generate schedule ' hash_block reps #48,@:sch 'i = 16..63 setinds w+16,w+16-15+7 'indb = @w[i], inda = @w[i-15+7] setinda --7 's0 = (w[i-15] -> 7) ^ (w[i-15] -> 18) ^ (w[i-15] >> 3) mov indb,inda-- mov x,indb rol x,#18-7 xor x,indb ror x,#18 shr indb,#3 xor indb,x add indb,inda 'w[i] = s0 + w[i-16] setinda ++14 's1 = (w[i-2] -> 17) ^ (w[i-2] -> 19) ^ (w[i-2] >> 10) mov x,inda mov y,x rol y,#19-17 xor y,x ror y,#19 shr x,#10 xor x,y add indb,x 'w[i] = s0 + w[i-16] + s1 setinda --5 'w[i] = s0 + w[i-16] + s1 + w[i-7] :sch add indb++,inda ' Load variables from hash reps #8,#1 'copy hash[0..7] into a..h setinds a,hash mov indb++,inda++ ' Do 64 hash iterations on variables reps #64,@:itr 'i = 0..63 setinds k+0,w+0 'indb = @k[i], inda = @w[i] mov x,g 'ch = (e & f) ^ (!e & g) xor x,f and x,e xor x,g mov y,e 's1 = (e -> 6) ^ (e -> 11) ^ (e -> 25) rol y,#11-6 xor y,e rol y,#25-11 xor y,e ror y,#25 add x,y 't1 = ch + s1 add x,indb++ 't1 = ch + s1 + k[i] add x,inda++ 't1 = ch + s1 + k[i] + w[i] add x,h 't1 = ch + s1 + k[i] + w[i] + h mov y,c 'maj = (a & b) ^ (b & c) ^ (c & a) and y,b or y,a mov h,c or h,b and y,h mov h,a 's0 = (a -> 2) ^ (a -> 13) ^ (a -> 22) rol h,#13-2 xor h,a rol h,#22-13 xor h,a ror h,#22 add y,h 't2 = maj + s0 mov h,g 'h = g mov g,f 'g = f mov f,e 'f = e mov e,d 'e = d mov d,c 'd = c mov c,b 'c = b mov b,a 'b = a add e,x 'e = e + t1 mov a,x 'a = t1 + t2 :itr add a,y ' Add variables back into hash reps #8,#1 'add a..h into hash[0..7] setinds hash,a add indb++,inda++ hash_block_ret ret ' ' ' Defined data ' zero long 0 d0 long 1 << 9 opad long $36363636 ^ $5C5C5C5C hash_init long $6A09E667, $BB67AE85, $3C6EF372, $A54FF53A, $510E527F, $9B05688C, $1F83D9AB, $5BE0CD19 'fractionals of square roots of primes 2..19 k long $428A2F98, $71374491, $B5C0FBCF, $E9B5DBA5, $3956C25B, $59F111F1, $923F82A4, $AB1C5ED5 'fractionals of cube roots of primes 2..311 long $D807AA98, $12835B01, $243185BE, $550C7DC3, $72BE5D74, $80DEB1FE, $9BDC06A7, $C19BF174 long $E49B69C1, $EFBE4786, $0FC19DC6, $240CA1CC, $2DE92C6F, $4A7484AA, $5CB0A9DC, $76F988DA long $983E5152, $A831C66D, $B00327C8, $BF597FC7, $C6E00BF3, $D5A79147, $06CA6351, $14292967 long $27B70A85, $2E1B2138, $4D2C6DFC, $53380D13, $650A7354, $766A0ABB, $81C2C92E, $92722C85 long $A2BFE8A1, $A81A664B, $C24B8B70, $C76C51A3, $D192E819, $D6990624, $F40E3585, $106AA070 long $19A4C116, $1E376C08, $2748774C, $34B0BCB5, $391C0CB3, $4ED8AA4A, $5B9CCA4F, $682E6FF3 long $748F82EE, $78A5636F, $84C87814, $8CC70208, $90BEFFFA, $A4506CEB, $BEF9A3F7, $C67178F2 ' ' ' Undefined data ' hmac res 1 bytes res 1 count res 1 length res 1 opad_key res 16 hash res 8 hashx res 8 w res 64 a res 1 b res 1 c res 1 d res 1 e res 1 f res 1 g res 1 h res 1 x res 1 y res 1a pc usually boots from hd, but in case of NEED can boot from a removable (and replacable/exchangable) other boot-media (floppy, cd, usb, even sd sometimes see your bios)
so if you want to use sd for firmwareupdates it has to boot before flash? Or do I see something wrong there? Else it will NEVER boot having flash soldered in?
As of now you can replace a eeprom on p1 (if you have it in a socket). You can mail that down to your technican/serviceguy like you can with a sd-card. He/she then can replace it and restart the device. Can you do this with Flash?
Lets say you have devices on telephone poles/trees or other not easy to reach places. If there would be internet or power they would use a pc. But it is not so they use a embedded device. How to update? Climbing up the pole and replacing sd/eeprom? possible.
Taking your laptop up there to reprogram flash? well - i do not think so.
If sd is to complicated PLEASE at least give us EEPROM. But we need SOME replaceable boot-media besides new flashing with a pc.
I think Clusos solution is pretty solid. Using the mbr for what it is supposed to be used for... as a boot-record.
one cluster is 32k - good for the p1 to load all of it - good for the p2 to load/encrypt a more serious bootloader.
ALL pc are doing this since dos 1.0. reading the boot-sector and getting the info needed to proceed. arm/x86/pc/linux does not matter. they all boot from floppy/hd/sd/usb-stick/dvd you name it. And they do all the same.
reading the bootsector and proceed from there. RAW access.
And we do not even need a Program running on all of our Osses to create a Prop-boot-sd - we need a prop with a sd-card. And the prop can create the boot-disk/make it Prop-bootable.
If you do not have a prop with a sd-card, you will have no need to create a bootable sd. If you have one - well then you can.
But you also need a PC/XT to create a sytem floppy for dos 3.2 or whatever. you can not do this in Win7 or linux. So yes you need a prop to make your sd prop-bootable. So what?
It is all in the standards, since 20?+ years. Nothing what Cluso is doing is against any rules of file-systems or 'using undocumented stuff'.
Its just plain and simple correct to do it this way. And easy to implement.
Read bootsector - look for signature - get sectoradress of first sector in cluster and read next 32k.
where is the problem here besides that sd should boot before flash?
confused!
Mike
.
2a You loose only one pin. You need one cs for the flash and another for the SD, the clock and data can share the SPI pins. The users that will have both the devices can also assign them to different pins because only one will be the boot device.
Probably the MISO MOSI pins can be shared also with tx and rx because when the CS is not asserted they are HiZ on the devices.
Regarding the boot order I prefer the serial first because in case of partially corrupted firmware you will not boot anymore. Otherwise you can download to ram only your diag software and test the hardware around the prop. As a second device I will choose the fastest device for the time critical apps.
For the ones that is against the SD boot consider for example the environmental monitoring stations, weather stations, data loggers, electronic advertisement panels and other applications in the field, perhaps running on alternative power (wind, solar) that can be located in strange places where it is not always possible to reach them with the NB to flash a new firmware. Now with one operation (exchange of SD) you can have your data and also the firmware changed/updated if the case.
1) boot from SD if it is part of hardware design, inserted and a viable boot image and functional
2) boot from TX/RX (allows intercept during development/debugging of Flash boot)
3) Flash boot (or EEPROM?) allows graceful, partial recovery from failed SD, allows debug monitor, allows more complex boot from a "non-standard" SD image
If your application has a need to boot/run from SD but can't conform to whatever standard is implemented, you put in Flash and code it to bootstrap to YOUR SD image standard. YOUR SD would fail the standard boot, and you would end up running YOUR flash bootstrap.
TX/RX boot is always an option just by connecting it and popping out any SD cards.
So, if there's SD then SD should be first, then if there's serial put it second (or if those two are switched it's not a big deal), then flash, and if that fails, reset.
And again, I'm not arguing for SD as such. I'm not in a position where I _should_ argue. I'm sure I, as a consumer, would be getting SD support indirectly anyway by loading something into flash which would then let me choose SD.
-Tor
We are better to have the SD initial boot code residing as a data file that does not move. That way, we only have to modify the MBR once. WIndows/*nix/Mac can then update our second stage file with ease (no special program).
Here is code that boots my SD (using the older offset of $1E0 for the 16 bytes). It is working. I used the previously posted code to write the boot data location in the MBR.
P2Boot_115c - Archive [Date 2012.08.14 Time 22.32].zip
The files fsrwFemto & sdspiFemto have been combined. There are still some major shrinking of code to be done to remove write calls and their support routines, and some FAT support. Also, the pasm driver requires the removal of eeprom support although much of this support may exist in the ROM to support the SPI Flash.
It's late so I will have a go tomorrow, time permitting.
Chip, thanks for posting the ROM code so far. I will be able to see what SPI routines we can use.
Hehe, I'm pretty sure you were pointing out the Raspberry Pi .. ah, was that only once, nah, more like ten times ... :P
Lol, no, keep up the good work. I'm all just talk. If you can put together a solution that works for most SD cards and fit it in the remaining ROM space I'm all for it.
Truly, cheers for getting stuck in.
Re your P2 pinouts...
rx_pin = 91
tx_pin = 90
spi_cs = 89
spi_ck = 88
spi_di = 87
spi_do = 86
Postedit.. changed recommendation after thinking it through further - possibility of adding second SPI Quad Flash chip
From the W25Q80 the Quad access uses these pins... (I am not sure about other Quad SPi chips)
rx_pin = 91
tx_pin = 90
spi_cs = 89
spi_ck = 88
spi_io3 = 87 = /Hold (not used in boot process)
spi_io2 = 86 = /WP (not used in boot process)
spi_io1 = 85
spi_io0 = 84
Then a second SPI Quad Flash can be added...
spi_io3 = 83 = /Hold
spi_io2 = 82 = /WP
spi_io1 = 81
spi_io0 = 80
This way, the nibble(s) would be aligned to a byte boundary and be in the correct bit order. I know it seems a waste to define these pins, but if the P2 is to support Quad mode efficiently, I would think this would be a better arrangement.
I saw some confusion above for what Flash is. It is similar to EEPROM in that it is programmed in a fairly similar fashion. Some Flash chips are available in DIP8 packages. They are now much cheaper than eeproms and larger and easier to buy. The choice of Flash is the right one.
I saw some discussion about booting the SD before Flash. The flash can still contain a simple stub loader to load from SD if required. No problems here. Checking the Flash before the SD is fine by me.
Can it be a FUSED option?
Ick! Not my preferred arrangement at all. I can understand the desire to have a bootable SD card override the hard-wired firmware, I'd also like to keep the debugging override as well. Ie: Use development/debugging software to control bootup execution when it's connected. Including control of SD card bootup, which also needs the SD card in place.
1) tx/rx
2) SD
3) flash
??
Just all those other times the R-Pi was brought up. :P