I don't understand why people look only for speed and proces time of loader.
Then don't always need be like fast as Light ---> but in many industrial applications need fill its function.
That said ---> I'm hop so I can see Propeller II in many that applications NOT only be adapted for small gouup of People.
To that it need be versatile --- That Professionals can use it
By just loading $1F8 longs, then an 8-long HMAC signature, you are dealing with a known quantity that will take an exact amount of time to process, and will provide enough code to start anything else (except SD card FAT).
I obviously don't know the market. How many competitive micro-controllers boot straight from an SD card?
Question about the boot order: if it's a consumer product boring from SD and the SD failed, the user is just looking at a dead widget. If boot order is SD, RX/TX, flash, then the developer could at least provide a minimal piece of code in a flash to indicate failure in some manner (depending on interfaces available).
Can any of this be chosen by fuse settings?
Not experienced or smart enough to contribute any more.
Ok, looks like I remembered wrong... You can read from the sst device in SPI mode...
Still, you're going to face a big problem... If you do initiate SQI mode and then do a system "reset" (without cycling power), the chip will remain in SQI mode.
What do you do then?
All SPI chips reset when the CS pin is de-asserted.
SPI chips use commands to do anything. So, if you do not send the READ-QUAD-SPI command but the READ-SLOW command the device will operate in 1-bit mode. The device will only send data on 4 pins if you send the READ-QUAD-SPI command.
"the chip will remain in SQI mode. What do you do then? "
Make sure you use a SPI version with RST and tie to it to Props Reset.
Reset
If the RST#/HOLD# pin is used as a reset pin, RST# pin provides a hardware method for resetting the
device. Driving the RST# pin high puts the device in normal operating mode. The RST# pin must be
driven low for a minimum of TRST time to reset the device.
Somebody already suggested linear scanning the SD card in RAW mode (Cluso?).
After thinking about it, it seems to me that the simplest method could somewhat encompass all the others. And without bothering with FAT at all.
Given code to read SPI bytes already in place, it should take only a few SD commands:
1. Perform initialization (send wakeup clocks, set up SPI mode, wait for card response).
2. Linearly read in one RAW sector at a time, until we find a specific signature, let's say one long equal to 0x43484950
3. From now on, we read in up to the whole HUB image.
.......
Case 3: we found the signature in the file area (but we're not concerned with "files" at all now, only when creating the card!)
* 3.1: a 32KB per sector FAT32 formatting guarantees at least 32KB contiguous, so if we use a boot file <32KB tha's enough for a 2nd stage bootloader, even if the rest of what we read in is garbage from the files following our one.
* 3.2: with the only requirement that we wrote a boot image on a freshly formatted FAT card, the whole 128KB image is valid, and it's all good.
I recant my previous suggestion, because this sounds like a pretty solid idea. I would like to add some things to this:
Eliminate the need for a secondary boot loader by embedding multiple specific signatures in the boot file should the file system place it non-contiguously when broken down into clusters.
Also place signatures at the beginning and ending of each cluster, to avoid copying some random data that coincidentally contains the same value as the signature.
All SPI chips reset when the CS pin is de-asserted.
SPI chips use commands to do anything. So, if you do not send the READ-QUAD-SPI command but the READ-SLOW command the device will operate in 1-bit mode. The device will only send data on 4 pins if you send the READ-QUAD-SPI command.
I'm not sure this is true. The SST datasheet I have says that after the chip is switched to quad I/O mode, only a power cycle or a "reset quad I/O" instruction will return it to SPI mode.
I'm not sure this is true. The SST datasheet I have says that after the chip is switched to quad I/O mode, only a power cycle or a "reset quad I/O" instruction will return it to SPI mode.
This is fact. It through me through a few loops...
Obviously the startup time will be a bit slower and less predictable than a flash ROM, so users needing fast start will be required to use the SPI flash anyway.
It seems like it could be significantly slower in the case of a large SD card and a file near the end of the card. How long will it take to scan 8gb of clusters?
I find this whole SD-card-boot discussion extremely off-putting. First of all, which high-roller volume purchasers would even use such a capability? My guess is very few. Moreover it won't be a selling point that will attract volume purchases. I see nothing at all wrong with booting from the lowest-common-denominator programmable device with the longest anticipated lifespan and lowest pin-count requirements, whether that be EEPROM or flash, and letting the users be responsible, from there, for more advanced booting. It would be a mistake, IMO, to let a vocal hobbyist minority dictate something as indelible as advanced ROM code. Parallax needs to recoup their investment from volume purchases. This discussion is not helping them meet that objective.
I find this whole SD-card-boot discussion extremely off-putting. First of all, which high-roller volume purchasers would even use such a capability? My guess is very few. Moreover it won't be a selling point that will attract volume purchases. I see nothing at all wrong with booting from the lowest-common-denominator programmable device with the longest anticipated lifespan, whether that be EEPROM or flash, and letting the users be responsible, from there, for more advanced booting. It would be a mistake, IMO, to let a vocal hobbyist minority dictate something as indelible as advanced ROM code. Parallax needs to recoup their investment from volume purchases. This discussion is not helping them meet that objective.
I'm not sure I agree... uSD cards are gigantic and dirt cheap... One of these and a Prop chip could be the only things you need to make a cool system...
I still think that for booting, you should forget FAT and just use block mode...
I'm not sure this is true. The SST datasheet I have says that after the chip is switched to quad I/O mode, only a power cycle or a "reset quad I/O" instruction will return it to SPI mode.
This is fact. It through me through a few loops...
How about making a separate partition to boot from? I know that Windows won't make a second partition without a special program on an external drive, but it looks like we'll need a special program no matter how we do it. This method will never accidentally load the wrong thing, which is likely with a 16GB SD card and a one long signature, and is probably the simplest that doesn't require more ram in the prop 2. All the boot loader would have for my method to do is read in the partition table and load the first 126KB of the first partition it finds with a certain partition type ID. This would also be convenient for a project that needed a custom file system for speed reasons, which could be appended to the end of the boot area instead of needing a third partition, which might choke the boot loader if it loaded the ram image in some other way. There is a list of used partition ids at http://www.win.tue.nl/~aeb/partitions/partition_types-1.html, so the boot loader could just use an otherwise unused one.
Does anybody have any quick pointers to what I need to learn?
How much memory could we expect to consistently be available to us in the MBR?
Chip: I am still digesting the overnight discussions here. So I will answer piecemeal. I have a number of links, just need to dig them out.
Currently, here is my suggestion...
Read the MBR sector (#0)
Use offset $1E0(16) = 12 byte specific identifier + 1 long sector pointer
Validate that offset $1E0 = a specific identifier such as "P2BOOTXX" (something specifically identifying the propeller or whatever Parallax is calling it)
Blindly use the next long at offset $1EC = a sector number to where the boot data will be found on the SD card.
Read the sector(s) (pointed to by the MBR) to load the boot code (as defined for the Flash so that encryption/security is maintained) - i.e. we don't want the SD to compromise the P2 security mechanism. There should be up to 32KB of code at this sector base (i.e. a cluster minimum of 32KB).
This may or may not be the data of the first file in the FAT directory (as the P2 ROM doesn't care)
The Flash ROM boot code has validation routines to perform a checksum and validate any encryption mechanism so further validation should be left to those routines already in the P2 ROM.
Use the boot code to locate a FAT file and boot from it.
This mechanism has the least risk.
Sure, it will require one small and simple program to access the SD card directly from Windows/*nix/Mac to locate the boot data file(s) and to then write the MBR sector. This should be fairly simple program. I can do this in spin quite simply, so a P1 with an SD card can perform this easily. it would take e a few minutes (really!) on a P1 since I have the code that could do this now (locates a file from the FAT and then reads/writes to this directly via the low level SD drivers) so I would only have to format the MBR record and write it. This squarely removes any problems/risk from the P2 ROM code.
Sectors are 512 bytes, although there is an extended version using 4096 bytes.
Sector 0 is the MBR (master boot record)
Sector 1 is the File System Information Sector
Sector 2 is an additional boot sector often containing routines called by the boot routines in the MBR
Sectors 6-8 is usually a copy of sectors 0, 1 & 2
Sector 12 is used as an extended Microsoft extended boot loader
The MBR sector contains...
+0(3): 3 byte JMP to skip over the subsequent data to the Boot Code
+3(8): OEM Name
+11(?) BIOS parameter block
+?(?) Boot code <
we will use the penultimate 16 bytes i.e. +$1E0 (length 16 bytes) for locating our boot code.
$1FE(2) $55 $AA boot signature
So, in essence, we can use the MBR at $1E0 (16 bytes) to store a special identifier (12 bytes) plus a pointer to the sector# (1 long) to the first data sector where the P2's Boot Code resides. This can then execute whatever subsequent code is required to support FAT16/32 or any other format that later comes along.
You have some good points, but I don't totally see it your way. If we can figure out a way to do intrinsic booting from an SD card as a third boot pathway (after serial RAM & SPI EEPROM), then that is a good thing because it offers some additional flexibility and the ability to modify an application without having to reflash an EEPROM or make a serial connection. Field upgrades could be a snap. That said, having an EEPROM with more robust code (as a default capability) for SD card is a minor additional cost for a hobbyist board and there are lots of pins anyway. I won't loose sleep if we still need to keep the EEPROM.
The question for me is if it can be done or not in the remaining ROM space? It is worth asking now because the decision Chip makes is one we will live with for years. Is there anything else that is compelling that we can do with the ROM space?
The question for me is if it can be done or not in the remaining ROM space? It is worth asking now because the decision Chip makes is one we will live with for years. Is there anything else that is compelling that we can do with the ROM space?
Another question is how long will it take to write the code and verify it against a huge number of SD cards to make sure it works and is delaying P2 for that long worth it?
All quadSPI chips support 1-bit and 4-bit transfer (maybe 2-bit). These chips startup natively in 1-bit mode and require a write to the status register to change to 4-bit mode.
I spent a day researching every SPI flash chip on the market to confirm this.
The new QSPI SRAM from Microchip also follow this rule
Operation Binary Hex Action
READ 0000 0011 0x03 Read data from memory array beginning at selected address
WRITE 0000 0010 0x02 Write data to memory array beginning at selected address
EDIO 0011 1011 0x3B Enter Dual I/O access
EQIO 0011 1000 0x38 Enter Quad I/O access
RSTIO 1111 1111 0xFF Reset Dual and Quad I/O access
RDMR 0000 0101 0x05 Read Mode Register
WRMR 0000 0001 0x01 Write Mode Register
EQIO is sticky, one in there, you need a RSTIO to exit.
I don't understand why people look only for speed and proces time of loader.
Then don't always need be like fast as Light ---> but in many industrial applications need fill its function.
That said ---> I'm hop so I can see Propeller II in many that applications NOT only be adapted for small gouup of People.
To that it need be versatile --- That Professionals can use it
It's not just speed of loading, but a quick determination of failure, in case we need to get onto the next booting medium.
You're probably right about the negligible effect SD boot support would have on volume customers, but isn't it somewhat curious why some of the biggest ARM vendors have it as an option? So what do you propose be put in the unused ROM space? If nothing else, then why not have the capability even if it's not used as a selling point, but something hidden in the bowels of the datasheet?
@David Betz
You bring up a very good point regarding the time a scan would take for a large capacity SD card. Though some probably won't like it, one solution is to make multiple partitions with the first one being perhaps just a couple MBs in size, and put the boot file there.
With all due respect, this entire process reeks. You need to get off the forum, find a quiet corner somewhere, write a simple, reliable bootloader that works from EEPROM or flash, test it, commit it to ROM, and get the Prop II OTFD. I'm really sorry to be so direct -- I mean that -- but somebody has to be the bad guy, and I can take the heat.
I find this whole SD-card-boot discussion extremely off-putting. First of all, which high-roller volume purchasers would even use such a capability? My guess is very few. Moreover it won't be a selling point that will attract volume purchases. I see nothing at all wrong with booting from the lowest-common-denominator programmable device with the longest anticipated lifespan and lowest pin-count requirements, whether that be EEPROM or flash, and letting the users be responsible, from there, for more advanced booting. It would be a mistake, IMO, to let a vocal hobbyist minority dictate something as indelible as advanced ROM code. Parallax needs to recoup their investment from volume purchases. This discussion is not helping them meet that objective.
-Phil
Totally disagree Phil. Since the prop has no internal flash, requiring an external flash chip to boot an SD card, if the commercial product has one, is an overhead that will be seen as stupid. I am looking at this solely from a commercial basis, not hobbyist. I have 100,000's of micro based products in the field that I designed and programmed. I do know there are enough detractors for the prop without internal flash, but to impose the requirement of a Flash chip just to boot from SD is insane.
Again, we are not talking about the support for FAT (or anything else) in ROM, just the ability to locate a set of sectors containing P2 boot code.
Still, you're going to face a big problem... If you do initiate SQI mode and then do a system "reset" (without cycling power), the chip will remain in SQI mode.
What do you do then?
To cover this, you need a reset preamble :
Winbond says :
Continuous read mode reset 0FFH 0FFH (2 bytes, this also clears the Sticky opcode bits)
So it looks a bit like handling i2c, where you send 16 clocks, with All Data Hi, and (hopefully) get into a known state.
There may be fish-hooks in this, as I see the Winbond RST mode overlays valid address in one mode, so maybe it needs
a CS de-assert after exactly 16 clocks ?
Microchip only shows two clocks for QSPI reset, (4 for DSPI), but does not say what happens if more are used ?
With all due respect, this entire process reeks. You need to get off the forum, find a quiet corner somewhere, write a simple, reliable bootloader that works from EEPROM or flash, test it, commit it to ROM, and get the Prop II OTFD. I'm really sorry to be so direct -- I mean that -- but somebody has to be the bad guy, and I can take the heat.
Cheers!
-Phil
No offense taken, Phil. It's just that the synthesis work is probably going to take another six weeks, we heard, so the ROM is not the critical path now. I have the booter already written that works with SPI flash and I could call it done tomorrow. Since we have time, I'm interested to know what else we can load onto the ship before it sails. You are not the only guy who's getting indigestion from this thread. I've received PM's from several here who have expressed similar disdain. Engineering can be ugly, I guess. This might be like making sausage.
If it requires hacking the SD card to get it to work with the Propeller, I don't think it should be done. It just sounds like a support nightmare for both Parallax and us here on the forums.
Kye,
Sorry to not bother quoting your comments. They are way out of perspective.
Heater and I (in ZiCog) were using the raw SD driver in fsrw (rokicki & Mike's IIRC) way before FAT was used. What I am promoting is exactly this. The raw routines to access the SPI Flash and SPI SD card should be identical. So we only need to command the card to read raw sectors. We only need to read the MBR sector to validate a specific identifier, and then blindly use a pointer to a sector base (start of cluster) where up to 32KB of boot code may be found. This code can be used identically to the Flash supplied code.
Then if new SD formats evolve, it will be easy for us users to make a new set of soft boot code or to fix whatever is wrong with booting from some SD cards.
I don't think there can be anything simpler that what I am proposing, and there is no risk, because if it fails, we are back to requiring Flash. But if it works, it makes a lot of sense to professionals likely to use the P2.
This code should be quite simple and not take too may longs to implement.
I shall dig out my old code later today to see what commands are required to be given to the SD card to just read it in raw mode. This is all we are doing from ROM.
Comments
I understand ---> BUT
I don't understand why people look only for speed and proces time of loader.
Then don't always need be like fast as Light ---> but in many industrial applications need fill its function.
That said ---> I'm hop so I can see Propeller II in many that applications NOT only be adapted for small gouup of People.
To that it need be versatile --- That Professionals can use it
Question about the boot order: if it's a consumer product boring from SD and the SD failed, the user is just looking at a dead widget. If boot order is SD, RX/TX, flash, then the developer could at least provide a minimal piece of code in a flash to indicate failure in some manner (depending on interfaces available).
Can any of this be chosen by fuse settings?
Not experienced or smart enough to contribute any more.
Still, you're going to face a big problem... If you do initiate SQI mode and then do a system "reset" (without cycling power), the chip will remain in SQI mode.
What do you do then?
SPI chips use commands to do anything. So, if you do not send the READ-QUAD-SPI command but the READ-SLOW command the device will operate in 1-bit mode. The device will only send data on 4 pins if you send the READ-QUAD-SPI command.
Make sure you use a SPI version with RST and tie to it to Props Reset.
Reset
If the RST#/HOLD# pin is used as a reset pin, RST# pin provides a hardware method for resetting the
device. Driving the RST# pin high puts the device in normal operating mode. The RST# pin must be
driven low for a minimum of TRST time to reset the device.
I recant my previous suggestion, because this sounds like a pretty solid idea. I would like to add some things to this:
Eliminate the need for a secondary boot loader by embedding multiple specific signatures in the boot file should the file system place it non-contiguously when broken down into clusters.
Also place signatures at the beginning and ending of each cluster, to avoid copying some random data that coincidentally contains the same value as the signature.
This is fact. It through me through a few loops...
-Phil
I still think that for booting, you should forget FAT and just use block mode...
Doesn't SD card behave the same way?
Chip: I am still digesting the overnight discussions here. So I will answer piecemeal. I have a number of links, just need to dig them out.
Currently, here is my suggestion...
- Read the MBR sector (#0)
- Use offset $1E0(16) = 12 byte specific identifier + 1 long sector pointer
- Validate that offset $1E0 = a specific identifier such as "P2BOOTXX" (something specifically identifying the propeller or whatever Parallax is calling it)
- Blindly use the next long at offset $1EC = a sector number to where the boot data will be found on the SD card.
- Read the sector(s) (pointed to by the MBR) to load the boot code (as defined for the Flash so that encryption/security is maintained) - i.e. we don't want the SD to compromise the P2 security mechanism. There should be up to 32KB of code at this sector base (i.e. a cluster minimum of 32KB).
- This may or may not be the data of the first file in the FAT directory (as the P2 ROM doesn't care)
- The Flash ROM boot code has validation routines to perform a checksum and validate any encryption mechanism so further validation should be left to those routines already in the P2 ROM.
- Use the boot code to locate a FAT file and boot from it.
This mechanism has the least risk.Sure, it will require one small and simple program to access the SD card directly from Windows/*nix/Mac to locate the boot data file(s) and to then write the MBR sector. This should be fairly simple program. I can do this in spin quite simply, so a P1 with an SD card can perform this easily. it would take e a few minutes (really!) on a P1 since I have the code that could do this now (locates a file from the FAT and then reads/writes to this directly via the low level SD drivers) so I would only have to format the MBR record and write it. This squarely removes any problems/risk from the P2 ROM code.
Chip, for your quick understanding of the FAT16/32 and SD structure...
http://en.wikipedia.org/wiki/File_Allocation_Table
- Sectors are 512 bytes, although there is an extended version using 4096 bytes.
- Sector 0 is the MBR (master boot record)
- Sector 1 is the File System Information Sector
- Sector 2 is an additional boot sector often containing routines called by the boot routines in the MBR
- Sectors 6-8 is usually a copy of sectors 0, 1 & 2
- Sector 12 is used as an extended Microsoft extended boot loader
The MBR sector contains...- +0(3): 3 byte JMP to skip over the subsequent data to the Boot Code
- +3(8): OEM Name
- +11(?) BIOS parameter block
- +?(?) Boot code <
- $1FE(2) $55 $AA boot signature
So, in essence, we can use the MBR at $1E0 (16 bytes) to store a special identifier (12 bytes) plus a pointer to the sector# (1 long) to the first data sector where the P2's Boot Code resides. This can then execute whatever subsequent code is required to support FAT16/32 or any other format that later comes along.we will use the penultimate 16 bytes i.e. +$1E0 (length 16 bytes) for locating our boot code.
You have some good points, but I don't totally see it your way. If we can figure out a way to do intrinsic booting from an SD card as a third boot pathway (after serial RAM & SPI EEPROM), then that is a good thing because it offers some additional flexibility and the ability to modify an application without having to reflash an EEPROM or make a serial connection. Field upgrades could be a snap. That said, having an EEPROM with more robust code (as a default capability) for SD card is a minor additional cost for a hobbyist board and there are lots of pins anyway. I won't loose sleep if we still need to keep the EEPROM.
The question for me is if it can be done or not in the remaining ROM space? It is worth asking now because the decision Chip makes is one we will live with for years. Is there anything else that is compelling that we can do with the ROM space?
The new QSPI SRAM from Microchip also follow this rule
EQIO is sticky, one in there, you need a RSTIO to exit.
It's not just speed of loading, but a quick determination of failure, in case we need to get onto the next booting medium.
You're probably right about the negligible effect SD boot support would have on volume customers, but isn't it somewhat curious why some of the biggest ARM vendors have it as an option? So what do you propose be put in the unused ROM space? If nothing else, then why not have the capability even if it's not used as a selling point, but something hidden in the bowels of the datasheet?
@David Betz
You bring up a very good point regarding the time a scan would take for a large capacity SD card. Though some probably won't like it, one solution is to make multiple partitions with the first one being perhaps just a couple MBs in size, and put the boot file there.
With all due respect, this entire process reeks. You need to get off the forum, find a quiet corner somewhere, write a simple, reliable bootloader that works from EEPROM or flash, test it, commit it to ROM, and get the Prop II OTFD. I'm really sorry to be so direct -- I mean that -- but somebody has to be the bad guy, and I can take the heat.
Cheers!
-Phil
Again, we are not talking about the support for FAT (or anything else) in ROM, just the ability to locate a set of sectors containing P2 boot code.
To cover this, you need a reset preamble :
Winbond says :
Continuous read mode reset 0FFH 0FFH (2 bytes, this also clears the Sticky opcode bits)
Microchip says :
RSTIO 1111 1111 0xFF Reset Dual and Quad I/O access
So it looks a bit like handling i2c, where you send 16 clocks, with All Data Hi, and (hopefully) get into a known state.
There may be fish-hooks in this, as I see the Winbond RST mode overlays valid address in one mode, so maybe it needs
a CS de-assert after exactly 16 clocks ?
Microchip only shows two clocks for QSPI reset, (4 for DSPI), but does not say what happens if more are used ?
No offense taken, Phil. It's just that the synthesis work is probably going to take another six weeks, we heard, so the ROM is not the critical path now. I have the booter already written that works with SPI flash and I could call it done tomorrow. Since we have time, I'm interested to know what else we can load onto the ship before it sails. You are not the only guy who's getting indigestion from this thread. I've received PM's from several here who have expressed similar disdain. Engineering can be ugly, I guess. This might be like making sausage.
If it's not in spec, don't do it.
Sorry to not bother quoting your comments. They are way out of perspective.
Heater and I (in ZiCog) were using the raw SD driver in fsrw (rokicki & Mike's IIRC) way before FAT was used. What I am promoting is exactly this. The raw routines to access the SPI Flash and SPI SD card should be identical. So we only need to command the card to read raw sectors. We only need to read the MBR sector to validate a specific identifier, and then blindly use a pointer to a sector base (start of cluster) where up to 32KB of boot code may be found. This code can be used identically to the Flash supplied code.
Then if new SD formats evolve, it will be easy for us users to make a new set of soft boot code or to fix whatever is wrong with booting from some SD cards.
I don't think there can be anything simpler that what I am proposing, and there is no risk, because if it fails, we are back to requiring Flash. But if it works, it makes a lot of sense to professionals likely to use the P2.
This code should be quite simple and not take too may longs to implement.
I shall dig out my old code later today to see what commands are required to be given to the SD card to just read it in raw mode. This is all we are doing from ROM.