For a refresher, there is a quirk that leaves some SD cards (all?) in an incorrect state outputting on DO ??? It was necessary to give the SD a number of clocks to clear the SD card from this state.
There are also a couple of SPI exit preambles needed too, given earlier.
Deep power down exit, and exit from Quad SPI are a couple, to get into a common-subset state.
Here's a deconstructed and simplified version of FSRW in a single file.
This is P1 spin.
This is with Spin version of SPI, so it's easier to understand and convert to P2.
Works for both regular SD and SDHC.
For P2, will have to slow down SPI to start, but then can speed up after initialization.
File write capability has been removed.
Total size is 900 longs, but 200 of that is fullduplexserial.
It opens a file named "NEWFILEX.TXT" and displays contents over serial link.
Note: It hasn't been tested a whole lot. Also, it can probably be whittled down a bit more...
I'm reviving this thread as I'm doing some work on this now although I am just backing up the P2 image to SD card and writing a simple signature onto sector zero. I am planning to write a very simple pasm routine that inits the SD card and performs a raw read of the card simply by checking the signature on sector zero for a valid boot file, then grabbing from here the starting sector of the boot file itself. No special FAT32 routines are required and in fact the card can be unformatted even. I will then use PNut to load this "SD bootloader" to verify boot operation so at the very least I will have a bootloader that works but it of course could be considered as the basis for the P2 bootrom SD loader. The serial Flash is easy enough although I would allow both simple SPI and QSPI although I would just get it functional first with SPI. Debates about this last time went on for over 7 pages and it was only getting more and more complicated. I'd rather have it running first and if nothing is resolved once again I will simply put this bootloader into a little PIC chip and have it load the P2 serially on reset.
P2 SECTOR ZERO BOOT SIGNATURE
The proposed ROM bootloader would only need to init the raw SD card without regard to FAT32 or any format in fact
and reads the signature on sector 0 and if it is the P2BOOT.IMG file name it will
read the starting sector for the boot image from offset $20 in sector 0
then using that sector number read in from there into memory for the image size specified in offset $24
An image size of less than the file size may be used for faster booting.
Therefore the card can be unformatted and a simple raw write used to backup memory to the card or
a backup may be performed with FAT32 if available.
An unformatted card image could just as easily begin from sector 1.
Notes:
The P2BOOT.IMG is expected to be contiguous and non-fragmented and I have always found this to be the case on SD cards.
So the bootloader can have a very simple SD init routine and a basic raw SD card read which does not require any knowledge of
the card format.
DUMP OF THE FIRST 64 BYTES OF SECTOR ZERO (only colored areas are checked by the bootloader)
Where
$00..$1F = Copy of directory entry for P2BOOT.IMG
$20..$23 = Starting sector of boot file in contiguous blocks
$24..$27 = Number of bytes to load
$28..$2B = Directory entry address encoded as sector<<4 + 4-bits as 1 of 16 directory entries/sector (64GB address range)
$30..$33 = HH:MM:SS time encoded as pure decimal (i.e. 3.45.12 PM is 34512 )
$34..$37 == YY:MM:DD date encoded as pure decimal
Checking this a little:
TF2# $20 $20 SD DUMPL
00.0020: 0000.F280 0010.0000 0004.002A 0000.0000 ........*.......
00.0030: 0002.5B47 0002.7369 0000.0000 0000.0000 G[..is.......... ok
Looking at the time and date stamps that were used
TF2# $30 x@ . 154439 ok
TF2# $34 x@ . 160617 ok
My SD backup routine in Tachyon for reference.
--- backup all of memory (for now) into P2BOOT.IMG for the size of the file
pub SDBACKUP
" P2BOOT.IMG" FOPEN$ 0EXIT --- try to open default boot image but exit if failed
@FILE --- use raw file's starting sector
0 FSIZE@ ADO I OVER SDWR DROP 1+ SPINNER BLKSIZ +LOOP --- write the image with a little spinning progress symbol | / - \
DROP --- discard sector
0 SECTOR ( str ) --- now go to sector zero to setup the signature
@DIRBUF SDBUF BL CMOVE --- make a copy of the directory entry for reference and detection
@FILE SDBUF $20 + ! --- Setup a pointer to the raw file starting sector
FSIZE@ SDBUF $24 + ! --- write the file size we are using (just preset to file size of 1M for now)
diradr *FILE @ SDBUF $28 + ! --- the sector<<4 + dir entry offset>>5 (64GB address on 32-byte dir boundaries)
TIME@ SDBUF $30 + ! --- time and date stamp
DATE@ SDBUF $34 + !
WRSECT --- write the sector to the card
;
Just load and execute the whole of block 0 as a boot block. The code in there can do the rest.
PS: I'm assuming this topic is targeting what Chip should be including in the Prop2 ROM.
That was also the simpler alternative but why limit yourself to sector 0 when sector 0 can point to the start sector of the contiguous sectors that make up the file. That method is just as easy and far more flexible as in the raw or formatted cases that I mentioned. So the simplest case would be that sector 0 contains a signature and maybe even a description but the important thing is that we only need the start sector which may indeed be sector 1 and however many more sectors that our boot image may need. KISS BOOTLOADER
The thing is you are loading block 0 before examining it's content for a whole bunch of details. Why not just leave all the complex optional parts to the code that can reside in block 0. That's what boot blocks are for.
On the other hand, if we were to support multi-boot for a single drive, ie: can automatically provide a dozen boot paths for differently detected hardware ... then there might be a good reason to make this more than a single boot block design.
Yes, flexibility is the key here as I see a single sector being that simple that it would limit things. The boot sig is to stop the P2 from trying to load sector 0 of a non-P2 boot but really the only other thing it needs is the starting sector and the almost non-optional file size. So there is nothing complex in reading sector 0, it's just used as a boot check and a pointer really. What I like is the fact that smaller systems don't really need to have FAT32 if they don't want to either or they can just implement raw SD access. Large images can also just be copied onto SD cards on the PC and field units firmware updated easily although I would recommend standard SD cards for this as we all know those microSD cards are easy to lose
@jmg - yes, I like my SIlabs parts too it's just that I also have 8-pin PICs in MSOP or DIP and they run at 33MHz and include UARTS etc so they are easy to press into service. In fact I would just connect the PIC rxd to the Prop's txd and the PIC txd to the Prop's rxd which is then connected via a current limit resistor to the rxd of the system (USB/PropPlug). That way the PIC can override the rxd when it wants to and load the P2 serially while monitoring or controlling the reset. The idea of having the PIC listen to the P2 txd is so that we have a way of updating the SD/Flash bootloader held in the PIC or an attahced EEPROM. An update sequence can be detected and the new code accepted ready to be loaded into the P2 on the next reset. To allow PNut to load up the P2 on reset we would monitor the rxd line with a simple bit-bash routine and if all's quiet then the PIC would take over.
I think Peter has about the right balance.
Some tolerance of SD card (mis)use needs to be there, and some check for a valid P2 boot image is also going to be useful.
It does not have to be a full, formal FAT scheme, so some field testing will be needed to get this to the
"As simple as possible, but no simpler" stage.
There also needs to be small code included, to exit any special SD modes before boot, as this may no always (re)start cold. (likewise for QuadSPI flash)
I'm reviving this thread as I'm doing some work on this now although I am just backing up the P2 image to SD card and writing a simple signature onto sector zero.
.
.
.
My SD backup routine in Tachyon for reference.
--- backup all of memory (for now) into P2BOOT.IMG for the size of the file
pub SDBACKUP
" P2BOOT.IMG" FOPEN$ 0EXIT --- try to open default boot image but exit if failed
@FILE --- use raw file's starting sector
0 FSIZE@ ADO I OVER SDWR DROP 1+ SPINNER BLKSIZ +LOOP --- write the image with a little spinning progress symbol | / - \
DROP --- discard sector
0 SECTOR ( str ) --- now go to sector zero to setup the signature
@DIRBUF SDBUF BL CMOVE --- make a copy of the directory entry for reference and detection
@FILE SDBUF $20 + ! --- Setup a pointer to the raw file starting sector
FSIZE@ SDBUF $24 + ! --- write the file size we are using (just preset to file size of 1M for now)
diradr *FILE @ SDBUF $28 + ! --- the sector<<4 + dir entry offset>>5 (64GB address on 32-byte dir boundaries)
TIME@ SDBUF $30 + ! --- time and date stamp
DATE@ SDBUF $34 + !
WRSECT --- write the sector to the card
;
... the only other thing it needs is the starting sector and the almost non-optional file size. So there is nothing complex in reading sector 0, it's just used as a boot check and a pointer really.
That can be integral to the code within the boot block. Entirely up to the boot block itself. The Prop2 ROM doesn't need to know.
As JMG says, the hard part is just making sure that a warm reset to SD SPI mode always works.
Large images can also just be copied onto SD cards on the PC and field units firmware updated easily although I would recommend standard SD cards for this as we all know those microSD cards are easy to lose
There's no difference between us here other than I'm proposing the boot block code takes control issuing the successive block reads of the large boot image rather than the ROM directly.
There's no difference between us here other than I'm proposing the boot block code takes control issuing the successive block reads of the large boot image rather than the ROM directly.
Is there any advantage to that as surely if the ROM can read one sector, it can read many. Why reinvent a sector read that the ROM already has?
Ah, now I understand the source of our differences.
No, not reinvented at all. The code loaded from the boot block will call the same ROM derived block read routines that were used for loading the boot block.
Ah, now I understand the source of our differences.
No, not reinvented at all. The code loaded from the boot block will call the same ROM derived block read routines that were used for loading the boot block.
Are you saying that your bootloader in sector 0 would be doing something special that the ROM would not? Isn't it just as easy to say this is the starting sector of the raw file and load it for this many by bytes (or sectors)?
Before this thread goes down another 7 page rabbit hole on how this should "optimally" work how about we let Peter produce something then the group has an actual working "boot thing" to talk about and optimize. plz
I think we can accommodate both modes of boot as it is nothing for the ROM to detect a "boot from sector x" or "just load and execute this code" mode. So I will structure things with this in mind as I have said that flexibility is the key plus these are still early days and we don't want to lock everything into cement that we are going to have to jackhammer later.
When I say the ROM I mean of course the part that I will write and test out along with the group to become a candidate for inclusion in the ROM.
Why don't you just check sector 0 for a boot sig and then run it, but give that boot sector code access to the SD card block read function stored in ROM that was used to load sector 0? That way, it could just load a contiguous stretch of sectors, but it could also be made to do some tests to figure out what it should load, or maybe it could even actually read the filesystem, if it could be crammed into that tiny space.
Why don't you just check sector 0 for a boot sig and then run it, but give that boot sector code access to the SD card block read function stored in ROM that was used to load sector 0? That way, it could just load a contiguous stretch of sectors, but it could also be made to do some tests to figure out what it should load, or maybe it could even actually read the filesystem, if it could be crammed into that tiny space.
Cramming into a tiny space is not a requirement or a necessity so to do so flies in the face of common sense although it might have made sense back in floppy disk days. But why limit the option to do so? Of course you can if you want but the ROM can also be told just to load sectors into memory too. Flexibility, simplicity.
Why don't you just check sector 0 for a boot sig and then run it, but give that boot sector code access to the SD card block read function stored in ROM that was used to load sector 0? That way, it could just load a contiguous stretch of sectors, but it could also be made to do some tests to figure out what it should load, or maybe it could even actually read the filesystem, if it could be crammed into that tiny space.
Cramming into a tiny space is not a requirement or a necessity so to do so flies in the face of common sense although it might have made sense back in floppy disk days. But why limit the option to do so? Of course you can if you want but the ROM can also be told just to load sectors into memory too. Flexibility, simplicity.
My method can load a stretch of blocks, just like yours, with a CALL to the SD block reader function and an ADD or two, all inside of a REP. A boot sector writer program would have to modify just a few more bytes of the boot sector to work my way. My method is slightly more complicated for the user than yours, in that the boot sector writer program has to worry about a short loop that would otherwise be in ROM.
I don't understand how my method is less flexible than yours. It's more flexible, if anything. My method, in addition to being able to do what your method can do, can also do things such as load different images depending on hardware tests, or actually properly follow the FAT cluster chain of a fragmented file. The 446 bytes of bootloader space in the MBR is 111 instructions, slightly more than 1/5 of cog ram. That's probably plenty, especially if the actual SD block reader function that was used to load the boot sector is already in RAM.
I don't get your floppy disk argument. SD cards may be way bigger than floppies, but the boot sector is still only 512 bytes, and there are still only 446 bytes in the boot area.
Guys, you are doing it again, getting hung up on ONE particular method, in this case a 2-stage bootloader where the ROM bootloader loads the SD bootloader which loads the boot image. I'm saying ok ok, but we can have both just as easily. What's the problem???
Thanks, but that was a Q&D while this version is a tad better. If you want to backup 64k just type 64 SDBACKUP and it only takes half a second to backup the image at 80MHz.
--- backup memory into P2BOOT.IMG for the size specified in k and setup sector 0
--- takes around 7 seconds on CV-A9 at 80MHz for 1MB image ~= 3ms/blk or 0.5sec for 64k
pub SDBACKUP ( kbytes -- ) --- size in kilobytes where 128 = 128k
" P2BOOT.IMG" FOPEN$ --- try to open default boot image but exit if failed
IF
1 MAX 1024 MIN 10 << --- limit between 1k and 1M and shift up kbytes to bytes
@FILE 0 3RD --- use raw file's starting sector from start for size
ADO I OVER SDWR DROP 1+ SPINNER BLKSIZ +LOOP DROP --- write the image with a little spinning progress symbol | / - \
@DIRBUF 0 XADR BL CMOVE --- make a copy of the directory entry for reference and detection in sector 0
@FILE $20 X! --- Setup a pointer to the raw file starting sector
DUP $24 X! --- write the file size we are using
diradr *FILE @ $28 X! --- save 64GB encoded pointer to the directory entry
TIME@ $30 X! --- time and date stamp
DATE@ $34 X!
WRSECT --- write the sector to the card
SWAP 0 PRINTDEC PRINT" bytes written"
THEN
DROP --- discard size or wrsect flag
;
Comments
There are also a couple of SPI exit preambles needed too, given earlier.
Deep power down exit, and exit from Quad SPI are a couple, to get into a common-subset state.
This is P1 spin.
This is with Spin version of SPI, so it's easier to understand and convert to P2.
Works for both regular SD and SDHC.
For P2, will have to slow down SPI to start, but then can speed up after initialization.
File write capability has been removed.
Total size is 900 longs, but 200 of that is fullduplexserial.
It opens a file named "NEWFILEX.TXT" and displays contents over serial link.
Note: It hasn't been tested a whole lot. Also, it can probably be whittled down a bit more...
P2 SECTOR ZERO BOOT SIGNATURE
The proposed ROM bootloader would only need to init the raw SD card without regard to FAT32 or any format in fact
and reads the signature on sector 0 and if it is the P2BOOT.IMG file name it will
read the starting sector for the boot image from offset $20 in sector 0
then using that sector number read in from there into memory for the image size specified in offset $24
An image size of less than the file size may be used for faster booting.
Therefore the card can be unformatted and a simple raw write used to backup memory to the card or
a backup may be performed with FAT32 if available.
An unformatted card image could just as easily begin from sector 1.
Notes:
The P2BOOT.IMG is expected to be contiguous and non-fragmented and I have always found this to be the case on SD cards.
So the bootloader can have a very simple SD init routine and a basic raw SD card read which does not require any knowledge of
the card format.
DUMP OF THE FIRST 64 BYTES OF SECTOR ZERO (only colored areas are checked by the bootloader)
Where
$00..$1F = Copy of directory entry for P2BOOT.IMG
$20..$23 = Starting sector of boot file in contiguous blocks
$24..$27 = Number of bytes to load
$28..$2B = Directory entry address encoded as sector<<4 + 4-bits as 1 of 16 directory entries/sector (64GB address range)
$30..$33 = HH:MM:SS time encoded as pure decimal (i.e. 3.45.12 PM is 34512 )
$34..$37 == YY:MM:DD date encoded as pure decimal
Checking this a little:
TF2# $20 $20 SD DUMPL
00.0020: 0000.F280 0010.0000 0004.002A 0000.0000 ........*.......
00.0030: 0002.5B47 0002.7369 0000.0000 0000.0000 G[..is.......... ok
Looking at the time and date stamps that were used
TF2# $30 x@ . 154439 ok
TF2# $34 x@ . 160617 ok
My SD backup routine in Tachyon for reference.
PS: I'm assuming this topic is targeting what Chip should be including in the Prop2 ROM.
@jmg - yes, I like my SIlabs parts too it's just that I also have 8-pin PICs in MSOP or DIP and they run at 33MHz and include UARTS etc so they are easy to press into service. In fact I would just connect the PIC rxd to the Prop's txd and the PIC txd to the Prop's rxd which is then connected via a current limit resistor to the rxd of the system (USB/PropPlug). That way the PIC can override the rxd when it wants to and load the P2 serially while monitoring or controlling the reset. The idea of having the PIC listen to the P2 txd is so that we have a way of updating the SD/Flash bootloader held in the PIC or an attahced EEPROM. An update sequence can be detected and the new code accepted ready to be loaded into the P2 on the next reset. To allow PNut to load up the P2 on reset we would monitor the rxd line with a simple bit-bash routine and if all's quiet then the PIC would take over.
Some tolerance of SD card (mis)use needs to be there, and some check for a valid P2 boot image is also going to be useful.
It does not have to be a full, formal FAT scheme, so some field testing will be needed to get this to the
"As simple as possible, but no simpler" stage.
There also needs to be small code included, to exit any special SD modes before boot, as this may no always (re)start cold. (likewise for QuadSPI flash)
Elegant thought compression Peter.
As JMG says, the hard part is just making sure that a warm reset to SD SPI mode always works.
There's no difference between us here other than I'm proposing the boot block code takes control issuing the successive block reads of the large boot image rather than the ROM directly.
Is there any advantage to that as surely if the ROM can read one sector, it can read many. Why reinvent a sector read that the ROM already has?
No, not reinvented at all. The code loaded from the boot block will call the same ROM derived block read routines that were used for loading the boot block.
Are you saying that your bootloader in sector 0 would be doing something special that the ROM would not? Isn't it just as easy to say this is the starting sector of the raw file and load it for this many by bytes (or sectors)?
Yep, they both simple.
When I say the ROM I mean of course the part that I will write and test out along with the group to become a candidate for inclusion in the ROM.
Cramming into a tiny space is not a requirement or a necessity so to do so flies in the face of common sense although it might have made sense back in floppy disk days. But why limit the option to do so? Of course you can if you want but the ROM can also be told just to load sectors into memory too. Flexibility, simplicity.
My method can load a stretch of blocks, just like yours, with a CALL to the SD block reader function and an ADD or two, all inside of a REP. A boot sector writer program would have to modify just a few more bytes of the boot sector to work my way. My method is slightly more complicated for the user than yours, in that the boot sector writer program has to worry about a short loop that would otherwise be in ROM.
I don't understand how my method is less flexible than yours. It's more flexible, if anything. My method, in addition to being able to do what your method can do, can also do things such as load different images depending on hardware tests, or actually properly follow the FAT cluster chain of a fragmented file. The 446 bytes of bootloader space in the MBR is 111 instructions, slightly more than 1/5 of cog ram. That's probably plenty, especially if the actual SD block reader function that was used to load the boot sector is already in RAM.
I don't get your floppy disk argument. SD cards may be way bigger than floppies, but the boot sector is still only 512 bytes, and there are still only 446 bytes in the boot area.
Thanks, but that was a Q&D while this version is a tad better. If you want to backup 64k just type 64 SDBACKUP and it only takes half a second to backup the image at 80MHz.
Yes I agree, the long knives are coming out again for this thread.