@Sapieha - SD cards that do not have an MBR will not be supported. In fact... Non-SDHC (or newer SD cards) may not be supported. Older SD cards and MMC cards are full of a bunch of problems that Chip did not like hearing about when I told them about them. He is is favor of simple solutions.
@Cluso99 and @all - The MBR contains the size of the partition in sectors and the starting sector address of the partition. Thus, you could write a contiguous file to the FAT partition and then point the partition the propeller will boot from to the file. Thus, you have your solution where you can update by drag and drop. SIMPLE!
(But, you cannot trust where a file is on the FAT and if it's contiguous without having block level access to the SD card... So, the link between the partition pointer and the file could be broken)
@Hinv - You should have joined in with me and Chip's 3 hour discussion. There are many caveats with SD cards.
@Sapieha - SD cards that do not have an MBR will not be supported. In fact... Non-SDHC (or newer SD cards) may not be supported. Older SD cards and MMC cards are full of a bunch of problems that Chip did not like hearing about when I told them about them. He is is favor of simple solutions.
@Cluso99 and @all - The MBR contains the size of the partition in sectors and the starting sector address of the partition. Thus, you could write a contiguous file to the FAT partition and then point the partition the propeller will boot from to the file. Thus, you have your solution where you can update by drag and drop. SIMPLE!
(But, you cannot trust where a file is on the FAT and if it's contiguous without having block level access to the SD card... So, the link between the partition pointer and the file could be broken)
@Hinv - You should have joined in with me and Chip's 3 hour discussion. There are many caveats with SD cards.
If You place that file on freshly formated SD / HD / FLOPY
It always will start as first block in file area and be contiguous.
Hehe, yes, we know that works when it's a fresh format. But I'm pretty certain we can do better than having to reformat every time. Ie: I think the Raspberry Pi method sucks, personally.
NONE of OS's can place its SYSTEM files without reformatting.
That is price You need pay for possibility to BOOT from FLOPY / HD / SD and any other media.
Hehe, yes, we know that works when it's a fresh format. But I'm pretty certain we can do better than having to reformat every time. Ie: I think the Raspberry Pi method sucks, personally.
If the Propeller put a prop boot partition (not partition type 0) and a unformatted FAT partition on an SD card and then you put the card into a windows computer to format it, I'm pretty sure it would only format and know that it would only show the FAT partition. I have an SD card with partition 0 being a propeller partition (I needed fast disk access) and partition 1 being a FAT partition. I formatted it with Linux (GParted and then fdisk to change the partition type), and neither the Propeller nor the computer have had problems with it. In fact, I completely forgot about it until now!
NONE of OS's can place its SYSTEM files without reformatting.
While it may be the normal to do the reformat, and repartition too, it's not mandatory.
On top of that, with the Prop, the amount of data needed to be loaded, to support FAT32 filesystem, is very small. After this is loaded normal files can be used for the main program. On the other hand, if a filesystem is not desired then by all means set up a separate partition.
Either method can be supported as per current plans.
EDIT: Whatever the method chosen, a tool for managing the raw blocks (Whether it's a contiguous file, or using the reserved space, or a whole other partition) should be written or integrated into a front-end.
evanh: FAT16 & FAT32 cards are available preformatted today, but perhaps not tomorrow. Why force something that's not required?
For the minimal boot, I anticipate that only 1024 bytes will initially be required. We could use a few of the boot records but they have been defined with differing uses depending on where you look. That is why I suggested a pointer to a sector where 2 contiguous sectors could be found. As an alternative, use 2 pointers to 2 sectors if you wish (or a few more)?
Why such a commotion with what filesystem/structure needs to be on the SD card??? Use the MBR record as a pointer(s). This IS WORKING NOW !!! I posted code and so has ariba. We both only have to trim the code to simplified pasm. Our concepts are very similar.
Anyway, I have a little time so I will spend a bit more time on it. For those who wish to give this a try (or help), see my previous posts #535, #552 & #558 on page 14, #567 on page 15, and ariba's code on #786 on page 20). While it was more difficult in hindsight than starting again, I wanted to use code that I know works - I ship this code on all my RamBlades. It is just a matter of simplifying the current code and then converting the spin code to pasm.
Clusso, my code is already trimmed to 151 longs in PASM 1 code. The difference is I test only one long instead of a 12 char string, and this at location 400.
This new version combines the shortcut in the MBR (or what else is at sector 0) with the sectorwise searching for a signature at the begin of a boot-file. It does work here with SD cards (<=2GB) , and SDHC cards (>= 4GB).
For a bootable SD card you copy the bootfile to the fresh formated SD card - that was it.
Then the bootloader finds no shortcut in the MBR so it searches up to sector 19000 for a sector with the signature. This takes several seconds.
So if you want it faster use a (Spin)- Utility to write the shortcut, and now it boots in a few ms.
You can overwrte the boot file everytime with new code if it fits in the already allocated clusters. If you want disable booting you need to overwrite it with a file with wrong signature, just deleting will not help.
I have changed the place of the MBR shortcut a bit, so I have also attached the new utility.
' Minimal SD booter Test
' ----------------------
' by Andy Schenk, Insonix
con
_clkmode = xtal1 + pll16x
_xinfreq = 5_000_000
do = 8 'sd pins
clk = 9
di = 10
cs = 11
obj dbg: "FullDuplexSerial"
pub Main : i | tm
'
' This Spin code is not part of the boot code, it
' just starts the PASM cog and shows the result
'
waitcnt(clkfreq*3+cnt)
dbg.start(31,30,0,115200)
cognew(@entry, 0)
dbg.str(string("searching file...",13))
tm := cnt
repeat until (i := long[$7000-4])
if i < 0
dbg.str(string("booting failed: "))
dbg.dec(i)
if i== -10
dbg.str(string(13,"(no boot file)"))
if i== -13
dbg.str(string(13,"(timeout SD access)"))
else
dbg.str(string("found at sector: "))
dbg.dec(i)
dbg.str(string(" in ms:"))
dbg.dec((cnt-tm)/(clkfreq/1000))
dbg.tx(13)
byte[$707F] := 0 'show first 127 chars
dbg.str($7000)
repeat
dat
org
entry
' set up
mov blocksh,#0
or outa,clkmask
mov dira,outmask 'pins
neg phsb,#1
mov ctr2,onek '1000 x High for init
oneloop
call #sendiohi
djnz ctr2,#oneloop
mov starttime,cnt
mov ctr2,#9 'try a few times
reslp mov cmdo,#0
mov cmdp,#0
call #cmd
cmp accb,#1 wz
if_z jmp #tsthc
djnz ctr2,#reslp
tsthc mov cmdo,#8 'test if SD2
mov cmdp,#$1AA
call #cmd
cmp accb,#1 wz
if_nz jmp #resok '< no
mov ctr,#32
call #sendiocnt 'read volt
inilp mov cmdo,#55
mov cmdp,#0
call #cmd
mov cmdo,#41
mov cmdp,bit30
call #cmd
cmp accb,#0 wz
if_nz jmp #inilp
mov cmdo,#58 'test if SDHC
mov cmdp,#0
call #cmd
mov ctr,#32
call #sendiocnt
or outa,csmask
test accb,bit30 wz
if_z mov blocksh,#9
jmp #begin
resok or outa,csmask
call #sendiohi
initloop
mov cmdo,#55
call #cmd
mov cmdo,#41
call #cmd
cmp accb,#1 wz
if_z jmp #initloop
mov blocksh,#9 '0=SDHC 9=SD
' search signature and startsector in sector 0
begin mov blocknr,#0
mov bufad,hubbuf
call #rblock 'read sector 0
mov bufad,hubbuf
add bufad,#400
rdlong accb,bufad 'boot signature at 400 ?
cmp accb,signatur0 wz
if_nz jmp #search 'no signature: search sector
add bufad,#4
rdlong blocknr,bufad 'read boot startsector at 404
' search sector with boot file signature
search mov bufad,hubbuf
call #rblock
rdlong accb,hubbuf
cmp accb,signaturFile wz
if_z jmp #found
add blocknr,#1
cmp blocknr,lastblock wc
if_b jmp #search
neg accb,#10 'not found error: -10
fback sub hubbuf,#4
wrlong accb,hubbuf
halt jmp #halt
found mov lastblock,#3 'load boot code (only 3 sectors here)
:load call #rblock
djnz lastblock,#:load
mov accb,blocknr
jmp #fback 'okay
rblock
mov ctr2,sectorsz 'read block
mov starttime,cnt
mov cmdo,#17
mov cmdp,blocknr
call #cmd
call #readresp
rbyte
mov accb,#8
andn outa,clkmask
or outa,clkmask
test domask,ina wc
addx acca,acca
djnz accb,#rbyte+1
wrbyte acca,bufad
add bufad,#1
djnz ctr2,#rbyte
call #sendiohi
call #sendiohi
or outa,csmask
rblock_ret ret
sendio
rol phsb,#24 'SPI routines
sendiohi
mov ctr,#8
sendiocnt
mov accb,#0
bit andn outa,clkmask
rol phsb,#1 wc
muxc outa,dimask
or outa,clkmask
test domask,ina wc
addx accb,accb
djnz ctr,#bit
sendio_ret
sendiohi_ret
sendiocnt_ret
ret
checktime
mov duration,cnt
sub duration,starttime
cmp duration,clockfreq wc
checktime_ret
if_c ret
neg accb,#13 'timeout error: -13
jmp #fback
cmd
cmp cmdo,#41 wz
if_z or outa,csmask
nop
andn outa,csmask
neg phsb,#1
call #sendiohi
mov phsb,cmdo
add phsb,#$40
call #sendio
mov phsb,cmdp
shl phsb,blocksh
call #sendiohi
call #sendiohi
call #sendiohi
call #sendiohi
cmp cmdo,#8 wz
if_z mov phsb,#$87
if_nz mov phsb,#$95
call #sendio
cmp cmdo,#12 wz
if_z neg phsb,#1
if_z call #sendiohi
readresp
neg phsb,#1
call #sendiohi
call #checktime
cmp accb,#$ff wz
if_z jmp #readresp
cmd_ret
readresp_ret
ret
clockfreq long 80_000_000
onek long 1000
sectorsz long 512
bit30 long 1<<30
domask long 1<<do
csmask long 1<<cs
clkmask long 1<<clk
dimask long 1<<di
outmask long 1<<clk | 1<<di | 1<<cs
lastblock long 19000
signatur0 byte "BOOT"
signaturFile byte "P2BO"
hubbuf long $7000
acca res 1
accb res 1
cmdo res 1
cmdp res 1
ctr res 1
ctr2 res 1
starttime res 1
duration res 1
bufad res 1
blocknr res 1
blocksh res 1
Andy
Edit: removed the copyright. The code is based on the FSRW driver from rokicki and lonesock, but havy modified and portet to PASM by me.
A separate partition wouldn't leave you tied to FAT.
The obvious response is: Why is so many people wanting to use SD cards? Ease of transporting data I would have thought. A common easy to use filesystem would seem to be expected.
Yes, I was saying TWO partitions. One for the Prop boot image. Personally, I don't like the hacks in ROM. They are great, don't get me wrong. I think they would work too. It's not that.
It's all about using the device properly and doing so in a standards based way. If we don't have filesystem support, the right answer is a partition. That is why they exist. Doing it in the *right* way, standard in other words, is part of a pro-quality device, IMHO. Supporting filesystem hacks is different from supporting a partition too. Once again, great hacks, I'm not using that in a negative way here. Just would prefer standards compliant operation, that's all. I'll not say any more about it, other than I really don't care whether or not it boots SD at all. The other storage solution can very easily support an SD card, much like we do now on P1, and it could do so with a far more robust process, and can be updated / changed easily too.
Older SD cards and MMC cards are full of a bunch of problems...
Exactly!!! That is precisely why I think it is utter folly to say that P2 SD bootloader code must support every Tom, Dick, and Harry SD card in the world or else it shouldn't be done at all. What???
Of course I'm not suggesting leaving users in the dark, either. Just tell them "The P2 bootloader supports XYZ cards." No apology needed.
(The exact nature of XYZ remains to be determined. Maybe it's SDHC.)
That is precisely why I think it is utter folly to say that P2 SD bootloader code must support every Tom, Dick, and Harry SD card in the world or else it shouldn't be done at all. What???
Kye does have a bit of point:
- If the ROM was written a few years back we may not have support for SDHC. There would be a lot of frustrated people having to use non-HC's for booting but after that can use anything.
- Now, with SDXC just coming onto the market, how long before SDHC becomes expensive and old school?
It's all about using the device properly and doing so in a standards based way. If we don't have filesystem support, the right answer is a partition.
All three options are valid. Separate partition needs the image copied to it, reserved space similar but just the loader, contiguous file for loader requires a tool to ensure it stays contiguous when modified. They all need to update the first block with the pointer. Under these conditions the methods comply.
All three options are valid. Separate partition needs the image copied to it, reserved space similar but just the loader, contiguous file for loader requires a tool to ensure it stays contiguous when modified. They all need to update the first block with the pointer. Under these conditions the methods comply.
Why don't you just have the P-II use the P-I boot up system. It's dirt simple and easy to modify. Ideally the P-II should had flash like every other micro out there but it doesn't so you'll always need external memory.
Booting from SD's make sense when you have something like the Raspberry not a micro-controller.
Just been trying to work out how modern Linuxes get booted ... not that easy to identify, most documentation, particularly webpages like Wikipedia, just talks about the general process as a concept and completely miss the technical detail of how the main loader is loaded itself.
Finally found it on page 41 of the Grub2 manual under the heading of "GRUB image files". The critical file being diskboot.img. It's a single 512 byte block and contains a list of block numbers and counts for loading the rest of the loader images that contain the filesystem driver for the root partition and basic console/shell that then loads Linux itself.
So, Linux needs a tool to build booting just like everyone else using the PC, but it's done away with needing contiguous files or any reserved space.
EDIT: I should clarify by saying that that's just the norm for your average BIOS based PC. It's certainly not the only way Grub can operate, hence the vagueness of the general documentation.
Reformatting just for a 10kB file seems a bit crappy. This is the Prop. We'll be fiddling heaps. Lots of fixes.
In that case you can program a second-stage loader that looks in the FAT filesystem for the real file to load. Then you only have to do that once per SD card and you can rewrite the program loaded by the second-stage loader as many times as you want without reformatting.
Andy: I missed your previous code because I downloaded your file and looked at that. Then realised that the posted code was not included in the zip file - perhaps you may like to add it in.
Can we first get a consensus of what the MBR shall be modified to???
1. Offset for parameter(s)? I have chosen $180. Andy has chosen 400decimal. My preference is a hex value. I am also happy with $100 which also seems to be fine. Thoughts???
2. The identifier? I agree that 1 long is probably sufficient. So let us on what that long shall be??? Any suggestions??? "P2x"+$0 (i.e. a string?) or a hex value???
3. The pointer? a long to the first boot code sector??? Do we require a second sector pointer or just use the first +1. From what I understand has been said, the Flash uses 1024 bytes initially, so we should conform to this.
4. If we add Andy's search if this fails, what should the search string/long/whatever be???
If we can at least agree with this then we can interchange each others code to test without having to re-write the MBR each time.
How many block numbers are you planning on storing? If you are loading a 126KB image, potentially you could have 252 blocks.
I am not that familiar with clusters, but if you address clusters instead of blocks, and the clusters are only 4k, it could take 32 of them, and we cannot assume that they are one right after the other.
I like the idea of using a special partition that's a contiguous region on the SD card. As far as other operating systems are concerned, this partition is an area that's normally not to be touched, yet a utility can be written, like a special version of fdisk, that can create, delete, modify, update this partition and its contents. Such a utility can be done for any of the standard operating systems (Linux, Windows, and the MacOS). For that matter, a utility can be written for the Prop-1 to do this, then later modified for use with the Prop-2.
Hehe, yes, we know that works when it's a fresh format. But I'm pretty certain we can do better than having to reformat every time. Ie: I think the Raspberry Pi method sucks, personally.
If the boot file(s) are of a defined size, do you need to reformat every time ?
Or can one not rely on an overwrite doing exactly that ? - Not even as part of the Prop Tools flow ?
How many block numbers are you planning on storing?
The exact number is speculation and, for the moment, isn't important. The general idea is to enforce contiguous blocks with a tool though. And given it's minimal nature will not be modified very often.
If the boot file(s) are of a defined size, do you need to reformat every time ?
Or can one not rely on an overwrite doing exactly that ? - Not even as part of the Prop Tools flow ?
It can be written anywhere in the partition as long as it's contiguous. The pointer in block 0 gets updated accordingly each time.
So, no, reformatting wouldn't do anything useful.
EDIT: Marking this file as SYSTEM and READONLY would be a good idea. Gives a good hint not to touch it with general file handling.
I like the idea of using a special partition that's a contiguous region on the SD card. As far as other operating systems are concerned, this partition is an area that's normally not to be touched, yet a utility can be written, like a special version of fdisk, that can create, delete, modify, update this partition and its contents. Such a utility can be done for any of the standard operating systems (Linux, Windows, and the MacOS). For that matter, a utility can be written for the Prop-1 to do this, then later modified for use with the Prop-2.
This sounds ok, as anyone doing Boot changes, will have a full set of tools on their PC.
Could those tools can also VERIFY the Boot-rom pathway, the Prop would use ?
Boot content will be relatively stable, and more casual users can still just copy their code files, and a partition somewhat protects against clobbering.
If such tools can be batch/command operated, then someone in the field can create a NEW boot SD without being an expert.
Comments
@Cluso99 and @all - The MBR contains the size of the partition in sectors and the starting sector address of the partition. Thus, you could write a contiguous file to the FAT partition and then point the partition the propeller will boot from to the file. Thus, you have your solution where you can update by drag and drop. SIMPLE!
(But, you cannot trust where a file is on the FAT and if it's contiguous without having block level access to the SD card... So, the link between the partition pointer and the file could be broken)
@Hinv - You should have joined in with me and Chip's 3 hour discussion. There are many caveats with SD cards.
This time I need say ---- You are wrong.
Aribs (Andy's) code will boot any SD
And with Values I pointed on one of pictures ---- You can calculate starting point of FILE area on any SD You imagine.
You maybe are good on HIGH level FILE System programing
But I see ---- You are not so good on LOW level on it.
I have worked with that since 1980 and I think I know little how it function --- BUT still I don't say I know all on it
If You place that file on freshly formated SD / HD / FLOPY
It always will start as first block in file area and be contiguous.
I have much more to say on that SD / HD / Flopy handle ---- BUT I see it no need for that as someone already have said --- THAT IS NOT POSSIBLE.
Hehe, yes, we know that works when it's a fresh format. But I'm pretty certain we can do better than having to reformat every time. Ie: I think the Raspberry Pi method sucks, personally.
NONE of OS's can place its SYSTEM files without reformatting.
That is price You need pay for possibility to BOOT from FLOPY / HD / SD and any other media.
While it may be the normal to do the reformat, and repartition too, it's not mandatory.
On top of that, with the Prop, the amount of data needed to be loaded, to support FAT32 filesystem, is very small. After this is loaded normal files can be used for the main program. On the other hand, if a filesystem is not desired then by all means set up a separate partition.
Either method can be supported as per current plans.
EDIT: Whatever the method chosen, a tool for managing the raw blocks (Whether it's a contiguous file, or using the reserved space, or a whole other partition) should be written or integrated into a front-end.
Reason: I'd prefer if the SD loader was file system agnostic, and not tied to FAT*
The obvious response is: Why is so many people wanting to use SD cards? Ease of transporting data I would have thought. A common easy to use filesystem would seem to be expected.
Yes
Andy's BOOT system are most flexible --- As it can BOOT from any position on SD.
Both with FAT and without it
Else as Chip said --- Reuse space from END of BOOT sector to first sector that is used by Partition !!
It's all about using the device properly and doing so in a standards based way. If we don't have filesystem support, the right answer is a partition. That is why they exist. Doing it in the *right* way, standard in other words, is part of a pro-quality device, IMHO. Supporting filesystem hacks is different from supporting a partition too. Once again, great hacks, I'm not using that in a negative way here. Just would prefer standards compliant operation, that's all. I'll not say any more about it, other than I really don't care whether or not it boots SD at all. The other storage solution can very easily support an SD card, much like we do now on P1, and it could do so with a far more robust process, and can be updated / changed easily too.
Exactly!!! That is precisely why I think it is utter folly to say that P2 SD bootloader code must support every Tom, Dick, and Harry SD card in the world or else it shouldn't be done at all. What???
Of course I'm not suggesting leaving users in the dark, either. Just tell them "The P2 bootloader supports XYZ cards." No apology needed.
(The exact nature of XYZ remains to be determined. Maybe it's SDHC.)
Kye does have a bit of point:
- If the ROM was written a few years back we may not have support for SDHC. There would be a lot of frustrated people having to use non-HC's for booting but after that can use anything.
- Now, with SDXC just coming onto the market, how long before SDHC becomes expensive and old school?
All three options are valid. Separate partition needs the image copied to it, reserved space similar but just the loader, contiguous file for loader requires a tool to ensure it stays contiguous when modified. They all need to update the first block with the pointer. Under these conditions the methods comply.
BTW, the pointer is, in effect, the bootblock.
Why You need it to have possibility to modify..
As I know none of systems can that without reformatting
Booting from SD's make sense when you have something like the Raspberry not a micro-controller.
I don't think SDXC will ever be as small or as cheap as SDHC.
Finally found it on page 41 of the Grub2 manual under the heading of "GRUB image files". The critical file being diskboot.img. It's a single 512 byte block and contains a list of block numbers and counts for loading the rest of the loader images that contain the filesystem driver for the root partition and basic console/shell that then loads Linux itself.
So, Linux needs a tool to build booting just like everyone else using the PC, but it's done away with needing contiguous files or any reserved space.
EDIT: I should clarify by saying that that's just the norm for your average BIOS based PC. It's certainly not the only way Grub can operate, hence the vagueness of the general documentation.
How many block numbers are you planning on storing? If you are loading a 126KB image, potentially you could have 252 blocks.
I am not that familiar with clusters, but if you address clusters instead of blocks, and the clusters are only 4k, it could take 32 of them, and we cannot assume that they are one right after the other.
If the boot file(s) are of a defined size, do you need to reformat every time ?
Or can one not rely on an overwrite doing exactly that ? - Not even as part of the Prop Tools flow ?
The exact number is speculation and, for the moment, isn't important. The general idea is to enforce contiguous blocks with a tool though. And given it's minimal nature will not be modified very often.
It can be written anywhere in the partition as long as it's contiguous. The pointer in block 0 gets updated accordingly each time.
So, no, reformatting wouldn't do anything useful.
EDIT: Marking this file as SYSTEM and READONLY would be a good idea. Gives a good hint not to touch it with general file handling.
This sounds ok, as anyone doing Boot changes, will have a full set of tools on their PC.
Could those tools can also VERIFY the Boot-rom pathway, the Prop would use ?
Boot content will be relatively stable, and more casual users can still just copy their code files, and a partition somewhat protects against clobbering.
If such tools can be batch/command operated, then someone in the field can create a NEW boot SD without being an expert.