Does anyone bother reading the FSRW or FATEngine ASM code for talking to an SD card?
The protocol is VERY complex. It is not simple at all.
...
Are you saying that it is so complex to initialize an SDCard, and load a few blocks (253 of them for a full load) couldn't be done in 150 longs even given SPI code already?
Aren't SDCards usable as SPI devices?
I understand the desire to have boot capability from the SD card. But, the end result is that even more pins will need to be used by the boot loader statically. Right now the boot loader takes 6 pins. You'll need 7 for SD card boot. You can't share the CS pin between a SPI flash chip and an optional SD card. This will cause problems.
Me thinks you assume that everybody that wants to boot off of SD will have an SPI flash chip also. Many have pointed out that they want to NOT have and SPI flash chip.
Additionally, for anyone who wants real performance out of the P2 and wants to make a competitive product you'll not want to share the SPI bus between the SPI flash chip and the SD card. If you want to use external flash to store code you'll most likely want to have either quadSPI or octalSPI action going on attached to the P2.
If you don't you will make cache misses very expensive for the C LMM machine (or flexible SPIN interpreter).
I didn't know we were discussing using SD card as a XLMM type memory, I think previous posts were referring to SDRAM.
SPI mode for SD cards do not support clock rates faster than 25 MHz. This is on purpose by the SD card association to make the SPI bus not suitable for commercial products. With quadSPI or octalSPI parts you can drive the clock up to 100 MHz.
25MHz looks like it would load a 1008Kbit (126KB) full hub load in 1/25th of a second plus overhead...maybe conservatively 1/10th of a second. Sound's plenty fast for a lot of devices to me.
It sounds good to have SD card boot. But, the devil is in the details. Ignoring the fact that the SD card boot loader will not likely work with every SD card is not acceptable to build into the hardware. Building SD card boot into the hardware should only be done if Parallax plans to license the SD card protocol from the SD card association and do it right.
This could be tested out today on the P1 without licensing protocols.
And again... most people who want the P2 for speed and performance will not share the SD card with the Flash bus. The performance you can get out of Octal SPI will run circles around an SD card.
Did you know that an over-clocked to 100 MHz FSRW driver using the fastest block driver on the P1 hits about the maximum write speed in RAW mode that you can achieve on an SD card. Do you want to be stuck with sub 2 MBs speed limits? With 100 MHz octalSPI flash you can hit 100 MBs a second.
The only way to hit the really fast read and write speeds on an SD card is to have sequential reads and writes (not random reads and writes... which is what is needed for a execute in place setup), use 1.8V signaling and license the SD card spec form the SD card association.
SPI flash memory is actually designed for execute in place (XIP) performance. This means random reads on SPI flash memory are designed to be fast.
...
*An octal SPI flash setup is where you have 2 quadSPI chips hooked up. This setup cost a $1 for about 16 MB of space. An SD card will likely cost you more money. Why pay for a 8 GB SD card when you do not need one and it will not be as fast?*
It's a good thing we now have a few more pins to work with;^) With that kind of performance, it sounds like justification to use some of them.
It seems to me this is an argument for SPI instead of SD Card, when in fact the argument is about having only SPI, or having SPI plus SDCard.
I don't know of anyone here that has argued that booting from SPI flash should not be an option.
Sorry, for sounding angry in this post. I'm just a bit upset that many people seems to ignore the elephant in the room. Please look at the SD card init code in either the FAT Engine or FSRW and see how complex it is. Then understand that that neither the FATEngine or FSRW is perfect and the P2 needs to be.
This means Chip will have to devote a large portion of his time testing every SD card he can find under the sun for compatibility.
Once again, stated another way, are you saying that it cannot be done given a subset of reliable SDCards to work with, a SPI driver already in rom, and another 150 longs to work with?
Given how small you are talking - 1024 bytes, I'm thinking just boot the MBR instead. Let the code in the MBR do the dirty work, that's what it's purpose is. Obviously still isn't enough for filesystem support but would be enough to continue on as you've just outlined.
No. There is only about 440 bytes available on the MBR and there is a second 512byte sector used by Microsoft for further booting - but this maynot be guaranteed. Therefore better to just point to a location on the SD. Because this can be a file, it can easily be updated. But It should really only load further filesystem data to then load a proper user code file. However, as you can see, once we boot the 1024? byte loader, anything is possible and outside control (or restrictions) of the ROM.
When I was a budding SysAdmin at Motorola, we had over 1000 Sun3 machines that booted a boot block that loaded the kernel. In order to make a disk bootable, you had to run a utility called "installboot" which wrote the boot block telling where(what blocks) to find the kernel. I don't remember how big the kernel (SunOS version of unix) was, but it had to be in the 100s of KB at least. The way I figure it, if you are going for a cog ram only boot, that will take exactly 4 longs out of a 512Byte MBR for the block numbers. If you are going for a full hub load, it would take 252 longs, too much for a MBR(512 Bytes), but if you had block numbers followed by how many in a run(run length encoding), you could get it loaded in just about every case(except the most fragmented ones).
The beauty of this approach is it has already been done, works reliably(for hard disks) and doesn't mess with the filesystem.
If Chip decides to make the SD card boot loader he will just make it so that it looks for a special partition in the partition table that has the entire image in it. This is the only method that is feasable and that will work.
Sorry Kye, but this is a really bad idea. You are requiring a specifically formatted SD card. Why not be format agnostic and minimise any format requirements and therefore any risk in the ROM failing?
I have the code booting using my concept. It is simpler and does not force any requirements on the SD whatsoever, other than using a fixed 16 bytes of the MBR boot code (which has always been defined for booting info or data) on the MBR sector #0. No predefined partitions, no FAT filesystem, no other filesystem - old or new, and no raw data - anything is possible requiring the only known part being the MBR sector #0 and a fixed 16 bytes within the MBR boot code section (at offset $180 unles anyone has a better offset). The actual first stage boot code is stored as 1 or 2 sectors pointed to by the MBR. This can be anywhere on the SD card, but would typically be the data in a file using whatever filesystem was on the SD. Those 1 or 2 sectors would contain suffiient code to locate (if required) any further boot data for the secondary stage.
It cannot get any simpler than this. Therefore, the risk to the P2 ROM is absolutely minimised.
I just did some reading on bootable SD cards and partitions. Seems to me, this approach means being able to make a Prop II bootable SD card with standard Linux file system tools. mkfs and dd should be all that is needed. Lots of devices work this way. I like it. For windows, a program could be written to move the image, but fdisk can easily make the partition. Probably there is a path there too, I just don't know it yet. I personally don't do any low level disk stuff, besides a basic format under windows. That's what a *NIX is for!
Being able to setup an SD card with standard filesystem tools is a big plus in my book.
Has anyone mounted a multi-partition SD card on windows? I'm tempted to go off and create a simulated P2 one, with some goofy partition ID that's not been used before for the 128k image partition and a standard FAT type for a file system. Saw a ton of simple directions for building a partition of X size. This isn't a hard thing to do.
Seems to me, windows would just display that as two "disks", just as it does most everything else. Isn't it then easy to just put the files one needs onto the FAT filesystem, leaving the boot image and process clean, simple, consistent? For a ton of applications, that boot image partition only needs to be done once. I suppose updating it would make sense, if the application *were* the boot image too. Entirely possible. So then, do it in Linux with dd, or something similar with what ever OS somebody is running, or use another Prop to do it. I don't think that's hard either, just needs a simple document.
Could hide that partition too, I suppose. Maybe make the booter dumb, so it doesn't care if the partition is hidden, so long as it's there, it loads it, but OSes would just hide it, presenting the file system capable partition(s) only. On that note, I'm wondering if it even needs to respect the boot flag either. If it sees it's ID, then it goes and gets the blocks period. It's not like that partition would actually boot on anything else.
@Kye, would the booter read the partition size and get all the blocks, or just expect it to contain 128K? Could one make a smaller partition and load say, 32K in the case of it reading partition size? Or, could one make a bigger partition, insure the image is the first 128K, with extra data after that, in the case of it always fetching 128K?
Just wondering how the partition boot system might work more than anything else at this point.
That was as I said from start --- BUT no one either listen to me to.
BUT -- That not need NEW partition
ONLY requirement are that SD need be freshly formated.
Then first file that are written to SD need be BOOTxxxx.prg file.
So that file will be always as continuous block. and then can be read in RAW mode.
IT is that way DOS boots --- IT is why You don't can make Floppy disk bootable after You placed on it Yours personal files.
And to that You only need find start of that file after You looked in DIR if it EXIST !!
You can even make utility that with writing of that file to SD even PLACE info on it in BOOT sector like DOS do it and PLACE STARTING position of it TO in that boot info ---> and with BOOT.
Booting :
Simply READ boot sector
Find if BOOT file exist
load its placement starting position You placed in BOOT sector and LOAD
DONE
I had a long 3 hour conversation with Chip about all the things needed to support SD/MMC cards. There isn't space for FAT support. Justing mounting the SD card takes up about all the space. This leaves just enough room to read through the partition table and find a special propeller image partition and load it.
Everyone seems to keep ignoring me when I say that they have to supply code to mount the SD card. It's more than 100 longs on the P1 and will not likely be shrink with P2 opcodes. I have written this code before...
I advised Chip that the best solution for this problem of limited space would be to just create a propeller image partition on the SD card and store the image there. Most SD cards have large tracks of unused space before and after partition 0 on the SD card. So, a small 128K image can be tucked in this unused space. If there was no space then FAT partition can just be shrunken to make space.
The Propeller 2 would then just scan the master boot record for the the partition identifier character and then, if found, load up all the sectors on the partition. It's an easy solution to implement in ASM and will work on every card that supports a MBR.
Unfortunately, not all SD cards have an MBR. Such cards will not be supported as they are no longer manufactured. All SD cards from the SD2.0 standard released many years ago must have an MBR. This allows support for every SD card sold since SDHC cards came out.
---
Getting file level access is a lot of work. There are too many things to do to fit all that within 150 longs. Shortcuts can be taken... but, then compatibility will be lost. Please remember that files are not contiguous. Reading a file requires interrogation of the FAT as the file is read. Additionally, FAT32 and FAT16 auto switching is not easy.
A separate partition is the way to go. If you Google "MBR" and look at the byte layout you will see how trivially simple it is to support booting from a partition.
Okay, just been reading about FAT partitions on Wikipedia and, not surprisingly, there is preallocated reserved space at the start of the FAT partition. The default reserved size is 32 blocks (16 kB). Within this, blocks 0,1 and 6 are used for basic FAT32 structures. The rest are never used unless there is an OS booting from that partition. Funnily, that's exactly what we're doing.
So, without too much extra effort, the MBR could be skipped altogether in terms of poking data onto the SD card. And 14.5kB should be enough to do a decent boot from. The one assumption is that the SD card is formatted with one FAT32 partition, ie: how they are purchased.
There is no need for a special partition nor any filesystem support nor any hidden kludges. It can be completely legit just by using the provided space at the start of the regular FAT32 partition.
Those that want a bigger boot loader than 14.5kB can still reformat the singular partition and have the reserved space as large as desired and still be fully in spec.
I had a long 3 hour conversation with Chip about all the things needed to support SD/MMC cards. There isn't space for FAT support. Justing mounting the SD card takes up about all the space. This leaves just enough room to read through the partition table and find a special propeller image partition and load it.
Everyone seems to keep ignoring me when I say that they have to supply code to mount the SD card. It's more than 100 longs on the P1 and will not likely be shrink with P2 opcodes. I have written this code before...
I advised Chip that the best solution for this problem of limited space would be to just create a propeller image partition on the SD card and store the image there. Most SD cards have large tracks of unused space before and after partition 0 on the SD card. So, a small 128K image can be tucked in this unused space. If there was no space then FAT partition can just be shrunken to make space.
The Propeller 2 would then just scan the master boot record for the the partition identifier character and then, if found, load up all the sectors on the partition. It's an easy solution to implement in ASM and will work on every card that supports a MBR.
Unfortunately, not all SD cards have an MBR. Such cards will not be supported as they are no longer manufactured. All SD cards from the SD2.0 standard released many years ago must have an MBR. This allows support for every SD card sold since SDHC cards came out.
---
Getting file level access is a lot of work. There are too many things to do to fit all that within 150 longs. Shortcuts can be taken... but, then compatibility will be lost. Please remember that files are not contiguous. Reading a file requires interrogation of the FAT as the file is read. Additionally, FAT32 and FAT16 auto switching is not easy.
A separate partition is the way to go. If you Google "MBR" and look at the byte layout you will see how trivially simple it is to support booting from a partition.
Thanks,
Kye, as my previous post. You are overcomplicating this. Use the MBR as you and I both agree. But use this as a pointer to a physical data location, not a partition location. This way, if you (or Chip) want it to be in it's own partition, it can be. But you don't have to force that on the rest of us. We don't require a partition to be created (and the software to create it). We only require a simple program (see my code that I posted a few days ago - ariba has also since posted similar code - both of us have used a P1 to do this with older fsrw code than yours). It is possible for a partition to be used or not, a file to be used or not, or anything else.
One question is why more devices aren't exploiting that kind of thing? Could it be tragedy of the commons worries? Doing it with a partition is "in spec" standard, expected, etc... and the ROM supports a complete boot too. Maybe supports partials, or small boots, who knows? But it would be complete, robust, standard. That's what partitions are actually for.
One question is why more devices aren't exploiting that kind of thing? Could it be tragedy of the commons worries? Doing it with a partition is "in spec" standard, expected, etc... and the ROM supports a complete boot too. Maybe supports partials, or small boots, who knows? But it would be complete, robust, standard. That's what partitions are actually for.
The answer to that question may be, how many devices boot from SD card?
We don't really know how many stages there are to the R-Pi booting. It may well do exactly what I'm saying.
Windoze uses this area, according to Wikipedia, when it's booting from a FAT partition.
From what I can tell, lots of them! Google around like I did. It was actually an interesting read.
You know, if we do it standard, one could make a bootable SD card that contains Prop dev tools, and that would turn right around and boot on a Prop as well! LOL!!
Edit: In my post above, I meant why aren't more devices exploiting file system hacks as discussed, not why aren't they using partitions. Lots of stuff uses a partition.
There is no need for a special partition nor any filesystem support nor any hidden kludges. It can be completely legit just by using the provided space at the start of the regular FAT32 partition.
Those that want a bigger boot loader than 14.5kB can still reformat the singular partition and have the reserved space as large as desired and still be fully in spec.
evanh: FAT16 & FAT32 cards are available preformatted today, but perhaps not tomorrow. Why force something that's not required?
For the minimal boot, I anticipate that only 1024 bytes will initially be required. We could use a few of the boot records but they have been defined with differing uses depending on where you look. That is why I suggested a pointer to a sector where 2 contiguous sectors could be found. As an alternative, use 2 pointers to 2 sectors if you wish (or a few more)?
Why such a commotion with what filesystem/structure needs to be on the SD card??? Use the MBR record as a pointer(s). This IS WORKING NOW !!! I posted code and so has ariba. We both only have to trim the code to simplified pasm. Our concepts are very similar.
Anyway, I have a little time so I will spend a bit more time on it. For those who wish to give this a try (or help), see my previous posts #535, #552 & #558 on page 14, #567 on page 15, and ariba's code on #786 on page 20). While it was more difficult in hindsight than starting again, I wanted to use code that I know works - I ship this code on all my RamBlades. It is just a matter of simplifying the current code and then converting the spin code to pasm.
You know, if we do it standard, one could make a bootable SD card that contains Prop dev tools, and that would turn right around and boot on a Prop as well! LOL!!
Yeah, pretty cool all right. Someone mentioned that earlier too.
Sorry if I say that ---- I think You don't read my post's else others that SAY boot from standard partition.
PC don't support booting from SD at all --- NO worries it will it will be disturbed.
BUT still --- PC DOS don have any HIGH Level disk IO in BIOS --- only RAW read from floppy / HD.
And it is all that are needed in Propeller for BOOTING from SD.from and PARTITION ---->
in stages I described in previous posts
It is one NEED for have it in Partition ---> I will place more files on that SD after I have initiated BOOT on it.
As That boot will be only OS to load my USER program from SD.
ANT that apply to all USERS of bootable SD card
And are simplest to
evanh: FAT16 & FAT32 cards are available preformatted today, but perhaps not tomorrow. Why force something that's not required?
For the minimal boot, I anticipate that only 1024 bytes will initially be required. We could use a few of the boot records but they have been defined with differing uses depending on where you look. That is why I suggested a pointer to a sector where 2 contiguous sectors could be found. As an alternative, use 2 pointers to 2 sectors if you wish (or a few more)?
Why such a commotion with what filesystem/structure needs to be on the SD card??? Use the MBR record as a pointer(s). This IS WORKING NOW !!! I posted code and so has ariba. We both only have to trim the code to simplified pasm. Our concepts are very similar.
Anyway, I have a little time so I will spend a bit more time on it. For those who wish to give this a try (or help), see my previous posts #535, #552 & #558 on page 14, #567 on page 15, and ariba's code on #786 on page 20). While it was more difficult in hindsight than starting again, I wanted to use code that I know works - I ship this code on all my RamBlades. It is just a matter of simplifying the current code and then converting the spin code to pasm.
Yeah, pretty cool all right. Someone mentioned that earlier too.
You know that thing about like minds... Must have missed it.
Was thinking about having to do something to the SD card. No matter what, something has to be done to the SD card. It needs boot data added at a minimum, which means putting that data on the card. Pre-formatted, not formatted, strangely formatted all doesn't matter, does it? The card needs boot prep period. So then, the boot prep process gets executed and it really doesn't matter what that process is, just that it has to be done. What that means is there isn't any advantage in just using the cards "as shipped", and in fact that's a dependency, where just writing the data in a standard way isn't.
If somebody just wants a bootable card, they develop their image and run a coupla commands, done! If they want extras, they author a script to automate, done. A hack into exploiting "unused" space really isn't a gain, unless somebody can show where having a partition really botches things. Is there a worry some how over whether or not a P2 bootable card can receive files from a P1, or something like that?
I don't see the net gain for partial, multi-stage, just get it done only to get it done again booting when it's really simple to just boot it solid, complete, standard.
Re: Required
Well, that's a matter of semantics isn't it? Technically, a partition is required so that devices that utilize SD cards can insure portions of that SD card comply with their requirements. When stuff is just put on there, raw, that doesn't insure that devices that utilize SD cards comply with their requirements. It's a hack. Should a hack be in the ROM?
evanh: FAT16 & FAT32 cards are available preformatted today, but perhaps not tomorrow. Why force something that's not required?
Hurm?! I'm no M$ fan but FAT32 has a huge amount of life in it for simplistic file sharing. You don't have complex structures nor user privileges to deal with. Perfect for removable storage.
If you like, the starting block number can still be written into the MBR, just make that number point into the start of the FAT32 partition, say block 2. The ROM code doesn't need to even know there is a partition and at the same time the boot code is in standard location for a FAT partition. Load up block 2 and 3 to give the 1kB of initial boot code and from there it can carry on loading the remaining 13.5kB.
Get the best of both then.
EDIT: The one down side of modifying the MBR like this is some OS tools have a tendency to completely rebuild the MBR to how that one company thinks it should be. Wiping out what was there.
Yes it does --- IF You will have simple way to handle that SD even in PC to have possibility to place Yours USER program that YOU will load after You have BOOTED
Loader OS from it
You know that thing about like minds... Must have missed it.
Was thinking about having to do something to the SD card. No matter what, something has to be done to the SD card. It needs boot data added at a minimum, which means putting that data on the card. Pre-formatted, not formatted, strangely formatted all doesn't matter, does it? The card needs boot prep period. So then, the boot prep process gets executed and it really doesn't matter what that process is, just that it has to be done. What that means is there isn't any advantage in just using the cards "as shipped", and in fact that's a dependency, where just writing the data in a standard way isn't.
If somebody just wants a bootable card, they develop their image and run a coupla commands, done! If they want extras, they author a script to automate, done. A hack into exploiting "unused" space really isn't a gain, unless somebody can show where having a partition really botches things. Is there a worry some how over whether or not a P2 bootable card can receive files from a P1, or something like that?
I don't see the net gain for partial, multi-stage, just get it done only to get it done again booting when it's really simple to just boot it solid, complete, standard.
This has been true for all kinds of boot devices and it's all about the BIOS and what support comes "in the box" I just checked and two of my machines will boot right off of SD, one is a little netbook (which is too small for me to consider using, but it does boot from SD), and the other is an ordinary HP laptop that I currently do Propeller related development on.
Over time, boot support had to evolve. CD/DVD, probably Blu-Ray but I've not checked as I don't use the media at all in computing, Network, USB, SD, etc...
That's worth honoring the standards right there.
What's the real objection to a partition? It's not a hard thing, and it's not a hack. They are the basic structures used to "partition" off a portion of a media for a specific purpose, say booting. It's hard to justify a hack in a ROM that's going to be around for a long time.
@Saphia, the partition doesn't change adding user data. The user data goes into the file system capable partition. Are you saying you can't actually add files to an SD card that has partitions on it?
Getting file level access is a lot of work. There are too many things to do to fit all that within 150 longs. Shortcuts can be taken... but, then compatibility will be lost. Please remember that files are not contiguous. Reading a file requires interrogation of the FAT as the file is read. Additionally, FAT32 and FAT16 auto switching is not easy.
A separate partition is the way to go. If you Google "MBR" and look at the byte layout you will see how trivially simple it is to support booting from a partition.
Thanks,
And that goes back to my point that *if* SD boot is implemented, it must be stupid simple for customers to use. Writing to specific blocks on the SD will require a special Parallax program to either use the P2 as a gateway or a Windoze program that does it natively.
Ironically every UNIX made should already have the utility: dd if=firmware.bin of=/dev/sd0 seek=1024 bs=512 count=258048
This has been true for all kinds of boot devices and it's all about the BIOS and what support comes "in the box" I just checked and two of my machines will boot right off of SD, one is a little netbook (which is too small for me to consider using, but it does boot from SD), and the other is an ordinary HP laptop that I currently do Propeller related development on.
Over time, boot support had to evolve. CD/DVD, probably Blu-Ray but I've not checked as I don't use the media at all in computing, Network, USB, SD, etc...
Lots of us could write that. Seems a non-issue to me, and it's really only for Windows, and for windows where somebody doesn't already have a prop. Since they will, a prop can just setup the boot partition and write out all the blocks. Lots of us could write that too.
SD CARD TEST Here is my simple program that will locate a file called "_VER.CMD" (only chose because that is how I boot my SD from eeprom now). It will display the MBR (sector #0), then update the 16 bytes at offset $180 with "PropellerII" $0 $xxxxxxxx where xxxxxxxx is the pointer to the start to a preboot data sector(s). In my case this is a complete P1 Propeller OS binary file called _VER.CMD. The MBR is then updated and re-displayed. My code then continues to boot the SD card.
Here is a copy of my MBR after update (it had already been written - my boot data is at sector $00000600)...
Writing to specific blocks on the SD will require a special Parallax program to either use the P2 as a gateway or a Windoze program that does it natively.
It's the price of having a SD boot option at all. EDIT: Correction, It's the price of having a minimal SD boot option in ROM.
Booting from SPI flash can do a nice job of managing SD cards, without even booting a card at all. Including file management and even repartitioning tools. ... Put a card in on power up and it'll try to boot it. Otherwise, fall back to management menus.
pedward: Did you see my post above regarding security? Do you see a problem that would allow the serial to continually try to break the prop code by just loading the short block and if it fails, reset the prop, and try again (i.e. brute force attack)? I am a little worried about this, hence my suggestion to add a fuse (defined in the boot ROM) to prohibit booting from the Serial.
A by-product of this would be the re-use of the pins - not that I really see this at issue.
I am also interested to see if anyones SD card has any data on the MBR at offset $180 where I place the PropellerII identifier and pointer. Please post your results if you try this.
That's safe. I'm thinking there's nothing wrong with putting the special string at start of MBR either. If anything else modifies the MBR it'll either leave the first 446 bytes untouched or it'll replace the lot.
pedward: Did you see my post above regarding security? Do you see a problem that would allow the serial to continually try to break the prop code by just loading the short block and if it fails, reset the prop, and try again (i.e. brute force attack)? I am a little worried about this, hence my suggestion to add a fuse (defined in the boot ROM) to prohibit booting from the Serial.
A by-product of this would be the re-use of the pins - not that I really see this at issue.
Brute force is a non-issue. The authentication is managed via SHA-256 HMAC, which relies on the strength of the SHA-256 algorithm. At present the DoD rates SHA-256 and AES-128 as "safe for 20 years" in terms of data security. It would be much faster, yet still insanely long, to use GPU or "cloud" resources to brute force the key instead of making the P2 go through the boot sequence.
Comments
Are you saying that it is so complex to initialize an SDCard, and load a few blocks (253 of them for a full load) couldn't be done in 150 longs even given SPI code already?
Aren't SDCards usable as SPI devices?
Me thinks you assume that everybody that wants to boot off of SD will have an SPI flash chip also. Many have pointed out that they want to NOT have and SPI flash chip.
I didn't know we were discussing using SD card as a XLMM type memory, I think previous posts were referring to SDRAM. 25MHz looks like it would load a 1008Kbit (126KB) full hub load in 1/25th of a second plus overhead...maybe conservatively 1/10th of a second. Sound's plenty fast for a lot of devices to me. This could be tested out today on the P1 without licensing protocols. It's a good thing we now have a few more pins to work with;^) With that kind of performance, it sounds like justification to use some of them.
It seems to me this is an argument for SPI instead of SD Card, when in fact the argument is about having only SPI, or having SPI plus SDCard.
I don't know of anyone here that has argued that booting from SPI flash should not be an option. Once again, stated another way, are you saying that it cannot be done given a subset of reliable SDCards to work with, a SPI driver already in rom, and another 150 longs to work with?
The beauty of this approach is it has already been done, works reliably(for hard disks) and doesn't mess with the filesystem.
Just throwing in my 10 bits
Doug
Sorry Kye, but this is a really bad idea. You are requiring a specifically formatted SD card. Why not be format agnostic and minimise any format requirements and therefore any risk in the ROM failing?
I have the code booting using my concept. It is simpler and does not force any requirements on the SD whatsoever, other than using a fixed 16 bytes of the MBR boot code (which has always been defined for booting info or data) on the MBR sector #0. No predefined partitions, no FAT filesystem, no other filesystem - old or new, and no raw data - anything is possible requiring the only known part being the MBR sector #0 and a fixed 16 bytes within the MBR boot code section (at offset $180 unles anyone has a better offset). The actual first stage boot code is stored as 1 or 2 sectors pointed to by the MBR. This can be anywhere on the SD card, but would typically be the data in a file using whatever filesystem was on the SD. Those 1 or 2 sectors would contain suffiient code to locate (if required) any further boot data for the secondary stage.
It cannot get any simpler than this. Therefore, the risk to the P2 ROM is absolutely minimised.
Being able to setup an SD card with standard filesystem tools is a big plus in my book.
Has anyone mounted a multi-partition SD card on windows? I'm tempted to go off and create a simulated P2 one, with some goofy partition ID that's not been used before for the 128k image partition and a standard FAT type for a file system. Saw a ton of simple directions for building a partition of X size. This isn't a hard thing to do.
Seems to me, windows would just display that as two "disks", just as it does most everything else. Isn't it then easy to just put the files one needs onto the FAT filesystem, leaving the boot image and process clean, simple, consistent? For a ton of applications, that boot image partition only needs to be done once. I suppose updating it would make sense, if the application *were* the boot image too. Entirely possible. So then, do it in Linux with dd, or something similar with what ever OS somebody is running, or use another Prop to do it. I don't think that's hard either, just needs a simple document.
Could hide that partition too, I suppose. Maybe make the booter dumb, so it doesn't care if the partition is hidden, so long as it's there, it loads it, but OSes would just hide it, presenting the file system capable partition(s) only. On that note, I'm wondering if it even needs to respect the boot flag either. If it sees it's ID, then it goes and gets the blocks period. It's not like that partition would actually boot on anything else.
@Kye, would the booter read the partition size and get all the blocks, or just expect it to contain 128K? Could one make a smaller partition and load say, 32K in the case of it reading partition size? Or, could one make a bigger partition, insure the image is the first 128K, with extra data after that, in the case of it always fetching 128K?
Just wondering how the partition boot system might work more than anything else at this point.
Thanks.
That was as I said from start --- BUT no one either listen to me to.
BUT -- That not need NEW partition
ONLY requirement are that SD need be freshly formated.
Then first file that are written to SD need be BOOTxxxx.prg file.
So that file will be always as continuous block. and then can be read in RAW mode.
IT is that way DOS boots --- IT is why You don't can make Floppy disk bootable after You placed on it Yours personal files.
And to that You only need find start of that file after You looked in DIR if it EXIST !!
You can even make utility that with writing of that file to SD even PLACE info on it in BOOT sector like DOS do it and PLACE STARTING position of it TO in that boot info ---> and with BOOT.
Booting :
Simply READ boot sector
Find if BOOT file exist
load its placement starting position You placed in BOOT sector and LOAD
DONE
There is no need for a special partition nor any filesystem support nor any hidden kludges. It can be completely legit just by using the provided space at the start of the regular FAT32 partition.
Those that want a bigger boot loader than 14.5kB can still reformat the singular partition and have the reserved space as large as desired and still be fully in spec.
Kye, as my previous post. You are overcomplicating this. Use the MBR as you and I both agree. But use this as a pointer to a physical data location, not a partition location. This way, if you (or Chip) want it to be in it's own partition, it can be. But you don't have to force that on the rest of us. We don't require a partition to be created (and the software to create it). We only require a simple program (see my code that I posted a few days ago - ariba has also since posted similar code - both of us have used a P1 to do this with older fsrw code than yours). It is possible for a partition to be used or not, a file to be used or not, or anything else.
The answer to that question may be, how many devices boot from SD card?
We don't really know how many stages there are to the R-Pi booting. It may well do exactly what I'm saying.
Windoze uses this area, according to Wikipedia, when it's booting from a FAT partition.
You know, if we do it standard, one could make a bootable SD card that contains Prop dev tools, and that would turn right around and boot on a Prop as well! LOL!!
Edit: In my post above, I meant why aren't more devices exploiting file system hacks as discussed, not why aren't they using partitions. Lots of stuff uses a partition.
For the minimal boot, I anticipate that only 1024 bytes will initially be required. We could use a few of the boot records but they have been defined with differing uses depending on where you look. That is why I suggested a pointer to a sector where 2 contiguous sectors could be found. As an alternative, use 2 pointers to 2 sectors if you wish (or a few more)?
Why such a commotion with what filesystem/structure needs to be on the SD card??? Use the MBR record as a pointer(s). This IS WORKING NOW !!! I posted code and so has ariba. We both only have to trim the code to simplified pasm. Our concepts are very similar.
Anyway, I have a little time so I will spend a bit more time on it. For those who wish to give this a try (or help), see my previous posts #535, #552 & #558 on page 14, #567 on page 15, and ariba's code on #786 on page 20). While it was more difficult in hindsight than starting again, I wanted to use code that I know works - I ship this code on all my RamBlades. It is just a matter of simplifying the current code and then converting the spin code to pasm.
Yeah, pretty cool all right. Someone mentioned that earlier too.
Sorry if I say that ---- I think You don't read my post's else others that SAY boot from standard partition.
PC don't support booting from SD at all --- NO worries it will it will be disturbed.
BUT still --- PC DOS don have any HIGH Level disk IO in BIOS --- only RAW read from floppy / HD.
And it is all that are needed in Propeller for BOOTING from SD.from and PARTITION ---->
in stages I described in previous posts
It is one NEED for have it in Partition ---> I will place more files on that SD after I have initiated BOOT on it.
As That boot will be only OS to load my USER program from SD.
ANT that apply to all USERS of bootable SD card
And are simplest to
You know that thing about like minds... Must have missed it.
Was thinking about having to do something to the SD card. No matter what, something has to be done to the SD card. It needs boot data added at a minimum, which means putting that data on the card. Pre-formatted, not formatted, strangely formatted all doesn't matter, does it? The card needs boot prep period. So then, the boot prep process gets executed and it really doesn't matter what that process is, just that it has to be done. What that means is there isn't any advantage in just using the cards "as shipped", and in fact that's a dependency, where just writing the data in a standard way isn't.
If somebody just wants a bootable card, they develop their image and run a coupla commands, done! If they want extras, they author a script to automate, done. A hack into exploiting "unused" space really isn't a gain, unless somebody can show where having a partition really botches things. Is there a worry some how over whether or not a P2 bootable card can receive files from a P1, or something like that?
I don't see the net gain for partial, multi-stage, just get it done only to get it done again booting when it's really simple to just boot it solid, complete, standard.
Re: Required
Well, that's a matter of semantics isn't it? Technically, a partition is required so that devices that utilize SD cards can insure portions of that SD card comply with their requirements. When stuff is just put on there, raw, that doesn't insure that devices that utilize SD cards comply with their requirements. It's a hack. Should a hack be in the ROM?
Really?
If you like, the starting block number can still be written into the MBR, just make that number point into the start of the FAT32 partition, say block 2. The ROM code doesn't need to even know there is a partition and at the same time the boot code is in standard location for a FAT partition. Load up block 2 and 3 to give the 1kB of initial boot code and from there it can carry on loading the remaining 13.5kB.
Get the best of both then.
EDIT: The one down side of modifying the MBR like this is some OS tools have a tendency to completely rebuild the MBR to how that one company thinks it should be. Wiping out what was there.
Yes it does --- IF You will have simple way to handle that SD even in PC to have possibility to place Yours USER program that YOU will load after You have BOOTED
Loader OS from it
Well, older ones don't, but newer ones do.
This has been true for all kinds of boot devices and it's all about the BIOS and what support comes "in the box" I just checked and two of my machines will boot right off of SD, one is a little netbook (which is too small for me to consider using, but it does boot from SD), and the other is an ordinary HP laptop that I currently do Propeller related development on.
Over time, boot support had to evolve. CD/DVD, probably Blu-Ray but I've not checked as I don't use the media at all in computing, Network, USB, SD, etc...
That's worth honoring the standards right there.
What's the real objection to a partition? It's not a hard thing, and it's not a hack. They are the basic structures used to "partition" off a portion of a media for a specific purpose, say booting. It's hard to justify a hack in a ROM that's going to be around for a long time.
@Saphia, the partition doesn't change adding user data. The user data goes into the file system capable partition. Are you saying you can't actually add files to an SD card that has partitions on it?
And that goes back to my point that *if* SD boot is implemented, it must be stupid simple for customers to use. Writing to specific blocks on the SD will require a special Parallax program to either use the P2 as a gateway or a Windoze program that does it natively.
Ironically every UNIX made should already have the utility: dd if=firmware.bin of=/dev/sd0 seek=1024 bs=512 count=258048
OK -- thanks.
Have not tested PC booting from SD.
I know it will boot from USB -- Flash HD's
But have not tested SD.
BUT as YOU said ---- "That's worth honoring the standards right there. " -- that is most right way to go !!
Lots of us could write that. Seems a non-issue to me, and it's really only for Windows, and for windows where somebody doesn't already have a prop. Since they will, a prop can just setup the boot partition and write out all the blocks. Lots of us could write that too.
Here is my simple program that will locate a file called "_VER.CMD" (only chose because that is how I boot my SD from eeprom now). It will display the MBR (sector #0), then update the 16 bytes at offset $180 with "PropellerII" $0 $xxxxxxxx where xxxxxxxx is the pointer to the start to a preboot data sector(s). In my case this is a complete P1 Propeller OS binary file called _VER.CMD. The MBR is then updated and re-displayed. My code then continues to boot the SD card.
Here is a copy of my MBR after update (it had already been written - my boot data is at sector $00000600)...
[code]
RamBlade3 SD test loader for P2 V1.12
MBR sector 0...
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
50 72 6F 70 65 6C 6C 65 72 49 49 00 00 06 00 00 PropellerII.....
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 02 ................
04 00 06 0A CA CA 81 00 00 00 7F AF 3B 00 00 00 ....
It's the price of having a SD boot option at all. EDIT: Correction, It's the price of having a minimal SD boot option in ROM.
Booting from SPI flash can do a nice job of managing SD cards, without even booting a card at all. Including file management and even repartitioning tools. ... Put a card in on power up and it'll try to boot it. Otherwise, fall back to management menus.
A by-product of this would be the re-use of the pins - not that I really see this at issue.
Having standard PC compatible partition ---> Simplify placing of user files on that SD
That's safe. I'm thinking there's nothing wrong with putting the special string at start of MBR either. If anything else modifies the MBR it'll either leave the first 446 bytes untouched or it'll replace the lot.
Brute force is a non-issue. The authentication is managed via SHA-256 HMAC, which relies on the strength of the SHA-256 algorithm. At present the DoD rates SHA-256 and AES-128 as "safe for 20 years" in terms of data security. It would be much faster, yet still insanely long, to use GPU or "cloud" resources to brute force the key instead of making the P2 go through the boot sequence.