Quite some time ago when I wanted to be able to boot the P2 from SD my thoughts were to have a minimal boot on the MBR that also contained an embedded physical sector address of the subsequent boot code. This would be a contiguous group of sectors that were in fact a file stored in a partition.
This would mean that, apart from the MBR sector being in a fixed location, the boot code would just be a contiguous block of code which would actually be a file in some file system. The boot code would not know anything about the file system (until after the secondary boot code was loaded) on the SD card. This would future proof the boot process. Therefore, the card could be any FAT16/32 or other system, with any partition system too.
Remember, the MBR only needs sufficient code to load a group of sectors from a fixed location on the SD card, verify it, and then jump to execute it. This would also permit encryption to be used.
The only code required in the P2 ROM would be to setup the SD card and read the MBR sector, verify it, and jump to execute it.
Note: The boot file (contiguous sectors) could be limited to the cluster size. If more sectors are required then that can be found (located) using the file system - the cluster size should be sufficient to boot a file system.
@Dave Hein
I tend to NOT think of the MBR as a partition, but as a 'Read first record of 512 bytes'. And I am not even sure it is specifically 8086 code (like a BIOS). It seems to just be a series of data fields to determine the kind of file system and a limited number of partition locations.
In theory, one might put anything in it for the Propeller to respond to. Non-conformance with prior standards would actually be a way to identify that the SDcard is intended to boot a Propeller.
There is a ton of history here as the MBR has been around for ages, but it can be used to simply provide locations on the SDcard to jump to. I generally DO NOT partition SDcards. And I generally adhere to the FAT16/32 formatting. At times, I have had them formated in Linux/Unix Ext3, but always felt that this might create oddball problems as the SDcard 'Association' seems to have never desired to support alternatives other than FAT16/32 and are not telling the world what exactly is under the hood.
The Propeller seems to have mature objects for the FAT16/32 use of SDcards. So the developmental focus would mainly be with a designated boot file system code for the Propeller, and not much more. One could later in Windows or Linux toggle the boot to other partitions and their related OSes. The main thing here is that all OSes mutually ignore the boot or mount of a file system that that is outside their scope -- unless there is good reason and special procedure for doing so.
The Propeller 2 would just confirm and ignore or use by reading the MBR, and move on.
Frankly, I really liked this idea, but I found Kye's skepticism compelling. Did the exploration work anyway, just sort of hoping an overall answer would be found.
Good on you for having a go, PH. To me it would be quite a tragedy if those comments by Kye were enough to thwart such a feature.
That's not to say the issues Kye hinted at won't surface, its just important to have a handle on their statistical influence. For instance it may be that we have to use slow (or fast) class of SD cards, or avoid really cheap brands, use newer cards, or provide guidance to end users, etc.
Best way we forumistas could contribute would be to crowd-test some P1/P1V/P2 "boot code" in the MBR, test with the range of SD cards we already possess, and see what issues surface.
On the contrary, the reality is enough to thwart the feature. All of those things you mention are fine, given some software. I'm not so sure that makes any sense being baked in.
From what I gather, many of the compatibility issues people experience when trying to interface SD cards to uCs stem from using the cards in SPI mode. I've read threads elsewhere where users contacted the card manufacturer and were told that their products weren't guaranteed to work in SPI mode (even though it's mandatory according to SD spec. I assume it's just easier for their support team to just use it as an excuse).
OK... I found my problem. For anyone else who runs into this issue, it is important to remember to send an extra 0xFF after getting responses. This gives the card an extra eight clock cycles to prepare itself for the next command. Some cards don't seem to need it (the Transcends that I'm using for example), but others require it.
I actually put a simple loop at the beginning of my 'write command' routine that sends 0xFF until it gets 0xFF as a response just so I don't have to go to all the different places where I read responses to make sure I put an extra send 0xFF. Because as far as the SD card is (usually) concerned in SPI mode, if there are no clock cycles coming in, time stands still.
The above solution also worked for a user with a Class 4 card, so the issue isn't just limited to dinky old cards.
edit: It would likely be preferable to operate in SD mode, but it looks like there would be a bit more overhead. Any thoughts on that?
Cards may support various combinations of the following bus types and transfer modes. The SPI bus mode and one-bit SD bus mode are mandatory for all SD families, as explained in the next section. Once the host device and the SD card negotiate a bus interface mode, the usage of the numbered pins is the same for all card sizes.
SPI bus mode: Serial Peripheral Interface Bus is primarily used by embedded microcontrollers. This bus type supports only a 3.3-volt interface. This is the only bus type that does not require a host license.
One-bit SD bus mode: Separate command and data channels and a proprietary transfer format.
Four-bit SD bus mode: Uses extra pins plus some reassigned pins. UHS-I and UHS-II requires this bus type.
The license is $1000 / year, per that same reference. Hmmm... based on what we've seen from Chip, any solution is going to have to use the SPI mode.
Now, that's also claimed to work (somehow) with *any* SD card, as an "open" and or "legacy" alternative. Somehow, this should be possible, license free too.
I talked to Chip at great length about this feature back in 2012. Chip wanted to know if booting from an SD card was simple, reliable, fast, etc. For example, SPI flash chips all use the same command to read data one byte at a time from address X sequentially. I investigated all the available brands of SPI flash chips on the market to make this conclusion. Because all chips use the same simple command to read data it makes the boot loader easy to write and reliable. Testing hundreds of different variants is not required because all SPI flashes use the same command with with the same electrical characteristics and timing as specified by JEDEC standards.
SD cards are not so simple. It's not a formatting issue. It's an initialization issue. Booting up an SD card requires sending a lot of different commands to it - this costs code space. Every long of the boot loader means less hub ram for everyone. If booting up an SD card was a simple as a SPI flash then the feature would be easy to do. But, it's not. Chip stopped being interested in this feature once I detailed to him all the work that has to be done to boot up the card only somewhat reliably. There are a bunch of other reasons that make doing this unpalatable too... but, the idea was dead to Chip once I walked him through the minimum of what had to be done.
A small SPI flash chip can be had for less than 50 cents in quantity of one. There's no need to hard code SD card support into the chip.
I'd agree the SD support is not really a ROM target, but better done as a secondary boot process.
The ROM boot is designed to be Size-settable, so the first boot step can load only as much code as is needed for Secondary Boot, thus saving time.
One new technology that might be simple enough to add to ROM, (along with the defaults of with Serial and SPI) is the new Spansion Hyperbus - basically an 8 bit wide, SPI DDR interface.
Some small HW in the smart pins may be needed here too, for DDR handling.
Even there, SPI Flash memory starts at ~ 18c/1k in SO8 and the smaller 2x3mm packages seems to be more at ~ 46c/1k
Commercially, it's also about piece parts. If a Flash part is not required when SD card is present, then it makes sense. SD would only be the 3rd sequence, so nothing really would be lost if there were some restrictions. There is not a lot of instructions for minimal MBR booting.
Unfortunately some seem to think it's harder than it needs to be.
Commercially, it's also about piece parts. If a Flash part is not required when SD card is present, then it makes sense. SD would only be the 3rd sequence, so nothing really would be lost if there were some restrictions. There is not a lot of instructions for minimal MBR booting.
Unfortunately some seem to think it's harder than it needs to be.
That means some means to quickly skip (no) flash is needed - maybe reading FFFFFFFFH as length is treated as no-flash ?
To support more mainstream flash and Full SD use, SD would need to be separate pins, then it could include a trigger-string and the loader clocks until string match.
IIRC Antii did some clever stuff around minimal SD reading ?
Just as the P1 determines there is no eeprom and therefore shuts down the P1, the P2 would determine there is no flash, and the try SD and if not found it would shutdown.
Since the SD card and Flash are both inSPI mode in P2, the same spi routines would be used. The CLK, DI, DO pins would be the same for both Flash and SD because theboot ROM code would be Flash first and if not present then SD. The only remaing question is the CE pin... IIRC the flash also has a CE pin, so this would also be common forbooting. An external pulldown could be required for the SD CE if required.
In the case of both Flash and SD, then the SD would use a different CE pin on the prop.. This would be handled by the user boot code in the flash.
Now because we have common spi routines in rom, code to setup the SD card for SPI acess to read the MBR sector is minimal. Remember, this rom code will not know anything about partitions or fat, just how to read one 512 byte sector and then execute it. The rest of the boot process is started with this one 512 byte MBR sector!
So, as I have maintained all along, there is nothing too complicated here, and if it fails to work on some SD cards then we just failed to get it right andnothing islost from not doing it at all. However, if as I maintain, if we gget it right, then we have gained a lot.
It is not a problem to tailor an SD card image.. I did so yesterday for a raspberry pi image.
Fwiw under windows to put an SD image on an o card under windows is as simple as using WIN32IMAGER and selecting the appropriate xxx.img fike and the SD drive. Of course to create/write the MBR on the SD card would require a somewhat simple pc program.
@Cluso99 - I understand you want SD card support. It's a cool feature. I would like to have it to if it didn't cost anything. But, the SD card booting code is not minimal. It cost less than 20 longs to support booting from flash. Just send out command h03 on the SPI bus with address h000000 and then read as many bytes as you like into hub ram. Flash support easily fits in the same code space with the P2 terminal code and encryption system support code.
For SD card booting your looking at hundreds of longs to support it. Not a marginal cost. Nor would supporting it have easily fit into the same boot loader space. If you haven't yet, take a look at my file system driver and how many longs are in ASM to support SD cards. While it's true that you may be able to shave off a few longs around the edges for SD card support you're still looking at around 200 longs.
...
Since the SD card and Flash are both in SPI mode in P2, the same spi routines would be used. The CLK, DI, DO pins would be the same for both Flash and SD because theboot ROM code would be Flash first and if not present then SD.
Some caution may be needed on share the pins, as that opens a P2 system to SD-socket attack, and other weaknesses.
Most well engineered system would likely prefer a Flash boot which would have a simple POST system test, before trying a SD load.
Such a start-up, should work reliably, even on shorted/damaged SD instances. (shared pins fails that requirement )
That avoids ever having a stone-cold-dead board to diagnose.
There are a bunch of other reason that make doing this unpalatable too... but, the idea was dead to Chip once I walked him through the minimum of what had to be done.
Kye and/or Chip could you perhaps enumerate what these other issues are, for the sake of completeness?
These could well be more important, than simply the number of longs required to do things "somewhat reliably".
@Tubular - The complexity of the code was Chip's main concern. He wasn't interested in further examination after knowing all the stuff you had to do. The size of boot loading code lowers how many hub bytes you have. An extra 1K of boot loader code means everyone gets 1K less of hub ram. Other issues that were brought up include Parallax having to have technical support for the feature and continually testing it against a moving target of SD card lineups - this is not something Chip wanted to do. I had this conversation about three years ago, so I don't remember everything. But, there were also some issues about which pins to use for SD card support and the boot order along with timeouts and such. In the end, Chip wanted something that was simple and reliable and SD cards were not that.
The size of boot loading code lowers how many hub bytes you have.
I though there was going to be the full 512K of RAM, and ROM would be copied into RAM at startup, meaning we would get the full 512K RAM no matter how much ROM there is? Or is that not happenning?
@Tubular - The complexity of the code was Chip's main concern. He wasn't interested in further examination after knowing all the stuff you had to do. The size of boot loading code lowers how many hub bytes you have. An extra 1K of boot loader code means everyone gets 1K less of hub ram. Other issues that were brought up include Parallax having to have technical support for the feature and continually testing it against a moving target of SD card lineups - this is not something Chip wanted to do. I had this conversation about three years ago, so I don't remember everything. But, there were also some issues about which pins to use for SD card support and the boot order along with timeouts and such. In the end, Chip wanted something that was simple and reliable and SD cards were not that.
Ok, thanks Kye. It's good to understand how these things make it in or out, and why.
To me, it would be a really nice feature to be "engineered in", taking advantage of the knowledge you've gained through your drivers. I regard this as a bit like the inbuilt "22pf" crystal caps on the P1 - it's no big deal to add a few caps externally, like every other micro, but these having been 'engineered in' to the P1 is a nice touch that tells you the designer has gone a little bit further to make your life easier.
I was surprised we are only talking about a few hundred longs to do this. That compares well with the booter/sha256/monitor code sizes, plus we're now looking at 512kB space I believe
The size of boot loading code lowers how many hub bytes you have.
Just to clarify, since I think a lot of this conversation revolves around the research you did, the "hundreds of longs" is dedicated to reliable SPI communication with SD cards, correct? You're not including MBR awareness and basic FAT16/32 bootloading in that estimate, are you? I'm quite curious about the details of how SD cards don't all speak SPI uniformly as that seems like a surefire way to not get approval for the SD mark. Was it timing issues? Weird ambiguities in the implementation of SPI?
And if you have the chance to take a quick look, could you give your thoughts on the StackOverflow link posted by mark
It's about 200 longs just to read sector into ram. So, such a boot loader would just read the first 2KB into RAM and boot a cog with that. This does not include FAT support of any kind. It wouldn't cost that much extra to add some code that inspected the MBR for a special partition and booted with that. If you want file level access you're looking at filling the cog space completely. Read through all the stuff I do in my file system driver (link below) in the mount function to get an idea of how to setup access to the file system on the card.
As for initialization of an SD card, please just read the ASM code for my file system block driver: http://obex.parallax.com/object/16. The steps to initialize the card for basic sector reading and writing are detailed in the code. It's not simple enough to write here...
...
I did a lot of research to get my SD card driver working with a lot of SD cards. Even then, I've had people tell me some don't work still. All the material I've cross referenced though says I'm doing everything right. Power supply issues may have caused problems for other people. I've personally not experienced an SD card that didn't work. Still, because I wrote my driver using many third party sources and not official SD card documentation this has always bothered me. While the SD card association publishes the SD card protocol on their website they do not publish a free comprehensive guide to all the electrical interface and timing issues. You have to pay if you want official information on clock cycle level timing and clock cycle tolerances for commands. The free SD card spec is vague about all the important details on purpose. (Paying for such information would make the boot loader not open source BTW. It would be covered under an NDA).
...
It's always the case that you could remove code that handles special cases to shrink the code size. However, doing so will just result in people thinking the feature is useless when they try their non-explicitly supported SD card and it does not work. Unless the driver can be written so that it's universal it will cost Parallax time and money handling support requests about the feature. Not to mention possibly the need to upgrade the code in the future like when SD cards transitioned to SDHC (SD card readers cannot read SDHC cards - the SD association changed the protocol - SDHC cards are basically all newer SD cards). One should keep in mind that all SD cards are going to be SDXC in the future. My driver should support these cards but I've never tested against one because they were $200+ when I wrote my file system driver.
...
About the Stack Overflow thing:
The original SD card spec left manufacturer's with a lot of flexibility to implement their SD cards. There's this data structure in the SD card you have to read to get the timeout values for a bunch of different operations. For example, you should timeout after X milliseconds of waiting for a read response. Where you get X from reading a 16 byte data structure called the CSD from a card. The CSD contains lots of bits that you need to check to assure that you can access an SD card safety. There's bits telling you the access time, bits telling you the file system type, bits telling you the card's supported voltage levels, etc. For extra fun, the CSD is a packed data structure meaning values sometimes are across multiple bytes!!! The structure was clearly designed to be read by a hardware module and not software. Getting the card size in 512 blocks requires a crazy amount of work for a value that should be easy to read.
By giving manufacturers all this flexibility the spec basically requires a complex driver to access a card. With v2.0 of the SD card spec the association removed a lot of this flexibility and required basically all SD cards to act the same. Even then, you're still supposed to read the CSD and interrogate it just in case. My driver doesn't do this and just uses maximum recommended values for timeouts. This works - albeit I'm not following the protocol...
Some of the weirdness from the original spec like providing extra clocks for some cards and not for others still exists as you see in that stack overflow article. The reality is that unless Parallax pays for official documentation about what to do any driver will never be 100% guaranteed. Compare this to flash support where you know it will work always.
Thank you for the detailed response! This is very interesting, and I can see how the experience makes you bearish on adding software SD support to the ROM. That CSD sounds like a barrel of fun.
The SD card reading can be of course implemented in the hardware as a state machine; I did it in Verilog for DE2-115. The verilog module can initialize the card, then read or write a sector. It took some hundreds LEs and 1k of FPGA RAM.
I tried to do it as simple as I can so it can only use SDHC cards (>= 4GB) but then it worked with some different SDHC cards I have. So, SD initializer and low level driver can be done in hardware too.
Then we can do the loading in two different ways:
(1) As in IBM PC. The fresh FAT32 formatted SD has a boot file on it, written in first user available clusters and unfragmented (remember this: format C: /s ?)
(2) As in ARM things (Raspbery Pi and others) - there is a special hidden partition with boot data.
yeah. I spend a lot of time with FSRW and @Kye's Fat_Engine. Kye supports all different SD cards FSRW some. But nowadays them >=4GB are the way to go. This shrinks down the code a lot.
I do not know much about Verilog, but if I understand you right, you have implemented a SD block driver in hardware for SDHC SD cards. Very cool.
If I am not completely wrong here the needed ROM code in the P2 could be quite small then, reading a) the first file or b) the given partition.
Nice.
I am saving money for a while now to attend the upcoming seminar at Parallax and get the Parallax FPGA.
I do not know much about Verilog, but if I understand you right, you have implemented a SD block driver in hardware for SDHC SD cards. Very cool.
Mike.
Yes And here it is.. in the state I left it some months ago after using it in one of my projects (the presentation machine for seminar) with the Nios processor (there was no P1V then) - unfinished like all my code but working.
// 2014 pik33@o2.pl
// gpl 2.0 or higher
module sd1(clk,rst,sclk,miso,mosi,cs,st,addr,secnum,datain,dataout,stb,wr,rd,ready,err,debug);
input wire clk;
input wire rst;
output reg sclk;
output reg mosi;
output reg cs;
input wire miso;
input wire [6:0] addr;
input wire [31:0] secnum;
input wire stb; // write strobe
input wire wr; // start sector write
input wire rd; // start sector read
input wire [31:0] datain;
output reg [31:0] dataout;
output reg ready;
output reg err;
output reg [7:0] st;
output reg[7:0] debug;
reg [7:0] wordcnt;
reg [31:0] writebuf[128];
reg [31:0] readbuf[128];
reg[15:0] cnt;
reg[4:0] state;
reg slowspiclk;
reg fastspiclk;
reg oldrst;
reg oldstb;
reg oldwr;
reg oldrd;
reg[56:0] cmd;
reg[48:0] res;
reg[48:0] res2;
reg[26:0] cnt2;
reg clk2;
reg [23:0] watchdog;
reg[31:0]qqqq;
initial
begin
cnt<=0;
state<=0;
mosi<=1;
sclk<=1;
cs<=1;
res<=0;
ready<=0;
err<=0;
watchdog<=0;
// $readmemh("wb.hex",writebuf);
end
always
begin
st<={err,ready,1'b0,state};
if (state==0) sclk<=1;
else if ((state>=1) && (state<14)) sclk<=slowspiclk;
else sclk<=fastspiclk;
end
always@(posedge clk)
qqqq<=writebuf[wordcnt];
always@(posedge clk)
begin
slowspiclk<=cnt[7]; //195 kHz
fastspiclk<=cnt[0]; //25 MHz
dataout<=readbuf[addr];
oldrst<=rst;
oldstb<=stb;
if ((stb==1) && (oldstb==0)) writebuf[addr]<=datain;
if ((rst==1) && (oldrst==0)) //posedge reset
begin
cnt<=0;
err<=0;
state<=1;
mosi=1;
cs=1;
end
else // if not reset
begin
//generate err and ready
if (state==14)
begin
err<=0;
ready<=1;
watchdog<=0;
end
else
begin
ready<=0;
if (watchdog>10_000_000)
begin
err<=1;
watchdog<=10_000_001;
end
else
watchdog<=watchdog+1;
end
// init state machine
if (state==0) //idle after powerup
begin
cnt<=0;
mosi<=1;
cs<=1;
end
else if (state==1) //send 88 clocks
begin
if (cnt[15:8]==88)
begin
cmd<=56'b11111111_01_000000_00000000_00000000_00000000_00000000_10010101; //cmd0
cnt<=1;
state<=2;
cs=0;
end
else cnt<=cnt+1;
end
else if (state==2) // send cmd0
begin
if (cnt[7:0]==4) mosi<=cmd[55];
if (cnt[7:0]==8)
begin
cmd<=(cmd<<1) | 56'b1;
end
if (cnt[15:8]==56)
begin
state<=3;
cnt<=1;
mosi<=1;
end
else
cnt<=cnt+1;
end
else if (state==3) // wait for 0 on miso
begin
if ((cnt[7:0]==127) && (miso==0))
begin
state<=4;
cnt<=128;
res<=(res<<1) | miso;
end
else cnt<=cnt+1;
end
else if (state==4)
begin
if (cnt[7:0]==127) res<=(res<<1) | miso;
cnt<=cnt+1;
if (cnt[15:8]==8) // TODO! check if res=1
begin
state<=5;
cnt<=0;
cmd<=56'b11111111_01_001000_00000000_00000000_00000001_10101010_10000111; //cmd8
end
end
else if (state==5) // send cmd8
begin
if (cnt[7:0]==4)
mosi<=cmd[55];
if (cnt[7:0]==8)
begin
cmd<=(cmd<<1) | 56'b1;
end
if (cnt[15:8]==56)
begin
state<=6;
cnt<=1;
end
else
cnt<=cnt+1;
end
else if (state==6) // wait for 0 on miso
begin
if ((cnt[7:0]==127) &&(miso==0))
begin
state<=7;
cnt<=128;
end
else cnt<=cnt+1;
end
else if (state==7)
begin
if (cnt[7:0]==127)
res<=(res<<1) | miso;
cnt<=cnt+1;
if (cnt[15:8]==40)
begin
state<=8;
cnt<=0;
cmd<=56'hFF_77_00_00_00_00_FF;
end
end
else if (state==8) // send cmd55
begin
if (cnt[7:0]==4)
mosi<=cmd[55];
if (cnt[7:0]==8)
begin
cmd<=(cmd<<1) | 56'b1;
end
if (cnt[15:8]==56)
begin
state<=9;
cnt<=0;
end
else
cnt<=cnt+1;
end
else if (state==9) // wait for 0 on miso
begin
if ((cnt[7:0]==127) &&(miso==0))
begin
state<=10;
cnt<=128;
end
else cnt<=cnt+1;
end
else if (state==10)
begin
if (cnt[7:0]==127)
res<=(res<<1) | miso;
cnt<=cnt+1;
if (cnt[15:8]==8) // TODO! check if res=1
begin
state<=11;
cnt<=1;
cmd<=56'hFF_69_40_00_00_00_FF; //cmd41
end
end
else if (state==11) // send cmd41
begin
if (cnt[7:0]==4)
mosi<=cmd[55];
if (cnt[7:0]==8)
begin
cmd<=(cmd<<1) | 56'b1;
end
if (cnt[15:8]==56)
begin
state<=12;
cnt<=1;
end
else
cnt<=cnt+1;
end
else if (state==12) // wait for 0 on miso
begin
if ((cnt[7:0]==127) &&(miso==0))
begin
state<=13;
cnt<=128;
end
else cnt<=cnt+1;
end
else if (state==13)
begin
if (cnt[7:0]==127)
res<=(res<<1) | miso;
cnt<=cnt+1;
if (cnt[15:8]==8)
begin
if (res[7:0]==0)
begin
state<=14;
cnt<=0;
cs<=1;
end
else
begin
cmd<=56'hFF_77_00_00_00_00_FF; //cmd55
state<=8;
cnt<=1;
end
end
end
else if (state==14) // ready for read/write
begin
if ((rd==1) && (oldrd==0)) // posedge read
begin
cmd<= {16'hFF_51,secnum,8'hFF};// (56'hFF_51_00_00_00_00_FF | (secnum<<8));
state<=15;
cnt<=0;
cs<=0;
end
else if ((wr==1) && (oldwr==0)) // posedge write
begin
cmd<={16'hFF_58,secnum,8'hFF};//(56'hFF_58_00_00_00_00_FF | (secnum<<8));
state<=16;
cnt<=1;
cs<=0;
end
else
begin
state<=14;
cnt<=0;
cs<=1;
end
end
else if (state==15) // send cmd17
begin
if (cnt[0]==0)
begin
mosi<=cmd[55];
cmd<=(cmd<<1) | 56'b1;
end
if (cnt[7:1]==56)
begin
state<=17;
cnt<=1;
end
else
begin
state<=15;
cnt<=cnt+1;
end
end
else if (state==17) // wait for 0 on miso
begin
if (cnt[0]==1)
begin
mosi<=cmd[55];
res<=(cmd<<1) | miso;
end
if ((cnt[0]==1) && (miso==0))
begin
state<=18;
cnt<=0;
end
else
begin
cnt<=cnt+1;
state<=17;
end
end
else if (state==18) //get response
begin
if (cnt[0]==1) res<=(res<<1) | miso;
if (cnt[7:1]==8) // TODO! check if res=1
begin
debug<=res;
state<=19;
cnt<=1;
end
else
begin
cnt<=cnt+1;
state<=18;
end;
end
else if (state==19) // wait for 0 on miso => FE sent
begin
if (cnt[0]==1) res<=(res<<1) |miso;
if ((cnt[0]==1) && (miso==0))
begin
debug<=res;
state<=20;
cnt<=0;
wordcnt<=0;
end
else
begin
cnt<=cnt+1;
state<=19;
end
end
else if (state==20)
begin
if (cnt[0]==1) res<=(res<<1) | miso;
if (cnt[7:1]==32)
begin
cnt<=1;
wordcnt<=wordcnt+1;
if (wordcnt==129)
begin
state<=14;
cnt<=0;
cs<=1;
end
else if (wordcnt<128)
begin
readbuf[wordcnt]<=res;
end
end
else
begin
cnt<=cnt+1;
state<=20;
end
end
/// ------ write -----
else if (state==16) // send cmd24
begin
if (cnt[0]==0)
begin
mosi<=cmd[55];
cmd<=(cmd<<1) | 56'b1;
end
if (cnt[7:1]==56)
begin
state<=21;
cnt<=0;
end
else
cnt<=cnt+1;
end
else if (state==21) // wait for 0 on miso
begin
if (miso==0)
begin
state<=22;
cnt<=0;
end
else cnt<=cnt+1;
end
else if (state==22) //get response
begin
if (cnt[0]==1)
res<=(res<<1) | miso;
cnt<=cnt+1;
if (cnt[7:1]==8) // TODO! check if res=1
begin
state<=23;
cnt<=0;
cmd<=(48'hFF_FE_00_00_00_00 | (secnum<<8));
end
end
else if (state==23 ) //send ---- FFFE----!!!----
begin
res<=32'hFF_FF_FF_FE;
state<=24;
wordcnt=0;
end
else if (state==24) // send writebuf
begin
if (cnt[0]==0)
begin
mosi<=res[31];
res<=(res<<1) | 32'b1;
end
if (cnt[7:1]==32)
begin
cnt<=0;
wordcnt<=wordcnt+1;
if (wordcnt<128) res<=qqqq;
if (wordcnt==128)
begin
state<=25;
res<=32'hffffffff;
end
end
else cnt<=cnt+1;
end
else if (state==25)// send ffff
begin
if (cnt[0]==1) res<=(res<<1) | 32'b1;
if (cnt[7:1]==16)
begin
cnt<=0;
state<=26;
end
else cnt<=cnt+1;
end
else if (state==26) // wait for 0 on miso
begin
if ((cnt[0]==1) && (miso==0))
begin
state<=27;
cnt<=0;
end
else cnt<=cnt+1;
end
else if (state==27) //get response
begin
if (cnt[0]==1)
res<=(res<<1) | miso;
cnt<=cnt+1;
if (cnt[7:1]==8)
begin
debug<=res;
if (res[7:4]==4'b0101)
begin
state<=28;
cnt<=0;
end
end
end
else if (state==28) //wait until ff
begin
if (cnt[0]==1)
res<=(res<<1) | miso;
cnt<=cnt+1;
if (cnt[7:1]==8)
begin
// debug<=res;
if (res[7:0]==8'hFF)
begin
cs<=1;
state<=14;
cnt<=0;
end
end
end
else // error!!!
begin
cnt<=0;
mosi<=1;
cs<=1;
state<=31;
end
end
end
endmodule
Yes but look was their website says: "The company was spun-off as a Winbond affiliate in 2008 while Winbond continues to focus on its large Memory business."
Do you think that is easy for Parallax to do the same?. They finally gave up on using custom layout and are using an affiliate company of OnSemi. Every simple thing that needs to be added to the design (I2C, SPI, DDR, ...) means money. HUGE MONEY.
Comments
This would mean that, apart from the MBR sector being in a fixed location, the boot code would just be a contiguous block of code which would actually be a file in some file system. The boot code would not know anything about the file system (until after the secondary boot code was loaded) on the SD card. This would future proof the boot process. Therefore, the card could be any FAT16/32 or other system, with any partition system too.
Remember, the MBR only needs sufficient code to load a group of sectors from a fixed location on the SD card, verify it, and then jump to execute it. This would also permit encryption to be used.
The only code required in the P2 ROM would be to setup the SD card and read the MBR sector, verify it, and jump to execute it.
Note: The boot file (contiguous sectors) could be limited to the cluster size. If more sectors are required then that can be found (located) using the file system - the cluster size should be sufficient to boot a file system.
I tend to NOT think of the MBR as a partition, but as a 'Read first record of 512 bytes'. And I am not even sure it is specifically 8086 code (like a BIOS). It seems to just be a series of data fields to determine the kind of file system and a limited number of partition locations.
In theory, one might put anything in it for the Propeller to respond to. Non-conformance with prior standards would actually be a way to identify that the SDcard is intended to boot a Propeller.
There is a ton of history here as the MBR has been around for ages, but it can be used to simply provide locations on the SDcard to jump to. I generally DO NOT partition SDcards. And I generally adhere to the FAT16/32 formatting. At times, I have had them formated in Linux/Unix Ext3, but always felt that this might create oddball problems as the SDcard 'Association' seems to have never desired to support alternatives other than FAT16/32 and are not telling the world what exactly is under the hood.
The Propeller seems to have mature objects for the FAT16/32 use of SDcards. So the developmental focus would mainly be with a designated boot file system code for the Propeller, and not much more. One could later in Windows or Linux toggle the boot to other partitions and their related OSes. The main thing here is that all OSes mutually ignore the boot or mount of a file system that that is outside their scope -- unless there is good reason and special procedure for doing so.
The Propeller 2 would just confirm and ignore or use by reading the MBR, and move on.
http://en.wikipedia.org/wiki/Master_boot_record
Good on you for having a go, PH. To me it would be quite a tragedy if those comments by Kye were enough to thwart such a feature.
That's not to say the issues Kye hinted at won't surface, its just important to have a handle on their statistical influence. For instance it may be that we have to use slow (or fast) class of SD cards, or avoid really cheap brands, use newer cards, or provide guidance to end users, etc.
Best way we forumistas could contribute would be to crowd-test some P1/P1V/P2 "boot code" in the MBR, test with the range of SD cards we already possess, and see what issues surface.
Agreed on your last point.
Here's the SD testing I helped with: http://forums.parallax.com/showthread.php/141952-SD-Card-Test-%28Please-test-may-be-used-for-boot-code-for-the-Prop2%29/page9
I found the following StackOverflow thread to be quite interesting, particularly the OP's own solution which might just help with a lot of the stubborn cards out there http://stackoverflow.com/questions/2365897/initializing-sd-card-in-spi-issues
The above solution also worked for a user with a Class 4 card, so the issue isn't just limited to dinky old cards.
edit: It would likely be preferable to operate in SD mode, but it looks like there would be a bit more overhead. Any thoughts on that?
I recall someone on this forum mentioning something along those lines, maybe Chip?
The license is $1000 / year, per that same reference. Hmmm... based on what we've seen from Chip, any solution is going to have to use the SPI mode.
Now, that's also claimed to work (somehow) with *any* SD card, as an "open" and or "legacy" alternative. Somehow, this should be possible, license free too.
SD cards are not so simple. It's not a formatting issue. It's an initialization issue. Booting up an SD card requires sending a lot of different commands to it - this costs code space. Every long of the boot loader means less hub ram for everyone. If booting up an SD card was a simple as a SPI flash then the feature would be easy to do. But, it's not. Chip stopped being interested in this feature once I detailed to him all the work that has to be done to boot up the card only somewhat reliably. There are a bunch of other reasons that make doing this unpalatable too... but, the idea was dead to Chip once I walked him through the minimum of what had to be done.
A small SPI flash chip can be had for less than 50 cents in quantity of one. There's no need to hard code SD card support into the chip.
The ROM boot is designed to be Size-settable, so the first boot step can load only as much code as is needed for Secondary Boot, thus saving time.
One new technology that might be simple enough to add to ROM, (along with the defaults of with Serial and SPI) is the new Spansion Hyperbus - basically an 8 bit wide, SPI DDR interface.
Some small HW in the smart pins may be needed here too, for DDR handling.
Even there, SPI Flash memory starts at ~ 18c/1k in SO8 and the smaller 2x3mm packages seems to be more at ~ 46c/1k
Unfortunately some seem to think it's harder than it needs to be.
The way I'm reading the debate is there is, however, a lot of instructions for initialising an SD card to ensure it's talking in SPI mode.
The discussions over block handling is academic until card init is sorted.
To support more mainstream flash and Full SD use, SD would need to be separate pins, then it could include a trigger-string and the loader clocks until string match.
IIRC Antii did some clever stuff around minimal SD reading ?
Since the SD card and Flash are both inSPI mode in P2, the same spi routines would be used. The CLK, DI, DO pins would be the same for both Flash and SD because theboot ROM code would be Flash first and if not present then SD. The only remaing question is the CE pin... IIRC the flash also has a CE pin, so this would also be common forbooting. An external pulldown could be required for the SD CE if required.
In the case of both Flash and SD, then the SD would use a different CE pin on the prop.. This would be handled by the user boot code in the flash.
Now because we have common spi routines in rom, code to setup the SD card for SPI acess to read the MBR sector is minimal. Remember, this rom code will not know anything about partitions or fat, just how to read one 512 byte sector and then execute it. The rest of the boot process is started with this one 512 byte MBR sector!
So, as I have maintained all along, there is nothing too complicated here, and if it fails to work on some SD cards then we just failed to get it right andnothing islost from not doing it at all. However, if as I maintain, if we gget it right, then we have gained a lot.
It is not a problem to tailor an SD card image.. I did so yesterday for a raspberry pi image.
Fwiw under windows to put an SD image on an o card under windows is as simple as using WIN32IMAGER and selecting the appropriate xxx.img fike and the SD drive. Of course to create/write the MBR on the SD card would require a somewhat simple pc program.
For SD card booting your looking at hundreds of longs to support it. Not a marginal cost. Nor would supporting it have easily fit into the same boot loader space. If you haven't yet, take a look at my file system driver and how many longs are in ASM to support SD cards. While it's true that you may be able to shave off a few longs around the edges for SD card support you're still looking at around 200 longs.
Most well engineered system would likely prefer a Flash boot which would have a simple POST system test, before trying a SD load.
Such a start-up, should work reliably, even on shorted/damaged SD instances. (shared pins fails that requirement )
That avoids ever having a stone-cold-dead board to diagnose.
Kye and/or Chip could you perhaps enumerate what these other issues are, for the sake of completeness?
These could well be more important, than simply the number of longs required to do things "somewhat reliably".
I though there was going to be the full 512K of RAM, and ROM would be copied into RAM at startup, meaning we would get the full 512K RAM no matter how much ROM there is? Or is that not happenning?
Ok, thanks Kye. It's good to understand how these things make it in or out, and why.
To me, it would be a really nice feature to be "engineered in", taking advantage of the knowledge you've gained through your drivers. I regard this as a bit like the inbuilt "22pf" crystal caps on the P1 - it's no big deal to add a few caps externally, like every other micro, but these having been 'engineered in' to the P1 is a nice touch that tells you the designer has gone a little bit further to make your life easier.
I was surprised we are only talking about a few hundred longs to do this. That compares well with the booter/sha256/monitor code sizes, plus we're now looking at 512kB space I believe
However, I'd really just like a faster P1 with more memory right now. Feature creep is not needed. The P1 is becoming very dated...
Just to clarify, since I think a lot of this conversation revolves around the research you did, the "hundreds of longs" is dedicated to reliable SPI communication with SD cards, correct? You're not including MBR awareness and basic FAT16/32 bootloading in that estimate, are you? I'm quite curious about the details of how SD cards don't all speak SPI uniformly as that seems like a surefire way to not get approval for the SD mark. Was it timing issues? Weird ambiguities in the implementation of SPI?
And if you have the chance to take a quick look, could you give your thoughts on the StackOverflow link posted by mark
As for initialization of an SD card, please just read the ASM code for my file system block driver: http://obex.parallax.com/object/16. The steps to initialize the card for basic sector reading and writing are detailed in the code. It's not simple enough to write here...
...
I did a lot of research to get my SD card driver working with a lot of SD cards. Even then, I've had people tell me some don't work still. All the material I've cross referenced though says I'm doing everything right. Power supply issues may have caused problems for other people. I've personally not experienced an SD card that didn't work. Still, because I wrote my driver using many third party sources and not official SD card documentation this has always bothered me. While the SD card association publishes the SD card protocol on their website they do not publish a free comprehensive guide to all the electrical interface and timing issues. You have to pay if you want official information on clock cycle level timing and clock cycle tolerances for commands. The free SD card spec is vague about all the important details on purpose. (Paying for such information would make the boot loader not open source BTW. It would be covered under an NDA).
...
It's always the case that you could remove code that handles special cases to shrink the code size. However, doing so will just result in people thinking the feature is useless when they try their non-explicitly supported SD card and it does not work. Unless the driver can be written so that it's universal it will cost Parallax time and money handling support requests about the feature. Not to mention possibly the need to upgrade the code in the future like when SD cards transitioned to SDHC (SD card readers cannot read SDHC cards - the SD association changed the protocol - SDHC cards are basically all newer SD cards). One should keep in mind that all SD cards are going to be SDXC in the future. My driver should support these cards but I've never tested against one because they were $200+ when I wrote my file system driver.
...
About the Stack Overflow thing:
The original SD card spec left manufacturer's with a lot of flexibility to implement their SD cards. There's this data structure in the SD card you have to read to get the timeout values for a bunch of different operations. For example, you should timeout after X milliseconds of waiting for a read response. Where you get X from reading a 16 byte data structure called the CSD from a card. The CSD contains lots of bits that you need to check to assure that you can access an SD card safety. There's bits telling you the access time, bits telling you the file system type, bits telling you the card's supported voltage levels, etc. For extra fun, the CSD is a packed data structure meaning values sometimes are across multiple bytes!!! The structure was clearly designed to be read by a hardware module and not software. Getting the card size in 512 blocks requires a crazy amount of work for a value that should be easy to read.
By giving manufacturers all this flexibility the spec basically requires a complex driver to access a card. With v2.0 of the SD card spec the association removed a lot of this flexibility and required basically all SD cards to act the same. Even then, you're still supposed to read the CSD and interrogate it just in case. My driver doesn't do this and just uses maximum recommended values for timeouts. This works - albeit I'm not following the protocol...
Some of the weirdness from the original spec like providing extra clocks for some cards and not for others still exists as you see in that stack overflow article. The reality is that unless Parallax pays for official documentation about what to do any driver will never be 100% guaranteed. Compare this to flash support where you know it will work always.
I tried to do it as simple as I can so it can only use SDHC cards (>= 4GB) but then it worked with some different SDHC cards I have. So, SD initializer and low level driver can be done in hardware too.
Then we can do the loading in two different ways:
(1) As in IBM PC. The fresh FAT32 formatted SD has a boot file on it, written in first user available clusters and unfragmented (remember this: format C: /s ?)
(2) As in ARM things (Raspbery Pi and others) - there is a special hidden partition with boot data.
yeah. I spend a lot of time with FSRW and @Kye's Fat_Engine. Kye supports all different SD cards FSRW some. But nowadays them >=4GB are the way to go. This shrinks down the code a lot.
I do not know much about Verilog, but if I understand you right, you have implemented a SD block driver in hardware for SDHC SD cards. Very cool.
If I am not completely wrong here the needed ROM code in the P2 could be quite small then, reading a) the first file or b) the given partition.
Nice.
I am saving money for a while now to attend the upcoming seminar at Parallax and get the Parallax FPGA.
Enjoy!
Mike.
Yes And here it is.. in the state I left it some months ago after using it in one of my projects (the presentation machine for seminar) with the Nios processor (there was no P1V then) - unfinished like all my code but working.
http://forums.parallax.com/showthread.php/159734-Memory-Interface-and-IO-Peripherals-in-2015?p=1311017&viewfull=1#post1311017
Yes but look was their website says: "The company was spun-off as a Winbond affiliate in 2008 while Winbond continues to focus on its large Memory business."
Do you think that is easy for Parallax to do the same?. They finally gave up on using custom layout and are using an affiliate company of OnSemi. Every simple thing that needs to be added to the design (I2C, SPI, DDR, ...) means money. HUGE MONEY.