Shop OBEX P1 Docs P2 Docs Learn Events
Propeller II - Page 33 — Parallax Forums

Propeller II

1303133353645

Comments

  • AribaAriba Posts: 2,682
    edited 2012-08-19 18:04
    Kye wrote: »
    ....
    Something to think about... whatever code that will be used for the SD card driver needs to be able to get the card out of multi block mode. No one's mentioned this but the SD card booting scheme will fail because of multi block mode if the boot loader does not support it.

    This is because people will likely use the SD card in multi block mode during run time. But, if the boot loader code cannot get the card out of multi block mode your system will need to be power cycled to be reset.

    This is a serious issue... I alerted Chip to this in my conversation with him. It's a hidden danger.

    Thanks,

    I had the "reset multiblock" command in the initialization code at begin. These are only 3 instructions more than yet. But this command caused some trouble with one of my SDHC cards (this card will not boot with it), so I removed it. Perhaps I made an error with the implementation or some cards really don't like it. The sdspi_q2 driver in the ZIP has the command still in the code but commented out:
    ...
             mov ctr2,#9
    reslp   mov cmdo,#0
            mov cmdp,#0
            call #cmd
             cmp accb,#1  wz
      if_z   jmp #tsthc
    '         test accb,#4 wz
    '         mov cmdo,#12          'reset multiblock
    '  if_nz  call #cmd
             djnz ctr2,#reslp
    ...
    

    But once again: Chip (with the help of Kye) has to do the final code, and they are free to choose the scheme they find the best. We have no mandate to make a code, we just can make proposals.
    If it is done with a separate partition or with a pointer to a file in the MBR is not so important, I just not want to hear that it is not possible or that it does not fit in the ROM.

    Andy
  • evanhevanh Posts: 15,126
    edited 2012-08-19 18:22
    Ariba wrote: »
    If it is done with a separate partition or with a pointer to a file in the MBR is not so important, I just not want to hear that it is not possible or that it does not fit in the ROM.

    Pointer points to raw blocks in custom partition too.

    The way I see it is we are being asked to come up with a good working solution here. The target ROM size is no more that 165 longs, preferably less. We should be aiming for as final as possible for one simple reason, it will help.
  • Cluso99Cluso99 Posts: 18,066
    edited 2012-08-19 19:54
    Proposal for testing....

    Andy's code looks nice and is similar to the fsrw that I have been using. I would like to use Andy's code as the base.

    I would like to fork it to a version that just initialises the card...
    1. Checks SDSC - thats the first SD basic version <=2GB cards
    2. Checks SDHC - this is 2GB -- <= 32GB
    3. Unsure if we should check for SDXC - 32GB -- <= ?TB
    4. Display the commands sent to the card and the responses returned including if it is SDSC/SDHC/SDXC (remember, we are not doing any validation, just the minimal required to ensure we can read the card in raw mode, fixed 512byte sectors. This is at least common to both SDSC & SDHC)
    5. Read the MBR record and display it (see my last posting for spin code that displays this)

    List the card that was tested, and any failures found, and the PST print (can be copied wth cut/paste).
    I am also interested to see if there is any code in the MBR record, specifically at offset $100, $180, $190.

    I will start a new thread requesting help testing all SD cards. Due to lack of time, would anyone like to start this (Andy, since it's your code?, David?)
  • Cluso99Cluso99 Posts: 18,066
    edited 2012-08-19 23:24
    Found some time this afternoon...

    see this thread http://forums.parallax.com/showthread.php?141952-SD-Card-Test-(for-testing-boot-code-for-the-Prop2)&p=1119986#post1119986

    I have posted a modification to Andy's (Ariba) code that records the commands sent to the SD card and displays these and the MBR sector. I welcome you all to try your various SD cards and report any problems and outputs to the abovementioned thread.

    If the P2 ROM supports at least SDHC cards (>2GB.....<=32GB) we would have good coverage. I am hoping that we can also support SDXC but I don't have any to test. My 1GB & 2GB SD cards also work fine.
  • evanhevanh Posts: 15,126
    edited 2012-08-20 06:28
    Hmm, I should buy some Prop gear and do more than cheer from the sidelines.
  • KyeKye Posts: 2,200
    edited 2012-08-20 08:00
    SDXC cards do not use the FAT file system. They work the same as SDHC cards otherwise. exFAT support must be coded for the propeller. The problem is, there is no nice ***FREE*** Microsoft white paper for exFAT.

    SD card support can be added for the P2. It can fit in the ROM. I've just always been concerned about how well it can be done. If SD card support works good as any standard $10 dollar SD card reader that you can buy for a computer then I would be very happy.
  • evanhevanh Posts: 15,126
    edited 2012-08-20 08:19
    Kye wrote: »
    The problem is, there is no nice ***FREE*** Microsoft white paper for exFAT.

    That's crazy, there will be so many devices around now that will simply puke on exFAT. Oh well, just reformat them to FAT32 I guess.

    Good to here the controller side doesn't need any changes though.
  • dMajodMajo Posts: 855
    edited 2012-08-20 09:24
    Cluso99 wrote: »
    SECURITY
    I was thinking that it might be a possible weakness to allow downloading ability permitting brute force attacks.
    I wonder if it might be prudent (since we have a number of user fuses) to identify 3 fuses, one each for Serial, Flash and SD. If the individual fuse was blown then that boot ability would be blocked. i.e. if the Serial fuse bit was blown, then serial downloading would not be possible. If all 3 fuses were blown then perhaps the Flash only should be allowed.
    Any thoughts???

    Agreed for the security foses except that if all 3 fuse are blown I will default to standard boot sequence (serial, SD, flash). I think that when only the flash is allowed only the program itself can blow the last fuse and if done there is aprogrammed reason for that.
    evanh wrote: »
    Given how small you are talking - 1024 bytes, I'm thinking just boot the MBR instead. Let the code in the MBR do the dirty work, that's what it's purpose is. Obviously still isn't enough for filesystem support but would be enough to continue on as you've just outlined.

    If you reserve a partition for the firmware you can boot all the hub image, 1k or 128k there is no difference when it's only raw sequential read.
    Ariba wrote: »
    f you have enough code space, all the various suggested scanning off the SD boot sector parameters are doable.
    But it has to fit in about 160 longs, and you all seem to have no idea of how tight this is. The needed code for initialization of the card, the SPI routines and read sector code together takes 80% of the available memory. Only very few instructions left to check all the possible formating variations. (FAT16, FAT32 with/without MBR, cluster size, number of FATs on so on).

    So either you dictate how exactly the SD card must be formated, or you implement a strategy to find the boot file without all the FAT-boot-parameter stuff.
    Another simple way would be to provide a disk image that contains the boot file. The utility to write such images exist on all platforms. This concept is also used by the Raspberry PI for example. The nice thing: You know exactly the structure of the sectors, and all you have to do is to check a signature at a certain sector and boot from this sector if the signature is valid.

    Andy

    Edit: What really counts for me is that SD booting is doable in the available ROM code space. How exactly it will be implemented depends on how many instructions you have left to find the file.

    Any partition area scanning for filenames for known signatures seems dangerous, better to raw boot from a reserved partition. Read 0x01C0 (partition type) and verify for the correct value (I suggest for one of these 0x27, 0x78 or 07F ), if correct read from 0x01BD to 0x01BF, threat them as pointer to the firmware first sector and start sequential reading. This assumes the first partition is reserved for firmware.

    Kye wrote: »
    If Chip decides to make the SD card boot loader he will just make it so that it looks for a special partition in the partition table that has the entire image in it. This is the only method that is feasable and that will work.

    Agree 100%
    Kye wrote: »
    I had a long 3 hour conversation with Chip about all the things needed to support SD/MMC cards.....

    Unfortunately, not all SD cards have an MBR. Such cards will not be supported as they are no longer manufactured. All SD cards from the SD2.0 standard released many years ago must have an MBR. This allows support for every SD card sold since SDHC cards came out.
    ....

    The MBR is used when you have more than one partition. This do not exclude that you can have the MBR also with only one. Perhaps also the SDs using directly the VBR can be reformatted so that the MBR can be written in.



    Past reference
    http://forums.parallax.com/showthread.php?141706-Propeller-II&p=1118079&viewfull=1#post1118079
    http://forums.parallax.com/showthread.php?141952-SD-Card-Test-(Please-test-may-be-used-for-boot-code-for-the-Prop2)&p=1120057&viewfull=1#post1120057
  • potatoheadpotatohead Posts: 10,253
    edited 2012-08-20 09:27
    http://www.munted.org.uk/programming/exfat.tar.bz2

    That's a read only implementation.

    This disturbs me too: https://www.sdcard.org/downloads/formatter_3/

    "Please use this formatter for all SD...."

    Microsoft recently saw it's FAT patent undermined, if not invalidated by some people doing research to find the technique for long names used on the Atari ST computers.

    Re: Gear

    You have no Propeller!! Boy, are you missing out! :) If it were me, I would fix that ASAP.
  • dMajodMajo Posts: 855
    edited 2012-08-20 09:33
    evanh wrote: »
    The answer to that question may be, how many devices boot from SD card?

    I haven't tried but I think that every PC supporting USB boot (I am used to boot from USB keys) will boot from the SD if insserted in one these small SD (uSD) to USB adapters
  • evanhevanh Posts: 15,126
    edited 2012-08-20 09:36
    dMajo:
    It might be best if you start at the end and read back. We've gone over so many of these questions too many times already.
  • dMajodMajo Posts: 855
    edited 2012-08-20 09:44
    evanh wrote: »
    Hurm?! I'm no M$ fan but FAT32 has a huge amount of life in it for simplistic file sharing. You don't have complex structures nor user privileges to deal with. Perfect for removable storage.

    If you like, the starting block number can still be written into the MBR, just make that number point into the start of the FAT32 partition, say block 2. The ROM code doesn't need to even know there is a partition and at the same time the boot code is in standard location for a FAT partition. Load up block 2 and 3 to give the 1kB of initial boot code and from there it can carry on loading the remaining 13.5kB.

    Get the best of both then.

    EDIT: The one down side of modifying the MBR like this is some OS tools have a tendency to completely rebuild the MBR to how that one company thinks it should be. Wiping out what was there.

    That's why the MBR should exist and you should use the proper place: the partition table. If you use eg the first partition (reserved, not the FAT32 one) starting adress you have properly used an existing MBR parameter. No todays OS will alter this information because all of them support each other (ex linux, MS WIN and OS/2) on separated partitions.
  • dMajodMajo Posts: 855
    edited 2012-08-20 10:14
    jmg wrote: »
    If the boot file(s) are of a defined size, do you need to reformat every time ?
    Or can one not rely on an overwrite doing exactly that ? - Not even as part of the Prop Tools flow ?

    Usually the new one is written in what is assumed being free space and then the fat entry is updated to point to the new starting point, the old one became new free space. Thats one of the reasons for disk fragmentation.
  • dMajodMajo Posts: 855
    edited 2012-08-20 10:29
    evanh wrote: »
    dMajo:
    It might be best if you start at the end and read back. We've gone over so many of these questions too many times already.

    Sorry .... I will
  • cgraceycgracey Posts: 14,133
    edited 2012-08-20 14:48
    Heater. wrote: »
    SRLM,

    Traditionally RAM and ROM were different devices on a memory bus. One used the low order address bits were connected directly to the devices to select the required memory location. High order address bit were decode to produce chip select signals that would enable either the RAM or the ROM or whatever other devices were on the bus.

    It is that decoding of the high order address bits into separate chip elect signals that introduces extra delays into the memory access.

    This argument probably applies equally as well when the RAM, ROM and other devices are integrated onto the same chip. The same logic has to exist somewhere all be it much faster now as it is smaller and "closer to the action".

    However it seems that in the Prop HUB, the memory cells are all in one address space except that each cell can be configured as read/write or just read only. The configuration being done by some "simple" mask process. In this way the decoding logic described above is not required and no extra delays are introduced.

    Heater is right. There is only one type of hub memory on the chip. Inside, 6-transistor SRAM bit cells can be substituted for single NMOS transistors, making them ROM bits. So, it's an SRAM that we decided to make some bits permanent in. This saves a lot of space over having separate ROM instances which must be multiplexed.

    Sorry I've been absent since Thursday night from the discussion. We had an unexpected death occur in our extended family, so we've been out of town. On top of that, I got really sick from working too much. I'm getting better, but still pretty weak. I see this thread has 10 more pages than when I left off. I'll catch up.
  • cgraceycgracey Posts: 14,133
    edited 2012-08-20 14:59
    Ariba wrote: »
    f you have enough code space, all the various suggested scanning off the SD boot sector parameters are doable.
    But it has to fit in about 160 longs, and you all seem to have no idea of how tight this is. The needed code for initialization of the card, the SPI routines and read sector code together takes 80% of the available memory. Only very few instructions left to check all the possible formating variations. (FAT16, FAT32 with/without MBR, cluster size, number of FATs on so on).

    So either you dictate how exactly the SD card must be formated, or you implement a strategy to find the boot file without all the FAT-boot-parameter stuff.
    Another simple way would be to provide a disk image that contains the boot file. The utility to write such images exist on all platforms. This concept is also used by the Raspberry PI for example. The nice thing: You know exactly the structure of the sectors, and all you have to do is to check a signature at a certain sector and boot from this sector if the signature is valid.

    Andy

    Edit: What really counts for me is that SD booting is doable in the available ROM code space. How exactly it will be implemented depends on how many instructions you have left to find the file.

    Guys, we can make the ROM a little bigger. I've relocated it to the bottom of memory, though the first 16 byte locations ($00000..$0000F) are going to be SRAM. I think they are useful for storing some system-wide status data, like freq, clkmode, etc. ROM will start at $00010 and cog0 will boot from there on power-up. ROM doesn't absolutely have to end at $007FF. It could go to $008FF, or something like that. So, while a tight implementation of SD boot is needed, it doesn't need to fit in 160 longs.
  • cgraceycgracey Posts: 14,133
    edited 2012-08-20 15:04
    These 10 pages of posts are taking me a long time to plow through. Would it be okay if I just asked, at this point, whether a consensus has been reached on the best approach for SD boot?
  • KyeKye Posts: 2,200
    edited 2012-08-20 15:24
    No consensus has been reached. I still think looking for a partition to boot from in the MBR is the best idea.

    Well... I think their is a consensus that booting from the SD card should be available. I've been convinced that it can be done. I'm still concerned on how well it will work. I still remain disturbed by not being able to have universal compatibility for all MMC and SD cards in my driver after spending so much time trying to implement everything properly.

    If you buy the spec from the SD association then the boot loader cannot be open source. So, it needs to be implemented with what we already have.

    Anyway,
  • pedwardpedward Posts: 1,642
    edited 2012-08-20 15:46
    LOL, that was the only plausible solution I saw when talking to Chip about it a few months ago. I'm not too hot on the idea simply because it presents a solution that requires doing non-standard things or repartitioning the Sd card. I feel that a half baked solution that robs more ROM is worse than not having it at all.

    I mean no disrespect to anyone, I'm thinking from a customer-of-the-customer view. If I develop a product with the P2, I want it to be super simple and error-free for customers to execute field upgrades or card swaps. If you design your product to boot from SD, you effectively have to be a single source for the card because the process is too complicated and sensitive to allow the customer to do it.

    I don't want to supply SD cards to customers, I want to tell them to go to Best Buy and get a 4GB card. I also want the customer to be able to do firmware upgrades easily, or change the personality of a device by putting a different file on a card.

    The process has to be idiot proofed to be useful, otherwise you must use the SPI flash method anyway. What's the point of a solution that is so restrictive and special case that you cannot use it practically for the reason you invented it?

    Put simply, if it's more hassle and work for me to boot from SD than to put an SPI flash chip on the board, why should I use SD?

    You're talking about 50 cents, if the customer picks up the phone or sends you an email, you lose the value of that 50 cent savings.

    I recommend sticking with the smallest ROM possible and making the SD booter be a small chain loader that runs from SPI flash. This is essentially how GRUB works.
  • groggorygroggory Posts: 205
    edited 2012-08-20 15:55
    pedward wrote: »
    I recommend sticking with the smallest ROM possible and making the SD booter be a small chain loader that runs from SPI flash. This is essentially how GRUB works.

    This is my vote
  • Invent-O-DocInvent-O-Doc Posts: 768
    edited 2012-08-20 15:57
    I second pedward. At first, i did not understand that the ROM bytes are taking from RAM space, which is precious. Although the technical challenge can be solved, having an EEPROM or FLASH device, which is cheap allows the prop2 to maintain its elegance and symmetrical nature. We can always preload the flash with an SD loader and if standards change, or a technique isn't compatible enough it can be upgraded. ROM should be for super rock solid stuff only, in my opinion.
  • KyeKye Posts: 2,200
    edited 2012-08-20 16:06
    @pedward - I agree with you. I've just given up arguing. I want Chip to work on whatever will get the P2 out the door fastest. Which in my opinion is just go get SPI flash booting working and then go from there.

    But, Chip thinks we can get SD booting in there. So, I will provide help.
  • potatoheadpotatohead Posts: 10,253
    edited 2012-08-20 16:09
    I don't see SD as necessary. It is a nice to have.

    If we DO SD, I second Kye's recommendation that we look for a prop boot partition and load from there.

    Secondly, I too favor loading more robust code to idiot proof SD, which means not doing the ROM for SD boot. We can get proper file system support that way, and work to handle changes in the cards and how they will be formatted.

    @Chip: Man, that is always hard. Please accept my best thoughts during this difficult time. :(

    Edit: Don't boot SD. This phone will erase a lot of text! Sorry for being unclear.
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2012-08-20 16:18
    I've been avoiding this thread, hoping that common sense would prevail in the end. Although not a consensus without Chip's concurrence, I'm encouraged by the the latest comments of pedward, Kye, Invent-O-Doc, groggory, potatohead, et al. The SD thing is a rather bulky can of worms that's best left out of ROM to become a secondary boot method. But I guess it was necessary to wrestle it to the ground anyway before shooting it. Chip, here's the gun. Do what you have to do. :)

    Thanks,
    -Phil
  • jmgjmg Posts: 15,140
    edited 2012-08-20 17:12
    pedward wrote: »
    ...I'm thinking from a customer-of-the-customer view. If I develop a product with the P2, I want it to be super simple and error-free for customers to execute field upgrades or card swaps. If you design your product to boot from SD, you effectively have to be a single source for the card because the process is too complicated and sensitive to allow the customer to do it.

    This is a valid comment, but since you have a Flash in this customer-of-the-customer system anyway, the ROM alternative does not exclude any of your features. They are not mutually exclusive.

    It really comes down to the actual real ROM size impact, and there may be rounding-up issues in ROM size.
    Chip has not said how granular this is, if single RAM bytes can be ROM'd or if it needs to be RAM ROWs ?
  • RaymanRayman Posts: 13,797
    edited 2012-08-20 17:23
    I would still vote for being able to boot from SD card in block mode.
    That's the only way to update firmware without a PC connected...
  • cgraceycgracey Posts: 14,133
    edited 2012-08-20 17:24
    jmg wrote: »
    This is a valid comment, but since you have a Flash in this customer-of-the-customer system anyway, the ROM alternative does not exclude any of your features. They are not mutually exclusive.

    It really comes down to the actual real ROM size impact, and there may be rounding-up issues in ROM size.
    Chip has not said how granular this is, if single RAM bytes can be ROM'd or if it needs to be RAM ROWs ?

    The RAM/ROM granularity is bit-level. So, there are no rounding-up issues, at all.
  • cgraceycgracey Posts: 14,133
    edited 2012-08-20 17:27
    Rayman wrote: »
    I would still vote for being able to boot from SD card in block mode.
    That's the only way to update firmware without a PC connected...

    Whew.... Maybe so. Rayman has a point.

    Since anything less than a fully FAT-aware system is going to cut some corners, is there a totally crude approach we could take that WOULD involve specially prepping an SD card, but would get us around needing a PC for reprogramming the SPI flash?

    I was about to pull the trigger, Phil.
  • tonyp12tonyp12 Posts: 1,950
    edited 2012-08-20 17:36
    I vote SD boot, unlike SPi flash you can remove it and get it ready on a PC.

    How are you gone get a SPI flash ready?, and still avoiding FTDI chips.
    As we can get a USB Serial going through software once PropII have booted up something.
  • pedwardpedward Posts: 1,642
    edited 2012-08-20 17:42
    [QUOTE=cgracey;[URL="tel:1120196"]1120196[/URL]]Whew.... Maybe so. Rayman has a point.

    Since anything less than a fully FAT-aware system is going to cut some corners, is there a totally crude approach we could take that WOULD involve specially prepping an SD card, but would get us around needing a PC for reprogramming the SPI flash?

    I was about to pull the trigger, Phil.[/QUOTE]

    Yeah, write a bootloader that loads from SPI flash that will read and update firmware from SD card. Then it can do authentication to make certain the firmware will actually run on the chip. Potential for a DoS if it was to blindly load to flash.

    There is just too many checks and "do the right thing" code to fit in the ROM unless you grow it by 4-6KB. At this point the ROM isn't going to grow for free, so it's better to have a soft-loaded code that can do all of this fancy stuff.

    Like I said before, there's no point in having it if it doesn't work for 90% of the use cases. Right now it will only work for 10% of the use cases.
Sign In or Register to comment.