Shop Learn P1 Docs P2 Docs
Prop2 ROM code - Page 10 — Parallax Forums

Prop2 ROM code

17810121319

Comments

  • David Betz wrote: »
    Seairth wrote: »
    cgracey wrote: »
    ...And, I'm really glad you guys are concerned about this. Being able to boot from an SD card would be really clean. Hello, stand-alone bench computer!

    Couldn't that also be done with a two-stage boot? I'm not necessarily opposed to baking in SD support (I've stayed out of the debate), but is it really necessary? The very fact that agreement hasn't been reached means to me that there is more than one way customers will desire to use an SD card. Why pigeonhole everyone into just one way do SD boot?
    Some people want to avoid having to add a SPI flash part to their BOM.

    I figured that was the case. And I reject that argument. Except for the few that can use the prescribed SD boot method, all alternatives will still require the external flash chip. If it turns out that the SD boot is flawed (which is increasingly likely if the method is complicated), we will still require the external flash chip.
  • Rayman wrote: »
    I think any ideas about using MBRs or partitions will be a real turn off to most people.

    Most people will just want to copy the boot file from PC to SD and not have to think too much about it...
    That will be helped by a utility that can be automatically invoked by SimpleIDE as someone has already suggested. I don't think it is a good idea to do the wrong thing just to satisfy a few lazy people.

  • RaymanRayman Posts: 12,914
    It's not just about laziness.
    If you do something clever, like MBR or partition tricks, you then will have to tell people what an MBR is or what a partition is. It'd get ugly and boring real fast...
  • jmgjmg Posts: 14,979
    dMajo wrote: »
    BTW IMHO I will use also the first 512 bytes of the flash in the same way. Having a MBR also there. This will not change anything compared to like is now, but will add the opportunity to partition th flash and use a file system on it.
    and the same decision-making procedure:
    That sounds a good idea, and maybe a nice way to shrink the code size, which Chip seems to have issues with.

  • jmgjmg Posts: 14,979
    edited 2016-09-06 23:35
    Rayman wrote: »
    I think any ideas about using MBRs or partitions will be a real turn off to most people.
    Most people will just want to copy the boot file from PC to SD and not have to think too much about it...

    For some that is true, which is why I suggested support both choices.

    Rayman wrote: »
    It's not just about laziness.
    If you do something clever, like MBR or partition tricks, you then will have to tell people what an MBR is or what a partition is. It'd get ugly and boring real fast...
    I agree, the 'support and explain' side matters, especially in casual user areas like classrooms.

  • jmg wrote: »
    Rayman wrote: »
    I think any ideas about using MBRs or partitions will be a real turn off to most people.
    Most people will just want to copy the boot file from PC to SD and not have to think too much about it...

    For some that is true, which is why I suggested support both choices.

    Rayman wrote: »
    It's not just about laziness.
    If you do something clever, like MBR or partition tricks, you then will have to tell people what an MBR is or what a partition is. It'd get ugly and boring real fast...
    I agree, the 'support and explain' side matters, especially in casual user areas like classrooms.
    Do what you want. It seems like dumbing things down and having keyword searching mechanisms to find boot files is just going to be a recipe for trouble. Following established standards seems like a far better idea.

  • RaymanRayman Posts: 12,914
    Well, I don't think it's horrible to require implementations to require an SPI flash chip and leave SD loading to second or third stages.

    There's just something clean about the idea of there just being a uSD card there that does everything...
  • jmgjmg Posts: 14,979
    edited 2016-09-07 01:00
    David Betz wrote: »
    Do what you want. It seems like dumbing things down and having keyword searching mechanisms to find boot files is just going to be a recipe for trouble. Following established standards seems like a far better idea.

    I'm lost, what part of my suggestion of 'support both' was unclear to you ?
    There is no dumbing things down, when both are supported - quite the opposite.

    Rayman wrote: »
    Well, I don't think it's horrible to require implementations to require an SPI flash chip and leave SD loading to second or third stages.

    As another reference point, the cost of SPI flash is now down to 12~15c/1k for 4Mb
    - in the P2 or SD price-scape, that is quite a small percentage cost.

    Rayman wrote: »
    There's just something clean about the idea of there just being a uSD card there that does everything...
    True.

    I guess there could even be use cases where there is a mounted Flash, and then a probe/edge connected SD is used on the production line, to solve this question of 'how to first flash the flash' ::
    tonyp12 wrote: »
    a SD card boot option does help with the chicken-egg problem, as how are going to get the spi-flash programmed if you don't have usb boot strap?
    using a uart-dongle?, that is so 1990's

    This SD card could also have various self-test codes, not needed in the field Flash.
    Blank flash would skip onto SD, and valid but outdated flash would need a means to force SD ?

  • tonyp12tonyp12 Posts: 1,950
    edited 2016-09-07 00:41
    a SD card boot option does help with the chicken-egg problem, as how are going to get the spi-flash programmed if you don't have usb boot strap?
    using a uart-dongle?, that is so 1990's
    I vote FAT-read_only-main_folder_only, even if strictly FAT16 2GB limit if that can get it done in 256 longs.




  • tonyp12 wrote: »
    a SD card boot option does help with the chicken-egg problem, as how are going to get the spi-flash programmed if you don't have usb boot strap?
    using a uart-dongle?, that is so 1990's
    I vote FAT, even if strictly FAT16 2GB limit if that can get it done in 256 longs.



    Please, no FAT16. If we have to be locked into FAT make it FAT32 and support SDHC and SDXC cards.

  • David Betz wrote: »
    jmg wrote: »
    Rayman wrote: »
    I think any ideas about using MBRs or partitions will be a real turn off to most people.
    Most people will just want to copy the boot file from PC to SD and not have to think too much about it...

    For some that is true, which is why I suggested support both choices.

    Rayman wrote: »
    It's not just about laziness.
    If you do something clever, like MBR or partition tricks, you then will have to tell people what an MBR is or what a partition is. It'd get ugly and boring real fast...
    I agree, the 'support and explain' side matters, especially in casual user areas like classrooms.
    Do what you want. It seems like dumbing things down and having keyword searching mechanisms to find boot files is just going to be a recipe for trouble. Following established standards seems like a far better idea.

    Following established standards?!?!?! Poking special values into the MBR is an established standard? How??? Seriously? NO! You know what is an established standard? Reading the FAT file system to find a file. Has been for decades.
  • Roy Eltham wrote: »
    David Betz wrote: »
    jmg wrote: »
    Rayman wrote: »
    I think any ideas about using MBRs or partitions will be a real turn off to most people.
    Most people will just want to copy the boot file from PC to SD and not have to think too much about it...

    For some that is true, which is why I suggested support both choices.

    Rayman wrote: »
    It's not just about laziness.
    If you do something clever, like MBR or partition tricks, you then will have to tell people what an MBR is or what a partition is. It'd get ugly and boring real fast...
    I agree, the 'support and explain' side matters, especially in casual user areas like classrooms.
    Do what you want. It seems like dumbing things down and having keyword searching mechanisms to find boot files is just going to be a recipe for trouble. Following established standards seems like a far better idea.

    Following established standards?!?!?! Poking special values into the MBR is an established standard? How??? Seriously? NO! You know what is an established standard? Reading the FAT file system to find a file. Has been for decades.
    Isn't the partition table there to define different partitions on the media? How is adding a boot partition "poking special values into the MBR"?

  • jmgjmg Posts: 14,979
    Roy Eltham wrote: »
    Poking special values into the MBR is an established standard? How??? Seriously? NO!
    Well, yes, it is already called a Master Boot Record, because that is an established standard use.
    Roy Eltham wrote: »
    You know what is an established standard? Reading the FAT file system to find a file. Has been for decades.

    I think there are actually two standards, - Consumer Standards and Manufacturer/OEM Standards.

    Consumer and casual users would use FAT, but Manufacturer/OEMs have no problems with their established standard of Boot records.

    It seems to me it may be possible to cover both 'standards' ?

  • Roy ElthamRoy Eltham Posts: 2,995
    edited 2016-09-07 01:52
    jmg, the suggestion is to poke a value into the MBR's 4th partition table entry, overwriting the LBA, that points to a file in another partition. That is not part of any standard.

    Other suggestions are to write the value into some other part of the MBR, also not standard.

    Manufacturers that use SD card standards (from the SD card org) use FAT.
  • RaymanRayman Posts: 12,914
    Well, the target audience is a key factor here. Maybe the most important factor.
  • Rayman wrote: »
    Well, the target audience is a key factor here. Maybe the most important factor.
    And the target audience for the P2 is hobbyists?
  • Such strong and casual opinions....

    For production programming of a serial Flash I would probably allow for those SPI pins to be on my programming header along with the serial coms much as I do now with the P1. All it takes is a little dongle that gets plugged in for a second and the code is loaded. An SD only solution would normally be copied onto a preformatted FAT32 card it's true but might never be used for FAT32 otherwise as FAT32 is proprietary anyway plus it wasn't designed for Flash systems and wear-leveling. It is a certain matter for an SD card to be read and updated infrequently and quite a different matter for a log file to continually grow and wear out the Flash because it keeps writing to the fixed directory entry.

    @Chip - is it possible to release an image with the boot rom without SD boot so that it will be easier to play with approaches and derive a solution SOON? Then you compare which solution is best.
  • RaymanRayman Posts: 12,914
    If Chip would say: give me FAT file boot in 256 longs and I'll put it in ROM, I might be motivated to make that happen...
  • Along with releasing the ROM, it might also make sense to release a version that simulates as if some fuses have been set for an encrypted boot. Further, if there's an ability to do a chained multi-stage boot, provide a simple example?
  • An SD only solution would normally be copied onto a preformatted FAT32 card it's true but might never be used for FAT32 otherwise as FAT32 is proprietary anyway plus it wasn't designed for Flash systems and wear-leveling.
    I thought that the SD card firmware had wear-leveling built in.

  • Just a couple of points. (And really, this is mostly rock throwing and not constructive. I just feel it must be aired.)

    1) IIRC, FAT is patent encumbered.

    2) The proper way to perform booting is the way you boot your PC, using the MBR. Anything else is a hack, even if it works.

    3) The tool used to develop for the P2 (SimpleIDE,PropTool,whatever) can be responsible for writing the boot image and updating the MBR for the SD card. This hides all of the details from "dumb users" who just want to boot from SD. If there are compatibility issues caused by how the SD card is formatted, the tool can be updated to solve these problems.

    4) If you want, you can write your own 2nd stage bootloader that can load and execute an image off of a FAT partition.*
    *As noted in previous posts, this may not be necessary, as the tool could ensure that the boot image is written as contiguous file.

    5) Everyone can have everything they want. A boot image in the FAT filesystem, a simple boot process, an SD card that plays nice with other systems, a boot process that is transparent to users. All without building a full FAT engine in the ROM.

    Again, this is mostly rock throwing. Call me out where I am wrong. I'm not going to post about this again, as I really don't have anything new to contribute.

    Thanks for everyone giving their input, as it means a lot of us care about how the P2 turns out.

  • jmgjmg Posts: 14,979
    Rayman wrote: »
    If Chip would say: give me FAT file boot in 256 longs and I'll put it in ROM, I might be motivated to make that happen...
    I'm still unclear on how the Boot process 'overflows' above the 1 COG limit.
    Can it reload other code into the boot-COG, or does the 16k Byte ROM image load once only, into COG+HUB memory, which would mean Hubexec would be used above 2K ?

  • The FAT filesystem is not patent encumbered. Nor is is proprietary.

    Long file names stored in FAT directories the way that windows 95 and newer do it is what is patented. ALso, they are patents from the mid 90s, so they will expire soon.

    If you want to "properly boot with the MBR" that would mean putting boot code into the MBRs boot code areas and running that to then read the rest of the boot code in files in the FAT file system. Of course, things are a lot more complex these days...

    Poking an address value into the partition table is not proper or standard or safe for all possible devices you plug the SD card into.
  • Roy Eltham wrote: »
    Poking an address value into the partition table is not proper or standard or safe for all possible devices you plug the SD card into.
    I don't think that is what is being suggested. I think the idea is to create a valid partition that contains the boot image. The partition entry would point to that image just like it points to a range of sectors formated as a FAT filesystem.

  • RaymanRayman Posts: 12,914
    I think Microsoft tried to patent FAT, but got overturned.

    They may own patent for their version of long file names.
    Their version is quiet simple and easy to implement, if not for ownership issues.
  • AribaAriba Posts: 2,599
    edited 2016-09-07 03:49
    I agree with Roy Eltham, that the usual way should be to use the FAT to load a bootfile with a specific name. That's the simplest for the user, and it's the standard filesystem for SD cards.

    But as a fallback there should also be a test for a certain partition in the MBR partition table. There are partition types defined that will not be observed by Windows or Linux or other systems, only the Prop2 booter needs to know about it.

    If such a partition entry has no effect for other systems you can also use it just as a pointer to a file instead of a separate partition. This is more dangerous because the file can get deleted, and the pointer is still there, but it seems to be the prefered methode for Cluso or Peter.

    So if you want it safe: make a hidden partition for your bootcode - the partition tool will set the table entry, if you want the pointer methode: copy the file and use a tool to set the pointer inside the partition table in MBR, if you want it easy: copy your file with the right filename and let the booter find it per FAT.

    What some here seem to forget: It must be easy for the enduser of a Prop2 product, not for us developers. This ordinary user has no PropPlug or a installed PropellerTool and is normally not willing to download SD card tools to make a damaged SD card bootable again.

    Andy


  • Cluso99Cluso99 Posts: 18,063
    Today, microcontrollers that require external Flash to boot are quite antiquated, and could be seen as yet another reason to skip the P2, and for that matter the P1.

    It is proposed to locate the 16 bytes in the MBR as the last 16 bytes in the "Boot Code" section of the MBR. Unless there is physical boot code located there, and the current info leads us to believe this is no longer used for anything these days, then it is free for us to use.

    This comes before the 4 Partition Tables, so nothing is being interfered with here. The last 2 bytes are a signature which is checked by a lot of software. Again, we are not touching that.

    Now, if your card is formatted as FAT16/32, then you can copy your boot file into the first partition as a normal file.

    If you want to create another Partition for your boot file, you can do this too.

    If you don't want to have any preformatting, you can do that also, although you will likely need special software like dd or similar.

    If a new format type becomes available (like an open source FAT64), that will work too, because the MBR is a backward compatible requirement that is quite unlikely to be superceeded.

    Now, the only bit of software required is to locate your "Boot File" sector address and length (in sectors), and write that info plus the "identifier" into the specific part of the MBR. This can be a simple P2 program downloaded and run, or a simple UNIX/dos/Windows program that can locate the file and write the MBR.

    I have P1 running code but I am stripping out SD V1 and MMC support as IMHO these are basically obsolete.
  • Peter JakackiPeter Jakacki Posts: 10,193
    edited 2016-09-07 07:00
    Cluso99 wrote: »
    Today, microcontrollers that require external Flash to boot are quite antiquated, and could be seen as yet another reason to skip the P2, and for that matter the P1.

    It is proposed to locate the 16 bytes in the MBR as the last 16 bytes in the "Boot Code" section of the MBR. Unless there is physical boot code located there, and the current info leads us to believe this is no longer used for anything these days, then it is free for us to use.

    This comes before the 4 Partition Tables, so nothing is being interfered with here. The last 2 bytes are a signature which is checked by a lot of software. Again, we are not touching that.

    Now, if your card is formatted as FAT16/32, then you can copy your boot file into the first partition as a normal file.

    If you want to create another Partition for your boot file, you can do this too.

    If you don't want to have any preformatting, you can do that also, although you will likely need special software like dd or similar.

    If a new format type becomes available (like an open source FAT64), that will work too, because the MBR is a backward compatible requirement that is quite unlikely to be superceeded.

    Now, the only bit of software required is to locate your "Boot File" sector address and length (in sectors), and write that info plus the "identifier" into the specific part of the MBR. This can be a simple P2 program downloaded and run, or a simple UNIX/dos/Windows program that can locate the file and write the MBR.

    I have P1 running code but I am stripping out SD V1 and MMC support as IMHO these are basically obsolete.

    Now this is the crux of the matter, when the Propeller was launched FAT16 was common and now FAT32 is on the wane although still used for compatibility. None are Flash friendly in that they do not take wear-leveling into account and an SD can come to grief in a very short time from wear. A wear-leveled Flash memory can slow down and lose capacity as it wears at the very least.

    We must take the long term view and assuming for the present that SDHC will still be supported for many years as there is not much we can do about that but there is no need to be reliant upon an already fading FAT32. But sectors will be sectors, just as they were from the days of floppy disks and FAT12, and the MBR or sector 0 is now only used for partition table information as no system actually tries to boot from there anymore. Let's just use sectors, and as I mentioned previously a simple bit of FAT32 software downloaded into the P2 can locate a FAT32 boot image and write that information to sector 0 at a location that does not interfere with the partition table along with a boot signature. Now, how hard is that?
  • David Betz,
    Go back and reread what people have said. The suggestion was "copy the file normally to a valid partition" then run something to mod the MBR to point to that file. That is not standard in any way.
    If the suggestion is the make a new partition type that is "P2 boot image" and it just contains the image, then that at least fits the MBR standards, but it still requires some special tool or code to create the partition and put the image in it. That's a PITA.

    There is enough ROM space, just make it read FAT and be done with it. The code can read FAT16, FAT32, or exFAT (which is just FAT32 extended). That will cover the current and most recent SD standards.

    The only limitation is trivial, don't use long file names, so there is no issues with the patents.
  • dMajodMajo Posts: 851
    edited 2016-09-07 09:20
    Roy Eltham wrote: »
    jmg, the suggestion is to poke a value into the MBR's 4th partition table entry, overwriting the LBA, that points to a file in another partition. That is not part of any standard.

    Other suggestions are to write the value into some other part of the MBR, also not standard.

    Manufacturers that use SD card standards (from the SD card org) use FAT.

    No, the suggestion is to have dedicated partition for firmware images (eventually more than one).
    At this point the LBA address, which is the lower boundary of the partition, automatically becomes the firmware starting point (no need to poke anything and any standard tool will do the job).
    The read is performed as raw block access since the partition is not formatted (declared, see partition type value) with any file-system. It is only reserved space on the card, out of reach for any OS, a safe area for your firmware.
    Now if you make eg 100MB partition and define the max allowed fw size eg 2MB you can use partition entry's status information as fw image selector and calculate the fw starting block from: LBAAddress + Status*MaxSize.
    And this is standard, it complies with all rules, and no OS will mess with your "hidden" partition".

    But, if you prefer to keep your firmware as a regular file in other partition (eg n°1 under FAT, so that you can simply drag&drop it there from the PC) than you write its starting point in partition entry n°4 using LBA address. This is slightly out of standards: the only "improper" value in the whole partition entry is the LBA address in this case. But again no OS will mess with your "poked" value in the partition entry area if its partition type/status have right values.



    And with that said if you consider that in case of use of flash, it will certainly be bigger than hub ram because users will have there also other data like images, logos, lookup tables, relocatable code chunks ....
    In this case having the MBR also in the flash will:
    - yes, waste the first 512 bytes, the only drawback
    - but, beside low level access (commands), will make. logically, flash boot equal to SD boot
    - will allow for several firmware images for safe in field upgrades (in case eg of power outage during the update)
    - will also prevent wear issues of frequent updates (specially under development)
    - and last but not least, with a little more clever boot code, offer a sort of HA in the boot media. In this case the loader can switch from a corrupted image to a valid one, of course two copies of the same firmware needs to be written into flash/SD


    Edit:
    BTW: file-systems like user needs during the time changes. MBR and low-level, raw block access will always remain the same. And from code perspective a sequential block read is much easier to do reliably rather than to interpret and follow a file-system, which involves its awareness fixed in ROM and limits user's freedom of choice.
Sign In or Register to comment.