How to do an EEPROM update safely
ManAtWork
Posts: 2,178
Hi,
I have a propeller connected to an ethernet interface. I'm planning to provide a way for customers who don't own a Prop plug to update the software via network. Of course there's always some risk that the update fails (power failure or software crash while writing to the EEPROM) and the user is left with an incomplete EEPROM image which might result in a "dead" device since the bootloader is no longer working. This risk can be minimized to some degree with good error checking and by making the update as fast as possible. But it can't be totally avoided, I think, except by providing a "double buffer" backup with bank switching a double sized EEPROM.
A small risk is acceptable as long as it's guaranteed that in the case of a failure the system stays in a safe state, i.e. does nothing. The device controls a CNC machine and unwanted toggling of outputs could cause dangerous things to happen.
So my idea was to do it like this:
1. clear the first EEPROM page or somehow marking it as "bad" to prevent the propeller booting from a bad image.
2. update the rest of the EEPROM
3. write the final contents to the first page to make it valid again
So the question is: What should I write to the EEPROM in step #1 to prevent further program execution? Can I force the propeller to halt or execute an endless loop? I know that the first 16 bytes have a special meaning and there must be some documentation about it. But I can't find it at the moment.
I have a propeller connected to an ethernet interface. I'm planning to provide a way for customers who don't own a Prop plug to update the software via network. Of course there's always some risk that the update fails (power failure or software crash while writing to the EEPROM) and the user is left with an incomplete EEPROM image which might result in a "dead" device since the bootloader is no longer working. This risk can be minimized to some degree with good error checking and by making the update as fast as possible. But it can't be totally avoided, I think, except by providing a "double buffer" backup with bank switching a double sized EEPROM.
A small risk is acceptable as long as it's guaranteed that in the case of a failure the system stays in a safe state, i.e. does nothing. The device controls a CNC machine and unwanted toggling of outputs could cause dangerous things to happen.
So my idea was to do it like this:
1. clear the first EEPROM page or somehow marking it as "bad" to prevent the propeller booting from a bad image.
2. update the rest of the EEPROM
3. write the final contents to the first page to make it valid again
So the question is: What should I write to the EEPROM in step #1 to prevent further program execution? Can I force the propeller to halt or execute an endless loop? I know that the first 16 bytes have a special meaning and there must be some documentation about it. But I can't find it at the moment.
Comments
The propellent loader has three load options
1 Load RAM and run (EEPROM is untouched)
2 Load EEPROM and run
3 Load EEPROM and don't run
Jeff T.
you probably haven't undestood my question. The problem is not how to download the program to RAM or EEPROM but how to prevent booting an incomplete or damaged EEPROM image. BTW I can't and don't want to use propellent. I run a custom protocol over ethernet (physical layer, no IP) and no serial link (RS232 or USB). I have no problems writing to EEPROM but I can't load to RAM, at least not to all 32k of it because the network driver and protocol uses a fair amout of it that must not be overwritten while downloading.
A lot depends... but you might download the new firmware to the upper eeprom, run some checksums, then move to lower eeprom if ok.
OR
There might have a bootloader running in lower eeprom which enables the user to download new firmware, or else boots from upper eeprom. In the case of new download, read new firmware into ram 16K at a time (assuming your bootloader code uses less than half the size of available ram!) checksum each half, and write to upper eeprom. when bootloader finishes, it boots from upper, or if it fails then the device reboots and waits for the user to attempt the firmware update again.
a) A power failure while writing to EEPROM would still result in a corrupted image
b) my network driver code and buffer doesn't fit into one cog
I agree, this would be the optimum solution. If I sold the complete machines to the end customer I'd also agree theat $1 extra cost wouldn't matter much (32k EEPROM is ~$0.25, 64k is >$1.30) if it saves some trouble. But I fear that my customer who pays me only for the controller board is not willing to pay anything extra for a feature that is rarely needed.
I'm sure there is a solution that works as I said in the first post. I just have to mark the EEPROM invalid before the update and valid again after. This saves the majority of users $1 (or more profit to my customer) and will cause some extra cost and trouble to a minority who manages to kill their device by pulling the plug while updating... It's out of my scope to judge if this makes sense or not... The customer wants it that way.
Somebody mentioned there's a documented source of the propeller's internal bootloader somewhere. I think this would help me so I could figure out what the propeller does to decide if there's a valid EEPROM to boot from or not.
I can see you are not really interested in this solution because of the extra $, but just to answer your points anyway should it help for another project...
a- You will always face that problem; hence the bootloader is always cleaner- when copying the new image to upper eeprom, always write the critical byte last (whatever that is for you to make sure the program runs- maybe the first byte so upload the image 'backwards' ?). So if you do get a unbootable image it does not matter because the propeller will keep rebooting by itself and running the bootloader again until it finds a valid upper image. The bootloader can keep trying to contact the update server (or whatever the logic is for updates), until it manages to download a valid image.
b- That does not matter- I think our current bootloader uses 4 cogs. In our case we just needed a spare 16KB to temporarily store the image in 2 parts as it is downloaded from a webserver. So your bootloader can even use all 8 cogs- all you need is some spare memory to use as a buffer. Whatever the spare buffer size you have will be the download block size for the new firmware- even if that needs to be 8 or 4 KB blocks, or whatever you can spare.
Going back to your preferred single eeprom plan... It seems you already have the solution. Just invalidate the first byte of eeprom, then copy the downloaded new firmware to eeprom backwards (so 7fff to 0000). It means if the update fails, then the propeller will never boot the "bad" image because the first byte will not be valid (please check this, but this is from memory my understanding). In this case, the propeller will just keep rebooting itself (which might represent a "bricked" state), and so perhaps require someone to program the eeprom directly with the prop. switched off, or use propplug if the board has those connections available.
enjoy !
Ah, this is exactly what I was looking for. From the booter.spin file:
' Load ram from eeprom and launch ' boot mov smode,#0 'clear mode in case error call #ee_read 'send read command :loop call #ee_receive 'get eeprom byte wrbyte eedata,address 'write to ram add address,#1 'inc address djnz count,#:loop 'loop until done call #ee_stop 'end read (followed by launch) ' ' ' Launch program in ram ' launch rdword address,#$0004+2 'if pbase address invalid, shutdown [B] cmp address,#$0010 wz[/B] if_nz jmp #shutdownSo the word at address $0006 has to contain #$0010, otherwise the propeller will shutdown. So all I have to do is to start writing the EEPROM from address $0000 to $7FFF with the first 8 bytes erased to $FF. Then the address counter wraps around to $0000 again and I can re-write the final, valid contents to the first 8 bytes.
Thanks, Kuroneko, Maxwin and Johnny