Shop OBEX P1 Docs P2 Docs Learn Events
How to authenticate/validate file uploads to Prop/Spinneret? — Parallax Forums

How to authenticate/validate file uploads to Prop/Spinneret?

agsags Posts: 386
edited 2011-02-08 17:46 in Accessories
In the process of writing a bootloader (with help from folks here and many existing examples) I've seen some that will calculate an MD5 hash and display/return the value.

I'd like to (I think) create a more automatic process to validate that the file arrived intact, but don't know precisely how to do so following any existing protocols or standards. I don't have much experience with HTTP. I'd appreciate any insight that others could share on the topic. Note that I'm not (at this point) concerned with malicious tampering but am simply trying to validate an error-free file download (thus avoiding unnecessary trips to the PropPlug to un-brick the device).

At this point I'm going down the path of calculating the MD5 digest for the file to be downloaded on the client (push) side. I'd like to embed the MD5 digest in the same transaction (not sure what it's properly called) as the file transfer. Then the server (Prop/Spinneret) would store the file, calculating the MD5 value, and then compare the value it calculated with the value provided from the client. If they are equal, then I can be certain (enough) that the file received was the file sent (without transmission error).

Is there a standard way to do this, or at least a way that isn't in direct violation of protocol? Also, is there an easy way to "package" the MD5 value with the file and send it, without setting up an HTTP server on the PC side (for instance, using Telnet)?

Thanks in advance for any help.

Comments

  • David CarrierDavid Carrier Posts: 294
    edited 2011-02-07 16:52
    I don't know of any standards for transmitting a file and verifying its transmission. TCP protocol includes a checksum in each packet, and Propeller ".binary" and ".eeprom" files have their own checksum, which eliminates the need for additional checksums in most situations. Neither checksum is very secure; the TCP checksum is a 16-bit modulo of the entire packet, which is up to 1,500 bytes long and the Propeller checksum is an 8-bit modulo of the file, which may be up to 32,768 bytes long. This means that if you give a random number for a TCP checksum, it has a one in 65,536 chance of matching, and a random Propeller file checksum has a one in 256 chance of matching.

    If you are transmitting over a reliable connection, which would include most internet connections, Wi-Fi, and Zigbee, this should be enough. If you were transmitting over a wireless connection that did not have better packet authentication built into the protocol, then you would need an extra level of support.

    If you use a secure file transfer protocol, some of which are just standard file transfer protocols over a secure tunnel, then the security of the protocol will ensure that all packets are received unmodified. These protocols would include https, ftps, sftp, and scp. None of them are currently implemented on the Spinneret Web Server, and some of them require a significant amount of processing, which means they may take a noticeable amount of time to negotiate a connection, or the transfer rate may be somewhat slower.

    — David Carrier
    Parallax Inc.
  • Mike GMike G Posts: 2,702
    edited 2011-02-07 18:48
    Also, is there an easy way to "package" the MD5 value with the file and send it, without setting up an HTTP server on the PC side
    I found myself reading this page a lot. Might be of some help as there an MD5 header section with links to RFC 1864.
    http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html

    You'll need to send an HTTP request from the client.
  • agsags Posts: 386
    edited 2011-02-08 10:49
    Mike G wrote: »
    I found myself reading this page a lot. Might be of some help as there an MD5 header section with links to RFC 1864.
    http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html

    You'll need to send an HTTP request from the client.

    Yes, that's a great resource (along with the IANA site listing all registered media types (and more)) and I've added a bookmark to it for future reference.

    There is, indeed, a Content-MD5: <value> field supported. I can take care of capturing that and using it on the Spinneret side to check the downloaded binary. What I can't figure out is what tool I can use (other than writing it myself) to take a file as input, automatically calculate the base64 MD5 digest from that file, and then package both and send to the designated URL (in this case, the Spinneret). Any suggestions? I've looked at curl (http://curl.haxx.se) but no luck there. [of course, as a workaround I could calculate the MD5 digest, send that to the Spinneret in a short message and then follow with a separate message with the actual binary file. That will work, but it's a horrendous hack...]

    Thanks.
  • Mike GMike G Posts: 2,702
    edited 2011-02-08 17:46
    I don't know of any ready to go application. It should be short work to whip up a client in C#. You could literally grab 90% of the HTTP POST code from Microsoft. All you would have to do is serialize the file, calculate the hash (there's an class for that), add the MD5 header. I imagine the same goes for any high level language.
Sign In or Register to comment.