Programming the propeller via wireless, fast? What happened to this?
Oldbitcollector (Jeff)
Posts: 8,091
A while back there was an official looking project that appeared to have Jeff Martin's attention in which he demonstrated the programming of a Propeller using a wireless link.
I can't seem to lay my hands on this, but IIRC it was not quite ready for release and was to be released soon.
Did this ever see the final light of day and I miss the link?
Thanks
I can't seem to lay my hands on this, but IIRC it was not quite ready for release and was to be released soon.
Did this ever see the final light of day and I miss the link?
Thanks
Comments
http://forums.parallax.com/showthread.php/139534-Wireless-Programming-of-a-Propeller-chip-%28video%29
No, you didn't miss the link. We had a few projects in the works and other kinks that delayed pushing this forward in the time we originally planned. One of the issues was that we seriously needed to produce a board with the right connections on it to make this very easy to use. Another was that the existing firmware (at the time) on the XBee Wi-Fi had a couple bugs that we were working with Digi to get fixed (and they have).
We built a couple prototypes (the one seen in the video below and another that tests an auto-switching USB/Wi-Fi circuit we had devised) and tested those extensively. We are now about to send off the design for the revised Propeller Activity Board which will be the first one that directly supports it.
We need to make some loader modifications and absorb the Wi-Fi design into it to complete the picture. We'll have a couple iOS apps due to be released around mid-March that use it and we'll be actively adding support elsewhere too
Here's a link to the materials I created and used for R&D.
http://www.parallax.com/downloads/propeller-ip-loader
It describes the protocol that I created during the Wi-Fi downloader R&D I did (and is implemented in our Prop Loader iOS app).
Here's the repository link for the Prop Loader app that we used as part of the R&D on the project:
https://github.com/parallaxinc/Prop-Loader_iOS
Here's a little demo video that was derived from implementing all that:
https://plus.google.com/+JeffMartinOfParallax/posts/CiFSkZv3S67
Here's a short video quickly demonstrating an iOS app that Mike Westerfield is making. It currently looks somewhat like SimpleIDE (same icon) because that's the direction we gave him back then, but it may look different and be called something else when we release.
https://drive.google.com/file/d/0B0tev9Y3WaL4MmZFNUg1cHE4Z0E/view?usp=sharing
http://www.parallax.com/news/2015-01-26/wireless-programming-and-propeller-activity-board-wx-software-and-hardware
The iPad technical guts about programming via WiFi are portable to whatever system we want, as Jeff mentioned.
Be the first to post a comment on the page.
Ken Gracey
Ken Gracey
@Jeff: Thank you! That post will give me some information to digest!
It makes sense that you used the Xbee's in your example, you do sell them, but I'm hopeful to see this adapted to the other more generic wireless ttl serial devices as well.
In my first glance through your materials it looks promising.
It is faster than our current wired (like USB-based virtual COM port) implementations... about 6x faster in practice.
We'll be updating the wired implementation to use some of the same techniques too, so it will shore up the speed difference.
Everybody will need a Propeller Activity Board WX just to try to this out.
Ken Gracey
I have a couple of MR-703 that I have used over wifi with an Asus EeePC in Debian Linux to load Propeller binaries or as a Forth on Propeller wifi interface. This could also work easily in Windows 7 on the same platform.
I presume that any wifi platform that can transfer a text file may be used with these mini-routers. Once inside the router, a .spin file may be compiled to binary and loaded.
I'm not sure but I guess if we are talking ethernet we are taking IP and probably TCP/IP
In which case one can never be sure that the blobs that one sends as a "packet" are actually received as a the same blob. It could well be chopped into smaller IP packets or combined with others. Given that you always have to worry about network propagation and latency times.
Please don't use terms like "stupid" here. It's a very negative value judgement.
As far as I understand ethernet knows nothing of UDP or TCP or even IP. It's at a lower level of the protocol stack. After all, both UDP and TCP can be moved over good old fashioned serial links using SLIP or PPP.
I'm sure ethernet does not split packets, UDP or otherwise. But I'm not sure about TCP. TCP provides a byte stream, it makes no guarantees about which bytes you write end up in which packets. That's the whole reason to have TCP, to abstract all that packet stuff away. This has caught many a programmer who writes "write(something)" at one end but then is confused when "read(something)" at the other end does not come out the same.
As far as I know TCP is quite likely to have it's packets split and merged on the way from sender to receiver through various routers. The only guarantee is that the bytes come out in the right order at the far end.
Anyway, at the end of the day, if the the thing works on the local LAN that is great.
What do I need, then to program a Propeller over the internet ?
How are security issues handled?
Again, this whole wireless thing is primarily intended to support classroom use of a number of Propeller and Stamp boards. It is designed to use the existing ROM loader and a minimum of additional hardware. If you want higher security, you would need some kind of intermediate buffer that securely downloads a Propeller binary image (maybe via SFTP or a VPN), then loads that into attached Propeller boards. This would be an expensive but secure alternative.
My understanding of the hardware changes is that it puts the S6B effectively in parallel with the USB-serial adapter's serial port so that the Propeller's existing ROM boot code can load from either device. The first (UDP) block received contains the "handshake" data and the 1st stage loader. None of this has any "security issues" taken care of. If there's access to the S6B from the broader internet, someone could reset the Propeller and load any desired code into it. You have to use the LAN's firewall to prevent that.
Thanks David! After many weeks of struggling with this, the idea for a single first-packet micro boot loader finally popped into my head (woke me up in the middle of the night, if I recall). After that, it was many hours of squishing the handshake+connection response, creating a thin enough boot loader, and lot's of timing tests and pondering before that "dream" was finally realized.
As it turns out, the micro boot loader itself (second-stage loader) doesn't have enough elbow room in the packet to contain all the smarts to do everything, so I broke it down even further.
* The micro boot loader in the first packet knows how to receive further packets, provide an ack/nak response, and store the new data into RAM. The packets are about 1,400 bytes each.
* Then, once the target application is fully received, the host sends a couple more packets (as necessary) that contain executable micro-boot loader code... the loader knows to simply receive these and execute them as-is.
* The last few packets deliver the RAM verification code (that the micro boot loader uses for RAM checksum calculation and response), the EEPROM programmer, the EEPROM verification code, and finally the launcher code. The EEPROM parts I still need to implement... I have the code written, but got side tracked and haven't fully tested or released it yet.
Thanks for chiming in about the "stupid" comment, Mike! As it turns out, I wasn't offended by that comment; after reading it a second time, I understood Heater's point (and we know each other well enough), but agree that comments like that are not always understood the way they were intended, so please be careful. No harm here though.
Regarding splitting packets:
Routers have a maximum packet size expectation. The XBee actually has a limitation on packet size that is more constrained than typical routers these days. I wrote it to fit within the XBee's constraints, which is 1,400 if I recall. To my knowledge, and via all testing I've done, UDP and TCP packets never get split into even smaller packets upon transport unless they exceed the router's/switch's maximum packet size- and even if they did get split into smaller sizes (for network transport), it's the responsibility of a deeper network layer to reconstruct them as they were originally transmitted before delivering to the layer that interprets the payload.
So, with TCP's overhead, packets are "guaranteed" to be received, and they will be received out-of-order, but will be re-ordered before the application layer receives the data... so the stream should always be in-tact unless a fatal break in communication happens. If a programmer is receiving them out-of-order, I think they are working at a lower-level and not the API level (but I could be wrong). I've seen it happen while tracing network traffic, but never have received packet payloads out-of-order at the programming level I was working in, when using TCP.
Regardless, I found TCP to be way too much overhead; the delays between packets were so incredibly varied and extreme that some critical timing requirements (like the post-reset-delay between reset pulse generation and reception of the first packet of data) was impractical to predict across devices, even in a clean network.
I switched to UDP because, 1) delays between packets were much less variable, way more reliable, and much smaller, 2) I could create thinner (ie: "just right") overhead to handle packet loss, retransmission, and out-of-order reception that works way better than TCP (as far as speed) for our application, and 3) XBee's Application Service (which is separate from it's Serial Service) requires UDP anyway, and we use that to verify and change XBee settings to fit the needs of the download attempt. In addition, I used a couple hardware/software features of the XBee Wi-Fi to time the reset pulse reliably and the post-reset-delay as well.
Regarding "over the Internet":
This works well and fairly automatically on the same LAN segment. Getting it to work "over the Internet" is another deal... it's still possible (very reachable, in fact, from what I've seen) but due to factors outside our control, requires some deeper knowledge and control of the network involved. The packet source/destination IP's get NAT'd, which eliminates using broadcast to automatically "find" attached XBee Wi-Fi modules in other subnets, and also the XBee's Serial Service needs to know the destination address to use for data received from the Propeller in this case; that's information that the host can't easily determine for itself, it takes human knowledge and usually some port-forwarding rule setting on the routers involved.
Regarding Security:
Mike, you nailed it! Thanks. 'nuff said. :-)
My comment about programmers being surprised about what they get from TCP was not that they get data out of order but rather they a surprised to get more or less data than they expected on receive. For example:
The sender does:
write("This is my message")
write("This is my other message")
The receiver does:
read() - Gets: "This is my mes"
read() - Gets: "sageThis is my oth"
read() - Gets: "er message"
Then they wonder why their parser could not understand the incoming data.
Anyway. I agree. UDP is a good solution for this. For speed and simplicity.
As for "over the internet", I'll be leaving that to programming a Prop over a serial link from a 20 euro WIFI router that handles secure connections over SSH / HTTPS. I guess not everyone wants to do that.
Oh, I see what you mean now. Yes, with longer data than would fit into a packet, or with intermediate events that end up triggering the net stack's nagel timer, the logical "stream" the program is sending gets automatically broken up into smaller packets. That is indeed something that would surprise an unsuspecting programmer.
In this case, we're forming the packets in very determined ways so as to not see such things happen.
Regarding security concerns over the Internet: that sounds like a good solution. It also makes me think a secured point-to-point VPN circuit can be employed and then the same XBee Wi-Fi based solution can be used assuming the host system let's the user enter the required IP Address and port metrics to successfully perform the download. Many possible variations to achieve the same effects, which is great.
As for 'atomic' datagrams - one could also use Ethernet frames, but then it would only work over actual Ethernet. UDP frames seems a good solution. It certainly does wonders for latency, it's only because I use a UDP-based VPN solution that I can be in Japan and connect to computers in Norway as if it were a local connection (almost). TCP over UDP is actually vastly more usable than TCP over TCP, if there's any latency involved - TCP over TCP (as with tcp/ip-based VPN solutions) gets astronomical total latencies and is useless when you're on the other side of the globe. TCP over UDP creates nearly no overhead compared to direct TCP, and some times it's even faster - has to do with the routing or something I guess.
-Tor
https://www.kickstarter.com/projects/1133560316/black-swift-tiny-wireless-computer
Oh Smile. Thanks. Now I have to shell out on yet another Kickstarter project:)
Certainly Black Swift can be used to program Props wirelessly. It's a MIPS CPU running OpenWRT and we already have OpenSpin and the propeller loader running in that environment. I'm sure it can also be used to serve up our web based Spin IDE. No more installing of IDE applications like the Prop Tool or Propeller IDE, required. Just connect to it with a browser. All self contained.
Almost as cheap as a Prop Plug around here.