O Arduino, How Terrible Art Thou? (A Rant)
localroger
Posts: 3,452
I recently had an application for which the ESP8266 wasn't adequate, so I had to look into its new big brother the ESP32. And the NodeMCU Lua firmware I've been using for the 8266 wasn't ready for the 32. (As I write this it's in beta, which for a commercial project is just as bad.) So after reviewing my options I installed the Arduino IDE and the ESP32 plugin. I have little use for Arduino class devices when the Propeller exists, so this was my first rodeo with the whole ecosystem.
And I will say one thing the Arduino environment does well (in fact the only thing it seems to do well) is install itself on a variety of boxes without a lot of drama. The Windows self-installer worked fine, and I needed a video tutorial but it successfully guided me through the install of the ESP32 add-on. Within an hour I was blinken de light on a NodeMCU ESP32s.
Things went rapidly downhill from there.
My first task was a relatively simple relay that needed to accept a HTTP connection, relay the GET request to a serially connected server, and relay the reply to the original socket before closing it. This should have been straightforward and I did in fact get the basic functionality working after about a day of fighting the development system. There wasn't any example code that quite did what I needed, but I've been programming for almost 40 years and I was able to pick out the bits and pieces I needed.
Problem was, it took nearly six seconds to relay a 6 kilobyte webpage. Okay serial is slow but it's not that slow. I was able to prove with serial debug outputs of the ESP cycle count that the delays were in the TCP stack. The example code I'd followed worked one character at a time, calling a send function that accepted a uint-8 character argument. Obviously there was some overhead.
Now C++ has this terrible, horrible stupid feature called "overloading" which allows you to have multiple functions with the same name that are differentiated by the arguments they are declared to receive. The only purpose of this seems to be to maximize the confusion in debugging other peoples' code, which becomes effectively unreadable since it's unclear what any of the nonstandard (or even really standard) functions are doing. Now you'd expect that if there are four versions of a function which take different arguments and handle them differently, that somewhere there would be a list of them explaining their strengths, weaknesses, and limitations. In Arduinoland? Nope. Not for the core functions (which typically only example the simplest form of each function) and not for the extensions (which sometimes don't provide examples at all). Not being a C++ head it took me nearly half a day to figure out just how to use the "IPaddr" data type which actually comes from the ExpressIF ESP32 SDK and convert it to and from strings. There are of course simple functions for that, but there is no documentation of them and rooting in the source code first requires you to FIND the source code, which in my case required using a tool called Agent Ransack to find the ESP32 libs about nine folders deep beneath documents/arduino, and then comparing IPaddr function declares many of which were overloaded same-name functions doing quite different things with similar code from libraries I did have examples for and could understand.
And OK, I realize I'm dipping my toe in a really unfamiliar pool, but what mystifies me is that this is considered an appropriate environment, even what one might call the current default or index environment, for teaching programming to n00bs. If I'm using the PropTool and I want to know how the library works, all the "objects" are in the left hand panel and if I click on one there's the source, usually with a nice comment block at the top describing how to use it. In Arduinoland? Nope. Even if you find the source there are no comments, because why bother? Who would ever find it anyway in C:\Users\MyName\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\libraries\WiFi\src ?
So I suspected my problem with the byte-at-a-time driver might be solved if I collected all my data in a buffer (ESP32 has plenty of RAM for what I'm doing) and executed a single call to blow it out to the stack. This required me to get into the weeds of how you pass a pointer to a C byte array slash string to a function, which is another bit of non-obvious arcana. I really don't need the efficiency of pointer manipulation for this but it turns out via the fourth or fifth tutorial I scrounge up via the GOOG that C array names ARE pointers, and you can even set a non permanent pointer equal to one at which point it becomes a manipulable pointer, and walla. And that's basically the core concept in C for how you turn a byte array into what looks like a string for a function call. Stuff like this should be on page 1, fellas.
And yep, collecting my junk into an array and blasting it to the ESP32 TCP stack with one call solved the overhead problem which should have had a 24-point warning attached to it in the nonexistent documentation.
Now I've been programming as I said for nearly 40 years, in far more primitive and far more obtuse languages, and I can adapt. I've adapted before. But that requires me to know what I need to adapt to, and the Arduino environment seems almost defiantly arranged to prevent anyone from learning anything. Yes, you can get it set up and blink an LED and run the demo app you downloaded, but as soon as you want to do something even a little different from what the example shows, you are well and truly screwed.
So having made the serial relay work I was asked to do something else, which required me to run a simple non-relay webserver and a UDP server at the same time. The only UDP examples I could find used something called asyncUDP, obviously a fancy library with extra functions, but when I pasted that code into my app along with the webserver it crashed whenever it received a UDP packet. I got rid of the async web server and manually accepted the TCP connection and replied to it, but UDP still crashed. I knew the basic WiFi object had UDP support but with little experience it was not clear to me just how to convert the C++ object-oriented function definitions in the source to calls that could be made in C-like code. Finally, after hours of searching, I found one example which like about 3/4 of them I had encountered was on a discussion board under the question "Here is my code that doesn't work, what is wrong with it?" Fortunately what was wrong with it wasn't the UDP calls and I was able to get my own stuff to work using those as a template for what I was doing.
So after weeks of fighting with this, I ask again: What sadist thought this was a good way to teach programming? I now know why the coworker who does hobby robotics with Arduino doesn't know how 2's complement math works or how to convert a string into a numeric value to do math on it. He couldn't figure out how to unpack a 2-digit BCD value from a clock chip into binary to do math on it. Because really, he's been using this tool for several years but if you can't just copy boilerplate code and use it as found, it kicks your butt and is completely discouraging even if you know your way around a computer in both directions at high and low speed. It was easier hand compiling 6502 opcodes into hexadecimal and hand keying them into a Prolog ROM programmer than this Smile.
And that's not even counting the hilariously bad user interface. (Orange on black status for downloads and errors? Really? and Sending 0%..4%..(long wait)..Done!) And we're supposed to do something more like this than Spin and the PropTool because... ???
And I will say one thing the Arduino environment does well (in fact the only thing it seems to do well) is install itself on a variety of boxes without a lot of drama. The Windows self-installer worked fine, and I needed a video tutorial but it successfully guided me through the install of the ESP32 add-on. Within an hour I was blinken de light on a NodeMCU ESP32s.
Things went rapidly downhill from there.
My first task was a relatively simple relay that needed to accept a HTTP connection, relay the GET request to a serially connected server, and relay the reply to the original socket before closing it. This should have been straightforward and I did in fact get the basic functionality working after about a day of fighting the development system. There wasn't any example code that quite did what I needed, but I've been programming for almost 40 years and I was able to pick out the bits and pieces I needed.
Problem was, it took nearly six seconds to relay a 6 kilobyte webpage. Okay serial is slow but it's not that slow. I was able to prove with serial debug outputs of the ESP cycle count that the delays were in the TCP stack. The example code I'd followed worked one character at a time, calling a send function that accepted a uint-8 character argument. Obviously there was some overhead.
Now C++ has this terrible, horrible stupid feature called "overloading" which allows you to have multiple functions with the same name that are differentiated by the arguments they are declared to receive. The only purpose of this seems to be to maximize the confusion in debugging other peoples' code, which becomes effectively unreadable since it's unclear what any of the nonstandard (or even really standard) functions are doing. Now you'd expect that if there are four versions of a function which take different arguments and handle them differently, that somewhere there would be a list of them explaining their strengths, weaknesses, and limitations. In Arduinoland? Nope. Not for the core functions (which typically only example the simplest form of each function) and not for the extensions (which sometimes don't provide examples at all). Not being a C++ head it took me nearly half a day to figure out just how to use the "IPaddr" data type which actually comes from the ExpressIF ESP32 SDK and convert it to and from strings. There are of course simple functions for that, but there is no documentation of them and rooting in the source code first requires you to FIND the source code, which in my case required using a tool called Agent Ransack to find the ESP32 libs about nine folders deep beneath documents/arduino, and then comparing IPaddr function declares many of which were overloaded same-name functions doing quite different things with similar code from libraries I did have examples for and could understand.
And OK, I realize I'm dipping my toe in a really unfamiliar pool, but what mystifies me is that this is considered an appropriate environment, even what one might call the current default or index environment, for teaching programming to n00bs. If I'm using the PropTool and I want to know how the library works, all the "objects" are in the left hand panel and if I click on one there's the source, usually with a nice comment block at the top describing how to use it. In Arduinoland? Nope. Even if you find the source there are no comments, because why bother? Who would ever find it anyway in C:\Users\MyName\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\libraries\WiFi\src ?
So I suspected my problem with the byte-at-a-time driver might be solved if I collected all my data in a buffer (ESP32 has plenty of RAM for what I'm doing) and executed a single call to blow it out to the stack. This required me to get into the weeds of how you pass a pointer to a C byte array slash string to a function, which is another bit of non-obvious arcana. I really don't need the efficiency of pointer manipulation for this but it turns out via the fourth or fifth tutorial I scrounge up via the GOOG that C array names ARE pointers, and you can even set a non permanent pointer equal to one at which point it becomes a manipulable pointer, and walla. And that's basically the core concept in C for how you turn a byte array into what looks like a string for a function call. Stuff like this should be on page 1, fellas.
And yep, collecting my junk into an array and blasting it to the ESP32 TCP stack with one call solved the overhead problem which should have had a 24-point warning attached to it in the nonexistent documentation.
Now I've been programming as I said for nearly 40 years, in far more primitive and far more obtuse languages, and I can adapt. I've adapted before. But that requires me to know what I need to adapt to, and the Arduino environment seems almost defiantly arranged to prevent anyone from learning anything. Yes, you can get it set up and blink an LED and run the demo app you downloaded, but as soon as you want to do something even a little different from what the example shows, you are well and truly screwed.
So having made the serial relay work I was asked to do something else, which required me to run a simple non-relay webserver and a UDP server at the same time. The only UDP examples I could find used something called asyncUDP, obviously a fancy library with extra functions, but when I pasted that code into my app along with the webserver it crashed whenever it received a UDP packet. I got rid of the async web server and manually accepted the TCP connection and replied to it, but UDP still crashed. I knew the basic WiFi object had UDP support but with little experience it was not clear to me just how to convert the C++ object-oriented function definitions in the source to calls that could be made in C-like code. Finally, after hours of searching, I found one example which like about 3/4 of them I had encountered was on a discussion board under the question "Here is my code that doesn't work, what is wrong with it?" Fortunately what was wrong with it wasn't the UDP calls and I was able to get my own stuff to work using those as a template for what I was doing.
So after weeks of fighting with this, I ask again: What sadist thought this was a good way to teach programming? I now know why the coworker who does hobby robotics with Arduino doesn't know how 2's complement math works or how to convert a string into a numeric value to do math on it. He couldn't figure out how to unpack a 2-digit BCD value from a clock chip into binary to do math on it. Because really, he's been using this tool for several years but if you can't just copy boilerplate code and use it as found, it kicks your butt and is completely discouraging even if you know your way around a computer in both directions at high and low speed. It was easier hand compiling 6502 opcodes into hexadecimal and hand keying them into a Prolog ROM programmer than this Smile.
And that's not even counting the hilariously bad user interface. (Orange on black status for downloads and errors? Really? and Sending 0%..4%..(long wait)..Done!) And we're supposed to do something more like this than Spin and the PropTool because... ???
Comments
I'm a big fan of the Propeller design ... particularly the P1. It'll take me a while to get used to the P2. I tend to favor plain C and Spin. They're both "close to the machine" conceptually which I like. (Remember PL360? A sort-of Algol-like assembly language for the IBM 360 architecture. No GOTOs ... well, only simple ones)
In fact @localroger that's why I rarely do anything with the batch of those dratted things I have here.
Agreed. No-one knows the basics and they just google a piece of code and hope it works!
@"Mike Green"
You started programming a little earlier than my early 70's. It was no problem to enter a complete small program in machine code and run it on the ICL mini. I taught many field engineers how to do this, as they needed to be able to do this in case of a hardware failure. It became second nature to me and I can still code in it if I wanted. We are now so far removed from the hardware we use!
The sensor lives in a security kiosk (attached image) on the big red PCB (13x11 inches) in the middle of the unit.
Yup, the DHT22 sensor with it's demented protocol is a tricky thing to get working.
I also tried various library programs which did not work (and whose coding was impossible to figure out) so I eventually coded my own program directly from the datasheet.
I think your problem here is more the bare-bones Arduino IDE than the overloading. In a proper C++ IDE, you can easily ask it what overload is being used (usually by hovering over it or some context menu option) and get autocomplete to tell you what types a function can accept.
I have never been a fan of C but Arduino examples are usually the first things to pop up and the examples you see can be anywhere from crude to elegant.
I remember someone asking about using an Adafruit LCD backup so I looked over the schematic and the LiquidCrystal library that is often cited.
The library is somewhat spaghetti code because it jumps around depending on which interface you are using and except for some of the C++ is fairly to follow.
IIRC, I think it was SPI interface I found strange because usually you put the 4 data bits into a nibble but it didn't work like that.
The Arduino is nice if you want instant gratification and many Objects have been ported or converted from Arduino code/libraries.
https://sites.google.com/site/annexwifi/downloads
If you want native code but a civilized programming syntax then look at B4R from anywheresoftware
I agree with most of what you've said here. But overloading, when properly implemented, can greatly simplify code.
For example, if I want to print out a line to the standard output - like println() - I want to be able to print just a blank line (no parameters) or to print an integer, or a string, etc. - anything that supports conversion to text. Clean, simple, easy to understand, and I don't need different function names.
That said...
Your point, of course, is that in your case the same function handled the arguments differently. Which should not happen. These should have been distinct functions, properly named, rather than overloads of the same function.
Unfortunately, this seems to be the direction that software is heading, aka open source.
Which may be fine for learning and non-critical software.
But open source's biggest problem is that submitted code isn't always consistent or validated. And once it's out there, it's essentially cast in stone.
How bad? Well they advised to use very long camelCase names for variables and functions to avoid naming collisions. This is not just a matter of poor style, it is objectively wrong because Lua is a loosely typed all-lowercase language which does not enforce variable declaration, so if you create a variable currentCpuMHz and you misname it CurrentCpuMhz, Lua will happily create a new variable with the lookalike name instead of warning you about it. Worse, Lua tables allow you to create namespaces which are error-checked for their existence because you can't index a nil (nonexistent) variable as if it's a table, so addressing current.cpu.mhz as current.cppu.mhz will generate an error. Their code is full of crappy decisions like this which totally ignore Lua's table structure and were obviously inherited from their C++ style guide, and whoever made all these decisions obviously had enough influence that nobody dared contradict them. In open source, someone would have objected loudly.
Let me repeat my point: "Open source's biggest problem is that submitted code isn't always consistent or validated."
I worked on a project for implementing scheduled tasks in Windows.
Not one of the available open source products at that time worked properly for even the most basic types of scheduling.
Anyways, I think that's just selection bias. OSS projects are more often public before they're actually finished/stable, whereas commercial closed source software is generally only released when it's done.
If you are lucky there is just one of them, but often different parts of the companies use different coding style manuals.
Thankfully I am out of this rat race, but commercial source code is usually better styled/commented, more reviewed and with less personal preferences as Open Source code. The main reason is the Company reuses the source while hiring and firing programmer, while in OS a lot of the time the project is managed and steered by just a handful programmers, part time and well some move to other projects of interest and the source code sort of - hmm - degresses? mutate? devolves? IDK, but there it comes from.
In the propeller world things where easy with PropTool, Librarys and Example folder. The OBEX was easy to use and thanks to PropTools Archive feature one has one file to publish/download containing everything one needed and is automatically time stamped.
The Arduino World is huge and wide spread with hundreds of different sources for libraries and sources, but inconsistent
So what can we learn from this?
Thee classic PARALLAX way was (maybe still is) to have one set of Libraries provided by them and ONE place to go to publish something or look for published content. There is work in progress to move OBEX to GIT, hopefully this will get a bit refined, currently it needs someone at PARALLAX to manage pull requests.
I think PARALLAX should not use a GIT-HUB account, but run a GIT-HUB server (alike the OBEX) where developer register their projects (alike in OBEX), can update their projects by them self without PARALLAX (alike the OBEX) and everybody could use GIT if he wants to or just download single Projects/Driver/whatever without the need to clone the whole PARALLAX repository.
Mike
as of reasons stated above the COMPANY PARALLAX has to do this not some unknown SW developer out of Clearlake Oaks, who decides that a vegetable garden or a fish pond is more interesting then MC from said company and, just leaves.
And for that reason PARALLAX would need to install Gitea and manage it if needed.
But that would maybe have less administrative need as someone executing all the pull requests at PARALLAX.
that is, at least my thinking behind it.
Mike
Other topics (AVR, BSD, CP/M, ..., you name it, Z80) have an active distributed community generating some redundancy. Why don't we have this? E.g.: Then probably someone would still have the Gadget Gangster howtos, designs and schematics mirrored...
If someone starts a GIT based community OBEX, we will find ways to mirror it.
Currently my GIT(EA) knowledge does not include how to mirror stuff including(!) issues and pull requests and their discussions, but as migrating projects with all these components from GITHUB to GITLAB is possible (I was told so), it probably is doable with GITEA too.
Fossil would fit this job even better but the world "speaks GIT" now and so this P2P VCS even is widely unknown. Yes... I too decided to put my time into GIT instead... :-(
But this probably will end like the other cries before...
wow, this is very interesting to read.
Thanks,
Mike
It is interesting that he tried to create the first version of Wiring with the Javelin Stamp.
It is isn't.
There _is_ a lesson there, but damned if I can figure out what. Give it away and you get tons of good will but don't earn a wooden Kroner. Keep it proprietary and you make money for a while till something better and cheaper comes along and then you're last week's leftovers.