O Arduino, How Terrible Art Thou? (A Rant)

I recently had an application for which the ESP8266 wasn't adequate, so I had to look into its new big brother the ESP32. And the NodeMCU Lua firmware I've been using for the 8266 wasn't ready for the 32. (As I write this it's in beta, which for a commercial project is just as bad.) So after reviewing my options I installed the Arduino IDE and the ESP32 plugin. I have little use for Arduino class devices when the Propeller exists, so this was my first rodeo with the whole ecosystem.

And I will say one thing the Arduino environment does well (in fact the only thing it seems to do well) is install itself on a variety of boxes without a lot of drama. The Windows self-installer worked fine, and I needed a video tutorial but it successfully guided me through the install of the ESP32 add-on. Within an hour I was blinken de light on a NodeMCU ESP32s.

Things went rapidly downhill from there.

My first task was a relatively simple relay that needed to accept a HTTP connection, relay the GET request to a serially connected server, and relay the reply to the original socket before closing it. This should have been straightforward and I did in fact get the basic functionality working after about a day of fighting the development system. There wasn't any example code that quite did what I needed, but I've been programming for almost 40 years and I was able to pick out the bits and pieces I needed.

Problem was, it took nearly six seconds to relay a 6 kilobyte webpage. Okay serial is slow but it's not that slow. I was able to prove with serial debug outputs of the ESP cycle count that the delays were in the TCP stack. The example code I'd followed worked one character at a time, calling a send function that accepted a uint-8 character argument. Obviously there was some overhead.

Now C++ has this terrible, horrible stupid feature called "overloading" which allows you to have multiple functions with the same name that are differentiated by the arguments they are declared to receive. The only purpose of this seems to be to maximize the confusion in debugging other peoples' code, which becomes effectively unreadable since it's unclear what any of the nonstandard (or even really standard) functions are doing. Now you'd expect that if there are four versions of a function which take different arguments and handle them differently, that somewhere there would be a list of them explaining their strengths, weaknesses, and limitations. In Arduinoland? Nope. Not for the core functions (which typically only example the simplest form of each function) and not for the extensions (which sometimes don't provide examples at all). Not being a C++ head it took me nearly half a day to figure out just how to use the "IPaddr" data type which actually comes from the ExpressIF ESP32 SDK and convert it to and from strings. There are of course simple functions for that, but there is no documentation of them and rooting in the source code first requires you to FIND the source code, which in my case required using a tool called Agent Ransack to find the ESP32 libs about nine folders deep beneath documents/arduino, and then comparing IPaddr function declares many of which were overloaded same-name functions doing quite different things with similar code from libraries I did have examples for and could understand.

And OK, I realize I'm dipping my toe in a really unfamiliar pool, but what mystifies me is that this is considered an appropriate environment, even what one might call the current default or index environment, for teaching programming to n00bs. If I'm using the PropTool and I want to know how the library works, all the "objects" are in the left hand panel and if I click on one there's the source, usually with a nice comment block at the top describing how to use it. In Arduinoland? Nope. Even if you find the source there are no comments, because why bother? Who would ever find it anyway in C:\Users\MyName\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\libraries\WiFi\src ?

So I suspected my problem with the byte-at-a-time driver might be solved if I collected all my data in a buffer (ESP32 has plenty of RAM for what I'm doing) and executed a single call to blow it out to the stack. This required me to get into the weeds of how you pass a pointer to a C byte array slash string to a function, which is another bit of non-obvious arcana. I really don't need the efficiency of pointer manipulation for this but it turns out via the fourth or fifth tutorial I scrounge up via the GOOG that C array names ARE pointers, and you can even set a non permanent pointer equal to one at which point it becomes a manipulable pointer, and walla. And that's basically the core concept in C for how you turn a byte array into what looks like a string for a function call. Stuff like this should be on page 1, fellas.

And yep, collecting my junk into an array and blasting it to the ESP32 TCP stack with one call solved the overhead problem which should have had a 24-point warning attached to it in the nonexistent documentation.

Now I've been programming as I said for nearly 40 years, in far more primitive and far more obtuse languages, and I can adapt. I've adapted before. But that requires me to know what I need to adapt to, and the Arduino environment seems almost defiantly arranged to prevent anyone from learning anything. Yes, you can get it set up and blink an LED and run the demo app you downloaded, but as soon as you want to do something even a little different from what the example shows, you are well and truly screwed.

So having made the serial relay work I was asked to do something else, which required me to run a simple non-relay webserver and a UDP server at the same time. The only UDP examples I could find used something called asyncUDP, obviously a fancy library with extra functions, but when I pasted that code into my app along with the webserver it crashed whenever it received a UDP packet. I got rid of the async web server and manually accepted the TCP connection and replied to it, but UDP still crashed. I knew the basic WiFi object had UDP support but with little experience it was not clear to me just how to convert the C++ object-oriented function definitions in the source to calls that could be made in C-like code. Finally, after hours of searching, I found one example which like about 3/4 of them I had encountered was on a discussion board under the question "Here is my code that doesn't work, what is wrong with it?" Fortunately what was wrong with it wasn't the UDP calls and I was able to get my own stuff to work using those as a template for what I was doing.

So after weeks of fighting with this, I ask again: What sadist thought this was a good way to teach programming? I now know why the coworker who does hobby robotics with Arduino doesn't know how 2's complement math works or how to convert a string into a numeric value to do math on it. He couldn't figure out how to unpack a 2-digit BCD value from a clock chip into binary to do math on it. Because really, he's been using this tool for several years but if you can't just copy boilerplate code and use it as found, it kicks your butt and is completely discouraging even if you know your way around a computer in both directions at high and low speed. It was easier hand compiling 6502 opcodes into hexadecimal and hand keying them into a Prolog ROM programmer than this crap.

And that's not even counting the hilariously bad user interface. (Orange on black status for downloads and errors? Really? and Sending 0%..4%..(long wait)..Done!) And we're supposed to do something more like this than Spin and the PropTool because... ???

Comments

  • Mike GreenMike Green Posts: 23,007
    edited 2020-05-29 - 01:11:52
    On the other hand, if you want to do something with a new device that's pretty much what's already working in some sample code ... Arduino and C++ or Python may get you going more quickly than the current Propeller tools, particularly on a Mac or Linux box. I've had that experience more than once with fancy sensors from SparkFun or Adafruit and I started coding around the mid-1960s.

    I'm a big fan of the Propeller design ... particularly the P1. It'll take me a while to get used to the P2. I tend to favor plain C and Spin. They're both "close to the machine" conceptually which I like. (Remember PL360? A sort-of Algol-like assembly language for the IBM 360 architecture. No GOTOs ... well, only simple ones)
  • And @localroger and @Mike Green I agree.

    In fact @localroger that's why I rarely do anything with the batch of those dratted things I have here.
  • @localroger
    Agreed. No-one knows the basics and they just google a piece of code and hope it works!

    @Mike Green
    You started programming a little earlier than my early 70's. It was no problem to enter a complete small program in machine code and run it on the ICL mini. I taught many field engineers how to do this, as they needed to be able to do this in case of a hardware failure. It became second nature to me and I can still code in it if I wanted. We are now so far removed from the hardware we use!
  • I had a similar experience with a DHT22 sensor. At first, the only example code I could find was for Arduino and I couldn't find any meat under the hood to talk to this sensor. Nowadays I write code purely in Assembly language, but if I find a decent example in another language I can usually decipher it into Assembly code. The datasheets for this sensor are terrible unless you can read Chinese. All of the code I did find for this sensor assumes that you have nothing going on in the background and nothing better to do with your CPU time which is not the case for me with this current project. It's not that this sensor is slow, but there is a 18ms negotiation that starts out slow at the beginning before it decides to spit out 40 bits of data. Each data bit starts with a LOW pulse 50us wide followed by a HIGH pulse that can vary between 30us and 70us. A logic "1" is a 30us pulse while a logic "0" is a 70us pulse. I ended up creating an interrupt on change (IOC) on the signal pin from a HIGH to LOW transition and setting a timer so that it would overflow after 100us but NOT generate an interrupt upon it's overflow. When the IOC generates an interrupt it looks at the overflow bit from the timer to receive and concatenate the bit stream into 5 data BYTES before resetting the timer for the next bit.

    The sensor lives in a security kiosk (attached image) on the big red PCB (13x11 inches) in the middle of the unit.

    777 x 1600 - 189K
  • Beau,
    Yup, the DHT22 sensor with it's demented protocol is a tricky thing to get working.
    I also tried various library programs which did not work (and whose coding was impossible to figure out) so I eventually coded my own program directly from the datasheet.
  • localroger wrote: »
    Now C++ has this terrible, horrible stupid feature called "overloading" which allows you to have multiple functions with the same name that are differentiated by the arguments they are declared to receive. The only purpose of this seems to be to maximize the confusion in debugging other peoples' code, which becomes effectively unreadable since it's unclear what any of the nonstandard (or even really standard) functions are doing.

    I think your problem here is more the bare-bones Arduino IDE than the overloading. In a proper C++ IDE, you can easily ask it what overload is being used (usually by hovering over it or some context menu option) and get autocomplete to tell you what types a function can accept.
  • localroger,

    I have never been a fan of C but Arduino examples are usually the first things to pop up and the examples you see can be anywhere from crude to elegant.

    I remember someone asking about using an Adafruit LCD backup so I looked over the schematic and the LiquidCrystal library that is often cited.
    The library is somewhat spaghetti code because it jumps around depending on which interface you are using and except for some of the C++ is fairly to follow.
    IIRC, I think it was SPI interface I found strange because usually you put the 4 data bits into a nibble but it didn't work like that.

    The Arduino is nice if you want instant gratification and many Objects have been ported or converted from Arduino code/libraries.
  • If you just need to get the job done and get paid (ESP32 and 8266):

    https://sites.google.com/site/annexwifi/downloads

    If you want native code but a civilized programming syntax then look at B4R from anywheresoftware

    :smile:
  • Look at MicroPython. It is an interpreter firmware that takes python code almost verbatim. The only differences are that you must import things like usocket instead of socket and some of the u versions of the libraries have different commands. If you want an http or https request, you would use the urequests module (I think). Overall, an amazing system!
  • MicroPython is available for the P2, but the thread about it has gone dormant for some months. I don't think anybody is interested in it right now.
  • localroger wrote: »
    Now C++ has this terrible, horrible stupid feature called "overloading".... Now you'd expect that if there are four versions of a function which take different arguments and handle them differently...
    @localroger,
    I agree with most of what you've said here. But overloading, when properly implemented, can greatly simplify code.
    For example, if I want to print out a line to the standard output - like println() - I want to be able to print just a blank line (no parameters) or to print an integer, or a string, etc. - anything that supports conversion to text. Clean, simple, easy to understand, and I don't need different function names.

    That said...
    Your point, of course, is that in your case the same function handled the arguments differently. Which should not happen. These should have been distinct functions, properly named, rather than overloads of the same function.
    Unfortunately, this seems to be the direction that software is heading, aka open source.
    Which may be fine for learning and non-critical software.
    But open source's biggest problem is that submitted code isn't always consistent or validated. And once it's out there, it's essentially cast in stone.
  • Not sure i would agree with crediting Open Source for that. Yes, the range of skill and style will always remain "wide" in open source, but we have a choice to use or not use open source parts. With closed source, the skill and style range while hopefully not as wide it's still choice of two, use it or don't while not knowing whether the underlying code was quality or floor sweepings pushed under the rug to meet the "work it may, ship it must" paradigm. (d*mn, always wanted to use that word). Open source you see what you are getting.
  • TorTor Posts: 1,999
    edited 2020-06-04 - 12:30:08
    It's absurd to drag Open Source into this. If you ever looked at closed source you'd find much more of, let's say, not so particularly forward-thinking ways of implementing things. Way, way more. It's just that you don't get to see it, and the software isn't nearly as much eye-balled as the source everybody can see.
  • Tor is right, the sins of closed-source software can make the sins of open-source software look like trivial peccadilloes by comparison. I work with a device that is equipped with a Lua scripting engine. The software style guide we were sent advised many bad practices, which are followed consistently by the factory engineers, and which make their code unreadable, unmaintainable, and unusable. I basically created wrappers for all their crappy API functions and wrote my own version of the scripted part of the firmware.

    How bad? Well they advised to use very long camelCase names for variables and functions to avoid naming collisions. This is not just a matter of poor style, it is objectively wrong because Lua is a loosely typed all-lowercase language which does not enforce variable declaration, so if you create a variable currentCpuMHz and you misname it CurrentCpuMhz, Lua will happily create a new variable with the lookalike name instead of warning you about it. Worse, Lua tables allow you to create namespaces which are error-checked for their existence because you can't index a nil (nonexistent) variable as if it's a table, so addressing current.cpu.mhz as current.cppu.mhz will generate an error. Their code is full of crappy decisions like this which totally ignore Lua's table structure and were obviously inherited from their C++ style guide, and whoever made all these decisions obviously had enough influence that nobody dared contradict them. In open source, someone would have objected loudly.
  • Disclaimer: I write closed software for a living, so yes, I'm biased.
    Let me repeat my point: "Open source's biggest problem is that submitted code isn't always consistent or validated."
    I worked on a project for implementing scheduled tasks in Windows.
    Not one of the available open source products at that time worked properly for even the most basic types of scheduling.
  • wmosscrop wrote: »
    Disclaimer: I write closed software for a living, so yes, I'm biased.
    Let me repeat my point: "Open source's biggest problem is that submitted code isn't always consistent or validated."
    I worked on a project for implementing scheduled tasks in Windows.
    Not one of the available open source products at that time worked properly for even the most basic types of scheduling.
    Doesn't Windows have scheduled tasks built-in? Or was this really long ago on Win9x or smth?

    Anyways, I think that's just selection bias. OSS projects are more often public before they're actually finished/stable, whereas commercial closed source software is generally only released when it's done.
  • Well I also did my time in programming closed source and especially in larger companies is a quite strict code review and always some coding style reference on has religiously to follow.

    If you are lucky there is just one of them, but often different parts of the companies use different coding style manuals.

    Thankfully I am out of this rat race, but commercial source code is usually better styled/commented, more reviewed and with less personal preferences as Open Source code. The main reason is the Company reuses the source while hiring and firing programmer, while in OS a lot of the time the project is managed and steered by just a handful programmers, part time and well some move to other projects of interest and the source code sort of - hmm - degresses? mutate? devolves? IDK, but there it comes from.

    In the propeller world things where easy with PropTool, Librarys and Example folder. The OBEX was easy to use and thanks to PropTools Archive feature one has one file to publish/download containing everything one needed and is automatically time stamped.

    The Arduino World is huge and wide spread with hundreds of different sources for libraries and sources, but inconsistent

    So what can we learn from this?

    Thee classic PARALLAX way was (maybe still is) to have one set of Libraries provided by them and ONE place to go to publish something or look for published content. There is work in progress to move OBEX to GIT, hopefully this will get a bit refined, currently it needs someone at PARALLAX to manage pull requests.

    I think PARALLAX should not use a GIT-HUB account, but run a GIT-HUB server (alike the OBEX) where developer register their projects (alike in OBEX), can update their projects by them self without PARALLAX (alike the OBEX) and everybody could use GIT if he wants to or just download single Projects/Driver/whatever without the need to clone the whole PARALLAX repository.

    Mike
  • I second the git server idea.
  • yetiyeti Posts: 723
    edited 2020-06-05 - 19:32:03
    I second the git server idea.
    Get a (v)server, install Gitea and do it!
  • No @yeti,

    as of reasons stated above the COMPANY PARALLAX has to do this not some unknown SW developer out of Clearlake Oaks, who decides that a vegetable garden or a fish pond is more interesting then MC from said company and, just leaves.

    And for that reason PARALLAX would need to install Gitea and manage it if needed.

    But that would maybe have less administrative need as someone executing all the pull requests at PARALLAX.

    that is, at least my thinking behind it.

    Mike
  • yetiyeti Posts: 723
    edited 2020-06-06 - 06:38:49
    We need redundancy. Only relying on Parallax is as healthy as a RAID0 driveset in your server.

    Other topics (AVR, BSD, CP/M, ..., you name it, Z80) have an active distributed community generating some redundancy. Why don't we have this? E.g.: Then probably someone would still have the Gadget Gangster howtos, designs and schematics mirrored...

    If someone starts a GIT based community OBEX, we will find ways to mirror it.

    Currently my GIT(EA) knowledge does not include how to mirror stuff including(!) issues and pull requests and their discussions, but as migrating projects with all these components from GITHUB to GITLAB is possible (I was told so), it probably is doable with GITEA too.

    Fossil would fit this job even better but the world "speaks GIT" now and so this P2P VCS even is widely unknown. Yes... I too decided to put my time into GIT instead... :-(

    But this probably will end like the other cries before...
  • In case you didn't think Arduino was bad enough already, read The Untold History of Arduino: https://arduinohistory.github.io/
  • @Electrodude,

    wow, this is very interesting to read.

    Thanks,

    Mike
  • JonnyMacJonnyMac Posts: 6,737
    edited 2020-06-14 - 00:56:58
    Abstracting the microcontroller pins as numbers was, without a doubt, a major decision, possible because the syntax was defined prior to implementation in any hardware platform.
    PBASIC did that in 1993 with the BS1....

    It is interesting that he tried to create the first version of Wiring with the Javelin Stamp.
  • JonnyMac wrote: »
    Abstracting the microcontroller pins as numbers was, without a doubt, a major decision, possible because the syntax was defined prior to implementation in any hardware platform.
    PBASIC did that in 1993 with the BS1....

    It is interesting that he tried to create the first version of Wiring with the Javelin Stamp.

    It is isn't.
  • TorTor Posts: 1,999
    edited 2020-06-18 - 11:46:12
    The Javelin Stamp "lost" at becoming Wiring which became the Arduiono because the toolchain was (Parallax) proprietary. The PIC too, for the same reason (2003). There's a lesson there..
  • Tor wrote: »
    The Javelin Stamp "lost" at becoming Wiring which became the Arduiono because the toolchain was (Parallax) proprietary. The PIC too, for the same reason (2003). There's a lesson there..

    There _is_ a lesson there, but damned if I can figure out what. Give it away and you get tons of good will but don't earn a wooden Kroner. Keep it proprietary and you make money for a while till something better and cheaper comes along and then you're last week's leftovers.
  • They made the hardware open source from the beginning. That coincided with the ability of China to clone it and provide cheaper versions on Amazon and ebay.
Sign In or Register to comment.