Propeller C : Tools and Status
chintal
Posts: 7
in Propeller 1
I've recently been trying to take a closer look at the propeller (1), which I noticed in passing about a half a decade ago, for a computationally intensive embedded application with no IO requirements at all. I've looked through much of what I could find and have some basic questions about the platform and whether it would make sense for me to use a propeller, opposed to a more traditional Linux capable device. To get the context out of the way, I have an easily parallelizable workload that involves multivariate optimization. This is not especially time critical and basically takes some numerical input and returns numerical output. I have my reasons for needing to do it in hardware, and let's leave it at that. The propeller has a nice number of high speed cores and a decent amount of memory (though I will probably need more, been looking at the SRAM application note), which is why I'm taking a very serious look at it. I like that it's available in DIP and more importantly QFP, which makes soldering possible for me. The prop plug is not particularly expensive and is, quite surprisingly, available in my country.
I do not with to offend or inflame anyone, or question the platform as a whole, but I do have some concerns as an outsider considering the platform for a relatively serious application. I might well be very wrong about some or all of the things listed here, but the whole point I'm trying to make (in addition to hopefully getting answers) is that it shouldn't be so hard to find out these things. Here are the issues I have which make me concerned about taking the plunge :
1. I'm not entirely sure what the future for the Propeller 1 is. It seems to have an almost cult-like popularity, and the longevity of the STAMP is comforting (though I never used it or know anything about it). However, information regarding the propeller seems strangely fragmented and confused. Parallax itself seems to center it's website on the Propeller as a learning platform, with information about the platform itself being tucked away in the corner. The information is there, which is nice, but it seems the parallax platform takes second seat to the learning and educational tutorials. This, coupled with with the Propeller 2 being on the horizon, makes me slightly doubtful about getting into a platform on the brink of obsoletion where the core drivers of the platform are literally academic exercises.
2. I'm going to be developing on linux. Windows tools would be nice to also have, but my first requirement is dev tools for linux. The spin programming language and tools confuse me. After all of my looking, (and yes, no actual installing or trying), I'm still not entirely sure whether spin tools support linux well. More importantly, I'm not sure I've seen a recent, authoritative source from parallax discussing the tool and in comparison with C. There were some (3-4 year old) mentions of the situation changing with propeller 2, suggesting a move away from spin. An indicator of platform stability to me is the availability of tools on linux distributions. Ubuntu (and probably Debian) don't seem to have any propeller or Parallax tools in the standard repository.
3. I'm not afraid of compiling tools myself, though I do tend to avoid it as far as I can. However, though there is a lot of talk about the C compiler for the propeller, I can't seem to find a good, authoritative source for which compiler this is. I came across a couple of gcc ports on github, which were last updated in 2015 or earlier. There is talk of various googlecode projects, which hasn't been a thing for a year or two now atleast. There is a SimpleIDE thing on parallax website which seems to partially or fully windows only. I'm not convinced that I know what the reliable, supported C / C++ compiler for propeller is. I'm also not sure I'm convinced the C compiler is ready for serious use, with mentions here and there about how spin tools produce better code with lower memory usage on the device because the gcc ports are not very optimized. In fairness, those claims are all pretty old, circa 2012 / 14, but at the same time the compilers I'm looking at seem just as old.
4. I do have big chunks of C code I would like to be able to reuse with the propeller. Would it be possible to mix C and spin code on the propeller, perhaps even at the level of running C code on some cores and spin code on the others? Are the spin to C converters and the C compilers reliable and optimized (3) enough to base applications off of, if I want to be able to use stuff from the OBEX?
5. Assuming I have a propeller and the prop plug, is that all I'd need to debug applications? There are mentions of gdb being used in the wild, but again from quite some time ago. I can't seem to find any specific mention of the debug hardware used in those cases (JTAG? Something proprietary? Does the prop plug support it by itself?). Is there a preferred, well supported open source linux host combination for compile (Preferably C, acceptably Spin), download / program, and debug?
For a silicon manufacturer that has a seemingly very powerful chip (on paper atleast) at a very reasonable price point, with fully open designs, which has existed for as long as it has, I find it quite strange that the answers to these questions are not readily available. Combined with the higher than usual noise level on seemingly 'official' websites, with the occasional dead links, links to dated posts, git repositories with no activity for years, and a home page that looks built to sell kits for high school and undergraduate courses has me concerned. Again, I apologize if any / all of the above are incorrect claims, and I'm sure many of them are wrong since parallax still exists in 2018. But having looked at the ecosystem for all of two days, which is as much as I really would be prepared to do in the general case, I'm left with more questions than I started with.
I do not with to offend or inflame anyone, or question the platform as a whole, but I do have some concerns as an outsider considering the platform for a relatively serious application. I might well be very wrong about some or all of the things listed here, but the whole point I'm trying to make (in addition to hopefully getting answers) is that it shouldn't be so hard to find out these things. Here are the issues I have which make me concerned about taking the plunge :
1. I'm not entirely sure what the future for the Propeller 1 is. It seems to have an almost cult-like popularity, and the longevity of the STAMP is comforting (though I never used it or know anything about it). However, information regarding the propeller seems strangely fragmented and confused. Parallax itself seems to center it's website on the Propeller as a learning platform, with information about the platform itself being tucked away in the corner. The information is there, which is nice, but it seems the parallax platform takes second seat to the learning and educational tutorials. This, coupled with with the Propeller 2 being on the horizon, makes me slightly doubtful about getting into a platform on the brink of obsoletion where the core drivers of the platform are literally academic exercises.
2. I'm going to be developing on linux. Windows tools would be nice to also have, but my first requirement is dev tools for linux. The spin programming language and tools confuse me. After all of my looking, (and yes, no actual installing or trying), I'm still not entirely sure whether spin tools support linux well. More importantly, I'm not sure I've seen a recent, authoritative source from parallax discussing the tool and in comparison with C. There were some (3-4 year old) mentions of the situation changing with propeller 2, suggesting a move away from spin. An indicator of platform stability to me is the availability of tools on linux distributions. Ubuntu (and probably Debian) don't seem to have any propeller or Parallax tools in the standard repository.
3. I'm not afraid of compiling tools myself, though I do tend to avoid it as far as I can. However, though there is a lot of talk about the C compiler for the propeller, I can't seem to find a good, authoritative source for which compiler this is. I came across a couple of gcc ports on github, which were last updated in 2015 or earlier. There is talk of various googlecode projects, which hasn't been a thing for a year or two now atleast. There is a SimpleIDE thing on parallax website which seems to partially or fully windows only. I'm not convinced that I know what the reliable, supported C / C++ compiler for propeller is. I'm also not sure I'm convinced the C compiler is ready for serious use, with mentions here and there about how spin tools produce better code with lower memory usage on the device because the gcc ports are not very optimized. In fairness, those claims are all pretty old, circa 2012 / 14, but at the same time the compilers I'm looking at seem just as old.
4. I do have big chunks of C code I would like to be able to reuse with the propeller. Would it be possible to mix C and spin code on the propeller, perhaps even at the level of running C code on some cores and spin code on the others? Are the spin to C converters and the C compilers reliable and optimized (3) enough to base applications off of, if I want to be able to use stuff from the OBEX?
5. Assuming I have a propeller and the prop plug, is that all I'd need to debug applications? There are mentions of gdb being used in the wild, but again from quite some time ago. I can't seem to find any specific mention of the debug hardware used in those cases (JTAG? Something proprietary? Does the prop plug support it by itself?). Is there a preferred, well supported open source linux host combination for compile (Preferably C, acceptably Spin), download / program, and debug?
For a silicon manufacturer that has a seemingly very powerful chip (on paper atleast) at a very reasonable price point, with fully open designs, which has existed for as long as it has, I find it quite strange that the answers to these questions are not readily available. Combined with the higher than usual noise level on seemingly 'official' websites, with the occasional dead links, links to dated posts, git repositories with no activity for years, and a home page that looks built to sell kits for high school and undergraduate courses has me concerned. Again, I apologize if any / all of the above are incorrect claims, and I'm sure many of them are wrong since parallax still exists in 2018. But having looked at the ecosystem for all of two days, which is as much as I really would be prepared to do in the general case, I'm left with more questions than I started with.
Comments
2. The official Spin tool for linux is called openspin. It works fine for me on all the distros I've tried. It's a command line only tool. There are also 3rd party Spin compilers: bstc, homespun, and fastspin. All have Linux ports. fastspin differs from the others in that it produces native code rather than the standard interpreted Spin bytecode.
There is a cross platform GUI called PropellerIDE that's fairly nice, but seems a bit unfinished (the last version had problems with openspin and only worked with bstc).
3. The "official" port of gcc to the propeller is gcc 4.6.1. There is also a beta port of gcc 6.0.0. Those are probably the two you found on github. Both work reasonably well, although the gcc 6.0.0 port is incompatible with the 4.6.1 port in some way that affects the Parallax educational libraries, and so it's never been officially adopted. Basically gcc 4.6.1 worked well enough for Parallax's purposes, so it's been frozen (which is why the repositories haven't been updated for a long time). The IDE for gcc 4.6.1 is called SimpleIDE. I only use the command line version myself.
4. Note that Spin is traditionally compiled to bytecode, and C to native code, so the interoperation does require some work. It's possible to mix Spin and C code, and there are a variety of ways to do this: spin2cpp converts Spin to C++, and spinwrap takes Spin code and wraps it up so it can be called from C. There's also a tool (I think it's called spinc) to convert C code to Spin code.
5. The gcc 6.0.0 binutils includes a gdb port. It's very much beta quality software -- it works, but has rough edges.
I think the bottom line is that Parallax is a small company with limited resources, so most of the knowledge and tools come from the community. This has pluses and minuses, but it does mean that (unfortunately) there's not really a centralized location for all of this. @DavidZemon has a private repository that does have builds of most of the C tools. He also has a very nice set of C++ libraries for the propeller.
gdb is available in PropGCC with GCC v4.6 as well, at least as its built TeamCity (don't know about the version shipped with SimpleIDE).
@chintal, I haven't had the opportunity to say this in quite some time, but I feel like I finally can: you're the person I developed PropWare for. I've had a terrible time supporting Windows users, but as a Linux user you're in luck. I provide a .deb installation file (and a .rpm) so you won't have trouble getting yourself setup. You'll need to download PropGCC separately, but that's easy enough if you follow this link. Extract to /opt/parallax if you want PropWare to just find it, or set PROPGCC_PREFIX environment variable if you want it somewhere else (or just ensure that propeller-elf-gcc is on PATH).
Okay, so what is PropWare? PropWare is three key pieces, none of which require the other two:
1. A CMake build system for building your Propeller applications. It's primary purpose is to invoke PropGCC for you, but there's also a hook to build via spin2cpp. PropWare's build system lets you write your code in whatever IDE you choose, such as JetBrains CLion (my personal choice, and where I develop PropWare), Eclipse, or vim/emacs/gedit/etc. It provides both the build target as well as targets for pushing the code to your device, opening a terminal, and even debugging with GDB if you so choose. If this aspect intrigues you, start here with the "CMake by Example" and then see the full documentation for PropWare's CMake functionality on "CMake Reference". Of course, if CMake is entirely new to you, start here.
2. My own set of C++ classes for the Propeller. See the list of classes available via Doxygen, or just browse the folder hierarchy on GitHub.
3. Pre-compiled Parallax and community provided libraries. Parallax don't provide any easy way to utilize the Simple libraries outside of SimpleIDE, so part of PropWare's build system includes logic to compile all of Simple into static libraries for each memory model. Along with that, I also include libpropeller (one of the key projects that inspired much of PropWare's own classes), libArduino (a decent port of the Arduino classes), as well as a small JSON library that has nothing to do with the Propeller, but works nicely if you need JSON.
When you're ready, head on over to the Getting Started page and let me know if you have questions, either here, in the PropWare thread, or in PropWare's issue tracker.
Re: gdb
No special hardware necessary. The Propeller does not have any hardware debugging capabilities, so the smart guys behind PropGCC did something in software to make GDB work. I've no idea how it works, and I'm sure it's detrimental for performance, but it's there.
Your computations: you're aware the Propeller has no hardware multiple/divide, let alone floating point support, right? So long as you don't need that, it's 8-core/80 MHz design should suit you quite well.
2) SimpleIDE (for C/C++ development) and PropellerIDE (for Spin development) are both available for Windows, Linux, and MacOS (and are open source). Both provide for assembly code. PropellerIDE and SimpleIDE for Linux are about 2 years old (see here).
4) Propeller 1 is not really suitable for big chunks of C code. Shared (hub) RAM is only 32K and is used for code and data. There is a compiler mode (XMM) where code is stored on an SD card and paged in as needed, but this is slow and only supported now by invoking the compiler (and linker) from the command line as far as I remember. Propeller 2 should do much better with large programs. It can execute code directly from shared RAM.
You can't realistically mix Spin and C code. Spin is actually an interpreted virtual machine with the interpreter occupying a cog (or multiple copies in multiple cogs). C code can be compiled to run completely within a cog or uses a small interpreter in the cog with the code in shared (hub) RAM. Data can be in the cog or in hub RAM. The interpreters and hub RAM layout are different so you can't really mix C and Spin except to use a preprocessor to translate Spin to C, then combine with other C code.
5) There's no debug hardware for Propeller 1. Typically one cog is reserved for debug use and this communicates with a PC over a serial/USB connection. One very nice implementation of this is ViewPort for Propeller 1. Propeller 2 has some nice hardware features for debugging, but the software to use them will need to be developed.
Propeller 1 is not really comparable to say an ARM processor. It doesn't have the memory or even the raw speed. There's no hardware floating point, no memory management unit, not even a hardware multiply. The cores have little memory (512 32-bit words). On the other hand, it has a very clean instruction set that works very well for software-defined I/O drivers, instruction emulators, signal processing.
You raise a lot of concerns there which I see others have attempted to answer. So I'd like to comment on your very first sentence: The Propeller 1 executes instructions at 20 MIPS. That is limited to program sizes of 250 odd instructions that can fit in the COG RAM space. Bigger programs require an interpreter loop to run instructions from HUB memory, which cuts performance by a factor of 4 or so. Search here for Large Memory Model (LMM) as used by the C compilers.
It has no multiply or divide in hardware. It is an integer only machine, no floating point.
As such I would say it is one of the least suitable micro-controllers available one could think of if ones requirements are "computationally intensive".
It does have 8 cores which can of course be used to boost performance a bit by spreading the computation load around. Depending on what your problem actually is don't expect that to boost performance by a factor of 8. My FFT for the Propeller can make use of all 8 cores my using OpenMP. I don't recall exactly but after 2 cores the performance gains are almost negligible. See Amdahl's Law.
The Propeller 1 has 32 I/O pins tightly coupled to the processor cores. As you seem to have little in the way of I/O requirements that would be wasted in your application.
Unless you have more details of your application to share I can't help thinking it is not a good fit for your stated requirements.
Everyone's mentioned the lack of the hardware FP support and not even an MPY. This is and has been a concern to me. It's probably fine, to be honest, but I don't at this time know. In fact, while I did not explicitly check, I did not really expect it to have it to begin with. To better explain why _I'm_ looking at the propeller today or what makes the chip interesting to me, the calculations I intend to do are currently done on an AVR for a single set of parameters, and while it's slow, it works. The next step now involves doing tens of thousands of sets of these calculations with varying parameters and effectively optimizing them. I come from a perspective that as long as there's C, emulated floating point is possible even if it's slower. The AVR itself won't do, certainly, but my options at present are :
* AVR, MSP430, and their ilk. The MSP430 is 16bit includes a hardware (integer) MPY, but that's not going to make a real difference. 25MHz and a single core gives me maybe 8MIPS total on a RISC architecture. The AVR will be about the same, if not lower.
* ARM Cortex M3, operating at 96MHz. These are available in QFP as well, I've used them, have them avaialble. They dont really have FP either. Even with pipelining working perfectly, it's about 100MIPS (they claim 125 at 100MHz, which I never quite understood, but whatever).
* ARM Cortex M4 at 125MHz-ish. These do have FPUs, but I don't quite have M4s on hand. I can't take advantage of the fact that the workload can be parallelized quite easily. I expect maybe about 125MIPS, so possibly about as fast as a propeller 1 with all cores crunching away).
* Move to a ARM A series processor running linux, like the Raspberry Pi or it's ilk. I can write python code and use SciPy, which is tempting, but this means BGA processors, BGA RAM. There's no way I'm reliably soldering that board, or even for that matter going to be able to design it in a relatively short period of time. Bootstrapping the kernel to a custom board is entirely alien territory for me. So I'm going to be depndent on someone else's hardware. I won't be able to add the security features on-board that I need. I'll be wasting money on all manner of HDMI and audio and networking connectors that I will have to desolder. All around not my favourite plan, though a current frontrunner.
* Use a propeller, lets its many high speed cores crunch through the problem in a more streamlined manner. I can control the hardware, use less power, marginally higher cost (not really a production run, so I don't get the volume discounts an RPi does), but less depressing and seems more amusing.
I've always thought of the Propeller as an AVR on steroids. Now that I actually look at the documentation and capabilities besides the press releases or so that I saw years ago, that continues to be the case, though the lack of hardware peripherals is surprising. I understand the multi-core architecture is intended to replace them, and that's fine for the most part. My current application does not need or want peripherals besides I2C / SPI (for external memory and some kind of secure key and maybe parameter storage), and dedicating more of the chip to processing power is good for me.
I understand Parallax is a small company. I suspect the problems I am seeing are more of a communication issue than actual lack of support. In specific, Parallax might want to do a better job of curating the introductory experience to the Propeller.
The information provided by @ersmith and @"Mike Green" is specific and concise. However, for the benefit of Parallax let me list the experience I have when I enter that rabbithole.
Openspin, the 'official' spin. Google for it takes me to http://developer.parallax.com/openspin/, which provides a windows download. Linux versions are listed as from external sources, with disclaimed responsibility. This is fine for someone playing around, but is something of a red flag when the question is more of the form of "Here's something I need to do for this customer. What is the most likely way to do it, how long will it take, and what commitment can I make to the customer." As someone who has never used a propeller before, I have zero basis for choosing one of the other spin compilers. My background experience suggests whatever the manufacturer puts out is likely to have the best immediate support from the manufacturer or it's immediate community (support as in : hello world tutorials, peripheral driver libraries, sample code, application notes, consistency with datasheet nomenclature for registers, etc. ), unless the manufacturer puts out some proprietary tools AND there exists a reasonably stable, official, gcc port.
I'm perfectly fine with command line tools. Personally, I typically tend to use Kdevelop for writing the code and the command line for compiling it, and gdb over command line. I don't _want_ simpleIDE, but I'm alright with installing it in order to get all the tools pulled in. This is what actually happens with wiring or whatever the strange toy like thing arduino puts out which is now available for the MSP430 as well as a port. I go to the SimpleIDE page. I click on linux. I'm taken to github, but for _just_ simpleIDE. It then lists a bunch of other git repositories for the other tools. I end up at https://github.com/parallaxinc/propgcc. That readme immediately scares me. Markdown isn't that hard a markup. Clear information about what's there, where it works, how it should be installed, would certainly help. I wonder if it's an unstable, WIP compiler suite. Last update in 2015. I wonder if it's some strange 3rd party fork. Nope. Repository owned by propellerinc.
I'm actually ok with GCC 4.6 as long as it generates decent optimized output. GCC 6.x would certainly be nice, but that may well just be a psychological difference. I don't actually know where this beta is, but if GCC 4.6 is officially supported, that's where I would like to start anyway. Until
Oops. I'll set gdb aside for now until I get it basically operational and have a better sense of the ecosystem.
And then @DavidZemon comes to the rescue with
Whew. Also, this says to me, "gdb might not be a first class debug tool in the propeller world", which is fine. It would just be nice to have a good, clean, curated list somewhere with parallax's stamp on it saying
* These are the tools we have or know of that work, are current, you can use to start with.
* These are the tradeoffs involved in choice of language and toolchain.
* This is a set of tools that you should start with if you have no external preconditions, because they give you best performance or usability or feature support.
While most other manufacturers do do this to an extent, the bit I typically care about is them saying, "Hey, this is our tool. You can pay so much for it. There's also gcc, and there's a port over there that we know about, we don't do ourselves, we can't support, but we know more or less works. Our tool is this much better than gcc.". They all have slightly different ASM anyway, and they have different instruction sets. Parallax talks a fair bit about PASM, which is nice but not something that's critical at the first introduction to the platform to most of us. The addition of Spin is the problem. It confuses matters greatly, and perhaps it's a better language for the platform, but the added confusion needs to be mitigated by a clean, sober, discussion of what the language entails, whether standard C is supported, what the corresponding toolchains look like, what is the extent to which interoperability is or is not supported. And I don't mean with screenshots of toy IDEs (no offense). Your IDE is less important than what the underlying tools are, and unless you're JetBrains your IDE isn't likely to change the world. It doesn't help that spin itself has a number of compilers available without a canonical side by side canonical comparison.
@DavidZemon PropWare sounds very promising. I will be taking a look at it (I have not yet with sufficient detail). CMake is perfectly fine. I use it with kdevelop and maintain a personal CMake based build system for the AVR and the MSP430 (https://github.com/chintal/toolchains) which I use to reuse my library code across those platforms. In reality the bulk of CMake's job in my case is done through the command line, but kdevelop adds in code completion and some just-enough-to-be-useful-not-enough-to-be-beautiful intellisense. I do typically prefer C over C++ on constrained platforms, but it is nice to know there are at least C++ libraries out there. I would ideally want to be able to add on a propeller toolchain to the architecture there (My problem, not yours or parallax's) so that I can move around my modbus library, for instance, but that's some ways down the road.
I would not worry about how Parallax supports OpenSpin. OpenSpin just works. It's easy to compile on any platform. Like many Open Source tools it is quite likely better supported than a manufacturers compiler.
I don't expect most people to get into PASM much, but I have to say it is one of the simplest, nicest assembly languages I have ever used. Very regular and logical. The fact that within a COG, memory is registers, registers are memory makes it so simple to understand.
You can use one cog for doing debugging by loading a serial I/O driver that's been modified to allow multiple cogs to insert lines of text into the output buffer ... also not a big deal. If memory's tight, there's one driver that keeps its buffer in the cog's memory. Essentially you get a log for the cost of one cog and one I/O pin.
Spin is a language and interpreter for a stack machine. The bytecode structure is optimized for space, but is also quite fast. Operators and built-in functions represent primitive operations of the hardware and there's an assembler as well. The compiler/assembler is very fast so programs are kept as source and always recompiled. High speed device drivers are usually written in assembler with interfaces (usually through shared memory) written in Spin and packaged as Spin "objects" with documentation, definitions and functions (methods) together. Programs have a main "object" that refers to lower level "objects". These are normally loaded with the program and copied to cogs from shared (hub) memory during initialization. The Spin interpreter is included in ROM on the Propeller 1 chip and is loaded into cog 0 from there during chip initialization. Any forking of additional cogs either loads the cog from hub RAM or from the ROM copy of the Spin interpreter.
I normally use a Mac for development with openspin for Spin/Assembly and proploader for downloading compiled programs to the Propeller. I use BBEdit for editing and that can start a script for compiling and downloading. I've also used a RaspberryPi version of PropellerIDE (and SimpleIDE for C) and I've run the Linux versions as well.
When the Propeller 1 came on the market, Spin and Assembly were all that was available. The Spin bytecodes were proprietary. It took several years for that to change and for alternative Spin/Assembly compilers and C/C++ compilers to become available. There are several good Forth implementations
To briefly answer your first question about why the Propeller 1 doesn't get the same visibility and traction as the educational tools on our web site - it's a difficulty we've had with fragmenting our customers and serving commercial and educational customers equally. For a long time, we attempted to run Parallax Semiconductor for the commercial users so the educational customer wouldn't be confused with the product design and engineering applications, and so that the commercial user wouldn't look at the Propeller 1 and think it's some kind of toy with the many robot products we offer.
To properly support Parallax Semiconductor, our customers expected lots more than we were able to deliver: more chips, faster; more professional tools and support; FAEs; etc. As we're a bootstrapped company and our Propeller 2 didn't materialize (it's basically done now, 13 years later) we eventually closed Parallax Semiconductor and focused on educational markets.
Education is about 85% of our revenue and the bulk of our profit. Robots (maybe a million of them), teachers and students have made both Propeller 1 and Propeller 2 possible from a financial perspective. But that's peanuts compared to Chip's work on both devices, which now spans over 20 years. The former represents the business sense; the latter is the long-term goal.
On this forum you will find nearly all engineers, product developers, and Propeller 2 contributors. This is the place to start, really.
As we move forward with Propeller 2 you will see us attempt to serve both customer bases again. It should be easier this time.
Propeller 1 isn't going away, so you know. In fact, we're just now taking on a second foundry source to handle projected shortages from our primary supplier. And, Propeller 2 is minutes away from being committed to silicon.
Ken Gracey, CEO
Parallax Inc.
The tools required to program the prop should be front and centre, not buried four layers deep in a support directory! Until I saw Mike's post I had no idea anything, other than the Parallax software, was even available on the Parallax website. Glad I finally found it but it should have been easier. The software required to program the prop is just as important as the chip itself.
Sandy
That's a very wide 'footprint' of candidates you have there, as well as some strange 'need to's' - why do you need to desolder anything from a Pi ? use a Pi Zero if you just want an engine.
What 'security' features do you need ?
you also say "currently done on an AVR for a single set of parameters, and while it's slow, it works. The next step now involves doing tens of thousands of sets of these calculations with varying parameters and effectively optimizing them"
That's a quantum leap in workload, so nothing at the low end of your curve will be able to do that.
You really need to define how many calculations you need, and what precision you need, and what size data you have to work with.
" tens of thousands of sets" really sounds outside microcontroller space.
So why not stick with the AVR?
At 20MHZ an AVR does 20MIPS (give or take depending on instruction mix but certainly way more than your 8MIPS figure).
It's only 8 cores. And moving data between cores, once you go off-chip, is no easier than moving data between single core chips.
I have a set of equations taking in a few numbers (call them B ) and estimating some other numbers (call them C). This relationship is something that needs to be protected, because it represents some form of intellectual property. This IP does not belong to me, and I'm not in a position to question the IP owner's estimation of their value or how they wish to expose it. These aren't pretty equations, but there are no real time requirements for this. They're perfectly fine waiting a few seconds for it to happen.
We now have an additional set of equations which can estimate those original few numbers (B) based on some other numbers (A), and by optimizing this new set of numbers along with constraints set on A, B, and C, a suitable set of results can be obtained. This is the goal. And yes, it is very tiresome to try to explain it without explaining it.
I can run the original equations in just under the second on the AVR. If the optimization algorithm is efficient and / or the space available for the inputs is small, something like the propeller would have gotten me there. I do not now believe either is the case. The available space is massive, and the algorithm needs a lot of work, and a uC is no place to try to do that. I've recommended looking at it again when there is a real optimization algorithm in place.
In practice, for such a workload, I expected the propeller to give me the following advantages over the AVR :
- 32 bit over 8 bit means arithmetic operations should be faster in terms of instructions per operation, even without FP or MUL/DIV, since both lack that kind of hardware.
- 80MHz clock over 8 cores, even if a couple of cores are tied up doing other maintenance type things, should get me to atleast 5 times (originally expected 20 times, see below) faster than the AVR in terms of raw MIPS.
- 512x32b RAM / 2k bytes of RAM is about as much RAM as I get with a single AVR. It's not a lot, sure, but it's not very little either, given each core has that much RAM available.
I did not realize the AVR does do some basic pipelining as well as has a lot of single clock instructions (thanks @"Brian Fairchild" ), giving it about 1MIPS/MHz. This is actually surprising to me, and yes, it does take away a significant chunk of the advantage of the Propeller's higher clock, at least on paper.
a) Why do you replace system CMake entirely? Surely there should be a simpler way to install CMake modules/support packages (I presume that's why you do this?) without replacing the CMake install itself? Or at the very least, install into /opt or /usr/local with a $PATH edit that makes the switch. I've found CMake to be relatively finicky about CMake versions. This switching could in principle break other people's CMake projects or have them start spitting out new deprecation warnings. I expect this also moves the CMake packages folder to /usr/pwcmake/share/... (not a fan of this, by the way. You should be installing into /usr/share or, even better, /usr/local/share), which means any CMake packages I have installed in /usr/share/cmake-3.10 would probably no longer be found.
b) Does the installer not include gcc et all? If so, perhaps add that step to the getting started guide?
c) Have you considered putting the deb onto a PPA or so, allowing updates to be pushed through there?
Don't worry about the discussion. It's very useful for forum members to see what thoughts might go into a decision about whether a Propeller is suitable for some particular application, particularly someone who may not have used a Propeller before. As you may have figured out, the Propeller is a fairly unique microcontroller. Even with the relatively limited memory, it can do voice synthesis of 4-part harmony in stereo (for spatial positioning of the parts). It can do emulation of an 8080 or a Z-80 running CP/M with mass storage, TV text output, and PS/2 keyboard processing, all at faster than the original hardware. It can compile Spin code and assembly code under a simple "operating system" with TV or VGA display, PS/2 keyboard, and SD card support. There is a paging library written in Spin as well as the not-well-supported XMM mode for the C compiler. If you're not in a hurry, your application could probably be done in Spin.
Why not do it with a RaspberryPi Zero W? Maybe in Python? The PiZW is $10 from Adafruit. Case, HDMI and USB adapters, SD card, and power cube will cost several times that, but still awfully cheap. Development would be a lot easier. I don't quite understand what you need to connect your engine to, but the PiZW can certainly handle simple interfacing to other hardware.
That is quite common here and simultaneously endlessly frustrating and useful. I feel for you
Regarding your PropWare questions:
a) why replace the system-installed CMake
Without hand-writing the deb and rpm packages, I don't have any way to determine where the CMake modules should be installed because the directory has the version number in it. That's a real bummer... but I don't know any way around it other than a script that executes "cmake --version" and then scrapes the output to build the value "/usr/share/cmake-<VERSION>"... but that implies hand-written deb/rpm instead of letting CPack do it all for me.
As for the paths being used... you're absolutely right. I just tested a fix for x86 Deb-based OSes and it's working well. I'll need to update some things for the other OSes before publishing, but follow this ticket for progress. The new deb installs CMake directly under /usr (instead of /usr/pwcmake) and PropWare moves from /usr/PropWare to /usr/share/PropWare. I've also added an exclude in there for CMake's "doc" folder since the standard deb package from Ubuntu doesn't provide that either.
You're also not the first to bring up the issue of conflicting CMake installations. For exactly that reason, I broke the build system away from the C++ code. You can drop the new repo into your own CMake modules directory or use a Git submodule to add it to your project's directory in exactly the same fashion as PropWare does. Here's the build system repo: https://github.com/DavidZemon/PropWare-Build.
b) No compiler install instructions
The installer does not provide any compilers. You're absolutely right, that should be explicitly stated on the Getting Started page along with how to download them. I've just added a comment to one of my existing tickets to ensure this gets fixed - feel free to subscribe for updates
c) No PPA
I've considered a PPA. I'd love to make that happen because it would be a great way to push out updates wouldn't it? Unfortunately, PPA requires source packages which CPack doesn't generate for me.
Why point him there instead of PropGCC?
That wouldn't be the vocabulary used, but yes. You have a few different way to execute code at the speed of PASM:
1) Take the DAT section from a Spin file, compile it with the likes of spin2cpp into its own binary, then run that binary blob in a cog. That would be exactly the same thing as Spin, but can be done with C/C++ too.
2) Write a file in assembly (typically a .s or .S extension) and build it into a binary blob. Run it like #1.
3) Write a C/C++ file that gets compiled into native binary (as opposed to CMM or LMM binary) and then run that code just like #1. Files like this are named .cogc (or in PropWare you get the additional cogcpp).
4) Choose the "COG" memory model, which just means your entire application runs in a single cog without any HUB execution like CMM and LMM. Not terribly useful... but it's there. (Also, I know CMM and LMM don't execute from HUB... but way beyond the scope of the question or this answer.)
5) Use FCache. I've said it before and I'll say it 10,000 more times because, in case you didn't know, I love FCache. FCache gives PropGCC users super duper performance without needing dedicated cogs
See FCache example here: https://github.com/parallaxinc/PropWare/blob/develop/PropWare/serial/uart/uarttx.h#L103
EDITED to try and not start another language war
Fair enough. Wasn't sure if there was some point that he brought up which made you think Catalina would be a better choice or just, as you say, a different choice.
Where does the restriction on the PPAs (needing source packages) come from? I suppose Launchpad or so might have specific requirements, but as an apt repository itself I remember setting up a private apt repository with almost exclusively binary packages. That is quite a lot of extra work, though. I might decide to try to set one up in the next couple of months for some other reason. If you like I could see about creating a second one along side it.
I understand the problem with the CMake Module installation process. I'm still convinced there should be a 'better way', but I confess I never found one either. Since I'm probably the only person using my toolchains, I never spent a great deal of time looking for that solution. I see the CMake build system you have is quite complex, though if it helps, the way I get around the problem is to not touch system CMake modules at all. I don't even bother about non-linux platforms, though, and my approach would take you far from your intent of making it Arduino-easy.
I keep a ~/code/toolchains folder containing the various build systems I use. The toolchain files are provided to the project only at the time cmake is run against it (cmake -DCMAKE_TOOLCHAIN_FILE= ... ). The toolchain files then point to the correct paths to look for CMake Modules and installed packages.
The project sources themselves make no reference to the toolchain at all, though it might provide a list of supported devices. Each toolchain, when asked to make all in that project, builds the library or executable for each supported device it recognizes. Each of my toolchains also uses a Platform file that provides ADD_PLATFORM_EXECUTABLE and ADD_PLATFORM_LIBRARY (perhaps similar to your ADD_SIMPLE_EXECUTABLE). This results in significant limitations compared to the native ADD_* functions, and a number of useful cmake features are either not available or very difficult to use because of this. However, my libraries themselves remain portable. I can create multiple build folders for multiple platforms, and for each platform can trigger builds for multiple devices at one go.
I'm certainly not opposing to using one that's already setup. I wonder if there's an easy Docker image I could stand up on my server... that'd be cool and easy. Hadn't thought about using a self-hosted repository before.
I'm definitely open to a 'better way' if you ever come up with one. And like I said, the harder but "better" way of doing it is downloading the build system by itself and then using it with an existing CMake installation. I'll add instructions for doing that someday.
Oh yea - that is certainly the way CMake seems to recommend doing things. I don't like it at all though - I think it should be much easier than that!
"This results in significant limitations"? Are you not using the add_* functions at the core? PropWare's create_* functions are just wrappers around add_*, so you still have all the functionality available, such as linking your own libraries with target_link_libraries and transitive dependencies.
I do, but I end up changing the function signature slightly to allow the build system to mangle the library/device names to include the target device name. This includes providing the dependencies via the function call itself so that those names can also be mangled to make sure I use the right library.
FUNCTION(add_platform_library LIBRARY_NAME LIBRARY_TYPE DEPENDENCIES)
FUNCTION(add_platform_executable EXECUTABLE_NAME DEPENDENCIES)
In practice, I don't actually know the target names at the time the CMakeLists.txt is written. When I write a modbus library, for instance, I have an ADD_PLATFORM_LIBRARY call which just takes "modbus" as the name. When combined with the msp430 build system, the library targets look like "modbus-msp430f5529" etc. (for each device in the intersection of the devices the build system is configured to build for and the set of devices the library claims it supports), with the AVR build system I get "modbus-atmega644" etc.
This can be cleaned up somewhat, but the way CMake handles function arguments doesn't quite give me the kind of flexibility I would need to get all the newer features working properly. I think I could, in principle, make it work by individually wrapping _every_ CMake function that deals with targets. I'm not sure it's worth the effort.