Shop OBEX P1 Docs P2 Docs Learn Events
[FYI] libpropeller: MIT C++ library — Parallax Forums

[FYI] libpropeller: MIT C++ library

SRLMSRLM Posts: 5,045
edited 2014-01-14 12:17 in Propeller 1
This post introduces libpropeller, a collection of PropGCC C++ objects free for use under the MIT license.

libpropeller is a collection of PropGCC C++ classes that I use every day for my work. I've been developing these classes for 12 months now, and have finally received permission to share them under the MIT license. Each class is optimized for efficiency in both code size and run time, and has unit testing. Using many of these classes I've made a complex real time application that remains small at less than 22KB (downloaded).

This library is a constant work in progress, so be sure to make a local copy and check any repository changes before updating. Many classes are not stable and the API may change dramatically. And as always with code, there are things that I would like to change and update. But, I've decided to go ahead and share the code in order to get it out there.

This library is open sourced under the MIT license. It's been supported generously by Kenneth Bedolla and Red9 Sensor. I'd like to thank both sponsors for allowing me to work on it, and agreeing to open source that work. As appropriate, the copyright of code that they supported has been documented. I'm the author of all of the current code, although much of it has been taken from Spin MIT code. Where appropriate this history has been documented and credit given. I'd also like to thank ersmith, jazzed, and David Betz for all their help with PropellerGCC issues and spin2cpp.

libpropeller Features:
- C++ for the Propeller!
- Optimized classes for many tasks
- Actively in development
- Extensive unit and field testing
- MIT Licensed

Relevant links:
Source: https://github.com/libpropeller/libpropeller
Wiki: wiki.libpropeller.org/
API Documentation: api.libpropeller.org

Everything is hosted on github here.

attachment.php?attachmentid=104598&d=1382850401

Comments

  • danielstrittdanielstritt Posts: 43
    edited 2013-10-26 22:02
    Clicking the Source: libpropeller.org link gave me a 404. Just so you know

    Daniel
  • SRLMSRLM Posts: 5,045
    edited 2013-10-26 22:04
    That's interesting. I didn't think to try it with Firefox (and I don't have IE8). It appears that firefox automatically adds a www, which is technically a subdomain. Anyway, I've fixed it for now. Thanks!
  • Heater.Heater. Posts: 21,230
    edited 2013-10-26 23:14
    That's wonderful.

    I have to take issue with statement regarding putting all the code in header files.
    It's not so good for .cpp files: you have to define those explicitly in your makefile.

    The idea of a Make file is not just to get files built into your project but to be able to rebuild only those parts that change.

    If my project has multiple classes in traditional .cpp and .h files and uses a Makefile I would like to have the .cpp files dependence on any headers they use expressed in the Makefile. This ensures that .cpp files will be recompiled if I make any changes to their included headers.

    Of course on small Propeller programs compile time is not such an issue and perhaps one can live with compiling everything all the time.

    Also, I'm wondering now what happens when I include the same library header in two or more of my source files?

    I would have thought the same code would be compiled into all files that use the header thus making the program bigger.

    Magically it turns out this is not true. Probably because functions and data in headers get compiled with "weak" linkage and only one copy is actually included in the final code.
  • SRLMSRLM Posts: 5,045
    edited 2013-10-27 05:18
    Heater. wrote: »
    That's wonderful.

    Thanks!
    Heater. wrote: »
    The idea of a Make file is not just to get files built into your project but to be able to rebuild only those parts that change.

    If my project has multiple classes in traditional .cpp and .h files and uses a Makefile I would like to have the .cpp files dependence on any headers they use expressed in the Makefile. This ensures that .cpp files will be recompiled if I make any changes to their included headers.

    But isn't all that a hassle? Makefiles are one of the most annoying things about working on a new project. They're great for the larger projects: you can set it up exactly the way that you want, with lot's of control. But for a smaller project it's a major drawback to doing quick experimentation. My solution has been to come up with a makefile that uses "*.cpp" in the parameters to GCC. This helps, but it's still not a total solution.

    The best build system I've seen so far is Java's Maven build. All libraries are automatically downloaded as needed, it finds the source, and there's all sorts of plugins. From what I've seen so far it's pretty neat.

    And you're right about the weak linkage. With C++ Inline functions only get included once, no matter how many times you #include them. It's one of the nice features that make inline headers work for this library.
  • Heater.Heater. Posts: 21,230
    edited 2013-10-27 09:04
    SRLM,
    But isn't all that a hassle?

    I totally agree. Header files are a pain and Make file are the spawn of Satan.

    Seems the only reason we have headers is so that people can bundle up code into "secret" binary only libraries that they want to sell you. The header files enable you to compile your code to match
    pre-compiled binaries.

    I guess they probably also helped compilers when machines were much slower and smaller.

    Logically headers make no sense. Who wants to write all those function signatures twice and maintain two files instead of one? Especially in the open source world, just give me the source and I'll compile my app against it
    thankyou.

    You start to appreciate this when using Pascal. No headers. No Makefiles. Just point your compiler at your main program module and it finds everything it needs for itself. Just like Spin does. Why can't C/C++ do that?

    Seems there are many others thinking this way and there are proposals to give C++ some kind of module system (no headers) in a future standard.

    Surely under your system my application would only have one .cpp file for main(). After that everything else is pulled in via headers, my other modules and any libraries.
  • Heater.Heater. Posts: 21,230
    edited 2013-10-27 09:05
    SRLM,

    We should be a bit clearer about the use of the phrase "inline functions".

    The normally accepted idea of "inlining" is that the compiler can choose not to generate code for a function and calls to that function but rather generate the code for the function at every place it is used. This save the overheads of generating the call and return instructions along with handling local variables on the stack etc. It can be a great speed up for small functions.

    Your usage of "inlining" is about including source code into a file via by placing it in header files. This is not the same thing. Of course the compiler needs the code to be in the header so that it can see it at compile time but it may or may not choose to actually inline the function bodies as I described.

    If code is inlined it can speed things up, it can also make the binaries bigger as there are now many copies of a function spread throughhout the executable.
  • SRLMSRLM Posts: 5,045
    edited 2013-10-27 09:45
    I agree that the word "inline" is misleading, since it is just a suggestion. Technically, any definition in a class definition is defined as inline. In that case the inline keyword is optional. From the 2012 C++ standard working draft:
    A function defined within a class definition is an inline function.

    I guess we could relabel them "inline-marked" and "inline-implemented". And you're right: if code is inline-implemented then it can make the binaries bigger, but GCC seems to only inline-implement code very rarely. Most of the time inline-marked code is treated as just a regular function, without any inline-implementation or code size increase*. That, plus the maintainability of the code, are the main reasons I decided to put everything in headers.

    * As the wiki page explains, in practice inline-marking seems to decrease the binary size due to the optimizations that can be applied.

    ps @Heater: I'd just like to say thanks for sharing about Node.js on these forums. I had to make a website recently and so I gave it a shot. Turns out that it's really fun, and is web programming the way that it should be.


    For completeness, below seems to be the main points about inlining:
    There can be more than one definition of a class type (Clause 9), enumeration type (7.2), inline function with
    external linkage (7.1.2), class template (Clause 14), non-static function template (14.5.6), static data member
    of a class template (14.5.1.3), member function of a class template (14.5.1.1), or template specialization for
    which some template parameters are not specified (14.7, 14.5.5) in a program provided that each definition
    appears in a different translation unit, and provided the definitions satisfy the following requirements...

    And for completeness, this is what it says about inlining:
    7.1.2
    2 A function declaration (8.3.5, 9.3, 11.3) with an inline specifier declares an inline function. The inline
    specifier indicates to the implementation that inline substitution of the function body at the point of call
    is to be preferred to the usual function call mechanism. An implementation is not required to perform this
    inline substitution at the point of call; however, even if this inline substitution is omitted, the other rules
    for inline functions defined by 7.1.2 shall still be respected.
    3 A function defined within a class definition is an inline function. The inline specifier shall not appear on
    a block scope function declaration.* If the inline specifier is used in a friend declaration, that declaration
    shall be a definition or the function shall have previously been declared inline.
    4 An inline function shall be defined in every translation unit in which it is odr-used and shall have exactly
    the same definition in every case (3.2). [ Note: A call to the inline function may be encountered before its
    definition appears in the translation unit. — end note ] If the definition of a function appears in a translation
    unit before its first declaration as inline, the program is ill-formed. If a function with external linkage is
    declared inline in one translation unit, it shall be declared inline in all translation units in which it appears;
    no diagnostic is required. An inline function with external linkage shall have the same address in all
    translation units. A static local variable in an extern inline function always refers to the same object.
    A string literal in the body of an extern inline function is the same object in different translation units.
    [ Note: A string literal appearing in a default argument is not in the body of an inline function merely
    because the expression is used in a function call from that inline function. — end note ] A type defined
    within the body of an extern inline function is the same type in every translation unit.

    * The inline keyword has no effect on the linkage of a function.
  • Heater.Heater. Posts: 21,230
    edited 2013-10-27 10:53
    It's kind of hard to out think the optimizers in compilers now a days. Inlining for example looks like it would make binaries bigger, obviously, you are sticking lot's of bits of code into your program instead of having it just once. BUT if the code is small and the overheads of calling and returning and tweaking the stack are bigger then the overall result is less code. And then once you have that code inline in the caller perhaps the optimizer can see other things it can do to help which it could not do before.

    My approach now is not to think about it much. Write the simplest clearest code you can and let the compiler work it out.

    On modern processors with deep pipelines, multiple execution units and caches it gets even harder to arrive at optimal solutions. For example it used to be that unrolling loops was often a good speed up. Maybe not so now. If you unroll that loop it might get too big for cache and suddenly gets a big performance hit.

    Glad to hear you like Node.js. I have to try and find time to get back to the Propeller-Node thing.
  • KyeKye Posts: 2,200
    edited 2013-10-28 07:29
    This is wonderful SRLM!
  • jazzedjazzed Posts: 11,803
    edited 2013-10-28 11:20
    @SRLM, it's really great that your sponsors were kind enough to let you share your work.

    Thanks.
  • Ken GraceyKen Gracey Posts: 7,386
    edited 2014-01-11 08:18
    Hello SRLM,

    We received your reply on making libpropeller MIT C++ library an Open Propeller Project here http://www.parallax.com/news/2014-01-10/open-propeller-project (thank you).

    The idea with Open Propeller Projects is just like I described - that we can rally more people behind these goals that benefit the Propeller community. As I see it, libpropeller provides the first formalized C++ selection, built primarily on our core C libraries. From the GitHub https://github.com/libpropeller/libpropeller it's clear what's been produced and what status pertains to the different classes of code, what coding standard is used, and how people can participate by submitting files to the authors. It seems to meet the intent the Open Propeller Project. The only part I'm not sure about is whether this source would become the C++ standard - and I don't think it's necessary to make that decision right now anyway. The point is you've produced a set of C++ classes, you would like more involvement to improve them, and everything is available with the MIT license. Do I have it mostly correct?

    The next steps that I'd take would be to see that Parallax gets behind the effort by recognizing it much more as an Open Propeller Project. But there are some things we need to know to make that productive. To bring in more people, we should define the needs and benefits together.

    1. What kinds of help would you like from the community to grow the libpropeller C++ libraries? To develop and submit new classes, or improve on existing ones?
    2. Which kinds of object-oriented examples should be made available to demonstrate libpropeller to C++ newbies or first-timers to the Propeller? Could it provide a way for Parallax's C programmers grown on learn.parallax.com to come over to C++? What kinds of examples would we want produced to make this possible, if it's a goal? I believe it's a goal, but I'm not a C programmer (at least I don't know much beyond what we've introduced on Learn.parallax.com).
    3. Would you also want a quick-start type of tutorial to show how to load these files into the SIDE environment to make them easy to get started with?

    Please define more of what you'd like to see around libpropeller C++. I believe we could activate this effort much more once I understand some of the needs that you have as the creator, along with the benefits to users.

    I think we're close to rallying more energy behind this effort - just need to know more high-level background to make it happen.

    Look forward to your reply!

    Sincerely,

    Ken Gracey
  • SRLMSRLM Posts: 5,045
    edited 2014-01-14 12:17
    Hi Ken,

    Thanks for taking an interest. In regards to your enumerated questions:

    1. Any sort of feedback or contribution is welcome. In order to maintain code quality, all contributions will be checked by me to make sure that they fit well but other than that it's pretty open.
    2a. The nice thing about unit testing is that it provides a very complete example of how to use the code. Nearly all the objects have units tests, so that's the easiest way to see how to use the code. I don't have any plans for releasing "traditional" examples (ie, demos) for that reason, but I may release some example applications.
    2b. I think it's an excellent way to go from C to C++. My personal opinion is to learn C++ first, than back down to C. But anyway, all the code in libpropeller is in classes, and there's a wide variety of techniques showcased there.
    3. I haven't worked with SIDE much. The times I have used it C++ was pretty easy. One question that remains to be tested is inclusion of .S files, but the Linux SIDE version is lagging and I'm waiting for an update on that. I suppose I could build from source, but I haven't had time yet.

    A little bit of philosophy here:

    There are a few reasons why I decided to go with GitHub over posting this code to the Obex. Much of the reasons are listed and discussed in the thread I created to discuss moving PropGCC to GitHub (here).

    What I would really like to see is just more usage of C++ on the Propeller. I think it's just as efficient as C, but with with many useful features that are particularly suited to the Propeller. In fact, C++ better mirrors Spin so it makes a transition easier IMHO. libpropeller is intended to jumpstart that transition to C++, and to prove that C++ on a 32KB microcontroller can be both efficient and lean.
Sign In or Register to comment.