Shop OBEX P1 Docs P2 Docs Learn Events
[FYI] PropWare: Complete build system and library for PropGCC - Page 4 — Parallax Forums

[FYI] PropWare: Complete build system and library for PropGCC

124

Comments

  • WOW.

    PropWare's first commit was made April 6, 2013 at 7:03pm. I didn't know proper Doxygen syntax. I didn't know about stdint.h. I didn't know about #pragma once. I was using C (and not even good C). There were no tests, and only one demo app.

    I was using Make, but only as part of the demo. PropWare would not get a reusable Makefile for another 14 minutes. I was still using Eclipse (CLion wouldn't be released for a very long time still).

    The only objects were SPI, GPIO, and a very, very incomplete SD implementation.

    Today, more than two and a half years later, I am please to announce the first ever, official release of PropWare: version 2.0.0.

    Following this release, version numbers will be incremented in a sane fashion. Nightly branches will use a build number as the fourth part of the version; release branch versions will increment with every push to the remote repository. There is still lots more work to be done and I have plenty of open tickets in the GitHub issue tracker. My first step will be to prioritize which ones are marked as v2.1 releases, and which remain as "someday". One of my top priorities will be to finally create a buffered UART object. I know this is something used by almost every Propeller project, and PropWare is still to this day missing it.

    Also, tomorrow is my four year anniversary! My last anniversary before I am wed in May. This is a HAPPY WEEK! :D
  • Congratulations, David (on many levels...)! PropWare is an achievement you can be proud of.

    Eric
  • Version 2.0? I guess I need to try this out! I completely missed version 1.x. :-(

    Four year anniversary of what? Not wedding anniversary if you're not even getting married until May. Anyway, congratulations!!
  • Ha, I made git tags for version 1.0, 1.1, and 1.2 a long time ago back when I was on make. Looking back, that was premature. I should have such with 0.x versions for a while longer.

    Four years we've been dating, as of today. :)
  • DavidZemon wrote: »
    Ha, I made git tags for version 1.0, 1.1, and 1.2 a long time ago back when I was on make. Looking back, that was premature. I should have such with 0.x versions for a while longer.

    Four years we've been dating, as of today. :)
    Four years dating? I guess you know each other pretty well by now. Congratulations!

    I just looked at the PropWare installation instructions for the Mac. I already have cmake installed. Can't I skip many of those steps? Ideally, it would be nice to be able to install without having to use sudo.

  • PropWare requires a number of special cmake files. To make life easy, and to minimize the lines of code required in a project's CMakeLists.txt file, I've placed these files in the installation directory for cmake. PropWare also currently has a very strict requirement on cmake version: I'm using a feature in 3.3 and discovered a bug in 3.4. All this means it is way easier to deploy cmake as part of PropWare.

    Deploying without sudo is definitely possible. Install cmake wherever you choose, and be sure to add it to PATH (or create an alias or invoke it with the full path of whatever you want to do). When you extract the PropWare zip, with libs and includes, at an environment variable to the root of that directory called PROPWARE_PATH. I'll get this added to the instructions page
  • DavidZemon wrote: »
    PropWare requires a number of special cmake files. To make life easy, and to minimize the lines of code required in a project's CMakeLists.txt file, I've placed these files in the installation directory for cmake. PropWare also currently has a very strict requirement on cmake version: I'm using a feature in 3.3 and discovered a bug in 3.4. All this means it is way easier to deploy cmake as part of PropWare.

    Deploying without sudo is definitely possible. Install cmake wherever you choose, and be sure to add it to PATH (or create an alias or invoke it with the full path of whatever you want to do). When you extract the PropWare zip, with libs and includes, at an environment variable to the root of that directory called PROPWARE_PATH. I'll get this added to the instructions page
    Looks like I'm in trouble:
    david-betzs-macbook-pro:esp8266-loader dbetz$ cmake --version
    cmake version 3.4.1
    

  • Lol yep. The version shipped with PropWare is 3.3.2.
  • DavidZemon wrote: »
    Lol yep. The version shipped with PropWare is 3.3.2.
    Interestingly, I installed cmake a while back so I could build Dave Hein's P2 C compiler. I believe you wrote the cmake scripts for that and they worked fine under 3.4.1.

  • Yes. There's a bug that seemed to only be affecting PropWare. Haven't narrowed it down enough to report it to kitware yet :/
  • DavidZemon wrote: »
    Yes. There's a bug that seemed to only be affecting PropWare. Haven't narrowed it down enough to report it to kitware yet :/

    Well, as is usual, I spoke too soon. This "bug" I spoke of was in fact a documented change with v3.4.0. I've added a check in PropWare's build system to check the version and do the appropriate action depending on whether the version is newer or older.

    In other news, I just discovered that PropWare was being compiled with exceptions enabled. Whoops! I've added "-fno-exceptions" as a default compile flag and everyone can grab the latest binaries from the develop branch on my build server as usual. I encourage all users to either download the latest version, or add the following line to their CMakeLists.txt files:
    set(COMMON_FLAGS "${COMMON_FLAGS} -fno-exceptions")
    

    This should decrease code size by a noticable amount. In one of the demos in PropWare, the size decreased from ~17kB to ~16kB.
  • PropWare 2.1 is coming along quite nicely. Already there are 10 tickets closed (4 to go) including better use of CMake (per-target flags rather than per-directory), a "create_library" function, Spin support in the form of Spin -> C++, and even a (unsupported) Raspberry Pi 2 Debian package!

    The Pi package is "unsupported" because I unfortunately have no good way to have it build automatically with the other releases, so it requires me booting up a Pi and running a special build, then uploading it to a server and posting a new link.

    You can all grab the latest development packages by hovering over "View" in the "Artifacts" column on this page.

    David Betz:
    The special CMake files are now packaged up in a standalone zip, so you can add them to your existing CMake installation if you wish. Simply download PropWare-XYZ-Generic-standalone-cmake.zip and extract the contents into <cmake install>/share/cmake-X.Y/. There should be a folder named "Modules" under cmake-X.Y, and the zip file has a folder named Modules as well - just merge the two (there should not be any conflicts - only new files).
  • DavidZemon wrote: »
    David Betz:
    The special CMake files are now packaged up in a standalone zip, so you can add them to your existing CMake installation if you wish. Simply download PropWare-XYZ-Generic-standalone-cmake.zip and extract the contents into <cmake install>/share/cmake-X.Y/. There should be a folder named "Modules" under cmake-X.Y, and the zip file has a folder named Modules as well - just merge the two (there should not be any conflicts - only new files).
    Thanks. However, I don't understand why PropWare requires extensions to CMake that have to be installed in a system directory. Can't its CMake files be local to the PropWare directory so no modifications to the system are required?

  • DavidZemonDavidZemon Posts: 2,973
    edited 2016-03-23 03:22
    David Betz wrote: »
    DavidZemon wrote: »
    David Betz:
    The special CMake files are now packaged up in a standalone zip, so you can add them to your existing CMake installation if you wish. Simply download PropWare-XYZ-Generic-standalone-cmake.zip and extract the contents into <cmake install>/share/cmake-X.Y/. There should be a folder named "Modules" under cmake-X.Y, and the zip file has a folder named Modules as well - just merge the two (there should not be any conflicts - only new files).
    Thanks. However, I don't understand why PropWare requires extensions to CMake that have to be installed in a system directory. Can't its CMake files be local to the PropWare directory so no modifications to the system are required?

    I chose the least bad of a few different options. Everything in the CMakeModules directory of PropWare's source code is treated as a native extension, not a simple "import" or "include" in the sense of a normal programming language. Treating PropWare's build system source as such makes the CMake configuration for a user project very natural and easy (as in: very CMake-like)
    cmake_minimum_required(VERSION 3.3)
    find_package(PropWare REQUIRED)
    set(MODEL cog)
    project(Blinky_Demo)
    create_simple_executable(${PROJECT_NAME} Blink_Demo.cpp)
    

    So, somehow CMake needs to know about these extra files (for a normal CMake installation, how is it supposed to know about the "PropWare" package?). One way is to just drop them in the CMake installation, so that they truly become part of CMake. The other option is to tell CMake that another directory should be added to its search path. You then have two different options for that extra search path: local to your project, or shared by multiple projects. Of course for PropWare's source code, I keep it local to the project. If you were to share it with multiple projects of your own though, it might be easiest to set an environment variable and then reference it from your own project (CMake allows the syntax "$ENV{MY_ENV_VAR_NAME}" to easily reference environment variables from within a CMake script).

    However, explaining this to you has caused me to finally confirm, once and for all, that PropWare must be split into two separate projects: the build system and the C++ source code. I've pondered this for quite some time, and enjoyed the theory of it but never seen a need until now.

    I've created an issue on GitHub for this. It will allow more experienced users (folks who don't mind running an extra Git command and adding another line to their CMake config) to use their unmolested, system-installed CMake installation in PropWare-based projects. All you'll need to do is include PropWare's build system as a git submodule in your project, and then add this line to your root CMake config file somewhere between line 1 and the call to find_package(PropWare)
    list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_LIST_DIR}/CMakeModules")
    
  • DavidZemonDavidZemon Posts: 2,973
    edited 2016-03-23 03:04
    deleted
  • DavidZemonDavidZemon Posts: 2,973
    edited 2016-04-03 06:15
    PropWare version 2.1.0 has been released. Numerous bugs fixed, not all of which made it into the issue tracker. Some of the more prominent features in this release include:

    * New EEPROM class
    * FatFS has learned to fully support multiple partitions
    * Build system improved in numerous ways, partly by settings flags on a per-target basis instead of per-folder
    * Raspberry Pi Debian packages can now be built (though this is considered "unsupported" and they will only be built by request)

    Please checkout out the release's changelog on GitHub and give it a try. https://github.com/parallaxinc/PropWare/releases/v2.1.0

    I can't wait to get started on v3.0 which is going to be another major overhaul of PropWare. See what I have planned here: https://github.com/parallaxinc/PropWare/milestones/Version 3.0
  • The first thing I'm going to do for the v3.0 release is separate the build system from the C++ source. To do this, I need a name and I hate names. This is actually the biggest reason I didn't do it earlier - I have no idea what to call it.

    So please, help me. What do I name the build system that comes with PropWare, if "PropWare" already refers to the C++ source code? PropWare-Build? Anyone have something better or more creative?
  • DavidZemonDavidZemon Posts: 2,973
    edited 2016-04-04 01:07
    v3.0 is moving right along. Here's my current plan for the file reorganization

    propware_tree.jpg

    I'd love to get some outside opinions on this. For anyone wondering, "hmi" is "human-machine-interface" and therefore houses any object that aids with the interaction between a Propeller and a human. I see the input and output folders under hmi growing out of hand some day... so i'm looking for ideas for sub-categories. For output, I could certainly break it down by led/text/graphic. Not sure about input though.

    I'm also not entirely happy with the "utility" folder. It feels like "miscellaneous" and that I don't like. I thought about a folder for data structures (there may be more than just queue some day) but wasn't sure what to call it since "datastructures" is quite long.
  • Looks like you have mostly header files and hardly any .cpp files. Did you put all of the code in the headers?
  • David Betz wrote: »
    Looks like you have mostly header files and hardly any .cpp files. Did you put all of the code in the headers?

    Yes. I try very hard to limit the code that gets compiled into the libraries because of the research SRLM did with libpropeller.
  • David BetzDavid Betz Posts: 14,511
    edited 2016-04-04 02:18
    DavidZemon wrote: »
    David Betz wrote: »
    Looks like you have mostly header files and hardly any .cpp files. Did you put all of the code in the headers?

    Yes. I try very hard to limit the code that gets compiled into the libraries because of the research SRLM did with libpropeller.
    Ugh. It's too bad that it results in slightly smaller code. Everything in one place is something I never liked about Java. It has its advantages I suppose but it's nice to have a clean definition of an interface that isn't cluttered with implementation details.

    What if the program using PropWare is split into multiple separately compiled files that each use the same PropWare classes. Is the code size still reduced? Or does the entire program have to adopt this philosophy and consist of a single main .cpp file that includes everything else as a header?
  • DavidZemonDavidZemon Posts: 2,973
    edited 2016-04-04 03:00
    David Betz wrote: »
    DavidZemon wrote: »
    David Betz wrote: »
    Looks like you have mostly header files and hardly any .cpp files. Did you put all of the code in the headers?

    Yes. I try very hard to limit the code that gets compiled into the libraries because of the research SRLM did with libpropeller.
    Ugh. It's too bad that it results in slightly smaller code. Everything in one place is something I never liked about Java. It has its advantages I suppose but it's nice to have a clean definition of an interface that isn't cluttered with implementation details.

    What if the program using PropWare is split into multiple separately compiled files that each use the same PropWare classes. Is the code size still reduced? Or does the entire program have to adopt this philosophy and consist of a single main .cpp file that includes everything else as a header?

    Haven't tested it, but I would imagine the fewer compilation units the better. Being a Java programmer by day, I actually really like having it all in the header. I rely on my IDE and its autocomplete combined with pop-up documentation to provide everything a standard header would. And no source file means I don't have to keep two different copies of the signatures in sync
  • DavidZemon wrote: »
    David Betz wrote: »
    DavidZemon wrote: »
    David Betz wrote: »
    Looks like you have mostly header files and hardly any .cpp files. Did you put all of the code in the headers?

    Yes. I try very hard to limit the code that gets compiled into the libraries because of the research SRLM did with libpropeller.
    Ugh. It's too bad that it results in slightly smaller code. Everything in one place is something I never liked about Java. It has its advantages I suppose but it's nice to have a clean definition of an interface that isn't cluttered with implementation details.

    What if the program using PropWare is split into multiple separately compiled files that each use the same PropWare classes. Is the code size still reduced? Or does the entire program have to adopt this philosophy and consist of a single main .cpp file that includes everything else as a header?

    Haven't tested it, but I would imagine the fewer compilation units the better. Being a Java programmer by day, I actually really like having it all in the header. I rely on my IDE and its autocomplete combined with pop-up documentation to provide everything a standard header would. And no source file means I don't have to keep two different copies of the signatures in sync
    It seems to me that it's safer to use a language the way it was designed to be used rather than trying to make it into another language. However, I suppose "safer" isn't always better. In any case, it seems you've done a lot of good work on PropWare.

  • DavidZemon wrote: »
    The first thing I'm going to do for the v3.0 release is separate the build system from the C++ source. To do this, I need a name and I hate names. This is actually the biggest reason I didn't do it earlier - I have no idea what to call it.

    So please, help me. What do I name the build system that comes with PropWare, if "PropWare" already refers to the C++ source code? PropWare-Build? Anyone have something better or more creative?

    Keep it simple and just name it 'BuildSystem'. It is what it is.

    Enjoy!

    Mike



  • .
    msrobots wrote: »
    DavidZemon wrote: »
    The first thing I'm going to do for the v3.0 release is separate the build system from the C++ source. To do this, I need a name and I hate names. This is actually the biggest reason I didn't do it earlier - I have no idea what to call it.

    So please, help me. What do I name the build system that comes with PropWare, if "PropWare" already refers to the C++ source code? PropWare-Build? Anyone have something better or more creative?

    Keep it simple and just name it 'BuildSystem'. It is what it is.

    Enjoy!

    Mike

    If that were always within the context of a Propeller project, I'd say great. But that seems pretty... ambitious without context. It sounds like PropWare's build system is The build system. The one and only. The holy grail of build systems. As proud as I am of PropWare, I don't think it's quite that great :D
  • PW-BS?

    nah that sounds not good either.

    Mike
  • Pwmake or pwbuild

    (The only downside is Android autocomplete had trouble swallowing pwmake! )
  • mindrobots wrote: »
    Pwmake or pwbuild

    (The only downside is Android autocomplete had trouble swallowing pwmake! )

    I like PWMake.
    msrobots wrote: »
    PW-BS?

    nah that sounds not good either.

    Mike

    Ha! You made me smile at least, and this has been a rough day, so thank you :D
  • DavidZemonDavidZemon Posts: 2,973
    edited 2016-04-06 03:03
    Great news! I've been working on issue 83, which is to break up the serial classes into independent transmit and receive classes. This is a better separation of concerns and makes the penalties that go along with the "virtual" keyword smaller. By doing this, I've decreased the code size for numerous applications utilizing PropWare's classes. This work can be found on the related feature branch until I fix a little bug still revolving around the COG memory model.

    In fact, I can finally run the following in cog mode (though just barely)
    #include <PropWare/serial/uart/uarttx.h>
    
    const char string[] = "Hello\r\n";
    
    int main () {
        PropWare::UARTTX tx;
        tx.send_array(string, sizeof(string));
    
        return 0;
    }
    

    That beauty takes up 1,952 bytes total. I know... still seems like a lot to just print a string. I'll keep working on it :) but at least it fits now!

    Now I'd like to share some numbers on the following program:
    #ifndef TEST_PROPWARE
    int _cfg_rxpin    = -1;
    int _cfg_txpin    = -1;
    int _cfg_baudrate = -1;
    #endif
    
    int main () {
        uint32_t i = 0;
    
    #ifdef TEST_FDSERIAL
        fdserial *serial = fdserial_open(_cfg_rxpin, _cfg_txpin, 0, _cfg_baudrate);
    #elif defined TEST_LIBPROPELLER
        libpropeller::Serial serial;
        serial.Start(_cfg_rxpin, _cfg_txpin, _cfg_baudrate);
    #endif
    
        while (1) {
    #ifdef TEST_PROPWARE
            pwOut.printf("Hello, world! %03d 0x%02X\n", i, i);
    #elif defined TEST_SIMPLE
            printi("Hello, world! %03d 0x%02x\n", i, i);
    #elif defined TEST_FDSERIAL
            dprinti(serial, "Hello, world! %03d 0x%02x\n", i, i);
    #elif defined TEST_LIBPROPELLER
            serial.PutFormatted("Hello, world! %03d 0x%02X\r\n", i, i);
    #endif
            i++;
            waitcnt(250 * MILLISECOND + CNT);
        }
    
        return 0;
    }
    

    With PropWare v2.1 and a recent version of PropGCC GCC4 from my build server, I got the following code sizes for LMM:

    PropWare: 5,872
    Simple: 4,972
    FdSerial: 6,172
    libpropeller: 6,020

    But with the new PropWare v3.0, I am down to 5,544.
    But wait! If I change PropWare's line to...
    pwOut << "Hello, world!" << i << " 0x";
    pwOut.put_uint(i, 16, 2);
    pwOut << '\n';
    

    I get a measily 4,748, which actually beats Simple for the first time! Now that's not entirely fair, because I'm invoking that special put_uint function and passing extra parameters. So let's simplify the test... to this:
    #ifdef TEST_PROPWARE
            pwOut << "Hello, world! " << i << '\n';
    #elif defined TEST_SIMPLE
            printi("Hello, world! %03d\n", i);
    // ....
    

    PropWare: 4,628
    Simple: 4,952

    Excellent! Now some of you might say that's unfair, because I'm using the entire printi function in Simple which is too big! I should compare apples to apples and use Simple's putStr, putInt, and putChar. To you, I say bogus. For easy printing, the intention was to use print or printi for Simple and << for PropWare. These are near equivalents. But alas... just to satisfy curiosity...
    putStr("Hello, world! ");
    putDec(i);
    putChar('\n');
    

    Produces an output of 4,428. Only a couple hundred bytes smaller than PropWare's << operator, but no where near as pretty!

    I'd say I done gooder :D
  • DavidZemonDavidZemon Posts: 2,973
    edited 2016-04-08 02:59
    This is excellent! I've shrunk the Printer class even more and have officially beaten Simple for size using my Hello demo. The new code now looks like this:
    #ifdef TEST_PROPWARE
            pwOut << "Hello, world! " << Printer::Format(3, '0') << i << " 0x" << Printer::Format(2, '0', 16) << i << '\n';
    #elif defined TEST_PROPWARE_PRINTF
            pwOut.printf("Hello, world! %03d 0x%02X\n", i, i);
    #elif defined TEST_SIMPLE
            printi("Hello, world! %03d 0x%02x\n", i, i);
    // .....
    

    Similar to standard C++'s setw()/setfill()/etc functions, PropWare now supports modifying the format via the << operator. This means I can replicate all the same functionality provided in Simple's printi without having to resort to the hefty printf function. Printer::printf is still available (and I added a 'b' format char for printing binary), but the user now has complete control via the leaner << operator.

    Code Sizes

    For the above three cases, the code sizes come in at...

    TEST_PROPWARE: 4,872
    TEST_PROPWARE_PRINTF: 5,680
    TEST_SIMPLE: 4,972

    You can see PropWare's printf function grew relative to a couple nights ago by about 140 bytes, but I think that's well worth the benefits provided by the change (and it includes the 'b' char).

    Of course, all this and we're still just talking about a few hundred bytes - not too big of a deal relative to 32k. But I've been trying to shrink smaller than Simple for quite a while, and am thrilled that I've managed to do it while still retaining the full power of the Printer class.

    In other news, PropWare::UARTTX can now run in COG mode! FINALLY.

    This program requires 1,348 bytes...
    #include <PropWare/serial/uart/uarttx.h>
    
    int main () {
        PropWare::UARTTX tx;
    
        tx.puts("Hello\r\n");
    
        while (1);
    }
    




    And this little guy comes in at 1,804 bytes.
    #include <PropWare/serial/uart/uarttx.h>
    #include <PropWare/serial/uart/uartrx.h>
    
    int main () {
        PropWare::UARTTX tx;
        PropWare::UARTRX rx;
    
        while (1) {
            tx.put_char(rx.receive());
        }
    }
    

    Both are still larger than most native PASM drivers out there, but this is a great start. And now I can use these classes to write COG drivers for buffered UART implementations.
Sign In or Register to comment.