LLVM Backend for Propeller 2

124»

Comments

  • DavidZemonDavidZemon Posts: 2,900
    edited 2020-08-21 - 13:09:56
    kuba wrote: »
    For anyone building anything using cmake: please forget that gnu make exists. It's highly counter-productive. Use ninja everywhere. It's the defacto modern build tool. Once you experience its performance, going back to gnu make will sound and feel ridonkulous. make tries to be a general purpose tool, and its code base and performance bears that fact. ninja tries to do one job only and does it extremely well.`cmake -G Ninja` all the way. Thank me later :) Yes, you need to install ninja.

    I've tried Ninja before. It made an insignificant difference in both of the two projects I tried it on.
  • DavidZemon wrote: »
    kuba wrote: »
    For anyone building anything using cmake: please forget that gnu make exists. It's highly counter-productive. Use ninja everywhere. It's the defacto modern build tool. Once you experience its performance, going back to gnu make will sound and feel ridonkulous. make tries to be a general purpose tool, and its code base and performance bears that fact. ninja tries to do one job only and does it extremely well.`cmake -G Ninja` all the way. Thank me later :) Yes, you need to install ninja.

    I've tried Ninja before. It made an insignificant difference in both of the two projects I tried it on.

    Nor do I think the exact build system you use matters here (and isn't relevant to this discussion)... I used Ninja at work, couldn't really tell a difference in what it actually does vs make and I know make. But if anyone wants to use Ninja instead, feel free to! Just change the generator type when running cmake, as kuba wrote above.
  • Been quiet so quick update: I’ve rewritten how instructions are formatted to support conditions (if_z, etc) and effect flags (wz, etc) as operands to each instruction. This has created a unique thing situation that hasn’t been tested in LLVM before: instructions where the instruction mnemonic doesn’t come first and there are optional operands (the effect flag). As a result, something in the assembly parser isn’t happy. I think this is an actual bug in LLVM and I’ve isolated to where it happens, I just need to figure out a good fix. Hopefully this weekend, I’ll be able to post next version which fixes that, as well as supports more of the c library, particularly dynamic memory allocation (with locks and such)
  • very cool. good find :)
  • Wait, doesn't x86 have instruction prefixes (REP, LOCK, etc), too?
  • Wuerfel_21 wrote: »
    Wait, doesn't x86 have instruction prefixes (REP, LOCK, etc), too?

    Maybe, I don't know x86, but I do the know the x86 backend in LLVM is HUGE and has lots of custom parsing code, so it's likely the target-specific code handles this rather than the LLVM internals. The constructs that I'm using for having optional arguments AND instruction prefixes seems to not work. Anyway, I know where to fix the problem, just need to figure out how.
  • I wonder if there is a way to treat each variation of the instruction with different trailing argument (like wz, wc, wcz) more like just 3 new/different instructions, one of which you will choose to emit depending on what needs to be done.
  • rogloh wrote: »
    I wonder if there is a way to treat each variation of the instruction with different trailing argument (like wz, wc, wcz) more like just 3 new/different instructions, one of which you will choose to emit depending on what needs to be done.

    Yes that’s totally possible, and is how I initially did it for a few specific instructions, but would be a bit of a headache to write all those out for every instruction--i've got them working as operands, just need to clean a few things up.
  • latest version of compiler and library: https://github.com/ne75/p2llvm/releases/tag/v0.3, implementing things from above with instruction/effect flags. Next going to keep expanding c standard library functionality and adding instructions as needed.
  • Fwiw, this LLVM backend could also open the door to support for Go (golang) via TinyGo. I went ahead and made this to help track the work involved: https://github.com/tinygo-org/tinygo/issues/1336

    Is there any info on what would be required to get this fork upstreamed into the main LLVM repo? Perhaps there a discussion that should be started with the LLVM maintainers on this? Just from a cursory check it seems like they do contributions via patches http://llvm.org/docs/Contributing.html#how-to-submit-a-patch

    @DavidZemon made this suggestion near the top of the thread and I think it's an import aspect of helping make this effort successful. It might be worth submitting a patch of the current code to the main LLVM project to see what their feedback is?
  • bgp wrote: »
    Fwiw, this LLVM backend could also open the door to support for Go (golang) via TinyGo. I went ahead and made this to help track the work involved: https://github.com/tinygo-org/tinygo/issues/1336

    Is there any info on what would be required to get this fork upstreamed into the main LLVM repo? Perhaps there a discussion that should be started with the LLVM maintainers on this? Just from a cursory check it seems like they do contributions via patches http://llvm.org/docs/Contributing.html#how-to-submit-a-patch

    @DavidZemon made this suggestion near the top of the thread and I think it's an import aspect of helping make this effort successful. It might be worth submitting a patch of the current code to the main LLVM project to see what their feedback is?

    I agree, I do want to get it into mainline LLVM, but I think that requires a) it be more complete, b) have the C std library more implemented than it currently is (which I'm working on) and c) someone else to help me push that effort forward. My branch of this is now a proper fork so we can merge in new LLVM features as they are made before putting in the patch request. I anticipate only 1 place where I made a bandaid fix in the actual LLVM code that needs to be properly addressed, but the rest I think would be able to be merged in without issue.

    In other news, the FILE driver interface from propgcc has been ported over to the dev branch and it seems to work. I haven't tested it much, but I modified the SimpleSerialDriver with smart pins to be used as the default for stdin/stdout/stderr as it was on P1.
  • This sounds really good @n_ermosh.

    So once it is good shape will there be distributable binaries for the different plaforms in the end? After all the efforts I did to try to build it and ultimately failing myself with Yosemite, I guess some people won't easily be able to use these tools otherwise, depending on their systems. Basically will anyone who wants to use LLVM for P2 always need to build their own toolchain, or once it is part of mainline LLVM does that mean we will get binaries (e.g. for MacOS X)?

    It sort of appears that should be the case...
    https://releases.llvm.org/download.html
  • rogloh wrote: »
    This sounds really good @n_ermosh.

    So once it is good shape will there be distributable binaries for the different plaforms in the end? After all the efforts I did to try to build it and ultimately failing myself with Yosemite, I guess some people won't easily be able to use these tools otherwise, depending on their systems. Basically will anyone who wants to use LLVM for P2 always need to build their own toolchain, or once it is part of mainline LLVM does that mean we will get binaries (e.g. for MacOS X)?

    It sort of appears that should be the case...
    https://releases.llvm.org/download.html

    Yes I fully intend to provide a binary you can directly use without compiling anything, although it will be tough to support older OS’s if I’m running newer ones with newer libraries. Each tag I have on GitHub current contains the binaries I compiled on macOS, are you able to run those? If not, I’ll need to figure out a way to make it compatible with older versions of macOS/OS X, but that won’t be as high a priority for me.
  • I also have no intention of taking down my CI server any time soon, so you'll be able to find Linux binaries there for the foreseeable future. My guess is, since the build system is CMake, that I could pretty easily cross-compile for Windows as well (as opposed to other projects where that has historically taken a significant development effort).
  • DavidZemon wrote: »
    I also have no intention of taking down my CI server any time soon, so you'll be able to find Linux binaries there for the foreseeable future. My guess is, since the build system is CMake, that I could pretty easily cross-compile for Windows as well (as opposed to other projects where that has historically taken a significant development effort).

    Which branch does your CI server pull from? master? And do you mind if I just pull the binaries you build and include them in my tags?

    For windows, I think I’ll leave that effort to someone else who really wants it. I haven’t written code on Windows in like 7-8 years and am super unfamiliar with how to port these things over
  • n_ermosh wrote: »
    DavidZemon wrote: »
    I also have no intention of taking down my CI server any time soon, so you'll be able to find Linux binaries there for the foreseeable future. My guess is, since the build system is CMake, that I could pretty easily cross-compile for Windows as well (as opposed to other projects where that has historically taken a significant development effort).

    Which branch does your CI server pull from? master? And do you mind if I just pull the binaries you build and include them in my tags?

    For windows, I think I’ll leave that effort to someone else who really wants it. I haven’t written code on Windows in like 7-8 years and am super unfamiliar with how to port these things over

    It builds both dev and master branches. And you're most welcome to pull binaries direct from the CI server, or link to them, whatever you'd like.

    I tried a Windows build... it died pretty quickly. Maybe cross compiling with clang would be better than MinGW. An idea for another time.
  • @n_ermosh, current dev branch is failing on the CI server. Does it build for you? Maybe this is another GCC v. LLVM difference, and it would work if I built it with clang instead?

    Build failure: https://ci.zemon.name/buildConfiguration/P2llvm_LlvmLinuxX64/7792?guest=1&showLog=7792_4356_4267.4356
  • n_ermoshn_ermosh Posts: 107
    edited 2020-09-03 - 00:38:02
    That looks like a bug I introduced. I’ll try a scratch build and let you know if I can fix it.
  • Yep--I pulled in upstream LLVM commits but never tested it and they changed some stuff. Fixed now, let me know if your build still breaks.
  • Sorry for the long delay. Yes, it built successfully now :)
    There was a failed build when i woke up this morning, but that's because I had forgotten to restrict the "linux" build configuration to Linux agents. So it tried to build on a Mac agent and failed... what a surprise :P
  • Sorry for the quiet, work has been super busy lately.

    I'm currently working on setting up several example programs to demonstrate the various features, especially those related to LLVM. One of these will be my LVDS driver, which I can just compile with fastspin and include a binary since it's all assembly anyway, but I want to see how close I can get LLVM's assembly parser to mimic fastspin's assembly parser to keep it all in one tool. Not the end of the world if it doesn't work.

    I also emailed the LLVM devs about merging this into mainline LLVM, haven't heard back, but from what I've read online, one thing they desire is unit tests. I have no idea how to go about writing unit tests for this, especially without a solid P2 emulator that can be used to verify the code runs as expected in a controlled environment with software feedback of the chip's state. If anyone has ideas, let me know. I started thinking about writing a Python-based P2 emulator, but quickly decided against that much scope creep on this project.
  • You probably first want to write simple unit tests that exercises most C functionality to prove the generated code works without obvious error. You may have already done a lot of that. How stringent they would want their unit tests to be I'm not sure. You could possibly write expect scripts and have something automated that uses printf with output you can match on to tell that the code did the correct thing.

    for loops
    if
    case statements
    goto
    function calls
    function pointers
    assignment statements
    pointer increment/decrements
    expressions
    etc...

    Also some heap allocation stuff would be good to test.

    It could just about go on endlessly though if you wanted to test all possible functionality and need to look for memory corruption or modifications outside of valid register areas etc. Just start simple and build up over time.
  • I'd be figuring what unit tests look like for other embedded systems that llvm supports. Without knowing what those look like though, I might hand write an assembly file and verify that clang generates the right listing. Then somehow acquire a valid binary (fastspin? Trial and error with llvm? Whatever) and verify that file matches what llvm's assembler outputs. Since llvm isn't responsible for loading into the p2, I would say your tests don't need to verify anything past writing the correct file to disk.
Sign In or Register to comment.