Merging P2LLVM into upstream: what we need
I'm getting to a point with p2llvm that I want to submit the target in the LLVM codebase to the upstream repo, so we can regularly take advantage of updates to LLVM and clang without it constantly breaking the target code. I sent an email to the LLVM folks with a series of questions and requirements, which I've linked below. There's quite a few things to do still in terms of testing and cleanup, but one major one that is needed is "active community support". I intend to keep supporting the project, but it will be difficult to do it entirely alone, and the llvm team would also want more than one person who is able to support it, run and maintain tests, etc.
Is anyone willing to sign up to be a code owner for this effort? My hope is that the effort is minimal, and I will still take the bulk of the work, but having more than one person the llvm team can reach out to would be good for getting this supported long term.
My hope is that if we get this submitted into the upstream repo, we can have a stable, modern, and complete C++ toolchain option for P2, the community can put most effort into developing libraries and hardware support code, rather than trying to solve compiler problems. There's still plenty of features I want to add, but for the most part, the target is working and you can compile most programs you would want.
Discussion with LLVM: https://groups.google.com/g/llvm-dev/c/QJgTO9vBIqU
Comments
What is the current status of p2llvm? Can I use it to compile arbitrary C/C++ code for the P2? I’ve been inactive for a while and didn’t know this project had gotten to a usable state. How good is the code generation compare to Eric’s compiler and Catalina?
Nikita, congrats again on all this work. Well done.
Looks like a lot of work is still needed - both in testing, and breaking the change into digestable patches.
And finding a community to get behind it.
I wonder if merging into upstream GCC was similarly difficult, and why Parallax never helped to make it happen.
If I ever get back into Propeller things as a hobby, I'll let you know and I'll volunteer to help if there's still more to help with.
Yes, you can compiler arbitrary C/C++ with it. I haven't tested across all C++ features so maybe there's still weird corners that break, but for most things, it does work, including (very basic) floating point support. Things I have tried (with basic tests/examples):
I ported over libc from propgcc to get myself going with the C library, but it's just the bare minimum I needed to get going. I ported over compiler-rt from LLVM for the floating point support.
I haven't included all floating point builtins yet, I need to create some test cases that actually use them all, but they are implemented. The things that needs more work (and is near the top of my list) is running native cog code. currently, you can happily load a function or binary to a cog directly, but you can't load multiple functions into the cog RAM that can call each other.
I don't have clear benchmarks on comparing to Catalina or Eric's compiler system, if you know of good benchmark programs that people have been using, please let me know so I can do a comparison and I'll post results in another thread. It'll be good to test code I didn't write, too. There's probably a lot of performance improvements to be made (optimizing loops to use rep, instruction skipping, block moves, etc), but glancing at assembly for various programs compiled using -Oz optimization, it looks fairly good.
Yes, learning the LLVM test system is my next step. I'm hoping that going through that process will inherently help me break things apart since the tests will be focused around testing individual components.
There's an FFT benchmark that's been used (partially because it exists as both Spin and C), not sure if I have it laying around. @ersmith has it for sure.
I think I found it: https://github.com/ZiCog/fftbench/blob/master/fftbench.c. Will start a new thread to discuss the results.
I for one are all for a C++ compiler. Most of the people are SPIN coders so I don't know how much traction you're going to get. In fact, there are no repos for C libraries for the P2.
I have converted most of the hardware drivers I use on the P1 over to the P2 and found it to be straight forward using Spin2cpp. Having another C computer though, only muddies the waters for the few users that there are.
I would be very interested in this project and willing to help if I can.
MIke
IF there is any real interest, I might be convinced to update PropWare to support LLVM & P2. But I'd probably need/ask for help converting many of the core serial routines to use the P2's hardware peripherals instead of bit-banging.
I'd say because FlexC is the main C compiler and it has full Spin interop, people don't write C libraries, because it is more useful to write them in Spin.
In theory you should be able to spin2cpp any spin library into a C/C++ one for use with any compiler, but IDK how well that works on P2.
Sounds good. What is the link for downloading it? Will it build on the Mac?
It's good to hear FlexC is the main C compiler for the P2. I'm glad Eric is getting some support. I haven't tried it lately. Is there any solution to what to do about programs that require separate compilation and libraries? Last I knew everything had to be compiled at once. Is that still true?
Still like that and unlikely to change. Not sure when that'd ever be a problem. LTO in the big compilers is effectively the same thing.
Clone here: https://github.com/ne75/p2llvm
I develop on Big Sur so I know it builds there for sure. Quick start instructions:
Compile anything as:
Load and run with loadp2. Currently, patching in clock modes doesn't work, it will start with defaults and then should be changed in software if necessary.
I'll get a CI build system set up eventually to make both macOS and Linux builds.
I already can tell you there are issues building fftbench.c. It compiles and runs fine (with tweaks to get the correct includes and clock setup) but gives incorrect results. Digging into that now.
I noticed I was having artifact upload problems a while ago, but finally am digging into it now and will get it resolved hopefully tonight, but definitely soon. Apparently the p2llvm builds are taking an hour now
(they run in a VM that is allowed to use half of my AMD 3700x CPU) so I've triggered another build but won't know if it succeeds until I wake up tomorrow.
I had no idea you developed on a Mac. If you're interested in working with me to set up a build agent on your machine, we could configure my CI server to send Mac builds to you and then you'd have Mac binaries available for download.
Any idea what might have gone wrong here? I'm building under OS X Monterey.
[100%] Building CXX object tools/lld/tools/lld/CMakeFiles/lld.dir/lld.cpp.o
[100%] Linking CXX executable ../../../../bin/lld
[100%] Built target lld
Traceback (most recent call last):
File "/Users/dbetz/Dropbox/p2/p2llvm/build.py", line 183, in
main();
File "/Users/dbetz/Dropbox/p2/p2llvm/build.py", line 166, in main
r = build_libp2(build_dir, llvm, clean)
File "/Users/dbetz/Dropbox/p2/p2llvm/build.py", line 79, in build_libp2
p = subprocess.Popen(['make', 'LLVM=' + llvm], cwd=os.path.join(LIBP2_DIR, 'build'))
File "/Users/dbetz/.pyenv/versions/3.10.0/lib/python3.10/subprocess.py", line 966, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "/Users/dbetz/.pyenv/versions/3.10.0/lib/python3.10/subprocess.py", line 1842, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'libp2/build'
I see that flexcc does support a -c option to compile an individual file into a ".o" file.
Flexc is just like Arduino compiles and libraries. Everything gets compiled at the same time. No archives.
Mike
An h
That would be great, though I use 2 Macs. my Macbook (Intel) and a Mac Mini (M1). Are you able to build for both architectures? > @"David Betz" said:
Yeah there's a mistake in the build script--I never actually added the code to configure the build directory for
libp2. Fixed now, so pull latest from p2llvm master. Manual fix is:At some point, I should probably make everything build with one top level cmake project, but that's a problem for a different day.
Yep, shouldn't be a problem at all. The build agent is a Java app, so assuming someone has made a JVM for the M1, it should work great. At least.... that's the theory
I got close this time but I still ran into an error:
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/dbetz/Dropbox/p2/p2llvm/libp2/build
[ 3%] Building C object lib/builtins/CMakeFiles/builtins.dir/adddf3.c.obj
[ 6%] Building C object lib/builtins/CMakeFiles/builtins.dir/addsf3.c.obj
[ 9%] Building C object lib/builtins/CMakeFiles/builtins.dir/ashldi3.c.obj
[ 12%] Building C object lib/builtins/CMakeFiles/builtins.dir/ashrdi3.c.obj
[ 15%] Building C object lib/builtins/CMakeFiles/builtins.dir/comparedf2.c.obj
[ 18%] Building C object lib/builtins/CMakeFiles/builtins.dir/comparesf2.c.obj
[ 21%] Building C object lib/builtins/CMakeFiles/builtins.dir/divdi3.c.obj
[ 24%] Building C object lib/builtins/CMakeFiles/builtins.dir/divdf3.c.obj
[ 27%] Building C object lib/builtins/CMakeFiles/builtins.dir/divsi3.c.obj
[ 30%] Building C object lib/builtins/CMakeFiles/builtins.dir/fixunsdfdi.c.obj
[ 33%] Building C object lib/builtins/CMakeFiles/builtins.dir/fixunsdfsi.c.obj
[ 36%] Building C object lib/builtins/CMakeFiles/builtins.dir/fixunssfdi.c.obj
[ 39%] Building C object lib/builtins/CMakeFiles/builtins.dir/fixunssfsi.c.obj
[ 42%] Building C object lib/builtins/CMakeFiles/builtins.dir/floatundidf.c.obj
/Users/dbetz/Dropbox/p2/p2llvm/libp2/lib/builtins/floatundidf.c:22:10: fatal error: 'stdio.h' file not found
include <stdio.h>
1 error generated.
make[2]: *** [lib/builtins/CMakeFiles/builtins.dir/floatundidf.c.obj] Error 1
make[1]: *** [lib/builtins/CMakeFiles/builtins.dir/all] Error 2
make: *** [all] Error 2
Ah that's due to a circular dependency--I never noticed cause I had libc already installed. Fixed. Found a couple of other errors that arise only on clean builds I also fixed. I should have CI set up by the end of the day on David Z's server so you don't need to build from scratch.
Now I get this:
[ 81%] Built target builtins
[ 84%] Building C object lib/CMakeFiles/p2.dir/crt0.c.obj
/Users/dbetz/Dropbox/p2/p2llvm/libp2/lib/crt0.c:5:10: fatal error: 'propeller2.h' file not found
include "propeller2.h"
1 error generated.
make[2]: *** [lib/CMakeFiles/p2.dir/crt0.c.obj] Error 1
make[1]: *** [lib/CMakeFiles/p2.dir/all] Error 2
make: *** [all] Error 2
oops, I fixed that but never pushed it... fixed now
That did it. The build completed successfully and I was able to build a hello world program. Now I have to find my P2 eval board to see if it works. Thanks for your help!
I compiled this program and loaded it into my P2 Eval RevC board and the load seemed to complete but I didn't see any output. Do I need to set the baud rate to something other than the default? I also tried loading it on a P2 Edge JonnyMac board and a RevA Edge module with the same results.
#include <stdio.h> int main(void) { printf("Hello, world!\n"); while (1) ; return 0; }I also tried the loadp2 command with -b 230400 and still saw no output. I originally didn't have the while loop after the printf but I thought maybe the output buffer wouldn't be flushed before the program exited.
You need to configure clock and serial to match the host side. Check out this example: https://github.com/ne75/p2llvm/blob/master/test/src/t1.cpp
For reference, if you don't change anything, the boot clock setup is used (rcfast, ~24MHz) and serial baud rate is set to 3Mbps. Patching in other options at load time is not yet supported, but can be changed to anything at runtime.
This opens up a new discussion of what should be considered "default", so that the application code can be as decoupled as possible from the actual hardware configuration. i.e. the above program to print hello world should run on any board without extra setup.
For all of my C applications I settled on 200mhz as the default speed and have it set in the compiler that way.
For baud rate it's not as simple, Arduino and other platforms seem to like 115200, I am using 230400.
Mike
Okay, setting up the clock speed and baud rate explicitly seems to work:
#include <stdio.h> #define P2_TARGET_MHZ 200 #include <sys/p2es_clock.h> int main(void) { _clkset(_SETFREQ, _CLOCKFREQ); _uart_init(DBG_UART_RX_PIN, DBG_UART_TX_PIN, 115200); printf("Hello, world!\n"); while (1) ; return 0; }It would be kind of nice to be able to run a standard C program without having to add that code though.
I see this command will work:
3Mbps seems amazingly fast to me. I remember thinking that my Hayes Smartmodem that ran at 300bps was blindly fast when coming from an acoustic coupler that ran at 110bps!
It does work like that in theory. If you don't do any clock or serial set up, you should be able to printf over serial, just configure the host to 3Mbps. The load command you wrote should work. I don't know how well that will work across boards though given the variability in the RC clock.
Welcome to the future
. 3M seems to be the fastest stable speed, I don't think it makes sense to limit it to anything slower since it's a debugging interface.
It would be even nicer to be able to patch the clock at load time so that it doesn't need to be configured at runtime. loadp2 does this, BUT the addresses it writes to intersect with the entry function. So I'd need to add an intermediate one to skip over those addresses to keep things consistent across all compilers, or modify loadp2 to be able to dynamically change which addresses are written to. The former seems like the more correct option, but it's a little bit messier, since that would mean there are multiple startup functions that just keep skipping addresses
Not really. 2M (the default debugging speed in Spin2) already drops data sometimes, at least with a standard FTDI USB serial port under Windows. 1M is rock solid though.
I've been lurking here for years, and have lost plenty of sleep wishing I could dive into LLVM so I could get Rust running on the P2. I know more than one of you have expressed the same wish.
If/when I get time, I'd love to help with the unit tests (C/C++) to support P2 LLVM being upstreamed.
An addition to the Guide to Rustc Development mentioned by @DavidZemon, Rust has an incredibly supportive community. I've been lurking on Rust's Zulip chat system. Most newcomers wanting to port Rust to new targets are invited to check out the dev guide and the #t-compiler/wg-llvm stream.