Building PropGCC with Docker
DavidZemon
Posts: 2,973
I really like this Docker thing. The crowds were right - it's a cool tool when used in the right places.
I've created a new Docker image (davidzemon/propgccbuilder - https://hub.docker.com/r/davidzemon/propgccbuilder/) which is capable of building PropGCC from source. I've so far tested the following:
Using David Betz's repository: https://github.com/dbetz/propeller-gcc
- master branch only
GCC 4
- Linux x86-64
- Windows 32-bit
- ARMv6 (Raspberry Pi)
GCC 6
- Linux x86-64
- Windows 32-bit
- ARMv6 (Raspberry Pi)
I have not tested the old version of PropGCC from Parallax's repository (https://github.com/parallaxinc/propgcc) and I do not know how to use Docker to build for Mac (or if that's even possible).
Okay, so you want to try this for yourself?
System Requirements
Docker
Git
Setup (Written for Linux, tweak for Windows where necessary)
(For Windows, check out @iseries reply to this thread: https://forums.parallax.com/discussion/comment/1455187/#Comment_1455187)
Start by cloning PropGCC and entering the Docker container
Note that the "-u $(id -u):$(id -g)" is probably not necessary for Windows users. Excluding that argument on Linux will yield a bunch of files owned by root and that'd just be a pain wouldn't it?
At this point, you are now inside the Docker container and should be able to follow the instructions in the README. The important part about doing this in a Docker container is that all system requirements have already been installed, namely: a version of GCC that doesn't throw any errors when building GCC 4.x, MinGW, ARMv6 toolchain, texinfo.
Building PropGCC for Linux
The following assumes you are in the Docker container as configured above
At this point, you'll be dropped to your host command prompt and PropGCC with GCCv4 for 64-bit Linux will have been installed to propgcc-parent/propgcc4-linux-bin. You can then move that wherever necessary on your host box.
Building PropGCC with GCCv6 is almost identical of course:
Cross-Compiling PropGCC
Note for cross-compiling PropGCC, you MUST have already compiled (or downloaded) PropGCC for Linux (even if your host computer is Windows, the Docker image is always Linux and therefore you must have compiled PropGCC for Linux). The Linux version of PropGCC must also be based off of the same version of GCC as you are wanting to cross-compile. If you now want to build PropGCC with GCCv4 for Windows, then you must first build/download PropGCC with GCCv4 for Linux. If you want to build PropGCC with GCCv6 for Raspberry Pi, then you must first build/download PropGCC with GCCv6 for Linux.
So, let's assume you want to build PropGCC with GCCv6 for Windows and you've already built PropGCC with GCCv6 for Linux, as described above. We're starting at your host command prompt in the propgcc-parent directory, not from within the Docker container:
Voila! In propgcc-parent/propgcc-win32-bin, you now have Windows binaries for PropGCC built with the latest GCC 6 (or whatever version of GCC it is at the time of you reading this).
Some notes on cross-compiling:
The key here is the extra argument to Docker: "-v `pwd`/propgcc-linux-bin:/opt/parallax:ro". This has four parts:
"-v" is short for "--volume", and it says that a directory on your host computer should be available in the Docker container
The value is then split into three parts with the colons:
"`pwd`/propgcc-linux-bin" means that the folder on our host computer which is going to be available in Docker is the PropGCC binaries. This is the ONLY configurable part. You can change this part of the argument to whatever you need, based on wherever you have installed/downloaded/compiled PropGCC.
"/opt/parallax" is the target directory in the Docker container. One of the steps to creating the Docker image was adding "/opt/parallax/bin" to $PATH, which is why things "just work" with this arg.
"ro" means read only. Not entirely necessary, but a good safety measure that prevents accidental mucking around with your working PropGCC binaries.
I do not expect this to be used by the masses. The masses should just download whichever binaries they need either from Parallax directly or from my build server (https://ci.zemon.name?guest=1). However, if you want to play around with compiling PropGCC, this should make it much more reliable than trying to figure out compatible versions of GCC, since newer versions of GCC (such as what ships in Ubuntu 17.10) throw errors when trying to build PropGCC.
That's all folks! Let me know if you have questions.
I've created a new Docker image (davidzemon/propgccbuilder - https://hub.docker.com/r/davidzemon/propgccbuilder/) which is capable of building PropGCC from source. I've so far tested the following:
Using David Betz's repository: https://github.com/dbetz/propeller-gcc
- master branch only
GCC 4
- Linux x86-64
- Windows 32-bit
- ARMv6 (Raspberry Pi)
GCC 6
- Linux x86-64
- Windows 32-bit
- ARMv6 (Raspberry Pi)
I have not tested the old version of PropGCC from Parallax's repository (https://github.com/parallaxinc/propgcc) and I do not know how to use Docker to build for Mac (or if that's even possible).
Okay, so you want to try this for yourself?
System Requirements
Docker
Git
Setup (Written for Linux, tweak for Windows where necessary)
(For Windows, check out @iseries reply to this thread: https://forums.parallax.com/discussion/comment/1455187/#Comment_1455187)
Start by cloning PropGCC and entering the Docker container
> mkdir propgcc-parent > cd propgcc-parent > git clone --recursive https://github.com/dbetz/propeller-gcc.git > docker pull davidzemon/propgccbuilder > docker run -it -u $(id -u):$(id -g) -v `pwd`:`pwd` -w `pwd` davidzemon/propgccbuilder
Note that the "-u $(id -u):$(id -g)" is probably not necessary for Windows users. Excluding that argument on Linux will yield a bunch of files owned by root and that'd just be a pain wouldn't it?
At this point, you are now inside the Docker container and should be able to follow the instructions in the README. The important part about doing this in a Docker container is that all system requirements have already been installed, namely: a version of GCC that doesn't throw any errors when building GCC 4.x, MinGW, ARMv6 toolchain, texinfo.
Building PropGCC for Linux
The following assumes you are in the Docker container as configured above
> cd propeller-gcc/gcc > ./contrib/download_prerequisites > cd .. > make GCCDIR=gcc4 -j4 > make GCCDIR=gcc4 INSTALL=`pwd`/../propgcc4-linux-bin install > exit("-j4" can be changed for your rig, but serves as a good default)
At this point, you'll be dropped to your host command prompt and PropGCC with GCCv4 for 64-bit Linux will have been installed to propgcc-parent/propgcc4-linux-bin. You can then move that wherever necessary on your host box.
Building PropGCC with GCCv6 is almost identical of course:
> cd propeller-gcc/gcc > ./contrib/download_prerequisites > cd .. > make GCCDIR=gcc -j4 > make GCCDIR=gcc INSTALL=`pwd`/../propgcc-linux-bin install > exit
Cross-Compiling PropGCC
Note for cross-compiling PropGCC, you MUST have already compiled (or downloaded) PropGCC for Linux (even if your host computer is Windows, the Docker image is always Linux and therefore you must have compiled PropGCC for Linux). The Linux version of PropGCC must also be based off of the same version of GCC as you are wanting to cross-compile. If you now want to build PropGCC with GCCv4 for Windows, then you must first build/download PropGCC with GCCv4 for Linux. If you want to build PropGCC with GCCv6 for Raspberry Pi, then you must first build/download PropGCC with GCCv6 for Linux.
So, let's assume you want to build PropGCC with GCCv6 for Windows and you've already built PropGCC with GCCv6 for Linux, as described above. We're starting at your host command prompt in the propgcc-parent directory, not from within the Docker container:
> docker run -it -u $(id -u):$(id -g) -v `pwd`:`pwd` -v `pwd`/propgcc-linux-bin:/opt/parallax:ro -w `pwd` davidzemon/propgccbuilder > cd propeller-gcc > make GCCDIR=gcc CROSS=win32 -j4 > make GCCDIR=gcc CROSS=win32 INSTALL=`pwd`/../propgcc-win32-bin install > exit
Voila! In propgcc-parent/propgcc-win32-bin, you now have Windows binaries for PropGCC built with the latest GCC 6 (or whatever version of GCC it is at the time of you reading this).
Some notes on cross-compiling:
The key here is the extra argument to Docker: "-v `pwd`/propgcc-linux-bin:/opt/parallax:ro". This has four parts:
"-v" is short for "--volume", and it says that a directory on your host computer should be available in the Docker container
The value is then split into three parts with the colons:
"`pwd`/propgcc-linux-bin" means that the folder on our host computer which is going to be available in Docker is the PropGCC binaries. This is the ONLY configurable part. You can change this part of the argument to whatever you need, based on wherever you have installed/downloaded/compiled PropGCC.
"/opt/parallax" is the target directory in the Docker container. One of the steps to creating the Docker image was adding "/opt/parallax/bin" to $PATH, which is why things "just work" with this arg.
"ro" means read only. Not entirely necessary, but a good safety measure that prevents accidental mucking around with your working PropGCC binaries.
I do not expect this to be used by the masses. The masses should just download whichever binaries they need either from Parallax directly or from my build server (https://ci.zemon.name?guest=1). However, if you want to play around with compiling PropGCC, this should make it much more reliable than trying to figure out compatible versions of GCC, since newer versions of GCC (such as what ships in Ubuntu 17.10) throw errors when trying to build PropGCC.
That's all folks! Let me know if you have questions.
Comments
The free version is all you need. However, if you're on a Debian-based distro, simply "apt-get install docker.io" will install it. You likely want to add yourself to the "docker" group and reboot though, else you'll need to run docker with sudo every time.
Yes - the whole idea of docker is that anywhere Docker is installed, the images will be identical, whether I'm running Linux and you're on Mac, or I'm on Linux and you're on some custom OS (good luck getting Docker installed on your custom OS though :P ).
The base Ubuntu images in Docker are extremely stripped down, so no. Basically what you get is the Ubuntu folder structure, apt-get, and the Ubuntu mirrors configured for apt-get. Aside from that... not much more than ls and cp :P. What I've added to it can be seen from my Dockerfile: https://github.com/DavidZemon/PropGCCBuilder/blob/master/Dockerfile
Eric
You do not have to repeat the steps each time. Any file changed in the mounted volume (everything under propgcc-parent) will remain after the container exits. So, the downloaded prereqs and all the build output will still be there when you launch the next container and remount the same volume.
Glad you like it! I'm really enjoying being able to run the latest and greatest bleeding edge OS while still being able build old apps. Docker definitely helps to solve the legacy software problem.
OH NO! Don't do that! Linux is the answer. Second only to 42, of course.
WSL is much lighter weight than a VM
The WSL has some advantages:
As noted above it is much lighter than a VM.
The whole Windows machine is available as /mnt/c so no messing with setting up shares.
No messing with setting up networking.
Cutting and pasting just works.
The WSL has some disadvantages:
It does not have a Linux kernel under it. So anything that needs that is off the table. A recent annoying example of that is that one cannot generate core dumps.
No X Windows. I guess one could get a Windows X server running for that. Generally I don't need X Windows.
I still have Cygwin installed. It's getting less and less use...
Thanks for sharing, I'll give docker a try.
I'm currently compiling PropGCC with Ubuntu 16.04 and gcc 5.4.0 without problems, I don't remember if I have applied a patch for it. If interested I can try to build the original sources and see what happens.
Also, I have added targets for 64 bit Windows. I don't have a PC with 64 bit Windows to test but compile works fine. If interested the patches are for the makefiles to use the 64 bit version of mingw cross compiler.
I always use dbetz/propeller-gcc but should be applicable for the parallax repository too, they only add ifeq CROSS... lines to set the cross compilers.
I can try, does it work for the submodules ? The patch involves all tools.
I'm attaching it here, just in case.
And, did you build as gcc4 or gcc? (I get errors building gcc4, currently still building as gcc without an error, yet...).
EDIT: Wow, gcc (gccv6) completed its build:
dgately
Yes, this is normal. This is because, inside the docker image, there is no user with the UID specified via the "-u" parameter. Weird that Linux even functions when you're logged in as a user whose UID doesn't exist... but it works
What error did you get for GCC 4?
I'm currently trying a new make after a make clean, but here's what I get with make, right now after previous make attempts:
dgately
BTW: Once I got gccv6 to build, I was only able to exec propeller-elf-gcc within Docker (because of course, it is an x86 elf executable, which won't run under macOS native). So, I've got to figure out how to create a macOS variant, similar to how you created a WIN variant? Or, am I totally off here?
dgately