propgcc now in the Parallax github

David Betz · 2015-03-18 06:59

SwimDude0614 wrote: »
Ugh. I'm stumped! I keep hitting this error with the rpi build:
Configuring ncurses
/bin/sh: 1: /var/www/vhosts/david.zemon.name/propgcc-build/src/ncurses/configure: not found
make: *** [/var/www/vhosts/david.zemon.name/propgcc-build/src/../build-rpi/ncurses/ncurses-configured] Error 127
I don't get it. It's looking for a directory called ncurses in the source files.... but there's definitely no ncurses directory in the repository. I thought maybe I just didn't have an environment variable set correctly, or was missing a dependency, but the Makefile doesn't show any obvious issues (referring to line 254: https://github.com/DavidZemon/propgcc/blob/master/Makefile#L254). The one odd thing I noticed was that the target "$(BUILD)/ncurses/ncurses-configured" depends on "$(BUILD)/ncurses/ncurses-created" which doesn't exist in that file. So... maybe ncurses-created is supposed to create the ncurses directory? Makes sense based on the name. Where's that target defined?

--edit--
grep just finished searching the source directory and found nothing
dzweb@zemon1204:~/propgcc-build/src$ grep ncurses-created ./* -r
./Makefile:$(BUILD)/ncurses/ncurses-configured: $(BUILD)/ncurses/ncurses-created
dzweb@zemon1204:~/propgcc-build/src$

I have no idea what is going on with the ncurses stuff. I didn't add it. However, the xxx-created files are just sentinels used to indicate that the xxx directory has been created. You can't use the directory itself as a dependency because it changes every time a file in it changes and causes lots of unnecessary rebuilds.

ersmith · 2015-03-18 19:14

ncurses is only required for the raspberry pi, and then (I think) only for gdb. So it's probably safe to comment out the ncurses requirement in the Makefile (the CURSES=ncurses line for the rpi build). That said, I'm surprised that (a) ncurses didn't get checked in (and you're right, it didn't), and (b) nobody has complained about that before. I guess rpi versions of gdb were low on people's priority lists.

I'll see if I can figure out where I got the ncurses source code from and make sure it's OK to check in.

Eric

DavidZemon · 2015-03-18 19:16

ersmith wrote: »

ncurses is only required for the raspberry pi, and then (I think) only for gdb. So it's probably safe to comment out the ncurses requirement in the Makefile (the CURSES=ncurses line for the rpi build). That said, I'm surprised that (a) ncurses didn't get checked in (and you're right, it didn't), and (b) nobody has complained about that before. I guess rpi versions of gdb were low on people's priority lists.

I'll see if I can figure out where I got the ncurses source code from and make sure it's OK to check in.

Eric

great!

David, can you merge my pull request? soon as it's merged, I can point TeamCity at your repo instead of mine, then as soon as eric commits the ncurses source code, TeamCity will build us a new raspberry pi binary!

Heater. · 2015-03-19 08:34

ncurses is just fine on Raspberry Pi. I just install the libncurses5-dev package. gdb builds OK.

Why put ncurses source in the propgcc repository?

That is assuming we are building from David's propgcc repo on github. Master branch.

Here are my build steps on the Pi itself:

0) Ensure you have 3GB of free space file system. I use an NFS mount for this.

1) Install packages required for building:

$ sudo apt-get install build-essential
$ sudo apt-get install g++
$ sudo apt-get install texinfo
$ sudo apt-get install libncurses5-dev
$ sudo apt-get install bison
$ sudo apt-get install flex

2) Check out propgcc from github:

$ git clone https://github.com/dbetz/propgcc.git

3) Fetch and build openspin and copy it into the propgcc release directory:

$ git clone https://github.com/parallaxinc/OpenSpin.git
$ cd OpenSpin/
$ make
$ cp openspin ../propgcc/release/openspin.linux
$ cd ..

4) Make the directory that progcc is to be installed into:

$ sudo mkdir /opt/parallax
$ sudo chown pi:pi /opt/parallax

5) Build propgcc:

$ export PATH=$PATH:/opt/parallax/bin/
$ cd propgcc
$ make

DavidZemon · 2015-03-19 09:02

That's very curious. I'll give this a try on my pi next chance i get. I'm working on a bug in PropWare (cogc issues again

) at the moment, but soon as that's fixed I'll get back to PropGCC raspberry pi builds

David Betz · 2015-03-19 09:59

SwimDude0614 wrote: »

great!

David, can you merge my pull request? soon as it's merged, I can point TeamCity at your repo instead of mine, then as soon as eric commits the ncurses source code, TeamCity will build us a new raspberry pi binary!

What did you change? I didn't get email saying you had submitted a pull request.

David Betz · 2015-03-19 10:03

Heater. wrote: »

ncurses is just fine on Raspberry Pi. I just install the libncurses5-dev package. gdb builds OK.

Why put ncurses source in the propgcc repository?

That is assuming we are building from David's propgcc repo on github. Master branch.

Here are my build steps on the Pi itself:

0) Ensure you have 3GB of free space file system. I use an NFS mount for this.

1) Install packages required for building:

$ sudo apt-get install build-essential
$ sudo apt-get install g++
$ sudo apt-get install texinfo
$ sudo apt-get install libncurses5-dev
$ sudo apt-get install bison
$ sudo apt-get install flex

2) Check out propgcc from github:

$ git clone https://github.com/dbetz/propgcc.git

3) Fetch and build openspin and copy it into the propgcc release directory:

$ git clone https://github.com/parallaxinc/OpenSpin.git
$ cd OpenSpin/
$ make
$ cp openspin ../propgcc/release/openspin.linux
$ cd ..

4) Make the directory that progcc is to be installed into:

$ sudo mkdir /opt/parallax
$ sudo chown pi:pi /opt/parallax

5) Build propgcc:

$ export PATH=$PATH:/opt/parallax/bin/
$ cd propgcc
$ make

If you're up for a challenge, try building my new propeller-gcc repository. It should automatically pull openspin and spin2cpp from their respective repositories and build them along with the rest of the gcc toolchain. If you decide to try this, make sure you read through the README.md file because there are a couple of additional steps required from what is needed by propgcc. Also, this repository uses Eric's port of the new version of gcc as well as binutils and lib.

https://github.com/dbetz/propeller-gcc

Heater. · 2015-03-19 10:09

OK, That will have to wait till tomorrow though.

DavidZemon · 2015-03-19 10:44

David Betz wrote: »

What did you change? I didn't get email saying you had submitted a pull request.

Weird. It's just two small changes. 1) Add a path in front of the spin compiler variable so that the PATH variable doesn't have be messed with. 2) allow for longer path names (build and prefix)

https://github.com/dbetz/propgcc/pull/73

DavidZemon · 2015-03-19 10:45

David Betz wrote: »

If you're up for a challenge, try building my new propeller-gcc repository. It should automatically pull openspin and spin2cpp from their respective repositories and build them along with the rest of the gcc toolchain. If you decide to try this, make sure you read through the README.md file because there are a couple of additional steps required from what is needed by propgcc. Also, this repository uses Eric's port of the new version of gcc as well as binutils and lib.

https://github.com/dbetz/propeller-gcc

Didn't know this was live!

I'll give this a try too

ersmith · 2015-03-19 16:07

Heater. wrote: »

ncurses is just fine on Raspberry Pi. I just install the libncurses5-dev package. gdb builds OK.

Why put ncurses source in the propgcc repository?

It was for cross-compilers. I built the RPi binaries on a PC, and the ARM cross-tools didn't come with ncurses, for some reason. But I guess it's arguable whether ncurses should be in the repo or not.

All this will become easier with David's new propeller-gcc repository, which makes the subdirectories into git submodules. Then I think whether or not to pull ncurses from its git (which we won't have to host) will be up to the builder.

Eric

David Betz · 2015-03-19 17:22

I just modified the Makefile for propeller-gcc to write its generated code to ../build/target instead of directly to /opt/parallax. It also pushes ../build/target/bin onto the front of PATH before building any of the targets so the already generated binaries should be used to build the remaining binaries. I've also added a new "install" target that will copy the generated files to /opt/parallax.

So to build propeller-gcc:

make
sudo make install

If for some reason you want to install to some directory other than /opt/parallax:

INSTALL=/my/install/directory make install

Whether you need to use sudo with that command depends on whether you have write access to /my/install/directory.

I will make these same changes to the propgcc repository shortly.

DavidZemon · 2015-03-19 17:27

David Betz wrote: »
I just modified the Makefile for propeller-gcc to write its generated code to ../build/target instead of directly to /opt/parallax. It also pushes ../build/target/bin onto the front of PATH before building any of the targets so the already generated binaries should be used to build the remaining binaries. I've also added a new "install" target that will copy the generated files to /opt/parallax.

So to build propeller-gcc:
make
sudo make install
If for some reason you want to install to some directory other than /opt/parallax:
INSTALL=/my/install/directory make install
Whether you need to use sudo with that command depends on whether you have write access to /my/install/directory.

I will make these same changes to the propgcc repository shortly.

Great! I like that a lot more. Would that mean that PREFIX doesn't need to be defined uniquely for cross-compiling? I notice the variable is still used for cross compiling at the moment

David Betz · 2015-03-19 17:30

SwimDude0614 wrote: »

Great! I like that a lot more. Would that mean that PREFIX doesn't need to be defined uniquely for cross-compiling? I notice the variable is still used for cross compiling at the moment

Actually, PREFIX still needs to be redefined for cross compiling. In fact, I may have broken cross compiling with this edit. I'll have to check and fix that if I did. I'm still a little nervous about the idea of using ../build as a workspace. It bothers me to use a directory outside of the root of the tree where you type "make". In any case, you can still define PREFIX on the command line.

David Betz · 2015-03-19 17:34

David Betz wrote: »

Actually, PREFIX still needs to be redefined for cross compiling. In fact, I may have broken cross compiling with this edit. I'll have to check and fix that if I did. I'm still a little nervous about the idea of using ../build as a workspace. It bothers me to use a directory outside of the root of the tree where you type "make". In any case, you can still define PREFIX on the command line.

Okay, I moved the code that determines the default value for PREFIX to before where the cross compilation gets setup. I hope that will fix cross compiles. Let me know if you have trouble.

DavidZemon · 2015-03-19 17:45

All the changes i made to the propgcc repo are now gone, so I'm finding them and fixing again. Haven't gotten a successful build to finish yet.

I'm not a fan of the workspace being ../build either. I'd love to see it get moved underneath the checkout directory. It'd make building in TeamCity a lot easier too, if everything was contained inside the checkout directory.

David Betz · 2015-03-19 18:07

SwimDude0614 wrote: »

All the changes i made to the propgcc repo are now gone, so I'm finding them and fixing again. Haven't gotten a successful build to finish yet.

I'm not a fan of the workspace being ../build either. I'd love to see it get moved underneath the checkout directory. It'd make building in TeamCity a lot easier too, if everything was contained inside the checkout directory.

How did your changes to propgcc get lost? Did I do something wrong?

I don't see any reason it couldn't be ./build instead of ../build. Maybe Eric will comment on why it is the way it is.

DavidZemon · 2015-03-19 18:09

Sorry - i didn't mean to imply that. I meant they didn't carry over to the new repository.

David Betz · 2015-03-19 18:10

SwimDude0614 wrote: »

Sorry - i didn't mean to imply that. I meant they didn't carry over to the new repository.

I just copied the Makefile from propgcc when I made propeller-gcc. I guess I must have made propeller-gcc before you made your changes.

ersmith · 2015-03-20 05:44

SwimDude0614 wrote: »

I'm not a fan of the workspace being ../build either. I'd love to see it get moved underneath the checkout directory. It'd make building in TeamCity a lot easier too, if everything was contained inside the checkout directory.

There's a pretty strong argument that building should not modify the checkout directory. This is particularly important if you're doing cross-compiles and need to frequently switch build environments. If all the changes are encapsulated in one directory, which can then be placed in .gitignore, then it's not too bad, but I think it's even better to make the object files "out of tree" in a separate directory, leaving the original checked out sources unchanged.

Eric

Heater. · 2015-03-20 06:02

Yes, keeping the build directory out of the source tree is a common approach. Qtcreator for example builds into what it calls "shadow builds", basically directories outside the source tree. This is basically so that you can do cross compilations or use different compilers, clang for example.

It might be nice not to use such a simple generic name like "build" for this directory. But something more specific or user defined. qtcreator for example comes up with names like "/opt/qt5.3.1/Examples/Qt-5.3/widgets/mainwindows/build-application-Desktop_Qt_5_3_GCC_64bit-Debug"

yeti · 2015-03-20 06:13

If the builddir were just named like the git directory with ".build" appended, having the sources of propgcc and propeller-gcc side by side would not make the build directories clash on the same name.

And a default for PREFIX not being "/opt/parallax" for the new propeller-gcc while being work in progress would be nice. "/opt/parallax-wip"?

David Betz · 2015-03-20 06:34

I guess ../propeller-gcc-build would be a good compromise and maybe even ../propeller-gcc-<cross>-build for cross builds.

Heater. · 2015-03-20 06:38

Sounds good to me.

Heater. · 2015-03-20 08:14

Ooops, propeller-gcc build failed on the Pi.

I guess I should raise an issue on Eric's gcc repo or what?

Did you say this was a port of a new version of GCC ?

make[2]: Entering directory '/mnt/nfs/home/client1/build/gcc/gcc'
g++ -c   -g -O2 -DIN_GCC  -DCROSS_DIRECTORY_STRUCTURE  -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wwrite-strings -Wcast-qual -Wmissing-format-attribute -Woverloaded-virtual -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -fno-common  -DHAVE_CONFIG_H -I. -I. -I/home/pi/propeller-gcc/gcc/gcc -I/home/pi/propeller-gcc/gcc/gcc/. -I/home/pi/propeller-gcc/gcc/gcc/../include -I/home/pi/propeller-gcc/gcc/gcc/../libcpp/include -I/mnt/nfs/home/client1/build/gcc/./gmp -I/home/pi/propeller-gcc/gcc/gmp -I/mnt/nfs/home/client1/build/gcc/./mpfr -I/home/pi/propeller-gcc/gcc/mpfr -I/home/pi/propeller-gcc/gcc/mpc/src  -I/home/pi/propeller-gcc/gcc/gcc/../libdecnumber -I/home/pi/propeller-gcc/gcc/gcc/../libdecnumber/dpd -I../libdecnumber -I/home/pi/propeller-gcc/gcc/gcc/../libbacktrace -I/mnt/nfs/home/client1/build/gcc/./isl/include -I/home/pi/propeller-gcc/gcc/isl/include  -o expr.o -MT expr.o -MMD -MP -MF ./.deps/expr.TPo /home/pi/propeller-gcc/gcc/gcc/expr.c
/home/pi/propeller-gcc/gcc/gcc/expr.c: In function &#8216;tree_node* get_inner_reference(tree, long long int*, long long int*, tree_node**, machine_mode*, int*, int*, bool)&#8217;:
/home/pi/propeller-gcc/gcc/gcc/expr.c:6969:1: internal compiler error: output_operand: invalid shift operand
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.6/README.Bugs> for instructions.

Preprocessed source stored into /tmp/ccWv9nrE.out file, please attach this to your bugreport.

ersmith · 2015-03-20 12:14

That's weird that it doesn't build on the pi. I was able to cross build a pi compiler on my x86 Linux machine, so I don't see any reason that it shouldn't build natively. Actually the "internal compiler error" makes it sound like there is in fact a bug in the gcc 4.6 that you're using on the Raspberry P, rather than a propgcc problem (since propgcc is version 5.0)i. My cross compiler (arm-linux-gnueabihf-gcc) is version 4.8.3. Any chance you can try with a more recent native gcc for the RPi?

Heater. · 2015-03-20 12:21

Eric,

I was suspecting this was a compiler version issue.

Problem is that no, I have Raspbian installed on the Pi and whatever GCC version that is then that is what it is.

There is no way I am messing with getting a more recent GCC on the machine. Not until Jessie becomes stable and we upgrade brining whatever GCC version that has with it.

David Betz · 2015-03-20 13:25

I may try to move the master branch of propeller-gcc back to the old gcc/binutils/lib and move the new gcc/binutils/lib to a branch if I can figure out how to do that with submodules.

Dave Hein: Are you following this thread? Do you have spinsim in a git repository somewhere so I can include it as a submodule of propeller-gcc?

David Betz · 2015-03-21 05:41

I just pushed changes to the propeller-gcc repository that should fix cross builds and also make the default build directory ../propeller-gcc-build. This moves the generated files out of the source tree. I'd really like to remove the ncurses stuff from the Makefile and just make installing the ncurses library a prerequisite for building propeller-gcc on a platform. It seems to me that that is the normal way of handling this sort of dependency. Does anyone have any objections? I'd rather not pull the sources for every dependent library into the propeller-gcc tree just because some platform doesn't come with it installed by default. Comments?

Heater. · 2015-03-21 06:04

I agree. I see no reason to pull in ncurses source code or any other such dependency.