heater might chime in with a comment, but I believe the first C compiled on the propeller was BDS C running on the CP/M emulation. CP/M needs a minimum of 64k of external ram to do this.
But - it was painfully slow. The emulation was already running about half the speed that CP/M usually ran, so was taking about 2-3 minutes to compile a 20k program.
With a faster CP/M emulation (using cached ram) it probably could be done a lot faster.
I'd love to see a more modern self hosting C compiler running on the propeller. The vibe seems to be that it is possible, but hasn't actually been done as yet.
One concept the C experts might be able to help with - is it possible to split a C program into parts (eg 'objects', or 'dll's') and then precompile those and link them in at run time, and in doing so, speed up the compilation process?
I discussed this with several C people once. It was deemed a gimmick by at least one person. Certainly, I wouldn't bother trying to self-host GCC on Propeller P1 or even a P2.
One might consider porting the Tiny C Compiler (C89 with some C99 extensions). Porting could take months; it targets x86 at the moment. http://bellard.org/tcc/
Yes of course. We have been compiling C on the Propeller for years. It's done with Leor Zolmans BDSC compiler running under CP/M running on a Z80 emulator.
It has the down side of course is that it produces Z80 code!
That's no problem just compile the code under CP/M and then boot into the Z80 emulator and run the compiled program directly without any CP/M present. Think of it like a CMM mode kernel for the Prop.
Or all we need is some genius like Leor to write a C to native Prop compiler in assembler like he did!
One concept the C experts might be able to help with - is it possible to split a C program into parts (eg 'objects', or 'dll's') and then precompile those and link them in at run time, and in doing so, speed up the compilation process?
Yes (I assume you're talking about loading/unloading functions at run time, not splitting up source code to different files). In general terms you make a bunch of .so files (dynamic libraries), and you use 'dlopen()' to load the library in run time, from your application instead of letting the runtime linker do it. At work we use this on normal server systems too, as a method to add new processing functions to a plugin-based generic processor.
Some years ago I used the dlopen() mechanism so that I could handle 8-bit and 12-bit versions of the jpeg library in the same application (you either have to compile the standard jpeg lib for 12 bits or for 8 bits, it can't do both at the same time). I had two versions of the library in .so form and loaded the variant I needed.
Using dlopen() (and dclose()) to switch what code segment you're currently running is a more elegant way of implementing the equivalent of overlays as was used in e.g. Turbo Pascal once upon a time.
What I don't know is the current state of dlopen() support in propgcc, but as the dynamic runtime linker needs that anyway I assume it's supported or can be supported easily.
Actually I just now noticed you said 'speeding up the compilation process' - but that only needs the source code separated into different files so that when you e.g. make a change to a single function you'll only need to compile the source file containing that function, not all of the source. That's standard procedure. You can make your own libraries (even static ones) and fill them with functions, single functions or several functions per file, and link your main program to the library, or you can just forgo the library and split your main program into many files (e.g. one 'main.c', one 'searching.c' and so on). You'll need some header files for prototypes, include those in main.c and you're all set.
The same guy did the TCC port. OTCC can compile itself to 15K bytes of x86 code (without libraries of course).
OTCC is a subset of C and the most obvious difference is missing types in function signatures. Types also appear to be limited to int and char* strings.
Now if a "partial" x86 emulator could be written to handle OTCC's output, you would have a "not unreasonable" C dialect solution.
Here's one interesting point from the otccex.c program:
/* #!/usr/local/bin/otcc */
/*
* Sample OTCC C example. You can uncomment the first line and install
* otcc in /usr/local/bin to make otcc scripts !
*/
Added ...
Looking at the non-obfuscated code, the implementation is so simple that one might even be able to port the file to use the SPIN interpreter.
There are many limitations of course ....
Interesting compiler, but the non-obfuscated code is only for "documentation". So unless someone can un-obfuscate the working files, it's probably a waste of time.
Is there any chance that, perhaps using external memory, one could compile C code on the Propeller?
Yes, this is feasible. Catalina can (or at least could - I haven't tried it for a while!) compile itself.
However, it would take hours (perhaps even days) to compile even a very small C program on the Propeller this way, so my response whenever this question comes up is ... why on earth would you want to?
If you need a small, self-contained or embedded development environment - and one that is far cheaper than any possible Propeller-based solution - then just buy a Raspberry Pi. Under $50 and you can host all the development tools you could ever possibly want!
Tor, that is a very interesting post. Ok, could this work for, say, stdio using dlopen?
First I should add that I forgot for a moment there that we didn't just talk about running C programs on the Propeller, but about actually compiling the code on the Propeller as well.
stdio though, that's a problem.. it's part of the standard C runtime library, and that's the one that you won't be able to dlopen() by yourself, unless you rewrite the runtime library so that stdio is left out (unfortunately I'm not running propgcc yet so I'm not sure, but I think I've seen something about there being more than one runtime library version available, so maybe it's been done already?). Anyway, the standard setup is that the runtime loader (a little program running at the very start of execution of every C program, in the normal setup) actually does a dlopen() on the C runtime library (libc.so) before calling your main() function.
However, if you want small, you don't want stdio... printf, sprintf, all those variants, they're large. Formatted output requires large internal tables (the same is true for e.g. Fortran's formatted output). That's what makes 'hello.c' with just printf ("Hello, World\n"); so large. And functions like fread, fwrite are also stdio functions. In the traditional implementation they require 4096 bytes of internal buffer space. So you would avoid those too (NB: AFAIK these are not implemented in the traditional way on propgcc, not yet at least, they're more like kernel functions of the read/write type). So, what you need for memory-constrained environments is a runtime library which doesn't provide stdio, but simpler I/O alternatives instead.
Then there's of course the size of the compiler itself.. gcc isn't known to be small. So alternative compilers - I think a few have been mentioned in this thread - should probably be considered.
I have to say that I don't think dynamic libs are the equivalent of overlays. Overlays are a technique used enable running of larger program than fit in memory by chopping it up into bits that get loaded as required. All parts of the program are linked and symbols resolved at build time, it only needs to pull in the binary chunks and jump to them at run time swapping chunk for chunk as the program proceeds.
Dynamic libs are intended as a way of packaging up functionality that can be used by many different programs while not actually being in the program binaries. It allows many programs to run at the same time on a multitasking system and use the same library code. Dynamic libs are, as the name implies, linked (symbols resolved) at run time.
I really don't we want dynamic libs for C on the Prop. You are going to need a lot of OS support and it's big and it's slow. I don't see what it you would gain from it.
Dynamic libraries are usually used as you say - for sharing libs between different applications running at the same time, for example.
But dynamic libraries can equally well be used to implement an overlay scheme, it's part of the specification - the standard variant is just one specific use of dlopen(). It's just that the runtime pre-main() code does the dopen() for you before you reach main(). But you can attach and detach dynamic libraries programmatically, at wish, during runtime. At work, for example, we routinely deploy this mechanism to load specific types of data processors to a single application. We just list what functions, and which dynamic libs they live in (the .so) in an .xml file, load the file at startup, and then go on executing, dlopen() as we go. The difference from an old-time overlay scheme is that we do it in order to switch code dynamically post-build, and not to save memory.
.
Overlays, as used in the past, can be fully implemented via dlopen() and shared libraries. What you resolve, or rather define, at build time are just the hooks to call the code in the shared libraries. It's even easy to do, think up some kind of overlay scheme you want to explore and implement it via shared libs and dlopen()/dlclose(), it's surprisingly simple.
I agree that you can use dynamic libs like overlays, I only posit that they are very different animals.
With overlays your symbol resolution (linking) happens at build time with dynamic libs it happens at run (load time). This has major consequences:
Effectively an executable and its overlays are all one single program. You can't use those same overlay files with different programs. You can't use the program with different overlay files.
With dynamic libs the program executable and the libs are independent entities. You can build them separately. You can swap different versions of the lib for your program and you can use the same libs in different programs. As long as the right symbols are in there with the right API all is good.
As you say, in your apps you load different functionality as desired at run time. Clearly you can write new processing functions and create a new lib and it will work with your program without rebuilding the program itself. You cannot do that with overlays.
I agree there are differences, but in the sense that old-fashioned overlays are more limited than the functionality you can have with *nix-style dynamic libs
dlopen() isn't supported in the propeller libraries, and the propeller toolchain does not presently have support for dynamic libraries. I'm not sure how useful these would be in the propeller environment; it'd be kind of cool in the abstract to have shared libraries on an SD card, but for practical purposes SD cards are so much bigger than the propeller's RAM that there's no real downside to static linking.
Rayman
Posts: 14,665
Comments
Catalina should be able to recompile itself; you may need to also compile Roy's Spin compiler (written in C).
It should fit on a Morpheus with at least one 2MB Mem+ boards (depends on if it needs 2.5MB or 4.5MB of ram total).
PropGCC potentially would also work, but I am guessing it would need more ram as the gcc tools tend to be larger.
On Prop2 with SDRAM and a uSD card it should be possible to self-host GCC.
But - it was painfully slow. The emulation was already running about half the speed that CP/M usually ran, so was taking about 2-3 minutes to compile a 20k program.
With a faster CP/M emulation (using cached ram) it probably could be done a lot faster.
I'd love to see a more modern self hosting C compiler running on the propeller. The vibe seems to be that it is possible, but hasn't actually been done as yet.
One concept the C experts might be able to help with - is it possible to split a C program into parts (eg 'objects', or 'dll's') and then precompile those and link them in at run time, and in doing so, speed up the compilation process?
Try Peter's SmallC retargetable compiler (K&R only) http://forums.parallax.com/showthread.php/92683-quot-Small-C-quot-Compiler?p=697988&highlight=retargetable#post697988
One might consider porting the Tiny C Compiler (C89 with some C99 extensions). Porting could take months; it targets x86 at the moment. http://bellard.org/tcc/
It has the down side of course is that it produces Z80 code!
That's no problem just compile the code under CP/M and then boot into the Z80 emulator and run the compiled program directly without any CP/M present. Think of it like a CMM mode kernel for the Prop.
Or all we need is some genius like Leor to write a C to native Prop compiler in assembler like he did!
Some years ago I used the dlopen() mechanism so that I could handle 8-bit and 12-bit versions of the jpeg library in the same application (you either have to compile the standard jpeg lib for 12 bits or for 8 bits, it can't do both at the same time). I had two versions of the library in .so form and loaded the variant I needed.
Using dlopen() (and dclose()) to switch what code segment you're currently running is a more elegant way of implementing the equivalent of overlays as was used in e.g. Turbo Pascal once upon a time.
What I don't know is the current state of dlopen() support in propgcc, but as the dynamic runtime linker needs that anyway I assume it's supported or can be supported easily.
Actually I just now noticed you said 'speeding up the compilation process' - but that only needs the source code separated into different files so that when you e.g. make a change to a single function you'll only need to compile the source file containing that function, not all of the source. That's standard procedure. You can make your own libraries (even static ones) and fill them with functions, single functions or several functions per file, and link your main program to the library, or you can just forgo the library and split your main program into many files (e.g. one 'main.c', one 'searching.c' and so on). You'll need some header files for prototypes, include those in main.c and you're all set.
-Tor
The same guy did the TCC port. OTCC can compile itself to 15K bytes of x86 code (without libraries of course).
OTCC is a subset of C and the most obvious difference is missing types in function signatures. Types also appear to be limited to int and char* strings.
Now if a "partial" x86 emulator could be written to handle OTCC's output, you would have a "not unreasonable" C dialect solution.
Here's one interesting point from the otccex.c program:
Added ...
Looking at the non-obfuscated code, the implementation is so simple that one might even be able to port the file to use the SPIN interpreter.
There are many limitations of course ....
Interesting compiler, but the non-obfuscated code is only for "documentation". So unless someone can un-obfuscate the working files, it's probably a waste of time.
Yes, this is feasible. Catalina can (or at least could - I haven't tried it for a while!) compile itself.
However, it would take hours (perhaps even days) to compile even a very small C program on the Propeller this way, so my response whenever this question comes up is ... why on earth would you want to?
If you need a small, self-contained or embedded development environment - and one that is far cheaper than any possible Propeller-based solution - then just buy a Raspberry Pi. Under $50 and you can host all the development tools you could ever possibly want!
Ross.
stdio though, that's a problem.. it's part of the standard C runtime library, and that's the one that you won't be able to dlopen() by yourself, unless you rewrite the runtime library so that stdio is left out (unfortunately I'm not running propgcc yet so I'm not sure, but I think I've seen something about there being more than one runtime library version available, so maybe it's been done already?). Anyway, the standard setup is that the runtime loader (a little program running at the very start of execution of every C program, in the normal setup) actually does a dlopen() on the C runtime library (libc.so) before calling your main() function.
However, if you want small, you don't want stdio... printf, sprintf, all those variants, they're large. Formatted output requires large internal tables (the same is true for e.g. Fortran's formatted output). That's what makes 'hello.c' with just printf ("Hello, World\n"); so large. And functions like fread, fwrite are also stdio functions. In the traditional implementation they require 4096 bytes of internal buffer space. So you would avoid those too (NB: AFAIK these are not implemented in the traditional way on propgcc, not yet at least, they're more like kernel functions of the read/write type). So, what you need for memory-constrained environments is a runtime library which doesn't provide stdio, but simpler I/O alternatives instead.
Then there's of course the size of the compiler itself.. gcc isn't known to be small. So alternative compilers - I think a few have been mentioned in this thread - should probably be considered.
-Tor
I have to say that I don't think dynamic libs are the equivalent of overlays. Overlays are a technique used enable running of larger program than fit in memory by chopping it up into bits that get loaded as required. All parts of the program are linked and symbols resolved at build time, it only needs to pull in the binary chunks and jump to them at run time swapping chunk for chunk as the program proceeds.
Dynamic libs are intended as a way of packaging up functionality that can be used by many different programs while not actually being in the program binaries. It allows many programs to run at the same time on a multitasking system and use the same library code. Dynamic libs are, as the name implies, linked (symbols resolved) at run time.
I really don't we want dynamic libs for C on the Prop. You are going to need a lot of OS support and it's big and it's slow. I don't see what it you would gain from it.
Dynamic libraries are usually used as you say - for sharing libs between different applications running at the same time, for example.
But dynamic libraries can equally well be used to implement an overlay scheme, it's part of the specification - the standard variant is just one specific use of dlopen(). It's just that the runtime pre-main() code does the dopen() for you before you reach main(). But you can attach and detach dynamic libraries programmatically, at wish, during runtime. At work, for example, we routinely deploy this mechanism to load specific types of data processors to a single application. We just list what functions, and which dynamic libs they live in (the .so) in an .xml file, load the file at startup, and then go on executing, dlopen() as we go. The difference from an old-time overlay scheme is that we do it in order to switch code dynamically post-build, and not to save memory.
.
Overlays, as used in the past, can be fully implemented via dlopen() and shared libraries. What you resolve, or rather define, at build time are just the hooks to call the code in the shared libraries. It's even easy to do, think up some kind of overlay scheme you want to explore and implement it via shared libs and dlopen()/dlclose(), it's surprisingly simple.
-Tor
With overlays your symbol resolution (linking) happens at build time with dynamic libs it happens at run (load time). This has major consequences:
Effectively an executable and its overlays are all one single program. You can't use those same overlay files with different programs. You can't use the program with different overlay files.
With dynamic libs the program executable and the libs are independent entities. You can build them separately. You can swap different versions of the lib for your program and you can use the same libs in different programs. As long as the right symbols are in there with the right API all is good.
As you say, in your apps you load different functionality as desired at run time. Clearly you can write new processing functions and create a new lib and it will work with your program without rebuilding the program itself. You cannot do that with overlays.
-Tor
Eric