I'm hesitant to volunteer too much on the build system... But I do think I can at least help with that too if necessary.
Each of the subsystems has its own build system already in place. Mostly, we need an overall Makefile that builds each component and puts together a release. The Makefile in my propeller-gcc GitHub project does that but I'm sure it can be improved. We could start with it though. It builds gcc, binutils, the libraries, the loader, and gdb and also openspin since that is used in building some of the other tools.
I think we really need to start with Parallax making some decisions and looking at working with GCC if the goal is to make this an official port.
It might be the case that using something other than gas as the assembler might make an official port very difficult if at all possible.
Some early guidance from the GCC team might make things go more smoothly.
Personally I would like to see Parallax provide or appoint a project manager to oversee this and help keep those of us that can volunteer some time on track and focused.
I think we really need to start with Parallax making some decisions and looking at working with GCC if the goal is to make this an official port.
There is supposed to be a meeting about this later this month. I was just trying to get a handle on who might be interested.
It might be the case that using something other than gas as the assembler might make an official port very difficult if at all possible.
It isn't impossible. We did that many years ago at a company Eric and I both worked at. It does require some work in integrating the assembler into the gcc tool flow though.
Some early guidance from the GCC team might make things go more smoothly.
I assume you mean the team supporting GCC as a whole. As far as I know, there is no Parallax GCC team yet.
Personally I would like to see Parallax provide or appoint a project manager to oversee this and help keep those of us that can volunteer some time on track and focused.
Yes, that would be best. Steve Denson did that for the P1 port. Having a single contact person for Parallax would probably help.
With updating the tools we should start by getting P1 working before going on to P2.
If we get P1 working and tested with existing environments then we build some confidence in moving forward.
Mike
I'm concerned that will take away resources from the P2 effort and more concerned that it may result in the P2 version being a "good enough" bending of the P1 version to make it run on the P2.
I would rather see a fresh start for the P2 effort and then make that version have support for the P1.
I don't think we should worry about P1 at this point. Parallax is happy with release_1_0 for the time being and we need to get started on a toolchain for P2.
The advantages to starting with P1 first is that it already exists, and we can test it on existing hardware. Also the differences in the code generator between the P1 and P2 are trivial. Other than the memory model, the main differences are in converting wz, wc, max, maxs, min, mins and jmpret to wcz, fle, fles, fge, fges and calld. As far as converting the memory model, I suggest that we start with LMM and modify the LMM pseudo-ops to work with hubexec. That's it as far as the code generator.
One we have a working generator we can used the p2gcc utilities to assemble, link and load. At that point we have a working P2 compiler. There's still a lot of work remaining, but we can build on a functional system. We will need to optimize the code generator for the P2. The assembler should generate the ELF format. This can either be done by modifying p2asm or using GAS. The linker needs to also understand ELF. Maybe we can use the GNU linker, or maybe we us p2link -- I'm not sure of the best approach.
The reason that I've advocated using one of the existing P1 compilers is that we already have a functional compiler. However, it seems that the group is leaning toward using the latest version of GCC. This will require getting the latest version, setting up a repository, and then integrating the P1 stuff from an existing build. Hopefully, we can get some guidance from Eric on how to do the integration.
And of course, we need to produce a P2 library. Most of the library is written in C, and is fairly generic. I have been able to compile a lot of this using p2gcc. The library does contain a significant amount of P1 assembly and device drivers. These will need to be modified for the P2.
I think we should start with Eric's GCC repository. It uses a more recent version of GCC. We can clone it and check it into the Parallax GitHub account if they will allow that. Or maybe we can ask them to create a parallax-dev GitHub account to keep the stuff under development separate from the release Parallax products.
If we're not going to use p2asm then we should probably just use all of binutils which includes both gas and the linker. There probably aren't many changes needed in the linker to make it work with P2. The biggest job is adding the P2 assembler to gas.
The linker will need to know how to handle AUGS. It will also need to handle relative addressing for CALLD and JMP, unless the compiler always forces absolute addresses. It will also need to handle the 20-bit address format used with CALLD, JMP and LOC. Maybe it's not a big deal to add more addressing modes.
I like the idea of starting from Eric's repository. We could upgrade to the latest GCC version later on if needed.
The linker will need to know how to handle AUGS. It will also need to handle relative addressing for CALLD and JMP, unless the compiler always forces absolute addresses. It will also need to handle the 20-bit address format used with CALLD, JMP and LOC. Maybe it's not a big deal to add more addressing modes.
I don't think relative addressing is going to be a problem. Many architectures that GCC supports use it.
I like the idea of starting from Eric's repository. We could upgrade to the latest GCC version later on if needed.
Yes, it means we have a working compiler out of the box.
The linker will need to know how to handle AUGS. It will also need to handle relative addressing for CALLD and JMP, unless the compiler always forces absolute addresses. It will also need to handle the 20-bit address format used with CALLD, JMP and LOC. Maybe it's not a big deal to add more addressing modes.
I don't think relative addressing is going to be a problem. Many architectures that GCC supports use it.
I like the idea of starting from Eric's repository. We could upgrade to the latest GCC version later on if needed.
Yes, it means we have a working compiler out of the box.
As nice as "the latest and greatest" would be, GCC 6 is plenty new enough for most people I think. Sounds like a great compromise between a new(er) compiler and reusing existing work.
We can clone it and check it into the Parallax GitHub account if they will allow that. Or maybe we can ask them to create a parallax-dev GitHub account to keep the stuff under development separate from the release Parallax products.
If Eric is willing, we can actually transfer ownership to the Parallax org. This will keep history for bugs and issues that were opened (and closed). We can also then mark the repository as private if we really need to... but personally, I wouldn't bother. Simply a big warning at the top of the README "WORK IN PROGRESS! Use at your own risk" or something similar should be sufficient, right?
There shouldn't be a need for any sort of warning about in process development, regardless of whether the effort is starting from scratch or leveraging previous work, so long as a sensible branching strategy is used. My workplace uses "A successful Git branching model" which I use for my personal projects as well.
As far as the C and C++ standard libraries are concerned, if things are starting from scratch I would suggest the following:
1) With the C standard I/O facilities, take some inspiration from avr-libc's implementation. While the existing Propeller C standard library's out of the box support for I/O is very convenient, it seems to have some undocumented limitations (such as buffer sizing and overflow) and lacks the flexibility available with avr-libc (though this flexibility is non-standard). The convenience of the existing implementation could still be made available through an auxiliary library developed to meet Parallax's education needs.
2) For the C++ standard library, initially only develop a free standing implementation, that possibly omits run time type info and exception support, in order to reduce the amount of initial work that must be done. A full C++ standard library could then be delivered in later releases.
For standard libs, it should stick to the standards more than do nice non-standard things. Non-standard stuff should be done as separate libs made for the P1/P2.
There is already some C++ standard library work done for the P1, should be able to leverage that for this?
The advantages to starting with P1 first is that it already exists, and we can test it on existing hardware. Also the differences in the code generator between the P1 and P2 are trivial. Other than the memory model, the main differences are in converting wz, wc, max, maxs, min, mins and jmpret to wcz, fle, fles, fge, fges and calld. As far as converting the memory model, I suggest that we start with LMM and modify the LMM pseudo-ops to work with hubexec. That's it as far as the code generator.
Yes, that makes sense. As you say, there's not too much difference between P1 and P2 PASM for the kind of code the compiler generates, so changing the compiler is probably not going to be the hard part. Getting a full ecosystem (assembler, linker, library) for P2 is probably the more difficult task.
(Of course there will be a lot of room for optimization in the compiler, and for adding P2 specific builtin functions for exotic hardware features, but all of that can come later.)
For standard libs, it should stick to the standards more than do nice non-standard things. Non-standard stuff should be done as separate libs made for the P1/P2.
There is already some C++ standard library work done for the P1, should be able to leverage that for this?
Certainly, the PropGCC P1 C and C++ libraries will be a starting point. Most of the code should move over easily but there is some assembly code at the lowest levels of the drivers that will need to be ported.
We can clone it and check it into the Parallax GitHub account if they will allow that. Or maybe we can ask them to create a parallax-dev GitHub account to keep the stuff under development separate from the release Parallax products.
If Eric is willing, we can actually transfer ownership to the Parallax org. This will keep history for bugs and issues that were opened (and closed).
I don't see any reason to transfer ownership -- the beauty of github is that it's easy to fork projects, and you certainly don't need my permission to do that (although of course you have it!).
As nice as "the latest and greatest" would be, GCC 6 is plenty new enough for most people I think. Sounds like a great compromise between a new(er) compiler and reusing existing work.
Someone mentioned V8.x IIRC ? That seems to be part of the problem with GCC, the moving-target pick-a-version problem....
I'm concerned that will take away resources from the P2 effort and more concerned that it may result in the P2 version being a "good enough" bending of the P1 version to make it run on the P2.
I would rather see a fresh start for the P2 effort and then make that version have support for the P1.
As above, it makes sense to start with something that works, and a later-GCC stepping stone, if possible.
Is there a list anywhere of what the actual porting-P1-gcc-to-newer-GCC issues were ?
The point about "good enough" is valid, but I think any GCC edifice is going to be doomed to always be 'good enough'.
The in-line assembler syntax I've seen of GCC is in the 'pretty terrible' basket - are there plans to improve that, or are we stuck with 'What-GCC-does'
Those types of very user-visible issues, are what gives something like fastpsin multi language support such appeal.
I've seen some textbooks write multiple assembly statements within asm() as:
asm("
movl $4, %eax
movl $2, %ebx
addl %eax, %ebx
...
");
However, my compiler (GCC) doesn't recognize this syntax. Instead, I must rely on "\t\n" separating each line or using multiple asm():
asm("movl $4, %eax");
asm("movl $2, %ebx");
asm("addl %eax, %ebx");
...
How do I enable the "clean" syntax with no "\t\n" or repeated asm()?
good question indeed - one reply
Another option is to use C line continuation character \ (backslash). Although the following will generate excessive white space in generate assembly code it will compile as expected:
int main()
{
__asm__("movl $4, %eax; \
movl $2, %ebx; \
addl %eax, %ebx");
}
Although this is a way of doing it, I'm not suggesting that this is good form. I have a preference for the form you use in your second example using \n\t without line continuation characters.
Wow, a preference to write \n\t on every line ?! Maybe that is why this has never been fixed
but then someone says
MSVC/C++
32-bit Microsoft C and C++ compilers support an extension to the language that allows you to place multi-line inline assembly between __asm { and }. Using this mechanism you don't place the inline assembly in a C string; don't need to use line continuation; and no need to end a statement with with a ; (semicolon).
An example of this would be:
__asm {
mov eax, 4
mov ebx, 2
add ebx, eax
} Thank you @Michael - Finally an answer that address my question.
The post is for MSVC/C++, does that syntax actually work in GCC ? If yes, in which version(s) ?
Is there a list anywhere of what the actual porting-to-newer-GCC issues were ?
I expect it's a non-trivial list, different for every backend, and on the mailing list in a form that's hard to find.
I did notice one thing of interest though when I was browsing the gcc mailing list yesterday, avr was among the backends being marked for deprecation because it hadn't updated from internally deprecated model/method/function.
Back to the P2:
With the addition of a fairly solid test harness and modern-gcc in git we can probably automate the move to master until tests start to fail. Fix, rinse, repeat.
I have the skills to put this kind of framework together and automate it. I don't have the skills to fix the bugs as they're found.
A test harness is one of those jobs that's extremely valuable and has to be done that can be done independently without fear of incompatible commits from multiple developers at a time.
If you change the inline assembly syntax you'll probably make it a lot harder for our changes to be upstreamed.
Understood, I was trying to find out what the current/lastest GCC inline assembly syntax rules actually are.
Is the MSVC/C++ answer now applicable to GCC ? - the OP says it answered his question, less clear is he if actually test it, on what version
Is there a list anywhere of what the actual porting-to-newer-GCC issues were ?
I expect it's a non-trivial list, different for every backend, and on the mailing list in a form that's hard to find.
Oops, I was unclear, I was meaning the issues with upmove of P1 - ie why exactly did Parallax shelve the move to newer GCC ? Cleary some issues were bumped into ?
It seems ersmith did some porting to a middle version of GCC, did that still have any issues ?
I suggest we focus on getting the P2 code generator to work first, and not do any other changes until we reach that milestone. We're working with a small team with limited expertise in making changes to the GCC tools. Any guidance we can get from Eric would be much appreciated, but I understand if he has limited time to devote to this project.
So would somebody like to volunteer to fork/copy Eric's repository to a repository that we can contribute too? If possible we should limit push capability to only those who have volunteered to write P2 tool code. Everybody should be able to clone and/or download the code and build the tools for their platform. Also, with the continuous build feature people will be able to download and test the latest version of the code.
I suggest we focus on getting the P2 code generator to work first, and not do any other changes until we reach that milestone. We're working with a small team with limited expertise in making changes to the GCC tools. Any guidance we can get from Eric would be much appreciated, but I understand if he has limited time to devote to this project.
So would somebody like to volunteer to fork/copy Eric's repository to a repository that we can contribute too? If possible we should limit push capability to only those who have volunteered to write P2 tool code. Everybody should be able to clone and/or download the code and build the tools for their platform. Also, with the continuous build feature people will be able to download and test the latest version of the code.
We probably want to wait for Parallax to hold their tools meeting before we do much more than discuss this. They may have opinions as to where to host the repository. My understanding is that this meeting will happen after we receive our P2-ES boards.
Any guidance we can get from Eric would be much appreciated, but I understand if he has limited time to devote to this project.
Yes, unfortunately I do have pretty limited time and resources to devote to P2 development. I can give some advice, but that's about it. To be honest I've kind of burned out on GCC, particularly after GCC 6 didn't really get any traction. But it's encouraging to see a new set of developers take up the torch.
We can clone it and check it into the Parallax GitHub account if they will allow that. Or maybe we can ask them to create a parallax-dev GitHub account to keep the stuff under development separate from the release Parallax products.
If Eric is willing, we can actually transfer ownership to the Parallax org. This will keep history for bugs and issues that were opened (and closed).
I don't see any reason to transfer ownership -- the beauty of github is that it's easy to fork projects, and you certainly don't need my permission to do that (although of course you have it!).
A "forked" repository on GitHub always shows up as a forked repository. Usually when I see that at the top of a page, I think "oh, I should really go get the upstream changes, this is just some random guy's playground." That's not always the case, but most of the time it is. So I think it'd be better if the repository that Parallax is going tout as the "official" one not have "forked from ...." at the top
Other than that, I agree that forking would be sufficient. My original post about this revolved around the issue tracker and keeping its history.... but the repository in question only has a single issue with a single comment.
A "forked" repository on GitHub always shows up as a forked repository. Usually when I see that at the top of a page, I think "oh, I should really go get the upstream changes, this is just some random guy's playground." That's not always the case, but most of the time it is. So I think it'd be better if the repository that Parallax is going tout as the "official" one not have "forked from ...." at the top
That's a good point to mention to Ken.
Given that, my suggestion would be to start with the master, released P1 gcc as the root, and then place ersmith's newer-version gcc as a derivative from there, and hopefully that can be made to operate with libraries and newer GCCs and become both a new p1.gcc, and from there, a basis for P2 work.
Comments
I'm hesitant to volunteer too much on the build system... But I do think I can at least help with that too if necessary.
As soon a we get a repository on GitHub, all of these can (should) get logged in the issue tracker and then eventually expanded into many more tickets
It might be the case that using something other than gas as the assembler might make an official port very difficult if at all possible.
Some early guidance from the GCC team might make things go more smoothly.
Personally I would like to see Parallax provide or appoint a project manager to oversee this and help keep those of us that can volunteer some time on track and focused.
If we get P1 working and tested with existing environments then we build some confidence in moving forward.
Mike
That is correct David.
I'm concerned that will take away resources from the P2 effort and more concerned that it may result in the P2 version being a "good enough" bending of the P1 version to make it run on the P2.
I would rather see a fresh start for the P2 effort and then make that version have support for the P1.
C.W.
One we have a working generator we can used the p2gcc utilities to assemble, link and load. At that point we have a working P2 compiler. There's still a lot of work remaining, but we can build on a functional system. We will need to optimize the code generator for the P2. The assembler should generate the ELF format. This can either be done by modifying p2asm or using GAS. The linker needs to also understand ELF. Maybe we can use the GNU linker, or maybe we us p2link -- I'm not sure of the best approach.
The reason that I've advocated using one of the existing P1 compilers is that we already have a functional compiler. However, it seems that the group is leaning toward using the latest version of GCC. This will require getting the latest version, setting up a repository, and then integrating the P1 stuff from an existing build. Hopefully, we can get some guidance from Eric on how to do the integration.
And of course, we need to produce a P2 library. Most of the library is written in C, and is fairly generic. I have been able to compile a lot of this using p2gcc. The library does contain a significant amount of P1 assembly and device drivers. These will need to be modified for the P2.
I like the idea of starting from Eric's repository. We could upgrade to the latest GCC version later on if needed.
As nice as "the latest and greatest" would be, GCC 6 is plenty new enough for most people I think. Sounds like a great compromise between a new(er) compiler and reusing existing work.
If Eric is willing, we can actually transfer ownership to the Parallax org. This will keep history for bugs and issues that were opened (and closed). We can also then mark the repository as private if we really need to... but personally, I wouldn't bother. Simply a big warning at the top of the README "WORK IN PROGRESS! Use at your own risk" or something similar should be sufficient, right?
As far as the C and C++ standard libraries are concerned, if things are starting from scratch I would suggest the following:
1) With the C standard I/O facilities, take some inspiration from avr-libc's implementation. While the existing Propeller C standard library's out of the box support for I/O is very convenient, it seems to have some undocumented limitations (such as buffer sizing and overflow) and lacks the flexibility available with avr-libc (though this flexibility is non-standard). The convenience of the existing implementation could still be made available through an auxiliary library developed to meet Parallax's education needs.
2) For the C++ standard library, initially only develop a free standing implementation, that possibly omits run time type info and exception support, in order to reduce the amount of initial work that must be done. A full C++ standard library could then be delivered in later releases.
There is already some C++ standard library work done for the P1, should be able to leverage that for this?
(Of course there will be a lot of room for optimization in the compiler, and for adding P2 specific builtin functions for exotic hardware features, but all of that can come later.)
I don't see any reason to transfer ownership -- the beauty of github is that it's easy to fork projects, and you certainly don't need my permission to do that (although of course you have it!).
Certainly a newer GCC verion, is going to be less work to port to a version even newer..
As above, it makes sense to start with something that works, and a later-GCC stepping stone, if possible.
Is there a list anywhere of what the actual porting-P1-gcc-to-newer-GCC issues were ?
The point about "good enough" is valid, but I think any GCC edifice is going to be doomed to always be 'good enough'.
The in-line assembler syntax I've seen of GCC is in the 'pretty terrible' basket - are there plans to improve that, or are we stuck with 'What-GCC-does'
Those types of very user-visible issues, are what gives something like fastpsin multi language support such appeal.
https://stackoverflow.com/questions/43053197/how-to-write-multiple-assembly-statements-within-asm-without-t-n-separating
good question indeed - one reply
Wow, a preference to write \n\t on every line ?! Maybe that is why this has never been fixed
but then someone says
The post is for MSVC/C++, does that syntax actually work in GCC ? If yes, in which version(s) ?
I expect it's a non-trivial list, different for every backend, and on the mailing list in a form that's hard to find.
I did notice one thing of interest though when I was browsing the gcc mailing list yesterday, avr was among the backends being marked for deprecation because it hadn't updated from internally deprecated model/method/function.
Back to the P2:
With the addition of a fairly solid test harness and modern-gcc in git we can probably automate the move to master until tests start to fail. Fix, rinse, repeat.
I have the skills to put this kind of framework together and automate it. I don't have the skills to fix the bugs as they're found.
A test harness is one of those jobs that's extremely valuable and has to be done that can be done independently without fear of incompatible commits from multiple developers at a time.
Understood, I was trying to find out what the current/lastest GCC inline assembly syntax rules actually are.
Is the MSVC/C++ answer now applicable to GCC ? - the OP says it answered his question, less clear is he if actually test it, on what version
It seems ersmith did some porting to a middle version of GCC, did that still have any issues ?
So would somebody like to volunteer to fork/copy Eric's repository to a repository that we can contribute too? If possible we should limit push capability to only those who have volunteered to write P2 tool code. Everybody should be able to clone and/or download the code and build the tools for their platform. Also, with the continuous build feature people will be able to download and test the latest version of the code.
A "forked" repository on GitHub always shows up as a forked repository. Usually when I see that at the top of a page, I think "oh, I should really go get the upstream changes, this is just some random guy's playground." That's not always the case, but most of the time it is. So I think it'd be better if the repository that Parallax is going tout as the "official" one not have "forked from ...." at the top
Other than that, I agree that forking would be sufficient. My original post about this revolved around the issue tracker and keeping its history.... but the repository in question only has a single issue with a single comment.
That's a good point to mention to Ken.
Given that, my suggestion would be to start with the master, released P1 gcc as the root, and then place ersmith's newer-version gcc as a derivative from there, and hopefully that can be made to operate with libraries and newer GCCs and become both a new p1.gcc, and from there, a basis for P2 work.