Unless you use C for everything, you almost certainly have been mixing languages in your PC or mainframe projects. Most libraries are written in C or assembler. Any PC program that invokes a Windows system DLL is calling C code. It all works because Microsoft declared a standard calling interface for system calls.
Apart from standard libraries, almost every interpreter (Python, Lua, Java) has some kind of foreign function interface for calling C libraries. They work. Sometimes they're a pain to use, sometimes not, but people do accomplish things with them.
Seriously, having *some* standard calling convention (not one that's imposed on all programmers at all times, but rather one that's available for standard libraries) is extremely useful.
It's also quite orthogonal to the questoin of inter-cog communication. Now that hubexec exists it is eminently feasible for Spin code to call PASM routines on the same cog, and it seems like something that would be very useful. Should we really need to fire up a new cog just to accelerate one function? And if Spin can call PASM, then why not adopt the same mechanism so that Spin can call C, which ends up being translated to PASM?
But Eric, it's not just a question of a calling convention. As in how we pass parameters on the stack or in registers and how we get return values, and what those passed and return data type may actually be.
Those "foreign function" interfaces are not trivial.
Python to C, OK
Lua to C, OK
Javascript to C, OK
Any of those in reverse, OOPs.
Javascript to Python, OOPs
Having said that, I agree, any language to PASM, without starting a COG to run that PASM in is a worthy goal.
Of course that PASM (machine instructions) may not have come from PASM at all, but from a C compiler, say.
That still leaves my question. How would one actually build such a polyglot program?
But Eric, it's not just a question of a calling convention. As in how we pass parameters on the stack or in registers and how we get return values, and what those passed and return data type may actually be.
But a calling convention (ABI) is a fundamental start, and it's pretty standard for a microprocessor to have a defined ABI.
Perhaps the title of this thread was too grandiose. I'm not proposing that we solve every possible language interoperability issue, nor that any function written in any language should be able to call any other function in a different language. What I would like to do is to define one calling convention (or at most two) which, *if* a language supports it, and if a PASM routine is written in accordance with it, would allow the language to call PASM and vice-versa. And I think Spin and C, as the "major" languages for P2, should have some kind of support for this ABI. Maybe to use it in Spin you'll have to jump through some hoops, and maybe calling Spin functions from C will be a bit awkward and require an OBEX wrapper. But let's not rule it out before we've even tried!
That still leaves my question. How would one actually build such a polyglot program?
I can imagine many ways -- the GNU linker, Spin FILE statements including a binary blob, PASM blocks in a Spin program, etc. -- but I don't think we need to solve that right now. I don't even think we need/want to mandate one way to link things together.
But right now Chip is building Spin2, so now is the time to establish any low level interface standards to allow the interpreter to call PASM (and hopefully vice-versa).
No, no. Just a simple pipes though which different processes could exchange bytes. No matter what language they are written in.
In that regard I think it would be better if no COG could access the HUB RAM of any other COG. Then a hardware channel through which bytes could be exchanged between COGs was the only method of communication.
Given that: Language A calling language B calling language C ... X, Y, Z is a pipe dream. Never mind vice versa or in any other random combination, that leaves us with inter process communication.
That brings us to a debate that is a decade old on this forum. When the topic was how to reuse PASM drivers from, Spin objects, running in a COG from C. Be it Catalina and then GCC.
At the time nobody could agree on a standard way to do this.
I don't see any hope of improvement.
Actually, the argument was against finding a way, not finding an agreeable way! This is why a number of us were so ******.
There is still the question of how do you actually build a single project that is made from languages A, B, C..X,Y,Z, into a single binary for down load ?
Agreed. But we also need a way to call and pass parameters.
If the stack pointer of all the languages used the same one (either PTRA or PTRB), and in the same direction, then it would be possible for the calling language to place its' return address and calling parameters on the stack, and then call the called languages' method/routine.
Any type casting would be the responsibility of the calling program.
Next, is how do we setup a pointer/address table for each public method/routine, and where does it reside?
in P1 we usually supply a Hub address in PAR and read a couple of longs for coginit of a COG (or patch values in DAT from SPIN which is bad for C)
Later (when COG is running) we usually use also a block of longs to supply a command + parameters at a certain HUB address (mailbox)
Reading that is usually done in P1 with self modifying code. We all did that in some variations. Its common practice.
P2 offers the nice feature to read a couple of longs from HUB to COG (or LUT) with setqx + rdlong (or something alike), nice two liner to get a block of parameter out of the HUB into the COG.
P2 also offers to restart a loaded COG at a given address, without loading it again.
But common (and useful) is to have parameters and command in some block in HUB and call the COG.
That will work from any Language without C and SPIN or Tachion or whoever sharing the same Stack.
So the first goal for interoperability should be - Now more patching the PASM image, use Parameter blocks in HUB.
Then C and Spin (or whoever) can share PASM blobs without hassle as long as they know where the mailbox is.
That's a start.
Calling from SPIN to C for functions/methods should also be simple with inline PASM, provided we know the HUB address and the C stack pointer. Or we could also use HUB-Blocks.
Calling SPIN from C is more complicated, there we need some simple way.
Using shared COG registers for fastcalls makes no sense with SPIN running from a COG, C can't access the SPIN-interpreter-COG registers, Does not work.
That leaves the problem with loading multiple languages at the same time into the P2.
@Chip will be no help there, but I am sure @Roy will be able to work with Eric/David/Dave to produce something a ELF linker can link together.
SPIN2 needs to have its interpreter linked to the SPIN2 program anyways, so SPIN2 will need some linker, why not go for ELF there.
A SPIN2 program is basically a PASM2 binary (the interpreter) and a Data Block containing the SPIN2 byte Codes.
I see no real problem for a linker to combine that with a C program.
we even could work in C and SPIN2 with something alike
external function "xxx" to provide information to a linker?
A SPIN2 program is a PASM2 binary (the interpreter) and a Data Block containing the Byte Codes. We also need a Stack (-segment?).
If OpenSpin2 could produce ELF compatible output, linking with C (or whatever) should be doable.
Then Symbols can be shared and C could reach a SPIN variables and SPIN could reach one out of C.
For C calling SPIN (SPIN external method XXX) we would need a PASM stub in hub handling parameters/return value between C and SPIN2.
Same for SPIN2 to C.
This is standard stuff linker are doing for ages. There might be even support for overlays in GCC&CO,, not sure about that, but dynamic linking and shared libraries (dll-hell) would be possible then.
If we're only concerned about C/Spin compatibility it seems like the easiest solution is to use spin2cpp, and link everything together as C. If you want another language to work with Spin or C you would need to write an xxxx2cpp converter for it.
If we're only concerned about C/Spin compatibility it seems like the easiest solution is to use spin2cpp, and link everything together as C. If you want another language to work with Spin or C you would need to write an xxxx2cpp converter for it.
For Prop1, yes, I mostly agree. But I think for Spin2 I'm not sure this is a complete solution:
(1) spin2cpp doesn't support Spin2 yet. I'll probably add it, but I might get hit by a bus (or by a load of work) and not be able to.
(2) I really think we want a way for Spin2 to call out to hubexec PASM routines anyway.
My experience of mixing languages in a single project/program has always been a royal pain in the butt.
Even calling between compiled languages like C to Pascal or vice versa.
Heck, even calling from C to C++ and C++ to C is a pain.
Then mixing a compiled language with an interpreted one. Sure you can start a Lua interpreter from C then interact with it, but that is a pain. Or perhaps you can add C functionality to Python or Javascript. Also a pain.
Then the killer, you want something like Lua to make calls to Python or vice versa. Good luck with that.
Microsoft's .Net framework was supposed to make it easy to use a bunch of different languages in a single program. I have no idea how well it works but I'm sure it's not coming to the Propeller soon.
As such, I think the only hope of language Interoperability in a single Propeller project is cog to cog communication. If that were standardized, perhaps even baked into the hardware it would stand a chance.
I agree 100% with this sentiment. Different languages in a single program may be possible but will be a lot of pain for little gain.
A standard for cog to cog communications on the other hand could be implemented in a manner that makes it possible for programs written in different languages to communicate with each other. That would be much simpler to implement and use. Somewhat like a turbocharged version of pipes.
Comments
Apart from standard libraries, almost every interpreter (Python, Lua, Java) has some kind of foreign function interface for calling C libraries. They work. Sometimes they're a pain to use, sometimes not, but people do accomplish things with them.
Seriously, having *some* standard calling convention (not one that's imposed on all programmers at all times, but rather one that's available for standard libraries) is extremely useful.
It's also quite orthogonal to the questoin of inter-cog communication. Now that hubexec exists it is eminently feasible for Spin code to call PASM routines on the same cog, and it seems like something that would be very useful. Should we really need to fire up a new cog just to accelerate one function? And if Spin can call PASM, then why not adopt the same mechanism so that Spin can call C, which ends up being translated to PASM?
Eric
Those "foreign function" interfaces are not trivial.
Python to C, OK
Lua to C, OK
Javascript to C, OK
Any of those in reverse, OOPs.
Javascript to Python, OOPs
Having said that, I agree, any language to PASM, without starting a COG to run that PASM in is a worthy goal.
Of course that PASM (machine instructions) may not have come from PASM at all, but from a C compiler, say.
That still leaves my question. How would one actually build such a polyglot program?
LMAO!
Perhaps the title of this thread was too grandiose. I'm not proposing that we solve every possible language interoperability issue, nor that any function written in any language should be able to call any other function in a different language. What I would like to do is to define one calling convention (or at most two) which, *if* a language supports it, and if a PASM routine is written in accordance with it, would allow the language to call PASM and vice-versa. And I think Spin and C, as the "major" languages for P2, should have some kind of support for this ABI. Maybe to use it in Spin you'll have to jump through some hoops, and maybe calling Spin functions from C will be a bit awkward and require an OBEX wrapper. But let's not rule it out before we've even tried!
I can imagine many ways -- the GNU linker, Spin FILE statements including a binary blob, PASM blocks in a Spin program, etc. -- but I don't think we need to solve that right now. I don't even think we need/want to mandate one way to link things together.
But right now Chip is building Spin2, so now is the time to establish any low level interface standards to allow the interpreter to call PASM (and hopefully vice-versa).
Eric
No, no. Just a simple pipes though which different processes could exchange bytes. No matter what language they are written in.
In that regard I think it would be better if no COG could access the HUB RAM of any other COG. Then a hardware channel through which bytes could be exchanged between COGs was the only method of communication.
Interoperability problem solved.
Of course, that is not going to happen.
A simple calling convention for external binaries isn't so hard we can't bake it in.
I feel ersmith is barking up a good tree here. Just enough to make some hard things possible.
Scope is super important.
A "simple" calling convention is probably too much for any Propeller language builder to follow unless they have a really good reason to do so.
Agreed. But we also need a way to call and pass parameters.
If the stack pointer of all the languages used the same one (either PTRA or PTRB), and in the same direction, then it would be possible for the calling language to place its' return address and calling parameters on the stack, and then call the called languages' method/routine.
Any type casting would be the responsibility of the calling program.
Next, is how do we setup a pointer/address table for each public method/routine, and where does it reside?
What happened with the RED and BLUE there?
We are hypothesizing about languages for the P2 that do not exist yet. Heck, the P2 does not exist yet.
One can lay down whatever standards one likes now.
Those languages that come in the future will ignore it.
I think it's hopeless.
If, say it's easy to encapsulate some binary into a SPIN PASM procedure, that will get used.
At a minimum, there will be whatever a HUBEXEC body of code needs to work in SPIN.
We may get a simple thing for free. Maybe we revisit this in a short while.
in P1 we usually supply a Hub address in PAR and read a couple of longs for coginit of a COG (or patch values in DAT from SPIN which is bad for C)
Later (when COG is running) we usually use also a block of longs to supply a command + parameters at a certain HUB address (mailbox)
Reading that is usually done in P1 with self modifying code. We all did that in some variations. Its common practice.
P2 offers the nice feature to read a couple of longs from HUB to COG (or LUT) with setqx + rdlong (or something alike), nice two liner to get a block of parameter out of the HUB into the COG.
P2 also offers to restart a loaded COG at a given address, without loading it again.
But common (and useful) is to have parameters and command in some block in HUB and call the COG.
That will work from any Language without C and SPIN or Tachion or whoever sharing the same Stack.
So the first goal for interoperability should be - Now more patching the PASM image, use Parameter blocks in HUB.
Then C and Spin (or whoever) can share PASM blobs without hassle as long as they know where the mailbox is.
That's a start.
Calling from SPIN to C for functions/methods should also be simple with inline PASM, provided we know the HUB address and the C stack pointer. Or we could also use HUB-Blocks.
Calling SPIN from C is more complicated, there we need some simple way.
Using shared COG registers for fastcalls makes no sense with SPIN running from a COG, C can't access the SPIN-interpreter-COG registers, Does not work.
That leaves the problem with loading multiple languages at the same time into the P2.
@Chip will be no help there, but I am sure @Roy will be able to work with Eric/David/Dave to produce something a ELF linker can link together.
SPIN2 needs to have its interpreter linked to the SPIN2 program anyways, so SPIN2 will need some linker, why not go for ELF there.
A SPIN2 program is basically a PASM2 binary (the interpreter) and a Data Block containing the SPIN2 byte Codes.
I see no real problem for a linker to combine that with a C program.
we even could work in C and SPIN2 with something alike
external function "xxx" to provide information to a linker?
Enjoy!
Mike
A SPIN2 program is a PASM2 binary (the interpreter) and a Data Block containing the Byte Codes. We also need a Stack (-segment?).
If OpenSpin2 could produce ELF compatible output, linking with C (or whatever) should be doable.
Then Symbols can be shared and C could reach a SPIN variables and SPIN could reach one out of C.
For C calling SPIN (SPIN external method XXX) we would need a PASM stub in hub handling parameters/return value between C and SPIN2.
Same for SPIN2 to C.
This is standard stuff linker are doing for ages. There might be even support for overlays in GCC&CO,, not sure about that, but dynamic linking and shared libraries (dll-hell) would be possible then.
I shut up now
Mike
For Prop1, yes, I mostly agree. But I think for Spin2 I'm not sure this is a complete solution:
(1) spin2cpp doesn't support Spin2 yet. I'll probably add it, but I might get hit by a bus (or by a load of work) and not be able to.
(2) I really think we want a way for Spin2 to call out to hubexec PASM routines anyway.
I agree 100% with this sentiment. Different languages in a single program may be possible but will be a lot of pain for little gain.
A standard for cog to cog communications on the other hand could be implemented in a manner that makes it possible for programs written in different languages to communicate with each other. That would be much simpler to implement and use. Somewhat like a turbocharged version of pipes.