I really think that is something that should be built into the IDE or whatever, and not the compiler/language.
I agree that the compiler should handle Core Spin and nothing more. But I also think that Spin should be extensible. If the compiler provided the hooks for extensibility, then extensions could be expected to occur in a uniform fashion. If each IDE had to provide its own extension methods, then the language itself becomes IDE-dependent. I see the PRE section as kind of self-defense for the compiler and as a guard against forking of the language itself. Without PRE or some mechanism like it, more people will be tempted to mess with the compiler itself, and I do not believe the consequences of that will be for the good.
Now, a technical question about your compiler. At what stage during compilation does the compiler gather all of the objects in the "project?" Is this a pre-compilation matter, or does it occur as each object in the object tree gets compiled. The reason I ask has to do with processing namespaces in a preprocessor, rather than in the compiler itself.
PRE
[b]sudo[/b] rm -rf *
PUB start | i
debug "Testing the Spin DEBUG function:\r\r"
repeat i from -10 to 10
debug " The value of i is ", dec3 i, ", and i-squared is "
I'm not sure how to make any preprocessor "safe." If it has read/write access to the filesystem, as it must, it will be intrinsically unsafe. A safe scripting language, like JavaScript, would not be a suitable candidate, since it does not have the necessary file access. And, with Perl or Python, anything is possible. Maybe it would be possible for the compiler or IDE to invoke the preprocessors in a sandbox to ensure that they play nice with the system.
I agree that the compiler should handle Core Spin and nothing more. But I also think that Spin should be extensible. If the compiler provided the hooks for extensibility, then extensions could be expected to occur in a uniform fashion. If each IDE had to provide its own extension methods, then the language itself becomes IDE-dependent. I see the PRE section as kind of self-defense for the compiler and as a guard against forking of the language itself. Without PRE or some mechanism like it, more people will be tempted to mess with the compiler itself, and I do not believe the consequences of that will be for the good.
But what you're proposing essentially builds the language forking right into the language itself; that is, people will only be able to build a program with a PRE section if they have all the tools mentioned in the PRE. I guess if we build a scripting language into the Spin compiler itself, and write the PRE extensions in that scripting language, the problem would be avoided, but that's a pretty significant change.
The simpler solution is that extended versions of Spin are identified by the IDE/Makefile/whatever, probably by means of the filename. That is, "foo.dspin" is a Spin file that has to be passed through the debug.exe preprocessor. People will know that a .dspin file is not regular Spin and needs a different (or additional) tool.
(The idea of an extensible language with a scripting language built in has some appeal -- it's part of what makes Scheme so powerful. But then comes the thorny question of what scripting language should be built in...)
A pre-processor is a very important piece for any language.
I hope Roy can get to the pre-processor items soon. We need them for propeller-gcc items to replace BSTC entirely. There are some #ifdefs in the cache-cog spin/pasm files that open-source spin can't handle. I would ask for the #ifdefs to be removed, but that's probably the wrong direction to go.
But what you're proposing essentially builds the language forking right into the language itself ...
Correct. But the reason that this is less of a problem than with multiple different compilers is that by building extensibility into Core Spin, you don't end up with Spin programs that are incompatible with each other, as long as people have the necessary preprocessors. If the forking occurs at the compiler level, you might end up with one object that requires compiler A and another that requires compiler B. There would be no way to combine both objects in the same program, because they can't be compiled together. But with built-in extensibility, compatibility can be maintained. Moreover, with access to the Core Spin preprocessor output, objects can be distributed without the preprocessors.
For these reasons, I see core extensibility as a defense against incompatible forking, rather than a mechanism that promotes it. One way or another, forking will occur. If all the forked code can play nice together, that would be a benefit rather than a detriment.
Don't forget: the Spin compiler is not just a compiler: it's also a linker and loader. Without separating these functions, there has to be a way to compile disparate sources. Extensibility via preprocessing hooks is a way to do just that.
Without PRE or some mechanism like it, more people will be tempted to mess with the compiler itself, and I do not believe the consequences of that will be for the good.
I agree with the gist of what you are saying but you have hit on a very important point.
As it stands the Spin language has no formal definition, there are no rigorous specifications, there is no BNF definition of the syntax. Rather the language is defined by whatever Chips compiler accepts and does with it (Correct me if I am wrong sombody). Contrast to C or languages going back to ALGOL where the language syntax and semantics is very stricktly defined regardless of any particular implementation.
In theory (and practice) I could take the GCC C compiler and add features or "fix" this and that feature of the C language to my hearts content. Would anyone use it? Not if they are smart or only if my modifications are amazingly good and usefull. But then I have created a new language, it is no longer C as defined by the standards and I should call it something else (C++ for example).
So, given a rigorous definition of Spin we need not worry about all those possible forks of the compiler. They are just experimental offshoots that have features that may or may not get folded back to the Standard Spin.
Didn't you mean the following? [re: sudo rm -rf *]
You have a very good point. It applies to make files as well. And any other random stuff I fetch from the net. You have to "trust your sources". I still don't think including that kind of scripting into source code is a good idea. Better to "separate concerns" as they say.
...by building extensibility into Core Spin, you don't end up with Spin programs that are incompatible with each other,
I don't think this is true in general. Imagine:
I might create a Spin program that uses extentions that require some preprocessor. OK your PRE idea ensures that the required preprocessor is used. Think "ifdef ..." and such. As long as the user has the right preprocessor he is good to go.
BUT I also use a Spin compiler hacked up from the mainstream open source version. My hacked up version changes the sematics of operators , say I would rather use C style >= rather tha Spin style =>. Well at that point the poor sap trying to use my code is totally screwed.
In general I cannot see a way around this, there has to be a "one true language standard", perhaps with backwards compatible enhancements as features get added. Or ultimately the thing changes so much as to be a new language as in C and C++. Don't forget that C++ was a preprocessor for C originally.
BUT I also use a Spin compiler hacked up from the mainstream open source version. My hacked up version changes the sematics of operators , say I would rather use C style >= rather tha Spin style =>. Well at that point the poor sap trying to use my code is totally screwed.
I don't see the problem. Rather than using a hacked-up compiler, a preprocessor could just as easily make the operator substitutions. It all depends upon how deeply the preprocessor is written to parse the incoming code. Heck, it would even be possible for a preprocessor to accept a subset of C and spit out Core Spin, if someone wanted to go that far.
The whole idea is to make hacked-up compilers unnecessary.
Don't forget: the Spin compiler is not just a compiler: it's also a linker and loader. Without separating these functions, there has to be a way to compile disparate sources.
No.
I belive that if the sources you are trying to build into a single program are disparate (i.e. comming from different Spin language versions) then things have gotten so complex that it's time to give up and use something like GCC that can link any object file from anywhere if the format is compatible no matter what language it ´was originally written in.
Spin is a simple language, ideal for beginners, those that hardly know what a program is. Or for experineced programmers it let's them do simple thing simply. Let's keep it simple.
For those that grow in their programming demands and hit the limits there are already a thousand other solutions. No point in reinventing the wheel and killing Spin with "feature creep".
My hacked up Spin compiler uses C style >= for greater than or equal. Some preprocessor fixes that to be => in Spin style to be compiled by a "standard" Spin compiler. BUT my program also uses => to mean something complete different. What then?
That's just my point: don't use a hacked-up compiler! If you want C-style >=, write a preprocessor that replaces them with => so it can be compiled correctly by the Core Spin compiler, and name that preprocessor in your source file under the PRE section. It would only apply to that one file and not to other objects that may have their own PRE sections.
The whole idea is to make hacked-up compilers unnecessary.
The problem with doing enhancements using only a pre-processor is that you will lose some efficiency in the final code. As an example, if we were to add structures we would have to reference them as an indexed array element. However, a modified compiler would know the address of a statically defined structure element, and would just use the appropriate offset to access it. Of course, this could be done as a post-processing step on the bytecodes, but that would require modifying jump offsets, which I think would be more complicated.
Dave, you're right, of course: there might be some loss of code efficiency if Core Spin is not expressive enough to accommodate these kinds of changes without going directly to the byte codes. I guess the question then becomes, "To what extent is this a problem, and can we live with it?" In the case of structures, I'm sure that a preprocessor could produce usable Core Spin code. One just has to decide if the presumptive loss of efficiency is severe enough to justify forking the core language instead.
I suppose another option would be to permit embedded bytecodes in Core Spin, but that seems very ugly to me, and I'm sure there are some contextual issues that would prevent it from working. (I can well image that Roy is cringing, at this point. )
If you want C-style >=, write a preprocessor that replaces them with => so it can be compiled correctly by the Core Spin compiler, and name that preprocessor in your source file under the PRE section. It would only apply to that one file and not to other objects that may have their own PRE sections.
That does mean that people reading the source code will never know what it means, though, since any and all language constructs could be over-ridden by PRE. Similarly, code can't be copied between objects -- you can share the whole object, but cannot cut and paste or try to learn and duplicate. This seems like a fairly serious drawback in the education market (although perhaps they're all switching over to C now anyway).
If the preprocessor script is given in-line at least the reader would have some chance to figure out what changes the source code is undergoing.
Perhaps the best solution would be a new language ESpin for those advanced Spin hackers that want a flexible language syntax. ESpin could be implemented as a preprocessor and output regular .spin files, so it could interoperate with existing Spin solutions, and ESpin objects could be exported for use with other compilers. Such a preprocessor would be following the well established tradition of C++ (which started life as a preprocessor for C), Ratfor, and so on.
That does mean that people reading the source code will never know what it means, though, since any and all language constructs could be over-ridden by PRE. Similarly, code can't be copied between objects -- you can share the whole object, but cannot cut and paste or try to learn and duplicate. This seems like a fairly serious drawback in the education market (although perhaps they're all switching over to C now anyway).
There's no disadvantage there at all, since the Core Spin preprocessor output will always be available. A specialized compiler would be a problem, though, since it would obscure the differences between the specialized source and Core Spin.
There's no disadvantage there at all, since the Core Spin preprocessor output will always be available. A specialized compiler would be a problem, though, since it would obscure the differences between the specialized source and Core Spin.
Sorry, I think you have this backward. The Core Spin preprocessor output is only available after the compiler is run, and is not the "preferred" version of the source (pretty much by definition). So if a beginner picks up a .spin file that contains a PRE section (i.e. it hasn't been preprocessed yet) and tries to modify it, or to take source code from it and use it elsewhere, they could get into big trouble. Moreover, there's no guarantee that the output Core Spin code will work correctly in an Enhanced Spin file, since the preprocessor may change the meanings of things (to use Heater's example, the Core Spin construct <= may mean something very different in an Enhanced Spin file). I guess they could work around this by never trying to use the Enhanced Spin source code directly, but then what's the point?
If Enhanced Spin files were called something different, though (like .espin) and required a different compiler, then it would be immediately obvious that they're not necessarily interoperable with "regular" Spin. It's making Enhanced Spin and Core Spin look the same that obscures the difference, I think.
No, Eric, you have it backwards. Core Spin is, and always will be, the lowest common denominator from which compatibility springs. Obviously, if you try to import Enhanced Spin A statements into an Enhanced Spin B program, you're going to have problems. No one is arguing that. But, since both programs can be reduced by preprocessing to Core Spin, you can cut and paste their products with impunity into a hybrid Core Spin program. But if one object requires an eSpin A compiler and another an eSpin B compiler, you're screwed, since there's no common source expression to which both dialects map. Core Spin, in my proposal, is the lingua franca or Esperanto, if you will, for commonality. Otherwise, people will be forced to cut and paste byte codes -- well, that's obviously absurd, but you get my drift.
That's just my point: don't use a hacked-up compiler! If you want C-style >=, write a preprocessor that replaces them with => so it can be compiled correctly by the Core Spin compiler, and name that preprocessor in your source file under the PRE section. It would only apply to that one file and not to other objects that may have their own PRE sections.
-Phil
If you commit to the notion that the SPIN compiler will be altered and the SPIN language will be extended, then a lot of your fears are assuaged.
SPIN can't stay stagnant, just like C and C++ have revisions to the language, SPIN must march forward as well. Trying to bolt on features using a pre-processor is architecturally the wrong way to go about it. The pre-processor should be used to solve simple problems, example: C.
Trying to overload a SPIN file to do what Make does really well, well is just not the right way to go about it.
Yes, I'm saying things like "the right way".
To add things like the C >= syntax is trivially easy:
Trying to bolt on features using a pre-processor is architecturally the wrong way to go about it.
The only thing "wrong" with it is that it may not produce byte codes that are as efficient as those produced by a native compiler. I'm willing to concede that. And if your view of a preprocessor is limited by those that do shallow parsing and simple text substitution then, yes, this is the "wrong" approach for more complex language modification. But that's not a limitation I'm willing to accept. What I'm talking about when I say "preprocessor" or, better, "precompiler" is a program that can, if necessary, do deep parsing and compile an extended Spin program into a Core Spin target object. I did this with the S2 GUI, which compiles graphical icons into Spin. It works, and it works well.
The thing about preprocessors is that they are a democratizing force for extensibility. In assembly, for example, one has only to look as far as macros and how user-defined macros can simplify programming for particular apps. Yet, the same criticisms leveled here against preprocessors could easily be applied to macros. But who's ready to buck decades of acceptance for macros? Yet, many democratizing forces are met with fear and resistance by those seeking to control the narrative. Maybe that's a factor here; it's hard to say.
I would also argue that preprocessor hooks are a good way to promote experimentation with the language without someone having to understand compiler code that someone else has written. And, who knows? The fittest language modifications could easily make their way someday into Core Spin. But that will never happen without giving evolutionary forces a chance with bolt-on preprocessor tools.
And, yes, as you seem well aware, the "right way to go about it" is colored by experience and the way one has always done things. Spin broke a lot of "rules" when it was introduced, and there's no reason for that to stop now.
Phil,
My main problem with your idea is that it adds a new external dependency that now must be provided across all platforms in order to make said spin code usable by anyone. Also, I think ersmith is correct that making the source code that someone works with be NOT core spin and require a preprocessor in the PRE section to turn it into core spin is going to cause issues. You can't just move "source" code around, you have to preprocess it into core spin and then move the result around. That's just ugly. Alternatively you add the preprocessor dependency to the destination of the copy. Also, ugly. The more I think about it the more it's just a BIG can of worms. I'm really confused why you like it so much and want it?
We have plans to make Spin better, and add new features that will work for everyone on all platforms without external dependencies. I think this is the more sane path and beneficial for everyone.
My problem with your approach boils down to the question, "Who gets to decide which features are added?" We've suffered for six years already by the refusal of the compiler custodian(s) to make requested improvements. You're too busy with your new job to respond to requests, which means somebody else would have to grok your compiler to make changes to the language. And that's okay, because that's what open source is all about. But if two or more people do it, then the language gets forked, and that's a Bad ThingTM, because there will be objects out there that are irreversibly incompatible. That's why I think that Core Spin should remain the way it is and be used as a target language for enhancements -- at least until those enhancements prove themselves in the marketplace of ideas.
-Phil
Addendum: BTW, my idea for a PRE section is neither radical nor unprecedented: it's just a fancy way of providing Spin files with a shebang pragma.
my idea for a PRE section is neither radical nor unprecedented:
Any links to examples of such precedents?
"Who gets to decide which features are added?"
We need a committee
At least we need a language definition that is not dependent on any one piece of source code that happens to be "the" compiler today.
That's why I think that Core Spin should remain the way it is and be used as a target language for enhancements
I agree. "standard" Spin may evolve, perhaps more slowly than some would like. For those who want to get on with experimenting with new features a preprocessor may well be the way to go if they don't want to have a local hack of the compiler. As was done with C++. Eventually any good ideas that turn out to be usefull may get rolled back into standard Spin. Many others will fall by the wayside.
However, people capable of doing that sort of thing are also capable of running their own preprocessor over their sources before compiling them with the Spin compiler. Just use a script, or make. No need to muddy the Spin language from the get go with a PRE.
FWIW, here is a compiler/bytecode interpreter I wrote back in 1998. It's called Gencomp (generic compiler) and was used to take 2 files, a template that had source code and various bits of logic for how it was emitted, and a second file that was a list of parameters that the source code generate used to determine it's operation.
In practice the compiler was used to generate native source code for C and Java to interface with our database libraries. When you created a new stored procedure, you simply created a file that described the name and the input, output parameters, and whether the routine returned a row, rows, or just returned an output parameter.
It actually has 2 lexer/parsers in the program, 1 for the data definition file and 1 for the code template.
To invoke it: genc -t test.gram -o javaout=out.txt test.sp
You would name the output streams as parameters to the compiler, so you could have several file descriptors that you wrote to in the template. The block convention used brackets instead of braces because my boss wanted C source to be inlined without escaping.
I just want to reiterate, my intention is not to fork SPIN, it's to contribute to the main code line that represents SPIN going forward. The concerns over forks and "core" SPIN are unfounded.
There isn't any reason why Parallax wouldn't accept/endorse the changes, as long as a good argument can be made for the changes and it's done in the proper way.
I'm completely willing to hear Phil out on pre-processors, but our opinion on what constitutes a pre-processor seems to differ a bit. Things like the C pre-processor and assembly macros are good IMHO, anything that causes an external dependency, outside of standard SPIN dependencies, is a bad thing.
Phil,
I've only been busy in the short term because of moving and a bunch of "events" in the last month that I went to. I've flown more in the last few months than I have in the prior 40 years. Anyway, I will be working less hours by far at my new job than I was at my old one. I also have more freedom to work on side projects. Like I said earlier, I am working with Jeff on the validation tool, then we fully plan to start implementing new features. Chip is going to be involved in defining how Spin changes for the Prop 2, and Jeff and I are going to make those work with Prop 1 as much as we can. For example, Chip and I want to add the tertiary operator (? in C/C++ and vhdl), we plan to make that work across both Prop 1 and 2, it should be trivial.
There are things we still need to decide, but expect that Jeff, Chip, and I (and others most likely) to be doing a fair amount of work on Spin in the coming months. It'll be heavily Prop 2 slanted, but Jeff and I want there to be one compiler that can take in Prop 1 or Prop 2 source and output Prop 1 or Prop 2 binaries (obviously if you use features that are only on Prop 2, then you can't produce a Prop 1 binary, there are some reverse cases of this also, the way the counters and video hardware work are different enough that it's unlikely we could easily produce a Prop 2 binary for something that uses them significantly, especially when PASM is involved). The binary format will be significantly different between the two, most notably Prop 2 will be including the Spin Interpreter in the binary and the Prop 2 loading process with the encryption factors in as well, the byte codes will need to be different also.
An ideal goal is that you can take an existing spin program for the Prop 1, and compile it targeting the Prop 2 and have it work. This can work for a lot of stuff, but not everything.
Anyway, since ersmith graciously provided a patch for preprocessing spin with my compiler, I'll be able to get that in quicker than planned. I've already got his patch applied locally, and am working on some minor tweaks/additions to make it more complete. I've got it so that by default it's off, and you turn it on with a -p option. This is because I need to do more testing before I have it on by default, but with it off by default I can put it up there for people sooner.
Unnofficial - because Parallax hasn't chimed in on an plans or intentions for a Spin roadmap into the future
Propeller - well, duh, that's why we're here
Expo - that's so we have at least 3 opportunities to have committee steering meetings
2016 - trust me on this it will be here before much is agreed on in committee
Voting members/ technical committee:
Roy
Phil
Heater
Ersmith
Pedward
Dave Hein
Jazzed
Chip and I want to add the tertiary operator (? in C/C++ and vhdl)
Why in God's name?
Isn't Spin supposed to be simple for those new to programming? Those who hardly know what a program is even. For them the least amount of ugly, obfuscated, line noise you can have in the source the better. Do we really want Spin to look like C?
We have to take this idea to the Spin governing bodies subcommittee on operators tertiary operators working group. If there is no quorum it does not pass.
Comments
Now, a technical question about your compiler. At what stage during compilation does the compiler gather all of the objects in the "project?" Is this a pre-compilation matter, or does it occur as each object in the object tree gets compiled. The reason I ask has to do with processing namespaces in a preprocessor, rather than in the compiler itself.
Thanks,
-Phil
Didn't you mean the following?
PRE [b]sudo[/b] rm -rf * PUB start | i debug "Testing the Spin DEBUG function:\r\r" repeat i from -10 to 10 debug " The value of i is ", dec3 i, ", and i-squared is "I'm not sure how to make any preprocessor "safe." If it has read/write access to the filesystem, as it must, it will be intrinsically unsafe. A safe scripting language, like JavaScript, would not be a suitable candidate, since it does not have the necessary file access. And, with Perl or Python, anything is possible. Maybe it would be possible for the compiler or IDE to invoke the preprocessors in a sandbox to ensure that they play nice with the system.
-Phil
The simpler solution is that extended versions of Spin are identified by the IDE/Makefile/whatever, probably by means of the filename. That is, "foo.dspin" is a Spin file that has to be passed through the debug.exe preprocessor. People will know that a .dspin file is not regular Spin and needs a different (or additional) tool.
(The idea of an extensible language with a scripting language built in has some appeal -- it's part of what makes Scheme so powerful. But then comes the thorny question of what scripting language should be built in...)
Eric
I hope Roy can get to the pre-processor items soon. We need them for propeller-gcc items to replace BSTC entirely. There are some #ifdefs in the cache-cog spin/pasm files that open-source spin can't handle. I would ask for the #ifdefs to be removed, but that's probably the wrong direction to go.
Thanks.
For these reasons, I see core extensibility as a defense against incompatible forking, rather than a mechanism that promotes it. One way or another, forking will occur. If all the forked code can play nice together, that would be a benefit rather than a detriment.
Don't forget: the Spin compiler is not just a compiler: it's also a linker and loader. Without separating these functions, there has to be a way to compile disparate sources. Extensibility via preprocessing hooks is a way to do just that.
-Phil
I agree with the gist of what you are saying but you have hit on a very important point.
As it stands the Spin language has no formal definition, there are no rigorous specifications, there is no BNF definition of the syntax. Rather the language is defined by whatever Chips compiler accepts and does with it (Correct me if I am wrong sombody). Contrast to C or languages going back to ALGOL where the language syntax and semantics is very stricktly defined regardless of any particular implementation.
In theory (and practice) I could take the GCC C compiler and add features or "fix" this and that feature of the C language to my hearts content. Would anyone use it? Not if they are smart or only if my modifications are amazingly good and usefull. But then I have created a new language, it is no longer C as defined by the standards and I should call it something else (C++ for example).
So, given a rigorous definition of Spin we need not worry about all those possible forks of the compiler. They are just experimental offshoots that have features that may or may not get folded back to the Standard Spin.
You have a very good point. It applies to make files as well. And any other random stuff I fetch from the net. You have to "trust your sources". I still don't think including that kind of scripting into source code is a good idea. Better to "separate concerns" as they say.
I don't think this is true in general. Imagine:
I might create a Spin program that uses extentions that require some preprocessor. OK your PRE idea ensures that the required preprocessor is used. Think "ifdef ..." and such. As long as the user has the right preprocessor he is good to go.
BUT I also use a Spin compiler hacked up from the mainstream open source version. My hacked up version changes the sematics of operators , say I would rather use C style >= rather tha Spin style =>. Well at that point the poor sap trying to use my code is totally screwed.
In general I cannot see a way around this, there has to be a "one true language standard", perhaps with backwards compatible enhancements as features get added. Or ultimately the thing changes so much as to be a new language as in C and C++. Don't forget that C++ was a preprocessor for C originally.
I don't see the problem. Rather than using a hacked-up compiler, a preprocessor could just as easily make the operator substitutions. It all depends upon how deeply the preprocessor is written to parse the incoming code. Heck, it would even be possible for a preprocessor to accept a subset of C and spit out Core Spin, if someone wanted to go that far.
The whole idea is to make hacked-up compilers unnecessary.
-Phil
No.
I belive that if the sources you are trying to build into a single program are disparate (i.e. comming from different Spin language versions) then things have gotten so complex that it's time to give up and use something like GCC that can link any object file from anywhere if the format is compatible no matter what language it ´was originally written in.
Spin is a simple language, ideal for beginners, those that hardly know what a program is. Or for experineced programmers it let's them do simple thing simply. Let's keep it simple.
For those that grow in their programming demands and hit the limits there are already a thousand other solutions. No point in reinventing the wheel and killing Spin with "feature creep".
So it could. But in general it does not solve the problem. Imagine:
My hacked up Spin compiler uses C style >= for greater than or equal.
Some preprocessor fixes that to be => in Spin style to be compiled by a "standard" Spin compiler.
BUT my program also uses => to mean something complete different. What then?
A preprocessor cannot know about the sematics of what I have written. Only my compiler can.
P.S. Have I got my C a nd Spin styles the right way around, I'm confusing myself:)
I really like that idea. I can think of several dozen that I can live without
That's just my point: don't use a hacked-up compiler! If you want C-style >=, write a preprocessor that replaces them with => so it can be compiled correctly by the Core Spin compiler, and name that preprocessor in your source file under the PRE section. It would only apply to that one file and not to other objects that may have their own PRE sections.
-Phil
I suppose another option would be to permit embedded bytecodes in Core Spin, but that seems very ugly to me, and I'm sure there are some contextual issues that would prevent it from working. (I can well image that Roy is cringing, at this point.
-Phil
That does mean that people reading the source code will never know what it means, though, since any and all language constructs could be over-ridden by PRE. Similarly, code can't be copied between objects -- you can share the whole object, but cannot cut and paste or try to learn and duplicate. This seems like a fairly serious drawback in the education market (although perhaps they're all switching over to C now anyway).
If the preprocessor script is given in-line at least the reader would have some chance to figure out what changes the source code is undergoing.
Perhaps the best solution would be a new language ESpin for those advanced Spin hackers that want a flexible language syntax. ESpin could be implemented as a preprocessor and output regular .spin files, so it could interoperate with existing Spin solutions, and ESpin objects could be exported for use with other compilers. Such a preprocessor would be following the well established tradition of C++ (which started life as a preprocessor for C), Ratfor, and so on.
Eric
There's no disadvantage there at all, since the Core Spin preprocessor output will always be available. A specialized compiler would be a problem, though, since it would obscure the differences between the specialized source and Core Spin.
-Phil
Sorry, I think you have this backward. The Core Spin preprocessor output is only available after the compiler is run, and is not the "preferred" version of the source (pretty much by definition). So if a beginner picks up a .spin file that contains a PRE section (i.e. it hasn't been preprocessed yet) and tries to modify it, or to take source code from it and use it elsewhere, they could get into big trouble. Moreover, there's no guarantee that the output Core Spin code will work correctly in an Enhanced Spin file, since the preprocessor may change the meanings of things (to use Heater's example, the Core Spin construct <= may mean something very different in an Enhanced Spin file). I guess they could work around this by never trying to use the Enhanced Spin source code directly, but then what's the point?
If Enhanced Spin files were called something different, though (like .espin) and required a different compiler, then it would be immediately obvious that they're not necessarily interoperable with "regular" Spin. It's making Enhanced Spin and Core Spin look the same that obscures the difference, I think.
Eric
-Phil
If you commit to the notion that the SPIN compiler will be altered and the SPIN language will be extended, then a lot of your fears are assuaged.
SPIN can't stay stagnant, just like C and C++ have revisions to the language, SPIN must march forward as well. Trying to bolt on features using a pre-processor is architecturally the wrong way to go about it. The pre-processor should be used to solve simple problems, example: C.
Trying to overload a SPIN file to do what Make does really well, well is just not the right way to go about it.
Yes, I'm saying things like "the right way".
To add things like the C >= syntax is trivially easy:
{type_binary, 7, ">=", op_cmp_ae, false},Not all changes will be as simple as this.
The thing about preprocessors is that they are a democratizing force for extensibility. In assembly, for example, one has only to look as far as macros and how user-defined macros can simplify programming for particular apps. Yet, the same criticisms leveled here against preprocessors could easily be applied to macros. But who's ready to buck decades of acceptance for macros? Yet, many democratizing forces are met with fear and resistance by those seeking to control the narrative. Maybe that's a factor here; it's hard to say.
I would also argue that preprocessor hooks are a good way to promote experimentation with the language without someone having to understand compiler code that someone else has written. And, who knows? The fittest language modifications could easily make their way someday into Core Spin. But that will never happen without giving evolutionary forces a chance with bolt-on preprocessor tools.
And, yes, as you seem well aware, the "right way to go about it" is colored by experience and the way one has always done things. Spin broke a lot of "rules" when it was introduced, and there's no reason for that to stop now.
-Phil
My main problem with your idea is that it adds a new external dependency that now must be provided across all platforms in order to make said spin code usable by anyone. Also, I think ersmith is correct that making the source code that someone works with be NOT core spin and require a preprocessor in the PRE section to turn it into core spin is going to cause issues. You can't just move "source" code around, you have to preprocess it into core spin and then move the result around. That's just ugly. Alternatively you add the preprocessor dependency to the destination of the copy. Also, ugly. The more I think about it the more it's just a BIG can of worms. I'm really confused why you like it so much and want it?
We have plans to make Spin better, and add new features that will work for everyone on all platforms without external dependencies. I think this is the more sane path and beneficial for everyone.
Roy
My problem with your approach boils down to the question, "Who gets to decide which features are added?" We've suffered for six years already by the refusal of the compiler custodian(s) to make requested improvements. You're too busy with your new job to respond to requests, which means somebody else would have to grok your compiler to make changes to the language. And that's okay, because that's what open source is all about. But if two or more people do it, then the language gets forked, and that's a Bad ThingTM, because there will be objects out there that are irreversibly incompatible. That's why I think that Core Spin should remain the way it is and be used as a target language for enhancements -- at least until those enhancements prove themselves in the marketplace of ideas.
-Phil
Addendum: BTW, my idea for a PRE section is neither radical nor unprecedented: it's just a fancy way of providing Spin files with a shebang pragma.
Any links to examples of such precedents?
We need a committee
At least we need a language definition that is not dependent on any one piece of source code that happens to be "the" compiler today.
I agree. "standard" Spin may evolve, perhaps more slowly than some would like. For those who want to get on with experimenting with new features a preprocessor may well be the way to go if they don't want to have a local hack of the compiler. As was done with C++. Eventually any good ideas that turn out to be usefull may get rolled back into standard Spin. Many others will fall by the wayside.
However, people capable of doing that sort of thing are also capable of running their own preprocessor over their sources before compiling them with the Spin compiler. Just use a script, or make. No need to muddy the Spin language from the get go with a PRE.
In practice the compiler was used to generate native source code for C and Java to interface with our database libraries. When you created a new stored procedure, you simply created a file that described the name and the input, output parameters, and whether the routine returned a row, rows, or just returned an output parameter.
It actually has 2 lexer/parsers in the program, 1 for the data definition file and 1 for the code template.
To invoke it: genc -t test.gram -o javaout=out.txt test.sp
You would name the output streams as parameters to the compiler, so you could have several file descriptors that you wrote to in the template. The block convention used brackets instead of braces because my boss wanted C source to be inlined without escaping.
There are 4 major parts to the compiler:
There isn't any reason why Parallax wouldn't accept/endorse the changes, as long as a good argument can be made for the changes and it's done in the proper way.
I'm completely willing to hear Phil out on pre-processors, but our opinion on what constitutes a pre-processor seems to differ a bit. Things like the C pre-processor and assembly macros are good IMHO, anything that causes an external dependency, outside of standard SPIN dependencies, is a bad thing.
You can. It's open-source. Make a clone, and get to work.
I've only been busy in the short term because of moving and a bunch of "events" in the last month that I went to. I've flown more in the last few months than I have in the prior 40 years. Anyway, I will be working less hours by far at my new job than I was at my old one.
There are things we still need to decide, but expect that Jeff, Chip, and I (and others most likely) to be doing a fair amount of work on Spin in the coming months. It'll be heavily Prop 2 slanted, but Jeff and I want there to be one compiler that can take in Prop 1 or Prop 2 source and output Prop 1 or Prop 2 binaries (obviously if you use features that are only on Prop 2, then you can't produce a Prop 1 binary, there are some reverse cases of this also, the way the counters and video hardware work are different enough that it's unlikely we could easily produce a Prop 2 binary for something that uses them significantly, especially when PASM is involved). The binary format will be significantly different between the two, most notably Prop 2 will be including the Spin Interpreter in the binary and the Prop 2 loading process with the encryption factors in as well, the byte codes will need to be different also.
An ideal goal is that you can take an existing spin program for the Prop 1, and compile it targeting the Prop 2 and have it work. This can work for a lot of stuff, but not everything.
Anyway, since ersmith graciously provided a patch for preprocessing spin with my compiler, I'll be able to get that in quicker than planned. I've already got his patch applied locally, and am working on some minor tweaks/additions to make it more complete. I've got it so that by default it's off, and you turn it on with a -p option. This is because I need to do more testing before I have it on by default, but with it off by default I can put it up there for people sooner.
Roy
Unofficial Propeller Expo Spin 2016
Unnofficial - because Parallax hasn't chimed in on an plans or intentions for a Spin roadmap into the future
Propeller - well, duh, that's why we're here
Expo - that's so we have at least 3 opportunities to have committee steering meetings
2016 - trust me on this it will be here before much is agreed on in committee
Voting members/ technical committee:
Roy
Phil
Heater
Ersmith
Pedward
Dave Hein
Jazzed
Recording Secretary/referee/floor sweeper:
Mindrobots
Neutral corners will be assigned, debates will be in 5 minute rounds, you are allowed to "tag out" to someone on your sub-committee.
The 2016 target date is just to have a formal definition of the language, working tools will be released by 2018.
Play nice because in the end this is for the children....or the grandchildren depending on how effective the committee is.
Ok, now we have a committee! Let's git 'er done!!!
Why in God's name?
Isn't Spin supposed to be simple for those new to programming? Those who hardly know what a program is even. For them the least amount of ugly, obfuscated, line noise you can have in the source the better. Do we really want Spin to look like C?
We have to take this idea to the Spin governing bodies subcommittee on operators tertiary operators working group. If there is no quorum it does not pass.