@ke4pjw said:
Having a VER block could also (possibly?) help with changes in how directives behave. ie. The REPEAT loop enumeration change that happened a while back. Though, I bet that would make Chip and Eric's jobs more complex.
If I'm not wrong, the repeat behaviour is implemented in the runtime interpreter, the compiler should have a copy of the old interpreter (or be able to patch it) and upload the appropriate version.
While this may be "trivial", I remember sometime ago that the introduction of a new keyword (don't rememeber what, maybe field) caused a renumber of all bytecodes (existing binaries were not compatible with the new interpreter). In such case, the compiler not only needs the old interpreter but also generate a completely different bytecode. This would be very bad!
We almost need to have each version of PNut available and have the source code name which version it needs.
The source code does need to name which version it needs (I thought we had pretty much converged on having a version identifier or section at the start of files that need new keywords?)
But I don't think we need to keep old versions of PNut around, or old interpreters. We just need to avoid making breaking changes to the existing keywords (of the REPEAT loop sort). Instead any new behavior should be marked by new keywords.
Maybe we could take a stealthy approach, where if we see a keyword being used a method, variable, or constant name, that keyword is removed from the symbol table and reentered as the type the user has declared?
I think that's kind of what @macca is doing. That will work well for new keywords that are restricted to PUB and PRI sections, but less well for things that update OBJ, CON, VAR, or DAT (where keywords and declarations can be mixed).
@ke4pjw said:
Having a VER block could also (possibly?) help with changes in how directives behave. ie. The REPEAT loop enumeration change that happened a while back. Though, I bet that would make Chip and Eric's jobs more complex.
If I'm not wrong, the repeat behaviour is implemented in the runtime interpreter, the compiler should have a copy of the old interpreter (or be able to patch it) and upload the appropriate version.
While this may be "trivial", I remember sometime ago that the introduction of a new keyword (don't rememeber what, maybe field) caused a renumber of all bytecodes (existing binaries were not compatible with the new interpreter). In such case, the compiler not only needs the old interpreter but also generate a completely different bytecode. This would be very bad!
We almost need to have each version of PNut available and have the source code name which version it needs.
The source code does need to name which version it needs (I thought we had pretty much converged on having a version identifier or section at the start of files that need new keywords?)
But I don't think we need to keep old versions of PNut around, or old interpreters. We just need to avoid making breaking changes to the existing keywords (of the REPEAT loop sort). Instead any new behavior should be marked by new keywords.
Maybe we could take a stealthy approach, where if we see a keyword being used a method, variable, or constant name, that keyword is removed from the symbol table and reentered as the type the user has declared?
I think that's kind of what @macca is doing. That will work well for new keywords that are restricted to PUB and PRI sections, but less well for things that update OBJ, CON, VAR, or DAT (where keywords and declarations can be mixed).
We could identify all user symbols in each section. Why not? This could totally solve the problem, couldn't it? A warning could pop up after compile, identifying which keywords were canceled and became user symbols.
The REPEAT-var debacle involves v35n-v37. That is a real pain to handle. Edit: And it's impossible to resolve when mixing objects that used either type of REPEAT. This will have to be chalked up as a youthful indiscretion with lasting consequences.
Maybe we could take a stealthy approach, where if we see a keyword being used a method, variable, or constant name, that keyword is removed from the symbol table and reentered as the type the user has declared?
I think that's kind of what @macca is doing. That will work well for new keywords that are restricted to PUB and PRI sections, but less well for things that update OBJ, CON, VAR, or DAT (where keywords and declarations can be mixed).
We could identify all user symbols in each section. Why not? This could totally solve the problem, couldn't it? A warning could pop up after compile, identifying which keywords were canceled and became user symbols.
That's fine for something like a compiler that's already doing multiple passes over the source code, but it seems like it could be a real pain for other tools like editors. EDIT: Actually come to think of it it may not always be fine: if the user redefines something like IF or WHILE then reading the code could become very "interesting". I still think the VER 43 solution (or something like it) is more straightforward in general.
The REPEAT-var debacle involves v35n-v37. That is a real pain to handle. Edit: And it's impossible to resolve when mixing objects that used either type of REPEAT. This will have to be chalked up as a youthful indiscretion with lasting consequences.
Yes. I think we have to draw a line and say v41 (or something similar) is the "base" Spin2 and nothing earlier is supported. After that though we should try not to break existing code, so that we can get a stable OBEX.
Maybe we could take a stealthy approach, where if we see a keyword being used a method, variable, or constant name, that keyword is removed from the symbol table and reentered as the type the user has declared?
I think that's kind of what @macca is doing. That will work well for new keywords that are restricted to PUB and PRI sections, but less well for things that update OBJ, CON, VAR, or DAT (where keywords and declarations can be mixed).
We could identify all user symbols in each section. Why not? This could totally solve the problem, couldn't it? A warning could pop up after compile, identifying which keywords were canceled and became user symbols.
That's fine for something like a compiler that's already doing multiple passes over the source code, but it seems like it could be a real pain for other tools like editors. EDIT: Actually come to think of it it may not always be fine: if the user redefines something like IF or WHILE then reading the code could become very "interesting". I still think the VER 43 solution (or something like it) is more straightforward in general.
The REPEAT-var debacle involves v35n-v37. That is a real pain to handle. Edit: And it's impossible to resolve when mixing objects that used either type of REPEAT. This will have to be chalked up as a youthful indiscretion with lasting consequences.
Yes. I think we have to draw a line and say v41 (or something similar) is the "base" Spin2 and nothing earlier is supported. After that though we should try not to break existing code, so that we can get a stable OBEX.
If you are going to create a new block/section name just for versioning, I'm wondering whether it could be useful to make it more of a general container for including any future compiler version requirements and/or switches we come up with, rather than just a statement for a single version number. For now we might just need a version number in it but later if you do change any behaviour again or add features that might break stuff then it could be indicated by another keyword in this section. So instead of just a VER 41 or something we could have this as a USES block:
USES
V41
REPEAT_VAR_END
' ... (any future things that aren't just version related but might alter the compiler behavior - like compiler command line switches)
PUB blah()
This would allow us to "solve" the repeat var stuff at least as a one off. If the compiler sees the REPEAT_VAR_END (or a suitable name) it could still generate the repeat code that "special" way vs the current/original way. Then the only thing that has to change (for the small number of people who really insist it needs to work like that because they can't use the latest compiler otherwise), is the code requiring that behaviour (just once) to include this USES block in the file if it hasn't already been changed back to behave the current way. Anyone running an old compiler will simple barf on the USES block seen after that, forcing the upgrade to the newer version that supports it.
This new USES block should remain fairly generic and could also indicate other things later like libraries if the language ever evolves in that way for example.
Moving forward however, you will really need to make your compilers support the superset of all old versions, honouring their old features and performing the legacy behaviour indicated by the older version number if it differs to the new, otherwise old code will not be usable anymore with upgraded compilers. And the worst thing then will be if you bring in someone's older code as a shared object and you have to then go hunt down an older matching compiler version just to build it, which may not be compatible with your own code if it uses any newer features requiring a newer compiler. Ending up in situations like that would truly suck hard.
USES could be some other short name like NEEDS, USE, REQ (requires/request) etc. Examples...
USE
V41 ' minimum spin compiler version
OPTS ' optimization enabled
FP_MATH ' floating point math library included
NO_DEAD_CODE ' dead code removal
I'm not very keen on flags that imply changes to the interpreter's behavior, but one possibility is to modularize the interpreter's source with the use of preprocessor directives and recompile it along with the program and appropriate flags (this of course requires PNut to implement the preprocessor directives, Spin Tools IDE and flexprop already implement them, albeit in a slightly incompatible variant). This will not solve the problem of two sources requiring different behavior, this should generate an error whenever the source requirements are not compatible with the interpreter. And it would be a real pain if the change required renumbering the bytecodes as has happened in the past.
Let's not let the perfect become the enemy of the good. Please, can we just agree on a very simple, basic version standard that will allow us to extend the Spin2 language without breaking existing code? If we decide some other new cool features are needed in the future, we'll be able to add those.
The proposal that seemed to have the most momentum was a VER block with a single entry (a version number) and which has to come first in the file (preferably on the first line). Once that's in, we can expand on it.
The only disadvantages I see to the VER block are (1) it introduces yet another keyword which could break existing code, and (2) code with a VER block cannot compile with an older compiler, even if the code does not actually use any new features. Both of these could be fixed by putting the version declaration in a comment. But again, let's not let the perfect get in the way of the good. (2) is not really that big a deal in practice, and (1) can be avoided if the VER keyword can only appear first in the file (the compiler can remove "VER" from the keyword table if we see some other section like CON or DAT before VER). So I'm perfectly happy to add VER to flexspin as is.
Ultimately the choice is up to @cgracey though, because his compiler defines the Spin2 standard. All I will say is that the status quo isn't very sustainable, unless we want the OBEX to bit-rot.
@ersmith said:
The only disadvantages I see to the VER block are (1) it introduces yet another keyword which could break existing code, and (2) code with a VER block cannot compile with an older compiler, even if the code does not actually use any new features. Both of these could be fixed by putting the version declaration in a comment....
Those were my immediate thoughts too. Using a {comment tag} or verbose CON statement (just like the DEBUG configuration statements do), would totally avoid any further breakage, and also set a precedent for future expansion (just as the CON debug_* options exist and might grow, then the CON compiler_* options might grow).
Introducing yet another keyword just to work around some other new keyword problem doesn't feel like the right solution to me. I'd rather see the original issue dealt with - ie. use BYTE() etc.. instead of introducing BYTES(). And set a simple comment flag for moving forward. Otherwise we're making a mountain out of a molehill.
I always liked the elegant way the BasicSTAMP compiler dealt with the language versions and would have liked that to be adopted for spin. Would have saved all the file switching pain with PropellerTool too.
{$P1} {$P2} {$V32} etc..
Between comments and CON blocks, there's already two ways there to set the compiler version and processor type, without needing to add new keywords and break things.
Don't go the Arduino direction !
After some time, you can't build you working code again with the updated tool.
Frustrated many times, finding break points inside tested and validated programs, due to updated IDE, and al the surrounding stuffs
And just like Jon told, how to deal with different version extracted from the obex?
Can a compiler, jump on every different version, inside the "OBJ" part of a Spin2 program?
This is one reason, I do't really use the P2 now, I only play, and look.
Waiting the P2 surounding get mature and stable
@cgracey : Have you thought about how, or if, you want to provide version protection in PNut?
For now in flexprop I've done kind of what Macca does: by default the new keywords (starting at FIELD) can be overridden by user variables. When this happens a warning is printed. I've also added an option to have an explicit version like {$ver 41} in the file, in which case the keywords up to that version become "hard" and can no longer be overridden, and keywords with a later version are ignored completely. This is probably all overkill, but in the absence of official guidance I'm trying to keep the compiler flexible.
@ersmith said:
@cgracey : Have you thought about how, or if, you want to provide version protection in PNut?
For now in flexprop I've done kind of what Macca does: by default the new keywords (starting at FIELD) can be overridden by user variables. When this happens a warning is printed. I've also added an option to have an explicit version like {$ver 41} in the file, in which case the keywords up to that version become "hard" and can no longer be overridden, and keywords with a later version are ignored completely. This is probably all overkill, but in the absence of official guidance I'm trying to keep the compiler flexible.
I am thinking about all this.
I kind of like the syntax {v42}, since it doesn't need a new block and it's terse. It would just have to be documented up front.
I am kind of bummed about requiring ANYTHING to be added for new programs. That is my hang-up.
What appeals to me is checking for name conflicts against NEW keywords, and then letting them be user names, with a warning.
@ersmith said:
@cgracey : Have you thought about how, or if, you want to provide version protection in PNut?
For now in flexprop I've done kind of what Macca does: by default the new keywords (starting at FIELD) can be overridden by user variables. When this happens a warning is printed. I've also added an option to have an explicit version like {$ver 41} in the file, in which case the keywords up to that version become "hard" and can no longer be overridden, and keywords with a later version are ignored completely. This is probably all overkill, but in the absence of official guidance I'm trying to keep the compiler flexible.
I am thinking about all this.
I kind of like the syntax {v42}, since it doesn't need a new block and it's terse. It would just have to be documented up front.
I am kind of bummed about requiring ANYTHING to be added for new programs. That is my hang-up.
What appeals to me is checking for name conflicts against NEW keywords, and then letting them be user names, with a warning.
If you go this route, please make it e.g. {$v42} and not just {v42}. The latter is just a funny-looking comment, while the former is clearly a directive.
But, as others have said, all these versioning problems could be eliminated by just calling the new keywords byte, word, and long. Every syntax highlighter already has to deal with those keywords (as well as e.g. and and or) already having multiple fundamentally different meanings - what's one more overloading?
@ersmith said:
@cgracey : Have you thought about how, or if, you want to provide version protection in PNut?
For now in flexprop I've done kind of what Macca does: by default the new keywords (starting at FIELD) can be overridden by user variables. When this happens a warning is printed. I've also added an option to have an explicit version like {$ver 41} in the file, in which case the keywords up to that version become "hard" and can no longer be overridden, and keywords with a later version are ignored completely. This is probably all overkill, but in the absence of official guidance I'm trying to keep the compiler flexible.
I am thinking about all this.
I kind of like the syntax {v42}, since it doesn't need a new block and it's terse. It would just have to be documented up front.
I am kind of bummed about requiring ANYTHING to be added for new programs. That is my hang-up.
What appeals to me is checking for name conflicts against NEW keywords, and then letting them be user names, with a warning.
If you go this route, please make it e.g. {$v42} and not just {v42}. The latter is just a funny-looking comment, while the former is clearly a directive.
But, as others have said, all these versioning problems could be eliminated by just calling the new keywords byte, word, and long. Every syntax highlighter already has to deal with those keywords (as well as e.g. and and or) already having multiple fundamentally different meanings - what's one more overloading?
We are going back to BYTE(), WORD(), and LONG().
I was thinking comment words like Spin2_v42 could be used to set compiler level. You could put a $ in front of it. Maybe we'll require it. Not sure.
@ersmith said:
@cgracey : Have you thought about how, or if, you want to provide version protection in PNut?
For now in flexprop I've done kind of what Macca does: by default the new keywords (starting at FIELD) can be overridden by user variables. When this happens a warning is printed. I've also added an option to have an explicit version like {$ver 41} in the file, in which case the keywords up to that version become "hard" and can no longer be overridden, and keywords with a later version are ignored completely. This is probably all overkill, but in the absence of official guidance I'm trying to keep the compiler flexible.
I am thinking about all this.
I kind of like the syntax {v42}, since it doesn't need a new block and it's terse. It would just have to be documented up front.
I am kind of bummed about requiring ANYTHING to be added for new programs. That is my hang-up.
What appeals to me is checking for name conflicts against NEW keywords, and then letting them be user names, with a warning.
If you go this route, please make it e.g. {$v42} and not just {v42}. The latter is just a funny-looking comment, while the former is clearly a directive.
But, as others have said, all these versioning problems could be eliminated by just calling the new keywords byte, word, and long. Every syntax highlighter already has to deal with those keywords (as well as e.g. and and or) already having multiple fundamentally different meanings - what's one more overloading?
We are going back to BYTE(), WORD(), and LONG().
I was thinking comment words like Spin2_v42 could be used to set compiler level. You could put a $ in front of it. Maybe we'll require it. Not sure.
Going back to BYTE(), WORD(), and LONG() won't help with LSTRING(). Honestly I think it's time to bite the bullet and provide some kind of long term solution for the compiler, whether it's a version number or having a way for user variables to override new keywords. Both @macca and I have already implemented the new keywords in a backwards compatible way in our respective compilers.
Comments
The source code does need to name which version it needs (I thought we had pretty much converged on having a version identifier or section at the start of files that need new keywords?)
But I don't think we need to keep old versions of PNut around, or old interpreters. We just need to avoid making breaking changes to the existing keywords (of the REPEAT loop sort). Instead any new behavior should be marked by new keywords.
I think that's kind of what @macca is doing. That will work well for new keywords that are restricted to PUB and PRI sections, but less well for things that update OBJ, CON, VAR, or DAT (where keywords and declarations can be mixed).
We could identify all user symbols in each section. Why not? This could totally solve the problem, couldn't it? A warning could pop up after compile, identifying which keywords were canceled and became user symbols.
The REPEAT-var debacle involves v35n-v37. That is a real pain to handle. Edit: And it's impossible to resolve when mixing objects that used either type of REPEAT. This will have to be chalked up as a youthful indiscretion with lasting consequences.
That's fine for something like a compiler that's already doing multiple passes over the source code, but it seems like it could be a real pain for other tools like editors. EDIT: Actually come to think of it it may not always be fine: if the user redefines something like
IForWHILEthen reading the code could become very "interesting". I still think theVER 43solution (or something like it) is more straightforward in general.Yes. I think we have to draw a line and say v41 (or something similar) is the "base" Spin2 and nothing earlier is supported. After that though we should try not to break existing code, so that we can get a stable OBEX.
Agreed.
If you are going to create a new block/section name just for versioning, I'm wondering whether it could be useful to make it more of a general container for including any future compiler version requirements and/or switches we come up with, rather than just a statement for a single version number. For now we might just need a version number in it but later if you do change any behaviour again or add features that might break stuff then it could be indicated by another keyword in this section. So instead of just a VER 41 or something we could have this as a USES block:
This would allow us to "solve" the repeat var stuff at least as a one off. If the compiler sees the REPEAT_VAR_END (or a suitable name) it could still generate the repeat code that "special" way vs the current/original way. Then the only thing that has to change (for the small number of people who really insist it needs to work like that because they can't use the latest compiler otherwise), is the code requiring that behaviour (just once) to include this USES block in the file if it hasn't already been changed back to behave the current way. Anyone running an old compiler will simple barf on the USES block seen after that, forcing the upgrade to the newer version that supports it.
This new USES block should remain fairly generic and could also indicate other things later like libraries if the language ever evolves in that way for example.
Moving forward however, you will really need to make your compilers support the superset of all old versions, honouring their old features and performing the legacy behaviour indicated by the older version number if it differs to the new, otherwise old code will not be usable anymore with upgraded compilers. And the worst thing then will be if you bring in someone's older code as a shared object and you have to then go hunt down an older matching compiler version just to build it, which may not be compatible with your own code if it uses any newer features requiring a newer compiler. Ending up in situations like that would truly suck hard.
USES could be some other short name like NEEDS, USE, REQ (requires/request) etc. Examples...
USE V41 ' minimum spin compiler version OPTS ' optimization enabled FP_MATH ' floating point math library included NO_DEAD_CODE ' dead code removalI'm not very keen on flags that imply changes to the interpreter's behavior, but one possibility is to modularize the interpreter's source with the use of preprocessor directives and recompile it along with the program and appropriate flags (this of course requires PNut to implement the preprocessor directives, Spin Tools IDE and flexprop already implement them, albeit in a slightly incompatible variant). This will not solve the problem of two sources requiring different behavior, this should generate an error whenever the source requirements are not compatible with the interpreter. And it would be a real pain if the change required renumbering the bytecodes as has happened in the past.
Let's not let the perfect become the enemy of the good. Please, can we just agree on a very simple, basic version standard that will allow us to extend the Spin2 language without breaking existing code? If we decide some other new cool features are needed in the future, we'll be able to add those.
The proposal that seemed to have the most momentum was a VER block with a single entry (a version number) and which has to come first in the file (preferably on the first line). Once that's in, we can expand on it.
The only disadvantages I see to the VER block are (1) it introduces yet another keyword which could break existing code, and (2) code with a VER block cannot compile with an older compiler, even if the code does not actually use any new features. Both of these could be fixed by putting the version declaration in a comment. But again, let's not let the perfect get in the way of the good. (2) is not really that big a deal in practice, and (1) can be avoided if the VER keyword can only appear first in the file (the compiler can remove "VER" from the keyword table if we see some other section like CON or DAT before VER). So I'm perfectly happy to add VER to flexspin as is.
Ultimately the choice is up to @cgracey though, because his compiler defines the Spin2 standard. All I will say is that the status quo isn't very sustainable, unless we want the OBEX to bit-rot.
Those were my immediate thoughts too. Using a {comment tag} or verbose CON statement (just like the DEBUG configuration statements do), would totally avoid any further breakage, and also set a precedent for future expansion (just as the CON debug_* options exist and might grow, then the CON compiler_* options might grow).
Introducing yet another keyword just to work around some other new keyword problem doesn't feel like the right solution to me. I'd rather see the original issue dealt with - ie. use BYTE() etc.. instead of introducing BYTES(). And set a simple comment flag for moving forward. Otherwise we're making a mountain out of a molehill.
I always liked the elegant way the BasicSTAMP compiler dealt with the language versions and would have liked that to be adopted for spin. Would have saved all the file switching pain with PropellerTool too.
{$P1} {$P2} {$V32} etc..
Between comments and CON blocks, there's already two ways there to set the compiler version and processor type, without needing to add new keywords and break things.
Don't go the Arduino direction !
After some time, you can't build you working code again with the updated tool.
Frustrated many times, finding break points inside tested and validated programs, due to updated IDE, and al the surrounding stuffs
And just like Jon told, how to deal with different version extracted from the obex?
Can a compiler, jump on every different version, inside the "OBJ" part of a Spin2 program?
This is one reason, I do't really use the P2 now, I only play, and look.
Waiting the P2 surounding get mature and stable
Guess this is the reason PropTool has the option of including the tool itself in the archive...
Must have been a time when Spin1 code was also evolving...
@cgracey : Have you thought about how, or if, you want to provide version protection in PNut?
For now in flexprop I've done kind of what Macca does: by default the new keywords (starting at FIELD) can be overridden by user variables. When this happens a warning is printed. I've also added an option to have an explicit version like
{$ver 41}in the file, in which case the keywords up to that version become "hard" and can no longer be overridden, and keywords with a later version are ignored completely. This is probably all overkill, but in the absence of official guidance I'm trying to keep the compiler flexible.I am thinking about all this.
I kind of like the syntax {v42}, since it doesn't need a new block and it's terse. It would just have to be documented up front.
I am kind of bummed about requiring ANYTHING to be added for new programs. That is my hang-up.
What appeals to me is checking for name conflicts against NEW keywords, and then letting them be user names, with a warning.
If you go this route, please make it e.g.
{$v42}and not just{v42}. The latter is just a funny-looking comment, while the former is clearly a directive.But, as others have said, all these versioning problems could be eliminated by just calling the new keywords
byte,word, andlong. Every syntax highlighter already has to deal with those keywords (as well as e.g.andandor) already having multiple fundamentally different meanings - what's one more overloading?We are going back to BYTE(), WORD(), and LONG().
I was thinking comment words like Spin2_v42 could be used to set compiler level. You could put a $ in front of it. Maybe we'll require it. Not sure.
Going back to BYTE(), WORD(), and LONG() won't help with LSTRING(). Honestly I think it's time to bite the bullet and provide some kind of long term solution for the compiler, whether it's a version number or having a way for user variables to override new keywords. Both @macca and I have already implemented the new keywords in a backwards compatible way in our respective compilers.