Shop OBEX P1 Docs P2 Docs Learn Events
SPIN improvements — Parallax Forums

SPIN improvements

pedwardpedward Posts: 1,642
edited 2012-10-11 19:25 in Propeller 1
I've been thinking about what SPIN needs to improve as a language. I've been thinking of hacking on the OSS compiler to add support for a few things. I saw in the P2 polishing thread there are others with a similar interest in improving SPIN.

I'd like this thread to be a place where people can enumerate their opinions on what needs to be added to SPIN.

I'm thinking about things that are compiler only, so the runtime doesn't need to change. This would essentially be SPINv1.5 and SPIN2.0 would coincide with the Prop 2 and that runtime. SPIN 1.5 would be constrained to only features that can be implemented with the existing runtime, so it would be compatible with the Prop1. SPIN2.0 would be exclusive to the Prop 2.

Some things I've discussed with others:

-Static type checking support (typing checking of parameters passed to methods, perhaps dynamic type conversion)
-Float (single precision) native 32bit support
-Strings
-Structures
-Name space (using the PERL :: notation?)
«13456710

Comments

  • David BetzDavid Betz Posts: 14,516
    edited 2012-10-02 11:50
    pedward wrote: »
    I've been thinking about what SPIN needs to improve as a language. I've been thinking of hacking on the OSS compiler to add support for a few things. I saw in the P2 polishing thread there are others with a similar interest in improving SPIN.

    I'd like this thread to be a place where people can enumerate their opinions on what needs to be added to SPIN.

    I'm thinking about things that are compiler only, so the runtime doesn't need to change. This would essentially be SPINv1.5 and SPIN2.0 would coincide with the Prop 2 and that runtime. SPIN 1.5 would be constrained to only features that can be implemented with the existing runtime, so it would be compatible with the Prop1. SPIN2.0 would be exclusive to the Prop 2.

    Some things I've discussed with others:

    -Static type checking support (typing checking of parameters passed to methods, perhaps dynamic type conversion)
    -Float (single precision) native 32bit support
    -Strings
    -Structures
    -Name space (using the PERL :: notation?)
    By OSS compiler I assume you mean the one that Roy wrote, right? If you're going to start hacking it, could you add the preprocessor stuff from BSTC along with #include? That would really help the PropGCC project! :-)
  • jazzedjazzed Posts: 11,803
    edited 2012-10-02 11:59
    Must be 32 bit address capable of course. The current interpreter is mostly 16 bits, but a 32 bit version is not impossible if the code, data, and stack pointers are moved beyond the first 16 byte object header.
  • Heater.Heater. Posts: 21,230
    edited 2012-10-02 12:01
    Don't forget that the Prop II will not have it's Spin interpreter in ROM but rather the interpreter will be downloaded to RAM along with the rest of your code.

    That means anyone is at liberty to add, subtract, change anything they want in the Spin iterpreter to support whatever language features they like. You want unsigned longs? No problem.

    I'm bit worried about forking the Spin compiler and adding/changing functionality. Last thing we want is a plethora of incompatible langage dialects. It would be great if there were only one "True Spin" maintained by Roy and Parallax. People could fork and hack whatever versions they like and if anything good comes out and they are accpeted those changes could be submitted back to the "True Spin" Backwards compatibility would have to be maintained.
  • pedwardpedward Posts: 1,642
    edited 2012-10-02 12:13
    Regarding "True SPIN", it's my intent to make these changes in accordance with Parallax's wishes. That would mean I would do what I can to make it the "official" SPIN compiler.

    There hasn't been an initiative from Ken at Parallax to do this, but everyone agrees that SPIN needs to be improve. I was taking the initiative to do this without someone asking. I want to see it get done, and I think this is the best way, just do it.

    The beauty is that I'm just asking for suggestions, because I feel there are other people with more experience with SPIN that can contribute ideas. I don't necessarily need adopt all of the ideas provided, but I'm not naive enough to think that I know everything.
  • ersmithersmith Posts: 6,054
    edited 2012-10-02 12:13
    David Betz wrote: »
    By OSS compiler I assume you mean the one that Roy wrote, right? If you're going to start hacking it, could you add the preprocessor stuff from BSTC along with #include? That would really help the PropGCC project! :-)

    The spin2cpp preprocessor is designed to stand on its own and just take a file as input and a buffer of characters as output. It could probably be incorporated into Roy's compiler.
  • David BetzDavid Betz Posts: 14,516
    edited 2012-10-02 13:09
    ersmith wrote: »
    The spin2cpp preprocessor is designed to stand on its own and just take a file as input and a buffer of characters as output. It could probably be incorporated into Roy's compiler.
    That would be great! Does your preprocessor support #include?
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2012-10-02 13:30
    I don't think one needs to, or should, mess with Roy's compiler at all. It would be better to maintain the language it compiles as Core Spin and apply any enhancements to the language in a precompilation stage that maps to Core Spin. That way forking becomes less of a problem and people will be free to experiment with the assurance that they can always publish a precompiled version of their code that will work with BST, the Prop Tool, or Roy's compiler.

    Granted, this approach may not accommodate every enhancement one might wish to make, but things like object name spaces and embedded floating point would not be an issue.

    -Phil
  • David BetzDavid Betz Posts: 14,516
    edited 2012-10-02 13:33
    I don't think one needs to, or should, mess with Roy's compiler at all. It would be better to maintain the language it compiles as Core Spin and apply any enhancements to the language in a precompilation stage that maps to Core Spin. That way forking becomes less of a problem and people will be free to experiment with the assurance that they can always publish a precompiled version of their code that will work with BST, the Prop Tool, or Roy's compiler.

    Granted, this approach may not accommodate every enhancement one might wish to make, but things like object name spaces and embedded floating point would not be an issue.

    -Phil
    How would you get the Propeller Tool to invoke the preprocessor before invoking the compiler? Is there some support for Propeller Tool plug-ins?
  • ersmithersmith Posts: 6,054
    edited 2012-10-02 13:58
    David Betz wrote: »
    That would be great! Does your preprocessor support #include?

    Yes.
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2012-10-02 14:02
    Dave Betz wrote:
    How would you get the Propeller Tool to invoke the preprocessor before invoking the compiler? Is there some support for Propeller Tool plug-ins?

    The preprocessor has to invoke the compiler. From the Prop Tool you could to it the way I do with a Perl/Tk script that attaches a button to the Prop Tool and, when invoked, sends the Prop Tool keystrokes and mouse commands. That's the way my '.debug processor works, which controls both the Prop Tool and PST and uses Propellent as the back-end compiler. Granted, it would be easier if the Prop Tool provided hooks for stuff like this, but it doesn't and probably never will.

    -Phil
  • Invent-O-DocInvent-O-Doc Posts: 768
    edited 2012-10-02 14:04
    An architectural question here. (I like the thread, by the way.) I'm sure we can enhance spin in a number of ways, but isn't a spin interpreter limited in size due to having to run inside a 512 long COG?

    I mean, many enhancements have to take place at the 'pre-compilation' stage, right?
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2012-10-02 14:07
    None of the enhancements being considered would require changes to the interpreter, AFAIK. My proposal is to make them all at the preprocessor level, but it's also possible to compile the enhanced stuff more directly into the current byte codes. The latter might, in some cases, produce more efficient code, but at a greater risk of creating irreversible forks to the language itself.

    One possible downside to the preprocessor model is the reporting of compilation errors, since those will occur in the Core Spin target and have to be mapped back to the original code.

    -Phil
  • David BetzDavid Betz Posts: 14,516
    edited 2012-10-02 14:13
    None of the enhancements being considered would require changes to the interpreter, AFAIK. My proposal is to make them all at the preprocessor level, but it's also possible to compile the enhanced stuff more directly into the current byte codes. The latter might, in some cases, produce more efficient code, but at a greater risk of creating irreversible forks to the language itself.

    One possible downside to the preprocessor model is the reporting of compilation errors, since those will occur in the Core Spin target and have to be mapped back to the original code.

    -Phil
    I wonder if it would be possible to build Roy's compiler as a DLL that can be called from the Propeller Tool in the same way that the current compiler is. That way it could replace the current Spin compiler even in the Propeller Tool and there would be only one compiler to maintain. I'm assuming BST will go away eventually since it isn't maintained or developed anymore. This would allow extensions like the preprocessor directives to be incorporated directly into Roy's compiler. I think Roy planned on doing that anyway.
  • Heater.Heater. Posts: 21,230
    edited 2012-10-02 14:23
    Isn't the Propeller Tool being end of lifed? That is to say it will not be pushed forward for the Prop II.
    I'm assuming the future is SimpleIDE and Roys compiler. Works fine for me so far.
  • Roy ElthamRoy Eltham Posts: 3,000
    edited 2012-10-02 15:16
    First off, sorry for being a bit absent recently. It's mostly because of moving to a new state and starting a new job, but also a few trips mixed in there to consume most of the weekends recently. Things have settled down a bit now, so I am starting to do work on my side projects again, including the compiler.

    I am currently working with Jeff Martin on a validation tool for the compiler. This is an important step in getting the compiler to be used as the "official" one. Once the validation tool is up and running, we can make sure my compiler fully works properly (as much as possibly anyway) in both success and failure (properly reporting errors) cases. I have talked with Jeff about substituting my compiler in for the current one in Prop Tool, it wouldn't be that hard, so it may happen (depending on Jeff's time which is rather packed). Anyway, we want to get the validation tool working first thing so that we can use it to validate stuff when we make changes and additions to the compiler. I know it adds another delay before getting new features, but it's the safest course of action for keeping the compiler stable.

    Anyway, I was originally planning to use a pre-exisiting full preprocessor with my compiler, but there are several problems with that (primarily with Spin/PASM syntax confusing it). I've decided to just implement stuff similar to what BST does, and additionally adding #include. It's first on my list after the validation tool.

    Roy
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2012-10-02 15:54
    Roy,

    I don't know how BST does it, but it would be nice to be able to recognize a preprocessor directive (in a PRE section perhaps) that included one or more programs to be run in succession from the command line. Each program in the list would take the original source (or the output from the previous one) in STDIN, and produce the processed output on STDOUT (or errors in some standard form on STDERR). The compiler would get the output from the last program in the list.

    -Phil
  • pedwardpedward Posts: 1,642
    edited 2012-10-02 16:02
    Roy,

    I don't know how BST does it, but it would be nice to be able to recognize a preprocessor directive (in a PRE section perhaps) that included one or more programs to be run in succession from the command line. Each program in the list would take the original source (or the output from the previous one) in STDIN, and produce the processed output on STDOUT (or errors in some standard form on STDERR). The compiler would get the output from the last program in the list.

    -Phil

    And thus was born Make...
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2012-10-02 18:37
    pedward wrote:
    And thus was born Make...
    I fail to see the similarity. Make is invoked externally. What I'm proposing is invoked internally by the source code itself. Each distinct object can invoke a different preprocessor or set of preprocessors, depending upon which precompile features it implements.

    -Phil
  • David BetzDavid Betz Posts: 14,516
    edited 2012-10-02 18:39
    I fail to see the similarity. Make is invoked externally. What I'm proposing is invoked internally by the source code itself. Each distinct object can invoke a different preprocessor or set of preprocessors, depending upon which precompile features it implements.

    -Phil
    I'm not sure I understand your proposal. Could you post some sample code to show how it would look?
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2012-10-02 19:02
    David,

    Here's an example of how my debug preprocessor might be called, absent the special treatment from the Prop Tool IDE button:
    CON
    
      _clkmode      = xtal1 + pll16x
      _xinfreq      = 5_000_000
    
    PRE
    
      debug.exe
    
    PUB  start | i
    
      debug "Testing the Spin DEBUG function:\r\r"
      repeat i from -10 to 10
        debug "  The value of i is ", dec3 i, ", and i-squared is ", dec i*i, ".", #CR
    

    The PRE section lists the command-line tool(s) required to process the source code before the compiler sees it. This would very much simplify the procedure I use now, which is to force the IDE to create an archive, extract the files from the archive into a separate directory, process them, then pass the TopLevel file to Propellent. Of course, it would not address the extra control I exert over PST, but that's beyond the scope of the example you asked for.

    -Phil
  • David BetzDavid Betz Posts: 14,516
    edited 2012-10-02 19:11
    David,

    Here's an example of how my debug preprocessor might be called, absent the special treatment from the Prop Tool IDE button:
    CON
    
      _clkmode      = xtal1 + pll16x
      _xinfreq      = 5_000_000
    
    PRE
    
      debug.exe
    
    PUB  start | i
    
      debug "Testing the Spin DEBUG function:\r\r"
      repeat i from -10 to 10
        debug "  The value of i is ", dec3 i, ", and i-squared is ", dec i*i, ".", #CR
    

    The PRE section lists the command-line tool(s) required to process the source code before the compiler sees it. This would very much simplify the procedure I use now, which is to force the IDE to create an archive, extract the files from the archive into a separate directory, process them, then pass the TopLevel file to Propellent. Of course, it would not address the extra control I exert over PST, but that's beyond the scope of the example you asked for.

    -Phil
    Hi Phil,

    Thanks for the example! As you say, multiple programs could be listed to form what Unix/Linux users would call a pipeline where the output of one feeds the input of the next culminating in the Spin compiler itself.
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2012-10-02 19:15
    Yes, the pipeline would also work in DOS/Windows, so the PRE directive could easily consist of a single line. Using multiple lines, though, with each line invoked separately by the compiler, would allow the preprocessing to be short-circuited in case of an error reported in STDERR.

    Also, I assume from my example that the preprocessor path is an environment variable, separately defined from the default source file path.

    -Phil
  • ersmithersmith Posts: 6,054
    edited 2012-10-02 19:32
    David Betz wrote: »
    That would be great! Does your preprocessor support #include?

    I've attached a patch to the open source spin compiler to add the spin2cpp preprocessor. It supports #define of simple macros (no parameters), #include, #ifdef/#else/#endif etc. This isn't a complete patch, since it doesn't do the UTF-8 to PASCII translation (the preprocessor always outputs UTF-8, even from Unicode 16 input) and doesn't have command line support for defines yet. Both of those features will be straightforward to add. To use the patch, unzip the file in the root of your spin compiler source code (checked out from svn) and then apply the patch in patch.diff.
  • ersmithersmith Posts: 6,054
    edited 2012-10-02 19:36
    Here's an example of how my debug preprocessor might be called, absent the special treatment from the Prop Tool IDE button:
    PRE
    
      debug.exe
    
    PUB  start | i
    
      debug "Testing the Spin DEBUG function:\r\r"
      repeat i from -10 to 10
        debug "  The value of i is ", dec3 i, ", and i-squared is ", dec i*i, ".", #CR
    

    That looks very cool, but what happens on Mac and Linux? debug.exe is presumably a Windows tool, so in general it won't run on other platforms :-(. So the Spin source code would become host platform specific, which seems awkward.

    A more general solution might be to have a scripting language built in to the PRE section itself, so the preprocessing could be specified in a platform independent manner. But that's getting very ambitious, I think!
  • David BetzDavid Betz Posts: 14,516
    edited 2012-10-02 19:43
    ersmith wrote: »
    That looks very cool, but what happens on Mac and Linux? debug.exe is presumably a Windows tool, so in general it won't run on other platforms :-(. So the Spin source code would become host platform specific, which seems awkward.

    A more general solution might be to have a scripting language built in to the PRE section itself, so the preprocessing could be specified in a platform independent manner. But that's getting very ambitious, I think!
    Maybe you could just leave off the ".exe" part and have the implementation on each target platform supply the appropriate executable extension. This would assume that these programs are in the current path or that the Spin preprocessor knows how to find them.
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2012-10-02 19:59
    David Betz wrote:
    Maybe you could just leave off the ".exe" part and have the implementation on each target platform supply the appropriate executable extension.
    Of course. :) This also works with scripting language sources, like Perl. Unfortunately, Windows does not understand the shebang (#!) schema, so universality is still a bit compromised.

    -Phil
  • David BetzDavid Betz Posts: 14,516
    edited 2012-10-02 20:02
    Of course. :) This also works with scripting language sources, like Perl. Unfortunately, Windows does not understand the shebang (#!) schema, so universality is still a bit compromised.

    -Phil
    Sorry, it didn't occur to me that you'd want to write these filters in a scripting language.
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2012-10-02 20:26
    David Betz wrote:
    Sorry, it didn't occur to me that you'd want to write these filters in a scripting language.
    'No reason to be. For Windows, I typically distribute Perl code prepackaged as an .exe by perl2exe, which includes the Perl runtime, since most Windows installations do not include Perl. Linux and OS/X installations typically do include Perl, however, so the packaging step is unnecessary..

    -Phil
  • Heater.Heater. Posts: 21,230
    edited 2012-10-03 00:40
    Phil,

    Here is a usefull example of Phil's PRE directive.
    PRE
        rm -rf *
    PUB  start | i
        debug "Testing the Spin DEBUG function:\r\r"
        repeat i from -10 to 10
            debug "  The value of i is ", dec3 i, ", and i-squared is "
    

    You might guess I'm not taken with the idea.
    To make the thing crossplatform and safe probably means making it more like a scripting language built into the Spin Source.

    But then it starts to look like a built in make file.

    Why not use Make like normal people?
  • Roy ElthamRoy Eltham Posts: 3,000
    edited 2012-10-03 04:14
    Phil,
    I really think that is something that should be built into the IDE or whatever, and not the compiler/language.

    ersmith,
    I'll take a look at that patch this weekend (when I next work on the compiler project stuff). Seems like it should just drop in, if so then it should be available by Monday. Thanks!
Sign In or Register to comment.