Shop OBEX P1 Docs P2 Docs Learn Events
SPIN improvements - Page 8 — Parallax Forums

SPIN improvements

1456810

Comments

  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2012-10-08 20:29
    David Betz wrote:
    I'd like to make sure that I don't have to copy every object that I want to use in a project into a subdirectory of that project. I'd like to be able to find frequently used objects in a common place.
    Yes, of course. What I propose would work as follows. Suppose I have a project directory (c:\jobs\propeller\projects), my own personal library (c:\jobs\propeller\lib) and the default Propeller library (c:\program files\parallax inc\propeller tool v1.3\library). My path variable (global or IDE-based) would look like this, in the desired search order:
    c:\jobs\propeller\projects;c:\jobs\propeller\lib;c:\program files\parallax inc\propeller tool v1.3\library

    Now, in my program, I have an object spec that looks like this:
    OBJ
    
      sio: "IO::Serial::FullDuplexSerial"
      u : "umath"
    
    For the first object, the search order will be:
    c:\jobs\propeller\projects\IO\Serial\FullDuplexSerial.spin, then
    c:\jobs\propeller\lib\IO\Serial\FullDuplexSerial.spin, and finally, if it's not found in either of those two places,
    c:\program files\parallax inc\propeller tool v1.3\library\IO\Serial\FullDuplexSerial.spin

    This leads not only to hierarchical organization, but a way to override library entries that are further down the search chain.

    Moreover, I might want to test a new version of FullDuplexSerial that I'm working on, so I could specify:
    OBJ c:\jobs\propeller\testlib
    
      sio: "IO::Serial::FullDuplexSerial"
      u : "umath"
    
    This would prepend c:\jobs\propeller\testlib to the search path for this program only, in order to capture my new version of FullDuplexSerial in c:\jobs\propeller\testlib\IO\Serial\FullDuplexSerial.spin.

    Finally, using multiple OBJ sections, the path-prepend could be different for each one in a testing environment.

    Beyond the simple absolute system or IDE path list, this is all compiler stuff.

    -Phil
  • jazzedjazzed Posts: 11,803
    edited 2012-10-08 20:35
    Steve, this is not an IDE issue: it's a compiler issue, unless Roy chooses not to implement my OBJ suggestions. The /lib paths submitted to the compiler are merely installation-dependent absolute paths to be expanded upon by further relative paths in the OBJ section. It's up to the compiler to resolve the final path to each object. You're off the hook, once you submit the initial (and, for the most part, unchanging) /lib path to the compiler.

    -Phil
    It is an IDE problem unless the compiler provides me a "tree of files" and a list of files and paths with no duplicates.

    It's clearer to me every day the direction I need to go.
  • David BetzDavid Betz Posts: 14,516
    edited 2012-10-08 20:39
    Yes, of course. What I propose would work as follows. Suppose I have a project directory (c:\jobs\propeller\projects), my own personal library (c:\jobs\propeller\lib) and the default Propeller library (c:\program files\parallax inc\propeller tool v1.3\library). My path variable (global or IDE-based) would look like this, in the desired search order:
    c:\jobs\propeller\projects;c:\jobs\propeller\lib;c:\program files\parallax inc\propeller tool v1.3\library

    Now, in my program, I have an object spec that looks like this:
    OBJ
    
      sio: "IO::Serial::FullDuplexSerial"
      u : "umath"
    
    For the first object, the search order will be:
    c:\jobs\propeller\projects\IO\Serial\FullDuplexSerial.spin, then
    c:\jobs\propeller\lib\IO\Serial\FullDuplexSerial.spin, and finally, if it's not found in either of those two places,
    c:\program files\parallax inc\propeller tool v1.3\library\IO\Serial\FullDuplexSerial.spin

    This leads not only to hierarchical organization, but a way to override library entries that are further down the search chain.

    This all sounds good to me.
    Moreover, I might want to test a new version of FullDuplexSerial that I'm working on, so I could specify:
    OBJ c:\jobs\propeller\testlib
    
      sio: "IO::Serial::FullDuplexSerial"
      u : "umath"
    
    This would prepend c:\jobs\propeller\testlib to the search path for this program only, in order to capture my new version of FullDuplexSerial in c:\jobs\propeller\testlib\IO\Serial\FullDuplexSerial.spin.

    Finally, using multiple OBJ sections, the path-prepend could be different for each one in a testing environment.

    Beyond the simple absolute system or IDE path list, this is all compiler stuff.

    -Phil

    I'm not so sure I like this. Wouldn't this path be better placed in some sort of project file? I'm concerned that the path specified after OBJ is not only project-specific, it is also platform and machine specific making it hard to move this file to some other machine.
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2012-10-08 20:39
    jazzed wrote:
    It is an IDE problem unless the compiler provides me a "tree of files" and a list of files and paths with no duplicates.
    No. It's. Not. You provide the compiler with a simple absolute path list. The compiler uses that to resolve further each reference in the OBJ section, and so on down the object tree. The IDE does not have to submit each object to the compiler or even know what they are -- just the top object and a library path list. The compiler does the rest. It's that simple. You're too accustomed to make files, Grasshopper! :) Spin is much simpler.

    -Phil
  • jazzedjazzed Posts: 11,803
    edited 2012-10-08 20:42
    No. It's. Not. You provide the compiler with a simple absolute path list. The compiler uses that to resolve further each reference in the OBJ section. The IDE does not have to submit each object to the compiler -- just the top object and a library path list. The compiler does the rest. It's that simple.

    -Phil

    How do I get that stupid list of files then?
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2012-10-08 20:47
    David Betz wrote:
    Wouldn't this path be better placed in some sort of project file? I'm concerned that the path specified after OBJ is not only project-specific, it is also platform and machine specific making it hard to move this file to some other machine.
    Spin does not, and should never, require project files. As I explained in a prior post, the OBJ <path> override is definitely not a feature you'd use for files that you distribute, but only during development or in OS-specific installations that require it for some reason.

    -Phil
  • David BetzDavid Betz Posts: 14,516
    edited 2012-10-08 20:51
    Spin does not, and should never, require project files. As I explained in a prior post, the OBJ <path> override is definitely not a feature you'd use for files that you distribute, but only during development or in OS-specific installations that require it for some reason.

    -Phil
    Okay.
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2012-10-08 20:52
    jazzed wrote:
    How do I get that stupid list of files then?
    You don't. You don't need it. That onus is on the compiler. Once you've handed off the path list and the name of the top object file to the compiler, the IDE's job is done. If the compiler finds an error in one of the sub-objects, it's the compiler's job to return a full path spec to the offending file and a line, column number, and error descriptor, which you can then use to provide further service to the user.

    -Phil
  • Roy ElthamRoy Eltham Posts: 3,000
    edited 2012-10-08 20:54
    Because I might have more than one dev effort going on at the same time and don't want to keep switching back and forth.

    Steve, this is not an IDE issue: it's a compiler issue, unless Roy chooses not to implement my OBJ suggestions. The /lib paths submitted to the compiler are merely installation-dependent absolute paths to be expanded upon by further relative paths in the OBJ section. It's up to the compiler to resolve the final path to each object. You're off the hook, once you submit the initial (and, for the most part, unchanging) /lib path to the compiler.

    -Phil

    Actually Phil, the compiler itself doesn't do any file stuff at all. The "wrapper" (PropTool, or Propellent, or my port of some of Propellent) does it. What happens is the wrapper loads the top object into a buffer, then feeds that to the first pass of the compiler which fills up an array of filenames to be loaded from the OBJ section and other sections (like the FILE keyword in DAT sections). Then the wrapper loads each of those files and does first passes on them (repeating recursively). Then as the Recursion unwinds the second pass is called on each file which does the bulk of the actual compiling. This mechanism is how the PropTool can feed the compiler it's in member buffers instead of files from disk. Also, the wrapper does the bit where it takes the source Unicode 16 or ASCII (and now UTF-8) file data and converts it to PASCII for the compiler to work with.

    If you say that everything inside of my "spin.exe" is the compiler, then you are correct, but I still keep things separate in my mind, because the "compiler" part is what Chip wrote and I ported from x86. The "wrapper" part was written by Jeff (and maybe Chip helped) in delphi, and I ported portions of that in order to make spin.exe work. However, with the source code up on google, you can make a .lib file of just the compiler portion. It wouldn't take much to make a DLL of that also. However, it's not very friendly to use it directly like that. I'm hoping to find time to clean up and "library-ize" the wrapper stuff so that it's easier for people to integrate a usable Spin compiler into their projects. In PropTool, all the file handling is done by the IDE.

    It's pretty likely that I will implement something like you want for the OBJ section, because I kind of like the idea and it makes sense. However, I'm NOT doing the bit where you put a path (e.g. c:\blah\blah\) on the OBJ line or anywhere in the Spin source. That is just not going to happen sorry. Way too many issues with making that work cross platform, plus it's gross.
    However, if Jeff & Chip don't like it then they could veto me.
  • jazzedjazzed Posts: 11,803
    edited 2012-10-08 20:59
    You don't. You don't need it. ....
    The attached list of files or something like it. Then there is the list for the .zip feature.
    370 x 357 - 32K
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2012-10-08 21:04
    Roy Eltham wrote:
    It's pretty likely that I will implement something like you want for the OBJ section, because I kind of like the idea and it makes sense.
    Thanks, Roy, I appreciate it, as I'm sure many other users will, too.
    However, I'm NOT doing the bit where you put a path (e.g. c:\blah\blah\) on the OBJ line or anywhere in the Spin source. That is just not going to happen sorry. Way too many issues with making that work cross platform, plus it's gross.
    That's okay. I can certainly understand your reservations about it, and I agree it's gross. It's gross in Perl, too, but I've had to resort to it in cases where my web host didn't want to install the modules I needed in the global library.

    -Phil
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2012-10-08 21:10
    jazzed wrote:
    The attached list of files or something like it. Then there is the list for the .zip feature.
    You don't need that for compilation. But, yes, creating an archive is a different matter. Roy, can you help out here? In the Propeller Tool, archives have to be compiled before they're created to make sure there are no errors. Is there any way that your compiler could be "encouraged" to do the archiving so Steve doesn't have to mess with the object list? Obviously, the archive would have to include relative directory info now, too, but that's already a zip option.

    Thanks,
    -Phil
  • Roy ElthamRoy Eltham Posts: 3,000
    edited 2012-10-08 21:15
    jazzed,
    You were supposed to add an issue up on google code requesting the feature to output the list of files. I need to know more exactly what you want/need, and I should be able to provide it.
  • jazzedjazzed Posts: 11,803
    edited 2012-10-08 21:22
    Roy, if you can finish the pre-processor stuff, that would be much more useful.
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2012-10-08 21:25
    Roy,

    There's actually another bit of grossness that's already part of Spin: the file directive in the DAT section. Is there any way to make that path-relative? Or are we stuck with backwards-compatibility issues there?

    -Phil
  • David BetzDavid Betz Posts: 14,516
    edited 2012-10-08 21:45
    Roy Eltham wrote: »
    It's pretty likely that I will implement something like you want for the OBJ section, because I kind of like the idea and it makes sense. However, I'm NOT doing the bit where you put a path (e.g. c:\blah\blah\) on the OBJ line or anywhere in the Spin source. That is just not going to happen sorry. Way too many issues with making that work cross platform, plus it's gross.
    However, if Jeff & Chip don't like it then they could veto me.
    Hi Roy,

    Can you tell us what other chance you have planned? Any chance you'll add a way to create references to objects that can be passed as parameters?
  • Roy ElthamRoy Eltham Posts: 3,000
    edited 2012-10-08 22:04
    jazzed wrote: »
    Roy, if you can finish the pre-processor stuff, that would be much more useful.
    That's going in tonight. Just doing testing/debugging now.

    Still, go add the issue on the google code site for the file list thing. It seems like it would be easy to add.
  • Heater.Heater. Posts: 21,230
    edited 2012-10-08 22:25
    If I download an object from somewhwere that uses this object path scheme and contains a sub object like so:
    OBJ
        sco: "Bla::Bla::Bla::SomeCleverObject"
    

    I am I now going to be forced to create the Bla/Bla/Bla directory tree somewhere just to put SomeCleverObject in?
    Or edit the "Bla::Bla::Bla" out of the original source.

    This would not make me happy.
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2012-10-08 23:13
    Heater,

    You'll get used to it. Typically, such a directory structure is reserved for library usage, where the subdirectories form an agreed-upon hierarchy that everyone adheres to. The Perl CPAN library is a perfect example of this, and installing CPAN modules automatically creates the necessary subdirectories as needed. This is a system which has worked well for decades, as attested to by the success of CPAN. Once the OBEX reaches a critical mass, such a library structure becomes a necessity, lest it crumble under its own monolithic bulk. Taking care of this issue now will be a lot less painful than would forcing the transition when it becomes inevitable.

    But, better still, you can organize your own private library along hierarchical lines, rather than having everything in one huge directory.

    Here's another scenario. You download an archive from the forum and unpack it into a subdirectory of your own project directory. The zip file includes relative directory info, put there by the compiler/IDE, and these directories are created during the unzip process automatically. That's already a feature of zip.

    -Phil
  • Heater.Heater. Posts: 21,230
    edited 2012-10-08 23:29
    Phil,
    Typically, such a directory structure is reserved for library usage, where the subdirectories form an agreed-upon hierarchy that everyone adheres to.

    So, you are saying that the "bla::bla::bla" Is really only for shall we call them "system" objects or otherwise common "standard" objects.

    When developing a project I may have a bunch of objects I have created within it. I need those to be found in the same directory as the main module, most of the time, or perhaps a sub directory of my project. In that case I don't uses "bla::bla::..." just the object names and the projects directory is always first in the search path, right?
  • SapiehaSapieha Posts: 2,964
    edited 2012-10-09 02:33
    Hi jazzed.

    Sorry I'm late to this party.
    My health was not good last weeks.

    But to answer Yours question.

    I have 5 different systems that RUN same MAIN code ---- BUT only difference are in LIBRARY's used to compile that main programs.

    jazzed wrote: »
    I would like to know how many people have had that problem.
    Ok, now we can have a discussion (I was beginning to think this was the room for abuse).

    So if this is a valid concern (i'm not convinced it is), it impacts what Parallax and I have already considered for the new version of SimpleIDE. My concern is to have fewest possible modes for SimpleIDE for simplicity.

    Why do you need more libraries in the first place?

    The current idea is to have 2 or more library paths + the local propeller program path files.
    The main propeller library files distributed by Parallax can be over-ridden by user library path files.
    All library files can be over-ridden by what is in the local propeller program path files

    Why is it crazy to set one or more library path or paths in the IDE for a program to be compiled in a given propeller development effort?

    I have 2 library paths and one regular path that happens to have the main project code. Why would I ever need more than that?

    At this point the feature is not implemented, so it's not as important now, but if someone decides it's important, and actually implements it, I need to plan for it and so will Parallax.

    Please answer all this and help me reach some reasonable conclusions about SimpleIDE requirements. I have that other list and a few other suggestions. Dealing with this issue is not on the original spinnyness thread list.
  • eldonb46eldonb46 Posts: 70
    edited 2012-10-09 02:37
    OK, while "mom and dad" are still fighting about "how" to Improve Spin, I'll suggest another improvement that I would like to see. But, because I do not know the "magic" available to provide my request, I'll state just it, and let others provide the details.

    I would like to see a New Counter and Control Methods similar to "cnt" that count seconds instead of system clock ticks, similar to Epach as used in a UNIX system.

    Yes, I know it can be done by dedicating a 20MIP COG or using a RTC chip, but that seem like a little overkill.

    Why not provide a simple and easy to access "sec" counter?

    For a lot of short timing requirements, the current system "cnt" counter is fine, but because it rollover at about 52 seconds its usefulness is diminished. Any Prop users that has dealt with Wall Clock Time requirement knows what I mean, and the current use of "cnt" mystifies the Noobe (as I was, not too long ago).

    Thanks for the consideration.

    Eldon - ( now back to the informative fighting :-)

    -
  • Roy ElthamRoy Eltham Posts: 3,000
    edited 2012-10-09 03:55
    I've posted an update of the compiler on google code. There are some issues to resolve, but most of you will not run into them, so it should work as is for you. I've messaged ersmith about it, and we'll get stuff fixed shortly.
    It's disabled by default, enable it with the -p command line option. There is also a -D option to allow you to define things on the command line. (you can only do -D SYMBOL, not anything fancier like -D SYMBOL=X or whatever).
  • David BetzDavid Betz Posts: 14,516
    edited 2012-10-09 04:33
    Roy Eltham wrote: »
    I've posted an update of the compiler on google code. There are some issues to resolve, but most of you will not run into them, so it should work as is for you. I've messaged ersmith about it, and we'll get stuff fixed shortly.
    It's disabled by default, enable it with the -p command line option. There is also a -D option to allow you to define things on the command line. (you can only do -D SYMBOL, not anything fancier like -D SYMBOL=X or whatever).
    Great! Thanks Roy and Eric for adding the preprocessor.

    I have been using Mercurial for a while so I may have forgotten how to use SVN but I did a 'svn update' and then 'make clean; make' and got the following errors. This is under Mac OS X. Did I need to do something other than just 'svn update'?
    g++ -o spin -DGCC -Wall -g SpinSource/spin.cpp SpinSource/flexbuf.c SpinSource/preprocess.c PropellerCompiler/libspin.a
    SpinSource/spin.cpp: In function &#8216;void UnicodeToPASCII(char*, int, char*)&#8217;:
    SpinSource/spin.cpp:765: warning: comparison is always false due to limited range of data type
    SpinSource/spin.cpp:765: warning: comparison is always false due to limited range of data type
    SpinSource/spin.cpp:765: warning: comparison is always false due to limited range of data type
    SpinSource/preprocess.c: In function &#8216;void handle_define(preprocess*, ParseState*, int)&#8217;:
    SpinSource/preprocess.c:648: error: &#8216;_strdup&#8217; was not declared in this scope
    SpinSource/preprocess.c: In function &#8216;void handle_include(preprocess*, ParseState*)&#8217;:
    SpinSource/preprocess.c:674: error: &#8216;_strdup&#8217; was not declared in this scope
    
  • jazzedjazzed Posts: 11,803
    edited 2012-10-09 07:29
    Sapieha wrote: »
    I have 5 different systems that RUN same MAIN code ---- BUT only difference are in LIBRARY's used to compile that main programs.

    How do you deal with it today? How can it be made better?
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2012-10-09 08:25
    heater wrote:
    So, you are saying that the "bla::bla::bla" Is really only for shall we call them "system" objects or otherwise common "standard" objects.
    For programs that are meant for distribution, largely yes.
    When developing a project I may have a bunch of objects I have created within it. I need those to be found in the same directory as the main module, most of the time, or perhaps a sub directory of my project. In that case I don't uses "bla::bla::..." just the object names and the projects directory is always first in the search path, right?
    That's an interesting question. Typically, if your projects directory is, say /propeller/projects and you have objects in /propeller/projects/objects, you could specify:
    OBJ
    
      ob : "objects::myobject"
    

    Or, if the objects subdirectory contained your entire personal library, you could include it in the /lib path instead and leave off the objects::.

    In my case, if I'm developing code for the OBEX, I might create a directory hierarchy for the new object in my own library that mirrors what it would be in the default, or OBEX, library. In that case, I would write:
    OBJ
    
      cam : "sensors::vision::PropCAM"
    

    Once accepted in the OBEX, I could simply move the new object over to my own OBEX library without having to change the OBJ reference in my demo code and other top level programs.

    Now, when archives are created, what happens is this: source files are gathered from their disparate path locations (system library, projects directory, personal library) into one base directory, with subdirectories structured as given by the various OBJ sections. Then the sources are zipped, preserving that subdirectory structure. When unzipped into any directory that structrue is restored, and the Top Object can be run without change, and without having to redistribute the included objects to their various libraries. Alternatively, for OBEX objects, the zip can just be expanded in the OBEX library, and things will end up where they need to be, with subdriectories created on the spot as necessary.

    Looking further down the road, it would not be unreasonable to have install scripts for various OSes that simply access the OBEX, download the selected object, and expand it in the user's OBEX library: one-step install.

    -Phil
  • SapiehaSapieha Posts: 2,964
    edited 2012-10-09 08:28
    Hi jazzed.

    Today I need 5 different Directory's that hold both its own MAIN and Library files to be compiled.

    My thinking was -- IF it is possible that I can have one directive in SPIN that give preferred Library-DIR.

    But it is not any thing I have looked on -- That need thinking
    jazzed wrote: »
    How do you deal with it today? How can it be made better?


    EDIT:

    Maybe have same system as VHDL.

    2 directives.
    Library: xxxxxxxxxxxxxxxxx
    USE: xxxxxxxxxxxxxxxx

    That system maybe can be used even for more possibilitys!
  • Roy ElthamRoy Eltham Posts: 3,000
    edited 2012-10-09 10:15
    David Betz wrote: »
    Great! Thanks Roy and Eric for adding the preprocessor.

    I have been using Mercurial for a while so I may have forgotten how to use SVN but I did a 'svn update' and then 'make clean; make' and got the following errors. This is under Mac OS X. Did I need to do something other than just 'svn update'?
    g++ -o spin -DGCC -Wall -g SpinSource/spin.cpp SpinSource/flexbuf.c SpinSource/preprocess.c PropellerCompiler/libspin.a
    SpinSource/spin.cpp: In function ‘void UnicodeToPASCII(char*, int, char*)’:
    SpinSource/spin.cpp:765: warning: comparison is always false due to limited range of data type
    SpinSource/spin.cpp:765: warning: comparison is always false due to limited range of data type
    SpinSource/spin.cpp:765: warning: comparison is always false due to limited range of data type
    SpinSource/preprocess.c: In function ‘void handle_define(preprocess*, ParseState*, int)’:
    SpinSource/preprocess.c:648: error: ‘_strdup’ was not declared in this scope
    SpinSource/preprocess.c: In function ‘void handle_include(preprocess*, ParseState*)’:
    SpinSource/preprocess.c:674: error: ‘_strdup’ was not declared in this scope
    

    That's just me not realizing I need to do something different for the Mac. Eric's code used strdup() and MSVC complained about it, so I switched it to _strdup() which compiled fine for MSVC and for GCC in MinGW. I guess I need to do some sort of #ifdef in there to pick between _strdup and strdup. I'll try to get to that tonight. in the short term, you could try locally modding preprocess.c to change _strdup() to strdup() and see if that works on your Mac.
  • David BetzDavid Betz Posts: 14,516
    edited 2012-10-09 10:21
    Roy Eltham wrote: »
    That's just me not realizing I need to do something different for the Mac. Eric's code used strdup() and MSVC complained about it, so I switched it to _strdup() which compiled fine for MSVC and for GCC in MinGW. I guess I need to do some sort of #ifdef in there to pick between _strdup and strdup. I'll try to get to that tonight. in the short term, you could try locally modding preprocess.c to change _strdup() to strdup() and see if that works on your Mac.
    Okay, for now I just added the following line to the top of preprocess.c:
    #define _strdup strdup
    
    Now it compiles but I still get those comparison warnings. Should I worry about those?
  • Roy ElthamRoy Eltham Posts: 3,000
    edited 2012-10-09 10:57
    David,
    Sadly, that code isn't going to work right. I've just put up new code that should fix everything for you, do another svn update please. :)

    Make sure you get r41 (r40 compiles but had a bug).
Sign In or Register to comment.