Shop OBEX P1 Docs P2 Docs Learn Events
Function Pointers in Spin2 - Page 4 — Parallax Forums

Function Pointers in Spin2

124»

Comments

  • David BetzDavid Betz Posts: 14,516
    edited 2013-05-11 11:03
    Heater. wrote: »
    Not much like a closure.

    I know little of lisp but in Scheme and JavaScript you can return a function from a function. Perhaps I should say return a reference to an inner function from a function.

    When do that the returned function has access to all variables that were in scope when it was created as an inner function including all the local variables that one might expect "vaporized" when the outer function returned.

    Not only that but every time you get a reference to that inner function each one has it's own set of different variables. It gets new "referencing environment" as they say.

    If that all sounds tough to follow I agree, as a programmer in languages of the style of C for ages this whole closure thing was a bit stomach churning when I met it for the first time last year. It actually makes JavaScript a more expressive language that Java or C++ both of which are hoping to intoduce closure soon I beleive.
    In general a closure is just a some code and some data packaged up together. You're right that in Lisp you tend to close over the environment that is in effect when you create the closure. This isn't really necessary though and there are other languages (can't remember which off the top of my head) where you have to specify which variables to close over. In any case, there really isn't any reason why a "smart pointer" should need more than two 32 bit fields given the right implementation underneath. Maybe part of the problem here is that Spin has no types and that third long is basically a pointer to the "class" that is needed because the pointer itself doesn't carry that information in its type.
  • Heater.Heater. Posts: 21,230
    edited 2013-05-11 11:33
    Seems the term "closure" changed in meaning a little bit over the years. It's all on wikipedia if any one wants the historic detatails.

    Looks like Python is an example of a language where you have to specify which variables to close over. It has the nonlocal keyword to do that.

    I agree. I would even naively assume a single 32 bit value is enough to identify any object in the system. Is it the overhead of such indirection that we are worried about here?

    Currently we don't have any dynamic object creation so a compiler could surely build a table whoes entries containin code and data pointers for all the object methods in the system if need be. After that our mega_pointer is just an index into that table. An operation like mptr := @method or mptr := @sio.method would get you the index into that table. I suspect having dynamic object creation would make that much arder.

    Bah, what do I know of such things, ignore me.
  • KC_RobKC_Rob Posts: 465
    edited 2013-05-11 11:38
    Heater. wrote: »
    I wonder what a language designed by public web fourum might look like....
    Quite possibly the scariest scenario of all.
  • Heater.Heater. Posts: 21,230
    edited 2013-05-11 11:47
    KC_Rob,
    Quite possibly the scariest scenario of all.

    Scares me. Perhaps we already have an example. That convoluted, contorted, pig ugly mess they call HTML5.
    CSS, HTML, JavaScript, all thrown into the pot and stirred. Not to mention the PHP or templating thrown into it at the server end to make it work. And the XML to get AJAX going and...and...and...

    Sometimes I wonder how it all hangs together at all.

    Anyway, I have confidence in Chip. He will listen to all input but then goes away and does his own thing.
  • cgraceycgracey Posts: 14,206
    edited 2013-05-11 11:49
    There are three things needed to make a fully-contextual method pointer:

    1) a pointer to the VAR area (unique to each instance of the object)
    2) a pointer to the PUB/PRI/DAT area (static for all instances of the object)
    3) an index byte to the method of interest

    The Spin compiler only sees the object that it is currently compiling. Of child objects, it only knows method names and their related indexes, along with their constants (for object#constant_name usage by the parent). So, the Spin compiler is telescopic in its approach, only concerning itself with the details of the currently-compiling object. After it's all done, there's a compressor that doesn't know anything about names of objects, but recognizes and eliminates all redundant static object instances, while patching up the pointers to the eliminated copies with the new unique addresses.

    The Spin compiler would need to get a lot more complicated to achieve total object awareness.
  • cgraceycgracey Posts: 14,206
    edited 2013-05-11 12:08
    Some of you are worried that we'll end up with something like this:

    http://www.youtube.com/watch?v=CpPuYGPcvD4
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2013-05-11 12:11
    LOL! I knew that would be the reference before I even clicked. A classic!

    -Phil
  • David BetzDavid Betz Posts: 14,516
    edited 2013-05-11 12:14
    cgracey wrote: »
    Some of you are worried that we'll end up with something like this:

    http://www.youtube.com/watch?v=CpPuYGPcvD4

    That looks pretty good although it's kind of a stripped down model. Could we add a few more features? :-)
  • KC_RobKC_Rob Posts: 465
    edited 2013-05-11 12:17
    cgracey wrote: »
    Some of you are worried that we'll end up with something like this:

    http://www.youtube.com/watch?v=CpPuYGPcvD4
    Yeah, something like that... LOL!!
  • David BetzDavid Betz Posts: 14,516
    edited 2013-05-11 12:19
    cgracey wrote: »
    There are three things needed to make a fully-contextual method pointer:

    1) a pointer to the VAR area (unique to each instance of the object)
    2) a pointer to the PUB/PRI/DAT area (static for all instances of the object)
    3) an index byte to the method of interest

    The Spin compiler only sees the object that it is currently compiling. Of child objects, it only knows method names and their related indexes, along with their constants (for object#constant_name usage by the parent). So, the Spin compiler is telescopic in its approach, only concerning itself with the details of the currently-compiling object. After it's all done, there's a compressor that doesn't know anything about names of objects, but recognizes and eliminates all redundant static object instances, while patching up the pointers to the eliminated copies with the new unique addresses.

    The Spin compiler would need to get a lot more complicated to achieve total object awareness.
    I guess the second long is what is generally called a vtable pointer in C++. You could make that the first long in every object and then you wouldn't have to carry it along with every object reference but it would increase the size of each object by four bytes.
  • cgraceycgracey Posts: 14,206
    edited 2013-05-11 12:25
    David Betz wrote: »
    I guess the second long is what is generally called a vtable pointer in C++. You could make that the first long in every object and then you wouldn't have to carry it along with every object reference but it would increase the size of each object by four bytes.

    But if you knew where to find it, you wouldn't need it, it seems.

    As Spin executes and you go deeper into objects, the new offsets get added to VBASE and PBASE at each level, so it's kind of organic.
  • David BetzDavid Betz Posts: 14,516
    edited 2013-05-11 12:32
    cgracey wrote: »
    But if you knew where to find it, you wouldn't need it, it seems.

    As Spin executes and you go deeper into objects, the new offsets get added to VBASE and PBASE at each level, so it's kind of organic.
    If you knew the type of the object at any method call site then you would know the address of its static data but with no types in Spin you kind of have to carry that along with every object reference. In C++ the vtable field is left out of objects that don't have any virtual methods because the compiler knows about all of the non-virtual methods based on the object type. This could be true of Spin as well if you'd keep track of type information even if it was just for objects and not other data types. This might involve significant restructuring of the current Spin compiler though. Are you attempting to make Spin2 out of Spin by making minimal compiler changes or are you open to creating a totally new compiler?
  • cgraceycgracey Posts: 14,206
    edited 2013-05-11 12:38
    David Betz wrote: »
    If you knew the type of the object at any method call site then you would know the address of its static data but with no types in Spin you kind of have to carry that along with every object reference. In C++ the vtable field is left out of objects that don't have any virtual methods because the compiler knows about all of the non-virtual methods based on the object type. This could be true of Spin as well if you'd keep track of type information even if it was just for objects and not other data types. This might involve significant restructuring of the current Spin compiler though. Are you attempting to make Spin2 out of Spin by making minimal compiler changes or are you open to creating a totally new compiler?

    I want to use what I've got, at first. To get ANY object-method pointer, you would have to communicate a pointer through the tree at run-time. Kind of a pain, for non-directly-related objects.
  • David BetzDavid Betz Posts: 14,516
    edited 2013-05-11 12:47
    cgracey wrote: »
    I want to use what I've got, at first. To get ANY object-method pointer, you would have to communicate a pointer through the tree at run-time. Kind of a pain, for non-directly-related objects.
    I think I must not understand how the Spin VM works. Is there a description of the VBASE and PBASE semantics somewhere and how they are used to effect method calls?
  • cgraceycgracey Posts: 14,206
    edited 2013-05-11 12:59
    David Betz wrote: »
    I think I must not understand how the Spin VM works. Is there a description of the VBASE and PBASE semantics somewhere and how they are used to effect method calls?

    No. I can't remember all the details, myself. I need to delve back into it to get an exact handle on it.

    What I know, for sure, though, is that we can implement object.method pointers. How cumbersome it would be due to current compiler limitations is unknown, but it wouldn't be that bad.
  • Cluso99Cluso99 Posts: 18,069
    edited 2013-05-11 15:35
    I have not commented much because a lot of what you are referring is out of my league. I am not a c programmer and I have no wish to be. So perhaps my input is useless, but here it is anyway.

    I consider myself an expert microcoder -microprocessor/microcontroller in assembly. I also have written commercial software in both assembler (an ICL mini) and VB3-6 (for PCs).

    Where I want access to bare metal I use assembler. Were high level code can be used, I use VB.

    On the Propeller, I choose spin for any high level because of its' simplicity. Unfortunately it does not do everything I need, so in one of my projects I use Catalina as well (mostly coded by another person). For bare metal I use pasm.

    I don't really consider spin to be easy because of the often used complex syntax (similar to c) eg >=, |= and so forth.
    I like the forced indentation - I think many dont realise the PropTool's option to display this better.
    However, a smart editor could replace the indentation with block notation ("{" and "}" for theC fans, "begin..." and "end..." for the basic fans) with a switch of a button.
    I would prefer that the syntax was a little further akin to VB (Basic) - eg case statements - did I mention I dislike C

    What I do miss and have found wanting, is to be able to call any object method from within a compiled program. What I wanted was to call a method froman object where its current heirarchy doesnt permit it because it was down another tree. eg A is the top object and has 2 sub objects B and C. While in B I cannot call a method in C unless I add object C in again as child object of B called D. I have no idea how this can be achieved.

    The other thing that I would like to do is be able to dynamically redirect input/output (as in a primitive OS akin to CPM, not *nix)

    This would permit redirection. That is the main reason others and I have advocated a mailbox type system for passing Input & Output.

    So, to me, there does not seem to be anything really missing in the object structure as a blob. But there should be an intermediate output option where the PUB objects are listed in the object code by name and reference number within the object.

    I hope this helps give a picture of what I would like to see.
  • Heater.Heater. Posts: 21,230
    edited 2013-05-11 20:47
    Cluso,

    The object pointer mechanism under discussion here is exactly trying to address your two problems.
    If I can summarize them.

    1) You want to be able to call some objects method(s) but can't because the object s out of scope.
    2) You would like to be able to dynamically swap which object is being used to perform a task.

    Serial output is the canonical example. Many times people have been asking how they can call sio.out from anywhere in their program. Say for debugging or logging etc. Being able to redirect serial output is a natural extension.

    These needs could be met if there were some kind of "mega_pointer" as it has become known in this thread. The mega_pointer would be created and set pointing to some object, or method of an object. The mega_pointer could be passed around from parent to child objects in the object tree. Child objects could use the functionality pointed at by the mega_pointer. e.g. "mptr.tx('a')" would actually call sio.tx if that is what the parent set it to.

    As a bonus by changing which object the mega_pointer points to one gets the ability to redirect I/O on the fly.

    This could all be very neat if done in a way that is a natural for for Spin. Or it could turn into a horrible C++ nigh mare of confusion.
  • stevenmess2004stevenmess2004 Posts: 1,102
    edited 2013-05-12 04:16
    Hi chip

    I haven't commented on the forums for a long time but as this is the last thing I was working on.

    I specifically wanted object pointers for a couple of reasons. One was so that I could swap bits of code to and from an sd card/EEPROM to either the stack or a heap. This would be really useful when we run out of hub room for code and also make it easy to make a mini os that doesn't need to reload everything when you want to run a new program.

    The other reason is to be able to determin the qty of objects in an array at run time. For example reading a file of contacts or something into a linked list or binary tree.

    It was relatively easy in the original spin because the vbase and pbase pointers were only 16 bits each and both would fit into a long. I would suggest it may be worth adding a double long variable type to spin to handle this. It would also be useful fou working with the clock, multiplier and divider. In DOL I was trying to get something like the following to work.
    Main object code
    OBJ
    DOL:"dol" 'this object provides a method that loads an object into the hub and returns the absolute hub address of pbase and vbase
    #output:"outputObjectType" 'the '#' is to indicate that no actual object will be loaded but a space will be reserved in the object table
    VAR
    long objPointer
    Pub start
      objPointer=DOL.loadObjectSD("serialDriver") 'loads the object into the heap and returns the absolute address of pbase and vbase
      output=objPointer 'calculates the offset address of pbase and vbase and saves it into the object table. ObjPointer isn't really necessary in this example but is useful if you want to make an array of objects.
      output.string("hello world") 'run a method in the object like you normally would
    

    Additionally, a "this" directive and allow address lookup of Objects like this would be useful
    OBJ
    output:"outputObjectType"
    Obj2:"someOtherObj"
    Pub start
      Obj2.doSomething(@output) 'passes the absolute address of pbase and vbase to doSomething so that obj2 doesn't need to know where the object is at compile time
    

    For this to work you would also need to add some basic inheritance. Because of the way the object table and variables are set out in memory you could only inherit from a single parent and there could be no prototypes. DAT sections could also be a bit problematic.

    Most of this could be done pretty easily by simple compiler changes. I did actually make most of the required changes to sphinx but I had a couple of helper functions.
  • stevenmess2004stevenmess2004 Posts: 1,102
    edited 2013-05-12 04:22
    cgracey wrote: »
    No. I can't remember all the details, myself. I need to delve back into it to get an exact handle on it.

    What I know, for sure, though, is that we can implement object.method pointers. How cumbersome it would be due to current compiler limitations is unknown, but it wouldn't be that bad.
    Thereis an object table in each object that holds an offset that is added to the current pbase and vbase. There is also a method table that is similar. Calling an object is simply takes the index of the object in the object table adds the looked up values to pvase and cbase and then similar to get to the method desired.
  • Dr_AculaDr_Acula Posts: 5,484
    edited 2013-05-12 05:04
    I ask because doing function pointing within an object is straightforward, but having function pointers work beyond the scope of the object is difficult - it requires two 32-bit pointers, exceeding a handy long.

    There must be a way around this!

    I haven't commented either up to now, but there are some things here that really resonate.

    To me, an object is group of functions. So it is like a box, and you can send data into the box, get data from the box, and ask the box to do things.

    The serial object has been mentioned. If it so happens that an object I choose to use happens to use the serial object, then the compiler ought to know about this (it might need to do a pre-compile on the code). So I could ask the compiler for all the objects already included. And if I write "sio.", as soon as the . is typed, that could bring up a drop-down menu of all the methods.

    Deep down at the core of the compiler, I'm not sure why this needs to be complicated for the user. Let's say any object happens to include the sio. object. It starts off life as a group of text files and these can be rewritten as one big text file. I did this once with a group of spin objects, and it does work except that variable names need to be unique within each object. But the compiler can handle that - it can change all variable names a,b,c to sio_a, sio_b,sio_c.

    If the compiler then rewrites to code in the same way as a typical C program, with all the functions declared first, then the actual main routine right at the end, it makes the compilation process a lot easier. Let's rename all the functions too, so that tx and rx within the sio object become functions called sio_tx and sio_rx. This can all be invisible to the user, but this intermediate file could be visible as part of the compilation process if required.

    It seems to me that this means any part of the large program can access any method in any object. Variables remain unique to the object. The object is self contained. And no more longs need to be passed than the number needed to call a local method.

    Of course, you can mess up the code if you write it badly. The sub object could call a start method, and you could forget it is doing this and call the start method again in your code with different values. But by and large it is going to simplify things and avoid duplication in the code. Avoiding duplication is a big advantage - I once had a program using Kye's SD driver which existed in a sub-object, I couldn't access it easily, and the simplest solution was to duplicate the object in the main routine. Only problem - that pretty much chewed up all the hub ram.

    There are some other cool things we could do with spin right now as well as global objects. eg loading cog code off external memory (eeprom,flash,sd card) and through a common 2k block and save 14k of hub ram. We could also add caching and run code from external ram, like the C boffins are doing. Spin programs could then be as big as you like.

    All of this can be done with code that exists now. In fact, much of it is changes to the IDE and compiler rather than to the Spin language itself. In fact, adding global access to object methods, and big spin programs could be done with no changes at all to Spin - these things are both compiler and IDE changes only.

    Keep it simple.

    sio.tx(n) would be great to be able to use anywhere in the program. And I don't think it needs some of the complex code examples that have been posted - the complexities can be hidden from the user.
  • Cluso99Cluso99 Posts: 18,069
    edited 2013-05-12 05:23
    Thanks heater.

    There was one other thing a number of us missed being able to do. It was added in bst and/or homespun and was only a compiler issue. I forget the exact details now, but it was solved with @@@.
  • Heater.Heater. Posts: 21,230
    edited 2013-05-12 07:08
    Cluso,

    Ah yes the dreaded @,@, @@@ nightmare.

    This all got so confusing I havn't managed to keep it straight in my mind ever since.

    Bottom line for me was that:

    1) @ gives you some kind of address/offset of something
    2) @@ gives you a different kind of address/offset of something
    3) @@@ gives you what you actually want. The friggin HUB address of whatever it is.

    Then there are nice features like the fact that @ gives a different result (when applied to the same thing) depending on whether it is used in Spin code or a DAT initialization.

    BST has @@@, perhaps HomeSpun as well.

    Anyway I think this is why some of us flinched on seeing @ potentially getting overloaded again with yet another meaning for object pointers.
  • Heater.Heater. Posts: 21,230
    edited 2013-05-12 07:26
    Dr A,
    And if I write "sio.", as soon as the . is typed, that could bring up a drop-down menu of all the methods.

    Don't forget that the language is not the IDE, They are two totally separate things. The language is not even the compiler, there can be many of those for the same language.

    The language is just a a set of symbols. Plus the syntactic rules that govern how you can arrange those symbols into a legal program. Plus the semantics that govern what the resulting programs will do.

    The IDE, well, that's whatever program you use to help edit files in the language and get the projects, programming and whatever else sorted out.

    That, said. It seems to be simpler to create the kind of intelli-sense you are describing for some languages than others. So a language designer might want to think about that a bit.

    The challenge for Chip is to be able to add nice features to the language, like object references, without having to totally re-engineer the compiler and especially without contorting our beautifully simple Spin language into some hideous, wart covered, multi-tentacled monster that no one wants to look at never mind use.

    See C++ for an example of how this can go horribly wrong:) I posted an example in post #86 of this thread.
  • TinkersALotTinkersALot Posts: 535
    edited 2013-05-19 16:37
    As I read through this thread, the concept that occurs to me is "namespaces".

    If an object can be declared to exist in a namespace, then could not any other object "find" objects through namespaces? Perhaps the namespace could be the entity that contains the mega-pointers and the mega pointers could be produced on the fly, as needed, by the compiler when it is doing its "linker pass" ?

    This is pretty heady stuff in this thread, but I am concerned (like others here) that "trying to do tricky things" will end up "tricking me the most" :)
  • Heater.Heater. Posts: 21,230
    edited 2013-05-19 22:05
    TinkersALot,

    I was going to say that namespaces, as in C++ and such, aren't really intended to solve this "passing objects around" problem. Namespaces are simply about trying to avoid name clashes between things in different libs or classes.

    BUT:

    You may have said something profound.

    In Spin we cannot create instances of objects dynamically.They are all statically constructed by the compiler at compile time.
    That means all the information you need to use any objects methods and vars from another object is available in your binary somewhere.
    You just have to find it and have a way to use it.

    So;

    As you say, a "namespace" could be like some kind of OBJ section that is somehow made globally available. Which contains all the info to access all objects that are in that name space.

    ....Before I think this thought any further. It occurs to me that this "name space" idea does not address the issue of being able to dynamically change the currently used objects for another, as in switching serial ports. Whilst we only have statically instantiated objects if we had object pointers we could dynamically change what those pointers point to. Not so with name spaces.
  • TinkersALotTinkersALot Posts: 535
    edited 2013-05-20 05:59
    Heater. wrote: »
    ...That means all the information you need to use any objects methods and vars from another object is available in your binary somewhere. You just have to find it and have a way to use it....As you say, a "namespace" could be like some kind of OBJ section that is somehow made globally available. Which contains all the info to access all objects that are in that name space.

    I think you caught the germ of the idea that I was trying to express. Perhaps another "overloaded phrase" would be a "registry" that exists within the compiled output. So for example in one instance I use Object-A in a program variant, and Object-A "gets registered" in the "namespace" as MyVeryCoolFoo. Next week, I need to swap out Object-A with Object-Z-from-outer-space -- and this object now becomes "registered" in the "namespace" under the same "key" (MyVeryCoolFoo).

    Another possible use of this notion could be: Both Object-A and Object-Z... are included in the compiled output. One of the "registered" as "output-option-a" and the other "registered" as "output-option-z". Then if program logic could "select a or z" for use then perhaps that is another option to consider?

    Now if consumers/users of these objects could just "look them up" as needed, then that is a bit more of the idea that I was trying to describe -- I realize it may not be what is being sought with function pointers, mega pointers, etc -- but still thought I would put this notion into the mix to see if it could help.

    cheers!
  • pedwardpedward Posts: 1,642
    edited 2013-05-24 08:46
    I was thinking about this subject again. I would support the notion of having function pointers only to local functions within the scope of the object. This would be useful for jump tables to handle vectors tied to input data. For the farther call, I still think that object references should be supported for pointer jumps; if you point to a handler in an object, the whole object context should be conveyed.
  • Dave HeinDave Hein Posts: 6,347
    edited 2013-05-24 11:16
    Back in post #4 of this thread I suggested looking at the MethodPointer object in the obex at http://obex.parallax.com/object/384 . Did anybody ever bother to look at it? What's wrong with implementing function pointers that way? Yes, the descriptor does require 2 longs, and it would be slightly larger for P2, but you would just use a pointer, and never have to worry about the descriptor space. The compiler would hide the descriptor struct from the programmer. That's the way it works in C, so I don't see why it won't work for Spin. The descriptor contains the value of VBASE, so it points to a specific instance of an object. It works for local methods as well as methods in another object.
  • TinkersALotTinkersALot Posts: 535
    edited 2013-05-24 11:30
    Dave Hein wrote: »
    Back in post #4 of this thread I suggested looking at the MethodPointer object in the obex at http://obex.parallax.com/object/384 . Did anybody ever bother to look at it? What's wrong with implementing function pointers that way? Yes, the descriptor does require 2 longs, and it would be slightly larger for P2, but you would just use a pointer, and never have to worry about the descriptor space. The compiler would hide the descriptor struct from the programmer. That's the way it works in C, so I don't see why it won't work for Spin. The descriptor contains the value of VBASE, so it points to a specific instance of an object. It works for local methods as well as methods in another object.

    I did look at it. It looked itneresting to me. I have not had any chence to test with it though yet.
Sign In or Register to comment.