Configuring sub-objects from the parent object

ersmith

A recurrent problem that comes up in Spin programming is the issue of how to configure a sub-object from the parent object. @cgracey recently added constant overrides, so we can now do things like:

obj d : "driver" | BASEPIN=16, HZ=120

to override the values of constants BASEPIN and HZ in the child object. This helps a lot, and is a very useful feature. But it doesn't quite cover all use cases. In particular, there are cases where we want to be able to change which objects the child object uses, or to glue together objects. For example, it'd be nice if my ANSI text driver could be used with different display objects (VGA, HDMI, etc.); or we may want the upper level program to tell the child object which memory driver it needs to include.

Another common use case is to want to pass a serial or display object initialized in the parent to the child objects so they can print status. The SEND and RECV keywords can help with this to a degree, but they don't cover all situations.

So this thread is to brainstorm possible ways to extend the various Spin2 compilers to allow parents more control over child objects. I'd be particularly interested in feedback from @cgracey and @macca over how their compilers work and what would be feasible for them to implement.

ersmith

Option (1): Using a preprocessor, and allowing children to #define symbols in the child

I think @macca's compiler has a preprocessor, although I don't know the extent of it. PNut doesn't yet, although openspin does and there have been suggestions to add one to PNut.

This could look in the parent something like:

OBJ d : "driver" # MEMORY="edge32"

This would have the effect of compiling the object d with a preprocessor symbol MEMORY defined to be the string "edge32", just as though

#define MEMORY "edge32"

appeared at the start of driver.spin2. In the child we'd do something like:

#ifndef MEMORY
#define MEMORY "default_mem_driver"
#endif
OBJ mem: MEMORY

which would then allow the parent object to change which object the child used for mem.

Option (2): Allowing constant strings to be used to specify object names. In the parent this would look like:

CON our_mem_driver=string("edge32")
OBJ d: "driver" | MEMORY=our_mem_driver

and in the child like:

CON MEMORY = string("default_mem_driver")

OBJ mem : MEMORY

This doesn't require a preprocessor. It does require a change to the compiler to allow string literals to be defined as constants, and for those string literals to be used in object declarations. I'm not sure how easy/hard this would be for the other compilers. I think it's quite do-able for flexspin, with a bit of tweaking.

The disadvantage of these two options is that they don't address the use case of wanting to pass a specific object (like a configured serial object) to the child, but at least they would allow a great deal more configuration of child objects.

ersmith

Option (3): Allow child objects to directly access some parts of objects specified by the parent. This is a much bigger language change, but offers more power. The syntax I'm thinking of is something along the lines of:

' parent object
OBJ
  m: "edge32"
  d: "driver" | LOOKDOWN mem=m

and in the child something like:

OBJ mem: LOOKUP "mem_interface"
...
mem.erase(block, count) ' does an indirect call on the parent's `m.erase` method.

These so-called "LOOKUP" objects would actually be method pointers which "look up" to the object in the LOOKDOWN clause of the object declaration. "mem_interface" would be a simple object with the names of just a few methods provided, like:

' mem_interface: methods to be implemented by any memory driver
' it is an error if the object specified as LOOKDOWN in the parent doesn't have these
PUB LOOKUP erase(destaddr, count)
PUB LOOKUP write(destaddr, srcaddr, count) : r
PUB LOOKUP read(destaddr, srcaddr, count) : r

At compile time the PUB LOOKUP methods in "mem_interface" would be initialized with the corresponding methods from the object provided by the parent

I'm sure the syntax could be improved, but I hope I've gotten the idea across: that we would have special kinds of objects (I've called them "lookup objects" so we don't need a new keyword, but maybe "interface objects" would be more appropriate) which would be configured in the child to call methods in an object provided by the parent.

macca

@ersmith said:
Option (1): Using a preprocessor, and allowing children to #define symbols in the child

I think @macca's compiler has a preprocessor, although I don't know the extent of it. PNut doesn't yet, although openspin does and there have been suggestions to add one to PNut.

Yes, my compiler supports preprocessor directives, some features are not implemented (keyword substitution is only partially implemented), however conditionals can use any constant definition, so something like this:

CON
    TYPE = 1

OBJ
  #if TYPE == 1
    driver : "object1"
  #elseif TYPE == 2
    driver : "object2"
  #endif

Is overridable in the parent with the standard parameters syntax:

OBJ
    child : "child_object" | TYPE=2

Option (3): Allow child objects to directly access some parts of objects specified by the parent. This is a much bigger language change, but offers more power. The syntax I'm thinking of is something along the lines of:

' parent object
OBJ
  m: "edge32"
  d: "driver" | LOOKDOWN mem=m

This may be a problem for the current spin interpreter. I'm not sure but AFAIK, objects are reordered in memory so the address offset is always positive, the proposed method requires the child object to have a reference to the parent resulting in a negative offset. Also the variable offset may be problematic for the same reason. Can the intepreter handle that ? I don't know...

Electrodude

Why don't you just add per-object vtbls, as in PNUT? Then you can have direct object pointers, and simultaneously remove the need for GC allocations for method pointers. Making an object conform to an interface would then just be a matter of ensuring certain methods are found at the correct indices in the vtbl. And then overriding child objects just becomes a matter of passing an object pointer to the child object.

ersmith

@macca said:

@ersmith said:
Option (1): Using a preprocessor, and allowing children to #define symbols in the child

I think @macca's compiler has a preprocessor, although I don't know the extent of it. PNut doesn't yet, although openspin does and there have been suggestions to add one to PNut.

Yes, my compiler supports preprocessor directives, some features are not implemented (keyword substitution is only partially implemented), however conditionals can use any constant definition, so something like this:

CON
    TYPE = 1

OBJ
  #if TYPE == 1
    driver : "object1"
  #elseif TYPE == 2
    driver : "object2"
  #endif

Ah, interesting -- your preprocessor isn't actually a preprocessor then but is integrated into the compiler itself and uses compiler CON definitions rather than #define? It's quite different from flexspin, openspin, and Catalina then, which run the preprocessor before compilation. I think your approach is more powerful, but it may be harder for the other tools to implement (a traditional preprocessor can even be a stand-alone program, which I think is how Catalina does it).

Option (3): Allow child objects to directly access some parts of objects specified by the parent. This is a much bigger language change, but offers more power. The syntax I'm thinking of is something along the lines of:

' parent object
OBJ
  m: "edge32"
  d: "driver" | LOOKDOWN mem=m

This may be a problem for the current spin interpreter. I'm not sure but AFAIK, objects are reordered in memory so the address offset is always positive, the proposed method requires the child object to have a reference to the parent resulting in a negative offset. Also the variable offset may be problematic for the same reason. Can the intepreter handle that ? I don't know...

Sorry, I didn't explain myself well. The actual "mem" object in the child (the interface object) is a stub which doesn't have any actual methods, only method pointers for the set of methods the child will need. At compile time those get filled out by the compiler based on the override parameter. At run time all the calls in the child get made through method pointers. This has the advantage that the parent object doesn't have to have any set memory layout or ordering of methods, and can have extra methods. The compiler just creates method pointers for the methods the child's interface object needs, and throws an error if for example the interface object has a method "erase" but the object provided by the parent does not. Does that make sense?

I suppose another way to do it would be to implement object pointers. At one time there was a proposal for object pointers, and I implemented something in flexspin that was supposed to correspond to that proposal, but I don't think it ever made it into PNut. But object pointers would have the disadvantage that the object the parent provides would have to have its methods laid out in exactly the same order as the interface object in the child, so we'd need some additional syntax to specify that (basically object inheritence, I think).

ersmith

@Electrodude said:
Why don't you just add per-object vtbls, as in PNUT? Then you can have direct object pointers, and simultaneously remove the need for GC allocations for method pointers.

It would mean an additional memory lookup on every function call, which is expensive, and also 4 extra bytes of memory for each instance of the object. Not a problem for Spin2 objects, probably, but C and BASIC use the same code for their structs and classes.

Making an object conform to an interface would then just be a matter of ensuring certain methods are found at the correct indices in the vtbl. And then overriding child objects just becomes a matter of passing an object pointer to the child object.

How would we ensure that the ordering of methods in the vtbl is correct? We'd have to specify somehow that the "edge32.spin2" object is going to implement the "mem_driver.spin2" interface so that all the appropriate methods get put in the beginning of the vtbl, I guess.

Electrodude

@ersmith said:

@Electrodude said:
Why don't you just add per-object vtbls, as in PNUT? Then you can have direct object pointers, and simultaneously remove the need for GC allocations for method pointers.

It would mean an additional memory lookup on every function call, which is expensive, and also 4 extra bytes of memory for each instance of the object. Not a problem for Spin2 objects, probably, but C and BASIC use the same code for their structs and classes.

Assuming you don't want fancy things like a class hierarchy that lets you override some methods and inherit others, you can cheat and only go through the vtbl for calls through object and method references and in all other cases call functions directly. You can leave methods out of the vtbl that are only ever called directly because they aren't ever called by object or method reference. You can leave the vtbl reference out of objects with empty vtbls.

Making an object conform to an interface would then just be a matter of ensuring certain methods are found at the correct indices in the vtbl. And then overriding child objects just becomes a matter of passing an object pointer to the child object.

How would we ensure that the ordering of methods in the vtbl is correct? We'd have to specify somehow that the "edge32.spin2" object is going to implement the "mem_driver.spin2" interface so that all the appropriate methods get put in the beginning of the vtbl, I guess.

Yes. Perhaps the parent class can declare the interface if the child class doesn't. Maybe I'm not familiar enough with flexspin's source to see the problem here.

macca

@ersmith said:
Ah, interesting -- your preprocessor isn't actually a preprocessor then but is integrated into the compiler itself and uses compiler CON definitions rather than #define? It's quite different from flexspin, openspin, and Catalina then, which run the preprocessor before compilation. I think your approach is more powerful, but it may be harder for the other tools to implement (a traditional preprocessor can even be a stand-alone program, which I think is how Catalina does it).

It can use both, #define and CON definitions.
Also if #define are inherited by child objects, the object parameters are not needed anymore for such cases (need to implement this...).

Sorry, I didn't explain myself well. The actual "mem" object in the child (the interface object) is a stub which doesn't have any actual methods, only method pointers for the set of methods the child will need. At compile time those get filled out by the compiler based on the override parameter. At run time all the calls in the child get made through method pointers. This has the advantage that the parent object doesn't have to have any set memory layout or ordering of methods, and can have extra methods. The compiler just creates method pointers for the methods the child's interface object needs, and throws an error if for example the interface object has a method "erase" but the object provided by the parent does not. Does that make sense?

Yes, this makes more sense, but after thinking a bit about it, I'm not sure how it would be different from the other options. I mean, the child already have an object reference (yes a stub with a bunch of functions that do nothing, but could also be an actual default implementation). The LOOKDOWN parameter replaces the default object in the child with the one specified. Seems much like the preprocessor substitution. Maybe a specific keyword would tell the compiler to not duplicate the variable space (as it happens with the normal object declaration), still requires some reordering but seems more feasible.

Maybe the object parameters could be extended to reference the OBJ declarations:

memory.spin2
OBJ
    driver : "default_driver"

parent.spin2
OBJ
    mem : "memory" | driver="edge32" ' Will replace "default_driver"

- or -

OBJ
    our : "edge32"
    mem : "memory" | driver=our ' Will reuse the object defined above, keeps VAR space

Of course the compiler will throw errors if the child object uses methods not present in the replaced or reused object.

ersmith

@macca said:

@ersmith said:
Ah, interesting -- your preprocessor isn't actually a preprocessor then but is integrated into the compiler itself and uses compiler CON definitions rather than #define? It's quite different from flexspin, openspin, and Catalina then, which run the preprocessor before compilation. I think your approach is more powerful, but it may be harder for the other tools to implement (a traditional preprocessor can even be a stand-alone program, which I think is how Catalina does it).

It can use both, #define and CON definitions.
Also if #define are inherited by child objects, the object parameters are not needed anymore for such cases (need to implement this...).

In flexspin and openspin the #defines are not inherited by children, because the preprocessor is run on a file by file basis.

Sorry, I didn't explain myself well. The actual "mem" object in the child (the interface object) is a stub which doesn't have any actual methods, only method pointers for the set of methods the child will need. At compile time those get filled out by the compiler based on the override parameter. At run time all the calls in the child get made through method pointers...

Yes, this makes more sense, but after thinking a bit about it, I'm not sure how it would be different from the other options. I mean, the child already have an object reference (yes a stub with a bunch of functions that do nothing, but could also be an actual default implementation).

The main difference is that it's by reference rather than by value. That is, if the parent does:

OBJ
  ser: "my_serial" | TXPIN = 30, RXPIN=31
  c: "child1" | LOOKDOWN cs=ser
  d: "child2" | LOOKDOWN ds=ser

then method calls to c.cs, d.ds, and ser all refer to the same object with the same configuration, whereas with something like:

CON serdriver=string("my_serial")
OBJ
  ser: serdriver | TXPIN=30, RXPIN=31
  c: "child1" | serial_name = serdriver
  d: "child2" | serial_name=serdriver

there are 3 different serial objects created (and only the top level one uses pins 30 and 31, the children will use whatever the default pins are).

It is more complicated to implement LOOKUP/LOOKDOWN objects in the compiler though. For the LOOKUP object (the one in the child) all the method declarations would have to be replaced with method pointers instead, possibly pointing to a default implementation. Then when the LOOKDOWN object is given in the parent, the method pointers would be updated so that they point to the parent object instead.

(I don't much like LOOKUP/LOOKDOWN to connect the objects, I think IMPLEMENTS/INTERFACE would be more natural, but that would require new keywords which could break existing code.)

Wuerfel_21

I vote against anything involving method pointer bloat ;3

macca

@ersmith said:
The main difference is that it's by reference rather than by value. That is, if the parent does:

OBJ
  ser: "my_serial" | TXPIN = 30, RXPIN=31
  c: "child1" | LOOKDOWN cs=ser
  d: "child2" | LOOKDOWN ds=ser

then method calls to c.cs, d.ds, and ser all refer to the same object with the same configuration, whereas with something like:

CON serdriver=string("my_serial")
OBJ
  ser: serdriver | TXPIN=30, RXPIN=31
  c: "child1" | serial_name = serdriver
  d: "child2" | serial_name=serdriver

there are 3 different serial objects created (and only the top level one uses pins 30 and 31, the children will use whatever the default pins are).

Good, that's basically what I have tought.

It is more complicated to implement LOOKUP/LOOKDOWN objects in the compiler though. For the LOOKUP object (the one in the child) all the method declarations would have to be replaced with method pointers instead, possibly pointing to a default implementation. Then when the LOOKDOWN object is given in the parent, the method pointers would be updated so that they point to the parent object instead.

Maybe I have to do an implementation test but I don't see that complications, the syntax replace the object instance in the child, before the child is compiled so it can be handled like any other object. The only critical part is the pointer to the VAR space that needs to be the same as the parent (and the reordering of the objects in memory).

ersmith

@Wuerfel_21 said:
I vote against anything involving method pointer bloat ;3

What do you propose then? If anything? Maybe the status quo is fine, but it does seem that a lot of people would like to be able to pass objects from parent to child. Are flexspin's current object pointers enough, even though you have to know the exact object type for them?

Wuerfel_21

@ersmith said:

@Wuerfel_21 said:
I vote against anything involving method pointer bloat ;3

What do you propose then? If anything? Maybe the status quo is fine, but it does seem that a lot of people would like to be able to pass objects from parent to child. Are flexspin's current object pointers enough, even though you have to know the exact object type for them?

The define-based approach would be my favorite because it can generate direct method calls. Also, any approach based on virtualized objects wouldn't allow access to the object's constants, which seems vital to me.

ersmith

Re-visiting a fairly old thread... it appears that @macca 's SpinTools makes #define global. That's not how flexspin works, but it turns out to be fairly easy to add an option to make a define apply across other files. So in flexspin you'll be able to write something like:

#define DRIVEROBJ "psram2"
#pragma exportdef DRIVEROBJ

OBJ m: "memory"

and the DRIVEROBJ macro will be visibile inside "memory.spin2", just as if you'd specified -DDRIVEROBJ="psram2" on the command line.

avsa242

Cool...this would be a lot cleaner than specifying it on the command line with the object filename in escaped quotes