New to SPIN: Need help implementing a list of Strings

Fonzi

Hello everyone,

I'm working on a project that could be very nicely implemented using a list of Strings. Essentially, I'm passing·a list of function names from one .spin file to another. The latter·.spin file will use these function names to execute the code·defined in the respective functions.

In my VAR block I have·my array declaration and initialization:

VAR
···byte funcNames[noparse][[/noparse]2]
·· funcNames[noparse][[/noparse]0] = String("start")
·· funcNames[noparse][[/noparse]1] = String("computeSomething")

Then I have a function defined to pass this list to the other .spin file:

PUB getfuncNames
·· Return funcNames

Then, the other spin file can create an object of this type and use the getfuncNames function to retrieve the list of Strings and call these respective functions.

My question is, is this the right way to do it? I have a C/C++ programming background. The way I see it, I'm creating a list of pointers (since the String("something") expression returns the address of the string), but I'm not sure if this·implementation works quite the same way in SPIN. Also, is there an easy way to concatenate two·Strings? The reason I ask is because the .spin file that retrieves the function names will need to concat the object name it has initialized with a·dot and with the actual function name.

For instance, if the latter .spin file instantiates the object as

someObject: "Object.spin"

Then it needs to become Object.funcNames[noparse][[/noparse]0] to call the appropriate function.

Any and all help will be greatly appreciated. [noparse]:)[/noparse]


···

stevenmess2004

First of all, welcome to the forums

And now for the answers that you didn't want to hear.
1. You can't instantiate VAR variables in the VAR section. I don't know why but that is the way it is. (It would probably make the compiler more complicated than they wanted to)
2. You can instantiate variables in the DAT section though, but, the DAT section gets shared by all instances of an object (kind of like static variables) while a seperate copy of the VAR variables is made for each instance of the object.
3. You can't use String function in a VAR section. You can only use it in a PUB or PRI section.
4. "Return funcNames" returns the first item in the array. There is actually no distinction between an array and a normal variable in spin. You can use "[noparse][[/noparse] x]" on any variable if you want to. To get the address use the "@" operator.
5. There is no way to use the string that is the function name to call the function.
6. Spin is compiled to byte code so passing a string and concatenating it with the object name won't do anything useful.

So that is all the bad news. If I understand what you are trying to do it is the same thing that I attempted a while ago. That is to be able to dynamically change the object or method you are calling. Unfortunately there is no easy way to do this in Spin. It is possible to do this but it takes a fair amount of mucking around with different pointers. Have a look at this thread http://forums.parallax.com/showthread.php?p=701497 which is a program I wrote to do this. It needs some improvements but I got a bit side tracked doing other things. This thread http://forums.parallax.com/showthread.php?p=718722 also talks about something similar.

Hope this helps.

Mike Green

You can also declare the strings as byte arrays and have a table of offsets into the group of byte arrays and a method that takes a string number and uses that as an index into the table of offsets, adds the start of the byte arrays and returns the address of the string.

To catenate strings, you need a destination buffer. If the strings are zero terminated you can use STRSIZE to get the size of the string and BYTEMOVE to copy the pieces to the destination. One method could be given the string number and simply catenates that string to the end of the buffer.

stevenmess2004

There is some code for dealing with strings here propeller.wikispaces.com/Strings

Fonzi

Thank you for the replies. So what I'm seeing is Strings aren't the way to do this. Let me elaborate a bit more on what I'm trying to do.

I will have a CogAllocator.spin file which will be responsible for allocating cogs to specific tasks. The way it does this is it instantiates an object that contains the functions that will be run on each cog. What I originally thought (which I'm seeing doesn't work anymore) was that I could pass a list of Strings, where each entry in the list contained the method that was to be run on each cog by that task. The CogAllocator.spin file would then extract each of these Strings and do a cognew call such as,

···· cognew(funcNames[noparse][[/noparse]1], @stack)

I figured the name of the function (as a String) would start the cog running the appropriate function (obviously this is not the case).

So my question is, when you call cognew with a function name, does the function name simply represent a pointer to the start of the executable code? If that's the case, instead of passing in a list of Strings, could I pass in a list of addresses, where each address represents the start of executable code for each function?

So in one of the auxilliary files, I could write

funcAddr[noparse][[/noparse]0] = @start················ 'where start is the name of the method
funcAddr[noparse][[/noparse]1] = @someOtherMethod
...
...
...

Any input on this design would be appreciated [noparse]:)[/noparse]

Mike Green

It doesn't really work that way. COGNEW / COGINIT is more than a simple function call. The compiler generates code to set up the initial stack frame including the parameters for the call. You'd have to analyze the generated code to see exactly how it does this in your case. I believe that the ordinal number of the method within the current object is what is actually used and the interpreter uses that to fetch the actual address from a table of pointers constructed at compile time. It's possible to locate this table at run-time and patch it. You'd have a dummy entry for use with each cog and set the dummy entry to the actual pointer you want to use. You need a cog specific dummy entry because it takes a while for the cog to actually start and you can't reuse that entry until the new cog is completely done using it.

Please look at the links stevenmess4 has provided. If you want to "mess around" with the internals of the Spin interpreter, you need to study how it works first. Otherwise, you should stick with straightforward Spin code, like a single method that uses a CASE statement to select one of a group of methods to execute in the new cog.

Post Edited (Mike Green) : 3/29/2008 7:34:30 PM GMT

Fred Hawkins

Fonzi said...

·passing·a list of function names from one .spin file to another. The latter·.spin file will use these function names to execute the code·defined in the respective functions.

Sounds like forth to me. Why not use that?

But my real question is: why is the first .spin file so much smarter than the second? Can't the second be made smart enough to obviate the need for the first?

stevenmess2004

Fonzi, have a look at DOL in the links above. I think it does exactly what you want. Also, this page shows how method calls in spin work propeller.wikispaces.com/Method+Calls.
Just quickly.

There is a table at the start of each object. Everything in it is an offset not the actual address. The things in it are
1. pointer to the next object
2. pointers to pub methods in the object
3. pointers to pri methods in the object
4. pointers to sub objects

Calls to methods in the same are made using an opcode that looks up the offset in the table. Calls to other objects do a similar thing.

This diagram is from some documentation that hippy did.

An object with sub-object -


                              .----------.
        .---------------|__________|    +0      Pointer to next object
        |            .--|__________|    +1      Pointer to first PUB method
        |        .---|--|__________|    +2      Pointer to second PUB method
        |        | .-|--|__________|    +3      Pointer to first PRI method
        |    .---|-|-|--|__________|    +4      Pointer to sub-object
        |    |   | | |  | DAT      |            Data Area
        |    |   | | |  |----------|
        |    |   | | `->| PUB a    |            First Method
        |    |   | `--->| PRI b    |            Second Method
        |    |   `----->| PUB c    |            Third method
        |    |          `----------'
        |    |
        |    |          .----------.
        `->  `-> .------|__________|    +0      Pointer to next object
                 |   .--|__________|    +1      Pointer to first method
                 |   |  |__________|
                 | .-|--|__________|    +3      Pointer to last method
                 | | |  | DAT      |            Sub-Object's Data Area
                 | | |  |----------|
                 | | `->| PUB      |
                 | |    :          :            Sub-Object's Methods
                 | `--->| PRI      |
                 |      `----------'
                 `----->

Edit: I'll attach the file since this diagram doesn't work very well.

Fonzi

Thanks Steven, I understand it a little better now after looking over the DOL and reading the other link you provided.

From what I can tell, one of the differences between what I'm trying to do and what you've implemented for the DOL is that you need to allocate space for the objects that you are loading from the SD card, whereas in my case these objects are already loaded into main RAM; am I correct? For this reason, for my implementation I wouldn't need to calculate the length of different sections (as I'm not writing into memory). Instead, would it suffice to access the correct offsets in the method pointer table of each object and use this value in the cognew() call (instead of what is usually done when one provides the name of a SPIN method)?

For instance, in my case, CogAllocator.spin is the main program. Say I want to have a cog run the first PUB method provided by the next object in the list of links (I'm going by the diagram you provided in your last post). Therefore, would the cognew call look something like this?

cognew(long[noparse][[/noparse]0]+1, @stack)

This reads the memory address at +0, which gives a pointer to the next object; 1 is then added to this address to produce the address of +1 of the next object. I'm not sure if this would be the correct syntax; please correct me if it's wrong.

I apologize for all the questions; I'm quite new to this language. An implementation in C would be so much easier, as a list of function pointers could simply be past in, so it's taking me some time to learn some of SPIN's subtleties. Thank you for all the help so far though, I greatly appreciate it [noparse]:)[/noparse].

StefanL38

Hello Fonzi,

maybe a simple or a "thumb"-question:

is 32KB of RAM not enough to create ONE BIG method with a CogAllocator-method

case Parameter 
  value1 : cognew(Method1...)
  value2 : cognew(Method2...)
  etc

value1, value2 can be constants in the CON-section to make the values explain themselves

What will be the advantage of trying to create something with a pointerlist ?

Do you REALLY need to run so many methods in its own cog ? Is it REALLY nescessary that so many methods run
completly independently from each other ?

usually you start a new cog if it is REALLY nescessary that something runs independently from other things
(like fullduplexserial=UART-functionality, or TV-Signal-creation etc.)

Is the reason speed? Do you want to do 2-8 things parallel because it takes too much time if they were done sequentially?

Notice: there were "only" 8 cogs in one chip and setting up a cog by the command cognew take some time too !
cognew copies content of HUB-RAM into the cog-RAM. A fully dynamic allocating/disallocating of cogs everytime only for some 100 microseconds
will be slower than one static case-routine branching to several sub-methods in the same cog

fulldynamic allocating might be good programming-style in C++ for PC-processors but my opinion is
it does not fit with the environment Propellerchip and SPIN

best regards

Stefan

Fonzi

Let me make the discussion more concrete with some code I have written. If any of the code I provide is wrong, please let me know.

Assume we have an auxilliary .spin file called "someTask.spin". This is the code contained in that file:

CON
·_clkmode = xtal1
·_xinfreq = 5_000_000
·numCogs = 1····· 'number of cogs this task requires to operate



VAR
· long methodOffsets[noparse][[/noparse]numCogs]·· 'the offsets of the methods to be run on each cog by this task

PUB initialize
· methodOffsets[noparse][[/noparse]0] := 1···· 'this task (for simplicity) only requires one cog running the first PUB method (called "start"). Thus, its offset is +1.

PUB start
·'do something here

PUB getNumCogs········································· 'simply return the number of cogs this task requires
· Return numCogs

PUB getBaseOffset······································ 'returns the base address of the array methodOffsets
· Return @methodOffsets

Now, assume there is a .spin file called "CogAllocator.spin". This file is responsible for instantiating an object of each task that is to be allocated, and then it attempts to allocate the required cogs for the tasks, telling the cogs to run the methods provided by the source files of each task:

CON
· _clkmode····· = xtal1 + pll16x
· _xinfreq····· = 5_000_000

VAR
· long stack[noparse][[/noparse]100]

OBJ

··dosomething: "someTask.spin"
· 'other task instantiations go here


PUB start
· allocateTask(dosomething.getNumCogs, dosomething.getBaseOffset)

PUB allocateTask(numCogs, baseOffset) | count, cogIDs
· Repeat count from 0 to numCogs - 1
··· cogIDs.byte[noparse]/noparse]count] := cognew(long[noparse][[/noparse]long[noparse][[/noparse]word[noparse][[/noparse]6 + baseOffset.long[noparse]/noparse]count, @stack[noparse][[/noparse]count*10])················· 'store allocated cog id in cogIDs array
··· if(cogIDs.byte[noparse][[/noparse]count] == -1)··············································· 'if cog allocation failed
····· count := count - 1······················································· 'revert back to last successful cog allocation (if any)
····· Repeat While count >= 0
······· cogstop(cogIDs.byte[noparse][[/noparse]count])············································ 'deallocate previously-allocated cogs (so that they may be used by another task)
······· count := count - 1·
····· Return false

· Return true·································································· 'all cogs successfully allocated
·

Let me explain my reasoning behind this expression:

cognew(long[noparse]/noparse]long[noparse][[/noparse]word[noparse][[/noparse]6 + baseOffset.long[noparse]/noparse]count, @stack[noparse][[/noparse]count*10])

- word[noparse][[/noparse]6] contains a pointer to the base of the program (this is read from the application description block, as described in the MainSpec.txt steven attached in a previous post).
- long[noparse]/noparse]word[noparse][[/noparse]6 reads the value pointed to by the base program pointer. This value is the +0 offset of the main program. Specifically, a pointer to the next object in the list.
- The final expression long[noparse]/noparse]long[noparse][[/noparse]word[noparse][[/noparse]6 + baseOffset.long[noparse]/noparse]count then dereferences this value and adds it to the appropriate offset provided by the expression baseOffset.long[noparse][[/noparse]count]
in order to access the first PUB method in someTask.spin.

I hope this added more clarity than confusion.

Fonzi

Hello Stefan,

I see what you are saying. Unfortunately, I cannot elaborate much on certain aspects of the project as I signed an NDA. However, one of the goals of my design for this application is to provide a fairly automated way of allocating particular tasks to different cogs. And yes, the application is real-time, thus it is imperative that the methods run on different cogs in parallel.

From what I understand of your design, changes in the auxilliary methods would require further change in the CogAllocator.spin file (for instance if a given task·defined in an auxilliary file suddenly requires more cogs and·its method names change). My goal is to avoid changes being made·in the CogAllocator.spin file and to localize those changes to the auxilliary files. I think this is part of my reasoning for employing·full·dynamic allocation.·

hippy

Fonzi said...
would the cognew call look something like this?

cognew(long[noparse][[/noparse]0]+1, @stack)

No. The Spin compiler is quite specific in what it will accept as the first parameter; it can only be a named method or an @ assembler code start address. From that it determines what the necessary bytecode sequence will be to start up the required method / assembler code. You'd need ...

cognew(MyMethod,@stack)

You then have two choices; either poke the bytecode which that generates so something other than MyMethod were started or overwrite the 'method call table' so the MyMethod information were that of the method you actually wanted to call. This needs to be done before invoking CogNew().

A variant on the first method would be to build up a bytecode sequence in a byte array as CogNew would create with the correct reference to what you want to run and patch the existing bytecode to then execute that bytecode held in the byte array. This is perhaps a more flexible way of handling things especially if the methods to be ultimately launched had varying numbers of parameters.

Altering the 'method call table' is the easiest option.

stevenmess2004

First of all an important point. You can't start a cog with a method from another object. The method must be in the same object that you are starting the cog from.

Your cognew won't work like that. (Well it will compile but it will be setup for using assembly and not spin.) In spin, to launch another spin cog you must use a method name in the cognew call.

Time for a big announcement. This compiles but I haven't tried it on a prop yet to see if it really works.

OBJ
  tv:"TV_Text"
  vga:"VGA_Text"

PUB start
  tv[noparse][[/noparse] 0].start(12)
  tv[noparse][[/noparse] 1].start(12)
  tv[noparse][[/noparse] 3].start(12)'very dangerous but it still compiles

This will be great for DOL...

Any way for how it will help you Fonzi,
If each object has the first few methods with exactly the same name, number of arguments and number of local variables than you should be able to use the index notation so you only need one method to start any object. One of these common methods in the objects would be a start method that attempted to start the object and return success or the cog it started in.

Sorry this is a bit short but its bedtime here so if you want to know more just ask.

Edit: well I must have been looking at this for around 3/4 of an hour cause I didn't see hippy's post

hippy

stevenmess2004 said...
tv[noparse][[/noparse] 3].start(12)'very dangerous but it still compiles

That is quite an observation, so for ...

OBJ
  tv      : "TV_Object"
  serial  : "Serial_Object"

providing say "PUB Out(n)" were the Nth method in both of those objects, tv[noparse][[/noparse] 0 ].Out("X") and tv[noparse][[/noparse] 1 ].Out("X") would programmatically route the output to TV or Serial, no hacking involved.

That really does open up some possibilities.

stevenmess2004

Yeah, but it does put a lot on the programmer to ensure that the methods are all in the same order and have the same number of arguments. Not sure about local arguments though. Are they setup by the calling method or the called method?

Actually, maybe we should move this to a new thread instead of taking Fonzi's one OT.

Fonzi

Steven,

Unfortunately assuming that all the objects have the same number of arguments and local variables isn't practical for this application, as the tasks run on the different cogs can be of varying and unpredictable complexity.

Going back to Hippy's idea of patching in the bytecode, would someone mind elaborating on how I could accomplish that? Does bytecode patching refer to overwriting the memory addresses that correspond to the appropriate method call table?

This is probably an amateur question, but can arguments be past in upon cognew calls? For instance,

cognew(MyMethod(@array), @stack)

If that is legal, would it be possible for array to contain the bytecode needed by the given function and have MyMethod be a universal and generic method that simply has the cog execute the bytecode contained in array? Is there a directive that allows a program to begin executing arbitrary code without touching the method call table (for instance, to execute the·bytecode contained in array by having the instruction pointer point to that area of memory)?

And thank you for the continued help guys. This is definitely more involved than simple function pointers that I was hoping for.·

hippy

Patching bytecodes isn't particularly easy, not least you need to determine the addresses to patch and that in itself isn't simple. You'd need a lot of understanding of bytecode and the execution behaviour of the bytecode. I haven't entirely understood the CogNew/CogInit mechanism yet.

Re : cognew(MyMethod(@array), @stack) - yes that would work.

There's no provided mechanism to execute arbitrary code, but that's easier to patch up although one still needs to determine the address to do the patching. It certainly is more complicated than you were hoping for

stevenmess2004

Fonzi, with my idea only the first x methods in the objects would have to be the same. (thinking about it I'm not sure if you need the number of local variable to be the same or not) I would make on of these methods a case statement like this

PUB methodSelector(method,var1,var2)
  case method
    con1:
      method1(var1,var2)
    con2:
      cognew(method2(var2,var1),@stack)
'.....

You could also have a method that returned the number of methods and the constants (effectively pseudo method names or pointers) for all the methods. I think that this will do what you want without mucking with the spin bytecode.

If you have a look at DOL it plays around with the object/method pointer table at runtime. However, every object that uses it must have the first method look like this

PUB methodSelector(methodName,var1,var2)

You could also patch the actual byte code but that would be an absolute pain starting with where to find what needs patching.

Of course if you want to help with making an open source compiler to do this kind of thing than you are more than welcome

The bytecode is flexible enough to do all this but its pretty difficult with the current compiler.

Fonzi

Steven,

I'll think about your idea some more tonight and early tomorrow. Isn't the cognew call a bit problematic though, considering the methods will be contained in other objects?

As far as patching the bytecode is concerned, other than patching the correct method pointer with a new one, does anything else need to be done to get a new method to execute in place of an old one? Would I have to worry about the VAR section of different objects or would that be taken care of automatically depending on what object contains the method being jumped to?

stevenmess2004

Yes, you do have to worry about the VAR sections. The pointer is two parts. One is the offset from the start of the current object to the start of the second object and the other is the offset from the start of the start of the VAR section of the current object to the start of the VAR section of the second object.

I would just put the cognew in the other objects like this

PUB methodSelector(method,var1,var2)
  case method
    con1:
      method1(var1,var2)
    con2:
      return cognew(method2(var2,var1),@stack)
'.....

This will return the cog number that started which I think is everything you need to know.

Fonzi

Steven,

Thank you for the methodSelector suggestions. It seems like the best route to go and it's far easier to implement than messing with the bytecode.

My question now concerns what you mentioned previously:

stevenmess2004 said...

OBJ
  tv:"TV_Text"
  vga:"VGA_Text"

PUB start
  tv[noparse][[/noparse] 0].start(12)
  tv[noparse][[/noparse] 1].start(12)
  tv[noparse][[/noparse] 3].start(12)'very dangerous but it still compiles

Could you explain this indexing is more detail and what it does? Does tv[noparse][[/noparse]0] use the "TV_Text" object and tv[noparse][[/noparse]1] the "VGA_Text" object (with tv[noparse][[/noparse]3] somewhat undefined, since there isn't a third object in the OBJ section)? If so, what I'm getting at is trying to use this notation to call the respective objects in the CogAllocator object without necessarily knowing the name of the objects so that it can run in an automated loop. For instance, the following is a condensed version of the CogAllocator.spin file:

OBJ
· something1: "something1.spin"
· something2: "something2.spin"

PUB start
· allocateTask(something1.getNumCogs, 0)
··allocateTask(something2.getNumCogs, 1)

PUB allocateTask(numCogs, i) | count, cogIDs
· Repeat count from 0 to numCogs - 1
··· cogIDs.byte[noparse][[/noparse]count] := something1[noparse][[/noparse]i].methodSelector(count)
··· ...
··· ...
··· ...

Would the index notation work like this here so that allocateTask is able to call each object instantiation in the OBJ section in turn? I want to avoid having to write the actual name of the object in the allocateTask method, as that would defeat the looping nature of the method and would require different code depending on the different object names.

Any input again is appreciated. I'm finally starting to understand some of this.

stevenmess2004

I haven't had a chance to test this yet but here is what I think happens.

When you do a object[noparse][[/noparse] x].method(a,b,c,....) the interpreter starts at the object object and then steps x more objects and uses that one. So in my example above tv[noparse][[/noparse] 1].start(12) would start the VGA driver. tv[noparse][[/noparse] 3] would be very bad because there are only two objects and we are trying to point to the fourth one etc.

Fonzi said...
Would the index notation work like this here so that allocateTask is able to call each object instantiation in the OBJ section in turn? I want to avoid having to write the actual name of the object in the allocateTask method, as that would defeat the looping nature of the method and would require different code depending on the different object names.

Yes, I think this is how it works and it will allow more general functions to be made. It wouldn't be a bad idea to name some constants with the numbers so that it makes the code easier to read. In my example you could use

CON
TV=0
VGA=1

I'm not sure exactly why you want to step through all the methods in an object in allocateTask but you may not be able to tell us that.

Hope this helps.

Fonzi

Thanks Steven. I'll get a chance to try this out on the chip most likely tomorrow. I'll post back on this thread to confirm whether or not that syntax seems to be working as expected.

Thanks again!

Fonzi

Hey everyone,

I just wanted to post an update to let everyone know that the object.[noparse][[/noparse]X] syntax does in fact work as Steven described. Thanks to the flexibility that this syntax allows, I was able to code what I needed as I originally set out to do. The only small issue left is that cog 0's processing power is taken up by the actual CogAllocator.spin code and cannot flexibly be devoted to fulfilling a task (for instance, a task that fails allocation because it requires only one more cog, could successfully run if it could use cog 0 to run whatever code it needs to be run). Unless I modify the bytecode or make this cog assignment in a non-automated way, I don't see another solution to it. For this application however, it's not too big of a deal.

In summary, I'd like to thank hippy and particularly Steven for their input on the forums and for teaching me a lot about SPIN. That object.[noparse][[/noparse]X] syntax was certainly a lifesaver. [noparse]:)[/noparse]

Thanks again!

stevenmess2004

Great to hear you got it working. Its good when people post a follow up post. Can you show us a picture of it working or is that not allowed by your NDA?