Formatted output using callbacks (indirect procedure calls).

Phil Pilgrim (PhiPi)

A few days ago I posted a thread relating to callbacks, along with some code purporting to implement them. Well, that code was far from complete. There is more to setting up an indirect procedure call than I had imagined. What I left out completely was making sure that the callback routine's VARs were pointed to correctly. This, it turns out, is tricky when the callback is created in one object and called from another. But I think those details have been ironed out now.

To verify the new code, I converted my format object (a printf-like formatter) to use indirect calls to the character output routine. That way, it can be used with tv_text, tv_wtext, FullDuplexSerial, Simple_Serial, or any character-based output object that has a public routine accepting a single parameter to output one character. Because the calls to these objects are indirect, format doesn't have to declare them or even know they exist.

In order to keep calls to the format methods simple, I've incorporated a use method that remembers which callback object to call. That way, the callback object doesn't have to be included in each call to a format method. But there's a downside to this: it means that all calls to format methods have to be made from the object that called use. If this turns out to be an onerous limitation, I may have to change it.

Attached is a demo program that includes the format, tv_wtext, and callback objects. To run it, you will need a TV monitor connected to your Propeller. Again, this code is highly experimental. It contains routines that modify Spin object code on the fly! So use it at your own risk!

Thanks much to Chip, who provided some necessary clues about object layout in memory!

-Phil

TyreBiter

Phil:

That is impressive... and you saved me a few month of pocking around!!!! Thanks

David.

Phil Pilgrim (PhiPi)

One caveat about the callback object: Because (due to the self-modifying Spin code) it's no-longer reentrant, it's not multi-cog safe either. To use callback in multiple cogs, make copies of it, assign unique names to the files containing the copies, and use a different filename in the OBJ section of each cog that uses it. That way, a separate copy of the object will be compiled into each cog's object code, and non-reentrancy will no longer be an issue.

-Phil

cgracey

The compiler will see that even though these files are named differently, they are identical. Then, it will get rid of the redundancy to make your code smaller. It does this at the binary level, so there's no way to trick it -·unless, you make each object contain different data.

Phil Pilgrim (PhiPi) said...
...and use a different filename in the OBJ section of each cog that uses it. That way, a separate copy of the object will be compiled into each cog's object code, and non-reentrancy will no longer be an issue.

-Phil

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔


Chip Gracey
Parallax, Inc.

Phil Pilgrim (PhiPi)

Chip,

Holy smokes! You've thought of everything! Okay, well, I guess including a one-long DAT section with different data for each file would do it then. Right?

-Phil

cgracey

That would do it.

Phil Pilgrim (PhiPi) said...
Chip,

Holy smokes! You've thought of everything! Okay, well, I guess including a one-long DAT section with different data for each file would do it then. Right?

-Phil

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔


Chip Gracey
Parallax, Inc.

TyreBiter

Phil/ Chip

The comment about the long is a cryptic. Could you be more specific

Thanks

David

Phil Pilgrim (PhiPi)

David,

What Chip was saying is that the code generated by the copies has to be different in order to be loaded as separate entities. One way to do this without changing the actual Spin code is to add a DAT section after the Spin code containing a single long, like this:

[b]DAT[/b]
                
CopyNo         [b]long[/b]      $0000_0001

In each copy, the value after the long would be different. The DAT section doesn't do anything. It just guarantees that each copy will be unique and loaded separately.

-Phil

TyreBiter

phil,

after being away from the propeller for months, I am back and trying to expand on your demo by adding some additional
functions.

I adding the following code to callback and callbackdummy. these two objects compile, but when compiling and running the format demo without actualy calling the added routines, The image seems to crash even if the added routines are not called....
well mabe crash is a bit much actualy the programs down loads fine and then goes into la la land when it is executed by the
propeller. I am wondering what the line

byte[noparse][[/noparse]@@$xxx] := self & $3f

is about and what the value of @@$xxx should be for successively added functions to callback dummy.
I am hoping to expand your example to seven functions total.

perhaps you could suggest some reading material and/or example code fragments.

Regards...

David




********************************

PUB _4(a,b,c,d)

********************************

PUB do4(self, a, b, c, d)

''Call a callback routine indicated by the callback object, self. The routine called should take
''four arguments.

if self
word[noparse][[/noparse]@@$18] := -(self >> 17 & $7ffc)
word[noparse][[/noparse]@@$1a] := (self >> 4 & $7ffc) - @var_base
byte[noparse][[/noparse]@@$135] := self & $3f
return dummy._4(a, b, c, d)

********************************

Phil Pilgrim (PhiPi)

David,

The line byte[noparse][[/noparse]@@xxx] will be different for each routine and refers to an address within the routine itself. But it's worse than that. By adding another routine, the addresses of all the other ones shift due to an additional table entry at the top of the code. So basically, you've got to analyze a dump of the compiled code and recompute all the addresses. That's the reason your program crashes, even though you don't call the added code.

What the byte[noparse][[/noparse]@@xxx] does is replace the index of the dummy routine in the call with the index of the callback routine in the calling program. Assuming _1 through _4 are in order in dummy.spin, you will be looking for these patterns:

····01 33 32
····02 33 32
····03 33 32
····04 33 32

It's the dump address of the first byte in each, minus $10, that needs to be inserted for xxx. But be sure to check a new dump of the corrected code. Simply changing these numbers could also cause things to shift! Also, if I recall correctly (it's been awhile!), you will need to bump the $18 and $1a each by 4 for each routine that you add.

(Once Chip adds array constructors to Spin, this will be a whole lot easier, since all callback routines could then take a single argument: an array with a variable number of elements.)

-Phil

TyreBiter

Phil,

Thanks for the rapid reply and the info....

Has there been any serious thaught given to putting togather an "internals" document on the prop/ spin???
I for one would be willing to part with a reasonable amount for cash for it...

by the by how goes the camera? ( sorry off topic )

David

Phil Pilgrim (PhiPi)

David,

An accurate internals document would have to come from Parallax. Otherwise, there'd just be too much guesswork and the likelihood of wrong information. (In developing the callback program, I was burned a couple times by false assumptions.)

As to the PropCAM, I've ordered 300 lenses and holders, which should be shipping from China today in fact!

-Phil

Paul Baker

An internals document is not likely and would be very far from now if it ever came into existance. There are much more important things to release (datasheet being the highest). Also there is the expectation of tech support for anything we release and such a document would potentially create a tech-support nightmare. This is the reason there is no official documentation for the PBASIC bytecodes either.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Paul Baker
Propeller Applications Engineer

Parallax, Inc.

Phil Pilgrim (PhiPi)

Hey Paul,

I certainly didn't mean to imply that Parallax should create such a document — only that they'd be the only possible source for a reliable one. I expect a lot of this stuff might even change over time. So an "official" internals document could tie Parallax's hands needlessly, if one were published.

Somebody out there, other than Parallax, will likely attempt such a document. (It won't be me.) But if they do, it's contents would have to be regarded with a large grain of salt.

Nontheless, by studying the internals on one's own, it's possible to glean a lot of insight and possibly create some useful tools. I don't think anyone would discourage that!

-Phil

Paul Baker

Phil, I understood, I was just trying to dash any hopes that such a thing was in the works. We might be able to provide minimal feedback on something·someone produced, time permitting of course.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Paul Baker
Propeller Applications Engineer

Parallax, Inc.

Post Edited (Paul Baker (Parallax)) : 11/7/2006 1:33:04 AM GMT

Cliff L. Biffle

I've started reverse engineering the SPIN bytecodes and runtime model, but I don't expect to put a whole lot of work into it. (I've figured out enough to get machine code up and running, and that's about enough for me.)

My info's on my web page; I'd be happy to Creative Commons license it if others want to expand on it.

TyreBiter

Thanks Phil, That worked like a charm...

Would you enlighten me about the two word address values and where in the hex dump to find them?

word[noparse][[/noparse]@@$18] := -(self >> 17 & $7ffc)
word[noparse][[/noparse]@@$1a] := (self >> 4 & $7ffc) - @var_base


Waiting with baited breath for the propCam

And Thanks again

David

Zack Bloom

I'm pretty confused as to the usage of callbacks.· I need to output a string combining constant strings and values via serial or create a string to be outputed to a seperate serial function.· Could anyone provide an example of this object using the FullDuplexSerial,·another serial method or outputing strings for use in FullDuplexSerial functions?

I tried the:

tx(63)
tx(val)

method, but the delay was too much for what I was controlling.· It needs to occur in one call to the serial object.

Post Edited (Zack Bloom) : 1/10/2007 3:18:32 AM GMT

Paul Baker

I dont think you need to use callbask in order to accomplish what you are trying, here is an extended serial object Martin released a frew days ago: http://forums.parallax.com/showthread.php?p=624624

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Paul Baker
Propeller Applications Engineer

Parallax, Inc.

deSilva

@PhiPi: you have "advertised" these routines in your most recent posting which has cought my interest. I think I broadly understand what you are doing......

As there is considerable overhead when using your style of "call back", I thought it might be considerably easier to just have a static dispatcher as:

obj
  extern : "???"
con
r1,r2,r3,r4

pub callback1(r, p1)
case r
  r1: extern.callback1(p1)
  r2: callback2(p1)
pub callback2(r,p1,p2)
case r
  r3: callback3(p1,p2)
  r4: callback4(p1,p2)

This looks very dowdy and it certainly is....

My main problem however is - and this is where I need help - that there seems to be no way to access A SPECIFIC instantiation of - in this case - EXTERN. In fact this code will even create a new one..

Of course I can double your trick, substituting the "base" in the appropriate internal location, which however would need an initialization call from that instantiation, which will leave me with the awkwardnesses of both worlds....