Almost time for another update. I've got sphinx modified so that it now spits out the index of an object in the object table [noparse]:)[/noparse]. Now just to test my other modification to sphinx... Should be new code posted this weekend.
Some, but nothing ready to release yet. I made one (to get the index in the object table) of the mods to sphinx and then couldn't figure out how to make the second (remove objects that are meant to be loaded dynamically).
I've started on another new version that cuts back on some of it's functionality to try and make it easier to use and take up less space.
I've also run into a problem of sorts, the heap object I'm using is byte aligned rather than long which is needed.
I also need to start using the latest version of fsrw.
And the biggest problem was that I had an sd card failure and haven't got around to fixing it yet...
I'm going to be away from my computer next week but after that I'll put some time into it, especially if someone here sends me a friendly reminder in my pms...
In fact, I'm looking at the code right now [noparse]:)[/noparse]
Dave Hein said...
Steven, do you have a description of how you modify the offsets? I'm having trouble understanding what your code is doing.
That's probably not surprising, sometimes I have problems with it
Okay, first reading the header, this should be simple and the code can be improved now that fsrw has a seek command
'read ProgramStart, VARStart and StackStart from header
'subtracting these will give us program length and var length
'set objectSize - from header
'read all the data in the header and use whatever we need
[b]repeat[/b] 8
sd.pgetc
objectLength:=sd.pgetc '8 'byte of varBASE
objectLength.[b]byte[/b][noparse][[/noparse]­1]:=sd.pgetc '9 'byte of varBase
objectLength:=objectLength-$10
varSize:=sd.pgetc 'A 'byte of stack start
varSize.[b]byte[/b][noparse][[/noparse]­1]:=sd.pgetc 'B 'byte of stack start
varSize:=VARSize-$10-objectLength-8 '-$8 fudge factor to be figured out
[b]repeat[/b] 4 'get to end of header
sd.pgetc
This gives us two important pieces of information, the object length (DAT+PUB+PRI+Object/method table) and the size of the VAR area. A little math is needed to figure it all out and I wouldn't have a clue why there is an extra -8 in the varSize calculation but it needed it to work.
Next is allocating memory and putting the addresses together in a useful format. I stripped it of a lot of extra stuff to get only the essentials.
'allocate space for methods
objectStartBase:=mem.allocate(objectLength+3) 'need the extra 3 bytes so we can make sure that the object is long aligned
'load methods
sd.pread(objectStartBase,objectLength)
'allocate space for VAR
objectVarBase:=mem.allocate(varSize) 'need the extra 3 to ensure vars are long aligned
'zero the var area
[b]bytefill[/b](objectVarBase,0,varSize)
objectAddr.[b]word[/b][noparse][[/noparse]­0]:=objectStartBase
objectAddr.[b]word[/b][noparse][[/noparse]­1]:=objectVarBase
[b]return[/b] objectAddr
Hopefully all easy to understand. Note that objectAddr is the "absolute" address of the object and not an offset.
The last important bit is what to do with objectAddr and that is in objsTemp
The arguments to this are the objectAddr we got previously from DOL and the position. position is the position of the address in the object table. To get that you start at 0 and add 1 for every PUB, PRI and Object (including each instance of an array object) to the object you want to change. This method when called will fix up the offsets so that then you can call the object with a normal call. Note that I had to split the calculation into two parts so that overflows during the calculation didn't become a problem (maybe the problem you were having with C-DOS?).
Now, there are still a couple of problems.
1. How do we get the position? I sorted that with a modification to sphinx. Pretty simple but I have to find what I did because I think I lost the modified code on a bad sd card [noparse]:([/noparse]
To use the mod you just did this
CON
objectIndex=@object
worked nicely though only in a con block. Was an easy mod.
2. We still need a space in the object table. I started modifying sphinx so that you could use a notation like
object:@"object"
which would compile everything as normal and then just not include object in the final linking process. There was a free byte I used in the sop file to mark the unused objects but I couldn't figure out an easy way to not include the file.
Hopefully that helps. If anythings not clear than ask away.
Thanks for the explanation.· It's much clearer now.· I am still a bit confused by the VAR section.· I see that you allocate memory for it, and set the VAR offset in the table to point to it.· I thought the VAR entry in the table was an offset from the previous object's VAR space.· It appears that your using it as an absolute address.· I'll need to study that some more to understand it.
In this bit of code the second line fixes up the offset to the object and the third line fixes up the offset to the var section. "this" is a variable that was figured out earlier that conatains the absolute addresses of the object and var sections.
Here is a new version of DOL. This one will copy create a new copy of the VAR section of an object in the hub allowing you to add and remove objects in the hub. Should be good for things like dynamic linked lists or binary trees.
The demo is pretty basic and shows creating five new objects and assigning a different VAR variable to each, than calling each and seeing all the separate values.
There is code in DOL for loading objects from an SD card but it hasn't been tested yet.
Thanks for the link to this thread from the cogject thread. This is a fascinating piece of work. I'm coming up to speed with how this works - in post #1 is the source code and in that is a mouse driver code and I can see there is a 'methodrunner' which replaces calls to each method.
The cogjects that are now working are keyboard, serial and the two cog VGA driver.
TV, Mouse and other VGA drivers are next on the list. The underlying SD card driver is Kye's code which happens to fill a cog, so this hub space can be recycled for other cogjects. 8k of ram already saved.
Combine that with the code here and it ought to be possible to move both cog code and the supporting spin code in and out of memory, either from an SD card, or from external memory. That means one can have arbitrarily large programs on a standard propeller.
How many DOL objects have been created so far, and how 'hard' are they to create?
Sorry, should have said to look at the version in the last post. There are some significant differences in how things work that should simplify a lot of things.
I've been thinking about DOL and cogject a bit and if you only want to create an object that load an existing object into a cog and then get unloaded it should be really easy. I'm currently at work but will post some ideas tonight.
Writing objects that fully take advantage of DOL is a bit of a pain because it really needs two features that none of the compilers support. These are 1. Getting the index of an item in the method table and 2. Not including a referenced object in the binary output file. There are workarounds for both but they are messy and prone to errors.
If the sole purpose is only to load cogs and then free memory we can get around both problems easily by putting all the hard bits in a single object that doesn't need changing.
Re the compilers not supporting these two things, I wonder if the authors of the BST/homespun might be able to help?
Is it simplistic to think of something that the compiler produces, a huge detailed text file for instance, with links to all the data and where it ended up etc, and you could then use this information? "Not including a referenced object"... there must be something one could do there. I'm thinking back to the CP/M project where we needed #ifdefs and then like and Brad was able to help out there with custom commands.
What would the syntax look like? Something that starts with # and doesn't include a block of code?
At the moment with cogjects, the aim is to be able to load and reload code as needed. For instance, consider a program that loads up a text graphic driver and asks the user some questions, then unloads the graphics driver and keyboard and loads up a graphics driver and mouse to play a game. At the moment you can recycle the cog space, but you end up with all the spin code from previous objects still in hub memory.
It would be great to be able to unload the spin code too. Treat an object as something that can be loaded and unloaded as often as you like. I think you already have this working but more objects are needed. I guess the essential ones are sd card, keyboard, mouse, TV, VGA (text and graphics).
With cogjects, I am trying to change the standard objects as little as possible. The pasm code stays exactly the same. Generally, all the 'start' methods are having one extra variable being passed which is the location of the cog loader space in hub ram. Most of the methods remain unchanged. Generally the start methods are requiring a lot of rewriting - eg any values that are implanted into pasm by way of variables now have to be done with a longmove to a pre-calculated area.
If we spliced that in with your code, I think you add a method driver which references all the methods.
Of all the existing objects I have coded, the keyboard one was probably the simplest because it has hardly any methods and all the values are passed to pasm with a PAR. I wonder what the standard keyboard object would look like in DOL?
First, attached is a new version of DOL that implements loading objects from sd cards and also a loadcogject method that loads an object from an sd card, gets it started in a cog and then exits. To use it you will first have to compile FullDuplexSerialCogjectCog.spin and save it to the root directry of your sd card with the filename "fdscc.bin". Obviously all the pin settings will need setting up for your board. Please note that this isn't tested at all because I don't currently have a spare sd card. Don't try it if there is stuff on your sd card you haven't backed up. If you have any problems with it post the stuff that gets dumped to the serial terminal and I'll have a look.
To get back your questions.
Brad indicated a year or two ago that it should be pretty easy to add the support needed but other stuff was understandably higher on his priority list at the time. We need two things indicated in the code box below.
OBJ
obj1 : #"dummyObject" 'This leaves a spot in the object table for the object and lets us use the object but doesn't actually include the object in the binary
CON
objIndex:=@obj1 'or
objIndex:=#obj1 'This puts the index of obj1 in the object/method table into objIndex
'I got this working in sphinx but I would have to find it again and I can't remember which syntax I used...
If you have a look at the attached FullDuplexSerial changes, all I had to do was separate the bit that gets loaded into the cog from the spin parts of the object. In this case it was easy. Some of the other drivers may be harder. Once that is done we simply change the spin part of the driver to call DOL to load the cog instead of doing it all in the one file.
Drac: as a temporary solution, could you use a special string in the line above to denote an address that is required. Then perhaps, using the listing you could scan and find the relevant addresses. ???
Comments
--trodoss
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
New ICON coming, gotta wait for the INK to heal, now we have colour!
I've started on another new version that cuts back on some of it's functionality to try and make it easier to use and take up less space.
I've also run into a problem of sorts, the heap object I'm using is byte aligned rather than long which is needed.
I also need to start using the latest version of fsrw.
And the biggest problem was that I had an sd card failure and haven't got around to fixing it yet...
I'm going to be away from my computer next week but after that I'll put some time into it, especially if someone here sends me a friendly reminder in my pms...
In fact, I'm looking at the code right now [noparse]:)[/noparse]
I want to make a linked list or something for it that will take advantage of the dynamic nature of it to help test with...
That's probably not surprising, sometimes I have problems with it
Okay, first reading the header, this should be simple and the code can be improved now that fsrw has a seek command
'read ProgramStart, VARStart and StackStart from header 'subtracting these will give us program length and var length 'set objectSize - from header 'read all the data in the header and use whatever we need [b]repeat[/b] 8 sd.pgetc objectLength:=sd.pgetc '8 'byte of varBASE objectLength.[b]byte[/b][noparse][[/noparse]­1]:=sd.pgetc '9 'byte of varBase objectLength:=objectLength-$10 varSize:=sd.pgetc 'A 'byte of stack start varSize.[b]byte[/b][noparse][[/noparse]­1]:=sd.pgetc 'B 'byte of stack start varSize:=VARSize-$10-objectLength-8 '-$8 fudge factor to be figured out [b]repeat[/b] 4 'get to end of header sd.pgetcThis gives us two important pieces of information, the object length (DAT+PUB+PRI+Object/method table) and the size of the VAR area. A little math is needed to figure it all out and I wouldn't have a clue why there is an extra -8 in the varSize calculation but it needed it to work.
Next is allocating memory and putting the addresses together in a useful format. I stripped it of a lot of extra stuff to get only the essentials.
Hopefully all easy to understand. Note that objectAddr is the "absolute" address of the object and not an offset.
The last important bit is what to do with objectAddr and that is in objsTemp
The arguments to this are the objectAddr we got previously from DOL and the position. position is the position of the address in the object table. To get that you start at 0 and add 1 for every PUB, PRI and Object (including each instance of an array object) to the object you want to change. This method when called will fix up the offsets so that then you can call the object with a normal call. Note that I had to split the calculation into two parts so that overflows during the calculation didn't become a problem (maybe the problem you were having with C-DOS?).
Now, there are still a couple of problems.
1. How do we get the position? I sorted that with a modification to sphinx. Pretty simple but I have to find what I did because I think I lost the modified code on a bad sd card [noparse]:([/noparse]
To use the mod you just did this
CON objectIndex=@objectworked nicely though only in a con block. Was an easy mod.
2. We still need a space in the object table. I started modifying sphinx so that you could use a notation like
object:@"object"
which would compile everything as normal and then just not include object in the final linking process. There was a free byte I used in the sop file to mark the unused objects but I couldn't figure out an easy way to not include the file.
Hopefully that helps. If anythings not clear than ask away.
Thanks for the explanation.· It's much clearer now.· I am still a bit confused by the VAR section.· I see that you allocate memory for it, and set the VAR offset in the table to point to it.· I thought the VAR entry in the table was an offset from the previous object's VAR space.· It appears that your using it as an absolute address.· I'll need to study that some more to understand it.
Dave
In this bit of code the second line fixes up the offset to the object and the third line fixes up the offset to the var section. "this" is a variable that was figured out earlier that conatains the absolute addresses of the object and var sections.
The demo is pretty basic and shows creating five new objects and assigning a different VAR variable to each, than calling each and seeing all the separate values.
There is code in DOL for loading objects from an SD card but it hasn't been tested yet.
The cogjects that are now working are keyboard, serial and the two cog VGA driver.
TV, Mouse and other VGA drivers are next on the list. The underlying SD card driver is Kye's code which happens to fill a cog, so this hub space can be recycled for other cogjects. 8k of ram already saved.
Combine that with the code here and it ought to be possible to move both cog code and the supporting spin code in and out of memory, either from an SD card, or from external memory. That means one can have arbitrarily large programs on a standard propeller.
How many DOL objects have been created so far, and how 'hard' are they to create?
I've been thinking about DOL and cogject a bit and if you only want to create an object that load an existing object into a cog and then get unloaded it should be really easy. I'm currently at work but will post some ideas tonight.
Writing objects that fully take advantage of DOL is a bit of a pain because it really needs two features that none of the compilers support. These are 1. Getting the index of an item in the method table and 2. Not including a referenced object in the binary output file. There are workarounds for both but they are messy and prone to errors.
If the sole purpose is only to load cogs and then free memory we can get around both problems easily by putting all the hard bits in a single object that doesn't need changing.
Is it simplistic to think of something that the compiler produces, a huge detailed text file for instance, with links to all the data and where it ended up etc, and you could then use this information? "Not including a referenced object"... there must be something one could do there. I'm thinking back to the CP/M project where we needed #ifdefs and then like and Brad was able to help out there with custom commands.
What would the syntax look like? Something that starts with # and doesn't include a block of code?
At the moment with cogjects, the aim is to be able to load and reload code as needed. For instance, consider a program that loads up a text graphic driver and asks the user some questions, then unloads the graphics driver and keyboard and loads up a graphics driver and mouse to play a game. At the moment you can recycle the cog space, but you end up with all the spin code from previous objects still in hub memory.
It would be great to be able to unload the spin code too. Treat an object as something that can be loaded and unloaded as often as you like. I think you already have this working but more objects are needed. I guess the essential ones are sd card, keyboard, mouse, TV, VGA (text and graphics).
With cogjects, I am trying to change the standard objects as little as possible. The pasm code stays exactly the same. Generally, all the 'start' methods are having one extra variable being passed which is the location of the cog loader space in hub ram. Most of the methods remain unchanged. Generally the start methods are requiring a lot of rewriting - eg any values that are implanted into pasm by way of variables now have to be done with a longmove to a pre-calculated area.
If we spliced that in with your code, I think you add a method driver which references all the methods.
Of all the existing objects I have coded, the keyboard one was probably the simplest because it has hardly any methods and all the values are passed to pasm with a PAR. I wonder what the standard keyboard object would look like in DOL?
First, attached is a new version of DOL that implements loading objects from sd cards and also a loadcogject method that loads an object from an sd card, gets it started in a cog and then exits. To use it you will first have to compile FullDuplexSerialCogjectCog.spin and save it to the root directry of your sd card with the filename "fdscc.bin". Obviously all the pin settings will need setting up for your board. Please note that this isn't tested at all because I don't currently have a spare sd card. Don't try it if there is stuff on your sd card you haven't backed up. If you have any problems with it post the stuff that gets dumped to the serial terminal and I'll have a look.
To get back your questions.
Brad indicated a year or two ago that it should be pretty easy to add the support needed but other stuff was understandably higher on his priority list at the time. We need two things indicated in the code box below.
OBJ obj1 : #"dummyObject" 'This leaves a spot in the object table for the object and lets us use the object but doesn't actually include the object in the binary CON objIndex:=@obj1 'or objIndex:=#obj1 'This puts the index of obj1 in the object/method table into objIndex 'I got this working in sphinx but I would have to find it again and I can't remember which syntax I used...If you have a look at the attached FullDuplexSerial changes, all I had to do was separate the bit that gets loaded into the cog from the spin parts of the object. In this case it was easy. Some of the other drivers may be harder. Once that is done we simply change the spin part of the driver to call DOL to load the cog instead of doing it all in the one file.