If "byte" is a keyword then the parser should be able to tell that byte[myarray][3] is not a nested array. It could be confusing for humans though. Maybe a syntax like "[byte myarray][3]" instead?
Another possibility might be to only support byte pointers instead of byte arrays, so you'd do something like:
myptr := @myarray[0]
byte[myptr + 3] := x
I will say that if you ever think you might want types in your language, it's best to bite the bullet early and get them in. Adding consistent types to fastspin has been one of the bigger challenges in getting it working with BASIC. They do introduce a lot of headaches (like what to do when operating with mixed type sizes) and it's only worse if the type information has to be grafted into a language infrastructure that wasn't originally designed for it.
If "byte" is a keyword then the parser should be able to tell that byte[myarray][3] is not a nested array. It could be confusing for humans though. Maybe a syntax like "[byte myarray][3]" instead?
Another possibility might be to only support byte pointers instead of byte arrays, so you'd do something like:
myptr := @myarray[0]
byte[myptr + 3] := x
I will say that if you ever think you might want types in your language, it's best to bite the bullet early and get them in. Adding consistent types to fastspin has been one of the bigger challenges in getting it working with BASIC. They do introduce a lot of headaches (like what to do when operating with mixed type sizes) and it's only worse if the type information has to be grafted into a language infrastructure that wasn't originally designed for it.
I'm not sure I want to explicitly support pointers. So far the only way to create a pointer is to pass an array, string, or object to a function. Like in C, these are passed by reference. Integers are passed by value. If I go to far I may as well just implement a C compiler. That, of course, would have explicit types. I'm sort of at an impasse with this language as I want it to be simple but also powerful and the two are hard to achieve at the same time. Maybe it would be better as a dynamically typed language like Javascript. That doesn't work well in the constrained memory of the Propeller though. I think what I'll probably do right now is the minimum I need to get my parser implemented and then think about whether I want to take it further. This is really mostly an exercise for me since I don't think anyone else will be interested in it. It's fun designing languages though.
I'm having a problem with my PASM VM and am wondering if I've missed something in the code below. The idea here is to copy a state vector in hub memory into and out of COG registers. I used to do this with a sequence of RDLONG and WRLONG instructions but I changed it to a loop to save COG memory. I've looked at this for a long time and can't see a problem with it. Can anyone else notice any issues?
DAT
regs
' virtual machine registers
tos long 0
sp long 0
fp long 0
pc long 0
efp long 0
' stack limits
stack long 0
stackTop long 0
' set if single stepping
stepping long 0
end_regs
' temporaries used by the VM instructions
r1 long 0
r2 long 0
' pointer to the state vector in hub memory
state_ptr long 0
' code to load the hub state vector into COG registers
mov r1,state_ptr
movd :rloop, #regs
mov r2,#(end_regs - regs)
:rloop rdlong 0-0,r1
add r1,#4
add :rloop, dstinc
djnz r2,#:rloop
' code to store the COG registers into the state vector
mov r1,state_ptr
movd :sloop, #regs
mov r2,#(end_regs - regs)
:sloop wrlong 0-0,r1
add r1,#4
add :sloop, dstinc
djnz r2,#:sloop
dstinc long $200
I just thought of something. Does the code "#(end_regs - regs)" subtract the long addresses of these labels or the byte addresses? In other words, does it evaluate to 8 or 32?
regs
' virtual machine registers
tos long 0
sp long 0
fp long 0
pc long 0
efp long 0
' stack limits
stack long 0
stackTop long 0
' set if single stepping
stepping long 0
end_regs
' code to load the hub state vector into COG registers
mov r1,state_ptr
movd :rloop, #regs
mov r2,#(end_regs - regs)
:rloop rdlong 0-0,r1
add r1,#4
add :rloop, dstinc
djnz r2,#:rloop
According to both bstc and fastspin "#(end_regs - regs)" is the same as "#8", so it is doing long addresses.
Are you absolutely sure this is the failing code? You had some other minor reorganizations; is it possible a typo crept in there? If you replace this loop with the old code do things work? If you're anything like me, you suspect this code for the good reason that it's the most complicated piece of your changes, but this part does seem OK.
' code to store the COG registers into the state vector
movd :sloop, #regs
mov r1,state_ptr
mov r2,#(end_regs - regs)
:sloop wrlong 0-0,r1
add :sloop, dstinc
add r1,#4
djnz r2,#:sloop
' code to store the COG registers into the state vector
movd :sloop, #regs
mov r1,state_ptr
mov r2,#(end_regs - regs)
:sloop wrlong 0-0,r1
add :sloop, dstinc
add r1,#4
djnz r2,#:sloop
I looked it up and you only need one instruction between the instruction that modifies and the execution of the modified instruction.
According to both bstc and fastspin "#(end_regs - regs)" is the same as "#8", so it is doing long addresses.
Are you absolutely sure this is the failing code? You had some other minor reorganizations; is it possible a typo crept in there? If you replace this loop with the old code do things work? If you're anything like me, you suspect this code for the good reason that it's the most complicated piece of your changes, but this part does seem OK.
Eric
Yeah, it's probably something else. I just thought, as you said, this change was a bit tricky involving self-modifying code. I'll keep looking. Thanks for checking my loops.
According to both bstc and fastspin "#(end_regs - regs)" is the same as "#8", so it is doing long addresses.
Are you absolutely sure this is the failing code? You had some other minor reorganizations; is it possible a typo crept in there? If you replace this loop with the old code do things work? If you're anything like me, you suspect this code for the good reason that it's the most complicated piece of your changes, but this part does seem OK.
Eric
Yeah, it's probably something else. I just thought, as you said, this change was a bit tricky involving self-modifying code. I'll keep looking. Thanks for checking my loops.
I fixed the bug and you were right. It wasn't in the state load/store code. Now on to byte addressing of arrays. I'm going to go with the foo.byte[n] syntax since it seems consistent with my property access syntax. You can think of "byte" as a property of an array or object that lets you address it as an array of bytes. Just that final addition should give me what I need to write my parser.
I've been thinking about this and wondering why anyone would want to use this language rather than just use C or C++ which are already available on the Propeller. Here is what I came up with. This language (what shall I call it?) has a fairly powerful way of describing initialized data. The network of locations and objects that typically exist in a text adventure game requires lots of pointers. There needs to be a pointer from the "living room" object to the "hall" object if it is possible to move between those two locations. There also has to be a list of objects contained in a location so it can be shown when a player enters that location or if a player tries to manipulate an object. This means that the data structures of the language must support setting up this network. The way I do that is to assume that any reference you make to an as yet undefined object when specifying the initial value of an array element or a property is assumed to be a reference to an array or object that has not yet been defined. In this way you can make forward references to objects or arrays that have not yet been defined hence allowing you to create the network of objects needed to represent the game map. It seems to me that this sort of network of objects could possibly be useful for other purposes as well.
Also, this language has an object system that is more like JavaScript than it is like C++. There really aren't any classes. Every object has a set of properties and an object can inherit from another object which is referred to as its parent. Actually, I do use the term CLASS at present but it really just means the parent of the object. If you use an expression like "foo.bar", the "bar" part is a property name. This resolves to a small integer at compile time. The runtime code then looks for a property with that "tag" and returns its value. If there is no property with that tag in the object itself, it's parent is checked and then it's parent's parent, etc. This allows some properties to be shared among all objects that inherit from the same parent.
Beyond that, the language supports a fairly large subset of C statements and expressions. One big omission currently is switch statements although a simple version of them would be easy to add.
The compiler translates the source language into byte codes and the byte codes are executed by a PASM virtual machine. At present, only one COG can run one of these programs although there is nothing preventing me from allowing you to launch another COG sharing the same data as long as you are careful to avoid access conflicts. Even those could be resolved with locks if necessary.
Anyway, I'm not sure any of these non-C features would be enough to convince anyone to try this. I guess it depends on whether they help with any problem you're trying to solve.
Another thing I've considered is just making a data structure compiler that would produce these networks of objects and a C library that would allow you to access them. That wouldn't be difficult.
OK, then you'll need to ensure that you add enough padding to handle the stack and VAR space if you use Eric's method. Or you could concatenate the compiled AdvSys2 code after the Spin binary, and then adjust the DBASE, DCURR and VBASE values in the Spin header. Each of these three values would be increased by the size of the AdvSys2 code. You would also need to adjust the checksum in the header.
I know how to adjust DBASE, DCURR, and VBASE and recompute the checksum but I'm not sure how to actually find the appended image in the Spin code. How do I know its address when I compile the Spin code? Is there a built-in symbol that points to where VBASE points in the binary image?
OK, then you'll need to ensure that you add enough padding to handle the stack and VAR space if you use Eric's method. Or you could concatenate the compiled AdvSys2 code after the Spin binary, and then adjust the DBASE, DCURR and VBASE values in the Spin header. Each of these three values would be increased by the size of the AdvSys2 code. You would also need to adjust the checksum in the header.
I know how to adjust DBASE, DCURR, and VBASE and recompute the checksum but I'm not sure how to actually find the appended image in the Spin code. How do I know its address when I compile the Spin code? Is there a built-in symbol that points to where VBASE points in the binary image?
I just realized that I can probably use the same scheme I did for patching baud rate and rx/tx pins into PropGCC programs. Just locate the first DAT section and make sure that the first few locations contain the information that needs to be patched including the offset to the VM image I'm appending to the Spin binary.
OK, then you'll need to ensure that you add enough padding to handle the stack and VAR space if you use Eric's method. Or you could concatenate the compiled AdvSys2 code after the Spin binary, and then adjust the DBASE, DCURR and VBASE values in the Spin header. Each of these three values would be increased by the size of the AdvSys2 code. You would also need to adjust the checksum in the header.
I know how to adjust DBASE, DCURR, and VBASE and recompute the checksum but I'm not sure how to actually find the appended image in the Spin code. How do I know its address when I compile the Spin code? Is there a built-in symbol that points to where VBASE points in the binary image?
I just padded the Spin binary out to a fixed size (with an external tool) so that the appended image always started at the same address. I think that's the simplest solution, although you could also look up VBASE in the first few words of memory as well.
OK, then you'll need to ensure that you add enough padding to handle the stack and VAR space if you use Eric's method. Or you could concatenate the compiled AdvSys2 code after the Spin binary, and then adjust the DBASE, DCURR and VBASE values in the Spin header. Each of these three values would be increased by the size of the AdvSys2 code. You would also need to adjust the checksum in the header.
I know how to adjust DBASE, DCURR, and VBASE and recompute the checksum but I'm not sure how to actually find the appended image in the Spin code. How do I know its address when I compile the Spin code? Is there a built-in symbol that points to where VBASE points in the binary image?
I just padded the Spin binary out to a fixed size (with an external tool) so that the appended image always started at the same address. I think that's the simplest solution, although you could also look up VBASE in the first few words of memory as well.
Eric
Interesting. I had forgotten that the .binary file header remains in memory after the program has started. That's an interesting idea.
OK, then you'll need to ensure that you add enough padding to handle the stack and VAR space if you use Eric's method. Or you could concatenate the compiled AdvSys2 code after the Spin binary, and then adjust the DBASE, DCURR and VBASE values in the Spin header. Each of these three values would be increased by the size of the AdvSys2 code. You would also need to adjust the checksum in the header.
I know how to adjust DBASE, DCURR, and VBASE and recompute the checksum but I'm not sure how to actually find the appended image in the Spin code. How do I know its address when I compile the Spin code? Is there a built-in symbol that points to where VBASE points in the binary image?
I just padded the Spin binary out to a fixed size (with an external tool) so that the appended image always started at the same address. I think that's the simplest solution, although you could also look up VBASE in the first few words of memory as well.
Eric
Do you have any idea whether vbase is guaranteed to be on a long boundary? Will I need to pad to a long boundary if my image requires that?
Yes, VBASE is always on a long boundary. If you use Eric's padding method you will need to ensure that the pad is large enough to handle your stack needs. The stack will grow upward toward where your AdvSys2 code is residing, so you need to make sure the stack won't overwrite your code.
If you use the other method of adjusting the Spin state variables in the header you could store a pointer to the AdvSys2 code in a variable at the beginning of the DAT space in the top object, as you suggested. This will be located just after the method table, and it is fairly easy to compute where that is located. I used that method to store system configuration variables in the boot program for spinix.
Yes, VBASE is always on a long boundary. If you use Eric's padding method you will need to ensure that the pad is large enough to handle your stack needs. The stack will grow upward toward where your AdvSys2 code is residing, so you need to make sure the stack won't overwrite your code.
If you use the other method of adjusting the Spin state variables in the header you could store a pointer to the AdvSys2 code in a variable at the beginning of the DAT space in the top object, as you suggested. This will be located just after the method table, and it is fairly easy to compute where that is located. I used that method to store system configuration variables in the boot program for spinix.
I wondered why Eric had used the padding method until I realized that that allows his compiler to assign physical addresses to things. My compiler only assigns addresses relative to a base address because it doesn't know where the image will live in memory during runtime. Unfortunately, that means I need to add that base address to pointers constantly.
Thanks for the suggestions about appending an image to a Propeller .binary file. I've checked in a simple program called propbinary that takes a .dat file created by adv2com and creates a .binary file from it. Next I'll integrate it into the compiler itself as an option. That will allow you to compile an advsys2 program into a Propeller .binary file directly without needing openspin installed.
Here is the code I wrote to combine my VM image with some Spin template code. The inputs to the function are an array containing the compiled Spin template and another array containing the VM image. The output is the resulting .binary file. This seems to work but I was wondering if anyone who knows the .binary file format better could look at it to see if I've made any wrong assumptions.
/* spin binary file header structure */
typedef struct {
uint32_t clkfreq;
uint8_t clkmode;
uint8_t chksum;
uint16_t pbase;
uint16_t vbase;
uint16_t dbase;
uint16_t pcurr;
uint16_t dcurr;
} SpinHdr;
/* spin object */
typedef struct {
uint16_t next;
uint8_t pubcnt;
uint8_t objcnt;
uint16_t pubstart;
uint16_t numlocals;
} SpinObj;
/* DAT header in serial_helper.spin */
typedef struct {
uint32_t baudrate;
uint8_t rxpin;
uint8_t txpin;
uint16_t imagebase;
} DatHdr;
/* BuildBinary - build a .binary file from a template and a VM image */
static uint8_t *BuildBinary(uint8_t *template, int templateSize, uint8_t *image, int imageSize, int *pBinarySize)
{
int binarySize, paddedImageSize;
uint8_t *binary;
SpinHdr *hdr;
SpinObj *obj;
DatHdr *dat;
/* pad the image to a long boundary */
paddedImageSize = (imageSize + sizeof(uint32_t) - 1) & ~(sizeof(uint32_t) - 1);
/* compute the size of the binary file including the image */
binarySize = templateSize + paddedImageSize;
/* allocate space for the binary file */
if (!(binary = (uint8_t *)malloc(binarySize)))
return NULL;
memset(binary, 0, binarySize);
/* copy the template to the start of the file */
memcpy(binary, template, templateSize);
/* update the binary file header to include the VM image */
hdr = (SpinHdr *)binary;
obj = (SpinObj *)(binary + hdr->pbase);
dat = (DatHdr *)((uint8_t *)obj + (obj->pubcnt + obj->objcnt) * sizeof(uint32_t));
dat->imagebase = hdr->vbase;
memcpy(binary + hdr->vbase, image, imageSize);
hdr->vbase += paddedImageSize;
hdr->dbase += paddedImageSize;
hdr->dcurr += paddedImageSize;
UpdateChecksum(binary, binarySize);
/* return the binary */
*pBinarySize = binarySize;
return binary;
}
/* target checksum for a binary file */
#define SPIN_TARGET_CHECKSUM 0x14
/* UpdateChecksum - recompute the checksum */
static void UpdateChecksum(uint8_t *binary, int size)
{
SpinHdr *hdr = (SpinHdr *)binary;
int chksum = 0;
hdr->chksum = 0;
while (--size >= 0)
chksum += *binary++;
hdr->chksum = SPIN_TARGET_CHECKSUM - chksum;
}
I've never heard of the STACK statement. Where is that documented? BST doesn't recognize the STACK symbol. I'm on a new computer, and I currently don't have the Prop Tool installed, so I can't try it there. AFAIK the Spin interpreter does not check for a stack limit.
EDIT: I'm not even sure what the Spin interpreter would do if it did support a stack limit. Silently not write to the stack? I'm pretty sure there is no STACK statement.
_STACK Constant: Pre-defined, one-time settable constant for specifying the size of an application’s stack space.
CON _STACK = Expression
Expression is an integer expression that indicates the number of longs to reserve for stack space.
Explanation _STACK is a pre-defined, one-time settable optional constant that specifies the required stack space of an application. This value is added to _FREE if specified, to determine the total amount of stack/free memory space to reserve for a Propeller application. Use _STACK if an application requires a minimum amount of stack space in order to run properly. If the resulting compiled application is too large to allow the specified stack space, an error message will be displayed. For example: CON _STACK = 3000
The _STACK declaration in the above CON block indicates that the application needs to have at least 3,000 longs of stack space left over after compilation. If the resulting compiled application does not have that much room left over, an error message will indicate by how much it was exceeded. This is a good way to prevent successful compiles of an application that will fail to run properly due to lack of memory. Note that only the top object file can set the value of _STACK. Any child object’s _STACK declarations will be ignored. The stack space reserved by this constant is used by the application’s main cog to store temporary data such as call stacks, parameters, and intermediate expression results
This sets the minimum needed stack size
and there is _FREE Propeller Help Page 110
_FREE Constant: Pre-defined, one-time settable constant for specifying the size of an application’s free space.
CON _FREE = Expression
Expression is an integer expression that indicates the number of longs to reserve for free space.
Explanation _FREE is a pre-defined, one-time settable optional constant that specifies the required free memory space of an application. This value is added to _STACK, if specified, to determine the total amount of free/stack memory space to reserve for a Propeller Application. Use _FREE if an application requires a minimum amount of free memory in order to run properly. If the resulting compiled application is too large to allow the specified free memory, an error message will be displayed. For example: CON _FREE = 1000
The _FREE declaration in the above CON block indicates that the application needs to have at least 1,000 longs of free memory left over after compilation. If the resulting compiled application does not have that much room left over, an error message will indicate by how much it was exceeded. This is a good way to prevent successful compiles of an application that will fail to run properly due to lack of memory. Note that only the top object file can set the value of _FREE. Any child object’s _FREE declarations will be ignored.
Comments
Mike
Another possibility might be to only support byte pointers instead of byte arrays, so you'd do something like:
I will say that if you ever think you might want types in your language, it's best to bite the bullet early and get them in. Adding consistent types to fastspin has been one of the bigger challenges in getting it working with BASIC. They do introduce a lot of headaches (like what to do when operating with mixed type sizes) and it's only worse if the type information has to be grafted into a language infrastructure that wasn't originally designed for it.
Could you post the working non-loop code ?
Bean
Are you absolutely sure this is the failing code? You had some other minor reorganizations; is it possible a typo crept in there? If you replace this loop with the old code do things work? If you're anything like me, you suspect this code for the good reason that it's the most complicated piece of your changes, but this part does seem OK.
Eric
switch etc
Also, this language has an object system that is more like JavaScript than it is like C++. There really aren't any classes. Every object has a set of properties and an object can inherit from another object which is referred to as its parent. Actually, I do use the term CLASS at present but it really just means the parent of the object. If you use an expression like "foo.bar", the "bar" part is a property name. This resolves to a small integer at compile time. The runtime code then looks for a property with that "tag" and returns its value. If there is no property with that tag in the object itself, it's parent is checked and then it's parent's parent, etc. This allows some properties to be shared among all objects that inherit from the same parent.
Beyond that, the language supports a fairly large subset of C statements and expressions. One big omission currently is switch statements although a simple version of them would be easy to add.
The compiler translates the source language into byte codes and the byte codes are executed by a PASM virtual machine. At present, only one COG can run one of these programs although there is nothing preventing me from allowing you to launch another COG sharing the same data as long as you are careful to avoid access conflicts. Even those could be resolved with locks if necessary.
Anyway, I'm not sure any of these non-C features would be enough to convince anyone to try this. I guess it depends on whether they help with any problem you're trying to solve.
Another thing I've considered is just making a data structure compiler that would produce these networks of objects and a C library that would allow you to access them. That wouldn't be difficult.
This code seems to work:
I just padded the Spin binary out to a fixed size (with an external tool) so that the appended image always started at the same address. I think that's the simplest solution, although you could also look up VBASE in the first few words of memory as well.
Eric
If you use the other method of adjusting the Spin state variables in the header you could store a pointer to the AdvSys2 code in a variable at the beginning of the DAT space in the top object, as you suggested. This will be located just after the method table, and it is fairly easy to compute where that is located. I used that method to store system configuration variables in the boot program for spinix.
Enjoy!
Mike
EDIT: I'm not even sure what the Spin interpreter would do if it did support a stack limit. Silently not write to the stack? I'm pretty sure there is no STACK statement.
_STACK Constant: Pre-defined, one-time settable constant for specifying the size of an application’s stack space.
CON _STACK = Expression
Expression is an integer expression that indicates the number of longs to reserve for stack space.
Explanation _STACK is a pre-defined, one-time settable optional constant that specifies the required stack space of an application. This value is added to _FREE if specified, to determine the total amount of stack/free memory space to reserve for a Propeller application. Use _STACK if an application requires a minimum amount of stack space in order to run properly. If the resulting compiled application is too large to allow the specified stack space, an error message will be displayed. For example: CON _STACK = 3000
The _STACK declaration in the above CON block indicates that the application needs to have at least 3,000 longs of stack space left over after compilation. If the resulting compiled application does not have that much room left over, an error message will indicate by how much it was exceeded. This is a good way to prevent successful compiles of an application that will fail to run properly due to lack of memory. Note that only the top object file can set the value of _STACK. Any child object’s _STACK declarations will be ignored. The stack space reserved by this constant is used by the application’s main cog to store temporary data such as call stacks, parameters, and intermediate expression results
This sets the minimum needed stack size
and there is _FREE Propeller Help Page 110
_FREE Constant: Pre-defined, one-time settable constant for specifying the size of an application’s free space.
CON _FREE = Expression
Expression is an integer expression that indicates the number of longs to reserve for free space.
Explanation _FREE is a pre-defined, one-time settable optional constant that specifies the required free memory space of an application. This value is added to _STACK, if specified, to determine the total amount of free/stack memory space to reserve for a Propeller Application. Use _FREE if an application requires a minimum amount of free memory in order to run properly. If the resulting compiled application is too large to allow the specified free memory, an error message will be displayed. For example: CON _FREE = 1000
The _FREE declaration in the above CON block indicates that the application needs to have at least 1,000 longs of free memory left over after compilation. If the resulting compiled application does not have that much room left over, an error message will indicate by how much it was exceeded. This is a good way to prevent successful compiles of an application that will fail to run properly due to lack of memory. Note that only the top object file can set the value of _FREE. Any child object’s _FREE declarations will be ignored.
Mike