Shop OBEX P1 Docs P2 Docs Learn Events
AdvSys2 Development - RIP - Page 2 — Parallax Forums

AdvSys2 Development - RIP

2

Comments

  • at least unsigned byte and zero terminated string not sure if word is really needed

    Mike
  • If "byte" is a keyword then the parser should be able to tell that byte[myarray][3] is not a nested array. It could be confusing for humans though. Maybe a syntax like "[byte myarray][3]" instead?

    Another possibility might be to only support byte pointers instead of byte arrays, so you'd do something like:
      myptr := @myarray[0]
      byte[myptr + 3] := x
    

    I will say that if you ever think you might want types in your language, it's best to bite the bullet early and get them in. Adding consistent types to fastspin has been one of the bigger challenges in getting it working with BASIC. They do introduce a lot of headaches (like what to do when operating with mixed type sizes) and it's only worse if the type information has to be grafted into a language infrastructure that wasn't originally designed for it.
  • ersmith wrote: »
    If "byte" is a keyword then the parser should be able to tell that byte[myarray][3] is not a nested array. It could be confusing for humans though. Maybe a syntax like "[byte myarray][3]" instead?

    Another possibility might be to only support byte pointers instead of byte arrays, so you'd do something like:
      myptr := @myarray[0]
      byte[myptr + 3] := x
    

    I will say that if you ever think you might want types in your language, it's best to bite the bullet early and get them in. Adding consistent types to fastspin has been one of the bigger challenges in getting it working with BASIC. They do introduce a lot of headaches (like what to do when operating with mixed type sizes) and it's only worse if the type information has to be grafted into a language infrastructure that wasn't originally designed for it.
    I'm not sure I want to explicitly support pointers. So far the only way to create a pointer is to pass an array, string, or object to a function. Like in C, these are passed by reference. Integers are passed by value. If I go to far I may as well just implement a C compiler. That, of course, would have explicit types. I'm sort of at an impasse with this language as I want it to be simple but also powerful and the two are hard to achieve at the same time. Maybe it would be better as a dynamically typed language like Javascript. That doesn't work well in the constrained memory of the Propeller though. I think what I'll probably do right now is the minimum I need to get my parser implemented and then think about whether I want to take it further. This is really mostly an exercise for me since I don't think anyone else will be interested in it. It's fun designing languages though.
  • David BetzDavid Betz Posts: 14,511
    edited 2018-09-19 12:28
    I'm having a problem with my PASM VM and am wondering if I've missed something in the code below. The idea here is to copy a state vector in hub memory into and out of COG registers. I used to do this with a sequence of RDLONG and WRLONG instructions but I changed it to a loop to save COG memory. I've looked at this for a long time and can't see a problem with it. Can anyone else notice any issues?
    DAT
    
    regs
    
    ' virtual machine registers
    tos         long    0
    sp          long    0
    fp          long    0
    pc          long    0
    efp         long    0
    
    ' stack limits
    stack       long    0
    stackTop    long    0
    
    ' set if single stepping
    stepping    long    0
    
    end_regs
    
    ' temporaries used by the VM instructions
    r1          long    0
    r2          long    0
    
    ' pointer to the state vector in hub memory
    state_ptr   long    0
    
    ' code to load the hub state vector into COG registers
            mov     r1,state_ptr
            movd    :rloop, #regs
            mov     r2,#(end_regs - regs)
    :rloop  rdlong  0-0,r1
            add     r1,#4
            add     :rloop, dstinc
            djnz    r2,#:rloop
    
    ' code to store the COG registers into the state vector
            mov     r1,state_ptr
            movd    :sloop, #regs
            mov     r2,#(end_regs - regs)
    :sloop  wrlong  0-0,r1
            add     r1,#4
            add     :sloop, dstinc
            djnz    r2,#:sloop
    
    dstinc  long    $200
    
  • BeanBean Posts: 8,129
    David,
    Could you post the working non-loop code ?

    Bean
  • It looked something like this although I've rearranged the order some.
            mov     r1,state_ptr
            rdlong  fp,r1           ' load fp
            add     r1,#4
            rdlong  sp,r1           ' load sp
            add     r1,#4
            rdlong  tos,r1          ' load tos
            add     r1,#4
            rdlong  pc,r1           ' load pc
            add     r1,#4
            rdlong  stepping,r1     ' load stepping
            add     r1,#4
            rdlong  stack,r1        ' load stack
            add     r1,#4
            rdlong  stackTop,r1     ' load the stack size
    
  • Here is the exact code. There is still some reordering though.
    load_state
            mov     r1,state_ptr
            rdlong  fp,r1           ' load fp
            add     r1,#4
            rdlong  sp,r1           ' load sp
            add     r1,#4
            rdlong  tos,r1          ' load tos
            add     r1,#4
            rdlong  pc,r1           ' load pc
            add     r1,#4
            rdlong  efp,r1          ' load efp
            add     r1,#4
            rdlong  stepping,r1     ' load stepping
    
    store_state
            mov     r2,state_ptr
            wrlong  fp,r2       ' store fp
            add     r2,#4
            wrlong  sp,r2       ' store sp
            add     r2,#4
            wrlong  tos,r2      ' store tos
            add     r2,#4
            wrlong  pc,r2       ' store pc
            add     r2,#4
            wrlong  efp,r2      ' store efp
            add     r2,#4
            wrlong  stepping,r2 ' store stepping
            add     r2,#4
            wrlong  stack,r2    ' store stack
            add     r2,#4
            wrlong  stackTop,r2 ' store stackTop
    
  • David BetzDavid Betz Posts: 14,511
    edited 2018-09-19 21:37
    I just thought of something. Does the code "#(end_regs - regs)" subtract the long addresses of these labels or the byte addresses? In other words, does it evaluate to 8 or 32?
    regs
    
    ' virtual machine registers
    tos         long    0
    sp          long    0
    fp          long    0
    pc          long    0
    efp         long    0
    
    ' stack limits
    stack       long    0
    stackTop    long    0
    
    ' set if single stepping
    stepping    long    0
    
    end_regs
    
    ' code to load the hub state vector into COG registers
            mov     r1,state_ptr
            movd    :rloop, #regs
            mov     r2,#(end_regs - regs)
    :rloop  rdlong  0-0,r1
            add     r1,#4
            add     :rloop, dstinc
            djnz    r2,#:rloop
    
  • According to both bstc and fastspin "#(end_regs - regs)" is the same as "#8", so it is doing long addresses.

    Are you absolutely sure this is the failing code? You had some other minor reorganizations; is it possible a typo crept in there? If you replace this loop with the old code do things work? If you're anything like me, you suspect this code for the good reason that it's the most complicated piece of your changes, but this part does seem OK.

    Eric
  • msrobotsmsrobots Posts: 3,701
    edited 2018-09-19 22:33
    wasn't that two ins between modifying and using?

    switch
            add     r1,#4
            add     :rloop, dstinc
    to
            add     :rloop, dstinc
            add     r1,#4
    
    etc
    ' code to store the COG registers into the state vector
            movd    :sloop, #regs
            mov     r1,state_ptr
            mov     r2,#(end_regs - regs)
    :sloop  wrlong  0-0,r1
            add     :sloop, dstinc
            add     r1,#4
            djnz    r2,#:sloop
    
    
  • msrobots wrote: »
    wasn't that two ins between modifying and using?

    switch
            add     r1,#4
            add     :rloop, dstinc
    to
            add     :rloop, dstinc
            add     r1,#4
    
    etc
    ' code to store the COG registers into the state vector
            movd    :sloop, #regs
            mov     r1,state_ptr
            mov     r2,#(end_regs - regs)
    :sloop  wrlong  0-0,r1
            add     :sloop, dstinc
            add     r1,#4
            djnz    r2,#:sloop
    
    
    I looked it up and you only need one instruction between the instruction that modifies and the execution of the modified instruction.
  • ersmith wrote: »
    According to both bstc and fastspin "#(end_regs - regs)" is the same as "#8", so it is doing long addresses.

    Are you absolutely sure this is the failing code? You had some other minor reorganizations; is it possible a typo crept in there? If you replace this loop with the old code do things work? If you're anything like me, you suspect this code for the good reason that it's the most complicated piece of your changes, but this part does seem OK.

    Eric
    Yeah, it's probably something else. I just thought, as you said, this change was a bit tricky involving self-modifying code. I'll keep looking. Thanks for checking my loops.

  • David Betz wrote: »
    ersmith wrote: »
    According to both bstc and fastspin "#(end_regs - regs)" is the same as "#8", so it is doing long addresses.

    Are you absolutely sure this is the failing code? You had some other minor reorganizations; is it possible a typo crept in there? If you replace this loop with the old code do things work? If you're anything like me, you suspect this code for the good reason that it's the most complicated piece of your changes, but this part does seem OK.

    Eric
    Yeah, it's probably something else. I just thought, as you said, this change was a bit tricky involving self-modifying code. I'll keep looking. Thanks for checking my loops.
    I fixed the bug and you were right. It wasn't in the state load/store code. Now on to byte addressing of arrays. I'm going to go with the foo.byte[n] syntax since it seems consistent with my property access syntax. You can think of "byte" as a property of an array or object that lets you address it as an array of bytes. Just that final addition should give me what I need to write my parser.
  • I got access to COG registers working. Here is a blinking LED example:
    def LED1_PIN = 26;
    def LED1_MASK = 1 << LED1_PIN;
    
    def LED2_PIN = 27;
    def LED2_MASK = 1 << LED2_PIN;
    
    def LEDS_MASK = LED1_MASK | LED2_MASK;
    
    def waitcnt(n)
    {
        asm {
            LADDR 0
            LOAD
            NATIVE mov      t1, cnt
            NATIVE add      t1, tos
            NATIVE waitcnt  t1, #0
            DROP
        }
    }
    
    def main()
    {
        // set the LED pins to outputs
        dira |= LEDS_MASK;
    
        // turn on LED1 and off LED2
        outa |= LED1_MASK;
        outa &= ~LED2_MASK;
    
        while (1) {
    
            // toggle both LEDs
            outa ^= LEDS_MASK;
    
            // wait half a second
    	waitcnt(40000000);
        }
    }
    
  • I've been thinking about this and wondering why anyone would want to use this language rather than just use C or C++ which are already available on the Propeller. Here is what I came up with. This language (what shall I call it?) has a fairly powerful way of describing initialized data. The network of locations and objects that typically exist in a text adventure game requires lots of pointers. There needs to be a pointer from the "living room" object to the "hall" object if it is possible to move between those two locations. There also has to be a list of objects contained in a location so it can be shown when a player enters that location or if a player tries to manipulate an object. This means that the data structures of the language must support setting up this network. The way I do that is to assume that any reference you make to an as yet undefined object when specifying the initial value of an array element or a property is assumed to be a reference to an array or object that has not yet been defined. In this way you can make forward references to objects or arrays that have not yet been defined hence allowing you to create the network of objects needed to represent the game map. It seems to me that this sort of network of objects could possibly be useful for other purposes as well.

    Also, this language has an object system that is more like JavaScript than it is like C++. There really aren't any classes. Every object has a set of properties and an object can inherit from another object which is referred to as its parent. Actually, I do use the term CLASS at present but it really just means the parent of the object. If you use an expression like "foo.bar", the "bar" part is a property name. This resolves to a small integer at compile time. The runtime code then looks for a property with that "tag" and returns its value. If there is no property with that tag in the object itself, it's parent is checked and then it's parent's parent, etc. This allows some properties to be shared among all objects that inherit from the same parent.

    Beyond that, the language supports a fairly large subset of C statements and expressions. One big omission currently is switch statements although a simple version of them would be easy to add.

    The compiler translates the source language into byte codes and the byte codes are executed by a PASM virtual machine. At present, only one COG can run one of these programs although there is nothing preventing me from allowing you to launch another COG sharing the same data as long as you are careful to avoid access conflicts. Even those could be resolved with locks if necessary.

    Anyway, I'm not sure any of these non-C features would be enough to convince anyone to try this. I guess it depends on whether they help with any problem you're trying to solve.

    Another thing I've considered is just making a data structure compiler that would produce these networks of objects and a C library that would allow you to access them. That wouldn't be difficult.
  • I got byte addressing working but unfortunately broke something in the PASM VM in the process. Ugh.

    This code seems to work:
    var foo, myArray[2];
    
    def main()
    {
        var bar;
        foo = bar;
        bar = foo;
        foo = myArray[1];
        foo = myArray.byte[1];
        myArray[1] = foo;
        myArray.byte[1] = foo;
        myArray.byte[2]++;
        ++myArray.byte[2];
        myArray.byte[3] += 4;
    }
    
  • Dave Hein wrote: »
    OK, then you'll need to ensure that you add enough padding to handle the stack and VAR space if you use Eric's method. Or you could concatenate the compiled AdvSys2 code after the Spin binary, and then adjust the DBASE, DCURR and VBASE values in the Spin header. Each of these three values would be increased by the size of the AdvSys2 code. You would also need to adjust the checksum in the header.
    I know how to adjust DBASE, DCURR, and VBASE and recompute the checksum but I'm not sure how to actually find the appended image in the Spin code. How do I know its address when I compile the Spin code? Is there a built-in symbol that points to where VBASE points in the binary image?

  • David Betz wrote: »
    Dave Hein wrote: »
    OK, then you'll need to ensure that you add enough padding to handle the stack and VAR space if you use Eric's method. Or you could concatenate the compiled AdvSys2 code after the Spin binary, and then adjust the DBASE, DCURR and VBASE values in the Spin header. Each of these three values would be increased by the size of the AdvSys2 code. You would also need to adjust the checksum in the header.
    I know how to adjust DBASE, DCURR, and VBASE and recompute the checksum but I'm not sure how to actually find the appended image in the Spin code. How do I know its address when I compile the Spin code? Is there a built-in symbol that points to where VBASE points in the binary image?
    I just realized that I can probably use the same scheme I did for patching baud rate and rx/tx pins into PropGCC programs. Just locate the first DAT section and make sure that the first few locations contain the information that needs to be patched including the offset to the VM image I'm appending to the Spin binary.

  • David Betz wrote: »
    Dave Hein wrote: »
    OK, then you'll need to ensure that you add enough padding to handle the stack and VAR space if you use Eric's method. Or you could concatenate the compiled AdvSys2 code after the Spin binary, and then adjust the DBASE, DCURR and VBASE values in the Spin header. Each of these three values would be increased by the size of the AdvSys2 code. You would also need to adjust the checksum in the header.
    I know how to adjust DBASE, DCURR, and VBASE and recompute the checksum but I'm not sure how to actually find the appended image in the Spin code. How do I know its address when I compile the Spin code? Is there a built-in symbol that points to where VBASE points in the binary image?

    I just padded the Spin binary out to a fixed size (with an external tool) so that the appended image always started at the same address. I think that's the simplest solution, although you could also look up VBASE in the first few words of memory as well.

    Eric
  • ersmith wrote: »
    David Betz wrote: »
    Dave Hein wrote: »
    OK, then you'll need to ensure that you add enough padding to handle the stack and VAR space if you use Eric's method. Or you could concatenate the compiled AdvSys2 code after the Spin binary, and then adjust the DBASE, DCURR and VBASE values in the Spin header. Each of these three values would be increased by the size of the AdvSys2 code. You would also need to adjust the checksum in the header.
    I know how to adjust DBASE, DCURR, and VBASE and recompute the checksum but I'm not sure how to actually find the appended image in the Spin code. How do I know its address when I compile the Spin code? Is there a built-in symbol that points to where VBASE points in the binary image?

    I just padded the Spin binary out to a fixed size (with an external tool) so that the appended image always started at the same address. I think that's the simplest solution, although you could also look up VBASE in the first few words of memory as well.

    Eric
    Interesting. I had forgotten that the .binary file header remains in memory after the program has started. That's an interesting idea.

  • ersmith wrote: »
    David Betz wrote: »
    Dave Hein wrote: »
    OK, then you'll need to ensure that you add enough padding to handle the stack and VAR space if you use Eric's method. Or you could concatenate the compiled AdvSys2 code after the Spin binary, and then adjust the DBASE, DCURR and VBASE values in the Spin header. Each of these three values would be increased by the size of the AdvSys2 code. You would also need to adjust the checksum in the header.
    I know how to adjust DBASE, DCURR, and VBASE and recompute the checksum but I'm not sure how to actually find the appended image in the Spin code. How do I know its address when I compile the Spin code? Is there a built-in symbol that points to where VBASE points in the binary image?

    I just padded the Spin binary out to a fixed size (with an external tool) so that the appended image always started at the same address. I think that's the simplest solution, although you could also look up VBASE in the first few words of memory as well.

    Eric
    Do you have any idea whether vbase is guaranteed to be on a long boundary? Will I need to pad to a long boundary if my image requires that?
  • Yes, VBASE is always on a long boundary. If you use Eric's padding method you will need to ensure that the pad is large enough to handle your stack needs. The stack will grow upward toward where your AdvSys2 code is residing, so you need to make sure the stack won't overwrite your code.

    If you use the other method of adjusting the Spin state variables in the header you could store a pointer to the AdvSys2 code in a variable at the beginning of the DAT space in the top object, as you suggested. This will be located just after the method table, and it is fairly easy to compute where that is located. I used that method to store system configuration variables in the boot program for spinix.
  • Dave Hein wrote: »
    Yes, VBASE is always on a long boundary. If you use Eric's padding method you will need to ensure that the pad is large enough to handle your stack needs. The stack will grow upward toward where your AdvSys2 code is residing, so you need to make sure the stack won't overwrite your code.

    If you use the other method of adjusting the Spin state variables in the header you could store a pointer to the AdvSys2 code in a variable at the beginning of the DAT space in the top object, as you suggested. This will be located just after the method table, and it is fairly easy to compute where that is located. I used that method to store system configuration variables in the boot program for spinix.
    I wondered why Eric had used the padding method until I realized that that allows his compiler to assign physical addresses to things. My compiler only assigns addresses relative to a base address because it doesn't know where the image will live in memory during runtime. Unfortunately, that means I need to add that base address to pointers constantly.
  • David BetzDavid Betz Posts: 14,511
    edited 2018-09-26 02:05
    Thanks for the suggestions about appending an image to a Propeller .binary file. I've checked in a simple program called propbinary that takes a .dat file created by adv2com and creates a .binary file from it. Next I'll integrate it into the compiler itself as an option. That will allow you to compile an advsys2 program into a Propeller .binary file directly without needing openspin installed.
  • David BetzDavid Betz Posts: 14,511
    edited 2018-09-27 01:12
    Here is the code I wrote to combine my VM image with some Spin template code. The inputs to the function are an array containing the compiled Spin template and another array containing the VM image. The output is the resulting .binary file. This seems to work but I was wondering if anyone who knows the .binary file format better could look at it to see if I've made any wrong assumptions.
    /* spin binary file header structure */
    typedef struct {
        uint32_t    clkfreq;
        uint8_t     clkmode;
        uint8_t     chksum;
        uint16_t    pbase;
        uint16_t    vbase;
        uint16_t    dbase;
        uint16_t    pcurr;
        uint16_t    dcurr;
    } SpinHdr;
    
    /* spin object */
    typedef struct {
        uint16_t next;
        uint8_t pubcnt;
        uint8_t objcnt;
        uint16_t pubstart;
        uint16_t numlocals;
    } SpinObj;
    
    /* DAT header in serial_helper.spin */
    typedef struct {
        uint32_t baudrate;
        uint8_t rxpin;
        uint8_t txpin;
        uint16_t imagebase;
    } DatHdr;
    
    /* BuildBinary - build a .binary file from a template and a VM image */
    static uint8_t *BuildBinary(uint8_t *template, int templateSize, uint8_t *image, int imageSize, int *pBinarySize)
    {
        int binarySize, paddedImageSize;
        uint8_t *binary;
        SpinHdr *hdr;
        SpinObj *obj;
        DatHdr *dat;
        
        /* pad the image to a long boundary */
        paddedImageSize = (imageSize + sizeof(uint32_t) - 1) & ~(sizeof(uint32_t) - 1); 
        
        /* compute the size of the binary file including the image */
        binarySize = templateSize + paddedImageSize;
        
        /* allocate space for the binary file */
        if (!(binary = (uint8_t *)malloc(binarySize)))
            return NULL;
        memset(binary, 0, binarySize);
        
        /* copy the template to the start of the file */
        memcpy(binary, template, templateSize);
        
        /* update the binary file header to include the VM image */
        hdr =  (SpinHdr *)binary;
        obj =  (SpinObj *)(binary + hdr->pbase);
        dat =  (DatHdr *)((uint8_t *)obj + (obj->pubcnt + obj->objcnt) * sizeof(uint32_t));
        dat->imagebase = hdr->vbase;
        memcpy(binary + hdr->vbase, image, imageSize);
        hdr->vbase += paddedImageSize;
        hdr->dbase += paddedImageSize;
        hdr->dcurr += paddedImageSize;
        UpdateChecksum(binary, binarySize);
        
        /* return the binary */
        *pBinarySize = binarySize;
        return binary;
    }
    
    /* target checksum for a binary file */
    #define SPIN_TARGET_CHECKSUM    0x14
    
    /* UpdateChecksum - recompute the checksum */
    static void UpdateChecksum(uint8_t *binary, int size)
    {
        SpinHdr *hdr = (SpinHdr *)binary;
        int chksum = 0;
        hdr->chksum = 0;
        while (--size >= 0)
            chksum += *binary++;
        hdr->chksum = SPIN_TARGET_CHECKSUM - chksum;
    }
    
  • The code looks good to me.
  • Dave Hein wrote: »
    The code looks good to me.
    Thanks for looking it over.

  • The SPIN language has a STACK statement that allows you to set the top end of the SPIN stack, and prevent SPIN from writing over your added binary.

    Enjoy!
    Mike
  • Dave HeinDave Hein Posts: 6,347
    edited 2018-09-27 21:53
    I've never heard of the STACK statement. Where is that documented? BST doesn't recognize the STACK symbol. I'm on a new computer, and I currently don't have the Prop Tool installed, so I can't try it there. AFAIK the Spin interpreter does not check for a stack limit.

    EDIT: I'm not even sure what the Spin interpreter would do if it did support a stack limit. Silently not write to the stack? I'm pretty sure there is no STACK statement.
  • msrobotsmsrobots Posts: 3,701
    edited 2018-09-27 23:09
    Propeller Help Page 201

    _STACK Constant: Pre-defined, one-time settable constant for specifying the size of an application’s stack space.

    CON _STACK = Expression
     Expression is an integer expression that indicates the number of longs to reserve for stack space.

    Explanation _STACK is a pre-defined, one-time settable optional constant that specifies the required stack space of an application. This value is added to _FREE if specified, to determine the total amount of stack/free memory space to reserve for a Propeller application. Use _STACK if an application requires a minimum amount of stack space in order to run properly. If the resulting compiled application is too large to allow the specified stack space, an error message will be displayed. For example: CON _STACK = 3000

    The _STACK declaration in the above CON block indicates that the application needs to have at least 3,000 longs of stack space left over after compilation. If the resulting compiled application does not have that much room left over, an error message will indicate by how much it was exceeded. This is a good way to prevent successful compiles of an application that will fail to run properly due to lack of memory. Note that only the top object file can set the value of _STACK. Any child object’s _STACK declarations will be ignored. The stack space reserved by this constant is used by the application’s main cog to store temporary data such as call stacks, parameters, and intermediate expression results

    This sets the minimum needed stack size

    and there is _FREE Propeller Help Page 110

    _FREE Constant: Pre-defined, one-time settable constant for specifying the size of an application’s free space.

    CON _FREE = Expression
     Expression is an integer expression that indicates the number of longs to reserve for free space.

    Explanation _FREE is a pre-defined, one-time settable optional constant that specifies the required free memory space of an application. This value is added to _STACK, if specified, to determine the total amount of free/stack memory space to reserve for a Propeller Application. Use _FREE if an application requires a minimum amount of free memory in order to run properly. If the resulting compiled application is too large to allow the specified free memory, an error message will be displayed. For example: CON _FREE = 1000

    The _FREE declaration in the above CON block indicates that the application needs to have at least 1,000 longs of free memory left over after compilation. If the resulting compiled application does not have that much room left over, an error message will indicate by how much it was exceeded. This is a good way to prevent successful compiles of an application that will fail to run properly due to lack of memory. Note that only the top object file can set the value of _FREE. Any child object’s _FREE declarations will be ignored.

    Mike
Sign In or Register to comment.