Shop OBEX P1 Docs P2 Docs Learn Events
Parallax seeking contractor to convert Spin/ASM to C/ASM for Scribbler 3 Robot - Page 2 — Parallax Forums

Parallax seeking contractor to convert Spin/ASM to C/ASM for Scribbler 3 Robot

24

Comments

  • ersmith wrote: »
    A very interesting thread :). Thanks to @DavidZemon for reporting the spin2cpp problems he encountered. I've fixed some of those in the current master branch in Git, and scribbler.spin compiles now with spin2cpp and the default settings. Using --gas to preserve the PASM code does not work yet, nor have I actually tested that the code converted without --gas works (I don't have a robot to test with).
    I'm sure Parallax would be willing to fix that if you are interested in helping out with this! :-)

  • @ersmith,
    That's a very helpful list of "failings", thanks! Can you drop that in your readme? maybe under the "limitations" section?
  • DavidZemon wrote: »
    @ersmith,
    That's a very helpful list of "failings", thanks! Can you drop that in your readme? maybe under the "limitations" section?

    Yes, good idea -- I've done that. I've also fixed one of the issues (using switch for case instead of if/then/else in the event that all the case items are constant). Anyone planning to use spin2cpp for this project should definitely check out the master branch from git and use that.

    Eric
  • jmgjmg Posts: 15,148
    edited 2016-09-28 01:17
    ersmith wrote: »
    A very interesting thread :). Thanks to @DavidZemon for reporting the spin2cpp problems he encountered. I've fixed some of those in the current master branch in Git, and scribbler.spin compiles now with spin2cpp and the default settings. Using --gas to preserve the PASM code does not work yet, nor have I actually tested that the code converted without --gas works (I don't have a robot to test with).

    Another approach here, would be for Ken to fun*fund improving spin2cpp , (& associated pathways) so it can convert the sources ?

    Seems to be quite close, and a better Spin translator is a global resource, valid across many projects.
    ersmith wrote: »
    That said, spin2cpp could only be a starting point. Its output isn't the high quality C code that Parallax and its customers would expect. The results of spin2cpp is designed mainly to be correct, and only secondarily to be idiomatic C.

    That depends on how well the translation works.

    If there is minimal, or no editing needed, then code presentation matters much less.
    The master code repository is the Spin source and the C code is another listing file.

    It sounds quite near to no-editing-needed translation ?

    * oops, but it does sound like fun too !
  • jmg wrote: »
    Another approach here, would be for Ken to fun improving spin2cpp , (& associated pathways) so it can convert the sources ?

    Seems to be quite close, and a better Spin translator is a global resource, valid across many projects.
    That sounds like a great idea!

  • I've run the Spin code through spin2cpp and was able to build an executable using SimpleIDE. I did have to make a few changes to work around the problem that David Zemon found. I re-ordered a few lines in the CON section to get around one problem, and renamed some local variables to get around the other problem. However, when I first tried to build it the binary was greater than 32K, so the build failed. I reduced the size of the tone queue from 1200 words to 800 words to get it to build. I have attached a zip file containing the SimpleIDE project. I haven't tested it since I don't have the hardware either.

    I didn't try using the --gas option, so the converted files contain a dat array that contains all of the DAT variables and assembled PASM code. I think a better approach would be to move the PASM code to a separate .s file, and retain the rest of the DAT variables in the C source file with their original names.

    Enhancing spin2cpp to produce clean C files sounds like a good idea, but I think it would be a much bigger effort than just cleaning the C files up by hand. There is some advantage to using the Spin code as the master source, and generating the C code whenever changes are made to the Spin code. The alternative is to maintain separate Spin and C files, and ensure that when changes are made they are made to both code bases. I think enhancing spin2cpp would require Ken to spend more money on the project, and it would probably take significantly longer.

  • Cluso99Cluso99 Posts: 18,069
    I sent a reply to Ken over the w/end here in Oz.

    Here is another proposal...

    1. A small group of us do this project for a small contra fee (1x S3 of course since we will require that for testing purposes) and perhaps some other Parallax goodies/chips/boards.

    2. We have varied experience in different areas.

    For instance,

    I am proficient at
    * PASM
    * SPIN (to a lesser degree perhaps than others)
    * Code documentation
    * Hardware design (in case we have issues that could be with hardware misunderstanding)
    I am not so proficient at
    * C - I have done a small amount and a light understanding.
    * C++ - Nothing/nada
    * spin2cpp - never used, about to learn???
    * SimpleIDE - tried a long time ago,so basically nothing.
    * GCC for Propeller - nothing/nada, but is this a requirement???

    I have plenty of available time for the last 3 weeks of October.I have a few things I am finishing up now. Getting the SD boot code for P2 and discussing the booting in general.
    First week in October is not that full either, so could get started pretty quickly.

    I have looked at the spin code (from Ken's zip) and there is nothing too daunting there. Nor is there a huge amount of code.

    What does concern me is that the spin code might blow out to be much larger than the P1 can handle. If so, then it might be necessary to place some routines within the PASM cogs. Not sure if this is possible as I didn'tcompile the code, yet anyway.

    If a few of you are in, and with Ken's agrement, here is a suggested plan...

    * Appoint a manager - I volunteer if no-one else wants the task.
    * Compile with listing the current code (Cluso using bst and/or homespun - both have listings)
    * Convert the SPIN code thru' SPIN2CPP (been done already, but perhaps Eric can fix some simple problems???)
    * Compile with SimpleIDE for errors (ALL of us need to do this)
    * Allocate code Blocks for tidying up and documenting (Each of us a block or more. I will add extra comments if reqd)
    * Testing (ALL of us need to do this, especially out parts). Note we need to get some basics running first before we can actually start testing other bits.
    * Files need to be up on github? or somewhere where we can all get to them. (ALL - You will have to tell me how to check out and in code changes)

    Anyone interested in this???

    Ken, interested or do you want a single person???
    BTW I am still happy to proceed on my own.




  • ersmithersmith Posts: 5,914
    edited 2016-09-28 12:37
    Dave Hein wrote: »
    I've run the Spin code through spin2cpp and was able to build an executable using SimpleIDE. I did have to make a few changes to work around the problem that David Zemon found. I re-ordered a few lines in the CON section to get around one problem, and renamed some local variables to get around the other problem.

    I see from the headers that you used spin2cpp v1.94. The current source in git (3.2.0-beta2) has fixes for the bugs David Zemon found. It's also smart enough to remove the "self" parameter from C functions that don't need it, and has a few other minor improvements to the C output. If you use it to drive the compilation with:
    spin2cpp --binary --ccode -Os -mcmm scribbler.spin
    
    it also removes unused functions from the output, which allows the original Scribbler to fit (the command line above produces a 12820 byte binary). (EDIT: actually scribbler_default.spin is the top level file with the test harness, and that does not fit :( ).

    Moving the PASM into separate .s files is probably a good approach. spin2cpp can help with this:
    spin2cpp --dat --gas scribbler.spin
    
    produces a single .S file that's supposed to be the source code of the dat. I just looked and the current master has messed that up by adding in the asm() directive and turning the lines into strings (like the inline version of the PASM code in a C file). It isn't supposed to do that, so I'll work on fixing fix it.

    (Edited to add: found the problem with --dat --gas: it was a small typo in a function call parameter. Producing .S files should work again with the current git source. I'll compile everything up and make a new binary release Real Soon Now.)

    Regards,
    Eric
  • Cluso99 wrote: »
    I sent a reply to Ken over the w/end here in Oz.

    Here is another proposal...

    1. A small group of us do this project for a small contra fee (1x S3 of course since we will require that for testing purposes) and perhaps some other Parallax goodies/chips/boards.

    2. We have varied experience in different areas.

    For instance,

    I am proficient at
    * PASM
    * SPIN (to a lesser degree perhaps than others)
    * Code documentation
    * Hardware design (in case we have issues that could be with hardware misunderstanding)
    I am not so proficient at
    * C - I have done a small amount and a light understanding.
    * C++ - Nothing/nada
    * spin2cpp - never used, about to learn???
    * SimpleIDE - tried a long time ago,so basically nothing.
    * GCC for Propeller - nothing/nada, but is this a requirement???

    I have plenty of available time for the last 3 weeks of October.I have a few things I am finishing up now. Getting the SD boot code for P2 and discussing the booting in general.
    First week in October is not that full either, so could get started pretty quickly.

    I have looked at the spin code (from Ken's zip) and there is nothing too daunting there. Nor is there a huge amount of code.

    What does concern me is that the spin code might blow out to be much larger than the P1 can handle. If so, then it might be necessary to place some routines within the PASM cogs. Not sure if this is possible as I didn'tcompile the code, yet anyway.

    If a few of you are in, and with Ken's agrement, here is a suggested plan...

    * Appoint a manager - I volunteer if no-one else wants the task.
    * Compile with listing the current code (Cluso using bst and/or homespun - both have listings)
    * Convert the SPIN code thru' SPIN2CPP (been done already, but perhaps Eric can fix some simple problems???)
    * Compile with SimpleIDE for errors (ALL of us need to do this)
    * Allocate code Blocks for tidying up and documenting (Each of us a block or more. I will add extra comments if reqd)
    * Testing (ALL of us need to do this, especially out parts). Note we need to get some basics running first before we can actually start testing other bits.
    * Files need to be up on github? or somewhere where we can all get to them. (ALL - You will have to tell me how to check out and in code changes)

    Anyone interested in this???

    Ken, interested or do you want a single person???
    BTW I am still happy to proceed on my own.

    If Ken doesn't want to deal with multiple contractors, you could always just be a proxy and subcontract parts out. Then Ken still has one person he can say is fully responsible and only one person that he has to pay. Your subcontractors could be paid in a simpler fashion without all the paperwork that parallax would have to file.
  • Cluso99: Your plan sounds reasonable, and I'd definitely be in for fixing spin2cpp problems, but managing forum members is probably going to be a bit harder than herding cats :). It might be more straightforward if there's a single person doing the conversion. But that's Ken's choice -- I don't know what constraints Parallax have. I'd sign up for doing the work, but I have other things on the go and can't commit to a new project right now. I will help out with spin2cpp improvements and with testing as time allows.

    Eric
  • Cluso99Cluso99 Posts: 18,069
    Am I correct in assuming that we need to get "Scribbler_test.spin" working?
    This uses and tests the various sections of the S3 code, and as such that is what I would be expecting to do.

    Now, this causes a big problem!
    The code extends to $6908 where the FF FF F9 FF FF FF F9 FF final string is placed (ie to $690F).
    Then there is stack space required above this for the base cog.

    This does not leave sufficient space to be able to convert to C as essentially the SPIN is bytecode and very compressed.

    So how is this "C" code intended to fit into S3. Do we have to load various blocks from EEPROM on the fly, so to speak??? I didn't recall seeing any SRAM on the circuit that we could use quickly.

    Even with unused code removal, we still get to $6444.

    BTW the listing is ~17,000+ lines for both homespun and bst.

    Am I missing something???
  • From what I could tell in the spin code, scribbler.spin simply contains a number of reusable blocks - no main function. Scribbler_test.spin was then the test harness that invoked ask of the different driver functions. It may be all we need to do is split scribbler_test.spin into two files - I doubt that will be a serious issue for Ken
  • It looks like scribbler_default.spin is the top level file, and building that fails (region 'hub' overflowed by 3208 bytes). It looks like we may have to break the tests up into multiple files.
  • Cluso99Cluso99 Posts: 18,069
    Wouldn't a user expect to be able to do something similar to the test?

    In other words, how much code space does the user expect to be able have to be able to use the S3 properly? It's no use if all of the code space is fully utilised by the drivers, leaving none for the user to do anything useful.

    I believe we need more information as to the specific requirements.
  • Dave Hein wrote: »
    I haven't tested it since I don't have the hardware either.


    Dave, I tried the SimpleIDE Project on my S3, and it did not work. Can you try with 3.0.2. I will test that.

    Anyone with code to test, my S3 is always at my side.

  • It looks like I used the wrong object as the top object. scribbler_default.spin seems to be the correct object for the top. Unfortunately, as others have pointed out the compiled C code doesn't fit in hub memory. I tried spin2cpp 3.1.2 and I got 4,004 bytes more than 32K. The Spin binary has about 2KB left for stack space, so we would need to reclaim about 6,000 bytes to have the same stack space in C.

    According to the S3 schematic, it uses a 512Kb EEPROM, so there would be space on the EEPROM to store cog images. It looks like there's about 3,400 bytes used for the cog images. This still leaves about 2,600 bytes that would need to be reclaimed. Maybe the stack doesn't need 2,000 bytes, and a few hundred bytes might be sufficient. We would need to check the stack during run time to verify that. Hopefully, there are other areas where memory usage can be reduced.
  • Dave Hein wrote: »
    It looks like I used the wrong object as the top object. scribbler_default.spin seems to be the correct object for the top. Unfortunately, as others have pointed out the compiled C code doesn't fit in hub memory. I tried spin2cpp 3.1.2 and I got 4,004 bytes more than 32K. The Spin binary has about 2KB left for stack space, so we would need to reclaim about 6,000 bytes to have the same stack space in C.

    According to the S3 schematic, it uses a 512Kb EEPROM, so there would be space on the EEPROM to store cog images. It looks like there's about 3,400 bytes used for the cog images. This still leaves about 2,600 bytes that would need to be reclaimed. Maybe the stack doesn't need 2,000 bytes, and a few hundred bytes might be sufficient. We would need to check the stack during run time to verify that. Hopefully, there are other areas where memory usage can be reduced.
    You could try using XMM using the EEPROM external memory driver. Not sure if the performance would be adequate though.

  • David Betz wrote: »
    You could try using XMM using the EEPROM external memory driver. Not sure if the performance would be adequate though.

    Unfortunately the S3 code seems to rely on running Spin methods on other cogs, which in C terms means running C code on another cog. We can't do that in XMM using the SimpleIDE version of PropGCC, although we can with more recent versions of PropGCC.

    Eric
  • If I grep the Spin files for cognew I get:
    fds.spin:  okay := cog := cognew(@entry, @rx_head) + 1
    FullDuplexSerial.spin:  okay := cog := cognew(@entry, @rx_head) + 1
    PWMx8.spin:  if (cogno := cognew(@pwm, @fcb) + 1)
    scribbler.spin:  if (Adc_cog := cognew(@adc_all, Model_number << 2) + 1)
    scribbler.spin:    result := (Motor_cog := cognew(@motor_driver, @Motor_cmd) + 1) > 0
    scribbler.spin:    result := (Tone_cog := cognew(@tone_seq, 0) + 1) > 0
    scribbler.spin:    return (Mic_cog := cognew(@env_det, @Envelope) + 1) > 0
    scribbler_test.spin:    cog_list[cog_index] := cognew(@busy, 0)
    scribbler_test.spin:    cog_list[cog_index] := cognew(@busy, 0)
    
    I also searched for coginit, but it's not used. All of the cognew's refer to PASM code. Are there other Spin files that I'm not aware of?

    At any rate it's not clear that XMM would fit in the EEPROM, and it would probably be quite a bit slower than CMM.
  • ersmith wrote: »
    David Betz wrote: »
    You could try using XMM using the EEPROM external memory driver. Not sure if the performance would be adequate though.

    Unfortunately the S3 code seems to rely on running Spin methods on other cogs, which in C terms means running C code on another cog. We can't do that in XMM using the SimpleIDE version of PropGCC, although we can with more recent versions of PropGCC.

    Eric
    That brings up an interesting question. How do you handle stack sizes when translating Spin to C? When you call cognew in Spin you pass in a stack but the size of that stack is probably different in the C translation. It will certainly be bigger if you use XMM since the cache is taken out of the stack space.

  • Dave Hein wrote: »
    I also searched for coginit, but it's not used. All of the cognew's refer to PASM code. Are there other Spin files that I'm not aware of?
    Huh, you're right. I'm sorry, I had run across the cognew problem with SimpleIDE recently, and I was sure that it was in the Scribbler source code. But all the cognews do indeed seem to be from DAT, so they shouldn't be a problem. So XMM may be an option here. Thanks for correcting me!

    Eric
  • David Betz wrote: »
    That brings up an interesting question. How do you handle stack sizes when translating Spin to C? When you call cognew in Spin you pass in a stack but the size of that stack is probably different in the C translation. It will certainly be bigger if you use XMM since the cache is taken out of the stack space.

    There is a problem in the Spin version of cognew/coginit, since Spin only takes the bottom of the stack and does not give a size. C needs to know the top of the stack. spin2cpp handles this automatically if the stack base given is an array, otherwise it gives up and prints an error.

    The stack for the initial COG (the one that will be taken away from in XMM) is set up by the run time in both Spin and C, so spin2cpp doesn't have to do anything for that.

    You are correct of course that the stack requirements may differ between Spin and C, so that's something else that anyone porting the code will have to watch out for.

    Eric
  • ersmith wrote: »
    David Betz wrote: »
    That brings up an interesting question. How do you handle stack sizes when translating Spin to C? When you call cognew in Spin you pass in a stack but the size of that stack is probably different in the C translation. It will certainly be bigger if you use XMM since the cache is taken out of the stack space.

    There is a problem in the Spin version of cognew/coginit, since Spin only takes the bottom of the stack and does not give a size. C needs to know the top of the stack. spin2cpp handles this automatically if the stack base given is an array, otherwise it gives up and prints an error.

    The stack for the initial COG (the one that will be taken away from in XMM) is set up by the run time in both Spin and C, so spin2cpp doesn't have to do anything for that.

    You are correct of course that the stack requirements may differ between Spin and C, so that's something else that anyone porting the code will have to watch out for.

    Eric
    If there is no Spin code launched with COGNEW or COGINIT then I guess stack size is less of a problem. However, I wonder if spin2cpp should recognize some stylized comments to indicate where values differ between code intended to run on the Spin VM and code to be translated to C. If you don't embed that information in the sources somehow then keeping the master source in Spin and auto-translating whenever the Spin code changes doesn't work as well. The resulting C code would have to be tweaked each time.

  • jmgjmg Posts: 15,148
    ersmith wrote: »
    Cluso99: Your plan sounds reasonable, and I'd definitely be in for fixing spin2cpp problems, but managing forum members is probably going to be a bit harder than herding cats :).

    hehe, yes...
    What we've found works well, is to have just a couple: one focuses on code, and one focuses on testing.
    Different time zones can even help here...
    Sounds like you and Cluso have that covered ?
    (unless it removes Cluso from getting SD boot into P2 ... )


  • Dave HeinDave Hein Posts: 6,347
    edited 2016-09-28 21:07
    I created two different projects called scribbler_rundemos and scribbler_runtests. It seems like these are two different modes for the scribbler. The demos run stand-along, and the tests require a serial connection. The demos do not require a lot of memory, and easily fit in the hub RAM. The tests overflow hub RAM by 148 bytes, so I reduced the tone queue size from 1200 to 800 to get it to fit. This only leaves about 800 bytes for the stack, which may not be enough. Since the tests are more of a diagnostic program we could probably split the tests up into 2 different programs to provide more stack space.

    The files are in the attached zip file. Publison, could you give these a try on your S3? The binary files are located in the cmm subdirectory.
  • PublisonPublison Posts: 12,366
    edited 2016-09-28 22:31
    Dave Hein wrote: »
    I created two different projects called scribbler_rundemos and scribbler_runtests. It seems like these are two different modes for the scribbler. The demos run stand-along, and the tests require a serial connection. The demos do not require a lot of memory, and easily fit in the hub RAM. The tests overflow hub RAM by 148 bytes, so I reduced the tone queue size from 1200 to 800 to get it to fit. This only leaves about 800 bytes for the stack, which may not be enough. Since the tests are more of a diagnostic program we could probably split the tests up into 2 different programs to provide more stack space.

    The files are in the attached zip file. Publison, could you give these a try on your S3? The binary files are located in the cmm subdirectory.

    I gave both binaries a shot. Both do not run. The scribbler_runtests just echos inputs to the screen. It should bring up the test menu.

    Thanks for trying!



  • I also tried building both through SimpleIDE, same results. No response to any inputs.
  • David Betz wrote: »
    ersmith wrote: »
    You are correct of course that the stack requirements may differ between Spin and C, so that's something else that anyone porting the code will have to watch out for.
    If there is no Spin code launched with COGNEW or COGINIT then I guess stack size is less of a problem. However, I wonder if spin2cpp should recognize some stylized comments to indicate where values differ between code intended to run on the Spin VM and code to be translated to C. If you don't embed that information in the sources somehow then keeping the master source in Spin and auto-translating whenever the Spin code changes doesn't work as well. The resulting C code would have to be tweaked each time.

    I'd say the simplest way to handle that would be:
    con
    #ifdef __SPIN2CPP__
      stksize = 16   ' stack needed for C version
    #else
      stksize = 20   ' stack needed for Spin bytecode
    #endif
    ...
    long mystack[stksize]
    

    The __SPIN2CPP__ symbol is always defined by spin2cpp when compiling C or C++ code (and __SPIN2PASM__ is defined when compiling assembly code). As long as none of the other Spin compilers define those symbols we should be fine :).

    Note that since C does not store variables on the stack, and does not require the stack for arithmetic, it may not need as much stack space as Spin. This will be quite code dependent.

    Eric
  • jmg wrote: »
    ersmith wrote: »
    Cluso99: Your plan sounds reasonable, and I'd definitely be in for fixing spin2cpp problems, but managing forum members is probably going to be a bit harder than herding cats :).

    hehe, yes...
    What we've found works well, is to have just a couple: one focuses on code, and one focuses on testing.
    Different time zones can even help here...
    Sounds like you and Cluso have that covered ?

    Not me, I'm afraid. I'll help out with spin2cpp bugs and problems (as time permits!), but the manual code conversion and fixing the code up will have to be done by someone else.

    Eric
  • Cluso99Cluso99 Posts: 18,069
    Dave,
    When you say the demos easily fit into hub ram, I presume you mean the compiled C version?
    How much space is available?

    Remember, some kids/adults may make rather large demos once they get used to the S3. There does need to be lots of expansion room for the user.
Sign In or Register to comment.