Shop OBEX P1 Docs P2 Docs Learn Events
Catalina - ANSI C for the Propeller 1 & 2 - Page 2 — Parallax Forums

Catalina - ANSI C for the Propeller 1 & 2

2456711

Comments

  • RossHRossH Posts: 5,336

    A linux release of Catalina 5.0 has been added to SourceForge - see https://sourceforge.net/projects/catalina-c/files/releases/5.0/

  • RossHRossH Posts: 5,336

    I could have predicted this would happen as soon as I finished the release. It always seems to! :)

    I wondered why one of my Lua example programs (ex9.lua) seemed to need much more stack space than the other examples to execute reliably, even though it didn't really do much more. I never could figure out why, so I also used it as an example of how Lua's garbage collector can help when memory gets tight. But it turns out this example does not need the larger stack at all - it was apparently the result of an obscure bug in Lua itself, which I seem to have now fixed quite by chance while working on something else.

    There are no functional changes to either Lua or the new multi-threading module, and in any case I will probably leave the example intact (except for the stack size) as an example of managing Lua's garbage collector, but I'll wait to see if anything else crops up before I re-package release 5.0 (or release 5.1 if I find anything more serious).

    Ross.

  • RossHRossH Posts: 5,336

    Catalina 5.0.1 has been released here.

    This release adds no significant new functionality, but it fixes one major Lua issue and a couple of other minor issues.

    Here is the relevant part of the README.TXT:

    RELEASE 5.0.1
    
    New Functionality
    -----------------
    
    1. Modified various Catalyst commands to allow combined command-line 
       options - e.g. -dh now means the same as -d -h
    
    
    Other Changes
    -------------
    
    1. Fixed a bug in the DOSFS file system which allowed a file to be opened
       as a directory using the DFS_OPENDIR option to DFS_OpenFile instead of
       returning that a directory of that name did not exist. This confused
       the Catalyst cp command when copying files into a directory.
    
    2. Fixed a bug in the Catalyst rm command that prevented recursive file
       deletions.
    
    3. Fixed a bug in the Lua threads module that prevented the number of 
       factories being reduced. Also, the Lua threads module documentation
       has been updated to reflect the fact that reducing the number of 
       factories now necessarily terminates all workers first (they are 
       restarted after the number of factories has been reduced) and that 
       this should therefore only be done from the main thread and when that 
       thread is executing on factory 1.
    
    4. Modified Lua to zero all newly allocated blocks of memory. Lua seems to
       assume all new blocks of memory will be initialized to zero, which is 
       not actually guaranteed by C, so this led to some Lua programs behaving
       unexpectedly. While this fix resolves the issue, it does so at a small 
       run-time cost, so further investigation is required. Multi-threaded Lua
       example 9 was badly affected by this issue and has been updated. The 
       Lua threads module documentation has also been updated.
    

    I have also updated the version of Multi-threaded Lua pre-compiled for the P2 Evaluation board which is attached to an earlier post.

    Ross.

  • RossHRossH Posts: 5,336

    As usual, I wish I had thought to do this before the release! :/

    Now that I have fixed Lua's memory issue, I thought I'd try compiling Multi-threaded Lua in NATIVE mode rather than COMPACT mode. Doing so means there is less memory available for Lua programs, but the programs will execute much faster. The command I used was:

    build_all P2_EVAL SIMPLE VT100 OPTIMIZE

    Here are the results for the 10 tutorial examples:

    ex1.lua, ex2.lua, ex3.lua, ex5.lua, ex7.lua, ex10.lua : All good. Can be executed from source (using mlua), or compiled (using lauc and mluax).

    ex4.lua : The increased execution speed highlights that the original program has a race condition. Adding a small delay to the ping and pong functions fixes the problem:

       function ping()
          t = threads
          for i = 1, t.shared("count") do
            ball = t.get("ping")
            t.output("ping ")
            t.msleep(10)             -- DELAY ADDED
            t.aput("pong", ball)
          end
       end
    
       function pong()
          t = threads
          for i = 1, t.shared("count") do
            ball = t.get("pong")
            t.output("pong\n")
            t.msleep(10)             -- DELAY ADDED
            t.aput("ping", ball)
          end
       end
    

    ex6.lua : Must now be compiled. Otherwise, all good.

    ex8.lua : There is only enough memory available to execute 6 workers concurrently (not 12) and the increased execution speed is such that a small delay needs to be added to the Thread function to illustrate that all the workers are indeed executing in parallel:

       count = 6
    
       function Thread(me, iter)
          return function()
            t = threads
            -- wait till we are told to go
            repeat until t.shared("go")
            repeat
              t.output(me .. " ")
              t.msleep(10)           -- DELAY ADDED
              iter = iter-1
            until iter == 0
          end
       end
    

    ex9.lua : There is only enough memory available to recycle up to 2 workers, not all 4:

       t.recycle(2)                  -- WAS 4
    

    Other than example 9, these programs are too small and simple to show much improvement in execution speed ... except (in strict accordance with Murphey's Law) where it highlights a problem in my example programs! :(

    Ross.

  • RossHRossH Posts: 5,336
    edited 2022-04-06 06:53

    There is an interesting synergy between Lua co-routines and threads. I will add this to the threads documentation in the next release, but the example program below can be executed with the current release, so I thought I would post a preview here because it shows what a "rich" language Lua is ...

    Even without the threads module, Lua has a simple type of multi-tasking. It has non-preemptive multi-tasking in the form of co-routines. The threads module adds preemptive multi-tasking. However, this has interesting implications when co-routines are executed using the threads module.

    Here is a simple example:

    -- Generate a function which can be executed 
    -- as either a coroutine or as a thread
    function Generate(me)
      return function ()
        for i = 1, 10 do
          threads.print(me ..":" .. i)
          threads.msleep(100)
          coroutine.yield()
        end
      end
    end
    
    -- Execute the functions as coroutines
    threads.print("\nFUNCTIONS AS COROUTINES")
    
    -- create 4 functions as coroutines and also
    -- wrap the coroutines inside another function
    fee = coroutine.wrap(Generate("fee"))
    fi  = coroutine.wrap(Generate("fi"))
    fo  = coroutine.wrap(Generate("fo"))
    fum = coroutine.wrap(Generate("fum"))
    
    -- execute the coroutines 
    fee()
    fi()
    fo()
    fum()
    -- and again
    fee()
    fi()
    fo()
    fum()
    
    -- Execute the functions as threads
    threads.print("\nFUNCTIONS AS THREADS")
    
    -- we only need one worker to execute any number of 
    -- threads if those threads explicitly "yield" 
    threads.workers(1)
    
    -- create (and start) 4 functions as threads
    threads.new(Generate("fee"))
    threads.new(Generate("fi"))
    threads.new(Generate("fo"))
    threads.new(Generate("fum"))
    
    -- wait for all threads to complete
    threads.wait()
    

    Here is the output this program produces:

    FUNCTIONS AS COROUTINES
    fee:1
    fi:1
    fo:1
    fum:1
    fee:2
    fi:2
    fo:2
    fum:2
    
    FUNCTIONS AS THREADS
    fee:1
    fee:2
    fee:3
    fi:1
    fee:4
    fi:2
    fo:1
    fee:5
    fi:3
    fum:1
    fo:2
    fee:6
    fi:4
    fum:2
    fo:3
    fee:7
    fi:5
    fum:3
    fo:4
    fee:8
    fi:6
    fum:4
    fo:5
    fee:9
    fi:7
    fum:5
    fo:6
    fee:10
    fi:8
    fum:6
    fo:7
    fi:9
    fum:7
    fo:8
    fi:10
    fum:8
    fo:9
    fum:9
    fo:10
    fum:10
    
    

    When the functions are executed as co-routines, then each time the function is called, it executes one iteration and then "yields". This "yield" call is the key to co-routine behavior. It returns control to the caller, but "suspends" the co-routine rather than exiting it. When the function is called again, it does not start from the beginning as a normal function would, it instead continues execution from where it was last suspended - i.e. just after the last "yield". This is normal co-routine behavior (for more details on co-routines, see the Lua documentation).

    The interesting thing is the consequence of calling "yield" when the functions are executed as threads instead of co-routines. When a thread yields then the thread is suspended, but the worker executing that thread is not - it moves on to resume execution of the next suspended thread. If there is no other suspended thread it will resume execution of the same thread. Since we started all our functions as threads, this means that one worker (which is one Posix thread) will execute all the functions concurrently (using non preemptive multitasking) provided those functions are written as co-routines. If the functions are not written as co-routines, then you need multiple workers available to execute them concurrently (using preemptive multitasking). Which of course is very easily done in this example - just change the number of workers from 1 to 4. You can try it, but in this simple example it makes little or no obvious difference to the output.

    In summary, with 1 worker the program uses non preemptive multitasking. With 4 workers it uses fully preemptive multitasking. With 2 or 3 workers, it would use a mixture of both.

    Isn't Lua an interesting language? :)

    Ross.

  • RossHRossH Posts: 5,336

    Catalina 5.0.2 has been released here.

    This release is a patch release for both Windows and Linux. It must be installed over Catalina 5.0.1. It adds no new functionality, but it fixes one Lua issue and adds a couple of new Lua examples. If you are not interested in Lua you don't need it - the changes will also be included in the next full release.

    Here is the relevant part of the README.TXT:

    RELEASE 5.0.2
    
    New Functionality
    -----------------
    
       This release contains no new functionality.
    
    Other Changes
    -------------
    
    1. Fixed a bug in the Lua threads module when sending and receiving messages. 
       The program could lock up under certain circumstances. Since the Lua
       threads module is only supported on the Propeller 2, this affected the
       Propeller 2 only.
    
    2. Multi-threaded Lua examples 4 has been updated to eliminate a race 
       condition.
    
    3. Multi-threaded Lua example 8 has been updated to make it more evident the
       example is using preemptive multi-tasking.
    
    4. Multi-threaded Lua examples 11 and 12 have been added to demonstrate
       interactions between co-routines and threads.
    
    5. The document Lua on the Propeller 2 with Catalina has been updated to
       reflect the above changes. Also, some terminology issues have been
       clarified.
    

    I have also updated the version of Lua pre-compiled for the P2 Evaluation board which is attached to an earlier post.

    Ross.

  • RossHRossH Posts: 5,336

    Interesting. Now that Lua 5.1 seems rock solid (yes, I realize I'm tempting fate here :smile: ) I have been re-looking at Lua 5.4, which is the latest and greatest version.

    When I looked at it a while ago, the first thing I noticed was that the initial code/data footprint of Lua 5.4 is much larger than that of Lua 5.1 - by about 40kb even in COMPACT mode. This was enough for me to stay with 5.1 as the default version even though 5.4 seemed to have some useful improvements.

    But it turns out I didn't look deeply enough - yes, the initial footprint of Lua itself is larger, but the footprint of each concurrent thread is smaller - by about 5kb. This means that if you have 8 or more workers (as you might if you have 8 cogs!), then Lua 5.4 actually ends up using less memory than Lua 5.1.

    Only one of the tutorial examples (ex8.lua) actually creates this many workers - this particular example is specifically designed to see how many workers it can execute concurrently. It turns out that you can execute more workers with Lua 5.4 than you can with Lua 5.1 - something I had not thought to check!

    As well as better memory management, Lua 5.4 also includes integer numeric types (like some versions of Basic, Lua originally only had floats) and bitwise operators, which makes it much more suitable for the type of applications common on the Propeller.

    Catalina will continue to support all versions of Lua (from 5.1 onwards) but unless I find a showstopper, I am likely to switch the default to Lua 5.4 in the next release.

    Ross.

  • RossHRossH Posts: 5,336

    I can't believe I missed this! I just found a version of the classic startrek game ported to Lua! I already had C, Pascal and Basic versions included with Catalina, but I had never thought to even look for a Lua version. It seems I am not the only one that remembers this game very fondly!

    The sources of the original Lua port are available here: https://github.com/emabolo/superstartrek

    In the attached zip file are two Lua versions that I have modified very slightly, because the originals were written before Lua 5.4 was released, and the math library has changed slightly. They also needed a few tweaks to the garbage collector to stop them running out of memory on the Propeller. One version is a faithful port of the original 1978 game (sst.lua) and the other is an "enhanced" version (sst-tos.lua) that incorporates dialog taken from the original scripts.

    I have not done extensive testing, but I have played them both for a few minutes each and they both seem to run ok when compiled with luac and executed using mluax.

    For instance, in Catalyst, you would use commands like:

    luac -o sst.out sst.lua
    mluax  sst.out
    
    

    or

    luac -o sst-tos.out sst-tos.lua
    mluax  sst-tos.out
    
    

    Enjoy! :)

  • RossHRossH Posts: 5,336
    edited 2022-06-06 13:28

    I have been testing Catalina on the new P2 Edge module, and can confirm that Catalina works ok on it.

    However, while testing various things I found a bug in Catalina's SD Card plugin that may have prevented programs from being able to write to some types of SD Cards (reading was ok). I will include the updated version of the plugin with the next release of Catalina, but I have also attached it here - it can simply be copied over the current version in the target_p2 directory. Of course, you will have to recompile any programs that use it.

    Note that this bug affects the Propeller 2 only.

  • RossHRossH Posts: 5,336

    I have just posted Catalina 5.1 to SourceForge here.

    I am not getting much time for Catalina at the moment so there are not many changes, but the new SD Card plugin is included so I thought I had better do a release.

    Here is the relevant extract from the README.TXT:

    RELEASE 5.1
    
    New Functionality
    -----------------
    
    1. Decoding the CATALINA_DEFINE environment variable was taking place AFTER 
       the symbols to specify the clock frequency were defined, so the Catalina 
       symbols MHZ_220, MHZ_260 & MHZ_300 only worked when specified on the 
       command line, and had no effect if specified using CATALINA_DEFINE.
    
    2. Added MHZ_200 as a Catalina symbol, to make it easier to build Catalyst 
       using 200Mhz on the Propeller 2. Note that the Propeller 2 reference 
       manual, the tutorial Getting Started with Catalina and the documentation
       in the Propeller 2 VGA plugin source file all mistakenly referred to 
       MHZ_200 when they should have referred to the existing MHZ_220 symbol. 
       All these documents have now been updated.
    
    3. The Lua "propeller" module has had the functions sleep, msleep and sbrk
       added. These functions are the same as the ones in the "threads" module,
       but are handy when the propeller module is used in a non-threaded Lua
       program.
    
    4. The Lua "threads" and "propeller" modules now accepts a string parameter. 
       If "lua" is specified, these functions return the version of Lua. 
       Otherwise they return the version of the module
    
    5. Lua versions of the classic Star Trek game (sst.lua and sst-tos.lua) 
       are now included as Lua test programs.
    
    6. A change was made to the gettoken procedure in Dumbo Basic 1.0, which was 
       introduced in Catalina release 4.7. This change led to a syntax error when 
       executing TREK15.BAS. This change has been reverted in Dumbo Basic 1.1, but
       if this breaks any existing basic code, it can be re-implemented by 
       modifying the file basic.c and changing the #define of NEW_GETTOKEN to 1.
    
    7. When compiled for the Propeller, Dumbo Basic now has a delay on exit of 
       100ms to allow time for any pending output to be printed. This prevents 
       error messages or final program output from being truncated.
    
    
    Other Changes
    -------------
    
    1. The command-line options to set the clock frequency (-f, -F and -E) were
       fully implemented, but not described in the documentation. These options 
       apply to the Propeller 2 only.
    
    2. On the Propeller 2, it is now recommended that Catalyst be compiled using
       a 200Mhz clock speed. This is a work-around for an obscure bug which only
       seems to affect some programs loaded from the SD card and which use a clock
       speed of 180Mhz.
    
    3. A bug in the SD Card plugin was preventing programs being able to write to 
       some SD cards (reading was ok). This affected the Propeller 2 only.
    
    
  • RossHRossH Posts: 5,336
    edited 2022-08-17 12:35

    I have just posted Catalina 5.1.1 to SourceForge here.

    This is a very small patch release for the Propeller 1 only, mostly to fix a silly bug that was introduced in Catalina 4.9.3.

    This patch release must be installed over an existing installation of Catalina 5.1. Here is an extract of the README.TXT file:

    RELEASE 5.1.1
    
    New Functionality
    -----------------
    
    1. Added a new demo folder ("xeprom") containing programs that demonstrate how
       to read from the EEPROM from XEPROM XMM programs. In such programs, the
       EEPROM cannot be read independently (e.g. using an I2C driver) because it 
       is constantly in use by the cache. This applies only to the Propeller 1, 
       and requires a platform with an EEPROM larger than 32k, such as a FLIP, a 
       C3 or a HYDRA. 
    
    Other Changes
    -------------
    
    1. Fixed a bug that meant the -e command line switch to Catalina, which is
       used to specify that a ".eeprom" file be generated instead of a ".binary"
       file did not work correctly in some instances. In particular, it could not
       be used in conjunction with the -C XEPROM option. This affected the 
       Propeller 1 only, and only Catalina version 4.9.3 or later.
    
    2. The catalina_clean utility script had not been updated recently, and
       would miss cleaning some directories when the optional "all" parameter
       was specified.
    
  • RossHRossH Posts: 5,336

    I have just posted Catalina 5.1.2 to SourceForge here.

    This is another small patch release, mostly to fix another silly Propeller 1 bug that was introduced in Catalina 5.0, but it also incorporates a few changes to Lua that affect the Propeller 2.

    This patch release can be installed over an existing installation of Catalina 5.1 or Catalina 5.1.1. Here is an extract of the README.TXT file:

    RELEASE 5.1.2
    
    New Functionality
    -----------------
    
    1. The path that Lua uses to search for modules has changed. The old path
       was the one used on Windows, which was not appropriate for the Propeller. 
       The path is now:
    
          "?.lua;?/init.lua;lua/?.lua;lua/?/init.lua"
    
       This means that (for example) if you say "require 'm'" then Lua will 
       search for "m.lua", "m/init.lua", "lua/m.lua" and "lua/m/init.lua"
       in that order.
    
    2. The Lua demos sst.lua and sst-tos.lua now use "propeller.sbrk()" instead of
       "threads.sbrk()", which means the mem command (which displays the top of
       the C heap) now works when the program is run under both lua and mlua.
    
    Other Changes
    -------------
    
    1. Fixed a bug in the threads library, which prevented some multithreaded
       programs from compiling. Affected only Propeller 1 programs which were
       compiled in LMM TINY mode.
    
    2. Eliminated the file minit.c from Lua. This was a temporary replacement 
       for the Lua file linit.c which included the threads module. Now linit.c 
       will include the "threads" module if LUA_THREADS is defined in the file
       luaconf.h. For completeness, there is also a corresponding LUA_PROPELLER 
       that is used to specify that the "propeller" module be included. This is 
       more in keeping with the way Lua normally defines compile-time options, 
       eliminates the need to have the extra file, and it also means that when 
       compiling the executables (e.g. lua and mlua or luax and mluax) under 
       Catalina, Catalina can automatically detect the compile options and 
       include the correct modules. But if you compile mlua or mluax using gcc 
       and posix threads, you will now need to explicitly define LUA_THREADS in 
       luaconf.h to include the "threads" module.
    
    3. Fixed a bug that meant the Lua "propeller" module would not compile using
       any compiler other than Catalina. Although it is only functional when
       compiled using Catalina, it should still have compiled using other
       compilers (such as gcc).
    
    4. The number of threads the "test_maximum_threads.c" demo program creates 
       has been reduced to 145, since at 150 there are some plugin combinations
       that do not have enough free Hub RAM to support 150 threads. Affected 
       the Propeller 1 only.
    
    5. The build_all scripts and Makefiles now always build Lua in COMPACT mode
       on the Propeller 2, because NATIVE mode does not leave enough Hub RAM 
       for the various demo programs. 
    
    

    I have to apologize for the recent patch releases. I have been working almost exclusively on P2 and Lua support recently, and I had not bothered to run the validation suite before the releases - if I had done so the validation suite would have picked the problems up straight away. I will do so before future releases :(

  • RossHRossH Posts: 5,336

    I have just posted Catalina 5.2 to SourceForge here.

    This is a full release, primarily to update Catalyst to add some outstanding features that were always intended to be there, but I had never bothered to finish - like wildcard support for various Catalyst commands (i.e. ls, cp mv, rm, cat).

    I have become a real fan of Lua, and I tend to mainly use my P2 as a stand-alone Lua development platform these days. These additions make developing Lua software on the P2 for the P2 very much easier ... as long as you are also a fan of the vi text editor. Let the editor flame wars commence! :)

    Here is an extract of the README.TXT file:

    RELEASE 5.2
    
    New Functionality
    -----------------
    
    1. Added a new inline pasm demo ("test_inline_pasm_5.c") to demonstrate how 
       to CALL an inline PASM function from within an inline PASM function in a 
       COMPACT program.
    
    2. Updated the validation suite. Now logs more details to make it easier to
       identify what failed, and also added "short" scripts to do faster 
       validation.
    
    3. Updated the file 'globbing' demo (in folder "demos\globbing"). Tidied it
       up, fixed a few bugs and it now uses the correct DOS filename syntax, 
       prints the filenames properly, and is now correctly case insensitive in
       all cases. The listDir() function has been re-written to be an instance 
       of a new generic doDir() function, which calls a specified function on 
       each matching filename.
    
    4. Wildcard matching ('globbing') has been added to many of the Catalyst
       utilities. You can now specify wildcard expressions like *.bin or 
       [a-g]*.b?? - see "glob.c" for details. The options and syntax of the 
       utilities have not changed, except for 'ls' (see 5 below). 
       The commands affected are:
    
          ls - list files and/or directories
          rm - delete files or directories
          cp - copy files
          mv - move files
          cat - concatenate and print files
    
       For example, you can now say things like:
    
          ls *.bin *.dat
          mv [a-f]*.bin bin
          rm ???.dat
          cat *.txt
    
       Note that wildcards can only be used in the file name part of a path, not 
       in the directory part, so you cannot specify an argument like "/b??/*.*". 
    
       For arguments that may be interpreted as files or directories, adding a 
       trailing "/" ensures they will be treated as directories. For example:
    
          ls bin      -- list just the entry "bin" (if it exists)
          ls bin/     -- list the contents of directory "bin" (if it exists)
    
    5. The Catalyst "ls" command now uses a short listing format (similar to
       that used by Unix or the built-in "dir" command) by default. The previous 
       default listing format can be specified using the "-l" option, and the 
       previous long listing format (which used to be specified by "-l") must 
       now be specified by "-l -l" or "-ll". Also fixed interactive mode, which
       was not working in some cases. 
    
    6. The Catalyst "cp", "mv" and "rm" commands now accept 'a' or 'A' (for "all")
       at the interactive prompt, to indicate that the command should assume "yes"
       for all subsequent files.
    
    7. The catalyst "build_all" script now does "clean_all" before building, to
       ensure there are no binaries left from previous builds.
    
    Other Changes
    -------------
    
    1. Updated the file "catalina_compact.inc" in the target and target_p2 
       directories to add more notes about writing inline PASM in COMPACT 
       programs.
    
    2. The tiny library had a version of the C stdio function gets(). This 
       prevented programs that used gets() from linking with the tiny library,
       because it ended up being multiply defined. The version in the tiny
       library has been renamed tiny_gets() in this library, but like the other
       tiny functions (i.e. tiny_printf() etc) it will be used automatically if 
       a program is linked with the tiny library. Affected the Propeller 1 and 
       Propeller 2.
    
    3. Fixed a bug in the Propeller 1 SD Card plugin - multiple block writes 
       must not be used on platforms that must disable the SD Card to use XMM
       RAM (e.g. because they share pins). Affected the Propeller 1 only.
    
    4. Fixed a bug in Catalina's "unmanaged" file close function which meant that
       only a limited number of files could ever be opened. Did not affect the
       normal stdio open/close functions, but it affected the Catalyst "cat"
       command, which uses the unmanaged versions of open/close to save space.
    
    5. Catalyst's example basic files are now all Unix format, not DOS format.
       This makes no difference to their functionality, but makes them easier
       to edit usng "vi" or list using "cat".
    
    6. Fixed a bug in DOSFS which affected the ability to delete files with
       zero length (e.g. using the Catalyst "rm" command) - doing so may
       have corrupted the file system.
    
    
  • evanhevanh Posts: 15,126
    edited 2022-09-11 01:35

    Is that self-hosting?

    EDIT: Never mind, found the docs:

    While it can be used with any Propeller programs, Catalyst is specifically intended to
    facilitate loading and using programs compiled with the Catalina C compiler.
    It is not currently possible to compile Catalina C programs on the Propeller, but
    Catalyst comes with various other applications that can be used for self-hosted
    Propeller development, including the vi text editor, and such tools as a BASIC
    interpreter, a Pascal compiler and interpreter, and the Lua scripting language. It
    could also be use to edit, compile and then run SPIN programs using with the Sphinx
    SPIN compiler (not included – see http://www.sphinxcompiler.com/).

    Looking up Spinx I see it's been around since 2009. So self-hosted Spin programming has been an option for a while then. I've not seen anyone mention this before when the topic of self-hosting gets its regular airing.

  • RossHRossH Posts: 5,336

    Hi @evanh

    No, it's not self-hosting because you cannot compile the Lua compiler itself on the Propeller - you have to compile that using Catalina. There is a version of Lua written entirely in Lua, but I think it would be too large to compile on the P2, and too slow to compile on the P1 even if you had megabytes of XMM RAM available. I have not tried it, and I don't intend to do so - I've never understood why people are so obsessed with self-hosted development on a microcontroller, when you can buy a full-blown computer that will do the job better and cheaper.

    Still, Catalyst is fine for developing and executing Lua programs on the Propeller, and it's a fun way to learn Lua!

  • evanhevanh Posts: 15,126
    edited 2022-09-11 06:43

    :) I've not been a big follower myself but Chip encourages it and there have been a number of requests for self-hosting a Spin development environment. It occured me, reading you release notes, that it's probably not that hard to be achieved using Catalyst + Spinx.

  • RossHRossH Posts: 5,336

    Why is it that as soon as you do a release, you notice a stupid little problem ... Murphey's Law, I guess ...

    One of the Lua demos - fact.lua - was written when all Lua numbers were floating point. Recent versions of Lua have both floats and integers, but it is still an untyped language - so an expression like n = 1 makes n an integer, whereas n = 1.0 makes n a floating point.

    So fact.lua should now look like this, with one minor change on line 18:

    -- function closures are powerful
    
    -- traditional fixed-point operator from functional programming
    Y = function (g)
          local a = function (f) return f(f) end
          return a(function (f)
                     return g(function (x)
                                 local c=f(f)
                                 return c(x)
                               end)
                   end)
    end
    
    
    -- factorial without recursion
    F = function (f)
          return function (n)
                   if n == 0 then return 1.0      -- <<< changed 1 to 1.0 to return a float instead of an int
                   else return n*f(n-1) end
                 end
        end
    
    factorial = Y(F)   -- factorial is the fixed point of F
    
    -- now test it
    function test(x)
       io.write(x,"! = ",factorial(x),"\n")
    end
    
    for n=0,16 do
       test(n)
    end
    
    -- added for Catalina/Catalyst, which may clear the screen on program exit
    print("\nPress any key to continue")
    any = io.stdin:read() 
    
    

    I'll leave it to you to figure out how this program actually works, and how it can claim to be non-recursive.

    Lua looks a lot like any other procedural language (Pascal, Modula, C, C++ etc) - but it is very much not. It is so expressive and powerful that it can seriously do your head in. That's probably why I like it :)

  • evanhevanh Posts: 15,126

    @RossH said:
    Lua looks a lot like any other procedural language (Pascal, Modula, C, C++ etc) - but it is very much not.

    Well, without reading anything from the Web, first thing of note is the apparent function call to factorial(x) and that that is the one thing that isn't defined as a function.

  • RossHRossH Posts: 5,336
    edited 2022-09-11 22:14

    @evanh said:

    @RossH said:
    Lua looks a lot like any other procedural language (Pascal, Modula, C, C++ etc) - but it is very much not.

    Well, without reading anything from the Web, first thing of note is the apparent function call to factorial(x) and that that is the one thing that isn't defined as a function.

    Ok. here goes ... **spoiler alert! **...

    It is best to start from the bottom ...

    The line "factorial = Y(F)" tells you that factorial is what is returned from calling function Y with F as a parameter. But since Lua is an untyped language, we don't know what that is - at least not from just this line.

    The lines "F = function(f) return function (n) ..." tells you that F is a function that accepts an argument f - and we learn a few lines down that f has to be a function (remember Lua is typeless - we only know f has to be a function because F uses it as such). But note that F itself does not call f - instead F uses f to generate another function, which accepts a parameter n - and that function would calculate factorials --- provided F was passed a function f that does exactly what the generated function does (big hint there!).

    The lines "Y = function (g) ..." is where the magic happens. This tells us that Y is a function that accepts a parameter g, and it produces the "fixed point of g" - i.e. it calculates "fix g" = g(fix g) = g(g(fix g)) = ... etc ...

    So, when "fix g" is F, Y returns a function that takes one numeric argument, and that function calculates the factorial of its numeric argument.

    I warned you that Lua can do your head in! But it is a surprisingly small, clean and highly orthogonal language that can be used to do procedural programming, object-oriented programming, or (as in this case) functional programming.

  • RossH,

    I have seen 'uncle' Murphy pay a visit too many times when either there was an insane schedule to meet or there was a piece of equipment on the verge of death.

    Another thing that I have found is that it's best to wait a day or two before doing your final checks, because when your mind is fresh mistakes and/or problems are easier to spot.

  • RossHRossH Posts: 5,336

    @Genetix said:
    RossH,

    Another thing that I have found is that it's best to wait a day or two before doing your final checks, because when your mind is fresh mistakes and/or problems are easier to spot.

    Good advice. However, I generally only have limited time to devote to this at the moment, so I have to get it out there while I can! :(

  • @RossH said:

    Good advice. However, I generally only have limited time to devote to this at the moment, so I have to get it out there while I can! :(

    I myself prefer to take my time and focus on quality, because I have found that if I just try to get it done, then sooner or later I will have to fix it.

  • RossHRossH Posts: 5,336
    edited 2022-09-23 06:23

    Attached is a preview of what will be coming in the next release of Catalina. If I had realized how easy this was going to be, I would have held up the last release for it, but no matter - I am sure I will find a few more annoying bugs that also need fixing.

    I have decided to make Lua the scripting language for Catalyst itself (an obvious thing to do really, but I had just not bothered, because I had not realized both how easy and how useful it would be until I had learned more about Lua).

    You could of course always invoke Lua manually, and you still can - but now Catalyst will look for Lua programs to execute the same way it executes any normal Propeller binaries, so it is much easier to extend Catalyst's basic functionality using Lua and then invoke these programs seamlessly from the command line as if they plain old Propeller binaries.

    The attached file includes a working prototype of the new versions of both Catalyst and Lua, compiled for the P2 EDGE or P2 EVAL boards (you need an SD Card and a VT100 terminal emulator such as putty or payload). A version of payload is included in the attachment. If you use putty you may need to tweak its line termination options.

    It includes three demos designed to highlight the new Lua functionality, and illustrate how easily Lua now integrates with Catalyst:

    • list.lua - a simple directory listing program (similar to Unix "ls").
    • find.lua - a simple file searching program (similar to Unix "find").
    • freq.lua - a simple word frequency counting program.

    Load the programs onto an SD Card, set up your P2 to boot from it, and then use the following payload command (where X is the USB port connected to your P2):

       payload -i -b230400 -pX
    
    

    If you reboot your P2, you should see the Catalyst prompt:

       Catalyst 5.2.1
       >
    
    

    As well as all the usual Catalyst commands, you can try the following new commands:

       list                       <-- list file details of all files
       list *.lua                 <-- list file details of Lua files 
       find *.bas PRINT           <-- find PRINT statements in BASIC files
       freq *.txt                 <-- count word frequency in all text files
    
    

    To see how easy stuff like this is to do, here is a complete (but quite mimimalist) version of a directory listing program, implemented in Lua:

    -- print the file name only
    function print_entry(name, attr, size)
       print("   " .. name)
    end
    
    -- mount the DOSFS volume
    propeller.mount()
    
    -- scan the DOSFS directory specified in arg[1] and call print_entry 
    -- on any files matching the wildcard pattern specified in arg[2]
    propeller.scan(print_entry, arg[1], arg[2])
    
    

    The real version (in list.lua) has a few more bells and whistles, but is basically the same program.

    You may notice that this program uses two new functions that have been added to the Lua propeller module:

       mount - mount a DOSFS file system
    
       scan - scan the mounted DOSFS file system and execute a Lua function on each matching file
    
    

    This is using essentially the same wildcard 'globbing' functionality now used in the new Catalyst utilities (ls, mv, cp, rm etc), and it makes it especially easy to write new file processing utilities in Lua.

    So, is it worth rewriting all the existing Catalyst utilities (e.g. ls) as Lua scripts?

    Well, in short "no". The C versions are faster and can run on any Propeller - 1 or 2, whereas the Lua versions could only run on a Propeller 2, or on a Propeller 1 with sufficient XMM RAM. And while loading Lua each time you need it does work ok on a Propeller 1, on some platforms - especially those that use FLASH memory as XMM RAM - it is painfully slow!

    Lua is fun! :)

    EDIT: Updated freq.lua in the attached zip file (to add garbage collection).

    EDIT: I will leave this zip file here, because it is a stand-alone demo - but a slightly more advanced version of the same functionality has now been released as part of Catalina 5.3, so it is generally recommended to download that instead.

  • RossHRossH Posts: 5,336

    I have just posted Catalina 5.3 to SourceForge here.

    This is a full release, primarily to update Catalyst to add Lua scripting capabilities.

    Lua can now be used as a simple method of extending Catalyst functionality, as well as being a powerful high-level language for developing programs that can exploit the Propeller's unique capabilities.

    Lua provides the kind of capabilities in a high-level language that I wanted the Propeller to have from day 1, but which were only previously available from low-level languages like Spin, Forth, C and C++.

    Here is an extract of the README.TXT file:

    RELEASE 5.3
    
    New Functionality
    -----------------
    
    1. The Catalyst utilities cp, mv, rm, ls and cat now ignore volume id entries,
       except that the ls utility will still include them when long or very long
       listing output is specified.
    
    2. Catalyst now has the ability to execute a once-only command on startup. 
       This is similar to the existing AUTOEXEC.TXT file processing when the
       AUTODELETE option was also enabled. The AUTOEXEC.TXT and AUTODELETE 
       functionality has been retained, but AUTODELETE is not enabled by default, 
       which means that normally the AUTOEXEC.TXT is executed on every Propeller 
       reboot. The new once-only execution functionality overrides this if the
       file EXECONCE.TXT exists. If it does then it will be executed and also 
       deleted. This new functionality is used by the Lua propeller module 
       "execute" function, described in point 3 below. It requires that the
       SD Card be writable.
    
    3. The Lua propeller module now includes new propeller-specific functions.
    
       Lua analogues of the C DOSFS wildcard/globbing functions:
          mount()
          scan(function, directory, filename)
    
       The filename can include wildcards. See the Catalyst Reference Manual
       for details, and see list.lua for an example of using these functions.
    
       A Lua function to execute any Catalyst command (reboots the Propeller):
          execute(command, filename)
    
       For example:
          propeller.execute("list *.bas")
    
       The first parameter is the command (which may include parameters), and the 
       second (optional) parameter is the file name to write this command to. 
       This defaults to "EXECONCE.TXT", which means the command will only be 
       executed once, but it can be used to write the command to any file. For 
       instance:
          propeller.execute("lua", "AUTOEXEC.TXT")
       will cause the Propeller to execute "lua" on each reboot. To disable this
       from within Lua, just delete the file by executing a Lua command like:
          os.remove("AUTOEXEC.TXT")
    
       Lua analogues of the keyboard HMI functions:
          k_get, k_wait, k_new, k_ready, k_clear
    
       Lua analogues of the screen HMI functions:
          t_geometry, t_mode, t_setpos, t_getpos, t_scroll, 
          t_color, t_color_fg, t_color_bg
    
       Lua analogues of the mouse HMI functions:
          m_button, m_abs_x, m_abs_y, m_delta_x, m_delta_y, m_reset, 
          m_bound_limits, m_bound_scales, m_bound_x, m_bound_y 
    
       Each of these HMI functions accepts the same parameters and returns the 
       same values as their C counterparts. See the Catalina Reference Manual 
       for details.
    
       Note that to save space, the mouse functions will not be included if the 
       Catalina symbol NO_MOUSE is defined when Lua is compiled (which it will be
       if you use the "build_all" scripts to build Lua - to change this you can
       edit the file Makefile.Catalina to remove the "-C NO_MOUSE" option).
       You can detect whether mouse functions have been included from within 
       Lua itself by testing if any of them are nil, such as:
    
          if (propeller.m_button) then
             -- mouse functions have been included
          else
             -- no mouse functions
          end
    
    4. Catalyst will now try to execute commands as Lua scripts, in addition to
       just executing binary files. The order of priority for Catalyst command 
       execution is now:
    
       1. As a built-in command (e.g. "dir")
       2. As a Lua script, adding a ".lua" extension if none is specified
       3. As a binary file, using the command as the file name
       4. As a binary file, adding the following extensions (in this order
          if the command does not specify an extension):
            .BIN
            .BIX (propeller 2 only)
            .XMM
            .SMM
            .LMM
    
       Note that all command types accept command line arguments, including
       Lua scripts. Several example Lua scripts are included:
    
         list.lua - a simple directory listing program (similar to Unix "ls").
         find.lua - a simple file searching program (similar to Unix "find").
         freq.lua - a simple word frequency counting program.
    
       You invoke Lua scripts from the Catalyst command line just like any other 
       command. For example:
    
          list                       <-- list file details of all files
          list *.lua                 <-- list file details of Lua files 
          find *.bas PRINT           <-- find PRINT statements in all BASIC files
          freq *.txt                 <-- count word frequency in all text files
    
       Note that Lua scripts can be compiled to improve load and execution 
       times, but they should still have the extension ".lua".
    
       As a consequence of this, the Lua versions of the Super Star Trek demo
       programs have changed name, so that executing the command "sst" at the 
       Catalyst prompt will still invoke the C version ("sst.bin") rather than
       now executing the Lua version (which used to be called "sst.lua"):
    
          sst.lua      --> now called star.lua
          sst-tos.lua  --> now called star-tos.lua
    
    5. The Propeller 2 Catalina command line argument processing has been modified
       to match that of the Propeller 1. On startup, all Catalina programs check 
       if CogStore is running. If it is, the program fetches any arguments stored
       in it. Previously on the Propeller 2, if CogStore was not running then 
       Catalina would set argc to zero and argv[0] to NULL - but some C programs
       expect argc to always be at least 1, so now if CogStore is not running 
       argc will be set to 1 and argv[0] will point to a string with the value 
       "null" (since without CogStore the real program name is not available). 
       This functionality was already implemented on the Propeller 1.
    
    6. The wildcard/globbing doDir() function in storage.h and storage.c has been 
       extended to make it more useful - it now calls the file processing 
       function with the file size and the DOSFS file attributes of each file 
       in addition to the file name. One advantage of this is that programs
       that use doDir() do not need to use DOSFS functions to retrieve the file 
       attributes themselves, and therefore only need to use standard C stdio 
       file functions.
    
    7. The build_catalyst script now detects whether it is being run in the
       current directory. If so, it builds catalyst in this directory,
       otherwise it builds it in the demos\catalyst folder in the Catalina 
       installation tree (as it did in previous releases). The purpose of this 
       is that if you copy the Catalyst folder to your own local user directory, 
       you can build Catalyst locally and do not need to have write permission 
       for the Catalina installation tree. Note that the build_all script already
       supported local builds, it was only the build_catalyst script that did
       not.
    
    8. The build_utilities script now detects whether it is being run in the
       current directory. If so, it builds the utilities in this directory,
       otherwise it builds them in the utilities folder in the Catalina 
       installation tree (as it did in previous releases). The purpose of this 
       is that if you copy the utilities folder to your own local user directory, 
       you can build the utilities locally and do not need to have write 
       permission for the Catalina installation tree. Also, the build_utilities 
       script has been modified to prompt whether the utilities binaries should 
       also be copied to Catalina's bin directory, or simply left in the current
       directory. Copying the utilities to Catalina's bin directory is convenient
       since payload looks for them there if it does not find them in the current
       directory, but it requires write permission to the Catalina installation
       tree. If you do not have this permission, you can now copy the utilities 
       folder to your own user directory, build the utilities there, then copy 
       the binaries to each directory from which programs will be loaded - which 
       may actually be a better solution if you have multiple Propeller platforms
       or configurations which need different versions of the utilities. Note 
       that the utilities are required only for Propeller 1 platforms - there 
       are currently none needed for any of the supported Propeller 2 platforms.
    
    9. The build_all script in the utilities folder has been removed (it was
       deprecated quite a few releases ago). Use the build_utilities script
       instead. The main difference is that build_all used to build binaries 
       for all CPUs of multi-CPU platforms (such as the TRIBLADEPROP) whereas 
       the build_utilities script has to be re-run for each CPU.
    
    Other Changes
    -------------
    
    1. Implemented a workaround for an OpenSpin/Spinnaker bug that meant compiling
       Spin programs with too many short symbol names could fail unexpectedly. The
       current workaround is to simply use longer symbol names. Affected only 
       Propeller 1 programs that used the Catalina Optimizer (which may generate 
       many additional symbol names).
    
    2. Fixed a typo in the DOSFS demo program that prevented it from compiling.
    
    3. Updated the version of Lua used in payload and blackbox to Lua 5.4.4, 
       which is the current version.  Lua 5.1.5 is still included as a Catalyst
       demo program, but is no longer compiled by default (Lua 5.4.4 is now used
       everywhere) and Lua 5.1.5 is now deprecated and may be removed from a
       future release.
    
    4. Since Lua is now compiled in COMPACT mode by default, the pre-compiled P2 
       demo versions of Catalyst (in P2_EVAL.ZIP and P2_VGA.ZIP) are now compiled
       in NATIVE mode, which improves execution speed, but at the expense of
       larger executables for some of the demo programs. If this is a problem, 
       simply recompile Catalyst, specifying COMPACT mode as one of the 
       parameters to the "build_all" script.
    
    5. Updated the notes about compiling Catalyst - a memory model should only be
       specified as an argument to the "build_all" script when compiling for the
       Propeller 2, not for the Propeller 1.
    
    
  • RossHRossH Posts: 5,336
    edited 2022-09-27 06:41

    I have just updated the Catalyst spin demo (located in the "demos/catalyst/demo" folder of the Catalina installation tree) to add a version for the Propeller 2. Unzip the attached zip file over the contents of that folder (it is a complete replacement) or to a separate folder.

    This demo shows how Spin programs can be executed from Catalyst and passed command-line arguments, just like C and Lua programs.

    Demo.spin only supports the Propeller 1, but I have added Demo.spin2 for the Propeller 2. You can use the Propeller Tool to compile them both. I used version 2.7.0.

    Ross.

    EDIT: This is also included in the Catalina 5.3.1 patch, which is now available.

  • RossHRossH Posts: 5,336

    I have just posted Catalina 5.3.1 to SourceForge here.

    This is a patch release that must be installed over Catalina 5.3 (Windows or Linux). The purpose of this patch is to fix a bug I found (finally!) in the Propeller 2 SD Card plugin. There are a few other minor additions and fixes. Here is the relevant portion of the README.TXT:

    RELEASE 5.3.1
    
    New Functionality
    -----------------
    
    1. Catalyst now understands ".lux" as a filename extension, as well as ".lua".
       Catalyst assumes ".lux" represents a compiled Lua script, whereas ".lua"
       can represent either a compiled or a non-compiled Lua script. By default, 
       files with a ".lux" extension will be executed with LUAX.BIN and files 
       with a ".lua" extension will be executed with LUA.BIN. If no extension
       is specified on the command line and files with both extensions exist, 
       ".lux" will be used.
    
    2. There is now a Propeller 2 version of the Catalyst Spin demo that shows how 
       to use Catalyst command-line arguments in Spin2 files. It is in the folder
       demos/catalyst/demo and called Demo.spin2. 
    
    Other Changes
    -------------
    
    1. Fixed a race condition in the SD Card plugin, which meant that it could 
       fail when a sequence of SD Card operations was performed at a specific 
       rate. Affected the Propeller 2 only.
    
    2. The demo program ex_time.c was not clearing the daylight savings flag when
       setting the time, which meant that when the time was retrieved, the hour 
       would sometimes be one hour different than what was set. Affected both
       the Propeller 1 and Propeller 2.
    
    3. Dumbo Basic was not flushing its output buffers, so prompts and input was
       not appearing in some cases when executing input statements until an 
       end-of-line character was entered.
    
  • RossHRossH Posts: 5,336
    edited 2022-10-17 20:49

    I have just posted Catalina 5.4 to SourceForge here.

    This is a full release, primarily to update Catalina to add PSRAM support on the Propeller 2, and Catalyst to add HIRES_VGA support on the Propeller 1, and also add enhanced Catalyst scripting capabilities on both the Propeller 1 and 2. Plus a few significant bug fixes.

    Here is an extract of the README.TXT file concerning the new functionality - the full file is too long to include here, but is attached:

    RELEASE 5.4
    
    New Functionality
    -----------------
    
    1. Roger Loh's 16 Bit PSRAM driver has been added as a Catalina plugin. 
       It is supported on the Propeller 2 only. It is enabled by linking with 
       the new psram library (i.e. adding -lpsram to the Catalina command). 
       An example of its use has been added in "demos\examples\ex_psram.c". 
       The configuration parameters for the driver must be specified in the 
       platform files in the target_p2 directory, such as P2_EDGE.inc. 
    
       You would compile the PSRAM example program with a command like:
    
          catalina -p2 -lc -lpsram -C P2_EDGE ex_psram.c
    
       The only tested platform is the P2 EDGE, and like the driver itself,
       Catalina's support for it should be considered a beta version until
       further notice.
    
    2. As a demonstration of the use of the PSRAM plugin, Lua now has the option 
       to store code in PSRAM on those platforms that support it, such as the 
       P2 EDGE.  This allows larger Lua programs to be executed at the cost of a
       slight speed reduction. Supported on the Propeller 2 only. 
    
       To enable the use of PSRAM in Lua, specify ENABLE_PSRAM to the Catalyst or 
       Lua build_all scripts. For example: 
    
          build_all P2_EDGE SIMPLE VT100 ENABLE_PSRAM
    
       Note that PSRAM is supported only by the Lua execution engine (luax) which
       executes compiled Lua programs, and not for the interactive version (lua)
       that executes text programs or the Multiprocessing version (mlua or mluax).
       So if ENABLE_PSRAM is specified, only luax will be built. This means you 
       may need to build Lua twice - once to build the lua programs that do not 
       use PSRAM, and then again to build luax (only) to use PSRAM.
    
       For example, to compile Lua in directory "demos\catalyst\lua-5.4.4" and put 
       the executables in "demos\catalyst\bin" and call the PSRAM version luaxp 
       rather than overwrite the standard luax binary, you might use commands 
       such as:
    
          cd demos\catalyst\lua-5.4.4
          build_all P2_EDGE SIMPLE VT100
          copy src\*.bin ..\bin\
          build_all P2_EDGE SIMPLE VT100 ENABLE_PSRAM 
          copy src\luax.bin ..\bin\luaxp.bin
    
       Note that specifying ENABLE_PSRAM is applicable only when using the 
       Catalyst and Lua build_all scripts and Makefiles - it is not a general 
       Catalina symbol that can be used on the Catalina command-line to enable 
       PSRAM in other cases (which is done via the usual mechanism of linking 
       the program with the psram library - i.e. adding -lpsram to the catalina
       command).
    
       Finally, note that for small Lua programs, the Hub RAM usage of the PSRAM 
       version may not be much smaller than that of the non-PSRAM version - it 
       may even be larger. This is not only because of the additional PSRAM 
       support code required, it is also because the PSRAM version allocates a 
       fixed amount of Hub RAM on startup to use as a PSRAM cache, and for small
       Lua programs the cache may be larger than the program being loaded. 
       However, the amount of Hub RAM used for Lua code will never increase 
       beyond the cache size no matter how big the program code gets.
    
    3. Catalyst on the Propeller 1 can now use the HIRES_VGA option. However, 
       Hub RAM limitations mean that Catalyst itself needs to be built as an 
       EEPROM program, and some of the Catalyst utilities may only work if 
       they are built as LARGE programs.
    
       This means that while Catalyst itself will work in HIRES_VGA mode on all 
       platforms, some utilities (such as cp & mv) may only work in HIRES_VGA 
       mode on platforms with XMM RAM. Alternatively, you might build Catalyst
       itself in HIRES_VGA mode, but the utilities in LORES_VGA mode.
    
       To facilitate this, two new options have been added that can be used with
       the Catalyst build_all scripts:
    
          EEPROM_CATALYST   specifies that the Catalyst binary should be
                            built as an EEPROM program.
    
          LARGE_UTILITIES   specifies that the Catalyst utilities should be
                            built as LARGE programs.
    
       Whether you need to specify one or both of these options can depend on the
       other options used. For instance, if you need to use the cache to access
       XMM RAM, then you will generally need to use both of these options to 
       build Catalyst in HIRES_VGA mode. For example, here is how you might
       build Catalyst to use HIRES_VGA on the C3:
    
          build_all C3 FLASH CACHED_1K HIRES_VGA EEPROM_CATALYST LARGE_UTILITIES
    
       When you build Catalyst to use LORES_VGA or HIRES_TV HMI option, you may 
       also find that catalyst.binary exceeds the size of Hub RAM and in that 
       case you can specify the EEPROM_CATALYST option, but you may not need the
       LARGE_UTILITIES option - e.g:
    
          build_all C3 FLASH CACHED_1K HIRES_TV EEPROM_CATALYST 
    
       Note that to program Catalyst into EEPROM when the CATALYST_EEPROM option
       is used, you will need to run the build_utilities script to build the 
       EEPROM loader, and then use that with payload. For example, to build and 
       load Catalyst to use HIRES_VGA on the C3, you might use commands like:
    
          cd demos\catalyst
          build_all C3 FLASH CACHED_1K HIRES_VGA EEPROM_CATALYST LARGE_UTILITIES
          build_utilities
          payload -o1 EEPROM ..\bin\catalyst.bin 
    
       Note that these new options are applicable only when using the Catalyst 
       build_all scripts and Makefiles - they are not Catalina symbol that can 
       be used in other circumstances (i.e. specifying -C EEPROM_CATALYST when 
       compiling catalyst.c manually will not have any effect. It is the Makefile
       that intercepts this symbol and instead uses -C EEPROM, but only when 
       building catalyst.binary).
    
    4. The Catalyst ONCE capability (i.e. to execute a command once on reboot)
       has been extended to execute MORE than one command. If enabled when 
       building Catalyst (by setting both ENABLE_ONCE and ENABLE_MORE to 1) then 
       the file (EXECONCE.TXT by default) can contain more than one command. 
       The commands in the file will be executed in sequence on successive 
       reboots. The Lua "execute" function can be used to easily add multiple 
       commands to the file, one per line.  For example, if the MORE capability
       is enabled, the Lua statement:
    
          propeller.execute("vi abc\n vi def")
    
       will cause the propeller to first reboot and execute the command "vi abc", 
       and then when vi exits, the propeller will reboot and execute the command 
       "vi def".
    
       This also allows a very basic scripting capability to be implemented.
       For example, if you have a file called commands.txt which contains all 
       the commands you want executed, then executing the command:
    
          cp command.txt execonce.txt
    
       at the Catalyst prompt, or executing the Lua statement:
    
          propeller.execute("cp command.txt execonce.txt")
    
       will cause all the commands in the file to be executed in sequence. 
    
       Note that this capability is enabled by default on the Propeller 2, and
       on the Propeller 1 unless the HIRES_VGA HMI option is used, because there
       is not enough Hub RAM available. If you want to enable it, you may need to 
       disable something else, such as the capability to allow Lua commands to be
       executed directly from the command line (note that you can still execute 
       them by specifying them as parameters to Lua - e.g. by entering a command 
       like "lua list.lua" instead of just "list"). This can be disabled by 
       editing "demos\catalyst\core\catalyst.h" and rebuilding Catalyst. 
    
    5. The internal cat, dir and help commands have been removed from Catalyst.
       The help and cat commands are now always external. The dir command has
       simply been dropped since the ls command is far more capable. However, if 
       you prefer typing "dir" to "ls", then just make a copy of "ls.bin" called 
       "dir.bin" - i.e:
    
          cp ls.bin dir.bin
    
    6. Added a new Lua demo called "wild.lua" that can be used to add wildcard 
       capability to a Catalyst command that accepts multiple file names.
    
       For example, if the command specified in wild.lua is "vi" (which it is
       by default) then 
    
          luac -o xvi.lux wild.lua
    
       will create a command "xvi" which can then be used on the command line
       to invoke vi on multiple files - e.g:
    
          xvi ex*.lua 
    
       will execute vi on all the Lua example files.
    
    7. The length of a Catalyst command line has been increased to 300
       characters on the Propeller 1, and 1200 characters on the Propeller 2.
       Hub RAM is limited on the Propeller 1, but 300 is enough to accommodate 
       24 MSDOS 8.3 filenames, which could be generated when using wildcards, 
       and 1200 is the maximum that CogStore can store. This maximises the 
       potential usefulness of the wildcard functionality added in the last
       few releases.
    
    8. A new "getrand()" function has been added to the C library. It is defined 
       in "propeller.h" and is implemented on both the Propeller 1 and the
       Propeller 2 (but differently - see below). A program that demonstrates 
       the use of the function has been added in "demos\examples\ex_random.c"
    
       On the Propeller 1, the first time this function is called it calls
       srand() with the current CNT value and is therefore best called after 
       some user input or other random source of delay. It then returns the 
       result of 3 combined calls to rand() to make up 32 random bits (rand 
       itself only returns 15 bits).
    
       On the Propeller 2, the first time this function is called it calls
       srand() with the result of the GETRND opcode, and also returns that 
       value. Thereafter it just returns the result of the GETRND opcode.
    
       This means you can either use just this function, or use this function 
       once to generate a seed for srand() and thereafter use rand(), which is 
       what most traditional C programs would typically do.
    
       Note that rand() only returns a value between 0 and RAND_MAX (inclusive) 
       (i.e. 0 .. 32727 on the Propeller) whereas getrand() returns 32 bits. 
       To simulate rand() using getrand(), use an expression like:
    
              (getrand() % (RAND_MAX + 1))
    
    9. On the P2_EDGE, the base pin for VGA has been changed to 0 (was 32), 
       and the base pin for USB has been changed to 8 (was 40). This makes it
       possible to use the P2-ES VGA and USB accessory boards with Catalina
       on the P2_EDGE. The base pins can be edited if required in the file 
       P2_EDGE.inc in the target_p2 folder.
    
  • RossHRossH Posts: 5,336
    edited 2022-10-30 05:14

    Oops! A silly mistake in the Propeller 1 XMM startup code meant an XMM SMALL program loaded using Catalyst could not access command-line arguments. Does not affect other program types on the Propeller 1, or the Propeller 2.

    To fix it, just drop the attached file over the existing one in Catalina's target directory and recompile any affected programs (you don't need to recompile either Catalina or Catalyst).

    This will also be included in the next release, which is likely to be a very small patch release unless I find any other problems.

    UPDATE: This file is now also included in Catalina patch release 5.4.1

    Ross.

  • RossHRossH Posts: 5,336

    I have just posted Catalina 5.4.1 to SourceForge here.

    This is a patch release that must be installed over Catalina 5.4 (Windows or Linux). The purpose of this patch is to fix the bug that was preventing XMM SMALL programs using command line arguments when loaded using Catalyst, plus a few other minor changes. Here is the relevant portion of the README.TXT:

    RELEASE 5.4.1
    
    New Functionality
    -----------------
    
    1. p2asm now supports the "IF_NN" and "IF_NOT_NN" (where N=0,1 or X) PASM 
       instruction prefixes, as well as "IF_SAME" & "IF_DIFF". For example, 
       the following two statements will generate the same PASM instruction:
    
          if_c_or_z  mov r0, #1
          if_not_00  mov r0, #1
    
    
    Other Changes
    -------------
    
    1. Programs compiled in SMALL mode and loaded using Catalyst were not setting 
       up the argc and argv parameters to the C main function correctly. Affected 
       the Propeller 1 only.
    
    2. The arg.c argument diagnostic program in the utilities folder was not
       working correctly on the Propeller 1.
    
    3. It was not clear in the Catalyst documentation that when Catalyst is
       used to load XMM programs, then both Catalyst and the XMM programs
       must use the same caching and flash options. For example, if Catalyst
       is compiled using FLASH CACHED_1K then so must all XMM programs (but
       note that they can be SMALL or LARGE programs). This only applies to
       XMM programs, and not to LMM, CMM, SMM, Spin or Lua programs. Applies 
       to the Propeller 1 only.
    
    4. The Catalina Optimizer was failing to perform some optimizations when the 
       optimization level was 3 or greater, so that -O3 could result in a larger
       code size than -O2. Affected all memory models except COMPACT on both the
       Propeller 1 and the Propeller 2. 
    
    
  • RossHRossH Posts: 5,336
    edited 2022-11-01 07:26

    Just a quick - but important - update on Lua on the Propeller 2 ...

    I have just realized that Lua can very easily take advantage of the Propeller 2's ability to read and write data to Hub RAM without requiring the address to be aligned on a particular boundary (e.g. you can read or write a long to any address, even if that address is not a multiple of 4).

    This gets around a significant problem using Lua in memory constrained environments, which occurs because Lua is a dynamically typed language. That means that every value in Lua (even a simple integer) must be tagged to tell Lua what type it is (e.g. that the value should be interpreted as an integer and not as a float or a pointer to a string). This requirement means that on most processors, Lua requires 8 bytes to store each 4 byte value - 4 for the value and another 4 for the tag, because even though the tag only needs 1 byte, the alignment requirements of most processors force anything that follows the tag to the next 4 byte boundary, so effectively the tag also takes 4 bytes. For things like arrays this can be a real problem because it can double the memory required!

    But the Propeller 2 can do this in 5 bytes rather than 8 bytes - 4 bytes for the value and 1 byte for the tag, at the trivial cost of one additional clock cycle per access. Chalk one up for the Propeller 2! :)

    By itself, this can reduce a typical Lua program's memory requirement by 5-10% - more in some cases. When combined with the PSRAM support, it can reduce a typical Lua program's memory requirement by 20-30% - more in some cases.

    This will be in the next release of Catalina. But it turns out to be so easy that if anyone is interested in a preview, just drop the attached files over the existing ones in demos\catalyst\lua-5.4.4\src and then rebuild Lua. Note that the alignment fix will work on all Propeller 2 platforms, but PSRAM is currently only supported on the P2 EDGE, and to enable it you must add ENABLE_PSRAM on the command line - e.g:

    cd demos\catalyst\lua-5.4.4
    build_all P2_EDGE SIMPLE ENABLE_PSRAM
    

    Ross.

Sign In or Register to comment.