Shop OBEX P1 Docs P2 Docs Learn Events
Catalina 3.0 - Page 15 — Parallax Forums

Catalina 3.0

11112131517

Comments

  • RossHRossH Posts: 5,519
    edited 2011-07-09 06:23
    All,

    I've just uploaded a new patch release (3.0.4) for Catalina, which contains the new spinc utility that allows the execution of Spin programs from C. The patch is available from the Catalina SourceForge site.

    Ross.
  • RossHRossH Posts: 5,519
    edited 2011-07-10 21:26
    All,

    In recent offline discussions with various people, I have committed to addressing a complaint expressed by a few Catalina users - i.e. that it is complex to add new drivers to Catalina.

    Partly, this will be a documentation exercise to let people know how (and, more importantly, when) to either add a new driver to an existing target package, or to create a new target package for a particular purpose (e.g. for a new board, or for a specific application) - but I also intend to do some more cleaning up of the standard target package. I don't anticipate any functional changes - just a general refactoring.

    However, in addition I am planning to add a new basic target package. This new target package will be in addition to the existing custom target package (which was intended to illustrate how to create a plugin, and supports only the TINY memory model).

    The new basic target will contain all the kernel, memory model and loader selection logic from the standard target, but will not load any other plugins or drivers (possibly excepting the floating point plugins, since these are required by some of the kernels).

    The basic target will also be designed to support only one platform at a time (instead of the dozens currently supported by the standard target and selected by defining various symbols on the command line). You can use the new basic target "as is" if you don't need any additional drivers, or if you intend to load your drivers at run-time (perhaps using Dr_A's cogjects, or the newly released capability for executing any OBEX Spin objects).

    It is the need to support so many different platforms (i.e. different boards, along with their various configurations and optional peripherals) that complicates the standard target, so this will also make it much easier to create a new target for a specific purpose - i.e. just copy and modify the basic target to suit your needs. In fact, this was the original Catalina mechanism for supporting multiple platforms, so all the infrastructure required for doing this is already present - you just set your CATALINA_TARGET environment variable to point to the appropriate target directory before compiling your C program.

    This work will form part of release 3.1 - in fact, since I have just released (as a patch) the only other major item that was originally going to be included in 3.1, this may be all that is included in the next release.

    I will post a note here when it's ready.

    Ross.
  • Dr_AculaDr_Acula Posts: 5,484
    edited 2011-07-10 21:52
    So Catalina also keeps a record in the XMM code of your last memory access, and knows if the next address you are accessing needs the RAM card to be sent a new address or not. Similarly on other platforms Catalina keeps a record of the settings of the latches, and knows whether the next address needs the memory latches to be updated or not.

    These tricks are necessary to get the speed up - but the downside is that only one cog can access the XMM RAM - if another cog tried it would upset the addressing, and any attempt to synchronize multiple accesses would make the whole thing too slow to be useful.

    This is great news. I am close to building a new board with TV and much faster ram using one latch and 20 or 21 pins. (I've been distracted just recently getting a new TV driver and all the quite complex image processing code needed to get the speed and resolution right, but today I watched a 5 minute grab from "The Matrix" on the propeller using Bagger's 256x96 display with Floyd Steinberg dithering, and it looks really good).

    The catch - only 54 longs left as it uses 24k of hub ram for the video buffer, plus the code so not much left to add Kye's nifty audio driver. So silent movies for a little while yet.

    Which is a motivation to get the same code running at a decent speed in XMM in C so it can run movies with sound.

    I doubt I'd be needing two cogs accessing the XMM memory at the same time - the solution which you helped code a few months ago works really well, which is for an XMM program to load up a new cog, put itself to sleep, and for the code in the new cog to restart the XMM processor when it has finished accessing external memory. So no data clashes.
  • RossHRossH Posts: 5,519
    edited 2011-07-13 16:03
    All,

    I'm in the process of testing Catalina 3.1. The changes in this release will be largely cosmetic, but they will make it trivially easy to add new plugins and drivers to Catalina - including plugins based on existing OBEX Spin objects (I will include a fully worked example of doing this).

    One question - I would like to eliminate the existing RESERVE_COG feature and I wondered if anyone was currently using it (if you don't know what this is, you can ignore the rest of this post!).

    This capability allowed you to reserve one or more cogs so that Catalina would not affect them during the program load and initialization phases. It was mainly intended to allow Jazzed to support his 32Mb SDRAM card - but I don't think it was ever used for that, and in any case the same result can now be achieved by using the Caching XMM driver.

    The other possible use for this feature was that it could be used to run Spin programs concurrently with Catalina - but the new ability for Catalina to launch any Spin objects probably makes it redundant in this case as well.

    On balance, I don't think the RESERVE_COG functionality is worth preserving - and it would save me some internal complexity and some testing time to eliminate it altogether. However, if anyone is currently using it then post a message here and I will make sure your needs are catered for in the new release of Catalina.

    Ross.
  • jazzedjazzed Posts: 11,803
    edited 2011-07-13 16:16
    RossH wrote: »
    This capability allowed you to reserve one or more cogs so that Catalina would not affect them during the program load and initialization phases. It was mainly intended to allow Jazzed to support his 32Mb SDRAM card - but I don't think it was ever used for that, and in any case the same result can now be achieved by using the Caching XMM driver.
    My intent is to get the SDRAM driver and the Flash drivers running this summer. So your're saying the cache driver remains loaded and running during the start-up process once started and thus refresh would remain active after loading the code to SDRAM? If so this is a welcome improvement.
  • RossHRossH Posts: 5,519
    edited 2011-07-13 17:57
    jazzed wrote: »
    My intent is to get the SDRAM driver and the Flash drivers running this summer. So your're saying the cache driver remains loaded and running during the start-up process once started and thus refresh would remain active after loading the code to SDRAM? If so this is a welcome improvement.

    Yes, that's the idea - the caching driver works similar to a RESERVE_COG - i.e. you can load and start the caching XMM driver at any point during the Catalina load process - or even prior to the Catlaina load process (as Catalyst does) - and Catalina will not stop that particular cog during the remainder of the load, initialization or execution phases.

    The RESERVE_COG feature came along first, but I then decided to implement much the same functionality in the caching XMM driver anyway - just in case anyone wanted to use both features. So I never ended up using the RESERVE_COG feature for its original purpose - and now it just seems unnecessary.

    Ross.
  • RossHRossH Posts: 5,519
    edited 2011-07-21 07:40
    All,

    Here is an example of using a Spin program as a Catalina plugin - I am posting this here rather than in the "Integrating Spin with C" thread since plugins are currently Catalina-specific functionality.

    In Catalina, the HMI plugin is quite complex since it may contain a screen, keyboard and mouse driver. In this example, I will replace the Catalina HMI plugin with a simplified Spin equivalent, but only include a screen and keyboard driver (using standard OBEX drivers). However, adding a mouse driver would be quite simple.

    First, here is our Spin "plugin" (HMI.spin) - this program is a completely normal Spin program - it just happens to do much the same job as the normal Catalina HMI plugin:
    '' HMI - a demonstration of using a Spin program as a Catalina plugin.
    ''
    ''       This is not a complete HMI plugin replacement, as the
    ''       real HMI plugins do a lot of other work, such as
    ''       cursor management and character translation - but even
    ''       this very simple HMI replacement is quite functional! 
    ''
    CON
    
    ' Define the HMI commands required to implement full C stdio library
    ' support - there are only two, which correspond to getc and putc!
    
    K_WAIT  = 3  ' get character from the keyboard (wait if necessary)
    T_CHAR  = 22 ' put character to the screen (at the current location)
    
    OBJ
    
    common : "Catalina_Common"
    kbd    : "Keyboard"
    scr    : "TV_Text"
    
    PUB Start | cog, command, data
    
       ' start the standard OBEX keyboard and screen drivers
       kbd.Start (26, 27)   ' pin numbers for C3 keyboard
       scr.Start (12)       ' pin numbers for C3 TV output
    
       ' register ourselves as a HMI plugin
       ' (so the Catalina kernel can find us!) 
       cog := cogid
       common.Register(cog, common#LMM_HMI)
    
       ' now process commmands as they appear in the registry
       repeat
    
          command := long[common#REQUESTS][2 * cog]
          case (command >> 24)
          
             K_WAIT:
                data := kbd.newkey & $FF
                if (data == 13) ' translate CR to LF
                   data := 10
                long[common#REQUESTS][2 * cog + 1] := data
                long[common#REQUESTS][2 * cog] := 0
                            
             T_CHAR:
                data := command & $FF
                if (data == 10) ' translate LF to CR
                   scr.out(13)
                else
                   scr.out(data)
                long[common#REQUESTS][2 * cog] := 0
    
    Now here is the C program (run_tiny_hmi.c) that loads the Spin plugin at run time, then uses it via the standard C stdio library functions (e.g. getchar(), printf() etc):
    // include some standard header files
    #include <stdio.h>
    
    // include the utilities required to launch SPIN programs
    #include "spin_utilities.h"
    
    // include the Spin HMI program (as generated by the SpinC utility)
    #include "tiny_hmi.c"
    
    void main() {
    
       char ch;
    
       // declare local storage for the Spin HMI program
       char var[TINY_HMI_VAR_SIZE];
       char stack[TINY_HMI_STACK_SIZE];
    
       // start the Spin HMI (which will start the tv and keyboard drivers)
       start_TINY_HMI(var, stack);
    
       // print a welcome message (via stdio!)
       printf("Welcome to Catalina\n");
    
       // now read and process characters (via stdio!)
       while (1) {
    
          // get a key
          ch = getchar();
    
          // interpret the keys '0' .. '9' specially 
          // (note that getchar echoes the key itself)
          switch (ch) {
             case '1': printf(" one ");   break;
             case '2': printf(" two ");   break;
             case '3': printf(" three "); break;
             case '4': printf(" four ");  break;
             case '5': printf(" five ");  break;
             case '6': printf(" six ");   break;
             case '7': printf(" seven "); break;
             case '8': printf(" eight "); break;
             case '9': printf(" nine ");  break;
             case '0': printf(" zero ");  break;
          }
    
       }
    
    }
    
    The commands I used to compile these programs are as follows (note you need Catalina 3.0.4 to do this!):
    homespun hmi.spin -L "C:\Program Files\Catalina\target" -b
    spinc HMI.binary -t -n TINY_HMI -s 200 > tiny_hmi.c
    catalina -lc run_tiny_hmi.c spin_utilities.c -D NO_HMI -D C3
    
    Note the use of the NO_HMI option on the catalina command - we don't want Catalina to include its usual HMI plugins, since our C program wants to use the Spin plugin to do that job. Once the Spin plugin has been loaded and started, the C program can't tell the difference - it can just use all the C stdio library functions as usual!

    I'll include this (and other examples) in the forthcoming Catalina 3.1 release.

    Ross.
  • David BetzDavid Betz Posts: 14,516
    edited 2011-07-21 09:21
    Hi Ross!

    This interface looks pretty straight forward. I assume that the tiny_hmi.c file created by spinc contains the definitions for TINY_HMI_VAR_SIZE and TINY_HMI_STACK_SIZE and that it defines the function start_TINY_HMI which calls coginit to start the Spin binary code. I assume also that the Catalina runtime code always uses the common#LMM_HMI mailbox to read/write console characters. Is that correct? Does that mean that any code that I write that uses that mailbox and obeys your mailbox protocol will work to supply console I/O to Catalina?

    Thanks,
    David
  • RossHRossH Posts: 5,519
    edited 2011-07-21 15:50
    David Betz wrote: »
    Hi Ross!

    This interface looks pretty straight forward. I assume that the tiny_hmi.c file created by spinc contains the definitions for TINY_HMI_VAR_SIZE and TINY_HMI_STACK_SIZE and that it defines the function start_TINY_HMI which calls coginit to start the Spin binary code. I assume also that the Catalina runtime code always uses the common#LMM_HMI mailbox to read/write console characters. Is that correct? Does that mean that any code that I write that uses that mailbox and obeys your mailbox protocol will work to supply console I/O to Catalina?

    All correct. You choose the name you want to use when you run the spinc program (in this case I chose TINY_HMI). Then you choose the plugin type at runtime. By registering as a HMI plugin, this program is telling Catalina it should be sent all requests for stream I/O on stdin, stdout and stderr. If it registered instead as a FIL plugin, it would be sent all disk sector read/write requests. If it registered as a CLK plugin it would be sent time related requests.

    This is Catalina's "hardware abstraction layer" in action. Catalina doesn't care who provides these services, or how they are provided. You could even write them in GCC if you were sufficiently masochistic! :)

    Ross.
  • Bill HenningBill Henning Posts: 6,445
    edited 2011-07-22 07:46
    Hi Ross,

    This is fantastic; it makes integrating OBEX objects with Catalina a snap.

    Great work!

    Bill
    RossH wrote: »
    All,

    Here is an example of using a Spin program as a Catalina plugin - I am posting this here rather than in the "Integrating Spin with C" thread since plugins are currently Catalina-specific functionality.

    In Catalina, the HMI plugin is quite complex since it may contain a screen, keyboard and mouse driver. In this example, I will replace the Catalina HMI plugin with a simplified Spin equivalent, but only include a screen and keyboard driver (using standard OBEX drivers). However, adding a mouse driver would be quite simple.

    First, here is our Spin "plugin" (HMI.spin) - this program is a completely normal Spin program - it just happens to do much the same job as the normal Catalina HMI plugin:

    ...

    I'll include this (and other examples) in the forthcoming Catalina 3.1 release.

    Ross.
  • RossHRossH Posts: 5,519
    edited 2011-07-22 17:08
    Hi Bill,

    Yes, that's the idea. It will be even easier in release 3.1 because I have added some Spin support functions to simplify integrating Spin programs with the Catalina registry.

    This will never be my "preferred" method of integration, because Spin plugins are neither as fast nor as memory efficient as their PASM equivalents (this may surprise some people - but remember that PASM plugins need not consume any Hub RAM, whereas Spin plugins always do). However, it certainly makes it easy to "plug and play" with OBEX objects from C.

    Catalina 3.1 will also include a new and simplified target package that will also assist with this. The new "basic" target is much simpler than the standard target package since by default it loads no plugins - it is specifically intended be used by those who want to load their own OBEX objects and/or their own plugins (Spin or PASM).

    Release 3.1 is nearly ready. It just takes a bit of time to work through the tests on all the various platforms and confirgurations it has to support.

    Ross.
  • Cluso99Cluso99 Posts: 18,069
    edited 2011-07-22 18:58
    Great work Ross. I should be onto Catalina sometime late next week :) This makes it so easy to add spin/pasm. Often, spin speed is not a concern.

    Still sending pcbs out for Mondays production.
  • RossHRossH Posts: 5,519
    edited 2011-07-26 18:13
    All,

    A quick update, and then a question ...

    I've finished testing Catalina 3.1 on Windows. It's really looking really quite nifty now - it is much tidier, and also much easier to add new drivers or support new platforms. It took slightly longer than I expected because of yet more annoying Windows 7 "features".

    Now I need to spend some time on Linux support - it seems I really need to issue two different Linux releases - one 32 bit and one 64 bit. Then there are a few documentation updates to do. Expect a full release for all platforms in about a week.

    Now onto the question. First some background: Some people are confused by the current Catalina build_all batch files/bash scripts, and would prefer to use make. This is simple enough to do, but I have resisted it since I don't really want to have to support one mechanism for building all the utilities and demos under Windows, and another for Linux - and Windows has no native make capability. At the moment, Catalina doesn't require the installation of any other software just to use it - you only need to install additional things things if want to rebuild it.

    So - would anyone have a problem with having to install a Unix-like environment (probably MinGW) just to be able to use Catalina?

    Ross.
  • jazzedjazzed Posts: 11,803
    edited 2011-07-26 18:47
    GNU Make is free and does not require MinGW. http://www.gnu.org/software/make/
  • RossHRossH Posts: 5,519
    edited 2011-07-26 19:58
    jazzed wrote: »
    GNU Make is free and does not require MinGW. http://www.gnu.org/software/make/

    Thanks for the link jazzed, but I'm not quite sadistic enough to expose Windows users directly to GNU Make without the assistance of the other utilities (such as a bash shell) that help to make it useful.

    Ross.
  • jazzedjazzed Posts: 11,803
    edited 2011-07-26 20:17
    RossH wrote: »
    Thanks for the link jazzed, but I'm not quite sadistic enough to expose Windows users directly to GNU Make without the assistance of the other utilities (such as a bash shell) that help to make it useful.

    Ross.

    Huh? I've often used it from DOS cmd shell.
  • RossHRossH Posts: 5,519
    edited 2011-07-26 20:44
    jazzed wrote: »
    Huh? I've often used it from DOS cmd shell.

    Of course - it's not invoking it that's the issue. It's the way you have to write makefiles in a DOS environment. Not only is that frustrating enough to make grown men weep, it also means that makefiles taken from other sources (often copied along with the code itself) are virtually unusable by anyone not very familiar with make.

    With MinGW you generally only need to make minor amendments to an existing makefile to make it work under DOS. Without it ... well, let's just say it can get real ugly real fast!

    Ross.
  • jazzedjazzed Posts: 11,803
    edited 2011-07-26 20:52
    RossH wrote: »
    Of course - it's not invoking it that's the issue. It's the way you have to write makefiles in a DOS environment. Not only is that frustrating enough to make grown men weep, it also means that makefiles taken from other sources (often copied along with the code itself) are virtually unusable by anyone not very familiar with make.

    With MinGW you generally only need to make minor amendments to an existing makefile to make it work under DOS. Without it ... well, let's just say it can get real ugly real fast!

    Ross.
    RossH wrote: »
    Now onto the question. First some background: Some people are confused by the current Catalina build_all batch files/bash scripts, and would prefer to use make. This is simple enough to do, but I have resisted it since I don't really want to have to support one mechanism for building all the utilities and demos under Windows, and another for Linux - and Windows has no native make capability. At the moment, Catalina doesn't require the installation of any other software just to use it - you only need to install additional things things if want to rebuild it.

    So - would anyone have a problem with having to install a Unix-like environment (probably MinGW) just to be able to use Catalina?

    So you brought up the question just to have an argument?
  • Cluso99Cluso99 Posts: 18,069
    edited 2011-07-26 20:56
    For us newbies when it comes to these things... please KISS. While I like the old systems such as CPM, I hate all the commands required to "compile" code. VB removed such things from the users sights and made it easy. PropTool is fundamentally simple, and so is bst (windoze).

    Not to start a war, but me thinks *nix is a regression :(
  • jazzedjazzed Posts: 11,803
    edited 2011-07-26 21:16
    Simple customers need simple things.
  • RossHRossH Posts: 5,519
    edited 2011-07-26 21:18
    jazzed wrote: »
    So you brought up the question just to have an argument?

    Not sure what you mean. The original question was (paraphrasing dramatically here) "would anyone object to installing MinGW". You responded with "just install make". I responded with "make by itself is not much use". So the original question still stands.

    Ross.
  • RossHRossH Posts: 5,519
    edited 2011-07-26 21:23
    jazzed wrote: »
    Simple customers need simple things.

    Hmmm. Why would sophisticated customers not also appreciate simple things?

    Ross.
  • jazzedjazzed Posts: 11,803
    edited 2011-07-26 21:24
    Make by itself would allow customers who require that feature to build programs using Catalina without installing MinGW.

    Many corporate customers require build infrastructures. They are used to simplify things for the organization as a whole. Most people reading here wouldn't understand that.
  • potatoheadpotatohead Posts: 10,261
    edited 2011-07-26 21:41
    I vote for having the system work on a Windows computer that just contains Windows. The more experienced bears can always make additions and create build environments that suit them, and those can be rather easily documented, and even packaged, if people so desire.

    Overall, I like lean, and having it work on "just Windows" is lean. Pretty much that's my default, unless I see value in thicker environments. Lean isn't always wise, but that depends on a very large number of things. IMHO, that variance more or less justifies the minor league investment to insure it's a user choice. Besides, that's less overall dependencies on the development end. Seems to me, that's gotta pay off too, or it's thick for no real return, and that's generally a undesirable state.
  • RossHRossH Posts: 5,519
    edited 2011-07-26 21:48
    jazzed wrote: »
    Make by itself would allow customers who require that feature to build programs using Catalina without installing MinGW.
    Quite right. But I don't need to do anything to support users who want to use gnu make (or nmake or cmake or any of the other available make variants) to build their Catalina programs. They can already do so.

    My question was intended to be addressed to people who don't already use such tools, but who want to build the Catalina demos and utilities - obviously I can't write a different makefile for every possible make variant out there, so would the requirement to download and install MinGW be a problem?

    It's a simple enough question.
    jazzed wrote: »
    Many corporate customers require build infrastructures. They are used to simplify things for the organization as a whole. Most people reading here wouldn't understand that.

    So sophisticated users do appreciate simple solutions!

    Ross.
  • RossHRossH Posts: 5,519
    edited 2011-07-26 21:52
    potatohead wrote: »
    I vote for having the system work on a Windows computer that just contains Windows. The more experienced bears can always make additions and create build environments that suit them, and those can be rather easily documented, and even packaged, if people so desire.

    Overall, I like lean, and having it work on "just Windows" is lean. Pretty much that's my default, unless I see value in thicker environments. Lean isn't always wise, but that depends on a very large number of things. IMHO, that variance more or less justifies the minor league investment to insure it's a user choice. Besides, that's less overall dependencies on the development end. Seems to me, that's gotta pay off too, or it's thick for no real return, and that's generally a undesirable state.

    I tend to agree. Perhaps it was a mistake to even offer build scripts in the first place (after all, they were originally intended to make my life simpler, not the users!).

    Maybe I should instead move everything over to Code::Blocks and discard both the makefiles and the build scripts. I could just as easily do this.

    Ross.
  • jazzedjazzed Posts: 11,803
    edited 2011-07-26 21:57
    RossH wrote: »
    So sophisticated users do appreciate simple solutions!
    If it doesn't fit their needs, they won't buy it.
  • RossHRossH Posts: 5,519
    edited 2011-07-26 22:35
    RossH wrote: »
    potatohead wrote: »
    I vote for having the system work on a Windows computer that just contains Windows ...

    Maybe I should instead move everything over to Code::Blocks and discard both the makefiles and the build scripts. I could just as easily do this.

    Ok - sorry to keep on about this, but perhaps I have inadvertently answered my own question. Perhaps my original question was actually a bit silly, and potatohead is right - why would anyone who doesn't already use Unix want to download and use any version of "make"? And why on earth should I force them to do so?

    One thing I've learned from Catalina is that that only a small fraction of users (like me) still prefer to use a command line compiler, and would therefore be even remotely interested in such things.

    What got me thinking is that I already require users to download, install and configure Code::Blocks separately. Requiring them to also download, configure and learn MinGW (or gnu make, or anything else for that matter) could be the straw that broke the camel's back - and for what? With Code::Blocks you don't need such things (it has functionality equivalent to "make" built in).

    Perhaps I should instead just distribute my own version of Code::Blocks with Catalina already fully integrated - the user experience would then be much more like the Parallax Propeller Tool - no MinGW, no make, no nothing - just one download, one install and you're up and running!

    I'll get release 3.1 out, and then look into this a bit further. Any comments welcome.

    Ross.
  • jazzedjazzed Posts: 11,803
    edited 2011-07-26 22:58
    RossH wrote: »
    Perhaps I should instead just distribute my own version of Code::Blocks with Catalina already fully integrated - the user experience would then be much more like the Parallax Propeller Tool
    Sounds like a good plan to me as long as code blocks is optional.
  • Dr_AculaDr_Acula Posts: 5,484
    edited 2011-07-26 23:15
    I don't know much about Unix so will leave comments about that to the experts.

    The command line format has been easy to integrate into a vb.net IDE to the point that it is similar to the proptool - write some code, hit one button and it is downloaded and running.

    I must confess I haven't tried the code::blocks version from a few weeks back as the install looked a little daunting.

    I think the idea of distributing a customised code::blocks/catalina version sounds brilliant.
Sign In or Register to comment.