Catalina - ANSI C for the Propeller 1 & 2

Wingineer19 · 2023-07-11 17:18

Hi RossH,

Assuming that altering the value of XMM_Len within the DUALSRAM driver was the cause of the lockup problem in the above test program, I modified the XMM Direct functions to prevent this by copying the value of XMM_Len into register RamLoop and then only change the value of RamLoop.

I made the same modification to the XMM Cached functions because previously they too were decrementing the value of XMM_Len, yet seemed to work perfectly fine while doing that.

Summarizing the test results after the latest driver modification to address the XMM_Len issue:

The test program works fine when running in either XMM LARGE or XMM SMALL when the cache is used.
It also works fine when running in XMM SMALL without using the cache.
It works in XMM LARGE without cache up until the point that the values within union are changed, then the program locks up and requires a Prop reset.

With the exception of when the Kernel calls the XMM_WriteLong or XMM_ReadLong function the value of XMM_Len isn't changed within the XMM Direct or XMM Cached functions anymore.

But when the Kernel calls either XMM_WriteLong or XMM_ReadLong I assumed that it didn't set the value of XMM_Len to four before calling them so these two functions do that. If the Kernel does set the value, then I will remove the mov XMM_Len,#04 instruction at the start of each of these functions.

I've attached the revised DUALSRAM driver for your review.

Of course if the Kernel always provided the value of XMM_Len regardless of the function called, and allowed me to change its value, that would save me about six LONGs in my driver

RossH · 2023-07-11 22:34

Hello @Wingineer19

Hmmm. Please try the attached XMM kernel (unzip it over the existing one in the target directory, but save the original) and tell me if it fixes the problem with using LARGE mode without the cache. This version should allow you to change the value of XMM_Len in the various XMM functions.

I have not had time to review your drivers yet, but you should not need to set XMM_Len to 4 when using XMM_ReadLong and XMM_WriteLong unless these functions use XMM_ReadMult and XMM_WriteMult internally (which some of my XMM drivers do to save space).

Also, can you tell me if your implementation of XMM_ReadLong, XMM_WriteLong, XMM_ReadMult and XMM_WriteMult all increment XMM_Addr by the appropriate amount (i.e. either by 4 or by the original value of XMM_Len). If not, this might explain the problem.

Ross.

Wingineer19 · 2023-07-12 00:14

Hi RossH,

Yes, this new XMM Kernel appears to have fixed the problem.

Yes, all of the XMM Direct and XMM Cached functions do increment the XMM_Addr by the XMM_Len value provided by the Kernel.

I've modified the DUALSRAM driver to eliminate the mov XMM_Len,#04 instruction from both the XMM_WriteLong and XMM_ReadLong since the Kernel provides this value when calling these functions. I've also revamped all of the functions so that they once again decrement the XMM_Len value.

The aforementioned changes allowed me to shrink the XMM Direct functions (XMM_Activate, XMM_TriState, SendRamMemReq, XMM_WriteLong/XMM_WriteMult, and XMM_ReadLong/XMM_ReadMult) down to a total of 84 LONGs if my math is correct.

I've attached the newest and revised version of the DUALSRAM driver for review at your convenience. I will continue to examine it to see if I can shrink the size down further but probably not.

Thanks for the quick revision to the XMM Kernel to address the XMM LARGE problem. I will continue the testing and let you know if any other anomalies arise.

Wingineer19 · 2023-07-16 04:41

Hi RossH,

The testing of the DUALSRAM driver running XMM LARGE is ongoing and looking good, so I would like to briefly digress from that to address another issue, specifically the EEPROM Loader.

You previously said that although the existing Loader is old you thought it could handle EEPROMs up to 128 kilobytes in size.

It's possible to access up to 512 kilobytes on an I2C bus using various combinations of EEPROMs. I mention this because in addition to the two SRAM chips on my DUALSRAM board, it has some EEPROMs which allow the FLiP to access up to 512 kilobytes total.

I didn't want to mutilate the FLiP, so my board just adds an additional 448 kilobytes to its existing 64 kilobytes for a total of 512 kilobytes. It did this by including an additional 64 kilobyte EEPROM, a 128 kilobyte EEPROM, and a 256 kilobyte EEPROM.

Each EEPROM type either makes direct reference to the [A2, A1, A0] address lines within the Device Address Byte or redefines them in whole or in part. For example, the 64 kilobyte chip calls them [A2, A1, A0], the 128 kilobyte chip calls them [A2, A1, P0], and the 256 kilobyte chip calls them [A2, A17, A16].

Regardless of how each chip defines them, given the following Memory Map, these bits within the Device Address Byte can effectively be considered to be, and behave as, memory address bits [A18, A17, A16], with the lower memory address bits of [A15 to A0] contained within the First Word Address Byte and Second Word Address Byte:

                                   DUALSRAM Expansion Module
EEPROM Device   Size    A2  A1  A0   Memory Map Range (Hex)         Comments
-------------   ----    --  --  --   ---------------------    --------------------- 
FLiP Internal   64KB    0   0   0     0x00000 -> 0x0FFFF      [GND: A2,A1,A0]

DUALSRAM 64KB   64KB    0   0   1     0x10000 -> 0x1FFFF      [GND: A2,A1],[3V3: A0]

DUALSRAM 128KB  128KB   0   1   -     0x20000 -> 0x3FFFF      [GND: A2],[3V3: A1],[No A0]

DUALSRAM 256KB  256KB   1   -   -     0x40000 -> 0x7FFFF      [3V3: A2],[No A1 or A0]

I compiled an XMM program using the -C EEPROM option and attempted to send it to EEPROM using the payload EEPROM designation, but the EEPROM Loader was very unhappy:

I suspect the Loader could be attempting to write across the 0x0FFFF to 0x10000 boundary, which it shouldn't do even using 32 byte pages, but it seems like a reasonable explanation.

Tachyon had no problem with the physical EEPROM boundaries between chips and happily gave me this result:

Tachyon Commands:
$0000 $80000 $5A EFILL -->  ok
$0000 $80000 EE DUMP --> 

FLiP 64KB EEPROM (0 -> 64KB):
0000.0000:   5A 5A 5A 5A  5A 5A 5A 5A  5A 5A 5A 5A  5A 5A 5A 5A    ZZZZZZZZZZZZZZZZ
0000.FFF0:   5A 5A 5A 5A  5A 5A 5A 5A  5A 5A 5A 5A  5A 5A 5A 5A    ZZZZZZZZZZZZZZZZ

MemBoard 64KB EEPROM (64KB -> 128KB):
0001.0000:   5A 5A 5A 5A  5A 5A 5A 5A  5A 5A 5A 5A  5A 5A 5A 5A    ZZZZZZZZZZZZZZZZ
0001.FFF0:   5A 5A 5A 5A  5A 5A 5A 5A  5A 5A 5A 5A  5A 5A 5A 5A    ZZZZZZZZZZZZZZZZ

MemBoard 128KB EEPROM (128KB -> 256KB):
0002.0000:   5A 5A 5A 5A  5A 5A 5A 5A  5A 5A 5A 5A  5A 5A 5A 5A    ZZZZZZZZZZZZZZZZ
0003.FFF0:   5A 5A 5A 5A  5A 5A 5A 5A  5A 5A 5A 5A  5A 5A 5A 5A    ZZZZZZZZZZZZZZZZ

MemBoard 256KB EEPROM (256KB -> 512KB)
0004.0000:   5A 5A 5A 5A  5A 5A 5A 5A  5A 5A 5A 5A  5A 5A 5A 5A    ZZZZZZZZZZZZZZZZ
0007.FFF0:   5A 5A 5A 5A  5A 5A 5A 5A  5A 5A 5A 5A  5A 5A 5A 5A    ZZZZZZZZZZZZZZZZ ok

So whatever technique Tachyon is using it was able to seamlessly write/read across these physical boundaries without any problem.

A long time ago I asked Peter about this and he provided this explanation:

Tachyon will fill and copy to EEPROM in a page size to suit the device which is
typically 128 bytes for 64KB chips. Each time it performs a write it calculates the
device address so it will cross device boundaries seamlessly. So each page write is:

<START> <DEVICE ADDR> <WRITE> <16-bit MEMORY ADDRESS> <128 BYTES> <STOP>

Because the device may be busy programming a page of data, the device address
is polled for an ack to indicate when it is ready along with a timeout to prevent 
hanging.

OK, I can understand how a write/read/verify approach would work to cross device boundaries when the FLiP programs the EEPROM. If the FLiP encountered a NACK from the EEPROM it could switch to reading back the bytes it just sent to determine the failure address. It could then initiate a new write operation starting at that address to hopefully resume page writes from there.

But how could the FLiP determine EEPROM device boundaries when strictly reading from EEPROM since the FLiP is the one which provides the ACK/NACK signal to the EEPROM in that case? I suppose one could implement a Random Read command to read each individual byte within EEPROM. That would allow the seamless crossing of device boundaries, but require an inordinate amount of time just to read from the EEPROMs.

Then Peter added:

Practically all modern I2C EEPROMs accept 16-bits for their address and even for 
128kB and larger devices. Whereas you would expect A16 to be the b1 or the lsb 
of the 7-bit address, it is in fact b3 for Atmel/Microchip x1025 devices, which is 
rather non-standard. 

Tachyon doesn't perform any voodoo, it just translates A16,A17,A18 into the device 
address which most normal larger devices handle, well normally. I favor the 
M24M0x EEPROMs.

Page writes can never cross boundaries because they always start on an aligned 
page address. You cannot start a page on the last byte for instance. But page writes
take the same amount of time as a byte write, probably because that's what they 
are doing internally.

Now with this dissertation in mind, is it possible that the EEPROM Loader is attempting to write across device boundaries resulting in failure? If so, how difficult would it be to fix?

RossH · 2023-07-16 07:13

@Wingineer19 said:
Now with this dissertation in mind, is it possible that the EEPROM Loader is attempting to write across device boundaries resulting in failure? If so, how difficult would it be to fix?

Hello @Wingineer19

It is possible. However, I can't even think about it for at least another week. We are preparing to sell our current property and are busy getting it ready to put on the market. We have been flat out for weeks preparing it (our property is a bush retreat with 4 cabins on 100 acres) and we still have lots to do. But photographs are due to be taken next week and after that we can relax a little and let fate take its course. Until then I can't do much more than acknowledge your post, but I promise to read it in detail after that.

Ross.

RossH · 2023-07-23 07:53

Hello @Wingineer19

I've finally had a chance to look into this a little. Catalina uses Mike Green's sdspiFemto.spin I2C/SPI module (Catalina's version is in the target directory, called catalina_sdspiFemto.spin - but the only difference is that Catalina's version has the Spin code removed, leaving only the PASM functions).

This module says it can read and write EEPROMs up to 512kb if the EEPROM is implemented internally as multiples of 64kb EEPROMs. It certainly works for me on the 128kb EEPROM used on the HYDRA, which is an Atmel 24C1024. I have nothing else that has EEPROMs larger than 64kb that I can try it on.

So I think that if you can get this module working with your EEPROMs, then Catalina should work. You can download Mike Green's FemtoBasic (which uses this module) here - https://github.com/parallaxinc/propeller/tree/master/libraries/community/p1/All/FemtoBasic

Ross.

Wingineer19 · 2023-07-24 06:54

Hi RossH,

Thanks for the info and I hope the situation with the sale of your property is going well.

Suppose I compiled a program in Catalina resulting in a binary file called test.binary. My thought was that maybe I can just write a program that would copy this binary file verbatim to the eeprom. Then I remembered that I already did that years ago.

The first program is called Upload.c and is compiled using the OpenWatcom C++ compiler yielding a Windows executable called Upload.exe. Here's the source code:

// Program Is UpLoad.c
// Sends EEPROM File To Prop1
// Last Revision On 25Feb20

#define CBR_230400 230400L
#define Limit  16

#include <windows.h>
#include <stdio.h>
#include <conio.h>
#include <stdlib.h>
#include <stdarg.h>
#include <string.h>
#include <stddef.h>
#include <time.h>
#include <math.h>
#include <process.h>

HANDLE hCom;

FILE *InPutFile;

char FileNameStr[80];

int GetChar(void)
{
 BOOL Status;
 DWORD BytesRead;
 char RxD[10];
 Status=ReadFile(hCom,RxD,1,&BytesRead,NULL);
 if(BytesRead < 1) return(-1);
 else return(RxD[0]);
}

char LoopBack(char OutPut)
{
 int Result;
 time_t TimeStart;
 time_t TimeStop;
 TransmitCommChar(hCom,OutPut);
 time(&TimeStart);
 for(;;)
  {
   time(&TimeStop);
   if(difftime(TimeStop,TimeStart) > 1.0) return(0);
   Result=GetChar();
   if(Result == OutPut) return(OutPut);
  }
 return(0);
}

char OutBack(char OutPut, char InPut)
{
 time_t TimeStart;
 time_t TimeStop;
 TransmitCommChar(hCom,OutPut);
 time(&TimeStart);
 for(;;)
  {
   time(&TimeStop);
   if(difftime(TimeStop,TimeStart) > 1.0) return(0);
   if(GetChar() == InPut) return(1);
  }
 return(0);
}

char FindComPort(void)
{
 char Index;
 char ComPort[20];
 DCB dcbSerialParams; 
 COMMTIMEOUTS timeouts; 
 COMMPROP comprop;
 dcbSerialParams.DCBlength = sizeof(dcbSerialParams); 
 for(Index=0; Index<10; Index++)
  {
   sprintf(ComPort,"COM%d",Index);
   hCom=CreateFile(ComPort,GENERIC_READ | GENERIC_WRITE,0,NULL,OPEN_EXISTING,FILE_ATTRIBUTE_NORMAL,NULL);
   if(hCom == INVALID_HANDLE_VALUE) continue;
   if(SetupComm(hCom, 1024, 1024) == FALSE) continue;
   if(GetCommProperties(hCom,&comprop) == FALSE) continue;
   if(GetCommState(hCom, &dcbSerialParams) == FALSE) continue; 
   dcbSerialParams.BaudRate = CBR_115200;
   dcbSerialParams.ByteSize = 8;
   dcbSerialParams.StopBits = ONESTOPBIT;
   dcbSerialParams.Parity = NOPARITY; 
   if(SetCommState(hCom, &dcbSerialParams) == FALSE) continue; 
   timeouts.ReadIntervalTimeout         = MAXDWORD;
   timeouts.ReadTotalTimeoutConstant    = 0;
   timeouts.ReadTotalTimeoutMultiplier  = 0;
   timeouts.WriteTotalTimeoutConstant   = 0;
   timeouts.WriteTotalTimeoutMultiplier = 0;
   if(SetCommTimeouts(hCom, &timeouts) == FALSE) continue;
   if(OutBack('?','P') > 0) break;
  }
 if(Index > 9) return('N');
 printf("Found On %s\n",ComPort);
 return('Y');
}

char Command(void)
{
 long MemAdd=0;
 long Value=0;
 char TempStr[80];
 char RxD;
 char Result;
 int  Index;
 int  InPutData;
 printf("Searching For Propeller...");
 if(FindComPort() == 'N') return('N');
 printf("Opening %s Data File\n",FileNameStr);
 InPutFile=fopen(FileNameStr,"rb");
 if(InPutFile == NULL) return('E');
 printf("Sending EEPROM Data To Propeller\n");
 for(;;)
  {
   sprintf(TempStr,"%.8X",MemAdd);
   if(LoopBack('M') == 0) return('T');
   for(Index=0; Index<8; Index++)
    {
     Result=LoopBack(TempStr[Index]);
     if(Result == 0) return('T');
     printf("%c",Result);
    }
   printf(" ");
   if(LoopBack('L') == 0) return('T');
   for(Value=0; Value<Limit; Value++)
    {
     InPutData=fgetc(InPutFile);
     switch(InPutData)
      {
       case EOF: fclose(InPutFile);
                 if(Value == 0) return('D');
                 if(LoopBack('W') == 0) return('T');
                 return('D');
                 break;
       default:  RxD=(char) InPutData;
                 sprintf(TempStr,"%.2X",RxD);
                 if(LoopBack(TempStr[0]) == 0) return('T');
                 if(LoopBack(TempStr[1]) == 0) return('T');
                 printf("%s ",TempStr);
                 break;
       }
    }
   if(LoopBack('W') == 0) return('T');
   printf("\r");
   MemAdd=MemAdd + Limit;
  }
 return('T');
}

void main(int argc, char *argv[])
{
 char Cmd;
 time_t TaskStart;
 time_t TaskStop;
 strcpy(FileNameStr,argv[1]);
 system("cls");
 time(&TaskStart);
 system("payload propload.binary");
 Cmd=Command();
 switch(Cmd)
  {
   case 'E': printf("\nError Opening EEPROM File\n");
             break;
   case 'N': printf("\nPropeller Not Found\n");
             CloseHandle(hCom);
             break;
   case 'D': printf("\nCompleted Sending EEPROM Data\n");
             time(&TaskStop);
             printf("Task Duration=%f Seconds\n",difftime(TaskStop,TaskStart)); 
             CloseHandle(hCom);
             break;
   case 'T': printf("\nTimeout Error Occurred\n");
             CloseHandle(hCom);
   default:  break;
  }
}

The second program is called PropLoad.c and is compiled by Catalina itself yielding a binary file called PropLoad.binary. Since it's written in C there was no need to mess with Spin or assembly. Here's the source code:

// Program Is PropLoad.c
// Last Revision On 25Feb20

#include <stdio.h>
#include <stdlib.h>
#include <stdarg.h>
#include <math.h>
#include <string.h>
#include <time.h>
#include <propeller.h>
#include <catalina_rtc.h>


#define     SDA_Pin                     29
#define     SCL_Pin                     28

#define     SDA_Hi                      1 << SDA_Pin
#define     SCL_Hi                      1 << SCL_Pin

#define     SDA_Lo                      SDA_Hi ^ 0xffffffff
#define     SCL_Lo                      SCL_Hi ^ 0xffffffff

struct Memee
{
 unsigned int A00     :1;
 unsigned int A01     :1;
 unsigned int A02     :1;
 unsigned int A03     :1;
 unsigned int A04     :1;
 unsigned int A05     :1;
 unsigned int A06     :1;
 unsigned int A07     :1;

 unsigned int A08     :1;
 unsigned int A09     :1;
 unsigned int A10     :1;
 unsigned int A11     :1;
 unsigned int A12     :1;
 unsigned int A13     :1;
 unsigned int A14     :1;
 unsigned int A15     :1;

 unsigned int A16     :1;
 unsigned int A17     :1;
 unsigned int A18     :1;
 unsigned int A19     :1;
 unsigned int A20     :1;
 unsigned int A21     :1;
 unsigned int A22     :1;
 unsigned int A23     :1;

 unsigned int RW      :1;
 unsigned int E16     :1;
 unsigned int E17     :1;
 unsigned int E18     :1;
 unsigned int Device  :4;
};

union Morpee
{
 struct   Memee     DevAdd;
          int       Address;
          char      Byte[4];
};

union Morpee EEPROM;
int Address=0;
int Value;
char InStr[512];
char OutStr[512];


void i2c_time_delay(int ticks)
{
 WAIT(ticks);
}

void i2c_sda_input(void)
{
 DIRA &= SDA_Lo;
}

void i2c_sda_output(void)
{
 DIRA |= SDA_Hi;
}

void i2c_init(void)
{
 OUTA |= SCL_Hi;                   // SCL=Hi
 OUTA |= SDA_Hi;                   // SDA=Hi
 DIRA |= SCL_Hi;                   // Set SCL As Output
 DIRA |= SDA_Hi;                   // Set SDA As Output
}

void i2c_scl_hi(void)
{
 DIRA |= SCL_Hi;
 OUTA |= SCL_Hi;
}

void i2c_scl_lo(void)
{
 DIRA |= SCL_Hi;
 OUTA &= SCL_Lo;
}

void i2c_sda_hi(void)
{
 DIRA |= SDA_Hi;
 OUTA |= SDA_Hi;
}

void i2c_sda_lo(void)
{
 DIRA |= SDA_Hi;
 OUTA &= SDA_Lo;
}

void i2c_start(void)
{
 i2c_sda_lo();                     // SDA=Lo
 i2c_time_delay(528);
 i2c_scl_lo();                     // SCL=Lo
}

void i2c_stop(void)
{
 i2c_scl_hi();                     // SCL=Hi
 i2c_time_delay(528);
 i2c_sda_hi();                     // SDA=Hi
}

void i2c_reset(void)
{
 char index;
 i2c_start();
 for(index=0; index<9; index++)
  {
   i2c_scl_hi();
   i2c_time_delay(528);
   i2c_scl_lo();
   i2c_time_delay(528);
  }
 i2c_start();
 i2c_stop();
}

char i2c_byte_write(char Byte)
{
 char result;
 char index;
 char mask=0x80;
 for(index=0; index<8; index++)
  {
   result=mask & Byte;
   if(result > 0) i2c_sda_hi();
   else i2c_sda_lo();
   mask=mask >> 1;
   i2c_time_delay(264);
   i2c_scl_hi();
   i2c_time_delay(264);
   i2c_scl_lo();
   if(index == 7) i2c_sda_input();
   i2c_time_delay(264);
  }
 i2c_time_delay(132);
 i2c_scl_hi();
 i2c_time_delay(396);
 result=(char) ((INA & SDA_Hi) >> SDA_Pin); 
 i2c_scl_lo();
 if(result == 0) i2c_sda_lo();
 i2c_time_delay(264);
 return(result);
}

char EEWrite(void)
{
 int Index;
 char HiMem;
 EEPROM.Address=Address;
 EEPROM.DevAdd.Device=0x0a;
 EEPROM.DevAdd.RW=0;
 EEPROM.DevAdd.E16=EEPROM.DevAdd.A16;
 EEPROM.DevAdd.E17=EEPROM.DevAdd.A17;
 EEPROM.DevAdd.E18=EEPROM.DevAdd.A18;
 HiMem=(EEPROM.DevAdd.E18 << 2) | (EEPROM.DevAdd.E17 << 1) | EEPROM.DevAdd.E16;
 i2c_start();
 if(i2c_byte_write(EEPROM.Byte[3]) != 0x00) return('N');
 if(i2c_byte_write(EEPROM.Byte[1]) != 0x00) return('N');
 if(i2c_byte_write(EEPROM.Byte[0]) != 0x00) return('N');
 for(Index=0; Index<Value; Index++) if(i2c_byte_write(InStr[Index]) != 0x00) return('N');
 i2c_stop();
 return('Y');
}

void main(void)
{
 int InPut;
 char AorD;
 char RxD;
 char TxD;
 char Left;
 char Right;
 char Data;
 char Flip=0;
 i2c_init();
 for(;;)
  {
   InPut=tty_rxcheck();
   if(InPut == -1) continue;
   RxD=(char) InPut;
   switch(RxD)
    {
     case '0':
     case '1':
     case '2':
     case '3':
     case '4':
     case '5':
     case '6':
     case '7':
     case '8':
     case '9':
     case 'A':
     case 'B':
     case 'C':
     case 'D':
     case 'E':
     case 'F': TxD=RxD;
               if(RxD >= 'A') RxD=RxD - 'A' + 0x0a;
               else RxD=RxD - '0';
               switch(AorD)
                {
                 case 1: Address=(Address << 4) | RxD;
                         break;
                 case 0: switch(Flip)
                          {
                           case 0: Data=RxD << 4;
                                   Flip=1;
                                   break;
                           case 1: Data=Data | RxD;
                                   InStr[Value++]=Data;
                                   Flip=0;
                                   break;
                          }
                         break;
                 }
               tty_tx(TxD);
               break;
     case 'M': AorD=1;
               Address=0;
               tty_tx('M');
               break;
     case 'L': AorD=0;
               Flip=0;
               Value=0;
               tty_tx('L');
               break;
     case 'W': if(EEWrite() == 'Y') tty_tx('W');
               else tty_tx('T');
               break;
     case '?': tty_tx('P');
     default:  break;
    }
  }
}

So from the Windows command line I would enter Upload test.binary to send the file to the Prop1. The UpLoad program would then invoke your payload program which would send the propload.binary file to the prop. Upon completion, Upload would then establish contact with the uploaded propload program and commence the test.binary file upload. It worked, but was painfully slow.

If I knew how payload communicates with the secondary loader then maybe I can write a C program compiled by Catalina to replace the eeprom.binary secondary loader with one maybe called myeeprom.binary. Then I could upload test.binary to the Prop1 using payload myeeprom test.binary instead of the convoluted process used by PropLoad.c and UpLoad.c

RossH · 2023-07-24 07:26

@Wingineer19 said:
If I knew how payload communicates with the secondary loader then maybe I can write a C program compiled by Catalina to replace the eeprom.binary secondary loader with one maybe called myeeprom.binary. Then I could upload test.binary to the Prop1 using payload myeeprom test.binary instead of the convoluted process used by PropLoad.c and UpLoad.c

It is a very simple protocol. See the section titled "A Note about the Catalina Loader Protocol" in the Catalina Reference Manual for the Propeller 1. On or about page 157.

RossH · 2023-07-25 08:59

Hello @Wingineer19

I realized my description of the Catalina secondary loader protocol is a little out of date. Two things have changed since I wrote that:

The upper 8 bits of the address field contains the CPU for which the packet is intended. The address of the packet is in the lower 24 bits only.
The first packet sent is always a dummy packet with address 0, to verify that a secondary loader is loaded and executing.

Apologies. Below is a complete secondary loader example written in C, which you can modify to suit your needs:

Ross.

/******************************************************************************
 *                                                                            *
 *              Catalina Secondary Loader Example Program                     *
 *                                                                            *
 * This program demonstrates a Catalina secondary loader implemented in C.    *
 * It accepts, acknowledges and decodes the packets sent by the payload       *
 * loader when loading second and subsequent files specified on the command   *
 * line (the first file uses the Parallax protocol, unless the payload -x     *
 * option is specified) but it does nothing with the packets.                 *
 *                                                                            *
 * To see the diagnostic messages generated from this program, you must use a *
 * non-serial HMI option, such as VGA or TV. If you do not have a non-serial  *
 * HMI option, you should instead use NO_HMI                                  *
 *                                                                            *
 * You can load any binary file as the second file to be loaded, but the      *
 * examples below use the same binary for both the first and second load.     * 
 *                                                                            *
 * On the Propeller 1, you must use the tty256 serial library (not tty        *
 * because it does not have a large enough buffer), so compile                *
 * this program with a command like:                                          *
 *                                                                            *
 *    catalina load.c -lci -ltty256 -C VGA                                    *
 * or                                                                         *
 *    catalina load.c -lci -ltty256 -C NO_HMI                                 *
 *                                                                            *
 * Then test it using a payload command like:                                 *
 *                                                                            *
 *    payload load.binary load.binary -d                                      *
 *                                                                            *
 * The payload -d option causes it to display progress messages, including    *
 * the address of each packet as it is loaded.                                *
 *                                                                            *
 * On the Propeller 2, you can use the 2-port serial library, so compile      *
 * this program with a command like:                                          *
 *                                                                            *
 *    catalina -p2 load.c -lci -lserial2 -C VGA                               *
 * or                                                                         *
 *    catalina -p2 load.c -lci -lserial2 -C NO_HMI                            *
 *                                                                            *
 * Then test it using a payload command like:                                 *
 *                                                                            *
 *    payload load.bin load.bin -d                                            *
 *                                                                            *
 ******************************************************************************/

#include <stdio.h>
#include <stdint.h>

#define CPU             1             // Payload defaults to CPU 1
#define TIMEOUT         1000          // read timeout
#define MAX_SIZE        512           // Payload uses a 512 byte packet size
#define SIO_EOP         0x00FFFFFE    // end of packets marker

#ifdef __CATALINA_P2
// On the Propeller 2, we can use the 2-port serial plugin
#include <catalina_serial2.h>
#define PORT 0
#define tx(ch)        s2_tx(PORT, ch)
#define rx(timeout)   s2_rxtime(PORT, timeout);
#else
// On the Propeller 1, we must use the tty256 plugin
#include <catalina_tty.h>
#define tx(ch)        tty_tx(ch)
#define rx(timeout)   tty_rxtime(timeout);
#endif

int diagnose = 1;      // 0 = no diagnostics, increment for more diagnostics

// ReadByte - read a byte, unstuffing if necessary
//            returns 0 .. 0xFF on success, -1 on timeout, -2 on sync
int ReadByte(int cpu) {
   int result = -1;
   int i;

   static int save_data = 0;

   if (save_data & 0x100) {
      if (save_data != 0x1ff) {
        result = save_data & 0xff;
        save_data = 0;
        if (diagnose > 1) {
           printf("Read byte 0x%02x\n", result);
        }
        return result;
      }
   }

   while (1) {
      for (i = 0; i < 5; i++) {
         result = rx(TIMEOUT);
         if (diagnose > 2) {
            printf("Read result %d\n", result);
         }
         if (result != -1) {
            break;
         }
      }
      if (result < 0) {
         if (diagnose > 0) {
            printf("Read result %d\n", result);
         }
         return result;
      }
      if (save_data == 0x1ff) {
        if (result == cpu) {
           if (diagnose > 1) {
              printf("Read sync for cpu %d\n", cpu);
           }
           return -2;
        }
        if (result == 0) {
           result = 0xff;
           save_data = 0;
        }
        else {
           save_data = result | 0x100;
           result = 0xff;
        }
        if (diagnose > 1) {
           printf("[%02x] ", result);
        }
        return result;
      }
      else {
         if (result == 0xff) {
            save_data = 0x1ff;
         }
         else {
            save_data = 0;
            if (diagnose > 1) {
               printf("[%02x] ", result);
            }
            return result;
         }
      }
   }
   return result;
}

// ReadSync - returns 1 when sync signal read (note no unstuffing!)
int ReadSync(int cpu) {
   int result = 0;

   while (1) {
      result = rx(TIMEOUT);
      //if (result < 0) {
         //return result;
      //}
      if (result == 0xff) {
         result = rx(TIMEOUT);
         if (result == cpu) {
            return 1;
         }
      }
   }
   return result;
}

// WriteSync - write a sync (note no stuffing!)
void WriteSync(uint8_t cpu) {
   tx(0xff);
   tx(cpu);
}

// ReadLong - read a long value, unstuffing if necessary
//            updates long value l
//            returns 1 on success, -1 on timeout, -2 on sync
int ReadLong(uint32_t *l) {
   int i;
   int b;
   uint32_t tmp = 0;

   for (i = 0; i < 4; i++) {
      b = ReadByte(CPU);
      if (b < 0) {
         return b;
      }
      else {
         tmp |= (b & 0xFF) << (i * 8);
      }
   }
   *l = tmp;
   return 1;
}

// WriteByte - write a byte value, stuffing if necessary
void WriteByte(uint8_t b) {
   if (diagnose > 1) {
      printf("<0x%02x> ", b);
   }
   if (b == 0xff) {
      tx(b);
      tx(0);
   }
   else {
      tx(b);
   }
}

// WriteLong - write a long value, stuffing if necessary
void WriteLong(unsigned long l) {
   int i;

   for (i = 0; i < 4; i++) {
      WriteByte(l & 0xff);
      l >>= 8;
   }
}

// WritePacket - write a packet, stuffing if necessary
void WritePacket(uint8_t *buff, uint32_t addr, int size, uint8_t cpu) {
   int i;

   if (diagnose > 1) {
      printf("Sending Buffer %08lX\n", addr);
   }
   WriteLong((cpu << 24) | addr);
   WriteLong(size);
   for (i = 0; i < size; i++) {
      WriteByte(buff[i]);
   }
}


// ReadPacket - read a packet, unstuffing if necessary
//              updates values in addr, size and cpu
//              returns cpu on success, 0 on wrong cpu, -1 on other error
int ReadPacket(uint8_t *buff, uint32_t *addr, uint32_t *size, uint8_t *cpu) {
   uint32_t cpu_addr = 0;
   int b;
   int i;

   if (diagnose > 1) {
      printf("Receiving Buffer\n");
   }
   ReadLong(&cpu_addr);
   *addr = cpu_addr & 0xFFFFFF;
   *cpu = cpu_addr >> 24;
   ReadLong(size);
   if (diagnose > 1) {
      printf("CPU %d, addr %08lX, size %d\n", *cpu, *addr, *size);
   }
   for (i = 0; i < *size; i++) {
      b = ReadByte(CPU);
      if (b < 0) {
         return -1;
      }
      if (i < MAX_SIZE) {
        buff[i] = b;
      }
      else {
         if (diagnose > 0) {
            printf("Buffer Overflow!\n");
         }
      }
   }
   if (*cpu != CPU) {
      return 0;
   }
   if (*size > MAX_SIZE) {
      return -1;
   }
   return *cpu;
}


// LRC - calculate LRC of buffer
uint8_t LRC(uint8_t *buff, int size) {
   int i;
   uint8_t result = 0;

   for (i = 0; i < size; i++) {
      result ^= buff[i];
   }
   return result;
}

void main() {
   int result;
   uint32_t addr;
   uint32_t size;
   uint8_t cpu;
   uint8_t lrc;
   uint8_t buffer[MAX_SIZE];

   if (diagnose > 0) {
      printf("Secondary Loader\n");
   }

   // wait for our sync signal ...
   if (diagnose > 0) {
      printf("Awaiting Sync\n");
   }
   ReadSync(CPU);
   if (diagnose > 0) {
      printf("Received Sync\n");
   }

   // the first packet is a dummy, used by payload
   // to verify that a secondary loader is executing
   result = ReadPacket(buffer, &addr, &size, &cpu);
   if (result > 0) {
      lrc = LRC(buffer, size);
      if (diagnose > 1) {
         printf("Received Dummy Buffer, LRC = %02x\n", lrc);
      }
      WriteSync(CPU);
      WriteByte(lrc);
   }

   // now read real packets until SIO_EOP detected
   while ((result > 0) && (addr != SIO_EOP)) {
      result = ReadPacket(buffer, &addr, &size, &cpu);
      if ((result > 0) && (addr != SIO_EOP)) {
         lrc = LRC(buffer, size);
         if (diagnose > 0) {
            printf("Received Buffer, Addr = %06X\n", addr);
         }
         WriteSync(CPU);
         WriteByte(lrc);
      }

      /************************************************
       *                                              *
       *  insert code to process each packet here!!!  *
       *                                              *
       ************************************************/

   }

#ifndef __CATALINA_NO_HMI
   if (diagnose > 0) {
      printf("Press a key to exit\n");
      k_wait();
   }
#endif

}

EDIT: minor update - use catalina_tty.h and catalina_serial2.h in place of tty.h and serial2.h, and refer to "packets" rather than "pages" or "buffers" for consistency.

Wingineer19 · 2023-08-02 15:23

Hi RossH,

I've been busy working on other projects recently but just copied your Secondary Downloader Example and will examine it in detail as soon as possible. Thanks.

Wingineer19 · 2023-08-03 03:31

Hi RossH,

Well, it looks like I'm having problems again moving Structures around using the non-cached XMM memory, but only when I'm attempting to use the Multi-Memory-Model.

In this case, I'm using the non-cached LARGE memory model in the Primary code while using the COMPACT memory model in the Secondary. Unlike before, this problem also afflicts the non-cached SMALL memory model. But in each case the problem only surfaced when using the Multi-Memory-Model, and so far only when I try copying Structures within the Primary.

For this test, whenever the Primary wishes to output a particular Menu Item, it places the contents into a string within the Shared Structure and then sets a flag. The Secondary then grabs the string and outputs it to the Serial Port. Upon completion, the Secondary clears the flag. The Primary then grabs the next Menu Item, places it into the shared string, and the process repeats.

Please note that I've haven't encountered any problem with the Shared Structure. The problem is with a Data Structure confined exclusively to the Primary (i.e. not shared with the Secondary), and moving Structure contents from one element to another will cause a program lockup. Exact same problem you addressed previously by releasing a new XMM Kernel. Again, the problem only surfaced once more when using the Multi-Memory-Model.

I can email the code if you have the time and/or inclination to take a look at it. Let me know.

RossH · 2023-08-03 05:36

@Wingineer19 said:

I can email the code if you have the time and/or inclination to take a look at it. Let me know.

Yes, you can email me the code directly or post it here. It is possible it is a memory allocation or a stack space problem, so make sure to include the exact Catalina commands you use to build the program, including any memory allocation parameters (i.e. any -P and -R values you use).

Ross.

Wingineer19 · 2023-08-03 06:39

Hi RossH,

OK, I packed the entire program into an email and sent it to you.

Hopefully the code contains some good clues as to what is going on.

Thanks for taking a look at it.

It's late here (or really early depending upon how you want to look at it), so I'm going to call it a night (or early morning)

RossH · 2023-08-03 08:41

Hi @Wingineer19

Got your code. Found an issue that might be causing your problem. See my email.

Ross.

RossH · 2023-08-12 05:13

Catalina 6.0.1 has been released here.

The main purpose of this release is to add the self-hosted version of Catalina to a supported release (Catalina 6.0 was a beta release).

Self-hosted C development is a component of the Catalyst operating system for the Propeller. Self-hosted C development requires a Propeller 2 platform with at least 16MB of PSRAM. This currently includes the P2 Edge with onboard PSRAM (i.e. the P2-EC32MB) or a P2 Edge or P2 Evaluation board equipped with the HyperFlash/HyperRAM add-on. If you do not have one of these platforms, or do not want to try the self-hosting C development capability, there is no compelling reason to upgrade from Catalina 5.9.3.

Note that the following pre-built versions of Catalina for the P2_EVAL and P2_EDGE all include the self-hosted version of Catalina:

P2_EVAL.ZIP - Built for a P2 Evaluation board with the HyperRAM add-on and a serial interface.
P2_EVAL_VGA.ZIP - Built for a P2 Evaluation board with the HyperRAM add-on and a VGA interface and USB keyboard.
P2_EDGE.ZIP - Built for a P2 Edge board with 32Mb PSRAM on board and a serial interface.
P2_EDGE_VGA.ZIP - Built for a P2 Edge board with 32Mb PSRAM on board and a VGA interface and USB keyboard.

The following pre-built version, which is intended to be used on any P2 Edge or P2 Evaluation board even if it does not have PSRAM, does not include the self-hosted version of Catalina:

P2_DEMO.ZIP - Built for any P2 Edge or P2 Evaluation board (with or without PSRAM or HyperRAM) with a serial interface.

Because the Catalina directory structure changed significantly in version 6.0, it is recommended that you do NOT install any version 6 release over any version 5 (or earlier) release. Either uninstall the previous version, or install this version to a different location.

Here are the relevant extracts from the README.TXT since Catalina 5.9.3 (note that Catalina 5.9.4 was not released separately, but has instead been incorporated into this release):

RELEASE 6.0.1

New Functionality
-----------------

1. Catalina now has the option of caching SD card sector reads and writes 
   when PSRAM or HYPER RAM is available. The cache supports two modes:

      Write Through    if the sector is in the cache then it is used for
                       reads, but all writes are done both to the cache 
                       and to the SD card. This is the default mode.

      Write Back       if the sector is in the cache then it is used for
                       reads, and all writes are done only to the cache,
                       with the sector in the cache marked as dirty. To 
                       write all the dirty sectors in the cache back to the 
                       SD card, an explicit cache flush must be performed. 
                       It is fine not to flush the cache if the cache has 
                       not been filled, but once it has been filled then 
                       any sectors not cached will be written directly to 
                       the SD card, and so if the cache is NOT subsequently 
                       flushed then the SD card can end up in a corrupt 
                       or inconsistent state.

   The cache is supported only on the P2_EDGE and P2_EVAL boards, and is 
   enabled by compiling the library with the PSRAM or HYPER option specified 
   and then linking the programs that use it with both the extended C library
   and the appropriate PSRAM or HYPER library (i.e. -lcx or -lcix and -lpsram 
   or -lhyper). Since all programs linked with the library built to use the 
   cache will HAVE to be linked this way, it is recommended that the general
   version of the library NOT enable the cache, but that a separate version
   of the library be compiled to use with this option. For an example, see the 
   build_psram and build_hyper scripts in the source/lib folder, and how
   these are used by the build_all script in demos/catalyst/catalina.

   The P2_EDGE board with onboard PSRAM has 32MB of PSRAM, so Catalina uses 
   the lower 16MB as XMM RAM, and the upper 16MB as SD card cache. Catalina
   can only use 16MB of XMM RAM, but if some of the PSRAM is required for 
   other purposes, the size of the SD card cache can be adjusted - see 
   source/lib/include/sd_cache.h.

   The HyperFlash/HyperRAM add-on board has 16MB of PSRAM, so by default
   Catalina uses the lower 12MB as XMM RAM, and the upper 4MB as SD card 
   cache. The amount used for the SD card cache can be adjusted - see 
   source/lib/include/sd_cache.h. Note that no checking is done on the use
   of XMM RAM - so if the program uses XMM RAM above the 12MB (e.g. if it
   allocates too much heap space), then it will overwrite the cache. In
   that case, it is better to not use the cache at all.

   The SD card cache is currently only used by the self-hosted version of
   Catalina. When compiled as a Catalyst demo, the following catalina-related
   programs use the cache (in Write Back mode) to significantly improve SD 
   card performance:

      cpp
      rcc
      bcc
      spp
      pstrip
      p2asm

   Using the cache can reduce the time required for the self-hosted version 
   of Catalina to compile a C program by 20% to 33%.

   For details of the functions supported by the cache, see the file 
   source\lib\include\sd_cache.h

Other Changes
-------------

1. The reduction in SD card driver delay time made in release 5.9.2 has been 
   reverted because it did not work reliably on all SD cards, some of which 
   appear to require the longer delay. However, this change only made a 
   small improvement in very SD card intensive programs (such as catalina
   itself) when compared to the new sd card cache option.

RELEASE 6.0

New Functionality
-----------------

1. There is a new Catalyst demo - Catalina itself! The demos\catalyst folder
   has a new subdirectory called 'catalina', in which a self-hosting version
   of Catalina can be built. Since it is supported only on a limited number
   of Propeller 2 platforms, this new demo is not built by default when you
   build Catalyst. If you have a P2 EDGE board with PSRAM or a P2 EVAL board
   with HyperRAM then you can build it manually - go to the 'catalina' 
   subdirectory, and use the build_all script with the same options you
   used to build Catalyst. For example

      cd catalina
      build_all P2_EDGE SIMPLE VT100 CR_ON_LF USE_COLOR OPTIMIZE MHZ_200
   or
      cd catalina
      build_all P2_EVAL VGA COLOR_4 OPTIMIZE MHZ_200

   Note that the build_all script will automatically use the copy_all script
   to copy the results to the Catalyst image folder.

   The self-hosted version of Catalina currently has the following limitations:

      - it supports the Propeller 2 only.
      - it supports the TINY, COMPACT and NATIVE modes only.
      - it does not support the Catalina debugger, optimizer or parallelizer.

   Since the self-hosting version of catalina introduces a 'bin' directory to 
   hold the catalina executables, the build_all and copy_all scripts now put 
   their output in a directory called 'image' instead of 'bin'. See the
   Catalyst Reference Manual for more details on the self-hosted version of 
   Catalina.

2. Local user libraries must now be created in a local subdirectory called
   'lib'. For example, if you specify -liberty on the command line, previous 
   versions of Catalina would first look for a local library in a subdirectory
   in the current directory called libiberty, but now Catalina will look for a
   it in a local subdirectory called lib\iberty.

3. A new Catalina symbol P2_REV_A can now be defined (e.g. using -C P2_REV_A
   to the catalina command) to indicate that p2asm should not generate
   instructions not supported by the Rev A version of the Propeller 2.
   Previously, the p2_asm batch script had to be edited to do this. Note
   that this applies only to the Catalina compiler itself, which no longer 
   uses the p2_asm script. The p2_asm script must still be manually edited 
   (to remove the -v33 option to p2asm) if the Catalina Optimizer is used.

RELEASE 5.9.4

New Functionality
-----------------

1. Support for the DUALSRAM add-on XMM board for the Propeller 1. See the
   file DUALSRAM_README.TXT in the Propeller 1 target directory for
   more details.

2. A new demo program showing how to implement a Catalina secondary loader in
   C for use with the payload loader has been added. See the README.TXT file
   in the folder demos\loader for more details. Supported on both the Propeller
   1 and 2.

Ross.

Addendum: There is a typo in the README.TXT file - in places it refers to 14MB used as XMM RAM on the HYPER add-on board - this should be 12MB. When the SD card cache is used with the HyperRAM board, 12MB of the available 16MB is used as XMM RAM, with 4MB used as the SD cache. This is configurable.

RossH · 2023-08-15 07:01

Attached is a small errata file for Catalina release 6.0.1. Or you can download it from the usual place here.

Just unzip it over an existing Windows or Linux installation of Catalina 6.0.1.

No functional changes. The main reason for it is that it contains an updated Catalyst Reference Manual, which documents some of the recent functional changes, and also the important point that you need an SD Card formatted as FAT32 and with a 32kb cluster size to be able to use the self-hosting version of Catalina.

Ross.

RossH · 2023-08-24 00:43

All

Apparently I introduced a bug in recent versions of Catalina - the more accurate timing functions (_waitsec, _waitms, _waitus) may not work on the Propeller 1 for XMM programs (i.e. in SMALL or LARGE mode). Programs that use them may just hang.

They are ok in non-XMM modes (i.e. TINY or COMPACT) on the Propeller 1, and ok in all modes (i.e. TINY, COMPACT, NATIVE, SMALL, LARGE) on the Propeller 2.

I am testing a fix and will post an update soon.

If you need a workaround in the meantime, you can simulate the previous timing behavior using the WAIT macro. For example:

#define old_waitsec(secs) {if (secs>0) {WAIT(secs*_clockfreq());}}
#define old_waitms(msecs) {if (msecs>0) {WAIT(msecs*(_clockfreq()/1000));}}
#define old_waitus(usecs) {if (usecs>=150) {WAIT(usecs*(_clockfreq()/1000000));}}

The value of 150 used above in 'old_waitus' represents the smallest wait time that does not "hang" - it is based on trial and error. The value of 150 works in all modes on my C3. The problem is that this value depends on the speed of the XMM RAM - getting it correct is one thing the newer versions tried to automate

Ross.

RossH · 2023-09-03 12:24

Just a progress update, in case anyone was wondering ...

I've fixed the bug in the timing functions, but what has taken me so long has been just how difficult it is to get anything like an accurate timing function that works in all the huge number of program variants that Catalina now supports:

2 different processor architectures
5 different memory models
8 different cache options
2 threading options
2 interrupt options
12 different XMM memory implementations

I have finally decided that getting microsecond accurate timing of a possibly multi-threaded, interrupt-driven, XMM memory-based program that uses a cache (which means that even the timing function itself may be spread across multiple cached pages and require several XMM page access to execute) is so insanely difficult that it is simply not worth spending any more time on (pun intended!). If the timing works accurately on entirely hub-resident programs, and does not do crazy things in the other cases, I am prepared to call it a win! Use Catalina's multi-model support to put the time critical parts of the program into a Hub RAM resident component that uses dedicated cogs, and the rest of the program can then happily execute at slower speeds and with less accurate timing from XMM RAM if necessary.

The rest of my time recently has been spent in reviewing all the many dozens of Catalina demo programs, and making sure they all work properly. Some bugs had crept into recent releases, mostly to do with the Propeller 1 support (which had not been receiving the attention it deserved in recent releases). But the next release - 6.1 - will be a much more solid and robust release, and will become the recommended release for both Propeller 1 and Propeller 2 users.

Ross.

RossH · 2023-09-05 07:28

Catalina 6.1 has been released here.

The main purpose of this release is to fix issues with the timer functions and improve support for inline PASM. This release also addresses quite a few minor issues with the Propeller 1 support, and so it is strongly recommended that both Propeller 1 users and Propeller 2 users update to this release.

Here are the relevant extracts from the README.TXT since Catalina 6.0.1:

RELEASE 6.1

New Functionality
-----------------

1. The inline PASM capabilities have been enhanced by the addition of
   a new _PASM() macro, which simplifies the use of C identifiers in 
   inline PASM strings used as arguments to the PASM() function. New 
   demo programs have been added to demos\inline_pasm to illustrate this, 
   and all the existing demo programs have been updated as well. Also, 
   all the inline PASM demo programs now work "as is" on the C3, HYDRA, 
   P2_EVAL and P2_EDGE platforms. The Catalina Reference Manual has also 
   been updated with more information about inline PASM.

2. The 'build_utilities' script now has the option of copying the utilities
   it builds to the current directory as well as to Catalina's bin directory.
   If neither of these options is selected, the utilities are built and left 
   only in the utilities directory itself. Recall that the script will use
   the user's utilities folder if one exists in the user's home directory, 
   otherwise it will use the one specified by the directory in the LCCDIR 
   environment variable.

3. The accuracy of the following timer functions has been improved:
      _waitus()
      _waitms()
      _waitsec()
      _iwaitus()
      _iwaitms()
      _iwaitsec()

   A new demo program has been added to the demos\examples folder - the
   ex_timers.c program will show the accuracy of all the timer functions.

   Note that the these functions are most accurate for programs executed 
   entirely from Hub RAM (i.e. TINY, COMPACT or NATIVE programs) and when 
   neither multi-threading nor interrupts are used. They are less accurate 
   when used in multi-threaded, interrupt-driven, or SMALL or LARGE XMM 
   programs, especially when the cache has to be used and the cache size is 
   small. Catalina's multi-model support can be used to place time-critical 
   sections of code entirely in Hub RAM and use dedicated cogs. Other parts
   of the programs can be executed from XMM RAM.

2. The Comms VT100 terminal emulator has a new option added to the Edit
   menu - Clear Buffer, which clears the entire buffer and resets the
   cursor, screen and view to the beginning of the buffer.

Other Changes
-------------

1. A bug that may have caused the _waitus(), _waitms() and _waitsec() 
   timer functions to hang for up to 53 seconds has been fixed. Affected 
   only the Propeller 1.

2. Some programs built in COMPACT mode were not working correctly, such
   as demos\graphics\graphics_demo.c - this was due to an error in the
   catalina_compact.inc file.  Affected only the Propeller 1.

3. The Catalina blackbox debugger was not working on programs in COMPACT 
   mode due to an error in the file catalina_compact.inc. Affected only 
   the Propeller 1.

4. The file pthread.h should have included the file rtc.h in order to 
   correctly define the rtc_settime() function. Affected both the 
   Propeller 1 and 2.

5. The examples/ex_leds_2.c demo program was using the wrong value for a 
   LED on the P2_EDGE board. Affected only the Propeller 2.

6. Various demo programs have been tidied up. Many minor issues have been 
   corrected. More significant issues have been addressed in the following 
   demo programs:

   - demos/games/chimaera.c was not being compiled with the extended
     C library, so the logging function was not working. Affected both
     the Propeller 1 and 2.

   - demos/benchmark/ackerman_2.c still had a call to printf(), which 
     meant that instead of being smaller than ackerman_1.c, it ended up 
     larger. Affected both the Propeller 1 and 2.

   - demos/spinc/test_pasm.c now builds for both TINY and COMPACT mode.
     Also, the Makefile now warns correctly if these programs are built 
     for a Propeller 2 (spinc is specific to Propeller 1).

7. The Makefile in the demos\examples folder knows more about the options
   required to build particular programs, and which programs apply only to 
   the Propeller 1 or 2, which means less error messages are now generated 
   when the 'build_all' batch script is used. Affected both the Propeller 
   1 and 2.

I intend taking a short break from Catalina development. However, I will continue to monitor the forums, and will fix any issues discovered in this release.

Ross.

RossH · 2023-09-06 06:26

Not a bug, and strictly speaking not even Propeller-related, but ...

Catalina's multi-processing version of lua (i.e. mlua) uses the industry standard Posix threads library, and therefore should work on both the Propeller and on Windows (and on Linux for that matter, but I have not tried it).

However, there is a bug in the Windows Makefile that means it does not build correctly when compiled using mingw (the Windows version of gcc).

Attached is a modified Makefile. Not that this is not the file Catalina uses - it has its own Makefile (Makefile.Catalina) which is fine.

To compile mlua for Windows, you must have mingw installed. Then unzip the attached Makefile over the one in demos\catalyst\lua-5.4.4\src, (not demos\catalyst\lua-5.4.4) and then

cd demos\catalyst\lua-5.4.4
make clean mingw

Then all the Catalina mlua demos (ex1.lua .. ex12.lua) should also work under Windows - but note that the factory-based examples (ex9.lua and ex12.lua) will need the number of factories reduced from 4 to 1, because Windows supports only one factory, and the LED-flashing example (ex10.lua) won't do anything visible because there are no LEDs to flash!

Ross.

Wingineer19 · 2023-09-09 05:00

Hi RossH,

I'm continuing to experiment with the XMM mode on my FLiP while using the DUALSRAM module and driver. The XMM cog has been consigned to providing the operator interface with lots of Menus to accept user input, while several other cogs have been running in CMM mode performing the "workhorse" functions of WiFi interface and position estimation using multilateration as mentioned in our previous discussions. The next step is to interface to the GPS receiver but I'm not quite there yet.

Ideally, I would like to get the FLiP to do everything itself without requiring external hardware like my DUALSRAM module. I might be able to do this by moving all of the fixed Menu items into EEPROM then read them on the fly when required.

Since I probably won't be able to get everything to work by strictly running in CMM mode, I can continue running an XMM cog but have it use XEPROM mode. I'm not yet sure if the 64 kilobyte EEPROM of the FLiP will allow me to do everything that's required, but i think it's worth a try. I might have to replace the 64 kilobyte EEPROM with a 128 kilobyte one. Hopefully, someday, Parallax will offer a FLiP module with a 128 kilobyte EEPROM.

For this to work I must be able to read the Menu strings from the EEPROM while running in XEPROM mode. We've discussed this before with the thought being that we can use the XMM kernel to grab what is needed from EEPROM.

So, instead of attempting to read entire strings from EEPROM, I initially opted for a simple test to read some blocks of memory that just contained LONG values (actually 32 bit INTs). Sadly, it didn't work.

Below is a copy of the program, with the commentary included. I don't know why it's not working so I'm hoping you can provide some feedback as to why not. Thanks.

 //Program Is TestProg.c
//Last Revision On 08Sep23

//Uses A Prop1 FLiP Module and compiled using the following:
//catalina testprog.c -lc -lma -lserial4 -C FLIP -C NO_HMI -C XEPROM -C SMALL -C CACHED_1K

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdarg.h>
#include <math.h>
#include <time.h>
#include <propeller.h>

#define            ConPort              0

#define            A0I0                 0x0000C090
#define            A0I1                 0x0000C094
#define            A0I2                 0x0000C098
#define            A0I3                 0x0000C09C
#define            A0I4                 0x0000C0A0
#define            A0I5                 0x0000C0A4
#define            A0I6                 0x0000C0A8
#define            A0I7                 0x0000C0AC

/*
Commentary:

Program attempts to read the EEPROM when running in XEPROM mode by using this function:

unsigned long GetMemInt(unsigned long Address)
{
 PASM("mov RI, r2");
 PASM("jmp #RLNG");
 return PASM("mov r0, BC");
}

But it doesn't work!

Previously, the EEPROM was programmed with these values:

 contents of A0I0 actually are 0xc0b0
 contents of A0I1 actually are 0xc0c0
 contents of A0I2 actually are 0xc0e8
 contents of A0I3 actually are 0xc124
 contents of A0I4 actually are 0xc15c
 contents of A0I5 actually are 0xc180
 contents of A0I6 actually are 0xc1a4
 contents of A0I7 actually are 0xc1c8

And Confirmed By A Tachyon Dump Here:

$C090 $20 EE DUMP -->                                                      
0000.C090:   B0 C0 00 00  C0 C0 00 00  E8 C0 00 00  24 C1 00 00    ............$
...                                                                             
0000.C0A0:   5C C1 00 00  80 C1 00 00  A4 C1 00 00  C8 C1 00 00    \............
... ok                                                                          
...                                                                             

But the program always outputs this:

Address=C090,Contents=FFFF                                                      
Address=C094,Contents=FFFF                                                      
Address=C098,Contents=FFFF                                                      
Address=C09C,Contents=FFFF                                                      
Address=C0A0,Contents=FFFF                                                      
Address=C0A4,Contents=FFFF                                                      
Address=C0A8,Contents=FFFF                                                      
Address=C0AC,Contents=FFFF  

Instead of this:

Address=C090,Contents=C0B0                                                      
Address=C094,Contents=C0C0                                                      
Address=C098,Contents=C0E8                                                      
Address=C09C,Contents=C124                                                      
Address=C0A0,Contents=C15C                                                      
Address=C0A4,Contents=C180                                                      
Address=C0A8,Contents=C1A4                                                      
Address=C0AC,Contents=C1C8  


Recalling the 0x10 offset, I also tried this but it didn't work, either:

 result=GetMemInt(address-0x10);                                                                             

*/


unsigned long GetMemInt(unsigned long Address)
{
 PASM("mov RI, r2");
 PASM("jmp #RLNG");
 return PASM("mov r0, BC");
}

void main(void)
{
 unsigned long address=A0I0;
 unsigned long result;
 for(;;)
  {
   s4_str(ConPort,"Address=");
   s4_hex(ConPort,address,4);
   s4_str(ConPort,",");
   s4_str(ConPort,"Contents=");
   result=GetMemInt(address);
   s4_hex(ConPort,result,4);
   s4_str(ConPort,"\r\n");
   address=address + 0x04;
   if(address > A0I7)
    {
     address=A0I0;
     s4_str(ConPort,"\r\n");
     sleep(1);
    }
  }
}

RossH · 2023-09-10 12:59

Hello @Wingineer19

I'm not 100% sure what Tachyon does, or is showing you - but you were on the right track.

Here is a modified version of your program which I think does what you want. It prints the string data from the EEPROM (Note that the string is also in Hub RAM at the address specified, but we know it is reading the EEPROM version and not Hub RAM version because it uses RLNG, and RLNG in a SMALL program only reads from EEPROM).

Actually, to be precise, RLNG reads from the cache, which is in turn read from the EEPROM using the XMM access functions provided by XEPROM:

//Program Is TestProg.c

//Uses A Prop1 C3 Module and compiled using the following:
//
//  catalina testprog.c -lc -lserial4 -C C3 -C NO_HMI -C XEPROM -C SMALL -C CACHED_1K
//
//Loaded and executed with the following:
//
//  payload EEPROM testprog.binary -i -q1

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdarg.h>
#include <math.h>
#include <time.h>
#include <propeller.h>

#define ConPort 0

unsigned long GetMemInt(unsigned long Address)
{
 PASM("mov RI, r2");
 PASM("jmp #RLNG");
 return PASM("mov r0, BC");
}

// declare a string so it will end up in the EEPROM
static char my_string[] = "HELLO THERE!!!";

void main(void)
{
 unsigned long cs_base;
 unsigned long address;
 unsigned long result;
 int i;

 // get address of my_string
 address = (unsigned long)&my_string;
 s4_str(ConPort,"my_string is at ");
 s4_hex(ConPort,address>>16,4);
 s4_hex(ConPort,address,4);
 s4_str(ConPort,"\r\n");
 s4_str(ConPort,"\r\n");

 // print Hub RAM version of my_string
 s4_str(ConPort,"my_string=");
 s4_str(ConPort,my_string);
 s4_str(ConPort,"\r\n");

 // erase Hub RAM version of my_string
 // (EEPROM version will remain intact!)
 for (i = 0; i < strlen(my_string); i++) {
     my_string[i] = 0;
 }

 // print Hub RAM version of my_string again
 s4_str(ConPort,"my_string=");
 s4_str(ConPort,my_string);
 s4_str(ConPort,"\r\n");
 s4_str(ConPort,"\r\n");

 // now print EEPROM version of hub string
 while (1) {
   for (i = 0; i < 10; i++) {
     // read some data from cached EEPROM 
     s4_str(ConPort,"Address=");
     s4_hex(ConPort,address,4);
     s4_str(ConPort,",");
     s4_str(ConPort,"Contents=");
     result=GetMemInt(address);
     s4_hex(ConPort,result,2);
     s4_str(ConPort," ");
     s4_hex(ConPort,result>>8,2);
     s4_str(ConPort," ");
     s4_hex(ConPort,result>>16,2);
     s4_str(ConPort," ");
     s4_hex(ConPort,result>>24,2);
     s4_str(ConPort," ");
     s4_str(ConPort,"\r\n");
     address += 4;
    }
    s4_str(ConPort,"Press a key to continue\r\n");
    s4_rx();
 }
}

This works on my C3. Is this what you were trying to do?

Ross.

EDIT: Modified the program to make it clear that RLNG is reading from EEPROM, not Hub RAM.

Wingineer19 · 2023-09-10 23:58

Hi RossH,

Unfortunately I didn't provide a newer post to dissuade you from looking at this until I could provide an update. I see I was too late. Apologies.

OK, let me see if I can explain all of this without resorting to a long dissertation.

Previously, we discussed a way to store various strings within EEPROM but yet keep them safely away from being overwritten by the XEPROM program itself. You suggested just simply putting a Dummy variable in front of the strings which would effectively push them higher up into EEPROM memory. It worked!

Here's our friendly neighborhood menusonly.c program that does just that:

//Program Is Menusonly.c
//catalina menusonly.c -C FLIP -C NO_HMI -C EEPROM -y -M128k

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <math.h>
#include <propeller.h>

const char Dummy[16384]="\0";
const char M0I0[]="\x1b[1m\x1b[40m\x1b[2J";
const char M0I1[]="\x1b[6;30H\x1b[37;40mMiniPlate Participant";
const char M0I2[]="\x1b[8;20H\x1b[37;40mFirmWare Version 1.0, Rev Data: 14 Sep 22";
const char M0I3[]="\x1b[10;21H\x1b[37;40mCopyright(c) 2022, All Rights Reserved";
const char M0I4[]="\x1b[12;31H\x1b[37;40m(1) WiFi Main Menu";
const char M0I5[]="\x1b[14;31H\x1b[37;40m(2) GPS Main Menu ";
const char M0I6[]="\x1b[16;31H\x1b[37;40m(3) Diagnostics   ";
const char M0I7[]="\x1b[18;28H\x1b[37;40mMake Selection (1 To 3):";

void main(void)
{
}

We would then compile the menusonly.c program using this command line:

catalina menusonly.c -C FLIP -C NO_HMI -C EEPROM -y -M128k

This generated an assembly list file called menusonly.lst which shows not only the assembly instructions but also the address location of each string.

We then uploaded the menusonly.binary file to the EEPROM using this command:

payload eeprom menusonly

Next, I manually created a list of the EEPROM memory locations of each string by examining the menusonly.lst file and placed them into a series of #defines.

Finally, I could write a function that would grab each string from EEPROM and output it using the Catalina s4_str() output function. (Note: I'm using the s4 function because I will be using four serial ports. Otherwise, I could have used the tty or even the printf() or puts() functions).

Here's the code sample that outputs the strings when running in XEPROM mode:

#define            M0I0                 0x000040B0
#define            M0I1                 0x000040C0
#define            M0I2                 0x000040E8
#define            M0I3                 0x00004124
#define            M0I4                 0x0000415C
#define            M0I5                 0x00004180
#define            M0I6                 0x000041A4
#define            M0I7                 0x000041C8

char OutStr[256];

char GetMemChr(char *src,char *dst)
{
 PASM("mov RI, r3");
 PASM("jmp #RBYT");
 PASM("mov r0, BC");
 PASM("wrbyte r0, r2");
 return PASM("and r0, #$FF");
}

void GetStrVal(unsigned long Address)
{
 char *source;
 char *destiny;
 char ndex;
 char temp;
 source=(char *) Address;
 destiny=&OutStr[0];
 for(ndex=0; ndex<200; ndex++)
  {
   temp=GetMemChr(source++,destiny++);
   OutStr[ndex+1]=0x00;
  }
 s4_str(0,OutStr);
} 

void ShowMenuZero(void)
{
 unsigned long menulist[]={M0I0,M0I1,M0I2,M0I3,M0I4,M0I5,M0I6,M0I7};
 short index;
 for(index=0; index<8; index++)    GetStrVal(menulist[index]-0x10);
}

void main(void)
{
 for(;;) ShowMenuZero();
}

Notice in the function GetStrVal(menulist[index]-0x10) there's an offset of 0x10 that must be subtracted from the menu address locations. I'd forgotten about that, added it, but encountered some other issues making things worse (effectively hammering my program), which then prompted my previous post. Oops.

This is all well and good, but notice that in the ShowMenuZero(void) function I have this:

unsigned long menulist[]={M0I0,M0I1,M0I2,M0I3,M0I4,M0I5,M0I6,M0I7};

These are eight variables that must be created, and even though they are done on the Stack, the compiler must generate code to do that. When dealing with the Prop1 the objective is to minimize the use of memory.

Now suppose I modified menusonly.c to contain a lookup table containing eight long variables placed in front of the strings. Each long variable would contain the address of a corresponding string. Our menusonly.c program would now look like this:

//Program Is Menusonly.c
//catalina menusonly.c -C FLIP -C NO_HMI -C EEPROM -y -M128k

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <math.h>
#include <propeller.h>

const char Dummy[16384]="\0";
const int  A0I0=0x00;
const int  A0I1=0x00;
const int  A0I2=0x00;
const int  A0I3=0x00;
const int  A0I4=0x00;
const int  A0I5=0x00;
const int  A0I6=0x00;
const int  A0I7=0x00;
const char M0I0[]="\x1b[1m\x1b[40m\x1b[2J";
const char M0I1[]="\x1b[6;30H\x1b[37;40mMiniPlate Participant";
const char M0I2[]="\x1b[8;20H\x1b[37;40mFirmWare Version 1.0, Rev Data: 14 Sep 22";
const char M0I3[]="\x1b[10;21H\x1b[37;40mCopyright(c) 2022, All Rights Reserved";
const char M0I4[]="\x1b[12;31H\x1b[37;40m(1) WiFi Main Menu";
const char M0I5[]="\x1b[14;31H\x1b[37;40m(2) GPS Main Menu ";
const char M0I6[]="\x1b[16;31H\x1b[37;40m(3) Diagnostics   ";
const char M0I7[]="\x1b[18;28H\x1b[37;40mMake Selection (1 To 3):";

void main(void)
{
}

So variable A0I0 would contain the address of string M0I0, A0I1 of M0I1, A0I2 of M0I2 and so forth. But not initially since each variable is assigned a value of zero as shown.

We need to update the A0I0 to A0I7 variables with the actual address of M0I0 to M0I7, respectively. We compile the menusonly.c just like before to generate the menusonly.lst file.

I could manually parse the menusonly.lst file, find the memory locations of the M0I0 to M0I7 variables, edit the menusonly.c file to assign these string addresses to A0I0 to A0I7, recompile, then manually parse the menusonly.c again to finally generate my list of #defines for A0I0 to A0I7 and M0I0 to M0I7.

That's a lot of work to do. Alternatively, I could just write a menufix.cpp Windows program to do all of this for me, including updating the menusonly.c file with all of the correct values:

//Program Is Menufix.cpp
//Last Revision On 10Sep23

#include <stdio.h>
#include <stdlib.h>
#include <stdarg.h>
#include <ctype.h>
#include <conio.h>
#include <string.h>
#include <math.h>
#include <time.h>
#include <unistd.h>
#include <iostream.h>
#include <string>
#include <windows.h>

void main(int argc,char *argv[])
{
 FILE *asmfile;
 char *asmptr;
 char  asmname[40];
 char  asmindex[40];
 char  asmline[128];
 FILE *menufile;
 char *menuptr;
 char  menuline[128];
 FILE *codefile;
 char *codeptr;
 char  codename[40];
 char  codeline[128];
 char  codestra[128];
 char  codestrb[128];
 FILE *newfile;
 char  SysStr[40];
 long offset=0x8000;
 long asmaddress;
 if(argc < 3)
  {
   printf("Must Specify MemoryType And SourceFile!\n");
   return;
  }
 strupr(argv[1]);
 switch(argv[1][0])
  {
   case 'X': offset=0;
             break;
   case 'C': 
   case 'L': break;
   default:  printf("Invalid Memory Type!\n");
             return;
  }
 strcpy(asmname,argv[2]);
 asmptr=strstr(asmname,".");
 if(asmptr != NULL) snprintf(asmname,strlen(asmname) - strlen(asmptr) + 1,"%s",asmname);
 sprintf(codename,"%s.c",asmname);
 strcat(asmname,".lst");
 if((asmfile=fopen(asmname,"r")) == NULL)
  {
   printf("ERROR: Can't Find %s! Try Recompiling %s\n",asmname,codename);
   return;
  } 
 if((menufile=fopen("menulist.h","w")) == NULL)
  {
   printf("ERROR: Unable To Create menulist.h File!\n");
   return;
  }
 fprintf(menufile,"//Segment Is menufile.h\n");
 fprintf(menufile,"//Last Revision On 08Sep23\n");
 fprintf(menufile,"//Scans Catalina LST File And Determines Locations Of Variables Within Memory\n\n");
 while(fgets(asmline,100,asmfile) != NULL)
  {
   if(strstr(asmline,"<symbol:") == NULL) continue;
   if(strstr(asmline,"<symbol:main>") != NULL) continue;
   snprintf(asmindex,5,"%s",asmline);
   asmaddress=strtol(asmindex,NULL,16) + offset;
   asmptr=strstr(asmline,"<symbol:") + 8;
   strcpy(asmline,asmptr);
   asmptr=strstr(asmline,">");
   snprintf(asmline,strlen(asmline) - strlen(asmptr) + 1,"%s",asmline);
   while(strlen(asmline) < 20) strcat(asmline," ");
   fprintf(menufile,"#define            %s 0x%.8X\n",asmline,asmaddress);
  }
 fclose(asmfile);
 if((newfile=fopen("tempcode.c","w")) == NULL)
  {
   printf("ERROR: Unable To Open A Temporary File\n");
   return;
  }
 if((codefile=fopen(codename,"r")) == NULL)
  {
   printf("ERROR: Unable to Open %s!\n",codename);
   return;
  } 
 freopen("menulist.h","r",menufile);
 while(fgets(codeline,100,codefile) != NULL)
  {
   if(strstr(codeline,"const int") != NULL)
    {
     codeptr=strstr(codeline,"=");
     strncpy(codestra,codeline,strlen(codeline) - strlen(codeptr));
     codeptr=strstr(codestra,"A");
     strcpy(codestra,codeptr);
     strcpy(codestrb,codestra);
     codestrb[0]='M';
     while(fgets(menuline,100,menufile) != NULL)
      {
       if(strstr(menuline,codestrb) == NULL) continue;
       menuptr=strstr(menuline,"0x");
       asmaddress=strtol(menuptr,NULL,16);  
       fprintf(newfile,"const int  %s=0x%.8X;\n",codestra,asmaddress);
       break;
      }
    }
   else fprintf(newfile,"%s",codeline); 
  }
 fclose(newfile);
 fclose(codefile);
 fclose(menufile);
 sprintf(SysStr,"del /q %s\n",codename);
 system(SysStr);
 sprintf(SysStr,"rename tempcode.c %s\n",codename);
 system(SysStr);
 printf("**IMPORTANT** Don't Forget To Recompile %s Prior To Uploading To EEPROM!\n",codename);
}

Now we do this:
Step 1: Compile the above menusonly.c file with the unpopulated A0I0 to A0I7 variables:
catalina menusonly.c -C FLIP -C NO_HMI -C EEPROM -y -M128k

Step 2: Run the MenuFix program. (It updates the menusonly.c variables A0I0 to A0I7)
menufix xmm menusonly

Step 3: Recompile the menusonly.c file
catalina menusonly.c -C FLIP -C NO_HMI -C EEPROM -y -M128k

Step 4: Upload this newest menusonly.c file to EEPROM:
payload eeprom menusonly

Notice that the menufix.cpp program created a file called menulist.h:

#define            Dummy                0x00000090
#define            A0I0                 0x00004090
#define            A0I1                 0x00004094
#define            A0I2                 0x00004098
#define            A0I3                 0x0000409C
#define            A0I4                 0x000040A0
#define            A0I5                 0x000040A4
#define            A0I6                 0x000040A8
#define            A0I7                 0x000040AC
#define            M0I0                 0x000040B0
#define            M0I1                 0x000040C0
#define            M0I2                 0x000040E8
#define            M0I3                 0x00004124
#define            M0I4                 0x0000415C
#define            M0I5                 0x00004180
#define            M0I6                 0x000041A4
#define            M0I7                 0x000041C8

Now we can revisit the string output code:

#define XmmOffSet   0x10

#include "menulist.h"

char OutStr[256];

unsigned long GetLongVal(unsigned long address)
{
 PASM("mov RI, r2");
 PASM("jmp #RLNG");
 return PASM("mov r0, BC");
}

char GetMemChr(char *src,char *dst)
{
 PASM("mov RI, r3");
 PASM("jmp #RBYT");
 PASM("mov r0, BC");
 PASM("wrbyte r0, r2");
 return PASM("and r0, #$FF");
}

void GetStrVal(unsigned long Address)
{
 char *source;
 char *destiny;
 char ndex;
 char temp;
 source=(char *) Address;
 destiny=&OutStr[0];
 for(ndex=0; ndex<200; ndex++)
  {
   temp=GetMemChr(source++,destiny++);
   OutStr[ndex+1]=0x00;
  }
 s4_str(0,OutStr);
} 

void ShowMenuZero(void)
{
 unsigned long address=AOIO;
 unsigned long menuaddress;
 short index;
 for(index=0; index<8; index++) 
  {
   menuaddress=GetLongVal(address - XmmOffSet);
   GetStrVal(menuaddress - XmmOffSet);
   address=address + 4;
  }
}

void main(void)
{
 for(;;) ShowMenuZero();
}

So (hopefully) saving memory using this technique was the whole purpose of this exercise.

Unfortunately, I didn't completely chase down the bugs before hastily submitting my previous post.

Nevertheless, I appreciate your response and hope you find something of value and interest with what I'm doing here

RossH · 2023-09-11 00:52

@Wingineer19 said:

Nevertheless, I appreciate your response and hope you find something of value and interest with what I'm doing here

Yes, I do. Doing anything in C on the Propeller 1 that needs both speed and a significant code size is likely to require the use of the multi-model capability, since the Hub RAM is so limited but the XMM execution speed is so slow.

But as you have discovered, building multi-model programs can be complex - so I will look at your solution and see if it suggests any improvements I can make in Catalina.

Ross.

Wingineer19 · 2023-09-11 02:40

Hello RossH,

As we all know, one of the fastest way to consume memory on the Prop1, or any microcontroller for that matter, is to use the library printf() function. I've looked around on the Web and found some pretty good rewrites of it which drastically reduced the memory footprint. Unfortunately, many of the solutions don't support floating point, which in my case is essential.

So, I decided to attempt my own rewrite by examining what others have done and adding my own tweaks and refinements. It supports floating point but I noticed there is some loss in precision.

I was going to post this on a separate thread (and may yet do that) with a title like "The Great Printf() Challenge" and see what the code gurus can do with it. The objective is to make it even smaller, more efficient, and more accurate. The only prize I have to offer is gratitude and the admiration of other coders who need this type of function.

Anyway, I've listed it below. Feel free to do with it as you please. This is a public forum and I posted it with the hopes that others can find it useful without any licenses or restrictions. Hopefully you can make some refinements to it and maybe consider adding it as an option to Catalina.

//Segment Is Conprint.c
//Last Revision On 09Sep23
//Attempts To Replace The Library printf() function -- but in a much smaller package

#define ln10  2.302585093

char ConTxDStr[256];

char *int2asc(char *InStr,unsigned int value,char sign,char wide,char uplow,char base)
{
 unsigned int temp=value;
 unsigned int remain;
 char diff;
 char deep=1;
 char *iptr=InStr;
 if((base < 2) || (base > 16)) return InStr;
 if(sign != 0) *iptr++ = sign;
 if(value > 0) deep=(char) (log(value) / log(base)) + 1;
 iptr=iptr + deep;
 for(diff=deep; diff<wide; diff++) iptr++;
 InStr=iptr;
 *iptr-- = 0;
 do
  {
   temp=value;
   value=value / base;
   remain=temp - value*base;
   if(remain < 10) *iptr-- = remain + '0';  
   else *iptr-- = remain - 10 + ((uplow == 'X') ? 'A': 'a');
  }while(value);
 for(diff=deep; diff<wide; diff++) *iptr-- = '0';
 return(InStr);
}

float ipowten(short x)
{
 return(exp(x * ln10));
}

char *eftoa(char *InStr,float fval,short wide,short precise,char sign,char type)
{
 char *fptr=InStr;
 char fsign;
 float fexpo;
 float emult;
 float pmult;
 short expo;
 short index;
 short ival;
 short point=0;
 fsign=(fval < 0.0) ? -1 : (fval > 0.0) ? 1 : 0;
 switch(fsign)
  {
   case 0:  expo=0;
            *fptr++='0';
            *fptr++='.';
            for(index=0; index<precise; index++) *fptr++ = '0';
            if(type == 'e') goto makex;
            break; 
   case -1: fval=-fval;
            sign='-';
   case 1:  if(sign != 0) *fptr++ = sign;
            fexpo=log10(fval);
            expo=(short) fexpo;
            if(fexpo < 0.0) expo=expo - 1;           
            emult=ipowten(expo);
            pmult=ipowten(precise);
            fval=fval / emult;
            if(emult > 1.0) pmult=pmult * emult;
            fval=(fval * pmult + 0.5) / pmult;
            switch(type)
             {
              case 'f':  if(expo > 0)
                          {
                           precise=precise + expo;
                           point=expo;
                          }
                         else
                          {
                           fval=fval * emult;
                           expo=0;
                          }                        
                         for(index=0; index<(wide - expo - 1); index++) *fptr++ = '0';
              case 'e':  for(index=0; index<=precise; index++)
                          {
                           ival=(short) fval;
                           *fptr++ = '0' + ival;
                           fval=10.0 * (fval - (float) ival);
                           if(index == point) *fptr++ = '.';
                          }
                         if(type != 'e') break;
makex:                   *fptr++='E';
                         if(expo < 0)
                          {
                           *fptr++='-';
                           expo=-expo;
                          }
                         else *fptr++='+';
                         *fptr++=(expo / 10) + '0';
                         *fptr++=expo - (expo/10)*10 + '0';
                         break;
             }
           break;
  } 
 *fptr=0;
 return(fptr);
}

char *strprint(char *result,char *InStr,va_list valist)
{
 char *optr=result;
 char *nptr;
 char *sptr;
 char val;
 char sign=0;
 char flag=0;
 int wide=0;
 int deep=0;
 int sval;
 unsigned int uval;
 float fval;
 while((val=*InStr++) != 0x00)
  {
   switch(flag)
    {
     case 0:  switch(val)
               {
                case '%': wide=0;
                          flag=1;
                          break;
                default:  sign=0;
                          *optr++=val;
                          *optr=0;
                          break;
               }
              break;
     default: nptr=optr;
              switch(val)
               {
                case '%': *optr++=val;
                          *optr=0;
                          break;
                case '+': sign='+';
                          break;
                case 'b': uval=va_arg(valist,unsigned int);
                          optr=int2asc(optr,uval,0,wide,0,2);
                          break;
                case 'o': uval=va_arg(valist,unsigned int);
                          optr=int2asc(optr,uval,0,wide,0,8);
                          break;
                case 'c': val=(char) va_arg(valist,int);
                          *optr++=val;
                          *optr=0;
                          break;
                case 'd': sval=va_arg(valist,int);
                          if(sval < 0)
                           {
                            uval=-sval;
                            sign='-';
                           }
                          else uval=sval;
                          optr=int2asc(optr,uval,sign,wide,0,10);
                          break;
                case 'E': val='e';
                case 'e':
                case 'f': fval=(float) va_arg(valist,double);
                          if(fval < 0.0)
                           {
                            fval=-fval;
                            sign='-';
                           }
                          optr=eftoa(optr,fval,wide,deep,sign,val);
                          break;
                case 's': sptr=va_arg(valist,char*);
                          do
                           {
                            *optr++ = *sptr++;
                           }
                          while(*sptr != 0);
                          break;
                case 'u': uval=va_arg(valist,unsigned int);
                          optr=int2asc(optr,uval,0,wide,0,10);
                          break;
                case 'x':
                case 'X': uval=va_arg(valist,unsigned int);
                          optr=int2asc(optr,uval,0,wide,val,16);
                          break;
                case '.': deep=0;
                          flag=2;
                          break;
                default:  if(val < '0') break;
                          if(val > '9') break;
                          if(flag == 1) wide=wide * 10 + (val - '0');
                          else deep=deep * 10 + (val - '0');
                          break;
               }
              if(nptr == optr) break;
              nptr=optr;
              flag=0;
              break;
     }
   }
 va_end(valist);
 *optr=0;
 return(result);
}

void ConPrint(char *format,...)
{
 char *txd=ConTxDStr;
 va_list ptr;
 va_start(ptr,format);
 txd=strprint(txd,format,ptr);
 va_end(ptr);
 s4_str(0,ConTxDStr);
}

Wingineer19 · 2023-09-11 06:43

@RossH said:
...Doing anything in C on the Propeller 1 that needs both speed and a significant code size is likely to require the use of the multi-model capability, since the Hub RAM is so limited but the XMM execution speed is so slow.

Indeed, but the DUALSRAM module and driver running in XMM LARGE worked great and produced acceptable execution speeds for what I needed for Menu displaying and operator input.

But it's external hardware. As mentioned previously, if there's any way I can get away from external hardware I'm going to give it a try. Right now the only XMM mode I know that can be run without external hardware on the Prop1 is XEPROM.

As my coding and testing progresses, I'm quite confident that I will need to replace the 64 kilobyte EEPROM with a larger one. So far I haven't found any 256 kilobyte ones that fit within the same form factor, but 128 kilobyte ones appear at the ready.

I haven't seen any that operate at 3.4Mbps on the I2C bus, but 1Mbps ones seem to be common. Between tweaking the I2C bus speed and choosing sufficient, but not excessive, cache I might be able to get some decent execution speeds on the XEPROM XMM cog comparable to what I was getting with the DUALSRAM running XMM LARGE

RossH · 2023-09-11 08:46

Hello @Wingineer19

I've been prototyping a simpler solution for you.

Here is an example of a program that loads your menus into EEPROM (the -D MENU_ADDR=XXXXXX option specifies the address to store the menu data):

/*
 * loadmenus.c - load menu data into EEPROM
 *
 * Compile with a command like:
 *    catalina loadmenus.c -C C3 -C SMALL -C XEPROM -C CACHED_1K -D MENU_ADDR=16384
 *
 * Note that MENU_ADDR must be specified as a decimal number. This program
 * does nothing - it is simply used to store the menu data into EEPROM. The 
 * menu data is stored as an array of addresses starting at MENU_ADDR, followed
 * by the data of each menu. If the number of menus is not known in advance, 
 * terminate the array by adding a known end marker value such as a zero.
 *
 * Load with a command like:
 *    payload EEPROM loadmenus.binary
 */

// stringizing functions (required to process MENU_ADDR):
#define STRING_VALUE(x) STRING_VALUE__(x)
#define STRING_VALUE__(x) #x

// start menus by padding to the address:
#define START_MENUS(addr) \
   PASM( \
      "\n" \
      "' Catalina Init\n" \
      " alignl\n" \
        "Menu_Pad\n" \
        " byte $00["STRING_VALUE(addr) " - @Menu_Pad]\n" \
   );

// declare a menu - the name must be unique and the value must be a string:
#define DECLARE_MENU(name, value) \
   PASM( \
      "\n" \
      "' Catalina Init\n" \
      " alignl\n" \
      ""#name" long @"#name"_val\n" \
      "' end\n" \
      "' Catalina Data\n" \
      " alignl\n" \
      ""#name"_val byte "#value"\n" \
      "            byte 0\n" \
      "' end\n" \
   );

// add an optional end marker (this allows for a variable number of menus)
#define END_MENUS(marker) \
   PASM( \
      "\n" \
      "' Catalina Init\n" \
      " alignl\n" \
      " long "#marker"\n" \
   );

void main(void) {

   START_MENUS(MENU_ADDR); // the menu addresses will be stored here
   DECLARE_MENU(HELLO, "HELLO AND WELCOME!!!");
   DECLARE_MENU(M0I0, "\x1b[1m\x1b[40m\x1b[2J");
   DECLARE_MENU(M0I1, "\x1b[6;30H\x1b[37;40mMiniPlate Participant");
   DECLARE_MENU(M0I2, "\x1b[8;20H\x1b[37;40mFirmWare Version 1.0, Rev Data: 14 Sep 22");
   DECLARE_MENU(M0I3, "\x1b[10;21H\x1b[37;40mCopyright(c) 2022, All Rights Reserved");
   DECLARE_MENU(M0I4, "\x1b[12;31H\x1b[37;40m(1) WiFi Main Menu");
   DECLARE_MENU(M0I5, "\x1b[14;31H\x1b[37;40m(2) GPS Main Menu ");
   DECLARE_MENU(M0I6, "\x1b[16;31H\x1b[37;40m(3) Diagnostics   ");
   DECLARE_MENU(M0I7, "\x1b[18;28H\x1b[37;40mMake Selection (1 To 3):");
   END_MENUS(0); // a marker can be added to indicate the end of the menus

}

And here is an example of a program that reads them (the -D MENU_ADDR=XXXXXX option specifies the address to read the menu data):

/*
 * readmenus.c - read menu data from EEPROM
 *
 * Compile with a command like:
 *    catalina readmenus.c -C C3 -C TTY -lci -C XEPROM -C SMALL -C CACHED_1K -D MENU_ADDR=16384
 *
 * Note that MENU_ADDR must be specified as a decimal number. The menus must
 * have been previously loaded into EEPROM using loadmenus.c
 *
 * Load with a command like:
 *    payload EEPROM readmenus.binary -i
 */

unsigned long GetMemAddr(unsigned long Address) {
   PASM("mov RI, r2");
   PASM("jmp #RLNG");
   return PASM("mov r0, BC");
}

char GetMemChar(unsigned long Address) {
   PASM("mov RI, r2");
   PASM("jmp #RBYT");
   return PASM("mov r0, BC");
}

void main(void) {
   int i;
   unsigned long menu_addr;
   unsigned long addr;
   char ch;

   menu_addr = MENU_ADDR; 

   while ((addr=GetMemAddr(menu_addr)) != 0) {
      t_printf("menu address = %8x\n", addr);
      t_printf("menu string  = \"");
      while ((ch = GetMemChar(addr)) != 0) {
         t_printf("%c", ch);
         addr++;
      }
      t_printf("\"\n\n");
      menu_addr += 4;
   }

   while(1);
}

I think you should be able to adapt this method to your needs.

Ross.

EDIT: Both loadmenus.c and readmenus.c now use the same mechanism to specify the address - i.e. via a -D MENU_ADDR=XXXXXX command line option - note that the address must be specified as a decimal value such as 32768 - it cannot be a hex value such as 0x8000 or $8000. There is now a START_MENUS macro that can use this value, so the -P parameter is no longer required.

RossH · 2023-09-12 02:16

@Wingineer19 said:
Hello RossH,

Hopefully you can make some refinements to it and maybe consider adding it as an option to Catalina.

Cool! I've tried it and it seems to work ok, apart from the field width, which doesn't seem to work the same way as standard printf - I'll have a look at why when I get some time.

Note that you should add the following at the top of your file:

#include <stdarg.h>
#include <math.h>

Catalina has something similar - t_printf() and friends - Check out the Catalina Reference Manual or see source\lib\catalina\tprintf.c. These functions work with all the standard HMI options - they are not as sophisticated as either the standard printf() or your ConPrint() but are smaller than both because some of it is implemented in the HMI plugins themselves - but I could add a version to the serial library. If I do so I may extend t_printf() using some of your code.

Ross.

RossH · 2023-09-12 07:33

Hello @Wingineer19

I've updaded the loadmenus.c and readmenus.c programs to make them more consistent, and to remove the need to use magic numbers in the code.

Ross.

Wingineer19 · 2023-09-12 22:57

Hi RossH,

The loadmenus.c and readmenus.c idea is really cool but I can't get it to work on my FLiP.

The only parameter I changed for compiling both programs was to -C FLIP instead of -C C3.

I compiled loadmenus.c for my FLiP and uploaded it to EEPROM. I then compiled the readmenus.c for the FLiP, uploaded it to EEPROM, then ran it. All it did was print a bunch of spaces and then scrolled the screen.

I used Tachyon to perform an EEPROM dump for every location from 0x0000 to 0xFFFF. I only included the portion where the strings are stored.

See if the storage locations look correct. Here's the output:

$0000 $FFFF EE DUMP --> 
0000.8000:   00 1B B7 00  00 91 10 00  1C 40 24 40  14 40 28 40    .........@$@.@(@
0000.8010:   0C 40 02 00  04 40 00 00  FC 01 00 00  30 02 00 00    .@...@......0...
0000.8020:   02 00 00 00  00 02 00 00  38 00 00 00  38 00 00 00    ........8...8...
0000.8030:   40 00 00 00  04 40 00 00  00 02 00 00  38 00 00 00    @....@......8...
0000.8040:   04 40 00 00  00 02 00 00  00 00 00 00  6C 7E 00 00    .@..........l~..
0000.8050:   48 45 4C 4C  4F 20 41 4E  44 20 57 45  4C 43 4F 4D    HELLO AND WELCOM
0000.8060:   45 21 21 21  00 00 00 00  5C 00 00 00  5C 78 31 62    E!!!....\...\x1b
0000.8070:   5B 31 6D 5C  78 31 62 5B  34 30 6D 5C  78 31 62 5B    [1m\x1b[40m\x1b[
0000.8080:   32 4A 00 00  78 00 00 00  5C 78 31 62  5B 36 3B 33    2J..x...\x1b[6;3
0000.8090:   30 48 5C 78  31 62 5B 33  37 3B 34 30  6D 4D 69 6E    0H\x1b[37;40mMin
0000.80A0:   69 50 6C 61  74 65 20 50  61 72 74 69  63 69 70 61    iPlate Participa
0000.80B0:   6E 74 00 00  A8 00 00 00  5C 78 31 62  5B 38 3B 32    nt......\x1b[8;2
0000.80C0:   30 48 5C 78  31 62 5B 33  37 3B 34 30  6D 46 69 72    0H\x1b[37;40mFir
0000.80D0:   6D 57 61 72  65 20 56 65  72 73 69 6F  6E 20 31 2E    mWare Version 1.
0000.80E0:   30 2C 20 52  65 76 20 44  61 74 61 3A  20 31 34 20    0, Rev Data: 14 
0000.80F0:   53 65 70 20  32 32 00 00  EC 00 00 00  5C 78 31 62    Sep 22......\x1b
0000.8100:   5B 31 30 3B  32 31 48 5C  78 31 62 5B  33 37 3B 34    [10;21H\x1b[37;4
0000.8110:   30 6D 43 6F  70 79 72 69  67 68 74 28  63 29 20 32    0mCopyright(c) 2
0000.8120:   30 32 32 2C  20 41 6C 6C  20 52 69 67  68 74 73 20    022, All Rights 
0000.8130:   52 65 73 65  72 76 65 64  00 00 00 00  30 01 00 00    Reserved....0...
0000.8140:   5C 78 31 62  5B 31 32 3B  33 31 48 5C  78 31 62 5B    \x1b[12;31H\x1b[
0000.8150:   33 37 3B 34  30 6D 28 31  29 20 57 69  46 69 20 4D    37;40m(1) WiFi M
0000.8160:   61 69 6E 20  4D 65 6E 75  00 00 00 00  60 01 00 00    ain Menu....`...
0000.8170:   5C 78 31 62  5B 31 34 3B  33 31 48 5C  78 31 62 5B    \x1b[14;31H\x1b[
0000.8180:   33 37 3B 34  30 6D 28 32  29 20 47 50  53 20 4D 61    37;40m(2) GPS Ma
0000.8190:   69 6E 20 4D  65 6E 75 20  00 00 00 00  90 01 00 00    in Menu ........
0000.81A0:   5C 78 31 62  5B 31 36 3B  33 31 48 5C  78 31 62 5B    \x1b[16;31H\x1b[
0000.81B0:   33 37 3B 34  30 6D 28 33  29 20 44 69  61 67 6E 6F    37;40m(3) Diagno
0000.81C0:   73 74 69 63  73 20 20 20  00 00 00 00  C0 01 00 00    stics   ........
0000.81D0:   5C 78 31 62  5B 31 38 3B  32 38 48 5C  78 31 62 5B    \x1b[18;28H\x1b[
0000.81E0:   33 37 3B 34  30 6D 4D 61  6B 65 20 53  65 6C 65 63    37;40mMake Selec
0000.81F0:   74 69 6F 6E  20 28 31 20  54 6F 20 33  29 3A 00 00    tion (1 To 3):..
0000.8200:   00 00 00 00  0E 00 7C 5C  00 02 00 00  00 00 00 00    ......|\........
0000.8210:   80 66 FC A0  00 66 7C 0C  19 00 7C 5C  0C 00 7C 5C    .f...f|...|\..|\
0000.8220:   03 00 7C 5C  3C 00 00 00  2E 5E BC 08  2F 6C BC 04    ..|\<....^../l..
0000.8230:   02 5E FC 80  2F 6A BC 04  0C 00 7C 5C  0C 00 7C 5C    .^../j....|\..|\
0000.8240:   0B 00 7C 5C  10 02 00 00  00 00 00 00  00 00 00 00    ..|\............