Update: I was wrong about not being able to execute C code that used floating point from the LUT. The C code executed from the LUT can include any C code that will fit, including all the basic floating point operations - it just cannot call any other C functions (except those that can be 'inlined' by the optimizer).
This means it cannot use any standard C library functions, which includes the floating point library functions.
Removing this limitation will take a lot of work for not much extra benefit, so I will just included it "as is" in the next release.
If anyone can think of a cool demo for this capability then please let me know. My current examples are all fairly trivial.
REP loops will benefit. Stuff like bit-bashed SPI can run much tighter. So much so that I/O staging latencies will need accounted for in the rx phase-lag timing.
Well, this LUT C execution lark has been an interesting little exercise.
It turns out to be easy to remove all the limitations on executing C from the LUT, except (naturally) for the size limitation. I can have LUT C able to do anything normal C can do, including calling other C functions (LUT or normal).
The downside is that to make it work I have to remove some of the code size reduction techniques I use, so the code ends up being larger. And the speed improvements of LUT execution over Hub execution are very marginal at best. Particularly for C code small enough to fit in the LUT in the first place.
So I have decided not to bother. LUT execution of 'leaf' C code will be supported for NATIVE C programs in the next release (since I get that for nothing) but the main use for LUT execution is for executing PASM, not C.
This is a full release. The main new feature is addition of the LUT Execution macros, which can now be used in all Propeller 2 memory models (NATIVE, COMPACT, TINY, XMM SMALL or XMM LARGE).
Also, I have now added Lua to the Catalina libraries. While not a new feature (Lua has been supported since the Propeller 1 days), this release makes it much easier to build C programs with embedded Lua scripting. Almost completely trivial, in fact - a fully working demo program is less than ten lines of C code:
#include <lua.h>
#include <lualib.h>
#include <lauxlib.h>
int main(int argc, char *argv[]) {
// create a new Lua state
lua_State *L = luaL_newstate();
// open the standard Lua libraries
luaL_openlibs(L);
// execute the script contained in the file "script.lua"
luaL_dofile(L, "script.lua");
}
And here is an example script.lua that this program might load and execute:
# a simple Lua script
print('Hello, World (from "script.lua")\n');
io.write("enter number A: ");
A = io.read("n");
io.write("enter number B: ");
B = io.read("n");
io.write("A + B = " .. A + B .. "\n");
Here is the relevant extract from the README.TXT:
RELEASE 7.5
New Functionality
-----------------
1. Lua has now been integrated into the Catalina library. This makes it much
easier to build C programs that embed Lua. Previously to do this the whole
Lua distribution had to be compiled with the C program, but now only the
Lua initialization module (linit.c) is required because the rest of Lua
can be included by simply specifying either -llua or -lluax on the Catalina
command line. Examples of doing this (and an explanation of the
difference between -llua and -lluax) are provided in a new demo folder
called demos/lua. See the README.TXT file in that folder for more details.
This also means that building C programs that embed Lua is now possible
using the self-hosted version of Catalina on the Propeller 2. However,
while this is now supported, it is not yet very practical since it takes
hours to compile even a very simple C program that embeds Lua (lhello.c).
Note that the Catalyst version of Lua does NOT use the compiled version
in the library. This is because some of the options offered by the
Catalyst version of Lua (e.g. ENABLE_PSRAM, ENABLE_HYPER) require Lua to
be recompiled from source.
Applies to Windows and Linux for the Propeller 1 and 2, and Catalyst
for the Propeller 2.
2. The self-hosted version of catalina has an additional option -W which can
be used to send options to the compiler (specifically, to rcc). For
instance, to suppress warnings use the option -W-w which will pass the
option -w to rcc. Applies to Catalyst on the Propeller 2.
3. A new include file has been added (lut_exec.h) which simplifies the
definition of inline PASM and C code to be loaded and executed from the
LUT (i.e. using LUT execution mode). Code executed from the LUT is
limited to 254 longs.
Inline PASM executed from the LUT is supported in all memory models.
C code executed from the LUT is only supported for the NATIVE memory model,
and has the additional limitation that it must be 'leaf' code - i.e. it
cannot call any other C functions (except those that can be 'inlined' by
the optimizer).
Demos of LUT execution have been provided in a new demos/lut_exec folder.
Both PASM and C examples are included. Applies to the Propeller 2 only.
Other Changes
--------------
1. The default Propeller 2 clock parameters are specified in each platform
file (e.g. P2_EDGE.inc, P2_EVAL.inc, P2_CUSTOM.inc etc). These could be
overridden on the command line via the -f, -F & -E command line options,
which are used to calculate appropriate values for _XDIV, _XMUL and _XDIVP
(see the platform include files or the Propeller specifications for more
details on these values).
The defaults in previous releases were as follows:
_XDIV = 4 '\ crystal divider to give 5.0MHz
_XMUL = 72 '| crystal / div * mul to give 360MHz
_XDIVP = 2 '/ crystal / div * mul / divp to give 180MHz
These defaults have been changed in this release to:
_XDIV = 1 '\ crystal divider to give 20MHz
_XMUL = 9 '| crystal / div * mul to give 180MHz
_XDIVP = 1 '/ crystal / div * mul / divp to give 180MHz
This results in the same default clock frequency (180Mhz) but the new
values should mean the clock is more stable. Applies to the Propeller
2 only.
2. Eliminated errors and warnings issued when building the Catalina libraries.
Most of these were because some of the libraries were being compiled for a
Propeller platform (1 or 2) that did not support them. Now only the
supported libraries are compiled for each Propeller platform.
3. Eliminated warnings about deprecated code when compiling catdbgfilegen,
which is the Catalina utility that generates debugging information.
I just noticed there are two different sets of instructions for rebuilding Catalina under Linux. The instructions in README.Linux are out of date. The up-to-date instructions are the ones in the file BUILD.TXT.
Catalina's multi-model capability - e.g. being able to define one part of a program to execute as XMM code while another part executes as NATIVE code - is potentially one of Catalina's most powerful features - but it can be quite complex to figure out how to compile such programs correctly. Even I struggle with it, and have to re-learn it every time I use it.
I have been working on something to make it simpler to do, and the result is Catalina Catapult - now available as part of Catalina 7.6.
Here is an extract of the documentation - which is now all contained in the document Getting Started with Catapult:
Catalina Catapult is a utility intended to simplify the process of developing,
debugging and maintaining Catalina multi-model programs.
A multi-model program typically consists of a primary program in which speed
is not critical (and so it can be executed from XMM RAM) which then loads and
executes subsidiary programs that execute in Hub RAM whenever speed, precise
timing or functionality not supported by XMM programs - e.g. multi-threading
or interrupts (on the Propeller 2) is required.
Alternatively, the primary program may consist of a Hub-based primary program
which loads other Hub-based secondary programs as overlays from files as
necessary.
In either case, the main advantage of multi-model programs is that secondary
programs do not consume Hub RAM except when they are actually executing.
Multi-model support therefore allows C programs larger than Hub RAM to execute
while still allowing them to access all the functionality of the Propeller
when needed.
The main disadvantage of multi-model programs is that they are more difficult
to develop, build, debug and maintain than ordinary C programs. Catapult helps
by allowing multi-model programs to be developed and kept in a single C source
file that contains both the primary program and all the secondary programs.
Catapult also helps by splitting the source file into separate program files,
compiling each one using the appropriate memory model and options, invoking
the utility required to turn these into data files to be loaded when required
by the primary program, and then compiling the primary program to produce a
single binary ready for execution.
The best way to see how Catapult simplifies this process is to build and run a
demo program. There are several versions of the same demo program in this
folder - some specifically for the Propeller 1 (e.g. demo_p1.c) and others
specifically for the Propeller 2 (e.g. demo_p2.c).
The reason there are different versions of the program for the Propeller 1
and Propeller 2 is that the Propeller 1 and 2 support different memory models
and different address ranges - both of which are important when building
multi-model programs.
The catapult command always accepts a single C source file name as its
argument.
To compile the Propeller 1 version of the demo program:
catapult demo_p1.c
or, to compile the Propeller 2 version of the demo program:
catapult demo_p2.c
Both these commands will take the specified C source file, split it up (in
this case) into one primary and two secondary programs, compile those
programs separately, and then combine them back into a single output binary.
A command file will be created called either _catapult.bat (Windows) or
_catapult.cmd (Linux).
Doing this manually is quite possible, but it can be quite complex. For an
example of this, see the Makefile in the demos\multimodel folder. Catapult
eliminates the need to write such complex Makefiles. Examine the command file
generated by building the demo program (_catapult.bat or _catapult.cmd) to
see that Catapult generates the necessary commands for (in this case) the
three Catalina commands and two invocations of the spinc utility required
to build the final binary.
The resulting binary program can be loaded and executed using payload.
For the Propeller 1:
payload -i demo_p1
or, for the Propeller 2:
payload -i demo_p2
Note that if you compile BOTH the Propeller 1 and Propeller 2 versions, you
may need to add the extension in the payload command (i.e. .binary for the
Propeller 1, or .bin for the Propeller 2) or else use the -o1 or -o2 payload
option to specify whether you are loading a Propeller 1 or Propeller 2.
Now for the details ...
Catapult pragmas
================
Catapult is implemented by adding pragmas to the C source file. There are
three catapult pragmas:
#pragma catapult common [ name ] [ attributes ]
This pragma introduces a common segment, which should contain all
the definitions and types that are common to the primary and all
the secondary programs - the common segment will be converted into a C
header file shared by all other segments. If no name is specified, the
default name is "common". Note that all code not included in any other
segment will be included in the common segment, even if that code
precedes the actual pragma (or there is no common pragma). The common
segment should include only C definitions and types, and not data,
variables or functions - those should be specified in the appropriate
primary or secondary segment.
#pragma catapult secondary [ name [ (type) ] ] [ attributes ]
This pragma introduces a secondary segment, which should contain
all the file scope data and code required for a secondary program,
but no C main function. Instead, the segment should include a C
function with the name of the segment that accepts a pointer to the
specified type. The secondary segment is converted into a C program
file. If no name is specified, the default name is "secondary". If
no type is specified, the default type is the name of the segment
with "_t" appended. There can be none, one or more secondary segments.
Each secondary segment must be given a unique name. Each secondary
segment should include all the data, variables and functions required
by the secondary function. See also the section below on Sharing Data
between Catapult Segments.
#pragma catapult primary [ name ] [attributes ]
This pragma introduces the primary segment, which should contain
all the file scope data, variables and code required for the primary
program, including a C main function. The primary segment is converted
into a C program file that includes either the binaries of all the
secondary segments as an array of longs, or else the name of the
overlay files containing this data. If no name is specified, the
default name is "primary". There must be exactly one primary segment.
If there are multiple primary pragmas, they must all be identical.
See also the section below on Sharing Data between Catapult Segments.
Each pragma can specify additional attributes, which are described below.
There can be multiple instances of each pragma, but only one segment of each
type will be created with each specified segment name, so each instance of a
pragma type (common, secondary or primary) with the same name must be
identical. Catapult will warn you if this is not the case.
The primary and secondary segment names must be unique, but the common
segment may share the same name as another segment since the common segment
is generated as a C header file (.h), and the primary and secondary segments
are generated as C programs files (.c).
All the code introduced by the same pragma type with the same name will
be combined in the same output file for compilation. For the common segment
this file will be given the name of the segment plus a ".h" extension. For
a primary or secondary segment it will have the segment name plus a ".c"
extension. Catapult will complain if this would result in the same file name
as the source file, to prevent accidentally overwriting that file. However,
the binary output can be given that name by using the 'binary' attribute.
Catapult pragma attributes
==========================
Each of the catapult pragmas can also include various attributes:
address(value) - specifies the address to use for a secondary segment.
The value can be a decimal value (e.g. 16384) or a hex
value (e.g. 0x4000). This attribute applies only to
secondary pragmas.
stack(value) - specifies the stack size to use for a secondary segment.
The value should be a decimal value (e.g. 500).
This attribute applies only to secondary pragmas.
mode(name) - specifies the memory model to be used for the segment.
For secondary segments, the mode name can be any of:
COMPACT or CMM : use the COMPACT memory model
TINY or LMM : use the TINY memory model
NATIVE or NMM : use the NATIVE memory model (P2 only)
For the primary segment, this can also be:
SMALL or XMM SMALL : use the SMALL memory model
LARGE or XMM LARGE or XMM : use the LARGE memory model
options(list) - specifies Catalina command line options to be added to
the catalina command when compiling the segment.
For example: options(-lc -lm -C TTY -O5)
All Catalina command line options are valid, but some
should not be used since catapult will generate them
automatically, such as -o for the name of the output.
To define Catalina symbols use the usual -C syntax,
and to define C symbols, use the usual -D syntax.
Note that options can be specified on common, secondary
or primary pragmas - any options specified on common
pragmas will be combined with the options specified
for the primary or secondary pragma when the relevant
programs are compiled. For instance, if a platform is
included in a common pragma - e.g. options(-C C3) - it
will apply to the compilations of both primary and
secondary programs.
binary(name) - specifies the name of the binary output to be generated.
Can be used in primary and secondary pragmas. If not
specified, then the name of the segment is used. Use
this attribute to name the binary output instead of
specifying -o in the options attribute. Do not include
an extension - the appropriate extension (i.e. .binary
or .bin) will be added automatically.
overlay(name) - specifies the name of an overlay file to be generated.
Can be used in secondary pragmas only. Use this attribute
to generate a file containing the overlay instead of
an array of longs in the primary programs data segment.
The overlay files must be placed on an SD card, and will
be loaded automatically at execution time whenever the
secondary program is started. The overlay name can
include an extension. Note that this attribute CAN - but
does not HAVE TO - be used in conjunction with the
OVERLAY macros (described later). See the section on
using Catapult with Overlay Files for more details.
The names used in the pragmas cannot be the same as any of the possible
attributes (so for example, 'address' cannot be used as a segment name).
Just as a final note to highlight what this is all about without diving into any technical details ... let's say that during your program development you run out of Hub RAM (all too easy to do on the Propeller 1) and realize that you have to change your primary program so that it runs from XMM RAM instead of Hub RAM. What would you have to do?
This is now literally a one word change. For instance, in demo_p1.c you just change this line:
#pragma catapult primary main mode(COMPACT) binary(demo_p1)
... to this ...
#pragma catapult primary main mode(SMALL) binary(demo_p1)
... or this ...
#pragma catapult primary main mode(LARGE) binary(demo_p1)
... and then re-execute the catapult command:
catapult p1_demo.c
... and you're done.
Ross.
EDIT: Update documentation extract and remove Windows binary - Catapult is now available for both Windows and Linux as part of Catalina 7.6
If we got the memory location wrong for the secondary.c code the program will let us know at runtime what the proper address should be so we can correct it and go through this whole compiling process again.
I combined the secondary.c and primary.c code into a new file called prisec.c:
I did comment out the check_memory() function within the primary portion which previously did the memory checking under the assumption that Catapult might do it for me instead.
The prisec.c code won't compile, much less run. Could you look over the code and see what I did wrong? Thanks.
This is a full release. The main new feature is addition of the Catalina Catapult utility. See the previous post for a quick overview of Catapult, or download the document Getting Started with Catapult from the link above.
Here is the relevant extract of the README.TXT file:
RELEASE 7.6
New Functionality
-----------------
1. Catalina Catapult is a utility intended to simplify the process of
developing, debugging and maintaining Catalina multi-model programs.
For details, see the document Getting Started with Catapult, and/or
the demo programs in the demos/catapult folder.
2. The C library now has two overlay load functions. The existing function:
_load_overlay() - which uses the C stdio file system.
And a new function:
_load_overlay_unmanaged() - which uses the Catalina file system.
The new Catalina file system version takes much less Hub RAM than the
previous version, and is recommended for use on the Propeller 1 for
programs that do not otherwise require stdio. However, it requires the
_mount() function to be called before any overlays are loaded. Typically,
this would be called in the main program as:
_mount(0, 0);
3. The spinc utility now generates code that can use either the existing
_load_overlay() function or the new _load_overlay_unamanaged() function
depending on whether the Catalina symbol FS_OVERLAY is defined.
For example:
catalina -C FS_OVERLAY -lcx overlay.c
The difference is that while the unmanaged version still needs to be
compiled with the extended file system (e.g. -lcix or -lcx) it uses the
Catalina file system functions, which are much smaller than the stdio
file system functions. However, note that the _mount() function must be
called before any overlays are loaded. Typically, this would be called in
the main program as:
_mount(0, 0);
4. The spinc utility now accepts hex values (e.g. 0x200) as arguments for the
-s stack size command line option. For example:
spinc -s 0x200 program.binary > xxx.inc
5. Catalina now accepts the definition of the Catalina symbol P2 to mean
the same as the -p2 command line option. This allows the propeller
version to be specified using CATALINA_DEFINE.
Other Changes
--------------
1. None.
Because the type is not named secmain_t (which is the default assumed for a segment named secmain) you need to explicitly name it in the secondary pragma (or else call it secmain_t instead of the name I chose, which was shared_t):
Also (but this one is just a matter of style, intended to simplify maintenance) I have taken all the common options out of the primary and secondary pragmas, and added them to the common pragma instead - i.e.:
Finally, note that some of the options you had (e.g. -C NO_ARGS) are no longer required because they are added automatically by catapult where required.
But unfortunately, it still didn't work. No error display but no proper operation, either.
I suspect the Secondary isn't being loaded to the proper address. Perhaps the CATAPULT_ERROR is immediately sent but the display screen isn't fast enough to show me the memory location error right after EEPROM upload.
And if the CATAPULT_ERROR is only sent once, if I miss it then I miss it and that's too bad so sad...
If you look at my check_memory() function which is commented out, you will notice that if the memory location is wrong, the code enters into an infinite loop and will display the error every time the operator hits the ENTER key on the keyboard. So, if the first time the error was displayed is missed, the operator just hits the ENTER key and it will display it again.
Can the CATAPULT_ERROR be modified to do something similar?
I did modify the CATAPULT_ERROR to do what I wanted -- it will continually display the error message whenever any key is hit.
Hence, if for some reason the error message was initially missed, all one needs to do is hit any key to get the message displayed again.
//Program is prisec.c
//Combines the previous secondary.c and primary.c programs into one
//Last revision on 27 June 2024
#pragma catapult common options(-lc -lm -lserial4 -C NO_HMI -C FLIP -y)
#include "catapult.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdarg.h>
#include <math.h>
#include <time.h>
#include <propeller.h>
#define Yes 1
#define No 0
#define conport 0
#define print_command 1
#define ROUNDUP 128
typedef struct shared
{
char request;
char *format;
char *outstr;
va_list ptr;
} shared_t;
// since we are not using a HMI, we must redefine CATAPULT_ERROR
#undef CATAPULT_ERROR
#define CATAPULT_ERROR(name,address)\
{\
for(;;) \
{ \
s4_str(conport,"Catapult Error -- Change The "); \
s4_str(conport,name); \
s4_str(conport," Memory Location To 0x"); \
s4_hex(conport,(int) address,6); \
s4_str(conport,"\r\n"); \
while(s4_rxcheck(conport) < 0); \
} \
}
#pragma catapult secondary secmain(shared_t) mode(COMPACT) address(0x1234) stack(512)
struct shared *secondary;
char outstr[256];
void secprint(char *format,...)
{
va_list ptr;
va_start(ptr,format);
(*secondary).format=format;
(*secondary).ptr=ptr;
(*secondary).outstr=&outstr[0];
(*secondary).request=print_command;
while((*secondary).request != 0);
s4_str(conport,outstr);
}
void secmain(struct shared *primary)
{
float theta;
secondary=primary;
for(;;)
{
for(theta=0.0; theta<6.283185307; theta+=0.0062831853)
{
secprint("theta=%f,sine=%f,cosine=%f\r\n",theta,sin(theta),cos(theta));
}
}
}
#pragma catapult primary main mode(SMALL) binary(primary) options(-C XEPROM -C CACHED_4K)
struct shared primary;
void priprint(void)
{
vsprintf(primary.outstr,primary.format,primary.ptr);
va_end(primary.ptr);
primary.request=0;
}
void main(void)
{
RESERVE_SPACE(secmain);
int cogsec;
RESERVED_START(secmain,primary,ANY_COG,cogsec);
for(;;)
{
if(primary.request == print_command) priprint();
}
}
In this example notice that I've deliberately set the secmain secondary address to 0x1234.
Now, after compiling and uploading it will display this error and will continue to do so whenever any key is hit. So if you hit any key 100 times then you will get this message displayed 100 times:
Catapult Error -- Change The secmain Memory Location To 0x0057FC
It looks like Catapult will be a great timesaver and really simplifies the compiling process.
Before, I had to manually compile the secondary.c file.
Then run spinc on it to generate the secondary.inc file.
Then compile the primary.c file to generate the primary.binary output and upload it to the Prop.
If the memory location was wrong, I had to change it to the proper location and go through this whole process again.
Now, I only need to use Catapult on a single file after making the change. Very easy to do now and eliminates mistakes during the compiling process -- like forgetting to run spinc which was a common mistake.
One thing to keep in mind is that the CATAPULT_ERROR will need to be redefined for whatever operator interface is being used. Here, it's redefined for the 4-Port serial driver, but the same will be needed for the tty and tty256 and the 8-Port serial driver for the Prop2.
I'll continue to experiment with Catapult and let you know how it goes.
Any idea why using the Optimizer would introduce a huge time delay before the code would start executing?
No. I've used the optimizer on my own programs with no issues. I'll investigate.
EDIT: Yes, I see the same thing as you. Odd. But it's probably not catapult, which just manages the compilation process. It could be an optimizer quirk - I'll have to examine the optimized code.
EDIT: A clue is that the delay is about 53 seconds - which is the delay you get when you do a WAITCNT for a count that has just passed and have to wait for the 32 bit counter to "go round the clock". This means I think I know where the problem is - I'll confirm it and issue a patch.
Still working on this issue. As I suspected, it is a WAITCNT instruction going astray, but it is deeper than that - the instruction is not working because of a memory corruption.
So far, I can tell you what the problem is NOT:
It is not catapult.
It is not the optimizer.
It is not the compiler.
It is not payload.
It is not running out of stack space or Hub RAM.
On my C3, the program works when executed from Hub RAM, FLASH or SRAM, and when it is executed from EEPROM it is being loaded correctly. I have two possibilities still to investigate: (1) the XEPROM XMM drivers, and (2) the variable argument macros (see below).
Since it is so repeatable, I am sure I will track it down.
A couple of things to note about this particular program:
It should use -lma or -lmb instead of -lm. It currently works with -lm because the main program is an XMM program, which always loads a floating point plugin - it stops working if (for example) it is executed entirely from Hub RAM because -lm does not load the floating point plugin which the COMPACT secondary program needs. There is a note about this in the catapult documentation.
Your use of the variable argument macros is suspicious - they are being executed by different C programs running on different cogs and using different stacks. Using a va_list in one C program after setting it up (via va_start) in another C program could be causing the problem (not sure yet).
Then finally upload the primary.binary to the Prop:
payload -i eeprom primary.binary
After going through the usual memory relocation process, and after uploading the memory corrected compiled code, the code begins to immediately execute (within just a few seconds or so) and shows theta, sine, and cosine.
If the va_list and va_start macros are causing problems, they only seemed to surface when using Catapult and not with the individual compiling process used above.
It could be that the -lm option is causing the problem but I left it in both compile commands and the code still works.
Just thought I would add this test to the mix to muddy the water even more
@Wingineer19 said:
After going through the usual memory relocation process, and after uploading the memory corrected compiled code, the code begins to immediately execute (within just a few seconds or so) and shows theta, sine, and cosine.
If the va_list and va_start macros are causing problems, they only seemed to surface when using Catapult and not with the individual compiling process used above.
It could be that the -lm option is causing the problem but I left it in both compile commands and the code still works.
Just thought I would add this test to the mix to muddy the water even more
All good. I don't think it is catapult (which basically just does what you are doing manually) and I have done some more investigating into the variable argument macros and I think your use of them is basically ok. However, code like this would probably not be portable to another C compiler. The C manual says:
Each invocation of va_start() must be matched by a corresponding invocation of va_end() in the same function. After the call va_end(ap) the variable ap is undefined.
While technically you are using the variable initialized by va_start() before va_end() is invoked, you are doing so in another C program compiled separately. I don't think the designers of the macros considered that case!
The -lm issue doesn't matter as long as your primary program is an XMM program. But it does when I turn it into an LMM program to run it entirely from Hub RAM.
I am on the track of another possibility now. Will let you know what I find.
@RossH said:
All good. I don't think it is catapult (which basically just does what you are doing manually) and I have done some more investigating into the variable argument macros and I think your use of them is basically ok. However, code like this would probably not be portable to another C compiler. The C manual says:
Each invocation of va_start() must be matched by a corresponding invocation of va_end() in the same function. After the call va_end(ap) the variable ap is undefined.
While technically you are using the variable initialized by va_start() before va_end() is invoked, you are doing so in another C program compiled separately. I don't think the designers of the macros considered that case!
LOL, yes, I had to do some experimentation with the code to get it to do what I wanted it to.
However I did something similar in Windows using the OpenWatcom compiler while configuring the secondary as a separate thread. In that case I used a va_copy to store the va_list ptr within the shared structure in order to make it work:
Basically, I was "winging it", hence my handle Wingineer19 as the technicians out at work always accused us engineers of "winging it". Truth be told, they were right more often than not.
I think it's approaching late afternoon where you live, but it's getting late here in the States so I'm going to call it a night. I'll check back tomorrow morning and see if you found any other interesting stuff.
OK, I just completed 32 iterations of this code and got some interesting results.
First, here's the baseline code I tested. Variations are made for each iteration to account for the cache size, optimization, and the type of math library tested.
//Program is prisec.c
//Combines the previous secondary.c and primary.c programs into one
//Last revision on 28 June 2024
#pragma catapult common options(-lc -lm -lserial4 -C NO_HMI -C FLIP -y)
#include "catapult.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdarg.h>
#include <math.h>
#include <time.h>
#include <propeller.h>
#define Yes 1
#define No 0
#define conport 0
#define print_command 1
#define ROUNDUP 128
typedef struct shared
{
char request;
char *format;
char *outstr;
va_list ptr;
} shared_t;
// since we are not using a HMI, we must redefine CATAPULT_ERROR
#undef CATAPULT_ERROR
#define CATAPULT_ERROR(name,address)\
{\
for(;;) \
{ \
s4_str(conport,"Catapult Error -- Change The "); \
s4_str(conport,name); \
s4_str(conport," Memory Location To 0x"); \
s4_hex(conport,(int) address,6); \
s4_str(conport,"\r\n"); \
while(s4_rxcheck(conport) < 0); \
} \
}
#pragma catapult secondary secmain(shared_t) mode(COMPACT) address(0x4ec4) stack(512)
struct shared *secondary;
char outstr[256];
void secprint(char *format,...)
{
va_list ptr;
va_start(ptr,format);
(*secondary).format=format;
(*secondary).ptr=ptr;
va_end(ptr);
(*secondary).outstr=&outstr[0];
(*secondary).request=print_command;
while((*secondary).request != 0);
s4_str(conport,outstr);
}
void secmain(struct shared *primary)
{
float theta;
secondary=primary;
for(;;)
{
for(theta=0.0; theta<6.283185307; theta+=0.0062831853)
{
secprint("theta=%f,sine=%f,cosine=%f\r\n",theta,sin(theta),cos(theta));
}
}
}
#pragma catapult primary main mode(SMALL) binary(primary) options(-C XEPROM -C CACHED_1K)
struct shared primary;
void priprint(void)
{
vsprintf(primary.outstr,primary.format,primary.ptr);
primary.request=0;
}
void main(void)
{
RESERVE_SPACE(secmain);
int cogsec;
RESERVED_START(secmain,primary,ANY_COG,cogsec);
for(;;)
{
if(primary.request == print_command) priprint();
}
}
One thing I discovered is that code execution didn't care if you did this:
Cache Size Optimization Code Execution Delay
1K Y > 50 sec
2K Y > 50 sec
4K Y > 50 sec
8K Y < 5 sec
I haven't tried this yet so it will be interesting to see what happens:
#pragma catapult common options(-lc -lmb -lserial4 -C NO_HMI -C FLIP -y)
or
#pragma catapult common options(-lc -lmb -lserial4 -O5 -C NO_HMI -C FLIP -y)
So it does appear that using the -lma plugin will introduce the delay regardless of cache size and optimization with the exception of CACHE_8K and optimization. That's strange.
What is also odd is that using the -lm will only introduce the delay using CACHE_4K and optimization.
Hopefully this additional information will provide more clues as to what is happening.
So, I guess there's an issue with using the -lmb plugin as well...
I have tried the -lmb option, with essentially the same results - i.e. I can make the problem appear or disappear just by changing a few instructions. I don't even need to recompile the program.
I know exactly what is being corrupted and when. But I cannot figure out how or why
@RossH said:
I have tried the -lmb option, with essentially the same results - i.e. I can make the problem appear or disappear just by changing a few instructions. I don't even need to recompile the program.
I know exactly what is being corrupted and when. But I cannot figure out how or why
My supposition is that something bad is happening with the va_list data within the shared structure as this info is passed from the secondary over to the primary via this shared structure.
If so, why doesn't the problem arise for EVERY instance of testing regardless of cache size, optimization, or math library plugin? It doesn't make any sense.
Could the way the vsprintf() function is used within priprint() be causing the problem?
This is an example of a major problem being caused by a small program. Totally insane.
Yes, it is. I now have the problem isolated to a single instruction. Execute it and the program goes haywire. I just can't figure out why. The instruction is completely unrelated to the memory location that is being corrupted.
I'm missing something obvious. It will no doubt end up as one of those "D'oh!" moments
It's not related to the instruction being executed, it's related to the timing of the instruction being executed. Which is why the optimizer makes a difference. I's likely to be a very tight race condition between two or more cogs.
That opens up the field again, but at least it makes a bit more sense.
My supposition is that something bad is happening with the va_list data within the shared structure as this info is passed from the secondary over to the primary via this shared structure.
If so, why doesn't the problem arise for EVERY instance of testing regardless of cache size, optimization, or math library plugin? It doesn't make any sense.
Could the way the vsprintf() function is used within priprint() be causing the problem?
This is an example of a major problem being caused by a small program. Totally insane.
No your program is fine. It's definitely a problem with Catalina's XEPROM support. I now have several test programs that all work perfectly when executed from FLASH, SRAM, Hub RAM, and even from EEPROM - and with any combination of cache size, optimization, and maths plugins - but which all fail with the same symptoms when executed via XEPROM.
I have a workaround for this specific instance of the problem which makes all these programs work reliably. I will include it in the next release because it's also more robust than the code that I currently use to start a C program on another cog.
However, it is only a workaround, not a solution. I still need to find the underlying cause or the problem might show up again elsewhere. I now believe it is in the XEPROM XMM driver, which I haven't looked at for close to a decade
Ross.
P.S. Your test program is indeed a "small" program, and so looks deceptively simple - but the underlying complexity of actually building and executing it is very high. So thank goodness for Catapult! Without it, recompiling this program manually so many times with so many different combinations of options would be a nightmare!
I have a workaround for this specific instance of the problem which makes all these programs work reliably. I will include it in the next release because it's also more robust than the code that I currently use to start a C program on another cog.
I still need to find the underlying cause or the problem might show up again elsewhere. I now believe it is in the XEPROM XMM driver, which I haven't looked at for close to a decade
Well, that's a bummer. Having to rewrite an old driver doesn't sound like fun. But if you do, you might find ways to make it even better and more robust.
The XEPROM driver using the SMALL memory model that stores code in EEPROM, but doesn't use external SRAM, is an attractive option for running an XMM program that's heavy on code but doesn't need lots of data memory. Circuit complexity is certainly reduced without the need for external memory and hardware.
Hopefully, Parallax will eventually offer the FLiP with a 128 kilobyte EEPROM (or a 256 kilobyte one if they're in a really generous mood) that will make it more useful than the 64 kilobyte one installed now.
There are newer 128 kilobyte EEPROMs that support up to 3.4 MHz transfer speed like the MicroChip 24CSM01. That combined with caching should yield a little bit better performance than the current ones.
P.S. Your test program is indeed a "small" program, and so looks deceptively simple - but the underlying complexity of actually building and executing it is very high. So thank goodness for Catapult! Without it, recompiling this program manually so many times with so many different combinations of options would be a nightmare!
Absolutely! Catapult really proved itself when I did 40 iterations of test code yesterday. Each iteration required a single compile command followed by a single upload command. If the secondary memory needed to be changed then one more compile and upload would be required.
Without Catapult each iteration would have required:
1. Compile secondary
2. Run spinc
3. Compile primary
4. Upload to EEPROM
5. If secondary memory needed to be changed, then repeat steps 1 through 4 again.
So Catapult essentially cut the required number of steps by half. I think any XMM programmer will find it to be a very handy utility.
However, it is only a workaround, not a solution. I still need to find the underlying cause or the problem might show up again elsewhere. I now believe it is in the XEPROM XMM driver, which I haven't looked at for close to a decade
Found it. I thought this would probably involve a D'oh! moment, and I was at least right about that!
The problem has nothing to do with XEPROM. It is the floating point plugin that is corrupting Hub RAM. After enough tests, I noticed that the data being written was not a valid Propeller instruction - it looked suspiciously like an IEEE-754 floating point number, which indeed it was! I was not correctly initializing the communication between a dynamically loaded COMPACT kernel and the floating point plugin. This would be a problem if the secondary C program was a COMPACT program, and executed a floating point operation before the primary program had finished waiting for it to start, such as the program used by @Wingineer19 does (it also depended on the timing, so it was indeed a "race" condition).
My "workaround" was to eliminate that wait so that this could no longer happen, but now I have a proper fix.
No major new functionality, just fixes for a few issues. It is a full release because it requires a rebuild of all the C libraries.
Here is the relevant extract of the README.TXT file:
RELEASE 7.6.1
New Functionality
-----------------
1. The _clockfreq() function now returns a default value (80Mhz on a Propeller
1 or 180Mhz on a Propeller 2) if it finds no frequency value has been set
in the appropriate Hub RAM location ($0 on a Propeller 1, or $14 on a
Propeller 2).
2. The catapult STOP macro, which is the recommended way of terminating an
executing secondary function in a multi-model program, now also unregisters
the cog. If the STOP macro is not used, the cog should be explicitly
unregistered instead.
Other Changes
-------------
1. The Catapult "start" macros (i.e. RESERVE_AND_START, START_RESERVED,
START_FIXED and START_OVERLAY) were always using the value ANY_COG
for the cog to be started instead of using the value passed to the
macro in the 'cog' argument. Affected both the Propeller 1 and the
Propeller 2.
2. When the COMPACT kernel was dynamically loaded, the data block used
to transfer data to/from the Floating point plugin was not being set up
correctly, and so the first two longs of Hub RAM were being used - which
would overwrite the Frequency and clock mode, and break any functions
that relied on that value, such as _clockfreq(), _waitcnt() etc.
Apologies for the delay in release 7.6.1 - it is now available again. I have not had as much time as I thought I would have, so instead of holding up the release any further I have pared it back to just the necessary bug fixes. I will add the other changes (fairly minor) into a future release as I get time.
Comments
Update: I was wrong about not being able to execute C code that used floating point from the LUT. The C code executed from the LUT can include any C code that will fit, including all the basic floating point operations - it just cannot call any other C functions (except those that can be 'inlined' by the optimizer).
This means it cannot use any standard C library functions, which includes the floating point library functions.
Removing this limitation will take a lot of work for not much extra benefit, so I will just included it "as is" in the next release.
If anyone can think of a cool demo for this capability then please let me know. My current examples are all fairly trivial.
Ross.
REP loops will benefit. Stuff like bit-bashed SPI can run much tighter. So much so that I/O staging latencies will need accounted for in the rx phase-lag timing.
Well, this LUT C execution lark has been an interesting little exercise.
It turns out to be easy to remove all the limitations on executing C from the LUT, except (naturally) for the size limitation. I can have LUT C able to do anything normal C can do, including calling other C functions (LUT or normal).
The downside is that to make it work I have to remove some of the code size reduction techniques I use, so the code ends up being larger. And the speed improvements of LUT execution over Hub execution are very marginal at best. Particularly for C code small enough to fit in the LUT in the first place.
So I have decided not to bother. LUT execution of 'leaf' C code will be supported for NATIVE C programs in the next release (since I get that for nothing) but the main use for LUT execution is for executing PASM, not C.
Ross.
Catalina 7.5 has been released here.
This is a full release. The main new feature is addition of the LUT Execution macros, which can now be used in all Propeller 2 memory models (NATIVE, COMPACT, TINY, XMM SMALL or XMM LARGE).
Also, I have now added Lua to the Catalina libraries. While not a new feature (Lua has been supported since the Propeller 1 days), this release makes it much easier to build C programs with embedded Lua scripting. Almost completely trivial, in fact - a fully working demo program is less than ten lines of C code:
#include <lua.h> #include <lualib.h> #include <lauxlib.h> int main(int argc, char *argv[]) { // create a new Lua state lua_State *L = luaL_newstate(); // open the standard Lua libraries luaL_openlibs(L); // execute the script contained in the file "script.lua" luaL_dofile(L, "script.lua"); }And here is an example script.lua that this program might load and execute:
# a simple Lua script print('Hello, World (from "script.lua")\n'); io.write("enter number A: "); A = io.read("n"); io.write("enter number B: "); B = io.read("n"); io.write("A + B = " .. A + B .. "\n");Here is the relevant extract from the README.TXT:
RELEASE 7.5 New Functionality ----------------- 1. Lua has now been integrated into the Catalina library. This makes it much easier to build C programs that embed Lua. Previously to do this the whole Lua distribution had to be compiled with the C program, but now only the Lua initialization module (linit.c) is required because the rest of Lua can be included by simply specifying either -llua or -lluax on the Catalina command line. Examples of doing this (and an explanation of the difference between -llua and -lluax) are provided in a new demo folder called demos/lua. See the README.TXT file in that folder for more details. This also means that building C programs that embed Lua is now possible using the self-hosted version of Catalina on the Propeller 2. However, while this is now supported, it is not yet very practical since it takes hours to compile even a very simple C program that embeds Lua (lhello.c). Note that the Catalyst version of Lua does NOT use the compiled version in the library. This is because some of the options offered by the Catalyst version of Lua (e.g. ENABLE_PSRAM, ENABLE_HYPER) require Lua to be recompiled from source. Applies to Windows and Linux for the Propeller 1 and 2, and Catalyst for the Propeller 2. 2. The self-hosted version of catalina has an additional option -W which can be used to send options to the compiler (specifically, to rcc). For instance, to suppress warnings use the option -W-w which will pass the option -w to rcc. Applies to Catalyst on the Propeller 2. 3. A new include file has been added (lut_exec.h) which simplifies the definition of inline PASM and C code to be loaded and executed from the LUT (i.e. using LUT execution mode). Code executed from the LUT is limited to 254 longs. Inline PASM executed from the LUT is supported in all memory models. C code executed from the LUT is only supported for the NATIVE memory model, and has the additional limitation that it must be 'leaf' code - i.e. it cannot call any other C functions (except those that can be 'inlined' by the optimizer). Demos of LUT execution have been provided in a new demos/lut_exec folder. Both PASM and C examples are included. Applies to the Propeller 2 only. Other Changes -------------- 1. The default Propeller 2 clock parameters are specified in each platform file (e.g. P2_EDGE.inc, P2_EVAL.inc, P2_CUSTOM.inc etc). These could be overridden on the command line via the -f, -F & -E command line options, which are used to calculate appropriate values for _XDIV, _XMUL and _XDIVP (see the platform include files or the Propeller specifications for more details on these values). The defaults in previous releases were as follows: _XDIV = 4 '\ crystal divider to give 5.0MHz _XMUL = 72 '| crystal / div * mul to give 360MHz _XDIVP = 2 '/ crystal / div * mul / divp to give 180MHz These defaults have been changed in this release to: _XDIV = 1 '\ crystal divider to give 20MHz _XMUL = 9 '| crystal / div * mul to give 180MHz _XDIVP = 1 '/ crystal / div * mul / divp to give 180MHz This results in the same default clock frequency (180Mhz) but the new values should mean the clock is more stable. Applies to the Propeller 2 only. 2. Eliminated errors and warnings issued when building the Catalina libraries. Most of these were because some of the libraries were being compiled for a Propeller platform (1 or 2) that did not support them. Now only the supported libraries are compiled for each Propeller platform. 3. Eliminated warnings about deprecated code when compiling catdbgfilegen, which is the Catalina utility that generates debugging information.Ross.
I just noticed there are two different sets of instructions for rebuilding Catalina under Linux. The instructions in README.Linux are out of date. The up-to-date instructions are the ones in the file BUILD.TXT.
Ross.
Catalina's multi-model capability - e.g. being able to define one part of a program to execute as XMM code while another part executes as NATIVE code - is potentially one of Catalina's most powerful features - but it can be quite complex to figure out how to compile such programs correctly. Even I struggle with it, and have to re-learn it every time I use it.
I have been working on something to make it simpler to do, and the result is Catalina Catapult - now available as part of Catalina 7.6.
Here is an extract of the documentation - which is now all contained in the document Getting Started with Catapult:
Catalina Catapult is a utility intended to simplify the process of developing, debugging and maintaining Catalina multi-model programs. A multi-model program typically consists of a primary program in which speed is not critical (and so it can be executed from XMM RAM) which then loads and executes subsidiary programs that execute in Hub RAM whenever speed, precise timing or functionality not supported by XMM programs - e.g. multi-threading or interrupts (on the Propeller 2) is required. Alternatively, the primary program may consist of a Hub-based primary program which loads other Hub-based secondary programs as overlays from files as necessary. In either case, the main advantage of multi-model programs is that secondary programs do not consume Hub RAM except when they are actually executing. Multi-model support therefore allows C programs larger than Hub RAM to execute while still allowing them to access all the functionality of the Propeller when needed. The main disadvantage of multi-model programs is that they are more difficult to develop, build, debug and maintain than ordinary C programs. Catapult helps by allowing multi-model programs to be developed and kept in a single C source file that contains both the primary program and all the secondary programs. Catapult also helps by splitting the source file into separate program files, compiling each one using the appropriate memory model and options, invoking the utility required to turn these into data files to be loaded when required by the primary program, and then compiling the primary program to produce a single binary ready for execution. The best way to see how Catapult simplifies this process is to build and run a demo program. There are several versions of the same demo program in this folder - some specifically for the Propeller 1 (e.g. demo_p1.c) and others specifically for the Propeller 2 (e.g. demo_p2.c). The reason there are different versions of the program for the Propeller 1 and Propeller 2 is that the Propeller 1 and 2 support different memory models and different address ranges - both of which are important when building multi-model programs. The catapult command always accepts a single C source file name as its argument. To compile the Propeller 1 version of the demo program: catapult demo_p1.c or, to compile the Propeller 2 version of the demo program: catapult demo_p2.c Both these commands will take the specified C source file, split it up (in this case) into one primary and two secondary programs, compile those programs separately, and then combine them back into a single output binary. A command file will be created called either _catapult.bat (Windows) or _catapult.cmd (Linux). Doing this manually is quite possible, but it can be quite complex. For an example of this, see the Makefile in the demos\multimodel folder. Catapult eliminates the need to write such complex Makefiles. Examine the command file generated by building the demo program (_catapult.bat or _catapult.cmd) to see that Catapult generates the necessary commands for (in this case) the three Catalina commands and two invocations of the spinc utility required to build the final binary. The resulting binary program can be loaded and executed using payload. For the Propeller 1: payload -i demo_p1 or, for the Propeller 2: payload -i demo_p2 Note that if you compile BOTH the Propeller 1 and Propeller 2 versions, you may need to add the extension in the payload command (i.e. .binary for the Propeller 1, or .bin for the Propeller 2) or else use the -o1 or -o2 payload option to specify whether you are loading a Propeller 1 or Propeller 2. Now for the details ... Catapult pragmas ================ Catapult is implemented by adding pragmas to the C source file. There are three catapult pragmas: #pragma catapult common [ name ] [ attributes ] This pragma introduces a common segment, which should contain all the definitions and types that are common to the primary and all the secondary programs - the common segment will be converted into a C header file shared by all other segments. If no name is specified, the default name is "common". Note that all code not included in any other segment will be included in the common segment, even if that code precedes the actual pragma (or there is no common pragma). The common segment should include only C definitions and types, and not data, variables or functions - those should be specified in the appropriate primary or secondary segment. #pragma catapult secondary [ name [ (type) ] ] [ attributes ] This pragma introduces a secondary segment, which should contain all the file scope data and code required for a secondary program, but no C main function. Instead, the segment should include a C function with the name of the segment that accepts a pointer to the specified type. The secondary segment is converted into a C program file. If no name is specified, the default name is "secondary". If no type is specified, the default type is the name of the segment with "_t" appended. There can be none, one or more secondary segments. Each secondary segment must be given a unique name. Each secondary segment should include all the data, variables and functions required by the secondary function. See also the section below on Sharing Data between Catapult Segments. #pragma catapult primary [ name ] [attributes ] This pragma introduces the primary segment, which should contain all the file scope data, variables and code required for the primary program, including a C main function. The primary segment is converted into a C program file that includes either the binaries of all the secondary segments as an array of longs, or else the name of the overlay files containing this data. If no name is specified, the default name is "primary". There must be exactly one primary segment. If there are multiple primary pragmas, they must all be identical. See also the section below on Sharing Data between Catapult Segments. Each pragma can specify additional attributes, which are described below. There can be multiple instances of each pragma, but only one segment of each type will be created with each specified segment name, so each instance of a pragma type (common, secondary or primary) with the same name must be identical. Catapult will warn you if this is not the case. The primary and secondary segment names must be unique, but the common segment may share the same name as another segment since the common segment is generated as a C header file (.h), and the primary and secondary segments are generated as C programs files (.c). All the code introduced by the same pragma type with the same name will be combined in the same output file for compilation. For the common segment this file will be given the name of the segment plus a ".h" extension. For a primary or secondary segment it will have the segment name plus a ".c" extension. Catapult will complain if this would result in the same file name as the source file, to prevent accidentally overwriting that file. However, the binary output can be given that name by using the 'binary' attribute. Catapult pragma attributes ========================== Each of the catapult pragmas can also include various attributes: address(value) - specifies the address to use for a secondary segment. The value can be a decimal value (e.g. 16384) or a hex value (e.g. 0x4000). This attribute applies only to secondary pragmas. stack(value) - specifies the stack size to use for a secondary segment. The value should be a decimal value (e.g. 500). This attribute applies only to secondary pragmas. mode(name) - specifies the memory model to be used for the segment. For secondary segments, the mode name can be any of: COMPACT or CMM : use the COMPACT memory model TINY or LMM : use the TINY memory model NATIVE or NMM : use the NATIVE memory model (P2 only) For the primary segment, this can also be: SMALL or XMM SMALL : use the SMALL memory model LARGE or XMM LARGE or XMM : use the LARGE memory model options(list) - specifies Catalina command line options to be added to the catalina command when compiling the segment. For example: options(-lc -lm -C TTY -O5) All Catalina command line options are valid, but some should not be used since catapult will generate them automatically, such as -o for the name of the output. To define Catalina symbols use the usual -C syntax, and to define C symbols, use the usual -D syntax. Note that options can be specified on common, secondary or primary pragmas - any options specified on common pragmas will be combined with the options specified for the primary or secondary pragma when the relevant programs are compiled. For instance, if a platform is included in a common pragma - e.g. options(-C C3) - it will apply to the compilations of both primary and secondary programs. binary(name) - specifies the name of the binary output to be generated. Can be used in primary and secondary pragmas. If not specified, then the name of the segment is used. Use this attribute to name the binary output instead of specifying -o in the options attribute. Do not include an extension - the appropriate extension (i.e. .binary or .bin) will be added automatically. overlay(name) - specifies the name of an overlay file to be generated. Can be used in secondary pragmas only. Use this attribute to generate a file containing the overlay instead of an array of longs in the primary programs data segment. The overlay files must be placed on an SD card, and will be loaded automatically at execution time whenever the secondary program is started. The overlay name can include an extension. Note that this attribute CAN - but does not HAVE TO - be used in conjunction with the OVERLAY macros (described later). See the section on using Catapult with Overlay Files for more details. The names used in the pragmas cannot be the same as any of the possible attributes (so for example, 'address' cannot be used as a segment name).Just as a final note to highlight what this is all about without diving into any technical details ... let's say that during your program development you run out of Hub RAM (all too easy to do on the Propeller 1) and realize that you have to change your primary program so that it runs from XMM RAM instead of Hub RAM. What would you have to do?
This is now literally a one word change. For instance, in demo_p1.c you just change this line:
#pragma catapult primary main mode(COMPACT) binary(demo_p1)... to this ...
#pragma catapult primary main mode(SMALL) binary(demo_p1)... or this ...
#pragma catapult primary main mode(LARGE) binary(demo_p1)... and then re-execute the catapult command:
catapult p1_demo.c... and you're done.
Ross.
EDIT: Update documentation extract and remove Windows binary - Catapult is now available for both Windows and Linux as part of Catalina 7.6
Hi @RossH,
Catapult looks very interesting and a great timesaver if I can get it to work on my FLiP.
Let's quickly revisit our friendly secondary.c and primary.c programs posted under The Great printf() Challenge topic.
Start by looking at the secondary.c segment:
//segment is secondary.c //last revision on 07Nov23 //catalina secondary.c -lc -lm -lserial4 -C NO_HMI -C NO_ARGS -C FLIP -C COMPACT -R 0x57d8 -M64k -y //spinc -B2 -n SECONDARY -s 512 -c -l secondary.binary > secondary.inc #include <stdio.h> #include <stdlib.h> #include <string.h> #include <stdarg.h> #include <math.h> #include <time.h> #include <propeller.h> #define Yes 1 #define No 0 #define conport 0 #define print_command 1 #define ROUNDUP 128 struct shared { char request; char *format; char *outstr; va_list ptr; }; struct shared *secondary; char outstr[256]; void secprint(char *format,...) { va_list ptr; va_start(ptr,format); (*secondary).format=format; (*secondary).ptr=ptr; (*secondary).outstr=&outstr[0]; (*secondary).request=print_command; while((*secondary).request != 0); s4_str(conport,outstr); } void main(struct shared *primary) { float theta; secondary=primary; for(;;) { for(theta=0.0; theta<6.283185307; theta+=0.0062831853) { secprint("theta=%f,sine=%f,cosine=%f\r\n",theta,sin(theta),cos(theta)); } } }And the way we originally compiled secondary.c:
Now, let's look at primary.c:
//segment is primary.c //last revision on 07Nov23 // catalina primary.c -lc -lm -lserial4 -C NO_HMI -C FLIP -C XEPROM -C SMALL -C CACHED_4K -y #include <stdio.h> #include <stdlib.h> #include <string.h> #include <stdarg.h> #include <math.h> #include <time.h> #include <propeller.h> #include "secondary.inc" #define Yes 1 #define No 0 #define conport 0 #define print_command 1 #define ROUNDUP 128 void check_memory(void *secondary_memory) { if(SECONDARY_CODE_ADDRESS == (int) secondary_memory) return; for(;;) { s4_str(conport,"Error - Change secondary Memory To 0x"); s4_hex(conport,secondary_memory,4); s4_str(conport,"\r\n"); while(s4_rxcheck(conport) < 0); } } struct shared { char request; char *format; char *outstr; va_list ptr; }; static struct shared primary; void priprint(void) { vsprintf(primary.outstr,primary.format,primary.ptr); va_end(primary.ptr); primary.request=0; } void main(void) { char SECONDARY_RESERVED_SPACE[ROUNDUP * ((SECONDARY_RUNTIME_SIZE + ROUNDUP -1)/ROUNDUP)]; check_memory(SECONDARY_RESERVED_SPACE); start_SECONDARY(&primary,ANY_COG); for(;;) { if(primary.request == print_command) priprint(); } }And here's how we compiled primary.c:
If we got the memory location wrong for the secondary.c code the program will let us know at runtime what the proper address should be so we can correct it and go through this whole compiling process again.
I combined the secondary.c and primary.c code into a new file called prisec.c:
//Program is prisec.c //Combines the previous secondary.c and primary.c programs into one //Last revision on 22 June 2024 //Attempting to compile using Catapult but without success... #pragma catapult common #include "catapult.h" #include <stdio.h> #include <stdlib.h> #include <string.h> #include <stdarg.h> #include <math.h> #include <time.h> #include <propeller.h> #define Yes 1 #define No 0 #define conport 0 #define print_command 1 #define ROUNDUP 128 struct shared { char request; char *format; char *outstr; va_list ptr; }; #pragma catapult secondary secmain mode(COMPACT) address(0x57d8) stack(512) options(-lc -lm -lserial4 -C NO_HMI -C NO_ARGS -C FLIP -M64k -y) struct shared *secondary; char outstr[256]; void secprint(char *format,...) { va_list ptr; va_start(ptr,format); (*secondary).format=format; (*secondary).ptr=ptr; (*secondary).outstr=&outstr[0]; (*secondary).request=print_command; while((*secondary).request != 0); s4_str(conport,outstr); } void secmain(struct shared *primary) { float theta; secondary=primary; for(;;) { for(theta=0.0; theta<6.283185307; theta+=0.0062831853) { secprint("theta=%f,sine=%f,cosine=%f\r\n",theta,sin(theta),cos(theta)); } } } #pragma catapult primary main mode(SMALL) options(-lc -lm -lserial4 -C NO_HMI -C FLIP -C XEPROM -C CACHED_4K -y) binary(primary) struct shared primary; /* void check_memory(void *secondary_memory) { if(SECONDARY_CODE_ADDRESS == (int) secondary_memory) return; for(;;) { s4_str(conport,"Error - Change secondary Memory To 0x"); s4_hex(conport,secondary_memory,4); s4_str(conport,"\r\n"); while(s4_rxcheck(conport) < 0); } } */ void priprint(void) { vsprintf(primary.outstr,primary.format,primary.ptr); va_end(primary.ptr); primary.request=0; } void main(void) { // char SECONDARY_RESERVED_SPACE[ROUNDUP * ((SECONDARY_RUNTIME_SIZE + ROUNDUP -1)/ROUNDUP)]; // check_memory(SECONDARY_RESERVED_SPACE); // start_SECONDARY(&primary,ANY_COG); int cogsec; FIXED_START(secmain,&primary,ANY_COG,cogsec); for(;;) { if(primary.request == print_command) priprint(); } }But when I ran Catapult I got nothing but errors:
I did comment out the check_memory() function within the primary portion which previously did the memory checking under the assumption that Catapult might do it for me instead.
The prisec.c code won't compile, much less run. Could you look over the code and see what I did wrong? Thanks.
Catalina 7.6 has been released here.
This is a full release. The main new feature is addition of the Catalina Catapult utility. See the previous post for a quick overview of Catapult, or download the document Getting Started with Catapult from the link above.
Here is the relevant extract of the README.TXT file:
RELEASE 7.6 New Functionality ----------------- 1. Catalina Catapult is a utility intended to simplify the process of developing, debugging and maintaining Catalina multi-model programs. For details, see the document Getting Started with Catapult, and/or the demo programs in the demos/catapult folder. 2. The C library now has two overlay load functions. The existing function: _load_overlay() - which uses the C stdio file system. And a new function: _load_overlay_unmanaged() - which uses the Catalina file system. The new Catalina file system version takes much less Hub RAM than the previous version, and is recommended for use on the Propeller 1 for programs that do not otherwise require stdio. However, it requires the _mount() function to be called before any overlays are loaded. Typically, this would be called in the main program as: _mount(0, 0); 3. The spinc utility now generates code that can use either the existing _load_overlay() function or the new _load_overlay_unamanaged() function depending on whether the Catalina symbol FS_OVERLAY is defined. For example: catalina -C FS_OVERLAY -lcx overlay.c The difference is that while the unmanaged version still needs to be compiled with the extended file system (e.g. -lcix or -lcx) it uses the Catalina file system functions, which are much smaller than the stdio file system functions. However, note that the _mount() function must be called before any overlays are loaded. Typically, this would be called in the main program as: _mount(0, 0); 4. The spinc utility now accepts hex values (e.g. 0x200) as arguments for the -s stack size command line option. For example: spinc -s 0x200 program.binary > xxx.inc 5. Catalina now accepts the definition of the Catalina symbol P2 to mean the same as the -p2 command line option. This allows the propeller version to be specified using CATALINA_DEFINE. Other Changes -------------- 1. None.Will do. But probably not till the weekend.
Hello @Wingineer19
Here is some amended code - it now compiles using the command
catapult prisec.c//Program is prisec.c //Combines the previous secondary.c and primary.c programs into one //Last revision on 22 June 2024 #pragma catapult common options(-lc -lm -lserial4 -C NO_HMI -C FLIP -y) #include "catapult.h" #include <stdio.h> #include <stdlib.h> #include <string.h> #include <stdarg.h> #include <math.h> #include <time.h> #include <propeller.h> #define Yes 1 #define No 0 #define conport 0 #define print_command 1 #define ROUNDUP 128 typedef struct shared { char request; char *format; char *outstr; va_list ptr; } shared_t; #pragma catapult secondary secmain(shared_t) mode(COMPACT) address(0x57d8) stack(512) struct shared *secondary; char outstr[256]; void secprint(char *format,...) { va_list ptr; va_start(ptr,format); (*secondary).format=format; (*secondary).ptr=ptr; (*secondary).outstr=&outstr[0]; (*secondary).request=print_command; while((*secondary).request != 0); s4_str(conport,outstr); } void secmain(struct shared *primary) { float theta; secondary=primary; for(;;) { for(theta=0.0; theta<6.283185307; theta+=0.0062831853) { secprint("theta=%f,sine=%f,cosine=%f\r\n",theta,sin(theta),cos(theta)); } } } #pragma catapult primary main mode(SMALL) binary(primary) options(-C XEPROM -C CACHED_4K) struct shared primary; /* void check_memory(void *secondary_memory) { if(SECONDARY_CODE_ADDRESS == (int) secondary_memory) return; for(;;) { s4_str(conport,"Error - Change secondary Memory To 0x"); s4_hex(conport,secondary_memory,4); s4_str(conport,"\r\n"); while(s4_rxcheck(conport) < 0); } } */ void priprint(void) { vsprintf(primary.outstr,primary.format,primary.ptr); va_end(primary.ptr); primary.request=0; } void main(void) { // char SECONDARY_RESERVED_SPACE[ROUNDUP * ((SECONDARY_RUNTIME_SIZE + ROUNDUP -1)/ROUNDUP)]; // check_memory(SECONDARY_RESERVED_SPACE); // start_SECONDARY(&primary,ANY_COG); int cogsec; FIXED_START(secmain,primary,ANY_COG,cogsec); for(;;) { if(primary.request == print_command) priprint(); } }There were three issues:
typedef struct shared { char request; char *format; char *outstr; va_list ptr; } shared_t;#pragma catapult secondary secmain(shared_t) mode(COMPACT) address(0x57d8) stack(512)FIXED_START(secmain,primary,ANY_COG,cogsec);Also (but this one is just a matter of style, intended to simplify maintenance) I have taken all the common options out of the primary and secondary pragmas, and added them to the common pragma instead - i.e.:
#pragma catapult common options(-lc -lm -lserial4 -C NO_HMI -C FLIP -y)Finally, note that some of the options you had (e.g.
-C NO_ARGS) are no longer required because they are added automatically by catapult where required.I have not run it - let me know how you go.
Ross.
Hi RossH,
I was able to get it to compile fine, but after uploading to the FLiP it did absolutely nothing. No errors, no warnings, nothing.
Since my code uses the 4-Port Serial Driver, I even edited the CATAPULT_ERROR to this:
#ifndef CATAPULT_ERROR #define CATAPULT_ERROR(name, address) \ { \ s4_str(1, "Error: secondary "); \ s4_str(1, name); \ s4_str(1, " must specify address(0x"); \ s4_hex(1, (int)address); \ s4_str(1, ")\n"); \ } #endifBut unfortunately, it still didn't work. No error display but no proper operation, either.
I suspect the Secondary isn't being loaded to the proper address. Perhaps the CATAPULT_ERROR is immediately sent but the display screen isn't fast enough to show me the memory location error right after EEPROM upload.
And if the CATAPULT_ERROR is only sent once, if I miss it then I miss it and that's too bad so sad...
If you look at my check_memory() function which is commented out, you will notice that if the memory location is wrong, the code enters into an infinite loop and will display the error every time the operator hits the ENTER key on the keyboard. So, if the first time the error was displayed is missed, the operator just hits the ENTER key and it will display it again.
Can the CATAPULT_ERROR be modified to do something similar?
Hello @Wingineer19
You were on the right track - you do indeed have to redefine CATAPULT_ERROR - but you have to #undef it first - your code does not do that.
Also, I modified your program to use RESERVED_START instead of FIXED_START (which makes it more like your original).
Here is a version that works ok on my C3:
//Program is prisec.c //Combines the previous secondary.c and primary.c programs into one //Last revision on 22 June 2024 #pragma catapult common options(-lc -lm -lserial4 -C NO_HMI -C FLIP -y) #include "catapult.h" #include <stdio.h> #include <stdlib.h> #include <string.h> #include <stdarg.h> #include <math.h> #include <time.h> #include <propeller.h> #define Yes 1 #define No 0 #define conport 0 #define print_command 1 #define ROUNDUP 128 typedef struct shared { char request; char *format; char *outstr; va_list ptr; } shared_t; // since we are not using a HMI, we must redefine CATAPULT_ERROR #undef CATAPULT_ERROR #define CATAPULT_ERROR(name, address) \ {\ s4_str(conport, "Catapult Error : "); s4_str(conport, name); \ s4_str(conport, " should be at 0x"); s4_hex(conport, (int)address, 6); \ s4_str(conport, "\n"); \ } #pragma catapult secondary secmain(shared_t) mode(COMPACT) address(0x57FC) stack(512) struct shared *secondary; char outstr[256]; void secprint(char *format,...) { va_list ptr; va_start(ptr,format); (*secondary).format=format; (*secondary).ptr=ptr; (*secondary).outstr=&outstr[0]; (*secondary).request=print_command; while((*secondary).request != 0); s4_str(conport,outstr); } void secmain(struct shared *primary) { float theta; secondary=primary; for(;;) { for(theta=0.0; theta<6.283185307; theta+=0.0062831853) { secprint("theta=%f,sine=%f,cosine=%f\r\n",theta,sin(theta),cos(theta)); } } } #pragma catapult primary main mode(SMALL) binary(primary) options(-C XEPROM -C CACHED_4K) struct shared primary; void priprint(void) { vsprintf(primary.outstr,primary.format,primary.ptr); va_end(primary.ptr); primary.request=0; } void main(void) { RESERVE_SPACE(secmain); int cogsec; RESERVED_START(secmain,primary,ANY_COG,cogsec); for(;;) { if(primary.request == print_command) priprint(); } }Ross.
Hi RossH,
Excellent. It works great!
I did modify the CATAPULT_ERROR to do what I wanted -- it will continually display the error message whenever any key is hit.
Hence, if for some reason the error message was initially missed, all one needs to do is hit any key to get the message displayed again.
//Program is prisec.c //Combines the previous secondary.c and primary.c programs into one //Last revision on 27 June 2024 #pragma catapult common options(-lc -lm -lserial4 -C NO_HMI -C FLIP -y) #include "catapult.h" #include <stdio.h> #include <stdlib.h> #include <string.h> #include <stdarg.h> #include <math.h> #include <time.h> #include <propeller.h> #define Yes 1 #define No 0 #define conport 0 #define print_command 1 #define ROUNDUP 128 typedef struct shared { char request; char *format; char *outstr; va_list ptr; } shared_t; // since we are not using a HMI, we must redefine CATAPULT_ERROR #undef CATAPULT_ERROR #define CATAPULT_ERROR(name,address)\ {\ for(;;) \ { \ s4_str(conport,"Catapult Error -- Change The "); \ s4_str(conport,name); \ s4_str(conport," Memory Location To 0x"); \ s4_hex(conport,(int) address,6); \ s4_str(conport,"\r\n"); \ while(s4_rxcheck(conport) < 0); \ } \ } #pragma catapult secondary secmain(shared_t) mode(COMPACT) address(0x1234) stack(512) struct shared *secondary; char outstr[256]; void secprint(char *format,...) { va_list ptr; va_start(ptr,format); (*secondary).format=format; (*secondary).ptr=ptr; (*secondary).outstr=&outstr[0]; (*secondary).request=print_command; while((*secondary).request != 0); s4_str(conport,outstr); } void secmain(struct shared *primary) { float theta; secondary=primary; for(;;) { for(theta=0.0; theta<6.283185307; theta+=0.0062831853) { secprint("theta=%f,sine=%f,cosine=%f\r\n",theta,sin(theta),cos(theta)); } } } #pragma catapult primary main mode(SMALL) binary(primary) options(-C XEPROM -C CACHED_4K) struct shared primary; void priprint(void) { vsprintf(primary.outstr,primary.format,primary.ptr); va_end(primary.ptr); primary.request=0; } void main(void) { RESERVE_SPACE(secmain); int cogsec; RESERVED_START(secmain,primary,ANY_COG,cogsec); for(;;) { if(primary.request == print_command) priprint(); } }In this example notice that I've deliberately set the secmain secondary address to 0x1234.
Now, after compiling and uploading it will display this error and will continue to do so whenever any key is hit. So if you hit any key 100 times then you will get this message displayed 100 times:
It looks like Catapult will be a great timesaver and really simplifies the compiling process.
Before, I had to manually compile the secondary.c file.
Then run spinc on it to generate the secondary.inc file.
Then compile the primary.c file to generate the primary.binary output and upload it to the Prop.
If the memory location was wrong, I had to change it to the proper location and go through this whole process again.
Now, I only need to use Catapult on a single file after making the change. Very easy to do now and eliminates mistakes during the compiling process -- like forgetting to run spinc which was a common mistake.
One thing to keep in mind is that the CATAPULT_ERROR will need to be redefined for whatever operator interface is being used. Here, it's redefined for the 4-Port serial driver, but the same will be needed for the tty and tty256 and the 8-Port serial driver for the Prop2.
I'll continue to experiment with Catapult and let you know how it goes.
Hi RossH,
I added the Optimizer to the mix as part of the common options:
//Program is prisec.c //Combines the previous secondary.c and primary.c programs into one //Last revision on 27 June 2024 #pragma catapult common options(-lc -lm -lserial4 -O5 -C NO_HMI -C FLIP -y) #include "catapult.h" #include <stdio.h> #include <stdlib.h> #include <string.h> #include <stdarg.h> #include <math.h> #include <time.h> #include <propeller.h> #define Yes 1 #define No 0 #define conport 0 #define print_command 1 #define ROUNDUP 128 typedef struct shared { char request; char *format; char *outstr; va_list ptr; } shared_t; // since we are not using a HMI, we must redefine CATAPULT_ERROR #undef CATAPULT_ERROR #define CATAPULT_ERROR(name,address)\ {\ for(;;) \ { \ s4_str(conport,"Catapult Error -- Change The "); \ s4_str(conport,name); \ s4_str(conport," Memory Location To 0x"); \ s4_hex(conport,(int) address,6); \ s4_str(conport,"\r\n"); \ while(s4_rxcheck(conport) < 0); \ } \ } #pragma catapult secondary secmain(shared_t) mode(COMPACT) address(0x1234) stack(512) struct shared *secondary; char outstr[256]; void secprint(char *format,...) { va_list ptr; va_start(ptr,format); (*secondary).format=format; (*secondary).ptr=ptr; (*secondary).outstr=&outstr[0]; (*secondary).request=print_command; while((*secondary).request != 0); s4_str(conport,outstr); } void secmain(struct shared *primary) { float theta; secondary=primary; for(;;) { for(theta=0.0; theta<6.283185307; theta+=0.0062831853) { secprint("theta=%f,sine=%f,cosine=%f\r\n",theta,sin(theta),cos(theta)); } } } #pragma catapult primary main mode(SMALL) binary(primary) options(-C XEPROM -C CACHED_4K) struct shared primary; void priprint(void) { vsprintf(primary.outstr,primary.format,primary.ptr); va_end(primary.ptr); primary.request=0; } void main(void) { RESERVE_SPACE(secmain); int cogsec; RESERVED_START(secmain,primary,ANY_COG,cogsec); for(;;) { if(primary.request == print_command) priprint(); } }The first time you use Catapult to compile and run you will get this error:
After making the change, compiling using Catapult, and uploading to the Prop, you won't receive an error.
But the code doesn't appear to be doing anything, either.
Then, after about a minute or so, it will start displaying the theta, sine, and cosine values.
Any idea why using the Optimizer would introduce a huge time delay before the code would start executing?
No. I've used the optimizer on my own programs with no issues. I'll investigate.
EDIT: Yes, I see the same thing as you. Odd. But it's probably not catapult, which just manages the compilation process. It could be an optimizer quirk - I'll have to examine the optimized code.
EDIT: A clue is that the delay is about 53 seconds - which is the delay you get when you do a WAITCNT for a count that has just passed and have to wait for the 32 bit counter to "go round the clock". This means I think I know where the problem is - I'll confirm it and issue a patch.
Ross.
Hello @Wingineer19
Still working on this issue. As I suspected, it is a WAITCNT instruction going astray, but it is deeper than that - the instruction is not working because of a memory corruption.
So far, I can tell you what the problem is NOT:
On my C3, the program works when executed from Hub RAM, FLASH or SRAM, and when it is executed from EEPROM it is being loaded correctly. I have two possibilities still to investigate: (1) the XEPROM XMM drivers, and (2) the variable argument macros (see below).
Since it is so repeatable, I am sure I will track it down.
A couple of things to note about this particular program:
Ross.
Hi RossH,
In light of your discovery, I decided to do a regression and split prisec.c back into its constituent parts, namely secondary.c and primary.c.
Here's secondary.c:
//segment is secondary.c //last revision on 07Nov23 //catalina secondary.c -lc -lm -lserial4 -O5 -C NO_HMI -C NO_ARGS -C FLIP -C COMPACT -R 0x5ad4 -M64k -y //spinc -B2 -n SECONDARY -s 512 -c -l secondary.binary > secondary.inc #include <stdio.h> #include <stdlib.h> #include <string.h> #include <stdarg.h> #include <math.h> #include <time.h> #include <propeller.h> #define Yes 1 #define No 0 #define conport 0 #define print_command 1 #define ROUNDUP 128 struct shared { char request; char *format; char *outstr; va_list ptr; }; struct shared *secondary; char outstr[256]; void secprint(char *format,...) { va_list ptr; va_start(ptr,format); (*secondary).format=format; (*secondary).ptr=ptr; (*secondary).outstr=&outstr[0]; (*secondary).request=print_command; while((*secondary).request != 0); s4_str(conport,outstr); } void main(struct shared *primary) { float theta; secondary=primary; for(;;) { for(theta=0.0; theta<6.283185307; theta+=0.0062831853) { secprint("theta=%f,sine=%f,cosine=%f\r\n",theta,sin(theta),cos(theta)); } } }And here's primary.c:
//segment is primary.c //last revision on 07Nov23 // catalina primary.c -lc -lm -lserial4 -O5 -C NO_HMI -C FLIP -C XEPROM -C SMALL -C CACHED_4K -y #include <stdio.h> #include <stdlib.h> #include <string.h> #include <stdarg.h> #include <math.h> #include <time.h> #include <propeller.h> #include "secondary.inc" #define Yes 1 #define No 0 #define conport 0 #define print_command 1 #define ROUNDUP 128 void check_memory(void *secondary_memory) { if(SECONDARY_CODE_ADDRESS == (int) secondary_memory) return; for(;;) { s4_str(conport,"Error - Change secondary Memory To 0x"); s4_hex(conport,secondary_memory,4); s4_str(conport,"\r\n"); while(s4_rxcheck(conport) < 0); } } struct shared { char request; char *format; char *outstr; va_list ptr; }; static struct shared primary; void priprint(void) { vsprintf(primary.outstr,primary.format,primary.ptr); va_end(primary.ptr); primary.request=0; } void main(void) { char SECONDARY_RESERVED_SPACE[ROUNDUP * ((SECONDARY_RUNTIME_SIZE + ROUNDUP -1)/ROUNDUP)]; check_memory(SECONDARY_RESERVED_SPACE); start_SECONDARY(&primary,ANY_COG); for(;;) { if(primary.request == print_command) priprint(); } }This means I need to compile secondary.c first, then run spinc, then compile primary.c, then upload the primary.binary to the Prop using payload.
This time, I include the optimizer option -O5 within each compile command for secondary.c and primary.c, respectively.
Compile command for secondary.c:
Followed by the spinc command:
Compile command for primary.c:
Then finally upload the primary.binary to the Prop:
After going through the usual memory relocation process, and after uploading the memory corrected compiled code, the code begins to immediately execute (within just a few seconds or so) and shows theta, sine, and cosine.
If the va_list and va_start macros are causing problems, they only seemed to surface when using Catapult and not with the individual compiling process used above.
It could be that the -lm option is causing the problem but I left it in both compile commands and the code still works.
Just thought I would add this test to the mix to muddy the water even more
All good. I don't think it is catapult (which basically just does what you are doing manually) and I have done some more investigating into the variable argument macros and I think your use of them is basically ok. However, code like this would probably not be portable to another C compiler. The C manual says:
While technically you are using the variable initialized by va_start() before va_end() is invoked, you are doing so in another C program compiled separately. I don't think the designers of the macros considered that case!
The -lm issue doesn't matter as long as your primary program is an XMM program. But it does when I turn it into an LMM program to run it entirely from Hub RAM.
I am on the track of another possibility now. Will let you know what I find.
Ross.
LOL, yes, I had to do some experimentation with the code to get it to do what I wanted it to.
However I did something similar in Windows using the OpenWatcom compiler while configuring the secondary as a separate thread. In that case I used a va_copy to store the va_list ptr within the shared structure in order to make it work:
void secprint(char *format,...) { va_list ptr; va_start(ptr,format); va_copy((*secondary).ptr,ptr); (*secondary).format=format; (*secondary).outstr=&outstr[0]; (*secondary).request=print_command; while((*secondary).request != 0); }Basically, I was "winging it", hence my handle Wingineer19 as the technicians out at work always accused us engineers of "winging it". Truth be told, they were right more often than not.
I think it's approaching late afternoon where you live, but it's getting late here in the States so I'm going to call it a night. I'll check back tomorrow morning and see if you found any other interesting stuff.
Hi RossH,
OK, I just completed 32 iterations of this code and got some interesting results.
First, here's the baseline code I tested. Variations are made for each iteration to account for the cache size, optimization, and the type of math library tested.
//Program is prisec.c //Combines the previous secondary.c and primary.c programs into one //Last revision on 28 June 2024 #pragma catapult common options(-lc -lm -lserial4 -C NO_HMI -C FLIP -y) #include "catapult.h" #include <stdio.h> #include <stdlib.h> #include <string.h> #include <stdarg.h> #include <math.h> #include <time.h> #include <propeller.h> #define Yes 1 #define No 0 #define conport 0 #define print_command 1 #define ROUNDUP 128 typedef struct shared { char request; char *format; char *outstr; va_list ptr; } shared_t; // since we are not using a HMI, we must redefine CATAPULT_ERROR #undef CATAPULT_ERROR #define CATAPULT_ERROR(name,address)\ {\ for(;;) \ { \ s4_str(conport,"Catapult Error -- Change The "); \ s4_str(conport,name); \ s4_str(conport," Memory Location To 0x"); \ s4_hex(conport,(int) address,6); \ s4_str(conport,"\r\n"); \ while(s4_rxcheck(conport) < 0); \ } \ } #pragma catapult secondary secmain(shared_t) mode(COMPACT) address(0x4ec4) stack(512) struct shared *secondary; char outstr[256]; void secprint(char *format,...) { va_list ptr; va_start(ptr,format); (*secondary).format=format; (*secondary).ptr=ptr; va_end(ptr); (*secondary).outstr=&outstr[0]; (*secondary).request=print_command; while((*secondary).request != 0); s4_str(conport,outstr); } void secmain(struct shared *primary) { float theta; secondary=primary; for(;;) { for(theta=0.0; theta<6.283185307; theta+=0.0062831853) { secprint("theta=%f,sine=%f,cosine=%f\r\n",theta,sin(theta),cos(theta)); } } } #pragma catapult primary main mode(SMALL) binary(primary) options(-C XEPROM -C CACHED_1K) struct shared primary; void priprint(void) { vsprintf(primary.outstr,primary.format,primary.ptr); primary.request=0; } void main(void) { RESERVE_SPACE(secmain); int cogsec; RESERVED_START(secmain,primary,ANY_COG,cogsec); for(;;) { if(primary.request == print_command) priprint(); } }One thing I discovered is that code execution didn't care if you did this:
void secprint(char *format,...) { va_list ptr; va_start(ptr,format); (*secondary).format=format; (*secondary).ptr=ptr; va_end(ptr); (*secondary).outstr=&outstr[0]; (*secondary).request=print_command; while((*secondary).request != 0); s4_str(conport,outstr); } void priprint(void) { vsprintf(primary.outstr,primary.format,primary.ptr); primary.request=0; }Or if you did this:
void secprint(char *format,...) { va_list ptr; va_start(ptr,format); (*secondary).format=format; (*secondary).ptr=ptr; (*secondary).outstr=&outstr[0]; (*secondary).request=print_command; while((*secondary).request != 0); s4_str(conport,outstr); } void priprint(void) { vsprintf(primary.outstr,primary.format,primary.ptr); va_end(ptr); primary.request=0; }Take your pick because it didn't matter.
What did matter was using the optimizer and what type of math library you used.
If you did this:
You got this:
Cache Size Optimization Code Execution Delay 1K N < 5 sec 2K N < 5 sec 4K N < 5 sec 8K N < 5 secIf you did this:
You got this:
Cache Size Optimization Code Execution Delay 1K Y < 5 sec 2K Y < 5 sec 4K Y > 50 sec 8K Y < 5 secNow if you did this:
You got this:
Cache Size Optimization Code Execution Delay 1K N > 50 sec 2K N > 50 sec 4K N > 50 sec 8K N > 50 secAnd if you did this:
You got this:
Cache Size Optimization Code Execution Delay 1K Y > 50 sec 2K Y > 50 sec 4K Y > 50 sec 8K Y < 5 secI haven't tried this yet so it will be interesting to see what happens:
So it does appear that using the -lma plugin will introduce the delay regardless of cache size and optimization with the exception of CACHE_8K and optimization. That's strange.
What is also odd is that using the -lm will only introduce the delay using CACHE_4K and optimization.
Hopefully this additional information will provide more clues as to what is happening.
UPDATE:
I tried this:
And this:
While using cache sizes CACHE_1K, CACHE_2K, CACHE_4K, and CACHE_8K. In each case, the code never appears to start.
So, I guess there's an issue with using the -lmb plugin as well...
I have tried the -lmb option, with essentially the same results - i.e. I can make the problem appear or disappear just by changing a few instructions. I don't even need to recompile the program.
I know exactly what is being corrupted and when. But I cannot figure out how or why
My supposition is that something bad is happening with the va_list data within the shared structure as this info is passed from the secondary over to the primary via this shared structure.
If so, why doesn't the problem arise for EVERY instance of testing regardless of cache size, optimization, or math library plugin? It doesn't make any sense.
Could the way the vsprintf() function is used within priprint() be causing the problem?
This is an example of a major problem being caused by a small program. Totally insane.
Yes, it is. I now have the problem isolated to a single instruction. Execute it and the program goes haywire. I just can't figure out why. The instruction is completely unrelated to the memory location that is being corrupted.
I'm missing something obvious. It will no doubt end up as one of those "D'oh!" moments
Ross.
Not much progress to report ... but some ...
It's not related to the instruction being executed, it's related to the timing of the instruction being executed. Which is why the optimizer makes a difference. I's likely to be a very tight race condition between two or more cogs.
That opens up the field again, but at least it makes a bit more sense.
Ross.
No your program is fine. It's definitely a problem with Catalina's XEPROM support. I now have several test programs that all work perfectly when executed from FLASH, SRAM, Hub RAM, and even from EEPROM - and with any combination of cache size, optimization, and maths plugins - but which all fail with the same symptoms when executed via XEPROM.
I have a workaround for this specific instance of the problem which makes all these programs work reliably. I will include it in the next release because it's also more robust than the code that I currently use to start a C program on another cog.
However, it is only a workaround, not a solution. I still need to find the underlying cause or the problem might show up again elsewhere. I now believe it is in the XEPROM XMM driver, which I haven't looked at for close to a decade
Ross.
P.S. Your test program is indeed a "small" program, and so looks deceptively simple - but the underlying complexity of actually building and executing it is very high. So thank goodness for Catapult! Without it, recompiling this program manually so many times with so many different combinations of options would be a nightmare!
Well, that's a bummer. Having to rewrite an old driver doesn't sound like fun. But if you do, you might find ways to make it even better and more robust.
The XEPROM driver using the SMALL memory model that stores code in EEPROM, but doesn't use external SRAM, is an attractive option for running an XMM program that's heavy on code but doesn't need lots of data memory. Circuit complexity is certainly reduced without the need for external memory and hardware.
Hopefully, Parallax will eventually offer the FLiP with a 128 kilobyte EEPROM (or a 256 kilobyte one if they're in a really generous mood) that will make it more useful than the 64 kilobyte one installed now.
There are newer 128 kilobyte EEPROMs that support up to 3.4 MHz transfer speed like the MicroChip 24CSM01. That combined with caching should yield a little bit better performance than the current ones.
Absolutely! Catapult really proved itself when I did 40 iterations of test code yesterday. Each iteration required a single compile command followed by a single upload command. If the secondary memory needed to be changed then one more compile and upload would be required.
Without Catapult each iteration would have required:
1. Compile secondary
2. Run spinc
3. Compile primary
4. Upload to EEPROM
5. If secondary memory needed to be changed, then repeat steps 1 through 4 again.
So Catapult essentially cut the required number of steps by half. I think any XMM programmer will find it to be a very handy utility.
Found it. I thought this would probably involve a D'oh! moment, and I was at least right about that!
The problem has nothing to do with XEPROM. It is the floating point plugin that is corrupting Hub RAM. After enough tests, I noticed that the data being written was not a valid Propeller instruction - it looked suspiciously like an IEEE-754 floating point number, which indeed it was! I was not correctly initializing the communication between a dynamically loaded COMPACT kernel and the floating point plugin. This would be a problem if the secondary C program was a COMPACT program, and executed a floating point operation before the primary program had finished waiting for it to start, such as the program used by @Wingineer19 does (it also depended on the timing, so it was indeed a "race" condition).
My "workaround" was to eliminate that wait so that this could no longer happen, but now I have a proper fix.
I hope to release a a new version tomorrow.
Ross.
Catalina 7.6.1 has been released here.
No major new functionality, just fixes for a few issues. It is a full release because it requires a rebuild of all the C libraries.
Here is the relevant extract of the README.TXT file:
EDIT: README extract updated.
Oops! Just thought of something I'd forgotten - release 7.6.1 has been taken down but will be available again shortly!
Ross.
Apologies for the delay in release 7.6.1 - it is now available again. I have not had as much time as I thought I would have, so instead of holding up the release any further I have pared it back to just the necessary bug fixes. I will add the other changes (fairly minor) into a future release as I get time.
Ross.