@Wuerfel_21 said:
On Windows NT, especially if you have any sort of security software enabled, spawning a new process and attaching it to the console subsystem is comically slow. Reading directory information is also slow. This can combine into a make+gcc style build taking orders of magnitude longer than reasonable.
Yes, disabling as much Windows security as possible does indeed speed things up - but not enough. Also, there are good reasons why you absolutely want as much security enabled in Windows as you can get (and then go add some more!) so it is not something users should have to do.
Using MinGW in GH Actions is perfectly fine (assuming use as a cross-compiler on a Linux action runner), I set it up for flexspin/spin2cpp.
I may be missing something here. Why install MinGW on Linux and cross-compile? Why not just use a Windows action runner? Because it would be too slow?
The real big thing with actions IMO is that you can use it to auto-run any test scripts you have. For flexspin I made a modified version of spinsim, so P1 code can be tested end-to-end.
Yes, I can see that would be useful. It would have saved me from a few embarrassing releases
I know there was a P2 version of spinsim - do you use that?
@Wuerfel_21 said:
On Windows NT, especially if you have any sort of security software enabled, spawning a new process and attaching it to the console subsystem is comically slow. Reading directory information is also slow. This can combine into a make+gcc style build taking orders of magnitude longer than reasonable.
Yes, disabling as much Windows security as possible does indeed speed things up - but not enough. Also, there are good reasons why you absolutely want as much security enabled in Windows as you can get (and then go add some more!) so it is not something users should have to do.
Installing security software isn't something anyone should ever have to do. The OS should already be inherently secure. If it isn't, it isn't a serious OS.
Using MinGW in GH Actions is perfectly fine (assuming use as a cross-compiler on a Linux action runner), I set it up for flexspin/spin2cpp.
I may be missing something here. Why install MinGW on Linux and cross-compile? Why not just use a Windows action runner? Because it would be too slow?
Doing compilation (or, frankly, anything more complicated than word processing) on Windows just doesn't make sense. Every popular OS besides Windows - Linux, BSD, even Mac OS, is better, faster, more secure, easier to write correct programs for, and more consistent with other OSes. Official builds of PuTTY, a program which decidedly only makes sense on Windows because Linux (and newer versions of Windows) already comes with ssh, telnet, and nice serial port programs, are cross-compiled for Windows by a Linux machine (and there's a Linux build for testing purposes) because it's so much simpler than bothering with building something inside Windows.
@Electrodude said:
Doing compilation (or, frankly, anything more complicated than word processing) on Windows just doesn't make sense. Every popular OS besides Windows - Linux, BSD, even Mac OS, is better, faster, more secure, easier to write correct programs for, and more consistent with other OSes. Official builds of PuTTY, a program which decidedly only makes sense on Windows because Linux (and newer versions of Windows) already comes with ssh, telnet, and nice serial port programs, are cross-compiled for Windows by a Linux machine (and there's a Linux build for testing purposes) because it's so much simpler than bothering with building something inside Windows.
I get where you're coming from, but OS wars are a bit like language wars and editor wars - neither side is ever likely to convince the other. I routinely use both Windows and Linux so I don't really have a "side" on this one.
I think a more important question when it comes to developing free open-source software is whether you can develop it using free open-source software. I can still do that on both Windows and Linux, but I dropped MacOS when this became impossible. I don't get paid for developing my software, so I don't see why I should have to pay to develop it.
As long as it remains possible to do this on Windows, I will probably continue to do so. Painful and slow as it sometimes is.
@Wuerfel_21 said:
On Windows NT, especially if you have any sort of security software enabled, spawning a new process and attaching it to the console subsystem is comically slow. Reading directory information is also slow. This can combine into a make+gcc style build taking orders of magnitude longer than reasonable.
Yes, disabling as much Windows security as possible does indeed speed things up - but not enough. Also, there are good reasons why you absolutely want as much security enabled in Windows as you can get (and then go add some more!) so it is not something users should have to do.
Using MinGW in GH Actions is perfectly fine (assuming use as a cross-compiler on a Linux action runner), I set it up for flexspin/spin2cpp.
I may be missing something here. Why install MinGW on Linux and cross-compile? Why not just use a Windows action runner? Because it would be too slow?
Because it's just easier to deal with. The makefile was already set-up to allow for cross-compilation and it's easy to grab MinGW, Bison, etc from the ubuntu package manager. Also I think windows runners used to not exist / cost extra?
Incidentally, for linux binary builds, I set it up to use musl-gcc to link the entire C library in to avoid dependency conflicts. It doesn't seem to increase the file size noticeably.
The real big thing with actions IMO is that you can use it to auto-run any test scripts you have. For flexspin I made a modified version of spinsim, so P1 code can be tested end-to-end.
Yes, I can see that would be useful. It would have saved me from a few embarrassing releases
I know there was a P2 version of spinsim - do you use that?
There's some P2-themed code in there, but I remember I couldn't get it to work. Possibly based on some FPGA prototype version...
This is just a bit of advice for Catalina users ...
I've been doing some more investigation into memory issues, because despite many recent Lua improvements there is still an outstanding issue that can affect both C and Lua programs.
The issue occurs when the heap (which grows up from low memory) and the stack (which grows down from high memory) are both allocated in Hub RAM and collide. There is no simple fix for this issue. It cannot be solved just by having the dynamic memory allocation functions (i.e. malloc() and friends) check the stack before allocating from the heap. It also requires that the heap be checked before every operation that allocates from the stack - e.g. on every C function call. This would be horribly expensive, but I will consider adding it to a future release as a debugging aid.
This issue only affects programs where the heap and stack are both in Hub RAM (i.e. CMM, NMM, LMM and XMM SMALL programs, but not XMM LARGE programs). Lua programs are more prone to suffer from it because they typically do more dynamic memory allocation than C programs, and recent Catalina releases - which allow larger and more complex Lua programs to execute from Hub RAM - have made it more likely to occur.
Hoist with my own petard!
The best option currently available is to minimize the need for programs to allocate more heap space just because they do not have a single contiguous block of suitable size by de-fragmenting the heap.
In C, there is a library function to do this. Add a line like the following to the C program:
malloc_defragment(); // defragment the C heap
In Lua (as documented in Lua on the Propeller 2 with Catalina ) the sbrk() function can be used. Add a line like the following to the Lua program:
propeller.sbrk(true) -- defragment the C heap
In many cases just adding one line resolves the issue completely. The best place to add such a line is just before executing code that does large amounts of dynamic memory allocation (e.g. in Lua, lots of string operations). If it is not easy to identify a specific place in the program, the next best option would be to do it periodically.
I've (re)discovered Catalina, and recently downloaded 8.1.something. And it's all doing what it should, and fun is being had. But..... I'm having difficulties with payload. It wants DLLs, which don't appear to be in the download, DLLs which I don't have, and although googling has found me things of about the right name, they don't result in a working payload.
Is there a definitive place where I can get the right DLLs?
For giggles, trawling through backups got me Catalina from a retired PC, and payload version 3.3 works, but strangely, it seems to lack features...
DLL names? Well, I think , from error messages, it is libintl-8.dll and libiconv-2.dll, versions of which I've found, and there are also versions without the hyphen, eg libiconv2.dll. I'm hoping for a definitive answer as to whats required. Delving into the source tree hasn't provided the illumination I sought.
I've (re)discovered Catalina, and recently downloaded 8.1.something. And it's all doing what it should, and fun is being had. But..... I'm having difficulties with payload. It wants DLLs, which don't appear to be in the download, DLLs which I don't have, and although googling has found me things of about the right name, they don't result in a working payload.
Is there a definitive place where I can get the right DLLs?
For giggles, trawling through backups got me Catalina from a retired PC, and payload version 3.3 works, but strangely, it seems to lack features...
Thanks,. David.
Hi David
If you downloaded from SourceForge, the DLLs should be installed by the Windows installer.
If you downloaded from GitHub, the DLLs should be in the GitHub binary assets - perhaps in a previous release if there have been no binary changes. For instance, for 8.1.1 or 8.1.2, use the binary assets for 8.1.1 here and here. You need both.
And yes, payload 3.3 is very old and will be missing lots of features! It is probably one of those features that requires the additional DLLs.
Ross.
Edited: You need both the Windows binary asset and the Windows Geany binary asset to get all the necessary DLLs.
@"David Buckley" said:
DLL names? Well, I think , from error messages, it is libintl-8.dll and libiconv-2.dll, versions of which I've found, and there are also versions without the hyphen, eg libiconv2.dll. I'm hoping for a definitive answer as to whats required. Delving into the source tree hasn't provided the illumination I sought.
Ah yes - it seems you now also need both libintl-8.dll and libiconv-2.dll in your PATH. I have so many things in my PATH variable (including paths to other versions of these DLLs) that I'd not noticed that omission before - thanks.
These files exist in "%LCCDIR%"\catalina_geany\bin, so to do this either copy those two files from "%LCCDIR%"\catalina_geany\bin to "%LCCDIR%"\bin or add "%LCCDIR%"\catalina_geany\bin to your path.
For example, execute the following in a Catalina Command Line window:
I thought I would have had time by now to do a new Catalina release, but I have not. So just to fix the issue about missing DLLs being reported by payload, I have updated the current Windows binary releases of Catalina 8.1.2 (the Linux releases do not need DLLs).
On SourceForge I have updated the Windows installer for 8.1.2 to include the missing DLLs. Or you can just use the copy commands given here) to do this on an existing installation of 8.1.2 - that's the only change.
On GitHub I have added new binary assets for Windows to release 8.1.2. Previously, this release said to use the assets from 8.1.1, but those are missing the DLLs.
I have just ordered a new Raspberry Pi (a Pi 500), but in preparation and while waiting for it to arrive, I dug out my old Pi 3 and rebuilt Catalina using that. I haven't done that for literally years, but it all went well. It's actually much easier now than it used to be - the Pi OS seems to have improved since I last used it.
It's a bit of a squeeze on the Pi 3 because it only has one GB of RAM, but it works. It should be better on the Pi 500 (which has a faster chip and also has 8GB of RAM).
On the Pi you currently have to build Catalina from source. I found one issue in the bin/p2_asm script (seems to be a difference between Ubuntu and Debian) and one problem with the xterm program used by Catalina Geany (ditto) which I will look into - but other than that everything I have tried so far works fine. If anyone finds any issues, please let me know.
I have held off upgrading to Windows 11 so far because every version of Windows seems worse than the one before, but I will eventually have to do so since Windows 10 support is finally coming to an end. But if the Pi 500 works out well, I am tempted to make the Pi my default development platform. On the Pi 500, I expect it to be faster to build Catalina than it will be on my Core i7 running Windows 11.
I have updated the BUILD.TXT file to include detailed Pi instructions. These updates are currently on Github only. I will include them in the SourceForge packages in the next release.
I found one issue in the bin/p2_asm script (seems to be a difference between Ubuntu and Debian) ...
More on this one. It's a bit weird. On the Pi, using #!bin/bash -login as the first line of a bash script breaks a few things downstream, including make
Really this line should have been #!bin/bash --login (i.e. two dashes, not one) - but in most cases the scripts don't need login shells anyway, so I have simply removed the -login option, and now everything seems to work properly on the Pi.
What bash -login actually does on the Pi is a bit of a mystery.
Times (minutes) to build all the usual Catalyst P2 demo binaries from source
(i.e. P2_DEMO.ZIP, P2_EDGE.ZIP, P2_EDGE_VGA.ZIP. P2_EVAL.ZIP, P2_EVAL_VGA.ZIP):
I found one issue in the bin/p2_asm script (seems to be a difference between Ubuntu and Debian) ...
More on this one. It's a bit weird. On the Pi, using #!bin/bash -login as the first line of a bash script breaks a few things downstream, including make
Really this line should have been #!bin/bash --login (i.e. two dashes, not one) - but in most cases the scripts don't need login shells anyway, so I have simply removed the -login option, and now everything seems to work properly on the Pi.
What bash -login actually does on the Pi is a bit of a mystery.
As described in §6.2 of the bash manual, it makes bash source /etc/profile and ~/.bashrc, which typically do things only suitable for interactive shells, before running your script, so you definitely don't want -login in a hashbang. Is there a reason you had it there to begin with?
As described in §6.2 of the bash manual, it makes bash source /etc/profile and ~/.bashrc, which typically do things only suitable for interactive shells, before running your script, so you definitely don't want -login in a hashbang. Is there a reason you had it there to begin with?
I originally added the --login option so users could have Catalina set up automatically in each new interactive shell (e.g. by adding commands to their .bashrc file). But it is not strictly necessary unless the scripts are going to be initiated other than from an interactive shell (e.g. if they were to be initiated from a desktop menu or icon).
For example, I do NOT set up Catalina automatically because I am typically using multiple versions of Catalina at the same time (e.g. the current release and the next release). So I would typically do something like the following to use release X.X in a particular shell:
cd ~/Catalina_X.X
export LCCDIR=`pwd`
source use_catalina
I've been doing some reading on both bash and make, and I think it may have to do with the fact that interactive shells use different job control to non-interactive shells. The details of job control are platform dependent, and it is possible that on the Pi the bash job control and the make job control are interacting in a different way to other flavors of Linux.
I've revised the Pi OS instructions (in BUILD.TXT) to fix an issue with the suggested .Xdefaults settings, and also to recommend that Catalina should be installed in /opt/catalina. I've also added some desktop icons - one for Catalina Command Line and another for Catalina Geany.
The icons have to use hardcoded paths, so they use /opt/catalina - which means they will work "out of the box" if you installed Catalina as recommended. Otherwise they can be edited to suit (they are simple text files). Using the icons means you no longer have to manually set the LCCDIR and PATH variables (although you still can if you want). This eliminates the need to use login shells.
As usual, Github has been updated. Sourceforge will be updated on the next release.
Since the Raspberry Pi OS is really just Debian with a custom desktop, I thought that Catalina's Pi OS build instructions might work for other Debian installations, and indeed they do (with one minor tweak). So Catalina now builds on Windows, Pi OS, Ubuntu and Debian.
I found one Debian issue to do with the sizes of some GNOME icons which made some Geany dialog boxes too large to fit on the screen. This is probably to do with the version of GTK used, but I fixed it by simply removing the icons and just putting text labels on the affected buttons.
I've added a new payload option to fix an issue with it double spacing lines on Pi OS (and other Debian-based Linux systems) when used in interactive mode.
I'm not sure why Debian line termination handling is different to Ubuntu - possibly a difference in the readline and/or ncurses library functions. In any case, payload now does NOT move the cursor by default when it receives a CR and LF from the Propeller (it used to do this even if the CR or LF was otherwise ignored).
The previous payload behaviour (i.e. DO move the cursor) can be restored by adding 32 as the -q option - e.g. -q32. This can be combined with other -q options, so if you previously used -q1 to get correct line termination behaviour, you might now use -q33 (or -q34 instead of -q2 etc).
The complications with line termination in C programs originates from two main sources:
The C standards do not specify the behaviour of "carriage return" and "line feed" as the original terminals (often hardcopy terminals) of the day were actually built - instead of "carriage return" and "line feed", the C standard chose to use the terms "carriage return" and "new line". However, "new line" is NOT the same as "line feed" - it actually means "carriage return PLUS line feed", which is what the teletype terminals of the day had to be sent to implement a "new line". For some reason the designers of C decided there was no longer need for a plain old "line feed" function even though that is what "new line" is still called in the ASCII standard to this day.
Programs originally written for Windows, Macintosh and Unix systems all generated and expected different line termination sequences (i.e. CR LF, CR and LF respectively). To try and accommodate these discrepancies when it came to serial comms, intermediaries such as the serial drivers (e.g. tty), terminal programs (e.g. xterm and payload) and command shells (e.g. bash) often provided their own - and often incompatible - ways of "correcting" for this.
The source of payload.c has been updated on Github. You will need to recompile the binaries (e..g use the build_binaries script in the Catalina source directory).
Just been doing a bit more testing on various serial comms combinations, and I was surprised to discover that Catalyst no longer works with the Parallax Serial Terminal.
The culprit turned out to be the linenoise library, which enables command line history and editing. Using this library also requires a VT100 terminal, so it would not normally be used with the Parallax Serial Terminal, but I was surprised to find that it breaks it so badly
As a stopgap, I have added an option to the Catalyst Makefile to allow linenoise to be disabled - just add the option NO_LINENOISE to the Catalyst build_all command (along with CR_ON_LF, which is required by the Parallax Serial Terminal). For example:
build_all P2_EDGE TTY CR_ON_LF NO_LINENOISE
As usual, Github has been updated. SourceForge will have to wait for the next release.
@RossH said:
Just been doing a bit more testing on various serial comms combinations, and I was surprised to discover that Catalyst no longer works with the Parallax Serial Terminal.
The culprit turned out to be the linenoise library, which enables command line history and editing. Using this library also requires a VT100 terminal, so it would not normally be used with the Parallax Serial Terminal, but I was surprised to find that it breaks it so badly
wow, thanks, I suspect I'd've hit that soon, too. doing some re-org of shop to clear a place for a Catalyst serial terminal on P1 project.
Ross,
I presume you've noted my work on bringing up full 4-bit SD mode support for SD cards within FlexC? I've had good success with porting to Spin2 and making it work with Kye's FAT32 FS layer. And that same driver should be easy to drop into FSRW as well but I've not found a known good version of that one.
Anyway, it's a finished driver now even if it's not yet getting use due to needing different wiring than the SPI mode drivers.
@evanh said:
Ross,
I presume you've noted my work on bringing up full 4-bit SD mode support for SD cards within FlexC?
Not been monitoring it closely, but definitely interested.
The speed of the SD card access is definitely one of the major limiting factors in compiling C programs on the P2, so I will look into your progress with interest when I get some time.
@RossH said:
Just been doing a bit more testing on various serial comms combinations, and I was surprised to discover that Catalyst no longer works with the Parallax Serial Terminal.
The culprit turned out to be the linenoise library, which enables command line history and editing. Using this library also requires a VT100 terminal, so it would not normally be used with the Parallax Serial Terminal, but I was surprised to find that it breaks it so badly
wow, thanks, I suspect I'd've hit that soon, too. doing some re-org of shop to clear a place for a Catalyst serial terminal on P1 project.
I think I will be reducing my support for the Parallax Serial Terminal. It's not really a very useful option**. There are plenty of alternatives that can do more than it can do, including the two options (payload and Comms) already bundled with Catalina. Then there are things like puTTY, Tera Term and minicom. And probably a zillion others.
Of course it will still be possible to use the Parallax Serial Terminal, but only if you disable much of the really neat functionality now offered by Catalyst, like command line editing, command completion and command history.
Ross.
** Not set in stone yet. If anyone knows of a reason for continuing support of the Parallax Serial Terminal, please let me know.
This is a full release, but it is mostly just a few bug fixes, and to rationalize payload's CR/LF handling, which was getting too unwieldy by trying to support the Parallax Serial Terminal. Now, you have to recompile if you want to use that as your serial terminal emulator.
Here is the relevant extract from the README.TXT
New Functionality
-----------------
1. Payload now accepts multiple -q options, with all the -q options being
'or'ed together to determine the final modes. So, for example, specifying
'-q1 -q2' is now the same as specifying '-q3'.
2. Payload now accepts an additional -q option (-q32) which specifies that
payload should interpret CR and LF and move the screen cursor even if
these keys are otherwise being ignored. This means that payload handles
the screen movements rather than the ncurses package on Linux (or the
pdcurses package on Windows). This can solve some compatibility issues
with C programs that expect LF to behave like a C style 'new line' but
also expect the ENTER key to generate a 'carriage return' rather than a
'line feed' (this is something that ncurses/pdcurses cannot do).
3. Some payload options have changed descriptions to be more correct, and
also to correspond more closely with their Comms program equivalents. For
instance, the description of the payload -q16 option has changed from
"CR to LF on Output" to "Auto CR on LF Output", which is the same as used
by the Comms program (note that the functionality has not changed - the
description in payload was incorrect).
4. A CR/LF test program (called crlftest.c) has been added to the demos
folder. It can be used to test the effects of various payload and Comms
options, and also various Catalina symbols that affect CR and LF
processing.
5. It is now possible to build Catalyst and Lua without the 'linenoise'
library, which was added by default when building Catalyst (for the
Propeller 2), and Lua for the Propeller 1 or 2). The 'linenoise' library
can now be disabled by defining the Catalina symbol NO_LINENOISE.
Using the 'linenoise' library works ok with both payload internal and
external terminal emulators, but not with the Parallax Serial Terminal.
6. None of the demo programs are now built by default with the Catalina symbol
CR_ON_LF defined. This was originally done to make programs such as Catalyst
compatible with the Parallax Serial Terminal, but since the Parallax Serial
Terminal has issues with the 'linenoise' library (which adds command line
editing and command history to both Catalyst and Lua), it now needs a
special build anyway. So the defaults of both payload and Comms have been
unified to eliminate the need to be compatible with the CR_ON_LF option,
which eliminates the need to use the payload -q option or adjust the
"Advanced Options" in the Comms program.
The Parallax Serial Terminal can still be used if programs are compiled
with the Catalins symbols CR_ON_LF and NO_LINENOISE defined.
7. Since the Propeller 2 Evaluation board is no longer readily available,
the Propeller 2 CUSTOM definitions have been modified to use the P2_EDGE
definitions rather than the P2_EVAL definitions. Also, the demo programs
in P2_DEMO.ZIP are now built for the P2_EDGE rather than the P2_EVAL.
8. Separate instructions for building Catalina from source under the Raspberry
Pi OS have been added to BUILD.TXT. These instructions will also work on
other Debian-based Linux releases.
9. The self-hosted version of Catalina now includes the spinc utility. Since
the output of this utility used to be sent to stdout and would have to be
redirected to a file (which is not possible under Catalyst) a new command
line option has also been added: '-g file' will now generate a file with
with the output instead of sending it to stdout.
Other Changes
-------------
1. Defining the symbol P2 either on the command line or in CATALINA_DEFINE
incorrectly forced NATIVE mode, overriding any other specified mode
(e.g. TINY, CUSTOM, SMALL, LARGE). Affected the Propeller 2 only on
Windows, Linux, Pi OS and Catalyst.
2. The payload -q4 and -q8 options were being incorrectly applied to the
output (to the screen) rather than the input (from the keyboard). Affected
the Propeller 1 and 2 on Windows, Linux and Pi OS.
3. Various Linux scripts have been modified to remove the bash '-login'
option, which made these scripts not work in the Pi OS or Debian-based
Linux distributions. Affected the Propeller 1 and 2 on Linux and Pi OS.
4. The document "Getting Started with Catalina" was very out of date, and has
been extensively revised and updated.
5. The Catalina Reference Manual (Propeller 1 and Propeller 2) said that the
CR_ON_LF option translated CR to CR LF, but what it actually does is
translate LF to CR LF.
6. The payload -o option now always sets the default baud rate as appropriate
for the Propeller 1 (115200) or Propeller 2 (230400) unless a baud rate is
explicitly set. Previously, it would only do so if a file to load was
specified, not if only interactive mode was specified.
Finally, this will probably be the last Ubuntu release (Ubuntu binaries were released on SourceForge). Since I have decided to move to Pi OS as my primary development environment (which is essentially Debian Linux) I have decided to stop struggling with Ubuntu.
I will continue to build Windows releases but perhaps only for major releases. It takes 10 times as long to build and test on Windows than it does on Pi OS or Linux.
@RossH said:
4. The document "Getting Started with Catalina" was very out of date, and has been extensively revised and updated.
I appreciate that.
Since I have decided to move to Pi OS as my primary development environment (which is essentially Debian Linux I have decided to stop struggling with Ubuntu.
can a Pi400 handle it? I still have one of those that I used to do some development on.
@refaQtor said:
can a Pi400 handle it? I still have one of those that I used to do some development on.
Yes, I'm sure it would. Pi OS is basically Debian 12 (bookworm), and the current version runs on all Pi models. I also ran it on a Pi 3 and it worked, but it was a little sluggish. The Pi 400 should be fine.
Comments
Yes, disabling as much Windows security as possible does indeed speed things up - but not enough. Also, there are good reasons why you absolutely want as much security enabled in Windows as you can get (and then go add some more!) so it is not something users should have to do.
I may be missing something here. Why install MinGW on Linux and cross-compile? Why not just use a Windows action runner? Because it would be too slow?
Yes, I can see that would be useful. It would have saved me from a few embarrassing releases
I know there was a P2 version of spinsim - do you use that?
Ross.
Installing security software isn't something anyone should ever have to do. The OS should already be inherently secure. If it isn't, it isn't a serious OS.
Doing compilation (or, frankly, anything more complicated than word processing) on Windows just doesn't make sense. Every popular OS besides Windows - Linux, BSD, even Mac OS, is better, faster, more secure, easier to write correct programs for, and more consistent with other OSes. Official builds of PuTTY, a program which decidedly only makes sense on Windows because Linux (and newer versions of Windows) already comes with
ssh,telnet, and nice serial port programs, are cross-compiled for Windows by a Linux machine (and there's a Linux build for testing purposes) because it's so much simpler than bothering with building something inside Windows.I get where you're coming from, but OS wars are a bit like language wars and editor wars - neither side is ever likely to convince the other. I routinely use both Windows and Linux so I don't really have a "side" on this one.
I think a more important question when it comes to developing free open-source software is whether you can develop it using free open-source software. I can still do that on both Windows and Linux, but I dropped MacOS when this became impossible. I don't get paid for developing my software, so I don't see why I should have to pay to develop it.
As long as it remains possible to do this on Windows, I will probably continue to do so. Painful and slow as it sometimes is.
Ross.
Because it's just easier to deal with. The makefile was already set-up to allow for cross-compilation and it's easy to grab MinGW, Bison, etc from the ubuntu package manager. Also I think windows runners used to not exist / cost extra?
Incidentally, for linux binary builds, I set it up to use musl-gcc to link the entire C library in to avoid dependency conflicts. It doesn't seem to increase the file size noticeably.
There's some P2-themed code in there, but I remember I couldn't get it to work. Possibly based on some FPGA prototype version...
This is just a bit of advice for Catalina users ...
I've been doing some more investigation into memory issues, because despite many recent Lua improvements there is still an outstanding issue that can affect both C and Lua programs.
The issue occurs when the heap (which grows up from low memory) and the stack (which grows down from high memory) are both allocated in Hub RAM and collide. There is no simple fix for this issue. It cannot be solved just by having the dynamic memory allocation functions (i.e. malloc() and friends) check the stack before allocating from the heap. It also requires that the heap be checked before every operation that allocates from the stack - e.g. on every C function call. This would be horribly expensive, but I will consider adding it to a future release as a debugging aid.
This issue only affects programs where the heap and stack are both in Hub RAM (i.e. CMM, NMM, LMM and XMM SMALL programs, but not XMM LARGE programs). Lua programs are more prone to suffer from it because they typically do more dynamic memory allocation than C programs, and recent Catalina releases - which allow larger and more complex Lua programs to execute from Hub RAM - have made it more likely to occur.
Hoist with my own petard!
The best option currently available is to minimize the need for programs to allocate more heap space just because they do not have a single contiguous block of suitable size by de-fragmenting the heap.
In C, there is a library function to do this. Add a line like the following to the C program:
malloc_defragment(); // defragment the C heapIn Lua (as documented in Lua on the Propeller 2 with Catalina ) the sbrk() function can be used. Add a line like the following to the Lua program:
propeller.sbrk(true) -- defragment the C heapIn many cases just adding one line resolves the issue completely. The best place to add such a line is just before executing code that does large amounts of dynamic memory allocation (e.g. in Lua, lots of string operations). If it is not easy to identify a specific place in the program, the next best option would be to do it periodically.
Ross.
Help!
I've (re)discovered Catalina, and recently downloaded 8.1.something. And it's all doing what it should, and fun is being had. But..... I'm having difficulties with payload. It wants DLLs, which don't appear to be in the download, DLLs which I don't have, and although googling has found me things of about the right name, they don't result in a working payload.
Is there a definitive place where I can get the right DLLs?
For giggles, trawling through backups got me Catalina from a retired PC, and payload version 3.3 works, but strangely, it seems to lack features...
Thanks,. David.
The DLL names?
DLL names? Well, I think , from error messages, it is libintl-8.dll and libiconv-2.dll, versions of which I've found, and there are also versions without the hyphen, eg libiconv2.dll. I'm hoping for a definitive answer as to whats required. Delving into the source tree hasn't provided the illumination I sought.
Hi David
If you downloaded from SourceForge, the DLLs should be installed by the Windows installer.
If you downloaded from GitHub, the DLLs should be in the GitHub binary assets - perhaps in a previous release if there have been no binary changes. For instance, for 8.1.1 or 8.1.2, use the binary assets for 8.1.1 here and here. You need both.
And yes, payload 3.3 is very old and will be missing lots of features! It is probably one of those features that requires the additional DLLs.
Ross.
Edited: You need both the Windows binary asset and the Windows Geany binary asset to get all the necessary DLLs.
Ah yes - it seems you now also need both libintl-8.dll and libiconv-2.dll in your PATH. I have so many things in my PATH variable (including paths to other versions of these DLLs) that I'd not noticed that omission before - thanks.
These files exist in "%LCCDIR%"\catalina_geany\bin, so to do this either copy those two files from "%LCCDIR%"\catalina_geany\bin to "%LCCDIR%"\bin or add "%LCCDIR%"\catalina_geany\bin to your path.
For example, execute the following in a Catalina Command Line window:
Ross.
And with those two copy commands, all is well and working.
Thank you Ross!
No worries. I'll fix this properly in the next release.
I thought I would have had time by now to do a new Catalina release, but I have not. So just to fix the issue about missing DLLs being reported by payload, I have updated the current Windows binary releases of Catalina 8.1.2 (the Linux releases do not need DLLs).
On SourceForge I have updated the Windows installer for 8.1.2 to include the missing DLLs. Or you can just use the copy commands given here) to do this on an existing installation of 8.1.2 - that's the only change.
On GitHub I have added new binary assets for Windows to release 8.1.2. Previously, this release said to use the assets from 8.1.1, but those are missing the DLLs.
Ross.
I have just ordered a new Raspberry Pi (a Pi 500), but in preparation and while waiting for it to arrive, I dug out my old Pi 3 and rebuilt Catalina using that. I haven't done that for literally years, but it all went well. It's actually much easier now than it used to be - the Pi OS seems to have improved since I last used it.
It's a bit of a squeeze on the Pi 3 because it only has one GB of RAM, but it works. It should be better on the Pi 500 (which has a faster chip and also has 8GB of RAM).
On the Pi you currently have to build Catalina from source. I found one issue in the bin/p2_asm script (seems to be a difference between Ubuntu and Debian) and one problem with the xterm program used by Catalina Geany (ditto) which I will look into - but other than that everything I have tried so far works fine. If anyone finds any issues, please let me know.
I have held off upgrading to Windows 11 so far because every version of Windows seems worse than the one before, but I will eventually have to do so since Windows 10 support is finally coming to an end. But if the Pi 500 works out well, I am tempted to make the Pi my default development platform. On the Pi 500, I expect it to be faster to build Catalina than it will be on my Core i7 running Windows 11.
I have updated the BUILD.TXT file to include detailed Pi instructions. These updates are currently on Github only. I will include them in the SourceForge packages in the next release.
Ross.
More on this one. It's a bit weird. On the Pi, using
#!bin/bash -loginas the first line of a bash script breaks a few things downstream, including makeReally this line should have been
#!bin/bash --login(i.e. two dashes, not one) - but in most cases the scripts don't need login shells anyway, so I have simply removed the-loginoption, and now everything seems to work properly on the Pi.What
bash -loginactually does on the Pi is a bit of a mystery.GitHub has been updated.
Interesting ...
Pi 500 arrived today. Works a treat! And - as expected - much faster than the Pi 3. And faster that Windows on my PC.
Times (minutes) to build all Catalina binaries and libraries from source:
Times (minutes) to build all the usual Catalyst P2 demo binaries from source
(i.e. P2_DEMO.ZIP, P2_EDGE.ZIP, P2_EDGE_VGA.ZIP. P2_EVAL.ZIP, P2_EVAL_VGA.ZIP):
My Windows PC cost several thousand dollars. The Pi 500 costs under $100.
I certainly won't be replacing my PC when it dies - or using Windows as my primary development environment from now on.
Nice. I guess that shows the ability of a good cache when the CPU/RAM is performant. The SD card won't be a patch on the SSD.
As described in §6.2 of the bash manual, it makes bash source
/etc/profileand~/.bashrc, which typically do things only suitable for interactive shells, before running your script, so you definitely don't want-loginin a hashbang. Is there a reason you had it there to begin with?I originally added the --login option so users could have Catalina set up automatically in each new interactive shell (e.g. by adding commands to their .bashrc file). But it is not strictly necessary unless the scripts are going to be initiated other than from an interactive shell (e.g. if they were to be initiated from a desktop menu or icon).
For example, I do NOT set up Catalina automatically because I am typically using multiple versions of Catalina at the same time (e.g. the current release and the next release). So I would typically do something like the following to use release X.X in a particular shell:
I've been doing some reading on both bash and make, and I think it may have to do with the fact that interactive shells use different job control to non-interactive shells. The details of job control are platform dependent, and it is possible that on the Pi the bash job control and the make job control are interacting in a different way to other flavors of Linux.
Ross.
I've revised the Pi OS instructions (in BUILD.TXT) to fix an issue with the suggested .Xdefaults settings, and also to recommend that Catalina should be installed in /opt/catalina. I've also added some desktop icons - one for Catalina Command Line and another for Catalina Geany.
The icons have to use hardcoded paths, so they use /opt/catalina - which means they will work "out of the box" if you installed Catalina as recommended. Otherwise they can be edited to suit (they are simple text files). Using the icons means you no longer have to manually set the LCCDIR and PATH variables (although you still can if you want). This eliminates the need to use login shells.
As usual, Github has been updated. Sourceforge will be updated on the next release.
Since the Raspberry Pi OS is really just Debian with a custom desktop, I thought that Catalina's Pi OS build instructions might work for other Debian installations, and indeed they do (with one minor tweak). So Catalina now builds on Windows, Pi OS, Ubuntu and Debian.
I found one Debian issue to do with the sizes of some GNOME icons which made some Geany dialog boxes too large to fit on the screen. This is probably to do with the version of GTK used, but I fixed it by simply removing the icons and just putting text labels on the affected buttons.
Github has been updated.
I've added a new payload option to fix an issue with it double spacing lines on Pi OS (and other Debian-based Linux systems) when used in interactive mode.
I'm not sure why Debian line termination handling is different to Ubuntu - possibly a difference in the readline and/or ncurses library functions. In any case, payload now does NOT move the cursor by default when it receives a CR and LF from the Propeller (it used to do this even if the CR or LF was otherwise ignored).
The previous payload behaviour (i.e. DO move the cursor) can be restored by adding 32 as the -q option - e.g. -q32. This can be combined with other -q options, so if you previously used -q1 to get correct line termination behaviour, you might now use -q33 (or -q34 instead of -q2 etc).
The complications with line termination in C programs originates from two main sources:
The source of payload.c has been updated on Github. You will need to recompile the binaries (e..g use the build_binaries script in the Catalina source directory).
Ross.
Just been doing a bit more testing on various serial comms combinations, and I was surprised to discover that Catalyst no longer works with the Parallax Serial Terminal.
The culprit turned out to be the linenoise library, which enables command line history and editing. Using this library also requires a VT100 terminal, so it would not normally be used with the Parallax Serial Terminal, but I was surprised to find that it breaks it so badly
As a stopgap, I have added an option to the Catalyst Makefile to allow linenoise to be disabled - just add the option NO_LINENOISE to the Catalyst build_all command (along with CR_ON_LF, which is required by the Parallax Serial Terminal). For example:
build_all P2_EDGE TTY CR_ON_LF NO_LINENOISEAs usual, Github has been updated. SourceForge will have to wait for the next release.
wow, thanks, I suspect I'd've hit that soon, too. doing some re-org of shop to clear a place for a Catalyst serial terminal on P1 project.
Ross,
I presume you've noted my work on bringing up full 4-bit SD mode support for SD cards within FlexC? I've had good success with porting to Spin2 and making it work with Kye's FAT32 FS layer. And that same driver should be easy to drop into FSRW as well but I've not found a known good version of that one.
Anyway, it's a finished driver now even if it's not yet getting use due to needing different wiring than the SPI mode drivers.
Wiring and discussion - https://forums.parallax.com/discussion/174988/new-sd-mode-p2-accessory-board/p1
FlexC driver and tester - https://forums.parallax.com/discussion/comment/1564225/#Comment_1564225
Spin2 driver - https://forums.parallax.com/discussion/comment/1563777/#Comment_1563777
Spin2 tester - https://forums.parallax.com/discussion/comment/1563936/#Comment_1563936
Also, along the way I did a Spin2 cleaned up SPI mode driver that corrected buggy behaviour - https://forums.parallax.com/discussion/comment/1563810/#Comment_1563810
Not been monitoring it closely, but definitely interested.
The speed of the SD card access is definitely one of the major limiting factors in compiling C programs on the P2, so I will look into your progress with interest when I get some time.
I think I will be reducing my support for the Parallax Serial Terminal. It's not really a very useful option**. There are plenty of alternatives that can do more than it can do, including the two options (payload and Comms) already bundled with Catalina. Then there are things like puTTY, Tera Term and minicom. And probably a zillion others.
Of course it will still be possible to use the Parallax Serial Terminal, but only if you disable much of the really neat functionality now offered by Catalyst, like command line editing, command completion and command history.
Ross.
** Not set in stone yet. If anyone knows of a reason for continuing support of the Parallax Serial Terminal, please let me know.
Catalina 8.3 has been released on GitHub and SourceForge
This is a full release, but it is mostly just a few bug fixes, and to rationalize payload's CR/LF handling, which was getting too unwieldy by trying to support the Parallax Serial Terminal. Now, you have to recompile if you want to use that as your serial terminal emulator.
Here is the relevant extract from the README.TXT
Finally, this will probably be the last Ubuntu release (Ubuntu binaries were released on SourceForge). Since I have decided to move to Pi OS as my primary development environment (which is essentially Debian Linux) I have decided to stop struggling with Ubuntu.
I will continue to build Windows releases but perhaps only for major releases. It takes 10 times as long to build and test on Windows than it does on Pi OS or Linux.
I appreciate that.
can a Pi400 handle it? I still have one of those that I used to do some development on.
Yes, I'm sure it would. Pi OS is basically Debian 12 (bookworm), and the current version runs on all Pi models. I also ran it on a Pi 3 and it worked, but it was a little sluggish. The Pi 400 should be fine.
Ross.