New version of yymodem - various bug fixes and improvements, and now tested and working on Propeller 1 and Propeller 2 (but see the README.TXT file about Propeller 1 limitations)
YMODEM FOR CATALINA/CATALYST
============================
This folder contains an implementation of the YMODEM file transfer protocol
for Catalina and Catalyst. It is derived from a free public domain version
of YMODEM written by Fredrik Hederstierna.
YMODEM can be used to send and receive both text and binary files using the
serial connection from the Propeller to a host PC, or vice-versa. Transfers
are done in 128 (YModem) or 1024 byte (YModem-1K) packets, and each packet
is protected by a 16 bit CRC, and retried if it fails, so transfers are very
reliable and transmission errors are extremely rare.
The impetus for this program was that I was damaging my Propeller boards and
also my PC by inserting and removing SD Cards so often - typically because I
had to update the software on them. Both the Host and the Propeller SD Card
sockets can eventually be damaged or become unreliable, so I wanted a simple
file transfer mechanism that would help me reduce the need for doing this -
and YMODEM (although it dates from the days of modems and bulletin boards)
does the job nicely!
To execute on the Propeller, the program must be compiled to use a serial
plugin and library (e.g. using the tty256 serial plugin on the Propeller 1,
or the 2 port serial plugin on the Propeller 2) and cannot also use a serial
HMI option (if this is the default, specify -C NO_HMI). If the plugin supports
multiple ports then by default the program uses serial port 0.
There is only one YMODEM program file (fymodem.c) plus a serial support file
(rs232.c) which allows it to be compiled using gcc, but the program can be
compiled as either a sender or a receiver by defining the symbols YMODEM_SEND
or YMODEM_RECEIVE in the compile commmand. For example, the following commands
might be used to compile a YMODEM receive and YMODEM send program:
catalina -p2 fymodem.c -D YMODEM_RECEIVE -C NO_HMI -lcx -lserial2 -o receive
catalina -p2 fymodem.c -D YMODEM_SEND -C NO_HMI -lcx -lserial2 -o send
The programs can be compiled for the Host PC using MinGW under Windows, or
gcc under Linux. Note that you need to include the rs232.c serial interface
code. For example:
gcc fymodem.c rs232.c -D YMODEM_RECEIVE -o receive
gcc fymodem.c rs232.c -D YMODEM_SEND -o send
There is a build_all script provided that does all this for you. For example,
you can just say:
build_all P2_EDGE
or
build_all C3 FLASH CACHED_1K
This will build two PC executables:
send.exe <-- or just 'send' for Linux
receive.exe <-- or just 'receive' for Linux
which will build two Propeller executables:
send.bin <-- or 'send.binary' for the Propeller 1
receive.bin <-- or 'receive.binary' for the Propeller 1
The format of the commands under Windows or Linux is:
send [-h] [-v] [-d] [-x] [-p PORT] [-b BAUD] [-t TIMEOUT] [-s DELAY] file
receive [-h] [-v] [-d] [-x] [-p PORT] [-b BAUD] [-t TIMEOUT] [file]
The format of the commands on the Propeller is similar, except for the baud
rate option. On the Propeller the baud rate cannot be specified on the command
line - it must be pre-configured in the platform configuration files (on the
Propeller 1, this is the file Extras.spin, on the Propeller 2 it is in the
platform.inc file - e.g. P2_EDGE.inc). Also note that the Makefile provided
adds -C NO_HMI when compiling the program for the Propeller, which makes the
-h, -v and -d options a bit useless. This is because ymodem generally uses
the same port as a serial HMI (which is often the default HMI), so the HMI
cannot be used at the same time. However, if your platform can use a
non-serial HMI option (such as VGA) you can edit the Makefile to remove
the -C NO_HMI option and add your preferred HMI option (e.g. -C VGA).
The default baud rate is 230400 baud. This is fine on the Propeller 2 but is
probably too fast on the Propeller 1, where a baud rate of 115200 is generally
the default. Also, the Propeller 1's smaller serial buffer sizes and slower
serial plugins means the -s option must usually be specified for the sender
(the -s option applies ONLY to the sender). This option does two things:
1. Tells the sender to only send 128 byte blocks, not 1024 byte blocks -
i.e. use YModem and not YModem-1K.
2. Adds a delay of the specified number of milliseconds between each
character sent - often -s0 will work, but if not try -s5, -s10 etc.
Currently, only one file can be transferred at a time. This may change in
a future release.
The send command must include the name of the file to be sent, but in the
receive command this is optional. It will be used if specified, but if not
then the received file will be given the same name as the sent file.
You can terminate an executing receive or send program by entering two
successive [CTRL-X] characters.
From Windows you can use the stand-alone ymodem programs on both the Propeller
and the Host PC, or you can use them only on the Propeller and use payload
(which now supports YMODEM) on the PC. Start payload as normal in interactive
mode, even if the Propeller uses a non-serial HMI option. Execute the ymodem
send or receive program on the Propeller first, and then within payload, press
CTRL-A and you will see a menu appear. Press Y for YModem and then either S
for a YModem Send, T for a YModem-1K Send or R for a YModem Receive. You must
enter a file name when Sending, but this is optional when Receiving.
A terminal emulator program that supports YMODEM may also be used, such as
Tera Term or ExtraPuTTY (on Windows) or minicom or picocom (on Linux).
However, note that the Propeller 1 can only support YModem transfers, not
YModem-1K, and some terminal emulators (e.g. Tera Term and ExtraPuTTY) assume
they can always use YModem-1K.
When the YMODEM program is started and you are using a terminal emulator,
you may see 'C' characters being printed repeatedly in the terminal window -
this is normal, and is how the YMODEM send and receive programs synchronize
with each other.
IMPORTANT NOTE WHEN USING WINDOWS:
==================================
Windows recently introduced a new console mode that is very buggy. Programs
such as payload that use console mode can fail unexpectedly if the new console
mode is used. This requires more investigation, but until Microsoft fixes the
issues, it is recommended that the "legacy console mode" be used with payload
and other Catalina programs. To ensure this, select Properties from the system
menu of any console window and in the Options tab, ensure that the option
to "Use legacy console" is selected.
EDIT: Updated to match the version included in Catalina 5.5
@msrobots said:
how about using the sd/host filesystem @ersmith is using in Flex?
It is a nice implementation, it is neat integrated to use from C and Basic, just the Spin support is a bit lacking.
Having that in Catalina/Catalyst would be fantastic.
Mike
I'm not familiar with Flex. Does it support transferring files between the host PC and the Propeller SD Card? That's what I needed, and now that I have yymodem working reliably at 230400 baud, it's easy to use when using Catalyst via a terminal emulator, since many of them already have support for it built in. Catalina's own terminal emulator (payload) currently does not, but it will be easy to add, so I will probably do that for the next release.
The FAT filesystem in FlexProp is elm-chan.org/fsw/ff/00index_e.html one, which is pretty widely used in the microcontroller world. For the host PC file system I used a Plan 9 protocol server on the PC side and wrote my own code on the Prop side; it's a very clean protocol and easy to implement (not surprising considering where it came from!). The PC side code is in both loadp2 and my version of proploader, so it works on both P1 and P2. To copy files from host to SD I just run a shell that mounts both the host FS and FAT FS, and then use "copy /host/file.txt /sd/file.txt".
Having said all that, YMODEM is a fine solution for file copies as well, it just serves a slightly different function (although you may well be able to put a file system interface on top of the YMODEM protocol?)
The nice thing about the Plan 9 protocol is that in principle it gives a standard way to talk to any file or device, so we could (theoretically) expose I2C, SPI, and any kind of disk drive (including FAT) via a 9P interface, which would make it easy for code to interoperate between Catalina, FlexProp, Spin2, or whatever. 9P does impose some overhead compared to raw device accesses though, so it isn't widely used in the embedded space.
Sounds useful. Does the Plan 9 system mount the PC disk as a file system on the Propeller, or the Propeller SD Card as a file system on the PC? Or perhaps it can do either? Could it also be used to mount one Propeller SD Card file system on another Propeller?
I did something similar using my proxy SD card driver on the P1 - i.e. you can mount an SD card from one Propeller as the file system on another, using a simple 2-pin serial connection. This was used in the various multi-Propeller systems such Cluso's and Bill Henning's boards. There was a craze for these multi-Propeller P1 systems before the P2 came along, when people were desperate for more pins, more RAM and more speed! But I have not bothered porting it to the P2 yet because there are no multi-Propeller 2 systems yet (AFAIK!). But give it time, and I'm sure some new lunatic will come along ...
The Plan 9 software might be better than mine if it is written in C - mine was written in P1 assembler, so it will need porting and re-testing for the P2. But on the other hand mine also supports other proxy devices such as the mouse, screen and keyboard. I'll look into the Plan 9 stuff when I get some time. For the moment, yymodem solves the problem I needed solved, and already had software support on the PC side. The current version now works at full speed with all the terminal emulators I have tried so far - e.g. minicom, PuTTY, Tera Term.
I've updated the post above (here) with a new version of ymodem. The version has various bug fixes and improvements, and has now been now tested and is working on both the Propeller 1 and the Propeller 2.
This version will also be included in the next release of Catalina.
YModem support has now been added to the payload loader. This is convenient on Windows, but it is required for Linux - on Windows you can easily use the stand-alone YModem send and receive programs, but Linux toggles DTR (which resets the Propeller) whenever you close and re-open a serial port, so you have to use a terminal emulator that supports YModem so that it does not have do that before the YModem transfer can be started. This version of payload will be included in the next release, but it is not worth doing a release just for this.
I have never found a general solution for this Linux/DTR issue - if anyone knows of one please let me know!
@RossH said:
I have never found a general solution for this Linux/DTR issue - if anyone knows of one please let me know!
Ross.
Linux normally does this because serial ports were traditionally all connected to modems, for which this is the right thing to do. You can turn it off with stty -hup -F /dev/whatever, but turning it off causes DTR to transition, resetting the P2, so make sure you run that command as soon as you plug in the device, before you try programming it.
It should be possible to write a udev rule to make stty -hup get called automatically whenever you plug in a serial device.
I wouldn't be surprised if some of the more "helpful" GUI terminal emulators turn HUP back on for you, but I just confirmed that the one I usually use, picocom, leaves the setting alone (unless you explicitly ask it to change it for you).
EDIT: It's probably also a good idea to add -ixon -ixoff to stty's options to make the terminal not interpret Ctrl-S and Ctrl-Q as flow control characters.
@RossH said:
I have never found a general solution for this Linux/DTR issue - if anyone knows of one please let me know!
Ross.
Linux normally does this because serial ports were traditionally all connected to modems, for which this is the right thing to do. You can turn it off with stty -hup -F /dev/whatever, but turning it off causes DTR to transition, resetting the P2, so make sure you run that command as soon as you plug in the device, before you try programming it.
It should be possible to write a udev rule to make stty -hup get called automatically whenever you plug in a serial device.
I wouldn't be surprised if some of the more "helpful" GUI terminal emulators turn HUP back on for you, but I just confirmed that the one I usually use, picocom, leaves the setting alone (unless you explicitly ask it to change it for you).
EDIT: It's probably also a good idea to add -ixon -ixoff to stty's options to make the terminal not interpret Ctrl-S and Ctrl-Q as flow control characters.
Thanks, but I've tried all the stty and ioctl tricks that are reported by various people to solve this problem and none of them make any difference for me. At least not on my version of Linux (Ubuntu 18). I don't think there is a general solution to this problem. I know some people have resorted to writing their own kernel drivers in desperation, but that's difficult and also version-specific.
@RossH said:
I have never found a general solution for this Linux/DTR issue - if anyone knows of one please let me know!
Ross.
Linux normally does this because serial ports were traditionally all connected to modems, for which this is the right thing to do. You can turn it off with stty -hup -F /dev/whatever, but turning it off causes DTR to transition, resetting the P2, so make sure you run that command as soon as you plug in the device, before you try programming it.
It should be possible to write a udev rule to make stty -hup get called automatically whenever you plug in a serial device.
I wouldn't be surprised if some of the more "helpful" GUI terminal emulators turn HUP back on for you, but I just confirmed that the one I usually use, picocom, leaves the setting alone (unless you explicitly ask it to change it for you).
EDIT: It's probably also a good idea to add -ixon -ixoff to stty's options to make the terminal not interpret Ctrl-S and Ctrl-Q as flow control characters.
Thanks, but I've tried all the stty and ioctl tricks that are reported by various people to solve this problem and none of them make any difference for me. At least not on my version of Linux (Ubuntu 18). I don't think there is a general solution to this problem. I know some people have resorted to writing their own kernel drivers in desperation, but that's difficult and also version-specific.
Ross.
That's very strange. That's the standard way of doing it - it should work. What USB-serial converter are you using? What terminal emulator have you been using on Linux?
@Electrodude said:
That's very strange. That's the standard way of doing it - it should work. What USB-serial converter are you using? What terminal emulator have you been using on Linux?
I am just using a Prop Plug. I generally use my own terminal emulator (payload) and stty hup or stty -hup etc makes no difference. But I have just tried picocom and interestingly it can open the port without resetting the Prop. Again, stty hup or stty -hup makes no difference to it, but clearly it is possible. There must be a combination of ioctl options I have not tried which overrides the port settings.
I'll take a look at the picocom source code to see how it does it.
@RossH said:
I'll take a look at the picocom source code to see how it does it.
Thanks.
Ha! Comparing the picocom code to the code I use, everything looked fine - except that the code I use contains some ADDITIONAL code specifically designed to avoid toggling DTR! When I take that code out, payload behaves the same way as picocom.
It is possible I added that code in the early days of Catalina when I was using a different flavor of Linux. Or perhaps the Linux serial driver changed at some point. It's not worth going back to find out. I will just add a suitable note, make it configurable, and include the change in the next release.
@RossH said:
I'll take a look at the picocom source code to see how it does it.
Thanks.
Ha! Comparing the picocom code to the code I use, everything looked fine - except that the code I use contains some ADDITIONAL code specifically designed to avoid toggling DTR! When I take that code out, payload behaves the same way as picocom.
It is possible I added that code in the early days of Catalina when I was using a different flavor of Linux. Or perhaps the Linux serial driver changed at some point. It's not worth going back to find out. I will just add a suitable note, make it configurable, and include the change in the next release.
Ross.
Are you sure you ever actually needed that code to begin with, or is it possible you just put it in because, say, you thought it'd fix a problem that was really somewhere else? Why don't you just tear it out, with no configuration option, until somebody complains, and only then consider putting some configuration option in. Less code means less room for future bugs...
@Electrodude said:
Are you sure you ever actually needed that code to begin with, or is it possible you just put it in because, say, you thought it'd fix a problem that was really somewhere else? Why don't you just tear it out, with no configuration option, until somebody complains, and only then consider putting some configuration option in. Less code means less room for future bugs...
I have wrestled with this problem several times over the years, and so has Linux (evident from looking at various old Linux bug reports about DTR/DSR). I just went back through old versions of Catalina, and it looks like I did indeed write this particular code. So I think it was probably both necessary and worked at the time - but I would probably have been using Red Hat/Fedora at that point, not Debian/Ubuntu.
In any case I have effectively removed it by putting #if ... #endif around it and also putting a note in the source file so it will be obvious in future for anyone who looks that enabling it might offer a solution.
I have just posted Catalina 5.5 to SourceForge here.
This is a full release, primarily to update Catalyst and Payload to add YMODEM support, and also fix the Payload DTR issue on Linux. Plus a few minor things.
Here is the relevant extract of the README.TXT file:
RELEASE 5.5
New Functionality
-----------------
1. A version of the YModem serial file transfer protocol has been added to
Catalyst. When Catalyst is built, there will be two more Propeller programs
added to the demos\catalyst\bin directory:
send.bin - send a file from the Propeller to the host.
receive.bin - receive a file on the Propeller sent from the host.
Also, in the demos\catalyst\fymodem directory, there will be two
corresponding host binaries (on Windows these will be .exe files):
send - send a file from the host to the Propeller
receive - receive a file on the host sent from the Propeller
See the README.TXT file in demos\catalyst\fymodem for more details.
2. A version of the YModem serial protocol has been integrated into payload's
interactive mode. This can be used for file transfers even if the Propeller
does not use a serial HMI option.
3. Payload now supports configuring the line termination mode from within
interactive mode as well as on the command line (i.e. using the -q command
line option).
4. Payload now has menus. Pressing the "attention" key (CTRL-A by default)
in payload's interactive mode now brings up a menu from which you can
select either Terminal Configuration, YModem Transfer, or to Exit from
payload (as an alternative to pressing CTRL-D twice).
5. The ex_leds_1.c and ex_leds_2.c example programs in demos\examples now
support the P2_EDGE board.
6. Payload now responds correctly to the DEC VT100 Cursor Position Requests
(CUP) and responds to a Device Status Report (DSR) specifying parameter 6
with a Cursor Position Report (CPR). Taken together, this allows programs
expecting a VT100 compatible terminal to determine the actual terminal
window size, rather than simply assuming it will always be 80x24.
7. The Catalyst vi program now uses the DEC VT100 CUP and DSR requests to
elicit a CPR, which allows it to determine the actual window size.
This means that vi is no longer limited to 80x24 windows when a serial
HMI option is used. In particular, when payload is used, the actual window
size will now be reported correctly to vi on startup, and vi will use it
rather than defaulting to 80x24 whenever a serial HMI is used. The default
VT100 window size (80x24) will still be used if VT100 is specified in the
build and the terminal emulator cannot respond to CUP and DSR requests.
For example, to use vi in a terminal window of 50 lines and 120 columns,
and assuming you are using a serial HMI and specified the VT100 option
when vi was built, then you might start payload as follows (assuming N is
the comms port connected to your Propeller):
payload -i -g120,50 -b230400 -pN
Then within payload just start vi as normal and it will use the correct
window size - e.g:
vi catalyst.txt
Other Changes
-------------
1. The files s2_serial.h, s4_serial.h and s8_serial.h were not correctly
defining the symbols s2_getc, s4_getc and s8_getc respectively. These
names were only intended to be used to emulate the original Spin names -
the actual underlying C functions (i.e. s2_rxcheck, s4_rxcheck and
s8_rxcheck respectively) were all fine.
2. The Catalina Optimizer may have reported undefined symbols when optimizing
programs that used the serial2 or serial8 libraries.
3. The header files for the tty, tty256, serial2, serial4 and serial8
libraries now check and issue a message (using #error) if the libraries
they represent are not supported on the Propeller type for which the
program is being compiled:
- The serial2 and serial8 libraries are only supported on the Propeller 2.
- The tty, tty256 and serial4 libraries are only supported on the Propeller
1 (but note this has nothing to do with the TTY HMI option, which applies
to both the Propeller 1 and Propeller 2 - the TTY HMI option does not
use these libraries).
4. The rs232 interface module (rs232.h, rs232.c) used by payload and fymodem
included code intended to prevent DTR being toggled when opening a serial
port. This code no longer works correctly on Debian Linux and has been
disabled. If it is required on other platforms, it can be re-enabled by
setting rs232_DTR_FIX to 1 in the file rs232.h. Note that copies of this
file exist in both the folder source/catalina as well as the folder
demos/catalyst/fymodem folder, and will need to be modified in both places.
5. The help text for payload's -q command line option had the values for
enabling CR_TO_LF and LF_TO_CR translation the wrong way around.
There is not much new stuff in this release, but I needed to get it out because I won't as have much time for Catalina for the next few months - it is coming into our "busy" season here. Also, I think this may well be the last "monolithic" release. The overheads of doing this are now much too high. I will think about the best way to break it down into more manageable chunks in future - at the very least, I will probably have separate releases for Catalina and Catalyst.
I have just posted Catalina 5.5.1 to SourceForge here.
This is a patch release, and there are no changes to Catalina itself. The purpose of the release is to introduce a new Windows-based terminal emulator, and to update payload to use it.
Payload's built-in interactive mode is still available, but I got tired of fighting with Microsoft's console mode, so I decided it was best to stop using it. I dusted off a terminal emulator I developed a while ago - for essentially the same reason - and added a few Propeller-friendly specifics (basically, the ability to control DTR, support for additional baud rates common on the Propeller, and YModem support).
Note that the new payload option allows any terminal emulator to be used. Examples of using it to invoke PuTTY (on Windows) or minicom (on Linux) are included. Also note that the new terminal emulator is completely separate from Catalina, and can also be used stand-alone (it is also available here.
Here is the relevant extract of the README.TXT file:
RELEASE 5.5.1
New Functionality
-----------------
1. The Windows version of Catalina now includes a new serial comms program
with a full-function terminal emulator, which can be used as a replacement
for payload's very simple interactive mode by specifying "-I terminal"
instead of just "-i" on the payload command line - see below for more
details).
The binary for the program (comms.exe) is in Catalina's bin folder,
and the source is in source\comms, but note that the source is not built
by default if Catalina is rebuilt, since it requires GNAT (the GNU Ada
Translator) and the GNAT Studio development environment to be installed.
See the BUILD.TXT file in source\comms for details on how to rebuild the
comms program from source.
2. Payload now allows an external terminal emulator program to be executed in
place of the simple internal one. The -i option still enables the internal
emulator, but a new -I option (i.e. upper case) calls an external terminal
emulator program. The -I option accepts one parameter which specifies the
command to be used. The port name (not the port number) and baud rate will
be passed to this program on startup.
For example, if the payload command executed was:
payload program.bin -p11 -b230400 -I vt100
Then after loading the program.bin file, the command executed (on Windows)
would be:
vt100 COM11 230400
On Linux, the command would be something like:
vt100 /dev/ttyUSB0 230400
A suitably named script (i.e. on Windows a batch file called vt100.bat,
or on Linux a shell script called vt100) can be used to specify any other
required parameters.
The following Windows scripts are provided (each one is a batch file):
pc <-- start comms, specifying PC emulation
vt52 <-- start comms, specifying VT52 emulation
vt100 <-- start comms, specifying VT100 emulation
vt101 <-- start comms, specifying VT101 emulation
vt102 <-- start comms, specifying VT102 emulation
vt220 <-- start comms, specifying VT220 emulation
vt320 <-- start comms, specifying VT320 emulation
vt420 <-- start comms, specifying VT420 emulation
vt100_putty <-- start PuTTY, specifying VT100 emulation
The following Linux scripts are provided (each one is a shell script):
ansi <-- start minicom, specifying ANSI emulation
vt100 <-- start minicom, specifying VT100 emulation
vt102 <-- start minicom, specifying VT102 emulation
Note that these scripts can be used independently of payload. They all
accept two parameters - the com port to use, and the baud rate.
For example:
vt100 COM11 230400
vt102 /dev/ttyUSB1 115200
Note that the comms program executable is provided as part of the
Windows version of Catalina, but PuTTY and minicom executable will have
to be installed separately. Ensure the appropriate executable is in the
PATH.
Other Changes
-------------
1. Payload now interprets either ESC [ ? 6 n or ESC [ 6 n as a DEC DSR
request and responds with a DEC CUP. Strictly speaking, ESC [ ? 6 n
is not a correct DEC DSR command, and would not be recognized by
other terminal emulators, so the Catalyst xvi program now uses the
more correct ESC [ 6 n
A change to the clock and SD Card plugin code (made in release 5.4) means that the Catalyst Super Star Trek demo now needs to be explicitly built with the -C CLOCK option.
I will fix this in the next release. but in the meantime, just edit the file demos\catalyst\sst\MAKEFILE.CAT and find the following line:
Another quick patch - release 5.5.2, available here.
Again, no changes to Catalina itself. Mainly just an update for the vi text editor.
Here is the relevant portion of README.TXT:
RELEASE 5.5.2
New Functionality
-----------------
1. The Catalyst vi editor (xvi) demo program has been updated to version 2.51,
which is the latest version and contains some new functionality and some
bug fixes. See the documents in demos\catalyst\xvi-2.51\doc for details.
2. The Catalyst vi editor now processes the keypad keys PgUp, PgDown, Home,
End, Ins & Del appropriately.
2. The Catalyst vi editor can now be compiled to use colour when the VT100 HMI
option is used, since colour is supported by the new external terminal
emulator (comms.exe). Just define the Catalina symbol USE_COLOR in addition
to VT100, either on the Catalina command line or by adding it as a parameter
to the build_all command. For example:
cd demos/catalyst/xvi-2.51
build_all P2_EDGE CR_ON_LF VT100 USE_COLOR
Note: Only foreground colours can be set within vi itself - the current
background colour (typically the background colour in use before vi was
started) will be used. The default colours are that text will be grey,
the status line will be green (normal files) or red (read-only files).
Try the vi command ":help" to see an example.
You can manually set the colours (for the text, status line, or read-only
status line) in vi as follows:
:set colour=n
:set statuscolour=n
:set roscolour=n
where n is:
0 : black
1 : red
2 : green
3 : yellow
4 : blue
5 : magenta
6 : cyan
7 : grey
9 : default foreground colour
10 : bold and bright black (dark grey)
11 : bold and bright red
12 : bold and bright green
13 : bold and bright yellow
14 : bold and bright blue
15 : bold and bright magenta
16 : bold and bright cyan
17 : bold and bright grey (white)
Note that when vi is compiled to use colour, the default vi colours will
override the current foreground colour on startup. To get it back again,
use the vi command ":set colour=9"
Any value other than those specified will set the colour to the default
foreground colour (typically, the foreground colour before vi was started).
If you are using comms.exe then you can also select the default colours
using the Format menu (remember to select the Format->Set All To Colours
menu item after choosing your new colours, or the changes will apply only
to new characters, not existing ones).
Note that you can also have other effects, such as inverse video -
(i.e. black text on white bg). To do this, start vi and then set the
foreground colour to black (i.e. ":set colour=0") - note that any text
on the screen may temporarily disappear. Then use the Format->Bg Color
menu item to select a white background, then use the Format->Set All
to Colour menu item to refresh the colours of the characters already
on display. You can choose to have any background colour you want.
Other Changes
-------------
1. A change to the clock and SD Card plugin code (made in release 5.4) means
that the Catalyst Super Star Trek demo now needs to be explicitly built
with the -C CLOCK option. The Makefile has been modified accordingly.
2. When the external terminal emulator's YModem dialog box is open, it is now
always on top. Otherwise, it could get hidden by other windows and it was
not obvious why the main comms window would not respond.
3. The external terminal emulator's YModem dialog box "Start" button is no
longer styled as if it was a default button, since it was not. There is no
default button for this dialog box.
4. The new external terminal emulator now locks the screen and view. This
makes it easier to set the initial screen size (can now be done just using
the /ScreenSize option, instead of also requiring the /ViewSize option).
This is more appropriate when emulating a physical terminal, where the
screen size would normally be the same as the view size. They can be
unlocked again (if required) via the Options->Advanced menu.
5. The scripts that call the external terminal emulator (vt100.bat etc) now
add /Wrap to enable wrap mode on startup. This option can also be changed
via the Options->Advanced menu. Setting it on by default makes the external
terminal emulator behave more like the internal terminal emulator.
6. The payload internal terminal emulator now limits the number of restart
retries on ymodem_receive, to avoid locking up permanently in case the
other end has not been started.
7. The file P2_EVAL.ZIP has been renamed P2_DEMO.ZIP, and the README_P2.TXT
file has been updated to indicate that this demo version of Catalyst
should work on the P2 EVAL, P2 EDGE or any other P2 board with a similar
SD Card configuration.
Just a quick note about Catalyst on the Propeller 2 ...
I discovered recently that the P2 EDGE and P2 EVAL boards (which are the only P2 boards I have) both boot programs much more reliably from FLASH than from the SD Card. Not sure why this is, but to take advantage of it Catalyst can be booted from FLASH just as easily as it can from the SD Card.
Compile Catalyst as usual and copy the results from the demos\catalyst\bin directory to an SD Card as usual, but then set the FLASH dip switch ON (and the P59^ and P59v dip switches OFF ) and use flash_payload to program Catalyst into FLASH. For example, if you are using the serial interface and also want to use the new vt100 terminal emulator, you would use commands like:
cd demos\catalyst\bin
flash_payload catalyst.bin -I vt100
Catalyst and all its demo programs should function as usual. Turn the FLASH dip switch OFF to revert to booting from the SD Card.
A note to this effect will be included in the next release.
Okay ... I said I wouldn't do it ... I said it was a mistake on the Propeller 1 ... but then I came down with Covid-19 and couldn't do much of anything else for the past week, so ...
I now have Catalina executing C programs using PSRAM as XMM on the Propeller 2. This means Catalina C programs can now be up to 16MB.
I currently only support the PSRAM on the P2 EDGE, but I am using the driver written by @rogloh so while I don't have one of the HyperRAM boards, I believe it would also work on that with just a few tweaks.
The P2 EDGE has 32MB of PSRAM, but there are some internal design decisions from the Propeller 1 days (when anything bigger than 32kb was a big deal) that limit programs to 16MB - but you can use the other 16MB as extra storage and access it from C.
My test suite executes correctly, and I just tried a simple benchmark program. The early results are quite promising:
So executing the code using XMM mode (i.e. from PSRAM) is about 4 times slower than executing the same code using NATIVE mode (aka P2 HUBEXEC mode), or about 2 times slower than the LMM or COMPACT modes (which are identical to the same modes on the Propeller 1). You can also have different cogs executing the different modes simultaneously, so you can have those parts of the program that need speed executing in NATIVE mode and the rest executing in XMM mode.
There is still a heap more to do and test and tidy up, but I hope to get at least a beta release out before Christmas.
Hell froze over! LOL. You've seen the light! Well done. Merry Xmas.
It's great that you used my driver as other COGs can still share the memory with QoS guarantees, like video drivers, etc - the executed code will slow down a bit if/when they do, but in many cases that's still okay. And as you say there are other memory types that would be supportable. I did HyperRAM and some smallish 2MB SRAM and still plan to do one for (Q)SPI FLASH which could give you some XIP (probably rather slow unless cached).
@rogloh said:
Hell froze over! LOL. You've seen the light! Well done. Merry Xmas.
It's great that you used my driver as other COGs can still share the memory with QoS guarantees, like video drivers, etc
Yes, the fact that I can use your driver from multiple cogs was a key factor. Eventually I hope to have XMM programs executing in multiple cogs (currently only one cog can be executing in XMM mode, the others have to execute one of the other modes). If that had not been a possibility I probably would not have bothered. But that will have to come in a later release.
The bad news is that I got held up by a nasty bug (no, not Covid-19 - I mean the programming kind), so I will not have a release of Catalina that executes programs from external RAM ready by Christmas. The bug was because the shift instructions (in this case SHR) behave differently on the Propeller 2 than they did on the Propeller 1. I mean, who on earth would mess about with an instruction as fundamental as that? (no, you don't need to tell me, I know the answer )
This put me behind schedule by a few days. And Christmas is now approaching fast!
The good news is now that I have fixed that, everything seems to be working fine. I have the largest, most complex program I routinely use (i.e. the Lua language interpreter) now executing from XMM RAM (i.e. PSRAM) on the Propeller 2. This program exercises pretty much everything - memory, file systems, various plugins etc. It means that I can now execute much larger Lua programs than was previously possible (which was in fact the whole point of the exercise).
Of course, just like on the Propeller 1, the performance of programs executing from external RAM on the Propeller 2 will never be anything to write home about. But as the Russian proverb says "The marvel is not that the bear dances well, but that the bear dances at all." But one reason is because I am still using just an 8kb cache to cache access to up to 16Mb of XMM RAM. This relatively tiny cache size is a hang over from the Propeller 1 days, when Hub RAM was so limited. On the Propeller 2 I will be able to make the cache much larger (say 32kb or 64kb) and that should improve the performance. I just haven't had time to do it yet.
Instead of a full release, I may post a demo version of just one program (probably the Lua interpreter), Anyone with a suitable P2 EDGE (you need the one with 32MB of PSRAM, i.e. the P2-EC32MB) will be able to load it and execute it.
The standalone Propeller 2 XMM Demo has been removed from SourceForge since an improved (but incompatible) version of Propeller 2 XMM support is now incorporated in Catalina release 5.6.
I had a suspicion my original benchmark was too optimistic, and I was right. The choice of benchmarks makes a huge difference tp the results. Also, I have now had time to expand the cache size to 64kb, and this improves things, but not as much as I had hoped.
Benchmark 1 is too trivial to be much use - it is a variation of the original benchmark, and it understates the execution time of XMM. Benchmarks 2 and 3 are closer to reality - which is that executing from external RAM can be twenty times slower than executing natively from Hub RAM. I think this can be improved with a better cache - the current one - a hangover from the Propeller 1 - was primarily optimized for size, not speed, and just making it larger has not improved things as much as I had hoped. A redesign looks to be in order.
Still, things are not as bad as all that. Benchmark 3 represents pretty much the largest program that can execute either in Native or LMM mode on the P2. For programs larger than that, it comes down to a choice between either CMM or XMM, and XMM is only three times slower than CMM, but allows programs thirty two times larger.
EDIT: Updated final paragraph, which was overly pessimistic.
A quick post-Christmas update. I've now implemented an XMM SMALL mode in addition to the XMM LARGE mode on the Propeller 2 (the Propeller 1 already had both of these). The good news is that SMALL mode is significantly faster than LARGE mode.
For those unfamiliar with XMM execution (which would probably be many Propeller 2 only users), Catalina offers several different program execution modes:
NMM : Propeller 2 Native (HUBEXEC) mode. Combined Code, Data, Stack and Heap limited to 512kb (i.e. Hub RAM). LMM : A legacy mode carried over from the Propeller 1 (essentially, the Propeller 1's equivalent to HUBEXEC mode). Similar code sizes and limits to NMM. CMM : A compact version of LMM. Combined Code, Data, Stack and Heap still limited to 512kb, but Code size is generally about half that of LMM or NMM.
XMM SMALL : Code in external RAM, Data, Stack and Heap in Hub RAM. Maximum code size is 16Mb, maximum combined Data, Stack and Heap is 512kb.
XMM LARGE : Code, Data and Heap in external RAM, Stack in Hub RAM. Maximum combined Code, Data and Heap size is 16Mb, maximum Stack size is 512kb.
I have also made a few other small improvements, so here are the most recent benchmark results, with values adjusted to make them all relative to NMM mode:
NMM = 1
LMM = 4
CMM = 6
XMM SMALL = 13
XMM LARGE = 18
LMM is not particularly useful on the Propeller 2 (it was retained mainly because I knew it would provide an easy path to XMM). Once your program won't fit in Hub RAM and so you cannot use NMM any longer (which is obviously still the preferred option) you would first move to CMM and at that point you take a pretty large performance hit. Even so, CMM can still useful (on the Propeller 1, CMM is comparable in speed to Spin). For instance, many of Catalina's own utilities (such as Catalyst, and the Lua interpreter) are compiled in CMM mode. But once you need even more code or data space after that then you now have two further options - SMALL and LARGE mode. The latest addition (SMALL) allows code sizes up to 32 times larger than CMM but at the cost of executing at half the speed. You would move to LARGE mode only once your data size (i.e. not just code size) exceeds the 512kb of available Hub RAM. Also, keep in mind that if some parts of your program absolutely need speed, they can be executing in NMM mode on some cogs while other parts of your program are executing in one or more of the slower modes on other cogs.
I have a few ideas about how to improve the performance of XMM further, but they will require a substantial amount of re-work (which implementing the existing XMM modes did not, since they were already supported on the Propeller 1) and I want to get the new Propeller 2 implementations bedded down properly first.
The bad news is that this is still a little way off - it is so long since I did any XMM work on the Propeller that I had forgotten that most of the complexity is not in the execution itself, it is in all the supporting utilities (program loaders etc) that have to be updated
I've been following your Prop2 XMM thread and see that you started with the Prop1 XMM capability as the baseline for it.
To some extent you've already covered the following questions but I want to make sure I understand where the Prop2 XMM development stands right now:
The Prop2 XMM code can only be executed from a single cog, correct?
But the remaining cogs can execute code using any of the other non-XMM models (NMM,LMM,CMM)?
Does the Prop2 XMM model use a C structure to share common variables among the other cogs like the Prop1 XMM?
Do you eventually plan on allowing multiple cogs to simultaneously execute Prop2 XMM code?
Do you anticipate any of the techniques used in the Prop2 XMM to backfeed into an enhanced Prop1 XMM? Like, for example, allowing more than one cog to execute XMM code on the Prop1?
(Somewhere I thought I came across a writeup wherein the GCC developers claim to have enabled the Prop1 to execute XMM code on multiple cogs with as little as 2K cache for each cog).
Obviously multi-cog XMM execution on either the Prop1 or Prop2 wouldn't be a speed demon and performance would depend upon what type of external memory is used (among other factors) but it might be an interesting capability nevertheless.
But allowing multi-cog XMM execution seems to be quite the task as the caching cog(s) would have to prioritize and deconflict the access to external memory. A parallel dual port memory arrangement might help in that regard but it appears to be much more difficult attempting that with standard serial RAM.
It would be interesting to see what your solution to that problem would be.
The Prop2 XMM code can only be executed from a single cog, correct?
Yes. The same as on the Propeller 1. But this is largely a limitation of the current cache plugin, not the XMM plugin.
But the remaining cogs can execute code using any of the other non-XMM models (NMM,LMM,CMM)?
Yes. The same as on the Propeller 1.
Does the Prop2 XMM model use a C structure to share common variables among the other cogs like the Prop1 XMM?
Yes. The same as on the Propeller 1.
Do you eventually plan on allowing multiple cogs to simultaneously execute Prop2 XMM code?
Yes. This should be possible on the Propeller 2.
Do you anticipate any of the techniques used in the Prop2 XMM to backfeed into an enhanced Prop1 XMM? Like, for example, allowing more than one cog to execute XMM code on the Prop1?
Not likely. On the Propeller 1 both Hub space and cog space are too limited. But with the Propeller, you should never say never!
(Somewhere I thought I came across a writeup wherein the GCC developers claim to have enabled the Prop1 to execute XMM code on multiple cogs with as little as 2K cache for each cog).
Not with the cache architecture I am currently using, which was originally the same one GCC used. But I didn't really stay in touch with the GCC progress, because Catalina was always so much more capable and GCC could never do what I needed. But I'll check to see if they later came up with a better cache architecture that the one which I adopted. If so, I could perhaps use it on both the Propeller 1 and the Propeller 2. If you or anyone else has a reference or link, please let me know.
Obviously multi-cog XMM execution on either the Prop1 or Prop2 wouldn't be a speed demon and performance would depend upon what type of external memory is used (among other factors) but it might be an interesting capability nevertheless.
Yes it is definitely not a speed demon! My main purpose in implementing XMM on the Propeller 2 is to enable larger Lua programs, and eventually multi-processing Lua programs. Lua is my favorite language these days, and it is such a rich language that speed is not really a major issue. C and Lua are highly inter-operable - you use C where speed is necessary, and Lua where things are simply too difficult to code in C. Actually, you have just raised a good point - I don't yet have a means of invoking non XMM C from Lua. You can invoke C but it will always execute as XMM C. I should make it possible to invoke NMM C from Lua for when you need full speed. Just another thing to add to the "to do" list!
But allowing multi-cog XMM execution seems to be quite the task as the caching cog(s) would have to prioritize and deconflict the access to external memory. A parallel dual port memory arrangement might help in that regard but it appears to be much more difficult attempting that with standard serial RAM.
The PSRAM XMM plugin already allows multiple cog access, so a simplistic solution would be to simply have multiple independent caches all using the same XMM plugin. But this would consume too many cogs to be much use. It may be a valid approach if we ever see the 16 cog version of the Propeller 2, but I have another approach in mind. However, I haven't had time to investigate it yet.
It would be interesting to see what your solution to that problem would be.
Comments
New version of yymodem - various bug fixes and improvements, and now tested and working on Propeller 1 and Propeller 2 (but see the README.TXT file about Propeller 1 limitations)
YMODEM FOR CATALINA/CATALYST ============================ This folder contains an implementation of the YMODEM file transfer protocol for Catalina and Catalyst. It is derived from a free public domain version of YMODEM written by Fredrik Hederstierna. YMODEM can be used to send and receive both text and binary files using the serial connection from the Propeller to a host PC, or vice-versa. Transfers are done in 128 (YModem) or 1024 byte (YModem-1K) packets, and each packet is protected by a 16 bit CRC, and retried if it fails, so transfers are very reliable and transmission errors are extremely rare. The impetus for this program was that I was damaging my Propeller boards and also my PC by inserting and removing SD Cards so often - typically because I had to update the software on them. Both the Host and the Propeller SD Card sockets can eventually be damaged or become unreliable, so I wanted a simple file transfer mechanism that would help me reduce the need for doing this - and YMODEM (although it dates from the days of modems and bulletin boards) does the job nicely! To execute on the Propeller, the program must be compiled to use a serial plugin and library (e.g. using the tty256 serial plugin on the Propeller 1, or the 2 port serial plugin on the Propeller 2) and cannot also use a serial HMI option (if this is the default, specify -C NO_HMI). If the plugin supports multiple ports then by default the program uses serial port 0. There is only one YMODEM program file (fymodem.c) plus a serial support file (rs232.c) which allows it to be compiled using gcc, but the program can be compiled as either a sender or a receiver by defining the symbols YMODEM_SEND or YMODEM_RECEIVE in the compile commmand. For example, the following commands might be used to compile a YMODEM receive and YMODEM send program: catalina -p2 fymodem.c -D YMODEM_RECEIVE -C NO_HMI -lcx -lserial2 -o receive catalina -p2 fymodem.c -D YMODEM_SEND -C NO_HMI -lcx -lserial2 -o send The programs can be compiled for the Host PC using MinGW under Windows, or gcc under Linux. Note that you need to include the rs232.c serial interface code. For example: gcc fymodem.c rs232.c -D YMODEM_RECEIVE -o receive gcc fymodem.c rs232.c -D YMODEM_SEND -o send There is a build_all script provided that does all this for you. For example, you can just say: build_all P2_EDGE or build_all C3 FLASH CACHED_1K This will build two PC executables: send.exe <-- or just 'send' for Linux receive.exe <-- or just 'receive' for Linux which will build two Propeller executables: send.bin <-- or 'send.binary' for the Propeller 1 receive.bin <-- or 'receive.binary' for the Propeller 1 The format of the commands under Windows or Linux is: send [-h] [-v] [-d] [-x] [-p PORT] [-b BAUD] [-t TIMEOUT] [-s DELAY] file receive [-h] [-v] [-d] [-x] [-p PORT] [-b BAUD] [-t TIMEOUT] [file] The format of the commands on the Propeller is similar, except for the baud rate option. On the Propeller the baud rate cannot be specified on the command line - it must be pre-configured in the platform configuration files (on the Propeller 1, this is the file Extras.spin, on the Propeller 2 it is in the platform.inc file - e.g. P2_EDGE.inc). Also note that the Makefile provided adds -C NO_HMI when compiling the program for the Propeller, which makes the -h, -v and -d options a bit useless. This is because ymodem generally uses the same port as a serial HMI (which is often the default HMI), so the HMI cannot be used at the same time. However, if your platform can use a non-serial HMI option (such as VGA) you can edit the Makefile to remove the -C NO_HMI option and add your preferred HMI option (e.g. -C VGA). The default baud rate is 230400 baud. This is fine on the Propeller 2 but is probably too fast on the Propeller 1, where a baud rate of 115200 is generally the default. Also, the Propeller 1's smaller serial buffer sizes and slower serial plugins means the -s option must usually be specified for the sender (the -s option applies ONLY to the sender). This option does two things: 1. Tells the sender to only send 128 byte blocks, not 1024 byte blocks - i.e. use YModem and not YModem-1K. 2. Adds a delay of the specified number of milliseconds between each character sent - often -s0 will work, but if not try -s5, -s10 etc. Currently, only one file can be transferred at a time. This may change in a future release. The send command must include the name of the file to be sent, but in the receive command this is optional. It will be used if specified, but if not then the received file will be given the same name as the sent file. You can terminate an executing receive or send program by entering two successive [CTRL-X] characters. From Windows you can use the stand-alone ymodem programs on both the Propeller and the Host PC, or you can use them only on the Propeller and use payload (which now supports YMODEM) on the PC. Start payload as normal in interactive mode, even if the Propeller uses a non-serial HMI option. Execute the ymodem send or receive program on the Propeller first, and then within payload, press CTRL-A and you will see a menu appear. Press Y for YModem and then either S for a YModem Send, T for a YModem-1K Send or R for a YModem Receive. You must enter a file name when Sending, but this is optional when Receiving. A terminal emulator program that supports YMODEM may also be used, such as Tera Term or ExtraPuTTY (on Windows) or minicom or picocom (on Linux). However, note that the Propeller 1 can only support YModem transfers, not YModem-1K, and some terminal emulators (e.g. Tera Term and ExtraPuTTY) assume they can always use YModem-1K. When the YMODEM program is started and you are using a terminal emulator, you may see 'C' characters being printed repeatedly in the terminal window - this is normal, and is how the YMODEM send and receive programs synchronize with each other. IMPORTANT NOTE WHEN USING WINDOWS: ================================== Windows recently introduced a new console mode that is very buggy. Programs such as payload that use console mode can fail unexpectedly if the new console mode is used. This requires more investigation, but until Microsoft fixes the issues, it is recommended that the "legacy console mode" be used with payload and other Catalina programs. To ensure this, select Properties from the system menu of any console window and in the Options tab, ensure that the option to "Use legacy console" is selected.EDIT: Updated to match the version included in Catalina 5.5
Deleted - this YMODEM issue has now been fixed in the above post, and also in Catalina 5.5
how about using the sd/host filesystem @ersmith is using in Flex?
It is a nice implementation, it is neat integrated to use from C and Basic, just the Spin support is a bit lacking.
Having that in Catalina/Catalyst would be fantastic.
Mike
I'm not familiar with Flex. Does it support transferring files between the host PC and the Propeller SD Card? That's what I needed, and now that I have yymodem working reliably at 230400 baud, it's easy to use when using Catalyst via a terminal emulator, since many of them already have support for it built in. Catalina's own terminal emulator (payload) currently does not, but it will be easy to add, so I will probably do that for the next release.
Ross.
The FAT filesystem in FlexProp is elm-chan.org/fsw/ff/00index_e.html one, which is pretty widely used in the microcontroller world. For the host PC file system I used a Plan 9 protocol server on the PC side and wrote my own code on the Prop side; it's a very clean protocol and easy to implement (not surprising considering where it came from!). The PC side code is in both loadp2 and my version of proploader, so it works on both P1 and P2. To copy files from host to SD I just run a shell that mounts both the host FS and FAT FS, and then use "copy /host/file.txt /sd/file.txt".
Having said all that, YMODEM is a fine solution for file copies as well, it just serves a slightly different function (although you may well be able to put a file system interface on top of the YMODEM protocol?)
The nice thing about the Plan 9 protocol is that in principle it gives a standard way to talk to any file or device, so we could (theoretically) expose I2C, SPI, and any kind of disk drive (including FAT) via a 9P interface, which would make it easy for code to interoperate between Catalina, FlexProp, Spin2, or whatever. 9P does impose some overhead compared to raw device accesses though, so it isn't widely used in the embedded space.
Hi @ersmith
Sounds useful. Does the Plan 9 system mount the PC disk as a file system on the Propeller, or the Propeller SD Card as a file system on the PC? Or perhaps it can do either? Could it also be used to mount one Propeller SD Card file system on another Propeller?
I did something similar using my proxy SD card driver on the P1 - i.e. you can mount an SD card from one Propeller as the file system on another, using a simple 2-pin serial connection. This was used in the various multi-Propeller systems such Cluso's and Bill Henning's boards. There was a craze for these multi-Propeller P1 systems before the P2 came along, when people were desperate for more pins, more RAM and more speed! But I have not bothered porting it to the P2 yet because there are no multi-Propeller 2 systems yet (AFAIK!). But give it time, and I'm sure some new lunatic will come along ...
The Plan 9 software might be better than mine if it is written in C - mine was written in P1 assembler, so it will need porting and re-testing for the P2. But on the other hand mine also supports other proxy devices such as the mouse, screen and keyboard. I'll look into the Plan 9 stuff when I get some time. For the moment, yymodem solves the problem I needed solved, and already had software support on the PC side. The current version now works at full speed with all the terminal emulators I have tried so far - e.g. minicom, PuTTY, Tera Term.
Ross.
All,
I've updated the post above (here) with a new version of ymodem. The version has various bug fixes and improvements, and has now been now tested and is working on both the Propeller 1 and the Propeller 2.
This version will also be included in the next release of Catalina.
Ross.
All,
YModem support has now been added to the payload loader. This is convenient on Windows, but it is required for Linux - on Windows you can easily use the stand-alone YModem send and receive programs, but Linux toggles DTR (which resets the Propeller) whenever you close and re-open a serial port, so you have to use a terminal emulator that supports YModem so that it does not have do that before the YModem transfer can be started. This version of payload will be included in the next release, but it is not worth doing a release just for this.
I have never found a general solution for this Linux/DTR issue - if anyone knows of one please let me know!
Ross.
Linux normally does this because serial ports were traditionally all connected to modems, for which this is the right thing to do. You can turn it off with
stty -hup -F /dev/whatever, but turning it off causes DTR to transition, resetting the P2, so make sure you run that command as soon as you plug in the device, before you try programming it.It should be possible to write a udev rule to make
stty -hupget called automatically whenever you plug in a serial device.I wouldn't be surprised if some of the more "helpful" GUI terminal emulators turn HUP back on for you, but I just confirmed that the one I usually use,
picocom, leaves the setting alone (unless you explicitly ask it to change it for you).EDIT: It's probably also a good idea to add
-ixon -ixofftostty's options to make the terminal not interpret Ctrl-S and Ctrl-Q as flow control characters.Thanks, but I've tried all the stty and ioctl tricks that are reported by various people to solve this problem and none of them make any difference for me. At least not on my version of Linux (Ubuntu 18). I don't think there is a general solution to this problem. I know some people have resorted to writing their own kernel drivers in desperation, but that's difficult and also version-specific.
Ross.
That's very strange. That's the standard way of doing it - it should work. What USB-serial converter are you using? What terminal emulator have you been using on Linux?
I am just using a Prop Plug. I generally use my own terminal emulator (payload) and stty hup or stty -hup etc makes no difference. But I have just tried picocom and interestingly it can open the port without resetting the Prop. Again, stty hup or stty -hup makes no difference to it, but clearly it is possible. There must be a combination of ioctl options I have not tried which overrides the port settings.
I'll take a look at the picocom source code to see how it does it.
Thanks.
Ha! Comparing the picocom code to the code I use, everything looked fine - except that the code I use contains some ADDITIONAL code specifically designed to avoid toggling DTR! When I take that code out, payload behaves the same way as picocom.
It is possible I added that code in the early days of Catalina when I was using a different flavor of Linux. Or perhaps the Linux serial driver changed at some point. It's not worth going back to find out. I will just add a suitable note, make it configurable, and include the change in the next release.
Ross.
Are you sure you ever actually needed that code to begin with, or is it possible you just put it in because, say, you thought it'd fix a problem that was really somewhere else? Why don't you just tear it out, with no configuration option, until somebody complains, and only then consider putting some configuration option in. Less code means less room for future bugs...
I have wrestled with this problem several times over the years, and so has Linux (evident from looking at various old Linux bug reports about DTR/DSR). I just went back through old versions of Catalina, and it looks like I did indeed write this particular code. So I think it was probably both necessary and worked at the time - but I would probably have been using Red Hat/Fedora at that point, not Debian/Ubuntu.
In any case I have effectively removed it by putting #if ... #endif around it and also putting a note in the source file so it will be obvious in future for anyone who looks that enabling it might offer a solution.
I have just posted Catalina 5.5 to SourceForge here.
This is a full release, primarily to update Catalyst and Payload to add YMODEM support, and also fix the Payload DTR issue on Linux. Plus a few minor things.
Here is the relevant extract of the README.TXT file:
RELEASE 5.5 New Functionality ----------------- 1. A version of the YModem serial file transfer protocol has been added to Catalyst. When Catalyst is built, there will be two more Propeller programs added to the demos\catalyst\bin directory: send.bin - send a file from the Propeller to the host. receive.bin - receive a file on the Propeller sent from the host. Also, in the demos\catalyst\fymodem directory, there will be two corresponding host binaries (on Windows these will be .exe files): send - send a file from the host to the Propeller receive - receive a file on the host sent from the Propeller See the README.TXT file in demos\catalyst\fymodem for more details. 2. A version of the YModem serial protocol has been integrated into payload's interactive mode. This can be used for file transfers even if the Propeller does not use a serial HMI option. 3. Payload now supports configuring the line termination mode from within interactive mode as well as on the command line (i.e. using the -q command line option). 4. Payload now has menus. Pressing the "attention" key (CTRL-A by default) in payload's interactive mode now brings up a menu from which you can select either Terminal Configuration, YModem Transfer, or to Exit from payload (as an alternative to pressing CTRL-D twice). 5. The ex_leds_1.c and ex_leds_2.c example programs in demos\examples now support the P2_EDGE board. 6. Payload now responds correctly to the DEC VT100 Cursor Position Requests (CUP) and responds to a Device Status Report (DSR) specifying parameter 6 with a Cursor Position Report (CPR). Taken together, this allows programs expecting a VT100 compatible terminal to determine the actual terminal window size, rather than simply assuming it will always be 80x24. 7. The Catalyst vi program now uses the DEC VT100 CUP and DSR requests to elicit a CPR, which allows it to determine the actual window size. This means that vi is no longer limited to 80x24 windows when a serial HMI option is used. In particular, when payload is used, the actual window size will now be reported correctly to vi on startup, and vi will use it rather than defaulting to 80x24 whenever a serial HMI is used. The default VT100 window size (80x24) will still be used if VT100 is specified in the build and the terminal emulator cannot respond to CUP and DSR requests. For example, to use vi in a terminal window of 50 lines and 120 columns, and assuming you are using a serial HMI and specified the VT100 option when vi was built, then you might start payload as follows (assuming N is the comms port connected to your Propeller): payload -i -g120,50 -b230400 -pN Then within payload just start vi as normal and it will use the correct window size - e.g: vi catalyst.txt Other Changes ------------- 1. The files s2_serial.h, s4_serial.h and s8_serial.h were not correctly defining the symbols s2_getc, s4_getc and s8_getc respectively. These names were only intended to be used to emulate the original Spin names - the actual underlying C functions (i.e. s2_rxcheck, s4_rxcheck and s8_rxcheck respectively) were all fine. 2. The Catalina Optimizer may have reported undefined symbols when optimizing programs that used the serial2 or serial8 libraries. 3. The header files for the tty, tty256, serial2, serial4 and serial8 libraries now check and issue a message (using #error) if the libraries they represent are not supported on the Propeller type for which the program is being compiled: - The serial2 and serial8 libraries are only supported on the Propeller 2. - The tty, tty256 and serial4 libraries are only supported on the Propeller 1 (but note this has nothing to do with the TTY HMI option, which applies to both the Propeller 1 and Propeller 2 - the TTY HMI option does not use these libraries). 4. The rs232 interface module (rs232.h, rs232.c) used by payload and fymodem included code intended to prevent DTR being toggled when opening a serial port. This code no longer works correctly on Debian Linux and has been disabled. If it is required on other platforms, it can be re-enabled by setting rs232_DTR_FIX to 1 in the file rs232.h. Note that copies of this file exist in both the folder source/catalina as well as the folder demos/catalyst/fymodem folder, and will need to be modified in both places. 5. The help text for payload's -q command line option had the values for enabling CR_TO_LF and LF_TO_CR translation the wrong way around.There is not much new stuff in this release, but I needed to get it out because I won't as have much time for Catalina for the next few months - it is coming into our "busy" season here. Also, I think this may well be the last "monolithic" release. The overheads of doing this are now much too high. I will think about the best way to break it down into more manageable chunks in future - at the very least, I will probably have separate releases for Catalina and Catalyst.
Ross.
I have just posted Catalina 5.5.1 to SourceForge here.
This is a patch release, and there are no changes to Catalina itself. The purpose of the release is to introduce a new Windows-based terminal emulator, and to update payload to use it.
Payload's built-in interactive mode is still available, but I got tired of fighting with Microsoft's console mode, so I decided it was best to stop using it. I dusted off a terminal emulator I developed a while ago - for essentially the same reason - and added a few Propeller-friendly specifics (basically, the ability to control DTR, support for additional baud rates common on the Propeller, and YModem support).
Note that the new payload option allows any terminal emulator to be used. Examples of using it to invoke PuTTY (on Windows) or minicom (on Linux) are included. Also note that the new terminal emulator is completely separate from Catalina, and can also be used stand-alone (it is also available here.
Here is the relevant extract of the README.TXT file:
RELEASE 5.5.1 New Functionality ----------------- 1. The Windows version of Catalina now includes a new serial comms program with a full-function terminal emulator, which can be used as a replacement for payload's very simple interactive mode by specifying "-I terminal" instead of just "-i" on the payload command line - see below for more details). The binary for the program (comms.exe) is in Catalina's bin folder, and the source is in source\comms, but note that the source is not built by default if Catalina is rebuilt, since it requires GNAT (the GNU Ada Translator) and the GNAT Studio development environment to be installed. See the BUILD.TXT file in source\comms for details on how to rebuild the comms program from source. 2. Payload now allows an external terminal emulator program to be executed in place of the simple internal one. The -i option still enables the internal emulator, but a new -I option (i.e. upper case) calls an external terminal emulator program. The -I option accepts one parameter which specifies the command to be used. The port name (not the port number) and baud rate will be passed to this program on startup. For example, if the payload command executed was: payload program.bin -p11 -b230400 -I vt100 Then after loading the program.bin file, the command executed (on Windows) would be: vt100 COM11 230400 On Linux, the command would be something like: vt100 /dev/ttyUSB0 230400 A suitably named script (i.e. on Windows a batch file called vt100.bat, or on Linux a shell script called vt100) can be used to specify any other required parameters. The following Windows scripts are provided (each one is a batch file): pc <-- start comms, specifying PC emulation vt52 <-- start comms, specifying VT52 emulation vt100 <-- start comms, specifying VT100 emulation vt101 <-- start comms, specifying VT101 emulation vt102 <-- start comms, specifying VT102 emulation vt220 <-- start comms, specifying VT220 emulation vt320 <-- start comms, specifying VT320 emulation vt420 <-- start comms, specifying VT420 emulation vt100_putty <-- start PuTTY, specifying VT100 emulation The following Linux scripts are provided (each one is a shell script): ansi <-- start minicom, specifying ANSI emulation vt100 <-- start minicom, specifying VT100 emulation vt102 <-- start minicom, specifying VT102 emulation Note that these scripts can be used independently of payload. They all accept two parameters - the com port to use, and the baud rate. For example: vt100 COM11 230400 vt102 /dev/ttyUSB1 115200 Note that the comms program executable is provided as part of the Windows version of Catalina, but PuTTY and minicom executable will have to be installed separately. Ensure the appropriate executable is in the PATH. Other Changes ------------- 1. Payload now interprets either ESC [ ? 6 n or ESC [ 6 n as a DEC DSR request and responds with a DEC CUP. Strictly speaking, ESC [ ? 6 n is not a correct DEC DSR command, and would not be recognized by other terminal emulators, so the Catalyst xvi program now uses the more correct ESC [ 6 nRoss.
A change to the clock and SD Card plugin code (made in release 5.4) means that the Catalyst Super Star Trek demo now needs to be explicitly built with the -C CLOCK option.
I will fix this in the next release. but in the meantime, just edit the file demos\catalyst\sst\MAKEFILE.CAT and find the following line:
CFLAGS += -lcx -lma -C NO_MOUSEand add -C CLOCK so that it reads as follows:
CFLAGS += -lcx -lma -C NO_MOUSE -C CLOCKThen rebuild.
EDIT: This has been fixed in patch release 5.5.2.
Ross.
Another quick patch - release 5.5.2, available here.
Again, no changes to Catalina itself. Mainly just an update for the vi text editor.
Here is the relevant portion of README.TXT:
RELEASE 5.5.2 New Functionality ----------------- 1. The Catalyst vi editor (xvi) demo program has been updated to version 2.51, which is the latest version and contains some new functionality and some bug fixes. See the documents in demos\catalyst\xvi-2.51\doc for details. 2. The Catalyst vi editor now processes the keypad keys PgUp, PgDown, Home, End, Ins & Del appropriately. 2. The Catalyst vi editor can now be compiled to use colour when the VT100 HMI option is used, since colour is supported by the new external terminal emulator (comms.exe). Just define the Catalina symbol USE_COLOR in addition to VT100, either on the Catalina command line or by adding it as a parameter to the build_all command. For example: cd demos/catalyst/xvi-2.51 build_all P2_EDGE CR_ON_LF VT100 USE_COLOR Note: Only foreground colours can be set within vi itself - the current background colour (typically the background colour in use before vi was started) will be used. The default colours are that text will be grey, the status line will be green (normal files) or red (read-only files). Try the vi command ":help" to see an example. You can manually set the colours (for the text, status line, or read-only status line) in vi as follows: :set colour=n :set statuscolour=n :set roscolour=n where n is: 0 : black 1 : red 2 : green 3 : yellow 4 : blue 5 : magenta 6 : cyan 7 : grey 9 : default foreground colour 10 : bold and bright black (dark grey) 11 : bold and bright red 12 : bold and bright green 13 : bold and bright yellow 14 : bold and bright blue 15 : bold and bright magenta 16 : bold and bright cyan 17 : bold and bright grey (white) Note that when vi is compiled to use colour, the default vi colours will override the current foreground colour on startup. To get it back again, use the vi command ":set colour=9" Any value other than those specified will set the colour to the default foreground colour (typically, the foreground colour before vi was started). If you are using comms.exe then you can also select the default colours using the Format menu (remember to select the Format->Set All To Colours menu item after choosing your new colours, or the changes will apply only to new characters, not existing ones). Note that you can also have other effects, such as inverse video - (i.e. black text on white bg). To do this, start vi and then set the foreground colour to black (i.e. ":set colour=0") - note that any text on the screen may temporarily disappear. Then use the Format->Bg Color menu item to select a white background, then use the Format->Set All to Colour menu item to refresh the colours of the characters already on display. You can choose to have any background colour you want. Other Changes ------------- 1. A change to the clock and SD Card plugin code (made in release 5.4) means that the Catalyst Super Star Trek demo now needs to be explicitly built with the -C CLOCK option. The Makefile has been modified accordingly. 2. When the external terminal emulator's YModem dialog box is open, it is now always on top. Otherwise, it could get hidden by other windows and it was not obvious why the main comms window would not respond. 3. The external terminal emulator's YModem dialog box "Start" button is no longer styled as if it was a default button, since it was not. There is no default button for this dialog box. 4. The new external terminal emulator now locks the screen and view. This makes it easier to set the initial screen size (can now be done just using the /ScreenSize option, instead of also requiring the /ViewSize option). This is more appropriate when emulating a physical terminal, where the screen size would normally be the same as the view size. They can be unlocked again (if required) via the Options->Advanced menu. 5. The scripts that call the external terminal emulator (vt100.bat etc) now add /Wrap to enable wrap mode on startup. This option can also be changed via the Options->Advanced menu. Setting it on by default makes the external terminal emulator behave more like the internal terminal emulator. 6. The payload internal terminal emulator now limits the number of restart retries on ymodem_receive, to avoid locking up permanently in case the other end has not been started. 7. The file P2_EVAL.ZIP has been renamed P2_DEMO.ZIP, and the README_P2.TXT file has been updated to indicate that this demo version of Catalyst should work on the P2 EVAL, P2 EDGE or any other P2 board with a similar SD Card configuration.Just a quick note about Catalyst on the Propeller 2 ...
I discovered recently that the P2 EDGE and P2 EVAL boards (which are the only P2 boards I have) both boot programs much more reliably from FLASH than from the SD Card. Not sure why this is, but to take advantage of it Catalyst can be booted from FLASH just as easily as it can from the SD Card.
Compile Catalyst as usual and copy the results from the demos\catalyst\bin directory to an SD Card as usual, but then set the FLASH dip switch ON (and the P59^ and P59v dip switches OFF ) and use flash_payload to program Catalyst into FLASH. For example, if you are using the serial interface and also want to use the new vt100 terminal emulator, you would use commands like:
Catalyst and all its demo programs should function as usual. Turn the FLASH dip switch OFF to revert to booting from the SD Card.
A note to this effect will be included in the next release.
Ross.
Okay ... I said I wouldn't do it ... I said it was a mistake on the Propeller 1 ... but then I came down with Covid-19 and couldn't do much of anything else for the past week, so ...
I now have Catalina executing C programs using PSRAM as XMM on the Propeller 2. This means Catalina C programs can now be up to 16MB.
I currently only support the PSRAM on the P2 EDGE, but I am using the driver written by @rogloh so while I don't have one of the HyperRAM boards, I believe it would also work on that with just a few tweaks.
The P2 EDGE has 32MB of PSRAM, but there are some internal design decisions from the Propeller 1 days (when anything bigger than 32kb was a big deal) that limit programs to 16MB - but you can use the other 16MB as extra storage and access it from C.
My test suite executes correctly, and I just tried a simple benchmark program. The early results are quite promising:
EXECUTION MODE TIME CODE SIZE (ms) (bytes) ================ ==== ========= NATIVE (HUBEXEC) 705 20584 LMM 1308 23736 CMM (COMPACT) 1551 12612 XMM (PSRAM) 3009 26960So executing the code using XMM mode (i.e. from PSRAM) is about 4 times slower than executing the same code using NATIVE mode (aka P2 HUBEXEC mode), or about 2 times slower than the LMM or COMPACT modes (which are identical to the same modes on the Propeller 1). You can also have different cogs executing the different modes simultaneously, so you can have those parts of the program that need speed executing in NATIVE mode and the rest executing in XMM mode.
There is still a heap more to do and test and tidy up, but I hope to get at least a beta release out before Christmas.
Merry Christmas!
Ross.
Hell froze over! LOL. You've seen the light! Well done.
Merry Xmas.
It's great that you used my driver as other COGs can still share the memory with QoS guarantees, like video drivers, etc - the executed code will slow down a bit if/when they do, but in many cases that's still okay. And as you say there are other memory types that would be supportable. I did HyperRAM and some smallish 2MB SRAM and still plan to do one for (Q)SPI FLASH which could give you some XIP (probably rather slow unless cached).
Yes, the fact that I can use your driver from multiple cogs was a key factor. Eventually I hope to have XMM programs executing in multiple cogs (currently only one cog can be executing in XMM mode, the others have to execute one of the other modes). If that had not been a possibility I probably would not have bothered. But that will have to come in a later release.
So much to do, and so little time!
Well, some bad news and some good news ...
The bad news is that I got held up by a nasty bug (no, not Covid-19 - I mean the programming kind), so I will not have a release of Catalina that executes programs from external RAM ready by Christmas. The bug was because the shift instructions (in this case SHR) behave differently on the Propeller 2 than they did on the Propeller 1. I mean, who on earth would mess about with an instruction as fundamental as that? (no, you don't need to tell me, I know the answer
)
This put me behind schedule by a few days. And Christmas is now approaching fast!
The good news is now that I have fixed that, everything seems to be working fine. I have the largest, most complex program I routinely use (i.e. the Lua language interpreter) now executing from XMM RAM (i.e. PSRAM) on the Propeller 2. This program exercises pretty much everything - memory, file systems, various plugins etc. It means that I can now execute much larger Lua programs than was previously possible (which was in fact the whole point of the exercise).
Of course, just like on the Propeller 1, the performance of programs executing from external RAM on the Propeller 2 will never be anything to write home about. But as the Russian proverb says "The marvel is not that the bear dances well, but that the bear dances at all." But one reason is because I am still using just an 8kb cache to cache access to up to 16Mb of XMM RAM. This relatively tiny cache size is a hang over from the Propeller 1 days, when Hub RAM was so limited. On the Propeller 2 I will be able to make the cache much larger (say 32kb or 64kb) and that should improve the performance. I just haven't had time to do it yet.
Instead of a full release, I may post a demo version of just one program (probably the Lua interpreter), Anyone with a suitable P2 EDGE (you need the one with 32MB of PSRAM, i.e. the P2-EC32MB) will be able to load it and execute it.
Ross.
Post deleted.
The standalone Propeller 2 XMM Demo has been removed from SourceForge since an improved (but incompatible) version of Propeller 2 XMM support is now incorporated in Catalina release 5.6.
Ross.
Some more benchmark data.
I had a suspicion my original benchmark was too optimistic, and I was right. The choice of benchmarks makes a huge difference tp the results. Also, I have now had time to expand the cache size to 64kb, and this improves things, but not as much as I had hoped.
Here are some better benchmark results:
BENCHMARK 1: ============ MODE MEMORY TYPE, SIZE TIME CACHE SIZE (bytes) (ms) ====== ========== ======= ===== NATIVE HUB RAM 6304 998 LMM HUB RAM 6284 1000 CMM HUB RAM 3884 1997 XMM PSRAM,64K 6760 2001 XMM PSRAM,8K 6760 2998 BENCHMARK 2: ============ MODE MEMORY TYPE, SIZE TIME CACHE SIZE (bytes) (ms) ====== ========== ======= ===== NATIVE HUB RAM 8288 34 LMM HUB RAM 8392 166 CMM HUB RAM 4872 236 XMM PSRAM,64K 9136 625 XMM PSRAM,8K 9136 699 BENCHMARK 3: ============ MODE MEMORY TYPE, SIZE TIME CACHE SIZE (bytes) (sec) ====== ========== ======= ===== NATIVE HUB RAM 438176 1 LMM HUB RAM 442912 4 CMM HUB RAM 232736 7 XMM PSRAM,64K 476732 20 XMM PSRAM,8K 476732 23Benchmark 1 is too trivial to be much use - it is a variation of the original benchmark, and it understates the execution time of XMM. Benchmarks 2 and 3 are closer to reality - which is that executing from external RAM can be twenty times slower than executing natively from Hub RAM. I think this can be improved with a better cache - the current one - a hangover from the Propeller 1 - was primarily optimized for size, not speed, and just making it larger has not improved things as much as I had hoped. A redesign looks to be in order.
Still, things are not as bad as all that. Benchmark 3 represents pretty much the largest program that can execute either in Native or LMM mode on the P2. For programs larger than that, it comes down to a choice between either CMM or XMM, and XMM is only three times slower than CMM, but allows programs thirty two times larger.
EDIT: Updated final paragraph, which was overly pessimistic.
A quick post-Christmas update. I've now implemented an XMM SMALL mode in addition to the XMM LARGE mode on the Propeller 2 (the Propeller 1 already had both of these). The good news is that SMALL mode is significantly faster than LARGE mode.
For those unfamiliar with XMM execution (which would probably be many Propeller 2 only users), Catalina offers several different program execution modes:
NMM : Propeller 2 Native (HUBEXEC) mode. Combined Code, Data, Stack and Heap limited to 512kb (i.e. Hub RAM).
LMM : A legacy mode carried over from the Propeller 1 (essentially, the Propeller 1's equivalent to HUBEXEC mode). Similar code sizes and limits to NMM.
CMM : A compact version of LMM. Combined Code, Data, Stack and Heap still limited to 512kb, but Code size is generally about half that of LMM or NMM.
XMM SMALL : Code in external RAM, Data, Stack and Heap in Hub RAM. Maximum code size is 16Mb, maximum combined Data, Stack and Heap is 512kb.
XMM LARGE : Code, Data and Heap in external RAM, Stack in Hub RAM. Maximum combined Code, Data and Heap size is 16Mb, maximum Stack size is 512kb.
I have also made a few other small improvements, so here are the most recent benchmark results, with values adjusted to make them all relative to NMM mode:
LMM is not particularly useful on the Propeller 2 (it was retained mainly because I knew it would provide an easy path to XMM). Once your program won't fit in Hub RAM and so you cannot use NMM any longer (which is obviously still the preferred option) you would first move to CMM and at that point you take a pretty large performance hit. Even so, CMM can still useful (on the Propeller 1, CMM is comparable in speed to Spin). For instance, many of Catalina's own utilities (such as Catalyst, and the Lua interpreter) are compiled in CMM mode. But once you need even more code or data space after that then you now have two further options - SMALL and LARGE mode. The latest addition (SMALL) allows code sizes up to 32 times larger than CMM but at the cost of executing at half the speed. You would move to LARGE mode only once your data size (i.e. not just code size) exceeds the 512kb of available Hub RAM. Also, keep in mind that if some parts of your program absolutely need speed, they can be executing in NMM mode on some cogs while other parts of your program are executing in one or more of the slower modes on other cogs.
I have a few ideas about how to improve the performance of XMM further, but they will require a substantial amount of re-work (which implementing the existing XMM modes did not, since they were already supported on the Propeller 1) and I want to get the new Propeller 2 implementations bedded down properly first.
The bad news is that this is still a little way off - it is so long since I did any XMM work on the Propeller that I had forgotten that most of the complexity is not in the execution itself, it is in all the supporting utilities (program loaders etc) that have to be updated
Ross.
Hi @RossH,
Happy New Year!
I've been following your Prop2 XMM thread and see that you started with the Prop1 XMM capability as the baseline for it.
To some extent you've already covered the following questions but I want to make sure I understand where the Prop2 XMM development stands right now:
The Prop2 XMM code can only be executed from a single cog, correct?
But the remaining cogs can execute code using any of the other non-XMM models (NMM,LMM,CMM)?
Does the Prop2 XMM model use a C structure to share common variables among the other cogs like the Prop1 XMM?
Do you eventually plan on allowing multiple cogs to simultaneously execute Prop2 XMM code?
Do you anticipate any of the techniques used in the Prop2 XMM to backfeed into an enhanced Prop1 XMM? Like, for example, allowing more than one cog to execute XMM code on the Prop1?
(Somewhere I thought I came across a writeup wherein the GCC developers claim to have enabled the Prop1 to execute XMM code on multiple cogs with as little as 2K cache for each cog).
Obviously multi-cog XMM execution on either the Prop1 or Prop2 wouldn't be a speed demon and performance would depend upon what type of external memory is used (among other factors) but it might be an interesting capability nevertheless.
But allowing multi-cog XMM execution seems to be quite the task as the caching cog(s) would have to prioritize and deconflict the access to external memory. A parallel dual port memory arrangement might help in that regard but it appears to be much more difficult attempting that with standard serial RAM.
It would be interesting to see what your solution to that problem would be.
And to you!
Yes. The same as on the Propeller 1. But this is largely a limitation of the current cache plugin, not the XMM plugin.
Yes. The same as on the Propeller 1.
Yes. The same as on the Propeller 1.
Yes. This should be possible on the Propeller 2.
Not likely. On the Propeller 1 both Hub space and cog space are too limited. But with the Propeller, you should never say never!
Not with the cache architecture I am currently using, which was originally the same one GCC used. But I didn't really stay in touch with the GCC progress, because Catalina was always so much more capable and GCC could never do what I needed. But I'll check to see if they later came up with a better cache architecture that the one which I adopted. If so, I could perhaps use it on both the Propeller 1 and the Propeller 2. If you or anyone else has a reference or link, please let me know.
Yes it is definitely not a speed demon! My main purpose in implementing XMM on the Propeller 2 is to enable larger Lua programs, and eventually multi-processing Lua programs. Lua is my favorite language these days, and it is such a rich language that speed is not really a major issue. C and Lua are highly inter-operable - you use C where speed is necessary, and Lua where things are simply too difficult to code in C. Actually, you have just raised a good point - I don't yet have a means of invoking non XMM C from Lua. You can invoke C but it will always execute as XMM C. I should make it possible to invoke NMM C from Lua for when you need full speed. Just another thing to add to the "to do" list!
The PSRAM XMM plugin already allows multiple cog access, so a simplistic solution would be to simply have multiple independent caches all using the same XMM plugin. But this would consume too many cogs to be much use. It may be a valid approach if we ever see the 16 cog version of the Propeller 2, but I have another approach in mind. However, I haven't had time to investigate it yet.
Yes, indeed it would!