@Wuerfel_21 said:
On Windows NT, especially if you have any sort of security software enabled, spawning a new process and attaching it to the console subsystem is comically slow. Reading directory information is also slow. This can combine into a make+gcc style build taking orders of magnitude longer than reasonable.
Yes, disabling as much Windows security as possible does indeed speed things up - but not enough. Also, there are good reasons why you absolutely want as much security enabled in Windows as you can get (and then go add some more!) so it is not something users should have to do.
Using MinGW in GH Actions is perfectly fine (assuming use as a cross-compiler on a Linux action runner), I set it up for flexspin/spin2cpp.
I may be missing something here. Why install MinGW on Linux and cross-compile? Why not just use a Windows action runner? Because it would be too slow?
The real big thing with actions IMO is that you can use it to auto-run any test scripts you have. For flexspin I made a modified version of spinsim, so P1 code can be tested end-to-end.
Yes, I can see that would be useful. It would have saved me from a few embarrassing releases
I know there was a P2 version of spinsim - do you use that?
@Wuerfel_21 said:
On Windows NT, especially if you have any sort of security software enabled, spawning a new process and attaching it to the console subsystem is comically slow. Reading directory information is also slow. This can combine into a make+gcc style build taking orders of magnitude longer than reasonable.
Yes, disabling as much Windows security as possible does indeed speed things up - but not enough. Also, there are good reasons why you absolutely want as much security enabled in Windows as you can get (and then go add some more!) so it is not something users should have to do.
Installing security software isn't something anyone should ever have to do. The OS should already be inherently secure. If it isn't, it isn't a serious OS.
Using MinGW in GH Actions is perfectly fine (assuming use as a cross-compiler on a Linux action runner), I set it up for flexspin/spin2cpp.
I may be missing something here. Why install MinGW on Linux and cross-compile? Why not just use a Windows action runner? Because it would be too slow?
Doing compilation (or, frankly, anything more complicated than word processing) on Windows just doesn't make sense. Every popular OS besides Windows - Linux, BSD, even Mac OS, is better, faster, more secure, easier to write correct programs for, and more consistent with other OSes. Official builds of PuTTY, a program which decidedly only makes sense on Windows because Linux (and newer versions of Windows) already comes with ssh, telnet, and nice serial port programs, are cross-compiled for Windows by a Linux machine (and there's a Linux build for testing purposes) because it's so much simpler than bothering with building something inside Windows.
@Electrodude said:
Doing compilation (or, frankly, anything more complicated than word processing) on Windows just doesn't make sense. Every popular OS besides Windows - Linux, BSD, even Mac OS, is better, faster, more secure, easier to write correct programs for, and more consistent with other OSes. Official builds of PuTTY, a program which decidedly only makes sense on Windows because Linux (and newer versions of Windows) already comes with ssh, telnet, and nice serial port programs, are cross-compiled for Windows by a Linux machine (and there's a Linux build for testing purposes) because it's so much simpler than bothering with building something inside Windows.
I get where you're coming from, but OS wars are a bit like language wars and editor wars - neither side is ever likely to convince the other. I routinely use both Windows and Linux so I don't really have a "side" on this one.
I think a more important question when it comes to developing free open-source software is whether you can develop it using free open-source software. I can still do that on both Windows and Linux, but I dropped MacOS when this became impossible. I don't get paid for developing my software, so I don't see why I should have to pay to develop it.
As long as it remains possible to do this on Windows, I will probably continue to do so. Painful and slow as it sometimes is.
@Wuerfel_21 said:
On Windows NT, especially if you have any sort of security software enabled, spawning a new process and attaching it to the console subsystem is comically slow. Reading directory information is also slow. This can combine into a make+gcc style build taking orders of magnitude longer than reasonable.
Yes, disabling as much Windows security as possible does indeed speed things up - but not enough. Also, there are good reasons why you absolutely want as much security enabled in Windows as you can get (and then go add some more!) so it is not something users should have to do.
Using MinGW in GH Actions is perfectly fine (assuming use as a cross-compiler on a Linux action runner), I set it up for flexspin/spin2cpp.
I may be missing something here. Why install MinGW on Linux and cross-compile? Why not just use a Windows action runner? Because it would be too slow?
Because it's just easier to deal with. The makefile was already set-up to allow for cross-compilation and it's easy to grab MinGW, Bison, etc from the ubuntu package manager. Also I think windows runners used to not exist / cost extra?
Incidentally, for linux binary builds, I set it up to use musl-gcc to link the entire C library in to avoid dependency conflicts. It doesn't seem to increase the file size noticeably.
The real big thing with actions IMO is that you can use it to auto-run any test scripts you have. For flexspin I made a modified version of spinsim, so P1 code can be tested end-to-end.
Yes, I can see that would be useful. It would have saved me from a few embarrassing releases
I know there was a P2 version of spinsim - do you use that?
There's some P2-themed code in there, but I remember I couldn't get it to work. Possibly based on some FPGA prototype version...
This is just a bit of advice for Catalina users ...
I've been doing some more investigation into memory issues, because despite many recent Lua improvements there is still an outstanding issue that can affect both C and Lua programs.
The issue occurs when the heap (which grows up from low memory) and the stack (which grows down from high memory) are both allocated in Hub RAM and collide. There is no simple fix for this issue. It cannot be solved just by having the dynamic memory allocation functions (i.e. malloc() and friends) check the stack before allocating from the heap. It also requires that the heap be checked before every operation that allocates from the stack - e.g. on every C function call. This would be horribly expensive, but I will consider adding it to a future release as a debugging aid.
This issue only affects programs where the heap and stack are both in Hub RAM (i.e. CMM, NMM, LMM and XMM SMALL programs, but not XMM LARGE programs). Lua programs are more prone to suffer from it because they typically do more dynamic memory allocation than C programs, and recent Catalina releases - which allow larger and more complex Lua programs to execute from Hub RAM - have made it more likely to occur.
Hoist with my own petard!
The best option currently available is to minimize the need for programs to allocate more heap space just because they do not have a single contiguous block of suitable size by de-fragmenting the heap.
In C, there is a library function to do this. Add a line like the following to the C program:
malloc_defragment(); // defragment the C heap
In Lua (as documented in Lua on the Propeller 2 with Catalina ) the sbrk() function can be used. Add a line like the following to the Lua program:
propeller.sbrk(true) -- defragment the C heap
In many cases just adding one line resolves the issue completely. The best place to add such a line is just before executing code that does large amounts of dynamic memory allocation (e.g. in Lua, lots of string operations). If it is not easy to identify a specific place in the program, the next best option would be to do it periodically.
I've (re)discovered Catalina, and recently downloaded 8.1.something. And it's all doing what it should, and fun is being had. But..... I'm having difficulties with payload. It wants DLLs, which don't appear to be in the download, DLLs which I don't have, and although googling has found me things of about the right name, they don't result in a working payload.
Is there a definitive place where I can get the right DLLs?
For giggles, trawling through backups got me Catalina from a retired PC, and payload version 3.3 works, but strangely, it seems to lack features...
DLL names? Well, I think , from error messages, it is libintl-8.dll and libiconv-2.dll, versions of which I've found, and there are also versions without the hyphen, eg libiconv2.dll. I'm hoping for a definitive answer as to whats required. Delving into the source tree hasn't provided the illumination I sought.
I've (re)discovered Catalina, and recently downloaded 8.1.something. And it's all doing what it should, and fun is being had. But..... I'm having difficulties with payload. It wants DLLs, which don't appear to be in the download, DLLs which I don't have, and although googling has found me things of about the right name, they don't result in a working payload.
Is there a definitive place where I can get the right DLLs?
For giggles, trawling through backups got me Catalina from a retired PC, and payload version 3.3 works, but strangely, it seems to lack features...
Thanks,. David.
Hi David
If you downloaded from SourceForge, the DLLs should be installed by the Windows installer.
If you downloaded from GitHub, the DLLs should be in the GitHub binary assets - perhaps in a previous release if there have been no binary changes. For instance, for 8.1.1 or 8.1.2, use the binary assets for 8.1.1 here and here. You need both.
And yes, payload 3.3 is very old and will be missing lots of features! It is probably one of those features that requires the additional DLLs.
Ross.
Edited: You need both the Windows binary asset and the Windows Geany binary asset to get all the necessary DLLs.
@"David Buckley" said:
DLL names? Well, I think , from error messages, it is libintl-8.dll and libiconv-2.dll, versions of which I've found, and there are also versions without the hyphen, eg libiconv2.dll. I'm hoping for a definitive answer as to whats required. Delving into the source tree hasn't provided the illumination I sought.
Ah yes - it seems you now also need both libintl-8.dll and libiconv-2.dll in your PATH. I have so many things in my PATH variable (including paths to other versions of these DLLs) that I'd not noticed that omission before - thanks.
These files exist in "%LCCDIR%"\catalina_geany\bin, so to do this either copy those two files from "%LCCDIR%"\catalina_geany\bin to "%LCCDIR%"\bin or add "%LCCDIR%"\catalina_geany\bin to your path.
For example, execute the following in a Catalina Command Line window:
I thought I would have had time by now to do a new Catalina release, but I have not. So just to fix the issue about missing DLLs being reported by payload, I have updated the current Windows binary releases of Catalina 8.1.2 (the Linux releases do not need DLLs).
On SourceForge I have updated the Windows installer for 8.1.2 to include the missing DLLs. Or you can just use the copy commands given here) to do this on an existing installation of 8.1.2 - that's the only change.
On GitHub I have added new binary assets for Windows to release 8.1.2. Previously, this release said to use the assets from 8.1.1, but those are missing the DLLs.
Comments
Yes, disabling as much Windows security as possible does indeed speed things up - but not enough. Also, there are good reasons why you absolutely want as much security enabled in Windows as you can get (and then go add some more!) so it is not something users should have to do.
I may be missing something here. Why install MinGW on Linux and cross-compile? Why not just use a Windows action runner? Because it would be too slow?
Yes, I can see that would be useful. It would have saved me from a few embarrassing releases
I know there was a P2 version of spinsim - do you use that?
Ross.
Installing security software isn't something anyone should ever have to do. The OS should already be inherently secure. If it isn't, it isn't a serious OS.
Doing compilation (or, frankly, anything more complicated than word processing) on Windows just doesn't make sense. Every popular OS besides Windows - Linux, BSD, even Mac OS, is better, faster, more secure, easier to write correct programs for, and more consistent with other OSes. Official builds of PuTTY, a program which decidedly only makes sense on Windows because Linux (and newer versions of Windows) already comes with
ssh,telnet, and nice serial port programs, are cross-compiled for Windows by a Linux machine (and there's a Linux build for testing purposes) because it's so much simpler than bothering with building something inside Windows.I get where you're coming from, but OS wars are a bit like language wars and editor wars - neither side is ever likely to convince the other. I routinely use both Windows and Linux so I don't really have a "side" on this one.
I think a more important question when it comes to developing free open-source software is whether you can develop it using free open-source software. I can still do that on both Windows and Linux, but I dropped MacOS when this became impossible. I don't get paid for developing my software, so I don't see why I should have to pay to develop it.
As long as it remains possible to do this on Windows, I will probably continue to do so. Painful and slow as it sometimes is.
Ross.
Because it's just easier to deal with. The makefile was already set-up to allow for cross-compilation and it's easy to grab MinGW, Bison, etc from the ubuntu package manager. Also I think windows runners used to not exist / cost extra?
Incidentally, for linux binary builds, I set it up to use musl-gcc to link the entire C library in to avoid dependency conflicts. It doesn't seem to increase the file size noticeably.
There's some P2-themed code in there, but I remember I couldn't get it to work. Possibly based on some FPGA prototype version...
This is just a bit of advice for Catalina users ...
I've been doing some more investigation into memory issues, because despite many recent Lua improvements there is still an outstanding issue that can affect both C and Lua programs.
The issue occurs when the heap (which grows up from low memory) and the stack (which grows down from high memory) are both allocated in Hub RAM and collide. There is no simple fix for this issue. It cannot be solved just by having the dynamic memory allocation functions (i.e. malloc() and friends) check the stack before allocating from the heap. It also requires that the heap be checked before every operation that allocates from the stack - e.g. on every C function call. This would be horribly expensive, but I will consider adding it to a future release as a debugging aid.
This issue only affects programs where the heap and stack are both in Hub RAM (i.e. CMM, NMM, LMM and XMM SMALL programs, but not XMM LARGE programs). Lua programs are more prone to suffer from it because they typically do more dynamic memory allocation than C programs, and recent Catalina releases - which allow larger and more complex Lua programs to execute from Hub RAM - have made it more likely to occur.
Hoist with my own petard!
The best option currently available is to minimize the need for programs to allocate more heap space just because they do not have a single contiguous block of suitable size by de-fragmenting the heap.
In C, there is a library function to do this. Add a line like the following to the C program:
malloc_defragment(); // defragment the C heapIn Lua (as documented in Lua on the Propeller 2 with Catalina ) the sbrk() function can be used. Add a line like the following to the Lua program:
propeller.sbrk(true) -- defragment the C heapIn many cases just adding one line resolves the issue completely. The best place to add such a line is just before executing code that does large amounts of dynamic memory allocation (e.g. in Lua, lots of string operations). If it is not easy to identify a specific place in the program, the next best option would be to do it periodically.
Ross.
Help!
I've (re)discovered Catalina, and recently downloaded 8.1.something. And it's all doing what it should, and fun is being had. But..... I'm having difficulties with payload. It wants DLLs, which don't appear to be in the download, DLLs which I don't have, and although googling has found me things of about the right name, they don't result in a working payload.
Is there a definitive place where I can get the right DLLs?
For giggles, trawling through backups got me Catalina from a retired PC, and payload version 3.3 works, but strangely, it seems to lack features...
Thanks,. David.
The DLL names?
DLL names? Well, I think , from error messages, it is libintl-8.dll and libiconv-2.dll, versions of which I've found, and there are also versions without the hyphen, eg libiconv2.dll. I'm hoping for a definitive answer as to whats required. Delving into the source tree hasn't provided the illumination I sought.
Hi David
If you downloaded from SourceForge, the DLLs should be installed by the Windows installer.
If you downloaded from GitHub, the DLLs should be in the GitHub binary assets - perhaps in a previous release if there have been no binary changes. For instance, for 8.1.1 or 8.1.2, use the binary assets for 8.1.1 here and here. You need both.
And yes, payload 3.3 is very old and will be missing lots of features! It is probably one of those features that requires the additional DLLs.
Ross.
Edited: You need both the Windows binary asset and the Windows Geany binary asset to get all the necessary DLLs.
Ah yes - it seems you now also need both libintl-8.dll and libiconv-2.dll in your PATH. I have so many things in my PATH variable (including paths to other versions of these DLLs) that I'd not noticed that omission before - thanks.
These files exist in "%LCCDIR%"\catalina_geany\bin, so to do this either copy those two files from "%LCCDIR%"\catalina_geany\bin to "%LCCDIR%"\bin or add "%LCCDIR%"\catalina_geany\bin to your path.
For example, execute the following in a Catalina Command Line window:
Ross.
And with those two copy commands, all is well and working.
Thank you Ross!
No worries. I'll fix this properly in the next release.
I thought I would have had time by now to do a new Catalina release, but I have not. So just to fix the issue about missing DLLs being reported by payload, I have updated the current Windows binary releases of Catalina 8.1.2 (the Linux releases do not need DLLs).
On SourceForge I have updated the Windows installer for 8.1.2 to include the missing DLLs. Or you can just use the copy commands given here) to do this on an existing installation of 8.1.2 - that's the only change.
On GitHub I have added new binary assets for Windows to release 8.1.2. Previously, this release said to use the assets from 8.1.1, but those are missing the DLLs.
Ross.