I have always found payload to be extremely slow (>30 seconds). None of my laptops have USB3.0 and they all exhibit the problem. I have the latest FTDI drivers.
Hmmm. Maybe it's not USB 3 specific - maybe it's more subtle (USB chipset, perhaps). But it definitely affects only some USB ports and not others.
What has fit to do with it? Besides, fit takes just about anything. It's not currently limited to $1F0 AFAICT. Branch labels OTOH are limited to below $1F0.
RossH:
I looked more carefully at the @ operator change, and I think I agree with you. If I understand correctly, it allows you to use @ on asm local symbols (e.g. @:symbol). I think I'll just include that as a "fix".
The $1FF vs $1F0 change is to allow you to compile stuff that would write into the shadow registers at cog load time, right? I think this probably needs to be enabled by an option, because it could cause issues for someone who doesn't understand how that stuff works in detail.
For the preprocessor, as I understand it in my version parent objects do not "see" #defines from their child objects. The child objects "see" the #defines of their parent and the #defines of other children compiled before them. With your changes, the child objects only "see" the #defines of the parent and *not* the #defines of the child objects compiled before them. My version is more like the way C/C++ preprocessing works, but missing one key part. The parent should "see" the #defines from it's children. That would match how I think it should work, #defines being "global" from the time they are parsed until they are #undef'd or preprocessing ends. Of course, this is treating OBJ stuff like they are #include's which is not really right either. So one could argue that OBJ's should be treated like a separate compile and thus shouldn't see any #defines from it's parent or children. It seems more useful to allow children to see their parent's #defines and parents to see their children's #defines, but children should probably not see their sibling's #defines. That isn't possible without changes to the preprocessor state handling.
I think I'll just keep mine working as it is now, because I think it works like BSTC does (though I'm not 100% certain of that anymore), and add an option to do it like homespun does.
Thoughts?
Roy
Hi Roy. It is always risky to take C/C++ assumptions into the Spin world. Spin has a whole new set of its own peculiarities. My main purpose was more shortsighted - I just needed to get my homespun code to compile with openspin! But in general I think my solution respects the concepts of Spin "objects" a bit more. They should definitely not propagate stuff to their parent unexpectedly.
What has fit to do with it? Besides, fit takes just about anything. It's not currently limited to $1F0 AFAICT. Branch labels OTOH are limited to below $1F0.
The imposed limit doesn't matter. At cog load time you can't preset SPRs anyway (except 0) so I don't really see any advantage here. Do you have an example?
The imposed limit doesn't matter. At cog load time you can't preset SPRs anyway (except 0) so I don't really see any advantage here. Do you have an example?
Or are we talking overlays here?
I thought it was a FIT limitation I ran into. It may have been a src or dst address limitation instead. Whatever the limitation was, homespun allowed it, and the code worked. Openspin didn't allow it, but worked when I changed it.
There are actually a few places in the compiler that limit to the $1F0 range. You changed them all in your version to $1FF.
The checks that exist are to limit the compiler generating code or data (including res) beyond $1F0. Having data or res stuff above $1F0 doesn't seem like it would work really.
FIT defaults to $1F0 when you don't specify a size, else it lets the size be whatever you specify.
Having data or res stuff above $1F0 doesn't seem like it would work really.
Yes it does - if you know what you're doing. I can understand the need to limit it for general usage - but sometimes you need to break the rules to get stuff done.
A Spin compiler should certainly restrict you to what is "safe". But a C compiler? Not so much
Linux 32 and 64 bit versions uploadied now (to here).
As usual, when compiling the Linux versions, I found a couple of minor script and filename capitalization issues, so I have also updated the Windows version (since the Windows version also contains all the Linux scripts and sources).
However, if you have already downloaded the Windows version, don't bother downloading it again - the changes are very minor and will not affect you.
Uninstalled and reinstalled on the work machine and Catalina now compiles and uploads hello_world. Still takes about 55 sec to download to a quickstart.
Uninstalled and reinstalled on the work machine and Catalina now compiles and uploads hello_world. Still takes about 55 sec to download to a quickstart.
Glad you got it working.I had a play with all the machines I had handy, and there is definitely a difference in times between machines - but not between ports on the same machine. And the worst was still only 30s.
The only time I have ever seen times significantly larger than that was when I was downloading via a USB hub - I changed the hub and that seemed to fix it. Are you by any chance using a hub?
Also, it does not seem to be USB 2 vs USB 3 related - some of the machines I used had both, and there was no appreciable difference.
So my best guess is that it is either driver or chipset related. I'll hunt around for some more machines to see if I can find any that show download times of 55s or more.
I think I may have found the"slow download" problem. I found a machine that takes 55 seconds to load a simple "hello world" program (a load that should take only 5 seconds or so). I have attached to this post a test version of payload that seems to fix the problem.
Can anyone who has been seeing download times more than about 30 seconds please execute the same load using the program in the attached zip file, and report the time difference between this program and the normal payload.
I think I may have found the"slow download" problem. I found a machine that takes 55 seconds to load a simple "hello world" program (a load that should take only 5 seconds or so). I have attached to this post a test version of payload that seems to fix the problem.
Can anyone who has been seeing download times more than about 30 seconds please execute the same load using the program in the attached zip file, and report the time difference between this program and the normal payload.
Thanks to those who tested paytest - the problem seems to be with the way buffering is handled in some USB serial drivers. The fixed version makes no difference at all on many machines, but a massive difference on others.
I'll release a patch file suitable for all versions of Catalina 3.11 (i.e. Windows, Linux32, Linux64) later today.
Thanks to those who tested paytest - the problem seems to be with the way buffering is handled in some USB serial drivers. The fixed version makes no difference at all on many machines, but a massive difference on others.
I'll release a patch file suitable for all versions of Catalina 3.11 (i.e. Windows, Linux32, Linux64) later today.
Ross.
Congratulations on finding this. What setting did you have to change?
Congratulations on finding this. What setting did you have to change?
No settings - I just added some buffering. It was apparently only working at full speed on machines where sufficient buffering was being provided by the serial driver. I never spotted it because it seemed to work at full speed on all the machines I tried - even some very old, very slow machines still running Windows XP.
But it seems some serial drivers don't provide much (or perhaps any) internal buffering, and these are the ones that have problems. Maybe this is configurable somewhere, but it no longer matters - payload will now work correctly in this case as well.
I always had the slow payload program problem. Never had trouble with the other loaders.
Will it load a normal spin binary? I could test with that, but it seems to be under control now.
I always had the slow payload program problem. Never had trouble with the other loaders.
Will it load a normal spin binary? I could test with that, but it seems to be under control now.
Thanks Jazzed. Yes, payload will load normal Spin binaries - but there is no need since the problem does indeed seem to be sorted.
I had some previous anecdotal reports of slow loads, but I could never reproduce anything, and no-one had ever highlighted just how bad the problem was - 10 times slower! Yikes!
Until I get the patch out (later today) you can just use paytest in place of payload. Or you can rename the executable as payload.exe and overwrite the existing version in the catalina\bin directory - that way it will work from Code::Blocks as well.
I have attached an "errata" file to the second post in this thread with the new payload sources and binaries. It is also available on SourceForge (here).
See the README.TXT file in the errata file for more details.
No settings - I just added some buffering. It was apparently only working at full speed on machines where sufficient buffering was being provided by the serial driver. I never spotted it because it seemed to work at full speed on all the machines I tried - even some very old, very slow machines still running Windows XP.
But it seems some serial drivers don't provide much (or perhaps any) internal buffering, and these are the ones that have problems. Maybe this is configurable somewhere, but it no longer matters - payload will now work correctly in this case as well.
Ross.
Are you saying you were doing something like this before:
char buf[1];
write(fd, buf, 1);
and now you're doing something like this?
char buf[1024];
write(fd, buf, 1024);
I guess that could make a big difference if every call to write generated a Linux system call.
Essentially, yes. Works fine if the serial driver multitasks properly and buffers up all your writes (which will always occur much faster than any serial hardware can transmit each byte), but really slow if it instead won't respond to each call to send another byte until it has finished sending the previous byte
My fault, I know - but who on earth would design a serial driver with a one byte buffer?
Something of a noob question, a number of objects that I use in Obex are mainly wrappers for pasm routines. Is there a straight forward way of using these objects in Catalina? Are there license issues ( or just common courtesy issues ) with just pulling chunks of code from another's work and plugging into your own source code?
Something of a noob question, a number of objects that I use in Obex are mainly wrappers for pasm routines. Is there a straight forward way of using these objects in Catalina? Are there license issues ( or just common courtesy issues ) with just pulling chunks of code from another's work and plugging into your own source code?
The simple answer is "yes" - but in fact this is quite a complex issue, and has to be dealt with on a case-by-case basis. In the simplest case, it is certainly possible to launch Spin and/or PASM programs "unmodified" from Catalina, or even run them concurrently (provided they adhere to some simple conventions), The complexity comes in if you want to be able to communicate between them.
Spin provides no common mechanism for doing this. Catalina does, but of course most Spin programs don't use it. I wrote up a proposal a while ago for this (see file attached) but it didn't get much traction. I don't see much chance of the Spin community ever agreeing on a single mechanism, which means that there will always be some modification required. With Catalina it is fairly small.
The best place to look for a quick introduction is in the README.TXT file in catalina\demos\spinc\ - this describes how to launch Spin objects from a Catalina program. For just including PASM code chunks, there are other methods, described in the Catalina Reference Manual.
As for licensing - everything in the OBEX is MIT licensed, which means you are free to do what you like. Generally it is considered courteous to include an acknowledgement.
I've just updated the Catalina Reference Manual to fix a few outdated references (e.g. it still referred in many places to Homespun as being the default Spin compiler).
I have included the new version of this document in the Catalina 3.11 Errata file (see the second post in this thread). If you have already downloaded the Errata file, you can also get just the updated document by itself, directly from SourceForge (here).
Comments
Hmmm. Maybe it's not USB 3 specific - maybe it's more subtle (USB chipset, perhaps). But it definitely affects only some USB ports and not others.
Hi Roy. It is always risky to take C/C++ assumptions into the Spin world. Spin has a whole new set of its own peculiarities. My main purpose was more shortsighted - I just needed to get my homespun code to compile with openspin! But in general I think my solution respects the concepts of Spin "objects" a bit more. They should definitely not propagate stuff to their parent unexpectedly.
It is in openspin.
Or are we talking overlays here?
I thought it was a FIT limitation I ran into. It may have been a src or dst address limitation instead. Whatever the limitation was, homespun allowed it, and the code worked. Openspin didn't allow it, but worked when I changed it.
The checks that exist are to limit the compiler generating code or data (including res) beyond $1F0. Having data or res stuff above $1F0 doesn't seem like it would work really.
FIT defaults to $1F0 when you don't specify a size, else it lets the size be whatever you specify.
I may have been a bit over enthusiastic
Yes it does - if you know what you're doing. I can understand the need to limit it for general usage - but sometimes you need to break the rules to get stuff done.
A Spin compiler should certainly restrict you to what is "safe". But a C compiler? Not so much
As usual, when compiling the Linux versions, I found a couple of minor script and filename capitalization issues, so I have also updated the Windows version (since the Windows version also contains all the Linux scripts and sources).
However, if you have already downloaded the Windows version, don't bother downloading it again - the changes are very minor and will not affect you.
Ross.
Glad you got it working.I had a play with all the machines I had handy, and there is definitely a difference in times between machines - but not between ports on the same machine. And the worst was still only 30s.
The only time I have ever seen times significantly larger than that was when I was downloading via a USB hub - I changed the hub and that seemed to fix it. Are you by any chance using a hub?
Also, it does not seem to be USB 2 vs USB 3 related - some of the machines I used had both, and there was no appreciable difference.
So my best guess is that it is either driver or chipset related. I'll hunt around for some more machines to see if I can find any that show download times of 55s or more.
Ross.
I think I may have found the"slow download" problem. I found a machine that takes 55 seconds to load a simple "hello world" program (a load that should take only 5 seconds or so). I have attached to this post a test version of payload that seems to fix the problem.
Can anyone who has been seeing download times more than about 30 seconds please execute the same load using the program in the attached zip file, and report the time difference between this program and the normal payload.
Thanks!
Ross.
payload_test.zip
Ross,
Tested with:
Quickstart Board
Propeller Project Board
Propeller Servo Controller
Each board loaded fine with paytest, about 6 seconds each.
I had about 50% failure loading with payload, "Error Loading Program", but when it did load, it was about 1 minute as experienced by others.
BTW payload.exe was from a fresh install of 3.11.
Jim
paytest 6 sec
payload 55 sec
I'll release a patch file suitable for all versions of Catalina 3.11 (i.e. Windows, Linux32, Linux64) later today.
Ross.
Forgot to mention my platform:
Windows XP with USB 1.0 , (old machine).
Will test on my Windows 7 Pro AMD 64 with USB 2.0 tomorrow. (or is it today there)
No settings - I just added some buffering. It was apparently only working at full speed on machines where sufficient buffering was being provided by the serial driver. I never spotted it because it seemed to work at full speed on all the machines I tried - even some very old, very slow machines still running Windows XP.
But it seems some serial drivers don't provide much (or perhaps any) internal buffering, and these are the ones that have problems. Maybe this is configurable somewhere, but it no longer matters - payload will now work correctly in this case as well.
Ross.
Will it load a normal spin binary? I could test with that, but it seems to be under control now.
QuickStart:
4 loads payload.exe Successful 55-56 seconds
4 loads paytest.exe Successful 5-6 seconds
Propeller Project Board:
4 loads payload.exe Successful 55-56 seconds
4 loads paytest.exe Successful 5-6 seconds
Propeller Servo Board:
4 loads payload.exe Successful 55-56 seconds
4 loads paytest.exe Successful 5-6 seconds
paytest is a winner!
Congrats Ross.
EDIT All tests were with USB 2.0 ports on this machine.
Jim
Thanks Jazzed. Yes, payload will load normal Spin binaries - but there is no need since the problem does indeed seem to be sorted.
I had some previous anecdotal reports of slow loads, but I could never reproduce anything, and no-one had ever highlighted just how bad the problem was - 10 times slower! Yikes!
Ross.
Until I get the patch out (later today) you can just use paytest in place of payload. Or you can rename the executable as payload.exe and overwrite the existing version in the catalina\bin directory - that way it will work from Code::Blocks as well.
Ross.
A very small binary is now 2-3 seconds whereas the same with payload is ~19 seconds.
All,
I have attached an "errata" file to the second post in this thread with the new payload sources and binaries. It is also available on SourceForge (here).
See the README.TXT file in the errata file for more details.
Ross.
and now you're doing something like this?
I guess that could make a big difference if every call to write generated a Linux system call.
Essentially, yes. Works fine if the serial driver multitasks properly and buffers up all your writes (which will always occur much faster than any serial hardware can transmit each byte), but really slow if it instead won't respond to each call to send another byte until it has finished sending the previous byte
My fault, I know - but who on earth would design a serial driver with a one byte buffer?
Ross.
The simple answer is "yes" - but in fact this is quite a complex issue, and has to be dealt with on a case-by-case basis. In the simplest case, it is certainly possible to launch Spin and/or PASM programs "unmodified" from Catalina, or even run them concurrently (provided they adhere to some simple conventions), The complexity comes in if you want to be able to communicate between them.
Spin provides no common mechanism for doing this. Catalina does, but of course most Spin programs don't use it. I wrote up a proposal a while ago for this (see file attached) but it didn't get much traction. I don't see much chance of the Spin community ever agreeing on a single mechanism, which means that there will always be some modification required. With Catalina it is fairly small.
The best place to look for a quick introduction is in the README.TXT file in catalina\demos\spinc\ - this describes how to launch Spin objects from a Catalina program. For just including PASM code chunks, there are other methods, described in the Catalina Reference Manual.
As for licensing - everything in the OBEX is MIT licensed, which means you are free to do what you like. Generally it is considered courteous to include an acknowledgement.
Ross.
I've just updated the Catalina Reference Manual to fix a few outdated references (e.g. it still referred in many places to Homespun as being the default Spin compiler).
I have included the new version of this document in the Catalina 3.11 Errata file (see the second post in this thread). If you have already downloaded the Errata file, you can also get just the updated document by itself, directly from SourceForge (here).
Ross.