I guess the only reason you would choose Python was if you wanted to use the same code on the P2 under micropython. I would expect that you would not ever be able to use C to target the P2 for the compiler as it would use up the entire memory with C, if indeed it would fit at all.
Chip,
OpenSpin has 12,002 lines of code, 1899 lines of comments, and 1399 blank lines.
Also, it's C++, but it's C++ lite, as in I don't use that much of C++ features in it.
You can look at all the source files here: https://github.com/parallaxinc/OpenSpin
Most of the code is in the PropellerCompiler folder. The SpinSource folder is the wrapper code that does the file i/o and calls into the compiler.
I would consider Micropython itself to be a reasonably complex C application when you look at its source, yet it only takes up in the order of half of the P2 memory. So a first estimate it seems reasonable to assume the executable components of a SPIN2 compiler could probably fit too if coded efficiently. You sort of need to write something to see how big it gets, but C can generate pretty tight code if done well. The concern may be if the data structures get too large however even then it might be possible to make it work on the P2 if coded in a certain way ahead of time that is amenable to using external memory storage for larger data structures using something like HyperRAM or SPI RAM or even an SD card (not ideal). I am assuming the executable itself still fits in the 512k (still a big unknown), and if it didn't it could then need overlay loaders etc.
For example, let's say you are keeping a large amount of symbol table data to maintain symbols with long names and other properties etc and you need to randomly access them, you could try to abstract things in C to work on individual elements at a time and read/write back to memory when they are modified. You could even try to temporarily cache these too in the software where it makes sense to avoid repeated accesses where unnecessary.
The getSymbol and updateSymbol functions on a PC implementation could just access the symbol table data from an array directly from the global memory space, but on a P2 this function could be made to read/write a element stored in external memory holding the much larger array. For a P2 implementation you'd need a small wrapper layer for the different data structures that knows where they are allocated in external memory. It could work this way but may run a bit slower unless the tool is aware of the access overhead and can streamline its accesses to make best use of the read/write burst capabilities of the external memory.
If the full executable didn't fit then some overlay/chaining of code may work too reading out of external RAM but it is much harder to setup and would likely need some serious GCC linker script hacking to setup the overlay code at some common addresses etc, which is a very big ask. Probably doable but very difficult.
Even if the resource use becomes problematic to actually run the SPIN2 compiler on the same system as the target it could still be good to have the ability to use such a compiler tool on a second P2 board, potentially hosting its own IDE and debugger to attach a control a second target P2, serially. Then you can be in full control of your own destiny and not need to rely on other systems to build P2 software.
Update: of course all the above assumes a suitable filesystem layer is present on the P2 that works with the C code generated by P2gcc/flexC. Not sure if we are 100% there yet.
Nobody seems to be in any particular hurry to get python 3 to create .exe programs however.
If you want to run .py scripts, you need python installed.
For what it's worth, there are options to generate .exe and remove dependence on the end user having to have Python (or specific versions/dependencies) installed. I've done this with PyInstaller.
Beeware > Briefcase project is interesting but is in it's early beta days.
Maybe some interesting examples of python applications, though rather complex
* Adafruit's Mu
* PlatformIO (which plugs into VS Code)
I have had very good experiences with the TCC C compiler. It could hardly be simpler. You don't need a Makefile, no linker, you just open a CMD window and type:
tcc mysource.c
and you get a mysource.exe as executable in the same directory.
I have written a Compiler and an Assembler for my own FPGA processor. It was also no problem to transfer the whole thing later to a RasPi.
Of course you should not write GUI applications with it, but for simple console applications I find TCC ingenious.
So and now I am curious how horrified the C/C++ gurus will be about this suggestion.
Question: If I want to write C code for the PC, what compiler/debugger should I use? I assume that these days, there are some free options? Is any compiler worth spending money on?
Visual Studio 2017 is free for more than professional needs for c/c++/basic/.net/android/....
How do 32 bit systems address more than 4GB of RAM? Do they use some kind of paging?
They can't. Your app relays on the OS and windows 32bit can't address more than 4GB. (To be true it even can't address the full 4GB)
But you can compile for 32 or 64 bit system
I guess the only reason you would choose Python was if you wanted to use the same code on the P2 under micropython. I would expect that you would not ever be able to use C to target the P2 for the compiler as it would use up the entire memory with C, if indeed it would fit at all.
Huh? MicroPython takes up almost all of the P2 memory. I doubt there is any way that a Spin compiler written in Python would ever fit on the P2. On the other hand, C doesn't take up any space at all other than what your own code takes and the libraries that you use all of which are necessary to run the application. There is no interpreter. If Chip is careful in how he manages memory in his compiler I don't see why it couldn't run on the P2 compiled with Eric's FastSpin C compiler.
Yes, and the code compiled with the 32 bit tool version should still run on older non-64 bit machines.
Is there much need for being 32-bit compatible these days, or is everything 64-bit now?
I think Evanh and TonyB_ have older machines for curious reasons.
My PC has a 32-bit CPU and it's quick because the Windows swap file is on a RAM drive, which could hold all of the Windows system files if I really wanted. I can use Eric's FastSpin without any problem and I don't see the necessity for switching from 386 assembler to 64-bit C instead of 32-bit C for PNut, if that's what we are discussing here.
I don't use PNUT because it doesn't support include files, I think, therefore adding #include to PNUT would be very useful. I've written a program that scans a PASM2 source file and writes skip patterns to a separate file for including under CON. A 2500+ line source takes about one second to process on my 1200MHz PC. Creating skip patterns manually is crazy.
What? Doesn't that defeat the purpose of having a swapfile? Wouldn't it be faster if you just completely disabled the swapfile altogether? Or is this because it's a 32-bit CPU with more than 4GB RAM?
It works the other way around on Linux: if RAM drives and the rest of the system are competing for memory, the swapper can swap parts of RAM drives to disk just as if it were any other piece of RAM.
I don't understand. Why would you want to run Chip's tools under a C interpreter? Why not compile them to native code with a C compiler that targets the ESP32?
I have had very good experiences with the TCC C compiler.
So and now I am curious how horrified the C/C++ gurus will be about this suggestion.
Ariba, I was thinking the exact same thing last night and figured there would be a lot of comments about how its not OSS, not advanced enough, etc, etc.
Chip should use it, and then for giggles someone here could compile it on GCC and see if there is any significant difference.
Would seem to be a far better use of Chip's time.
What? Doesn't that defeat the purpose of having a swapfile? Wouldn't it be faster if you just completely disabled the swapfile altogether? Or is this because it's a 32-bit CPU with more than 4GB RAM?
It works the other way around on Linux: if RAM drives and the rest of the system are competing for memory, the swapper can swap parts of RAM drives to disk just as if it were any other piece of RAM.
Yes swap file is normally part of windows. If anything Linux copied the idea. Older 9x versions are very aggressive about pushing less used RAM chunks to disk because that was the need back then. The typical 8 or 16 megabytes system back then is tiny!
I don't understand. Why would you want to run Chip's tools under a C interpreter? Why not compile them to native code with a C compiler that targets the ESP32?
I was more thinking about the "how" rather than actual use of that C interpreter... I liked the web-based interface to the ESP32, its editor, & the ability to just upload to its file system, etc... If there was an ESP32-based compiler for P2, rather than a C interpreter, you would upload or edit your source via the web, then compile and load the P2 from the ESP32.
I don't understand. Why would you want to run Chip's tools under a C interpreter? Why not compile them to native code with a C compiler that targets the ESP32?
I was more thinking about the "how" rather than actual use of that C interpreter... I liked the web-based interface to the ESP32, its editor, & the ability to just upload to its file system, etc... If there was an ESP32-based compiler for P2, rather than a C interpreter, you would upload or edit your source via the web, then compile and load the P2 from the ESP32.
You're talking about an ESP32-based Spin2 compiler and IDE? I'm sure that would be interesting to some. I doubt I'd use it because I don't have a problem with using a PC to develop code. I don't understand the obsession with self hosting. I do see a use for an interactive language on the target platform but we already have that in ROM with Taqoz. MicroPython or Javascript might be interesting as well. Being able to develop C or Spin2 code on an ESP32 doesn't sound that compelling to me. I'm sure it does to others though.
Self-hosting would be good for the education market...
I think Chip said that...
Okay but what language do you want to provide? I assume it would be Spin2? Or do you want C? That might be more difficult unless FastSpin will fit on the ESP32. I'm virtually certain that GCC won't. Maybe Catalina would be an option as well. What ever happened to RossH?
this whole thread seems backwards....now were are at c++ i could understand this if linux was a possibility...but even its at a 64 bit level these days
so why c++
now gcc...im not throwing that on anything but a virtual box!
spin is more like python anyway
if ur teaching kids: big buttons...less options
if ur self hosting to push the envelope with advanced level programming: write a multi-pass compiler
if ur trying to get spin2 going: its done it seems
HEY TEACHERS....WE WILL HAVE DIGITAL CALCULATORS IN 20 YEARS...JUST NOT THE INTERNET
Lua was designed from the ground up to be extremely lightweight and self-hosting. It manages to do this on ESP8266 with room left over for actual applications. The table structures are highly optimized and can be used to form namespaces and rudimentary object-oriented structures if you like, or not. It is extremely easy to wrap C function libraries into the Lua scripting system when compiling a firmware image. For development it's really cool to be able to open a serial terminal on a system that's running, because everything is event driven with callbacks, and type print(var) or var = new_value in real time.
Anyone who likes programming in assembly language is going to hate everything about C++. I really wish the NodeMCU people would get their stuff together and update the custom firmware build page to support the ESP32. I'm currently working with the Arduino IDE because it's a lot easier to get up and running than the Expressif SDK, but I had to run a search program just to find where the libraries were hidden on my hard drive so I could try to figure out how they worked. Hiding code and data like this is one of the supposed "virtues" of object-oriented programming which I've always thought was batshit crazy. Sure having a data type named IP_addr makes for real pretty and clean-looking code, until you need to convert four actual integers read from a file into one of those IP_addr structs and there is no guidance anywhere in the documentation on how to do that.
I'm working on something like that. It's been slow going though. With this COVID-19 crisis I've been working even more hours than before. I'm jealous of the people who are on forced vacation.
David Betz,
I am working full time too. Sometimes wish I had all the time off, but then I'd not get paid (like many that are on "forced vacation" right now).
David Betz,
I am working full time too. Sometimes wish I had all the time off, but then I'd not get paid (like many that are on "forced vacation" right now).
My brother and his wife are on forced "vacation" with full pay. That probably won't last long though. As you say, it's good to have some money coming in. What I'm doing is interesting as well.
Don't get too excited. This work is derived from various other BASIC interpreters that I wrote for P1. I'd like to try to get it to generate PASM instead of byte code this time though. I don't have any illusions that people will actually use this thing. It's just a fun project at present. I'll try to make it easy to add functions to interface to hardware though so it would be possible to use it in some of the same ways that Femto BASIC was used. However, Taqoz is much better for that. Also, as has been mentioned elsewhere, the P2 has enough RAM that more standard languages may be possible like MicroPython, Javascript, or Lua.
Comments
Roy, how many lines of C source does OpenSpin have?
I guess the only reason you would choose Python was if you wanted to use the same code on the P2 under micropython. I would expect that you would not ever be able to use C to target the P2 for the compiler as it would use up the entire memory with C, if indeed it would fit at all.
OpenSpin has 12,002 lines of code, 1899 lines of comments, and 1399 blank lines.
Also, it's C++, but it's C++ lite, as in I don't use that much of C++ features in it.
You can look at all the source files here: https://github.com/parallaxinc/OpenSpin
Most of the code is in the PropellerCompiler folder. The SpinSource folder is the wrapper code that does the file i/o and calls into the compiler.
I would consider Micropython itself to be a reasonably complex C application when you look at its source, yet it only takes up in the order of half of the P2 memory. So a first estimate it seems reasonable to assume the executable components of a SPIN2 compiler could probably fit too if coded efficiently. You sort of need to write something to see how big it gets, but C can generate pretty tight code if done well. The concern may be if the data structures get too large however even then it might be possible to make it work on the P2 if coded in a certain way ahead of time that is amenable to using external memory storage for larger data structures using something like HyperRAM or SPI RAM or even an SD card (not ideal). I am assuming the executable itself still fits in the 512k (still a big unknown), and if it didn't it could then need overlay loaders etc.
For example, let's say you are keeping a large amount of symbol table data to maintain symbols with long names and other properties etc and you need to randomly access them, you could try to abstract things in C to work on individual elements at a time and read/write back to memory when they are modified. You could even try to temporarily cache these too in the software where it makes sense to avoid repeated accesses where unnecessary.
The getSymbol and updateSymbol functions on a PC implementation could just access the symbol table data from an array directly from the global memory space, but on a P2 this function could be made to read/write a element stored in external memory holding the much larger array. For a P2 implementation you'd need a small wrapper layer for the different data structures that knows where they are allocated in external memory. It could work this way but may run a bit slower unless the tool is aware of the access overhead and can streamline its accesses to make best use of the read/write burst capabilities of the external memory.
If the full executable didn't fit then some overlay/chaining of code may work too reading out of external RAM but it is much harder to setup and would likely need some serious GCC linker script hacking to setup the overlay code at some common addresses etc, which is a very big ask. Probably doable but very difficult.
Even if the resource use becomes problematic to actually run the SPIN2 compiler on the same system as the target it could still be good to have the ability to use such a compiler tool on a second P2 board, potentially hosting its own IDE and debugger to attach a control a second target P2, serially. Then you can be in full control of your own destiny and not need to rely on other systems to build P2 software.
Update: of course all the above assumes a suitable filesystem layer is present on the P2 that works with the C code generated by P2gcc/flexC. Not sure if we are 100% there yet.
For what it's worth, there are options to generate .exe and remove dependence on the end user having to have Python (or specific versions/dependencies) installed. I've done this with PyInstaller.
Beeware > Briefcase project is interesting but is in it's early beta days.
Maybe some interesting examples of python applications, though rather complex
* Adafruit's Mu
* PlatformIO (which plugs into VS Code)
tcc mysource.c
and you get a mysource.exe as executable in the same directory.
I have written a Compiler and an Assembler for my own FPGA processor. It was also no problem to transfer the whole thing later to a RasPi.
Of course you should not write GUI applications with it, but for simple console applications I find TCC ingenious.
So and now I am curious how horrified the C/C++ gurus will be about this suggestion.
Visual Studio 2017 is free for more than professional needs for c/c++/basic/.net/android/....
They can't. Your app relays on the OS and windows 32bit can't address more than 4GB. (To be true it even can't address the full 4GB)
But you can compile for 32 or 64 bit system
My PC has a 32-bit CPU and it's quick because the Windows swap file is on a RAM drive, which could hold all of the Windows system files if I really wanted. I can use Eric's FastSpin without any problem and I don't see the necessity for switching from 386 assembler to 64-bit C instead of 32-bit C for PNut, if that's what we are discussing here.
I don't use PNUT because it doesn't support include files, I think, therefore adding #include to PNUT would be very useful. I've written a program that scans a PASM2 source file and writes skip patterns to a separate file for including under CON. A 2500+ line source takes about one second to process on my 1200MHz PC. Creating skip patterns manually is crazy.
What? Doesn't that defeat the purpose of having a swapfile? Wouldn't it be faster if you just completely disabled the swapfile altogether? Or is this because it's a 32-bit CPU with more than 4GB RAM?
It works the other way around on Linux: if RAM drives and the rest of the system are competing for memory, the swapper can swap parts of RAM drives to disk just as if it were any other piece of RAM.
https://codeproject.com/Articles/5061494/Run-a-C-Language-Interpreter-on-Your-ESP32
dgately
Ariba, I was thinking the exact same thing last night and figured there would be a lot of comments about how its not OSS, not advanced enough, etc, etc.
Chip should use it, and then for giggles someone here could compile it on GCC and see if there is any significant difference.
Would seem to be a far better use of Chip's time.
Yes swap file is normally part of windows. If anything Linux copied the idea. Older 9x versions are very aggressive about pushing less used RAM chunks to disk because that was the need back then. The typical 8 or 16 megabytes system back then is tiny!
You're talking about an ESP32-based Spin2 compiler and IDE? I'm sure that would be interesting to some. I doubt I'd use it because I don't have a problem with using a PC to develop code. I don't understand the obsession with self hosting. I do see a use for an interactive language on the target platform but we already have that in ROM with Taqoz. MicroPython or Javascript might be interesting as well. Being able to develop C or Spin2 code on an ESP32 doesn't sound that compelling to me. I'm sure it does to others though.
I think Chip said that...
That would be easy to make self hosted...
this whole thread seems backwards....now were are at c++ i could understand this if linux was a possibility...but even its at a 64 bit level these days
so why c++
now gcc...im not throwing that on anything but a virtual box!
spin is more like python anyway
if ur teaching kids: big buttons...less options
if ur self hosting to push the envelope with advanced level programming: write a multi-pass compiler
if ur trying to get spin2 going: its done it seems
HEY TEACHERS....WE WILL HAVE DIGITAL CALCULATORS IN 20 YEARS...JUST NOT THE INTERNET
That's certainly a tad incoherent.
Anyone who likes programming in assembly language is going to hate everything about C++. I really wish the NodeMCU people would get their stuff together and update the custom firmware build page to support the ESP32. I'm currently working with the Arduino IDE because it's a lot easier to get up and running than the Expressif SDK, but I had to run a search program just to find where the libraries were hidden on my hard drive so I could try to figure out how they worked. Hiding code and data like this is one of the supposed "virtues" of object-oriented programming which I've always thought was batshit crazy. Sure having a data type named IP_addr makes for real pretty and clean-looking code, until you need to convert four actual integers read from a file into one of those IP_addr structs and there is no guidance anywhere in the documentation on how to do that.
I am working full time too. Sometimes wish I had all the time off, but then I'd not get paid (like many that are on "forced vacation" right now).
Sounds interesting..