Interactive Forth as a development language and tool vs "sealed" compiled code
Peter Jakacki
Posts: 10,193
As I was going about my business updating documentation on my Tachyon Forth and source code I was also exercising a lot of the functions that I have built in, testing and timing them. The thought occurred to me, "where would I be if I wasn't using this stuff?".
That's because with my Forth I have such intimate connection with the Propeller chip itself as well as as the software modules that make up an application. The level of interaction with these is astounding and one that I can never achieve with a conventional compiled language that generates some binary bits that get loaded into EEPROM and it's all sealed up in there. Yes, I have written all kinds of debugging extensions for Spin programs as well as other languages but despite the effort put into this the results pale in comparison to what I can do with a system equipped with Forth. This is even more the case as Tachyon has so much stuff in there to see what's going on and to be able to change it easily.
Is Forth too slow or too fat that it wouldn't be used? No, quite the opposite, it's lean and mean in that department, yet it is rich with development and debugging tools loaded right into the Prop itself, why wouldn't it be more popular? Oh, you know, the RPN thing. So typing 0 100 DUMP vs DUMP 0,100 ? (If any other system would let you that is) Yeah, I get it although in Forth I can just as easily type BUFFERS 40 DUMP etc. Must be a real mindset thing but I get so much done and so quickly and efficiently with Forth than I would otherwise.
There are so many times I read the forum and someone is stuck trying to do something simple which I know in Forth would be even simpler, a line or two of code and seconds later you press enter, and presto! Even the so called complex stuff with all it's objects is for the most part much much simpler in Tachyon. I somehow feel that this great resource is overlooked too often maybe because it doesn't fit into what most would expect.
Hmmm, I know I should be quite, but every once in a while I can't help but stand up on my little soapbox and say something, even a little something. I know I'm not missing out.
That's because with my Forth I have such intimate connection with the Propeller chip itself as well as as the software modules that make up an application. The level of interaction with these is astounding and one that I can never achieve with a conventional compiled language that generates some binary bits that get loaded into EEPROM and it's all sealed up in there. Yes, I have written all kinds of debugging extensions for Spin programs as well as other languages but despite the effort put into this the results pale in comparison to what I can do with a system equipped with Forth. This is even more the case as Tachyon has so much stuff in there to see what's going on and to be able to change it easily.
Is Forth too slow or too fat that it wouldn't be used? No, quite the opposite, it's lean and mean in that department, yet it is rich with development and debugging tools loaded right into the Prop itself, why wouldn't it be more popular? Oh, you know, the RPN thing. So typing 0 100 DUMP vs DUMP 0,100 ? (If any other system would let you that is) Yeah, I get it although in Forth I can just as easily type BUFFERS 40 DUMP etc. Must be a real mindset thing but I get so much done and so quickly and efficiently with Forth than I would otherwise.
There are so many times I read the forum and someone is stuck trying to do something simple which I know in Forth would be even simpler, a line or two of code and seconds later you press enter, and presto! Even the so called complex stuff with all it's objects is for the most part much much simpler in Tachyon. I somehow feel that this great resource is overlooked too often maybe because it doesn't fit into what most would expect.
Hmmm, I know I should be quite, but every once in a while I can't help but stand up on my little soapbox and say something, even a little something. I know I'm not missing out.
Comments
The good part of Forth is indeed the interactive she'll, it makes trial and error experiments pretty trivial as you can just type in commands and see what happens. I find I don't need to write as much serial output debug code as I do with Spin or C++. I can also write and debug some code on the PC which I can't do in Spin.
Yeah, I hate the stack manipulation too although it's a small price to pay, but that is also why I don't implement PICK and ROLL, 3 to 4 levels is deep enough otherwise there are "local" variables. The weak typing is also a strength too I find, depends which way you look at it and I am free to treat the data whichever way I want, usually to my advantage.
Peter, is there any reason these couldn't be incorporated into forth?
-Phil
Forth as it is is simple and transparent, there is no special voodoo happening in the back room, just the voodoo that you do yourself. I mentioned before about not implementing PICK and ROLL and I try to keep my words simple without unnecessary stack manipulation, hence the reason for a loop stack for instance so that I can also refer to stack indices in a called function etc. So except for the occasional "ROT OVER ROT" the stack is easily handled.
But I take your suggestion on-board Phil and I will think about how this could be implemented and I welcome any suggestions, keeping in mind we have limited resources in 32K.
Another method would be to keep a bit array image of the main stack, with 1's representing marks and 0's regular data. mark would just set the bit in the bit array corresponding to the current stack pointer. cleartomark would pop the stack until the corresponding bit in the bit array was a 1.
-Phil
There is very little useful open source Forth code that can be re-used by others. Even if you find a useful snippet of Forth code it cannot be ported directly into many Forth implementations without having to rewrite the code.
Forth is good for bootstrapping onto a new processor that has no other development tools. Forth is good for interactively peeking and poking, and setting and resetting bits in hardware. Forth is good as a learning tool to teach how computers work. Forth is good as a scripting tool. Forth is good for quick and dirty programs to test out an idea or quickly calculate a value using simple expressions.
I think the second one tends to cause the first, and I think the second is a big problem. For example, I was looking at AmForth recently and I realized that it's a 16 bit Forth, so none of the fixed point arithmetic code I wrote would port to it unmodified. But I can write C code and easily port between the Propeller and the Arduino.
I think you have to get your own soap box Dave! Yep, Forth sucks at syntax checking as you create your own syntax, not the forever stuck this is the way you must do it syntax. Yep, what error checking? That's not a bug, that's a feature, I don't want nobody telling me I can't do it that way. Yep, all I need is right there on the stack ready for picking.
What good is open source anything unless it has been targeted for the Prop anyway? I think the bulk of the software that has been written for the Prop didn't make use of OSS anyway. Why should Forth be singled out in this regard? Besides, once I've written it, it is OSS!
Yep, Forth is good for bootstrapping new processors even if there are "development tools" available, they are not in the same league. Yep, Forth is good for interacting with the hardware but not in the clumsy way of what peeking and poking might imply. Yep, Forth is good as a learning tool to teach how computers work. Yep, Forth is good as a scripting too. Yep, Forth is good for quick and dirty programs that get the job done when you need it done.
Glad you agree with me then Dave
BTW, I wasn't trying to compare "languages" anyway because the language is only part of what Forth offers, and that is an OS, compiler, scripting tool, and debugger built into the Prop
The Prop hasn't used much open source before because it only supported proprietary languages. However, with C and C++ support the Prop is already making use of open source code, and this will increase in the future. And PropGCC supports the low-level features of the Prop, such as input/output ports and the special instructions that we love on the Prop.
I have not issues with stack, but forth is quite unique under many other aspects.
While with my programming level all the others language are more or less the same (Ok, I'm a very lame programmer), forth is quite different.
I really like the interactive approach, the possibility to modify remotely and on the fly.
I have issues learning how to use it, and how to exploit the prop power. Syntax is quite different, and needs time to digest.
One of the main prop features in my opinion is the obex. For instance I use a lot of stuff requiring serial protocol.
With spin I simply grab the 4port object and I'm done. Same applies to many other examples. A forth way to load an adapted pasm module would be great (v3?).
I used a lot the obex also for learning prop programming. I started with very simple objects and the demo file, and started playing with it, but that's my way.
I for sure think many of my past and future projects would benefit from forth, but in my case the learning curve is quite steep.
I posted a figure of eight done with tachyon and it was fun, but I have a lot to learn.
So bottom line is:
thanks for opening a new programming world to me, even if it is causing me frustration...
Massimo
...is how we do it. If you want to rename or alias these as cleartomark, you are free to do so, which is what they did.
also
... for "stack clear" clears the stack.
Same for assembler. Therefore we should get rid of assembler?
The FORTH code is for the most part so simple that you can just look at it and see whats different. If its too complicated, you would have had to re-write it anyway.
This is why we generally go by the "KISS" method.
No one is suggesting getting rid of Forth, PASM or any other language. I have always said that the best tool should be used for the job. As I said in my first post, there are certain things that Forth is good at.
Your comment about simple Forth code is good. Forth is good for writing simple pieces of code. It's not good for large complex projects. If a piece of Forth code becomes too complicated to the point where you have to rewrite it and break it up into smaller words it's probably better to use another language that can handle it more efficiently.
Yes, I noticed that about missing the mark but I figured there would be a Doh moment eventually.
Forth is good for writing and testing small pieces of code, that's the way it should be and being able to factor something complicated into smaller simpler bits AND being able to test them easily is surely a big advantage. I don't know what you might be calling "complicated" but I have an application right at the moment which combines SD card file access, FTP and TELNET servers and various other communications protocols all running from the one cog at present (SPI, HLL, the lot). I know that when I have attempted this with standard libraries that I very quickly run out of resources so that it becomes impossible or impractical to implement. That's why I did it with the quick and dirty method that gets it done, in Forth.
The Propeller in particular has no concept of stacks built into it.
Peter,
On the face of it I should love Forth.
It has interactivity which from time to time I really appreciate in systems with a REPL like JavaScript. Check out the Espruino project for how nice that can be.
It has speed and it has a minimal size. Thousands of times smaller than JavaScript run times, even the tiny one on the Espruino.
The RPN is not an issue. As you say writing an expression like: is hardly more onerous or hard to understand than: If languages like C had RPN expressions it would save all that uncertainty of operator precedence and parenthesis.
However somehow Forth does not work for me. It's impossible to get anything done. It's seems to be easier to write stuff in PASM. Sadly you seem to have missed my discussion of my problems on some thread or other a few months back. Despite a long debate with a few Forth enthusiasts I could not get anyone to tell me how to make an array or even named global variables. Then actually getting any non-trivial calculation going was impossible.
I think the most complex thing that I've written in Forth is the ted, which is my tiny editor. I basically modified line-by-line my Spin version of the code to do it. It makes use of many named variables, and it would have been harder to program and understand using stack-based variables. The most complex Forth program that I've gotten running is the chess program. Fortunately, it was written in ANS Forth, and it was fairly easy to port.
http://forums.parallax.com/showthread.php/151826-Scara-robot-using-servos-and-hardware-store-components
It creates arrays and does fixed point trigonometry. The last time I checked it also compiles with gforth because I defined a few compatiblity words and avoided create does>*. As far as the difficulty in writing it. I've done fixed point trigonometry in PBasic and inverse kinematics in Spin and C++. I would rate it as easier to write then trig in PBasic because you don't need to worry about PBasic's unsigned division. It was also easier to write than the Spin code because I did most of the work on the PC using gforth. It was definately harder to write than corresponding C++ because I was able to rework some existing code I found online.
* I think the absence of create does> from Propforth is a shame. I know Sal and the Prof have their reasons, but it is a barrier to adoption.
[1] Forth is good for bootstrapping onto a new processor that has no other development tools.
[2] Forth is good for interactively peeking and poking, and setting and resetting bits in hardware.
[3] Forth is good as a learning tool to teach how computers work.
[4] Forth is good as a scripting tool.
[5] Forth is good for quick and dirty programs to test out an idea or quickly calculate a value using simple expressions.[/QUOTE]
That 's a lot of good. I was absolutely ready to give up on the Propeller when I had to choose between Spin and PASM.
You have managed to clearly list all the reasons it is useful for a new learner as an alternative entry point to using the Propeller.
Of course, I may need C or PASM, for more ambitious projects.. but we all need to start somewhere before we get to the big projects that require more language resources. I am fully aware that many times, i have to write my own tool to have in Forth a feature that is simply provided in other languages. But learning to write such tools might just be an item [6] to add to your list of good things.
And [7] might be that Forth can be a good way to learn PASM in the context of an elaborate example... at least that is why I am so involved with pfth code. I am finally reading and able to verify what I am reading in PASM via the pfth.spin file.
Tachyon and PropForth are a bit more challenging to use to learn PASM.. because they are more complex and more optimized. But all three are worthy of a loyal following on the Propeller. Even more so when the Propeller 2 arrives and users want to peek and poke new features.
It becomes obvious that the Propeller and Forth were a match made in heaven. Why so? One really needs 32K of ram to have enough space for a Forth dictionary on an microcontroller. And then having 8 cpu's sharing the same dictionary AND 32bit maths just upped the power and glory of what it can do.
It is fun, it is interesting, and it is helpful to new learners. I'm happy with Forth.
Interesting, thanks.
What's with the "Heater's Square Root"? It's pretty certain I did not invent it. I don't even remember posting it anywhere now:)
@Heater,
As best I recalled your 'non-trival' calculation application was Fast-Fourier Transforms... simply an application that might best be optimized in assembler on any machine. Admittedly, Forth does not readily embrace precision floating point. FFT almost cries out for a dedicated math co-processor.
Tachyon and PropForth have done some things different than ANS Forth, but pfth has made a brave effort to conform.
If you want a Global Variable, or an Array, these things are well-documented in a pdf called "And so Forth" by Hans Bezemer. ... for ANS Forth
http://www.forth.org/tutorials.html
~~~~~~~~~~~~~~~
I am very surprised by Dave Hein's comments at this point as he has accomplished so much with pfth. But I do have to admit that he has a lot of programming talent and experience in languages other than Forth and may just feel it has become a drag on productivity.
But I certainly do have to admit that to program Forth, one absolutely has to learn to type correctly or be constantly chasing typos down. That is very un-forgiving and the price to pay for such a simple minimal interpreter.
I have enjoyed comparing the pfth.spin code to the eForth model for creating Forth on a microcontroller... learned a huge amount and opened up new horizons.
I found it here: https://code.google.com/p/propforth/wiki/SquareRootExample which references this page as the origin: http://forums.parallax.com/showthread.php/144414-The-Forth-Dimension./page3?p=1148471&viewfull=1#post1148471
So it looks like you either wrote it, or found it elsewhere.
Ah yes, the "I can feel my sanity slipping away..." post. I do remember that so I guess my mind did not totally slip away.
I take full responsibility for that Forth code but I have no idea now what I based in on, probably some C snippet from somewhere.
I'm amazed it made it to google code!
With that said, I can understand that Forth seems perfectly logical to some people, and they can't imagine programming in any other language. That's cool. I think most people are wired to program in languages like C, Basic and Spin, which is why I think it's better to suggest that approach for newbies. At some point, a programmer should try a variety of languages. There's something to learn from every language.
OT I am discussing Forth as a development language beyond the Q&D or toying with it stage, no, rather using it to develop full-blown applications. I do this daily and if I listened to all the reasons why I can't then I would not have been able to do many of the things that I have done. I have also had many opportunities where an application was developed in Forth and gotten off the ground in short time and with little resources vs later "we can do it better" conventional language approach that already had the application foundation to work off. More money, more people, more libraries, more time - becomes - more money, more people, more time, no more room for libraries.....
But the focus here is on Forth, for it to be considered seriously because even those newbies might be more inclined to click with Forth which I think are perhaps in the minority. Maybe that's a right brain left brain thing or something, but certainly some click with Forth, some never get it.
BTW, Thanks for the kudos Dave, I don't think my nascent code is altogether perfect or complete yet, but it is certainly workable, and it's out there.
-Phil
It's been a while since I've read C compiler output, but I recall function prologue and epilog code contributing a fairly sizeable amount to the program. They had to define register call linkage to speed things up, but that only worked for functions that didn't call other functions.
Forth having the programmer operate directly on the stack eliminates much of that code, so that alone should shrink your code. But in theory Forth encourages smaller functions which allow for further reusable blocks of code. Overall my detour into Forth programming has been interesting.
In what way is forth threaded?
Did I miss a point here, are you talking about the normal meaning of "thread" (multiple execution paths at the same time) or something else?
More here: http://en.wikipedia.org/wiki/Threaded_code
-Phil
Next question, what was ENTER, EXIT, and NEXT?
The first two sound like x86 "enter" and "leave" instructions and similar on other processors, what about "next"?