Forth as Communication Protocol between Raspberry Pi2 and Propeller
Christof Eb.
Posts: 1,106
Hi
the idea is to control a lathe using the following Setup:
* Forth as a communication protocol, tachyon?
* Raspberry pi2 for the user Interface and high level programs, graphics, storage
* propeller board doing low level tasks, at the beginning at least. Start would be step motor control with linear Interpolation.
What do you think of this setup?
(I have always found forth very fascinating but I am not convinced, that it it is best for me to do everything with forth and the propeller.)
Could someone give me some hints where to start? I have not yet found a place where to download an ide for raspi. Edit: found it. :-)
Many thank, Christof
the idea is to control a lathe using the following Setup:
* Forth as a communication protocol, tachyon?
* Raspberry pi2 for the user Interface and high level programs, graphics, storage
* propeller board doing low level tasks, at the beginning at least. Start would be step motor control with linear Interpolation.
What do you think of this setup?
(I have always found forth very fascinating but I am not convinced, that it it is best for me to do everything with forth and the propeller.)
Could someone give me some hints where to start? I have not yet found a place where to download an ide for raspi. Edit: found it. :-)
Many thank, Christof
Comments
I am finally getting around to implementing a design using around 42 Propeller chips on distributed buses in one enclosed system. Guess what? they are all running Tachyon since it was designed for demanding industrial applications and since these buses are half-duplex I decided that I would still like the full-duplex mode interactive experience with any one device so I am dedicating one of my many spare cogs to the task of looking after the bus and simulating full-duplex, all without any changes to the kernel. By typing an ALT+key I can select any Prop and talk to it as if it were connected directly to the console while the bus is still in operation. So I can debug and reprogram any single Prop easily. This is while they are madly handling critical timing and logging huge volumes of data to their individual Flash memories or SD cards, but then again that's what Prop's are good for too.
As for graphics I have another Prop module (one of my Pixie modules) connected to a VGA monitor and a keyboard for a bit-mapped graphic display, no need for an RPi there.
The smaller monitoring and operator displays are 320x240 TFT displays as well.
A WIZnet W5500 chip and my Easynet server software completes the remote access part so besides FTP and Web interfaces I can telnet in as if I were connected locally.
I'm even tempted to dedicate an FPGA board emulating the P2 and running Tachyon for the main controller and later on when P2 becomes available will hopefully be when production is ramping so a switch can be made then.
As for stepping motor I have used various driver chips such as the L6470/80 etc with ease.
I too think that there is merit in using small packets of Forth to communicate between widely different systems - in your example Propeller and Pi.
In more general terms I see Forth as a kind of Lingua Franca, which allows small packets of data and executable code to be freely exchanged between different micros, within a context framework (tags) that instructs the recipient to what they are, and what to do with them. This has a a slight smell of Chuck Moore's ColorForth, but the way the tagging is implemented may be different.
In my workplace and hobby environment, I have a mix of ARM M3,M4, M7, Atmel AVR, MSP430 (FRAM based) and now Propeller. I also have J1 Forth cpu and Oberon system running on Spartan 6 FPGAs.
http://www.saanlima.com/pepino/index.php?title=Welcome_to_Pepino
Trying to bind that lot together in a meaningful way is a little difficult, unless you can reduce everything down to a common Forth, or Forth-like language.
I had a couple of ideas to make the proposed environment easier to work in. After all, Chuck Moore developed Forth to give himself an easier, more productive computing environment - where he was in control.
So I came up with the (unoriginal) idea of BootForth - which is the smallest Forth kernel needed on an mcu, so that it can support a serial communication port, and compile source code from a terminal - allowing an application to be loaded.
There are some TinyForths out there - I found this one for the ATmega - that when written is assembly language is around 1800 bytes. It's also offered in C.
http://middleriver.chagasi.com/electronics/tforth.html
(Use GoogleTranslate to translate from the Japanese)
BootForth is really just an extended serial bootloader. It is included in protected Flash in the bootloader segment on every mcu you wish to use - and it's role is to provide the load mechanism - and present a standard programming interface.
In this way, mcu type A can communicate with mcu type B - regardless of what manufacturer or family they belong to.
I got some way to generating a BootForth for various micros - written in C for portability. I have so far used it on ATmega, STM32Fxxx ARM and MSP430. In it's smallest state - it's around 2Kbytes of Fash (or FRAM).
BootForth uses single ascii characters as word tokens. This drastically reduced the overhead of decoding words and jumping to the code.
IIRC, the ability to scan through an array of ascii characters and jump to an executable code address based on the value of the character can be reduced to a few hundred bytes - including UART code for key and emit and (getc and putc).
Forth is only verbose (6 significant character plus "length" word format) to make it more human readable, and to allow a large dictionary.
If we reduce our dictionary to 5 bit or 6 bit tokens that immediately reduces the side of the array needed to hold the source. We hold the expanded names of the kernel in ROM - so that we can regenerate the more verbose sourcecode later if needed.
Primitives are coded up as lower case alpha and punctuation characters, whilst user definable words use uppercase alpha characters. That restricts the number of useful words it recognises to about 80 - but it does make the source code readable as a compacted form of Forth - rather than random ascii gobbledeegook.
Restricting the code to ascii chars, makes the source very compact and quick to load and compile.
Source code can be loaded and compiled extremely quickly with a 1Mbaud+ serial terminal - or equally the sort of thing that could be sent in over the air wireless packets, 140 character tweets, (SMS messages), BLE or any other short packet communications method.
By way of example:
Consider the classic washing machine example consisting of the following high level words:
FILL
AGITATE
DRAIN
FILL
RINSE
DRAIN
SPIN
This can condense down to a 7 character string "FADFRDS" plus return.
Integer numeric parameters may be added before the words - without whitespace, allowing a compact 16 byte program
"5F100AD5FRD1300S"
5 litre FILL
100 Agitations
Drain
5 Litre FILL
Rinse
Drain
1300rpm SPIN
In Forth this would be 5 FILL 100 AGITATE DRAIN 5 FILL RINSE DRAIN 1300 SPIN
About 53 characters - 70% of which are redundant to the actual logical meaning of the application.
The complete WASH application can be initiated just by sending a single upper case character "W" - followed by return.
Go back and try applying this to your RasPi sending serial instructions to your lathe controller - and you will see just how compact it can be.
If a word requires multiple parameters on the stack, then you need a separation character, which when interpreted pushes the stack down to make way for the new number. I use the space for this - which makes the code fairly readable.
e.g. Plot a pixel at x, y (100,200)
100 200P
This conveniently brings me onto idea #2 - Graphical commands
***********************************************************************
I got interested in the use of GUIs for creating a better programming environment and less arduous experience. Sequential access into source code using a traditional terminal - constantly scrolling up and down to find a particular block of code is something that is rooted in the teletype and CRT terminal days of original Forth.
As we now have LCD monitors of high resolution, why not lay out the upper levels of the source code out as a 2 dimensional picture of objects - using the whole screen to show the application.
Use of colour, text boxes, drop down menus and a mouse-clickable objects presents the application code in a highly visible format and allows you to drill down to the individual definitions you wish to edit.
We are essentially recreating a late 1980's style home microcomputer, of the 512kByte to 1Mbyte class (think Commodore Amiga, Atari 1040ST, Acorn Archimedes) but with a Forth inspired GUI - all running on 1 or 2 Propellers.
Plus we now have all the benefits of touchscreen LCDs, micro SD cards for storage, large screen monitors, PS/2 keyboards and mice and very low cost RAM and cpu technology - we couldn't even dream of 30 years ago.
Here's how I envisage it working:
In it's simplest form, it is used to display the words in the dictionary as clickable coloured text boxes on the screen. (You are able to choose the colour to give meaning to your specific application) Clicking a word executes it - in immediate mode. There also needs to be a text box that allows you to enter the various parameters onto the stack before the word is called.
Right clicking the word opens up the source code behind it - showing all the dependencies. You can further click on these dependencies - each of which open a source window until you get down to the primitives.
Consider the Washing Machine example above - with the five definitions FILL, AGITATE, DRAIN, RINSE and SPIN arranged as small text boxes on the screen - all below the top level word WASH.
Beneath these words would be textboxes for any other dependencies - until you get right back to the Forth primitives.
At a glance, the screen shows graphically the whole application - and you can right mouse click on any word to "open it up" and begin to edit the associated source code.
This scheme can also be used for writing new source code.
You have a colon command ":" that creates a new code object - i.e. a word. On the GUI screen this has the effect of creating a new text box - into which you write the name, and the code associated with that new definition.
Included in the source you may use definitions that have not yet been defined - and the first pass of the compiler detects these and generates the accompanying text boxes in readiness for you to add the missing definitions and their associated code.
You should be able to tab between words, and a single right click of the mouse to "see" the source.
A mouse click to expand a word, generates a short Forth sequence that requests the mcu to send the source for that word - basically it sends the "see" word followed by the name of the word that you wish to decompile. The mcu sends the source within a wrapper of GUI commands that ensures that it is displayed on the screen where you want it.
Remember that it is the Propeller that is generating this GUI content - all the PC is doing is displaying it conveniently, object by object. The PC provides the mechanics of the display, a set of displayable objects - but not the content.
All of the content, in terms of the objects to display and position, colour etc is controlled by the target mcu. In this way, if you have no PC present, the whole thing becomes self hosting, with the Propeller running a small O/S which handles the Forth Application Window, keystrokes from PS/2 keyboard, mouse data and clicks, editor and interprets the commands over serial.
Forth Application Windows, Keyboard, Editor, Serial = "FAWKES"
At the other end of the serial (or other comms) link, the PC can be used as a disk filesystem, if there is no local SD on the Propeller target - with file open, close read and write requests being sent over the serial connection. This could utilise the traditional Forth BLOCK, LOAD SAVE words, but I suspect that we have moved on a little since such a regimented system.
As I am not a PC application programmer, I would use something like "Processing" to generate the GUI and Disk O/S - "GUIDO". It's cross platform - so will run on any PC including the Raspberry Pi. That makes it accessible to and hackable by, - anyone.
https://processing.org/
In terms of the hardware, I started looking at the FT81x series of embedded video engine (EVE) chips to create this GUI. They use a simple list of 32-bit words to build up the screen, object by object.
So I thought that the "display list" that is normally sent as SPI commands to the FT81x graphics controller IC, could be sent over a serial link to an application running on a PC - which emulates the FT810.
It was about that point where I re-stumbled across Propeller, and realised that the whole thing could be done using Props generating VGA and supporting a mouse and keyboard.
I then came across Ray's Logic "PropMonitor" and realised that this was mostly what would be needed on the PC end.
http://www.rayslogic.com/propeller/programming/PropMonitor/PropMonitor.htm
There's a lot of coding to get this to work, but I hope that you get a flavour of what I am trying to implement. The ideas behind this concept are purposefully kept simple and generic. Whilst it might apply to Propellers and ARMs today - who knows what we might be programming in the near future.
Comment and Critique Appreciated!
Ken
London
Ernestly: if a communication is expected to be corrupted, recovery methods have to be implemented, like parity bit, what allows to detect one bit errors, crc, what allows to detect errors and repair them by having redundant information in the data stream. The programming language itself should not influence the number of failures due to the physical behavior of the control system. But there is a difference in who easy to introduce a weak point into a program. A type check prevents many errors. But sophisticated programming is self modifying the code and it is very unlikely, a compiler/linker can forsee an error situation.
So Tachyon is not a bad choice, aside from the fact, you will not find too much support in the forum
Interesting comments, interesting because they are your personal views but some aspects of which I totally disagree with, not from a personal viewpoint however. I would think that something not being suitable for safety-critical systems had more to do with implementation rather than the language and besides I have used Forth in safety-critical systems including industrial machine shops. Communications protocols are just that, they are not specified for some particular language although what Ken is proposing is simply using Forth in a more efficient yet still flexible way rather than a fixed syntax/format method. The thing is that even this type of "protocol" is still a layer which may be delivered straight from the PC's serial port without any error checking or it may be delivered over MODBUS protocol with some rudimentary checksum or it may be encapsulated in an Ethernet packet with its own error handling. So Forth does not make it less suitable but in this proposed instance it has many advantages.
The other thought I had was that many "Forths" that some have been exposed to have been little more than quick hacks just to have "a Forth" for xyz. Try doing that with any compiler or code etc and you may not be impressed with the results, however handy and quick it may have been. So from my point of view I know WHY I wrote Tachyon and HOW it should operate, so there was no need for ANSI "PC" standards, only for a robust and fast Forth for industrial applications. The "proof of the pudding" I already have in very many commerical products all working reliably 24/7.
@Ken: That was a bit to chew through and I just haven't had time to reply properly yet but sometimes if you just dive in and try it then it helps to sort out the specification and also makes it easier to talk about too. I can see what you are getting at in reducing the size of the transmission and simplifying the parsing which is a a good idea. More later.
Thanks for your encouragement.
It was finding your Tachyon Forth implemetation on Propeller devices that really inspired me to forge ahead.
I must admit that some of my postings can ramble on a bit - but the concept can be reduced to a couple of key sentences:
A lightweight, but expandable kernel running on the target microcontroller device, which has the means to accept additional word definitions, add them to memory and execute them.
A "heavy lifting" package running on the PC, which provides GUI support, and hence the means to provide a graphical illustration of the application code - thus making the Forth easier for mere mortals.
There is no doubt that Forth can be hard to master, for many people, but it is such a neat, elegant, lightweight and compact programming language - which can be applied to virtually any micro.
It's no worse than java bytecode - but unlike bytecode - doesn't need a big sweaty compiler to produce it.
I just love the way that I can get immediate interaction with a piece of target hardware - just by typing a few words.
My interest lies currently in educational products where I need seamless interaction between widely disparate devices. I need a Lingua Franca - that you only need to teach the kids once.
The devices should be self-hosting, or at least be capable of providing an interactive environment with a modern PC or portable computing device - which includes tablets and smartphones.
The current trend is to move to touchscreens - with their limited text entry capability. The development environment should reflect these trends.
It doesn't matter whether it's a 10 year old making smiley face patterns on an RGB LED matrix, or a factory electrician making program updates to a mission critical plc - the process of interaction is all important.
When I started programming in 1979 it was a self hosting Z80 CP/M machine. Later, I was hand-assembling Z80 machine code - where a pencil and paper were the preferred tools. I have sweated with the hard stuff - I now want an easier programming environment.
I want simple engineering tools that don't consume weeks of my time to learn and gigabytes on my hard drive. ( Altera, IAR, Keil, Xilinx - please all take note).
Forth has certainly been niche in the past - but with so many more people now being exposed to coding, and "new" languages virtually every year, there is still a very good reason to continue to promote Forth - as an alternative way of doing things. But not in it's previous guise - but with a pretty, shiny new wrapper that makes it much more appealing. Sugar coating a bitter pill - that's intended to make things better?
Ken
Yes, but that was basically a bytecode Forth built into the boot ROM and also bytecode (FCode) in the peripherals and especially the plug-in cards so it didn't matter which processor it ran on as long as it had OpenFirmware built in. OLPC and even PowerMacs also used this. The Forth shell could even be entered at boot time for diagnostics and developing drivers. I think it was Mitch Bradley from memory that implemented this originally as simply a way to test new processors and hardware for Sun Microsystems since compilers take time and they need proven hardware too. So this Forth was the boot system in ROM for these computers as it was necessary for development and testing of hardware and then the portable bytecode idea probably came a bit later.
Ken
We were running machine tools, Gerber plotters and linotypes off PDP-8s in the late 1960s.
I don't think they needed all that bloat?
OK - I suspect I have posted in the wrong thread?
Make sure that both sides have a common vocabulary - how hard can it be to support a few dozen Forth words?
The PC side can support a bunch of GUI drawing objects, the mcu needs only to specify what it wants drawn and where. Uses PC as glorified graphics co-processor, and file system.
Ken
There is only compatible bytecode on the cards that is essentially the machine code of the VM running on the processor much like a Z80 on the motherboard will run Z80 machine code on a plug-in card etc. OpenFirmware is both a Forth VM and BIOS too.