EtherCAT experiments
ManAtWork
Posts: 2,178
Today I've made the first successful communication with an EtherCAT servo. First, I had to scratch my head several times because the device refused to respond while WireShark displayed a perfectly valid EtherCAT packet. But I found out that the first packet seems to be necessary to pull the servo out of some kind of sleep mode. It responds to every packet from the second one.
Of course, this is a very rudementary test and there's still a long way to go to make the motor actually run. All that is needed for an EtherCAT master implementation is a normal Ethernet accessory board (the left one). I've already made an accessory board for a slave device (the right one). The later is a bit more complicated, has a LAN9252 and two RJ45 jacks on it and requires the QSPI driver to work before doing anything useful.
Comments
Interesting 👍
Good work. Going where no man has gone before. I wouldn't have even contemplated pulling off raw RMII let alone whatever is stacked on top.
Out of interest, how many cogs are being used to get this far?
For the master interface two cogs are used. Because sending a frame and receiving the response occurs overlapping we need full duplex with the receiver cog always ready for a frame to arrive. The sender has lots of free resources so I will add a command interpreter to it so that the full protocol stack won't need more than the two cogs.
The slave interface currently uses no cog at all and consists only of a cog-less QSPI driver. All low level protocol handling is done in the slave controller chip LAN9252. However, the final implementation of the full slave protocol stack will require at least one cog because it needs to be ready for incoming mailbox read/write request at any time.
And the EtherCAT so far? Is it just sitting in with the sender cog?
I haven't implemented the full protocol stack, so far. The data link layer is relatively simple and only has read and write commands. To run motion control devices like servo drives we need another protocol on top of that. It's called CoE or "CanOpen over Ethercat" and is basically the same as the older CanOpen standard with just some extensions. But yes, I think that can be implemented in the sender cog.
That of course, presumes that the master already knows which variables of the process image have to go to which "objects" (registers or mailboxes) of the slave devices connected. For a full blown universal PLC or CNC controller application you need a way for the user to configure multiple devices from different manufacturers which requires a lot more resources. The "object directory" (description of registers, their data types, meanings and valid ranges) of a single servo drive is several megabytes long! You need a lot of memory, a GUI and a filesystem to implement that.
So my plan is to divide that task into a program running on a PC and an external device with a P2. The P2 scans the field bus and enumerates the devices connected. The P2 uploads the list to the PC and the user can configure the whole machine. The PC then downloads two lists of commands to be processed to the P2, one for one-time configuration like speed and acceleration limits, control loop gains etc. and one for cyclic processing like position or torque commands for the tool trajectory. The P2 then only needs to keep the real time part of the process image in it's memory and not all the data of all objects of all devices at the same time.
This is great! My friend Doug uses ethernet servos in his automation jobs, but he is driving them with ultra-expensive Allen-Bradley PLCs ($4k).
$4K doesn't buy much AB hardware...Here's the programming software:
Studio 5000 Professional Pricing:
Option 1A: Annual subscription + 12mo. 8-5 support, PN# 9324C-RLDT31
$3,246.18 (12-01-22)
Option 1B: Annual subscription + Legacy + 12mo. 8-5 support, PN# 9324C-RLDT41
$4,118.00 (12-01-22)
Option 2A: Perpetual license + 12mo. 8-5 support, PN#9324M-RLDT31
$10,079.35 (12-01-22)
Option 2B: Perpetual license + Legacy + 12mo. 8-5 support, PN#9324M-RLDT41
$12,661.62 (12-01-22
This is my pet peeve....The cartel traditional suppliers are ripping-off industry and they allow themselves to get sucked right-in.
This nonsense happens all the time in my world and this is a tiny example:
So now you have an entire production-line, dead in the water and a bunch of hourlies with nothing to do....Now what?
The same applies to PLCs, PLC-modules, servo-drives....everything.
If my HMI dies (never happened after 12 years, now):
Just grab a tablet from Walmart or use an Android phone. Download the app, pair the Bluetooth and we're back in business.
Craig
Having spent a few weekends frantically trying to revive a dead factory while AB and MS sort out what broke when Windows updated...
Allen Bradley.. You can buy better, but you will never pay more!
On the other hand, they have excellent integration with most design tools. It is SOOOO easy to spend all that money at design-time!
Any more... since I am now in a position to set the specs..
If the PLC doesn't use Codesys.. it doesn't get designed in.
If the custom board's control chip doesn't understand SPIN.. it's got to have something else REALLY special...
If the computer or hmi runs Windows.. it is soon replaced.
Rock On @ManAtWork!
@Mickster
Those tablets are SOOO handy for control and the operator can carry it where they can see what is happening.
But, a welding laser will do a very messy job of cutting one in half... Don't ask how I know...
R Baggett,
Any hints for a newbie doing a Codesys demo setup on Kubuntu in a tower PC? Is there any free options?
Oh absolutely. With some equipment, one needs to recruit another pair of eyes, during part-setup, to watch for collisions. This is not necessary with a portable HMI. Furthermore, I utilize the onboard sensors for when I have an oh-sh*t about to happen. Just tilt the tablet and the machine stops dead.
Same thing applies for machine diagnostics; one can directly trigger a prox-sensor and see for themselves if the signal is getting through.
A certain product requires a specific tooling-die or adjustment and other personnel need to know: Tablet records still or video images of the setup and these recordings are stored with the part-program under "Setup Notes"
Schematics: They're on the device along with how-to videos.
Remote support: Real VNC or Teamviewer
Graphical HMI: No sweat
WiFi: Standard...why tie-up the Prop
Bluetooth: Standard
Battery backup
Memory
Storage
I have hundreds of Windows-based controllers throughout NA, mainly in big-automotive; backup/restore always has issues. With Android, we download an app and Voila. A bit fiddly if one has to temporarily use a smartphone but at least, parts are coming off the line.
Welding laser?
Actually, I wanna know what happened 
But yeah, fork-trucks happily plow through [insert overpriced-and-probably-unobtainium-HMI] control consoles as well.
Craig
@Mickster I was probably a bit peeved about you frequently hijacking my thread and posting examples of overpriced industrial controls or critizism about Beckhoff and EtherCAT. But I begin to understand your antipathy. I've spent a lot of time browsing the EtherCAT specification and related documentation from Beckhoff. The basic EtherCAT protocol (data link layer) and other details (sync managers, distributed clocks...) are described quite well. But as soon as it comes to a real example how those things work together further information is really hard to find. I already thought about playing around a bit with TwinCAT and reverse engineer the ethernet packets recorded with WireShark. But I already gave up on finding out what license I need. Their web page is all shiny with lots of impressive pictures but zero information. So I asked a friend who I know that he has worked with TwinCAT. He warned me not to even think about getting any result without first learning to program CODESYS.
I think all this stuff is just not meant for companies that don't have multiple man-years and milions of dollars budget.
But that's not the fault of EtherCAT itself but that of the implementations of Beckhoff, Allen Bradley or other big players. The basic idea of taking existing fast and inexpensive networking hardware and modify the protocol to optimize it for real time applications and machine control is ingenius. It's a pity that I haven't got the idea myself. I've worked with Interbus in the 90s which is a very simple and efficient field bus, basically daisy chained devices acting like a big shift register. This is very similar to how EtherCAT works, just with more modern hardware.
I think it's time to make a user friendly implementation that doesn't suffer from extreme feature-itis, that is affordable and that an average user can get to work within one day.
@ManAtWork
Something happened after the mid-80's and it's a conspiracy, I tell ya
All of a sudden, multi-axis motion control was possible with just two IC's. A CPU (68008) and an FPGA and naturally they appeared on a ISA-slot card. I purchased this card along with a 3-axis linear amplifier and three DC motors with encoders. Having never written a program before, I loaded-up the BASICA sample code on my PC-XT and after setting gains, I was instantly able to not only position these motors but coordinate them using ASCII commands...absolute child's play.
In 2024, if the AB/Beckhoff/Siemens equivalent shows up on your desk for the first time......there's no chance in hell that you could do what I just described. Why not?
Somehow the big players have kept things complicated and expensive and the situation reminds me of The Emperor's New Clothes. If an engineer doesn't toe-the-line and agree that these ridiculously complicated systems are the best way to go, then they are not worthy. Plus, engineers are always thinking about their own marketability and résumés (let someone else pay for my education).
Purely for my own satisfaction, I am building something that can be fully maintained by the guy on the plant floor with his screwdriver and DMM
This conspiracy was already there in the late 80s! I think it was around 1989 when a friend called me up and described his problem. He was about to build a paint mixer machine (one of the first ones that you can see at every DIY construction store nowadays). It consisted of basically 20 barrels of paint of different colours, 20 ball valves and a electronic weight scale under a bucket and a PC. He asked Siemens how to control the ball valves with the PC to be able to precisely dispense the paint. They suggested 20 servo drives for $4000 each and a big motion controller. The would have been able to pay that but they made experiments and it turned out that the motors were too weak and stalled when there was dried paint in the valves. So they would have needed even bigger and more expensive motors. They asked me if I knew something else more affordable and reliable.
I took a windshield wiper motor from a truck and designed some PCBs with a big 24V 20A PWM power stage and an 8 bit ADC to read the position of a potentiometer. We made some tests and were able to precisely control the valve position with a resolution of 90°/256. It took less than half a second to turn the full 90° and my friend was quite happy. Then he wanted to test the torque. He had the crazy idea to stick a broomstick into the 1" ball valve to see what happens. To his surprise the drive didn't show a positioning error when he commanded it to move. The motor didn't even slow down remarkably. But when he pulled out the broomstick he saw that it was cut in halve!
The motors + potis were around $150 per unit. We added a big "demultiplexer" of 20 relays so that one power stage served 20 motors. I charged around $500 for the power electronics which was way overpriced from my point of view (BOM cost ~$50) but saved them $76,000 per machine compared to the Siemens solution.
@ManAtWork
My kinda talk 
One of these days, I'll be over for a few (proper) beers
Craig
Operator set up the path for laser welding a steel panel.. then set the tablet down and performed a few other tasks, left the cage, unlocked, and hit start.
mm or it would have been really 'spensive.
Of course, the tablet was right on top of the weld path.... of the 5 KW fiber laser. The bottom of the nozzle cleared it by
how does less than sign followed by 5 mm translate into a heart?
Yep,
Codesys control runs for free with a time limit. The license is cheap to go ahead and make full use of it. Very easy to install on Linux (Yes, even on RasPi)
The programming environment (Unfortunately not available for Linux) has a very good simulator as well.
All flavors will serve up an HMI web page.
I would also point out that Codesys supports Modbus serial, and we have a nice Modbus object for P1.. Quick and dirty I/O and then some. Looks easy to adapt for P2 also, but I haven't messed with that yet.
I am not aware of any ethernet protocols it does not support.
Yeah you had me worried for a minute
That's the new AI rendering - it was a heart stopping moment !
Oh, bugger. That was the part I wanted to demo on Kubuntu.
Strike! No need to spend money for TwinCAT. There is a free version of "EC-Engineer". I think it does everything I need. At least I can scan the connected devices and display the registers and CoE objects.
When I turn the motor shaft the value "actual position" changes. So I can reverse-engineer what's going on on the network and then look up in the specification what the data actually means. Doing it the other way round is totally impossible becaused the docs are so confused you don't find anything.
I have NO experience with it, but I wanted to double-check and found this:
https://forge.codesys.com/tol/codesys-4-linux/home/Home/
Trying to run Codesys programming environ on Linux originally looked like a hard trip with high likelihood of failure, but with this, I might give it a go.
Huh, that's actually hosted on their own site too. There's a number of warnings though, the most concerning being the first one - that it's now outdated and no longer works because Codesys has changed too much.
EDIT: Ah, maybe efforts on the Wine solution has ceased because of work on "Codesys Go" web based tools. It'll be lots of javascript I presume. I'll count my blessings I guess, at least it'll be cross-platform. Just have to wait for V1.0 to come out ...
@ManAtWork
!
Good news: Now I can communicate with the servo at least at the lowest level. There are four states in the protocol state machine: init, pre-op, safe-op and operational. In the init state only reads and writes to registers in the physical address space are allowed. I can scan the bus and "enumerate" the devices, that means I can assign a fixed address to them by first using position addressing (the cable forms an topological ring which results in a natural numbering order, so no DIP switches needed).
Phew, this wasn't easy.
The Beckhoff documents are not only confusing and and written in an extremely theoretical language. They do not use a consistent format for the layout of registers and data types. Little-endian and big-endian representation is mixed arbitrarily. The most efficient was to find out is really looking at the WireShark display which provides a nice view of what's in the packet.
The second obstacle was that the slave devices have a watchdog that cuts the connection when there is no valid network traffic for several milliseconds. There is no way you can single-step debug the communication. Even doing too much printf()s over the ProgPlug port can slow down things too much.
Third, have I already said that I don't feel comfortable with C?
Well, it's the lesser evil. Coding the protocol machine in Spin or assembler would be even harder. But I thought that I could at least use the structured data types of C to some advantage. After all, it's not as easy as expected. There are some special alignment rules in FlexC that don't mach that of other compilers. So I couldn't use the definitions found in example code on the internet but instead had to apply special tricks. But I don't blame Eric because it's really only the tip of the iceberg.
Ugly constructs like this are necessary to advance from one datagram to the next because their sizes can be any number, even or odd.
The next thing on my to-do list is setting up the Sync Managers. Then I can switch to the next level and try out mailbox communication (CoE or CanOpen tunneled via EtherCat).
@ManAtWork
Is this part of the hot-swap deal or being able to switch off a node? Sounds like a debug PITA. I have my watchdogs bypass-able for this reason. CoE next?!?! Yer a better man than me
Craig
I'm just different (not so much different from you but from the average engineer, I think). I recently had a chat with a friend who is also a software and hardware developper. I mentioned that I plan to write an EtherCat master. He said "Are you crazy?! There are existing implementations available as open source. So doing that again is a waste of time!". I answered "You can call me stubborn, but if I have an idea of how to implement something I just have to do it in my way." So in reality I'm very slow and inefficient compared to other programmers who often prefer "glueing" existing libraries together over creating new code. ... or even use ChatGpt to let it do their work like discussed in the latest live forum event.
I'm also not good at market analysis. At most 10% of my ideas ever make it to a real product and probybly only 1% is really successful. But I find it better to take risks and do what I like and the way I like it rather than doing boring work.
Back on topic... After fiddeling around a bit with the Sync Manager configurations I finally can switch to Pre-Op state.
There is a reason why progress takes time! If you come up with a simple solution, you will see resistance. By those who believe that they master the complexity. What they actually don't. On the other hand, people trust those, that promote simple solutions for problems that are complex in the core! So don't hesitate to walk on the Gracey lane ;-)
I forgot where, but I once heard someone say smth to the extent of "Telling a programmer that there's already a library for that is like telling a songwriter that there's already songs about love"