Shop OBEX P1 Docs P2 Docs Learn Events
EtherCAT experiments - Page 2 — Parallax Forums

EtherCAT experiments

2»

Comments

  • evanhevanh Posts: 15,043

    Does that mean that everyone using libraries isn't a programmer?

  • Of course not. According to the hacker's code of honour no intelligent person who is able and willing to be creative should ever be forced to do boring work for solving problems that others already solved. That's what libraries are meant for. It's completely OK and recommended to use them if they do what they should and don't bloat the code more than necessary. I only meant to blame those programmers who don't really know how to program but just call functions of libraries other's created. That's still OK, or to use Ada's analogy, there are bands that only play cover songs. Some of them are really good. But there is software that is ridicously bloated or inefficient because the programmers know nothing about algorithms and use big libraries because they are lazy.

    The setup software for Yaskawa VFDs are an example. The tool needs >4GB to download. After installing it it takes minutes to load and then tells you to download another couple of GBs of updates. And all that only to set some numerical values over USB.

  • OK, I think I begin to understand the mailbox protocol. It's really sick. The CanOpen protocol used to be designed for the CAN bus, of course, which always suffered from bandwidth drought. They squeezed the complete header (type, size and some status bits) into one single byte! Depending on the command type the size field has more or fewer bits. They re-used some of the command types depending on if its a request or response. So you always have to watch out what it means depending on the communication progress and context.

    But after I've corrected a few mistakes I finally get some results. The watchdog cuts the connection and resets the state of the state machine to Init if it times out. But it doesn't clear the SyncManager registers. They are write protected once the Sync Manager is activated. So I can't write to them in all but the first session if I don't clear the "activated" bit before writing.

    Datagram i=12 c=5 a=03e8:0120 s=2 l=14
    send frame...
    frame received
    Datagram i=12 dg=14 wc=1
    success!
    Datagram i=13 c=4 a=03e8:0130 s=6 l=18
    send frame...
    frame received
    Datagram i=13 dg=18 wc=1
    success!
    request state=12 actual state=02 code=0000
    Datagram i=14 c=5 a=03e8:1000 s=32 l=44
    send frame...
    frame received
    Datagram i=14 dg=44 wc=1
    success!
    Datagram i=15 c=5 a=03e8:107f s=1 l=13
    send frame...
    frame received
    Datagram i=15 dg=13 wc=1
    success!
    Datagram i=16 c=4 a=03e8:1200 s=32 l=44
    send frame...
    frame received
    Datagram i=16 dg=44 wc=1
    success!
    Datagram i=17 c=4 a=03e8:12ff s=1 l=13
    send frame...
    frame received
    Datagram i=17 dg=13 wc=1
    success!
    SDO res=43 data=66668888
    

    Address a=03e8:1000 is the outgoing mailbox and a=03e8:1200 the ingoing mailbox. The final result indicates that I have read $66668888 from service data object index $1018 subindex $01 which is the vendor ID of JMC (servo manufacturer). B)
    Oh man, I need a beer.

  • MicksterMickster Posts: 2,581

    @ManAtWork said:

    The setup software for Yaskawa VFDs are an example. The tool needs >4GB to download. After installing it it takes minutes to load and then tells you to download another couple of GBs of updates. And all that only to set some numerical values over USB.

    Except that it won't connect or you get a bunch of nonsensical errors. Supplier offers to take control of your laptop via TeamViewer and that starts a war with "the IT guy" who doesn't want to provide access and when you finally get it, the WiFi signal is too weak to support a TeamViewer session :D

    Craig

  • Some slow progress: I can also write SDOs, now. But sometimes the mailbox buffers get messed up and the communication stalls.

    The mailbox mechanism is actually quite clever. The ingoing and outgoing mailboxes are buffers in dual ported RAM which can be accessed both from the network side and from the slave microcontroller. Access is automatically interlocked. When the last byte is written from the network side the buffer is transferred to the slave controller side and further access by the master is blocked. When the slave has read the last byte the buffer is transferred to the master side again and writes are re-enabled. The same mechanism is implemented in the opposite side just with read and write access reversed.

    Each datagram has an index counter that is incremented for each communication cycle. So theoretically lost packets could be detected automatically by missing numbers. But because it's unclear if the original (write) data packet or the response (read) was lost the master could accidentally repeat a packet that was actually written to the mailbox. Only the returning packet was lost. But the false repeat could also be detected by the index (same number). But automatic error handling would require some interconnection beween the write and read mailbox logic that was never implemented. So if something goes wrong the interlock mechanism that should originally help does actually make things even worse. You have to set a special "repeat request bit" in a register or clear the so called Sync Manager (buffer interlock handler) manually and start all over, again.

    I know, I'm complaining too much... I was warned :#

  • MicksterMickster Posts: 2,581

    @ManAtWork
    What's really frustrating when undertaking such developments is not having someone to bounce ideas around with. So thus far, you haven't needed anything that is proprietary to Beckhoff?

    Craig

  • evanhevanh Posts: 15,043
    edited 2024-02-05 00:08

    @ManAtWork said:
    The ingoing and outgoing mailboxes are buffers in dual ported RAM which can be accessed both from the network side and from the slave microcontroller. Access is automatically interlocked. When the last byte is written from the network side the buffer is transferred to the slave controller side and further access by the master is blocked. When the slave has read the last byte the buffer is transferred to the master side again and writes are re-enabled. The same mechanism is implemented in the opposite side just with read and write access reversed.

    Is that automatic interlock applied to the whole dual-ported RAM and thereby only handling one packet at a time or does it provide multiple independent locks for concurrent packets allotted across multiple buffers at once?

  • @Mickster said:
    @ManAtWork
    What's really frustrating when undertaking such developments is not having someone to bounce ideas around with.

    Yes, my comments here are probably not of much real use to anyone. But sometimes I need to discuss the problems with myself. To formulate and write down my thoughts helps to clear my mind and possibly find solutions I haven't thought of earlier.

    So thus far, you haven't needed anything that is proprietary to Beckhoff?

    I haven't used any part of their source code if you mean that. Of course I have needed the protocol specifications to look up the theory and algorithms. BTW, I've found out that the servo drive manual from SanyoDenki explains much better how everything works. They provide a step-by-step procedure with a flow chart of how everything has to be setup for operation. The "conspiracy" theory behind that is obvious: Beckhoff wants to sell their training courses and consulting. So they don't want you to be able to figure out everything on your own. SanyoDenki wants to sell their drives. If you get them operational faster than those of the competitors it's to their advantage.

    @evanh said:
    Is that automatic interlock applied to the whole dual-ported RAM and thereby only handling one packet at a time or does it provide multiple independent locks for concurrent packets allotted across multiple buffers at once?

    There are multiple Sync Managers (interlock units) per slave and each controls only a subrange of the available RAM. Two are used for the ingoing and outgoing mailbox for the SDO communication. Another two are used for the PDO communication. SDOs (service data objects) are used for one-time initialization and are not time critical (for example control loop gain). PDOs (process data objects) are used for cyclic transmission of real-time process data (for example target position and actual torque). Multiple PDOs can be mapped to the PDO buffer memory.

    So only one SDO can be handled at a time per slave. But that's no problem because the SDOs are not time critical. The communication mechanism for the PDOs is very efficient, though. Up to 4 times 12 PDOs can be mapped per slave and all PDOs of all slaves can be transmitted in a single Ethernet frame for both write and read direction (if they fit into 1.5kB, of course). So the whole initialization process is very complex. But once everything is set up and runs it is very efficient.

  • Until now I have processed all commands one-by-one meaning that every single read or write transaction was translated to an Ethernet packet with a single datagram in it. This was easier to debug but terribly inefficient, of course. The EtherCat master should be able to bundle multiple commands into a single packet and process them concurrently. So I changed the linear code to a state machine. There are multiple FIFO queues for new commands to be processed, those that are waiting for reply and those that timed out and need a retry. The state machine looks if there is something to do, builds the datagrams and sends them all in a single packet.

    I still need to debug this before proceeding to the next step: setting up the distributed clock system for synchronisation of all slave devices.

  • I have extended the Ethernet PHY driver to record timestamps for sending and receiving network packets. Each slave device also latches the local time when a packet arrives at any of its ports. Each packet is passed from the master to port0 of the first slave, processed internally by the EtherCat processing unit, sent out at port 1 to port 0 of the next slave and so on. If a port is closed (nothing connected) the packet is looped around to the next open port. The last slave has only one cable connected so any packet that arrives at port 0 is looped back and sent out at port 0 again to the previous slave. This continues until it is sent back to the master.

    As I only have one slave device at the moment I've closed the loop at port #1 with jumper wires.

    Now I can measure the timing of the packets travelling through the loop.

    TX Datagram i=19 c=4 a=03e8:0900 s=16 l=26
    send frame...
    RX Datagram i=19 dg=28 wc=1
    send ts=15f261a0 recv ts=15f2641a loop dt=3170ns
    slave #1 ports=5a37 rt0=423cbc64 rt1=423cbebc
    loop dt=600ns
    

    The total round trip takes 3170ns. This includes the smart pin shift registers of the P2, the delay of the Ethernet PHYs and the EtherCat processing in the slave. The "time of flight" in the last cables including the PHY delay of port #1 out & in is 600ns. If I add 2m (=4m forth and back) of cable I get 640ns.

  • It's been quiet for a while but that doesn't mean that nothing is done. I have decided not to rush for an ASAP "get the motor to turn" solution in free running mode instead of fully synchonized distributed clock mode. This does not only require to measure the delay of the packets travelling through the cables but also adjusting the clock speeds so that there is no long-term drift due to the tolerances of the individual crystals. To compensate the drift the time differences between the local and the refernece clock is measured. This is fed into a PID control loop of which the optput "tunes" each local clock by adding a small fraction (up to 100ppm) of the time passed to the clock. This works, now.

    We also had to do a new layout for the slave board (see post #1 right side) to change the P2 pinout and add an EEPROM for the configuration data.

    I discovered a strange thing: one ethernet packet is lost exactly once every 100ms. I haven't figured out what causes this, so far. I thought it could be a watchdog inside the EtherCat device because 100ms is the default timeout. But I've changed the watchdog clock divider and the 100ms stayed the same. And I don't think it's related to noise or EMI. We have 50Hz mains AC frequency so it should be much more likely to happen every 10 or 20ms. I also doubt that it's a bug of my Ethernet driver. No counter overflows at a 10Hz interval. If I send a packet every millisecond then the lost packet ratio is 1:100. If I send every 10ms it's 1:10.

    I need the slave boards to check if it's a bug in the chinese servo drives.

    Another piece of hard work shows some progress. I can parse the XML files, now, that contain the descriptions of the device properties like mailbox buffer sizes, timeouts and the SDO/PDO object dictionary. I have coded this all from scratch because I need full control so that only the relevant information is passed from the PC to the P2. Storing and parsing them on the P2 would require an SD card interface and lots of time. As I mentioned before the files are really big, several megabytes per device.

  • MicksterMickster Posts: 2,581

    Oh, we know what it means when you go quiet :D

    Your tenacity is commendable B)

    However, it's making me less and less a fan of such a ridiculously complicated network. I'll stick with RS422, 4-wire with a simple protocol (KISS principle) :)

    Craig

  • evanhevanh Posts: 15,043
    edited 2024-02-22 21:11

    Everything evolves messy. I've been shifting some Modbus on RS485 equipment over to ModbusTCP on Ethernet, and that offers 32-bit "registers" but only in a wacky ordering and addressing way. I'm guessing there's some sort of compatibility reasoning but am just going with the flow and not delving into the why's really.

  • MicksterMickster Posts: 2,581

    @evanh said:
    Everything evolves messy.

    It absolutely does and I don't understand why. Too many cooks, I guess.
    I am convinced that the industry giants are inventing complexity and convincing the market that they really need it.

    I lurk here and all I do is SMH. All of these problems don't need to be.

    Craig

  • Yes, that happens quite often. Same thing with "Eagle" (PCB layout editor). The acronym stands for "easy applicable graphical layout editor" and 20+ years ago it was exactly that, easy to use, compact and efficient. I think the original software was written in 386 assembler and was quite fast on VGA. Then Autodesk bought the company and now it's bloated and painful to use. The footprint libraries are so big that it takes several seconds to load even on a fast PC and you don't find anything. I still use the old version from 1995 which luckily still runs on Windows 10. The VGA stuff is now emulated which means that the screen update is actually slower on a modern 3GHz PC compared to a 12MHz XT it was designed for.

    The problem is that big companies always try to please everyone. This leads to every project passing the magical "one man knows it all" threshold. Once the "critical mass" is reached the complexity grows uncontrollably.

    Back to EtherCat... I still hope that the whole complexity is only required for configuration. All that megabytes of XML files and SDO object definitions only really matters for an expert who needs to figure out how each device needs to be configured, i.e. which value needs to be written to which register or "service object" to get the thing running. Once this is done everything gets easy and works very efficiently. It all comes down to two buffers, one for the inputs and one for the outputs. The first n bytes of the output buffers are mapped to the first device, the next m to the second and so on. The output buffer is sent out to the network cyclically and automatically and the input states are received and placed in the input buffer. It behaves like all the I/O was connected directly to the master and the buffers were just a bunch of hardware registers.

    Fortunatelly, all servo drives behave very similar and the layout of registers or PDOs (process data objects) is mostly the same. Example:
    object 0x607A is the target position,
    0x6064 is the actual position,
    0x6077 is the actual torque.
    Of course, every motor is different. The maximum current, torque per current factor, encoder resolution and so on are different. But that's not a problem. Torque for example can be displayed as a percentage of the nominal value so everything above 100% means overload. It's the same as every Boeing pilot can sit down in the cockpit of any Boeing plane and can tell from the engine diplay if the engine is performing like it is meant to without even knowing which engine type is actually installed. An N1 of 90+% means takeoff thrust and ~30% means idle no matter to what actual RPM that corresponds to.

    My plan is to provide a library of hardware abstraction files for "known good devices". They only need to contain a dozen or maybe 20 values. An expert who knows how the servo drives and the EtrherCat setup works can add new devices to the library but Joe random user doesn't need to. He can just plug in the cables and the software looks up in the data base if the connected devices are in the library. If they are then everything configures automatically. All the complex EtherCat protocol runs in the background and the user doesn't need to care about. At least that's the goal.

  • MicksterMickster Posts: 2,581

    @ManAtWork
    This is what I'm shooting for :+1:

    Servo drive. I don't care, old/new, any brand as long as it can be set to torque-mode and has an analogue command. All it needs to be is a transconductance device that commutates. My controller handles everything else.
    See these guys on the plant-floor, hooking-up to devices with a laptop, trying to remember how to use the setup software, what's the password, who has the proprietary 485 cable with the weird connector. :s

  • @Mickster said:
    See these guys on the plant-floor, hooking-up to devices with a laptop, trying to remember how to use the setup software, what's the password, who has the proprietary 485 cable with the weird connector. :s

    Yes exactly. My plan is to make any custom setup software and proprietary cables unnecessary. The cheap chinese drives offer EtherCat connectivity but still need a serial port and PC software to tune them. But the more user-friendly and higher priced drives from the big names (Sanyo Denki, Yaskawa, Panasonic...) have ALL tuning parameters accessible over EtherCat. This not only means that they can be tuned online with the same software that runs the machine. It also means that it's no longer required to "customize" each drive by storing parameters in it which limits their use to a particular axis in a particular machine if you don't know how to change the parameters (which overstrains most users).

    If all the parameters are stored in the master controller then if one of the drives breaks down I can ship an "empty" drive to the customer. He just pulls out the connectors, replaces the drive, plugs the cables back in, switch the machine on and it works, again. That's a BIG advantage and makes service and stock of replacement parts much easier.

  • MicksterMickster Posts: 2,581

    @ManAtWork said:

    If all the parameters are stored in the master controller then if one of the drives breaks down I can ship an "empty" drive to the customer. He just pulls out the connectors, replaces the drive, plugs the cables back in, switch the machine on and it works, again. That's a BIG advantage and makes service and stock of replacement parts much easier.

    :+1:B):+1:

  • AAArrrggghh! I have totally forgotten that I actually have a 100ms period in my Ethernet driver. It frequently polls the PHY status register to check the link status. This register is accessed over the SMI (serial management interface) which shares pins with the RMII. I still have to find out what the exact problem is but somehow those status polls interfere with the network data transfer and seem to cause the loss of one packet per 100ms.

    But after all, that bug wasn't all that bad. It has helped me finding at least three other bugs which only showed up due the packet loss. First, the repeat request of the mailbox protocol was not handled correctly. Second, a cyclic datagram was not re-scheduled if is was sent together with a non-cyclic datagram. The third bug was a rather stupid one:

            uint8_t match= task->number - dataHead->index;
            if (match < 0) EnqueueTask (&RetryTasks, task); // task older, frame was lost
            else if (match > 0) break; // task not found, abort
            else { // index match
    

    Of course, uint8_t is always positive so the function after the first 'if' was never executed. I deleted the 'u' and it works as it should, now. So I can pretend that the packet loss was there intentionally as a stress test of the robustness of the protocol. :Do:)

  • ErNaErNa Posts: 1,738
    edited 2024-02-27 16:15

    every dump guy at least can serve as a bad example! ;-) Everything is fine as long as you do not connect +25V to the Vcc ;-)
    Just to be more precise: I made an setup for an upcoming demonstration and to be sure, that all the wires are properly connected, I reordered them and in doing so, I connected the +24V to the wrong row, so as I switched on the power, the power switched of a lot of IC's! And now I look for the upside ;-)

  • Ok, the driver is fixed, no more lost frames. I hope I can focus on the real thing, now: get the motor running.

Sign In or Register to comment.