EC-Inspector: Troubleshooting, Diagnosis and Monitoring Tool for EtherCAT® Networks

EC-Inspector is a Windows application for finding faults in EtherCAT networks, checking their health and watching what they do. It works independently of the EtherCAT MainDevice and with controllers from many manufacturers – Beckhoff, Omron, Bosch-Rexroth, Lenze, acontis and others – and it can either listen to a running network through a TAP device or operate the network itself when the machine is stopped.

Use cases: Troubleshooting, diagnosis, monitoring

EC-Inspector is used for three different jobs, and it is worth separating them, because each one asks a different question and each one is served by a different operation mode.

Troubleshooting is the investigation of communication issues. Are Ethernet frames lost? Where are they lost? How can it be fixed? This is the situation where a machine stops for no apparent reason and the controller reports nothing useful.

Diagnosis is the routine check: Is the network healthy, are all SubDevices in the state they should be in, and can that be documented? A report proving error-free communication after commissioning belongs here.

Monitoring is the continuous observation of a running machine – the velocity of a servo drive over time, all the parameters that are written during operation. Recorded over weeks, this is the raw material for predictive maintenance.

Because EC-Inspector needs no integration with the controller and no changes to existing software, all three are available on new installations (greenfield) as well as on machines that have been running for years (brownfield).

Operation modes

EC-Inspector opens with a start page that asks what you would like to do and answers with four pictures. Each mode is drawn as the system architecture it represents, so the right one can be recognised rather than looked up.

The start page: Choose the operation mode, then supply only the parameters that mode needs.

In Active as MainDevice, EC-Inspector operates the network itself using the integrated EC-Master MainDevice software. The standard controller is disconnected and the network is connected directly to the PC. No TAP device and no ENI file are required.

In Passive via TAP, the machine keeps running under its own controller. A TAP device copies all EtherCAT frames to EC-Inspector; it is not recognised by the controller and does not have to be configured.

Offline PCAP analyses previously captured EtherCAT traffic, including captures made with Wireshark. Offline Snapshot reads back a complete EC-Inspector session that was recorded earlier.

Troubleshooting: Locating communication faults

About 95 % of all network issues are cables and connectors. EtherCAT offers excellent diagnostic capabilities for exactly this class of fault, but many EtherCAT controllers do not evaluate that information, or do not show it. EC-Inspector does.

Communication errors show up as frames that the MainDevice sends but does not receive back intact. Every SubDevice counts these events itself, separately for each port – invalid frames with CRC errors, RX errors from the physical transmission, and lost links. Because the counters belong to a port, the fault can be narrowed down to the connection between two specific devices.

The Self-Test Scan

With the machine stopped and the controller disconnected, EC-Inspector takes over the network and loads it to 100 % capacity, so that even sporadic faults appear within a short time. It then evaluates the error counters of every SubDevice. The SubDevices are held in INIT throughout, so no output can be activated.

Active as MainDevice: No TAP device and no controller are needed to find a bad cable.

The result is a picture of the network rather than a verdict. The topology view shows every SubDevice port coloured by the quality of its connection, and the message window logs each error with time stamp, SubDevice, port and error type.

After the scan: Topology view, per-port error counters and messages in one window.

Warning icons mark the devices at both ends of the faulty link in the topology tree, its diagnostics view separates state machine errors from the physical condition of each port, and a message states in plain words what to do – naming both devices whose link is affected.

More about the Self-Test Scan and the start page

Passive via TAP: Investigating a running machine

When a machine must not be stopped, the TAP device provides all EtherCAT frames to EC-Inspector while the standard controller keeps operating the network. EC-Inspector decodes the traffic, counts lost and incoming frames and evaluates the error counters read from the SubDevice registers.

Passive via TAP: The machine keeps running while EC-Inspector listens.

Diagnosis: Checking the health of a running network

Diagnosis and monitoring both work in the mode Passive via TAP. The TAP device provides all EtherCAT frames to EC-Inspector while the standard controller operates the network, and the ENI file supplies the configuration EC-Inspector needs to decode them – or the Learning Mode creates it.

Diagnosis and monitoring: The ENI file configures EC-Inspector, the TAP supplies the frames.

Warnings and errors, with instructions

While the machine is producing, EC-Inspector evaluates every frame and the error counters of every SubDevice, and reports problems at the place where they occur. Devices with warnings or errors are marked in the Project Explorer, and the message window logs each event with time stamp, SubDevice and type – invalid frames with CRC errors, lost links and so on.

A SubDevice with an error on one port and a warning on the other, and the instructions EC-Inspector derives from them.

Selecting a marked device shows its diagnostics port by port, at two levels of severity. In the example, port 0 of SubDevice 1003 has a bad connection, reported as an error, and port 1 a disturbed connection, reported as a warning; the summary reads “multiple warnings and errors”. EC-Inspector turns this into instructions – check the connection between SubDevice 1003 and 1002, and between 1003 and 1004 – one instruction per affected link. The device is still in Op while all of this is reported. A connection that is degrading becomes visible while the machine keeps running, not only after it has stopped.

SubDevice states at a glance

The colour of each SubDevice in the Project Explorer shows its EtherCAT state: Green for Op, yellow for SafeOp, blue for Pre-Op and red for Init. A device that drops out of Op stands out immediately, without opening a single register.

SubDevice states by colour: Green Op, yellow SafeOp, blue Pre-Op, red Init.

Monitoring: Process data, triggers and parameter changes

Monitoring uses the same setup as diagnosis: The machine runs under its own controller, and EC-Inspector listens through the TAP.

One of the core functions of EC-Inspector is to display process data – inputs and outputs – in a live chart. All symbols are available with their data types and offsets, the list can be filtered per SubDevice, and the chart shows the last 500 values of the selected variable over time. The Break button freezes the chart so that a moment can be examined in detail, and the variables that matter can be collected in a watch list for faster access.

Process variables with data type, offset and value, and the live chart of the selected input.

The chart can also be stopped automatically by trigger conditions. A trigger compares an input or output variable with a fixed value or with another variable, using a choice of comparison operators, and several triggers can be defined at once. Once a trigger has been hit, everything else – the other variables, the SubDevice states, the error counters – can be examined at that exact moment. A trigger does not have to stop the chart: It can simply count how often its condition occurs, and it can be linked to a PCAP recording or to a saved backtrace.

Beyond cyclic process data, EC-Inspector also traces the parameters written to the SubDevices during operation. The CoE object dictionary shows the latest value of each parameter, and the mailbox protocol log keeps the complete history: Every CoE download with time stamp, SubDevice, index, value and result. Which parameters were changed, with which value, at which time – and were those changes expected and correct? Rejected writes appear in the log as well, for example a value outside the permitted range of a parameter.

Latest values in the object dictionary, the complete log of all CoE downloads below – including two rejected writes.

Offline analysis: PCAP files and snapshots

EC-Inspector can monitor live data and write the complete traffic to a PCAP file in parallel. That file can be loaded again afterwards and evaluated cycle by cycle; PCAP files recorded by Wireshark can be investigated the same way.

Left: Live diagnosis while the complete traffic is written to a PCAP file. Right: The captured file is read back and analysed offline.

The operation resembles an oscilloscope whose input signals are taken from the file: A process variable is selected and EC-Inspector shows how the signal developed over time. Playback runs at adjustable speed or in single steps, and trigger and search functions stop processing at a specific situation – when a variable reaches a value, or when a SubDevice leaves the operational state.

A snapshot takes a different approach: Instead of raw traffic it stores the complete diagnosis state in a single file. Snapshots can be analysed in greater detail later on, emailed to a colleague or an expert for a second opinion, and used as a status report and proof of quality, since each one records the state of a network at a given point in time.

Advantages

  • Independent from EtherCAT MainDevice software and hardware
  • No integration with the existing controller and no changes to existing software required
  • For troubleshooting the physical layer, no TAP device and no controller are needed at all
  • Passive and offline modes have no impact on timing or load of the existing network; in active mode the SubDevices are set to INIT only, so outputs cannot be activated
  • No specific TAP device required for passive operation, even a regular switch may be used
  • Very small engineering effort: Use the existing ENI file, or let EC-Inspector learn it
  • One tool for machines operated by controllers from different manufacturers (Beckhoff, Bosch-Rexroth, Omron, Yaskawa, etc.)
  • User-friendly: No EtherCAT expertise required
  • Suitable for new (greenfield) and existing (brownfield) installations
  • No installation of a Windows packet capture library or driver (WinPcap / Npcap) required

Recommended TAP (Test Access Point) devices

Supported MainDevice (master) software and controllers

  • Controllers using acontis EC-Master: ACS, Bosch Rexroth, Lenze, Mitsubishi, Omron, OSAI, Keyence, Yaskawa, etc.
  • Beckhoff TwinCAT
  • Codesys
  • SOEM (Simple Open EtherCAT Master)
  • IgH Etherlab EtherCAT Master

New features by version

More information

Developer Center: User manuals, tutorials and more. Data sheets and evaluation versions are available in the download area.