The quality of EtherCAT communication depends heavily on the accuracy of the EtherCAT cycle. A high amount of jitter in the cycle time has a negative impact on the process data transmission. For demanding EtherCAT applications with fast cycle times, the real-time requirements are even more important, and therefore on a multitasking operating system (OS) like Linux additional effort is necessary in order to improve the real-time behavior. For some simple use cases the standard “out-of-the-box” Linux environment is sufficient, but the majority of EtherCAT applications usually have more stringent real-time requirements. The most common and widely used solution to improve real-time behavior of a Linux based system is to apply a patch to the kernel known as the PREEMPT_RT patch. The PREEMPT_RT patch has been around for a long time, and until recently could be challenging to implement properly and required advanced knowledge of Linux kernels and operating systems in general. Fortunately, since April 2022 it is possible to use a prebuilt real-time Linux kernel with the PREEMPT_RT patch already applied thanks to the release of Ubuntu 22.04 LTS https://ubuntu.com/blog/real-time-linux-qa.

Here at acontis, we have already tested this new release of Ubuntu and the included real-time Linux kernel, and now we would like to share our results with you.

In order to install the real-time kernel on Ubuntu 22.04, just follow the instructions from Ubuntu's website: https://ubuntu.com/blog/real-time-ubuntu-released. Previously in beta, recently Canonical made the 22.04 release generally available: https://ubuntu.com/blog/real-time-ubuntu-is-now-generally-available. In order to enable the real-time kernel, two options are available; Ubuntu Server 22.04 LTS via Ubuntu Pro, and Ubuntu Core 22 with an App Store. A subscription to Ubuntu Pro is free for personal use, but for your specific needs we recommend you contact Canonical about your licensing options.

After attaching the machine to an Ubuntu Pro subscription, the real-time kernel can be enabled with one simple command:

pro enable realtime-kernel

For better real-time performance we highly recommend to set-up firmware and BIOS according to our recommendations in our Developer Center: https://developer.acontis.com/rt-faq/items/FAQ1013.html and https://developer.acontis.com/pcob.html.

The recommendations above give the basic setup and prepare the system.

Further optimizations can be done with kernel parameters. The kernel parameters provide a lot of ptions for fine-tuning the OS. A full list of the kernel command line parameters can be found here: https://www.kernel.org/doc/html/v5.15/admin-guide/kernel-parameters.html.

For our purposes we can focus just on a few of the kernel parameters:

1. 1. “Power and clocking management” group:
   • • cpuidle.off=1
     • intel_idle.max_cstate=0
     • cpufreq.off=1
     • processor.max_cstate=0
     • processor_idle.max_cstate=0
     • intel_pstate=disable
     • rcupdate.rcu_cpu_stall_suppress=1
     • idle=poll
2. 2. “Core isolation” group:
   • • nohz_full=3
     • isolcpus=3
     • rcu_nocbs=3
     • irqaffinity=0
3. 3. “Graphics Settings” group:
   • • i915.enable_rc6=0
     • i915.enable_dc=0
     • i915.disable_power_well=0
     • i915.enable_guc=2

The main idea with the kernel parameters is to combine and mix these parameters for the best results depending on the particular use cases.

• Hardware: Intel® Core™ i5-8279U CPU 2.40GHz, 16Gb RAM, Intel I225-V Ethernet controller
• Ubuntu 22.04 LTS with real-time kernel 5.15
• EC-Master 3.1.4 for Linux (free evaluation here: https://www.acontis.com/en/ethercat-support-eval-request.html)

We are going to measure the actual cycle time of the EcMasterDemo example application using the built-in performance measurement capability included with EC-Master. With this performance measurement turned on, the EcMasterDemo application periodically outputs the measured minimal, average, and maximum values for a variety of important measurements including the cycle time. An example excerpt from the output of this performance measurement feature is below. For our purposes, the critical value is the top row showing the overall cycle time. In the example output below the maximum cycle time value of 1053.5 microseconds indicates a worst-case jitter of 53.5 microseconds:

0001363481: ===========================================================
0001363481: PerfMsmt 'Cycle Time                                          ' (min/avg/max) [usec]: 948.3/1000.0/1053.5
0001363481: PerfMsmt 'Task Duration (JOB_Total + App)    ' (min/avg/max) [usec]:      7.4/    12.2/    77.0
0001363481: PerfMsmt 'JOB_Total                                            ' (min/avg/max) [usec]:      7.0/    11.4/    67.5
0001363481: PerfMsmt 'JOB_ProcessAllRxFrames Duration' (min/avg/max) [usec]:      1.3/      3.4/    46.2
0001363481: PerfMsmt 'JOB_SendAllCycFrames Duration   ' (min/avg/max) [usec]:      3.0/      3.9/    41.5
0001363481: PerfMsmt 'JOB_MasterTimer Duration             ' (min/avg/max) [usec]:      0.4/      1.5/    37.9
0001363481: PerfMsmt 'JOB_SendAcycFrames Duration      ' (min/avg/max) [usec]:      1.5/      2.4/    36.9
0001363481: PerfMsmt 'myAppWorkPd                                   ' (min/avg/max) [usec]:      0.0/      0.2/    27.9

In order to simulate a load on the system we will use the BurnInTest v5.0 for Linux (http://www.passmark.com/products/bit.htm) with the following settings:

Each test runs for at least 24 hours.

The application will be executed isolated on core 3, using command line switch ‘-a’ available from the example application:

./EcMasterDemo –a 3 …

Below is a summary of the results of some test runs in the table. As you can see, it is possible to improve the real-time behavior of a prebuilt Linux kernel using only kernel parameters.

Notes:
(*1) 2 hour test run
(*2) cpuidle.off=1 intel_idle.max_cstate=0 cpufreq.off=1 nohz_full=3 isolcpus=3
(*3) cpuidle.off=1 intel_idle.max_cstate=0 cpufreq.off=1 nohz_full=3 isolcpus=3 rcu_nocbs=3
(*4) cpuidle.off=1 intel_idle.max_cstate=0 cpufreq.off=1 processor_idle.max_cstate=0 processor.max_cstate=0 intel_pstate=disable nohz_full=3 isolcpus=3 rcu_nocbs=3 rcupdate.rcu_cpu_stall_suppress=1 i915.enable_rc6=0 i915.enable_dc=0 i915.disable_power_well=0 i915.enable_guc=2 nohz=off rcu_nocb_poll idle=poll irqaffinity=0