Low latency Linux for industrial embedded systems – Part I

Welcome to this mini blog series on the low latency Linux kernel for industrial embedded systems! 

The real-time patch, which is not fully upstream yet, has had many developers wonder about stable alternatives for their projects adopting an embedded Linux operating system (OS) with latency requirements in the milliseconds’ range. The low-latency Ubuntu Linux kernel from Canonical is less costly to maintain than real-time alternatives. PREEMPT_RT is an intrusive patchset that may not be compatible with all required drivers and may require debugging/reworking, whereas low latency is a configuration flavour of mainline.

The low latency Ubuntu kernel has the maximum preemption currently available in mainline (PREEMPT), coupled with four times the timer granularity of the generic Ubuntu kernel (HZ_1000 for low latency vs HZ_250 for generic). 

If this sentence is crystal-clear to you and if you are familiar with the concepts, you may want to jump ahead to the remaining two blogs of his three-part blog series. Part I of the series is for those at the beginning of their learning journey. Here, we will provide a basic introduction to preemptable processes in multiuser systems, and memory segregation into kernel and user space. Building on this knowledge, we will tackle preemption and frequency of the timer interrupt in Part II.

Finally, Part III of this three-part blog series will put everything together and delve into the considerations behind adopting low-latency Ubuntu for your embedded applications.

We have a lot to cover, so let’s get started.

Industrial embedded systems 

While executing, the programs of an industrial embedded system contend for resources like the central processing unit (CPU). Whereas some industrial embedded systems comprise high-end CPUs and large memory footprints, many are resource-constrained and have low-powered CPUs. It follows having hardware-enforced protection of system resources (CPU, memory, I/O) is particularly relevant for industrial embedded Linux applications.

But let’s not get ahead of ourselves and get back to the basics.

The Linux kernel adopts design and architectural considerations from the Unix family of kernels. Here, we are not concerned with the internals of Unix but need to mention some of its features for a proper understanding of the low-latency Ubuntu kernel.

Unix is a multiuser system with preemptable processes enforcing hardware protection of system resources. As such, several processes, or instances of a program in execution, can occur concurrently and contend for those resources, the CPU being one of them.

The processes of a multiuser system must be preemptable in that the operating system tracks how long each of those holds the CPU and periodically activates the scheduler. On the other hand, only one process uses the CPU on uniprocessor systems, with only one execution flow progressing at a time. Programs in Unix kernels execute in two states: the User Mode and the Kernel Mode.

Kernel space vs user space

As system memory in Linux segregates into kernel space and user space, it follows the CPU of an embedded Linux OS can run in either User Mode or Kernel Mode. The CPU cannot directly access the kernel extensions and data structures when application software executes in userland (or user space) because of hierarchical protection domains (also referred to as rings). 

The access-protected kernel space is the memory area reserved for executing the core of the OS (Ring 0 on the x86 architecture), whereas user space is the system memory, where user processes execute (Ring 3 in Linux x86 protection rings lingo).

A program usually executes in the User Mode in userland and activates a kernel routine only when requesting one of its services. Once the Linux kernel has serviced the request, it then resumes execution in the user address space. Processes switch to Kernel Mode when, e.g., issuing system calls or due to devices invoking an interrupt signal. 

At the time of Linux 1.x, when an instance of a program issued a system call requiring to switch to the kernel address space, no other task could be scheduled to execute until the kernel serviced the original request. Why was that the case? Because Linux introduced kernel preemption only with version 2.5.4.

Preemption plays a central role in the latency-throughput tradeoff. When vendors consider adopting a real-time OS they have already assessed latency is more important than throughput. On the other hand, the low-latency Ubuntu Linux kernel enables a well-balanced solution reducing overhead while maintaining responsiveness.

By shipping a preemptable kernel, Linux enabled higher-priority processes to force interrupt lower-priority ones already executing in kernel space, despite their requests not being serviced yet.

What are the preemption models available in the Linux kernel? And which config option does the low latency Ubuntu kernel adopt? Dive straight into Part II of this mini-series to find out!

Conclusion

The low-latency Ubuntu Linux kernel efficiently services most real-world low-latency, low-jitter workloads, achieved via the maximum preemption available in mainline coupled with a rate of the timer interrupt handler higher than in a generic kernel.

The meaning of these concepts will be better understood in the next part of this three-part blog series. In this part, we aimed at a quick introductory overview on the processes of a multiuser system and memory segregation into kernel and user space.

Part II of the series will address preemption in Linux and the timer interrupt frequency of the low latency Ubuntu kernel. See you there!

Further reading

Why is Linux the OS of choice for embedded systems? Find out with the ultimate guide to Linux for embedded applications.

Interested in a detailed comparison of Yocto and Ubuntu Core? Watch the Yocto or Ubuntu Core for your embedded Linux project? webinar.