Measuring Real-Time Operating System Performance – Part I: Finding and Porting a Benchmark Suite
September 8, 2022 | 8:05 am
CDT
It is often necessary for certain operations to meet a specific deadline when working on real-time systems. Efficient operating system services, such as task switching or inter-task synchronization, play a crucial part in meeting these deadlines. Most of these services come at a cost: control must be transferred from the user tasks to the operating system and, therefore, latencies outside of our control may occur.
The first blog in this two-post series shows how benchmarks can be used to measure these latencies. This provides an estimate of the suitability of certain operating systems for a specific real-time application. The follow-up post measures the performance of two open-source, real-time operating systems: FreeRTOS and Zephyr.
Finding a Benchmark Suite
Before measuring anything, we need a benchmark suite fitted to our needs. Many real-time benchmark suites are concerned with typical real-time applications, such as video encoding. These suites provide benchmarks in the form of algorithms often used in these kinds of applications. Examining the services provided by the platform’s underlying operating system requires a different set of test cases. RTOSBench is a benchmark suite specifically designed for that purpose.
RTOSBench is the result of a master thesis from the University of Montréal. It includes test cases for services present in most operating system kernels: (cooperative) context switches, mutexes, semaphores, message queues and interrupts. Unfortunately, the original project seems to be no longer maintained and has smaller bugs in it. We forked the project to fix these bugs and made that version available here.
Porting RTOSBench to FreeRTOS and Zephyr
One key idea behind RTOSBench is portability, meaning it can be adapted to previously unsupported operating systems. This is done by encapsulating the respective kernel’s API inside a thin porting layer. To run RTOSBench on a new operating system, we must provide two things: a configuration header (named config.h) containing macros and data type definitions and an implementation of the porting layer’s function API (e.g., in zephyr_porting_layer.c).
Providing Type Definitions
To gain a better understanding of the porting process, let us implement semaphore support for FreeRTOS and Zephyr. First, we must define RTOSBench’s data type no_sem_t in such a way that it uses the respective operating system’s semaphore. FreeRTOS uses the data type SemaphoreHandle_t. Therefore, we have to add the following type definition to our config.h for FreeRTOS:
typedef SemaphoreHandle_T no_sem_t;
Zephyr’s semaphore data type is called struct k_sem, so we have to add the following type definition to the Zephyr port’s config.h:
typedef struct k_sem no_sem_t;
Implementing API Functions
After providing RTOSBench with a definition of no_sem_t, we can move on to implementing the semaphore API functions. Overall, we must implement three semaphore-related functions in our porting_layer.c: no_sem_create, no_sem_wait and no_sem_signal. The function no_sem_create() is used to initialize a semaphore sem with an initial value. Zephyr provides the same service via the function k_sem_init(). It takes a pointer to a semaphore and initializes it accordingly. We can implement no_sem_create() as follows:
void
no_sem_create(no_sem_t *sem, int value)
{
k_sem_init(sem, (unsigned int)value, K_SEM_MAX_LIMIT);
}
FreeRTOS does this is a bit differently. We can use the function xSemaphoreCreateCounting() to create a new semaphore. Unlike Zephyr’s k_sem_init(), this function does not receive a pointer to the semaphore, but allocates a semaphore from FreeRTOS’s heap memory and returns it. Therefore, the FreeRTOS implementation of no_sem_create() looks like this:
void
no_sem_create(no_sem_t *sem, int value)
{
/*
* The possible maximum value depends on your platform. On the
* example platform UBaseType_T is a typedef for unsigned long.
*/
*sem = xSemaphoreCreateCounting(ULONG_MAX, value);
/* Was the semaphore allocation successful? */
if (*sem == NULL) {
no_serial_write("sem_create: error");
Error_Handler();
}
}
The two remaining semaphore functions can be ported in the same manner and after proceeding in a similar way with all other kernel services, we should be able to execute RTOSBench on both operating systems. If you are interested in more details, you can find our Zephyr port on GitHub. Please note that it only supports ARM Cortex chipsets with a Nested Vector Interrupt Controller (NVIC).
Conclusion
Porting RTOSBench to Zephyr is a relatively simple process. Zephyr’s toolchain makes it easy to create new applications and its code base provides good abstractions of the target platform’s low-level details. Porting RTOSBench to FreeRTOS, however, is a bit more complicated. The main reason is that FreeRTOS is more minimalist and does neither provide a common hardware abstraction layer nor a toolchain like Zephyr’s. Therefore, hardware manufacturers must provide these. This means that parts of RTOSBench’s porting layer might have to be re-implemented for each platform that RTOSBench should be executed on when benchmarking FreeRTOS.