You are here: Home / RTLWS 1999-2017 / RTLWS Submitted Papers / 
2022-07-04 - 04:41

Dates and Events:

OSADL Articles:

2022-01-13 12:00

Phase #3 of OSADL project on OPC UA PubSub over TSN successfully completed

Another important milestone on the way to interoperable Open Source real-time Ethernet has been reached

2021-02-09 12:00

Open Source OPC UA PubSub over TSN project phase #3 launched

Letter of Intent with call for participation is now available

2016-11-12 12:00

Raspberry Pi and real-time Linux

Let's have a look at the OSADL QA Farm data

2016-09-17 12:00

Preemption latency of real-time Linux systems

How to measure it – and how to fix it, if it's too high?

Real Time Linux Workshops

1999 - 2000 - 2001 - 2002 - 2003 - 2004 - 2005 - 2006 - 2007 - 2008 - 2009 - 2010 - 2011 - 2012 - 2013 - 2014 - 2015 - 2017

17th Real Time Linux Workshop, October 21 to 22, 2015 at the Virtual Vehicle Research Center, Graz, Austria

Announcement - Call for participation (ASCII) - Hotels - Directions - Agenda - Paper Abstracts - Presentations - Registration - Abstract Submission - Exhibitors and Sponsors - Gallery

Test Suite Coverage Measurement and Reporting for Testing an Operating System without Instrumentation

Hermann Felbinger, Virtual Vehicle Research Center
Joel Sherrill, OAR Corporation
Gedare Bloom, George Washington University
Franz Wotawa, Graz University of Technology

Measuring the coverage of a test suite provides common metrics to assess the quality of a test suite. In safety-critical applications as in the domains of avionics and automotive complete coverage is required for certification. Usual approaches to measure the coverage require instrumentation of the source code or the object code of the system under test to obtain processable execution traces. However, instrumentation might change the behavior of the system under test. In this paper we show an approach to measure the coverage of a test suite and to generate human-readable reports without instrumentation of the system under test. As a system under test we use an operating system. Our approach is based on the execution traces obtained from an instrumented QEMU CPU emulator. We use this emulator to execute the operating system and the test cases. From the execution of the test cases we obtain execution traces. We provide a framework to map these execution traces back to the source code and to generate a detailed report exposing execution and branching (taken/not taken) information at the assembly language level and high level language level.

To evaluate our approach we generate coverage reports for the RTEMS real time operating system. We provide detailed coverage results for RTEMS running on different CPUs in this paper. Coverage of a test suite can be used by operating system developers to assess test suite quality and guide test case creation. Our approach is due to the lack of instrumentation of source code and object code broadly applicable for development of embedded systems applications.