Model-based Testing of Real-Time Embedded Systems in the Automotive Domain

Zusammenfassung:

Software aspects of embedded systems are expected to have the greatest impact on industry, market and everyday life in the near future. This motivates the investigation of this field. Furthermore, the creation of consistent, reusable, and well-documented models becomes an important stage in the development of embedded systems. Design decisions that used to be made at the code level are increasingly made at a higher level of abstraction. The relevance of models and the efficiency of model-based development have been demonstrated for software engineering. A comparable approach is applicable to quality-assurance activities including testing. The concept of model-based testing is emerging in its application for embedded systems.

Nowadays, 44% of embedded system designs meet only 20% of functionality and performance expectations (as given by Helmerich et al., 2005). This is partially attributed to the lack of an appropriate test approach for functional validation and verification. Hence, the problem addressed by this innovation relates to quality-assurance processes at model level, when neither code nor hardware exists. A systematic, structured, and abstract test specification is in the primary focus of the innovation. In addition, automation of the test process is targeted as it can considerably cut the efforts and cost of development.   The main contribution of this thesis applies to the software built into embedded systems. In particular, it refers to the software models from which systems are built. An approach to functional black-box testing based on the system models by providing a test model is developed. It is contrasted with the currently applied test methods that form dedicated solutions, usually specialized in a concrete testing context. The test framework proposed herewith, is realized in the MATLAB®/Simulink®/Stateflow® environment and is called Model-in-the-Loop for Embedded System Test (MiLEST).   The developed signal-feature – oriented paradigm allows the abstract description of signals and their properties. It addresses the problem of missing reference signal flows as well as the issue of systematic test data selection. Numerous signal features are identified. Furthermore, predefined test patterns help build hierarchical test specifications, which enables a construction of the test specification along modular divide-and-conquer principles. The processing of both discrete and continuous signals is possible, so that the hybrid behavior of embedded systems can be addressed. The testing with MiLEST starts in the requirements phase and goes down to the test execution level. The essential steps in this test process are automated, such as the test data generation and test evaluation to name the most important.   Three case studies based on adaptive cruise control are presented. These examples correspond to component, component-in-the-loop, and integration level tests. Moreover, the quality of the test specification process, the test model, and the resulting test cases is investigated in depth. The resulting test quality metrics are applied during the test design and test execution phases so as to assess whether and how the proposed method is more effective than established techniques. A quality gain of at least 20% has been estimated.   The Approach   The starting point of the MiLEST approach is to design a test specification model. Since at the early stage of system development reference signals are not available, a new method for describing the required system under test (SUT) behavior is given. Here a signal feature (SigF) concept is applied. It is a formal description of certain predefined attributes of a signal. In other words, it is an identifiable, descriptive

property of a signal. It can be used to describe particular shapes of individual signals by providing means to address abstract characteristics of a signal. Giving some examples – step response characteristics, step, minimum etc. are considerable SigFs.   Graphical instances of SigFs are given in Figure 1. The signal presented on the diagram is fragmented in time according to its descriptive properties resulting in: decrease, constant, increase, local maximum, decrease, and response, respectively. This forms the backgrounds of the solution presented in this work.  

  Figure 1: A Descriptive Approach to Signal’s Feature.   A feature can be predicated by other features, logical connectives, or timing relations. These can be defined either between features within one signal or throughout more signals, e.g.: − within(A1)A2        − if SigF A1 occurs, SigF A2 occurs at least once at the time when SigF A1 is active − after(y ms)A&B − SigF A and SigF B occur together after y milliseconds − during(A)B        − if SigF A occurs, SigF B occurs continuously during the activation time of SigF A − A׀׀¬C                − SigF A or negated event C occur − A=v                   − a set of SigFs A (e.g., maximum) which values are equal to v.   Further on, generic test data patterns are retrieved automatically from the marked portions of the test specification. The test data generator concretizes the test data based on the classification tree method and aims at a systematic signal production. The SUT input partitions and boundaries are used to find the meaningful representatives. Additionally, the SUT outputs are considered too. Hence, instead of searching for a scenario that fulfills the test objective it is assumed that this has been already achieved by defining the test specification. Further on, the method enables to deploy a searching strategy for finding different variants of such scenarios and the time points when they should start/stop.     Contact: Justyna.Zander-Nowicka _at_ fokus.fraunhofer.de