**2. State-of-the-Art**

The simulation of perception sensors has been a part of worldwide research in recent years. Vehicles equipped with Automated Driving or Advanced Driver Assistance functions perform their tasks based on information provided by sensors that sense the environment, such as different types of radar sensors, front-, rear-, surround-view and night-vision cameras, LIDAR sensors, ultrasonic sensors [13]. The reliability of the warning and intervention hardware components and the algorithms that control them is strongly influenced by the quality of the information provided by the perception sensors. One of the goals of virtual testing is to model the behaviour of these sensors to match their behaviour under real environmental conditions.

In the simplest case, it is sufficient for the sensor model to give a perfect representation of the environment, using information artificially generated from the scenario, such as the relative distance, relative velocity, angular position, classification etc., taking into account the sensor's sensing properties, such as the maximum sensing range, horizontal and vertical coverage (FOV). This so called geometric or ideal sensor model is well suited to test the dynamic behaviour of complex systems at the whole vehicle level [14].

A more realistic modelling approach would be to implement physical sensor models simulating the behaviour of the complex interaction between sensors and the environment, such as the reflectivity, sight obstruction by other objects, multipath fluctuation, interference, damping etc.; however, the computational time and effort involved make the use of such models for vehicle testing impractical [15].

However, some sensor phenomena need to be modelled to test the performance of ADAS/AD systems with higher complexity. To test such systems under sub-optimal sensing conditions, sensor models are needed that reflect typical sensor phenomena based on the results of field tests, such as reduced range or increased noise levels in bad weather conditions, erroneous, missing or incomplete information on some sensed parameters, tracking errors, loss of objects etc. [16]. Models describing the phenomenological effects of different sensing technologies under similar conditions allow the performance and fault tolerance of the whole system chain to be investigated, taking into account an appropriate balance between the fidelity of the simulation, the complexity of the parameter settings and the computational power.

#### *2.1. Classification of Virtual Sensor Models*

Depending on the use case for virtual sensors, a general classification of sensor modelling approaches can be given. Using the basic simulation methodology, a model can be classified as a High-, Mid- or Low-Fidelity sensor model [10] or as black-, gray- or white-box [11]. As Peng in [17] and Schaermann in [18] denoted, there are three different modelling approaches for active perception sensors. These are the deterministic, the statistical and the field propagation approach; see Table 1. The deterministic modelling approach is based on mathematical formulations, which are represented by a multitude of parameters. If a sufficiently large amount of data is collected, the parameters can be trained and ideal sensor behaviour can be mapped.

The statistical approach is based on statistical distribution functions. This model is also called phenomenological sensor model and the realisations are drawn from a distribution function, which has to be determined before. This model architecture represents a good trade-off between computational effort and realism. This paper will focus on the validation of the third modelling approach, the field-propagation models. These models are simulating the propagation using Maxwell's equations. As these equations only can be solved for few geometries, a numerical approximation must be used.

In the case of a radar sensor, the propagation of the wave can be approximated with the Finite-Difference Time-Domain (FDTD). In order to achieve a real-time capable simulation, FDTD is too expensive in terms of computational power. However, other methods to approximate Maxwell's equations include the ray-optical approaches [17], simulating the propagation of electromagnetic waves by optical rays. Ray-tracing methods, like the published approach in [19], can model electromagnetic waves with various physical effects. For every radar transmitter–receiver pair, the wave propagation can be analysed to output the range and Doppler frequency for every detected target.

A major disadvantage of models based only on optics theory is that the effect of scattering is not included. Therefore, a scattering model must be implemented in the simulation. This can be either a stochastic scattering approach or a micro-facet-based scattering model as shown in [20]. By using these field-propagation models in the simulation environment, unprocessed environment sensor data with physical attributes are generated. This data is on a low-level in the signal processing chain, since it has not ye<sup>t</sup> undergone any object detection or fusion algorithm.



#### *2.2. Assessment Methods of Virtual Sensors*

Virtual testing, for all its potential benefits, has limited fidelity because of its generalisation to the real environment. Sensor models represent reality only roughly or by focusing on a specific property of a sensor type. Therefore, it is necessary to assess whether a sensor model can sufficiently represent the real world to validate the safety of ADAS/AD systems. In the literature, several methods have been published on verification and validation methods for perception–sensor models [21–23]. All these approaches have their advantages and disadvantages and specific areas of application.

Depending on the general validation strategy and the accuracy of the available perceptual sensor models, different goals can be achieved with the virtual ADAS/AD validation. Therefore, it is difficult to quantify the performance of the different techniques since no accepted methodology to assess the performance of sensor models in the automotive spectrum exists today. One simple approach to assess the accuracy of a virtual sensor model can be to compare the performance of the ADAS/AD function within the virtual environment with its real world performance, running the same driving scenario [8]. This approach assumes that, for the assessment, models describing the ADAS/AD function under test are available in addition to sensor models of sufficient fidelity and that at least one prototype of the real hardware is available for real-world testing.

Since our hardware resources are limited to open-interface sensors freely available on the market, our sensor modelling efforts are designed accordingly. The present research introduces a novel approach that we call *Dynamic Ground Truth—Sensor Model Validation* (DGT-SMV) for performance assessment of perception–sensor models. The method is based on a statistical comparison of simulated and measured low-level radar data and aims to provide a quantifiable evaluation of the low-level radar-sensor model used. The method is presented in the next section.
