A methodology for validation of a radar simulation for virtual testing of autonomous driving

Abstract

Autonomous driving offers great potential for reducing the number of accidents as well as optimizing traffic flow. The safety validation of such an autonomous system is an extremely difficult problem and new approaches are needed because the conventional statistical safety proof based on field testing is not feasible. The combination of real-world and simulation-based tests is a promising approach to significantly reduce the validation effort of autonomous driving. As environment sensors such as lidar, camera, and radar are key technologies for a self-driving vehicle, they have to be validated to be able to rely on virtual tests using synthetically generated sensor data. In particular, radar has traditionally been one of the most complex sensor to model. Since a sensor simulation is an approximation of the real sensor, a discrepancy between real sensor measurements and synthetic data can be assumed. However, there exists no systematic and sound method for validating a sensor model, especially for radar models. Therefore, this work makes several contributions to address this problem with the objective to gain an understanding of the capabilities and limitations of sensor simulation for virtual testing of autonomous driving. Considering that high fidelity radar simulations face challenges regarding the required execution time, a sensitivity analysis approach is introduced with the goal to identify the sensor effects that has the biggest impact on a downstream sensor data processing algorithm. In this way, the modeling effort can be focused on the most important components in terms of fidelity, while minimizing the overall computation time required. Furthermore, a novel machine learning-based metric is proposed for evaluating the accuracy of synthetic radar data. By learning the latent features that distinguish real and simulated radar point clouds, it can be demonstrated that the developed metric outperforms conventional metrics in terms of its capability to measure characteristic differences. Additionally, after training, this removes the need for real radar measurements as a reference to evaluate the fidelity of a sensor simulation. Moreover, a multi-layered evaluation approach is developed to measure the gap between radar simulation and reality, consisting of an explicit and an implicit sensor model evaluation. The former directly assesses the realism of the simulated data, whereas the latter refers to an evaluation of a subsequent perception application. It can be shown that by introducing multiple levels of evaluation, the existing discrepancies can be revealed in detail and the sensor model fidelity can be accurately measured across different scenarios in a holistic manner.