Validation of a machine learning model for certification using symbolic regression and a behaviour envelope

Abstract

Aviation is highly regulated with a strong focus on safety. If machine learning models are to be used in aviation, their correctness must be proven as part of the certification process. As the number of data points is limited in real-world applications, a new approach is needed to ensure that the behaviour between the test points is correct. Due to the complexity, it is unlikely that a method for a complete evaluation with a reasonable runtime will be found. It is demonstrated in this methodology study how, in addition to the data set, the expected behaviour of the system the model is designed for can be considered. Using domain knowledge, a specific “behaviour envelope” defines the area the model is expected to stay within. In case the model stays within the behaviour envelope, which can be mathematically evaluated, it can be ensured that the behaviour between the test points is always physically meaningful. Since the effort for the evaluation increases with the complexity, it is proposed to use symbolic regression, a method where a search procedure combines elementary functions to create a compact symbolic model. This shifts the effort more towards model creation and simplifies the subsequent validation.