Property-based testing : evaluating its applicability and effectiveness for AUTOSAR basic software

Abstract

Previous work has shown that Property-based Testing (PBT) can be successfully applied to testing synchronous software. For example, it has been demonstrated that PBT can be applied to testing cloud services, web services and telecoms software. But less research has been carried out to evaluate this approach to testing asynchronous code as in automotive software. In the work presented in this Master thesis, the data generation feature of PBT is exploited to test the functionality of a software module based on the AUTOSAR Adaptive Platform. Properties are defined considering the system as a black-box targeting its functionality on an abstract level. First, we apply stateless properties to test a single functionality and thereby find the communication delay needed to incorporate in our testing at system level. Later, we implement a test infrastructure based on stateful properties using the Python tool Hypothesis for the demonstration of research based on PBT. The testing framework is interfaced with the runtime environment to integrate the former with the system being tested. The test inputs generated in this approach are evaluated for their effectiveness and efficiency in testing the software module under test. Finally, experts in the testing field have been interviewed to draw comparisons between PBT and traditional methods of testing.