Performance testing in continuous integration environments

Abstract

The purpose of this case study is to evaluate how and which performance testing tools which can be used in continuous integration (CI) environments. By doing so, developers can see the effects of changes immediately and react against performance problems of their applications. This will help companies to eliminate performance issues which the media is reporting about more often every day. CI provides the reference platform for executing the performance tests and the performance testing tools provide metrics like response time and percentage of errors. These metrics can be combined through CI plugins. The results of this combination can be visualized in form of graphs and tables. Through this case study, we give a short market overview of current CI servers and performance testing tools. In respect of the requirements by adesso AG, we will only evaluate performance testing tools, which can be integrated into the Atlassian Bamboo or Jenkins CI. We evaluated six performance testing tools of which four were integratable into the CI servers. Based on the results of our evaluation we will give a recommendation.