Automated categorization of performance problem diagnosis results

Abstract

In times of big data and global networking, software performance becomes a major issue in software development. Many enterprise applications were not designed for the current highly frequented usage. Thus, more and more organizations are using Application Performance Monitoring tools, to detect bottlenecks and other performance problems in their product. Meanwhile, there are many different tools on the market which provide a detailed performance-aware analysis of enterprise applications. However the common Application Performance Management tools do not provide any additional automated performance problem diagnosis. The performance expert has to find the cause on its own. diagnoseIT [4], a framework for automatic diagnosis of performance problems in enterprise applications addresses this problem. diagnoseIT extracts performance problem instances from execution traces. The resulting set of problem instances can become huge and the analysis of the problems is very time-consuming. We extend the general concept of diagnoseIT and categorize the resulting set of problem instances into a manageable number of problem categories. Therefore, a concept of categorization is elaborated. We analyze several different categorization approaches and evaluate the performance and the quality of the result. We perform a sensitivity analysis, which analyzes the influence of each attribute of a problem instance on a clustering result. The results of the sensitivity analysis indicates, that there is a potential for optimization. Thus, we introduce a concept of optimization, which optimizes the process of categorization by weighting the attributes and we compare different manual and automatic optimization approaches with regard to the improvement compared to default weights. In the evaluation, we examine the accuracy and the performance of the approaches. The evaluation shows that k-means clustering provides the most promising and best results. Additionally, the evaluation indicates a high potential for optimization. However, the results of the evaluation show that it is difficult to optimize the weights without any knowledge about the analyzed system.