Automatic splitting in data-parallel complex event processing systems

Abstract

Parallel Complex Event Processing (CEP) systems handle today’s heavy loaded event streams from smart homes, network traffic systems or stock trading systems by distributing the incoming event stream to several pattern detection systems. The correct splitting is currently done by CEP experts which ensure the consistent splitting without generating false-positive or false-negative complex events in comparison with centralized CEP systems. In this work an approach is developed which automatically generates a splitting model from the pattern definition which ensures the consistent distribution without generating false positives or false negatives. This approach enables a parallel CEP system to be configured and used the same way as a centralized CEP system. Further, a method which combines window based splitting and key based splitting is presented to reduce the network load and the CPU load on pattern detection operators. The functionality of the automatic splitting and the optimization is validated with common CEP scenarios based on generated and real world data to ensure a wide applicability of the approach.