Data parallelization in complex event processing without a dedicated splitter

Abstract

With the popularity of Internet of Things(IoT), Complex Event Processing (or CEP) shows its power in detecting specified patterns from input event stream. There are existing parallel CEP architectures to improve the capacity of CEP system. The major data parallel CEP architecture is the Split-Process-Merge architecture, which is able to provide unbounded parallelism degree. However, it has limitation when the splitting decision becomes computational heavy, which leads the splitter becoming a bottleneck. E.g. splitting decision depends on comparing two images to check if they contain the same object such as a person. The result is that the single splitter, instead of operator instances, is doing the computational expensive job. To help analyze the cause of "heavy" splitting decision, this thesis proposes an Extended SNOOP query language, which combines features from both SNOOP and TESLA, two of the leading event specification languages. Then this thesis derives an architecture, which avoids the splitting decision, from Split-Process-Merge architecture. The Split-Process-Merge architecture splits the input event stream into sub-streams and each operator instance handles one or more sub-streams. Instead, the new architecture creates Tasks by combining every incoming event to all existing Partial Matches, and operator instances process the Tasks. The Task Creation Algorithm is content independent. It won’t check the content, like the image data, in events. Therefore, the computational heavy splitting decision is avoided. Together with this thesis, an example implementation of new architecture for a specific query is given. The Evaluation results of implementation show the new architecture obtains a good scalability as number of CPU cores increasing and as the cost of operation increasing.