Using complex event processing to gather information from infrastructure

This work aims to present a new approach on data centers monitoring. We make use of Complex Event Processing (CEP) to provide both efficient data visualization and significant threats and opportunities patterns detection. Our architecture defines a data bus that receives events from several data sources. Configuration items like servers, routers or applications send events to the data bus, while the CEP engine processes and correlates all these data.

At runtime, sysadmins can easily express patterns like "warn my team when session count falls more than 20% on a load balancer in less then one minute and cache fails on connecting to backend" using Event Processing Language (EPL), as well as create real time and historical charts. Once a pattern is detected, the system may take the corresponding actions to warn who are interested.

We have applied the solution in iG, one of the biggest brazilian ISP's and content providers. We have created 18 different patterns to prevent and detect several infrastructure and application usual problems.