Marco on CEP

As the ruleCore CEP Server is available as a service you really don’t have to care about any details of how things work. But as we techies are curious by nature I get regularly questions about the internals of ruleCore.

So here’s the first ruleCore Internals short article…

If we look at the ruleCore architecture we find two distinct components. The Event I/O front-end and the back-end of rule processing engines.

Event I/O: When an inbound events is sent into ruleCore it hits a layer of event transport protocol adapters. We use the Mule ESB in order to avoid spending time in implementing every possible transport protocol ourselves. Mule gives us 30+ protocols so it should be enough for most cases. In the illustration there is just some of the most interesting protocols mentioned. Apart from the obvious ones like raw TCP/IP sockets and web service there’s a cool one called XMPP, which is the protocol used by Jabber and Google Talk. It allows you to use your IM client to talk to ruleCore. Cool eh!

the transport protocol layer can also include features such as throttling and accounting or other thing that you like to do as close to the client as possible.

We use most protocols in a slightly special way. As ruleCore is event driven and in the best of worlds everyone would like to send events using JMS, WebSphere MQ or other messaging products, but they don’t so we have to deal with synchronous transport protocols. We have a asynchronous feeling to all transport protocols. For example if you send an event using a WebService you will get a "ok" as return and the only argument to the call is a string containin the event in XML. So even the synchronous protocols are used in a asynchronous fashion. Results are sent back to the client using outbound events (not shown in the illustration above)

As the inbound stream of events is generated by totally unsynchronized sources we feed these, after a security check, into a Sequencer.

Sequencer The sequencer re-arranges and sorts events from multiple sources into one nicely ordered event stream. This allows the rules to operate on a event stream which have known properties. The most important one being that the events are ordered and that duplicates are removed. This allows the rule model to be much simpler.

The rules see only one single stream of events. Making like much much simple when writing rules. There simply no stream abstraction to care about. The rules just see a logical stream of events and its not interesting from what system they come from.

Sometimes we call this our Event Fusion layer. It fuses events from different sources and creates a structured stream out of them. Other functionality that can be put into this layer if to pre-filer or aggregate events.

You could for example do throttling by sending events to the engines each 10th second if the rules does not need more resolution. You could also aggregate events into a higher level event.

After sequencing the events are distributed to a number of back-end rule evaluation servers. This is done using a very simple (internal) protocol which uses IP to send of the events to each engine. You need ofcourse be careful here to which engine you send events to that you don’t mess upp the rule evaluation.

Complex Event Processing