Different Types of CEP
As a part of an interesting discussion in our plans to take over the world^H^H^H^H^H^H to understand what our clients think is difficult with CEP I ended up in doing a improvised presentation on what EDA and CEP is good at.
I ended up with the following bullets which I made up along the way:
Detect when a complex state change happens.
Here I tried to do an analogy to a more traditional RDBMS system. How would you detect that a number of tables have been changed over time in a very specific way? I hoped that this would give a good mental model on what CEP is trying to achieve.
In database terms, it rather messy to write the SQL and/or procedural code to detect changes to multiple tables. All agreed that things get even more messy when there’s timing constraints involved. Suggestions ranged from triggers to flashback queries but even the hardcore DBAs and database coders agreed that they would rather avoid looking at these type of queries after a couple of years – "What the hell did I do here??"
Drive calculations
Another good example of what can be done with CEP is to drive online calculations. Rapidly changing data fed to an algorithm which produces a constant stream of output values. I showed some example of Coral8, StreamBase to get an overall understanding of how these could be used in the capital markets. As no one worked with anything even remotely related to the world of finance we sortof had to invent use-cases.
Instead driving calculation of business metrics was agreed to be one of the major areas for this type of processing as these can be found in more or less all companies. There’s always a dashboard which needs to be as close to real-time as possible. For example "Tempo of production during the latest 13 minutes."
Event warehouse
A database full of events can be used to do some really interesting analytics. If we have long sequences of events which can be related to each other using different kinds of correlation rules we can find lots of interesting things. Here I tried to make people understand that there’s nothing magic about CEP. If you know how to do data mining you can apply the same techniques to events. Events are just a special type of data from this point of view. Just call the events for data structures of tuples (which actually some CEP vendors do) and you can pretend that you are processing normal data.
Simulation, what-if?
If everything is driven by events then we can save the stream of events a reply it. This is like being able to tweak the history and do it all over again to understand how we should have done in order to avoid certain situations. We can also replay history and try to build rules which detect new types of situations.
Control systems
High performance processing of sensor data is something that has been done for ages. Google for SCADA and you’ll find tons of systems doing that. The ‘problem’ is that all these systems are custom built from ground up. There has been missing critical infrastructure software to build upon. When building a traditional dataprocessing system one normally starts with a database and perhaps an application server. These kind of system components have been missing for real-time processing of data. Within CEP, there’s now a category of CEP software which is suitable for this.
The response to this "CEP thing" was mixed. A couple of persons realized the value of CEP at once and a large portion remained very sceptical. Which partly validates my theory that CEP is not for everyone (no it will not replace databases even though some CEP vendors say (said) so). A good subset of business problems are suited for CEP and that’s enough to keep at last a couple of CEP vendors running.
Recent Comments