On Event Models
It might not come as a surprise that the definition of an event is central to any type of event processing. Call it Complex Event Processing (CEP) , Event Stream Processing (ESP) or just Event Processing, but the concepts are the same.
In some products, like Coral8 and StreamBase, they actually started with the notion of data and after a while re-named the tuples to events. So the notion of an event seems to be a bit flexible. Different vendors have different ideas on what an event is. Our ruleCore started from the other end, starting with an strict 'everything is event approach' with a more strict definition of an event and using events as the native interface for everything. Other products seems to end up somewhre in between. I don't think either of there approaches are wrong. They just reflect two different types of event processing.
In order to better understand all these different event models it's great to have some possible features that a conceptual event might have.
The event model used by an event processing system defines to great extent the feel of the system. Small details in the event model have a huge impact on the expressive power and capabilities of an event processor. Just compare StreamBase with ruleCore and you'll see two type of products even though both are marketed as CEP, and they are both CEP!
An event model consists normally of:
- Event semantics
- Event syntax
Let's break up the text with a spider:
In addition to headers and other meta-data an event normally contains some payload. It's the data which tells us more about the event. If the event type tells us what happened, the payload tells the details about the event that occurred.
This data can be in different forms. Key/value pairs, structured or unstructured. Popular event encodings such as XML enforces some restrictions on the payload. Using a binary event encoding opens up for all kinds of creative content in the event body; Pictures, video or other types of rich content can easily be added into the body of a binary event. Although many architects and designers favor keeping their events small.
So, what are the fundamental base properties of an event? That is, what kind of properties can we expect to find in most events?
We need to consider both the syntax and semantics of the properties. Syntax is actually the easy part. Semantics can be a bit harder to define.
We have some prime candidates for event core properties
- Event Id
- Event Type
- Detection timestamp
- Entity
- Probability
- Location
- Occurrence time stamp
- Event class or category
- Caused by.
Event Example
Lets look at some practical examples. Here's an event encoded in XML (actually in the ruleCore markup language, but the concept is the same independent of the exact format. Feel free to add others in the comment field):
<ProximityAlert id="bd79d85a-12c1-4a1f-9047-3dc34fcab2cc" detection_time="2011-06-06 08:29:22">
<Header>
<CausedBy>39493849384938984930</CausedBy>
</Header>
<Body>
<EntityId type="semitrailer">AB-123</EntityId>
<Location>-23.3495 34.4955</Location>
</Body>
</ProximityAlert>
I’m using XML here as an example, but it could as well be something else… In this example we have an event of type ProximityAlert, it was detected just a couple of minutes ago and its globally unique id is bd79d85a-12c1-4a1f-9047-3dc34fcab2cc.
As we can see I envision the event to contains two containers, one if the header and the other is the body.
The header contains information about the event itself, for example which events it depends on, security related information, information about the event sending system and other meta data type of information.
The body is the actual payload of the event.
So what’s the difference then between ruleCore events and Streambase events? I don’t get the point here.
The main difference would be that StreamBase events are modelled more to fit into a SQL world. So they are more like rows in a database. This provides a good fit into their SQL based approach. The ruleCore event would not fit at all into a SQL based approach but fits nicely into the rule based approach used. So I'm not sure if there's a point to get, its just two variants of event models designed to fit into their respective processing model.
The biggest difference seems to be in the semantics. The StreamBase events are a bit more flexible where the user can assign his own semantics to the event and the ruleCore event semantics are a bit more fixed.
Hey Marco, thanks for your quick answer. I am, however, still a bit confused about things here:
Maybe I’m just not understanding it correctly, but it seems to me that you are mixing up two things: Event representation and query language (accessing events).
In my world, events are generally to be seen as attribute-value pairs. Whether they are queried by an SQL based language such as CQL or by specifying restrictions on attributes similar to content-based publish/subscribe system does not seem to make any difference here. For the CEP system the way a user issues a query should not make any difference, it is just a matter of representation which one should be able to translate into the respective other.
The interesting part is, whether we have other implicit restrictions here. E.g. if you speak of SQL world events, do you mean that there has to be a static schema that is to be known in advance? Then, however, I don’t get the part of this approach being more flexible, because I’d suppose it would be the other way round. Or what are the other differences concerning the processing models of the two engines?
Kind regards
Marco
Well, my intention was not at all to mix up events and the languages used to process them…
attribute-value pair type of events are just one type of events. There are more complex semantics that might be required and more complex data structures needed for the payload of data.