Marco on CEP

Continuing my series of ruleCore Internals today It’s time to look at our development environment… So if you’re not a geek. This is not interesting to you.

Work here is done completely on Linux. Everything runs Linux. The servers on which the ruleCore ‘cloud’ is running are Linux and so are the developer’s workstations. We use Linux as it’s the best tool for developers. It gets the work done and allows us to be flexible as it’s easy to find tools for all parts of the development cycle.

But lately we have starting to see Mac as an alternative. There are a number of small problems with Linux which might lead us to Mac some day. We love tools that just work, so that we can spend more time creating ruleCore code. With Linux,  we can spend hours with seemingly simple tasks.. Like trying to figure out why printing suddenly stopped working? Mac OS X seems to be a bit more polished and "just works", so we’ll certainly keep it on our radar screen as an alternative for developing on. I would love to see see a more polished Linux distribution which works in a commercial software development setting where there’s always too little time to get the next set of features ready.

Under Linux we write code (obviously). Most of the code is written using Netbeans. Why not Eclipse? Our experience shows us that we have much less problems with Netbeans. We prefer to spend time coding and not fighting with our IDE. Basically we don’t need any advanced features from our IDE. Basic editing and debugging is enough.

For source code management we have our trusty subversion. Normally we check out and commit from within inside Netbeans. But sometimes the subversion plugin breaks down and we need to use the good’ol command line.That’s one of the strengths of Linux, you can most of the times track down problems and find workarounds. Compared to a Microsoft environment where finding problems can be totally impossible and you have to spend days with reinstalling everything.

Most of the code is written in Java and Python. With some additional Unix shell (bash) scripts to glue everything together. Both Java (offcourse) and Python is well supported in Netbeans. Java support just great but sometimes the Pyhon mode behaves strangely. That’s basically our biggest complaint against netbeans. It’s bad support for Python. The other thing that could improve is how to share projects among developers. It’s a pain to check in everything required to run a project. But overall netbeans is a good tools and gets the work done (Well, not in the same class as visual studio from Microsoft, but it’s enough for our needs…)

When coding in Java we use Glassfish as our app server and PostgreSQL as database. For GUI work we are in the framework hell using a clever combination of JSF, Seam, JPA and IceFaces. Java and all it’s frameworks are great in one way, but takes some time and skill to set up everything.

In the Python world we tend to use just plain Python, no frameworks or few other dependencies. So all Python coding feels very simple as there is not multiple frameworks to learn, just Python.

Almost forgot. There’s actually another database besides PostgreSQL used; BerkeleyDB. It’s used for internal purposes by the engine to provide the basis on which we build transactional crash recovery.

When everything is done, it’s deployed on Linux too. We don’t run those servers ourselves. There’s other guys that are awesome in building vmware clusters with all kind of features for scalability and uptime so we really don’t want to do that ourselves. So we just buy processing power and storage from these guys.

We also use the Mule ESB as a way to transport and distribute events. We don’t want to spend time in implementing transport protocols for event transport or spending time in creating integration infrastructure. Mule gives us that more or less free. We’ll not completely free. We had to extend Mule to read it’s configuration from a database (postgres) and do the configuration programmatically instead of using the standard way of XML. Mule comes with so many transport protocols so we can easily get events in and out of ruleCore to just about every system out there.

The event sequencer is running as a component inside Mule. It could run in an app server of as a stand-alone service but it fits nicely into Mule for now.

The most common question I get about this setup is why we use Python? Is not Python very slow? Well, it’s slower that Java most of the times. But sometimes developer productivity is more important than raw speed. My guess is that we do five times more in Python than in Java. With the event-driven architecture we have, we don’t base our performance on running fast on one single server anyway. Our scalability strategy is instead to add lots of cheap PCs. And sometimes a slow system is better than a non-existing one. Python allows us to create things that would be too expensive (taking too much time) otherwise.