This article is based on the book Open Source SOA scheduled to print January 2009. This article is courtesy of Manning Publications. The ebook is available and sold exclusively through Manning Publications.
The open source community includes many early advocates of the recent wave of emerging SOA-related technology projects. Historically, however, open source has sometimes been considered a "late follower," with commercial products first to hit the market, and then followed by "me-too" open source alternatives. One reason frequently cited by critics of open source is that open source projects are often not innovators, but imitators (of course, some might argue Microsoft has done very well by following the imitation model). There may be some truth to that, but many of the products we'll be examining are very innovative and cutting-edge. In some instances, the reason development has lagged vis-à-vis commercial offerings is simply because of resource challenges - open source projects are often supported and staffed by a very small team of developers, many of whom have full-time responsibilities elsewhere.
Overall, it did take some time before a comprehensive collection of open source projects achieved sufficient breath and maturity as to offer a compelling alternative to the highly priced commercial alternatives. Now, there are many choices for crafting an entirely open source SOA environment. This excerpt forms the basis for the remainder of the book, as it identifies the open source products that we will be examining in greater detail. The selected products will form the basis for our OpenSOA Platform, and I will illustrate how these products can be integrated together in a coherent fashion so that, combined, they will equal or surpass in value the offerings by the commercial SOA vendors. Figure 1 recaps the technologies involved in the OpenSOA Platform and highlights (in double-width lines) those that will be investigated moving forward (as you recall, JMS, Application Servers and GUIs are covered thoroughly by other publications or are fairly commoditized in functionality).
Over the past 5 years I personally have had the opportunity to participate in "real-life" projects that have used many of the open source products discussed here. In choosing which ones will constitute our OpenSOA Platform, I pick "winners" from the competing selection of products. This is not intended to suggest those that are not selected are any less worthy. As with any evaluation process, the final choice is based on some combination of objective and subjective facts (obviously, we all want to believe we only use objective facts, but human nature often dictates against such logic).
Before we dive into each of the technology categories and the open source possibilities within each of them, we'll first try to set to establish some general, universal criteria that we can use when evaluating any of the products.
Open Source Evaluation Criteria
Some general criteria exist for examining all of the technology products that constitute the OpenSOA Platform. They are described in Table 1.
Now that we have identified the general evaluation criteria that we can apply toward evaluating any of the technologies that constitute the OpenSOA Platform, we will now look individually into each technology category and identify for each an open source product that we will use moving forward. In this process, we will identify competing open source solutions and address: 1) the criteria used for evaluating the products within a category; and 2) the justification for why a given product was chosen. We will start with Business Process Management, otherwise popularly known as BPM.
Business Process Management (BPM)
BPM refers to software that can be used to model and execute workflow processes (Figure 2 recaps BPM's role in the OpenSOA Platform).
BPM can be considered another form of application development, albeit a more visual process. Design and early development can often be performed by subject matter experts instead of hardcore developers (that said, the latter is often still required, at least in later stages of the development cycle). BPM is considered part of the SOA equation because it directly benefits, and is enabled by, the exposing of reusable services. With BPM, you can create business processes that span across multiple, previously stovepiped, applications. In this sense, BPM applications are often fundamentally different than conventional ones, and are less focused on performing a specific task and more oriented toward an entire business process. For example, a new hire process within a company may involve setting up the individual in a multitude of different systems, from benefits and payroll to registering them in a 401k system. A BPM process models that entire workflow and is not isolated to just populating one of the applications with data.
What Is a "Stovepiped" Application?
A "stovepiped" application is considered one in which, by its design, is completely isolated and self-contained. Legacy applications, which were often developed with little notion of integrating with external data or systems, are often considered "stovepiped." SOA tools provide the ability to unlock the business rules and operations of these stovepiped applications into services that can be invoked externally. Existing investments can be leveraged without having to resort to replacing critical business systems.
It is perhaps worthwhile to distinguish between some of the terms used in BPM, as the terminology can sometimes be rather confusing:
- A workflow is generally understood as series of human and/or automated tasks that are performed to produce some desired outcome.
- Closely related to workflow is a process. It is defined as "a set of activities and transactions that an organization conducts on a regular basis in order to achieve its objectives...It can exist within a single department, run throughout the entire enterprise, or extend across the whole value chain" [BPMBasics]. A process may involve one or more workflows.
- A task represents a specific work item that must be performed, most typically by a user. Tasks constitute the work within the workflow.
- A node is a generic command or step within a process. It can be a task, a wait state, a decision, etc. A business process is comprised of nodes.
- A transition (or, in XPDL nomenclature, edge) defines how nodes are connected.
BPM systems, by their nature, involve modeling what can be very complex workflows, or orchestrations. Mathematical algorithms are often used as the basis for implementation, and can be fairly arcane to understand for those not steeped in its principles. The requirement to visually model workflows also represents a significant development challenge. These are perhaps the reasons why open source BPM solutions were, at first, slow to emerge. Recently that has changed, and there are now several excellent open source BPM systems from which you can choose. We will discuss how to choose between them in the next section.
What's the Difference Between BPM and BPEL?
BPEL (or Business Process Execution Language) can be considered a subset of BPM. BPEL provides a semantically rich language for creating business processes that are composed of SOAP-based web services. It is a specification for how to materialize a business process that is composed of SOAP-based web services. BPEL, by itself, has no specific provisions for human activity-based tasks or queues (though the emerging BPEL4People will address some of these deficiencies), which are typically associated with workflow-based BPM processes. The BPEL standard also does not specifically address reporting, analysis, or monitoring, though some BPEL vendors have augmented their offerings to include such features. In other words, BPM represents a product offering whereas BPEL is a standard focused on web service orchestrations.
BPM Product Evaluation Criteria
As you recall, we discussed general criteria for evaluating open source SOA software. There are obviously some additional BPM-specific criteria that we'll want to consider. They are listed in Table 2.
Obviously, this only scratches the surface of the underlying functionality typical in any BPM solution. However, it does touch on some of the most important features, and provides us guidance on identifying what really constitutes a BPM so that we can identify possible open source products, which is the next topic.
Open Source BPM Products
As I pointed out previously, BPM solutions tend to be fairly complex in nature. This is both because of the visual modeling requirements and the complex workflow algorithms that drive the BPM engine. Fortunately, within the past few years, we've seen exciting developments in the open source community surrounding BPM, and there are now several excellent products to choose from. Table 3 lists the most active BPM open source products available today.
While the overview in Table 3 does not delve deeply into the feature sets of each available solution, the criteria we established does point to Apache ODE, JBoss JBPM, or Bonita as the most appealing of the solutions. We'll address the reasons for this next.
Selecting a BPM Solution
For several of the products listed, licensing issues were a major consideration in why they were excluded from further consideration. In the case of Intalio, their entire product is not open source. With several others the open source releases are more of a teaser to encourage upgrading to a commercial product (Shark/JaWe, ActiveBPEL). While Apache ODE can be fairly easily extended, it doesn't come with any built-in support for human-interface tasks, which are an essential part of a BPM (that's not a part of the core BPEL standard). Also, given that it is a BPEL execution engine, it is limited to working with SOAP-based web services, and cannot, for example, directly invoke a Java class or populate a JMS message (granted, you could extend it to support this, but then it's no longer truly supporting the BPEL standard). For these reasons, ODE was not selected as the BPM product.
• • •
This article is based on the book Open Source SOA scheduled to print January 2009. This article is courtesy of Manning Publications. The ebook is available and sold exclusively through Manning Publications.
ObjectWeb's Bonita offers a very attractive open source solution. It has a proven heritage dating back to its 1.0 release, and with the release of version 2, added support for XPDL. Unfortunately, Bonita does not come with a XPDL editor. Instead, they suggest using one of the available open source or commercial editors. This raises a concern, as the open source XPDL editors do not appear to be sufficiently robust (at least compared with their commercial alternatives). An additional concern is the newer version's reliance on the JOnAS application server. This will limit the ability to embed the engine within other applications. Because of these reasons, Bonita was not considered moving forward.
This leaves JBoss jBPM - it is a simple-to-use, but very powerful, workflow engine. As mentioned, jPDL is the XML vocabulary used to express business processes, and they can be created visually using the jPDL Designer, an Eclipse-based plugin. jBPM has the financial backing of JBoss and enjoys a fairly vibrant user community, based on forum and mail list activity. It also is being extended to support those who want to use BPEL scripts for workflow execution (at its core, it's really a graph-based process engine). For these reasons, it was selected as the BPM solution for our SOA technology stack. Let's take a more in-depth look at jBPM.
Introducing JBoss jBPM
The jBPM project's first official release was in early 2004, followed by the 2.0 release in late fall of that year. At approximately the same time, the jBPM team merged with JBoss, officially making jBPM a part of the JBoss family of products. Since the merger, the product has been managed and led by largely the same team, which has resulted in a solid, robust, and time-tested product. Some have expressed concern about its use of the LGPL license, which stipulates that enhancements/changes to the product must be shared with the community.
According to JBoss, this was done to encourage the building of a jBPM "community," and to prevent the splitting off of the project into proprietary versions. Given the widespread popularity of the LGPL license, this does not seem problematic, at least in my opinion. If the intention is to take the code and build a new commercial proprietary product based on it, yes, that would be prohibited. However, the vast majority of users would not be impacted by the LGPL restrictions, and would likely only benefit from them.
JBoss describes jBPM as "a flexible, extensible framework for process languages," or alternatively as a "platform for graph-based languages." The jBPM Process Definition Language, or JPDL, was the first, or "native" process language developed on this framework. More recently, they have also introduced a BPEL-based version of jBPM that is now in general release. While the BPEL release shows great promise as a standards-compliant BPEL engine, for the reasons cited earlier, BPEL is not necessarily well suited, in its current state, for BPM (again, this may change as the standard continues to evolve). Instead, our focus will be on the standard jBPM, JPDL-based release, which as of this writing had reached a milestone version of 3.2.
jBPM comes with the JPDL Eclipse plug-in Designer for easily creating business processes, and a web application pageflow framework for creating human-based tasks. It supports persistence of process instances by storing them within nearly any open source or commercial database (uses the well-respected Hibernate object-relational DB mapping framework). Any process variables used during the duration of a process instance are stored in the database, along with the actors who performed the work. These are important considerations in a world of SOX compliance requirements.
Enterprise Decision Management
Enterprise Decision Management (EDS), otherwise less glamorously known as a Business Rule Management Systems (BRMS), is an approach to automating and improving the decisions a business makes on a day-to-day basis. It plays an important role in our OpenSOA platform, as it provides the underlying business rules/logic that are used by many of the other components of the platform. The role of EDM in the OpenSOA Platform is shown in Figure 3.
Fundamentally, an EDM means extracting those decisions and rules that are today embedded into applications or people and systematically exposing them as true assets that can be centrally managed and authored. Some have gone so far as to proclaim a "Business Rule Revolution" is underway, insofar as it "represents an emerging undeniable need for the right people to know what a business's rules are, to be able to change those rules on demand according to changing objectives, to be accountable for those rules, and to predict, as closely as possible, the impact of rule changes on the business, its customers, its partners, its competition, and its regulators" [VonHalleGoldberg].
The value of managing business rules in a centralized fashion, and making them maintainable by business users instead of developers, has long been recognized as a laudable goal. Unfortunately, tapping into those rules from external applications and processes was often a considerable challenge. Early BPM vendors had their own proprietary API, often in one or two supported languages. This made integrating the business rules difficult and ensured vendor lock-in. The advent of web services and SOA opened up a vast new opportunity for incorporating a BRMS. Since web services are designed as language and platform neutral, centralized business rules can now be accessed by virtually any application. Further, composite applications, such as business processes designed using a BPM, can easily tap into a BRMS for decision and content-based routing rules. Perhaps the hyperbole of a "Business Rules Revolution" isn't such an exaggeration after all. In this case, the foundations of SOA become an enabling force to this exciting, even enterprise-changing, technology.
• • •
This article is based on the book Open Source SOA scheduled to print January 2009. This article is courtesy of Manning Publications. The ebook is available and sold exclusively through Manning Publications.
What's the Difference Between BRMS and EDS?
EDS, besides sounding a bit sexier and less boring than BRMS, is also considered to be a superset of BRMS. By that, it also includes leveraging analytical models that can be derived from data warehouse or business intelligence capabilities to conceivably create self-tuning rule sets. The reason we chose EDS for this book was that EDS appears to be becoming the more recognized acronym for rule-based systems. Consider it similar to how workflow slowly became subsumed by the more glitzy-sounding business process management (after all, workflow does sound pretty dry).
Figure 4 depicts the main components of an EDS.
In Figure 4, we see a repository of rules broadly categorized according to the types of rules they are, such as "Constraint Rules," which serves, for instance, to impose limits such as the maximum amount of credit to extend to a customer. These rules constitute the rule repository, which obviously has a central role in a rules system. The Rule Engine component, sometimes referred to as the "inference" or "execution" engine, represents the algorithms and logic that define how the engine works. The Rete algorithm, for instance, is a popular pattern-matching algorithm that is in most commercial and open source rule systems. The API/Web Service layer is how you interface with the system. Many EDS' include multiple language-specific libraries/APIs, and often a SOAP- or REST-based web service interface. The Authoring IDE is the tool for writing, editing, testing, and categorizing rules. An important aspect of the authoring environment is whether support for "domain-specific languages," or DSLs, is available. This refers to the ability to express rules in a language that is natural to the business user yet has rigorous semantics. Consider it analogous to building your own programming language using a business vocabulary (hence, it's sometimes referred to as "Language Oriented Programming"). The External Apps are those applications that are utilizing the rules engine.
What is the role of EDS in SOA? One of the principal tenets of SOA is designing systems that are flexible and agile. Rules engines are instrumental in advancing this concept, as they allow business rules to be change independent of making application modifications. This effectively eliminates having to go through drawn-out development and testing cycles. This obviously is also a form of loose coupling (another tenet of SOA), as the binding between an application and its business rules are no longer as tightly intertwined. The next section will delve more deeply into the criteria used for evaluating an EDM offering.
EDM Product Evaluation Criteria
I had identified some general criteria for evaluating the various SOA technologies, and an EDS obviously has some additional product-specific requirements. Table 4 identifies some key requirements we'll use when analyzing the suitability of the various open source rule systems.
Now that we have a foundation for assessing an EDM, we can now turn toward identifying the possible open source EDM candidates.
Open Source EDM Products
While commercial business rule solutions have been around for a decade or more, it's only been within the past five years or so that open source alternatives have become available. This is no doubt because of the increased visibility that has become associated with the "business rule approach," and the success stories presented by the commercial vendors. Table 5 identifies the open source EDM products.
Based upon the results in Table 5, it appears as though the only two real choices are OpenLexicon and JBoss Rules (hereafter referred to as "Drools", its historical name). Let's examine the reasons next.
Selecting an EDM
Mandarax, while maintained by a fairly small team of developers, does offer some very innovative features. They include an elegant way of tapping into working memory from a variety of data sources and a novel API approach to creating functions and predicate-style clauses using standard Java classes. Documentation is adequate. The biggest concern with Mandarax is that it's maintained by a very small team and appears to have a very limited use base. The concern is that, over time, without a strong user base, the project could fall into quiescence, and no longer be actively maintained (a fate that afflicts the majority of open source projects). For this reason, Mandarax was not considered.
Both OpenRules and Jess were excluded from consideration due to their licensing restrictions. OpenRules, while proclaiming itself as "open source," does not fit our criteria of open source. Namely, to use it in certain commercial capacities requires purchasing a license. While I am an advocate of purchasing support for those open source applications that have a sponsoring company whose revenue model is based on that (such as JBoss), I think its disingenuous to pitch a product as "open source" when a license must be purchased for commercial use. On the other hand, Jess clearly doesn't aim to mislead and does not position itself as open source (free versions for certain types of usage are available).
OpenLexicon shows great long-term promise, but the fact remains that it is still relatively new and lacks comprehensive documentation. Its nicely integrated BRMS features and well-designed user interface should definitely place it on anyone's open source short list. This leaves Drools, which has a long and proven track-record, and now has been enhanced with more enterprise BRMS features, such as repository management.
Introducing JBoss Rules (Drools)
The Drools project began in 2001, and the first production-ready release was the 2.x version that appeared in 2003. By 2005, Drools had become a very popular open source rules engine, so much so that in October of that year, it joined the JBoss family of products. With the deeper pockets afforded by JBoss (and then RedHat, which, in turn, acquired JBoss in 2006), the 3.0 release of Drools offered some significant enhancements by way of performance and introduced an authoring/IDE Eclipse environment. In addition, a new rule language, DRL, simplified rule creation. Even more substantial improvements accompanied the 4.0 release. The rule language was enhanced; a hibernate framework introduced for populating working memory; performance was further improved; and, perhaps most significantly, BRMS functionality was added. Drools can now claim to be a true feature-rich alternative to commercial BRMS offerings.
The Drools team at JBoss now includes over 12 full-time staffers, along with a fairly large contingent of non-JBoss contributors. The project has excellent documentation, which can be somewhat of a rarity in the open source world. The mailing list is also quite active. If there is a knock against Drools, it is that there is not currently a pre-built web services interface.
• • •
This article is based on the book Open Source SOA scheduled to print January 2009. This article is courtesy of Manning Publications. The ebook is available and sold exclusively through Manning Publications.