SOA and the real Service Integration issue

A real problem considering a SOA project, either in its beginning or in extending an all ready existing one, is the Web Services (ws) hell, that is hiding in the corner. Beginning the SOA approach, especially when having limited time to deliver and / or pure business analysis results, defining and architecting a solution that is compliant with your client (both in the kick-off architecture and in the way of working – aka good adaptation of the new environment) is possibly to provide a sea of ws. From the other hand, extending or taking a project in an already existing SOA environment that was developed by someone else, you have a small daemon hiding in the corner that always play with you in order to make you build more and more new ws in order to build your components.

In any case, in the SOAs you are always face the problem of exponential increase of the ws pool. In order to avoid such a problem, you have to give effort in focusing on the standardization of the core architecture elements (imagine that in an existing case, developed by someone else, you have to understand and comply with the already taken way) and furthermore to establish an optimized integration approach for existing assets of the environment.

Having these in mind, you have either to use some existing ground of work to extent by, or to … re-invent the wheel.

Re-inventing the wheel, means that you are prepared to give much more effort in business analysis and re-engineering and furthermore to develop all the necessary tools and bridges, both for the interoperability elements of the infrastructure, as well as for the semantics and conceptual transformations that will take place in order to implement the needed physical (system) bridges.

Or using some existing assets…

Java Business Integration (JBI) is an effort focused on standardizing the core architecture elements of integration architectures. It is a specification developed under the Java Community Process (JCP) for an approach to implementing a Service Oriented Architecture (SOA). The JCP reference is JSR-208. JBI extends Java EE and Java SE with business integration service provider interfaces (SPIs). It enables the creation of a Java business integration environment for the creation of composite applications. It defines a standard runtime architecture for assembling integration components to enable a SOA in an enterprise information system.

Following the JBI road, you might be able to use another ally for the SOA stacks you gonna need.

Open ESB is an open source integration platform based on JBI technology. It implements an Enterprise Service Bus (ESB) using JBI as the foundation. This allows easy integration of Web Services to create loosely coupled, enterprise-level integration solution.

Open ESB Architecture

Because Open ESB is built on top of the JBI specification, it makes sense that, before diving into the architecture of Open ESB, we take a look at the JBI architecture, which is illustrated in Figure 1.

As the figure shows, JBI adopts a pluggable architecture. At the heart of a JBI runtime is a messaging infrastructure called Normalized Message Router (NMR) that is connected to a bunch of JBI components. JBI components are architectural building blocks of a JBI instance and are plugged into the NMR to interact with each other. The interaction among components carries out the functional logics of the JBI instance.

Normalized Message Router

The primary function of the NMR is to route normalized messages from one component to another. It enables and mediates the inter-component communication. When a component needs to interact with another component, the component does so by generating a normalized message and sending it to the NMR. The NMR will route the message to the destined component based on some routing rules. After the destined component gets and processes the message, it will generate a response message if required, and will send the response message to the NMR. It is the NMR's responsibility to deliver the response message back to the original component.

The NMR uses a WSDL-based messaging model to mediate the message exchanges between components. From the NMR's point-of-view, all JBI components are service providers and/or service consumers. The WSDL-based model defines operations as a message exchange between a service provider and a service consumer. A component can be a service provider, a service consumer, or both. Service consumers identify needed services by a WSDL service name rather than end-point address. This provides the necessary level of abstraction and decouples the consumer from the provider, allowing the NMR to select the appropriate service provider transparently to the consumer.

An instance of an end-to-end interaction between a service consumer and a service provider is referred to as a service invocation. JBI mandates four types of interactions: One-Way, Reliable One-Way, Request-Response, and Request Optional-Response. These interactions map to the message exchange patterns (MEPs) defined by WSDL 2.0 Predefined Extensions.

The NMR also supports various levels of quality of service for message delivery depending on application needs and the nature of the messages being delivered.Component - Service Engine and Binding Component
A JBI component is a collection of software artifacts that provide or consume Web Services. As mentioned previously, JBI components are plugged into the NMR to interact with other components. JBI defines two types of components, Service Engines (SEs) and Binding Components (BCs).

A Service Engine is a component that provides or consumes services locally within the JBI environment. Service Engines are business logic drivers of the JBI system. An XSLT Service Engine, for example, can provide data transformation services, while a BPEL Service Engine can execute a BPEL process to orchestrate services, or enable execution of long-lived business processes. A Service Engine can be a service provider, a service consumer, or both.

A Binding Component is used to send and receive messages via particular protocols and transports to systems that are external to the JBI environment. They serve to isolate the JBI environment from the particular protocol by providing normalization and de-normalization from and to the protocol-specific format, allowing the JBI environment to deal only with normalized messages.

Distinguishing between these two types of components is more functional. In fact, JBI uses only a flag to distinguish these two. The programming model and APIs of these two types are otherwise identical. However, by convention, Service Engines and Binding Components implement different functionality in JBI.

Service Unit and Service Assembly

JBI runtime hosts components and so acts as a container for components. Components in turn act as containers for Service Units (SUs). A Service Unit is a collection of component-specific configuration artifacts to be installed on SEs or BCs. One can also think of a Service Unit as a single deployment package destined for a single component. The content of a Service Unit are completely opaque to JBI, but transparent to the component it is deployed to. An SU contains a single JBI-defined descriptor file that defines the static services provided and consumed by the Service Unit.

Service Units are often grouped into an aggregated deployment file called a Service Assembly (SA). A Service Assembly includes a composite service deployment descriptor, detailing to which component each SU contained in the SA is to be deployed. A service assembly represents a composite service.

Lifecycle Management

JBI also defines a JMX-based infrastructure for lifecycle management, environmental inspection, administration, and reconfiguration to ensure a predictable environment for reliable operations.

Components interfere with JBI via two mechanisms: service provider interfaces (SPIs) and application program interfaces (APIs). SPIs are interfaces implemented by the binding or engine; APIs are interfaces exposed to bindings or engines by the framework. The contracts between framework and component define the obligations of both framework and JBI component to achieve particular functional goals within the JBI environment.

Unless you're doing component development, it's unlikely that you need to work with these SPIs and APIs directly.

Open ESB

Once we've understood the architecture of JBI, Open ESB becomes really simple. Open ESB is an implementation of the JBI specification. It extends the JBI specification by creating an ESB from multiple JBI instances. The instances are linked by a proxy-binding based on Java Message Service (JMS). This lets components in separate JBI instances interoperate in the same fashion as local ones (see Figure 2).

ESB administration is done by the Centralized Administration Server (CAS), a bus member that lets the administrator control the system directly.

Open ESB includes a variety of JBI components, such as the HTTP SOAP Binding Component, the Java EE Service Engine, and the BPEL Service Engine

 

A Sample Service Integration Scenario

In this section, we'll examine a simple use case and illustrate a possible service integration solution using Open ESB.

Problem Description

ABC Movie Theatres is a fast-growing movie theatre chain. To better serve its customers, the management has decided to put in place a new ticket booking service, the Booking Service, which will be responsible for handling most of the ticket purchasing requests generated from various systems such as Web-based applications, ticket vending machines and points of sale (POS) at box offices.

The business logic of handling a booking request is somewhat complicate. But in a nutshell, it involves the following steps:

1. When a booking request is received, the Booking Service will first try to process the ticket information. This could include checking for ticket availability; holding the tickets for the customer if tickets are indeed available or provide alternatives to the customer otherwise; applying any applicable promotions and calculating the total dollar amount
2. Then the Booking Service will charge the customer using the payment information included in the booking request
3. Finally the Booking Service will send a confirmation to the customer using the contact information included in the request.

High-Level Solution Description

Considering the fact that a significant amount of the logic needed by the new booking service has been implemented in different applications over time, and that these applications have been proved stable over time, it makes a lot sense to leverage these existing IT assets for cost efficiency. So the architect team at ABC comes up with the solution of service-enabling and -consolidating existing logics, bringing the services into the ESB and exposing them as new composite services that are made available via different protocols.

To do this, they have to identify candidates for service-enabling. Currently the company has a billing system developed and used by the finance department and it has been working perfectly over years. This system runs over the HTTP protocol. Another system is the notification system developed by the customer relationship department. This system listens to a JMS message queue and, when it gets a new message, sends out notifications to the customer by the means specified in the message, e.g., an e-mail or a voice mail message. These two systems become the ideal candidates for billing customers and sending confirmations.

The company also has a system for processing ticket orders. This system, however, is severely old and impossible to scale up to handle the ever-increasing transaction volume due to recent acquisitions. The architect team has decided it's time to write a replacement application; it's also decided that the replacement application, the Ticket System, will be built using EJB technologies due to the transactional nature of the application.

The architect team also decided that the new Booking Service will be made available via the SOAP-over-HTTP protocol as well as the File protocol to support different client systems.

Solution Details

When it comes to Open ESB development, it's all about creating JBI Service Units and packing them in a Composite Application (or Service Assembly, in JBI terms). Figure 3 illustrates the Service Units for this solution and the interactions among those units. As mentioned earlier, Service Units are deployed to their corresponding JBI components. For simplicity's sake, we'll use the term Service Unit and Component interchangeably provided it doesn't cause any confusion in the particular context.

At a high level, two BCs, the Booking Service SOAP BC and the Booking Service File BC, are created to enable the Booking Service and expose it to the outside world via the SOAP and File protocol respectively, allowing different kinds of clients to consume the service. A client application can invoke the Booking Service via either of these protocols. Invocation requests generated by client applications are then routed to the Booking Process, a BPEL Service Engine. The Booking Process orchestrates the Ticket Service, the Billing Service, and the Notification Service to fulfil the request.

File and SOAP BC - The Booking Service

In Open ESB, a File BC is a JBI binding component that binds file systems to WS-I Web Services. A File BC scans a pre-configured file location for new files. If a new file is found, the component generates a Web Service call using the content in the file as the payload of the Web Service input message. Response messages will also be written to the file system by the component. Many of the properties of a File BC, such as the file location, file name, and time interval for the component to scan the specified location for new file, can be configured.

A SOAP BC works the same way as the File BC, only instead of scanning a file system directory, a SOAP BC accepts WS-I SOAP messages over the HTTP protocol.

A File BC and a SOAP BC are created in this solution to enable the Booking Service and expose it to external systems via these protocols. External systems that wish to consume the service do so by either sending a WS-I-compliant message, or dropping the message in the specific file location.

External requests received by the previous two BCs will be routed, by the NMR, to the Booking Process, a BPEL Service Engine that executes a BPEL business process to orchestrate services.

BPEL Service Engine - The Booking Process

A BPEL Service Engine is a JBI runtime component that provides services for executing WS-BPEL-compliant business processes. The contract between a business process and partner services is described in WSDL.

A BPEL SE can save business process data to a persistent store if configured to do so. This is required for recovering from system failure and running long-lived processes. A BPEL SE can be deployed to a clustered environment to achieve high scalability. The service engine's clustering algorithm automatically distributes processing across multiple engines. When the business process is configured for clustering, the BPEL Service Engine's failover capabilities ensure throughput of running business process instances. When business process instances encounter an engine failure, any suspended instances are picked up by all available BPEL Service Engines.

The Booking Process in this solution is a BPEL SE and is at the heart of this solution. It does some simple message transformation and, most importantly, invokes the Ticket Service, Billing Service and the Notification Service. Figure 4 shows a simplified version of the Booking Service process.

Java EE Service Engine - The Ticket Service

A Java EE Service Engine brings Java EE components into the Open ESB runtime as Web Services. A Java EE Service Engine acts as a bridge between a Java EE application server and a JBI environment for Web Service providers and Web Service consumers deployed in the application server. Java EE Web components or EJB components that are packaged and deployed as Web Services on a Java EE container can be transparently exposed as service providers in JBI environment.

In this solution, the Ticket System is implemented using EJB Session Beans and wrapped as a JAX-WS Web Service. The service is then brought into the Open ESB runtime by the Ticket Service SE.

HTTP and JMS BC - The Billing Service & the Notification Service

Let's face it - all BCs work the same way. This is the beauty of Open ESB architecture. We've looked at two BCs, the File and the SOAP BC. Similarly an HTTP BC binds the HTTP protocol to the Web Service, and a JMS BC binds the JMS protocol.

In this solution, the HTTP BC and the JMS BC are used to bring the Billing Service and the Notification Service into the Open ESB runtime respectively - as stated previously, the Billing Service runs over the HTTP protocol and the Notification Service over the JMS.

It's important to be aware that, although these components are shown connected directly in the figure, they never communicate directly to each other. Instead, components send massages to and receive messages from the NMR. The NMR is responsible for transforming messages and routing messages to the appropriate destinations.

GlassFish & NetBeans - Development & Deployment

Open ESB runs on any OSGi R4-compliant runtime. GlassFish has a built-in JBI runtime and is bundled with NetBeans for easy development. NetBeans provides a comprehensive GUI development environment. The java.net community is working on a new project, GlassFish ESB aimed at creating a community-driven ESB for the Glassfish Enterprise Server platform.

Developing the process in NetBeans involves creating the needed JBI modules and including them in a composite application. Figure 5 shows a screenshot of creating the composite application in the NetBeans IDE.

Conclusion
Open ESB provides a robust and flexible platform for building service-oriented integration solutions. Its component-based architecture allows maximum extensibility and interoperability. It's based on industry standards and is easy to use. It seamlessly integrates with other Java enterprise technologies.

References

• JSR 208: Java Business Integration (JBI)
• Open ESB Project
• The Community Wiki for JBI, JBI Component Development and Open ESB.
• JDJ for useful resources and the scenario 

Open JBI Components

The overall goal of Project Open JBI Components is to foster community-based development of JBI components that conform to the Java Business Integration specification (JSR208). You can join this project as a JBI component developer or as part of an existing JBI component development team.

About JBI Components

The JSR208 specification provides for three installable JBI components: Service Engines, Bindings, and Shared Libraries. JBI components operate within a JBI container, which is defined by the JSR208 specification. Two popular implementations of JBI containers are Project Open ESB and ServiceMix, an alternative approach which has been mentioned in previous posts.