Business Intelligence and SOA - a Powerful Combination

BI and SOA – a Powerful Combination The combination of SOA (Service Oriented Architecture) and BI (Business Intelligence) represents a powerful paradigm shift, which promises to help manage complex BI environments more cost efficiently, and to leverage ana lytic information from BI applications to improve the performance of complex operational and management processes. This whitepaper provides an overview of how SOA concepts can be leveraged in a BI environment on the one hand, and of the value that BI can provide in an SOA environment on the other. The BI Challenge: Managing the Complexity of Large- Scale BI Environments Business Intelligence has become a cornerstone of the application landscape in almost every large enterprise. BI applications are used on many levels, from the analysis and optimization of individual business processes to Corporate Performance Management (CPM) on the global level. Figure 1: The Problem of Traditional BI However, most BI environments have grown organically over many years, have often been built in an ad-hoc fashion to satisfy individual customer requirements, and are suffering from data redundancies and inconsistencies, thus limiting the potential benefits and becoming costly to operate and adjust. Additionally, many relevant information sources are still untapped and are not accessible for decision processes. All in all, they are not following the enterprise architecture principles that would help manage the high level of complexity that many of these systems have now attained. In this respect, today’s complex BI environments are very similar to the rest of the application landscape in most large enterprises. SOA was developed as a way of addressing the problems of the existing heterogeneous enterprise application landscapes. Therefore it seems only consistent to also apply SOA to help manage the complexity of BI environments. This whitepaper describes how to best apply the principles of SOA to help manage the increasing complexity of large BI environments more effectively. The SOA Challenge: Complex Operational and Management Processes Require Analytic Information When we look at the combination of SOA and BI, we quickly discover that not only can SOA provide added value in a BI environment, but also that large SOA deployments can equally benefit from BI. Today, SOA is largely seen as an architectural principle in which basic services with stable functionality are used as the building blocks for managing complex, frequently changing operational process chains, often managed by what we call Business Process Management Systems (BPMS). This process-driven approach to integrating disparate application systems has many benefits, including improved agility and lower TCO (Total Cost of Ownership). However, it seems that the now widely accepted SOA and BPM architecture is missing a crucial element: many complex business processes – operational as well as management processes – require not only operational but also analytic information in order to make intelligent decisions about the process control flow. For example, an automated process which manages the claims processing for an insurance company requires information about the customer, the related contract claim history and the account, as well as the actua l claim itself. In an SOA, this information is normally provided to the process through a number of basic SOA services, which encapsulate operational systems to make such information available in real time. However, the claims process also requires analytic information in order to be efficient. For example, in order to decide whether the claim should be processed automatically or actually be routed to an insurance clerk for manual verification to eliminate false claims, the process will have to access analytic information as a basis for this decision. This analytic profiling is very likely to come from a BI system. Consequently, it seems natural to also include BI services as “analytic services” in an SOA. This whitepaper will later explain in more detail how this can be achieved. SOA Basics Let’s first look at the basics of an SOA, before discussing how SOA and BI fit together technically. In an SOA, a service usually represents a piece of business logic. The service encapsulates application logic and data, and makes it accessible though a well defined interface. Interfacing technologies include modern and widely accepted standards like WSDL (Web Services Description Logic) and SOAP (Simple Object Access Protocol), but also older standards like CORBA (Common Object Request Broker Architecture). Also, many enterprise SOA environments include message-oriented middleware as well as bulk data exchange technology, such as ftp (File Transfer Protocol). Figure 2: SOA Basics The actual semantics of a service are usually defined in what is known as the service contract. Such a service contract specifies the purpose, functionality, constraints, and usage of the service in an informal way. The service contract usually also includes a formal interface definition. Because services are designed to provide the basic building blocks for more complex process implementations , they have to be managed efficiently in order to foster the re-use of existing services. Usually, service contracts and service interfaces are managed in a central service repository. A leading standard for managing WSDL- based services is UDDI (Universal Description, Discovery & Integration). Basic services are usually coarse-grained, business-oriented services with relatively stable functionality. In order for a service consumer to access a service, most SOAs leverage a service infrastructure, e.g. based on an ESB (Enterprise Service Bus). The Service Bus usually provides many value-added features in addition to basic interfacing functionality, including guaranteed delivery, security, transactionality, and so on. Some SOA environments also leverage Business Process Management Systems (BPMS) for service orchestration and the execution of complex processes. For example, a BPMS can access basic services in order to access operational data which is required for managing the process control efficiently. Basic Building Blocks for BI SOA: Analytic Services How do these basic principles of an SOA apply to a BI environment? Typically, in this modern approach, BI application components are leveraged as SOA services. The resulting analytic services follow the basic SOA principles, as described above. The main constitue nts of analytic SOA services are the service contract and the a nalytic service logic. The service contract includes the analytic service interface, a description of the underlying analytic model, as well as the data domain definition. The a nalytic service interface is a well defined interface which makes the analytic service accessible through the ESB (Enterprise Service Bus), much like any other SOA service. It describes the semantics and the format of input and output data (e.g. data grids, graphics, etc.). The a nalytical model is also documented. The data domain definition describes the data which the service accesses in order to fulfil its analytic operations. The a nalytic service logic provides the actual implementation of the service, e.g. leveraging queries, analytical combination and restrictions, business rules, etc. The a nalytic service logic also includes data access logic which aggregates, cleanses and quality assures the data upon which the analytic operations are performed. Finally, an analytic service might also include a specialized analytic service data mart, if required, e.g. for performance reasons . This is usually a custom-designed data mart which is needed to fulfil specific needs of a particular service. Figure 3: Analytic Services Leveraging the concepts of SOA for basic analytic services is a key prerequisite for helping to manage complex BI environments more efficiently. Building up catalogues of re-usable analytic services and describing their behaviour in service contracts helps reduce redundancies and inconsistencies, and helps build a more manageable BI architecture. For example, UDDI could be leveraged for managing the catalogue of BI services available in an enterprise. Benefit of using Basic Analytic Services The Basic Analytic service becomes a “black box”, and the service contract becomes the only binding link to the consumer. This means that the analytic service becomes the basic unit of management, which helps manage semantics on the right level of granularity. c Better managed complexity Composite Analytic Services The next step in applying SOA principles to BI is to leverage the SOA concept of composite services. In an SOA, a composite service is usually a more complex service, which acts as a consumer of existing basic services. Figure 4: Composite Analytic Service Similarly, a composite analytic service leverages basic SOA services as the building blocks for creating more sophisticated analytical services. These basic services can be either basic analytic services or normal services, e.g. services which make operational and other data accessible. A good example of a composite analytic service is a service which accesses information about a stock portfolio from a local operational service and combines it with data from an externa l service providing information about stock performance on a stock exchange. The composite service can now combine the information about the stock portfolio and the market data to yield analytic information about the overall portfolio performance. Benefit of using Composite Analytic Services Composite Analytic Services leverage existing functionality of existing analytic and other basic SOA services. Composite services can be assembled quickly and efficiently.