Talend nutzt Open Source, um kostengünstige und einfach zu nutzende, unternehmensweite Datenintegrationslösung anzubieten

WHITE P APER Talend Uses Open Source to Deliver Low-Cost, Easy-to- Use Enterprise Data Integration Sponsored by: Talend Carl W. Olofson September 2008 IDC OPINION www.idc.com It has been generally accepted that the open source model works well for delivering software support and other services on top of well-understood software technology in commodity markets such as operating systems and general-purpose relational DBMS. Talend is demonstrating, however, that this model may also be applied as a F.508.935.4015 go-to-market strategy to build a business based on new technology, such as that of the still rapidly evolving data integration market. Talend's product line, which is positioned to offer scalable, incremental enterprise data integration with components that are easy for nontechnical staff to use, breaks the mold for open source, and its success could shake up the software business. P.508.872.8200 METHODOLOGY Information for this white paper was derived from IDC research in the data integration and access software market, general IDC knowledge about software and software Global Headquarters: 5 Speen Street Framingham, MA 01701 USA markets, and specific research regarding Talend and the Talend products and business model. Information regarding Talend and Talend products was compiled through interviews with Talend executives and technicians, review of Talend material, and conversations with Talend customers. IN THIS WHITE P APER This white paper discusses the data integration and access market, looking specifically at two segments: Managed data movement, including extract, transform, and load (ETL) technologies and variants used for data synchronization and migration Data quality and profiling The document considers barriers to adoption of this technology in some companies, especially those related to the purchase of software licenses, and considers the market potential in the absence of such barriers. It examines the approach to this market taken by Talend, which is to offer a fresh approach to data integration (including data quality) software, delivering it under the open source model. This white paper discusses the business rationale underlying this strategy, considering its implications for software in general. It shows how the combination of Talend's technical approach and its open source licensing overcomes the barriers to adoption of data integration. SITUATION OVERVIEW An Overview of Data Integration Data Integration Origins in Project-Based Development Most enterprises initially created their software applications as isolated, self-contained systems designed to perform some narrowly circumscribed set of business functions. These applications had no relation to one another and were not designed to share data, except as part of hard-coded, usually batch processes designed to satisfy fixed operational requirements (e.g., moving orders data into a load file for the inventory management and shipping systems). When it was necessary to move data from one application to another, to coordinate common data (such as customer data), or to blend data from multiple applications for reporting purposes, enterprises typically used hard-coded programs or scripts. As the idea of a data warehouse, an omnibus repository of business-oriented data used for business analytics and reporting, emerged, the efforts involved in moving data to the data warehouse soon became too expensive, and the results too hard to manage, to be left to hand-coding. So, firms turned to ETL tools designed to generalize the process of extracting data from multiple sources, converting the data to the format of the target system, merging or reconciling the data, and loading it into the target. Soon, these tools were also used to replace hand-coded data synchronization utilities, and for data migration as well. These ETL tools were designed to run as batch jobs, extracting, transforming, reconciling, and loading data in very large lots. They were not especially easy to use and were not designed for smaller-scale data movement or access functions. Also, since most enterprises naïvely assumed that their data was OK, or that their data was valid even after simple transformations, they did not usually invest anything in data quality. Emergence of Data Integration Platforms In recent years, demand has been building for more flexible and responsive data integration functionality, with data quality included. Users are looking for data movement that can be scheduled in smaller lots, in so-called "real-time" data integration or data synchronization (data is moved as it changes from source to target), and data migration, all with the option of data quality and profiling (analysis of data to find patterns and hidden application "rules" in the data), all sharing common metadata (definitions of the data and the rules that govern the data). In response, vendors of ETL products have developed (from their existing products) or acquired data synchronization technology and acquired data quality and profiling tools, seeking to integrate them together as data integration platforms. Usually, there is an effort to blend their metadata as well. Generally speaking, these data integration platform offerings involve collections of products that require some considerable setup and configuration time because they are based on different code bases and technical approaches. This adds personnel and service cost to the project. Only recently have most data integration vendors also embraced the concept of incremental data integration, where the work of one project helps accelerate successive projects, thanks to persistent, shared metadata. Such functionality is typically hampered by the integration of products that were not originally designed to work together. 2 #214012 ©2008 IDC Barriers to Broad Adoption of Data Integration Most data integration platform offerings are based on products designed to be used for large projects involving big batch data movement and are commonly applied to large data warehouse, master data management, and other similar data movement tasks. As such, they are expensive, require teams of experts to implement, tend to be inflexible, and usually involve fairly long-term consulting engagements. For implementers of small to medium-sized data movement projects, these characteristics make them unattractive options, and they prefer the hand-coded approach, with all its attendant risk and maintenance cost. Of course, this approach does not offer a basis for further data integration development, since there is no metadata to be reused in successive projects, and it adds to the inventory of software that the enterprise must then maintain; therefore, it really represents a short-term cost savings but longer-term additional cost. Ideally, enterprises should seek to integrate data using a data integration platform that supports incremental development. Barriers to adoption of such a platform include: Initial cost. The license fee is more than enterprises can justify in the context of a given project's budget. Complexity or lack of flexibility in deployment. To do a job, enterprises need to learn a lot and execute a fairly heavyweight project just to install and configure the software for the job at hand. Project scope.