Daten Migration - Eine Publikation von Bloor Research

Data Migration A White Paper by Bloor Research Author : Philip Howard Publish date : October 2007 This document is Copyright © 2007 Bloor Research In our view, data migration has historically been under-valued, under-resourced and not treated with the attention it deserves Philip Howard This document is sponsored by: Data Migration page 1 Introduction According to research conducted by Bloor Research (see http://www.bloor-research.com/ research/survey/876/data_migration_survey.html) industry currently spends in excess of $5bn per annum on data migration, including software, services, consulting and so forth. Yet there is no market for data migration per se. Nor is it recognised as an independent discipline within data management. Indeed, the tasks required in data migration are typically treated as being just a part of what you need to do when migrating, implementing or upgrading application software. These application projects frequently overrun their budgets, get delayed or, in extreme circumstances, get cancelled. We believe that a major reason behind these failures is precisely because the techniques and disciplines of data migration are not treated seriously enough or are not well enough understood. This is supported by our research, which indicates that more than 60% of data migration projects have overruns on time and/or budget. In our view it is time to reverse this situation. It is time for data migration to be recognised as a discipline it is own right and for its exponents to get due credit when their job is well done. It is time for data migration to be treated as a major component, with its own requirements, within larger application projects rather than as an afterthought. This paper is not supposed to be a how-to guide for data migration. Such can be found in John Morris’ “Practical Data Migration” published by the British Computer Society. What it is intended to do is to highlight the major issues involved in data migration. We hope that an understanding of the complexities involved will help readers to understand why we believe that data migration, and the people who do it, need to have their status elevated. At the least, we hope that the issues involved in data migration will be taken more seriously, especially in the planning stages of broader application projects. Data Migration page 2 What is data migration? According to the Wikipedia, data migration is changed in the process. For the same reason, via change data capture – does not involve “the transferring of data between storage types, if you move your applications from an Oracle data migration. formats or computer systems… it is required database to MySQL, say, but retain your when organizations or individuals change existing applications (and therefore, the same The first conclusion we can draw from this is computer systems or upgrade to new systems.” schema) then that is a data movement that data migration never exists in isolation: it According to the Webopedia it is 1) the process of exercise and not one of data migration. is always linked to applications in some way, translating data from one format to another… whether that be an application migration, necessary when an organization decides to use a A further consideration is with respect to implementation or upgrade. This is one reason new computing system or database management conventional ETL (extract, transform and load) why there is no such thing as a data migration system that is incompatible with the current processes. Under both of the definitions discipline per se, because it is always a subset system and/or 2) the process of moving data quoted above, ETL, at least as it relates to of some broader effort. However, this may be from one storage device to another, in which loading data into a data warehouse, would be a reason but it is not an excuse: the use of ETL case data migration is the same as Hierarchical considered a form of data migration. Certainly is part of a broader discipline but it does have Storage Management (HSM).” some of the same techniques might be its own discipline area. employed but we do not believe that this is In part because of the qualifications (“it is how data migration is commonly understood. Getting back specifically to the question of required…”, “necessary when…”) that form both So, we need a definition of data migration that how we define data migration, the second of these definitions, neither is correct. For excludes both pure data movement and the conclusion we can draw from the use cases example, the Wikipedia definition implies that sort of ETL processes that are used to load a above is that the usage of the data, if by that if you upgrade your system then you will need data warehouse. we mean whether the data is used for to migrate your data, which isn’t true. operational or analytic purposes, remains Similarly, you might infer from the Webopedia Finally, a further distinction must be made constant. With data migration you never definition that compatibility is a key issue. between data migration and data integration migrate from an operational environment to Indeed it is but this definition might suggest more generally. The former is a one-time an analytic (business intelligence) that existing systems are compatible when exercise whereas most other forms of data environment or vice versa. they are not. integration, such as loading a data warehouse, are on-going. Note that we are defining usage here as Migration literally means movement from one distinct from context. Loading data into a place to another. However, if you were to move It is worth considering a number of common warehouse changes the context in which it is from Boston to New York or from Manchester use cases to see whether these actually used, as does migrating data from an ERP to London one would not normally describe involve data migration or not: system into a CRM system. However, not all that as migration. On the other hand, if you data migrations change context. For example, moved (on a permanent basis) from Boston to • Moving from one or more databases or the consolidation of heterogeneous CRM Paris then you would be (e)migrating from storage types to another, with all other systems does not change the context of the your perspective or (im)migrating from the things remaining constant – not data data: it is still CRM. point of view of a French person. The migration. difference is that you are moving from one Our definition is therefore as follows: cultural, linguistic and political milieu to • Loading a data warehouse – not data another, which will involve significant change. migration. “Data migration is any movement of persistent This is the essential difference between data that involves some sort of restructuring of moving and migrating: the latter involves • Archival, where older or less used that data while the usage of that data major change while the former does not. information is moved to secondary storage (operational versus analytical) remains – not data migration. constant.” To take a simple example, if you wish to consolidate multiple (Oracle) Siebel CRM • Migrating from one application to another Note that you could optionally replace systems to a single instance of Siebel, and all (such as Oracle to SAP) – will require data “movement of persistent data” in this definition of those systems use the same version of that migration. with “one-time movement of data”. Either of software (and the systems have not been these qualifiers will exclude the sort of data customised, which admittedly is unlikely), then • Implementing a new application where integration that involves the transformation of this would represent a data movement task (some of) the data for the new application things such as SWIFT and EDI messages, and not data migration. Conversely, already exists within existing applications which is an on-going process for what are consolidating multiple, heterogeneous CRM – will require data migration. essentially transitory objects (though these systems into one new one would data require may be persisted this is done for auditing data migration.