Data Integration Blog
In this blog you can find posts with useful links to, news on and analysis of things like data integration, mashups, data quality, data warehousing, application integration (EAI), data management…the list goes on.
Monday 20 October 2008, 2:20 PM
"Clean Up Your Room" Turns into "Clean Up Your Data"
But nonetheless, clean up you should. If your source systems and the customer leads are a mess, of course. The obsession with clean data is only justified in this world of Business Intelligence, where looking at the picture as a whole and thinking big is not an encouraged, yet infrequent practice anymore, but a requirement.
One of the key elements to having your data clean and having a global view of your organization’s lifecycle is data migration. Wise data migration with an appropriate strategy and the right tools, not the sort where you splash money and remain in the same spot you started.
So how do successful companies approach this process? Simple, really. After you're solid in your decision to move you data, determine exactly what you're migrating and what it'll take. That is, identify the time, costs and human efforts that might be associated with this. It also helps to reherse. Unfortunately, not all integration vendors offer quality simulation/visualization tools, but that''s not too hard to fix.
Moving data around like that can also become a great opportunity to improve your data quality. That is, by checking its accuracy and consistency before sending it on to the new system.
PreviousComments on this post
Hi Alena,
A data migration not only offers a great opportunity to improve data quality - it often demands it. It’s likely that the target system will have different data standards and data structures. Whilst ETL tools can provide simple transformations of structured data, they’re inadequate when it comes to semantic content and requirements such as textual parsing and fuzzy matching.
Failure to address data quality as part of a data migration is a major cause of project failures and poor adoption of the new system. Standish Group estimates that 30% of data migration projects fail, whilst a Bloor Research study found that 84% of data migrations fail to meet expectations.
An audit of the source data at the outset of the project, including an assessment of its fitness for purpose in the target, should be a pre-requisite for any data migration. It’s the essential first step in removing the risk of failure due to inadequate data quality.
The company I work for recently upgraded their entire computer system, at HQ and all the stores. During the process some customer data failed to make the transition.Also some non-essential apps were placed on some local servers resulting in a week of downtime. The upgrade was from windows 2000 to XP. This created frustration and embarassment to employees and customers alike. We were often asked what OS we were running and the minute we said windows almost all customers said that was our first mistake. The majority said we should use MAC's, and most of the rest said it should be UNIX based. When I started working there they were using a UNIX system and had zero downtime and then they make a deal with Redmond and now it is costing them with poor customer service, constant errors, downtime, and daily reboots.
Steve,
I wasn't aware of this research. 84% - auch, that's alarming! I just don't understand how come, with the number of data integration tools to choose from on the market, companies still choose to suffer from poor data quality and data loss?
Ator,
Sounds like your company first rushed into the decision of migrating without considering the pro's and con's and other options. As a result, the steps necessary to protect your customer data were insufficient (a thing as basic as backup would prevent data loss). If this issue was given more thought in the beginning, you'd probably still be using UNIX. =)
To add a comment, fill out the form below
