Data Integration Blog
In this blog you can find posts with useful links to, news on and analysis of things like data integration, mashups, data quality, data warehousing, application integration (EAI), data management…the list goes on.
Monday 13 October 2008, 12:21 PM
ETL of your own - wise or not?
ETL - Extracting and reading data from the original source, Transforming it to suit your business, that is, cleansing and formatting it, and finally Loading(sending) it to your system/database/warehouse
So, ETL tools. Make one of your own or buy one from a trusted vendor? What's best for your company?
Before jumping into anything or rushing to a popular vendor, analyze the degree to which an ETL tool will benefit your company and wether it might be wiser to build one of your own.
Building a customized solution of your own implies hiring technical staff for that purpose, but you have a greater chance of your final solution to be simple and match just your business needs.
Purchasing an ETL tool you are more limited by the market offerings in terms of customization. To add to that, you are likely to face the complexity of educating your in-house staff, which might take a load of your company's time and resources. Regardless of this, however, most vendors still resort to purchasing ready-made solutions. How come? Well, try asking yourself the following questions and you might actually come to the same decision.
1. What are your goals, why do you need an ETL solution? Is this a one-time procedure with limited conditions or are you planning to make it a part of your organization's structure and strategy? What do you want from your ETL tool? Specify priorities.
2. How much can you spend on your solution? How much do you want to spend?
3. How many data sources are you working with, and what kind are they? What functionality do they already have that might be helpful at the extraction/transformation.
4. How much time can you painlessly allow for the transformation process?, for your entire ETL process?
5. How much human resources and time can you dedicate to this project? (don't forget about education)
6. If you decide to build your own solution, who is going to educate your staff? Are you competent enough in the process (etl, warehousing)?
7. Just how many ETL experts do you have in your company and how do you estimate their potential and skill? Are they replaceable?
Just these 7 for starters...
So, ETL tools. Make one of your own or buy one from a trusted vendor? What's best for your company?
Before jumping into anything or rushing to a popular vendor, analyze the degree to which an ETL tool will benefit your company and wether it might be wiser to build one of your own.
Building a customized solution of your own implies hiring technical staff for that purpose, but you have a greater chance of your final solution to be simple and match just your business needs.
Purchasing an ETL tool you are more limited by the market offerings in terms of customization. To add to that, you are likely to face the complexity of educating your in-house staff, which might take a load of your company's time and resources. Regardless of this, however, most vendors still resort to purchasing ready-made solutions. How come? Well, try asking yourself the following questions and you might actually come to the same decision.
1. What are your goals, why do you need an ETL solution? Is this a one-time procedure with limited conditions or are you planning to make it a part of your organization's structure and strategy? What do you want from your ETL tool? Specify priorities.
2. How much can you spend on your solution? How much do you want to spend?
3. How many data sources are you working with, and what kind are they? What functionality do they already have that might be helpful at the extraction/transformation.
4. How much time can you painlessly allow for the transformation process?, for your entire ETL process?
5. How much human resources and time can you dedicate to this project? (don't forget about education)
6. If you decide to build your own solution, who is going to educate your staff? Are you competent enough in the process (etl, warehousing)?
7. Just how many ETL experts do you have in your company and how do you estimate their potential and skill? Are they replaceable?
Just these 7 for starters...
PreviousComments on this post
what is ETL ?
Thanks for bringing this to my attention, fixed it in the first paragraph. =)
To add a comment, fill out the form below
