Data Warehouse – One – Enterprise ETL Yes or No

So the question here is should you have one enterprise ETL team and one ETL infrastructure for your entire enterprise/company/corporation

In my opinion Yes as soon as possible, but as always there is a but.

You can only have this is if you have standards in place (Or you are creating the standards with your first project) and when you have maturity in the team /enterprise to support and maintain this.
You need to clearly define the tools that you use for ETL – decide on your etl partner this partner is there to stay.

Create a process that can be re-used for each following etl job.

OK so what am I saying “re-usable etl job” – basically all ets job should for example support

  1. Batch_Run_id – in order to trace when data was inserted it should
  2. Batch_Date
  3. County Code – enterprises solution must scale across countries/States.
  4. Using the same Error table for logging
  5. Using the same metadata layer
  6. Using the same audit function
  7. Be re-executable.
  8. Be traceable – ie when where did it fail.
  9. If you use surrogate keys then all must use the same method to derive the key.
  10. Create components small components that can be re-used.

We can go on and on but you get the picture.

So what is your opinion?


Tagged: ,

2 thoughts on “Data Warehouse – One – Enterprise ETL Yes or No

  1. Bob Blackburn November 7, 2012 at 1:13 pm Reply

    I agree. The next step is dealing with mergers and multiple systems and tools. And, chances are if a company is big enough to have an EDW they are likely to have been through a merger/acquisition or two.

    • christianvdheever November 7, 2012 at 2:34 pm Reply

      I am currently at a company that is word wide thus we have a EDW.
      With a few challenges.
      We will have later blogs covering some of these challenges like (MDM, Currency/mergers etc)

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: