Data Warehousing and Business Intelligence

3 December 2012

Why Do We Need a Data Warehouse?

Filed under: Data Warehousing — Vincent Rainardi @ 6:52 pm

A lot of times people question the value of data warehousing. Why do we spend 1 year building a data warehouse? We can’t wait that long. Let’s just install QlikView/Spotfire and feed the transaction system direct to it and we have a BI!

Absolutely! You can. You can buy BO, MicroStrategy, QlikView, Spotfire or any BI tool you like, then report straight from the transaction system. Or, if you fancy, you can create a cube first (SSAS, Cognos or Hyperion), then install appropriate client tool (Tableau, Strategy Companion, etc). Of course you can. And this is the best way to learn about Data Warehousing: by not doing it.

What you will experience is:

a)    Data quality issues
b)    Low confident level from users
c)    Quick turn around of report but data is unstable
d)    Issues with data consistency
e)    Issues with performance

The whole year spent on building a data warehouse essentially for providing a quality data source. A data warehouse has the following characteristics:

a)    Integrated
b)    Consistent
c)    Contains historical data
d)    Tested and verified
e)    Performant

A data warehouse integrates data from multiple sources correctly. This integration doesn’t happen overnight. A Business Analyst spent weeks analysing the sources and wrote down a specification of how the data should be integrated. A Data Architect looked at that spec and designed a performant star schema to host the data. An ETL Architect looked at the star schema design and wrote an ETL population spec. An ETL developer studied the ETL spec and built the workflows. And finally, a tester verified the data.

That takes months, but as a result, we have integrated, consistent, clean data source containing the correct and valid data. And it is performant. Your query doesn’t need to join 15 tables in a horrible way. All the data is in a centralised place, ready for you to query.

But, as I said, experience is the best teacher. Once you experience how difficult it is to build business intelligence without a solid data warehouse, then you will truly appreciate the value of a data warehouse.

About these ads

3 Comments »

  1. […] Why Do We Need a Data Warehouse? […]

    Pingback by Why You Need a Data Warehouse | James Serra's Blog — 18 July 2013 @ 3:02 pm | Reply

  2. […] Why Do We Need a Data Warehouse? […]

    Pingback by Why You Need a Data Warehouse - SQL Server - SQL Server - Toad World — 18 July 2013 @ 3:16 pm | Reply

  3. […] data warehouse diehards will tell you something similar to what Vincent Rainardi says in his bloghttp://dwbi1.wordpress.com/2012/12/03/why-do-we-need-a-data-warehouse/  that a data warehouse is worth it because it is: a)      Integrated […]

    Pingback by Does Data Virtualization Foreshadow the End of Data Warehouses as We Know them Today? | Stone Bond Technologies — 11 February 2014 @ 3:03 pm | Reply


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

The Rubric Theme. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 199 other followers

%d bloggers like this: