the death of the data warehouse michigan oracle user summit 14 november 2012
TRANSCRIPT
The Death of the Data Warehouse
Michigan Oracle User Summit14 November 2012
©2004 Dataspace Incorporated.Any unauthorized use of these materials violates copyright and trademark laws.
W W W . D A T A S P A C E . C O M
The Business Problems We’re Trying to Solve w/ DW & BI?
Business people can’t get to their data
Running summary reports out of transaction databases is very slow
Performance issues of transaction DB
Reporting is complex
Disparate databases - No integrated view of the whole company
Transaction systems discard history
©2004 Dataspace Incorporated.Any unauthorized use of these materials violates copyright and trademark laws.
W W W . D A T A S P A C E . C O M
What we need to solve these
Subject Oriented
Integrated
Time variant
Non volatile
©2004 Dataspace Incorporated.Any unauthorized use of these materials violates copyright and trademark laws.
W W W . D A T A S P A C E . C O M
What we create
©2004 Dataspace Incorporated.Any unauthorized use of these materials violates copyright and trademark laws.
W W W . D A T A S P A C E . C O M
The Traditional DW Model
Complexities– Technologies to master
• Data modeling• ETL• BI• DBA
– Workplan steps to complete• Design data mart databases• Design DW databases• Design BI tool metadata• Build flows from source systems to DW• Build flows from DW to data marts• Build BI metadata
Result– Time consuming– Brittle (e.g. change to one column in the
source ripples through architecture)
©2004 Dataspace Incorporated.Any unauthorized use of these materials violates copyright and trademark laws.
W W W . D A T A S P A C E . C O M
Traditional BI Development
$
Success?
©2004 Dataspace Incorporated.Any unauthorized use of these materials violates copyright and trademark laws.
W W W . D A T A S P A C E . C O M
Data Warehouse Definition – The Physical DB Implications
Subject Oriented
Integrated
Time variant
Non volatile
This is a LOGICAL definition, not a physical one – it says nothing about how the data must be stored or accessed
©2004 Dataspace Incorporated.Any unauthorized use of these materials violates copyright and trademark laws.
W W W . D A T A S P A C E . C O M
New Generation of BI Tools (QlikView, Tableau, etc.)
They contain their own, non-relational, self-managing data stores.
They can import data from multiple sources into a single, accessible data store.
They join related data together, like a relational database.
They provide predictable, blisteringly fast query performance
They provide very easy, user-friendly user interfaces.
They can contain, and rapidly summarize, atomic-level, granular data.
They can be incrementally refreshed, enabling the storage of history.
These tools meet the definition of a data warehouse but are far more efficient
©2004 Dataspace Incorporated.Any unauthorized use of these materials violates copyright and trademark laws.
W W W . D A T A S P A C E . C O M
The Traditional DW Model
TRADITIONAL
Complexities– Technologies to master
• Data modeling• ETL• BI• DBA
– Workplan steps to complete• Design data mart databases• Design DW databases• Design BI tool metadata• Build flows from source systems to DW• Build flows from DW to data marts• Build BI metadata
Result– Time consuming– Brittle (e.g. change to one column in the
source ripples through architecture)
NEW WORLD
Complexities– Technologies to master
• In memory tool
– Workplan steps to complete
• Build flows from source systems to DW
• Build reports
Result– Agile– Easily revised
©2004 Dataspace Incorporated.Any unauthorized use of these materials violates copyright and trademark laws.
W W W . D A T A S P A C E . C O M
Preferred Model of BI Development
$
UserInput
Dev &Rvw
Quit
No
UserInput
Dev &Rvw
Yes
Quit
No
UserInput
Dev &Rvw
Yes
Develop DW in Parallel with Input from BI (If Necessary)
©2004 Dataspace Incorporated.Any unauthorized use of these materials violates copyright and trademark laws.
W W W . D A T A S P A C E . C O M
In Memory Advantages & Disadvantages
Replace DW
• Isolate operational systems from query demands
• Improve query response times with data structures optimized for query
• Provide a place to store history that might otherwise be lost
• Provide a place where users can access data integrated from multiple systems
Users prefer the in-memory / visualization approach
Less administration vs. traditional BI
Rapid development / rapid prototyping / incremental delivery
Data set size
Real time / Operational reporting
No access from other tools
Great for visualization & analysis - not for ‘greenbar’ replacement
Data cleansing & complex integration
MDM
©2004 Dataspace Incorporated.Any unauthorized use of these materials violates copyright and trademark laws.
W W W . D A T A S P A C E . C O M
Questions?
©2004 Dataspace Incorporated.Any unauthorized use of these materials violates copyright and trademark laws.
W W W . D A T A S P A C E . C O M
©2004 Dataspace Incorporated.Any unauthorized use of these materials violates copyright and trademark laws.
W W W . D A T A S P A C E . C O M
©2004 Dataspace Incorporated.Any unauthorized use of these materials violates copyright and trademark laws.
W W W . D A T A S P A C E . C O M
Traditional BI Architecture (e.g. Cognos Rpt Studio)
Point & click to generate SQL
Database –Operational or Informational
Format presentation
Source DB 1
Source DB 2
©2004 Dataspace Incorporated.Any unauthorized use of these materials violates copyright and trademark laws.
W W W . D A T A S P A C E . C O M
QlikView Architecture
Point & click to generate Query
Format presentation
Source DB 1
Data Warehouse
Associative DB
©2004 Dataspace Incorporated.Any unauthorized use of these materials violates copyright and trademark laws.
W W W . D A T A S P A C E . C O M
Demo