bi forum 2009 - bi mega trends
TRANSCRIPT
BI Mega Trends
Where we are today!
Thomas Zimmer
HP Business Intelligence
BI Forum 2009
2 27 November 2009
Over the past 20 years, BI has evolved
Y2K bug
fails to bite
Query and
reporting
technologies
converge
Advent of
e-business
Oracle SQL
RDBMS
SAP BW
1.0
Structured and
unstructured data
converge
Operational BI
embedded in
business process
Data privacy
and security
Managing
information
as an asset
Packaged BI
applications
emerge
Business
Drivers
Information
as a strategic
differentiator
Data
integration
technologies
converge
Business
process
reengineering
Balanced
scorecard
introduced
Rise of the
technology-enabled
knowledge worker
ETL
emergesTDWI is
founded
The Health
Insurance
Portability and
Accountability
Act becomes
law
Dr. E.F. Codd
defines the
principles of
OLAP
Howard
Dresner defines
“business
Intelligence”
Bill Inmon
defines “data
warehousing”
Cutter Consortium
survey finds that
20% of DW
projects fail
Technology
Drivers
Sarbanes-
Oxley
Patriot Act
becomes law
META Group
survey finds
that more than
50% of DW
projects fail
1985 2000 2005 20101990 1995
3 27 November 2009
What Businesses are Saying…• “Data is growing at a compounding rate and we
have multiple non-integrated source systems. Our IT environment is too expensive to operate, manage, and maintain.”
• “We have multiple versions of the „truth‟ and I can‟t reconcile these reports.”
• “We‟re having problems complying with regulations because our metrics are inconsistent.”
• “Our sources often supply data of dubious qualityand I don‟t have control over the sources.”
• “We spend too much time fixing data and not enough time analyzing data. “
• “Our users are asking for analysis reports and intelligence in near-real time and we can‟t keep up with the demands.”
• “While our users continually scream for wider access we need to protect our data assets from unauthorized access and threats.”
4 27 November 2009
Pro
duct P
erf
orm
ance
Time or Engineering Effort
Executive Information System
Independent Data Mart
Enterprise Data Warehouse
Data Provisioning
Platform
BI technology S-Curve
5 27 November 2009
Executive Information System (EIS)P
roduct P
erf
orm
ance
Time or Engineering Effort
• Very few users, typically management
• Standard reports, usually monthly
• Often run on production system
• High IT requirement
• Full centralized control, minimal user flexibility
• No persistent analytic data store
• Minimal data integration
Dimensional
Proprietary
PC based (if you‟re lucky)
6 27 November 2009
Data Warehousing: Independent DMsP
roduct P
erf
orm
ance
Time or Engineering Effort
• Rise of the power user
• Query capability and analysis added
• Dimensional systems; typically departmental
• Introduction of BI tools and normalization for more user flexibility
• Rise of data redundancy and synchronization problems
• Heavy management overhead
• Data integrated from multiple sources via ETL
Dimensional (star schema)
OLAP
SMP based
Oracle
7 27 November 2009
Data Warehousing: Enterprise DWP
roduct P
erf
orm
ance
Time or Engineering Effort
• Eliminates redundant data; single version of
the truth• Ad hoc query and cross-functional analysis
• Reduces ETL costs; continuous data loading
possible• Re-centralizes control
• Integrates data from multiple sources, but requires agreement on a single data model
• Complex to manage, not adaptable to change
Normalised
ROLAP, MOLAP
MPP
Teradata, DB2
Blue Chip
Out Of Business
(fast)
Intelligence
Speed
Out Of Business
(slowly)
The Winners !
The Critical Balancing Act
9 27 November 2009
Data Warehouse Evolution Model
Strategic, predictive
Strategic + tactical, post-predictive
What Why did it What will What is What do I
happened? happen? happen? happening? want to happen?
Stage 1
REPORTING
Stage 2
ANALYTICAL
Stage 3
PREDICTIVE
Stage 4
OPERATIONAL
Stage 5
ADAPTIVE
Inactive Reactive Proactive /Autonomous
Custom scripts ETL + scripts EAI + ETL + scripts TDM or maxing out EAI
BATCH RIGHT-TIME NEAR REAL REAL TIME
Weeks Days Hours Minutes Seconds Sub-seconds
CH
AS
M
10 27 November 2009
Data Provisioning?? Operational BI??P
roduct P
erf
orm
ance
Time or Engineering Effort
Appliances?
Columnar?
Exadata?
Teradata?
??
??
Neoview
HP Confidential 11
• Strategic
• Reporting
• Standalone
• Weekly batch updates
• Simple ETL
• Single-function departmental data marts
• Few users doing strategic analysis
• Data volume < 1 TB
• Response time and availability not critical
• Summarized data
• Operational
• Automating action
• Mission-critical component
• Continuous online updates
• Sophisticated data integration
• EDW supporting “single version of the truth” for multiple applications
• Thousands of users performing many types of tasks
• Data volumes at more than 100 TB
• Near-real-time response and 24x7 online everything
• Detail plus years of history
Business intelligence is evolving to become an integral part of business operations
Why business intelligence matters
Capitalize on regulatory change
Respond to market opportunities
Increase profitability
Improve efficiency and productivity
Optimize growth opportunities
Reduce the cost of running the business
Maximize up sell and cross sell
Manage operational and financial risk
Grow and retain customer base
Connect your business to new opportunities
New World Requirements
Workload
MixQuery
Complexity
Active Data Warehousing
3-5 Way
Joins
Normalized
TB’s
MB’s
GB’s
Query Data
Volumes
10 TB
Old World
New World
15 TB
20 TB
Multiple, Integrated
Stars and Normalized
15+ way Joins +
OLAP operations +
Aggregation +
Complex “Where”
constraints +
Views
Parallelism
Batch Reporting,
Repetitive Queries
“Iterative”, Ad Hoc Queries
Data Analysis/Mining
Near Real Time Data Feeds
Simple
Star
Multiple,
Integrated
Stars
Data Storage
Schema
Sophistication
5-10 Way
Joins
5 TB
# of
Concurrent
Queries
1,000
15 27 November 2009
Questions?
“Even if you‟re on the right track, you‟ll get
run over if you just sit there.”Will Rogers
Copyright © 2007 Hewlett Packard. All rights reserved.