modernizing your data warehouse for...
TRANSCRIPT
Modernizing Your Data Warehouse for Hadoop
Christian Coté
Big data. Small data. All data.
The traditional data warehouse
“…data warehousing has reached the most significant
tipping point since its inception.
The biggest, possibly most elaborate data
management system in IT is changing.”
– Gartner, “The State of Data Warehousing in 2012”
The traditional data warehouse
Real time data2
Increasing datavolumes
1
Cloud-borndata
4
Increasing datavolumes
1 New data sourcesand types
3
The modern data warehouse
Microsoft’s modern data warehouse
Data Platform
PDW
SQL Server 2014
Microsoft Azure HDInsight
Scale out technologies
in Parallel Data Warehouse
0TB 6PB
APS /
HDInsight
APS
APS /
HDInsight
APS /
HDInsight
APS /
HDInsight
APS /
HDInsight
APS /
HDInsight
From terabytes to multi-petabytesScale out relational data to petabytes
In-memory performanceIn-memory Columnstore for next-generation performance
Columnstore
index representation
Concurrency and mixed workloadsGreat performance for mixed workloads
Query
Results
Data complexity: variety and velocity
Petabytes
What is big data?
Hadoop Cluster
What is Hadoop?
Hive
Distributed, scalable system on commodity HW
Core Services
Operational services Data services
HDFS
SQOOP
FLUME
NFS
LOAD & EXTRACT
WebHDFS
OOZIE
AMBARI
YARN
MAP REDUCE
HIVE &HCATALOG
PIG
HBASEFALCON
compute
&
storage
. . .
. . .
. . compute
&
storage
.
.
Hadoop clusters provide scale-out storage and distributed data processing on commodity hardware
Web app
optimization
Smart meter
monitoring
Equipment
monitoring
Advertising
analysis
Life sciences
research
Fraud
detection
Healthcare
outcomesWeather forecasting
Social network
analysis
Churn
analysis
Traffic flow
optimization
IT infrastructure
optimization
Legal
discovery
Natural resource
exploration
Hadoop offerings on-premise and cloudReal-time with complex event processing
Microsoft Azure
Architecture
Analyze unstructured data
in Excel
Combine different types of data with Power
Query
Analyze your data with Power Pivot and
Power View and perform analysis
Features and benefits
Build a cluster in minutes and
tear it down when you’re done
Optimize cluster-size for time to
insight or cost-savings
Features and benefits
Try HDInsight at www.windowsazure.com/bigdata
Try SQL Server for data warehousing in Microsoft Azure VMs atwww.windowsazure.com
Try Hortonworks Data Platform for Windows at www. hortonworks.com/products/hdp-windows/
Try SQL Server 2014 CTP1 at http://www.microsoft.com/en-us/sqlserver/sql-server-2014.aspx