deploying big data to the cloud: roadmap for · pdf filedeploying big data to the cloud:...
TRANSCRIPT
Deploying Big Data to the Cloud:
Roadmap for Success
James Kobielus
•Chair, CSCC Big Data in
the Cloud Working Group
•IBM Big Data Evangelist. IBM
Data Magazine, Editor-in-
Chief. IBM Senior Program
Director, Product Marketing,
Big Data Analytics
Twitter: @jameskobielus
Clouds with big data are a foundation of the smarter planet
Mobile Social
Cloud
Internet of Things
Key Takeaways
• Big data is integral many cloud services applications.
• Big data platforms ensure scalability, flexibility, and cost-effectiveness for cloud analytics: – massively parallel processing, in-database execution, storage
optimization, data virtualization, and mixed-workload management
• Cloud services realize big data objectives: – ubiquitous, convenient, on-demand network access to data analytics
– shared pool of configurable data analytic computing resources
– dynamic provisioning and release of data analytic resources
• CSCC has recently published guidance for organizations to realize value from cloud-based big data initiatives: – “Deploying Big Data Analytics Applications to The Cloud: Roadmap for
Success" (http://bit.ly/1iQ8aSy)
Cloud Ensures Scalability for Big Data Application Workloads
from surveillance cameras trade events per second
meter readings per annum
Analyze sentiment
Predict power consumption
Monitor events of interest Identify potential fraud
Prevent churn
call detail records per day are images, video, documents…
Improve constituent satisfaction
Volume Velocity Variety
5 100’s of Tweets create daily
12 terabytes
video
feeds million
350 billion 500 million 80% data
growth
Big Data = realizing differentiated value from advanced analytics and
trustworthy data at cloud scales.
Why is Big Data in the Cloud Important Now?
The power of all data coming together…
…with the power of cloud services
…to deliver improved outcomes
1. Enrich your information base
with Big Data Exploration
5. Prevent crime with Security and Intelligence Extension
3. Optimize operations
with Operations Analysis
4. Gain IT efficiency and scale
with Data Warehouse Augmentation
2. Improve customer interaction with Enhanced 360º View of the Customer
5
Top 5 Use Cases We’ve Observed
Cloud is the Latest Major Wave of Technology
Back Office Computing
Client Server PC - 1981
World Wide Web and eBusiness
Confluence of Social Mobile Cloud Big Data / Analytics
90’s 80’s 60’s We are here
Many users are already on the
way to cloud with consolidation
and virtualization efforts
CONSOLIDATE Physical Infrastructure
CLOUD Dynamic provisioning for workloads
VIRTUALIZE Increase Utilization
STANDARDIZE Operational Efficiency
AUTOMATE Flexible delivery & Self Service
SHARED RESOURCES Common workload profiles
Traditional IT
Industry Movement from Traditional
Environments to Clouds
7
Leon Katsnelson ([email protected])
Smartphones and 1.2 billion mobile employees by 2014
1Billion
Extended Reach
Speed Value
view cloud as critical to their plans
90%
of digital content in 2012, up 50% from 2011
2.7ZB
New Insights Intelligent business assets
20B+
Responsiveness
1. Technology factors
2. People skills
3. Market factors
4. Macro-economic factors
5. Regulatory concerns
6. Globalization
Factors impacting organizations: 1
Source: IBM CEO Study 2012
Industry Puts Cloud At the Forefront of Their
Business Strategies
8
What Industries Are Doing with Big Data in the
Cloud
Utilities Weather analysis Smart grid management
Retail Marketing Campaign
Efficiency
Targeted Marketing
MicroSegmentation
Law Enforcement Multimodal surveillance
Cyber security detection
Transportation Logistics optimization
Traffic congestion
Financial Services Fraud detection
360° View of the Customer
IT System Log Analysis Cybersecurity Outage prevention Resource Prediction Warehouse Integration
Health & Life Sciences
Epidemic early warning
ICU monitoring
Telecommunications Geomapping / marketing
Network monitoring
9
Multiple Industries Customer Retention
Customer Acquisition
Manufacturing Manufacturing
Efficiency
Cloud-based Big Data Spans Systems of Record &
Engagement
Established Approach Structured, analytical, logical
Systems of
Record
Emerging Approach Creative, holistic thought,
intuition
Systems of
Engagement Multimedia
Data Warehouse
Web Logs
Social Data
Sensor data:
images
RFID
Internal App Data
Transaction Data
Mainframe Data
OLTP System Data
Traditional Sources
ERP Data
Structured Repeatable
Linear
Unstructured Exploratory
Dynamic
Text Data:
emails
Hadoop and Streams
New Sources
How to get there?
Step 1: Build your business case for big data in the
cloud
How to get there?
Step 2: Assess which big-data functions are best deployed
in cloud
•Enterprise apps already
hosted in cloud?
•High-volume data
requiring extensive
preprocessing?
•Tactical apps beyond
capabilities of legacy
platforms?
•Elastic provisioning of
very data-intensive but
shortlived analytic and
data management apps?
How to get there?
Step 3: Develop your cloud big-data technical
approach
•Public vs. private vs.
hybrid vs….
•MPP RDBMS vs.
Hadoop vs. NoSQL vs…
•Consolidated vs. multi-
tier vs. federated vs…
•Homogeneous vs. hybrid
How to get there?
Step 4: Maintain tight controls over your big data in the
cloud
•Governance
•Security
•Privacy
•Risk
•Accountability
•Compliance
How to get there?
Step 5: Deploy, integrate, & operationalize your big-
data cloud
Don’t do big data in the cloud unless you can
make it production-ready from Day One!
•Converge Big Data cloud operational
siloes
•Administer Big Data cloud through
consolidated system management tools
•Provide Big Data cloud users with a
“single throat to choke” on Big Data
cloud support
•Automate Big Data cloud support
functions to maximum extent feasible
•Deliver consulting support to users
considering implementing new Big Data
cloud initiatives
