Download - Big Data on OpenStack
![Page 1: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/1.jpg)
Big Data on
OpenStack
@natishalom
![Page 2: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/2.jpg)
About GigaSpaces
Managing Big Data on the Cloud
100s of Enterprise Customers
![Page 3: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/3.jpg)
My Data Out of My
hands..
No Way!
![Page 4: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/4.jpg)
The Reality of Big Data…
2.7 ZB
0.5 Petabytes
66%
Global Digital Data
Two years’ tweets
Plan to use Big Data/Cloud
43% think that their
organization’s data analytics could be improved if data analytics was part of
cloud services
![Page 5: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/5.jpg)
Large ISV Case Study
• Application– Call Center surveillance
• Background– Previously – voice data
• Goal for a new system– Monitor data & voice– Multiple data sources – Advanced correlations
![Page 6: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/6.jpg)
The Challenges…
Ever Growing Data
Deeper Correlation
Tight Performance
![Page 7: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/7.jpg)
A Classic Case for..
![Page 8: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/8.jpg)
A Typical Big Data System
![Page 9: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/9.jpg)
The Challenge
Cost Business Impact
Lower Margins
Competiveness
Time to Market
Customer Satisfaction
Infrastructure
Operational
![Page 10: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/10.jpg)
The Solution Big Data
in the Cloud
![Page 11: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/11.jpg)
Big Data in the Cloud- 3 Reasons
• Skills– Do you really need/want this all in-
house?• Huge amounts of external data – Does it make sense to move and
manage all this data behind your firewall?
• Focus on the value of your data– Instead of big data management
Holger Kisker
![Page 12: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/12.jpg)
Managing Big Data on the
Cloud
• Auto start VMs• Install and configure
app components • Monitor • Repair • (Auto) Scale• Burst…
![Page 13: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/13.jpg)
Big Data in the Cloud
Reduce the Infrastructure
Cost
Choose the Right Cloud for the Job
Running Bare-Metal for high I/O workloads, Public cloud for sporadic workloads
![Page 14: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/14.jpg)
Big Data in the Cloud
Reducing the Operational Complexity
• Consistent Management
• Automation Through the Entire Stack
![Page 15: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/15.jpg)
Big Data on
OpenStack
![Page 16: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/16.jpg)
General Approach …
![Page 17: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/17.jpg)
Reducing the Complexity
17
My Recipes
Wrap all your system elements into easy-to-use recipes, providing you with consistent, automated management of your Big Data
Consistent ManagementTypical Big Data System
Scale
Monitor
Update
Deploy
One manager easily & consistently handles all system functions.
![Page 18: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/18.jpg)
Reducing the Infrastructure Cost
18
Consistent Management
Abstraction
Typical Big Data System
Creates an abstraction between your Big Data system recipe/blueprint and the target environment. This means you can take the same blueprint and simply point it at different environments without making any changes to your application.
Testing Production
Development
Client Environment
Scale
Monitor
Update
Deploy
![Page 19: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/19.jpg)
Is that Good
Enough?
What about-Performance?-Deterministic Latency?
![Page 20: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/20.jpg)
Bare Metal vs. Virtualization Benchmark
Source: Petestrenna
8.84%
14.36%
24.46%
2.41X
10.84X
Disk I/O
CPU and Memory
Network I/O
Disk Latency
Micro-operations
![Page 21: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/21.jpg)
Bare Metal vs. Virtualization Benchmark
Source: NTT DOCOMO
![Page 22: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/22.jpg)
The Impact on Big Data
Apps
3X more compute resources
for the same workload!
Non Deterministic Latency
![Page 23: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/23.jpg)
Bare Metal OpenStack Support
![Page 24: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/24.jpg)
Case-Study“We took this single image, picked it up from public cloud into a Rackspace-powered private cloud and saw a
4X increased efficiency running that workload.”
Jim O’Neill CIO at HubSpot
![Page 25: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/25.jpg)
Automation Frameworks
Configuration Centric APP Centric (PaaS)
![Page 26: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/26.jpg)
Big Data Apps, on Any Cloud, Your Way
Open source (Apache2)
![Page 27: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/27.jpg)
Built-in Support for Big Data StacksReal Time Relational DB
ClustersNoSQL Clusters Hadoop
Storm MySQL MongoDB Hadoop (Hive, Pig,..)
GigaSpaces XAP Postgress Cassandra ZooKeeper
Couchbase
ElasticSearch
![Page 28: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/28.jpg)
Moving from Existing Data Center to OpenStack?
Consistent Management
ScaleDeplo
y
Monitor
Update
Non Virtualized Data Center OpenStack Cloud
Cloud Driver
![Page 29: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/29.jpg)
Demo Time…
![Page 30: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/30.jpg)
Storm on OpenStack
![Page 31: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/31.jpg)
BigData Services Catalogue on OpenStack (HP)
![Page 32: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/32.jpg)
Large ISV Case Study
• Application– Call Center surveillance system
• Background– Previously – voice data
• Goal for a new systemMonitor data & voiceMultiple data sources Advanced correlations Mission
Accomplished
![Page 33: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/33.jpg)
Additional Benefits
• True Cloud Economics
• One product -> any Customer Environment
• Increased Agility
![Page 34: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/34.jpg)
Thank You!
References: http://www.cloudifysource.org http://github.com/CloudifySource
![Page 35: Big Data on OpenStack](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cd84a7959af718b4738/html5/thumbnails/35.jpg)
Additional References
• Bare Metal Cloud/PaaS• OpenStack Baremetal Project • Big Data in the Cloud• Big Data in the Cloud using Cloudify• Putting Hadoop On Any Cloud (A video presentation)• In Memory Computing (Data Grid) for Big Data• Using the Cloudify Player as an Open Source Framework for Buil
ding Your Own Cloud Application Marketplace on OpenStack• Going native: The move to bare-metal cloud services• New bare metal cloud offerings emerging• How much overhead does x86/x64 virtualization have?• Amazon EC2 versus Bare Metal and KVM? The inside story on w
hat you thought you knew about EC2