scaling your data horizontally on the openstack magnetodb - trove day 2014
DESCRIPTION
Presentation from OpenStack Trove Day 2014 by Keith Newstadt, Technical Director, Symantec CorporationTRANSCRIPT
MagnetoDBScaling Your Data Horizontally on OpenStack
August 19, 2014
Keith NewstadtTechnical Director, Symantec Corporation@knewstadt
tesora.com 2
Management Platform
Identity Notifications Storage Data Sync Events
Norton Family
8/19/14
Infrastructure
tesora.com 3
Symantec Cloud Platform Engineering
8/19/14
Infrastructure as a Service
Infrastructure
Platform Services
Servers Networks Disks Images Ops Identity
Storage Databases Big Data Monitoring Deployment
tesora.com 4
What Kinds of Databases Do We Need?
8/19/14
Relational
NoSQLGood for relational dataIn teams’ comfort zone
Can be difficult to operate
Scales well horizontallyLearning curve
Can be difficult to operate
tesora.com 5
What Kinds of Models Do We Need?
8/19/14
Customer Operated As a
ServicePlatform provides
operational services
Platform provides the database itself
tesora.com 6
MagnetoDB – NoSQL + as a Service
8/19/14
• Open source, OpenStack standards
• Large data sets• Throughput, availability, latency• Secure multi-tenant• Integrated with Keystone
TablesCreate
Load
Index
Query Scan
Configurable Consistency
Data Types (maps, sets,
etc.)
Conditional Operations
MagnetoDB
tesora.com 7
MagnetoDB – A Simple Architecture
8/19/14
API Proxy Layer
Database Layer
MagnetoDB Driver Layer
• Simple ReST (ish) Web service API• Modeled after DynamoDB, plus…
• We’re using Cassandra• HBase also in the works
• Supports different backend databases
tesora.com 8
Some Symantec Use Cases
8/19/14
MagnetoDB
User Profile Data
Searchable Swift Object Metadata
Import from Big Data Processes
Real-time Metrics
DemoUsing MagnetoDB8/19/14
tesora.com 10
Demo – Top 100 Music Charts
8/19/14
Songs
Key Schema Secondary Index
Artist Artist
Track High
Charts
Key Schema
Year
Week-Position
http://bullfrogspond.com/
MagnetoDB
• Tables• Queries
tesora.com 11
Local VM
Environment
8/19/14
MagnetoDB
https://wiki.openstack.org/wiki/MagnetoDB/QA/Tests_on_env_with_devstack
Q&AKeith Newstadt@[email protected]
8/19/14