mysql in the cloud – endless possibilities
DESCRIPTION
MySQL in the Cloud – Endless Possibilities. Grant McAlister, Senior Principal Engineer. What is the CLOUD?. Cloud computing is Internet-based computing, whereby shared resources, software and information are provided to computers and other devices on-demand, like a public utility. - Wikipedia. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/1.jpg)
MYSQL IN THE CLOUD – ENDLESS POSSIBILITIESGrant McAlister, Senior Principal Engineer
![Page 2: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/2.jpg)
• Cloud computing is Internet-based computing, whereby shared resources, software and information are provided to computers and other devices on-demand, like a public utility. - Wikipedia
WHAT IS THE CLOUD?
![Page 3: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/3.jpg)
• Build your own– AWS – EC2, EBS & S3– Rackspace – CloudServers & CloudFiles
• Relational Database Services– AWS RDS– FathomDB
WHAT IS MYSQL IN THE CLOUD?
![Page 4: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/4.jpg)
BREAKING APART SCALING
CPU MEM
Storage IOPS
CPU
StorageIOPS
![Page 5: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/5.jpg)
• Drive capacity is doubling every 18-24 months for the same price
• IOPS are not CHEAP and are not getting much cheaper
• You can now split IOPS from storage to reduce costs
STORAGE IS CHEAP!!!!
![Page 6: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/6.jpg)
• Tape Sucks!– Slow and error prone– Humans are involved
• Cloud Storage– Disk based backups– Incremental backups – save time and money
• Recoveries that work!!!!
USABLE BACKUPS
![Page 7: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/7.jpg)
• Multiple locations across the world– Reduce latency to customer
• Locations can have multiple datacenters– Increase durability and availability
LOCATION, LOCATION, LOCATION
![Page 8: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/8.jpg)
• OLD– Typically measured in years– Almost never kill off a database
• NEW– Measured in hours or minutes– Application based creation and deletion of db
DATABASE LIFETIMES
![Page 9: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/9.jpg)
• Standard DB Instance creation– Get a new host – weeks to months– Get the host setup with OS and software – hours to days– Setup db instance, monitoring and backups – hours to days
• Cloud DB Instance creation– From API calls to running DB instance – 5 to 10 minutes
• Demo deleting and creating 10 DB Instances
RAPID CREATION AND DELETION
![Page 10: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/10.jpg)
• Automated Backups• Patching• Monitoring & Recovery of DB Instance and Hosts• Replication / HA
CLOUD DB SERVICE CAPABILITIES
![Page 11: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/11.jpg)
• New methods of using databases• Allow us to build applications that would have been
difficult or impossible to do.– Scaling up or down based on workload– Large number of partitions or shards– A hierarchical storage database system– Testing
WHAT ARE ENDLESS POSSIBILITIES
![Page 12: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/12.jpg)
ON DEMAND DAILY SCALING
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 240
2
4
6
8
10
12
14Daily CPU Load
Hour
Load 25% Savings
![Page 13: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/13.jpg)
ON DEMAND YEARLY SCALING
1 5 9 13 17 21 25 29 33 37 41 45 490
2
4
6
8
10
12
Yearly CPU Load
Week
Load
50% Savings
![Page 14: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/14.jpg)
VERTICAL PARTITIONING
Very Large CPU
Customers
Orders
Addresses
Order Items
Catalog
Very Large CPU Very Large CPU Large CPU
Customers
Addresses
Orders
Order Items
Catalog
Before After
![Page 15: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/15.jpg)
SHARDING HORIZONTALLY
Very Large CPU
Customers
Orders
Addresses
Order Items
Before After• Hashed by CustomerID
Very Large CPU
Customers
Orders
Addresses
Order Items
Very Large CPU
Customers
Orders
Addresses
Order Items
Very Large CPU
Customers
Orders
Addresses
Order Items
Very Large CPU
Customers
Orders
Addresses
Order Items
![Page 16: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/16.jpg)
• Cloud Service Provides– Provisioning and management of large numbers of
instances– Backup/Restore to facilitate making new partitions– Allows you to both scale up and scale out – Should only have to scale for CPU/MEM
INITIAL PARTITIONING & RE-SHARDING
![Page 17: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/17.jpg)
DATABASE OVER TIME
Storage
Small Server
Storage
Large Server
Storage
Very Large Server
Storage
Extra Large Server
![Page 18: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/18.jpg)
Small
HSM DB SYSTEM
Application Layer
Large CPU
Week 1
Large CPU
Week 1
Week 2
Small
Week 1
Week 2
Week 3
Small
Week 1
Week 2
Week 3
Week 4
Small
Week 1
Week 2
Week 3
Week 4
Week 5
Week 1
Week 2
Week 3
Week4
Week 5
Week 6
Give me Week1 data
Writes
Reads
![Page 19: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/19.jpg)
TYPICAL TESTING SETUP
PROD
Very Large Server
Storage
TEST
Large Server
Storage
WEEKLY REFRESH
![Page 20: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/20.jpg)
CLOUD TESTING SETUP
PROD
Very Large Server
Storage
Bob Test
Very Small Server
Storage
Ted Test
Very Small Server
Storage
Mary Test
Very Small Server
Storage
7 AM6 PM
![Page 21: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/21.jpg)
DEMO TESTING 10 DIFFERENT CONFIGURATIONS
Test Driver
Instance TPCC-0
innodb_thread_concurrency=0
Test Driver
Instance TPCC-1
innodb_thread_concurrency=1
Test Driver
Instance TPCC-2
innodb_thread_concurrency=2
Test Driver
Instance TPCC-3
innodb_thread_concurrency=3
Test Driver
Instance TPCC-4
innodb_thread_concurrency=4
Test Driver
Instance TPCC-5
innodb_thread_concurrency=5
Test Driver
Instance TPCC-6
innodb_thread_concurrency=6
Test Driver
Instance TPCC-7
innodb_thread_concurrency=7
Test Driver
Instance TPCC-8
innodb_thread_concurrency=8
Test Driver
Instance TPCC-9
innodb_thread_concurrency=9
![Page 22: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/22.jpg)
• The Cloud provides very powerful features • The new features can enable new use cases
• Still very early in cloud computing for databases• Just scratching the surface
CONCLUSIONS
![Page 23: MySQL in the Cloud – Endless Possibilities](https://reader035.vdocument.in/reader035/viewer/2022070421/568161ba550346895dd18db2/html5/thumbnails/23.jpg)
THANK [email protected]