![Page 1: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/1.jpg)
©Continuent 2014
Geographically Distributed Multi-Master MySQL Clusters
Jeff Mace, Director of Professional Services
![Page 2: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/2.jpg)
©Continuent 2014
Introducing Continuent
2
• The leading provider of clustering and replication for open source DBMS
• Our Product: Continuent Tungsten
• Clustering - Commercial-grade HA, performance scaling and data management for MySQL
• Replication - Flexible, high-performance data movement
![Page 3: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/3.jpg)
©Continuent 2014
Quick Continuent Facts
• Largest Tungsten installation by data volume processes over 800 million transactions per day on 225 terabytes of relational data
• Largest installation by transaction volume handles up to 8 billion transactions daily
• Wide variety of topologies including MySQL, Oracle, Vertica, and Hadoop in production
• Cross-site topologies widely deployed
3
![Page 4: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/4.jpg)
©Continuent 2014©Continuent 2014
Continuent Tungsten Customers
4
1
![Page 5: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/5.jpg)
©Continuent 2014 5
The Importance of Geographic Distribution for
Business Data
![Page 6: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/6.jpg)
©Continuent 2014
Use case: Registering and updating hardware devices
• Users may be located in any region
• DNS routes users to an available site with lowest round-trip latency
• Each site offers full-service processing to users
• BENEFIT: Transactions always process at the fastest location for user
6
![Page 7: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/7.jpg)
©Continuent 2014
Use case: Credit card transaction processing
• Customer connectivity to a particular Internet site is never guaranteed
• Credit card processing apps use a list of potential sites
• Credit card app connects to the first responsive site and issues transaction
• BENEFIT: No credit card transaction is ever refused
7
![Page 8: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/8.jpg)
©Continuent 2014
Use case: GPS device personalization and upgrade service
• Local sites may be subject to prolonged failures that would otherwise
• Business processing continues at other sites
• Transactions reconcile automatically when down site(s) are restored
• BENEFIT: Business processing continues without complex site failover or data loss
8
![Page 9: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/9.jpg)
©Continuent 2014
Automatic Failover
9
![Page 10: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/10.jpg)
©Continuent 2014
Automatic Local Failover
10
Application
Tungsten Connector
![Page 11: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/11.jpg)
©Continuent 2014
Automatic Local Failover
11
Application
Tungsten Connector
1. Detect non-responsive master
2. Halt master connections
![Page 12: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/12.jpg)
©Continuent 2014
Automatic Local Failover
12
Application
Tungsten Connector
3. Select most advanced slave and wait for pending transactions to apply
![Page 13: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/13.jpg)
©Continuent 2014
Automatic Local Failover
13
Application
Tungsten Connector
4. Promote to master, reconfigure slave(s), and re-enable connections
![Page 14: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/14.jpg)
©Continuent 2014
Automatic Local Failover
14
Application
Tungsten Connector
5. Recover failed master
![Page 15: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/15.jpg)
©Continuent 2014
Support Multiple Hosting Providers
15
![Page 16: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/16.jpg)
©Continuent 2014
Support Multiple Hosting Providers
16
AWS Rackspace
On-Premises
App Server + Connector
![Page 17: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/17.jpg)
©Continuent 2014
Bi-Directional Replication
17
AWS Rackspace
master
slave rack
rack
aws
aws
master
slave
Important Note: Continuent recommends 3 nodes for clusters
![Page 18: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/18.jpg)
©Continuent 2014
Provider Failover
18
![Page 19: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/19.jpg)
©Continuent 2014
Location Failover
19
AWS Rackspace
On-Premises
App Server + Connector
![Page 20: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/20.jpg)
©Continuent 2014
Location Failover
20
AWS Rackspace
On-Premises
App Server + Connector
![Page 21: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/21.jpg)
©Continuent 2014
Global Application Availability
21
![Page 22: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/22.jpg)
©Continuent 2014
Global Application Availability
22
AWS Rackspace
On-Premises
App Server + Connector
App Server + Connector
App Server + Connector
![Page 23: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/23.jpg)
©Continuent 2014
Demo
23
![Page 24: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/24.jpg)
©Continuent 2014
Design for MSMM
24
![Page 25: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/25.jpg)
©Continuent 2014
Application and Schema Design
• Use auto_increment or UUID for primary keys
• Avoid unique keys that may conflict from different locations
• Shard users into different schemas if possible
• Heavy INSERT workloads operate best
• Avoid UPDATE/DELETE statements of the same data from different locations
25
![Page 26: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/26.jpg)
©Continuent 2014
Server Configuration
• Use auto_increment_increment and auto_increment_offset
• Handle DNS management with Global Traffic Managers
• 24/7 monitoring and response is critical if you are operating in all sites
26
![Page 27: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/27.jpg)
©Continuent 2014 27
Wrap-Up
![Page 28: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/28.jpg)
©Continuent 2014
Standard clustering designs provably fail when operating over distance
• Synchronous replication adds severe performance drag (Daniel Abadi, Yale Univ.)
• Global locks create exploding deadlock problems (Jim Gray, Microsoft)
• Strong consistency between DBMS requires some/all regions to stop when network fails (CAP proof by Nancy Lynch, MIT)
28
![Page 29: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/29.jpg)
©Continuent 2014
Async multi-master is overall best approach for multi-site operation
29
Europe
North America
Asia/Pacific
SQL transaction processing in any region
Optimized performance
for users
Local high availability in each region
Continuous updates across
regions
![Page 30: Geographically Distributed Multi-Master MySQL Clusters](https://reader033.vdocument.in/reader033/viewer/2022052906/5589e369d8b42a8e558b4603/html5/thumbnails/30.jpg)
©Continuent 2014
www.continuent.com Follow us on Twitter @continuent
!
Tungsten Replicator: http://code.google.com/p/tungsten-replicator
Our Blogs: http://scale-out-blog.blogspot.com http://datacharmer.org/blog http://www.continuent.com/news/blogs http://flyingclusters.blogspot.com/
560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel +1 (866) 998-3642 Fax +1 (408) 668-1009 e-mail: [email protected]