tungsten university: geographically distributed multi-master mysql clusters
DESCRIPTION
Do you need run multi-master MySQL database clusters across sites? In this webinar we build on the multi-master capabilities of Continuent Tungsten, covered in our advanced replication topologies course, to help you build and manage systems that spread data across dozens or even hundreds of sites. We cover topics such as large scale topologies, transaction filtering, security, and moving replication from simple data movement to application messaging. Course Topics: - Review of multi-master replication topologies - Setting up Continuent Tungsten clusters for local site HA - Linking Continuent Tungsten clusters across sites - Security for cross-site replication - Handling local as well as site-wide failures - Off-the-shelf filters to improve multi-master replication - Custom filter development to implement business rules for application messaging.TRANSCRIPT
©Continuent 2013
Tungsten University: Geographically Distributed
Multi-Master MySQL ClustersRobert Hodges, CEO
Giuseppe Maxia, Director of QA
©Continuent 2013
Introducing Continuent
2
• The leading provider of clustering and replication for open source DBMS
• Our Product: Continuent Tungsten
• Clustering - Commercial-grade HA, performance scaling and data management for MySQL
• Replication - Flexible, high-performance data movement
©Continuent 2013
Geographic Distribution of Data
3
©Continuent 2013
Why Multi-Master Across Sites?
4
• Move applications close to customers
• Online game pro"les
• Mapping app information
• Protect against site failures
• Hosted call centers
• Credit card processing
Multi-master is NOT a performance scaling solution
©Continuent 2013
Tungsten Cluster Architecturein 3 Minutes or Less
5
©Continuent 2013
Overview of a Tungsten Cluster
6
Application
Tungsten Connector
Application
Tungsten Connector
Replicator
Db2
Replicator
Db3Db1
Replicator
Slave SlaveMaster
Manager Manager Manager
Monitoring and Control
Monitoring and Control
Data Service: sjc
©Continuent 2013
Tungsten Replicator Architecturein 2 Minutes or Less
7
©Continuent 2013
Tungsten Replicator Overview
8
Master
(Transactions + Metadata)
Slave
THL
DBMSLogs
Replicator
(Transactions + Metadata)
THLReplicator
Download transactions via network
Apply using JDBC
©Continuent 2013
Simple Multi-Master Con"guration
9
London New YorkReplicator
ny (master)
lon (slave)
Replicator
ny (slave)
lon (master)
Multiple services per replicator
©Continuent 2013
Cross-Site Clustering in Action
10
©Continuent 2013
Clustered Multi-Master Con"guration
11
London New York
master
slave
ny
ny
lon
lon master
slave
te1
te2
te3
te4
Cross-site replicators read from slaves
Cross-site updates are unlogged
©Continuent 2013
Reconnect to Slave after Switch
12
slave
master
master
slave
(Master switch to te2)
London New York
ny
ny
lon
lonte1
te2
te3
te4
©Continuent 2013
Connect to Master when Slave Fails
13
master
master
slave
slave
London New York
ny
ny
lon
lonte1
te2
te3
te4
(Slave failure/maintenance)
©Continuent 2013
Multi-Master Best Practices
14
©Continuent 2013
Eventually Consistent Design
15
• Each site is independent application with local HA cluster
• Replication links clusters
• Transfer is asynchronous
• Think through business implications
• What does it mean if a site is down for several days?
• How do I get up-to-date picture of data?
©Continuent 2013
Avoid Con#icting Primary Keys
16
• Method #1: Auto-increment o$sets
[my.cnf]server-id=1auto-increment-offset = 1auto-increment-increment = 4
• Method #2:
create table test.keys ( server_id int primary key, counter int);insert into test.keys(server_id, counter) values(@@server_id, 0);
©Continuent 2013
Use Row Replication
17
• Row replication sends changes to each row:
[my.cnf]binlog_format=row
• Statement replication results in ambiguous changes depending on state of replication
update employees set salary = salary * 1.1;
Can affect different numbers of employees depending on state of
replication between sites
©Continuent 2013
Watch Out for...
18
• Triggers -- Tungsten cannot turn them o$; you have to alter the code
• Scheduled Events -- Be careful where they run!
• MyISAM -- Not crash safe and just generally bad for replication
• Very large transactions --
©Continuent 2013
Look Out for Con#icts
19
• UPDATE/DELETE on same rows across di$erent sites
• Generation of globally unbroken sequences (e.g., invoice numbers)
• Reports that need global state from all sites
©Continuent 2013
Use Filters to Detect/Avoid Con#icts
20
Name Purpose
ignore_server Drop transactions from server-id
rename Rename schemas/tables/columns
replicate Control schemas/table replication
shardfilter Control shard replication
You can also write your own filters!
©Continuent 2013
Test, Test, Test!!!
21
• Set up a QA environment
• Put load on every master on every site
• Test backup and restore on every DBMS node
• Test switch
• Test failover
• Compare data between sites
• Develop a reset procedure for fast test cycles
©Continuent 2013
Be Ready For Trouble
22
• Ensure you have multiple days of logs so you can be o%ine for a while
• Learn how to use data repair tools like pt-table-checksum
• Remove constraints to make repair easier
• Monitor replication!
• Get a support contract for when things go wrong
©Continuent 2013
Advanced Multi-Site Clustering
23
©Continuent 2013
Complex Multi-Master Topologies
24
StarReplication
AllMasters
©Continuent 2013
Snow#ake Topology
25
HQ
Region 1 Region 2
CM1 CM2 CM1 CM2
©Continuent 2013
Wrapping Up
26
©Continuent 2013
Tungsten University Sessions
27
• Replicate between MySQL and Oracle(May 2 and 7)
Send any feedback to: [email protected]
©Continuent 2012.
Continuent Web Page:http://www.continuent.com
Tungsten Replicator 2.0:http://code.google.com/p/tungsten-replicator
Our Blogs:http://scale-out-blog.blogspot.comhttp://datacharmer.org/bloghttp://www.continuent.com/news/blogs
560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel +1 (866) 998-3642 Fax +1 (408) 668-1009e-mail: [email protected]