my sql 5.6_replwebinar_may12

MySQL 5.6 Replication: Supporting High Scale Web & Cloud Services Mat Keep MySQL Product Management [email protected]

Andrew Morgan MySQL Product Management [email protected]

The image cannot be

<Insert Picture Here>

Session Agenda

•  Development Priorities

•  MySQL 5.6 Replication Enhancements

•  Resources to Get Started

Copyright 2012 Oracle Corporation 2

The image cannot be

The presentation is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


The image cannot be

Cloud

Web & Enterprise OEM & ISVs

Industry Leaders Rely on MySQL Replication is Key to Scale & HA


The image cannot be

•  MySQL 5.6 builds on MySQL 5.5 by improving: •  Performance and Scalability •  Optimizer for better query execution times, diagnostics •  Performance Schema for better instrumentation •  InnoDB for better transactional throughput •  NoSQL API for better flexibility •  Replication largest set of enhancements ever released!

•  Available now under GPL •  Development Milestone Release model enabling early

evaluation

MySQL Database 5.6: A Better MySQL.

The image cannot be

HIGH PERFORMANCE AUTOMATED FAILOVER & RECOVERY

DATA INTEGRITY DEV/OPS AGILITY

MySQL Replication: Development Priorities


The image cannot be

Translating to User Needs

•  Improves read consistency from slaves •  Reduces risk of data loss if master fails

PERFORMANCE

•  Minimize service interruption •  Self-Healing

FAILOVER & RECOVERY

•  Correct, consistent & accessible

DATA INTEGRITY

•  Reducing TCO •  Quickly responding to business demand

DEV/OPS AGILITY


The image cannot be

MySQL 5.6 Replication

•  Multi-Threaded Slaves •  Optimized Row-Based Replication

PERFORMANCE

•  Global Transaction Identifiers •  Replication Failover & Admin Utilities •  Crash Safe Slaves & Binlog

FAILOVER & RECOVERY

•  Replication Event Checksums

DATA INTEGRITY

•  Replication Utilities •  Time-Delayed Replication •  Informational Log Events •  Remote Binlog Backup •  Server UUIDs

DEV/OPS AGILITY


The image cannot be

PERFORMANCE

The image cannot be

5x Higher Performance with MySQL 5.6

0 5 10 QPS 58.11 144.4 282.53

0

50

100

150

200

250

300

Que

ries

per S

econ

d

Worker Threads

Multi-Threaded Slave Performance

The image cannot be

Multi-Threaded Slaves •  Increases slave throughput,

reducing lag •  All transactions received into

slave’s relay log •  Implements multiple SQL

threads, based on database •  Applies events to different

databases in parallel •  Great for systems which

isolate application data using databases – e.g. multi-tenant


SQL-A

I/O

Relay Log

T1 T2 T3 T4

SQL-B

SQL-C

DB-A DB-B DB-C

T1

T4

T2 T3

Coordinator

The image cannot be

Multi-Threaded Slaves

•  Throughput of slave increased by allowing multiple slave threads: •  0 – functionality disabled •  Up to 1024

• Exec_Master_Log_Posn in SHOW SLAVE STATUS now shows the low-water-mark

•  Configure using:

slave-parallel-workers=4 •  Only ensures sequencing within a database

•  Unsuitable if ordering of updates between databases is not required


The image cannot be

Multi-Threaded Slaves

Master INSERT INTO dba.tab1...

INSERT INTO dba.tab2...

INSERT INTO dbb.tab1...

INSERT INTO dbc.tab2...


INSERT INTO dbb.tab3...


INSERT INTO dbc.tab2...


Slave thread1 dba.tab1 dba.tab2 dba.tab1 dba.tab3 dba.tab4

dbb.tab1 dbc.tab2 dbb.tab3 dbc.tab2

Slave thread2


•  Transactions applied in parallel on the slave

•  Sequencing of transactions within a given database will be the same

The image cannot be

Optimized Row Based Replication

•  Increases replication throughput for master and slave •  Reduces Binlog size, memory requirements & network

bandwidth

•  Only replicates those elements of the Row image that have changed


Primary Key Changed Columns

The image cannot be

Optimized Row Based Replication

•  Default is to include full before & after image for every changed row

•  New option: binlog-row-image=minimal •  Reduces space:

•  Inserts: only after image •  Deletes: only before image •  Updates: after image + primary key or indexed columns of

before image


The image cannot be

AUTOMATED FAILOVER & RECOVERY

The image cannot be

Global Transaction Identifiers

•  Foundation for reliable, automatic failover & recovery •  Unique identifier for each replication event written to the Binlog

•  Simple to track & compare replication across the cluster •  Automatically identify the most up-to-date slave for failover •  Deploy complex replication topologies •  Eliminates Dev/Ops overhead

Master

GTID=123456

GTID=123456

GTID=123456 GTID=123456


The image cannot be

Before Global Transaction IDs


Master

{my-bin.000132, 782}

{my-bin.000101, 873}

{my-bin.000099, 123}

{my-bin.00088, 1027}

The image cannot be

Before Global Transaction IDs Failover & Recovery


Master

{my-bin.000132, 782}

{my-bin.000101, 873}

{my-bin.000099, 123}

{my-bin.00088, 1027}

CHANGE MASTER TO??

Must convert {file,pos} from old master to one from the new master

The image cannot be

With Global Transaction IDs


Master

GTID=123456

GTID=123456

GTID=123456 GTID=123456

The image cannot be

With Global Transaction IDs Failover & Recovery


Master

GTID=123456

GTID=123456

GTID=123456 GTID=123456

CHANGE MASTER TO??

Slave(s) can start from same GTID

The image cannot be

Global Transaction ID Utilities •  Self-Healing, Low Administration replication cluster •  Automated failover & recovery

• mysqlfailover Utility

•  Switchover & administration • mysqlrpladmin Utility

•  Delivers HA within the core MySQL distribution •  Eliminates the need to integrate 3rd party HA frameworks •  Allows extensibility to support variety of HA mechanisms

The image cannot be

Replication Failover Utility

Failover Utility

Monitoring

•  Automatic failover & slave promotion •  Default is to promote most up-to-date

slave, based on GTID •  Slave promotion policies are fully

configurable; select slave(s) that are candidates for promotion

•  Implemented as a command-line utility

•  Option to bind in your own pre/post failover scripts

•  For example, initiate VIP failover

•  Never lose replicated events •  Nominated slave automatically retrieves any

missing updates from other slaves


Fail-Over

Failed Master

Slaves

Auto-Failover & Slave Promotion

Promoted Master

The image cannot be

Use of mysqlfailover $ mysqlfailover --master=root@rod:3306 --discover-slaves-login=root MySQL Replication Monitor and Failover Utility Failover Mode = auto Next Interval = Mon Mar 19 15:56:03 2012 Master Information ------------------ Binary Log File Position Binlog_Do_DB Binlog_Ignore_DB mysql-bin.000001 571

Replication Health Status +------------+-------+---------+--------+------------+---------+ | host | port | role | state | gtid_mode | health | +------------+-------+---------+--------+------------+---------+ | rod | 3306 | MASTER | UP | ON | OK |

| jane | 3306 | SLAVE | UP | ON | OK | | jane | 3307 | SLAVE | UP | ON | OK | | freddy | 3306 | SLAVE | UP | ON | OK | +------------+-------+---------+--------+------------+---------+ Q-quit R-refresh H-health G-GTID Lists U-UUIDs L-log entries


Fail-Over

The image cannot be

mysqlfailover – master failed Failover starting... # Candidate slave jane:3306 will become the new master. # Preparing candidate for failover. # Creating replication user if it does not exist. # Stopping slaves. # Performing STOP on all slaves. # Switching slaves to new master. # Starting slaves. # Performing START on all slaves. # Checking slaves for errors. # Failover complete. # Discovering slaves for master at jane:3306 Failover console will restart in 5 seconds.


Fail-Over

The image cannot be

mysqlfailover – monitoring resumes

MySQL Replication Monitor and Failover Utility Failover Mode = auto Next Interval = Mon Mar 19 16:05:12 2012 Master Information ------------------ Binary Log File Position Binlog_Do_DB Binlog_Ignore_DB mysql-bin.000001 1117 UUIDs +------------+-------+---------+--------+------------+---------+ | host | port | role | state | gtid_mode | health | +------------+-------+---------+--------+------------+---------+ | jane | 3306 | MASTER | UP | ON | OK | | jane | 3307 | SLAVE | UP | ON | OK | | freddy | 3306 | SLAVE | UP | ON | OK | +------------+-------+---------+--------+------------+---------+


Fail-Over

The image cannot be

Replication Administration Utility •  Perform switchover to eliminate

downtime during planned maintenance

•  Start and stop slaves

•  Slave discovery & monitoring

•  Slave status, thread status •  Replication processing, including any lag •  Configure slave promotion policies

•  Non GTID-functionality can be used with pre-MySQL 5.6 releases

Master

Slaves

Administration Utility

Status & Switchover


Admin

The image cannot be

Planned switchover to new master with mysqlrpladmin

$ mysqlrpladmin --master=root@rod:3306 --slaves=root@jane:3306,root@jane:3307,root@freddy:3306 --new-master=root@jane:3306 --demote-master switchover # Performing switchover from master at rod:3306 to slave at jane:3306.

# Checking candidate slave prerequisites. # Waiting for slaves to catch up to old master. # Stopping slaves. # Performing STOP on all slaves. # Demoting old master to be a slave to the new master. # Switching slaves to new master.

# Starting all slaves. # Performing START on all slaves. # Checking slaves for errors. # Switchover complete. # ...done.


Switch-Over

The image cannot be

Crash-Safe Slaves & Binlog

Before: –  Transaction Data: in tables –  Replication Info: in files


MySQL 5.6 –  Transaction Data: in tables –  Replication Info: in tables

Data

Position Info

CRASH!

Time

Data

Position Info Time

•  Automatic recovery of a slave and Binlog after a failure •  Binlog and table data are transactionally

consistent

•  Resumes replication without Dev/Op intervention •  Automatically rolling back replication to

last committed event

•  Eliminates risk of data loss or corruption

Atomic

Atomic

The image cannot be

Crash-Safe Slaves

•  Writes and reads back only complete events or transactions from the binary log

• master.info & relay-log.info files can be replaced with tables •  ACID transaction prevent them getting out of sync with InnoDB

tables

--master-info-repository=TABLE --relay-log-info-repository=TABLE mysql.slave_master_info mysql.slave_relay_log_info


The image cannot be

DATA INTEGRITY

The image cannot be

Replication Event Checksums

•  Ensures replicated data is correct, consistent and accessible

•  Detects corrupt replication events before they’re applied •  Returns an error

•  Protects entire replication path •  Memory •  Disk •  Network •  Bugs

Master

#

Slave

#


The image cannot be

Replication Event Checksums

•  Implemented in the binary and relay logs •  New mysqld options:

• binlog-checksum=CRC32 Checksums written to the binary log

• master-verify-checksum=1 Master validates checksum read from the binary log

• slave-sql-verify-checksum=1 Slave validates checksum when reading from the relay log


The image cannot be

DEV/OPS AGILITY

The image cannot be

MySQL Utilities •  Automate common Dev/Ops tasks

•  Replication: provisioning, testing, monitoring and failover •  Database comparisons: consistency checking •  Database administration: users, connections, tables, etc •  New utilities in development, ie log analysis

•  Implemented as Python scripts, plug-in for MySQL Workbench •  Also available from LaunchPad •  Extensible to include custom scripting

•  Resources: Documentation & Community Forum •  On-Demand Webinar: http://t.co/7bFAV083 •  http://dev.mysql.com/doc/workbench/en/mysql-utils-man.html •  http://forums.mysql.com/list.php?155


The image cannot be

Utility Workflow for Replication

•  Check: Verifies pre-requisites for Replication •  Repl: Initiates Replication to the new slave •  Show: Display Replication topology •  Fail-Over & Admin: Detects and failovers (or switches)

from master to slave. Status monitoring

Check Repl Show Fail-Over & Admin

The image cannot be

Time Delayed Replication


•  Configure time period before replication events applied to slave •  Per-slave, via execution of SQL

Thread •  Second-level granularity, up to 68

years!

•  Protects against operational error •  Dropping a table, etc.

•  Allows database to be inspected without loading a back-up

Master :3306

Binary Logs

Relay Log

Relay Log

Slave 1 :3307

Slave2 :3308

10 Minute Delay

The image cannot be

Time Delayed Replication - configuring

slave2> CHANGE MASTER TO -> MASTER_HOST = 'localhost‘, -> MASTER_PORT = 3306, -> MASTER_USER = 'repl_user‘, -> MASTER_PASSWORD = 'pw‘, -> MASTER_DELAY = 600;

slave2> START SLAVE;


The image cannot be

Time Delayed Replication – Doh!

slave2> STOP SLAVE; Breath slowly!

master> SHOW BINLOG EVENTS\G *************************** 12. row *************************** Log_name: ws2-bin.000001 Pos: 984 Event_type: Query Server_id: 1 End_log_pos: 1096 Info: use `clusterdb`; DROP TABLE `towns` /* generated by server */

slave2> START SLAVE UNTIL -> MASTER_LOG_FILE='ws2-bin.000001', -> MASTER_LOG_POS=984;


The image cannot be

Informational Log Events

•  Simplifies debugging and auditing when using Row Based Replication •  Original query written to the Binlog •  Replicated with row event to the slave

•  Activate with

binlog-rows-query-log-events=TRUE master> INSERT INTO simples VALUES (20),(21),(22);


The image cannot be


BEFORE: mysqlbinlog (-v): ### INSERT INTO clusterdb.simples ### SET ### @1=20 /* INT meta=0 nullable=0 is_null=0 */ ### INSERT INTO clusterdb.simples ### SET ### @1=21 /* INT meta=0 nullable=0 is_null=0 */ ### INSERT INTO clusterdb.simples ### SET ### @1=22 /* INT meta=0 nullable=0 is_null=0 */


The image cannot be


AFTER: mysqlbinlog (-vv): # at 443 #111128 16:04:24 server id 1 end_log_pos 504 Rows_query

# INSERT INTO simples VALUES (20),(21),(22) # at 504


The image cannot be

Other Useful Enhancements

•  Remote BinLog Backup •  Creates real-time backup of binary log on remote server •  Eliminates requirement for:

•  MySQL instance to parse binary log into SQL statements •  User needing privileges to each master for binary log

backup

•  Universally Unique Server IDs •  Enables auto-discovery and analysis of replication topology •  Simplifies management of large and highly dynamic MySQL

replication environments

The image cannot be

How to Evaluate MySQL 5.6

http://dev.mysql.com/downloads/mysql/#downloads Select Development Release tab


The image cannot be

Next Steps •  Evaluate the new features

•  DevZone Article: http://dev.mysql.com/tech-resources/articles/mysql-5.6-replication.html •  Questions & Feedback: forums.mysql.com/list.php?26 •  Bugs: bugs.mysql.com/

•  Replication documentation •  dev.mysql.com/doc/refman/5.6/en/replication.html


MySQL 5.6 Replication: Supporting High Scale Web & Cloud Services

The image cannot be

Remote Binary Log Back-up


MySQL Master

MySQL Slave

Relay Log

Backup

Binary Log Backup

•  Enhances operational efficiency •  Creates real-time backup of binary

log on remote server •  Adds a “raw” flag to the mysqlbinlog command

•  Sent via the replication channel

•  No need for….. •  MySQL instance to parse binary log

into SQL statements •  User needing privileges to each

master for binary log backup

The image cannot be

Remote Binary Log Backup

• Writes to a local file with the same name as the original

$> mysqlbinlog --read-from-remote-server --raw –h secret_server -P 3306 -u root mysql-bin.000001


The image cannot be

Universally Unique Server IDs

•  Enables auto-discovery and analysis of replication topology •  UUID automatically generated for each new server that

becomes part of the replication cluster •  Can be viewed with systems monitoring tools, i.e.

MySQL Enterprise Monitor

•  Simplifies management of large and highly dynamic MySQL replication environments


!

The image cannot be

Universally Unique Server IDs

•  A UUID generated when the MySQL server first starts •  Stored in auto.cnf file •  Available as: • server_uuid system variable for local host • SHOW SLAVE HOSTS for UUIDs of slaves • SHOW SLAVE STATUS for UUIDs of master

slave> SHOW SLAVE STATUS\G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event

....

Master_UUID: e43ac311-19cc-11e1-a06e-002318c418fd ....


The image cannot be

How to get Utilities •  Available on Launchpad

• https://launchpad.net/mysql-utilities • bzr branch lp:mysql-utilities • Requires Connector/Python

• https://launchpad.net/myconnpy • bzr branch lp:myconnpy

•  Available as a plugin in MySQL Workbench • http://www.mysql.com/downloads/workbench/

•  Documentation is here: • http://dev.mysql.com/doc/workbench/en/mysql-utilities.html

my sql 5.6_replwebinar_may12

Technology

row image

mysql replication

image updates

changedthe image

image deletes

performancethe image

multitenantthe image

replication throughput