tungsten university: configure and provision tungsten clusters

40
©Continuent 2013 Tungsten University: Configure & provision Tungsten clusters Je! Mace, Neil Armitage

Upload: continuent

Post on 28-Jun-2015

304 views

Category:

Technology


0 download

DESCRIPTION

Are you unsure of the steps needed to get your Continuent Tungsten cluster up-and-running? In this virtual course, we teach you how to get from a single database server to a scalable cluster, or from a brittle MySQL replication system to a transparent, manageable Tungsten cluster. We discuss the benefits of leveraging Continuent Tungsten clustering with MySQL, and walk you through the steps to implement a Tungsten cluster in Amazon EC2. We cover the prerequisites, installing and configuring Tungsten, and best practices that are part of most production installations and proof-of-concepts. Course Topics - Configuring MySQL and the OS for proper installation - Installing a cross-site cluster - Schema upgrade on the master database server with minimal application downtime (switch operation) - Automated fail over when a MySQL database server crashes - Recovery of a failed master to a fully operational slave with a single command (recover operation) - Switching database operations to a remote site (geo-clustering, cross-site 'switch' operation) We will also discuss and demonstrate basic operations, such as adding and removing a cluster node, basic monitoring and troubleshooting, and discuss the basic failure scenarios. Learn how to quickly configure and provision highly optimized Continuent Tungsten deployments in the cloud or on-premises!

TRANSCRIPT

Page 1: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Tungsten University: Configure & provision

Tungsten clustersJe! Mace, Neil Armitage

Page 2: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

About Continuent

• The leading provider of clustering and replication for open source DBMS

• Tungsten Clustering - Commercial-grade HA, performance scaling and data management for MySQL

• Tungsten Replication - Flexible, high-performance replication

2

Page 3: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Introducing Tungsten

3

Page 4: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Application Application

Replicator Replicator Replicator

MySQL Client API MySQL Client API

MasterSlave Slave

Existing MySQL Replication

4

Page 5: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Application Application

Replicator Replicator Replicator

MySQL Client API MySQL Client API

MasterSlave Slave

Step 1: Replace MySQL Replication

5

Page 6: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Application Application

Replicator Replicator Replicator

MySQL Client API MySQL Client API

Manager Manager Manager

MasterSlave Slave

Step 2: Add Manager Process

6

Page 7: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Application Application

Replicator Replicator Replicator

Manager Manager Manager

Tungsten Connector Tungsten Connector

MasterSlave Slave

Step 3. Add Intelligent Connector

7

Page 8: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Data Service

Application

Replicator Replicator Replicator

Application

Manager Manager Manager

Monitoring and control

Monitoring and control

MasterSlave Slave

Step 4. Connector Connectivity and Communication

Tungsten Connector Tungsten Connector

8

Page 9: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Application Application

NYC

Page 10: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013 10

Manual Failover

NYC London

Page 11: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Evaluating Tungsten

• Con"guring servers

• Viewing cluster status

• Connectivity

• Testing cluster operations

• Next steps

• Questions

11

Page 12: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Con"guring Servers

12

Page 13: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Create a Security Group

• Create a security group for all Tungsten servers

• It should be created in the AZ you will use for servers

13

All TCP traffic within the security groupAll UDP traffic within the security groupAll ICMP traffic within the security groupTCP port 22, 9999 and 13306 from '0.0.0.0/0'

Page 14: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Launch Servers

• Create 5 EC2 servers in a single AZ

• They should be m1.large or greater

• Use the Amazon Linux AMI

• Set the root volume to be large enough for your test data set

14

Page 15: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Logging Into the Servers

# SSH access must be done to the ec2-user account$> ssh [email protected]

# Then use sudo to gain access to the root user$ ip-184-72-189-135> sudo su -

15

Page 16: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Set Server Hostnames

# We will use the following hostnames# db1.nyc.tu# db2.nyc.tu# db3.nyc.tu# db1.london.tu# db2.london.tu

$ ip-184-72-189-135> hostname db1.nyc.tu$ ip-184-72-189-135> sed -i "/^HOSTNAME=/c\HOSTNAME=`hostname`" /etc/sysconfig/network

# You must logout completely and back in # for the change to take effect

16

Page 17: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Modify /etc/hosts

• Add entries to /etc/hosts on each server

• Use the private IP address for each host

17

$ db1> /sbin/ifconfig eth0 | grep "inet addr" inet addr: 10.112.24.214 Bcast: 10.112.25.255 Mask:255.255.254.0

$ db1> echo "10.112.24.214 db1.nyc.tu10.10.219.125 db2.nyc.tu10.242.134.18 db3.nyc.tu10.10.102.83 db1.london.tu10.112.74.196 db2.london.tu" >> /etc/hosts

Page 18: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Install Software Packages

$ db1> yum -y install mysql-server which curl bc rsync wget java-1.6.0-openjdk ruby

$ db1> rpm -i http://www.percona.com/redir/downloads/XtraBackup/LATEST/RPM/rhel6/x86_64/percona-xtrabackup-2.0.4-484.rhel6.x86_64.rpm

18

Page 19: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Create MySQL Users

$ db1> mysql -e "grant all on *.* to 'tungsten'@'%' identified by 'secret' with grant option"

$ db1> mysql -e "grant all on *.* to 'app'@'%' identified by 'secret'"$ db1> mysql -e "revoke super on *.* from 'app'@'%'"

$ db1> mysql -e "delete from mysql.user where user=''"$ db1> mysql -e "flush privileges"

19

Page 20: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Rinse & Repeat

• The requirements must be completed on each server

• Use a di!erent server-id

• Use the same SSH id_rsa and id_rsa.pub

• Full details at https://docs.continuent.com/wiki/display/TEDOC/System+Requirements

• Sample Install scripts will be posted on docs.continuent.com

20

Page 21: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Installation

21

• Installation completed from a staging directory using tungsten-cookbook or tpm

• tungsten-cookbook runs tpm plus some additional tests

• Staging con"guration stored in $CONTINUENT_PROFILES

Page 22: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Installation

22

• Installation completes several steps

• Copy software to each server

• Validate con"guration

• Write con"guration "les

• Start services

Page 23: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Viewing Cluster Status

23

$ db1> trepctl status

$ db1> cctrl -multiTungsten Enterprise 1.5.3 build 59connect to '[email protected]'usa: session established

[LOGICAL] / > ls

[LOGICAL] / > use usa[LOGICAL] /usa > ls

[LOGICAL] /usa > use europe[LOGICAL] /europe > ls

[LOGICAL] /europe > use world[LOGICAL] /world > ls

Page 24: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Connectivity

24

$ db1> mysql -h`hostname` -P9999 -uapp -pmysql> select @@hostname;

[LOGICAL] /usa > switchmysql> select @@hostname;

mysql> begin;mysql> select @@hostname;[LOGICAL] /usa > switchmysql> select @@hostname;

Page 25: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Read/Write Splitting

$ db1> vi $CONTINUENT_ROOT/tungsten/tungsten-connector/conf/user.map

mysql> select @@hostname;

mysql> select @@hostname for update;

mysql> begin; select @@hostname; rollback;

25

Page 26: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Testing Cluster Operations

26

Page 27: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Switching the Master Server

27

$ db1> cctrl -multi[LOGICAL] /> use usa[LOGICAL] /usa> switch[LOGICAL] /usa> ls

[LOGICAL] /usa> switch to db1.nyc.tu[LOGICAL] /usa> ls

[LOGICAL] /usa> use world[LOGICAL] /world> ls

[LOGICAL] /world> switch to europe[LOGICAL] /world> ls

[LOGICAL] /world> use europe[LOGICAL] /europe> ls

Page 28: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Automatic Failover

$ db1> cctrl -multi[LOGICAL] /> use europe[LOGICAL] /europe> lsmysql> select @@hostname for update;

$ db1> ssh db1.london.tu sudo /sbin/service mysqld stop[LOGICAL] /europe> lsmysql> select @@hostname for update;

$ db1> ssh db1.london.tu sudo /sbin/service mysqld start[LOGICAL] /europe> datasource db1.london.tu recover[LOGICAL] /europe> ls

[LOGICAL] /europe> switch to db1.london.tu[LOGICAL] /europe> ls

28

Page 29: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Site Failover

$ db1> ssh db1.london.tu /opt/continuent/tungsten/cluster-home/bin/stopall;ssh db2.london.tu /opt/continuent/tungsten/cluster-home/bin/stopall;ssh db1.london.tu sudo /sbin/service mysqld stop;ssh db2.london.tu sudo /sbin/service mysqld stop

$ db1> cctrl -multi[LOGICAL] /> use world[LOGICAL] /world> ls

[LOGICAL] /world> datasource europe fail

[LOGICAL] /world> failover[LOGICAL] /world> ls

29

Page 30: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Site Recovery

$ db1> ssh db1.london.tu /opt/continuent/tungsten/cluster-home/bin/startall;ssh db2.london.tu /opt/continuent/tungsten/cluster-home/bin/startall;ssh db1.london.tu sudo /sbin/service mysqld start;ssh db2.london.tu sudo /sbin/service mysqld start

$ db1> cctrl -multi[LOGICAL] /> use europe[LOGICAL] /europe> ls

[LOGICAL] /europe> use world[LOGICAL] /world> recover using db1.london.tu[LOGICAL] /world> ls

30

Page 31: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Online Upgrades

31

Page 32: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Upgrade the Slaves

32

[LOGICAL] /usa> switch to db1.nyc.tu[LOGICAL] /usa> datasource db2.nyc.tu backup[LOGICAL] /usa> datasource db2.nyc.tu shun$ db1> mysql -hdb2.nyc.tu -P13306 -utungsten -p

mysql> # Apply backwards compatible changes to db2.nyc.tu

# If unsuccessful[LOGICAL] /usa> datasource db2.nyc.tu restore# Restart the process

# If successful[LOGICAL] /usa> datasource db2.nyc.tu welcome# Repeat the above steps for all slaves

Page 33: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Upgrade the Master

[LOGICAL] /usa> switch to db2.nyc.tu[LOGICAL] /usa> datasource db1.nyc.tu backup[LOGICAL] /usa> datasource db1.nyc.tu shun$ db1> mysql -hdb2.nyc.tu -P13306 -utungsten -p

mysql> # Apply backwards compatible changes to db1.nyc.tu

# If unsuccessful[LOGICAL] /usa> datasource db1.nyc.tu restore# Restart the process

# If successful[LOGICAL] /usa> datasource db1.nyc.tu welcome[LOGICAL] /usa> switch to db1.nyc.tu

33

Page 34: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Changing Tungsten Con"guration

34

• Con"guration changes are made through tpm in the staging directory

• Use tpm configure to make changes

• Use tpm update to push changes out to each server and restart services

Page 35: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Changing Tungsten Con"guration

$ db1> cd /opt/continuent/software$ db1> cd tungsten-enterprise-1.5.3-89$ db1> ./tools/tpm configure usa \--dataservice-hosts=db1.nyc.tu,db2.nyc.tu,db3.nyc.tu \--dataservice-connectors=db1.nyc.tu,db2.nyc.tu,db3.nyc.tu$ db1> ./tools/tpm update world

$ db1> cctrl -multi

35

Page 36: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Next Steps

• Register at http://www.continuent.com/downloads/software

• Initiate a POC with Continuent

• Testing production hardware and data sets

• Application testing

• Integration into monitoring and alerting

• Training for operations personnel

36

Page 37: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Questions

37

Page 38: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Feedback

38

• Send any feedback to

[email protected]

Page 39: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013

Tungsten University Sessions

39

• Con"gure & provision Tungsten clustersTuesday, January 22 @ 15:00 GMT/16:00 CET

• Setup & operate Tungsten ReplicatorThursday, January 31 @ 10 am PT/1 pm ET

• Setup & operate Tungsten ReplicatorTuesday, February 5 @ 15:00 GMT/16:00 CET

Page 40: Tungsten University: Configure and provision Tungsten clusters

©Continuent 2013 40

Continuent Website:http://www.continuent.com

Tungsten Replicator 2.0:http://code.google.com/p/tungsten-replicator

Our Blogs:http://scale-out-blog.blogspot.comhttp://datacharmer.blogspot.comhttp://flyingclusters.blogspot.comhttp://continuent-tungsten.blogspot.com

560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel +1 (866) 998-3642 Fax +1 (408) 668-1009e-mail: [email protected]