mysql enterprise backup (meb)

MySQL Enterprise Backup (MEB)

Remote DBA Team

Overview

• MySQLConsulting• MySQLSupport• RemoteDBAsupport.• ExpertMySQLsolutions.• 24/7MySQLMonitoringandSupport• MariaDB,Percona,Galera,TokuDBaresupportedtoo.

Overview

•  MySQL Enterprise Backup provides enterprise-grade backup and recovery for MySQL. It delivers hot, online, non-blocking backups on multiple platforms.

•  This can be primarily used to take hot backup of InnoDB

tables. This can also backup MyISAM and other non-InnoDB tables, however that is performed by locking tables.

•  The backup of other non-InnoDB & non-MyISAM tables

taken is not guaranteed to be consistent.

Important Features

•  "Hot" Online Backups •  High Performance •  Incremental, Partial, full instance backup •  Fast Recovery •  Advanced LZ4 Compression •  Direct Cloud Storage Backup •  Online "Hot" Selective Restore •  Backup Validation •  Table renaming

Backup and Restore

●  The backup and restore involves conceptually three

steps:

(1) backup operation.

(2) Apply log operation to prepare backup for restore.

(3) copy-back operation to restore backup to server repository.

Option Files Read

•  Default options are read from the following files in the

given order

/etc/my.cnf

/etc/mysql/my.cnf

/usr/local/mysql/etc/my.cnf

~/.my.cnf

•  This can be overridden as --defaults-file=/file/path

Operations Supported

•  Backup:

performs backup to the backup directory, result is raw backup.

•  Backup-to-image: performs backup to single backup image.

•  Apply-log: Prepares the backup for restore.

•  Backup-and-Apply-log: performs the backup and apply-log in a single step.

•  Apply-incremental-backup: applies incremental backup on full backup.

Operations Supported

•  copy-back: Restores data, index, log files from backup directory to server repository. Server should be offline.

•  copy-back-and-apply-log: Restores a single-file image backup or raw backup into datadir and then applies log.

•  image-to-backup-dir: Extracts the given backup image contents to specified directory.

•  backup-dir-to-image: Creates backup image of given backup dir.

Operations Supported

•  list-image:

List the content of backup image.

•  extract: Extracts the contents of given backup image.

•  validate: Checks for corruption in the backup image by comparing the checksums.

Option Classification

•  The options are classified for the ease of remembrance

1.  Standard Options

2.  Connection Options

3.  Server Repository Options

4.  Backup Repository Options

5.  Parallelization Options

6.  Encryption Options

7.  Other options

Usage

•  The command usage is as below

# MEB [STD-OPTIONS]

[CONNECTION-OPTIONS] [SERVER-REPOSITORY-OPTIONS] [BACKUP-REPOSITORY-OPTIONS] [PARALLELIZATION-OPTIONS] [ENCRYPTION-OPTIONS] [OTHER-OPTIONS] OPERATION

Standard Options

•  This option may be given at the first.

--print-defaults --no-defaults --defaults-file=PATH --help --force --show-progress

Connection Options

•  They define the options to connect to the server.

-u, --user=name -p, --password= --host=name --port=# --protocol=name --socket=name --connect_timeout --connect-if-online

Server Repository Options

•  Provides information regarding the server.

--datadir=PATH --innodb_data_file_path=VALUE

--innodb_data_home_dir=PATH --innodb_log_group_home_dir=PATH --innodb_log_files_in_group=N --innodb_log_file_size=SIZE --innodb_page_size=SIZE --innodb_checksum_algorithm=NAME

Backup Repository Options

•  Provides information regarding the backup destination.

--backup_dir=PATH --backup_innodb_data_home_dir=PATH --backup_innodb_log_group_home_dir=PATH --backup_innodb_log_files_in_group=N

--backup_innodb_log_file_size=SIZE --backup_innodb_page_size=SIZE --backup_innodb_checksum_algorithm=NAME

Parallelization Options

•  These options speed up the backup process

--number-of-buffers=NUMBER --process-threads=NUMBER --read-threads=NUMBER --write-threads=NUMBER

Encryption Options

•  These options define encryption.

--encrypt [The encryption method is AES-256-CBC] --decrypt --key=KEY --key-file=FILENAME

Other Options

•  These contains some of the important option. --backup-image=path --compress --compress-level=LEVEL [0-9] --compress-method=ALGORITHM[def LZ4] --exclude-tables=REGEXP --include-tables=REGEXP --incremental --incremental_backup_dir=PATH --incremental_base=BACKUP --start-lsn=LSN

Other Options

•  These contains some of the important option. --incremental-with-redo-log-only --limit-memory=MB --only-innodb --on-disk-full={abort|warn|abort_and_remove}] --slave-info --uncompress --use-tts={with-minimum-locking|with-full-locking}] --include=REGEXP --databases=LIST

Full Backup

•  By default when you invoke the MEB it goes for a full

backup # MEB/bin/mysqlbackup -u backup -p --datadir=/var/lib/mysql --backup-dir=/var/lib/backup backup

Backup Progression

Backup Progression

Backup Progression

Backup dir contents

•  Below are the content of the backup dir.

Preparing Phase

● In prepare phase log is applied

# MEB/bin/mysqlbackup --backup-dir=/var/lib/backup apply-log

Preparing Phase

Restore of backup

•  `copy-back` option is used to restore the backup to the

datadir mentioned in the cnf file or under [mysqlbackup].

Note: The server should be offline before restoring

# MEB/bin/mysqlbackup --backup-dir=/var/lib/backup copy-back

Restore of Backup

Backup-and-apply-log

•  Back and preparation can be combined as below.

# MEB/bin/mysqlbackup --datadir=/var/lib/mysql --backup-dir=/var/lib/backup backup-and-apply-log

Restore and prepare

•  The restore and prepare phase can be combined as

below # MEB/bin/mysqlbackup --datadir=/var/lib/mysql --backup-dir=/var/lib/backup copy-back-and-apply-log

•  Server should be offline

Incremental backups

•  Incremental backup copies the changes made after the

previous full backup(base) or by using the LSN. # MEB/bin/mysqlbackup --datadir=/var/lib/mysql --incremental --incremental-backup-dir=/var/lib/inc --incremental-base=dir:/var/lib/backup backup # MEB/bin/mysqlbackup --datadir=/var/lib/mysql --incremental --start-lsn=424059414 --with-timestamp --incremental-backup-dir=/var/lib/inc backup

Applying Incremental backup to full backup

•  Step 1: The full backup should be prepared first

# MEB/bin/mysqlbackup --backup-dir=/var/lib/backup apply-log

•  Step 2: Now apply the incremental backup to the prepared full backup in sequence. # MEB/bin/mysqlbackup --backup-dir=/var/lib/backup --incremental-backup-dir=/var/lib/inc apply-incremental-backup

Redo log backup

•  The redo log backup copies the changes in the redo-logs

based on the LSN.

# MEB/bin/mysqlbackup --incremental-with-redo-log-only --start-lsn=431175291 --with-timestamp --incremental-backup-dir=/var/lib/inc backup

# MEB/bin/mysqlbackup --incremental-with-redo-log-only --incremental-base=dir:/var/lib/backup --with-timestamp --incremental-backup-dir=/var/lib/inc backup

Redo log backup

•  Note: For redo-log backup the LSN should exit in the

circular logs, if the LSN is overwritten, you will get the error as below.

•  Error:

mysqlbackup: ERROR: Cannot do incremental redo-only backup of InnoDB tables: InnoDB log at LSN 424059414 is already overwritten. InnoDB saves the only the latest redo log records in InnoDB log files. Older records are overwritten when their distance from the tail of the log exceeds the combined size of InnoDB log files.

Speeding up the backup

•  The Backup process can be increased by allocating

resources efficiently # MEB/bin/mysqlbackup -u backup -p --datadir=/var/lib/mysql --limit-memory=400 --process-thread=8 --read-thread=2 --write-thread=2 --backup-dir=/var/lib/backup backup

Backup log: ----------- 160419 18:58:52 mysqlbackup: INFO: Full Backup operation starts with following threads 2 read-threads 8 process-threads 2 write-threads

Compression

•  MEB support compression at various levels of 0-9 , The

default Algorithm is lz4 supports zlib and lzma # MEB/bin/mysqlbackup -u backup -p --datadir=/var/lib/mysql --compress --compress-level=6 --compress-method=zlib --process-threads=8 --read-threads=2 --write-threads=2 --backup-dir=/var/lib/backup backup Uncompressed backup size: 255M /tmp/var/lib/backup Compressed backup size: 58M /var/lib/backup

Decompress

•  Decompression of backup happens along with the

prepare phase as below.

# MEB/bin/mysqlbackup --uncompress --process-threads=8 --read-threads=2 --write-threads=2 --backup-dir=/var/lib/backup apply-log

Single File backup

•  The Backup file ends with extension `.mbi` the operation

used is `backup-to-image` # MEB/bin/mysqlbackup -u backup -p --datadir=/var/lib/mysql --with-timestamp --backup-image=backup.mbi --backup-dir=/var/lib/backup backup-to-image File: /var/lib/backup/2016-04-19_19-28-14/backup.mbi

List and Extract single file backup

•  list the content of the backup file as below

# MEB/bin/mysqlbackup --backup-image=/var/lib/backup/2016-04-19_19-28-14/backup.mbi list-image

•  To extract the content as below, by def it is extracted to the current directory. # MEB/bin/mysqlbackup --backup-image=/var/lib/backup/2016-04-19_19-28-14/backup.mbi --backup-dir=/var/lib/backup extract

Streaming single file backup

•  The single-file backup could be streamed to a remote

host # MEB/bin/mysqlbackup --user=backup --password='backup' --datadir=/var/lib/mysql --backup-dir=/var/lib/backup --backup-image=- --compress backup-to-image | ssh mydbops@xx.xx.xxx -p 2345 '/home/mydbops/meb/bin/mysqlbackup --backup-dir=/home/mydbops/mysqlbackup --datadir=/home/mydbops/data --innodb_log_group_home_dir=. --innodb_log_files_in_group=2 --innodb_log_file_size=268435456 --uncompress --backup-image=- copy-back-and-apply-log'

Streaming single file backup

●  Backup transferred:

[mydbops@master.mydbops.com ~$] du -sh /home/mydbops/data 855M /home/mydbops/data

Encrypting backup

•  MEBsupportshighlevelsecuritytothebackupbyencryp8ngthemwithAES,Encryp8onworksonthebackupimages.

Crea8ngkey-file:-----------------------#opensslrand32-hex>meb_keyfile

Thiskeyisusedforbothencryp8on&decryp8on

Encrypting backup

•  Below is the command which provides encryption.

# MEB/bin/mysqlbackup -u backup -p --datadir=/var/lib/mysql --backup-image=encrpt_backup.enc --backup-dir=/var/lib/backup --encrypt --key-file=meb_keyfile backup-to-image

Decrypting Backup

•  The same key used for encryption has to be used for

decryption of the backup.

# MEB/bin/mysqlbackup --backup-image=/var/lib/backup/encrpt_backup.enc --backup-dir=/var/lib/backup --with-timestamp --decrypt --key-file=meb_keyfile extract

Highlights

•  MySQL Enterprise ( Proprietary ) •  Written in C •  Supports Windows , Unix and other platforms •  Faster and secure •  Compression. •  Incremental and full backup.

Note : Xtrabackup is an good alternative too

Thank You