mysql enterprise backup (meb)
TRANSCRIPT
MySQL Enterprise Backup (MEB)
Remote DBA Team
Overview
• MySQLConsulting• MySQLSupport• RemoteDBAsupport.• ExpertMySQLsolutions.• 24/7MySQLMonitoringandSupport• MariaDB,Percona,Galera,TokuDBaresupportedtoo.
Overview
• MySQL Enterprise Backup provides enterprise-grade backup and recovery for MySQL. It delivers hot, online, non-blocking backups on multiple platforms.
• This can be primarily used to take hot backup of InnoDB
tables. This can also backup MyISAM and other non-InnoDB tables, however that is performed by locking tables.
• The backup of other non-InnoDB & non-MyISAM tables
taken is not guaranteed to be consistent.
Important Features
• "Hot" Online Backups • High Performance • Incremental, Partial, full instance backup • Fast Recovery • Advanced LZ4 Compression • Direct Cloud Storage Backup • Online "Hot" Selective Restore • Backup Validation • Table renaming
Backup and Restore
● The backup and restore involves conceptually three
steps:
(1) backup operation.
(2) Apply log operation to prepare backup for restore.
(3) copy-back operation to restore backup to server repository.
Option Files Read
• Default options are read from the following files in the
given order
/etc/my.cnf
/etc/mysql/my.cnf
/usr/local/mysql/etc/my.cnf
~/.my.cnf
• This can be overridden as --defaults-file=/file/path
Operations Supported
• Backup:
performs backup to the backup directory, result is raw backup.
• Backup-to-image: performs backup to single backup image.
• Apply-log: Prepares the backup for restore.
• Backup-and-Apply-log: performs the backup and apply-log in a single step.
• Apply-incremental-backup: applies incremental backup on full backup.
Operations Supported
• copy-back: Restores data, index, log files from backup directory to server repository. Server should be offline.
• copy-back-and-apply-log: Restores a single-file image backup or raw backup into datadir and then applies log.
• image-to-backup-dir: Extracts the given backup image contents to specified directory.
• backup-dir-to-image: Creates backup image of given backup dir.
Operations Supported
• list-image:
List the content of backup image.
• extract: Extracts the contents of given backup image.
• validate: Checks for corruption in the backup image by comparing the checksums.
Option Classification
• The options are classified for the ease of remembrance
1. Standard Options
2. Connection Options
3. Server Repository Options
4. Backup Repository Options
5. Parallelization Options
6. Encryption Options
7. Other options
Usage
• The command usage is as below
# MEB [STD-OPTIONS]
[CONNECTION-OPTIONS] [SERVER-REPOSITORY-OPTIONS] [BACKUP-REPOSITORY-OPTIONS] [PARALLELIZATION-OPTIONS] [ENCRYPTION-OPTIONS] [OTHER-OPTIONS] OPERATION
Standard Options
• This option may be given at the first.
--print-defaults --no-defaults --defaults-file=PATH --help --force --show-progress
Connection Options
• They define the options to connect to the server.
-u, --user=name -p, --password= --host=name --port=# --protocol=name --socket=name --connect_timeout --connect-if-online
Server Repository Options
• Provides information regarding the server.
--datadir=PATH --innodb_data_file_path=VALUE
--innodb_data_home_dir=PATH --innodb_log_group_home_dir=PATH --innodb_log_files_in_group=N --innodb_log_file_size=SIZE --innodb_page_size=SIZE --innodb_checksum_algorithm=NAME
Backup Repository Options
• Provides information regarding the backup destination.
--backup_dir=PATH --backup_innodb_data_home_dir=PATH --backup_innodb_log_group_home_dir=PATH --backup_innodb_log_files_in_group=N
--backup_innodb_log_file_size=SIZE --backup_innodb_page_size=SIZE --backup_innodb_checksum_algorithm=NAME
Parallelization Options
• These options speed up the backup process
--number-of-buffers=NUMBER --process-threads=NUMBER --read-threads=NUMBER --write-threads=NUMBER
Encryption Options
• These options define encryption.
--encrypt [The encryption method is AES-256-CBC] --decrypt --key=KEY --key-file=FILENAME
Other Options
• These contains some of the important option. --backup-image=path --compress --compress-level=LEVEL [0-9] --compress-method=ALGORITHM[def LZ4] --exclude-tables=REGEXP --include-tables=REGEXP --incremental --incremental_backup_dir=PATH --incremental_base=BACKUP --start-lsn=LSN
Other Options
• These contains some of the important option. --incremental-with-redo-log-only --limit-memory=MB --only-innodb --on-disk-full={abort|warn|abort_and_remove}] --slave-info --uncompress --use-tts={with-minimum-locking|with-full-locking}] --include=REGEXP --databases=LIST
Full Backup
• By default when you invoke the MEB it goes for a full
backup # MEB/bin/mysqlbackup -u backup -p --datadir=/var/lib/mysql --backup-dir=/var/lib/backup backup
Backup Progression
Backup Progression
Backup Progression
Backup dir contents
• Below are the content of the backup dir.
Preparing Phase
● In prepare phase log is applied
# MEB/bin/mysqlbackup --backup-dir=/var/lib/backup apply-log
Preparing Phase
Restore of backup
• `copy-back` option is used to restore the backup to the
datadir mentioned in the cnf file or under [mysqlbackup].
Note: The server should be offline before restoring
# MEB/bin/mysqlbackup --backup-dir=/var/lib/backup copy-back
Restore of Backup
Backup-and-apply-log
• Back and preparation can be combined as below.
# MEB/bin/mysqlbackup --datadir=/var/lib/mysql --backup-dir=/var/lib/backup backup-and-apply-log
Restore and prepare
• The restore and prepare phase can be combined as
below # MEB/bin/mysqlbackup --datadir=/var/lib/mysql --backup-dir=/var/lib/backup copy-back-and-apply-log
• Server should be offline
Incremental backups
• Incremental backup copies the changes made after the
previous full backup(base) or by using the LSN. # MEB/bin/mysqlbackup --datadir=/var/lib/mysql --incremental --incremental-backup-dir=/var/lib/inc --incremental-base=dir:/var/lib/backup backup # MEB/bin/mysqlbackup --datadir=/var/lib/mysql --incremental --start-lsn=424059414 --with-timestamp --incremental-backup-dir=/var/lib/inc backup
Applying Incremental backup to full backup
• Step 1: The full backup should be prepared first
# MEB/bin/mysqlbackup --backup-dir=/var/lib/backup apply-log
• Step 2: Now apply the incremental backup to the prepared full backup in sequence. # MEB/bin/mysqlbackup --backup-dir=/var/lib/backup --incremental-backup-dir=/var/lib/inc apply-incremental-backup
Redo log backup
• The redo log backup copies the changes in the redo-logs
based on the LSN.
# MEB/bin/mysqlbackup --incremental-with-redo-log-only --start-lsn=431175291 --with-timestamp --incremental-backup-dir=/var/lib/inc backup
# MEB/bin/mysqlbackup --incremental-with-redo-log-only --incremental-base=dir:/var/lib/backup --with-timestamp --incremental-backup-dir=/var/lib/inc backup
Redo log backup
• Note: For redo-log backup the LSN should exit in the
circular logs, if the LSN is overwritten, you will get the error as below.
• Error:
mysqlbackup: ERROR: Cannot do incremental redo-only backup of InnoDB tables: InnoDB log at LSN 424059414 is already overwritten. InnoDB saves the only the latest redo log records in InnoDB log files. Older records are overwritten when their distance from the tail of the log exceeds the combined size of InnoDB log files.
Speeding up the backup
• The Backup process can be increased by allocating
resources efficiently # MEB/bin/mysqlbackup -u backup -p --datadir=/var/lib/mysql --limit-memory=400 --process-thread=8 --read-thread=2 --write-thread=2 --backup-dir=/var/lib/backup backup
Backup log: ----------- 160419 18:58:52 mysqlbackup: INFO: Full Backup operation starts with following threads 2 read-threads 8 process-threads 2 write-threads
Compression
• MEB support compression at various levels of 0-9 , The
default Algorithm is lz4 supports zlib and lzma # MEB/bin/mysqlbackup -u backup -p --datadir=/var/lib/mysql --compress --compress-level=6 --compress-method=zlib --process-threads=8 --read-threads=2 --write-threads=2 --backup-dir=/var/lib/backup backup Uncompressed backup size: 255M /tmp/var/lib/backup Compressed backup size: 58M /var/lib/backup
Decompress
• Decompression of backup happens along with the
prepare phase as below.
# MEB/bin/mysqlbackup --uncompress --process-threads=8 --read-threads=2 --write-threads=2 --backup-dir=/var/lib/backup apply-log
Single File backup
• The Backup file ends with extension `.mbi` the operation
used is `backup-to-image` # MEB/bin/mysqlbackup -u backup -p --datadir=/var/lib/mysql --with-timestamp --backup-image=backup.mbi --backup-dir=/var/lib/backup backup-to-image File: /var/lib/backup/2016-04-19_19-28-14/backup.mbi
List and Extract single file backup
• list the content of the backup file as below
# MEB/bin/mysqlbackup --backup-image=/var/lib/backup/2016-04-19_19-28-14/backup.mbi list-image
• To extract the content as below, by def it is extracted to the current directory. # MEB/bin/mysqlbackup --backup-image=/var/lib/backup/2016-04-19_19-28-14/backup.mbi --backup-dir=/var/lib/backup extract
Streaming single file backup
• The single-file backup could be streamed to a remote
host # MEB/bin/mysqlbackup --user=backup --password='backup' --datadir=/var/lib/mysql --backup-dir=/var/lib/backup --backup-image=- --compress backup-to-image | ssh [email protected] -p 2345 '/home/mydbops/meb/bin/mysqlbackup --backup-dir=/home/mydbops/mysqlbackup --datadir=/home/mydbops/data --innodb_log_group_home_dir=. --innodb_log_files_in_group=2 --innodb_log_file_size=268435456 --uncompress --backup-image=- copy-back-and-apply-log'
Streaming single file backup
● Backup transferred:
[[email protected] ~$] du -sh /home/mydbops/data 855M /home/mydbops/data
Encrypting backup
• MEBsupportshighlevelsecuritytothebackupbyencryp8ngthemwithAES,Encryp8onworksonthebackupimages.
Crea8ngkey-file:-----------------------#opensslrand32-hex>meb_keyfile
Thiskeyisusedforbothencryp8on&decryp8on
Encrypting backup
• Below is the command which provides encryption.
# MEB/bin/mysqlbackup -u backup -p --datadir=/var/lib/mysql --backup-image=encrpt_backup.enc --backup-dir=/var/lib/backup --encrypt --key-file=meb_keyfile backup-to-image
Decrypting Backup
• The same key used for encryption has to be used for
decryption of the backup.
# MEB/bin/mysqlbackup --backup-image=/var/lib/backup/encrpt_backup.enc --backup-dir=/var/lib/backup --with-timestamp --decrypt --key-file=meb_keyfile extract
Highlights
• MySQL Enterprise ( Proprietary ) • Written in C • Supports Windows , Unix and other platforms • Faster and secure • Compression. • Incremental and full backup.
Note : Xtrabackup is an good alternative too
Thank You