mysql backups go near-continuous
TRANSCRIPT
![Page 1: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/1.jpg)
![Page 2: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/2.jpg)
R1Soft at a Glance
• High Performance Backup & Recovery– Linux & Windows servers & MySQL databases
• Near-Continuous Data Protection– Fast, backup (e.g .every 15 mins) & rapid restore
• Over 80,000 servers licensed• Used by major hosting companies• Division of BBS Technologies
– Based in Houston, Texas
– Systems management software for Windows & Linux
– 140+ staff , 3,000 customers
– 24*7 worldwide sales and support
– 49th fastest growing Technology company in US (Deloitte)
– 4th largest software company in Houston
![Page 3: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/3.jpg)
Topics
• Who Needs Data Protection?• Data Protection Ideas• Evaluating MySQL Backup Software• Available MySQL Backup Applications• Alpha MySQL Online Backup API• near-Continuous Backup for MySQL
![Page 4: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/4.jpg)
What Do We Protect Data From?
• Server Failure– Hard Drives Stop Spinning,
Storage Controllers Corrupt Data, etc
• Accidental Drop or Delete– The “Arggg Sh***!” factor
• Security Compromise– Server, Database, Application
• Business or Government Regulation– Historical Archiving Rules– HIPPA, EU Data Archiving Regulations
![Page 5: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/5.jpg)
Who Protects Data MySQL?
• Different Users Have Different Needs and Preferences for Data Protection
• Three Types of MySQL Administrators
1. Developers
2. Accidental Admin
3. Database Administration (DBA) Guru
![Page 6: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/6.jpg)
MySQL Developers
• Some develop on test systems - MySQL Data Protection Not Important
• In Practice Development often on live production environments! Oh My!
• Hidden Risk for Stakeholders• Single copy of PHP / Python / Perl Scripts Often Located on Server
PHP, Perl and Python used by ~85% of MySQL Developers http://dev.mysql.com/tech-resources/quickpolls/connectors-apis.html
![Page 7: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/7.jpg)
Accidental MySQL Admin
• Accidental MySQL Admin– Who Are They?
• Proprietor of Web 2.0 Business
• Hosting Business
• Network or System Administrators
• The Small Office PC Technician
– Simple is Better
– Binary Log, Clustering and Replication techniques often out of reach or unused
![Page 8: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/8.jpg)
MySQL DBA Guru
• MySQL DBA Guru– Often using Clusters and Replication
– Likes to performance tune and understand all details
– Custom data protection scripts and methods prevalent
– If a professional serving many clients simple data protection solutions desired
![Page 9: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/9.jpg)
MySQL Data Protection Ideas• RAID• Replication• Clusters• MySQL Backup and Restore
![Page 10: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/10.jpg)
RAID
• RAID Attributes– Software or Hardware RAID– Available on Windows, Linux, Unix– Some RAID levels protect against drive failure
• Good Data Protection?
Archiving None
Accidental Drop or Delete Data Loss!
Server Failure Data May Be Lost!
Security Compromise All Data Compromised!
![Page 11: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/11.jpg)
Replication
• One Writer and Many Readers– MySQL Master Slave Replication– Some Protection against server failure– Multiple copies of data– Replicates accidental data deletion!
• Good Data Protection?Archiving None
Accidental Drop or Delete Data Loss!
Server Failure Protected
Security Compromise Limited Protection
![Page 12: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/12.jpg)
Clusters• Many Writers and Many Readers
– Usually Linux / Unix Only
– Examples: DRBD, NDB, Continuent
– Protection against server failure
– Multiple copies of data
– Replicates accidental data deletion!
• Good Data Protection?
Archiving None
Accidental Drop or Delete Data Loss!
Server Failure Protected
Security Compromise Limited Protection
![Page 13: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/13.jpg)
Evaluating MySQL Backups• Archive Management?
– Does application easily manage historical archives
• Archive Type?– People either need disk or tape or both
• Supported Environment– What storage engines are supported?– What O/S’s are supported?
• Binlog Restore– Rollback to selected transaction using binary log
![Page 14: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/14.jpg)
Evaluating MySQL Backups Continued
• Zero Business Interruption (Hot Backup)– means no noticeable interruptions to connections or queries on
live database system
• Granular Restore– Means restore selected Databases or Tables to original or
alternate locations– Who Needs it Most?: Developers, DBA Guru
• Ease of Use– Graphical User Interface, Web Interface, etc– Who Needs it Most?: Developers, Accidental Admin
• Complete Server Protection– Does it just protect MySQL? Or Server Also?
![Page 15: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/15.jpg)
Evaluating MySQL Backups Continued 2
• Backup Type– Full
• All data transmitted and stored every backup– Incremental
• Complex combination of periodic full, and more frequent incremental backups
– Virtual Full Backups• One full backup, with deltas sent from then on. Each backup
appears as full backup, only deltas sent across network and stored.
• Backup Window– How long is the backup Window on Large Data Sets?
• Server I/O and Load Impact– Does it kill the server when data set is large?
![Page 16: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/16.jpg)
What Are Your Data Protection Priorities?
• Financial and Banking– Binlog Restore Critical– Tape Backup – Example:
• Typical Embedded OEM Network Device– Usually small databases– Proprietary device/app. Management handles backups using mysqldump or mysqlhotcopy – Exceptions are carrier grade clusters– Example:
• Massive Transactional Enterprise System– Master data located on mainframe or Unix not MySQL– Data replicated for read-only queries to MySQL– Use existing mainframe or Unix backup system for data protection– Example:
![Page 17: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/17.jpg)
What Are Your Data Protection Priorities?
• Web 2.0 / LAMP Stack Application / Web Hosting– Zero Business Interruption (Hot Backup) Critical
– Granular Restore for developers
– Ease to Use GUI
– Complete Server Protection
– Short backup window with minimal server impact
– Disk Based Backup Dominates
– Large Databases
– Examples:
![Page 18: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/18.jpg)
MySQL Backup Applications• Legacy Backup
– The Dump– Binary Log Backups– mysqlhotcopy – Volume Snapshot Hot Backup– InnoBase (Oracle) Backup– Replication Slave Backup
• Legacy Backup with GUI– ZRM, Bakbone, and others
• Alpha MySQL Online Backup API• near-Continuous Data Protection
![Page 19: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/19.jpg)
Everyone Loves Taking a Dumpmysqldump
Typical User / Complexity Developer, Accidental Admin, DBA Guru
Archive Type Disk
Supported Storage Engines All
Supported O/S All
Binlog Restore No
Zero Business Interruption No
Granular Restore No
Easy to Use Graphical UI No
Complete Server Protection No
Backup Type Full
Backup Window = Time to Read All Data
Server I/O and Load impact Heavy
![Page 20: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/20.jpg)
Binlog Backups for Gurusmysqldump, mysqlhotcopy
Typical User / Complexity DBA Guru
Archive Type Disk
Supported Storage Engines Most / All
Supported O/S All
Binlog Restore Yes
Zero Business Interruption Only during incremental
Granular Restore No
Easy to Use Graphical UI No
Complete Server Protection No
Backup Type Incremental
Backup Window = Time to Read All Data (when Full)
Server I/O and Load impact Heavy (when Full)
![Page 21: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/21.jpg)
Not So Hot mysqlhotcopyHolds table locks while copying MyISAM files
Typical User / Complexity Developer, Accidental Admin, DBA Guru
Archive Type Disk
Supported Storage Engines MyISAM
Supported O/S All
Binlog Restore No
Zero Business Interruption No (Blocks Queries Until Backup Finished)
Granular Restore No
Easy to Use Graphical UI No
Complete Server Protection No
Backup Type Full
Backup Window = Time to Read All Data (when Full)
Server I/O and Load impact Heavy (when Full)
![Page 22: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/22.jpg)
Volume Snapshot Hot Backup• Uses snapshot capability of volume manager on Linux (LVM) or Unix: Veritas VxFS, NetApp SnapManager • Linux: uses LVM snapshots• Usually implemented with custom scripts or programming by a DBA
1.Lock Tables in MySQL
2.Take Volume Snapshot
3.Unlock Tables in MySQL
4.Copy Raw Tables from Snapshot
![Page 23: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/23.jpg)
Volume Snapshot Hot BackupLinux LVM, Unix VxFS, NetApp Snapmanager etc
Typical User / Complexity DBA Guru
Archive Type Disk
Supported Storage Engines MyISAM, InnoDB
Supported O/S Linux and Unix MUST HAVE LVM, VxFS, NetApp
Binlog Restore No
Zero Business Interruption Yes
Granular Restore No
Easy to Use Graphical UI No
Complete Server Protection No
Backup Type Full
Backup Window = Time to Read All Data (when Full)
Server I/O and Load impact Heavy (when Full)
![Page 24: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/24.jpg)
Innobase (Oracle) Hot BackupCommercial InnoDB Hot Backup Utility (ibbackup)
Typical User / Complexity DBA Guru
Archive Type Disk
Supported Storage Engines InnoDB only
Supported O/S All
Binlog Restore Yes
Zero Business Interruption Yes
Granular Restore Yes
Easy to Use Graphical UI No
Complete Server Protection No
Backup Type Incremental
Backup Window = Time to Read All Data (when Full)
Server I/O and Load impact Heavy (when Full)
![Page 25: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/25.jpg)
Slaves Are Hard Workers• MySQL Replication Slave for Backup
• Replication Can be Paused on Slave to Allow Point-in-time backups
• Long backup operations can proceed without disrupting other servers
• Backup Disk I/O isolated away from customer facing servers
![Page 26: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/26.jpg)
Replication Slave for Backupmysqlhotcopy, mysqldump, ibbackup
Typical User / Complexity DBA Guru
Archive Type Disk
Supported Storage Engines Replication
Supported O/S Linux and Windows
Binlog Restore Possible with binlog backup
Zero Business Interruption Yes
Granular Restore No
Easy to Use Graphical UI No
Complete Server Protection No
Backup Type Full or Incremental
Backup Window Long Replication Pause Can be Problematic
Server I/O and Load impact No impact to live servers
![Page 27: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/27.jpg)
Big $$ for User Interfaces?• Most Commercial Software Vendors make Expensive
Wizards and User Interfaces for MySQL Backup
• What Do You Really Get? Is it worth your $$?
![Page 28: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/28.jpg)
What is Under The Hood?• Answer: Little more than
mysqldump, mysqlhotcopy, binlog !• Zmanda Enterprise for MySQL
– “Backups are done via mysqldump, various snapshots, mysqlhotcopy or MySQL replication.”http://www.zmanda.com/backup-mysql.html
• Zmanda for Linux and Unix Server Backups“Amanda uses industry standard tools (tar and dump)…” http://www.zmanda.com/amanda_advantage.html
• BakBone MySQL APM“mysqldump - This component allows you to perform backups/restores of multiple types of MySQL storage engines”http://www.bakbone.com/docs/NetVault_Backup_APM_for_MySQL_Users_Guide.pdf
![Page 29: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/29.jpg)
Zmanda Enterprise for MySQL Web GUI for mysqlhotcopy, mysqldump
Typical User / Complexity Accidental Admin, DBA Guru
Archive Type Disk or Tape
Supported Storage Engines All
Supported O/S Linux and Unix Only
Binlog Restore Yes
Zero Business Interruption No (Possible with LVM Snapshots)
Granular Restore No
Easy to Use Graphical UI Yes
Complete Server Protection Linux and Unix
Backup Type Full (incremental with binlog)
Backup Window Long
Server I/O and Load impact Heavy
![Page 30: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/30.jpg)
BakBone MySQL APM • A Win32 GUI for mysqldump and binlog backups
Typical User / Complexity Accidental Admin, DBA Guru
Archive Type Tape or Virtual Tape Library
Supported Storage Engines All
Supported O/S All
Binlog Restore Yes
Zero Business Interruption No
Granular Restore No
Easy to Use Graphical UI Yes
Complete Server Protection Linux, Unix, and Windows
Backup Type Full (incremental with binlog)
Backup Window Long
Server I/O and Load impact Heavy
![Page 31: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/31.jpg)
Alpha MySQL Online Backup API• Commercial vendors told MySQL AB: build as an API and we’ll build hot
backup apps!http://forge.mysql.com/wiki/OnlineBackup
• Implemented and Not Yet Released (MySQL 6.0.5 )– Online Backup without Interruptions for InnoDB and Falcon– SQL Interface for Initiating Backups
• BACKUP DATABASE databasename1, databasename2 TO "filename.backup";
• RESTORE FROM "filename.backup";
• Limitations Still Being Worked Out– Can’t backup replicated Databases– No backup of database privileges– No selective (table level) backup and restore– No Compression– No MyISAM support
![Page 32: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/32.jpg)
Alpha MySQL Online Backup APIBACKUP DATABASE databasename1, databasename2 TO "filename.backup";
Typical User / Complexity Developer, Accidental Admin, DBA Guru
Archive Type Disk
Supported Storage Engines InnoDB and Falcon
Supported O/S All
Binlog Restore Yes when used with binlog backups
Zero Business Interruption Yes (Only InnoDB and Falcon for now)
Granular Restore No (No Table level restore)
Easy to Use Graphical UI No
Complete Server Protection No
Backup Type Full (incremental with binlog)
Backup Window Long
Server I/O and Load impact Heavy
![Page 33: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/33.jpg)
MySQL near-Continuous Data Protection• Online Backups for MyISAM and InnoDB
– Similar to Linux LVM Backup Method except LVM not required!
– All block devices supported! Software RAID (MD), Multi-Path, ISCSI, Windows Dynamic Disks, etc
– Low Level Volume Snapshots with Device Driver (Linux), Volume Shadow Copy (Windows)
• Scheduled Synchronizations– E.g. Every 15 minutes, Hourly, Daily, etc.
• Changes to Disk Sectors Tracked in between Synchronizations = Short Backup Windows with Minimal Impact
• Granular Restore to Table Level
• Recovery Point Archiving– Only Deltas Stored after first full backup– Each archive point appears as virtual full backup
![Page 34: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/34.jpg)
near-CDP Online Backups for MyISAM & InnoDB
![Page 35: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/35.jpg)
near-CDP Performance
![Page 36: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/36.jpg)
near-CDP Archives
![Page 37: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/37.jpg)
MySQL Granular Restore
![Page 38: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/38.jpg)
![Page 39: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/39.jpg)
MySQL near-Continuous Data ProtectionR1Soft is the only near-CDP for MySQL
Typical User / Complexity Developer, Accidental Admin, DBA Guru
Archive Type Disk
Supported Storage Engines InnoDB, MyISAM
Supported O/S Linux Now Windows Q4 2008
Binlog Restore No
Zero Business Interruption Yes InnoDB and MyISAM
Granular Restore Yes Table Level
Easy to Use Graphical UI Yes
Complete Server Protection Yes Linux and Windows
Backup Type Virtual Full Backup
Backup Window Very Short (Time to Read Deltas)
Server I/O and Load impact As Minimal As Possible
![Page 40: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/40.jpg)
SummaryR1Soft is the only near-CDP for MySQL
Legacy Alpha API near-Continuous
Binlog Restore Yes Yes No
Zero Business Interruption No Yes Yes
Granular Restore No No Yes
Easy to Use Graphical UI Yes Soon Yes
Complete Server Protection Possible No Yes
near-Continuous No No Virtual Full Backup
Backup Window Long Long Very Short
Server I/O and Load impact Heavy Heavy Minimal
Version Support 4.1, 5.0, 5.1, 6.0
6.0.5 - Alpha
4.1, 5.0, 5.1, 6.0
![Page 41: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/41.jpg)
Questions and Comments
David Wartell can be reached at [email protected]
If you would like to discuss any part of this presentation your feedback is [email protected]
This Presentation Can Be Downloaded from http://www.r1soft.com
Please Visit our Data Protection Forum at http://forum.r1soft.com
![Page 42: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/42.jpg)
What Customers Say
![Page 43: MySQL Backups Go Near-continuous](https://reader033.vdocument.in/reader033/viewer/2022061115/5462fa54b1af9fa13d8b4e55/html5/thumbnails/43.jpg)
Sample R1Soft Customers