wonderful world of mysql storage engine - oscon 2008
Post on 30-May-2018
230 Views
Preview:
TRANSCRIPT
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 1/35
Wonderful world of MySQL StorageEngines
July 24, 2008
OSCON
Portland,OR
by Peter Zaitsev, Percona Inc
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 2/35
-2-
What are Storage Engines
• MySQL Server was for a while consisting of 2 layers
– “SQL Layer” - Responsible for all high level stuff
– “Storage Layer” - Storage, Transactions etc
• MySQL 5.1 changes – Storage Engine interface is modular
– Can compile storage engine separately and load it in theserver
– MySQL Actively cultivated both External and Internalstorage engine development• So there are many of them
– Partners come up with many (often close source) enginestoo.
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 3/35
Wonderful world of MySQL Storage Engines
Aproximate MySQL Architecture
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 4/35
Wonderful world of MySQL Storage Engines
What Storage Engines Are ?
• Storage Engine responsible for storage
– Can implement different storage concepts, file format,remote storage, efficient scans.
• Can't handle upper level functions – sorting, groupby, limit clause
– Future MySQL versions are expected to raise this limits
• Some vendors (like Kickfire) implement hacks to fully
intercept query and handle it via their ownprocessing engine
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 5/35
Wonderful world of MySQL Storage Engines
Extending by Storage Engines
• MySQL took unusual choice for extension for DBMS
– Practically no Pluggable index types, language features
– Though wide choice of storage engines
• Benefits
– Different applications may need different storage properties• Persistence, transactions, lock granularity, compression etc
• Drawbacks
– Performance Overhead
• 2 phase commit on transaction commit, separate logs.
– Complexity• Development and testing (all these interactions)
• Choosing storage engines
• Operational challenges – backup, balancing etc
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 6/35
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 7/35
Wonderful world of MySQL Storage Engines
Storage engines in Practice
• Chose one main storage engine for the application
– Innodb is de-facto standard at this point
• User other storage engines for what they are good
– MyISAM – compact, non transactonal, temporary storage – MEMORY – temporary tables
– Federated – Light duty remote data access etc
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 8/35
Wonderful world of MySQL Storage Engines
Storage Engines
General Purpose Storage EnginesOverview
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 9/35
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 10/35
Wonderful world of MySQL Storage Engines
When to use MyISAM
• If you have Read Only or Read Mostly data whichyou want to be compact
– Note: MyISAM is not always faster than Innodb for reads.
• When you need fast write performance – But not mixed reads/writes for the same table
– Logging, temporary tables, data crunching
• When recovery time is not critical
– Large MyISAM table can take many hours to repair after crash.
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 11/35
Wonderful world of MySQL Storage Engines
Innodb
• Originated by Heikki Tuuri
• Now owned by Oracle Corp
• Was “dormant” for years but new release came out
on MySQL Users Conference – “Plugin” features compression, fast index creation
• Advanced transactional storage engine
– MVCC, Row Level Locks, Clustered Keys
• Automatic crash recovery
• Support for foreign keys
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 12/35
Wonderful world of MySQL Storage Engines
When to use Innodb
• Good as default storage engine in many cases
• When you need transactions or foreign keys
• When you need high concurrency
– So readers do not block writers• If you do not want corrupted tables on power crash
• Tables (especially indexes) are larger than MyISAM
– 2-5 times larger in majority of cases
• If table gets corrupted recovery is complicated
• Import/Index built can be very slow before Pluginrelease.
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 13/35
Wonderful world of MySQL Storage Engines
Falcon
• Storage engine designed by Jim Starkey
• MySQL's prime planned transactional storage engine
– Though told not to be Innodb Competitor
• Focused on working on systems with many CPUsand large amount of memory
• Has a lot of Innovative (different?unproved?)decisions
• Not Clustered, Can't use covered Indexes
• Still Unstable
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 14/35
Wonderful world of MySQL Storage Engines
When would you use Falcon ?
• Very limited production use to tell
– Yes we've seen people running Slaves on Falcon inproduction to test it out
• Will be hard to replace Innodb for many applications
• May be more scalable if Innodb does not gets itsbottlenecks fixed
• Promises to be faster for small transactions
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 15/35
Wonderful world of MySQL Storage Engines
Maria
• Design and Development lead by Michael Widenious
– With few MySQL old timers
• The ideological breed between MyISAM and Innodb
– MyISAM storage efficiency, base sructure – Innodb's Multi Versioning, Transactions, Recoverability
• Can use Page level and Storage storage format
• Transaction support planned to be Optional
• Currently Crash Safe version available
• Not optimized for performance yet.
• Alpha stage
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 16/35
Wonderful world of MySQL Storage Engines
What Maria will be good for ?
• No production use feedback yet
• Good replacement for MyISAM
– Crash safe system tables, cachable large temporary tables
• Will likely be good alternative for Innodb for manycases
– When clustering by Primary key is not critical for application
• Operational ease of use of MyISAM – Repairing tables, moving tables between servers with
some preparation
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 17/35
Wonderful world of MySQL Storage Engines
PBXT
• Developed by Paul McCullagh from PrimeBase
• Developed specially for MySQL
• Has its already 3rd or 4th rewrite
– The last version is now ACID by design• A lot of innovative ideas.
– “Write Once”, Log per transaction etc
• Designed to deal with blobs very efficiently
• Rather unstable still
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 18/35
Wonderful world of MySQL Storage Engines
What PBXT may be used for ?
• To work together with MyBS – blob streaming
• Storing lots of log data
• May work well with SSD Drives
– Mainly bulky sequential drives• Should know better as we see more production use
• Generally showed good performance for somequeries.
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 19/35
Wonderful world of MySQL Storage Engines
What is about SolidDB
• Last year we looked into SolidDB
– And this year we removed it from consideration
• SolidDB for MySQL project stopped after Solid
Technologies was bought by IBM – So it never become real alternative
• The code is available on SourceForge
– But does not have user community to move it forward.
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 20/35
Wonderful world of MySQL Storage Engines
Storage Engines
Using Special Purpose MySQLStorage Engines
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 21/35
Wonderful world of MySQL Storage Engines
MEMORY
• Stores Data in Memory
– Contents lost on restart, though table remains
• Used internally for temporary tables
– To resolve certain queries• Very fast for storing temporary results sets
• Watch out ! Fixed size rows only.
• Pre-Loading data for fast access
• Be careful with replication !
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 22/35
Wonderful world of MySQL Storage Engines
ARCHIVE
• Fast and efficient compressed storage
• No Indexes – Full table scans only
• Non blocking inserts
• Can take space considerably less than MyISAM• Helpful for storing logs
– Keep one table per day or use Partitions for efficiency
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 23/35
Wonderful world of MySQL Storage Engines
FEDERATED
• Access data stored on remote server
• Good when you just need few rows from remoteserver
• Pretty bad with JOINs – requires many round trips• Beware of large result sets retrieved
– Server can run out of memory
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 24/35
Wonderful world of MySQL Storage Engines
BLACKHOLE
• Initially created for benchmarking and examplepurposes
• Though found to be very helpful for filtered
replication – Though some gotchas remain
– ALTER TABLE TBL ... ENGINE=MYISAM• Will convert blackhole engine to MyISAM
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 25/35
Wonderful world of MySQL Storage Engines
Benchmarks
Want some Benchmarks ?
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 26/35
Wonderful world of MySQL Storage Engines
DBT2 Data Sizes
• Loading 200 warehouse DBT2 database
• PBXT Crashed during the load
• Maria footprint is surprisingly large
Data Size (GB)
0
2
4
6
8
10
12
14
16
18
20
Innodb
Innodb Compress edFalcon
Maria
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 27/35
Wonderful world of MySQL Storage Engines
DBT2 Load Times
• 200W load Time
• 2*Dual Core Xeon 5148, 16G Ram
– 8GB buffers allocated to the engines
• Fast index creation was not used for Innodb
Load Time (min)
0
50
100
150
200
250
Innodb
Innodb Compress edFalcon
Maria
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 28/35
Wonderful world of MySQL Storage Engines
DBT2 Results
• DBT2 10W Results (CPU bound)
• Falcon significantly improved since last year run
• 5.1 is a bit slower than 5.0 with Innodb
• Maria results were too unstable to show
Data Size (GB)
0
5000
10000
15000
20000
25000
30000
Innodb 5.0
Innodb 5.1
Falcon
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 29/35
Wonderful world of MySQL Storage Engines
Primitives Benchmark
• CPU Bound, Results in Queries Per Sec
• 1.000.000 rows
• Dual Quad Core Xeon
• Focusing on the typical storage engine access pathsfor the data
• Details about queries and schema can be found
• http://www.mysqlperformanceblog.com/files/benchma
• Falcon excluded as crashing for some of the test
– So not completing the run
• PBXT was hanging on data load
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 30/35
Wonderful world of MySQL Storage Engines
Full Table Scan
1 4 16 64 256
0
1
2
3
4
5
6
7
8
1.65
4.52
6.45 6.45 6.42
1.22
4.66
7.21
6.89
6.49
Innodb Maria
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 31/35
Wonderful world of MySQL Storage Engines
Single row access by PK
1 4 16 64 256
0
10000
20000
30000
40000
50000
60000
70000
80000
90000
14740
49077
80791
67046
57661
6588
20715
2521222957 22102
15260
50015
7153069169
63240
Innodb Maria MyISAM
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 32/35
Wonderful world of MySQL Storage Engines
Access by Index
1 4 16 64 256
0
20
40
60
80
100
120
50
101
79
61
56
35
29
34 34 33
41
97 9795 95
Innodb Maria MyISAM
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 33/35
Wonderful world of MySQL Storage Engines
Access by Covering Index
1 4 16 64 256
0
50
100
150
200
250
300
350
214
307
325 329 325
139
183179 179 178
104
135
108113 113
Innodb Maria MyISAM
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 34/35
8/14/2019 Wonderful World of MySQL Storage Engine - OSCON 2008
http://slidepdf.com/reader/full/wonderful-world-of-mysql-storage-engine-oscon-2008 35/35
Wonderful world of MySQL Storage Engines
Thanks for Coming
• Questions ? Followup ? – pz@percona.com
• Yes, we do MySQL and Web Scaling Consulting
– http://www.percona.com• Check out our book
– Complete rewrite of 1st edition
top related