falcon is not innodb
DESCRIPTION
Falcon is not InnoDB. Kevin Lewis, Falcon Team Lead Ann Harrison, Falcon Team [email protected] , [email protected]. Falcon is not InnoDB. Different design Different concurrency methods Different sweet spots Different quirks Different performance. Design differences. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/1.jpg)
Presented by,
MySQL & O’Reilly Media, Inc.
Falcon is not InnoDB
Kevin Lewis, Falcon Team Lead
Ann Harrison, Falcon Team
![Page 2: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/2.jpg)
Falcon is not InnoDB
Different design Different concurrency methods Different sweet spots Different quirks Different performance
![Page 3: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/3.jpg)
Design differences
InnoDB is modeled after OracleClustered storage
Old versions stored in log
Mixed MVCC and locking
Influenced by MySQLStatement based logging
File per table / index
Table name rules
![Page 4: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/4.jpg)
Design differences
Falcon derives loosely from Rdb and InterBaseStarkey design
Pure MVCC
Originally had no log – careful write for durability
Designed for large memory multi-processorsPage cache plus record cache
Finely grained multi-threading
Synchronization (read/write) on shared structures
![Page 5: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/5.jpg)
Concurrency
Both default to Repeatable Read Neither is exactly Repeatable Read per
ISO/SQL Differ from each other in implementation
InnoDB mixes MVCC and locking
Falcon is pure MVCC
Differ from each other in quirks
![Page 6: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/6.jpg)
Different sweet spots
True of all databases If you design an application to make best use of
Database A, moving to Database B will be hard The more you know about A, the harder it will be
to move to B InnoDB is part of the MySQL family and will be
into the future
![Page 7: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/7.jpg)
Different quirks
Serializable is serializable Everything else is quirky in different ways Standard repeatable read
Select returns the same results
Plus any insert that gets committed
Equivalent to pure locking
read locks + write locks w/o predicate locks
Mix in MVCC and you get write anomalies
![Page 8: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/8.jpg)
Falcon quirk 1
On two tables Insert into t1 (f1) select count (*) from t1M
![Page 9: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/9.jpg)
Falcon quirk 1 Trans1:
Insert into t1 (f1)
select count (*)
from t1
Trans2:Insert into t1 (f1)
select count (*)
from t1
Repeat
mysql> select * from t1;+------+| f1 |+------+| 0 || 0 || 1 || 1 |+------+4 rows in set (0.00 sec)
![Page 10: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/10.jpg)
Falcon quirk 1 Innodb makes
transaction 2 wait for transaction 1’s commit, then stores the “right” values in the table
Transaction 2 has an inconsistent view of data
mysql> select count(*) from t1;+----------+| count(*) |+----------+| 0 |+----------+
mysql> insert into t1 (f1) select count(*) from t1;
mysql> select count(*) from t1;+----------+| count(*) |+----------+| 1 |+----------+
mysql> select * from t1;+------+| f1 |+------+| 2 |+------+
![Page 11: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/11.jpg)
Falcon quirk 2
Exchange values between two tables using two transactions
Neither “sees” the other’s changes
![Page 12: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/12.jpg)
Falcon quirk 2mysql> select * from dinner_menu;
+--------+-------+
| entree | price |
+--------+-------+
| steak | 25.00 |
+--------+-------+
mysql> select * from lunch_menu;
+--------+-------+
| entree | price |
+--------+-------+
| steak | 5.00 |
+--------+-------+
Transaction1:
mysql> update lunch_menu
-> set price =
-> (select price * 0.5 from
-> from dinner_menu where
-> dinner_menu.entree =
-> lunch_menu.entree);
Transaction 2:
mysql> update dinner_menu
-> set price =
-> (select price * 0.5
-> from lunch_menu where
-> lunch_menu.entree =
-> dinner_menu.entree);
![Page 13: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/13.jpg)
Falcon quirk 2mysql> select * from lunch_menu;
+--------+-------+
| entree | price |
+--------+-------+
| steak | 12.50 |
+--------+-------+
mysql> select * from dinner_menu;
+--------+-------+
| entree | price |
+--------+-------+
| steak | 10.00 |
+--------+-------+
InnoDB transaction 2 waits for transaction 1 to commit, then gets the “correct” result
![Page 14: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/14.jpg)
InnoDB quirk 1 Select for update sees a
different scope than normal select
With consistent-read off, Falcon does the same
mysql> select * from t1;
+------+
| f1 |
+------+
| 1 |
+------+
mysql> select * from t1 for update;
+------+
| f1 |
+------+
| 5 |
+------+
![Page 15: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/15.jpg)
InnoDB quirk 2 InnoDB does implicit
“select for update” in some subqueries
Falcon does not
mysql> select * from t1;
+------+
| f1 |
+------+
| 1 |
+------+
mysql> create table t2 as select * from t1;
mysql> select * from t2;
+------+
| f1 |
+------+
| 5 |
+------+
![Page 16: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/16.jpg)
Falcon Architecture – short form
Record cache
Page Cache
Serial Log
Tablespace Files
Gophers
Front end
Back end
I/O Threads
![Page 17: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/17.jpg)
Performance
Where we were last year Performance peaks were good Standard deviation excessive
![Page 18: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/18.jpg)
Performance problem 1
ProblemQuick benchmarks had bad results
SymptomAuto-commit / select * was slow
SolutionReuse read-only transactions
Reduce the cost of transaction startup
Non-blocking scavenge
![Page 19: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/19.jpg)
Falcon performance problem 2
ProblemDBT2 times degraded badly
SymptomRunning a monitoring task improved performance
SolutionFirst, slow down the front end
Put a limit on the number of Active transactions
that can be committed but not “write complete”Second, speed up the back end
![Page 20: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/20.jpg)
Falcon’s back end
From Log to Page Cache - Gopher threadsAdd a pool (‘herd’) of Gophers threads
From Page Cache to diskAdd a pool of I/O threads
Direct IO
Page Consolidation
Thread Prioritization
![Page 21: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/21.jpg)
Falcon performance problem 3
ProblemIndex access (read and insert) was disappointing
SymptomSignificant (>10%) time spent in locating index entry points
SolutionAdd Supernode lookup to each index page
Compromise between density of prefix compression and speed of binary search
![Page 22: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/22.jpg)
Falcon performance problem 4
ProblemDbt2 tests were disappointing and erratic
SymptomSome tests were OK, many weren’t
Standard deviation was large
SolutionHold the mutex in sync object to avoid missing a wake-up call between recognizing the need to wait and going to sleep.
![Page 23: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/23.jpg)
Falcon performance April 2008
CPU bound performance is better. 10 warehouse DBT2 (900 Mb) 16-way, 32GB Intel Caneland, 4 disk RAID 10
![Page 24: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/24.jpg)
Falcon vs InnoDB - Dec, 2007
![Page 25: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/25.jpg)
After Transaction Fix
![Page 26: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/26.jpg)
No Thread Stalls !
![Page 27: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/27.jpg)
Today
![Page 28: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/28.jpg)
Record Cache / Page Cache
100 Warehouses (9GB) with 2GB Falcon Cache
![Page 29: Falcon is not InnoDB](https://reader035.vdocument.in/reader035/viewer/2022081520/5681492b550346895db66451/html5/thumbnails/29.jpg)
Falcon Feature Preview
http://forge.mysql.com/wiki/Falcon_Feature_Preview
Questions ?