locking and concurrency control

<Insert Picture Here>

Locking and Concurrency Control Morgan Tocker, MySQL Community Manager

Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

MySQL 5.5MySQL Cluster 7.3

MySQL Enterprise Monitor 2.3 & 3.0MySQL Enterprise Backup Security Scalability HA Audit

MySQL 5.6MySQL Workbench 6.0

M y S Q L U t i l i t i e s

M y S Q L A p p l i e r f o r

H a d o o p

MySQL Workbench 5.2 & 6.0M y S Q L E n t e r p r i s e O r a c l e C e r t i f i c a t i o n s

4 Years of MySQL Innovation

M y S Q L C l u s t e r M a n a g e r

Windows installer & Tools

MySQL Cluster 7.2MySQL Cluster 7.1

MySQL Migration Wizard

MySQL 5.7

Agenda

1.Table Locks, Row Locks and MVCC 2.Live Demo 3.Deadlocks and Lock Waits Discussion 4.ACID Discussion 5.Summary

Copyright © 2012 Oracle and/or its affiliates. All rights reserved.

Table Locks, Row Locks and MVCC

Customer Accounting System

customerid name

1 Acorn Enterprises2 First National Squirrel Bank3 Squirrels 5000 Inc

invoiceid customerid ...

1 1 # Acorn Enterprises2 1 # Acorn Enterprises3 3 # Squirrels 5000 Inc

customer

invoice

itemid invoiceid Description Amount

1 1 1 hrs Professional Services

$10

2 2 Acorn Parts $12.50


$50

4 3 Delivery Fee $17.50

5 3 Replacement Acorn

$20

invoice_item

Start Backup

customerid name




customer

invoice



$10



$50



$20

invoice_item

Continues…

customerid name




customer

invoice



$10



$50



$20

invoice_item

Concurrent Access System - New Customer Added

customerid name

1 Acorn Enterprises2 First National Squirrel Bank3 Squirrels 5000 Inc4 Savvy Squirrel Global


1 1 # Acorn Enterprises2 1 # Acorn Enterprises3 3 # Squirrels 5000 Inc4 4 # Savvy Squirrels

customer

invoice



$10



$50



$20

6 4 Premium Acorn $999

invoice_item

Backup Continues

customerid name




customer

invoice



$10



$50



$20


invoice_item

Backup Continues…

customerid name




customer

invoice



$10



$50



$20


invoice_item

Backup Complete

customerid name




customer

invoice



$10



$50



$20


invoice_item

What was really backed up

customerid name




customer

invoice



$10



$50



$20


invoice_item

Next Operation

customerid name

1 Acorn Enterprises2 First National Squirrel Bank3 Squirrels 5000 Inc4 Acorn Intolerant Squirrels Inc



customer

invoice



$10



$50



$20


invoice_item

X X

• Operation needs to represent a single point in time • When does not matter.

• This race condition can exist in all database operations - not just backups.

Solution

MyISAM Implementation

customerid name




customer

invoice



$10



$50



$20


invoice_item

• Readers may co-exist. • Writers are forbidden. • Locks a table by default. • Programs like mysqldump responsibly lock all tables

they need access to before starting.

MyISAM

InnoDB Implementation

Trx ID Rollback Ptr customerid name

123 ## 1 Acorn Enterprises123 ## 2 First National Squirrel Bank123 ## 3 Squirrels 5000 Inc401 ## 4 Savvy Squirrel

Trx ID Rollback Ptr invoiceid customerid ...

123 ## 1 1 # Acorn Enterprises123 ## 2 1 # Acorn Enterprises123 ## 3 3 # Squirrels 5000 Inc401 ## 4 4 # Savvy

customer

invoice

Trx ID Rollback Ptr itemid invoiceid Description Amount

123 ## 1 1 1 hrs Profess

$10

123 ## 2 2 Acorn Parts

$12.50


$50

123 ## 4 3 Delivery Fee

$17.50

123 ## 5 3 Replacement

$20

401 ## 6 4 Premium

$999

invoice_item

• Hidden meta data attached to each row: • Row version • Rollback pointer to older versions if needed.

• Every transaction gets allocated a monotonic Transaction ID.

InnoDB

InnoDB Implementation

Trx ID Rollback Ptr customerid name

123 ## 1 Acorn Enterprises123 ## 2 First National Squirrel Bank123 ## 3 Squirrels 5000 Inc401 ## 4 Savvy Squirrel

Trx ID Rollback Ptr invoiceid customerid ...

123 ## 1 1 # Acorn Enterprises123 ## 2 1 # Acorn Enterprises123 ## 3 3 # Squirrels 5000 Inc401 ## 4 4 # Savvy

customer

invoice

Trx ID Rollback Ptr itemid invoiceid Description Amount


$10

123 ## 2 2 Acorn Parts

$12.50


$50

123 ## 4 3 Delivery Fee

$17.50

123 ## 5 3 Replacement

$20

401 ## 6 4 Premium

$999

invoice_item

Backup process is TRX ID #394. Remembers it is not supposed to see anything newer than itself. New changes are permitted to continue.

• Stands for “Multi-version concurrency control” - http://en.wikipedia.org/wiki/Multiversion_concurrency_control

• Readers don’t block writers! • And actually writers don’t block readers, because

they can read older versions.

MVCC Algorithm

http://en.wikipedia.org/wiki/Multiversion_concurrency_control

• Multiple versions of the same row may exist. • What you will see depends on the context of your

transaction. Even the number of rows in a table depends on your context.

MVCC (cont.)


Live Demo


Deadlocks & Lock Waits

• A situation when two sessions are trying to acquire each-other’s respective locks.

• Neither can proceed until the other backs down. • It’s no different to when two colleagues get in an

irrational argument and neither will back down :)

What is a deadlock?

• One step short of a deadlock. • You keep trying to do something, but it’s not in the

budget this quarter? • Not quite sure of the analogy here, but you are

free to retry. • But.. a fair amount of the time it still won’t work

What is a lock wait?

• InnoDB as a storage engine will detect deadlocks. • Algorithm is Graph detection http://en.wikipedia.org/

wiki/Cycle_detection - happens virtually instantly. • Kills the transaction that modified the least rows. • All changes are rolled back - application developer

must now handle/retry.

Deadlock Detection

http://en.wikipedia.org/wiki/Cycle_detection

• Deadlock is a concurrency issue. • Same solutions as when you have colleagues

arguing!

Preventing Deadlocks

• Don’t make them work-together. • Reduce concurrency / run known conflicting workers

in serial execution.

Solution #1

X

• Don’t make them work on big projects together. • aka lower the stakes! • Locks are held for the duration of a transaction:

• Put people who don’t work together only on small projects as needed (smaller transactions).

• Reduce locking required for transactions (more efficient queries, better indexing).

Solution #2

• Also, we should accept that some deadlocks are a fact of life. Continuing co-worker analogy: • Essential for business growth that people work

together. • Have to be prepared to handle disputes. !

• MyISAM as a storage engine does not actually need deadlock detection (table locking). • Where does this leave concurrency?

Preventing Deadlocks (cont.)

• You actually want databases to be optimistic. • That is set as few locks as possible, knowing that

sometimes oopses will happen. • Alternative is more locks and less concurrency.

Preventing Deadlocks (cont.)

• Database only owns responsibility of alerting application

• It may not reapply transaction, since state has changed.

• Application owns responsibility of handling errors. • Yet maybe 5% of applications do this :(

Handing Deadlocks

• Will proceed immediately if it can acquire the lock. • Since most transactions are less than 1 second, at

50 seconds seems unlikely retrying will work. • May need to voluntarily rollback and try again or give

a user error.

Lock Wait Timeout

• The problems probably won’t happen often. • May be easier to kick back to the user: • “try again - no changes have been applied”. • Exception may be asynchronous workers?

• Usually you’ve made the promise of work without opportunity to back-out.

“Requires a replay of the transaction”


ACID

• Atomicity - All or none. • Consistency - Generic term to say all constraints etc

must be honoured (primary key, unique key). • Isolation - Changes in one transaction to not

interfere with another transaction. • Durability - When you said that the atomic

transaction was committed, you will honour it. Even if the server were to crash.

The “ACID” Contract

• Most people know this one: • Transfer $20 from Account A --> Account B.

• Multiple failure states... • The money may be subtracted from A, but has

not reached B yet. • Wrap your code between START TRANSACTION

and COMMIT and you get this solved for free.

Atomicity

• Not too much more to add on this one. • InnoDB has foreign key constraints • MySQL does not feature CHECK constraints where

another database may allow more domain specific data validation here as well.

Consistency

• Actually a very expensive feature for a database to implement (may require lots of locking, stale data kept around etc.)

• Many databases make this somewhat tunable. • There are four levels of isolation describing what

potential unsafe conditions are possible.

Isolation

• READ-UNCOMMITTED - No isolation • READ-COMMITTED - Can see other session’s

committed rows. • REPEATABLE-READ - Won’t see other session’s

committed rows once your session starts (default). • SERIALIZABLE - Reads actually lock. Can’t modify

any data another session has changed.

Isolation Levels

• This one’s actually tunable as well. • InnoDB will not lie by default.

• It means on COMMIT everything should survive a power loss not be stuck in a buffer in memory.

Durable


Summary

• Wrap every logical unit of work in START TRANSACTION and COMMIT.

• Expect failure because failure is a database allowing concurrency.

• LOCK WAIT TIMEOUT = Retry “may” be possible. • DEADLOCK = Everything must be retried.

Application Responsibilities

• Common names given to database “internal locking”. • Today we are not going to differentiate between

them. • Not unique to databases, required to provide “thread

safety” in multi-threaded programs.

Mutexes, Latches & Semaphores

• InnoDB buffer pool is full • Free space needs to be created in order to be able to

load your desired page into memory. • In between freeing and loading, someone else fills

desired slot.

Example #1

• When a page becomes too full, it may need to split data into other pages.

• It is possible that there could be a thread deadlock if two pages left and right of each-other reach this condition at once.

• Can’t decide who should proceed first.

Example #2

• When mutexes have high contention other tasks are not runable.

• Which means that CPUs can appear idle, but nothing is happening.

Internal Locking (cont.)

• Responsibility to fix is on MySQL Engineering Team. • High Priority for Us.

• Some DBAs will have peripheral knowledge - know situations when it’s time to upgrade.

Fixes in newer versions

• Kernel mutex split (5.6) • Lock open mutex split (5.6) • Buffer pool mutex contention reduced w/8 buffer

pool instances (5.6) • Multiple rollback segments (5.5) • Log-sys mutex split (5.5) • Flush-list mutex split (5.5)

Time to Upgrade?

More mentioned at: http://www.tocker.ca/2013/11/27/what-is-a-mutex-anyway.html

http://www.tocker.ca/2013/11/27/what-is-a-mutex-anyway.html

locking and concurrency control

Documents