Transcript
Page 1: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Database Management SystemsCSEP 544

Lecture 9: Transactions and Recovery

1CSEP 544 - Fall 2017

Page 2: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Announcements• HW8 released

• OH tomorrow– Always check the class schedule page for up to

date info

• Last lecture today

• Finals on 12/9-10– Covers everything (lectures, HWs, readings)

2

Page 3: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Homework 8• A “flight reservation” transactional application

in Java based on HW3 and Azure• 2 weeks assignment

3

• Use your Azure credits to run and test

Page 4: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Homework 8

4

• Throughput contest (completely optional):– We will generate a random number of transactions

and measure the time taken to execute them– Fastest implementation wins

• 1st place: 2% extra credit on HW• 2nd place: 1% extra credit on HW• 3rd place: 0.5% extra credit on HW

– You can create any extra tables, indexes, classes, etc in your implementation

– Need to pass all grading test cases to be eligible for prizes

Page 5: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

5

Class overview• Data models

– Relational: SQL, RA, and Datalog– NoSQL: SQL++

• RDBMS internals– Query processing and optimization– Physical design

• Parallel query processing– Spark and Hadoop

• Conceptual design– E/R diagrams– Schema normalization

• Transactions– Locking and schedules– Writing DB applications

CSEP 544 - Fall 2017

Data models

UsingDBMS

Query Processing

Page 6: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

6

Class Recap• Data models

– Elements of a data model– Relational data model

• SQL, RA, and Datalog

– Non-relational data model• SQL++

• RDBMS internals– Relational algebra and basics of query processing– Algorithms for relational operators– Physical design and indexes– Query optimization

CSEP 544 - Fall 2017

Page 7: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

7

Class Recap• Parallel query processing

– Different algorithms for relational operators– MapReduce and Spark programming models

• Conceptual design– E/R diagrams– Normal forms and schema normalization

• Transactions and recovery– Schedules and locking-based scheduler– Recovery from failures

CSEP 544 - Fall 2017

Page 8: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Data Management Pipeline

Conceptual Schema

Physical Schema

Schema designer

Databaseadministrator

Applicationprogrammer

product

name

price

8

Page 9: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Transactions• We use database transactions everyday

– Bank $$$ transfers– Online shopping– Signing up for classes

• For this class, a transaction is a series of DB queries– Read / Write / Update / Delete / Insert– Unit of work issued by a user that is independent

from othersCSEP 544 - Fall 2017 9

Page 10: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

What’s the big deal?

CSEP 544 - Fall 2017 10

Page 11: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Challenges

• Want to execute many apps concurrently– All these apps read and write data to the same DB

• Simple solution: only serve one app at a time– What’s the problem?

• Want: multiple operations to be executed atomically over the same DBMS

CSEP 544 - Fall 2017 11

Page 12: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

What can go wrong?• Manager: balance budgets among projects

– Remove $10k from project A– Add $7k to project B– Add $3k to project C

• CEO: check company’s total balance– SELECT SUM(money) FROM budget;

• This is called a dirty / inconsistent read aka a WRITE-READ conflict

CSEP 544 - Fall 2017 12

Page 13: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

What can go wrong?• App 1:

SELECT inventory FROM products WHERE pid = 1

• App 2: UPDATE products SET inventory = 0 WHERE pid = 1

• App 1:SELECT inventory * price FROM products WHERE pid = 1

• This is known as an unrepeatable read aka READ-WRITE conflict

CSEP 544 - Fall 2017 13

Page 14: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

What can go wrong?Account 1 = $100Account 2 = $100

Total = $200• App 1:

– Set Account 1 = $200– Set Account 2 = $0

• App 2:– Set Account 2 = $200– Set Account 1 = $0

• At the end:– Total = $200

• App 1: Set Account 1 = $200

• App 2: Set Account 2 = $200

• App 1: Set Account 2 = $0

• App 2: Set Account 1 = $0

• At the end: – Total = $0

This is called the lost update aka WRITE-WRITE conflictCSEP 544 - Fall 2017 14

Page 15: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

What can go wrong?• Buying tickets to the next Bieber / Swift concert:

– Fill up form with your mailing address– Put in debit card number– Click submit– Screen shows money deducted from your account– [Your browser crashes]

CSEP 544 - Fall 2017 15

Lesson:Changes to the databaseshould be ALL or NOTHING

Page 16: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Transactions

• Collection of statements that are executed atomically (logically speaking)

16

BEGIN TRANSACTION [SQL statements]

COMMIT or ROLLBACK (=ABORT)

[single SQL statement]

If BEGIN… missing,then TXN consists

of a single instructionCSEP 544 - Fall 2017

Page 17: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

17

Know your chemistry transactions: ACID

• Atomic– State shows either all the effects of txn, or none of them

• Consistent– Txn moves from a DBMS state where integrity holds, to

another where integrity holds • remember integrity constraints?

• Isolated– Effect of txns is the same as txns running one after

another (i.e., looks like batch mode)• Durable

– Once a txn has committed, its effects remain in the database

CSEP 544 - Fall 2017

Page 18: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Atomic• Definition: A transaction is ATOMIC if all

its updates must happen or not at all.• Example: move $100 from A to B

– UPDATE accounts SET bal = bal – 100 WHERE acct = A;

– UPDATE accounts SET bal = bal + 100 WHERE acct = B;

– BEGIN TRANSACTION; UPDATE accounts SET bal = bal – 100 WHERE acct = A;UPDATE accounts SET bal = bal + 100 WHERE acct = B;COMMIT; 18CSEP 544 - Fall 2017

Page 19: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Isolated

• Definition An execution ensures that txns are isolated, if the effect of each txn is as if it were the only txn running on the system.

CSEP 544 - Fall 2017 19

Page 20: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Consistent• Recall: integrity constraints govern how values in

tables are related to each other– Can be enforced by the DBMS, or ensured by the app

• How consistency is achieved by the app:– App programmer ensures that txns only takes a

consistent DB state to another consistent state– DB makes sure that txns are executed atomically

• Can defer checking the validity of constraints until the end of a transaction

CSEP 544 - Fall 2017 20

Page 21: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Durable

• A transaction is durable if its effects continue to exist after the transaction and even after the program has terminated

• How? – By writing to disk!– (more later)

CSEP 544 - Fall 2017 21

Page 22: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Rollback transactions

• If the app gets to a state where it cannot complete the transaction successfully, execute ROLLBACK

• The DB returns to the state prior to the transaction

• What are examples of such program states?

CSEP 544 - Fall 2017 22

Page 23: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

23

ACID• Atomic• Consistent• Isolated• Durable

• Enjoy this in HW8!

• Again: by default each statement is its own txn– Unless auto-commit is off then each statement starts a

new txn

CSEP 544 - Fall 2017

Page 24: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Transaction Schedules

CSEP 544 - Fall 2017 24

Page 25: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Schedules

CSEP 544 - Fall 2017 25

A schedule is a sequenceof interleaved actions from all transactions

Page 26: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Serial Schedule

• A serial schedule is one in which transactions are executed one after the other, in some sequential order

• Fact: nothing can go wrong if the system executes transactions serially – (up to what we have learned so far)– But DBMS don’t do that because we want better overall

system performance26CSEP 544 - Fall 2017

Page 27: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Example

T1 T2READ(A, t) READ(A, s)t := t+100 s := s*2WRITE(A, t) WRITE(A,s)READ(B, t) READ(B,s)t := t+100 s := s*2WRITE(B,t) WRITE(B,s)

CSEP 544 - Fall 2017 27

A and B are elementsin the database

t and s are variables in txn source code

Page 28: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Example of a (Serial) ScheduleT1 T2READ(A, t)t := t+100WRITE(A, t)READ(B, t)t := t+100WRITE(B,t)

READ(A,s)s := s*2WRITE(A,s)READ(B,s)s := s*2WRITE(B,s)

CSEP 544 - Fall 2017 28

Tim

e

Page 29: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Another Serial ScheduleT1 T2

READ(A,s)s := s*2WRITE(A,s)READ(B,s)s := s*2WRITE(B,s)

READ(A, t)t := t+100WRITE(A, t)READ(B, t)t := t+100WRITE(B,t)

CSEP 544 - Fall 2017 29

Tim

e

Page 30: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Serializable Schedule

CSEP 544 - Fall 2017 30

A schedule is serializable if it is equivalent to a serial schedule

Page 31: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

A Serializable ScheduleT1 T2READ(A, t)t := t+100WRITE(A, t)

READ(A,s)s := s*2WRITE(A,s)

READ(B, t)t := t+100WRITE(B,t)

READ(B,s)s := s*2WRITE(B,s)

This is a serializable schedule.This is NOT a serial schedule

CSEP 544 - Fall 2017 31

Page 32: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

A Non-Serializable ScheduleT1 T2READ(A, t)t := t+100WRITE(A, t)

READ(A,s)s := s*2WRITE(A,s)READ(B,s)s := s*2WRITE(B,s)

READ(B, t)t := t+100WRITE(B,t)

CSEP 544 - Fall 2017 32

Page 33: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

How do We Know if a Schedule is Serializable?

CSEP 544 - Fall 2017 33

T1: r1(A); w1(A); r1(B); w1(B)T2: r2(A); w2(A); r2(B); w2(B)

Notation:

Key Idea: Focus on conflicting operations

Page 34: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Conflicts

• Write-Read – WR• Read-Write – RW• Write-Write – WW• Read-Read?

CSEP 544 - Fall 2017 34

Page 35: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Conflict SerializabilityConflicts: (i.e., swapping will change program behavior)

ri(X); wi(Y)Two actions by same transaction Ti:

wi(X); wj(X)Two writes by Ti, Tj to same element

wi(X); rj(X)Read/write by Ti, Tj to same element

ri(X); wj(X)CSEP 544 - Fall 2017 35

Page 36: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Conflict Serializability

• A schedule is conflict serializable if it can be transformed into a serial schedule by a series of swappings of adjacent non-conflicting actions

• Every conflict-serializable schedule is serializable

CSEP 544 - Fall 2017 36

Page 37: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Conflict Serializability

CSEP 544 - Fall 2017 37

Example:r1(A); w1(A); r2(A); w2(A); r1(B); w1(B); r2(B); w2(B)

Page 38: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Conflict Serializability

CSEP 544 - Fall 2017 38

Example:

r1(A); w1(A); r1(B); w1(B); r2(A); w2(A); r2(B); w2(B)

r1(A); w1(A); r2(A); w2(A); r1(B); w1(B); r2(B); w2(B)

Page 39: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Conflict Serializability

CSEP 544 - Fall 2017 39

Example:

r1(A); w1(A); r1(B); w1(B); r2(A); w2(A); r2(B); w2(B)

r1(A); w1(A); r2(A); w2(A); r1(B); w1(B); r2(B); w2(B)

Page 40: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Conflict Serializability

CSEP 544 - Fall 2017 40

Example:

r1(A); w1(A); r1(B); w1(B); r2(A); w2(A); r2(B); w2(B)

r1(A); w1(A); r2(A); w2(A); r1(B); w1(B); r2(B); w2(B)

r1(A); w1(A); r2(A); r1(B); w2(A); w1(B); r2(B); w2(B)

Page 41: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Conflict Serializability

CSEP 544 - Fall 2017 41

Example:

r1(A); w1(A); r1(B); w1(B); r2(A); w2(A); r2(B); w2(B)

r1(A); w1(A); r2(A); w2(A); r1(B); w1(B); r2(B); w2(B)

r1(A); w1(A); r2(A); r1(B); w2(A); w1(B); r2(B); w2(B)

r1(A); w1(A); r1(B); r2(A); w2(A); w1(B); r2(B); w2(B)

….

Page 42: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Testing for Conflict-Serializability

Precedence graph:• A node for each transaction Ti, • An edge from Ti to Tj whenever an action in Ti

conflicts with, and comes before an action in Tj

• The schedule is conflict-serializable iff the precedence graph is acyclic

CSEP 544 - Fall 2017 42

Page 43: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Example 1

CSEP 544 - Fall 2017 43

r2(A); r1(B); w2(A); r3(A); w1(B); w3(A); r2(B); w2(B)

1 2 3

Page 44: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Example 1

CSEP 544 - Fall 2017 44

r2(A); r1(B); w2(A); r3(A); w1(B); w3(A); r2(B); w2(B)

1 2 3

This schedule is conflict-serializable

AB

Page 45: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Example 2

CSEP 544 - Fall 2017 45

r2(A); r1(B); w2(A); r2(B); r3(A); w1(B); w3(A); w2(B)

1 2 3

Page 46: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Example 2

CSEP 544 - Fall 2017 46

1 2 3

This schedule is NOT conflict-serializable

AB

B

r2(A); r1(B); w2(A); r2(B); r3(A); w1(B); w3(A); w2(B)

Page 47: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Course Evalhttp://bit.do/544eval

CSEP 544 - Fall 2017 47

Page 48: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Implementing Transactions

CSEP 544 - Fall 2017 48

Page 49: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Scheduler

• Scheduler = the module that schedules the transaction’s actions, ensuring serializability

• Also called Concurrency Control Manager

• We discuss next how a scheduler may be implemented

CSEP 544 - Fall 2017 49

Page 50: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Implementing a Scheduler

Major differences between database vendors• Locking Scheduler

– Aka “pessimistic concurrency control”– SQLite, SQL Server, DB2

• Multiversion Concurrency Control (MVCC)– Aka “optimistic concurrency control”– Postgres, Oracle

We discuss only locking schedulers in this class50CSEP 544 - Fall 2017

Page 51: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Locking Scheduler

Simple idea:• Each element has a unique lock• Each transaction must first acquire the lock

before reading/writing that element• If the lock is taken by another transaction,

then wait• The transaction must release the lock(s)

CSEP 544 - Fall 2017 51By using locks scheduler ensures conflict-serializability

Page 52: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

What Data Elements are Locked?

Major differences between vendors:

• Lock on the entire database– SQLite

• Lock on individual records– SQL Server, DB2, etc

CSEP 544 - Fall 2017 52

Page 53: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

More Notations

Li(A) = transaction Ti acquires lock for element A

Ui(A) = transaction Ti releases lock for element A

53CSEP 544 - Fall 2017

Page 54: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

A Non-Serializable ScheduleT1 T2READ(A)A := A+100WRITE(A)

READ(A)A := A*2WRITE(A)READ(B)B := B*2WRITE(B)

READ(B)B := B+100WRITE(B)

54CSEP 544 - Fall 2017

Page 55: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

A Serializable ScheduleT1 T2READ(A, t)A := A+100WRITE(A)

READ(A)A := A*2WRITE(A)

READ(B)B := B+100WRITE(B)

READ(B)B := B*2WRITE(B)

CSEP 544 - Fall 2017 55

Page 56: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Enforcing Conflict-Serializabilitywith Locks

T1 T2L1(A); READ(A)A := A+100WRITE(A); U1(A); L1(B)

L2(A); READ(A)A := A*2WRITE(A); U2(A); L2(B); BLOCKED…

READ(B)B := B+100WRITE(B); U1(B);

…GRANTED; READ(B)B := B*2WRITE(B); U2(B);

56CSEP 544 - Fall 2017Scheduler has ensured a conflict-serializable schedule

Page 57: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

But…T1 T2L1(A); READ(A)A := A+100WRITE(A); U1(A);

L2(A); READ(A)A := A*2WRITE(A); U2(A);L2(B); READ(B)B := B*2WRITE(B); U2(B);

L1(B); READ(B)B := B+100WRITE(B); U1(B);

57Locks did not enforce conflict-serializability !!! What’s wrong ?

Page 58: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Two Phase Locking (2PL)

CSEP 544 - Fall 2017 58

In every transaction, all lock requests must precede all unlock requests

The 2PL rule:

Page 59: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Example: 2PL transactionsT1 T2L1(A); L1(B); READ(A)A := A+100WRITE(A); U1(A)

L2(A); READ(A)A := A*2WRITE(A); L2(B); BLOCKED…

READ(B)B := B+100WRITE(B); U1(B);

…GRANTED; READ(B)B := B*2WRITE(B); U2(A); U2(B); Now it is conflict-serializable

59CSEP 544 - Fall 2017

Page 60: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

A New Problem: Non-recoverable Schedule

T1 T2L1(A); L1(B); READ(A)A :=A+100WRITE(A); U1(A)

L2(A); READ(A)A := A*2WRITE(A); L2(B); BLOCKED…

READ(B)B :=B+100WRITE(B); U1(B);

…GRANTED; READ(B)B := B*2WRITE(B); U2(A); U2(B); Commit

Rollback60CSEP 544 - Fall 2017

Page 61: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Strict 2PL

CSEP 544 - Fall 2017 61

All locks are held until the transactioncommits or aborts.

The Strict 2PL rule:

With strict 2PL, we will get schedules thatare both conflict-serializable and recoverable

Page 62: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Strict 2PLT1 T2L1(A); READ(A)A :=A+100WRITE(A);

L2(A); BLOCKED…L1(B); READ(B)B :=B+100WRITE(B); Rollback

U1(A);U1(B); …GRANTED; READ(A)A := A*2WRITE(A); L2(B); READ(B)B := B*2WRITE(B); CommitU2(A); U2(B); 62

Page 63: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Another problem: Deadlocks• T1 waits for a lock held by T2;• T2 waits for a lock held by T3;• T3 waits for . . . .• . . .• Tn waits for a lock held by T1

63CSEP 544 - Fall 2017

SQL Lite: there is only one exclusive lock; thus, never deadlocks

SQL Server: checks periodically for deadlocks and aborts one TXN

Page 64: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Lock Modes

• S = shared lock (for READ)• X = exclusive lock (for WRITE)

64CSEP 544 - Fall 2017

None S XNone

SX

Lock compatibility matrix:

Page 65: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Lock Modes

• S = shared lock (for READ)• X = exclusive lock (for WRITE)

65CSEP 544 - Fall 2017

None S XNone ✔ ✔ ✔

S ✔ ✔ ✖

X ✔ ✖ ✖

Lock compatibility matrix:

Page 66: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

66

Lock Granularity

• Fine granularity locking (e.g., tuples)– High concurrency– High overhead in managing locks– E.g., SQL Server

• Coarse grain locking (e.g., tables, entire database)– Many false conflicts– Less overhead in managing locks– E.g., SQL Lite

• Solution: lock escalation changes granularity as needed

CSEP 544 - Fall 2017

Page 67: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Lock Performance

CSEP 544 - Fall 2017 67

Thro

ughp

ut (T

PS)

# Active Transactions

thrashing

Why ?

TPS =Transactionsper second

To avoid, use admission control

Page 68: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

68

Phantom Problem

• So far we have assumed the database to be a static collection of elements (=tuples)

• If tuples are inserted/deleted then the phantom problem appears

CSEP 544 - Fall 2017

Page 69: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Phantom Problem

Is this schedule serializable ?

T1 T2SELECT *FROM ProductWHERE color=‘blue’

INSERT INTO Product(name, color)VALUES (‘A3’,’blue’)

SELECT *FROM ProductWHERE color=‘blue’

Suppose there are two blue products, A1, A2:

CSEP 544 - Fall 2017 69

Page 70: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Phantom Problem

70

R1(A1);R1(A2);W2(A3);R1(A1);R1(A2);R1(A3)

T1 T2SELECT *FROM ProductWHERE color=‘blue’

INSERT INTO Product(name, color)VALUES (‘A3’,’blue’)

SELECT *FROM ProductWHERE color=‘blue’

CSEP 544 - Fall 2017

Suppose there are two blue products, A1, A2:

Page 71: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

W2(A3);R1(A1);R1(A2);R1(A1);R1(A2);R1(A3)

Phantom Problem

R1(A1);R1(A2);W2(A3);R1(A1);R1(A2);R1(A3)

T1 T2SELECT *FROM ProductWHERE color=‘blue’

INSERT INTO Product(name, color)VALUES (‘A3’,’blue’)

SELECT *FROM ProductWHERE color=‘blue’

Suppose there are two blue products, A1, A2:

Page 72: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

72

Phantom Problem

• A “phantom” is a tuple that is invisible during part of a transaction execution but not invisible during the entire execution

• In our example:– T1: reads list of products– T2: inserts a new product– T1: re-reads: a new product appears !

CSEP 544 - Fall 2017

Page 73: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Dealing With Phantoms

• Lock the entire table• Lock the index entry for ‘blue’

– If index is available• Or use predicate locks

– A lock on an arbitrary predicate

Dealing with phantoms is expensive !CSEP 544 - Fall 2017 73

Page 74: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

74

Isolation Levels in SQL

1. “Dirty reads”SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED

2. “Committed reads”SET TRANSACTION ISOLATION LEVEL READ COMMITTED

3. “Repeatable reads”SET TRANSACTION ISOLATION LEVEL REPEATABLE READ

4. Serializable transactionsSET TRANSACTION ISOLATION LEVEL SERIALIZABLE

ACID

CSEP 544 - Fall 2017

Page 75: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

1. Isolation Level: Dirty Reads

• “Long duration” WRITE locks– Strict 2PL

• No READ locks– Read-only transactions are never delayed

75

Possible problems: dirty and inconsistent reads

CSEP 544 - Fall 2017

Page 76: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

2. Isolation Level: Read Committed

• “Long duration” WRITE locks– Strict 2PL

• “Short duration” READ locks– Only acquire lock while reading (not 2PL)

76

Unrepeatable reads:When reading same element twice, may get two different values

CSEP 544 - Fall 2017

Page 77: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

3. Isolation Level: Repeatable Read

• “Long duration” WRITE locks– Strict 2PL

• “Long duration” READ locks– Strict 2PL

77

This is not serializable yet !!!

Why ?

CSEP 544 - Fall 2017

Page 78: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

4. Isolation Level Serializable

• “Long duration” WRITE locks– Strict 2PL

• “Long duration” READ locks– Strict 2PL

• Predicate locking– To deal with phantoms

78CSEP 544 - Fall 2017

Page 79: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Beware!In commercial DBMSs:• Default level is often NOT serializable• Default level differs between DBMSs• Some engines support subset of levels!• Serializable may not be exactly ACID

– Locking ensures isolation, not atomicity• Also, some DBMSs do NOT use locking and

different isolation levels can lead to different pbs• Bottom line: Read the doc for your DBMS!

CSEP 544 - Fall 2017 79

Page 80: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Recovery

CSEP 544 - Fall 2017 80

Page 81: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

81

Log-based Recovery

Basics (based on textbook Ch. 17.2-3)• Undo logging• Redo logging

CSEP 544 - Fall 2017

Page 82: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

82

Transaction Abstraction

• Database is composed of elements.

• 1 element can be either:– 1 page = physical logging– 1 record = logical logging

CSEP 544 - Fall 2017

Page 83: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

83

Primitive Operations of Transactions

• READ(X,t)– copy element X to transaction local variable t

• WRITE(X,t)– copy transaction local variable t to element X

• INPUT(X)– read element X to memory buffer

• OUTPUT(X)– write element X to disk

CSEP 544 - Fall 2017

Page 84: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

84

Running Example

Initially, A=B=8.

Atomicity requires that either(1) T commits and A=B=16, or(2) T does not commit and A=B=8.

CSEP 544 - Fall 2017

BEGIN TRANSACTIONREAD(A,t); t := t*2;WRITE(A,t); READ(B,t); t := t*2;WRITE(B,t)COMMIT;

Page 85: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Action t Mem A Mem B Disk A Disk BINPUT(A) 8 8 8

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8

INPUT(B) 16 16 8 8 8

READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

COMMIT

Main memory DiskTransaction

READ(A,t); t := t*2; WRITE(A,t); READ(B,t); t := t*2; WRITE(B,t)

Page 86: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Action t Mem A Mem B Disk A Disk BINPUT(A) 8 8 8

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8

INPUT(B) 16 16 8 8 8

READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

COMMIT

Is this bad ?

Crash !

Page 87: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Is this bad ?

Action t Mem A Mem B Disk A Disk BINPUT(A) 8 8 8

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8

INPUT(B) 16 16 8 8 8

READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

COMMIT

Yes it’s bad: A=16, B=8….

Crash !

Page 88: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Is this bad ?

Action t Mem A Mem B Disk A Disk BINPUT(A) 8 8 8

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8

INPUT(B) 16 16 8 8 8

READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

COMMITCrash !

Page 89: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Is this bad ?

Action t Mem A Mem B Disk A Disk BINPUT(A) 8 8 8

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8

INPUT(B) 16 16 8 8 8

READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

COMMIT

Yes it’s bad: A=B=16, but not committed

Crash !

Page 90: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Is this bad ?

Action t Mem A Mem B Disk A Disk BINPUT(A) 8 8 8

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8

INPUT(B) 16 16 8 8 8

READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

COMMIT

Crash !

Page 91: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Is this bad ?

Action t Mem A Mem B Disk A Disk BINPUT(A) 8 8 8

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8

INPUT(B) 16 16 8 8 8

READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

COMMIT

No: that’s OK

Crash !

Page 92: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Typically, OUTPUT is after COMMIT (why?)

Action t Mem A Mem B Disk A Disk BINPUT(A) 8 8 8

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8

INPUT(B) 16 16 8 8 8

READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8

COMMIT

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

Page 93: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Typically, OUTPUT is after COMMIT (why?)

Action t Mem A Mem B Disk A Disk BINPUT(A) 8 8 8

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8

INPUT(B) 16 16 8 8 8

READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8

COMMIT

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

Crash !

Page 94: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Atomic Transactions

• FORCE or NO-FORCE– Should all updates of a transaction be forced to

disk before the transaction commits?• STEAL or NO-STEAL

– Can an update made by an uncommitted transaction overwrite the most recent committed value of a data item on disk?

CSEP 544 - Fall 2017 94

Page 95: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Force/No-steal

• FORCE: Pages of committed transactions must be forced to disk before commit

• NO-STEAL: Pages of uncommitted transactions cannot be written to disk

CSEP 544 - Fall 2017 95

Easy to implement (how?) and ensures atomicity

Page 96: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

No-Force/Steal

• NO-FORCE: Pages of committed transactions need not be written to disk

• STEAL: Pages of uncommitted transactions may be written to disk

CSEP 544 - Fall 2017 96

In either case, Atomicity is violated; need WAL

Page 97: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

97

Write-Ahead LogThe Log: append-only file containing log records• Records every single action of every TXN• Force log entry to disk• After a system crash, use log to recoverThree types: UNDO, REDO, UNDO-REDO

CSEP 544 - Fall 2017

Page 98: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

UNDO Log

CSEP 544 - Fall 2017 98

FORCE and STEAL

Page 99: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

99

Undo LoggingLog records• <START T>

– transaction T has begun• <COMMIT T>

– T has committed• <ABORT T>

– T has aborted• <T,X,v>

– T has updated element X, and its old value was v

CSEP 544 - Fall 2017

Page 100: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

100

Action t Mem A Mem B Disk A Disk B UNDO Log

<START T>

INPUT(A) 8 8 8

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8 <T,A,8>

INPUT(B) 16 16 8 8 8READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8 <T,B,8>

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

COMMIT <COMMIT T>

Page 101: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

101WHAT DO WE DO ?

Action t Mem A Mem B Disk A Disk B UNDO Log

<START T>

INPUT(A) 8 8 8

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8 <T,A,8>

INPUT(B) 16 16 8 8 8READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8 <T,B,8>

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

COMMIT <COMMIT T>

Crash !

Page 102: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

102WHAT DO WE DO ?

Action t Mem A Mem B Disk A Disk B UNDO Log

<START T>

INPUT(A) 8 8 8

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8 <T,A,8>

INPUT(B) 16 16 8 8 8READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8 <T,B,8>

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

COMMIT <COMMIT T>

We UNDO by setting B=8 and A=8

Crash !

Page 103: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

103

Action t Mem A Mem B Disk A Disk B UNDO Log

<START T>

INPUT(A) 8 8 8

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8 <T,A,8>

INPUT(B) 16 16 8 8 8READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8 <T,B,8>

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

COMMIT <COMMIT T>

What do we do now ? Crash !

Page 104: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Action t Mem A Mem B Disk A Disk B UNDO Log

<START T>

INPUT(A) 8 8 8

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8 <T,A,8>

INPUT(B) 16 16 8 8 8READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8 <T,B,8>

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

COMMIT <COMMIT T>

What do we do now ? Crash !Nothing: log contains COMMIT

Page 105: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

105

Recovery with Undo Log……<T6,X6,v6>……<START T5><START T4><T1,X1,v1><T5,X5,v5><T4,X4,v4><COMMIT T5><T3,X3,v3><T2,X2,v2>

Question1: Which updatesare undone ?

Question 2:How far backdo we need toread in the log ?

Question 3:What happens if thereis a second crash,during recovery ?

Crash !

Page 106: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

Action t Mem A Mem B Disk A Disk B UNDO Log

<START T>

INPUT(A) 8 8 8

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8 <T,A,8>

INPUT(B) 16 16 8 8 8READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8 <T,B,8>

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

COMMIT <COMMIT T>

When mustwe force pagesto disk ?

Page 107: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

107

Action t Mem A Mem B Disk A Disk B UNDO Log

<START T>

INPUT(A) 8 8 8

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8 <T,A,8>

INPUT(B) 16 16 8 8 8READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8 <T,B,8>

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

COMMIT <COMMIT T>

RULES: log entry before OUTPUT before COMMIT

FORCE

Page 108: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

108

Undo-Logging Rules

U1: If T modifies X, then <T,X,v> must be written to disk before OUTPUT(X)

U2: If T commits, then OUTPUT(X) must be written to disk before <COMMIT T>

• Hence: OUTPUTs are done early, before the transaction commits

CSEP 544 - Fall 2017

FORCE

Page 109: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

REDO Log

CSEP 544 - Fall 2017 109

NO-FORCE and NO-STEAL

Page 110: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

110

Action t Mem A Mem B Disk A Disk B

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8

READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8

COMMIT

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

Is this bad ?

Crash !

Page 111: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

111

Action t Mem A Mem B Disk A Disk B

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8

READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8

COMMIT

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

Is this bad ? Yes, it’s bad: A=16, B=8

Crash !

Page 112: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

112

Action t Mem A Mem B Disk A Disk B

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8

READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8

COMMIT

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

Is this bad ?

Crash !

Page 113: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

113

Action t Mem A Mem B Disk A Disk B

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8

READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8

COMMIT

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

Is this bad ?

Crash !

Yes, it’s bad: lost update

Page 114: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

114

Action t Mem A Mem B Disk A Disk B

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8

READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8

COMMIT

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

Is this bad ?

Crash !

Page 115: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

115

Action t Mem A Mem B Disk A Disk B

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8

READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8

COMMIT

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

Is this bad ? No: that’s OK.

Crash !

Page 116: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

116

Redo Logging

One minor change to the undo log:

• <T,X,v>= T has updated element X, and its new value is v

CSEP 544 - Fall 2017

Page 117: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

117

Action t Mem A Mem B Disk A Disk B REDO Log

<START T>

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8 <T,A,16>

READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8 <T,B,16>

COMMIT <COMMIT T>

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

Page 118: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

118

Action t Mem A Mem B Disk A Disk B REDO Log

<START T>

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8 <T,A,16>

READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8 <T,B,16>

COMMIT <COMMIT T>

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

How do we recover ?

Crash !

Page 119: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

119

Action t Mem A Mem B Disk A Disk B REDO Log

<START T>

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8 <T,A,16>

READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8 <T,B,16>

COMMIT <COMMIT T>

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

How do we recover ? We REDO by setting A=16 and B=16

Crash !

Page 120: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

120

Recovery with Redo Log<START T1><T1,X1,v1><START T2><T2, X2, v2><START T3><T1,X3,v3><COMMIT T2><T3,X4,v4><T1,X5,v5>

CSEP 544 - Fall 2017

Show actionsduring recovery

Crash !

Page 121: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

121

Action t Mem A Mem B Disk A Disk B REDO Log

<START T>

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8 <T,A,16>

READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8 <T,B,16>

COMMIT <COMMIT T>

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

When mustwe force pagesto disk ?

Page 122: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

122

Action t Mem A Mem B Disk A Disk B REDO Log

<START T>

READ(A,t) 8 8 8 8

t:=t*2 16 8 8 8

WRITE(A,t) 16 16 8 8 <T,A,16>

READ(B,t) 8 16 8 8 8

t:=t*2 16 16 8 8 8

WRITE(B,t) 16 16 16 8 8 <T,B,16>

COMMIT <COMMIT T>

OUTPUT(A) 16 16 16 16 8

OUTPUT(B) 16 16 16 16 16

RULE: OUTPUT after COMMIT

NO-STEAL

Page 123: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

123

Redo-Logging Rules

R1: If T modifies X, then both <T,X,v> and <COMMIT T> must be written to disk before OUTPUT(X)

• Hence: OUTPUTs are done late

CSEP 544 - Fall 2017

NO-STEAL

Page 124: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

124

Comparison Undo/Redo• Undo logging: OUTPUT must be done

early: – Inefficient

• Redo logging: OUTPUT must be done late: – Inflexible

• Compromise: ARIES (see textbook)CSEP 544 - Fall 2017

Page 125: Lecture 9: Transactions and Recovery€¦ · Lecture 9: Transactions and Recovery CSEP 544 - Fall 2017 1. Announcements • HW8 released ... • RDBMS internals ... • Buying tickets

End of CSEP 544• “Big data” is here to stay• Requires unique techniques / abstractions

– Logic (SQL)– Algorithms (query processing)– Conceptual modeling (FD’s)– Transactions

• Technology evolving rapidly, but• Techniques/abstracts persist over may years,

e.g. What goes around comes around

CSEP 544 - Fall 2017 125


Top Related