Jerry PostCopyright © 2013

DATABASE

Database Management Systems

Chapter 7

Database Integrity and Transactions

1

2

Objectives

Why would you need to use procedural code when SQL is so powerful? How are SQL commands integrated into more traditional programming

structures? What capabilities exist in procedural code? How are business rules added to the database? How does a DBMS handle multiple transaction events? How do you prevent problems arising when two processes change the

same data? What are the primary rules to ensure integrity of transactions? How are key values generated? How can procedural code track row-by-row through a query? What issues arise when maintaining totals in the database?

3

Programming Environment

Create code(1) Within the query system

(2) In forms and reports

(3) Hosted in external programs

DBMSTables

Forms &Reports

Queries

If (Click) ThenMsgBox . . .

End If

If ( . . ) Then SELECT . . .

Else . . .UPDATE . . .

End If

C++

ExternalProgram

if (. . .) { // embed SQL SELECT …}

(2)

(1)

(3)

4

User-Defined Function

CREATE FUNCTION EstimateCosts (ListPrice Currency, ItemCategory VarChar)

RETURNS CurrencyBEGIN

IF (ItemCategory = ‘Clothing’) THENRETURN ListPrice * 0.5

ELSERETURN ListPrice * 0.75

END IFEND

5

Function to Perform Conditional Update

CREATE FUNCTION IncreaseSalary(EmpID INTEGER, Amt CURRENCY)

RETURNS CURRENCYBEGIN

IF (Amt > 50000) THENRETURN -1 -- error flag

ENDUPDATE Employee SET Salary = Salary + AmtWHERE EmployeeID = EmpID;RETURN Amt

END

6

Looking Up Data

CREATE FUNCTION IncreaseSalary(EmpID INTEGER, Amt CURRENCY)

RETURNS CURRENCYDECLARE

CURRENCY MaxAmount;BEGIN

SELECT MaxRaise INTO MaxAmountFROM CompanyLimitsWHERE LimitName = ‘Raise’;

IF (Amt > MaxAmount) THENRETURN -1 -- error flag

ENDUPDATE Employee SET Salary = Salary + AmtWHERE EmployeeID = EmpID;RETURN Amt;

END

Programming Tools

7

(1) Sequence(2) Variables(3) Conditions(4) Loops(5) Input and Output(6) Procedures and functions (subroutines)

(1) Sequence

8

INSERT INTO Customer (CustomerID, LastName, …)VALUES (1551, ‘Jones’, …);

INSERT INTO Sales(SaleDate, CustomerID)VALUES (Now(), 1551);

Control the order of execution for multiple tasks.

(2) Variables

9

Temporary locations in memory to hold data.

FUNCTION ComputeTax(SalesTotal)…DECLARE Tax as currency; TaxRate as float;BEGIN SELECT TaxRate = SalesTax FROM TaxTable WHERE … Tax = SalesTotal*TaxRate; RETURN Tax;END;

Scope: Within this function only.

(3) Conditions

10

Compare values or logical tests.

FUNCTION sample(…)BEGIN IF (amount < 0) THEN action 1 ELSE action 2 END IF

END;

(4) Loops

11

Repeat a set of operations.

i=0;DO action 1 i = i + 1;WHILE (i < 10);

C IS CURSOR FOR SELECT …DO UNTIL c.EOF examine one row of data FETCH(c)LOOP

Useful for examining one row of data at a time from a SELECT statement.

(5) Input and Output

12

In a DBMS, interact with a form or a data table.

DBMS, use SQLSELECT variables …INSERT INTO … variables

FORM or other devices, depends on DBMSand location.

Form code can access form fields,Trigger code cannot access forms.

(6) Procedures and Functions

13

Split code into meaningful pieces that are easier to debug.

CREATE PROCEDURE AS …CREATE FUNCTION AS …

To call a procedure or function:EXEC procedure …variable = function(…)

14

Data Trigger Events

Oracle additions: Tables ALTER, CREATE, DROP User LOGOFF, LOGON Database SERVERERROR, SHUTDOWN,

STARTUP

INSERTDELETEUPDATE

BEFORE AFTER

Example Trigger: Employee Salary

Goal: Keep a separate record of changes to salaries Create a new audit-log table with columns for time, old salary, new salary,

employee changed, and person making the change Trigger: AFTER UPDATE of SALARY on EMPLOYEE Basic code:

INSERT a row into the audit-log table with the tracking data

15

16

Statement v. Row Triggers

UPDATE EmployeeSET Salary = Salary + 10000WHERE EmployeeID=442 OR EmployeeID=558

SQL

time

Before UpdateOn table

After UpdateOn table

Before UpdateRow 442

After UpdateRow 442

UpdateRow 442

… other rows

Triggers for overall table

Triggers for each row

17

Data Trigger Example

CREATE TRIGGER LogSalaryChangesAFTER UPDATE OF Salary ON EmployeeREFERENCING OLD ROW as oldrow

NEW ROW AS newrowFOR EACH ROW

INSERT INTO SalaryChanges (EmpID, ChangeDate, User, OldValue, NewValue)VALUES (newrow.EmployeeID, CURRENT_TIMESTAMP,CURRENT_USER, oldrow.Salary, newrow.Salary);

18

Canceling Data Changes in Triggers

CREATE TRIGGER TestDeletePresidentBEFORE DELETE ON EmployeeREFERENCING OLD ROW AS oldrowFOR EACH ROW

WHEN (oldrow.Title = ‘President’)SIGNAL _CANNOT_DELETE_PRES;

19

Cascading Triggers

Sale(SaleID, SaleDate, …)

OrderItem(OrderID, ItemID, Quantity, …)

Order(OrderID, OrderDate, …)

Inventory(ItemID, QOH, …)

SaleItem(SaleID, ItemID, Quantity, …)

AFTER INSERTUPDATE InventorySET QOH = QOH – newrow.Quantity

AFTER UPDATEWHEN newrow.QOH < newrow.Reorder

INSERT {new order}INSERT {new OrderItem}

20

Trigger Loop

Employee(EID, Salary)

BonusPaid(EID, BonusDate, Amount)

StockOptions(EID, OptionDate, Amount, SalaryAdj)

AFTER UPDATEIF newrow.Salary > 100000 THEN

Add BonusEND

AFTER UPDATE Or INSERTIF newrow.Bonus > 50000 THEN

Reduce BonusAdd Options

END

AFTER UPDATE Or INSERTIF newrow.Amount > 100000 THEN

Reduce SalaryEND

21

Transactions

Some transactions result in multiple changes. These changes must all be completed

successfully, or the group must fail. Protection for hardware and

communication failures. example: bank customer transfers

money from savings account to checking account.

Decrease savings balance Increase checking balance Problem if one transaction and

machine crashes.

Possibly: give users a chance to reverse/undo a transaction.

Performance gain by executing transactions as a block.

Savings Accounts

Inez: 5340.924340.92

Checking Accounts

Inez: 1424.27

Transaction1. Subtract $1000 from savings.

(machine crashes)2. Add $1000 to Checking.

(money disappears)

$1000

22

Transaction Steps

Steps Savings Balance Checking Balance

0. Start 5,340.92 1,424.27

1. Subtract 1,000 4,340.92 1,424.27

2. Add 1,000 4,340.92 2,424.27

Problem arises if transaction is not completed

1. Subtract 1,000 4,340.92 1,424.27

2. Machine crashes 1,000 is gone

Transaction Logs

23

Update Table 1…Update Table 2…

Table 1 Table 2

Log File Transaction changes are written to the log file so they can be restarted if the computer crashes during a transaction.

Low-end DBMSs (e.g., Access) do not have log files.

24

Defining Transactions

The computer needs to be told which changes must be grouped into a transaction.Turn on transaction processing.Signify a transaction start.Signify the end.

Success: save all changesFailure: cancel all changes

Must be set in module codeCommitRollback

25

SQL Transaction CodeCREATE FUNCTION TransferMoney(Amount Currency, AccountFrom Number,

AccountTo Number) RETURNS NUMBERcurBalance Currency;BEGIN

DECLARE HANDLER FOR SQLEXCEPTIONBEGIN

ROLLBACK;Return -2; -- flag for completion error

END;START TRANSACTION; -- optionalSELECT CurrentBalance INTO curBalance FROM Accounts WHERE (AccountID = AccountFrom);IF (curBalance < Amount) THEN

RETURN -1; -- flag for insufficient fundsEND IFUPDATE Accounts SET CurrentBalance = CurrentBalance – AmountWHERE AccountID = AccountFrom;UPDATE AccountsSET CurrentBalance = CurrentBalance + AmountWHERE AccountID = AccountTo;COMMIT;RETURN 0; -- flag for success

END;

26

SAVEPOINT

START TRANSACTION;SELECT …UPDATE …SAVEPOINT StartOptional;UPDATE …UPDATE …If error THEN

ROLLBACK TO SAVEPOINT StartOptional;END IFCOMMIT;

time

startRequired elements

SAVEPOINTStartOptional

Risky stepscommit

Partial rollback

27

Concurrent Access

Concurrent Access Multiple users or processes

changing the same data at the same time.

Final data will be wrong! Force sequential

Locking Delayed, batch updates

Two processes Receive payment ($200) Place new order ($150)

Initial balance $800 Result should be $800 -200 +

150 = $750 Interference result is either $600

or $950

ID BalanceJones $800

$600$950

Customers

1) Read balance 8002) Subtract pmt -2004) Save new bal. 600

3) Read balance 8005) Add order 1506) Write balance 950

Receive Payment Place New Order

28

Concurrent Access Steps

Receive Payment Balance Place New Order

1. Read balance 8002. Subtract Pmt. -200

4. Save balance 600

800

600

950

3. Read balance800

5. Add order1506. Write balance

950

29

Optimistic Locks

Assume that collisions are rare Improved performance, fewer resources Allow all code to read any data (no locks) When code tries to write a new value

Check to see if the existing value is different from the one you were given earlier

If it is different, someone changed the database before you finished, so it is a collision--raise an error

Reread the value and try again

30

Optimistic Locks for Simple Update

(1) Read the balance(2) Add the new order value(3) Write the new balance(4) Check for errors(5) If there are errors, go back to step (1).

31

Optimistic Locks with SQLCREATE FUNCTION ReceivePayment (

AccountID NUMBER, Amount Currency) RETURNS NUMBERoldAmount Currency;testEnd Boolean = FALSE;BEGIN

DO UNTIL testEnd = TRUEBEGIN

SELECT Amount INTO oldAmountWHERE AccountNumber = AccountID;… (slow code)UPDATE AccountsSET AccountBalance = AccountBalance - AmountWHERE AccountNumber = AccountIDAND Amount = oldAmount;COMMIT;IF SQLCODE = 0 and nrows > 0 THEN

testEnd = TRUE;RETURN 0;

END IF-- keep a counter to avoid infinite loops

ENDEND

Optimistic UPDATE: WHERE

32

var: oldAmount = SELECT ……UPDATE …WHERE ID=AccountIDAND (Amount = oldAmount)

To handle missing values, better to write:((Amount = oldAmount) OR (Amount Is Null AND oldAmount Is Null) )

Minimize Concurrency Issues

33

Avoid concurrency issues by keeping code fast and avoiding delays.

Bad:Read Balance SELECT …Do slow stuffAdd new Bal + Bal + NewWrite new total UPDATE …

Better:UPDATE tableSET Balance = Balance + NewValueWHERE …

Delays can arise when you present screens of data to users. Particularly on Web forms.

But, when possible, perform the computations within the DBMS not on the Web server forms.

That means the data displayed to the user sometimes might be out of date, but concurrency within the DBMS is rare.

Also: Avoid storing totals. Just write time-stamped change logs and compute the totals when they are needed.

34

Pessimistic Locks: Serialization

One answer to concurrent access is to prevent it. When a transaction needs to alter data, it places a

SERIALIZABLE lock on the data used, so no other transactions can even read the data until the first transaction is completed.

ID BalanceJones $800

$600

Customers

1) Read balance 8002) Subtract pmt -2004) Save new bal. 600

3) Read balanceReceive error message that it is locked.

Receive Payment Place New Order

SET TRANSACTION SERIALIZABLE, READ WRITE

35

SQL Pessimistic Lock

CREATE FUNCTION ReceivePayment (AccountID NUMBER, Amount Currency) RETURNS NUMBER

BEGINDECLARE HANDLER FOR SQLEXCEPTIONBEGIN

ROLLBACK;RETURN -2;

ENDSET TRANSACTION SERIALIZABLE, READ WRITE;UPDATE AccountsSET AccountBalance = AccountBalance - AmountWHERE AccountNumber = AccountID;COMMIT;RETURN 0;

END

36

Serialization Effects

Receive Payment Balance Place New Order

1. Read balance 8002. Subtract Pmt. -200

4. Save balance 600

800

600

750

3. Read balanceError: Blocked3. Read balance 6004. Add order1505. Write balance 750

37

Transaction to Transfer Money

CREATE FUNCTION ReceivePayment (AccountID NUMBER, Amount Currency) RETURNS

NUMBERBEGIN

DECLARE HANDLER FOR SQLEXCEPTIONBEGIN

ROLLBACK;RETURN -2;

ENDSET TRANSACTION SERIALIZABLE, READ WRITE;UPDATE AccountsSET AccountBalance = AccountBalance - AmountWHERE AccountNumber = AccountID;COMMIT;RETURN 0;

END

38

Deadlock

DeadlockTwo (or more) processes have

placed locks on data and are waiting for the other’s data.

Many solutionsRandom wait timeGlobal lock managerTwo-phase commit - messages

Data A Data B

1) Lock Data A3) Wait for Data B

2) Lock Data B4) Wait for Data A

39

Deadlock Sequence

Process 1 Data A Data B Process2

1. Lock Data A

3. Wait for Data BLocked

by 1Locked

by 22. Lock Data B

4. Wait for Data A

40

Lock Manager

ResourceA

ResourceB

ResourceC

ResourceD

ResourceE

Process1 Lock Wait

Process2 Wait Lock

Process3 Lock

Process4 Lock Wait

Process5 Wait

Process6 Wait Lock

Process7 Wait Wait

41

ACID Transactions

Atomicity: all changes succeed or fail together. Consistency: all data remain internally consistent (when committed) and

can be validated by application checks. Isolation: The system gives each transaction the perception that it is

running in isolation. There are no concurrent access issues. Durability: When a transaction is committed, all changes are

permanently saved even if there is a hardware or system failure.

42

SQL 99/2003 Isolation Levels (Advanced)

READ UNCOMMITTED Problem: might read dirty data that is rolled back Restriction: not allowed to save any data

READ COMMITTED Problem: Second transaction might change or delete data Restriction: Need optimistic concurrency handling

REPEATABLE READ Problem: Phantom rows

SERIALIZABLE Provides same level of control as if all transactions were run sequentially. But, still might encounter locks and deadlocks

43

Phantom Rows(Advanced)

ItemID QOH Price

1115 15

1136 7

11712 30

1184 12

1197 22

1208 17

121 7 16

122 3 14

SELECT SUM(QOH)FROM InventoryWHERE Price BETWEEN 10 and 20Result: 5 + 4 + 8 = 17

INSERT INTO InventoryVALUES (121, 7, 16)INSERT INTO InventoryVALUES (122, 3, 14)

Included in first query

SELECT SUM(QOH)FROM InventoryWHERE Price BETWEEN 10 and 20Result: 5 + 4 + 8 + 7 + 3 = 27

Additional rows will be included in the second query

44

Generated Keys

Create an order for a new customer:

(1) Create new key for CustomerID(2) INSERT row into Customer(3) Create key for new OrderID(4) INSERT row into Order

Customer TableCustomerID, Name, …

Order TableOrderID, CustomerID, …

45

Methods to Generate Keys

1. The DBMS generates key values automatically whenever a row is inserted into a table.

Drawback: it is tricky to get the generated value to use it in a second table.

2. A separate key generator is called by a programmer to create a new key for a specified table.

Drawback: programmers have to write code to generate a key for every table and each row insertion.

Overall drawbacks: neither method is likely to be transportable. If you change the DBMS, you will have to rewrite the procedures to generate keys.

46

Auto-Generated Keys

Create an order for a new customer:

(1) INSERT row into Customer(2) Get the key value that was generated(3) Verify the key value is correct. How?(4) INSERT row into Order

Major problem:Step 2 requires that the DBMS return the key value that was most recently generated. How do you know it is the right value? What happens if two transactions generate keys at almost the same time on the same table?

47

Key-Generation Routine

Create an order for a new customer:

(1) Generate a key for CustomerID(2) INSERT row into Customer(3) Generate a key for OrderID(4) INSERT row into Order

This method ensures that unique keys are generated, and that you can use the keys in multiple tables because you know the value. But, none of it is automatic. It always requires procedures and sometimes data triggers.

48

Database Cursors

PurposeTrack through table or

query one row at a time.Data cursor is a pointer to

active row.

Why?Performance.SQL cannot do everything.

Complex calculations.Compare multiple rows.

Year Sales1998 104,3211999 145,9982000 276,0042001 362,736

1998 104,3211999 145,9982000 276,0042001 362,736

49

Database Cursor Program Structure

DECLARE cursor1 CURSOR FORSELECT AccountBalanceFROM Customer;

sumAccount, balance Currency;SQLSTATE Char(5);BEGIN

sumAccount = 0;OPEN cursor1;WHILE (SQLSTATE = ‘00000’)BEGIN

FETCH cursor1 INTO balance;IF (SQLSTATE = ‘00000’) THEN

sumAccount = sumAccount + balance;END IF

ENDCLOSE cursor1;-- display the sumAccount or do a calculation

END

50

Cursor Positioning with FETCH

DECLARE cursor2 SCROLL CURSOR FORSELECT …OPEN cursor2;FETCH LAST FROM cursor2 INTO …Loop…

FETCH PRIOR FROM cursor2 INTO …End loopCLOSE cursor2;

FETCH positioning options:FETCH NEXT next rowFETCH PRIOR prior rowFETCH FIRST first rowFETCH LAST last rowFETCH ABSOLUTE 5 fifth rowFETCH RELATIVE -3 back 3 rows

51

Problems with Multiple Users

Name SalesAlice 444,321Carl 254,998Donna 652,004Ed 411,736

Original Data

Name SalesAlice 444,321Bob 333,229Carl 254,998Donna 652,004Ed 411,736

Modified Data

New row isadded--whilecode is running.

The SQL standard can prevent this problem with the INSENSITIVE option:DECLARE cursor3 INSENSITIVE CURSOR FOR …

But, this is an expensive approach, because the DBMS usually makes a copy of the data. Instead, avoid moving backwards.

52

Changing Data with Cursors

Year Sales Gain

2000 151,039

2001 179,332

2002 195,453

2003 221,883

2004 223,748

DECLARE cursor1 CURSOR FORSELECT Year, Sales, GainFROM SalesTotalORDER BY YearFOR UPDATE OF Gain;priorSales, curYear, curSales, curGainBEGIN

priorSales = 0;OPEN cursor1;Loop:

FETCH cursor1 INTO curYear, curSales, curGainUPDATE SalesTotalSET Gain = Sales – priorSalesWHERE CURRENT OF cursor1;priorSales = curSales;

Until end of rowsCLOSE cursor1;COMMIT;

END

53

Dynamic Parameterized Cursor Queries

DECLARE cursor2 CURSOR FORSELECT ItemID, Description, PriceFROM InventoryWHERE Price < :maxPrice;maxPrice Currency;BEGIN

maxPrice = … -- from user or other queryOPEN cursor2; -- runs query with current valueLoop:

-- Do something with the rows retrievedUntil end of rowsCLOSE cursor2;

END

Parameters enable you to control the rows retrieved dynamically from within the procedure code. The value is applied when the cursor is opened.

54

Sally’s Pet Store Inventory

Inventory method 1: calculate the current quantity on hand by totaling all purchases and sales every time the total is needed. Drawback: performance

Inventory method 2: keep a running balance in the inventory table and update it when an item is purchased or sold. Drawback: tricky code

Also, you need an adjustment process for “inventory shrink”

55

Inventory QuantityOnHand

ItemIDDescriptionQuantityOnHandListPriceCategory

Merchandise

Add items purchasedSubtract items soldAdjust for shrink

SaleIDItemIDQuantitySalePrice

SaleItem

56

Inventory Events

For a new sale, a row is added to the SaleItem table.

A sale or an item could be removed because of a clerical error or the customer changes his or her mind. A SaleItem row will be deleted.

An item could be returned, or the quantity could be adjusted because of a counting error. The Quantity is updated in the SaleItem table.

An item is entered incorrectly. ItemID is updated in the SaleItem table.

SaleIDItemIDQuantitySalePrice

SaleItem

(1) Add a row.

(2) Delete a row.

(3) Update Quantity.

(4) Update ItemID.

57

New Sale: Insert SaleItem Row

CREATE TRIGGER NewSaleItemAFTER INSERT ON SaleItemREFERENCING NEW ROW AS newrowFOR EACH ROW

UPDATE MerchandiseSET QuantityOnHand = QuantityOnHand – newrow.QuantityWHERE ItemID = newrow.ItemID;

58

Delete SaleItem Row

CREATE TRIGGER DeleteSaleItemAFTER DELETE ON SaleItemREFERENCING OLD ROW AS oldrowFOR EACH ROW

UPDATE MerchandiseSET QuantityOnHand = QuantityOnHand + oldrow.QuantityWHERE ItemID = oldrow.ItemID;

59

Inventory Update SequenceSaleItem Clerk Event Code Merchandise

SaleID101

ItemID15

Quantity10

Quantity 8

1. Enter new sale item, enter Quantity of 10.

3. Change Quantity to 8.

2. Subtract Quantity 10 from QOH.

4. Subtract Quantity 8 from QOH.

ItemID15

QOH50

QOH40

QOH32

Solution that corrects for change

SaleID101

ItemID15

Quantity10

Quantity 8

1. Enter new sale item, enter Quantity of 10.

3. Change Quantity to 8.

2. Subtract Quantity 10 from QOH.

4. Add original Quantity 10 back and subtract Quantity 8 from QOH.

ItemID15

QOH50

QOH40

QOH42

60

Quantity Changed Event

CREATE TRIGGER UpdateSaleItemAFTER UPDATE ON SaleItemREFERENCING OLD ROW AS oldrow

NEW ROW AS newrowFOR EACH ROW

UPDATE MerchandiseSET QuantityOnHand = QuantityOnHand

+ oldrow.Quantity – newrow.QuantityWHERE ItemID = oldrow.ItemID;

61

ItemID or Quantity Changed Event

CREATE TRIGGER UpdateSaleItemAFTER UPDATE ON SaleItemREFERENCING OLD ROW AS oldrow

NEW ROW AS newrowFOR EACH ROWBEGIN

UPDATE MerchandiseSET QuantityOnHand = QuantityOnHand + oldRow.QuantityWHERE ItemID = oldrow.ItemID;

UPDATE MerchandiseSET QuantityOnHand = QuantityOnHand – newRow.QuantityWHERE ItemID = newrow.ItemID;COMMIT;

END