1 transaction reordering speaker :po-hung lai adviser : chih-hung lin date :2011.1.4

42
1 Transaction reordering Speaker :Po-Hung Lai Speaker :Po-Hung Lai Adviser : Chih-Hung Lin Adviser : Chih-Hung Lin Date :2011.1.4 Date :2011.1.4

Upload: lily-russell

Post on 04-Jan-2016

217 views

Category:

Documents


0 download

TRANSCRIPT

11

Transaction reordering

Speaker :Po-Hung LaiSpeaker :Po-Hung Lai

Adviser : Chih-Hung LinAdviser : Chih-Hung Lin

Date :2011.1.4Date :2011.1.4

22

OutlineOutline

IntroductionIntroduction Transaction reordering frameworkTransaction reordering framework MethodMethod ExperimentExperiment conclusionconclusion

33

IntroductionIntroduction

Traditional Traditional :: allows a certain number of complex queriesallows a certain number of complex queries

a crasha crash

focus on the current system statusfocus on the current system status

Our Our :: information about the interaction information about the interaction

wait runwait run

current system statuscurrent system status

44

IntroductionIntroduction Why transaction reordering be effectiveWhy transaction reordering be effective

– system independent system independent

eliminates some intrinsic lock conflictseliminates some intrinsic lock conflicts– system dependentsystem dependent

55

IntroductionIntroduction Advantage/ flawAdvantage/ flaw

insideinside outsideoutside

Advantage : Advantage :

current state of the systemcurrent state of the system

synchronized scanssynchronized scans more opportunities for more opportunities for

reorderingreordering

Don’t change database engineDon’t change database engine

Flaw :Flaw :Need set system and reorder Need set system and reorder

modelmodelSystem will be miss transaction System will be miss transaction

an additional parsing of each tran additional parsing of each transaction ansaction

66

IntroductionIntroduction

general transaction reordering frameworkgeneral transaction reordering framework– current system status current system status – queued and running transactions queued and running transactions

77

IntroductionIntroduction

reordering algorithms reordering algorithms – Run all the transactions Run all the transactions

– Analyze and record transactions Analyze and record transactions

88

IntroductionIntroduction

Our reordering algorithms Our reordering algorithms – Luck conflictsLuck conflicts– buffer pool buffer pool

99

Transaction reordering frameworkTransaction reordering framework

Two method to increase the transaction throughput: Two method to increase the transaction throughput:

– increasing concurrency preventing conflicting transactions preventing conflicting transactions

– sharing resources No deadlock No deadlock

PurposePurpose– improve RDBMS performance

1010

Our methodOur method typical operational data warehouse typical operational data warehouse [16][16]

– OLTP data warehouseOLTP data warehouse(Operational Load Transaction Process)(Operational Load Transaction Process)

– Acknowledge client input somethingAcknowledge client input something– Ensuring data’s durabilityEnsuring data’s durability

Fig :operational data warehouseFig :operational data warehouse

1111

Our methodOur method

loading data continuously into an RDBMSloading data continuously into an RDBMS

Files Files 、 、 message message 、 、 queues …queues …

1212

Our methodOur method

Continuous load utilities Continuous load utilities definitiondefinition– ([20]Oracle([20]Oracle ,, [6]Teradata[6]Teradata))

– like RDBMS like RDBMS series of transactions series of transactions – free to arbitrarily groupfree to arbitrarily group

free to arbitrarily group Characteristic– It’s not anomalies

1313

Our methodOur method

applications tolerate such anomalies applications tolerate such anomalies

Ex Ex 刪除記錄後 紀錄表刪除記錄後 紀錄表 尚未更新記錄前尚未更新記錄前 交易隨即移動走交易隨即移動走 使其發生之異常使其發生之異常

1414

Our methodOur method

application ensures not allow such anomalieapplication ensures not allow such anomaliess

ex ex 在新增交易在新增交易資料完成之前資料完成之前不允許做任何不允許做任何動作動作

1515

Our methodOur method

Increase concurrencyIncrease concurrency

Files Files 、 、 message message 、 、 queues …queues …

1616

Our methodOur method

Materialized view maintenanceMaterialized view maintenance– Important class of Materialized view call join vieImportant class of Materialized view call join vie

ww

Deadluck occurDeadluck occur– becausebecause

allow data load into multiple relations allow data load into multiple relations RDBMS’s data as up-tpdateRDBMS’s data as up-tpdate JV is defined on multiple base relations JV is defined on multiple base relations

1717

Our methodOur method

Deadluck probabilityDeadluck probability– k > 1 concurrent transactik > 1 concurrent transacti

ons ons – n modification operations n modification operations – p = A’s probabilityp = A’s probability– P-1 = B’s probabilityP-1 = B’s probability– S = totally S = totally S >> knS >> kn

)2/()1)(1( 2 snkpp

1818

Our methodOur method

ExampleExample– p = 50%p = 50% ,, k = 8k = 8 ,, n = 32 and s = 10000n = 32 and s = 10000

– Deadluck probability = 9%Deadluck probability = 9%

– n = 64 n = 64 , , Deadluck probability = 36%Deadluck probability = 36%

1919

Our methodOur method

Solution follow as ruleSolution follow as rule– data can only be loaded into one base relation data can only be loaded into one base relation

of JV of JV – Modification operations Modification operations – uses a high concurrency locking protocol uses a high concurrency locking protocol

2020

Our methodOur method

we schedule transactions as follows we schedule transactions as follows – Action 1 Action 1

2121

Our methodOur method

– Action 2Action 2

2222

Our methodOur method

– Action 3Action 3– schedule a transaction to the RDBMS schedule a transaction to the RDBMS

desirable transaction T or first transactions desirable transaction T or first transactions

starvation problemstarvation problem– Use timestamp(TS) to solutionUse timestamp(TS) to solution– give pointer r ; r = 0 ; give pointer r ; r = 0 ; (0<=r<=k)(0<=r<=k)

2323

Our methodOur method

Situation Situation – Qm Qm (1<=m<=k)(1<=m<=k) is empty is empty ,, r = 0r = 0– Anytime r =0; transaction coming Qm Anytime r =0; transaction coming Qm r = m r = m– r = m ; first transaction leave ; r = r+1r = m ; first transaction leave ; r = r+1

if (m = kif (m = k , , r =1)r =1)

v = r last number ; TS is pre-defined v = r last number ; TS is pre-defined – If v > ts then Qv transaction become firstIf v > ts then Qv transaction become first

2424

Our methodOur method

2525

Our methodOur method

buffer pool analysis buffer pool analysis – buffer management methods cannot utilize the sbuffer management methods cannot utilize the s

ynchronized scan technique efficiently ynchronized scan technique efficiently

– solution to this problem solution to this problem synchronized scan techniquesynchronized scan technique

2626

Our methodOur method lower RDBMS’s capability problemlower RDBMS’s capability problem

2727

Our methodOur method lower RDBMS’s capability problemlower RDBMS’s capability problem

– involve full table scans involve full table scans

2828

Our methodOur method

Solution : synchronized scan techniqueSolution : synchronized scan technique

2929

Our methodOur method

Our methodOur method– addition addition K1 buffer; K1 buffer;

Hold recent Hold recent resourceresource

– DS ;DS ; Record T1 need all resourceRecord T1 need all resource

– fast scan; slow scanfast scan; slow scan

3030

Our methodOur method

T2 start can check DBT2 start can check DB

3131

Our methodOur method

T2 start scanT2 start scan

3232

Our methodOur method

DefectDefect– t1 doing transactiont1 doing transaction– t2 transaction end t2 transaction end – New transaction can’t enter share resourceNew transaction can’t enter share resource

3333

Our methodOur method

ImproveImprove– Applying transaction reorderingApplying transaction reordering

Technique1 maintain Technique1 maintain hash tablehash table (HT) and find (HT) and find desirable transactiondesirable transaction

Technique2 new transaction , check DSTechnique2 new transaction , check DS

ProposePropose– improve the throughput improve the throughput – economize I/O resource economize I/O resource

3434

experimentexperiment

IBM DB2IBM DB2– ConcurrencyConcurrency

deadlock probability and throughput testing resultsdeadlock probability and throughput testing results

– deadlock probability. Fig. 5 and 6deadlock probability. Fig. 5 and 6

P =50% ; s=10000 ; n=1 to 128 ; K =2 or 4 or 6 P =50% ; s=10000 ; n=1 to 128 ; K =2 or 4 or 6 or 8 or 8

3535

experimentexperiment

ExEx

P =50% ; s=10000 ; n=64 ; K =2P =50% ; s=10000 ; n=64 ; K =2

Deadluck probability = 5% Deadluck probability = 5%

3636

experimentexperiment

Predicted deadlock probability Measured deadlock probability of the naive

of the naive method method

3737

experimentexperiment

Throughput of the naive method Throughput of the reordering

method

3838

experimentexperiment

throughput throughput

Naive’Naive’

probabilityprobability

Naive’Naive’

throughputthroughput

ReorderReorder

probabilityprobability

ReorderReorder

throughputthroughput

deadluckdeadluck badbad BadBad GoodGood goodgood

No deadlNo deadluckuck

SimilerSimiler

ReorderReorder

SimilerSimiler

ReorderReorder

SimilerSimiler

NaiveNaive

SimilerSimiler

NaiveNaive

3939

experimentexperiment

Data ratio test results Data ratio test results – K=16; n=64; p=50%; s= 5000 to 20000K=16; n=64; p=50%; s= 5000 to 20000– Fig10Fig10

4040

experimentexperiment

Data ratio test resultsData ratio test results– throughput of Reprdering and naivethroughput of Reprdering and naive

4141

experimentexperiment

Transaction ratio test results Transaction ratio test results

4242

conclusionconclusion

Improve the performance of an RDBMS

Combine running transaction and wait transaction

Predict the occurrence of deadlock