managing memory & locks - series 2 transactions & lock management

46
www.dageop.com Managing Memory and locks Managing Memory & Locks ® ML-02: Transactions & Lock Management DR. SUBRAMANI PARAMASIVAM (MANI)

Upload: dageop-ltd

Post on 18-Jan-2017

276 views

Category:

Data & Analytics


8 download

TRANSCRIPT

Page 1: Managing  Memory & Locks - Series 2 Transactions & Lock management

www.dageop.comManaging Memory and locks

Managing Memory & Locks

®

ML-02: Transactions & Lock Management

DR. SUBRAMANI PARAMASIVAM (MANI)

Page 2: Managing  Memory & Locks - Series 2 Transactions & Lock management

About me

Dr. SubraMANI ParamasivamPhD., MCT, MCSE, MCITP, MCP, MCTS, MCSACEO, Principal Consultant & Trainer @ DAGEOP (UK) Email: [email protected] Blog: http://dataap.org/blog

Follow Us

https://www.facebook.com/pages/YOUR-SQL-MAN-LTD/http://www.youtube.com/user/YourSQLMAN

https://twitter.com/dageop

https://uk.linkedin.com/in/dageop

Proud Sponsor

• SQLBits• SQL Saturdays• MCT Summit• SQL Server Geeks

Summit• Data Awareness

Programme• Dageop’s Data Day

®

www.DataAP.org

SPEAKER

Page 3: Managing  Memory & Locks - Series 2 Transactions & Lock management

ContentsML-02: Transactions & Lock Management

• Lock types and their compatibility• Isolation levels explained• Transactions and lock duration• Resolving contention problems• Implementing row versioning

www.dageop.comManaging Memory and locks

Page 4: Managing  Memory & Locks - Series 2 Transactions & Lock management

ML02 - TRANASACTIONS

www.dageop.comManaging Memory and locks

Page 5: Managing  Memory & Locks - Series 2 Transactions & Lock management

Lock types and their compatibility

• To synchronize access by multiple users to the

same piece of data at the same time.

• Granularity:

• Smaller: Rows, increases concurrency but has a

higher overhead because of more locks.

• Larger: Tables, are expensive in terms of concurrency

because locking an entire table restricts access to

any part of the table by other transactions. But it has

a lower overhead because of fewer locks.

www.dageop.comManaging Memory and locks

YSMTable1

User1

User2

User3

User4

Read

Read

Write

Read

Read / Write

Page 6: Managing  Memory & Locks - Series 2 Transactions & Lock management

Lock types and their compatibility

Group of Locks at multiple levels of granularity (lock hierarchy)

www.dageop.comManaging Memory and locks

In Order Resource Levels Description1 RID Row Identifier locks a single row in a heap table.

2 KEY Row lock within an index used to protect key ranges in serializable transactions.

3 PAGE 8KB page in a database (data or index pages)

4 EXTENT 8 X 8KB Pages (Mixed or Uniform)

5 HoBT Data pages in a heap table that does not have a clustered index.

6 TABLE The entire table (all data and indexes).

7 FILE A database file (mdf, ndf, ldf)

8 APPLICATION An application-specified resource.

9 METADATA Metadata level locks.

10 ALLOCATION_UNIT An allocation unit.

11 DATABASE The entire database.

Page 7: Managing  Memory & Locks - Series 2 Transactions & Lock management

Lock types and their compatibility

Resource locks with different lock modes for concurrent transactions

www.dageop.comManaging Memory and locks

Lock mode Description

Shared (S) A simple SELECT statement to read the data. No other transactions can modify the data

Update (U) Prevents deadlocks. During a Repeatable read or serializable transaction it creates a Shared lock (on page or row), then acquires Exclusive lock for modification of data.

Exclusive (X) Used for data-modification operations, such as INSERT, UPDATE, or DELETE. Exclusive (X) locks prevent access to a resource by concurrent transactions.

Intent (IS, IX, SIX) a) prevent other transactions from modifying the higher-level resource to invalidate the lower level lock.b) improve the efficiency of the Database Engine in detecting lock conflicts at the higher level of granularity.

Schema a) Schema modification (Sch-M) locks the table while the schema changes like DDL is appliedb) schema stability (Sch-S) locks the table when compiling and executing queries and do not block any transactional locks, including exclusive (X) locks.

Bulk Update (BU) Used when bulk copying data into a table also when TABLOCK hint is specified or the table lock on bulk load table option is set using sp_tableoption.

Key-range (RangeS-S, RangeS-U, Rangel-N, RangeX-X)

Protects the range of rows read by a query when using the serializable transaction isolation level. Ensures that other transactions cannot insert rows that would qualify for the queries if the queries were run again.

Page 8: Managing  Memory & Locks - Series 2 Transactions & Lock management

Lock types and their compatibility

www.dageop.comManaging Memory and locks

Resource Levels

RID

KEY

PAGE

EXTENT

HoBT

TABLE

FILE

APPLICATION

METADATA

ALLOCATION_UNIT

DATABASE

Lock mode

Shared (S)

Update (U)

Exclusive (X)

Intent

Schema

Bulk Update (BU)

Key-range

User1

User2

User3

Page 9: Managing  Memory & Locks - Series 2 Transactions & Lock management

Lock types and their compatibility

INTENT LOCKS

www.dageop.comManaging Memory and locks

Lock mode Description

Intent shared (IS) Protects requested or acquired shared locks on some (but not all) resources lower in the hierarchy.

Intent exclusive (IX) Protects requested or acquired exclusive locks on some (but not all) resources lower in the hierarchy. IX is a superset of IS, and it also protects requesting shared locks on lower level resources.

Shared with intent exclusive (SIX)

Protects requested or acquired shared locks on all resources lower in the hierarchy and intent exclusive locks on some (but not all) of the lower level resources. Concurrent IS locks at the top-level resource are allowed.

Intent update (IU) Protects requested or acquired update locks on all resources lower in the hierarchy. IU locks are used only on page resources. IU locks are converted to IX locks if an update operation takes place.

Shared intent update (SIU) A combination of S and IU locks, as a result of acquiring these locks separately and simultaneously holding both locks. For example, a transaction executes a query with the PAGLOCK hint and then executes an update operation. The query with the PAGLOCK hint acquires the S lock, and the update operation acquires the IU lock.

Update intent exclusive (UIX) A combination of U and IX locks, as a result of acquiring these locks separately and simultaneously holding both locks.

Page 10: Managing  Memory & Locks - Series 2 Transactions & Lock management

ISOLOATION LEVELS Explained

www.dageop.comManaging Memory and locks

Page 11: Managing  Memory & Locks - Series 2 Transactions & Lock management

• Isolation levels in SQL Server control the locking and row versioning behaviour of T-SQL statements.

• Defines the degree to which one transaction must be isolated from resource made by other transactions

• Isolation levels are described whether the transaction extracts dirty reads or phantom reads.

• It controls

• Types of locks

• Locks acquired when reading data

• How long the read locks are held

NOTE: A transaction always gets an exclusive lock on any data it modifies, and holds that lock until the

transaction completes, regardless of the isolation level chosen for that transaction.

www.dageop.comManaging Memory and locks

ISOLOATION LEVELS Explained

Page 12: Managing  Memory & Locks - Series 2 Transactions & Lock management

www.dageop.comManaging Memory and locks

ISOLOATION LEVELS ExplainedIsolation level Description Dirty read Non-repeatable

read Phantom

Read uncommitted • Specifies that statements can read rows that have been modified by other transactions but not yet committed. • The lowest level where transactions are isolated only enough to ensure that physically corrupt data is not read

Yes Yes Yes

Read committed (Server default)

Specifies that statements cannot read data that has been modified but not committed by other transactions. This prevents dirty reads. Data can be changed by other transactions between individual statements within the current transaction, resulting in non-repeatable reads or phantom data. This option is the SQL.

No Yes Yes

Repeatable read Specifies that statements cannot read data that has been modified but not yet committed by other transactions and that no other transactions can modify data that has been read by the current transaction until the current transaction completes.

No No Yes

Snapshot • Specifies that data read by any statement in a transaction will be the transaction ally consistent version of the data that existed at the start of the transaction.

• The transaction can only recognize data modifications that were committed before the start of the transaction.

No No No

Serializable • Statements cannot read data that has been modified but not yet committed by other transactions.• No other transactions can modify data that has been read by the current transaction until the current

transaction completes.• Other transactions cannot insert new rows with key values that would fall in the range of keys read by any

statements in the current transaction until the current transaction completes.• the highest level, where transactions are completely isolated from one another

No No No

Page 13: Managing  Memory & Locks - Series 2 Transactions & Lock management

Step1 Step2

www.dageop.comManaging Memory and locks

ISOLOATION LEVELS Explained

Page 14: Managing  Memory & Locks - Series 2 Transactions & Lock management

DEMO

www.dageop.comManaging Memory and locks

Page 15: Managing  Memory & Locks - Series 2 Transactions & Lock management

Transactions and lock duration

www.dageop.comManaging Memory and locks

Page 16: Managing  Memory & Locks - Series 2 Transactions & Lock management

Transactions and lock duration• Lock duration depends on the Isolation Levels.

• duration of the entire transaction.• duration of the statement execution.

• Transaction Isolation Levels• Read Committed, • Read Uncommitted, • Repeatable Read, • Serializable• Snapshot (available after 2005 onwards)• Read Committed Snapshot (available after 2005 onwards)

• Each isolation level protects against a specific concurrency problem

www.dageop.comManaging Memory and locks

Page 17: Managing  Memory & Locks - Series 2 Transactions & Lock management

Transactions and lock duration

LOST UPDATE

• Transaction A

• Begin Transaction

• Updates a table

• Transaction B

• Begin Transaction

• updates the same table again

• Commit Transaction

• Commit Transaction

www.dageop.comManaging Memory and locks

Page 18: Managing  Memory & Locks - Series 2 Transactions & Lock management

Transactions and lock duration

DIRTY READ

• Transaction A

• Begin Transaction

• Read data and put in a variable

• Transaction B • Begin Transaction

• updates the table.

• Commit Transaction

• Update table with variable value

• Commit Transactionwww.dageop.com

Managing Memory and locks

Page 19: Managing  Memory & Locks - Series 2 Transactions & Lock management

Transactions and lock durationNON REPEATABLE READ

• Transaction A

• Begin Transaction

• If table exists BEGIN

• Transaction B

• Begin Transaction

• Delete the table.

• Commit Transaction

• Read from the table (Deleted by Transaction B)

• Commit Transaction

www.dageop.comManaging Memory and locks

Page 20: Managing  Memory & Locks - Series 2 Transactions & Lock management

Transactions and lock durationPHANTOM READ

• Transaction A

• Begin Transaction

• Read from a table (1 row only)

• Transaction B

• Begin Transaction

• Insert value into the table. (1 row)

• Commit Transaction

• Read from the same table (Returns 2 rows)

• Commit Transaction

www.dageop.comManaging Memory and locks

Page 21: Managing  Memory & Locks - Series 2 Transactions & Lock management

Transactions and lock duration• To solve all these problems SQL Server uses

• Shared (S) Lock

• Several shared locks on any resource (row or a page) is possible

• Compatible with other shared locks and also called as READ locks

• Exclusive (X) Lock

• Only one exclusive lock can exist on a resource at any time

• Not compatible with other locks, including shared locks and it is also called as WRITE locks

• Update (U) Lock

• Combination of shared and exclusive locks.

• Compatible with shared locks, but not with Update locks.

www.dageop.comManaging Memory and locks

Page 22: Managing  Memory & Locks - Series 2 Transactions & Lock management

DEMO

www.dageop.comManaging Memory and locks

Page 23: Managing  Memory & Locks - Series 2 Transactions & Lock management

Resolving contention problems

www.dageop.comManaging Memory and locks

Page 24: Managing  Memory & Locks - Series 2 Transactions & Lock management

Resolving contention problems

• Blocking with short durations (not a problem)

• Lots of longer wait times

• contention problems.

www.dageop.comManaging Memory and locks

Page 25: Managing  Memory & Locks - Series 2 Transactions & Lock management

Resolving contention problems

• The following query shows all processes that are blocked and have been waiting

for longer than 10 seconds.

www.dageop.comManaging Memory and locks

SELECT spid, blocked, waittime FROM sysprocesses WHERE blocked <> 0 AND waittime > 10000 ORDER by waittime DESC

Page 26: Managing  Memory & Locks - Series 2 Transactions & Lock management

Resolving contention problems

• Identify

• Transactions

• Why locks more than certain time

• Wait time (Duration)

• Wait type (Lock modes)

• Data resources (Level of granularity)

www.dageop.comManaging Memory and locks

SP_LOCK

Page 27: Managing  Memory & Locks - Series 2 Transactions & Lock management

Resolving contention problems

• Understand the transaction

• DBCC INPUTBUFFER

• Sometimes we may miss it and smart move

with Traces

• SP_WHO2 (My favourite)

• Other DMVs

www.dageop.comManaging Memory and locks

Page 28: Managing  Memory & Locks - Series 2 Transactions & Lock management

Resolving contention problems

TIPS• Don’t pause for user input inside of a transaction

• Optimize physical IO to keep transactions short by addressing fragmentation problems, choosing appropriate indexing strategies, etc.

• Tune long running and inefficient SQL statements

• Don’t select more data than you need for the transaction

• Try to do all the reads in your transaction before the modifications to reduce the duration of exclusive locks

• Don’t open transactions while users are browsing data, wait until they decide to modify it

www.dageop.comManaging Memory and locks

• Break up long transactions into smaller transactions wherever possible

• Choose the least restrictive level of transaction isolation possible. In many cases when you are reading data which is old or doesn’t change much, or when absolute accuracy is not critical Read Uncommitted works fine

• Use stored procedures to minimize server client communication inside the transaction, and to force transactions to complete or abort

• Include only the code necessary for the transaction inside the transaction

Page 29: Managing  Memory & Locks - Series 2 Transactions & Lock management

Resolving contention problems

Choose appropriate locking strategies: • Consider making the default lock level a table lock for lookup tables that don’t change often. Use SP_INDEXOPTION to alter default lock levels.

• Consider forcing row level locking on tables with lots of blocking by using SP_INDEXOPTION to turn off page level locking

• Consider tuning queries using locking hints

• Consider changing SQL Server’s locking behavior by setting a lock timeout

www.dageop.comManaging Memory and locks

Page 30: Managing  Memory & Locks - Series 2 Transactions & Lock management

Resolving contention problems

More on Contentions

Associated increase in concurrency because of the increased CPU cores also causes contention problems according to SQLCAT team.

• Types of Contentions• Latch Contention

• Tempdb Contention

www.dageop.comManaging Memory and locks

Page 31: Managing  Memory & Locks - Series 2 Transactions & Lock management

Resolving contention problems

Latch Contention

Lightweight synchronization process to guarantee consistency of in-memory structures like data pages, index pages.

SQL Server uses

• Buffer Latches to protect pages in the buffer pool

• I/O Latches to protect pages not yet loaded to the buffer pool.

Contention on page latches is the most common scenario encountered on multi-CPU systems www.dageop.com

Managing Memory and locks

Data

Plan

Log Buffe

r Poo

l

1

2

3

4

mdf

ldf

Whenever data is written to or read from a page in the SQL Server buffer pool a worker thread must first acquire a buffer latch for the page.

• PAGELATCH_EX• PAGELATCH_SH

Page 32: Managing  Memory & Locks - Series 2 Transactions & Lock management

Resolving contention problems

www.dageop.comManaging Memory and locks

LATCH LOCK

Guarantee consistency of in-memory structures Guarantee consistency of transactions

SQL Server engine User

Performance Cost is Low High

Held for the duration of physical operation of the in-memory structures.

Held for the duration of the transaction

sys.dm_os_wait_stats sys.dm_os_latch_stats

sys.dm_tran_lockssys.dm_exec_sessions

Page 33: Managing  Memory & Locks - Series 2 Transactions & Lock management

Resolving contention problems

www.dageop.comManaging Memory and locks

• 1 or more affecting factors• High number of logical CPU’s

• SQLCAT research paper says 16+ CPU cores causes Latch Contention• Schema design and access patterns

• Size & density of rows / page and access patterns (read/write/delete)• At the application level

• High level of concurrent requests• Layout of logical files

• Caused by allocation structures (IAM, GAM, SGAM, PFS)• I/O subsystem performance

• SQL Server waiting on I/O subsystem

Page 34: Managing  Memory & Locks - Series 2 Transactions & Lock management

Resolving contention problems

www.dageop.comManaging Memory and locks

Tempdb Contention• Refers to a bottleneck for threads trying to access allocation pages in-

memory. • Hundreds of concurrent queries (Create/Write/Read/Drop)

sys.dm_os_waiting_tasks DMV helps to see the waiting tasks.

Page 35: Managing  Memory & Locks - Series 2 Transactions & Lock management

Resolving contention problems

www.dageop.comManaging Memory and locks

1 or more reasons for tempdb contention

• Global (##) and local (#) temporary table, indexes on temporary table.• Table variable (@), single and multi-statement Table Valued Function (TVF).• Using CURSOR.• Snapshot isolation for row-versioning.• Index creation using SORT_IN_TEMPDB.• Online index operation.• DISTINCT, GROUP BY, ORDER BY, or UNION queries results in work table creation in tempdb.• Memory pressure and smaller amount of grant memory for query execution.• Work tables creation from DBCC CHECKDB.

Page 36: Managing  Memory & Locks - Series 2 Transactions & Lock management

DEMO

www.dageop.comManaging Memory and locks

Page 37: Managing  Memory & Locks - Series 2 Transactions & Lock management

Implementing row versioning

www.dageop.comManaging Memory and locks

Page 38: Managing  Memory & Locks - Series 2 Transactions & Lock management

Implementing row versioning

Integrity of Transactions & consistency of databases

• Locks

• Row Versioning

• When row versioning based isolation level is enabled, the Database Engine maintains versions of

each row that is modified

• Applications can specify that a transaction use the row versions to view data as it existed at the start

of the transaction or query instead of protecting all reads with locks.

• By using row versioning, the chance that a read operation will block other transactions is greatly

reduced. www.dageop.com

Managing Memory and locks

Page 39: Managing  Memory & Locks - Series 2 Transactions & Lock management

Implementing row versioning

Row versioning framework supports• Triggers• Multiple Active Results Sets (MARS)• Online indexing

ISOLATION LEVELS

• READ_COMMITTED_SNAPSHOT database option = ON

• READ_COMMITTED transactions provide statement-level read consistency using row versioning.

• ALLOW_SNAPSHOT_ISOLATION database option = ON

• SNAPSHOT transactions provide transaction-level read consistency using row versioning.www.dageop.com

Managing Memory and locks

Page 40: Managing  Memory & Locks - Series 2 Transactions & Lock management

Implementing row versioning

STEP 1: Enable at database level

ALTER DATABASE AdventureWorks2008R2

SET READ_COMMITTED_SNAPSHOT ON;

GO;

ALTER DATABASE AdventureWorks2008R2

SET ALLOW_SNAPSHOT_ISOLATION ON;

STEP 2: Use it before any transactions

SET TRANSACTION ISOLATION LEVEL SNAPSHOT;

BEGIN TRANSACTION

STEP 3: See the output in System objects & Counterswww.dageop.com

Managing Memory and locks

Page 41: Managing  Memory & Locks - Series 2 Transactions & Lock management

Implementing row versioning

• Row versioning Performance Counter• SQL Server: Transactions Object

• T-SQL Functions & Views• sys.databases• sys.dm_tran_top_version_generators• sys.dm_tran_active_snapshot_database_transactions• sys.dm_tran_transactions_snapshot• sys.dm_tran_version_store• sys.dm_tran_current_transaction• sys.dm_tran_current_snapshot• sys.dm_db_file_space_usage• sys.dm_db_session_space_usage• sys.dm_db_task_space_usage

www.dageop.comManaging Memory and locks

Page 42: Managing  Memory & Locks - Series 2 Transactions & Lock management

@@DBTS Vs Min_Active_RowVersion

• Min_Active_RowVersion returns the minimum active row version in current database. Active row versions refers to the row versions in transactions. • @@DBTS returns the last row version has been used regardless the

transaction status.

Implementing row versioning

www.dageop.comManaging Memory and locks

Page 43: Managing  Memory & Locks - Series 2 Transactions & Lock management

DEMO

www.dageop.comManaging Memory and locks

Page 44: Managing  Memory & Locks - Series 2 Transactions & Lock management

Review

Transactions & Lock Management Lock types and their compatibility Isolation levels explained Transactions and lock duration Resolving contention problems Implementing row versioning

www.dageop.comManaging Memory and locks

Page 45: Managing  Memory & Locks - Series 2 Transactions & Lock management

Q & A

www.dageop.comManaging Memory and locks

Page 46: Managing  Memory & Locks - Series 2 Transactions & Lock management

®

www.dageop.com