1 recovery tuning main techniques put the log on a dedicated disk delay writing updates to the...
TRANSCRIPT
![Page 1: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/1.jpg)
1
Recovery Tuning
Main techniques Put the log on a dedicated disk Delay writing updates to the database
disks as long as possible Setting proper intervals for DB
dumping and checkpointing Reduce the size of large update
transactions
![Page 2: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/2.jpg)
2
Separate Disk for the Log DB2 UDB v7.1 on
Windows 2000 5 % performance
improvement if log is located on a different disk
Controller cache hides negative impact
mid-range server, with Adaptec RAID controller (80Mb RAM) and 2x18Gb disk drives.
0
50
100
150
200
250
300
350
controller cache no controller cache
Th
rou
gh
put
(tu
ples
/sec
)
Log on same disk
Log on separate disk
Figure 2.18 in the textbook shows a 30% improvement
![Page 3: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/3.jpg)
3
Tuning Database Writes Database writes caused by transactions tend
to be random Better to be delayed as much as possible
Sufficient info in the log for recovery But eventually they need to be written to the disk
To reduce recovery time When to write?
Forced: when the buffer is full (or nearly full) Opportunistic: when no extra overhead for disk seeking Checkpoint: force all committed writes to disk
![Page 4: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/4.jpg)
4
Writing Dirty Pages to the Disk When the number of dirty pages is greater
than a given parameter (Oracle 8) When the number of dirty pages crosses a
given threshold (less than 3% of free pages in the database buffer for SQL Server 7)
When the log is full, a checkpoint is forced. This can have a significant impact on performance.
![Page 5: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/5.jpg)
5
Tune Checkpoint Intervals Oracle 8i on Windows
2000 A checkpoint (partial
flush of dirty pages to disk) occurs at regular intervals or when the log is full:
Impacts the performance of on-line processing
+ Reduces the size of log+ Reduces time to
recover from a crash
0
0.2
0.4
0.6
0.8
1
1.2
0 checkpoint 4 checkpoints
Thro
ughp
ut R
atio
![Page 6: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/6.jpg)
6
Group Commit Log-writing a bottleneck if every committing
transaction needs a write to the log Group commit
Write the logs of multiple transactions in batch Need to use a “log buffer” (another thing to
tune!) Better throughput if many concurrent short
update transactions Longer response time for individual transactions
This is a problem if they hold lock Early release of locks can cause problems, but the risk
is remote
![Page 7: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/7.jpg)
7
Log IO - Data
Settings:lineitem ( L_ORDERKEY, L_PARTKEY , L_SUPPKEY,
L_LINENUMBER , L_QUANTITY, L_EXTENDEDPRICE , L_DISCOUNT, L_TAX , L_RETURNFLAG, L_LINESTATUS , L_SHIPDATE, L_COMMITDATE, L_RECEIPTDATE, L_SHIPINSTRUCT , L_SHIPMODE , L_COMMENT );
READ COMMITTED isolation level Empty table Dual Xeon (550MHz,512Kb), 1Gb RAM, Internal RAID
controller from Adaptec (80Mb), 4x18Gb drives (10000RPM), Windows 2000.
![Page 8: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/8.jpg)
8
Log IO - Transactions
No Concurrent Transactions:Insertions [300 000 inserts, 10 threads],
e.g., insert into lineitem values (1,7760,401,1,17,28351.92,0.04,0.02,'N','O','1996-03-13','1996-02-12','1996-03-22','DELIVER IN PERSON','TRUCK','blithely regular ideas caj');
![Page 9: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/9.jpg)
9
Group Commits DB2 UDB v7.1 on
Windows 2000 Log records of many
transactions are written together
Increases throughput by reducing the number of writes
At cost of increased minimum response time.
0
50
100
150
200
250
300
350
1 25
Size of Group Commit
Th
rou
gh
pu
t (t
up
les/
sec)
![Page 10: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/10.jpg)
10
Transaction Chopping Some transactions, in particular
batch transactions, can be very long A lot of log information Very costly for recovery
Solution Transaction chopping An easy to understand concept
Formal work in appendix B of the textbook
![Page 11: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/11.jpg)
11
Summary
In this module, we have covered: The principles of recovery How to optimise recovery-related
options Put the log on a dedicated disk Delay writing updates Using checkpoint and dump properly Reduce the size of update transactions
![Page 12: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/12.jpg)
CS5226 Week 6
Operating System & Database Performance Tuning
![Page 13: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/13.jpg)
13
Outline
Part 1: Operating systems and DBMS
Part 2: OS-related tuning
![Page 14: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/14.jpg)
14
Operating System• Operating system is an
interface between hardware and other software, supporting:
• Processes and threads;
• Paging, buffering and IO scheduling
• Multi-tasking
• File system
• Other utilities such as timing, networking and performing monitoring
hardware
![Page 15: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/15.jpg)
15
Scheduling Process versus thread
Scheduling based on time-slicing, IO, priority etc
Different from transaction scheduling The cost of content switching
When switch is desirable? And when is not? The administrator can set priorities to
processes/threads Case 1: The DBMS runs at a lower priority Case 2: Different transactions run at different priority Case 3: Online transactions with higher priority than
offline transactions
![Page 16: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/16.jpg)
16
Priority Inversion
Let priorities T1 > T2s > T3
T1
T2s
T3
Lock x
Request X
… a solution: priority inheritance
![Page 17: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/17.jpg)
17
Database Buffers
Application buffers
DBMS buffers
OS buffers
•An application can have its own in-memory buffers (e.g., variables in the program; cursors);
•A logical read/write will be issued to the DBMS if the data needs to be read/written to the DBMS;
•A physical read/write is issued by the DBMS using its systematic page replacement algorithm. And such a request is passed to the OS.
•OS may initiate IO operations to support the virtual memory the DBMS buffer is built on.
![Page 18: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/18.jpg)
18
Database Buffer Size Buffer too small, then hit
ratio too smallhit ratio =
(logical acc. - physical acc.) / (logical acc.)
Buffer too large, paging Recommended strategy:
monitor hit ratio and increase buffer size until hit ratio flattens out. If there is still paging, then buy memory.
LOG DATADATA
RAM
Paging Disk
DATABASE PROCESSES
DATABASEBUFFER
![Page 19: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/19.jpg)
19
Buffer Size - Data
Settings:employees(ssnum, name, lat, long, hundreds1,
hundreds2);
clustered index c on employees(lat); (unused) 10 distinct values of lat and long, 100 distinct values
of hundreds1 and hundreds2 20000000 rows (630 Mb); Warm Buffer Dual Xeon (550MHz,512Kb), 1Gb RAM, Internal RAID
controller from Adaptec (80Mb), 4x18Gb drives (10000 RPM), Windows 2000.
![Page 20: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/20.jpg)
20
Buffer Size - Queries
Queries: Scan Queryselect sum(long) from employees;
Multipoint queryselect * from employees where lat = ?;
![Page 21: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/21.jpg)
21
Database Buffer Size SQL Server 7 on
Windows 2000 Scan query:
LRU (least recently used) does badly when table spills to disk as Stonebraker observed 20 years ago.
Multipoint query: Throughput increases
with buffer size until all data is accessed from RAM.
Multipoint Query
0
40
80
120
160
0 200 400 600 800 1000
Buffer Size (Mb)
Th
rou
gh
pu
t (Q
uer
ies/
sec)
Scan Query
0
0.02
0.04
0.06
0.08
0.1
0 200 400 600 800 1000
Buffer Size (Mb)
Th
rou
gh
pu
t (Q
ue
rie
s/s
ec
)
![Page 22: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/22.jpg)
22
Multiprogramming Levels More concurrent users
Better utilization of CPU cycles (and other system resources)
Risk of excessive page swapping More lock conflicts
So how many exactly Depends on transaction profiles
Experiments to find the best value And this parameter may change when application
patterns change Feedback control mechanism
![Page 23: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/23.jpg)
23
Disk Layout and Access Larger disk allocation chunks
improves write performance At the cost of disk utilisation
Setting disk usage factor Low when expecting updates/inserts Higher for scan-type of queries
Prefetching within DBMS; not OS For non-random accesses
![Page 24: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/24.jpg)
24
Scan Performance - Data
Settings:lineitem ( L_ORDERKEY, L_PARTKEY , L_SUPPKEY,
L_LINENUMBER , L_QUANTITY, L_EXTENDEDPRICE , L_DISCOUNT, L_TAX , L_RETURNFLAG, L_LINESTATUS , L_SHIPDATE, L_COMMITDATE, L_RECEIPTDATE, L_SHIPINSTRUCT , L_SHIPMODE , L_COMMENT );
600 000 rows Lineitem tuples are ~ 160 bytes long Cold Buffer Dual Xeon (550MHz,512Kb), 1Gb RAM, Internal RAID
controller from Adaptec (80Mb), 4x18Gb drives (10000RPM), Windows 2000.
![Page 25: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/25.jpg)
25
Scan Performance - Queries
Queries:select avg(l_discount) from lineitem;
![Page 26: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/26.jpg)
26
Usage Factor DB2 UDB v7.1 on
Windows 2000 Usage factor is the
percentage of the page used by tuples and auxiliary data structures (the rest is reserved for future)
Scan throughput increases with usage factor.
0
0.05
0.1
0.15
0.2
70 80 90 100
Usage Factor (%)
Thro
ughp
ut (T
rans
/sec
)
scan
![Page 27: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/27.jpg)
27
Prefetching DB2 UDB v7.1 on
Windows 2000
Throughput increases up to a certain point when prefetching size increases.
0
0.05
0.1
0.15
0.2
32Kb 64Kb 128Kb 256Kb
Prefetching
Thro
ughp
ut (T
rans
/sec
)
scan
![Page 28: 1 Recovery Tuning Main techniques Put the log on a dedicated disk Delay writing updates to the database disks as long as possible Setting proper intervals](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eb75503460f94bc0e95/html5/thumbnails/28.jpg)
28
Summary
In this module, we have covered: A review of OS from the DBMS
perspective How to optimise OS-related
parameters and options Thread Buffer, and File system