cache-aware database systems internalsgrust/teaching/ws0607/mmdbms/dbms-cpu-7-s… · © 2006/07...
TRANSCRIPT
![Page 1: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/1.jpg)
Cache-AwareDatabase Systems Internals
Chapter 7
![Page 2: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/2.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Data Placement in RDBMSs
• A careful analysis of query processing operators and data placement schemes in RDBMS reveals a paradox:
• Workloads perform sequential traversals through parts of the relation that are not referenced.
1. Requested items (attributes) and transfer unit between memory and processor mismatch in size,
2. data placement schemes store data in different order than they are accessed.
2
![Page 3: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/3.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
The “Whole Tuple” Model
• Query operators typically access only a small fraction of a whole tuple (record).
1. Loading the cache with useless data wastes bandwidth.
2. Cache is polluted with useless data.
3. Useless data may enforce the replacement of valuable data that is actually needed by the query processor.
3
![Page 4: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/4.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
N-Ary Storage Model (NSM)
4
RID SSN Name Age
1 0962 Jane 30
2 7658 John 45
3 3859 Jim 20
4 5523 Susa 52
5 9743 Leon 43
6 0618 Dan 47
page header RH1 0962
Jane 30 RH2 7658 John
45 RH3 3859 Jim 20 RH4
RH4 5523 Susan 52
• Record header (RH) contains NULL bitmap, offset to variable length attributes, etc.
![Page 5: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/5.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
NSM and Cache Pollution
• Depending on the query, NSM may exhibit pool cache performance:
• To evaluate the predicate, employ a scan operator to retrieve the Age attribute values.
• If cache block size < record size, the scan operator will incur one cache miss per record.
5
SELECT NameFROM RWHERE Age < 40
![Page 6: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/6.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Decomposed Storage Model• The Decomposed Storage Model
(DSM) is the other extreme. Relations undergo full vertical partitioning.
- n-ary relation ⇒ n binary subrelations (RID, attr).
- Sequential accesses (scan) lead to high spatial locality if queries touch few attributes.
- But: Multi-attribute queries call for subrelation joins.(... ORDER BY Age, Name ?)
6
RID Age
1 30
2 45
3 20
4 52
5 43
6 47
![Page 7: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/7.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
PAX Data Placement• Partition Attributes Across (PAX):
- Try to maximize inter-record spatial locality within each column and within each page.
- Try to minimize record construction cost.
- Low impact on the rest of the DBMS kernel.
• PAX:
- Cache-friendly placement of attributes inside each page (“DSM inside an NSM page”).
7
![Page 8: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/8.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
PAX Data Page Layout
8
page header RH1 0962
Jane 30 RH2 7658 John
45 RH3 3859 Jim 20 RH4
RH4 5523 Susan 52
page header 0962 7658
3589 5523
Jane John Jim Susan
30 52 45 20
• Data remains on its original NSM page. NSM and PAX storage requirements on par.
• Attribute values are grouped into minipages.
![Page 9: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/9.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
NSM vs. DSM vs. PAX
• Local changes to the page manipulation code of an NSM-style DBMS kernel.
• Note: NSM may still win for transaction-processing-style workloads (“wide queries”), DSM may still win for complex decision support queries (“narrow”).
9
NSM DSM PAXInter-record spatial locality ✘ ✔ ✔
Low record reconstruction cost ✔ ✘ ✔
![Page 10: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/10.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
DBMS Instruction Footprints• In DBMS, memory stall times are dominated by L2
data cache misses and L1 instruction cache misses.
• Typical query workloads exhbibit active instruction footprints which clearly exceed L1 i-cache size.
• Volcano model: operator returns control to parent operator immediately after generating one tuple.
Instructions for this operator get evicted fromL1 i-cache and need to be reloaded for next tuple⇒ Instruction cache thrashing.
10
![Page 11: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/11.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Branch History Thrashing• The classical query execution model also incurs
another of i-cache pollution: branch mispredictions.
• Operators check for nullability, data types, comparison outcome, over/underflows, error conditions, ... per record.
• CPU keeps track of branch histories (typically 512–4096 history slots).
• Large instructions footprints with many branch instructions will thrash the branch history unit.
11
![Page 12: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/12.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Operator Instruction Footprints (PostgreSQL)
12
Instruction Footprints
TableScan without predicates 9Kwith predicates 13K
IndexScan 14K
Sort 14K
Nested Loop Join (inner) TableScan 11K
(inner) IndexScan 11K
MergeJoin 12K
HashJoin build 10K
probe 12K
Aggregation COUNT < 1KMAX, MIN < 1.6K
![Page 13: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/13.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Instruction Footprint Analysis• Naive: Study DBMS code to construct static call graph
of functions reachable from an operator.
- Sums up sizes of object code of all possibly called functions ⇒ significant overestimation of size.
• Assume that dynamic function call behavior is largely data independent:Calibrate core DBMS by performing a set of queries over “mini” data instances covering the DBMS’s algebra.
- Shared code only counts once.13
![Page 14: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/14.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
pBuffer Operators
• Given a parent–child pair (p–c) of operators in a query plan, introduce a new buffer operator in between.
1. During query execution, the buffer operator stores a large array of pointers to tuples generated by c.
2. Buffer operators fills its pointer array, control is not returned to p until buffer is fully filled.
3. Once buffer filled (or c exhausted), return control to p and serve tuples from the buffer.
14
c
![Page 15: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/15.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Execution Patterns
• Query operator interleaving without buffer operator:
pcpcpcpcpcpcpcpcpcpcpcpcpcpcp
• With buffer operator (buffer size: 5 tuple pointers):
pcccccpppppcccccpppppcccccppppp
• Avoid i-cache thrashing if the instruction size of p plus c exceeds i-cache size. At least 4 out of 5 executions of operators will find their instructions cached.
15
![Page 16: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/16.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Placing Buffer Operators
• We aim to improve spatial instruction locality. Place a buffer operator above a group of operators whose effective instruction footprint fits into theL1 i-cache:
1. Consider each plan leaf operator to be an execution group of its own.
2. Traverse query tree bottom-up, add operator to current group if adding its footprint does not grow group beyond L1 i-cache size.
16
![Page 17: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/17.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Buffering “Too Much”• Buffer operators introduce overhead during query
execution. Do not introduce buffer operator for operator group, if
- expected cardinality of tuple flow in group is small: operators are only executed a few times(“calibrate” DBMS: measure plan performance with/without buffering at various cardinalities), or
- operator group contains a blocking operator (sort, hash-table building).
17
![Page 18: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/18.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Rescheduling Operators • Do not alter given plan but try to schedule identical
operators together (share object code):
18
scan scan
⋈
π
nested-loops
SR
• Scheduling the two table scans together will save i-cache capacity.
• Relations R,S need to be materialized in memory, however.
scan
![Page 19: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/19.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Hash Join• Hash join is one (of many possible) physical
implementations of the equi-join operator of the relational algebra.
• Hash join is a viable choice to implement join, whenever
- there is no exploitable order on the input relations,or
- the input relations are intermediate results with no index support ⇒ try to construct index on-the-fly.
19
![Page 20: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/20.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Building Temporary Indexes
• The construction of a temporary index for a single join may pay off if the input cardinalties are high.
• In the case of hash join, build a hash table on the join attribute of the smaller input relation (build input).
- Make sure that hash table fits into main memory:
Due to the nature of hashing (no locality), almost all lookups in the hash table will lead to page misses.
20
![Page 21: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/21.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Partitioned Hash Join
21
R
S
R
S
• Partition both input relations. Potentially matching tuples will reside in corresponding partitions.
⇒Resulting partitions fit into memory: hash table for partitions will fit into memory and not incur page misses.
![Page 22: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/22.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
In-Memory Partitioned Hash Join
22
SR
=
=
=
• Choose partition size to match the capacity of the L2 (or L1) cache.
![Page 23: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/23.jpg)
© 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture
One-Pass Hash Clustering
23
R57
17
03
47
92
81
…
03
17
57
…
81
92
H
n
• Apply hash function H to distribute tuples into their partitions.
• Partition size chosen to fit into L1 cache ⇒ large n.
• Random access pattern that writes into n locations.
⇒Cache and TLB thrashing.
![Page 24: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/24.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Controlling the Fan-Out• Problem: Need to split into H partitions to fit into L1
cache blocks size — but that many data streams will trash the TLB or exceed the number of cache blocks.
• Idea: Radix Clustering:Split the relation in multiple, say P, passes.
- In each pass p (1 ≤ p ≤ P) only consider only Bp bits of the join column to control the fan-out.
- That pass will write to Hp = 2Bp partitions. Choose Bp so as to not thrash the TLB/cache.
- Choose P and considered bits such that ∏p ∈ 1...P Hp ≥ H.
24
![Page 25: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/25.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Two-Pass Radix Clustering
25
R
57 (001)
17 (001)
03 (011)
47 (111)
92 (100)
81 (001)
20 (100)
06 (110)
96 (000)
37 (101)
66 (010)
75 (001)
H1(R)
00 57 (001)
17 (001)
81 (001)
96 (000)
75 (001)
01 03 (011)
66 (010)
10 92 (100)
20 (100)
37 (101)
11 47 (111)
06 (110)
H1
H2(H1(R))
0 96 (000)
1 57 (001)
17 (001)
81 (001)
75 (001)
0 66 (010)
1 03 (011)
0 92 (100)
20 (100)
1 37 (101)
0 06 (110)
1 47 (111)
H2
![Page 26: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/26.jpg)
© 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture
Joining Radix-Clustered Relations
• After relation R has been radix-clustered on B = ∑ p ∈1...P Bp bits, each
partition will typically contain∣R∣/ 2B tuples.
• Use regular (nested-loops) join algorithm to join tuples in associated partitions.
26
R(000) 96(001) 57(001) 17(001) 81(001) 75(010) 66(011) 3(100) 92(100) 20(101) 37(110) 6(111) 47
S96 (000)
(000)32(001)17(010)10(010)2(010)66
3 (011)
35 (011)
20 (100)
47 (111)
![Page 27: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/27.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Radix-Cluster: TLB Impact
27
3.4.2 Radix Cluster
To analyze the impact of all three parameters (B, P ,Bp) on radix clustering, we conduct two series of ex-periments, keeping one parameter fixed and varyingthe remaining two.
First, we conduct experiments with various num-bers of bits and passes, distributing the bits evenlyacross the passes. The points in Figure 9 depict the re-sults for a BAT of 8M tuples—the remaining cardinali-ties (! 64M) behave the same way. Up to 6 bits, usingjust one pass yields the best performance (cf. “mil-lisecs”). Then, as the number of clusters to be filledconcurrently exceeds the number of TLB entries (64),the number of TLB misses increases tremendously (cf.“TLB misses”), decreasing the performance. Withmore than 6 bits, two passes perform better than one.The costs of an additional pass are more than compen-sated by having significantly less TLB misses in eachpass using half the number of bits. Analogously, threepasses should be used with more than 12 bits, and fourpasses with more than 18 bits. Thus, the number ofclusters per pass is limited to at most the number ofTLB entries. A second more moderate increase in TLBmisses occurs when the number of clusters exceeds thenumber of L2 cache lines, a behavior which we cannotreally explain.
Similarly, the number of L1 cache misses and L2cache misses significantly increases whenever the num-
1e+07
TLB L1 TLB2 L2 TLB
3L1
2TLB
4
L1 m
isses
4 passes3 passes2 passes1 pass
1e+06
1e+07
L2 m
isses
1e+04
1e+05
1e+06
1e+07
TLB
mis
ses
1e+03
1e+04
5 10 15 20
mill
isecs
number of bits
Figure 9: Performance and Model of Radix-Cluster
ber of clusters per pass exceeds the number of L1 cachelines (1024) and L2 cache lines (32,768), respectively.The impact of the additional L2 misses on the totalperformance is obvious for one pass (it doesn’t oc-cur with more than one pass, as then at most 13 bitsare used per pass). The impact of the additional L1misses on the total performance nearly completely van-ishes due to the heavier penalty of TLB misses and L2misses.
Finally, we notice that the best-case execution timeincreases with the number of bits used.
The following model calculates the total executioncosts for a radix cluster depending on the number ofpasses, the number of bits, and the cardinality:
Tc(P,B,C) =
P "!C"wc+ML1,c
!B
P,C
""lL2+ML2,c
!B
P,C
""lMem
+MTLB,c
!B
P,C
""lTLB
"
with MLi,c(Bp, C) =
2 " |Re|Li +
#$$%
$$&
C " Hp
|Li|Li, if Hp ! |Li|Li
C "!1 + log
!Hp
|Li|Li
"", if Hp < |Li|Li
and MTLB,c(Bp, C) =
2 " |Re|Pg +
#$$%
$$&
|Re|Pg "!
Hp
|TLB|
"if Hp ! |TLB|
C "!1# |TLB|
Hp
", if Hp > |TLB|
|Re|Li and |Cl|Li denote the number of cache lines perrelation and cluster, respectively, |Re|Pg the numberof pages per relation, |Li|Li the total number of cachelines, both for the L1 (i = 1) and L2 (i = 2) caches,and |TLB| the number of TLB entries.
The first term of MLi,c equals the minimal numberof Li misses per pass for fetching the input and storingthe output. The second term counts the number of ad-ditional Li misses, when the number of clusters eitherapproaches the number of available Li or even exceedsthis. MTLB,c is made up analogously. Due to spacelimits, we omit the term that models the additionalTLB misses when the number of clusters exceeds thenumber of available L2 lines. A detailed description ofthese and the following formulae is given in [MBK99].
The lines in Figure 9 represent our model for a BATof 8M tuples. The model shows to be very accurate4.
The question remaining is how to distribute thenumber of bits over the passes. The experimen-tal results—not presented here due to space limits(cf. [MBK99])—showed, that the performance stronglydepend on even distribution of bits.
4On our Origin2000 (250 Mhz) we calibrated lTLB = 228ns,LL2 = 24ns, LMem = 412ns, and wc = 50ns.• MIPS R10000: 64 TLB entries, L1 cache: 1024 32-byte
blocks (32K), L2 cache: 32768 128-byte blocks (4 MB).
+ 1 pass × 2 passes ⋇ 3 passes◻ 4 passes
![Page 28: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/28.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Radix-Cluster: L1/L2 Impact
28
+ 1 pass × 2 passes ⋇ 3 passes◻ 4 passes
3.4.2 Radix Cluster
To analyze the impact of all three parameters (B, P ,Bp) on radix clustering, we conduct two series of ex-periments, keeping one parameter fixed and varyingthe remaining two.
First, we conduct experiments with various num-bers of bits and passes, distributing the bits evenlyacross the passes. The points in Figure 9 depict the re-sults for a BAT of 8M tuples—the remaining cardinali-ties (! 64M) behave the same way. Up to 6 bits, usingjust one pass yields the best performance (cf. “mil-lisecs”). Then, as the number of clusters to be filledconcurrently exceeds the number of TLB entries (64),the number of TLB misses increases tremendously (cf.“TLB misses”), decreasing the performance. Withmore than 6 bits, two passes perform better than one.The costs of an additional pass are more than compen-sated by having significantly less TLB misses in eachpass using half the number of bits. Analogously, threepasses should be used with more than 12 bits, and fourpasses with more than 18 bits. Thus, the number ofclusters per pass is limited to at most the number ofTLB entries. A second more moderate increase in TLBmisses occurs when the number of clusters exceeds thenumber of L2 cache lines, a behavior which we cannotreally explain.
Similarly, the number of L1 cache misses and L2cache misses significantly increases whenever the num-
1e+07
TLB L1 TLB2 L2 TLB
3L1
2TLB
4
L1
mis
se
s
4 passes3 passes2 passes1 pass
1e+06
1e+07
L2
mis
se
s
1e+04
1e+05
1e+06
1e+07
TL
B m
isse
s
1e+03
1e+04
5 10 15 20
mill
ise
cs
number of bits
Figure 9: Performance and Model of Radix-Cluster
ber of clusters per pass exceeds the number of L1 cachelines (1024) and L2 cache lines (32,768), respectively.The impact of the additional L2 misses on the totalperformance is obvious for one pass (it doesn’t oc-cur with more than one pass, as then at most 13 bitsare used per pass). The impact of the additional L1misses on the total performance nearly completely van-ishes due to the heavier penalty of TLB misses and L2misses.
Finally, we notice that the best-case execution timeincreases with the number of bits used.
The following model calculates the total executioncosts for a radix cluster depending on the number ofpasses, the number of bits, and the cardinality:
Tc(P,B,C) =
P "!C"wc+ML1,c
!B
P,C
""lL2+ML2,c
!B
P,C
""lMem
+MTLB,c
!B
P,C
""lTLB
"
with MLi,c(Bp, C) =
2 " |Re|Li +
#$$%
$$&
C " Hp
|Li|Li, if Hp ! |Li|Li
C "!1 + log
!Hp
|Li|Li
"", if Hp < |Li|Li
and MTLB,c(Bp, C) =
2 " |Re|Pg +
#$$%
$$&
|Re|Pg "!
Hp
|TLB|
"if Hp ! |TLB|
C "!1# |TLB|
Hp
", if Hp > |TLB|
|Re|Li and |Cl|Li denote the number of cache lines perrelation and cluster, respectively, |Re|Pg the numberof pages per relation, |Li|Li the total number of cachelines, both for the L1 (i = 1) and L2 (i = 2) caches,and |TLB| the number of TLB entries.
The first term of MLi,c equals the minimal numberof Li misses per pass for fetching the input and storingthe output. The second term counts the number of ad-ditional Li misses, when the number of clusters eitherapproaches the number of available Li or even exceedsthis. MTLB,c is made up analogously. Due to spacelimits, we omit the term that models the additionalTLB misses when the number of clusters exceeds thenumber of available L2 lines. A detailed description ofthese and the following formulae is given in [MBK99].
The lines in Figure 9 represent our model for a BATof 8M tuples. The model shows to be very accurate4.
The question remaining is how to distribute thenumber of bits over the passes. The experimen-tal results—not presented here due to space limits(cf. [MBK99])—showed, that the performance stronglydepend on even distribution of bits.
4On our Origin2000 (250 Mhz) we calibrated lTLB = 228ns,LL2 = 24ns, LMem = 412ns, and wc = 50ns.
3.4.2 Radix Cluster
To analyze the impact of all three parameters (B, P ,Bp) on radix clustering, we conduct two series of ex-periments, keeping one parameter fixed and varyingthe remaining two.
First, we conduct experiments with various num-bers of bits and passes, distributing the bits evenlyacross the passes. The points in Figure 9 depict the re-sults for a BAT of 8M tuples—the remaining cardinali-ties (! 64M) behave the same way. Up to 6 bits, usingjust one pass yields the best performance (cf. “mil-lisecs”). Then, as the number of clusters to be filledconcurrently exceeds the number of TLB entries (64),the number of TLB misses increases tremendously (cf.“TLB misses”), decreasing the performance. Withmore than 6 bits, two passes perform better than one.The costs of an additional pass are more than compen-sated by having significantly less TLB misses in eachpass using half the number of bits. Analogously, threepasses should be used with more than 12 bits, and fourpasses with more than 18 bits. Thus, the number ofclusters per pass is limited to at most the number ofTLB entries. A second more moderate increase in TLBmisses occurs when the number of clusters exceeds thenumber of L2 cache lines, a behavior which we cannotreally explain.
Similarly, the number of L1 cache misses and L2cache misses significantly increases whenever the num-
1e+07
TLB L1 TLB2 L2 TLB
3L1
2TLB
4
L1
mis
se
s
4 passes3 passes2 passes1 pass
1e+06
1e+07
L2
mis
se
s
1e+04
1e+05
1e+06
1e+07
TL
B m
isse
s
1e+03
1e+04
5 10 15 20
mill
ise
cs
number of bits
Figure 9: Performance and Model of Radix-Cluster
ber of clusters per pass exceeds the number of L1 cachelines (1024) and L2 cache lines (32,768), respectively.The impact of the additional L2 misses on the totalperformance is obvious for one pass (it doesn’t oc-cur with more than one pass, as then at most 13 bitsare used per pass). The impact of the additional L1misses on the total performance nearly completely van-ishes due to the heavier penalty of TLB misses and L2misses.
Finally, we notice that the best-case execution timeincreases with the number of bits used.
The following model calculates the total executioncosts for a radix cluster depending on the number ofpasses, the number of bits, and the cardinality:
Tc(P,B,C) =
P "!C"wc+ML1,c
!B
P,C
""lL2+ML2,c
!B
P,C
""lMem
+MTLB,c
!B
P,C
""lTLB
"
with MLi,c(Bp, C) =
2 " |Re|Li +
#$$%
$$&
C " Hp
|Li|Li, if Hp ! |Li|Li
C "!1 + log
!Hp
|Li|Li
"", if Hp < |Li|Li
and MTLB,c(Bp, C) =
2 " |Re|Pg +
#$$%
$$&
|Re|Pg "!
Hp
|TLB|
"if Hp ! |TLB|
C "!1# |TLB|
Hp
", if Hp > |TLB|
|Re|Li and |Cl|Li denote the number of cache lines perrelation and cluster, respectively, |Re|Pg the numberof pages per relation, |Li|Li the total number of cachelines, both for the L1 (i = 1) and L2 (i = 2) caches,and |TLB| the number of TLB entries.
The first term of MLi,c equals the minimal numberof Li misses per pass for fetching the input and storingthe output. The second term counts the number of ad-ditional Li misses, when the number of clusters eitherapproaches the number of available Li or even exceedsthis. MTLB,c is made up analogously. Due to spacelimits, we omit the term that models the additionalTLB misses when the number of clusters exceeds thenumber of available L2 lines. A detailed description ofthese and the following formulae is given in [MBK99].
The lines in Figure 9 represent our model for a BATof 8M tuples. The model shows to be very accurate4.
The question remaining is how to distribute thenumber of bits over the passes. The experimen-tal results—not presented here due to space limits(cf. [MBK99])—showed, that the performance stronglydepend on even distribution of bits.
4On our Origin2000 (250 Mhz) we calibrated lTLB = 228ns,LL2 = 24ns, LMem = 412ns, and wc = 50ns.
1024 L1 cache entries
32768 L2 cache entries
![Page 29: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/29.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Joins and Projections in DSM
• Like any other relation, in DSM a join result consists of two columns — regardless of the schema of the query containing the join.
- Consider:
• How will the projection (SELECT clause) be handled?29
SELECT R.A1,...,R.An, S.B1,...,S.BmFROM R,SWHERE R.key = S.key
![Page 30: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/30.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Pre–Projection (NSM)
30
key A1 A2 key B1 B2
π πA1,...,An,key B1,...,Bm,key
√radix-cluster
√radix-cluster
⋈hash join
![Page 31: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/31.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Joins and Projections in DSM• DSM join result contains pairs of row identifiers
(RIDs) of qualifying tuples (a so-called join index).
• To construct the fully projected join result, perform RID-based lookups in the join input relations.
31
R ⋈ S3 3
3 5
4 4
0 2
7 0
7 1
RA1 A2 A3
0 A 0 E 0 x
1 A 1 E 1 z
2 B 2 F 2 z
3 D 3 H 3 y
4 C 4 G 4 z
5 D 5 H 5 z
6 D 6 H 6 y
7 D 7 H 7 y
![Page 32: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/32.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Post–Projection
32
A2 A1 key key B1 B2
π πkey key
√radix-cluster
√radix-cluster
⋈hash join
π π
![Page 33: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/33.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Post–Projection andPartitioned Hash Join
• But: Due to the nature of partitioned hash join, the RIDs (MonetDB: oid values) of neither input relation will appear sorted in the two-column join index.Standard improvement:
1. Sort join index in order of RIDs of larger relation.
2. Access tuples of larger relation using a sequential scan (spatial locality).
3. Tuples of smaller relation accessed randomly but may fit into the cache (temporal locality).
33
![Page 34: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/34.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Projection UsingRadix Cluster
• Fully sorting the join index is overkill. A partial ordering can also achieve the same effect:
• Cluster the join index, each cluster contains RIDs of a disjoint range.
• Choose cluster size such that the contained RIDs point to a limited region in the projection column.
• If this region fits into the cache, post-projection will approach optimal cache (re-)usage.
34
![Page 35: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/35.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Partial Radix Cluster
35
R ⋈ S3 3
3 5
4 4
0 2
7 07 1
(0??) 3 3
(0??) 3 5
(0??) 0 2
(1??) 4 4
(1??) 7 0
(1??) 7 1
0 3
1 3
2 0
3 4
4 7
5 7
RA1 A2 A3
0 A 0 E 0 x1 A 1 E 1 z2 B 2 F 2 z3 D 3 H 3 y4 C 4 G 4 z5 D 5 H 5 z6 D 6 H 6 y7 D 7 H 7 y
πR(R ⋈ S)A1 A2 A3
0 D 0 H 0 y
1 D 1 H 1 y
2 A 2 E 2 x
3 C 3 G 3 z
4 D 4 H 4 y
5 D 5 H 5 y
Partial RadixCluster (1 bit)
MonetDB/MIL: mark()
![Page 36: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/36.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Database Operation andSimultaneous Multithreading• Simulatenous multithreading (SMT) improves CPU
performance by supporting thread-level parallelism on a single processor.
- An SMT processor pretends to be multiple logical processors.
- Multiple threads may issue instructions on each cycle and share processor resources.
• SMT on Intel® CPUs (Xeon, Pentium 4): “Hyper-Threading Technology”
36
![Page 37: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/37.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Simultaneous Multithreading• In an SMT systems, threads share the memory bus
and the caches.
• SMT intrinsically performs worse than a system with multiple physical CPUs(threads may even compete for a shared resource).
• Improvement of single-threaded CPUs: Higher instruction throughput (e.g., continue to run thread A while thread B waits for cache miss).
• Idea: Use shared cache to communicate betweenthreads.
37
![Page 38: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/38.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Bi-Threaded Operators
• Try to perform thread-based data cache preloading:
1. Operator execution itself happens in a main thread.
2. A separate helper thread works ahead of the main thread and preloads data thatwill soon be needed by the main thread.
• Data preloading in helper thread can be overlapped with CPU computation in main thread.
• Helper thread suffers most of the memory latency,main thread will experience a higher cache hit rate.
38
![Page 39: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/39.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Work-Ahead Set1. Main thread posts soon-to-be-needed memory
references p (plus a small amount of state information s) into work-ahead set: { (p,s) }.
2. Helper thread reads (p,_) from work-ahead set, accesses p and thus populates the data cache.
• Note: Correct operation of main thread is not dependent on helper thread.
39
work-aheadset
(p1,s1)
(p2,s2)
(p3,s3)
…
mainthread post (p,s)
helperthreadread (p,_)
![Page 40: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/40.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Work-Ahead Set
• Why not use prefetching in the main thread?
- Requires manual computation of prefetching distance and scheduling of prefetch instructions.
- Prefetches are not binding and may be droppedby the CPU (e.g., if a TLB miss would result).
40
(p1,s1)(p2,s2)(p3,s3)…
mainthread post (p,s)
helperthreadread (p,_)
RAMCache
![Page 41: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/41.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Staged Data Access• Main operator threads need to be written to access
data in stages.
- Example: In hash join, separate probe of hash table and construction of output tuple (post-projection).Two stages for a probe/construction:
1. Main thread posts address p of probed hash bucket to work-ahead set. Then continues with other computation (e.g., hashing of further join keys).
2. Later, main thread removes p from work-ahead set and only then accesses the buckets contents.
41
![Page 42: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/42.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Work-Ahead Set: Post• Simple implementation of work-ahead set:
Circular list or array.
• Main thread posts at next available array slot, in round-robin fashion.
- If slot already contained a (p,s) pair, this is returned to the main thread.
- At the end of the computation, main thread may need to post dummy references to retrieve remaining entries.
42
work-aheadset
(p1,s1)
(p2,s2)
(p3,s3)
…
![Page 43: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/43.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Work-Ahead Set: Read• Helper thread maintains its own offset i into the array.
Reads entries in circular fashion — no coordination with main thread.
- Helper thread performs a read access to given memory reference. In C:
• If helper thread runs faster than main thread, the same data will be loaded multiple times.
- If combination〈i,p〉has been seen before, put
thread in spin loop, polling for new entries.43
temp += *((int*)p);
![Page 44: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/44.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Work-Ahead Set Entries• One operation might touch data types larger than a
cache line or might need multiple references to execute a single instruction, e.g.
- Divide execution into three stages.But: Too many stages risk intermediate cache eviction.
- Alter structure of work-ahead set entry: (p1,p2,p3,s).Three times fewer entries, helper thread is supposed to access all three references together.
44
*a = *b + *c;
![Page 45: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/45.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Work-Ahead Set Entries• Use work-ahead set entries with M memory
references each (M: maximum of references needed per stage in main thread).
- If an operation needs k < M entries, patch the remaining M - k entries with a dummy address d: (p1,p2,d,d,d,d,s)
- No special treatment in helper thread, always access all M references (reduce branches).
- Dummy cache line at d will be accessed often and thus reside in the L1 cache. Negligible overhead.
45
![Page 46: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/46.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Staged Database Operators
• Requires reimplementation of each query operator. Helper thread is general and can be reused.
• Identify stages in operator execution:When a memory location p2 is only known after accessing another memory location p1 has been read, process p1 and p2 in different stages.
• Operators typically perform only simple computations per tuple ⇒ # of required memory references ≈ operator arity.
46
![Page 47: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/47.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Staged Index Traversal
• Stage 1: On lookup for key K, traverse B+-tree node and compare with keyi values. When branch number is known, post (child node address,K) to work-ahead set.
• Stage 2:Via post receive (child node address′,K′). Access that node and resume search for key K′.
⇒Multiple index probes active “in parallel.”47
key2 ≤ K < key3
key0 key1 key2 key3-∞ ∞
![Page 48: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/48.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Staged Hash Join
• Hash table, chained hash cells(= tuple of build table hashed to the associated bucket).
• In hash cell:
48
( RID,h(K′) )
• Operator receives (RID,K) from probe input.
• Perform hash probe in four stages. In each stage: (1) compute; (2) issue memory reference for next stage.
![Page 49: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/49.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
next stage
Staged Hash Join
• Stages:
1. Compute bucket number via hash function h(K), post (hash bucket addr, (h(K),2) ).
2. Visit hash bucket, post (first cell addr, (h(K),3) ).
3. Visit cell, evaluate h(K) = h(K′). If equal, post (RID, (K,4)), else post(next cell addr, (h(K),3)).
4. Visit potentially matching tuple in build relation, compare its key with K. If equal, build result tuple, else post(next cell addr, (h(K),3)).
49
![Page 50: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/50.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Work-Ahead Set Size• Work-ahead set too small:
- Insufficient time for helper thread to preload.
- High probability that main and helper thread access same cache line in work-ahead set. Post of main thread dirties cache line: flush to RAM, reload.
• Work-ahead set too large:
- Loads performed by helper thread may be evicted from cache before used by main thread.Keep size of preloaded data less than (a fraction of) L2 cache size.
50
![Page 51: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/51.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Main / Helper Thread Speed• Helper thread faster than main thread:
- Helper thread preloads entire work-ahead set, catches up with main thread. Goes into spin-loop.
- When main thread advances, helper thread will wake up, also advances one slot, then spin-loops.Same cache line written / read, dirty cache lines, flushing, reloading.
• One solution: Let helper thread process the work-ahead set in a backwards fashion.
51
![Page 52: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/52.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
B+–Trees andCache Performance
• B+–trees still are the prevailing index structures in database systems.
• The many B+–tree supported operations of a database kernel (selection, access to inner relation of a join) break down into two low-level access patterns:
1. Descending from root to leaf while searching for a key K,
2. scanning a region of the index leaf level while retrieving the RIDs for a range ℓ < K ≤ r.
52
![Page 53: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/53.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
B+–Trees andCache Performance
• Both access patterns effectively follow a linked list of pointers.
- Each traversal step potentially incurs the cost of page miss (disk-based indexes) or cache miss (memory-based index), since index node size and unit of transfer are aligned.
- Research has found B+–tree key searches to spend more than 65% of the overall search time waiting for data cache misses.
53
![Page 54: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/54.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
B+–Tree Index Node Size• In-memory, the unit of transfer is two orders of
magnitude smaller than for disk-based indexes(≈ 64 byte cache line vs. 4 KB disk page).
⇒ In-memory indexes are considerably deeper.
- As tree height determines number of cache misses during key search, try to increase fan-out (branching factor):
- One approach: (Almost) No pointers in inner index nodes, instead point to node group. Double fan-out.
54
![Page 55: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/55.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Prefetching B+–Trees• Remember that for a modern CPU and memory sub-
system, it is not the number of cache misses but the amount of exposed miss latency that counts.
• Use prefetching in B+–trees to limit exposed miss latency.
- Searching and scanning in B+–trees pose a challenging problem since both constitutepointer-chasing problems.
- Adapt the original B+–tree concept to enable prefetching that reaches sufficiently far ahead.
55
![Page 56: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/56.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Wider Index Nodes• Expect one cache miss while moving down a level of
the tree. How to reduce tree depth?
- B+–tree of 1000 keys, node size 64 bytes (cache line), key size = pointer size = RID size = 4 bytes,B+–tree will contain ≥ 4 levels. Miss latency 150 cy:
- But: Simply making nodes wider than transfer unit can hurt performance.
56
node 1 node 2 node 3 node 4
time (cycles)
cache misses
0 150 300 450 600
![Page 57: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/57.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Wider Index Nodes• Double the node width. One node occupies two
cache lines (128 bytes). New tree depth: 3 levels.
• Cache misses during B+–tree key search:
- Search now suffers six cache misses (instead of four), decreasing search performance.
57
node 1 node 2 node 3
time (cycles)
cache misses
0 150 300 450 600 900750
![Page 58: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/58.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Wide Index NodesPlus Prefetching
• Prefetch second half of each two-cache-line-widenode already upon node entry.
- Second half of node at known (contiguous) address.
• Assume: Memory can serve two back-to-back cache misses with a 10 cycle delay (10 cy ≪ 150 cy).
58
node 1 node 2 node 3
time (cycles)
cache misses
0 160 320 480
![Page 59: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/59.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Minor Modifications to B+–Tree Algorithms
• Search:Upon node entry, before starting binary search, prefetch all cache lines that comprise a node.
• Insert:Perform index search, brings root-to-leaf path into cache. Cache misses only for newly allocated nodes.
• Deletion:Perform index search, root-to-leaf path in cache. Redistribute only on completely empty nodes. In that case, prefetch sibling node (not on prefetched path).
59
![Page 60: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/60.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Prefetching Index Scans• Given start and end key, perform index search for
start key to obtain start leaf node.
- Then let the scan follow next leaf pointers, visiting leaf nodes in order.
- Copy scanned RIDs (or tuples) into result buffer.
- Stop scan if end key reached or result buffer full (may resume scan later on).
• Studies found 84% data cache stall time during index scans.
60
![Page 61: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/61.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Wide Leaf Nodes• Apply prefetching idea to yield wider leaf nodes:
• But index scans lead to a longer sequence of node accesses. Goal: Try to prefetch farther ahead:
61
leaf 1 leaf 2
time (cycles)
cache misses
0 160 320
time (cycles)
cache misses
0 180
![Page 62: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/62.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Pointer-Chasing Problem• Assume that three leaf nodes worth of computation
are needed to hide miss latency.
- When node ni is visited, a prefetch of node ni+3 should be initiated.Following the pointer chain through nodes ni+1 and ni+2 is no option.
• Possible solutions:
1. Data linearization — only for read-only workloads.
2. Jump pointers (embed prefetch address in node).
62
![Page 63: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/63.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Jump Pointers
• Pull jump pointers out of nodes into separate jump pointer array. Added flexibility:
- Adjust prefetching distance easily (adapt to changing performance conditions, model different latencies, e.g., database host or memory vs. disk):
63
ni ni+1 ni+2 ni+3 ni+4… …
ni ni+1 ni+2 ni+3 ni+4… …
ni ni+1 ni+2 ni+3 ni+3
3
![Page 64: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/64.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Updating Jump Pointers
• B+–tree insertion may lead to node split at leaf level: Shift jump pointer array, update back pointers.
64
B+–tree
leaf level
jump pointers
B+–tree
leaf level
chunked jump pointerswith holes
…
![Page 65: Cache-Aware Database Systems Internalsgrust/teaching/ws0607/MMDBMS/DBMS-CPU-7-s… · © 2006/07 • Prof. Dr. Torsten Grust Database Systems and Modern CPU Architecture ... -Low](https://reader031.vdocument.in/reader031/viewer/2022011801/5ae3a0a67f8b9a90138ddc34/html5/thumbnails/65.jpg)
Database Systems and Modern CPU Architecture© 2006/07 • Prof. Dr. Torsten Grust
Internal Jump Pointers• Avoid B+–tree external jump pointer array. Instead
reuse pointers from leaf parent level.
• Chain leaf parents in order to support rapid prefetching (overhead: one pointer per node).
• No back pointers needed: remember branching number in leaf parent during the search for start key.
65
B+–tree
leaf level
leaf parents