how tokudb fractal treetm indexes work - mit opencourseware · how tokudb fractal tree tm indexes...
TRANSCRIPT
![Page 1: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/1.jpg)
How TokuDB Fractal TreeTM Indexes Work
Bradley C. Kuszmaul
Guest Lecture in MIT 6.172 Performance Engineering, 18 November 2010.
6.172 —How Fractal Trees Work 1
![Page 2: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/2.jpg)
My Background• I’m an MIT alum:
MIT Degrees = 2 × S.B + S.M. + Ph.D. • I was a principal architect of the
Connection Machine CM-5 supercomputer at Thinking Machines.
• I was Assistant Professor at Yale. • I was Akamai working on network mapping and
billing.• I am research faculty in the SuperTech group,
working with Charles.6.172 —How Fractal Trees Work 2
![Page 3: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/3.jpg)
TokutekA few years ago I started collaborating with Michael Bender and Martin Farach-Colton on how to store data on disk to achieve high performance.
We started Tokutek to commercialize the research.
6.172 —How Fractal Trees Work 3
![Page 4: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/4.jpg)
I/O is a Big Bottleneck
Disk
Sensor
Sensor
Sensor
Sensor
Query
Query
Query
per second
Millions of data
elements arriveQuery recently
arrived data
using indexes.
Systems include sensors and storage, and want to perform queries on recent data.
6.172 —How Fractal Trees Work 4
![Page 5: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/5.jpg)
The Data Indexing Problem
• Data arrives in one order (say, sorted by the time of the observation). • Data is queried in another order (say, by URL or
location).
Disk
Sensor
Sensor
Sensor
Sensor
Query
Query
Query
per second
Millions of data
elements arriveQuery recently
arrived data
using indexes.
6.172 —How Fractal Trees Work 5
![Page 6: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/6.jpg)
Why Not Simply Sort?
Data Sorted by Time
Sort
Data Sorted by URL
This is what data • warehouses do.
• The problem is that you must wait to sort the data before querying it: typically an overnight delay.
The system must maintain data in (effectively) several sorted orders. This problem is called maintaining indexes. 6.172 —How Fractal Trees Work 6
![Page 7: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/7.jpg)
B-Trees are Everywhere
B-Trees show up in database indexes (such as MyISAM and InnoDB), file systems (such as XFS), and many other storage systems.
6.172 —How Fractal Trees Work 7
![Page 8: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/8.jpg)
B-Trees are Fast at Sequential InsertsB In Memory
B B
· · ·
· · ·
Insertions are into this leaf node
• One disk I/O per leaf (which contains many rows).• Sequential disk I/O. • Performance is limited by disk bandwidth.
6.172 —How Fractal Trees Work 8
![Page 9: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/9.jpg)
B-Trees are Slow for High-Entropy InsertsB In Memory
B B
· · ·
· · ·
• Most nodes are not in main memory. • Most insertions require a random disk I/O. • Performance is limited by disk head movement. • Only 100’s of inserts/s/disk (≤ 0.2% of disk
bandwidth).
6.172 —How Fractal Trees Work 9
![Page 10: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/10.jpg)
New B-Trees Run Fast Range Queries B
B B
· · ·
· · ·
Range Scan
• In newly created B-trees, the leaf nodes are often laid out sequentially on disk. • Can get near 100% of disk bandwidth. • About 100MB/s per disk.
6.172 —How Fractal Trees Work 10
![Page 11: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/11.jpg)
Aged B-Trees Run Slow Range QueriesB
B B
· · ·
· · · · · ·
Leaf Blocks Scattered Over Disk
• In aged trees, the leaf blocks end up scattered over disk. • For 16KB nodes, as little as 1.6% of disk
bandwidth.• About 16KB/s per disk.
6.172 —How Fractal Trees Work 11
![Page 12: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/12.jpg)
• Looking up anything requires a table scan.
Append-to-file Beats B-Trees at InsertionsHere’s a data structure that is very fast for insertions:
5 4 2 7 9 4
Write next key here
Write to the end of a file.
Pros: • Achieve disk bandwidth even for random keys.
Cons:
6.172 —How Fractal Trees Work 12
![Page 13: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/13.jpg)
Append-to-file Beats B-Trees at InsertionsHere’s a data structure that is very fast for insertions:
5 4 2 7 9 4
Write next key here
Write to the end of a file.
Pros: Achieve disk bandwidth even for random keys. •
Cons: Looking up anything requires a table scan. •
6.172 —How Fractal Trees Work 13
![Page 14: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/14.jpg)
Fractal Tree Good Good Good
Yes, Fractal Trees!
A Performance Tradeoff?Structure Inserts Point Queries Range Queries B-Tree Horrible Good Good (young) Append Wonderful Horrible Horrible
• B-trees are good at lookup, but bad at insert. • Append-to-file is good at insert, but bad at lookup.• Is there a data structure that is about as good as a
B-tree for lookup, but has insertion performance closer to append?
6.172 —How Fractal Trees Work 14
![Page 15: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/15.jpg)
A Performance Tradeoff?Structure Inserts Point Queries Range Queries B-Tree Horrible Good Good (young) Append Wonderful Horrible Horrible Fractal Tree Good Good Good • B-trees are good at lookup, but bad at insert. • Append-to-file is good at insert, but bad at lookup.• Is there a data structure that is about as good as a
B-tree for lookup, but has insertion performance closer to append?
Yes, Fractal Trees!
6.172 —How Fractal Trees Work 15
![Page 16: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/16.jpg)
An Algorithmic Performance Model To analyze performance we use the Disk-Access Machine (DAM) model. [Aggrawal, Vitter 88]
• Two levels of memory.
• Two parameters: block size B, andmemory size M.
• The game: Minimize the numberof block transfers. Don’t worryabout CPU cycles.
6.172 —How Fractal Trees Work 16
© Source unknown. All rights reserved. This content is excludedfrom our Creative Commons license. For more information,see http://ocw.mit.edu/fairuse.
![Page 17: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/17.jpg)
Theoretical ResultsStructure Insert Point Query B-Tree O
log N log B
O logN log B
Append O 1 B
O N B
log N logNFractal Tree O O
B1−ε ε log B1−ε
6.172 —How Fractal Trees Work 17
![Page 18: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/18.jpg)
Example of Insertion Cost
1 billion 128-byte rows. N = 230; log(N) = 30.• • 1MB block holds 8192 rows. B = 8192; logB = 13.
logN 30B-Tree: O
log B = O
13 ≈ 3
Fractal Tree: O logN
= O 30 ≈ 0.003.
B 8192
Fractal Trees use << 1 disk I/O per insertion.
6.172 —How Fractal Trees Work 18
![Page 19: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/19.jpg)
A Simplified Fractal Tree
5 10
3 6 8 12 17 23 26 30
• log N arrays, one array for each power of two.
• Each array is completely full or empty.
• Each array is sorted.
6.172 —How Fractal Trees Work 19
![Page 20: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/20.jpg)
Example (4 elements)If there are 4 elements in our fractal tree, the structure looks like this:
23 30 12 17
6.172 —How Fractal Trees Work 20
![Page 21: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/21.jpg)
If there are 10 elements in our fractal tree, the structure might look like this:
5 10
3 6 8 12 17 23 26 30
But there is some freedom. • Each array is full or empty, so the 2-array and the
8-array must be full.• However, which elements go where isn’t
completely specified.
6.172 —How Fractal Trees Work 21
![Page 22: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/22.jpg)
Searching in a Simplified Fractal Tree• Idea: Perform a binary
search in each array.
5 10 • Pros: It works. It’s fasterthan a table scan.
3 6 8 12 17 23 26 30 Cons: It’s slower than a• B-tree at O(log2 N) block transfers.
Let’s put search aside, and consider insert.
6.172 —How Fractal Trees Work 22
![Page 23: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/23.jpg)
Inserting in a Simplified Fractal Tree
5 10
3 6 8 12 17 23 26 30
Add another array of each size for temporary storage.
At the beginning of each step, the temporary arrays are empty.
6.172 —How Fractal Trees Work 23
![Page 24: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/24.jpg)
Insert 15To insert 15, there is only one place to put it: In the1-array.
5 10
3 6 8 12 17 23 26 30
15
6.172 —How Fractal Trees Work 24
![Page 25: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/25.jpg)
Insert 7To insert 7, no space in the 1-array. Put it in the temp1-array.
5 10
3 6 8 12 17 23 26 30
15 7
Then merge the two 1-arrays to make a new 2-array.
5 10
3 6 8 12 17 23 26 30
7 15
6.172 —How Fractal Trees Work 25
![Page 26: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/26.jpg)
Not done inserting 7
5 10
3 6 8 12 17 23 26 30
7 15
Must merge the 2-arrays to make a 4-array.
3 6 8 12 17 23 26 30
10 155 7
6.172 —How Fractal Trees Work 26
![Page 27: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/27.jpg)
An Insert Can Cause Many Merges
9 5 10 2 18 33 40 3 6 8 12 17 23 26 30
31
9 5 10 2 18 33 40 3 6 8 12 17 23 26 30
9 31
5 10 2 18 33 40 3 6 8 12 17 23 26 30
5 9 10 31
2 18 33 40 3 6 8 12 17 23 26 30
2 5 9 10 18 31 33 40
3 6 8 12 17 23 26 30
2 3 5 6 8 9 10 12 17 18 23 26 30 31 33 40
6.172 —How Fractal Trees Work 27
![Page 28: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/28.jpg)
•
Analysis of Insertion into SimplifiedFractal Tree
• Cost to merge 2 arrays of size X is O(X/B) block I/Os.
5 7 10 15 Merge is very I/O efficient. 3 6 8 12 17 23 26 30
• Cost per element to merge is O(1/B) since O(X) elements were merged.Max # of times each element is merged is O(log N).
log NAverage insert cost is O .•
B 6.172 —How Fractal Trees Work 28
![Page 29: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/29.jpg)
Improving Worst-Case InsertionAlthough the average cost of a merge is low,occasionally we merge a lot of stuff.
3 6 8 12 17 23 26 30 4 7 9 19 20 21 27 29
Idea: A separate thread merges arrays. An insertreturns quickly.
Lemma: As long as we merge Ω(logN) elements forevery insertion, the merge thread won’t fall behind.
6.172 —How Fractal Trees Work 29
![Page 30: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/30.jpg)
Speeding up Search
At log2 N, search is too expensive.Now let’s shave a factor of log N.
3 6 8 12 17 23 26 30
5 7 10 15
The idea: Having searched an array for a row, we know where that row would belong in the array. We can gain information about where the row belongs in the next array
6.172 —How Fractal Trees Work 30
![Page 31: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/31.jpg)
Forward Pointers9 Each element gets a forward2 14 pointer to where that element5 7 13 25 goes in the next array using3 6 8 12 17 23 26 30 Fractional Cascading. [Chazelle, Guibas 1986]
If you are careful, you can arrange for forward pointers to land frequently (separated by at most a constant). Search becomes O(log N) levels, each looking at a constant number of elements, for O(log N) I/Os.
6.172 —How Fractal Trees Work 31
![Page 32: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/32.jpg)
Industrial-Grade Fractal TreesA real implementation, like TokuDB, must deal with • Variable-sized rows; • Deletions as well as insertions; • Transactions, logging, and ACID-compliant crash
recovery;• Must optimize sequential inserts more; • Better search cost: O(logB N), not O(log2 N); • Compression; and • Multithreading.
6.172 —How Fractal Trees Work 32
![Page 33: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/33.jpg)
iiBench Insert Benchmark
iiBench was developed by us and Mark Callaghan tomeasure insert performance.Percona took these measurements about a year ago.6.172 —How Fractal Trees Work 33
![Page 34: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/34.jpg)
iiBench on SSD
TokuDB on rotating disk beats InnoDB on SSD.
6.172 —How Fractal Trees Work 34
![Page 35: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/35.jpg)
Disk Size and Price Technology Trends
• SSD is getting cheaper. • Rotating disk is getting cheaper faster. Seagate
indicates that 67TB drives will be here in 2017.• Moore’s law for silicon lithography is slower over
the next decade than Moore’s law for rotating disks.
Conclusion: big data stored on disk isn’t going awayany time soon.Fractal Tree indexes are good on disk.One cannot simply indexes in main memory. Onemust use disk efficiently.
6.172 —How Fractal Trees Work 35
![Page 36: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/36.jpg)
Speed Trends
• Bandwidth off a rotating disk will hit about500MB/s at 67TB.• Seek time will not change much.
Conclusion: Scaling with bandwidth is good. Scaling with seek time is bad.
Fractal Tree indexes scale with bandwidth.
Unlike B-trees, Fractal Tree indexes can consume many CPU cycles.
6.172 —How Fractal Trees Work 36
![Page 37: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/37.jpg)
Power Trends
• Big disks are much more power efficient per byte stored than little disks. • Making good use of disk bandwidth offers further
power savings.
Fractal Tree indexes can use 1/100th the power of B-trees.
6.172 —How Fractal Trees Work 37
![Page 38: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/38.jpg)
CPU Trends
• CPU power will grow dramatically inside serversover the next few years. 100-core machines arearound the corner. 1000-core machines are on thehorizon.• Memory bandwidth will also increase. • I/O bus bandwidth will also grow.
Conclusion: Scale-up machines will be impressive.
Fractal Tree indexes will make good use of cores.
6.172 —How Fractal Trees Work 38
![Page 39: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/39.jpg)
The Future
• Fractal Tree indexes dominate B-trees theoretically.• Fractal Tree indexes ride the right technology
trends. • In the future, all storage systems will use Fractal
Tree indexes.
6.172 —How Fractal Trees Work 39
![Page 40: How TokuDB Fractal TreeTM Indexes Work - MIT OpenCourseWare · How TokuDB Fractal Tree TM Indexes Work Bradley C. Kuszmaul Guest Lecture in MIT 6.172 Performance Engineering, 18 November](https://reader031.vdocument.in/reader031/viewer/2022030821/5b35559a7f8b9aad388bc0ab/html5/thumbnails/40.jpg)
MIT OpenCourseWarehttp://ocw.mit.edu
6.172 Performance Engineering of Software Systems Fall 2010
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.