high performance managed languages - qcon london … · high performance managed languages martin...
TRANSCRIPT
![Page 1: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/1.jpg)
High Performance
Managed Languages
Martin Thompson - @mjpt777
![Page 2: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/2.jpg)
Really, what is your preferred
platform for building HFT
applications?
![Page 3: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/3.jpg)
![Page 4: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/4.jpg)
Why do you build low-latency
applications on a
GC’ed platform?
![Page 5: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/5.jpg)
![Page 6: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/6.jpg)
1. Let’s set some Context
2. Runtime Optimisation
3. Garbage Collection
4. Algorithms & Design
Agenda
![Page 7: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/7.jpg)
Some Context
![Page 8: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/8.jpg)
Let’s be clear
A Managed Runtime is not
always the best choice…
![Page 9: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/9.jpg)
Latency Arbitrage?
![Page 10: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/10.jpg)
![Page 11: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/11.jpg)
Two questions…
![Page 12: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/12.jpg)
Why build on a
Managed Runtime?
![Page 13: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/13.jpg)
Can managed languages
provide good performance?
![Page 14: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/14.jpg)
We need to follow the
evidence…
![Page 15: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/15.jpg)
![Page 16: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/16.jpg)
Are native languages faster?
![Page 17: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/17.jpg)
Time?
Skills & Resources?
![Page 18: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/18.jpg)
What can, or should, be
outsourced?
![Page 19: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/19.jpg)
CPU vs Memory
Performance
![Page 20: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/20.jpg)
How much time to perform an
addition operation on 2 integers?
![Page 21: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/21.jpg)
1 CPU Cycle
< 1ns
![Page 22: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/22.jpg)
Sequential Access
-
Average time in ns/op to sum all
longs in a 1GB array?
![Page 23: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/23.jpg)
Access Pattern Benchmark
Benchmark Mode Score Error Units
testSequential avgt 0.832 ± 0.006 ns/op
~1 ns/op
![Page 24: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/24.jpg)
Really???
Less than 1ns per operation?
![Page 25: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/25.jpg)
![Page 26: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/26.jpg)
Random walk per OS Page
-
Average time in ns/op to sum all
longs in a 1GB array?
![Page 27: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/27.jpg)
Access Pattern Benchmark
Benchmark Mode Score Error Units
testSequential avgt 0.832 ± 0.006 ns/op
testRandomPage avgt 2.703 ± 0.025 ns/op
~3 ns/op
![Page 28: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/28.jpg)
Data dependant walk per OS Page
-
Average time in ns/op to sum all
longs in a 1GB array?
![Page 29: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/29.jpg)
Access Pattern Benchmark
Benchmark Mode Score Error Units
testSequential avgt 0.832 ± 0.006 ns/op
testRandomPage avgt 2.703 ± 0.025 ns/op
testDependentRandomPage avgt 7.102 ± 0.326 ns/op
~7 ns/op
![Page 30: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/30.jpg)
Random heap walk
-
Average time in ns/op to sum all
longs in a 1GB array?
![Page 31: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/31.jpg)
Access Pattern Benchmark
Benchmark Mode Score Error Units
testSequential avgt 0.832 ± 0.006 ns/op
testRandomPage avgt 2.703 ± 0.025 ns/op
testDependentRandomPage avgt 7.102 ± 0.326 ns/op
testRandomHeap avgt 19.896 ± 3.110 ns/op
~20 ns/op
![Page 32: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/32.jpg)
Data dependant heap walk
-
Average time in ns/op to sum all
longs in a 1GB array?
![Page 33: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/33.jpg)
Access Pattern Benchmark
Benchmark Mode Score Error Units
testSequential avgt 0.832 ± 0.006 ns/op
testRandomPage avgt 2.703 ± 0.025 ns/op
testDependentRandomPage avgt 7.102 ± 0.326 ns/op
testRandomHeap avgt 19.896 ± 3.110 ns/op
testDependentRandomHeap avgt 89.516 ± 4.573 ns/op
~90 ns/op
![Page 34: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/34.jpg)
Then ADD 40+ ns/opfor NUMA access on a server!!!!
![Page 35: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/35.jpg)
Data Dependent Loads
aka “Pointer Chasing”!!!
![Page 36: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/36.jpg)
Performance 101
![Page 37: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/37.jpg)
1. Memory is transported in Cachelines
Performance 101
![Page 38: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/38.jpg)
1. Memory is transported in Cachelines
2. Memory is managed in OS Pages
Performance 101
![Page 39: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/39.jpg)
1. Memory is transported in Cachelines
2. Memory is managed in OS Pages
3. Memory is pre-fetched on
predictable access patterns
Performance 101
![Page 40: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/40.jpg)
Runtime Optimisation
![Page 41: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/41.jpg)
1. Profile guided optimisations
Runtime JIT
![Page 42: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/42.jpg)
1. Profile guided optimisations
2. Bets can be taken and later revoked
Runtime JIT
![Page 43: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/43.jpg)
Branches
void foo()
{
// code
if (condition)
{
// code
}
// code
}
![Page 44: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/44.jpg)
Block A
Branches
void foo()
{
// code
if (condition)
{
// code
}
// code
}
![Page 45: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/45.jpg)
Block A
Block B
Branches
void foo()
{
// code
if (condition)
{
// code
}
// code
}
![Page 46: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/46.jpg)
Block A
Block C
Block B
Branches
void foo()
{
// code
if (condition)
{
// code
}
// code
}
![Page 47: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/47.jpg)
Block A
Block C
Block B
Branches
void foo()
{
// code
if (condition)
{
// code
}
// code
}
Block A
Block C
![Page 48: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/48.jpg)
Block A
Block C
Block B
Branches
void foo()
{
// code
if (condition)
{
// code
}
// code
}
Block A
Block C
Block B
![Page 49: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/49.jpg)
Subtle Branches
int result = (i > 7) ? a : b;
![Page 50: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/50.jpg)
Subtle Branches
int result = (i > 7) ? a : b;
CMOV vs Branch Prediction?
![Page 51: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/51.jpg)
Method/Function Inlining
void foo()
{
// code
bar();
// code
}
![Page 52: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/52.jpg)
Method/Function Inlining
void foo()
{
// code
bar();
// code
}
Block A
![Page 53: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/53.jpg)
Method/Function Inlining
void foo()
{
// code
bar();
// code
}
Block A
bar()
![Page 54: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/54.jpg)
Method/Function Inlining
void foo()
{
// code
bar();
// code
}
Block A
bar()
![Page 55: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/55.jpg)
Method/Function Inlining
void foo()
{
// code
bar();
// code
}
Block A
Block B
bar()
![Page 56: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/56.jpg)
Method/Function Inlining
void foo()
{
// code
bar();
// code
}
Block A
Block B
bar()
Block A
![Page 57: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/57.jpg)
Method/Function Inlining
void foo()
{
// code
bar();
// code
}
Block A
Block B
bar()
Block A
bar()
![Page 58: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/58.jpg)
Method/Function Inlining
void foo()
{
// code
bar();
// code
}
Block A
Block B
bar()
Block A
Block B
bar()
![Page 59: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/59.jpg)
Method/Function Inlining
void foo()
{
// code
bar();
// code
}
i-cache
& code bloat?
![Page 60: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/60.jpg)
“Inlining is THE optimisation.”
- Cliff Click
Method/Function Inlining
![Page 61: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/61.jpg)
Bounds Checking
void foo(int[] array, int length)
{
// code
for (int i = 0; i < length; i++)
{
bar(array[i]);
}
// code
}
![Page 62: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/62.jpg)
Bounds Checking
void foo(int[] array)
{
// code
for (int i = 0; i < array.length; i++)
{
bar(array[i]);
}
// code
}
![Page 63: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/63.jpg)
Subtype Polymorphism
void draw(Shape[] shapes)
{
for (int i = 0; i < shapes.length; i++)
{
shapes[i].draw();
}
}
void bar(Shape shape)
{
bar(shape.isVisible());
}
![Page 64: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/64.jpg)
Subtype Polymorphism
void draw(Shape[] shapes)
{
for (int i = 0; i < shapes.length; i++)
{
shapes[i].draw();
}
}
void bar(Shape shape)
{
bar(shape.isVisible());
}
Class Hierarchy Analysis
& Inline Caching
![Page 65: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/65.jpg)
1. Profile guided optimisations
2. Bets can be taken and later revoked
Runtime JIT
![Page 66: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/66.jpg)
Garbage Collection
![Page 67: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/67.jpg)
Generational Garbage Collection
“Only the good die young.”
- Billy Joel
![Page 68: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/68.jpg)
Eden Survivor 0 Survivor 1
Young/New Generation
TLA
B
TLA
B
Tenured
Virtual
Virtual
Old Generation
Generational Garbage Collection
![Page 69: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/69.jpg)
Modern Hardware (Intel Sandy Bridge EP)
C 1 C n C 1 C nRegisters/Buffers <1ns
L1 L1 L1 L1~4 cycles ~1ns
L2 L2 L2 L2~12 cycles ~3ns
L3 L3~40 cycles ~15ns
~60 cycles ~20ns (dirty hit)
~65ns
DRAM
QPI ~40nsMC MC
DRAM
DRAM
DRAM
DRAM
DRAM
DRAM
DRAM
...
...
...
...
...
...
QPI QPIPCI-e 3 PCI-e 3
40XIO
40XIO
* Assumption: 3GHz Processor
![Page 70: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/70.jpg)
Broadwell EX – 24 cores & 60MB L3 Cache
![Page 71: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/71.jpg)
Eden
Young/New Generation
TLA
B
TLA
B
Thread Local Allocation Buffers
![Page 72: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/72.jpg)
Eden
Young/New Generation
TLA
B
TLA
B
Thread Local Allocation Buffers
• Affords locality of reference
• Avoid false sharing
• Can have NUMA aware allocation
![Page 73: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/73.jpg)
Eden Survivor 0 Survivor 1
Young/New Generation
TLA
B
TLA
B
Virtual
Object Survival
![Page 74: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/74.jpg)
Eden Survivor 0 Survivor 1
Young/New Generation
TLA
B
TLA
B
Virtual
Object Survival
• Aging Policies
• Compacting Copy
• NUMA Interleave
• Fast Parallel Scavenging
• Only the survivors require work
![Page 75: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/75.jpg)
Eden Survivor 0 Survivor 1
Young/New Generation
TLA
B
TLA
B
Tenured
Virtual
Virtual
Old Generation
Object Promotion
![Page 76: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/76.jpg)
Eden Survivor 0 Survivor 1
Young/New Generation
TLA
B
TLA
B
Tenured
Virtual
Virtual
Old Generation
Object Promotion
• Concurrent Collection
• String Deduplication
![Page 77: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/77.jpg)
Compacting Collections
![Page 78: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/78.jpg)
Compacting Collections – Depth first copy
![Page 79: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/79.jpg)
Compacting Collections
![Page 80: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/80.jpg)
Compacting Collections
OS Pages and
cache lines?
![Page 81: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/81.jpg)
EdenE
O
S
O
S
O E
S OO
E
O E O O
OSurvivor
Old
Unused
O
S
E
H
Humongous H
H
O O
G1 – Concurrent Compaction
![Page 82: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/82.jpg)
Azul Zing C4
True Concurrent Compacting
Collector
![Page 83: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/83.jpg)
Where next for GC?
![Page 84: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/84.jpg)
Object Inlining/Aggregation
![Page 85: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/85.jpg)
GC vs Manual Memory Management
Not easy to pick clear winner…
![Page 86: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/86.jpg)
GC vs Manual Memory Management
Managed GC
• GC Implementation
• Card Marking
• Read/Write Barriers
• Object Headers
• Background Overhead
in CPU and Memory
Not easy to pick clear winner…
![Page 87: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/87.jpg)
GC vs Manual Memory Management
Managed GC
• GC Implementation
• Card Marking
• Read/Write Barriers
• Object Headers
• Background Overhead
in CPU and Memory
Not easy to pick clear winner…
Native
• Malloc Implementation
• Arena/pool contention
• Bin Wastage
• Fragmentation
• Debugging Effort
• Inter-thread costs
![Page 88: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/88.jpg)
Algorithms & Design
![Page 89: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/89.jpg)
What is most important to
performance?
![Page 90: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/90.jpg)
• Avoiding cache misses
• Strength Reduction
• Avoiding duplicate work
• Amortising expensive operations
• Mechanical Sympathy
• Choice of Data Structures
• Choice of Algorithms
• API Design
• Overall Design
![Page 91: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/91.jpg)
In a large codebase it is really
difficult to do everything well
![Page 92: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/92.jpg)
It also takes some “uncommon”
disciplines such as:
profiling, telemetry, modelling…
![Page 93: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/93.jpg)
“If I had more time, I would
have written a shorter letter.”
- Blaise Pascal
![Page 94: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/94.jpg)
The story of Aeron
![Page 95: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/95.jpg)
Aeron is an interesting lesson in
“time to performance”
![Page 96: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/96.jpg)
Lots of others exists such at the
C# Roslyn compiler
![Page 97: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/97.jpg)
Time spent on
Mechanical Sympathy
vs
Debugging Pointers
???
![Page 98: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/98.jpg)
Immutable Data & Concurrency
![Page 99: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/99.jpg)
Functional Programming
![Page 100: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/100.jpg)
In Closing …
![Page 101: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/101.jpg)
What does the future hold?
![Page 102: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/102.jpg)
Remember
Assembly vs Compiled
Languages
![Page 103: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/103.jpg)
What about the issues of
footprint, startup time,
GC pauses, etc. ???
![Page 104: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/104.jpg)
![Page 105: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/105.jpg)
![Page 106: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/106.jpg)
![Page 107: High Performance Managed Languages - QCon London … · High Performance Managed Languages Martin Thompson - @mjpt777. ... - Billy Joel. Eden Survivor 0 ... • Concurrent Collection](https://reader031.vdocument.in/reader031/viewer/2022022602/5b58fab37f8b9a4e1b8caa7a/html5/thumbnails/107.jpg)
Blog: http://mechanical-sympathy.blogspot.com/
Twitter: @mjpt777
“Any intelligent fool can make things bigger, more complex, and more violent.
It takes a touch of genius, and a lot of courage, to move in the opposite direction.”
- Albert Einstein
Questions?