Download - Debunking Dynamic Optimization Myths
IBM Research
Feb 18, 2005 NC State Seminar © 2005 IBM Corporation
Debunking Dynamic Optimization Myths
Michael HindBased on PLDI’04 Tutorial by Fink, Grove, and Hind
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation2
Well-known Facts
1. Because they execute at runtime, dynamic compilers must be blazingly fast
2. Dynamic class loading is a fundamental roadblock to cross-method optimization
3. Sophisticated profiling is too expensive to perform online
4. A static compiler will always produce better code than a dynamic compiler
5. Program optimization is a dead field
6. The infrastructure investment stifles innovation in this field
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation3
Well-known Facts ???
1. Because they execute at runtime, dynamic compilers must be blazingly fast
2. Dynamic class loading is a fundamental roadblock to cross-method optimization
3. Sophisticated profiling is too expensive to perform online
4. A static compiler will always produce better code than a dynamic compiler
5. Program optimization is a dead field
6. The infrastructure requirements stifles innovation in this field
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation4
Terminology
Virtual Machine (for this talk): a software execution engine for a program written in a machine-independent language• Ex, Java bytecodes, CLI, Pascal p-code, Smalltalk v-code
Program Loader
ThreadScheduler
SecurityServices
LibrariesMemoryManagement
RuntimeSupport
Mechanisms
Dynamic type checkingIntrospection, etc.
Tracing,Profiling, etc(ex. JVMPI)
Interpreter Compiler(s) Adaptive Optimization System
VM != JIT
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation5
Adaptive Optimization Hall of Fame
1958-1962
1974
1980-1984
1986-1994
1995-present
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation6
Adaptive Optimization Hall of Fame
1958-1962: LISP
1974: Adaptive Fortran
1980-1984: ParcPlace Smalltalk
1986-1994: Self
1995-present: Java
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation7
Quick History of VMs
LISP Interpreters [McCarthy’78]• First widely used VM
• Pioneered VM services
– memory management, Eval -> dynamic loading
Adaptive Fortran [Hansen’74]• First in-depth exploration of adaptive optimization
• Selective optimization, models, multiple optimization levels, online profiling and control systems
ParcPlace Smalltalk [Deutsch&Schiffman’84]• First modern VM
• Introduced full-fledge JIT compiler, inline caches, native code caches
• Demonstrated software-only VMs were viable
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation8
Quick History of VMs
Self [Chambers&Ungar’91, Hölzle&Ungar’94]• Developed many advanced VM techniques
• Introduced polymorphic inline caches, on-stack replacement, dynamic de-optimization, advanced selective optimization, type prediction and splitting, profile-directed inlining integrated with adaptive recompilation
• Pure OO language that supports an interactive development environment
• Much of the technology was transferred to Sun’s HotSpot JVM
Java/JVM [Gosling et al ‘95]• First VM with mainstream market penetration
• Java vendors embraced and improved Smalltalk and Self technology
• Encouraged VM adoption by others -> CLR
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation9
Other VMs in this Talk
IBM DK for Java [’95- ]• Port of Sun Classic JVM with JIT, GC, and synch enhancements
• World class performance
Jikes RVM (Jalapeno) [’97- ]• VM for Java, written in (mostly) Java
• Independently developed VM + GNU Classpath libs
• Open source, popular with researchers, not a full JVM
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation10
Myth 1 - Dynamic compilers must be blazingly fast
Myth: Because they execute at runtime, dynamic compilers must be blazingly fast. • they cannot perform sophisticated optimizations, such as SSA,
graph-coloring register allocation, inlining, etc.
Reality:• Production JITs perform all the classical optimizations
• Selective optimization strategies successfully focus compilation effort where needed
What is Selective Optimization?
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation11
How are Programs Executed?
1. Interpretation• Popular approach for high-level languages
– Ex, APL, SNOBOL, BCPL, Perl, Python, MATLAB
• Useful for memory-challenged environments
• Low startup overhead, but much slower than native code execution
2. Classic just-in-time compilation• Compile each method to native code on first invocation
– Ex, ParcPlace Smalltalk-80, Self-91– Initial high (time & space) overhead for each compilation– Precludes use of sophisticated optimizations (eg. SSA, etc.)
• Responsible for many of today’s myths
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation12
Selective Optimization
Hypothesis: most execution is spent in a small pct. of methods
Idea: use two execution strategies1. Interpreter or non-optimizing compiler
2. Full-fledged optimizing compiler
• Strategy: • Use option 1 for initial execution of all methods
• Profile application to find “hot” subset of methods
• Use option 2 on this subset
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation13
Selective Optimization Examples
Adaptive Fortran: interpreter + 2 compilers
Self’93: non-optimizing + optimizing compilers
JVMs• Interpreter + compilers: Sun’s HotSpot, IBM DK for Java, IBM’s J9
• Multiple compilers: Jikes RVM, Intel’s Judo/ORP, BEA’s JRocket
CLR?• Currently, only 1 runtime compiler, i.e., a classic JIT
– But, also use ahead-of-time (AOT) compilation
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation14
Profiling: How to Find Candidates for Optimization
Counters
Call Stack Sampling
Combinations
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation15
How to Find Candidates for Optimization: Counters
Insert method-specific counter on method entry and loop back edge
Counts how often a method is called • approximates how much time is spent in a method
Very popular approach: Self, HotSpot Issues: overhead for incrementing counter can be
significant• Not present in optimized codefoo ( … ) {
fooCounter++;
if (fooCounter > Threshold) {
recompile( … );
}
. . .
}
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation16
How to Find Candidates for Optimization: Call Stack Sampling
Periodically record which method(s) are on the call stack Approximates amount of time spent in each method Does not necessarily need to be compiled into the code
• Ex) Jikes RVM, JRocket Issues: timer-based sampling is not deterministic
ABC
AB
A AB
ABC
ABC
......
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation17
How to Find Candidates for Optimization: Call Stack Sampling
Periodically record method(s) on call stack• Approximates amount of time spent in each method
Does not necessarily need to be compiled into the code• Ex) Jikes RVM, JRocket
Issues: timer-based sampling is not deterministic
ABC
AB
A AB
ABC
ABC
......
Sample
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation18
How to Find Candidates for Optimization: Combinations
Use counters initially and sampling later on• Ex) IBM DK for Java
foo ( … ) {
fooCounter++;
if (fooCounter > Threshold) {
recompile( … );
}
. . .
}
ABC
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation19
Recompilation Policies: Which Candidates to Optimize?
Problem: given optimization candidates, which ones should be optimized?
Counters: 1. Optimize method that surpasses threshold
– Simple, but hard to tune, doesn’t consider context
2. Optimize method on the call stack based on inlining policies (Self, HotSpot)
– Addresses context issue Call Stack Sampling:
1. Optimize all methods that are sampled
− Simple, but doesn’t consider frequency of sampled methods
2. Use Cost/benefit model (Jikes RVM)
– Seemingly complicated, but easy to engineer– Maintenance free– Naturally supports multiple optimization levels
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation20
Jikes RVM Architecture [Arnold et al. ’00, ‘04]
AOS Database
Compilation Queue
Event Queue
Controller
OPT Compiler
CompilationThread
ExecutingCode
Hot MethodOrganizer
MethodSamples
Take Sample
Runtime Measurements
AOS
Recompilation Subsystem
Samples occur at taken yield points (approx 100/sec)Organizer thread communicates sampled methods to controller
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation21
Jikes RVM: Recompilation Policy – Cost/Benefit Model
Define• cur, current opt level for method m
• Exe(j), expected future execution time at level j
• Comp(j), compilation cost at opt level j Choose j > cur that minimizes Exe(j) + Comp(j)
If Exe(j) + Comp(j) < Exe(cur) recompile at level j
Assumptions• Sample data determines how long a method has executed
• Method will execute as much in the future as it has in the past
• Compilation cost and speedup are offline averages
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation22
IBM DK for Java [Suganuma et al. ’01]
Execution Levels (excluding Specialization)
MMI (Mixed Mode Interpreter)•Fast interpreter implemented in assembler
Quick compilation•Reduced set of optimizations for fast compilation, little inlining
Full compilation•Full optimizations only for selected hot methods
Methods can progress sequentially through the levels
1st LevelCompiled Code
2nd LevelCompiled Code
Mixed Mode Interpreter
Hot Method Sampling
Dynamic Compiler
Profiling System
Quick OptCompiler
Sampling Profiler
Full OptCompiler
MMI Profiler
ByteCode
Invocation FrequencyLoop Iteration
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation23
Startup Programs: Jikes RVM
0
1
2
3
4
5
db/10
jack/10
ipsi
xql/
short
jess
/10
jbb
/12
000
mtr
t/10
java
c10
xerc
es/
short
mpeg/10
com
pre
ss/10
daik
on/sh
ort
soot/
short
jack/10
0
xerc
es/
long
java
c/10
0
jess
/10
0
mrt
r/10
0
db/10
0
ipsi
xql/
long
soot/
long
jbb
/200
000
com
pre
s/10
0
mpeg/10
0
daik
on/lo
ng
Geom
Speedup o
ver
Base
line
JIT 0 JIT 1 JIT 2
No FDO, Mar’04, AIX/PPC
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation24
Startup Programs: Jikes RVM
0
1
2
3
4
5
db/1
0
jack
/10
ipsi
xql
/sho
rt
jess
/10
jbb/1
2000
mtr
t/10
java
c10
xer
ces/
shor
t
mpe
g/10
com
pres
s/10
daik
on/s
hort
soot
/sho
rt
jack
/10
0
xer
ces/
long
java
c/10
0
jess
/10
0
mrt
r/10
0
db/1
00
ipsi
xql
/lon
g
soot
/lon
g
jbb/2
0000
0
com
pres
/100
mpe
g/10
0
daik
on/l
ong
Geo
m
Spe
edup
ove
r Bas
elin
e
JIT 0 JIT 1 JIT 2 Model
No FDO, Mar’04, AIX/PPC
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation25
Steady State: Jikes RVM
0
1
2
3
4
5
6
7
jbb
-30
0
ipsi
xql
com
pres
s
jess db
java
c
mpe
gaud
io
mtr
t
jack
Geo
mea
n
Spe
edup
ove
r Bas
elin
e
J I T 0 J I T 1 J I T 2
No FDO, Mar’04, AIX/PPC
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation26
Steady State: Jikes RVM, no FDO (Mar ’04)
0
1
2
3
4
5
6
7
jbb
-30
0
ipsi
xql
com
pres
s
jess db
java
c
mpe
gaud
io
mtr
t
jack
Geo
mea
n
Spe
edup
ove
r Bas
elin
e
J I T 0 J I T 1 J I T 2 Model
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation27
Execution Profile: Jikes RVM (Jul ’02)
86.6%
6.2%
0.1%
0.0%
0.0%
0.6%
6.5%
Application Threads
Garbage Collection
Controller
Method organizer
Decay organizer
Inlining organizer
Opt. Recompilation
Size 100, SPECjvm98, 1 run each
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation28
Myth 1 - Dynamic compilers must be blazingly fast
Myth: Because they execute at runtime, dynamic compilers must be blazingly fast. • they cannot perform sophisticated optimizations, such as SSA,
graph-coloring register allocation, inlining, etc.
Reality:• Production JITs perform all the classical optimizations
• Selective optimization strategies successfully focus compilation effort where needed
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation29
Understanding System Behavior
PLDI’04 tutorial contains• Performance results for IBM DK for Java
• Code size usage (IBM DK for Java)
• Recompilation information
– Pct/total methods recompiled (Jikes RVM)– Activity over time (Both)
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation30
Other issues
Synchronous vs. asynchronous recompilation• Is optimization performed in the background?
Static or dynamic view of profile data• Is profile data packaged or used in flight?
Skipping optimization levels• When to do it?
Collecting dead compiled code• When is it safe?
Installing new compiled code• Stack rewriting, code patching, etc.
RAS issues• How repeatable?
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation31
Myth II - Dynamic class loading is a fundamental roadblock to cross-method optimization
Myth: Because you never have the whole program, interprocedural optimizations are not possible ex) virtual method resolution, virtual inlining, escape analysis, etc.
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation32
Myth II - Dynamic class loading is a fundamental roadblock to cross-method optimization
Myth: Because you never have the whole program, interprocedural optimizations are not possible ex) virtual method resolution, virtual inlining, escape analysis, etc.
Reality:• Can speculatively optimize with respect to current class
hierarchy
• Sophisticated invalidation technology well-understood; mitigates need for overly conservative assumptions
• Speculative optimization can be more aggressive than conservative, static compilation
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation33
Aggressive Speculation and Invalidation Techniques
Speculative code generation• Generate code that would be incorrect if some condition changes
• Invalidate generated code to recover if needed
Why speculate? • Hard to analyze features (reflection, native code, classloading)
• Heavier usage of OO language features, generic frameworks
• Constraints on compilation resources
How to invalidate speculative code?• On-Stack Replacement [Chambers,Hölzle&Ungar’91-94, Fink&Qian’03]
• Other approaches
– Pre-existence inlining [Detlefs & Agesen’99]– Thin Guards [Arnold’02]
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation34
Invalidation via On-Stack Replacement (OSR)
stack
PC
frame
m2
m2
stack
PC
frame
m1
m1
Transfer execution from compiled code m1 to compiled code m2even while m1 runs on some thread's stack
Extremely general mechanism minimal restrictions on speculation
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation35
Applications of OSR
Deferred compilation [SELF-91, HotSpot, Whaley 2001] • Don't compile uncommon cases
• Improve dataflow optimization and reduce compile-time Runtime optimization of long-running activations [SELF-93]
• Promote long-running loops to higher optimization level Debug optimized code via dynamic deoptimization
[HCU92]• At breakpoint, deoptimize activation to recover program state
Safe invalidation for speculative optimization• Class-hierarchy-based inlining [HotSpot]
• Type prediction [SELF-91]
• Escape Analysis [Whaley 2001]
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation36
OSR Challenges
Engineering Complexity• Need to capture/recover program state without constraining
optimizations or introducing runtime overhead to the point where the optimization benefits are lost
• Retro-fitting to existing systems (especially JITs) can be quite complex
• Support for multi-threading, weak memory model SMPs, etc. RAS implications
• Code that is both complex and infrequently executed is a prime location for bugs
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation37
Myth II - Dynamic class loading is a fundamental roadblock to cross-method optimization
Myth: Because you never have the whole program, interprocedural optimizations are not possible ex) virtual method resolution, virtual inlining, escape analysis, etc.
Reality:• Can speculatively optimize with respect to current class
hierarchy
• Sophisticated invalidation technology well-understood; mitigates need for overly conservative assumptions
• Speculative optimization can be more aggressive than conservative, static compilation
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation38
Myth III
Myth: A static compiler will give better performance than a dynamic compiler • Static compiler can use an unlimited amount of analysis time
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation39
Myth III
Myth: A static compiler will give better performance than a dynamic compiler • Static compiler can use an unlimited amount of analysis time
Reality:• Many Production JITs implement all classical static
optimizations
• Feedback-directed optimization can be more effective than unlimited IPA without profile information
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation40
Feedback-Directed Optimization (FDO)
Exploit information gathered at run-time to optimize execution• “selective optimization”: what to optimize
• “FDO” : how to optimize
Advantages of FDO [Smith 2000]• Can exploit dynamic information that cannot be inferred statically
• System can change and revert decisions when conditions change
• Runtime binding allows more flexible systems
Challenges for fully automatic online FDO• Compensate for profiling overhead
• Compensate for runtime transformation overhead
• Account for partial profile available and changing conditions
• Complexity issues
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation41
FDO Can Improve Performance in Long Running Applications: Jikes RVM (Mar ’04) (LOWER BOUND!)
0
1
2
3
4
5
6
7
8jb
b-3
00
ipsi
xql
com
pres
s
jess db
java
c
mpe
gaud
io
mtr
t
jack
Geo
mea
n
Spe
edup
ove
r Bas
elin
e
Default Default+FDO
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation42
Common FDO Techniques
Compiler optimizations• Inlining
• Code Layout
• Multiversioning
• Potpourri Run-time system optimizations
• Caching (PICs)
• Speculative meta-data representations (object headers, etc)
• GC Acceleration (adjusting heap, switching collectors, etc)
• Locality optimizations
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation43
FDO PotpourriMany opportunities to use profile info during various compiler phasesAlmost any heuristic-based decision can be informed by profile data
Examples: Loop unrolling
• Unroll “hot” loops only Register allocation
• Spill in “cold” paths first Global code motion
• Move computation from hot to cold blocks Exception handling optimizations
• Avoid expensive runtime handlers for frequent exceptional flow Speculative stack allocation
• Stack allocate objects that only escape on cold paths Software prefetching
• Profile data guides placement of prefetch instructions
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation44
Challenge: Don’t Degrade Performance Too Much When Collecting Profile Data [Arnold et al. ’02]
-4
-2
0
2
4
6
8
10
12
14
16
18
0 50 100 150 200
Total time (seconds)
Pe
rfo
rma
nc
e Im
pro
ve
me
nt
(%)
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation45
Myth III
Myth: A static compiler will give better performance than a dynamic compiler • Static compiler can use an unlimited amount of analysis time
Reality:• Many production JITs implement all classical static
optimizations
• Feedback-directed optimization can be more effective than unlimited IPA without profile information
• Legacy C compiler backends can’t exploit type information and other semantics that JITs routinely optimize
• However, ahead-of-time compilation still needed sometimes:
– Fast startup of large interactive apps– Small footprint (e.g. embedded) devices
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation46
Myths IV
Myth: Sophisticated profiling is too expensive to perform online
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation47
Myths IV
Myth: Sophisticated profiling is too expensive to perform online
Reality:• Sampling-based profiling is cheap and can collect
sophisticated information
• e.g. IBM DK dynamic instrumentation
• e.g. Arnold-Ryder full-duplication framework
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation48
Profiling
4 Categories• Runtime service monitors
– e.g. dispatch tables, synchronization services, GC• Hardware performance monitors
– see Adl-Tabatabai et al.’04• Sampling
– e.g. sample method running, call stack at context switch • Program instrumentation
– e.g. basic block counters, value profiling
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation49
IBM DK Profiler [Suganuma et al ’01,’02]
Sampling• Used to identify already compiled methods for re-optimization
Dynamic instrumentation1. Patch entry to a method with jump to instrumented version
2. Run until threshold
– Time bound– Desired quantity of data collected
3. Undo patch
sub esp, 50
mov [esp-50], ebx
mov [esp-50], ebx
mov [esp-50], ebx
B’s compiled codeB’sInstrumented
code
jmp instr_code
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation50
Arnold-Ryder [PLDI 01]: Full Duplication Profiling
Full-Duplication Framework
Duplicated CodeChecking Code
Method Entry
Checks
EntryBackedges
CheckPlacement
Generate two copies of a method•Execute “fast path” most of the time•Jump to “slow path” occasionally to collect desired profile•Demonstrated low overhead, high accuracy, tunable•Adopted by IBM J9 VM; aka “Bursty Tracing” [Hirzel&Chilimbi’01]
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation51
Myth V
Myth: Program optimization is a dead field, law of diminishing returns
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation52
Myth V
Myth: Program optimization is a dead field, law of diminishing returnsReality: No shortage of research topics
• FDO
– What optimizations should be performed?– How should a VM use FDO?– When should a VM switch from one form of FDO to another
because of a program phase change?
• Theoretically-grounded selective optimization
• Empirical optimization policies
• Deep online/staged analysis
• Higher-level programming models (e.g. J2EE, XML, Web Services, BPEL)
• Resource-constrained devices (space, power …)
• Dealing with the complexity of dynamic systems
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation53
Myth VI - The infrastructure requirements stifles innovation in this field
Myth: Small independent academic research group cannot afford infrastructure investment to innovate in this field
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation54
Myth VI - The infrastructure requirements stifles innovation in this field
Myth: Small independent academic research group cannot afford infrastructure investment to innovate in this field
Reality:• High-quality open-source virtual machines are available
– Jikes RVMhttp://www.ibm.com/developerworks/oss/jikesrvm
– ORP http://orp.sourceforge.net
– Monohttp://go-mono.com
– <Insert your favorite infrastructure here>
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation55
Comparison between HLL VMs and Dynamic Binary Optimizers
HLL VM• Applies only to programs in target languages
• Exploits program structure and high-level semantics (e.g. types)
•Large gains from run-time optimization (10X vs. interpreter)
• Most effective optimizations: inlining, register allocation
• Optimizer usually expensive, employed selectively
Dynamic Binary Optimizer• Applies to any program
•Views stream of executed instructions, can infer limited program structure and low-level semantics
• Smaller gains from run-time optimization (10% would be good?)
• Most effective optimizations: instruction scheduling, code placement
• Optimizer usually cheap, often employed ubiquitously
Trends suggest that more programs will be written to managed HLLs• Open question: for managed HLL, does binary optimizer add value?
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation56
Conclusions (but not last slide!)
Commercial JVM implementations are on the cutting edge of research in adaptive optimization
• Researchers need to pick up the slack and lead the field
Some Java-related research areas are not relevant to commercial settings
• Whole program analysis of “Java” (to improve performance)
Is HLL dynamic optimizationA. Static compilation applied at runtime
or
B. Speculative runtime optimizations with richer semantics?
• CGO’05 Keynote teaser: • We need to think more like a computer architect• “Virtual Machine Learning: Thinking like a computer architect”
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation57
Acknowledgements
Stephen Fink and Dave Grove for co-authoring the slides
Toshio Suganuma for data and slides on IBM DK for Java
Matthew Arnold for slides and general feedback
Tamiya Onodera for feedback
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation58
Additional Information
PLDI’04 Tutorial Slides• Available on my web page• Will be presented at ETAPS/CC’’05, April 3 in Edinburgh
“A Survey of Adaptive Optimization in Virtual Machines” • by Arnold, Fink, Grove, Hind and Sweeney. • Proceedings of the IEEE, Feb 2005 • Contains tons of citations• Linked from my web page
3-day Virtual Execution Environment Workshop, Sept’04• 32 experts, hosted by IBM• Slides and video for most talk and discussion are available• Link off of my web page
VEE’05, merger of USENIX JVM/VM conference and SIGPLAN IVME workshop• 2-day conference, co-located with PLDI’05• Broad scope (includes language-level and OS-level VMs)• Submission deadline: Fri, Feb 18
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation59
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation60
Code Size Comparison, startup: IBM DK for Java
SwingSet Java2D ICE Browser HotJava IchitaroArk WebSphere
0
1
2
Rela
tive C
ode S
ize t
o N
o O
pt
quick-only full-only MMI-quick MMI-full MMI-all
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation61
Code Size Comparison, steady state: IBM DK for Java
mtrt jess compress db mpeg jack javac SPECjbb
0
1
2
3
4
Rela
tive C
ode S
ize t
o N
o O
pt
quick-only full-only MMI-quick MMI-full MMI-all
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation62
Recomp. Decisions, 20 iterations for each benchmarkJikes RVM
compressjess
dbjavac
mpegaudiomtrt
jack
0
10
20
30
40
50
60
70
80
90
100
Pct
Execu
ted
Meth
od
s
2->2
B->0->1->2
B->0->2
B->1->2
B->2
B->0->1
B->1
B->0
Base
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation63
Recomp. Decisions, 20 iterations for each benchmarkJikes RVM
compressjess
dbjavac
mpegaudiomtrt
jack
0
100
200
300
400
500
600
700
800
Nu
m M
eth
od
s R
ecom
piled
2->2
B->0->1->2
B->0->2
B->1->2
B->2
B->0->1
B->1
B->0
Base
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation64
Recompilation Activity: Jikes RVM (Jul ’02)
0
5
10
15
20
25
Nu
m M
eth
od
s R
eco
mp
ile
d Opt 2
Opt 1
Opt 0
compress
jess db
javac
mpegaudio
mtrt jack
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation65
Recompilation Activity (IBM DK for Java)m
trt
jes
s
co
mp
res
s
db
mp
eg
jac
k
jav
ac
0
10
20
30
40
50
60
70
80
90
Nu
mb
er
of
Co
mp
iled
Me
tho
ds
(b
arc
ha
rt)
0
0.5
1
Ex
ec
uti
on
Tim
e R
ati
o (
line
gra
ph
)
quick-optfull-opt
special-opt
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation66
Startup: IBM DK for Java, no Specialization[Suganuma et al. ’01]
SwingSet Java2D ICE Browser HotJava IchitaroArk WebSphere Geo. Mean0
1
2
3
Re
lati
ve
Pe
rform
an
ce t
o N
o O
pt MMI-only quick-only full-only MMI-quick MMI-full
IBM Research
NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths
© 2005 IBM Corporation67
Steady State: IBM DK for Java, no Specialization [Suganuma et al ’01]
mtrt jess compress db mpegaudio jack javac SPECjbb Geo Mean0
1
2
3
4
Rela
tive
Perf
orm
ance
to N
o O
pt
MMI-only quick-only full-only MMI-quick MMI-full