Transcript

IBM Research

Feb 18, 2005 NC State Seminar © 2005 IBM Corporation

Debunking Dynamic Optimization Myths

Michael HindBased on PLDI’04 Tutorial by Fink, Grove, and Hind

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation2

Well-known Facts

1. Because they execute at runtime, dynamic compilers must be blazingly fast

2. Dynamic class loading is a fundamental roadblock to cross-method optimization

3. Sophisticated profiling is too expensive to perform online

4. A static compiler will always produce better code than a dynamic compiler

5. Program optimization is a dead field

6. The infrastructure investment stifles innovation in this field

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation3

Well-known Facts ???

1. Because they execute at runtime, dynamic compilers must be blazingly fast

2. Dynamic class loading is a fundamental roadblock to cross-method optimization

3. Sophisticated profiling is too expensive to perform online

4. A static compiler will always produce better code than a dynamic compiler

5. Program optimization is a dead field

6. The infrastructure requirements stifles innovation in this field

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation4

Terminology

Virtual Machine (for this talk): a software execution engine for a program written in a machine-independent language• Ex, Java bytecodes, CLI, Pascal p-code, Smalltalk v-code

Program Loader

ThreadScheduler

SecurityServices

LibrariesMemoryManagement

RuntimeSupport

Mechanisms

Dynamic type checkingIntrospection, etc.

Tracing,Profiling, etc(ex. JVMPI)

Interpreter Compiler(s) Adaptive Optimization System

VM != JIT

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation5

Adaptive Optimization Hall of Fame

1958-1962

1974

1980-1984

1986-1994

1995-present

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation6

Adaptive Optimization Hall of Fame

1958-1962: LISP

1974: Adaptive Fortran

1980-1984: ParcPlace Smalltalk

1986-1994: Self

1995-present: Java

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation7

Quick History of VMs

LISP Interpreters [McCarthy’78]• First widely used VM

• Pioneered VM services

– memory management, Eval -> dynamic loading

Adaptive Fortran [Hansen’74]• First in-depth exploration of adaptive optimization

• Selective optimization, models, multiple optimization levels, online profiling and control systems

ParcPlace Smalltalk [Deutsch&Schiffman’84]• First modern VM

• Introduced full-fledge JIT compiler, inline caches, native code caches

• Demonstrated software-only VMs were viable

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation8

Quick History of VMs

Self [Chambers&Ungar’91, Hölzle&Ungar’94]• Developed many advanced VM techniques

• Introduced polymorphic inline caches, on-stack replacement, dynamic de-optimization, advanced selective optimization, type prediction and splitting, profile-directed inlining integrated with adaptive recompilation

• Pure OO language that supports an interactive development environment

• Much of the technology was transferred to Sun’s HotSpot JVM

Java/JVM [Gosling et al ‘95]• First VM with mainstream market penetration

• Java vendors embraced and improved Smalltalk and Self technology

• Encouraged VM adoption by others -> CLR

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation9

Other VMs in this Talk

IBM DK for Java [’95- ]• Port of Sun Classic JVM with JIT, GC, and synch enhancements

• World class performance

Jikes RVM (Jalapeno) [’97- ]• VM for Java, written in (mostly) Java

• Independently developed VM + GNU Classpath libs

• Open source, popular with researchers, not a full JVM

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation10

Myth 1 - Dynamic compilers must be blazingly fast

Myth: Because they execute at runtime, dynamic compilers must be blazingly fast. • they cannot perform sophisticated optimizations, such as SSA,

graph-coloring register allocation, inlining, etc.

Reality:• Production JITs perform all the classical optimizations

• Selective optimization strategies successfully focus compilation effort where needed

What is Selective Optimization?

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation11

How are Programs Executed?

1. Interpretation• Popular approach for high-level languages

– Ex, APL, SNOBOL, BCPL, Perl, Python, MATLAB

• Useful for memory-challenged environments

• Low startup overhead, but much slower than native code execution

2. Classic just-in-time compilation• Compile each method to native code on first invocation

– Ex, ParcPlace Smalltalk-80, Self-91– Initial high (time & space) overhead for each compilation– Precludes use of sophisticated optimizations (eg. SSA, etc.)

• Responsible for many of today’s myths

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation12

Selective Optimization

Hypothesis: most execution is spent in a small pct. of methods

Idea: use two execution strategies1. Interpreter or non-optimizing compiler

2. Full-fledged optimizing compiler

• Strategy: • Use option 1 for initial execution of all methods

• Profile application to find “hot” subset of methods

• Use option 2 on this subset

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation13

Selective Optimization Examples

Adaptive Fortran: interpreter + 2 compilers

Self’93: non-optimizing + optimizing compilers

JVMs• Interpreter + compilers: Sun’s HotSpot, IBM DK for Java, IBM’s J9

• Multiple compilers: Jikes RVM, Intel’s Judo/ORP, BEA’s JRocket

CLR?• Currently, only 1 runtime compiler, i.e., a classic JIT

– But, also use ahead-of-time (AOT) compilation

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation14

Profiling: How to Find Candidates for Optimization

Counters

Call Stack Sampling

Combinations

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation15

How to Find Candidates for Optimization: Counters

Insert method-specific counter on method entry and loop back edge

Counts how often a method is called • approximates how much time is spent in a method

Very popular approach: Self, HotSpot Issues: overhead for incrementing counter can be

significant• Not present in optimized codefoo ( … ) {

fooCounter++;

if (fooCounter > Threshold) {

recompile( … );

}

. . .

}

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation16

How to Find Candidates for Optimization: Call Stack Sampling

Periodically record which method(s) are on the call stack Approximates amount of time spent in each method Does not necessarily need to be compiled into the code

• Ex) Jikes RVM, JRocket Issues: timer-based sampling is not deterministic

ABC

AB

A AB

ABC

ABC

......

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation17

How to Find Candidates for Optimization: Call Stack Sampling

Periodically record method(s) on call stack• Approximates amount of time spent in each method

Does not necessarily need to be compiled into the code• Ex) Jikes RVM, JRocket

Issues: timer-based sampling is not deterministic

ABC

AB

A AB

ABC

ABC

......

Sample

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation18

How to Find Candidates for Optimization: Combinations

Use counters initially and sampling later on• Ex) IBM DK for Java

foo ( … ) {

fooCounter++;

if (fooCounter > Threshold) {

recompile( … );

}

. . .

}

ABC

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation19

Recompilation Policies: Which Candidates to Optimize?

Problem: given optimization candidates, which ones should be optimized?

Counters: 1. Optimize method that surpasses threshold

– Simple, but hard to tune, doesn’t consider context

2. Optimize method on the call stack based on inlining policies (Self, HotSpot)

– Addresses context issue Call Stack Sampling:

1. Optimize all methods that are sampled

− Simple, but doesn’t consider frequency of sampled methods

2. Use Cost/benefit model (Jikes RVM)

– Seemingly complicated, but easy to engineer– Maintenance free– Naturally supports multiple optimization levels

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation20

Jikes RVM Architecture [Arnold et al. ’00, ‘04]

AOS Database

Compilation Queue

Event Queue

Controller

OPT Compiler

CompilationThread

ExecutingCode

Hot MethodOrganizer

MethodSamples

Take Sample

Runtime Measurements

AOS

Recompilation Subsystem

Samples occur at taken yield points (approx 100/sec)Organizer thread communicates sampled methods to controller

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation21

Jikes RVM: Recompilation Policy – Cost/Benefit Model

Define• cur, current opt level for method m

• Exe(j), expected future execution time at level j

• Comp(j), compilation cost at opt level j Choose j > cur that minimizes Exe(j) + Comp(j)

If Exe(j) + Comp(j) < Exe(cur) recompile at level j

Assumptions• Sample data determines how long a method has executed

• Method will execute as much in the future as it has in the past

• Compilation cost and speedup are offline averages

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation22

IBM DK for Java [Suganuma et al. ’01]

Execution Levels (excluding Specialization)

MMI (Mixed Mode Interpreter)•Fast interpreter implemented in assembler

Quick compilation•Reduced set of optimizations for fast compilation, little inlining

Full compilation•Full optimizations only for selected hot methods

Methods can progress sequentially through the levels

1st LevelCompiled Code

2nd LevelCompiled Code

Mixed Mode Interpreter

Hot Method Sampling

Dynamic Compiler

Profiling System

Quick OptCompiler

Sampling Profiler

Full OptCompiler

MMI Profiler

ByteCode

Invocation FrequencyLoop Iteration

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation23

Startup Programs: Jikes RVM

0

1

2

3

4

5

db/10

jack/10

ipsi

xql/

short

jess

/10

jbb

/12

000

mtr

t/10

java

c10

xerc

es/

short

mpeg/10

com

pre

ss/10

daik

on/sh

ort

soot/

short

jack/10

0

xerc

es/

long

java

c/10

0

jess

/10

0

mrt

r/10

0

db/10

0

ipsi

xql/

long

soot/

long

jbb

/200

000

com

pre

s/10

0

mpeg/10

0

daik

on/lo

ng

Geom

Speedup o

ver

Base

line

JIT 0 JIT 1 JIT 2

No FDO, Mar’04, AIX/PPC

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation24

Startup Programs: Jikes RVM

0

1

2

3

4

5

db/1

0

jack

/10

ipsi

xql

/sho

rt

jess

/10

jbb/1

2000

mtr

t/10

java

c10

xer

ces/

shor

t

mpe

g/10

com

pres

s/10

daik

on/s

hort

soot

/sho

rt

jack

/10

0

xer

ces/

long

java

c/10

0

jess

/10

0

mrt

r/10

0

db/1

00

ipsi

xql

/lon

g

soot

/lon

g

jbb/2

0000

0

com

pres

/100

mpe

g/10

0

daik

on/l

ong

Geo

m

Spe

edup

ove

r Bas

elin

e

JIT 0 JIT 1 JIT 2 Model

No FDO, Mar’04, AIX/PPC

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation25

Steady State: Jikes RVM

0

1

2

3

4

5

6

7

jbb

-30

0

ipsi

xql

com

pres

s

jess db

java

c

mpe

gaud

io

mtr

t

jack

Geo

mea

n

Spe

edup

ove

r Bas

elin

e

J I T 0 J I T 1 J I T 2

No FDO, Mar’04, AIX/PPC

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation26

Steady State: Jikes RVM, no FDO (Mar ’04)

0

1

2

3

4

5

6

7

jbb

-30

0

ipsi

xql

com

pres

s

jess db

java

c

mpe

gaud

io

mtr

t

jack

Geo

mea

n

Spe

edup

ove

r Bas

elin

e

J I T 0 J I T 1 J I T 2 Model

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation27

Execution Profile: Jikes RVM (Jul ’02)

86.6%

6.2%

0.1%

0.0%

0.0%

0.6%

6.5%

Application Threads

Garbage Collection

Controller

Method organizer

Decay organizer

Inlining organizer

Opt. Recompilation

Size 100, SPECjvm98, 1 run each

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation28

Myth 1 - Dynamic compilers must be blazingly fast

Myth: Because they execute at runtime, dynamic compilers must be blazingly fast. • they cannot perform sophisticated optimizations, such as SSA,

graph-coloring register allocation, inlining, etc.

Reality:• Production JITs perform all the classical optimizations

• Selective optimization strategies successfully focus compilation effort where needed

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation29

Understanding System Behavior

PLDI’04 tutorial contains• Performance results for IBM DK for Java

• Code size usage (IBM DK for Java)

• Recompilation information

– Pct/total methods recompiled (Jikes RVM)– Activity over time (Both)

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation30

Other issues

Synchronous vs. asynchronous recompilation• Is optimization performed in the background?

Static or dynamic view of profile data• Is profile data packaged or used in flight?

Skipping optimization levels• When to do it?

Collecting dead compiled code• When is it safe?

Installing new compiled code• Stack rewriting, code patching, etc.

RAS issues• How repeatable?

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation31

Myth II - Dynamic class loading is a fundamental roadblock to cross-method optimization

Myth: Because you never have the whole program, interprocedural optimizations are not possible ex) virtual method resolution, virtual inlining, escape analysis, etc.

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation32

Myth II - Dynamic class loading is a fundamental roadblock to cross-method optimization

Myth: Because you never have the whole program, interprocedural optimizations are not possible ex) virtual method resolution, virtual inlining, escape analysis, etc.

Reality:• Can speculatively optimize with respect to current class

hierarchy

• Sophisticated invalidation technology well-understood; mitigates need for overly conservative assumptions

• Speculative optimization can be more aggressive than conservative, static compilation

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation33

Aggressive Speculation and Invalidation Techniques

Speculative code generation• Generate code that would be incorrect if some condition changes

• Invalidate generated code to recover if needed

Why speculate? • Hard to analyze features (reflection, native code, classloading)

• Heavier usage of OO language features, generic frameworks

• Constraints on compilation resources

How to invalidate speculative code?• On-Stack Replacement [Chambers,Hölzle&Ungar’91-94, Fink&Qian’03]

• Other approaches

– Pre-existence inlining [Detlefs & Agesen’99]– Thin Guards [Arnold’02]

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation34

Invalidation via On-Stack Replacement (OSR)

stack

PC

frame

m2

m2

stack

PC

frame

m1

m1

Transfer execution from compiled code m1 to compiled code m2even while m1 runs on some thread's stack

Extremely general mechanism minimal restrictions on speculation

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation35

Applications of OSR

Deferred compilation [SELF-91, HotSpot, Whaley 2001] • Don't compile uncommon cases

• Improve dataflow optimization and reduce compile-time Runtime optimization of long-running activations [SELF-93]

• Promote long-running loops to higher optimization level Debug optimized code via dynamic deoptimization

[HCU92]• At breakpoint, deoptimize activation to recover program state

Safe invalidation for speculative optimization• Class-hierarchy-based inlining [HotSpot]

• Type prediction [SELF-91]

• Escape Analysis [Whaley 2001]

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation36

OSR Challenges

Engineering Complexity• Need to capture/recover program state without constraining

optimizations or introducing runtime overhead to the point where the optimization benefits are lost

• Retro-fitting to existing systems (especially JITs) can be quite complex

• Support for multi-threading, weak memory model SMPs, etc. RAS implications

• Code that is both complex and infrequently executed is a prime location for bugs

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation37

Myth II - Dynamic class loading is a fundamental roadblock to cross-method optimization

Myth: Because you never have the whole program, interprocedural optimizations are not possible ex) virtual method resolution, virtual inlining, escape analysis, etc.

Reality:• Can speculatively optimize with respect to current class

hierarchy

• Sophisticated invalidation technology well-understood; mitigates need for overly conservative assumptions

• Speculative optimization can be more aggressive than conservative, static compilation

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation38

Myth III

Myth: A static compiler will give better performance than a dynamic compiler • Static compiler can use an unlimited amount of analysis time

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation39

Myth III

Myth: A static compiler will give better performance than a dynamic compiler • Static compiler can use an unlimited amount of analysis time

Reality:• Many Production JITs implement all classical static

optimizations

• Feedback-directed optimization can be more effective than unlimited IPA without profile information

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation40

Feedback-Directed Optimization (FDO)

Exploit information gathered at run-time to optimize execution• “selective optimization”: what to optimize

• “FDO” : how to optimize

Advantages of FDO [Smith 2000]• Can exploit dynamic information that cannot be inferred statically

• System can change and revert decisions when conditions change

• Runtime binding allows more flexible systems

Challenges for fully automatic online FDO• Compensate for profiling overhead

• Compensate for runtime transformation overhead

• Account for partial profile available and changing conditions

• Complexity issues

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation41

FDO Can Improve Performance in Long Running Applications: Jikes RVM (Mar ’04) (LOWER BOUND!)

0

1

2

3

4

5

6

7

8jb

b-3

00

ipsi

xql

com

pres

s

jess db

java

c

mpe

gaud

io

mtr

t

jack

Geo

mea

n

Spe

edup

ove

r Bas

elin

e

Default Default+FDO

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation42

Common FDO Techniques

Compiler optimizations• Inlining

• Code Layout

• Multiversioning

• Potpourri Run-time system optimizations

• Caching (PICs)

• Speculative meta-data representations (object headers, etc)

• GC Acceleration (adjusting heap, switching collectors, etc)

• Locality optimizations

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation43

FDO PotpourriMany opportunities to use profile info during various compiler phasesAlmost any heuristic-based decision can be informed by profile data

Examples: Loop unrolling

• Unroll “hot” loops only Register allocation

• Spill in “cold” paths first Global code motion

• Move computation from hot to cold blocks Exception handling optimizations

• Avoid expensive runtime handlers for frequent exceptional flow Speculative stack allocation

• Stack allocate objects that only escape on cold paths Software prefetching

• Profile data guides placement of prefetch instructions

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation44

Challenge: Don’t Degrade Performance Too Much When Collecting Profile Data [Arnold et al. ’02]

-4

-2

0

2

4

6

8

10

12

14

16

18

0 50 100 150 200

Total time (seconds)

Pe

rfo

rma

nc

e Im

pro

ve

me

nt

(%)

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation45

Myth III

Myth: A static compiler will give better performance than a dynamic compiler • Static compiler can use an unlimited amount of analysis time

Reality:• Many production JITs implement all classical static

optimizations

• Feedback-directed optimization can be more effective than unlimited IPA without profile information

• Legacy C compiler backends can’t exploit type information and other semantics that JITs routinely optimize

• However, ahead-of-time compilation still needed sometimes:

– Fast startup of large interactive apps– Small footprint (e.g. embedded) devices

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation46

Myths IV

Myth: Sophisticated profiling is too expensive to perform online

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation47

Myths IV

Myth: Sophisticated profiling is too expensive to perform online

Reality:• Sampling-based profiling is cheap and can collect

sophisticated information

• e.g. IBM DK dynamic instrumentation

• e.g. Arnold-Ryder full-duplication framework

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation48

Profiling

4 Categories• Runtime service monitors

– e.g. dispatch tables, synchronization services, GC• Hardware performance monitors

– see Adl-Tabatabai et al.’04• Sampling

– e.g. sample method running, call stack at context switch • Program instrumentation

– e.g. basic block counters, value profiling

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation49

IBM DK Profiler [Suganuma et al ’01,’02]

Sampling• Used to identify already compiled methods for re-optimization

Dynamic instrumentation1. Patch entry to a method with jump to instrumented version

2. Run until threshold

– Time bound– Desired quantity of data collected

3. Undo patch

sub esp, 50

mov [esp-50], ebx

mov [esp-50], ebx

mov [esp-50], ebx

B’s compiled codeB’sInstrumented

code

jmp instr_code

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation50

Arnold-Ryder [PLDI 01]: Full Duplication Profiling

Full-Duplication Framework

Duplicated CodeChecking Code

Method Entry

Checks

EntryBackedges

CheckPlacement

Generate two copies of a method•Execute “fast path” most of the time•Jump to “slow path” occasionally to collect desired profile•Demonstrated low overhead, high accuracy, tunable•Adopted by IBM J9 VM; aka “Bursty Tracing” [Hirzel&Chilimbi’01]

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation51

Myth V

Myth: Program optimization is a dead field, law of diminishing returns

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation52

Myth V

Myth: Program optimization is a dead field, law of diminishing returnsReality: No shortage of research topics

• FDO

– What optimizations should be performed?– How should a VM use FDO?– When should a VM switch from one form of FDO to another

because of a program phase change?

• Theoretically-grounded selective optimization

• Empirical optimization policies

• Deep online/staged analysis

• Higher-level programming models (e.g. J2EE, XML, Web Services, BPEL)

• Resource-constrained devices (space, power …)

• Dealing with the complexity of dynamic systems

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation53

Myth VI - The infrastructure requirements stifles innovation in this field

Myth: Small independent academic research group cannot afford infrastructure investment to innovate in this field

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation54

Myth VI - The infrastructure requirements stifles innovation in this field

Myth: Small independent academic research group cannot afford infrastructure investment to innovate in this field

Reality:• High-quality open-source virtual machines are available

– Jikes RVMhttp://www.ibm.com/developerworks/oss/jikesrvm

– ORP http://orp.sourceforge.net

– Monohttp://go-mono.com

– <Insert your favorite infrastructure here>

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation55

Comparison between HLL VMs and Dynamic Binary Optimizers

HLL VM• Applies only to programs in target languages

• Exploits program structure and high-level semantics (e.g. types)

•Large gains from run-time optimization (10X vs. interpreter)

• Most effective optimizations: inlining, register allocation

• Optimizer usually expensive, employed selectively

Dynamic Binary Optimizer• Applies to any program

•Views stream of executed instructions, can infer limited program structure and low-level semantics

• Smaller gains from run-time optimization (10% would be good?)

• Most effective optimizations: instruction scheduling, code placement

• Optimizer usually cheap, often employed ubiquitously

Trends suggest that more programs will be written to managed HLLs• Open question: for managed HLL, does binary optimizer add value?

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation56

Conclusions (but not last slide!)

Commercial JVM implementations are on the cutting edge of research in adaptive optimization

• Researchers need to pick up the slack and lead the field

Some Java-related research areas are not relevant to commercial settings

• Whole program analysis of “Java” (to improve performance)

Is HLL dynamic optimizationA. Static compilation applied at runtime

or

B. Speculative runtime optimizations with richer semantics?

• CGO’05 Keynote teaser: • We need to think more like a computer architect• “Virtual Machine Learning: Thinking like a computer architect”

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation57

Acknowledgements

Stephen Fink and Dave Grove for co-authoring the slides

Toshio Suganuma for data and slides on IBM DK for Java

Matthew Arnold for slides and general feedback

Tamiya Onodera for feedback

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation58

Additional Information

PLDI’04 Tutorial Slides• Available on my web page• Will be presented at ETAPS/CC’’05, April 3 in Edinburgh

“A Survey of Adaptive Optimization in Virtual Machines” • by Arnold, Fink, Grove, Hind and Sweeney. • Proceedings of the IEEE, Feb 2005 • Contains tons of citations• Linked from my web page

3-day Virtual Execution Environment Workshop, Sept’04• 32 experts, hosted by IBM• Slides and video for most talk and discussion are available• Link off of my web page

VEE’05, merger of USENIX JVM/VM conference and SIGPLAN IVME workshop• 2-day conference, co-located with PLDI’05• Broad scope (includes language-level and OS-level VMs)• Submission deadline: Fri, Feb 18

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation59

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation60

Code Size Comparison, startup: IBM DK for Java

SwingSet Java2D ICE Browser HotJava IchitaroArk WebSphere

0

1

2

Rela

tive C

ode S

ize t

o N

o O

pt

quick-only full-only MMI-quick MMI-full MMI-all

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation61

Code Size Comparison, steady state: IBM DK for Java

mtrt jess compress db mpeg jack javac SPECjbb

0

1

2

3

4

Rela

tive C

ode S

ize t

o N

o O

pt

quick-only full-only MMI-quick MMI-full MMI-all

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation62

Recomp. Decisions, 20 iterations for each benchmarkJikes RVM

compressjess

dbjavac

mpegaudiomtrt

jack

0

10

20

30

40

50

60

70

80

90

100

Pct

Execu

ted

Meth

od

s

2->2

B->0->1->2

B->0->2

B->1->2

B->2

B->0->1

B->1

B->0

Base

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation63

Recomp. Decisions, 20 iterations for each benchmarkJikes RVM

compressjess

dbjavac

mpegaudiomtrt

jack

0

100

200

300

400

500

600

700

800

Nu

m M

eth

od

s R

ecom

piled

2->2

B->0->1->2

B->0->2

B->1->2

B->2

B->0->1

B->1

B->0

Base

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation64

Recompilation Activity: Jikes RVM (Jul ’02)

0

5

10

15

20

25

Nu

m M

eth

od

s R

eco

mp

ile

d Opt 2

Opt 1

Opt 0

compress

jess db

javac

mpegaudio

mtrt jack

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation65

Recompilation Activity (IBM DK for Java)m

trt

jes

s

co

mp

res

s

db

mp

eg

jac

k

jav

ac

0

10

20

30

40

50

60

70

80

90

Nu

mb

er

of

Co

mp

iled

Me

tho

ds

(b

arc

ha

rt)

0

0.5

1

Ex

ec

uti

on

Tim

e R

ati

o (

line

gra

ph

)

quick-optfull-opt

special-opt

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation66

Startup: IBM DK for Java, no Specialization[Suganuma et al. ’01]

SwingSet Java2D ICE Browser HotJava IchitaroArk WebSphere Geo. Mean0

1

2

3

Re

lati

ve

Pe

rform

an

ce t

o N

o O

pt MMI-only quick-only full-only MMI-quick MMI-full

IBM Research

NC State Seminar February 18, 2005 Debunking Dynamic Optimization Myths

© 2005 IBM Corporation67

Steady State: IBM DK for Java, no Specialization [Suganuma et al ’01]

mtrt jess compress db mpegaudio jack javac SPECjbb Geo Mean0

1

2

3

4

Rela

tive

Perf

orm

ance

to N

o O

pt

MMI-only quick-only full-only MMI-quick MMI-full


Top Related