levon stepanian, angela demke brown computer systems group

36
Inlining Java Native Calls at Runtime (CASCON 2005 – 4 th Workshop on Compiler Driven Performance) Levon Stepanian, Angela Demke Brown Computer Systems Group Department of Computer Science, University of Toronto Allan Kielstra, Gita Koblents, Kevin Stoodley IBM Toronto Software Lab

Upload: wyatt

Post on 01-Feb-2016

57 views

Category:

Documents


0 download

DESCRIPTION

Inlining Java Native Calls at Runtime (CASCON 2005 – 4 th Workshop on Compiler Driven Performance). Levon Stepanian, Angela Demke Brown Computer Systems Group Department of Computer Science, University of Toronto Allan Kielstra, Gita Koblents, Kevin Stoodley IBM Toronto Software Lab. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Levon Stepanian, Angela Demke Brown Computer Systems Group

Inlining Java Native Calls at Runtime

(CASCON 2005 – 4th Workshop on Compiler Driven Performance)

Levon Stepanian, Angela Demke BrownComputer Systems Group

Department of Computer Science, University of Toronto

Allan Kielstra, Gita Koblents, Kevin StoodleyIBM Toronto Software Lab

Page 2: Levon Stepanian, Angela Demke Brown Computer Systems Group

In a nutshell

• Runtime native function inlining into Java• Optimizing transformations on inlined JNI calls• Opaque and binary-compatible while boosting

performance

Java Code

Native CodeNative Function Call

Page 3: Levon Stepanian, Angela Demke Brown Computer Systems Group

In a nutshell

• Runtime native function inlining into Java• Optimizing transformations on inlined JNI calls• Opaque and binary-compatible while boosting

performance

Java Code

Native Codeinlined

Page 4: Levon Stepanian, Angela Demke Brown Computer Systems Group

In a nutshell

• Runtime native function inlining into Java• Optimizing transformations on inlined JNI calls• Opaque and binary-compatible while boosting

performance

Java Code

Native Codeinlined

+optimized

Page 5: Levon Stepanian, Angela Demke Brown Computer Systems Group

Motivation

• The JNI• Java’s interoperability API• Callouts and callbacks• Opaque• Binary-compatible

Java App

Native (Host) Environment

Native App/Lib

JVM + JIT

Callout

Callback

JNI

Page 6: Levon Stepanian, Angela Demke Brown Computer Systems Group

Motivation

• The JNI• Pervasive

• Legacy codes• Performance-critical, architecture-

dependent• Features unavailable in Java (files, sockets

etc.)

Page 7: Levon Stepanian, Angela Demke Brown Computer Systems Group

Motivation

• Callouts run to 2 to 3x slower than Java calls• Callback overheads are an order of magnitude larger

• JVM handshaking requirements for threads leaving and re-entering JVM context

• i.e. stack switching, reference collection, exception handling

• JIT compiler can’t predict side-effects of native function call

Page 8: Levon Stepanian, Angela Demke Brown Computer Systems Group

Our Solution

• JIT compiler based optimization that inlines native code into Java

• JIT compiler transforms inlined JNI function calls to constants, cheaper operations

• Inlined code exposed to JIT compiler optimizations

Page 9: Levon Stepanian, Angela Demke Brown Computer Systems Group

Infrastructure

• IBM TR JIT Compiler + IBM J9 VM• Native IL to JIT IL conversion mechanism

• Exploit Native IL stored in native libraries • W-Code to TR-IL at runtime

TR JITMachine

codeStatic

compiler IL

+

Page 10: Levon Stepanian, Angela Demke Brown Computer Systems Group

Outline

• Background Information • Method• Results• Future Work

Page 11: Levon Stepanian, Angela Demke Brown Computer Systems Group

Sample Java Class

class SetFieldXToFive{

public int x;

public native foo();

static{

System.loadLibrary(…);

}

}

Page 12: Levon Stepanian, Angela Demke Brown Computer Systems Group

Sample Java Class

class SetFieldXToFive{

public int x;

public native foo();

static{

System.loadLibrary(…);

}

}

Page 13: Levon Stepanian, Angela Demke Brown Computer Systems Group

Sample Native Code

JNIEXPORT void JNICALL Java_SetFieldXToFive_foo (JNIEnv * env, jobject obj){

jclass cls = (*env)->GetObjectClass(env,obj); jfieldID fid = (*env)->GetFieldID(env,cls,“x","I"); if (fid == NULL) return;

(*env)->SetIntField(env,obj,fid,5);}

GOAL: obj.x = 5

Page 14: Levon Stepanian, Angela Demke Brown Computer Systems Group

Sample Native Code

JNIEXPORT void JNICALL Java_SetFieldXToFive_foo (JNIEnv * env, jobject obj){

jclass cls = (*env)->GetObjectClass(env,obj); jfieldID fid = (*env)->GetFieldID(env,cls,“x","I"); if (fid == NULL) return;

(*env)->SetIntField(env,obj,fid,5);}

GOAL: obj.x = 5

SetFieldXToFive

Page 15: Levon Stepanian, Angela Demke Brown Computer Systems Group

Sample Native Code

JNIEXPORT void JNICALL Java_SetFieldXToFive_foo (JNIEnv * env, jobject obj){

jclass cls = (*env)->GetObjectClass(env,obj); jfieldID fid = (*env)->GetFieldID(env,cls,“x","I"); if (fid == NULL) return;

(*env)->SetIntField(env,obj,fid,5);}

GOAL: obj.x = 5

Page 16: Levon Stepanian, Angela Demke Brown Computer Systems Group

Sample Native Code

JNIEXPORT void JNICALL Java_SetFieldXToFive_foo (JNIEnv * env, jobject obj){

jclass cls = (*env)->GetObjectClass(env,obj); jfieldID fid = (*env)->GetFieldID(env,cls,“x","I"); if (fid == NULL) return;

(*env)->SetIntField(env,obj,fid,5);}

GOAL: obj.x = 5

Page 17: Levon Stepanian, Angela Demke Brown Computer Systems Group

Sample Native Code

JNIEXPORT void JNICALL Java_SetFieldXToFive_foo (JNIEnv * env, jobject obj){

jclass cls = (*env)->GetObjectClass(env,obj); jfieldID fid = (*env)->GetFieldID(env,cls,“x","I"); if (fid == NULL) return;

(*env)->SetIntField(env,obj,fid,5);}

GOAL: obj.x = 5

Page 18: Levon Stepanian, Angela Demke Brown Computer Systems Group

Native Inlining Overview

1. Inliner detects a native callsite

2. Extracts and converts Native IL to JIT IL

3. Identifies inlined JNI calls

4. Transforms inlined JNI calls

5. Finishes inlining

Page 19: Levon Stepanian, Angela Demke Brown Computer Systems Group

Method – Step 1

1. Inliner detects a native callsiteInliner

Java Code

Call to obj.foo()

foo(){…}(Native code)

TR JIT

Page 20: Levon Stepanian, Angela Demke Brown Computer Systems Group

Method – Step 2

1. Inliner detects a native callsite

2. Extracts and converts Native IL to JIT IL

Native IL

JIT IL

Page 21: Levon Stepanian, Angela Demke Brown Computer Systems Group

Method – Step 3

1. Inliner detects a native callsite

2. Extracts and converts Native IL to JIT IL

3. Identifies inlined JNI calls

JIT IL

/* call to GetObjectClass */

/* call to GetFieldID */

/* call to SetFieldID */

Pre-constructed IL

shapes

Page 22: Levon Stepanian, Angela Demke Brown Computer Systems Group

Method – Step 4

1. Inliner detects a native callsite

2. Extracts and converts Native IL to JIT IL

3. Identifies inlined JNI calls

4. Transforms inlined JNI calls

jclass cls = (*env)->GetObjectClass(env,obj);

jfieldID fid = (*env)->GetFieldID(env,cls,“x","I");if (fid == NULL) return;

(*env)->SetIntField(env,obj,fid,5);

Page 23: Levon Stepanian, Angela Demke Brown Computer Systems Group

Method – Step 4

1. Inliner detects a native callsite

2. Extracts and converts Native IL to JIT IL

3. Identifies inlined JNI calls

4. Transforms inlined JNI calls

(*env)->SetIntField(env,obj,fid,5);

Constant: SetFieldXToFive class data structure

Constant: Offset of field “x”

JIT IL: obj.x = 5

jclass cls = (*env)->GetObjectClass(env,obj);

jfieldID fid = (*env)->GetFieldID(env,cls,“x","I");if (fid == NULL) return;

Page 24: Levon Stepanian, Angela Demke Brown Computer Systems Group

The Big Picture

1. Inliner detects a native callsite

2. Extracts and converts Native IL to JIT IL

3. Identifies inlined JNI calls

4. Transforms inlined JNI calls

5. Finishes inlining

Before Native Inlining & Callback

Transformations

Inliner

Java Code

Call to obj.foo()

TR JIT

foo(){…}(Native code)

Page 25: Levon Stepanian, Angela Demke Brown Computer Systems Group

The Big Picture

After Native Inlining & Callback

Transformations

1. Inliner detects a native callsite

2. Extracts and converts Native IL to JIT IL

3. Identifies inlined JNI calls

4. Transforms inlined JNI calls

5. Finishes inlining

Inliner

Java Code

obj.x = 5

TR JIT

foo(){…}(Native code)

Page 26: Levon Stepanian, Angela Demke Brown Computer Systems Group

The Big Picture

After Native Inlining & Callback

Transformations

1. Inliner detects a native callsite

2. Extracts and converts Native IL to JIT IL

3. Identifies inlined JNI calls

4. Transforms inlined JNI calls

5. Finishes inlining

Inliner

Java Code

obj.x = 5

TR JIT

Page 27: Levon Stepanian, Angela Demke Brown Computer Systems Group

Outline

• Background Information • Method • Results• Future Work

Page 28: Levon Stepanian, Angela Demke Brown Computer Systems Group

Experimental Setup

• Native function microbenchmarks• Average of 300 million runs

• 1.4 GHz Power4 setup• Prototype implementation

Page 29: Levon Stepanian, Angela Demke Brown Computer Systems Group

Cost of IL Conversion

• 5.3 microseconds per W-Code

0

1

2

3

4

5

6

7

bzip2

crafty

gap

gcc

gzip

mcf

parser

perlbmktwolf

vortex

vpr

SPEC CINT2000 Benchmarks

Tim

e p

er

Op

co

de

(m

icro

se

cs

.)

Page 30: Levon Stepanian, Angela Demke Brown Computer Systems Group

Inlining Null Callouts

• Null native method microbenchmarks• Varying numbers of args (0, 1, 3, 5)

• Complete removal of call/return overhead• Gain back 2 to 3x slowdown

• confirmed our expectations

Page 31: Levon Stepanian, Angela Demke Brown Computer Systems Group

Inlining Non-Null Callouts

Speedup (X)

Microbenchmark Test Instance Static

hash 5.5 1.8

•smaller speedups for natives performing work

•instance vs. static speedup

Page 32: Levon Stepanian, Angela Demke Brown Computer Systems Group

Inlining & Transforming Callbacks

•Reclaim order of magnitude overhead

Speedup (X)

Microbenchmark Test Instance Static

CallVoidMethod 12.9 11.8

Page 33: Levon Stepanian, Angela Demke Brown Computer Systems Group

Data-Copy Speedups

• Transformed GetIntArrayRegion

Array Length

Speedu

p (

X)

Page 34: Levon Stepanian, Angela Demke Brown Computer Systems Group

Exposing Inlined Code To JIT Optimizations

Microbenchmark Test

Speedup (X)

GetArrayLength 93.4

FindClass

GetMethodID

NewCharArray

GetArrayLength

Page 35: Levon Stepanian, Angela Demke Brown Computer Systems Group

Conclusion

• Runtime native function inlining into Java code• Optimizing transformations on inlined Java Native

Interface (JNI) calls• JIT optimize inlined native code• Opaque and binary-compatible while boosting

performance

• Future Work• Engineering issues• Heuristics• Larger interoperability framework

Page 36: Levon Stepanian, Angela Demke Brown Computer Systems Group

Fin