Runtime Feedback in a Meta-TracingJIT for Efficient Dynamic Languages

Writer: Carl Friedrich Bolz

Introduced by Ryotaro IKEDAat 2011/09/06

OverviewThis paper describes about…

How to make it more efficient to apply JIT compiler with PyPyPyPy :

Well-known as fast Python implementation.However, in actual, it is one of framework to implement interpreter with JIT and GC!( Python implementation is just a demo! )

What is PyPy?

Framework which enables to write interpreter implementation with Restricted Python

The project mainly intends to give environments toimplement dynamic interpreter much efficient

PyPy’s JIT Automatic Implementation

Architecture

PyPy’s RPython interpreter

Any interpreter that is written in RPython

Target code that is written in any language

Give some “hints”to enable to run JIT compiler efficiently

Run

It is implementedby PyPy user!

The most bottom one performs JIT compilation and optimization to the middle one

In result, JIT compiler that is suitable for any language is automatically implemented

How to treat non-language-specific JIT compilationTypical JIT Compiler

Uses language-specific feature because each JIT compiler is dedicated to compile only one language

PyPy’s JIT Compiler

Though it is for RPython, PyPy can’t use any language-specific feature which PyPy user want to implement.

It is what we called “ Meta-Tracing “

How can we make it much faster with applying efficient method for Meta-Tracing?? = Objective

What Merit Using PyPyRather than JIT of other implementation

Widen compilation / optimization area

Typical JIT Implementation

It is too challenging for JIT compiler to target data structure operation

PyPy’s JITImplementation

It traces ,and only looks to whole RPython code,so it can target data structure operationwhich written in RPython by developer.

Hinting MechanismPyPy

RPython

Code Hint

Main conceptGiving hints to enable JIT compiler to compile

efficiently is the most important

A hint to turn arbitrary variables into constants in the trace by feeding back runtime information into compilation

A way to annotate operations which the constant folding opti-mization then recognizes and exploits.

General techniques for refactoring code to expose constant folding opportunities of likely runtime constants.

MAIN HINTS

☆

☆

☆

PyPy’s Meta-Tracing JIT Compilers

Tracing

To check and determine which control path to compile

Cond

OpOp

x = 100y = 200

x = x + y

Trace (cycles, to be compiled): Cond -> x = x + 200 -> Cond ….

Also constant-folded

Cycle: TraceOptimizations are also

performed during this trace form

for x in sequence : t = x + ….……

PyPy’s TracerTrace Area

PyPy (can / by default) traces only “hot” paths.-> Trace will be invoked frequently executed path

1000Counter It indicates how many times

the loop is executedWhen it crossesthreshold, it isregarded as “hot”

☆ As mentioned before, PyPy’s tracer doesn’t trace user program directly, but interpreter implementation written in RPython instead.

Optimization Passes

• Remove/simplify operations in the trace– Constant folding– Common subexpression elimination– Allocation removal– Store/load propagation– Loop invariant code motion

These can be applied because traces are absolutely linear form

Operate during RPython form

Running Example

Arrangements of shown examples

Simple and bare-bones object model.

• Just supports classes and instances• No inheritance• Class contains method and variable• Instance have a class, if no requested method / variable found in the instance, it searches among the class.

Example ImplementationUse dictionaryto manage class method

Use dictionaryto manage instance attributes(variables/methods)

To search requested method

To register given method

Dictionary’s “get” method costs too much.To solve the problem, it is required to make it target to JIT compilation( The way to do this is described later discussion )

Hints for Controlling Optimization

• Two hints that enables to increase the optimization opportunities for constant folding

Applied only to interpreter written in RPython, not user program.

Promotion

Trace-Elidable

Enable propagation to find “Constant-foldable” variables via trace guard

Annotation to notify which variablesare assumed as constant variable

☆ Though each of them never break code’s behavior,Using them incorrectly will definitely deteriorate its speed.

What “Guard” isDynamic Language

test = x + y;That both x and y are number, or string is OK

Static Language

test = x + y; That both x and y are either number or string, types

cannot be canged

It is necessary to assure each variables’ type are sameto compile Dynamic Language to Static Language

Native code is one of static language, it’s needed

“Guard”

How “Guard” worksGuard assures that interpreter is running compiled trace insame condition as when it is compiled at first time.

y = 10z = 100for x in sequence:

x = y + zy += 1…. = func(x)

100

Now it Becomes

hot!

Source codeguard(x == int() )guard(y = int())guard(z = 100)x = y + zy += 1…. = func(x)

Trace resultAssure conditionsto compile themto machine code

During executionof compiled

machine code…

If conditions described in guard is true,it continues to run.If conditions described in guard is false,it stops to run and switch to interpreter exec.

Traced root (will be / already compiled)Normal execution root (interpreter)

Promotion• Technique to operate constant-fold using guard

x = somefunc()y = func(x)

Source Codex = somefunc()guard( x == 200)y = func(x)

Trace ResultPROMOTE

x = somefunc()guard( x == 200)y = func(200)

Result after Promotion

x = somefunc()Trace tree

guard(x == 200)

y = func(200)

TRUE

y = func(x)FALSE

“Promote” how to

Later discussion!(Soon!)

Use “promote()” embedded methodwhich is given by PyPy RPython interpreterto give it a “hint” that indicates“promote” can be applied during this scope.

Assume the trace here usually be with acondition that self and val are expected to not so frequently varied.

Guard-fall is expected not sooccurs frequently

It may not consume overheadso much and can be expectedthat constant-folding will bringgreat improvement.

“Trace-Elidable” helps to apply “Promote”

• To tell the truth, promote cannot be invoked without @elidable annotation in the example.

Trace-Elidable: Assure specific method never change any variables.

Though tracer want to “promote” method “f” ,tracer doesn’t know whether self.c() returnsalways same value or not…

Tracer considers not to usevalue-specific guard buttype-specific guard… never “promoted”.

@elidable annotations shows that given method is immutable

This “hint” enables tracer to promote f()!

Result trace after these 2 hints applied

Before AfterCOMMON

This trace is created without any hints given.

Constant-folding is appliedvia @elidable and promote.

Technique to increate “trace-elidable”Putting It All

TogetherIncreasing the amount of Trace-elidable method increaseschance to apply constant-folding and to help Promote.

Prepare original “Map” classto manage Instance’s attributesinstead of using dictionary

To append @elidableannotations!

for index map (described in next slide)

Index map

• Efficient / Suitable data structure for PyPyMap: To manage data location (index)

“v1” : 0“string” : 1

“x” : 3

List: Stores actual data

1234 Hello,world! 3.141592 ….

Prepare getindex with this impelemntation,though it is immutable, trace-elidable can be used!

How does Instance use the “Map”?This class which is used for manage instances no longer uses dictionary!

Whole methods belong to “map”are “trace-elidable”.So the promote will work correctly!

No longer use dictionary

Versioning of Classes

Using only trace-elidable don’t satisfy requirements

In Python, though @elidable annotation is given,the method may yield not same value because any attributes can be changed.

class A: def __init__(self): x = 100 @elidable def X(self): return x

inst = A()

How do you feel if“inst.x = -1” is executed?

It is necessary to handlethis possible changes

They propose “Versioning”

Use Guard Feature to VersioningDummy class to use guard feature

When some of methods is changed,Yield new VersionTag and save it to self.version

This promote helps to create value-specifiedguard with current “version”.So, it is still trace-elidable but can handlemethods changing.

EvaluationsEnvironment: Intel Core2 Duo P8400 processor with 2.26 GHz and 3072 KB of cache on a machine with 3GB RAM running Linux 2.6.35

No hints given

Algorithmfor board game

BZ2 decoder

OS KernelSimulation

Decimal floatingPoint calculations

It uses many OOP’s features

Conclusions

• Two hints that can be used in the source code of an interpreter written with PyPy.

• They give control over runtime feedback and optimization to the language implementor.

• They are expressive enough for building well-known virtual machine optimization techniques, such as maps and inlining.

Effects to my Study

• Use PyPy as infrastructure – It can emit C source code from RPython

implementation• Applying P.T seems easy

– Parallelized Template for Rpython• This paper performs optimizations in RPython form.

How do you think that I consider to implementtemplate code in RPython?