Partial Evaluation:

Types, Binding Times and Optimal Specialisation

Lecture 3: The Types Involved in Partial evaluation

Neil D. Jones

DIKU, University of Copenhagen (prof. emeritus)

I: QUICK REVIEW OF 1. ORDER PARTIAL EVALUATION

Programs are data objects in a first order data domain D, for

example

D = Atom ∪D ×D

A programming language L is a set L-programs together with a

semantic function

[[ ]]L : L-programs→ D ⇀ D

Program meanings are partial functions:

[[p]]L : D ⇀ D

(Omit L if clear from context.) Examples for L = Lisp:

[[(quote ALPHA)]]L = ALPHA

[[(lambda (x) (+ x x))]]L 3 = 6

— 2 —

INTERPRETER, COMPILER, PARTIAL EVALUATOR

An interpreter int (for S written in L) must satisfy:

[[source]]S(d).= [[int]](source.d)

A compiler comp (from S to T, written in L)

[[source]]S(d).= [[[[comp]](source)]]T(d)

A partial evaluator (for L) is a program spec satisfying, for any

program p and data s, d:

[[p]](s.d).= [[[[spec]](p.s)]](d)

— 3 —

TECHNIQUES FOR PARTIAL EVALUATION

I Applying base functions to known data

I unfolding function calls

I creating one or more specialised program points

Example. Ackermann’s function with known n = 2:

a(m,n) = if m=0 then n+1 else

if n=0 then a(m-1,1)

else a(m-1,a(m,n-1))

Specialised program:

a2(n) = if n=0 then 3 else a1(a2(n-1))

a1(n) = if n=0 then 2 else a1(n-1)+1

Less than half as many arithmetic operations as the original:

since all tests on and computations involving m have been re-

moved.

— 4 —

THE FUTAMURA PROJECTIONS

1. A partial evaluator can compile:

targetdef= [[spec]](int.source)

2. A partial evaluator can generate a compiler:

compdef= [[spec]](spec.int)

3. A partial evaluator can generate a compiler generator:

cogendef= [[spec]](spec.spec)

— 5 —

CORRECTNESS OF THE FUTAMURA PROJECTIONS

Simple equational reasoning to verify:

1. [[target]](d).= [[source]]S(d)

2. target.= [[comp]](source)

3. comp.= [[cogen]](int)

(Surprise! It works well on the computer too... )

Practice: tricky it took a year to get right the first time, in

1984.)

— 6 —

II. UNDERBAR TYPES FOR PARTIAL EVALUATION

Isn’t there a type error somewhere?

Self-application f(f) requires f -type A = A→ A (?)

A symbolic version of an operation on values is a corresponding

operation on program texts.

The symbolic composition of programs p, q yields an output

program r.

The meaning of r =

the composition of the meanings of p and q.

Partial evaluation =

the symbolic specialisation of a function

to a known first argument value.

— 7 —

REMAINDER OF THIS TALK

I A notation for the types of symbolic operations. The nota-

tion distinguishes

• the types of values from

• the types of program texts

I Natural definitions of type correctness of a first-order inter-

preter, compiler or partial evaluator.

I State the problem of optimal partial evaluation.

I Show why it’s difficult for typed languages (even first-order).

I Reference a solution by Henning Makholm.

— 8 —

UNDERBAR TYPES FOR SYMBOLIC COMPUTATION

t: type ::= firstorder | type× type | type→type

| typeX

Type firstorder describes values in D.

For each language X and type t, a type constructor

tX

Meaning: the set of X-programs that denote values of type t.

Examples

I Atom ALPHA has type firstorder

I Lisp program (quote ALPHA) has type firstorderLisp

— 9 —

THE MEANING OF TYPE EXPRESSION T IS [[T ]]

[[firstorder]] = D

[[t1→t2]] = [[[t1]] → [[t2]]]

[[t1 × t2]] = {(t1, t2) | t1 ∈ [[t1]], t2 ∈ [[t2]]}

[[ tX

]] = { p ∈ D | [[p]]X∈ [[t]] }

Some type inference rules:

exp1 : t2 →t1, exp2 : t2exp1exp2 : t1

exp : tX

[[exp]]X : t

firstordervalue : firstorder

exp : tX

exp : firstorder

— 10 —

SYMBOLIC COMPOSITION

(α→ γ)(α→ β)× (β → γ)

(α→ γ)(α→ β)× (β → γ)

[[-]] × [[-]]

;

[[-]]

;

-

-

??'

&

$

%

'

&

$

%

'

&

$

%

'

&

$

%

(α→ β) = the set of all programs that

compute a function from α to β.

— 11 —

COMPOSITION OF FINITE TRANSDUCERS

M ; NNM

6

a/c-(q, s)(p, r)

6

r sb/c

'

&

$

%

'

&

$

%-a/b qp

�

-

'

&

$

%

cacba

���

@@@ -----

Point: no intermediate symbol b is ever produced.

— 12 —

COMPOSITION OF PROGRAMS

Consider composition oneto ; squares ; sum where

oneto(n) = [n, n− 1, . . . , 2, 1]

squares[a1, a2, . . . an] = [a21, a

22, . . . , a

2n]

sum[a1, a2, . . . an] = a1 + a2 + . . .+ an.

Straightforward program:

f(n) = sum(squares(oneto(n)))

squares(l) = if l = [] then [] else

cons(head(l)**2, squares(tail(l)))

sum(l) = if l = [] then 0 else

head(l) + sum(tail(l))

oneto(n) = if n = 0 then [] else cons(n, oneto(n-1))

Result of “deforestation”:

g(n) = if n = 0 then 0 else n**2+g(n-1)

— 13 —

PARTIAL EVALUATION

(β → γ)(α× β → γ)× α

(β → γ)(α× β → γ)× α

[[-]] × Identity

fcn-spec

[[-]]

pgm-spec

-

-

??'

&

$

%

'

&

$

%

'

&

$

%

'

&

$

%

— 14 —

A BETTER DEFINITION OF PARTIAL EVALUATION

Type in the diagram:

pgm− spec : (α× β → γ × α)→ (β → γ)

First Curry:

α→(β→γ)→α→β→γThen generalize:

spec : ∀α .∀τ . α→ τ → α→ τ

Usually α must be first order.

Definition. Program spec ∈ D is a partial evaluator if for all

p, s ∈ D,

[[p]] s.= [[[[spec]] p s]]

— 15 —

INTERPRETERS, COMPILERS, ETC. REVISITED

An interpreter int (for S written in L) must satisfy:

[[source]]S.= [[int]]source

A compiler comp (from S to T, written in L)

[[source]]S.= [[[[comp]]source]]T

A partial evaluator (for L) is a program spec satisfying, for any

program p and data s:

[[p]] s.= [[[[spec]] p s]]

— 16 —

TYPE INFERENCE FOR SELF-APPLICATION

The Futamura projections:

[[spec]] int sourcedef= target

[[spec]] spec intdef= compiler

[[spec]] spec specdef= cogen

Do these type-check?

Recall our type inference rules:

exp1 : t2 →t1, exp2 : t2exp1exp2 : t1

exp : tX

[[exp]]X : t

firstordervalue : firstorder

exp : tX

exp : firstorder

— 17 —

TYPES OF INTERPRETERS, ETC.

1. Type of source : τS

2. Type of [[int]] : ∀τ . τS→ τ

3. Type of [[compiler]] : ∀τ . τS→ τ

T

4. Type of [[spec]] : ∀α .∀β . α→ β → α→ β

where α is first order

Remark: Line 3 gives

• the type of the compiling function.

• The type of the compiler text is:

compiler : ∀τ . τS→ τ

T

— 18 —

TYPES DURING COMPILATION

We wish to find the type of

targetdef= [[spec]] int source

Assume program source has type τS

. A deduction:

[[spec]] : ρ→σ→ρ→σ

[[spec]] : τS→τ→τ

S→τ

int : τS→τ

[[spec]] int : τS→τ

source : τS

[[spec]] int source : τ

— 19 —

TYPES DURING COMPILATION

Thus target has type

τ = τL

(as expected).

The deduction uses only the type inference rules and general-

ization of polymorphic variables.

— 20 —

TYPES DURING COMPILER GENERATION: 1

Recall that: compilerdef= [[spec]] spec int where

interpreter int has type ∀τ . τS→τ .

We show: If p has type α→βthen [[spec]] spec p has type α→β

Deduction:

[[spec]] : ρ→σ→ρ→σ

[[spec]] : α→β→α→β →α→β→α→βspec : α→β→α→β

[[spec]] spec : α→β→α→βp : α→β

[[spec]] spec p : α→β

— 21 —

TYPES DURING COMPILER GENERATION: 2

Recall that:

compilerdef= [[spec]] spec int

where interpreter int has type ∀τ . τS→τ .

We just showed: If p has type α→βthen [[spec]] spec p has type α→β

Substituting α = τS, β = τ , we get

compiler = [[spec]] spec int : τS→τ

and so (as desired)

[[compiler]] : τS→τ

— 22 —

TYPES DURING COMPILER GENERATION: 3

We just showed that:

[[compiler]] : τS→τ

Furthermore τ was arbitrary, so

[[compiler]] : ∀τ . τS→τ

By similar reasoning (too big a tree to show!):

[[cogen]] : ∀α∀β . α→β→α→β

— 23 —

III: OPTIMAL PARTIAL EVALUATION

Suppose sint is a self-interpreter and p, p′ are programs such

that

p′ = [[spec]] sint p

Correctness of spec implies

[[p′]] = [[p]]

but p, p′ need not be the same programs.

— 24 —

DEFINITION OF OPTIMAL PARTIAL EVALUATION

Definition Partial evaluator spec is optimal if it removes all

interpretational overhead:

For a natural self-interpreter sint and any program p and input

d,

timep′(d) ≤ timep(d)

Intuitively: spec has removed an entire layer of interpretation.

— 25 —

PARTIAL EVALUATION EXAMPLE

Example. Ackermann’s function with known n = 2:

a(m,n) = if m=0 then n+1 else

if n=0 then a(m-1,1)

else a(m-1,a(m,n-1))

Specialised program:

a2(n) = if n=0 then 3 else a1(a2(n-1))

a1(n) = if n=0 then 2 else a1(n-1)+1

where a1(n) = a(1,n) and a2(n) = a(2,n) are specialised ver-

sions of function a.

— 26 —

TECHNIQUES FOR PARTIAL EVALUATION

I Applying base functions to known data

I unfolding function calls

I creating one or more specialised functions

Specialised Ackermann’s function performs less than half as

many arithmetic operations as the original:

All computations involving m have been removed.

— 27 —

EXAMPLE: PART OF A FIRST-ORDER INTERPRETER

A well-known trick: split the environment into two parallel lists:

ns = (n1,. . . ,nk) names

vs = (v1,. . . ,vk) values

Part of the interpreter text:

eval(exp,ns,vs,pgm) = case exp of

"X" : lookup X ns vs

"e1+e2" : eval(e1,ns,vs,pgm) + eval(e2,ns,vs,pgm)

...

— 28 —

SPECIALISATION OF THE FIRST-ORDER INTERPRETER

Binding times: exp,ns,pgm are static,

while vs is dynamic.

Consequence: all functions in p′ = [[spec]] sint p have form:

evalexp,ns,pgm(vs) = . . .

An annoying problem:

there is only only one argument in each p′ function (!)

This cannot be optimal, i.e., as fast as p !

— 29 —

INHERITED LIMITS DURING SPECIALISATION

This problem: specialised program p′ = [[spec]] sint p inherits

a limit from sint: a specialised function

fa,b(x, y) = . . .

has k′ ≤ k arguments, if sint function f has k arguments.

Thus no function in p′ has more than k arguments(!)

For interpreter function eval, this problem can be solved by

variable splitting, also called arity raising.

Observation: for a fixed p, the interpreter’s variable vs always

has a constant length k.

— 30 —

TECHNICAL SOLUTION

Split evalexp,ns,pgm(vs) = . . . into

evalexp,ns,pgm(v1, . . . , vk) = . . .

By this and similar tricks, a first-order “optimal” spec can be

built.

For the “optimal” spec, p′ = [[spec]] sint p is

identical to p, up to the naming of variables

(and thereby just as fast).

— 31 —

OPTIMALITY IS HARDER FOR TYPED LANGUAGES!

Interpreter example with types (first-order):

eval : Exp -> Names -> Values -> Univ

Univ = Int integer | Pair Univ * Univ | ...

eval exp ns vs = case exp of

"X" : env X

"e1:e2" : Pair (eval e1 ns vs) (eval e2 ns vs)

...

Suppose source program has type [[p]] : N → N .

Then specialised program has a different type:

[[p′]] : Univ→ Univ

Significantly less efficient. With higher-order types: even worse!

— 32 —

A CHALLENGING PROBLEM

To achieve optimal specialisation for a typed programming lan-

guage.

I Stated in 1987

I Unsuccessfully attempted for a number of years

I Solved by Henning Makholm in 1999. Reported in SAIG

2000 (ICFP workshop at Montreal)

— 33 —

MAKHOLM’S SOLUTION

Type of a specialiser:

[[spec]] : Pgm→Data→Pgm

This is correct, but it “doesn’t tell the whole story”

To clarify the problem, extend α→β L to

One version for L-programs: α→βPgm

and another version for data types: αData

αData is a subtype of Data:

the set of encodings of all values of type α

— 34 —

OPTIMALITY REVISITED

The type of a specialiser’s meaning, redone:

[[spec]] :α→β→γ

Pgm→

α

Data→

β→γPgm

Type of a self-interpreter’s meaning:

∀α, β . [[sint]] :α→βPgm

→α

Univ→

β

Univand thus

∀α, β . sint :

α→βPgm →

αUniv →

βUniv

Pgm

Here Univ is a universal data type.

— 35 —

OPTIMALITY AND THE UNIVERSAL DATA TYPE

The optimality criterion: p′ = [[spec]] sint p should be as good

as p.

Alas this is impossible since:

[[p]] : α→β

but

[[p′]] = [[[[spec]] sint p]] :α

Univ→

β

Univ

— 36 —

OPTIMALITY REFORMULATED

A way out: use a self-interpreter with type

[[sintα→β]] :α→βPgm

→ α → β

This can be mechanically obtained from

∀α, β . [[sint]] :α→βPgm

→α

Univ→

β

Univby

[[sintα→β]] p a = decodeβ([[sint]] p encodeα(a))

Optimality reformulated: for any [[p]] : α→β the program

p′ = [[spec]] sintα→β p

is at least as fast as p.

— 37 —

OPTIMALITY ACHIEVED

1. L = a first-order call-by-value language with

2. types unit, integer and sum and product types.

3. The self-interpreter uses a universal type Univ.

4. The self-interpreter has been proven correct (Morten Welin-

der’s phd thesis).

— 38 —

OPTIMALITY ACHIEVED

I Phase 1: specialise using unsophisticated techniques.

The output program uses a universal type Univ.

I Phase 2: Retype output program, using

• Type erasure analysis that uses

• non-standard type inference for

• types that are infinite regular trees.

I Phase 3: an Identity elimination phase, e.g., η-reductions

for product and sum types.

Punch line: It works, and even achieves:

[[spec]] sint sint =α sint

— 39 —

CONCLUSIONS

Contributions:

I A notation for the types of symbolic operations. It distin-

guishes types of values from types of program texts.

I Natural definitions of type correctness of an interpreter or

compiler.

I Makholm: succeeded in solving a long-standing open prob-

lem using the underbar type notation (after some refine-

ment)

More to do:

I Better mathematical understanding of the underbar types.

I How to prove that an interpreter or compiler has the desired

type?

— 40 —