type inference with run-time logs

Type Inferencewith Run-time Logs

Ravi Chugh

2

Motivation: Dynamic Languages• Dynamically-typed languages– Enable rapid prototyping– Facilitate inter-language development

• Statically-typed languages– Prevent certain run-time errors– Enable optimized execution– Provide checked documentation

• Many research efforts to combine both– Recent popularity of Python, Ruby, and JavaScript

has stoked the fire

3

Static Type Systems

• Many attempts at fully static type systems– Flow sensitive types, occurrence types, ...– Often require refactoring and annotation– Some features just cannot be statically typed

• Retrofitting existing language unlikely– Could work for a new language– But we want to migrate existing programs

4

Gradual Type Systems• Typing is not all-or-nothing– Some expressions have types, others are dynamic

• Typed portions checked in standard ways• Untyped portions fall back on run-time checks• Challenges– Granularity– Guarantees– Blame tracking– and ...

5

... Inference!

• Goal: migrate programs from dynamic langs

• Programmer annotation burden is currently 0

• A migration strategy must require ~0 work– Even modifying 1-10% LOC in a large codebase

can be prohibitive

6

The Challenge: Inference Goal: practical type system

polymorphism

def id(x) { return x };1 + id(2);“hello” ^ id(“world”);

polymorphism

id : ∀X. X → X

7

The Challenge: Inference Goal: practical type system

subtypingpolymorphism

def succA(x) { return (1 + x.a)};succA({a=1});succA({a=1;b=“hi”});

subtyping

{a:Int;b:Str} <: {a:Int}

8

The Challenge: Inference

bounded quantification

Goal: practical type system


def incA(x) { x.a := 1 + x.a; return x};incA({a=1}).a;incA({a=1;b=“hi”}).b;


incA : ∀X<:{a:Int}. X →

X

9





dynamic

n := if b then 0 else “bad”;m := if b then n + 1 else 0;

dynamic

n : dynamic

10





dynamicother features...

11






System ML ✓

12






System F ✗

13

The Idea

• Use run-time executions to help inference

• Program may have many valid types– But particular execution might rule out some

• Dynamic language programmers test a lot– Use existing test suites to help migration

14

Route: Inference w/ Run-time Logs

bounded quantificationsubtyping+ polymorphism

omit higher-order functions

System E

System E≤

15

First Stop


omit higher-order functions

System E

System E≤

System E−

16

• Function definitions

• Function calls

• Program is sequence of function definitions

Typed vs. Untyped Syntax

def y[A1,...,An](x:τ){ e }

def y(x){ e’ }

y[τ1,...,τn](e) y(e’)

17

E− Type System• Expression and function types

τ ::= | Int | Bool | ... | {fi:τi}

| X

σ ::= ∀Xi. τ1 → τ2

• Typing rules prevent field-not-found and primitive operation errors

• No rule for if-expressions yet

18

def id (x) { x }

def id[X] (x:X) { x } : ∀X. X → X

def id[] (x:Int) { x } : Int → Int

def id[Y,Z] (x:Y*Z) { x } : ∀Y,Z. Y*Z → Y*Z

• Infinitely many valid types for id...

19

• ... but ∀X. X → X is the principal type

∀X. X → X

∀Y,Z. Y*Z → Y*ZInt → Int

20

• ... but ∀X. X → X is the principal type

• More general than every other type• Allows id to be used in most different ways

∀Y,Z. Y*Z → Y*Z

∀X. X*X → X*X

Int*Int → Int*IntInt*Bool → Int*Bool

∀X. X → X

Int → Int

21

def readA (x) { x.a }

def readA[A] (x:{a:A}) { x.a } : ∀A. {a:A} → A

• This is the best type

∀A. {a:A} → A

∀A,B. {a:A;b:B} → A{a:Int} → Int

∀B. {a:Int;b:B} → Int

22

def foo (o) { let _ = o.a in o }

• Two valid types:

• Neither is better than the other

∀A. {a:A} → {a:A} ∀A,B. {a:A;b:B} → {a:A;b:B}

allowsfoo({a=1;b=2}).b

allowsfoo({a=1})

23

E− Static Type Inference

• E− lacks principal types– Cannot assign type just from definition– Need to consider calling contexts

• Our approach is iterative– Impose minimal constraints on argument– If a calling context requires an additional field to

be tracked, backtrack and redo the function

24

τ ::= | Int | Bool | ... | {fi:τi}

| X

Annotated Types

25

“this type variable is for parameter of y and then projected on sequence of fields l”

τ ::= | Int | Bool | ... | {fi:τi}

| Xy.l

Annotated Types

26

def id (x) { x }

τ ::= | Int | Bool | ... | {fi:τi}

| Xy.l

Annotated Types

∀X. X → X

∀Xid. Xid → Xid

27

def readA (x) { x.a }

τ ::= | Int | Bool | ... | {fi:τi}

| Xy.l

Annotated Types

∀A. {a:A} → A

∀XreadA.a. {a:XreadA.a} → XreadA.a

28

def readAB (x) { x.a.b }

τ ::= | Int | Bool | ... | {fi:τi}

| Xy.l

Annotated Types

∀AB. {a:{b:AB}} → AB

∀XreadA.a.b. {a:{b:XreadA.a.b}} → XreadA.a.b

29

τ ::= | Int | Bool | ... | {fi:τi}

| Xy.l

Annotated Types

30

“this record type came from parameter of y andthen projected on sequence of fields l”

τ ::= | Int | Bool | ... | {fi:τi} | {fi:τi}y.l

| Xy.l

Annotated Types

31


Annotated Types

∀A. {a:A} → {a:A}

∀Xfoo.a. {a:Xfoo.a} → {a:Xfoo.a}foo


| Xy.l

32


Annotated Types

∀A,B. {a:A;b:B} → {a:A;b:B}

∀Xfoo.a,Xfoo.b. {a:Xfoo.a;b:Xfoo.b} → {a:Xfoo.a;b:Xfoo.b}foo


| Xy.l

33


Iterative Inference – Example 1

Iteration 0

Processing foo...

Xfoo <: {a : Xfoo.a}

foo : ∀Xfoo.a. {a:Xfoo.a} → {a:Xfoo.a}foo

def main () { foo({a=1}) }

34

def foo (o) { let _ = o.a in o } def main () { foo({a=1}) }


Iteration 0

Processing main...

{a=1} : {a:Int}


✓{a:Int} ≤ {a:Xfoo.a} ?

35



Iteration 0

Processing foo...



def main () { let z = {a=1;b=2} in foo(z).b }

36

def foo (o) { let _ = o.a in o } def main () { let z = {a=1;b=2} in foo(z).b }


Iteration 0

Processing main...z : {a:Int;b:Int}


✓{a:Int;b:Int} ≤ {a:Xfoo.a} ?

37

✗*



Iteration 0

Processing main...

foo(z) : {a:Int}foo


{a:Int}foo ≤ {b:X} ? ✗

Caller-induced constraints

Xfoo <: {b : Xfoo.b}

z : {a:Int;b:Int}

{a:Int;b:Int} ≤ {a:Xfoo.a} ?

38



Iteration 1

Processing foo...

foo : ∀Xfoo.a. {a:Xfoo.a;b:Xfoo.b} → {a:Xfoo.a;b:Xfoo.b}foo




39



Iteration 1




Processing main...z : {a:Int;b:Int}

✓{a:Int;b:Int} ≤ {a:Xfoo.a;b:Xfoo.b} ?

40



Iteration 1




Processing main...

foo(z) : {a:Int;b:Int}foo

{a:Int;b:Int}foo ≤ {b:X} ?✓

z : {a:Int;b:Int}

{a:Int;b:Int} ≤ {a:Xfoo.a;b:Xfoo.b} ?

41

def main () { let _ = foo({a=1}) in let z = {a=1;b=2} in foo(z).b }



Iteration 0


✓

✓

42




Iteration 0


✗*

43


Iteration 1


✗

Calling contexts impose incompatibleconstraints on type of foo



44

E− Static Type Inference• In iteration 0, don’t have calling context info

• Iteratively constraint Xfoo as needed

• Process foo and its callers again

• No principal types, but still static inference

• Can run-time information improve algorithm?

45

E− Type Inference with Run-time Logs• Rig evaluation to log caller-induced constraints

1. Wrap all values with sets of type variables

2. When a value passed to function y, add Xy tag

3. When a value with tag Xy.l projected on field f, record Xy.l <: {f : Xy.l.f}

• Iteration 0 of inference looks in log forcaller-induced constraints

46


main()⇒ let z = {a=1;b=2} in foo(z).b⇒ let z = [{a=1;b=2},{}] in foo(z).b⇒ foo([{a=1;b=2},{Xfoo}]).b

⇒ (let _ = [{a=1;b=2},{Xfoo}].a in [{a=1;b=2},{Xfoo}]).b

⇒ (let _ = [1,{Xfoo.a}] in [{a=1;b=2},{Xfoo}]).b

⇒ [{a=1;b=2},{Xfoo}].b

⇒ [2,{Xfoo.b}]

Evaluation

Run-time log


Xfoo <: {a : Xfoo.a}Caller-inducedconstraint

47

System E− Summary

• Fully static inference needs to iterate

• Can wrap run-time values with sets of type variables and record field read constraints

• If all expressions executed, then– log contains all caller-induced constraints– no need for iteration

48

Next Stop


System ESystem E

System E≤

System E−

49

E Type System• Type of if-expression is join of branch types

if b then 1 else 2

if b then 1 else true

if b then {f=1; g=“”} else {f=2; h=true}

if b then {f=“”} else {f=true}

: Int

: {f:Int}

✗

✗

def bar (x,y) { if1 x.n > 0 then x else y }

50

def bar (o) { if1 o.1.n > 0 then o.1 else o.2 }

τ1*τ2 shorthand for {1:τ1;2:τ2}

shorthand for


• Three valid incomparable types:

51

∀A. {n:Int}*{} → {}

∀A. {n:Int}*{n:Int} → {n:Int}

∀B. {n:Int;b:B}*{n:Int;b:B} →

{n:Int;b:B}


| Xy.l

52

New Annotated Types

τ ::= | Int | Bool | ... | {fi:τi} | {fi:τi}y.l | {fi:τi}k.l

| Xy.l

53

“this record type came from if-expression k andthen projected on sequence of fields l”

New Annotated Types

def bar (x,y) { if1 x.n > 0 then x else y } def main () { let o = bar({n=1;b=true},{n=2;b=false}) in let _ = o.n + 1 in not o.b }

54


Iteration

Constraints from bar

Xbar.1 <: {n : Xbar.1.n}

0

55

New caller-induced constraints from mainType for bar


Iteration


Xbar.1.n = Int

0


56



Iteration


Xbar.1.n = Int Xbar.1 Xbar.2

Π

0


57



Iteration



Π

0


58


{n:Int} * {} → {}1


Iteration



ΠXbar.1 <: {n : Xbar.1.n}

59


{n:Int} * {} → {}1 Xbar.1 <: {n : Xbar.1.n}Xbar.2 <: {n : Xbar.2.n}

0


New caller-induced constraints from mainIteration


Type for bar


Π

{n:Int} * {} → {}1 0

{n:Int} * {n:Int} → {n:Int}1 1

Xbar.1 <: {n : Xbar.1.n}Xbar.2 <: {n : Xbar.2.n}

Xbar.1 <: {b : Xbar.1.b}Xbar.2 <: {b : Xbar.2.b}


60


New caller-induced constraints from mainIteration


Type for bar


Π

{n:Int} * {} → {}1 0

{n:Int} * {n:Int} → {n:Int}1 1

Xbar.1 <: {n : Xbar.1.n}Xbar.2 <: {n : Xbar.2.n}

∀B. {n:Int;b:B} * {n:Int;b:B} → {n:Int;b:B}1

2

Xbar.1 <: {b : Xbar.1.b}Xbar.2 <: {b : Xbar.2.b}


61

62

System E Summary

• Lacks principal types

• Has iterative inference

• Can run-time info help?– Simple extension to if-expression evaluation– Removes need for iteration, as before

63

Last Stop


System E≤System E≤

System E

System E−

64

Bounded Quantification

def y[ A1<:τ1 , ... , An<:τn ] (x:τ) { e }

Type parameters now have bounds

65


• Now there is a best type for foo• Variable X binds all other fields at call sites– X is {a:Int} for foo({a=1})– X is {a:Int;b:Int} for foo({a=1;b=2}).b

65

∀A. {a:A} → {a:A} ∀A,B. {a:A;b:B} → {a:A;b:B}

∀A, X<:{a:A}. X → X


66

∀Xbar.1<:{n:Int}.

Xbar.1 * Xbar.1 → Xbar.1

{n:Int}*{} → {}

{n:Int}*{n:Int} → {n:Int}

∀B. {n:Int;b:B}*{n:Int;b:B} →

{n:Int;b:B}


67

∀Xbar.1<:{n:Int}.


∀Xbar.1<:{n:Int},Xbar.2<:{}.

Xbar.1 * Xbar.2 → {}

{n:Int}*{} → {}


68

∀Xbar.1<:{n:Int}.


∀Xbar.1<:{n:Int},Xbar.2<:{}.

Xbar.1 * Xbar.2 → {}

allowsbar({n=1},{n=2}).n

allowsbar({n=1},{})

andbar({n=1,b=true},

{n=2,b=false}).b

• Neither type is better

69

Sharing Bounded Type Variables

• E≤ lacks principal types

• Can type variables for x and y be shared?

• If their separate bounds are compatible– X<:{n:Int} and Y<:{} can be combined to {n:Int}– X<:{n:Int} and Y<:{n:Bool} cannot be combined

70

E≤ Inference

• Strategy

1. If possible, use same type variableA. All call sites well-typedB. Otherwise, restart and keep vars separate

2. Otherwise, use separate variables

• Keeping separate variables is just like System E

• Now there is a second cause for restarting

71

E≤ Inference with Run-time Logs

• Know when to share from run-time info?

• Can’t determine if all call sites well-typed– even if actual has a field, its static type might not

• Can determine if some call site is not– if an actual doesn’t have field, static type will not

• So can eliminate some iteration but not all

72

Summary


System E≤

System E

System E−

73

Summary

• Iterative static inference for E≤ – first-order functions, records– polymorphism, subtyping– bounded quantification

• Run-time information improves algorithms– reduces worst-case complexity– (not counting overhead for instrumented eval)

74

Future Work• Direction #0: Prove formal properties for E≤

• Direction #1: Extend E≤

– recursion, recursive types, dynamic, classes, ...– is there still static inference?– how can run-time info help?– can existing programs be encoded in E?

• Direction #2: Inference for F– does inference become easier with run-time info?– if so, extend with more features– if not, heuristic-based approaches for migration

75

Related Work• Complete inference for ML + records [Remy 89]

– limited by ML polymorphism– and fields cannot be forgotten

• Type inference for F is undecidable [Wells 94]

• Local type inference for F [Pierce et al, Odersky et al, et al]

– require top-level annotations, try to fill in rest– even partial inference for F undecidable [Pfenning 88]

• Type checking for F≤ is undecidable [Pierce 91]

76

Related Work• Static type systems for dynamic languages– type systems for JavaScript [Thiemann 05, et al]

– Typed Scheme [Tobin-Hochstadt and Felleisen 08]

– Diamondback Ruby [Furr et al 09 11]

• Gradual type systems– functions [Thatte 90, Cartwright and Fagan 91, Siek and Taha 06]

– objects [Siek and Taha 07, Wrigstad et al 10, Bierman et al 10]

• Coercion insertion [Henglein/Rehof 95, Siek/Vachharajani 08] – do not deal with records

77

Thanks!

Released under Creative Commons Attribution 2.0 Generic LicenseOriginal photo by Alaskan Dude

http://creativecommons.org/licenses/by/2.0/deed.en

http://www.flickr.com/photos/72213316@N00/4566711365/

78

Extra Slides

79

Programs• Program is a sequence of function definitions

def y1[A1,...,An] (x1:τ1) { e1 }

def y2[B1,...,Bm] (x2:τ2) { e2 }

def main[] (z:Unit) { e3 }

• Untyped definitions

def y1 (x1) { e1’ }

80

Programs• Expressions

e ::= | n | b | e1 + e2 | not e | ...

| {fi=ei} | e.f

| y[τ1,...,τn](e)

| if e1 then e2 else e3

| let x = e1 in e2

• Untyped function calls

y(e’)

τ ::= | Int | Bool | ... | {fi:τi} | {fi:τi}y.l | {fi:τi}k.l

| Xy.l | Xk.l

81

“this record type came from if-expression k andthen projected on sequence of fields l”

New Annotated Types for E

“this type variable is for if-expression k and then projected on sequence of fields l”

type inference with run-time logs

Documents

return x

occurrence types

dynamictyped portions

standard waysuntyped

new languagebut

idworld polymorphismid

large codebasecan

nothingsome expressions