ESOP

March 19, 2013

Mihai Budiu, Microsoft Research, Silicon Valley

Joel Galenson, UC Berkeley

Gordon Plotkin, University of Edinburgh

• Motivating example

–Declarative parallel programming with LINQ and DryadLINQ

• Divide-and-conquer compilation

–Compilers and Partial Compilers

• Building real compilers – LINQ, DryadLINQ, and matrix computations

2

LINQ Summary

Input

3

.Net objects of type T

LINQ Summary

Where (filter)

Input

4

.Net objects of type T

LINQ Summary

Where (filter)

Select (map)

Input

5

.Net objects of type T

LINQ Summary

Where (filter)

Select (map)

GroupBy

Input

6

.Net objects of type T

LINQ Summary

Where (filter)

Select (map)

GroupBy

OrderBy (sort)

Input

7

.Net objects of type T

LINQ Summary

Where (filter)

Select (map)

GroupBy

OrderBy (sort)

Aggregate (fold)

Input

8

.Net objects of type T

LINQ Summary

Where (filter)

Select (map)

GroupBy

OrderBy (sort)

Aggregate (fold)

Join

Input

9

.Net objects of type T

Distributed Collections

10

Distributed Collections

11

Partition

Distributed Collection

Select (Map)

f( )=

12

Select (Map)

f( )=

Select(x => f(x))

13

Select (Map)

f( )=

Select(x => f(x))

+

14

Select (Map)

f( )=

Select(x => f(x))

+

+

15

Aggregate (Fold)

f( , )=

associative

16

Aggregate (Fold)

f( , )=

Aggregate((x,y) => f(x,y))

associative

17

Aggregate (Fold)

f( , )=

Aggregate((x,y) => f(x,y))

+

associative

18

Aggregate (Fold)

f( , )=

Aggregate((x,y) => f(x,y))

+

associative

+

19

Aggregate (Fold)

f( , )=

Aggregate((x,y) => f(x,y))

+

associative

+

20

GroupBy

GroupBy(x => K(x))

21

GroupBy

GroupBy(x => K(x))

o

22

GroupBy

GroupBy(x => K(x))

o

o

PartitionBy(x => K(x))

23

GroupBy

GroupBy(x => K(x))

o

o

o

PartitionBy(x => K(x))

GroupBy(x => K(x))

Equal under permutation

24

• Motivating example

–Declarative parallel programming with LINQ and DryadLINQ

• Divide-and-conquer compilation

–Compilers and Partial Compilers

• Building real compilers – LINQ, DryadLINQ, and matrix computations

25

Compilers

Compiler

S T

26

𝐶 ∶ source → target

Partial compilers

Child

Compiler

target source

source’ target’

Partial compiler

27

Partial compilers

Child

Compiler

target source

source’ target’

Partial compiler

28

𝑃𝐶 ∶ source → source′ × target′ → target

Partial compilers

Child

Compiler

target source

source’ target’

Partial compiler R

re

du

ce G

gen

erat

e

29

𝑃𝐶 ∶ source → source′ × target′ → target

Compiler Composition target source

source’ target’

PC

C

30

Compiler Composition target source

source’ target’

PC

C

31

PC’

Partial Compiler Composition

PC

32

target source

source’ target’

source’’ target’’

Composition Laws

33

Composition Laws

Id

34

Compilers as First-Class Objects

PC PC’ ⊗

Tensor: 𝑃𝐶 ⊗ 𝑃𝐶′ 35

Multiple Children

SEQ

C1 C2

S T

36

Multiple Children

SEQ

C1 C2

Example: 𝑆 = 𝑓 ∘ 𝑔 𝑇 = 𝐶1 𝑓 ∘ 𝐶2(𝑔)

S T

37

Star

38

C

*

source* target*

Star

39

C

*

source* target*

Star

40

C

*

source* target* … …

Conditional

if gpu

GPU CPU

gpu : source -> bool

41

Case

42

case

Select GroupBy Aggregate

• Motivating example

–Declarative parallel programming with LINQ and DryadLINQ

• Divide-and-conquer compilation

–Compilers and Partial Compilers

• Building real compilers – LINQ, DryadLINQ, and matrix computations

43

A LINQ Compiler

44

Select GroupBy Aggregate

A LINQ Compiler

45

case

Select GroupBy Aggregate

A LINQ Compiler

46

case

Select GroupBy Aggregate

*

A LINQ Compiler

47

case

Select GroupBy Aggregate

*

SEQ

A Multi-Core LINQ Compiler

48

case

Select GroupBy Aggregate

MC

*

SEQ

A Multi-Core LINQ Compiler

49

case

Select GroupBy Aggregate

MC

*

SEQ

MC

A Multi-Core LINQ Compiler

50

case

Select GroupBy Aggregate

MC

*

SEQ

MC

S

T

A Multi-Core LINQ Compiler

51

case

Select GroupBy Aggregate

MC

*

SEQ

MC

S

o

o

T

T T

A Distributed, Multi-Core

LINQ Compiler

52

case

Select GroupBy Aggregate

MC

*

Cluster

SEQ

A Distributed, Multi-Core

LINQ Compiler

53

case

Select GroupBy Aggregate

MC

*

Cluster

SEQ

Cluster

A Distributed, Multi-Core

LINQ Compiler

54

case

Select GroupBy Aggregate

MC

*

Cluster

SEQ

Cluster

S

T

A Distributed, Multi-Core

LINQ Compiler

55

case

Select GroupBy Aggregate

MC

*

Cluster

SEQ

Cluster

S

o

o

T

T T

A Distributed Matrix Compiler

56

*

A Distributed Matrix Compiler

57

Tile Tile *

Cluster

A Distributed Matrix Compiler

58

Matrix

*

SEQ

Tile

Tile Tile *

Cluster

A Distributed Matrix Compiler

59

Matrix

*

SEQ

case

+ * …

Tile Tile *

M1.Times(M2).Plus(M3)

60

M1

M2

M3

*

+

Join. GroupBy.

Select( Aggregate) Concat

GroupBy. Select(

Aggregate)

+

PCTILE

PCTILE

PCTILE

CLINQ

CLINQ

*

PCMATRIX

+

PCMATRIX

PCSEQ

M1.Times(M2).Plus(M3)

61

M1

M2

M3

*

+

Join. GroupBy.

Select( Aggregate) Concat

GroupBy. Select(

Aggregate)

+

PCTILE

PCTILE

PCTILE

CLINQ

CLINQ

*

PCMATRIX

+

PCMATRIX

PCSEQ

M1.Times(M2).Plus(M3)

62

M1

M2

M3

*

+

Join. GroupBy.

Select( Aggregate) Concat

GroupBy. Select(

Aggregate)

+

PCTILE

PCTILE

PCTILE

CLINQ

CLINQ

PCMATRIX

PCMATRIX

PCSEQ

M1.Times(M2).Plus(M3)

63

Partial Compiler Correctness

64

if gpu

GPU CPU

Partial Compiler Correctness

65

𝑔𝑝𝑢 𝑆 𝐶𝐺𝑃𝑈 {¬𝑔𝑝𝑢 𝑆 }𝐶𝐶𝑃𝑈

𝑇 𝐼𝐹 𝑔𝑝𝑢 𝑇𝐻𝐸𝑁 𝐶𝐺𝑃𝑈 𝑒𝑙𝑠𝑒 𝐶𝐶𝑃𝑈 if gpu

GPU CPU

Partial Compiler Correctness

66

𝑔𝑝𝑢 𝑆 𝐶𝐺𝑃𝑈 {¬𝑔𝑝𝑢 𝑆 }𝐶𝐶𝑃𝑈

𝑇 𝐼𝐹 𝑔𝑝𝑢 𝑇𝐻𝐸𝑁 𝐶𝐺𝑃𝑈 𝑒𝑙𝑠𝑒 𝐶𝐶𝑃𝑈

Totally correct compiler from partially correct parts!

if gpu

GPU CPU

Related Work

• Dialectica category

– Inspired partial compilers and their operators.

• Milner’s tactics

– Partial compilers are a typed form of tactics.

• Multistage compilers

– Fit as a composition of unary partial compilers.

• Federated databases, cooperating analyses

– Interesting applications.

67

Thank you

68

Backup slides

69

Homomorphisms

70

ℎ ∶ 𝐴 → 𝐵 ℎ 𝑎1 +𝐴 𝑎2 = ℎ 𝑎1 +𝐵 ℎ(𝑎2)

collections

concatenation

concatenation

Nested Parallelism

71

Machine

Nested Parallelism

72

Machine Core

Correctness Definition

73

⊨ ⊆ 𝑡𝑎𝑟𝑔𝑒𝑡 × 𝑠𝑜𝑢𝑟𝑐𝑒 𝑇 ⊨ 𝑆

T implements the meaning of S

𝐶 ∶ 𝑠𝑜𝑢𝑟𝑐𝑒 → 𝑡𝑎𝑟𝑔𝑒𝑡 𝐶 𝑖𝑠 𝑐𝑜𝑟𝑟𝑒𝑐𝑡 𝑤. 𝑟𝑒𝑠𝑝. 𝑡𝑜 ⊨ iff

∀𝑆 ∈ 𝑠𝑜𝑢𝑟𝑐𝑒. 𝐶 𝑆 ⊨ 𝑆

C

T S

Partial Compiler Correctness

74

⊨ ⊆ 𝑠𝑜𝑢𝑟𝑐𝑒 × 𝑡𝑎𝑟𝑔𝑒𝑡 ⊨′ ⊆ 𝑠𝑜𝑢𝑟𝑐𝑒′ × 𝑡𝑎𝑟𝑔𝑒𝑡′

PC is correct iff ∀𝑆, 𝑇′. 𝑇′ ⊨′ 𝑆′ ⇒ 𝑇 ⊨ 𝑆

T S

T’ S’

Correctness Theorems

75

PC, C correct ⇒ correct

𝑃𝐶 ⊗ 𝑃𝐶′ correct

𝐶∗ correct

etc.

Partial Correctness

76

Correct only for some programs

Correct only for some programs

Generated sub-programs satisfy this predicate

Partial Correctness

77

Correct only for some programs

Correct only for some programs

Generated sub-programs satisfy this predicate

{Precondition} {Postcondition}

Functor

78

functor

CPU

Trans Id

Id : target -> target’ Trans : source -> source’

Iteration

79

iteration

Opt

A Distributed Matrix Compiler

80

Matrix -> sets of tiles

Matrix operations Distributed LINQ

Tile operations LINQ

Staged Compilers

81