compiler construction overview & lessons learned from teaching a class at uw jim hogg program...

Compiler Construction

Overview & Lessons Learned from Teaching a class at UW

Jim HoggProgram Manager - C++ Compiler Team - Microsoft

October 2014

L-1

H-2

What is this Talk About?

• I taught a class in Compilers, earlier this year, at UW

• Part of UW's "Professional Masters Program"

• I'll describe:

• The Course - content, organization• "Inside a Compiler" (at a gallop)• Lessons Learned

H-3

UW's "Professional Master's Program

For "fully-employed professionals"

Part-time (1 evening per week)

Average 2.5 years to reach Masters degree

~ $14k per year tuition fees

Average student: 5 years experience; GRE 83%; GPA 3.5

"The best computer science department you've never heard of" (NY Times)

In recent years, ~50% of attendees employed at Microsoft

H-4

PMP - Classes

1. Compiler Construction2. Principles of Software

Engineering3. Programming Languages4. Concurrency5. Human Computer

Interaction6. Applied Algorithms7. Parallel Computations8. Database Management

Systems9. Transaction Processing10.Data Mining11.Computer Architecture12.Computer Operating

Systems13.Current Trends in Computer

Graphics14.Network Systems

15.Design & Implementation of Digital Systems

16.Applications of Artificial Intelligence

17.Computer Vision18.Distributed Systems19.Computer Animation

Production20.Computational Biology21.Practical Aspects of Modern

Cryptograph22.Complexity Theory23.Alternative Programming

Paradigms24.Software Entrepreneurship25.Business Basics for CS

Professionals

H-5

How did I get into this?

One day, at a seminar, I was talking to the guy beside me . . .

H-6

Course Admin

10 lectures. One each week. Tuesday evenings 6:30-9:30

Broadcast from Microsoft Campus (Building 99)

Recorded at: http://courses.cs.washington.edu/courses/csep501/14sp/

Homework, Tutorials, Project, Final ExamSet of slides from previous years (Hal Perkins)

Teaching Assistant (Nat Mote)

http://courses.cs.washington.edu/courses/csep501/14sp/



H-8

My First Lecture

It went like this . . .

Jim Hogg

Part-time Job: Visiting Lecturer at UW

Day Job: Program Manager in Microsoft’s C++ Compiler Team

Backend – optimizations & codegen

Previous Software Computational Physics; Operations Research; Oil Exploration Operating Systems; Device Drivers Languages & Compilers

Previous Hardware IBM; Cray; Xerox; CDC; DEC-10; VAX; Alpha; PC

Spring 2014 Jim Hogg - UW - CSE - P501 A-9

Who Am I?


Execute this:

How? Computers only know 1’s and 0’s

55 8b . . .

int p = 0;int k = 0;while (k < length) {

if (a[k] > 0) p++;k++;

}

Compilers, from 10,000 feet

55 push ebp8b ec mov ebp, esp83 ec 58 sub esp, 88c7 45 f8 00 00 00 00 mov _p$[ebp], 0c7 45 fc 00 00 00 00 mov _k$[ebp], 0$LN3:83 7d fc 14 cmp _k$[ebp], 207d 1e jge $LN48b 45 fc mov eax, _k$[ebp]83 7c 85 a8 00 cmp _a$[ebp+eax*4], 0

. . . . .


x86 Target Code


Be a better programmer Insight into languages, compilers, and hardware What’s all that stuff in the debugger?

Compiler techniques are everywhere Little languages, verifiers, Lint, query languages, Verilog,

Mathematica Draws from many corners of CS

Finite automata, regex, grammars, graphs Links to Hardware

ISA, pipeline, multi-issue, cache, SIMD, multi-core, memory consistency

Jobs available! http://www.compilerjobs.com/

Why Study Compilers?

http://www.compilerjobs.com/

http://www.compilerjobs.com/

Jim Hogg - UW - CSE - P501 L-13

1966 Alan Perlis

1972 Edsger Dijkstra

1974 Donald Knuth

1976 Rabin & Scott

1977 John Backus

1978 Bob Floyd

1979 Ken Iverson

1980 Tony Hoare

1984 Niklaus Wirth

1987 John Cocke

1991 Robin Milner

2001 Dahl & Nygaard

2003 Alan Kay

2005 Peter Naur

2006 Fran Allen

2008 Barbara Liskov

Spring 2014

Compiler-related Turing Awards


Structure of a Compiler, approximately Front end: analyze

Read source program; understand its structure and meaning

Specific to the source language used

Back end: synthesize (well, partly) Generate equivalent target language program Mostly unaware of the source language use

Source TargetFront End Back End

Compilers, from 1,000 feet


01-Apr Tue Class 1 – Overview, Regex07-Apr Mon Homework 1 – regex08-Apr Tue Class 2 – grammar, LR(1)14-Apr Mon Homework 2 - grammars15-Apr Tue Class 3 – LL(1), ASTs, IR21-Apr Mon Project 1 - Scanner21-Apr Mon Homework 3 - grammars22-Apr Tue Class 4 – Semantics, x8628-Apr Mon Project 2 – Parser, ASTs29-Apr Tue Class 5 – Codeshape06-May Tue Class 6 – Optimizations, Dataflow13-May Tue Class 7 – Loops, SSA19-May Mon Project 3 – Semantics, Symbol Table20-May Tue Class 8 – Instruction Selection, Scheduling, Reg alloc27-May Tue Class 9 – Calling Conventions28-May Wed Exam03-Jun Tue Class 10 – Inlining, Multi-thread, GC09-Jun Mon Project 4 – CodeGen10-Jun Tue Project 5 - Report

Calendar

A-16

Compilers: Principles, Techniques and Tools Aho, Lam, Sethi, Ullman; 2e; 2011; “The Dragon Book” A Classic

Modern Compiler Implementation in Java Appel; 2e; 2013 Where the project comes from; good, but tough, text

Engineering a Compiler Cooper & Torczon; 2e; 2011 Solid; understandable; practical advice from the coal-face

Optimizing Compilers for Modern Architectures Allen & Kennedy; 2001 Good on SIMD & multi-core

Books


Advanced Compiler Design & Implementation Muchnick; 1e; 1997 Detailed optimization text; magnum opus

Compiler Construction Wirth; 1996 “A refreshing antidote to heavy theoretical tomes” Now free, from: http://www.ethoberon.ethz.ch/WirthPubl/CBEAll.pdf

Programming Language Processors in Java Watt & Brown; 2000 Pragmatic; lots of code (but no LR parsing)

More Books

http://www.ethoberon.ethz.ch/WirthPubl/CBEAll.pdf



1801 - Joseph Marie Jacquard uses punch cards to instruct a loom to weave "hello, world" into a tapestry. Redditers of the time are not impressed due to the lack of tail call recursion, concurrency, or proper capitalization

L-18

1940s - Various "computers" are "programmed" using direct wiring and switches. Engineers do this in order to avoid the tabs vs spaces debate1970 - Niklaus Wirth creates Pascal, a procedural language. Critics immediately denounce Pascal because it uses "x := x + y" syntax instead of the more familiar C-like "x = x + y". This criticism happens in spite of the fact that C has not yet been invented.1983 - Bjarne Stroustrup bolts everything he's ever heard of onto C to create C++. The resulting language is so complex that programs must be sent to the future to be compiled by the Skynet artificial intelligence. Build times suffer.

http://james-iry.blogspot.com/2009/05/brief-incomplete-and-mostly-wrong.html

Best way to learn about compilers is to build one! Course project

MiniJava compiler: classes, objects, inheritance Built using Java Build scanner using JFlex; and parser using CUP Optionally use an IDE (Eclipse, etc) Optionally use a source code repository (SVN, etc) Generate executable x86 code (via assembler) & run it Complete in steps through the quarter

See MiniJava page at: http://www.cambridge.org/resources/052182060X/


The Project

http://www.cambridge.org/resources/052182060X/



Code Flow (for Windows)

Spring 2014M-20

parser.cup

CUP

parser.java

sym.java

scanner.jflex

JFlex scanner.java

javac

mjc.java

ast

visitor

. . .

mjc.class

prog.java

prog.asm

ML

prog.obj

boot.c

CL

boot.obj

link

prog.exe

H-21

Inside a Compiler - at a gallop

8 Phases ("passes"), as follows:

Spring 2014 Jim Hogg - UW - CSE - P501 H-22


Scan

chars

tokens

AST

IR

‘Middle End’

Optimize

Select Instructions

Parse

Semantics

Allocate Registers

Emit

Machine Code

IR

IR

IR

IR

IR

Convert

AST

AST

AST = Abstract Syntax Tree

IR = Intermediate Representation

Reminder: a token is . . .

Spring 2014 Jim Hogg - UW - CSE P501 A-23

class C { public int fac(int n) { // factorial int nn; if (n < 1) nn = 1; else nn = n * this.fac(n-1); return nn; }}

class∙C∙{◊∙∙public∙int∙fac(int∙n)∙{∙∙//∙factorial◊∙∙∙∙int∙nn;◊∙∙∙∙if(n∙<∙1)◊∙∙∙∙∙∙nn∙=∙1;◊∙∙∙∙else◊∙∙∙∙nn∙=∙n∙*∙(this.fac(n-1));◊∙∙∙∙return∙nn;◊∙∙}◊}

Key for Char Stream:

◊ newline \n∙ space

CLASS ID:C LBRACE PUBLIC INT ID:fac LPAREN INT ID:n RPAREN LBRACE INT ID:nn SEMI IF LPAREN ID:n LT ILIT:1 RPAREN ID:nn EQ ILIT:1 ELSE ID:nn EQ ID:n TIMES LPAREN ID:this DOT ID:fac LPAREN ID:n MINUS ILIT:1 RPAREN RPAREN SEMI RETURN ID:nn SEMI RBRACE RBRACE

Token Spotting

Spring 2014 Jim Hogg - UW - CSE P501 B-24

if(a<=3)++grades[1]; // what are the tokens? (no spaces)

public int fac(int n) { // what are the tokens? (need spaces?)

Counter-example: fixed-format FORTRAN:

DO 50 I = 1,99 // DO loopDO 50 I = 1.2 // assignment: DO50I = 1.2



Scan

chars

tokens

AST

IR

‘Middle End’

Optimize

Select Instructions

Parse

Semantics

Allocate Registers

Emit

Machine Code

IR

IR

IR

IR

IR

Convert

AST

AST



Valid Tokens != Valid Program

Spring 2014 Jim Hogg - UW - CSE - P501 C-26

So a MiniJava Scanner would happily accept the following program:

int ; = true { while ( x < true * if { or 123 ) goto count_99

We rely on a MiniJava Parser to reject this kind of gibberish

MiniJava includes the following tokens (among many others):

class int [ ( . true < this ) + * ; while = if id ilit ! / new {

But how do we specify what makes a valid MiniJava program?

Grammar for the Hokum Language

1. Prog Stm ; Prog | Stm2. Stm AsStm | IfStm 3. AsStm Var = Exp4. IfStm if Exp then AsStm5. VorC Var | Const6. Exp VorC | VorC + VorC7. Var [a-z]8. Const [0-9]


• Context-Free Grammar ~ CFG ~ Grammar ~ Backus-Naur Form ~ BNF

• Productions, or Rules• Terminals & Non-Terminals; Start (Symbol)• Multiple languages present in the description

Context-Free Grammars (CFG)

Example Hokum Programs

Prog Stm ; Prog | StmStm AsStm | IfStm AsStm Var = ExpIfStm if Exp then AsStmVorC Var | ConstExp VorC | VorC + VorCVar [a-z]Const [0-9]


a = 1; b = a + 4z = 1if b + 3 then z = 2

a = x < 20b = a + 4 + 5 ;z = 1 ;if (a == 33) z < 2 ;

Hokum BNF Grammar

But how do we know which programs are legal or illegal, in Hokum?

Legal

Illegal

Derivation

Prog => Stm ; Prog=> AsStm ; Prog=> Var = Exp ; Prog=> a = Exp ; Prog=> a = VorC ; Prog=> a = Const ; Prog=> a = 1 ; Prog=> a = 1 ; Stm=> a = 1 ; IfStm=> a = 1 ; if Exp then AsStm=> a = 1 ; if VorC + VorC then AsStm=> a = 1 ; if Var + VorC then AsStm


=> a = 1 ; if a + VorC then AsStm=> a = 1 ; if a + Const then AsStm=> a = 1 ; if a + 1 then AsStm=> a = 1 ; if a + 1 then Var = Exp=> a = 1 ; if a + 1 then b = Exp=> a = 1 ; if a + 1 then b = VorC=> a = 1 ; if a + 1 then b = Const=> a = 1 ; if a + 1 then b = 2

=> versus Leftmost, rightmost, middlemost Sentential Form & Sentence What is a Context-Sensitive

Grammar?


Parse TreeProg

Stm

1

Spring 2014

; Prog

AsStm

= Exp

Var

a VorC

Const

Stm

IfStm

thenExp

if AsStm

+ VorCVorC

Var

a 1

Const

2

= Exp

Var

b VorC

Const


Jim Hogg - UW - CSE - P501 C-30

Junk Nodes in the Parse TreeProg

Stm

1

Spring 2014

; Prog

AsStm

= Exp

Var

a VorC

Const

Stm

IfStm

thenExp

if AsStm

+ VorCVorC

Var

a 1

Const

2

= Exp

Var

b VorC

Const


AST (Abstract Syntax Tree)

Prog

Spring 2014

=

Var:a Const:1

IfStm

+

Var:a Const:1

=

Var:b Const:2




Scan

chars

tokens

AST

IR

‘Middle End’

Optimize

Select Instructions

Parse

Semantics

Allocate Registers

Emit

Machine Code

IR

IR

IR

IR

IR

Convert

AST

AST



Spring 2014 Jim Hogg - UW - CSE - P501 I-34

Beyond Syntax - Semantic Checks

There is a level of correctness not captured by a CFG:

Has a variable been declared before it is used? Are types consistent in an expression? In the assignment x=y, is y assignable to x? Does a method call have right number and types of

parameters? In a selector p.q, is q a method or field of object p? Is variable x guaranteed to be initialized before it is used? Could p be null when p.q is executed? Etc

Spring 2014 Jim Hogg - UW - CSE - P501 I-35

We can specify Java micro-syntax with a few dozen regex Then find a tool (JFlex) to create a scanner from these regex

We can specify Java syntax with a few pages of BNF Then find a tool (CUP) to create a parser from that BNF

What about the huge collection of constraint checks? (760 pages in the Java Language Reference Manual)

Attribute Grammars? Then find a tool (???) to create a semantic checker for that Attribute

Grammar?

It gets progressively harder



Scan

chars

tokens

AST

IR

‘Middle End’

Optimize

Select Instructions

Parse

Semantics

Allocate Registers

Emit

Machine Code

IR

IR

IR

IR

IR

Convert

AST

AST



Spring 2014 Jim Hogg - UW - CSE - P501 Q-37

Example Optimization Classic example: Array references in a loop

for (k = 0; k < n; k++) a[k] = 0;

Naive codegen for a[k] = 0 in loop body mov eax, 4 // elemsize = 4 bytes imul eax, [ebp+offsetk] // k * elemsize add eax, [ebp+offseta] // &a[0] + k * elemsize mov [eax], 0 // a[k] = 0

Better! mov eax, [ebp+offseta] // &a[0], once-off

mov [eax], 0 // a[0]=0, a[1]=0, etc add eax, 4 // eax = &a[1],

&a[2], etcNote: pointers allow a user to do this directly in C or C++Eg: for (p = a; p < a + n; ) *p++ = 0;

Any Loops in this Code?

Spring 2014 Jim Hogg - UW - CSE - P501 T-38

i = 0goto L8

L7: i++L8: if (i < N) goto L9

s = 0j = 0goto L5

L4: j++L5: N--

if(j >= N) goto L3if (a[j+1] >= a[j]) goto L2t = a[j+1]a[j+1] = a[j]a[j] = ts = 1

L2: goto L4L3: if(s != ) goto L1 else goto L9L1: goto L7L9: return

Anyone recognize or guess the algorithm?

1 i = 12 j = 13 t1 = 10 * i4 t2 = t1 + j5 t3 = 8 * t26 t4 = t3 - 887 a[t4] = 08 j = j + 19 if j <= 10 goto #3

10 i = i + 111 if i <= 10 goto #212 i = 113 t5 = i - 114 t6 = 88 * t515 a[t6] = 116 i = i + 117 if i <= 10 goto #13

Jim Hogg - UW - CSE - P501 G-39

Typical "tuple stew" - IR generated by traversing an AST

Partition into Basic Blocks:• Sequence of consecutive instructions• No jumps into the middle of a BB• No jumps out of the middles of a BB• "I've started, so I'll finish"• (Ignore exceptions)

Basic Blocks

1 i = 12 j = 13 t1 = 10 * i4 t2 = t1 + j5 t3 = 8 * t26 t4 = t3 - 887 a[t4] = 08 j = j + 19 if j <= 10 goto #3

10 i = i + 111 if i <= 10 goto #212 i = 113 t5 = i - 114 t6 = 88 * t515 a[t6] = 116 i = i + 117 if i <= 10 goto #13

Spring 2014 Jim Hogg - UW - CSE - P501 G-40

Identify Leaders (first instruction in a basic block):• First instruction is a leader• Any target of a branch/jump/goto• Any instruction immediately after a branch/jump/goto

Leaders in red. Why is each leader a leader?

Basic Blocks : Leaders

Jim Hogg - UW - CSE - P501 G-41

i = 1

j = 1

t1 = 10 * it2 = t1 + jt3 = 8 * t2t4 = t3 - 88a[t4] = 0j = j + 1if j <= 10 goto B3

B1

B2

B3

i = i + 1if i <= 10 goto B2

B4

i = 1B5

t5 = i - 1t6 = 88 * t5a[t6] = 1i = i + 1if i <= 10 goto B6

B6

EXIT

ENTRY

Control Flow Graph ("CFG", again!)

• 3 loops total• 2 of the loops are nested

Most of the executions likely spent in loop bodies; that's where to focus efforts at optimization

Basic Blocks : Flowgraph

Loop in a Flowgraph: Intuition

Spring 2014 Jim Hogg - UW - CSE - P501 T-42

Header Node

• Cluster of nodes, such that:

• There's one node called the "header"• I can reach all nodes in the cluster from the header• I can get back to the header from all nodes in the cluster• Only once entrance - via the header• One or more exits



Scan

chars

tokens

AST

IR

‘Middle End’

Optimize

Select Instructions

Parse

Semantics

Allocate Registers

Emit

Machine Code

IR

IR

IR

IR

IR

Convert

AST

AST



Spring 2014 Jim Hogg - UW - CSE - P501 J-44

Call Example

; n = sum(1, 2)

push 2 ; push argspush 1call sum ; push L and jump

L:add esp, 8 ; pop argsmov [ebp+offsetn], eax ; store result into n


Example Function

int sum(int x, int y) { int a; int b; a = x; b = a + y; return b;}

Simply calculates x+y, but with a few extraneous statements to illustrate the generated code


Assembly Language Version

sum PROCpush ebp ; prologmov ebp, esp ; prologsub esp, 8 ;

prolog

mov eax, [ebp+8] ; eax = xmov [ebp-4], eax ; a = eax

mov ecx, [ebp-4] ; ecx = aadd ecx, [ebp+12] ; ecx += ymov [ebp-8], ecx ; b = ecx

mov eax, [ebp-8] ; eax = b

mov esp, ebp ; epilogpop ebp ; epilogret 0

sum ENDP

int sum(int x, int y) { int a; int b; a = x; b = a + y; return b;}


Assembly Language Version

sum PROCpush ebp ; prologmov ebp, esp ;

prologsub esp, 8 ; prolog

mov eax, [ebp+8] ; eax = xmov [ebp-4], eax ; a = eax

mov ecx, [ebp-4] ; ecx = aadd ecx, [ebp+12] ; ecx +=

ymov [ebp-8], ecx ; b = ecx

mov eax, [ebp-8] ; eax = b

mov esp, ebp ; epilog

pop ebp ; epilog

retsum ENDP

0000 0001

esp

???? ????

old ebp ebp

???? ????

esp

0000 0002y

x

0000 0001

0000 0002y

x

a

b

stack grows up the page

00A5 F988retaddr

00A5 F988

-

+


cdecl - Responsibilities Caller:

if you need the values currently in eax, ecx or edx then save them!

push arguments right-to-left execute call instruction pop arguments from stack after ret restore any of eax, ecx or edx that you saved before call

Callee: allocate space in stack frame for local variables if you will use any of ebx, edi or esi then save them! execute function body move result into eax restore any of ebx, edi or esi that you saved earlier Pop the stack frame so that return-address is top-of-stack execute ret instruction

H-49

Lessons Learned

Example 1st - Theory 2nd

Students are smarter than you think

Check for solutions to the project on the Internet!

You are bound by the syllabus

Thinking on your feet is impossible

Don't try to explain "The Visitor Pattern"

It's much more work than you thought

We reach the exciting stuff, just as the course ends

Dry-run first with your colleagues

H-50

Future?

Repeat the course, but better?

• Skip the classic lexer/parser tools?• Extend the project?

Devise a new, follow-on course on Optimizations?

• LLVM based?

compiler construction overview & lessons learned from teaching a class at uw jim hogg program...

Documents

microsoft slide

jim hogg uw

cs professionals slide

ebp eax

jim hogg parttime job

computer vision

computer graphics

computer architecture