![Page 1: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/1.jpg)
Compiler Construction
Abstract Syntax Tree
Rina Zviel-Girshin and Ohad ShachamSchool of Computer Science
Tel-Aviv University
![Page 2: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/2.jpg)
22
Compiler
ICProgram
ic
x86 executable
exeLexicalAnalysi
s
Syntax Analysi
s
Parsing
AST Symbol
Tableetc.
Inter.Rep.(IR)
CodeGeneration
IC compiler
![Page 3: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/3.jpg)
33
From text to abstract syntax5 + (7 * x)
num+(num*id)
Lexical Analyzer
program text
token stream
Parser
Grammar:E id E numE E + EE E * EE ( E ) num(5)
E
E E+
E * E
( E )
num(7) id(x)
+
Num(5)
Num(7) id(x)
*Abstract syntax tree
parse tree
validsyntaxerror
![Page 4: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/4.jpg)
44
PA2
FieldsOrMethods ::= | Field FieldsOrMethods |
Method FieldsOrMethods
ICClass ::= CLASS CLASS_ID EXTENDS CLASS_ID LCBR FieldsOrMethods RCBR | CLASS CLASS_ID LCBR FieldsOrMethods RCBR
![Page 5: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/5.jpg)
55
PA2
FieldsOrMethods ::= |Field FieldsOrMethods |
Method FieldsOrMethods
ICClass ::= CLASS CLASS_ID EXTENDS CLASS_ID LCBR FieldsOrMethods RCBR | CLASS CLASS_ID LCBR FieldsOrMethods RCBR
prog
class_list
ICClass
FieldsOrMethods
Field FieldsOrMethods
type ID(state)
BOOLEAN
Method
FieldsOrMethods
…
…
![Page 6: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/6.jpg)
66
AST
FieldsOrMethods ::= {: RESULT = new FieldsMethods(); :} | Field:field FieldsOrMethods:next | {: RESULT = next; RESULT. addField(field); :} |
Method:method FieldsOrMethods:next {: RESULT = next; RESULT.addMethod(method); :}
![Page 7: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/7.jpg)
77
Parsing and AST
prog
class_list
ICClass
FieldsOrMethods
Field FieldsOrMethods
type ID(state)
BOOLEAN
Method
FieldsOrMethods
…
…
Syntax tree builtduring parsing
ProgAST
ClassAST
classList
FieldAST[0]type:BoolTypename:state
MethodAST[0]
MethodAST[1]
MethodAST[2]
…
…
methodListfieldList
![Page 8: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/8.jpg)
88
Designing an ASTpublic abstract class ASTNode {
// common AST nodes functionality}
public class Expr extends ASTNode {private int value;private Expr left;private Expr right;private String operator;
public Expr(Integer val) {value = val.intValue();
}public Expr(Expr operand, String op) {
this.left = operand;this.operator = op;
}public Expr(Expr left, Expr right, String op) {
this.left = left;this.right = right;this.operator = op;
}}
![Page 9: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/9.jpg)
99
Partial AST hierarchy for ICASTNode
Program ICClass Statement
LocalVariable
Statement
StatementBlock
If While ...
Expression
Literal New ...Call
Method
VirtualMethod
StaticMethod
...Field
LibraryMethod
declaration
![Page 10: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/10.jpg)
1010
AST node contents
class ICClass extends ASTNode {
String name;List<Field> fields;List<Method> methods;...
}
abstract class ASTNode {
int getLine();
...
}
class Program extends ASTNode {
List<ICClass> classes;...
}
![Page 11: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/11.jpg)
1111
non terminal Program program;non terminal ICClass class;non terminal List<ICClass> class_list;
program ::= class_list:cl {: RESULT = new Program(getLine(), cl); :};
class_list ::= class:c {: RESULT = new LinkedList<ICClass>(); RESULT.add(c); :} | class_list:cl class:c {: cl.add(c); RESULT = cl; :};
Actions part of IC.cup
Reduceclass_list ::= class_list:cl class:c {: cl.add(c); RESULT = cl :}
object stackSymbol stack
class_list
class
cl
c
RESULTclass_list
Object stackSymbol stack
pop classpop class_listpush class_list
In reality, usevalue field of Symbol
![Page 12: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/12.jpg)
1212
AST traversal
Once AST stable want to operate on treeAST traversal for type-checkingAST traversal for transformation (IR)AST traversal for pretty-printing (-dump-ast)
Each operation in separate pass
![Page 13: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/13.jpg)
1313
Non-Object Oriented approachprettyPrint(ASTNode node) { if (node instanceof Program) { Program prog = (Program) node; for (ICClass icc : prog.classes) { prettyPrint(icc); } } else if (node instanceof ICClass) { ICClass icc = (ICClass) node; printClass(icc); } else if (node instanceof BinaryExpression) { BinaryExpression be = (BinaryExpression) node; prettyPrint(be.lhs); System.out.println(be.operator); prettyPrint(be.rhs); } ...}
Messy code instanceof + down-casting error-prone Not extensible
![Page 14: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/14.jpg)
1414
Single dispatch - polymorphism
class A { void op() {…}}
class B extends A { void op() {…}}
class C extends A { void op() {…}}
op
A
B
C
index
conceptually,one-dimensional table
![Page 15: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/15.jpg)
1515
What if we need more operations?
class A { void op1() {…} void op2() {…} void op3() {…} }
class B extends A { void op1() {…} void op2() {…} void op3() {…}}
class C extends A { void op1() {…} void op2() {…} void op3() {…}}
Want to separate complicated operations from data structures
![Page 16: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/16.jpg)
1616
class A {}
class B extends A {}
class C extends A {}
class op1 extends op{ … // lots of code}
class op2 extends op{ … // lots of code}
class op3 extends op{ … // lots of code}
×
What if we need more operations?
![Page 17: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/17.jpg)
1717
class A {}
class B extends A {}
class C extends A {}
class op1 extends op{ doOp(A a){
… // lots of code }} doOp(B b){ … // lots of code }}
What if we need more operations?
Overloading is static
class op2 extends op{ doOp(A a){
… // lots of code }} doOp(B b){ … // lots of code }}
class op3 extends op{ doOp(A a){
… // lots of code }} doOp(B b){ … // lots of code }}
op o = new op3(); A a = new B();
o.doOp(a);
![Page 18: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/18.jpg)
1818
Separate operations on objects of a data structure from object representation
Each operation (pass) may be implemented as separate visitor
Use double-dispatch to find right method for object
Instance of a design pattern
Visitor Pattern
![Page 19: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/19.jpg)
1919
Visitor pattern in Javainterface Visitor { visit(A a); visit(B b); visit(C c);}
class A { A x; accept(Visitor v) { v.visit(this); }}
class B extends A { accept(Visitor v) { v.visit(this); }}
class op1 implements Visitor { visit(A a) {…} visit(B b) {…} visit(C c) {…}}×class op2 implements Visitor { visit(A a) {…} visit(B b) {…} visit(C c) {…}}
class op3 implements Visitor { visit(A a) {…} visit(B b) {…} visit(C c) {…}}
class C extends A { accept(Visitor v) { v.visit(this); }}
![Page 20: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/20.jpg)
2020
Double dispatch example
Visitor v = new op1(); // op1/2/3 A x = new B(); // x can be A/B/C x.accept(v);
class op1 implements Visitor { visit(A a) { } visit(B b) { … }}
class B { accept(Visitor v) { // always calls visit(B b) v.visit(this); }}
1st dispatch
2nd dispatch
![Page 21: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/21.jpg)
2121
Double dispatch example
Visitor v = new op1(); // op1/2/3 A x = new B(); // x can be A/B/C x.accept(v);
class op1 implements Visitor { visit(A a) { } visit(B b) { … }}
class B { accept(Visitor v) { // always calls visit(B b) v.visit(this); }}
1st dispatch
2nd dispatch
op1op2op3
A
B
C
1st dispatch
x.accept(v)
v.visit(this)2nd dispatch
op1.visit(B b)
Visitor pattern conceptuallyimplements two-dimensional table
![Page 22: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/22.jpg)
2222
Straight Line Program example
prog stmt_list
stmt_list stmtstmt_list stmt_list stmtstmt var = expr;stmt print(expr);
expr expr + exprexpr expr - exprexpr expr * exprexpr expr / exprexpr - exprexpr ( expr )expr numberexpr readi()expr var
ASTNode
Stmt Expr
PrintStmt
AssignStmt
BinaryOpExpr
UnaryOpExpr
NumberExpr
ReadIExpr
StmtList
VarExpr
(Code available on web site.Demonstrates scanning, parsing, AST + visitors)
![Page 23: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/23.jpg)
2323
Printing visitor example
interface Visitor { void visit(StmtList stmts); void visit(Stmt stmt); void visit(PrintStmt stmt); void visit(AssignStmt stmt); void visit(Expr expr); void visit(ReadIExpr expr); void visit(VarExpr expr); void visit(NumberExpr expr); void visit(UnaryOpExpr expr); void visit(BinaryOpExpr expr);}
public class PrettyPrinter implements Visitor { public void print(ASTNode root) { root.accept(this); }
public void visit(StmtList stmts) { for (Stmt s : stmts.statements) { s.accept(this); System.out.println(); } } // x = 2*7 public void visit(AssignStmt stmt) { stmt.varExpr.accept(this); System.out.print("="); stmt.rhs.accept(this); System.out.print(";"); } // x public void visit(VarExpr expr) { System.out.print(expr.name); } // 2*7 public void visit(BinaryOpExpr expr) { expr.lhs.accept(this); System.out.print(expr.op); expr.rhs.accept(this); } ...}
![Page 24: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/24.jpg)
2424
Visitor variationsinterface PropagatingVisitor {
/** Visits a statement node with a given * context object (book-keeping) * and returns the result * of the computation on this node. */ Object visit(Stmt st, Object context);Object visit(Expr e, Object context);Object visit(BinaryOpExpr e, Object context);...
}
Propagate values down the AST (and back)
![Page 25: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/25.jpg)
2525
Evaluating visitor examplepublic class SLPEvaluator implements PropagatingVisitor { public void evaluate(ASTNode root) { root.accept(this); } /** x = 2*7 */ public Object visit(AssignStmt stmt, Object env) { Expr rhs = stmt.rhs; Integer expressionValue = (Integer) rhs.accept(this, env); VarExpr var = stmt.varExpr; ((Environment)env).update(var, expressionValue); return null; }
/** expressions like 2*7 and 2*y */ public Object visit(BinaryOpExpr expr, Object env) { Integer lhsValue = (Integer) expr.lhs.accept(this, env); Integer rhsValue = (Integer) expr.rhs.accept(this, env); int result; switch (expr.op) { case PLUS: result = lhsValue.intValue() + rhsValue.intValue() ; ... } return new Integer(result); } ...}
class Environment { Integer get(VarExpr ve) {…} void update(VarExpr ve, int value) {…}}
![Page 26: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/26.jpg)
2626
NumberExpr
value = 1
BinaryOpExpr +
left right
NumberExpr
value= 2
VarExpr
name = x
BinaryOpExpr +
left right
class BinaryOpExpr extends Expression {
Object accept(Visitor v) {
return v.visit(this);
}
Expression lhs, rhs;
}
class NumberExpr extends Expression {
Object accept(Visitor v) {
return v.visit(this);
}
int val;
}
SLPEvaluator ev = new SLPEvaluator();Integer result = (Integer)root.accept(ev);
root
visit(lhs)
visit(lhs)
visit(rhs)
visit(rhs)
1 2
3
6
3
AST traversal
public class SLPEvaluator … { public Object visit(BinaryOpExpr e, Object env) { Integer lhsValue=(Integer)e.lhs.accept(this,env); Integer rhsValue=(Integer)e.rhs.accept(this,env); int result; switch (expr.op) { case PLUS: result=lhsValue.intValue()+rhsValue.intValue(); ... } return new Integer(result); } public Object visit(NumberExpr e,Object env) { return e.value; } public Object visit(VarExpr e, Object env) { return ((Environment)env).get(e); }
1+2+x
![Page 27: Compiler Construction Abstract Syntax Tree Rina Zviel-Girshin and Ohad Shacham School of Computer Science Tel-Aviv University](https://reader031.vdocument.in/reader031/viewer/2022031901/56649d555503460f94a31f92/html5/thumbnails/27.jpg)
2727
Error recovery
How to catch errors public void syntax_error(Symbol cur_token)
Optional error recovery Use error token in your grammar stmt ::= expr SEMI
| while_stmt SEMI | if_stmt SEMI | ... | error SEMI ;