topic #2: infix to postfix ee 456 – compiling techniques prof. carl sable fall 2003
TRANSCRIPT
Why this topic?
• Program to convert infix to postfix
– A simple, one-pass compiler!– Infix notation is source language– Postfix is intermediate code– No code generation (but could be actions)
• A good example that introduces many aspects of writing a compiler
Syntax vs. Semantics
• Syntax– Describes what is allowable in a language– Relatively easy to describe (we will use context free
grammars)
• Semantics– Describes the meaning of a program (or expression)– Very difficult to describe in a formal sense
• We are discussing programming languages, but these definitions also apply to natural languages
Structure of Simple Compiler
• A lexical analyzer will tokenize the input
• A “syntax-directed translator” combines syntax analysis and intermediate code generation
Context-free Grammars
• Often used to describe the hierarchical structure of a programming language
• Every CFG has four components:– A set of tokens (terminal symbols)– A set of nonterminals– A set of productions (rules)– A start symbol (left side of first rule)
Example rule: stmt → if (expr) stmt else stmt
Productions
• Every production consists of left side, an arrow (“can have the form of”), right side
• Left side is a always nonterminal; the production is “for” this nonterminal
• Right side consists of terminals and/or non-terminals
• Convention, nonterminals will be italicized
Example rule: stmt → if (expr) stmt else stmt
Example CFG
• List of digits separated by plus or minus signs:
• For convenience, nonterminals can be grouped using ‘|’ (“or”)
• The tokens of this grammar: + - 0 1 2 3 4 5 6 7 8 9• The start symbol: list
list → list + digitlist → list – digitlist → digitdigit → 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
list → list + digit | list – digit | digitdigit → 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
CFG for Expressions
expr → expr + term | expr – term | termterm → term * factor | term / factor | factorfactor → digit | (expr)digit → 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
Strings
• A string of tokens is a sequence of zero or more tokens
• The string containing zero tokens is the empty string: ‘ε’
• A grammar “derives” strings– starting with start symbol– repeatedly replacing nonterminals with right side of
productions for the nonterminals– The set of strings that can be derived from the start
symbol is the “language” for the grammar
Parse Trees
• Shows how the start symbol of a grammar can derive a string in the language
• A tree with the following properties:– The root is labeled with a start symbol– Each leaf labeled with a token or ε– Each interior node is labeled by a nonterminal
– If A is the label for an interior node, and X1,X2,…,Xn (nonterminals or tokens) are the labels of its children, then the following production must exist:
A → X1 X2 … Xn
More on Parse Trees
• A language can be defined as all strings that can be generated by some parse tree
• The process of finding a parse tree for a given string is called “parsing” the string
Ambiguous Grammars
• If any string has more than one parse tree, grammar is said to be ambiguous
• Need to avoid for compilation, since string can have more than one meaning
• List of digits separated by plus or minus signs:
• Example merges notion of digit and list into single nonterminal string
• Same strings are derivable, but some strings have multiple parse trees (possible meanings)
string → string + string | string – string |0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
Precedence and Associativity
• Precedence– Determines the order in which different operators are
evaluated when they occur in the same expression– Operators of higher precedence are applied before
operators of lower precedence
• Associativity– Determines the order in which operators of equal
precedence are evaluated when they occur in the same expression
– Most operators have a left-to-right associativity, but some have right-to-left associativity
Postfix Notation
• Formal rules, infix → postfix– If E is variable or constant, E → E– If E is expression of form E1 op E2, where op is binary operator, E1 → E1’, and E2 → E2’, then E → E1’ E2’ op
– If E is expression of form (E1) and E1 → E1’, then E → E1’
• Parentheses are not needed!
Syntax-Directed Definitions
• CFG to specify syntactic structure of input• Each grammar symbol has associated attributes• Each production has associated semantic rules
for computing values of attributes• Grammar and semantic rules together constitute
the syntax-directed definition• A parse tree showing the attribute values at
each node is called an annotated parse tree
Two Types of Attributes
• Synthesized Attributes– Value at a parse-tree node can be determined
based on values of attributes of children– Can be evaluated during a single bottom-up
traversal of parse tree
• Inherited Attributes– More complicated– Will be discussed later in course
Example Syntax-Directed Definition
Production Semantic Rule
expr expr1 + term expr.t := expr1.t || term.t || ‘+’
expr expr1 - term expr.t := expr1.t || term.t || ‘-’
expr term expr.t := term.t
term 0 term.t := ‘0’
term 1 term.t := ‘1’
… …
term 9 term.t := ‘9’
Depth-First Search
• Recursively visit all children before evaluating semantic rules of given node
• Suitable if all attributes are synthesized
procedure visit (n:node)begin
for each child m of n, from left to right dovisit(m)
evaluate semantic rules at node nend
Translation Schemes
• Adds to a CFG
• Includes “semantic actions” embedded within productions
• Similar to syntax-directed definition, but order of evaluation is explicit
Example Translation Scheme
expr expr + term { print(‘+’) }expr expr – term { print(‘-’) }expr termterm 0 { print(‘0’) }term 1 { print(‘1’) }
…term 9 { print(‘9’) }
Equivalent Translation Scheme
expr term restrest + term { print(‘+’) } restrest - term { print(‘-’) } restrest εterm 0 { print(‘0’) }term 1 { print(‘1’) }
…term 9 { print(‘9’) }
Parsing
• Parsing is the process of determining if a string of tokens can be generated by a grammar
• For any CFG, there is a parser that takes at most O(n^3) time
• For most programming languages that arise in practice, linear time parsers exist
Top-down Parsing
• Recursively apply the following steps:– At node n with nonterminal A, select a production for A
– Construct children at n for symbols on right side of selected production
– Find next node for which subtree needs to be constructed
• Top-down parsing uses a “lookahead” symbol• Selecting production may involve trial-and-error
and backtracking
Predictive Parsing
• Recursive-descent parsing is a recursive, top-down approach to parsing
• A procedure is associated with each nonterminal of the grammar
• Predictive parsing– Special case of recursive-descent parsing– The lookahead symbol unambiguously
determines the procedure for each nonterminal
Procedures for Nonterminals
• Production with right side α used if lookahead is in FIRST(α)– FIRST(α) is set of all symbols that can be first symbol
of α– If lookahead symbol is not in FIRST set for any
production, can use production with right side of ε– If two or more possibilities, can not use this method– If no possibilities, an error is declared
• Nonterminals on right side of selected production are recursively expanded
Left Recursion
• Left-recursive productions can cause recursive-descent parsers to loop forever
• Example: example example + term
• Can eliminate left recursion
A A α | βA β RR α R | ε
Eliminating Left Recursionexpr expr + term { print(‘+’) }expr expr – term { print(‘-’) }expr termterm 0 { print(‘0’) }term 1 { print(‘1’) }
…term 9 { print(‘9’) }
expr term restrest + term { print(‘+’) } restrest - term { print(‘-’) } restrest εterm 0 { print(‘0’) }term 1 { print(‘1’) }
…term 9 { print(‘9’) }
Infix to Prefix Code: Part 1#include <stdio.h>#include <ctype.h>
int lookahead;
void expr(void);void rest(void);void term(void);void match(int);void error(void);
int main(void){ lookahead = getchar(); expr(); putchar('\n'); /* adds trailing newline character */}…
Infix to Prefix Code: Part 2…void expr(void){ term(); rest();}
void term(void){ if (isdigit(lookahead)) { putchar(lookahead); match(lookahead); } else error();}…
Infix to Prefix Code: Part 3…void rest(void){ if (lookahead == '+') { match('+'); term(); putchar('+'); rest(); } else if (lookahead == '-') { match('-'); term(); putchar('-'); rest(); }}…
Infix to Prefix Code: Part 4…void match(int t){ if (lookahead == t) lookahead = getchar(); else error();}
void error(void){ printf("syntax error\n"); /* print error message */ exit(1); /* then halt */}
Code Optimization 1void rest(void){REST: if (lookahead == '+') { match('+'); term(); putchar('+'); goto REST; } else if (lookahead == '-') { match('-'); term(); putchar('-'); goto REST; }}
Code Optimization 2void expr(void){ term(); while (1) { if (lookahead == '+') { match('+'); term(); putchar('+'); } else if (lookahead == '-') { match('-'); term(); putchar('-'); } else break; }}
Improvements Remaining
• Want to ignore whitespace
• Allow numbers
• Allow identifiers
• Allow additional operators (multiplications and division)
• Allow multiple expressions (separated by semicolons)
Lexical Analyzer
• Eliminates whitespace (and comments)
• Reads numbers (not just single digits)
• Reads identifiers and keywords
Allowable Tokens
• expected tokens: +, -, *, /, DIV, MOD, (, ), ID, NUM, DONE
• ID represents an identifier, NUM represents a number, DONE represents EOF
Tokens and Attributes
LEXEME TOKEN ATTRIBUTE VALUE
white space --- ---
sequence of digits NUM numeric value of sequence
div DIV ---
mod MOD ---
letter followed by letters and digits
ID index into symbol table
EOF DONE ---
any other character that character NONE
A Simple Symbol Table
• Each record of symbol table contains a token type and a string (lexeme or keyword)
• Symbol table has fixed size• All lexemes in array of fixed size• Will be able to insert and search for tokens:
– insert(s, t): creates entry with string s and token t, returns index into symbol table
– lookup(s): searches for entry with string s, returns index if found, 0 otherwise
• Keywords (div and mod) will be inserted into symbol table, they can not be used as identifiers
Updated Translation Scheme
start list eoflist expr; list | ε expr expr + term { print(‘+’) } | expr – term { print(‘-’) } | termterm term * factor { print(‘*’) } | term / factor { print(‘/’) } | term div factor { print(‘DIV’) } | term mod factor { print(‘MOD’) } | factorfactor (expr) | id { print(id.lexeme) } | num { print(num.value) }
After Eliminating Left Recursion
start list eoflist expr; list | ε expr term moretermsmoreterms + term { print(‘+’) } moreterms | - term { print(‘-’) } moreterms | εterm factor morefactorsmorefactors * factor { print(‘*’) } morefactors | / factor { print(‘/’) } morefactors | div factor { print(‘DIV’) } morefactors | mod factor { print(‘MOD’) } morefactors | εfactor (expr) | id { print(id.lexeme) } | num { print(num.value) }