Compiler Compiler ConstructionConstruction

Compiler Compiler ConstructionConstruction

Sohail Aslam

Lecture 9

2

DFA MinimizationDFA MinimizationDFA MinimizationDFA Minimization The generated DFA may

have a large number of states.

Hopcroft’s algorithm: minimizes DFA states

3

DFA MinimizationDFA MinimizationDFA MinimizationDFA Minimization The generated DFA may

have a large number of states.

Hopcroft’s algorithm: minimizes DFA states

4

DFA MinimizationDFA MinimizationDFA MinimizationDFA Minimization Idea: find groups of

equivalent states. All transitions from

states in one group G1 go to states in the same group G2

5

DFA MinimizationDFA MinimizationDFA MinimizationDFA Minimization Idea: find groups of

equivalent states. All transitions from

states in one group G1 go to states in the same group G2

6

DFA MinimizationDFA MinimizationDFA MinimizationDFA Minimization

Construct the minimized DFA such that there is one state for each group of states from the initial DFA.

7

DFA MinimizationDFA MinimizationDFA MinimizationDFA Minimization

DFA for (a | b )*abb

A

a

b a

b bEB D

C

a

b

b

a

a

8

DFA MinimizationDFA MinimizationDFA MinimizationDFA Minimization

Minimized DFA for (a | b )*abb

A,C

ab

b bEB D

a

b

a

a

9

Optimized AcceptorOptimized AcceptorOptimized AcceptorOptimized Acceptor

input string

RE

w

R

yes, if w L(R)

no, if w L(R)

RE=>NFA

NFA=>DFA

Min. DFA

SimulateDFA

10

Lexical AnalyzersLexical AnalyzersLexical AnalyzersLexical Analyzers Lexical analyzers (scanners)

use the same mechanism but they:

• Have multiple RE descriptions for multiple tokens

• Have a character stream at the input

11

Lexical AnalyzersLexical AnalyzersLexical AnalyzersLexical Analyzers Lexical analyzers (scanners)

use the same mechanism but they:

• Have multiple RE descriptions for multiple tokens

• Have a character stream at the input

12

Lexical AnalyzersLexical AnalyzersLexical AnalyzersLexical Analyzers Lexical analyzers (scanners)

use the same mechanism but they:

• Have multiple RE descriptions for multiple tokens

• Have a character stream at the input

13

Lexical AnalyzersLexical AnalyzersLexical AnalyzersLexical Analyzers

• Return a sequence of matching tokens at the output (or an error)

• Always return the longest matching token

14

Lexical AnalyzersLexical AnalyzersLexical AnalyzersLexical Analyzers

• Return a sequence of matching tokens at the output (or an error)

• Always return the longest matching token

15

Lexical AnalyzersLexical AnalyzersLexical AnalyzersLexical Analyzers

characterstream

R1…R2

Tokenstream

RE=>NFA

NFA=>DFA

Min. DFA

SimulateDFA

R1…R2

16

Lexical Analyzer GeneratorsLexical Analyzer GeneratorsLexical Analyzer GeneratorsLexical Analyzer GeneratorsThe lexical analysis process

can automatedWe only need to specify

• Regular expressions for tokens

• Rule priorities for multiple longest match cases

17

Lexical Analyzer GeneratorsLexical Analyzer GeneratorsLexical Analyzer GeneratorsLexical Analyzer GeneratorsThe lexical analysis process

can automatedWe only need to specify

• Regular expressions for tokens

• Rule priorities for multiple longest match cases

18

Lexical Analyzer GeneratorsLexical Analyzer GeneratorsLexical Analyzer GeneratorsLexical Analyzer GeneratorsFlex

generates lexical analyzer in C or C++

Jlexwritten in Java. Generates lexical analyzer in Java

19

Lexical Analyzer GeneratorsLexical Analyzer GeneratorsLexical Analyzer GeneratorsLexical Analyzer GeneratorsFlex

generates lexical analyzer in C or C++

Jlexwritten in Java. Generates lexical analyzer in Java

20

Using FlexUsing FlexUsing FlexUsing FlexProvide a specification file

Flex reads this file and produces C or C++ output file contains the scanner.

The file consists of three sections

21

Using FlexUsing FlexUsing FlexUsing FlexProvide a specification file

Flex reads this file and produces C or C++ output file contains the scanner.

The file consists of three sections

22

Using FlexUsing FlexUsing FlexUsing FlexProvide a specification file

Flex reads this file and produces C or C++ output file contains the scanner.

The file consists of three sections

23

Flex Specification FileFlex Specification FileFlex Specification FileFlex Specification File

C or C++ and flex definitions 1

24

Flex Specification FileFlex Specification FileFlex Specification FileFlex Specification File

C or C++ and flex definitions

%%

token definitions and actions

1

2

25

Flex Specification FileFlex Specification FileFlex Specification FileFlex Specification FileC or C++ and flex definitions

%%

token definitions and actions

%%

user code 3

1

2

26

Specification File Specification File lex.llex.lSpecification File Specification File lex.llex.l%{#include “tokdefs.h”%}D [0-9]L [a-zA-Z_]id {L}({L}|{D})*%%"void" {return(TOK_VOID);}"int" {return(TOK_INT);}"if" {return(TOK_IF);}

27

Specification File Specification File lex.llex.lSpecification File Specification File lex.llex.l"else" {return(TOK_ELSE);}"while"{return(TOK_WHILE)};"<=" {return(TOK_LE);}">=" {return(TOK_GE);}"==" {return(TOK_EQ);}"!=" {return(TOK_NE);}{D}+ {return(TOK_INT);}{id} {return(TOK_ID);}[\n]|[\t]|[ ];%%

28

File File tokdefs.htokdefs.hFile File tokdefs.htokdefs.h#define TOK_VOID 1#define TOK_INT 2#define TOK_IF 3#define TOK_ELSE 4#define TOK_WHILE 5#define TOK_LE 6#define TOK_GE 7#define TOK_EQ 8#define TOK_NE 9#define TOK_INT 10#define TOK_ID 111

29

Invoking FlexInvoking FlexInvoking FlexInvoking Flex

lex.l lex.cppflex

30

Using Generated ScannerUsing Generated ScannerUsing Generated ScannerUsing Generated Scannervoid main(){ FlexLexer lex; int tc = lex.yylex(); while(tc != 0) cout << tc << “,” <<lex.YYText() << endl; tc = lex.yylex();}

31

Creating Scanner EXECreating Scanner EXECreating Scanner EXECreating Scanner EXE

flex lex.lg++ –c lex.cppg++ –c main.cppg++ –o lex.exe lex.o main.o

lex <main.cpp