AN/N LOG N ALGORITHM FOR MINIMIZINGSTATES IN kF I N ITE AUTOMATON

BY

JOHN HOPCROFT

STAN-CS-71-190January, 1971

COMPUTER SCIENCE DEPARTMENT

School of Humanities and Sciences

STANFORD UN IVERS ITY

AN N LOG N ALGORITHM FOR MINIMIZING STATES IN A FINITE AUTOMATON

John Hopcroft

Abstract

An algorithm is given for minimizing the number of states in a finite automaton or for

determining if two finite automata are equivalent. The asymptotic running time of the

algorithm is bounded by knlogn where k is some constant and n is the number of

states. The constant k depends linearly on the size of the input alphabet.

This research was supported by the National Science Foundation under grant number NSF-GJ-96,and the Office of Naval Research under grant number N-00014-67-A-0112-0057 NR 044-402.Reproduction in whole or in part is permitted for any purpose of the United States Government.

AN n log n ALGORITHM FOR MINIMIZING STATES IN A FINITE AUTOMATON

John Hopcroft

Stanford University

Introduction

Most basic texts on finite automata give algorithms for minimizing the number of states in a finite

automaton [l, 21. However, a worst case analysis of these algorithms indicate that they are n2 processes

where n is the number of states. For finite automata with large numbers of states, these algorithms are

grossly inefficient. Thus in this paper we describe an algorithm for minimizing the states in which the

asymptotic running time in a worst case analysis grows as n log n . The constant of proportionality depends

linearly on the number of input symbols. Clearly the same algorithm can be used to determine if two finite

automata are equivalent.

The essence of previously published algorithms was to first partition the states according to their

outputs. The blocks of the partitions are then repeatedly refined by examining the successor state on a

given input for each state in the block. States whose successor states on a given input are in different

blocks are placed in separate blocks. When no further refinement is possible, all states in the same block of

the partition can be shown to be equivalent. Consider the example in Figure 1. The initial partition is

(1,2,3,4,5)(6) . Since on input 0 , the successor

states of states 1, 2, 3 and 4 are in the first blockInput

State 0 1 Output__ - - - -

of the partition and the successor of state 5 is in 1 2 1 0

the second block, the first iteration refines the 2 3 2 0

3

I! '

4 3 0partition into the blocks (1,2,3,4)( 5) and (6) .

4. 5 4 0

Successive refinements yield (1,2,3)(4)(5) (6) ; 5 6 5 0

(1,2)(3)(4)(5)(6) and (l)(2)(3)(4)(5)(6) - Thus,6 6 6 1

Nextin this example all pairs of states are inequivalent. State

For this example it is seen that as many as nFigure 1

iterations may be required and the total number of

steps needed to execute the algorithm if implemented in a straightforward fashion on a digital computer

is n2 .

The algorithm proposed in this paper may also require n iterations but the work per iteration summed

over all iterations yields only n log n . We illustrate the algorithm by an example before specifying it in

detail. Extensive use of list processing is employed to reduce the computation time. First the state table

is inverted to obtain the table shown in Figure 2. The states are partitioned according to their outputs

($293,495) (6) . Next a block and an input symbol on which the partition is refined are selected. Assume

the block (6) and input 0 are selected. The states in each block are further partitioned depending on

whether on input 0 their next state is in block (6) or not. Thus the next partition is (1,2,3,4)(5)(6) .

Note that had we partitioned on the block (1,2,3,4,5) and input 0 we would have obtained the same result.

1

More generally, once we have partitioned on a block

and an input symbol, we need never partition on that

block and input symbol again until the block is

split and then we need only partition on one of

the two subblocks. Since the time needed to

partition on a block is proportional to the

transitions into the block and since we can always

select the half with fewer transitions, the total

number of steps in the algorithm is bounded by

4 3 4 0

5 4 5 0

6 5,6 .6 1

previousstate

Figure 2

n log n .

Formal description of the algorithm

Let A = (S,I,G,F) be a finite automaton where S is a finite set of states, I is a finite set of

inputs, 6 is a mapping from S x I into S and F c S is the set of final states. No initial state is

specified since it is of no importance in what follows. The mapping 6 is extended to SxI* in the usual

*manner where I denotes the set of all finite length strings of symbols from I . States s and t are

*said to be equivalent if for each x in I , 6(s,x) is in F if and only if 6(t,x) is in F .

The algorithm for finding the equivalence classes of S is described below.

1.Step trseS and ae1 construct

6-'(s,a) = {tlE(t,a) = s] .

Step 2. Construct B(1) = F , B(2) = S-F and for each a in I and l_< i 5 2 construct

a(i) = {s]ssB(i) and E-'(s,a) # @] .

3 .Step Set k=3

Step 4. VasI construct

Cl3 if la(l) \ 5 142) IL(a) = .

c23 otherwise

5 .Step Select a in I and i in L(a) . The algorithm terminates when L(a) = $ for each a in I .

Step 6. Delete i from L(a) .

Step 7. Vj < k st 3% in B(j) with s(t,a) es(i) perform steps 7a, 7b,a7c, and 7d.

Step 7a. Partition B(j) into B'(j) = {tls(t,a) e&(i)] and B"(j) = B(j)-B'(j) .

7b.Step Replace B(j) by B'(j) and construct B(k) = B"(j) . Construct corresponding a(j)

and a(k) for each a in I .

7 c .Step TagI modify L(a) as follows.

L(a) U CjlL(a) =

if j{L(a) and 0 < la(j)1 _< la(k)\

L(a) U ikj otherwise

Step 7d. Set k = k+l .

8 .Step Return to Step 5.

Correctness of algorithm

The claim is made that on termination of the algorithm two states are equivalent if and only if they are

in the same block. The algorithm must terminate since the only times that an index is added to L(a) for

some a in I are in Step 4 which is executed only once and in Step 7c. An index is added at Step 7c only

after a refinement of a block of the partition. Each time Step 6 is executed, an index is removed from L(a)

for some a . Thus the algorithm must terminate.

It is easily shown by induction on the number of times Step 7a is executed that if s is in B(i) and

t is in B(j) , i # j , then s is not equivalent to t . Clearly, it is true the first time Step 7a is

executed since only two blocks exist, B(1) containing only final states and B(2) containing only nonfinal

states. Blocks are refined at Step 7a only when successor states on a given input have previously been

shown to be inequivalent.

To see that two inequivalent states cannot be in the same block when the algorithm terminates, assume

that states s and t are in B(i) and that s and t are not equivalent. Without loss of generality,

assume 8( s, a) is in B(j) and s(t,a) is in B(k) where j f k . (If s(s,a) and s(t,a) are in the

same block then there exists a shortest x such that 6(s,x) and 6(t,x) are in distinct blocks. Clearly

an x exists and hence a shortest x since for same x one or the other of 6(s,x) and 6(t,x) but not

both is in a final state and each block consists solely of final or solely of nonfinal states. Let a be the

last symbol of x and write x = ya . Then 6(s,y) and F(t,y) are in the same block, ~(s,Y) and 6(t,y)

are not equivalent and 6(6(s,y),a) and 6(6(t,y),a) are in different blocks. Replace s by 6(s,y) and

replace t by 6(t,y) .) Consider the point at which the block containing 6(s,a) and &(t,a) was

partitioned so that o(s,a) and s(t,a) first appeared in separate subblocks. At that point one of the

two subblocks was placed in L(a) . When this subblock is removed from L(a) , the block containing s and

t is partitioned with s and t going into separate subblocks. Thus s and t cannot both be in B(i) ,

a contradiction.

Analysis of the running time

The running time of the algorithm is clearly dependent on the implementation. The algorithm has been

programmed in AIGOL. Since the implementation consists of approximately 300 ALGOL statements we shall simply

indicate how the various steps were implemented and discuss informally their running time.

-3

Tlie sets sui:h as 6 -'(s,a) , L(a) , etc. were represented by linked lists in such a way that an item

could be added o-- deleted at the beginning of the l'st <r. a fixed number of steps. Vectors were also

maintained to indicate if a state was or was x;ot on a c:'-r~,n list. This eliminates searching a list simply

to determine if the item is on the list and is essen',in .n Step 7c. The sets B(i) and a(i) were

represented 8s dou'[Ijr linked lists SO that an iwtem could oe added or deleted an)qJhere in the list in a fixed

number of steps once the position is given. The struct:,Ye was such that given a state s , its position in

B(i) and a(i) could be determined in a fixed number of steps.

Steps 1, I', and 4 are executed only once and eal:ire time proportional to the product of the number of

states ti!?rs 'lf: number of input symbols. f,teps :, thr3';-‘tI 9 f orm a simple loop. The time necessary to

traverse t?e l,;op for given a in I and i in I,(a) is proportional to the number of state transitions

on input a 'erminating on states in B(i) . (To see tl.i,s, note that Steps 5, 6 and 8 are finite.

Step 7 WI do not need to examine B(j) for eacil J < 'I I-o see if there exists a t in B(j) with 8(t,a)

in a(i) . !:ather we look at each state in a(i) and i,hen consult the inverse state table to find each t

such t!,a: s(t,a) is in a(i) . Each time a new t ic found, the block containing t is located and t

placed or- : list of states to be split off from the :jl~.:. The block is then placed on a list of blocks

which na-,e becr~ refined if it is not already on the list. Finally we go down the list of blocks which have

been refined and actually partition them. The number of blocks we must look at must be less than the number

of state t-.a-,s'tians on input a terminatin,? on states in B(i) . The time necessary to actually partition

a block is piorzrtional to the number of states to be split off. When the number is summed over all blocks

which are part;t.ioned it adds up to the number of state transitions on input a terminating on states in

B(i) .) I; be the constant of proportionality.

Consider :,i,e time spent in the loop of Step j thro@~ 8 for a given input symbol a . Assume that at

step > the blocks of the partition are B(l),B(Z),...,B(m) and that L(a) = {il,i2, . . ., ir] . Let

{ir+-l,ip32,...,im] = {1,2,...,m] -L(a) . The clati, is made that the total time spent on traversals of the

loop for which input symbol a is selected in Step 5 until the program terminates is bounded by

T=k(? aim

j=l jlog ai. + C ai. loe(ai ./T)! -

J j=r+l J 3

Clearly the bound is valid if the algorithm has terminated. If the loop is traversed for an input symbol

other than a , then the time spent is not included in T . However, since blocks get split and the set

L(a) modified we must show that the new value of T , call it !?' , is less than or equal to the old value

of r . If a block whose index is in L(a) is partitioned, then a term of the form b log b is replaced by

the expression c log c + (b-c) log(b-c) which decreases the value of T . If a block whose index is not

in L(a) is partitioned, then a term of the form b log b/2 is replaced by the expression

-c log c + (b -c) log(b -c)/2

where _c <b-c . Since c _<b/2 and (-b-c)/2 _< b/2 ,

c log c + {b -c) log@ -c)/2 < c log b/2 + (b -c) log b/2

_< blogb/? .

ither case $ is less than T . Finally, ass-Lme r # 0 and a and some B in L(a) has been

4

selected at Step 5. As we showed earlier, the time around the loop is bounded by kal . Thus by induction,

the total time is bounded by

k[ap + a1 log(aJ?) + i a.m

ljlog a. +

j=l lj c aij=r+l j

W3(ai /2) 1 .j

jb I

We must show that this eqression is less than or equal to T . That is, we need show

&P + aa log a!/2 _< a1 log ae .

Clearly al +a1 log a!/2 = ap(log a!/2 + log 2) = ap log aI . This completes the proof of the claim.

The first time Step 5 is executed the formula for T is bounded by knlog n . Multiply by the number

of input symbols and adding in the time for Steps 1 through 4 yields a total bound proportional to n log n .

Experimental results and conclusions

In order to obtain timing information, the algorithm was applied to two classes of finite automata.

Automata in the first class are given by A(n) = ( fL2, . . ..n].{O,l],s,{l]) where 5(1,0) = 6(1,1) = 1 and

6(i,O) = i-l and 6(i,l) = i for 2 _< i _< n . Automata in the second class are given (for even n) by

B(n) = ([1,2,..., n3, CO,ll,~, Cill _< i,_< n/23) where 5(i,O) = 6(i,l) = n/2 + 2i-1 and

6(n/4+ i,O) = 6(n/4+ i,l) = 2i-1 for l_< i _< n/4 and S(n/2+i,O) = 6(n/2+i,l) = 2i-1 for

n/2 <i_<n . The running times on an IBM 360/67 for the two classes are listed in Table 1.

n

100

1000

2003

Table 1.

A(n) B(n)

time in seconds

Note that A(n) is the example which required n2 steps for previous algorithms.

Our algorithm is particularly suited for A(n) and a detailed analysis shows that the running time

should grow linearly-with the number of states as the experimental evidence indicates. The worst case for

our algorithm is typified by B(n) in which blocks are always partitioned equally. The running time for

B(n) should grow as n log n for both the current algorithm and for previously published algorithms. The

results seem to indicate that the algorithm is practical for minimizing states in finite automata (or testing

equivalence of finite automata) of up to several thousand states.

References

1. Harrison, M. A., Introduction to Switching and Automata Theory, McGraw-Hill, New York, 1965.

2. McCluskey, E. J., Introduction to the Theory of Switching Circuits, McGraw-Hill, New York, 1965.

5

l-.-.r,-,

i-/

:I

,. .

CL1

--

- <<

.- r

- *

t:;

--

: (, t

1-

-^>

:L‘!

--

L:::

t?

-?3,

164

--

C^

(C

3-

:,:j

r, -

-o

ce7

--

*; *:

f $3

- -

1; ,;

t ‘1

- -

$,;iC

+

a

1:7

1 -

-01

: 7 2

- -

‘::7

3 -

-:c74

-4

:c75

--

QC

7h

4-

:Ci7

--

:c73

--

cc79

--

!; r)

I? fj

- -

ZC

El

--

,?C

E2

--

‘:C

f3 -

4,CE4

--

>I(-,?5

--

LZEf?

--

cce7

--

CC&

?8 -

-i;

,;Eq

--

:rco

--

n .c

GL‘

) -

-1

cc9

2 -

-c

c5

3 4

-Ii794

--

cc<5

--

CO

96 -

4C

CC

7 -

-

Cc5

8 -

-LC

CO

--

)1

,0

-

-01r1

4-

ClC

Z -

-0

13

3

--

OlC

4 -

-C

lC5

--

c!lC

h -

-O

lC7

--

CL

C8

-4

ClC

3 -

-O

LLC

’ -

-2

11

1 -

-C

A12

--

31

13

4-

Cl1

4 -

-c115

5-

@1

16

--

?11

1 -

-“

11

? e

-T

ll’>

--

Cli

j

--

-2 1

L 1

- *

CLLI --

-@123 h-

:‘LL't --

712s

--

rllih -6

Cli7 --

)lZ

Y -

-:1

is -

5J

1 3

,I-

-n

13

1

:-c132

--

1: -.

J --

11

’Cl?4 t-

2135

--

?L?F,

--

:137 -6

31

3,;

--

Cl?

‘2

6-

Cl443

--

1:1t1

--

OlL

.2

-5Cl45

--

Gi44

--

Cl45

-5

0146

-4

Cl47

--

Cl48 4-

0143

--

c1c31

--

T'lC1

-/t

Cl5

7 -

-Q

lC3

4-

Cl54

--

Cl53

--

ClE

h -

4(7

157

--

<lC

E

4-

$15)

--

Clt

t; -

-31t1

-4

CL

Cl

--

31

63

4-

c1t4

--

3tt3

--

7166

-4

Cl67

--

ilk

+!

-3,l

t’

--

31

7:

--

,:1

71

--

01

72

--

01

73

-

-C

l74

!-

!F EL3

E C

C I

h @C

C{

3L

’:C

KL

IST

ZE

H(J

r3);

I:r,k

!CC

CK

L I

ST

ZE

4C

(2):

=2;

Ch

S;

(p~-

ivc”

~+:-

( L

)>=

‘?l

AhC

4

PfJ

F(C

h;_

(l

)<=

‘,tJP

r?‘4

F(L

)

1 TI

-“_N

SF

;Ih AC

C (

!:L

CC

YL

I S

TC

hrE

, 1)

;C

hB

LcC

,YL

IST

C+

1’(

1)

:=l;

E hl!

:F P

EG

Ih Act

(s?

Li‘S

KL

IS

Tcr.

~r2

) ;

Ch

hL

Sr.

KL

IST

CN

E(Z

):=

2;

cr.

r;

.r”

LccK

:=3;

Cl7

5 -

-2

17

0 -

-C

L7

7 -

-8

17

8

--

c175

-3

CA

E’:

--

31S

l 3-

ala

? -

-0

18

3

--

Cl@

4 -

-3

1&

5

--

Clfb -3

ClE

7 -

-C

l88

--

c1e4

--

01

90

3-

01

51

--

Cl9

2 -

-C

l53

4-

01

94

--

01

95

--

OlS

b -

-C

lS7

--

OlS

d -

-0

15

9 -

-c2

c3

--

OZ

Cl

-4

02c2

--

02

23

-

3r)

2C4

--

02

C5

--

G2

Cb

--

zic

7

3-

C2C

H -

-0

2C

9 -

-3

21

0

4-

02

11

--

c/2

12

--

c213

--

02

14

--

02

15

--

C2

16

--

-32

17

--

C2lR

-4

32

14

-

-c220

-3

c2

21

--

02

22

-

-0

22

.4

--0

22

4 -

-c225

--

02

26

--

02

27

--

022t

3 -

-C

ZiQ

3-

02

3ll

--

C23

L -

-0

23

2 -

-

1;9

CE

CE

”HE

R 1

97

0 a;

12

: 1

7P

4CE

4

J:=S~Tb(~LCCKtIST7~a~l~

~F

”CV

~F

I:lIY

(i:L

CC

KL

I~T

ZE

~T

:l;

Ch

QL

CC

KL

T~

TZ

ER

C(J

l:=

@:

r,t:

Tf-

1 !?

I\,

‘<S

tJ;

=h

cC

LT

E

I=~>C~T~;LI~;TC~~I=~ TkEh

P "C

IN

~:=

C~

TJ(?

L~

CK

LIS

TC

~F

);R

F~

Cv~

FR

~V

(eL

CC

KL

IST

CN

F);

Ch

CL

CC

KC

I3T

CN

C(J

):=

3;

GC

TO

DIV

CN

E;

EhC

EL

SF

CC

TC

FIh

ISkE

C;

CIV

ZE

FC

:S

TA

TE

:=Z

CR

Ob

FX

T(J

I;kH

ILE

S

TA

TF

-r=O

D

Cf!

EG

Ih CE

LL

:=P

RE

vC

hZ

ER

O(S

TA

TE

-fo

b;

WkI

CE

CE

LL

.-=

0 C

OY

EG

Ih LA

ST

cTA

TE

:=C

AT

A(C

EL

L):

K:=

dL

CC

K(L

AS

TS

TA

TE

) ;

Arr

>( $

PL

IT(K

I ,

LA

ST

ST

AT

!l ;

if “

LC

CK

SP

L I

T (K

)=O

TH

FY

AC

i)(

5PL

IT

f?L

r:C

KS

,K 1

;h

UM

lhS

PL

IT(K

1 :

=h

UC

IhS

”L

IT(K

)+l;

HL

CC

KS

PL

IT(K

) :

=l;

CE

LL

:=N

EX

T(C

EL

Lb

;E

NP

;S

TA

Tf

:=tE

FC

\EX

f(S

Tb

TE

);Er

JC ;

r,r

TC

FE

TU

Ph

;C

ILC

AE

:.ii

TA

TE

:=C

IjEh

FX

T(

J)

;h

l-IL

E S

TA

TE

-=0

DO

SE

GIh C

EL

L:=

PP

EV

CN

GN

E(

ST

AT

E-N

);kt-

ILE

CF

LL

1=9

CC

HE

C I

N LA

ST

ST

AT

E:=

CA

TA

(CE

LL

J;

K:=

HL

CC

K(t

AS

TS

TA

TE

) ;

AC

Z(S

PL

IT(t

otC

AS

TS

TA

TC

);I

F

EL

CC

KS

PL

IT(K

)=O

T

HE

N

AC

II(S

PL

ITB

LC

CK

S,K

~;

hU

YIN

SP

LIT

(K):

=h

U~

INS

~L

IT(K

)+~

;P

LC

CK

SP

LIT

(Kl:

=l;

CE

LL

:=N

EX

T(C

EL

L)

;E

hC

;S

TA

TE

:=

CN

Fh

EX

T(

ST

AT

E)

:E

F\C

;C

C~

CE

hT

~*n

*s4~

~~

4*~

~d

****

*++

**~

~~

**~

~~

~*~

~~

~~

~*~

~~

~~

**~

~~

~~

$~

~*~

*~*

*R

Ecfh

E

BL

CC

KS

*~$

1S

$a

9~

**4

****

~$

~~

~~

*~~

~**

~*~

~~

*~~

~~

~~

*~~

~~

*~~

~~

~*~

~~

~~

~**

~~

~R

fTU

Rh

:I

F

(Sf’

LIT

t?L

CC

KS

=C

l T

t-E

N G

U T

C

RE

PF

AT

;R

SP

LIT

:-C

AT

4(S

PL

ITB

LC

CK

S);

PE

?C

VF

F~

Cb

’(S

PL

ITP

LC

CK

SI;

~L

CC

K’P

LIT

(~S

PL

IT):

=C

;I

F

hU

NI~

~S

LO

CK

(B~

PL

IJ)=

NU

~I~

~S

PL

IT@

SP

LT

)

TtI

Eh

HE

CIh h

L~

IN’F

LIT

(BS

PL

IT):

=0

;h

t-IL

F

ISF

L!T

(SS

PL

IT)-

.=O

)

C13

RE

FfO

VF

:~K

~lY

~S

PC

IT~

R~

PL

!T~

I;r,C

TO

F:r

TL

Ry\

‘;

c

“2

33

-3

(32

?L

--

C’

2 3

5- -

i’?

30

--

CL37

-

-I;

23

8

3-

c2

3q

--

c2

4u

--

22

41

--

.)2L

L -

-:!

24.$

--

c744

--

22

45

-

-(

:2

46

--

32

47

4-

cc?4

3 -

-c

2 4

'3-

-c25:

--

?L

Cl

--

Cic

2 -

-d

17c

j

--

._ L

,C

L54

-4

I?/';5

--

c2rf

: 4-

X%

57 -

-czj;

j -

-'3

25

';

--fl

26C

i -

-S

ZC

l -

-2

26

2

--

Tzt3

-4

:2t4

-

3O

Lt5

--

c2t5

--

02

67

--

12

69

-

-~o

Lt.9

--

32 7

':3 -

CL

jl -

-

2IL

iZ -

-

t-2

73

-3

0274

--

CL7

5

?-

32

76

-

-2

27

7

--

CL

79 -

?!)

27c;

--

d/

F)

--

32

&l

3-

0/"

2 -

-)

2?

+

--

ZiG

/l

-3

c2e5

--

CL

56

3-

':')

Fl

--

C?

[>j

--

,!Lc;

'J -

3‘,!'

;,-

--

ST

~:~F

I:R

C

@LC

oL

w(V

EQ

S I

IXh

z l

P!O

L6S

100

>E

CF

MR

CR

1976

ci

12

: 1

7P

AG

F

6

c L b‘1

--

32

F2

--

02

93

--

ci9

4 -

-0

24

5 -

-c?

i=t

!-C

2S7

--

CL

58 -

-c299

--

n3cc

--

C3C

L -

-ll

3c2

--

@3C

3 4

-0

30

4 -

-03c5

--

03G

6 -

4c3

c7

-3

c3ce

-2

c3

c9

-1

I?,S

Z.?

3 S

EC

CN

CS

IN

CC

MP

ILA

TIC

h,

lCR

32

3Y

TE

S C

F C

CD

E GE

hE

RA

TE

C