efficient computational algorithms for searching for good designs according to the generalized...

This article was downloaded by: [University of Windsor]On: 16 July 2014, At: 19:53Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,UK

American Journal ofMathematical and ManagementSciencesPublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/umms20

Efficient ComputationalAlgorithms for Searching forGood Designs According tothe Generalized MinimumAberration CriterionDebra Ingrama & Boxin Tangb

a Department of Computer Science and MathematicsArkansas State University, State University, AR72467, U.S.A.b Department of Mathematical Sciences TheUniversity of Memphis, Memphis, TN 38152, U.S.A.Published online: 14 Aug 2013.

To cite this article: Debra Ingram & Boxin Tang (2001) Efficient ComputationalAlgorithms for Searching for Good Designs According to the Generalized MinimumAberration Criterion, American Journal of Mathematical and Management Sciences,21:3-4, 325-344, DOI: 10.1080/01966324.2001.10737564

To link to this article: http://dx.doi.org/10.1080/01966324.2001.10737564

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all theinformation (the “Content”) contained in the publications on our platform.However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness,or suitability for any purpose of the Content. Any opinions and views

http://www.tandfonline.com/loi/umms20

http://www.tandfonline.com/action/showCitFormats?doi=10.1080/01966324.2001.10737564

http://dx.doi.org/10.1080/01966324.2001.10737564

expressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of theContent should not be relied upon and should be independently verified withprimary sources of information. Taylor and Francis shall not be liable for anylosses, actions, claims, proceedings, demands, costs, expenses, damages,and other liabilities whatsoever or howsoever caused arising directly orindirectly in connection with, in relation to or arising out of the use of theContent.

This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan,sub-licensing, systematic supply, or distribution in any form to anyone isexpressly forbidden. Terms & Conditions of access and use can be found athttp://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Uni

vers

ity o

f W

inds

or]

at 1

9:53

16

July

201

4

http://www.tandfonline.com/page/terms-and-conditions

AMERICAN JOURNAL OF MATHEMATICAL AND MANAGEMENT SCIENCES Copyright@ 2001 by American Sciences Press, Inc.

EFFICIENT COMPUTATIONAL ALGORITHMS FOR SEARCHING FOR GOOD DESIGNS ACCORDING TO THE

GENERALIZED MINIMUM ABERRATION CRITERION

Debra Ingram 1 and Boxin Tang2

1 Department of Computer Science and Mathematics Arkansas State University, State University, AR 72467, USA.

2 Department of Mathematical Sciences The University of Memphis, Memphis, TN 38152, USA.

SYNOPTIC ABSTRACT

Generalized minimum aberration was introduced by Deng and Tang ( 1999)

as a criterion for comparing and assessing the "goodness" of nonregular fractional

factorial designs. For small run sizes, generalized minimum aberration designs can be

obtained by a complete search of Hadamard matrices; however, for larger run sizes

a complete search is impractical. The main purpose of this paper is to study the use

of efficient computational algorithms to construct generalized minimum aberration

designs. Application to small run sizes shows that our algorithms perform almost as

well as complete search, but with much less computing time. This strongly suggests

that our algorithms should be successful in finding good designs for larger run sizes.

Key Words and Phrases: nonregular fractional factorial; Hadamard matrix;

generalized minimum aberration

2001, VOL. 21, NOS. 3 & 4, 325-344 0196-6324/01/030325-20 $25.00

325

Dow

nloa

ded

by [

Uni

vers

ity o

f W

inds

or]

at 1

9:53

16

July

201

4

326 DEBRA INGRAM & BOXIN TANG

I. INTRODUCTION.

We consider two-level factorial designs with the two levels denoted by+ I and

-1 (or simply+ and - ). The most commonly used two-level factorial designs are the

regular fractional factorials. These designs are determined by their defining relations,

which specify the relationship among the columns. In a regular design, any two

effects can be estimated independently of each other or are fully confounded. The

number of runs in a regular design must be a power of2 (e.g. , 4, 8, I 6, 32, 64, . .. ),

leaving large gaps in the available choices of run size.

A much broader class of orthogonal two-level factorial designs can be

constructed by choosing columns from Hadamard matrices. A Hadamard matrix of

order n, say M, is an orthogonal n x n matrix of± 1; that is, M 'M = n I where I is

the nth order identity matrix. Except for the trivial cases n = 1 and n = 2, every

Hadamard matrix has order n = 4t for some positive integer t, and a library of

Hadamard matrices of every order upton= 256 is available at N.J.A. Sloane's web

site, www. research. aft. coml-njas. The designs constructed from Hadamard matrices

allow orthogonal (i.e., uncorrelated) main effect estimates and are often called

orthogonal main effect plans. In contrast to the regular fractional factorials, designs

constructed from Hadamard matrices are nonregular in that they exhibit more

complex aliasing structures - ones that cannot be specified by defining relations- and

there exist effects that are neither orthogonal nor fully confounded. Hamada and Wu

( 1992) discussed several applications of nonregular designs, and they showed that the

complex aliasing structure of a nonregular design can be utilized to detect interaction

effects. Moreover, the number of runs in designs from Hadamard matrices can be a

power of two or a multiple offour, and this flexibility in run size provides a distinct

advantage over the regular designs.

Hadamard matrices can be normalized so that all the entries in the first column

equal + 1. Deleting this first column gives a Hadamard design, H , with n runs and

n - I columns. In the construction of a design of n runs and m factors, the factors are

assigned to a subset of m columns from then - 1 available so the resulting design has

Dow

nloa

ded

by [

Uni

vers

ity o

f W

inds

or]

at 1

9:53

16

July

201

4

SEARCHING FOR GENERALIZED MINIMUM ABERRATION DESIGNS 327

good properties according to some criterion. This paper examines the construction

of nonregular fractional factorial designs that possess good qualities according to the

generalized minimum aberration criterion. Introduced in Deng and Tang (1999),

generalized minimum aberration is a criterion by which the "goodness" of nonregular

and regular factorials may be assessed and compared in a systematic fashion. Designs

that are the best according to the generalized minimum aberration criterion are called

generalized minimum aberration designs. By complete search of all possible designs,

Deng, Li, and Tang (2000) and Deng and Tang (2001) produced a comprehensive

catalog of generalized minimum aberration designs of 12, 16, and 20 runs and a partial

catalog of designs of 24 runs (accommodating up to eight factors). However, a

complete search for larger designs is impossible or, at best, impractical. For example,

a complete search for a design of24 runs and 10 factors requires the comparison of

all (i~) = 1,144,066 possible designs. To further complicate the complete search,

there are 60 nonequivalent Hadamard matrices of order 24 from which designs of24

runs can be chosen, multiplying the number of possible designs by 60. The main

purpose of this work is to study the use of efficient computational algorithms that

search a Hadamard design for a subset of columns that provides a good design

according to the generalized minimum aberration criterion. These efficient algorithms

make it possible to find good designs when a complete search is not possible.

In Section 2, we review the generalized minimum aberration criterion, which

is a natural extension of the minimum aberration criterion proposed by Fries and

Hunter (1980). It is based on the confounding frequency vector, a concept that

generalizes the word length pattern for regular designs. The confounding frequency

vector ( CFV) of a design describes the confounding present in that design, beginning

with the confounding present between main effects and two-factor interactions,

followed by the confounding between two-factor interactions and other two-factor

interactions, and so on.

In Section 3, we define the column confounding frequency vector (CCFV).

For a column cJ' the confounding frequency vector of cJ' CCFV(c), describes the

Dow

nloa

ded

by [

Uni

vers

ity o

f W

inds

or]

at 1

9:53

16

July

201

4


extent of that column's involvement in subsets of three columns exhibiting serious

confounding, followed by its involvement in subsets of four columns exhibiting

serious confounding, etc. Using the column confounding frequency vector to classify

each column as "good" or "bad," we propose backward elimination and forward

selection as efficient algorithms for searching for generalized minimum aberration

designs. We also provide the user with our recommendations for maximizing the

potential of these algorithms for finding the best design. In Section 4, the designs

constructed from efficient algorithms are compared with those from complete search.

The algorithms perform almost as well as complete search, but with much less

computing time.

2. GENERALIZED MINIMUM ABERRATION

Nonregular designs exhibit more complex confounding structures than regular

designs (i.e., structures that allow partial confounding). The minimum aberration

criterion is not applicable to nonregular designs and there was no systematic criterion

for comparing different nonregular designs until recently. Deng and Tang (1999)

proposed generalized resolution and the generalized minimum aberration criterion for

this purpose.

For a set of k vectors, s = { v1, v2, • •. , v.}, define

where v,1

is the 11h component of vector vr When s is a subset of k design columns

with entries +and - , Jk(s) provides a measure of confounding among the k columns

ins. For a regular design of n runs, the only possible values of Jis) are 0 and n,

corresponding to orthogonality and full confounding, respectively . If Jb) = n, the

kcolumns ins give a word oflength kin the defining relation. F ornonregular designs

of n runs, J.(s) can be between 0 and n, and this corresponds to partial confounding.

Larger values of Jis) (closer ton) indicate more serious confounding among the k

Dow

nloa

ded

by [

Uni

vers

ity o

f W

inds

or]

at 1

9:53

16

July

201

4


columns ins, while smaller Jk(s) values indicate less serious confounding. The designs

studied in this paper are constructed from the columns of a Hadamard design and are

thus orthogonal, yielding J1 (s) = 0 for all subsets s containing one design column, and

J2(s) = 0 for all subsets s containing two design columns. Deng and Tang {1999,

2001) proved additional results about the possible values of Jis). First, Jis) will

always be a multiple of 4. Moreover, for a design of n runs, if n is a multiple of 8,

then the possible values of Jis) are [ n, n-8, .. . , 0]. If n is not a multiple of8, then the

possible values of Jis) depend on the value of k. When k = 3, 4, 7, 8, II, 12, ... ,

Jis) may take on values [n, n-8, ... , 4], and fork= 5, 6, 9, 10, 13, 14, ... , the

possible values are [n-4, n-12, ... , 0].

Two designs of the same size are compared by their confounding frequency

vectors, and the confounding frequency vector of a design is built up from its Jk(s)

values. As shown in Deng and Tang (2001), an abbreviated confounding frequency

vector, constructed only from theJ3(s), Jis), andJ5(s) values, requires less computing

time and can be used as a surrogate of the full confounding frequency vector. Let D1,

D2, and D3 denote the top three generalized minimum aberration designs of 20 runs

and 12 columns, as in Deng, Li, and Tang (2000). In each design, there are ( 1n =

220 different subsets of three columns, ( ~2) = 495 different subsets of four columns,

and ( 152) = 792 subsets of five columns. For a design of20 runs, the possible J3(s)

and Jis) values are [20, 12, 4] and the possible J5(s) values are [16, 8, 0]. Table I

gives the frequency of the different Jt(s) values (fork= 3, 4, 5) for each design. In

D 1, for example, none of the 220 subsets of three columns yieldJ3(s) = 20, while eight

subsets of three columns yieldJ3(s) = 12, and the remaining 212 subsets yieldJ3(s) =

4. None of the three designs in Table I have subsets producing J 3(s) = 20, and

therefore full confounding between main effects and two-factor interactions is not

present in any ofthese designs. It is the second column of Table I, #Jls) = 12, that

distinguishes D1 from D2 and D3• The best design is D1 since it contains only eight

subsets in which the most serious partial confounding between main effects and two

factor interactions is taking place, while D2 and D3 contain I 0 of these subsets.

Dow

nloa

ded

by [

Uni

vers

ity o

f W

inds

or]

at 1

9:53

16

July

201

4


Similarly, it is the fifth column of Table 1, #Jis) = 12, that distinguishes D2 and D3.

In terms of the confounding between main effects and two-factor interactions

exhibited in each design, D2 and D3 are equivalent. However, D3 contains more

subsets in which the most serious partial confounding between two-factor interactions

and other two-factor interactions is taking place, with 35 subsets yielding Jis) = 12,

compared to 33 for D2• We might note that here we assume main effects are more

important than two-factor interactions, two-factor interactions are more important

than three-factor interactions, and so on.

From the entries in Table I, we construct the (abbreviated) confounding

frequency vectors of each design. In general, the following notation will be used.

The abbreviated confounding frequency vector of design D is given by CFV(D) =

[CFV3(D) , CFV4(D), CFV5(D)] where CFViD) contains the frequencies of the

different Jk(s) values for design D, say f.lD),J.iD), etc. f. 1(D) is the number of k

subsets of D producing the largest Jb) value,fk2(D) is the number of k-subsets of D

producing the second largest Jis) value, and so forth. In particular, for the designs

in Table I, we have CFV(D1) = [(0, 8)3, (0, 39)4, (0, 240)s], CFV(D2) = [(0, I 0)3, (0,

33)4 , (0, 244)s], and CFV(D3) = [(0, I 0)3 , (0, 35)4 , (0, 242)s]. Notice the entries for

#Jis) = 4, #J4(s) = 4, and #J5(s) = 0 are not used in the confounding frequency vector

since these are determined by the previous entries in CFV3, CFV4, and CFV5,

respectively .

Table 1. Frequency of Jis) values for the top three generalized minimum aberration designs for n = 20, m = 12

#J3(s) #Jis) #J3(s) #Jis) #J4(s) #Jis) #Js(s) #Js(s) #Js(s)

II II II II II II II II II 20 12 4 20 12 4 16 8 0

D, 0 8 212 0 39 456 0 240 552

D2 0 10 210 0 33 462 0 244 548

D3 0 10 210 0 35 460 0 242 550

Dow

nloa

ded

by [

Uni

vers

ity o

f W

inds

or]

at 1

9:53

16

July

201

4


In comparing two designs using the generalized minimum aberration criterion,

we compare their confounding frequency vectors entry-by-entry moving from left to

right, in much the same way as the word length patterns of two regular designs are

compared in the minimum aberration criterion. Letfr(D1) andfr(D2) be the rth entries

of the confounding frequency vectors of two designs D 1 and D2• If r is the smallest

integer such thatfr(D1) ,. fr(D2) andfr(D1) < fr(D2), then D 1 has less generalized

aberration than D2 and is preferred. Of the three designs in Table I, it is easy to see

that D 1 has generalized minimum aberration.

3. EFFICIENT COMPUTATIONAL ALGORITHMS

FOR SEARCHING FOR

GENERALIZED MINIMUM ABERRATION DESIGNS

The confounding frequency vector of a design provides a measure of the

confounding present in the design, beginning with the confounding present between

main effects and two-factor interactions, followed by the confounding between two

factor interactions and other two-factor interactions, and so on. We extend this

notion and define the column confounding frequency vector.

We will use notation similar to that in Section 2. The column confounding

frequency vector of column c, is given by CCFV(c) = (CCFV3(c), CCFVic)) . The

first component, CCFVlc), lists the number of subsets containing c1

that yield each

Jls) value. These frequencies are denoted by i"J 1(c),/32(c),f13(c), and so on, where

h,(c) is the number of subsets containing c1

(and two other columns) that produce the

ith largest J3(s) value. For example,f-J1(c) is the number of subsets containing c, (and

two other columns) that produce the largest J3(s) value, f-J 2(c) is the number of

subsets containing c1 that produce the second largest Jls) value, andf-Jlc) is the

number of subsets containing c, that produce the third largest J3(s) value. Likewise,

CCFV4(c) lists the number of subsets containing c, that yield each J4(s) value, and

these frequencies are denoted in a similar fashion: !;,(c) is the number of subsets

containing c, (and three other columns) that produce the itll largest J4(s) value.

Dow

nloa

ded

by [

Uni

vers

ity o

f W

inds

or]

at 1

9:53

16

July

201

4


High Jls) values provide the worst scenario- serious confounding between

the main effects and the two-factor interactions present in subsets. Accordingly, it

makes sense to classify a column as "bad" when it is contained in a large number of

subsets of three columns that yield high J3(s) values, and "good" when it is contained

in only a small number of subsets that yield highJls) values. Then, using the notation

above, a column that produces a large_h 1 value is bad while a column that produces

a small_h1 value is good. Columns yielding identicalh1 values are then compared by

their_h2 values, and so on. Columns with identical entries throughout their CCFV3's

are equally bad (or equally good) in regards to their involvement in the confounding

occurring between main effects and two-factor interactions. The columns are then

compared by their CCFV4's, which quantify the extent of their involvement in subsets

exhibiting serious confounding between two-factor interactions and other two-factor

interactions, that is, subsets producing high J.(s) values.

Let H be a Hadamard design of n runs and n- I columns. In constructing a

design of n runs and m factors, we wish to choose a subset of m columns from then- I

available so the resulting design has generalized minimum aberration. Furthermore,

we require this subset of columns be chosen in a manner that is efficient and that

eliminates the necessity for a complete search. We now explain how the column

confounding frequency vector is used in the development of efficient algorithms for

searching for generalized minimum aberration designs.

3.1 BACKWARD ELIMINATION ALGORITHM

As usual, let H be a Hadamard design of n runs and n-1 columns. Start with

the full design D •. 1 consisting of all n-1 columns of H . At each iteration, the "worst"

column is eliminated from the current design D. Repeat until we reach the target

design Dm of m columns. The elimination rules are as follows: For each column din

D, calculate all the Jls) values for subsets s containing d and two other columns in

the current design D. Construct CCFV3(Q) = (h 1(Q) ,hiQ), ... )which lists the number

of subsets containing dthat yield the largestJ3(s) value, the second largestJ3(s) value,

Dow

nloa

ded

by [

Uni

vers

ity o

f W

inds

or]

at 1

9:53

16

July

201

4


etc. The CCFV4's will not be calculated unless necessary . For each column din the

current design, CCFVld) quantifies the extent of d's involvement in confounding

occurring between main effects and two-factor interactions. Let max!J 1 denote

max{h 1(d)} where the maximization is over all columns in the current design. If

exactly one column attains max!J 1, then delete it from the current design. In this case,

there is one column that can be identified as the "worst." In some cases, a column

that is uniquely the worst does not exist. Suppose more than one column attains

max!J 1• Then these columns are equally "bad" in terms of their involvement in the

most serious confounding among three columns, and are all candidates for

elimination. Shorten the list of candidates for elimination by comparing the next

entries of their CCFV3's. Supposed, and~ are columns identified as candidates for

elimination because they attain max!J1. If hl~) > h 2(d,) for all i * j, then eliminate

~ from the current design because it is the column that is uniquely the worst. Let

maxh2 =max {hld)} , where the maximization is over all columns that are candidates

for elimination. If more than one of the candidate columns attains max!J2, then a

unique column for elimination has not been found, but the list of candidates for

elimination is shortened to include only those columns that attain both malifJ 1 and

max!J2. Continue to shorten the list of candidates for elimination by comparing entries

in the CCFV3 until the column that is uniquely the worst has been found or there are

no more CCFV3 entries to compare. Suppose, after exhausting the CCFV3 entries, we

have still only identified a subset of columns tied for worst. These columns are

equally bad in regards to their involvement in the confounding occurring between

main effects and two-factor interactions. Therefore, they can be distinguished only

by their involvement in the confounding occurring between two-factor interactions

and other two-factor interactions and so proceed to compare their CCFV4 entries.

Although we may continue to calculate and compare entries of, say, CCFV5 and

CCFV6, this represents a significant increase in computing time. Consequently, if

comparing the CCFV4 entries still results in a subset of columns tied for worst, then

we choose a column at random from this subset for deletion.

Dow

nloa

ded

by [

Uni

vers

ity o

f W

inds

or]

at 1

9:53

16

July

201

4

334 DEBRA INGRAM & BOX!N TANG

Example 1. We apply the backward elimination algorithm to a Hadamard design of

order 24, and search for a design of24 runs and 12 columns. More specifically, we

search the columns ofthe Hadamard matrix labeled as had.24.40 in N.J.A. Sloane's

web site: www.research.att.com/-njas. We start with the full design of all 23

columns. At each iteration of the algorithm, we eliminate one column from the

current design according to the elimination rules described above. During the first

nine of the eleven iterations, a column that is uniquely the worst is identified without

having to calculate the CCFV4 entries at all. For example, the 18'h column is

eliminated during the first iteration, with CCFV3(d18) = (2, 2, 73, 154)3 = {h 1(d18),

/ 32(d18),i"Jld 18),i"Jid18)). In this case, no other column attained maxi"J 1 = 2.

Later iterations, however, require the comparison of the CCFV4 entries to

shorten the list of candidates for elimination. The 1Oth iteration results in the

identification of two columns, d6 and d1, as candidates for elimination based on their

equally bad CCFV3's . In an effort to single out one of them as the worst, the CCFV4

entries for these candidate columns are calculated and produce CCFV4(d6) =

CCFVid1) = (1 , 3, 107, 175)4 • The candidates for elimination have identical CCFV4

entries, and therefore a column (d7 in this instance) is eliminated at random from the

subset of two columns tied for worst. In the last iteration, two columns are again tied

for worst (d6 and d 13) based on their CCFV3's. This time, a column that is uniquely

the worst is identified using the CCFV4 entries, and a random pick is not called for.

Comparing the entries of CCFV4(d6) = (0, 3, 87, 130) and CCFVid13 ) = (0, 4, 83,

133), column d13 is chosen for elimination. The design constructed has abbreviated

confounding frequency vector [(0, 0, 83)3, (4, 3, 174)4, (0, 8, 276)5] and appears in

Table 2.

3.2 FORWARD SELECTION ALGORITHM

Let Hbe a Hadamard design of n runs and n-1 columns. Choose two columns

at random from H to begin the construction of design D. We call this the O'h iteration.

This random selection in the Oth iteration is appropriate since, due to the orthogonal

Dow

nloa

ded

by [

Uni

vers

ity o

f W

inds

or]

at 1

9:53

16

July

201

4


Table 2. Design of24 runs and 12 columns constructed

using backward elimination algorithm

d1 d4 dS d6 d 10 dll d14 d1 5 d19 d21 d 22 d 23 1 1 1 1 1 1 1 1 1 1 -1 - 1 -1 - 1 -1 -1 1 1 1 - 1 - 1 - 1 1 -1 1 -1 - 1 - 1 1 1 1 - 1 - 1 -1 -1 1 - 1 1 1 -1 -1 -1 - 1 1 1 -1 -1 - 1 -1 1 - 1 -1 -1 - 1 - 1 - 1 1 1 1

- 1 1 -1 - 1 - 1 - 1 1 1 - 1 - 1 - 1 1 - 1 1 - 1 - 1 1 - 1 - 1 - 1

-1 -1 1 - 1 1 - 1 - 1 1 1

- 1 -1 - 1 - 1 - 1 1 1 - 1 -1 - 1 1 - 1 1 1 - 1 1 -1 - 1

- 1 1 -1 -1 - 1 1 - 1

-1 - 1 1 1 - 1 - 1 - 1 1 - 1 - 1 1 1 -1 1 -1 -1 - 1 -1 1 - 1

1 - 1 1 -1 1 - 1 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 1 - 1 - 1 1 - 1

1 1 -1 -1 1 - 1 - 1 1 - 1 1 - 1 - 1 - 1 - 1 - 1 1 - 1 1 1 - 1 1 - 1 - 1 1 - 1 - 1 1 1 1 - 1 1 - 1 - 1 1 1 1 - 1 1 -1 1 1 -1 - 1 1 - 1 - 1 - 1 1 1 - 1 -1

- 1 - 1 1 -1 - 1 - 1 1 1 - 1 1 1 - 1 - 1 -1 1 1 - 1 - 1 1 1 - 1

-1 -1 -1 - 1 -1 1 - 1 -1 - 1 - 1 - 1

property of Hadamard designs, any two columns of Hare as "good" as any other two

columns of H. The columns in H that are not in the current design Dare denoted by

H I D, which may be referred to as D-complement. At each subsequent iteration, the

"best" column in H \ D is selected for entry into the current design. Thus at each

iteration, D gains one column and H \ D loses one column. Repeat until we reach the

target design Dm of m columns and n runs. The selection rules are as follows: For

each column h in H\ D, calculate all the J3(s) values for subsets s containing hand

two columns already in the current design D. From the Jls) values calculated for h,

construct CCFV3(h) = (h 1(h),f32(h), .. . ). For each column h in H \ D, CCFV3(h)

quantifies the confounding between main effects and two-factor interactions that h

would bring into the current design. Let rnin_t; 1 denote min{f; 1(h)}, where the

minimization is over all columns in H\ D. If exactly one column attains mil!f; 1, then

that column leaves H \ D and enters the current design. Suppose more than one

Dow

nloa

ded

by [

Uni

vers

ity o

f W

inds

or]

at 1

9:53

16

July

201

4


column attains min}; 1• Then these columns are equally "good" in terms of the amount

of serious confounding among three columns they bring into the current design, and

the columns are all candidates for entry. Suppose h, andh1

are columns inH\ Dthat

have been identified as candidates for entry because they attain min}; 1• If};z(h) <

};2(h,) for all i "'j, then h1

leaves H \ D and enters the current design. Let min};2 =

min {hz(h)}, where the minimization is over all columns that are candidates for entry.

If more than one candidate column attains min};2, then a unique column for entry has

not been identified, but the list of candidates for entry is shortened to include only

those that attain min}; 1 and mi!1h2• As in the backward elimination algorithm,

compare the rest of the CCFV3 entries, and, if necessary, the CCFV4 entries, to single

out the candidate column that is uniquely the best. If, after comparing the CCFV4

entries, we still have a subset of columns tied for best, then choose a column at

random from this subset to enter the current design.

Clearly, the selection rules for the forward selection algorithm resemble the

elimination rules for the backward elimination algorithm. One important distinction

is in the formation of the subsets on which the column confounding frequency vectors

are based. In backward elimination, the first component of the column confounding

frequency vector of~. CCFV,(~), is based on subsets that contain ~ and two other

columns in the current design, where ~ is itself a column in the current design. In

forward selection, the first component of the column confounding frequency vector

of h1

in H \ D is based on subsets that contain h1

and two columns already in the

current design D.

3.3 USER-SPECIFIED OPTIONS

Both algorithms include the following built-in options for the user to specify:

(1) the number of columns, m, in the target design, (2) the Hadamard matrix from

which columns are to be chosen (which in turn specifies the run size), and (3) the

number of searches to be performed. We will use the phrase "search matrix" to refer

to the Hadamard matrix from which design columns are to be chosen. Indeed, a

Dow

nloa

ded

by [

Uni

vers

ity o

f W

inds

or]

at 1

9:53

16

July

201

4


complication of any search method that chooses columns from a Hadamard matrix is

that for given order n, several nonequivalent Hadamard matrices exist. Furthermore,

the number of nonequivalent Hadamard matrices is known only for n :s; 28. There is

a unique (up to equivalence) Hadamard matrix of order 12. There are five

nonequivalent Hadamard matrices of order 16 (Hall 1961 ), three of order 20 (Hall

1965), 60 of order 24 (Kimura 1989), and 487 nonequivalent Hadamard matrices for

order 28 (Spence 1995). Deng, Li, and Tang (2000) and Deng and Tang (2001)

found the top ten generalized minimum aberration designs of 12, 16, and 20 runs for

all m, and up to m = 8 factors for 24 runs by a complete search of all the

nonequivalent Hadamard matrices of orders 12, 16, 20, and 24. As discussed earlier,

a complete search for larger designs is impractical or even impossible, and efficient

computational algorithms are indispensable in searching for larger designs.

In the 24-run case, it was determined that the generalized minimum aberration

designs of 24 runs and eight or fewer columns can always be found from the

Hadamard matrix of order 24 labeled as had.24.11 in N.J.A. Sloane' s web site:

www.research.att.com/-njas. In using our algorithms to search for generalized

minimum aberration designs of 24 runs (and more than eight columns), the user

should first search had.24.11 since this matrix provided the best designs during the

complete search. In addition, the user may certainly search any or all of the other 59

Hadamard matrices of order 24, and choose the best design obtained from these

searches.

There are no complete search results available for designs of 28 runs (or

higher), and it is not clear which of the 487 Hadamard matrices of order 28 might

contain the best designs. Most users will find it too time-consuming to search all487

matrices, even when using efficient algorithms. Searching a few of these (chosen at

random) will produce several designs (along with their confounding frequency

vectors) from which the best will be chosen. For designs of 32 or more runs, the

number of nonequivalent Hadamard matrices is unknown, and it is impossible to

consider more than the number of Hadamard matrices that can be given by

Dow

nloa

ded

by [

Uni

vers

ity o

f W

inds

or]

at 1

9:53

16

July

201

4


construction methods like those of Paley (1933) and Williamson (1944). Some

Hadamard matrices of order 32 and higher can be found in N.J .A. Sloane's web site,

where he gives at least one Hadamard matrix of every order up through 256.

There is a random component in each algorithm. In each iteration that a single

"worst" column (as in backward elimination) or a single "best" column (as in forward

selection) cannot be identified, a column is chosen randomly from those tied for worst

or those tied for best, respectively. What is more, the forward selection algorithm has

a random start. Therefore, any two searches using the same algorithm, and specifying

the same Hadamard matrix and number of columns, can give two different designs.

The user should take advantage of this situation by performing several searches on the

same search matrix during one execution of the algorithm. For example, the user may

specify the following options: (1) m = 10 columns, (2) search matrix= had.24.11 , (3)

number of searches = 15, and this will produce 15 good designs of 24 runs and 10

columns chosen from had.24.11 , along with their confounding frequency vectors.

From these 15 designs, the design with generalized minimum aberration is selected.

In Section 4, we will see that there are designs attained using forward

selection that are not attained using backward elimination, and vise versa. Evidently,

a good search strategy should include the use of both backward elimination and

forward selection algorithms. In addition, we should search more than one Hadamard

matrix when possible. For example, there are two Hadamard matrices of order 36

available in N.J.A. Sloane's library , labeled as had.36.pal2 and had.36.will. A good

strategy for finding a generalized minimum aberration design of 36 runs and 9

columns is to perform 15 searches using backward elimination and 15 searches using

forward selection on both had.36.pal2 and had.36.will. The final result is 60 good

designs, most likely not all the same, from which the best is chosen.

4. PERFORMANCE

In this section, we study the performance of our efficient algorithms. Our

main concern is to examine their performance as compared with complete search. If

Dow

nloa

ded

by [

Uni

vers

ity o

f W

inds

or]

at 1

9:53

16

July

201

4


the efficient algorithms perform well for those run sizes for which complete search

results are available, then it is reasonable to expect that they will perform well in

general. We also make some comparisons between the backward elimination and

forward selection algorithms themselves, and some useful observations are given.

4.1 EFFICIENT ALGORITHM PERFORMANCE SUMMARY

We examine the performance of the algorithms by making a direct comparison

of the designs obtained by the algorithms to those obtained by complete search. Our

algorithms are applied to each combination of run size, number of columns, and

search matrix for which complete search results are available (see Deng, Li, and Tang

(2000) and Deng and Tang (2001)). By "search matrix" we mean the Hadamard

design from which design columns are chosen. In Table 3, the combinations of run

size/column size/search matrix are listed along with the results of 50 searches using

backward elimination and 50 searches using forward selection. If these searches

result in a set of designs that includes the best design, a checkmark is placed under

"best" and "in the top two". If the result is a set of designs that includes the second

best design, but not the best design, then a checkmark is placed only under "in the top

two." The fact that 50 searches produced a set of 50 designs that include the best

design does not imply that 50 searches were required to obtain the best design. For

example, for 20 runs and 12 columns, the 50 searches by forward selection produced

12 best designs, 15 second best designs, and various other designs in the top ten.

It is evident from the data in Table 3 that searching for generalized minimum

aberration designs by efficient algorithm is very effective. In fact, each algorithm

found the best design at least 85% of the time, and found one of the top two designs

at least 91% of the time. Combining the designs obtained from backward elimination

and forward selection (as recommended in Section 3) yields a set of designs that

includes the best design 96% of the time and includes one of the top two designs

I 00% of the time. Based on these results, we feel confident that search by efficient

algorithm will be successful in finding very good designs for larger run sizes.

Dow

nloa

ded

by [

Uni

vers

ity o

f W

inds

or]

at 1

9:53

16

July

201

4


Table 3. Search Results by Efficient Algorithm

search Backward Elimination Forward Selection # runs # columns matrix*

best in the top two best in the top two

12 5 had.l2 ./ ./ ./ ./

12 6 had.12 ./ ./ ./ ./

16 4 had.l6 .0 ./ ./ ./ ./

16 5 had.l6.0 ./ ./ ./

16 6 had. l6.0 ./ ./ ./

16 7 had. l6.0 ./ ./ ./

16 8 had.16.0 ./ ./ ./

16 9 had.l6 .0 ./ ./ ./ ./

16 10 had.l6.0 ./ · ./ ./ ./

16 II had.l6.0 ./ ./ ./ ./

16 12 had. l6.0 ./ ./ ./ ./

16 13 had. l6.0 ./ ./ ./ ./

16 14 had.l6 .0 ./ ./ ./ ./

16 9 had. l6.2 ./ ./

16 10 had. l6.2 ./ ./ ./ ./

16 II had.l6.2 ./ ./ ./ ./

16 12 had. l6.2 ./ ./ ./ ./

16 13 had.16.2 ./ ./ ./ ./

16 14 had. l6.2 ./ ./ ./ ./

16 II had.l6.4 ./ ./ ./ ./

16 12 had.l6.4 ,/ ,/ ,/ ,/

16 13 had. l6.4 ./ ./ ./ ./

16 14 had. l6.4 ./ ./ ./ ./

16 13 had. I6.3 ./ ./ ./ ./

16 14 had.16.3 ./ ,/ ./ ./

Dow

nloa

ded

by [

Uni

vers

ity o

f W

inds

or]

at 1

9:53

16

July

201

4


Table 3. Search Results by Efficient Algorithm (continued)

search Backward Elimination Forward Selection #runs #columns matrix•

best in the top two best in the top two

20 4 had.20.toncheviv ./ ./ ./ ./

20 5 had.20.toncheviv ./ ./ ./ ./

20 6 had.20.toncheviv ./ ./ ./ ./

20 7 had.20.toncheviv ./ ./ ./ ./

20 8 had.20.toncheviv ./ ./ ./ ./

20 9 had.20.toncheviv ./ ./ ./ ./

20 10 had.20.toncheviv ./ ./ ./ ./

20 11 had.20.toncheviv ./ ./ ./ ./

20 12 had.20.toncheviv ./ ./ ./ ./

20 13 had.20.toncheviv ./ ./ ./ ./

20 14 had.20.toncheviv ./ ./ ./ ./

20 15 had.20.toncheviv ./ ./ ./ ./

20 16 had.20.toncheviv ./ ./ ./ ./

20 9 had.20.julin ./ ./ ./ ./

20 10 had.20.julin ./ ./

20 II had.20.julin ./ ./ ./ ./

24 4 had.24.11 ./ ./ ./ ./

24 5 had.24.11 ./ ./

24 6 had.24.ll ./ ./

24 7 had.24.ll ./ ./

24 8 had.24.11 ./ ./

*The abbrevtatwns for the search matrtces are as they appear m N.J.A. Sloane's web stte:

www. research. att. com/-njas. The corresponding abbreviations used in Deng, Li, and Tang

(2000) and Deng and Tang (2001) are as follows . For n = 16, had.l6.0, had.l6.1, had.l6.2,

had. l6.3, and had.l6.4 are listed as Types I, II, III, IV, and V, respectively. For n = 20,

had.20.toncheviv and had.20.julin correspond to Types N and P. For n = 24, had.24.11 is

listed as Type II in Deng, Li, and Tang (2000).

Dow

nloa

ded

by [

Uni

vers

ity o

f W

inds

or]

at 1

9:53

16

July

201

4

342 DEBRA INGRAM & BOXJN TANG

4.2 BACKWARD ELIMINATION AND

FORWARD SELECTION -A COMPARlSON

Although our main concern is in comparing the search results of efficient

algorithm with those of complete search, some general observations can be made in

comparing the algorithms themselves.

There are designs that are attainable using forward selection but are not

attainable using backward elimination, and vise versa, as can be seen in Table 3. By

nature of the algorithms, some designs may be unattainable. For example, suppose

that the best 16 x 5 design cannot be embedded in the best 16 x 6 design. In using

backward elimination to search for a 16 x 5 design, if the second-to-last iteration

results in the best 16 x 6 design, then the best 16 x 5 design will not be found .

When there are few or no iterations that involve choosing a column at random

(for entry or elimination), the result of several searches may be the same design over

and over. Suppose our target design is 24 x 14 and we are searching the Hadamard

design had.24.40 as in Example I of Section 3. In this case, a column that is uniquely

the "worst" is identified in every iteration of the backward elimination algorithm, and

so every search by backward elimination will yield the same design. Using forward

selection, with its random start, should provide a wider variety of designs from which

to choose.

Actually, there is no evidence that backward elimination works better than

forward selection when the number of columns is greater than n I 2, and backward

elimination almost always uses more computing time. Supposing that the CCFV3's

distinguish the uniquely worst column at each iteration of backward elimination or the

uniquely best column at each iteration of forward selection, one may compare

computing times of the backward and forward algorithms by the minimum numbers

ofJ3(s) values that need to be calculated. Table4 shows the values ofm (the number

of columns) for which the minimum number of calculations required by forward

selection exceeds that required by backward elimination. For example, when

constructing designs of24 runs, backward elimination uses less computing time than

Dow

nloa

ded

by [

Uni

vers

ity o

f W

inds

or]

at 1

9:53

16

July

201

4


forward selection only when the number of columns is greater than 20. Clearly,

forward selection saves computing time except when m is very close to n.

Table 4. Values of n (#of runs) and m (#of columns) for which backward elimination uses less computing time than forward selection

#of #of #of #of #of #of #of #of runs columns runs columns runs columns runs columns

16 m ;d4 44 m ~ 40 72 m ~ 65 100 m ~ 91

20 m ~ 18 48 m ~ 43 76 m ~69 104 m ~ 94

24 m ~ 21 52 m ~ 47 80 m ~ 72 108 m ~ 98

28 m ~25 56 m ~ 51 84 m ~ 76 112 m ~ 101

32 m ~ 29 60 m ~ 54 88 m ~ so 116 m ~ 105

36 m ~ 32 64 m ~ 58 92 m ~ 83 120 m ~ 109

40 m ~ 36 68 m ~ 61 96 m ~ 87 124 m ~ 112

5. CONCLUDING REMARKS

We present two simple algorithms that are efficient and successful in finding

generalized minimum aberration designs. We are currently exploring improvements

to backward elimination and forward selection. The challenge is to make

improvements that do not add a significant amount of computing time.

ACKNOWLEDGEMENTS

The authors thank the referees and Dr. Daniel Voss for their constructive

comments. The research of Boxin Tang is supported by the National Science

Foundation DMS-9971212.

Dow

nloa

ded

by [

Uni

vers

ity o

f W

inds

or]

at 1

9:53

16

July

201

4


REFERENCES

Deng, L.Y., Li, Y., and Tang, B. (2000). "Catalogue of small runs nonregular designs from Hadamard matrices based on generalized minimum aberration criterion," Communications in Statistics- Theory and Methods, 29, 1379-1395

Deng, L.Y. and Tang, B. (1999). "Generalized resolution and minimum aberration criteria for P1ackett-Burrnan and other nonregular factorial designs," Statistica Sinica, 9, 1071-1082.

Deng, L.Y. and Tang, B. (2001). "Design selection and classification for Hadamard matrices using generalized minimum aberration criterion," Technometrics, to appear.

Fries, A. and Hunter, W.O. (1980). "Minimum aberration 2k·P designs," Technometrics, 22, 601-608.

Hall, M., Jr. (1961). "Hadamard matrix of order 16," Jet Propulsion Laboratory Research Summary, 1, 21-76.

Hall, M. Jr. (1965). "Hadamard matrix of order 20," Jet Propulsion Laboratory Technical Report, 1, 32-76.

Hamada, M. and Wu, C.F .J. ( 1992). "Analysis of designed experiments with complex aliasing," Journal of Quality Technology, 24, 130-137.

Kimura, H. (1989). "New Hadamard matrix of order 24," Graphs and Combinatorics, 5, 235-242.

Paley, R.E.A.C. (1933). "On orthogonal matrices," Journal of Mathematics and Physics, 12,311-320.

Sloane, N.J.A. A Library of Hadamard Matrices. {Online} Available http: //www.research.att.com/-njas/hadamard

Spence, E. (1995). "Classification of Hadamard matrices of order 24 and 28," Discrete Mathematics, 140, 185-243.

Williamson, J. (1944). "Hadamard's determinant theorem and the sum of four squares," Duke Mathematics Journal, 11, 65-81.

Dow

nloa

ded

by [

Uni

vers

ity o

f W

inds

or]

at 1

9:53

16

July

201

4

efficient computational algorithms for searching for good designs according to the generalized...

Documents