friday, may 14, 2004 isys3015 analytical methods for is professionals school of it, the university...

Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals

School of IT, The University of Sydney 1

Factorial DesignsFactorial Designs

Week 9 Lecture 2



AgendaAgenda

• Basic factorial design concepts

• Main and interaction effect

• Factorial design in computer system performance analysis



What are factorial designsWhat are factorial designs

• Two or more independent variables are manipulated in a single experiment

• They are referred to as factors• The major purpose of the research is to

explore their effects jointly• Factorial design produce efficient

experiments, each observation supplies information about all of the factors



A simple exampleA simple example

• Investigate an education program with a variety of variations to find out the best combination– Amount of time receiving

instruction• 1 hour per week vs. 4 hour

per week– Settings

• In-class vs. pull out• 2 X 2 factorial design

– Number of numbers tells how many factors

– Number values tell how many levels

– The result of multiplying tells how many treatment groups that we have in a factorial design



Null outcomeNull outcome

• None of the treatment has any effect

• Main effect– is an outcome that is a

consistent difference between levels of a factor.

• Interaction effect– An interaction effect

exists when differences on one factor depend on the level you are on another factor.



Main effectsMain effects

• Main effect of time• Main effect of setting• Main effects on both



Interaction effectInteraction effect

• An interaction effect exists when differences on one factor depend on the level of another factor

• How do we know if there is an interaction in a factorial design?– Statistical analysis will report all main effects and

interactions.– If you can not talk about effect on one factor

without mentioning the other factor– Spot an interaction in the graphs – whenever there

are lines that are not parallel there is an interaction present!



Interaction effectInteraction effect

• Interaction as a difference in magnitude of response

• Interaction as a difference in direction of response



Factorial design variationsFactorial design variations

• A 2 X 3 example• study the effect of

different treatment combinations for cocaine abuse. – Factor 1: treatment

• psychotherapy • behavior modification

– Factor 2:• inpatient • day treatment• outpatient

– Dependent variable• severity of illness rating



Factorial designs in computer Factorial designs in computer system performance analysis system performance analysis

• Personal workstation design– Processor: 68000, Z80, 8086– Memory size: 512K 2M or 8M bytes– Number of disks: one, two or three– Workload: Secretarial, managerial or

scientific– User education: high school, college, post-

graduate level

• Dependent variable– Throughput, response time



2222 factorial design factorial design

• Two factors, each at two levels

• Example: workstation design– Factor 1: memory

size– Factor 2: cache size– DV: performance in

MIPS0

20

40

60

80

4M 8M

Memory size

Perf

orm

ance

in M

IPS

1K

2K

Cache size

Memory size

4M byte 8M byte

1K 15 45

2K 25 75



Quantify effectsQuantify effects

• We want to learn which factor contribute more to the performance.– Define two variable

– The regression model

Missizememoryif

Missizememoryifxa 161

41

Kissizecacheif

Kissizecacheifxb 21

11

baabbbaa xxqxqxqqy 0



Quantifying results (cont)Quantifying results (cont)

• Resolving those coefficients

• We get

• How do you read this?

abba

abba

abba

abba

qqqq

qqqq

qqqq

qqqq

0

0

0

0

75

25

45

15

baba xxxxy 5102040



Quantify effects by sign tableQuantify effects by sign table

• Sign table method

I A B AB y

1 -1 -1 1 15

1 1 -1 -1 45

1 -1 1 -1 25

1 1 1 1 75

160 80 40 20 Total

40 20 10 5 Total/4



22KK factorial design factorial design

• K factors, each at two level

• 2K experiments• 23 design example

– In designing a personal workstation, the three factors needed to be studied are: cache size, memory size and number of processors

Factor Level -1 Level 1

Memory size 4Mbytes 16Mbytes

Catch size 1Kbytes 2Kbytes

Number of processors

1 2

Cache size (Kbytes)

4 Mbytes 16 Mbytes

1 2 1 2

1 14 46 22 58

2 10 50 34 86



22kk factorial design with factorial design with replicationreplication

• r replications of 2k experiments– 2Kr observations– Allows estimation of experimental errors– 223 design example

• The memory-cache experiments were repeated three times each. The result is shown below

Cache size Memory size

4M 8M

1 K 15, 18, 12 45, 48,51

2K 25, 28, 19 75,75,81



Full and fractional factorial Full and fractional factorial design design

• Full factorial design– Study all combinations– Can find effect of all factors– May try 2K factorial design first

• Fractional (incomplete) factorial design– Leave some treatment groups empty– Less information– May not get all interactions– No problem if interaction is negligible



22k-pk-pFractional factorial design Fractional factorial design

• Large number of factors– Large number of experiments– Full factorial design too expensive– Use a fractional factorial design

• 2k-p design allows analyzing k factors with only 2k-pexperiments.– 2k-1 design requires only half as many

experiments– 2k-2 design requires only one quarter of the

experiments



Example: 2Example: 27-47-4 DesignDesign

Exp No. A B C C E F G

1 -1 -1 -1 1 1 1 -1

2 1 -1 -1 -1 -1 1 1

3 -1 1 -1 -1 1 -1 1

4 1 1 -1 1 -1 -1 -1

5 -1 -1 1 1 -1 -1 1

6 1 -1 1 -1 1 -1 -1

7 -1 1 1 -1 -1 1 -1

8 1 1 1 1 1 1 1

• Study 7 factors with only 8 experiments• When quantify the effects, just calculate the main

effects• Will be able to eliminate some factors in further study.



Quantify modelQuantify model

ggffeeddccbbaa xqxqxqxqxqxqxqqy 0

I A B C C E F G

1 -1 -1 -1 1 1 1 -1 20

1 1 -1 -1 -1 -1 1 1 35

1 -1 1 -1 -1 1 -1 1 7

1 1 1 -1 1 -1 -1 -1 42

1 -1 -1 1 1 -1 -1 1 36

1 1 -1 1 -1 1 -1 -1 50

1 -1 1 1 -1 -1 1 -1 45

1 1 1 1 1 1 1 1 82

317 101 35 109 43 1 47 3 Total

39.62 12.62 4.37 13.62 5.37 0.125 5.87 0.37 Total/8



Preparing the sign tablePreparing the sign table

• Choose k - p factors and prepare a complete sign table for a full factorial design with k-p factors

• Of the 2k-p –k +p -1 column on the right, choose p columns and mark them with the p factors that were not chosen in step 1.

friday, may 14, 2004 isys3015 analytical methods for is professionals school of it, the university...

Documents

interaction effect interaction

effect main effect

isys3015 analytical

professionals school

university of sydney

factorial design slide

factorial design variations

interaction present