1
Systems Management
Control Concepts
“It is the mark of a truly intelligent person to be moved by statistics.”
George Bernard Shaw
2
R. A. Fisher
• Studied math and astronomy at Cambridge
• Conducted agricultural research in Britain
• Developed powerful methods of experimental design
Sir Ronald A. Fisher1890 - 1962
Lady Tasting Tea• At a summer tea party in Cambridge, England, a
lady states that tea poured into milk has a different taste than milk poured into tea
• Most guests, including distinguished scientists, think this nonsense, but Ronald Fisher proposes a method to scientifically test the lady's claim
• In Fisher’s experiment, the lady correctly identified every cup!
Salsburg, D., The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century,” W.H. Freeman, 2001
A Cure for Scurvy• Scurvy once common among sailors
• Dr. James Lind, a Scottish surgeon in the British Royal Navy, designed experiments in 1754
– 12 scurvy patients on the HMS Salisbury selected
– Divided into six pairs and given remedies
• Cider, vitriol, seawater, garlic, vinegar, citrus
– The pair on oranges and lemons fit for duty in six days
• First controlled clinical trial
• Origin of “limey” as term for British sailors
Lind, J. , A Treatise of the Scurvy, 1753
5
Design of Experiments
• As we vary process parameters to reduce the loss function, it may be hard to determine what changes cause what effects
• Traditional method is to vary one parameter at a time– Inefficient– Provides no information about interactions
6
Example
• Goal: improve some quality characteristic
• Factors: temperature (T) and pressure (P)
• Approach: conduct a series of experiments– First, the classic one
factor at a time design
– Baseline plus 2 trials
Trial T P Result
1 L L 10
2 L H 30
3 H L 20
7
One Factor at a Time Design
• The “T-effect” increase result by 10
• The “P-effect” increase result by 20
• Each estimate based on two trials
• No estimate of the interaction effect
Trial T P Result
1 L L 10
2 L H 30
3 H L 20
8
Factorial Design
• The T-effect
(20+26)/2 – (10+30)/2 = 3• The P-effect
(30+26)/2 – (20+10)/2 = 13• Each estimate based on
four trials• Next, interaction effect
Trial T P R
1 L L 10
2 L H 30
3 H L 20
4 H H 26
9
Factorial DesignTxP Interaction Effect
• T-effect (P Low) 20 – 10 = 10
• T-effect (P High)26 – 30 = -4
• By convention TxP effect is half the difference
(10+4) / 2 = 7• Average of two T-effects
equal to main effect of T[10 + (-4)] / 2 = 3
Trial T P R
1 L L 10
2 L H 30
3 H L 20
4 H H 26
10
Factorial DesignInteraction Effect
0
5
10
15
20
25
30
35
T-Low T-High
P-Low P-High
T P R
1 L L 10
2 L H 30
3 H L 20
4 H H 26
Interaction: Effect of T changesdepending on level of P
Best result
11
Factorial DesignPxT Interaction Effect
• P-effect (T Low)
30 – 10 = 20
• P-effect (T High)
26 – 20 = 6
• Half the difference is the PxT effect (same as TxP)
(20-6) / 2 = 7
• Average of two P-effects equal to main effect of P
[20 + 6] / 2 = 13
Trial T P R
1 L L 10
2 L H 30
3 H L 20
4 H H 26
12
Factorial DesignLurking Variable
• Suppose Operator also affects the response
• T and O perfectly correlated
• Which variable caused the effect attributed to T?
• Control what you can
• Randomize trial order to break link with variables you can’t control (or don’t know about)
Trial T P R O
1 L L 10 1
2 L H 30 1
3 H L 20 2
4 H H 26 2
13
Factorial DesignSummary
• With only four trials, we get– Main effect of T– Main effect of P– 2-factor interaction effect
• Each estimate based on all four trails
• Estimating effects ANalysis Of Means (ANOM)
• Determining which effects are statistically significant ANalysis Of VAriances (ANOVA)
T P R
1 L L 10
2 L H 30
3 H L 20
4 H H 26
With good design, analysis can be simple
14
Factorial Design
• Same as earlier example
• Here with algebraic notation
• Recall that
T-effect = 3
P-effect = 13
T x P effect = 7
• Much easier to get these results with this notation
Trial T P TxP R
1 + 10
2 + 30
3 + 20
4 + + + 26
Calculating Effects in the 23 Design
A B C AB AC BC ABC
+ + + + + + +
+ + +
+ + +
+ + +
+ + +
+ + +
+ + +
+ + +
15
The Fractional Factorial
A B C AB AC BC ABC
+ + + + + + +
+ + +
+ + +
+ + +
16
• One-half fraction of a full 23 design: 23-1 design
• Note that effects B, AB and BC, ABC are confounded
• We always lose something with a fractional design
Fractional Factorial Designs
• As the number of factors increases, the runs required for a full factorial may outgrow the experimental resources available
• Fractional factorial designs are used for screening experiments: to determine which factors are truly significant
• Goal: distinguish main effects and 2-factor interactions
17
Alias structure for 26-2 designEffect Alias Alias Alias
A BCE CDF ABDEF
B ACE DEF ABCDF
C ABE ADF BCDEF
D ADF BEF ABCDE
E ABC BDF ACDEF
F ACD BDE ABCEF
AB CE BCDF ADEF
AC BE DF ABCDEF
AD CF BCDE ABEF
AE BC CDEF ABDF
AF CD BCEF ABDE
BD EF ACDE ABCF
BF DE ABCD ACEF 18
Design Resolution• Resolution III Designs (23-1)
– No main effects are aliased with any other main effect, but main effects are aliased with 2-factor interactions
• Resolution IV Designs (24-1)– No main effect is aliased with any other main effect or 2-
factor interaction, but 2-factor interactions are aliased with each other
• Resolution V Designs (25-1)– No main effect or 2-factor interaction is aliased with any
other main effect or 2-factor interaction
19
20
George Box
• Born in Britain
• Distinguished career at the University of Wisconsin
• Box, Hunter, and Hunter, Statistics for Experimenters
• Dr. John MacGregor one of Box’s Ph.D. students George E. P. Box
1919 -
21
Douglas Montgomery
• Ph.D. VPI, 1969
• G.T. faculty, 1969 - 84
• ASU faculty, 1988 – present
• ACQC Shewhart Medal, 1997
• Supervised Philip Coyle’s M.S. thesis, "An Adaptation of Bayesian Statistical Methods to the Determination of Optimal Sample Sizes for Operational Testing"
How to use statistical techniques
• Find out as much as you can about the problem• Get help from experts on the process• Define objectives• Don’t invest more than one quarter of the
experimental effort (budget) in a first design
• Box, G., Hunter, W., Hunter, S., Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building, John Wiley and Sons, 1978
22
How to use statistical techniques
• Use your non-statistical knowledge of the problem• Keep the design and analysis as simple as possible• Recognize the difference between practical and
statistical significance• Experiments are usually iterative
• Montgomery, D., Design and Analysis of Experiments, 2nd edition, John Wiley and Sons, 1984
23