doe evaluation presentation
TRANSCRIPT
-
7/27/2019 DOE Evaluation Presentation
1/29
Selecting Effective Designs
Presented by: Larry Scott
Process TechnologiesNorthville, Michigan
248-347-1522
Welcome to:
DOE Evaluation:
Agenda
Intro to DOEOverall Strategy of DOEDesign Evaluation: Focus onFraction of Design Space (FDS) Full Factorial Designs Fractional Designs
CCD, Box Benken, D-Optimal, MLC Simplex Designs (Formulations)
-
7/27/2019 DOE Evaluation Presentation
2/29
ProcessProcess
Controllable Factors (X)Controllable Factors (X)
Responses (Y)Responses (Y)
Uncontrollable Variables (Z)Uncontrollable Variables (Z)
What is DOE:What is DOE: 66--sigma enthusiast Y = f (xsigma enthusiast Y = f (x ii))
DOE is:
A series of tests,
in which purposeful changes
are made to input factors,
so that you may identify causes
for significant changes
in the output responses.
HistoryFisher & Yates: DOE concepts 1920s
Plackett & Burman designs 1940s
1st Textbook: Box, Hunter & Hunter - 1960
Optimal Designs: a- , d- 1960s
Douglas Montgomery resurgence 1976
DOE software commercially available 1980s
Six Sigma drives interest in DOE 1990sMinimum Run Designs; Whitcomb & Oley 1999
Fraction of Design Space (FDS) Introduced - 2007
-
7/27/2019 DOE Evaluation Presentation
3/29
Phase: Screening
Designs: resolution IV 2k-p
Phase: Characterization
Designs: 2k, resolution V 2k-p
Phase: Optimization
Designs : CCD, BB, etc.
Phase: Verification
Designs: resolution III 2k-p
yes
Strategy of Experimentation
Factor effectsand interactions
ResponseSurfacemethods
Curvature?
Confirm?
KnownFactors
UnknownFactors
Screening
Backup
Celebrate!
no
no
yes
Trivialmany
Vital few
Presentation Intent
Applying design evaluationstechniques that ensure effective designproperties for YOUR experiments.
Concept of Power: an under utilizedtool in DOE, until NOW!until NOW!
-
7/27/2019 DOE Evaluation Presentation
4/29
Input Requirements for Design
Signal estimates
Noise (variance) estimates
Resolution: Main Effects & Interactions
Resource Availability
# of Model Coefficients
Signal / Noise
The golf ball represents the signal: .
The grass represents the noise: .
/ = 1/2
p = 8.6 %/ = 1
p = 19.5 %/ = 2
p = 57.2 %
-
7/27/2019 DOE Evaluation Presentation
5/29
System Signal
is reflected in the magnitude of the output.
is a function of the range or spread
of the input variables.
X1: Temperature - Low = 50 deg C
High = 100 deg C
X2: Pressure - Low = 10 psiHigh = 15 psi
Input Requirements for Design
Signal estimates
Noise (variance) estimates
Resolution: Main Effects & Interactions
Resource Availability
# of Model Coefficients
-
7/27/2019 DOE Evaluation Presentation
6/29
-
7/27/2019 DOE Evaluation Presentation
7/29
Selecting # of Design Points
Given how many factors (k) you want tostudy and the number of coefficients (p) inthe model you select, the design will bebuilt as follows:
Model: estimation of all coefficients.
Lack-of-Fit: test how well modelrepresents actual behavior.
Replicates: estimate pure error.
Why these inputs ?????
POWERPOWER !!!!!The ability to find a factor effect!
-
7/27/2019 DOE Evaluation Presentation
8/29
Power Depends On:
The size of the difference y being measured.the larger the difference the higher the power.
The size of the experimental error :the smaller the higher the power.
The risk chosen:the larger the higher the power.
The number of replicates:the more runs the higher the power.
Choose design appropriate to the problem:more orthogonal & larger designs have more power.
Agenda
Intro to DOE
Overall Strategy of DOE
Power using Fraction of Design Space
Full Factorial Designs: traditional (%)
Fractional Designs: FDS
CCD, Box Benken, D-Optimal, MLC
Simplex Designs (Formulations): FDS
-
7/27/2019 DOE Evaluation Presentation
9/29
Good Response Surface Designs
1. Allows chosen polynomial to be estimatedwell.
2. Sufficient information to test for lack of fit.
Have more unique design points thancoefficients in model.
Replicates to estimate pure error.
3. Remain insensitive to outliers, influential
values and bias from modelmisspecification.
4. Be robust to errors in the factor levels.
Good Response Surface Designs
5. Permit blocking and sequentialexperimentation.
6. Provide a check on varianceassumptions, e.g., test that residuals areN(0, 2).
7. Generate useful information throughout
the region of interest, i.e., provide aconstant distribution of variance acrossdesign space.
8. Do not contain an excessively largenumber of runs.
-
7/27/2019 DOE Evaluation Presentation
10/29
RSM vs OFATOFAT
-2 -1 0 1 2
30
45
60
75
90
Factor A
Response
-2 -1 0 1 2
60
65
70
75
80
85
90
Factor B
Response
Response
65
73
80
88
95
Response
-4-2
02
4
-4
-2
0
2
4
Factor A
Factor B
Response
LCD Case Study
The objective is to find the best
color/typeface combination to maximize
readability on a LCD video display terminal.
Response: Time (seconds)
-
7/27/2019 DOE Evaluation Presentation
11/29
1. Identify opportunity and define objective.Find best color/typeface combination tomaximize readability.
2. State objective in terms of measurableresponses.Response = Time in seconds.
a. Define the change (y) that is important todetect for each response. = 1 second
b. Estimate experimental error () for eachresponse. = 1 second.
c. Use the signal to noise ratio (//// = 1.0)to estimate power.
DOE Process: LCD Case Study
3. Enter the factor names, levels and units.
continue>>
Building the DesignLCD Case Study
-
7/27/2019 DOE Evaluation Presentation
12/29
4. Enter the response name, units, y (1), (1)and a / of 1 is calculated.
continue>>
Building the DesignLCD Case Study
Evaluating Power: LCD Case Study
Reading Time: = 1 sec, = 1 sec, //// = 1.0
Want power of at least 80% for effects ofinterest!
-
7/27/2019 DOE Evaluation Presentation
13/29
Power Depends On:
The size of the difference y being measured.the larger the difference the higher the power.
The size of the experimental error :the smaller the higher the power.
The risk chosen:the larger the higher the power.
The number of replicates:the more runs the higher the power.
Choose design appropriate to the problem:more orthogonal & larger designs have more power.
4.Build a 23 factorial with 2 replicates or 16runs.
Replicate to Increase Power:
-
7/27/2019 DOE Evaluation Presentation
14/29
5. Build a 23 factorial with 5 replicates or 40runs.
Replicate to Increase Power
However, As Collinearity Increases
Calculating power becomes a less
effective tool.. So..
We rely on a new tool..
Fraction of Design Space (FDS)
Plus:
StdErr: Prediction error for the design.
-
7/27/2019 DOE Evaluation Presentation
15/29
Stent Delivery System
A stent is a wire mesh tube used to prop open an
artery recently cleared using angioplasty. The stent
is collapsed to a small diameter over a balloon
catheter. It's then moved into the area of the
blockage.
When the balloon is inflated, the stent expands,
locks in place and forms a scaffold. This holds the
artery open. The stent stays in the arterypermanently, holding it open to improve blood flow
to the heart muscle.
Stent Delivery SystemThis case study is meant to illustrate typical DOE
use in research to develop an improved product; in
this case a stent delivery system. Typical factors
include:
Lengths and diameters of various components,
e.g. tip, balloon, catheter, etc.
Materials used for the components.
Assembly parameters, e.g. weld locations, howthe balloon is folded, etc.
Stent geometry, wall thickness, how it is
crimped on the balloon, etc.
-
7/27/2019 DOE Evaluation Presentation
16/29
Stent Delivery SystemMR-5 Factorial Design
Twelve factors (11 numeric and 1 categoric)were studied in a factorial design. Afteranalysis of the design:
A CCD design with seven factors wasrun.
The seven factors and the region of interest are:
(The actual factor names and levels are proprietary.)
Stent Delivery SystemMR-5 CCD Design
Factor Type Low Level()
High Level(+)
A numeric 1 +1
B numeric 1 +1
C numeric 1 +1
D numeric 1 +1
E numeric 1 +1
F numeric 1 +1
G numeric 1 +1
-
7/27/2019 DOE Evaluation Presentation
17/29
Stent Delivery SystemMR-5 CCD Design
Possible design choices:
27 CCD (143 builds and 152 runs)
27-1 CCD (79 builds and 88 runs)
MR-5 CCD (47 builds and 50 runs)
Small CCD (37 builds and 41 runs)
BB (57 builds and 62 runs)
Stent Delivery System
MR-5 CCD Design
Use Design-Expert to build a 7-factor MR-5 CCDdesign.
-
7/27/2019 DOE Evaluation Presentation
18/29
MR-5 DesignsProvide Considerable Savings
46610243010625614
3261024259225613232512218025612
212512206812811
192512195612810
17251218461289
1542561738648
1382561630647
1222561522326
MR52k-pkMR52k-pk
Check Size of Design
Confirm that the design has enough runs togive the results needed.
! "
This tool bar allows the userto estimate StdErr Mean for aspecific design based onthese inputs:
-
7/27/2019 DOE Evaluation Presentation
19/29
Check Size of Design
80% of the design space will be able to estimate theresponse with the needed precision.
# $
%&
0.00 0.25 0.50 0.75 1.00
0.000
0.475
0.950
1.425
1.899
#
Design-Expert Software
Min StdErr Mean: 0.302Max StdErr Mean: 1.899Cuboidalradius = 1Points = 10000t(0.05/2,14) = 2.14479Reference X = 0.80Reference Y = 0.619
FDS Graph
Fraction of Design Space
StdErrMea
0.00 0.25 0.50 0.75 1.00
0.000
0.475
0.950
1.425
1.899
Software Screenshot
# $
%&
-
7/27/2019 DOE Evaluation Presentation
20/29
Remember, As Collinearity Increases
Power calculations becomes a less
effective tool..
So, in Simplex designs we rely
extensively on ..
Fraction of Design Space (FDS)
calculations.
Simplex-Lattice Designs
X = 11
X = 13X = 12
{3, 3} Simplex Lattice {3,3} Simplex Latticeaugmented & 4 reps
,
X = 11
X = 13X = 12
-
7/27/2019 DOE Evaluation Presentation
21/29
Compare to Orthogonal Designs:
2
3
factorial with 3 level factorialcenter point
Design Evaluation
Two Simplex-Lattice Designs
X = 11
X = 13X = 12
{3, 3} Simplex Lattice {3,3} Simplex Latticeaugmented & 4 reps
,
X = 11
X = 13X = 12
-
7/27/2019 DOE Evaluation Presentation
22/29
Design Matrix Evaluation for MixtureSpecial Cubic Model
No aliases found
Degrees of Freedom for Evaluation
Model 6
Residuals 3
Lack 0f Fit 3
Pure Error 0
Corr Total 9
Simplex-LatticeWithout Augmentation or Replicates
&
Design Matrix Evaluation for MixtureSpecial Cubic Model
No aliases found
Degrees of Freedom for Evaluation
Model 6
Residuals 10Lack 0f Fit 6
Pure Error 4
Corr Total 16
Simplex-Lattice
With Augmentation and 4 Replicates
'
-
7/27/2019 DOE Evaluation Presentation
23/29
Term 0.5 Std. Dev. 1 Std. Dev. 2 Std. Dev.
A 6.2 % 9.8 % 23.8 %
B 6.2 % 9.8 % 23.8 %
C 6.2 % 9.8 % 23.8 %
AB 6.2 % 10.0 % 24.5 %
AC 6.2 % 10.0 % 24.5 %
BC 6.2 % 10.0 % 24.5 %
ABC 6.0 % 9.2 % 21.6 %
Simplex-LatticeWithout Augmentation or Replicates
( ! ) * $+
A 8.6 % 19.9 % 60.1 %
B 8.6 % 19.9 % 60.1 %
C 8.6 % 19.9 % 60.1 %
AB 7.6 % 15.6 % 46.7 %
AC 7.6 % 15.6 % 46.7 %
BC 7.6 % 15.6 % 46.7 %
ABC 7.8 % 16.5 % 49.6 %
Simplex-Lattice
With Augmentation and 4 Replicates
( ! ) * $+
, ) ) - )
-
7/27/2019 DOE Evaluation Presentation
24/29
Fraction of Design Space
FDS:
Calculates the volume of the design spacehaving a standard error (StdErr) less than orequal to a specified value.
The ratio of this volume to the total volume ofthe design volume is the fraction of designspace.
Produces a single plot showing the cumulative
fraction of the design space on the x-axis(from zero to one) versus the StdErr on the y-axis.
Simplex-Lattice
Without Augmentation or Replicates
A: A1.00
B: B
1.00
C: C
1.00
0.00 0.00
0.00
0.63 0.63
0.63
0.780.78
0.78
0.78
0.78 0.78
0.78
StdErrMean
0.00 0.25 0.50 0.75 1.00
0.00
0.25
0.50
0.75
1.00
0.63
0.78
Fraction of Design Space% $
-
7/27/2019 DOE Evaluation Presentation
25/29
Simplex-LatticeWith Augmentation and 4 Replicates
A: A1.00
B: B1.00
C: C1.00
0.00 0.00
0.00
StdErr of Design
0.470.47
0.47
0.570.57
0.57
0.57 0.57
0.57
0.572
2
2 2
Fraction of Design Space
StdErrMean
0.00 0.25 0.50 0.75 1.00
0.00
0.25
0.50
0.75
1.00
0.47
0.57
% $
Simplex-Lattice: 3DWithout With
A (1.000)
B (0.000)
C (1.000)C (0.000)
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
StdErrofDesign
A (0.000)
B (1.000)
StdErr
A (1.000)
B (0.000)
C (1.000)C (0.000)
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
StdErrofDesign
A (0.000)
B (1.000)
StdErr
-
7/27/2019 DOE Evaluation Presentation
26/29
Evaluating Mixture Designs
All mixture DOEs have collinearity due to the equalityconstraint, ie. proportions.. X1 + X2 + X3 = 100%
As other constraints are added collinearity worsens.The fall out is:
Power depend on orthogonality and thereforelooses value as collinearity increases.
Standard error plots and FDS are better design
evaluation tools in the presence of collinearity.
A : T E A - L S
28 .00
B : C o c a m i d e9 .00
C: Lauramide9 .00
1 .0 0 1 .0 0
20 .00
Height
140
150
160
160
160
170
176
22
22
22
22
22
P re d ic ti 1 7 6 .9 39 5 % L o 1 6 1 .9 29 5 % Hi 1 9 1 .9 4S E m e a 2 .8 0 65 4S E p re d 6 .8 8 8X 1 2 4 .4 7X 2 3 .4 9X 3 2 .0 4
Simplex: Simple Constraints
-
7/27/2019 DOE Evaluation Presentation
27/29
90
50
70
30
10
90
50
70
30
10
90
50
70
30
10
X1
X2
X3
1. 0.1 A
2. A 0.5
3.0
.1B
4.B
0.7
5. C 0.7
Simplex: Multi-component Constraints
90
50
70
30
10
90
50
70
30
10
90
50
70
30
10
X1
X2
X3
1
2
3
4
5
8
6
7
Additional Extreme Vertices
-
7/27/2019 DOE Evaluation Presentation
28/29
DOE Software
.(
-/
1) Readily generate many design options
2) Quick, powerful data analysis
3) Generate useful response surface models
4) Design evaluation include:
Power: percentage, contour & FDS plotsFDS plots
StdErr estimates for:
Mean optimization
Point prediction
Estimating differences
If you always do what you always did;
youll always get what you always got.
- wise but unknown philosopher
Old Habits Methods Die Hard
-
7/27/2019 DOE Evaluation Presentation
29/29
www.doetraining.com
Software: Design-Expertfrom Stat-Ease, Inc.
0)( ! 1