modeling experiments
DESCRIPTION
Modeling ExperimentsTRANSCRIPT
© 2003 Business Process Improvements, LLC. All Rights Reserved.
Full Factorial Experiments
Process Certification 301
2Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Objectives
Introduce the concepts of Modeling Experiments2k Factorials
Use Minitab with Standard Order Designs to:ConstructAnalyzeInterpret
Learn how to Improve Standard ErrorReduce the Model
Develop a DOE Analysis Roadmap
3Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Where are We?
IdentifyAssignable
CausesIs
Cpk > 1.33?Evaluate Process
Improvements
Implement updated Control
Plan
N
Certify the Process
Prioritize Variables
Y
N Optimize / Model the Process
Implement Control Methods
Create / UpdateProcess FMEA
Create / UpdateBaseline
Control Plan
EvaluateMeasurement
System(s)
ImplementCorrective
Actions
GR&R <20% of
ET?
IsProcess InControl?
ImplementCorrective
Actions
ReviewProcess
Documentation
ReviewProcess
Documentation
Create / updateProcess Map
(KPOs, KPIs, KCs)
Select Process and Charter
Team
Collect & Analyze
Process Data
Select Process and Charter
TeamStart
A
N
Assess ProcessCapabilityB
Maintain the Certification
Go to Next Process
B
A
EstablishProcess Controls
1. Define
2. Measure
3. Assess
4. Improve
5. Implement
6. Certify
Y Y
4Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
What’s in Step 4?
IMPROVE the Process and verify
the gains made
IMPROVE the Process and verify
the gains made4IMPLEMENT a control plan
to hold the gainsIMPLEMENT a control plan
to hold the gains
CERTIFY, maintain and standardize the processCERTIFY, maintain and standardize the process
Purpose:Apply P-D-C-A strategies to improve the process by attacking those top priority problems identified in Step 3.
Plan: Plan a well-thought-out set of experiments, tests, and observations meant to address a well-defined problem.
Do: Do the things you planned! Collect the data.
Check: Analyze the results using statistically valid methods.
Act: Take appropriate action based on the results.
Tools:Multi-Vari Studies: Establishing the contribution of input variables to the overall process variability.
Hypothesis testing: Comparing means, variances, and proportions; goodness-of-fit; contingency tables; and others...
Experimentation (DOE): Factorial experiments, Response surface experiments, “steepest assent” strategies, and others...
Modeling: Regression analysis, Analysis of Variance, and others…
DEFINE the processcertification opportunities
DEFINE the processcertification opportunities1
MEASURE and baseline the process key characteristics
MEASURE and baseline the process key characteristics2ASSESS and analyze the
current state of the processASSESS and analyze the
current state of the process3
5
6
5Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Design Classification
Design Type Screening Design or Fractional Factorial
Full Factorial
Central Composite
Design
Factors Explored:
3 to 15 2 to 6 2 to 5
Effects Estimated:
Individual Individual, Interaction
Individual, Interaction, Curvature
Result: Identify
Important Factors
Understand System
Behavior
High Quality Prediction,
Optimization
6Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Experiments
When we discuss experiments, we are interested in the following:
Number of factors we wish to investigate • Represented by the letter ‘k’
Number of Levels we want to set each Factor to during the experiment
• Represented by the letter ‘n’
Number of runs we need to conduct to look at all combinations of the Factors at their respective Levels
• Represented by the relationship nk
– Note: this is valid for designs where each Factor has the same number of Levels
7Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
2k Factorials
A 2k factorial refers to k factors, each with 2 levels
A 22 factorial is also represented as a 2x2 factorial
This design has two factors with two levels and can be done in 2x2, or 4, runs
Likewise a 23 factorial includes 3 factors, each with two levels
This experiment can be done in 2x2x2, or 8, runs
8Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Construction
9Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Standard Order of 2k Designs
Designs are usually shown in standard orderThe low level of a factor is designated with a “-” or -1
The high level is designated with a “+” or +1
Air Pressure Solids Content-1 -1+1 -1-1 +1+1 +1
10Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
23 Design
Air Pressure Solids Content Nozzle Type-1 -1 -1+1 -1 -1-1 +1 -1+1 +1 -1-1 -1 +1+1 -1 +1-1 +1 +1+1 +1 +1
11Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Exercise
Create a 24 Factorial Design Matrix on paper
What are the minimum number of runs needed?
12Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Experiment Construction Roadmap
Create the Factorial Design
Select type of design
Select Number of Factors
Select design type
Enter names and levels of Factors
Set randomization criteria
Enter response column
13Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Creating a Factorial Design
Improve> Modeling> Create Factorial Design
Select type of designSelect type of design
Select number of factorsSelect number of factors
14Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Creating a Factorial Design
Select the Design ...Select the Design ...
15Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Creating Factorial Design
Enter Factor informationEnter Factor information
16Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Create Factorial Design
Deselect randomization for this exercise
Deselect randomization for this exercise
Design OptionsDesign Options
17Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
24 Full Factorial Design in Standard Order
18Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Exercise
Procedure:Use the following catapult Factors and their respective Levels
• Hinge Point: 1 and 3
• Stop Position: 2 and 4
• Ball Type: Ping Pong and Golf
Response:• Distance to first impact
Design a worksheet using Minitab in Standard Order
Conduct the experiment and load your data into Minitab
19Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Full Factorial DesignAnalysis
20Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Example Background
The coating thickness across a part is inconsistent
The problem is thought to be associated with the spraying of the material
The aim is to study the effect of 3 Factors on the Response – thickness variation across the product
Response values closer to zero (0) are desiredThis shows no difference across the part
21Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
23 Design: 3 Factors, 8 Runs
Assign Factors and Levels
Variable -1 Level +1 LevelAir Pressure 90 PSI 100 PSISolids Content 20% 30%Nozzle Type A B
22Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Design with Actual Levels
Run Air Pressure Solids Content Nozzle Type Coating Variation1 90 20% A2 100 20% A3 90 30% A4 100 30% A5 90 20% B6 100 20% B7 90 30% B8 100 30% B
23Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Response Values Added
Run Air Pressure Solids Content Nozzle TypeCoating Variation
1 90 20% A 4.302 100 20% A 3.813 90 30% A 2.734 100 30% A 4.005 90 20% B 5.706 100 20% B 4.217 90 30% B 4.708 100 30% B 4.33
24Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Design with Coded Levels
Run Air Pressure Solids Content Nozzle TypeCoating Variation
1 -1 -1 -1 4.302 +1 -1 -1 3.813 -1 +1 -1 2.734 +1 +1 -1 4.005 -1 -1 +1 5.706 +1 -1 +1 4.217 -1 +1 +1 4.708 +1 +1 +1 4.33
25Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Analysis Mathematics
We will learn how to:Calculate the effect of the variables (main effects)
Calculate coefficients• For variables and interactions
Interpret graphical output from Minitab
There are two ways of analyzing factorial designs:
Using main effects and using coefficients
Minitab does them both!
There are two ways of analyzing factorial designs:
Using main effects and using coefficients
Minitab does them both!
26Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Calculating Main Effects
Nozzle TypeCoating Variation
-1 4.30-1 3.81-1 2.73-1 4.00+1 5.70+1 4.21+1 4.70+1 4.33
The main effect, also called the average response, is defined as:(average of response at +1 level) - ( average of response at -1 level)
So the main effect of Nozzle Type is:So the main effect of Nozzle Type is:
4.74 - 3.71 = 1.03
(5.70 + 4.21 + 4.70 + 4.33)/4 = 4.74
For Nozzle Type, the average of the +1 level is:
For Nozzle Type, the average of the +1 level is:
(4.30 + 3.81 + 2.73 + 4.00)/4 = 3.71
And the average of the -1 level is:And the average of the -1 level is:
27Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Graphical Representation
The main effect for the Nozzle Type variable is 1.03
This means Coating Variation (Y) increases by 1.03 units when changing from the -1 level (Nozzle A) to the +1 level (Nozzle B)
The Main Effects Plot shows this
Nozzle Type
Mea
n of
Coa
ting
Var
iati
on
BA
4.8
4.6
4.4
4.2
4.0
3.8
3.6
Main Effects Plot (data means) for Coating Variation
28Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Other VariablesM
ean
of C
oati
ng V
aria
tion
10090
4.5
4.4
4.3
4.2
4.1
4.0
3.93020
Air Pressure Solids Content
Main Effects Plot (data means) for Coating Variation
29Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Exercise
Calculate Main Effects for your catapult Factors (Hinge Point, Stop Position, Ball Type)
Sketch the Main Effects Plots for each
Construct a list of the Factors in descending order of size of effect
Which Factor has the biggest effect?
Which Factor has the smallest effect?
Which Factors have statistically significant effects?
30Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Interactions
Sometimes there is a bigger effect when two Factors are changed at the same time than what we would expect from the effect of each Factor on their own
We call this effect an interactioninteraction
We can calculate the interactions from the design matrix by adding some extra columns
Notice that each column has a pattern that is different from all other columns
This is called orthogonality
What would we expect the correlation coefficients to be for all these columns?
31Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Design with Interactions
We construct the interaction columns by cross-multiplying the relevant Factors
We construct the interaction columns by cross-multiplying the relevant Factors
Example: for the A*B interaction column multiply the coded values for each Factor
RunAir Pressure
(A)Solids
Content (B)Nozzle Type
(C) A*B A*C B*C A*B*CCoating Variation
1 -1 -1 -1 1 1 1 -1 4.302 +1 -1 -1 -1 -1 1 1 3.813 -1 +1 -1 -1 1 -1 1 2.734 +1 +1 -1 1 -1 -1 -1 4.005 -1 -1 +1 1 -1 -1 1 5.706 +1 -1 +1 -1 1 -1 -1 4.217 -1 +1 +1 -1 -1 1 -1 4.708 +1 +1 +1 1 1 1 1 4.33
This is repeated for all columns and rows
(Minitab does this for us)
This is repeated for all columns and rows
(Minitab does this for us)
32Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
What is a Coefficient?
A coefficient is a number that indicates how much the Response (Y) changes for a change of 1 coded unit in the Factor (X)
The coefficient gives the slope of the line
The coefficient gives the slope of the line
Effe
ct
-1 +1
33Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
The Coefficients
We use the letter β to indicate our coefficientsA coefficient is the change in Y for a change of one coded unit in X
Variablesβ1 tells us how big is the effect of A individuallyβ2 tells us how big is the effect of B individuallyβ3 tells us how big is the effect of C individually
Interactionsβ12 tells us how big is the effect of changing A and B togetherβ13 tells us how big is the effect of changing A and C togetherβ23 tells us how big is the effect of changing B and C togetherβ123 tells us how big is the effect of changing A AND B AND C together
34Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Interpretation
Listed are the Factor names, the mathematical coefficient
symbols and their values
Minitab calculates these values for us
Listed are the Factor names, the mathematical coefficient
symbols and their values
Minitab calculates these values for us
Factor Coefficient ValueConstant β0 4.2225
Air Pressure (A) β1 -0.1350Solids Content (B) β2 -0.2825Nozzle Type (C) β3 0.5125
A*B β12 0.3600A*C β13 -0.3300B*C β23 0.0625
A*B*C β123 -0.0800
Which variable has the biggest effect?Which variable has the smallest effect?
Which variables have statistically significant effects?
35Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
DOE Analysis
36Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Experiment Analysis Roadmap
Collect Response Data
Determine Response(s) to be analyzed
Select terms to be investigated(Factors and 2nd Order Interactions for
Full FactorialsFactors Only for Fractional Factorials)
Interpret graphical outputPareto and Normal Plot of
Effects
Reduce the model
37Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Experiment Analysis
Improve> Modeling> Analyze Factorial Design
Open File: DOE.MPJ
Worksheet: Coating Variation
Open File: DOE.MPJDOE.MPJ
Worksheet: Coating VariationCoating Variation
Select the TermsSelect the Terms
38Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Select Graphs
Select Normal and Pareto Effects PlotsSelect Normal and Pareto Effects Plots
We will look at these first to decide which variables and
interactions appear important
We will look at these first to decide which variables and
interactions appear important
39Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Pareto Chart of Standardized Effects
We use a Pareto chart of the effects to compare the relative magnitude and the statistical significance of both main and interaction effects
MINITAB plots the effects in decreasing order of the absolute value of the standardized effects and draws a reference line on the chart
Any effect that extends past this reference line is considered tAny effect that extends past this reference line is considered to o be significantbe significant
By default, MINITAB uses an alpha-level of 0.05
Why isn’t anything
important?
Why isn’t anything
important?
Term
Effect
BC
ABC
A
B
AC
AB
C
3.53.02.52.01.51.00.50.0
3.190Factor NameA A ir PressureB Solids C ontentC Nozzle Ty pe
Pareto Chart of the Effects(response is Coating Variation, Alpha = .05)
Lenth's PSE = 0.8475
40Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Pareto Chart of Standardized Effects
Hypothesis TestH0 there are no significant effects
Ha there are some significant effects
The red line is a confidence limit based on the alpha level
Any effect to the right of the red line is considered significant
The calculation of the line is based on:The alpha level selected
The standard error in the experiments
Total number of runs in the experiment
The number of terms included in the model
We can change the number of terms in the model and the alpha level and thus change the position of the line
41Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Normal Probability Plot
MINITAB draws a line to indicate where the points would be expected to fall if there were no effect of any of the Factors and their interactions
Significant effects are larger and farther from the line than insignificant effects
By default, MINITAB uses an alpha-level of 0.05 and labels any effect that is significant
Effect
Perc
ent
210-1-2
99
95
90
80
7060504030
20
10
5
1
Factor NameA A ir PressureB Solids C ontentC Nozzle Ty pe
Effect TypeNot SignificantSignificant
Normal Probability Plot of the Effects(response is Coating Variation, Alpha = .05)
Lenth's PSE = 0.8475
Looks like nothing is important…
… but let’s look at the Analytical results
before we give up
Looks like nothing is important…
… but let’s look at the Analytical results
before we give up
42Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
ANOVA Table
Note there are no degrees of freedom for the error termWithout any DF for Error calculation, we can not get any P values!
We need to remove a term from the modelWe will remove the term with the smallest effect, the B*C interaction
Here are the coefficients calculated
earlier
Here are the coefficients calculated
earlier
Here are the effects
calculated earlier
Here are the effects
calculated earlier
43Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Model Reduction
Remove the smallest effect
Examine:Probability plots
Pareto plots ANOVA table
Repeat with next smallest effect
Continue until model is considered “the best”
Tips on “best model” selection:
Do not remove too many terms
Include one insignificant term to ensure that a significant one was not removed by mistake
Tips on “best model” selection:
Do not remove too many terms
Include one insignificant term to ensure that a significant one was not removed by mistake
Analyze Residuals and Fits for “best” model
44Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Reducing the Model
We remove the BC Interaction term firstWe remove the BC Interaction term first
Improve> Modeling> Analyze Factorial Design
45Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Pareto Chart of “Reduced” Model
Still no variables show to be significant
We can remove the A*B*C interaction term nextIt is unusual to have a third order interaction term that is statistically and practically significant
Term
Standardized Effect
ABC
A
B
AC
AB
C
14121086420
12.71Factor NameA A ir PressureB Solids C ontentC Nozzle Ty pe
Pareto Chart of the Standardized Effects(response is Coating Variation, Alpha = .05)
46Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
“Best” Reduced Model
3 terms remain statistically significantTwo interactions have been removed from the model
Notice that two factors appear to not be significant (A and B)
Notice that they are BOTH involved in interactions though!
Term
Standardized Effect
A
B
AC
AB
C
876543210
4.303Factor NameA A ir P ressureB Solids C ontentC Nozzle Ty pe
Pareto Chart of the Standardized Effects(response is Coating Variation, Alpha = .05)
47Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
ANOVA Table for “Best” Model
We use the p-values to determine whether the term is significant
Any < 0.05 we will consider as significant
48Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Exercise
Conduct the analysis for your catapult Factors (Hinge Point, Stop Position, Ball Type)
Reduce the model as necessary
Identify the significance of the Factors and Interactions
Which Factor has the biggest effect?
Which Factor has the smallest effect?
Which Factors and Interactions have statistically significant effects?
49Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Interpretation
50Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Analysis Interpretation Roadmap
Examine the ANOVA table
Examine plots:Interactions, Main Effects,
Cube, Variances, Residuals
Determine practical significance of terms (% contribution)
Finalize model
Finalize recommendations next steps
Conduct confirmation runs / validate the model
51Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Interpretation
There are many plots to help understand the results
Interaction Plots• Look at significant, highest order interactions first
• Do this before looking at the effects plots
Main Effects Plot• Look at significant factors
Cube Plots• Looks at multiple Factors at the same time
Residuals Plots• Examines the validity of the model
Equal Variance Plot• Tests the underlying assumptions in DOE
52Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Interaction Plots
Select Interaction Plot and SetupSelect Interaction Plot and Setup
Insert “Coating Variation”as the Response
Insert “Coating Variation”as the Response
We will look at all the interactions
We will look at all the interactions
Improve> Modeling> Factorial Plots
53Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Interaction Plots
A ir Pressure
3020 BA
4.8
4.0
3.2
Solids Content
4.8
4.0
3.2
Nozzle T ype
Air Pressure90
100
SolidsContent
2030
Interaction Plot (data means) for Coating Variation
Look for non-parallel and crossing lines
The plot helps identify interactions we want to look at further (individually) for setting factor levels
Look for non-parallel and crossing lines
The plot helps identify interactions we want to look at further (individually) for setting factor levels
54Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Interaction Plot Interpretation
This plot shows a strong interaction between Air Pressure and Solids Content
Notice that the level of Coating Variation “depends” on the levels of the two variables
To achieve the lowest average Coating Variation:Use low Air Pressure and high Solids Content
To achieve the lowest variation in Coating Variation:Use high Air Pressure (especially if Solids Content is hard to control)
Solids Content
Mea
n
3020
5.0
4.8
4.6
4.4
4.2
4.0
3.8
3.6
Air Pressure90
100
Interaction Plot (data means) for Coating Variation
55Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Interaction Plot Interpretation
This plot shows a strong interaction between Air Pressure and Nozzle Type
What levels of each variable should be chosen to achieve the lowest average Coating Variation?
What about to achieve the lowest variation in Coating Variation?
Nozzle Type
Mea
n
BA
5.25
5.00
4.75
4.50
4.25
4.00
3.75
3.50
Air Pressure90
100
Interaction Plot (data means) for Coating Variation
56Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Main Effects Plot
Select Main Effects Plot and Setup
Select Main Effects Plot and Setup
Improve> Modeling> Factorial Plots
We will plot all Factors
We will plot all Factors
Insert “Coating Variation” as the
response
Insert “Coating Variation” as the
response
57Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Main Effects Plot
Main Effects plots should be interpreted after all significant interactions plots have been reviewed
Notice the influence of Nozzle Type versus the other factors
Since we have already chosen levels for our three Factors from the interaction plots, we really don’t need the Main Effects plot
Main Effects plots should be interpreted after all significant interactions plots have been reviewed
Notice the influence of Nozzle Type versus the other factors
Since we have already chosen levels for our three Factors from the interaction plots, we really don’t need the Main Effects plot
Mea
n of
Coa
ting
Var
iati
on
10090
4.6
4.4
4.2
4.0
3.8
3020
BA
4.6
4.4
4.2
4.0
3.8
Air Pressure Solids Content
Nozzle Type
Main Effects Plot (data means) for Coating Variation
58Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Cube Plot
Select Cube Plot and SetupSelect Cube Plot and Setup
Insert “Coating Variation” as the
response
Insert “Coating Variation” as the
response
We will plot all Factors
We will plot all Factors
Improve> Modeling> Factorial Plots
59Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Cube Plot
B
A
30
2010090
Nozzle Type
Solids Content
Air Pressure
4.33
4.215.70
4.70
4.00
3.814.30
2.73
Cube Plot (data means) for Coating Variation
Cube plots are useful for identifying the combination of Factors that gives the highest or lowest average response
Cube plots are useful for identifying the combination of Factors that gives the highest or lowest average response
60Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Factor Level Selection
Based on the Interaction plots, the Main Effects plot, and the Cube Plot we can determine the best factor levels
In this example, we want lower Coating Variation
Therefore, our factor levels should be:Air Pressure: Low
Solids Content: High
Nozzle Type: A
We would expect the Coating Variation to be about 2.73 for this combination
Reference the Cube Plot
61Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Residual Plots
Improve> Modeling> Analyze Factorial Design
62Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Residual Plots
Residual
Per
cent
0.20.10.0-0.1-0.2
99
90
50
10
1
Fitted Value
Res
idua
l
6543
0.1
0.0
-0.1
Residual
Freq
uenc
y
0.150.100.050.00-0.05-0.10-0.15
4
3
2
1
0
Observation Order
Res
idua
l
87654321
0.1
0.0
-0.1
Normal Probability Plot of the Residuals Residuals Versus the Fitted Values
Histogram of the Residuals Residuals Versus the Order of the Data
Residual Plots for Coating Variation
We expect stability
We expect stability
We expect random
variation, no
patterns
We expect random
variation, no
patternsWe expect
normally distributed
error
We expect
normally distributed
error
63Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Test for Equal Variances
An underlying assumption is that the variances within factor levels (of the same factor) are equal
Minitab can not test all three factors at the same time, but it can handle two
Test the variances for Nozzle Type first, then for Air Pressure and Solids Content together
Improve> Test for Equal Variances
64Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Equal Variance Plots
Test for Air Pressure and Solids Content
p = 0.454
Variances are equal
Test for Air Pressure and Solids Content
p = 0.454
Variances are equal
Assumptions are met
Assumptions are met
Test for Nozzle Type
p = 0.986
Variances are equal
Test for Nozzle Type
p = 0.986
Variances are equal
Noz
zle
Typ
e
95% Bonferroni Confidence Intervals for StDevs
B
A
3.53.02.52.01.51.00.50.0
Noz
zle
Typ
e
Coating Variation
B
A
6.05.55.04.54.03.53.0
F-Test
0.946
Test Statistic 1.02P-Value 0.986
Levene's Test
Test Statistic 0.01P-Value
Test for Equal Variances for Coating Variation
95% Bonferroni Confidence Intervals for StDevs
Air Pressure Solids Content
100
90
30
20
30
20
200150100500
Bartlett's Test
Test Statistic 2.62P-Value 0.454
Test for Equal Variances for Coating Variation
65Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Exercise
Complete the Graphical Analysis of the catapult data
Are there any significant Interactions? How did you interpret the graph?
What level of each Factor would you choose if you are trying to increase Distance?
66Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Determining Practical Significance
Statistical significance of the terms / model is only one step – we need to determine the practical significance of each factor
We use the General Linear Model to calculate the SS for each term and the SSTotal for comparison
NOTE: we need to change the display into codedcodedunits first
Improve> Display Design
67Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
General Linear Model (GLM)
DOE> Practical Significance> General Linear Model
All terms from our model are included
Note that the Interaction is made by adding an ‘*’
between Factors
All terms from our model are included
Note that the Interaction is made by adding an ‘*’
between Factors
68Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
General Linear Model: ANOVA
We need the SS for each Term, and the total SS, to determine the % Contribution of each term
We can copy the data from the ANOVA table to new columns in the worksheet using the Alt-Left Mouse Drag / Copy method
69Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Practical Significance
% Contribution is just the proportion of the Total SS for each Factor and Interaction (source)
% Contribution is just the proportion of the Total SS for each Factor and Interaction (source)
70Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Practical Results
There are four statistically significant terms
However, Nozzle Type contributes 43% of all the observed variation in our experiment
What action would you take based on these results?
What about the other sources of variation?
71Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Mathematical Models
The coefficients are used to produce a mathematical model - an equation
The coefficients are used to produce a mathematical model - an equation
Factor Coefficient ValueConstant β0 4.2225
Air Pressure (A) β1 -0.1350Solids Content (B) β2 -0.2825Nozzle Type (C) β3 0.5125
A*B β12 0.3600A*C β13 -0.3300B*C β23 0.0625
A*B*C β123 -0.0800
The Full Equation
Y = β0 + β1A + β2B + β3C + β12AB + β13AC + β23BC + β123ABC + error
The Factors are in coded values
The Full Equation
Y = β0 + β1A + β2B + β3C + β12AB + β13AC + β23BC + β123ABC + error
The Factors are in coded values
72Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Reduced Model
Based on our Factor level selection, the predicted output is then:
Coating Variation (Y) = 4.2225 + (-0.1350)(-1) + (-0.2825)(1) + (0.5125)(-1) + (0.3600)(-1)(1) + (-0.3300)(-1)(-1) + error
Based on our Factor level selection, the predicted output is then:
Coating Variation (Y) = 4.2225 + (-0.1350)(-1) + (-0.2825)(1) + (0.5125)(-1) + (0.3600)(-1)(1) + (-0.3300)(-1)(-1) + error
Predicted Coating Variation = 2.7510 (+ error)
Predicted Coating Variation = 2.7510 (+ error)
The Reduced Model Equation
Y = β0 + β1A + β2B + β3C + β12AB + β13AC + error
The Factors are in coded values
The Reduced Model Equation
Y = β0 + β1A + β2B + β3C + β12AB + β13AC + error
The Factors are in coded values
Factor Coefficient ValueConstant β0 4.2225
Air Pressure (A) β1 -0.1350Solids Content (B) β2 -0.2825Nozzle Type (C) β3 0.5125
A*B β12 0.3600A*C β13 -0.3300
73Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Exercise
Using your catapult data, what Factor(s) or Interaction(s) are practically significant?
How much of the total variation is explained by these terms?
What is your final equation?
Calculate the expected maximum distance based on your equation
Take 8 shots at the ‘best’ settings and compare the results to the prediction
What statistical tool should you use here???
74Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Summary
Full factorial experiments provide the opportunity to look at all interactions of all Factors included
A predictive model (equation) can be developed based on the results
Practical significance of the terms indicates what variables to control / pay particular attention to first
Residuals analysis can identify missing variables if unusual patterns exist
75Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Experiment Construction Roadmap
Improve> Modeling > Create Factorial DesignCreate the Factorial Design
Select type of design
Select Number of Factors
Improve> Modeling > Create Factorial Design> DesignsSelect design type
Enter names and levels of Factors Improve> Modeling > Create Factorial Design> Factors
Set randomization criteria Improve> Modeling > Create Factorial Design> Options
Enter response column
76Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Experiment Analysis Roadmap
Collect Response Data
Determine Response(s) to be analyzed
Select terms to be investigated(Factors and 2nd Order Interactions for
Full FactorialsFactors Only for Fractional Factorials)
Improve> Modeling > Analyze Factorial Design> Terms
Interpret graphical outputPareto and Normal Plot of
EffectsImprove> Modeling > Analyze Factorial Design> Graphs
Reduce the model
77Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Model Reduction
Remove the smallest effect Improve> Modeling > Analyze Factorial Design> Terms
Examine:Probability plots
Pareto plots ANOVA table
Improve> Modeling > Analyze Factorial Design> Graphs
Repeat with next smallest effect
Continue until model is considered “the best”
Analyze Residuals and Fits for “best” model
Improve> Modeling > Analyze Factorial Design> Graphs
Tips on “best model” selection:
Do not remove too many terms
Include one insignificant term to ensure that a significant one was not removed by mistake
Tips on “best model” selection:
Do not remove too many terms
Include one insignificant term to ensure that a significant one was not removed by mistake
78Licensed for use within United Technologies Corporation© 2003 Business Process Improvements, LLC. All Rights Reserved. Module: Modeling Experiments Rev: 1
Analysis Interpretation Roadmap
Examine the ANOVA table Improve> Modeling > Analyze Factorial Design
Examine plots:Interactions, Main Effects,
Cube, Variances, ResidualsImprove> Modeling > Factorial Plots
Determine practical significance of terms (% contribution)
Improve> Practical Significance >General Linear Model
Finalize model
Finalize recommendations next steps
Conduct confirmation runs / validate the model