javier garcia - verdugo sanchez - six sigma training - w2 design of experiments (doe) intro

Introduction to DOE (D i f E i t)(Design of Experiment)

C t Pl t f A b t

55 58 61

1

Contour Plot of Ausbeute

64 67 70 73 76

0

Kat

al

76 79 82

1

160 170 180

-1

Temp

Hold values: Konz: 20,0

2k factorial designsWeek 2

Knorr-Bremse Group

2 factorial designs

About this Module

The Design of Experiment is one of the most effectiveThe Design of Experiment is one of the most effective tools in the DMAIC cycle.

You receive with low effort many information about processes, products and services. E.g. only 8 trials are p , p g ysufficient to determine the effects of three variables on one or more results (responses). The evaluation in ( p )Minitab is easy and user friendly.

The 2k factorial experiments give you an uncomplicated introduction to this technique. All further experimental designs are similar but in a modified form.

Knorr-Bremse Group 09 BB W2 DOE Intro 08, D. Szemkus/H. Winkler Page 2/78

Content of this Module

• Ways to learn

• Components of an experiment

• Experimental validation

St f l i i t• Steps for planning an experiment

• 2k factorial design2 factorial design

• Practical exercises

• Final report


DOE within the DMAIC Cycle

ControlMaintain Improvements

SPC

DefineProject charterSPC

Control PlansDocumentation

Project charter (SMART)

Business Score CardQFD + VOC

D QFD VOC

Strategic GoalsProject strategy

C M

MeasureBaseline AnalysisImprove

AIBaseline Analysis

Process MapC + E Matrix

Measurement SystemAnalyze

Definition of

pAdjustment to the

OptimumFMEA y

Process CapabilityDefinition of

critical InputsFMEA

S

FMEA

Statistical TestsSimulation

Statistical TestsMulti-Vari Studies

Regression

Tolerancing


Ways to Learn…

•• EmpiricEmpiric – Observation of natural, significant events (M lti i St di )(Multivari-Studies)

• If you are lucky, a significant informative event occurs while you are presentpresent

•• ExperimentalExperimental – Induce a informative event

• Manipulate Input-Variables in a way, that the effect on the Output-Manipulate Input Variables in a way, that the effect on the OutputVariables can be investigated

• Provoke the occurrence of an informative event

• Correctly performed experiments are:

• beneficial

• meaningful


What is an Experimental Design?

• A systematic series of runs where we manipulate directly various Input Variables (X’s) while we observe the effects on the Output Variables (Y’s).

• To determine which X’s have the strongest impact on the Yg p

• To determine how to set the influential X’s to center the Y on the targettarget

• To determine how to set influential X’s to minimize the variability of Y

• To determine how to set influential X’s to minimize the influence of Noise Variables

• A well planed experiment eliminates all possible causes which will have an effect on the Y, except those we want to test! If an effect

th Y th it di tl b i d t th i toccurs on the Y, then it can directly be assigned to the input variables (X’s) which are under investigation. This is possible because these input variables are independent of each other.


because these input variables are independent of each other.

The Process

Analyze the processControllable Inputs

X1 X2 X3

Controllable Inputs

Quality

LSL USL

Quality characteristics:

Outputs

The Process

Inputs:

Raw material, parts etc

Y1, Y2, …

parts etc.

Z1 Z2 Z3Not controllable Inputs


Not controllable Inputs

The Process

Improve the processControllable Inputs

X1 X2 X3

X

Controllable Inputs

QualityXLSL USL


Outputs

The ProcessXInputs:


Y1, Y2, …

parts etc.

X LSL USL

Z1 Z2 Z3

XNot controllable Inputs



The Process

Control the processControllable Inputs

X1 X2 X3

Controllable Inputs

Quality

LSL USL


Outputs

The Process

Inputs:


Y1, Y2, …

parts etc.

Z1 Z2 Z3Not controllable Inputs



The Benefits of Experimentation

• Process characterization

• Determination which X’s most strongly influence the YDetermination which X s most strongly influence the Y

• Includes controllable factors as well as noise factors (X’s)

• Identifies critical process variables (mean and variation)• Identifies critical process variables (mean and variation)

• Identifies variables which have to be controlled

• Gives procedures for controlling inputs instead of control charts for outputs

P ti i ti• Process optimization

• Determination of the setting of critical inputs

• Definition of appropriate specification limits

• Product designg

• Helps to understand the X’s early in the design phase

• Gives a guide line for robust designs


• Gives a guide line for robust designs

The Strategy of Experimentation

Collect information Fractional FactorialCollect information

Validate factors

A l b h i f

Mirror Plackett-Burnam

2k F t i lAnalyze behavior of important factors

E t bli h d l

2k FactorialCenter PointsBlockingEstablish a model

Determine optimized dj t t

BlockingFull Factorial

Box-Behnkenadjustments

RSM

Taguchi

EVOP

Taguchi


Knowledge and complexity define the type of experiment

The Strategy of Experimentation

Fractional factorial designs

Sort out uncritical factors Fold overSort out uncritical factors, Fold over

Plackett Burman Designs

2k factorial designs2 factorial designs

Center points

BlocksBlocks

Evaluate co variables

Full factorial designsg

RSM

Box Behnken

Evop

Taguchi

Mixed design


Knowledge and complexity define the type of experiment

Try and Error• The problem: The actual gas consumption of a car is 12l/100km.

We like to achieve 8l/100km.

• Some possible actions:

• Change the brand of the gasChange the brand of the gas

• Change the type of gas

• Drive slower

• Tune the car

• Wash and polish the car

• Assemble new tires

• Change the tire pressure

• What happens if it helps?


• What happens if it doesn’t help?

Sequential Experimentation “OFAT”

Problem: Fuel consumption of a car is about 12l/100km

Speed km/h Octane Tire pressure l /100 km105 91 2,1 11,5105 91 2 4 10 4105 91 2,4 10,4105 95 2,1 10,890 91 2,1 9,5

How may runs do we need to find out the settings of variables?

90 9 , 9,5

How may runs do we need to find out the settings of variables?

How do we explain the above results?

If there were more variables, how long would it take to get a optimized solution?

What if there is a combination of two or more variables that lead to the best fuel consumption?


Full Factorial Experiment

Problem: Fuel consumption of a car is about 12l/100km

Speed km/h Octane Tire pressure l / 100 km90 91 2,1 9,5

105 91 2,1 11,590 95 2,1 9,2

105 95 2,1 10,890 91 2,4 8,2

105 91 2,4 10,490 95 2,4 7,5

OFAT Runs

105 95 2,4 10,1

OFAT Runs


What conclusion do you draw now?

Components of Experiments• Output variables (responses)

• Current performance (Baseline)

• Measurement system capability

• Controllable input variables (factors)

• Variables

• Levels of setting

• Noise variables

• Narrow the validity

• Random order of the treatments

• Blocking

• Experimental Design

• Sample size

• Minimal size of the effect (delta)

• Alpha risk


• Beta risk (Power of the test)

Noise Factors

• Should be controlled what sometimes is difficult and not always possible for all factors

• Possibilities for control

• Use randomization (random experimental design)Use randomization (random experimental design)

• Try to hold the variables constant

• Blocking Make this factor part of o r e periment• Blocking: Make this factor part of your experiment

• Try to replicate your experiment

• Try to hold noise variables constant

• An experiment with one machine / line only

• Conduct the experiment on one day or one shift

• Use the same operators for entire experimentUse the same operators for entire experiment

H d h th d i fl l i ?


How does each method influences your conclusion?

Randomization• Randomize the experimental runs

• Randomize the allocation of treatments to samples and the runRandomize the allocation of treatments to samples and the run order in which the individual runs are performed

• Randomize the investigation of samples in the lab

Scenario:Scenario:

• You have just bought a new hunting gun

• You have four different types of ammunition

• Your are planning an experiment in order to find out which ammunition hits the target most accurate at a distance of 500 m

• Design an experiment

H d i th t i t?


How can you randomize that experiment?

Blocking

Include the noise factor into your experiment!

• A block is similar to rational subgroups as already mentioned in the g p ysections Capability Analysis and Statistical Process Control (SPC)

• The variability between the blocks should be larger than the• The variability between the blocks should be larger than the variability within a block

• Add day, shift or batch as a factor to your experiment

Minitab has the possibility to evaluate the effect of block factors. You receive additional and valuable information.

H thi th d i fl l i ?


How can this method influence your conclusion?

Repeats vs. Replicates• Repeats:

• Several samples with the same set up of experiment in a rapidSeveral samples with the same set up of experiment in a rapid succession

• Replicates:Replicates:

• Samples with the same set up of the experiment (same factor level settings or treatment) at different timessettings or treatment) at different times

• You may apply both methods to the same experiment.

• Both methods are directly linked to the sample size of the experiment.

• Conduct the hunting gun experiment firstly based on repeats than another experiment based on replicatesp p

What are the differences between your conclusion when you compare repetition with replication?


compare repetition with replication?

Validity Space

• The scopes within we draw conclusion based on the results

• Two scopes: tight or broadTwo scopes: tight or broad

• Tight scope:

• Experiment focused on a certain part of an overall process

• Example: Only one shift one operator one machine one batch etc• Example: Only one shift, one operator, one machine, one batch etc…

• Less influence of noise variables

• Broad scope:

• Investigates the overall process (all machines all operators several• Investigates the overall process (all machines, all operators, several batches, etc.). More data have to be collected over a longer time frame. Is more affected by noise variables

Usually tight experimental designs will be used to evaluate the noise variables Broad experimental designs will be used to forecast productivity


variables. Broad experimental designs will be used to forecast productivity

Validity Space

• Internal validity:

The limits within we draw conclusion based on the results

Internal validity:

• Do the input variables of the experiment really influence the output variables (response) or is a noise variable the cause?variables (response) or is a noise variable the cause?

• Aims at short time studies.

• External validity:

• May we transfer our experience to similar processes, production lines, other time periods etc.?

• Aims at long time studies.

• Validity of the statistical conclusion:


• Is there enough information for a valid statistical decision?

Risks for the Statistical Validity

• Little statistical power: Sample size to small• Little statistical power: Sample size to small

• Inaccurate measurement systems broaden the error• Inaccurate measurement systems broaden the error

• Happenstance data increase the variation of the• Happenstance data increase the variation of the measurements

• Randomization and appropriate sample size selection help to overcome these shortcomingshelp to overcome these shortcomings.


Standard Order of a 2k Experiment

The design matrix for 2k factorial experiments is usually displayed in standard order A “ ” or “ 1” is the notation for the low level of astandard order. A - or -1 is the notation for the low level of a factor, a “+” or “+1” is the notation for the high level.

2The example below shows the design matrix for a 22 factorial:

A 23 factorial looks like:Speed Octane

-1 -11 -11 -1-1 11 1

Speed Octane Tire pressure-1 -1 -11 -1 -1

Note: the 2² Factorial

-1 1 -11 1 -1-1 -1 11 1 1is embedded in the 23

Factorial

1 -1 1-1 1 11 1 1


Calculation of the EffectsNow we calculate the effects of the experiment.

We start with the factor speed. We sum the results (responses) of the notations “1” and “-1” and calculate the difference of the average values of “1” and “-1”.

Speed Octane Tire pressure l / 100 km-1 -1 -1 9,51 -1 -1 11,51 1 1 9 2-1 1 -1 9,21 1 -1 10,8-1 -1 1 8,21 -1 1 10,4-1 1 1 7,51 1 1 10,1

Effect 2,1 -0,5 -1,2

( ) ( )1,26,87,10

4

5,72,82,95,9

4

1,104,108,105,11=−=

+++−

+++=Speed

We state that there is an increase of average consumption of 2.1 units by

44


an increase of speed from 90 km/h to 105 km/h.

DOE Design Possibilities in MinitabStat

>DOE

>Factorial

>Create Fact. Design

>Display Avail. Design…

We want to use the designs in the GREEN GREEN - fields

The number of runs is noted on the left side, the number of factors on the top of the matrix


Exercise: Generate a matrix for 4 factors with Minitab (Full factorial)

Define a DOE Design in MinitabStat

>DOE

>Factorial

If the design has been not created by Minitab, the factors and their settings low and

high has to be defined first!>Factorial

>Define Custom Fact. Design

high has to be defined first!


Interactions

• First we have calculated the main effects of thisFirst we have calculated the main effects of this experiment. That means we have determined the main effects of speed, octane number and tire pressure.effects of speed, octane number and tire pressure.

• We are also interested in the interactions of theseWe are also interested in the interactions of these three factors. Is there a meaningful combination of input settings which impact the fuel consumption? p g p p

• Lets have a look on the 22 factorial experiment again p gand how we can determine interactions in a statistical way. Then we will come back to our example.


Effects of Interactions

• The effects of the interactions are calculated in the same way like the main effects. First we have to d t i th “l l ” (1 d 1) f th i t tidetermine the “levels” (1 and -1) of the interaction column.

• The “level” values of the interaction are the product of the involved factors.

• Using the 2x2 example we can determine the interactions of speed x octane by multiplication the

S d O t O t

interactions of speed x octane by multiplication the values of speed and octane.

Speed Octane v x Oct-1 -1 11 1 11 -1 -1-1 1 -11 1 1


1 1 1


• Each interaction has the same number of runs for each level like the single factors.

• The values for each factor and for each interaction are independent We call this “Orthogonal”independent. We call this Orthogonal .

• Exercise: If you enter this matrix into Minitab andExercise: If you enter this matrix into Minitab and correlate all the columns which each other what correlation coefficient results?

Speed (v) Octane (Oct) Tire pressure (p) v x Oct v x p Oct x p v x Oct x p l/100 km1 1 1 1 1 1 1 9 5-1 -1 -1 1 1 1 -1 9,51 -1 -1 -1 -1 1 1 11,5-1 1 -1 -1 1 -1 1 9,21 1 -1 1 -1 -1 -1 10,81 1 1 1 1 1 1 8 2-1 -1 1 1 -1 -1 1 8,21 -1 1 -1 1 -1 -1 10,4-1 1 1 -1 -1 1 -1 7,51 1 1 1 1 1 1 10,1



Speed (v) Octane (Oct) Tire pressure (p) v x Oct v x p Oct x p v x Oct x p l/100 km-1 -1 -1 1 1 1 -1 9,51 -1 -1 -1 -1 1 1 11,5-1 1 -1 -1 1 -1 1 9,21 1 -1 1 -1 -1 -1 10,8-1 -1 1 1 -1 -1 1 8,21 -1 1 -1 1 -1 -1 10,4-1 1 1 -1 -1 1 -1 7,51 1 1 1 1 1 1 10,1

Effect 2,1 -0,5 -1,2 0 0,3 0 0,2

The challenge is to figure out which of the effects are e c a e ge s to gu e out c o t e e ects a emeaningful (significant). Minitab helps us to select the factors and interactions which are significant. g

Precisely:

Which effect is significant and how large is the acceptable (α-risk) risk to make an error?


acceptab e (α s ) s to a e a e o

Evaluation in MinitabStat

>DOE

>Factorial

1

>Factorial

>Analyze Fact. Design…

32

3


1st Evaluation with all effects

Factorial Fit: Fuel Consumption versus Speed; Octane; Tire Pressure

Estimated Effects and Coefficients for Fuel Consumption (coded units)

Term Effect CoefConstant 9,6500S d 2 1000 1 0500 A

1,129

A SpeedFactor Name

Pareto Chart of the Effects(response is Fuel Consumption, Alpha = 0,05)

Speed 2,1000 1,0500Octane -0,5000 -0,2500Tire Pressure -1,2000 -0,6000Speed*Octane 0,0000 0,0000Speed*Tire Pressure 0 3000 0 1500

AC

B

C

A

Term

A SpeedB O ctaneC Tire Pressure

Speed*Tire Pressure 0,3000 0,1500Octane*Tire Pressure -0,0000 -0,0000Speed*Octane*Tire Pressure 0,2000 0,1000 BC

AB

ABC

2,01,51,00,50,0Effect

S = * PRESS = *

Analysis of Variance for Fuel Consumption (coded units)

Effect

Lenth's PSE = 0,3

The pareto shows that speedy p ( )

Source DF Seq SS Adj SS Adj MS F PMain Effects 3 12,2000 12,2000 4,06667 * *2-Way Interactions 3 0,1800 0,1800 0,06000 * *

The pareto shows that speed and tire pressure are

significant. In the next step d th d l ( ll t3-Way Interactions 1 0,0800 0,0800 0,08000 * *

Residual Error 0 * * *Total 7 12,4600

we reduce the model (all not significant effects will be removed, step by step)!


2nd Evaluation with all effects

Factorial Fit: Fuel Consumption versus Speed; Octane; Tire Pressure

S d

2,78

Pareto Chart of the Standardized Effects(response is Fuel Consumption, Alpha = 0,05)

Estimated Effects and Coefficients for Fuel Consumption (coded units) Tire Pressure

Speed

Term

Term Effect Coef SE Coef T PConstant 9,6500 0,09014 107,06 0,000Speed 2,1000 1,0500 0,09014 11,65 0,000Octane -0,5000 -0,2500 0,09014 -2,77 0,050

Octane

121086420Standardized Effect

Tire Pressure -1,2000 -0,6000 0,09014 -6,66 0,003

S = 0,254951 PRESS = 1,049 9 9 6 j 96 3

The session window shows now clearly that the 3 main

R-Sq = 97,91% R-Sq(pred) = 91,65% R-Sq(adj) = 96,35%

Analysis of Variance for Fuel Consumption (coded units)

yfactors are significant only

with P values ≤ 0,05. The R-Sq indicates that the variation

Source DF Seq SS Adj SS Adj MS F PMain Effects 3 12,2000 12,2000 4,06667 62,56 0,001Residual Error 4 0,2600 0,2600 0,06500Total 7 12 4600

Sq indicates that the variation in the fuel consumption is

caused by the change of the main factor setting withTotal 7 12,4600 main factor setting with

97.91%. The remaining 2% of variation are just by chance

( )


(random variation)

The graphical evaluationStat

>Quality Tools

>Multi Vari Chart>Multi-Vari Chart…

Multi-Vari Chart for Fuel Consumption by Octane - Speed

Here we can see in which range the fuel consumption

can be adjusted by the 12

2,42,1

90 1059195

Octane

j yfactor setting.

Caution: This is a simplified

11

10

on

sum

pti

on

Caution: This is a simplified experiment which gives us a linear relation. In reality we may investigate also

9

8

Fue

l Co

we may investigate also quadratic effects, e.g. for

the factor speed. 2,42,1

7

Tire Pressure

P l i bl S d


Panel variable: Speed

Example

The yield of a reactor depends on the settings of temperature, concentration and presence/absence of a catalyst

Stat

>DOEof a catalyst.

We create a factorial experiment to figure out which effect each of the single factors has

>Factorial

>Create fact. Design

>number of factors = 3effect each of the single factors has.

A graphical evaluation will help to rate the effects. Th th ti l d l ill b t bli h d d th

>number of factors = 3

>Design = full fact

The mathematical model will be established and the optimal results adjusted accordingly.

File: 2k factorial Design.mtw

As “factor” you enter names and values

RunOrder CenterPt Blocks Temp Con. Catal. Yield1 1 1 160 20 -1 60

File: 2k factorial Design.mtw

names and values.

Temp 160 180

Con. 20 40

2 1 1 180 20 -1 723 1 1 160 40 -1 544 1 1 180 40 -1 68

Catal. -1 1

Enter the yield in the work sheet.

5 1 1 160 20 1 526 1 1 180 20 1 837 1 1 160 40 1 458 1 1 180 40 1 80


o s eet 8 1 1 180 40 1 80

The Steps in Minitab

1Stat

>DOE

>Factorial>Factorial


32


The Mathematical Model of the ResultsResults for: 2K FACTORIAL DESIGN.MTW

Factorial Fit: Yield versus Temp; Conc.; Catal.

Estimated Effects and Coefficients for Yield (coded units)

Term Effect CoefConstant 64,250T 23 000 11 500

Using the coefficients we can calculate the Fits:

Yield = 64 25 + 11 5xTemp -2 5xCon + 0 75xCatal +Temp 23,000 11,500Conc. -5,000 -2,500Catal. 1,500 0,750Temp*Conc. 1,500 0,750Temp*Catal 10 000 5 000

Yield = 64,25 + 11,5xTemp -2,5xCon. + 0,75xCatal. + 0,75xTemp*Con. + 5xTemp*Catal. + 0xConc*Catal. + 0.25xTemp*Con.*Catal.

A measure for the quality of our model is derivedTemp*Catal. 10,000 5,000Conc.*Catal. -0,000 -0,000Temp*Conc.*Catal. 0,500 0,250

A measure for the quality of our model is derived from Measurement values - Fits = Residuals. The Residuals should be as small as possible and their mean should be 0mean should be 0

First statement about the variances

Analysis of Variance for Yield (coded units)

Source DF Seq SS Adj SS Adj MS F P

Some values can’t be calculated because the model includes all the interactions On the other side weMain Effects 3 1112,50 1112,50 370,833 * *

2-Way Interactions 3 204,50 204,50 68,167 * *3-Way Interactions 1 0,50 0,50 0,500 * *Residual Error 0 * * *

interactions. On the other side we have just single values.


Total 7 1317,50

The Graphical Evaluation of the Effects

Pareto Chart Normal distribution

We see the effects declining in accordance to their importance. The red line is our chosen

Minitab shows the effects in a normal distribution diagram. The significant deviation of terms is

significance limit. highlighted.

A

5,97

A TempB C onc

Factor Name

Pareto Chart of the Effects(response is Yield, Alpha = 0,10)

99

95Not SignificantSignificant

Effect Type

Normal Plot of the Effects(response is Yield, Alpha = 0,10)

AB

B

AC

Term

B C onc.C C atal. 90

80

7060504030P

erc

en

t

A TempB C onc.C C atal.

F actor Name

g

AC

A

BC

ABC

C

22000 22000

30

20

10

5

12520151050

Effect

Lenth's PSE = 2,25

2520151050-5Effect

Lenth's PSE = 2,25


Simplification of the ModelAs introduced before we are allowed to simplify our model to the significant factors and interactions. The result will be more accurate because the degrees of freedom are assigned more realistically Reduce step by step!degrees of freedom are assigned more realistically. Reduce step by step!

Estimated Effects and Coefficients for Yield (coded units)

T Eff t C f SE C f T P

Significant factors usually show a P – value < 0,05. (Probability f i fl 95%)

Term Effect Coef SE Coef T PConstant 64,250 0,4564 140,76 0,000Temp 23,000 11,500 0,4564 25,20 0,000Conc. -5,000 -2,500 0,4564 -5,48 0,012Catal 1 500 0 750 0 4564 1 64 0 199 of influence > 95%)Catal. 1,500 0,750 0,4564 1,64 0,199Temp*Catal. 10,000 5,000 0,4564 10,95 0,002

S = 1 29099 PRESS = 35 5556S 1,29099 PRESS 35,5556R-Sq = 99,62% R-Sq(pred) = 97,30% R-Sq(adj) = 99,11%

Analysis of Variance for Yield (coded units)y ( )

Source DF Seq SS Adj SS Adj MS F PMain Effects 3 1112,50 1112,50 370,833 222,50 0,0012-Way Interactions 1 200,00 200,00 200,000 120,00 0,002

A small portion of variance is not explained in the reduced model, in

Residual Error 3 5,00 5,00 1,667Total 7 1317,50


Minitab noted as error.

R2 and R2 adj.: Practical Significance

S = 1,29099 PRESS = 35,5556R-Sq = 99,62% R-Sq(pred) = 97,30% R-Sq(adj) = 99,11%

• R² is a method within the statistics, to show the practical 9962,0

5,1312Re2 === gressionSSRstatistics, to show the practical

significance of an effect. 996,0

5,1317TotalSS

• R² adj. is a similar method to explain the practical significance of anR adj. is a similar method to explain the practical significance of an effect. It is helpful, if we use several factors in a model. E.g. R2 adj. gets smaller, if an additional factor is added in the model, because every reduction of SS can be balanced by the loss of degrees of freedomreduction of SS error can be balanced by the loss of degrees of freedom. The values for R² adj. are always a little bit smaller than for R².

6671MS9911,0

75,1317

667,1112 =−=−=

Total

Total

Error

DF

SSMS

adjR

Total

• S is the pooled standard deviation (averaged within group variation) The square root of S is the MS Error


square root of S is the MS Error.

PRESS and R2 (pred): Significance for PredictionPrediction Sum of Squares (PRESS)

The predictive ability of the model will be assessed with the statistic PRESS. In l th ll th l PRESS th b tt th di ti bilit f th d lgeneral, the smaller the value PRESS, the better the predictive ability of the model.

PRESS is used for the calculation of predicted R2. The interpretation of R2 (pred) is in general more intuitive. The combination of these statistics can help to avoid an over adjustment of model because it uses observations for the calculation which are notadjustment of model because it uses observations for the calculation which are not considered in the model estimation. An over adjustment of models exists, which explains apparently the relation between predictor and response variable based on the data set used for the model calculation but which don’t deliver valid prediction forthe data set used for the model calculation, but which don t deliver valid prediction for new observations.

PRESS is similar to the residual (Error) Sum of Squares (SSE) and presents the sum of squares prediction error. PRESS is different to SSE, because each adjusted value, ith, will be calculated for PRESS in the following procedure: Initially every ith

observation will be excluded from the data set. Subsequently the regression equation ill b ti t d b d th i i 1 b ti d th di t d lwill be estimated based on the remaining n -1 observations and the predicted value

will be calculated with the help of the adjusted regression equation .

The predicted R2 indicates how well the model predicts responses for new observationsThe predicted R2 indicates, how well the model predicts responses for new observations.

973,051317

56,3511)(2 =−=−=

SS

PRESSpredR


5,1317)(

TotalSSp

Graphical Support at the EvaluationAl t t ith i t ti di d l t th i ifi tStat

>DOE

>Factorial

Always start with an interaction diagram and select the significant effects. In the second step review the significant main effects which have no interactions.

85

80160180

Temp

Interaction Plot for YieldData Means

>Factorial

>Factorial Plots…

75

70

65Me

an

Interaction plot

The slope indicates the significance of the 60

55

50

interactions.

M i Eff t Pl t f Yi ld1-1

Catal.67

66

Main Effects Plot for YieldData Means

Mean value of all trials

65

64Me

an

Mean value of all trials

with high level setting

4020

63

62 Mean value of all trials with low level setting


4020Conc.

low level setting

Graphical Support at the EvaluationStat

>DOE

>Factorial>Factorial

>Factorial Plots…

The highest value for yield will be received with the following setting:

Temp: 180 degrees CTemp: 180 degrees C

Catalyst: with Catalyst 1

Concentration: 20%Concentration: 20%


The Multivari ChartStat

>Quality Tools

>Multivari ChartMulti-Vari Chart for Yield by Catal. - Temp

>Multivari Chart…

80

4020

160 180 Catal.-11

Yie

ld

70

6060

50

Conc.4020

40

Panel variable: Temp

The catalysts affect yield differently as a function of temperature . Changes

Panel variable: Temp

in the concentration have no big effect on the yield, the results are similar.

High temperature (if controllable) in combination with catalyst 1 shows the b t lt i d d t f t ti tti


best results independent of concentration setting.

The Significance of the Model

Possibilities of model diagnostics in Minitab

• Normal distribution of the residuals: The observations should follow a straight line in this diagram. Small deflections at both ends are acceptable. Points fairly outside indicate an effect not considered in this model.

• Histogram of residuals: Usually one expects a normal distribution with a mean of 0. Strong deviations are indications of effects from other factors (not included in this experiment)other factors (not included in this experiment)

• Run chart, (I-Chart) of residuals: Shows trends of the experiment. Special causes will be highlighted by Minitab.

• Residuals against fits ( Calculated results): This plot should show a g ( ) prandom pattern of the residuals on both sides of the baseline. Pay attention to patterns indicating trends.


Residual DiagramsStat

>DOE

>Factorial

Normal Probability Plot of the Residuals Residuals Versus the Fitted Values

Residual Plots for Yield

>Factorial


>Graphs…

rce

nt

99

90

50 sid

ual

1,0

0,5

0,0

Normal Probability Plot of the Residuals Residuals Versus the Fitted Values

Residual

Per

210-1-2

10

1

Fitted Value

Res

80706050

-0,5

-1,0

ncy

2,0

1,5

ual

1,0

0,5

Histogram of the Residuals Residuals Versus the Order of the DataF

requ

en

1,00,50,0-0,5-1,0

1,0

0,5

0,0

Re

sidu

87654321

0,0

-0,5

-1,0

Residual,,,,,

Observation Order


Problem, Electrical Test of PCB

The portion of the retest is 28,8%... As a critical factor the quality of the electrical contact has been determined.

The properties of the factor metal pin were investigated with a DOE.

DOE Planning :• 3 factors, with 2 levels each:

Supplier : A - B

Diameter : thick - thin (diameter of the contact pin)

Material type : Cu - Ag

• Full factorial design with 2 repeats(2 tests per combination)

• Scope: 2 sets with 25 pieces each

• Result = Amount of retest for 2 x 25 pieces for each factor combination


combination

Application of the Planned Design

th mat sup retest

thin c A 6

Stat

>DOE

>Factorial

1

thin c A 6

thick a A 7

thick c A 3

>Factorial


thin c A 7

thick c A 4

thick a B 4thick a B 4

thin c B 5

thin a A 5

23

thick c B 2

thick c B 1

thick a B 4thick a B 4

thin c B 4

thin a A 6

File: DOE Retest.mtw


First Evaluation

Aim: what´s significant

Factorial Fit: retest versus th; mat; sup

Estimated Effects and Coefficients for retest (coded units) Pareto Chart of the Standardized Effects(response is retest, Alpha = 0,05)

Term Effect Coef SE Coef T PConstant 4,6250 0,1976 23,40 0,000th -1,7500 -0,8750 0,1976 -4,43 0,002mat 1,2500 0,6250 0,1976 3,16 0,013sup -1,5000 -0,7500 0,1976 -3,79 0,005

C

A

2,306

A thB matC sup

Factor Name

p , , , , ,th*mat 1,2500 0,6250 0,1976 3,16 0,013th*sup -0,5000 -0,2500 0,1976 -1,26 0,242mat*sup 0,5000 0,2500 0,1976 1,26 0,242th*mat*sup -0,5000 -0,2500 0,1976 -1,26 0,242

BC

ABC

B

AB

Term

S = 0,790569 PRESS = 20R-Sq = 88,02% R-Sq(pred) = 52,10% R-Sq(adj) = 77,54%

AC

BC


Analysis of Variance for retest (coded units)

Source DF Seq SS Adj SS Adj MS F PMain Effects 3 27,5000 27,5000 9,1667 14,67 0,0012-Way Interactions 3 8,2500 8,2500 2,7500 4,40 0,042y , , , , ,3-Way Interactions 1 1,0000 1,0000 1,0000 1,60 0,242Residual Error 8 5,0000 5,0000 0,6250Pure Error 8 5,0000 5,0000 0,6250

Total 15 41,7500


The Reduced Model

Here we get the best model from our experimental design

Pareto Chart of the Standardized Effects(response is retest, Alpha = 0,05)

Factorial Fit: retest versus th; mat; sup

Estimated Effects and Coefficients for retest (coded units)

A

2,201

A thB matC sup

Factor Name

Term Effect Coef SE Coef T PConstant 4,6250 0,2132 21,69 0,000th -1,7500 -0,8750 0,2132 -4,10 0,002mat 1,2500 0,6250 0,2132 2,93 0,014sup -1,5000 -0,7500 0,2132 -3,52 0,005

AB

C

Term

p , , , , ,th*mat 1,2500 0,6250 0,2132 2,93 0,014

S = 0,852803 PRESS = 16,9256R-Sq = 80,84% R-Sq(pred) = 59,46% R-Sq(adj) = 73,87%

B


q , q(p ) , q( j) ,

Analysis of Variance for retest (coded units)y ( )

Source DF Seq SS Adj SS Adj MS F PMain Effects 3 27,500 27,500 9,1667 12,60 0,0012-Way Interactions 1 6,250 6,250 6,2500 8,59 0,014Residual Error 11 8,000 8,000 0,7273, , ,Lack of Fit 3 3,000 3,000 1,0000 1,60 0,264Pure Error 8 5,000 5,000 0,6250

Total 15 41,750


The Necessary Model DiagnosticStat

>DOE

>Factorial

Normal Probability Plot Versus Fits

Residual Plots for retest

>Factorial


>Graphs…99

90

50cent

1,0

0,5

0 0dual

y

21012

50

10

1

Per

c

65432

0,0

-0,5

-1,0

Re

sid

210-1-2Residual

65432Fitted Value

Histogram Versus Order

4,8

3,6

2,4

equ

ency

1,0

0,5

0,0

esid

ual

1,51,00,50,0-0,5-1,0

1,2

0,0

R id l

Fre

16151413121110987654321

-0,5

-1,0

Observation Order

Re


Residual Observation Order

Graphical Presentation of the ResultsStat

>DOE

>FactorialInteraction Plot (data means) for retest

First we interpret the interaction diagram…

>Factorial

>Factorial Plots…

5,5

5,0

4,5

thinthick

th

Me

an

4,0

3,5

matac

3,0

2,5… subsequent the main effectsMain Effects Plot (data means) for retest

5,5

5,0

4,5

th mat

a ec s o (da a ea s) o e es

Me

an

of

rete

st

thickthin

4,0

3,5ac

5,5sup

5,0

4,5

4,0

3,5


BA

Translation to Reality

Multi-Vari Chart for retest by th - sup

Stat

>Quality Tools

>Multivari Chart

7

ac

A Bthin

th

y p>Multivari Chart…

7

6

thinthick

ete

st

5

4re

3

2

1

matac

Panel variable: sup


The Practical Meaning of the Results

C o s t 23 key A-products identified, test time average 60,24 secnew test time average at 45 90 sec : improvement 25 6%new test time average at 45,90 sec : improvement 25,6%

Retest rate measured for

Q u a l i t y

Retest rate measured for 11 key A-products : 28,8%new retest rate at 6,9% :new retest rate at 6,9% :improvement 21,9%

Stability Maintenance time for 23 key A-products at 84 min/daynew maint. time at 49 min/day : improvement 41,6%


The Helicopter Exercise

•• Purpose:Purpose: Investigation of a helicopter design with a 2k factorial experimentexperiment

•• Goal:Goal: OptimizeOptimize airborne time

•• Output:Output: Airborne: time between launch until first ground contact

•• Procedure:Procedure:•• Procedure:Procedure:

• Select 3 factors for the investigation.

• Perform a 2x2x2 factorial experimental design with two repeats per treatment.

H ld th fli ht h i ht t t!• Hold the flight height constant!

• Define the measurement system first!!!

• Follow the analysis “roadmap” and present your results on a flipchart.


The Helicopter Exercise

Wing length

2 2

Weight

• short

• long

1 1• 1st clip

• 2nd clip• long • 2nd clip

DOE 1DOE 1

Shaft length

• shortShaft width

• small 2 21 1

• longsmall

• wide11


2 2

The Helicopter ExerciseSteps of the experiment

• Define the problem

• Establish the goal

• Select the output variablesp

• Select the input variables

• Define the levels of the variablesDefine the levels of the variables

• Select the experimental design

• Determine the tasks in the team• Determine the tasks in the team

• Collect the data

A l th d t• Analyze the data

• Draw statistical conclusions

• Replicate the results

• Develop practical solutions


• Implement the solutions

Example: Customer Survey

C t h th i j d t i d h th t b t• Customers have their own judgment in order whether to buy or not to buy this product.

• Lets take 3 important criteria. We want to figure out how they affect the decision of potential buyers.

• Here are criteria for buying a laptop

Weight (g) Battery Capacity (h) Display Size (cm)

1500 2 202200 3 302200 3 30


Example: Customer Survey

Please rate the interest for a purchase for the following combination of factors (Types of Laptops) on a scale of 1 to 10 (10 = strongest interest)

First as an individual, than in the team

RunOrder Weight Capacity Display Rating1 1500 2 202 2200 2 203 1500 3 204 2200 3 204 2200 3 205 1500 2 306 2200 2 306 2200 2 307 1500 3 308 2200 3 30

Now enter your rating in the Minitab file Laptop1.mpj


y g p p pj

Final Report

The report should include the following items:

• SummarySummary

• Problem description and background

• GoalsGoals

• Output variables

• Input variablesInput variables

• Experimental design

• Process / ProcedureProcess / Procedure

• Results and data analysis

• ConclusionsConclusions

Attachments

• Detailed data analysisDetailed data analysis

• Original data, if available

• Details of the instruments and procedure


Details of the instruments and procedure

Be Pro-active

DOE i ti t l• DOE is a pro-active tool.

• There are no bad experiments – only poorly planned and There are no bad experiments only poorly planned and performed ones.

N t i t ill h ith b kth h di i• Not every experiment will show up with breakthrough discoveries for the world.

• Every experiment teaches you something.

N d t b i ti d lt i f ll• New data bring new questions and results in follow up studies.


Summary

• Ways to learn

• Components of an experiment

• Experimental validation

• Steps for planning an experiment• Steps for planning an experiment

• 2k factorial designg

• Practical exercises

• Final report


Appendix DOE

Terminology

Planning

Examples


TerminologyDesign (Layout): Complete specification of experimental runs, which include block building, randomization, replicates, repeats and attribution of factor/level combinations to experimental unitsand attribution of factor/level combinations to experimental units.

2k x 3k x 3k… factorial: Description of a basic design. A 2 x 3 x 3 design having three input variables, one with two levels and two with three levels. The number of experimental runs (treatments) is the product of the levels In this case we have 18 treatmentsis the product of the levels. In this case we have 18 treatments.

Response unit: A unit under observation and measured during the experiment Also called analysis unitexperiment. Also called analysis unit.

Treatment combination: An experimental run with defined levels f ffor all input variables, referred to as cell.

Balanced design: A design with equal numbers of experimental g g q pruns in every treatment combination or run.

Unbalanced design: A design with a uneven number of


Unbalanced design: A design with a uneven number of experimental runs per treatment combination.

Terminology, Continued

Repetition: Running several samples on one treatment combination.

Replication: Replication (repeating) of the entire experiment.

Effect (main effect): The average change in the response variable due to the change of a factor from one level to another.

Interaction: Exists when an effects of one factor of the response depends on the setting of other factors.

Experimental area: All possible factor / level combinations where an experiment could be carried out.

Test run: One or more observations of the output variable for a single combination of the experiment.g p

Confounding: One or more effects that cannot be separated properly and assigned to a factor or a interaction


properly and assigned to a factor or a interaction.

Exercise

A friend of yours brags that he is able to differentiate several types of beer from each other, especial “Jever” by tasting these beers.

You ask him to prove it.

You plan an experiment:

Define purpose & goal− Define purpose & goal

− Outputs

− Inputs

> Controllable inputs

> Not controllable inputs

(noise) Review and rate your draft for(noise)

− Scheme of randomization

Review and rate your draft for internal and external validity.

Present your results on a flip chart.


General Items for a DOE Planning

• Involvement of a team (cross-functional)

• Maximize prior knowledge

Pursue measurable objectives• Pursue measurable objectives

• Plan the execution of all phases (including confirmation)

• Rigorous sample size determination

• Allocate sufficient resources for data collection and analysis

• Write and review proposal p p

See also week 1

Module 05 “Thought Map”

last 3 pages


last 3 pages

Steps of Planning

• Define the problem with business scores (RTY, COQ, Capacity, Productivity)Productivity)

• Name the goals of the experiment

• Define the output variables

• Define the input variables (factor selection)

• Define the levels for the input variablesp

• Choice an design for the experiment

S l k 1

• Plan and provide equipment, material, operator

• Review you proposal See also week 1


Review you proposal


last 3 pages

Questions with Respect to Planning

• What is the measurable goal?

• What will it cost?

• H d t bli h th l i ?• How do we establish the sample size?

• How does the plan for randomization looks like?

• Are our internal customers informed?

• What time will it take?• What time will it take?

• How do we analyze the data?

• Did we set up a control plan?See also week 1


last 3 pages


last 3 pages

Experimental GoalsCentering or reduction of variation?

How small are the changes you want to detect (experimentalHow small are the changes you want to detect (experimental “Delta”)?

Examples:Examples:

• Determination of the effect of material change on product reliability

• Defining of causes for the variation for a critical process

• Evaluation of the effect of cheaper material on the product performanceEvaluation of the effect of cheaper material on the product performance

• Determination of the effect of variation of the operator on the final product/serviceproduct/service

• Evaluation of cause – effect relations on process inputs and product characteristicscharacteristics

Usually investigation and determination of the effect of several factors on


the response (output)

Definition of the Output Variables• Is the output qualitative or quantitative?

• Objective: centering variation reduction or both?Objective: centering, variation reduction, or both?

• What is the baseline? (mean and standard deviation evaluation)

• Is the output under statistical control?

• Does the output vary over time?• Does the output vary over time?

• How large a change in the output do you want to detect?

• Is the output normal distributed?

• How do we measure the output?

• Is the measurement system adequate?y q

• Are there multiple outputs? What are the priorities for these?


Selection of Factors

The initial identification of factors should include at least the following sources:

• Process map

• Cause and effect matrix

• FMEA

• Multivari

• Literature review

• Brainstormingg

• Scientific theory

• Operator experience

• Customer/supplier inputs

• Ranking methods (or nominal group technique)g ( g )

Final selection of factors based on prioritization techniques, technical knowledge, FMEA RPN’s, etc.


Factor Selection

List all KPIV’s and KPOV’s of the complete process

Identif al e adding and non al e adding steps in the processIdentify value adding and non value adding steps in the process

List for all sub process inputs and outputs

Split the inputs (factors) in controllable, non controllable (noise) and SOP partsp

Define the process critical inputs of the current process

Use prioritizing tools to select the factors


Factor Level SelectionAfter factor definition the level settings have to be chosen. The first Goal: Distinguish the vital inputs from a big number of inputs (Screening)(Screening)

• If we vary the factors to extremes we will see an effect on select broad or “bold” levels to include all possible components ofbroad or bold levels to include all possible components of variation of the current process

• the response if there is one• the response if there is one

• May exaggerate the variation (unrealistic variation)

• “unrealistic” variation may be created

Examples of broad settings:p g

• Qualitative:

Method A vs B or Reactor 1 vs Reactor 2Method A vs. B or Reactor 1 vs. Reactor 2

• Quantitative:


5 minutes vs. 15 minutes or 2 bar vs. 4 bar

Factor Level SelectionIf the vital factors have been determined the goal is now to understand the interactions between the factors (Ch t i ti )(Characterization)

− Information form earlier experiments will used to adjust the f t tti di l t i th i t ti f thfactor settings accordingly to recognize the interactions of the inputs

U ll th l l ill b t l F ll hi h l ti− Usually the levels will be set closer. Full or high resolution fractional factorials are recommended

N t l i t id tif th ti i d f f i tNext goal is to identify the operating window of a group of input variables and understand the experimental space near the optimum (Optimization)p ( p )

− Levels of factor will be set closer together, usually

Th i t h ll b f f t ith− The experiments have a smaller number of factors with an increased number of levels

S i l DOE d i ill b ft d ( W k 3 & 4)


− Special DOE designs will be often used (see Week 3 & 4)

Steps of the Experiment

Define the problem

Establish the goalEstablish the goal

Select the output variables

S l t th i t i blSelect the input variables

Define the levels of the variables

Select the experimental design

Determine the tasks in the team

Collect the data

Analyze the dataAnalyze the data

Draw statistical conclusions

Replicate the resultsReplicate the results

Develop practical solutions


Implement the solutions

General Recommendations

• Make sure that a business benefit is connected with your project ( i t)(experiment).

• Try not to answer all questions with one study. Rely on several studies in a sequence.. (Rule of the thump: Spend less than 25% of your budget for the first experiment)

• Use design with 2 levels in early project stages

• Proof your results always with a confirmation run• Proof your results always with a confirmation run

• Be prepared for changes!

• A final report is required!


javier garcia - verdugo sanchez - six sigma training - w2 design of experiments (doe) intro

Engineering