design and analysis of experiments lecture 3...design and analysis of experiments lecture 3.1 1....

Lecture 3.1 1

© 2016 Michael Stuart

Design and Analysis of Experiments

Lecture 3.1

1. Review of Lecture 2.2

– 2-level factors

– Homework 2.2.1

2. A 23 experiment

3. 24 in 16 runs with no replicates

Postgraduate Certificate in Statistics


Lecture 3.1 2


2k Factorial Designs

• Designs with k factors each at 2 levels

• 2k factor level combinations

– 22 = 4

– 23 = 8

L H

A

H

B

L

L H

A

H

B

L

H

L C

Ref: Lecture 1.1

Multi-factor Designs



Lecture 3.1 3


Why use 2- level factorial designs?

3 factors each with 2 levels: 23 = 8



3 factors with levels 3, 4 and 5, respectively:

3x4x5 = 60

More reasons later!



Lecture 3.1 4


A 22 experiment

Project:

optimisation of a chemical process yield

Factors (with levels):

operating temperature (Low, High)

catalyst (C1, C2)

Design:

Process run at all four possible combinations of

factor levels, in duplicate, in random order.

2k Factorial Designs



Lecture 3.1 5


Design matrix

Replicate 1

Replicate 2

Columns designate experimental factors

Rows designate experimental conditions

Design

Point Temperature Catalyst

1 Low 1

2 High 1

3 Low 2

4 High 2

5 Low 1

6 High 1

7 Low 2

8 High 2



Lecture 3.1 6


Results (standard order)

Standard Order

Run Order

Temperature Catalyst Yield

1 6 Low 1 60

2 8 High 1 72

3 1 Low 2 52

4 4 High 2 83

5 3 Low 1 54

6 7 High 1 68

7 2 Low 2 45

8 5 High 2 80



Lecture 3.1 7


Effects as calculated by Minitab

Estimated Effects and Coefficients for Yield

Term Effect Coef SE Coef T P

Constant 64.25 1.31 49.01 0.000

Temperature 23.0 11.50 1.31 8.77 0.001

Catalyst 1.5 0.75 1.31 0.57 0.598

Temperature*Catalyst 10.0 5.00 1.31 3.81 0.019

S = 3.70810

Effect = Coef x 2

SE(Effect) = SE(Coef) x 2

T effect: 23 = Mean Yield at HighT – Mean Yield at Low T



Lecture 3.1 8


Direct Calculation

Temp Cat Rep

1 Rep

2 Rep1 – Rep2

Variance =

½(Rep1–Rep2)² Degrees of Freedom

Low 1 60 54 6 18 1

High 1 72 68 4 8 1

Low 2 52 45 7 24.5 1

High 2 83 80 3 4.5 1

Sum 55 4

Mean = s² 13.75

s 3.7

LowY HighY LowHigh YY52.75 75.75 23

2/s4/s2)YY(SE 2LowHigh 2.6

6.2

23

)YY(SE

YYt

LowHigh

LowHigh8.8



Lecture 3.1 9


Direct Calculation

Temp Cat Rep

1 Rep

2 Rep1 – Rep2

Variance =

½(Rep1–Rep2)² Degrees of Freedom

Low 1 60 54 6 18 1

High 1 72 68 4 8 1

Low 2 52 45 7 24.5 1

High 2 83 80 3 4.5 1

Sum 55 4

Mean = s² 13.75

s 3.7

LowY HighY LowHigh YY52.75 75.75 23

2/s4/s2)YY(SE 2LowHigh 2.6

6.2

23

)YY(SE

YYt

LowHigh

LowHigh8.8



Lecture 3.1 10


Classwork 2.2.2

Calculate a confidence interval for the Temperature

effect.

t4, .05 = 2.78

CI: 23 2.78 x 2.6

23 7.2

15.8 to 30.2

23YY LowHigh

6.2)YY(SE LowHigh



Lecture 3.1 11


Design matrix

Design

Point Temperature Catalyst

1 Low 1

2 High 1

3 Low 2

4 High 2

5 Low 1

6 High 1

7 Low 2

8 High 2



Lecture 3.1 12


Design matrix:

generic notation

+ = "High"; – = "Low"

Design

Point

Temperature

A

Catalyst

B

1 – –

2 + –

3 – +

4 + +

5 – –

6 + –

7 – +

8 + +



Lecture 3.1 13


Design Matrix with Y’s

Main effect estimates:

¼ (Y2+Y4+Y6+Y8) – ¼ (Y1+Y3+Y5+Y7)

¼ (Y3+Y4+Y7+Y8) – ¼ (Y1+Y2+Y5+Y6)

Design

Point

Temperature

A

Catalyst

B Yield

1 – – Y1

2 + – Y2

3 – + Y3

4 + + Y4

5 – – Y5

6 + – Y6

7 – + Y7

8 + + Y8

Â

B̂Postgraduate Certificate in Statistics


Lecture 3.1 14



Main effect estimates:

¼ (Y2+Y4+Y6+Y8) – ¼ (Y1+Y3+Y5+Y7)

¼ (Y3+Y4+Y7+Y8) – ¼ (Y1+Y2+Y5+Y6)

Design

Point

Temperature

A

Catalyst

B Yield

1 – – Y1

2 + – Y2

3 – + Y3

4 + + Y4

5 – – Y5

6 + – Y6

7 – + Y7

8 + + Y8

Â

B̂Postgraduate Certificate in Statistics


Lecture 3.1 15



Interaction effect estimates:

½ (Y4+Y8) – ½ (Y3+Y7)

½ (Y2+Y6) – ½ (Y1+Y5)



Design

Point

Temperature

A

Catalyst

B Yield

1 – – Y1

2 + – Y2

3 – + Y3

4 + + Y4

5 – – Y5

6 + – Y6

7 – + Y7

8 + + Y8

BHighatÂ

BLowatÂ

Lecture 3.1 16


The interaction effect

A effect at High B: ½ (Y4 + Y8) – ½ (Y3 + Y7)

A effect at Low B: ½ (Y2 + Y6) – ½ (Y1 + Y5)

½ difference: ¼ (Y4 + Y8) – ¼ (Y3 + Y7)

– ¼ (Y2 + Y6) + ¼ (Y1 + Y5)

= ¼ (Y4 + Y8 + Y1 + Y5) – ¼ (Y3 + Y7 + Y2 + Y6)



Lecture 3.1 17


Design Matrix: applying the signs

Effect estimates: Sum/4 Sum/4

Classwork 3.1.1: Check correspondence with Slide 14.

Â B̂

Temperature

A

Catalyst

B

– Y1 – Y1

+ Y2 – Y2

– Y3 + Y3

+ Y4 + Y4

– Y5 – Y5

+ Y6 – Y6

– Y7 + Y7

+ Y8 + Y8



Lecture 3.1 18


Augmented Design Matrix with Y’s

Interaction effect:

(Y1 – Y2 – Y3 + Y4 + Y5 – Y6 – Y7 + Y8)/4

Classwork 3.1.2: Check correspondence with Slide 16

Design Point

A B AB Yield

1 – – + Y1 2 + – – Y2 3 – + – Y3

4 + + + Y4

5 – – + Y5

6 + – – Y6

7 – + – Y7

8 + + + Y8



Lecture 3.1 19


Augmented Design Matrix with Data

Classwork 3.1.3: Calculate the estimate of the AB

interaction.

Design Point

A B AB Yield

1 – – + 60

2 + – – 72

3 – + – 52

4 + + + 83

5 – – + 54

6 + – – 68

7 – + – 45

8 + + + 80



Lecture 3.1 20


Dual role of the design matrix

• Prior to the experiment, the rows designate the

design points, the sets of conditions under which

the process is to be run.

• After the experiment, the columns designate the

contrasts, the combinations of design point

means which measure the main effects of the

factors.

• The extended design matrix facilitates the

calculation of interaction effects



Lecture 3.1 21


Lecture 3.1


– 2-level factors

– Homework 2.2.1

2. A 23 experiment




Lecture 3.1 22


Homework 2.2.1

Test the statistical significance of and

calculate confidence intervals for

the Catalyst effect and

the Temperature by Catalyst interaction effect.



Lecture 3.1 23


B AB Yield

– + 60

– – 72

+ – 52

+ + 83

– + 54

– – 68

+ – 45

+ + 80



)YY(SE

YYt

Lecture 3.1 24


Lecture 3.1


– 2-level factors

– Homework 2.2.1

2. A 23 experiment




Lecture 3.1 25


Part 2 A 23 experiment

3 factors each at 2 levels

Project:

optimisation of a chemical process yield

Factors (with levels):

operating Temperature T (°C) (160, 180)

raw material Concentration C (%) (20, 40)

catalyst K (A, B)

Design:

Process run at all eight possible combinations of

factor levels, in duplicate, in random order.



Lecture 3.1 26


Design matrix

(standard order)

Run T C K

1 – + –

2 + – –

3 – + +

4 + – –

5 + + –

6 – – –

7 + + +

8 – – +

9 + – +

10 + + –

11 – + +

12 – – +

13 – – –

14 + – +

15 + + +

16 – + –

A 23 experiment

Run order for design points

(in duplicate)

Factor

Design Point

A = T B = C C = K

1 – – –

2 + – –

3 – + –

4 + + –

5 – – +

6 + – +

7 – + +

8 + + +



Lecture 3.1 27


A 23 experiment

Results, in standard order T C K Yield Mean SD

– – – 59 61 60 1.41

+ – – 74 70 72 2.83

– + – 50 58 54 5.66

+ + – 69 67 68 1.41

– – + 50 54 52 2.83

+ – + 81 85 83 2.83

– + + 46 44 45 1.41

+ + + 79 81 80 1.41

Ref: PilotPlant.xls



Lecture 3.1 28


A 23 experiment

– Initial analysis

– Calculating effects

– Calculating s

– Minitab analysis



Lecture 3.1 29


Initial analysis



Lecture 3.1 30


Initial analysis



Lecture 3.1 31


Calculating main effects

Classwork 3.1.4: Calculate K main effect

T C K Mean – – – 60 + – – 72 – + – 54 + + – 68 – – + 52 + – + 83 – + + 45 + + + 80



Lecture 3.1 32


Calculating interaction effects,

the extended design matrix

Classwork 3.1.5: Complete the missing columns.

Calculate the TK and TCK interactions

Design Point

T C K TC TK CK TCK Mean

1 – – – + + 60 2 + – – – – 72 3 – + – – + 54 4 + + – + – 68 5 – – + + – 52 6 + – + – + 83 7 – + + – – 45 8 + + + + + 80



Lecture 3.1 33


Calculating s

T C K Yield Variance =½(diff)2

– – – 59 61

+ – – 74 70

– + – 50 58

+ + – 69 67

– – + 50 54

+ – + 81 85

– + + 46 44

+ + + 79 81



Lecture 3.1 34


Calculating s

T C K Yield Variance =½(diff)2

– – – 59 61

+ – – 74 70

– + – 50 58

+ + – 69 67

– – + 50 54

+ – + 81 85

– + + 46 44

+ + + 79 81

2

8

32

2

8

8

2

2

Total 64

s2 8

s 2.828



Lecture 3.1 35


Classwork 3.1.6

Calculate the t-ratio for the Temperature effect and

the 3-factor interaction. What conclusions do you

draw?



Lecture 3.1 36


Minitab analysis

Estimated Effects for Yield

Term Effect SE T P

T 23.0 1.414 16.26 0.000

C -5.0 1.414 -3.54 0.008

K 1.5 1.414 1.06 0.320

T*C 1.5 1.414 1.06 0.320

T*K 10.0 1.414 7.07 0.000

C*K 0.0 1.414 0.00 1.000

T*C*K 0.5 1.414 0.35 0.733

S = 2.82843



Lecture 3.1 37


Diagnostic analysis

De

lete

d R

es

idu

al

Score

3

2

1

0

-1

-2

-3

210-1-2

Normal Probability Plot of the Residuals

(response is Y)



Lecture 3.1 38


Diagnostic analysis

Fitted Value

De

lete

d R

es

idu

al

8070605040

3

2

1

0

-1

-2

-3

Residuals Versus the Fitted Values

(response is Y)



Lecture 3.1 39


Exercise 3.1.1

An experiment was run to assess the effects of

three factors on the life of a cutting tool

A: Cutting speed

B: Tool geometry

C: Cutting angle.

The full 23 design was replicated three times. The

results are shown in the next slide and are available

in Excel file Tool Life.xls.

Carry out a full analysis and report.



Lecture 3.1 40


Exercise 3.1.1

Cutting Speed

Tool Geometry

Cutting Angle

Tool Life

- - - 22 31 25 + - - 32 43 29 - + - 35 34 50 + + - 55 47 46 - - + 44 45 38 + - + 40 37 36 - + + 60 50 54 + + + 39 41 47

Exercise 3.1.2:

Web Exercises

Ref: Tool Life.xls



Lecture 3.1 41


Lecture 3.1


2. A 23 experiment


– Normal plot, Pareto chart, Lenth's method

– Reduced model method

– Design projection method



Lecture 3.1 42


Project: Improving the filtration rate of a

chemical manufacturing process

Key factor: Formaldehyde concentration.

Problem: Reducing the level of formaldehyde

concentration reduces the filtration rate

to an unacceptably low level.

Proposal: Raise levels of

– Temperature,

– Pressure and

– Stirring rate.

Design: 24 unreplicated, random run order

Part 3 24 in 16 runs, no replicates



Lecture 3.1 43


Data

Temperature Pressure Formaldehyde Concentration

Stirring Rate

Filtration Rate

Low Low Low Low 45 High Low Low Low 71 Low High Low Low 48 High High Low Low 65 Low Low High Low 68 High Low High Low 60 Low High High Low 80 High High High Low 65 Low Low Low High 43 High Low Low High 100 Low High Low High 45 High High Low High 104 Low Low High High 75 High Low High High 86 Low High High High 70 High High High High 96

Ref: Formaldehyde.xls



Lecture 3.1 44


Initial analysis



Lecture 3.1 45


Initial analysis



Lecture 3.1 46


Minitab / DOE / Factorial /

Analyse Factorial Design

Estimated Effects Term Effect

T 21.625

P 3.125

F 9.875

S 14.625

T*P 0.125

T*F -18.125

T*S 16.625

P*F 2.375

P*S -0.375

F*S -1.125

T*P*F 1.875

T*P*S 4.125

T*F*S -1.625

P*F*S -2.625

T*P*F*S 1.375 Postgraduate Certificate in Statistics


Lecture 3.1 47


No replication: alternative analyses

• Normal plots of effects

– if no effects present, estimated effects reflect

chance variation, follow Normal model

– a few real effects will appear as exceptions in a

Normal plot

• Lenth method

– alternative estimate of s, given few real effects

• Best approach: combine both!



Lecture 3.1 48


Normal Effects Plot

20

10

0

-10

-20

210-1-2

Eff

ect

Score

A T

B P

C F

D S

Factor Name

Not Significant

Significant

Effect TypeAD

AC

D

C

A

Normal Plot of the Effects

Lenth's PSE = 2.625



Lecture 3.1 49


Alternative view: Pareto Chart

• vital few versus trivial many (Juran) Postgraduate Certificate in Statistics


Lecture 3.1 50


Minitab Pareto Chart



Lecture 3.1 51


Lenth's method

Given several Normal values with mean 0

and given their absolute values (magnitudes, or values

without signs), then it may be shown that

SD(Normal values) ≈ 1.5 × median(Absolute values).

Denote this by s0

Given a small number of effects with mean ≠ 0, then

SD(Normal values) is inflated.

Refinement: PSE ≈ 1.5 × median(Abs values < 2.5 × s0 )



Lecture 3.1 52


Calculating PSE

Term Effect

T 21.625

P 3.125

F 9.875

S 14.625

T*P 0.125

T*F -18.125

T*S 16.625

P*F 2.375

P*S -0.375

F*S -1.125

T*P*F 1.875

T*P*S 4.125

T*F*S -1.625

P*F*S -2.625

T*P*F*S 1.375



Lecture 3.1 53


Calculating PSE, ignore signs

Term Effect

T 21.625

P 3.125

F 9.875

S 14.625

T*P 0.125

T*F 18.125

T*S 16.625

P*F 2.375

P*S 0.375

F*S 1.125

T*P*F 1.875

T*P*S 4.125

T*F*S 1.625

P*F*S 2.625

T*P*F*S 1.375



Lecture 3.1 54


No. Term Effect

1 T*P 0.125

2 P*S 0.375

3 F*S 1.125

4 T*P*F*S 1.375

5 T*F*S 1.625

6 T*P*F 1.875

7 P*F 2.375

8 P*F*S 2.625

9 P 3.125

10 T*P*S 4.125

11 F 9.875

12 S 14.625

13 T*S 16.625

14 T*F 18.125

15 T 21.625

Calculating PSE, order the effects



Lecture 3.1 55


No. Term Effect

1 T*P 0.125

2 P*S 0.375

3 F*S 1.125

4 T*P*F*S 1.375

5 T*F*S 1.625

6 T*P*F 1.875

7 P*F 2.375

8 P*F*S 2.625

9 P 3.125

10 T*P*S 4.125

11 F 9.875

12 S 14.625

13 T*S 16.625

14 T*F 18.125

15 T 21.625

x 2.5 = 9.84 x 1.5 = 3.94

x 1.5 = 2.625 avge = 1.75

Calculating PSE, order the effects

s0

PSE



Lecture 3.1 56


From Excel, find median(Absolute Values) = 2.625,

so initial SE is s0 = 1.5 × 2.625 = 3.9375.

2.5 × s0 = 9.84375, 5 values exceed.

The median of the remaining 10 is

mean of 1.625 and 1.875 = 1.75

Hence, PSE = 1.5 × 1.75 = 2.625.

Check Slide 50

Calculating PSE



Lecture 3.1 57


Assessing statistical significance

Critical value for effect is t.05,df × PSE

df ≈ (number of effects)/3

t.05,5 = 2.57

PSE = 2.625

Critical value = 6.75

Check Slide 50



Lecture 3.1 58


Estimating s

PSE = 2.625 is the (pseudo) standard error of an

estimated effect.

SE(effect) = (s2/8 + s2/8) = s/2.

s ≈ 2 × 2.625 = 5.25



Lecture 3.1 59


Lecture 3.1


2. A 23 experiment


– Normal plot, Pareto chart, Lenth's method





Lecture 3.1 60


Reduced Model method

• Select identified terms for a fitted model

– omitted terms provide basis for estimating s

→check degrees of freedom

• Estimate effects

– ANOVA used to calculate s

• Check diagnostics



Lecture 3.1 61


Reduced Model method

Y equals overall mean

plus

T effect

plus

F effect

plus

S effect

plus

TF interaction effect

plus

TS interaction effect

plus

chance variation Postgraduate Certificate in Statistics


Lecture 3.1 62


Estimated effects

(Minitab)

Estimated Effects and Coefficients for R (coded units)


Constant 70.063 1.104 63.44 0.000

T 21.625 10.812 1.104 9.79 0.000

F 9.875 4.938 1.104 4.47 0.001

S 14.625 7.312 1.104 6.62 0.000

T*F -18.125 -9.062 1.104 -8.21 0.000

T*S 16.625 8.313 1.104 7.53 0.000

S = 4.41730



Lecture 3.1 63


Analysis of Variance (basis for s)

Source DF SS MS F-Value P-Value

T 1 1870.6 1870.56 95.86 0.000

F 1 390.1 390.06 19.99 0.001

S 1 855.6 855.56 43.85 0.000

T*F 1 1314.1 1314.06 67.34 0.000

T*S 1 1105.6 1105.56 56.66 0.000

Error 10 195.1 19.51

Total 15 5730.9



Lecture 3.1 64


Diagnostics

100908070605040

2

1

0

-1

-2

Fitted Value

De

lete

d R

esid

ua

l

Versus Fits(response is R)



Lecture 3.1 65


Diagnostics

3

2

1

0

-1

-2

-3

210-1-2

De

lete

d R

esid

ua

l

Score

Normal Probability Plot(response is R)



Lecture 3.1 66


Lecture 3.1


2. A 23 experiment


– Normal plot, Pareto chart

– Lenth's method





Lecture 3.1 67


Design Projection Method

Pressure not

statistically significant,

exclude

leave 23, duplicated

T F S FR

– – – 45

– – – 48

+ – – 71

+ – – 65

– + – 68

– + – 80

+ + – 60

+ + – 65

– – + 43

– – + 45

+ – + 100

+ – + 104

– + + 75

– + + 70

+ + + 86

+ + + 96 Postgraduate Certificate in Statistics


Lecture 3.1 68



Initial analysis



Lecture 3.1 69



Initial analysis



Lecture 3.1 70



Minitab analyis

Estimated Effects and Coefficients for R (coded units)


Constant 70.063 1.184 59.16 0.000

T 21.625 10.812 1.184 9.13 0.000

F 9.875 4.938 1.184 4.17 0.003

S 14.625 7.312 1.184 6.18 0.000

T*F -18.125 -9.062 1.184 -7.65 0.000

T*S 16.625 8.313 1.184 7.02 0.000

F*S -1.125 -0.562 1.184 -0.48 0.647

T*F*S -1.625 -0.813 1.184 -0.69 0.512

S = 4.73682



Lecture 3.1 71



Minitab analyis



Lecture 3.1 72



Minitab analyis



Lecture 3.1 73


Identify optimal operating conditions

List means and SE's for 8 design points:

T*F*S Mean SE Mean

– – – 46.50 3.349

+ – – 68.00 3.349

– + – 74.00 3.349

+ + – 62.50 3.349

– – + 44.00 3.349

+ – + 102.00 3.349

– + + 72.50 3.349

+ + + 91.00 3.349

Postgraduate Certificate in Statistics Design and Analysis of Experiments

Lecture 3.1 74



Calculate confidence interval for optimum yield.

CI = 102.0 2.2 × 3.35 = ( 94.63 , 109.37 )

Classwork 3.1.7

Test the statistical significance of the difference

between the optimal yield and the 'next best' yield

Calculate a confidence interval for the 'next best'

yield.



Lecture 3.1 75



T*F*S Mean SE Mean

– – – 46.50 3.349

+ – – 68.00 3.349

– + – 74.00 3.349

+ + – 62.50 3.349

– – + 44.00 3.349

+ – + 102.00 3.349

– + + 72.50 3.349

+ + + 91.00 3.349


Lecture 3.1 76


Lecture 3.1


2. A 23 experiment




Lecture 3.1 77


Laboratory 1, Thursday February 4, 6-8



Lecture 3.1 78



• Students will work in pairs (or threes where

necessary) but not singly, with a view to

– exploiting synergy in solving Laboratory

problems,

– promoting collaborative learning.

• Laboratory tasks involve analysing, discussing

and reporting on the results of designed

experiments.

• Laboratory handouts give detailed guidance on

the use of Mintab.



Lecture 3.1 79



• The laboratory handouts include the following:

Invitations to consider the results of Minitab

analysis and their statistical and substantive

interpretations are printed in italics.

Take some time for this; consult your neighbour

or tutor.

Enter your responses in a Word document, as a

log of your work and as draft contributions to a

report on the experiments and their analyses.



Lecture 3.1 80



• The laboratory handouts include Learning

Objectives.

• Students should check these objectives as they

proceed through the Laboratory.

• At the end of each Laboratory, students are

invited to

– review the Learning Objectives

and

– check whether they have been achieved.


Lecture 3.1 81



• A Feedback document will be circulated at the

end of the Laboratory, containing

– solutions to the Laboratory tasks

– asides, constituting Extra Notes on relevant

topics.

• Laboratory handout will appear on the module

web page tomorrow,

• Feedback document will appear a week later.

• There will be a Minute Test at the end.



Lecture 3.1 82


Reading

EM §5.3, §5.4, §5.6

DCM §6-2, §6-3 to p.221, §6.5 to p. 237

(BHH § 5.14 (Lenth plots) and all of Ch. 5!)

Extra Notes:

Lenth’s PSE



design and analysis of experiments lecture 3...design and analysis of experiments lecture 3.1 1....

Documents