doe - design of experiments - case study

Design Of ExperimentsExample: case study I

(7.1)DOE - KVL(031023)

Day 1 1. 13:00-13:45h - Introduction; what was experimental design again? 2. 14:00-14:45h - Design in one formula: y = X.b3. 15:00-15:45h - Statistical inference and ANOVA

Day 2 4. 09:00-12:00h - Hands-on DOE: computer exercise I *)

5. 13:00-13:45h - Data inspection and plotting (+ some PCA) 6. 14:00-14:45h - Miscellaneous subjects 7. 15:00-15:45h - Example: case study I

Day 3 8. 09:00-12:00h - Hands-on DOE: computer exercise II *)

9. 13:00-13:45h - Blocking and split-plot10. 14:00-14:45h - Example: case study I11. 15:00-15:45h - Introduction to mixed linear models

*) Participants can choose to take part in the theory (3 afternoons) sessions or theory + computer exercise (2 mornings) sessions.

Case study IDesign Of Experiments

(7.2)DOE - KVL

Case study IColor change during meat storage

(7.3)DOE - KVL

Design Of ExperimentsDifferent research areas

(7.4)

• Replicates are ‘perfect clones’• Experiments are cheap• Simulations give ‘perfect statistics’

DOE - KVL

• Motivation for a lot of developments• Reasonably reproducible• Sparse designs• THE example in literature

• Natural variation• Experiments are expensive • More replicates• More input of the experimenter


(7.5)DOE - KVL

Loss of redness of pork-meat during storage time under different packaging conditions

© Lisbeth Dahlgård Nannerup, MLI/LMC Kvali-MAP project, Department of Food Chemistry, KVL, Denmark

• Response variable is the color (redness) of the meat

• Measured as a so-called a-value

• Higher redness is more attractive towards consumers (optimization)


(7.6)DOE - KVL

The effects investigated are packaging conditions

• Storage days (12 levels)

• Percentage oxygen (4)

• Product/Headspace ratio (3)

• Temperature (3)

• Light exposure (3)

8 replicates ( )


(7.7)DOE - KVL

The experiment is a five (or six if you count replicates) dimensional factorial designDays x Oxygen x Product/Headspace x Temperature x Light (x Animals)

1D 2D 3D … 6D

?


(7.8)DOE - KVL

0.0 0.5 1.0 1.5 O2 (%)

1:1

P/H 1:1.5

1:3

10T(°C) 8

5

L (Lux)0 600 1200

t (days)0 21 34

• 4 x 3 x 3 x 3 x 8 = 864 samples• 864 x 12 = 10368 measurements

Color loss in porkQuality of the response

(7.9)DOE - KVL

• Each measurement outcome is based on 5 readings• Minolta color measurement (a, b and L-value)• 8mm diameter serves area• a-value is used to express redness of meat

0 5 10 15 20 25 30 350

10

day

a-va

lue

Standard deviation over readings (σ)

Color loss in porkQuality of the response

(7.10)DOE - KVL

0

3

σ Sam

ples

0 5 10 15 20 25 30 351

2

day

σ poo

led

σpooled all samples = 1.59

Measurement error :1.58/√5 = 0.7 a-values

10 (randomly selected) samples are used to computethe pooled standard deviation

Color loss in porkReplicates

(7.11)DOE - KVL

-5

0

5

10

O2 = 0.0% P/H = 1:1 T = 5°C L = 0Lux

0 5 10 15 20 25 30 35-5

0

5

10 O2 = 1.5% P/H = 1:3 T = 10°C L = 1200Lux

day

a-va

lue

a-va

lue

Two design points

x 8

Color loss in porkReplicates?

(7.12)DOE - KVL

First experimental run

2

8

0 352

8

0 35day

a-va

lue

day

a-va

lue

Second experimental run

x2 per production runMean a-value over

all design points

Storage (days)

Color loss in porkExpected trends

(7.13)DOE - KVL

Mean a-value overall design/time points

0.0 0.5 1.0 1.52

8

1:1 1:1.5 1:3

5 102

8

0 600 12008

Light (Lux)

a-va

lue

a-va

lue

Product/Headspace

Temperature (°C)

Oxygen (%)

Color loss in porkResponse surface model for day 21

(7.14)DOE - KVL

Storage time in the setup of this experiment requires special methods.We will treat the data from day 21 as single design (108 points x 8 replicates)

0.0 0.5 1.0 1.5 O2 (%)

1:1

P/H 1:1.5

1:3

10T(°C) 8

5

L (Lux)0 600 1200 Day 21


(7.15)DOE - KVL

An ANalysis Of Variance shows that the temperature as main effect and interactions are not important.

√√X√√X√X√XX√XXX

Color loss in porkResponse surface model for day 2, 21 and 34

(7.16)DOE - KVL


Day 2 Day 21 Day 34


(7.17)DOE - KVL



(7.18)DOE - KVL

a-value = b0 + b1 xO2 + b2 xP/H + b3 xL + b4 xO2 xP/H + b5 xO2 xL + b6 xP/H xL + b7 xO2.P/H xL

-0.8x10-3-1.6x10-3-1.2x10-3b7 (O2.P/H.L)

-0.0x10-3-0.7x10-3-0.4x10-3b6 (P/H.L)

2.2x10-30.8x10-31.5x10-3b5 (O2.L)

0.20-0.34-0.07b4 (O2.P/H)

0.5x10-3-0.8x10-3-0.2x10-3b3 (L)

-0.05-0.56-0.30b2 (P/H)

0.48-0.63-0.08b1 (O2)

11.8910.8511.37b0

Confidence interval

=y = X.b b = (X’.X)-1.X.y=

Color loss in porkModel residuals for day 21

(7.19)DOE - KVL

-6 -4 -2 0 2 4 -6 -4 -2 0 2 40.001

0.01

0.05 0.10 0.25

0.50

0.75 0.90 0.95

0.99

0.999

Residuals (mildly) skewed towards low values, but for now we assume ANOVA is ‘robust’ against this non-normality

Color loss in porkModel residuals for day 21

(7.20)DOE - KVL

0 10-10

0

10

0 10-10

0

10

Product/Headspace1:11:1.51:3

Light0

6001200

Color loss in porkResponse surface for day 21

(7.21)DOE - KVL

1:1 1:30

0.5

1

1.5

P/H

O2

(%)

Light = 0Lux

6.2

6.4

6.4

6.4

6.6

6.6

6.6

6.8

6.8

6.8

7

1:1.5


(7.22)DOE - KVL

3.63.84

4.2

4.44.4

4.6

4.6

4.8

4.85

5

55.2

5.2

5.25.4

5.4

5.45.6

5.65.6

5.65.8

5.85.8

5.86

66

6.26.2

6.2

6.46.4

6.4

6.66.6

6.6

1:1 1:30

0.5

1

1.5

P/H

O2

(%)

Light = 600Lux

1:1.5


(7.23)DOE - KVL

11.21.41.61.82

2.2

2.42.4

2.62.6

2.82.8

33

3.23.2

3.43.4

3.6

3.6

3.63.8

3.8

3.84

4

44.2

4.2

4.24.4

4.4

4.44.6

4.6

4.64.8

4.84.8

4.85

55

55.2

5.25.2

5.4

5.4

5.4

5.65.6

5.6

5.85.8

5.8

66

6

6.26.2

6.2

6.4

6.4

6.4

6.6

1:1 1:30

0.5

1

1.5

P/H

O2

(%)

Light = 1200Lux

1:1.5


(7.24)DOE - KVL

1:1 1:1.51:3 0.0

0.51.0

1.51

8

Light = 1200Lux

Light = 600Lux

Light = 0Lux

a-va

lue

Oxygen (%)

Product/Headspace


(7.25)DOE - KVL

1:1 1:1.51:3 0.0

0.51.0

1.51

8

Light = 1200Lux

Light = 600Lux

Light = 0Lux

a-va

lue

Oxygen (%)

Product/Headspace


(7.26)DOE - KVL

1:1 1:1.51:3 0.0

0.51.0

1.51

8

Light = 1200Lux

Light = 600Lux

Light = 0Lux

a-va

lue

Oxygen (%)

Product/Headspace

Color loss in porkResponse surface for days 2, 21 and 34

(7.27)DOE - KVL

Day 2 Day 21 Day 34

A workable model of for all the design factors, supported by inspecting the collecting (raw) data, and…

‘Models are to be used, not believed’ (Henri Theil)

Data transformationsNon-normal residuals?

(7.28)DOE - KVL

-6 -4 -2 0 2 40.001

0.01 0.05 0.10 0.25 0.50 0.75 0.90 0.95 0.99

0.999

21)constantlog(

xyxyx

y

xy

sbsandxbaythen

bxayif

xy

===

+=

=

+=+=

Log-normal distributions are often fond in nature (weight, size, etc.), time-series andgrowth models

y = b0exp(b1x) ln(y) = ln(b0) + b1xy’ = b0’ + b1x

(linear transformation leave the residual distributions unattached)

Data transformationVariance stabilizing

(7.29)DOE - KVL

0.5 1 1.5 2 2.5 3 3.5 4 4.5 5λ

SSe

Sometimes a so-called Box-Cox transformation of the form y = (x + constant)λcan help to ‘stabilize’ (make more similar) the model errors

The ‘optimal’ transformation is the one that minimizes the sum-of-squares of residuals as a function of λ(Maximum Likelihood):

( )

( )nxex

xxyxxy

/ln

1

)ln()0(

)1(

∑=

=

−= −λ

λ

λλ

Color loss in porkDay 21 after transformation

(7.30)DOE - KVL

-80 -60 -40 -20 0 20 40 60 80 -80 -60 -40 -20 0 20 40 600.001

0.01

0.05 0.10 0.25

0.50

0.75 0.90 0.95

0.99

0.999


(7.31)DOE - KVL

Beforetransformation


(7.32)DOE - KVL

0 100 200-100

0

100

200

0 100 200-100

0

100

200

Product/Headspace1:11:1.51:3

Light0

6001200


(7.33)DOE - KVL

1:1 1:1.51:3 0.0

0.51.0

1.51

8

Light = 1200Lux

Light = 600Lux

Light = 0Lux

a-va

lue

Oxygen (%)

Product/Headspace

GEMANOVAMultiplicative-model analysis of the DOE-data

(7.34)

GEneralized Multiplicative ANalysis Of VAriannce (GEMANOVA)

Classical ANOVA - Additive model:color = b0 + b1 xO2 + b2 xP/H + b3 xL + b4 xO2.P/H + b5 xO2.L + b6 xP/H.L + b7 xO2.P/H.L

GEMANOVA - Multiplicative model:color = c0 cA ct cO2 cP/H cT cL

DOE - KVL

GEMANOVA = PARAFACParallel factor analysis

(7.35)

X E= + +

X E= + +

Principal Component Analysis (PCA)

PARAFAC

A factor model in three (or more!) dimensional space

with scores/loadings in three (or more) directions

DOE - KVL

GEMANOVAOne factor model, all effects free

(7.36)

c0 = 1092color = c0 cA ct cO2 cP/H cT cL

2 4 6 8

0.33

0.37

0 10 20 30

0.28

0.29

0 0.5 1 1.5

0.49

0.52

1:1 1:1.5 1:30.52

0.60

5 8 10

0.577

0.579

600 1200

0.56

0.62

0

Animal

Storage (days)

Prod./Headsp.

Light (Lux)Temperature (°C)

Oxygen (%)

DOE - KVL

GEMANOVAOne factor model, all effects free

(7.37)

color = c0 cA ct cO2 cP/H cT cL + e

Data range = [0.98 – 14.82]; RMSPfit = 1.47 color-values; R2 = 0.41

-6 -4 -2 0 2 4

DOE - KVL

Error

Jackknife re-samplingUncertainty estimation

(7.38)

( )( ) ( )

( ) ( )

( ) ( )θθθθ

θ

θθθ

αθ

θ

θ

αα

αα

ˆˆˆ

ˆˆ

ˆˆ

1)ˆ(

ˆˆ

*

***

^

2/1;

^

2/;

2/12/

−⇔−

==

−≤≤−

−=≤≤

==

=

−

−

x

x

uFt

setset

ssP

uFt

Ft

dfdf

F : Cumulative Distribution Function

x : data; plug-in principle

α-coverage probability

Uncertainty estimate

Estimates from new distributionfound from some re-sampling strategy (*)

The re-sampling assumption!

DOE - KVL

Jackknife re-samplingUncertainty estimation

(7.39)

( ) ( )( )( ) ( )

( ) ( )^

2/1

^

2/

2*(.)

*)(

^

*(.)

^

ˆˆ

ˆˆ1

ˆˆ1

sezsez

nnse

nbias

iJ

J

αα θθθ

θθ

θθ

−−≤≤−

−−

=

−−=

∑

n = #samples (= 8 animals) (.) : re-sampling expectation

Jackknife bias estimate

Jackknife standard error estimate

Assume normal distribution

DOE - KVL

GEMANOVAOne factor model, Jackknife 5% coverage probability

(7.40)

color = c0 cA ct cO2 cP/H cT cL c0 = 1052 - 1092 - 1133

2 4 6 8

0.32

0.38

0 10 20 300.27

0.29

0 0.5 1 1.50.48

0.52

1:1 1:1.5 1:3

0.52

0.60

5 8 10

0.57

0.58

0 600 12000.54

0.62

Anim

al

Storage (days)

Prod./Headsp.

Light (Lux)T (°C)

Oxygen (%)

DOE - KVL

RMSPfit = 1.47R2 = 0.41

No effect!

GEMANOVAOne factor model, Temperature effect eliminated

(7.41)

color = c0 cA ct cO2 cP/H 1T cL c0 = 607 - 631 - 654

2 4 6 8

0.32

0.38

0 10 20 300.27

0.29

0 0.5 1 1.50.48

0.52

1:1 1:1.5 1:3

0.52

0.60

5 8 10

1

0 600 12000.54

0.62

Anim

al

Storage (days)

Prod./Headsp.

Light (Lux)

Oxygen (%)

Temperature (°C)

RMSPfit = 1.47R2 = 0.41(same performance)

DOE - KVL

doe - design of experiments - case study

Business

h design

doe kvlmean

doe kvldesign

doe kvlloss

doe kvl031023day

doe kvl03samples0

h example

h introduction