multiobjective optimization with desirability … optimization with desirability functions and...

Multiobjective Optimization with DesirabilityFunctions and Desirability Indices

Dr. Heike Trautmann

Statistics Faculty, TU Dortmund University

Drug Design Workshop, Leiden,21st of July, 2009

Desirability Index Robust DI Optimization Desirabilities vs. Pareto-Optimality DI Process Control References

Content

1 The Desirability Index (DI) as a method for multicriteria optimizationDesirability FunctionsDesirability Index

2 Robust Desirability Index optimizationThe Distribution of the DI

3 Desirabilities vs. Pareto-OptimalityDFs as a tool for incorporating preferences in EMOA

4 Process Control as a new application field for the DI

5 References

2 / 35


MCO methods and expert knowledge

Prior knowledge: Link criterion

Specification of expert knowledge by means of link criterion before

optimization

→ (Possibly) unambiguous solution

Examples: Desirability index, weighted sum

Posterior knowledge: Multiobjective Optimization

Optimization without usage of prior knowledge:

→ Set of optimal solutions

→ Selection of set of desired solutions after optimization based on

expert knowledge

Example: Pareto optimization

3 / 35


Desirability Index Example: Production of Fruit Juice

1. Which factors influence the process quality ?

X1 : Orange Juice (%), X2 : Pineapple Juice (%), X3 : Grapefruit J. (%)

2. By which factors can the process quality be measured?

Y1 : Content of Vitamin C (mg/l) Y1 = f1(X1, . . . ,X3)

Y2 : Overall content of fruit acid (g/l) Y2 = f2(X1, . . . ,X3)

Y3 : Relative Density of fruit ingredients Y3 = f3(X1, . . . ,X3)

3. How desirable are different values of the quality criteria?→ Specification of Desirability Functions di (Yi )

4 / 35



0 100 200 300 400

0.0

0.2

0.4

0.6

0.8

1.0

d 1

Content of Vitamin C (Y1)3 4 5 6 7 8

0.0

0.2

0.4

0.6

0.8

1.0

d 2

Overall content of fruit acid (Y2)

1.00 1.05 1.10 1.15 1.20 1.25 1.30

0.0

0.2

0.4

0.6

0.8

1.0

d 3

Relative Density of fruit ingredients (Y3)

5 / 35



4. Combination into an overall unitless quality criterion:

Desirability Index (DI) D := (3∏

i=1

di (Yi ))1/3

5. Maximization of the DI, i.e. overall process quality:

As Yi = fi (X1, . . . ,X3) :

D(X1, . . . ,X3) = (3∏

i=1

di [fi (X1, . . . ,X3)])1/3

X opt = maxX1,...,X3

D(X1, . . . ,X3) = (60, 17, 23)′

6. Process is set up based on optimal factor levels

6 / 35


Desirability Index (DI) Optimization

1. Influence factors: X1, . . . ,Xn

2. Quality criteria: Y1, . . . ,Yk with Yi = fi (X1, . . . ,Xn, εi )

3. How desirable are different values of the quality criteria (DFs)?

di (Yi )(i = 1, . . . , k), d : R→ [0, 1]

4. Combination into an overall unitless quality criterion (DI):

D := f (d1, . . . , dk), D : [0, 1]k → [0, 1]

5. Maximization of the DI as a function of influence factors:

maxX1,...,Xn

D(X1, . . . ,Xn) = k

√√√√ k∏i=1

di (fi (X1, ...,Xn, 0))

7 / 35


Main types of DFs (I)

Harrington (1965)

−3 −2 −1 0 1 2 3

0.0

0.2

0.4

0.6

0.8

1.0

di (Y′i ) = exp(−|Y ′i |ni )

Y ′i =2Yi − (USLi + LSLi )

USLi − LSLi

Derringer/Suich (1980)

0.0

0.4

0.8

Yi

d i

LSLi Ti USLi

li = 1 ri = 1

li = 0.2 ri = 2.5

di (Yi ) =

0, Yi < LSLi

( Yi−LSLi

Ti−LSLi)li , LSLi ≤ Yi ≤ Ti

( Yi−USLi

Ti−USLi)ri , Ti < Yi ≤ USLi

0, Yi > USLi

8 / 35


Main types of DFs (II)

Harrington (1965)

0.0

0.4

0.8

Yi

d i

●

●

●

●

Value pairs (Y(1)i , d

(1)i ), (Y

(2)i , d

(2)i )

di (Y′(j)i ) = exp(− exp(−Y

′(j)i ))

Y′(j)i = b0i + b1iY

(j)i , j = 1, 2.

Derringer/Suich (1980)

0.0

0.4

0.8

Yi

d i

USLiTi

ri = 1

ri = 0.5

ri = 3.5

di (Yi ) =1, Yi ≤ Ti

( Yi−USLi

Ti−USLi)ri , Ti < Yi < USLi

0, Yi ≥ USLi

9 / 35


The Desirability Index has become widely accepted

1. Different types of DIs:

Dg := (k∏

i=1

di )1/k , D :=

k∏i=1

di , Dmin := mini=1,...,k

di , D := 1/kk∑

i=1

di

2. Applications:

Mostly in chemistry, chemical and mechanical engineering

Optimization of manufacturing-, production- or chemical processes:Examples: Optimization of a force balance in wind tunnel tests,Optimization of solid-state bioconversion of wheat straw, . . .

3. Remarks:

Mostly Derringer-Suich DFs and the geometric mean as a DI

Balanced ratio of numerical and graphical optimization techniques

10 / 35


The Desirability Index in noisy environments

The DI is a random variable

DI maximization using the models linking quality criteria to influencefactors:

classical approach:

maxX1,...,Xn

D(X1, . . . ,Xn) = k

√√√√ k∏i=1

di (fi (X1, ...,Xn, 0))

ideal approach → Robust DI Optimization:

maxX1,...,Xn

E [D(X1, . . . ,Xn)] = E

k

√√√√ k∏i=1

di (fi (X1, ...,Xn, εi ))

with in general εi ∼ N (0, σ2

i ), i = 1, . . . , k

11 / 35


Example of Derringer/Suich (1980)

Tire Tread Example

Quality Criteria:

Y1: PICO Abrasion Index

Y2: 200 % - Modulus

Y3: Elongation at Break

Y4: Hardness

Influence factors (phr: parts per hundred):

X1: (phr silica − 1.2)/0.5

X2: (phr silane − 50)/10

X3: (phr sulfur − 2.3)/0.5

12 / 35


Tire Tread Example

Mathematical models

Y1(X ) = 139.1 + 16.5X1 + 17.9X2 + 10.9X3 − 4.0X 21 − 3.5X 2

2 − 1.6X 23

+5.1X1X2 + 7.1X1X3 + 7.9X2X3; σ1 = 5.6,

Y2(X ) = 1261.1 + 268.2X1 + 246.5X2 + 139.5X3 − 83.6X 21 − 124.8X 2

2

+199.2X 23 + 69.4X1X2 + 94.1X1X3 + 104.4X2X3; σ2 = 328.7,

Y3(x) = 400.4− 99.7X1 − 31.4X2 − 73.9X3 + 7.9X 21 + 17.3X 2

2

+0.4X 23 + 8.8X1X2 + 6.3X1X3 + 1.3X2X3; σ3 = 20.6,

Y4(X ) = 68.9− 1.4X1 + 4.3X2 + 1.6X3 + 1.6X 21 + 0.1X 2

2 − 0.3X 23

−1.6X1X2 + 0.1X1X3 − 0.3X2X3; σ4 = 1.27

−1.633 ≤ Xi ≤ 1.633.

13 / 35


Tire Tread Example

Desirability Functions

PICO Abrasion 200 % Modulus

120 140 160

0.0

0.4

0.8

Value

De

sir

ab

ility

1000 1150 1300

0.0

0.4

0.8

Value

De

sir

ab

ility

Elong. at Break Hardness

400 500 600

0.0

0.4

0.8

Value

De

sir

ab

ility

60 65 70 75

0.0

0.4

0.8

Value

De

sir

ab

ility

14 / 35


Tire Tread Example

Classical DI optimization (without noise)

Maximization of geometric mean:

maxX∈[−1.633,1.633]

D(Y ) =4∏

i=1

Yi

Optimum found:

X = (−0.05, 0.145,−0.868)

Y = (129.5, 1300, 465.7, 68)

d(Y ) = (0.189, 1, 0.656, 0.932)

D = 0.583

→ acceptable desirability

15 / 35


Tire Tread Example

How robust is the optimum?

10000 realisations of quality criteria using optimal factor setting:

Yi = fi (X1, . . . ,X4) + εi , εi ∼ N (0, σ2i )

Y1

De

nsity

110 130 150

0.0

00

.03

0.0

6

Y2

De

nsity

0 500 1500 25000.0

00

00

.00

06

0.0

01

2

Y3

De

nsity

400 450 500 550

0.0

00

0.0

10

Y4

De

nsity

64 66 68 70 72

0.0

00

.15

0.3

0

16 / 35


Tire Tread Example


Desirability Index:

D

Density

0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

Desirability Functions:

d1

De

nsity

0.0 0.2 0.4 0.6

0.0

1.0

2.0

3.0

d2

De

nsity

0.0 0.4 0.8

02

46

8

d3

De

nsity

0.0 0.4 0.8

0.0

1.0

2.0

d4

De

nsity

0.4 0.6 0.8 1.0

01

23

417 / 35


Tire Tread Example


Empirical distributionfunction of D

0.0 0.1 0.2 0.3 0.4 0.5 0.6

0.0

0.2

0.4

0.6

0.8

1.0

D

Fn(x

)

Summary:

min(D) = 0

q0.25(D) = 0.1097

med(D) = 0.1872

mean(D) = 0.1898

q0.75(D) = 0.2633

max(D) = 0.5757

sd(D) = 0.108

But: D(Xopt)) = 0.583

→ not realistic in the courseof the process!!

18 / 35


The Desirability Index in noisy environments

Robust DI Optimization

⇓

Optimization of E (D) instead of D

→ Derivation of the distribution of the DI

Yi = fi (x) + εi , εi ∼ N (0, σ2i )

⇒ Yi ∼ N (fi (x), σ2i )

⇒ Distribution of DFs

⇒ Distribution of DI

19 / 35


Distribution of the DI (Geometric Mean) (I)

1. One-Sided Specification:

Density Function:

fD(D) ≈ − 1√2π · σ∗ · log(D) · D

· exp

[− 1

2σ∗2(log(−k · log(D))− µ∗)2

]

Distribution Function:

FD(D) ≈ 1− Φ

[log(k) + log(− log(D))− µ∗

σ∗

]Qα ≈ exp(− exp(σ∗ · z1−α − log(k) + µ∗)) with

med(fD(D)) ≈ exp[− exp(µ∗)/k]

20 / 35


Distribution of the DI (Geometric Mean) (II)

2. Two-Sided Specification:

Density Function

fZ (z) =

√2

4√

π(σ22 + σ1

2)·

exp

− (z − µ1 − µ2)2

2(σ22 + σ1

2)

erf

(zσ22 − µ1σ2

2 + µ2σ12)

σ2σ1√

2√

σ22 + σ1

2

+ exp

− (z − µ1 + µ2)2

2(σ22 + σ1

2)

erf

(zσ22 − µ1σ2

2 − µ2σ12)

σ2σ1√

2√

σ22 + σ1

2

+ exp

− (z + µ1 − µ2)2

2(σ22 + σ1

2)

erf

(zσ22 + µ1σ2

2 + µ2σ12)

σ2σ1√

2√

σ22 + σ1

2

+ exp

− (z + µ1 + µ2)2

2(σ22 + σ1

2)

erf

(zσ22 + µ1σ2

2 − µ2σ12)

σ2σ1√

2√

σ22 + σ1

2

+ exp

− (z − µ1 − µ2)2

2(σ22 + σ1

2)

erf

(zσ12 + µ1σ2

2 − µ2σ12)

σ2σ1√

2√

σ22 + σ1

2

+ exp

− (z − µ1 + µ2)2

2(σ22 + σ1

2)

erf

(zσ12 + µ1σ2

2 + µ2σ12)

σ2σ1√

2√

σ22 + σ1

2

+ exp

− (z + µ1 − µ2)2

2(σ22 + σ1

2)

erf

(zσ12 − µ1σ2

2 − µ2σ12)

σ2σ1√

2√

σ22 + σ1

2

+ exp

− (z + µ1 + µ2)2

2(σ22 + σ1

2)

erf

(zσ12 − µ1σ2

2 + µ2σ12)

σ2σ1√

2√

σ22 + σ1

2

with erf (x) = 2 · Φ(√

2x)− 1 (Gaussian Error Function).

21 / 35


Visualization

One-Sided Specification

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0

2.5

3.0

Wünschbarkeitsindex D

f D(D

) (1) (2)

0.0 0.2 0.4 0.6 0.8 1.00

12

34

5


f D(D

) (3)

(4)(5)

0.0 0.2 0.4 0.6 0.8 1.0

01

23

45


f D(D

)

(6)

(7)(8)

Two-Sided Specification

0.0 0.2 0.4 0.6 0.8 1.00.0

1.0

2.0

3.0

D

f D(D

)

22 / 35


Distribution of the DI (Minimum of the DFs)

D := mini=1,...,kdi ⇒ FD(D) = 1−k∏

i=1

[1− FDi (di )]

1. One-Sided Specification:

fD(D) = −1

D · log(D)

k∑i=1

1

σiφ

(log(− log(D)) − µi

σi

) k∏j=1j 6=i

Φ

(log(− log(D)) − µj

σj

)2. Two-Sided Specification:

fD(D) =

−k∑

i=1

((− log(D))1/ni

niD log(D)σi

(φ

[(− log(D))1/ni − µi

σi

]+ φ

[(− log(D))1/ni + µi

σi

])

·k∏

j=1j 6=i

(−1 + Φ

[(− log(D))1/nj − µj

σj

]+ Φ

[(− log(D))1/nj + µj

σj

]))23 / 35


Visualization

One-Sided Specification

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0

2.5

3.0


f D(D

) (1) (2)

0.0 0.2 0.4 0.6 0.8 1.0

01

23

45

6Wünschbarkeitsindex D

f D(D

)

(3)

(4)

(5)

0.0 0.2 0.4 0.6 0.8 1.0

01

23

45


f D(D

)

(6)

(7)

(8)

Two-Sided Specification

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0

2.5

3.0


f D(D

) (1)

(3)(2)

0.0 0.2 0.4 0.6 0.8 1.0

01

23

45

67


f D(D

)

(4)

(6)

(5)

0.0 0.2 0.4 0.6 0.8 1.00.

00.

51.

01.

52.

0Wünschbarkeitsindex D

f D(D

)

(7)

(8)

(9)

24 / 35


Desirabilities vs. Pareto-Optimality

Prior knowledge: Link criterion

Specification of expert knowledge by means of link criterion before

optimization

→ (Possibly) unambiguous solution

Examples: Desirability index, weighted sum

Posterior knowledge: Multiobjective Optimization

Optimization without usage of prior knowledge:

→ Set of optimal solutions

→ Selection of set of desired solutions after optimization based on

expert knowledge

Example: Pareto optimization

25 / 35


Desirabilities vs. Pareto-Optimality

Pareto-Optimality

A realization of quality criteria (QC) Y = (Y1, . . . ,Yk)′ is said to bepareto-optimal if there is no other realization that keeps up the processquality regarding all criteria and improves at least one criterion.

→ The process cannot be improved upon without deteriorating at leastone quality criterion.

A corresponding influence factor setting X = (X1, . . . ,Xn)′ then ispareto-optimal in factor space if the corresponding criteria vector Y ispareto-optimal in criteria space.

Problems:Determination of the Pareto-Set, Selection of optimal factor setting

26 / 35


Pareto-Optimality of the optimal solution

Geometric Mean

The optimized factor levels based on the DI are pareto-optimal

The DI can be understood as a method for selecting apareto-optimal solution from the Pareto-Set.

Minimum of DFs

Pareto-Optimality of X opt is not guaranteed.

But: easy to interprete

The applying experts have to be aware of this“non-pareto-optimality“

27 / 35


DFs as a tool for incorporating preferences in EMOA

Desirability MCO problem:

Minimize −d(Y ) = −d [f (X )] = −(d1[f1(X )], . . . , dk [fk(X )])

with Y = (Y1, . . . ,Yk) ∈ Y, objectives

X = (X1, . . . ,Xn) ∈ X , decision variables

di ∈ [0, 1] i = 1, . . . , k , desirability functions

fi , i = 1, . . . , k . mathematical models

→ Focussing on relevant parts of the Pareto front becomes possible

28 / 35


Binh-problem, Harrington DFs (d1,d2)

NSGA-II, µ = 200, ]FE = 10000

0 10 20 30 40 500.0

0.2

0.4

0.6

0.8

1.0

Y

d(Y

)

●

●

●

●

0 10 20 30 40 500.0

0.2

0.4

0.6

0.8

1.0

Y

d(Y

)

●

●

●

●

0 10 30 500.0

0.2

0.4

0.6

0.8

1.0

Y

d(Y

)

●

●

●

●

0 10 20 30 40 50

010

2030

4050

f1

f 2

0 10 20 30 40 50

010

2030

4050

f1

f 2

0 10 20 30 40 500

1020

3040

50f1

f 2

29 / 35


The Desirability Index for Process Control?

DI was already used for Process Optimization→ Ideal measure for evaluating the “degree of optimality“ over time

Interpretation of Out-Of-Control Signals possible→ Control limits for the DI can be transferred back to DFs

Complexity reduction compared to separate univariate control chartsfor quality criteria

Exemplary Control Chart

0 20 40 60 80 100

0.2

0.4

0.6

0.8

1.0

t

UCL

T

LCL

Out−Of−Control

Out−Of−Control

30 / 35


Suitable Control Charts for the Desirability Index

Problem: Most univariate control charts assume normality!

→ Adapted Single-Measurements-Control Chart :

LCL/UCL = Q0.005/Q0.995 resp. LWL/UWL = Q0.025/Q0.975,

Examples

0 50 100 150 2000.2

0.4

0.6

0.8

t

D

LCL

UCL

0 50 100 150 200

0.2

0.4

0.6

0.8

Wünschbark

eitsin

dex (

D)

LCL

UCL

31 / 35


Extreme-Value-Chart for DI (Sample Size g)

Control Limits: LCL = Q(1− g√0.99)/2, UCL = Q(1+ g√0.99)/2

Warning Limits: LWL = Q(1− g√0.95)/2, UWL = Q(1+ g√0.95)/2

Example

1 2 3 4 5 6

0.4

0.6

0.8

1.0

t

D

LCL

UCL

32 / 35


Thank you very much!

33 / 35


References

Harrington, J. (1965): The desirability function, Industrial QualityControl 21 (10), pp. 494 - 498

Derringer, G.C. and Suich, D. (1980): Simultaneous optimization ofseveral response variables, Journal of Quality Technology 12 (4), pp. 214 -219

Trautmann, H. and Mehnen, J. (2008): Preference-BasedPareto-Optimization in Certain and Noisy Environments; EngineeringOptimization 41, pp. 23-28

Mehnen, J., Trautmann, H. and Tiwari, A. (2007): Introducing UserPreference Using Desirability Functions in Multi-Objective EvolutionaryOptimisation of Noisy Processes. CEC 2007, IEEE Congress onEvolutionary Computation, pp. 2687-2694, Kay Chen Tan, Jian-Xin Xu(eds.), Singapore, 2007.

34 / 35


References

Mehnen, J. and Trautmann, H. (2006): Integration of Expert’sPreferences in Pareto Optimization by Desirability Function Techniques;In: Proceedings of the 5th CIRP International Seminar on IntelligentComputation in Manufacturing Engineering (CIRP ICME ’06), Ischia,Italy, R. Teti (ed.), pp. 293-298

Trautmann, H. and Weihs, C. (2006): On the Distribution of theDesirability Index using Harrington’s Desirability Function, Metrika 63 (2),pp. 207-213

Trautmann, H. (2004a): Qualitatskontrolle in der Industrie anhand vonKontrollkarten fur Wunschbarkeitsindizes - AnwendungsfeldLagerverwaltung; Dissertation at Dortmund University,http://hdl.handle.net/2003/2794

Trautmann, H. (2004b): The Desirability Index as an Instrument forMultivariate Process Control; Technical Report 43/04, SFB 475,Dortmund University.

35 / 35

multiobjective optimization with desirability … optimization with desirability functions and...

Documents