generating and comparing pareto fronts of experiment ... · 3 one weird trick to make your...
TRANSCRIPT
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Generating and Comparing Pareto Fronts ofExperiment Designs to Simultaneously Account
for Multiple Experimental Objectives
Byran Smucker
Department of Statistics
Miami University, Oxford, OH
Joint work with Yongtao Cao and Tim Robinson
December 16, 2015
DEMA2015, Sydney, Australia
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Overview
1 Multiple criteria are important in optimal design
2 The weighted sum approach may not work
3 A hybrid algorithm for multiobjective design
4 Conclusion
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Clickbait
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Clickbait
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Clickbait
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Clickbait
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Overview: Clickbait version
1 They considered multiple criteria simultaneously: whathappened next will shock you
2 After you see this table you’ll never use the weighted sumapproach again
3 One weird trick to make your multi-objective designconstruction a breeze
4 He concluded his talk; what followed brought the audienceto tears
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Overview: Clickbait version
1 They considered multiple criteria simultaneously: whathappened next will shock you
2 After you see this table you’ll never use the weighted sumapproach again
3 One weird trick to make your multi-objective designconstruction a breeze
4 He concluded his talk; what followed brought the audienceto tears
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Overview: Clickbait version
1 They considered multiple criteria simultaneously: whathappened next will shock you
2 After you see this table you’ll never use the weighted sumapproach again
3 One weird trick to make your multi-objective designconstruction a breeze
4 He concluded his talk; what followed brought the audienceto tears
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Overview: Clickbait version
1 They considered multiple criteria simultaneously: whathappened next will shock you
2 After you see this table you’ll never use the weighted sumapproach again
3 One weird trick to make your multi-objective designconstruction a breeze
4 He concluded his talk; what followed brought the audienceto tears
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Overview: Clickbait version
1 They considered multiple criteria simultaneously: whathappened next will shock you
2 After you see this table you’ll never use the weighted sumapproach again
3 One weird trick to make your multi-objective designconstruction a breeze
4 He concluded his talk; what followed brought the audienceto tears
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Outline
1 Multiple criteria are important in optimal design
2 The weighted sum approach may not work
3 A hybrid algorithm for multiobjective design
4 Conclusion
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Multiple criteria: popping up recently in the designliterature
Jones and Nachtsheim (2011, 1), Jones and Nachtsheim (2011,2): aliasing vs. D-efficiency
Lu, Anderson-Cook, and Robinson (2011): algorithms to handlemore than one criteria
Gilmour and Trinca (2012): lack-of-fit vs. pure error estimation
Sambo, Borrotti, and Mylona (2014): D-optimal vs. I-optimal
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Typical approaches used for multiobjective design
1. Optimizing a linear combination of the criteria
φ(ξN ) = λφ1(ξN ) + (1 − λ)φ2(ξN )
2. Construct a Pareto front, a set of non-dominated designs.
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Dominated/Non-dominated Designs
f2
f1
Pareto
Dominated
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Example 1: Pareto front (PF)
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Example 2: Pareto front
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Outline
1 Multiple criteria are important in optimal design
2 The weighted sum approach may not work
3 A hybrid algorithm for multiobjective design
4 Conclusion
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Example 3: Pareto front
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Weighted sum approach and nonconvex parts of the PF
It has been proven theoretically that the weighted sum won’tdetect elements in a nonconvex part of the Pareto front (Dasand Dennis 1997).
Empirically, this does not seem quite as clear-cut.
Still, point 2 is in trouble.
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Example 3 Pareto front
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Outline
1 Multiple criteria are important in optimal design
2 The weighted sum approach may not work
3 A hybrid algorithm for multiobjective design
4 Conclusion
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
A Hybrid Framework
Hybrid algorithms are becoming increasingly popular in themulti-objective optimization literature.
Our proposal: Combine the coordinate exchange operator withan elitism operator from evolutionary algorithms.
Call it the Elitist Pareto-based Coordinate Exchange Algorithm(EPCEA).
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Pareto-based coordinate exchange operator
At each possible exchange, make two comparisons:
1 Does the new design dominate the old one?
2 Does the new design belong in the current Pareto set ofdesigns?
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
EPCEA Algorithm
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Existing Algorithms
Multi-objective genetic algorithm (MOGA); Park (2009)
Pareto aggregate point exchange (PAPE) algorithm; Lu,Anderson-Cook, and Robinson (2011)
Weighted sum approach: a coordinate-exchange two-phase localsearch algorithm (CE-TPLS); Sambo, Borrotti, and Mylona(2014).
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Example 3 Pareto front
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Example 3 Pareto front
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Example 3 Pareto front
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Example 3 Pareto front
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Example 3 Pareto front
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Example 3 Pareto front
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Three-dimensional example
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Hypervolume Indicator in 2D
Contribution Rate (CR) =IH
(PF
′i ,r
)IH(PFS ,r)
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Comparing 3-D Pareto fronts
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Comparing 3-D Pareto fronts
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Comparing 3-D Pareto fronts
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Comparing 3-D Pareto fronts
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Comparing 3-D Pareto fronts
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Outline
1 Multiple criteria are important in optimal design
2 The weighted sum approach may not work
3 A hybrid algorithm for multiobjective design
4 Conclusion
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Recap
1 Multiple criteria are important to consider in optimaldesign - shocking development?
2 The weighted sum approach won’t necessarily find allelements of a Pareto front - that one table
3 Hybrid algorithm - one weird trick
Remaining questions: More powerful hybrid algorithms? Howto choose a design to run from the Pareto front?
Cao et al. Pareto Fronts of Designs
Multiple criteria are important in optimal designThe weighted sum approach may not work
A hybrid algorithm for multiobjective designConclusion
Papers
Cao, Y., Smucker, B.J., and Robinson, T.J. (2015) “On usingthe hypervolume indicator to compare Pareto fronts:Applications to multiple optimal experiment design.” Journalof Statistical Planning & Inference. 160:60-74.
Cao, Y., Smucker, B.J., and Robinson, T.J. “A Hybrid ElitistPareto-based Coordinate Exchange Algorithm for ConstructingMulti-Criterion Optimal Experimental Designs.” Tentativelyaccepted to Statistics & Computing.
Cao et al. Pareto Fronts of Designs