current challenges and the application of computer ...dae/pdf/d2 william myers.pdf · innovation...
TRANSCRIPT
Current Challenges and the Application of Computer Experiments in Industry
William Myers
The Procter & Gamble Company
DAE 2015: Design & Analysis of Experiments Conference – Cary, NC
2
Outline
The Business Value of Computer Experiments in Industry Brief Introduction to Computer Experiments Unique Challenges in Industry
o Design o Modeling
• Industry Example
3
Competitive Advantage in Industry
In a highly competitive and complex product development environment, reliance on computer experiments is crucial - often allow faster innovation and cheaper development costs
o When a series of design decisions must be made upfront prior to the
availability of a physical prototype o Physical experimentation can often be too time consuming and too
expensive or even impossible
“explore and develop virtually and confirm physically”
4
Simulates the Performance of a Package on Drop Testing
5
Simulates the Performance of a Package on a Standard Conveying Line
Baseline (original) Option 2, best option
7
Computer Experiments Computer simulations are often computationally intensive and can take
too much time for the purpose of sensitivity analysis or product design optimization - a single computer code run can take from an hour to several days
The goal of computer experiments is to find an approximate model –
metamodel approach
9
Modeling Computer Experiments
The Gaussian Process (GP) (ordinary kriging) model is a popular choice for use as a metamodel for computer experiments
They are flexible – fit a wide variety of surfaces GP models interpolate the data (adopted from the spatial statistics literature) GP models treat the deterministic output response as a realization of a random
stochastic process
( ) ( )Y x Z xµ= +
2( ) ~ (0, )Z x GP Rσ
2( , ) exp ( )ij k ik jkk
R X x xθ θ = − − ∑
10
Design Challenges
Design Region Constraints
Good Projection Properties Design with Qualitative Variables
Sample Size
11
Important Design Properties for Computer Experiments
Good Space-filling Properties o Spread points evenly throughout the design region (help to bound the bias and impact
predictive accuracy of surrogate model) o Assume no prior knowledge (model form) about the underlying system o Have the ability to fit a variety of models
Large number of levels for each input factor Good Projection Properties (minimize the average reciprocal distance)
o Not all the factors will be important – Effect Sparsity o Consider the projections of the design points onto its subspaces
o “Non-collapsing” – space-filling in lower dimensional projections o Poor projection properties will potentially produce quasi-replicates or sub-optimal coverage
X2
X1
X2
X1
12
Constrained Design Space
Examples o Geometric constraints on bottle and cap/closure o Constraint with respect to packaging component or equipment o Constraint with respect to different equipment parts
Example 1
Example 2
13
Constrained Design Space
Create space-filling designs in a constrained design space o Stinstra et al. (2003). Constrained Maximin Designs for Computer Experiments. Technometrics o Draguljic et al. (2012). Non-collapsing Designs for Bounded Non-rectangular Regions. Technometrics o Lekivetz et al. (2014). Fast Flexible Space-Filling Designs for Nonrectangular Regions. Quality & Reliability Engineering
International
Software is beginning to provide options
Still needs to be more evaluation of the different approaches
14
Projection Properties
MmLHDS have problems with projection properties in 2, 3, …, p-1 dimensions
Joseph, V.R., Gul, E. and Ba, S. Maximum Projection Designs for Computer Experiments (soon to appear in Biometrika)
MmLHDS MaxPro
15
Qualitative Variables in Computer Experiments
Typically when people think of computer experiments and space-filling
designs you think of continuous variables
Very common to have qualitative variables in computer experiments – At least at P&G!
o Examples o design/shape of equipment part o presence or absence of a piece of equipment o type or shape of packaging material o type of conveyor material
16
Qualitative Variables in Computer Experiments
Sliced Space-filling Designs (Qian et al. Biometrika 2009)
Sliced Latin-hypercube Designs (Qian – JASA 2012) o A sliced Latin hypercube design is a special Latin hypercube design that can be partitioned into slices of smaller
Latin hypercube designs based on the categorical variables. o R code available
Desirable properties
SLHD should have good space-filling properties for the whole design as well as for each slice Good projection properties
17
Qualitative Variables in Computer Experiments
Ba, S., Brenneman, W. A., and Myers, W. R. Optimal Sliced Latin Hypercube Designs (soon to appear in Technometrics) o First constructs the small LHD for each slice and then arrange their levels to form an overall SLHD o General algorithm which can search for the optimal SLHD efficiently o R code available
Overall
By Slice
18
Run Size for Computer Experiment “How large of a computer experiment do I need to run?” Loeppky, J. L., Sacks, J. and Welch, W. J. (2009). Choosing the Sample Size of a
Computer Experiment: A Practical Guide. Technometrics. o “10d runs will provide reasonable prediction accuracy for tractable functions and are sufficient to diagnose more
difficult problems”
In practice, there are situations where 10d is impractical Under effect sparsity
o only a subset, d0, of the input variables are important) then a lower bound on the number runs could be implemented
o (10 x d0) We often will implement this if there are budget constraints or if the computer simulations are very time consuming o Want to ensure good projection properties
19
Modeling Issues/Challenges
Stationary GP Model is inadequate
Qualitative Variables
20
When Stationary GP Model is Inadequate
Ba, S and Joseph, V. R. (2013). Composite Gaussian Process Models for Emulating Expensive Functions. Annuals of Applied Statistics o When design is sparse the CGP is more stable and more accurately approximates complex surface o A composite of two Gaussian processes 1) models the smooth global trend 2) models local details o JMP script
GP with nugget CGP
21
Industry Example
GP (with a nugget) CGP
LOO CV Error =7086 LOO CV Error =1363
Obs
erve
d
Obs
erve
d Jackknife Predicted Jackknife Predicted
22
Modeling Challenges with Qualitative Variables
As previously pointed out it is not uncommon to have qualitative variables in computer experiments
Simple approach – t different response curves for each qualitative level Can fit separate GP model to each response surface If the response surfaces are similar, then it would be “best” to borrow information from the other surfaces
References
o Qian et al. (2008) Gaussian Process Models for Computer Experiments with Qualitative and Quantitative Factors Technometrics
o Han et al. (2009) Prediction for Computer Experiments Having Quantitative and Qualitative Input Variables Technometrics
o Zhou et al. (2011) A Simple Approach to Emulation for Computer Models with Qualitative and Quantitative Factors Technometrics
o Zhang et al. (2015) Computer Experiments with Qualitative and Quantitative Variables: A Review and Reexamination Quality Engineering –puts it directly in the hands of the practitioner
23
Industry Example – Package Design
Quality of Model Fit
Sensitivity Analysis
24
Industry Example – Package Design
Understand Interactions to Gain Better Insight
25
Final Thought
There are still many interesting challenges in the area of computer experiments
from an industry perspective – both in the area of design and modeling
Also, many research opportunities!