myopic policies for budgeted optimization with constrained experiments
DESCRIPTION
Myopic Policies for Budgeted Optimization with Constrained Experiments. Javad Azimi , Xiaoli Fern, Alan Fern Oregon State University AAAI, July 2010. Motivation: Electricity Production in a Microbial Fuel Cell (MFC). This is how an MFC works. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/1.jpg)
1
Myopic Policies for Budgeted Optimization
with Constrained Experiments
Javad Azimi, Xiaoli Fern, Alan Fern
Oregon State University AAAI, July 2010
![Page 2: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/2.jpg)
2
Motivation: Electricity Production in a Microbial Fuel Cell (MFC)
AnodeCathode
bact
eria
Oxidation products
(CO2)
Fuel (organic matter)
e-
e-
O2
H2OH+
This is how an MFC works
SEM image of bacteria sp. on Ni nanoparticle enhanced carbon fibers.
Nano-structure of anode significantly impact the electricity production.
We should optimize anode nano-structure to maximize power.
![Page 3: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/3.jpg)
3
Experiment Selection• Experiments are costly and there is a fixed budget.• How to select the best sequence of experiments.
Current Experiments Scientist selects Experiment
Run Experiment
![Page 4: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/4.jpg)
Bayesian Optimization (BO)• Since Running experiment is very expensive we use BO.
• Select one experiment to run at a time based on results of previous experiments.Current Experiments Gaussian Process Surface Select Single Experiment
Run Experiment 4
![Page 5: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/5.jpg)
5
Bayesian Optimization (BO)• BO assumes that we can ask for specific experiment.• This is unreasonable assumption in many applications.
– In Fuel Cell it takes many trials to create a nano-structure with specific requested properties.
– Costly to fulfill.Space of Experiments
Average Circularity
Ave
rage
d A
rea
![Page 6: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/6.jpg)
6
Constrained Experiments• It is less costly to fulfill a request that specifies ranges for the nanostructure
properties
• E.g. run an experiment with Averaged Area in range r1 and Average Circularity in range r2
• We will call such requests “constrained experiments”Space of Experiments
Average Circularity
Ave
rage
d A
rea
Constrained Experiment 1• large ranges • low cost• high uncertainty about which experiment will be run
Constrained Experiment 2• small ranges• high cost• low uncertainty about which experiment will be run
![Page 7: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/7.jpg)
7
BO for Constrained Experiment
• Given a fixed budget, select the best constrained experiments.
Run Experiment
Current Experiments Gaussian Process Surface Select Single ExperimentSelect Constrained Experiment
![Page 8: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/8.jpg)
8
Constrained Experiment• We generalized BO heuristics to constrained
experiments.
• Two challenges:– How to compute heuristics for constrained experiment?– How to take experimental cost into account?(which has
been ignored by most of the approaches in BO).
![Page 9: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/9.jpg)
9
Standard BO Heuristics• Standard heuristics are statistics of the posterior p(y|x,D) where D is our
current observation.
• Maximum Upper bound Interval (MUI)– Select point with highest 95% upper confidence bound– Purely explorative approach.
• Maximum Probability of Improvement (MPI)– It computes the probability that the output is more than (1+m) times of the best current
observation , m>0. – Explorative and Exploitative.
• Maximum Expected of Improvement (MEI)– Similar to MPI but parameter free– It simply computes the expected amount of improvement after sampling at any point.
![Page 10: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/10.jpg)
10
Generalizing BO for Constrained Experiment
• Having the posterior distribution of p(y|x,D) and px(.|D) we can calculate the posterior of the output of each constrained experiment which has a closed form solution.
• Therefore we can compute standard BO heuristics for constrained experiments.– There are closed form solution for these heuristics.
Input spaceDiscretization
Level
![Page 11: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/11.jpg)
11
Budgeted Constrained Experiments• We are limited with Budget B.• Unfortunately heuristics will typically select the smallest and most
costly constrained experiments which is not a good use of budget.
• How can we consider the cost of each constrained experiment in making the decision?– Cost Normalized Policy (CN)– Constraint Minimum Cost Policy(CMC)
-Low uncertainty
-High uncertainty
-Better heuristic value
-Lower heuristic value
-Expensive -Cheap
![Page 12: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/12.jpg)
12
Cost Normalized Policy• It selects the constrained experiment achieving
the highest expected improvement per unit cost.
• We report this approach for MEI policy only.
![Page 13: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/13.jpg)
13
Constraint Minimum Cost Policy (CMC)• Motivation:
1. Approximately maximizes the heuristic value.2. Has expected improvement at least as great as spending
the same amount of budget on random experiments.• Example:
Very expensive: 10 random experiments likely to be better
Selected Constrained experiment
Poor heuristic value: not select due to 1st
condition
Cost=4 random Cost=10 random Cost=5 random
![Page 14: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/14.jpg)
14
Experimental Results(Setup)
• Gaussian process is used as our model with squared exponential kernel.
• Cost function is defined as:
– There is a constant cost for running any constrained experiment plus an additional cost depending on the size of the experiment.
– The value of slope dictates how fast the cost increases as the size of a constrained experiment decreases.
Space of Experiments
r1
r2
![Page 15: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/15.jpg)
15
Experimental Results(Real Applications)
• Two Real data sets:– Fuel Cell:• Fuel Cell electricity generation
– Hydrogene:• Biosolar hydrogen production
![Page 16: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/16.jpg)
16
Experimental Results(Benchmarks Functions)
• 3 popular benchmarks used in BO literature.
Rosenbrock Discontinuous Cosines
![Page 17: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/17.jpg)
17
Overall Performance• The normalized regret of each framework is shown which is calculated
as y*- ymax over Random performance for budget 15 where y* is the highest possible output.
Average regret of each approach over all frameworks.
![Page 18: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/18.jpg)
Different Budget(1)
18
Random
Cosines
Fuel CellReal
Rosenbrock
CMC-MUI
![Page 19: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/19.jpg)
Different Budget(2)
19
CN-MEI
Cosines
Fuel CellReal
Rosenbrock
![Page 20: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/20.jpg)
Different Budget(3)
20
CMC-MPI(0.2)
Cosines
Fuel CellReal
Rosenbrock
![Page 21: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/21.jpg)
21
Different Budget(4)
CMC-MEICosines
Fuel CellReal
Rosenbrock
![Page 22: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/22.jpg)
22
Conclusion• We introduced a new constrained experiment
framework which asks for hyper-rectangle rather than exact point.
• We extended model-free BO heuristics to our frame work.
• We introduced two approaches to optimize our budgeted framework.
• CMC-MEI is working better than other approaches.
![Page 23: Myopic Policies for Budgeted Optimization with Constrained Experiments](https://reader035.vdocument.in/reader035/viewer/2022062520/56815f0e550346895dcdcdab/html5/thumbnails/23.jpg)
23
Thanks for attendance
Question?