satzilla-07: the design and analysis of an algorithm portfolio for sat
DESCRIPTION
SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT. Lin Xu, Frank Hutter, Holger H. Hoos and Kevin Leyton-Brown University of British Columbia {xulin730, hutter, hoos, kevinlb}@cs.ubc.ca. Outline. Motivation History of SATzilla and related work SATzilla methodology - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/1.jpg)
SATzilla-07: The Design and SATzilla-07: The Design and Analysis of an Algorithm Analysis of an Algorithm
Portfolio for SATPortfolio for SAT
Lin Xu, Frank Hutter,Lin Xu, Frank Hutter,Holger H. Hoos and Kevin Leyton-Brown Holger H. Hoos and Kevin Leyton-Brown
University of British ColumbiaUniversity of British Columbia
{xulin730, hutter, hoos, kevinlb}@cs.ubc.ca{xulin730, hutter, hoos, kevinlb}@cs.ubc.ca
![Page 2: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/2.jpg)
2
Outline
Motivation
History of SATzilla and related work
SATzilla methodology
Example Problem
SATzilla for the SAT Competition
Conclusions and ongoing research
![Page 3: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/3.jpg)
Motivation
Lots of high performance solvers, but …. No single SAT solver dominates all others on
all types of instances
Question: How to select the best solver for a given SAT instance?
![Page 4: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/4.jpg)
4
Algorithm Selection Problem [Rice, 1976]
Reference: Select solvers based on previous experience or research papers
“Winner-Take-All”: Test solvers on some samples from target distribution;
select the solver with best performance.
SATzilla: Automatically based on instance characteristics’
![Page 5: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/5.jpg)
Related work: Portfolio of stochastic algorithm [Gomes & Selman,1997]
Running multiple algorithms at the same time
Reinforcement learning [Lagoudakis & Littman, 2001]
Select branching rule at each decision point
Branch & bound algorithm selection [Lobjois & LemaÎter, 1998]
Based on an estimation of search tree size
![Page 6: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/6.jpg)
6
History of SATzilla Old SATzilla [Nudelman, Devkar, et. al, 2003 ]
2nd Random 2nd Handmade (SAT) 3rd Handmade
SATzilla-07 1st Handmade 1st Handmade (UNSAT) 1st Random 2nd Handmade (SAT) 3rd Random (UNSAT)
![Page 7: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/7.jpg)
7
SATzilla-07 Methodology (offline)
Target
Distribution
Solvers
Features
Final solver
selection
Build Model
Winner-
Take-All
Collect Data
Pre-Solvers
![Page 8: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/8.jpg)
8
SATzilla-07 Methodology (online)
Run
Pre-solver
Compute
Features
Predict
Runtime
Run Winner-
Take-All
Run Best
If error
If error AND time left; run second best
![Page 9: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/9.jpg)
9
Solvers Used Eureka [Nadel, Gordon, Palti & Hanna,
2006]
Kcnfs2006 [Dubois & Dequen, 2006]
March_dl2004 [Heule & Maaren, 2006]
Minisat2.0 [Eén & Sörensson, 2006]
OKsolver [Kullmann, 2002]
Rsat [Pipatsrisawat & Darwiche, 2006]
Vallst [Vallstrom, 2005]
Zchaff_Rand [Mahajan, Fu & Malik, 2005]
![Page 10: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/10.jpg)
SATzilla-07 Example
Using quasi-group completion Problems (QCP) to validate our
general approach
![Page 11: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/11.jpg)
11
SATzilla-07 Example Problem Problem distribution
QCP problems generated near phase transition [Gomes & Selman,
1997]
Solvers Eureka, OKsolver, Zchaff_Rand
Features Same as in previous work [Nudelman, et al. 2004]
Collect Data Compute instances’ features and determine solvers’ runtime
Pre-Solver & “Winner take all” Build Models Final solver selection
![Page 12: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/12.jpg)
12
Empirical Hardness Model (EHM) The Core of SATzilla --- EHM
Accurately predict algorithm’s runtime based on cheaply computable features
Linear basis function regression
Features () Runtime (y)
fw() = wT • 23.34
• 7.21
• …
• …
![Page 13: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/13.jpg)
13
Improve EHM (deal with censoring) Heavy-tailed behavior and censoring
Three ways for censored data Drop them Keep them as if finished at cutoff Censored sampling
Schmee & Hahn ‘s approach [1979] REPEAT
1. Estimate runtime conditional on EHM and real runtime bigger than cutoff runtime
2. Build new EHM with estimated runtimeUNTIL no more changes in EHM
![Page 14: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/14.jpg)
14
How to deal with censored data
A: Drop themB: Finished at cutoff
C: Censor sampling Censor point
![Page 15: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/15.jpg)
15
How to deal with censored data
A: Drop them
B: Finished at cutoffC: Censor sampling
Censor pointCensor point
![Page 16: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/16.jpg)
16
How to deal with censored data
A: Drop them
B: Finished at cutoff
C: Censor sampling Censor point
Censor point
![Page 17: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/17.jpg)
17
Improve EHM (using Hierarchal Hardness Models )
EHM often more accurate, much simpler when trained with SAT/UNSAT samples only [Nudelman, et al. 2004]
Building hierarchal hardness models by
approximate model selection Oracle Mixture of experts problem with fixed experts
![Page 18: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/18.jpg)
SATzilla-07 for QCP
Average Runtime
![Page 19: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/19.jpg)
SATzilla-07 for QCP
Empirical CDF
![Page 20: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/20.jpg)
2007 SAT Competition
Three submissions for 2007 SAT Competition
BIG_MIX for all three category (demo)RANDOM
HANDMADE
![Page 21: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/21.jpg)
21
SATzilla-07 for SAT Competition Target Distribution
Previous SAT competition and SAT Race Solver (with/without preprocessing, Hyper)
Eureka, Kcnfs2006, March_dl2004, Minisat2.0 Vallsat, Rsat, Zchaff_Rand
Features Reduce probing time to 1 second Only cheap features, total about 3 seconds
Pre-Solvers March_dl 5 seconds, SAPS 2 seconds
![Page 22: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/22.jpg)
22
SATzilla-07 for SAT Competition “ Winner take all” solver
March_dl2004 Final candidates
BIG_MIX Eureka, Kcnfs2006, March_dl2004, Rsat RANDOM March_dl2004, Kcnfs2006, Minisat2.0+ HANDMADE March_dl2004, Vallst, March_dl2004+,
Minisat2.0+, Zchaff_Random+
![Page 23: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/23.jpg)
23
SATzilla-07 for BIG_MIX
![Page 24: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/24.jpg)
24
SATzilla-07 for BIG_MIX
Pre-solvers
Feature time
![Page 25: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/25.jpg)
25
SATzilla-07 for RANDOM
![Page 26: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/26.jpg)
26
SATzilla-07 for RANDOM
Pre-solvers
Feature time
![Page 27: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/27.jpg)
27
SATzilla-07 for HANDMADE
![Page 28: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/28.jpg)
28
SATzilla-07 for HANDMADE
Pre-solvers
Feature time
![Page 29: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/29.jpg)
Conclusions
![Page 30: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/30.jpg)
30
Conclusions Can combine algorithms into portfolios,
improving performance and robustness SATzilla approach has been proven to be
successful in real world competition
With more training data and more solvers, SATzilla can be even better
![Page 31: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/31.jpg)
Ongoing research SATzilla for industrial category
Use the same approach, SATzilla is 25% faster and solves 5% more instances
Score function Optimize objective function other than runtime
Local search Improve SATzilla performance by using local
search solvers as component
![Page 32: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/32.jpg)
32
Special Thanks Creators of solvers
Alexander Nadel, Moran Gordon, Amit Palti and Ziyad Hanna (Eureka)
Marijn Heule, Hans van Maaren (March_dl2004) Niklas Eén, Niklas Sörensson (Minisat2.0) Oliver Kullmann (OKsolver) Knot Pipatsrisawat and Adnan Darwiche (Rsat 1.04) Daniel Vallstrom (Vallst) Yogesh S. Mahajan, Zhaohui Fu and Sharad Malik
(Zchaff_Rand)
![Page 33: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/33.jpg)
SATzilla Pick for BIG_MIX
![Page 34: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/34.jpg)
SATzilla Pick for RANDOM
![Page 35: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT](https://reader034.vdocument.in/reader034/viewer/2022051821/5681573e550346895dc4e39a/html5/thumbnails/35.jpg)
SATzilla Pick for HANDMADE