csp examples tamara berg cs 560 artificial intelligence many slides throughout the course adapted...
TRANSCRIPT
CSP Examples
Tamara BergCS 560 Artificial Intelligence
Many slides throughout the course adapted from Svetlana Lazebnik, Dan Klein, Stuart Russell, Andrew Moore, Percy Liang, Luke Zettlemoyer
Announcements/Reminders
• HW1 due date extended to Sunday, Sept 13, 11:59pm on classroom server – submit one solution per group and email Ric (
[email protected]) with the location of your directory and names of your group members
Backtracking search
• In CSP’s, variable assignments are commutative– For example, [WA = red then NT = green] is the same as
[NT = green then WA = red]• We only need to consider assignments to a single variable at
each level (i.e., we fix the order of assignments)• Depth-first search for CSPs with single-variable assignments is
called backtracking search
Improving Backtracking Efficiency
• Ordering– How should we order variables for assignment?– How should we order values from the domains?
Improving Backtracking Efficiency
• Ordering– How should we order variables for assignment?– How should we order values from the domains?
• Filtering– Can we detect inevitable failures early?
Early detection of failure
Apply inference to reduce the space of possible assignments and detect failure early
Early detection of failure:Forward checking
• Keep track of remaining legal values for unassigned variables• Terminate search when any variable has no legal values
• Simplest form of constraint propagation makes each pair of variables consistent:– X Y is consistent iff for every value of X there is some allowed value of Y
Arc consistency
Consistent!
• Simplest form of constraint propagation makes each pair of variables consistent:– X Y is consistent iff for every value of X there is some allowed value of Y
Arc consistency
• Simplest form of propagation makes each pair of variables consistent:– X Y is consistent iff for every value of X there is some allowed value of Y– When checking X Y, throw out any values of X for which there isn’t an
allowed value of Y
• If X loses a value, all pairs Z X need to be rechecked
Arc consistency
• Simplest form of propagation makes each pair of variables consistent:– X Y is consistent iff for every value of X there is some allowed value of Y– When checking X Y, throw out any values of X for which there isn’t an
allowed value of Y
• Arc consistency detects failure earlier than forward checking• Can be run before or after each assignment
Arc consistency
Improving Backtracking Efficiency
• Ordering– How should we order variables for assignment?– How should we order values from the domains?
• Filtering– Can we detect inevitable failures early?
• Structure– Can we exploit problem structure?
Tasmania and the mainland are separate structures.
We can solve them independently and then combine their solutions.
More generally, we can decompose the CSP into separate connected components (sub-problems of the CSP).
Then the solution is the union of the sub-problem solutions.
This can be much more efficient than solving the entire CSP as one problem.
Algorithm for tree-structured CSPs• Choose one variable as root, order variables from root to leaves
such that every node's parent precedes it in the ordering
http://cs188ai.wikia.com/wiki/Tree_Structure_CSPs
Algorithm for tree-structured CSPs• Choose one variable as root, order variables from root to leaves
such that every node's parent precedes it in the ordering• Backward removal phase: check arc consistency starting from the
rightmost node and going backwards
http://cs188ai.wikia.com/wiki/Tree_Structure_CSPs
Algorithm for tree-structured CSPs• Choose one variable as root, order variables from root to leaves
such that every node's parent precedes it in the ordering• Backward removal phase: check arc consistency starting from the
rightmost node and going backwards• Forward assignment phase: select an element from the domain of
each variable going left to right. We are guaranteed that there will be a valid assignment because each arc is arc consistent
http://cs188ai.wikia.com/wiki/Tree_Structure_CSPs
Local Search Methods
• Local Search: Improve what you have until you can’t make it better
• Generally much faster and more memory efficient than back-tracking search (but not necessarily complete)
Local search for CSPs• Start with an initial assignment of variables to values • Allow states with unsatisfied constraints• Attempt to improve states by reassigning variable values• Hill-climbing search:
– In each iteration, randomly select any conflicted variable and choose value that violates the fewest constraints
– I.e., attempt to greedily minimize total number of violated constraints
h = number of conflicts
Local search for CSPs• Start with “complete” states, i.e., all variables assigned • Allow states with unsatisfied constraints• Attempt to improve states by reassigning variable values• Hill-climbing search:
– In each iteration, randomly select any conflicted variable and choose value that violates the fewest constraints
– I.e., attempt to greedily minimize total number of violated constraints– Problem: local minima
h = 1
CSP in computer vision:Line drawing interpretation
An example polyhedron:
David Waltz, 1975
Desired output:
Goal: Label lines in drawing as convex (+), concave (-), or boundary (>).
How could we set this up as a CSP?
Variables?
Domains?
Constraints?
CSP in computer vision:Line drawing interpretation
Four vertex types:
Constraints imposed by each vertex type:
David Waltz, 1975
Inferring Temporal Order of Images from 3d Structure
Grant Schindler, Frank Dellaert, Sing Bing Kang
IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) , 2007.
http://www.cc.gatech.edu/~phlosoft/
Ultimate Goal
Produce time-varying 3d models of cities from historical photographs
Requires: Building 3d modelsReasoning about time
Temporal Inference Problem1. When was each photograph taken?2. When did each building first appear?3. When was each building removed?
Set of Photographs:Set of Objects:
Buildings
Temporal Inference Problem
(a, b) Time Interval
t Time
Temporal Inference ProblemGiven observations Z and scene geometry X, what are the temporal parameters T?
Images
Structure from Motion
(x,y,z) Position
(x,y,z) Translation(y,p,r) Rotation f Focal Length
Structure
Cameras
Correspondences
3D
Overview
3D Images + TimeStructure + Time Intervals
1891
1934
2007
Buildings
1885 - 1953
1906 - 2009
1968 - 2009
4DCorrespondences
Images
Overview
3D Images + TimeStructure + Time Intervals
1891
1934
2007
Buildings
1885 - 1953
1906 - 2009
1968 - 2009
4DCorrespondences
Images
Overview
3D Images + TimeStructure + Time Intervals
1891
1934
2007
Buildings
1885 - 1953
1906 - 2009
1968 - 2009
4DCorrespondences
Images
Overview
3D Images + TimeStructure + Time Intervals
1891
1934
2007
Buildings
1885 - 1953
1906 - 2009
1968 - 2009
4DCorrespondences
Images
1937 2001 2009
3D ReconstructionImages 3D Point Cloud
1891
1934
20071885 - 1953 1906 - 2009 1968 - 2009
Structure from MotionBundler Software by Noah Snavely(Snavely et al SIGGRAPH 2006)
Scene Reconstruction• Automatically estimate
– position, orientation, and focal length of cameras
– 3D positions of feature points
Feature detection
Pairwisefeature matching
Incremental structure
from motion
Correspondence estimation
Correspondence estimation
• Link up pairwise matches to form matches across several images
Image 1 Image 2 Image 3 Image 4
Structure from motion
Camera 1
Camera 2
Camera 3R1,t1
R2,t2
R3,t3
p1
p4
p3
p2
p5
p6
p7
minimize
f (R, T, P)
Incremental structure from motion
See: e.g. http://www.cs.cornell.edu/projects/bigsfm/ for more info
Grouping Points into Buildings3D Point Cloud 3D Buildings
Building Geometry1. Convex hull of each group2. Fit ground plane to camera centers3. Extend convex hulls to ground
1891
1934
20071885 - 1953 1906 - 2009 1968 - 2009
Point GroupingGroup points according to both: a. Distance < threshold in 3D b. Observed simultaneously in 1+ images
3D Reconstruction: Points vs. ObjectsLower Manhattan
454 images 83,860 points 960 Buildings1928-2010
Reasoning About Time:From Constraint Satisfaction…
• Key Idea: Visibility of 3D points constrains image ordering
• Variables? Domains? Constraints?
• Disadvantage: Only relative image ordering
Visibility Reasoning
• Goal: reorder images (columns) to have as few violations as possible
observed missing occluded
Columns: imagesRows: points
Violates constraints:
Satisfies constraints:
Temporal Ordering via CSP• Goal: reorder images (columns) to have as few violations as possible
• Backtracking search: intractable (n! solutions for n images)
• Local search: start with random ordering of columns, swap columns or groups of columns to reduce the number of conflicts
Temporal Ordering via CSP• Goal: reorder images (columns) to have as few violations as possible• Local search: start with random ordering of columns, swap columns
or groups of columns to reduce the number of conflicts
• Can also reorder the rows to group together points that appear and disappear at the same time – that gives you buildings
Conclusions
• Authors demonstrated that constraint satisfaction problems provide a powerful framework in which to solve temporal ordering problems
• Presented the first known method for solving this ordering problem.
• Remaining challenges? – Dealing with noise in classification of points as
observed/missing/occluded.