qualifying exam: contour grouping vida movahedi supervisor: james elder supervisory committee: minas...
TRANSCRIPT
Qualifying Exam:Qualifying Exam:
Contour GroupingContour GroupingVida MovahediVida Movahedi
Supervisor: James ElderSupervisor: James Elder
Supervisory Committee:Supervisory Committee:
Minas Spetsakis, Jeff EdmondsMinas Spetsakis, Jeff Edmonds
York UniversityYork University
Summer 2009Summer 2009
ContentsContents• Introduction
• Preliminary Concepts– Pre-processing
– Gestalt cues
• Methods– Local & Heuristic
– Local & Probabilistic
– Global Saliency
• Evaluation
• Conclusion & open problems
ContentsContents• Introduction
• Preliminary Concepts– Pre-processing
– Gestalt cues
• Methods– Local & Heuristic
– Local & Probabilistic
– Global Saliency
• Evaluation
• Conclusion & open problems
IntroductionIntroduction
• SegmentationPartition an image into regions, each corresponding to
an object or entity
• Figure-Ground segmentation
Segmentation MethodsSegmentation Methods• Regional Segmentation
– Use regional info, optimize labelling of regional tokens, e.g. clustering
– Depending on uniformity in object region
• Active Contour Models– Use regional (external) & boundary (internal) info,
optimize deformation of model
– Sensitivity to initialization, too smooth
• Contour Grouping– Use boundary info (& regional info), optimize grouping of
contour fragments
Problem DefinitionProblem Definition• Input: Color image
• Goal: Figure-ground segmentation
• Method: Contour Grouping
• Other available info: None
- No motion, stereo or video information
- No user interactions
- No assumptions on object types, shapes, color, etc.
- No assumptions on background or lighting conditions
ChallengesChallenges• High-dimensional data space, lots of information,
many cues
• Unknown cue integration
• Global optimization in a non-convex multidimensional space
• Camera, imaging, quantization noise
• Clutter in natural scenes
• Occluded or overlapping objects
ContentsContents• Introduction
• Preliminary Concepts– Pre-processing
– Gestalt cues
• Methods– Local & Heuristic
– Local & Probabilistic
– Global Saliency
• Evaluation
• Conclusion & open problems
StepsStepsImage
Grouping Algorithm
Saliency
Computations
Optimization
Algorithm
Figure/Ground Segmentation
Pre-processing
Edge
Detection
Line /Curve
Approximation
Learned Parameters or Distributions
Pre-processingPre-processing
Image Edge Map Line Map Contour
Gestalt CuesGestalt Cues
How is grouping done in human vision?
• Proximity
• Similarity– Brightness– Contrast
• Good continuation – Parallelism– Co-circularity
ContentsContents• Introduction
• Preliminary Concepts– Pre-processing
– Gestalt cues
• Methods– Local & Heuristic
– Local & Probabilistic
– Global Saliency
• Evaluation
• Conclusion & open problems
Grouping MethodsGrouping Methods• Local Heuristic methods
– Defining a heuristic cost for contour hypotheses, find the optimal one
• Local Probabilistic methods– Find posterior probability of contour
hypotheses given cues, find the optimal one
• Global methods– An extra step of calculating global saliencies
based on local measures
ContentsContents• Introduction
• Preliminary Concepts– Pre-processing
– Gestalt cues
• Methods– Local & Heuristic
– Local & Probabilistic
– Global Saliency
• Evaluation
• Conclusion & open problems
Local & HeuristicLocal & HeuristicExample: Ratio Contour Method Example: Ratio Contour Method
(Wang et. al, PAMI’05)(Wang et. al, PAMI’05)
• Detected/ virtual fragments
• Contour cost= curvature & gap per unit length
• Graph model
• Alternate cycle
Local & HeuristicLocal & HeuristicExample: Ratio Contour Method Example: Ratio Contour Method
(Wang et. al, PAMI’05)(Wang et. al, PAMI’05)
• Edge/ Link costs
• Ratio Contour Algorithm
)(
2 )]()([)(eB
dtttew
)(
)(eB
dtel
Ce
Ce
el
ewC
)(
)()(
fragment real aon is )( if0
fragment virtualaon is )( f1)(
tv
tvit
Sample Results for RC methodSample Results for RC method
Image RC RRC
RC
Image
(from Stahl & Wang, TIP’07)
(from Wang et al., PAMI’05)
ContentsContents• Introduction
• Preliminary Concepts– Pre-processing
– Gestalt cues
• Methods– Local & Heuristic
– Local & Probabilistic
– Global Saliency
• Evaluation
• Conclusion & open problems
Local & ProbabilisticLocal & Probabilistic(Elder et al., PAMI’03)(Elder et al., PAMI’03)
• Bayesian Rule:
• Contour saliency= posterior probability of contour
• Assumptions:– Markov Chain Assumption– Independence of evidence from cues– Comparing contours of same length
11
1
)(
)().|()|(
LPDp
HpHDpDHp
)(
)(,
)|(
)|(
Hp
HpP
HDp
HDpL
)|(maxarg* DCcpcc
}...1{},...1{,),...,(11
niNCttc in
ctt
cij
ct
oi
jii
ppDCcp
),(
)|(
Local & Probabilistic Local & Probabilistic (Elder et al., PAMI’03)(Elder et al., PAMI’03)
• Graph Model
• Node weight
• Link weight
• Shortest path/cycle
• Approximate search
Nipvw oii
o ..1 ),log()( iv
ije jv
Njipew cijij
c ..1, ),log()(
cjicijii Pvvij
c
Pvi
o
ctt
cij
ct
oi ewvwppDCcp
),(),(
)()()log()log()|(log
Sample Results for Probabilistic MethodsSample Results for Probabilistic Methods
(from Estrada & Elder- CVPRW’06)
ContentsContents• Introduction
• Preliminary Concepts– Pre-processing
– Gestalt cues
• Methods– Local & Heuristic
– Local & Probabilistic
– Global Saliency
• Evaluation
• Conclusion & open problems
Global ModelGlobal Model
Local weights Global weights
Global SaliencyGlobal Saliency• Edge/Link Affinity
Based on collinearity, proximity, etc.
• Edge/ Link Saliency
Relative number of closed random walks which visit that edge/link (Mahamud et al., PAMI’03)
• Shown to be relevant to the eigenvalues and eigenvectors of the affinity matrix
• Grouping based on global saliency
Some Results of the Untangling methodSome Results of the Untangling method
(from Zhu; Song; Shi- ICCV’07)
ContentsContents• Introduction
• Preliminary Concepts– Pre-processing
– Gestalt cues
• Methods– Local & Heuristic
– Local & Probabilistic
– Global Saliency
• Evaluation
• Conclusion & open problems
EvaluationEvaluation
• Empirical discrepancy methods
The output of algorithms is compared with a reference segmentation or ground truth
• Requirements– A ground truth dataset– An error measure
SOD: Salient Object DatasetSOD: Salient Object Dataset
• Based on Berkeley Segmentation Dataset (BSD)
• 300 images, randomly showing 818 segmentations (half of BSD) to each of 7 subjects
• 12,110 object boundaries obtained
1
1
1
1
1
Region-based Error MeasuresRegion-based Error Measures• Example
• Not sensitive to some large shape features (e.g., spikes, wiggles)
BA
BAB
BA
BAA
BA
BA
RR
RRR
RR
RRRRR
RRBARIM
||||
1),(
Boundary-based Error Boundary-based Error MeasuresMeasures
• Not sensitive to object topology and some large shape features (e.g., loop-backs, wiggles)
),(minmax)),(max(),( badBASDBAhBbAa
B
}),({),( AaadBASD BB
),(),,(max),( ABhBAhBAH
AabadadBb
B
,),(min)(
Mixed Error MeasuresMixed Error Measures
fpfn N
kkB
fp
N
jjA
fn
qdN
pdN
BAMM11
)(1
)(1
2
1),(
• Example
• Not sensitive to some large shape features. Does not respect ordering along contours.
pj, j=1..Nfn are pixels in the false negative region (RB-RA)
qk, k=1..Nfp are pixels in the false positive region (RA-RB)
Contour Mapping MeasureContour Mapping Measure
• Based upon a matching between all points on the two boundaries
• Monotonically non-decreasing
• Allowing one-to-one, many-to-one, and one-to-many matching
• Error= average distance between matched pairs
• Dynamic Programming
Contour Mapping Distance=7.73
ContentsContents• Introduction
• Preliminary Concepts– Pre-processing
– Gestalt cues
• Methods– Local & Heuristic
– Local & Probabilistic
– Global Saliency
• Evaluation
• Conclusion & open problems
Conclusion & Open ProblemsConclusion & Open Problems• Cue selection and combination
• Grouping Model– Global saliency
– Probabilistic models
• Optimization Algorithms
• Hierarchical and multi-scale algorithms
• Quantitative evaluation