building a dictionary of image fragments
Post on 03-Jan-2016
25 Views
Preview:
DESCRIPTION
TRANSCRIPT
11
Building a Dictionary of Image Fragments
Zicheng Liao Ali Farhadi Yang Wang Ian Endres David Forsyth
Department of Computer Science, University of Illinois at Urbana-Champaign
2
OutlineOutline
► IntroductionIntroduction►Related WorkRelated Work►Building a Dictionary of Fragments►Applications►Conclusion
3
IntroductionIntroduction
4
IntroductionIntroduction
► Image fragment – regions that could Image fragment – regions that could represent a single object, an object in represent a single object, an object in context or a piece of a scene – form a context or a piece of a scene – form a natural representation of objects.natural representation of objects.
►Key step - to build a large, rich dictionary of image fragments automatically.
5
IntroductionIntroduction
► Four steps:Four steps: 1.G1.Generate a set of fragment proposals from
image sets. 2.Verifythe qualities of the generated fragment
proposals with a discriminative method. The selected fragments are grouped and indexed by the labels of their source images.
3.Use a clean-up procedure to remove anomalies from the dictionary within each category.
4.Matte the resulting fragments out of the training images to get the best possible boundaries.
6
Related WorkRelated Work
►Automated object segmentation (a) that it is useful to work with more than
one segmentation of a particular image, then choose good fragments
(b) that these multiple segmentations can yield estimates of support
(c) that it is possible to identify segments that seem to form a single object, without knowing what the object is
7
Related WorkRelated Work
►Exemplar-based image classification There exist methods for image
classification using region-based exemplar matching and image-based exemplar matching
8
Building a Dictionary of Fragments
►1. 1. Generating fragment proposals►2. Fragment verification►3. Dictionary clean-up►4. Matting dictionary fragments
9
Building a Dictionary of Fragments
10
Building a Dictionary of Fragments-Generating fragment
proposals►Proposing regions: Proposing regions: This step aims to
generate a large and diverse set of proposals that are likely to be object regions.
11
Building a Dictionary of Fragments-Generating fragment
proposals►Ranking proposals:
The next step of is to rank all the proposals in an image so that object regions are ranked higher than non-object regions.
Use a rank-SVM formulation based on various features computed from proposal regions, such as color, texture, geometric surfaces, boundaries, etc.
12
Building a Dictionary of Fragments-Fragment
Verification
13
Building a Dictionary of Fragments-Fragment
Verification► Flickr: These images are downloaded from Flickr.
They are mainly about humans, objects, activities, pets, and familiar scenes (indoor and outdoor).
►Caltech256: This dataset is widely used for object recognition in the computer vision literature.
►PASCAL VOC2010: This is another widely used object recognition dataset. The images are more complex than those in Caltech256.
14
Building a Dictionary of Fragments-Fragment
Verification
15
Building a Dictionary of Fragments-Fragment
Verification►A good fragment can be a whole object, a
meaningful component of a larger object , or a scene that consists of part of an scene.
►A good fragment’s effective size should cover neither too little nor too much of the entire image domain. If a fragment covers too little of the image, it may contain little discriminative information and is unlikely to be useful for image compositing. On the other hand, if a fragment covers too much of the image, its distinction from the whole image is small.
16
Building a Dictionary of Fragments-Dictionary Cleanup
►Fragments associated with the same tag (e.g. “cat”) in the source images are grouped as “cat” fragments.
►In this step we want to remove such within-class fragment anomalies from the dictionary.
17
Building a Dictionary of Fragments-Dictionary Cleanup
►How to cleanup For each new incoming fragment, we use
an adapted asymmetric region-to-image matching algorithm to measure its distance to the fragment set and count the top k (5 in experiment) best matches.
18
Building a Dictionary of Fragments-Dictionary Cleanup
19
Building a Dictionary of Fragments-Matting Dictionary
Fragments►The closed-form matting algorithm
simplifies the matting equation with a local window color smoothness assumption, and transforms the problem into a quadratic optimization problem, which can be solved efficiently via a sparse linear solver.
20
ApplicationsApplications
►1. 1. Image Classification►2. Object Localization►3. Image Composition
21
Applications- Applications- Image Classification
►Fragments may have a small advantage because contextual information creates noise, or because our fragment selection procedure will prefer images with high contrast between fragment and background.
22
Applications- Applications- Image Classification
23
Applications- Applications- Object Localization
►The spatial support of fragments can be used to accurately localize objects in a query image.
24
Applications- Applications- Object Localization
25
Applications-Image Applications-Image CompositionComposition
26
Applications-Image Applications-Image CompositionComposition
27
ConclusionsConclusions
►Use the highly localized information of fragment-based matching to do object detection.
28
►Thanks for your listening.Thanks for your listening.
top related