MACHINE LEARNING: FOUNDATIONS COURSETAU – 2012APROF. YISHAY MANSOUR
o TextonBoost :Joint Appearance, Shape and Context Modeling for Multi-Class Object Recognition and Segmentation
J. Shotton*, J. Winn†, C. Rother†, and A. Criminisi†
o * University of Cambridgeo † Microsoft Research Ltd, Cambridge, UK
Yaniv BarMarch 2013
GOAL
o Simultaneous recognition and segmentation:Efficiently detect a large number of object classes and give a pixel-perfect segmentation of an image into these classes.
DATA AND CLASSES Original Paper: 3 DBs.
Main DB: MSRC 21. MSRC 21-Class Object
Recognition Database 591 hand-labelled images
Original main DB was updated to MSRC 23.
MSRC 23-Class Object Recognition Database 592 hand-labelled images
HIGH LEVEL APPROACH
High-level description of approach:Learn classifier based on relative texture locations for each class. Classification is then refined.
Given an image, for each pixel:
- Texture-Layout features are calculated
- A boosting classifier gives the probability of the pixel belonging to each class
- The discriminative model combines the boosting output with low-level color, location, and edge information; image receives final label.
Most important part of the model is the Shape/Context Potential – it is significant for object recognition and very rough segmentation results.
Other potential such as Edge and Color refine the segmentation results.
(a) Original image, (b) Shape, (c) (b)+edge, (d) (c)+color
TEXTURE LAYOUT FEATURES
For modeling object shape, appearance and context we use a New texton-based features.This feature (texton) compact and efficient characterisation of local texture.
o The task is to recognize surfaces made from different materials on the basis of their texture appearance.o Different materials show different texture appearance.Moreover, texture appearance of the same material changes dramatically due to different viewpoint/lighting settings (specularities, shadows, and occlusions).
WHAT ARE TEXTONS
Computing texton maps:
Texton mapColours Texton Indices
Input image
Clustering
Filter BankConvolve 17-D filter bank (composed of gaussians, dogs, logs) with all training images
Responses are clustered with K-means
Each pixel is assigned a texton number
CALCULATING TEXTURE-LAYOUT FEATURES
Capturing appearance:
How Texture-Layout features jointly model texture and layout:
LEARNING
Learning is done with Joint Boost algorithm – A version of Multi class gentle boost algorithm.
I’ve used both AdaBoost.M1 and AdaBoost.Mh (multiclass reduction to binary which is due to the fact that AdaBoosting is only for binary classification).
THE GOOD AND BAD
The Good: Provides reasonable recognition + segmentation for many classes. Also, combines several good ideas. Most of previous works didn’t tackle the problem as a whole – rather, problems were treated separately.
The Bad: Does not beat past work (in terms of quantitative recognition results) and a bit hacky.
CODE-SEQUENCE OF EXECUTION
1. imagesTextonization.m(extract efficient images characterization)
2. calcModelFeatures.m(calculate the appearance (shape) potential
context)
3. trainModel.m(build a classification model)
4. testModel.m(test the classification model with test data)
RESULTS