learning specific-class segmentation from diverse data m. pawan kumar, haitherm turki, dan preston...
TRANSCRIPT
![Page 1: Learning Specific-Class Segmentation from Diverse Data M. Pawan Kumar, Haitherm Turki, Dan Preston and Daphne Koller at ICCV 2011 VGG reading group, 29](https://reader034.vdocument.in/reader034/viewer/2022051110/551ac01f5503466b6a8b4c02/html5/thumbnails/1.jpg)
Learning Specific-Class Segmentation from Diverse Data
M. Pawan Kumar, Haitherm Turki, Dan Preston and Daphne Koller at ICCV 2011
VGG reading group, 29 Nov 2011, presented by Varun Gulshan
![Page 2: Learning Specific-Class Segmentation from Diverse Data M. Pawan Kumar, Haitherm Turki, Dan Preston and Daphne Koller at ICCV 2011 VGG reading group, 29](https://reader034.vdocument.in/reader034/viewer/2022051110/551ac01f5503466b6a8b4c02/html5/thumbnails/2.jpg)
Semantic image segmentation
![Page 3: Learning Specific-Class Segmentation from Diverse Data M. Pawan Kumar, Haitherm Turki, Dan Preston and Daphne Koller at ICCV 2011 VGG reading group, 29](https://reader034.vdocument.in/reader034/viewer/2022051110/551ac01f5503466b6a8b4c02/html5/thumbnails/3.jpg)
Main idea
• High level: Getting fully labelled data for training is expensive, use other easily available ‘diverse’ data for learning (bounding boxes, classification labels for image).
Tags: Car, peoplePerson bounding box
![Page 4: Learning Specific-Class Segmentation from Diverse Data M. Pawan Kumar, Haitherm Turki, Dan Preston and Daphne Koller at ICCV 2011 VGG reading group, 29](https://reader034.vdocument.in/reader034/viewer/2022051110/551ac01f5503466b6a8b4c02/html5/thumbnails/4.jpg)
Implementing the idea
• The bounding box/image classification data is incomplete for segmentation, fill in the missing information using latent variables.
• Setup the training cost function using latent variables. Use their self-paced learning algorithm for Latent-SVM’s [NIPS2010] to optimise the training cost function.
• While inferring latent variables, make sure latent variable estimation is consistent with the weak annotation. Setting up the inference problems to ensure this condition.
![Page 5: Learning Specific-Class Segmentation from Diverse Data M. Pawan Kumar, Haitherm Turki, Dan Preston and Daphne Koller at ICCV 2011 VGG reading group, 29](https://reader034.vdocument.in/reader034/viewer/2022051110/551ac01f5503466b6a8b4c02/html5/thumbnails/5.jpg)
Energy function without latent variables
Notation:
Image
Parameters to be trained
Joint feature vector (essentially the terms of a CRF)
![Page 6: Learning Specific-Class Segmentation from Diverse Data M. Pawan Kumar, Haitherm Turki, Dan Preston and Daphne Koller at ICCV 2011 VGG reading group, 29](https://reader034.vdocument.in/reader034/viewer/2022051110/551ac01f5503466b6a8b4c02/html5/thumbnails/6.jpg)
Structured output training
Ground truth labels
Loss function
![Page 7: Learning Specific-Class Segmentation from Diverse Data M. Pawan Kumar, Haitherm Turki, Dan Preston and Daphne Koller at ICCV 2011 VGG reading group, 29](https://reader034.vdocument.in/reader034/viewer/2022051110/551ac01f5503466b6a8b4c02/html5/thumbnails/7.jpg)
Introducing latent variables
![Page 8: Learning Specific-Class Segmentation from Diverse Data M. Pawan Kumar, Haitherm Turki, Dan Preston and Daphne Koller at ICCV 2011 VGG reading group, 29](https://reader034.vdocument.in/reader034/viewer/2022051110/551ac01f5503466b6a8b4c02/html5/thumbnails/8.jpg)
Introducing latent variables
But we don’t know what hk is (its latent), so maximise it out.
![Page 9: Learning Specific-Class Segmentation from Diverse Data M. Pawan Kumar, Haitherm Turki, Dan Preston and Daphne Koller at ICCV 2011 VGG reading group, 29](https://reader034.vdocument.in/reader034/viewer/2022051110/551ac01f5503466b6a8b4c02/html5/thumbnails/9.jpg)
Introducing latent variables
![Page 10: Learning Specific-Class Segmentation from Diverse Data M. Pawan Kumar, Haitherm Turki, Dan Preston and Daphne Koller at ICCV 2011 VGG reading group, 29](https://reader034.vdocument.in/reader034/viewer/2022051110/551ac01f5503466b6a8b4c02/html5/thumbnails/10.jpg)
Self-paced optimisation
![Page 11: Learning Specific-Class Segmentation from Diverse Data M. Pawan Kumar, Haitherm Turki, Dan Preston and Daphne Koller at ICCV 2011 VGG reading group, 29](https://reader034.vdocument.in/reader034/viewer/2022051110/551ac01f5503466b6a8b4c02/html5/thumbnails/11.jpg)
Self-paced optimisation
Indicator variable to switch off the harder cases.
![Page 12: Learning Specific-Class Segmentation from Diverse Data M. Pawan Kumar, Haitherm Turki, Dan Preston and Daphne Koller at ICCV 2011 VGG reading group, 29](https://reader034.vdocument.in/reader034/viewer/2022051110/551ac01f5503466b6a8b4c02/html5/thumbnails/12.jpg)
Second idea: Latent variable estimation
The algorithm involves estimating annotation consistent latent variables in the following equation:
More precisely
![Page 13: Learning Specific-Class Segmentation from Diverse Data M. Pawan Kumar, Haitherm Turki, Dan Preston and Daphne Koller at ICCV 2011 VGG reading group, 29](https://reader034.vdocument.in/reader034/viewer/2022051110/551ac01f5503466b6a8b4c02/html5/thumbnails/13.jpg)
Move to white-board
Me
You
Beware of Equations