lecture27
TRANSCRIPT
![Page 1: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/1.jpg)
CSE486Robert Collins
Lecture 27:
Skin Color
![Page 2: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/2.jpg)
CSE486Robert Collins
Review: Light Transport
Source emits photons
Photons travel in astraight line
They hit an object. Some areabsorbed, some bounce offin a new direction.
And then some reachan eye/camera andare measured.
![Page 3: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/3.jpg)
CSE486Robert Collins
Color of Light Source
Spectral Power Distribution:
UV IRVisible
Wavelength λ
Amplitude
Relative amount of lightenergy at each wavelength
![Page 4: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/4.jpg)
CSE486Robert Collins
Spectral albedo for several different leaves
Spectral AlbedoRatio of incoming to outgoing radiation at different wavelengths.
![Page 5: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/5.jpg)
CSE486Robert Collins
Spectral Radiance
to a
SpectralIrradiance
SpectralRadiance
Spectral Albedo
![Page 6: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/6.jpg)
CSE486Robert Collins
Human Eye: Rods and Cones
S cones (blue)
M cones (green)
L cones (red)
rods (overall intensity)
![Page 7: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/7.jpg)
CSE486Robert Collins
Putting it all Together = Color
3 cones
COLOR!
![Page 8: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/8.jpg)
CSE486Robert Collins
Describing Color
Today we consider a sample material, human skin,and look at two approaches to describe the color of skin in order to find it in images.
1) physics-based approach2) learning-based approach
![Page 9: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/9.jpg)
CSE486Robert Collins
Goal: Label Skin Pixels in an Image
Applications: Person finding/tracking Gesture recognition
![Page 10: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/10.jpg)
CSE486Robert Collins
The Physics of Skin Color
Analytic derivation:Moritz Storring, Hans Andersen and Eric Granum, “Skin ColourDetection under Changing Lighting Conditions,” 7th Symposium onIntelligent Robotics Systems, Coimbra Portugal, July 1999.
Experimental measurement:Birgitta Martinkauppi, “Face Colour Under Varying Illumination: Analysis and Applications,” Ph.D. Thesis, Oulu University Press, Oulu Finland, 2002.
![Page 11: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/11.jpg)
CSE486Robert Collins
Problem: Color Variation
Sample from Oulu Physics-Based Face Database
Apparent color varies due to lighting color and and camera spectral response.
![Page 12: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/12.jpg)
CSE486Robert Collins
Skin Reflectance Model
Skin is well-modeled by a dichromatic reflectance model. transparent medium (dermis) pigmentations (hemaglobin, melanin) specular reflection (oil on skin)
Dichromatic reflectance model
![Page 13: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/13.jpg)
CSE486Robert Collins
Measuring Spectral Albedo of Skin
![Page 14: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/14.jpg)
CSE486Robert Collins
Understanding Skin Albedo
bluered
![Page 15: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/15.jpg)
CSE486Robert Collins
Understanding Skin AlbedoIncrease in melanin yields darker skin, masking the absorbtion band pattern of the hemaglobin.
![Page 16: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/16.jpg)
CSE486Robert Collins
Analytic ModelGenerate different skin albedos by using observed curve forcaucasian, and calculate the reduction in reflectance due to anincrease in melanin (a substance that has a known absorbtion)
I1(λ) ~ s I2(λ) ; λ = wavelengthSimpler approximation:s = scale factor
![Page 17: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/17.jpg)
CSE486Robert Collins
Illuminant SPD
Artificial light sourcesBlackbody sources(for theoretical calculations)
![Page 18: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/18.jpg)
CSE486Robert Collins
Camera Spectral Response
SONY DXC-755P 3CCD(manufacturer can supply this)
![Page 19: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/19.jpg)
CSE486Robert Collins
Skin Color Locus : Analytic Computation
“Normalized Color”recall simple scalingof SPD curves
![Page 20: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/20.jpg)
CSE486Robert Collins
Skin Color Locus : Experimental Measure
Fairly good agreement!
![Page 21: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/21.jpg)
CSE486Robert Collins
Skin Locus Examples
Histograms of skin color for different lightingconditions. Red: high values, blue: low values.
![Page 22: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/22.jpg)
CSE486Robert Collins
Tighter Bounds
If you know the camera and light source, you canderive much tighter analytic bounds on skin color.
![Page 23: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/23.jpg)
CSE486Robert Collins
Example
Same individual under different lighting conditions.
Subject 1:Caucasian
Subject 2:Asian Indian
![Page 24: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/24.jpg)
CSE486Robert Collins
Sample ApplicationFace tracking under varying illumination conditions
![Page 25: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/25.jpg)
CSE486Robert Collins
Jones and Rehg, 2002
“Statistical Color Models with Application to Skin Detection”, M. J. Jones and J. M. Rehg,Int. J. of Computer Vision, 46(1):81-96, Jan 2002
General Idea:• Drop the physics. Learn from examples instead.• Learn distributions of skin and nonskin color• Nonparametric distributions: color histograms• Bayesian classification of skin pixels
![Page 26: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/26.jpg)
CSE486Robert Collins
Learning from Examples
P(rgb | skin) = number of times rgb seen for a skin pixel total number of skin pixels seen
P(rgb | not skin) = number of times rgb seen for a non-skin pixel total number of non-skin pixels seen
These statistics stored in two 32x32x32 RGB histograms
R
GB
R
GB
Skin histogram Non-Skin histogram
First, have some poor grad student hand label thousands of images
![Page 27: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/27.jpg)
CSE486Robert Collins
Learned Distributions
Skin colorNon-Skin colorP(rgb | skin)
P(rgb | not skin)
![Page 28: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/28.jpg)
CSE486Robert Collins
Likelihood Ratio
Label a pixel skin if P(rgb | skin)
P(rgb | not skin)> Θ
Θ = (cost of false positive) P( seeing not skin)
(cost of false negative) P( seeing skin)
0 <= Θ <= 1
![Page 29: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/29.jpg)
CSE486Robert Collins
Sample Pixel Classifications
Θ = .4
![Page 30: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/30.jpg)
CSE486Robert Collins
Sample Application: HCI
Haiying Guan, Matthew Turk, UCSB
![Page 31: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/31.jpg)
CSE486Robert Collins
Sample Application: HCIHaiying Guan, Matthew Turk, UCSB
![Page 32: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/32.jpg)
CSE486Robert Collins
Sample Use: Adult Image Classification
• Percentage of pixels detected as skin.• Average probability of the skin pixels.• Size in pixels of the largest connected component of skin.• Number of connected components of skin.• Percentage of colors with no entries in the skin and non-skin histograms
Based on Five Features:
Jones and Rehg
![Page 33: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/33.jpg)
CSE486Robert Collins
Adult Image Classification
![Page 34: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/34.jpg)
CSE486Robert Collins
Combining Color and Text
![Page 35: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/35.jpg)
CSE486Robert Collins
Adult Image Classification
Other related work:
M.M. Fleck, D.A. Forsyth and C. Bregler, “FindingNaked People,” Proc. European Conf. on ComputerVision, Springer-Verlag, 1996. p. 593-602
James Wang, Jia Li, Gio Wiederhold and OscarFirschein, “System for Screen ObjectionableImages” Computer Communications, Vol 21(15),pp.1355-1369, Elsevier, 1998.
![Page 36: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/36.jpg)
CSE486Robert Collins
Back to Jones and Rehg ModelA compact description is provided by converting thehistogram-based model into a Gaussian Mixture model.
![Page 37: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/37.jpg)
CSE486Robert Collins
Jones and Rehg Mixture Model
![Page 38: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/38.jpg)
CSE486Robert Collins
Jones and Rehg Mixture Model
![Page 39: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/39.jpg)
CSE486Robert Collins
Homework: Due Friday Dec 7
•Download jrmogskin.m from the course web site•Try it on your own images!
![Page 40: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/40.jpg)
CSE486Robert Collins
What to Hand In
A short report, in Angel: 1) one example where it works wonderfully well 2) one example showing false positives (things that are not skin, but that are labeled as skin). 3) one example showing false negatives (a patch of skin that is not labeled), along with an educated guess about why it was missed.
![Page 41: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/41.jpg)
CSE486Robert Collins
Examples Working Well
![Page 42: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/42.jpg)
CSE486Robert Collins
Example of False Positives
![Page 43: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/43.jpg)
CSE486Robert Collins
Examples
Example ofFalse Negatives
Explanation:paint on skinchanges thespectral albedo
![Page 44: Lecture27](https://reader034.vdocument.in/reader034/viewer/2022042516/5584a87dd8b42afd5b8b4b8a/html5/thumbnails/44.jpg)
CSE486Robert Collins
Important Constraint
No X-rated images!!!! Keep it clean for the report.