illumination modeling and chromophore identification in ... · 8 estimates the maximum diffuse...
TRANSCRIPT
Zhao Liu, Josiane Zerubia
Ayin Research Team, INRIA Sophia Antipolis Méditerranée (France)
Publication available on https://team.inria.fr/ayin/publications-hal/
Illumination Modeling and Chromophore Identification in
Dermatological Photographs for Skin Disease Analysis
Ayin
2
INTRODUCTION
Objectives:
Imaging artifact removal from skin images
Chromophore concentration identification
Applications:
Computer-aided melanoma diagnosis
Automatic acne detection and differentiation
3
SKIN IMAGES
Fig. 1 Skin lesion images from a free public database [DermQuest, 2013] with illumination artifacts. (a) A superficial
spreading melanoma image with obvious horizontal shading effects. (b) Another superficial spreading melanoma
image with palpable highlight effects within the lesion areas.
Visual perception of skin color is not only credited to major
chromophores (melanin and hemoglobin) underneath the skin surface,
but is also affected by external illumination and spectral responses of
imaging detectors.
Pigmentation information may partially concealed by imaging artifacts.
4
OPTICS OF HUMAN SKIN
Fig.2 A three-layered skin reflectance model.
5
OPTICS OF HUMAN SKIN (Cont.)
Diffuse reflectance only:
where Qλ stands for sensor characteristics, kd is diffuse reflection constant, Ed,λ is intensity of diffuse components of light source. wd is wavelength-independent shading variation due to scene geometry. μm,λ and μh,λ are wavelength-dependent absorptive coefficients of melanin and hemoglobin, respectively.𝑙m,λ and 𝑙h,λ are the lengths of light penetration in epidermis and dermis
layers. cm and ch are densities of melanin and hemoglobin in a sampled volume of skin.
• Previous studies [Claridge, 2003; Yamamoto, 2008; Madooei, 2012] consider
human skin as a diffuse reflectance model only.
• No specular reflectance on the skin surface.
• Shading effects can be removed by dividing two spectral bands.
6
OPTICS OF HUMAN SKIN (Cont.)
Specular and diffuse reflectance:
where ks is specular reflection constant, Es,λ is intensity of specular components of light source.
ws is the specular factor giving rise of highlight, and α is a material relevant constant.
• Joint effects of specular and diffuse reflectance.
• Address artifacts (e.g. highlight, shading) normally existing in large-
dynamic-range intensity skin images captured under uncontrolled
environment.
7
METHOD
Specular Reflection and Diffuse Reflection Separation
Decomposition of Diffuse Reflection Image
Melanin Index and Hemoglobin Index Estimation
8
Estimates the maximum diffuse image [Yang, 2010] and apply it as intensity
guidance in a weighted average process to remove specular reflection.
Surface texture will be filtered out as specular component and resultant
diffuse skin image turns out blurred.
Localize the candidate specular reflection areas and then decide the highlight
regions by selecting pixel intensity larger than a threshold.
To avoid the problem of fake intensity (black spots) values, the diffuse
chromaticity of the detected specular pixels is computed by a B-spline image
interpolation [Pan, 2003].
Specular Reflection and Diffuse Reflection Separation
9
Specular Reflection and Diffuse Reflection Separation (Cont.)
Fig. 3 Specular reflection removal in skin color image. (a) The original melanoma images having highlight and
shading as artifacts in Fig.1. (b) Candidate specular reflectance. (c) Blurred diffuse reflectance image by
subtracting specular reflection (b) from (a), pointing out the areas having fake intensity pixels by ellipses. (d)
Specular reflectance component after thresholding. (e) Diffuse reflectance image using a B-spline pixel
interpolation calculated by our method.
(a)
(b) (c)
(d) (e)
10
Decomposition of Diffuse Reflection Image
Diffuse image intensity can be formulated by a linear combination of
chromophore coefficients, optical parameters of light source, and effects of
scene geometry in an inverse logarithmic form:
Applied an adaptive bilateral filter to the inverse logarithmic diffuse skin
image, where image intensity gradient is applied as reference to make the
spatial and range standard deviations in Gaussian kernel adaptive to each
pixel.
Considering that shading is normally a low frequency component, the high
frequency chromophore elements can be smooth out through embedding the
bilateral filtering into an iterative process.
11
Decomposition of Diffuse Reflection Image (Cont.)
Varied spatial and range standard deviations at different image location
make the illumination estimation unnatural and less homogeneous.
A weighted parametric polynomial curve fitting is subsequently introduced
to generate the final illumination image.
Subtracting the illumination map from the diffuse reflection component, a
skin image can be decomposed into a base layer and a detail layer.
12
Fig. 4 Base layers and detail layers of the diffuse melanoma image in Fig. 3(e). (a) Derived illumination component. (b) Corrected image after
specular reflection and shading effect removal by the proposed method. (c) Spectral bands of diffuse reflection image. (d) Intensity gradient
expressed by a monotonically increasing function. Blue color refers to small value, while red color stands for large one. (e) Initial illumination
approximation by the adaptive bilateral filtering. (f) Base layers (𝐼𝑏𝑎𝑠𝑒) by the polynomial curve fitting, taking (e) as a prior. (g) Detail layers
𝐼𝑑𝑒𝑡𝑎𝑖𝑙 after subtraction. For comparison with the original spectral band in (c), (e), (f) and (g) are shown in the exponential form. 1st – 3rd columns:
Red, Green and, Blue channels, respectively.
13
Melanin Index and Hemoglobin Index Estimation
Fig. 5 Spectral absorption of major
chromophores from 400nm to 1,000nm based
on published data [Jacques, 1998], and their
relation with spectral responses of conventional
RGB digital cameras.
From Fig. 6, the diffuse reflectance of
skin in a specific spectral band can
then be considered as an effect
attributed to particular chromophores.
Taking the absorptive coefficients and
light penetration lengths from the
previous publications [Keller, 2001,
Jacques, 1998], the melanin density
and hemoglobin density can be
estimated.
14
Melanin Index and Hemoglobin Index Estimation (Cont.)
Fig. 6 Melanin and hemoglobin concentrations calculated by different algorithms. (a) Melanin index and (b)
hemoglobin index derived from the proposed approach. (c) First and (d) second independent components by
the ICA method [Madooei, 2012].
15
EXPERIMENTS AND RESULTS
Melanoma Diagnosis
Lesion segmentation after illumination correction.
Melanoma classification using chromophore descriptors.
Acne Detection and Differentiation
Semi-automatic acne detection.
Inflammatory acne and hyperpigmentation discrimination.
16
Melanoma Diagnosis
Experimental dataset
A number of 201 conventional RGB skin lesion images (62MMs, 139 benign nevi (BN)) are collected from free public databases [DermQuest, 2013; DermIS, 2013] to form an experimental dataset.
Of these lesions, 135 were reported to be excised and examined by histopathology, giving 62 MMs and 73 BN diagnosed as 34 Dysplastic nevi, 25 Common Acquired nevi1, 8 Blue nevi, 4 Spitz nevi, and 2 Seborrheic Keratosis.
The remaining 66 lesions did not undertake excision biopsy due to no evidence of malignancy under clinical examinations.
17
Lesion Segmentation
The proposed illumination modeling method are compared with two other
illumination correction approaches [Cavalcanti, 2011, Glaister, 2013], both
of which were developed specifically for skin lesion images.
The manual segmentation outlined by an experienced dermatologist is
applied as the reference truth.
Skin images after illumination modeling are segmented into lesion and
non-lesion areas by the Otsu’s method [Otsu, 1979].
18
Fig. 7 Comparison of different illumination modeling methods for skin lesion images. (1st row) Six example lesion images from free public
databases [DermQuest, 2013, DermIS, 2013] with highlight and shading as artifacts. (2nd row) Results from Cavalcanti’s method [Cavalcanti,
2011]. (3rd row) Results from MSIM method [Glaister, 2013]. (4th row) Results by the proposed method.
Illumination Modeling Comparison
19
Segmentation Comparison
Fig. 8 Skin lesion segmentation on images after illumination correction by different methods in Fig. 7.
Black line: manual segmentation by an experienced dermatologist. Blue line: segmentation on the
original image. Green line: segmentation on the corrected image by the Cavalcanti’s method. Yellow
line: segmentation on the corrected image by the MSIM method. Red line: segmentation on the
corrected image by the proposed method.
20
Lesion Segmentation (Cont.)
Average TC ± Standard Deviation (%)
Original 65.74 ± 21.87
Cavalcanti [Cavalcanti, 2011] 79.40 ± 11.73
MSIM [Glaister, 2013] 81.06 ± 13.92
Our method 88.47 ± 8.16
Table 1. Comparison of segmentation accuracy on skin lesion images after illumination
correction using different algorithms.
The Tanimoto coefficient (TC) [Tan, 2005] defined below is used to evaluate
the automatic segmentation accuracy.
21
Melanoma Classification
SE (%) SP (%) Acc. (%) AUC
RGB Original 66.13 69.78 68.65 0.653
RGB Cavalcanti [Cavalcanti, 2011] 70.97 77.70 75.62 0.713
RGB MSIM [Glaister, 2013] 70.97 81.30 78.11 0.752
RGB, Our method 72.58 88.50 83.58 0.852
MI+HI, ICA [Madooei, 2012] 77.42 86.33 83.58 0.866
MI+HI, Our method 83.87 84.89 84. 57 0.878
Table 2. Melanoma diagnostic results applying diagnostic features derived from RGB colorimetry,
MI+HI from the ICA method [Madooei, 2012], and MI+HI from the proposed method.
A 12D diagnostic feature vector, including 3 absolute color variation measures, 3
relative color variation measures, and 6 pigmentation asymmetry measures, is applied
as the diagnostic features for melanoma classification.
A linear support vector machine is used as the classifier, and a 10-fold cross validation
is the training-testing strategy.
Sensitivity (SE), specificity (SP) and overall diagnostic accuracy (Acc.) are recorded to
evaluate classification performance.
22
Acne Detection and Differentiation
Acne vulgaris is believed resulting from proliferation of propionibacterium
acnes in pilosebaceous unit [Webster, 1995].
Inflammatory acne, visible as red papules, is due to increase of
oxyhemoglobin level.
Dark scar, known as secondary acne hyperpigmentation, will appear
thereafter thanks to excess melanin production during the healing process
[Webster, 1995].
23
Acne Detection and Differentiation (Cont.)
The combination of melanin and hemoglobin indices CI is used as the only
pigmentation descriptor and forwarded to a MRF model [Liu, 2013] to
isolate acne from surrounding normal skin.
A number of 50 challenging RGB acne images, captured under uncontrolled
environment from a free public database [Dermnet NZ, 2013], have been
tested.
Professional assessment of acne images is under process by an experienced
dermatologist in the Hôpital L’Archet 2, CHU de Nice.
24
Fig. 9 Acne segmentation of two conventional RGB images using chromophore descriptors derived from different methods. (a)(b) Original
images with highlight/shading. (c)(e) Results by the bilateral decomposition. (d)(f) Results by the ICA method [Madooei, 2012].
25
Conclusions
Proposes a novel illumination modeling and chromophore identification
method in dermatological photographs for skin disease analysis.
The derived melanin and hemoglobin indexes are well identified, and prove
robust to highlight and strong shading in skin color images captured under
uncontrolled imaging setting.
Experiments show that the proposed method increases automatic melanoma
diagnostis, and exhibits a high potential for other skin disease analysis.
For the next step study, the accuracy of automatic acne segmentation will be
compared to the manual segmentation outlined by the experienced
dermatologist when the assessment is available.
26
References
[DermQuest, 2013] DermQuest. Sept. 23th 2013. [Online] Available: http://www.dermquest.com.
[Claridge, 2003] E. Claridge, S. Cotton, P. Hall, M. Moncrieff. “From color to tissue histology: Physics-based
interpretation of images of pigmented skin lesions,” Med. Image Anal., vol. 7, pp. 489–502, 2003.
[Yamamoto, 2008] T. Yamamoto, H. Takiwaki, H. Ohshima. “Derivation and clinical application of special
imaging by means of digital cameras and image freeware for quantification of erythema and pigmentation,” Skin
Res. Technol., vol. 14, pp. 26–34, 2008.
[Madooei, 2012] A. Madooei, M. Drew, M. Sadeghi. M. Atkins. “Intrinsic Melanin and Hemoglobin Color
Components for Skin Lesion Malignancy Detection,” In proc. of 15th Medical Image Computing and Computer
Intervention (MICCAI), vol. 7510, pp. 315-322, Nice, 2012.
[Yang, 2010] Q. Yang, S. Wang, N. Ahuja. “Real-time Specular Highlight Removal Using Bilateral Filtering,” In
proc. of 11th European Conference on Computer Vision (ECCV), vol. IV, pp. 87-100, Greece, 2010.
[Pan, 2003] J. Pan. “Image Interpolation using Spline Curves,” Dept. of Mechanical Engineering MEC572 term
paper, 2003.
[Jacques, 1998] S. Jacques. “Skin optics,” Oregon Medical Laser Center News, January, 1998.
[Keller, 2001] G. Keller, V. Lacombe, P. Lee, J. Watson. Lasers in Aesthetic Surgery, Thieme, 2001.
27
References
[DermIS, 2013] DermIS. Sept. 23th 2013. [Online] Available: www.dermis.net.
[Cavalcanti, 2011] P. G. Cavalcanti and J. Scharcanski, “Automated prescreening of pigmented skin lesions using
standard cameras,” Comput.Med. Imag. Graph., vol. 35, pp. 481–491, 2011.
[Glaister, 2013] J. Glaister, R. Amelard, A. Wong, and D. A. Clausi. "MSIM: Multistage Illumination Modeling of
Dermatological Photographs for Illumination-Corrected Skin Lesion Analysis," IEEE Trans. Biomed. Eng., vol.60,
no.7, pp.1873-1883, 2013.
[Otsu, 1979] N. Otsu. "A threshold selection method from gray-level histograms". IEEE Trans. Sys., Man., Cyber.
Vol.9, no.1, pp.62–66, 1979.
[Tan, 2005] P. Tan, M. Steinbach, V. Kumar. Introduction to data mining, 1st edition, Boston, Pearson Addison
Wesley, 2005.
[Webster, 1995] G. Webster. “Inflammation of acne vulgaris,” J. Am. Acad. Dermatol., vol. 33, pp. 247–253,
1995.
[Liu, 2013] Z. Liu, J. Zerubia. “Towards automatic acne detection using a MRF model with chromophore
descriptors.” The European Signal Processing Conference (EUSIPCO), in press, Marrakech, 2013.
[Dermnet NZ, 2013] Dermnet NZ. Oct. 22nd 2013. [Online] Available: http://www.dermnetnz.org/.