feature-preserving artifact removal from dermoscopy images
DESCRIPTION
Feature-preserving Artifact Removal from Dermoscopy Images. Howard Zhou 1 , Mei Chen 2 , Richard Gass 2 , James M. Rehg 1 , Laura Ferris 3 , Jonhan Ho 3 , Laura Drogowski 3. 1 School of Interactive Computing, Georgia Tech 2 Intel Research Pittsburgh - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/1.jpg)
Feature-preserving Artifact Removal from Dermoscopy Images
Howard Zhou1, Mei Chen2,
Richard Gass2, James M. Rehg1,
Laura Ferris3, Jonhan Ho3, Laura Drogowski3
1School of Interactive Computing, Georgia Tech2Intel Research Pittsburgh
3Department of Dermatology, University of Pittsburgh
![Page 2: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/2.jpg)
Skin cancer and melanoma Skin cancer : most common of all cancers
![Page 3: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/3.jpg)
Skin cancer and melanoma Skin cancer : most common of all cancers
[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]
Basal Cell Carcinoma
Hemangioma
Compound nevus Seborrheic keratosis
![Page 4: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/4.jpg)
Skin cancer and melanoma Skin cancer : most common of all cancers Melanoma : leading cause of mortality
[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]
Basal Cell Carcinoma
Hemangioma
Compound nevus Seborrheic keratosis
Melanoma
Melanoma
![Page 5: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/5.jpg)
Skin cancer and melanoma Skin cancer : most common of all cancers Melanoma : leading cause of mortality Early detection significantly reduces mortality
[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]
Basal Cell Carcinoma
Hemangioma
Compound nevus Seborrheic keratosis
Melanoma
Melanoma
![Page 6: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/6.jpg)
[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]
Clinical ViewDermoscopy view
![Page 7: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/7.jpg)
Dermoscopy Skin surface microscopy Improve diagnostic accuracy by 30% for trained,
experienced physicians Requires 5 or more years of experience Computer-aided diagnosis (CAD) to assist less
experienced physicians
Clinical view Dermoscopy view
![Page 8: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/8.jpg)
Artifacts in dermoscopy images Hair, air-bubbles,… Interfering with computer-aided diagnosis
[ Image courtesy of Grana et al. 2006]
![Page 9: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/9.jpg)
Hair, air-bubbles,… Interfering with computer-aided diagnosis
[ Image courtesy of Grana et al. 2006]
Artifacts in dermoscopy images
![Page 10: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/10.jpg)
Artifacts in dermoscopy images Hair, air-bubbles,… Interfering with computer-aided diagnosis
[ Image courtesy of Grana et al. 2006]
Hair lesion boundary
![Page 11: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/11.jpg)
Artifacts in dermoscopy images Hair, air-bubbles,… Interfering with computer-aided diagnosis
[ Image courtesy of Grana et al. 2006]
Hair lesion boundary
![Page 12: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/12.jpg)
Artifacts in dermoscopy images Hair, air-bubbles,… Interfering with computer-aided diagnosis
[ Image courtesy of Grana et al. 2006]
Hair lesion boundary Hair pigmented network
![Page 13: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/13.jpg)
Previous work Hair detection and tracing
Fleming et al. 1998 Thresholding and averaging
“DullRazor”, Tim K. Lee et al. 1997 Schmid et al. 2003
Thresholding and inpainting Paul Wighton et al. 2008 (right here in the
conference)
![Page 14: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/14.jpg)
Detection: thresholding Removal: morphological
operations
Schmid et al.
![Page 15: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/15.jpg)
Schmid et al. Thresholding false
detection Accidental removal of
diagnostic features
Schmid et al. 2003
ThresholdingThresholding
![Page 16: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/16.jpg)
Schmid et al. Morphological operation
(neighbors’ average) blurring
Morphological operationMorphological operation
Schmid et al. 2003
![Page 17: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/17.jpg)
Feature-preserving Feature-preserving artifact removal artifact removal (FAR)(FAR)
Detection: Explicit curve modeling
Removal: Exemplar-based inpainting
Our method (FAR)Schmid et al. 2003
![Page 18: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/18.jpg)
Our method (FAR)
FAR Curve modeling
more accurate hair detection
ThresholdingThresholding Curve modelingCurve modeling
Schmid et al. 2003
![Page 19: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/19.jpg)
Our method (FAR)
FAR Exemplar-based
inpainting preserving features
ThresholdingThresholding Curve modelingCurve modelingMorphological operationMorphological operation Exemplar-based inpaintingExemplar-based inpainting
Schmid et al. 2003
![Page 20: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/20.jpg)
Our method (FAR)
FAR Exemplar-based
inpainting preserving features
ThresholdingThresholding Curve modelingCurve modelingMorphological operationMorphological operation Exemplar-based inpaintingExemplar-based inpainting
Schmid et al. 2003
![Page 21: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/21.jpg)
Our method (FAR)
FAR Exemplar-based
inpainting preserving features
Schmid et al. 2003
![Page 22: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/22.jpg)
Our method (FAR)
FAR Exemplar-based
inpainting preserving features
Schmid et al. 2003
![Page 23: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/23.jpg)
Our method (FAR)
FAR Exemplar-based
inpainting preserving features
Schmid et al. 2003
![Page 24: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/24.jpg)
System overview
Threholding
Curve fitting & intersection analysis
Exemplar patches
Exemplar-based inpainting
Dermoscopy image
Hair removed
Luminance difference dark thin structure
Line points
Line segments
Parameterized curves
Mask
Line points linking
Line points Detection
![Page 25: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/25.jpg)
Input dermoscopy image
![Page 26: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/26.jpg)
Enhancing dark-thin structure Luminosity channel in CIE L*u*v* Difference b/a morphological closing
[ Schmid-Saugeona et al. 2003, “Towards a computer-aided diagnosis system for pigmented skin lesions” ]
![Page 27: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/27.jpg)
Detecting line points
[ Steger 1998, ”An Unbiased Detector of Curvilinear Structures” ]
Curve B(t)
![Page 28: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/28.jpg)
Detecting line points
[ Steger 1998, ”An Unbiased Detector of Curvilinear Structures” ]
n(t)
Curve B(t)Cross section
n(t)
f(x)
![Page 29: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/29.jpg)
Detecting line points
[ Steger 1998, ”An Unbiased Detector of Curvilinear Structures” ]
n(t)
Cross section
n(t)
f(x)
Curve B(t)
![Page 30: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/30.jpg)
Detecting line points
[ Steger 1998, ”An Unbiased Detector of Curvilinear Structures” ]
n(t)
Cross section
n(t)
f(x)
f’ = 0|f’’| large
Curve B(t)
![Page 31: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/31.jpg)
Detecting line points
[ Steger 1998, ”An Unbiased Detector of Curvilinear Structures” ]
n(t)
Cross section
n(t)
f(x)
f’ = 0|f’’| large
Curve B(t)
n(t) : direction ┴ curve B(t)
eigenvector corresponding to the maximum absolute eigenvalue of the local Hessian
![Page 32: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/32.jpg)
Detecting line points
[ Steger 1998, ”An Unbiased Detector of Curvilinear Structures” ]
n(t)
![Page 33: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/33.jpg)
Detecting line points
[ Steger 1998, ”An Unbiased Detector of Curvilinear Structures” ]
![Page 34: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/34.jpg)
Linking line points
Link the neighboring points to get line segments (sets of ordered line points)
![Page 35: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/35.jpg)
Fitting polynomial curves
A set of ordered points Pi s
P
![Page 36: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/36.jpg)
Fitting polynomial curves
A set of ordered points Pi s Parametric curveP
![Page 37: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/37.jpg)
Fitting polynomial curves
A set of ordered points Pi s Parametric curve
B(t)
P
![Page 38: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/38.jpg)
Fitting polynomial curves
B(t)
P A set of ordered points Pi s Parametric curve
Minimize sum of squared distance
![Page 39: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/39.jpg)
Fitting polynomial curves
A set of ordered points Pi s Parametric curve
Minimize sum of squared distance
Linear system (can be solved by Gaussian elimination)
B(t)
P
![Page 40: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/40.jpg)
Handling hair intersection
Configurations:
Hair intersection Line segments Intersection analysisLink Line segment
……
![Page 41: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/41.jpg)
Before curve fitting and linking
Line segments
![Page 42: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/42.jpg)
After curve fitting and linking
Parameterized curves
![Page 43: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/43.jpg)
After curve fitting and linking
Parameterized curves
![Page 44: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/44.jpg)
After curve fitting and linking
Hair mask
![Page 45: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/45.jpg)
After curve fitting and linking
Hair mask
![Page 46: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/46.jpg)
Exemplar-based inpainting
[ Criminisi et al. 2003, “Object removal by exemplar-based inpainting” ]
[ Image courtesy of Criminisi et al. 2003 ]
Fill in with patches from the image itself Patch ordering structure propagation.
![Page 47: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/47.jpg)
Exemplar-based inpainting Fill in with patches from the image itself Patch ordering structure propagation.
[ Criminisi et al. 2003, “Object removal by exemplar-based inpainting” ]
![Page 48: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/48.jpg)
Exemplar-based inpainting Fill in with patches from the image itself Patch ordering structure propagation.
[ Criminisi et al. 2003, “Object removal by exemplar-based inpainting” ]
![Page 49: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/49.jpg)
Exemplar-based inpainting Fill in with patches from the image itself Patch ordering structure propagation.
[ Criminisi et al. 2003, “Object removal by exemplar-based inpainting” ]
![Page 50: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/50.jpg)
Exemplar-based inpainting Fill in with patches from the image itself Patch ordering structure propagation.
[ Criminisi et al. 2003, “Object removal by exemplar-based inpainting” ]
![Page 51: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/51.jpg)
Exemplar-based inpainting Fill in with patches from the image itself Patch ordering structure propagation.
[ Criminisi et al. 2003, “Object removal by exemplar-based inpainting” ]
![Page 52: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/52.jpg)
Exemplar-based inpainting Fill in with patches from the image itself Patch ordering structure propagation.
[ Criminisi et al. 2003, “Object removal by exemplar-based inpainting” ]
![Page 53: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/53.jpg)
Exemplar-based inpainting Fill in with patches from the image itself Patch ordering structure propagation.
[ Criminisi et al. 2003, “Object removal by exemplar-based inpainting” ]
![Page 54: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/54.jpg)
Before FAR
![Page 55: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/55.jpg)
After FAR
![Page 56: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/56.jpg)
More results Explicit curve
modeling Exemplar-based
inpainting
Our method (FAR)Schmid et al. 2003
![Page 57: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/57.jpg)
More results Explicit curve
modeling Exemplar-based
inpainting
Our method (FAR)Schmid et al. 2003
![Page 58: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/58.jpg)
Our method (FAR)
FAR Exemplar-based
inpainting preserving features
Schmid et al. 2003
![Page 59: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/59.jpg)
When is FAR not suitable ?
Oops, too much hair!
![Page 60: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/60.jpg)
When is FAR not suitable ?
Too much hair Makes explicit
modeling difficult
Schemid et al. 2003 (DullRazor) Our method (FAR)
![Page 61: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/61.jpg)
Conclusion
Automatic system that detects and removes curvilinear artifacts
Feature-preserving artifact removal: Explicit curve modeling Exemplar-based
inpainting
![Page 62: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/62.jpg)
Future work
Speed up exemplar-based inpainting
![Page 63: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/63.jpg)
Future work
Speed up exemplar-based inpainting Handle hair with arbitrary intensity
![Page 64: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/64.jpg)
Future work
Speed up exemplar-based inpainting Handle hair with arbitrary intensity Extend to removing air bubbles
![Page 65: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/65.jpg)
Questions ?
![Page 66: Feature-preserving Artifact Removal from Dermoscopy Images](https://reader036.vdocument.in/reader036/viewer/2022062801/568143d7550346895db06677/html5/thumbnails/66.jpg)
Additional results
Our method (FAR)Original Dermoscopy image