camera-based signage detection and recognition for blind persons
DESCRIPTION
Portable and Mobile Systems in Assistive Technology - Camera-based Signage Detection and Recognition for Blind Persons - Tian, Yingli (f)TRANSCRIPT
![Page 1: Camera-based Signage Detection and Recognition for Blind Persons](https://reader034.vdocument.in/reader034/viewer/2022051314/55961ee41a28ab7b0e8b47b1/html5/thumbnails/1.jpg)
Shuihua Wang and Yingli Tian
{swang15, ytian}@ccny.cuny.edu
Presented by: Shizhi Chen
Department of Electrical Engineering
The City College of New York
![Page 2: Camera-based Signage Detection and Recognition for Blind Persons](https://reader034.vdocument.in/reader034/viewer/2022051314/55961ee41a28ab7b0e8b47b1/html5/thumbnails/2.jpg)
OutlineMotivation
Proposed algorithm
Experimental results
![Page 3: Camera-based Signage Detection and Recognition for Blind Persons](https://reader034.vdocument.in/reader034/viewer/2022051314/55961ee41a28ab7b0e8b47b1/html5/thumbnails/3.jpg)
Motivation Access unfamiliar environment
Recognize restroom signage
Available technology
(a) (b) (c)
![Page 4: Camera-based Signage Detection and Recognition for Blind Persons](https://reader034.vdocument.in/reader034/viewer/2022051314/55961ee41a28ab7b0e8b47b1/html5/thumbnails/4.jpg)
Proposed Algorithm
![Page 5: Camera-based Signage Detection and Recognition for Blind Persons](https://reader034.vdocument.in/reader034/viewer/2022051314/55961ee41a28ab7b0e8b47b1/html5/thumbnails/5.jpg)
Image Preprocess
Original
Image
Gray
Image
Binary
Image
Connected
Components
![Page 6: Camera-based Signage Detection and Recognition for Blind Persons](https://reader034.vdocument.in/reader034/viewer/2022051314/55961ee41a28ab7b0e8b47b1/html5/thumbnails/6.jpg)
Signage Detection: Head Based on shape of Connected Components (CC)
Head shape is circle
![Page 7: Camera-based Signage Detection and Recognition for Blind Persons](https://reader034.vdocument.in/reader034/viewer/2022051314/55961ee41a28ab7b0e8b47b1/html5/thumbnails/7.jpg)
Signage Detection: Body More variations
Close to head
Based on shape
![Page 8: Camera-based Signage Detection and Recognition for Blind Persons](https://reader034.vdocument.in/reader034/viewer/2022051314/55961ee41a28ab7b0e8b47b1/html5/thumbnails/8.jpg)
Signage Detection Results Scale invariant
Rotation invariant
Illumination invariant
![Page 9: Camera-based Signage Detection and Recognition for Blind Persons](https://reader034.vdocument.in/reader034/viewer/2022051314/55961ee41a28ab7b0e8b47b1/html5/thumbnails/9.jpg)
Signage Recognition: Find Corners Search within detected signage region
SIFT (Scale Invariant Feature Transform) detector
Search over all scales
Template
Signage
Detected
Signage
![Page 10: Camera-based Signage Detection and Recognition for Blind Persons](https://reader034.vdocument.in/reader034/viewer/2022051314/55961ee41a28ab7b0e8b47b1/html5/thumbnails/10.jpg)
Signage Recognition: Match Corners SIFT descriptor
Histogram of gradients
Rotation and scale invariant
Matching pair of corners: minimal Euclidean distance
Find the template with maximum matching pairs
Template
Signage
Detected
Signage
![Page 11: Camera-based Signage Detection and Recognition for Blind Persons](https://reader034.vdocument.in/reader034/viewer/2022051314/55961ee41a28ab7b0e8b47b1/html5/thumbnails/11.jpg)
Signage Database 102 Signage: Men (50); Women(42); Disabled(10)
![Page 12: Camera-based Signage Detection and Recognition for Blind Persons](https://reader034.vdocument.in/reader034/viewer/2022051314/55961ee41a28ab7b0e8b47b1/html5/thumbnails/12.jpg)
Experiment Results 89.2% detection rate (91 out of 102 images)
84.3% recognition rate (86 out of 102 images)
Confusion matrix: column is the ground truth
![Page 13: Camera-based Signage Detection and Recognition for Blind Persons](https://reader034.vdocument.in/reader034/viewer/2022051314/55961ee41a28ab7b0e8b47b1/html5/thumbnails/13.jpg)
Original
Image
Binary
Image
Connected
Component
Signage
Recognition
W MDW M D
Intermediate Results
![Page 14: Camera-based Signage Detection and Recognition for Blind Persons](https://reader034.vdocument.in/reader034/viewer/2022051314/55961ee41a28ab7b0e8b47b1/html5/thumbnails/14.jpg)
MM
MM W
W
WW
W
W
W
W
M
MD
D DD
Recognition Success
![Page 15: Camera-based Signage Detection and Recognition for Blind Persons](https://reader034.vdocument.in/reader034/viewer/2022051314/55961ee41a28ab7b0e8b47b1/html5/thumbnails/15.jpg)
Detection Fails
Significant view angle changes
Complex Background
![Page 16: Camera-based Signage Detection and Recognition for Blind Persons](https://reader034.vdocument.in/reader034/viewer/2022051314/55961ee41a28ab7b0e8b47b1/html5/thumbnails/16.jpg)
Acknowledgement Supported:
NIH 1R21EY020990,
NSF grants IIS-0957016
EFRI-1137172.
16
![Page 17: Camera-based Signage Detection and Recognition for Blind Persons](https://reader034.vdocument.in/reader034/viewer/2022051314/55961ee41a28ab7b0e8b47b1/html5/thumbnails/17.jpg)
Thank you!
![Page 18: Camera-based Signage Detection and Recognition for Blind Persons](https://reader034.vdocument.in/reader034/viewer/2022051314/55961ee41a28ab7b0e8b47b1/html5/thumbnails/18.jpg)
Author Contact
Shuihua Wang and Yingli Tian{swang15, ytian}@ccny.cuny.edu