con-text: text detection using background connectivity for fine-grained object classification
DESCRIPTION
Con-Text: Text Detection Using Background Connectivity for Fine-Grained Object Classification. Sezer Karaoglu, Jan van Gemert, Theo Gevers. Can we achieve a better object recognition with the help of scene-text ?. Goal. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Con-Text: Text Detection Using Background Connectivity for Fine-Grained Object Classification](https://reader036.vdocument.in/reader036/viewer/2022062519/568151dc550346895dc014d9/html5/thumbnails/1.jpg)
Con-Text: Text Detection Using Background Connectivity for Fine-Grained Object Classification
Sezer Karaoglu, Jan van Gemert, Theo Gevers
1
![Page 2: Con-Text: Text Detection Using Background Connectivity for Fine-Grained Object Classification](https://reader036.vdocument.in/reader036/viewer/2022062519/568151dc550346895dc014d9/html5/thumbnails/2.jpg)
Can we achieve a better object recognition with the help of scene-text?
2
![Page 3: Con-Text: Text Detection Using Background Connectivity for Fine-Grained Object Classification](https://reader036.vdocument.in/reader036/viewer/2022062519/568151dc550346895dc014d9/html5/thumbnails/3.jpg)
Goal
• Exploit hidden details by text in the scene to improve visual classification of very similar instances.
Application : Linking images from Google street view to textual business inforation as e.g. the Yellow pages, Geo-referencing, Information retrieval
3
SKYSKYSKY
CAR CAR
DJ SUBS Breakfast Starbucks Coffee Starbucks Coffee
![Page 4: Con-Text: Text Detection Using Background Connectivity for Fine-Grained Object Classification](https://reader036.vdocument.in/reader036/viewer/2022062519/568151dc550346895dc014d9/html5/thumbnails/4.jpg)
Challenges of Text Detection in Natural Scene Images
o Lightingo Surface Reflectionso Unknown backgroundo Non-Planar objectso Unknown Text Fonto Unknown Text Sizeo Blur
4
![Page 5: Con-Text: Text Detection Using Background Connectivity for Fine-Grained Object Classification](https://reader036.vdocument.in/reader036/viewer/2022062519/568151dc550346895dc014d9/html5/thumbnails/5.jpg)
Literature Review Text Detection
• Texture Based: Wang et al. “End-to-End Scene Text Recognition”
ICCV ‘11
Computational ComplexityDataset specific
Do not rely on heuristic rules
• Region Based: Epshtein et al. “Detecting Text in Natural Scenes
with Stroke Width Transform ” CVPR ‘10
Hard to define connectivity Segmentation helps to improve ocr performance
5
![Page 6: Con-Text: Text Detection Using Background Connectivity for Fine-Grained Object Classification](https://reader036.vdocument.in/reader036/viewer/2022062519/568151dc550346895dc014d9/html5/thumbnails/6.jpg)
Motivation to remove background for Text Detection
• To reduce majority of image regions for further processes.• To reduce false positives caused by text like image regions (fences, bricks,
windows, and vegetation).• To reduce dependency on text style.
![Page 7: Con-Text: Text Detection Using Background Connectivity for Fine-Grained Object Classification](https://reader036.vdocument.in/reader036/viewer/2022062519/568151dc550346895dc014d9/html5/thumbnails/7.jpg)
7
Automatic BG seed selection BG reconstructionText detection by BG
substraction
Proposed Text Detection Method
![Page 8: Con-Text: Text Detection Using Background Connectivity for Fine-Grained Object Classification](https://reader036.vdocument.in/reader036/viewer/2022062519/568151dc550346895dc014d9/html5/thumbnails/8.jpg)
Background Seed Selection
• Color, contrast and objectness responses are used as feature.• Random Forest classifier with 100 trees based on out-of-bag error are used to create forest.• Each tree is constructed with three random features.• The splitting of the nodes is made based on GINI criterion.
Original Image Color Boosting Contrast Objectness
![Page 9: Con-Text: Text Detection Using Background Connectivity for Fine-Grained Object Classification](https://reader036.vdocument.in/reader036/viewer/2022062519/568151dc550346895dc014d9/html5/thumbnails/9.jpg)
Conditional Dilation for BG connectivity
where B is the structring element (3 by-3 square), M is the binary image where bg seeds are ones and X is the gray level input image
untilrepeat
![Page 10: Con-Text: Text Detection Using Background Connectivity for Fine-Grained Object Classification](https://reader036.vdocument.in/reader036/viewer/2022062519/568151dc550346895dc014d9/html5/thumbnails/10.jpg)
Text Recognition Experiments
• ICDAR’03 Dataset with 251 test images, 5370 characters, 1106 words.
10
![Page 11: Con-Text: Text Detection Using Background Connectivity for Fine-Grained Object Classification](https://reader036.vdocument.in/reader036/viewer/2022062519/568151dc550346895dc014d9/html5/thumbnails/11.jpg)
ICDAR 2003 Dataset Char. Recognition Results
11
Method Cl. Rate (%)ABBYY 36Karaoglu et. al. 62Proposed 63
The proposed system removes 87% of the non-text regions where on average 91% of the test set contains non-text regions. It retains approximately %98 of text regions.
![Page 12: Con-Text: Text Detection Using Background Connectivity for Fine-Grained Object Classification](https://reader036.vdocument.in/reader036/viewer/2022062519/568151dc550346895dc014d9/html5/thumbnails/12.jpg)
ImageNet Dataset
• ImageNet building and place of business dataset ( 24255 images 28 classes, largest dataset ever used for scene tekst recognition)
• The images do not necessarily contain scene text.• Visual features : 4000 visual words, standard gray SIFT only.• Text features: Bag-of-bigrams , ocr results obtained for each image
in the dataset.• 3 repeats, to compute standard deviations in Avg. Precision.• Histogram Intersection Kernel in libsvm.• Text only, Visual only and Fused results are compared.
Steak PizzeriaFuneralBakery Discount HouseCountry House
![Page 13: Con-Text: Text Detection Using Background Connectivity for Fine-Grained Object Classification](https://reader036.vdocument.in/reader036/viewer/2022062519/568151dc550346895dc014d9/html5/thumbnails/13.jpg)
Fine-Grained Building Classification Results
ocr : 15.6 ± 0.4 Bow : 32.9 ± 1.7
Text Visual Fusion
Bow + ocr : 39.0 ± 2.6
#269 #431 #584 #2752
#1 #4 #5 #8
Visual
Text
Proposed
Discount House
#1 #4 #5 #8
![Page 14: Con-Text: Text Detection Using Background Connectivity for Fine-Grained Object Classification](https://reader036.vdocument.in/reader036/viewer/2022062519/568151dc550346895dc014d9/html5/thumbnails/14.jpg)
Conclusion
• Background removal is a suitable approach for scene text detection • A new text detection method, using background connectivity and, color, contrast and objectness cues is proposed.• Improved performance to scene text recognition. • Improved Fine-Grained Object Classification performance with visual and scene text information fusion.
14
![Page 15: Con-Text: Text Detection Using Background Connectivity for Fine-Grained Object Classification](https://reader036.vdocument.in/reader036/viewer/2022062519/568151dc550346895dc014d9/html5/thumbnails/15.jpg)
DEMO
TRY HERE