self-directing text detection and removal from images with smoothing

A Presentation

on

Self-Directing Text Detection and Removal from

Images with SmoothingBy

WAGH PRIYANKA DEELIPUnder the Guidance of

PROF. D. R. PATIL

Department of Computer Engineering

SES’s R.C.Patel Institute of Technology, Shirpur

Maharashtra State, India

2014-151

Outline

Introduction

Literature Survey

Problem Definition

Objectives

Methodology

Experimental Results

Conclusions

References

Publications

2

Introduction

Images consists of Text in various forms such as: logos,

subtitles, captions, banners etc.

Figure: Examples of Logos, captions, subtitles

3

Introduction Contd..

Drawback of having text in Image:

Occludes important portion of the Image and easy to index.

That’s why Text detection and removal system is required.

4


The stages of Automatic Text detection and removal

scheme:

1) Self-directing Text detection

2) An Effective hole filling after the text removal [1].

Figure: Text Detection and Removal results

5


Figure: Existing Architecture for Automatic Text Detection and

Removal from Image System

6


An Automated Text Detection:

Computer system should answer:

Where is a text string in image?

Using such system text embedded in complex backgrounds

can be automatically detected.

7


Figure: Basic Architecture for Automatic Text Detection from Image

System

8


Applications of Automated Text Detection:

Optical Character Recognition.

Content-based video coding or document coding.

License/container plate recognition.

Text based image indexing.

Automated text removal system.

9


An Automated Region filling:A process of filling holes with surrounding region of image

without help of user.Basic methods available for region filling:

1. Texture Synthesis: For large holes.

2. Inpainting: For small scratches.

Figure: Region filling10


Texture Synthesis:

A process of algorithmically constructing a large digital im-

age from a small digital sample image.

Figure: Texture Synthesis

11


Inpainting:

Inpainting is the process of reconstructing lost or deterio-

rated parts of images and videos.

Figure: Inpainting

12

Introduction Contd...

Applications of Region filling:

Restoration of old photographs and damaged films.

Removal of superimposed texts.

Object Removal.

Occlusion or shadow removal.

Adding special effects.

Getting the clear visibility of original image while removing

unwanted elements of images for research purpose in insti-

tutes like NASA or defense DRDO.

To get clear view and gap filling of satellite images.

13

Literature Survey

Mohammad Khodadadi et. al. have proposed:

A method for text localization, extraction and inpainting.

Localization using: stroke filter and segmentation.

Extraction using: Background and text color estimation and

color histogram of candidate text blocks.

Inpainting: Based on matching algorithm and priority for in-

painting of pixel [1].

14

Literature Survey Contd..

J. Malobabic et. al. have proposed:

A method for text detection and localization for superimposed

video text using horizontal difference magnitude measure

and morphological processing.

Smoothing and multiple binarisation is used for enhancing

result.

The presented evaluation is based upon manually selected

best detection results.

Further research is essential to bring automaticity in this pro-

cess [2].

15


Boris Epshtein et. al. have proposed:

Stroke width Transformation for text detection.

No need for multi-scale computation or scanning windows.

Reliable, Efficient and Language independent along with 15

times faster [3].

16


Ali Mosleh et. al. have proposed:

A method for text detection using Feature vector.

Feature vectors composed of: Directionality of gradient of

text edges, High contrast with background, geometric prop-

erties of text components.

K-means clustering for text and non-text distinguishing.

A novel bandlet based edge detector for obtaining edges [4].

17


Huizhong Chen et. al. have proposed:

A method to employ edge-enhanced Maximally stable ex-

tremal Regions as basic letter candidates.

Candidates filtering to exclude non-text objects by use of ge-

ometric and stroke width information.

Identification of text lines using letter pairing.

Dataset: ICDAR and mobile document database [5].

18


Marcelo Bertalmio et. al. have proposed:

Method of digital automatic inpainting.

Fill-in: Isophote lines arriving at the regions’ boundaries are

completed inside.

Advantage: No need for user to specify where the novel in-

formation comes from [6].

19


Lai-Man Po et. al. have proposed:

Multidirectional extrapolation hole filling method with com-

plex texture background.

Hole filling direction estimation using neighbor pixel’s texture

features.

Better virtual views synthesis with high quality depth map for

large hole fillings [8].

20

Problem Definition

Currently existing options for text detection and removal are

based on users and their technical skills to use those tools.

These tools and softwares consumes lot of time and ef-

forts utilization for text detection and as well as region filling.

Current systems are not at all common/non-technical user

friendly.

21

Objectives

To study and use the effects of smoothing on automatic text

detection system in order to improve the performance of

system.

To embed new inpainting method to bring more visually

plausible hole fillings.

To compare the results of smoothing based method, new

inpainting based method with existing methods.

22

Methodology

Text detection using text localization and extraction with smooth-

ing and Exemplar based Inpainting method. i.e. (TLES+EBI)

Main stages of the Automatic video Text detection and re-moval system are as [1]:

1. Text Localization: to approximately detect text regions.

2. Text Extraction: extract more perfect text regions from un-

wanted regions.

3. Inpainting: Fill the holes generated using surrounding region

data.

23

Methodology Contd...

Modified architecture of text detection and removal to im-

prove performance of text detection:

Figure: 4 Modified architecture

24


25


What is Smoothing?A process to create an approximating function that attempts:

To capture important patterns in the data.

To leave out noise or other fine-scale structures/rapid phe-

nomena.

Individual points in signal are reduced.

Points lower than adjacent points are increased leading to a

smoother signal.

26


Effect of smoothing [9]:

Figure: 5 Original Image and Image after L0 Smoothing

27


Why L0 gradient minimization?

Its effective method for sharpening edges [9]:

1. by increasing steepness of transition.

2. by eliminating low amplitude structures in statistical manner.

Doesn’t depend on local features but globally locates impor-

tant edges.

Retains primary color change by restricting drastic color change

of many pixels and reduces fractional diversities of patters

in image.

Characterizes and enhances fundamental image constituents.

28


Figure: 5 Basic steps of L0 Gradient minimization Smoothing

29


30


Figure: Text Detection Algorithm [1]

31


32


Why Exemplar Based Inpainting?

Proved better visibly plausible results in field of inpainting

where NBMI lags.

Combines advantages of two methods:

1. Texture Synthesis: Filling large holes with use of large

region generation from sample texture.

2. Inpainting: Filling small scratches or gaps or holes using

diffusion.

Simultaneous texture and structure information propagation

achieved by single efficient algorithm.

Block based sampling provides computational efficiency [7].

33


Working methodology [7]:

Figure: Exemplar based inpainting methodology

34

Implementation

Figure: Main Window

35

Implementation Contd...

Figure: Browse Window

36


Figure: Text Detection Results

37


Figure: Text Removal and Inpainting Results

38

Experimental Results

Text Detection Evaluation:

Figure: Detection Rate Evaluation

39

Experimental Results cont...

Comparison for text detection rate:

Figure: Text Detection Results

40


PSNR and MSE values for each method:

Figure: PSNR and MSE Result Table

41


Comparison for MSE of inpaiting outputs:

Figure: MSE Comparison

42


Comparison for PSNR of inpaiting outputs:

Figure: PSNR Comparison

43

Conclusions

The smoothing procedure can improve text detection rate

while decreasing false positive and false negative values.

In case of 5.4560% text image detection rate has been

achieved to 66.0444.

The exemplar based inpainting give more visually plausible

output than existing method.

The MSE decreases while PSNR values increases with

use of smoothing and exemplar based inpainting method in

text detection and removal system to 1.9235e-04 and

85.2899 respectively.

44

ReferencesM. Khodadadi and A. Behrad, “Text Localization, Extraction and Inpainting in color Images", In proc. of Iranian

Conference on Electrical Engineering (ICEE2012), pp.1035-1040, May, 2012.

J. Malobabic and N. OŠConnor and N. Murphy and S. Marlow, “Automatic Detection and Extraction of Artificial

Text in Video", WIAMIS- 5th International Workshop on Image Analysis for Multimedia Interactive Services,

2004.

Boris Epshtein and Eyal Ofek and Yonatan Wexler, “Detecting text in natural scenes with stroke width trans-

form", In proc. of IEEE conference on Computer Vision and Pattern Recognition(CVPR), 2010, pp. 2963-2970,

June, 2010.

Ali Mosleh and Nizar Bouguila and A. Ben Hamza, “Image Text Detection Using a Bandlet-Based Edge De-

tector and Stroke Width Transform", In proc. of British Machine Vision Conference, pp. 63.1-63.12, 2012

Huizhong Chen and Sam S. Tsai and Georg Schroth and David M. Chen and Radek Grzeszczuk and Bernd

Girod, “Robust Text Detection in Natural Images with Edge-Enhanced Maximally Stable Extremal Regions",

In proc. of IEEE International Conference on Image Processing, Brussels, Sept. 2011.

Marcelo Bertalmio and Guillermo Sapiro and Vicent Caselles and Coloma Ballester, “Image Inpainting", In

proc. of Proceedings of the 27th annual conference on Computer graphics and interactive techniques, pp.

417-424, 2000.

A. Criminisi and P. Perez and K. Toyama, “Region Filling and Object Removal by Exemplar-Based Image

Inpainting", IEEE Transactions on Image Processsing, vol. 13, pp. 1200-1212, 2004.

Lai-Man Po and Shihang Zhang and Xuyuan Xu and Yuesheng Zhu, “A new multidirectional extrapolation

hole-filling method for Depth-Image-Based Rendering", In proc. of 18th IEEE International Conference on

Image Processing (ICIP), pp. 2589 - 2592, 2011.

Li Xu and Cewu Lu and Yi Xu and Jiaya Jia, “Image Smoothing via L0 Gradiet Minimization", IACM Transac-

tions on Graphics (SIGGRAPH Asia), vol. 30, pp. 174:1-174:12, 2011.

45

Publications

Priyanka Deelip Wagh and D. R. Patil, “Survey on Automatic Text Detection, Extraction and Removal from

video or images", In Proc. of National Conference on Emerging Trends in Computer Technology(NCETCT),

pp. 19-23, December 2014.

Priyanka Deelip Wagh and D. R. Patil, “Automatic Text Detection, Extraction and Removal from Video or

images", In Proc. of International Conference on Science and Technology 2K14, Indapur, Maharashtra, 2014.

Priyanka Deelip Wagh and D. R. Patil, “Text Detection and Removal from Image using Inpainting with Smooth-

ing", In Proc. of International Conference on Pervasive Computing 2015, Sinhagad college of Engineering,

Pune, 8-10 January 2015.

Priyanka Deelip Wagh and D. R. Patil, “Self-directing Superimposed Text Detection and Removal from images

using Inpainting for Region Filling with Smoothing", In proc. of International Conference on Advances in En-

gineering and Technology, Anjuman College of Engineering and Technology, Sadar, Nagpur, 25-26 February

2015.

46

Thank You !!!

47

self-directing text detection and removal from images with smoothing

Technology

system text

text localization

text string

automated text removal

selfdirecting text detection

text based image indexing

introduction images

superimposed video text