event info extraction from mobile camera imageskw303mf0907/zhang...event info extraction from mobile...

Event info extraction from mobile camera images Haoran Li, Hao Zhang and Yang Zhang Goal Every day, we encounter a lot of events information from the paper flyers posted on the hallway walls, entrance doors and elevators in our buildings. Will it be a presentation, a forum discussion or a concert; they all come with a data & time and venue. Because these information are not digitized, it is very inconvenient for us to manually input such information into our smartphone calendars. Therefore, we are proposing to build an OCR‐based mobile image processing system that can automatic extract event information from flyer pictures and directly integrate them into the users’ digital calendars. Our goal is to design a small tool on Android platform that enables the user to put the event info (time and location) into his personal calendar by simply snapping a picture on the related flyer/poster. Project designs Obviously, the main part of the project is to build an OCR‐based mobile image processing system, using the techniques we learn on 368 class [1][2] .We propose the following workflow in our design: 1) Input a single image captured from the smartphone camera. 2) Apply several preprocessing steps to the original image to improve the image quality, for example, illumination balancing/histogram equalization, denoising, spatial transform[ 2] to correct geometric distortion caused by tilted capturing angle. 3) Detect the region of interest (ROI) using a Maximally stable extremal regions (MSER) [3] detector. By using some criterions, we hope to filter out areas that are not likely to be the text we are interested in. 4) Binarize the image for text recognition. 5) Use an Android‐based OCR engine (Tesseract‐ocr [4] ) to recognize the actual text in the ROI. 6) Parse the text we get from the OCR output, search specifically for the dates/venue/event name information among the text and extract them. 7) Invoke the system’s calendar system, send this event info to the calendar system automatically [5] , and the user just need to decide whether he/she would want to add it to schedule. Literature Survey We did some survey on the Internet, and did not found any mobile app on Android store that does the exact functionality we proposed. On the technical side, there are many literatures on building image recognition systems on mobile phones , [6] for example, and implementation wise, there are already many applications on image recognition and OCR detection like Google Goggles, but these apps are not specifically targeting event‐calendar management, and do not provide good user experience for our purpose.

Upload: others

Post on 06-Aug-2020

0 views

Category:

Documents

0 download

Report

Download

Embed Size (px):

TRANSCRIPT

Eventinfoextractionfrommobilecameraimages

HaoranLi,HaoZhangandYangZhang

GoalEveryday,weencounteralotofeventsinformationfromthepaperflyerspostedonthehallwaywalls,entrancedoorsandelevatorsinourbuildings.Willitbeapresentation,aforumdiscussionoraconcert;theyallcomewithadata&timeandvenue.Becausetheseinformationarenotdigitized,itisvery

inconvenientforustomanuallyinputsuchinformationintooursmartphonecalendars.Therefore,weareproposingtobuildanOCR‐basedmobileimageprocessingsystemthatcanautomaticextracteventinformationfromflyerpicturesanddirectlyintegratethemintotheusers’digitalcalendars.Ourgoalis

todesignasmalltoolonAndroidplatformthatenablestheusertoputtheeventinfo(timeandlocation)intohispersonalcalendarbysimplysnappingapictureontherelatedflyer/poster.

ProjectdesignsObviously,themainpartoftheprojectistobuildanOCR‐basedmobileimageprocessingsystem,usingthetechniqueswelearnon368class[1][2].Weproposethefollowingworkflowinourdesign:

1) Inputasingleimagecapturedfromthesmartphonecamera.

2) Applyseveralpreprocessingstepstotheoriginalimagetoimprovetheimagequality,forexample,illuminationbalancing/histogramequalization,denoising,spatialtransform[2]tocorrectgeometricdistortioncausedbytiltedcapturingangle.

3) Detecttheregionofinterest(ROI)usingaMaximallystableextremalregions(MSER)[3]detector.Byusingsomecriterions,wehopetofilteroutareasthatarenotlikelytobethetextweareinterestedin.

4) Binarizetheimagefortextrecognition.5) UseanAndroid‐basedOCRengine(Tesseract‐ocr[4])torecognizetheactualtextintheROI.6) ParsethetextwegetfromtheOCRoutput,searchspecificallyforthedates/venue/eventname

informationamongthetextandextractthem.7) Invokethesystem’scalendarsystem,sendthiseventinfotothecalendarsystem

automatically[5],andtheuserjustneedtodecidewhetherhe/shewouldwanttoadditto

schedule.

LiteratureSurveyWedidsomesurveyontheInternet,anddidnotfoundanymobileapponAndroidstorethatdoestheexactfunctionalityweproposed.Onthetechnicalside,therearemanyliteraturesonbuildingimagerecognitionsystemsonmobilephones,[6]forexample,andimplementationwise,therearealready

manyapplicationsonimagerecognitionandOCRdetectionlikeGoogleGoggles,buttheseappsarenotspecificallytargetingevent‐calendarmanagement,anddonotprovidegooduserexperienceforourpurpose.

Page 2: Event info extraction from mobile camera imageskw303mf0907/Zhang...Event info extraction from mobile camera images Haoran Li, Hao Zhang and Yang Zhang Goal Every day, we encounter

References1. EE368,DigitalImageProcessing,http://www.stanford.edu/class/ee368/2. ComputerVision:AlgorithmsandApplications,byRichardSzeliski,2010.SpringerPress.3. J.Matas,O.Chum,M.Urban,andT.Pajdla."Robustwidebaselinestereofrommaximallystable

extremalregions."Proc.ofBritishMachineVisionConference,pp384‐396,20024. TesseractOCRengine,https://code.google.com/p/tesseract‐ocr/5. AndroidCalendarTutorial,http://www.vogella.com/tutorials/AndroidCalendar/article.html

6. JonathanJ.Hull,XuLiu,BernaErol,JameyGraham,andJorgeMoraleda.2010.Mobileimagerecognition:architecturesandtradeoffs.InProceedingsoftheEleventhWorkshoponMobileComputingSystems&Applications(HotMobile'10).ACM,NewYork,NY,USA,84‐88

AsnapshotoftheeventflyersintheelevatorofMitchellBldgoncampus.

TEXTLINE INFORMATION EXTRACTION FROM GRAYSCALE CAMERA …bukhari/data/papers/6-Bukhari-Grayscale... · TEXTLINE INFORMATION EXTRACTION FROM GRAYSCALE CAMERA-CAPTURED DOCUMENT IMAGES

Intelligent Database Systems Lab Presenter: WU, MIN-CONG Authors: Yongzheng Zhang, Rajyashree Mukherjee, Benny Soetarman 2012, ACM Concept Extraction for

1 An Efficient Method for Chip-Level Statistical Capacitance Extraction Considering Process Variations with Spatial Correlation W. Zhang, W. Yu, Z. Wang,

Camera Calibration with Lens Distortion from Low …perception.csl.illinois.edu/matrix-rank/Files/...Camera Calibration with Lens Distortion from Low-rank Textures Zhengdong Zhang

Video Surveillance using a Multi-Camera Tracking and ... · Video Surveillance using a Multi-Camera Tracking and Fusion System Zhong Zhang, Andrew Scanlon, Weihong Yin, Li Yu, Péter

A Self-Reconﬁgurable Camera Array€¦ · Eurographics Symposium on Rendering (2004) H. W. Jensen, A. Keller (Editors) A Self-Reconﬁgurable Camera Array Cha Zhang and Tsuhan Chen

Live Video Montage with a Rotating Camera · 2014-05-22 · Live Video Montage with a Rotating Camera Zilong Dong, Lei Jiang, Guofeng Zhang, Qing Wang†, Hujun Bao† State Key Lab

New RainbowRow: Fast Optical Camera Communication · 2020. 10. 13. · RainbowRow: Fast Optical Camera Communication Xiao Zhang, Li Xiao Department of Computer Science and Engineering

Camera Calibration, by Zhengyou Zhang - Computer · PDF fileChapter 2 CAMERA CALIBRATION Zhengyou Zhang Camera calibration is a necessary step in 3D computer vision in order to extract

A study of copper extraction kinetics with LIX … study of copper extraction kinetics with LIX 984N using a Lewis cell Presented by Wensheng Zhang 8 July 2011 by Wensheng Zhang, Fuping

Photo-realistic 3D Model Extraction from Camera Array Capture