from a nthrax to z ip codes - the handwriting is on the wall
DESCRIPTION
From A nthrax to Z IP Codes - The Handwriting is on the Wall. Venu Govindaraju Dept. of Computer Science & Engineering University at Buffalo [email protected]. Outline. Success in Postal Application Role of Handwritten Word Recognition Word Recognition - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/1.jpg)
From Anthrax to ZIP Codes-The Handwriting is on the Wall
Venu GovindarajuDept. of Computer Science & Engineering
University at Buffalo
![Page 2: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/2.jpg)
Outline
• Success in Postal Application
• Role of Handwritten Word Recognition
• Word Recognition– Lexicon Driven Word Recognition
– Lexicon Free Word Recognition
• New Models– Interactive Cognitive Models
• New Research Areas– Lexicon density
– Lexicon Reduction and Combination
• Other Applications
![Page 3: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/3.jpg)
USPS HWAI Background
• Postal Sponsorship Started – 1984
• 370 Academic Articles Published
• Millions of Letters Examined
• Many Experimental Systems Built and Tested
• Migrated from Hardware to Software System
• Only Postal Research Continuously Funded
![Page 4: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/4.jpg)
Items to be Recognized, Read, and Evaluated (Machine printed and Script)
• Delivery address, sender´s address, endorsements
• Linear Codes, Mail Class
• Indicia (2D-Codes, Meter Marks)
Meter Mark
Sender’s Address
Delivery Address
Linear Code
Digital Post MarkEndorsem
entIn Case of Undeliverable as Addressed Return to Sender
Pattern Recognition Tasks
![Page 5: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/5.jpg)
Deployed..
• USA– 250 P&DC sites
– 27 Remote Encoding Centers
– 25 Billion Images Processed Annually
– 89% Automated Bar-coding
• UK– 67 Processing Centers
– 27 Million Pieces Per Day,
– 9.7 Million Pieces Per Hour Peak
• Australia
![Page 6: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/6.jpg)
Scope - Others
• Royal Mail– 67 Processing Centers– 27 Million Pieces Per Day– 9.7 Million Pieces Per Hour Peak
• Australia Post– Similar to Royal Mail
![Page 7: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/7.jpg)
C o l l e c t i o nM a i l
R C R
B a rC o d eS o r t e r s
P O S T N E Tb a r c o d e R e s u l t s
R e m o t eE n c o d i n gS i t e
M L O C R
H a n d w r i t t e n
F I M ( p r e - b a r c o d e d )
M a c h i n e
M a i l
A F C SA F C SA F C S
P r e -B a r c o d e d
M L O C RM L O C R
U S P S M a i l F l o wM L O C RR e j e c t s
D e l i v e r yP o i n t B a r - c o d e dM a i l p i e c e s
M L O C RA s s i g n e d
I P S S
![Page 8: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/8.jpg)
RCR Overview
Bar Code Sorter
RemoteEncodin
g
Advanced Facer
CancelerMulti-Line
OCR
Image
RCR
![Page 9: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/9.jpg)
The Right Technology
• Technological Nexus
– Sophisticated Algorithms
– High Speed Processors
– Large Disk Capacities
– High Speed Memories
![Page 10: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/10.jpg)
At the Right Price
Processing Type Cost/1000 Pieces
Manual $47.78
Mechanized $27.46
Automated $5.30
![Page 11: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/11.jpg)
80% encode rate and counting!
Handwriting Encode Rate
0%
10%20%
30%40%
50%
60%70%
80%
Date
En
co
de
Ra
te
![Page 12: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/12.jpg)
Impact
• Applications of CEDAR research helping to automate tasks at IRS and USPS– 1st year that USPS used CEDAR-developed
software to read handwritten addresses on envelopes, saved $100 million
– 1997-1999 USPS deployment of CEDAR-developed RCRs, USPS saved 12 million work hours and over $340 million
• 500 scientific publications and 10 patents
![Page 13: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/13.jpg)
Outline
• Success in Postal Application
• Role of Handwritten Word Recognition
• Word Recognition– Lexicon Driven Word Recognition
– Lexicon Free Word Recognition
• New Models– Interactive Cognitive Models
• New Research Areas– Lexicon density
– Lexicon Reduction and Combination
• Other Applications
![Page 14: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/14.jpg)
![Page 15: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/15.jpg)
Chaincode Generation
Pre-scan with Digit Recognizer
Line Segmentation
Word Separation
Parsinga) shapeb) syntax
Digit String Recognition
Database Queries
Phrase Recognition
Encoding Strategy
5, 9, or 11 digit encode
OR reject
Address Block Image
Yes
Finalized?
Pass 1or
Pass 2
Adaptive Image
Enhancement
No
Pass 1 Pass 2
14221 3851 11
Input
Output
Output
Handwritten Address Interpretation (HWAI)
![Page 16: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/16.jpg)
• <ZIP Code, Primary Number>– Create street name lexicon
<06478, 110>• DPF yields 8 street names
• ZIP+4 yields 31 street names (on average about 5 times more)
HAWLEY RD 1034NEWGATE RD 1533BEE MOUNTAIN RD 1615DORMAN RD 1642BOWERS HILL RD 1757FREEMAN RD 1781PUNKUP RD 1784PARK RD 6124
Context Provided by Postal Directories
![Page 17: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/17.jpg)
• One record per delivery point in USA
• Provided weekly by USPS, San Mateo
• Raw DPF – 138 million records– 15 GB (114 bytes per record);– 41,889 ZIP Code files
• Fields of interest to HWAI– ZIP Code, record type (eg., street, firm, PO
Box ..), street name, primary number, secondary number, add-on
Delivery Point FileCEDAR
![Page 18: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/18.jpg)
• ZIP Code– 30% of ZIP Codes contain a single street name– 5% of ZIP Codes contain a single primary number– 2% of ZIP Codes contain a single add-on
• <ZIP Code, primary number>– Maximum number of records returned is 3,071
• <ZIP Code, add-on>– Maximum number of records returned is 3,070
Relevant Statistics
CEDAR
![Page 19: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/19.jpg)
Outline
• Success in Postal Application
• Role of Handwritten Word Recognition
• Word Recognition– Lexicon Driven Word Recognition
– Lexicon Free Word Recognition
• New Models– Interactive Cognitive Models
• New Research Areas– Lexicon density
– Lexicon Reduction and Combination
• Other Applications
![Page 20: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/20.jpg)
Word Recognition Engine
Bryant 2.3Boston 1.8Bidwell 2.6James 4.7Buffalo 8.9:::::
Rankedlexiconwith distance scores
Signal
Handwriting Recognition
BostonBuffaloWilliamsvilleBidwellJamesByrant....
ContextLexicon
![Page 21: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/21.jpg)
WMR
1 2 3 4 5 6 7 8 9
w[7.6]
w[7.2]r[3.8]
w[5.0]
w[8.6]
o[7.6]r[6.3]
d[4.9]
w[5.0]
o[6.6]
o[6.0]
o[7.2]o[10.6] d[6.5]
d[4.4]
r[7.5]r[6.4]
o[7.8]r[8.6]
o[8.7]r[7.4]
r[7.6]
o[8.3]
o[7.7]r[5.8]
1 2 3 4 5 6 7 8 9
o[6.1]
Find the best way of accounting for characters ‘w’, ‘o’, ‘r’, ‘d’ buy consuming all segments 1 to 8 in the process
Distance between lexicon entry ‘word’ first character ‘w’ and the image between:- segments 1 and 4 is 5.0- segments 1 and 3 is 7.2- segments 1 and 2 is 7.6
![Page 22: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/22.jpg)
CMR
4
5
67 82 3
1
1 32 4 5 6 7 8i[.8], l[.8] u[.5], v[.2]
w[.6], m[.3]
w[.7]
i[.7]u[.3]
m[.2]m[.1]
r[.4]
d[.8]o[.5]
-Image from 1 to 3 is a in with 0.5 confidence-Image from segment 1 to 4 is a ‘w’ with 0.7 confidence-Image from segment 1 to 5 is a ‘w’ with 0.6 confidence and an ‘m’ with 0.3 confidence
Find the best path in graph from segment 1 to 8
w o r d
![Page 23: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/23.jpg)
Outline
• Success in postal application
• Role of Handwritten Word Recognition
• Word Recognition– Lexicon Driven Word Recognition
– Lexicon Free Word Recognition
• New Models– Interactive Cognitive Models
• New Research Areas– Lexicon density
– Lexicon Reduction and Combination
• Other Applications
![Page 24: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/24.jpg)
Multiple Choice Paradigm
a) Amherst b) Buffalo c) Bostond) None of the above
![Page 25: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/25.jpg)
Grapheme Models
![Page 26: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/26.jpg)
Stochastic Models and Continuous Attributes
grapheme pos orientation angle
Down cusp
3.0 -90o
Up loop
Down arc
![Page 27: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/27.jpg)
ResultsLex size Top WMR % SM CA%
10 1 96.86 96.56
2 98.80 98.77
100 1 91.36 89.12
2 95.30 94.06
1000 1 79.58 75.38
2 88.29 86.29
50 98.00 98.40
20000 1 62.43 58.14
2 71.07 66.49
100 93.59 93.39
![Page 28: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/28.jpg)
Interactive Models[McClelland and Rumelhart, Psychological Review, 1981]
ABLE TRIPTRAP
A TN
Words
Letters
Features
![Page 29: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/29.jpg)
Cognitive Handwritten Word Recognition
T-crossings, loops, ascenders, descenders, length
West Central StreetWest Main StreetSunset Avenue
West Central StreetEast Central StreetSunset Avenue
West Central StreetWest Central AvenueSunset Avenue
Lexicon 1 Lexicon 2 Lexicon 3
Interactive Model
features
image
![Page 30: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/30.jpg)
Adaptive Character Recognition[Park and Govindaraju, IEEE CVPR 2000]
•Adaptive selection of features
•Adaptive number of features
•Adaptive resolutions
•Adaptive sequencing of features
•Adaptive termination conditions
![Page 31: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/31.jpg)
Features4 gradient features 5 moment features
Vector code book
![Page 32: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/32.jpg)
Feature Space
• |V| x |Nc| x |Ixy|
• 29 x 10 x 85 (quad tree, 4 levels)
• Recognition rate and feature |V|
• GSC: |V| : 2512
• Tradeoffs: space vs accuracy– Hierarchical space with additional
resolution and features as needed
![Page 33: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/33.jpg)
Active Recognition Using Quad Trees
![Page 34: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/34.jpg)
Experimental Results
![Page 35: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/35.jpg)
![Page 36: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/36.jpg)
ResultsClassifier Active Model Neural
NetKNN
Top 1% 95.7 % 96.4% 95.7%
Templates 612 976 3,777
Msec/char 1.45 11.5 384
Training hrs 1 24 1
10 class digit recognition
25656 training and 12242 test (Postal +NIST)
![Page 37: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/37.jpg)
Outline
• Success in Postal Application
• Role of Handwritten Word Recognition
• Word Recognition– Lexicon Driven Word Recognition
– Lexicon Free Word Recognition
• New Models– Interactive Cognitive Models
• New Research Areas– Lexicon Reduction and Combination
– Lexicon Density and Prediction of Performance
• Other Applications
![Page 38: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/38.jpg)
Combination and Dynamic Selection[Govindaraju and Ianakiev, MCS 2000]
WR 1
WR 2
WR 3+Lexicon
1
Top 5
<55Top 50
image
•Optimization problem
•Combinatorial explosion in
•arrangement of recognizers
•lexicon reduction levels
![Page 39: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/39.jpg)
Lexicon Density[Govindaraju, Slavik, and Xue, IEEE PAMI 2002]
Lexicon 1 Lexicon 2
Me MeHe MemoSo MemoryTo MemoirsIn Mellon
![Page 40: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/40.jpg)
Classifier Performance Prediction[Xue and Govindaraju, IEEE PAMI 2002]
q: probability that recognizer make a unit distance errors
D: average distance between any two words in the lexicons
n: lexicon size; p: performance; a, k,: model parameters
ln (-ln p) = (ln q) D + a ln ln n + ln k
![Page 41: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/41.jpg)
Outline
• Success in Postal Application
• Role of Handwritten Word Recognition
• Word Recognition– Lexicon Driven Word Recognition
– Lexicon Free Word Recognition
• New Models– Interactive Cognitive Models
• New Research Areas– Lexicon density
– Lexicon Reduction and Combination
• Other Applications
![Page 42: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/42.jpg)
Bank Check Recognition
![Page 43: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/43.jpg)
PCR Trend Analysis
![Page 44: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/44.jpg)
NYS EMS PCR FormNYS PCR Example
Thousands are filed a day.Passed from EMS to Hospital.
PCR Purpose:– Medical care/diagnosis– Legal Documentation– Quality Assurance
EMS AbbreviationsCOPD Chronic Obstructive Pulmonary DiseaseCHF Congestive Heart FailureD/S Dextrose in SalinePID Pelvic Inflammatory DiseaseGSW Gunshot WoundNKA No known allergiesKVO Keep vein openNaCL Sodium Chloride
![Page 45: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/45.jpg)
Medical Text Recognition and Data Mining
![Page 46: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/46.jpg)
Reading Census Forms
Lexicon Anomalies
Space: “sales man” and “salesman”
Morphology: “acct manager” and “account management”
Abbreviation
Plural: “school” and “schools”
Typographical: “managar” and “manager”
![Page 47: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/47.jpg)
Binarization
![Page 48: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/48.jpg)
Historic Manuscripts
![Page 49: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/49.jpg)
Mapping Snippets with Transcribed Text
![Page 50: From A nthrax to Z IP Codes - The Handwriting is on the Wall](https://reader035.vdocument.in/reader035/viewer/2022062422/568137e4550346895d9f92bf/html5/thumbnails/50.jpg)
Summary
• Handwriting recognition technology
• Pattern recognition task
• Lexicon holds domain specific knowledge
• Adaptive methods
• Classifier combination methods
• Many applications