3.2 kofax partner connect 2013 - transformation modules - advanced track and what is new in ktm

Kofax Transformation Modules – Expertenforum & Neuigkeiten in KTM?

PartnerConnect Deutschland Frankfurt, 31. Januar 2013

Stephan Mayer / Stefan Skrok Presales EMEA

Agenda

How to build a successful KTM project?

New features in KTM

2

KTM for Forms Kofax Capture Add On – Features

Technology Enhancements

Trainable Document Seperation LimLoc Enhancements Kofax Search and Matching Server Mix Print Detection Clustering Utility

This and That Recostar 5 Normalization Format Locator Enhancements Locator Dialog - Testing Script Rotation

Advanced Track & What’s new in KTM - Overview

On Ramp KIC - PDF and Color Documents

Productivity Enhancements – Design Time Benchmarking

Separation Classification Extraction

Knowledge Base Conflict Management Project Merge Tool Project Builder –Test Documents New Xdoc Browser

Productivity Enhancements – Users Localisation Thin Client Enhancements Field and Table Drop Down Lists Sticky Notes (Annotations) Advanced Routing Docking and Zooming

3

The Golden Rule of KTM

4

User productivity? (i.e. how many docs can a

user process per hour)

Automation? (i.e. how much data is

extracted automatically)

The Fallacy of OCR Accuracy

What OCR accuracy do you have?

What is the straight-through processing rate?

How much can we automate?

85% straight-through processing

23 fields → 99.29% field accuracy

6 chars/field → 99.89% character accuracy

What is the cost of the other 15%?

You will lose this deal against an OCR Provider because this deal is being fought over features and tech, and not business value

5

User Productivity Example

6

Große Supermarkt- kette in der Türkei

Pan European Wholesaler invoices/person/day Improvement Before Kofax 800 After 3 months of Automation 1200 50% After 2 weeks of „user productivity“ >2000 66% (150%!)

What are the goals of a KTM Project?

Every KTM project can be reduced to the following goal

Increase documents/person/hour.

Decrease clicks/document.

Can a user correct a problem faster than your complex solution?

The goal is not

Perfect OCR

Perfect UI

Be guided by simplicity, order, speed

Do not chase accuracy, chase docs/person/hour.

7

Anyone can do KTM

Classify Separate

Folder Extract Validate Learning

8

All you need is paper and highlighters

Classify Separate

Folder Extract Validate

9

Build a Benchmark

Add the Fields you need

Classify (F5)

Validate (F8)

Save Xdocs ( )

Train Xdocs (F10)

Train Project ( )

Tools/ExtractionBenchmark/AllClasses

Save Benchmark

Open in Microsoft Excel

10

Goals of every KTM Project

1. Human Productivity 2. Eliminate False Positives

bad data leaving Kofax 3. Reduce False Negatives

user pressing ENTER 4. Few True Negatives

OCR Accuracy, Database problems & learning

11

Benchmark Before

12

Benchmark During

13

Benchmark After

14

How to improve extraction quality beyond just OCR?

We probably have much of the information to be extracted in our dictionaries/databases already!

15

Database Matching

%

Slide 16

Kofax Clustering Utility

New Utility for Clustering Unknown Documents

What it does

Requirements

Step-by-step

Importing into KTM

18

What does the Kofax Clustering Utility do?

When configuring KTM content classification, the customer needs to provide samples for each class.

What KTM requires:

19


When configuring KTM content classification, the customer needs to provide samples for each class.

What customers usually provide:

20


presorting a document set into clusters of similar documents

User labels some of these clusters

Utility learns from labeling and pre-sorts again

Several iterations of labeling and pre-sorting

Export of sorted documents as learn-set for KTM content classification

21


new KTM project

Customer uses Utility to provide KPSG or partner with sorted documents

KPSG or partner uses Utility to sort documents from customer

Understanding what are the biggest subsets of documents in a customer’s monthly mailroom volume

enhancing a KTM project

Customer adds new classes to project and needs samples for classification

22

Requirements

Kofax Clustering Utility works with XDocuments

XDocuments must be created with KTM OCR Server tool

KTM (5.5 SP2) must be installed to use Clustering Utility.

23

Using the KTM OCR Server reduces the KTM base volume count

Eval licenses supported

Hardware requirements same as for KC/KTM

Files to be clustered should be local for performance

Need write access to file location

Requirements

24

Step by Step – KTM OCR Server

Configuring the KTM OCR Server:

Select path to unsorted images

Enable „Save XDoc files“ and „Save text files“

Under OCR Settings, select proper language

Leave rest at default

Running the KTM OCR Server:

Simply press the Start button

25

Step by Step – Kofax Clustering Utility

1. Import

Point „Import directory“ to same directory of unsorted documents

For each document, an .xdc file and a .txt file must exist

Select „Start Discovery“.

Takes a while, ~0.5 sec per document

Converts XDocs into internal format

Identifies initial clusters

26


2. Discovery

Label initial 3 clusters

You see the most representative document of each cluster

Provide a name for each cluster, will be used as class name in KTM

27


2. Discovery

28


2. Discovery

You can stop discovery when 80-90% of the documents are discovered or continue until all documents are discovered

At 80-90% the most common document types are often known, remaining documents are likely in very small clusters

Click „Review“ to continue to next step

29


3. Review

Sort by categories (labels)

Examine the categories for consistency

Confirm some documents if you want to cluster again

30


3. Review

31


4. Export

Select any directory for export

Sub directories will be created for each category/label

.txt files (and tif/xdoc for reference) will be exported, since only .txt files are used to train KTM content classification later

32

Importing into KTM

In Project Builder, point New Project dialog Content Classifier settings to exported directory

Select „Discovered documents“ sub directory

33

Importing into KTM

A class is created in Project Builder for each category

Training documents are imported

Select „Train“ in Project Builder main menu

Verify in Classification Benchmark (Result Matrix)

34

Importing into KTM

35

Setting this up manually and finding/organizing the proper training documents

takes hours or days.

With the Kofax Clustering Utility, this example took

20 minutes.

KTM for Forms

Kofax Transformation Modules vs Xtrata

Kofax Transformation for fixed-form and free-form documents,

Xtrata limited to fixed forms only.

KTM key applications: Automatic Indexing for Archive. Workflow (Mailroom) Automation. Forms Processing. Accounts Payable Automation. Automatic Document Separation. Records management.

Xtrata key applications forms processing only.

37

Kofax Transformation Modules vs Xtrata

Advantages of using KTM for forms processing applications

Ability to perform database matching to improve extraction rates - improves ROI.

More powerful and flexible validation interface (with Xtrata you have to use the KC validation module) - improves productivity.

More classification methods, i.e. layout and context vs layout only in Xtrata – improves classification accuracy (requires Full Base license)

Scripting for more advanced applications - improves flexibility.

38

KTM for Forms

Basic Information

http://www.kofax.com/downloads/datasheets/ds-kc10-license-update-en.pdf

Features

Layout-based classification

Unlimited extraction fields

Advanced Zone Locator

Barcode Locator

ABBYY FineReader OCR

Document Review (thick client)

Validation, Verification, Correction (thick and thin client)

39



Not supported in KTM for Forms

All locators not mentioned in the previous question

Content based classification

Any OCR besides Abbyy

Trainable Document Separation

40

KTM 5.5 – On Ramp

KTM 5.5 – On Ramp

Fax

Email

Export Connector

Scan

Business Processes

Kofax Transformation

Modules Native PDF

Support

Kofax Capture Scan

Original Format

Kofax Capture Import

Connector

Folder

Web service

42

KTM 5.5 – PDF

43

Supports PDF

Supports Color Supports Advanced Binarization for full

compatibility with all KTM functions

Advanced Binarization settings at project level

KTM 5.5 – PDF

Image layer is ignored!

Extracts “perfect” original PDF text layer.

No OCR required!

Productivity Enhancements – Design Time

KTM 5.5 – Benchmarking

Separation Benchmarking Classification Benchmarking Extraction Benchmarking

46

KTM 5.5 – Separation Benchmarking

Separation Benchmark

47

Document Separation Test



48


Golden Batch

Golden Files – Extraction Benchmarking


Separation Benchmark How can a Golden Batch be created?

Kofax Capture (before Export Connector)

KTM Project Builder

49



50

Separation Benchmark – Quality?


51

Correct Documents

Rejected Documents

Incorrect Documents

Incorrectly classified

Additional splits

Missed splits

But confidently

Document Review...?

The worst of all three categories

False Postive

KTM 5.5 – Classification Benchmarking

Classification Benchmark

52



53



54



55



56

Extraction Benchmark

KTM 5.5 – Extraction Benchmarking

Slide 57 57

Extraction Benchmark

KTM 5.5- Extraction Benchmarking

EV = Extracted Value GFV = Golden File Value (perfect file)

EV = GFV Work

EV ≠ GFV Work

EV ≠ GFV False positives

EV = GFV Super

Project quality Project design

Slide 58 58


Extraction Benchmark - Comparison

59


60

Extraction Benchmark - Enhancements Selection List

Sorting By Column Content By Status

Open in Document Viewer

Re-arrange columns

Productivity Enhancements - Conflict Management

KTM 5.5 – Conflict Management

Resolve conflicts...

62


Resolve Conflicts Dialogue

63

Toolbar Navigate between conflicts Synchronize Zoom Show All Fields



A Conflict Document Delete document Delete field File name Page navigation Conflicting field Field area on document


Progress / Summary

66


Delete field

67


Resolve conflict – Select the correct value with the mouse

68


Resolve Conflicts

69

Productivity Enhancements Project Merge Tool

Multiple Users – One Project

KTM 5.5 – Project Merge Tool

71


Project Master

72


Copy the Project Master for each aditional user

73


Project Master

74


Copy 1

75


Copy 2

76


Merge Copy 1

77


Source and Destination projects

78


Select Classes

79


Summary

80


Save changes to destination project (Project Master)

81


Merge Copy 2

82


Source and Destination projects

83


Select Classes

84


Summary

85


Save changes to destination project (Project Master)

86


Project Master after merging

87


Elements that can be merged...

88

Classes

Fields

Locators

Validation Rules

Script

Localization

Validation Forms


89


Elements

90


91


92


93


94


95


96


Summary

97


Save changes

98


The merged project

99

Productivity Enhancements xDoc Browser

KTM 5.5 – xDoc Browser

Design Time Enhancement

Shows more xDoc Structure

Representation • Boxes (used for Mixed Print recognition)

Annotations (Sticky Notes)

XValues

Helps the user to learn the KTM COM object

Useful assistance for scripting

101

KTM 5.5 – Project Builder – Test Documents

Additional Project Builder Enhancements

Open xDoc in Xdoc Browser from Project Builder

102



Show selected document(s) in Windows Explorer

103



Choose Details...

104


New look

105


106

Refresh

Document List

Previous / Next Document

Previous / Next Page

Zooming

Highlighting On/Off

Annotations On/Off


Refresh

107


Highlighting

108


Annotations

109


XValues

110


XValues 1. AC_BATCH_NAME - Batch Name (read-only) 2. AC_BATCH_CLASS_NAME - Batch Class Name (read-only) 3. AC_BATCH_PRIORITY - Priority (read/write) 4. AC_BATCH_DIRECTORY - ImageDirectory (read-only) 5. AC_BATCH_EXTERNAL_BATCHID - ExternalBatchID (read-only) 6. AC_BATCH_GUID - BatchGUID (read-only) 7. AC_BATCH_CREATIONDATETIME - BatchCreationDateTime (read-only) 8. AC_BATCH_CREATIONSITENAME - CreationSiteName (read-only) 9. AC_BATCH_CREATIONUSERID - CreationUserID (read-only) 10. AC_BATCH_OPERATORUSERID - OperatorUserID (user ID of last batch history entry) (read-only) 11. AC_BATCH_USERID - UserID (read-only) 12. AC_BATCH_USERNAME - UserName (read-only) 13. AC_BATCH_WINDOWSUSERNAME - WindowsUserName (read-only) 14. AC_FIELD - Kofax Capture document fields (read only) 15. AC_TABLE - Kofax Capture table fields (read only) 16. AC_FORMTYPE - Kofax Capture form type (read only) 17. AC_CSS - Kofax Capture Custom Storage String at document level 18. AC_CSS_PAGE<n> - Kofax Capture Custom Storage String at page level 19. AC_REJECTED_DOCUMENT - Indicates if the document has been rejected in Kofax Capture 20. AC_REJECTED_DOCUMENT_NOTE - The rejection note 21. AC_REJECTED_PAGE<n> - Rejected page 22. AC_REJECTED_PAGE_NOTE<n> - Rejected page note

111


Copy Text to Clipboard

112

pXDoc.Locators.ItemByName("LineItems").Alternatives(0).Table

Productivity Enhancements Localisation

KTM 5.5 – Localisation

KTM Languages

English

German

114


Additional KTM Languages

# Language Pack Language ID 1 Brazilian pt-BR 2 Chinese zh-CN 3 Czech cs 4 French fr 5 Italian it 6 Japanese ja 7 Polish pl 8 Russian ru 9 Spanish es

10 Swedish sv-SE

Additional KTM Languages Graphic User Interface

Project Builder and runtime modules

Component based messages KTM Server

Documentation (runtime modules and Userguide.pdf)

1. Document Review 2. Correction 3. Validation 4. Verification


116


Project Settings - Localization

117


Project Settings - Localization

118


.Net concept

Primary language

Secondary language

119

English en

English (United Kingdom) en-UK English (United Stated) en-US


Fall back principle

120


Fall back principle

121

Localise

End

Primary – Secondary language translation?

Yes

No

Primary language translation?

Use default value for display name Use translation value for display name

Yes

No


KTM GUI, Server and Active Language

122


KTM GUI Language, Server and Active Language

123

The Project.ActiveLanguage overrides the Region and Language settings


Summary KTM Graphic User Interface language KTM Server language Project language (Project.ActiveLanguage)

124

- -

-


What can be localised?

125

KTM Element Yes/No Note Fields Table Columns Formatting Methods Component messages used Validation Methods Regular Expression only

Component messages used Validation Form Tab captions

Field label Simple label Button captions DB button captions Group captions Script Resources


Fields

126


Tables

127


Project – Script Resources

128


Project – Script Resources

129

Project.Resources.GetString("Error_Example")

KTM project folder structure Default language in *.fpr file Additional languages


130

Document Review

Default language

Localised languages


Localisation.xml

External editor Language ID Example: Field

• Default value • Localised translation

131


XML Update


Project design language

133

Productivity Enhancements Thin Client

New and Improved Functionality Inside KTM TC 5.5

Validation Form Layouts

Annotations

Additional Batch Editing Operations

User Settings

Advanced Login Capabilities

Combo-boxes With Descriptions

Combo-boxes Inside Tables

Other “Small” Things

KTM TC 5.5 Improvements

135

Support Validation Form Layouts

Different font types and sizes

Mini-viewers

Custom buttons

Location of fields

Anchoring

Layout localization


136

Support Annotations

Display annotations created by KTM modules

Create new annotations inside Thin Client

Edit annotations

Delete annotations

Move annotations

Hide/Display annotations


137

Additional Batch Editing Operations

Delete pages Move, merge, delete documents Move, merge, delete, split, create folders


138

Preserve User Settings

User name at login screen Batch Open dialog box: size, columns, sorting settings Panels: size, expanded states Zoom settings: fit width, fit height, custom zoom Annotation settings: hide/display annotations


139

Advanced Login Capabilities

Domain login for linked users Single sign-on support for Active Directory users


140

Combo-boxes Inside Tables, Items With Descriptions

Display descriptions, values or both Support empty strings consistently for all combo-boxes Paging control for over 100 items Type-ahead filtering capabilities New script events to initialize scripted combo-boxes


141

Other “Small” Things…

Batch loading performance improvements (project caching) PDF support Reject/Unreject documents – support scripting on the server Allow to install Thin Client Server on top of previous version Propagate user changes in config files to a new version


142

Productivity Enhancements - Advanced Routing

KTM 5.5 – Advanced Routing

Batch routing was new in KTM 5.0 with KC 9.0 Kofax Catpture Service Packs allow more functionality: KTM 5.0

Batch Routing (routing of documents) is available in KC 9 Batch Routing (routing of folders) is available with KC 9 SP1

KTM 5.5 Assigning a new batch class to the child batches. KC 9 SP2 required

144

KTM 5.5 – Advanced Routing

Setting an XValue assigns a new batch class to a child batch:

KTM_DOCUMENTROUTING_NEWBATCHCLASS_<PlaceHolder>

Technology Enhancements - TDS / Project Planner

KTM 5.5 – TDS Enhancements

KTM AFC – Documents

KTM 5.5 AFC - Pages

147

1st

Middle

Last

• AFC or SVM

• TDS Separation Algorythm unchanged

• Re-use training sets

• Re-build model

SVM AFC

KTM 5.5 – TDS Enhancements

SVM vs AFC

SVM

AFC

Similar accuracey, but the AFC produces fewer missed splits

AFC allows for more frequent benchmarking

100 doc. types

100,000 pages

30,000 docs

Training Set

148

Technology Enhancements – Line Item Matching

KTM 5.5 – Line Item Matching Locator

Use cases for new features:

1. Multi PO discovery

2. Online Learning

3. Release Matching information to ERP

4. Getting more data

150

• Multi PO discovery


151


152 Slide 152

KTM Server

Marked for Learning

KTM 5.0

Validation clerk KTM KB Learning Server

Learned

KTM Server

Marked for Learning

KTM 5.5

Validation clerk KTM KB Learning Server

Learned Marked for Learning


Match Remarks Information about Under-/Over delivery, Ambiguous matches,

etc. are now stored in new global column for Match Remarks

153


154

Additional columns Table Locator can be used to find additional columns on the

invoice (e.g. Supplier Article code)

LIM Loc as input to Table Locator

Table Header pack for column detection


155

Additional columns

Additional Database columns (e.g. Cost Center ID) can be copied to the XDoc Table

Technology Enhancements Kofax Search and Matching Server (KSMS)

KTM 5.5 – Search and Matching Server

Business Value New in KTM 5.5 Faster client startup time – Instant feedback

Client Server instead of local copy (No Loading Delay – No Local Memory Usage)

Access large enterprise DBs Unlimited DB Size due to 64 bit support (50 Mio Records Tested)

Fast response time Multithreaded design with full support of multi core architecture

Industry standard connectivity MS SQL, Oracle, ODBC and CSV Low Maintenance Automatic DB Update Scheduler in

background

157

KTM Validation / KTM Server KSMS

Administrator

158

Technical Background

Technical Background

Instant access, no loading time Automatic update Direct access to databases Made for 64 bit systems and big databases Load balancing available Multiple KSMS Server

Security Active Directory support Secure communication

Administration through KTM remote or KTM local client possible Separate installer

159

Kofax Search and Matching Server

160

Enterprise Customer DB

0,00

5,00

10,00

15,00

20,00

25,00

30,00

35,00

40,00

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Sear

ch O

pera

tions

/ se

cond

KSMS Search Speed - 1 million records

Server (8 cores + HT)

Server (24 Cores - no HT)

Technology Enhancements – Mixed Print Detection

KTM 5.5 – Mixed Print Detection

Use cases 1. Mixture of OCR and ICR 2. Signatures 3. Read only with OCR or ICR 4. Classification

162


Recognition Profile - MixedPrint

163

OCR recognition profile – full page only

ICR recognition profile – full page or zonal

Threshold for ICR

Mixed OCR and ICR OCR or ICR


XDoc – Representation - Boxes

164

• LongTag = 1 – ICR • LongTag = 2 – Signature • Boxes are not created for OCR


XDoc – XValues

165

• PercentageHP_PageN: X% Can be used for classification


Example Project

166

This and That

KTM 5.5 – This and That

Drop Down Boxes in Tables Cells Drop Down Boxes Description | Value

RecoStar 5 Normalization Format Locator Enhancements Locator Dialogue & Testing Sticky Notes (Annotations) Docking and Zooming Rotation Script

168

KTM 5.5 – This and That: Dropdown Boxes in Table Cells

Validation Form Designer Validation Form Same script events as for normal combo boxes

169

KTM 5.5 – This and That: Recostar 5

Country and Language

170


Dictionaries

171


Zonal

172

KTM 5.5 – This and That: Normalization

Batch and Document structures Memory or on disk?

173

KTM 5.5 – This and That: Format Locator Enhancements

Use and Sorting

174

KTM 5.5 – This and That: Locator Dialogue & Testing

175

KTM 5.5 – This and That: Sticky Notes (Annotations)

176

KTM 5.5 – This and That: Sticky Notes (Annotations) Script events Application_AnnotationCreated Application_AnnotationSaved

177

KTM 5.5 – This and That: Docking and Zooming Allow user to change the view - True/False Docking

178

KTM 5.5 – This and That: Docking and Zooming

179

Top

Fields

Bottom

Right Left

Allow user to change the view - True/False Docking The zoom value is stored seperately for [Left/Right] and [Top/Bottom]

KTM 5.5 – This and That: Docking and Zooming Fit to Width

180

KTM 5.5 – This and That: Docking and Zooming

181

KTM 5.5 – This and That: Rotation

Use case: User rights

182

KTM 5.5 – This and That: Rotation

Project script: Document_XDocPageRotated

183

KTM 5.5 – This and That: Scripting

Class Script ValidationForm_ButtonDialogClosed ValidationForm_AfterViewerLassoDrawn

184

3.2 kofax partner connect 2013 - transformation modules - advanced track and what is new in ktm

Documents

ktm ocr server tool

step ktm ocr server

ktm overview

ramp ktm

ktm project1

step kofax clustering

successful ktm project

new ktm project customer