cesto - home | ipkey · 5.7.* 11g development ... apache http server 2 jee6 full-profile (ejb 3.1)...

CESTO IP Key LA workshop at INDECOPI, Lima, Peru

Martin Beckman, Emmanuel Collin, Antonio Grau, Arjeton Uzairi

November 2018

Project overview

• Features and functionalities

• Architecture

• CESTO

• OSA 2 – Search algorithm

Development

• Framework, Servers and Tools

• Technology stack

• CESTO/OSA 2 – Data Design Model

• ESPEAK & LAME

• CESTO UI, WS, Batch Process & Persistance

Deployment

• CESTO – Deployment models

Agenda

Overview

Tool enabling automated searches to provide

support on Absolute / Relative Grounds to

examiners by presenting all the relevant information

and knowledge dispersed in several databases on

one screen.

CESTO – Purpose

CESTO = Common Examiner Support TOol

CESTO – Main features

• Free search

• Pre-generated reports for examiners

• Configuration per Office

• Detailed TM comparison

• Statistics generation

• Traffic light configuration

• Customize data sources

• Report management

• Administration functionalities

CESTO – Data sources and updates

OHIM SEARCH ALGORITHM (OSA)

TRADEMARKS

FTP Server

PLANT VARIETIES

Web Service

INTERNATIONAL NON-PROPRIETARY

NAMES (INN)

Web Service

GI

CESTO

6ter PARIS

ZIP files

EXTERNAL DATABASES

NATIONAL DATA LIST (NDL)

CESTO

INTERNAL DATABASES

CPVO WHO

ONLINE SEARCH

MANUALLY

AUTOMATICALLY MANUALLY

DAILY UPDATED MONTHLY UPDATED MANUALLY

WIPO TMview DOOR

E-BACCHUS E-SPIRIT

NATIONAL OFFICES

Searches in CESTO

CPVO INN WHO

GI WIPO NATIONAL OFFICES

TM Database

CESTO – Architecture

CESTO – General functional view

CESTO OSA2

TM

Database View/Request

CESTO Reports

Web UI

View/Request

CESTO Reports

Web Services

Configure Reports

Generate Stats

Generate Stats

Free Search /

View Reports

Trade Marks

periodically uploaded

to TM DB by POs

OSA Trade Marks

connection to TM

DB

OSA 2 indexes

External

Repositories

National Data List

GI uptodaded to OSA

rep by POs

OSA 2 – OHIM Search Algorithm 2

OSA 2 is the underlying search algorithm behind

CESTO, assessing the similarity between two

expressions.

• Literal element similarity: finding part/modifications of the searched

expressions

• Phonetics similarity: potential similar results based on pronunciation

• Conceptual similarity: it is capable of searching for “symbols” (5,$, etc)

both ways

SIMILAR IPR

REF. DENOMINATION

NATIONAL OFFICES

NICE CLASS

LANGUAGES

1 Normalization Distinctive

words Word keys

2 Language Similarity

General Similarity

Fuzzy queries


1 Normalization Distinctive

words Word keys

2 Language Similarity

General Similarity

Fuzzy queries

CAPITALISE

RID OF DIACRITICS

WORD SPLITTING

OCCURENCES

WITH DIFFERENT

OWNER + RULES

MINIMUM

SIMILARITY (CUTTING POINT DISTANCE)

ALPHABETS LATIN, GREEK, CYRILLIC

LITERAL SIGNS + TRANSLITERATION

PHONETIC 23 LANGUAGES

WORD DISTANCE AND

DISTINCTIVENESS

WEIGHT VALUE BASED ON LANGUAGE

SIMILARITY

POSSIBLE

MATCHING TERMS

WITH A MAXIMUM

EDIT DISTANCE


OSA 2 – Calculating Minimum Similarity % for each Word Key

1. lengthDeflactor = 2*numberOfWordsOfMark 2. partialDF = (1/(1-wordDistintiveness/2)-1)/lengthDeflactor)

3. cuttingPointDistance = 4*partialDF*log2(wordLength) 4. luceneCuttingPoint = 10 - round(cuttingPointDistance)

These exceptional cases are dealt at this point:

If wordLength is 3 or 4 and luceneCuttingPoint greater than 7 then the

luceneCuttingPoint is set to 6 If ipaTranscription length of the word is greater than 10 and

luceneCuttingPoint is less than 8 then the luceneCuttingPoint is set to 8

5. minimumSimilarity % = luceneCuttingPoint * 10

• INCLUDED o CESTO Source Code (Standalone version)

• CESTO

• OSA2

o Tools (eSpeak, Lame)

o Development, Deployment & Server configuration

• NOT INCLUDED o Data Sources: Trademarks, GI, WIPO 6ter

o External Web Services: Plant Variaties (CPVO) and INN WHO

o Bilateral Agreements for Web Services: CPVO, WHO

CESTO – Package

Development

3.1.1 1.6 & 1.8

4 1.1

2.0 3.5

1.7.1

Development – Frameworks

8.2

7.0.x

2.2.3

5.7.*

11g

Development – Servers

3.0.4

Oxygen

1.46

3.99

Development – Tools

Presentation

Tier

Service

Tier

Information

Tier

System Product Specification / Framework

Web Server Apache HTTP Server

jQuery 1,7,1

Bootstrap

Backbone

Servlet Container Apache Tomcat 7 Spring: MVC, Security

Java Servlet / JSP

Application Server

Apache Tomcat 7

Wildfly 8.2

Apache HTTP Server 2

JEE6 Full-Profile (EJB 3.1)

Database MySQL 5.7

Oracle 11g

JPA 2.0

Hibernate 4

Search Engine Apache Lucene Solr 3.5

Development – Technology stack

Architecture diagram

Development – CESTO Data Model Design

Development – OSA2 Data Model Design

Development – Database

• Open source speech synthesizer for English and other languages

• Available for Linux and Windows

• Options:

o -v: Use voice file from espeak-data/voices (ie. “-ven”)

o -q: quiet

o --ipa: Write phonemes to stdout using International Phonetic Alphabet

• Bash script running in HTTPD server in Unix

• PHP script runing PHP server in Windows

ESPEAK – http://espeak.sourceforge.net/

Bash script PHP script

Web result JSON

ESPEAK – http://espeak.sourceforge.net/

Web result JSON

LAME – http://lame.sourceforge.net/

• Open source enterprise search platform (JAVA)

• Advanced Full-Text Search capabilities

• Optimize for High Volume Traffic

• Standard Based interface (XML, JSON & HTTP)

• Highly Scalable & Fault Tolerant

• Real-Time Indexing

• Multiple search indices

SOLR – http://lucene.apache.org/solr/

• Indexing

• Querying

• Mapping

• Ranking the outcome


• Solr Indexation from TM-Database

• Solr Indexation from NDL (National Data List)

• Solr Indexation from GI (Geographical Indications)

• Solr Indexation from 6ter Paris

name: The name for the field

type: Mandatory - the name of a previously defined type from

indexed: True if this field should be indexed (searchable or sortable)

stored: True if this field should be retrievable

multiValued: True if this field may contain multiple values per document

omitNorms: (Expert) set to true to omit the norms associated with this


NAME INDEX STORED TYPE MULTIVALUED

TERM VECTORS

OMIT NORMS

ApplicationNumber TRUE TRUE int FALSE FALSE FALSE

GoodsServicesClassNumber TRUE TRUE string TRUE FALSE FALSE

MarkCurrentStatusCode TRUE TRUE string FALSE FALSE TRUE

RegistrationOfficeCode TRUE TRUE string FALSE FALSE TRUE

MarkVerbalElementText TRUE TRUE entity_names FALSE TRUE TRUE

SortedMarkVerbalElementText TRUE TRUE string FALSE FALSE FALSE

CTMApplicantName FALSE FALSE entity_names TRUE FALSE FALSE

ApplicantName TRUE FALSE entity_names TRUE FALSE TRUE

ApplicantNameString TRUE TRUE contains_string TRUE FALSE TRUE

ApplicationDate TRUE TRUE tdate FALSE FALSE FALSE

MarkFeature TRUE TRUE string FALSE FALSE FALSE

MarkImageURI TRUE TRUE string FALSE FALSE FALSE

ST13 TRUE TRUE string FALSE FALSE FALSE

TradeMarkURI TRUE TRUE ignored FALSE FALSE FALSE

ViennaCode TRUE TRUE string TRUE FALSE FALSE

CTMRepresentativeIdentifier FALSE FALSE string TRUE FALSE FALSE

CTMRepresentativeName FALSE FALSE entity_names TRUE FALSE FALSE

IRRepresentativeURI FALSE FALSE string TRUE FALSE FALSE

IRRepresentativeIdentifier FALSE FALSE string TRUE FALSE FALSE

IRRepresentativeName FALSE FALSE entity_names TRUE FALSE FALSE

RepresentativeIdentifier TRUE TRUE string TRUE FALSE FALSE

RepresentativeName TRUE TRUE entity_names TRUE FALSE FALSE

RepresentativeNameString TRUE TRUE string TRUE FALSE FALSE

IpaDoublese FALSE TRUE string TRUE FALSE TRUE

IpaDoubleen FALSE TRUE string TRUE FALSE TRUE

IpaDoublees FALSE TRUE string TRUE FALSE TRUE

IpaDoublede FALSE TRUE string TRUE FALSE TRUE

IpaDoublefr FALSE TRUE string TRUE FALSE TRUE

IpaDoublebg FALSE TRUE string TRUE FALSE TRUE

IpaDoubleel FALSE TRUE string TRUE FALSE TRUE

IpaDoublefi FALSE TRUE string TRUE FALSE TRUE

IpaDoublehu FALSE TRUE string TRUE FALSE TRUE

IpaDoublelt FALSE TRUE string TRUE FALSE TRUE

IpaDoublelv FALSE TRUE string TRUE FALSE TRUE

IpaDoublenl FALSE TRUE string TRUE FALSE TRUE

IpaDoublepl FALSE TRUE string TRUE FALSE TRUE

IpaDoublept FALSE TRUE string TRUE FALSE TRUE

IpaDoublero FALSE TRUE string TRUE FALSE TRUE

IpaDoublesk FALSE TRUE string TRUE FALSE TRUE

IpaDoublesv FALSE TRUE string TRUE FALSE TRUE

IpaDoubleit FALSE TRUE string TRUE FALSE TRUE

IpaDoublecs FALSE TRUE string TRUE FALSE TRUE

IpaDoubleda FALSE TRUE string TRUE FALSE TRUE

IpaDoubleet FALSE TRUE string TRUE FALSE TRUE

SOLR – Trademark index

NAME INDEX STORED TYPE MULTIVALUED

TERM VECTORS

OMIT NORMS

uniqueid TRUE TRUE string FALSE FALSE FALSE

office TRUE TRUE string FALSE FALSE FALSE

idAgreement TRUE TRUE string FALSE FALSE FALSE

shortDescription TRUE TRUE string FALSE FALSE FALSE

Denomination TRUE TRUE entity_names FALSE TRUE TRUE

ApplicationNumber TRUE TRUE int FALSE FALSE FALSE

GoodsServicesClassNumber TRUE TRUE string TRUE FALSE FALSE

MarkCurrentStatusCode TRUE TRUE string FALSE FALSE TRUE

RegistrationOfficeCode TRUE TRUE string FALSE FALSE TRUE

MarkVerbalElementText TRUE TRUE entity_names FALSE TRUE TRUE

SortedMarkVerbalElementText TRUE TRUE string FALSE FALSE FALSE

CTMApplicantName FALSE FALSE entity_names TRUE FALSE FALSE

ApplicantName TRUE FALSE entity_names TRUE FALSE TRUE

ApplicantNameString TRUE TRUE contains_string TRUE FALSE TRUE

ApplicationDate TRUE TRUE tdate FALSE FALSE FALSE

MarkFeature TRUE TRUE string FALSE FALSE FALSE

MarkImageURI TRUE TRUE string FALSE FALSE FALSE

ST13 TRUE TRUE string FALSE FALSE FALSE

TradeMarkURI TRUE TRUE ignored FALSE FALSE FALSE

ViennaCode TRUE TRUE string TRUE FALSE FALSE

IpaDoublese FALSE TRUE string TRUE FALSE TRUE

IpaDoubleen FALSE TRUE string TRUE FALSE TRUE

IpaDoublees FALSE TRUE string TRUE FALSE TRUE

IpaDoublede FALSE TRUE string TRUE FALSE TRUE

IpaDoublefr FALSE TRUE string TRUE FALSE TRUE

IpaDoublebg FALSE TRUE string TRUE FALSE TRUE

IpaDoubleel FALSE TRUE string TRUE FALSE TRUE

IpaDoublefi FALSE TRUE string TRUE FALSE TRUE

IpaDoublehu FALSE TRUE string TRUE FALSE TRUE

IpaDoublelt FALSE TRUE string TRUE FALSE TRUE

IpaDoublelv FALSE TRUE string TRUE FALSE TRUE

IpaDoublenl FALSE TRUE string TRUE FALSE TRUE

IpaDoublepl FALSE TRUE string TRUE FALSE TRUE

IpaDoublept FALSE TRUE string TRUE FALSE TRUE

IpaDoublero FALSE TRUE string TRUE FALSE TRUE

IpaDoublesk FALSE TRUE string TRUE FALSE TRUE

IpaDoublesv FALSE TRUE string TRUE FALSE TRUE

IpaDoubleit FALSE TRUE string TRUE FALSE TRUE

IpaDoublecs FALSE TRUE string TRUE FALSE TRUE

IpaDoubleda FALSE TRUE string TRUE FALSE TRUE

IpaDoubleet FALSE TRUE string TRUE FALSE TRUE

IpaDoublesl FALSE TRUE string TRUE FALSE TRUE

IpaDoublemt FALSE TRUE string TRUE FALSE TRUE

IpaDoubletr FALSE TRUE string TRUE FALSE TRUE

SOLR – GI index

NAME INDEX STORED TYPE MULTIVALUED TERM VECTORS

OMIT NORMS

ndluniqueid TRUE TRUE string FALSE FALSE FALSE

idList TRUE TRUE string FALSE FALSE FALSE

office TRUE TRUE string FALSE FALSE FALSE

listDescription FALS

E TRUE string FALSE FALSE FALSE

listShortDescription FALS


isListPublic TRUE TRUE boolean FALSE FALSE FALSE

idTerm FALS


term TRUE TRUE string FALSE FALSE FALSE

comment FALS


termDescription FALS


SOLR – National Data List index

OSA – DS Maintenance

1.- CESTO UI 2.- CESTO WebService

https://www.tmdn.org/cesto-ui

3.- CESTO

Batch Process 4.- CESTO

Persistence

CESTO

• Spring Batch framework

• Daily Schedule (out office hours)

• Generate and storage Reports for TM

o New trademarks from OSA

o Past failed reports

• Purging the old reports

• Use of CESTO Web Services

CESTO – BATCH

• Manage the CESTO database through Hibernate.

o Relational data persistence is managed with JPA 2.0.

• CESTO persistence uses a mechanism to store XML files

on disk.

o Document persistence is managed with JCR 2.0.

CESTO – PERSISTANCE

• Internal interfaces will be Java interfaces

• WSDL definitions (v1.1 for SOAP, v2.0 for REST)

• JAXB 2.0

• JPA 2.0

• JCR 2.0

• SPRING 3 MVC REST

• TOMCAT

Business logic to perform the different tasks:

-Administration functionalities

-CESTO Report generation, storage and retrieval

-Search for and retrieve CESTO reports

-Statistics generation

CESTO – WEB SERVICE

Operations

Operation: Search for and view a CESTO report

-URL: http://<server>/cesto-webservice/report/search/<idTm>

-Http Method: GET


INN International Non-propietary Names

WHO World Health Organization

CPVO Community Plant Variety Office

https://extranet.who.int/tools/inn_global_data_hub https://cpvoextranet.cpvo.europa.eu

ftp://ftpird.wipo.int/wipo/6ter/

WIPO 6ter Paris

BILATERAL AGREEMENT

• DB ORGANIZATION

• PROPERTY OFFICE

CESTO – External Web Services & WIPO 6ter FTP

https://extranet.who.int/tools/inn_global_data_hub

https://extranet.who.int/tools/inn_global_data_hub

https://cpvoextranet.cpvo.europa.eu/

https://cpvoextranet.cpvo.europa.eu/

ftp://ftpird.wipo.int/wipo/6ter

1.- CESTO UI 2.- CESTO WebService


3.- CESTO

Batch Process 4.- CESTO

Persistence

CESTO

Web-based user interface:

• Spring Web MVC frameworks

Roles:

• Office administrator CESTO report / traffic light configuration

• Authenticated and non-authenticated user Search: Free search

• Authenticated PO user Search: Can perform search for, view and

re-generation of a CESTO report

• Administrator Statistics generation

CESTO – UI

CESTO – UI: Login

CESTO – UI: Internationalization

~650 terms

~2000 words

CESTO – UI: Internationalization


CESTO

Deployment

Deployment – CESTO Infrastructure model

Deployment models vary from the office needs and requirements of

the Office:

1. Standalone or Portable deployment

• Single server/VM/PC with 16GB RAM, 100GB HDD

2. Distributed

• Every component of CESTO to be installed in separate server/VM with

8GB RAM, 100GB HDD

3. Clustered

• Every component of CESTO to be installed in separate server/VM with

16GB RAM, 100 HDD, in addition to have a proper replication

configuration of the most critical components of CESTO such as OSA2

and SOLR indexes with master/slave configurations. Load balancer will

be required to balance the number of request.

*Jmeter – tool to test the performance of the deployment model

Deployment – CESTO Infrastructure model

Deployment – OSA2 Infrastructure model

Presentation Tier

Service Tier

Information Tier

Servlet ContainersNode #2


Application ServerNode #2


Web ServerNode #2

Web ServerNode #2

DMZ

ZONE 1

ZONE 2

Internet

Web ServerNode #1

Web ServerNode #1





Database ServerDatabase Server

Server #3 and #4CPU: 2RAM: 4 GBDISK: 50 GBOS: CentOS 64bitSW: Tomcat

Server #5 and #6CPU: 2RAM: 4 GBDISK: 50 GBOS: CentOS 64bitSW: JBoss AS

Server #7CPU: 2RAM: 4 GBDISK: 50 GBOS: CentOS 64bitSW: JBoss AS

Server #1 and #2CPU: 2RAM: 4 GBDISK: 50 GBOS: CentOS 64bitSW: JBoss AS

Load Balancer


Servlet ContainersServlet Containers


Application ServerApplication Server

Web ServerWeb Server

DMZ

ZONE 1

ZONE 2

Presentation Tier

Service Tier

Information Tier

Internet

Server #1CPU: 2RAM: 4 GBDISK: 50 GBOS: CentOS 64bitSW: Tomcat

Server #2CPU: 2RAM: 4 GBDISK: 50 GBOS: CentOS 64bitSW: JBoss AS

DEVELOPMENTProduction Development

Deployment – OSA2 Infrastructure model

Type Specification Usage Qty

File Server

RAM = 2 GB, Disk = 60 GB Reports storage 1

Web Server RAM = 4 GB, Disk = 40 GB CESTO UI /

CESTO batch

1

Web Server RAM = 4 GB, Disk = 40 GB CESTO web services /

CESTO batch

1

Database Server RAM = 2 GB, CPU = 2x2GHz, Disk = 50 GB CESTO data base

1

Type Specification Usage Qty

Solr Master RAM = 4 GB,

Disk =100 GB

Solr Indexation from TM-Database

Solr Indexation from NDL DS

Solr Indexation from GI DS

Solr Indexation from 6ter DS

1

Solr Slaves RAM = 4 GB,

Disk =100 GB

Replication of Solr Master indexes 1..N

OSA Servers RAM = 4 GB,

Disk =50 GB

OSA Search (TM, NDL, GI , 6ter) 1..N

OSA-MAINTENANCE-GUI RAM = 2 GB,

Disk =25 GB

GUI OF OSA-MAINTENANCE OF NDL, GI, and 6ter 1

Deployment – Infrastructure model

Thank You

cesto - home | ipkey · 5.7.* 11g development ... apache http server 2 jee6 full-profile (ejb 3.1)...

Documents