kantanfest: dimitar shterionov - part 2

KantanNeural™ from A to Z3/3: NMT in 4 weeks → 4 days → 4 hours

Dimitar Shterionov

What is NMT?

31/07/2017 KantanFest, Dublin, Ireland 2

What is NMT?


x1 x2 x3 c y1 y2 y3

How to NMT – The Recipe

Hardware + Software: GPUs, torch, Theano

nematus, OpenNMT

Know-how, Support

Integration,Deployment

Training data


How to NMT – KantanNeural™

Hardware + Software: GPUs, torch, theano

nematus, OpenNMT

Know-how, Support

Integration,Deployment

Training data

KantanNeural™


KantanNeural™: black board to production

Proof of Concept:

AWS, NVIDIA K520 GPUs

Nematus, ADAM, BPE, SCN

MT (engines) build: 4 weeks

Quality: impressive

01 Nov 2016


• ADAM: Parameter update algorithm

• Byte-pair encoding (BPE)• Single-character n-gram (SCN)

lower → low ertallest → tall estalmost → al most

lowesttallerallow

KantanNeural™ α:

OpenNMT, ADAM, BPE

ΜΤ build time: 4 days

Quality: on a par with nematus

KantanFleet™

01 Nov 2016 01 Feb 2017



KantanNeural™ β:

Build-your-own NMT

Available to all clients (no extra charge)

Extended KantanFleet™

01 Nov 2016 01 Feb 2017 15 March 2017



01 Nov 2016 01 Feb 2017 15 March 2017

Currently:

Build-your-own NMT

NVIDIA K80 GPUs

AdaptiveMT

Incremental Retraining

4 hours?

30 June 2017



KantanMT.com – A Complete Platform

Build

Improve

Deploy

Kantan

Templates

Kantan

NER

Kantan

Llibrary

Kantan

Fleet

Kantan

BuildAnalytics

Kantan

Analytics

Kantan

PEX

Kantan

LQR

Adaptive

MT

Kantan

GENTRY

Kantan

TotalRecall

KantanNeural™Kantan

Translate

Kantan

Swift

Kantan

API

Kantan

AutoScale

Kantan

OfficeMT

Kantan

Connectors

Kantan

Snippets

KantanNeural™



Build Improve Deploy




Select a KantanFleet™ engine

KantanFleet™ Neural (18 language

pairs)

Multiple domains

Create new NMT engine

Import library data

Import your own data

Convert an SMT profile:

… just two clicks away from NMT




Select a KantanFleet™ engine




Create a blank KantanNeural™

engine




Convert a PBSMT engine into KantanNeural™

engine





Artificial Neural Networks train iteratively:

While stopping condition not met:

While training data not exhausted:

Take a batch

Learn from it

Repeat



Augment data

Parallel corporaPreprocessing rules

(PEX, tokeniser excep., etc.)

F-Measure, BLEU, TERKantanLQR(Error typology, AB Testing)

New Preprocessing rulesNew data

Augment data Augment data Augment data




Augment data









Augment data







4 hours?



API

Connectors

KantanWidgets™

As every other KantanMT engine


Conclusions…

KantanMT:

A complete MT platform for both NMT and PBSMT engines

Easy access to powerful MT technology

How to train, improve and deploy KantanNeural™ engines

Seamless switch from PBSMT to NMT

Incremental retraining to improve, adapt and specialize engines

Conclusions…

KantanMT:

A complete MT platform for both NMT and PBSMT engines

Easy access to powerful MT technology

How to train, improve and deploy KantanNeural™ engines

Seamless switch from PBSMT to NMT

Incremental retraining to improve, adapt and specialize engines

4 hours training?

… and future work

Better control:

Terminology

Tags

NTAs

Learn from postedits:

Exploit feedback from KantanLQR™

Exploit feedback from connectors

Models:

Add language knowledge

Hybrid MT

Convolutional Neural Networks (CNN)

…

Solving

Thank you…

Laura Casanellas: [email protected] Shterionov: [email protected]

KantanLabs: [email protected]

KantanMT: [email protected]

kantanfest: dimitar shterionov - part 2

Technology