applying machine learning - abdessamad echihabi at sdl connect 16
Post on 27-Jan-2017
54 Views
Preview:
TRANSCRIPT
Applying Machine Learning
11/17/2016
Samad Echihabi
2
Machine Learning
Machine
data
model
input output
3
Machine Learning
Machine
data
model
input output
4
Machine Learning: French to English Translation
data
model
Méditerranée: 3200 personnes secourues en cinq jours
Mediterranean: 3200 people rescued in five days
5
Machine Translation Models
Language Model
Translation Model
Source Target Good Translations?bonjour hellobonjour bluebonjour morningbonjour good morningbonjour hi
Source Target Good Translations?bonjour hello ✔bonjour blue ✗bonjour morning ~ bonjour good morning ✔bonjour hi ✔
MT: Translation Model
Target Good Language?Be the change that you wish to see in the world.Be the world that you wish to see in the change.The be change which you wish to see on the world. Be that the you world to in wish see change . the
Target Good Language?Be the change that you wish to see in the world. ✔
Be the world that you wish to see in the change. ✗The be change which you wish to see on the world. ✗✗Be that the you world to in wish see change . the ✗✗✗✗✗
MT: Language Model
MT: Training
Statistical Analysis
Translation Model
la the 80%
la a 12%
la 8%
capitale capital 70%
capitale death 30%
de of 53%
de from 47%
france france 100%
Is est 75%
Is was 25%
paris paris 100%
Language Model
the death of 54%
the capital of 34%
a capital of 11%
capital of france 41%
capital from france 9%
of france is 45%
of the france 2%
france is paris 23%
france was paris 22%
………………english………
P(s/t) P(t)
parallelmonolingual
………………english………
………………french………
Statistical Analysis
MT: Decoding
Statistical Search
Translation Score
the capital of france is paris 94%
capital of france is paris 71%
a capital of france is paris 65%
... …
a death from france was paris 3%
Translation Model
la the 80%
la a 12%
la 8%
capitale capital 70%
capitale death 30%
de of 53%
de from 47%
france france 100%
Is est 75%
Is was 25%
paris paris 100%
Language Model
the death of 54%
the capital of 34%
a capital of 11%
capital of france 41%
capital from france 9%
of france is 45%
of the france 2%
france is paris 23%
france was paris 22%
Input
la capitale de la france est paris
SMT Models
Adaptive ModelsNeural Models
• Translation Model P(s/t)• Language Model P(t)• Distortion• Alignment• Phrase• POS• Syntactic Translation• Syntactic Language• Reordering• Lexicalized Reordering• Preordering• Word Deletion• Lexicalized Smoothing• Capitalization• Morphology• Transliteration• Semantic• Informal Models• Social Media Components
Applying Machine Learning – Use cases
Social Media Translation
Character Repetition
Spelling Errors
Dialect
Morphology
Romanization
Metadata
Social Media Translation Challengesأااااااحسن احسنالخ�لييييييج� لخليجا Normalization
نزيفه
نظيفةوظيفةنظيفنزيفنزيفه
Spelling Correction
# القدم_ كرة #soccerSocial Metadata
المرفهين Morphologicalين + مرفه + ال
Segmentation
bessa7a
wel3afya
habibi والعافيةحبيبي بالصحةhabibi Deromanizationبساحة
+62% Improvement
Source Generic MT
Social Media Translation
• la2a hia katir fi lakhbar. • ma 3ajbanish kida. Lazim t3'iyyer
l3ounouane • Enty habla ? • Kalemni lama t3raf ezay tebatal
teshtemni• 3andy soda3 fi rassi... 5oshy namy badal
chat. a7san lik Ah sa7
• La2a hia katir Fi lakhbar. • Ma 3ajbanish kida. lazim T3 (iyyer
L3ounouane • enty habla? • kalemni Lama T3RAF ezay tebatal
teshtemni • 3Andy soda3 Fi rassi ... 5oshy namy
badal Chat. A7San lik Ah SA7
Source Social Media MT
Social Media Translation
• la2a hia katir fi lakhbar. • ma 3ajbanish kida. Lazim t3'iyyer
l3ounouane • Enty habla ? • Kalemni lama t3raf ezay tebatal
teshtemni• 3andy soda3 fi rassi... 5oshy namy badal
chat. a7san lik Ah sa7
• No, it is very much in the news. • I don't like this. We must change the
title• Are you an idiot? • Talk to me when you know how to stop
insulting me• I have a headache in my head. Go to
sleep, instead of chat. It is better for you, Yes, sa7
Broadcast NewsTranslations
Broadcast News Translation
Speech Recognition
Machine Translation Distillation
Audio Channels
Video Channels
Actionable Information
Received Tuesday in Warsaw by Bronislaw Komorowski, Barack Obama has participated in ceremonies marking the twenty-fifth anniversary of the first democratic elections in Poland
Broadcast News Translation
Reçu mardi à Varsovie par Bronislaw Komorowski, Barack Obama a participé aux cérémonies marquant le vingt-cinquième anniversaire des premières élections démocratiques en Pologne.
Received Tuesday in Warsaw by Bronislaw Komorowski, Barack Obama has participated in ceremonies marking the twenty-fifth anniversary of the first democratic elections in Poland
✗
Travel Reviews
Travel User Reviews Translation
published
Translated User Reviews
post-edited
good translationbad translation
Automatic Quality Prediction
Post-Editing Machine Translation
Post-Editing Machine Translation
Post-Editing Adaptive Machine Translation
Post-Editing Adaptive Machine Translation
Applying Machine Learning
Volume
Quality
Data Domain
Models
Delivery
Security Speed
Privacy
Evaluation
Integration
Adaptation
ANSWERS&
QUESTIONS
Copyright © 2008-2017 SDL plc. All rights reserved. All company names, brand names, trademarks,
service marks, images and logos are the property of their respective owners.
This presentation and its content are SDL confidential unless otherwise specified, and may not be
copied, used or distributed except as authorised by SDL.
Software and Services for Human Understanding
top related