data-driven machine translation for sign languages
DESCRIPTION
Data-Driven Machine Translation for Sign Languages. Sara Morrissey PhD topic NCLT/CNGL Workshop 23 rd July 2008. outline. background main problems data-driven MT for SLs experiments and results conclusions. background. communication interpreters and technological aids - PowerPoint PPT PresentationTRANSCRIPT
Data-Driven Machine Translation for Sign Languages
Sara MorrisseyPhD topic
NCLT/CNGL Workshop23rd July 2008
outline background
main problems
data-driven MT for SLs
experiments and results
conclusions
background communication interpreters and technological aids machine translation
– automatic and confidential– native language of users
rule-based approaches (Veale et al., 1998, Marshall & Sáfár, 2002)
data-driven approaches – Bauer et al., 1999, Stein et al., 2006, Wu et al.,
2007
main problems representation
no formally adopted writing system linguistic analysis
little research appropriate data
difficult to find evaluation
visual-spatial nature rules out automatic
data-driven MT for SLs initial prototype system using Dutch SL MaTrEx system Air Traffic Information System (ATIS)
Corpus 595 English sentences multi-lingual – ISL parallel corpus
creation manual annotation with semantic
glosses
data representation
(Early morning flights between Cork and Belfast)
EARLY MORNING BETWEEN be-CORK CORK FLY BELFAST BETWEEN ref-BELFAST ref-CORK
MATREX: data-driven machine translation
English ISL
bilingual database
translation directions
SL Recognition SL Generation
SL Annotation
Spoken Language Text
experiments and results
machine translation experiments 2 segmentation methodologies
type 1 chunks uses Marker Hypothesis (Green, 1979)
type 2 uses dual segmentation method
1. Early morning flights between Cork and Belfast
2. <ADJ> early morning flights <PREP> between Cork <CONJ> and Belfast
experiments and results
System BLEU WER PER
EN—ISL
ISL—EN
Baseline
+ T1 chunks
+ T2 chunks
Baseline
+ T1 chunks
+ T2 chunks
38.85
39.11
39.05
52.18
51.31
50.32
46.02
45.90
46.02
38.48
37.39
39.56
34.33
34.20
34.21
29.67
30.63
32.32
animation real human signing preferred (Naqvi, 2007)
but impractical avatar animation criteria: realistic, consistent, functional,
fluid Poser Animation Software Version 6.0 50 randomly selected sentences, 66
hand-crafted videos problem of fluidity
animation
‘or’
‘e’
how much
flight
http://www.computing.dcu.ie/~smorri/ISL_AnimationDemo.html
human evaluation experiments 4 native Deaf human monitors web-based evaluation of 50 ISL
translations evaluated intelligibility and fidelity 82% animations = intelligible 72% animations = good-excellent
translations HCI analysis using Nielsen’s approach
experiments and results
conclusion MT methodology never before applied to SLs multi-component system, bidirectional
system practical, technological alternative to help
alleviate communication and comprehension for Deaf community
positive automatic and manual evaluation scope for incorporating different SL
representation methodologies and segmentation techniques