automatic transcription of video files sig media
TRANSCRIPT
![Page 1: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/1.jpg)
Automatic transcription of video
filesCarlos Turró
Universitat Politecnica de Valencia
![Page 2: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/2.jpg)
Agenda• Why automatic transcription• State of the art: The transLectures project• Automatic transcription of Lecture Recordings: The Opencast Project• Notes & the near future
![Page 3: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/3.jpg)
Why automatic transcription of video files?• Accessibility
![Page 4: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/4.jpg)
Why automatic transcription of video files?• Accessibility
• Searching into a video file• Searching into a video repository• Topic identification• …and much more
![Page 5: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/5.jpg)
The transLectures project• Development of an engine for Automated Speech Recognition (ASR) for
lectures & educational content• Development of translation tools for that content
• Implementation• Case studies: Videolectures.NET & Polimedia (UPV video repository)• Real-life evaluation• Integration into Opencast
http://www.translectures.eu
5
![Page 6: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/6.jpg)
transLectures partners
12 Nov 2013
Name Country
1 Universitat Politècnica de València (MLLP) Spain2 Xerox SAS France3 Institut Jožef Stefan Slovenia3+ Knowledge for All Foundation UK4 RWTH Aachen University Germany5 EML – European Media Laboratory Germany6 DDS – Deluxe Digital Studios UK
36 Months
November 2014
![Page 7: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/7.jpg)
Statistical transcription (and translation)
Acustic Model
LanguageModel
TRANSCRIPTION
Sound ASR Engine
![Page 8: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/8.jpg)
Statistical transcription (and translation)
Acustic Model
LanguageModel
Manually transcriptedvoice Modeling Engine
![Page 9: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/9.jpg)
Architecture of TransLecturesLecture
Language Model
Slides
Extracontent
Result
Intelligent interaction
Transcription Translation
![Page 10: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/10.jpg)
Languages
12 Nov 2013 10
• Transcription (ASR)• EN • SL • ES
• Translation (MT)• EN>SL , SL>EN• EN>ES , ES>EN• EN>FR• EN>DE
![Page 11: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/11.jpg)
Transcription and Translation Platform
![Page 12: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/12.jpg)
Transcription and Translation Platform API
![Page 13: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/13.jpg)
Transcription and Translation Platform• Post-editing web interface (in HTML5)
![Page 14: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/14.jpg)
Example video• https://media.upv.es/?id=b444d12e-db23-9a4f-9b3b-d1d9275d4cb4
![Page 15: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/15.jpg)
Scientifical Evaluations• WER = Word Error Ratio
• The lower the better
• Usually, a human transcriptor has a WER around 12
![Page 16: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/16.jpg)
Beyond transLectures
![Page 17: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/17.jpg)
Beyond transLecturesWER
Language M10 M17Dutch 25.7 24.5Italian 21.2 17.7Portuguese 45.9 43.0Spanish 15.9 14.4Estonian N/A 27.1French N/A 22.7
![Page 18: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/18.jpg)
Beyond transLectures
![Page 19: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/19.jpg)
The Opencast Community is…Universities, companies and people:• concerned with academic video• attracted to the Opencast values of openly exchanging ideas,
experience, knowledge and code• committed to building and maintaining a robust, flexible, high-quality
open source lecture capture and academic video management solution.
Now also part of
![Page 20: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/20.jpg)
Full-featured Lecture Recording ecosystem
![Page 21: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/21.jpg)
Who uses Opencast?Around the world, with strong adoption in Europe especially.
43 Adopters with public information (May 2014)
30+ commercial partner clients
http://opencast.org/matterhorn-adopters
![Page 22: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/22.jpg)
Yesterday’s tweet
![Page 23: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/23.jpg)
Indexing in Opencast• Opencast has built-in OCR indexing capabilities
Video (slides) -> OCR (hunspell) -> Word list filter -> Apache Lucene search server
• New operations can be addedVideo (slides) -> transcription (tL) -> Apache Lucene search serverorVideo (slides) -> OCR (hunspell) -> transcription (tL) -> Word list filter ->Apache Lucene search server
![Page 24: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/24.jpg)
Why do I need an indexing server?• Powerful, Accurate and Efficient Search Algorithms
• ranked searching -- best results returned first• many powerful query types: phrase queries, wildcard queries, proximity
queries, range queries and more• fielded searching (e.g. title, author, contents)• sorting by any field• multiple-index searching with merged results• allows simultaneous update and searching• flexible faceting, highlighting, joins and result grouping• fast, memory-efficient and typo-tolerant suggesters
![Page 26: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/26.jpg)
Notes & the near future• ASR Technology is enough good for automated transcription of videos
… with enough good sound
• There are lecture recording systems that enables to plug transcriptions for searching
…like Opencast
• There are already things to solve• Transcription speed (in good progress)• Topic indentification• Adding more languages
![Page 27: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/27.jpg)
Thanks!Questions?
![Page 28: Automatic transcription of video files sig media](https://reader036.vdocument.in/reader036/viewer/2022081521/58eda3b71a28ab73158b471f/html5/thumbnails/28.jpg)
Learning more ….transLectureshttp://translectures.eu
Video in a multilingual context (EMMA)http://association.media-and-learning.eu/portal/resource/ml-webinar-video-multilingual-context
Opencast State of the Projecthttp://lanyrd.com/2015/apereo/sdmpry/