speechless speech libian_camerafinalproject
DESCRIPTION
TRANSCRIPT
Speechless Speech
MIT Media Lab Li Bian | www.chakuli.com
Hey Mary, a bunch of us are going to Legal Seafood tonight.
Would you like to come along?
Jeremy and David are coming too.
We are thinking of leaving at around 9PM.
Audio Visual Speech Recognition
------- a technique that uses image processing capabilities in lip reading to aid speech recognition systems in recognizing undeterministic phones or giving preponderance among near probability decisions.
IBM (ViaVoice)
Affective Computing Group, Media Lab (facial feature -> emocons)
DARPA (Advanced Speech Encoding thru tuned electromagnetic
resonator)
Haar classifier Flash light + Luminance Equ. Compression
Y = 0.2126 R + 0.7152 G + 0.0722 B Luminance (relative):
Haar classifier_frontal face
Haar classifier_mouth
Lossy Compression
BMW car
BMW car
Original transcript:
Hello, hi, my name is Mary, and I am calling from BMW regarding your
car washing services.
Could you please come to the factory to pick up your car this Saturday
afternoon?
Thank you.
BMW car
Person 1 (male, native English speaker)
calling about…card
Person 2 (male, native English speaker)
Hello my name is Mary, calling from BMW, if you can give me a call… this factory..
Person 3 (male, non-native English speaker)
pick up...
BMW car
Person 1 (male, native English speaker)
Hello my name is Mary, I am calling about....factory...pick up the car
Person 2 (male, native English speaker)
Hello my name is Mary, I am calling from BMW...pick up your car Saturday morning...
Person 3 (male, non-native English speaker)
I am doing...to pick up...
BMW car
Person 1 (male, native English speaker)
Hello Mary
Person 2 (male, native English speaker) Mary...calling from BMW...
Person 3 (male, non-native English speaker)
I am ..calling from BMW...call me back....
Dinner Call
Dinner Call
Original transcript:
Hi Lizzie, would you like to go for dinner tonight at 7PM? um, can we
meet at Kendall Square?
Alright, give me a call back if you get this message.
Thank you.
Dinner Call
Person 1 (male, native English speaker)
...Please call back
Person 2 (male, native English speaker)
...call back..
Person 3 (male, non-native English speaker)
...
Dinner Call
Person 1 (male, native English speaker)
Hi, would you like to go...?...Please call back, thanks
Person 2 (male, native English speaker)
give me a call back...
Person 3 (male, non-native English speaker)
Let's meet at...call me back for...
Dinner Call
Person 1 (male, native English speaker)
Hi, would you like to go...?7PM...Please call back. Thank you, bye
Person 2 (male, native English speaker)
calling to ask you about dinner, at 7PM...thank you, bye
Person 3 (male, non-native English speaker)
...Let's meet at...call me soon...
Homework Call
Homework Call
Original transcript:
Hi John, um just calling to see if you have finished the homework # 4
that's due tonight.
If you are still working on it, can we work on it together later tonight?
Alright, give me a call back if you get this message.
Thank you. Bye.
Homework Call
Person 1 (male, native English speaker)
...bye
Person 2 (male, native English speaker)
...bye
Person 3 (male, non-native English speaker)
...
Homework Call
Person 1 (male, native English speaker)
...bye
Person 2 (male, native English speaker)
...thank you, bye
Person 3 (male, non-native English speaker)
Hi…I am from...phone number...bye
Homework Call
Person 1 (male, native English speaker)
Hello John...work on....thank you, bye
Person 2 (male, native English speaker)
Hello John....please give me a call when you get this message...ok, bye
Person 3 (male, non-native English speaker)
Hello John...text me, bye
Legal Seafood
Legal Seafood
Original transcript:
Hey Mary, a bunch of us are going to Legal Seafood tonight.
Would you like to come along? Jeremy and David are coming too.
We are thinking of leaving at around 9PM.
Legal Seafood
Person 1 (male, native English speaker)
...
Person 2 (male, native English speaker)
Mary...terminal
Person 3 (male, non-native English speaker)
Larry....I...
Legal Seafood
Person 1 (male, native English speaker)
...Mary....I'm going to...would you like to come, too?
Person 2 (male, native English speaker)
Mary...coming too...around 9PM...
Person 3 (male, non-native English speaker)
..come along...come too
Legal Seafood
Person 1 (male, native English speaker)
...Would you like to come, too?
Person 2 (male, native English speaker)
we are meeting to go to Legal Seafood.....is coming too, would you like
to come along?
Person 3 (male, non-native English speaker)
...Would you mind to come along...
Visual Audio enhance
Visual Audio
Speech Recognition Computer Vision
enhance
train
Visual Audio enhance
Speech Recognition Computer Vision train
Facial features Text Messages translate
Thank you!