Human Auditory Cognition

Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,

Mehmet Ozdas, Lakshmi Krishnan, Michael Crosse, Nima Mesgarani, Jose L Pepe Contreras-Vidal, Thusitha Chandrapala

Organizers: Shihab Shamma (Univ. of Maryland), Malcolm Slaney (Microsoft),Barbara Shinn-Cunningham (Boston University),

Edmund Lalor (Trinity College, Dublin)

Imagined Sounds

• We hear sounds

• Cocktail Party Effect

• We imagine and expect sounds

EEG Research

• Others• Muscles…• Simple stimuli (beeps, etc)• Simple responses (ERPs)

• Us• Natural speech• Not imagined movement• Imagined speech!!!

One basket, one big egg

Decoding

• Record EEG signals

• Three Approaches:• Mick/Nima

• Covariance of channels/SVM• Malcolm

• DCT of EEG/Nearest Neighbor• James

• Decode audio envelope/Correlate

Scientific Results

Perc

enta

ge C

orre

ct

Demo

• Ask question• Imagine answer• Decode/Classify• Display result

What would the Queen not like to see?

Quiet please

• Applause

What's up in the sky?

Quiet please

• Applause

Andromeda galaxy is formed by billions of ...

Quiet please

• Applause

Over the Thames river you can find...

Quiet please

• Applause

What is the Capital of England?

Quiet please

• Applause

Why should you use sunscreen?

Quiet please

• Applause

What was Hitler's dream?

Quiet please

• Applause

What can be seen in the night sky?

Quiet please

• Applause

What would cause gridlock in a major city?

Quiet please

• Applause

What melody did Mozart compose as a child?

Quiet please

• Applause

What is the airspeed velocity of an unladen swallow?

Quiet please

• Applause

Thank you

What is the most shining thing you've ever seen?

What is the main source of energy for our planet?

When I look up into the sky I see...

What would an Irishman like to see?