Human Auditory Cognition

Adriano Claro Monteiro, Alain de Cheveign, Anahita Mehta, Byron Galbraith, Dimitra Emmanouilidou, Deniz Erdogmus, James OSullivan,

Mehmet Ozdas, Lakshmi Krishnan, Michael Crosse, Nima Mesgarani, Jose L Pepe Contreras-Vidal, Thusitha Chandrapala

Organizers: Shihab Shamma (Univ. of Maryland), Malcolm Slaney (Microsoft),Barbara Shinn-Cunningham (Boston University),

Edmund Lalor (Trinity College, Dublin)

Imagined Sounds• We hear sounds

• Cocktail Party Effect

• We imagine and expect sounds

EEG Research• Others• Muscles…• Simple stimuli (beeps, etc)• Simple responses (ERPs)

• Us• Natural speech• Not imagined movement• Imagined speech!!!

One basket, one big egg

Decoding• Record EEG signals

• Three Approaches:• Mick/Nima

• Covariance of channels/SVM• Malcolm

• DCT of EEG/Nearest Neighbor• James

• Decode audio envelope/Correlate

Scientific Results

Perc

enta

ge C

orre

ct

Demo• Ask question• Imagine answer• Decode/Classify• Display result

What would the Queen not like to see?

Quiet please• Applause

What's up in the sky?

Quiet please• Applause

Andromeda galaxy is formed by billions of ...

Quiet please• Applause

Over the Thames river you can find...

Quiet please• Applause

What is the Capital of England?

Quiet please• Applause

Why should you use sunscreen?

Quiet please• Applause

What was Hitler's dream?

Quiet please• Applause

What can be seen in the night sky?

Quiet please• Applause

What would cause gridlock in a major city?

Quiet please• Applause

What melody did Mozart compose as a child?

Quiet please• Applause

What is the airspeed velocity of an unladen swallow?

Quiet please• Applause

Thank you

What is the most shining thing you've ever seen?

What is the main source of energy for our planet?

When I look up into the sky I see...

What would an Irishman like to see?