unconscious memory representations in perception

288

Upload: pillow-poe

Post on 24-Oct-2015

115 views

Category:

Documents


0 download

DESCRIPTION

Editors: István Czigler and István Winkler

TRANSCRIPT

Page 1: Unconscious Memory Representations in Perception
Page 2: Unconscious Memory Representations in Perception

Unconscious Memory Representations in Perception

Page 3: Unconscious Memory Representations in Perception

Volume 78

Unconscious Memory Representations in Perception. Processes and mechanisms in the brainEdited by István Czigler and István Winkler

Advances in Consciousness Research (AiCR)Provides a forum for scholars from different scientific disciplines and fields of knowledge who study consciousness in its multifaceted aspects. Thus the Series includes (but is not limited to) the various areas of cognitive science, including cognitive psychology, brain science, philosophy and linguistics. The orientation of the series is toward developing new interdisciplinary and integrative approaches for the investigation, description and theory of consciousness, as well as the practical consequences of this research for the individual in society. From 1999 the Series consists of two subseries that cover the most important types of contributions to consciousness studies: Series A: Theory and Method. Contributions to the development of theory and method in the study of consciousness; Series B: Research in Progress. Experimental, descriptive and clinical research in consciousness.This book is a contribution to Series B.

EditorMaxim I. StamenovBulgarian Academy of Sciences

Editorial Board David J. ChalmersAustralian National University

Axel CleeremansUniversité Libre de Bruxelles

Gordon G. GlobusUniversity of California Irvine

Christof KochCalifornia Institute of Technology

Stephen M. KosslynHarvard University

Steven LaureysUniversity of Liège, Belgium

George MandlerUniversity of California at San Diego

John R. SearleUniversity of California at Berkeley

Petra StoerigUniversität Düsseldorf

Page 4: Unconscious Memory Representations in Perception

Unconscious Memory Representations in PerceptionProcesses and mechanisms in the brain

Edited by

István CziglerInstitute for Psychology, Hungarian Academy of Sciences/Debrecen University, Hungary

István WinklerInstitute for Psychology, Hungarian Academy of Sciences/ University of Szeged, Hungary

John Benjamins Publishing Company

Amsterdam / Philadelphia

Page 5: Unconscious Memory Representations in Perception

Library of Congress Cataloging-in-Publication Data

Unconscious memory representations in perception : processes and mechanisms in the brain / edited by István Czigler, István Winkler.

p. cm. (Advances in Consciousness Research, issn 1381-589X ; v. 78)Includes bibliographical references and index.1. Implicit memory. I. Czigler, István. II. Winkler, István. BF378.I55U53 2010153.1’3--dc22 2009053346isbn 978 90 272 5214 2 (Hb ; alk. paper)isbn 978 90 272 8835 6 (Eb)

© 2010 – John Benjamins B.V.No part of this book may be reproduced in any form, by print, photoprint, microfilm, or any other means, without written permission from the publisher.

John Benjamins Publishing Co. · P.O. Box 36224 · 1020 me Amsterdam · The NetherlandsJohn Benjamins North America · P.O. Box 27519 · Philadelphia pa 19118-0519 · usa

The paper used in this publication meets the minimum requirements of American National Standard for Information Sciences – Permanence of Paper for Printed Library Materials, ansi z39.48-1984.

8 TM

Page 6: Unconscious Memory Representations in Perception

Table of contents

Contributors� viiPreface� ix

1.� Conscious an�� unconscious as�ects of �or�in�� �e�or��Conscious�an���unconscious�as�ects�of��or�in����e�or��� 1Amanda L. Gilchrist and Nelson Cowan

2.� Mar�ers�of�a�areness?�EEG��otentials�evo�e���b���faint��� an����as�e���events,��ith�s�ecial�reference�to�the�“attentional�blin�”� 37

Rolf Verleger

3.� In�search�for�au��itor���object�re�resentations�� 71István Winkler

4.� Re�resentation�of�re��ularities�in�visual�sti�ulation:�Event-relate����� �otentials�reveal�the�auto�atic�acquisition� 107

István Czigler

5.� Au��itor���learnin���in�the���evelo�in���brain� 133Minna Huotilainen and Tuomas Teinonen

6.� Neuroco��utational��o��els�of��erce�tual�or��anization� 147Susan L. Denham, Salvador Dura-Bernal, Martin Coath

and Emili Balaguer-Ballester

7.� Are���ou�listenin��?�Lan��ua��e�outsi��e�the�focus�of�attention� 179Yury Shtyrov and Friedemann Pulvermüller

8.� Unconscious��e�or���re�resentations�un��erl��in����usic-s��ntactic��� �rocessin���an����rocessin���of�au��itor���o����balls�� 209

Stefan Koelsch

Page 7: Unconscious Memory Representations in Perception

vi� Unconscious�Me�or���Re�resentations�in�Perce�tion�

9.� On�the��s��cho�h��siolo�����of�aesthetics:�Auto�atic�an���controlle����� �rocesses�of�aesthetic�a��reciation� 245

Thomas Jacobsen

A��en��ix:�Usin���electro�h��siolo�����to�stu�����unconscious��e�or����re�resentations� 259

Alexandra Bendixen

In��ex� 273

Page 8: Unconscious Memory Representations in Perception

Alexan��ra�Ben��ixenInstitute�for�Ps��cholo�����IUniversit���of�Lei�zi��Seebur��str.�14-20D-04103�Lei�zi��,�Ger�an��alexan��ra.ben��ixen@uni-lei�zi��.��e

A�an��a�L.�Gilchrist�an���Nelson�Co�an�De�art�ent�of�Ps��cholo��ical�Sciences18�McAlester�HallUniversit���of�MissouriColu�bia,�MO�65211�USACo�anN@�issouri.e��u

István�Czi��lerInstitute�for�Ps��cholo����Hun��arian�Aca��e����of�Sciences1394�Bu��a�est,�P.�O.�Box�398,�Hun��ar��czi��ler@co���s���h��.hu

Susan�L.�Denha�Centre�for�Theoretical�an���Co��utational�NeuroscienceUniversit���of�Pl���outhDra�e�Circus�Pl���outh�PL4�8AA,�UKs��enha�@�l���outh.ac.u�

Minna�Huotilainen�an���Tuo�as�TeinonenUniversit���of�Helsin�iP.O.�Box�9�(Siltavuoren�en��er�20�D)FIN-00014�Universit���of�Helsin�i�inna.huotilainen@helsin�i.fi

Tho�as�JacobsenEx�eri�ental�Ps��cholo�����UnitFacult���of�Hu�anities�an���Social�SciencesHel�ut�Sch�i��t�Universit���Universit���of�the�Fe��eral�Ar�e����Forces�Ha�bur��Holstenhof�e���8522043�Ha�bur��,�Ger�anJacobsen@hsu-hh.��e

Stefan�KoelschDe�art�ent�of�Ps��cholo����Pevense���Buil��in��Universit���of�SussexBri��hton,�BN1�9QH,�UK�oelsch@cbs.����.��e

Yur���Sht��rov�an���Frie��e�ann�Pulver�üllerMe��ical�Research�Council,�Co��nition�an���Brain�Sciences�Unit15�Chaucer�Roa��Ca�bri����e,�CB2�7EF,�UK��ur��.sht��rov@�rc-cbu.ca�.ac.u�

Rolf�Verle��erDe�art�ent�of�Neurolo����Universit���of�Lübec�23538�Lübec�,�Ger�an��rolf.verle��[email protected]�.��e

István�Win�lerInstitute�for�Ps��cholo����Hun��arian�Aca��e����of�SciencesH-1394�Bu��a�est,�P.�O.�Box�398,�Hun��ar��i�in�ler@co���s���h��.hu

Contributors

Page 9: Unconscious Memory Representations in Perception
Page 10: Unconscious Memory Representations in Perception

Preface

The�stu�����of��rocesses�un��erl��in���conscious�ex�erience�is�a�tra��itional�to�ic�of��hiloso�h��,��s��cholo����,�an���neuroscience.�In�contrast,�until�recentl��,�the�role�of�i��licit�(non-conscious)��e�or���s��ste�s�nee��e���to�establish�veri��ical��erce�-tion�receive����uch�less���iscussion.�With��an���ne��results�an���novel�theoretical��o��els�e�er��in���in�recent���ears,�it�is�ti�e�to�ta�e�a�loo��at�the�state�of�art�in��s��-cholo�����an���neuroscience�about�the�role�non-conscious��rocesses�an����e�or���re�resentations��la���in��erce�tion.

Althou��h�behavioral�stu��ies�(e.��.��ri�in��)��in�co��nitive��s��cholo�������iel��e���in��irect�evi��ence�about��an���of�the�characteristics�of�i��licit��e�or���re�resen-tations,�co��nitive�neuroscience�an���co��utational��o��elin���can��rovi��e��ore���irect� insi��hts� into� the�structure�an���contents�of� these�re�resentations�an���the�into�various��rocesses�relate���to�the�.�This�boo���resents�several���ifferent�a�-�roaches�of�the�stu�����of�i��licit��e�or���re�resentations,�usin���both��s��cholo��i-cal,�neuroscience�an���co��utational��o��elin����etho��s,�assessin��� these�re�re-sentations�in�relativel���si��le�situations,�such�as��erceivin�����iscrete�au��itor���an���visual�sti�ulus�events,�as��ell�as�in�hi��h-level�co��nitive�functions,�such�as�s�eech�an����usic��erce�tion�an���aesthetic�ex�erience.

Each�cha�ter� revie�s�a���ifferent� to�ic,� its� theoretical�an����ajor�e��irical�issues.�A�lar��e��art�of�the�results�revie�e���in�this�boo���ere�obtaine���throu��h�recor��in���event-relate���brain��otentials�(ERP).�This�is�because�ERPs�offer�hi��h�te��oral�resolution,�an���as�a�consequence,�the���are�sensitive�in��icators�of�the�����na�ics� of� hu�an� infor�ation� �rocessin��.� Non-ex�ert� rea��ers� are� offere����ith�an�a��en��ix� to�hel�� to�un��erstan���an���assess� the�ERP���ata� revie�e��� in�the�various�cha�ters.�Ho�ever,�ERPs��rovi��e�less��recise�infor�ation�re��ar��in���the�sources�of�the�observe���neural�activit���in�the�brain.�Therefore,�an�increas-in��� a�ount� of� research� utilizes� �ore� recent� brain� i�a��in��� �etho��s,� such� as�functional��a��netic�resonance�i�a��in���(fMRI),��hose�s�atial�resolution�excee���that� of� the� ERPs.� Althou��h� the� ex�eri�ental� �ara��i���s� use��� in� brain� i�a��-in���are�often�not���et�as�hi��hl���s�ecific�as�those��or�e���out�throu��h�the���ears�in�behavioral�or�ERP�research,� so�e�of� these� stu��ies�alrea�����brou��ht� si��nificant�ne��insi��hts�into�the��or�in���of�i��licit��e�ories.�Co��on�to�all�cha�ters�is�

Page 11: Unconscious Memory Representations in Perception

�� Preface

the�a��roach�that�e��irical�evi��ence�is�evaluate���in�ter�s�of�its�i��ortance�to��o��els�of�the��rocesses�un��erl��in���conscious��erce�tion.

In�the�first�cha�ter,�Gilchrist and Cowan�revie��the�role�of��or�in����e�or���in�conscious�as��ell�as�in�non-conscious�co��nitive��rocessin��.�Verleger’s�cha�ter ��iscusses�ERP�co��onents� relate��� to�subjective�ex�erience�an���non-conscious��rocesses.�Winkler su����ests�that��re��ictive�re�resentations�of�acoustic�re��ulari-ties� for�� the� core� of� au��itor��� �erce�tual� objects,� �hile� Denham, Dura-Berna, Coath and Balaguer-Ballester ��escribe�a�neuroco��uational�a��roach�to��o��el�the��rocesses�un��erl��in����erce�tual�objects.�The�role�of�sensor����e�or���in�au-to�atic�visual�chan��e���etection�an����erce�tion�is���iscusse���in�Czigler’s�cha�ter.�Huotilainen and Teinonen�focus�on�the�role�of�learnin���an���i��licit��e�or���in��erce�tual� ��evelo��ent.� Shtyrov and Pulvermüller’s� cha�ter� ��escribes� a� novel��ara��i����for�stu����in���the��e�or���re�resentations� involve��� in�s�eech��erce�-tion.�The���evaluate�the�relevant�theories�in�li��ht�of�their�results�an����rovi��e�ne��insi��hts�into�the�non-conscious��rocesses�un��erl��in���s�eech��erce�tion.�Koelsch �resents�an�a��vance���theor���of�the��erce�tion�of��usical�structure,�e��hasizin���the�role�of�non-conscious��rocesses�an���i��licit��e�or���re�resentations.�Finall��,�the� role� of� non-conscious� an��� conscious� �rocesses� in� aesthetic� a��reciation� is�ex�lore���in�Jacobsen’s cha�ter.�In�the�A��en��ix,�Bendixen��rovi��es�a�concise���e-scri�tion�of�the�ERP��etho��.

In�su��ar��,�this�boo���rovi��es�a�theoretical�an���e��irical�overvie��of�the�various�to�ics�of�an���a��roaches�to�the�question:�What�is�the�role�of�non-con-scious��e�or���re�resentations�in��erce�tion?

We�than��Maxi��Sta�enov�for�his�initiative�an���sti�ulatin���a��vice.�Zsuzsa���’Albini’s�technical�hel��is���reatfull���ac�no�le����e��.�Our��or���as�su��orte���b���the�Hun��arian�Research�Fun���(OTKA�71600)�an��� the�Euro�ean�Co��unit��’s�Seventh�Fra�e�or��Pro��ra��e�(��rant�n°�231168�–�SCANDLE).

� István�Czi��ler�an���István�Win�ler� Bu��a�est,�Nove�ber�2009

Page 12: Unconscious Memory Representations in Perception

chapter�1

Conscious and unconscious aspects of working memory

A�an��a�L.�Gilchrist�an���Nelson�Co�anUniversit���of�Missouri,�Colu�bia,�USA

1.1 Working-memory models and consciousness

A�si��le�(an��,�alas,�oversi��lifie��)�histor���of��or�in����e�or����o��els�hel�s�to�len����ers�ective�to�the��resent�en��eavor.�Geor��e�Miller�(1956)�is���enerall���cre��-ite����ith��ic�in���off�a�renaissance�in�the�stu�����of�te��orar����e�or���relate���to�the�conscious��in��,�in�his��ublishe���s�eech�on�the�li�it�of�te��orar����e�or���to�“the��a��ical�nu�ber�seven��lus�or��inus�t�o”�(thou��h��an���researchers�faile���to�un��erstan���his�hu�orous�intent�in���escribin���a��a��ical�nu�ber�surroun��e���b���confi��ence�intervals).�Shortl���after�ar��,�Miller�et�al.�(1960)�coine���the�ter��working memory�to���escribe�the��e�or���for�current���oals�an���a�s�all�a�ount�of�infor�ation�that�allo�s�one’s�i��e��iate��lans�to�be�carrie���out.�Several��o��els�usin���so�ethin���li�e�this��or�in����e�or���conce�t��ill�no��be���escribe��.�

1.1.1 Infor�ation-flo���o��els

Broa��bent�(1958)��ro�ose���a�ver���si��le��o��el�of��e�or���that� too��into�ac-count�the�latest�evi��ence;�in�that��o��el,�a�lar��e�a�ount�of�sensor���infor�ation��as�enco��e���in�a�te��orar���buffer�(li�e��hat�is�no��calle����or�in����e�or��).�Ho�ever,�onl���a�s�all��ro�ortion�of�it�ever��a��e�its��a���into�the�li�ite��-ca�acit����art�of��e�or��,��here�it��as�anal��ze���an���cate��orize����ith�the�benefit�of�the�vast�a�ount�of�infor�ation�save���in�lon��-ter���e�or��,�an���eventuall���a����e���to�that�lon��-ter��ban��of��no�le����e.�At�inson�an���Shiffrin�(1968)�then���evelo�e���that�sort�of��e�or��� fra�e�or�� into�a��ore�ex�licit��o��el� in��hich��athe�atical�si�ulations��ere��a��e�of�the�transfer�of�infor�ation�fro��one�store�to�another�throu��h�enco��in��,�retrieval,�an���rehearsal��rocesses.�

Page 13: Unconscious Memory Representations in Perception

2� A�an��a�L.�Gilchrist�an���Nelson�Co�an

1.1.2 Multi-co��onent��o��els

Ba����ele��� an��� Hitch� (1974)� soon� foun��� that� this� si��le� treat�ent� of� �or�in����e�or����oul���not���o.�In�a����ition�to�infor�ation�te��oraril���hel���in�a�for��li�e-l���to�be�relate���to�consciousness�(Ja�es�1890),�the���ar��ue���in�favor�of�other�stor-a��e�faculties�that�o�erate���auto�aticall��,�outsi��e�of�voluntar���control.�Broa��bent�(1958)�alrea�����ha���conceive���of�sensor����e�or���that��a��,�but�sensor����e�or����as�not�enou��h.�There��as�sai���to�be�a��honolo��icall��-base���te��orar����e�or���store�that��as�vulnerable�to�interference�fro��a����itional�verbal�ite�s,�no��atter��hether�their�source��as�visual�or�au��itor��.�Si�ilarl��,�there��as�a�visuo-s�atial�te��orar����e�or���store�res�onsive�to�the�s�atial�la��out�an���visual�qualities�of�ite�s;�in�theor��,�this�store��oul���be�vulnerable�to�interference�fro��s�atial�infor-�ation�even�if�its�source��ere�au��itor��.�Ba����ele���(1986)�later�thou��ht�he�coul�����o��ithout�the�central�store�or��ri�ar����e�or���entirel��;��hereas�Ba����ele���(2000)�essentiall���restore���it�in�the�for��of�an�e�iso��ic�buffer.�

1.1.3 Activation-base����o��els

A� for�� of� �o��el� that� �ill� beco�e� the� center�iece� of� this� cha�ter� is� the� acti-vation-base����o��el.�In�that�sort�of��o��el�(An��erson�1983;�Co�an�1988;�Hebb�1949;�Nor�an�1968;�Treis�an�1964)�a�te��orar���for��of�stora��e�such�as��or�in����e�or���is�vie�e���not�as�a�se�arate�location�in�the��in���or�brain,�but�rather�as�a�state�of�activation�of�so�e�of�the�infor�ation�in�the�lon��-ter���e�or���s��ste�.�Hebb�(1949)�envisione���this�activate����e�or���as�a�cell�asse�bl���that�consiste���of�a�neural�firin����attern�occurrin���re�eate��l���so�lon���as�the�i��ea�it�re�resente����as�active.�Treis�an�(1964)�an���Nor�an�(1968)���istin��uishe���bet�een�subli�i-nal�a�ounts�of�activation,��hich��i��ht�influence�behavior�but�not�consciousl��,�an���su�rali�inal�a�ounts,��hich�attracte���attention�to�the�activate���ite�,�es-�eciall���if�it��as��ertinent�to�the��erson’s�current���oals�an���concerns.�An��erson�(1983)�use���the��rocess�of�activation�as�a��a���to�brin���lon��-ter���no�le����e�into��or�in����e�or��.�

The�instance�of�the�activation-base����o��els�that��e��ill���iscuss�in��ost���e-tail� is� the� one� that� Co�an� (1988,� 1999)� ��esi��ne��� as� a� �a��� to� ex�ress� �hat� he�believe���to�be��no�n�at�the�ti�e.�There��ere�a���ar���as�ects�of�the�box-base����o��els�of�the�ti�e,�such�as�the��ell-�no�n�ones�of�At�inson�an���Shiffrin�(1968)�an���Ba����ele���(1986).�These��o��els���i���not�rea��il���ex�ress�the�assu��tion�that�infor�ation�co-exists� in�several� for�s�at�once.�Sensor���infor�ation�can�be�ac-co��anie���b����ore�cate��orical�infor�ation�that�is���ra�n�fro��lon��-ter���e�-or���into��or�in����e�or��.�For�exa��le,�the�acoustic��ro�erties�of�a�s�eech�soun���elicit�the�a��ro�riate�soun���cate��or���that�is�store���in�lon��-ter���e�or��.�Other�

Page 14: Unconscious Memory Representations in Perception

� Conscious�an���unconscious�as�ects�of��or�in����e�or��� 3

sensor���infor�ation��ersists��ithout�invo�in���lon��-ter���e�or���infor�ation.�Co�an� (1988,�1999)� re�resente��� this� state�of� affairs��ith�e�be����e����rocesses.�Te��oraril��-activate���features�of��e�or���can�inclu��e�both�sensor���an���abstract�features.�So�e�of�these�features�beco�e�active�enou��h,�or�active�in�the�ri��ht��a��,�to�beco�e��art�of�the�in��ivi��ual’s�current�focus�of�attention,��hich�is�of�li�ite���ca�acit��.�The�focus�of�attention�is�controlle���in��art�voluntaril��,�an���in��art�invol-untaril���throu��h�the�brain’s�reactions�to�abru�t��h��sical�chan��es�in�the�environ-�ent�(such�as�a�su����en�chan��e�in�li��htin���or�a�lou���noise).�

In�the�ori��inal��o��el�of�Broa��bent�(1958),�unatten��e���sensor���infor�ation��as�filtere���out�so�that�it�fa��e���a�a����ithout�ever�reachin���the�ca�acit��-li�ite���store.�This��o��el�beca�e�untenable��hen�it��as�foun���that�as�ects�of�unatten��e���soun��s�can�attract�attention�(as��hen�one’s�o�n�na�e� is��resente��;� see�Mora���1959)� so� Treis�an� (1964)� �ro�ose��� that� the� filter� onl��� attenuate��,� rather� than�co��letel���bloc�in���out,�sensor���in�ut�that��as�unatten��e��.�Potentiall����ertinent�in�ut�coul���still�result� in�attention�bein���attracte���to�that� in�ut.�Co�an�(1988)�solve��� this� �roble�� in� another� �a��,� b��� in��icatin��� that� all� sensor��� in�ut� �a��e�contact��ith�lon��-ter���e�or��,�but��ith�extra�activation�for�the���eliberatel��-at-ten��e���in�ut�or�the�abru�tl��-chan��in���in�ut�that�attracts�attention�involuntaril��.�This�is�the�converse�of�the�filterin���conce�t.�

Co�an�(2001)� su����este��� that� the� focus�of�attention� in� the�e�be����e����ro-cess��o��el�of�Co�an�(1988,�1999)� is� li�ite���to�about�4�se�arate� ite�s�in�nor-�al�a��ults.�This�leaves�o�en�the�basis�of�in��ivi��ual���ifferences�in�ca�acit���li�its.�Co�an�an���collea��es�have�su����este���that�there�is�a�basic���ifference�bet�een�in-��ivi��uals�an���a��e���rou�s�in�the�size�of�the�focus�of�attention�(Chen�an���Co�an�2009a;�Co�an�2001;�Co�an�et�al.�2005,�2009;�Gilchrist�et�al.�2008,�2009).�An�al-ternative�basis�of�in��ivi��ual���ifferences�an���a��e���ifferences,�still�in��ee�in����ith�the��o��el,�is�that�in��ivi��uals�a��ear�to���iffer�in�the�abilit���to�control�or�allocate�attention�or�executive�function�(En��le�et�al.�1999;�Mi��a�e�et�al.�2001)�or,��ore�s�ecificall��,�to�inhibit�irrelevant�ite�s�an����revent�the��fro��clutterin���u��the�li�ite��-ca�acit���store�(Lusti���et�al.�2007).�These��ossibilities�are�not��utuall���exclusive�an���both�of�the��are�co��atible��ith�the�basic�e�be����e����rocesses��o��el.�(In��ee��,�for�a�structural�equation��o��el�si�ilar�to�En��le�et�al.�or�Mi��a�e�et�al.,�see�Co�an�et�al.�2005.)

Co�an’s�e�be����e����rocess��o��el�assu�es�that�stora��e�an����rocessin���both�use�attention,�an���that�certain��rocessin���challen��es��ust�be��e�t�in�chec��us-in���attentional�ca�acit���that�other�ise�coul���be���evote���to�the�te��orar���stor-a��e� of� infor�ation.� These� challen��es� inclu��e� sti�uli� that� elicit� stron��� habitual�res�onses,�an���irrelevant���istracters.�In��ee��,�recent�research�has�su����este���that��rocessin��,�such�as�tone�i��entification,�can�interfere��ith�ver�����ifferent�stora��e,�such�as�that�of�visual�arra��s�of�objects�(Stevanovs�i�an���Jolicoeur�2007);�an���that,�

Page 15: Unconscious Memory Representations in Perception

4� A�an��a�L.�Gilchrist�an���Nelson�Co�an

conversel��,�stora��e,�such�as�that�of�verbal�list�ite�s,�can�interfere��ith�ver�����if-ferent��rocessin��,�such�as�that�in�a�s�ee��e��,�nonverbal,�choice�reaction�ti�e�tas��(Chen�an���Co�an�2009b).�It�re�ains�to�be�seen��hether�attention�hol��s��ulti�le�ite�s�at�once���irectl���or,�alternativel��,�is�use���to�refresh�ite�s�one�at�a�ti�e�before�their�activation�is�lost�(Barrouillet�et�al.�2007).

1.1.4 Conscious�an���unconscious��rocessin���in���ifferent��o��els

Althou��h� these� �o��els� �ere� constructe��� �ith� the� ai�� of� ex�lainin��� behavior,�the���also�see��relevant�to�the�issue�of�consciousness.�Fi��ure�1�sho�s�three�of�the��o��els��ith�our�inter�retation�of��hat,��ithin�each��o��el,�is�li�el���to�corres�on���to�conscious�an���unconscious��rocesses.�Fi��ure�1A���e�icts�the�ori��inal��o��el�of�Broa��bent� (1958).� In� that� �o��el,� inco�in��� sti�ulation� fro�� the� environ�ent��as�sai���to�be�hel���in�a�te��orar���store,�or�buffer,�an���to���o�no�here�exce�t�for�the�s�all�a�ount�that�is�enco��e���into�the�li�ite��-ca�acit���buffer.�Given�that�sen-sor����e�or���is��uch�richer�than��hat�in��ivi��uals�can�re�ort�(�hich��as�sho�n�for�au��itor���sti�uli�b���Broa��bent�hi�self,�a�on���others,�an���for�visual�sti�uli�b���S�erlin���1960),�the��o��el��ust�consi��er�the�contents�of�the�sensor���buffer�to�be�unconscious.�Li�e�ise,�the�vast�infor�ation�in�lon��-ter���e�or���coul���not�all�be�in�consciousness�at�the�sa�e�ti�e.�It�is�the�li�ite��-ca�acit���buffer�that��oul���hol���the�contents�of�conscious�a�areness.

Fi��ure�1B�sho�s�the��o��el�of�Ba����ele���(2000).�Ba����ele��’s��o��els,�ever�since�Ba����ele���an���Hitch�(1974),�i��licitl���have�assu�e����ore�auto�atic��rocessin���than��as��a��e�clear�b���Broa��bent� (1958).� In��articular,� the�entr���of� infor�a-tion�into�the��honolo��ical�an���visuo-s�atial�buffers�is�sai���to�occur�auto�aticall��,�resultin��� in�rea����-to-use�co��es.�These,�ho�ever,�nee���not�be� in�consciousness.�For�exa��le,�Ba����ele���(1986)�su��arize���research�in��icatin���that�unatten��e���s�eech�coul���interfere��ith��or�in����e�or���for��rinte����or��s,���iven�that��ho-nolo��ical�co��es�are�in��e�or���for�both�t���es�of�sti�ulus�at�once,�lea��in���to�cor-ru�tion�of�the��aterial�to�be�re�e�bere��.�In�contrast,�the�e�iso��ic�buffer,��hich��as�a����e���to�the��o��el�b���Ba����ele���(2000)�in�or��er�to�account�for�te��orar����no�le����e�of�abstract�features,�see�s��ore�li�el���to�be�lin�e���to�conscious�a�are-ness.�Althou��h�to�our��no�le����e�Ba����ele���has�not�state���as��uch,�abstract�infor-�ation�has�the�a��vanta��e�that�it�coul���refer�to�an���sti�ulus.�Consciousness�see�s�to�have�the�qualit���of�unit���in�that�our�subjective�ex�eriences�are�such�that�there�is�onl���one��ool�of�consciousness;�for�exa��le,�if��e�are�loo�in���at�a��or��of�art�an���listenin���to�a�tour���ui��e’s���escri�tion�of�it,��e�feel�conscious�of�the��or��of�art,�the�verbal���escri�tion,�an����erha�s�the�tour���ui��e�all�at�once.�In�the�Ba����ele���(2000)��o��el,�the�onl���buffer�ca�able�of�re�resentin���an���inter-relatin���all�of�this�infor�ation�is�the�e�iso��ic�buffer.

Page 16: Unconscious Memory Representations in Perception

� Conscious�an���unconscious�as�ects�of��or�in����e�or��� 5

A��ore���ifficult,�thorn���issue�is��hether��e�are�conscious�of�central�executive�functions.�These�are�functions���riven�b����otivations�an���intentions,�as�one�can�tell�b���testin���in��ivi��uals�un��er���ifferent�sets�of�instructions�or�re�ar���contin-��encies.�The�central�executive�is,�b�����efinition,�the�set�of��echanis�s�res�onsible�for��lannin��,�sche��ulin��,�an���executin���infor�ation��rocessin���that�transfor�s�co��es� in�buffers� an����e�or��� stores,� affects��hat� infor�ation� is� atten��e��,� an���affects��hat�actions�are��erfor�e��.�Althou��h��e�are�at� least� t���icall���a�are�of�the�ite�s�to��hich�attention�has�been���irecte���an���the�ite�s�that�are�bein���trans-for�e���fro��one�co��e�to�another�or,�in�the��arlance�of�the��o��el,�shuttle���fro��

Figure 1. A���e�iction�of�three��o��els�of��or�in����e�or���an���infor�ation��rocessin��,�an���the�status��e��resu�e�for�each�co��onent�of�each��o��el��ith�res�ect�to�its�inclu-sion�in�conscious�a�areness�(conscious)�or�exclusion�fro��it�(unconscious).�Where��e���i���not�have�a��resu��tion�one��a���or�the�other,��e�a����e���a�question��ar�.�The�arro�s�co�in���fro��the�left-han���si��e�of�each�fi��ure�re�resent�the�entr���of�sensor���infor�ation�into�co��onents�of�the�s��ste��as�sho�n.�A.�Broa��bent’s�earl��-filter��o��el.�B.�Ba����ele��’s��ultico��onent��o��el.�C.�Co�an’s�e�be����e��-�rocesses��o��el.�

Sensory

Buffer (Un-

conscious)

Filter

(?)

Capacity-

limited

Buffer

(Conscious)

A. Broadbent (1958) Model

Long-term

Memory

(Un-

conscious)

B. Baddeley (2000) Model

Visuo-spatial

Buffer (Un-

conscious)

Central Executive Processes (?)

Long-Term Store (Unconscious)

Phonological

Buffer (Un-

conscious)

Episodic

Buffer

(Conscious)

Central Executive Processes (?)

C. Cowan (1988, 1999) Model

Long-term

memory (Un-

conscious)Activated

memory (Un-

conscious)

Focus of

attention

(Conscious)

**

*

Page 17: Unconscious Memory Representations in Perception

�� A�an��a�L.�Gilchrist�an���Nelson�Co�an

one�buffer�to�another,��e��a���not�be�a�are�of�the��rocess�b����hich�the�transfor-�ations�are�acco��lishe��.�Because�of�the�theoretical���ifficult���of�this�question,��e�leave�a�question��ar��in�the�central�executive.�Basicall��,�it�is�not�clear�to�us��hether�it�is�feasible�to�consi��er��rocesses�to�be�in�consciousness,�or��hether�it�is�onl���infor�ation�in�stores�that�is�hel���o�en�to�conscious�reflection.�One�reso-lution�of� this�question� is� that��rocesses��a�����ive�rise� to�store���re�resentations�that� are�hel��� in�consciousness,��hereas�other��rocesses�o�erate��ithout�atten-tion�an���consciousness���irecte���to�the�.�In��ee��,��hen�one�beco�es�too�a�are�of� the�central�executive��rocesses���overnin���a�co��lex�behavior,� it�can� lea���to��oor��erfor�ance�or�“cho�in��”�(Beiloc��an���Carr�2005);�as�F��o��or�Dostoevs���’s�character�co��laine���in�the�short�novel,�Notes From Underground,�in�such�cases�consciousness�is�a���isease.

Finall��,�Fi��ure�1C�sho�s� the�activation-base��,�e�be����e����rocesses��o��el�of�Co�an�(1988,�1999).�For�our��ur�oses,�this��o��el�has�the�a��vanta��e�that�the�access�to�consciousness�is�a��e���as�ect�of�the��o��el.�This�access�to�conscious-ness�is�s��non���ous��ith�the�focus�of�attention.�Ele�ents�of��e�or���outsi��e�of�the�focus�of�attention,��hether�active�or�not,�are�outsi��e�of�conscious�a�areness.�If� the� line�bet�een� the� focus�of�attention�an��� the�rest�of�activate����e�or��� is�seen�as�a�blurr���or���ra��ual�one,�then�the�focus�of�attention��i��ht�be�vie�e���as�bein���the�sa�e�as��ri�ar����e�or���as���escribe���b���Ja�es�(1890).�One�reason�to���e�arcate�conscious�versus�unconscious��rocessin���in�the��o��el�is�that�the�infor�ation�in�conscious�a�areness�is�assu�e���to���ain�access�to��uch���ee�er�an���extensive��erce�tual�an���conce�tual�anal��sis�usin���lon��-ter���e�or���in-for�ation� than� infor�ation� that� is�outsi��e�of�conscious�a�areness.�Therefore,���irectin���conscious�a�areness�is�an�i��ortant�s�ill�for�harnessin���infor�ation��rocessin���to�succee���at�a�tas�.�

One� of� the� stron��� suits� of� Ba����ele��’s� �o��els� is� that� the��� have� been� �ell-��roun��e��� in� research� on� �atients� �ith� brain� injur��.� One� can� fin��� instances�in��hich��atients�a��ear� to�be���eficient� in� te��orar����honolo��ical� stora��e�or�rehearsal� (e.��.,� Ba����ele��� et� al.� 1988),� attention-relate��� �or�in��� �e�or��� (e.��.,��Jefferies�et�al.�2004),�or�lon��-ter��e�iso��ic�stora��e�(e.��.,�Ba����ele���an���Warrin��ton�1970).�None�of�these�cases�are�es�eciall����roble�atic�for�the�Co�an��o��el,�ei-ther.�It�is�ac�no�le����e���in�the��o��el�that��honolo��ical�activation�is�one�t���e�of�activate����e�or���an���that�rehearsal�of�it�can�ta�e��lace��ithout��uch�attention.�Attention-relate����or�in����e�or���an���auto�atic�activation�are�both�intrinsic�to�the�Co�an��o��el,�as�is�lon��-ter��stora��e.�There�is�no�reason��h���s�ecific�le-sions�coul���not�selectivel��� i��air�these�as�ects�of��e�or��.�Althou��h��atients��ith�lon��-ter��e�iso��ic�stora��e�i��air�ents�obviousl����oul���be�unable�to�ac-tivate�infor�ation�that��as�not�store���in�the�first��lace,�infor�ation�in�se�antic�

Page 18: Unconscious Memory Representations in Perception

� Conscious�an���unconscious�as�ects�of��or�in����e�or��� 7

�e�or���can�beco�e�te��oraril���activate���b���ne��sti�uli�an��,�in�that��a��,�use���in�short-ter��stora��e�tas�s.�

An� i��ortant� issue� that� �as� a����resse��� in� enou��h� ��etail� b��� Co�an� (1988,�1999)�is��here�infor�ation�about�the�in��ivi��ual’s���oals�is�hel��.�When�the���oals��ee�� shiftin��� or� �hen� there� is� infor�ation� that� un��er�ines� the� ��oal,� the� ��oal��resu�abl����ust�be��e�t�in�the�focus�of�attention,�usin���u��so�e�of�the�li�ite���ca�acit���of�that�store.�An�exa��le�of�this�is�the�Stroo��effect,�in��hich�the���oal�is�to�na�e�the�color�of�in��but�the��or���s�ells�out�a���ifferent�color.�Kane�an���En��le�(2003)�sho�e���that�in��ivi��uals��ith�hi��h��or�in����e�or���s�an�carr���out�this�tas���ore�successfull���than�lo��s�ans�if�a�conflict�onl���occurs�on�a�s�all��ro�or-tion�of�the�trials.�In�contrast,�if�the���oal�beco�es�routine�an���it�is�not�un��er�ine���b���conflictin���sti�uli,�that���oal�nee���not�use�u��so�e�of�the�li�ite���ca�acit���of�the�focus�of�attention�an����a���be�hel���instea���in�a��ortion�of�lon��-ter���e�or���(or�in�the�central�executive�itself�if�it�can�be�consi��ere���to�have�a��e�or��,��hich��as�so�eti�es�assu�e���b���Ba����ele��,�but�not�b���Co�an).�

1.1.5 More���etail�on�the�e�be����e��-�rocesses��o��el

Thus,�the�e�be����e��-�rocesses��o��el,�rather�than�bein���co��ose���of�se�arable�structures,� inclu��es� �ithin� the� long-term memory� s��ste�� a� currentl��� activated�subset�of��e�or���an��,��ithin�that�in�turn,�a�subset�that�is�the�current� focus of attention.�The�activate����ortion�of��e�or����as�sai���to�inclu��e�unli�ite���infor-�ation�at�once�but�it��as�assu�e���to�lose�its�activation�throu��h�ti�e-base�����e-ca����ithin�10� to�20�secon��s,��rovi��e��� that�activate��� ite�s�are�not�atten��e���or�rehearse��.�The�focus�of�attention�co��rises�a� li�ite���nu�ber�of� ite�s�that�are�not� onl��� hi��hl��� activate��,� but� are� also� �ost� ra�i��l��� accessible� an��� are� in� one’s�a�areness� (‘in� �in��’).� In� contrast� to� the� ti�e-li�ite��� ��eca��� of� activate��� lon��-ter���e�or��,�the�focus�of�attention��as�assu�e���not�to���eca���but��as�li�ite���to�onl���several�se�arate,��eanin��ful�ite�s�at�once.�

The�focus�of�attention�functions�as�a�zoo��lens��ith�res�ect�to�its�co��risin���infor�ation�(see�also�LaBer��e�an���Bro�n�1989).�It�can�‘zoo��in’�on�a��articular�ite�,�lea��in���to��ore�focuse���an����recise�allocation�of�attention�to�a�sin��le�ite�,��ith�re�ainin���ite�s�in�the�focus�receivin���consi��erabl���less�attention�an�����etail;�this�su����ests�that�attentional�resources�can�be�allocate���on�the�basis�of��rioritiza-tion�if�necessar��.�In�contrast,�if��ulti�le�ite�s�nee���to�be��rocesse���si�ultaneous-l��,�the�focus�of�attention�can�‘zoo��out’,�lea��in���to�a���reater�brea��th�of��rocessin��,�but��ith�less��recision�overall.�

Ho��infor�ation�enters�into�a�areness���e�en��s�u�on�several���ifferent�fac-tors� that�are�accounte��� for� in� the��o��el.�The�first� factor� involves� the�ori��in�of�

Page 19: Unconscious Memory Representations in Perception

�� A�an��a�L.�Gilchrist�an���Nelson�Co�an

infor�ation;� ite�s� �a��� enter� into� the� focus� of� attention� fro�� either� an� inter-nal�(e.��.,�infor�ation�containe���in�lon��-ter���e�or��)�or�an�external�(e.��.,�sen-sor���infor�ation�fro��the�environ�ent)�source.�The�e�be����e��-�rocess��o��el�accounts� for� such� ��ifferences� throu��h� the� inclusion� of� t�o� ��ifferent� �eans� to�control�the�focus�of�attention.�Attention�is���irecte���to�ar��s�internall��-��enerate���ite�s�in��e�or���via�a�central�executive.�Co�an�(1999)���efine���the�central�execu-tive�as�a� subset�of�co��nitive��rocesses� that�are��o��ifie���b��� tas�-base��� instruc-tions,���oals,�an���incentives.�Consi��er�ho��such�effortful�allocation�of�attention��i��ht�o�erate�in�a���eneral�fluenc���tas�.�Partici�ants�are��resente����ith�a���iven�cate��or��,�such�as�‘ani�als’;�the���are�to�na�e�as��an���ani�als�as�the���can�fro���e�or��.�In�this�case,�the�central�executive�can���irect�a�search�of��e�or��,�select-in��� onl��� those� ite�s� that� are� relevant� to� the� tas�� (e.��.,� ‘ostrich’,� but� not� ‘ostro-��oth’).�Attention�is���irecte���to�ar��s�these�ite�s,�allo�in���access�to�the�focus�of�attention.�A����itionall��,��artici�ants��ust��onitor�their�res�onses,�to��ee��fro��recallin���a�na�e���ite���ulti�le�ti�es�–�the�central�executive�can�further�select�onl���those�ani�al�na�es�that�have�not���et�been�recalle��.�These�t�o��rocesses��a���be�re�eate���until�no�further�ani�al�na�es�can�be�recalle���–�the�central�executive��a���ter�inate��e�or���search�an���retrieval�of�cate��or���exe��lars,�thus�en��in���the�tas�.�Contrastin���the�above���escri�tion,�allocatin���attention�to�external�sen-sor���infor�ation�occurs�throu��h�filterin���throu��h�the�central�executive�(si�ilar�to�internall��-��enerate���infor�ation),�as��ell�as�throu��h�orientin���to�chan��es�in�environ�ental�sti�uli.�Infor�ation�fro��the�sensor���stores�enters�the�contents�of��e�or��;��hether�infor�ation���ains�access�to�the�focus�of�attention���e�en��s�u�on�ite��features�(as��hen�one�listens�to�one��erson’s�voice��hile�i��norin���other��eo�le�in�the�roo�).�

Processin����ithin�the�e�be����e��-�rocess��o��el�also���e�en��s�u�on�the�rela-tive�fa�iliarit���or�novelt���of��resente���sti�uli.�In���eneral,� features�that�re�ain�fixe���over�ti�e�or�are�not�novel���o�not�ca�ture�attention,�thou��h�these�features��a���be�activate���in��e�or���if�the���are�alrea�����sufficientl���fa�iliar�an���have�been���ee�e���tas�-relevant�via�the�central�executive.�Rather,�unchan��in���features�are�often�subject�to�habituation�an���lose��erce�tual�salience�over�ti�e;�it�shoul���be�note���that�for�these�cases,�access�to�the�focus�of�attention��ill�be��ore�li�el���to�occur� throu��h� voluntar��� control� of� attention� throu��h� the� central� executive.� In�contrast,�novel�or�����na�ic�features��resent�in�the�environ�ent���e�an���orient-in���an���allocation�of�attention,�an���are�thus�involuntaril���ca�ture���in�the�focus�of�attention.

Recent� research� �rovi��es� a����itional� evi��ence� for� a� �roce��ural,� activation-base��� �or�in��� �e�or��� s��ste�.� Oberauer� (2002,� 2005)� �ro�oses� a� �o��el� that�is�quite�si�ilar�to�the�e�be����e��-�rocess��o��el���escribe���above.�All��e�or���is�containe����ithin�the�contents�of�lon��-ter���e�or��,�a�subset�of��hich�is�activate��.�

Page 20: Unconscious Memory Representations in Perception

� Conscious�an���unconscious�as�ects�of��or�in����e�or��� 9

The��o��el��ro�ose���b���Oberauer���iffers�fro��the��o��el��ro�ose���b���Co�an�in�one�i��ortant�res�ect�–�the�size�of�the�focus�of�attention.�Whereas�the�e�be����e��-��rocess��o��el�has�a�focus�of�attention�containin���four�ite�s,�the�Oberauer��or�-in����e�or����o��el���efines� this� as�a� region of direct access.�Li�e�Co�an’s� focus�of�attention,�the�re��ion�of���irect�access�hol��s�a��roxi�atel���three�or�four�ite�s�or�chun�s�that�are�i��e��iatel���accessible�an���available�for�co��nitive��rocessin��.�Unli�e�the�e�be����e��-�rocess��o��el,�this���irect-access�re��ion�contains�an�a����i-tional�neste����rocess,��hich�is�the�focus�of�attention.�Here,�the�focus�of�attention�hol��s�onl���one�ite��or�chun��to�use�for�a�subsequent�co��nitive��rocess.�

1.1.� Further�e�be����in���of�Oberauer�(2002)

In�a�series�of�ex�eri�ents�exa�inin���shifts�of�attention�an���u���atin���in�lists�of��or��s�or���i��its,�Oberauer�(2002,�2005)�foun���that�the���irect�access�re��ion��ro-�ose�����e�en��s�on�currentl��-relevant�lists,�an���that�the�focus�of�attention�hol��s�the� ite��that��as��ost� recentl���u���ate��� in�a� tas�.�Partici�ants��ere��resente����ith�t�o�lists�of�either�1�or�3�ite�s,��hich��ere�to�be�u���ate����hen�a��articular�ite���as�cue��.�For�exa��le,�a��artici�ant��i��ht�receive�a�visuall��-�resente�����i��it�list:�4�7�2.�One�of�these�list�ite�s�(e.��.,�the�last�ite�)��i��ht�be�cue���for�u���atin����ith�–2,�in��icatin���to�subtract�t�o�fro��the�last�ite�’s�current�value.�Thus,�the�ne��list�shoul���no��be�4�7�0�–�this��ust�be�retaine���for�a�later�test�of�recall.�In�a����ition� to� in��icatin��� �hich� corres�on��in��� ite�� �as� to� be� u���ate��,� cues� also�in��icate����hich�of�the�t�o�lists��as�currentl���relevant�to�the�tas�.�A�critical��a-ni�ulation�occurre����hen�another�cuein���an���subsequent�u���atin���instruction��as��resente���to��artici�ants.�Partici�ants��a���be�as�e���to�u���ate�a��reviousl��-u���ate��� ite���ithin�the�tas�-relevant� list� (i.e.,�no�s�itch)�or�another� ite��in�a�tas�-relevant� list� (i.e.,�an�object�s�itch);�finall��,� the����a���be� instructe���to�shift�their�attention�to�ar��s�the��reviousl��-irrelevant�list,�ter�e���a�list�s�itch.�For�u�-��atin���ite�s��ithin�the�sa�e�list,�res�onse�ti�e�for�object�s�itches�is�lon��er�than��hen�no�ite��s�itch�is�require��.�All�relevant-list�ite�s�are�containe����ithin�the�re��ion�of���irect�access;�the���are�i��e��iatel���available�for��rocessin��.�The���iffer-ences�in�res�onse�ti�es�for�s�itchin���ite�s�versus�no�ite��s�itch�su����est�a�shift�of� the� focus�of�attention� to�ar��s� the�ne�l��-cue��� ite�;��resu�abl��,� if� all� ite�s��ere�equall���hel��,�as�the���are�in�the�Co�an��o��el,�ite��s�itches��ithin�an�ac-tive�list�shoul���not�result�in�increase���res�onse�ti�e.�In�a����ition�to�su��ort�for�a�one-ite�-li�ite���focus�of�attention,�Oberauer�has�foun���that�list,�but�not�object,�s�itches�result� in� increase���res�onse�ti�e�for�u���atin���an���recall�of� list� ite�s.�This�has�been�ta�en�as�evi��ence�that�s�itchin���lists��a���involve��ovin����revious-l��-relevant�ite�s�fro��the���irect-access�re��ion�into�activate���lon��-ter���e�or��,�

Page 21: Unconscious Memory Representations in Perception

10� A�an��a�L.�Gilchrist�an���Nelson�Co�an

�ith� ne�l��-relevant� ite�s� follo�in��� the� o��osite� �rocess.� Object� s�itches� �ill�require�less�ti�e�to��rocess,�as�all�of�the�ite�s�are�in�the�re��ion�of���irect�access;�ite�s��ill�si��l���be��ove���in�an���out�of�the�focus�of�attention.�Finall��,�Oberauer�foun���that�res�onse�ti�e�increases��ith�increasin���list� len��th�(i.e.,�a� list-len��th�effect),�but�onl���for�lists�that��ere��reviousl���tas�-relevant.�As�s�itchin���lists�re-quires��ovin���ite�s�bet�een�activate���lon��-ter���e�or���an���the���irect-access�re��ion,�this�su����ests�that��ovin���ite�s�out�of�the�focus�of�attention�an�����irect-access�re��ion�is��ore�effortful�an���ti�e-��e�an��in���than��ovin���ite�s�into�these�neste����rocessin���re��ions.�S�ecificall��,��ovin���ite�s�out�of�the�re��ion�of���irect�access�occurs�in�a�serial�fashion,�ta�in����ore�ti�e�as�list�len��th�increases.�

The� results� above� �rovi��e� stron��� evi��ence� for� the� further� e�be����e��� �ro-cessin�����escribe���in�Oberauer’s��o��el.�As��e��ill���iscuss�in�a�later�section,�this��o��el�fits��ell��ith�current�theories�of�conscious��rocessin��,��articularl���those��here�one�ite��receives�conscious��rocessin���at�a�ti�e,�such�as�Global�Wor�s�ace�Theor���(GWT)�(Baars�et�al.�2003;�Baars�an���Fran�lin�2003,�2007).�Ho�ever,��e�are�of�the�o�inion�that��ore�ex�loration��ith�the�above��ara��i����is�necessar��.�A��articular�concern�relatin���to�the�above��ara��i����is�the�use�of�incre��ibl���s�all�set�sizes�to�test�effects�of�list�len��th�an���tas��relevance.�We�ac�no�le����e�that��or�in����e�or���is�li�ite���to�a�s�all�subset�of�ite�s,�an���thus�s�aller�sets�are�necessar��,�but�the�inclusion�of�set�sizes�is�not�sufficientl���exhaustive.�For�exa��le,�Oberauer�(2005)�notes�that�list�len��ths�of�2�an���4��ere�teste���in��ilot�research,�but�the�tas���as�too���ifficult�for��artici�ants�un��er�these�con��itions.�For�the��ost�co��re-hensive�account�of�the���irect-access�re��ion�an���focus�of�attention,�these�set�sizes�shoul���be�inclu��e���in�ex�eri�ental�con��itions.�This�is��articularl���so�for�lists�of�one�versus�t�o�ite�s,��here�it�is��ossible�that�attention��ust�chan��e�fro��a�sin-��ular�focus�to�bein�����ivi��e���in�half.�A����itionall��,�set�sizes�that�excee���s�an��ust�also�be�consi��ere���to�enrich�this��o��el�further.�As�Oberauer�(2002,�2005)�onl���exa�ine���subs�an�lists�of�ite�s,�it�is�unclear�ho���ovin���ite�s�bet�een�the�e�-be����e���co��onents��ill�be�affecte��.�Surel���interference�bet�een�the�ite�s��oul���be�far��ore�li�el��,�an���this��ust�so�eho��be�accounte���for.�

Des�ite�the���ifferences�bet�een�the�t�o��o��els,�the���are�not�necessaril���in�conflict� �ith� re��ar��s� to� �rocessin��� of� conscious,� ��irectl��-accessible� ite�s.� Al-thou��h�there�is�no�further�neste����rocessin���re��ions��ithin�the�e�be����e��-�ro-cessin����o��el,�Co�an�(2001)�note��� that�attention� is� li�el���not�equall�����ivi��e���bet�een� the� ite�s� in� the� focus� of� attention,� �articularl��� �hen� the� nu�ber� of�ite�s��aintaine���is�subs�an�(see�also�Zhan���an���Luc��2008).�In�cases��here�the�nu�ber�of�ite�s�is�belo���or�in����e�or���s�an,��ore�than�one�fixe���slot�can��rocess�a���iven�ite�.�This�i��lies�that�so�e�ite�s��ithin�the�focus��a���require��ore�attentional� allocation,� an��� thus���reater��rocessin��,� than�others.�The� i��ea�that�li�ite��-ca�acit���attention�is�allocate���to�active�ite�s�is�not�a�ne��one.�Yantis�

Page 22: Unconscious Memory Representations in Perception

� Conscious�an���unconscious�as�ects�of��or�in����e�or��� 11

an���Johnson�(1990)�exa�ine���various��o��els�of�such��rioritization�in�a�series�of� visual� search� tas�s.� Partici�ants� �ere� instructe��� to� search� for� a� tar��et� letter�in�an�arra���of�a����itional� letters.�Letters� in� the�search�arra���either��ere��resent�in�a��rior,��as�e����lacehol��er�arra��,��no�n�as�no-onset�letters,�or�a��eare���in�locations�that��ere�blan��in�the��revious�arra���(i.e.,�abru�t-onset�ite�s).�Fittin���the�resultin�����ata�to�the��o��els,�Yantis�an���Johnson�foun���evi��ence�for�atten-tional��rioritization� in�visual��e�or��.�S�ecificall��,� all� abru�t-onset� ite�s��ere��rocesse���first,���ue�to�auto�atic�re��istration;�follo�in���this,�no-onset�letters�re-ceive���attention.�Ite�s�that�are���ee�e���hi��h��riorit����a���be��rocesse���via�either�a��ro�ose���queue��echanis��or�throu��h�ta��s�that���eca���over�ti�e,�thou��h�it�is�unclear�ho��the�or��er�of��rocessin���these�ite�s�is�attaine��.�Most�li�el��,�for�cases�involvin���visual� search,�hi��h��riorit��� ite�s�are� li�el����rocesse���on� the�basis�of�si�ilarit���to�a�tar��et�(i.e.,�tas�-relevant�features),��ith�ite�s��ost�rese�blin���the�tar��et�receivin���attention�first.�A����itionall��,�ite�s�that�are��ost��erce�tuall���sa-lient�are��ost�li�el���to�ca�ture�attention�initiall��,��ith�less�salient�ite�s�follo�in���suit.�With�this�in��in��,�the�e�be����e��-�rocess��o��el��a���utilize�such��rioritiza-tion��ithin�the�focus�of�attention,�‘zoo�in���in’�on�one�ite��that�receives�initial�attention�for�reasons�of�tas��relevance�or�salience�of�its�co��risin���features.�This�is�analo��ous�to�Oberauer’s�one-ite��focus�of�attention.�We�shoul���note�that��ore�research�an���ex�loration�is�nee��e���to���eter�ine�the�un��erl��in����echanis�s�for���eter�inin����riorit��,�for�both�visual�an���verbal�as�ects�of��or�in����e�or��.�In�contrast�to�the�t�o���ifferences�that�are�initiall���assu�e��,�the��o��el��ro�ose���b���Oberauer�actuall���onl�����iffers�in�one�res�ect:�Oberauer’s��o��el�si��l���contains�an�a����itional�level�of�nestin���be��on���that�of�the�e�be����e��-�rocess��o��el.�The��anner�in��hich�conscious,���irectl��-accessible�ite�s�are��rocesse����ithin�their�res�ective�re��ions�in�the�t�o��o��els,�ho�ever,���oes�not���iffer.�

To��resent�in�full� the�evi��ence�for�an���a��ainst�each�of�these��o��els��oul���be�the�to�ic�of�a�lon���boo��rather�than�a�cha�ter.�Belo�,��e��ill�si��l����resent�so�e�of�the�best-�no�n�relevant�research�in�or��er�to��rovi��e�an�un��erstan��in���of�so�e�of�the��otivation�for�the�construction�of�the�e�be����e��-�rocesses��o��el,�to�illustrate�the��otivation�behin���the�assu��tions�about�its�relation�to�conscious�a�areness,�an���to�ex�lore�the�current�status�of�these�assu��tions.

1.2 Research basis of consciousness and embedded processes in working memory

Co�an� (1988,� 1995,� 1999,� 2005)� su��arize��� ��iverse� evi��ence� relevant� to� the�conscious�versus�unconscious� status� of� infor�ation� in�activate����e�or���ver-sus�the�focus�of�attention.�What�is�critical�is�that�sensor���features�fro���ulti�le�

Page 23: Unconscious Memory Representations in Perception

12� A�an��a�L.�Gilchrist�an���Nelson�Co�an

sti�uli�can�beco�e�activate���in��e�or���at�the�sa�e�ti�e,��ithout�evi��ence�that�the���enter�a�areness.�Ho�ever,���ee�er��rocessin���see�s�to�require�a�areness,��a�in���the�s��ste���ore�li�e�that�of�Broa��bent�(1958).�Co�an�(1988)��as�non-co��ittal�about��hether�the�auto�aticall���activate���infor�ation�coul���inclu��e�se�antic� infor�ation.� Unli�e� Broa��bent� �e� still� assu�e� that� it� �a��� but,� li�e�Broa��bent,��e���o�not�believe�that�auto�atic�activation�of�se�antic�features�ca�-tures�attention.�The�evi��ence�co�es�fro��se�arate�au��itor���an���visual�stu��ies.�

1.2.1 Au��itor���stu��ies�relate���to�consciousness�an����or�in����e�or��

Broa��bent�(1958)�su��arize����or��of�this�t���e�b���several�investi��ators.�In�so�e�of� the��or�,�a���ichotic� listenin����etho����as�use��� in��hich���ifferent��essa��es��ere��resente��� in�the�t�o�ears�usin���a� ta�e�recor��in��.�The��essa��e� in�one�ear��as�to�be�atten��e���an���re�eate���(sha��o�e��)��hereas�a���ifferent��essa��e�in�the�other�ear��as�to�be�i��nore��.�Occasionall��,�the�ta�e��as�sto��e���an���the�research��artici�ant��as�to�re�eat�as��uch�as��ossible�fro��the�ear��ith�the��essa��e�to�be�i��nore��.�Onl���the��ost�recent�infor�ation�fro��that��essa��e�coul���be�re�e�-bere��;�the�last�fe��secon��s�at�best.�Man���versions�of���ichotic�listenin���ex�eri�ent���iel��e���infor�ation�consistent��ith�the�conclusion�that�infor�ation�is�hel���in�a�sensor���for��an���is�for��otten��ithin�a�fe��secon��s�unless�it�is�entere���into�atten-tion�an���consciousness.�

Johnston�an���Heinz�(1978)��ere�able�to�verif���the�assu��tion�that�attention�an���a�areness���o�to��ether�an���are�foun���for�one�channel�in�selective�listenin��.�The����resente���t�o�au��itor���channels�that��ere���istin��uishe���either�in�ter�s�of��h��sical�cues�li�e�location�an���voice�qualit��,�as�in��ost�of�the�stu��ies���escribe���b���Broa��bent�(1958),�or��ere��resente���in��h��sicall���i��entical��eans,���ifferin���onl���in�the�se�antic�to�ic�of�the��essa��e.�The��e����ani�ulation��as�that,�in�a����ition�to�sha��o�in���one�channel,��artici�ants�also��ere� to�carr���out�a� the�subsi��iar���tas��of��ressin���a��e���as�quic�l���as��ossible��hen�a�visual�si��nal�a��eare��.�The�subsi��iar���tas��reaction�ti�e��as��no�n�fro��other��or��to�be�a���oo����easure�of�ho���uch�attention�is�free���es�ite�the�selective�listenin���tas�.�It��as�foun���that�the�subsi��iar���tas��reaction�ti�e��as��uch�faster��hen�the��essa��es��ere��h��si-call���se�arate���than��hen�the����ere�onl���se�anticall���se�arate��.�Within�the�e�-be����e��-�rocesses��o��el,�this�in��icates�that�the�focus�of�attention�can�be���irecte���at�one��h��sical�strea��of�infor�ation��ith�little���ifficult��.�This�cannot�be���one��hen�the��essa��es���iffer�onl���in�their�se�antic��eanin��s�because�the�anal��sis�of�se�antics�is�not�auto�atic;�it�requires�attention�an���effort.�It�see�s��ost�li�el���that�both��essa��es�are�hel���in�the�focus�of�attention�until�the��ortions�relate���to�a�se�anticall���coherent��essa��e�can�be�extracte��.�Given�the�nee���to�hol���so��uch�

Page 24: Unconscious Memory Representations in Perception

� Conscious�an���unconscious�as�ects�of��or�in����e�or��� 13

infor�ation��hile�inter�retin���it�on�the�basis�of��no�le����e�an�����uess�or�,�this�is�an�effortful��rocess�that�leaves�little�attention�to�be�use���for�the�subsi��iar���tas�.

Both�the�e�be����e��-�rocesses��o��el�(Co�an�1988,�1999)�an���the�Broa��bent�(1958)��o��el�are�consistent��ith�these�fin��in��s�in��icatin���that�not��uch�auto-�atic� se�antic��rocessin���contributes� to� sha��o�in����erfor�ance.�One��a��� in��hich�the�����iffer�is�in�the��anner�of�the�control�of�attention.�Broa��bent��osits�an�attentional�filter�an���one��i��ht�thin��that�the��ore�channels�there�are�to�be�rejecte��,�the�har��er�the�filter��ust��or�.�In�Co�an’s��o��el,�in�contrast,�there�is�no�filterin���an���the�focus�of�attention��ust�select�one�channel� to�be��rocesse���se�anticall��,�a��rocess�that�nee���not���e�en���on�ho���an���channels�there�are�to�be�rejecte��.�A�further�ex�eri�ent�of�Johnston�an���Heinz�(1978)�see�s�to�su��ort�this�a����itional��re��iction.�A�con��ition�in��hich�there��ere�onl���t�o��h��sicall�����ifferent��essa��es,�one�to�be�atten��e���an���one�to�be�i��nore��,��as�co��are���to�a� con��ition� in� �hich� there� �ere� three� �h��sicall��� ��ifferent� �essa��es,� one� to� be�atten��e���(a�fe�ale�voice)�an���t�o�to�be�i��nore���(t�o���ifferent��ale�voices).�The�subsi��iar���tas��reaction�ti�e��as�the�sa�e�no��atter��hether�it��as�necessar���to�i��nore�one�or�t�o�channels,�as�ex�ecte���accor��in���to�the�e�be����e����rocesses��o��el�in��hich�there�is�no�real�filterin���out�of�irrelevant�sti�uli,�just�accentuation�of�the�relevant�channel�or��essa��e�hel���in�the�focus�of�attention.

At�one�ti�e�there��as�an�i��ortant��ove�ent�to�ar����o��els�in��hich�infor-�ation�lea�s�throu��h�the�attentional�filter,�such�as�the�attenuatin���filter��o��el�of�Treis�an�(1964).�The��ain�reason�for�it��as�that�there��ere�a�nu�ber�of�stu��ies�su����estin���that�infor�ation���i�����et�throu��h.�The��ost�fa�ous�of�those��as��hen�the��artici�ant’s�o�n�na�e��as��resente���in�the�unatten��e���channel�in�selective�listenin���(Mora���1959).�The�fin��in����as�that��artici�ants�so�eti�es�coul���reco��-nize�an���re�e�ber�hearin���their�o�n�na�es.�Woo���an���Co�an�(1995a,�1995b)�confir�e���that��artici�ants�coul���notice�subtle�chan��es�in�the�unatten��e���chan-nel,�inclu��in���their�o�n�na�es.�It��as�about�1/3�of�the��artici�ants��ho�notice���their�na�es,�about�the�sa�e��ro�ortion�obtaine���in�the�s�aller�stu�����b���Mora��.�Ho�ever,�there�are���ifferent��a��s�that�this�coul���co�e�about.�It�coul���be�that�in-for�ation�about�the�na�e�in��ee���lea�e���throu��h�the�filter�(or,�in�the�e�be����e����rocesses��o��el,� that�acoustic� co��onents�of� the�na�e��a��e�contact��ith� the�lon��-ter���e�or���re�resentation�of�the�na�e�an���thereb���attracte���attention).�Alternativel��,�it�coul���be�that�attention�often��an��ere���fro��the�assi��ne����essa��e�over�to�the�unatten��e����essa��e,�or��as�s�lit�bet�een��essa��es,�an���therefore�that�the�na�e��as�alrea�����atten��e����hen�it��as�hear��.�It�a��ears�that�that�is�the�case�inas�uch�as�one�stu�����(Con�a��,�Co�an,�an���Buntin���2001)�sho�e���that�65%�of�in��ivi��uals�in�the�lo�est�quartile�of��or�in����e�or���notice���their�na�es,��here-as�onl���20%�of�in��ivi��uals�in�the�hi��hest�quartile�of��or�in����e�or���notice���their�na�es.�The��ost�strai��htfor�ar���inter�retation�is�that�the�hi��h�s�ans��ere�on�tas��

Page 25: Unconscious Memory Representations in Perception

14� A�an��a�L.�Gilchrist�an���Nelson�Co�an

�ore�often�than�the�lo��s�ans.�Colflesh�an���Con�a���(2007)�sho�e���that��hen�the�tas���as�to�tr���to��onitor�both��essa��es�at�once,�hi��h-s�an�in��ivi��uals�notice���their�na�es��ore�often�than�lo��s�ans.

1.2.2 Visual�stu��ies�relate���to�consciousness�an����or�in����e�or��

S�erlin���(1960)�carrie���out�a�se�inal�stu�����that�can�be�consi��ere���the�basis�of�the�visual��or�� relatin����or�in����e�or��� to�consciousness,��uch�as� the��or��revie�e��� b��� Broa��bent� (1958)� can� be� consi��ere��� the� basis� of� the� corres�on��-in���au��itor����or�.�S�erlin����resente���an�arra���of�characters�(�rinte���letters�an���nu�bers)�on�a�co��uter�screen,�so�eti�es�follo�e���shortl���after�ar���b���a�tone�cue� in��icatin����hether�the�to�,��i����le,�or�botto��ro��of�characters��as�to�be�re�orte��.�With�no�tone�cue,��artici�ants�coul���recall�about�4�ite�s�fro��the�arra��.�When�the�tone�cue�follo�e���closel���after�the�arra��,�the���coul���recall�about�4�ite�s�fro��the�selecte���ro�.�This�in��icate���that�there�are�t�o�li�its�in�recall.�First,�there�is�a�li�it�of�about�4�ite�s�in�ho���uch�infor�ation�can�be�extracte���into��or�-in����e�or��.�Secon��,�there�is�a�li�it�in�ho���an���ite�s�can�be�enco��e���fro��the�arra���but�it�is�not�nearl���as�severe�as�4�ite�s.�Instea��,�it�a��ears�that��ost�or�all�of�the�arra���ite�s�are�enco��e��,���iven�that�the��artici�ants���i���not��no��in�a��vance��hich�ro���oul���be�cue���but�still�coul���successfull���recall�4�ite�s�fro��that�ro�.�The�retention�of��ost�of�the�arra����as�sai���to�be�in�the�for��of�sensor����e�or��,�usable�if�attention�is�turne���to�it�but�not�re�ortable�in�an��thin���near�its�entiret��.�This�sensor����e�or���therefore�see�s�to�be�hel���outsi��e�of�the�focus�of�attention�an���a�areness.�In�contrast,�the�s�all�a�ount�of��or�in����e�or���that�is�re�ort-able�(about�4�ite�s)�is�in�attention�an���a�areness.�

Luc��an���Vo��el�(1997)�create���a�cleaner�version�of�S�erlin��’s��roce��ure�that��as�inten��e���to�quantif���visual��or�in����e�or��.�The���use���colore���nonverbal�sha�es�rather�than�verbal�characters�to�sho��that��or�in����e�or���for�even�non-verbal�infor�ation�is�li�ite���to�about�4�ite�s.�The�arra���of�objects�to�be�re�e�-bere����as�follo�e���b���a��robe�that�coul���even�be�a�sin��le�ite�,�the�tas��bein���to�in��icate��hether�the�ite��chan��e���fro��the�corres�on��in���ite��in�the�ori��inal�arra��.�Perfor�ance��as�nearl����erfect��hen�there��ere�3�or�4�ite�s�in�the�arra��,�an�����ecline���as�the�arra���size�increase��.�Co�an�(2001)���erive���a�si��le�for�ula�to� esti�ate� the� nu�ber� of� arra��� ite�s� in� �or�in��� �e�or��� for� the� sin��le-ite���robe�case,�an���Rou��er�et�al.�(2008)�foun���a���oo����athe�atical�fit�of���etaile�����ata�to�the�for�ula.�

The�t���e�of��e�or���observe���b���Luc��an���Vo��el�(1997)�is�often�consi��ere���to�be�visual��or�in����e�or��,��hich�coul����a�e�it�a�rather�auto�atic�store�li�e�the�visuos�atial�buffer�of�Alan�Ba����ele��’s��o��els,��otentiall���o�eratin����ithout�

Page 26: Unconscious Memory Representations in Perception

� Conscious�an���unconscious�as�ects�of��or�in����e�or��� 15

conscious�a�areness.�Instea��,��e�believe�that�it�is�a�central��e�or���that�has��ore�to���o��ith�the�focus�of�attention�an���consciousness.�More���an���Co�an�(2004)�sho�e���that�interference�occurs�fro��a�s�o�en�verbal��e�or���loa���(seven�ran-��o����i��its),�but�not�fro��a�s�o�en�series�that�bloc�s�articulation�but���oes�not�loa����e�or���(a��no�n�seven-��i��it�tele�hone�nu�ber).�Saults�an���Co�an�(2007)�use�����i��its��resente��� in���ifferent�voices� fro�� four� lou��s�ea�ers� to��revent� re-hearsal�an���foun���that�the��e�or���for�the���i��its�tra��e���off��ith��e�or���for�the�colore���visual�objects;��artici�ants�coul���re�e�ber�about�3.5�visual�objects�or,�if�as�e���to�re�e�ber�both��o��alities,�the���coul���re�e�ber�fe�er�visual�objects�but�about�3.5�objects�total,�visual��lus�au��itor��.�Co�an�an���More���(2007)�sho�e���that�this�interference�bet�een�visuos�atial�an���au��itor��-verbal�ite�s�occurre���in�the��or�in����e�or���retention��hase,�not�just�in�the�enco��in���of�the�sti�uli.�It�see�s,�therefore,�that��e�are���ealin����ith�a�central��or�in����e�or���store�that�is�li�ite���to�3�or�4�ite�s�in�a��ults.�Not�sur�risin��l��,�this�central�store��oul���be�the�sa�e�that�S�erlin���(1960)�observe����ith��rinte���verbal�characters.�This�li�ite��-ca�acit���store��atches�the�conscious�infor�ation�that�the�e�be����e��-�rocesses��o��el�conceives�as�the�focus�of�attention�an���a�areness.

Stu��ies�revie�e���b���Co�an�(2001)�sho�e���that�in��ivi��uals�var���quite�a�bit�in�their��or�in����e�or���ca�acit��;�a��ults�var���fro��2�to�6��ith��ost�in��ivi��uals�in�the�ran��e�of�3�to�5�ite�s,�an���chil��ren�an���ol��er�a��ults��ore�t���icall���about�1�ite��fe�er.�There�are�at�least�t�o��echanis�s�b����hich�the�visual�store�in��exe���b���the�tas��of�Luc��an���Vo��el�(1997)�coul���var���a�on���a��e���rou�s�an���in��ivi��u-als.�First,�it�coul���be�that�so�e�in��ivi��uals�are��ore���istractible�an���therefore�are�less�li�el���to�enco��e�the�correct�infor�ation�into��or�in����e�or��.�This�can�be�teste���b���inclu��in���so�e�objects�that�are�to�be�i��nore���(e.��.,�re���bars)�an���others�that�are�to�be�atten��e���(e.��.,���reen�bars;�atten���to�their�orientations).�A���istract-ible�in��ivi��ual��oul���incorrectl���treat�the�re���bars�as�if�the����ere�relevant�an���therefore��oul���act�as�if�their��or�in����e�or����ere�heavil���loa��e��.�In�contrast,�a�less���istractible�in��ivi��ual��oul���exclu��e�the�re���bars�an���therefore��oul���act�as�if�their��or�in����e�or����as�not�so�heavil���loa��e��.�Vo��el�et�al.�(2005)�use���an�event-relate����otential��easure�of��or�in����e�or���loa���an���foun���that�in��ivi��-uals��ho�re�e�bere����ore�ite�s��ere�less���istractible.�McNab�an���Klin��ber���(2008)� ��u�licate��� this� result� in� an� fMRI� stu����,� sho�in��� in��ivi��ual� ��ifferences�in�subcortical�centers�involve���in�exclu��in���irrelevant�objects.�Of�course,�either�enco��in���of�infor�ation�into�an�unconscious�store�or�enco��in���of�infor�ation�into�a�conscious�store�coul���be�influence���b���the�abilit���to�filter�out�or�exclu��e�irrelevant� ite�s.� This� t���e� of� �echanis�� also� a��rees� �ell� �ith� the� notion� that�in��ivi��uals��ith�better�s�an�are�those��ho�are�better�able�to�exclu��e�or�inhibit�irrelevant�infor�ation�(Lusti���et�al.�2007).�

Page 27: Unconscious Memory Representations in Perception

1�� A�an��a�L.�Gilchrist�an���Nelson�Co�an

There�is�other�evi��ence,�thou��h,�of�another��echanis���hereb���in��ivi��uals���iffer�in�ca�acit��.�This��echanis��is�not��ell�un��erstoo���but�it�is�a���ifference�in�stora��e�abilit����er�se.�Gol���et�al.�(2006)�exa�ine���this� in�a�behavioral�stu�����of�in��ivi��uals��ith�schizo�hrenia�an���health���controls�usin���arra��s�of�objects�to�be�atten��e���an���i��nore��.�The���usuall����robe���objects�to�be�atten��e���but�occasion-all��� �robe��� objects� to� be� i��nore��.� The� su�� of� the� atten��e��� an��� i��nore��� ite�s�in��or�in����e�or�����ave�an�esti�ate�of� the�total�a�ount� in��or�in����e�or��,��hereas�the���ifference�bet�een�atten��e���an���i��nore���ite�s,�favorin���the�atten��e���ones,���ave�an�esti�ate�of�the�efficienc���of�the�attentional�filterin����rocess.�Sur�ris-in��l��,�it�turne���out�that�the���ifference�bet�een�the���rou�s��as�al�ost�entirel���in�the�total�a�ount�in��or�in����e�or��,��ith�ver���little���ifference�in�the�filterin���efficienc��.�Co�an�et�al.�(2010)�re�licate���this�fin��in���for���oun���chil��ren�(7�to�8���ears�ol��)�in�co��arison�to�ol��er�chil��ren�(12�to�13���ears�ol��)�or�a��ults,��hen�the�arra��s�inclu��e���2�relevant�ite�s�(e.��.,�circles;�atten���to�their�colors)�an���2�ir-relevant�ite�s�(e.��.,�trian��les).�Ho�ever,��hen�the�arra��s�inclu��e���3�relevant�an���3�irrelevant�ite�s,�the�lar��e�a��e���ifference�in�ca�acit���(as��easure���b���the�su��of��e�or���for�relevant�an���irrelevant�ite�s)��as�no��su��le�ente���b���a�s�aller�a��e���ifference�in�the�abilit���to�filter�out�or�exclu��e�irrelevant�ite�s�(as��easure���b���the���ifference�bet�een��e�or���for�relevant�versus�irrelevant�ite�s).�This�fin��in���su����ests�that�the�abilit���to�filter�out�or�exclu��e�irrelevant�ite�s�is�co��ro�ise����hen��or�in����e�or���is�overloa��e��.�That�is�to�be�ex�ecte���onl���if�the��or�in����e�or���store�in�question�is�a�central�store�that�is�attention-relate���in�itself,�as�is�the�focus�of�attention�as�a�stora��e���evice.�

So�e� controvers��� has� arisen� re��ar��in��� the� nature� of� �e�or��� in� the� Luc��an���Vo��el�(1997)�arra���tas��base���on�the��ro�erties�of�bin��in���bet�een���ifferent�features�of�objects.�An�exa��le� is� re�e�berin����hich� sha�e��ent��ith��hich�color,�an��/or��hich�color��as�foun���at��hich� location�in�the�arra��.�Our� initial�ex�ectation�a�fe����ears�a��o��as�that�intro��ucin���an�attentional���istraction��oul�����ifferentiall��� i��air� the� abilit��� to� re�e�ber� bin��in��� infor�ation� as� o��ose���to�feature�infor�ation.�That�is�because�the�focus�of�attention��as�vie�e���as�the�vehicle�for�the�retention�of�ne��bin��in��s�bet�een�features�(Co�an�1995,�1999,�2001).�The�activation�of�infor�ation�fro��lon��-ter���e�or���coul���not�inclu��e�ne��bin��in��s�bet�een� features,�as�one�fin��s��hen�ne��or��eanin��less�objects�are��resente��.�It�coul���inclu��e�the�infor�ation�that�an�a��le�is�re��,�for�exa��le,�but�not�the�infor�ation�that�an�abstract�circle�that�has�been��resente���is��ur�le.�Si�ilarl��,��erce�tual�theor����o�ular�in�the�fiel���ha���lon���hel���that�it�ta�es�atten-tion�to�for��bin��in��s�bet�een�features�(Treis�an�an���Gela��e�1980).�T�o�stu��ies,�ho�ever,�sho�e���that�this��as�not�the�case.�A���istractin���tas����urin���the�arra���retention�tas���as�just�as�har�ful�for�bin��in���infor�ation�as�it��as�for�the�feature�infor�ation�(Allen�et�al.�2006;�Co�an�et�al.�2006).�

Page 28: Unconscious Memory Representations in Perception

� Conscious�an���unconscious�as�ects�of��or�in����e�or��� 17

Does�this�fin��in���contra��ict�the�un��erstan��in���that�retention�of�infor�ation�in�this�tas��is�attention-relate��,�as�in�the�focus�of�attention?�Not�necessaril��.�One��a���to�un��erstan���this�fin��in���is�that�brin��in���an�ite��into�the�focus�of�attention�necessaril���entails�its�features�bein���boun��.�If�the�focus�of�attention��ere�the��ain�facult���in��hich�infor�ation�is�store���in�the�tas�,�intro��ucin�����istraction��oul���interfere��ith�stora��e�of�the�ite�s�an���therefore��oul���affect�both��e�or���for��e�or���for�features�an����e�or���for�the�bin��in���bet�een�these�features.�

1.2.3 Units�of��or�in����e�or���an���consciousness

Miller�(1956)�e��hasize���that��or�in����e�or�����e�en��s�on�the�abilit���to���rou��ite�s�to��ether�to�for���eanin��ful�chun�s.�The�letters�PRF��a���for��3�chun�s�if�the���for��no��no�n�acron���;�the�letters�USA��a���for��a�sin��le�chun��if�the�ac-ron����Unite���States�of�A�erica�is��no�n�to�the��artici�ant.�In��an����roce��ures,�the� abilit��� to� ��rou�� ite�s� to��ether� is� li�ite��� (because� the� ite�s� are� �resente���quic�l���an���un�re��ictabl��,�as�in�the�visual�arra���tas��or�in�runnin����e�or���s�an;�or�because�verbal�re�etition�of�a��or���or�short��hrase�to�eli�inate�verbal�rehears-al�is�require��)�an���the�usual�result�is�that�about�3�to�5�ite�s�are�recalle���b���a��ults�(Co�an�2001).�It�re�ains�unclear�fro��this�research,�thou��h,�ho����eneral�is�this�ans�er�re��ar��in���the�nu�ber�of�ite�s�that�can�be�hel���in�the�focus�of�attention.�

The�ite��li�it�a��ears�to�hol���even��hen�ite�s�are���rou�e���to��ether.�Chen�an��� Co�an� (2009b)� sho�� this� �ost� clearl��.� The��� tau��ht� �artici�ants� �airs� of��or��s�to�100%�correct�cue���recall,�an���also��resente���un�aire����or���sin��letons.�Thus,� there� �ere� 1-� an��� 2-�or��� chun�s.� Durin��� the� recall� of� lists� of� �or��s� of�various�len��ths�ran��in���fro��4�to�12��or��s,�the���sho�e���that�the�recall�li�it��as�narro�l�����efine���in�chun�s.�In��ivi��uals�coul���recall�about�3�sin��letons�fro��a�list�of�those�ite�s,�or�the���coul���recall�about�3��airs�fro��a�list�of�those�ite�s,�re��ar��less�of�the�list�len��th.�Thus�it���oes�not�a��ear�that�the�li�it�in�the�focus�of�attention�is�in�ho���an����or��s�can�be�hel��,�but�rather�in�ho���an����eanin��-ful�units�can�be�hel��.�Si�ilarl��,�Gilchrist�et�al.�(2008)��resente���lists�of�unrelate���s�o�en�sentences�an���foun���that���oun���a��ults�coul���recall�the��e����or��s�fro��about�3�sentences;�ol��er�a��ults�coul���recall�so�e�hat�fe�er.�It� is�not�necessar-il���the�case�that�ever����or���fro��a��no�n�chun��is�hel���in��or�in����e�or���at�the�sa�e�ti�e,�but�a��ar�er�of�the�unit�in�lon��-ter���e�or���is�hel���in��or�in����e�or��.�For�instance,���ou��robabl���can�re�e�ber�the�trio�of�son��s,�Three Blind Mice, The Star-Spangled Banner, an�� Taps�all�at�once�an�����ou��i��ht�be�able� to�hu��the��one�at�a�ti�e��ithout�a�re�in��er.�Ho�ever,�this���oes�not��ean�that�all�of�the��or��s�an���notes�fro��all�three�son��s�are���u��e���into�the�focus�of�attention�at�the�sa�e�ti�e.�The�focus�shifts�so�that�as���ou���et�to�each�son��,�the�ele�ents�of�the�son���are�shifte���into��or�in����e�or���as�nee��e��.

Page 29: Unconscious Memory Representations in Perception

1�� A�an��a�L.�Gilchrist�an���Nelson�Co�an

An�on��oin�����ebate�is��hether�the�resources�that�allo��infor�ation�to�be�hel���in� the� central� �art� of� �or�in��� �e�or��� or� the� focus� of� attention� is� ��iscrete� or�continuous.�A���iscrete�resource��oul���allo��u��to�a�certain�li�ite���nu�ber�of�chun�s�to�be�hel���(cf.�VanRullen�an���Koch�2003).�In�contrast,�a�continuous�re-source��oul���be��ore�flui���an���coul���be���ivi��e���u��a�on�����ifferent� ite�s� in�an����a���that��as���esire��.�This�is�actuall���a��uch��ore���ifficult�question�than�it��oul���see��at�first���lance.�The�fin��in���that��or�in����e�or��,��ithout�rehearsal�or���rou�in��,�is�usuall���li�ite���to�a�fe��ite�s�at�a�ti�e�coul���be�inter�rete���as�the�result� of� a� continuous� resource.� If� the� resource� �ere� s�rea��� unevenl��� over� the�ite�s�or�these�ite�s��ere�variabl���enco��e��,�it��oul���still�be�the�case�that�so�e�of�the���oul���sur�ass�a�threshol���for��e�or���an���others��oul���not.�A�cou�le�of� recent� stu��ies� have� use��� �roce��ures� in� �hich� the� �robe� ite�’s� ��iscre�anc���fro��the�corres�on��in���arra���ite��varies�(Ba��s�an���Husain�2008),�or�in��hich�the� �robe� has� to� be� recalle��� on� a� sli��in��� scale� an��� the� a�ount� of� ��iscre�anc���bet�een�the�sti�ulus�ite��an���the�res�onse�can�be��easure���(Zhan���an���Luc��2008).�These�stu��ies���iffer�in�their�inter�retation�an����ore�research�is�nee��e���to���eter�ine��ith�certaint����ho�is�ri��ht.�Mean�hile,�Rou��er�et�al.�(2008)�foun���a�tra��eoff�bet�een�chan��e���etection�an���no-chan��e���etection,���e�en��in���on�the�frequenc���of�each�t���e�of�trial�in�the�trial�bloc�,�that�is�linear�as�ex�ecte���accor��-in���to�the��o��el�in��hich�a���iscrete�nu�ber�of�ite�s�is�enco��e���into��or�in����e�or���(Co�an�2001).�

It�is��ossible�that���iscrete��or�in����e�or���is�a��ro��ra��able�stance�that�the��artici�ant�a��o�ts�rather�than�an�inevitable�qualit���of�the��in��.�Given�resources�insufficient�to�retain�all�ite�s�in�a�visual�arra��,��artici�ants��a���t���icall���choose�to�retain�a�fe��ite�s��ell.�It�is��no�n,�ho�ever,�that�in��ivi��uals�also�are�able�to�ju����e�the�statistical��ro�erties�of�a�lar��e�nu�ber�of�ite�s,�such�as�the�avera��e�size�of�circles�in�a�lar��e�fiel���(Chon���an���Treis�an�2005).�In�future,�it��ill�be�inter-estin���to�see��hether�attention�can�be�flui��l���s�rea���across�the�fiel���for�this�tas��or��hether�an�attention�li�ite���to�a�fe��ite�s�can�be�use���in�a��anner�co��lex�enou��h�to�ex�lain��or�in����e�or���for�statistical��ro�erties.

The� inter�la��� bet�een� conscious� an��� unconscious� ele�ents� of� �or�in����e�or����a�es�for�so�e�fascinatin����heno�ena.�A�on���the��is�inattentional�blin��ness� (Si�ons�an���Rensin��2005).�Even� thou��h� it�a��ears�as� if� the�visual�fiel��� before� ��ou� is� in� ��our� attention,� that� is� the� case� onl��� in� an� i��overishe����anner�an��,�actuall��,�onl���a�fe��ite�s�are�full���atten��e��.�That�is��h���it�is���if-ficult�to�reco��nize�a�chan��e�in�an�arra���of��ore�than�about�4�ite�s.�In�so�e�ex-�eri�ents,�a�realistic�scene�a��ears�an���then���isa��ears��o�entaril���an��,��hen�it�returns,�so�ethin���i��ortant�has�chan��e���(such�as�the�color�of�shirt�that�an�actor�is��earin���or�the��resence�of�a�sto��si��n�as�o��ose���to�a���iel���si��n�on�the�roa��).�Such�chan��es�are�ver���often��isse���an���it�see�s�that�onl���a�fe����etails�

Page 30: Unconscious Memory Representations in Perception

� Conscious�an���unconscious�as�ects�of��or�in����e�or��� 19

of�each�scene,�the�ones�in�the�focus�of�attention,�are�retaine����ell�enou��h�that�a�chan��e�can�be���etecte��.�If���ou�s�ill��asta�sauce�on���our�shirt�at�an�event,�feel�free�to�re�lace���our�shirt��ith�one�of�a���ifferent�color;�chances�are�s�all�that�an��one�at�the�event��ill�notice�the�chan��e.

Ar�e����ith�this�bo�����of�research,��e�no��return�to�a��ore�theoretical�treat-�ent� of� the� notion� of� conscious� an��� unconscious� �rocesses� an��� ho�� the��� are�involve���in��or�in����e�or��.�

1.3 Conscious and unconscious processing more generally considered

To�un��erstan���ho��conscious�an���unconscious�sti�uli��i��ht�o�erate��ithin�a��or�in����e�or���fra�e�or�,�it�is�i��ortant�to�un��erstan���ho��unconscious�an���conscious��rocesses��or�,�as��ell�as�ho��the����anifest�in�co��nitive�behavior.�In�the�current�section,��e�broa��l�����efine��hat�it�is�for�so�ethin���to�be�conscious�or,�conversel��,�unconscious.�We�follo��this��ith�e��irical�fin��in��s�that��rovi��e�su�-�ort�for�ho��conscious�an���unconscious�sti�uli�influence�behavior�in�co��nitive�tas�s.�We��resent�relevant�effects�that��rovi��e�su��ort�for���issociatin���conscious�an���unconscious�influences:��ri�in���an����erce�tion.�Unconscious�an���conscious��rocesses�o�erate�in�a�variet���of�co��nitive�tas�s�but�the�fin��in��s�fro��these�t�o�ar-eas�allo��for�a�clear���issociation�bet�een�conscious�an���unconscious��rocesses.�

Usin���a�broa�����efinition,�conscious�infor�ation�can�be�consi��ere���an���sti�-ulus,�either�externall��-�or�internall��-��enerate��,��hich��e�are�a�are�of�at�an�����iven�ti�e�–�thus,�these�ite�s�are�‘in��in��’.�In�contrast,�unconscious�sti�uli�are�those�ite�s��hich�are�currentl���not�in�a�areness,�an���have�no�re�ortabilit��.�Althou��h�one�is�t���icall���unable�to�classif���unconscious�infor�ation,��revious�stu��ies�fin���that�these�still�exert�effects�on�behavior,��articularl���on�in��irect�or�i��licit�tests�of� �e�or��.� Both� conscious� an��� unconscious� �echanis�s� �a��� be� involve��� in����ria��� co��nitive� tas�s,� o�eratin��� in� a� si�ilar� �anner� as� auto�atic� fa�iliar-it��� an��� controlle��� recollection,� as� ��escribe��� b��� �rocess� ��issociation� �o��els� of��e�or���(Jacob���1991).�The���e��ree�of�involve�ent�of�conscious�an���unconscious��rocessin���has�been���iscusse���for�various�co��nitive��heno�ena;�a�subset�of�these�is���iscusse���in���etail�belo�.�

1.3.1 Pri�in��

First,�consi��er�effects�of��ri�in��.�In�a��rotot���ical��roce��ure,��artici�ants�are��resente����ith�a��or���(e.��.,���octor).�This��or���is�often�consciousl����rocesse��,�thou��h�it��a���also�be��resente���belo��a�threshol���of�a�areness,�as� is���one�in�

Page 31: Unconscious Memory Representations in Perception

20� A�an��a�L.�Gilchrist�an���Nelson�Co�an

subli�inal��ri�in����roce��ures���iscusse���belo�.�Partici�ants�are�then��resente����ith�an�in��irect,�or�i��licit,��e�or���test;�this�often�involves��resentin���scra�-ble���ana��ra�s�or�inco��lete��or���ste�s��here�one��ust��rovi��e�a��er�itte����or���(e.��.,�n________).�Due�to�s�rea��in���se�antic�activation,�stron��l��-relate���associates�to�the��or���‘��octor’�also�receive�activation,�thus�increasin���their��rob-abilit���of�bein���re�orte���in�a��e�or���test.�In�the�exa��le��rovi��e��,�‘nurse’�is�a� ver��� stron��� associate� to� ‘��octor’;� �artici�ants� often� co��lete� the� �or��� ste���ith�this�or�other�stron��l��-relate���associates,�rather�than��ith�unassociate���(e.��.,�nei��hbor)�or��ea�l��-associate����or��s�(e.��.,�neurolo����).�A����itionall��,�res�onses�to� tar��ets�relate���to��ri�es�ten���to�be�si��nificantl��� faster.�Here,� the��resente����or��� is� clearl��� conscious;� ho�ever,� �ri�e��� associates� are� often� no� �ore� con-scious�than�an���other�ite��in�lon��-ter���e�or��.�Yet�so�eho��overt�res�onses�an���behavior�are�altere��.�Whereas�both�conscious�an���unconscious��echanis�s�contribute� to� the� ��eneral� �ri�in��� effect,� the� ��e��ree� of� involve�ent� of� each� is���ifficult�to���eter�ine.�Ho�ever,�it�is�clear�that�the�un��erl��in���o�erations�of�the��ri�in���effect�are�not�unifor�l���conscious�in�nature.�

Stu��ies�have�also�exa�ine���subli�inal��ri�in��.�As�briefl����entione���above,�this��articular��ara��i����involves�si�ilar��etho��olo�����to�a���eneral��ri�in���tas���ith�one�i��ortant���ifference�–��ri�es�are��resente���at�a�threshol���belo��con-scious�a�areness.�Often�ite�s�are��resente���so�briefl���that��artici�ants�re�ort�not�seein���an���ite�.�One�variation�of�this��resentation�involves��as�in���ite�s�sufficientl���fast�enou��h�that�the�initial� ite��is�not�re��istere���in�conscious��er-ce�tion�or� i��entification.�In�one�tas��use���b���More���et�al.�(2008),��artici�ants��ere� �resente��� �ith� a� �as�e��� ��i��it� �ri�e;� �artici�ants� rarel��� re�orte��� bein���able� to� ��etect� this� �ri�e.� Follo�in��� �resentation� of� a� belo�-threshol��� �ri�e,��artici�ants�are��resente����ith�a�tar��et���i��it,��hich�is�consciousl����erceive���an���i��entifie��.�Partici�ants�are�si��l���as�e����hether�the�tar��et���i��it�is�less�than�or���reater�than�a���iven�nu�ber�(e.��.,���reater�or�less�than�5).�What�is�critical�in�this��articular��ara��i����is�the�relationshi��bet�een�the��ri�e�an���the�tar��et���i��it.�Pri�es��a���either�be�con��ruent��ith�the�tar��et�(i.e.,�both��ri�e�an���tar��et�are�less�than�or���reater�than�5)�or�incon��ruent��ith�the�tar��et�(e.��.,��ri�e�is���reater�than�5,�tar��et�is�less�than�5).�For�this��articular��ara��i���,��ri�in���effects�are��anifest�in�res�onse�ti�e�to�verif��in���the�tar��et���i��it,�rather�than��rovi��in���an�overt�res�onse.�S�ecificall��,� the�ti�e�nee��e���to�res�on���to�the�tar��et� is� lon��er��hen�the��ri�e�is�incon��ruent.�Des�ite�the��ri�e�bein���irrelevant�to��erfor�-in���the�tas�s,�as�res�onses�are�onl���relate���to�the�tar��et�itself,�so�e�unconscious�re��istration� of� the� ‘unseen’���i��it�occurs,��hich�affects� tas���erfor�ance.�This�fin��in���is�base���on�earlier�stu��ies,�such�as�that�of�Marcel�(1983;�Ex�eri�ent�4),��ho�foun���a�si�ilar�subli�inal�associative��ri�in���effect�in�a�lexical���ecision�tas�.�Ite�s�that��ere��resente���at�a�subjective���etection�threshol���level,�either�

Page 32: Unconscious Memory Representations in Perception

� Conscious�an���unconscious�as�ects�of��or�in����e�or��� 21

un�as�e���or��attern-�as�e���to�both�e��es,�influence���res�on��in���to�associate����or��s,��ith�faster�res�on��in���to�tar��ets.�More���et�al.�(2008),�ho�ever,��rovi��e�a�statistical�ar��u�ent�that�our�un��erstan��in���of�the�subli�inal��ri�in����ara��i����is�insufficient���ue�to�an�incorrect�assu��tion�of�the�null�h���othesis.�

Even� if� subli�inal� �ri�in��� ��oes� exist,� there� are� li�its� to� ho�� it� o�erates.�Conscious�sti�uli�influence�unconscious�behavior�in�classic��ri�in����roce��ures.�Can�it�be�sai���that�unconscious��ri�es�influence�conscious�behavior?�Not�reall��;�accor��in��� to� one� stu����� at� least,� it� onl��� influences� other� unconscious� infor�a-tion.�S�ecificall��,�Balota�(1983)�foun���se�antic��ri�in���fro��both��ri�es�that��ere�in�a�areness�an����ri�es�that��ere�outsi��e�of�a�areness�(i.e.,�i��erce�tible).�The� t�o��in��s�of��ri�es���iffere��,�ho�ever,��hen� it� ca�e� to� influencin�����irect��e�or���tests.�Recollection�of�havin���receive���a�tar��et��as���reater��hen�it��as��aire����ith�a�relate����or���that�ha���been��resente���as�a�su�rali�inal��ri�e,�co�-�are���to�a�relate����or���that�ha���not�been��resente��.�In�contrast,��hen�a�tar��et��as� �resente��� for� recollection� �ith� a� relate��� �or��� that� ha��� been� �resente��� as�an� unconscious� �ri�e,� it� ha��� no� beneficial� effect� on� recollection� co��are��� to��hen�it��as��aire����ith�a���ifferent�relate����or��.�In�the�ter�s�of�the�e�be����e����rocesses��o��el�of��or�in����e�or���(Co�an�1988,�1999)�it�a��ears�that�there�is�such�a�thin���as�auto�atic�se�antic�activation�but�that�this�t���e�of�activation���oes�not�influence��hat�infor�ation�reaches�the�focus�of�attention.�

The�unconscious�activation�still�a��ears�to���e�en���to�so�e�extent�on�atten-tion�to�the�sti�uli�at�the�ti�e�of�their��resentation.�Woo���et�al.�(1997)�carrie���out�a��roce��ure�in��hich�s�ecial��or����airs��ere��resente���in�the�channel�to�be�i��-nore���in�selective�listenin��.�An�exa��le�is�the��or����air�taxi-fare.�The�first��art�of�the��or����air���isa�bi��uates�the�secon����art.�When��artici�ants�later��ere�as�e���to�s�ell�a��or���that�ha���t�o��ossible�s�ellin��s�(e.��.,�fare�versus�fair),�those��ho�ha���receive���the�corres�on��in�����isa�bi��uatin����or����air��ore�often�s�elle���the��or���in�a��anner�consistent��ith�that��air.�Woo���et�al.�foun��,�thou��h,�that�this�effect�occurre���onl����hen�the�atten��e���channel��as��resente���at�a�ver���slo��rate.�When� it��as��resente���at� a� rate��ore� t���ical�of� selective� listenin����roce��ures,�this� so�histicate���version�of�unconscious�se�antic�activation���isa��eare��.�A�-�arentl��,�the�slo���resentation�allo�e���attention�to��an��er�on�to�the��essa��e�that��as�su��ose���to�be�i��nore��,��hich�containe���the���isa�bi��uatin����or����airs.

1.3.2 Perce�tion�

In� ��eneral,� �uch� of� �hat� �e� �erceive� requires� so�e� sort� of� initial� conscious�re��istration,��hich�is�auto�atic�in�nature�an���a�in�to�the�attentional�orientin�����escribe���in�the�e�be����e��-�rocess��o��el�(Co�an�1988,�1999).�An���conscious�

Page 33: Unconscious Memory Representations in Perception

22� A�an��a�L.�Gilchrist�an���Nelson�Co�an

�erce�tion�of�environ�ental�sti�uli�that�follo�s�this�re��istration�is��o��ulate���b���controlle��,�selective�attention.�With�this�in��in��,�is�it��ossible�that��erce�-tion��a���occur�belo��a�conscious�threshol��?�Whereas�fin��in��s�fro��subli�inal��ri�in��� �a�e� a� sli��htl��� stron��er� case� for� unconscious� activation� in� �e�or��,�researchers�are���ivi��e���as�to��hether��erce�tion�can�occur��ithout�a�areness.�Objects�that�coul���be���escribe���as�subli�inal�coul���alternativel���be���escribe���as��ea�l���conscious.�

One� can� test� a�areness� of� sti�uli� b��� �resentin��� �erce�ts� at� a� subjective�threshol���(i.e.,�base���off�of��hat��artici�ants�re�ort)�or�b���settin��� threshol���at�an�objective�value�(e.��.,���’,�a�co��on���etection��ara�eter,�set�to�0).�Prior�stu��ies��rovi��e�su��ort�for�unconscious�or�subli�inal�influences�u�on��hat��e��erceive.�Sno����rass�an���Shevrin�(2006)��resente����artici�ants��ith�a�series�of�un�as�e����or��s�briefl���flashe���on�a�co��uter�screen,��hich�the����ere�later�as�e���to�i��en-tif���in�a�force��-choice�tas�.�As�ite�s��ere��resente���onscreen,��artici�ants��ere�instructe���to�follo��one�of�t�o�strate��ies:�‘loo�’,��here�instructions�in��icate���to�atte��t�to�intentionall���i��entif���a��or��,�or�‘�o�’,��here��artici�ants��ere�to��en-tion� the� first� �or��� that� a��eare��� in� �in��.� Partici�ants� �ere� later� as�e��� about��hich� strate����� �as� �referre��.� Initial� tests� sho�e��� a� strate����� con��ruence� effect�on� �erce�tion� –� �artici�ant� �erfor�ance� �as� t���icall��� above� chance� for� their��referre���strate����,��hereas��erfor�ance�on�the�non�referre���strate������as�belo��chance�(see�Sno����rass�et�al.�2004�for�an�in-��e�th���iscussion).�Interestin��l��,���ran����eans�for�all�con��itions�centere���aroun���chance�levels.�

Accor��in���to�Sno����rass�an���Shevrin�(2006),�if�subli�inal��erce�tion�is�si�-�l����ea��conscious��erce�tion,�this� i��lies�a�hierarch����here�hi��her-or��er�re-s�onses�are�not��ossible��ithout�un��erl��in��,�lo�er-or��er�res�onses;�thus,�there�is�a���irect�relationshi��bet�een�all�res�onses.�If�this�is�so,�correct�i��entification�cannot�occur�if�the�abilit���to���etect�a�sti�ulus�is�at�chance.�Yet,�the�results�sho��that���es�ite�objective�belo�-threshol�����etection�levels�(��’�at�chance),�a��referre���strate�����facilitate����erfor�ance,��ith�i��entification�above�chance.�

Stu��ies�b���Marcel�(1983)��rovi��e�further�evi��ence�a��ainst�this�hierarchical��rocessin���structure.�Here,��as�in����erce�tual�sti�uli�at�a�subjective� thresh-ol���level��as�less�li�el���to�affect�se�antic��rocessin��,�or�even���ra�hic��rocess-in��,�than�ite����etection.�Decreasin���the�ti�e�bet�een�sti�ulus�an����as��le���to�chance��erfor�ance�on���etection�first,�follo�e���b�����ra�hic�an���se�antic��ro-cessin���res�ectivel���(Ex�eri�ent�1).�A����itionall��,�re�etitions�of�a��as�e����or���le��� to� ��reater� associative� �ri�in��� in� a� lexical� ��ecision� tas�,� but� ha��� no� effect�u�on�an�ite�’s��robabilit���of�bein�����etecte���(Ex�eri�ent�5).�These�results�su��-��est�that��erfor�ance�on�these�t���es�of�tas�s� is�not�onl�����ue�to�conscious�as-�ects�of��erce�tion,�an���that�unconscious�as�ects�of��erce�tion�are�actually un-conscious.�These�results��ere�fra�e���in�ter�s�of�unconscious�attributions�fro��

Page 34: Unconscious Memory Representations in Perception

� Conscious�an���unconscious�as�ects�of��or�in����e�or��� 23

�artici�ants.�A�brief�flash�of�a���iven��or���increases�its�activation,�re��ar��less�of��hether�it�is�consciousl����erceive��;��hether�the�activation�is�use���to�correctl���i��entif���a�sti�ulus���e�en��s�entirel���on��hat�the�current�an����referre���strate�����instructions�are.�If�a��referre���strate�����is�instructe��,�the�activation�increase�in-fluences�correct�res�on��in���(Sno����rass�an���Shevrin�2006).�

In� the�e�be����e��-�rocesses��o��el,� these�effects�coul���be�vie�e���as�conse-quences� of� conscious� �rocesses� in� �re�arin��� the� fiel��� of� activate��� �e�or��� for��erce�tion.�The�central�executive’s�shiftin���of�the�focus�of�attention�leaves�in�its��a�e�an�activation�fiel���an���the��a���in��hich�it�is�set�u��can��a�e�a���ifference�for��erce�tion.�Different�in��ivi��uals�a��arentl���have���ifferent��referre����etho��s�of��re�arin���the�activation�fiel���for��erce�tion.�This�is�an�i��ortant�area�for�future�research.

1.4 Theories of consciousness and unconscious processing

There�are�various� theories�an����o��els� that���iscuss�ho��conscious�an���uncon-scious��rocesses�o�erate�in�hu�an�co��nition.�Given�s�ace�constraints��e���iscuss�those�theories�of�consciousness�that�are��ost�a��licable�to�the��o��el�of��or�-in����e�or���that��e��ro�ose,��ith�e��hases�on��ertinent�behavioral�an���neu-roscientific�fin��in��s.�We���iscuss�Global�Wor�s�ace�Theor���(GWT),�a�theoretical�fra�e�or��of�consciousness�that�has��otential�to�ex�lain�both�co��nitive�an���neu-roscientific��heno�ena�relatin���to�conscious��rocessin��,�an���a�taxono����of�con-sciousness��ro�ose���b���Dehaene�et�al.�(2006).�This�latter�fra�e�or�,�as��e�note�in� follo�in��� sections,�fits��ell��ithin� the��ara�eters�of� the� e�be����e��-�rocess��o��el.� Relevant� conce�tions� of� conscious� �rocessin��� �ill� be� a����resse��� belo�;�this��ill�follo���ith�a�h���othesize����o��el�of�conscious�a�areness�an����rocessin���in��or�in����e�or��.�

1.4.1 Global��or�s�ace�theor��

GWT,��ro�ose���b���Baars�an���collea��ues�(Baars�et�al.�2003;�Baars�an���Fran�lin�2003,�2007),�is�quite��ossibl���the��ost�relevant��o��el�of�conscious��rocessin���as�it��ertains�to��or�in����e�or��.�Accor��in���to�this��o��el,�there�is�a�bi��irectional�infor�ation�flo�,��ith�conscious��rocesses�influencin���unconscious��rocesses,�an���vice�versa.�The��ri�in���effects���iscusse���above,��here�conscious�infor�a-tion� influences�unconscious� infor�ation� (an���vice�versa),��rovi��e� su��ort� for�this.�Within�the��or�s�ace��ro�ose���b���Baars,�sti�uli�that�are�unconscious�are��rocesse��� on� a� local� scale,� �ith� activation� restricte��� to� areas� ��evote��� to� that�

Page 35: Unconscious Memory Representations in Perception

24� A�an��a�L.�Gilchrist�an���Nelson�Co�an

�articular��o��alit��.�For�exa��le,�consi��er�a�soun����a��e��hile���ou�are�slee�-in��.� Most� li�el��,� unless� ��ou� are� in� the� li��htest� sta��es� of� slee�,� this� shoul��� not���isturb���ou.�In�contrast,�the�brain�is�able�to��rocess�this�soun��,���es�ite�the�lo��level�of�arousal.�Ho�ever,�this��rocessin����ill�onl���be��resent�in�neural�re��ions���irectl���res�onsible�for�hearin���–��ri�ar���areas�in�the�te��oral�lobes,�exten��in���no�farther�(see�Rees�et�al.�2002,�for�a�revie��of�si�ilar�fin��in��s�re��ar��in���vision).�Si�ilar� results� �rovi��in��� su��ort� for� local� �rocessin��� of� unconscious� sti�uli�have�also�been�foun���for�co�atose��atients�(Kotchoube���et�al.�2002),�as��ell�as�those��ith�blin��si��ht�(Co�e���an���Stoeri���1997).�

When�arousal�levels�are�hi��her,�an����e�have�the�ca�acit���to�be�conscious�of�internal�an���external�sti�uli�in�our�environ�ent,�the�GWT��re��icts�a���ifferent��anner�of��rocessin��.�In�contrast�to�local��rocessin���of�unconscious�sti�uli,�con-scious�sti�uli�evo�e�sufficient�activation�that�not�onl���affects�local�sensor���areas,�but�also�lea��s�to�overall���lobal�activation,��articularl���for�frontal�an����arietal�re-��ions�(e.��.,�Seth�et�al.�2005;�Vuille�ier�et�al.�2001).�The��rocessin���of�conscious�infor�ation�allo�s�for�the�conver��ence�of�various�sensor���s��ste�s�an����rocess-in���net�or�s�that��oul���other�ise�o�erate�in��e�en��entl��;�this�allo�s�s�ecialize���net�or�s�to�access�other�net�or�s,�coor��inate�co��nitive�o�erations,�an���co�bine�various�in�uts� into�a�boun����hole.�Thus,�conscious�in�uts�allo��this��ro�ose����or�s�ace�to�function.�

Accor��in���to�this�theor��,�onl���one�conscious�in�ut�can���o�inate�attention�an���co��nitive��rocessin���in��or�in����e�or���at�an�����iven�ti�e;�Baars�notes�that�this�is�analo��ous�to�a�s�otli��ht�focuse���on�a�re��ion�of�theater�sta��e�(Baars�et�al.�2003).�The�‘s�otli��ht’�of�consciousness�is�focuse���an�����irecte���both�b���attention�an���executive�control.�Follo�in���Baars’s�analo����,�the�re�ain��er�of�the�sta��e�an���the�theater�is�consi��erabl�����ar�er;�thus,�as�attention�an���executive�control�are���irecte��� to�ar��s�one��articular� ite�� in�consciousness,�other� ite�s��resent� in��or�in����e�or���or�in�lon��-ter���e�or���are�consequentl���unconscious.�So�e�unconscious�ite�s�can�exert�effects�on�conscious��rocessin���–�these�are��no�n�as� contexts.� For� exa��le,� structures� in� the� �arietal� lobe,� t���icall��� critical� for�reco��nizin���object��otion�or� location,�are�not�critical� for�reco��nizin���objects;�ho�ever,�these�re��ions�have�s�atiall��-oriente����a�s�that�can�sha�e�one’s��erce�-tual�ex�erience�of�an�object.�In�no�case�is�this��a��e�clearer�than�in�that�of�con-tralateral�ne��lect:��ersons��ith���a�a��e�to�re��ions�of��arietal�cortex�ex�erience�the�loss�of�half�of�their�conscious�visual�fiel��.�Althou��h�the��arietal�cortex�is�not�critical� for� object� i��entification,� it� obviousl��� �la��s� a� crucial� role� in� one’s� con-scious�visual�ex�erience,�as�ne��lect�sho�s.�Via�attention,�executive��rocessin��,�an���various�contexts,�conscious�content�is�broa��cast�to�the�theater�‘au��ience’.�In�ter�s�of�neural�activation,�correlates�of�this�one-ite��s�otli��ht�t���icall���involve�association�areas�of�a�s�ecific�sensor����o��alit��.�Association�areas�contain�fiber�

Page 36: Unconscious Memory Representations in Perception

� Conscious�an���unconscious�as�ects�of��or�in����e�or��� 25

�rojections�to�various�re��ions�of�cortex,�es�eciall���fronto�arietal�re��ions;�thus,�this�can�contribute�to���lobal�activation�of�conscious�in�ut.�Other�sensor���re��ions�are�not�activate��;�it�is�li�el���that�activation�fro��conscious�in�uts��utuall���in-hibits�co��etin���sensor���in�uts,��ee�in����rocessin���for�these�ite�s��ithin�their�res�ective�re��ions.�

This��o��el�fits��ell��ith�activation-base����o��els�of��or�in����e�or��.�One�can�see�fro��the�e��hasis�on�a�one-ite��s�otli��ht�of�consciousness�that�Oberau-er’s��o��el�is�the��ost�si�ilar,���ue�to�his��ro�ose���one-ite��focus�of�attention.�As��reviousl����entione���above,�ho�ever,�the�e�be����e��-�rocess��o��el��ro�ose���b���Co�an�is�not�necessaril���rule���out,�as�ite�s��ithin�a�four-ite��focus�of�attention��a���receive��rocessin���on�the�basis�of��riorit��;�it�is�entirel����ossible�that�one�at-tention-��e�an��in���ite��coul���receive�a�consi��erable�share�of�available�resources.�These��o��els�treat�the�contents�of��e�or���as�a��or�s�ace,��here�ite�s��a���be�activate��,�accesse��,�atten��e��,�chun�e��,�an���so�on.�A����itionall��,�li�e�these��or�-in����e�or����o��els,�GWT�hi��hli��hts�the�role�of���irecte���attention;�ite�s��ust�receive�attention�to�be�conscious.�

Des�ite�the�si�ilarities�bet�een�the�t�o��or�in����e�or����o��els�an���GWT,��hich�is�a�si��nificant�stren��th,�this��o��el�of�consciousness�has�one��ea�ness�of�interest.� Here,� consciousness� is� assu�e��� to� be� ��ichoto�ous� –� ite�s� are� either�conscious�an���receive�a�s�otli��ht�of�attention,�or�the���re�ain�unconscious.�This���oes�not�a��ear�to�follo��fro��Baars’s�theater�analo����.�Li��ht�not�onl���shines�on�a��articular�object,�but�it�also�ra��iates�out�to�ar��s�other�ite�s�at�a���i�inishin���rate.�Thus,�shinin���an�attentional�s�otli��ht�on�a���iven�ite��not�onl���hel�s�one�ite��re�ain�in�conscious�a�areness,�but�also�‘illu�inates’�ite�s�in�close��roxi�it���to�a�lesser���e��ree.�Proxi�al�ite�s�are�not�co��letel���hi����en�in���ar�ness,���et�are�not�as�visible�as�an�ite��in�the�s�otli��ht;�rather,�the���resi��e�so�e�here�in�the��i����le�of�these�t�o�extre�es.�Follo�in���this�analo����,��e�assu�e�that�this��i����le�area�has�a��i��e�ran��e�of�variance�–�an�ite���a���be�barel���visible,�or��a���be�relativel���noticeable,���e�en��in���on��an���con��itions:�ho��lar��e�the�s�otli��ht�is,�ho���uch�li��ht�so�ethin���on�sta��e�receives,�the���i�ensions�of�the�theater,�an���so�on.�Fro��this�i��ea,�it�follo�s�that��hile�ite�s��roxi�al�to�an�atten��e���ite��are�clearl���not�conscious,�these�ite�s�are�not�necessaril���unconscious�either.�Instea��,�conscious-ness�is��ore�continuous,��ith�these�ite�s�existin���on�so�e��i����le��lane�bet�een�t�o� extre�es;� ho�� close� a� ��iven� ite�� is� to� one� of� these� extre�es� ��e�en��s� on��an��� factors,� such� as� ho�� �uch� attention� a� conscious� ite�� receives,� �hether�attention�is���ivi��e��,�an���efficac���in�executive�control.�An�alternative�taxono����of�consciousness�has�been���escribe���b���Dehaene�et�al.�(2006),�an���is���iscusse���belo�,�as�this��ill�have�si��nificant�i��lications�for�ho��consciousness�relates�to�our��ro�ose����o��el�of��or�in����e�or��.�

Page 37: Unconscious Memory Representations in Perception

2�� A�an��a�L.�Gilchrist�an���Nelson�Co�an

1.4.2 Dehaene’s�taxono����of�conscious��rocessin��

As��e��entione���above,��e���o�not�necessaril���a��vocate�the�i��ea�that�conscious-ness�is�all-or-none.�Rather,�consciousness�coul���exist�on�a�continuu�,��ith�so�e�ite�s�bein���conscious,�so�e�bein���unconscious,�an����an���fallin���in�the��i����le�of�these�t�o�extre�es.�The�taxono�����ro�ose���b���Dehaene�et�al.�(2006)�i��lies�a�tri�artite���ivision�of�conscious��rocessin��,��here�sti�uli�are�classifie���on�the�in-teractions�of�botto�-u���rocesses,�t���icall����erce�tion,�an���to�-��o�n��rocesses,�such�as�attention.�

Unconscious�infor�ation�an����rocessin���are�‘subliminal’�–�here,�botto�-u���erce�tual�influences�are�not�stron���enou��h�to��rovi��e�sufficient�activation�that�coul���allo��an�ite��the��otential�to�enter�consciousness.�These�ite�s�are�co�-�letel��� inaccessible,� an��� have� li�ite��� neural� activation� be��on��� localize��� areas�that��rocess�a���iven��erce�tual��o��alit��.�Interestin��l��,�this�taxono����allo�s�sub-li�inal�infor�ation�to�receive�attention,�as�this�is��resu�e���to�have��o��ulator���effects�on��rocessin��.�For�exa��le,�allocatin���attention�to�a��ri�e-tar��et��air�in�a�subli�inal��ri�in����ara��i�����ro��uces�the���esire����ri�in���effect;��hen�that�attention�is�unable�to�be�allocate��,�the�effect���isa��ears.�Si�ilar�results�have�been�foun���in��atients��ith�blin��si��ht�b����resentin���a�conscious�cue�for�a�tar��et�in�the�blin���visual�fiel��.�Althou��h�ite�s�still�re�ain�unseen�to��artici�ants,�allocatin���attention�in�the�absence�of�stron����erce�tual��rocessin���a��lifies�subli�inal��ro-cessin���effects;�if�these�ite�s�cannot�be�atten��e��,�these�effects��ill�be�less�li�el���to�occur.�

In�contrast�to�subli�inal��rocesses,��hich�have�insufficient�botto�-u��sti�u-lus� stren��th,� preconscious �rocesses� have� enou��h� �erce�tual� sti�ulus� stren��th,�an��� sufficient� neural� �rocessin��,� to� beco�e� activate��.� But� these� ite�s� are� not�in�conscious�a�areness,�as�there�is�no�to�-��o�n�allocation�of�attention.�Unli�e�subli�inal�sti�uli,��hich�are�not�accessible�at�all,��reconscious�sti�uli�have�the��otential� for� conscious� access,� �rovi��e��� that� attention� is� ��ra�n� to�ar��s� these�ite�s.�In�ter�s�of�neural��rocessin��,��reconscious�activation�is�not�as�localize���as��rocessin���for�subli�inal�sti�uli.�Dehaene�et�al.� (2006)��ro�ose�that�neural�activation��ill�s�rea���be��on���localize����rocessin���areas,�into��ulti�le�sensor��-�otor�re��ions;�ho�ever,�the�lac��of�to�-��o�n��rocessin����ee�s�neural�activation�fro��beco�in����ore���lobal�b���exten��in���to�fronto�arietal�re��ions,�a�hall�ar��of��rocessin���ite�s�in�conscious�a�areness.�

As�su����este��,�necessar���con��itions�for�conscious�a�areness�inclu��e�sufficient��erce�tual�stren��th�an���sufficient�allocation�of�attention.�Ite�s�that��eet�these�criteria� are� consi��ere��� conscious� in� the� �ro�ose��� taxono���;� ��lobal� neural� ac-tivation�occurs�for�these�ite�s�via�activation�of�critical�frontal�an����arietal�lobe�re��ions.�The�conce�ts��ro�ose���here�illustrate�the�various�connections�bet�een�

Page 38: Unconscious Memory Representations in Perception

� Conscious�an���unconscious�as�ects�of��or�in����e�or��� 27

the�stren��th�of�a�sti�ulus,�attention,�an���consciousness,�an���i��ose�a�three-�art���istinction� of� conscious� �rocessin��� instea��� of� a� ��ichoto���.� This� �ill� have� i�-�ortant� i��lications� for� ho�� conscious� �rocessin��� occurs,� as� ��escribe��� b��� the�e�be����e��-�rocess��o��el�of��or�in����e�or��.

1.5 The embedded-process model and consciousness reconsidered

With�a� stron��er�un��erstan��in���of��hat� constitutes� conscious�an���unconscious��rocessin��,� as� �ell� as� ho�� these� �a��� o�erate� �ithin� co��nitive� tas�s� an��� theo-retical�fra�e�or�s,��e�use�the�current�section�to��ro�ose�ho��conscious�an���un-conscious��rocessin����a���o�erate��ithin�the�e�be����e��-�rocess��o��el�outline���above.� As� �as� �entione��� �reviousl��,� this� �o��el� of� �or�in��� �e�or��� contains�three�levels�of�nestin���for�ite�s�containe����ithin��e�or��.�These�levels�of�nest-in���require�a�fra�e�or��of�consciousness�that�is�not���ichotic�–�for�this�reason,��e�fra�e��rocessin���of�infor�ation�in��or�in����e�or���in�ter�s�of�the�taxon-o�����ro�ose���b���Dehaene�et�al.� (2006).�Belo�,��e��ill���iscuss�conscious,��re-conscious,�an���unconscious��rocessin��,�as�the����ertain�to�an�e�be����e��-�rocess��o��el�of��or�in����e�or��.

1.5.1 Wor�in����e�or���an���conscious��rocessin��

The�e�be����e��-�rocess��o��el���escribe����reviousl����rovi��es�a��vanta��es�in�clari-f��in���our�un��erstan��in���of�conscious�an���unconscious��rocessin���over�structural��or�in����e�or����o��els.�This�is���ue�in��art�to�an�e��hasis�on�ho��ite�s�are��rocesse��,�rather�than�the�co��risin���subco��onents�of��or�in����e�or��.�In�the��o��el��e��ro�ose,�ite�s�are��ore�li�el���to�enter�into�conscious�a�areness��ith�increasin���activation.�Thus,��rovi��e���that�current�tas����oals�re�ain�unchan��e��,�ite�s��resent��ithin�activate��� lon��-ter���e�or���have�a�hi��her��robabilit���of�enterin���the�focus�of�attention�than���o�lon��-ter���e�or���re�resentations�that�are� not� activate��.� On� the� other� han��,� if� these� ��oals� are� ����na�ic,� activation� of�contents� �ill� also� chan��e� ra�i��l��� as� ne�� ite�s� beco�e� tas�-relevant;� an� ite�’s�current�level�of�activation��ill�no�lon��er�serve�as�a�reliable�in��ex�for�its��otential�to�enter�consciousness.�For�infor�ation�in��e�or���that�is�alrea�����activate��,�the���eci��in���factor�in��hether�an�ite��beco�es�conscious�or�not�is�allocation�of�at-tention.�In�the�current��o��el,�the�focus�of�attention�is�consi��ere���an�analo��ue�of�consciousness��ro�er�–�ite�s�here�have�both�enhance���activation�an���a�areness,�ensurin���their�re�ortabilit���in�co��nitive�tas�s.�In��ee��,�it�is�a�rare�event�that�ite�s��resent��ithin�the�focus�are�for��otten�or�are�incorrectl���recalle��.�These�ite�s�are�

Page 39: Unconscious Memory Representations in Perception

2�� A�an��a�L.�Gilchrist�an���Nelson�Co�an

not� li�ite���b��� ti�e;�as� lon���as�attention� is��aintaine��,� these� ite�s��ill�not�be�lost�or���eactivate��.�Ho�ever,�as��entione����reviousl��,�the�focus�of�attention�onl���hol��s�aroun���four�ite�s�at�an�����iven�ti�e.�

1.5.2 Preconscious��rocessin��

If�conscious�a�areness�fin��s�its�analo��ue��ithin�the�contents�of�the�focus�of�atten-tion,�this��o��el���escribes�unconscious��rocessin���throu��h�its�treat�ent�of�lon��-ter���e�or���contents.�We�ex�ect�that��rocessin���of�lon��-ter��contents���iffer,�as�activate���lon��-ter��contents�have�a���reater�li�elihoo���of�access�to�conscious�a�areness�un��er�unchan��in���tas��con��itions.�Activate���ite�s�that�are�not��res-ent�in�the�focus�of�attention�fit��ell��ith�a���escri�tion�of�infor�ation�that�un��er-��oes�“�reconscious”��rocessin���that�Dehaene�et�al.�(2006)���iscusse��.�Ite�s�that�un��er��o�this�sort�of��rocessin���have�sufficient�neural�activation�to�be��resent�in�consciousness,�but�are�buffere���into�unconscious�stores�because�of�a�lac��of�to�-��o�n�attentional�allocation�that�is�si�ilar�to�the�effortful���irectin���of�attention�via�the�central�executive.�This�i��lies�that�these�ite�s�have�the��otential�for�enter-in���conscious�a�areness,��rovi��e���that�the���are���iven�sufficient�attention.�Unli�e�ite�s�in�the�focus�of�attention,�these�ite�s��a���be�subject�to�ti�e-base�����eca���an���interference,��hile�their�ca�acit���is��resu�e���to�be�li�itless.�Such�li�its�see��in-tuitive.�Preconscious�contents�have�stron����erce�tual�sti�ulus�stren��th,�but,���ue�to�a�ra�i��l���chan��in���environ�ent,�such�activation�shoul�����ecline��ith�ti�e�(e.��.,�S�erlin���1960).�Attention�not�onl���causes�these�ite�s�to�enter�consciousness,�but�also�sto�s�the�ti�e-base�����eca���of��erce�tual�sti�uli�in�uts.�

Des�ite� an� inabilit��� to� enter� conscious� a�areness,� ite�s� in� activate��� lon��-ter���e�or���are�still��rocesse���in�a��a���that��e��i��ht�call�“behin���the�scenes”�or�“in�the�bac���roun��”.�Thus,�these��rocesses�still�exert�influence�over�conscious��rocessin���in��or�in����e�or����ithout�havin���the�abilit���to�be�re�orte��.�For�ex-a��le,�Hassin�(2005)���escribes�a�stu�����in��hich��artici�ants�are�sho�n��atrices�of�e��t���or�fille�����is�s.�Dis�s��ere��resente���in�a�sequential�fashion�in�sets�of�five;��artici�ants��ere�to�recall�each�sequence�as�a�test�of��or�in����e�or��.�A�critical��ani�ulation�involve���i��licit�rules�that�sets�of���is�s�coul���follo��or�fail�to�fol-lo�.�In�a�rule set,�locations�an���qualitative�as�ects�of���is�s�(i.e.,�fille���vs.�unfille��)�follo�e���an�i��licit�rule.�Control sets�involve���ran��o���resentations�of���is�s�an���locations.�The�final�sets��ere��no�n�as�broken rule sets.�Here,�the�first�four�ite�s�of�a���is��sequence�follo�e���an�i��licit�rule,�si�ilar�to�rule�sets;�ho�ever,�the�final�ite���ithin�a�set���i���not�follo��the�inferre���rule.�Hassin’s�rationale�for�the�final��resentation�t���e��as�as�follo�s:�correct��erfor�ance�on�rule�sets�involves�an�i�-�licit�extraction�of�the�un��erl��in���rule.�As�ite�s�are�bein����resente���sequentiall���

Page 40: Unconscious Memory Representations in Perception

� Conscious�an���unconscious�as�ects�of��or�in����e�or��� 29

at�a�ra�i���s�ee��,�the�rule�is�hi��hl���unli�el���to�be�consciousl���un��erstoo���-�in��ee��,�all� �artici�ants� exce�t� one� �ere� able� to� reconstruct� a� �resente��� sequence� rule�in�a�follo�in���ex�eri�ent.�If�such�a�rule�is�extracte���i��licitl��,�the�final�ite��in�a�bro�en�rule�set�shoul���be�incorrectl���recalle���an���shoul���invo�e�a�slo�er�re-s�onse;�s�ecificall��,��artici�ants�shoul���recall�the�final�ite��in�a��a���that�follo�s�the�un��erl��in���rule.�Thus,�s��ste�atic�error��atterns�shoul���result.�Hassin�foun���that�reaction�ti�es�for�final�ite�s��ere�si��nificantl���slo�er�for�bro�en�rule�sets�than�for�rule�or�control�sets.�A����itionall��,�rule�sets�ha���faster�res�onse�ti�es�than�control�sets,��rovi��in���su��ort�for�unconscious�rule�extraction.�

In�the�above�exa��le,�the�conscious��ortion�of�the�tas�,��an���of�the�ite�s��resent�in�the�sequence,�is�store���an����aintaine���in�the�focus�of�attention.�Due�to� ite�� li�its� in� the� focus,� �e� ex�ect� that� so�e� ite�s� �a��� also� be� �resent� in�activate��� lon��-ter���e�or��,� lin�e��� to�currentl��-atten��e��� ite�s.�An���un��erl��-in���rules�are�not�critical�for���oo���tas���erfor�ance;�one�onl���nee��s�to�store�an����aintain�a���iven�sequence�until�it�is�to�be�recalle���later.�Relationshi�s�bet�een�the���is�s�are�irrelevant�to�the�tas�.�As�these�ite�s�are��resente���sequentiall��,��e��resu�e�that�local�relationshi�s�bet�een��roxi�al�ite�s�can�be�inferre���–�for�ex-a��le,�a��artici�ant��a���be�able�to�notice�an���recall�that�t�o�subsequent���is�s�are�in�consecutive���ri���locations.�This�can�be�a�conscious��rocess,�but��a���quic�l���be��ove���outsi��e�of�the�focus�as�subsequent���is�s�are��resente��.�A���lobal�relation-shi��bet�een�the�full�set�of�ite�s�(i.e.,�the�un��erl��in���rule)��a���be�unconsciousl���inferre��� an��� �rocesse��� �ithin� activate��� lon��-ter�� �e�or��� via� �ulti�le� local�relationshi�s�bet�een�consecutive���is�s.�A����itional��rocessin����a���occur�fro��allocation�of�resources��ithin�the�central�executive,��hich�has�been��ro�ose���to�serve�as�one��eans�of�activatin���ite�s�in�lon��-ter���e�or���(Ba����ele���1996).�

1.5.3 Unconscious��rocessin��

It�a��ears�that�in�ter�s�of�conscious�an����reconscious��rocessin��,�usin���the�e�-be����e��-�rocessin��� �o��el� as� a� ��ui��e,� loci� for� these� res�ective� �rocesses� occur��ithin� the� focus� of� attention� an��� activate��� lon��-ter�� �e�or��.� Less� is� �no�n�about�the�contents�of�lon��-ter���e�or����hich�are�currentl���inactive.�Li�e�ac-tivate���lon��-ter���e�or��,�no�ite��li�its�are��resent�for�these��aterials;�all�of�the�contents�of��e�or���are�store���an����aintaine��.�As�these�ite�s�are�not�acti-vate��,���uration�li�its�are�irrelevant,�thou��h��e�assu�e�that�ite�s,�once�in�lon��-ter���e�or��,�can�never�be� lost.�Ho�ever,�our�un��erstan��in���of�contents�cur-rentl����ithin�this��ortion�of�the��o��el�is�less�un��erstoo���than�for�ite�s��ithin�the�other�re��ions�of� the��o��el,��hich���e�en���on�activation�an��/or�attentional�allocation.� Goin��� bac�� to� Dehaene’s� taxono���,� �e� �ro�ose� that� ite�s� �ithin��

Page 41: Unconscious Memory Representations in Perception

30� A�an��a�L.�Gilchrist�an���Nelson�Co�an

lon��-ter���e�or��,�but�are�not�activate��,�are�trul���subli�inal.�These�ite�s���o�not�have�sufficient�sti�ulus�stren��th�to��arrant�activation,�nor���o�the���receive�suffi-cient�allocation�of�attention.�The�����o�not�contribute�to�conscious�or��reconscious��rocessin���in�an����a��.�While�Dehaene�et�al.�(2006)��ro�ose���that�subli�inal��ro-cessin���can�still�contribute�to�co��nitive�o�erations,��e�are�consi��erabl���less�sure�about� this� contribution.� It� is� our� ho�e� that� future� stu��ies� �ill� �rovi��e� further�un��erstan��in���of�ho��activate���an���inactivate���ite�s�in�lon��-ter���e�or���in-teract��ith�each�other,�as��ell�as�ho��both�contribute�to��rocessin���of�conscious�an���unconscious�sti�uli.�

1.� Conclusions

As�a�theoretical�conce�t,��or�in����e�or���is�inextricabl���tie���to�conscious��he-no�ena.�To��ee��such�infor�ation�accessible�for�short��erio��s�of�ti�e�for�various�co��nitive�o�erations,�this�i��lies�it��ust�be�hel���in�conscious�a�areness.�Ho�-ever,��e�su����est�that��or�in����e�or���also�necessaril���accounts�for��rocessin����hich�is�not�conscious.�While�not���irectl���re�ortable�or��ithin�a�areness,�both�unconscious�an����reconscious��rocessin��� influence�behavior�an����erfor�ance�on�co��nitive�tas�s,�inclu��in���those�of��ri�in���an����erce�tion.�Given�the�various��o��els� of� �or�in��� �e�or��� ��iscusse��� above,� �e� su����est� that� activation-base����o��els,��articularl���those�that�inclu��e�neste����rocesses�li�e�the�e�be����e��-�ro-cess��o��els�of�Co�an�(1988,�1999)�an���Oberauer�(2002,�2005),�are�at�an�a��vanta��e�in���elineatin���conscious,��reconscious,�an���subli�inal��rocessin��.�These��o��els�also�fit�relativel����ell��ith�certain�theories�re��ar��in���conscious�an���unconscious��heno�ena�(e.��.,�Baars�et�al.�2003;�Dehaene�et�al.�2006).�

Certainl���a����itional�research�is�nee��e���to���ain�clearer�un��erstan��in���of�the�connections�bet�een��or�in����e�or���an���conscious�(or�unconscious)��rocess-in��.� One� research� area� that� is� of� �articular� interest� to� us� is� ho�� ite�s� in� the���ifferent�neste����rocesses�of��or�in����e�or���interact��ith�each�other.�As��en-tione��� above,� �reconscious� contents� �ithin� activate��� lon��-ter�� �e�or��� �a���o�erate�‘behin���the�scenes’,��otentiall���influencin���conscious��rocessin����ithin�the� focus� of� attention,� but� it� is� unclear� ho�� such� influence� �i��ht� be� exerte��.�It�is��ossible�that�unconscious�contents��a���also�interact��ith��reconscious�or�conscious�contents��ithin��or�in����e�or��,�an���this��arrants�further�ex�lora-tion.�Another�issue�of�i��ortance�is��hether��rocesses��er�se�can�beco�e��art�of�consciousness�(e.��.,�the��rocess�of�selectin���one�object�at�the�ex�ense�of�another),�or�onl���ele�ents�or�objects�cast�into�the�focus�of�attention�as���irect�or�in��irect�consequences�of�these��rocesses.�B���usin���activation�as�a��eans�to���efine�various�

Page 42: Unconscious Memory Representations in Perception

� Conscious�an���unconscious�as�ects�of��or�in����e�or��� 31

levels�of�consciousness�in�i��e��iate��e�or��,��e�ho�e�that�future�stu��ies��ill�hel��us�better�un��erstan���the�s�ecial��heno�enon�of�consciousness�an���ho��it�inter�la��s��ith��rocessin���in��or�in����e�or��.

Acknowledgements

This��or���as�co��lete����ith�su��ort�for�NIH�Grant�R01-HD-21338.�

References

Allen,�R.�J.,�Ba����ele��,�A.�D.�an���Hitch,�G.�J.�(2006).�Is�the�bin��in���of�visual�features�in��or�-in��� �e�or��� resource-��e�an��in��?� Journal of Experimental Psychology: General,� 135,�298–313.�

An��erson,� J.� R.� (1983).� The architecture of cognition.� Ca�bri����e,� MA:� Harvar��� Universit���Press.�

At�inson,�R.�C.�an���Shiffrin,�R.�M.�(1968).�Hu�an��e�or��:�A��ro�ose���s��ste��an���its�control��rocesses.�In�K.�W.�S�ence�an���J.�T.�S�ence�(E��s.),�The psychology of learning and motiva-tion: Advances in research and theory�(Vol.�2,���.�89–195).�Ne��Yor�:�Aca��e�ic�Press.

Baars,�B.�J.�an���Fran�lin,�S.�(2007).�An�architectural��o��el�of�conscious�an���unconscious�brain�functions:�Global��or�s�ace�theor���an���IDA.�Neural Networks,�20,�955–961.�

Baars,�B.�J.,�Ra�sø��,�T.�Z.�an���Laure��s,�S.�(2003).�Brain,�conscious�ex�erience�an���the�observin���self.�Trends in Neurosciences,�26,�671–675.�

Baars,�B.�J.�an���Fran�lin,�S.�(2003).�Ho��conscious�ex�erience�an����or�in����e�or���interact.�Trends in Cognitive Sciences,�7,�166–172.�

Ba����ele��,�A.�D.�(1986).�Working memory.�Ne��Yor�:�Oxfor���Universit���Press.Ba����ele��,�A.�D.�(1996).�Ex�lorin���the�central�executive.�Quarterly Journal of Experimental Psy-

chology,�49,�5–28.Ba����ele��,�A.�D.�(2000).�The�e�iso��ic�buffer:�A�ne��co��onent�of��or�in����e�or��?�Trends in

Cognitive Sciences,�4,�417–423.�Ba����ele��,�A.�D.�an���Hitch,�G.�J.�(1974).�Wor�in����e�or��.�In�G.�H.�Bo�er�(E��.),�The psychology

of learning and motivation: Advances in research and theory.�Ne��Yor�:�Aca��e�ic�Press.�(Vol.�8,���.�47–89).

Ba����ele��,�A.�D.,�Pa�a��no,�C.�an���Vallar,�G.�(1988).�When�lon��-ter��learnin�����e�en��s�on�short-ter��stora��e.�Journal of Memory and Language,�27,�586–595.

Ba����ele��,�A.�D.�an���Warrin��ton,�E.�K.�(1970).�A�nesia�an���the���istinction�bet�een�lon��-�an���short-ter���e�or��.�Journal of Verbal Learning and Verbal Behavior,�9,�176–189.

Balota,�D.�A.�(1983).�Auto�atic�se�antic�activation�an���e�iso��ic��e�or���enco��in��.�Journal of Verbal Learning and Verbal Behavior,�22,�88–104.

Barrouillet,�P.,�Bernar��in,�S.,�Portrat,�S.,�Ver��au�e,�E.�an���Ca�os,�V.�(2007).�Ti�e�an���co��ni-tive�loa���in��or�in����e�or��.�Journal of Experimental Psychology: Learning, Memory, and Cognition,�33,�570–585.

Page 43: Unconscious Memory Representations in Perception

32� A�an��a�L.�Gilchrist�an���Nelson�Co�an

Ba��s,�P.�M.�an���Husain,�M.�(2008).�D��na�ic�shifts�of�li�ite����or�in����e�or���resources�in�hu�an�vision.�Science,�321,�851–854.

Beiloc�,�S.�L.�an���Carr,�T.�H.�(2005).�When�Hi��h-Po�ere���Peo�le�Fail:�Wor�in���Me�or���an���“Cho�in���Un��er�Pressure”�in�Math.�Psychological Science,�16,�101–105.�

Broa��bent,�D.�E.�(1958).�Perception and communication.�Lon��on:�Per��a�on�Press.Chen,�Z.�an���Co�an,�N.�(2005).�Chun��li�its�an��� len��th� li�its� in� i��e��iate�recall:�A�rec-

onciliation.� Journal of Experimental Psychology: Learning, Memory, and Cognition,� 31,�1235–1249.�

Chen,�Z.�an���Co�an,�N.�(2009a).�Ho��verbal��e�or���loa��s�consu�e�attention.�Memory and Cognition,�37,�829–836.

Chen,�Z.�an���Co�an,�N.�(2009b).�Core�verbal��or�in����e�or���ca�acit��:�The�li�it�in��or��s�retaine��� �ithout� covert� articulation.� Quarterly Journal of Experimental Psychology,� 62,�1420–1429.

Chon��,� S.� C.� an��� Treis�an,� A.� (2005).� Statistical� �rocessin��:� co��utin��� the� avera��e� size� in��erce�tual���rou�s.�Vision Research,�45,�891–900.�

Colflesh,� G.� J.� H.� an��� Con�a��,� A.� R.� A.� (2007).� In��ivi��ual� ��ifferences� in� �or�in��� �e�or���ca�acit���an�����ivi��e���attention�in���ichotic�listenin��.�Psychonomic Bulletin and Review,�14,�699–703.�

Con�a��,�A.�R.�A.�(2005).�On�the�ca�acit���of�attention:�Its�esti�ation�an���its�role�in��or�in����e�or���an���co��nitive�a�titu��es.�Cognitive Psychology,�51,�42–100.

Con�a��,�A.�R.�A.,�Co�an,�N.�an���Buntin��,�M.�F.�(2001).�The�coc�tail��art����heno�enon�re-visite��:�The�i��ortance�of��or�in����e�or���ca�acit��.�Psychonomic Bulletin and Review,�8,�331–335.

Co�an,�N.�(1988).�Evolvin���conce�tions�of��e�or���stora��e,�selective�attention,�an���their��u-tual�constraints��ithin�the�hu�an�infor�ation-�rocessin���s��ste�.�Psychological Bulletin,�104,�163–191.�

Co�an,�N.�(1995).�Attention and memory: An integrated framework.�Oxfor���Ps��cholo�����Series,�No.�26.�Ne��Yor�:�Oxfor���Universit���Press.

Co�an,�N.�(1999).�An�e�be����e��-�rocess��o��el�of��or�in����e�or��.�In�A.�Mi��a�e�an���P.�Shah�(E��s.),�Models of working memory: Mechanisms of active maintenance and executive control�(��.�62–101).�Ca�bri����e,�U.K.:�Ca�bri����e�Universit���Press.�

Co�an,�N.�(2001).�The��a��ical�nu�ber�4�in�short-ter���e�or��:�A�reconsi��eration�of��ental�stora��e�ca�acit��.�Behavioral and Brain Sciences,�24,�87–185.�

Co�an,�N.�(2005).�Working memory capacity.�Ne��Yor�:�Ps��cholo�����Press.�Co�an,�N.,�Elliott,�E.�M.,�Saults,�J.�S.,�More��,�C.�C.,�Mattox,�C.,�His�jatullina,�A.�an���Co�an,�N.�

an���More��,�C.�C.�(2007).�Ho��can���ual-tas���or�in����e�or���retention�li�its�be�investi-��ate��?�Psychological Science,�18,�686–688.�

Co�an,�N.,�More��,�C.�C.,�AuBuchon,�A.�M.,�Z�illin��,�C.�E.�an���Gilchrist,�A.�L.�(2010).�Seven-��ear-ol��s� allocate� attention� li�e� a��ults� unless� �or�in��� �e�or��� is� overloa��e��.� Develop-mental Science,�13,�120–133.

Co�an,�N.,�Naveh-Benja�in,�M.,�Kilb,�A.�an���Saults,�J.�S.�(2006).�Life-S�an���evelo��ent�of�visual��or�in����e�or��:�When�is�feature�bin��in�����ifficult?�Developmental Psychology,�42,�1089–1102.�

Dehaene,�S.,�Chan��eux,�J.-P.,�Naccache,�L.,�Sac�ur,�J.�an���Ser��ent,�C.�(2006).�Conscious,��re-conscious,�an���subli�inal��rocessin��:�A�testable�taxono���.�Trends in Cognitive Sciences,�10,�205–211.

Page 44: Unconscious Memory Representations in Perception

� Conscious�an���unconscious�as�ects�of��or�in����e�or��� 33

En��le,�R.�W.,�Tuhols�i,�S.�W.,�Lau��hlin,�J.�E.�an���Con�a��,�A.�R.�A.�(1999).�Wor�in����e�or��,�short-ter���e�or��,�an�����eneral�flui���intelli��ence:�A�latent-variable�a��roach.�Journal of Experimental Psychology: General,�128,�309–331.

Gilchrist,�A.�L.,�Co�an,�N.�an���Naveh-Benja�in,�M.� (2008).�Wor�in����e�or���ca�acit��� for�s�o�en�sentences���ecreases��ith�a��ult�a��in��:�Recall�of�fe�er,�but�not�s�aller�chun�s�in�ol��er�a��ults.�Memory,�16,�773–787.�

Gilchrist,�A.�L.,�Co�an,�N.�an���Naveh-Benja�in,�M.�(2009).�Investi��atin���the�chil��hoo�����e-velo��ent�of��or�in����e�or���usin���sentences:�Ne��evi��ence�for�the���ro�th�of�chun��ca�acit��.�Journal of Experimental Child Psychology,�104,�252–265.

Gol��,�J.�M.,�Fuller,�R.�L.,�Robinson,�B.�M.,�McMahon,�R.�P.,�Braun,�E.�L.�an���Luc�,�S.�J.�(2006).�Intact�attentional�control�of��or�in����e�or���enco��in���in�schizo�hrenia.�Journal of Ab-normal Psychology,�115,�658–673.�

Hassin,�R.�R.�(2005).�Non-conscious�control�an���i��licit��or�in����e�or��.�In�R.�R.�Hassin,�J.�S.�Ule�an�an���J.�A.�Bar��h�(E��s.),�The new unconscious�(��.�196–224).�Ne��Yor�:�Oxfor���Universit���Press.�

Hebb,�D.�O.�(1949).�Organization of behavior.�Ne��Yor�:�Wile��.�Jacob��,�L.�L.�(1991).�A��rocess���issociation�fra�e�or�:�Se�aratin���auto�atic�fro��intentional�

uses�of��e�or��.�Journal of Memory and Language,�30,�513–541.Ja�es,�W.�(1890).�The principles of psychology.�NY:�Henr���Holt.Jefferies,�E.,�Ral�h,�M.�A.�L.�an���Ba����ele��,�A.�D.�(2004).�Auto�atic�an���controlle����rocessin���

in�sentence�recall:�The�role�of� lon��-ter��an����or�in����e�or��.� Journal of Memory and Language,�51,�623–643.

Johnston,�W.�A.�an���Heinz,�S.�P.�(1978).�Flexibilit���an���ca�acit�����e�an��s�of�attention.�Journal of Experimental Psychology: General,�107,�420–435.

Kane,�M.�J.�an���En��le,�R.�W.�(2003).�Wor�in��-�e�or���ca�acit���an���the�control�of�attention:�The�contributions�of���oal�ne��lect,�res�onse�co��etition,�an���tas��set�to�Stroo��interfer-ence.�Journal of Experimental Psychology: General,�132,�47–70.�

Kotchoube��,�B.,�Lan��,�S.,�Bostanov,�V.�an���Birbau�er,�N.�(2002).�Is�there�a��in��?�Electro�h��si-olo�����of�unconscious��atients.�News in Physiological Science,�17,�38–42.

LaBer��e,� D.� an��� Bro�n,� V.� (1989).� Theor��� of� attentional� o�erations� in� sha�e� i��entification.�Psychological Review,�96,�101–124.�

Luc�,�S.� J.�an���Vo��el,�E.�K.� (1997).�The�ca�acit���of�visual��or�in����e�or��� for� features�an���conjunctions.�Nature,�390,�279–281.

Lusti��,�C.,�Hasher,�L.�an���Zac�s,�R.�T.�(2007).�Inhibitor�����eficit�theor��:�Recent���evelo��ents�in�a�“ne��vie�.”�In�D.�S.�Gorfein�an���C.�M.�MacLeo���(E��s.),�Inhibition in cognition�(��.�145–162).�Washin��ton,�D.C.:�A�erican�Ps��cholo��ical�Association.

Marcel,�A.�J.�(1983).�Conscious�an���unconscious��erce�tion:�Ex�eri�ents�on�visual��as�in���an����or���reco��nition.�Cognitive Psychology,�15,�197–237.�

McNab,�F.�an���Klin��ber��,�T.�(2008).�Prefrontal�cortex�an���basal���an��lia�control�access�to��or�-in����e�or��.�Nature Neuroscience,�11,�103–107.�

Miller,�G.�A.�(1956).�The��a��ical�nu�ber�seven,��lus�or��inus�t�o:�So�e�li�its�on�our�ca�acit���for��rocessin���infor�ation.�Psychological Review,�63,�81–97.

Miller,�G.�A.,�Galanter,�E.�an���Pribra�,�K.�H.�(1960).�Plans and the structure of behavior.�Ne��Yor�:�Holt,�Rinehart�an���Winston,�Inc.

Mi��a�e,�A.,�Frie���an,�N.�P.,�Rettin��er,�D.�A.,�Shah,�P.�an���He��art��,�M.�(2001).�Ho��are�visuo-s�atial��or�in����e�or��,�executive�functionin��,�an���s�atial�abilities�relate��?�A�latent�vari-able�anal��sis.�Journal of Experimental Psychology: General,�130,�621–640.

Page 45: Unconscious Memory Representations in Perception

34� A�an��a�L.�Gilchrist�an���Nelson�Co�an

Mi��a�e,�A.�an���Shah,�P.�(E��s.).�(1999).�Models of working memory: Mechanisms of active main-tenance and executive control.�Ca�bri����e,�U.K.:�Ca�bri����e�Universit���Press.

Mora��,�N.�(1959).�Attention�in���ichotic�listenin��:�Affective�cues�an���the�influence�of�instruc-tions.�Quarterly Journal of Experimental Psychology,�11,�56–60.

More��,�C.�C.�an���Co�an,�N.�(2004).�When�visual�an���verbal��e�ories�co��ete:�Evi��ence�of�cross-��o�ain�li�its�in��or�in����e�or��.�Psychonomic Bulletin and Review,�11,�296–301.

More��,�R.�D.,�Rou��er,�J.�N.�an���S�ec��an,�P.�L.�(2008).�A�statistical��o��el�for���iscri�inatin���bet�een�subli�inal�an���near-li�inal��erfor�ance.� Journal of Mathematical Psychology,�52,�21–36.

Nor�an,�D.�A.� (1968).�To�ar���a� theor���of��e�or���an���attention.�Psychological Review,�75,�522–536.�

Oberauer,�K.�(2002).�Access�to�infor�ation�in��or�in����e�or��:�Ex�lorin���the�focus�of�atten-tion.�Journal of Experimental Psychology: Learning, Memory, and Cognition,�28,�411–421.

Oberauer,�K.�(2005).�Control�of�the�contents�of��or�in����e�or���–�A�co��arison�of�t�o��ara-��i���s�an��� t�o�a��e���rou�s.� Journal of Experimental Psychology: Learning, Memory, and Cognition,�31,�714–728.�

Rees,�G.,�Krei�an,�G.�an���Koch,�C.�(2002).�Neural�correlates�of�consciousness�in�hu�ans.�Na-ture Reviews,�3,�261–270.�

Rou��er,�J.�N.,�More��,�R.�D.,�Co�an,�N.,�Z�illin��,�C.�E.,�More��,�C.�C.�an���Pratte,�M.�S.�(2008).�An�assess�ent�of�fixe��-ca�acit����o��els�in�visual��or�in����e�or��.�Proceedings of the Na-tional Academy of Sciences of the United States of America,�105,�5975–5979.�

Saults,�J.�S.�an���Co�an,�N.�(2007).�A�central�ca�acit���li�it�to�the�si�ultaneous�stora��e�of�visual�an���au��itor���arra��s�in��or�in����e�or��.�Journal of Experimental Psychology: General,�136,�663–684.

Seth,�A.�K.,�Baars,�B.�J.�an���E��el�an,�D.�B.�(2005).�Criteria�for�consciousness�in�hu�ans�an���other��a��als.�Consciousness and Cognition,�14,�119–139.�

Si�ons,�D.�J.�an���Rensin�,�R.�(2005).�Chan��e�blin��ness:�Past,��resent,�an���future.�Trends in Cognitive Sciences,�9,�16–20.

Sno����rass,�M.,�Bernat,�E.�an���Shevrin,�H.�(2004).�Unconscious��erce�tion:�A��o��el-base���a�-�roach�to��etho���an���evi��ence.�Perception and Psychophysics,�66,�846–867.�

Sno����rass,�M.�an���Shevrin,�H.�(2006).�Unconscious�inhibition�an���facilitation�at�the�objective���etection�threshol��:�Re�licable�an���qualitativel�����ifferent�unconscious��erce�tual�effects.�Cognition,�101,�43–79.�

S�erlin��,�G.�(1960).�The�infor�ation�available�in�brief�visual��resentations.�Psychological Mono-graphs,�74�(Whole�No.�498).

Stevanovs�i,�B.�an���Jolicoeur,�P.�(2007).�Visual�short-ter���e�or��:�Central�ca�acit��� li�ita-tions�in�short-ter��consoli��ation.�Visual Cognition,�15,�532–563.

Treis�an,�A.�M.�(1964b).�Selective�attention�in��an.�British Medical Bulletin,�20,�12–16.Treis�an,�A.�M.�an���Gela��e,�G.� (1980).�A� feature� inte��ration� theor���of� attention.�Cognitive

Psychology,�12,�97–136.VanRullen,�R.�an���Koch,�C.�(2003).�Is��erce�tion���iscrete�or�continuous?�Trends in Cognitive

Sciences,�7,�207–213.Vo��el,�E.�K.,�McCollou��h,�A.�W.�an���Machiza�a,�M.�G.�(2005).�Neural��easures�reveal�in��i-

vi��ual���ifferences�in�controllin���access�to��or�in����e�or��.�Nature,�438,�500–503.�

Page 46: Unconscious Memory Representations in Perception

� Conscious�an���unconscious�as�ects�of��or�in����e�or��� 35

Vuilleu�ier,�P.,�Sa��iv,�N.,�Hazeltine,�E.,�Pol��rac�,�R.,�S�ic�,�D.,�Rafal,�R.�D.�an���Gabrieli,�J.�D.�E.�(2001).�Neural�fate�of�seen�an���unseen�faces�in�visuos�atial�ne��lect:�A�co�bine���event-relate���functional�MRI�an���event-relate����otential�stu����.�Proceedings of the National Acad-emy of Sciences,�98,�3495–3500.

Woo��,�N.�an���Co�an,�N.�(1995a).�The�coc�tail��art����heno�enon�revisite��:�Ho��frequent�are�attention�shifts�to�one’s�na�e�in�an�irrelevant�au��itor���channel?�Journal of Experimental Psychology: Learning, Memory, and Cognition,�21,�255–260.�

Woo��,�N.�an���Co�an,�N.� (1995b).�The�coc�tail��art����heno�enon�revisite��:�Attention�an����e�or���in�the�classic�selective�listenin����roce��ure�of�Cherr���(1953).�Journal of Experi-mental Psychology: General,�124,�243–262.

Woo��,�N.�L.,�Sta��ler,�M.�A.�an���Co�an,�N.�(1997).�Is�there�i��licit��e�or����ithout�attention?�A�re-exa�ination�of�tas����e�an��s�in�Eich’s�(1984)��roce��ure.�Memory and Cognition,�25,�772–779.

Zhan��,�W.�an���Luc�,�S.�J.�(2008).�Discrete�fixe��-resolution�re�resentations�in�visual��or�in����e�or��.�Nature,�453,�233–235.�

Page 47: Unconscious Memory Representations in Perception
Page 48: Unconscious Memory Representations in Perception

chapter�2

Markers of awareness? EEG��otentials�evo�e���b���faint�an����as�e���events,��ith�s�ecial�reference�to�the�“attentional�blin�”

Rolf�Verle��er�Universit���of�Lübec�,�Ger�an��

2.1 Paradigms

This�revie���ill�start��ith�a�brief�re�in��er�about�the��ioneerin���research���one��ith�the��ara��i����on���etection�of�au��itor���si��nals�an���then��ill� focus�on�t�o��ore�recent�lines�of�research��hich�are�(1)�Detection�an���i��entification�of�faint�or��as�e���visual�sti�uli,�an���(2)�The�attentional�blin�.

Other��ara��i���s�that�have� to�be�s�i��e���for�reasons�of�s�ace� inclu��e�bin-ocular�rivalr���(e.��.,�Roeber�et�al.�2008),�chan��e�blin��ness�(e.��.,�Ei�er�an���Mazza�2005;�Fernan��ez-Duque�et�al.�2003;�Koivisto�an���Revonsuo�2003;�Schan�in�an���Wascher�2007,�2008;�Turatto�et�al.�2002;�see�Czi��ler�this�volu�e),�an����erce�tion�of�a�bi��uous�fi��ures�(e.��.,�Keil�et�al.�1999;�Korn�eier�an���Bach�2004;�S�ith�et�al.�2006).�

2.2 Detection of auditory signals

It�is�easil���overloo�e���in�no�a��a��s���iscussions�that�brain�correlates�of�a�areness��ere�alrea�����stu��ie���in�the�ver���be��innin��s�of�ERP�research.�This�earl���a��roach�use���the��ara��i����of���etection�of�faint�au��itor���si��nals,�startin����ith�Hill��ar���et�al.’s�(1971)�Science��a�er�on�“Evo�e����otential�correlates�of�au��itor���si��nal���etec-tion”.�The��ara��i����is�illustrate���in�Fi��ure�1.

A��arnin���li��ht��as�or��as�not�follo�e���0.5�s�later�b���a�brief�tone.�Pro��te���b���another�li��ht,�1�s�later,��artici�ants�ha���to�in��icate��hether�a�tone��as��res-ent�or�not.�Tone�intensit����as�varie���across�bloc�s,�reachin���fro��lo��intensities��here��artici�ants’�res�onses��ere�at�chance�level�to�clearl�����etectable�intensities.�

Page 49: Unconscious Memory Representations in Perception

3�� Rolf�Verle��er

This�stu�����an���the�follo�in���series�of��a�ers�establishe���that�the��rinci�al�cor-relate�of�the�subjective,�conscious���ecision�that�so�e�si��nal��as��resent��as�the�P3�co��onent� (Ker�hof� an���Uhlenbroe��1981;�Parasura�an�an���Beatt���1980;�Parasura�an�et�al.�1982;�Paul�an���Sutton�1972;�Ruch�in�et�al.�1980;�K.�Squires�et�al.�1973,�1975a,�1975b;�N.�Squires�et�al.�1978;�Sutton�et�al.�1982;�Wil�inson�an���Seales�1978).�

A� corres�on��in��� a��roach� �ith� so�atosensor��� sti�uli� �as� �ioneere��� b���Des�e��t�et�al.�alrea�����in�1965,�in�the�ver���first���escri�tion�of�the�P3�co��onent�(in��arallel�to�Sutton�et�al.�1965).�Ho�ever,�this�a��roach���i���not�have�i��e��iate�i��act,�bein����ublishe���as�a��ono��ra�h�in�French.�Coverin���the�literature�on�so-�atosensor���sti�uli��oul���excee���the�li�its�of�this�contribution.�It��a���suffice�to��ention�t�o�recent��a�ers�that�focuse���on�the�earliest�si��ns�of�conscious�feelin��s.�S�i���et�al.�(2004)���escribe���an�earl���N1-t���e�N60�that���istin��uishe���bet�een���e-tecte���an���un��etecte���s�all��assive��ove�ents�of�the�lo�er�ri��ht�le��.�Schubert�et�al.�(2006)��as�e���a�faint�electrical��ulse�to�the�left�in��ex�fin��er�b���a�stron����ulse�to�the�ri��ht�in��ex�fin��er,�an���obtaine���enhance���P100�an���N140�a��litu��es��hen�the�faint�sti�uli��ere���etecte���co��are���to��hen�the���re�aine���un��etecte��.�

2.3 P3 evoked by unidentified visual stimuli?

When�ERPs�beca�e�available�in�the�sixties,�subli�inal�visual��erce�tion�ha���be-co�e�a�to�ic�of���eneral�interest�(e.��.,�Dixon�1971).�Therefore,�so�e�of�the�earliest�a��lications�of�ERPs�covere���this�to�ic,�usin���faint�an���briefl����resente���sti�uli,�either��as�e���b���follo�in���sti�uli�(Donchin�an���Lin��sle���1965;�Sch�artz�an���Pritchar���1981)�or�not�(Shevrin�an���Fritzler�1968a,�1968b).�A�nu�ber�of�fin��in��s��ere�obtaine���(see�Shevrin�2001,�for�a�revie��on�his��or�).�But�these�stu��ies��ere�affecte��� b��� several� shortco�in��s� arisin��� fro�� circu�stances� �revailin��� at� that�ti�e,�of��hich��robabl���the��ost�i��ortant�one�is�that�ERPs��ere�not�recor��e���at�lateral��osterior�sites�(P7,�P8,�PO7,�PO8)��here,�as��as�later�foun��,��ost�evo�e���visual�co��onents�are�lar��est�an���are��ost�sensitive�to�ex�eri�ental�variation.�(Cf.�Rossion�an���Jacques�2008,�for�the�continuin���relevance�of�this��oint.)�

Figure 1. Au��itor���Detection�Tas��of�Hill��ar���et�al.�(1971).�See�text�for���escri�tion.

Page 50: Unconscious Memory Representations in Perception

� Mar�ers�of�a�areness?� 39

Of�the��ore�recent�stu��ies,�a�fe���a��e�the�clai��that�so�e�s�all�P3�co��o-nent,�thou��h�certainl���s�aller�than�the�P3�evo�e���b����ell�visible�sti�uli,�coul���be�obtaine����ith�infrequent�sti�uli��resente���a�on���frequent�sti�uli,�althou��h�all�these�sti�uli��ere�so�faint�an���so�briefl����resente���that�the���coul���not�be���is-tin��uishe���(Bráz��il�et�al.�1998;�Bernat�et�al.�2001).�This�fin��in����as�even��a��e�in�intracranial�recor��in��s�(Bráz��il�et�al.�2001).�These�results�bolstere���the�clai��that�so�e�t���e�of�co��nitive���iscri�ination,�reflecte���b���P3,��a���be��a��e�entirel���non-consciousl��.�Thereb��,�these�results�are�in���irect�conflict��ith�the�notion�that�P3�is�a�correlate�of�conscious���ecision�(cf.�above,�on�au��itor���si��nal���etection,�an���belo�).�Ho�ever,�i��entification��erfor�ance��i��ht�have�not�been�co��letel���at�chance�in�those�stu��ies.�

The� �ara��i���� use��� b��� Bráz��il� et� al.� (1998,� 2001)� is� ��e�icte��� in� Fi��ure� 2.�Bráz��il�et�al.�(2001)�re�orte���that,�b���absolute�nu�bers,�correct�hits�to�the�faint�si��nals�(brief�“X”)�occurre���onl���2/3�as�often�as�false�alar�s��ith�faint�non-si��-nals�(brief�“O”).�So�the���conclu��e���that�the�nu�ber�of�hits�re�aine����ell�belo��chance.�Ho�ever,��artici�ants��ne��ver����ell�that�the�O��as��resente���five�ti�es��ore�often�than�the�X,�because�the�faint�sti�uli��ere�inter�in��le���in�clearl���vis-ible�sti�uli.�So�the�ratio�of�numbers�of�hits�/�false�alar�s�has�to�be��ulti�lie���b���five�to�obtain�the�ratio�of�the�rates�of�false�alar�s�/�hits,���iel��in���2/3�*�5�=�3.3.�Thus,�correct�res�onses��ere�3.3�ti�es��ore��robable�than�false�alar�s,��hich�is�far�above�chance.�A�si�ilar�consi��eration��a���a��l���to�Bráz��il�et�al.�(1998)��here�nu�bers�of�false�alar�s��ere�not�re�orte��.

Fi��ure�3�illustrates�the��ara��i����use���b���Bernat�et�al.�(2001).�In�their�stu����,�the��or��s�LEFT�or�RIGHT��ere��resente���for�1��s,�one�of�these��or��s�occurrin����ith�a�frequenc���of�20%,�the�other��ith�80%.�Partici�ants’�ERPs�inclu��e���a�slo���ositivit���of�about�0.5�µV��ean�a��litu��e�in�res�onse�to�the�20%�sti�uli,�si��-nificantl���lar��er�than�in�res�onse�to�the�80%�sti�uli.�Since��artici�ants��ere�not�able�to�consciousl�����iscri�inate�these��or��s�fro��blan��fiel��s�in�a�se�arate�bloc�,�Bernat�et�al.�conclu��e���that�the�slo���ositivit���reflecte���subconscious���iscri�ina-tion�bet�een�the�rare�an���the�frequent�sti�uli.�

Figure 2. Sti�ulus�Series�in�Bráz��il�et�al.�(1998).�An�exa��le�for�a��ortion�of�the�sti�ulus�series�is���e�icte��.�The�interval�bet�een�sti�uli�varie���bet�een�2�s�an���5�s.�Partici�ants�ha���to��ress�a�button�in�res�onse�to�ever���X.�The�X�tar��ets�occurre����ith�a��robabilit���of�1/5�a�on���the�frequent�O�nontar��ets.�Half�the�sti�uli��ere��ell�visible�(��enote���b���blac��a��earance)�b���bein����resente���for�200��s.�The�other�half��as�har��l���visible�(��re���a��earance)�b���bein����resente���for�10��s.

Page 51: Unconscious Memory Representations in Perception

40� Rolf�Verle��er

But� infrequent�an���frequent�sti�uli��ere�not��resente��� in�co��letel���ran-��o��or��er��ithin�successive�bloc�s�of�five�trials.�Rather�the�o����ball�sti�uli��ere��resente���in�the�3r��,�4th,�or�5th�trial�of�five�consecutive�trials�(Verle��er�2001).�Thus,�a�critical�question�is��hether��artici�ants��ere�a�are�of�the�fact�that�sti�uli��ere� �resente��� in� this� bloc�� structure� of� five� (Bernat� 2001).� If� so� then� the� re-�orte���effect��i��ht�have�been���ue�to�a���ifference�bet�een�earlier�an���later��osi-tions��ithin�the�fixe���or��er�of�five�sti�uli�rather�than���ue�to�a���ifference�bet�een�subconsciousl����erceive���rare�an���frequent�sti�uli.�Thus,�these�fin��in��s�re�ain�controversial.�

2.4 ERP signature of conscious identification of faint or masked visual stimuli

This�section��ill�focus�on�stu��ies�that�searche���for�the�neural�correlate�of�con-sciousness,���efine���as�the���ifference�bet�een�i��entifie���an���uni��entifie���visual�sti�uli.�

2.4.1 The�P3�co��onent

As�a��ain�fin��in��,�li�e�in�the�above-�entione���au��itor���stu��ies,�the�P3�co��o-nent��as��resent�for�i��entifie���sti�uli�an����as�absent,�or�at�least��uch�re��uce��,�for�uni��entifie���sti�uli.�This�fin��in���is�illustrate���in�Fi��ures�4,�5,�an���6�for�the�stu��ies�b���Pins�an���ff��tche�(2003),�Er��enoğlu�et�al.�(2004),�an���Koivisto�et�al.�(2005).

Figure 3. Trial�in�Bernat�et�al.�(1998).�A�soun����rovi��e���the�start�si��nal.�In�res�onse,��artici�ants�ha���to�in��icate�the�trial�nu�ber�(“1,�rea����”�until�“300,�rea����”).�Then,�the��or���LEFT�or�RIGHT,��ith��robabilities�of�80/20,��as��resente���for�1��s.�After�1�s,�a���ouble�soun���in��icate���en���of�the�trial.�The�o����ball�(20%��robabilit��)�sti�ulus��as��resente���in�the�3r��,�4th,�or�5th�trial�of�a�bloc��of�five�trials.

Page 52: Unconscious Memory Representations in Perception

� Mar�ers�of�a�areness?� 41

Figure 4. Stu�����of�Pins�an���ff��tche�(2003).�Top:�Trial�structure.�The���ratin����as�shifte���across�trials��ithin�a�450��s�interval.�Its���uration��as�continuousl���a��a�te���such�that�it��as��erceive���in�50%�of�the�trials.�Bottom:�Gran����eans�of�recor��in��s�fro��O2�vs.�lin�e����astoi��s.�“Yes”�trials��here��artici�ants�re�orte���seein���the���ratin���are�in�blac�,�“no”�trials�in���ar����re��.�Catch�trials��ithout�an�����ratin���(use���in�4�of�the�5��artici�ants)�are�in�li��ht���re���(to��ether��ith�the���ashe���lines�as�a��easure�of�their�variabilit��).�Note:�0��s���enotes��arnin��-tone�onset�in�the�u��er���ra�h�but���enotes�onset�of�the���ratin���in�the�lo�er���ra�h.�“Yes”�trials���iffere���fro��“no”�trials�b���the�P3�co��onent�as��ell�as�b���the�earlier�P1�an���N2�co��onents�(see�text).�(The�lo�er��art�of�the�fi��ure��as�a��a�te����ith��er�ission�fro��Fi��.�6�of�Pins�an���ff��tche�2003.)

2.4.2 Earlier�co��onents:�P1

Pins�an���ff��tche�(2003)�even�obtaine���a�ver���earl�����ifference�(Fi��ure�4):�Gratin��s�that��ere���etecte���elicite���a�P1�co��onent�at�occi�ital�sites�at�100��s,��hereas�un��etecte��� ��ratin��s� ��i��� not.� The� authors� s�eculate��� that� this� P1� effect� �as� a�“�ri�ar���correlate�of�consciousness”�(�.�473),�unli�e�the�follo�in���effects�on�N2�an���P3�that��ere�assu�e���to�re�resent�“��o�nstrea��secon��ar����rocesses”�onl���(ibi��.),�not�contributin�����irectl���to��erce�tion.�This�far-reachin���inter�retation�

Page 53: Unconscious Memory Representations in Perception

42� Rolf�Verle��er

of�the�P1�effect���oes�not�a��ear�to�be�justifie��,�thou��h,�because�increases�of�visu-al�P1�have�often�been�obtaine����ith�un��etecte���chan��es�of�visual�in�ut�(Ki�ura�et�al.�2006,�2008;�cf.�the�cha�ter�b���Czi��ler�in�this�volu�e).�Thus,�the�P1�effect,�if�re�licable,��a���just�in��icate�that�so�e�chan��e��as�notice���b���the�visual�s��ste�,��ithout�an���consciousness� involve��.�Such�notice�of�chan��e��i��ht� in��ee���be�a�necessar����rerequisite�for�conscious�i��entification,�but�in�vie��of�Ki�ura�et�al.’s�(2006,�2008)�results�not�a�sufficient�one.�

Figure 5. Stu�����of�Er��enoğlu�et�al.�(2004).�Top:�Trial�structure.�A�faint�square��as�briefl����rojecte���at��erce�tual�threshol��.�Partici�ants��resse���a�button��hen�the����erceive���the�square.�Bottom:�Gran����eans�of�recor��in��s�(referre���to�lin�e���earlobes).�Trials��ith���etecte���sti�uli�(bol��)�are���istin��uishe���b���a��ar�e���P3�co��onent.�(The�EEG���ran����eans�are�re�rinte����ith��er�ission�fro��Fi��.�1�in�Er��enoğlu�et�al.�2004.)

Page 54: Unconscious Memory Representations in Perception

� Mar�ers�of�a�areness?� 43

2.4.3 N2-t���e�co��onents

There�are�a�nu�ber�of�re�orts�on�ERP���ifferences�bet�een�i��entifie���an���uni��en-tifie���sti�uli�after�this�ver���earl���P1�effect�an���before�the�late�P3�effect.�The�ques-tion�is�of�so�e�interest��hether�such���ifferences�are�“��o�nstrea���rocesses”�onl��.�Even�if�such�co��onents�are�“not�contributin�����irectl���to��erce�tion”�(Pins�an���ff��tche�2003,��.�473)�the����i��ht�nevertheless�be�reflections�of��rocesses�necessar���to��a�e�the�subject�beco�e�a�are�of�the�result�of��erce�tion.�

There�is�so�e�evi��ence�for�a�ne��ative�co��onent�that���istin��uishes�bet�een�i��entifie���an���uni��entifie���sti�uli.�Pins�an���ff��tche�(2003)�obtaine���a�N260�(Fi��-ure�4),��axi�u���robabl���at�P7�(their�Fi��.�6;�unfortunatel���the�authors���is�la��e����a�s�of�the���istribution�of�current�source���ensities�onl����here�the�P7�site,�b���be-in���situate���at�the��ar��in�of�the��onta��e,���oes�not�a��ear�an����ore).�A�si�ilar�ne��ative���ifference��ith��axi�u��at�about�250��s,�a��ain��robabl���lar��est�at�P7�(no�co��lete� to�o��ra�hic���ata��ere� re�orte��),��as�obtaine���b���Koivisto�et� al.�(2005,�Ex�.�2;� see�Fi��ure�6).�Differin���bet�een���etecte���an���un��etecte��� tar��et�

Figure 6. Stu�����of�Koivisto�et�al.�(2005).�Top:�Trial�structure.�The�letters�H,�T,��or�U��ere�briefl����resente���an���follo�e���b���a��as�.�One�of�the�letters��as�the�tar��et.�Bottom:�Gran���avera��e�recor��in���for�the�tar��et�letter��hen���etecte���(bol���line)�an����hen�un��etecte���(��ashe���line)�fro��P7�(=�T5;�left��anel)�an���fro��P3�(ri��ht��anel),�referre���to�nose.�Detecte���tar��ets�evo�e���lar��er�N2�an���P3�co��onents�than�un��etecte���tar��ets�(�hich���i���evo�e�these�co��onents�too,�in�contrast�to�the�tas�s���e�icte���in�Fi��s.�4�an���5,�because�at�least�the��as���as�clearl���visible).

Page 55: Unconscious Memory Representations in Perception

44� Rolf�Verle��er

letters,�this�N250��as�labele���“visual�a�areness�ne��ativit��”�b���Koivisto�et�al.�Ho�-ever,�a�si�ilar�(thou��h�s�aller)���ifference��as�obtaine���in�their�Ex�.1�bet�een�tar��et� an��� nontar��et� letters� �hen� both� �ere� uni��entifie��� (bein��� follo�e��� b��� a��as��after�33��s).�Therefore,�it�is�not��lausible�that�N250��as�a���irect�correlate�of�a�areness�in�these�ex�eri�ents�but�rather�a�correlate�of�selection�of�the�relevant�tar��et.�In��ee��,�the�authors�ter�e���this�N250�effect�“selection�ne��ativit��”��hen�it�occurre����ith�uni��entifie���sti�uli.�(There��as�a�ne��ative��otential�overla��in���this�selection�ne��ativit��,�as�a���ifference�bet�een��erceive���an���un�erceive���let-ters,�in�Koivisto�et�al.’s�Ex�.�1,��hich�the�authors�ter�e���“a�areness�ne��ativit��”.�Ho�ever,�this�effect��i��ht��ell�be�confoun��e���b����otentials�evo�e���b���the��as��because��as��ti�in���s��ste�aticall�����iffere���bet�een��erceive���an���un�erceive���letters.�This�confoun�����i���not�occur�in�their�Ex�eri�ent�2,��hich��as�therefore�selecte���here�for�re�ort).�

Ojanen�et�al.�(2003)�re�orte���a�shar��ne��ative��otential�at�about�450��s�af-ter� onset� of� briefl��� �resente��� �ictures� that� �ere� successfull��� rate��� as� intact� or�scra�ble���(Fi��ure�7).

Bein��� absent� after� �ictures� �here� �artici�ants� coul��� not� tell� the� ��ifference�bet�een�intact�an���scra�ble��,�this��otential��as�a��ain�calle���“visual�a�areness�ne��ativit��”.�Ho�ever,�this�“VAN”���iffers�fro��Koivisto�et�al.’s�(2005)�VAN�b���its��uch�later� latenc���(after�the�P3�vs.�before�P3),�b���its�to�o��ra�h���(lar��est�at�Cz�an���entirel���absent�at��osterior�sites�vs.�lar��est�at��osterior�sites)�an���b���its�shar��s�i���� �avefor�.� Further�ore,� the� ��ifference� �as� confoun��e��� b��� �h��sical� fac-tors�because��ost�of� the� i��entifie����ictures�ha���hi��h�contrast�an����ost�of� the�uni��entifie����ictures�ha���“�i����le”,� i.e.,� lo�er,� contrast.�Thus,� the� status�of� this�shar��Cz-focuse���N450�is�unclear.�Of�interest,�no���ifference�bet�een�i��entifie���an���uni��entifie����ictures��as�obtaine���for�P3.�Possibl��,�the�strict�hi��h-�ass�filter�(0.3�s�ti�e�constant)�ha���filtere����uch�of�the�P3�co��onent�(Duncan-Johnson�an���Donchin�1979).�

Dehaene�et�al.� (2001)��easure���ERPs� to�briefl����resente����or��s� that��ere�e�be����e���in��as�in���strea�s�of��atterns�of�s�all�qua��ran��les.�Wor��s��ere�vis-ible��hen�the�screen�briefl���(70��s)�re�aine���blan��before�an���after�the��or��.�Visible��or��s�elicite���lar��er��otentials�than��as�e����or��s,�inclu��in���P1�(li�e�Pins�an��� ff��tche� 2003),� �osterior� N250� (li�e� Pins� an��� ff��tche� 2003,� an��� Koivisto� et�al.�2005),�central�N340�(�ossibl����or��-s�ecific),�an���P3.�Unfortunatel��,�no��ave-for�s��ere�sho�n�in�that��a�er,�so�the�question�cannot�be�ans�ere���to��hat�ex-tent�so�e�of�these�effects��i��ht�be���ue�to���ifferent�overla��of�the�ERPs�evo�e���b���the��rece��in���an���follo�in���blan��vs.��as��screens.

Page 56: Unconscious Memory Representations in Perception

� Mar�ers�of�a�areness?� 45

Figure 7. Stu�����of�Ojanen�et�al.�(2003).�Top:�Coherent�an���scra�ble����ictures��ere���resente���for�27��s�at�hi��h,��i����le,�or�lo��contrast,�in�ran��o��or��er.�Bottom:�Gran����avera��es�(referre���to�lin�e����astoi��s).�Soli���lines���enote�trials�in��hich��artici�ants��ere�able�to�classif���the��ictures�as�intact�or�scra�ble��,���ashe���lines���enote�trials��here�the�“��on‘t��no�“�res�onse��as��a��e.�(This�fi��ure�is�co��ile����ith��er�ission�fro��Fi��.�1�an���Fi��.�2�of�Ojanen�et�al.�2003.)

Page 57: Unconscious Memory Representations in Perception

4�� Rolf�Verle��er

2.4.4 N2�c

A� �a��� to� ��eal� �ith� the� overla�� of� �otentials� evo�e��� b��� �as�e��� an��� �as�in���sti�uli�is�to��resent�tas�-relevant�sti�uli�on�one�si��e�of�fixation�an���irrelevant�sti�uli�on�the�other�si��e.�Then,�an�“N2�c”��a���be�recor��e���in�the���ifference��o-tential�bet�een�left�an���ri��ht�lateral��osterior�sites:��ore�ne��ativit���contralater-all���to�relevant�sti�uli�than�i�silaterall��,��ith�a��ea��at�about�250��s�after�sti�-ulus�onset�(Ei�er�1996;�Ho�f�et�al.�2006;�Luc��an���Hill��ar���1994;�Wascher�an���Wausch�uhn�1996).�N2�c��a���sensitivel�����isentan��le��artici�ants’�selection�of��as�e���an����as�in���sti�uli,�as�is�illustrate���in�Fi��ure�8�an����ill�be���etaile���in�the�follo�in���(��ata�fro��Jaś�o�s�i�et�al.�2002;�cf.�Verle��er�an���Jaś�o�s�i�2007).�

The��ara��i����is���escribe���in�the�le��en���of�Fi��ure�8.�Conventional�ERPs�are�sho�n�in�the�u��er�ERP��anel�of�Fi��ure�8.�The���are��oole���across� trials��ith�left�or�ri��ht�si��e�of�the�tar��et�sha�e�an���are�recor��e���a��ainst�co��on�reference�(the�nose-ti�).�The�first�sti�ulus��air,��resente���at�0��s,�evo�e���P1�an���N1�co�-�onents,�no��atter��hether�this��air��as�visible�or�not�(167��s�vs.�83��s�SOA).�The�N1�co��onent��robabl���inclu��e���contributions�fro��the�secon���sti�ulus,��resente���at�83��s�or�167��s.�Criticall��,� the�first�notable���istinction�bet�een�co��atible�an���inco��atible�trials�(i.e.,�trials��here�the�tar��et�sha�es��ere�at�the�sa�e�or�at���ifferent��ositions�in��as�e���an���visible�sti�ulus��airs)��as�the�N2�co��onent�that��as��uch�lar��er�in�inco��atible�trials.�This�N2��as��ost��rob-abl���evo�e���b���the�visible�sti�uli,��ea�in���at�the�fixe���latenc���of�a��roxi�atel���290��s�after�onset�of�the�visible�sti�ulus�both��ith�83��s�an����ith�167��s�SOA�cases�(i.e.,�at�370��s�an���455��s�after�onset�of�the��as�e���sti�ulus).�Therefore,�this��ea�� is���enote���“N2(visible)”� in�Fi��ure�8.�But�co��onents�evo�e���b��� the��as�e���sti�uli�are�not�rea��il���seen�in�these�conventional�ERPs.

For�obtainin���the�lo�er�ERP��anel�of�Fi��ure�8,�trials��ere�sorte���accor��in���to�si��e�of�the�tar��et�sha�e�in�the�visible�sti�ulus,�an���the���ifference�contralat-eral��inus�i�silateral�to�the�tar��et��as�for�e���se�aratel���for�left-si��e-tar��et�tri-als�(P8-P7,�i.e.,�ri��ht��inus�left�site)�an���ri��ht-si��e�tar��et�trials�(P7-P8).�N2�cs,�ter�e���“N2�c(visible)”,�are�evi��ent� in�the� lo�er��anel�at� the� latencies�of�N2�in�the�u��er��anel,��hich�sho�s�that�N2�has�a�lar��er�contribution�fro��the�cortex�contralateral�to�the�tar��et�sha�e�than�fro��the�i�silateral�one.�Of�i��ortance,�this�effect�is��rece��e���b���a���iver��ence�bet�een�contralateral�an���i�silateral�tri-als,��ea�in���at�280��s,��hich�is�evi��entl���the�N2�c�evo�e���b���the�tar��et�sha�e�in�the��as�e���sti�ulus,�“N2�c(�as�e��)”�in�Fi��ure�8,��ointin���u��ar��s�or���o�n-�ar��s,���e�en��in���on�si��e�of�the�tar��et�sha�e�in�the��as�e���sti�ulus�(sa�e�si��e�as�the�tar��et�in�the�visible�sti�ulus�tri����ers�an�u��ar�����eflection,�o��osite�si��e�a���o�n�ar�����eflection).�

Page 58: Unconscious Memory Representations in Perception

� Mar�ers�of�a�areness?� 47

Figure 8. Data�fro��Jaś�o�s�i�et�al.�(2002,�Ex�.�1).�Top:�A�first��air�of�sti�uli,�left�an���ri��ht�fro��fixation,��as�follo�e���at�the�sa�e�screen��ositions�b���a�secon��,�lar��er��air,��ith�sti�ulus-onset�as��nchron���of�either�83��s�or�167��s.�This�cause����etacontrast��as�in���of�the�first��air,��ore�or�less�co��letel���(83��s)�or�inco��letel���(167��s).�Left/ri��ht�choice�res�onses�ha���to�be��a��e�accor��in���to��osition�of�the�tar��et�in�the�vis-ible�sti�ulus.�(Tar��et��as���ia�on���or�square,�alternatin���bet�een�subjects).�The��as�e���sti�uli�coul���be�co��atible�or�inco��atible��ith�the�visible�sti�uli�(or�neutral,�not���is�la��e���here).�See�text�for���escri�tion�of�the�ERP�results.�Bol���lines�are�fro��trials��ith�167��s�SOA,�thin�lines�fro��83��s�SOA.�Blac��lines���enote�co��atible�trials,���re���lines�inco��atible�trials.

Page 59: Unconscious Memory Representations in Perception

4�� Rolf�Verle��er

Of�interest,�this�N2�c��as�onl���evo�e����ith�167��s�SOA,��hich�is��hen�the��as�e��� sti�ulus� re�aine��� visible� b��� its� relativel��� lon��� interval� to� the� follo�-in����as�in���sti�ulus,�but��as�insi��nificant��ith�83��s�SOA��hen�the��as�e���sti�ulus�coul���not�be�i��entifie��.�Thus,�this�N2�c(�as�e��)���istin��uishe���bet�een�sti�uli�that��ere�consciousl����erceive���an���sti�uli�that��ere�not.

Nevertheless,�Jaś�o�s�i�et�al.�(2002)��ere�reluctant�to�consi��er�N2�c�(ter�e���PCN�in�that��a�er)�as�a���irect�reflection�of�a�areness,�an����rote:�“We���o�not��ant�to�su����est�that�PCN�is�a���irect�reflection�of�conscious��erce�tion,�rather�it� �i��ht� reflect� a� �rocess� necessar��� for� conscious� �erce�tion,� na�el��� that� at-tention�be���irecte���to�the�sti�ulus”�(�.�53).�In��ee��,�a��reciable�N2�c��otentials��ere�evo�e���b����as�e���uni��entifie���sti�uli�in�stu��ies�of�Jaś�o�s�i�et�al.�(2003)�an���Woo���an�an���Luc��(2003)��hich��a�es�it���ifficult�to�associate�N2�c��ith�a�areness.�

2.5 ERP signature of conscious identification of T2 in the Attentional Blink Paradigm

A� s�ecial� case� of� �as�e��� visual� sti�uli� is� �rovi��e��� b��� the� “Attentional� Blin�”��ara��i���.�T�o�tar��et�sti�uli�(“T1”�an���“T2”)�have�to�be�i��entifie����ithin�a�ra�i���strea�� of� sti�ulation.� I��entification� of� T2� is� co��ro�ise��� b��� the� necessit��� to�i��entif���T1�in�a�ti�e-��e�en��ent��anner:�With�sti�uli�ra�i��l����resente���at�a�rate�of�10/secon���(see�Potter�2006,�for�so�e�variation�of�this��resentation�rate)�T2�is�i��entifie����ell�if��resente�����irectl���after�T1�(“la��-1-s�arin��”),��orst�at�an�inter-�e��iate�interval�of�200–300��s�after�T1,�an���then�a��ain�better��hen��resente���at� lon��er� la��s.�Exa��les�are���iven� in�Fi��ures�9a�an���10a.�Due�to�this� te��oral�s�ecificit��,�this�i��air�ent�has�been�na�e���a�“blin�”,�i.e.,�a�te��orar���bloc�in���of� in�ut��rocessin��.�Debate� is�on��oin����hether� this�attentional�blin�� is���ue� to�ca�acit���li�itations�of��or�in����e�or���(Dell’Acqua�et�al.�2009)�or���ue�to�T1-in-��uce���over-attentive��rocessin���of���istractors�(Olivers�2007)�or���ue�to��roble�s�in�shiftin��� to� the�ne��search�criterion�nee��e��� for���etectin���T2�(Di�Lollo�et�al.�2005;�Nieu�enstein�an���Potter�2006).�This���ebate�not�ithstan��in��,�it�is�evi��ent�that�the�attentional�blin��is��ainl���cause���b���interference�in�co��nitive��rocess-in���rather�than�b����erce�tual��as�in��,�because�T2�usuall���can�be��ell�re�orte���in�the�ver���sa�e�sequence�of�sti�uli�if�the�instruction�sa��s�that�onl���T2�has�to�be�re�orte���an���T1��a���be�i��nore���(see�Fi��ures�9b�an���10b).

Because�of�the�fast�re��ular�sequence�of�sti�uli,�a priori�the�earl���P1�an���N1�co��onents� evo�e��� b��� T2� are� �oor� in��icators� of� �erce�tion� because� the��� un-��er��o�habituation���ue�to�the��rece��in���sequence�of�sti�uli.�Moreover,�all�co��o-nents�are�affecte���b���overla��of��otentials�evo�e���b���the�fast�series�of�consecutive�

Page 60: Unconscious Memory Representations in Perception

� Mar�ers�of�a�areness?� 49

sti�uli,�si�ilarl���to�the�above-��escribe���sequences�of��as�e���an����as�in���sti�-uli.�To���eal��ith�the��roble��of�overla�,�ERP�effects�have�often�been��easure���in���ifference��otentials��ith�so�e�a��ro�riate�control�con��ition�bein���subtracte��,�e.��.�in�Fi��ure�9c.

Figure 9. “Attentional�blin�”�in�ra�i���serial�visual��resentation:�Para��i����an�����ata�fro��Vo��el�et�al.�(1998,�Ex�.�4).�(a)�T�o�tar��ets,�“T1”�an���“T2”,��ere��resente����ithin�a�series�of�ra�i��l���(83��s�SOA)��resente���blac��letters.�T1��as�a�blac����i��it�an���T2��as�a��hite�letter,�either�an�E�(in�15%�of�trials)�or�another�letter.�Inters�erse���bet�een�T1�an���T2��ere�0,�2,�or�6�sti�uli,�=�“la��s”�1,�3,�or�7.�(b)�When�both�T1�an���T2�ha���to�be�i��entifie���(soli���lines)�the�E�coul���be�i��entifie����ith�chance��robabilit���onl���at�la���3.�(Dashe���line:�Bloc���here�onl���T2�ha���to�be�i��entifie��).�(c)�Corres�on��in��l��,�in�the�ERPs,�there��as�no�P3�elicite���b���the�E�in�la���3�trials�(��ensel�����ashe���line).�Soli���line:�la���1�trials.�Dashe���line:�la���7�trials.�Ti�e-�oint�0��s�is�T2�onset.�Sho�n�are���ifference��avefor�s�bet�een�the�infrequent�E�trials�an���other�trials.�Recor��in���is�fro��Pz�versus�avera��e����astoi��s.�Bein���the�infrequent�tar��et,�the�E��as�ex�ecte���to�evo�e�a�P3,��hich�it���i���in�la���1�an���la���7�trials�but�not�in�la���3�trials.�(Parts�(b)�an���(c)�of�this�fi��ure��ere�a��a�te����ith��er-�ission�fro��Fi��s.�8�an���9�of�Vo��el�et�al.�1998.)

Page 61: Unconscious Memory Representations in Perception

50� Rolf�Verle��er

Figure 10. “Attentional�blin�”�in�ra�i���serial�visual��resentation:�Para��i����an�����ata�fro��Kranczioch�et�al.�(2003,�2007).�(a)�T�o�tar��ets,�“T1”�an���“T2”,��ere��resente����ithin�a�series�of�ra�i��l���(100��s�SOA)��resente���blac��letters.�T1��as�a���reen�let-ter�–�vo�el�or�consonant�–�an���T2��as�an�X,��resente���in�75%�of�the�trials.�Inters�erse���bet�een�T1�an���T2��ere�0,�1,�or�6�sti�uli,�=�“la��s”�1,�2,�or�7.�(b)�When�both�T1�an���T2�ha���to�be�i��entifie���(o�en�circles)�the�X��as�i��entifie���belo��chance��robabilit���at�la���2.�(Blac��circles:�Bloc���here�onl���T2�ha���to�be�i��entifie��).�(c)�an���(��)�ERPs�in�trials��ith�correctl���res�on��e���T2�(left)�an���in�trials��ith��isse���T2�(ri��ht).�Ti�e-�oint�0��s�is�T2�onset.�(c)�P3�effect:�In�the�ERPs�to�correctl���res�on��e���T2,�P3��as�s�allest��ith�la���1�(blac��line)�an���lar��est��ith�la���7�(��ar����re��),��ith�la���3�(li��ht���re��)�ran��in���in-be-t�een.�In�the�ERPs�evo�e���b����isse���T2,�no���istinct��otentials�are�visible.�Recor��in���is��oole���fro��Cz�an���nei��hborin���electro��es�versus�avera��e���recor��in��s.�(��)�N2�effect:�At�the�sha��e���latenc��,�there�is�a�ne��ative-��oin�����ifference�(u��er�blac��line)�bet�een���etecte���T2�trials�(��ar��line�in�the�left��anel)�an���trials��ithout�T2�(��re���line).�Recor��-in���is��oole���fro��left��arieto-occi�ital�electro��es�versus�avera��e���recor��in��s.�(Parts�(b)�an���(c)�of�this�fi��ure��ere�a��a�te����ith��er�ission�fro��Kranczioch�et�al.�2003,��art�(��)�fro��Kranczioch�et�al.�2007.)

Page 62: Unconscious Memory Representations in Perception

� Mar�ers�of�a�areness?� 51

2.5.1 The�P3�co��onent

Since�the�stu�����of�Luc��et�al.�(1996)�that��easure���ERPs�in�this��ara��i����for�the�first�ti�e�(focusin���on�the�N400�co��onent,�see�belo�),�several�stu��ies�a��ree���that� it� is� s�ecificall��� the�P3�co��onent� that� is� sensitive� to�success�or� failure� to�re�ort�T2.�T�o�ar��u�ents�su��ort�this�notion.�First,�P3�is�su��resse����hen�T2�is��resente���at�the�critical��osition�co��are���to�lon��er�la��s�(Vo��el�et�al.�1998;�Vo��el�an���Luc��2002;�Dell’Acqua�et�al.�2003;�A���üre��et�al.�2007;�Sessa�et�al.�2007).�One��i��ht� ar��ue� that� this� su��ression� is� ��ue� to� the� short� T1-T2� ��istance� of� about�300��s�causin���so�e�habituation�of�the�P3���enerator.�Of��articular�i��ortance,�therefore,�is�that�P3�at�the�critical�la����as�even�su��resse���in�co��arison�to�the�even�shorter�la���1�(Vo��el�et�al.�1998).�Vo��el�et�al.’s�result�is���e�icte���in�Fi��ure�9.

This� latter� result� �as,� ho�ever,� onl��� once� re�licate��,� in� the� MEG� stu����� of�Kessler�et�al.�(2006)�an���there�for�ri��ht�sources�onl��.�In�contrast,�Kranczioch�et�al.�(2003)��hich�is�to������no�le����e�the�onl���other�ERP�stu�����that�co��are���T2-evo�e���P3�bet�een�la���1�an���other�la��s,�obtaine���an�increase�of�P3�a��litu��es�across�la��s,�fro��no�P3�at�all�at�la���1�via��i����le-size���P3�at�la���2�(�hich�is��here�the�attentional�blin���as�lar��est�in�their��rece��in���behavioral�stu����)�to�lar��e�a�-�litu��es�at�la���7.�These���ata�are���e�icte���in�the�left�half�of��art�(c)�of�Fi��ure�10.�The�secon��,��ore�s�ecific�ar��u�ent�to�su��ort�the�notion�that�P3�is�a�correlate�of�the�failure�to�re�ort�T2�is�that,�at�the�critical�la��,�T2-evo�e���P3�is�su��resse���in� trials��here�T2�cannot�be�re�orte���co��are���to� trials��here�T2� is�re�orte���(Rol�e�et�al.�2001;�Kranczioch�et�al.�2003,�2007;�Ser��ent�et�al.�2005;�Martens�et�al.�2006a,�2006b;�Pesciarelli�et�al.�2007;�Koivisto�an���Revonsuo�2008).�This�result�is�illustrate���b���the�co��arison�bet�een�left�an���ri��ht�halves�of�Fi��ure�10c.�Thus,�the�co��lete�su��ression�of�P3�a��litu��es�in�Vo��el�et�al.’s�(1998)�la��-3���ata�(Fi��-ure�9c)�is��erha�s�a�consequence�of�all�trials�bein���inclu��e���in�their�avera��es,�ir-res�ective�of�T2�i��entification,�an���that,�therefore,�their�la���3���ata��ainl���consist�of�trials�in��hich�T2��as��isse���(cf.�Fi��ure�9b).

2.5.2 Earlier�co��onents:�N2,�P2,�P1,�N1

So�e�stu��ies��easure���an�earlier�N2-t���e�co��onent�in�T2-�inus-control���if-ference��aves.�In��ee��,�this��osterior-lateral,�left-enhance���N270��as�lar��er��hen�T2��as�i��entifie���than��hen�it��as�not�(Ser��ent�et�al.�2005;�Kranczioch�et�al.�2007;�Koivisto� an��� Revonsuo� 2008).� The� result� is� illustrate��� in� Fi��ure� 10��� �ith� ��ata�fro��Kranczioch�et�al.�(2007;�sa�e��ara��i����as�use���in�Kranczioch�et�al.�2003,�that�is�sho�n�in�the�other��arts�of�Fi��ure�10).�Ser��ent�et�al.�(2005),�a����itionall���

Page 63: Unconscious Memory Representations in Perception

52� Rolf�Verle��er

��escribe���a�subsequent,��ore�centrall���locate���N300,��hich��a���be�va��uel����er-ceive���also�in�the�left��anel�of�Fi��ure�10c.�

A�fronto-centrall����axi�u��P250��as�s�aller��hen�T2��as��resente���at�the�critical�la���3�than�at�other�la��s�(Vo��el�an���Luc��2002;�Vo��el�et�al.�1998),�but���i���not���iffer�bet�een���etecte���an���un��etecte���ite�s�(Kranczioch�et�al.�2003).

The�earl���P1�an���N1�co��onents�–�as�far�as��easurable�in�the��resence�of�habituation���ue�to�the�fast��resentation�–���i���not���iffer�bet�een�i��entifie���an���uni��entifie���T2�(Ser��ent�et�al.�2005)�or�bet�een�T2��ositions�(Vo��el�et�al.�1998).�

2.5.3 N400

Wor��s�are�ex�ecte���to�evo�e�an�N400�co��onent�(Lau�et�al.�2008).�N400�is�sensi-tive�to��ri�in��:�When�so�e�earlier��resente����or���ha���a��eanin���to��hich�the��resent��or���is�relate���then�N400�is�re��uce���(Kutas�an���Hill��ar���1980).�Of�i�-�ortance�–�in�the��resent�context,�b���bein�����e�en��ent�on�the��eanin���of�the�t�o��or��s,�this�su��ression�of�N400�in�interrelate����or��s�in��icates�that�both��or��s��ere��erceive���to�the��oint�that�their��eanin����as�bein����rocesse��.�A��l��in���this�rationale�to�the�attentional�blin�,�Luc��et�al.�(1996;�sa�e���ata�re�orte���in�Vo��el�et�al.�1998)��resente���one��or���before�the�fast�sequence�of�sti�uli,�as�“T0”�[����ter�,�R.V.]�an���another��or���as�T2�(a�i��st�senseless�letter�strin��s�as���istractors,��rece��e���b���a���i��it�strin���as�T1).�N400�evo�e���b���T2��as�re��uce����hen�T2��as�relate���to�T0.�Criticall��,�this�re��uction�of�N400���i���not���iffer�bet�een�la��s�even�thou��h�T2��or��s��ere�consciousl��� i��entifie���at� the�critical� la��� in�65%�of� trials�onl��,�bein���subject�to�the�attentional�blin�.�Luc��et�al.�(1996)�conclu��e���that�ac-cess�of��or����eanin����a����rocee����ithout�conscious�a�areness.�Giesbrecht�et�al.�(2007)���e�onstrate���that�there�is�in�fact�a�threshol���for�this�non-conscious��ro-cessin���of�se�antics:�N400�su��ression��as�abolishe����hen��erce�tual�loa���as-sociate����ith�T1�i��entification�increase��.�A��arentl���then�the�“attentional�blin�”�beca�e�so�intensive�as�to�involve�a�“se�antic�blin�”�[����ter�,�R.V.].

Of�note,�even�un��etecte����or��s�at�T2��osition�can�on�their�o�n��ri�e��ro-cessin��� of� a� thir��� �or��� (T3)� �resente��� after� the� trial,� re��ucin��� the� T3-evo�e���N400��hen�T2�an���T3�are�relate���(Rol�e�et�al.�2001).�Althou��h�clearl����resent,�this��ri�in���effect�on�the�ERP�evo�e���b���T3�ten��e���to�be�s�aller��hen�T2��as��isse���than��hen�T2��as�i��entifie���(Rol�e�et�al.�2001)�an��,�so�e�hat�sur�ris-in��l��,�in�a�re�lication�of�this��ara��i����(Pesciarelli�et�al.�2007)��as�restricte���to�an�earlier��ortion�of�the�T3-evo�e����avesha�e,�aroun���the�P2��ea��at�270��s,��hen�T2� �as� �isse��,� in� contrast� to� a� lon��-lastin��� �o��ulation� of� the� T3� �avesha�e,�inclu��in��� the�N400,��hen�T2��as� i��entifie��.�Ta�en� to��ether,� these���ata��here�T2�acts�as�a��ri�e�su��ort�the�conclusions�of�the�above�stu��ies�(Luc��et�al.�1996;�

Page 64: Unconscious Memory Representations in Perception

� Mar�ers�of�a�areness?� 53

Giesbrecht�et�al.�2007)�that�the��eanin���of�the�uni��entifie���T2��or���is��rocesse���u��to�a�certain���e��ree.

2.5.4 N2�c

So�e�recent� stu��ies�a��lie��� the�N2�c� rationale,�as���escribe���above� (Fi��ure�8),�to� the� attentional-blin�� �ara��i���.� Bein��� a� correlate� of� selective� �rocessin��� of�events,�N2�c��i��ht�be�a���oo���in��icator�of��hether�T2�sti�uli��ere���iscri�inate���a�on���the�strea��of�other�sti�uli.�In�or��er�to��easure�N2�c�in�the���ifference�fro�� recor��in��s� at� �osterior� sites� contralateral� an��� i�silateral� to� the� relevant�event,�the�stan��ar���tas��has�to�be�chan��e��,�thou��h.�Sti�uli�have�to�be��resent-e���laterall��,��ith�the�relevant�sti�ulus��resente���on�one�si��e�an���an�irrelevant�sti�ulus�on�the�other�si��e.�Thus,�there��ust�be�t�o�strea�s�of�sti�uli,�one�left�an���one�ri��ht,�at� least�fro��T2�on�ar��s.�Fi��ure�11�illustrates�the�tas��use���b���Verle��er�et�al.�(2009).

Figure 11. Dual-strea��ra�i���serial�visual��resentation�(Verle��er�et�al.�2009).�In�this��o��ification�of�the�attentional-blin���ara��i���,�T1�an���T2�are��resente���left�or�ri��ht�fro��fixation.�T1�(here���e�icte���as��hite)��as�re��.�The���ra�h��resents���ata�on�trials�in��hich�T1�an���T2��as�correctl���i��entifie��,�ex�resse���as��ercenta��e�of�all�trials�in��hich�T1��as�correctl���i��entifie��.�These���ata�are�fro��Ex�.�1�of�Verle��er�et�al.�(2009).�

Page 65: Unconscious Memory Representations in Perception

54� Rolf�Verle��er

Selectin��� bet�een� these� t�o� strea�s� co��licates� the� tas�,� �hich� beco�es�obvious��ith�“la��-1-s�arin��”:�While�T2�is�i��entifie���as��erfectl���as�in�the�ori��i-nal�tas���hen�i��e��iatel���follo�in���T1�in�the�sa�e�strea�,�this�la��-1-s�arin���is�abolishe���an���i��entification�rates���rasticall�����ecrease��hen�T2�follo�s�T1�at�a���ifferent�location�(e.��.,�Breit�e��er�et�al.�1999;�Verle��er�et�al.�2009;�see�Fi��ure�11).�These�consi��erations�not�ithstan��in��,�results�obtaine���for�N2�c�loo�e����ro�is-in���in�such��o��ifie���versions�of�the�attentional-blin��tas�.�

A��l��in���this�rationale�for�the�first�ti�e,�Jolicœur�et�al.�(2006)�obtaine����ar-allel��atterns�of� i��entification�rates�an���N2�c:�T2��as��ell� i��entifie���an���elicit-e��� a� lar��e� N2�c� �hen� T1� coul��� be� i��nore��,� i��entification� �as� �orse� an��� N2�c��as�s�aller��hen�both�T1�an���T2�ha���to�be�i��entifie��,�an���this��as��articularl���true��hen�the�la���bet�een�T1�an���T2�a�ounte���to�t�o�sti�uli,�co��are���to�a�la���of�ei��ht�sti�uli.�Parallel�effects�on�N2�c�an���behavior��ere�also�obtaine���b���Dell’Acqua�et�al.�(2006)�an���Robitaille�et�al.�(2007).�Dell’Acqua�et�al.�(2006)�use���one�T1-T2�la���onl���(150��s).�N2�c�an���i��entification�rates��ere�re��uce����hen�T1�ha���to�be��rocesse���co��are���to��hen�T1�coul���be�i��nore��,�an����ere�further�re��uce����hen��rocessin���loa���of�T1��as�hi��h.�Si�ilarl��,�usin���a�fixe���T1-T2�in-terval�of�350��s,�Robitaille�et�al.�(2007)�obtaine���re��uce���i��entification�of�T2�in��arallel��ith�sli��htl���but�reliabl���re��uce���N2�c�to�T2��hen�T1�require���an�infre-quent�rather�than�a�frequent�res�onse,�thereb����robabl�����e�an��in����ore�ca�ac-it���for�its��rocessin���an���leavin���less�ca�acit���for��rocessin���T2.�

Thus,�these�stu��ies�conver��e�on�havin���N2�c�as�an�ERP�reflection�of�the�at-tentional�blin�,�si�ilar�to�P3,�but�earlier�than�P3�an���therefore��ossibl���closer�to�the�actual�reason�for�the�failure�in��rocessin��,�as�a�failure�to�select�T2�fro��a�on���the���istractors.�Ho�ever,�none�of� these� stu��ies���e�onstrate��� that�N2�c�closel���covarie����ith�i��entification�b���sho�in���that�N2�c��as�lar��er��hen�T2��as�i��enti-fie���than��hen�it��as��isse��.�Dell’Acqua�et�al.�(2006)�actuall���re�orte���this�co�-�arison�but�coul���not�obtain�si��nificant���ifferences�in�N2�c�a��litu��es,��ossibl�����ue�to�the��oor�si��nal/noise�ratio�that�is�t���ical�for���ifference��otentials.�This�is�in�contrast�to�P3��here�this�co��arison�ha���been���one,�as�note���above.�Moreover,�to� exclu��e� the� objection� that� N2�c� re��uction� �ith� the� critical� la��� co��are��� to�lon��er�la��s�is�si��l�����ue�to�habituation�of�the�N2�c���enerator�at�short�la��s,�N2�c�shoul���be�lar��e�at�la���1��hen�there�is�la��-1�s�arin��.�Fi��ure�12���is�la��s�results�of�our�stu�����(Verle��er�et�al.�2009)��here�N2�c��as��easure���at�la��s�1,�2,�an���5.�

When�T1�an���T2��ere�on�the�sa�e�si��e,�N2�c�a��litu��es��ere�s�aller�at�the�short�la��s�1�an���2�than�at�la���5.�This�is�in�contrast�to�i��entification�rates,��hich��ere�hi��her�at�la��s�1�an���2�than�at�la���5�(Fi��ure�11).�When�T2�an���T1��ere�on���ifferent�si��es,�N2�c�a��litu��es�beca�e��ore�si�ilar�to�i��entification�rates.�To���etail,��hen�T2��as�on�the�ri��ht,�both�N2�c�a��litu��es�an���i��entification�rates��ere�re��uce���at�la��s�1�an���2�co��are���to�la���5.�When�T2��as�on�the�left,�both�

Page 66: Unconscious Memory Representations in Perception

� Mar�ers�of�a�areness?� 55

Figure 12. Dual-strea��ra�i���serial�visual��resentation:�Results�on�N2�c�fro��Verle��er�et�al.�(2009).�Results�of�N2�c�a��litu��e�in�the��ara��i������e�icte���in�Fi��.�11.�Data�are�fro��Ex�.�1�of�Verle��er�et�al.�(2009).�N2�c��as��easure���se�aratel���for�left�T1,�in�the�avera��e�across�trials�of�the���ifference�PO8-PO7,�an���for�ri��ht�T1,�in�the�avera��e�of�the���ifference�PO7-PO8,�se�aratel���for�sa�e-si��e�an�����ifferent-si��e�T1�an���T2.�These���if-ference��avefor�s�are���e�icte���in�the�u��er��anels.�There,�0��s�is�T1�onset,�T2�onset�is���enote���b���an�arro���ointin�����o�n�on�the�x-axis.�The�first�ne��ative��ea��is�the�N2�c�evo�e���b���T1.�This�is�follo�e���in�the�Left-T1���ata�(u��er�ro�)�b���a�secon���ne��ative��ea�,��ost��robabl���evo�e���b���the�sti�ulus�that�follo�s�T1.�N2�c�evo�e���b���T2�is�then�seen�as�a���iver��ence�of�the��avefor�s�for�left�T2�an���ri��ht�T2�(bol���an���thin�lines)�at�200-300��s�after�T2�onset�(“T2-evo�e��”).�In�the�lo�er���ra�h,�a��litu��es�of�T2-evo�e���N2�c�are�sho�n,���efine���as��ean�a��litu��es�200-300��s�after�T2�onset.�To��easure�these�a��litu��es�at�la��s�1�an���2,�the��avefor�s�of�la���5��ere�subtracte��.�To��easure�the�N2�c�a��litu��es�at�la���5,��ean�a��litu��es�at�600–650��s��ere�subtracte��.

Page 67: Unconscious Memory Representations in Perception

5�� Rolf�Verle��er

N2�c�a��litu��es�an���i��entification�rates��ere�hi��her�than��hen�T2��as�on�the�ri��ht.�Ho�ever,� even� in� this� case�N2�c�a��litu��es��ere�at� least� as� lar��e�at� la��s�1�an���2�as�at�la���5�althou��h�i��entification�rates�of�T2��ere��orse�at�la��s�1�an���2�than�at�la���5.�This�latter��inor���issociation�not�ithstan��in��,�for���ifferent-si��e�T1�an���T2�one��i��ht��ee��the�h���othesis�that�N2�c�a��litu��e�reflects�i��entification�rate.�For�sa�e-si��e�T2�the�relationshi��bet�een�i��entification��erfor�ance�an���N2�c�a��litu��e���oes�not�hol���because�N2�c�is��uch�re��uce����ith�the�short�la��s�1�an���2.�This�latter�result��i��ht�reflect�saturation�of�the�N2�c���enerator.�More���ata�are�nee��e���to�further�eluci��ate�the�role�of�N2�c.

It��a���be��entione���that�in�Jolicœur�et�al.’s�(2006)�stu����,�the��re�on��erance�of�ne��ativit���contralateral�to�T2�that��as��hasicall����anifeste���in�the�N2�c��ea��continue�����urin���the�entire��resentation�of���istractor�sti�uli� follo�in���T2.�B���an��� lar��e,� this� tonic�contralateral��re�on��erance,� ter�e���SPCN�b���Jolicœur�et�al.�(“sustaine����osterior�contralateral�ne��ativit��”),�varie����ith�the�ex�eri�ental�variables�in�that�stu�����as��ell�as�in�follo�in���ones�in�the�sa�e��a���as�N2�c.

2.� ERP signatures of preparatory states favorable for conscious awareness

2.�.1 Detection�of�faint�sti�uli

So�e� stu��ies� searche��� for� EEG� correlates� of� brain� states� that� �oul��� increase�the��robabilit���of���etectin���an�u�co�in���faint�si��nal.�Lutzenber��er�et�al.�(1979)�foun���that�a��o��erate�increase�of�sustaine���ne��ativit���(�easure���at�Cz�an���Fz)��as�relate���to�better�i��entification�of�a���a��in�the�outline�of�the�sche�atic�“roc�et”�that��ove���across�the�screen.�Ho�ever,�to�co��licate��atters,�this�roc�et�serve���at�the�sa�e�ti�e�as�fee��bac��si��nal�for�the�level�of�cortical�ne��ativit��,�so�the�results��i��ht�have�been�affecte���b���the��anner�ho���artici�ants���ealt� in�a���iven�trial��ith�the���ual�tas��of�controllin���their�brain�ne��ativit���an�����etectin���the�si��nal.�

Si�ilarl��,���etection� of� a� faint�brief� laser��oint� in� the� tas�� illustrate���above�in�Fi��ure�5��as��ore�li�el����hen�the�DC�level�of�EEG��as��ore�ne��ative�before�sti�ulus�onset�(Devri��et�al.�1999).�Data��ere�re�orte���fro��Oz�recor��in��s�onl��,�althou��h�EEG��as��easure���fro��other� locations,� too.� It��oul���be� interestin���to��no���hether�this�relation��as�in��ee���s�ecific�to�the�visual�cortex,�as�is�su��-��este���b���the�Oz�site.�Anal��zin���another�feature�of�these�sa�e���ata,�Er��enoğlu�et�al.�(2004)�re�orte���that�al�ha��o�er��as�lo�er�before�correctl�����etecte���si��nals,�s�ecificall���at��osterior�sites�above�visual�cortex�(�here�al�ha�is��ost��ronounce���an���a��,�so�the�s�ecificit���of�this�relation��i��ht�a��ain�be���oubte��).

Page 68: Unconscious Memory Representations in Perception

� Mar�ers�of�a�areness?� 57

Increase���ne��ativit���an���re��uce���al�ha��i��ht�be� reasonabl��� inter�rete���as�si��ns�of�increase����re�arator���state.�Therefore,�these�results��a���be��lausibl���in-ter�rete���as�in��icatin���that�increase����re�arator���state�is�hel�ful�in���etectin���faint�si��nals.�The���ata���o�not�s�ea��a��ainst�the��ossibilit���that�this�increase����re�ara-tor���state�is�s�ecific�to�the�visual�cortex,�but��a���also�be�inter�rete���as�reflections�of�hei��htene�����eneral�arousal.

2.�.2 Attentional�blin�

Si�ilar�evi��ence�has�been�collecte���for�the�attentional-blin��tas�.�It��as�inves-ti��ate����hether�subjective�states�before�the�trials,�as�reflecte���in�EEG�rh��th�s,�are�favorable�for�T2�i��entification.�Kranczioch�et�al.�(2007)�foun���that���ecrease���coherence�in�the�10�Hz,�13�Hz,�an���20�Hz�ban��s�before�T1�onset�an���increase���coherence�in�the�13�Hz�an���20�Hz�ban��s�bet�een�T1�an���T2��as�a��vanta��eous�for���etectin���T2.�Generall���increase���coherence�in�the�40�Hz�ban��,�both�before�T1�an���bet�een�T1�an���T2,��as�re�orte���b���Na�atani�et�al.�(2005)�to�be�a��vanta-��eous�for���etectin���T2.

The� attentional-blin�� �ara��i���� offers� o��ortunities� over� an��� above� the��as�in���stu��ies� for� investi��atin����recon��itions�of�sti�ulus� i��entification:�Be-cause�the��ain���eter�inant�of�T2�i��entification�is�the��rocessin���of�T1,�the�brain�res�onse�to�T1��a���be�a�crucial���eter�inant.�This��ossibilit����as�hi��hli��hte���b���McArthur�et�al.�(1999)��ho�li�ene���the�ti�e-course�of�the�attentional�blin��to�the�P300��avefor��elicite���b���T1�i��entification:�Just�as�the�P300��avefor��reaches�its��ea��rather�abru�tl���an���then���ra��uall���returns�to�baseline,�i��entifi-cation�rates�of�T2���ecrease�rather�abru�tl���to�ar��s�their��ini�u���ith�increas-in�����istance�of�T2�fro��T1�an���then���ra��uall���recover.�Thus,�McArthur�et�al.�(1999)�su����este���that�the�state�of�cerebral��ositivit���evo�e���b���T1�an���in��icate���b���P3��i��ht�be�a�cause�for�the�attentional�blin�.�In��ossible�conflict��ith�this�no-tion,�size�of�P3�evo�e���b���T1���i���not�correlate�across��artici�ants��ith�size�of�the�attentional�blin��in�McArthur�et�al.’s�(1999)���ata,�nor�in�another�EEG�stu�����that�teste���for�this�relationshi��(Martens�et�al.�2006b).�Such�correlation��as,�ho�ever,�obtaine���in�an�MEG�stu�����(Sha�iro�et�al.�2006).�The�latter�fin��in����i��ht�be�a�chance�result�but,�on�the�other�han��,�MEG�is�not�as�affecte���as�EEG�b���irrelevant�interin��ivi��uall��� var��in��� �ara�eters� li�e� s�ull� thic�ness,� len��in��� so�e� cre��-ibilit��� to� this�fin��in��.� In� the� sa�e�vein,�Sha�iro�et� al.� (2006)�an���Kranczioch�et�al.� (2007)�re�ort� that�T1-evo�e���P3� is� lar��er��hen�T2� is��isse��� than��hen�it� is� i��entifie��,��hich�i��lies�so�e�tra��e-off�of�ca�acities�bet�een�T1�an���T2.�Ho�ever,� the� T1-effect� ��i��� actuall��� not� reach� si��nificance,� neither� in� Sha�iro�et�al.�(2006)�nor�in�the�recor��in��s�actuall�����is�la��e���b���Kranczioch�et�al.�(2007,�

Page 69: Unconscious Memory Representations in Perception

5�� Rolf�Verle��er

C.�Kranczioch,��ersonal�co��unication,�Oct.�1,�2008).�More�una�bi��uous�than�a��litu��e�results�are�available���ata�on�P3�latenc��:�The��ea��of�T1-evo�e���P3��as�earlier�intrain��ivi��uall����hen�T2��as�i��entifie���than��hen�it��as�not�(Ser��ent�et�al.�2005)�an��,�bet�een�in��ivi��uals,�in��artici�ants��ho�coul����ell�i��entif���T2�co��are���to��artici�ants��ho�suffere���fro��the�attentional�blin��(Martens�et�al.�2006a).�Thus,�these���ata���o�su��ort�a�role�of�ti�in���of�T1��rocessin��,�as�reflecte���b���P3�latenc��,�for�i��entif��in���vs.��issin���T2.�

2.7 ERP signatures of motor activity evoked by indistinguishable signals

Mas�e��,�uni��entifie���sti�uli��a���evo�e��otor�activation.�This� is� illustrate��� in�Fi��ure�13�b�����ata�fro��Jaś�o�s�i�et�al.�(2002)�alon���the�lines�first���escribe���b���Leuthol���an���Ko���(1998).�The�left�or�ri��ht��e���ha���to�be��resse���in�res�onse�to�the�visible�sti�ulus��air�accor��in���to�instruction�(alternatin���across��artici�ants:��osition�of�either�square�or���ia�on����as�relevant,�an���to�/botto���as��a��e���to� left/ri��ht�or� to�ri��ht/left).� I��ortantl��,� this�visible��air��as��rece��e���b���t�o�s�aller�sha�es��hich��ere��as�e���b���the�visible�sha�es�throu��h��etacontrast.�The��osition�of�the�tar��et�sha�e�in�the��as�e����air��ro��uce���an�earl�����eflection�in�the�Lateralize���Rea��iness�Potential�(LRP),�i.e.,�in�the���ifference��otential�be-t�een�the�t�o��otor�cortices:�The�LRP�beca�e��ore�ne��ative�at�the��otor�cortex�contralateral�to�the�han���that��oul����ress�the��e���that��as�assi��ne���to�the�loca-tion�of�the�tar��et�sha�e.�Thus,��hen��olarit���of�the�LRP��as���efine����ith�res�ect�to�the�han���contralateral�to�the�ensuin���tar��et,�this�earl�����eflection�evo�e���b���the��ri�es��ent�to�the��ron�����irection��hen��ositions�of�the�tar��et�sha�es���iffere���bet�een� �as�e��� �ri�es� an��� �as�in��� visible� sti�ulus,� an��� starte��� alrea����� ��o-in���to�the�ri��ht���irection��hen��ositions�of�the�tar��et�sha�es��ere�the�sa�e�for��ri�es�an���visible�sti�uli.�

This�result�sho�s�that�the��otor�s��ste���a���be�tri����ere���alrea�����b���sti�uli�that�are�not�consciousl����erceive��,�at�least�in�such��ri�e-tar��et�situations��here�the��ri�es�occur�in�a�te��oral��in��o��ver���close�to�the�tar��ets�an����here�sti�u-lus-res�onse�associations�are�si��le.�In��ee��,�this�fin��in���of�an�earl���LRP���eflec-tion�evo�e���b����ri�es�has�often�been�re�licate���un��er�such�situations,��ith�three�t���es�of�sti�uli:�First��hen,�li�e�in�Fi��ure�13,�both�the��as�e����ri�es�an���the��as�in���i��erative�sti�uli��ere�to�-botto��or�left-ri��ht��airs�an���res�onses�ha���to�be��a��e��ith�the��e���corres�on��in���to�tar��et��osition�(Jaś�o�s�i�et�al.�2002;�Jaś�o�s�i�et�al.�2003;�Klotz�et�al.�2007;�Leuthol���an���Ko���1998;�Verle��er�et�al.�2008).�Secon��,��hen�both��ri�es�an���i��erative�sti�uli��ere�arro�s��ointin���left�or�ri��ht�(Ei�er�an���Schla��hec�en�1998;�Ei�er�1999;� Jaś�o�s�i�et�al.�2008;�

Page 70: Unconscious Memory Representations in Perception

� Mar�ers�of�a�areness?� 59

Figure 13. Motor�activation�b����as�e����ri�es�(Data�fro��Jaś�o�s�i�et�al.�2002,�Ex�.�2).�Top:�A�first��air�of�sti�uli,�left�an���ri��ht�fro��fixation,��as�follo�e���at�the�sa�e�screen��ositions�b���a�secon��,�lar��er��air,��ith�sti�ulus-onset�as��nchron���of�either�83��s�or�167��s.�This�cause����etacontrast��as�in���of�the�first��air,��ore�or�less�co��letel���(83��s)�or�inco��letel���(167��s).�Left/ri��ht�choice�res�onses�ha���to�be��a��e�accor��in���to�to�/��o�n��osition�of�the�tar��et�in�the�visible�sti�ulus.�(Tar��et��as���ia�on���or�square,�alternatin���bet�een�subjects).�The��as�e���sti�uli�coul���be�co��atible�or�inco��atible��ith�the�visible�sti�uli�(or�neutral,�not���is�la��e���here).�Bottom:�Difference��otentials�contralateral-i�silateral�to�the�res�on��in���han��.�Bol���lines�are�fro��trials��ith�167��s�SOA,�thin�lines�fro��83��s�SOA.�Blac��lines���enote�co��atible�trials,���re���lines�inco�-�atible�trials.�Diver��ence�of�blac��an�����re���lines,�both�for�bol���an���thin�lines,���enotes��otor��ri�in���b���the��as�e���sti�uli.�The�absence�of�such��atterns�at�|P7-P8|�(lo�er��anel)�s�ea�s�for�s�ecificit���of�this�effect�to�the��otor�s��ste�.

Page 71: Unconscious Memory Representations in Perception

�0� Rolf�Verle��er

Praa�stra� an��� Seiss� 2005;� Seiss� an��� Praa�stra� 2004;� Verle��er� et� al.� 2004).� As�sho�n�b���Ei�er�an���Schla��hec�en�(2003),�these�LRP�effects�are�not�onl�����ue�to�the�intrinsic�tas�-in��e�en��ent���irectional��eanin���of�arro�s�but�to�the�fact�that�arro�s��ere�tar��ets,�because�no�LRP���eflections��ere��ro��uce���b���arro���ri�es��hen� the� i��erative� sti�uli��ere� the� letters�L�an���R.�This�fin��in��� len��s�so�e�a����itional�cre��ibilit���to�the�LRP�effects�obtaine����ith�the�thir�����rou��of�sti�uli�that���o�not�have�an���s�atial�connotations:��ri�e-in��uce���LRP�effects�have�even�been�re�orte����hen��ri�es�an���i��erative�tar��ets��ere�the�s����etrical�s���bols�<>� an��� ><,� one� assi��ne��� to� a� left-,� the� other� to� a� ri��ht-han��� res�onse� (Ei�er�an���Schla��hec�en�2003)�an����hen��ri�es�an���tar��ets��ere�nu�bers�an���nu�ber��or��s�that�require���a�choice�bet�een�left�an���ri��ht�han�����e�en��in���on��hether�the��� �ere� s�aller� or� lar��er� than� five� (Dehaene� et� al.� 1998).� The� latter� fin��in����oul���require�re�lication,�thou��h,��ith�the�LRP�results�havin���been�anal��ze���an����resente���b���so�e�hat�unusual��etho��s.�

Ta�en�to��ether,�these���ata��rovi��e���oo���evi��ence�that��as�e���sti�uli��a����ro��uce� a� res�onse� ten��enc��� in� the� �otor� s��ste�� if� these� �as�e��� sti�uli� are�hi��hl���si�ilar� to� i��erative�sti�uli�an���are��resente���shortl���before� these� i�-�erative�sti�uli.�

2.7.1 Is�the��otor�s��ste���ore�sensitive�than�the��erce�tual�s��ste�?

Thus,�insofar�as��as�e���sti�uli�cannot�be�consciousl�����istin��uishe��,�the��otor�s��ste��is�sensitive�to�sti�ulation�belo��the�threshol���of��erce�tual�a�areness.�

Base��� on� these� fin��in��s� an��� on� behavioral� results� of� �as�e��� �ri�in��,� the��ore�far-reachin���clai��has�been��a��e�that�the��otor�s��ste��nee��e���to�res�on���to� sti�uli� is� more� sensitive� than� the� �erce�tual� s��ste�� nee��e��� for� i��entif��in���sti�uli.�(In�neuro�h��siolo��ical�short-han���notation:�That�the�“��orsal�strea�”�is��ore�sensitive�than�the�“ventral�strea�”.)�Yet,�as�re�orte���further�above,�uni��enti-fie���sti�uli���o��ro��uce�earl����erce�tual�ERP�effects�on�P1,�on�an�N2-t���e�ne��a-tivit��,�an���on�N2�c.�There�is�no�clear���istinction�bet�een�these��erce�tual�effects�an���the�LRP�effects�evo�e���b����as�e���sti�uli:�The�latter�effects���o�not�lea���to�overt�res�on��in���as�the�for�er�effects���o�not�lea���to�a�areness.�It�is�har���to�tell��hich�one�is��ore�sensitive�than�the�other.�

2.7.2 Qualitative���ifferences�bet�een�conscious�an���non-conscious��� �ri�in��?

A�further�controvers���arose�fro��Ei�er�an���Schla��hec�en’s�(1998)���iscover���that�the��ri�e-evo�e���LRP���eflection��as�follo�e���b���a���eflection�of�inverte����olarit���

Page 72: Unconscious Memory Representations in Perception

� Mar�ers�of�a�areness?� �1

�hen�the��as��that�follo�e���the��ri�e��as�not�alrea�����the�i��erative�sti�ulus�(an����hen�so�e�te��oral�an���s�atial�con��itions��ere�fulfille��).�See�Fi��ure�14�for�an�exa��le��ith���ata�fro��our�o�n��or�.�

This� inverte��� LRP� ��eflection� is� in��ee��� functional,� since� res�onses� to� the�follo�in��� i��erative� sti�ulus� are� ��ela��e��� rather� than� s�ee��e��� after� i��entical��ri�es,� an��� s�ee��e��� rather� than� ��ela��e��� after� o��osite� �ri�es.� The� inversion��as� re�licate��� in� a� nu�ber� of� stu��ies� (Ei�er� 1999;� Ei�er� an��� Schla��hec�en�

Figure 14. Polarit���inversion�of�the�LRP�in��as�e����ri�in��.�Top:�Sti�uli�of�the�t���e�first�use���b���Ei�er�an���Schla��hec�en�(1998)�(o�n�un�ublishe�����ata).�Tar��et�arro�s�(botto��fra�e)��ointe���left�or�ri��ht,�requirin���a�corres�on��in����e���ress.�Tar��ets��ere��rece��e���b���brief��ri�es,��hich��ere�also�arro�s,�either�(as�on�the�left)�i��entical�to�the�tar��ets�or�(as�on�the�ri��ht)�the�alternative�ones.�Pri�es��ere�or��ere�not�follo�e���b����as�s�consist-in���of�overla��in���left�an���ri��ht�arro�s.�Bottom:�Difference��otentials�contralateral-�i�silateral�to�the�res�on��in���han���(�ean�of�|C3-C4|�an���|FC3-FC4|,���ran����eans�fro��12���oun���a��ults).�Bol���lines�are�fro��trials��ith��as�e����ri�es,�thin�lines�fro��trials��ithout��as�s.�Blac��lines���enote�co��atible�trials,���re���lines�inco��atible�trials.�In�trials��ithout��as�s,�there�is�lar��e���iver��ence�bet�een�co��atible�an���inco��atible�tri-als,�si�ilar�to�Fi��ure�13.�In�trials��ith��as�s,�the�initial���ifference�of�activation�beco�es�inverte��.

Page 73: Unconscious Memory Representations in Perception

�2� Rolf�Verle��er

2003;�Jaś�o�s�i�et�al.�2008;�Praa�stra�an���Seiss�2005;�Seiss�an���Praa�stra�2004;��Verle��er�et�al.�2004).�We�an���others�clai�e���initiall���that�these�LRP�an���behav-ioral�results�essentiall����ere�an�artifact�of��erce�tual��er��in���of��ri�e�an����as���hich��i��ht��ro��uce�a��erce�t�of�the�alternative��ri�e�(Lleras�an���Enns�2004;�Verle��er�et�al.�2004;�Verle��er�et�al.�2006).�But� this�objection�coul���not�account�for� the�entire��heno�enon�(e.��.,�Schla��hec�en�an���Ei�er�2006).�On�the�other�extre�e,� it��as�clai�e���that� this�“self-inhibition”�(Schla��hec�en�et�al.�2006)�of�the�initial�activation�in��icate���a�qualitative���ifference�bet�een�conscious�an���un-conscious��rocessin���of�sti�uli�(Kla���an���Hin�le���2002).�Our��resent�stance�on�this�issue�(Jaś�o�s�i�et�al.�2008)�is�that�the�inverte���LRP���eflection�in��ee���reflects�inhibitor����rocessin��,�thou��h�not�auto�aticall���tri����ere���as�counterre��ulation�of�the��ri�e-in��uce���activation�(as�initiall����referre���b���Ei�er�an���Schla��hec�en�1998)�but�rather�tri����ere���b���the��as��(cf.�Schla��hec�en�et�al.�2006,�for�a�si�i-lar�vie�)�as�beco�es�obvious��hen�the��as��is�shifte���in�ti�e�(Jaś�o�s�i�et�al.�2008).�The�role�of�the��as��in�invo�in���this�inhibition�a��ears�to�consist�less�in��reventin���the��ri�e�fro��beco�in���visible�but�rather�in�constitutin���an�event�that�is���ifferent�fro���ri�e�an���tar��et�an���cannot�be�i��nore��,�because�a��ea�er�version�of�the�inverte���LRP���eflection�an���follo�in���effects�on�behavior�can�even�be�obtaine����hen�the��as��leaves�the��ri�es�full���visible�(Jaś�o�s�i�et�al.�2008;�cf.�Bo���et�al.�2008).�Thus,�the�inverte���LRP���oes�not�in��icate��a��s�of�subconscious��rocessin���that�are��rinci�all�����ifferent�fro��conscious��rocessin��.�

2.� Conclusions on ERP components as markers of awareness

2.�.1 Earl���visual��otentials:�P1�an���N1

Certainl��,�visual�sti�uli�have�to�be�re��istere���b�����e��icate���visual�areas�of�the�brain�in�or��er� to�reach�a�areness.�The�P1�an���N1�co��onents�are� in��icators�of� this��rocessin���in�the�visual�s��ste�.�Therefore,�their�occurrence�can�be�consi��ere���a�necessar����rerequisite�for�conscious��rocessin��.�(At�least��hen�sti�uli�are��re-sente���in�isolation;�other�ise,��ith�continuous�sti�ulation,�these�earl���co��o-nents��i��ht���isa��ear.)�Ho�ever,�the�clai��state���b���Pins�an���ff��tche�(2003)�that�the�P1�co��onent�is�a�sufficient��ar�er�of�conscious��rocessin���see�s�bol��.�First,��as�e��� uni��entifiable� sti�uli� evo�e� P1� co��onents� that� are� in��istin��uishable�fro���as�e���i��entifiable�sti�uli�(Fi��ure�8,�u��er�ERP��anel).�Secon��,�chan��es�in�sti�ulation�evo�e�an�increase�of�P1�even�if�re�ainin���unnotice���(Ki�ura�et�al.�2006,�2008).�

Page 74: Unconscious Memory Representations in Perception

� Mar�ers�of�a�areness?� �3

2.�.2 N2�c,�N2,�visual�a�areness�ne��ativit��

N2�c� in��icates� to�-��o�n� controlle��� selection� of� the� relevant� feature� (�e�t� in��or�in����e�or��)�in�so�e�laterall����resente���sti�ulus�(Ei�er�1996;�Ei�er�an���Kiss�2008;�Verle��er�an���Jaś�o�s�i�2007;�Wausch�uhn�et�al.�1998).�This�selective��rocessin��� �a��� be� calle��� “attentional”� �rocessin��,� because� �a��in��� attention� to�so�ethin���entails�its�selection�for��rocessin��.�

N2�c� �as� nicel��� relate��� to� conscious� a�areness� of� the� �as�e��� sti�uli� in�Jaś�o�s�i�et�al.�(2002;�Fi��ure�8,�above)�an���behave���si�ilar�to�i��entification�rates�in�the��o��ifie���attentional-blin��tas��exce�t�for�its�habituation-li�e��attern��hen�sti�uli�follo�e���each�other�at�the�sa�e��osition�(Fi��ures�11�an���12).�Thus,�N2�c��i��ht�be�consi��ere���a��ar�er�of�a�areness.�Ho�ever,�there�is�so�e�relevant�evi-��ence� a��ainst� this� notion.� N2�c� �as� unaffecte��� b��� the� co��on-onset� �as�in����roce��ure,� in� contrast� to� conscious� �erce�tion,� in� the� stu����� of� Woo���an� an���Luc�� (2003)� an��� li�e�ise� �as� quite� lar��e� in� the� �as�e��� chains� of� sti�uli� �re-sente���b���Jaś�o�s�i�et�al.�(2003).�Woo���an�an���Luc��(2003)�conclu��e���that�at-tention,�reflecte���b���N2�c,�an���visual�a�areness,�reflecte���b���i��entification�of�the�sti�uli,�can�in��ee���be���issociate��:�The�fact�that�so�e�sti�ulus�attracts�attention�is�not�sufficient�to���uarantee�that�it��ill�be�consciousl����erceive���(cf.�La��e�2003)�althou��h�both��o��es�of��rocessin���are�usuall���relate��.

The��osterior-lateral�N2�c�to�contralateral�sti�uli��resu�abl���catches�in��ure�for��the�as�ect�of�the��osteriorl���recor��e���N2�that�is�relate���to�attention��hen�sti�uli�are�centrall����resente���(Folstein�an���van�Petten�2008).�Such�an�N2,��ith��ostero-lateral�to�o��ra�h���an���a��axi�u��at�about�250��s�after�sti�ulus�onset,��as�also�frequentl���re��istere���in�the�revie�e���tas�s,�e.��.�b���Pins�an���ff��tche�(Fi��-ure�4,�above)�an���Koivisto�et�al.�(Fi��ure�6,�above)��ith�faint�an����as�e���sti�uli,�an���b���Ser��ent�et�al.�(2005),�Kranczioch�et�al.�(2007;�cf.�Fi��ure�10��,�above)�an���Koivisto�an���Revonsuo�(2008)�in�the�attentional-blin���ara��i���.�These�latter�au-thors�coine���the�ter��“visual�a�areness�ne��ativit��”�for�the�N2�obtaine���in�their�stu��ies.�But�fro��the�rea��in���of�the�evi��ence�as��resente���here,�the�label�“visual�attention�ne��ativit��”�is��ore�a��ro�riate�for�the��osterior-lateral�N2��hich,�li�e�N2�c,��a���reflect�a��rocess�that�is��erha�s�necessar���(thou��h�see�La��e�2003)�but��robabl���not�sufficient�for�conscious�a�areness.

2.�.3 P3

P3�is�the��ost�reliable��ar�er�of�conscious�a�areness.�This��as�alrea�����the�case�in�the�earl���stu��ies�on�au��itor���si��nal���etection�(Fi��ure�1,�above)�an���continue���in�the��ore�recent�stu��ies��ith�faint�an����as�e���sti�uli�(Fi��ures�4–6,�above)�an���in�the�attentional�blin��(Fi��ures�9c�an���10c�above).�

Page 75: Unconscious Memory Representations in Perception

�4� Rolf�Verle��er

It�has�been�clai�e���in�the�attentional-blin��literature�that�the�absence�of�P3��ith� uni��entifie��� T2� sti�uli� �rovi��es� su��ort� for� theories� that� lin�� the� atten-tional�blin��to�ca�acit����roble�s�of��or�in����e�or���(cf.�Sessa�et�al.�2007,�for�a���iscussion).�Ho�ever,�the�ver���sa�e�variation�of�P3�is�foun���in�au��itor���si��nal���etection�an���in�i��entification�of�faint�an����as�e���si��nals,��here�ca�acit����rob-le�s�of��or�in����e�or���are�irrelevant.�Therefore,�althou��h�the��or�in����e�or���h���othesis��i��ht�be�true�in�accountin���for�the�attentional-blin��(Dell’Acqua�et�al.�2009)�the�P3�variation�evi��entl���cannot�be�ta�en�as�su��ortin���evi��ence.�Rather,�P3�is�li�el���to�reflect�a��ore���eneral��rocess�that�occurs�in�all��ara��i���s�covere���b���this�revie�.�

Base���on�their�results�in�si��nal���etection,�Hill��ar���an���collea��ues�relate���the�P3�co��onent�to�the���ecision��rocess�(e.��.,�K.�Squires�et�al.�1973,��.�272).�Thus,�one��i��ht�sa���that�P3�is�evo�e���b���all�sti�uli�that�lea���to�a�tas�-relevant���ecision,��hich�also�is�the�vie��ta�en�b���this�author�(Verle��er�et�al.�2005;�Verle��er�2008).�

Relatin���P3�to�the���ecision�is,�of�course,�not�equivalent�to�re��ar��in���P3�as�a���i-rect�correlate�of�a�areness.�This�vie��oint�has�been�ta�en�b���Dehaene�et�al.�(2003),�base���on�the�attentional-blin���ara��i���.�Dehaene�et�al.�(2003)��ro�ose���that�P3�reflects�access�of�a��i��e�variet���of�brain�re��ions�to�a���lobal��or�s�ace,�for�in���the��h��siolo��ical�basis� for�a�areness.�Ho�ever,� this� is�obviousl���an�over��eneraliza-tion.�Sti�uli�that���o�not�require�a���ecision���o�not�evo�e�a�P3�but�are�nevertheless�consciousl����erceive��,�see�e.��.�Wu�an���Coulson�(2005),�Roeh��et�al.�(2007).�

Thus,� there� is�no�a�areness� co��onent�of� the�ERP.�Rather� there�are� co�-�onents�that�reflect��rocesses��ore�or�less�relate���to�a�areness�of�sti�uli�in�the�ex�eri�ental�tas�s.�

Acknowledgements

This� cha�ter� �as� �ritten� �ithin� the� �roject� “On� the� role� of� ri��ht-he�is�here���o�inance� for� constructin��� conscious� �erce�ts“� fun��e��� b��� Ger�an� Research�Foun��ation�(DFG,�VE110/15-1)�as��art�of�the�research��ro��ra��“Neuro-co��ni-tive��echanis�s�of�conscious�an���unconscious�visual��erce�tion”�(�ac�a��e��ro-�osal�PAK270).

References

A���üre�,�E.�G.,�Ri����ell,�P.�M.,�Toffanin,�P.�an���Ho��el,�B.�(2007).�A��a�tive�control�of�event�inte��ration:�Evi��ence�fro��event-relate����otentials.�Psychophysiology,�44,�383–391.

Bernat,�E.�(2001).�Re�l���to�Dr.�Verle��er.�Clinical Neurophysiology,�112,�1957.

Page 76: Unconscious Memory Representations in Perception

� Mar�ers�of�a�areness?� �5

Bernat,�E.,�Shevrin,�H.�an���Sno����rass,�M.� (2001).�Subli�inal�visual�o����ball� sti�uli�evo�e�a�P300�co��onent.�Clinical Neurophysiology,�112,�159–171.

Bo��,�F.,�Clar�e,�K.�an���Su�ner,�P.�(2008).�Mas��sti�ulus�tri����ers�inhibition�in�subli�inal�visuo-�otor��ri�in��.�Experimental Brain Research,�190,�111–116.

Bráz��il,�M.,�Re�tor,�I.,�Daniel,�P.,�Dufe�,�M.�an���Jurá�,�P.�(2001).�Intracerebral�event-relate����otentials�to�subthreshol���tar��et�sti�uli.�Clinical Neurophysiology,�112,�650–661.

Bráz��il,�M.,�Re�tor,�I.,�Dufe�,�M.,�Jurá�,�P.�an���Daniel,�P.�(1998�).�Effect�of�subthreshol���tar��et�sti�uli�on�event-relate����otentials. Electroencephalography and Clinical Neurophysiology,�107,�64–68.

Dehaene,�S.,�Naccache,�L.,�Le�Clec’H,�G.,�Koechlin,�E.,�Mueller,�M.,�Dehaene-La�bertz,�G.,�van���e�Moortele,�P.-F.�an���Le�Bihan,�D.�(1998).�I�a��in���unconscious�se�antic��ri�in��.�Nature,�395,�597–600.

Dehaene,� S.,� Naccache,� L.,� Cohen,� L.,� Le� Bihan,� D.,� Man��in,� J.-F.,� Poline,� J.B.� an��� Rivière,�D.�(2001).�Cerebral��echanis�s�of��or����as�in���an���unconscious� re�etition��ri�in��.�Nature Neuroscience,�4,�752–758.

Dehaene,�S.,�Ser��ent,�C.�an���Chan��eux,�J.�P.�(2003).�A�neuronal�net�or���o��el�lin�in���subjec-tive�re�orts�an���objective��h��siolo��ical���ata���urin���conscious��erce�tion.�Proceedings of the National Academy of Science of the U.S.A.,�100,�8520–8525.

Dell’Acqua,�R.,�Jolicœur,�P.,�Luria,�R.�an���Pluchino,�P.�(2009).�Re-evaluatin���enco��in��-ca�acit���li�itations�as�a�cause�of�the�attentional�blin�.�Journal of Experimental Psychology: Human Perception and Performance,�in��ress.

Dell’Acqua,�R.,�Jolicœur,�P.,�Pesciarelli,�F.,�Job,�R.�an���Palo�ba,�D.�(2003)�Electro�h��siolo��ical�evi��ence�of�visual�enco��in�����eficits�in�a�cross-�o��al�attentional�blin���ara��i���.�Psycho-physiology,�40,�629–639.

Dell’Acqua,�R.,�Sessa,�P.,�Jolicœur,�P.�an���Robitaille,�N.�(2006).�S�atial�attention�freezes���urin���the�attentional�blin�.�Psychophysiology,�43,�394–400.

Des�e��t,�J.�E.,�Debec�er,�J.�an���Manil,�J.�(1965).�Mise�en�évi��ence���’un�si��ne�électrique�céré-bral�associé�à�la���étection��ar�le�sujet���’un�sti�ulus�sensoriel�tactile.�Bulletin de l’Académie Royale de Médecine de Belgique,�5,�887–936.

Devri�,�M.,�De�iral�,�T.,�Kurt,�A.�an���Yücesir,�I.�(1999).�Slo��cortical��otential�shifts��o��u-late�the�sensor���threshol���in�hu�an�visual�s��ste�.�Neuroscience Letters,�270,�17–20.

Di�Lollo,�V.,�Ka�ahara,�J.,�Ghorashi,�S.�M.�S.�an���Enns,�J.�T.�(2005).�The�attentional�blin�:�Re-source���e�letion�or�te��orar���loss�of�control?�Psychological Research,�69,�191–200.

Dixon,�N.�F.�(1971).�Subliminal perception: The nature of a controversy.�Lon��on:�McGra�-Hill.Donchin,�E.�an���Lin��sle��,�D.�B.�(1965).�Visuall���evo�e���res�onse�correlates�of��erce�tual��as�-

in���an���enhance�ent.�Electroencephalography and Clinical Neurophysiology,�19,�325–335.Duncan-Johnson,�C.�C.�an���Donchin,�E.�(1979).�The�ti�e�constant�in�P300�recor��in��.�Psycho-

physiology,�16,�53–55.Ei�er,�M.�(1996).�The�N2�c�co��onent�as�an�in��icator�of�attentional�selectivit��.�Electroencepha-

lography and Clinical Neurophysiology,�9,�25–234.Ei�er,�M.�(1999).�Facilitator���an���inhibitor���effects�of��as�e����ri�e�sti�uli�on��otor�activa-

tion�an���behavioural��erfor�ance.�Acta Psychologica,�101,�293–313.�Ei�er,�M.�an���Mazza,�V.�(2005).�Electro�h��siolo��ical�correlates�of�chan��e���etection.�Psycho-

physiology,�42,�328–342.Ei�er,�M.�an���Schla��hec�en,�F.�(1998).�Effects�of��as�e���sti�uli�on��otor�activation:�behav-

ioral�an���electro�h��siolo��ical�evi��ence.�Journal of Experimental Psychology: Human Per-ception and Performance,�24,�1737–1747.

Page 77: Unconscious Memory Representations in Perception

��� Rolf�Verle��er

Ei�er,� M.� an��� Schla��hec�en,� F.� (2003).� Res�onse� facilitation� an��� inhibition� in� subli�inal��ri�in��.�Biological Psychology,�64,�7–26.

Er��eno��lu,�T.,�De�iral�,�T.,�Ba��ra�taro��lu,�Z.,�Er��en,�M.,�Be����a��i,�H.�an���Uresin,�Y.�(2004).�Al�ha�rh��th��of�the�EEG��o��ulates�visual���etection��erfor�ance�in�hu�ans.�Cognitive Brain Research,�20,�376–383.

Fernan��ez-Duque,�D.,�Grossi,�G.,�Thornton,�I.�M.�an���Neville,�H.�J.�(2003).�Re�resentation�of�chan��e:�Se�arate�electro�h��siolo��ical��ar�ers�of�attention,�a�areness,�an���i��licit��ro-cessin��.�Journal of Cognitive Neuroscience,�15,�491–507.

Folstein,�J.�R.�an���van�Petten,�C.�(2008).�Influence�of�co��nitive�control�an����is�atch�on�the�N2�co��onent�of�the�ERP:�A�revie�.�Psychophysiology,�45,�152–170.

Giesbrecht,� B.,� S��,� J.� L.� an��� Elliott,� J.� C.� (2007).� Electro�h��siolo��ical� evi��ence� for� both� �er-ce�tual� an��� �ost-�erce�tual� selection� ��urin��� the� attentional� blin�.� Journal of Cognitive�Neuroscience,�19,�2005–2018.

Hill��ar��,�S.�A.,�Squires,�K.�C.,�Bauer,�J.�W.�an���Lin��sa��,�P.�H.�(1971).�Evo�e����otential�correlates�of�au��itor���si��nal���etection.�Science,�172,�1357–1360.

Ho�f,�J.�M.,�Luc�,�S.�J.,�Boel�ans,�M.�A.,�Schoenfel��,�M.�A.,�Boehler,�C.�N.,�Rie��er,�J.�an���Heinze,�H.�J.�(2006).�The�neural�site�of�attention��atches�the�s�atial�scale�of��erce�tion.�The Journal of Neuroscience,�26,�3532–3540.

Jaś�o�s�i,�P.,�Białuńs�a,�A.,�To�ane�,�M.�an���Verle��er,�R.�(2008).�Mas�-�an�����istractor-tri��-��ere���inhibitor����rocesses�in�the��ri�in���of��otor�res�onses:�An�EEG�stu����.�Psychophysi-ology,�45,�70–85.

Jaś�o�s�i,�P.,�S�als�a,�B.�an���Verle��er,�R.� (2003).�Ho��the�self�controls� its�“auto�atic��ilot”��hen��rocessin���subli�inal�infor�ation.�Journal of Cognitive Neuroscience,�15,�911–920.

Jolicœur,�P.,�Sessa,�P.,�Dell’Acqua,�R.�an���Robitaille,�N.�(2006).�On�the�control�of�visual�s�atial�attention:�evi��ence�fro��hu�an�electro�h��siolo����.�Psychological Research,�70,�414–424.

Jaś�o�s�i,�P.,�van���er�Lubbe,�R.�H.�J.,�Schlotterbec�,�E.�an���Verle��er,�R.�(2002).�Traces�left�on�visual�selective�attention�b���sti�uli�that�are�not�consciousl���i��entifie��.�Psychological Sci-ence,�13,�48–54.

Keil,�A.,�Müller,�M.�M.,�Ra��,�W.�J.,�Gruber,�T.�an���Elbert,�T.�(1999).�Hu�an���a��a�ban���activit���an����erce�tion�of�a���estalt.�Journal of Neuroscience,�19,�7152–7161.

Ker�hof,�G.�A.�an���Uhlenbroe�,�J.�(1981).�P3�Latenc���in�threshol���si��nal���etection.�Biological Psychology,�13,�89–106.

Kessler,�K.,�Sch�itz,�F.,�Gross,�J.,�Ho��el,�B.,�Sha�iro,�K.�an���Schnitzler,�A.�(2005).�Cortical��echanis�s�of�attention�in�ti�e:�neural�correlates�of�the�La��-1-s�arin����heno�enon.�Eu-ropean Journal of Neuroscience,�21,�2563–2574.

Ki�ura,�M.,�Kata��a�a,�J.�an���Murohashi,�H.�(2006).�Probabilit��-in��e�en��ent�an���-��e�en��ent�ERPs�reflectin���visual�chan��e���etection.�Psychophysiology,�43,�180–189.

Ki�ura,� M.,� Kata��a�a,� J.� an��� Murohashi,� H.� (2008).� Involve�ent� of� �e�or��-co��arison-base���chan��e���etection�in�visual���istraction.�Psychophysiology,�45,�445–457.

Kla��,�S.�T.�an���Hin�le��,�L.�B.�(2002).�The�ne��ative�co��atibilit���effect:�Unconscious�inhibi-tion�influences�reaction�ti�e�an���res�onse�selection.�Journal of Experimental Psychology: General,�131,�255–269.

Klotz,�W.,�Heu�ann,�M.,�Ansor��e,�U.�an���Neu�ann,�O.�(2007).�Electro�h��siolo��ical�activation�b����as�e����ri�es:�In��e�en��ence�of��ri�e-relate���an���tar��et-relate���activities.�Advances in Cognitive Psychology,�3,�449–465.

Koivisto,�M.�an���Revonsuo,�A.�(2003).�An�ERP�stu�����of�chan��e���etection,�chan��e�blin��ness,�an���visual�a�areness.�Psychophysiology,�40,�423–429.

Page 78: Unconscious Memory Representations in Perception

� Mar�ers�of�a�areness?� �7

Koivisto,�M.�an���Revonsuo,�A.�(2008).�Co��arison�of�event-relate����otentials� in�attentional�blin��an���re�etition�blin��ness.�Brain Research,�1189,�115–126.

Koivisto,�M.,�Revonsuo,�A.�an���Sal�inen,�N.�(2005).�In��e�en��ence�of�visual�a�areness�fro��attention�at�earl����rocessin���sta��es.�NeuroReport,�16,�817–821.

Korn�eier,�J.�an���Bach,�M.�(2004).�Earl���neural�activit���in�Nec�er-cube�reversal:�Evi��ence�for�lo�-level��rocessin���of�a�Gestalt��heno�enon.�Psychophysiology,�41,�1–8.

Kranczioch,�C.,�Debener,�S.�an���En��el,�A.�K.�(2003).�Event-relate����otential�correlates�of�the�attentional�blin���heno�enon.�Cognitive Brain Research,�17,�177–187.

Kranczioch,�C.,�Debener,�S.,�Ma��e,�A.�an���En��el,�A.�K.�(2007).�Te��oral�����na�ics�of�access�to�consciousness�in�the�attentional�blin�.�NeuroImage,�37,�947–955.

Kutas,�M.�an���Hill��ar��,�S.�A.�(1980).�Rea��in���senseless�sentences:�brain��otentials�reflect�se-�antic�incon��ruit��.�Science,�207,�203–205.

La��e,�V.�A.�F.�(2003�).�Wh���visual�attention�an���a�areness�are���ifferent.�Trends in Cognitive Sciences,�7,�12–18.

Lau,�E.�F.,�Philli�s,�C.�an���Poe��el,�D.�(2008).�A�cortical�net�or��for�se�antics:�(��e)constructin���the�N400.�Nature Reviews: Neuroscience,�9,�920–933.

Leuthol��,�H.�an���Ko��,�B.�(1998).�Mechanis�s�of��ri�in���b����as�e���sti�uli:�Inferences�fro��event-relate���brain��otentials.�Psychological Science,�9,�263–269.

Lleras,�A.�an���Enns,�J.�T.�(2004).�Ne��ative�co��atibilit���or�object�u���atin��?�A�cautionar���tale�of��as�-��e�en��ent��ri�in��.�Journal of Experimental Psychology: General,�133,�475–493.

Luc�,�S.�J.�an���Hill��ar��,�S.�A.�(1994).�Electro�h��siolo��ical�correlates�of�feature�anal��sis���urin���visual�search.�Psychophysiology,�31,�291–308.

Luc�,�S.�J.,�Vo��el,�E.�K.�an���Sha�iro,�K.�L.�(1996).�Wor����eanin��s�can�be�accesse���but�not�re-�orte�����urin���the�attentional�blin�.�Nature,�383,�616–618.

Lutzenber��er,�W.,�Elbert,�T.,�Roc�stroh,�B.�an���Birbau�er,�N.�(1979).�The�effects�of�self-re��ula-tion�of� slo��cortical��otentials�on��erfor�ance� in�a� si��nal���etection� tas�.� International Journal of Neuroscience,�9,�175–183.

Martens,�S.,�Munne�e,�J.,�S�i��,�H.�an���Johnson,�A.�(2006a).�Quic���in��s���on’t�blin�:�Electro-�h��siolo��ical�correlates�of�in��ivi��ual���ifferences�in�attentional�selection.�Journal of Cogni-tive Neuroscience,�18,�1423–1438.

Martens,�S.,�El�allah,�K.,�Lon��on,�R.�an���Johnson,�A.�(2006b�).�Cuin���an���sti�ulus��robabilit���effects�on�the�P3�an���the�AB.�Acta Psychologica,�123,�204–218.

McArthur,�G.,�Bu����,�T.�an���Michie,�P.�(1999).�The�attentional�blin��an���P300.�NeuroReport,�10,�3691–3695.

Na�atani,� C.,� Ito,� J.,� Ni�olaev,� A.� R.,� Gon��,� P.� an��� van� Leeu�en,� C.� (2005).� Phase� s��nchro-nization�anal��sis�of�EEG���urin���attentional�blin�.�Journal of Cognitive Neuroscience,�17,�1969–1979.

Nieu�enstein,�M.�R.�an���Potter,�M.�C.�(2006).�Te��oral�li�its�of�selection�an����e�or���en-co��in��:�A�co��arison�of��hole�versus��artial�re�ort� in�ra�i���serial�visual��resentation.�Psychological Science,�17,�471–475.

Ojanen,�V.,�Revonsuo,�A.�an���Sa�s,�M.�(2003).�Visual�a�areness�of�lo�-contrast�sti�uli�is�re-flecte���in�event-relate���brain��otentials.�Psychophysiology,�40,�192–197.

Olivers,�C.�N.�L.�(2007).�The�ti�e�course�of�attention:�It�is�better�than��e�thou��ht.�Current Di-rections in Psychological Science,�16,�11–15.

Parasura�an,�R.�an���Beatt��,� J.� (1980).�Brain�events�un��erl��in�����etection�an���reco��nition�of��ea��sensor���si��nals.�Science,�210,�80–83.

Page 79: Unconscious Memory Representations in Perception

��� Rolf�Verle��er

Parasura�an,�R.,�Richer,�F.�an���Beatt��,�J.�(1982).�Detection�an���reco��nition:�concurrent��ro-cesses�in��erce�tion.�Perception and Psychophysics,�31,�1–12.

Paul,�D.�D.�an���Sutton,�S.�(1972).�Evo�e����otential�correlates�of�res�onse�criterion�in�au��itor���si��nal���etection.�Science,�177,�362–364.

Pesciarelli,�F.,�Kutas,�M.,�Dell’Acqua,�R.,�Peressotti,�F.,�Job,�R.�an���Urbach,�T.�P.�(2007).�Se�an-tic�an���re�etition��ri�in����ithin�the�attentional�blin�:�An�event-relate���brain��otential�(ERP)�investi��ation�stu����.�Biological Psychology,�76,�21–30.

Pins� an��� ff��tche,� D.� (2003).� The� neural� correlates� of� conscious� vision.� Cerebral Cortex,� 13,�461–474.

Potter,�M.�C.�(2006).�Co��etition�for�attention�in�s�ace�an���ti�e:�The�first�200��s.�In�H.�Öğ�en�an���B.�Breit�e��er�(E��s.),�The first half second: The microgenesis and temporal dynamics of unconscious and conscious visual processes�(��.�207–224).�Ca�bri����e,�MA:�MIT�Press.

Praa�stra,�P.�an���Seiss,�E.�(2005).�The�neuro�h��siolo�����of�res�onse�co��etition:�Motor�cor-tex�activation�an���inhibition�follo�in���subli�inal�res�onse��ri�in��.�Journal of Cognitive Neuroscience,�17,�483–493.

Robitaille,�N.,�Jolicœur,�P.,�Dell’Acqua,�R.�an���Sessa,�P.�(2007).�Short-ter��consoli��ation�of�vi-sual� �atterns� interferes� �ith� visuo-s�atial� attention:� Conver��in��� evi��ence� fro�� hu�an�electro�h��siolo����.�Brain Research,�1185,�158–169.

Roeber,�U.,�Wi���ann,�A.,�Trujillo-Barreto,�N.�J.,�Herr�ann,�C.�S.,�O’Shea,�R.�P.�an���Schrö��er,�E.�(2008).�Earl���effects�of�visual�a�areness�in�the�hu�an�brain:�Ti�e�an����lace�fro��event-relate���brain��otentials.�Journal of Vision,�8,�21,�1–12.

Roeh�,�D.,�Born�essel-Schlese�s���,�I.,�Rösler,�F.�an���Schlese�s���,�M.�(2007).�To��re��ict�or�not�to��re��ict:�Influences�of�tas��an���strate�����on�the��rocessin���of�se�antic�relations.�Journal of Cognitive Neuroscience,�19,�1259–1274.

Rol�e,�B.,�Heil,�M.,�Streb,�J.�an���Henni��hausen,�E.�(2001).�Misse����ri�e��or��s��ithin�the�at-tentional�blin��evo�e�an�N400�se�antic��ri�in���effect.�Psychophysiology,�38,�165–174.

Rossion,�B.�an���Jacques,�C.�(2008).�Does��h��sical�intersti�ulus�variance�account�for�earl���elec-tro�h��siolo��ical� face�sensitive�res�onses� in�the�hu�an�brain?�Ten� lessons�on�the�N170.�NeuroImage,�39,�1959–1979.

Ruch�in,�D.�S.,�Sutton,�S.,�Kietz�an,�M.�L.�an���Silver,�K.�(1980).�Slo���ave�an���P300�in�si��nal���etection.�Electroencephalography and Clinical Neurophysiology,�50,�35–47.

Schan�in,�A.�an���Wascher,�E.�(2007).�Electro�h��siolo��ical�correlates�of�sti�ulus��rocessin���in�chan��e�blin��ness.�Experimental Brain Research,�183,�95–105.

Schan�in,� A.� an��� Wascher,� E.� (2008).� Unvoluntar��� attentional� ca�ture� in� chan��e� blin��ness.�Psychophysiology,�45,�742–750.

Schla��hec�en,� F.� an��� Ei�er,� M.� (2006).� Active� �as�s� an��� active� inhibition:� A� co��ent� on�Lleras�an���Enns�(2004)�an���on�Verle��er,�Jaś�o�s�i,�A����e�ir,�van���er�Lubbe,�an���Groen�(2004).�Journal of Experimental Psychology: General,�135,�484–494.

Schla��hec�en,�F.,�Bo��an,�H.�an���Ei�er,�M.� (2006).�Dissociatin��� local�an�����lobal� levels�of��erce�tuo-�otor�control�in��as�e����ri�in��.�Journal of Experimental Psychology: Human Perception and Performance,�32,�618–632.

Schubert,�R.,�Blan�enbur��,�F.,�Le��,�S.,�Villrin��er,�A.�an���Curio,�G.�(2006).�No����ou�feel�it�–�no����ou���on’t:�ERP�correlates�of�so�atosensor���a�areness.�Psychophysiology,�43,�31–40.

Sch�artz,�M.�an���Pritchar��,�W.�S.�(1981).�AERs�an�����etection�in�tas�s���iel��in���U-sha�e���bac�-�ar����as�in���functions.�Psychophysiology,�18,�678–685.

Page 80: Unconscious Memory Representations in Perception

� Mar�ers�of�a�areness?� �9

Seiss,�E.�an���Praa�stra,�P.�(2004).�The�basal���an��lia�an���inhibitor����echanis�s�in�res�onse�selection:�evi��ence� fro��subli�inal��ri�in���of��otor� res�onses� in�Par�inson’s���isease.�Brain,�127,�330–339.

Ser��ent,�C.,�Baillet,�S.�an���Dehaene,�S.�(2005)�Ti�in���of�the�brain�events�un��erl��in���access�to�consciousness���urin���the�attentional�blin�.�Nature Neuroscience,�8,�1391–1400.

Sessa,�P.,�Luria,�R.,�Verle��er,�R.�an���Dell’Acqua,�R.�(2007).�P3�latenc���shifts�in�the�attentional�blin�:�Further�evi��ence�for�secon���tar��et��rocessin��.�Brain Research,�1137,�131–139.

Sha�iro,�K.,�Sch�itz,�F.,�Martens,�S.,�Ho��el,�B.�an���Schnitzler,�A.�(2006).�Resource�sharin���in�the�attentional�blin�.�NeuroReport,�17,�163–166.

Shevrin,�H.�(2001).�Event-relate����ar�ers�of�unconscious��rocesses.�International Journal of Psychophysiology,�42,�209–218.

Shevrin,�H.�an���Fritzler,�D.�(1968a).�Visual�evo�e���res�onse�correlates�of�unconscious��ental��rocesses.�Science,�161,�295–298.

Shevrin,�H.�an���Fritzler,�D.�(1968b).�Brain�res�onse�correlates�of�re�ressiveness.�Psychological Reports,�23,�887–892.

S�i��,�H.�G�.O.�M.,�Hauser,�U.,�Weiler,�H.�T.,�A�iszus,�F.,�Hinrichs,�H.�an���Heinze,�H.-J.�(2004).�Brain��otentials�an���behavioral�res�onses�associate����ith�attention�to�har��-�an���eas��-to-��iscri�inate��assive��nee�joint��ove�ents.�Psychophysiology,�41,�489–500.

S�ith,�M.�L.,�Gossel��n,�F.�an���Sch��ns,�P.�G.�(2006).�Perce�tual��o�ents�of�conscious�visual�ex�erience�inferre���fro��oscillator���brain�activit��.�Proceedings of the National Academy of Sciences,�103,�5626–5631.

Squires,�K.�C.,�Hill��ar��,�S.�A.�an���Lin��sa��,�P.�H.�(1973).�Vertex��otentials�evo�e�����urin���au-��itor��� si��nal� ��etection:� Relation� to� ��ecision� criteria.� Perception and Psychophysics,� 14,�265–272.

Squires,�K.�C.,�Squires,�N.�K.�an���Hill��ar��,�S.�A.�(1975a).�Decision-relate���cortical��otentials���urin���an�au��itor���si��nal���etection�tas���ith�cue���observation�intervals.�Journal of Experi-mental Psychology: Human Perception and Performance,�1,�268–279.

Squires,�K.�C.,�Squires,�N.�K.�an���Hill��ar��,�S.�A.�(1975b).�Vertex-evo�e����otentials�in�a�ratin��-scale���etection�tas�:�Relation�to�si��nal��robabilit��.�Behavioral Biology,�13,�21–34.

Squires,�N.�K.,�Squires,�K.�C.�an���Hill��ar��,�S.�A.�(1978).�Functional�equivalence�of�si��nal-�res-ent,�si��nal-absent,�an���threshol��-��etect�P3s.�In�D.�Otto�(E��.),�Multidisciplinary Perspec-tives in Event-Related Brain Potential Research�(��.�218–221).�Washin��ton,�D.C.:�U.S.�Gov-ern�ent�Printin���Office.

Sutton,�S.,�Braren,�M.,�Zubin,�J.�an���John,�E.�R.�(1965).�Evo�e��-�otential�correlates�of�sti�ulus�uncertaint��.�Science,�150,�1187–1188.

Sutton,�S.,�Ruch�in,�D.�S.,�Munson,�R.,�Kietz�an,�M.�L.�an���Ha��er,�M.�(1982).�Event-relate����otentials�in�a�t�o-interval�force��-choice���etection�tas�.�Perception and Psychophysics,�32,�360–374.

Turatto,�M.,�An��rilli,�A.,�Mazza,�V.,�U�iltà,�C.�an���Driver,�J.�(2002).�Loo�in����ithout�seein���the�bac���roun���chan��e:�Electro�h��siolo��ical�correlates�of�chan��e���etection�versus�chan��e�blin��ness.�Cognition,�84,�B1–B10.

Verle��er,�R.�(2001).�Effect�of�subli�inall����erceive���sti�ulus���ifferences�or�of�su�erli�inall����erceive��� or��er� ��ifferences?� Letter� to� the� E��itor.� Clinical Neurophysiology,� 112,� 1956–1957.

Verle��er,�R.�(2008).�P3b:�To�ar��s�so�e���ecision�about��e�or��.�Clinical Neurophysiology,�119,�968–970.

Page 81: Unconscious Memory Representations in Perception

70� Rolf�Verle��er

Verle��er,�R.�an���Jaś�o�s�i,�P.�(2007).�Disentan��lin���neural��rocessin���of��as�e���an����as�in���sti�ulus�b����eans�of�event-relate���contralateral–i�silateral���ifferences�of�EEG��otentials.�Advances in Cognitive Psychology,�3,�193–210.

Verle��er,�R.,�Jaś�o�s�i,�P.,�A����e�ir,�A.,�van���er�Lubbe,�R.�H.�J.�an���Groen,�M.�(2004).�Qualita-tive� ��ifferences� bet�een� conscious� an��� non-conscious� �rocessin��?� On� inverse� �ri�in���in��uce���b����as�e���arro�s.�Journal of Experimental Psychology: General,�133,�494–515.

Verle��er,�R.,�Jaś�o�s�i,�P.�an���Wascher,�E.�(2005).�Evi��ence�for�an�inte��rative�role�of�P3b�in�lin�in���reaction�to��erce�tion.�Journal of Psychophysiology,�19,�165–181.

Verle��er,� R.,� Kötter,� T.,� Jaś�o�s�i,� P.,� S�ren��er,� A.� an��� Siebner,� H.� (2006).� A� TMS� stu����� on�non-consciousl��� tri����ere���res�onse�ten��encies� in�the��otor�cortex.�Experimental Brain Research,�173,�115–129.

Verle��er,�R.,�Schu�necht,�S.-V.,�Jaś�o�s�i,�P.�an���Wa��ner,�U.�(2008).�Chan��es�in��rocessin���of��as�e���sti�uli�across�earl��-�an���late-ni��ht�slee�:�A�stu�����on�behavior�an���brain��oten-tials.�Brain and Cognition,�68,�180–192.

Verle��er,�R.,�S�ren��er,�A.,�Gebauer,�S.,�Fritz�annova,�M.,�Frie��rich,�M.,�Kraft,�S.�an���Jaś�o�s�i,�P.�(2009).�On��h���left�events�are�the�ri��ht�ones:�Neural��echanis�s�un��erl��in���the�left-he�ifiel���a��vanta��e�in�ra�i���serial�visual��resentation.�Journal of Cognitive Neuroscience,�21,�474–488.�

Vo��el,�E.�K.�an���Luc�,�S.�J.�(2002).�Dela��e����or�in����e�or���consoli��ation���urin���the�atten-tional�blin�.�Psychonomic Bulletin and Review,�9,�739–743.

Vo��el,�E.�K.,�Luc�,�S.�J.�an���Sha�iro,�K.�L.�(1998).�Electro�h��siolo��ical�evi��ence�for�a��ost�er-ce�tual�locus�of�su��ression���urin���the�attentional�blin�.�Journal of Experimental Psychol-ogy: Human Perception and Performance,�24,�1656–1674.

Wausch�uhn,�B.,�Verle��er,�R.,�Wascher,�E.,�Kloster�ann,�W.,�Bur�,�M.,�Hei��e,�W.�an���Kö��f,�D.�(1998).�Lateralise���hu�an�cortical�activit���for�shiftin���visuos�atial�attention�an���initiatin���sacca��es.�Journal of Neurophysiology,�80,�2900–2910.

Wascher,� E.� an��� Wausch�uhn,� B.� (1996).� The� interaction� of� sti�ulus-� an��� res�onse-relate����rocesses� �easure��� b��� event-relate��� lateralisations� of� the� EEG.� Electroencephalography and Clinical Neurophysiology,�99,�149–162.

Wil�inson,�R.�T.�an���Seales,�D.�M.�(1978).�EEG�event-relate����otentials�an���si��nal���etection.�Biological Psychology,�7,�13–28.

Woo���an,�G.�F.�an���Luc�,�S.�J.�(2003).�Dissociations�a�on���attention,��erce�tion,�an���a�are-ness���urin���object-substitution��as�in��.�Psychological Science,�1,�605–611.

Wu,�Y.�C.�an���Coulson,�S.�(2005).�Meanin��ful���estures:�electro�h��siolo��ical�in��ices�of�iconic���esture�co��rehension.�Psychophysiology,�42,�654–667.

Page 82: Unconscious Memory Representations in Perception

chapter�3

In search for auditory object representations

István�Win�ler�Institute�for�Ps��cholo����,�Hun��arian�Aca��e����of�Sciences,�Bu��a�est�/�Institute�of�Ps��cholo����,�Universit���of�Sze��e��,�Hun��ar��

3.1 Introduction

In�ever����a���situations,�the��orl���a��ears�to�us�as�constitutin���of��eanin��ful�enti-ties,�ter�e���objects,�such�as�a�bir���or�a�car.�We�can���iscover�features�of�these�ob-jects�an���follo��their�actions.�Ho�ever,�scientists�have��no�n�for�a�lon���ti�e�that�this� is�not�ho��infor�ation�arrives� to�our�senses.�Si��nals�reachin���the�sensor���or��ans�contain�a��ixture�of�infor�ation�relate���to���ifferent�objects��hich��a����artl���obscure�each�other�or�interact�in�various��a��s.�Therefore,�a��ri�ar���ai��of�the�stu�����of��erce�tion�in��o��ern��s��cholo�����has�been�to�un��erstan���ho��the�(hu�an)�brain�transfor�s�inco�in���infor�ation�into�the�re�resentations�that��e�beco�e�a�are�of.�This�line�of�research�has�been�tra��itionall�����o�inate���b���inves-ti��ations�con��ucte���in�the�visual��o��alit��.�In��ee��,�even�the�conce�t�of�“object”�has�its�root�in�visual�ex�erience.�For�exa��le,�the�Merria�-Webster�Dictionar�����efines�the�base��eanin���of�“object”�as�“so�ethin���that��a���be�seen�or�felt”�(Ob-ject:�The�Merria�-Webster�Dictionar��,�1997).�Further�ore,�a��ain�base���on�visual�ex�erience,��e�intuitivel���assu�e�that�the�objects�a��earin���in��erce�tion�al�ost�al�a��s�corres�on���to��h��sical�entities�in�the��orl��.�When�the�����o�not,��e�tal��about�illusions�an���ascribe�the���iscre�anc���to�insufficient�inco�in���infor�ation�an��/or�to��rocessin���“errors”�in�the�brain.

Ho�ever,�the�ver���conce�t�of�an�object�is�of��uch���is�ute�in�the�research�of�au��itor����erce�tion.�This�is�lar��el�����ue�to�the�fact�that�the�au��itor���si��nal��rovi��es�relativel��� little� infor�ation�re��ar��in���the��h��sical�object� it�e�anates� fro��(the�soun���source).�Rather,�soun��s�carr�����etaile���infor�ation�about�on��oin���activities�in�our�environ�ent.�This�le���so�e�theorists�to�su����est�the�existence�of�t�o���if-ferent�au��itor���objects�(Kubov���an���Van�Val�enbur���2001).�Basin���on��erce�tual�an���neuroscience�evi��ence�(fin��in��s�su����estin���the�existence�of�se�arable�“�hat”�an���“�here”��ath�a��s�in�the�hu�an�an����ri�ate�au��itor���s��ste�;�e.��.,�Berlin�an���

Page 83: Unconscious Memory Representations in Perception

72� István�Win�ler

Zatorre�2000;�Rauschec�er�an���Tian�2000),�Kubov���an���Van�Val�enbur���su����est�that�au��itor���infor�ation�is��rocesse���in��arallel�neural��ath�a��s�an���in�ter�s�of��hat�Wi��ht�an�an���Jenison�(1995)�calle���“concrete”�an���“abstract”�objects.�The�ter��“concrete�object”�refers�to�the�soun���source.�The�au��itor���si��nal�contains�so�e�infor�ation�about�the���irection�of�the�soun���source�relative�to�the�listener�as��ell�as�so�e�other�cues��hich�are��erceive���as�characteristics�of� the�source�itself,�such�as�s�ea�er�size�(S�ith�et�al.�2005).�The�ter��“abstract�object”�refers�to�the�activit���of�soun���sources,�such�as�reflecte���b���the��elo������la��e���b���an�orches-tra,�s�eech,�or�the�soun���of�a�car��assin���b��.�It�is�intuitivel���obvious�that�the�latter���o�inates�our�au��itor���ex�erience.

When�consi��erin����erce�tion�as�a��ro��uct�of�neural�activit��,�one�shoul����ee��in��in��� that� in�“neuroscientific� ter�s,� the�conce�ts�of�an�object�an���of�object�anal��sis�can�be�re��ar��e���as�inse�arable”�(Griffiths�an���Warren�2004,��.�887).�That�is,� �hat� a��ears� in� �erce�tion� is� ��overne��� b��� �hat� infor�ation� �rocessin��� al-��orith�s�are�i��le�ente���for�the���iven�sensor����o��alit���in�the�brain.�Further-�ore,�hu�ans� (ani�als,� in���eneral)� are�not� i��le��erceivers.�Ps��cholo��ists�have�lon���ar��ue���that�the�role�of��erce�tion�is�not�veri��ical�re�resentation�of�the�en-viron�ent.�Rather,��erce�tion�subserves�the���oals�of�the�or��anis��b����rovi��in���infor�ation�require���for�successful�interactions��ith�the�environ�ent�(Bruns�i��1955).�Therefore,��e�ar��ue�that��erce�tual�objects���o�not�necessaril���corres�on���to�entities�in�the��h��sical��orl��;�the����a�����loss�over���istinctions�irrelevant�to�the���oals�of�behavior�as��ell�as�contain��ore�infor�ation�than��hat�is�currentl���avail-able�to�the�sensor���or��ans.

The�notion�that��erce�tual��rocesses�a�����infor�ation�to�that�extracte���fro��the�sensor��� in�ut� is� the���efinin���tenet�of� the�e��iricist� tra��ition�of��erce�tual��s��cholo������hich�starte����ith�Hel�holtz’��or��(1867/1910)�on�visual��erce�-tion.�In�one�of�the��ost�influential�conte��orar���e��iricist�theories�of��erce�-tion,�Gre��or���(1980)�li�ens��erce�tion�to�the��or�in���of��o��ern�science.�S�e-cificall��,�Gre��or���su����ests�that��erce�tions�are� li�e�h���otheses�in�science.�The���co��lete� the� inco�in��� infor�ation� b��� �rovi��in��� �re��ictions� about� �arts� that�have�not�been�actuall���sense��.�For�exa��le,�the�table�that��e��erceive�has�four�le��s,�even�thou��h�fro��our�current��oint�of�vie�,�onl���three�of�the��are�visible.�The�fourth�le���is�a����e���(its�existence��re��icte��)�on�the�basis�of��ast�ex�erience��ith�tables,�inclu��in���the�one��e�actuall���see.�Other�t���es�of��re��ictions�in�the�visual���o�ain�inclu��e�the�results�of�various�transfor�ations�cause���b���the��ove-�ents�of�the��h��sical�object�(e.��.,�rotation)�or�the��erceiver.�Because�soun��s��la���out�in�ti�e,��re��ictions�in�au��itor����erce�tion��ust�refer�to�the�future.�That�is,�in�searchin���for�au��itor����erce�tual�objects��e�shoul���loo��for�neural�activit����hich�allo�s�one�to��re��ict�the�continuation�of�a�soun���base���on��ast�ex�erience��ith�si�ilar�soun��s�as��ell�as�the�i��e��iate��ast�of�the���iven�soun��.

Page 84: Unconscious Memory Representations in Perception

� In�search�for�au��itor���object�re�resentations� 73

What���o��e�then��ean�b����erce�tual�objects?�We�re��ar����erce�tual�objects�as�units�of�infor�ation��hich�can�enter�various��ental�o�erations,�inclu��in���be-in����e�orize��,�transfor�e��,�co��are���to�other�units,�etc.�We�assu�e�that�such�sets�of� infor�ation�are�boun���then�accesse���an����rocesse���to��ether�in�the�hu-�an�brain.�Thus��e�treat�the��as�objects�in�infor�ation��rocessin���o�erations.�It�is�further�assu�e���that��erce�tual�objects�re�resent�so�e�se�arable�as�ect�of�the� environ�ent,� such� that� their� utilit��� for� the� �erceiver� is� vali��ate��� b��� a��a�-tive�behavior��ithin�the�environ�ent.�Therefore,��e�refer�to��erce�tual�objects�as�object�re�resentations�(i.e.,�the���re�resent�so�ethin���in�the��orl��).�Finall��,�it�is�i��ortant�to�note�that�object�re�resentations�are�not�re��ar��e���here�as�units�of�a��assive�infor�ation�stora��e�s��ste��(such�as�the��e�or���s��ste��of��ost�current�co��uters).�In�line��ith�the�vie���ro�ose���b���Griffiths�an���Warren�(2004),�the���are�consi��ere���to�be�(neural)��rocesses��rovi��in���infor�ation.�That�is,�an�object�re�resentation�is�equal�to�the�anal��sis�of�the�object-relate���infor�ation.

In� the� current� �a�er,� �e� �ill� sho�� that� �re��ictive� soun��� re�resentations�exist�in�the�hu�an�au��itor���s��ste��an���su����est�that�such�re�resentations�for��the�basis�of�au��itor���objects�in�the�brain.�We�shall�then��ro�ose�a�conce�tual��o��el�of�au��itor���object�for�ation��hich���iscusses�in�a�co��on�fra�e�or��t�o� i��ortant� functions� of� the� au��itor��� s��ste�,� the� se�aration� of� au��itor���strea�s�(Bre���an�1990)�an���au��itor�����eviance���etection�(Näätänen�1990).�The�function�of�the��ro�ose���s��ste��is�to��aintain�a��o��el�of�the�current�au��itor���environ�ent�an����rovi��e��erce�tion��ith�soun���infor�ation�e�be����e���in�the�au��itor���context.

Pre��ictive� �erce�tual� �rocesses� are� often� not� accessible� to� consciousness�(Hel�holtz�ter�e���the��“unconscious�inferences”),�onl���their�result��a���reach�a�areness.1�Ho�ever,�no�a�ount�of�trainin���in�intros�ection�allo�s�one�to�ex�e-rience�the�table�that��e�actuall���“see”��ith�our�e��es.�Thus��uch�of�the��rocessin���that�lea��s�to�the�conscious��erce�tion�of�objects�cannot�be�voluntaril���accesse��.2�Because�our�ai��is�to�fin���the��oint�at��hich�au��itor���infor�ation�is�turne���into�au��itor���object�re�resentations,�the��etho���use���in��an���of�the�stu��ies�revie�e���here� is� base��� on� a� �h��siolo��ical� �easure� ter�e��� event-relate��� brain� �otential�(ERP;� for�an� intro��uction� to� the��h��siolo��ical�bases�of�ERPs,� see�Fabiani�et�al.�2000;� for� �easurin��� �ractices,� see� Re��an� 1989,� see� also� Ben��ixen,� this� volu�e).�ERPs�recor��e����ith�electro��es��lace���on�the�hu�an�scal��su��the�electrical�ac-tivit���of�a�ver��� lar��e�nu�ber�of�s��nchronousl����o��ulate���neurons.�The���char-acterize�i��ortant�as�ects�of�the�brain’s�res�onse�to�sti�uli�(an���in�so�e�cases,�to�the�lac��of�a�sti�ulus),�acco��an��in���the��rocessin���of�a�sti�ulus�event�fro���hen�it�first�contacts�the�central�nervous�s��ste��until�the�sti�ulus-relate���neural�activit��� �er��es� into� the� bac���roun��� of� the� �ultitu��e� of� concurrentl��� on��oin���

Page 85: Unconscious Memory Representations in Perception

74� István�Win�ler

neural��rocessin���in�the�brain.�The�ERP��etho��,�as��ill�be�sho�n,�allo�s�one�to�stu�����both�conscious�an���unconscious��rocesses�un��erl��in����erce�tion.

In�the�follo�in��,��e�first�intro��uce�an�ERP�co��onent��hich�reflects�the���e-tection�of�au��itor�����eviance�irres�ective�of�the���irection�of�attention.�It�has�been�sho�n�that�au��itor�����eviance���etection�relies�on��e�or���re�resentations�enco��-in���the�re��ular�features�of�the��rece��in���au��itor���sti�ulation.�We��ill�exa�ine�the�re��ularit���re�resentations�inferre���fro��stu��ies�of�au��itor�����eviance���etec-tion,�searchin���for�features�ex�ecte���of��erce�tual�object�re�resentations.�Next��e��ill�sho��that�these�re�resentations�have��re��ictive��ro�erties.�We�then�a��l���the�notion�of��re��ictive�re��ularit���re�resentations�to�ex�lain�au��itor���strea�in���(van�Noor��en�1975).�Au��itor���strea�in���is�an�i��ortant�testin�����roun���for�theories�of�au��itor���object�for�ation,�because�it�sho�s�ho��coherent�soun���strea�s�are�se�arate���fro��a��ixe���soun���sequence.�Finall��,�a�conce�tual��o��el�of�au��itor���object�for�ation�is��ro�ose��.

3.2 The mismatch negativity event-related brain potential

The�Mis�atch�Ne��ativit���(MMN)�is�a�co��onent�of�the�ERP�elicite���b���soun��s�violatin���so�e�re��ular�feature�of�the��rece��in���soun���sequence�(Näätänen�et�al.�1978;�for�a�recent�revie�,�see�Kujala�et�al.�2007).�The�si��lest��a���to�elicit�MMN�is�to�occasionall���exchan��e�a�re�eatin���soun���for�a���ifferent�soun��.�Fi��ure�1�sho�s�a�“natural”�version�of�this��ara��i����in��hich�the�re�eatin���soun���varies�an���the�au��itor���scene�also�inclu��es�other�unrelate���soun��s.�MMN�is�observe���as�a�ne��a-tive� �avefor�� over� fronto-central� scal�� locations,� a��earin��� bet�een� 100� an���250��s�fro��the�onset�of���eviation�(see�Fi��ure�1).�Because�a��ajor��art�of�MMN�is� ��enerate��� in� au��itor��� cortex,� it� a��ears� often,� but� not� al�a��s,� �ith� reverse����olarit���at�electro��es��lace���over�the�o��osite�si��es�of�the�S��lvian�fissure�(such�as�electro��es��lace���over�the�frontal�scal��an���the��astoi��s).�MMN�can�be���e-lineate���fro��other�overla��in���ERP�co��onents�b���subtractin���fro��the�ERP�res�onse�elicite���b���the���eviant�soun���the�ERP�elicite���b���a�control�soun����hich�is��referabl���acousticall���i��entical�to�the���eviant�an���is��resente����ith�the�sa�e�te��oral�an���sequential��robabilit���as�the���eviant�itself�(for���etails�about�the�ori-��in�an�����erivation�of�MMN,�see�Kujala�et�al.�2007).

It�has�been�establishe���that�MMN���eneration�involves��e�or���traces���escrib-in���the�i��e��iate�histor���of�acoustic�sti�ulation�(for�a�su��ar��,�see�Näätänen�&�Win�ler�1999).�MMN��as�ori��inall�����iscovere����ithin�the�context�of�the�au��i-tor���o����ball��ara��i���3�an���has�been�re��ar��e���as�a�res�onse�tri����ere���b���soun��s���eviatin���fro��a�re�eatin���soun���in�one�or��ore�au��itor���features�(Näätänen�et�al.�1978).�As�a�consequence,�the��e�or���trace�un��erl��in���the�elicitation�of�MMN�

Page 86: Unconscious Memory Representations in Perception

� In�search�for�au��itor���object�re�resentations� 75

(e.��.,�Näätänen�1990)��as�initiall���thou��ht�to�corres�on���to�the�notion�of�au��itor���sensor����e�or��,� as���efine��� in��s��cholo��ical� stu��ies�usin���behavioral��etho��s�(e.��.,�Co�an�1984).�Since�then,�a�lar��e�nu�ber�of�stu��ies�observe���MMNs�elicite���b���violatin���such�acoustic�re��ularities,�in��hich�no�soun����as�re�eate����ith�hi��h��robabilit���or�the���eviant��as�a�hi��h-�robabilit���soun���(for�a�revie�,�see,�Win�ler�2003).�For�exa��le,�MMN�is�elicite���b���re�eatin���a�tone�in�a�sequence�of�tones���escen��in��� in� frequenc���(Tervanie�i�et�al.�1994a)�or� in�a� sequence�alternatin���bet�een�t�o���ifferent�frequencies�(e.��.,�Nor��b���et�al.�1988).�Therefore,�the�current�inter�retation�of� the�MMN�res�onse� su����ests� that� the��e�or��� re�resentations�

Figure 1. The�Mis�atch�Ne��ativit���(MMN).�The paradigm (left panel).�Partici�ants��atche���a��ovie�(soun���source�1�–�to�)��ith�street�noise���elivere���b���a�lou��s�ea�er�behin���the��(soun���source�2�–�botto�).�Fro��ti�e�to�ti�e,�a�series�of���i��itize���natural�footste��soun��s�(all���ifferent)��ere��resente���to�the��artici�ant�fro��t�o�lou��s�ea�ers��lace���at�both�si��es�(soun���sources�3�an���4�–��i����le�lateral),���ivin���the�i��ression�as�if�so�eone��al�e���across�the�roo�.�Ten�of�the�eleven�footste��soun��s�(�ar�e���blac��on�the��i��e�arro�s)�fit�into�a�natural��al�in���sequence.�One�footste��soun��,�ho�ever,���ave�the�i��ression�as�if�so�eone�ste��e���on�a���ifferent�surface�(�ar�e����hite�on�the�ar-ro�s).�In�the�Test�Sequence,�the�“Deviant”�footste��soun���a��eare���in�the�tenth,��hereas�in�the�Control�Sequences,�in�the�secon����osition�(“Control”).�Partici�ants�ha���no�tas��relate���to�the�footste��soun��s.�ERP results (right panel).�Frontal�(Fz)�ERP�res�onses�are�sho�n�for�the�i��entical���eviant�(continuous���re��)�an���control�(��ashe�����re���line)�footste��soun��s�to��ether��ith�the���eviant-�inus-control���ifference��avefor��(blac��continuous�line).�The�y�axis�(the�res�onse�a��litu��e�in�µV)��ar�s�the�onset�of�the���eviant/con-trol�footste��soun��.�The�x�axis���ives�the�ti�e�of�the�res�onse�(calibrate���in��s).�When�a��earin���late�in�the�sequence,�the���eviant�footste��soun���elicite���the�MMN���eviance���etection�res�onse�(�ar�e���on�the�fi��ure�b���the�chec�erboar����attern).�This�result�illustrates�that�the�MMN-��eneratin����rocess���etects���eviance�fro��a��reviousl���estab-lishe���re��ularit���(chan��e�a��earin���late�in�the�sequence),�as�o��ose���to�chan��e�fro��the��rece��in���soun���(chan��e�occurrin���earl���in�the�sequence).�(A��a�te����ith��er�ission�fro��Win�ler�et�al.�2003a.)

Page 87: Unconscious Memory Representations in Perception

7�� István�Win�ler

involve���in�the�MMN-��eneratin����rocess�enco��e�the���etecte���re��ularities�of�the�soun���sequence�(Win�ler�2007).�The�assu�e����e�or���re�resentations���escribe�the� inter-soun���relationshi�s���efinin��� the�re��ularit���as��ell�as�containin���suffi-cient�a�ount�of�au��itor���sensor���infor�ation�to�bin���the���escri�tion�of�the�re��u-larit���to�the�concrete�soun���sequence�(for�a�su����estion�re��ar��in���the�for��of�such�a��e�or���re�resentation,�see�Win�ler�an���Co�an�2005).

The��rocesses�reflecte���b���MMN�have�been�assu�e���to��la���a�role�in��as-sive�attention� (Ja�es�1890).� Detectin���chan��es� in� the� environ�ent�allo�s� the�or��anis��to�react�an���a��a�t�to�the�.�Given�that��e�are�not�able�to�full����rocess�all� of� the� infor�ation� arrivin��� continuousl��� to� the� sensor��� or��ans,� exclu��in���si��nals�relate���to�invariant�as�ects�of�the�environ�ent�fro��in-��e�th�anal��sis�conserves�li�ite����rocessin���ca�acities�(Sin��onen�1999).�Thus�earl����o��els�of�MMN�(e.��.,�Näätänen�1990)� re��ar��e��� the�MMN-��eneratin����rocess� as�one�of�the��echanis�s�b����hich�unatten��e���chan��es�in�the�environ�ent�can�call�for�li�ite����rocessin���ca�acities�(for�a�current�revie��of�this�issue,�see�Escera�an���Corral�2007).�An�alternative�h���othesis�su����ests�that�the��rocess���ivin���rise�to�MMN�is�involve���in�u���atin���the�re�resentations�of�au��itor���re��ularities��hen�the�inco�in���sti�ulus��is�atches�the��(Win�ler�2007;�Win�ler�an���Czi��ler�1998).�A�co��arison�bet�een�the�t�o�alternatives�sho�e���the��rece��ence�of�the��aintenance�of� re��ularit��� re�resentations�over� the�call� for� further��rocessin���(Win�ler�an���Czi��ler�1998).4

Althou��h�the��rocesses� lea��in���to�MMN�elicitation��a���not�be� full���auto-�atic,�MMN�is�elicite���even��hen�one���oes�not�focus�on�the�soun��s�an���has�no�tas��relate���to�the��(for�a�recent�revie��of�the�relationshi��bet�een�MMN�an���attention,�see�Suss�an�2007).�MMN�can�be�observe��� in�res�onse� to�re��ularit���violations�occurrin���in�ecolo��icall���vali���soun���sequences,�such�as�a�chan��e�in�a�series�of�footste�s�hear���in�a�nois���roo��(Win�ler�et�al.�2003a;�Fi��ure�1).�In�fact,�fa�iliarit����ith�the�soun��s�or�the�t���e�of�re��ularit���increases�the�MMN�a��li-tu��e�(e.��.,�Jacobsen�et�al.�2005;�van�Zuijen�et�al.�2005).�In���eneral,�learnin���(in-clu��in���lan��ua��e�an����usic�trainin��)�as��ell�as�other�lon��-ter���e�or���effects�influence� MMN� throu��h� the� un��erl��in��� �e�or��� re�resentations� (for� relate���revie�s,� see� Näätänen� et� al.� 2001;� Pulver�üller� an��� Sht��rov� 2006;� Tervanie�i�an���Huotilainen�2003).�An��,�conversel��,�the��e�or���re�resentations�involve���in�MMN���eneration�sho��so�e��ro�erties�of�lon��er-ter���e�or���traces,�such�as�the��ossibilit���of�reactivation�b���a�sin��le�re�in��er�(for�a�revie�,�see�Win�ler�an���Co�an�2005).

In�su��ar��,�the�MMN�ERP�co��onent�can�be�use���to�stu������e�or���re�re-sentations�of�au��itor���re��ularities�in�the�absence�of�focuse���attention.�In�fact,�the�function�of�the��rocesses���ivin���rise�to�the�observable�MMN�res�onse�is��robabl���relate���to� the��aintenance�of� these�re�resentations��hich�exhibit�both�sensor���

Page 88: Unconscious Memory Representations in Perception

� In�search�for�au��itor���object�re�resentations� 77

e�iso��ic�an���abstract�lon��er-ter�-li�e�features.�The�elicitation�of�MMN,��hen�lis-teners�are�not�a�are�of�the�un��erl��in���re��ularit���(van�Zuijen�et�al.�2006),�in��icates�that�the��e�or���re�resentations�involve���in�MMN���eneration��a���be�of�i��lic-it�nature.�Ta�en�to��ether,� these�observations���ive�rise�to�the��ossibilit���that�the��e�or���re�resentations�inferre���fro��stu��ies��easurin���MMN��a���be�relate���to�au��itor���object�re�resentations.�In�the�follo�in���section,��e�shall�co��are�the��ro�erties�of�these�re�resentations��ith�those�assu�e���for��erce�tual�objects.

3.3 Object-like properties of auditory regularity representations

What� ��o� �e� ex�ect� of� �erce�tual� objects?� We� clai�e��� in� the� intro��uction� of�this�cha�ter�that�an�object�has��ro�erties��hich�can�be���iscovere��.�We�can�no��a�����that��erce�tual�objects�al�a��s�have�several� features.� It� is� the�co�bination�of� features� that���efines� the�object.�Therefore,� in� search� for�au��itor���objects� in�the�brain,��e��ust�fin���re�resentations�inte��ratin���bet�een�features.�The�first�subsection��ill�exa�ine�this�question�for�the�re��ularit���re�resentations�in��exe���b���the�MMN�res�onse.

Most���efinitions�of�the�notion�of�object�e��hasize�that�the�object� is�a�unit��hich�is�se�arable�fro��other�ele�ents�of�the��orl��.�That�is,��e�shoul���be�able�to���eci��e��hether�or�not�a���iven�ele�ent�is��art�of�the�object�or�not.�This��ust�then�be�an�i��ortant�criterion�for��erce�tual�objects�as��ell.�The�secon���subsection��ill�a����ress�this�issue.

Finall��,��e�ex�ect�objects�to�for��no��es�of�invariance�in�the�ever-chan��in���sensor����orl��.�That�is,��hereas�the�actual�sensor���in�ut��a���chan��e���rasticall���in� ti�e,� object� re�resentations� shoul��� be� stable.� D��na�ic� chan��es� of� the� ob-ject� or� the� �erceiver,� such� as� �al�in��� aroun��� a� table� or� listenin��� to� a� �elo�����trans�ose���in��itch�shoul���not�result�in�the�for�ation�of�se�arate�object�re�re-sentations.�Thus��erce�tual�object�re�resentations�shoul���inclu��e���eneralization�across�the���ifferent��a��s�in��hich�the�sa�e�object��a���a��ear�to�our�senses.�In�the�thir���subsection,��e�shall���iscuss���eneralizations�across���ifferent�exe��lars�of�the�sa�e�re��ularit��.

3.3.1 Au��itor���feature�bin��in��

Feature�inte��ration�(feature�bin��in��)�has�been�the�focus�of�an�i��ortant���ebate�in�ex�eri�ental��s��cholo�����for�the��ast�40���ears.�The�question�at�the�heart�of�the���e-bate�is,��hether�or�not�focuse���(s�atial)�attention�is�necessar���for�joinin���the�fea-tures�of�a�sti�ulus.�Treis�an’s�attentive�feature�inte��ration�theor���(Treis�an�an���

Page 89: Unconscious Memory Representations in Perception

7�� István�Win�ler

Gela��e�1980;�Treis�an�1993)�su����ests�that,��hereas�the��rocessin���of�in��ivi��ual�features�is�auto�atic�an����rocee��s�in��arallel�for�a��hole�visual�scene,�inte��ration�bet�een���ifferent�features�is�a�serial��rocess��rocee��in���fro��location�to�location�to��ether��ith�the�focus�of�attention.�Treis�an�assu�es�that� in��ivi��ual� features�are��re-attentivel����a��e���on�se�arate�s�atial��a�s.�Ho�ever,�feature��a�s�are�boun���to��ether�in�object�files�onl���for�one�s�atial�location�at�a�ti�e.�The�location�is�selecte���b���attention.�A�lar��e�nu�ber�of�visual�ex�eri�ents��rovi��e���evi��ence�co��atible��ith�this�theor��.�One�i��ortant��iece�of�su��ortin���evi��ence�is�that�illusor���feature�conjunctions�e�er��e�at�locations��hich�have�not�been��arse���at-tentivel���(Treis�an�an���Sch�i��t�1992).�That�is,�subjects��a���re�ort�the��resence�of�ite�s��ith�such�a�co�bination�of�features��hich�have�not�been�actuall����resent�on�the���iven�sti�ulus���is�la��.�Ho�ever,��ost�results�su��ortin���the�h���othesis�of�attentive�feature�inte��ration�can�also�be�ex�laine���b���object�base���accounts�of�visual� �erce�tion� �hich� su����est� that� �erce�tual� units� (objects)� are� for�e��� in-��e�en��entl���of�the���irection�of�the�focus�of�attention�(Duncan�an���Hu��hre��s�1989;�Wolfe�et�al.�1989;�for�a�revie�,�see�Scholl�2001).

Althou��h�both�theories�are��re��o�inantl���base���on�results�of�stu��ies�in�vi-sion,�the�issue�of�feature�bin��in���is�relevant�for�all�sensor����o��alities.�Unfortu-natel��,�co��are���to�vision,��uch�fe�er�stu��ies�tar��ete���feature�inte��ration�in�the�au��itor����o��alit��.�Illusor���conjunctions�of�au��itor���features�have�been�foun���for�unatten��e���soun��s�(Deutsch�1982;�Hall�et�al.�2000;�Tho��son�et�al.�2001).�Al-thou��h,�si�ilarl���to�their�visual�counter�arts,�these�results�su��ort�the�h���othesis�of�attentive�feature�inte��ration,�just� li�e�in�the�visual�case,� it� is�not�clear,��hich�sta��e�of�sti�ulus��rocessin�����ives�rise�to�this��heno�enon.

Stu��ies�testin���the�conjunction�of�au��itor���features�for�the��e�or���re�re-sentations�involve���in�MMN���eneration�(Go�es�et�al.�1997;�Suss�an�et�al.�1998;�Ta�e��ata�et�al.�1999;�2005;�Win�ler�et�al.�2005)�use���the�follo�in���lo��ic�(see�Fi��-ure�2).�T�o�(or��ore)�soun��s���ifferin���fro��each�other�in�t�o�au��itor���features�are��resente���frequentl���in�the�soun���sequences.�Occasionall��,�soun��s��ith�one�feature�i��entical�to�one�of�the�frequent�soun��s�an���another�feature�i��entical�to�the�other�frequent�soun���are���elivere���in�the�sequences.�For�exa��le,�Win�ler�et�al.�(2005)��resente���sequences�ran��o�ize���fro��lo�-frequenc���tones���eliv-ere���fro��the�left�lou��s�ea�er�(45%)�an���hi��h-frequenc���tones���elivere���fro��the�ri��ht�lou��s�ea�er�(45%).�Lo��tones���elivere���fro��the�ri��ht�(5%)�an���hi��h�tones���elivere���fro��the�left�(5%)�lou��s�ea�er�a��eare���occasionall����ithin�the�sequences.�Thus�in�this��ara��i���,�infrequent�soun��s���o�not�carr���features��hich�are� infrequent� in� the�sequence�per se.�Onl��� the�co�bination�of� the� features� is�infrequent.�Therefore,�MMN�elicitation�b���the�soun��s��ith�infrequent�co�bina-tion�of�the�t�o�features�sho�s�that�the�feature-co�binations�carrie���b���the�fre-quent�soun��s�have�been�re��istere���in�the��e�or���involve���in�MMN���eneration,�

Page 90: Unconscious Memory Representations in Perception

� In�search�for�au��itor���object�re�resentations� 79

allo�in���the���eviance���etection�s��ste��to�fin���the�soun��s��ith�the�infrequent�co�binations�of�these�features.�

All�of�the�above�liste���stu��ies,��hich�use�����ifferent��airs�of�features,�obtaine���MMN� for� feature-conjunction� ��eviants.� Further�ore,� Win�ler� et� al.� (2005)�sho�e���that�MMN�elicitation��as�not�affecte���b���the���ifficult���of�a��ithin-�o��al-it����ri�ar���tas�.�This�su����este���that�feature�bin��in�����i���not�require�li�ite���ca-�acities.�Ho�ever,�it�is��ossible�that�feature�inte��ration�is�onl���auto�atic��hen�the�sensor���in�ut�consists�of�onl���one�object�at�a�ti�e.�Ta�e��ata�et�al.�(2005)��resente���their�t�o�frequent�soun��s�(a�hi��h�violin�an���a�lo���iano�soun��)�si�ultaneousl��,�occasionall��� exchan��in��� the� �itch� bet�een� the� t�o� soun��s.� MMN� �as� elicite���even�in�this�case,�irres�ective�of�the���ifficult���of�the�visual��ri�ar���tas��that�sub-jects��erfor�e�����urin���the���eliver���of�the�au��itor���sti�uli.�The�MMN�a��litu��e�obtaine��� �ith� si�ultaneous� �resentation� of� t�o� soun��s� �as� so�e�hat� hi��her�than�that�obtaine���in�a�sti�ulus�bloc���resentin���both�the�frequent�an���the�infre-quent�soun��s�alone�(serial�sti�ulus��resentation;�see�Fi��ure�2).�If,�as�the�attentive�feature�inte��ration�theor���ex�ects,�no�feature�conjunctions��ere�for�e����ithout�attention,� then� no� MMN� shoul��� have� been� elicite��.� This� is� because,� if� illusor���feature�conjunctions�e�er��e����ithin��re-attentive��rocessin���then�the�MMN�a�-�litu��e�shoul���have�been�si��nificantl���lo�er�in�the�si�ultaneous�than�in�the�serial��resentation�con��ition,�because��an���stan��ar����airs��oul���have�been�enco��e���as���eviants�an���vice versa.�The�a��litu��e�relationshi��bet�een�the�si�ultaneous�an���serial� sti�ulus��resentation�con��itions� su����est� that� features��ere�correctl���conjoine��� for� all� soun��s.� Ho�ever,� illusor��� conjunctions� �ere� observe��� in� the�

Figure 2. Sche�atic���ia��ra��of�the��rotot���ical�sti�ulus��ara��i����e��lo��e���in�MMN�stu��ies�of�feature�inte��ration.�Frequent�soun��s���ifferin���in�t�o�au��itor���features�(e.��.�Feature�1:�soun���source�location,����axis;�Feature�2:�tone�frequenc��,�sha��in��)�are�in-ter�ixe����ith�infrequent�soun��s�co�binin���a�feature�fro��one�frequent�soun����ith�another�feature�fro��the�other�frequent�soun��.�Soun��s�are��ar�e���as�rectan��les;�ti�e�is�re�resente���b���the�x�axis.�(A��a�te����ith��er�ission�fro��Win�ler�et�al.�2005.)

Page 91: Unconscious Memory Representations in Perception

�0� István�Win�ler

sa�e�stu����,��hen��artici�ants��ere�instructe���to�tell,��hether�a���iven�soun����as�i��entical�to�one�of�the�t�o�si�ultaneousl����resente���test�soun��s.�For�exa��le,�in�one�trial,� the�hi��h��iano�an���the�lo��violin�soun����ere���elivere���to��ether.�The��artici�ant��as�then��resente����ith�a� lo���iano�soun���an���as�e����hether�this�soun����as�one�of�the�t�o�soun��s�he/she�hear���just�before.�This�co�bination�of�the�MMN�an���active�au��itor���search�results�su����est�that,�at�least�in�the�au��itor����o-��alit��,�illusor���feature�conjunctions��robabl���e�er��e��ithin�attentive��rocessin��.

In� su��ar��,� stu��ies� of� au��itor��� feature� bin��in��� sho�e��� that� the� �e�or���re�resentations� un��erl��in��� the� ��eviance� ��etection� function� reflecte��� b��� MMN�enco��e� the� co�bination� of� features� belon��in��� to� re��ular� (frequent)� soun��s� ir-res�ective�of�the���irection�of�attention.�Thus�these��e�or���re�resentations�sho��this�i��ortant��ro�ert���of�object�re�resentations.

3.3.2 Se�aration�of�concurrent�au��itor���objects

In��ost�ever����a���situations,�soun��s�ori��inatin���fro���ulti�le�sources�are��resent�at�an�����iven�ti�e.�Mo��ern�stu�����of�the�se�aration�of�inter�ixe���soun���sequences�in�the�brain�starte����ith�the���iscover���of�au��itor���strea�in���(van�Noor��en�1975),��hich��rove���to�be�an�excellent�testin�����roun���for�the��erce�tual��rocesses�in-volve���in�se�aratin���au��itor���objects.�In�the�au��itor���strea�in����ara��i����(Fi��-ure�3),�t�o�soun��s���ifferin���in�one�or��ore�features�(�ost�t���icall��,��ure�tones���ifferin��� in� frequenc��)� are� alternate��.� De�en��in��� on� the� acoustic� �ara�eters,�such� a� sequence� can� be� hear��� as� a� sin��le� strea�� of� soun��s� (ter�e��� the� “inte-��rate��”��erce�tion)�or�as�t�o�se�arate�soun���strea�s,�one�for�each�of�the�t�o���if-ferent�soun��s�(ter�e���“se��re��ate��”��erce�tion).�For�easier�testin���of�the�listener’s��erce�tion,�often�ever���secon����resentation�of�one�of�the�t�o�soun��s�is�o�itte��.�This�results�in�the�sequence�bein����erceive���as�a�series�of�soun���tri�lets���ivin���the� i��ression�of���allo�in��,�or�as� t�o�se�arate� soun���strea�s�havin�����ifferent�isochronous�rh��th�s�(see�Fi��ure�3).�

Figure 3. Cartoon�of�the��rotot���ical�au��itor���strea�in����ara��i���.�A�sequence�of�lo��(A)�an���hi��h�(B)�tones�is��resente���re�eate��l���in�ABA���rou�s.�This�sequence�can�be��erceive���as�a�sin��le�coherent�soun���strea���ith�a���allo�in���rh��th��(u��er�ri��ht),�or�as�t�o�se��re��ate���strea�s�(lo�er�ri��ht),�each��ith�a�(��ifferent)�isochronous�rh��th�.

Page 92: Unconscious Memory Representations in Perception

� In�search�for�au��itor���object�re�resentations� �1

Presentin���the�soun���sequence�sho�n�in�Fi��ure�3�at�a�hi��h��ace�(short�∆t)�an����ith�lar��e�frequenc���se�aration�(∆f)�bet�een�the�t�o�soun��s��ro�otes�se��-re��ation�of�t�o�strea�s�(van�Noor��en�1975);�an��,�conversel��,�slo���resentation�an��� s�all� frequenc��� se�aration� �ro�otes� the� �erce�tion� of� a� sin��le� strea�� of�soun��.�There�are�also�co�binations�of�the�t�o��ara�eters�(e.��.,��e��iu��to�lon���∆t�co�bine����ith��e��iu��to�lar��e�∆f)��ith��hich�the�or��anization�of�the�soun���sequence�can�be�voluntaril���selecte���(for�further���etails,�see�Section�3.4).�It�shoul���be�note���that,�in�ter�s�of�veri��ical��erce�tion�of�soun���sources,�au��itor���strea�-in���can�be�re��ar��e���an�illusion,�because�even��hen�the�soun���sequence�is���eliv-ere���b���the�sa�e��h��sical�source,�one��a����erceive�t�o�se�arate�strea�s.�Just�li�e��hen��e�hear�t�o�or��ore�concurrent�voices,�instru�ents,�etc.�on�the�ra��io.�Thus,�althou��h�soun���strea�s���o�not�necessaril���corres�on���to�soun���sources�(concrete�objects),�the���are�ex�erience���as�se�arate�(abstract)�objects.�Bre���an’s�(1990)�in-fluential�theor���of�“Au��itor���Scene�Anal��sis”�su����ests�that�the�brain�for�s�au��i-tor���strea�s�in�t�o�sta��es.�In�the�first�sta��e,�a�lar��e�nu�ber�of�heuristic���rou�in���al��orith�s�anal��ze�the�au��itor���in�ut,�searchin���for�the��resence�of�various�cues�that�si��nal�to�the�au��itor���s��ste���hich�ele�ents�of�the�inco�in���soun���belon���to��ether.�The�cues�that�these�al��orith�s�search�for�corres�on���to�the��erce�tual���rou�in����rinci�les�as�ori��inall�����escribe���b���researchers�of�the�Gestalt�school�of��s��cholo�����(e.��.,�Köhler�1947).�Grou�in����rinci�les,�such�as�si�ilar�ele�ents�or�ele�ents�encountere����ithin�a�short��erio���of�ti�e�usuall���belon���to��ether,�are�thus�utilize���b���the�brain�in�for�in����ossible�soun���or��anizations.�Bre���an�(1990)���istin��uishes�al��orith�s�searchin���for�i��e��iate/s�ectral�cues��hich���o�not�require�reference�to�the�histor���of�sti�ulation,�an���al��orith�s�searchin���for�sequential/te��oral�cues��hich�atte��t�to�fit��arts�of�the�au��itor���in�ut�to�the�tren��s� ��etecte��� �ithin� the� i��e��iatel��� �rece��in��� soun��s.� For� exa��le,� tonal�co��onents��hose�frequencies�are�inte��er��ulti�les�of�the�sa�e�base�frequenc���are�usuall�����rou�e���to��ether�an����erceive���as�a�co��lex�tone,��hereas�a�co�-�onent��hose�frequenc���is�not�an�inte��er��ulti�le�of�the�sa�e�base�frequenc���(a��istune����artial)�is�hear���se�aratel���(i��e��iate�cue).�Si�ilarl��,��ithin�the�con-text�of�a�soun����onotonousl�����escen��in���in��itch,�further��itch���escent�is�li�el���to�be�ta�en�as�the�continuation�of�the��revious�soun��,��hereas�a�su����en�ju���in��itch� is�usuall��� se��re��ate��� fro��it� (sequential�cue).�Another���istinction�can�be� �a��e� bet�een� �ri�itive� an��� sche�a-base��� ��rou�in��� al��orith�s.� Bre���an�(1990)�su����ests�that�the�for�er,�such�as�searchin���for��itch�se�aration,�is��rob-abl���innate,��hereas�the�latter,�such�as��hen�the�con��uctor�can�follo��the��elo�����carrie���b���a�sin��le�instru�ent��ithin�the�orchestra,�are�learne��.�Inevitabl��,�not�all�solutions�to�soun���or��anization�e�er��in���fro��the�various���rou�in���al��orith�s�are�co��atible��ith�each�other.�Therefore,�in�the�secon���sta��e�of�au��itor���scene�anal��sis,�inco��atibilities�are�resolve���b���co��arin���the�su��ort�for�the�various�

Page 93: Unconscious Memory Representations in Perception

�2� István�Win�ler

alternatives�(see�in��ore���etail�in�Section�3.5).�The���o�inant�or��anization�then�a��ears�in��erce�tion�(Bre���an�1990).

There�is,�of�course,��uch��ore�to�the�theor���of�au��itor���scene�anal��sis�than��hat�has�been�su��arize���above.�These�basic� features,�ho�ever,���ive�us�so�e�i��ea�about�the��ro�erties�of�au��itor���object�re�resentations.�Firstl��,�one�ex�ects�these� re�resentations� to� closel��� follo�� the� �erceive��� soun��� or��anization.� Sec-on��l��,�if�Bre���an’s�(1990)�theor���is�correct,�evi��ence�for�the�t�o��hases�of�au-��itor��� scene� anal��sis� shoul��� be� foun��� �ithin� the� au��itor��� s��ste�.� Thir��l��,� the���ifferent�t���es�of�heuristic�al��orith�s�(see�above�for�the���istinctions�su����este���b����Bre���an)�shoul���be���istin��uishable�b���the�un��erl��in����atterns�of�neural�activit��.�Finall��,��ith�the�notable�exce�tion�of���u�lex��erce�tion�(Fo�ler�an���Rosenblu��1990;�Ran���1974),�ele�ents�in�the�sensor���in�ut�shoul���belon���to�onl���one��er-ce�tual�object�re�resentation.

Testin����hether�or�not�the��e�or���re�resentations�reflecte���b���the�MMN�res�onse�follo���erceive���soun���or��anization�has�been�base���on�the��rinci�le�that�certain�au��itor���re��ularities�can�onl���be���iscovere����hen�soun��s�are��er-ceive��� accor��in��� to� a� ��iven� soun��� or��anization.� For� exa��le,� a� tune� can� be�hi����en�b���alternatin���ran��o��soun��s��ith�the�notes�in�the��elo����.�When�the�frequencies� of� the� ran��o�� intervenin��� soun��s� are� close� to� those� of� the� tune,�listeners�cannot��a�e�out�the�tune.�Ho�ever,��hen�the�intervenin���soun��s�can�be�se��re��ate���b����itch�se�aration�fro��those�of�the�tune,� the�tune�e�er��es� in��erce�tion�(Do�lin���1973).�Usin���this��heno�enon�(see�Fi��ure�4,�illustratin���a�t���ical��ara��i����of�this�t���e),�several�MMN�stu��ies��ixe���to��ether�t�o�soun���sequences�(co��onent�sequences),�each�havin���its�o�n�re��ularities�(De�Sanctis�et�al.�2008;�Müller�et�al.�2005;�Na��er�et�al.�2003;�Ritter�et�al.�2000,�2006;�Suss�an�2005;�Suss�an�et�al.�1998,�1999,�2001,�2005,�2007;�Win�ler�et�al.�2003a,�b,�c;�for�a�revie�,�see�Suss�an�2007).�If�the��e�or���re�resentation�involve���in�MMN���en-eration�reflecte���the��erceive���(��o�inant)�soun���or��anization,�then�occasional���eviants�violatin���the�re��ularit���of�one�or�the�other�co��onent�sequence��ere�ex�ecte��� to� elicit� MMN� �hen� the� t�o� co��onent� sequences� �ere� se��re��ate���but�not��hen�the����ere�inte��rate��.�Althou��h�not�all�of�the�above-liste���stu��ies�teste���the��erce�tion�of�the�soun���sequences�(rel��in���on��ara�eters��no�n�to��ro��uce�inte��ration�an���se��re��ation),�all�of�the��foun���results�co��atible��ith�the� notion� that� MMN� elicitation� �arallele��� the� se��re��ation� of� the� interleave���soun���sequences.�Thus�it�a��ears�that�the��e�or���re�resentations�involve���in�the���eneration�of�MMN�co-var����ith�the��erceive���soun���or��anization.

So�e�stu��ies�a��lie���the�o��osite�lo��ic,��resentin�����eviations��hich�coul���onl���elicit�MMN��ith�the�inte��rate���soun���or��anization�(Shinoza�i�et�al.�2003;�Win�ler�et�al.�2005;�Yabe�et�al.�2001).�Previous�stu��ies�sho�e���that�soun���o�is-sions� onl��� elicit� MMN� �hen� the� onset� of� consecutive� soun��s� (sti�ulus� onset�

Page 94: Unconscious Memory Representations in Perception

� In�search�for�au��itor���object�re�resentations� �3

as��nchron��,�SOA)�are�se�arate���b���less�than�ca.�170��s,�the�assu�e�����uration�of� the� te��oral� �in��o�� of� inte��ration� (Z�isloc�i� 1960;� Horváth� et� al.� 2007;��Tervanie�i�et�al.�1994b;�Yabe�et�al.�1998).�The�stu��ies�testin���the�occurrence�of�inte��rate���soun���or��anization�in�the�au��itor���strea�in����ara��i����utilize���this�o�ission��heno�enon.�When�the�overall�SOA�of�a�sequence�of�t�o�alternatin���soun��s�is�bet�een�85�an���170��s,�soun���o�issions�can�onl���elicit�MMN�if�all�soun��s�in�the�sequence�are�inte��rate���into�a�sin��le�strea�.�This�is�because,�if�the�t�o�soun��s��ere�se��re��ate���into�se�arate�strea�s,�the�SOA��ithin�each�strea��se�aratel��� excee��s� the� ti�e� �in��o�� �ithin� �hich� soun��� o�issions� elicit� the�

Figure 4. Sche�atic���ia��ra��of�three�sti�ulus�con��itions���esi��ne���to�test�soun���or��a-nization�b���re��ularit�����etection.�Rectan��les�re�resent��ure�tones�in�the�ti�e�–�frequenc�����ia��ra�.�Sha��in���of�the�rectan��les��ar�s�tone�intensit���(��ar�er���re���for�hi��her�inten-sit��).�The�“Re��ularit���Alone”�con��ition�sho�s�a�si��le�o����ball�sequence��ith�occasional�shorter���eviant�tones�(�ar�e����ith�the�chec�ere����attern).�These���eviants�are�ex�ecte���to�elicit�the�MMN�res�onse.�In�the�“Hi����en�Re��ularit��”�con��ition,�t�o�ran��o��tones�are�intro��uce���bet�een�successive�tones�of�the�o����ball�sequence.�The�intervenin���tones�var���in��itch,�intensit���an���also���uration.�Because�the��itch�ran��e�of�the�intervenin���tones�inclu��es�the��itch�of�the�tones�in�the�ori��inal�o����ball�sequence,�all�tones�are�inte��rate���into�a�sin��le�strea��in��erce�tion�an���the�initial�re��ularit���is�obscure���b���the�variabilit���of�the�intervenin���tones.�Without���etectin���a�re��ularit����hich��oul���be�violate���b���the�ori��inal���eviant�tones,�these�tones���o�not�elicit�MMN.�In�the�“Re��ularit���Reveale��”�con-��ition,�the��itch�ran��e�of�the�intervenin���tones�has�been�re�ove���fro��the�vicinit���of�the��itch�of�the�tones�in�o����ball�sequence.�As�a�consequence,�the�o����ball�sequence�can�be�se��re��ate���fro��the�intervenin���tones.�When�this�ha��ens,�the�re��ularit���of�the�o����ball�sequence�is���iscovere���an���the�short���eviant�tones�elicit�MMN�a��ain.�(A��a�te���fro���ith��er�ission�Win�ler�et�al.�2003c.)

Page 95: Unconscious Memory Representations in Perception

�4� István�Win�ler

MMN�(SOA�>�2�×�85��s�=�170��s).�Presentin����ure�tones���ifferin���in�frequen-c��� in�the���allo�in���version�of� the�au��itor���strea�in����ara��i����(see�Fi��ure�3),��Win�ler�et�al.�(2005)�as�e����artici�ants�to�continuousl���in��icate�their��erce�-tion�of�the�tone�sequence�b����ee�in���a�res�onse�button���e�resse����henever�the���hear��� the���allo�in���rh��th�.�In�one�con��ition,�sti�ulus��ara�eters��ro�ote���se��re��ation�of�the�t�o�tones�into�se�arate�strea�s�(strea�in���con��ition);�in�the�other�con��ition,��ara�eters�allo�e����erce�tion�to�fluctuate�bet�een�se��re��ation�an���inte��ration�(a�bi��uous�con��ition).�Occasional�tone�o�issions�elicite���t�o���ifferent�fronto-centrall���ne��ative�ERP�res�onses.�The�earlier�res�onse�(�ea�in���bet�een�56�an���72��s�fro��the�ti�e�of�the�o�ission)��as�elicite���b���o�issions�in�the�a�bi��uous�sti�ulus�sequence,�but�not�in�the�strea�in���sequence.�Ho�-ever,�the�elicitation�of�this�co��onent���i���not���e�en���on��hether�the��artici�ant��erceive���the�sequence�accor��in���to�the�inte��rate���or�the�se��re��ate���or��aniza-tion.� The� later� o�ission� res�onse� (�ea�in��� bet�een� 170� an��� 180��s� fro�� the�start�of�the�o�ission)��as�onl���elicite����hen��artici�ants��erceive���the�sequence�as�a�sin��le�inte��rate���strea�,�but�not��hen�the����erceive���t�o�strea�s.�Thus,�in�accor��ance��ith�the�results�of��revious�stu��ies,�tone�o�issions�onl���elicite���an�ERP�res�onse,��hen,���ue�to���rou�in���the�hi��h�an���lo��tones�to��ether,�succes-sive� tones� co��ence��� �ithin� the� �reviousl��� observe��� 170-�s� te��oral� �in-��o�.�Ho�ever,��hereas�the�earl���o�ission�res�onse��as�full�����eter�ine���b���the�sti�ulus��ara�eters� the� later�res�onse��arallele���the��artici�ant’s��erce�tion,�follo�in���its�����na�ic�fluctuation��ithin�the�sti�ulus�sequences.�We�inter�ret�these�results�as�reflectin���t�o���ifferent�sta��es�in�au��itor���strea��se��re��ation.�The�earl���res�onse�reflects�the�outco�e�of�a��rocess�(or��ulti�le��rocesses),��hich���eter�ine�se��re��ation�vs.�inte��ration�base���on�acoustic��ara�eters�(�ost�li�el��,�the��ithin-strea��inter-sti�ulus�interval,�because�this��as�the��ara�eter�varie���bet�een�the�t�o�sti�ulus�con��itions;�cf.�Bre���an�et�al.�2000).�In�contrast,�the�late�o�ission�res�onse�reflects�the�outco�e�of�au��itor���scene�anal��sis,�����na�i-call���co-var��in����ith��erce�tion.�

As��e�state���in�Section�3.2�soun��s�elicit�MMN,��hen�the���violate�so�e�re��u-lar�feature�of�the��rece��in���sequence.�Thus�the�results�obtaine����ith�the�MMN�res�onse� characterize� the� extraction� an��� re�resentation� of� sequential� re��ulari-ties.�An�ERP�res�onse,�ter�e���object�relate���ne��ativit���(ORN),�acco��an��in���an�i��e��iate�for��of�se��re��ation�has�been���iscovere���b���Alain�an���his�collea��ues�(Alain�an���McDonal���2007;�Alain�et�al.�2001;�Alain�et�al.�2002;�Zen��el�an���Alain�2009).�The�ORN�is�a�fronto-centrall����axi�al�ERP�co��onent��ea�in���bet�een�140�an���180��s�fro��sti�ulus�onset.�It�is�elicite���b����istunin���a�har�onic��artial�in�a�co��lex�tone.�ORN�is�elicite���even��ithout�focuse���attention.�The�elicita-tion�of�ORN�acco��anies��erce�tion�of�the��istune����artial�an���the�rest�of�the�

Page 96: Unconscious Memory Representations in Perception

� In�search�for�au��itor���object�re�resentations� �5

har�onic�co��lex�as�se�arate�soun��s.�Thus�it�a��ears�that�ORN�reflects�soun���se��re��ation�b���har�onic�(s�ectral)�cues.

Se��re��atin���soun���strea�s�b���frequenc���se�aration,�as��as�sho�n��an���ti�es�usin��� the� au��itor��� strea�in��� �ara��i���� (see� above),� has� been� re��ar��e��� as� one�of�the��ri�itive�for�s�of�strea��se��re��ation.�In�a��revious�stu�����(Win�ler�et�al.�2003b),��e�sho�e���that,�si�ilarl���to�a��ults,��ith�lar��e�frequenc���se�aration�an���fast��resentation�rates,�the�ne�born�au��itor���s��ste��se��re��ates�strea�s�of�tones.�Thus�the�al��orith�s�un��erl��in���au��itor���strea�in���b����itch�se�aration�are��rob-abl���innate,�as��as�assu�e���b���Bre���an�(1990).�On�the�other�han��,�sche�a-base�����rou�in���al��orith�s�are�ex�ecte��� to�be� learne��.�Countin��� the�notes� in�a�bar,�a�s�ill�often�use���b����usicians,�is�a�t���ical�sche�a-base���al��orith�.�We�teste���the�functionin���of�this�al��orith��b����resentin����rofessional��usicians�an���non-�u-sician��artici�ants��ith�sequences��a��e�u��of�tone���rou�s�se�arate���in�frequenc���an���tone���uration�(van�Zuijen�et�al.�2005;�see�Fi��ure�5).�Durin���the��resentation�of� the� tone� sequences,� �artici�ants� �atche��� a� silence��� �ovie� �ith� subtitles.� In�the� ‘Countin��’� con��ition,� stan��ar�����rou�s� (90%)�consistin���of�4� tones��ere� in-ter�ixe����ith���eviant���rou�s�(10%)�consistin���of�5� tones.� In� the� ‘Ti�e’�con��i-tion,�stan��ar���tone���rou�s�laste���for�750��s�an�����eviants�for�900��s.�Whereas�the�‘Ti�e’���eviants�elicite���MMN�in�both���rou�s�of��artici�ants,�‘Countin��’���eviants�onl���elicite���MMN�in��rofessional��usicians.�These�results�su����est�that��hereas�the�al��orith�s�extractin���the�ti�e�of�soun�����rou�s�are�functional�in��ost�a��ults�even��hen�the�soun��s�are�not�tas�-relevant,�al��orith�s�“countin��”�the�nu�ber�of�ite�s�in�a�soun�����rou��onl���beco�e�auto�atic���urin����usical�trainin��.�Thus�this�stu������rovi��e���evi��ence�for�the�learnin���of�a�sche�a-base�����rou�in���al��orith�.�Naturall��,�auto�atic�o�eration�is�not�a��re-requisite�of�the�functionin���of�a���iven���rou�in���al��orith�.�So�e���rou�in���al��orith�s��a���require�focuse���attention.�The�role�of�attention�in�au��itor���strea��se��re��ation�re�ains�a��oint�of���is�ute.�So�e�results�su����est�that�attention��a���be�nee��e���even�for��ri�itive�for�s�of�strea��se��re��ation�(Carl��on�et�al.�2001;�Cusac��et�al.�2004),��hereas�other�results�ar��ue�a��ainst�the�require�ent�of�focuse���attention�in�such�cases�(Suss�an�et�al.�2007;�Win�ler�et�al.�2003c).�Other�fin��in��s�su����est��arallel�functionin���of�attentive�an����re-attentive��rocesses�in�au��itor���strea�in���(Sn����er�et�al.�2006;�for�a�revie��of�the�role�of�attention�in�au��itor���strea��se��re��ation,�see�Sn����er�an���Alain�2007).

We�ar��ue���above�that�the�re��ularit���re�resentations�un��erl��in���MMN���en-eration�are�s�ecific�to�au��itor���strea�s.�But�is�this�also�true�for�each�in��ivi��ual�soun��?�Ritter�et�al.�(2000,�2006)�an���De�Sanctis�et�al.�(2008)�sho�e���that�soun��s�elicit�the�MMN�b���violatin���a�re��ularit���of�the�strea��that�the���belon���to,�but�not�b���violatin���a� re��ularit���of�another� soun���strea�.�This��eans� that� the���eviant�soun����as�exclusivel���assi��ne���to�one�strea�.�In�vision,�objects�are�se�arate���fro��each�other�b��� s�atio-te��oral�bor��ers.� In� search�of�au��itor���objects,� a� si�ilar��

Page 97: Unconscious Memory Representations in Perception

��� István�Win�ler

Figu

re 5

. St

i�ul

us�co

n��iti

ons�f

or�th

e�ex

�eri�

ent�c

o��a

rin���

��rou

�in��

�al��

orith

�s�i

n��r

ofes

siona

l��us

icia

ns�a

n���n

on��

usic

ians

.�Seq

uenc

es�

cons

iste��

�of�t

one�

��rou

�s�se

�ara

te���

in�fr

eque

nc���

an���

tone

���ur

atio

n;�to

nes��

ithin

�eac

h���r

ou��

ha���

the�

sa�

e�fr

eque

nc���

an���

��ura

tion.

�‘Cou

nt-

in��’

�con��

ition

�(left

):�Th

e��

ajor

it���o

f�ton

e���r

ou�s

�cons

iste��

�of�4

�tone

s;�oc

casio

nal���

evia

nt���

rou�

s�ha��

�5�to

nes.�

The�

��ura

tion�

of�th

e�to

ne���

rou�

s�va

rie���

ran��

o�l��

�bet

�ee

n�61

0�an

���89

0��

s.�‘T

i�e’�

con��

ition

�(ri��

ht):�

The�

��ura

tion�

of�th

e��

ajor

it���o

f�ton

e���r

ou�s

��as

�750

��s;�

occa

siona

l���ev

iant

���r

ou�s

��er

e�ex

ten��

e���to

�900

��s.�

The�

nu�

ber�o

f�the

�tone

s�in�

the�

��rou

��ra

n��o�

l���v

arie

���be

t�ee

n�2�

an���

6.�(A

��a�t

e����

ith��

er�

issio

n�fr

o��v

an�

Zuije

n�et

�al.�

2005

.)

Page 98: Unconscious Memory Representations in Perception

� In�search�for�au��itor���object�re�resentations� �7

�rinci�le��a���be�set�u�.�Ho�ever,�since��e�concentrate�on�abstract�au��itor���ob-jects,�s�ace�cannot�be�the��ro�ert���in�the�au��itor����o��alit����ithin��hich�au��i-tor��� objects� �a��� for�� bor��ers.� Kubov��� an��� Van� Val�enbur��� (2001)� ar��ue� for�the�existence�of�s�ectro-te��oral�bor��ers�of�au��itor���objects.�That� is,�au��itor���objects��a���be�se�arate���fro��each�other�b���bor��ers���ra�n�in�the�s�ectral�s�ace.�The�vali��it���of�this�notion�can�be�assesse���b���ta�in���another�analo�����fro��visual�objects:� exclusive� assi��n�ent� of� the� bor��ers.� When� t�o� visual� objects� overla��each�other�in�the�visual�in�ut�(i.e.,�fro��the��oint�of�vie��of�the�observer),�the�bor��er�bet�een�the��is�assi��ne���to�the�object�closer�to�the�observer�(the�fore-��roun��).�Un��er�nor�al�circu�stances,�usuall���a�sufficient�nu�ber�of���e�th�cues�are�available�for�the�observer�to�reliabl�����eter�ine�fore��roun���an���bac���roun��,�an���thus�the�assi��n�ent�of�the�bor��er�is�una�bi��uous.�It�is,�ho�ever,��ossible�to�construct���is�la��s,�in��hich�no�a����itional���e�th�cues�are�available�(e.��.,�Rubin’s�classical�face-vase�reversal;�Rubin�1915).�As�a�consequence,�fore��roun���an���bac�-��roun���are���eter�ine���b���the�assi��n�ent�of�the�bor��er.�The�object�receivin���the�bor��er�beco�es�fore��roun��,��hereas�the�other�the�bac���roun��.�In�or��er�to�assess�this��rinci�le�in�the�au��itor����o��alit��,��e�teste����hether�or�not�s�ectro-te��oral�bor��ers�are�assi��ne���exclusivel���to�one�of�t�o��ossible�au��itor���objects�(Win�ler�et�al.�2006).�Fi��ure�6�sho�s�the�sti�ulus��ara��i���.�Hi��h�(A,�C),��e��iu��(E),�an���lo���ure�tones�(B,�D)��ere��resente���in�a�c��cle��ith�a�fixe���or��er�(ABCDE).�The�frequencies�an���the�avera��e�SOA��as�set�so,�that�the�lo��an���hi��h�tones�coul���not�be� inte��rate��� into�a� sin��le� soun���strea�,��hereas� the��e��iu�-�itch� tones�coul���be���rou�e����ith�either�one�of�the�.�Fi��ure�6�sho�s�the��atterns�e�er��in����hen�the��e��iu�-�itch�tones�are���rou�e����ith�the�hi��h�(soli��-line�rectan��les)�or�the�lo��tones�(��ashe��-line�rectan��les).�SOA’s��ere�varie���ran��o�l��,�but��ithin�ran��es�allo�in���listeners�to��aintain�the�structure�of�the��atterns�(EAC�for�the��e��iu�-hi��h� an��� BDE� for� the� lo�-�e��iu�� ��rou�).� Occasionall��,� the� E� tone��as��resente���earl��,�resultin���in�the�for�ation�of���eviant��e��iu�-hi��h���eviant��atterns�(EACE�follo�e���b���AC;��ar�e����ith�thic��bor��ers�on�Fi��ure�6).�Ho�-ever,�the�lo�-�e��iu���attern��as�not�affecte���b���this��ani�ulation.�Partici�ants��ere�as�e���either� to���rou��to��ether� the��e��iu��an���hi��h�or� the��e��iu��an���lo��tones.�The����ere�to��aintain�this���rou�in���throu��hout�the�sti�ulus�bloc�s.�(Grou�in����as��onitore���throu��hout�the�ex�eri�ent�b���a�continuous���eviance���etection�tas�;�this� is�not��ar�e���on�the�fi��ure.)�Thus�the��e��iu�-�itch�tones�for�e���a�bor��er�bet�een�t�o�au��itor���strea�s�(the�hi��h�an���lo��strea�s)�an���listeners�voluntaril���assi��ne���this�bor��er�to�one�of�the�t�o�strea�s.�If�the�assi��n-�ent�of�the�bor��er��as�exclusive,�then�re��ularities�involvin���the�bor��er�shoul���onl���be���etecte���an���re�resente���for�the�strea��to��hich�the�bor��er��as�assi��ne���to.�That�is,�the���eviation�cause���b���the�occasional�earl����resentation�of�the��e-��iu�-�itch�tone�shoul���onl���elicit�the�MMN,��hen�the��artici�ant���rou�e���the�

Page 99: Unconscious Memory Representations in Perception

��� István�Win�ler

�e��iu�-�itch�tones��ith�the�hi��h�tones,�but�not��hen�he/she���rou�e���the�lo��an���the��e��iu�-�itch�tones�to��ether.�If,�ho�ever,�both��ossible���rou�in��s�of�the��e��iu�-�itch�tones�are�evaluate���b���the�au��itor���s��ste�,�MMN�shoul���be�elic-ite���b���the���eviant�tones�irres�ective�of�the��artici�ants’�voluntar�����rou�in��.�We�foun���that�MMN��as�onl���elicite����hen�the��artici�ant��aintaine���the���rou�in���that��ro��uce���the��attern��hich��as�violate���b���the���eviant.�This��eans�that�this��itch-ti�e�bor��er��as�exclusivel���assi��ne���to�one�strea�,�si�ilarl���to�the�exclu-sive�assi��n�ent�of�s�ace-ti�e�bor��ers�in�vision.

In�this�section��e�sho�e���that�(1)�re��ularit���re�resentations�are�for�e���ac-cor��in���to�the��erceive���soun���strea�s;�(2)�the�t�o�assu�e���sta��es�of�au��itor���scene�anal��sis�are�reflecte���b���ERP�res�onses�involvin���re��ularit���re�resentations;�(3)�i��e��iate�an���sequential�soun�����rou�in���al��orith�s�are�in��exe���b���se�arate�ERP� res�onses;� (4)� at� least� so�e� of� the� �ri�itive� soun��� ��rou�in��� �rocesses� af-fectin���the�re��ularit���re�resentations�are�innate,��hereas�so�e�sche�e-base���al-��orith�s�are� learne��;�an���(5)� soun��s�are�usuall���assi��ne��� to�onl���one�au��itor���strea��an���are�onl���ta�en�into�account��ith�res�ect�to�the�re��ularit���re�resenta-tions�belon��in���to�that�strea�.�In�su��ar��,�the��ro�erties�of�the�re��ularit���re�-resentations�involve���in�the�MMN-��eneratin����rocess��atch�those�ex�ecte���for�au��itor���object�re�resentations.

3.3.3 Generalizin���across���ifferent�sti�ulus�instances

In�the��revious�sections,��e�alrea�����ca�e�across�so�e��ara��i���s�in��hich�re��u-larities��ere�extracte���fro��acousticall���var��in���sti�uli.�The��ara��i����b����hich�MMN��as�intro��uce���in�Section�3.2�is�a���oo���exa��le�(Fi��ure�1;�Win�ler�et�al.�2003a).�In�this��ara��i���,�the�series�of�footste�s�for�in���the�stan��ar����as��a��e�u��of�ten���ifferent���i��itize���natural�footste��soun��s.�A��i��el�����ifferent�footste��

Figure 6. Sche�atic���ia��ra��of�the�sti�ulus��ara��i����for�stu����in���exclusive�bor��er�assi��n�ent�of�au��itor���objects.�Tones�are��ar�e����ith�lettere���rectan��les.�Tone�fre-quenc���is�calibrate���in�the����axis;�the�x�axis��ar�s�ti�e.�See�the�text�for�the�ex�lanation.�(A��a�te����ith��er�ission�fro��Win�ler�et�al.�2006.)

Page 100: Unconscious Memory Representations in Perception

� In�search�for�au��itor���object�re�resentations� �9

soun���elicite���the�MMN.�This�result�su����ests�that�the�re�resentation�un��erl��in�����eviance���etection���eneralize���across�the���ifferent,�but���enerall���si�ilar�foot-ste��soun��s,�re��ar��in���the��as�instances�of�the�sa�e�re��ularit��.

Several�stu��ies�assesse���the�effects�of�acoustic�variance�on�re��ularit���extrac-tion�via���eviance���etection.�The��ara��i���s�e��lo��e���in�these�stu��ies�fall�into�t�o�broa���cate��ories.�In�so�e�of�the�stu��ies,�acoustic�variance��as�intro��uce���in�a�feature��hich��as�irrelevant�for�the�re��ularit����hose���etection��as�teste��.�In,��erha�s,�the�best�exa��le�of�this�a��roach,�both�tone�frequenc���an���intensit����ere�varie���over�a��i��e�ran��e��hile�tone���uration��as��e�t�co��on�for��ost�tones.� Occasional� lon��er� tones� elicite��� the� MMN� res�onse� sho�in��� that� the�co��on�tone���uration��as�extracte���fro��a��i��el�����ifferent�set�of�tones�(Go�es�et�al.�1995).�In�other�stu��ies,�acoustic�variance��as�intro��uce���in�a�feature��hich��as�involve���in�the�re��ularit���to�be�teste��.�For�exa��le,�Win�ler�et�al.�(1990)�varie���tone�intensit���se�aratel����ithin���ifferent�ran��es.�Infrequent�tones�fallin���outsi��e�the�ran��e�of�variance�elicite���an�MMN�res�onse��hose�a��litu��e��as�inversel���relate���to�the��i��th�of�the�ran��e�of�variance.�There�are�also�a�nu�ber�of�stu��ies�a����ressin���the�issue�of�re��ularit���extraction�fro��var��in���exe��lars�in��hich�the�se�aration�bet�een�the�varie���an���the�re��ular�feature�is�not�so�clear-cut.�In�so�e�stu��ies,�the�re��ularit���teste����as�quite�co��lex�,�such�as�in�the�stu������resentin���the�natural�footste��soun��s�(Win�ler�et�al.�2003b)�or��hen���ifferent�exe��lars�of�the�sa�e�vo�el��ere�contraste����ith�a���ifferent�vo�el�(Aulan�o�et�al.�1993;�San��ri����e�an���Boothro�����1996).�In�other�stu��ies,�the�varie���feature��as�closel���lin�e���to�the�re��ularit��.�The�latter�t���e�of��ara��i���s�is�best�exe��lifie���b��� stu��ies� testin����hether� the�co��on���irection�or� size�of� the��itch� interval��ithin�a�series�of�tone��airs�can�be�extracte���fro��tone��airs�var��in���in�absolute��itch�(see�Fi��ure�7�for�an�illustration�of�this��ara��i���).�These�stu��ies�foun���that�infrequent���eviations�fro��the�co��on��itch���irection�or�interval�size�elicit�the�MMN�res�onse�in�a��ults�(Paavilainen�et�al.�1999,�2003;�Saarinen�et�al.�1992)�as��ell�as�in�ne�born�babies�(Carral�et�al.�2005;�Stefanics�et�al.�2009).

There� are� also� re��ularities� �hich� are� ��efine��� b��� the� relationshi�� bet�een�soun��s�in�the�sequence.�That�is,�variance�itself�can�be�re��ular.�Paavilainen�an���his�collea��ues�(2001)��resente���tone�sequences�in��hich��ost�soun��s�confor�e���to�a�rule���efine���as�“the�hi��her�the��itch,�the�hi��her�(or�lo�er)�the�intensit���of�the�tone”.�Occasional�tones�violatin���the�rule�elicite���MMN.

Finall��,�rules�base���on�learne���cate��ories�or�al��orith�s�also��rovi��e��eans�to���eneralize�across���ifferent�soun��s.�We�alrea������entione���the�result�sho�in���that��rofessional��usicians�extract�re��ularities�base���on�the�nu�ber�of�soun��s��ithin�tonal���rou�s�var��in���in��itch�(van�Zuijen�et�al.�2005;�see�Fi��ure�5).�Several���e�-onstrations�of�re��ularit���re�resentations�base���on�learne���cate��ories�have�been�obtaine���usin��� s�eech� sti�uli.�For� exa��le,�Philli�s� an���his� collea��ues� (2000)�

Page 101: Unconscious Memory Representations in Perception

90� István�Win�ler

�resente���s��nthesize���consonants�in�the�‘��’-’t’�voice�onset�ti�e�continuu�.�When�the��ajorit���of�exe��lars� fell�on�one�si��e�of� the�bor��er�bet�een� ‘��’�an��� ‘t’� the�ones�fallin���into�the�less�frequent�cate��or���elicite���MMN�(for�a�revie��of�si�ilar�lon��-ter��learnin���effects,�see�Näätänen�et�al.�2001).

Thus�it�a��ears�that�re��ularit���re�resentations�are�for�e���b���extractin����hat�is�co��on�a�on��st���ifferent�sti�uli.�Sti�uli,��hich�have�the�features���efinin���the�re��ularit��,�are�absorbe���b���the�re��ularit���re�resentation.�In�contrast,�sti�uli�that���o�not��ossess�the�features�inclu��e���in�the�re��ularit���re�resentation�are�treat-e���as���eviants.�Thus�the�re��ularit���re�resentation���eneralizes�across���ifferent�in-stances�exe��lif��in���the�co��on�feature��hether�the�var��in���feature�is�relevant�or�irrelevant�for�the���iven�re��ularit��.�The�for�ation�of�re��ularit���re�resentations�can�be�ai��e���b���learne���al��orith�s�or�cate��ories.�These�characteristics�of�the�re��-ularit���re�resentations�inferre���fro��stu��ies�of�au��itor�����eviance���etection�are�in�line��ith��hat�is�ex�ecte���of��erce�tual�object�re�resentations.

Figure 7. Sche�atic�illustration�of��ara��i���s�testin����hether�co��on��itch���irection�or�the�size�of�a��itch�interval�is�extracte���fro��tone��airs�var��in���in�absolute��itch.�Tones�are�sho�n�as�fille���squares.�Tone��airs�are�connecte���for�easier�visualization.�Re��ular��airs�are��ar�e����ith�“S”�(stan��ar��),��hereas�irre��ular�ones��ith�“D”�(��eviant).

Page 102: Unconscious Memory Representations in Perception

� In�search�for�au��itor���object�re�resentations� 91

3.4 Predictive regularity representations and auditory streaming

As��e�state���before,�the�au��itor���strea�in����ara��i����(van�Noor��en�1975;�Fi��-ure�3)�has�been�use���extensivel���to�test�ho��coherent�soun���sequences�are�se��-re��ate��� in� �erce�tion.� The� effects� of� sti�ulation� �ara�eters� on� the� �erce�tual�or��anization�e�er��in���after�listenin���to�short�tone�sequences�is�one�of�the��ost�reliable�fin��in��s�of�au��itor���scene�anal��sis�(Fi��ure�8).�van�Noor��en�foun���three���istinct� areas� in� the� frequenc��-ratio/SOA� �ara�eter� s�ace.� Belo�� the� “Fission�boun��ar��”,��artici�ants�coul����erceive�the���allo�in���rh��th��(inte��rate����erce�t),�but�not�t�o��arallel�strea�s.�Above�the�“Te��oral�coherence�boun��ar��”,��artici-�ants�coul���onl����erceive�the�sequence�in�ter�s�of�t�o�se�arate�strea�s�of�soun���(se��re��ate����erce�t).�Bet�een�the�t�o�boun��aries,��artici�ants�coul���voluntaril���choose�one�or�the�other��erce�t�(a�bi��uous�area).�These�fin��in��s�have�been�ta�-en�to�reflect�the�stren��th�of�the�su��ort�for�the�t�o�alternative�soun���or��aniza-tions��ithin�the�secon����hase�of�au��itor���scene�anal��sis�(Bre���an�1990).�When�the� lar��e� �ajorit��� of� the� �ri�itive� heuristic� �rocesses� su��ort� one� of� the� �os-sible�or��anizations,�the�resultin����erce�t�is�full�����eter�ine���b���sti�ulus-��riven�(botto�-u�)��rocesses.�When,�ho�ever,�su��ort�for�t�o�or��ore�alternatives�is�co��arabl���stron��,�to�-��o�n�influences�can�ti��the�balance�either��a��.

Figure 8. Sche�atic�illustration�of�the�effects�of��resentation�rate�an���frequenc���ratio�in�the�au��itor���strea�in����ara��i����(see�Fi��ure�3).�See�text�for���etails.�(A��a�te����ith��er�ission�fro��Denha��an���Win�ler�2005.)

Page 103: Unconscious Memory Representations in Perception

92� István�Win�ler

Another�a��arentl���reliable�fin��in��� is� that��artici�ants�al�ost�never�re�ort��erceivin���se��re��ation�at�the�be��innin���of�the�tone�sequences.�The��robabilit���of�the� se��re��ate����erce�t� increases���urin��� the�first�5–15�secon��s�of� the� sequence.�This�has�been�ter�e���the�buil��-u��of�au��itor���strea�in���(Bre���an�1990).�Neural�correlates� of� strea�� buil��-u�� have� been� obtaine��� b��� Sn����er� an��� his� collea��ues�(2006).�It�has�been�assu�e���that�initiall��,�the�brain�atte��ts�to�inte��rate�all�inco�-in���soun���into�a�sin��le�strea�.�Strea�s�are�onl���se��re��ate����hen�sufficient�evi-��ence�has�been���athere���in�su��ort�of�the��resence�of�t�o�or��ore�se�arate�coher-ent�sequences�bein����ixe���to��ether�in�the�au��itor���in�ut.�Further�ore,�it��as�also�assu�e���that�once�the�buil��-u���erio���is�over,��erce�tion�of�the�soun���sequence�re�ains�constant,�at�least�in�the�una�bi��uous�areas�of�the��ara�eter�s�ace.

Ho�ever,�fin��in��s�su��ortin���the�buil��-u��of�au��itor���strea�in����a���have�been�influence���b���the��etho��olo��ical�as�ects�of�the�ex�eri�ents.�Man���stu��ies�of�this��heno�enon�assu�e���that��artici�ants�can�onl���ex�erience�the�soun���se-quence�in�ter�s�of�one�of�t�o���istinct�or��anizations:�either�hearin���the���allo�in���rh��th��or,�in��arallel,�a�lo��an���a�hi��h�re�eatin���soun���sequence,�each��ith�its�o�n��resentation�rate.�As�a�consequence,�so�e�of�the�stu��ies���i���not���istin��uish�bet�een�the�lac��of�se��re��ation�an���inte��ration�or�force����artici�ants�to�choose�bet�een�the�t�o��re-��efine����rece�ts,�even��hen�their�actual��erce�tion���i���not��atch�either�one�of�the�.�In�a��ilot�stu�����(re�orte���in�Denha��et�al.�in��ress),��artici�ants�re�orte���a�nu�ber�of���ifferent��erce�ts���urin���lon���(4��inutes)�au-��itor���strea�in���sequences.�So�e�of�the��coul���be�cate��orize���as�inte��rate��,�be-cause�the���inclu��e���a�re�eatin����attern�inclu��in���both�lo��an���hi��h�tones;�others�as�se��re��ate��,�because�the���onl���inclu��e���re�eatin����atterns,�se�aratel����a��e�u��of�either�onl���lo��or�onl���hi��h�tones;��ulti�le�re�eatin����atterns,�at�least�one�of��hich� �as� inte��rate��� an��� another� se��re��ate��� (“both”� �erce�t);� an��� so�eti�es��artici�ants�hear���no�re�eatin����attern�at�all.�Thus�the�assu��tion�of�t�o��utu-all���exclusive���istinct��erce�ts�is��robabl���not�vali��.�Further�ore,�recent�stu��ies�foun���that,��hen��artici�ants�listen�to�lon���sequences�of�the�au��itor���strea�in���sti�ulus��ara��i���,��erce�tion���oes�not�settle�on�a�final�stable�or��anization�after�the�buil��-u���erio���(Denha��et�al.�in��ress;�Denha��&�Win�ler�2006;�Pressnitzer�an���Hu�é�2005,�2006;�Win�ler�et�al.�2005),�not�even�in�the�una�bi��uous�areas�of�the��ara�eter�s�ace�(Denha��et�al.�in��ress).�Thus�au��itor���strea�in���a��ears�to�be�a�bi-�or��ulti-stable��heno�enon,�si�ilarl���to�those�foun���in�visual��erce�tion�(for�revie�s,�see�Bla�e�an���Lo��othetis�2002;�Rees�et�al.�2002),�such�as�binocular�rivalr���(for�a�revie�,�see�Ton���et�al.�2006).

Denha��an���Win�ler�(2005)�for�ulate���a�ne��h���othesis�to�ex�lain�au��itor���strea��se��re��ation�an��,�s�ecificall��,�the�results�obtaine���in�the�au��itor���strea�-in����ara��i���.�We�su����este���that�au��itor���strea�in���can�be���escribe���in�ter�s�of�co��etition�bet�een��re��ictions�base���on�t�o�or��ore�alternative�re��ularit���

Page 104: Unconscious Memory Representations in Perception

� In�search�for�au��itor���object�re�resentations� 93

re�resentations.�A�sequence�of�t�o�alternatin���soun��s�can�be���escribe���b���at�least�t�o��rinci�all�����ifferent�rules.�One�rule��a�es��re��iction�about�the�soun���i��e-��iatel���follo�in���the�current�one:�“Soun���“A”��ill�be�follo�e���b���soun���“B”�an���vice verse”.�Another��ossible�rule��a�es��re��ictions�to�non-a��jacent�soun��s:�“Ev-er���secon���soun���is�“A”,��hile�ever���other�is�“B”.�It�is�eas���to�see�that�in�the�au��i-tor���strea�in����ara��i���,�the�first�rule�corres�on��s�to�the�inte��rate��,��hereas�the�secon���to�the�se��re��ate���soun���or��anization�(see�Fi��ure�3).�In���eneral,�re��ularit���re�resentations�are�base���on�lin�s�bet�een�soun��s��hich�are�use���to��re��ict��hat�soun����ill�be�encountere���in�the�future.�Follo�in���the�chain�of��re��ictions�for�a���iven�re��ularit���creates�a�coherent�(re��ular)�soun���strea�.�The�h���othesis��ut�for�ar���b���Denha��an���Win�ler�(2005)� is� that�au��itor���strea�s�are�base���on�such�re��ularit���re�resentations.�Further�ore,��e�assu�e�that�re��ularities��rovi��-in�����ifferent��re��ictions�(for�in�����ifferent���irecte���lin�s)�are�inco��atible��ith�each�other�an���thus�vie�for���o�inance.�Co��etition�continues�as�lon���as�inco�-�atible�re��ularit���re�resentations�exist�in�the�brain.�The�or��anization�base���on�the��o�entaril���stron��er�set�of�re��ularities�a��ears�in��erce�tion.

The�stren��th�of�a� lin��bet�een�t�o�soun��s�is���eter�ine���b���the�te��oral�an���s�atial�overla��bet�een�the�sti�ulus�after-effects�in�the�brain.�That�is,���iven�the��ell-�no�n� tonoto�ic�or��anization� in��an����arts�of� the�au��itor��� s��ste�,�t�o�tones�close�to�each�other�in�frequenc����ill�activate��ore�neurons�in�co�-�on�than�t�o�tones��ore�se�arate���in�frequenc��.�Further�ore,��hen�a�soun���follo�s�another��ithin�a�short��erio���of�ti�e,�it�encounters�stron��er�lin��erin���effect�of� the��revious� soun��� than� if� it� follo�e��� the�other� soun���after�a� lon��er�ti�e.�These�an���si�ilar�si��le��rinci�les���overn�ho��fast�a���iven�re��ularit���can�be���iscovere���b���the�brain�an���ho��stron���the�lin�s�bet�een�soun��s�(the�re��u-larit���re�resentation)�is.�Man���of�the���rou�in����rinci�les�initiall�����escribe���b���researchers�of� the�Gestalt� school�of��s��cholo����� (see,� e.��.,�Köhler�1947)�can�be�easil���translate���into�the�lan��ua��e�of�the�nervous�s��ste��this��a��.

In�su��ort�of�the�above���escri�tion�of�au��itor���strea�in��,�Denha��et�al.�(in��ress)�foun���in�the�au��itor���strea�in����ara��i����that��ith�ver���fast��resentation�rates� an��� lar��e� frequenc��� se�aration� bet�een� the� t�o� tones,� se��re��ation� often�e�er��es�as�the�first�re�orte����erce�t.�That�is,�no�buil��-u��can�be�observe��.�The�co��etition�bet�een�re��ularit���re�resentations�h���othesis��rovi��es�an�ex�lana-tion�to�this��heno�enon.�When�the�frequenc���se�aration�bet�een�the�t�o�tones�is�lar��e,�there�is�little�overla��bet�een�their�neural�after-effects.�Thus�in�the�au��i-tor���strea�in���tone�sequence,� lin�s�bet�een�a��jacent�soun��s��ill�be��ea��an���ta�e� lon��er� ti�e� to�buil��.�Lin�s�bet�een� i��entical� tones�are�not���e�en��ent�on�the�frequenc���se�aration�bet�een�the�t�o�tones.�Ho�ever,�buil��in���these� lin�s�is�ver���sensitive�to�the�interval�se�aratin���successive�i��entical�tones.�With�a�fast��resentation�rate,�the�ti�e�se�aratin���successive�i��entical�tones�is�short.�Thus�the�

Page 105: Unconscious Memory Representations in Perception

94� István�Win�ler

re��ularities�base���on�lin�s�bet�een�i��entical�tones��ill�be�stron���an���can�be���is-covere���fast.�These�t�o�effects�to��ether�ex�lain,��h���se��re��ation�(the�or��aniza-tion� base��� on� lin�s� bet�een� i��entical� tones)� a��ears� as� the� first� �erce�t� in� an�au��itor���strea�in���sequence��resente���at�a�hi��h�rate�an���havin���lar��e�frequenc���se�aration�bet�een�the�t�o�tones.

Another� �iece� of� evi��ence� su��ortin��� the� co��etition� bet�een� re��ularit���re�resentations�h���othesis�co�es�fro��observin���the�rate�of�s�itchin���bet�een�inte��rate���an���se��re��ate���re�resentations�as�a�function�of�the�sti�ulation��ara�-eters.�Tra��itional���escri�tions�of�au��itor���strea�in���assu�e�that�no�or�ver���s�all�a�ount� of� s�itchin��� shoul��� occur� in� the� una�bi��uous� areas� of� the� �ara�eter�s�ace.�S�itchin����a���be��resent�in�the�a�bi��uous�area.�In�contrast,�the�re��ularit���co��etition�h���othesis�su����ests�that��ost�s�itchin���shoul���occur��hen�both�re��-ularit���re�resentations�are�stron��.�That�is,��ost�s�itchin���shoul���occur��hen�the�frequenc���se�aration�bet�een�the�tones�is�s�all�(stren��thenin���the�lin�s�bet�een�a��jacent�tones;�the�inte��rate����erce�t)�an���the��resentation�rate�is�hi��h�(stren��th-enin���the�lin�s�bet�een�i��entical�tones;�the�se��re��ate����erce�t).�Althou��h��art�of�this��ara�eter�ran��e�falls� into�the�a�bi��uous�area,�(1)�it�also�exten��s�to�una�-bi��uous�areas�(see�Fi��ure�8)�an���(2)�a� lar��e��art�of�the�a�bi��uous�area�has� lo��to��e��iu��frequenc���se�aration�an����e��iu��to�slo���resentation�rates.�It��as�foun���(Denha��et�al.�in��ress)�that�the�hi��hest�rate�of�s�itchin���falls�exactl���to�the�area��re��icte���b���the�re��ularit���co��etition�h���othesis,��hereas�the�a�ount�of�s�itchin���in�a�lar��e��art�of�the�a�bi��uous�area�is��uch�lo�er�(see�Fi��ure�9).

The�inter�retation�of�Denha��et�al.’s�(2005,�in��ress)�results�su����ests�that�the�au��itor���s��ste��for�s�several�re��ularit���re�resentations�for�the�sa�e�soun���se-quence�in��arallel.�The���also�ar��ue�that�the�re��ularit���re�resentations��ust��rovi��e��re��ictions�about�soun��s�occurrin���later�in�the�sequence.�Can��e�then�fin���si��ns�of�that�(a)�soun���sequences�are���escribe���in��arallel�b����ulti�le�re��ularit���re�re-sentations�in�the�brain�an���(b)�re��ularit���re�resentations�are��re��ictive?�As�to�the�first�question,�Horváth�an���collea��ues�(2001)�sho�e���that�re�resentations�for�the�t�o�re��ularities���escribe���above�for�tone�alternation�(lin�s�bet�een�a��jacent�an���lin�s�bet�een�i��entical�tones)�are�active�in��arallel�in�the�hu�an�au��itor���s��ste�.�These�authors� sho�e��� that�violatin���either�one�of� the� t�o�re��ularities� (�ithout�violatin���the�other�one)�elicits�the�MMN�res�onse.�Several�stu��ies�foun���results�co��atible��ith�the��re��ictive�nature�of�the�re��ularit���re�resentation�un��erl��in���the�MMN�res�onse�(Horváth�et�al.�2001;�Paavilainen�et�al.�2007;�Tervanie�i�et�al.�1994a�an���b).�The��ost�convincin���evi��ence��as�obtaine���b���Paavilainen�an���collea��ues�(2007)��ho�sho�e���that�MMN�is�elicite���b���occasionall���violatin���a��re��iction�bet�een�a��jacent�soun��s.�Sequences��ere�constructe���fro��short�lo�,�short�hi��h,� lon���lo�,�an���lon���hi��h�tones.�Short�tones��ere�usuall���follo�e���b���lo�,�lon���ones�b���hi��h�tones.�The�short-lon���attribute��as�varie���ran��o�l����ith�

Page 106: Unconscious Memory Representations in Perception

� In�search�for�au��itor���object�re�resentations� 95

equal��robabilities.�Occasional�hi��h�tones�follo�in���short�tones�an���lo��tones�fol-lo�in���lon���tones�elicite���the�MMN�res�onse.�Because�in�this��ara��i���,�MMN�coul���onl���be�elicite��� if� lo��tones�beca�e�ex�ecte��� follo�in���a�short� tone�an���hi��h�tones�after�a� lon���tone,�Paavilainen�et�al.’s�results�(since�then�re�licate���b���Ben��ixen�et�al.�2008)�stron��l���ar��ue�for��re��ictions�bein�����enerate���on�the�basis�of� the� re��ularit��� re�resentations� involve��� in� MMN� ��eneration.� Recentl��,� ��irect�ERP�evi��ence�of�the��re��ictive�nature�of�au��itor���re��ularit���re�resentations�has�been�obtaine���b���Ben��ixen�an���collea��ues�(2009).

In�this�section,��e�su����este���a���escri�tion�of�au��itor���strea�in���in�ter�s�of�co��etition�bet�een�re��ularit���re�resentations��ro��ucin���inco��atible��re��ic-tions.�Evi��ence�favorin���this�h���othesis�over�the�tra��itional���escri�tion�of�au��i-tor���strea�in���has�been�revie�e��.�Finall��,��e�sho�e���that�the�co��etin���re��u-larit���re�resentations�are��aintaine���in��arallel�in�the�au��itor���s��ste��an���that��re��ictions�base���on�these�re��ularit���re�resentations�are�utilize���b���the�au��itor�����eviance���etection��rocess�in��exe���b���the�MMN�res�onse.

Figure 9. Grou�-�ean���istribution�of��erce�tual�s�itchin���as�a�function�of�frequenc���se�aration�(��f�in�se�itones)�an���the�sti�ulus�onset�as��nchron���(SOA�in��illisecon��s).�The���re���scale�in��icates�the��ean�nu�ber�of�s�itches�across��artici�ants�accu�ulate���throu��hout�the�tone�trains,�se�aratel���for�each�co�bination�of��ara�eters.�Note�that�the�surface�is�inter�olate���bet�een�the���iscrete�ex�eri�ental���ata��oints�in��icate���b���the�s�all�e��t���circles.�The�te��oral�coherence�an���the�fission�boun��ar���are��ar�e���b�����ashe���lines.

Page 107: Unconscious Memory Representations in Perception

9�� István�Win�ler

3.5 A conceptual model of auditory object formation

In� the� afore� ��oin��,� �e� ar��ue��� that� �re��ictive� re��ularit��� re�resentations� are�for�e���b���the�hu�an�au��itor���s��ste��an���u���ate����henever�their��re��iction�is��is�atche���b���the�soun���in�ut.�U���atin���initiate���b���such��is�atches�is�re-flecte���b���the�MMN�event-relate����otential�(Win�ler�2007).�We�have�also�sho�n�that�the��ro�erties�of�these�re��ularit���re�resentations��atch�those�ex�ecte���fro��au��itor���objects�an���that�au��itor���strea��se��re��ation,�the��echanis��b����hich�au��itor���objects�are�se�arate���fro��each�other��ithin�the�soun���in�ut,�can�be�conce�tualize��� as� co��etition� bet�een� �re��ictive� re��ularit��� re�resentations.�Finall��,�the�notion�of�re��ar��in����re��ictive�re��ularit���re�resentations�as�the�basic�buil��in���bric�s�of��erce�tion�is�full���co��atible��ith�Gre��or��’s�(1980)�theor���of��erce�tion,��hich�su����ests�that��erce�tion�is�a�constructive��rocess�a�in�to�the���eneration�of��re��ictive�scientific�h���otheses.�That�is,�the�objects�a��earin���in��erce�tion�contain���escri�tive�infor�ation�not�actuall����resent�in�the�sensor���in�ut,�such�as�unseen��arts�of�a�visuall����resente���object.�In�the�sa�e��anner,��re��ictive� au��itor��� re��ularit��� re�resentations� �rovi��e� assu��tions� re��ar��in���the�continuation�of�the�soun���object�(Win�ler�et�al.�2009).

Fi��ure�10� illustrates� the�outline�of�a�s��ste��co��atible��ith� the�above�no-tion�of�au��itor���objects.�The�u��er�half�of�the�fi��ure���e�icts�the�t�o-sta��e��o��el�of�au��itor���scene�anal��sis.�Partl���anal��ze���au��itor��� infor�ation�a��ears�at� the�in�ut� of� au��itor��� scene� anal��sis� (u��er� left� arro�� in� Fi��ure� 10).� The� heuristic���rou�in���al��orith�s�assu�e���b���Bre���an’s�(1990)�theor���are�then�activate��.�The�u��er�left�box�of�the�fi��ure�inclu��es�the���istinction�bet�een�i��e��iate�(s�ectral)�an���sequential�(te��oral)���rou�in����rocesses.�These��rocesses��rovi��e�can��i��ate���rou�in��s� for� the� secon��� sta��e� of� au��itor��� scene� anal��sis.� In� the� secon��� sta��e�co��atible�can��i��ate���rou�in��s�for��coalitions.�Su��ort�for�these���rou�s�is�then�co��are���to�establish�the���o�inant�(�ost�li�el��)�solution�(u��er�ri��ht�box).�The�out�ut�of�the�secon���sta��e�of�au��itor���scene�anal��sis�contains�soun���infor�ation�or��anize���in�ter�s�of�au��itor���objects�(u��er�ri��ht�arro�).�The�lo�er�half�of�the�fi��ure���e�icts�a�s��ste��that��aintains�the�te��oral/sequential�re��ularities���etect-e����ithin�the��rece��in���soun���in�ut.�Such�re��ularities�(lo�er�left�box)�are�use���to��rovi��e��re��ictions�for�the�sequential���rou�in���al��orith�s.�These��re��ictions�carr����ith�the��a��ei��ht��easure��hich�infor�s�the�s��ste��about�the�observe���reliabilit���(confi��ence)�of�the���iven�re��ularit��.�This�infor�ation�is�then�use���in�resolvin���the�co��etition�bet�een�alternative�soun���or��anizations.�That�is,� the�su��ort��rovi��e���b�����ifferent�al��orith�s�for�the�alternative�or��anization(s)�the���are�co��atible��ith�is�not�fixe��.�An�al��orith���hose��re��iction�has�been�consis-tentl���foun���to�be�vali����ithin�the�actual�au��itor���scene,��ill��rovi��e��ore�su�-�ort�for�the�soun���or��anization�it�is�co��atible��ith�than�one��hose��re��iction�is�

Page 108: Unconscious Memory Representations in Perception

� In�search�for�au��itor���object�re�resentations� 97

often�off�the��ar�.�It�shoul���be�note���that�the��ei��ht�of��re��ictions�is�influence���b���several�factors�outsi��e�the��rocess�reflecte���b���MMN.�For�exa��le,�initial�con-fi��ence�levels��robabl�����e�en���on�lon��-ter��ex�erience�an����a���even�be�s�ecific�to�the�actual�context�(e.��.,�certain�al��orith�s��a���have�been��roven��ore�reli-able�in�often�recurrin���situations,�etc.).�Further�ore,�to�-��o�n�effects��a���also�bias�the�choice�bet�een�alternative�soun���or��anizations�(e.��.,�active�search�for�a�soun����attern).�Finall��,�lon��er-ter��a��a�tation�to�re�eatin���soun��s��a���result�in�s�itches�bet�een�alternative�soun���or��anizations,� si�ilarl��� to� the�assu��tions��a��e�in��o��elin���bi-stable��erce�tual��heno�ena�in�the�visual��o��alit���(e.��.,�Klin��et�al.�2008;�Noest�et�al.�2007).

Ho�ever,�shoul���there�be�a�chan��e�of�the�au��itor���scene,��reviousl���unreli-able�al��orith�s��a���beco�e��ore�reliable��hile�the��reviousl���reliable�ones��a���beco�e�less�reliable.�The��ro�ose���s��ste��thus�sho�s�continuous�a��a�tation�to�the�chan��es�in�the�acoustic�environ�ent.�A��a�tation�of�the��ei��hts�is�achieve���b���fee��bac��fro��the�sta��e,��here�soun���or��anization�is�finalize��.�We���istin��uish�three��ossible�outco�es.�When�the��re��iction�fro��a�store���re��ularit���is��atche���b��� the� in�ut,� its� confi��ence� �ei��ht� is� �aintaine��.� When� it� is� �is�atche��,� the�confi��ence��ei��ht�is�re��uce��.�Finall��,�there�are�often�ele�ents��ithin�the�au��itor���in�ut��hich�have�not�been��re��icte���b���an���of�the�existin���re��ularit���re�resenta-tions.�This�is�t���icall���the�case,��hen�a�ne��soun���source�beco�es�active�in�the�

Figure 10. Sche�atic�illustration�of�object�for�ation�an����aintenance�in�the�au��itor���s��ste�.�See�text�for���etails.�(A��a�te����ith��er�ission�fro��Win�ler�2007.)

Page 109: Unconscious Memory Representations in Perception

9�� István�Win�ler

environ�ent.�Thus�that��art�of�the�inco�in���soun��,�for��hich�no��re��iction�has�been� �rovi��e��� b��� an��� of� the� existin��� re��ularit��� re�resentations,� is� anal��ze��� in�or��er�to���iscover�ne��re��ularities.�This���escri�tion�is�full���co��atible��ith�the��rinci�le�of�the�ol��-�lus-ne��strate�����(Bre���an�1990).�The�ol��-�lus-ne���rin-ci�le�su����ests�that�the�au��itor���s��ste��first�extracts�those��arts�of�the�soun���in�ut��hich�a��ear�to�continue��reviousl���establishe���au��itor���strea�s�then�treats�the�re�ainin���soun��s�as�the�onset�of�a�ne��strea�.

In�the�above�su����este����o��el,�the�re��ularit���re�resentations�extracte���fro��the��rece��in���soun���in�ut�can�be�re��ar��e���as�buil��in���bric�s�of�au��itor����er-ce�tual�objects.�T���icall��,� several� such� re��ularit��� re�resentations�are� involve���in�for�in���an����erceive���soun���object.�The�set�of�store����re��ictive�re��ularities�for��a�lo�-level��o��el�of�the�au��itor���environ�ent,��hich�re�resents��hat��in���of�sensor���re��ularities�have�been���etecte����ithin�the�current�au��itor���scene.�The�MMN�ERP�res�onse�is��ar�e���on�the�fi��ure�(botto��center),�sho�in���ho��it�relates�to�the�overall�function�of�the�s��ste��(i.e.,�it�is�activate��,��hen�the�confi-��ence��ei��ht�of�a�re��ularit���re�resentation�is���ecrease��;�cf.�Win�ler�an���Czi��ler�1998).� If�MMN�is���erive��� fro��the�outco�e�of�au��itor���scene�anal��sis��hich��laces�each�soun���into�the�context�of�the��hole�scene,�one�shoul���ex�ect�MMN�to�sho��effects�not�onl���of�the�i��e��iatel����rece��in���soun���sequence,�but�also�of�the�context�of�the��hole�scene.�Such�au��itor���context�effects�have�been�ob-serve���for�MMN�(for�a�revie�,�see�Suss�an�2007).�For�exa��le,� it��as�sho�n�that�a���iven�soun���is�either�re��ar��e���as�re��ular�or�irre��ular,�but�not�both�at�the�sa�e�ti�e�an���that�the�attribute�of�“bein���re��ular”�is�establishe���at�the�level�of�the��hole�sequence�(Suss�an�et�al.�2003).5�Further�ore,���eviations�are�evalu-ate��� in�accor��ance��ith� their� infor�ation�content��ithin� the���lobal� sti�ulus�sequence�(Suss�an�an���Win�ler�2001).�Suss�an�an���Win�ler�(2001)�foun���that��hen���eviants�onl���occur�in��airs���elivere����ithin�ca.�170��s��ithin�a�soun���sequence�(��ouble���eviants),�onl���the�first�of�the�t�o�successive���eviants�elicits�MMN.�Ho�ever,�intro��ucin���sin��le���eviants�into�the�sti�ulus�sequence�results�in���ouble���eviants�elicitin��� t�o�successive�MMNs;�subsequent�re�oval�of� the�sin��le���eviants�����na�icall���reverts� the�s��ste��into�the�state,� in��hich���ouble���eviants�a��ain�elicit�onl���one�MMN.�Finall��,�as��as�alrea�������iscusse��,�soun��s�are�onl���evaluate����ithin�the�au��itor���strea��the���belon���to.�That�is,�a�soun���onl���elicits�MMN,��hen�it�violates�so�e�re��ularit����ithin�its�o�n�strea�,�but�not��hen�violatin���so�e�re��ularit���of�another�strea��(De�Sanctis�et�al.�2008;�Ritter�et�al.�2000,�2006).

Flexibilit��� of� the� �ro�ose��� s��ste�� requires� fast� establish�ent� of� re��ularit���re�resentations,��hereas�its�stabilit���requires�that�establishe���re��ularit���re�resen-tations�shoul���be�retaine���for�relativel��� lon����erio��s�of� ti�e,�even�if� their��re-��ictions��rove�to�be�false�for�so�e�ti�e.�In�accor��ance��ith�these�ex�ectations,�

Page 110: Unconscious Memory Representations in Perception

� In�search�for�au��itor���object�re�resentations� 99

MMN�stu��ies�sho�e���that�re��ularit���re�resentations�are�establishe���b���ver���fe��(2–3)�re��ular�soun��s�(Ben��ixen�et�al.�2007;�Co�an�et�al.�1993;�Horváth�et�al.�2001;�Schrö��er� 1997;� Win�ler� et� al.� 1996b).� In� contrast,� establishe��� re��ularities� have�been� sho�n� to� survive� several� successive� ��eviant� events� (Win�ler� et� al.� 1996a�an���b)�an���can�be�reactivate���b���a�sin��le�re�in��er�even�after�quite�lon���brea�s�(e.��.,�30�s;�Win�ler�et�al.�2002;�for�a�revie�,�see�Win�ler�an���Co�an�2005).

Are� �e� a�are� of� the� re��ularit��� re�resentations� store��� for� a� ��iven� au��itor���scene?�Not�necessaril���so.�MMN�elicitation��as�observe���to�violatin���rules��ar-tici�ants��ere�not�a�are�of�even��hen�the���atten��e���the�soun���sequence�an����ere�as�e���to���etect�the���eviants�(Paavilainen�et�al.�2007;�Paavilainen�et�al.�2003;�van�Zuijen�et�al.�2006).�Ho�ever,�trainin���to��erceive�a�re��ularit���enhances�the�MMN�res�onse� (e.��.� Näätänen� et� al.� 1993)� an��� the� MMN� res�onse� reflects� various�lon��-ter��learnin���effects�(for�a�revie�,�see�Näätänen�et�al.�2001).�Further�ore,�voluntar���selection�of�a���iven�soun���or��anization�(�hen�this�is��er�itte���b���the�a�bi��uous�au��itor����a�e-u��of�the�sequence)�can���overn�the�elicitation�of�MMN�(Suss�an�et�al.�2002;�Win�ler�et�al.�2006).�Thus�the�for�ation�of�au��itor����erce�-tual�objects�is�at�the�crossroa���of�conscious�an���unconscious��rocesses.�Althou��h�it�is�lar��el���sti�ulus-��riven�an���often�i��enetrable�for�conscious��rocesses,�it�is�in��irectl���affecte���b���ex�licit�learnin���an��,�un��er�s�ecific�circu�stances,�can�be���irectl���affecte���b���intentions.

In�su��ar��,��e��ro�ose���a�fra�e�or��for�conce�tualizin���the�for�ation�of��erce�tual�au��itor���objects.�We�su����este���that�au��itor���objects�are�built�fro���re-��ictive�re��ularit���re�resentations��hich�are�extracte���fro��the�on��oin���au��itor���in�ut�an���continuousl���u���ate����ith�res�ect�to�their��re��ictive�success.�U���atin���is�at�least��artl�����one�b���a�s��ste����etectin���violations�of�au��itor���re��ularities.�The�u���atin����rocess,�an���throu��h� it� the�au��itor���re��ularit���re�resentations�store���in�the�brain�can�be�stu��ie����ith�the��is�atch�ne��ativit���(MMN)�event-relate���brain��otential.�Pre��ictive�object�re�resentations��rovi��e�obvious�a��vanta��es�for�hu�ans,�because�the���allo��actions�to�a��a�t�to�the�future,�thus�ali��nin���their�ef-fect��ith�the�evolvin���state�of�the�environ�ent.

Acknowledgements

This� research� �as� su��orte��� b��� the� Euro�ean� Co��ission’s� 7th� Fra�e�or��Pro��ra��e�for� ‘‘Infor�ation�an���Co��unication�Technolo��ies’’� (�roject� title:�SCANDLE,�acoustic�SCene�ANal��sis�for�Detectin���Livin���Entities,�contract�no.:�231168).�I�a����rateful�to�Dr.�Alexan��ra�Ben��ixen�for�the�hel�ful�co��ents�on�an�earlier�version�of�this��a�er.

Page 111: Unconscious Memory Representations in Perception

100� István�Win�ler

Notes

1. B���unconscious��e��ean�that�the��rocess�is�not�re�ortable,�but�it�has�i��e��iate�or�re�ote�behavioral�an��/or��s��cho�h��siolo��ical�consequences.2. Althou��h�this��ro�ert���of��erce�tion�has�been�re��ar��e���as�an�i��ortant�ar��u�ent�for�the��o��ularit���of��erce�tual��rocesses�(Fo��or�1983),�the�current�treat�ent���oes�not�assu�e�the��o��ularit����oint�of�vie�.3. In�an�au��itor���o����ball�sequence,�one�soun���is��resente����ith�hi��h��robabilit���(t���icall���>�75%),��hereas�one�or��ore�other�soun��s�are��resente����ith�lo���robabilities.�The�or��er�of���ifferent�soun��s�is�usuall���ran��o�ize��.4. Ho�ever,��lease�note�that�the�t�o�functions�are�not��utuall���exclusive.5. Note,�ho�ever,� that�conflicts��a���arise� fro��o��osite��re��ictions�base���on�the�au��itor���an���e.��.,�the�visual�context�(Wi���ann�et�al.�2004),�althou��h�such�conflicts��a���not�affect�the�MMN�(Ritter�et�al.�1999).

References

Alain,�C.,�Arnott,�S.�R.�an���Picton,�T.�W.�(2001).�Botto��u��an���to����o�n�influences�on�au��i-tor���scene�anal��sis:�Evi��ence�fro��event�relate���brain��otentials.�Journal of Experimental Psychology: Human Perception and Performance,�27,�1072–1089.

Alain,�C.�an���McDonal��,�K.�L.�(2007).�A��e-relate�����ifferences�in�neuro�a��netic�brain�activit���un��erl��in���concurrent�soun����erce�tion.�Journal of Neuroscience,�27,�1308–1314.

Alain,�C.,�Schuler,�B.�M.�an���McDonal��,�K.�L.�(2002).�Neural�activit���associate����ith���istin-��uishin��� concurrent� au��itor��� objects.� Journal of the Acoustical Society of America,� 111,�990–995.

Aulan�o,�R.,�Hari,�R.,�Lounas�aa,�O.�V.,�Näätänen,�R.�an���Sa�s,�M.�(1993).�Phonetic�invari-ance�in�the�hu�an�au��itor���cortex.�NeuroReport,�4,�1356–1358.

Belin,�P.�an���Zatorre,�R.�J.�(2000).�‘What’,�‘�here’�an���‘ho�’�in�au��itor���cortex.�Nature Neurosci-ence,�3,�965–966.

Ben��ixen,�A.,�Prinz,�W.,�Horváth,�J.,�Trujillo-Barreto,�N.�J.�an���Schrö��er,�E.�(2008).�Ra�i���ex-traction�of�au��itor���feature�contin��encies.�NeuroImage,�41,�1111–1119.

Ben��ixen,�A.,�Roeber,�U.�an���Schrö��er,�E.�(2007).�Re��ularit���extraction�an���a��lication�in�����-na�ic�au��itor���sti�ulus�sequences.�Journal of Cognitive Neuroscience,�19,�1664–1677.

Ben��ixen,�A.,�Schrö��er,�E.�an���Win�ler,�I.�(2009).�I�hear���that�co�in��:�ERP�evi��ence�for�sti�u-lus-��riven��re��iction�in�the�au��itor���s��ste�.�Journal of Neuroscience,�29,�8447–8451.

Bla�e,�R.�an���Lo��othetis,�N.�K.�2002.�Visual�co��etition.�Nature Reviews Neuroscience,�3,�13–21.

Bre���an,� A.� S.� (1990).� Auditory scene analysis: The perceptual organization of sound.� Ca�-bri����e,�MA.:�MIT�Press.

Bre���an,�A.�S.,�Aha��,�P.�A.,�Cru�,�P.�A.�C.�an���O’Reill��,�J.�(2000).�Effects�of�ti�e�intervals�an���tone���urations�on�au��itor���strea��se��re��ation.�Perception & Psychophysics,�62,�626–�636.

Bruns�i�,�E.�(1955).�In���efense�of��robabilistic�functionalis�:�A�re�l��.�Psychological Review,�62,�236–242.

Page 112: Unconscious Memory Representations in Perception

� In�search�for�au��itor���object�re�resentations� 101

Carl��on,�R.�P.,�Cusac�,�R.,�Foxton,�J.�M.,�an���Robertson,�I.�H.�(2001).�Effects�of�attention�an���unilateral�ne��lect�on�au��itor���strea��se��re��ation.�Journal of Experimental Psychology: Hu-man Perception and Performance,�27,�115–127.

Carral,�V.,�Huotilainen,�M.,�Ruusuvirta,�T.,�Fell�an,�V.,�Näätänen,�R.�an���Escera,�C.� (2005).�A� �in��� of� au��itor��� ‘�ri�itive� intelli��ence’� alrea����� �resent� at� birth.� European Journal of Neuroscience,�21,�3201–3204.

Co�an,�N.�(1984).�On�short�an���lon���au��itor���stores.�Psychological Bulletin,�96,�341–370.Co�an,�N.,�Win�ler,�I.,�Te��er,�W.�an���Näätänen,�R.�(1993).�Short��an���lon���ter���rerequisites�

of�the��is�atch�ne��ativit���in�the�au��itor���event�relate����otential�(ERP).�Journal of Experi-mental Psychology: Learning, Memory, and Cognition,�19,�909–921.

Cusac�,� R.,� Dee�s,� J.,� Ai��an,� G.,� an��� Carl��on,� R.� P.� (2004).� Effects� of� location,� frequenc���re��ion,�an���ti�e�course�of�selective�attention�on�au��itor���scene�anal��sis.�Journal of Experi-mental Psychology: Human Perception and Performance,�30,�643–656.

De�Sanctis,�P.,�Ritter,�W.,�Molhol�,�S.,�Kell��,�S.�P.�an���Foxe,�J.�J.�(2008).�Au��itor���scene�anal��sis:�The�interaction�of�sti�ulation�rate�an���frequenc���se�aration�on��re-attentive���rou�in��.�European Journal of Neuroscience,�27,�1271–126.

Denha�,� S.� L.,� G��i�esi,� K.,� � Stefanics,� G.� an��� Win�ler,� I.� (in� �ress).� Stabilit��� of� �erce�tual�or��anisation� in� au��itor��� strea�in��.� In� E.� A.� Lo�ez-Pove��a,� A.� R.� Pal�er� an��� R.� Me��-��is�(E��s.),�Advances in auditory research: Physiology, psychophysics and models.�Ne��Yor�:�S�rin��er.

Denha�,�S.�L.�an���Win�ler,�I.�(2006).�The�role�of��re��ictive��o��els�in�the�for�ation�of�au��itor���strea�s.�Journal of Neurophysiology, Paris,�100,�154–170.

Deutsch,�D.�(1982).�Grou�in����echanis�s� in��usic.� In�D.�Deutsch�(E��.),�The psychology of music.�Ne��Yor�:�Aca��e�ic�Press.

Do�lin��,�W.�J.�(1973).�Rh��th�ic���rou�s�an���subjective�chun�s�in��e�or���for��elo��ies.�Per-ception & Psychophysics,�14,�37�40.

Duncan,�J.�an���Hu��hre��s,�G.�(1989).�Visual�search�an���sti�ulus�si�ilarit��.�Psychological Re-view,�96,�433–458.

Escera,�C.�an���Corral,�M.�J.�(2007).�Role�of��is�atch�ne��ativit���an���novelt��-�3�in�involuntar���au��itor���attention.�Journal of Psychophysiology,�21,�251–264.

Fabiani,�M.,�Gratton,�G.� an���Coles,�M.�G.�H.� (2000).�Event� relate���brain��otentials.� In� J.�T.�Cacio��o,�L.�G.�Tassinar���an���G.�G.�Bernston�(E��s.),�Handbook of psychophysiology�(2n���e��ition,���.�53–84).�Ca�bri����e,�MA:�Ca�bri����e�Universit���Press.

Fo��or,�J.�(1983).�The Modularity of Mind.�Ca�bri����e,�MA:�MIT�Press.Fo�ler,�C.�A.�an���Rosenblu�,�L.�D.�(1990).�Du�lex��erce�tion:�a�co��arison�of��onos��llables�

an���sla��in�����oors.�Journal of Experimental Psychology: Human Perception and Perfor-mance,�16,�742–754.

Go�es,�H.,�Bernstein,�R.,�Ritter,�W.,�Vau��han,�H.�G.,�Jr.�an���Miller,�J.�(1997).�Stora��e�of�feature�conjunctions�in�transient�au��itor����e�or��.�Psychophysiology,�34,�712–716.

Go�es,�H.,�Ritter,�W.�an���Vau��han,�H.�G.,�Jr.�(1995).�The�nature�of��reattentive�stora��e�in�the�au��itor���s��ste�.�Journal of Cognitive Neuroscience,�7,�81–94.

Gre��or��,�R.�L.�(1980).�Perce�tions�as�h���otheses.�Philosophical Transactions of the Royal Society of London – Series B: Biological Sciences,�290,�181–197.

Griffiths,�T.�D.��an���Warren,�J.�D.�(2004).�What�is�an�au��itor���object?�Nature Reviews Neurosci-ence,�5,�887–892.

Hall,�M.�D.,�Pastore,�R.�E.,�Ac�er,�B.�E.�an���Huan��,�W.�(2000).�Evi��ence�for�au��itor���feature�inte��ration��ith�s�atiall�����istribute���ite�s.�Perception & Psychophysics,�62,�1243–1257.

Page 113: Unconscious Memory Representations in Perception

102� István�Win�ler

Hel�holtz,�H.�L.�(1867/1910).�Han��buch���er��h��siolo��ischen�O�ti�.�Lei�zi��:�L.�Voss.�Re�rin-te��,��ith�extensive�co��entar��,�in�A.�Gullstran��,�J.�von�Kries�an���W.�Na��el�(E��s.),�Hand-buch der physiologischen Optik�(3r���e��ition).�Ha�bur���an���Lei�zi��:�L.�Voss.

Horváth,�J.,�Czi��ler,�I.,�Suss�an,�E.�an���Win�ler,�I.�(2001).�Si�ultaneousl���active��re-attentive�re�resentations�of� local�an�����lobal�rules�for�soun���sequences.�Cognitive Brain Research,�12,�131–144.

Horváth,�J.,�Czi��ler,�I.,�Win�ler,�I.�an���Te��er-Sälejärvi,�W.�A.�(2007).�The�te��oral��in��o��of�inte��ration�in�el��erl���an�����oun���a��ults.�Neurobiology of Aging,�28,�964–975.

Jacobsen,�T.,�Schrö��er,�E.,�Win�ler,�I.�an���Horváth,�J.�(2005).�Fa�iliarit���affects�the��rocessin���of�tas�-irrelevant�i��nore���soun��s.�Journal of Cognitive Neuroscience,�17,�1704–1713.

Ja�es,�W.�(1890).�The principles of psychology.�Ne��Yor�:�Holt.Klin�,�P.�C.,�van�Ee,�R.,�Nijs,�M.�M.,�Brou�er,�G.�J.,�Noest,�A.�J.�an���van�Wezel,�R.�J.�A.�(2008).�

Earl���interactions�bet�een�neuronal�a��a�tation�an���voluntar���control���eter�ine��erce�-tual�choices�in�bistable�vision.�Journal of Vision,�8,�16.1–18.

Köhler,�W.�(1947).�Gestalt Psychology.�Ne��Yor�:�Liveri��ht.Kubov��,�M.�an���Van�Val�enbur��,�D.�(2001).�Au��itor���an���visual�objects.�Cognition,�80,�97–

126.Kujala,�T.,�Tervanie�i,�M.�an���Schrö��er,�E.�(2007).�The��is�atch�ne��ativit���in�co��nitive�an���

clinical�neuroscience:�Theoretical�an����etho��olo��ical�consi��erations.�Biological Psycho-logy,�74,�1–19.

Müller,�D.,�Wi���ann,�A.�an���Schrö��er,�E.�(2005).�Au��itor���strea�in���affects�the��rocessin���of�successive���eviant�an���stan��ar���soun��s.�Psychophysiology,�42,�668–676.

Näätänen,�R.�(1990).�The�role�of�attention�in�au��itor���infor�ation��rocessin���as�reveale���b���event�relate����otentials�an���other�brain��easures�of�co��nitive� function.�Behavioral and Brain Sciences,�13,�201–288.

Näätänen,�R.,�Gaillar��,�A.�W.�K.�an���Mänt��salo,�S.�(1978).�Earl���selective�attention�effect�on�evo�e����otential�reinter�rete��.�Acta Psychologica,�42,�313–329.

Näätänen,�R.,�Schrö��er,�E.,�Kara�as,�S.,�Tervanie�i,�M.�an���Paavilainen,�P.�(1993).�Develo��ent�of�a��e�or���trace�for�a�co��lex�soun���in�the�hu�an�brain.�NeuroReport,�4,�503–506.

Näätänen,�R.,�Tervanie�i,�M.,�Suss�an,�E.,�Paavilainen,�P.�an���Win�ler,�I.�(2001).� ‘Pri�itive�intelli��ence’�in�the�au��itor���cortex.�Trends in Neurosciences,�24,�283–288.

Näätänen,�R.�an���Win�ler,�I.�(1999).�The�conce�t�of�au��itor���sti�ulus�re�resentation�in�co��ni-tive�neuroscience.�Psychological Bulletin,�125,�826–859.

Na��er,�W.,�Te��er-Sälejärvi,�W.,�Kunze,�S.�an���Münte,�T.�F.� (2003).�Preattentive�evaluation�of��ulti�le��erce�tual�strea�s�in�hu�an�au��ition.�NeuroReport,�14,�871–874.

Noest,�A.�J.,�van�Ee,�R.,�Nijs,�M.�M.�an���van�Wezel,�R.�J.�A.�(2007).�Perce�t-choice�sequences���riven�b���interru�te���a�bi��uous�sti�uli:�A�lo�-level�neural��o��el.�Journal of Vision,�7,�10.1–14.

Nor��b��,�H.,�Roth,�W.�T.�an���Pfefferbau�,�A.�(1988).�Event�relate����otentials�to�brea�s�in�se-quences�of�alternatin����itches�or�intersti�ulus�intervals.�Psychophysiology,�25,�262–268.

Paavilainen,�P.,�Arajärvi,�P.�an���Ta�e��ata,�R.�(2007).�Preattentive���etection�of�nonsalient�contin-��encies�bet�een�au��itor���features.�NeuroReport,�18,�159–163.

Paavilainen,� P.,� De��er�an,� A.,� Ta�e��ata,� R.� an��� Win�ler,� I.� (2003).� S�ectral� an��� te��oral�sti�ulus�characteristics�in�the��rocessin���of�abstract�au��itor���features.�NeuroReport,�14,�715–718.

Page 114: Unconscious Memory Representations in Perception

� In�search�for�au��itor���object�re�resentations� 103

Paavilainen,�P.,� Jara�illo,�M.,�Näätänen,�R.�an���Win�ler,� I.� (1999).�Neuronal��o�ulations� in�the�hu�an�brain�extractin���invariant�relationshi�s�fro��acoustic�variance.�Neuroscience Letters,�265,�179–182.

Paavilainen,�P.,�Si�ola,�J.,�Jara�illo,�M.,�Näätänen,�R.�an���Win�ler,�I.�(2001).�Preattentive�ex-traction�of�abstract�feature�conjunctions�fro��au��itor���sti�ulation�as�reflecte���b���the��is-�atch�ne��ativit���(MMN).�Psychophysiology,�38,�359–365.

Philli�s,� C.,� Pellath��,� T.,� Marantz,� A.,� Yellin,� E.,� Wexler,� K.,� Poe��el,� D.,� McGinnis,� M.� an���Roberts,�T.�(2000).�Au��itor���cortex�accesses��honolo��ical�cate��ories:�An�MEG��is�atch�stu����.�Journal of Cognitive Neuroscience,�12,�1038–1055.

Pressnitzer,�D.�an���Hu�é,�J.�M.�(2005).�Is�au��itor���strea�in���a�bistable��erce�t?�Forum Acusti-cum, Budapest,���.�1557–1561.

Pressnitzer,�D.�an���Hu�é,�J.�M.�(2006).�Te��oral�����na�ics�of�au��itor���an���visual�bistabilit���reveal�co��on��rinci�les�of��erce�tual�or��anization.�Current Biology,�16,�1351–1357.

Pulver�üller,�F.�an���Sht��rov,�Y.�(2006).�Lan��ua��e�outsi��e�the�focus�of�attention:�the��is�atch�ne��ativit���as�a�tool�for�stu����in���hi��her�co��nitive��rocesses.�Progress in Neurobiology,�79,�49–71.

Ran��,�T.�C.�(1974).�Dichotic�release�fro���as�in���for�s�eech.�Journal of the Acoustical Society of America,�55,�678–680.

Rauschec�er,�J.�P.�an���Tian,�B.�(2000).�Mechanis�s�an���strea�s�for��rocessin���of�“�hat”�an���“�here”� in� au��itor��� cortex.� Proceedings of the National Academy of Sciences USA,� 97,�11800–11806.

Rees,�G.,�Krei�an,�G.,�Koch,�C.�(2002).�Neural�correlates�of�consciousness�in�hu�ans.�Nature Reviews Neuroscience,�3,�261–270.

Re��an,�D.�(1989).�Human Brain Electrophysiology: Evoked Potentials and Evoked Magnetic Fields in Science and Medicine.�Ne��Yor�:�Elsevier.

Ritter,�W.,�DeSanctis,�P.,�Molhol�,�S.,�Javitt,�D.�C.��an���Foxe,�J.�J.�(2006).�Preattentivel�����rou�e���tones���o�not�elicit�MMN��ith�res�ect�to�each�other.�Psychophysiology,�43,�423–430.

Ritter,�W.,�Suss�an,�E.,�Deacon,�D.,�Co�an,�N.�an���Vau��han,�H.�G.,�Jr.�(1999).�T�o�co��nitive�s��ste�s�si�ultaneousl����re�are���for�o��osite�events.�Psychophysiology,�36,�835–838.

Ritter,�W.,�Suss�an,�E.�an���Molhol�,�S.�(2000).�Evi��ence�that�the��is�atch�ne��ativit���s��ste���or�s�on�the�basis�of�objects.�NeuroReport,�11,�61–63.

Rubin,�E.�(1915).�Synoplevede Figurer.�Co�enha��en:�G��l��en��als�e.Saarinen,�J.,�Paavilainen,�P.,�Schrö��er,�E.,�Tervanie�i,�M.�an���Näätänen,�R.�(1992).�Re�resenta-

tion�of�abstract�attributes�of�au��itor���sti�uli�in�the�hu�an�brain.�NeuroReport,�3,�1149–1151.

San��ri����e,�S.�an���Boothro����,�A.�(1996).�Usin���naturall����ro��uce���s�eech�to�elicit��is�atch�ne��ativit��.�Journal of the American Academy of Audiology,�7,�105–112.

Scholl,�B.�J.�(2001).�Objects�an���attention:�The�state�of�the�art.�Cognition,�80,�1–46.Schrö��er,�E.�(1997).�On�the���etection�of�au��itor�����eviants:�A��re-attentive�activation��o��el.�

Psychophysiology,�34,�245–257.S�ith,�D.�R.,�Patterson,�R.�D.,�Turner,�R.,�Ka�ahara,�H.�an���Irino,�T.�(2005).�The��rocessin���

an����erce�tion�of�size�infor�ation�in�s�eech�soun��s.�Journal of the Acoustical Society of America,�117,�305–318.

Shinoza�i,�N.,�Yabe,�H.,�Sato,�Y.,�Hiru�a,�T.,�Sutoh,�T.,�Matsuo�a,�T.�an���Kane�o,�S.� (2003).�S�ectrote��oral��in��o��of�inte��ration�of�au��itor���infor�ation�in�the�hu�an�brain.�Cog-nitive Brain Research,�17,�563–571.

Page 115: Unconscious Memory Representations in Perception

104� István�Win�ler

Sin��onen,� J.� (1999).� Infor�ation� an��� resource� allocation.� In� R.� Ba����ele��,� P.� Hancoc�� an���P.�Föl��iá��(E��s.),�Information theory and the brain�(��.�241–254).�Ca�bri����e,�UK:�Ca�-bri����e�Universit���Press.

Sn����er,�J.�S.�an���Alain,�C.�(2007).�To�ar���a�neuro�h��siolo��ical�theor���of�au��itor���strea��se��-re��ation.�Psychological Bulletin,�133,�780–799.

Sn����er,�J.�S.,�Alain,�C.�an���Picton,�T.�W.�(2006).�Effects�of�attention�on�neuroelectric�correlates�of�au��itor���strea��se��re��ation.�Journal of Cognitive Neuroscience,�18,�1–13.

Stefanics,�G.,�Há��en,�G.�P.,�Sziller,�I.,�Balázs,�L.,�Be�e,�A.�an���Win�ler,�I.�(2009).�Ne�born�in-fants��rocess��itch�intervals.�Clinical Neurophysiology,�120,�304–308.

Suss�an,�E.�(2005).�Inte��ration�an���se��re��ation�in�au��itor���scene�anal��sis.�Journal of the Acous-tical Society of America,�117,�1285–1298.

Suss�an,�E.�S.�(2007).�A�ne��vie��on�the�MMN�an���attention���ebate:�The�role�of�context�in��rocessin���au��itor���events.�Journal of Psychophysiology,�21,�164–175.

Suss�an,�E.�S.,�Bre���an,�A.�S.,�Wan��,�W.�J.�an���Khan,�F.�J.�(2005).�Attentional��o��ulation�of�electro�h��siolo��ical�activit���in�au��itor���cortex�for�unatten��e���soun��s��ithin��ultistrea��au��itor���environ�ents.�Cognitive, Affective, & Behavioral Neuroscience,�5,�93–110.

Suss�an,� E.,� Če�onienė,� R.,� Shesta�ova,� A.,� Näätänen,� R.� an��� Win�ler,� I.� (2001).� Au��itor���strea��se��re��ation��rocesses�o�erate�si�ilarl���in�school�a��e���chil��ren�as�a��ults.�Hearing Research,�153,�108–114.

Suss�an,�E.,�Go�es,�H.,�Nousa�,�J.�M.�K.,�Ritter,�W.�an���Vau��han,�H.�G.,�Jr.�(1998).�Feature�conjunctions�an���au��itor���sensor����e�or��.�Brain Research,�793,�95–102.

Suss�an,�E.,�Horváth,�J.,�Win�ler,�I.�an���Orr,�M.�(2007).�The�role�of�attention�in�the�for�ation�of�au��itor���strea�s.�Perception & Psychophysics,�69,�136–152.

Suss�an,�E.,�Ritter,�W.�an���Vau��han,�H.�G.,�Jr.�(1998).�Attention�affects�the�or��anization�of�au��i-tor���in�ut�associate����ith�the��is�atch�ne��ativit���s��ste�.�Brain Research,�789,�130–138.

Suss�an,�E.,�Ritter,�W.�an���Vau��han,�H.�G.,�Jr.�(1999).�An�investi��ation�of�the�au��itor���strea�-in���effect�usin���event�relate���brain��otentials.�Psychophysiology,�36,�22–34.

Suss�an,�E.,�Sheri��an,�K.,�Kreuzer,�J.�an���Win�ler,�I.�(2003).�Re�resentation�of�the�stan��ar��:�Sti�ulus�context�effects�on�the��rocess���eneratin���the��is�atch�ne��ativit���co��onent�of�event-relate���brain��otentials.�Psychophysiology,�40,�465–471.

Suss�an,�E.�an���Win�ler,�I.�(2001).�D��na�ic�sensor���u���atin���in�the�au��itor���s��ste�.�Cogni-tive Brain Research,�12,�431–439.

Suss�an,�E.,�Win�ler,�I.,�Huotilainen,�M.,�Ritter,�W.�an���Näätänen,�R.�(2002).�To�-��o�n�effects�on�sti�ulus-��riven�au��itor���or��anization.�Cognitive Brain Research,�13,�393–405.

Ta�e��ata,�R.,�Brattico,�E.,�Tervanie�i,�M.,�Var��ia��ina,�O.,�Näätänen,�R.�an���Win�ler,�I.�(2005).�Pre-attentive�re�resentation�of�feature�conjunctions�for�si�ultaneous,�s�atiall�����istribute���au��itor���objects.�Cognitive Brain Research,�25,�169–179.

Ta�e��ata,�R.,�Paavilainen,�P.,�Näätänen,�R.�an���Win�ler,�I.�(1999).�In��e�en��ent��rocessin���of�chan��es�in�au��itor���sin��le�features�an���feature�conjunctions�in�hu�ans�as�in��exe���b���the��is�atch�ne��ativit��.�Neuroscience Letters,�266,�109–112.

Tervanie�i,�M.�an���Huotilainen,�M.�(2003).�The��ro�ises�of�chan��e-relate���brain��otentials�in�co��nitive�neuroscience�of��usic.�Neurosciences and Music,�999,�29–39.

Tervanie�i,�M.,�Maur��,�S.�an���Näätänen,�R.�(1994a).�Neural�re�resentations�of�abstract�sti�u-lus�features�in�the�hu�an�brain�as�reflecte���b���the��is�atch�ne��ativit��.�NeuroReport,�5,�844–846.

Page 116: Unconscious Memory Representations in Perception

� In�search�for�au��itor���object�re�resentations� 105

Tervanie�i,�M.,�Saarinen,�J.,�Paavilainen,�P.,�Danilova,�N.�an���Näätänen,�R.�(1994b).�Te��oral�inte��ration�of�au��itor���infor�ation�in�sensor����e�or���as�reflecte���b���the��is�atch�ne��a-tivit��.�Biological Psychology,�38,�157–167.

The Merriam-Webster Dictionary (1997).�S�rin��fiel��,�MA:�Merria�-Webster�Inc.Tho��son,�W.�F.,�Hall,�M.�D.�an���Pressin��,�J.�(2001).�Illusor���conjunctions�of��itch�an�����ura-

tion�in�unfa�iliar�tone�sequences.�Journal of Experimental Psychology: Human Perception and Performance,�27,�128–140.

Ton��,�F.,�Men��,�M.�an���Bla�e,�R.�(2006).�Neural�bases�of�binocular�rivalr��.�Trends in Cognitive Sciences,�10,�502–511.

Treis�an,�A.�(1993).�The��erce�tion�of�features�an���objects.�In�A.�Ba����ele���an���L.�Weis�rantz�(E��s.),�Attention: Selection, awareness, & control. A Tribute to Donald Broadbent�(��.�5–35).�Oxfor��:�Claren��on�Press.

Treis�an,�A.�M.�an���Gela��e,�G.�A.�(1980).�A�feature�inte��ration�theor���of�attention.�Cognitive Psychology,�12,�97–136.

Treis�an,�A.�M.�an���Sch�i��t,�H.�(1982).�Illusor���conjunctions�in�the��erce�tion�of�objects.�Cognitive Psychology,�14,�107–141.

van�Noor��en,�L.�P.�A.�S.�(1975).�Temporal coherence in the perception of tone sequences.�PhD�thesis,�Ein��hoven.

van� Zuijen,� T.� L.,� Suss�an,� E.,� Win�ler,� I.,� Näätänen,� R.� an��� Tervanie�i,� M.� (2005).� Au��i-tor���or��anization�of�soun���sequences�b���a�te��oral�or�nu�erical�re��ularit��:�A��is�atch�ne��ativit���stu�����co��arin����usicians�an���non-�usicians.�Cognitive Brain Research,�23,�270–276.

van�Zuijen,�T.�L.,�Si�oens,�V.�L.,�Paavilainen,�P.,�Näätänen,�R.�an���Tervanie�i,�M.�(2006).�I�-�licit,� intuitive,�an���ex�licit��no�le����e�of�abstract�re��ularities� in�a�soun���sequence:�An�event-relate���brain��otential�stu����.�Journal of Cognitive Neuroscience,�18,�1292–1303.

Wi���ann,�A.,�Kujala,�T.,�Tervanie�i,�M.,�Kujala,�A.�an���Schrö��er,�E.�(2004).�Fro��s���bols�to�soun��s:� Visual� s���bolic� infor�ation� activates� soun��� re�resentations.� Psychophysiology,�41,�709–715.

Wi��ht�an,�F.�L.�an���Jenison,�R.�(1995).�Au��itor���s�atial�la��out.�In�W.�E�stein�an���S.�J.�Ro��ers�(E��s.),�Perception of space and motion� (2n���e��ition,���.�365–400).�San�Die��o,�CA:�Aca-��e�ic�Press.

Win�ler,�I.�(2003).�Chan��e���etection�in�co��lex�au��itor���environ�ent:�Be��on���the�o����ball��ara��i���.�In�J.�Polich�(E��.),�Detection of change: Event-related potential and fMRI findings�(��.�61–81).�Boston:�Klu�er�Aca��e�ic�Publishers.

Win�ler,�I.�(2007).�Inter�retin���the��is�atch�ne��ativit���(MMN).�Journal of Psychophysiology,�21,�147–163.

Win�ler,�I.�an���Co�an,�N.�(2005).�Fro��sensor����e�or���to�lon���ter���e�or��:�Evi��ence�fro��au��itor����e�or���reactivation�stu��ies.�Experimental Psychology,�52,�3–20.

Win�ler,� I.,�Co�an,�N.,�Csé�e,�V.,�Czi��ler,� I.� an���Näätänen,�R.� (1996).� Interactions�bet�een�transient�an���lon��-ter��au��itor����e�or���as�reflecte���b���the��is�atch�ne��ativit��.�Journal of Cognitive Neuroscience,�8,�403–415.

Win�ler,�I.�an���Czi��ler,�I.�(1998).�Mis�atch�ne��ativit��:�Deviance���etection�or�the��aintenance�of�the�“stan��ar��”.�NeuroReport,�9,�3809–3813.

Win�ler,�I.,�Czi��ler,�I.,�Suss�an,�E.,�Horváth,�J.�an���Balázs,�L.�(2005).�Preattentive�bin��in���of�au��itor���an���visual�sti�ulus�features.�Journal of Cognitive Neuroscience,�17,�320–339.

Page 117: Unconscious Memory Representations in Perception

10�� István�Win�ler

Win�ler,�I.,�Denha�,�S.�L.�an���Nel�en,�I.�(2009).�Mo��elin���the�au��itor���scene:�Pre��ictive�re��u-larit���re�resentations�an����erce�tual�objects.�Trends in Cognitive Sciences,�13,�532–540.

Win�ler,�I.,�Horváth,�J.,�Te��er�Sälejärvi,�W.�A.,�Näätänen,�R.�an���Suss�an,�E.�(2003a).�Hu�an�au��itor���cortex�trac�s�tas�-irrelevant�soun���sources.�NeuroReport,�14,�2053–2056.

Win�ler,�I.,�Kar�os,�G.�an���Näätänen,�R.�(1996a).�A��a�tive��o��elin���of�the�unatten��e���acous-tic� environ�ent� reflecte��� in� the� �is�atch� ne��ativit��� event� relate��� �otential.� Brain Re-search,�742,�239–252.

Win�ler,�I.,�Korz���ov,�O.,�Gu�en��u�,�V.,�Co�an,�N.,�Lin�en�aer-Hansen,�K.,�Alho,�K.,�Il�o-nie�i,�R.�J.�an���Näätänen,�R.�(2002).�Te��orar���an���lon��er�retention�of�acoustic�infor�a-tion.�Psychophysiology,�39,�530–534.

Win�ler,�I.,�Kushneren�o,�E.,�Horváth,�J.,�Če�onienė,�R.,�Fell�an,�V.,�Huotilainen,�M.,�Näätän-en,�R.�an���Suss�an,�E.�(2003b).�Ne�born�infants�can�or��anize�the�au��itor����orl��.�Pro-ceedings of the National Academy of Sciences USA,�100,�1182–1185.

Win�ler,�I.,�Paavilainen,�P.,�Alho,�K.,�Reini�ainen,�K.,�Sa�s,�M.�an���Näätänen,�R.�(1990).�The�effect�of�s�all�variation�of�the�frequent�au��itor���sti�ulus�on�the�event-relate���brain��oten-tial�to�the�infrequent�sti�ulus.�Psychophysiology,�27,�228–235.

Win�ler,�I.,�Suss�an,�E.,�Tervanie�i,�M.,�Ritter,�W.,�Horváth�J.�an���Näätänen,�R.�(2003c).�Pre�at-tentive�au��itor���context�effects.�Cognitive, Affective, & Behavioral Neuroscience,�3,�57–77.

Win�ler,�I.,�Ta�e��ata,�R.�an���Suss�an,�E.�(2005).�Event-relate���brain��otentials�reveal��ulti�le�sta��es�in�the��erce�tual�or��anization�of�soun��.�Cognitive Brain Research,�25,�291–299.

Win�ler,�I.,�van�Zuijen,�T.,�Suss�an,�E.,�Horváth,�J.�an���Näätänen,�R.�(2006).�Object�re�resenta-tion�in�the�hu�an�au��itor���s��ste�.�European Journal of Neuroscience,�24,�625–634.

Wolfe,�J.�M.,�Cave,�K.�R.�an���Franzel,�S.�L.�(1989).�Gui��e���search:�an�alternative�to�feature�in-te��ration��o��el�for�visual�search.�Journal of Experimental Psychology: Human Perception and Performance,�15,�419–433.

Yabe,�H.,�Tervanie�i,�M.,�Sin��onen,�J.,�Huotilainen,�M.,�Il�onie�i,�R.�J.�an���Näätänen,�R.�(1998).�The�te��oral��in��o��of�inte��ration�of�au��itor���infor�ation�in�the�hu�an�brain.�Psychophysiology,�35,�615–619.

Yabe,� H.,� Win�ler,� I.,� Czi��ler,� I.,� Ko��a�a,� S.,� Ka�i��i,� R.,� Sutoh,� T.� et� al.� (2001).� Or��anizin���soun���sequences�in�the�hu�an�brain:�The�inter�la���of�au��itor���strea�in���an���te��oral�inte��ration.�Cognitive Brain Research,�897,�222–227.

Zen��el,�B.�R.�an���Alain,�C.�(2009).�Concurrent�soun���se��re��ation�is�enhance���in��usicians.�Journal of Cognitive Neuroscience,�21,�1488–1498.

Z�isloc�i,�J.�J.�(1960).�Theor���of�te��oral�au��itor���su��ation.�Journal of the Acoustical Society of America,�32,�1046–1060.

Page 118: Unconscious Memory Representations in Perception

chapter�4

Representation of regularities in visual stimulation Event-relate����otentials�reveal��the�auto�atic�acquisition

István�Czi��ler�Institute�for�Ps��cholo�����of�the�Hun��arian�Aca��e����of�Sciences,�Bu��a�est

4.1 Introduction

In�the�Web of Science���atabase�101�ite�s�can�be�retrieve���for�the�ter��“visual�con-sciousness”�an���onl���three��ublications��ention�the�ter��“au��itor���conscious-ness”.� It� see�s� that� the� ter�� “visual� consciousness”� is� an� acce�te��� ter�� in� the�scientific�co��unit��,��hereas�“au��itor���consciousness”�is�not.�The���atabase���oes�not�contain�the�ter�s�“visual�unconsciousness”�an���“au��itor���unconsciousness”.�In�s�ite�of�the�non-existence�of�these�ter�s,�a�lar��e�bo�����of�stu��ies�has�been���e-vote���to�non-conscious�sensor����heno�ena.�Effects�of�subli�inal�visual�sti�ula-tion�(e.��.��ri�in��),�eli�ination�of�conscious�ex�erience�(�as�in��,�cro���in��,�etc.)�are�classical�as��ell�as�recent�research�to�ics�(for�revie�s�see�Bloc��2007;�Dehaene�an���Naccache�2001;�Ki��an���Bla�e�2005;�Meril�e�1992,�1997).�In�the�au��itor����orl���a��ell-�no�n�auto�atic��echanis��unconsciousl���evaluates�re��ularities�(see�Win�ler� this�volu�e).� In� this�cha�ter�I�revie��stu��ies� in��icatin���a�si�ilar�‘�ri�itive�intelli��ence’�(Näätänen�et�al.�2001)�in�vision.�Necessit���of�such�s��ste��is�challen��e���b���recent�theories�(O’Re��an�an���Noë�2001),�but�as�evi��ences�fro��co��nitive�neuroscience�sho�,�so�e�re��ularities�of�the�visual��or���are��reserve���an���co��are���to�the�re�resentation�of�the�on��oin���sti�ulation.�Re�resentations�of�such�re��ularities���evelo��outsi��e�the�focus�of�attention,�an���these�re�resenta-tions�are�not�necessaril���accessible� to�consciousness.�Ho�ever,� the�violation�of�the� re��ularities� is� not� �ithout� behavioral� consequences.� First� I’ll� list� evi��ences�in��icatin���the�existence�of�a�hi��h-ca�acit���visual��e�or���s��ste�,�thereafter�so�e���ata��ill��resente���sho�in���the��ossibilit���of�i��licit�visual��e�ories�of�lon��er�

Page 119: Unconscious Memory Representations in Perception

10�� István�Czi��ler

��uration.�The�next��art�of�the�cha�ter�revie�s�stu��ies�sho�in���behavioral�effects�base��� on� such� �e�or��� re�resentations.� As� the� results� of� �s��cho�h��siolo��ical���ata,�so�e�characteristics�of�an�i��licit�visual��e�or���s��ste���ill�be���iscusse��.�I’ll�ar��ue�that�this��e�or���is�sensitive�to�the�re��ularities�of�sti�ulation,�an���re-s�on��s�to�events�inco��atible�to�such�re��ularities.�Finall��,�the�functional�si��nifi-cance�of� this�s��ste���ill�be���iscusse��.�An�ex�licit��ur�ose�of� the�cha�ter� is� to�buil��� a� bri����e� bet�een� the� “co��lex� unconscious� �in��”� of� �an��� branches� of��s��cholo�����an���the�“��u�b�unconsciousness”�of�co��nitive��s��cholo�����(Bar��h�an���Morsella�2008).�

4.2 Lack of conscious detection of visual changes and large-capacity visual memories

As�the�stri�in���change blindness phenomenon�sho�s,�even�lar��e�chan��es�of�visual�scenes�re�ain�unnotice���if�(1)�the�chan��in���objects�are�outsi��e�the�fiel���of� fo-cal�attention,�an���(2)�no�chan��e-transients�are�available�(for�revie�s�see�Rensin��2000;�Si�ons�an���Levin�1997).�There�are�several���e�onstration��ara��i���s�of�the�chan��e�blin��ness.�In�the��o�ular�flic�er��ara��i����alternative�scenes�are��resente���for�a�short��erio���of�ti�e�(e.��.�250��s),�an���a�blan��fiel���of�si�ilar���uration�is��resente���bet�een�the��resentations�of� the�alternative�scenes.�Several�accounts�of�the�effect�are�available�(see�Si�ons�2000�for�a�revie�).�It�is��ossible�that�(1)�no���etaile���visual��e�or���is�for�e���fro��the��re-chan��e�scene�(e.��.�O’Re��an�an���Noë�2001);�(2)���etaile����e�or���is�for�e���fro��the��re-chan��e�scene,�but�this�re�resentation�is�annulate���b���the�follo�in���scene�(e.��.�Bec�er�et�al.�2000);�(3)���e-taile���re�resentation�is�for�e���for�both��re-�an����ost-chan��e�scenes,�but�no�co�-�arison�is��ossible�bet�een�these�re�resentations.�This�is�because�co��arison�is�restricte���to�a�li�ite���set�of�the��re-�an����ost-chan��e�scenes�(e.��.�Hollin���orth�an���Hen��erson�2002).�Finall��,�(4)�even�if�there�is�a�co��arison��rocess,�exce�t�fro��a�s�all�subset�of�the��re-�an����ost-chan��e�ele�ents,�the�results�of�this��ro-cess�re�ain�unconscious�(i��licit).�Concernin���the�latter�o�tion,�there�are�so�e�questions:�(a)�Is�there�an����roof�for�a�hi��h�ca�acit����e�or���s��ste��o�eratin���in�the��erio���bet�een�the�offset�of�the��re-chan��e�an���the�onset�of�the��ost-chan��e�scene?�(b)�Are�there�other�i��licit��e�or���s��ste�s�o�eratin���short�after�the��re-sentation�of�visual�scenes?�(c)�Is�there�an����roof�that�unnotice���chan��es�influence�behavior�an��/or�brain�activit��?�(��)�What�is�the�functional�role�of�a�s��ste����etect-in���chan��es�if�the�results�this�outco�e�re�ains�unconscious?1

Page 120: Unconscious Memory Representations in Perception

� Re�resentation�of�re��ularities�in�visual�sti�ulation� 109

4.2.1 Visual��e�ories��ith�lar��e�ca�acit���

Lan���an�et�al.�(2003)���e�onstrate���the�existence�of�a�lar��e-ca�acit���visual�stor-a��e.�Partici�ants��ere�as�e���to�re�ort�a�chan��in���ele�ent��ithin�ei��ht-ele�ent���is�la��s.�Pre-�an����ost-chan��e���is�la��s��ere�se�arate���b���blan��intervals�of�1500��s.�Cues�in��icate���the�location�of�the��ossible�chan��in���ele�ents.�The�cues��ere�effective�if�the����ere��resente����ithin�the�blan���erio��,�i��licatin���the�activit���of��e�or���stora��e�for�the��hole�inter-��is�la����erio��.�Ho�ever,�the�effectiveness�of�the�cue���reatl�����i�inishe���at�the�onset�of�the�secon�����is�la��.�This�result�sho�s�that�the��resentation�of�the�secon�����is�la���over�rites�the�re�resentation�of�the�first�one,�at�least�at�the�level�of�consciousness.�Ho�ever,�the��ossibilit���re�ains�that�the��e�or���for�the��re-chan��e���is�la���has�unconscious�effects.

The�no�-classic�ex�eri�ents�on�iconic��e�or���(e.��.�S�erlin���1960)���e�on-strate���a�lar��e-ca�acit���visual�stora��e.�B���no��the�ter��“iconic��e�or��”�beca�e�a�bi��uous.� In� the� sixties� iconic��e�or����as�consi��ere���as�a��assive�non-cat-e��orical�stora��e,�the�‘ra���aterial’�of�co��nitive��rocesses.�The�content�of�iconic��e�or����as�su��ose���to�be�transferre���into�a�short-ter��store.�Transfer�fro��iconic�stora��e��as�consi��ere���as�a�relativel���slo�,�serial��rocess�(20��s/ite�,�see�e.��.�S�erlin���1967),��ith�the�consequence�of�ite��loss�before�the�transfer�to�the�“short-ter�”�stora��e.�Transfer��as�consi��ere���as�an�active��rocess,�in�sense�that�attentional�(controlle��)��rocesses��ere�su��ose���to�influence�the�or��er�transfer.�Bac��ar���visual��as�in���(�resentation�of��atterne���sti�uli�after�the�offset�of�a�visual�arra��)�su��ose��l�����estro��s�the�content�of�iconic��e�or��.�In��rinci�le,�this�t���e�of�iconic��e�or���is�ca�able�of�storin���the�content�of�the��re-chan��e�arra��s,�but��ost-chan��e�sti�uli��oul���over�rite�its�content.�Further�ore,�bein���a��as-sive�stora��e,�co��arison��rocesses�cannot�ta�e��lace��ithin�the�iconic��e�or��.�Data�li�e�those�re�orte���b���Lan���an�et�al.�(2003)�corres�on��s�to�such�conce�t�of�iconic/sensor����e�or��.�This�is�because�cue�effectiveness���ecrease���at�the�arrival�of�the��ost-chan��e���is�la��.�The�onl�����iver��ence�fro��the�iconic��e�or���results�is�the�lon��er�esti�ate���lifeti�e�of�the��e�or���in�the�Lan���an�et�al.�(2003)�stu����,�i.e.�in�the�tra��itional��easures�of�iconic��e�or���(�artial�re�ort�of�the�cue���ite�s)�the���uration�of�the�iconic�stora��e�is�shorter.�Res�onse���ifferences�(binar�����eci-sion)��a���ex�lain�the���iscre�anc��,�but�there�is�no��roof�(neither���irect�nor�in��i-rect)�on�the�connection�bet�een�the�stora��e�un��erl��in����artial�re�ort�su�eriorit���effects�an���the��e�or���un��erl��in���an����in��s�of�chan��e���etections.�

Contrar��� to� the� conce�t� of� a� �assive� iconic� stora��e� an��� slo�� acquisi-tion�of� cate��orize��� (lon��-ter�)��e�or��,� an�alternative�vie���as��ioneere���b����M.�Coltheart�(1980).�Accor��in���to�his�su����estion,�i��entification�(What�is�this?)�an���location�(Where�is�it?)�infor�ation�is�acquire���quic�l��,�an���the�coor��ination�of�the�t�o�t���es�of�infor�ation�is�auto�atic.�Iconic��e�or���is�the�result�of�such�

Page 121: Unconscious Memory Representations in Perception

110� István�Czi��ler

coor��inate���infor�ation.�Ho�ever,�the�result�of�the�coor��ination�is�fra��ile,�an���in�the�absence�of�consoli��ation��rocesses�it���eca��s.�Consoli��ation��as�su��ose���to� be� a� ca�acit��� controlle��� �rocess� (i.e.,� influence��� b��� to�-��o�n� effects).� An�o�en�question�is�the�fate�of�the�non-consoli��ate���contents.�If�such�contents���eca���quic�l��,�ex�lanations�of�the�chan��e-blin��ness��heno�ena�built�u�on�the�‘revise��’�icon��oul���be�si�ilar�those�base���on�the�tra��itional�vie��of�iconic��e�or��,�i.e.,�exce�t�fro��a�s�all�subset�of�ele�ents�(objects),�no��ro�er�ro�-�aterial��oul���be�available�for�chan��e���etection.�One��a���su��ose�that�the�role�of�the�ca�acit���li�ite����rocesses�is�the�acquisition�of��e�or���re�resentations�ca�able�of�re��ulat-in���the�on��oin���behavior�an��/or�influencin���the�content�of�conscious�ex�erience.�Ho�ever,�at�the�sa�e�ti�e,��e�or���re�resentations�outsi��e�the�ca�acit���li�ite����rocesses��a����reserve�their�contents�for�a�lon��er��erio���as�i��licit�bac���roun����no�le����e.�As�a��art�of�bac���roun����no�le����e,�this��e�or���s��ste��is�suitable�for��artici�atin���in�i��licit�chan��e���etection��rocesses.�Therefore�the��eanin���of�the�ter��“consoli��ation”�is�not�unequivocal.

In� the� follo�in��,�ar��u�ents��ill�be��resente��� favorin��� the��ersistence�of�a�lar��e-ca�acit���visual��e�or��.�In�an�influential�stu�����Roc��an���Gut�an�(1981)��resente���t�o��eanin��less�su�eri��ose���sha�es.�The�sha�es�ha�����ifferent�colors,�an���the�tas���as�to�ju����e�the�aesthetic�value�of�the�sha�e��ith�a��articular�color.�Reco��nition��erfor�ance�of�the�sha�es��ith�non-atten��e���color�in�an�inci��ental�(unex�ecte��)�tas���as�at�chance�level.�Accor��in��l��,�no�explicit��e�or���for�these�sha�es��as�available.�Ho�ever,�in�a�later�stu�����DeSche��er�an���Treis�an�(1996)�re�orte���evi��ence�for�an�implicit��e�or���effect�of�such�non-atten��e���sha�es.�In�this�ex�eri�ent�the��artici�ants�co��are���the�atten��e���sha�es�to�test�sha�es,�an���the��atchin���RT��as��easure��.�The���esi��n�involve���the��ossibilit���of�the�ne��a-tive��ri�in���effect.�In�the�ne��ative��ri�in���stu��ies�(for�a�revie��see�Ti��er�2001)��reviousl���rejecte���sti�uli�elicit�res�onses��ith�lon��er�RT�than�control�sti�uli.�In�the�DeSche��er�an���Treis�an�(1996)�stu����� in�so�e�trials� the�sha�es� in�the�irrelevant�color��ere��resente���a��ain,�but�this�ti�e�in�the�relevant�color.�RT�in-crease���to�such�sha�es,�i.e.�ne��ative��ri�in���effect�e�er��e��.�Accor��in��l��,�sti�uli��ithout� ex�licit� �e�or��� re�resentation� have� a� consequence� on� later� behavior.�The�ne��ative��ri�in���effect�has�sur�risin��l���lon�����uration;�the�effect�survive���200�intervenin���sti�uli.

Visual�search�is�one�of�the��ost�frequentl���e��lo��e����ara��i���s�in�research�on�visual��rocessin��.�In�various�t���es�of�the��ara��i�����artici�ants�have�to�in��icate�the��resence/absence�or�the�i��entit���of�a��articular�ite��(tar��et).�Latent�learnin���of�visual���is�la��s�is���e�onstrate���in�visual�search�tas�s.�Usin���a�contextual�cuin��2�

variation�of�the�search��ara��i���,�Jian���an���Leun���(2005)�as�e���the��artici�ants�to�search�for�a�blac��(or��hite)�T�a�on���blac��L�an����hite�L���istractors.�The�tas���as�the�i��entification�of�the�orientation�of�the�T-sha�e.�When�the�blac��T��as�the�

Page 122: Unconscious Memory Representations in Perception

� Re�resentation�of�re��ularities�in�visual�sti�ulation� 111

tar��et,�blac��Ls��ere�consi��ere���as�atten��e���context,��hereas�the��hite�Ls�serve���as�the�i��nore���context.�In�several�trials�either�the�atten��e���or�the�i��nore���context��as�re�eate��.�After�ar��s�the�atten��e���an���i��nore���context�chan��e���color.�Con-text�re�etition�effect��as�assesse���for�the�atten��e���an���i��nore���context,�an���the�effect�of�transfer��as�assesse���as�the��erfor�ance�after�the�chan��e�of�the�context�colors.�Re�etition�of�the�atten��e�����istractor�set�facilitate���the�search,�but�no�such�context�effect�a��eare���to�the�re�etition�of�the�i��nore���set.�Ho�ever,�reversin���the�color�relevance,�the��reviousl���i��nore���context� le���to�faster�RT�than�a�ne��arran��e�ent� (context)�of���istractors.�Accor��in��l��,� the�re�eate����resentation�of�an�arran��e�ent�of�non-atten��e�����istractors�resulte��� in� latent� learnin��,� i.e.,� the�acquisition�of�lon��-lastin���an���hi��h�ca�acit���visual��e�or���re�resentation.

The�role�of�i��licit��e�or���in�visual�search��as�also���e�onstrate���in�a�series�ex�eri�ents�b���Lleras�et�al.�(2005).�These�authors�investi��ate���tar��et�i��entifica-tion�RT�in�case�of�re�eate�����is�la��s�containin���16�or�32�ele�ents.�Distributions�of�search�RT��ere��ar�e��l�����ifferent�in�the�first�an���in�the�subsequent��resenta-tions.�As�an�exa��le�(Ex�eri�ent�1),�at�the�first��resentation�onl���4%�of�the�re-s�onses��ere�faster�than�500��s,��hile�in�the�secon���an���thir����resentations�RTs�shorter� than�500��s�a��eare����ith��robabilities�of�53%�an���52%,�res�ectivel��.�The�authors�ter�e���this�effect�as�‘ra�i���resu��tion’.�This�effect�is�lar��er�at�lon��er�sti�ulus���uration,�it�is�not���isru�te���b���the��resentation�of�a�ne��search���is�la��,�the���uration�of�the�effect�(as� in��icate���b���the�interval�bet�een�the�consecutive��resentations)�is�lon��er�than�3000��s.�It�is���ifficult�to�ex�lain�these�results��ithout�the�consi��eration�of�a�re�resentation�acquire���after�the�first��resentation.�Further�results�are�nee��e���to���isclose�the�ca�acit���of�this��e�or��.�The�authors�su����este���that�it�is�restricte���to�a�li�ite���set�of���is�la���ele�ents,�inclu��in���the�tar��et.�Ho�-ever,�this��e�or���is���efinitel���i��licit,�an���at�the�sa�e�ti�e�it�is�“active”�in�the�sense�that�it�contributes�to�the�i��entification�of�the�tar��et�at�the�next��resentations�of�the���is�la��.�As�Lleras�et�al.�(2005)�su����este��,�a��re��ictive��o��el��as�for�e���at�the�first��resentation,�an���at�the�consecutive��resentations�this��o��el��as�teste��.

4.2.2 Develo��ent�of��e�or���re�resentations�for�visual�sti�uli�

Several�atte��ts��ere��a��e�to���e�onstrate�the���ra��ual�buil��-u��of��e�or���re�-resentation�in�the�chan��e�blin��ness��ara��i���.�Rensin��et�al.�(2000)��resente���one�of�the�scenes�for�a�lon��er��erio���of�ti�e�before�the�flic�erin���sequence�of�chan��-in���scenes.�This��revie���as�consi��ere���as�an�o��ortunit���for�the�consoli��ation�of� the�re�resentation�of� the�scene.�This��revie��con��ition�resulte��� in�no�faster�(less�flic�er�c��cle)�chan��e���etection.�Ho�ever,�in�so�e�other�stu��ies�accu�ula-tion�of�infor�ation��as�observe���over�successive��resentations.�Object��revie��

Page 123: Unconscious Memory Representations in Perception

112� István�Czi��ler

increase���localization��erfor�ance�(Hollin���orth�2005).�Recall��e�or���at�suc-cessive�interru�te����resentation�of�a�scene�(another�scene��as��resente���bet�een�t�o�250��s��resentations�of�a�scene)��as�si�ilar�to�the�results��ithout�interru�-tion�(Melcher�2001).

In�a�recent�stu�����Vierc��an���Kiesel�(2008)�varie���the�nu�ber�of��re-chan��e�fra�es.�Either�one�(AB�c��cle),�t�o�(AABB�c��cle)�or�five�(AAAAABBBBB�c��cle)��ictures��ere��resente���before�the�chan��e.�In�case�of�infor�ation�accu�ulation,�the�lon��er�the�c��cle,�the�less�chan��e�is�nee��e���for�chan��e���etection.�The�results�confir�e���this�assu��tion.�Usin���an�“o����ball-li�e”�sequence��here�one�of� the�scenes� �ere� �resente��� frequentl��,� an��� the� other� one� infrequentl��� (AAAAAB),�an���co��arin���chan��e���etection��ith�a�sequence�of�equal�nu�ber�of�chan��e�in�a�“rovin��”�sequence�(AAABBB),�the�rovin����roce��ure��ro��uce���faster���etection.�This�result�sho�s�that� the�buil��-u��of�both�re�resentations�contributes� to�con-scious� chan��e-��etection.� Finall��,� the� authors� obtaine��� better� chan��e-��etection��erfor�ance�for�the�re��ular��resentation�of�the�scenes�(AABB)�than�for�the�ran-��o���resentation��ith�i��entical�nu�ber�of�re�etitions�an���chan��es.�It�see�s�that�the�ran��o���resentation��revente���the�buil��-u��of�se�arate��e�or���re�resenta-tion�for�the�t�o���ifferent�scenes.

Such�results�e��hasize�the�role�of�sti�ulus�re�etition.�It�see�s�that�re�eti-tion�is�not�onl���a�te��oral�su��ation��rocess.�Successive��resentations�can�be�conceive���as�successive�sa��les�fro��the�environ�ent.�I��entical�sa��les�are�ca-�able�of�creatin���re�resentation�of�re��ularities,��hereas�a�sin��le�sa��le,�even�if�the�sa��lin���ti�e�is�lon��,�is�just�a�sin��le�event.�Ho�ever,�at�this�ti�e�there�is�no���irect�su��ort�of�this�su����estion.�At�an���rate,�this�su����estion��a���account�for�the�lac��of�beneficial�effect�of��revie��in�the�Rensin��et�al.�(2000)�stu����.�

In�a���eneral�theor���of�visual�scene�re�resentation�Rensin��(2000)��ro�ose���that�coherent�object�re�resentation���e�en��s�on�focal�attentional��rocesses.�With-out�a�lo�-ca�acit���attentional�s��ste��onl���volatile�units�(proto-objects)�are�for�e��.�Proto-object���eneration�is�continuous,�an���in�the�absence�of�consoli��ate����e�-or���consecutive��roto-objects�re�lace�each�other�at�each�sti�ulus��resentation.3�Therefore��roto-objects�cannot��artici�ate�in�co��arison��rocesses�bet�een�the��re-�an����ost-chan��e�scenes.�Proto-objects�are�stabilize����hen�fee��-bac��arrives�fro��hi��h-level�connections�(calle�� nexus).�This��rocess�requires�attention.�Ob-jects� of� such� state� belon��� to� the� coherence field.� With��ra�al� of� attention� lea��s�the�loss�of�coherence,��ithout�an���re�ainin�����etaile���(visual)��e�or��4�Rensin��(2002)�also�h���othesize���fast,�non-attentive��rocesses�ca�able�of�reco��nizin���the�abstract��eanin���(��ist)�of� scenes,�an���ca�able�of��reservin��� the� la��out� (s�atial�arran��e�ent)�of�so�e�objects.�Gist�an���la��out�infor�ation��a���influence�s�atial�attention,�an���this��a���the���contribute�to�the�for�ation�of�object�coherence.�It�is�unclear,�ho�ever,��hat�is�the�status�of�the�‘��ist’�an���the�‘la��out’.�

Page 124: Unconscious Memory Representations in Perception

� Re�resentation�of�re��ularities�in�visual�sti�ulation� 113

Fast�an���auto�atic��rocesses�contribute�to�the�for�ation�of�short-term con-ceptual memory� (e.��.,�Potter�1976;� Intraub�1999).�Ho�ever,� such��e�ories�are�“fleetin��”�(V.�Coltheart�1999),�an���these�re�resentations�are�usuall���consi��ere���as�onl����recursors�of�stabilize����e�or���re�resentations.

Contrar��� to� tra��itional� �o��els� of� co��nitive� �s��cholo����,� e�er��ence� of� cat-egorical representations�is�sur�risin��l���fast.�As�an�exa��le,�VanRullen�an���Thor�e�(2001)�as�e����artici�ants�to���eci��e�on�cate��or����e�bershi��(ani�als�or�vehicles)�of��icture�contents.�Event-relate����otentials�(ERPs)���ifferentiate���the�t�o�cate��o-ries�as�earl���as�70–80��s�after�sti�ulus�onset,�an���lar��e���ifferences��ere�obtaine���150��s��ost�sti�ulus�(for�relate���results�see�e.��.�Mouchetant-Rostain���et�al.�2000;�Schen��an�et�al.�1998).�Sacca��ic�e��e��ove�ents�to�a�tar��et�cate��or����ere�initiate���as� earl��� as� 120� �s� (Kirchner� an��� Thro�e� 2005),� �ith� a� �e��ian� RT� of� 228��s.�Mini�al��anual�RT�in�the�cate��orization�tas���as�260��s,�but�the��e��ian��as�400��s,�i.e.,��uch�lon��er�(Delor�e�et�al.�2004).�Ph��sical��ara�eters�of�the��ic-tures,�li�e�contrast�influence���the��rocessin���ti�e�(Macé�et�al.�2005).

Develo��ent�of���ist,�as�a�re�resentation�of�visual�scenes�see�s�to�be�an�au-to�atic��rocess.�Li�et�al.�(2002)�obtaine���no��erfor�ance���ecre�ent�in�a�cate��o-rization�tas���here�the��ictures��ere��resente���eccentricall���in�non-cue����osi-tions,�an���a�si�ultaneous���e�an��in���tas���as��erfor�e���at� the�center�of� the�visual�fiel��.

On�the�basis�of�the��revious�results�one��a����re��ict�no�chan��e�blin��ness�in�case�of�(a)�chan��in�����ist��ith�si�ilar�ele�ents,�(b)��ith�scenes�containin���objects�onl����ithin�the�ca�acit���of�short-ter��cate��orical��e�or��.�Testin����ossibilit���(a)�is���ifficult�(but�not�i��ossible).�Concernin����ossibilit���(b),�results�of�a�recent�stu�����on�a�relate����heno�enon�are��ro�isin��.�As�the�inattentional blindness��heno�e-non�(Mac��an���Roc��1998)�sho�s,�even�a�su����en�a��earance�of�an�object�re�ains�unnotice���if�the�object�is��resente���to��ether��ith�tas�-relate���sti�uli.�Ho�ever,�as�Koivisto�an���Revonsuo�(2007)���e�onstrate��,�unex�ecte���sti�uli��ith�se�antic�relation�to�the�tas��sti�uli�avoi���the�inattentional�blin��ness��heno�enon.�Unex-�ecte���sti�uli��ere�reco��nize���even�if�the��h��sical�a��earance�of�the�tas��sti�uli�an���the�unex�ecte���sti�uli��ere���ifferent�(relate����or��s�an�� �ictures).�As�an�ex-�lanation,�se�antic�activation��i��ht�contribute�to�the�consoli��ation�of�the�(oth-er�ise�fleetin��)�re�resentation�of�the�unex�ecte���irrelevant�sti�uli.�

4.2.3 Behavioral�effects�of�non-conscious�chan��e�on���etection

Perce�tual��rocesses��ithout�a�areness�are��ell���e�onstrate���(for�a�revie��see�e.��.�Meri�le�et�al.�2001),�an���criteria�of��erce�tion��ithout�a�areness�are�carefull�����efine���(e.��.�Sch�i��t�an���Vorber���2006).�Se�antic��rocessin���of�verbal��aterial�

Page 125: Unconscious Memory Representations in Perception

114� István�Czi��ler

�ithout� a�areness� is� sho�n� in� �erio��s� of� attentional blink5 (Luc�� et� al.� 1996).�Ho�ever,�the�level�of��rocessin���see�s�to�be���ifferent��hen�conscious��rocessin���is��revente���b���insufficient�in�ut�qualit���(li�e��as�in��)�an���b���the�lac��of�control��rocesses�(li�e� inattention).� In� the� for�er�case�hi��her� level��rocessin��� is�ques-tionable�(Koui��er�an���Dehaene�2007),��hereas�in�the�latter�se�antic��rocessin���is��ell-��e�onstrate��,�even�at�the�level�of�brain�electric�activit���(Vo��el�et�al.�1998;�Rol�e�et�al.�2001).�Dehaene�(e.��.�Dehaene�et�al.�2006)�intro��uce���the�ter��‘�re-conscious’��hen�su�ra-threshol���sti�ulation�elicits�no�concurrent�an���fee��-bac��activit���fro��central�executive�(frontal)��echanis�.�Ma��be�this�is�the�case�in�the�i��licit���e�onstrations�of�se�antic��rocessin��.

In�a��i��el�����iscusse���stu�����Ferna��ez-Duque�an���Thornton�(2000)���esi��ne���a� version� of� unatten��e��� �ri�in��,� an��� co�bine��� it� �ith� the� chan��e� blin��ness�effect.� The� tas�� �as� the� ��etection� of� the� orientation� of� a� �hite� bar� a�on��� an�arra��� of� ��ar�� bars.� The� tar��et� �as� �resente��� for� a� short� ��uration,� an��� its� a�-�earance��as��rece��e���b���the��resentation�of�t�o�si�ilar�arra��s.�The�ti�in���of�the��rece��in���arra��s�corres�on��e���to��ara�eters�o�ti�al�for�chan��e�blin��ness�(250��s�sti�ulus���uration�an���250��s�inter-sti�ulus�interval).�In�certain�tri-als�one�of�the�bars�in�the�secon���arra����as�rotate���b���90���e��ree�(relative�to�the�orientation�in�the�first�arra��).�The��ost�interestin���fin��in���of�the�stu������as�an�increase���res�onse�accurac���in�trials��hen�(a)�the�chan��e���bar�ha���the�sa�e�ori-entation�as�the�test�bar�(con��ruenc���effect),�even�if�(b)��artici�ants���i���not�notice�the� rotation� (chan��e� blin��ness),� an��� (c)� increase��� �erfor�ance� a��eare��� also�in� locations�other� than� the�chan��in���ones� (i.e.,� res�onses��ere��ore�accurate�even�in�trials��here�the�test�bar�a��eare���in�a�location�o��osite�to�the�chan��in���bar).�Mitroff�et�al.�(2002)�challen��e���the�i��licit�chan��e���etection�ex�lanation.�The��� �ointe��� out� that� the� in� the� Ferna��ez-Duque� an��� Thornton� (2000)� stu�����the�chan��in���bar��i��ht�function�as�a�localization�cue�for�the�a��earance�of�the�test�bar�(the�test�bar�occurre���either�in�the��osition�of�chan��e�or�in�the��osition�o��osite�to�the�chan��e,�but�not�in�the�other�six��ossible��ositions).�Eli�inatin���the�s�atial�effect�(tests��ere��resente���in�each�location),�Mitroff�et�al.�(2002,�Ex-�eri�ent�4B)�obtaine����erfor�ance�increase�onl���the�in�locations�of�the�chan��-in���bar�of�the�secon���arra��.�Further�ore�in�the�ex�eri�ent�co�ie���the�ori��inal���esi��n�(Ex�eri�ent�4A)�the��artici�ants��ere�a�are�of�the�contin��enc���bet�een�the�location�of�the�chan��in���bar�an���the�test�bar.�Ho�ever,�Ferna��ez-Duque�an���Thornton�(2003)��ointe���out�that�Mitroff�et�al.�(2002)�obtaine���no�con��ruenc���effect�(i.e.,�in�the�“s�atiall���cue��”�trials��erfor�ance��as�hi��her�for�both�chan��e���an���unchan��e���bars,�irres�ective�of�the��artici�ants’�a�areness�of�the�chan��e).�The�lac��of�con��ruenc���effect��as�attribute���to�a�floor�effect�(��enerall���lo��accu-rac���an���slo��RT).�Therefore�Ferna��ez-Duque�an���Thornton�(2003)��ent�on�to�re�licate�the�stu�����usin���a�con��ition��ith�s�atial�uncertaint��,�but��ith�a�hi��her�

Page 126: Unconscious Memory Representations in Perception

� Re�resentation�of�re��ularities�in�visual�sti�ulation� 115

��eneral��erfor�ance�level�(��uration�of�the�test�arra����as�lon��er).�The���obtaine���con��ruence�effect�even� in� trials�of�non-��etecte���chan��e.� In� this�stu�����RT��as�shorter� in�the�con��ruent� trials.�Finall��,� in�a��ore�recent�stu�����Lalo��aux�et�al.�(2006)�atte��te���to�control�for�so�e��ossible��etho��ical��roble�s�of�the��revi-ous� stu��ies,� an��� increase��� the� nu�ber� of� �artici�ants.� The� results� reinforce���the�i��licit-chan��e-effect�vie��(in�una�are�chan��e�trials�both�RT�an���accurac����erfor�ance��as�hi��her��hen�the�orientation�of�the�test�sti�ulus��as�con��ruent��ith�that�of�the�bar�after�the�chan��e),�i.e.,�the�ori��inal�su����estions�as��ro�ose���b���Ferna��ez-Duque�an���Thornton�(2000).�

4.2.4 Neuroscience�of�un��etecte���chan��es

4.2.4.1 Change blindness related studiesUsin���the��etho��s�of�event-relate���brain��otentials�(ERPs),�Turatto�et�al.�(2002)�investi��ate���effects�of���etecte���an���un��etecte���chan��es��ithin�the�visual�bac�-��roun���an���fore��roun��.�To�this�en���the����resente�����re���circles�(fore��roun��)�over�vertical� blac�-an��-�hite� stri�es� (bac���roun��).� A� �air� of� such� sti�uli� �as� �re-sente��,�an���“sa�e-��ifferent”���ecision��as�require��.�While��artici�ants�s�ontane-ousl�����etecte���bri��htness�chan��es�in�the�fore��roun���(i.e.,�in�the�circles),�chan��es�of�the�bac���roun���(i.e.�chan��e�of�the���ar��stri�es�to��hite�an���vice�versa)��ere���etecte���onl���after�the�instruction�about�the��ossibilit���of�such�chan��es.�Unli�e�un��etecte���chan��es�(chan��e�in�the�bac���roun��)���etecte���chan��es�(chan��e�of�the�circles�an���chan��e�of�the�stri�es�after�the�instruction)�elicite���ERP�activit���(late��ositivit��).�This�activit���e�er��e���earlier�over�the�anterior�(frontal)�re��ions�than�over� the��osterior� (�arietal)� re��ions.�The�authors�conclu��e��� that� this� anterior-�osterior���irection��as�a�reflection�of�to�-��o�n�influences.�In�this�stu�����no�earli-er�activities��ere�anal��ze��,�so�it�is�un�no�n,��hether�un��etecte���chan��es�elicite���activit�����ifferent�fro��sti�ulus�re�etition.�In�the�latenc���ran��e�of�late��ositivit���there�are�various�sources�of�ERP�activit���(P3a,�P3b),�an���these�activities�are�not�hierarchicall��� ��e�en��ent.� Therefore� the� results� are� not� conclusive� in� assessin���to�-��o�n�influences.

Usin���al�hanu�eric�sti�uli�Nie��e����en�et�al.�(2001)�also�obtaine�����ifferences�in�the�late��ositivit���latenc���ran��e�bet�een���etecte���an���un��etecte���chan��es.�In�this�ex�eri�ent�either�the�i��entit���or�the�location�of�a�character�beca�e���iffer-ent.�The�alternative�sti�uli��ere��resente���in�re�eatin���c��cles.�Detecte���chan��es�elicite���the�late��ositivit��,��hereas��isse���chan��es�elicite���ERP�activit���si�ilar�to�the�no-chan��e�trials.�Interestin��l��,�late��ositivit���e�er��e���to�chan��in���sti�uli�in�the�sti�ulus�c��cle��rece��in���the�re�orte���chan��e,�i.e.,�ERP�a��eare���to�be�a��ore�sensible��easure�of� chan��e���etection� than� the�ex�licit� re�ort.�The�authors���i���

Page 127: Unconscious Memory Representations in Perception

11�� István�Czi��ler

not��easure�an���chan��e-relate���effects�in�earlier�ERP�latenc���ran��es.�It�shoul���be�note���that�the�avera��e���nu�ber�of�trials��as�relativel���lo�;�therefore�i��entifica-tion�of�ERP�co��onents��ith�lo�er�a��litu��es��oul���have�been���ifficult.

Koivisto� an��� Revonsuo� (2003)� ��esi��ne��� an� ERP� ex�eri�ent� usin��� sti�uli�si�ilar�to�that�of�the�Ferna��ez-Duque�an���Thornton�(2000)�stu����.�Pairs�of���is-�la��s�containin���vertical�an���horizontal�bars��ere��resente��,�an����artici�ants���e-ci��e����hether�the�secon����e�ber��as�i��entical�to�the�first�one,�or�one�of�the�bars�chan��e���orientation.�As�a�hint�of�i��licit�chan��e���etection,�the���obtaine���lon��er�RT�in�un��etecte���chan��e�trials�than�in�the�no-chan��e�trials�(see�also�Willia�s�an���Si�ons�2000).�Ho�ever,�there��as�no�ERP���ifference�bet�een�the�non-��etecte���chan��e�an���no-chan��e�con��itions.

Ei�er� an��� Mazza� (2005)� ca�italize��� on� results� sho�in��� that� attention� ��i-recte���to�one�si��e�of�the�visual���is�la���elicits�a�ne��ative�ERP�co��onent�over�the�contralateral��osterior� locations� (N2�c).�Pairs�of���is�la��s�containin��� faces��ere��resente��,��ith�the��ossibilit���of�a�chan��in���face�across�the�successive��airs.�The�tas���as�“sa�e-��ifferent”���ecision.�Unli�e�un��etecte���chan��es,���etecte���chan��es�elicite���the�N2�c�co��onent,�an���elicite���enhance���late��ositivit��.6�

Fernan��ez-Duque� et� al.� (2003)� �resente��� �ictures� of� co��lex� scenes� in� a�chan��e�blin��ness�situation.�The����ere�able�to�co��are�the�ERPs�to�un��etecte���chan��es� to� non-chan��in��� scenes� (i.e.,� �hen� �artici�ants� �ere� una�are� of� the�chan��e�versus�non-chan��in���scenes��hen�the��artici�ants�searche���for�a��ossible�chan��e).�Si��nificant�ERP���ifference�e�er��e���in�the�240–300��s�ran��e�as�a��osi-tive���eflection�over�the�anterior�locations�to�the�un��etecte���chan��e.�This�fin��in���o�ene���the��ossibilit���of�ERP�si��ns�of�i��licit�chan��e���etection.

In�a�recent�investi��ation�Ki�ura�et�al.�(2008c)�obtaine���an�anterior��ositive�ERP� co��onent� to� un��etecte��� irrelevant� color-chan��es.� The� relevant� sti�uli�(��ots)� a��eare��� at� the� center,� the� tas�� �as� the� ��etection� of� the� �ossible� size-chan��e�of�the�secon����e�ber�of�a���ot-�air.�These���ots��ere�surroun��e���b���other,�but�tas�-unrelate���colore�����ots.�In�several�trials�the�color�of�the�tas�-irrelevant���ots��as���ifferent�in�the�t�o��e�bers�of�the�sti�ulus��air.�Color�chan��e�elicite���a��ositive�ERP�co��onent.�Interestin��l��,�the�latenc���of�the��ositive�co��onent��as�shorter�(160–180��s)� than�the��ositivit���of� the�Fernan��ez-Duque�et�al.� (2003)�stu����.�The�authors�attribute���the���ifference�to�the�co��lexit�����ifference�bet�een�the���is�la��s�of�the�t�o�ex�eri�ents.

While�ERP��etho��s�are� sensitive� to� the� te��oral�as�ects�of�brain�activit��,�sources�of�activit���can�be�i��entifie���b���usin���brain�i�a��in����etho��s�(PET,�fMRI).�In�an�fMRI�stu�����Bec��et�al.� (2001)�co��are���the�activit���to���etecte���an���un-��etecte���chan��es��ithin�a�secon��ar��� tas�.�Un��etecte���sti�ulus�chan��e�elicite���activit���in��osterior�areas�(fusifor��an���lin��ual�����ri)�an���also�in�the�in�the�infe-rior�frontal�����rus.�This��attern��as��ar�e��l�����ifferent�fro��the�activit���follo�in���

Page 128: Unconscious Memory Representations in Perception

� Re�resentation�of�re��ularities�in�visual�sti�ulation� 117

��etecte���chan��es.�Detecte���chan��es�elicite���activit���in��arietal�an�����orsolateral�frontal� cortices,� an��� the� activit��� increase��� in� the� sti�ulus-s�ecific� brain� areas.�The�i��ortance�of�these�results�is�t�ofol��.�First,�un��etecte���chan��es�elicite���char-acteristic�brain�activit��,�an���secon��,�in�conscious���etection��refrontal�areas��ere�recruite��.7�

4.3 “Oddball” studies: The visual mismatch negativity

Results�fro��the�au��itor����o��alit����rovi��e���convincin���evi��ences�about�an�i�-�licit��e�or���s��ste��ca�able�of�the���etectin���environ�ental�re��ularities.�Sti�-uli�violatin���such�re��ularities�elicit�the��is�atch�ne��ativit���(MMN)�co��onent�of�ERPs.�MMN�is�elicite���even�if�the�irre��ular�sti�uli�are�irrelevant�unatten��e���events�(for�revie�s�see�Schrö��er�2007;�Win�ler�2007;�Win�ler�this�volu�e).�In�the��ajorit���of�stu��ies�MMN��as�investi��ate���in�the�passive oddball paradigm.�In� this��ara��i����re��ularit��� is�set�u��b��� i��entical� (or�al�ost� i��entical)�sti�uli�(stan��ar��),�an���the�re��ulation�is�violate���b���the��resentation�of�sti�uli��ith���if-ferent�characteristics�(e.��.���eviant��itch,�lou��ness,���uration,�etc.).�Such���eviant�sti�uli�are��resente���infrequentl��,�an���the�MMN�is�the���ifference�bet�een�the�ERPs� to� ��eviant� an��� re��ular� sti�uli.� The� �ara��i���� is� �assive,� because� in� the�t���ical�case�neither�the�stan��ar���nor�the���eviant�sti�uli�are�involve���into�the�on��oin���tas�.�Partici�ants�usuall���rea���interestin���boo�s,��la���vi��eo-��a�es,�or��erfor��various�tas�s.�

In�or��er�to�investi��ate�chan��e-relate���brain�electric��ithout�the�involve�ent�of�focal�attention�an���conscious�a�areness�in�vision,�the��ro�er��ara��i����has�to�be�si�ilar�to�the��assive�o����ball��ara��i����of�the�au��itor����o��alit��.�The�can��i-��ate�of���eviant-relate���ERP�co��onent��oul���be�the�visual��is�atch�ne��ativit���(vMMN).�In�fact,�occi�ital/occi�ito-te��oral�ne��ative��aves� to� tas�-irrelevant,�infrequentl����resente���(��eviant)�visual�sti�uli��ere�recor��e���as�earl���as�1990�an���1992�(Alho�et�al.�1992;�Czi��ler�an���Csibra�1990,�1992;�Woo��s�et�al.�1992).�Ho�-ever,�authors�of�these�stu��ie���notice���that�attentive��rocessin���of�the�irrelevant�sti�uli��ere�not� strictl���controlle��� in� these� stu��ies.�Later,� starte����ith�a� stu�����b���Tales�et�al.�(1999)�an���Heslenfel���(2003),�a�consi��erable�collection�of��a�ers�clai�e���to���e�onstrate�the�e�er��ence�of�ERP�co��onents�in��exin���auto�atic�(�reconscious,��reattentive)�chan��e���etection�(for�revie�s�see�Pazo-Alvarez�et�al.�2003;�Czi��ler�2007).�Fi��ure�1�sho�s�a�t���ical�vMMN.

In� o����ball� ex�eri�ents� vMMN� �as� elicite��� b��� various� ��eviant� sti�ulus�features,�li�e�color,�s�atial�frequenc��,�s�atial�contrast,��otion���irection,�sha�e,�line�orientation,� sti�ulus� location,� facial� ex�ression.8�In�or��er� to�consi��er� the�vMMN�(or�an���other�ERP�co��onents)�as�an�in��ex�of�i��licit��e�or��-relate���

Page 129: Unconscious Memory Representations in Perception

11�� István�Czi��ler

activit��,�t�o�i��ortant�issues�have�to�clarif��.�First,�in�o����ball�stu��ies�ERP���if-ference�bet�een�the�rare�(��eviant)�an���frequent�(stan��ar��)�sti�uli��a���e�er��e�fro��reasons�other�than�the��is�atch�bet�een�the��e�or���re�resentation�of�the�stan��ar���an��� the�re�resentation�of� the���eviant.�Frequent�sti�ulation��a���elicit�refractor���state�in�brain�structures;�therefore�res�onses�to�“fresh”�neural�substrate�to�the���eviant��a���elicit���ifferent�ERP�activit��.�Whereas�so�e�results�su��ort�such�inter�retation�(e.��.�Kene�ans�et�al.�2003;�Ki�ura�et�al.�2006),�in�stu��ies�I’ll�revie��the�refractoriness�inter�retation�is�ina��ro�riate�(for�research�s�ecificall��� investi��atin��� this� issue� see� Czi��ler� et� al.� 2002;� Czi��ler� et� al.� 2007;�Ki�ura�et�al.�2008b;�Pazo-Alvarez�et�al.�2004).�The�secon���issue�concerns�the�

Figure 1. Event-relate����otentials�an�����ifference��otentils�in�a��assive�visual�o����ball��ara��i���.�The�stan��ar��:��eviant�ratio��as�9:1.�Sti�uli��ere���ratin����atterns�of�lo��vs.�hi��h�s�atial�frequenc��.�In���ifferent�sequences�either�the�lo��or�the�hi��h�frequenc���sti�uli��ere�stan��ar���or���eviant.�Difference��otentials�(��eviant��inus�stan��ar��)��ere�calculate���for�the�sa�e�sti�uli�in���eviant�an���stan��ar���roles.�Grou��avera��e�of�12��artici�ants.�(Czi��ler,�un�ublishe�����ata.)

Page 130: Unconscious Memory Representations in Perception

� Re�resentation�of�re��ularities�in�visual�sti�ulation� 119

control�of�attention.�Unfortunatel���there�are�onl���fe��stu��ies��here�the��artici-�ants��ere�intervie�e����hether�the���ha���notice���the�vMMN-relate���sti�ulus�chan��e.�Ho�ever,�in�these�stu��ies�(Win�ler�et�al.�2005;�Czi��ler�an���Pató�2009)�vMMN��as�fairl���si�ilar�to�the�results�of�other�stu��ies�in�the�literature.

Hereinafter�those�vMMN�results��ill�be���iscusse���that���o�be��on���the�si��le�“rare�feature�versus���ifferent�feature”�o����ball��ara��i���,�sho�in���that�the�i��licit��e�or���s��ste��is��ore�co��lex�than�a�si��le�chan��e-��etection���evice.9�This�is�a�s��ste��ca�able�of�re��isterin���re��ularities�an���the�violation�of�re��ularities.

4.3.1 Visual��is�atch�ne��ativit���to�feature�conjunction��� (Win�ler,�Czi��ler,�Suss�an,�Horváth�an���Balázs�2005)

It�is�truis��that��e��erceive�objects�an���scenes,�not�isolate���features�li�e�color,�contour,� location,� etc.�The�role�of� attentive��rocesses� in�bin��in��� sti�ulus� fea-tures�is���ebate��.��So�e��revalent�theories�e��hasize�the�role�of�attention�(e.��.�Treis�an�an���Gela��e�1980;�Rensin��2000;�Quinlan�2003;�Wolfe�1994),��hereas�object-relate���theories�of�attention�e��hasize�the��ri�ac���of�object�for�ation�(an���accor��in��l���the�auto�atic�conjunction�of�features;�e.��.�Duncan�1984).�In�a�vMMN�stu������e�investi��ate���the��ossibilit���of�auto�atic�conjunction�of�t�o�fea-tures:�line�orientation�an���color.�Colore���(re��/blac��or�blue/blac�)���ratin����at-terns�(horizontal�or�vertical)��ere�constructe��.�T�o�conjunctions�(e.��.�blue-ver-tical�an���re��-horizontal)��ere�frequent��e�bers�of�a�sti�ulus�sequence,��hereas�the�other�t�o�conjunctions��ere�rare.�The�sti�ulus�sequence��as��resente���on�the�u��er�an���lover��art�of�a�screen�as�tas�-irrelevant�sti�uli.�Sti�ulus���uration��as� 17� �s,� an��� the� inter-sti�ulus� interval� �as� 350–450� �s.� Partici�ants� fix-ate���on�the�center�of�a���ar��stri�e,�an���atten��e���to�a�cross�in�this�location.�Fro��ti�e�to�ti�e�the�cross��as��a��e��i��er�or�lon��er.�Partici�ants�ha���to�in��icate�the�un�re��ictable�chan��e�of�the�cross.�ERPs��ere�co��are���to�the�rare�an���frequent�color/orientation�conjunctions.�ERP���ifference�e�er��e���as�a��osterior�ne��ativit����ith�128��s��ea��latenc���(vMMN),�follo�e���b���a�s�all��osterior��ositive��ave��ith�188��s� latenc��.�Havin���co��lete���this��art�of� the�stu������e�con��ucte���a�reco��nition� tas�.� Gratin��s� �ere� �resente��� an��� the� �artici�ants� �ere� as�e��� to�choose�the�frequent�ones.�Perfor�ance��as�at�chance�level.�In�the�secon����art�of�the�ex�eri�ental�session�the���ratin����atterns�beca�e�tas��relevant.�Partici�ants�ha���to���etect�one�of�the�t�o�rare�color-��irection�conjunctions�(tar��et�sti�ulus).�Gratin����atterns�elicite���characteristic�attention-relate���ERP�co��onents,�an���fro��the�recor��s��e��ere�able�to�reconstruct�also�the�vMMN.10�

The��ain�fin��in���of�this�stu�����is�the�i��entification�of�a��e�or���s��ste��ca-�able�of�re��isterin���the��robabilit���relationshi�s�a�on���the���ratin����atterns,�even�

Page 131: Unconscious Memory Representations in Perception

120� István�Czi��ler

if� there� is�no�conscious�ex�erience�on� such� relationshi�s.�The��e�or��� s��ste��is� o�eratin��� on� a� level� above� feature� bin��in��,� because� in� the� stu����� all� features�ha���equal��robabilities.�The�results� in��icate�the��ossibilit���of�auto�atic�feature�bin��in��,�at�least�in�case�of�a�relativel���si��le�visual�environ�ent.�Further�ore,�accor��in���to�the�results,�this��e�or���is�ca�able�of�storin����ore�than�one�re�re-sentation�(there��ere�t�o�frequent�conjunctions).

4.3.2 The�stora��e�un��erl��in���vMMN�is�not�a��assive�iconic��e�or����� (Czi��ler,�Weisz�an���Win�ler�2007)

In�the��ajorit���of�vMMN�stu��ies�the�inter-sti�ulus�interval��as�short,��ithin�the�ran��e�of�300–1200��s.�This���uration�is��ithin�the�life-ti�e�of�the�iconic��e�or��.�Sensitivit��� to�bac��ar���visual��as�in��� is�one�of� the���efinin���characteristics�of�iconic��e�or��.�In�this�stu�����both�the�stan��ar���an���the���eviant�sti�uli��ere�fol-lo�e���b����atterne����as�s.�We�varie���the�test-�as��interval.�In�case�of�a��as�-able��e�or���no�vMMN�is�ex�ecte���to�e�er��e�at�an���test-�as��of�the�intervals,�because�the�a��earance�of�a��as��bet�een�the��resentations�of�the�t�o�sti�uli���estro��s�the��e�or���content�of�the�“icon”.�In�case�of���evelo��ent�of�a�non-�as�-able��e�or��,�no�vMMN�is�ex�ecte���belo��the�critical�sti�ulus-res�onse�as��n-chron���(SOA),�an���vMMN�e�er��ence�is�ex�ecte���above�the�critical�SOA.�Fur-ther�ore,�the�value�of�the�critical�SOA��oul���in��icate�the���uration�necessar���for�the���evelo��ent�of�the��e�or���s��ste��un��erl��in���vMMN.

VMMN��as�investi��ate���to���reen/blac��an���re��/blac��equilu�inant�chec�-erboar��s�(test�sti�uli).�One�of�the�chec�erboar���a��eare���frequentl����ithin�the�sequence� (stan��ar��),� the� other� a��eare��� infrequentl��� (��eviant).� These� sti�uli��ere�follo�e���b����as�s�(ran��o��arran��e�ents�of�re���an�����reen�hexa��ons).�Both�test� an��� �as�� ��urations� �ere� 14� �s.� In� t�o� ex�eri�ents� the� SOA� �as� varie���bet�een�14�an���174��s.�Both�test�an����as��sti�uli��ere�irrelevant.�Partici�ants�atten��e���the�center�of�the�screen�an����erfor�e���a�tas��si�ilar�to�that�of�the��revi-ous�(feature�conjunction)�stu����.�In�case�of�short�SOA�(14�an���27��s)�no�vMMN�e�er��e��.�Ho�ever,�at�lon��er�SOAs��e�recor��e���a��osterior�ne��ativit���(vMMN)��ith�~130��s��ea��latenc��,�follo�e���b���a��osterior��ositive�co��onent.�VMMN�a��litu��es��ere�si�ilar�in�the�40�to�174��s�SOA�ran��e.�These�results��ere�fairl���si�ilar�to�those�of�the Win�ler�et�al.�(2005)�stu����.�In�se�arate�sequences��e�in-vesti��ate���the���etection��erfor�ance�un��er�si�ilar�con��itions.�Partici�ants�re-s�on��e���to�the�a��earance�of�the���eviant�chec�erboar��s.�At�14��s�SOA���etection��erfor�ance��as�at�chance�level.�Perfor�ance�increase���at�67��s�SOA.�At�174��s�SOA���etection��as�al�ost��erfect.

Page 132: Unconscious Memory Representations in Perception

� Re�resentation�of�re��ularities�in�visual�sti�ulation� 121

As�these�results�sho�,�a��as�,�above�a�critical�SOA�value���oes�not��revent�vMMN� e�er��ence.� Accor��in��l��,� the� �e�or��� un��erl��in��� vMMN� is� ��ifferent�fro��the��as�able�iconic�stora��e.�Ho�ever,�in�or��er�to���evelo��such�re�resenta-tion,�~30–40��s�is�nee��e��.�This�interval�is�co��arable�to�the���uration�necessar���for�the���evelo��ent�of�a��e�or���that�allo�s�cate��orical���ecision�(Kovacs�et�al.�1995).�Ho�ever,�on�the�level�of�ex�licit���iscri�ination��e�obtaine����artial�bac�-�ar����as�in���effects�even�at�hi��her�values�of�SOA.

4.3.3 Sequential�rules�an���vMMN�(Czi��ler,�Weisz�an���Win�ler�2006)

The��e�or���s��ste��un��erl��in��� the� (au��itor��)�MMN�is� sensitive� to� sequential�rules.�As�an�exa��le,�a�re�eate���sti�ulus��ithin�a�sequence�of�alternatin���sti�-uli�elicits�the�MMN�(e.��.�Horváth�et�al.�2001).�In�the�Czi��ler�et�al.�(2006)�stu�����re��ular�sequences��ere�constructe���fro��isolu�inant���reen/blac��an���re��/blac��chec�erboar��s�in�an�AABBAABB…�or��er.�Infrequentl���an�irre��ular�thir���i��en-tical� sti�ulus� �as� �resente��� (AABBAABBAAABB…).� Accor��in��l��,� sti�ulus�chan��e�per se��as�contraste���to�the�violation�of�re��ularit��.�The�tas���as�si�ilar�to� those�of�our��revious�stu��ies.�There��ere� t�o�ERPs�of� interest� in� this��ara-��i���,�ERP�to�the�irre��ular�re�etition�an���ERP�to�the�re��ular�chan��e.�These�ERPs��ere� co��are��� to� the� ERPs� to� re��ular� re�etitions.� Irre��ular� re�etition� elicite���lar��er�ne��ativit���as�earl���as�100–140��s,�an���a�further�ne��ative�co��onent�in�the�220–260��s�e�och.�Althou��h�the�latenc���of�the�latter�co��onent�is�lon��er�than�the�vMMNs�of�the��revious�stu��ies,�the���istribution��as�clearl����osterior,�an���the�latenc���is��ithin�the�latenc���ran��e�of�so�e�vMMN�stu��ies�(e.��.�Tales�et�al.�1999).�Re��ular�chan��e,�ho�ever,�elicite���no�vMMN.�Instea��,�ERP�to�such�chan��es��ere��ositive�in�relation�to�the�re��ular�re�etition.�It�is��ossible�that�such�ERP���ifference�is�relate���to�the�“chan��e��ositivit��”,�re�orte���b���Ki�ura�(Ki�ura�et�al.�2005,�2006,�2008a,�b).�The�ne��ativit���to�a�sti�ulus�re�etition�cannot�be�ex�laine���on�the�basis�of�chan��in����h��sical�a��earance,�or�as�the�refractoriness�of�an�other�ise�ne��a-tive�co��onent�e�er��in���in�this�ran��e.�As�a�conclusion,�vMMN,�li�e�its�au��itor���ho�olo��,� is� sensitive� to� the�violation�of� sequential� re��ularities,�not�onl��� to� the��h��sical�chan��e�of�sti�ulation.

4.3.4 I��licit�chan��e���etection�an���vMMN�(Czi��ler�an���Pató�2009)

To�use�vMMN�as�a�tool�for�investi��atin���the��ossibilit���of�i��licit�re��istration�of�a�chan��in���scene,�the��etho���has�to�acco��lish�t�o�require�ents.�First,�(as�the��Czi��ler�et�al.�2006�stu�����in��icate��) the�chan��e�has�to�violate�the�re��ularit���of�sti�u-lation.�Secon��,�a�careful�test�on�the�lac��of�conscious�chan��e���etection�is�nee��e��.�

Page 133: Unconscious Memory Representations in Perception

122� István�Czi��ler

In� or��er� to� establish� re��ularit��� �e� use��� a� “rovin��� stan��ar��”� �roce��ure.� In�this��roce��ure�a�sequence�of�i��entical�sti�uli�are��resente���(in�the��resent�stu�����the�len��th�of�the�sequence�varie���ran��o�l����ithin�the�ran��e�of�10–15�sti�uli);�thereafter�in�a�ne��sequence�a�ne��sti�ulus�t���e�is��resente��.�Such�c��cle�can�be�re�eate���several� ti�es�(18–22�in�the��resent�stu����).�The�sti�uli��ere���ri����at-terns�(either���reen�on�re���bac���roun���or�vice versa).�In�the�lo�er�half�of�the�visual�fiel���the�rectan��le�ele�ents�of�the���ri��s��ere�either�horizontal�or�vertical��ithin�a�sequence.�In�the�u��er�half-fiel���there��as�no�sti�ulus�chan��e.�The���ri����at-tern��as�irrelevant.�Partici�ants�res�on��e���to�the�size�chan��e�of�a�s�all�rectan��le��ithin�the�center�of�the�screen.�At�the�half-ti�e�of�the�ex�eri�ental�session��e�intro��uce���a�se�i-structure���intervie�.�If�there��as�the�s�allest�in��ication�that�a��artici�ant���etecte���the�chan��in����attern�in�the�lo�er�half�fiel��,�her/his���ata��ere�o�itte���fro��further��rocessin���an���the�session��as�ter�inate��.�Fro��the�initial�17��artici�ants�onl���3�re�orte���that�the���ri��s��ere�not�i��entical�at�all��resenta-tions�(no�one�re�orte���the�real�sti�ulus�chan��e).�

Before�the�secon����art�of�the�session�the�“rovin���stan��ar��”�construction��as�ex�laine���an�����e�onstrate��.�Partici�ants��ere�instructe���that�the�tas��re�aine���the�sa�e,�but�the���are�free�to�observe�the�chan��es�of�the��attern.

In�s�ite�of�the�lac��of�an�����etecte���chan��e�in�the�first��art�of�the�session,�in�the�250–400��s�latenc���ran��e�the�sti�uli�in�the�chan��e��osition�elicite���a�ri��ht��osterior� ne��ative� shift,� in� co��arison� to� the� ERPs� to� the� re��ular� sti�uli� (the�fifth�i��entical�sti�ulus��ithin�the�sequence).�Mar�e��l�����ifferent�chan��e-relate���ERP�effects�e�er��e���in�the�secon����art�of�the�session.�In�si�ilar�co��arisons��e�obtaine���t�o�ne��ativities.�The�earlier�e�er��e���in�the�205–220��s�latenc���ran��e�as��i��e�ne��ativit���over�the�ri��ht�he�is�heres,��hile�the�later�one�(in�the�305–330��s�latenc���ran��e)��as�a�bilateral��osterior�ne��ativit��.�Accor��in��l��,�ERP�correlates�of�conscious���etection��ere�not�onl���stron��er,�but�the����ere�qualitativel�����ifferent�fro��the�effects�of�i��licit�re��istration�of�sti�ulus�chan��e.

As� a� control� con��ition� �e� intro��uce��� re��ular,� AABBAABB…� �resentation�of�the�alternative���ri����atterns.�This�case�there��as�no��artici�ant��ho�re�orte���an���re��ularit����ithin�the�sti�ulation.�Co��arin���the�ERPs�to�the�chan��in���an���re�eatin���sti�uli,��e�obtaine���no�EPR�effects.

As�the�results�of�these�ex�eri�ents�sho�,�in�or��er�to�elicit�vMMN-li�e�ERP�effects,�it�is�necessar���to�establish�the�re�resentation�of�environ�ental�re��ularit��,�an���violate�the�re��ularit��.

Page 134: Unconscious Memory Representations in Perception

� Re�resentation�of�re��ularities�in�visual�sti�ulation� 123

4.4 Functional significance of implicit registration of regularities in vision

Havin�����e�onstrate���the��ossibilit���of�a��e�or���s��ste��ca�able�of�re��isterin���re��ularities�in�the�unatten��e���sti�ulus�bac���roun���an���reactin���to�the�violation�of�such�re��ularities,�the�question�is�obvious:�“Wh�����o��e�nee���such�a�s��ste�?”�Unli�e�the�transient�au��itor����or��,�visual�scenes�are�su��ose���to�be�continuousl���available,�as�“�e�or���outsi��e”�(O’Re��an�an���Noë�2001).�Ho�ever,�even�in�case�of�a�stea�����scene�(li�e�a��aintin��)�an���stron���fixation,�ones��er�three�secon��s�on�the�avera��e�e��e-blin�s�a��eare.�Duration�of�an�e��e-blin��is�sur�risin��l���lon��,���ar�-ness�lasts��ore�than�200��s�(e.��.�Caffier�et�al.�2005;�Casse�et�al.�2007).�Further-�ore,�in�case�of�nor�al�con��itions,�100��s��erio��s��ithout�sti�ulus��rocessin���(sacca��ic�su��ression)�occur�at�each�sacca��ic�e��e��ove�ent�(e.��.�Dia�on���et�al.�2000).�As�a�conservative�esti�ation,��e�s�en���7–10%�of�our�total�a�a�e����erio���in���ar�ness.�In�the��erio��s�of���ar�ness�‘local�transients’�cannot�cue�chan��es.�In�a�rich�visual�environ�ent�a�li�ite���ca�acit����e�or����oul���be�unable�to�follo��all��otentiall���i��ortant�chan��es.�Accor��in��l��,�in�or��er�to���etect��otentiall���i��or-tant�ne��objects/events�a�lar��e�ca�acit���visual��e�or��,�not�unli�e�the�au��itor����e�or��,�has�evolutional�a��vanta��e�in�servin���the�orientin���s��ste�.�Ho�ever,�the�o�eration�of�a�visual��e�or�����oes�not��ean�that��e�have�to�be�a�are�of�the�re-sults�of�its�o�erations�(e.��.�consciousl�����etect�the�violate���re��ularities).�On�one�han��,�a��heno�enon�li�e�chan��e�blin��ness�cannot���is�roof�the��ossibilit���of�i�-�licit���etection.�On�the�other�han��,�in��irect��easures�(�ri�in���effects�an���ERP���ata)��a���in��icate�the�contribution�of�the�i��licit�s��ste�.

In�a�recent��a�er�Bar��h�an���Morsella�(2008)���iscusse���the�contrast�bet�een�a���u�b�an���a�s�art�unconscious��in���(Loftus�an���Klin��er�1992).�Base���on�re-search�on�subli�inal,��as�e��,�etc.�(i.e.,�ill-�erceive��)�sti�ulation,�effects�of�non-conscious��rocesses� see�s� to�be�rather� li�ite��.�On�the�contrar��,�as�Bar��h�an���Morsella�(2008)�ar��ue,�the��resence�of�so�e�events�auto�aticall���activates�re�-resentations�an���res�onse�ten��encies�(contextual��ri�in��).�In�this�sense�uncon-sciousness�is�a�so�histicate��,�flexible,�an���a��a�tive�behavior���ui��ance�s��ste�.�The�s��ste����isclose���b���the�vMMN�research�is�“�o��eratel���s�art”.�Li�e�in�the�au��i-tor����o��alit��,�this�s��ste��is�a�“�ri�itive�intelli��ence”�(Näätänen�et�al.�2001),�ca-�able�of�re��isterin���not�onl���basic�visual�features�li�e�colour,�s�atial�frequenc��,���i-rection,�contrast,�sha�e,��ove�ent���irection,�but�also�the�conjunction�of�features,�object-relate��� chan��es� an��� te��oral� re��ularities.� Ho�ever,� there� are� no� ��irect�evi��ences�about� the�a��a�tive� functions�of� this��e�or��.�As�have��e�su����este��,�violation�of� re��ularities��a���call� for�orientation��rocesses� (attentional� ca�ture,�increase���autono�ic�activation).

Page 135: Unconscious Memory Representations in Perception

124� István�Czi��ler

The� s��ste�� un��erl��in��� au��itor��� �is�atch� ne��ativit��� is� su��ose��� to� serve�antici�ator���functions�(Win�ler�2007,�this�volu�e).�Re��istration�of�bac���roun���activit���is�necessar���for�veri��ical�au��itor����erce�tion.�A��re��ictive��o��el�of�the�re��ularities�of�acoustic�bac���roun���is�an�effective�tool�for�the�co��utation�of�the���ifference�bet�een�the�actual�net�sti�ulation�an���the�estimated�bac���roun���ac-tivit���(Win�ler�2007).�Veri��ical�visual��erce�tion�also�requires�the�re��istration�of�bac���roun���sti�ulation.�A�si��le�exa��le� is� the� li��htness�constanc��.� In�or��er�to��erceive�the�li��htness�(or�colour)�of�an�object,11�the��erce�tual�s��ste��has�to�consi��er�the���eneral�level�of�illu�ination.�Since�characteristics�or�the�bac���roun���illu�ination�an���the�bri��htness�of�an���objects�are�available�si�ultaneousl��,�one��a���sa��,�that�there�is�no�nee���of�re��isterin���either�the�re��ularities�of�bac���roun���illu�ination�or�the�intrinsic�li��htness��ro�erties�(reflectance)�of�the�objects�in�a��e�or���s��ste�.�A�art�fro��theoretical��ositions�(see�Pal�er�1999,���.�126–133�for�a�su��ar��),�the��o��els�of�li��htness�constanc���assu�e�the�co��utation�of�a�s�ace-avera��e���reflectance�value.�Co��utation�involves�both�fast�an���slo���roc-esses�(Shi�oza�i�et�al.�2001).�The�latter�is�calculate���across�sacca��ic��ove�ents�(Cornelissen�an���Brenner�1995),�an���su��ose��l��,�across�e��e-blin�s.�Calculation�of� reflectance� is� a� laborious� co��utation� (�articularl��� in� case� of� coloure��� ob-jects).�Therefore�re��istration�of�initial�values�an���results�of��revious�co��utations��oul���be�of�consi��erable�benefit.�This�initial�value�can�be�consi��ere���a��art�of�the�object�file�(Shi�oza�i�et�al.�1999).�Sti�ulus�bac���roun����a���chan��e���urin���the��erio��s�of� the�absence�of� infor�ation�u�ta�e�(e.��.�e��e-blin�,�sacca��ic�su��res-sion).�Accor��in��l��,�an�i��licit��e�or���s��ste��re��isterin���the�re��ularities�of�the�bac���roun���sti�ulation,�an���reactin���to�the�violation�of�the�re��ularities�has�real�a��a�tive�value.�

Recent�results�(Alvarez�an���Oliva�2008)�sho��the�o�eration�of�a��e�or���s��s-te��ca�able�of�storin���ense�bles�of�tas�-irrelevant�visual�features.�In�a��ulti�le�trac�in���tas��the��artici�ants�follo�e���the�trajector���of�four���ots,��hereas�four�other���ots��ere�irrelevant�in�the�trac�in���tas�.�All���ots���isa��eare���for�a�short��e-rio���of�ti�e.�After�ar��s�several���ots�rea��eare��.�In�one�of�the�con��ition�the��ar-tici�ants�in��icate���the�location�of�the�sole��issin�����ot�(atten��e���or�unatten��e��),�an���in�the�other�con��ition�the���in��icate���the�centroi���of�four�(either�the�atten��e���or�the�unatten��e��)��issin�����ots.�As�the�results�sho�,��erfor�ance��as��oor��hen�the� tas��require��� the� location�of�an�unatten��e�����ot,��hereas��erfor�ance��as��ell�above�chance� level��hen� the� tas��concerne��� the� i��entification�of� the�cen-troi��� of� the� unatten��e��� four� ��ots.� This� result� sho�s� that� the� re�resentation� of�statistical/holistic��ro�erties�of�the�visual�fiel�����oes�not�require�focal�attentional��rocesses.�This�fin��in���fits�to�our�su����estion�about�a��e�or���s��ste��ca�able�of�storin���the�re��ularities�of�the�bac���roun���sti�ulation,�an���su��ortin���our�s�ecu-lations�about�the�role�of�such��e�or���s��ste��in��erce�tual�constancies.12

Page 136: Unconscious Memory Representations in Perception

� Re�resentation�of�re��ularities�in�visual�sti�ulation� 125

Li�e� in� the�au��itor����o��alit��,�u���atin���of� the�content�of� i��licit��e�or���s��ste�s�can�be�consi��ere���as�an�event���ivin���rise�to�recor��able�brain�activit��,�i.e.�the�vMMN.�Research�on�the�involve�ent�of�an�i��licit��e�or���s��ste��in��er-ce�tual�constancies�is�a�to�ic�of�further�research.�

Acknowledgements

Su��orte���b��� the�National�Research�Foun���of�Hun��ar��� (OTKA�–�K716000).� I�than��for�László�Balázs,�Lívia�Pató,�István�Wi�ler�an���Júlia�Weisz�for�their�hel�.

Notes

1. The�ter�inolo�����in�this���escri�tion�is���ifferent�fro��a�frequentl���use���one.�La��e�(2003)�(see�also�Bloc��1996)��a��e�a���istinction�bet�een�“�heno�enal”�an���“access”�consciousness.�Access�consciousness�is�available�for�re�ort,��hereas��heno�enal�consciousness�is�an�inter�e-��iate�state,�e�er��in���as�a�consequence�of�fee��-for�ar���activation�in�the�hierarch���of�brain�areas�involve��� in� sti�ulus� �rocessin��� an��� the� activation� of� fee��-bac�� loo�s.� Co��etition� a�on���these�brain�areas�(attention)�is�su��ose���to�select�activation�un��erl��in���“access”�a�areness.�In�the��resent��a�er�the�ter��“consciousness”�is�use���in�sense�of�“access�consciousness”.�We�su�-�ose�that�re�resentation�un��erl��in���‘�heno�enal�consciousness’�has�the�ca�acit���of��ro��ucin���i��licit�chan��e���etection.�We�consi��er�this�re�resentation�as�unconscious.�In�the��resent��a�er�the�ter��‘unconscious’,�is�si�ilar�to�the�ter��‘ineffectivel���unconscious’�La��e’s�(2003).�A�art�fro��the���ifferent�usa��e�of�so�e�ter�s��e�share�the�vie��about�the�substantial�role�of�fee��bac���echanis�s�in�consciousness,�as�La��e�(2003)�su����este���it.�2. The�effectiveness�of�contextual�cuin���in�visual�search�is���ebate���(see�e.��.�Kunar�et�al.�2007).�Ho�ever,�anal��sis�of�the��rocesses�un��erl��in���visual�search��erfor�ance�is�be��on���the�sco�e�of�this��a�er.3. It�shoul���be�note���that�Johnson,�Hollin���orth�an���Luc��(2008)�obtaine���evi��ences�sho�-in���that�an�attention-��e�an��in���tas��ha���no�lar��er�interference�on���etection�of�conjunction�chan��e� than�the�effect�of�such�tas��on� feature�search.�Such�result�contra��icts� to� the�nee���of�sustaine���attention�in��reservin���feature�conjunctions�(Wheeler�an���Treis�an�2002).4. Note�that�in�this�res�ect�this��o��el�is���ifferent�fro��an�account��ro�ose���b���Kahne�an�an���Treis�an�(1992).�The�object file�in�the�latter��o��el��reserves�conjunction�of�features�fol-lo�in���a�chan��e�of�the�attentional�fiel��.5. See�Verle��er�(this�volu�e)�for�a�co��rehensiove���iscussion�of�the�attentional�blin���he-no�enon�an���the�results�of�event-relate����otential�stu��ies�usin���this��heno�enon.��. In�this�cha�ter��e�concentrate�on�effects�of�non-conscious�sti�ulation.�The�other�si��e�of�the�coin,�the�event-relate����otential�correlates�of�conscious��rocessin���is���iscusse���in�the�cha�-ter�b���Rolf�Verle��er.7. A� ��etaile��� revie�� on� the� neural� correlates� of� consciousness� is� be��on��� the� sco�e� of� this��a�er.�For�a�revie��see�Dehane�an���Naccache�(2001).

Page 137: Unconscious Memory Representations in Perception

12�� István�Czi��ler

�. VMMN�is�so�eti�es��rece��e���b���a��osterior��ositivit���(‘chan��e-relate����ositivit��’�(CRP);�e.��.� Ki�ura� et� al.� 2005,� 2006,� in� �ress).� Accor��in��� to� the� results� fro�� this� laborator��,� CRP�e�er��es���ifferent��in��s�of�sti�ulus�chan��e.�Ho�ever,�usin���o����ball��ara��i���,�fro����ifferent�laboratories�onl���one�stu�����re�orte���CRP-li�e�activit��� (Fonteneau�an���Davi��off�2007).�Fur-ther�ore,�attention-in��e�en��ence�of�this�co��onent�has�not�been�seriousl���investi��ate��.9. Onl���one��ublishe���stu�����atte��te���to�use�vMMN�for�investi��atin���non-conscious�re�re-sentation�the��re-chan��e���is�la��s�in�chan��e�blin��ness�situation.�Hen��erson�an���Orbach�(2006)��resente���arra��s�of�six��atches.�The�secon����e�ber�of�a�sti�ulus��air��ere�either�i��entical�to�the�first,�or�one�of�the��atches�chan��e���orientation.�The�tas���as�a�non-s�ee��e���“sa�e-��iffer-ent”���etection.�Throu��hout�the�session�the�sa�e��attern��as��resente���in�300�consecutive�tri-als,�an���in�the�next�300�trials��atches��ith�ortho��onal�orientation��ere��resente��.�The�location��ith�chan��in���orientation��as�correctl���cue��,��iscue���or�uncue��.�Chan��e�trials�elicite���a��os-terior�ne��ativit���in�the�180–320��s�latenc���ran��e.�This�ne��ativit���e�er��e���even�in�the��iscuin���con��ition.�In�the��iscuin���Detection��erfor�ance��as�onl���59.5%.�Ho�ever,�in�chan��e�trials�in�the�absence�of���etecte���chan��e�no�such��osterior�ne��ativit���e�er��e��.�These�results�are�not��ithout�controversies.�First,�even�if�the�sa�e�sti�uli��ere��resente���in�a�series�of�trials�as�the�first��e�ber�of�the��air,�the�nu�ber�of���ifferent�sti�uli��as�relativel���hi��h�(25%).�Therefore�the� chance� for� a� ��etecte��� re��ularit��� �as� relativel��� lo�.� Secon��,� even� if� �is�atch� ne��ativit���(both�visual�an���au��itor��)�is�consi��ere���to�be�elicite���b���unatten��e���sti�uli,�it�is��ossible�that�attentional��rocesses��o��ulate� the�co��onent�(as�a��ell-�no�n�exa��le� fro��the�au��itor����o��alit���see�Wol��orff�et�al.�1991).�Thir��,�as�su����este���also�b���the�authors,�the�chan��e��i��ht�be�not�lar��e�enou��h�to�elicit�the�vMMN.�10. The� ��ifference� bet�een� the� ERP� to� the� rare� conjuncion� �ithout� tas�-relate��� featues��(Color–/Direction–)�an���avera��e�of�the�ERPs�to�sti�uli��ith�on�tar��et-relate���feature�(Col-or+/Direction–�an���Color+/Direction+).11. Li��htness�refers�to�the�intrinsic��ro�ert���of�an�object,�the��ro�ortion�of�reflecte���an���ab-sorbe��� li��ht,� �hereas� bri��htness� refers� to� the� value� of� li��ht� reflecte��� b��� the� object,� i.e.,� it� is���e�en��ent�on�illu�inance�an���reflectance.12. An�obvious���ifference�is�the�conscious�access�to�ense�ble�features�in�the�Alvarez�an���Oliva�(2008)� stu�����an��� the�non-conscious� re�resentation�as���isclose���b��� the�vMMN�results.�Fur-ther�ore,�a���ue�to�the���ual-tas��nature�of�this�tas�,�the�non-trac�e�����ots��ere�not�full���unat-ten��e��.

References

Alho,�K.,�Woo��s,�D.�L.,�Al��azi,�A.�an���Näätänen,�R.�(1992).�Inter�o��al�selective�attention�2.�Effects�of�attentional� loa���on��rocessin���of�au��itor���an���visual�sti�uli� in�central�s�ace.�Electroencephylography and Clinical Neurophysiology,�82,�356–368.�

Alvarez,�G.�A.�an���Oliva,�A.�(2008).�The�re�resentation�of�si��le�ense�ble�visual�features�out-si��e�the�focus�of�attention.�Psychological Science,�19,�392–398.

Bar��h,�J.�A.�an���Morsella,�E.�(2008).�The�unconscious��in��.�Perspectives on Psychological Sci-ence,�3,�73–79.

Bec�,�D.�M.,�Rees,�G.,�Frith,�C.�D.�an���Lavie,�N.�(2001).�Neural�correlates�of�chan��e���etection�an���chan��e�blin��ness.�Nature Neuroscience,�4,�645–650.�

Page 138: Unconscious Memory Representations in Perception

� Re�resentation�of�re��ularities�in�visual�sti�ulation� 127

Bec�er,�M.�W.,�Pashler,�H.�an���Anstis,�S.�M.�(2000).�The�role�of�iconic��e�or���in�chan��e���etec-tion�tas�s.�Perception,�29,�273–286.

Bloc�,�N.�(2007).�Consciousness,�accessibilit��,�an���the��esh�bet�een��s��cholo�����an���neuro-science.�Behavioral and Brain Sciences,�30,�481–499.

Caffier,�P.�P.,�Er���ann,�U.�an���Ulls�er��er,�P.�(2005).�The�s�ontaneous�e��e-blin��as�slee�iness�in��icator�in��atients��ith�obstructive�slee��a�noe�s��n��ro�e:�A��ilot�stu����.�Sleep Medicine,�6,�155–162.

Casse,�G.,�Sauva��e,�J.�P.,�A��enis,�J.�P.�an���Robert,�P.�Y.�(2007).�Vi��eon��sta���o��ra�h���to�assess�blin�in��.�Graefes Archive for Clinical and Experimental Ophthalmology,�243,�1789–1796.

Coltheart,�M.�(1980).�Iconic��e�or���an���visible��ersistence.�Perception and Psychophysics,�27,�183–228.

Coltheart,�V.�(1999).�Intro��uction:�Perceivin���an���re�e�berin���brief�visual�sti�uli.�In�V.�Col-theart�(E��.),�Fleeting memories�(��.�1–12).�Ca�bri����e,�MA:�MIT�Press.

Cornelissen,�F.�W.�an���Brenner,�E.�(1995).�Si�ultaneous�colour�constanc���revisite��:�An�anal��sis�of�vie�in���strate��ies.�Vision Research,�35,�2431–2448.

Czi��ler,�I.�(2007).�Visual��is�atch�ne��ativit��:�Violatin���of�nonatten��e���environ�ental�re��u-larities.�Journal of Psychophysiology,�21,�224–230.

Czi��ler,�I.,�Balázs,�L.�an���Win�ler,�I.�(2002).�Me�or��-base�����etection�of�tas�-irrelevant�visual�chan��e.�Psychophysiology,�39,�869–873.

Czi��ler,�I.�an���Csibra,�G.�(1990).�Event-relate����otentials�in�a�visual���iscri�ination�tas�:�Ne��a-tive��aves�relate���to���etection�an�����iscri�ination.�Psychophysiology,�39,�869–873.

Czi��ler,�I.�an���Csibra,�G.�(1992).�Event-relate����otentials�an���the�i��entification�of���eviant�sti�-uli.�Psychophysiology,�29,�471–484.

Czi��ler,�I.�an���Pató,�L.�(2009).�Unnotice���re��ularit���violation�elicits�chan��e-relate���brain�acti-vit��.�Biological Psychology,�80,�339–347.

Czi��ler,� I.,� Weisz,� J.� an��� Win�ler,� I.� (2006).� ERPs� an��� ��eviance� ��etection:� Visual� �is�atch�ne��ativit���to�re�eate���visual�sti�uli.�Neuroscience Letters,�401,�178–182.

Czi��ler,�I.,�Weisz,�J.�an���Win�ler,�I.�(2007).�Bac��ar����as�in���an���visual��is�atch�ne��ativit��:�electro�h��siolo��ical�evi��ence�for��e�or��-base���auto�atic���etection�of���eviant�sti�uli.�Psychophysiology,�44,�610–619.

Dehaene,�S.,�Chan��eux,�J.�P.,�Naccache,�L.,�Sac�ur,�J.�an���Ser��ent,�C.�(2006).�Conscious,��re-conscious�an���subli�inal��rocessin��:�A�testable�taxono���.�Trends in Cognitive Sciences,�10,�204–211.

Dehane,�S.�an���Naccache,�L.�(2001).�To�ar��s�a�co��nitive�neuroscience�of�consciousness:�basic�evi��ence�an���a��or�s�ace�fra�e�or�.�Cognition,�79,�1–7.

Delor�e,� A.,� Rousselet,� G.� A.,� Mace,� M.� J.� M.� an��� Fabre-Thor�e,� M.� (2004).� Interaction� of�to�-��o�n�an���botto�-u���rocessin���in�the�fast�visual�anal��sis�of�natural�scenes.�Cognitive Brain Research,�19,�103–113.

DeSche��er,� B.� an��� Treis�an,� A.� (1996).� Visual� �e�or��� for� novel� sha�es:� I��licit� co��in����ithout�attention.�Journal of Experimental Psychology: Learning, Memory and Cognition,�22,�27–47.

Dia�on��,�M.�R.,�Ross,�J.�an���Morrone,�M.�C.�(2000).�Extraretinal�control�of�sacca��ic�su��res-sion.�Journal of Neuroscience,�20,�3449–3455.

Duncan,� J.� (1984).�Selective�attention�an��� the�or��anization�of�visual� infor�ation.� Journal of Experimental Psychology: General,�113,�501–517.

Page 139: Unconscious Memory Representations in Perception

12�� István�Czi��ler

Fernan��ez-Duque,�D.,�Grossi,�G.,�Thornton,�I.�M.�an���Neville,�H.�J.�(2003).�Re�resentation�of�chan��e:�Se�arate�electro�h��siolo��ical��ar�ers�of�attention,�a�areness,�an���i��licit��ro-cessin��.�Journal of Cognitive Neuroscience,�15,�491–507.

Fernan��ez-Duque,�D.�an���Thornton,� I.�M.� (2000).�Chan��e���etection��ithout�a�areness:�Do�ex�licit�re�orts�un��eresti�ate�the�re�resentation�of�chan��e�in�the�visual�s��ste�?�Visual Cognition,�7,�323–334.

Fernan��ez-Duque,�D.�an���Thornton,�I.�M.�(2003).�Ex�licit��echanis�s���o�not�account�for�i�-�licit�localization�an���i��entitication�of�chan��e:�An�e��irical�re�l���to�Mitroff�et�al.�(2002).�Journal of Experimental Psychology: Human Perception and Performance,�29,�846–857.

Hen��erson,�R.�M.�an���Orbach,�H.�S.�(2006).�Is�there��is�atch�ne��ativit�����urin���chan��e�blin��-ness?�NeuroReport,�17,�1011–1015.

Heslenfel��,� D.� J.� (2003).� Visual� �is�atch� ne��ativit��.� In� J.� Polich� (E��.),� Detection of change: Event-related potential and fMRI findings�(��.�41–59).�Boston:�Kluver�Aca��e�ic�Press.

Hollin���orth,�A.�(2006).�Scene�an����osition�s�ecificit���in�visual��e�or���for�objects.�Journal of Experimental Psychology: Learning Memory and Cognition,�32,�58–69.

Hollin���orth,�A.�an���Hen��erson,�J.�M.�(2002).�Accurate�visual��e�or���for��reviousl���atten��-e���objects� in�natural� scene.� Journal of Experimental Psychology: Human Perception and Performance,�28,�113–116.

Horváth,�J.,�Czi��ler,�I.,�Suss�an,�E.�an���Win�ler,�I.�(2001).�Si�ultaneousl���active��re-attentive�re�resentations�of�local�an�����lobal�rules�for�soun���sequences�in�hu�an�brain.�Cognitive Brain Research,�12,�131–144.

Intraub,�H.�(1999).�Un��erstan��in���an���re�e�berin���brief���li��se����ictures:�I��lications�for�visual�scannin���an����e�or��.�In�V.�Coltheart�(E��.),�Fleeting memories�(��.�47–70).�Ca�-bri����e,�MA:�MIT�Press.

Jian��,�Y.�H.�an���Leun��,�A.�W.�(2005).�I��licit�learnin���of�i��nore���visual�context.�Psychonomic Bulletin and Review,�12,�100–106.

Kahne�an,�D.,�Treis�an,�A.�an���Gibbs,�B.�J.�(1992).�The�revie�vin���of�object�files:�Object�s�e-cific�inte��ration�of�infor�ation.�Cognitive Psychology,�24,�175–219.

Kene�ans,�J.�L.,�Jon��,�T.�G.�an���Verbaten,�M.�N.�(2003).�Detection�of�visual�chan��e:�Mis�atch�or�rareness?�NeuroReport,�14,�1239–1242.

Ki�,�C.�Y.�an���Bla�e,�R.�(2005).�Ps��cho�h��sical��a��ic:�ren��erin���the�visible�‘invisible’.�Trends in Cognitive Sciences,�9,�381–388.

Ki�ura,�M.,�Kata��a�a,�J.�an���Murohashi,�H.�(2005).�Positive���ifference�in�ERPs�reflect�in��e-�en��ent��rocessin����f�visual�chan��es.�Psychophysiology,�42,�369–379.

Ki�ura,�M.,�Kata��a�a,�J.�an���Murohashi,�H.�(2006).�Probabilit��-in��e�en��ent�an���-��e�en��ent�ERPs�reflectin���visual�chan��e���etection.�Psychophysilogy,�43,�180–189.

Ki�ura,�M.,�Kata��a�a,�J.�an���Murohashi,�H.�(2008a).�Involve�ent�of��e�or��-co��arison-base���chan��e���etection�in�visual���istraction.�Psychophysiology,�45,�445–457.�

Ki�ura,�M.,�Kata��a�a,�J.�an���Murohashi,�H.�(2008b).�Implicit change detection: Evidence from event-related potential.�Poster��resente���at�the�29th�International�Con��ress�of�Ps��cholo����,�Berlin.

Ki�ura,�M.,�Kata��a�a,�J.�an���Ohira,�H.�(2008c).�Event-relate����otential�evi��ence�for�i��licit�chan��e� ��etection:� A� re�lication� of� Ferna��ez-Duque� et� al.� (2003).� Neurosceience Letters,�448,�236–239.�

Kirchner,�H.�an���Thor�e,�S.� J.� (2005).�Ultra-ra�i���object���etection��ith� sacca��ic�e��e��ove-�ents:�Visual��rocessin���se�����revisite��.�Vision Research,�46,�1762–1776.

Page 140: Unconscious Memory Representations in Perception

� Re�resentation�of�re��ularities�in�visual�sti�ulation� 129

Koivisto�M.�an���Revonsuo,�A.�(2003).�An�ERP�stu�����of�chan��e���etection,�chan��e�blin��ness,�an���visual�a�areness.�Psychophysiology,�40,�423–429.

Koivisto,�M.�an���Revonsuo,�A.�(2007).�Ho���eanin���sha�es�seein��.�Psychological Science,�18,�845–849.�

Koui��er,�S.�an���Dehaene,�S.�(2007).�Levels�of��rocessin�����urin���non-conscious��erce�tion:�a�critical�revie��of�visual��as�in��.�Philosophical Transactions of the Royal Academy of Sci-ences,�362,�857–875.

Kovacs,�G.,�Vo��els,�R.�an���Orban,�G.�A.�(1995).�Cortical�correlate�of��attern�bac��ar����as�in��.�Proceedings of the National Academy of Sciences, USA,�923,�5587–5591.

Kunar,�M.�A.,�Flusber��,�S.,�Horo�itz,�T.�S.� an���Wolfe,� J.�M.� (2007).�Does� contextual� cuein�����ui��e�the���e�lo���ent�of�attention?�Journal of Experimental Psychology: Human Perception and Performance,�33,�816–828.

Lalo��aux,�C.,�Destrebecqz,�A.�an���Cleere�ans,�A.�(2006).�In�licit�chan��e�i��entification:�A�re�-etition� of� Fernan��ez-Duque� an��� Thornton� (2003).� Journal of Experimental Psychology: Human Perception and Performance,�32,�1366–1397.

La��e,�V.�A.�F.�(2003).�Wh���visual�attention�an���a�areness�are���ifferent.�Trends in Cognitive Sciences,�7,�12–18.

Lan���an,� R.,� S�e�reijse,� H.,� an��� La��e,� A.� F.� (2003).� Lar��e� ca�acit��� stora��e� of� inte��rate���objects�before�chan��e�blin��ness.�Vision Research,�43,�149–164.

Li,�F.�F.,�VanRullen,�R.�an���Perona,�P.�(2002).�Ra�i���natural�scene�cate��orization�in�the�near�absence�of�attention.�Proceedings of the National Academy of Sciences of the United States of America,�99,�9596–9601.

Lleras,�A.,�Rensin�,�R.�A.�an���Enns,�J.�T.�(2005).�Ra�i���resu��tion�of�interru�te���visual�search.�Psychonomic Science,�16,�684–688.

Loftus,�E.�F.�an���Klin��er,�M.�R.�(1992).�Is�the�unconscious�s�art�or���u�b?�American Psycholo-gist,�47,�761–765.

Luc�,�S.�J.,�Vo��el,�E.�K.�an���Sha�iro,�K.�L.�(1996).�Wor����eanin���can�be�assesse���but�not�re-�orte�����urin���the�attentional�blin�.�Nature,�383,�616–618.

Macé,�M.�J.�M.,�Thor�e,�S.�J.�an���Fabre-Thor�e,�M.�(2005).�Ra�i���cate��orization�of�achro�atic�natural�scenes:�Ho��robust�at�ver���lo��contrasts?�European Journal of Neuroscience,�21,�2007–2018.

Mac�,�A.�an���Roc�,�I.�(1998).�Inattentional Blindness.�Ca�bri����e,�MA:�MIT�Press.�Melcher,�D.�(2001).�Persistence�for�visual��e�or���for�scenes.�Nature,�412,�401.Meri�le,�P.�M.,�S�ile�,�D.�an���East�oo��,�J.�D.�(2001).�Perce�tion��ithout�a�areness:�Pers�ec-

tives�fro��co��nitive��s��cholo����.�Cognition,�79,�115–134.Meril�e,�P.�M.�(1992).�Perce�tion��ithout�a�areness:�Critical�issues.�American Psychologist,�47,�

792–796.Mitroff,�S.�R.,�Si�ons�,�D.�J.�an���Franconeri,�S.�L.�(2002).�The�siren�son���of�i��licit�chan��e�

��etection.� Journal of Experimental Psychology: Human Perception and Performance,� 28,�798–815.

Mouchetant-Rostain��,�Y.,�Giar��,�M.�H.,�Bentin,�S.,�Au��era,�P.�A.�an���Pernier,�J.�(2000).�Neuro-�h��siolo��ical�correlates�of�face���en��er��rocessin���in�hu�ans.�European Journal of Neuro-science,�12,�303–310.

Müller,�D.,�Win�ler,�I.,�Roeber,�U.,�Schaffer,�S.,�Czi��ler,�I.�an���Schrö��er,�E.�(2008).�Pre-attentive�object�for�ation�is�reveale���b�����ifferential��rocessin���of���eviances�relate���to�sa�e�or���if-ferent�objects.�Perception,�37,�212–Su��l.,�S.

Page 141: Unconscious Memory Representations in Perception

130� István�Czi��ler

Näätänen,�R.,�Tervanie�i,�M.,�Suss�an,�E.,�Paavilainen,�P.�an���Win�ler,�I.�(2001).� ‘Pri�itive�intelli��ence’�in�the�au��itor���cortex.�Trends in Neurosciences,�24,�283–288.

Nie��e����en,�M.,�Wich�ann,�P.�an���Stoeri��,�P.�(2001).�Chan��e�blin��ness�an���ti�e�to�conscious-ness.�European Journal of Neuroscience,�14,�1719–1726.

Norvalis,�A.�an���Miller,�J.�(2008).�Unconscious���etection�of�chan��e:�Can��e���o��ithout�a�are-ness?�Personal�co��unication,�16th�A�ril,�2008.�

O’Re��an,�J.�K.�an���Noë,�A.�(2001).�A�sensori�otor�account�of�vision�an���visual�consciousness.�Behavioral and Brain Sciences,�24,�883–975.

Pal�er,�S.�E.�(1999).�Vision science.�Ca�bri����e,�MA:�MIT�Press.�Pazo-Alvarez,�P.,�A�ene��o,�E.�an���Ca��aveira,�F.�(2004).�Auto�atic���etection�of��otion���irec-

tion�chan��es�in�the�hu�an�brain.�European Journal of Neuroscience,�19,�1978–1986.Pazo-Alvarez,�P.,�Ca��aveira,�F.�an���A�en��eo,�E.�(2003).�MMN�in�the�visual��o��alit��:�A�revie�.�

Biological Psychology,�63,�199–236.Potter,�M.�C.�(1976).�Short-ter��conce�tual��e�or���for��ictures.�Journal of Experimental Psy-

chology: Human Learning and Memory,�2,�509–522.Quinlan,�P.�T.�(2003).�Visual�feature�inte��ration�theor��:�Past,��resent,�future.�Psychological Bul-

letin,�129,�643–673.Ra���on��,�J.�E.,�Sha�iro,�K.�L.�an���Arnell,�K.�M.�(1992).�Te��orar���su��ression�of�visual��ro-

cessin���in�an�RSVP�tas�:�An�attentional�blin�?�Journal of Experimental Psychology: Human Perception and Performance,�18,�849–860.

Rensin�,�R.�A.�(2000).�Seein��,�sensin���an���scrutinizin��.�Vision Research,�40,�1469–1487.Rensin�,�R.�A.�(2000).�The�����na�ic�re�resentation�of�scenes.�Visual Cognition,�7,�17–42.Rensin�,�R.�A.�(2002).�Internal vs. external information in visual perception.�Ha�thorne,�NY:�

S����osiu��on�s�art���ra�hics,�63–70.Rensin�,�R.�A.,�O’Re��an,�J.�K.�an���Clar�,�J.�J.�(2000).�On�the�failure�to���etect�chan��es�in�scenes�

across�brief�interru�tions.�Visual Cognition,�7,�127–145.Roc�,�I.�an���Gut�an,�D.�(1981).�The�effect�of�inattention�an���for���erce�tion.�Journal of Ex-

perimental Psychology: Human Perception and Performance,�7,�275–282.Rol�e,�B.,�Heil,�M.,�Streb,�J.�an���Hennin��hausen,�E.�(2001).�Misse����ro�e��or��s��ithin�the�at-

tentional�blin��evo�e�an�N400�se�antic��ri�in���effect.�Psychophysiology,�38,�165–174.Schen��an,�H.�E.,�Ganis,�G.�an���Kutas,�M.�(1998).�Neuro�h��siolo��ical�evi��ence�for�visual��er-

ce�tual�cate��orization�of��or��s�an���faces��ithin�150��s.�Psychophysiology,�35,�240–251.Sch�i��t,�T.�an���Vorber��,�D.�(2006).�Criteria�for�unconscious�co��nition:�Three�t���es�of���issocia-

tion.�Perception and Psychophysics,�68,�489–504.Schrö��er,�R.�(2007).�Mis�atch�ne��ativit��:�A��icro�hone�into�au��itor����e�or��.�Journal of Psy-

chophysiology,�21,�138–146.Shi�oza�i,�S.�S.,�Ec�stein,�M.�an���Tho�as,�J.�P.�(1999).�The��aintenance�of�a��arent�lu�inance�

of�an�object.�Journal of Experimental Psychology: Human Perception and Performance,�25,�1433–1453.

Shi�oza�i,�S.�S.,�Tho�as,�J.�P.�an���Ec�stein,�M.�P.�(2001).�Effects�of�lu�inance�oscillations�on�si�ulate���li��htness���iscri�ination.�Perception and Psychophysics,�63,�1048–1062.

Si�ons,�D.�J.�(2000).�Current�a��roaches�to�chan��e�blin��ness.�Visual Cognition,�7,�1–15.Si�ons,�D.�J.�an���Levin,�D.�T.�(1997).�Chan��e�blin��ness.�Trends in Cognitive Sciences,�1,�78–89.S�erlin��,�G.�(1960).�The�infor�ation�available�in�brief�visual��resentations.�Psychological Mono-

graph,�74,�1–29.S�erlin��,�G.�(1967).�Successive�a��roxi�ations�to�a��o��el�for�short�ter���e�or��.�Acta Psy-

chologica,�27,�285–292.

Page 142: Unconscious Memory Representations in Perception

� Re�resentation�of�re��ularities�in�visual�sti�ulation� 131

Tales,�A.,�Ne�ton,�P.,�Troscian�o,�T.�an���Butler,�S.�(1999).�Mis�atch�ne��ativit��� in�the�visual��o��alit��.�NeuroReport,�10,�3363–3367.

Ti��er,�S.�P.�(2001).�Does�ne��ative��ri�in���reflect�inhibitor����echanis�s?�A�revie��an���in-te��ration� of� conflictin��� vie�s.� Quarterly Journal of Experimental Psychology, Section A. Human Experimental Psychology,�54,�321–343.

Treis�an,�A.�M.�an���Gela��e,�G.� (1980).�A� feature� inte��ration�theor���of�arrention.�Cognitive Psychology,�12,�97–136.

Turatto,�M.,�An��rilli,�A.,�Mazza,�V.,�U�ilta,�C.�an���Driver,�J.�(2002).�Loo�in����ithout�seein���the�bac���roun���chan��e:�Electro�h��siolo��ical�correlates�of�chan��e���etection�versus�chan��e�blin��ness.�Cognition,�84,�B1–B10.

VanRullen,�R.�an���Thor�e,�S.�J.�(2001).�The�ti�e�course�of�visual��rocessin��:�Fro��earl����erce�-tion�to���ecision��a�in��.�Journal of Cognitive Neuroscience,�13,�454–461.

Vierc�,�E.�an���Kiesel,�A.�(2008).�Chan��e���etection:�Evi��ence�for�infor�ation�accu�ulation�in�flic�er��ara��i���.�Acta Psychologica,�127,�309–323.

Vo��el,�E.�K.,�Luc�,�S.�J.�an���Sha�iro,�K.�L.�(1998).�Electro�h��siolo��ical�evi��ence�for��ost�er-ce�tual�locus�of�su��ression���urin���attentional�blin�.�Journal of Experimental Psychology: Human Perception and Performance,�24,�1654–1674.

Wheeler,� M.� an��� Treis�an,� A.� M.� (2002).� Bin��in��� in� short-ter�� visual� �e�or��.� Journal of Experimental Psychology: General,�131,�48–64.

Willia�s,�P.�an���Si�ons,�D.�J.�(2000).�Detectin���chan��es�in�novel,�co��lex�three-��i�ensional��bjects.�Visual Cognition,�7,�297–332.

Win�ler,�I.�(2007).�Inter�retin���the��is�atch�ne��ativit��.�Journal of Psychophysiology,�21,�147–163.

Win�ler,�I.,�Czi��ler,�I.,�Suss�an,�E.,�Horváth,�J.�an���Balázs,�L.�(2005).�Preattentive�bin��in���of�au��itor���an���visual�sti�uli.�Journal of Cognitive Neuroscience,�17,�320–339.

Wol��orff,�M.�G.,�Hac�le��,�S.�A.�an���Hill��ar��,�S.�A.�(1991).�The�effect�of�channel-selecticve�atten-tion�on�the��is�atch�ne��ativit����ave�b�����eviant�tones.�Psychophysiology,�28,�30–42.

Wolfe,�J.�M.�(1994).�Gui��e���search�2.0:�A�revise����o��el�of�visual-search.�Psychonomic Bulletin and Review,�1,�202–238.

Woo��s,�D.�L.,�Alho,�K.�an���Al��azi,�A.�(1992).�Inter�o��al�selective�attention�1.�Effects�on�event–relate����otentials�to�lateralize���au��itor���an���visual�sti�uli.�Electrophyisiology and Clinical Neurophysiology,�82,�341–355.

Page 143: Unconscious Memory Representations in Perception
Page 144: Unconscious Memory Representations in Perception

chapter�5

Auditory learning in the developing brain

Minna�Huotilainen�an���Tuo�as�TeinonenUniversit���of�Helsin�i,�Finlan��

5.1 Early auditory learning and measurements of behaviour

To��a��,��e��no��that� learnin��� fro��the�au��itor���environ�ent� starts�alrea����� in feto.� A�on��� the� first� evi��ence� �as� the� notice��� �reference� of� ne�born� infants�to�ar��s� the�voice�of� their��other.�DeCas�er�an���Fifer� (1980)��ere� the�first� to���e�onstrate�this�effect.�The����erfor�e���t�o�ex�eri�ents�investi��atin���the��ref-erence�of�ne�born�infants�to�ar��s���ifferent�soun��s.�As�an�in��ex�of��reference,�the��� use��� a� behavioural� �roce��ure� in� �hich� the� infant� is� suc�in��� on� a� s�ecial��acifier��easurin���the�frequenc���of�the�suc�in���bursts.�Infants��ere�con��itione���to�learn�that�a�chan��e�in�the�suc�in���frequenc���fro��the�baseline��ro��uce���either�a�recor��in���of�their��other�rea��in���a�chil��ren’s�stor���or,��hen�the�chan��e�ha�-�ene���to�the�o��osite���irection,�a�recor��in���of�another�infant’s��other�rea��in���alou���the�sa�e�stor��.�The��roce��ure��as�counterbalance���b���reversin���the��at-tern�in�half�of�the�infants.�8�of�the�10�infants��easure���chose�to� listen�to�their��other’s�voice�b���shiftin���the�suc�in���frequenc���to�the�res�ective���irection.�As�the�infants��ere�less�than�4���a��s�ol��,�the�result��as�a�stron���in��ication�that�the�infants�ha���learne���their��other’s�voice���urin���the�fetal��erio��.�This�learnin���is��ost��robabl���stron��l���base���on�learnin���of�the�t���ical��roso��ic��atterns�of�the��other’s�s�eech,�an���less�so�for�infor�ation�on�hi��her�frequencies�(voice�qualit��,��hone�e�qualit��),��hich�are�attenuate���in�utero.�This�assu��tion�is�su��orte���b���recent�stu��ies�of�neonatal�learnin���(Voulou�anos�an���Wer�er�2007),�hi��hli��ht-in���the�i��ortance�of�lo��frequencies�for�neonates.�Si�ilar�learnin���results�have�also�been�acquire���for��usic�(He��er�1991;�Wil�in�1995),�but��ore�research�is�nee��e���to�confir��the�extent�of��re-natal��usical�learnin��.

Most�of�the�behavioural�research�of�infant�learnin���concentrates�on�infants�of�6��onths�or�ol��er,��hen��ore���evelo�e���behavioural��easures�are�available.�Juscz����an���Hohne�(1997)�teste���the��or��-learnin���s�ills�of�8-�onth-ol���infants�

Page 145: Unconscious Memory Representations in Perception

134� Minna�Huotilainen�an���Tuo�as�Teinonen

b��� ex�osin��� the�� ten� ti�es� over� a� t�o-�ee�� �erio��� to� 30� �inutes� of� s�eech�containin���three�recor��e���stories.�The�stories�containe����e����or��s�that�occurre���re�eate��l��.�After�a�2-�ee��interval�in��hich�the�infants���i���not�hear�the�stor��,�the�learnin���of�the��or��s��as�teste���usin���the�hea��-turn��reference��roce��ure:�a��roce��ure�in��hich�the�infants���is�la���reco��nition�of�fa�iliar�or�novel�ite�s�b����ee�in���their�hea��s�turne���si��nificantl���lon��er�to�ar��s�a�lou��s�ea�er��la��in���this�t���e�of�sti�ulus�(for���etails,�see�Juscz����1997:�244).�Onl���infants�ex�ose���to�the�stories���is�la��e���a�fa�iliarit����reference��hen��resente����ith��or��s�fro��the�stor���alon����ith�novel��or��s��ith�si�ilar�stress�an����honetic��ro�erties�that���i���not�occur�in�the�stor��.�Infants�in�a�control���rou�,��ho��ere�not�ex�ose���to�the�stor��,�sho�e���no��reference.�Subsequentl��,�Saffran�et�al.�(2000)�use���a�si�ilar�a��roach�to�investi��ate�learnin���of�a��usical��iece�b���seven-�onth-ol���infants.�The���foun���that�the�infants��ere�able�to�se�arate�the�Mozart��iano�sonata��ove-�ents�that�the���ha���been�ex�ose���to�fro��si�ilar�but�novel��usic.�

The��otential�for��e�orisin���s�eech�an����usic�exist�earl���in���evelo��ent.�Ho�ever,� the� ��eneral,� context-in��e�en��ent� �e�or��� re�resentations� for� �or��s�an����elo��ies��i��ht�not�be�available�until�later�in�life.�Houston�an���Juscz����(2003)�stu��ie���7.5��onth-ol��� infants� for� their��e�or���for��or��s.� In� the�first�session,�the���ex�ose���the�infants�to�a�short�stor���containin���30�re�etitions�of�the�tar��et��or��s.� After� one� ��a��,� the� infants� �ere� teste��� for� their� reco��nition� for� the� tar-��et��or��s�usin���the�hea��-turn��reference��roce��ure.�The�infants�reco��nise���the�tar��et� �or��s� �hen� the��� �ere� s�o�en� b��� the� sa�e� s�ea�er� as� in� the� ex�osure.�Ho�ever,� �hen� the� s�ea�er� i��entit��� �as� chan��e��,� the��� faile��� to� reco��nise� the��or��s.�This�su����ests�that�earl���in�the���evelo��ent,�the��e�or���for��or��s�is�at�least��artl���s�ea�er-s�ecific.�

Trainor�et�al.�(2004)�stu��ie���6-�onth-ol���infants’��e�or���for��usical��elo-��ies��ith�a�si�ilar�a��roach.�The���ex�ose���the�infants�to�a��elo�����3��inutes�a���a���for�seven���a��s.�On�the�ei��hth���a��,�the���teste���the�infants��ith�the�hea��-turn��reference��roce��ure.�The��� foun��� that� the� infants� reco��nise��� the��elo����� the���ha���been�listenin���to���urin���the��rece��in����ee��reflecte���b���a��reference�to�ar��s�an�alternative,�novel��elo����.�Ho�ever,��hen�either�the�te��o�or�the�ti�bre�of�the�fa�iliar��elo������as�chan��e��,�the� infants�sho�e���no��reference,� in��icatin���that�the���ha����e�orise���the�s�ecific�te��o�an���the�ti�bre�of�the��elo��ies.�Thus,�analo��icall���to�the�s�ea�er-s�ecific��e�or���re�resentations�for��or��s�in�s�eech,�the�authors�conclu��e��� that� the� infants’��e�or��� for��usic� is� ti�bre-�an��� te�-�o-s�ecific.�An�alternative�ex�lanation� is��ossible,�ho�ever.� It� shoul���be�note���that� the� infants�sho�e����reference� to�ar��s�a�novel��elo������hen�co��are��� to�a��no�n��elo�����in�its�ori��inal�te��o�an���ti�bre,�but�no��reference�to�ar��s�a�novel��elo������hen�co��are���to�a��no�n��elo�����in�a�chan��e���te��o�or�ti�bre.�This��a���be�a���eneral�in��ication�of�a��reference�to�ar��s�an���novel�ele�ents,�be�

Page 146: Unconscious Memory Representations in Perception

� Au��itor���learnin���in�the���evelo�in���brain� 135

it�a��elo�����that�is�co��letel���novel,�or�a�novel�te��o�or�ti�bre�in�an�other�ise�reco��nize��,��no�n��elo����.

Usin���fluent�natural�s�eech�as�sti�uli��ith�infants�is�challen��in���for�various�reasons.� Natural� s�eech� varies� in� �ulti�le� ��i�ensions,� an��� for� �an��� research�questions�it�is�i��ortant�to��ee��all�but�one���i�ension�constant.�The�natural�lan-��ua��e�is�i��ensel���co��lex,�an���still�to��a����e�lac��co��rehensive��no�le����e�of��hich�cues�the�infants�are�able�to�use�an���in��hich��a���the���use�the���hen��rocessin���natural� lan��ua��e.�Artificial� lan��ua��es�have�been���evelo�e���to��robe�sin��le�co��onents�of�lan��ua��e�learnin���(Gó�ez�an���Ger�en�2000).�An�artificial�lan��ua��e�t���icall���contains�a�s�all�set�of�si��le��or��s�or�s��llables�an���rules�that���efine�ho��the�lan��ua��e�is��ro��uce��.�The�stu��ies�focusin���on�statistical learning�an���rule-learning�belo��all�use�artificial�lan��ua��es�to�control�the��ro�erties�of�the�lan��ua��e�in�ut.

Statistical�learnin���has�been�su����este���as�one�of�the��e���co��onents�to�learn-in���in�infanc��.�It�co��rises�all�learnin����echanis�s�that�are�base���exclusivel���on�the�statistical��ro�erties�of�the�in�ut,�such�as�transitional��robabilities�an���statis-tical���istributions.�Within�the�au��itor�����o�ain,�statistical�learnin���coul���hel��the�infants�to�or��anise�the�s�eech�in�ut�into�intelli��ible�units,���ras��the�un��erl��in���structure�of��usic,�or�se�arate�essential� soun��s� fro��the�environ�ent.�Saffran�et�al.� (1996)� foun��� that�8-�onth-ol��� infants��ere�able� to� se�arate��or��s� fro��a�strea��of�s��llables�usin���onl���infor�ation�about�the�transitional��robabilities�bet�een�a��jacent�s��llables.�The���ex�ose���the�infants�over�a��erio���of�t�o��inutes�to�a�continuous�strea��of�s��llables�containin���four�e�be����e���three-s��llabic��seu-��o�or��s.�After�this��erio��,�the���teste����hether�the�infants��ere�able�to�se�arate�the��seu��o�or��s�fro��non-�or��s,�i.e.,��or��s��hich�containe���the�final�s��llable�of�one��seu��o�or���an���t�o�first�s��llables�of�another.�Thus,�the�s��llables�of�the�non-�or��s�ha���been��resent�in�the�ex�osure,�but��ere�not�statisticall���coherent�un-li�e�the��seu��o�or��s,�in��hich�the�three�s��llables�al�a��s�occurre���to��ether.�The�infants��ere�able�to���istin��uish�the��seu��o�or��s�fro��the�non-�or��s,��rovi��in���evi��ence�for�learnin���of�the�transitional��robabilities�e�be����e���in�the�sequence.�More�recentl��,�Teinonen�et�al.�(2009)���e�onstrate���this�sa�e�abilit���in�neonates.�Saffran�et�al.�(1999)�also�sho�e���that�8-�onth-ol���infants���is�la���si�ilar�learn-in���s�ills�also�for�sequences�of�tones.�Thus,�the�statistical�learnin���s�ills�are�not�tailore���for�lan��ua��e,�but�can�be�a��lie���to�other���o�ains�as��ell.

Ma��e�et�al.�(2002)�investi��ate���the�effects�of�statistical���istributions�of��ho-ne�es�on�learnin���of��hone�e�cate��ories.�The����resente���6-�an���8-�onth-ol���infants��ith�s��llables�ran��in���fro��/��a/�to�/ta/.�These�s��llables�follo�e���either�a�uni�o��al���istribution,�in��hich�the��i����le�soun��s�in�the�continuu���ere��re-sente����ith�the�hi��hest�frequenc���or�a�bi�o��al���istribution,�in��hich�the�soun��s�close�to�the�en���oints�of�the�continuu��ha���the�hi��hest�frequenc���of�occurrence.�

Page 147: Unconscious Memory Representations in Perception

13�� Minna�Huotilainen�an���Tuo�as�Teinonen

Usin���a��o��ifie���hea��-turn��reference��roce��ure,� the��� sho�e��� that�after�onl���2.3��inutes�of�ex�osure,�the�infants�ex�ose���to�the�bi�o��al���istribution�of�s��l-lables���iscri�inate���the�test�s��llables�/��a/�an���/ta/.�Ho�ever,�the�infants�ex�ose���to�the�uni�o��al���istribution�of�s��llables���i���not�sho��this���iscri�ination.�Ma��e�et�al.�(2002)�conclu��e���that�infants�are�sensitive�to�the�frequenc�����istribution�of�s�eech�soun��s�an���that�the���are�able�to�use�this�infor�ation�to���etect��honetic�cate��or���structure.�Recentl��,�Teinonen�et�al.�(2008)���e�onstrate���that�visual�in-for�ation�contributes�stron��l���to�learnin���of��hone�e�boun��aries.�The���sho�e���that�even�a�uni�o��al���istribution�of�s�eech�soun��s�can�be��erceive���bi�o��all���b���6-�onth-ol���infants��ith�the�hel��of�bi�o��al�visual�counter�arts.

Learnin���of�si��le�rules� in�7-�onth-ol��� infants��as�teste���b���Marcus�an���others� (1999).� The��� ��ivi��e��� the� infants� into� t�o� ��rou�s� that� �ere� ex�ose��� to�three-s��llabic��seu��o�or��s�constructe���accor��in���to�one�of�t�o�rules:�either�the�first�t�o�s��llables�of�the��or��s��ere�equal�(e.��.,�“le-le-��i”)�or�the�t�o�final�s��llables��ere�equal� (e.��.,� “ji-li-li”).�After�a�2-�inute-lon���ex�osure,� the����ere� teste��� in�a�variant�of�the�hea��-turn��reference��roce��ure��hether�the�����istin��uish�novel��or��s�follo�in���the�rule�that�the���ha���been�ex�ose���to�fro��novel��or��s�follo�-in���the�other�rule.�In�both���rou�s,�the�infants�atten��e���lon��er�the�test�ite�s�that���i���not�follo��the�rule�that�the���ha���been�ex�ose���to.�This���e�onstrate���that�the���ha���learne���the�rule�the����ere�ex�ose���to.�

Not� ever��thin��� encountere��� in� life� can� or� even� shoul��� be� learne��.� Infants��a�����eal��ith�the�abun��ance�of� infor�ation�b���havin�����ifferent�constraints� in�the�learnin����echanis�s.�It�is�li�el���that�the�evolution�has�sha�e���the�learnin����echanis�s�to�focus�on�the�infor�ation�relevant�to�us�an���to�i��nore�the��ajorit���that,�fro��a�learnin����ers�ective,�is�noise.�Saffran�(2002)�sho�e���that�a��ult��ar-tici�ants�acquire�an�artificial�lan��ua��e�easier,��hen�it�contains��re��ictive���e�en-��encies,�si�ilar�to�those��resent�in�the��hrases�of�natural�lan��ua��es.�The�results��ere�also���eneralise���to�7-�to�10-��ear-ol���chil��ren�an���to�non-lin��uistic�soun��s.�The�infant�research�a����ressin���the�constraints�or�the�biases�of�learnin���(or�in�a��ore�restricte�����o�ain,�statistical�learnin��)�has��uch�to�achieve.�To��a��,�the�re-search�is��ore�focuse���on�the�i��ortant�to�ic�of��hat�can�be�learne��,��hen�it�can�be�learne��,�an����hat��echanis�s�are�available�for�acquirin���that��articular�t���e�of�infor�ation.�The�future�ste��to�be�ta�en�is�to��i��en�the��ers�ective�into��hat�constraints� infants�have�for� learnin���fro����ifferent�t���es�of�sensor��� in�ut.�The�t�o�co�bine����ill���ive�us�a��ore�co��rehensive�vie��on�the�learnin����rocess.

Page 148: Unconscious Memory Representations in Perception

� Au��itor���learnin���in�the���evelo�in���brain� 137

5.2 Mismatch negativity and other brain indices of simple auditory learning

The� �is�atch� ne��ativit��� (Näätänen� et� al.� 1978)� or� MMN� is� an� event-relate����otential�extracte���fro��the�EEG�an���recor��e���as�a�res�onse�to�a�chan��e�in�an�other�ise�constant�au��itor���environ�ent.�It�is��ro�ose���to�in��ex�the�function-in���of�the�au��itor���short-ter���e�or���or�au��itor���sensor����e�or��.�MMN�has�turne���out� to�be�ver���useful� in� stu����in��� learnin�����urin��� the�course�of� the���e-velo��ent,��artl���because�it���oes�not�require�an���active��artici�ation�fro��the�subject.�In��ee��,�the�MMN�is�best�recor��e���in�neonates���urin���slee��(Fell�an�an����Huotilainen�2006;�Kushneren�o�et�al.�2002).

The�MMN�is�t���icall�����eter�ine���fro��the�subtraction�si��nal�obtaine���b���subtractin���the�res�onse�of�the�re�etitive�soun��,�stan��ar��,�fro��that�of�the���e-viatin���soun��,���eviant.�The�res�onse� is�observe��� in�a��ults� in� the�120–300��s�latenc���ran��e�as�a�ne��ative��ea��a��earin���es�eciall���stron���at�the�frontocentral�scal��sites�an���as�a��ositive��ea��at��astoi��s�(Kujala�et�al.�2007).�Naturall��,�all��ea�s�observe���in�the�subtraction�si��nal�are�not�necessaril���MMNs,�since�the�subtraction�si��nal��a���contain�t�o�t���es�of�other��ea�s:�Cate��or���1:�traces�of���ifference�in�the�obli��ator���res�onses�of�the���eviant�an���the�stan��ar��,�relate���to�habituation,�refractoriness,�tonoto���,�factors�correlatin����ith���uration�or�inten-sit�����ifferences�of�the�soun��s,�an���Cate��or���2:�later�res�onses�that�are�cause���b����rocesses�tri����ere���b���the�MMN,�relate���to,�for�exa��le,�u���atin���or�stren��th-enin��� of� the� �e�or��� trace,� involuntar��� attention� s�itchin��� (in� a��ults� calle���P3a),�voluntar���attention�s�itchin���(P3b/P300),�se�antic��rocessin���(N400),�or�re-orientin���after�an�attention�s�itch�(RON).�All��ea�s�observe���in�the�subtrac-tion�si��nal�shoul���be�inter�rete����ith�caution.�In��ractise,�if�the�stan��ar���an�����eviant�soun��s���o�not���iffer�acousticall���fro��each�other�ver����uch,�all��ea�s�observe���in�the�subtraction�si��nal�are�either�the�MMN�or�belon���to�Cate��or���2,�that�is,��rocesses�that�are�tri����ere���b���the�MMN�or�so�e�other,�still�un�no�n,��rocess�in�the�neonatal�brain�relate���to�chan��e-��etection.�Different�Cate��or���2�res�onses��a����artl���ex�lain�the���ifferent�scal����istributions,��olarities�an���la-tencies�foun���in�the���ifferent�ex�eri�ents�(Kushneren�o�et�al.�2007).�In�su�,�to�ans�er�questions�relate���to�learnin���in�infants,�it�is�often�sufficient�to��a�e�sure�that�the�res�onses�seen�in�the�subtraction�si��nal�are�not�Cate��or���1��ea�s�si�-�l���b����ee�in���the�acoustic���ifference�bet�een�the�soun��s�as�s�all�as��ossible.�Another��ossibilit���is�to�co��are�the�res�onses�to�the���eviant�an���an�i��entical�soun���recor��e���in�a�con��ition��ithout��e�or���trace�effects,�e.��.,�an�equi�rob-able�con��ition�(Kushneren�o�et�al.�2001a).�

The�first��itch�MMN�recor��in��s�(1000�Hz�vs�1200�Hz)�in�infants��ere��ub-lishe���b���Alho�et�al.�(1990a).�Thereafter,�several�stu��ies�have�sho�n�MMN-li�e�

Page 149: Unconscious Memory Representations in Perception

13�� Minna�Huotilainen�an���Tuo�as�Teinonen

res�onses�to�chan��es�in��itch:�Kurtzber���et�al.�(1995)�(1000�vs�1200�Hz)��ith�t�o��resentation�rates,�Cheour�et�al.� (1999)�an���control���rou�s�of�Ce�oniene�et�al.�(2000)�(1000�vs�1100�Hz),�Le��änen�et�al.�(1997)�(1000�vs�1300�an���1100�Hz)�in�ne�borns,�Tana�a�et�al.�(2001)�(750�vs�1000�Hz)�in�ne�borns,�Morr�et�al.�(2002)�(1000�vs�1200�an���2000�Hz)� in�a��i��e�a��e���rou��an���Novits�i� et� al.� (2007)� in�neonates�in�a��i��e�frequenc���ran��e.�These�results,�even�thou��h�var��in���in��olar-it���an���scal����istribution,�sho��that�infants���o�buil����e�or���re�resentations�of�re�etitive�au��itor���events�an���notice�chan��es�in�the��itch�of�sinusoi��al�an���har-�onic�tones.�Me�or���for��itch�is�essential�for�the�infants�in�their���oal�of�lan��ua��e�acquisition.

Res�onses�to�chan��es�in�soun�����uration�have�been�re�orte���b���Kushneren�o�et�al.�(2001a)�(200�vs�300�an���400��s)�an���b���Cheour�et�al.�(2002a)�(100�vs�40�an���200��s)�in�ne�borns.�Res�onses�to�chan��es�in��hone�e���uration�have�been�re-�orte���b���Pih�o�et�al.�(1999)�in�0-6-�onth-ol���infants,�b���Frie��erici�et�al.�(2002)�in�2-�onth-ol���infants,�an���b���Kushneren�o�et�al.�(2001b)��ith��seu��o�or��s�in�ne�borns.�Res�onses�to�silent���a�s�in�sinusoi��al�soun��s��ere�re�orte���b���Trainor�et�al.�(2001)�in�6–7-�onth-ol���infants.�These�stu��ies�sho��that�the�neonatal�brain�can�learn�a�s�ecific���uration�of�a�soun���an���use�the��e�or���trace�of�this���uration�quite�accuratel���in�co��arin���it�to�inco�in���soun��s.�

Several���rou�s�have�also�re�orte���infant�MMN�res�onses�to�chan��es�in��ho-ne�es.�For�exa��le,�Dehaene-La�bertz�an���Dehaene�(1994)�sho�e���res�onses�(/ba/�vs�/��a/)�in�2–3-�onth-ol���infants,�Cheour-Luhtanen�et�al.�(1995,�1996)�(/��/�vs�/i/�but�not�to�/��/�vs�/��-i/)�in�full-ter��an����re-ter��ne�borns,�Cheour�et�al.�(1997)�(/��/�vs�/i/�an���/��/�vs�/��-i/)�in�3-�onth-ol��s,�an���Mart��nova�et�al.�(2002)�(/o/�vs�/e/)�in�neonates.�Pan���et�al.�(1998)�sho�e���res�onses�to�/��a/-/ta/-contrast�in�8-�onth-ol���infants,�Cheour�et�al.�(1998)�to�Finnish�an���Estonian�vo�els�in�6–12-�onth-ol���infants.�

Before�the�,�also�Kurtzber���et�al.�(1986)�ha���recor��e���a�ver���slo��an���ne��a-tive�res�onse�to���eviant�/ta/-s��llables��resente���a�on��st�stan��ar���/��a/-s��llables,�but� the��� inter�rete��� this� res�onse�as��o��alit��-nons�ecific�an���not� rese�blin���the�MMN,�an���the�res�onses�recor��e���to�full��or��s�b���Dere��nier�et�al.�(2000)��resente���late�ne��ative�res�onses�in�an�equi�robable�con��ition.

These� results� sho�� that� the� neonatal� au��itor��� s��ste�� has� hi��hl��� accurate��e�or���ca�abilities�for��hone�es.�The��e�or���traces�are�for�e���quic�l���an���accuratel���enou��h�to�enable�the���etection�of�the�chan��e�of�a��hone�e�cate��or��.�This���etection�is,�ho�ever,�not�a��roof�of� the�existence�of��honetic�cate��ories.�The���ata�act��erel���as�a���e�onstration�of�a�sufficient�sensor����e�or���s��ste��an���a�sufficient�accurac���of�the���etection�of�the��hone�e-s�ecific�features�(for-�ant�frequencies�an���their�transitions)�to�allo��the�se�aration�of�the��hone�es�or�s��llables.

Page 150: Unconscious Memory Representations in Perception

� Au��itor���learnin���in�the���evelo�in���brain� 139

Durin���the�first���ear�of�life,�the�infant�acquires�a�so-calle����hone�e��a�.�On�this��a�,��ost��robabl���situate����ithin�au��itor���cortical�areas�but��ossibl���also��ithin� lo�er� brain� areas,� a� re�resentation� of� each� of� the� native-lan��ua��e� �ho-ne�es�is�store��.�These�lon��-ter���e�or���traces�of��hone�es�serve�as�te��lates�for�s�eech��erce�tion,��a�in��� it��ossible� to��erceive�s�eech�quic�l���an���auto-�aticall���an����ovin���the�focus�of�s�eech�soun����erce�tion�fro��acoustic� fea-tures�to�ar��s��ore���eneralise���te��lates.�Evi��entl�����e�en��in���on�the�nu�ber�of�vo�els�in�the�native�lan��ua��e,�these��hone�e��a�s���evelo��bet�een�the�a��es�of�6�to�12��onths�for�vo�els�an���later�for�consonants�(Kuhl�2004;�Cheour�et�al.�1998).�Durin���the��rocess�of�buil��in���the��hone�e��a�s,�the�au��itor���s��ste��is�co�-�ittin���to�the�native�lan��ua��e�–�the��erce�tion�of�native�lan��ua��e�is�enhance��,��hile�si�ultaneousl���the��erce�tion�of�forei��n�lan��ua��es�beco�es�less�an���less�accurate.�Still�at�the�a��e�of�3–6���ears,�the�forei��n��hone�es�can�be�a����e���to�the�native� lan��ua��e��hone�e��a��in� just�a� fe���onths�b���ex�osin���the�chil���to�a�forei��n�lan��ua��e���ail���(Cheour�et�al.�2002c),��hile�in�a��ults�this��oul����robabl���require���ears�of�lan��ua��e�lessons.�

The�basic�MMN�ex�eri�ents�in�neonates���e�onstrate,�in�su�,�that�the�neo-natal�au��itor���s��ste��is�accurate�enou��h�to���ifferentiate�soun��s��ith�s�all��itch�an�����uration���ifferences�an�����ifferences�in�natural�an���se�i-s��nthetic��hone�es.�The�results�also�sho��that���ifferences�in�the�te��oral�structure�of�the�soun���are���etecte��.�The�learnin���in�these�ex�eri�ents�occurs�in�the�ti�e�scale�of�a�fe��sec-on��s:��hen�the�stan��ar���soun��s�start�re�eatin��,�a��e�or���trace�is�i��e��iatel���for�e��.�Cheour�et�al.�(2002b),�ho�ever,��resente���health���neonates��ith�a��ore�challen��in���tas��in��hich�learnin���too��a�fe��hours.�The�infants��ere��resente����ith�se�i-s��nthetic��hone�es�/i/,�re�lace���occasionall����ith��hone�e�/��-i/,�s��n-thesise��� to� inclu��e� for�ant� infor�ation� that� �laces� the� �hone�e� in� the� �i��-�oint�of�/i/�an���/��/.�In�the�initial�ex�eri�ents,�the�neonatal�brain��as�not�able�to���etect�such�a�s�all�chan��e�in�for�ant�frequencies.�Thereafter,� the� infants��ere�ex�ose���to�the�sa�e��hone�es��hile�the����ere�slee�in��.�Next��ornin��,�the�neo-nates��resente���MMN�res�onses�to�the�sa�e�chan��e,���e�onstratin���that�learnin���ha���occurre���on�the�basis�of��ere�ex�osure�to�the��hone�es���urin���slee�.�These�i��ortant�results�have�unfortunatel���not�been�re�licate��.�It��oul���also�be�i��or-tant�to��no��to��hich�extent�these��e�or���traces�re�ain�an���at��hich�a��es�such�slee��learnin���can�ta�e��lace.�

5.3 Magnetoencephalography reveals fetal learning

Ma��netoence�halo��ra�h���(MEG)�(Huotilainen�2006)�is�a�useful��etho���for�in-vesti��atin���learnin���an���its�neuro�h��siolo��ical�correlates.�In�neonates,�MEG�has�

Page 151: Unconscious Memory Representations in Perception

140� Minna�Huotilainen�an���Tuo�as�Teinonen

been�use���to�recor���MMN��res�onses�(the��a��netic�counter�art�of�MMN)�in�neonates�to�chan��es�in�soun���frequenc���(Huotilainen�et�al.�2003)�an����hone�es�(Kujala�et�al.�2004;�Pih�o�et�al.�2004;�Cheour�et�al.�2004).

In�a����ition�to�si��le�MMN�ex�eri�ents,�MEG�is�a���oo���tool�also�for�stu��ies�of�hi��her�co��nitive� functions� in� infants�(Huotilainen�et�al.�2008).�For�exa��le,�Sa�beth�et�al.�stu��ie���attention�allocation�(2006)�an���s�eech��roso�������etection�(2008)�in�neonates��ith�MEG.�Most�i��ortantl��,�ho�ever,�the�MEG,��ith�s�e-cific�technical�arran��e�ents,�is�suitable�for�recor��in���the��a��netic�fiel����ro��uce���b���the�fetal�brain�co��letel���noninvasivel���outsi��e�the��other’s�ab��o�en.�Most�lar��e-scale�flat-botto��instru�ents���esi��ne���for��a��netocar��io��ra�h���are�suit-able�for�the��ur�ose,�an���a�s�ecificall�����esi��ne���SARA�is�also�available�(Es�aran�et�al.�2007).�Stu��ies��ith�these�instru�ents�have���e�onstrate���that�also�fetuses�have�the�MMN��res�onse�(Huotilainen�et�al.�2005;�re�licate���b���Dra��anova�et�al.�2005�an���2007��ith�SARA).�After�these��ro�isin���first�si��le�ex�eri�ents,�it�is�ex�ecte���that�MEG��ill��rovi��e�us��ith��ore���etaile�����ata�on�learnin���in�the�hu�an�fetus�also��ith��ore�natural�soun��s.�

5.4 Early learning in complex auditory environments

Recentl��,� the� ��evelo��ent� of� ERP� research� has� �ro��resse��� into� �ore� co��lex�research���esi��ns.�Consequentl��,�the�research�questions�have�ex�an��e���fro��ques-tions�such�as�‘is�the�brain�able�to���iscri�inate�A�fro��B’�to�questions�such�as�‘can�the�brain���etect�a�violation�of�rule�X’�or�rather�‘can�the�brain�learn�the�rule�X’.�This���evelo��ent�is�es�eciall���fruitful��ith�infants,�as�the�behavioural�research��eth-o��s�available�earl���in�life�are�li�ite��.�In�the�follo�in��,�so�e�of�the��ost�recent���e-velo��ents��ithin�the��ore�co��lex�stu��ies�of�au��itor���learnin���are��resente��.

A�crucial�tas��in�or��anisin���the�au��itor���environ�ent�is�se�aratin���the�soun��s��ro��uce��� b��� ��ifferent� sources.� This� �rocess,� calle��� auditory stream segregation�enables�us� to�selectivel���atten���to�a�soun����ro��uce���b���a�s�ecific�source��hile�i��norin���other�soun��s.�For�infants,�it��oul���be�useful�in�tas�s�such�as�lan��ua��e�learnin���b���enablin���the��to�focus�on,�sa��,�the��other’s�voice�in�a�nois���environ-�ent.�Win�ler�an���others�(2003)�stu��ie���au��itor���strea��se��re��ation�in�ne�born�infants.� The��� �resente��� the� infants� �ith� t�o� strea�s� of� au��itor��� tones� so� that�ever���thir���tone��as�fro��the�first�strea��an���the�rest�fro��the�secon���strea�.�The�first�strea��containe���stan��ar���tones�an���tones���eviatin���in�intensit���in�O����-ball�fashion.�In�the�secon���strea�,�the�tones�varie���both�in�intensit���an���in��itch.�When�the�tones�in�the�t�o�strea�s��ere�close�to��ether�in�frequenc���(Fi��ure�1a),�the���eviant�tones�e�be����e���in�the�first�strea����i���not�evo�e�an�MMN�res�onse,�su����estin���that�the�infants��ere�not�able�to�se��re��ate�the�t�o�strea�s.�Ho�ever,�

Page 152: Unconscious Memory Representations in Perception

� Au��itor���learnin���in�the���evelo�in���brain� 141

�hen�the�t�o�strea�s��ere��resente���in�se�arate�frequenc���ran��es�(Fi��ure�1b),�the���eviant�tones�evo�e���an�MMN�res�onse,�su����estin���that�the�infants��ere�able�to� se��re��ate� the� t�o� strea�s� an��� for�� a� �e�or��� trace� for� the� stan��ar��� tones�in�the�strea���ith�the�hi��her��itch.�The�results���e�onstrate�that�even�ne�born�infants�can�se�arate�the�soun���sources�in�their�au��itor���environ�ent.�In�real�life,�also�ti�bre�is�li�el���to�notabl���facilitate�the�au��itor���strea��se��re��ation.

Vanessa�Carral�an���others� (2005)�ex�ose���ne�born� infants� to� soun����airs��ith�a�si��le�abstract�rule:�in��ost�of�the��airs,�the�secon���soun���ha���a�hi��her��itch�co��are���to�the�first�soun��.�Ho�ever,�in�one�ei��hth�of�the�cases,�the�secon���soun���ha���a�lo�er��itch.�The�ERPs��easure���to�the�onset�of�the�secon���soun�����if-fere���si��nificantl���bet�een�the�t�o�cases.�As�the�soun����ro�erties��ere�other�ise�acousticall���balance��,� this��as�seen�as�evi��ence� that� the�ne�born� infants��ere�able�to�learn�the�abstract�rule,�i.e.,�that�t���icall���the��itch�of�the�secon���soun���is�hi��her�than�that�of�the�first�one.

Ruusuvirta�an���others�(2003)�use���a�novel�a��roach�to�stu�����au��itor���fea-ture�bin��in���in�ne�born�infants.�The����la��e���to�the�infants�tones�that�varie���in�frequenc���an���intensit��.�Alto��ether�four�tones��ere�use����ith�all��ossible�co�-binations�of�the�t�o�intensit���levels�(50�an���70���B�soun����ressure�level)�an���t�o�frequenc���levels�(750�an���1000�Hz).�T�o�of�the�tones�occurre���frequentl���an���the�other�t�o�rarel���(to��ether�one�tone�in�ten).�The�tones��ere�assi��ne���to�stan��ar��s�an�����eviants�so�that�the��robabilit�����istribution�of�the�levels�of�both�features��as�s����etrical�across�the�cate��ories.�This��a��,�a���eviant�coul���not�be���iscri�inate���fro�� a� stan��ar��� on� the� basis� of� either� frequenc��� or� intensit��� alone.� The� �ea-sure���ERPs���is�la��e���a�si��nificant���ifference�bet�een�the�stan��ar���an�����eviant�res�onses.�Thus,�the�ne�born�brain��as�able�to�co�bine�the�t�o�features�of�the�tones��hen�for�in����e�or���re�resentations�for�these�soun��s.�

Figure 1. In�the�one�strea��con��ition�(a),�intervenin���tones�varie���in�frequenc���an���intensit��.�For�the�t�o-strea��con��ition�(b),�the�frequencies�of�one�of�the�strea�s�(�hite�tones)��ere�lo�ere���fro��those�use���in�the�one-strea��con��ition,�but�the�intensit���val-ues��ere�retaine��.�(A��a�te����ith��er�ission�fro��Win�ler�et�al.�2003.)

Page 153: Unconscious Memory Representations in Perception

142� Minna�Huotilainen�an���Tuo�as�Teinonen

5.5 Conclusions

Our�vie��of�au��itor���learnin���in�the�infant�brain�is�constantl���challen��e���b���ne��results�sho�in����ore�co��lex,�faster,�an����ore�s�ecialise���learnin���than��hat��as�ex�ecte���before.�Partl���these�ne��results�ste��fro��ne���etho��s�an�����e-velo��ents�of���ata��rocessin���an���anal��sis� techniques� that���irectl���benefit� the�infant�learnin���stu��ies.�Still,��ost�of�the�ne��results�are�obtaine���b�����evelo�in���novel��ara��i���s�for�brain�research,�allo�in���scientists�to�a��roach��ore�co��lex�an���naturalistic�situations�of�infant�learnin��.�Es�eciall���the�neonatal�brain�is�ver����lastic�an���can�a��a�t�to��an����in��s�of�environ�ents.�The�li�its�of�this�a��a�tation�are�challen��e���in�infants��ho�have��ajor�brain���a�a��e�or��inor���isa��vanta��es��rior�to,���urin��,�or�i��e��iatel���after�birth.�In�or��er�to�un��erstan���the�nature�of�these��roble�s�an���es�eciall���the�challen��es�the����ose�for�s�eech���evelo��ent�an���other�attention-��e�an��in���tas�s� in� the� future,� the���evelo��ent�of�neona-tal�au��itor����rocessin��,��e�or��,�an���attention�allocation�shoul���be�stu��ie����ith�natural�soun��s.�This�excitin���fiel���is��ro��ressin���ra�i��l���an���it� is�ex�ecte���that�the�first�a��lications�of�earl���re�e��iation�of�au��itor����erce�tive��roble�s��ill�be�teste���in�the�near�future.�In��ee��,��ajor�results�are�ex�ecte�����ue�to�the��lasticit���of�the�brain��rior�to�a��e�12��onths.�These�ne��a��roaches��ill�be�es�eciall���hel�ful�to�infants��ith�brain���a�a��e�or�a�ver����re�ature�birth.

References

Alho,�K.,�Sainio,�K.,�Sajanie�i,�N.,�Reini�ainen,�K.�an���Näätänen,�R.�(1990).�Event-relate���brain��otential�of�hu�an�ne�borns�to��itch�chan��e�of�an�acoustic�sti�ulus.�Electroencephalog-raphy and Clinical Neurophysiology,�77,�151–155.

Carral,�V.,�Huotilainen,�M.,�Ruusuvirta,�T.,�Fell�an,�V.,�Näätänen,�R.�an���Escera,�C.� (2005).�A� �in��� of� au��itor��� ‘�ri�itive� intelli��ence’� alrea����� �resent� at� birth.� European Journal of Neuroscience,�21,�3201–3204.

Ce�oniene,�R.,�Hu��i,�J.,�Cheour,�M.,�Haa�anen,�M.�L.,�Kos�inen,�M.,�Alho,�K.�an���Näätänen,�R.�(2000).�D��sfunction�of�the�au��itor���cortex��ersists�in�infants��ith�certain�cleft�t���es.�Developmental Medicine and Child Neurology,�42,�258–265.

Cheour-Luhtanen,�M.,�Alho,�K.,�Kujala,�T.,�Sainio,�K.,�Reini�ainen,�K.,�Renlun��,�M.,�Aaltonen,�O.,�Eerola,�O.�an���Näätänen,�R.�(1995).�Mis�atch�ne��ativit���in��icates�vo�el���iscri�ina-tion�in�ne�borns.�Hearing Research,�82,�53–58.

Cheour-Luhtanen,� M.,� Alho,� K.,� Sainio,� K.,� Rinne,� T.,� Reini�ainen,� K.,� Pohjavuori,� M.,� Aal-tonen,�O.,�Eerola,�O.�an���Näätänen,�R.�(1996).�The�onto��eneticall���earliest���iscri�inative�res�onse�of�the�hu�an�brain.�Psychophysiology, Special Report,�33,�478–481.

Cheour,�M.,�Haa�anen,�M.�L.,�Hu��i,�J.,�Ce�oniene,�R.,�Kurjenluo�a,�S.,�Alho,�K.,�Tervanie�i,�M.,�Ranta,�R.�an���Näätänen,�R.�(1997).�The�first�neuro�h��siolo��ical�evi��ence�for�co��nitive�brain�����sfunctions�in�CATCH�chil��ren.�NeuroReport,�8,�1785–1787.

Page 154: Unconscious Memory Representations in Perception

� Au��itor���learnin���in�the���evelo�in���brain� 143

Cheour,�M.,�Ce�oniene,�R.,�Lehto�os�i,�A.,�Luu�,�A.,�Alli�,�J.,�Alho,�K.�an���Näätänen,�R.�(1998).�Develo��ent� of� lan��ua��e-s�ecific� �hone�e� re�resentations� in� the� infant� brain.� Nature Neuroscience,�1,�351–353.

Cheour,�M.,�Ce�oniene,�R.,�Hu��i,�J.,�Haa�anen,�M.-L.,�Näätänen,�R.�an���Alho,�K.�(1999).�Brain�����sfunction� in� neonates� �ith� cleft� �alate� reveale��� b��� the� �is�atch� ne��ativit��� (MMN).�Electroencephalography and Clinical Neurophysiology,�110,�324–328.

Cheour,�M.,�Kushneren�o,�E.,�Ce�oniene,�R.,�Fell�an,�V.�an���Näätänen,�R.�(2002a).�Electric�brain�res�onses�obtaine���fro��ne�born�infants�to�chan��es�in���uration�in�co��lex�har-�onic�tones.�Developmental Neuropsychology,�22,�471–480.

Cheour,�M.,�Mart��nova,�O.,�Näätänen,�R.,�Er��ola,�R.,�Sillan�ää,�M.,�Kero,�P.,�Raz,�A.,�Kai�io,�M.-L.,�Hiltunen,�J.,�Aaltonen,�O.,�Savela,�J.�an���Hä�äläinen,�H.�(2002b).�S�eech�soun��s�learne���b���slee�in���ne�borns.�Nature,�415,�599–600.

Cheour,� M.,� Shesta�ova,� A.,� Al�u,� P.,� Ce�oniene,� R.� an��� Näätänen,� R.� (2002c).� Mis�atch�ne��ativit���sho�s�that�3–6-��ear-ol���chil��ren�can�learn�to���iscri�inate�non-native�s�eech�soun��s��ithin�t�o��onths.�Neuroscience Letters,�325,�187–190.

Cheour,�M.,�I�a��a,�T.,�Taulu,�S.,�Ahonen,�A.,�Salonen,�J.�an���Kuhl,�P.�(2004).�Ma��netoence�ha-lo��ra�h���is�feasible�for�infant�assess�ent�of�au��itor�����iscri�ination.�Experimental Neurol-ogy,�190,�S44–51.

DeCas�er,�A.�J.�an���Fifer,�W.�P.�(1980).�Of�hu�an�bon��in��:�Ne�borns��refer�their��others’�voices.�Science,�208,�1174–1176.

Dehaene-La�bertz,�G.�an���Dehaene,�S.�(1994).�S�ee���an���cerebral�correlates�of�s��llable���is-cri�ination�in�infants.�Nature,�28,�293–294.

Dere��nier,�R.�A.,�Nelson,�C.�A.,�Tho�as,�K.�M.,�We�er�a,�S.�an���Ceor��ieff,�M.�K.�(2000).�Neu-ro�h��siolo��ic�evaluation�of�au��itor���reco��nition��e�or���in�health���ne�born�infants�an���infants�of���iabetic��others.�Journal of Pediatrics,�137,�777–784.

Dra��anova,�R.,�Es�aran,�H.,�Mur�h��,�P.,�Huotilainen,�M.,�Lo�er��,�C.�an���Preissl,�H.�(2005).�Soun���frequenc���chan��e���etection�in�fetuses�an���ne�borns,�a��a��netoence�halo��ra�hic�stu����.�NeuroImage,�28,�354–361.

Dra��anova,�R.,�Es�aran,�H.,�Mur�h��,�P.,�Lo�er��,�C.�an���Preissl,�H.� (2007).�Serial��a��neto-ence�halo��ra�hic�stu�����of�fetal�an���ne�born�au��itor�����iscri�inative�evo�e���res�onses.�Early Human Development,�83,�199–207.

Es�aran,�H.,�Ha����a��,�N.�I.,�Shihabu����in,�B.�S.,�Preissl,�H.,�Sie��el,�E.�R.,�Mur�h��,�P.�an���Lo�er��,�C.�L.� (2007).�Non-invasive���etection�an��� i��entification�of�brain�activit����atterns� in� the���evelo�in���fetus.�Clinical Neurophysiology,�118,�1940–1946.

Fell�an,�V.,�Kushneren�o,�E.,�Mi��ola,�K.,�Ce�oniene,�R.,�Lei�älä,�J.�an���Näätänen,�R.�(2004).�At���ical�au��itor���event-relate����otentials�in��reter��infants���urin���the�first���ear�of�life:�A��ossible�si��n�of�co��nitive�����sfunction?�Pediatric Research,�56,�291–297.

Fell�an,�V.�an���Huotilainen,�M.�(2006).�Cortical�au��itor���event-relate����otentials�in�ne�born�infants.�Seminars in Fetal and Neonatal Medicine,�11,�452–458.

Frie��erici,�A.�D.,�Frie��erich,�M.�an���Weber,�C.�(2002).�Neural��anifestation�of�co��nitive�an����reco��nitive��is�atch���etection�in�earl���infanc��.�NeuroReport,�13,�1251–1254.

Gó�ez,�R.�L.�an���Ger�en,�L.�(2000).�Infant�artificial�lan��ua��e�learnin���an���lan��ua��e�acquisi-tion.�Trends in Cognitive Sciences,�4,�178–186.

He��er,�P.�G.�(1991).�An�exa�ination�of�fetal�learnin���before�an���after�birth.�Irish Journal of Psychology,�12,�95–107.

Page 155: Unconscious Memory Representations in Perception

144� Minna�Huotilainen�an���Tuo�as�Teinonen

Houston,�D.�M.�an���Juscz���,�P.�W.�(2003).�Infants’�lon��-ter���e�or���for�the�soun����atterns�of��or��s�an���voices.�Journal of Experimental Psychology: Human Perception and Perfor-mance,�29,�1143–1154.

Huotilainen,�M.,�Kujala,�A.,�Hota�ainen,�M.,�Shesta�ova,�A.,�Kushneren�o,�E.,�Par��onen,�L.,�Fell�an,�V.�an���Näätänen,�R.�(2003).�Au��itor����a��netic�res�onses�of�health���ne�borns.�NeuroReport,�14,�1871–1875.

Huotilainen,�M.,�Kujala,�A.,�Hota�ainen,�M.,�Par��onen,�L.,�Taulu,�S.,�Si�ola,�J.,�Nenonen,�J.,�Karjalainen,� M.� an��� Näätänen,� R.� (2005).� Short-ter�� �e�or��� functions� of� the� hu�an�fetus�recor��e����ith��a��netoence�halo��ra�h��.�NeuroReport,�16,�81–84.

Huotilainen,�M.�(2006).�Ma��netoence�halo��ra�h���of�the�ne�born�brain.�Seminars in Fetal and Neonatal Medicine,�11,�437–443.

Huotilainen,�M.,�Shesta�ova,�A.�an���Hu��i,�J.�(2008).�Usin����a��netoence�halo��ra�h���in�as-sessin���au��itor���s�ills�in�infants�an���chil��ren.�International Journal of Psychophysiology,�68,�123–129.

Juscz���,�P.�W.�(1997).�The discovery of spoken language.�Ca�bri����e,�MA:�MIT�Press.Juscz���,�P.�W.�an���Hohne,�E.�A.�(1997).�Infants’��e�or���for�s�o�en��or��s.�Science,�277,�1984–

1986.Kuhl,�P.�(2004).�Earl���lan��ua��e�acquisition:�Crac�in���the�s�eech�co��e.�Nature Reviews Neurosci-

ence,�5,�831–843.Kujala,� A.,� Huotilainen,� M.,� Hota�ainen,� M.,� Lennes,� M.,� Par��onen,� L.,� Fell�an,� V.� an���

Näätänen,�R.�(2004).�S�eech-soun�����iscri�ination�in�neonates�as��easure����ith�MEG.�NeuroReport,�15,�2089–2092.

Kujala,�T.,�Tervanie�i,�M.�an���Schrö��er,�E.�(2007).�The��is�atch�ne��ativit���in�co��nitive�an���clinical�neuroscience:�Theoretical�an����etho��olo��ical�consi��erations.�Biological Psychol-ogy,�74,�1–19.

Kurtzber��,�D.,�Vau��han,�H.�G.�Jr.�an���Nova�,�G.�P.�(1986).�Discri�inative�brain�res�onses�to�s�eech�soun��s�in�the�ne�born�hi��h�ris��infant.�In�V.�Gallai�(E��.),�Maturation of the CNS and evoked potentials�(��.�253–259).�A�ster��a�:�Elsevier.

Kurtzber��,�D.,�Vau��han,�H.�G.�Jr.,�Kreuzer,�J.�A.�an���Flie��ler,�K.�Z.�(1995).�Develo��ental�stu��-ies�an���clinical�a��lication�of��is�atch�ne��ativit��:�Proble�s�an����ros�ects.�Ear and Hear-ing,�16,�105–117.

Kushneren�o,�E.,�Ce�oniene,�R.,�Fell�an,�V.,�Huotilainen,�M.�an���Win�ler,�I.�(2001a).�Event-relate����otential�correlates�of� soun�����uration:�Si�ilar��attern� fro��birth� to�a��ulthoo��.�NeuroReport,�12,�3777–3781.

Kushneren�o,�E.,�Cheour,�M.,�Ce�oniene,�R.,�Fell�an,�V.,�Renlun��,�M.,�Soininen,�K.,�Al�u�P.,�Kos�inen,�M.,�Sainio,�K.�an���Näätänen,�R.�(2001b).�Central�au��itor����rocessin���of���ura-tional�chan��es�in�co��lex�s�eech��atterns�b���ne�borns:�An�event-relate���brain��otential�stu����.�Developmental Neuropsychology,�19,�83–97.

Kushneren�o,�E.,�Ce�oniene,�R.,�Balan,�P.,�Fell�an,�V.�an���Näätänen,�R.�(2002).�Maturation�of�the�au��itor���chan��e���etection�res�onse�in�infants:�A�lon��itu��inal�ERP�stu����.�NeuroReport,�13,�1843–1848.

Kushneren�o,�E.,�Win�ler,�I.,�Horváth,�J.,�Näätänen,�R.,�Pavlov,�I.,�Fell�an,�V.�an���Huotilainen,�M.�(2007).�Processin���acoustic�chan��e�an���novelt���in�ne�born�infants.�European Journal of Neuroscience,�26,�265–274.

Le��änen,�P.�H.�T.,�E�lun��,�K.�M.�an���L����tinen,�H.� (1997).�Event-relate���brain��otentialsto�chan��e�in�ra�i��l����resente���acoustic�sti�uli�in�ne�borns.�Developmental Neurophysiology,�13,�175–204.�

Page 156: Unconscious Memory Representations in Perception

� Au��itor���learnin���in�the���evelo�in���brain� 145

Marcus,�G.�F.,�Vija��an,�S.,�Ban��i�Rao,�S.� an���Vishton,�P.�M.� (1999).�Rule� learnin��� in� seven-�onth-ol���infants.�Science,�283,�77–80.

Mart��nova,�O.,�Kirjavainen,�J.�an���Cheour,�M.�(2003).�Mis�atch�ne��ativit���an���late���iscri�i-native�ne��ativit���in�slee�in���hu�an�ne�borns.�Neuroscience Letters,�340,�75–78.

Ma��e,�J.,�Wer�er,�J.�F.�an���Ger�en,�L.�(2002).�Infant�sensitivit���to���istributional�infor�ation�can�affect��honetic���iscri�ination.�Cognition,�82,�B101–B111.

Morr,�M.�L.,� Shafer,�V.�L.,�Keruzer,� J.�A.� an���Kurtzber��,�D.� (2002).�Maturation�of��is�atch�ne��ativit��� in� t���icall��� ��evelo�in��� infants� an��� �reschool� chil��ren.� Ear and Hearing,� 23,�118–136.

Novits�i,�N.,�Huotilainen,�M.,�Tervanie�i,�M.,�Näätänen,�R.�an���Fell�an,�V.�(2007).�Neonatal�frequenc�����iscri�ination�in�250–4000-Hz�ran��e:�Electro�h��siolo��ical�evi��ence.�Clinical Neurophysiology,�118,�412–419.

Pan��,�E.�W.,�E���on��s,�G.�E.,�Desjar��ins,�R.,�Khan,�S.�C.,�Trainor,�L.�J.�an���Ta��lor,�M.�J.�(1998).�Mis�atch�ne��ativit��� to� s�eech� sti�uli� in�8-�onth-ol��� infants�an���a��ults.� International Journal of Psychophysiology,�29,�227–236.

Pih�o,�E.,�Le��änen,�P.�H.,�E�lun��,�K.�M.,�Cheour,�M.,�Guttor�,�T.�K.�an���L����tinen,�H.�(1999).�Cortical�res�onses�of�infants��ith�an����ithout�a���enetic�ris��for�����slexia:�I.�A��e�effects.�NeuroReport,�10,�901–905.

Ruusuvirta,�T.,�Huotilainen,�M.,�Fell�an,�V.�an���Näätänen,�R.� (2003).�The�ne�born�hu�an�brain�bin��s�soun���features�to��ether.�NeuroReport,�14,�2117–2119.

Saffran,�J.�R.,�Aslin,�R.�N.�an���Ne��ort,�E.�L.�(1996).�Statistical�learnin���b���8-�onth-ol���infants.�Science,�274,�1926–1928.

Saffran,�J.�R.,�Johson,�E.�K.,�Aslin,�R.�N.�an���Ne��ort,�E.�L.�(1999).�Statistical�learnin���of�tone�sequences�b���hu�an�infants�an���a��ults.�Cognition,�70,�27–52.

Saffran,�J.�R.,�Lo�an,�M.�M.�an���Robertson,�R.�R.�W.�(2000).�Infant��e�or���for��usical�ex�eri-ences.�Cognition,�77,�B15–B23.

Sa�beth,�A.,�Huotilainen,�M.,�Kushneren�o,�E.,�Fell�an,�V.�an���Pih�o,�E.�(2006).�Ne�borns���iscri�inate�novel�fro��har�onic�soun��s:�A�stu�����usin����a��netoence�halo��ra�h��.�Clini-cal Neurophysiology,�117,�496–503.

Sa�beth,�A.,�Ruohio,�K.,�Al�u,�P.,�Fell�an,�V.�an���Huotilainen,�M.�(2008).�Slee�in���ne�borns�extract��roso�����fro��continuous�s�eech.�Clinical Neurophysiology,�119,�332–341.

Tana�a,�M.,�O�ubo,�O.,�Fuchi��a�i,�T.�an���Hara��a,�K.�(2001).�A�stu�����of��is�atch�ne��ativit���in�ne�borns.�Pediatrics International,�43,�281–286.

Teinonen,�T.,�Aslin,�R.�N.,�Al�u,�P.�an���Csibra,�G.�(2008).�Visual�s�eech�contributes�to��honetic�learnin���in�6-�onth-ol���infants.�Cognition,�108,�850–855.

Teinonen,�T.,�Fell�an,�V.,�Näätänen,�R.,�Al�u,�P.�an���Huotilainen,�M.�(2009).�Statistical� lan-��ua��e�learnin���in�neonates�reveale���b���event-relate���brain��otentials.�BMC Neuroscience,�10,�21.

Tervanie�i,�M.,�Kallio,�J.,�Sin��onen,�J.,�Virtanen,�J.,�Il�onie�i,�R.�J.,�Salonen,�O.�an���Näätän-en,�R.�(2005).�Test-retest�stabilit���of�the��a��netic��is�atch�res�onse�(MMN�).�Clinical Neurophysiology,�116,�1897–1905.

Trainor,�L.�T.,�Sa�uel,�S.�S.,�Desjar��ins,�R.�N.�an���Sonna��ara,�R.�R.�(2001).�Measurin���te��oral�resolution�in�infants�usin����is�atch�ne��ativit��.�NeuroReport,�12,�2443–2448.

Trainor,�L.�J.,�Wu,�L.�an���Tsan��,�C.�D.�(2004).�Lon��-ter���e�or���for��usic:�Infants�re�e�ber�te��o�an���ti�bre.�Developmental Science,�7,�289–296.

Voulou�anos,�A.�an���Wer�er,�J.�F.�(2007).�Listenin���to�lan��ua��e�at�birth:�Evi��ence�for�a�bias�for�s�eech�in�neonates.�Developmental Science,�10,�159–164.

Page 157: Unconscious Memory Representations in Perception

14�� Minna�Huotilainen�an���Tuo�as�Teinonen

Wil�in,�P.�(1995).�A�co��arison�of�fetal�an���ne�born�res�onses�to��usic�an���soun���sti�uli��ith�an����ithout���ail���ex�osure�to�a�s�ecific��iece�of��usic.�Bulletin of the Council for Research in Music Education,�27,�163–169.

Win�ler,�I.,�Kusnheren�o,�E.,�Horváth,�J.,�Če�onienė,�R.,�Fell�an,�V.,�Huotilainen,�M.,�Näätän-en,�R.�an���Suss�an,�E.�(2003).�Ne�born�infants�can�or��anize�the�au��itor����orl��.�Proceed-ings of the National Academy of Sciences of the USA,�100,�11812–11815.

Page 158: Unconscious Memory Representations in Perception

chapter�6

Neurocomputational models of perceptual organization

Susan�L.�Denha�,*�Salva��or�Dura-Bernal,*�Martin�Coath*��an���E�ili�Bala��uer-Ballester***�Universit���of�Pl���outh,�UK�/�**�Universit���of�Hei��elber��,�Ger�an��

�.1 Introduction

Our��erce�tual� s��ste�s��rovi��e�us��ith�sensors�an���effectors� to��robe� the�ex-ternal��orl���an����rocessin���s��ste�s��hich�allo��us�to��a�e�sense�of�the��ealth�of�inco�in���infor�ation.�We�assu�e�here�that�the���oal�of��erce�tion�is�to�fin���si��lif��in���explanations�for�the�inco�in���si��nals�in�or��er�to�allo��us�to���etect,���ifferentiate�an����re��ict�the�behaviour�of�ani�ate�an���inani�ate�entities�in�the��orl��.�This�is�achieve���throu��h�the�for�ation�of�associations�bet�een���ifferent��arts�of�the�scene�an���bet�een���ifferent�events.�Perce�tual�s��ste�s�t���icall���re-ceive���iscontinuous�in�ut�sequences,�so�so�e��eans�for�lin�in�����iscrete�events�is� require��.� In� au��ition� this� is� self-evi��ent� as� the� si��nal� is� inherentl��� te��oral�an��,�for��ost�soun��s�of�interest,�inter�ittent.�Ho�ever,�even�thou��h��o��els�of�visual��erce�tion�often�treat�the��roble��as�one�of�se���entin���static�i�a��es,�the�nature�of� sacca��ic�e��e��ove�ents��eans� that� the�visual� s��ste��also��rocesses�sequences�of���iscrete�events�fro���hich�it�constructs�re�resentations�of�entities�in�the�environ�ent.�It�is�therefore�necessar���for��erce�tual�s��ste�s�to�create�an����aintain�te��orall����ersistent�re�resentations�of���etecte���entities.�It�is�these�re�-resentations��hich��e�consi��er�to�constitute�‘�erce�tual�objects’�an���the���oal�of��erce�tual�or��anisation.�

The�challen��e� for��erce�tual� s��ste�s� is� to� for��a��ro�riate� re�resentations�on�the�fl��,��hich�reflect�the�nature�of�external�entities�as�accuratel���as�necessar��.�I��ortantl��,�for�autono�ous�s��ste�s�this�also�requires�that�the�s��ste��has�so�e��eans�to�verif���its�re�resentations,�an���the�abilit���to�assess�the�efficac���of�its��o��-els�of�the��orl����ithout�su�ervision.�These�require�ents�constrain�an�����eter�ine�

Page 159: Unconscious Memory Representations in Perception

14�� Susan�L.�Denha��et�al.

the��rocessin���strate��ies�of��erce�tual�or��anisation�an���the�architecture�of��erce�-tual�s��ste�s�in�interestin����a��s.�

The� a��roach� that� �e� ar��ue� for� here� is� that,� as� ori��inall��� su����este��� b����Hel�holtz� (Hel�holtz�1860/1962),��erce�tion�can�be�un��erstoo���as�a��rocess�of�inference,�in��hich��robabilistic�inte��ration�of��rior��no�le����e�can�be�co�-bine����ith�an���influence�subsequent��rocessin��.�The�basic�i��ea�is�that��a�in����re��ictions�at��an���te��oral�an���featural�scales�is�an�effective�strate�����for���is-coverin���‘�hat’s�out�there’,�an���for�refinin���an���verif��in���the�accurac���of�re�re-sentations�of�the��orl��,�because�in�this��a���the��orl���can�act�as�its�o�n�chec�.�Mis�atches�bet�een�ex�ecte���an���actual�sensor���ex�erience�allo��us�to�i��en-tif���the�thin��s�that��e���on’t��no��about,�an���hence�fail�to��re��ict.�Unex�ecte���events�are�therefore�infor�ation�bearin���an���can�tell�us�ne��thin��s�about�the��orl��.�This�infor�ation�can�then�be�use���in�the�creation�an���refine�ent�or�u�-��atin���of�internal�re�resentations�or��o��els�of�the��orl��,��hich�in�turn�lea���to�better��re��ictions.�A�natural�consequence�of�these�i��eas�is�that�the��rocessin���architecture�an���sensitivities�shoul���reflect�the�structure�an���statistics�of�natural�sensor���in�uts�(Friston�2005;�Kiebel�et�al.�2008).�

In�su��ar��,��e�su����est�that�it�is�this��rocess�of�active�sensor���ex�loration�an����o��el�buil��in���that�un��er�ins�the���evelo��ent�an���on��oin���o�eration�of�intelli��ent��erce�tion.�The�i��ea�that�the�brain�is�continuall���an���auto�aticall���extractin����atterns�fro��the�inco�in���sensor���si��nals�is�stron��l���su��orte���b���recent�ex�eri�ental�fin��in��s�on�au��itor���co��nition�in�neonates;�su����estin���that�this� �rocessin��� strate����� is� innate.� Ex�eri�ents� have� sho�n� that� neonates� are�sensitive�to��atterns�at��an�����ifferent�ti�e�scales,�inclu��in���re��ularities�in�the�soun����avefor�� that���efine��itch� in��e�en��ent�of� ti�bre� (H�en�et� al.� 2009),�re��ularities�in�the�interval�relationshi�s�bet�een�successive��itches�(Stefanics�et�al.�2009),�re��ularities�in�the�relationshi��bet�een��itch�an���ti�bre��hich���efine�the��erceive���size�of�a�soun���source�(Vester��aar���et�al.�2009),�re��ularities�in�the�rh��th�ic�structure�of�events���efine���b���inter-onset�ti�in���intervals�(Win�ler�et�al.�2009)�an���re��ularities�in�tone�sequences���efine���b���re�eatin����atterns�of�notes�(Stefanics�et�al.�2007).�Thus�re��ularities�or��atterns,��hich��a���be���efine���b���a�ran��e�of� features�or� feature�conjunctions,�are�extracte���at��an�����ifferent�ti�es�scales�ri��ht�fro��birth,�if�not�before.

In�this�cha�ter,��e��ill�consi��er��o��els�of��erce�tual�or��anisation�in�the�vi-sual�an���au��itor����o��alities�an���tr���to��otivate�our�vie��of��erce�tion�as�a��ro-cess�of�inference�an���verification�throu��h�a�nu�ber�of�exa��les.�We�first��res-ent�so�e�basic�conce�ts�as�useful�buil��in���bloc�s�for�un��erstan��in����erce�tual�or��anisation,�an���then�sho��ho��these�i��eas�can��rovi��e�unif��in���insi��hts�into��o��els�of�visual�an���au��itor����erce�tion.�In�the�follo�in���sections��e�revie��a�nu�ber�of��o��els�of��erce�tual�or��anisation�an���focus�on�a��articular�class�of�

Page 160: Unconscious Memory Representations in Perception

� Neuroco��utational��o��els�of��erce�tual�or��anization� 149

�o��els,�hierarchical���enerative��o��els,��hich�have�at�their�core�notions�of��re-��iction,�verification�an����o��el�buil��in��,�or��anize����ithin�a�hierarchical�architec-ture.�Finall����e���iscuss�the��heno�enon�of��erce�tual�bistabilit����hich�is�foun���in�both��o��alities,�an���sho��ho��this�can�be� inter�rete����ithin�the��ro�ose�����eneric�fra�e�or�.

�.2 Perception as inference: Basic concepts

Althou��h�an�attractivel���si��le�i��ea,�inter�retin���an���i��le�entin����erce�tion�as� inference� is� a� co��lex��roble�,� requirin��� the��erce�tual� s��ste�� to���evelo��an� architecture� �hich� reflects� the� nature� of� the� external� �orl��;� i.e.� the� s��ste��requires�a�structure��hich�allo�s�it�to�re�resent�the��ulti-scale�nature�of�re��ulari-ties�foun���in�the�natural��orl��.�This�vie��of��erce�tion,��hich�has���aine���su��ort�fro���h��siolo��ical�stu��ies�an���has�ins�ire���a�nu�ber�of�co��utational��o��els,�hi��hli��hts�so�e�i��ortant��rocesses�that�are�necessar���for�a���eneric�inferential�architecture.�Here��e�outline�so�e�of�the�basic�conce�ts.

Change: Processin���ai�e���at�hi��hli��htin���chan��es�in�the�inco�in���activit���is��revalent��ithin�all�sensor���s��ste�s.�In�vision�a�chan��e�in�texture�or�contrast�t���icall�����efines�the�e����e�of�an�object,�an����an����o��els�of�e����e���etection��o-tivate���b���this�i��ea�have�been��ro�ose��;�e.��.�(Jehee�et�al.�2007).�In�the�au��itor���s��ste�,�shar�l���enhance���res�onses�are�foun���at�event�onsets�at�all�levels�of�the�s��ste�,�an����o��elin���chan��e���etection,��ri�aril���in�or��er�to�i��entif���event�on-sets,�has�been��i��el���stu��ie��;�e.��.�(Coath�et�al.�2005;�Fishbach�et�al.�2001).�

Regularities: Ho�ever,�chan��e� is�onl����eanin��ful��ithin�the�context�of�so�e�re��ularit��,�i.e.�chan��e�an���re��ularit���are�relative�conce�ts.�The��e���is�to�fin���an���re�resent��atterns�or�re��ularities�in�the�inco�in���activit���so�that�the�s��ste��is�sensitive�to�si��nificant�chan��es,�i.e.�ne��infor�ation,��hile�not�bein���unneces-saril���sensitive�to�insi��nificant�variations.�A��a�tation�to�sti�ulus�statistics�can�be�un��erstoo���fro��this��oint�of�vie�.�The�result�is�a�hei��htene���res�onse�to�sti�uli��hich���iffer�in�feature���istribution�fro��that�to��hich�the�s��ste��has�a��a�te��;�exa��les�inclu��e�the�au��itor���inferior�colliculus�(Dean�et�al.�2005)�an����ri�ar���so�atosensor���cortex�(Garcia-Lazaro�et�al.�2007).

Multi-scale analysis: Chan��e�an���re��ularit���are��eanin��ful�onl����ith-in�a��articular�ti�e�or�featural�scale,�an���the�scale�of�anal��sis�has�to�be�chosen�a��ro�riatel��.� For� exa��le,� the� ti�e� scale� necessar��� for� ��etectin��� the� �atterns��hich���efine��usical�for�,��oul���co��letel���obscure�the��atterns�in�the�ti�in���of�in��ivi��ual�events��hich�lea���to�a�sense�of�rh��th�,�or�the�even��ore�ra�i����eri-o��ic�variations�in��ressure��hich�lea���to�a�sense�of��itch;�conversel��,�the�s�atial/�featural�scale�necessar���for�resolvin���the�in��ivi��ual�letters�on�a��a��e�in�or��er�to�

Page 161: Unconscious Memory Representations in Perception

150� Susan�L.�Denha��et�al.

rea��� a� boo�� �oul��� ��enerate� far� too� co��lex� a� re�resentation� for� a��reciatin���lar��er� scale� objects.� Thus� a� �ulti-scale� anal��sis� of� the� �orl��� an��� the� abilit��� to�s�itch�flexibl���bet�een�scales�is�an�essential�as�ect�of��erce�tion.

Resolution versus integration: Althou��h�in�so�e�cases�it��a���be�neces-sar���to�inte��rate�infor�ation�over�lon���ti�e�scales�(or�lar��e�s�atial�scales)�in�or��er�to��erceive�re��ularities,�it�is�also�i��ortant�to�be�able�to�construct�shar��boun��-aries�bet�een�unrelate���objects;�i.e.��ixin���into�a��o��el�or�re�resentation�of�an�object�or�event�activit���belon��in���to�so�e�other�object�or�event��oul���conta�i-nate�the��o��el,�thereb���re��ucin���its��re��ictive��o�er.�This��roble��is��articularl���acute�at� the�boun��aries�of�objects�or�events;�an���a�shar��reset�of� the� lon���scale�inte��ration�is�require���if�infor�ation�relatin���to�so�e�other�object�is���etecte��.�

Spatial organisation of important features: The�extraction�an���re�-resentation�of�features�in�ter�s�of�so�e�for��of�to�o��ra�hic�or��anisation�is�an�intrinsic�as�ect�of�neural��rocessin���architectures.�Biolo��ical�sensor���s��ste�s�are�t���icall���or��anize���hierarchicall��,��ith��rocessin��� levels� sensitive� to� increasin���s�atial�(Felle�an�et�al.�1991)�an���te��oral�(Hasson�et�al.�2008)�scales�at�hi��her�levels�of�the�hierarch��.�This�allo�s�increasin��l����ore�co��lex�features,�s�annin���increasin��l���lon��er�ti�e�scales,�to�be�re�resente��.

Joint time/feature representations: Althou��h�ti�e��a���be�objectivel����easure��,� the� internal� re�resentation� of� ti�e,� ti�e� intervals� an��� ��urations,� is�also�fun��a�entall���i��ortant,�an���necessar���for���eneratin���a��ro�riatel���ti�e���behaviours,� for� exa��le.�An� i��ortant� insi��ht� co�in���out�of� au��itor����o��el-in���stu��ies�is�that�the�transfor�ation�of�ti�e�throu��h�the��rojection�into�a�to�-o��ra�hic� re�resentation� is� useful� at� �an��� ti�e� scales.� The� �re��iction� of� event�ti�in��� is� i��licit� in� this�re�resentation,�an�����eviations� in� the�ex�ecte���ti�in���of�events�can�be���etecte���b���chan��es�in�the��atterns�of�activit���across�the��a�,�rather�than�b���ex�licit�for�ar����rojections�in�ti�e.�Even��ore�i��ortantl���the�re�a��in���of�ti�e�can�be��a��e�at��an����laces�in�the��rocessin���hierarch��,��hich�lea��s�to�an�e�er��ent�joint�ti�e/feature�re�resentation,��hich��e�believe��a���be�a�fun��a�ental�as�ect�of��erce�tual��rocessin��.�

Competition between competing representations: Localize���lateral�inhibition�is�co��onl���foun���in�the�nervous�s��ste�,�an���ex�resses�a�local�co�-�etition�bet�een�co��etin���features��hich�can�serve�to�shar�en�re�resentations.�An� i��ortant�outco�e�of� this�or��anisation� is� that�finel���balance���co��etition�bet�een� ��ifferent� inter�retations� of� the� sensor��� in�ut� can� ensure� that� the� s��s-te��re�ains�in�a�critical�state,�easil���controllable�throu��h�to�-��o�n�si��nals.�An�influential��o��el�of�visual�attention,�the�‘biase���co��etition’��o��el,�is�base���on�the�i��ea�that�co��etition�at�all�levels�of�the�s��ste���rovi��es�a�substrate��hereb���attention�can�influence��rocessin���b����o��ulatin���or�biasin���the�co��etition�in�res�onse� to� tas�� ��e�an��s� (Bec�� et� al.� 2008;� Deco� et� al.� 2002;� Desi�one� et� al.�

Page 162: Unconscious Memory Representations in Perception

� Neuroco��utational��o��els�of��erce�tual�or��anization� 151

1995;�Duncan�1984),�causin��� the�s��ste��to�s�itch� to�or��aintain� the�require���or��anisation.�As���escribe���in�the�final�section�si�ilarities�bet�een�the�����na�ics�of�visual�an���au��itor���bistabilit���su����est�that�co��etition��a���be�a���eneric�as�ect�of�biolo��ical��rocessin��,�necessar���for�resolvin���the�a�bi��uities�inherent�in�the�sensor���in�ut.�

�.3 Hierarchical generative models in vision

It� has� lon��� been� a��reciate��� that� infor�ation� fallin��� on� the� retina� cannot� be��a��e���una�bi��uousl���bac��onto�the�real-�orl��;�ver�����ifferent�objects�can���ive�rise�to�si�ilar�retinal�sti�ulation,�an���the�sa�e�object�can���ive�rise�to�ver�����if-ferent�retinal�i�a��es.�So�ho��can�the�brain��erceive�an���un��erstan���the�outsi��e�visual��orl���base���on�these�a�bi��uous�t�o-��i�ensional�retinal�i�a��es?�A��os-sible� ex�lanation� co�es� fro�� the� ��enerative� �o��elin��� a��roach,� �hich� has� as�its���oal� the��a��in���of�external� causes� to� sensor��� in�uts.�B���buil��in��� internal��o��els�of�the��orl���the�brain�can�ex�lain�observe���in�uts�in�ter�s�of�inferre���causes.�This�in�turn�su����ests�the�visual�cortex��i��ht�have�evolve���to�reflect�the�hierarchical�causal�structure�of�the�environ�ent��hich���enerates�the�sensor�����ata�(Friston�2003b;�Friston�2005;�Friston�et�al.�2006a;�Friston�et�al.�2006b),�an���that�it�can�consequentl���e��lo����rocessin���analo��ous�to�hierarchical�Ba��esian�inference�to�infer�the�causes�of� its�sensations,�as���e�icte���in�Fi��ure�1.�Ma�in���inferences�about�causes���e�en��s�on�a��robabilistic�re�resentation�of�the���ifferent�values�the�cause�can�ta�e,�i.e.�a��robabilit�����istribution�of�the�causes.�This�su����ests�re�lacin���the�classical���eter�inistic�vie�,��here��atterns�are�treate���as�enco��in���a�feature�(e.��.�the�orientation�of�a�contour),��ith�a��robabilistic�a��roach��here��o�ulation�activit����atterns�re�resent�uncertaint���about�sti�uli�(e.��.�the��robabilit�����istribu-tion�over��ossible�contour�orientations).�

The��o��el��a�s��ell�onto�anato�ical,��h��siolo��ical�an����s��cho�h��sical�as-�ects� of� the� brain.� Visual� cortices� are� or��anize��� hierarchicall��� (Felle�an� et� al.�1991),� in� recurrent�architectures�usin�����istinct� for�ar���an���bac��ar���connec-tions� �ith� functional� as����etries.� While� fee��for�ar��� connections� are� �ainl�����rivin��,�fee��bac��connections�are��ostl����o��ulator���in�their�effects�(An��elucci�et�al.�2003;�Hu�e�et�al.�2001).�Evi��ence�sho�s�that�fee��bac��ori��inatin���in�hi��her�level�areas�such�as�V4,�IT�or�MT,��ith�bi����er�an����ore�co��lex�rece�tive�fiel��s,�can��o��if���an���sha�e�V1�res�onses,�accountin���for�contextual�or�extra-classical�rece�tive�fiel���effects�(Guo�et�al.�2007;�Harrison�et�al.�2007;�Huan���et�al.�2007;�Sillito�et�al.�2006).�As��e��ill�see�in�this�section,�hierarchical���enerative��o��els�are�re�iniscent�of�the���escribe���architecture,�sharin����an���of�the�structural�an���

Page 163: Unconscious Memory Representations in Perception

152� Susan�L.�Denha��et�al.

connectivit����ro�erties.�Moreover,�the����re��ict�basic�s��na�tic��h��siolo�����such�as�associative�an���s�i�e-ti�in��-��e�en��ent��lasticit��.

In�ter�s�of�the�neural��echanis�s�involve��,�althou��h�it�is�not���et��ractical�to�test�the��ro�ose���fra�e�or��in���etail,�there�are�so�e�relevant�fin��in��s�usin���functional��a��netic�resonance�i�a��in���(fMRI)�an���electro�h��siolo��ical�recor��-in��s.�Murra���et�al.� (Murra���et�al.�2004)�sho�e���that��hen�local� infor�ation�is��erce�tuall���or��anize���into��hole�objects,�activit���in�V1���ecreases��hile�activit���in�hi��her�areas�increases.�The���inter�rete���this�in�ter�s�of�hi��h-level�h���otheses�

Figure 1. Learne���internal��o��el�in�visual�cortex�reflects�hierarchical�causal�structure�of�the�environ�ent��hich���enerates�the�sensor���in�ut.�The�a�bi��uous�infor�ation��rovi��e���b���sensor���in�uts�(e.��.�2D�retinal�i�a��e)�is�onl���a�function�of�the�internal�state�of�the�Worl���(e.��.�3D�objects).�The�brain�(observer)�nee��s�to�inversel����a��this�func-tion�as��recisel���as��ossible�to���enerate�an�accurate�internal�re�resentation�of�the�Worl��.�The�hierarchical�or��anization�of�the�brain�su����ests�it�has�evolve���to�reflect�the�inherent�hierarchical�structure�of�the�Worl��.�(A��a�te����ith��er�ission�fro��Rao�1999.)

Page 164: Unconscious Memory Representations in Perception

� Neuroco��utational��o��els�of��erce�tual�or��anization� 153

or�causes�‘ex�lainin���a�a��’�the�inco�in���sensor�����ata.�Further,�Lee�an���Mu�for���(Lee�et�al.�2003)�stu��ie���the�te��oral�res�onse�of�earl���visual�areas�to���ifferent�visual�illusions,�conclu��in���there�are�increasin���levels�of�co��lexit���in�infor�a-tion��rocessin���an��� that� lo�-level�activit��� is�hi��hl��� interactive��ith� the� rest�of�the�visual�s��ste�.�Results�of�both�ex�eri�ents�are�consistent��ith�the���enerative��o��elin���a��roach.

The��ers�ective�of�Ba��esian�inference�also��rovi��es�a�unif��in���fra�e�or��for��o��elin���the��s��cho�h��sics�of�object��erce�tion�(Kersten�et�al.�2004;�Knill�et�al.�1996),�resolvin���its�co��lexities�an���a�bi��uities�b����robabilistic�inte��ration�of��rior�object��no�le����e��ith�i�a��e�features.�Si�ilarl��,�visual�illusions,��hich�are�t���icall���inter�rete���as�errors�of�so�e�i��recise�neural��echanis�,�can�in�fact�be�seen�as�the�o�ti�al�a��a�tation�of�a��erce�tual�s��ste��obe��in���rules�of�Ba��esian�inference�(Geisler�et�al.�2002).�In�further�su��ort�of�this�vie��(Kor��in���et�al.�2004)�conclu��e���that�the�central�nervous�s��ste��also�e��lo��s�an�inferential�a��roach���urin���sensori�otor�learnin��.�A����itionall��,��heno�ena�such�as�re�etition�su�-�ression�in�sin��le�unit�recor��in��s,��is�atch�ne��ativit���an���the�P300�in�electro-ence�halo��ra�h���can�also�be�ex�laine���in�this�fra�e�or��(Friston�et�al.�2006a).�It�is�not�sur�risin��,�therefore,�that�recent�revie�s�of�cortical�function�(Caran��ini�et�al.�2005;�Olshausen�et�al.�2005;�Sch�artz�et�al.�2007),��oint�in�the���irection�of��ore� ��lobal� a��roaches,� �hich� ta�e� into� account� the� fee��bac�� of� infor�ation�fro��hi��her�areas.

One�of�the�first�to��ro�ose�for�ulatin����erce�tion�in�ter�s�of�a���enerative��o��el��as�Mu�for���(Mu�for���1991,�1992,�1996)�,�base���on�i��eas�fro��Grena��-er’s��attern�theor���an���earlier�su����estions�b���Hel�holtz�(Hel�holtz�1860/1962).�A��lie���to�visual��erce�tion,�this�theor���states�that��hat��e��erceive�is�not�the�true�sensor���si��nal,�but�a�rational�reconstruction�of��hat�the�si��nal�shoul���be.�The�a�bi��uities��resent�in�the�earl���sta��es�of��rocessin���an�i�a��e,�never�beco�e�conscious�because�the�visual�s��ste��fin��s�an�ex�lanation�for�ever����eculiarit���of�the�i�a��e.�Pattern�theor���is�base���on�the�i��ea�that��attern�anal��sis�requires��attern�s��nthesis;�thereb���a����in���to�the��revious��urel���botto�-u��or�fee��for-�ar���structure,�a�to�-��o�n�or�fee��bac���rocess�in��hich�the�si��nal�or��attern�is�reconstructe��.

The�Hel�holtz��achine�(Da��an�et�al.�1995)�exten��e���these�i��eas�b���i��le-�entin���inferential��riors�usin���fee��bac�.�Here�the���enerative�an���reco��nition��o��els��ere�both�i��le�ente���as�structure���net�or�s��hose��ara�eters�have�to�be�learne��.�The�connectivit���of�the�s��ste��is�base���on�the�hierarchical�to�-��o�n�an���botto�-u��connections�in�the�cortex.�This�la��ere���hierarchical�connection-ist�net�or���rovi��es�a�tractable�i��le�entation�to�co��utin���the�ex�onential�nu�ber� of� �ossible� causes� un��erl��in��� each� �attern,� unli�e� other� a��roaches�such� as� the� Ex�ectation-Maxi�ization� al��orith�� �hich� run� into� �rohibitive�

Page 165: Unconscious Memory Representations in Perception

154� Susan�L.�Denha��et�al.

co��utational�costs.�The��e���insi��ht�is�to�rel���on�usin���an�ex�licit�reco��nition��o��el��ith�its�o�n��ara�eters�instea���of�usin���the���enerative��o��el��ara�eters�to��erfor��reco��nition�in�an�iterative��rocess.�

�.4 Predictive coding

A�si�ilar�a��roach,�ins�ire���b���conce�ts�of�Kal�an�filterin����as�ex�lore���b���(Rao�et�al.�1999).�In��re��ictive�co��in��,��hich�un��er�the�Ba��esian�fra�e�or��is���erive���fro���axi�izin���the��osterior��robabilit��,�each�level�of�the�hierarchical�structure�atte��ts�to��re��ict�the�res�onses�of�the�next�lo�er�level�via�fee��bac��connections.�The���ifference�bet�een�the��re��icte���an���actual�in�ut,�is�then�trans�itte���to�hi��h-er�or��er�areas�via�fee��for�ar���connections,�an���use���to�correct�the�esti�ate.�The��re��ictions�are��a��e�on��ro��ressivel���lar��er�scale�contexts,�so�for�exa��le,�if�the�‘surroun��’�can��re��ict�the�‘centre’,�little�res�onse�is�evo�e���b���the�‘error-��etectin��’�neurons.�In�other��or��s,��hen�to�-��o�n��re��ictions��atch�inco�in���sensor���in-for�ation,�the�lo�er-level�cortical�areas�are�relativel���inactive.�Ho�ever,��hen�the�central�sti�ulus�is�isolate���or���ifficult�to��re��ict�fro��the�surroun��in���context,�then�the�to�-��o�n��re��ictions�fail,�an���a�lar��e�res�onse�is�elicite��.�

The�basic�architecture,�sho�n�in�Fi��ure�2,��as�i��le�ente���usin���the�Kal-�an�filter�equation.�The��o��el�achieve���robust�se���entation�an���object�reco��-nition,�even��ith�nois���i�a��es�an���occlu��e���objects;�an����ana��e���to�sho��ho��rece�tive�fiel��s�(�o��el�basis�vectors)�si�ilar�to�those�re�orte���in�V1�si��le�cells,�coul���be�learnt�b���trainin���the�net�or���ith�natural�i�a��es.�It�also�ele��antl���ex-�lains�the�en��-sto��in���effect,�as�V1�error-si��nalin���neurons’�activit���is�re��uce���onl����hen�hi��her-levels�(�ith�lar��er�rece�tive�fiel��s)��ana��e�to�correctl����re��ict�the�centre�res�onse�usin���the�surroun���sti�ulus.

Figure 2. Pre��ictive�co��in���architecture�i��le�ente���usin���the�Kal�an�filter:�fee��bac���aths���enerate��re��ictions,��hile�fee��for�ar����aths�trans�it�the���ifferences�bet�een�the��re��icte���an���actual�in�ut;�i.e.�the��re��iction�errors.�(A��a�te����ith��er�ission�fro��Rao�an���Ballar���1999.)

Page 166: Unconscious Memory Representations in Perception

� Neuroco��utational��o��els�of��erce�tual�or��anization� 155

A�si�ilar��re��ictive�co��in���sche�e,�si�ulate���usin���a�t�o-la��er�neuronal�hi-erarch���(Friston�et�al.�2006a),��ana��e���to�re�ro��uce�the��heno�enon�of�re�eti-tion�su��ression�observe���in�the�brain.�The��o��el�illustrates�the�basic��rinci�les�of�a�rich�an���co��lex�cortical�theor����hich��ro�oses�that�brain�����na�ics�act�to��ini�ize�the�free�ener�����available,�or�un��er�si��lif��in���assu��tions,�su��ress��re��iction�errors.�A��l��in���hierarchical�Ba��esian��o��els� to� infer� the�causes�of�sensor���in�ut�arises�as�a�natural�consequence�of�this�theor��.�Su��ortin���ex�eri-�ental�evi��ence�sho�s�the�su��ression�of��re��iction�error�throu��h�fee��bac��con-nections�(Harrison�et�al.�2007)�as��re��icte���b���the�free�ener�����theor��.

�.5 Bayesian belief propagation (BBP)

A���ifferent�set�of�co��utational��o��els�are�base���on�Ba��esian�belief��ro�a��ation�(BBP),��hich�can�be�consi��ere���as�a���eneral�al��orith��for�the�i��le�entation�of�Ba��esian�inference.�In�this�a��roach��essa��es�(�robabilit�����istributions)�are��asse���bet�een�the��rocessin���no��es�of�a���ra�hical��o��el�(Ba��esian�net�or�)�to�co��ute�locall���each��osterior��robabilit��.�This�is�achieve���b���co�binin���at�each�no��e,�hi��her��ith�lo�er�level�evi��ence.�The�hierarchical�arran��e�ent�an���the�fact�that�all�co��utations�can�be�carrie���out�locall����a�es�the�al��orith��suitable�for�neural� i��le�entation.� In� this�section��e���escribe� three���ifferent�BBP��o��els��rovi��in���si��nificant�insi��hts�to�this�a��roach.

In�the�first�exa��le�of�this�a��roach,�it��as�sho�n�that�a��o��el,�i��le�ent-e��� as� a� sin��le-la��ere��� recurrent� net�or�,� �as� able� to� �erfor�� a� si��le� visual��otion���etection�tas��(Rao�2004,�2006).�The�firin���rate�of�each�neuron�enco��e���the�lo���of�the��osterior��robabilit���of�bein���in�a�s�ecific�state,��here�the���ifferent�states�re�resente���all��ossible�co�binations�of��osition�an�����irection�of��otion�of�the�sti�ulus.�To��o��el�the�li�elihoo���function,�the�in�ut�i�a��e�is�filtere���b���a�set�of�fee��for�ar����ei��hts�(��aussian�functions);��hile�the��rior�is�a��roxi�ate���b����ulti�l��in���the��revious��osterior��robabilit���b���a�set�of�recurrent��ei��hts��hich�re�resent�the�transition��robabilities�bet�een�states.�The��robabilit�����is-tributions�are�treate���as�the��essa��es�of�the�BBP�architecture.�The��ain�contri-bution�of�this��o��el�is�that�it��ana��e���to�i��le�ent�Ba��esian�inference�usin���equations�re�resentin���a�recurrentl���connecte���net�or��of�s�i�in���neurons.�The��o��el��as�later�exten��e���b���a����in���a�secon���la��er�of�Ba��esian���ecision-�a�in���neurons�that�calculate���a�lo��-�osterior�ratio�to��erfor��the�ran��o�-��ot��otion���etection� tas�.�A�si�ilar� i��le�entation�usin���a�3� level�hierarchical�net�or���ith�2� interconnecte����ath�a��s� for� features�an��� locations�(as�observe��� in� the�visual�s��ste�),��as�use���to��o��el�attention�(Rao�2004).

Page 167: Unconscious Memory Representations in Perception

15�� Susan�L.�Denha��et�al.

At�a��ore�abstract�level�the�BBP�a��roach��as�use���b���(Lee�et�al.�2003)�to�ex�lain��rocessin���in�the�ventral�visual��ath�a���(V1,�V2,�V4�an���IT).�The�visual�cortex��as�su����este���to�re�resent�beliefs�or�con��itional��robabilit�����istributions�on�feature�values,��hich�are��asse���for�ar���an���bac��ar���bet�een�the�areas�to�u���ate�each�other’s���istribution.�In�other��or��s,�each�area�is�assu�e���to�co�-�ute�or�infer�a�set�of�beliefs�base���on�the�i��e��iate�botto�-u����ata�conve��e���throu��h� the� fee��for�ar��� �ath�a��,� an��� the� to�-��o�n� ��ata� or� �riors� conve��e���throu��h�fee��bac��connections.�In�this��a��,�the�visual�areas�are�lin�e���to��ether�as�a�Mar�ov�chain,�so�that�each�area�is�continuall���u���ate���base���on�chan��es�to�the�con��itional��robabilities�in�both�lo�er�an���hi��her�areas�(e.��.�V2�is�influence���onl���b���V1�an���V4).�

This��o��el�i��le�ents�hierarchical�Ba��esian�inference�b���incor�oratin����ar-ticle�filterin��.�This��athe�atical�tool�a��roxi�ates�hi��h-��i�ensional��robabilit�����istributions�usin���a�set�of�sa��le��oints�or��articles�an���an�attache���set�of��ei��hts�that�re�resent�their��robabilities.�This�al��orith��has��rovi��e���outstan��in���results�in�co��uter�vision�an���artificial�intelli��ence,�an���is�ar��ue���to�be�a��ro�isin���tech-nique�for�statistical�inference�in�lar��e,��ulti��i�ensional���o�ains.�The�essential�i��ea�is�to�co��ute�for�each�area,�not�onl���one�h���othesis�for�the�true�value�of�its�sets�of�features,�but�a��o��erate�nu�ber�of�h���otheses.�This�allo�s��ulti�le�hi��h-�robabilit���values�to�sta���alive�until�lon��er�ran��e�fee��bac��loo�s�have�ha���a�chance�to�exert�an�influence.�Global�co��etition�involvin���all�as�ects�of�each�h���othesis�allo�s�the�s��ste��to�fin���the�one�that�o�ti�all���inte��rates�lo�-level�an���hi��h-level���ata,��a�in���the��robabilit���of�all�other��articles�su����enl���colla�se.

To�better�un��erstan���the�conce�t�of�ho��BBP�can�be�a��lie���to�visual��ro-cessin��,�consi��er�the�sha��o�e���face�exa��le�sho�n�in�Fi��ure�3.�Initiall��,�botto�-u��cues�fro��the�illu�inate����art�of�the�face�cause�a�‘face’�h���othesis�to�beco�e�activate���at�the�hi��her�levels.�Then�infor�ation�about�the�li�el���features�an����ro-�ortions�of�a�face�is�conve��e���throu��h�to�-��o�n�fee��bac��to�the�lo�er-level�hi��h�resolution�buffer.�Re-exa�ination�of�the���ata�results�in�a�reinter�retation�of�the�faint�e����e� in� the�sha��o�e���area�as�an� i��ortant��art�of� the� face�contour.�This�ne����etaile���infor�ation�can�then�be�use���b���the�hi��her�levels�to�infer�a����itional�characteristics�of�the�i�a��e,�such�as�the��recise�i��entit���of�the�face.�

It� is� i��ortant�to�also�note�that�the�Ba��esian�fra�e�or���rovi��es�a��a���of�reconcilin���t�o�existin���contra��ictor���a��roaches,�i.e.�a��a�tive�resonance�or�bi-ase���co��etition,��here�res�onses�are�shar�ene���(‘sto����ossi�in��’�effect),��ith��re��ictive�co��in��,��here�res�onses�are�su��resse�����ue�to�hi��h-level�ex�lainin���a�a���(‘shut-u�’�effect).�The���ile��a�is�resolve���b���consi��erin���t�o�functionall�����istinct� sub�o�ulations,� one� enco��in��� the� causes� of� the� sensor��� in�ut� or� cur-rent��re��iction;�an���the�other�enco��in���the��re��iction�error�use���to�refine�the��robabilit�����istribution�of�causes.�The�co��atibilit���of�these�t�o�a��roaches�has�

Page 168: Unconscious Memory Representations in Perception

� Neuroco��utational��o��els�of��erce�tual�or��anization� 157

recentl���been��athe�aticall����roven,�an���it�has�been�hi��hli��hte���that��revious�a��arent�inconsistencies��a���have�resulte���fro��the�stron���e��hasis��re��ictive�co��in����laces�on�error-��etectin���no��es,�an���the�corres�on��in���un��er-e��hasis�on��re��iction�no��es�(S�ratlin���2008).�The��re��iction�no��es,��hich�can�be�con-si��ere���equivalent�to�the�Belief�no��es,�are�res�onsible�for��aintainin���an�active�re�resentation�of� the�sti�ulus�an����ill� sho��an�enhance���res�onse��hen� to�-��o�n��no�le����e�correctl����re��icts�the�in�ut;��hile,�at�the�sa�e�ti�e,�the�error-��etectin���no��es�sho��su��ression.

An� interestin��� architecture,� an��� one� �hich� ta�es� into� account� te��oral� as��ell�as�s�atial� infor�ation,� is� the�Hierarchical�Te��oral�Me�or���(HTM)��ro-�ose���b���(Ha��ins�et�al.�2007).�A��ain,�this��o��el�assu�es�that�i�a��es�are���en-erate���b���a�hierarch���of�causes,�an���that�a��articular�cause�at�one�level�unfol��s�into�a�sequence�of�causes�at�a� lo�er� level.�HTMs�therefore�atte��t�ca�ture�the�al��orith�ic�an���structural��ro�erties�of�the�cortex,��hich�in�turn�are�ar��ue���to��atch�the�causal�hierarch���of�i�a��e���eneration�(Fi��ure�1).�When�an�HTM�sees�a�novel�in�ut�it���eter�ines�not�onl���the��ost�li�el���hi��h-level�cause�of�that�in�ut,�but�also�the�hierarch���of�sub-causes.

Figure 3. Ba��esian�belief��ro�a��ation�architecture.�(a)�Initiall��,�botto�-u��cues�fro��the�illu�inate����art�of�the�face�(B1)�cause�a�‘face’�h���othesis�to�beco�e�activate���at�the�hi��her�levels.�Then�infor�ation�about�the�li�el���features�an����ro�ortions�of�a�face�is�conve��e���throu��h�to�-��o�n�fee��bac��(B2)�to�the�lo�er-level�hi��h�resolution�buffer.�Re-exa�ination�of�the���ata�results�in�a�reinter�retation�of�the�faint�e����e�in�the�sha��o�e���area�as�an�i��ortant��art�of�the�face�contour.�(b)�Each�area�co��utes�a�set�of�beliefs,�Xi,�base���on�botto�-u��sensor�����ata�(Xi–1)�an���to�-��o�n��riors�(P(Xi/Xi+1),��hich�are�inte��rate���accor��in���to�the�Ba��esian�inference�equation.�Beliefs�are�continuall���u���ate���accor��in���to�chan��es�in�earlier�an���hi��her�areas�to�obtain�the��ost��robable���istribution�of�causes�at�each�level.�(A��a�te����ith��er�ission�fro��Lee�an���Mu�for���2003.)

Page 169: Unconscious Memory Representations in Perception

15�� Susan�L.�Denha��et�al.

Durin���a�learnin���sta��e,�HTMs�atte��t�to���iscover�the�causes�un��erl��in���the�sensor�����ata.�Each�no��e�consists�of�a�s�atial��ooler�that�learns�the��ost�co��on�in�ut�s�atial��atterns,�an���a�te��oral��ooler�that���rou�s�these��atterns�accor��in���to�their�te��oral��roxi�it���an���assi��ns�the��a�label.�For�exa��le�a�set�of�corner�lines�at���ifferent��ositions�(in�ut�s�atial��atterns),�coul���be���rou�e���into�a�co�-�on�te��oral���rou��labele���‘corner’.�To���o�this�the�in�ut��ust�be�a��otion��ic-ture��ith�objects��ovin���aroun��,�so�that�te��oral�infor�ation�can�be�extracte��.�The�s�atial��ooler�in�the��arent�no��e�co�bines�the�out�ut�of�several�lo�er-level�no��es,�i.e.�a��robabilit�����istribution�over�the�te��oral���rou�s�of�those�no��es.�This�allo�s�it�to�fin���the��ost�co��on�co-occurrin���te��oral���rou�s�belo�,��hich�then�beco�e�the�al�habet�of�‘s�atial��atterns’�in�the��arent�no��e,�e.��.�features�of�a�face�(e��es,�nose,��outh,�...)��hich�al�a��s��ove�to��ether.�The�learnin����rocess�is�re�eate���throu��hout�the�hierarch���to�obtain�the�causes�at�the�hi��hest�level.�As�a�result�a�tree�structure���Ba��esian�net�or��is�obtaine��,�base���on�the�s�atio-te��o-ral�characteristics�of�the�in�uts.

For�the�inference�sta��e,�as�hi��her�levels�have�conver��in���in�uts�fro���ulti�le�lo�-level�re��ions,�several��arallel�an���co��etin���h���otheses��ill�finall���conver��e�on�a�hi��h-level�cause�lea��in���to�reco��nition.�This�allo�s��rior�infor�ation�relate���to�that�object�to���isa�bi��uate�or�ex�lain�a�a���lo�er�level��atterns.�The�si�ula-tion�results�for�a�line���ra�in���reco��nition�s��ste�,�usin���this��o��el,�sho�e���ver���robust�scale,�translation�an�����istortion�invariance�even�for�ver���nois���in�uts.�It�also��rovi��es�a�Ba��esian�ex�lanation�for�the�ex�eri�ental�fin��in���that��hen�ac-tivit���is�increase���in�hi��her�level�object��rocessin���areas,�the�res�onses�in�lo�er�levels� is� re��uce��.� The� reason� is� that� the� hi��h-level,� �ore� ��lobal� inter�retation�narro�s�the�h���othesis�s�ace��aintaine���b���lo�er�level�re��ions,��hich�lea��s�to�a�re��uction�in�total�activit��.�

The��ain���ifference�bet�een�HTM�an���other��robabilistic��o��els,�such�as�the�Hel�holtz��achine,�is�the�use�of�te��oral�infor�ation�to�achieve��osition�an���scale�invariance.�A�si�ilar�invariance�a��roach��as�a��o�te���in�H�ax,�a�biolo��i-call���realistic�fee��for�ar����o��el�of�vision��ro�ose���b���Riesenhuber�et�al.�(1999),��hich�has��rove���successful�for�un��erstan��in���both��erce�tual�an����h��siolo��ical�as�ects�of�object�reco��nition�(Ca��ieu�et�al.�2007;�Serre�et�al.�2007).�In�fact�this��o��el�shares�the�sa�e�hierarchical�structure,�inclu��in���si��le�an���co��lex�la��-ers��hich�closel���rese�ble�HTM’s�s�atial�an���te��oral���rou�s.�Units�in�the�lo�er�level,��o��elle���usin���Gabor�filters,�sho��rece�tive�fiel���sizes,�s�atial�frequencies�an��� orientation� ban���i��ths� consistent� �ith� V1� si��le� cells.� These� are� �oole���over�s�ace�an���scale�usin���a�max�o�eration�to��o��el�the��ro�erties�of�co��lex�cells�(�osition�an���size�invariance).�Grou�s�of�co��lex�cells�are�co�bine���to�for��object�features,��hich�are�then�use���as��rotot���es�to�co��ute�the�inter�e��iate�level’s�res�onse�b����erfor�in���a�te��late-�atchin���o�eration�(selectivit��).�The�

Page 170: Unconscious Memory Representations in Perception

� Neuroco��utational��o��els�of��erce�tual�or��anization� 159

invariance�an���selectivit���o�erations�are�re�eate���u��the�hierarch��,�as�observe���in�the�ventral��ath�a��,�resultin���in�invariant�object-tune���units��hich�rese�ble�neurons�in�inferote��oral�cortex.�

Recentl����e�have�sho�n�that�this�fee��for�ar���architecture�can�also�be�inter-�rete���in��robabilistic�ter�s,�an���exten��e���it�to�inclu��e�fee��bac��usin���the�BBP�al��orith��(Dura�et�al.�2008).�Fee��for�ar���res�onses�are�un��erstoo���as��robabil-it�����istributions�over�the�set�of�features�at�the���ifferent��ositions,��hich�co�bine�hierarchicall���to�obtain,�at�the�to��level,�the��ost��robable�object�re�resente���b���the� in�ut� i�a��e.�The�store��� invariant��rotot���e�of� this�object� is� then� fe���bac����o�n�the�net�or��an���co�bine����ith�the�fee��for�ar���res�onse�at�each�level�to�obtain�the�resultin���belief�(see�Fi��ure�4).�Ho�ever,�hi��h-level�invariant�object��ro-tot���es���enerate�a�bi��uous�an�����iffuse�fee��bac���hich�nee��s�to�be�refine��.�The���isa�bi��uation��rocess,�base���on�cues�fro��the�existin���fee��for�ar���res�onses,�uses� a� local� extra�olation� al��orith�� �hich� ex�loits� collinearit��,� co-orientation�an�����oo���continuation�to���ui��e�an���a��a�t�the�fee��bac��(see�Fi��ure�5).�Previous�stu��ies�have�sho�n�a��lausible�i��le�entation�of�the�interactions�bet�een�fee��-bac��an���lo�er-level��erce�tual���rou�in���b���the�la�inar�circuits�of�V1�an���V2�(Raiza��a�et�al.�2003).�The�resultin���lo�er�level�beliefs,�after�to�-��o�n�influence,�are�then�fe���for�ar���to�co��lete�the�recurrent�c��cle��hich���ra��uall���buil��s�the�o�ti�al�beliefs�in�each�level.

The� �ro�ose��� �o��el� �erfor�s� successful� fee��for�ar��� object� reco��nition,�inclu��in��� cases� of� occlu��e��� an��� illusor��� i�a��es.� Reco��nition� is� both� �osition�an���size�invariant.�The��o��el�also��rovi��es�a�functional�inter�retation�of�fee��-bac��connectivit����hich�accounts�for�several�observe����heno�ena.�The��o��el�res�onses�qualitativel����atch�re�resentations�in�earl���visual�cortex�of�occlu��e���an���illusor���contours�(Lee�et�al.�2001;�Rauschenber��er�et�al.�2006),�resultin���fro��hi��her� levels� i��osin��� their� �no�le����e� on� lo�er� levels� throu��h� fee��bac�;� an���fMRI���ata�sho�in���that��hen�activit���is�increase���in�hi��h-level�object��rocessin���areas,�the�res�onse�in�lo�er�levels�is�re��uce��,�as�fee��bac��narro�s�the�h���othesis�s�ace��aintaine���b���lo�er�re��ions.�A����itionall���a�����na�ic��echanis��for�illu-sor���contour�for�ation,��hich�can�a��a�t�a�sin��le�hi��h-level�object��rotot���e�to�Kanizsa’s�fi��ures�of���ifferent�sizes,�sha�es�an����ositions,�is��ro�ose��.�The��echa-nis��is�consistent��ith�a�recent�revie��on�illusor���contours�(Hal�o�et�al.�2008),�an���is�su��orte���b���ex�eri�ental�stu��ies��hich�su����est�that�interactions�bet�een���lobal� contextual� fee��bac�� si��nals� an��� local� evi��ence,� �recisel��� re�resente��� in�V1,�is�res�onsible�for�contour�co��letion�(Lee�et�al.�2001;�Stanle���et�al.�2003).�B���i��osin���to�-level��riors�the��o��el�can�also�si�ulate�the�effects�of�s�atial�atten-tion,��ri�in���an���visual�search.

Page 171: Unconscious Memory Representations in Perception

1�0� Susan�L.�Denha��et�al.

Figure 4. Dia��ra��of�hierarchical��o��el�of�object�reco��nition�(Dura�et�al.�2009).�A�Kanizsa’s�square�in�ut�i�a��e�is�use���to�illustrate�the�out�ut�of�the��o��el�at���ifferent��oints.�Units�is�the�lo�er�level�(S1),��o��elle���usin���Gabor�filters,�sho��tunin����ara�e-ters�consistent��ith�V1�si��le�cells.�Their�out�uts�are��oole���over�s�ace�an���scale�usin���a�max�o�eration�to��o��el�the��ro�erties�of�co��lex�cells�(S2)��hich�sho���osition�an���size�invariance.�Grou�s�of�co��lex�cells�are�co�bine���to�for��object�features,��hich�are�then�use���as��rotot���es�to�co��ute�the�inter�e��iate�level’s�res�onse�(S2�an���C2)�b����erfor�in���a�te��late-�atchin���o�eration�(selectivit��).�The�invariance�an���selectivit���o�erations�are�re�eate���u��the�hierarch���resultin���in�invariant�object-tune���units�(S3�an���C3)��hich�rese�ble�neurons�in�inferote��oral�cortex.�Fee��for�ar���res�onses�can�also�be�un��erstoo���as��robabilit�����istributions�over�the�set�of�features�at�the���ifferent��ositions,��hich�co�bine�hierarchicall���to�obtain,�at�the�to��level,�the��ost��robable�

Page 172: Unconscious Memory Representations in Perception

� Neuroco��utational��o��els�of��erce�tual�or��anization� 1�1

Figure 5. Fee��bac����isa�bi��uation�usin���cues�fro��the�fee��for�ar���res�onse�(Dura�et�al.�2009).�Left:�Fee��bac��fro��sin��le�hi��h-level��rotot���e�co��letin�����istorte���versions�of�the�Kanizsa’s�square.�Note�the�contour�co��letion��ill�onl���occur��hen�the�variations�lies��ithin�the��osition�an���scale�invariance�ran��e�of�the�hi��h-level��rotot���e;�an����hen�there�is�enou��h�local�evi��ence�to���ui��e�the�local���isa�bi��uation��rocess�(inter�olation/extra�olation).�The�secon���con��ition�is�not�satisfie���in�the�fi��ure��ith�blurre���e����es.�Right:�Exa��le�of�fee��bac����isa�bi��uation�for�a�Kanizsa�in�ut�i�a��e�s�aller�than�the�store����rotot���e.�The�fee��bac��si��nal�C1F�is���erive���fro��a�chain�of��rocesses�startin���at�the�hi��hest�level��here�the�invariant�object��rotot���e�of�the�square�is�fe���bac�,�as�sho�n�in�Fi��ure�4.�The��recise�s�atial�infor�ation�containe���in�S1��rovi��es�the�cues�require���to�further�refine�C1F�an�����enerate�S1F.�In�this��a��,�Kanizsa’s�illusor���squares�of���ifferent�sizes�can�be�co��lete���usin���the�sa�e�hi��h-level�square��rotot���e.

object�(square)�re�resente���b���the�in�ut�i�a��e�(Kanizsa’s�illusor���square).�The�store���invariant��rotot���e�of�this�object�is�then�fe���bac��(C3F�an���S3F)���o�n�the�net�or��an���co�bine����ith�the�fee��for�ar���res�onse�at�each�level�to�obtain�the�resultin���Belief�(BS3,�BS2�an���BS1).�These�are�then�fe���for�ar���bac��u��the�net�or�,���eneratin���a�recurrent��rocess��hich,���ue�to�the�overla��in���of�units�an���the�extra�olation��echa-nis��bet�een�C1F�an���S1F,�buil��s�u��the�contour�of�the�illusor���square�in�the�lo�er�levels�as�observe���ex�eri�entall��.

Page 173: Unconscious Memory Representations in Perception

1�2� Susan�L.�Denha��et�al.

�.� Hierarchical generative models in auditory perception

The�i��ea�of�usin����re��ictive�co��in���techniques�in��rocessin���soun��s�is�not�ne�,�an���in�the�1960’s��as�first�a��lie���to�s�eech�co��ression.�The�i��ea��as�to��re��ict�each�ne��soun���sa��le�b���a��ei��hte���su��of��rece��in���sa��les,��ith�the��ei��hts���eter�ine���a��a�tivel���to��ini�ise�the��re��iction�error�(Schroe��er�1999).�Since�s�eech�soun��s,�an���hence�the��ei��hts,�chan��e��ore�slo�l���than�the�sa��lin���rate�of�the�si��nal,�a��ore�efficient�re�resentation�coul���be�achieve��.�This�technique�is�also�relevant�to�s�eech�reco��nition,�since�the��ei��hts�carr���infor�ation�about�the�soun���uttere��,�an���can�therefore�be�use���to�classif���the�soun��.�Further�ore,�the�hierarchical�nature�of�co��unication�si��nals� le���naturall��� to� the���evelo��ent�of�hierarchicall���structure���s�eech�reco��nition�s��ste�s.�At��ro��ressivel���hi��her�levels�of�the�s��ste�,�sequences�of�s�eech�soun��s,�re�resente���t���icall���b���hi����en�Mar�ov��o��els�are�transfor�e���into��or��s,�an���at�even�hi��her�levels,�lan��ua��e��o��els�are�use���to�transfor���or���sequences�into��hrases�or�sentences.�Infor�a-tion�fro��hi��her�levels�is�use���to�constrain�an�����ui��e�the�choices��a��e�at�lo�er�levels.�Most�auto�atic�s�eech�reco��nition�s��ste�s�have�been�constructe���to�solve�an�en��ineerin����roble��rather�than�to�ex�lain�neural��rocessin��,�nevertheless,�the�resultin���solutions�are�rather�si�ilar.�This��a�es�sense�as�the�constraints�are���eter�ine���b���the�structure�of�the�si��nal.

As� �e� have� �entione��� �reviousl��� an� ex�licit� account� of� cortical� or��anisa-tion�in�ter�s�of�hierarchical��re��ictive�co��in����as��ro�ose���b���Friston�(Friston�2003a,�2005).�In�this��o��el,�at�each�level�there�are�error�an���ex�ectation�units,�fee��for�ar���si��nals�enco��e��re��iction�errors�fro��the�level�belo�,�an���to�-��o�n�si��nals��rovi��e��o��ulator���infor�ation�re��ar��in����rior�ex�ectations.�More�re-centl���this��o��el��as�exten��e���to�ex�lore�the�notion�that�the�cortical�anato����shoul���reflect�the�te��oral�hierarch���of�the�environ�ent�(Kiebel�et�al.�2008).�The�vali��it���of�these�i��eas��ere���e�onstrate���in�a��o��el�of�bir��son���learnin����hich��as�able�to�extract�te��oral�structure�at�t�o�ti�e�scales�fro��a�ra�i��l���chan��in���sensor���in�ut.�The�h���othesis�is�that�sensor���s��ste�s�have�evolve���to�infer�the�causes�of�sensor���in�ut�throu��h�the�for�ation�of�re�resentations��hich��rovi��e�te��orall���stable��re��ictions�about�future�in�ut.�In�this�account�it�is�su����este���to�use�the�ter�,�‘conce�t’�to�refer�to�a�state��hich�exists�for�so�e�ti�e,�an���‘�erce�t’�to�refer�to��ore�transient�fluctuations.�Conce�ts��rovi��e�the�contextual�attractors��ithin��hich�the�lo�er�level�trajectories�unfol��.�In�essence,�a�conce�t�at�one�level,�subsequentl���beco�es�a��erce�t�in�relation�to�the�level�above,�an���so�on.�Co�-�etition�bet�een�re�resentations�at�each� level�ensures� that�sensor���s��ste�s�act�as�a�set�of�neste��,�internall���consistent�����na�ical�s��ste�s.�This�issue�of�internal�consistenc��/inconsistenc���is�one�that��e��ill�return�to���urin���the���iscussion�of��erce�tual�bistabilit��.�Clearl��,�the�notion�of�conce�ts�an����erce�ts���escribe���in�

Page 174: Unconscious Memory Representations in Perception

� Neuroco��utational��o��els�of��erce�tual�or��anization� 1�3

this��a���coinci��e��ith�i��eas�of�objects�an���object��arts.�The���istinction��a���be�useful�thou��h�as�it�hi��hli��hts�the�te��oral���ifferences�bet�een�re�resentations�at�successive�levels�in�the�hierarch��.

�.�.1 Pitch

An�as�ect�of�sensor����rocessin����hich�to�our��no�le����e�has�not�been��reviousl���stu��ie����ithin�the�hierarchical���enerative�fra�e�or��is�the�nee���to�consi��er�ho��a��ulti�ti�e-scale��o��el�can�co�e��ith�abru�t�chan��es�in�context;�i.e.�a�chan��e�in�the�un��erl��in���cause.�For�exa��le,�in�a�conversation�bet�een�t�o�s�ea�ers,�it�is�necessar���to�ensure�that�si��nals�fro��one��erson�are�not�confuse����ith�those�fro��the�other.�A��o��el��hich�uses�fixe���te��oral�inte��ration��in��o�s��oul���inevitabl���at�so�e�level��ix�the�t�o.�We�have�recentl���consi��ere���this��roble��at�a��uch�lo�er�level�of�the��erce�tual�hierarch���in�the�for�ation�of�a�hierarchical��re��ictive��o��el�of��itch��erce�tion�(Bala��uer-Ballester�et�al.�2009�).

Pitch,�one�of�the��ost�i��ortant�features�of�au��itor����erce�tion,�un��erl��in���the��erce�tion�of��elo�����in��usic�an����roso�����in�s�eech,�is�usuall���associate����ith��erio��icities�in�the�soun���si��nals�(Moore�2004).�Hence,�a�nu�ber�of��o��-els�of��itch��erce�tion�are�base���u�on�a�te��oral�anal��sis�of�the�neural�activit���evo�e���b���the�sti�ulus�(Cariani�et�al.�1996a,�1996b;�Lic�li��er�1951;�Me����is�et�al.�1991,�1997;�Slane���et�al.�1990).�As�su����este���in�the�intro��uction,�a�useful��a���to�thin��about�au��itor����erce�tion�is�in�ter�s�of�the�iterative�extraction�of�te��oral�re��ularities���etecte���in�the�in�ut�at�each�sta��e�of��rocessin��,�an���the��rojection�of�these�te��oral��atterns�alon���a�s�atial�axis�or��ere���in�ter�s�of�the�scale�of�the�re��ularit��.�For�exa��le,�at�the�first�sta��e�of��rocessin��,�the�re��ularities�inherent�in� the� in��ivi��ual� frequenc���co��onents�of� the� soun��s�are��rojecte���alon��� the�cochlea�scale���fro��short�(hi��h�frequenc��)�to�lon���(lo��frequenc��)�ti�e�scales.�The��erio��icities�u�on��hich�the�sense�of��itch���e�en��s�are�essentiall���secon���or��er� re��ularities� �hich� can� be� extracte��� b��� �rojectin��� re��ularities� in� the� �at-terns�of�au��itor���nerve�activit���onto���et�another�axis�or��ere���fro��short��erio��s�(hi��h��itches)�to�lon����erio��s�(lo���itches),��ithin�an�initiall���s�ectrall���localise���or��anisation.�The�subsequent�extraction�of��itches,�can�be���erive���b���co�binin����erio��icit����atterns��ithin�s�ectral�clusters��hich�are�for�e���throu��h�the���etec-tion�of�shar����iscontinuities�in�the�te��oral��atterns�across���ifferent�frequenc���channels�(Bala��uer-Ballester�et�al.�2007).�

A�si��le��athe�atical�technique�for�extractin����erio��icit���re��ularities�is�au-tocorrelation,��hich�hence�for�s�the�basis�for��ost�te��oral��o��els�of��itch��er-ce�tion.�Most�of�these��o��els�co��ute�a�for��of�short-ter��autocorrelation�of�the�si�ulate���au��itor���nerve�activit���usin���an�ex�onentiall����ei��hte���inte��ration�

Page 175: Unconscious Memory Representations in Perception

1�4� Susan�L.�Denha��et�al.

ti�e��in��o��(Bala��uer-Ballester�et�al.�2008;�Bernstein�et�al.�2005;���e�Chevei��ne�et�al.�2006;���e�Chevei��né�2005;�Me����is�et�al.�1991,�1997).�Autocorrelation��o��els�have�been�able�to��re��ict�the�re�orte����itches�of�a��i��e�ran��e�of�co��lex�sti�uli.�Ho�ever,�choosin���an�a��ro�riate�inte��ration�ti�e��in��o��has��rove����roble�-atic.�The��roble��is�that�in�certain�con��itions,�the�au��itor���s��ste��is�ca�able�of�inte��ratin����itch-relate���infor�ation�over�ti�e�scales�of�several�hun��re����illi-secon��s�(Hall�et�al.�1981),��hile�at�the�sa�e�ti�e�it�is�also�able�to�follo��chan��es�in��itch�or��itch�stren��th��ith�a�resolution�of�onl���a�fe���illisecon��s�(Wie��rebe�2001).�This�sti�ulus-��e�en��ent�nature�of��itch��erce�tion�su����ests�that�a�si�-�le�fixe���fee��-for�ar����o��el�can�never�a��equatel���account�for�the��heno�enon.�One��a���to�thin��about�it�is�as�a�chan��e���etection��roble�;�i.e.�if�the��attern���ou�are�intereste���in�e�er��es�over�a�fairl���lon���ti�e,�ho��is�it��ossible�to�for��et�the�ol���an���no�lon��er�relevant��attern�an���establish�the�ne��one�quic�l��?�This�su����ests�a�for�ulation�of��itch��rocessin���in�ter�s�of�a���enerative��o��ellin���a��roach.

The��ro�ose����o��el�consists�of�a�fee��-for�ar���as��ell�as�a�fee��bac���rocess,��hich��o��ifies�the��ara�eters�of�fee��-for�ar����rocessin���(Bala��uer-Ballester�et�al.�2009).�The�role�of�the�fee��-for�ar����rocess�is�to���etect��itches�in�the�inco�in���sti�ulus.�This�is�achieve���b���extractin���the��erio��icities��ithin�each�frequenc���channel�usin���autocorrelation�at�relativel���short�ti�e�scales,�an���then�inte��ratin���the�out�ut�of�this�sta��e�at�the�next�level�in�the�hierarch���usin���a��uch�lon��er�ti�e�constant.�In�contrast�the�role�of�the�fee��bac���rocessin���is�to���etect�unex�ecte���chan��es�in�the�in�ut�sti�ulus,�such�as�the�offset�of�a�tone�in�a�sequence,�an���to��o��ulate�the�te��oral�inte��ration��in��o�s�of�the�fee��-for�ar����rocessin����hen�such�chan��es�occur.�A�sti�ulus�chan��e�t���icall���requires�a�fast�s��ste��res�onse,�so� that� infor�ation� occurrin��� aroun��� the� ti�e� of� the� chan��e� can� be� u���ate���quic�l��.� Thus,� ��urin��� �erio��s� �hen� there� is� a� si��nificant� ��iscre�anc��� bet�een�the�current�an���ex�ecte����itch�esti�ates,�the�effective�inte��ration�ti�e��in��o�s�beco�e�ver���short,�so�that�the�“�e�or��”�co��onent�of�the��o��el�res�onse�is�re��uce���to�near�zero�an���essentiall���reset.�

The��rinci�al�novelt���of�the��ulti-scale��re��ictive��o��el��as�the�intro��uc-tion�of�fee��-bac���o��ulation�of�the�recurrent�inhibitor����rocesses��hich�o�erate�at�each�level�(Bala��uer-Ballester�et�al.�2009)�.�These�to�-��o�n�si��nals�serve�to�alter�the�“effective”� inte��ration��in��o�s�use���at� the���ifferent�sta��es.�As�a�result,� the�res�onses�of�the��o��el�a��a�t�to�recent�an���relevant�chan��es�in�the�in�ut�sti�u-lus.�This�a��roach�is�consistent��ith�the�available�neuroi�a��in�����ata:�a�sustaine����itch� res�onse� (SPR)� in� lateral�Heschl’s�G��rus�has�been�sho�n� to�a��a�t� to� the�recent�te��oral�context�of�a��itch�sequence,�enhancin���the�res�onse�to�rare�an���brief� events� (Gutschal��et� al.� 2007).�The�ra�i��� reset� cause���b����is�atches�be-t�een�the��re��ictions�at�various�levels�of�the��o��el�is�i��ortant�because�it�allo�s�the�re�resentation�of�ne��infor�ation�to�buil���u���ith�as�little�conta�ination�as�

Page 176: Unconscious Memory Representations in Perception

� Neuroco��utational��o��els�of��erce�tual�or��anization� 1�5

�ossible�fro���revious�activit��.�This�su����ests�that��recisel��-ti�e���offset�activit���shoul���also�be�observe���in�the�au��itor���s��ste�,�as�is�in��ee���the�case�(Ka��ner�et�al.�2008).�Further�ore,�the�observe���ra�i���offset�res�onse�is��o��ulate���b���fee��-bac�,�consistent��ith�the��ro�ose����o��el�circuit.�A��re��iction�of� the��o��el� is�that�bloc�in���the�fee��-bac��circuits,�i.e.�cortical��o��ulation�of�sub-cortical��ro-cessin��,� �oul��� i��air� the� se���entation� of� successive� soun��� events� (Bala��uer-Ballester�et�al.�2009).

An�i��ortant�co��utational��rinci�le�e�bo��ie���in�this��o��el�is�the�trans-for�ation� of� ti�e,� or� �ore� �recisel��� ti�e� intervals,� into� a� to�o��ra�hic� re�re-sentation� at� �an��� ti�e� scales.� Pre��iction� of� ex�ecte��� ti�in��� is� i��licit� in� the�re�resentation,�rather�than�involvin���an�ex�licit��rojection�for�ar���in�ti�e,�an�����eviations� in� the� ex�ecte��� ti�in��� of� events� can� be� ��etecte��� b��� chan��es� in� the��atterns�of�activit���across�the��a�.�This��rinci�le��as�ex�lore���in�another��o��el-in���stu�����in��hich�the�sa�e�architecture��as�a��lie���at�a��uch�hi��her�co��nitive�level�to�extract�rh��th�ic�structure,�an���to�si��nal��hen�chan��es�in�rh��th���oul���beco�e�salient.�

�.�.2 Rh��th�

One�of�the��ost�co��ellin���features�of��usic�is�the�sense�of�rh��th��it�en��en��ers.�Hu�ans�can�quic�l���an���accuratel���inter�ret��usical�rh��th�ic�structure,�an�����o�so�ver���flexibl��;�for�exa��le,�the���can�easil�����istin��uish�bet�een�rh��th�ic,�te��o�an��� ti�in��� chan��es� (Honin��� 2002).� What� are� the� re�resentations� an��� relevant�features�that�hu�ans�so�successfull���use�to�inter�ret�rh��th�?�One�i��ea�is�that�the��erceive����etrical�hierarch���e�er��es�fro��the�relative�coinci��ence�of�activit���at�one�or��ore�level�of�re�resentation,�i.e.�it�arises�fro��interactions�bet�een���iffer-ent�ti�e�scales�ex�resse���in�the��atterns�of�activit���in�the�various�feature�s�aces;�i.e.�the�joint�ti�e�feature�s�aces�outline���in�the�intro��uction.�Structural���rou�in���can�occur�across��hrases� lon��er�than�a�sin��le��easure,�an���can�be�achieve���b���associations� bet�een� accentuations� (inclu��in��� har�onic� an��� �elo��ic� i��lica-tions)�or�b���the�use�of�ex�ressive�ti�in����hich�brea�s�the�strict�re��ularit���of�the��reviousl���establishe����eter.�This�su����ests�that�the��a���in��hich�these�structural���rou�in���cues�affect��erce�tion�can�be�un��erstoo���in�ter�s�of�the�establish�ent�of�re��ularities��hich�are�violate���,�for�exa��le�b����eans�of�b���the�use�of�ex�ressive�ti�in��.�Althou��h�this�cue�is�rather�subtle,�it��a���nevertheless�be�sufficientl����er-ce�tible�to��a�e�the�events�at�that��oint�in�ti�e��ore�salient�(Honin���2002).�This�is�consistent��ith�ex�eri�ental�fin��in��s�that�the��erce�tion�of�ex�ressive�ti�in���is�influence���b���the�listener’s�cate��orical��erce�tion�of�ti�e,�since�the���ifferences�are���etecte���in�relation�to�the�re��ularities�of�the�current�context�(Desain�et�al.�2003).

Page 177: Unconscious Memory Representations in Perception

1��� Susan�L.�Denha��et�al.

In��o��elin���the��erce�tion�of�rh��th�,�one��roble��is�to���ifferentiate�vari-abilit���in�ti�in��,�such�as�that�arisin���fro��the�use�of�ex�ressive�ti�in��,�fro��an�actual�chan��e�in�te��o,�an���to�account�for�the���e�en��ence�of��erce�tual�sensi-tivit���u�on�te��o.�In�an�ex�lorator���stu����,�the��o��el���escribe���above��ith�an�a��ro�riate�choice�of�ti�e�constants�at���ifferent�levels�of�the�hierarch����as�able�to�re�licate�hu�an��erfor�ance�in�a�te��o���iscri�ination�tas�,�illustrate���above.�Reset�si��nals�in�the��o��el�cause���b����is�atches��ith�ex�ectations��ere�use���to�in��icate��hen�a�variation�in�ti�in����oul���cause�the��o��el�to���eci��e�that�so�e-thin���ne��ha���ha��ene��,�an����as� inconsistent��ith� its�current�re�resentation,�as�su����este���b���ex�eri�ental�stu��ies�(Vuust�et�al.�2009;�Vuust�et�al.�2005).�The��o��ifie����itch��o��el��as�able�to�accuratel����re��ict�the�te��o���e�en��ent�thresh-ol��s�foun���in�a��erce�tual�ex�eri�ent�(Vos�et�al.�1996).

�.�.3 Tonalit��

Natural�environ�ents�contain�hi��hl���structure���infor�ation�s��ste�s�to��hich��e�are�ex�ose���in�ever����a���life.�As�ar��ue���here,�the�hu�an�brain�internalizes�these�re��ularities�b����assive�ex�osure,�an���the�acquire���i��licit��no�le����e�influences��erce�tion�an����erfor�ance.�So�e�evi��ence�that�si�ilarit���of��usical�scales�an���consonance�ju�����ents�across�hu�ans�arise�fro��the�statistical�structure�of�natu-rall���occurrin����erio��ic�soun��s�has�been�re�orte���(Sch�artz�et�al.�2003).�Moti-vate���b���evi��ence� that� i�a��e�statistics��re��ict� the�res�onse��ro�erties�of�so�e�as�ects�of�visual��erce�tion,�the�relationshi��bet�een��usic�statistics�an���hu�an�

Figure 6. Mo��el��re��ictions�an���ex�eri�ental�threshol��s�(Vos�et�al.�1996)�of�te��o�chan��e���etection�in�relation�to�the�base�te��o�rates�sho�n�on�the�x�axis.�

Page 178: Unconscious Memory Representations in Perception

� Neuroco��utational��o��els�of��erce�tual�or��anization� 1�7

ju�����ents�of�tonalit����ere�investi��ate���(Serrà�et�al.�2008).�A�statistical�anal��sis�of�chro�a�features,�extracte���fro��au��io�recor��in��s�of�Western��usic��as�use���to�buil���a�tonal��rofile�of�the��usic.�This�e��irical��rofile��as�co��are����ith�a�ran��e�of�tonal��rofiles��ro�ose���in�the�literature,�either�co��nitivel���or�theoreti-call���ins�ire��.�The�ver���hi��h���e��ree�of�correlation�bet�een�the��rofiles�su��orts�the�notion�that�hu�an�sensitivit���to�tonalit���faithfull���reflects�the�statistics�of�the��usical�in�ut�to��hich�the���have�been�ex�ose��.�

Tonalit����rofiles�can�be�seen�as�another��in���of�re��ularit���foun���in��usic;�an���violations�of�tonalit���in�the�for��of�chro�aticis���a���act�in��recisel���the�sa�e��a��� as� rh��th�ic� violations� to� enhance� �articular� �oints� in� the� �usic.� I��licit�learnin���of�re��ularities�such�as�tonalit���can�be��o��ele���usin���covariances�in�the�sti�uli� (Chen�et�al.�2007).�This���oes�be��on���si��le��air-�ise�correlations;�be-cause�of�the�hierarchicall���structure���re�resentations�in��hich�increasin��l����ore�co��lex��atterns,�s�annin���lon��er�ti�e�scales�co�e�to�be�re�resente��,�i��licit�learnin���can�also�a��l���to�hi��her-or��er��atterns�of�coherent�co-variation�a�on���sti�ulus� �ro�erties.� Hence� this� ��eneric� �echanis�� of� �erce�tual� learnin��� can���ive�rise� to� internal�re�resentations�that�ca�ture�a�si�ilarit���structure��hich� is���ifferent�fro��that�available���irectl���fro��a�si��le�sequential�sin��le�scale�anal��sis�of�the�sensor���in�ut.�

Ho�ever,�the�ti�e�scale�for�tonalit���anal��sis�a��ears�to�be�sur�risin��l���cir-cu�scribe��.�In�the�stu�����of�Serra�et�al.�(Serrà�et�al.�2008)�the�covariance��rofiles�of�chro�a�features��ere�avera��e���over�0.5,�15�an���30�s�lon���se���ents,�re�resent-in����itch�class���istributions�over�three���ifferent�ti�e�scales.�The�hi��h�accurac���of�the�0.5�secon����rofiles�in��re��ictin���hu�an�ju�����ent�su����este���that��erce�tions�of�tonal�stabilit���are�stron��l���influence���b���the�short-ter���usic�listenin���ex�eri-ence,�consistent��ith�the�ex�eri�ental�fin��in��s�of�Till�an�an���Bi��an���(Till�ann�et�al.�2004)�hi��hli��htin���the�i��ortance�of�local�structure�in��usic��erce�tion.�Fro�� their� �or�� the��� conclu��e��� that� usin��� short� te��oral� �in��o�s,� listeners�un��erstan���the�local�function�of�ca��ences,�an����erceive�chan��es�in�tonalit��.�This�is�also�consistent��ith�the��or��of�Le�an�(Le�an�2000)��ho�sho�e���that��a��o��-el��hich�incor�orate����rocessin���at�ti�e�scales�consistent��ith�the�t���ical�s�an�of� short� ter��echoic��e�or���(~1.5�secon��s)�coul���si�ulate�hu�an�sensitivit���to� tonalit��� in�a��robe-tone�ratin���tas�,� thereb���refutin���the�clai��that� the�sen-sitivit���necessaril�����e�en��e���u�on�so�e�lon���ter��ex�osure�to�Western��usic.�Clearl���the���eneric�architecture���erive���fro��the��itch��o��el,�illustrate���in�Fi��-ure�7,�coul���be�a��lie���to�the��erce�tion�of�tonalit��,�an���the��roble��of���etectin���chan��es�in��e���in��usical�excer�ts;�a�subject�of�on��oin���investi��ation.

Page 179: Unconscious Memory Representations in Perception

1��� Susan�L.�Denha��et�al.

�.7 Flexibility in perceptual organization

The��heno�enon�of��erce�tual�bistabilit���has�been��no�n�for�so�e�ti�e�in�vi-sion.�For�exa��le,�if�a���ifferent�i�a��e�is��resente���to�each�e��e,��erce�tual�a�are-ness�t���icall���s�itches�fro��one�i�a��e�to�the�other�rather�than�for�in���so�e�sort�of�co��osite� i�a��e.�This�effect� is��no�n�as� ‘binocular�rivalr��’.�There�are��an���other�for�s�of�visual�bistabilit��,�such�as�the�Nec�er�cube�an���the�vase-face�exa�-�le.�Ho�ever,�it�has�onl���recentl���been�sho�n�that�au��itor����erce�tion�si�ilarl���exhibits�bistabilit���(Denha��et�al.�2006;�Pressnitzer�et�al.�2005,�2006;�Win�ler�et�al.�2005).�In�the�case�of�au��itor���strea�in��,�in�res�onse�to�a�sequence�of�tones,�ABA-ABA-ABA-�…,��here�A�an���B�are�t�o���ifferent�frequencies,�an���–�is�a�silent��erio��,�subjects�re�ort�hearin���a���allo�in����attern��hich�characterizes�the�inte-��rate���or��anisation,�or�an�isochronous��attern��hich�si��nifies�that�the�tones�have�been�se��re��ate���into�t�o�se�arate�strea�s.�Our�anal��sis�sho�e���that�there�are�t�o�fun��a�entall�����ifferent��hases�of��erce�tual�or��anisation�(Denha��et�al.�2009);�in�the�first��erce�tual��hase�(formation of associations),�alternative�inter�retations�of� the� au��itor��� in�ut� are� for�e��� on� the� basis� of� sti�ulus� features;� an��� in� the�

Figure 7. A���eneric�hierarchical�architecture�for�the�extraction�an����o��elin���of�features�containe���in�te��oral��atterns�of�activit��.�This���eneric�circuit�is�h���othesize���to�recur�in��arallel�in��an�����ifferent�feature�s�aces,��hich�are�conjoine���iterativel���to�s�an�increas-in��l���lon��er�ti�e�scales.�

Page 180: Unconscious Memory Representations in Perception

� Neuroco��utational��o��els�of��erce�tual�or��anization� 1�9

secon����hase� (coexistence of interpretations),��erce�tion� stochasticall��� s�itches�bet�een�the�alternatives,�thus��aintainin����erce�tual�flexibilit��.�Interestin��l���the�characteristics�of�the�secon����hase�are�ver���si�ilar�to�a�nu�ber�of�as�ects�of�vi-sual�bistabilit��,�su����estin���co��on�un��erl��in����echanis�s.

Ke���in��re��ients�in��ost��o��els�of��erce�tual�bistabilit���are�co��etition�be-t�een� rivalin����erce�ts,� a��a�tation� so� that� the��innin����erce�t�never� re�ains���o�inant�for�all�ti�e�an���a�hierarchical�or��anisation�in��hich�co��etition�occurs�si�ultaneousl���at���ifferent�levels�in�the�hierarch��;�e.��.�(Wilson�2003;�Brasca���et�al.�2006;�Noest�et�al.�2007).�Consi��er�the�anal��sis�of�the��erce�t�choices�an���s�itchin���illustrate���in�Fi��ure�8��hich���e�icts�the�����na�ics�of�the��o��el.�The���re��� lines� re�resent� the� stable� states� of� each� of� the� rivalin��� �o�ulations,� also��no�n�as� the�null clines�of� the�s��ste�.�The�blac��arro�s�sho��ho��the�s��ste��state��ill�evolve�fro��an����articular��oint�in�the�s�ace.�The�attractors�of�the�s��s-te��are�the��oints��here�the���re���lines�intersect.�The�central�one,��no�n�as�the�bifurcation��oint,�is�unstable,�an���the�t�o�outer�ones�are�stable�an���corres�on���to�each�of�the�alternative��erce�tual�or��anisations.�With�a��a�tation,��hich�has�the�effect�of��ovin���the�line�corres�on��in���to�the���o�inant��erce�t�as�illustrate���in�the�ri��ht�han�����ia��ra�,�a�stable�attractor�is���ra��uall���lost�an���a�s�itch�to�the�other�attractor�occurs.�

Ke���here�is��erce�tual�flexibilit��.�If�the�neural�s��ste��is��aintaine���in�a�state�close�to�the�bifurcation��oint,�then�it�is�relativel���eas���to�s�itch�bet�een���ifferent�alternatives,�an���for�attention�to�exert�its�influence.�This�is�onl����ossible�if�the�t�o�rivalin���s��ste�s�are��ore�or�less�balance��.�The���ia��ra���a�es�it�clear�that�a�si��-nificant�i�balance�bet�een�the�co��etin���in�uts��oul���eli�inate�the�bifurcation�

Figure 8. The�����na�ics�of��erce�t�choice�(a)�an����erce�t�s�itchin���(b).�(A��a�te����ith��er�ission�fro��Noest�et�al.�2007.)

Page 181: Unconscious Memory Representations in Perception

170� Susan�L.�Denha��et�al.

�oint�an���the�existence�of�t�o�stable�attractors,�such�as�is�the�case�in�the�ri��ht�han�����ia��ra�.�This�is�interestin���because�it�is��ell��no�n�that�a��a�tation�to�the�current�context�is��ervasive�throu��hout�sensor���s��ste�s.�For�exa��le,�a��a�tation�to�the���istribution�of�sti�ulus�a��litu��es�in�inferior colliculus�has�been���e�onstrate���to��ove�the�si���oi��al�rate�level�function�such�that�it�o�ti�all����atches�the���istribu-tion�of�in�ut�levels�(Dean�et�al.�2005).�The�result�is�that�the�activit���levels�of�neurons�are�hel���relativel���constant,��ith�fluctuations�aroun���the��ean�bein���accentuate��.�A� natural� consequence� of� such� a��a�tation� �oul��� be� to� �aintain� rou��hl��� equi-table�activit���levels�in�res�onse�to�rivalin����erce�ts,�an���thereb���to��ee��the�s��ste��close�to�critical�bifurcation��oints.�The�effects�of�attention�can�be�un��erstoo���in�the�sa�e��a��;�i.e.�‘biasin��’�the�co��etition�is�essentiall���the�sa�e�as��ovin���one�of�the�null-clines,�thou��h�in�this�case�in�the�o��osite���irection�to�a��a�tation.

We�can�also�consi��er�a��o��el�of�au��itor���strea�in���in�the�li��ht�of�the���e-neric��o��el��ro�ose���above.�The� for�ar����ath�a���extracts� the��itches�of� the�alternatin��� tones� (ABA-ABA-…)� in� the� sequence,� an��� results� in� sequences� of�activit���associate����ith�each�tone.�It�has�been�sho�n�that�activit���across��ri�ar���au��itor���cortex���ra��uall���clusters� into�three�re��ions,�one�res�on��in���to�the�A�tones,�one�to� the�B�tones,�an���an� inter�e��iate�re��ion�to�both�tones�(Fish�an�et�al.�2004,�2001;�Miche��l�et�al.�2005).�It�is�clear�that�each�re��ion��oul���then�be�characterize���b���a���ifferent��erio��icit����hich�coul���be��rojecte���to�hi��her�level�areas�as���ifferent�quasi-static��atterns,�as�su����este����reviousl��.�Rivalr���at�the�u�-�er�level�bet�een�the���ifferent��erio��icit����atterns��oul���result�in�one��attern���o�inatin���an���hence�reachin����erce�tual�a�areness.�A��a�tation�at�all� levels��oul�����ra��uall����ea�en�the�attractor�until�a� s�itch� to� the�other�or��anisation�occurre��.�Inci��entall���the��o��el�can�ex�lain��h���it�is�easier�to�s�itch�bet�een�A-A-A�an���B----B----B�than�bet�een�A-A-A�an���ABA-ABA�since�lateral�co�-�etition�across�the�tonoto�ic�axis��oul���allo��both�the�se�arate�strea�s�to�exist��hile�su��ressin���the�central�cluster,�an���vice�versa,�as�illustrate���in�Fi��ure�9.

Both�in�visual�(Brasca���et�al.�2006)�an���in�au��itor���(Denha��et�al.�2008)�bistabilit��,� there� have� been� re�orts� of� so-calle��� transition� ��urations� �hich,� al-thou��h�not�as�lon���as�the�stable�or��anisations�t���icall���ex�erience��,�are�neverthe-less�lon��er�than��oul���be�ex�ecte���fro��the�bifurcation��o��el���escribe���above.�An� ex�lanation� for� this� �heno�enon� e�er��es� fro�� the� �ro�ose��� hierarchical��o��el�as�follo�s.�While�the���oal�of�sensor����rocessin���is�to�for��a�self-consistent�e�be����e���set�of�����na�ical�s��ste�s,�our��o��elin���stu��ies�have�sho�n�that�it�is��ossible�that�the��inner�at�one�level�of�the�hierarch����a���be�inconsistent��ith�that�at�another�level,�an���that�it�ta�es�so�e�ti�e�for�the�fee��bac���rojections�to�rein-state�consistenc���bet�een�all�levels.�This��e�su����est�corres�on��s�to�the�transition��erio���in��hich�both�or��anisations�are�re�orte��.

Page 182: Unconscious Memory Representations in Perception

� Neuroco��utational��o��els�of��erce�tual�or��anization� 171

�.� Concluding remarks

In� this�cha�ter��e�have�revie�e���visual�an���au��itor����o��els�of��erce�tual�or-��anisation� an��� have� ar��ue��� for� an� inter�retation� of� �erce�tion� as� a� �rocess� of�inference,�in�su��ort�of�the�i��ortant�role�of�sensor���s��ste�s�to���iscover�‘�hat�is�out�there’.�A�consistent�fra�e�or��e�er��es�across��o��alities�in�the�for��of�a�hierarchical���enerative�architecture,��hich��erfor�s�a��ulti-scale�anal��sis�of�the�inco�in�����ata�both�in�sti�ulus�feature�s�aces�an���in�ti�e.�In�a�novel�extension�to�this�a��roach�it��as�also�sho�n�that�to�-��o�n��o��ulation�of�inhibition�at�lo�er�levels�can�result�in�an�a��ro�riate�reset�of��rocessin����hen�ne��infor�ation�is���etecte��,�an���in�this��a���the�s��ste��can�a��just�its��rocessin���ti�e�constants�to��atch�the�sensor���in�ut.�In�conclusion,��e�believe�that�an�inter�retation�of�senso-r����rocessin���as�infor�ation���atherin���can��rovi��e��an���insi��hts�into�the�li�el���architecture�an���functionalit���of�sensor���s��ste�s.

Figure 9. The�first�t�o�sta��es�in�the��erce�tual�hierarch���in�the��ro�ose����o��el�of�au��itor���strea�in��.�In�the�sti�ulus�ABA-ABA-�each�tone�has���uration�T.�Co��etition�bet�een�clusters�of�activit���corres�on��in���to���ifferent��erio��icit����atterns�at�the�u��er�level�results�in�the�e�er��ence�of�a���o�inant��erce�tual�or��anisation.�Fee��bac��to�the�lo�er�level�ensures�consistent�or��anisation�throu��hout�the�hierarch��.�A��a�tation���ra��u-all����ea�ens�the�u��er�level�attractor,�allo�in���a�s�itch�to�another�or��anisation.

Page 183: Unconscious Memory Representations in Perception

172� Susan�L.�Denha��et�al.

Acknowledgements

This��or���as�su��orte���b���the�Euro�ean�Research�Area�S�ecific�Tar��ete���Proj-ects�E�CAP�(IST-FP6-013123)�an���SCANDLE�(ICT-FP7-231168).

References

An��elucci,�A.�an���Bullier,�J.�(2003).�Reachin���be��on���the�classical�rece�tive�fiel���of�VI�neurons:�horizontal�or�fee��bac��axons?�Journal of Physiology-Paris,�97,�141–154.

Bala��uer-Ballester,� E.,� Clar�,� N.,� Coath,� M.,� Kru�bholz,� K.� an��� Denha�,� S.� L.� (2009).� Un-��erstan��in����itch��erce�tion�as�a�hierarchical��rocess��ith�to�-��o�n��o��ulation.�PLoS�Computational Biology,�5,�e1000301.

Bala��uer-Ballester,�E.,�Coath,�M.�an���Denha�,�S.�L.�(2007).�A��o��el�of��erce�tual�se��re��ation�base���on�clusterin���the�ti�e�series�of�the�si�ulate���au��itor���nerve�firin����robabilit��.�Bio-logical Cybernetics,�97,�479–491.

Bala��uer-Ballester,�E.,�Denha�,�S.�L.�an���Me����is,�R.�(2008).�A�casca��e�autocorrelation��o��el�of��itch��erce�tion.�Journal of the Acoustical Society of America,�124,�2186–2195.

Bec�,�D.�M.�an���Kastner,�S.�(2008).�To�-��o�n�an���botto�-u���echanis�s�in�biasin���co��eti-tion�in�the�hu�an�brain.�Vision Research,�49,�1154–1165.

Bernstein,�J.�G.�an���Oxenha�,�A.�J.�(2005).�An�autocorrelation��o��el��ith��lace���e�en��ence�to�account�for�the�effect�of�har�onic�nu�ber�on�fun��a�ental�frequenc�����iscri�ination.�Journal of the Acoustical Society of America,�117,�3816–3831.

Brasca��,� J.�W.,�van�Ee,�R.,�Noest,�A.� J.,� Jacobs,�R.�H.�an���van���en�Ber��,�A.�V.� (2006).�The�ti�e�course�of�binocular�rivalr���reveals�a�fun��a�ental�role�of�noise.�Journal of Vision,�6,�1244–1256.

Ca��ieu,�C.,�Kouh,�M.,�Pasu�ath��,�A.,�Connor,�C.�E.,�Riesenhuber,�M.�an���Po����io,�T.�(2007).�A��o��el�of�V4�sha�e�selectivit���an���invariance.�Journal of Neurophysiology,�98,�1733–1750.

Caran��ini,�M.,�De�b,�J.�B.,�Mante,�V.,�Tolhurst,�D.�J.,�Dan,�Y.,�Olshausen,�B.�A.,�Gallant,�J.�L.�an���Rust,�N.�C.�(2005).�Do��e��no���hat�the�earl���visual�s��ste����oes?�Journal of Neuroscience,�25,�10577–10597.

Cariani,�P.�A.�an���Del��utte,�B.�(1996a).�Neural�correlates�of�the��itch�of�co��lex�tones.�II.�Pitch�shift,��itch�a�bi��uit��,��hase� invariance,��itch�circularit��,�rate��itch,�an���the���o�inance�re��ion�for��itch.�Journal of Neurophysiology,�76,�1717–1734.

Cariani,�P.�A.�an���Del��utte,�B.�(1996b).�Neural�correlates�of�the��itch�of�co��lex�tones.�I.�Pitch�an����itch�salience.�Journal of Neurophysiology,�76,�1698–1716.

Chen,�Z.,�Ha���in,�S.,�E����er�ont,� J.� J.�an���Bec�er,�S.� (2007).�Correlative learning: A basis for brain and adaptive systems.�Hobo�en,�NJ:�John�Wile���an���Sons,�Hobo�en.

Coath,�M.,�Bra��er,�J.�M.,�Fusi,�S.�an���Denha�,�S.�(2005).�Multi�le�vie�s�of�the�res�onse�of�an�ense�ble� of� s�ectrote��oral� res�onsefeatures� su��orts� concurrent� classification� of� ut-terance,��roso����,�sex�an���s�ea�er�i��entit��.�Network: Computation in Neural Systems,�16,�285–300.

Da��an,�P.,�Hinton,�G.�E.,�Neal,�R.�M.�an���Ze�el,�R.�S.�(1995).�The�Hel�holtz�Machine.�Neural Computation,�7,�889–904.

Page 184: Unconscious Memory Representations in Perception

� Neuroco��utational��o��els�of��erce�tual�or��anization� 173

��e�Chevei��ne,�A.�an���Pressnitzer,�D.�(2006).�The�case�of�the��issin�����ela���lines:�S��nthetic���ela��s�obtaine���b���cross-channel��hase�interaction.�Journal of the Acoustical Society of America,�119,�3908–3918.

��e�Chevei��né,�A.�(2005).�Pitch��erce�tion��o��els.�In�C.�J.�Plac�,�A.�J.�Oxenha�,�R.�R.�Fa���an���A.�N.�Po��er�(E��s.),�Pitch: Neural coding and perception.�Ne��Yor�:�S�rin��er.

Dean,� I.,� Har�er,� N.� S.� an��� McAl�ine,� D.� (2005).� Neural� �o�ulation� co��in��� of� soun��� level�a��a�ts�to�sti�ulus�statistics.�Nature Neuroscience,�8,�1684–1689.

Deco,�G.�an���Lee,�T.�S.�(2002).�A�unifie����o��el�of�s�atial�an���object�attention�base���on�inter-cortical�biase���co��etition.�Neurocomputing,�44–46,�775–781.

Denha�,�S.�L.�an���Win�ler,�I.�(2006).�The�role�of��re��ictive��o��els�in�the�for�ation�of�au��itor���strea�s.�Journal of Physiology-Paris,�100,�154–170.

Denha�,�S.�L.,�G��i�esi,�K.,�Stefanics,�G.�an���Win�ler,�I.�(2009).�Perce�tual�bi-stabilit���in�au-��itor��� strea�in��:� Ho�� �uch� ��o� sti�ulus� features� �atter?� Biological Psychology,� un��er�revie�.

Desain,�P.�an���Honin��,�H.�(2003).�The�for�ation�of�rh��th�ic�cate��ories�an����etric��ri�in��.�Perception,�32,�341–365.

Desi�one,�R.�an���Duncan,� J.� (1995).�Neural��echanis�s�of�selective�visual�atten-� te��oral�neurons�in�the�a�a�e�rhesus��on�e��.�Annual Review of Neuroscience,�18,�193–222.

Duncan,� J.� (1984).�Selective�attention�an��� the�or��anization�of�visual� infor�ation.� Journal of Experimental Psychology-General,�113,�501–517.

Dura,�S.�an���Denha�,�S.�L.� (2008).�Fee��bac�� in�a�hierarchical��o��el�of�object� reco��nition:�A�Ba��esian�Inference�A��roach,�Local-Area�S��ste�s�an���Theoretical�Neuroscience�Da��,�Gatsb���Unit�–�UCL.�Lon��on.

Dura,�S.,�Wenne�ers,�T.�an���Denha�,�S.� (2009).�Fee��bac�� in�a�hierarchical��o��el�of�object�reco��nition�in�cortex.�BMC Neuroscience,�10,�P355.

Felle�an,�D.�J.�an���Van�Essen,�D.�C.�(1991).�Distribute���hierarchical��rocessin���in�the��ri�ate�cerebral�cortex.�Cerebral Cortex,�1,�1–47.

Fishbach,�A.,�Nel�en,�I.�an���Yeshurun,�Y.�(2001).�Au��itor���e����e���etection:�A�neural��o��el�for��h��siolo��ical�an����s��choacoustical�res�onses�to�a��litu��e�transients.�Journal of Neuro-physiology,�85,�2303–2323.

Fish�an,� Y.� I.,� Arezzo,� J.� C.� an��� Steinschnei��er,� M.� (2004).� Au��itor��� strea�� se��re��ation� in��on�e���au��itor���cortex:�Effects�of�frequenc���se�aration,��resentation�rate,�an���tone���ura-tion.�Journal of the Acoustical Society of America,�116,�1656–1670.

Fish�an,�Y.�I.,�Reser,�D.�H.,�Arezzo,�J.�C.�an���Steinschnei��er,�M.�(2001).�Neural�correlates�of�au��itor��� strea�� se��re��ation� in� �ri�ar��� au��itor��� cortex� of� the� a�a�e� �on�e��.� Hearing Research,�151,�167–187.

Friston,�K.�(2002).�Functional�inte��ration�an���inference�in�the�brain.�Progress in Neurobiology,�68,�113–143.

Friston,�K.�(2003a).�Learnin���an���inference�in�the�brain.�Network-Computation in Neural Sys-tems,�16,�1325–1352.

Friston,�K.�(2003b).�Learnin���an���inference�in�the�brain.�Network-Computation in Neural Sys-tems,�16,�1325–1352.

Friston,�K.�(2005).�A�theor���of�cortical�res�onses.�Philosophical Transactions of the Royal Society of London, B Biological Sciences,�360,�815–836.

Friston,�K.,�Kilner,�J.�an���Harrison,�L.�(2006).�A�free�ener������rinci�le�for�the�brain.�Journal of Physiology-Paris,�100,�70–87.

Page 185: Unconscious Memory Representations in Perception

174� Susan�L.�Denha��et�al.

Friston,�K.�J.,�Harrison,�L.�an���Penn��,�W.�(2003).�D��na�ic�causal��o��ellin��.�Neuroimage,�19,�1273–1302.

Garcia-Lazaro,�J.�A.,�Ho,�S.�S.,�Nair,�A.�an���Schnu��,�J.�W.�(2007).�Shiftin���an���scalin���a��a�ta-tion�to�����na�ic�sti�uli� in�so�atosensor���cortex.�European Journal of Neuroscience,�26,�2359–2368.

Geisler,�W.�S.�an���Kersten,�D.�(2002).�Illusions,��erce�tion�an���Ba��es.�Nature Neuroscience,�5,�508–510.

Guo,�K.,�Robertson,�R.�G.,�Pul��arin,�M.,�Neva��o,�A.,�Panzeri,�S.,�Thiele,�A.�an���Youn��,�M.�P.�(2007).� S�atio-te��oral� �re��iction� an��� inference� b��� V1� neurons.� European Journal of Neuroscience,�26,�1045–1054.

Gutschal�,�A.,�Patterson,�R.�D.,�Scher��,�M.,�U��en�a��,�S.�an���Ru��,�A.� (2007).�The�effect�of�te��oral�context�on�the�sustaine����itch�res�onse�in�hu�an�au��itor���cortex.�Cerebral Cortex,�17,�552–561.

Há��en,�G.�P.,�Stefanics,�G.,�Vester��aar��,�M.�D.,�Denha�,�S.�L.,�Sziller,�I.�an���Win�ler,�I.�(2009).�Ti�bre-in��e�en��ent�extraction�of��itch�in�ne�born�infants.�Psychophysiology,�46,�69–74.

Hal�o,�M.�A.,�Min��olla,�E.�an���So�ers,�D.�C.�(2008).�Multi�le��echanis�s�of�illusor���contour��erce�tion.�Journal of Vision,�8,�Article.

Hall,�J.�W.,�3r���an���Peters,�R.�W.�(1981).�Pitch�for�nonsi�ultaneous�successive�har�onics�in�quiet�an���noise.�Journal of the Acoustical Society of America,�69,�509–513.

Harrison,�L.�M.,�Ste�han,�K.�E.,�Rees,�G.�an���Friston,�K.�J.�(2007).�Extra-classical�rece�tive�fiel���effects��easure���in�striate�cortex��ith�fMRI.�Neuroimage,�34,�1199–1208.

Hasson,�U.,�Yan��,�E.,�Vallines,�I.,�Hee��er,�D.�J.�an���Rubin,�N.�(2008).�A�hierarch���of�te��oral�rece�tive��in��o�s�in�hu�an�cortex.�Journal of Neuroscience,�28,�2539–2550.

Ha��ins,�J.�an���Geor��e,�D.�(2007).�Hierarchical temporal theory: Concepts, theory and terminol-ogy.�Re���oo���Cit��,�CA:�Nu�enta�Inc.

Hel�holtz,�H.�L.�(1860/1962).�Handbuch der physiologischen Optik�(En��lish�translation).�Ne��Yor�:�Dover.

Honin��,�H.�(2002).�Structure�an���inter�retation�of�rh��th��an���ti�in��.�Tijdschrift voor muziek-theorie,�7,�227–232.

Huan��,�J.�Y.,�Wan��,�C.�an���Dreher,�B.�(2007).�The�effects�of�reversible�inactivation�of��ostero-te��oral�visual�cortex�on�neuronal�activities�in�cat’s�area�17.�Brain Research,�1138,�111–128.

Hu�e,�J.�M.,�Ja�es,�A.�C.,�Girar��,�P.,�Lo�ber,�S.�G.,�Pa��ne,�B.�R.�an���Bullier,�J.�(2001).�Fee��bac��connections� act� on� the� earl��� �art� of� the� res�onses� in� �on�e��� visual� cortex.� Journal of Neurophysiology,�85,�134–145.

Jehee,� J.� F.,� Roelfse�a,� P.� R.,� Deco,� G.,� Murre,� J.� M.� an��� La��e,� V.� A.� (2007).� Interactions�bet�een�hi��her�an���lo�er�visual�areas�i��rove�sha�e�selectivit���of�hi��her�level�neurons-ex�lainin���cro���in����heno�ena.�Brain Research,�1157,�167–176.

Ka��ner,�A.�an���Berrebi,�A.�S.�(2008).�Enco��in���of�te��oral�features�of�au��itor���sti�uli�in�the��e��ial�nucleus�of�the�tra�ezoi���bo�����an���su�erior��araolivar���nucleus�of�the�rat.�Neurosci-ence,�151,�868–887.

Kersten,�D.,�Ma�assian,�P.�an���Yuille,�A.�(2004).�Object��erce�tion�as�Ba��esian�inference.�An-nual Review of Psychology,�55,�271–304.

Kiebel,�S.�J.,�Daunizeau,�J.�an���Friston,�K.�J.�(2008).�A�hierarch���of�ti�e-scales�an���the�brain.�PLoS�Computational Biology,�4,�e1000209.

Page 186: Unconscious Memory Representations in Perception

� Neuroco��utational��o��els�of��erce�tual�or��anization� 175

Knill,�D.�C.,�Kersten,�D.�an���Ma�assian,�P.�(1996).�I��lications�of�a�Ba��esian�for�ulation�of�visual�infor�ation�for��rocessin���for��s��cho�h��sics.�In�D.�Knill�an���W.�Richar��s�(E��s.),�Perception as Bayesian Inference�(��.�239–286).�Ca�bri����e�Universit���Press.

�Kor��in��,�K.�P.�an���Wol�ert,�D.�M.�(2004).�Ba��esian�inte��ration�in�sensori�otor�learnin��.�Na-ture,�427,�244–247.

Lee,�T.�S.�an���Mu�for��,�D.�(2003).�Hierarchical�Ba��esian�inference�in�the�visual�cortex.�Journal of the Optical Society of America A-Optics Image Science and Vision,�20,�1434–1448.

Lee,�T.�S.�an���N��u��en,�M.�(2001).�D��na�ics�of�subjective�contour�for�ation�in�the�earl���visual�cortex.�Proceedings of the National Academy of Sciences of the United States of America,�98,�1907–1911.

Le�an,�M.�(2000).�An�au��itor����o��el�of�the�role�of�short�ter���e�or���in��robe-tone�ratin��s.�Music Perception,�17,�481–509.

Lic�li��er,�J.�C.�R.�(1951).�A���u�lex�theor���of��itch��erce�tion.�Experientia,�7,�128–134.Me����is,�R.�an���He�itt,�M.�J.�(1991).�Virtual��itch�an����hase�sensitivit���of�a�co��uter��o��el�of�

the�au��itor����eri�her��.�I:�Pitch�i��entification.�Journal of the Acoustical Society of America,�89,�2866.

Me����is,�R.�an���O’Mar��,�L.�(1997).�A�unitar����o��el�of��itch��erce�tion.�Journal of the Acousti-cal Society of America,�102,�1811–1820.

Miche��l,�C.,�Tian,�B.,�Carl��on,�R.�P.�an���Rauschec�er,�J.�P.�(2005).�Perce�tual�or��anization�of�tone�sequences�in�the�au��itor���cortex�of�a�a�e��acaques.�Neuron,�48,�139–148.

Moore,�B.�C.�J.�(2004).�An introduction to the psychology of hearing.�Lon��on:�Elsevier.Mu�for��,� D.� (1991).� On� the� co��utational� architecture� of� the� neocortex� I.� The� role� of� the�

thala�o-cortical�loo�.�Biological Cybernetics,�65,�135–145.Mu�for��,�D.�(1992).�On�the�co��utational�architecture�of� the�neocortex�II.�The�role�of� the�

cortico-cortical�loo�.�Biological Cybernetics,�66,�241–251.Mu�for��,�D.�(1996).�Pattern�theor��:�a�unif��in����ers�ective.�In�D.�R.�W.�Knill�(E��.),�Perception

as Bayesian Inference�(��.�25–62).�Ca�bri����e�Universit���Press.Murra��,�S.�O.,� Schrater,�P.� an���Kersten,�D.� (2004).�Perce�tual���rou�in���an��� the� interactions�

bet�een�visual�cortical�areas.�Neural Networks,�17,�695–705.Noest,�A.�J.,�van�Ee,�R.,�Nijs,�M.�M.�an���van�Wezel,�R.�J.�(2007).�Perce�t-choice�sequences���riven�

b���interru�te���a�bi��uous�sti�uli:�A�lo�-level�neural��o��el.�Journal of Vision,�7,�10.Olshausen,�B.�A.�an���Fiel��,�D.�J.�(2005).�Ho��close�are��e�to�un��erstan��in���v1?�Neural Com-

putation,�17,�1665–1699.Pressnitzer,�D.�an���Hu�é,�J.�M.�(2005).�Is�au��itor���strea�in���a�bistable��erce�t?�Forum Acusti-

cum, Budapest,�1557–1561.Pressnitzer,�D.�an���Hu�e,�J.�M.�(2006).�Te��oral�����na�ics�of�au��itor���an���visual�bistabilit���

reveal�co��on��rinci�les�of��erce�tual�or��anization.�Current Biology,�16,�1351–1357.Raiza��a,�R.�D.�S.�an���Grossber��,�S.�(2003).�To�ar��s�a�theor���of�the�la�inar�architecture�of�cere-

bral�cortex:�Co��utational�clues�fro��the�visual�s��ste�.�Cerebral Cortex,�13,�100–113.Rao,�R.�P.�N.�(1999).�An�o�ti�al�esti�ation�a��roach�to�visual��erce�tion�an���learnin��.�Vision

Research,�39,�1963–1989.Rao,�R.�P.�N.�(2004).�Ba��esian�co��utation�in�recurrent�neural�circuits.�Neural Computation,�

16,�1–38.Rao,�R.�P.�N.�(2006).�Neural�Mo��els�of�Ba��esian�Belief�Pro�a��ation.�In�K.�Do��a�(E��.),�Bayes-

ian brain: Probabilistic approaches to neural coding�(��.�239–268).�Ca�bri����e,�MA:�MIT�Press.

Page 187: Unconscious Memory Representations in Perception

17�� Susan�L.�Denha��et�al.

Rao,�R.�P.�N.�an���Ballar��,�D.�H.�(1999).�Pre��ictive�co��in���in�the�visual�cortex:�A�functional�in-ter�retation�of�so�e�extra-classical�rece�tive-fiel���effects.�Nature Neuroscience,�2,�79–87.

Rauschenber��er,�R.,�Liu,�T.,�Slotnic�,�S.�D.�an���Yantis,�S.�(2006).�Te��orall���unfol��in���neural�re�resentation�of��ictorial�occlusion.�Psychological Science,�17,�358–364.

Riesenhuber,�M.�an���Po����io,�T.� (1999).�Hierarchical��o��els�of�object�reco��nition� in�cortex.�Nature Neuroscience,�2,�1019–1025.

Schroe��er,�M.�R.�(1999).�Computer speech.�Ne��Yor�:�S�rin��er.Sch�artz,�D.�A.,�Ho�e,�C.�Q.�an���Purves,�D.�(2003).�The�statistical�structure�of�hu�an�s�eech�

soun��s��re��icts��usical�universals.�Journal of Neuroscience,�23,�7160–7168.Sch�artz,�O.,�Hsu,�A.�an���Da��an,�P.�(2007).�S�ace�an���ti�e�in�visual�context.�Nature Reviews

Neuroscience,�8,�522–535.Serrà,�J.,�Gó�ez,�E.,�Herrera,�P.�an���Serra,�X.�(2008).�Statistical�anal��sis�of�chro�a�features�in�

�estern��usic��re��icts�hu�an�ju����e�ents�of�tonalit��.�Journal of New Music Research,�37,�299–309.�

Serre,�T.,�Wolf,�L.,�Bileschi,�S.,�Riesenhuber,�M.�an���Po����io,�T.�(2007).�Robust�object�reco��ni-tion� �ith� cortex-li�e� �echanis�s.� IEEE Transactions on Pattern Analysis and Machine Intelligence,�29,�411–426.

Sillito,�A.�M.,�Cu��eiro,�J.�an���Jones,�H.�E.�(2006).�Al�a��s�returnin��:�Fee��bac��an���sensor����ro-cessin���in�visual�cortex�an���thala�us.�Trends in Neurosciences,�29,�307–316.

Slane��,� M.� an��� L��on,� R.� F.� (1990).� A� �erce�tual� �itch� ��etector.� International Conference on Acoustics, Speech, and Signal Processing ICASSP-90,�1,�357–360.

S�ratlin��,�M.�W.�(2008).�Pre��ictive�co��in���as�a��o��el�of�biase���co��etition�in�visual�attention.�Vision Research,�48,�1391–1408.

Stanle��,�D.�A.�an���Rubin,�N.�(2003).� fMRI�activation� in�res�onse�to� illusor���contours�an���salient�re��ions�in�the�hu�an�lateral�occi�ital�co��lex.�Neuron,�37,�323–331.

Stefanics,�G.,�Ha��en,�G.�P.,�Sziller,�I.,�Balazs,�L.,�Be�e,�A.�an���Win�ler,�I.�(2009).�Ne�born�in-fants��rocess��itch�intervals.�Clinical Neurophysiology,�120,�304–308.

Stefanics,� G.,� Ha��en,� G.,� Huotilainen,� M.,� Balazs,� L.,� Sziller,� I.,� Be�e,� A.,� Fell�an,� V.� an����Win�ler,�I.�(2007).�Au��itor���te��oral���rou�in���in�ne�born�infants.�Psychophysiology,�44,�697–702.

Till�ann,�B.�an���Bi��an��,�E.�(2004).�The�relative�i��ortance�of�local�an�����lobal�structures�in��usic��erce�tion.�The Journal of Aesthetics and Art Criticism,�62,�211–222.

Vester��aar��,�M.�D.,�Há��en,�G.�P.,�Sht��rov,�Y.,�Patterson,�R.�D.,�Pulver�üller,�F.,�Denha�,�S.�L.,�Sziller,�I.�an���Win�ler,�I.�(2009).�Au��itor���size-��eviant���etection�in�a��ults�an���ne�born�infants.�Biological Psychology,�82,�169–175.

Vos,�P.�G.,�van�Assen,�M.�an���Frane�,�M.�(1996).�Perceive���te��o�chan��e�is���e�en��ent�on�base�te��o�an�����irection�of�chan��e:�Evi��ence�for�a���eneralize���version�of�Schulze’s�(1978)�in-ternal�beat��o��el.�Psychological Research,�59,�240–247.

Vuust,�P.,�Oster��aar��,�L.,�Pallesen,�K.�J.,�Baile��,�C.�an���Roe�storff,�A.�(2009).�Pre��ictive�co��in���of��usic�–�Brain�res�onses�to�rh��th�ic�incon��ruit��.�Cortex,�45,�80–92.

Vuust,�P.,�Pallesen,�K.�J.,�Baile��,�C.,�van�Zuijen,�T.�L.,�Gje����e,�A.,�Roe�storff,�A.�an���Oster��aar��,�L.�(2005).�To��usicians,�the��essa��e�is�in�the��eter��re-attentive�neuronal�res�onses�to�incon��ruent�rh��th��are�left-lateralize���in��usicians.�Neuroimage,�24,�560–564.

Wie��rebe,�L.�(2001).�Searchin���for�the�ti�e�constant�of�neural��itch�extraction.�Journal of the Acoustical Society of America,�109,�1082–1091.

Wilson,�H.�R.�(2003).�Co��utational�evi��ence�for�a�rivalr���hierarch���in�vision.�Proceedings of the National Academy of Sciences of the United States of America,�100,�14499–14503.

Page 188: Unconscious Memory Representations in Perception

� Neuroco��utational��o��els�of��erce�tual�or��anization� 177

Win�ler,�I.,�Ta�e��ata,�R.�an���Suss�an,�E.�(2005).�Event-relate���brain��otentials�reveal��ulti�le�sta��es�in�the��erce�tual�or��anization�of�soun��.�Brain Research – Cognitive Brain Research,�25,�291–299.

Win�ler,�I.,�Ha��en,�G.�P.,�La��ini��,�O.,�Sziller,�I.�an���Honin��,�H.�(2009).�Ne�born�infants���etect�the�beat�in��usic.�Proceedings of the National Academy of Sciences of the United States of America,�106,�2468–2471.

Yuille,�A.�an���Kersten,�D.�(2006).�Vision�as�Ba��esian�inference:�Anal��sis�b���s��nthesis?�Trends in Cognitive Sciences,�10,�301–308.

Page 189: Unconscious Memory Representations in Perception
Page 190: Unconscious Memory Representations in Perception

chapter�7�

Are you listening? Lan��ua��e�outsi��e�the�focus�of�attention

Yur���Sht��rov�an���Frie��e�ann�Pulver�üllerMe��ical�Research�Council,�Co��nition�an���Brain�Science�Unit,��Ca�bri����e,�UK

7.1 Introduction

Do�all�sta��es�of�s�eech�anal��sis�require�our�active�attention�on�the�subject?�Can�so�e�of� the�� ta�e��lace� irres�ective�of��hether�or�not��e� focus�on� inco�in���s�eech�an���can�the���be,�in�this�sense,�auto�atic?�Are�various�t���es�of�lin��uistic�infor�ation��rocesse���b���our�brain�si�ultaneousl���or�in�certain�or��er?�These�an���other�questions�have�been�lon�����ebate���in�both��s��cholin��uistics�an���neurosci-ence�of� lan��ua��e.�Recent� stu��ies� investi��atin��� the�co��nitive��rocesses�un��erl��-in��� s�o�en� lan��ua��e��rocessin��� foun��� that�even�un��er�attentional��ith��ra�al,�size�an���to�o��ra�h���of�certain�brain�res�onses�reflect�the�activation�of��e�or���traces�for�lan��ua��e�ele�ents�in�the�hu�an�brain.�Fa�iliar�soun��s�of�one’s�native�lan��ua��e��a���elicit�a�lar��er�brain�activation�than�unfa�iliar�soun��s,�an���at�the�level�of��eanin��ful�lan��ua��e�units,��or��s�elicit�a�lar��er�res�onse�than��eanin��-less��or��-li�e�soun���co�binations.�This�su����ests�that�usin����o��ern�neuroi�a��-in���tools,��e�can�trace�the�activation�of��e�or���net�or�s�for�lan��ua��e�soun��s�an��� s�o�en� �or��s� even� �hen� no� attention� is� �ai��� to� the�.� Unatten��e��� �or���sti�uli�elicit�an�activation�sequence�startin����ith�su�erior-te��oral�cortex�an���ra�i��l����ro��ressin���to�left-inferior-frontal�lobe.�The�s�atio-te��oral��atterns�of�these�cortical�activation���e�en���on�lexical�an���se�antic��ro�erties�of��or���ste�s�an���affixes,�thus��rovi��in���clues�about�lexico-se�antic�infor�ation��rocessin��.�At�the�s��ntactic�level,��e�can�see�reflections�of���ra��atical�re��ularities�in��or���strin��s.�This���ro�in���bo�����of�results�su����ests�that�lexical,�se�antic�an���s��ntactic�infor�ation�can�be��rocesse���b���the�central�nervous�s��ste��outsi��e�the�focus�of�attention�in�a�lar��el���auto�atic��anner.�Anal��sis�of�s�atio-te��oral��atterns�of���enerator�activations�un��erl��in���such�attention-in��e�en��ent�res�onses�to�s�eech�

Page 191: Unconscious Memory Representations in Perception

1�0� Yur���Sht��rov�an���Frie��e�ann�Pulver�üller

�a���be�an�i��ortant�tool�for�investi��atin���the�brain�����na�ics�of�s�o�en�lan��ua��e��rocessin���an���the���istribute���cortical�net�or�s�involve��.

7.2 Types and stages of linguistic information processing: Theories and conflicting evidence in psychololinguisitics and neurophysiology

It�has�been�a��atter�of���ebate��hether�the���istinctivel���hu�an�ca�acit���to��rocess�lan��ua��e���ra�s�on�attentional�resources�or�is�auto�atic.�The�ease,��ith��hich��e�can��erceive�the�entire�co��lexit���of�inco�in���s�eech�an���see�in��l���instanta-neousl�����eco��e�s��ntactic,��or�holo��ical,�se�antic�an���other�infor�ation��hile���oin���so�ethin���else�at�the�sa�e�ti�e,��ro��te���su����estions�that�lin��uistic�activ-it����a���be��erfor�e���b���the�hu�an�brain�in�a�lar��el���auto�atic�fashion�(Fo��or�1983;�Garrett�1984;�Garro���an���Pic�erin���2004;�Pic�erin���an���Garro���2004).�Be-fore�consi��erin���the�ex�eri�ental���ata�that�s�ea��to�the�issue,�let�us�briefl���loo��at�a��ore���eneral��s��cholin��uistic�an���neuro�h��siolo��ical�bac���roun���of��hat�is��no�n�about�lin��uistic�infor�ation��rocessin��.

Tra��itional� �s��cholin��uistic� �o��els� of� lan��ua��e� co��rehension� (Morton�1969;� Fro��in� 1973;� Garrett� 1980;� Dell� 1986;� MacKa��� 1987),� also� reflecte��� in�current�a��roaches�to�s�eech�co��rehension�an����ro��uction�(Levelt�et�al.�1999;�Norris� et� al.� 2000),� ��istin��uishe��� a� fe�� t���es� of� infor�ation� involve��� in� these��rocesses.�Even�thou��h�these�theories��a���be�rather���ifferent,�the����ostl���a��ree�on�the��lausibilit���of�(1)��honolo��ical��rocessin���level,�at��hich�s�eech�soun��s�are�anal��se���for�their��honetic/�honolo��ical�features�(follo�in���the�level�of�basic�acoustic�anal��sis),�(2)�lexical��rocessin��,�so�eti�es�conce�tualise���as�the�loo�u��of�an�ite��in�a�“�ental�lexicon”,��hich�lists�onl����or���for�s�but�not�their��eanin���or�other�relate���infor�ation,�(3)�se�antic��rocessin��,��here�the�ite�’s��eanin���is�accesse��,�an���(4)�s��ntactic�level,��here���ra��atical�infor�ation�lin�in����or��s�in�sentences�to�each�other�is�anal��se��.�These���ifferent�“�o��ules”�of��rocessin���can�so�eti�es�be��er��e��,�o�itte���or�exten��e���to�inclu��e��ore�levels.�A���reat���ebate�in��s��cholin��uistics�is,�ho�ever,�bet�een��o��els�accor��in���to��hich��rocessin���of�these�infor�ation�t���es�is�consecutive,�an���access�to�the��co��ences�at�sub-stantiall��� ��ifferent� ti�es,� an��� �o��els� i��l��in��� that� these� �rocesses� ta�e� �lace�near-si�ultaneousl��,�even�in��arallel.�This�issue�coul����otentiall���be�resolve���on�the�basis�of�neuro�h��siolo��ical���ata,�usin���EEG�or�MEG��hich�can�trac��brain��rocesses��ith�hi��hest��ossible�te��oral�resolution,�on�a��illisecon���scale.

Consequentl��,�an�entire�bo�����of�neuro�h��siolo��ical�research���one��ith�vari-ous�lin��uistic��aterials�assiste���in�establishin���the���o�inance�of�a�vie��that��os-its�serial�or��er�of�lin��uistic�infor�ation�access�in�the�brain.�With�basic�acoustic�

Page 192: Unconscious Memory Representations in Perception

� Are���ou�listenin��?� 1�1

feature�extraction�co��encin���at�20–50��s�after�the�sti�ulus�onset�(Kru�bholz�et� al.� 2003;� Lut�enhoner� et� al.� 2003),� �honolo��ical� tas�s� �o��ulate��� res�onses��ith�latencies�of�100–200��s�(Poe��el�et�al.�1996;�Obleser�et�al.�2004).�‘Hi��her’�levels�of�infor�ation�a��eare���to�ta�e�substantiall���lon��er�to�be�assesse���b���the�hu�an�brain:�the�N400�co��onent,��hich��ea�s�aroun���400��s�after�the�onset�of�a�critical�visual��or���(Kutas�an���Hill��ar���1980)�is�tra��itionall���seen�as�the��ain�in��ex�of�se�antic��rocesses.�A�sli��htl���earlier�(350��s)��ea�in���sub-co��onent�of�it�is�so�eti�es�se�arate���as�an�in��ex�of�lexical��rocessin���(Bentin�et�al.�1999;�E�bic��et�al.�2001;�P��l��anen�et�al.�2002;�Stoc�all�et�al.�2004)�ai��in���the�elusive���istinction�bet�een�the�lexical�an���se�antic�levels�of��rocessin��.�An�even��ore�co��licate���situation�e�er��e���for�s��ntactic��rocessin��,��ith�an�earl���co��onent�(ELAN,� earl��� left� anterior� ne��ativit��)� a��earin��� at� ~100� �s� an��� reflectin��� the��hrase’s���ra��aticalit���(Neville�et�al.�1991;�Frie��erici�et�al.�1993);�this�is�co��li-�ente���b���later���ra��aticall���relate���frontal�ne��ativities��ith�lon��er�(>�250��s)�latencies�(Münte�et�al.�1998;�Gunter�et�al.�2000),�an��,�finall��,�b���the�P600,�a�late��ositive�shift�reachin���its��axi�u��at�~600��s�at�centro-�arietal�sites�(for�re-vie�,�see�Osterhout�et�al.�1997;�Osterhout�an���Ha��oort�1999).

Althou��h�not��ithout�controversies,�such���ata�su����este���a�ste��ise�access�to� ��ifferent� t���es� of� lin��uistic� infor�ation,� thus� for�in��� a� �ainstrea�� vie��in�the�neuro�h��siolo�����of�lan��ua��e�(for�revie�,�see�Frie��erici�2002).�This�vie�,�ho�ever,�ca�e�into�a�conflict��ith�a�bo�����of�behavioural�evi��ence�collecte���in�a�nu�ber�of��sh��cholin��uistic�stu��ies�that�in��icate����arallel��rocessin���of�cru-cial�infor�ation�about�inco�in����or��s�an���their�context�ver���earl��,��ithin�the�first�~200��illisecon��s�after�a�critical��or���can�be�reco��nize���(Marslen-Wilson�1973,�1987;�Rastle�et�al.�2000;�Mohr�an���Pulver�üller�2002).�For�exa��le,�sub-jects�can�alrea������a�e�reliable�button-�ress��otor�res�onses�to��ritten��or��s�accor��in��� to� their� evaluation� of� as�ects� of� �honolo��ical� an��� se�antic� sti�u-lus��ro�erties��ithin�400–450��s�after�their�onset�(Marslen-Wilson�an���T��ler�1975).� Therefore,� the� earliest� �or��-relate��� �s��cholin��uistic� �rocesses� as� such��ust� ta�e� �lace� substantiall��� before� this,� as� consi��erable� ti�e� is� require��� for�the��otor� res�onse��re�aration�an���execution.�Sur�risin��l��,� at� these�earl��� la-tencies,� lin��uistic� �rocessin��� �as� alrea����� influence��� b��� s��ntactic� an��� se�an-tic� infor�ation� (Marslen-Wilson� an��� T��ler� 1975).� Further�ore,� stu��ies� usin���sha��o�in��� technique�su����este��� that� the� lan��ua��e�out�ut��ust�be� initiate���b���150�to�200��s�alrea�����after�the�in�ut�onset�(Marslen-Wilson�1985).�Earl���behav-ioural�effects�reflectin���se�antic��rocessin���an���context�inte��ration��ere���ocu-�ente���in�cross��o��al��ri�in��,��here�s�ecific��no�le����e�about�an�u�co�in���s�o�en� �or��� coul��� be� ��e�onstrate��� to� be� �resent� �ell� before� its� en��,� �ithin�200��illisecon��s�after�the�acoustic�si��nal�allo�s�for�unique��or���i��entification�or�even��ell�ahea���of�this�(Z�itserloo���1989;�Moss�et�al.�1997;�T��ler�et�al.�2002).��

Page 193: Unconscious Memory Representations in Perception

1�2� Yur���Sht��rov�an���Frie��e�ann�Pulver�üller

E��e-trac�in���ex�eri�ents���e�onstrate���that�a�ran��e�of��s��cholin��uistic��ro�-erties� of� sti�uli� influence� short-latenc��� e��e� �ove�ent� res�onses� (Sereno� an���Ra��ner�2003).�The�vie��i��ose���b���such�stu��ies�is�that�near-si�ultaneous�access�to� the���ifferent� t���es�of� lin��uistic� infor�ation��a���co��ence��ithin�200��s�after�the�relevant�infor�ation�beco�es�available.�This�calls�in�question�the�ste�-�ise��o��els�of� lan��ua��e��rocessin���also��ut�for�ar���in��s��cholin��uistics�an���su��orte��,�at�least��artiall��,�b���earl���ERP�ex�eri�ents.

7.3 Mismatch negativity as a tool for language science

The�above�controvers���clearl����resents�a�nee���for�a��ore�refine����etho��olo�����that��oul���a����ress�this�issue.�As��e��ill�see�belo�,�this�issue�is�intrinsicall���involve��,�for� theoretical� an��� �etho��olo��ical� reasons,� �ith� the� issue� of� auto�aticit��� an���control�in�lin��uistic��rocessin��.�To�a����ress�these,��e��ill�focus�on���ata�obtaine���outsi��e�the�focus�of�attention�in�a��assive�o����ball��ara��i����usin���the�so-calle����is�atch�ne��ativit���(MMN)�brain�res�onse.�MMN�is�an�evo�e���brain�res�onse�elicite���b���rare�(so-calle�����eviant)�acoustic�sti�uli�occasionall����resente���in�a�se-quence�of�frequent�(stan��ar��)�sti�uli�(Alho�1995;�Näätänen�1995).�I��ortantl��,�MMN�can�be�elicite���in�the�absence�of�the�subject’s�attention�to�the�au��itor���in�ut�(Tiitinen�et�al.�1994;�Schrö��er�1996).�It�therefore�beca�e�consi��ere���to�reflect�the�brain’s� auto�atic� ��iscri�ination� of� chan��es� in� the� au��itor��� sensor��� in�ut� an���thus�to�be�a�unique�in��icator�of�auto�atic�cerebral��rocessin���of�acoustic�events�(Näätänen�1995).�A�nu�ber�of�MMN’s�s�ecific��ro�erties�le���to�the�su����estion�to�use�it�as�a�tool�for�investi��atin���the�neural��rocessin���of�s�eech�an���lan��ua��e�(Näätänen�2001;�Pulver�üller�an���Sht��rov�2006).�The��ain��otivations�for�a�-�l��in���MMN�to�ex�lorin���the�brain��rocessin���of�lan��ua��e�are:�(a)�MMN�is�earl��;�(b)�MMN�is�auto�atic;�(c)�MMN�is�a�res�onse�to�in��ivi��ual�soun��;�(��)�MMN�is�res�onse�to�a�chan��e.�Whereas�(a)�an���(b)�are�base���on�the��no�n��ro�erties�of�the�MMN�res�onse�tentativel���relate���to�its�neural��echanis�s,�(c)�an���(��)�are�i��ortant�fro��the�technical,��etho��olo��ical��oint�of�vie��on�recor��in���brain�res�onses�to�lan��ua��e.�

7.3.1 The�earliness�of�MMN�res�onse�

Various� behavioural� ��ata� su����este��� lin��uistic� �rocesses� in� the� brain� to� co�-�ence��ithin�200��s�after�the�relevant�acoustic�infor�ation�is�available.�Shoul���these��rocesses�be�reflecte���in�the�����na�ics�of�event-relate���activit���recor��e���on�the�scal��surface,�the�res�ective�brain�res�onses��ust�have�si�ilar,�if�not�earlier,�

Page 194: Unconscious Memory Representations in Perception

� Are���ou�listenin��?� 1�3

latencies�in�or��er�to�be�consi��ere�����irectl���relate���to�these��rocesses�rather�then�bein���their�re�ote�consequences.�This�clearl���eli�inates�late�shifts�(N400,�M350,�P600)�as��otential� in��icators�of�earl����rocessin��.�Earl���obli��ator���au��itor���re-s�onses�(P1,�N1)�have�not�been�foun���sensitive�to�lin��uistic�variables.�ELAN,�the�earl���s��ntax-relate���res�onse,�is��ell��ithin�the�ti�e��in��o�,�but�cannot�be�seen�for��ost�t���es�of�lin��uistic�(even�s��ntactic)�tas�s.�MMN,�on�the�contrar��,�is�both�earl���(usuall���re�orte���as�havin���latencies�of�100–200��s)�an���fir�l���lin�e���to�such�co��nitive��rocesses�as��e�or��,�attention�allocation,�an����ri�itive�au��itor���s��ste��intellect�(Win�ler�et�al.�1993;�Näätänen�1995,�2000;�Näätänen�an���Alho�1995;�Näätänen�et�al.�2001).�It�is��no�n�to�be�sensitive�to�hi��hl���abstract�features�of� au��itor��� si��nals� an��� is� therefore� a� ��oo��� can��i��ate� for� i��entif��in��� �utative�earl���lin��uistic�activations.�

7.3.2 Auto�aticit���of�the�MMN�

The�jur���is�on�about��hether�or�not,�an���to��hat�extent,�the�MMN�is�in��e�en��ent�of�or��o��ulate���b���attention�(Picton�et�al.�2000).�It�is�ho�ever�uncontroversial�that�MMN�can�be�easil���elicite���in�the�absence�of�attention�to�the�sti�ulus�in�ut�an���therefore���oes�not�require�active�voluntar����rocessin���of�sti�ulus��aterial�b���the�in��ivi��ual,��ho��a���be�en��a��e���in�an�unrelate����ri�ar���tas��(t���icall����atchin���a�vi��eofil�)��hile�MMN�res�onses�are�bein���evo�e���b���ran��o��in-frequent�chan��es�in�au��itor���sti�ulation.�In�this�ver���sense�the�MMN�can�be�consi��ere��� an� auto�atic� brain� res�onse,� at� least� until� a� better� ter�� is� foun��.�For�lan��ua��e�research,�this�has�an�i��ortant�i��lication.�T���icall��,�in�lan��ua��e�ex�eri�ents,�subjects�are�as�e���to�atten���to��resente����or��s�or�sentences�(e.��.,�Neville�et�al.�1991;�Osterhout�an���S�inne���1993;�Frie��erici�et�al.�2000).�The�tas��is�often�to��a�e�a�ju�����ent�of�the�sti�ulus��aterial�(e.��.�fa�iliar/unfa�iliar,�correct/incorrect)�or�even��erfor��a�s�ecific�lin��uistic�tas��(e.��.�lexical���ecision,���ra��ar�assess�ent).�When�attention�is�require��,�one�can�not�be�sure�to��hat�extent� the� re��istere��� res�onses� are� influence��� b��� brain� correlates� of� attention�rather�than�b���the�lan��ua��e-relate���activit���as�such.�Attention-relate����heno�-ena� are� �no�n� to� �o��ulate� a� variet��� of� brain’s� evo�e��� res�onses� involvin��� a�nu�ber� of� brain� structures� inclu��in��� those� close� to,� or� overla��in��� �ith,� the�core�lan��ua��e�areas�(see�e.��.�Picton�an���Hill��ar���1974;�Alho�1992;�Woo��s�et�al.�1993a,�1993b;�Tiitinen�et�al.�1997;�Escera�et�al.�1998;�Ya�asa�i�et�al.�2002;�Yantis�et�al.�2002).�It� is�also� li�el���that�subjects��a����ore�attention�to�unusual�or� in-correct�sti�uli�(�ost�frequentl���use���sti�ulus�t���es�inclu��e�e.��.��seu��o�or��s,�nonsense�sentences�or���ra��atical�violations)�as�the���tr���to��a�e�so�e�sense�of� the�,�or� that� the���use���ifferent� strate��ies� to��rocess��ro�er�an��� ill-for�e���

Page 195: Unconscious Memory Representations in Perception

1�4� Yur���Sht��rov�an���Frie��e�ann�Pulver�üller

ite�s.�Such���ifferent�sti�ulus-s�ecific�strate��ies�an���attention�variation��a���be�reflecte���in�the�event-relate����easures,�thus�obscurin��,��as�in��,��o��if��in���or�even�cancelin���an���true�lan��ua��e-relate���res�onses.�So,�to�re��ister�the�true�lan-��ua��e-relate���activit���of�the�brain,�it�is�essential�to�tease�it�a�art�fro��such�atten-tion-�an���tas�-relate���effects.�The�MMN��rovi��es�a�strai��htfor�ar���solution�to�this,�as�it�can�be�recor��e����hen�the�subjects�are���istracte���fro��the�sti�uli�an���are�not�en��a��e���in�an���sti�ulus-oriente���tas�s.

7.3.3 MMN�as�a�sin��le-ite��res�onse

Even�thou��h�earl����s��cholin��uistic��rocesses�ha���been�su����este��,�such��utative�earl���activit���re�aine����ostl���un��etecte���neuro�h��siolo��icall��.�One�reason�for�the�failure�of��ost�stu��ies�to���etect�an���earl���lan��ua��e�activation��a���be�of��eth-o��olo��ical�nature.�In��ost�brain�stu��ies�of�lan��ua��e,�lar��e���rou�s�of�sti�uli�are�in-vesti��ate���an���co��are����ith�each�other,�an���avera��e�res�onses�are�use���to���ra����eneral�conclusions�on�all��aterials�fallin���into�a�certain�cate��or��.�This�lea��s�to�the��roble��of��h��sical�sti�ulus�variance,��ith�sti�uli�havin�����ifferent��h��sical�features�(e.��.���uration,�s�ectral�characteristics,���istribution�of�soun��s�ener����,�etc.).�Differences�even�in�basic��h��sical�features��a���lea���to���ifferential�brain�activation�(Näätänen�an���Picton�1987;�Korth�an���N��u��en�1997)�that�coul���in��rinci�le�over-la���ith,��as��or�be��isinter�rete���as� lan��ua��e-relate���effects.�Equall��� i��or-tantl��,�it�raises��s��cholin��uistic�variance��roble�,��ith�sti�uli���ifferin���in�their�lin��uistic�features,�e.��.�the�frequenc���of�their�occurrence�in�the�lan��ua��e�or�their��or���reco��nition��ara�eters.�The�latter��a���be�es�eciall�����ifficult�to�control,�as���ifferent��or��s,�even�of�i��entical�len��th,�beco�e�uniquel���reco��nize���fro��their�lexical�co��etitor�environ�ent�at���ifferent�ti�es,�in�extre�e�cases�shortl���after�their�onset�or�onl���after�a�substantial��ost-offset��erio���(Marslen-Wilson�1987).�Althou��h�tra��itional�strate�����of��atchin���avera��e��ara�eters�across�sti�ulus�cat-e��ories�can�hel���iti��ate�this,�it�still�has�a�serious�caveat:�if�the�brain�res�onses�reflectin���earl���lin��uistic��rocesses�are�s�all�an���short-live���(as�all��no�n�earl���ERPs��ea�s�are),�the�variance�in�the�sti�ulus���rou���a���re��uce�or�even�re�ove�an���effects�in�the�avera��e�brain�res�onses�(Pulver�üller�1999;�Pulver�üller�an���Sht��rov�2006).�Fi��ure�1�illustrates�ho��variabilit���in�sti�ulus�set��a����otentiall���lea���to��ashin���out�of�ERP�effects.�Later�res�onses�(N400,�P600),�on�the�other�han��,��ill�li�el���survive�such�avera��in��,�as�the���are�lar��e�in�a��litu��e�an���s�an�across�hun��re��s�of��illisecon��s.�Therefore,�to�locate�the��utative�earl���effects��ith�an���certaint��,�sti�ulus�variance�shoul���be��axi�all���re��uce��.�As�MMN�is�t���i-call���a�res�onse�to�a�sin��le���eviant�ite���resente���ran��o�l���a�lar��e�nu�ber�of�ti�es�in�or��er�to�o�ti�ize�si��nal-to-noise�ratio�(SNR)�of�the�ERP,�this�offers�an�ulti�ate�control�over�the�sti�ulus�variance�b���re�ovin���it�alto��ether.

Page 196: Unconscious Memory Representations in Perception

� Are���ou�listenin��?� 1�5

7.3.4 MMN�as�a���ifference�res�onse

Mis�atch�ne��ativit���is�elicite���first�of�all�b���contrasts�bet�een�the�stan��ar���an�����eviant�sti�uli�an���is�co��ute���as�a���ifference�bet�een�res�onses�to�these�t�o�t���es.� In� turn,� this� offers� an� o��ortunit��� to� strictl��� control� acoustic� sti�ulus��ro�erties�in�lan��ua��e�ex�eri�ents.�This�can�be���one�b���usin���the�sa�e�i��entical�acoustic�contrasts�in���ifferent�ex�eri�ental�con��itions,��hile��ani�ulatin���their�lin��uistic��ro�erties.�Consi��er�the�follo�in���h���othetic�ex�eri�ental�con��itions:�(1)�the��or���‘ra��’�as�stan��ar���sti�ulus�vs.�‘ra��s’�as���eviant,�(2)�‘la��’�vs.�‘la��s’,�(3)�‘�a��’�vs.� ‘�aze’,�(4)�‘ta��’�vs.� ‘taze’.�In�all�of�these,�the�MMN�res�onses��oul���be�elicite���b���the��resence�of�sti�ulus�final�soun���[z]�in�the���eviant�as�o��ose���to�silence�in�the�stan��ar���one.�So,�a��urel���acoustic�MMN�elicite���b�����eviant-stan-��ar���sti�ulus�contrast�shoul���sta���the�sa�e.�Ho�ever,�the�four�con��itions���iffer�

Figure 1. Ph��sical�an����s��cholin��uistic�variance�bet�een���ifferent�s�o�en��or��s:�exa��le�acoustic��avefor�s�of�three�En��lish��or��s.�A��roxi�ate��ositions�of�in��ivi��ual��or���reco��nition��oints�(WRP)�are��ar�e����ith��hite�arro�hea��s.�A�h���othetical�short-live���brain�res�onse�that��i��ht�reflect�the�co��rehension�of�each��or���shortl���after�the��oint�in�ti�e��hen�it�can�be�uniquel���reco��nize���fro��the�acoustic�si��nal�is�sche�aticall���in��icate��.�Please�note�that��h��sical�features�an���WRP�latencies���iffer�substantiall���bet�een��or��s.�Avera��in���of�neuro�h��siolo��ical�activit���over�such�variable�sti�ulus�sets��ill�therefore�blur�an���thus��ini�ize�or�re�ove�an���earl���short-live���brain�res�onse�loc�e���to�the�reco��nition��oint.

Page 197: Unconscious Memory Representations in Perception

1��� Yur���Sht��rov�an���Frie��e�ann�Pulver�üller

�i��el���in�their�lin��uistic�context��hile�incor�oratin���the�i��entical�acoustic�con-trast:�in�con��ition�1,�this�contrast�constitutes�a�noun�inflection�(nu�ber�chan��e);�in�con��ition�2,�a�verb�is�inflecte���(3r����erson)�instea��.�In�con��ition�3,�the�sa�e�acoustic�contrast�si��nifies�a�co��lete�chan��e�of�the�sti�ulus�lexical�an���se�antic�features��hen�both��art-of-s�eech�infor�ation�an����eanin�����iver��e.�Finall��,�in�the� fourth�con��ition,� t�o��eanin��less��seu��o�or��s�are�contraste���offerin���an�a����itional�acoustic�control�for�an���effects�that�can�be�obtaine���in�the�other�three�con��itions.�So,�b���recor��in���MMNs�to�such�i��entical�acoustic�contrasts,�one��a���focus�on�effects�of�the���ifferent�lin��uistic�contexts��ithout�the�usual�confoun���of���iver��in���acoustic�features.�A����itionall��,�the�sa�e�i��entical���eviant�sti�uli�can�be��resente���in�a�se�arate�set�of�con��itions�as�frequent�stan��ar��s,�an���the�MMN�can�then�be�co��ute���b���usin���res�onses�to��h��sicall���i��entical�ite�s��resent-e���as�both���eviants�an���stan��ar��,�offerin���an�ulti�ate�control�over�the��h��sical�sti�ulus�features.�The�use�of�such�strictl���controlle���sti�ulation��rove���to�be�ver���successful�in�a�nu�ber�of�stu��ies.�Let�us�no��briefl���revie��this�evi��ence.

7.4 Current data: Early, parallel and automatic access to linguistic information

At�this�sta��e�(�inter�2008–2009),�ex�eri�ental�evi��ence�usin���MMN�an���non-atten�����esi��ns�has�been�collecte���in�all��ajor���o�ains�of�lin��uistic�infor�ation:��honolo��ical,�lexical,�se�antic�an���s��ntactic�(Fi��ures�2–5).�

7.4.1 Phonolo��ical��rocesses�

Sin��le��hone�es�an���their�si��le�co�binations,�s��llables,��ere�the�first�lin��uis-tic��aterials� to�be� stu��ie���usin���MMN��ara��i���.�These�ex�eri�ents� sho�e���that�native�lan��ua��e�soun��s,�e.��.�vo�els,�elicit�lar��er�MMN�res�onses�than�their�analo��ues�that���o�not�have�corres�on��in���re�resentations�in�one’s��honolo��ical�s��ste��(Dehaene-La�bertz�1997;�Näätänen�et�al.�1997)�This�ha��ene���irres�ec-tive�of�the��a��nitu��e�of�acoustic�contrasts�bet�een�the�stan��ar��s�an�����eviants.�Further�ore,��hile�MMN�is�usuall���a�bilateral�or�even�ri��ht-��o�inant�res�onse�(Paavilainen�et�al.�1991),�such��honetic�MMNs�sho�e���left-he�is�heric���o�i-nance,��otentiall���lin�in���their�ori��in�to�the�lan��ua��e-s�ecific�structures�house���in�the�left�he�is�here�(Näätänen�et�al.�1997;�Sht��rov�et�al.�1998,�2000).�Further�ex�eri�ents� even� sho�e��� ho�� chan��es� in� the� �attern� of� the� MMN� res�onse��a���reflect���evelo��ent�of�a�ne���honolo��ical�re�resentation�in�the��rocess�of�learnin��� a� lan��ua��e� b��� chil��ren� or� a��ults� (Cheour� et� al.� 1998;� Win�ler� et� al.�

Page 198: Unconscious Memory Representations in Perception

� Are���ou�listenin��?� 1�7

1999).�Later�ex�eri�ents���e�onstrate���MMN�sensitivit���to��honotactic��rob-abilities�(Dehaene-La�bertz�et�al.�2000;�Bonte�et�al.�2005,�2007),�stress��atterns��(Honbol����o�et�al.�2004;�Weber�et�al.�2004),�au��io-visual�inte��ration�of��honetic�infor�ation�(Colin�et�al.�2002,�2004)�an���various�other��honolo��ical�variables.�The� ti�in��� of� these� �honetic� an��� �honolo��ical� �is�atch� ne��ativities� ran��e���fro��close�to�100��s�(Rinne�et�al.�1999)�to�nearl���~200��s�(Sht��rov�et�al.�2000).�

Acco��o��ation�of�such���ata�in�the�theoretical�fra�e�or��of�the�MMN�re-quire���a�certain�revision�of�it.�It��as�su����este���that,�in�a����ition�to�chan��e-��etec-tion�an���short-ter���e�or����rocesses,�MMN�is�sensitive�to�lon��-ter���e�or���traces� �re-for�e��� in� the� subject’s� neural� s��ste�� in� the� �rocess� of� their� �revi-ous�ex�erience��ith�s�o�en�lan��ua��e�(Näätänen�et�al.�1997,�2001;�Sht��rov�et�al.�2000).�I��ortantl��,�this�i��lie���that�such�lon��-ter���e�or���traces�for�lan��ua��e�ele�ents�can�beco�e�activate���in�the�brain�b���a���eviant�sti�ulus�in�an�o����-ball�sequence�an���this�s�ecific�activation�can�be�recor��e���neuro�h��siolo��icall���even��ithout�attention�to�the�sti�ulus�or�an���sti�ulus-oriente���tas�.�This�le���to�fur-ther� MMN� ex�eri�ents� �ith� lan��ua��e� sti�uli� that� tar��ete��� ‘hi��her-or��er’� lin-��uistic��rocesses.

Figure 2. MMN�reflections�of�earl���auto�atic�access�to�lin��uistic�infor�ation:���honolo��ical�enhance�ent�for�s��llables�(a��a�te���fro��Sht��rov�et�al.�2000),�lexicall���en-hance����a��netic�MMN�for��eanin��ful��or��s�(a��a�te���fro��Sht��rov�et�al.�2005);�fMRI�counter�art�of�lexical�enhance�ent�for��or��s�(a��a�te���fro��Sht��rov�et�al.�2008).

Page 199: Unconscious Memory Representations in Perception

1��� Yur���Sht��rov�an���Frie��e�ann�Pulver�üller

7.4.2 Lexical��rocesses�

Si�ilar� to� the� �honolo��ical� enhance�ent� of� the� MMN� in� res�onse� to� native�lan��ua��e’s��hone�es�an��� s��llables,��e� foun��� that��is�atch�ne��ativit��� evo�e���b���in��ivi��ual��or��s��as���reater�than�that�to�co��arable��eanin��less��or��-li�e�(i.e.,�follo�in����honolo��ical�rules�of�the�lan��ua��e)�sti�uli.�In�a�series�of�stu��ies,��e��resente���subjects��ith�sets�of�acousticall����atche����or���an����seu��o�or���sti�uli�an���foun���an�increase���MMN�res�onse��henever�the���eviant�sti�ulus��as�a��eanin��ful��or���(Pulver�üller et�al.�2001,�2004;�Sht��rov�an���Pulver�üller�2002;�Sht��rov�et�al.�2005).�This�enhance�ent,�t���icall����ea�in���at�100–200��s,�is�best�ex�laine���b���the�activation�of�cortical��e�or���traces�for��or��s�realise���as���istribute���stron��l���connecte����o�ulations�of�neurones�(Pulver�üller�an���Sht��-rov�2006).�The�lexical�enhance�ent�of�the�MMN,�re�eate��l���confir�e���b���our���rou�,��as�also���e�onstrate���b���other���rou�s�usin���various�sti�ulus�set-u�s�an���lan��ua��es�(Kor�ilahti�et�al.�2001;�Kujala�et�al.�2002;�Sitti�ra�a�orn�et�al.�2003;�En��rass�et�al.�2004;�Petti��re��et�al.�2004)�incu��in���a�recent�vali��ation�of�its�su�e-rior-�(STC)�an����e��io-te��oral�(MTG)�sources�b���fMRI�(Sht��rov�et�al.�2008).�Our�stu��ies�in��icate���that�the�lexical�status�of�the���eviant�sti�ulus��as�relevant�for� elicitin��� the� MMN,� but� the� lexical� status� of� the� stan��ar��� sti�ulus� ��i��� not�si��nificantl���affect�the�MMN�a��litu��e�(Sht��rov�an���Pulver�üller�2002).�Other�re�orts�su����este���that�the�event-relate���brain�res�onse�evo�e���b���the�stan��ar���sti�ulus��a���also�be�affecte���b���its�lexical�status�(Diesch�et�al.�1998;�Jacobsen�et�al.�2004).�More���etaile���investi��ations�also�sho�e���that�MMN��a���be�sensitive�to��ore�than�a���eneral� lexicalit��,�an���can�serve�as�an� in��ex�of��or��-cate��or��-s�ecific��rocessin���su��ortin��,�for�exa��le,�the�notion�of�earl����rocessin���an���re�resentational���ifferences�bet�een�verbs�an���nouns�(Hastin���et�al.�2008).

Further� scrutin��� of� �or��-elicite��� MMN� activations� usin��� state-of-the-art�technolo����,�such�as��hole-hea���hi��h-��ensit���MEG�co�bine����ith�source�recon-struction�al��orith�s,�allo�e���for���etailin����or��-relate����rocesses�in�the�brain�at�un�rece��ente���s�atio-te��oral�scale.�Usin���MMN�res�onses�to�sin��le��or��s,��e�coul�����e�onstrate�ho��the�activation�a��ears�first�in�the�su�erior�te��oral�lobe�shortl��� (~130��s)�after� the� infor�ation�allo�s� for� sti�ulus� i��entification,�an��,�in�a���efine���or��er�(�ith�a���ela���of�~20��s),�s�rea��s�to�inferior-frontal�cortices,��otentiall���reflectin���the�flo��of�infor�ation�in�the�lan��ua��e-s�ecific��eris��lvian�areas�in�the��rocess�of�lexical�access�(Pulver�üller�et�al.�2003).�Further�ore,��e�foun���a�correlation�bet�een� the� latenc���of�MMN�res�onses� in� in��ivi��ual� sub-ject�an���the�sti�ulus-s�ecific��or���reco��nition��oints���eter�ine���behaviourall��,��hich�su����ests�that��or��-elicite���MMN�reflects�the��rocesses�of�in��ivi��ual��or���reco��nition�b���the�hu�an�brain�an���the�t�o��rocesses��a���therefore�be�lin�e���(Pulver�üller�et�al.�2006).

Page 200: Unconscious Memory Representations in Perception

� Are���ou�listenin��?� 1�9

The��attern�of�MMN�res�onse�to�in��ivi��ual��or�����eviants,�as�a�result�of�such�increasin��l���accu�ulatin���evi��ence,�beca�e�consi��ere���as�a��or��’s�neural�“si��-nature”,�its��e�or���trace�activate���in�the�brain.�This,�in�turn,�offere���a�un�aral-lele���o��ortunit��� to� investi��ate�neural��rocessin���of� lan��ua��e� in�non-attention���e�an��in��� an��� tas�-in��e�en��ent� fashion.� In��ee��,� investi��ations� of� lan��ua��e�function�usin���MMN�that� follo�e����ere�able� to��rovi��e�a���reat� level�of���etail�on�the�s�atio-te��oral��atterns�of�activation��otentiall���relate���to��rocessin���of�se�antic�infor�ation�attache���to�in��ivi��ual��or��s.

7.4.3 Se�antic��rocesses�

To�tar��et�se�antic��or����ro�erties�usin���MMN,�the�ver���first�stu��ies�in�this���o-�ain�utilise����re��ictions�of�the�se�antic�so�atoto�����o��el�(SSM,�Pulver�üller�1999,�2005).�SSM�is�a��art�of�a���eneral�fra�e�or���aintainin���that��or��-s�ecific��e�or���traces�exist�in�the�brain�as���istribute���neuronal�circuits�for�e���in�the��rocess�of��utual�connection�stren��thenin���bet�een���ifferent�(even���istant)�ar-eas,�as�actions,�objects�or�conce�ts�are�ex�erience���in�conjunction��ith�the��or��s�use���to���escribe�the��(Hebb�1949;�Pulver�üller�1999).�The�referential��eanin���is�an�inte��ral��art�of�a��or��’s�se�antics�(Fre��e�1980).�Action��or��s’�re��ular�usa��e�for�referrin���to�ar�/han���actions�(�or��s�such�as�pick, write)�or�le��/foot�actions�(kick, walk)�is�therefore�an�essential�characteristic�of�their��eanin���(even�thou��h�their� se�antics� �a��� not� be� exhauste��� b��� it).� If� lexical� re�resentations� beco�e��anifest� corticall��� as�neuronal� asse�blies� an��� the�actions� referre��� to�b��� these��or��s�are�so�atoto�icall���re�resente���in��otor�areas�of�the�brain�(Penfiel���an���Ras�ussen�1950),�the�se�antic�lin�s�bet�een�neuronal�sets�in�these�cortical�re-��ions�shoul���realise�the�se�antic�relationshi��bet�een�the��or���for�s�an���their�actions�(Pulver�üller�2001).�Cruciall��,�this� lea��s�to�the�s�ecific��re��iction�that�action��or��s��ith���ifferent�reference���o�ains�in�the�bo�����also�activate�the�cor-res�on��in���areas�of��otor�cortex.�This�clai�,��hich�receive���stron���su��ort�fro��conventional� neuroi�a��in��� stu��ies� (Hau� et� al. 2004;� Hau�� an��� Pulver�üller�2004),��as��ut�to�test�in�MMN�ex�eri�ents�usin�����ifferent��etho��s�(EEG,�MEG)�an�����ifferent�sti�uli�of�t�o�lan��ua��es�(En��lish,�Finnish).

In�a����ition�to�the�usuall���observe���su�erior�te��oral�MMN�sources,�the�ac-tivation�elicite���b���the��or��s�referrin���to�face�an��/or�ar���ove�ents�involve���inferior�fronto-central�areas�li�el���inclu��in���the�cortical�re�resentation�of�the�u�-�er�bo����� (Sht��rov�et� al.�2004;�Pulver�üller� et� al.�2005).�Further�ore,� the� le��-relate����or��s�elicite���a�stron��er�su�erior�central�source�co��atible��ith�the�le���sensori�otor� re�resentation.�This� le��-�or��� s�ecific� su�erior� fronto-central� ac-tivation��as�seen�later�(~170��s)�than�the��ore�lateral�activation�for�face-�an���

Page 201: Unconscious Memory Representations in Perception

190� Yur���Sht��rov�an���Frie��e�ann�Pulver�üller

ar�-relate����or��s�(~140��s).�These�s�atio-te��oral�characteristics�su����est�that�MMN�sources�in��eris��lvian�areas�alon����ith�near-si�ultaneous�activation���is-tant�fro��the�S��lvian�fissure�can�reflect�access�to��or����eanin���in�the�cortex.�The��ini�al���ela��s�bet�een�local�activations��a���be��e��iate���b���con��uction���ela��s�cause���b���the�travelin���of�action��otentials�bet�een�cortical�areas.�

These� ��ata� su����este��� that� �rocessin��� of� se�antic� features� of� action� �or��s�is�reflecte���in�the�MMN�as�earl���as�140–170��s�after�acoustic�si��nals�allo��for��or���i��entification.�Si�ilar�to�lexical�access�an���selection,��eanin���access��a���therefore�be�an�earl���brain��rocess�occurrin����ithin�the�first�200��s,�an���the�t�o��rocesses��a���thus�be�near-si�ultaneous.

Si�ilar,�if�not��ore��rovo�in��,�results��ere��ro��uce���b���MMN�ex�eri�ents�on�se�antic�context�inte��ration.�In�these,�co��lete��hrases�or��or���co�bina-tions��ere��resente���to�the�subjects�in�a��assive�non-atten���o����ball���esi��n��hile�the��h��sical�sti�ulus��ro�erties��ere�strictl���controlle���for.�So�eti�es,�the���e-viant�co�binations�inclu��e���a�se�antic��is�atch�bet�een�the��or��s,�si�ilar�to�the�establishe���se�antic�violation��ara��i�����no�n�to��ro��uce�an�N400�re-s�onse.� Sur�risin��l��,� these� contextual� sentence-level� violations� �o��ulate��� the�MMN�res�onse�rather�earl���in�its�ti�e�course�an��,�in�the�sa�e�ti�e,�elicite���no�N400-li�e�res�onse�in�the��assive�o����ball��ara��i���.�In�one�of�the�stu��ies,�this��o��ulation��as�seen�as�earl���as�~115��s�after� the��or��s�coul���be�reco��nise���as���ifferent�an����as�sho�n�to�be��e��iate���b���the�su�erior-te��oral�an���infe-rior-frontal�cortices�in�the�left�he�is�here�(Sht��rov�an���Pulver�üller�2007).�In�another�stu�����(Mennin���et�al.�2005),�the�se�antic�abnor�alities��ere���etecte���b���MMN�res�onses�at�150–200��s.�Cruciall��,�these�se�arate��roofs�of�MMN�re-flections�of�se�antic�context�inte��ration��ere�obtaine���in���ifferent�laboratories��hich� utilise��� ��iver��in��� techniques,� sti�ulation� �rotocols� an��� even� sti�ulus�lan��ua��es� (Finnish,� Ger�an).� These� MMN� stu��ies� sho�e��� hi��her-or��er� �ro-cesses�of�se�antic�inte��ration�of�s�o�en�lan��ua��e�as�occurrin���not�onl���outsi��e�the�focus�of�attention,�but�also��ell�before�the�N400�ti�e�ran��e,��ithin�200��s.�This�stron��l���su��orts�the�notion�of��arallel�access�to���ifferent�t���es�of�lin��uistic�infor�ation.�A����itional�su��ort�for�this�vie���as�foun���in�recent�visual�stu��ies�(Sereno�et�al.�2003;�Penolazzi�et�al.�2007)�in��icatin���that�the�se�antic�context�inte��ration�affect�visual�brain�res�onses�at�latencies�un��er�200��s.

An�even�hi��her-level�as�ects�of��ra���atic��rocessin��,�such�as�sentence�incor-�oration�in�the���iscourse�context�or�even��orl����no�le����e,��hich��ere�sho�n�to�have�effects�in�the�N400�ti�e�ran��e��hen�attention�is�allocate���to�sti�uli�(van�Ber�u��et�al.�2003;�Ch�illa�an���Kol��2005),�ha���so�far�not�been�touche���b���the�MMN�research�into�earl���auto�aticit��.�It� is��orth�notin���ho�ever�that� there� is�alrea�����a���iscussion�as�to��hat�extent�lin��uistic�MMNs�are�affecte���b���the���eneral�lan��ua��e� environ�ent.� For� exa��le,� in� one� stu����� (Peltola� an��� Aaltonen� 2005)�

Page 202: Unconscious Memory Representations in Perception

� Are���ou�listenin��?� 191

res�onses�to�the�sa�e�sti�uli�in�the�o����ball��ara��i����a��lie���to�bilin��ual�sub-jects���iffere����hen�the����ere�s�ecificall���tol����hich�of�the�t�o�lan��ua��es�(Finnish�or�En��lish)��ere�use���in�sti�ulation.�Ho�ever,� in�a���ifferent�stu�����(Win�ler�et�al.�2003),��here� the�ex�eri�ental� environ�ent� (interaction��ith�ex�eri�enter,�instructions,�conversations��rior�to�the�stu�����etc)��as�s�itche���bet�een�the�t�o�fa�iliar�lan��ua��es�(Hun��arian�vs.�Finnish),�this�environ�ental�chan��e�ha���no�ef-fect�on�the�lin��uistic�MMN�res�onses.�A����itional�research�is�therefore�nee��e���to�ex�lore�such��ossible�hi��h-level�interactions.

7.4.4 S��ntactic��rocesses�

The���o�ain�of��or�ho-s��ntactic�infor�ation��rocessin���in�the�brain��as�also�a����resse���in�a�nu�ber�of�ex�eri�ents.�To�control�exactl���for��h��sical�sti�ulus��ro�erties,��e�once�a��ain��resente��� i��entical� s�o�en�sti�uli� in���ifferent�con-texts.�In�these�ex�eri�ents,�the�critical��or���occurre���after�a�context��or����ith��hich�it��atche���in�s��ntactic�features�or��is�atche���s��ntacticall��;�this�has�been�a� stan��ar���a��roach�use��� in�neuro�h��siolo��ical� stu��ies�on� s��ntax� (Osterhout�1997).�We�a��ain�use�����ifferent��etho��s�(EEG�an���MEG)�an���lan��ua��es�(Finn-ish,�Ger�an�an���En��lish),�enablin���us�to���ra����eneralise���conclusions��hich�is�

Figure 3. Earl���neuro�h��siolo��ical�reflections�of�auto�atic�se�antic�infor�ation��ro-cessin��:�sin��le-�or���cate��or��-s�ecific�se�antic�effects�(a��a�te���fro��Pulver�üller�et�al.�2005)�an���se�antic�context�inte��ration�(a��a�te���fro��Sht��rov�an���Pulver�üller�2007).

Page 203: Unconscious Memory Representations in Perception

192� Yur���Sht��rov�an���Frie��e�ann�Pulver�üller

es�eciall���i��ortant�in�MMN�stu��ies��hen�onl���a�ver���li�ite���set�of�sti�uli�can�be�use���in�each�sin��le�stu����.�

The�first�ex�eri�ent� loo�e���at�the�neuro�h��siolo��ical�res�onses�to�Finnish��ronoun-verb��hrases�en��in��� in�a�verb� suffix��hich���i���or���i���not�a��ree��ith�the��ronoun� in��erson�an���nu�ber;� this��as���one� in�an�ortho��onal���esi��n� in��hich��h��sicall���i��entical�sti�uli��ar�e���a�s��ntacticall���con��ruent�or�incon��ru-ent�event�(e.��.�he brings�vs.�*we brings),�thus�full���controllin���for��h��sical�sti�ulus�features.�The�results�sho�e���an�increase�of�the��a��netic�MMN�to��or��s�in�un-��ra��atical�context�co��are����ith�the�MMNs�to�the�sa�e��or��s�in���ra��ati-cal�context.�Si�ilar�results��ere�also�seen� in�En��lish�an���Ger�an�ex�eri�ents�(Pulver�üller�an���Sht��rov�2003;�Pulver�üller�an���Assa��ollahi�2007).�The�laten-cies��here���ra��aticalit���effects��ere�foun���varie���so�e�hat�bet�een�stu��ies�but�res�onses��ere���enerall����resent��ithin�200��s�after�the��or���reco��nition��oint,�so�eti�es�startin���as�earl���as�at�100��s�(Pulver�üller�an���Sht��rov�2003;�Sht��rov et� al.� 2003).� The� cortical� loci� �here� the� �ain� sources� of� the� s��ntactic�MMN��ere� localize���also�varie��.�MEG�results� in��icate���a���istribute���su�erior�te��oral��ain�source��ith��ossibl���so�e��ea��effects�in�inferior�frontal�cortex,��hereas�the�EEG�su����este���the�o��osite,�a��ronounce�����ra��aticalit���effect�in�the�left�inferior�frontal�cortex��ith��ossibl����inor�su�erior-te��oral�contribu-tion.�This���iver��ence�reflects�the��revious�neuroi�a��in���literature�on�the�cortical�basis� of� s��ntax,� �here� this� �o��ule� is� so�eti�es� localize��� in� frontal� areas� an���so�eti�es�in�te��oral�lobes�(e.��.�Kaan�an���S�aab�2002;�Born�essel�et�al.�2005;�Kircher�et�al.�2005).�It�is�li�el���therefore�that���ifferent�areas�in��eris��lvian�cortex�contribute�to���ra��atical�an���s��ntactic��rocessin���an���their�activit����a���be���if-ferentiall���reflecte���in�EEG�an���MEG�recor��in��s���ue�to�the��no�n�s�ecificit���of�these�techniques�to��ositionin���an���orientation�of�current�sources�in�the�brain.

These�fin��in��s� of� earl��� s��ntactic� �rocessin��� outsi��e� the� focus� of� attention��ere�further�su��orte���b���stu��ies�of�other���rou�s��hich�use���si�ilar��ara��i���s�an���foun���a�s�ecific�MMN�res�onse��ea�in���at�150–200��s��henever�the��re-sente���sentence�sti�ulus�containe���a�s��ntactic��is�atch�(Mennin�� et�al.�2005;�Hastin���et�al.�2007).�Re�ar�abl��,�so�e�of�this��or��use���a��ore�elaborate�sti�u-lus� setu�,��ith� lon��er� sentences� (unli�e�our� t�o-�or����hrases;�e.��.�we *comes�in� Pulver�üller� an��� Sht��rov� 2003,� vs.� Die Frau düngt *den rosen im Mai in��Mennin���et�al.�2005),�an���foun���the�s��ntactic��is�atch�effects�nevertheless.�

One� of� these� ex�eri�ents� �as� s�ecificall��� ��esi��ne��� to� a����ress� the� ques-tion� of� �hether� the� s��ntactic� MMN� reflects� ��ra��aticalit��� �er� se,� or� si��l���follo�s�sequential�strin����robabilit��.�To�this�en��,�strin��s��ith�lo��an���hi��h�se-quential��robabilit��,�also�var��in���in�their���ra��aticalit��,��ere�use��.�The�results�clearl��� in��icate��� that� the� increase��� a��litu��e� of� s��ntacitc� MMN� is� lin�e��� to��

Page 204: Unconscious Memory Representations in Perception

� Are���ou�listenin��?� 193

��ra��aticalit���as�such�rather�then�the��ere��robabilit���of�ite��co-occurance�in�the�lan��ua��e�(Pulver�üller�an���Assa��ollahi�2007).

Further,�to�rule�out�the�re�etition�confoun��s�of�the�MMN���esi��n,�a���ifferent�stu�����(Hastin���an���Kotz�2008)�use���a� si�ilar�acousticall���controlle���a��roach��ithout��ulti�le�re�etitions,�an���foun���the�earl���s��ntactic�res�onse�in��e�en��ent�of�attention�in�the�sa�e�ti�e�ran��e.�

The� earl��� s��ntactic� MMN� rese�bles� the� ELAN� co��onent� (Neville� et� al.�1991;�Frie��erici et� al.� 1993)��hich�has�been� su����este��� to� in��ex�earl��� s��ntactic�structure�buil��in���(Frie��erici�2002).�S��ntactic�MMN�results�su��ort� this� inter-�retation.�I��ortantl��,�the���also�sho��that�the�earl���s��ntactic��rocessin���in�the�brain���oes�not�require�focuse���attention�on�the�lan��ua��e�in�ut.�In�this�sense,�earl���s��ntactic��rocessin���see�s�to�be�auto�atic.�This�a��ears�true�even�for�a��ree�ent�violations,� �hich� are� consi��ere��� �ore� ��e�an��in��� co��utationall��� an��� �hich���i��� not� elicit� ELAN� in� the� �ast.� The� late� �ositivit��� (P600),� �hich� is� abolishe���in�the��assive�o����ball�tas�,��a���in�turn�reflect�secon��ar���controlle���atte��ts�at��arsin��� a� strin��� after� initial� anal��sis� has� faile��� (Osterhout� an��� Holco�b� 1992;�Frie��erici�2002).

Figure 4. Earl���auto�atic�s��ntax�in�the�brain:�s��ntactic�MMN�(a��a�te���fro���Pulver�üller�an���Sht��rov�2003;�an���Sht��rov�et�al.�2003),�an���absence�of�attention�effects�on�the�earliest�sta��es�of�s��ntactic��rocessin���(a��a�te���fro��Pulver�üller�an���Sht��rov�2003;�an���Sht��rov�et�al.�2003).

Page 205: Unconscious Memory Representations in Perception

194� Yur���Sht��rov�an���Frie��e�ann�Pulver�üller

7.4.5 Attentional�control�or�auto�aticit��?

One��a���ar��ue�that���e�onstratin���lan��ua��e-relate���effects�in�a��ara��i�����here�subjects�are� instructe���to�atten���to�a�vi��eo�fil��or�rea���a�boo���hile� lan��ua��e�sti�uli�are��resente�����oes�not�control�strictl���for�attentional��ith��ra�al.�To���ra��conclusions�on�the�auto�aticit���of�the��rocesses�investi��ate���fro��such�stu��ies,�subjects��ust�strictl��� follo��the� instruction�to� tr��� to� i��nore� the�s�eech�sti�uli��hich��a���be���ifficult�to�verif��.�It��oul���be��ore���esirable�to�control�for�the�level�of�attention�in�each�subject�throu��hout�the�ex�eri�ent��ore�closel��.�Therefore,��e��erfor�e���an�ex�eri�ent�to�further�investi��ate�the�role�of�attention�in�lan��ua��e��rocessin��� b��� co��arin��� the� classic� MMN� �ara��i���� �ith� its� �o��erate� atten-tional��ith��ra�al�b���a�silent�vi��eo�fil���ith�a���istraction�tas���here�subjects�ha���to�continuousl����erfor��a�non-s�eech�acoustic���etection� tas�.�Lan��ua��e� (s��n-tactic)�sti�uli��ere�onl����la��e���throu��h�the�ri��ht�ear��hile�acoustic�sti�uli��ere���elivere���to�the�left�ear.�In�a�strea�in���con��ition,�subjects�ha���to��ress�a�button�to�a�“��eviant”�acoustic�sti�ulus�in�the�left�ear��hile,�at�the�sa�e�ti�e,�the�lan��ua��e�sti�uli� �ere� still� �la��e��� to� the� ri��ht� ear.� In� the� other� con��ition,� subjects� �ere�allo�e��� to� �atch� a� vi��eo� as� usual,� �ithout� further� ��istraction,� �hile� the� sa�e�sti�uli,�non-s�eech�soun��s�an���s��ntactic�sti�uli,��ere��resente��.�Results�sho�e���a�re�lication�of�the���ra��aticalit���effect,�i.e.�stron��er�MMNs�to�un��ra��atical��or���strin��s�than�to�un��ra��atical�ones.�U��to�a�latenc���of�~150��s,�there��as�no���ifference�bet�een�the�tas�s��ith���ifferent�attentional��ith��ra�al.�Onl���later,��e�foun���si��nificant�interactions�of�the�tas��an���attention�factors�in��icatin���that�at�these�later�sta��es�the���ra��ar��rocesses�reveale���b���the�MMN��ere�influence���b���attention�an���tas����e�an��.�We�inter�ret�this�as�stron���evi��ence�for�the�atten-tion�in��e�en��ence�of�the�earl����art�of�the�MMN�an���for�the�auto�aticit���of�earl���s��ntactic� anal��sis� (Pulver�uller� et� al.� 2008).� Re�ar�abl��,� �e� foun��� an� al�ost�i��entical�ti�e�course�of�lan��ua��e-attention�interaction�(earl���auto�aticit���u��to�~140–150��s��ith�attention-��e�en��ance�at�later�sta��es)��hen�investi��atin���lexi-cal�contrasts�(�or��s�vs��seu��o�or��s,�Gara��nani�et�al.�2009;�Sht��rov�et�al.�2010).�These���ata�resonate��ell��ith�earlier�su����estions�of�auto�atic�earl���sta��es�as�o�-�ose���to�attention-��e�en��ant�late��rocesses�in�e.��.�s��ntactic�anal��sis��rocessin���(Hahne�an���Frie��erici�1999;�Gunter�et�al.�2000;�Frie��erici�2002).

7.5 Discussion: Theories, challenges and directions

We�have�revie�e���the��re�ises�for�neuro�s��siolo��ical�research�into�the�earliest�sta��es�of� lan��ua��e��erce�tion�an���their�auto�aticit���an���ho��the�MMN�coul���be�use���to�hel��eluci��ate�the�.�Not�onl���has�the�MMN�been�foun���sensitive�to�

Page 206: Unconscious Memory Representations in Perception

� Are���ou�listenin��?� 195

these� ��ifferent� t���es� of� lin��uistic� infor�ation� but� it� has� also� �rove��� useful� for���isentan��lin���the�neural�correlates�of�corres�on��in����rocesses�fro��those�relate���to�attention�or�sti�ulus-oriente���tas�s�an���strate��ies.�These�stu��ies�also�allo�e���for�the�strictest��ossible�control�over�the�sti�ulus��ro�erties��ini�izin���the��os-sibilit���that�the�obtaine���effects�are�si��l�����ue�to�acousticall���or��s��cholin��uisti-call���unbalance���sti�ulus�sets.

First�of�all,�these�stu��ies�she���ne��li��ht�on�the�ti�e�course�of�access�to�lin-��uistic�infor�ation.�Throu��hout�these�ex�eri�ents,���ifferent�t���es�of�sti�uli,�in-clu��in����honetic/�honolo��ical,�lexical,�se�antic�an���s��ntactic�contrasts,��o��u-late���evo�e���res�onses�at�latencies��ithin�100–200��s�after�the�relevant�acoustic�infor�ation��as��resent�in�the�in�ut�(Fi��ure�5).�These���ata�su��orte��,�an���ver���stron��l��� so,��arallel�or�near-si�ultaneous�access� to���ifferent� t���es�of� lin��uistic�infor�ation�co��encin���ver���ra�i��l���in�the�brain�shortl���after�the��or��s�coul���be�uniquel���i��entifie��.�Such�access��as�envisa��e���in�so�e�earlier��s��cholin��uistic�stu��ies�an���has�re�aine���hotl�����ebate���until�no�.

Secon��l��,� in� s�atial���o�ain,� these� results��rovi��e���stron���su��ort� for� the�existence� of� ��istribute��� neural� circuits� �hich� �a��� un��erlie� the� �rocessin��� of�the� inco�in��� lin��uistic� infor�ation.� For� exa��le,� an� inter�la��� of� activations�bet�een� su�erior-te��oral� an��� inferior-frontal� cortices� �as� sho�n� as� occur-rin��� in� the��rocess�of��or����erce�tion�(Pulver�üller�et�al.�2003;�Sht��rov�an���Pulver�üller�2007).�De�en��in���on�the�exact�referential�se�antics�of�the��or��s��resente��,� constellations� of� areas� in� te��oral,� inferior-frontal,� fronto-central���orsal� cortices� �ere� s�ar�e��� b��� the� sti�uli,� so�eti�es� even� s�rea��in��� to� the�ri��ht�he�is�here�(Pulver�üller�et�al.�2004,�2005).�Even��ore�refine����atterns�of�activation���ela��s�bet�een�areas��ere�foun����hen�scrutinisn���MMN�source�����-na�ics�for�non-s�eech�noise�an�����ifferent�lin��uistic�sti�uli�(Pulver�üller�an���Sht��rov�2009).

To��rovi��e�an�account�of�earl���near-si�ultaneit���an���the�critical�latenc�����if-ferences� bet�een� cortical� area� activations,� a� �o��el� is� necessar��� that� s�ells� out�lan��ua��e�an���conce�tual��rocesses�in�ter�s�of��echanistic�neuronal�circuits�an���their�activation.�Mechanistic�brain-base����o��els�of�lan��ua��e�are�available�since�the�earl���1990s�(Da�asio�1990;�Mesula��1990;�Pulver�üller�an���Preissl�1991;�Braitenber���an���Schüz�1992).�We�hi��hli��ht�here�an�account�that��ostulates�lexi-co-se�antic�circuits��ith�s�ecific�cortical���istributions�(Pulver�üller�an���Preissl�1991;�Pulver�üller�1999,�2005),�as��e�believe�that�such�s�ecificit��� is�necessar���for� ex�lainin��� fine-��raine��� ��ela��s� bet�een� near-si�ultaneous� area� activations.�The��o��el��osits� that� stron��l��� connecte��� neuronal� ense�bles� s�rea��� out� over���ifferent�sets�of�areas�of�the�cortex�are�the�basis�of�co��nitive�an���lan��ua��e��rocess-in��.�The��o�entar���ex�losion-li�e� i��nition�of�one�of� these�net�or�s�accounts�for� near-si�ultaneit��� of� area-s�ecific� activations� an��� the� con��uction� ��ela��s� of�

Page 207: Unconscious Memory Representations in Perception

19�� Yur���Sht��rov�an���Frie��e�ann�Pulver�üller

Figure 5. Near-si�ultaneous�earl���neural�reflection�of��s��cholin��uistic�infor�ation�t���es�in�s�eech�co��rehension:�su��ar��.�Phonolo��ical��rocessin���beca�e��anifest�in�a��o��ulation�of�the�MMN�aroun���140��s�(Sht��rov�et�al.�2005)�an���lexicalit����as�reflecte���b���t�o�sources�in�su�erior-te��oral�an���inferior�frontal�cortex,�s�ar�e��,�res�ectivel��,�at�136�an���158��s�(Pulver�üller�et�al.�2003).�S��ntactic�violations�elicite���a�s��ntactic�MMN�at�about�the�sa�e�ti�e,��ith�sources�in�inferior�frontal�an���su�erior-te��oral�cortex�(Sht��rov�et�al.�2003;�Mennin���et�al.�2005).�Se�antic�effects��ere�seen�at�140–170��s��hen�the�sa�e�s��llables��ere��resente���in��or��s�that�in��icate���face/ar���ove�ents,�ar��or�le���actions�(Pulver�üller�et�al.�2005).�These�results�are�consistent��ith�near-si-�ultaneous�earl���access�to���ifferent�t���es�of��s��cholin��uistic�infor�ation.�Criticall��,�there��ere�fine-��raine���ti�e�la�����ifferences,�es�eciall���in�the�se�antic���o�ain:�Le��-relate����or��s�(e.��.,�“�ic�”)�activate���the�central-��orsal�sensori�otor�le���re�resentation�30��s�later�than�inferior-frontal�areas��ere�s�ar�e���b���face/ar�-relate����or��s�(“eat”,�“�ic�”).�This�sho�s�cate��or��-s�ecificit���in�the�te��oral�structure�of�se�antic�brain�activation.

Page 208: Unconscious Memory Representations in Perception

� Are���ou�listenin��?� 197

cortical�fibers��ithin�the�circuits�ex�lain�fine-��raine���activation�ti�e�la��s�in�the��illisecon���ran��e.

Neuronal�circuits��rocessin���s�o�en��or���for�s�co��rise�at�least�neurons�in� su�erior-te��oral� cortex� activate��� b��� �honetic� features� of� a� s�o�en� �or��,�neurons� in� inferior-frontal� cortex� �ro��ra��in��� an��� controllin��� articulator����ove�ents� an��� a����itional� neurons� connectin��� the� acoustic� an��� articulator����o�ulations.�The�strict� si�ultaneit���of� acoustic,��honolo��ical� an��� lexical��ro-cessin���in��exes�is�ex�laine���b���this��o��el,�as�neuronal��o�ulations�in�the�sa�e�local�structure,�in�su�erior-te��oral�cortex,�are�assu�e���to�contribute�to�acous-tic,��honolo��ical�an���lexical��rocesses.�Therefore,�con��uction�ti�es�of�the�au��i-tor��� in�ut� to� these� critical� sites� are� rou��hl��� the� sa�e.� The� su�erior-te��oral�lobe�in��ee���see�s�to�contribute�to�all�of�these��rocesses�(Scott�an���Johnsru��e�2003;�U��en�a�� et�al.�2006)�an���local�activation���ifferences�bet�een���ifferent�soun��s�reveale���b���fMRI�(as��ell�as�MEG;�Pulver�üller�an���Sht��rov�2009)��a���be�ex�laine��,�in��art,�b���their���ifferential�lin�a��e�to�articulator���circuits.�I��or-tantl��,� the�evi��ence� for� stron��er�cortical�activation�(Sht��rov�an���Pulver�üller�2002)�an����otor� lin�s�(Buccino�et�al.�2001)�of��or��s�co��are����ith��seu��o-�or��s� su��orts� the� existence� of� �erce�tion-action� circuits� for� s�o�en� �or��s.�Further� evi��ence� that� these� lexical� �e�or��� net�or�s� lin�� su�erior-te��oral�(acoustic)� circuits� to� inferior-frontal� (s�eech� �otor� �lannin��)� circuits� co�es�fro�� i�a��in����or��revealin���coactivation�of� these�areas� in� s�eech��rocessin���as��entione���above.�I��ortantl��,�close�exa�ination�sho�e���that�the�frontal�ar-eas�involve���are�s�ar�e���10–20��s�after�su�erior-te��oral�activation,��hich�is�consistent��ith�the�con��uction�ti�e�of�action��otentials� in�the��ost�co��on�cortico-cortical�fibers� in�hu�an�cortex� (Pulver�üller�et�al.�2003;�Sht��rov�an���Pulver�üller�2007).

The�MMN�stu��ies�of�s��ntactic��rocessin����a��,�in�turn,�contribute�to�theo-risin���about�neural�i��le�entation�of���ra��ar.�Differences�in�brain�res�onses�bet�een���ra��atical�an���un��ra��atical�strin��s�are�usuall���inter�rete���as�evi-��ence�that�so�e���ra��ar-relate����rocess�is�activate���b�����ra��atical���eviance.�The� MMN� stu��ies� �rovi��e��� an� alternative� vie�.� Because� �h��sicall��� i��entical�ite�s��ere�use���in���ra��atical�or�erroneous�contexts,�an���so�eti�es�outsi��e�of�an���context,��e�coul�����etail�the�res�onse�����na�ics��ore��recisel��.�Co��are����ith�the�control�con��itions��here��or��s��ere��resente���out�of�lin��uistic�context�(Pulver�üller�an���Sht��rov�2003),�the���ra��atical�violation���i���not�increase�si��-nificantl���the�earl���brain�res�onse,�but,�instea��,�the���ra��aticall���correct�strin����o��ulate���earl����or��-evo�e���activit���b���re��ucin���it.�The��ost�feasible�ex�lana-tion�for�this�is�that�neural�re�resentations�of�frequentl���co-occurrin���ite�s�(e.��.�a��ronoun�an���a�corres�on��in���verb�affix,�as� in�he brings)�beco�e�lin�e���via,�for�exa��le,�a�sequence���etector�(Knoblauch�an���Pulver�üller�2005).�Therefore,�

Page 209: Unconscious Memory Representations in Perception

19�� Yur���Sht��rov�an���Frie��e�ann�Pulver�üller

�resentation�of�one�of�such�ite�s�lea��s�not�onl���to�the�activation�of�its�o�n�re�-resentation�in�the�brain,�but�also�to��reactivation�of�the�relate���re�resentation.�This�is�equivalent�to��ri�in����hich�is��no�n�to�re��uce�the�size�of�neurobiolo��i-cal�res�onses.�In�s��ntactic�context,��or��/�or�he�e�re�resentations�can��ri�e�each�other,��ossibl���throu��h�a�sequence���etector�lin�in���the��to��ether,�so�that�the�brain�res�onse�to�the�critical��or����oul���be�attenuate���b�����ra��atical�con-text.�This��rocess�of�syntactic priming at the neuronal level�(Pulver�üller�2002)��oul���be�absent�both�for��or��s�in�un��ra��atical�an���non-lin��uistic�context.�The�s��ntactic�MMN-relate���effects�are�thus�consistent��ith�the�sequence���etec-tor��o��el�for�s��ntax,�but�are�not�easil���ex�laine���as�the�si��n�of�an�“error�si��nal”��ro��uce���b���the�brain-internal���ra��ar.

The� �or�� revie�e��� here� has� been� instru�ental� in� �ro�otin��� the� vie�� of�earl��� lin��uistic� activit��� as� in��e�en��ent� of� attention� an��� therefore� auto�atic.�Discrete��e�or���circuits� for� existin����or��s�a��ear� to�be� full���activate��� irre-s�ective�of�attention�level��hen�the�res�ective��or��s�are�hear��.�Stron����utual�connections�bet�een�the�sub�arts�of�such��e�or���net�or�s��a���be�the�reason��h���the�entire�circuit�beco�es�quic�l���activate���to�its�full�ca�acit���even��hen�little�or�no�attention�is�allocate���to�lan��ua��e.�Whereas��o��ulation�of�these�con-nections�b���attention� is� still��ossible,� it���oes�not�a��ear� relevant� for� the�earl���net�or��activit��,��hich��a��� thus� in��ee���be�auto�atic�an���even�enca�sulate���fro��other�co��nitive��rocesses.�

In�co��utational� ter�s,�attention�effects��a���be�ex�laine���as� stren��th�of�cortical�fee��bac��connections:��hereas�lo��attention�resources��a���be�realise���as�stron���fee��bac��inhibition,�re��uction�in�this�inhibition�lea��s�to���reater�avail-abilit���of�attentional�resources�(Gara��nani et�al.�2008).�If�such�a���lobal�a��just-�ent�of�cortical�activit���levels,�a�re��ulation�s��ste��that�can�be�a�basis�for�atten-tional�resources,�exists,�a�clear��re��iction�can�be��a��e.�As�attention�is�allocate���to�or��ith��ra�n�fro��the�lin��uistic�in�ut,�sti�uli��ithout�existin���re�resenta-tions��oul����artiall���activate�a�lar��er�ran��e�or�onl���a�fe��lexical�re�resentations�belon��in���to�the�sa�e�cohort�(Marslen-Wilson�1987).�In�the�sa�e�ti�e,�earl���ac-tivation��rocesses�for�existin����or��s�can�be�ex�ecte���to�be�relativel���i��une�to�attentional��o��ulation,�as��or��s��oul���auto�aticall���activate�their�existin�����is-crete��e�or���circuits�irres�ective�of�attention�variation�(Gara��nani�et�al.�2008).�Thus,�the�a��vanta��e�of��or��s�over��seu��o�or��s��a���onl���be�observe����hen�the�sti�uli�are�not�atten��e���(Sht��rov�et�al.�2010);��hen�attention�is�require���this�ef-fect��a���be�cancelle���or�even�reverse��.�The�latter�is�certainl���the�case�in�N400�ex�eri�ents,��here�an�N400�a��vanta��e�for��seu��o�or��s�over��or��s�is�t���icall���foun���(Frie��rich�et�al.�2006).�An�ex�eri�ental�test�of�this�su����estion�in�the�con-text�of�a�sin��le�ex�eri�ent�coul����rovi��e�a�further�vali��ation�of�the��ro�ose����o��el�(Gara��nani�et�al.�2009).

Page 210: Unconscious Memory Representations in Perception

� Are���ou�listenin��?� 199

It� is�still,�ho�ever,�the�issue�of�attention�control�in�MMN�ex�eri�ents�that�the���are�frequentl���criticise���for.�Tra��itional��assive�MMN��ara��i���s,�in��hich�the�subjects’��ri�ar���tas��is�to��atch�a�vi��eofil�,��la���a�co��uter���a�e�or�rea���a�boo�,�in��ee���cannot���uarantee�that�the�subject�cannot�so�eti�es�‘snea��a�little�listen’�to�the�ex�eri�ental�sti�uli�(Carl��on�2004).�One�of�the��ossible���irections�for�future�research,�therefore,�is�to�further�ex�lore��ore�strin��ent��rotocols,�in-clu��in�����istracters�in�the�sa�e��o��alit��,�in�or��er�to�be�able�to�full���se�arate�atten-tion-in��e�en��ent,�an���thereb����ossibl���auto�atic,�lin��uistic��rocesses�fro��those�that�require�the�in��ivi��ual’s�active��artici�ation.�As�revie�e���above,��e�have�been�successful�in���oin���this�to�vali��ate�the�s��ntactic�MMN�as�bein���attention-in��e-�en��ent�at�its�earliest�sta��es.�Althou��h�this,�co�bine����ith�other�results,�stron��l���su����ests�that�the�earliest�lin��uistic�access�is�auto�atic,�this�a��roach�nee��s�to�be�exten��e���to���ifferent�t���es�of�lin��uistic�infor�ation��rocessin���(cf.�Gara��nani�et�al.�2009;�Sht��rov�et�al.�2010).

The�revie�e�����ata�b���no��eans�falsif���the�earlier�stu��ies��hich���elineate���the�later�neuro�h��siolo��ical�in��ices�of�lan��ua��e��rocessin��.�Instea��,�the���a�����a�ne����i�ension� to� the�neurolin��uistic� research� su����estin��� that� these� later��rocesses�reflecte���in�e.��.�N400,�P600�or�s��ntactic�ne��ativities�(LAN)��a���buil���u�on�the�earl���auto�atic�ones�an���are��ossibl���relate���to�secon��ar����ental��rocesses�that�can�be�tri����ere���b���the�earlier�access�to�lin��uistic��e�or���re�resentations,�but���e�en���on�attention�an���tas�-relate���strate��ies.

As� �an��� �s��cholin��uistic� an��� conce�tual� �rocesses� see�� to� be� brain-re-flecte���both�earl���an���late,�it�is�i��ortant�to�clarif���the�relation�bet�een�the�t�o�ti�e�ran��es.�Are�the�earl���se�antic,�s��ntactic�an���lexical�effects�just�the�be��in-nin��� of� late� effects?� Their� ��ifferential� ����na�ics� s�ea�� a��ainst� this� �ossibilit��.�Woul���the�late��rocesses�just�re�eat�the�earl���ones,�or�occur�onl���if�the�earl���ones�are�unsuccessful?�Here,�our�o�n���ata�in��icate�that�the�earl���near-si�ultaneous��rocesses�exhibit�sur�risin���s�ecificit���to�infor�ation�t���es,�both�to�o��ra�hi-call���an���in�ter�s�of�cortical���enerators,��hereas�the�late�ones,�for�exa��le�the�N400,�see��equall����o��ulate���b�����ifferent�lin��uistic�features�(inclu��in����or���frequenc��,�lexicalit��,�an���se�antic��ro�erties,�Hau��et�al.�2006).�Still,�are�the�late�co��onents�reflections�of��rolon��e���s�ecificall���lin��uistic��rocesses,�in-��e�th-�or� re-�rocessin��,� or� �oul��� the��� rather� reflect� �ost-co��rehension� �rocesses�(Glenber���an���Kascha��2002)�follo�in���co��lete����s��cholin��uistic�infor�ation�access� an��� context� inte��ration?� An��� ho�� fixe��� are� the� la��s� bet�een� ��ifferent�activations?�It�has�been�recentl���sho�n�that�sti�ulus�context�can��o��ulate�the�ti�e� la���of�brain� res�onses� in��exin����or���an���object��rocessin��� (Barber�an���Kutas�2007).�Such�context���e�en��ence�an���flexibilit���is�of���reatest�relevance�in�the�stu�����of�co��nitive��rocesses�an����oints�the��a���to�fruitful�future�research.�The�relationshi��bet�een�s�ecific�activation�ti�es�of���efine���brain�areas�on�the�

Page 211: Unconscious Memory Representations in Perception

200� Yur���Sht��rov�an���Frie��e�ann�Pulver�üller

one�han���an���s�ecific�co��nitive��rocesses�on�the�other�is�one�of�the��ost�excitin���to�ics� in�co��nitive�neuroscience�at��resent.�A����ressin���this� issue�usin���MEG/EEG�an���source�localization�has�onl���beco�e��ossible�ver���recentl��.�If�success-ful,�the�ne��available��etho��s��ill��a�e�it��ossible�to�rea���the�activation�si��na-tures�of�cortical�circuits��rocessin���lan��ua��e�an���conce�ts�in�the�hu�an�brain�in��ore�an����ore���etail,�lea��in���the�science�of�lan��ua��e�to�a�ne��sta��e.

Acknowledgements

Su��orte��� b��� the� Me��ical� Research� Council,� UK� (U.1055.04.014.00001.01,�U.1055.04.003.00001.01).

References

Alho,�K.�(1992).�Selective�attention�in�au��itor����rocessin���as�reflecte���b���event-relate���brain��otentials.�Psychophysiology,�29,�247–263.

Alho,�K.�(1995).�Cerebral���enerators�of��is�atch�ne��ativit���(MMN)�an���its��a��netic�counter-�art�(MMN�)�elicite���b���soun���chan��es.�Ear and Hearing,�16,�38–51.

Barber,�H.�A.�an���Kutas,�M.� (2007).� Inter�la���bet�een�co��utational��o��els�an���co��nitive�electro�h��siolo�����in�visual��or���reco��nition.�Brain Research Review,�53,�98–123.

Bentin,�S.,�Mouchetant-Rostain��,�Y.,�Giar��,�M.�H.,�Echallier,�J.�F.�an���Pernier,�J.�(1999).�ERP��anifestations� of� �rocessin��� �rinte��� �or��s� at� ��ifferent� �s��cholin��uistic� levels:� Ti�e�course�an���scal����istribution.�Journal of Cognitive Neuroscience,�11,�235–260.

Bonte,�M.�L.,�Mitterer,�H.,�Zella��ui,�N.,�Poel�ans,�H.�an���Blo�ert,�L.�(2005).�Au��itor���cortical�tunin���to�statistical�re��ularities�in��honolo����.�Clinical Neurophysiology,�116,�2765–2774.

Bonte,�M.�L.,�Poel�ans,�H.�an���Blo�ert,�L.� (2007).�Deviant�neuro�h��siolo��ical� res�onses� to��honolo��ical�re��ularities�in�s�eech�in�����slexic�chil��ren.�Neuropsychologia,�45,�1427–1437.

Born�essel,�I.,�Z��sset,�S.,�Frie��erici,�A.�D.,�von�Cra�on,�D.�Y.�an���Schlese�s���,�M.�(2005).�Who���i����hat�to��ho�?�The�neural�basis�of�ar��u�ent�hierarchies���urin���lan��ua��e�co��rehen-sion.�Neuroimage,�26,�221–233.

Braitenber��,�V.�an���Schüz,�A.�(1992).�Basic�features�of�cortical�connectivit���an���so�e�consi��er-ations�on�lan��ua��e.�In�J.�Win��,�B.�Chiarelli,�B.�H.�Bicha�jian,�A.�Nocentini�an���A.�Jon�er�(E��s.),�Language origin: A multidisciplinary approach�(��.�89–102).�Dor��recht:�Klu�er.

Buccino,�G.,�Bin�ofs�i,�F.,�Fin�,�G.�R.,�Fa��i��a,�L.,�Fo��assi,�L.,�Gallese,�V.,�Seitz,�R.�J.,�Zilles,�K.,�Rizzolatti,�G.�an���Freun��,�H.�J.�(2001).�Action�observation�activates��re�otor�an����ari-etal�areas�in�a�so�atoto�ic��anner:�An�fMRI�stu����.�European Journal of Neuroscience,�13,�400–404.

Carl��on,�R.�P.�(2004).�Ho��the�brain�se�arates�soun��s.�Trends in Cognitive Sciences,�8,�465–471.

Cheour,� M.,� Alho,� K.,� Ce�oniene,� R.,� Reini�ainen,� K.,� Sainio,� K.,� Pohjavuori,� M.,� Aaltonen,�O.�an���Näätänen,�R.�(1998).�Maturation�of��is�atch�ne��ativit���in�infants.�International Journal of Psychophysiology,�29,�217–226.

Page 212: Unconscious Memory Representations in Perception

� Are���ou�listenin��?� 201

Ch�illa,�D.�J.�an���Kol�,�H.�H.�(2005).�Accessin����orl����no�le����e:�Evi��ence�fro��N400�an���reaction�ti�e��ri�in��.�Brain Research – Cognitive Brain Research,�25,�589–606.

Colin,�C.,�Ra��eau,�M.,�Soquet,�A.�an���Deltenre,�P.�(2004).�Generalization�of�the���eneration�of�an� MMN� b��� illusor��� McGur�� �erce�ts:� Voiceless� consonants.� Clinical Neurophysiology,�115,�1989–2000.

Colin,�C.,�Ra��eau,�M.,�Soquet,�A.,�De�olin,�D.,�Colin,�F.�an���Deltenre,�P.�(2002).�Mis�atch�ne��ativit��� evo�e��� b��� the� McGur�-MacDonal��� effect:� A� �honetic� re�resentation� �ithin�short-ter���e�or��.�Clinical Neurophysiology,�113,�495–506.

Da�asio,�A.�R.�(1990).�Cate��or��-relate���reco��nition���efects�as�a�clue�to�the�neural�substrates�of��no�le����e.�Trends in Neurosciences,�13,�95–98.

Dehaene-La�bertz,�G.�(1997).�Electro�h��siolo��ical�correlates�of�cate��orical��hone�e��erce�-tion�in�a��ults.�Neuroreport,�8,�919–924.

Dehaene-La�bertz,� G.,� Du�oux,� E.� an��� Gout,� A.� (2000).� Electro�h��siolo��ical� correlates� of��honolo��ical��rocessin��:�A�cross-lin��uistic� stu����.� Journal of Cognitive Neuroscience,�12,�635–647.

Dell,�G.�S.�(1986).�A�s�rea��in��-activation�theor���of�retreival�in�sentence��ro��uction.�Psycho-logical Review,�93,�283–321.

Diesch,�E.,�Bier�ann,�S.�an���Luce,�T.�(1998).�The��a��netic��is�atch�fiel���elicite���b����or��s�an����honolo��ical�non-�or��s.�Neuroreport,�9,�455–460.

E�bic�,�D.,�Hac�l,�M.,�Schaeffer,�J.,�Kele�ir,�M.�an���Marantz,�A.�(2001).�A��a��netoence�ha-lo��ra�hic�co��onent��hose�latenc���reflects�lexical�frequenc��.�Brain Research – Cognitive Brain Research,�10,�345–348.

En��rass,�T.,�Mohr,�B.�an���Pulver�üller,�F.�(2004).�Enhance����is�atch�ne��ativit���brain�res�onse�after�binaural��or����resentation.�European Journal of Neuroscience,�19,�1653–1660.

Escera,�C.,�Alho,�K.,�Win�ler,�I.�an���Näätänen,�R.�(1998).�Neural��echanis�s�of�involuntar���attention�to�acoustic�novelt���an���chan��e.�Journal of Cognitive Neuroscience,�10,�590–604.

Fo��or,�J.�A.�(1983).�The modulatity of mind.�Ca�bri����e,�MA:�MIT�Press.Fre��e,�G.�(1980).�Über�Sinn�un���Be��eutun���(first��ublishe���in�1892).�In�G.�Patzi���(E��.),�Funk-

tion, Begriff, Bedeutung�(��.�25–50).�Göttin��en:�Huber.Frie��erici,�A.�(2002).�To�ar��s�a�neural�basis�of�au��itor���sentence��rocessin��.�Trends in Cogni-

tive Sciences,�6,�78–84.Frie��erici,�A.�D.,�Pfeifer,�E.�an���Hahne,�A.�(1993).�Event-relate���brain��otentials���urin���natural�

s�eech��rocessin��:�Effects�of�se�antic,��or�holo��ical�an���s��ntactic�violations.�Cognitive Brain Research,�1,�183–192.

Frie��erici,�A.�D.,�Wan��,�Y.,�Herr�ann,�C.�S.,�Maess,�B.�an���Oertel,�U.�(2000).�Localization�of�earl��� s��ntactic� �rocesses� in� frontal� an��� te��oral� cortical� areas:� A� �a��netoence�halo-��ra�hic�stu����.�Human Brain Mapping,�11,�1–11.

Frie��rich,�C.�K.,�Eulitz,�C.�an���Lahiri,�A.�(2006).�Not�ever����seu��o�or�����isru�ts��or���reco��ni-tion:�An�ERP�stu����.�Behavior and Brain Functions,�2,�36.

Fro��in,�V.�A.�(1973).�Intro��uction.�In�V.�A.�Fro��in�(E��.),�Speech errors as linguistic evidence�(��.�11–45).�The�Ha��ue:�Mouton.

Gara��nani,�M.,�Sht��rov,�Y.�an���Pulver�üller,�F.�(2009).�Effects�of�attention�on��hat�is��no�n�an����hat�is�not:�MEG�evi��ence�for�functionall�����iscrete��e�or���circuits.�Frontiers in Hu-man Neuroscience,�3,�Article�10.

Gara��nani,� M.,� Wenne�ers,� T.� an��� Pulver�uller,� F.� (2008).� A� neuroanato�icall��� ��roun��e���Hebbian-learnin����o��el�of�attention-lan��ua��e�interactions�in�the�hu�an�brain.�European Journal of Neuroscience,�27,�492–513.

Page 213: Unconscious Memory Representations in Perception

202� Yur���Sht��rov�an���Frie��e�ann�Pulver�üller

Garrett,�M.�(1980).�Levels�of��rocessin���in�sentence��ro��uction.�In�B.�Butter�orth�(E��.),�Lan-guage Production I�(��.�177–220).�Lon��on:�Aca��e�ic�Press.

Garrett,� M.� (1984).� The� or��anization� of� �rocessin��� structures� for� lan��ua��e� �ro��uction.� In�D.�Ca�lan,�A.�R.�Lecours�an���A.�S�ith�(E��s.),�Biological perspectives on language�(��.�172–193).�Ca�bri����e,�MA:�MIT�Press.

Garro��,�S.�an���Pic�erin��,�M.�J.�(2004).�Wh���is�conversation�so�eas��?�Trends in Cognitive Sci-ences,�8,�8–11.

Glenber��,�A.�M.�an���Kascha�,�M.�P.�(2002).�Groun��in���lan��ua��e�in�action.�Psychonomic Bul-letin and Reviews,�9,�558–565.

Gunter,�T.�C.,�Frie��erici,�A.�D.�an���Schriefers,�H.�(2000).�S��ntactic���en��er�an���se�antic�ex�ec-tanc��:�ERPs�reveal�earl���autono����an���late�interaction.�Journal of Cognitve Neuroscience,�12,�556–568.

Hahne,�A.�an���Frie��erici,�A.�D.�(1999).�Electro�h��siolo��ical�evi��ence�for�t�o�ste�s�in�s��ntactic�anal��sis.�Earl���auto�atic�an���late�controlle����rocesses.�Journal of Cognitive Neuroscience,�11,�194–205.

Hastin��,�A.�S.�an���Kotz,�S.�A.�(2008).�S�ee��in���u��s��ntax:�On�the�relative�ti�in���an���auto�atic-it���of�local��hrase�structure�an����or�hos��ntactic��rocessin���as�reflecte���in�event-relate���brain��otentials.�Journal of Cognitive Neuroscience,�20,�1207–1219.

Hastin��,�A.�S.,�Kotz,�S.�A.�an���Frie��erici,�A.�D.�(2007).�Settin���the�sta��e�for�auto�atic�s��ntax��rocessin��:�The��is�atch�ne��ativit���as�an�in��icator�of�s��ntactic��ri�in��.�Journal of Cogni-tive Neuroscience,�19,�386–400.

Hastin��,�A.�S.,�Win�ler,� I.� an���Kotz,�S.�A.� (2008).�Earl�����ifferential��rocessin���of�verbs�an���nouns�in�the�hu�an�brain�as�in��exe���b���event-relate���brain��otentials.�European Journal of Neuroscience,�27,�1561–1565.

Hau�,�O.,�Davis,�M.�H.,�For��,�M.,�Pulver�uller,�F.�an���Marslen-Wilson,�W.�D.�(2006).�The�ti�e�course�of�visual��or���reco��nition�as�reveale���b��� linear�re��ression�anal��sis�of�ERP���ata.�Neuroimage,�30,�1383–1400.

Hau�,�O.,�Johnsru��e,�I.�an���Pulver�üller,�F.�(2004).�So�atoto�ic�re�resentation�of�action��or��s�in�hu�an��otor�an����re�otor�cortex.�Neuron,�41,�301–307.

Hau�,�O.�an���Pulver�üller,�F.�(2004).�Neuro�h��siolo��ical���istinction�of�action��or��s� in�the�fronto-central�cortex.�Human Brain Mapping,�21,�191–201.

Hebb,�D.�O.�(1949).�The organization of behavior. A neuropsychological theory.�Ne��Yor�:�John�Wile��.

Honbol����o,�F.,�Cse�e,�V.�an���Ra��o,�A.�(2004).�Su�rase���ental�s�eech�cues�are�auto�aticall����rocesse��� b��� the� hu�an� brain:� A� �is�atch� ne��ativit��� stu����.�Neuroscience Letters,� 363,�84–88.

Jacobsen,�T.,�Horvath,�J.,�Schro��er,�E.,�Lattner,�S.,�Wi���ann,�A.�an���Win�ler,�I.�(2004).�Pre-at-tentive�au��itor����rocessin���of�lexicalit��.�Brain and Language,�88,�54–67.

Kaan,�E.�an���S�aab,�T.�Y.� (2002).�The�brain�circuitr���of� s��ntactic�co��rehension.�Trends in Cognitive Sciences,�6,�350–356.

Kircher,�T.�T.,�Oh,�T.�M.,�Bra��er,�M.�J.�an���McGuire,�P.�K.�(2005).�Neural�correlates�of�s��ntax��ro��uction�in�schizo�hrenia.�British Journal of Psychiatry,�186,�209–214.

Knoblauch,�A.�an���Pulver�üller,�F.�(2005).�Sequence���etector�net�or�s�an���associative�learn-in���of���ra��atical�cate��ories.�In�S.�Wer�ter,�G.�Pal��an���M.�Elsha��(E��s.),�Biomimetic neural learning for intelligent robots.�Hei��elber��:�S�rin��er.

Page 214: Unconscious Memory Representations in Perception

� Are���ou�listenin��?� 203

Kor�ilahti,�P.,�Krause,�C.�M.,�Holo�ainen,�I.�an���Lan��,�A.�H.�(2001).�Earl���an���late��is�atch�ne��ativit���elicite���b����or��s�an���s�eech-li�e�sti�uli�in�chil��ren.�Brain and Language,�76,�332–339.

Korth,�M.�an���N��u��en,�N.�X.�(1997).�The�effect�of�sti�ulus�size�on�hu�an�cortical��otentials�evo�e���b���chro�atic��atterns.�Vision Research,�37,�649–657.

Kru�bholz,�K.,�Patterson,�R.�D.,�Seither-Preisler,�A.,�La��ert�ann,�C.�an���Lut�enhoner,�B.�(2003).�Neuro�a��netic�evi��ence�for�a��itch��rocessin���center�in�Heschl’s�����rus.�Cerebral Cortex,�13,�765–772.

Kujala,�A.,�Alho,�K.,�Valle,�S.,�Sivonen,�P.,�Il�onie�i,�R.�J.,�Al�u,�P.�an���Näätänen,�R.�(2002).�Context� �o��ulates� �rocessin��� of� s�eech� soun��s� in� the� ri��ht� au��itor��� cortex� of� hu�an�subjects.�Neuroscience Letters,�331,�91–94.

Kutas,�M.�an���Hill��ar��,�S.�A.�(1980).�Rea��in���senseless�sentences:�Brain��otentials�reflect�se-�antic�incon��ruit��.�Science,�207,�203–205.

Levelt,�W.�J.�M.,�Roelofs,�A.�an���Me��er,�A.�S.�(1999).�A�theor���of�lexical�access�in�s�eech��ro��uc-tion.�Behavioral and Brain Sciences,�22,�1–75.

Lut�enhoner,�B.,�Kru�bholz,�K.,�La��ert�ann,�C.,�Seither-Preisler,�A.,�Steinstrater,�O.�an���Patterson,�R.�D.�(2003).�Localization�of��ri�ar���au��itor���cortex�in�hu�ans�b����a��neto-ence�halo��ra�h��.�Neuroimage,�18,�58–66.

MacKa��,�D.�G.�(1987).�The organization of perception and action. A theory of language and other cognitive skills.�Ne��Yor�:�S�rin��er.

Marslen-Wilson,�W.�(1973).�Lin��uistic�structure�an���s�eech�sha��o�in���at�ver���short�latencies.�Nature,�244,�522–523.

Marslen-Wilson,�W.�an���T��ler,�L.�K.�(1975).�Processin���structure�of�sentence��erce�tion.�Na-ture,�257,�784–786.

Marslen-Wilson,� W.� D.� (1985).� S�eech� sha��o�in��� an��� s�eech� co��rehension.� Speech and Communication,�4,�55–74.

Marslen-Wilson,�W.�D.�(1987).�Functional��arallelis��in�s�o�en��or��-reco��nition.�Cognition,�25,�71–102.

Mennin��,� H.,� Z�itserloo��,� P.,� Schonin��,� S.,� Hihn,� H.,� Bolte,� J.,� Dobel,� C.,� Mathia�,� K.� an����Lut�enhoner,�B.�(2005).�Pre-attentive���etection�of�s��ntactic�an���se�antic�errors.�Neuro-report,�16,�77–80.

Mesula�,�M.�M.�(1990).�Lar��e-scale�neuroco��nitive�net�or�s�an�����istribute����rocessin���for�attention,�lan��ua��e,�an����e�or��.�Annals of Neurology,�28,�597–613.

Mohr,�B.�an���Pulver�üller,�F.�(2002).�Re��un��anc�����ains�an���costs�in�co��nitive��rocessin��:�Ef-fects�of�short�sti�ulus�onset�as��nchronies.�Journal of Experimental Psychology: Learning, Memory and Cognition,�28,�1200–1223.

Morton,�J.�(1969).�The�interaction�of�infor�ation�in��or���reco��nition.�Psychological Review,�76,�165–178.

Moss,�H.�E.,�McCor�ic�,�S.�F.�an���T��ler,�L.�(1997).�The�ti�e�course�of�activation�of�se�an-tic� infor�ation���urin��� s�o�en��or���reco��nition.�Language and Cognitive Processes,� 12,�695–731.

Münte,�T.�F.,�Schiltz,�K.�an���Kutas,�M.�(1998).�When�te��oral�ter�s�belie�conce�tual�or��er.�Nature,�395,�71–73.

Näätänen,�R.�(1995).�The��is�atch�ne��ativit��:�A��o�erful�tool�for�co��nitive�neuroscience.�Ear and Hearing,�16,�6–18.

Näätänen,�R.�(2000).�Mis�atch�ne��ativit���(MMN):�Pers�ectives�for�a��lication.�International Journal of Psychophysiology,�37,�3–10.

Page 215: Unconscious Memory Representations in Perception

204� Yur���Sht��rov�an���Frie��e�ann�Pulver�üller

Näätänen,�R.�(2001).�The��erce�tion�of�s�eech�soun��s�b���the�hu�an�brain�as�reflecte���b���the��is�atch�ne��ativit���(MMN)�an���its��a��netic�equivalent�(MMN�).�Psychophysiology,�38,�1–21.

Näätänen,�R.�an���Alho,�K.�(1995).�Mis�atch�ne��ativit���–�a�unique��easure�of�sensor����rocess-in���in�au��ition.�International Journal of Neuroscience,�80,�317–337.

Näätänen,�R.,�Lehto�os�i,�A.,�Lennes,�M.,�Cheour,�M.,�Huotilainen,�M.,�Iivonen,�A.,�Vainio,�M.,�Al�u,�P.,�Il�onie�i,�R.�J.,�Luu�,�A.,�Alli�,�J.,�Sin��onen,�J.�an���Alho,�K.�(1997).�Lan��ua��e-s�ecific��hone�e�re�resentations�reveale���b���electric�an����a��netic�brain�res�onses.�Na-ture,�385,�432–434.

Näätänen,�R.�an���Picton,�T.�(1987).�The�N1��ave�of�the�hu�an�electric�an����a��netic�res�onse�to� soun��:� A� revie�� an��� an� anal��sis� of� the� co��onent� structure.� Psychophysiology,� 24,�375–425.

Näätänen,�R.,�Tervanie�i,�M.,�Suss�an,�E.,�Paavilainen,�P.�an���Win�ler,�I.�(2001).� ‘Pri�itive�intelli��ence’�in�the�au��itor���cortex.�Trends in Neurosciences,�24,�283–288.

Neville,�H.,�Nicol,�J.�L.,�Barss,�A.,�Forster,�K.�I.�an���Garrett,�M.�F.�(1991).�S��ntacticall���base���sentence��rocessin���classes:�Evi��ence�fro��event-relate���brain��otentials.�Journal of Cog-nitive Neuroscience,�3,�151–165.

Norris,�D.,�McQueen,�J.�M.�an���Cutler,�A.�(2000).�Mer��in���infor�ation�in�s�eech�reco��nition:�Fee��bac��is�never�necessar��.�Behavioral and Brain Sciences,�23,�299–370.

Obleser,�J.,�Lahiri,�A.�an���Eulitz,�C.�(2004).�Ma��netic�brain�res�onse��irrors�extraction�of��ho-nolo��ical�features�fro��s�o�en�vo�els.�Journal of Cognitive Neuroscience,�16,�31–39.

Osterhout,� L.� (1997).� On� the� brain� res�onse� to� s��ntactic� ano�alies:� Mani�ulations� of� �or����osition�an����or���class�reveal�in��ivi��ual���ifferences.�Brain and Language,�59,�494–522.

Osterhout,�L.�an���Ha��oort,�P.�(1999).�A�su�erficial�rese�blance���oes�not�necessaril����ean���ou�are��art�of�the�fa�il��:�Counterar��u�ents�to�Coulson,�Kin���an���Kutas�(1998)�in�the�P600/SPS-P300���ebate.�Language and Cognitive Processes,�14,�1–14.

Osterhout,� L.� an��� Holco�b,� P.� J.� (1992).� Event-relate��� brain� �otentials� elicite��� b��� s��ntactic�ano�al��.�Journal of Memory and Language,�31,�785–806.

Osterhout,�L.,�McLau��hlin,�J.�an���Bersic�,�M.�(1997).�Event-relate���brain��otentials�an���hu�an�lan��ua��e.�Trends in Cognitive Sciences,�1,�203–209.

Osterhout,�L.�an���S�inne��,�D.�A.�(1993).�On�the�te��oral�course�of���a�-fillin�����urin���co��re-hension�of�verbal��assives.�Journal of Psycholinguistic Research,�22,�273–286.

Paavilainen,�P.,�Alho,�K.,�Reini�ainen,�K.,�Sa�s,�M.�an���Näätänen,�R.�(1991).�Ri��ht�he�is�here���o�inance�of���ifferent��is�atch�ne��ativities.�Electroencephalography and Clinical Neuro-physiology,�78,�466–479.

Peltola,�M.�S.�an���Aaltonen,�O.�(2005).�Lon��-ter���e�or���trace�activation�for�vo�els���e�en��s�on� the� �other� ton��ue� an��� the� lin��uistic� context.� Journal of Psychophysiology,� 19,� 159–164.

Penfiel��,�W.�an���Ras�ussen,�T.�(1950).�The cerebral cortex of man.�Ne��Yor�:�Mac�illan.Penolazzi,� B.,� Hau�,� O.� an��� Pulver�üller,� F.� (2007).� Earl��� se�antic� context� inte��ration� an���

lexical�access�as�reveale���b���event-relate���brain��otentials.�Biological Psychology,�74,�374–388.

Petti��re�,�C.�M.,�Mur��och,�B.�E.,�Ponton,�C.�W.,�Finni��an,�S.,�Al�u,�P.,�Kei,�J.,�Soc�alin��a�,�R.�an���Chener��,�H.�J.�(2004).�Auto�atic�au��itor����rocessin���of�En��lish��or��s�as�in��exe���b���the��is�atch�ne��ativit��,�usin���a��ulti�le���eviant��ara��i���.�Ear and Hearing,�25,�284–301.

Page 216: Unconscious Memory Representations in Perception

� Are���ou�listenin��?� 205

Pic�erin��,�M.�J.�an���Garro��,�S.�(2004).�To�ar���a��echanistic��s��cholo�����of���ialo��ue.�Behav-ioral Brain Sciences,�27,�169–190.

Picton,�T.�an���Hill��ar��,�S.�(1974).�Hu�an�au��itor���evo�e����otentials:�II.�Effects�of�attention.�Electroencephalography and Clinical Neurophysiology,�36,�191–200.

Picton,�T.�W.,�Alain,�C.,�Otten,�L.,�Ritter,�W.�an���Achi�,�A.�(2000).�Mis�atch�ne��ativit��:�Differ-ent��ater�in�the�sa�e�river.�Audiology and Neurootology,�5,�111–139.

Poe��el,�D.,�Yellin,�E.,�Philli�s,�C.,�Roberts,�T.�P.,�Ro�le��,�H.�A.,�Wexler,�K.�an���Marantz,�A.�(1996).�Tas�-in��uce���as����etr���of�the�au��itor���evo�e���M100�neuro�a��netic�fiel���elic-ite���b���s�eech�soun��s.�Brain Research – Cognitive Brain Research,�4,�231–242.

Pulver�üller,�F.�(1999).�Wor��s�in�the�brain’s�lan��ua��e.�Behavioral and Brain Sciences,�22,�253–336.

Pulver�üller,�F.�(2001).�Brain�reflections�of��or��s�an���their��eanin��.�Trends in Cognitive Sci-ences,�5,�517–524.

Pulver�üller,�F.�(2002).�A�brain��ers�ective�on�lan��ua��e��echanis�s:�Fro����iscrete�neuronal�ense�bles�to�serial�or��er.�Progress in Neurobiology,�67,�85–111.

Pulver�üller,�F.�(2005).�Brain��echanis�s�lin�in���lan��ua��e�an���action.�Nature Review Neuro-sciences,�6,�576–582.

Pulver�üller,�F.� an���Assa��ollahi,�R.� (2007).�Gra��ar�or� serial�or��er?:�Discrete�co�binato-rial�brain��echanis�s�reflecte���b���the�s��ntactic��is�atch�ne��ativit��.�Journal of Cognitive Neuroscience,�19,�971–980.

Pulver�üller,�F.,�Kujala,�T.,�Sht��rov,�Y.,�Si�ola,�J.,�Tiitinen,�H.,�Al�u,�P.,�Alho,�K.,�Martin�au��i,�S.,�Il�onie�i,�R.�J.�an���Näätänen,�R.�(2001).�Me�or���traces�for��or��s�as�reveale���b���the��is�atch�ne��ativit��.�Neuroimage,�14,�607–616.

Pulver�üller,�F.�an���Preissl,�H.�(1991).�A�cell�asse�bl����o��el�of�lan��ua��e.�Network: Computa-tion in Neural Systems,�2,�455–468.

Pulver�üller,�F.�an���Sht��rov,�Y.�(2003).�Auto�atic��rocessin���of���ra��ar�in�the�hu�an�brain�as�reveale���b���the��is�atch�ne��ativit��.�Neuroimage,�20,�159–172.

Pulver�üller,�F.�an���Sht��rov,�Y.�(2006).�Lan��ua��e�outsi��e�the�focus�of�attention:�The��is�atch�ne��ativit���as�a�tool�for�stu����in���hi��her�co��nitive��rocesses.�Progress in Neurobiology,�79,�49–71.

Pulver�üller,�F.�an���Sht��rov,�Y.�(2009).�S�atiote��oral�si��natures�of�lar��e-scale�s��nfire�chains�for�s�eech��rocessin���as�reveale���b���MEG.�Cerebral Cortex,�19,�79–88.

Pulver�üller,�F.,�Sht��rov,�Y.,�Hastin��,�A.�S.�an���Carl��on,�R.�P.�(2008).�S��ntax�as�a�reflex:�Neuro-�h��siolo��ical�evi��ence�for�earl���auto�aticit���of���ra��atical��rocessin��.�Brain and Lan-guage,�104,�244–253.

Pulver�üller,�F.,�Sht��rov,�Y.�an���Il�onie�i,�R.�(2003).�S�atiote��oral�����na�ics�of�neural�lan-��ua��e��rocessin��:�An�MEG�stu�����usin����ini�u�-nor��current�esti�ates.�Neuroimage,�20,�1020–1025.

Pulver�üller,�F.,�Sht��rov,�Y.�an���Il�onie�i,�R.�(2005).�Brain�si��natures�of��eanin���access�in�action��or���reco��nition.�Journal of Cognitive Neuroscience,�17,�884–892.

Pulver�üller,� F.,� Sht��rov,� Y.,� Il�onie�i,� R.� J.� an��� Marslen-Wilson,� W.� D.� (2006).� Trac�in���s�eech�co��rehension�in�s�ace�an���ti�e.�Neuroimage,�31,�1297–1305.

Pulver�üller,�F.,�Sht��rov,�Y.,�Kujala,�T.�an���Näätänen,�R.�(2004).�Wor��-s�ecific�cortical�activit���as�reveale���b���the��is�atch�ne��ativit��.�Psychophysiology,�41,�106–112.

P��l��anen,�L.,�Strin��fello�,�A.�an���Marantz,�A.�(2002).�Neuro�a��netic�evi��ence�for�the�ti�in���of�lexical�activation:�An�MEG�co��onent�sensitive�to��honotactic��robabilit���but�not�to�nei��hborhoo�����ensit��.�Brain and Language,�81,�666–678.

Page 217: Unconscious Memory Representations in Perception

20�� Yur���Sht��rov�an���Frie��e�ann�Pulver�üller

Rastle,�K.,�Davis,�M.�H.,�Marslen-Wilson,�W.�D.�an���T��ler,�L.�K.� (2000).�Mor�holo��ical�an���se�antic�effects�in�visual��or���reco��nition:�A�ti�e-course�stu����.�Language and Cognitive Processes,�15,�507–537.

Schrö��er,�E.�(1996).�On���etection�of�au��itor�����iviations:�A��re-attentive�activation��o��el.�Psy-chophysiology,�34,�245–257.

Scott,�S.�K.�an���Johnsru��e,�I.�S.�(2003).�The�neuroanato�ical�an���functional�or��anization�of�s�eech��erce�tion.�Trends in Neurosciences,�26,�100–107.

Sereno,�S.�C.,�Bre�er,�C.�C.�an���O’Donnell,�P.�J.�(2003).�Context�effects�in��or���reco��nition:�Evi��ence�for�earl���interactive��rocessin��.�Psychological Science,�14,�328–333.

Sereno,�S.�C.�an���Ra��ner,�K.�(2003).�Measurin����or���reco��nition�in�rea��in��:�E��e��ove�ents�an���event-relate����otentials.�Trends in Cognitive Sciences,�7,�489–493.

Sht��rov,�Y.,�Hau�,�O.�an���Pulver�üller,�F.�(2004).�Distribute���neuronal�net�or�s�for�enco��in���cate��or��-s�ecific�se�antic�infor�ation:�The��is�atch�ne��ativit���to�action��or��s.�Euro-pean Journal of Neuroscience,�19,�1083–1092.

Sht��rov,�Y.,�Kujala,�T.,�Ahveninen,�J.,�Tervanie�i,�M.,�Al�u,�P.,�Il�onie�i,�R.�J.�an���Näätänen,�R.�(1998).�Bac���roun���acoustic�noise�an���the�he�is�heric� lateralization�of�s�eech��ro-cessin��� in� the� hu�an� brain:� Ma��netic� �is�atch� ne��ativit��� stu����.� Neuroscience Letters,�251,�141–144.

Sht��rov,�Y.,�Kujala,�T.,�Palva,�S.,�Il�onie�i,�R.�J.�an���Näätänen,�R.�(2000).�Discri�ination�of�s�eech�an���of�co��lex�nons�eech�soun��s�of���ifferent�te��oral�structure�in�the�left�an���ri��ht�cerebral�he�is�heres.�Neuroimage,�12,�657–663.

Sht��rov,�Y.,�Kujala,�T.�an���Pulver�üller,�F.� (2010).� Interactions�bet�een� lan��ua��e�an���atten-tion�s��ste�s:�Earl���auto�atic�lexical��rocessin��?�Journal of Cognitive Neuroscience,�22,�in��ress.

Sht��rov,�Y.,�Oss�al��,�K.�an���Pulver�üller,�F.�(2008).�Me�or���traces�for�s�o�en��or��s�in�the�brain�as�reveale���b���the�he�o����na�ic�correlate�of�the��is�atch�ne��ativit��.�Cerebral Cor-tex,�18,�29–37.

Sht��rov,�Y.,�Pih�o,�E.�an���Pulver�üller,�F.�(2005).�Deter�inants�of���o�inance:�Is�lan��ua��e�lat-eralit���ex�laine���b����h��sical�or�lin��uistic�features�of�s�eech?�Neuroimage,�27,�37–47.

Sht��rov,� Y.� an��� Pulver�üller,� F.� (2002).� Neuro�h��siolo��ical� evi��ence� of� �e�or��� traces� for��or��s�in�the�hu�an�brain.�Neuroreport,�13,�521–525.

Sht��rov,�Y.�an���Pulver�üller,�F.�(2007).�Earl���MEG�activation�����na�ics�in�the�left�te��oral�an���inferior�frontal�cortex�reflect�se�antic�context�inte��ration.�Journal of Cognitive Neurosci-ence,�19,�1633–1642.

Sht��rov,�Y.,�Pulver�üller,�F.,�Näätänen,�R.�an���Il�onie�i,�R.�J.�(2003).�Gra��ar��rocessin���outsi��e�the�focus�of�attention:�An�MEG�stu����.�Journal of Cognitive Neuroscience,�15,�1195–1206.

Sitti�ra�a�orn,�W.,�Chin��a��uan��ratn,�C.,�Tervanie�i,�M.�an���Khotchabha���i,�N.�(2003).�Pre-attentive��rocessin���of�lexical�tone��erce�tion�b���the�hu�an�brain�as�in��exe���b���the��is-�atch�ne��ativit����ara��i���.�Annals of the New York Academy of Sciences,�999,�199–203.

Stoc�all,�L.,�Strin��fello�,�A.�an���Marantz,�A.�(2004).�The��recise�ti�e�course�of�lexical�activa-tion:�MEG��easure�ents�of� the�effects�of� frequenc��,��robabilit��,� an�����ensit��� in� lexical���ecision.�Brain and Language,�90,�88–94.

Tiitinen,�H.,�Ma��,�P.�an���Näätänen,�R.�(1997).�The�transient�40-Hz�res�onse,��is�atch�ne��ativ-it��,�an���attentional��rocesses�in�hu�ans.�Progress in Neuropsychopharmacology and Bio-logical Psychiatry,�21,�751–771.

Page 218: Unconscious Memory Representations in Perception

� Are���ou�listenin��?� 207

Tiitinen,�H.,�Ma��,�P.,�Reini�ainen,�K.�an���Näätänen,�R.�(1994).�Attentive�novelt�����etection�in�hu�ans�is���overne���b����re-attentive�sensor����e�or��.�Nature,�372,�90–92.

T��ler,�L.,�Moss,�H.�E.,�Gal�in,�A.�an���Voice,�J.�K.�(2002).�Activatin����eanin���in�ti�e:�The�role�of�i�a��eabilit���an���for�-class.�Language and Cognitive Processes,�17,�471–502.

U��en�a��,�S.,�Johnsru��e,�I.�S.,�Norris,�D.,�Marslen-Wilson,�W.�an���Patterson,�R.�D.�(2006).�Locatin���the�initial�sta��es�of�s�eech-soun����rocessin���in�hu�an�te��oral�cortex.�Neuro-image,�31,�1284–1296.

van�Ber�u�,� J.� J.,�Z�itserloo��,�P.,�Ha��oort,�P.�an���Bro�n,�C.�M.� (2003).�When�an���ho����o�listeners�relate�a�sentence�to�the��i��er���iscourse?�Evi��ence�fro��the�N400�effect.�Brain Research – Cognitive Brain Research,�17,�701–718.

Weber,�C.,�Hahne,�A.,�Frie��rich,�M.�an���Frie��erici,�A.�D.�(2004).�Discri�ination�of��or���stress�in�earl���infant��erce�tion:�Electro�h��siolo��ical�evi��ence.�Brain Research – Cognitive Brain Research,�18,�149–161.

Win�ler,�I.,�Kujala,�T.,�Al�u,�P.�an���Naatanen,�R.�(2003).�Lan��ua��e�context�an����honetic�chan��e���etection.�Brain Research – Cognitive Brain Research,�17,�833–844.

Win�ler,�I.,�Kujala,�T.,�Tiitinen,�H.,�Sivonen,�P.,�Al�u,�P.,�Lehto�os�i,�A.,�Czi��ler,�I.,�Cse�e,�V.,�Il�onie�i,�R.�J.�an���Näätänen,�R.�(1999).�Brain�res�onses�reveal�the�learnin���of�forei��n�lan��ua��e��hone�es.�Psychophysiology,�36,�638–642.

Win�ler,� I.,� Reini�ainen,� K.� an��� Näätänen,� R.� (1993).� Event-relate��� brain� �otentials� reflect�traces�of�echoic��e�or���in�hu�ans.�Perception and Psychophysics,�53,�443–449.

Woo��s,� D.� L.,� Alho,� K.� an��� Al��azi,� A.� (1993a).� Inter�o��al� selective� attention:� Evi��ence� for��rocessin���in�tonoto�ic�au��itor���fiel��s.�Psychophysiology,�30,�287–295.

Woo��s,�D.�L.,�Kni��ht,�R.�T.�an���Scabini,�D.�(1993b).�Anato�ical�substrates�of�au��itor���selec-tive�attention:�Behavioral�an���electro�h��siolo��ical�effects�of��osterior�association�cortex�lesions.�Brain Research – Cognitive Brain Research,�1,�227–240.

Ya�asa�i,�H.,�LaBar,�K.�S.�an���McCarth��,�G.�(2002).�Dissociable��refrontal�brain�s��ste�s�for�attention�an���e�otion.�Proceedings of the National Academy of Sciences USA,�99,�11447–11451.

Yantis,�S.,�Sch�arzbach,�J.,�Serences,�J.�T.,�Carlson,�R.�L.,�Stein�etz,�M.�A.,�Pe�ar,�J.�J.�an���Court-ne��,�S.�M.�(2002).�Transient�neural�activit���in�hu�an��arietal�cortex���urin���s�atial�atten-tion�shifts.�Nature Neuroscience,�5,�995–1002.

Z�itserloo��,�P.�(1989).�The�locus�of�the�effects�of�sentential-se�antic�context�in�s�o�en-�or����rocessin��.�Cognition,�32,�25–64.

Page 219: Unconscious Memory Representations in Perception
Page 220: Unconscious Memory Representations in Perception

chapter�8

Unconscious memory representations underlying music-syntactic processing and processing of auditory oddballs*

Stefan�Koelsch�Universit���of�Sussex,�Bri��hton,�UK�/�Max-Planc�-Institute��for�Hu�an�Co��nitive�an���Brain�Sciences,�Lei�zi��,�Ger�an���

�.1 Introduction

In�1992,�a�stu�����fro��Saarinen�an���collea��ues�chan��e���the�conce�t�of�the��is-�atch�ne��ativit���(MMN)���ra�aticall���(Saarinen�et�al.�1992).�Whereas��revious�stu��ies�ha���investi��ate���the�MMN�onl����ith��h��sical���eviants�(such�as�frequen-c��,� intensit��,� or� ti�bre� ��eviants),� Saarinen� et� al.� sho�e��� that� a� brain� res�onse�re�iniscent�of�the�MMN�can�be�elicite���b���chan��es�of�abstract�au��itor���features�(in�that�stu����,�stan��ar���sti�uli��ere�tone��airs��ith�frequenc���levels�that�varie���across�a��i��e�ran��e,�but��ere�al�a��s�risin���in��itch,��hereas���eviants��ere�tone��airs�fallin���in��itch).�B���intro��ucin���the�conce�t�of�an�“abstract�feature�MMN”�(henceforth�referre���to�as�afMMN),�Saarinen�et�al.�i��licitl���chan��e���the��revi-ous�conce�t�of�the�MMN�as�a�res�onse�to�a��h��sical���eviance��ithin�a�re�etitive�au��itor��� environ�ent� (henceforth� referre��� to� as� phMMN)� to� a� conce�t� of� the�MMN�as�a�ne��ative�event-relate����otential�(ERP)�res�onse�to��is�atches�in���en-eral,�i.e.�to��is�atches�that���o�not�necessaril���have�to�be��h��sical�in�nature�(for�other�stu��ies�re�ortin���abstract�feature�MMNs�see,�e.��.,�Paavilainen�et�al.�1998,�2001,�2003,�2007;�Korz��u�ov�et�al.�2003;�Schrö��er�et�al.�2007).�

Hence,��hen�a�fe����ears�after�the�stu�����fro��Saarinen�et�al.�(1992)�a�stu�����on�neuro�h��siolo��ical�correlates�of��usic��rocessin���re�orte���a��is�atch�re-s�onse�for��usic-s��ntactic�re��ularities�(Koelsch�et�al.�2000),� it��as���ifficult� to���eci��e��hether�or�not�this��is�atch�res�onse�shoul���be�referre���to�as�MMN:�In�that�stu�����(Koelsch�et�al.�2000),�sti�uli��ere�chor���sequences,�each�sequence�consistin��� of� five� chor��s.� There� �ere� three� sequence� t���es� of� interest:� (1)� se-quences�consistin���of��usic-s��ntacticall���re��ular�chor��s,�(2)�sequences��ith�a�

Page 221: Unconscious Memory Representations in Perception

210� Stefan�Koelsch

�usic-s��ntacticall���irre��ular�chor���at�the�thir����osition�(i.e.,�in�the��i����le)�of�the�sequence,�an���(3)�sequences��ith�a��usic-s��ntacticall���irre��ular�chor���at�the�fifth�(i.e.�final)��osition�of�the�sequence�(Fi��ure�1a;�for�stu��ies�usin���si�ilar�ex-�eri�ental�sti�uli�see�Loui�et�al.�2005;�Leino�et�al.�2007).�Irre��ular�chor��s��ere�so-calle���“Nea�olitan�sixth�chor��s”��hich�are�consonant�chor��s��hen��la��e���in�isolation,�but��hich�are�har�onicall���onl�����istantl���relate���to�the��rece��in���har�onic�context�an��,�hence,�soun���hi��hl���unex�ecte����hen��resente���at�the�en���of�a�chor���sequence�(ri��ht��anel�of�Fi��ure�1a).�The�sa�e�chor��s��resente���in� the� �i����le� of� these� chor��� sequences� (�i����le� �anel� of� Fi��ure� 1a),� ho�ever,�soun����uch�less�unex�ecte��,�but�relativel���acce�table�(�resu�abl���because�Nea-�olitan�sixth�chor��s�are�si�ilar�to�sub��o�inants,��hich�are��usic-s��ntacticall���re��ular� at� that��osition�of� the� sequence).� In� the�ex�eri�ents�of�Koelsch�et� al.�(2000),� chor���sequences��ere��resente��� in���irect� succession� (re�iniscent�of�a��usical��iece,�Fi��ure�1b),��ith�50��ercent�of�the�sti�uli�bein���re��ular�sequences,��

Figure 1. (a)�Exa��les�of�chor���sequences�containin���in-�e���chor��s�onl���(left),�a�Nea-�olitan�sixth�chor���at�the�thir���(�i����le),�an���at�the�fifth��osition�(ri��ht).�In�the�ex�eri-�ent,�sequences��ere��resente���in���irect�succession�(b).�Co��are���to�re��ular�in-�e���chor��s,�the��usic-s��ntacticall���irre��ular�Nea�olitan�chor��s�elicite���an�ERAN�(c).�Note�that��hen�Nea�olitans�are��resente���at�the�fifth��osition�of�a�chor���sequence�(�here�the���are��usic-s��ntacticall���hi��hl���irre��ular),�the�ERAN�has�a�lar��er�a��litu��e�co��are���to��hen�Nea�olitan�chor��s�are��resente���at�the�thir����osition�of�the�sequences�(�here�the���are��usic-s��ntacticall���less�irre��ular�than�at�the�fifth��osition).

Page 222: Unconscious Memory Representations in Perception

� Unconscious��e�or���re�resentations� 211

25��ercent�containin���an�irre��ular�chor���at�the�thir��,�an���25��ercent�an�irre��u-lar�chor���at�the�final��osition�of�the�sequence.�

The�irre��ular�chor��s�elicite���an�ERP�effect��hich�ha���a�stron���rese�blance��ith� the� MMN:� It� ha��� ne��ative� �olarit��,� �axi�al� a��litu��e� values� over� fron-tal� lea��s� (�ith� ri��ht-he�is�heric� �re��o�inance),� an��� a� �ea�� latenc��� of� about�150–180��s�(Fi��ure�1c).�Ho�ever,��hen�To��Gunter�an���I���iscovere���this�brain�res�onse,��e���i���not�label�it�as�“�usic-s��ntactic�MMN”,�but�as�early right anterior negativity�(ERAN,�Koelsch�et�al.�2000).�One�reason�for�this�ter�inolo������as�that�the�ERAN��as�also�stron��l���re�iniscent�of�an�ERP�effect�elicite���b���s��ntactic�ir-re��ularities���urin���lan��ua��e��erce�tion:�the�early left anterior negativity�(ELAN,�Frie��erici�2002;�see�also�belo�).�Denotin���the�ERP�res�onse�to�har�onic�irre��u-larities�as�ERAN,�thus,�e��hasize���the�notion�that�this�ERP��as�s�ecificall���re-late���to�the��rocessin���of��usical�s��ntax�(or�“�usical�structure”).�

Nevertheless,�so�e�subsequent�stu��ies�have�also�referre���to�this�effect�as��u-sic-s��ntactic�MMN�(Koelsch�et�al.�2002a,�2003a,�b,�c),�not�onl�����ue�to�the�rese�-blance��ith�the�MMN,�but�also�because�the�ter��earl���right�anterior�ne��ativit���falls�short��hen�the�effect�elicite���b���irre��ular�chor��s�is�not�si��nificantl���lateral-ize��.�Lac��of�lateralization�also�le���authors�to�label�effects�elicite���b����usic-s��n-tacticall���irre��ular�events�as�early anterior negativity�(Loui�et�al.�2005),�or�early negativity� (Steinbeis� et� al.� 2006).� Ho�ever,� other� stu��ies� use��� the� ter�� ERAN�even��hen�the�effect��as�not�si��nificantl���ri��ht-lateralize���over�the�scal��for�t�o�reasons:� (1)� Functional� neuroi�a��in��� stu��ies� consistentl��� sho�e��� ri��ht-he�i-s�heric��re��o�inance�for�the�activation�of�the�structures�that�are�assu�e���to�be�involve���in�the���eneration�of�the�ERAN�(see�also�section�about�the�neural���enera-tors�of� the�ERAN�in�this�cha�ter).�That� is,�althou��h�the�ERAN��as�so�eti�es�not�si��nificantl���lateralize���in�the�scal��to�o��ra�hies�of�electroence�halo��ra�hic�(EEG)���ata,�it�is�hi��hl���li�el���that�the�brain�activit���un��erl��in���the���eneration�of�the�ERAN��as�nevertheless�stron��er�in�the�ri��ht�than�in�the�left�he�is�here.�(2)�The�ter��ERAN��as�also�establishe����ith�re��ar���to�the�functional�si��nificance�of�this�ERP�co��onent,�rather�than�onl���for�its�scal����istribution�(Koelsch�et�al.�2007;�Maess�et�al.�2001;�Miran��a�an���Ull�an�2007;�Mai��hof�an���Koelsch�2007).�Note�that�si�ilar�conflicts�exist�for��ost�(if�not�all)�en��o��enous�ERP�co��onents:�E.��.,�the�P300�is�often�not��axi�al�aroun���300��s�(e.��.,�McCarth���an���Donchin�1981),�the�N400�elicite���b���violations�in�hi��h�cloze��robabilit���sentences�t���icall���starts�aroun���the�P2�latenc���ran��e�(Gunter�et�al.�2000;�van���en�Brin��et�al.�2001),�an���the�MMN�has�so�eti�es��ositive��olarit���in�infants�(e.��.,�Win�ler�et�al.�2003;�Frie��erici�et�al.�2002).�Thus,�the�ter��ERAN�shoul���be�use���for�(relativel���earl��)�ERP�effects�elicite���b����usic-s��ntactic�irre��ularities.�

Page 223: Unconscious Memory Representations in Perception

212� Stefan�Koelsch

�.2 Musical syntax

The�ERAN�reflects��usic-s��ntactic��rocessin��,� i.e.��rocessin���of�abstract� re��u-larit��-base��� au��itor��� infor�ation.� In� �ajor-�inor� tonal� �usic� (often� si��l���referre���to�as�“Western”��usic),��usic-s��ntactic��rocessin���of�chor���functions�(be� the���constitute���b���chor��s�or�b��� subsequent� tones)�co��rises� several� sub-�rocesses�(see�Fi��ure�2a�for�ex�lanation�of�chor���functions).�These�sub-�rocesses�are���rou�e���in�the�follo�in���into�several�core�as�ects,��ith�the�or��erin���of�these�as�ects�not�bein���inten��e���to�reflect�a�te��oral�or��er�of��usic-s��ntactic��rocess-in���(that�is,�the�sub-�rocesses��a����artl���ha��en�in��arallel).�Note�that��usical�s��ntax�also�co��rises�other�structural�as�ects,�such�as��elo��ic�(e.��.�voice�lea��-in��),�rh��th�ic,��etric,�an���ti�bral�structure.�S��ntactic��rocessin���of�such�struc-tural�as�ects�has,�ho�ever,�to������no�le����e�not�been�investi��ate���so�far.�

(1)�Extraction of a tonal centre.�In��ajor-�inor�tonal��usic,��usic-s��ntactic��ro-cessin���of�chor���functions�(be�the���constitute���of�chor��s�or��elo��ies)�relies�on�the�extraction�of�a�tonal�centre,�for�exa��le,�C�in�the�case�of�a��assa��e�in�C��a-jor.�The�tonal�centre�corres�on��s�to�the�root�of�a�tonal��e���(an���is��erceive���as�the�best�re�resentative�of�a�tonal��e��,�see�Kru�hansl�an���Kessler�1982).�In�ter�s�of� har�onic� function,� the� tonal� centre� is� also� the� root� of� the� tonic� chor��� (see�Fi��ure�2a�for�ex�lanation�of�the�ter��“tonic”),�an���thus�the�reference��oint�for�the�tonal�hierarch���or�chor���functions�(see�also�belo�).�

The��rocess�of�establishin���a�re�resentation�of�a�tonal�centre�is�nor�all���an�iterative��rocess�(Kru�hansl�an���Toivainen�2001),�an���this��rocess�has�to�be�en-��a��e���each�ti�e�the�tonal��e���chan��es.�To�extract�a�tonal�centre,�or�to���etect�a�shift�in�tonal��e���(an���thus�a�shift�of�the�tonal�centre),�listeners�have�to�sequence��usical�infor�ation,�abstract�a�tonal�centre�fro��the���ifferent�tones�of�a��usical��assa��e,��ee��the�re�resentation�of�the�tonal�centre�in�short-ter���e�or��,�an���realize��hen�the��e�or���re�resentation�of�a�tonal�centre���iffers�fro��that�of�ne���usical�infor�ation.�A���escri�tion�of�the�co��nitive�re�resentation�of�tonal��e��s�is��rovi��e��,�e.��.,�in�Kru�hansl�an���Kessler�(1982).�

Previous�stu��ies�have�sho�n�that�listeners�ten���to�inter�ret�the�first�chor���(or�tone)�of�a�sequence�as�the�tonic�(that�is,�as�the�tonal�centre;�Kru�hansl�an���Kessler�1982).�In�case�the�first�chor���is�not� the�tonic,� listeners�have�to��o��if���their� initial� inter�retation�of� the� tonal� centre���urin��� the��erce�tion�of� subse-quent� tones� or� chor��s� (Kru�hansl� an��� Kessler� 1982;� for� a� conce�tion� of� �e���i��entification� �ithin� the� tonal� i��io�� see� the� intervallic� rivalr��� �o��el� fro��Bro�n�et�al.�1994).�

Page 224: Unconscious Memory Representations in Perception

� Unconscious��e�or���re�resentations� 213

(2)�Successive�tones�an���chor��s�are�activel���relate���to�the�tonal�centre,�as��ell�as�to�each�other,�in�ter�s�of�har�onic���istance�(see�also�Fi��ure�3a).�E.��.,�in�C��ajor,�a�G��ajor�chor���is��ore�closel���relate���to�C��ajor�than�a�G#��ajor�chor��.

(3)�Base���on�the�har�onic�relations�bet�een�chor��s,�a�tonal�or��er�(or�“hierarch��”,�Bharucha�an���Kru�hansl�1983;�see�also�Till�ann�et�al.�2000,�2008)�is�establishe��,�accor��in���to��hich�the�confi��uration�of��reviousl���hear���chor���functions�for�s�a�tonal�structure�(or�a�structural�context).�For�exa��le,��ithin�the�tonal�“hierarch���of�stabilit��”�(Bharucha�an���Kru�hansl�1983)�the�tonic�chor���is�the��ost�“stable”�chor��,�follo�e���b���the���o�inant�an���the�sub��o�inant,��hereas�chor��s�such�as�the�sub�e��iant�an���the�su�ertonic�re�resent�less�stable�chor��s�(ibi��.).�

I��ortantl��,��hen�in��ivi��uals�fa�iliar��ith��ajor-�inor�tonal��usic�listen�to�such��usic,�the���transfer�the�sensual��itch�infor�ation�(that�is,�infor�ation�about�the��itches�of�the�tones�of��elo��ies�or�chor��s)�into�a�co��nitive�re�resenta-tion�of�the�location�of�tones�an���chor��s��ithin�the�tonal�hierarch���of�a��e��,�as��ell�as��ithin�the�(�ajor-�inor)�tonal��e���s�ace�(see�also�Fi��ure�3).�For�exa��le,��ithin�a�G��ajor�context,�the�sensor����erce�t�of�three�si�ultaneousl���soun��in���tones��ith��itches�for�in���a��ajor�tria���(such�as�c’,�e’,�an�����’)� is�transfor�e���into�the�location�relative�to�the�tonic�(Fi��ure�3a)�as��ell�as�relative�to�the�tonal�centre�(that�is,�relative�to�the�tonal�reference��oint;�see�Fi��ure�3).�The�ter��“loca-tion�relative�to�the�tonal�centre”�refers�to�the��lace�in�the��a��of��e��s�(or�on�the�torus�of��e��s;�Kru�hansl�an���Kessler�1982,�see�also�Fi��ure�3��)�in�relation�to�the�tonic,�an���the�ter��“relative�to�the�tonic”�refers�to�the�chor���function�(c-e-���is�the� sub��o�inant� in�G��ajor).� In�other��or��s,��hen��rocessin���har�onic� in-for�ation,�listeners�relate�ne��har�onic�infor�ation�to�the��revious�har�onic�context�in�ter�s�of�har�onic���istance,�an���in�ter�s�of�its�functional-har�onic�infor�ation.�

Note�that��itch�hei��ht�infor�ation�is�one-��i�ensional�(ran��in���fro��lo��to�hi��h),��hereas�the�co��nitive�re�resentation�of��ajor-�inor�tonal�s�ace�is�at�least�t�o-�(if�not�four-)���i�ensional�(Fi��ure�3���an���Kru�hansl�an���Kessler�1982,�see�ibi��.�for�a���escri�tion�of�the��ulti-��i�ensional�co��nitive�re�resentation�of��a-jor-�inor�tonal�s�ace;�see�also�Kru�hansl�et�al.�1982).�

The� ��istances� bet�een� chor��� functions� an��� �e��s� in� ter�s� of� �usic� theor���correlate��ith�acoustic�si�ilarit���(see�Le�an�2000,��ho�sho�e���that�the��e����ro-files�obtaine���b���Kru�hansl�an���Kesssler�1982,�can�lar��el���be�accounte���for�b����easures�of�acoustic�si�ilarit��).�Ho�ever,�acoustic�si�ilarit���of�t�o�chor��s�is�a�one-��i�ensional��easure,��hereas�the�tonal�s�ace�of�chor���functions�an����e��s�a��ears�to�be�re�resente���in�listeners�in��ore�than�one���i�ension�(see�above�an���Kru�hansl�an���Kessler�1982;�Kru�hansl�et�al.�1982).�That�is,��hen��e�listen�to�a�

Page 225: Unconscious Memory Representations in Perception

214� Stefan�Koelsch

�usical��iece�(at�least�if��e�are�fa�iliar��ith��ajor-�inor�tonal��usic),��e���o�not�onl����onitor�chor���transitions��ith�re��ar���to�their�acoustic�si�ilarit��,�but�also��ith�re��ar���to�both�the�tonal�s�ace�of��e��s,�an���the�hierarch���of�stabilit���(that�is,��ith�re��ar���to�their�chor���function).�This���ifference�in���i�ensionalit���is�an�i�-�ortant���ifference�bet�een�“acoustical”�an���“�usical”��rocessin��.�

(4)�Once�a�har�onic�hierarch��� is�establishe��,��ovin���a�a��� fro��a� tonal�cen-tre� �a��� be� ex�erience��� as� tensionin��,� an��� �ovin��� bac�� as� releasin��� (see� also��Ler��ahl�2001;�Patel�2003).�This� si��le� state�ent�has� several� i��ortant� i��li-cations:�Firstl��,��ovin���throu��h�tonal�s�ace�establishes�hierarchical��rocessin���(for���etails� see�Ler��ahl�2001�an���Patel�2003).�Secon��l��,� the� tension-resolution��atterns� e�er��in��� fro�� �ovin��� throu��h� tonal� s�ace� have� e�otional� qualit���(Me��er�1956;�Steinbeis�et�al.�2006;�Koelsch�et�al.�2008).�Thir��l��,��ovin���a�a��,�an���bac��to�a�tonal�centre�also�o�ens�the��ossibilit���for�recursion,�because��hile��ovin���a�a��� fro��a� tonal� centre� (e.��.,� to� the���o�inant,� that� is� in�C��ajor:� a�G �ajor�chor��),�a�chan��e�of��e����i��ht�ta�e��lace�(e.��.,�fro��C��ajor�to�G��ajor),�an����ithin�the�ne���e���(no��G��ajor)�–��hich�no��has�a�ne��tonal�centre�–�the��usic� �i��ht� a��ain� �ove� a�a��� fro�� the� tonal� centre� (e.��.� to� the� ��o�inant� of�G �ajor),�until�it�returns�to�the�tonal�centre�of�G,�an���then�to�the�tonal�centre�of�C��ajor�(for�stu��ies�investi��atin���neural�correlates�of�the��rocessin���of�chan��es�in�tonal��e����ith�electroence�halo��ra�h���or�functional��a��netic�resonance�i�-a��in���see�Koelsch�et�al.�2002c,�2003b,�2005b;�Janata�et�al.�2002b).

(5)�In�a����ition,�the�succession�of�chor���functions�follo�s�st��le-s�ecific�statistical�re��ularities,�that�is,��robabilities�of�chor���transitions�(Rie�ann�1877;�Rohr�eier�2005).�For�exa��le,�in�a�statistical�stu�����b���Rohr�eier�(2005)�on�the�frequen-cies� of� ��iatonic� chor��� �ro��ressions� in� Bach� chorales,� the� su�ertonic� �as� five�ti�es��ore�li�el���to�follo��the�sub��o�inant�than�to��rece��e�it.�These�statistical�re��ularities� are� an� i��ortant� characteristic� of� �usical� s��ntax� �ith� re��ar��s� to�the�har�onic�as�ects�of��ajor-�inor�tonal��usic�(other�characteristics��ertain,�e.��.,�the��rinci�les�of�voice-lea��in��).�The�re�resentations�of�such�re��ularities�are�store��� in� lon��-ter�� �e�or��,� an��� b��� its� ver��� nature� it� nee��s� listenin��� ex�eri-ence�to�extract�the�statistical��ro�erties�of�the��robabilities�for�the�transitions�of�chor���functions�(see�also�Till�ann�et�al.�2000).�These�statistical��ro�erties�are�i��licitl���learne���in�the�sense�that�the���are�extracte����ithout�conscious�effort�b���in��ivi��uals�(usuall���even��ithout�in��ivi��uals�bein���a�are�of�it),�an���store���in�a�lon��-ter���e�or���for�at.�

Page 226: Unconscious Memory Representations in Perception

� Unconscious��e�or���re�resentations� 215

It� is� i��ortant� to� un��erstan��� that,� �hile� listeners� fa�iliar� �ith� (Western)�tonal��usic��erceive�a�sequence�of�chor��s,�the���auto�aticall����a�e��re��ictions�of�li�el���chor���functions�to�follo�.�That�is,�listeners�extra�olate�ex�ectancies�for�subsequent�soun��s�of�re��ular�chor��s,�base���on�re�resentations�of��usic-s��ntac-tic�re��ularities;�chor��s�(or�tones)�that��is�atch��ith�the��usic-s��ntactic�soun���ex�ectanc���of�a�listener�elicit��rocesses�that�are�electricall���reflecte���in�an�ERAN�(Koelsch�et�al.�2000).�The��athe�atical��rinci�les�fro���hich�the��robabilities�for�chor���transitions��ithin�a�tonal��e����i��ht�have�e�er��e���are�un��er�current�investi��ation�(see,�e.��.,�Woolhouse�an���Cross�2006,�for�the�interval�c��cle-base����o��el�of��itch�attraction),�an���it�a��ears�that��an���of�these��rinci�les�re�resent�abstract,�rather�than��h��sical�(or�acoustical)�features�(Woolhouse�an���Cross�2006;�note�that,�in�a����ition�to�transition��robabilities�of�chor���functions,�frequencies�of�co-occurrences,�as��ell�as�frequencies�of�occurrences�of�chor���functions�an���tones�also�re�resent�statistical�re��ularities,�see�Till�ann�et�al.�2008).�

It�is�li�el���that�ste�s�(1)�an���(2)�can�–�at�least�a��roxi�atel���–�be��erfor�e���even�b���hu�ans��ithout��rior�ex�erience�of�Western��usic�(e.��.,�b���ne�borns,�or�b���a��ult� listeners�naive� to�Western��usic).�Ho�ever,� several�stu��ies�su����est�that�the�fine-��raine���co��nitive��rocesses�require���for�tonic�i��entification�that�are�t���icall���observe���in�Western�listeners�(even��hen�the���haven’t�receive���for�al��usical� trainin��)�are�base���on�extensive��usical� ex�erience� (e.��.,�La�ont�an���Cross�1994).�Li�e�ise,�calculatin���subtle���istances�bet�een�chor���functions�an���a�tonal�centre�a��ears�to�rel���on�learnin���(see�also�Te��an�an���Bharucha�1998).�

Whether�ste��(3)�can�be��erfor�e����ithout��rior�ex�erience�of�Western��u-sic�is�un�no�n,�but��revious�stu��ies�stron��l���su����est�that�the���etaile���nature�of�the�tonal�hierarch���sche�a�is�learnt�throu��h�earl���chil��hoo���(La�ont�an���Cross�1994).�That�is,��hile�it�is�conceivable�that�hu�ans�naive�to�Western��usic�fin���the� �robabilities� for� chor��� transitions� �lausible� (because� the��� follo�� abstract��athe�atical��rinci�les��hich�beco�e�a��arent�in�s�ecific�transitions�of�chor��s,�Woolhouse�an���Cross�2006),�re�eate���ex�erience�of�Western��usic�is�necessar���to�acquire�the��no�le����e�about�the��robabilities�of�the�transitions�of�chor���func-tions,�as��ell�as��no�le����e�about�frequencies�of�co-occurrences�of�chor���func-tions,�an���frequencies�of�occurrences�of�chor���functions�an���tones�(see�above).�Because�this��no�le����e�is�essential�for�the��re��iction�of�subsequent�chor���func-tions�(an��,�thus,�for�buil��in���u��a�har�onic�soun���ex�ectanc��),�it�is�hi��hl���li�el���that�the�ERAN��oul���not�be�elicite����ithout�such��no�le����e�(see�also�section�on��usical�trainin���in�this�cha�ter).�

Page 227: Unconscious Memory Representations in Perception

21�� Stefan�Koelsch

Figure 2. Chor���functions�are�the�chor��s�built�on�the�tones�of�a�scale�(a).�The�chor���on�the�first�scale�tone,�e.��.,�is���enote���as�the�tonic,�the�chor���on�the�secon���scale�tone�(in��ajor)�as�su�ertonic,�on�the�thir���scale�tone�as��e��iant,�on�the�fourth�scale�tone�as�sub-��o�inant,�an���the�chor���on�the�fifth�scale�tone�as�the���o�inant.�The��ajor�chor���on�the�

Page 228: Unconscious Memory Representations in Perception

� Unconscious��e�or���re�resentations� 217

secon���tone�of�a��ajor�scale�can�be�inter�rete���as�the���o�inant�to�the���o�inant�(square�brac�ets).�In��ajor-�inor�tonal��usic,�chor���functions�are�arran��e����ithin�har�onic�sequences�accor��in���to�certain�re��ularities.�One�exa��le�for�a�re��ularit��-base���arran��e-�ent�of�chor���functions�is�that�the���o�inant-tonic��ro��ression�is�a��ro�inent��ar�er�for�the�en���of�a�har�onic�sequence,��hereas�a�tonic-��o�inant��ro��ression�is�unacce�t-able�as�a��ar�er�of�the�en���of�a�har�onic�sequence�(see�text�for�further�exa��les).��A�sequence�en��in���on�a�re��ular���o�inant-tonic��ro��ression�is�sho�n�in�the�left��anel�of�(b).�The�final�chor���of�the�ri��ht��anel�of�(b)�is�a���o�inant�to�the���o�inant.�This�chor���function�is�irre��ular,�es�eciall���at�the�en���of�a�har�onic��ro��ression�(soun���exa��les�are�available�at����.stefan-�oelsch.��e/TC_DD).�In�contrast�to�the�sequences�sho�n�in�Fi��ure�1,�the�irre��ular�chor��s�are�acousticall���even��ore�si�ilar�to�the��rece��in���context�than�re��ular�chor��s�(see�text�for���etails;�a��a�te����ith��er�ission�fro��Koelsch�2005).�(c)�sho�s�the�ERPs�elicite���b���the�final�chor��s�of�these�t�o�sequence�t���es�(recor��e���fro��a�ri��ht-frontal�electro��e�site�[F4]�fro��t�elve�subjects,�fro��Koelsch�2005).�Both�sequence�t���es��ere��resente���in��seu��oran��o��or��er�equi�robabl���in�all�t�elve��ajor��e��s.�Althou��h��usic-s��ntacticall���irre��ular�chor��s��ere�acousticall����ore�si�ilar�to�the��rece��in���har�onic�context�than�re��ular�chor��s,�the�irre��ular�chor��s�still�elicit�an�ERAN.�The�ERAN�can�best�be�seen�in�the���ifference��ave�(soli���line),��hich�re�resents�re��ular�subtracte���fro��irre��ular�chor��s.�With�MEG,�the��a��netic�equivalent�of�the�ERAN��as�localize���in�the�inferior�frontolateral�cortex�(a��a�te����ith��er�ission�fro��Maess�et�al.�2001;�sin��le-subject���i�ole�solutions�are�in��icate���b���stri�e�����is�s,�the��hite���i�oles�in��icate�the���ran��-avera��e�of�these�source�reconstructions).�(e)�sho�s�activation�foci�(s�all�s�heres)�re�orte���b���functional�i�a��in���stu��ies�on��usic-s��ntactic��rocess-in���usin���chor���sequence��ara��i���s�(Koelsch�et�al.�2005;�Maess�et�al.�2001;�Till�ann�et�al.�2003;�Koelsch�et�al.�2002c)�an����elo��ies�(Janata�et�al.�2002b).�The�t�o�lar��er�s�heres�sho��the��ean�coor��inates�of�foci�(avera��e���for�each�he�is�here�across�stu��ies,�coor��i-nates�refer�to�stan��ar���stereotaxic�s�ace).�(A��a�te����ith��er�ission�fro��Koelsch�an���Siebel�2005.)�

Page 229: Unconscious Memory Representations in Perception

21�� Stefan�Koelsch

Figure 3. (a)�T�o-��i�ensional�re�resentation�of��e���relationshi�s�in�the�visualization�of�Kru�hansl�an���Kessler�(1982).�One�linear�axis�corres�on��s�to�the�Circle�of�Fifths,�the�other�to�the�hetero��eneous�axes�of��arallel�an���relative�thir����ajor/�inor�relationshi�s.�The�to�o��ra�hical���istances�bet�een��e��s�rou��hl���corres�on���to��erceive���har�onic���istances.�(b)�B���lin�in���relative�an����arallel�relations�(�ith�thir��s�on�the�horizontal��lane�an���fifths�on�the�vertical),�Gottfrie���Weber�(Weber�1817)�create���a�sche�atic���ia��ra��of�all��ajor�an����inor��e��s.�This�fi��ure�sho�s�a�stri��of��e��s�base���on�Weber’s�sche�atic���ia��ra�.�(c)�Curlin���of�the��e����a���resente���in�(b).�For�exa��le,���-�inor�occurs�as�the�relative��inor�of�B-�ajor,�an���as�the��arallel��inor�of�G-�ajor.�When�the�stri��is�curle���as�above,�these��e��s�can�be�overlai��.�(��)�Once�the�re��un��ant�stri�s�overla�,�the�curls�can�be�co��acte���into�a�sin��le�tube,�the�enhar�onic�equivalents�at�both�en��s�of�the�tube�can�be��ra��e���horizontall���to��ro��uce�a�three-��i�ensional�torus.�This�torus�rou��hl���re�resents��ajor-�inor�tonal��e���s�ace.�(A��a�te����ith��er�ission�fro��Pur�ins�et�al.�2007.)

Page 230: Unconscious Memory Representations in Perception

� Unconscious��e�or���re�resentations� 219

�.3 A neural mechanism underlying music-syntactic processing

With� re��ar��� to� inter�retin��� event-relate��� brain� �otentials� (ERPs)� such� as� the�ERAN,� it� is� i��ortant� to� be� a�are� of� the� ��ifficult��� that� �usic-s��ntacticall��� ir-re��ular�chor��s�often�also�re�resent�a��h��sical���eviance�(as��ill�be���escribe���be-lo�� in� �ore� ��etail).� This� is� the� case,� e.��.,� for� the� Nea�olitan� chor��s� sho�n� in�Fi��ure�1a:�The�re��ular�chor��s�belon��e���to�one�tonal��e��,�thus��ost�notes��la��e���in�an�ex�eri�ental�bloc��belon��e���to�this��e���(e.��.,�in�C��ajor�all��hite��e��s�on�a��e��boar��),��hereas�the�Nea�olitan�chor��s�intro��uce����itches�that�ha���not�been��resente���in�the��revious�har�onic�context�(see�the�flat�notes�of�the�Nea�olitan�chor��s�in�Fi��ure�1b).�Thus,�the�ERAN�elicite���b���those�chor��s��as��resu�abl����artl���overla��e���b���a��hMMN.�In�other��or��s,� the���etection�of� the� irre��ular�chor��s��as��artl�����ue�to�i��licit,�or�“unconscious”,��e�or���re�resentations�of��usic-s��ntactic�irre��ularities,�but�also��artl�����ue�to�unconscious�re�resentations�(or� so-calle��� sensor��� �e�or��� traces)� of� acoustic� re��ularities� (fro�� �hich� the�Nea�olitan�chor��s���eviate��).�

Nevertheless,� it� is�also� i��ortant� to�note� that� the�Nea�olitan�chor��s�at� the�thir����osition�of�the�sequences�serve���as�acoustical�control�sti�uli,�allo�in���an�esti�ate�of�the���e��ree�of��h��sical��is�atch�res�onses�to�Nea�olitan�chor��s.�The���ata�sho�e���that�the�ERAN�elicite���b���chor��s�at�the�final��osition�of�chor���se-quences��as�consi��erabl��� lar��er� than�the�ERAN�elicite���b���chor��s�at� the�thir����osition�of�the�sequences�(Fi��ure�1c).�This�sho�e���that�the�effects�elicite���b���the�Nea�olitan�chor��s�at�the�final��osition�of�the�chor���sequences�coul���not�si��l���be�an�MMN,�because�an�MMN��oul���not�have�sho�n���ifferent�a��litu��es�at���if-ferent��ositions��ithin�the�sti�ulus�sequence�(Koelsch�et�al.�2001;�in�that�stu�����the�ERAN,�but�neither�the��hMMN�nor�the�afMMN���iffere���bet�een��ositions�in�the�chor���sequences).�

Corroboratin���these�fin��in��s,�the�stu�����fro��Leino�et�al.�(2007)�sho�e���that�the�a��litu��e�of�the�ERAN,�but�not�the�a��litu��e�of�an�MMN�elicite���b����is-tune���chor��s,���iffere���bet�een���ifferent��ositions��ithin�chor���sequences.�A�ver���nice�feature�of�that�stu������as�that�chor���sequences�co��rise���of�seven�chor��s,�an���that�the����ere�co��ose���in�a��a���that�Nea�olitan�chor��s�occurrin���at�the�fifth��osition��ere��usic-s��ntacticall���less�irre��ular�than�Nea�olitans�at�the�thir����osition�(contrar��� to� the�sequences��resente��� in�Fi��ure�1a).�Consequentl��,� the�ERAN�elicite���at�the�fifth��osition��as�s�aller�than�the�ERAN�elicite���at�the�thir����osition�(an���the�ERAN��as� lar��est��hen�elicite���b���Nea�olitan�chor��s�at� the�seventh��osition,��here�the����ere��ost�irre��ular).�I��ortantl��,�irre��ular�chor��s�at�both�the�thir���an���at�the�seventh��osition�follo�e�����o�inant-seventh�chor��s;�therefore,�the�fact�that�the�ERAN��as�lar��er�at�the�seventh�than�at�the�thir����osi-tion�cannot�si��l���be�ex�laine���b���local�statistical��robabilities�for�the�transition�

Page 231: Unconscious Memory Representations in Perception

220� Stefan�Koelsch

of�chor���functions.�This�is�a�stron���hint�that�the���eneration�of�the�ERAN�involve���hierarchical� �rocessin��,� because� �ithin� the� hierarchical� structure� of� the� chor���sequences,�the���e��ree�of�irre��ularit����as�hi��her�at�the�seventh�than�at�the�thir����osition�of�the�chor���sequences.�

Ho�ever,� the� fact� that� the�ERAN�elicite���b����usic-s��ntacticall��� irre��ular�events� is� often� �artl��� overla��e��� b��� a� �hMMN� results� fro�� the� fact� that,� for�the��ost��art,��usic-s��ntactic�re��ularities�co-occur��ith�acoustic�si�ilarit��.�For�exa��le,�in�a�har�onic�sequence�in�C �ajor,�a�C# �ajor�chor���(that���oes�not�be-lon���to�C �ajor)�is��usic-s��ntacticall���irre��ular,�but�the�C# �ajor�chor���is�also�acousticall���less�si�ilar�to�the�C �ajor�context�than�an���other�chor���belon��in���to�C �ajor�(because�the�C# �ajor�chor���contains�t�o�tones�that���o�not�belon���to�the�C �ajor�scale).�Thus,�an���ex�eri�ental�effects�evo�e���b���such�a�C# �ajor�chor���can�not�si��l���be�attribute���to��usic-s��ntactic��rocessin��.�Because�such�a�C# �ajor�chor���is�(in�the�first�inversion)�the�enhar�onic�equivalent�of�a�Nea-�olitan�sixth�chor��,� it� is� li�el��� that�effects�elicite���b���such�chor��s� in��revious�stu��ies�(e.��.,�Koelsch�et�al.�2000;�Loui�et�al.�2005;�Leino�et�al.�2007)�are�not�onl�����ue�to��usic-s��ntactic��rocessin��,�but�also��artl�����ue�to�acoustic���eviances�that�occurre����ith�the��resentation�of�the�Nea�olitan�chor��s�(for�further���etails�see�also�Koelsch�et�al.�2007).�

In� fact,� tonal�hierarchies,�an����usic-s��ntactic�re��ularities�of��ajor-�inor�tonal��usic�are��artl�����roun��e���on�acoustic�si�ilarities�(e.��.,�Le�an�2000),��os-in��� consi��erable� ��ifficult��� on� the� investi��ation� of� �usic-s��ntactic� �rocessin��.�Ho�ever,�a�nu�ber�of�ERP�stu��ies�has�been��ublishe���so�far�that�ai�e���at���is-entan��lin���the�“co��nitive”��echanis�s�(relate���to��usic-s��ntactic��rocessin��)�fro��the�“sensor��”��echanis�s�(relate���to�the��rocessin���of�acoustic�infor�a-tion;�Re��nault�et�al.�2001;�Poulin-Charronnat�et�al.�2006;�Koelsch�2005,�2007;�Koelsch�an���Jentsch�e�2008;�for�i��ortant�behavioural�stu��ies�on�this�issue�see,�e.��.,�Bi��an���et�al.�1999;�Till�ann�et�al.�2008),�an���so�e�of�the��sho�e���that�the�ERAN�can�be�elicite���even��hen�the�s��ntacticall���irre��ular�chor��s�are�acousti-call����ore�si�ilar�to�a��rece��in���har�onic�context�than�s��ntacticall���re��ular�chor��s�(Koelsch�2005,�2007;�Koelsch�an���Jentsch�e�2008).�For�exa��le,�in�the�sequences�sho�n�in�Fi��ure�2b,�the��usic-s��ntacticall���re��ular�chor��s�(i.e.,�the�final�tonic�chor���of�the�sequence�sho�n�in�the�left��anel�of�Fi��ure�2b)�intro��uce���t�o�ne���itches,��hereas�the�irre��ular�chor��s�at�the�sequence�en��in���(so-calle�����ouble���o�inants,�sho�n�in�the�ri��ht��anel�of�Fi��ure�2b)�intro��uce���onl���one�ne���itch�(the�ne���itches�intro��uce���b���the�final�chor��s�are�in��icate���b���the�ar-ro�s�of�Fi��ure�2b).�Moreover,�the�s��ntacticall���irre��ular�chor��s�ha����ore��itches�in�co��on��ith�the��enulti�ate�chor���than�re��ular�chor��s,�thus�the�“sensor�����issonance”�(of��hich��itch�co��onalit���is�the��ajor�co��onent)�bet�een�fi-nal�an����enulti�ate�chor����as�not���reater�for�the�irre��ular�than�for�the�re��ular�

Page 232: Unconscious Memory Representations in Perception

� Unconscious��e�or���re�resentations� 221

sequence�en��in��s.�Nevertheless,�the�irre��ular�chor���functions�(occurrin����ith�a��robabilit���of�50��ercent)�elicite���a�clear�ERAN,�su����estin���that�electrical�reflec-tions�of��usic-s��ntactic��rocessin���can�be��easure����ithout�the��resence�of�a��h��sical�irre��ularit���(Fi��ure�2c).�

In� the� sequences� of� Fi��ure� 2b,� the� irre��ular� chor��s� (i.e.,� the� ��ouble� ��o�i-nants)���i���not�belon���to�the�tonal��e���establishe���b���the��rece��in���chor��s�(si�i-lar�to�the�Nea�olitan�chor��s�of��revious�stu��ies).�Ho�ever,�ex�eri�ents�usin���in-�e���chor��s�as��usic-s��ntacticall���irre��ular�chor��s�have�sho�n�that�an�ERAN�can�also�be�elicite���b���in-�e���chor��s,�in��icatin���that�the�elicitation�of�the�ERAN���oes�not�require�out-of-�e���notes�(Koelsch�et�al.�2007;�Koelsch�an���Jentsch�e�2008).�

The��ea�� latenc���of� the�ERAN�is�often�bet�een�170�an���220��s,��ith�the�exce�tion�of�four�stu��ies:�Koelsch�an���Mul��er�(2002)�re�orte���an�ERAN��ith�a�latenc���of�aroun���250��s,�Steinbeis�et�al.�(2006)�re�orte���an�ERAN��ith�a�laten-c���of�230��s�(in�the���rou��of�non-�usicians),�Ja�es�et�al.�(2008)�an�ERAN��ith�a�latenc���of�230��s�in�a���rou��of��usicians,�an���Patel�et�al.�(1998)�re�orte���an�ERAN-li�e�res�onse�(the�right anterior temporal negativity,�RATN)��ith�a��ea��latenc���of�aroun���350��s.�The�co��onalit���of�these�four�stu��ies��as�that�the���use���non-re�etitive�sequences,�in��hich�the��osition�at��hich�irre��ular�chor��s�coul���occur��as�un�re��ictable.�It�is�also�conceivable�that�the���reater�rh��th�ic�co��lexit���of�the�sti�uli�use���in�those�stu��ies�ha���effects�on�the���eneration�of�the�ERAN�(lea��in���to�lon��er�ERAN�latencies),�but��ossible�effects�of�rh��th�ic�structure� on� the� �rocessin��� of� har�onic� structure� re�ain� to� be� investi��ate��.�It�is�also�interestin���that�all�four�of�these�stu��ies�(Patel�et�al.�1998;�Koelsch�an����Mul��er�2002;�Steinbeis�et�al.�2006;�Ja�es�et�al.�2008)�re�orte���a��ore�centro-te��oral�than�frontal�scal����istribution�of�the�ERAN.�Ho�ever,�in�a�recent�stu�����that�also�use���non-re�etitive�sequences,�in��hich�the�occurrence�of�an�irre��ular�chor����as�un�re��ictable�(Koelsch�et�al.�2008),�the��ea��a��litu��e�of�the�ERAN��as�bet�een�158�an���180��s,�an���the��axi�u��of�the�ERAN��as�over�frontal�electro��es.�The�s�ecific�effects�on� latenc���an���scal����istribution�of� the�ERAN�are,�thus,�not���et�un��erstoo��.�

A�recent�stu�����on�oscillator���activit���associate����ith��usic-s��ntactic��ro-cessin���(Herrojo�et�al.�2009)�sho�e���that��usic-s��ntactic��rocessin���in�res�onse�to� irre��ular� chor��� functions� is� �ri�aril��� reflecte��� b��� lo�� frequenc��� (<� 8� Hz)�brain�oscillations.�The�s�ectral�ener�����(total�an���evo�e��)�of�both���elta�(<�4�Hz)�an��� theta� (4–7� Hz)� oscillations� increase��� in� res�onse� to� re��ular� chor��s� (�re-sente���at�the�final��osition�of�chor���sequences),�but�this�increase��as�stron��er�for� the� re��ular� than� for� the� irre��ular� chor��s.�The���ifference� in�oscillator���ac-tivit���bet�een�re��ular�an���irre��ular�chor��s��as�lar��er�over�the�ri��ht�than�over�the�left�he�is�here,�an����resu�abl�����ue�to��hase-resettin���of�oscillations�in�the�theta�ban��,�an���both��hase-resettin���an���increase�in�a��litu��e�of�oscillations�in�

Page 233: Unconscious Memory Representations in Perception

222� Stefan�Koelsch

the���elta�ban��.�Interestin��l��,�in�the�ERAN-ti�e��in��o��no�effects�of�irre��ular�chor��s��ere� foun��� in� the���a��a�ban���(��ifferences��ere� foun��,�ho�ever,� in�a�later�ti�e��in��o�,�aroun���500–550��s�after�sti�ulus�onset).�

The�ERAN�can�not�onl���be�elicite���b���chor��s.�T�o��revious�ERP�stu��ies��ith��elo��ies�sho�e���that�earl���anterior�brain�res�onses�can�also�be�elicite���b���sin��le�tones�(Miran��a�an���Ull�ann�2007;�Brattico�et�al.�2006;�the�latter�stu�����referre���to� these�brain�res�onses�as�MMN).�Moreover,�a� stu����� fro��Schön�an���Besson�(2005)�sho�e���that�the�ERAN�can�even�be�elicite���b���visuall���in��uce����usical�ex�ectanc���violations�(that�stu�����also�use����elo��ies).�

�.4 Processing of acoustic vs. music-syntactic irregularities

There�is�a�crucial���ifference�bet�een�the�neural��echanis�s�un��erl��in���the��ro-cessin���of�acoustic�irre��ularities�(as�reflecte���in��hMMN�an���afMMN)�on�the�one�si��e,�an����usic-s��ntactic��rocessin���(as�reflecte���in�the�ERAN)�on�the�other:�The���eneration�of�both��hMMN�an���afMMN�is�base���on�an�on-line�establish�ent�of�re��ularities�–�that� is,�base���on�re�resentations�of�re��ularities� that�are�extracte���on-line�fro��the�acoustic�environ�ent.�B���contrast,��usic-s��ntactic��rocessin���(as�reflecte���in�the�ERAN)�relies�on�re�resentations�of��usic-s��ntactic�re��ulari-ties�that�alrea�����exist�in�a�lon��-ter���e�or���for�at�(althou��h��usic-s��ntactic��rocessin���can��o��if���such�re�resentations).�That�is,�the�statistical��robabilities�that��a�e�u���usic-s��ntactic�re��ularities�are�not�learne����ithin�a�fe���o�ents,�an��� the� re�resentations�of� such� re��ularities� are� store��� in�a� lon��-ter���e�or���for�at�(as���escribe���above).�

With�re��ar��s�to�the�MMN,�it�is�i��ortant�to�not�confuse�the�on-line�estab-lish�ent�of�re��ularities��ith�lon��-ter��ex�erience�or�lon��-ter��re�resentations�that��i��ht�influence�the���eneration�of�the�MMN:�E.��.,��itch�infor�ation�can�be���eco��e����ith�hi��her�resolution�b���so�e��usical�ex�erts�(lea��in���to�a��hMMN�to�frequenc�����eviants�that�are�not���iscri�inable�for��ost�non-ex�erts;�Koelsch�et�al.�1999),�or�the���etection�of�a��hone�e�is�facilitate����hen�that��hone�e�is�a��rotot���e�of�one’s�lan��ua��e�(lea��in���to�a��hMMN�that�has�a�lar��er�a��litu��e�in�in��ivi��uals��ith�a�lon��-ter��re�resentation�of�a�certain��hone�e�co��are���to�in��ivi��uals��ho���o�not�have�such�a�re�resentation;�Näätänen�et�al.�1997;�Win�ler�et�al.�1999;�Ylinen�et�al.�2006).�In�this�re��ar��,�lon��-ter��ex�erience�has�clear�ef-fects�on�the��rocessin���of��h��sical�o����balls�(as�reflecte���in�the�MMN).�

Ho�ever,� in�all�of� these� stu��ies� (Koelsch�et� al.� 1999;�Näätänen�et� al.� 1997;�Win�ler�et�al.�1999;�Ylinen�et�al.�2006),�the���eneration�of�the�MMN��as���e�en-��ent�on�re�resentations�of�re��ularities�that��ere�extracte���on-line�fro��the�acous-tic�environ�ent:�For�exa��le,�in�the�classical�stu�����fro��Näätänen�et�al.�(1997),�

Page 234: Unconscious Memory Representations in Perception

� Unconscious��e�or���re�resentations� 223

the�stan��ar���sti�ulus��as�the��hone�e�/e/,�an���one�of�the���eviant�sti�uli��as�the��hone�e�/õ/,��hich�is�a��rotot���e�in�Estonian�(but�not�in�Finnish).�This���eviant�elicite���a�lar��er��hMMN�in�Estonians�than�in�Finnish�subjects,�reflectin���that�Es-tonians�have�a�lon��-ter��re�resentation�of�the��hone�e�/õ/�(an���that�Estonians��ere,� thus,��ore�sensitive�to���etect�this��hone�e).�Ho�ever,� the�re��ularit���for�this�ex�eri�ental�con��ition�(“/e/�is�the�stan��ar���an���/õ/�is�a���eviant”)��as�in��e-�en��ent�of�the�lon��-ter��re�resentation�of�the��hone�es,�an���this�re��ularit����as�establishe���on-line�b��� the�Estonian�subjects���urin��� the�ex�eri�ent� (an���coul���have�been�chan��e���easil���into�“/õ/�is�the�stan��ar���an���/e/�is�the���eviant”).�That�is,�the�statistical��robabilities�that��a�e�u��the�re��ularities�in�such�an�ex�eri�en-tal�con��ition�are�learne����ithin�a�fe���o�ents,�an���the�re�resentations�of�such�re��ularities�are�not�store���in�a�lon��-ter���e�or���for�at.�

With�re��ar��s�to�the��hMMN�an���the�afMMN,�Schrö��er�(2007)���escribes�four��rocesses�that�are�require���for�the�elicitation�of�an�MMN,��hich�are�relate���here�to�the��rocesses�un��erl��in���the���eneration�of�the�ERAN�(see�also�Fi��ure�4):�

(1)� Inco�in��� acoustic� in�ut� is� anal��ze��� in� �ulti�le� �a��s� resultin��� in� the�se�aration�of�soun���sources,�the�extraction�of�soun���features,�an���the�establish-�ent�of�re�resentations�of�au��itor���objects.�Basicall���the�sa�e��rocesses�are�a�require���for��usic-s��ntactic��rocessin��,�an���thus�for�the�elicitation�of�the�ERAN�(see�also�to��left�of�Fi��ure�4;�for�exce�tions�see�Wi���ann�et�al.�2004;�Schön�an���Besson�2005).�

(2)� Re��ularities� inherent� in� the� sequential� �resentation� of� ��iscrete� events�are���etecte���an���inte��rate���into�a��o��el�of�the�acoustic�environ�ent.�Si�ilarl��,�Win�ler�(2007)�states�that�MMN�can�onl���be�elicite����hen�soun��s�violate�so�e��reviousl�����etecte���inter-soun���relationshi�.�These�state�ents�nicel���illustrate�a�crucial���ifference�bet�een� the�co��nitive�o�erations�un��erl��in����usic-s��ntactic��rocessin���(as�reflecte���in�the�ERAN)�an����rocessin���of�acoustic�o����balls�(as�re-flecte���in�the�MMN):�As��entione���above,���urin���(�usic-)�s��ntactic��rocessin��,�re�resentations�of�re��ularities�alrea�����exist�in�a�lon��-ter���e�or���for�at�(si�i-larl���to�the��rocessin���of�s��ntactic�as�ects�of�lan��ua��e)�an�����eter�ine�auto�ati-call��,�or�unconsciousl��,�the��rocessin���of��usic-s��ntactic�infor�ation.�That�is,�the�re��ularities�the�selves���o�not�have�to�be���etecte��,�an���it�is�not�the�re��ularit���that�is�inte��rate���into�a��o��el�of�the�acoustic�environ�ent,�but�it�is�the�actual�soun���(or�chor��)�that�is�inte��rate���into�a�co��nitive�(structural)��o��el�accor��in���to�lon��-ter��re�resentations�of�re��ularities.�That�is,�the�re�resentations�of�(�usic-)�s��n-tactic�re��ularities�are�usuall���not�establishe���on-line,�an���the���are,��oreover,�not�necessaril���base���on�the�inter-soun���relationshi�s�of�the�acoustic�in�ut�(see�to��ri��ht�of�Fi��ure�4).�Note�that,���ue�to�its�relation�to�re�resentations�that�are�store���in�a� lon��-ter�� for�at,��usic-s��ntactic��rocessin��� is� intrinsicall��� connecte��� to�learnin���an����e�or��.�Unless�an�in��ivi��ual�is�for�all���traine���in��usic�theor��,�

Page 235: Unconscious Memory Representations in Perception

224� Stefan�Koelsch

these��e�or���re�resentations�are�unconscious�in�the�sense�that�the���are�i��licit,�an���that�the���cannot�(an���nee���not!)�be���escribe����ith��or��s,�or�conce�ts.�

It�is�also�i��ortant�to�note�that��usic-s��ntactic��rocessin���usuall���requires��rocessin���of�lon��-��istance���e�en��encies�at�a�level�of�co��lexit���ter�e����hrase�structure���ra��ar�(for�ex�lanation�of��hrase�structure���ra��ar�see�Fitch�an���Hauser�2004;�for�stu��ies�co��arin���neural�correlates�of��hrase�structure���ra�-�ar�an���finite�state���ra��ar�see,�e.��.,�Frie��erici�et�al.�2006;�O�itz�an���Frie��erici�2007).�For�instance,�as��entione���in�the�section�about��usical�s��ntax,��ovin���a�a���fro��a�tonal�centre�in��ajor-�inor�tonal�s�ace�creates�a�hierarchical�struc-ture�that�is�al�a��s�conclu��e���b���the�return�to�the�initial�tonal�centre�(�ith�the��ossibilit��� of� recursion).� B��� contrast,� �rocessin��� of� acoustic� o����balls� involves�

Figure 4. S��ste�atic�overvie��of��rocesses�require���to�elicit�MMN�an���ERAN�(see�text�for���etails).�Whereas�the�extraction�of�acoustic�features�is�i��entical�for�both�MMN�an���ERAN�(to��left�qua��rant),�MMN�an���ERAN���iffer��ith�re��ar���to�the�establish�ent�of�a��o��el�of�intersoun��-relationshi�s�(to��ri��ht�qua��rant):�In�the�case�of�the�MMN,�a��o��el�of�re��ularities�is�base���on�inter-soun���relationshi�s�that�are�extracte���on-line�fro��the�acoustic�environ�ent.�These��rocesses�are�lin�e���to�the�establish�ent�an����aintenance�of�re�resentations�of�the�acoustic�environ�ent,�an���thus�to�the��rocesses�of�au��itor���scene�anal��sis.�In�the�case�of�the�ERAN,�a��o��el�of�inter-soun���relationshi�s�is�built�base���on�re�resentations�of��usic-s��ntactic�re��ularities�that�alrea�����exist�in�a�lon��-ter���e�or���for�at.�The�botto��qua��rants�illustrate�that�the�neural�resources�for�the��re��ic-tion�of�subsequent�acoustic�events,�an���the�co��arison�of�ne��acoustic�infor�ation��ith�the��re��icte���soun��,��resu�abl���overla��stron��l���for�MMN�an���ERAN.�

Page 236: Unconscious Memory Representations in Perception

� Unconscious��e�or���re�resentations� 225

sequential��rocessin�����ui��e���b���local�or��anizational��rinci�les��ith�re��ularities�usuall���li�ite���to�nei��hbourin���units,�but�not�hierarchical��rocessin��.�

Interestin��l��,�the�abilit���to��rocess��hrase�structure���ra��ar�is�available�to�all�hu�ans,��hereas�non-hu�an��ri�ates�are�not�able�to��aster�such���ra��ars�(Fitch�an���Hauser�2004).�Thus,�it�is�hi��hl���li�el���that�onl���hu�ans�can�a��equatel����rocess��usic-s��ntactic�infor�ation�at�the��hrase�structure�level.�Au��itor���o����-balls,�b���contrast,�can�be���etecte���b���non-hu�an��a��als�such�as�cats�an����a-caques�(an���au��itor���o����balls�elicit�MMN-li�e�res�onses�in�these�ani�als;�for�a�brief�overvie��see�Näätänen�et�al.�2005,��.�26).�

(3)�Pre��ictions�about�forthco�in���au��itor���events�are���erive���fro��the��o��-el�(see�also�Win�ler�2007;�Garri��o�et�al.�2009).�This��rocess�is�si�ilar�(�resu�abl���at�least��artl���i��entical)�for�the�ERAN:�A�soun���ex�ectanc���(Koelsch�et�al.�2000)�for�follo�in����usical�events�(e.��.,�a�chor��)�is�establishe���base���on�the��revious�structural�context�an���the��no�le����e�about�the��ost�li�el���tone,�or�chor��,�to�fol-lo�.�As��entione���in�(2),�ho�ever,�in�the�case�of�the�MMN�the��re��ictions�are�base���on�re��ularities�that�are�establishe���on-line�base���on�the�inter-soun���rela-tionshi�s�of�the�acoustic�in�ut,��hereas�in�the�case�of�the�ERAN�the��re��ictions�are�base���on�re�resentations�(of��usic-s��ntactic�re��ularities)� that�alrea�����exist�in�a�lon��-ter���e�or���for�at.�In�other��or��s,�in�the�case�of�the��rocessin���of�au��itor���o����balls,�the��re��ictions�are�establishe���base���on�local�or��anizational��rinci�les,��hereas�in�the�case�of��usic-s��ntactic��rocessin��,�the��re��ictions�are�usuall���base���on�the��rocessin���of��hrase�structure���ra��ar�involvin����robabili-ties�for�the�transition�of�chor���functions��ithin�hierarchical�structures.�

To���ate�it�is�not��no�n�to��hich���e��ree�the��re��ictions�un��erl��in���the���en-eration�of�the�MMN�an���the�ERAN�are�establishe���in�the�sa�e�areas.�Because�the��re�otor�cortex�(PMC,�corres�on��in���to�Bro���ann’s�area�6)�has�been�i��licate���in�serial��re��iction�(Schubotz�2007),�it�is�li�el���that�PMC�serves�the�for�ation�of�both�(i)��re��ictions�un��erl��in����rocessin���of�au��itor���o����balls�an���(ii)��re��ic-tions�un��erl��in����rocessin���of��usic-s��ntactic� infor�ation.�Ho�ever,� it� is�also�li�el���that,�in�a����ition�to�such�overla�,��re��ictions�for�the�MMN�are�also���ener-ate���in�sensor��-relate���areas�(i.e.,�in�the�au��itor���cortex),�an���the��re��ictions�for�the�ERAN�(�erha�s�also�for�the�afMMN)�in�hetero-�o��al�areas�such�as�Broca’s�area�(BA�44/45;�see�botto��left�of�Fi��ure�4;�for�neural���enerators�of�the�ERAN�see�next�section).�

(4)�Re�resentations�of�the�inco�in���soun���an���the�soun����re��icte���b���the��o��el�are�co��are��.�For�the�ERAN,�this��rocess� is,�a��ain,��resu�abl���at� least��artl���the�sa�e�as�for�the�MMN�(see�botto��ri��ht�of�Fi��ure�4).�Ho�ever,�si�i-larl���to�(3)�it�is�un�no�n��hether�such��rocessin���co��rises��ri�aril���au��itor���areas�for�the�MMN�(�here�the�soun���re�resentation��i��ht�be��ore�concrete,�or�

Page 237: Unconscious Memory Representations in Perception

22�� Stefan�Koelsch

“sensor��”,���ue�to���irectl����rece��in���sti�uli�that�establishe���the�re��ularities),�an����ri�aril���frontal�areas�for�the�ERAN�(see�also�next�section�for�further���etails).�

In�a����ition,�Win�ler�(2007)�states�that�the��ri�ar���function�of�the�MMN-��eneratin����rocess�is�to��aintain�neuronal��o��els�un��erl��in���the���etection�an���se�aration�of�au��itor���objects.�This�also���ifferentiates�the��rocesses�un��erl��in���the�MMN�fro��those�un��erl��in����usic-s��ntactic��rocessin��,�because�s��ntactic��rocessin��� serves� the� co��utation� of� a� strin��� of� au��itor��� structural� ele�ents�that�–�in�their��hole�–�re�resent�a�for��that�conve��s��eanin����hich�can�be�un-��erstoo���b���a�listener�fa�iliar��ith�the�s��ntactic�re��ularities�(Koelsch�an���Siebel�2005;�Steinbeis�an���Koelsch�2008).�

�.5 Unconscious interactions between music- and language-syntactic processing

Durin���the�last���ears,�a�nu�ber�of�stu��ies�has�reveale���interactions�bet�een��u-sic-s��ntactic� an��� lan��ua��e-s��ntactic� �rocessin��� (Koelsch� et� al.� 2005;� Steinbeis�an���Koelsch�2008;�Slevc�et�al.�2009;�Fe��oren�o�et�al.�2009).�In�these�stu��ies,�chor���sequences��ere��resente���si�ultaneousl����ith�visuall����resente���sentences��hile��artici�ants��ere�as�e���to�focus�on�the�lan��ua��e-s��ntactic�infor�ation,�an���to�i��-nore�the��usic-s��ntactic�infor�ation.�Usin���EEG�an���chor���sequence��ara��i���s�si�ilar�to�those���escribe���in�Fi��ures�1,�t�o�stu��ies�sho�e���that�the�ERAN�elicite���b���irre��ular�chor��s�interacts��ith�the�left�anterior�ne��ativit���(LAN)�elicite���b���lin-��uistic�(�or�ho-s��ntactic)�violations�(Koelsch�et�al.�2005;�Steinbeis�an���Koelsch�2008):�The�LAN�elicite���b����or��s��as�re��uce����hen�the�irre��ular��or����as��re-sente���si�ultaneousl����ith�an�irre��ular�chor���(co��are���to��hen�the�irre��ular��or����as��resente����ith�a�re��ular�chor��).�In�the�stu�����fro��Koelsch�et�al.�(2005)�a�control�ex�eri�ent��as�con��ucte���in��hich�the�sa�e�sentences��ere��resente���si�ultaneousl����ith�sequences�of�sin��le�tones.�The�tone�sequences�en��e���either�on�a�stan��ar���tone�or�on�a�frequenc�����eviant.�The��hMMN�elicite���b���the�fre-quenc�����eviants���i���not� interact��ith� the�LAN�(in�contrast� to� the�ERAN),� in-��icatin��� that� the��rocessin���of�au��itor���o����balls� (as� reflecte��� in� the��hMMN)���oes�not�consu�e�resources�relate���to�s��ntactic��rocessin��.�Whether�the�afMMN�consu�es�such�resources�re�ains�to�be�investi��ate��.�Results�of�these�ERP�stu��ies�(Koelsch�et�al.�2005;�Steinbeis�an���Koelsch�2008)�in��icate�that�the�ERAN�reflects�s��ntactic��rocessin��,�rather� than���etection�an��� inte��ration�of� inter-soun���rela-tionshi�s�inherent�in�the�sequential��resentation�of���iscrete�events�into�a��o��el�of�the�acoustic�environ�ent.�In�other��or��s,�the�results�of�those�ERP�stu��ies�in-��icate� that� the� auto�atic� �rocessin��� of� �usic-s��ntactic� infor�ation� (base��� on�

Page 238: Unconscious Memory Representations in Perception

� Unconscious��e�or���re�resentations� 227

unconscious�lon��-ter���e�or���re�resentations),�but�not�the�auto�atic��rocess-in���of�acoustic�o����balls�(base���on�unconscious�sensor����e�or���re�resentations)�interacts��ith�lan��ua��e-s��ntactic��rocessin��.�

These�ERP�fin��in��s��ere�corroborate���b���behavioural�stu��ies:�In�a�stu�����fro��Slevc�et�al.�(2009)��artici�ants��erfor�e���a�self-�ace���rea��in���of�“��ar��en-�ath”�sentences�(that� is,�of�sentences��hich�have�a���ifferent�s��ntactic�structure� than�initiall��� ex�ecte��).� Wor��s� (�resente��� visuall��)� occurre��� si�ultaneousl��� �ith�chor��s�(�resente���au��itoril��).�When�a�s��ntacticall���unex�ecte����or���occurre���to��ether� �ith� a� �usic-s��ntacticall��� irre��ular� (out-of-�e��)� chor��,� �artici�ants�nee��e����ore�ti�e�to�rea���the��or���(that�is,��artici�ants�sho�e���stron��er���ar��en-�ath� effects).� No� such� interaction� bet�een� lan��ua��e-s��ntactic� an��� �usic-s��n-tactic��rocessin����as�observe����hen��or��s��ere�se�anticall���unex�ecte��,�nor��hen�the�chor����resente����ith�the�unex�ecte����or���ha���an�unex�ecte���ti�bre�(but��as�har�onicall���correct).�Si�ilar�results��ere�re�orte���b���Fe��oren�o�et�al.�(2009)�in�a�stu�����in��hich�sentences��ere�sun��.�Sentences��ere�either�subject-extracte���or�object-extracte���relative�clauses,�an���the�note�sun���on�the�critical��or���of�a�sentence��as�either�in-�e���or�out-of-�e��.�Partici�ants��ere�less�accurate�in�their�un��erstan��in���of�object-relate���extractions�co��are���to�subject-relate���extractions�(as�ex�ecte��).�I��ortantl��,�the���ifference�bet�een�the�co��rehen-sion�accuracies�of�these�t�o�sentence�t���es��as�lar��er��hen�the�critical��or���(the�last��or���of�a�relative�clause)��as�sun���on�an�out-of-�e���note.�No�such�interaction��as�observe����hen�the�critical��or����as�sun����ith���reater�lou��ness.�Thus,�both�of�these�stu��ies�(Slevc�et�al.�2009;�Fe��oren�o�et�al.�2009)�sho�e���that��usic-�an���lan��ua��e-s��ntactic��rocessin���s�ecificall���interact��ith�each�other,��resu�abl���because�the���both�rel���on�co��on��rocessin���resources.�

The�fin��in��s�of� the��entione���EEG-�an���behavioural� stu��ies�sho�in��� in-teractions� bet�een� lan��ua��e-� an��� �usic-s��ntactic� �rocessin��� have� been� cor-roborate���b���a�recent��atient�stu�����(Patel�et�al.�2008).�This�stu�����sho�e���that�in��ivi��uals��ith�Broca’s�a�hasia�also�sho��i��aire����usic-s��ntactic��rocessin���in�res�onse�to�out-of-�e���chor��s�occurrin���in�har�onic��ro��ressions�(note�that�all��atients�ha���Broca’s�a�hasia,�but�onl���so�e�of�the��ha���a�lesion�that�inclu��e���Broca’s�area).�

�.� Neural basis of music-syntactic processing

A� nu�ber� of� stu��ies� su����est� that� �usic-s��ntactic� �rocessin��� relies� on� neural�sources�locate���in�the��ars�o�ercularis�of�the�inferior�fronto-lateral�cortex�(cor-res�on��in��� to� inferior� Bro���ann’s� area� [BA]� 44),� �resu�abl��� �ith� a����itional�

Page 239: Unconscious Memory Representations in Perception

22�� Stefan�Koelsch

contributions�fro��the�ventrolateral��re�otor�cortex�an���the�anterior�su�erior�te��oral�����rus�(�lanu���olare;�Koelsch�2006).�A�stu������ith��a��netoence�ha-lo��ra�h���(Koelsch�2000)�usin���a�chor���sequence��ara��i�����ith�the�sti�uli���e-�icte���in�Fi��ure�1a�an���b,�re�orte���a���i�ole�solution�of�the�ERAN��ith�a�t�o-��i�ole��o��el,�the���i�oles�bein���locate���bilaterall���in�inferior�BA�44�(see�also�Maess�et�al.�2001,�an���Fi��ure�2��,�the���i�ole�stren��th��as�no�inall���stron��er�in�the�ri��ht�he�is�here,�but�this�he�is�heric���ifference��as�statisticall���not�si��nificant).�The��ain�frontal�contribution�to�the�ERAN�re�orte���in�that�stu�����sta��s�in�contrast�to�the��hMMN��hich�receives�its��ain�contributions�fro��neural�sources�locate����ithin�an���in�the�vicinit���of�the��ri�ar���au��itor���cortex,��ith�a����itional�(but�s�aller)�contributions�fro��frontal�cortical�areas�(Alho�et�al.�1996;�Alain�et�al.�1998;�Giar���et�al.�1990;�O�itz�et�al.�2002;�Liebenthal�et�al.�2003;�Molhol��et�al.�2005;�Rinne�et�al.�2005;�Schön�iesner�et�al.�2007;�for�a�revie��see�Deouell�2007).�Li�e�ise,�the��ain���enerators�of�the�afMMN�have�also�been�re�orte���to�be�lo-cate���in�the�te��oral�lobe�(Korz��u�ov�et�al.�2003).�That�is,��hereas�the��hMMN�(an���the�afMMN)�receives��ain�contributions�fro��te��oral�areas,�the�ERAN�a��ears�to�receive�its��ain�contributions�fro��frontal�areas.�

The�results�of�the�MEG�stu�����(Koelsch�2000)��ere�su��orte���b���functional�neuroi�a��in��� stu��ies� usin��� chor��� sequence� �ara��i���s� (Koelsch� et� al.� 2002;�Koelsch�et�al.�2005;�Till�ann�et�al.�2006),�“real”,��ol���honic��usic�(Janata�et�al.�2002a),�an����elo��ies�(Janata�et�al.�2002b).�These�stu��ies�sho�e���activations�of�inferior�fronto-lateral�cortex�at�coor��inates�hi��hl���si�ilar�to�those�re�orte���in�the�MEG�stu�����(Fi��ure�2e).�Particularl���the�functional��a��netic�resonance�i�a��in���(fMRI)�stu�����fro��Koelsch�et�al.�(2005)�su��orte���the�assu��tion�of�neural���enerators�of�the�ERAN�in�inferior�BA�44:�As��ill�be�re�orte���in��ore���etail�belo�,�the�ERAN�has�been�sho�n�to�be�lar��er�in��usicians�than�in�non-�usicians�(Koelsch�et�al.�2002),�an���in�the�fMRI�stu�����fro��Koelsch�et�al.�(2005)�effects�of��usical�trainin����ere�correlate����ith�activations�of�inferior�BA�44,�in�both�a��ults�an���chil��ren.�

Data�obtaine���fro���atients��ith�lesions�of�the�left�inferior�frontal�cortex�(left��ars�o�ercularis)�sho�e���that�the�scal����istribution�of�the�ERAN���iffers�fro��that�of�health���controls�(Sa��ler�2008),�su��ortin���the�assu��tion�that�neural�sourc-es�locate���in�the��ars�o�ercularis�are�involve���in�the�ERAN���eneration.�Moreover,���ata�recor��e���fro��intracranial���ri��-electro��es�fro���atients��ith�e�ile�s���i��en-tifie���t�o�ERAN�sources,�one�in�the�inferior�fronto-lateral�cortex,�an���one�in�the�su�erior�te��oral�����rus�(Sa��ler�2008;�the�latter�one��as�inconsistentl���locate���in�anterior,��i����le,�an����osterior�su�erior�te��oral�����rus).�

Further�su��ort�ste�s�fro��EEG�stu��ies� investi��atin���the�ERAN�an���the��hMMN� un��er� �ro�ofol� se��ation:� Whereas� the� �hMMN� is� stron��l��� re��uce��,�

Page 240: Unconscious Memory Representations in Perception

� Unconscious��e�or���re�resentations� 229

but�still�si��nificantl����resent�un��er���ee���ro�ofol�se��ation�(Mo��ifie���Observer’s�Assesse�ent�of�Alertness�an���Se��ation�Scale�level�2–3,��ean�Bis�ectral�In��ex�=�68),� the�ERAN�is�abolishe�����urin���this� level�of�se��ation�(Koelsch�et�al.�2006).�This�hi��hli��hts�the�i��ortance�of�the�frontal�cortex�for��usic-s��ntactic��rocess-in��,� because� �ro�ofol� se��ation� a��ears� to� affect� hetero�o��al� frontal� cortices�earlier,�an����ore�stron��l���than�uni�o��al�sensor���cortices�(Hein�e�et�al.�2004;�Hein�e�an���Koelsch�2005).�

An�EEG-stu�����fro��Ja�es�et�al.�(2008)�re�orte���a�localization�of�the��ain���enerators�of�an�ERAN��otential�in��e��ial�te��oral�areas�(su��ose��l���hi��o-ca��us� an��� a�������ala)� an��� the� insula,� but� this� source� anal��sis� a��ears� to� be�so�e�hat�questionable�because�(a)�conver��in���evi��ence�fro��a�nu�ber�of�EEG,�MEG,�an���fMRI�stu��ies�in��icates�that��ain�ERAN���enerators�are�locate���in�the�frontal�lobe,�an���(b)�no�control�ERP�co��onent�(such�as�the�P1)��as�localize���in�the�stu�����fro��Ja�es�et�al.�(2008).�

Finall��,�it�is�i��ortant�to�note�that�inferior�BA�44�(�hich�is�in�the�left�he�i-s�here�often�referre���to�as�Broca’s�area)��la��s�a�crucial�role�for�the��rocessin���of�s��ntactic�infor�ation���urin���lan��ua��e��erce�tion�(e.��.,�Frie��erici�2002),�as��ell�as�for�the�hierarchical��rocessin���of�action�sequences�(e.��.,�Koechlin�an���Jubault�2006),�an���for�the��rocessin���of�hierarchicall���or��anize����athe�atical�for�ulas�an���ter�ini�(Frie��rich�an���Frie��erici�2009).�Thus,��ith�re��ar���to�lan��ua��e-s��n-tactic��rocessin��,�the�neural�resources�for�the��rocessin���of��usical�an���lin��uistic�s��ntax�a��ear�to�overla��stron��l��,�an���this�vie��is��articularl���su��orte���b���the��entione���stu��ies�sho�in���interactions�bet�een��usic-s��ntactic�an���lan��ua��e-s��ntactic��rocessin���(Koelsch�et�al.�2005;�Steinbeis�an���Koelsch�2008;�Slevc�et�al.�2007).�On�a��ore�abstract�level,�it�is�hi��hl���li�el���that�Broca’s�area�is�involve���in�the��rocessin���of�hierarchicall���or��anize���sequences�in���eneral,�be�the����usical,�lin��uistic,�action-relate��,�or��athe�atical.�Interestin��l��,�the���ata�fro��the�stu��-ies�on��usic-s��ntactic��rocessin���sho��us�that�Broca’s�area�is�ca�able�of��rocess-in���hierarchical�infor�ation�even��hen�this�infor�ation�is�establishe���base���on�i��licit�(an���in�this�sense�unconscious)��e�or���re�resentations.�

Moreover,�5-��ear-ol���chil��ren��ith�s�ecific� lan��ua��e� i��air�ent�(charac-terize���b����ar�e�����ifficulties�in�lan��ua��e-s��ntactic��rocessin��)���o�not�sho��an�ERAN�(�hereas�chil��ren��ith�nor�al� lan��ua��e���evelo��ent���o;� Jentsch�e�et�al.�2008),�an���11-��ear-ol���chil��ren��ith��usical�trainin�����o�not�onl���sho��an�increase�of� the�ERAN�a��litu��e,�but�also�an�increase�of� the�a��litu��e�of� the�ELAN�(reflectin���lan��ua��e-s��ntactic��rocessin��,�Jentsch�e�et�al.�2005;�see�also�section�on���evelo��ent�belo�).�The�latter�fin��in����as�inter�rete���as�the�result�of�trainin���effects�in�the��usical���o�ain�on��rocesses�of�fast�an���auto�atic�s��ntac-tic�sequencin�����urin���the��erce�tion�of�lan��ua��e.�

Page 241: Unconscious Memory Representations in Perception

230� Stefan�Koelsch

�.7 A note on the lateralization of the ERAN

Althou��h�the�ERAN��as�si��nificantl���lateralize���in�so�e�stu��ies�(e.��.,�Koelsch�et�al.�2000,�2007;�Koelsch�an���Sa��ler�2007;�Koelsch�an���Jentsch�e�2008),� it��as�not�ri��ht-lateralize���in�a�fe��stu��ies�also�usin���chor��-sequence��ara��i���s�(Steinbeis�et�al.�2006;�Loui�et�al.�2005),�or��elo��ies�(Miran��a�an���Ull�an�2007).�An�i��ortant���ifference�bet�een�those�stu��ies�is�that�the�stu��ies�in��hich�the�ERAN��as�ri��ht-lateralize���ha���relativel���lar��e�nu�bers�of��artici�ants�(t�ent���or��ore;�Koelsch�et�al.�2007;�Koelsch�an���Sa��ler�2007;�Koelsch�an���Jentsch�e�2008),��hereas�stu��ies�in��hich�no�lateralization�of�the�ERAN��as�re�orte���have��ostl����easure���less�than�t�ent���subjects�(ei��hteen�in�the�stu�����fro��Loui�et�al.�2005;�ten�in�the�stu�����fro��Leino�et�al.�2007).�This���ifference�is�i��ortant�because� it� see�s�that� the�ERAN�is��ore�stron��l��� lateralize��� in��ales� than� in�fe�ales� (�ho�often�sho��a�rather�bilateral�ERAN,�Koelsch�et�al.�2003a);� thus,�a� relativel��� lar��e�nu�ber�of� subjects� is� require���until� the� lateralization�of� the�ERAN�reaches�statistical�si��nificance.�A����itional�factors�that��o��ulate�the�lat-eralization�of�the�ERAN��i��ht�inclu��e�the�salience�of�irre��ular�chor��s,�atten-tional�factors,�an���the�si��nal-to-noise�ratio�of�ERP���ata�(see�Koelsch�2009).�

Ho�ever,�as��entione���above,�a�nu�ber�of�functional�neuroi�a��in���stu��ies�sho�e���that,�on�avera��e,�the�neural�activit���un��erl��in���the��rocessin���of��usic-s��ntacticall���irre��ular�chor��s�has�a�ri��ht-he�is�heric��ei��htin���(see�also�Koelsch�2009).�Thus,�even�if�the�EEG�effect�is�so�eti�es�not�si��nificantl���lateralize��,�it�is�reasonable�to�assu�e�that�the�neural���enerators�of�the�ERAN�are�activate����ore�stron��l���in�the�ri��ht�than�in�the�left�he�is�here.�

�.� Automaticity: Unconscious processing of musical syntax

So�far,�several�ERP�stu��ies�have�use���the�ERAN�to�investi��ate�the�auto�aticit���of��usic-s��ntactic��rocessin��.�The�ERAN�has�been�observe����hile��artici�ants��la���a�vi��eo���a�e�(Koelsch�et�al.�2001),�rea���a�self-selecte���boo��(Koelsch�et�al.�2002b),�or��erfor��a�hi��hl���attention-��e�an��in���rea��in���co��rehension�tas��(Loui�et�al.�2005).� In� the� latter�stu����,��artici�ants��erfor�e���the�rea��in���tas���hile�i��norin���all�chor���sequences,�or�the���atten��e���to�the�chor���sequences�an�����etecte���chor��s��hich���eviate���in�their�soun���intensit���fro��stan��ar���chor��s.�These�con��itions�enable���the�authors�to�co��are�the��rocessin���of�tas�-irrele-vant�irre��ular�chor��s�un��er�an�atten���con��ition�(intensit�����etection�tas�)�an���an�i��nore�con��ition�(rea��in���co��rehension�tas�).�Results�sho�e���that�an�ERAN��as�elicite���in�both�con��itions�an���that�the�a��litu��e�of�the�ERAN��as�re��uce���(but�still�si��nificant)��hen�the��usical�sti�ulus��as�i��nore���(Fi��ure�5a;�because�

Page 242: Unconscious Memory Representations in Perception

� Unconscious��e�or���re�resentations� 231

the�ERAN��as�not�si��nificantl���lateralize��,�it��as���enote���as�early anterior nega-tivity�b���the�authors).�

Another�recent�stu�����(Mai��hof�an���Koelsch�2008)�sho�e���that�the�neural��echanis�s�un��erl��in��� the��rocessin���of��usic-s��ntactic� infor�ation� (as� re-flecte��� in�the�ERAN)�are�active�even��hen��artici�ants�selectivel���atten���to�a�s�eech�sti�ulus.�In�that�stu����,�s�eech�an����usic�sti�uli��ere��resente���si�ul-taneousl��� fro����ifferent� locations� (20°� an���340°� in� the�azi�uthal��lane).�The�ERAN� �as� elicite��� even� �hen� �artici�ants� selectivel��� atten��e��� to� the� s�eech�sti�ulus,� but� its� a��litu��e� �as� si��nificantl��� ��ecrease��� co��are��� to� the� con-��ition� in� �hich� �artici�ants� listene��� to� �usic� onl��.� The� fin��in��s� of� the� latter�t�o�stu��ies�(Loui�et�al.�2005;�Mai��hof�an���Koelsch�2008)�sho��that�the�neural��echanis�s�un��erl��in���the��rocessin���of�har�onic�structure�o�erate�in�the�ab-sence�of�attention,�but�that�the���can�be�clearl����o��ulate���b�����ifferent�attentional���e�an��s.�That� is,� in� the�conscious� in��ivi��ual,��usical� s��ntax� is��rocesse���b���

Figure 5. (a)�sho�s���ifference�ERPs�(tonic�subtracte���fro��Nea�olitan�chor��s)�elicite����hen�attention��as�focusse���on�the��usical�sti�ulus�(��re���line),�an����hen�attention��as�focusse���on�a�rea��in���co��rehension�tas��(blac��line).�The�E(R)AN�(in��icate���b���the�arro�)�clearl�����iffere���bet�een�con��itions,�bein���s�aller�in�the�unatten��-con��i-tion.�(A��a�te����ith��er�ission�fro��Loui�et�al.�2005.)�(b)�sho�s���ifference�ERPs�(tonic�subtracte���fro��Nea�olitan�chor��s)�elicite���in��usicians�(soli���line)�an���non-�usicians�(��otte���line).�The�ERAN�(arro�)�clearl�����iffere���bet�een���rou�s,�bein���s�aller�in�the���rou��of�non-�usicians.�

Page 243: Unconscious Memory Representations in Perception

232� Stefan�Koelsch

unconscious��echanis�s�in�the�sense�that�(a)�the�listener���oes�not�have�to��a���attention� to� the��usic,� an��� (b)� the�brains�of� listeners� re��ister��usic-s��ntactic�irre��ularities�even��hen�these�irre��ularities�are�not�even�consciousl���notice���b���the�listener.�A�recent�stu�����(Koelsch�et�al.�2006)�sho�e���that,�in�the�unconscious�in��ivi��ual,��usic-s��ntactic��rocessin���is��resu�abl���abolishe��:�That�stu�����re-�orte���that�no�ERAN��as�observable�un��er���ee���ro�ofol�se��ation�(�here��ar-tici�ants��ere� in�a� state� si�ilar� to�natural� slee�),� in�contrast� to� the��hMMN,��hich��as� stron��l��� re��uce��,�but� still� si��nificantl����resent���urin��� this� level�of�se��ation.�This�su����ests�that�the�elicitation�of�the�ERAN�requires�a���ifferent�state�of�consciousness�on�the��art�of�the�listeners�than�the��hMMN�(see�also�Hein�e�et�al.�2004;�Hein�e�an���Koelsch�2005).

With�re��ar���to�the�MMN,�several�stu��ies�have�sho�n�that�the�MMN�a�-�litu��e� can� be� re��uce��� in� so�e� cases� b��� attentional� factors� (for� a� revie�� see�Suss�an� 2007).� Ho�ever,� it� has� been� ar��ue��� that� such� �o��ulations� coul��� be�attribute���to�effects�of�attention�on�the�for�ation�of�re�resentations�for�stan��ar���sti�uli,�rather�than�to�the���eviant���etection��rocess�(Suss�an�2007),�an���that�MMN� is� lar��el���unaffecte���b���attentional��o��ulations� (Gri���an���Schrö��er�2005;�Suss�an�et�al.�2004;�Go�es�et�al.�2000).�That�is,� the�MMN�see�s�to�be�consi��erabl����ore�resistant�a��ainst�attentional��o��ulations�than�the�ERAN.

�.9 Effects of musical training

Li�e�the�MMN,�the�ERAN�can�be��o��ulate���b���both�lon��-ter��an���short-ter��trainin��.�Effects�of��usical�trainin���have�been�re�orte���for�the�MMN��ith�re-��ar���to�the��rocessin���of�te��oral�structure�(Rüsseler�2001),�the��rocessin���of�abstract�features�such�as�interval�an���contour�chan��es�(Tervanie�i�et�al.�2001;�Fujio�a�et�al.�2004),�as��ell�as�for�the��rocessin���of��itch�(Koelsch�et�al.�1999).�In� all� these� stu��ies,� the� MMN� �as� lar��er� in� in��ivi��uals� �ith� for�al� �usical�trainin��� (“�usicians”)� than� in� in��ivi��uals� �ithout� such� trainin��� (“non-�usi-cians”).�With�re��ar���to�the�ERAN,�four�stu��ies�have�so�far�investi��ate���effects�of��usical�lon��-ter��trainin���on�the�ERAN�(Koelsch�et�al.�2002,�2007;�Koelsch�an���Sa��ler�2007;�Koelsch�an���Jentsch�e�2008).�These�stu��ies�have�sho�n�that�the�ERAN�is�lar��er�in��usicians�(Fi��ure�5b;�Koelsch�et�al.�2002),�an���in�a�ateur��usicians�(Koelsch�et�al.�2007)�co��are���to�non-�usicians.�In�the�latter�stu����,�the���ifference�bet�een���rou�s��as�just�above�the�threshol���of�statistical�si��nifi-cance,�an���t�o�recent�stu��ies�re�orte���no�inall���lar��er�ERAN�a��litu��e�val-ues�for��usicians�(co��are���to�non-�usicians,�Koelsch�an���Sa��ler�2007)�an���

Page 244: Unconscious Memory Representations in Perception

� Unconscious��e�or���re�resentations� 233

a�ateur��usicians�(co��are���to�non-�usicians,�Koelsch�an���Jentsch�e�2008),�althou��h� the� ��rou�� ��ifferences� ��i��� not� reach� statistical� si��nificance� in� these�stu��ies.�Des�ite�this�lac��of�statistical�si��nificance�it�is�re�ar�able�that�all�stu��ies�that�have�so�far�investi��ate���trainin���effects�on�the�ERAN�observe���lar��er�ERAN�a��litu��es�for��usicall���traine���in��ivi��uals�co��are���to�non-�usicians.�Cor-roboratin���these�ERP���ata,�si��nificant�effects�of��usical�trainin���on�the��rocess-in���of��usic-s��ntactic�irre��ularities�have�also�been�sho�n�in�fMRI�ex�eri�ents�for�both�a��ults�an���11���ear-ol���chil��ren�(Koelsch�et�al.�2005b).�

The�evi��ence�fro��the��entione���stu��ies�in��icates�that�the�effects�of��usi-cal�lon��-ter��trainin���on�the�ERAN�are�s�all,�but�reliable�an���consistent�across�stu��ies.�This�is�in�line��ith�behavioural�stu��ies�sho�in���that��usicians�res�on���faster,� an��� �ore� accuratel��� to� �usic-structural� irre��ularities� than� non-�usi-cians�(e.��.,�Bi��an���et�al.�1999),�an����ith�ERP�stu��ies�on�the��rocessin���of��usi-cal� structure� sho�in��� effects� of� �usical� trainin��� on� the� ��eneration� of� the� P3�usin���chor��s�(Re��nault�et�al.�2001),�or�the�elicitation�of�a�late��ositive�co��onent�(LPC)�usin����elo��ies�(Besson�an���Faita�1995;�see�also�Schön�et�al.�2004;�Ma��ne�et�al.�2006;�Moreno�an���Besson�2006).�The�ERAN�is��resu�abl���lar��er�in��usi-cians�because��usicians�have�(as�an�effect�of�the��usical�trainin��)��ore�s�ecific�re�resentations�of��usic-s��ntactic�re��ularities�an���are,�therefore,��ore�sensitive�to�the�violation�of�these�re��ularities�(for�effects�of��usical�trainin���in�chil��ren�see�next�section).�

With�re��ar��s�to�short-ter��effects�on��usic-s��ntactic��rocessin��,�a�recent�ex�eri�ent��resente���t�o�sequence�t���es�(one�en��in���on�a�re��ular�tonic�chor��,�the�other�one�en��in���on�an�irre��ular�su�ertonic)�for�a��roxi�atel���t�o�hours�(Koelsch�an���Jentsch�e�2008;�subjects��ere��atchin���a�silent��ovie��ith�sub-titles).� The� ��ata� sho�e��� that� �usic-s��ntacticall��� irre��ular� chor��s� elicite��� an�ERAN,�an���that� the�a��litu��e�of� the�ERAN���ecrease���over�the�course�of� the�ex�eri�ental� session.� These� results,� thus,� in��icate��� that� neural� �echanis�s�un��erl��in��� the��rocessin���of��usic-s��ntactic� infor�ation�can�be��o��ifie���b���short-ter���usical�ex�erience.�Interestin��l��,�althou��h�the�ERAN�a��litu��e��as�si��nificantl���re��uce��,�it��as�still��resent�at�the�en���of�the�ex�eri�ent,�su����estin���that�co��nitive�re�resentations�of�basic��usic-s��ntactic�re��ularities�are�re�ar�-abl���stable,�an���cannot�easil���be��o��ifie��.�This�notion��as�corroborate���b���a�recent�stu�����(Carrión�an���Bl���2008)�sho�in���that�the�a��litu��e�of�the�ERAN�to�irre��ular�chor��s���oes�not�increase��hen��artici�ants�un��er��o�a�trainin���session�in��hich�the���are��resente����ith�ei��ht��-four�chor���trainin���sequences�that�en��-e���on�s��ntacticall���correct�chor���sequences.�Notabl��,�in�contrast�to�the�ERAN,�the�P3b�a��litu��e�increase���in�that�stu�����as�an�effect�of�trainin��.�

Page 245: Unconscious Memory Representations in Perception

234� Stefan�Koelsch

�.10 Development

The���oun��est�in��ivi��uals�in��ho���usic-s��ntactic��rocessin���has�been�investi-��ate���so�far��ith�ERPs��ere,�to������no�le����e,�4��onth-ol���babies.�These�babies���i���not�to�sho��an�ERAN�(un�ublishe�����ata�fro��our���rou�,�irre��ular�chor��s��ere�Nea�olitan�chor��s;�so�e�of�the�babies��ere�aslee����urin���the�ex�eri�ent).�In�2.5���ear-ol���chil��ren�(30��onths)��e�observe���an�ERAN�to�su�ertonics�as��ell�as�Nea�olitan�chor��s�(un�ublishe�����ata�fro��our���rou�).�In�this�a��e���rou�,�the�ERAN��as�quite�s�all,�su����estin���that�the���evelo��ent�of�the�neural��ech-anis�s�un��erl��in���the���eneration�of�the�ERAN�co��ence�aroun��,�or�not�lon���before�this�a��e�(for�behavioural�stu��ies�on�the���evelo��ent�of��usic-s��ntactic��rocessin���see,�e.��.,�Schellenber���et�al.�2005).�

B��� contrast,� MMN-li�e� res�onses� can� be� recor��e��� even� in� the� fetus��(Dra��anova� et� al.� 2005;� Huotilainen� et� al.� 2005),� an��� a� nu�ber� of� stu��ies� has�sho�n� MMN-li�e� ��iscri�inative� res�onses� in� ne�borns� (althou��h� so�e-ti�es��ith��ositive��olarit��;�Ruusuvirta�et�al.�2003,�2004;�Win�ler�et�al.�2003;��Maurer�et�al.�2003;�Stefanics�et�al.�2007)�to�both��h��sical���eviants�(e.��.,�Alho�et�al.�1990;�Cheour�et�al.�2002a,�2002b;�Kushneren�o�et�al.�2007;�Win�ler�et�al.�2003;�Stefanics�et�al.�2007)�an���abstract�feature���eviants�(Ruusuvirta�et�al.�2003,�2004;�Carral�et�al.�2005).�Cheour�et�al.�(2000)�re�orte���that,�in�so�e�ex�eri�ents,�the�a��litu��es�of�such�ERP�res�onses�are�onl���sli��htl���s�aller� in�infants�than�the�MMN�usuall���re�orte��� in�school-a��e�chil��ren�(but�see�also,�e.��.,�Maurer�et�al.�2003;�Kushneren�o�et�al.�2007;�Frie��erici�2005,�for���ifferences).�With�re��ar���to��itch��erce�tion,�H�en�et�al.�(2009)�re�orte���that�alrea�����neonates�can���eneral-ize��itch�across���ifferent� ti�bres.�Moreover,� reliable��itch��erce�tion�has�been�sho�n�in�neonates�for�co��lex�har�onic�tones�(Ce�oniene�et�al.�2002),�s�eech�soun��s�(Kujala�et�al.�2004),�environ�ental�soun��s�(Sa�beth�et�al.�2006),�an���even�noise�(Kushneren�o�et�al.�2007).�Neonates�are�less�sensitive�than�a��ults,�ho�ever,�for�chan��es�in�the��itch�hei��ht�of�sinusoi��al�tones�(Novits�i�et�al.�2007).�

The�fin��in��s�that�MMN-li�e�res�onses�can�be�recor��e���in�the�fetus�an���in�ne�born� infants�su��ort� the�notion�that� the���eneration�of�such���iscri�inative�res�onses�is�base���on�the�(innate)�ca�abilit���to�establish�re�resentations�of�inter-soun���re��ularities�that�are�extracte���on-line�fro��the�acoustic�environ�ent�(an���the�innate�ca�abilit���to��erfor��au��itor���scene�anal��sis),��hereas�the���eneration�of�the�ERAN�requires�re�resentations�of��usical�re��ularities�that�first�have�to�be�learne���throu��h�listenin���ex�erience,�involvin���the���etection�of�re��ularities�(i.e.�statistical��robabilities)�un��erl��in��,�e.��.,�the�succession�of�har�onic�functions.�

Chil��ren�at� the�a��e�of�5���ears�sho��a�clear�ERAN,�but��ith� lon��er� latenc���than�a��ults�(aroun���230–240��s;�Jentsch�e�et�al.�2008,�in�that�stu�����the�ERAN��as�elicite���b���su�ertonics).�Si�ilar�results��ere�obtaine���in�another�stu�����usin���

Page 246: Unconscious Memory Representations in Perception

� Unconscious��e�or���re�resentations� 235

Nea�olitans�as�irre��ular�chor��s�(Koelsch�et�al.�2003).�It�is�not��no�n��hether�the�lon��er�latenc���in�5-��ear-ol��s�(co��are���to�a��ults)�is���ue�to�neuro-anato�ical���if-ferences�(such�as�fe�er����elinate���axons),�or���ue�to�less�s�ecific�re�resentations�of��usic-s��ntactic�re��ularities�(or�both).�

At� the�a��e�of�nine,� the�ERAN�a��ears� to�be�ver��� si�ilar� to� the�ERAN�of�a��ults.�In�a�recent�stu����,�9-��ear-ol��s��ith��usical�trainin���sho�e���a�lar��er�ERAN�than�chil��ren��ithout��usical�trainin���(un�ublishe�����ata�fro��our���rou�),�an���the�latenc���of�the�ERAN��as�aroun���200��s,�in�both�chil��ren��ith�an����ithout��usical�trainin��,�thus�still�bein���lon��er�than�in�ol��er�chil��ren�an���a��ults.�With�fMRI,�it��as�observe���that�chil��ren�at�the�a��e�of�10�sho��an�activation��attern�in�the�ri��ht�he�is�here�that�is�stron��l���re�iniscent�to�that�of�a��ults�(�ith�clear�activations�of�inferior�frontolateral�cortex�elicite���b���Nea�olitan�chor��s;�Koelsch�et�al.�2005b).�In�this�stu����,�chil��ren�also�sho�e���an�effect�of��usical�trainin��,�notabl���a�stron��er�activation�of�the�ri��ht��ars�o�ercularis�in��usicall���traine���chil��ren�(as�in�a��ults,�see�above).�

In�11-��ear-ol��s,�the�ERAN�has�a�latenc���of�aroun���180��s,�re��ar��less�of��u-sical� trainin��,� an��� is� �racticall��� in��istin��uishable� fro�� the� ERAN� observe��� in�a��ults�(Jentsch�e�et�al.�2005).�As�in�9-��ear-ol��s,�11���ear-ol���chil��ren��ith��usical�trainin���sho��a�lar��er�ERAN�than�chil��ren��ithout��usical�trainin���(Jentsch�e�et�al.�2005).�

With�re��ar���to�its�scal����istribution,��e��reviousl���re�orte���that�5���ear-ol�����irls�sho�e���a�bilateral�ERAN,��hereas�the�ERAN��as�rather�left-lateralize���in�bo��s�(Koelsch,�Gross�ann�et�al.�2005;�irre��ular�chor��s��ere�Nea�olitans).�Ho�-ever,�in�another�stu������ith�5-��ear-ol��s�(Jentsch�e�et�al.�2008;�irre��ular�chor��s��ere�su�ertonics)�no�si��nificant���en��er���ifference��as�observe��,�an���no�inall���the�ERAN��as�even��ore�ri��ht-lateralize���in�bo��s,�an����ore�left-lateralize���in���irls.�Thus,��hen�inter�retin�����ata�obtaine���fro��chil��ren,���en��er���ifferences�in�scal����istribution�shoul���be�treate����ith�caution.�

Interestin��l��,�it�is�li�el���that,��articularl�����urin���earl���chil��hoo��,�the�MMN�s��ste��is�of�fun��a�ental�i��ortance�for��usic-s��ntactic��rocessin��:�MMN�is�inextricabl���lin�e���to�the�establish�ent�an����aintenance�of�re�resentations�of�the� acoustic� environ�ent,� an��� thus� to� the� �rocesses� of� au��itor��� scene� anal��-sis.�The��ain���eter�inants�of�MMN�co��rise�the�stan��ar���for�ation��rocess�(because���eviance���etection� is� reliant�on� the�stan��ar���re�resentations�hel��� in�sensor����e�or��),���etection�an���se�aration�of�au��itor���objects,�an���the�or��ani-zation�of�sequential�soun��s�in��e�or��.�These��rocesses�are�in��is�ensable�for�the�establish�ent�of��usic-s��ntactic��rocessin��,�e.��.��hen�har�onies�are��erceive����ithin�chor����ro��ressions,�an����hen�the�re�eate���ex�osure�to�chor����ro��res-sions� lea��s� to� the� extraction� an��� �e�orization� of� statistical� �robabilities� for�chor��-� or� soun��-transitions.� In� a����ition,� because� �usic-s��ntactic� irre��ularit���

Page 247: Unconscious Memory Representations in Perception

23�� Stefan�Koelsch

an���har�onic���istance�is�often�relate���to�acoustic���eviance�(see�section�about�functional�si��nificance�of�the�ERAN),�the�acoustic���eviance���etection��echa-nis���roliferates�so�eti�es�infor�ation�about�the�irre��ularit���(i.e.,�unex�ect-e��ness)�of� chor��-functions,� an����erha�s�even� the�har�onic���istance�bet�een�so�e�chor��s.�Such�infor�ation�ai��s�the���etection�of��usic-s��ntactic�re��ulari-ties,�an���the�buil��-u��of�a�structural��o��el.� I��ortantl��,��hen��rocessin���an�acousticall��� ��eviant� �usic-s��ntactic� irre��ularit��,� MMN-relate��� �rocesses� also���ra��attention�to��usic.�

�.11 Conclusions

In� su��ar��,� �rocessin��� of� �usic-s��ntacticall��� irre��ular� infor�ation� is� often�reflecte��� electricall��� in� an� earl��� ri��ht� anterior� ne��ativit��� (ERAN).� The� ERAN�rese�bles�the�MMN��ith�re��ar���to�a�nu�ber�of��ro�erties,��articularl����olarit��,�scal����istribution,�ti�e�course,�an���sensitivit���to�acoustic�events�that��is�atch��ith�a��rece��in���sequence�of�acoustic�events,�an���sensitivit���to��usical�train-in��.�Therefore,�the�ERAN�has��reviousl���also�been�referre���to�as��usic-s��ntac-tic�MMN.�In�co��nitive�ter�s,�the�si�ilarities�bet�een�both�MMN�an���ERAN�co��rise�the�extraction�of�acoustic�features�require���to�elicit�both�ERPs�(�hich�is� i��entical� for� both� ERPs),� the� �re��iction� of� subsequent� acoustic� events,� an���the�co��arison�of�ne��acoustic�infor�ation��ith�a��re��icte���soun��,��rocesses��hich��resu�abl���overla��stron��l���for�MMN�an���ERAN.�

Ho�ever,�there�are�also���ifferences�bet�een��usic-s��ntactic��rocessin���an���the��rocessin���of�au��itor���o����balls,�the��ost�critical�bein���that�the��rocessin���of�au��itor���o����balls�(as�reflecte���in��hMMN�an���afMMN)�is�base���on�a��o��el�of�re��ularities�that�is�establish�ent�base���on�intersoun��-relationshi�s��hich�are�extracte���on-line�fro��the�acoustic�environ�ent.�B���contrast,��usic-s��ntactic��rocessin���(as�reflecte���in�the�ERAN)�is�base���on�a�structural��o��el��hich�is�es-tablishe����ith�reference�to�re�resentations�of�s��ntactic�re��ularities�that�alrea�����exist�in�a�lon��-ter���e�or���for�at.�In other words, in the case of the processing of auditory oddballs, the predictions are established based on local organizational principles, whereas in the case of music-syntactic processing, the predictions are usually based on the processing of phrase structure grammar involving long-dis-tance dependencies and hierarchical processing.� That� is,� the� re�resentations� of�re��ularities� buil��in��� the� structural� �o��el� of� the� acoustic� environ�ent� are� in�the�case�of�the�MMN��ore�sensor��,�an���in�the�case�of�the�ERAN��ore�co��nitive�in�nature.�It�is��erha�s�this���ifference�bet�een�ERAN�an���MMN��hich�lea��s�to�the���ifferent�to�o��ra�hies�of�neural�resources�un��erl��in���the���eneration�of�both�

Page 248: Unconscious Memory Representations in Perception

� Unconscious��e�or���re�resentations� 237

co��onents,��ith� the�ERAN�usuall��� sho�in����ore� frontal�an��� less� te��oral�lobe�involve�ent�than�the�MMN.�

Notabl��,�MMN�is�inextricabl���lin�e���to�the�establish�ent�an����aintenance�of� re�resentations� of� the� acoustic� environ�ent,� an��� thus� to� the� �rocesses� of�au��itor��� scene� anal��sis.� These� �rocesses� are� in��is�ensable� for� the� acquisition�of� re�resentations� of� �usic-s��ntactic� re��ularities� ��urin��� earl��� chil��hoo��,� e.��.��hen�the�re�eate���ex�osure�to�chor����ro��ressions� lea��s�to�the�extraction�an����e�orization�of�statistical��robabilities�for�chor��-�or�soun��-transitions.�Thus,�the��echanis�s�require���for�the�MNN�also�re�resent�the�foun��ation�for��usic-s��ntactic��rocessin��.�

Acknowledgements

I�than��Erich�Schrö��er,�Matthe��Woolhouse,�Aniru����h�Patel,�Barbara�Till�ann,�Istvan�Win�ler,�El��se�Suss�an,�Daniela�Sa��ler,�Ni�o�Steinbeis,�an���Sebastian�Jentsch�e�for�hel�ful���iscussion.�

Note

* A� shortene��� version� of� this� cha�ter� has� been� �ublishe��� as� article� in� Ps��cho�h��siolo�����(Koelsch�2009).�

References

Alain,�C.,�Woo��s,�D.�L.�an���Kni��ht,�R.�T.�(1998).�A���istribute���cortical�net�or��for�au��itor���sensor����e�or���in�hu�ans.�Brain Research,�812,�23–37.�

Alho,�K.,�Tervanie�i,�M.,�Huotilainen,�M.,�Lavi�ainen,�J.,�Tiitinen,�H.,�Il�onie�i,�R.�J.,�Knu-utila,� J.�an���Näätänen,�R.�(1996).�Processin���of�co��lex�soun��s� in�the�hu�an�au��itor���cortex�as�reveale���b����a��netic�brain�res�onses.�Psychophysiology,�33,�369–375.�

Besson,�M.�an���Faita,�F.�(1995).�An�event-relate����otential�(ERP)�stu�����of��usical�ex�ectanc��:�Co��arison�of��usicians��ith�non-�usicians.�Journal of Experimantal Psychology: Hu-man Perception and Performance,�21,�1278–1296.�

Bharucha,� J.�an���Kru�hansl,�C.� (1983).�The�re�resentation�of�har�onic�structure� in��usic:�Hierarchies�of�stabilit���as�a�function�of�context.�Cognition,�13,�63–102.�

Bi��an��,� E.,� Ma��urell,� F.,� Till�ann,� B.� an��� Pineau,� M.� (1999).� Effect� of� ��lobal� structure� an���te��oral�or��anization�on�chor����rocessin��.�Journal of Experimental Psychology: Human Perception and Performance,�25,�184–197.�

Brattico,�E.,�Tervanie�i,�M.,�Näätänen,�R.�an���Peretz,�I.�(2006).�Musical�scale��ro�erties�are�auto�aticall����rocesse���in�the�hu�an�au��itor���cortex.�Brain Research,�1117,�162–174.�

Page 249: Unconscious Memory Representations in Perception

23�� Stefan�Koelsch

Bro�n,�H.,�Butler,�D.�an���Jones,�M.�R.�(1994).�Musical�an���te��oral�influences�on��e�����iscov-er��.�Music Perception,�11,�371–391.�

Carral,�V.,�Huotilainen,�M.,�Ruusuvirta,�T.,�Fell�an,�V.,�Näätänen,�R.�an���Escera,�C.� (2005).�A� �in��� of� au��itor��� ‘�ri�itive� intelli��ence’� alrea����� �resent� at� birth.� European Journal of Neuroscience,�21,�3201–3204.�

Carrión,�R.�E.�an���Bl��,�B.�M.�(2008).�The�effects�of�learnin���on�event-relate����otential�correlates�of��usical�ex�ectanc��.�Psychophysiology,�45,�759–775.�

Ce�oniene,� R.,� Kushneren�o,� E.,� Fell�an� V.,� Renlun��,� M.,� Suo�inen,� K.� an��� Näätänen,� R.�(2002).�Event-relate����otential�features�in��exin���central�au��itor�����iscri�ination�b���ne�-borns.�Brain Research – Cognitive Brain Research,�13,�101–113.�

Cheour,�M.,�Ce�oniene,�R.,�Le��anen,�P.,�Alho,�K.,�Kujala,�T.�an���Renlun��,�M.�(2002a).�The�au��itor���sensor����e�or���trace���eca��s�ra�i��l���in�ne�borns.�Scandinavian Journal of Psy-chology,�43,�33–39.

Cheour,�M.,�Kushneren�o,�E.,�Ce�oniene,�R.,�Fell�an,�V.�an���Naatanen,�R.�(2002b).�Electric�brain�res�onses�obtaine���fro��ne�born�infants�to�chan��es�in���uration�in�co��lex�har-�onic�tones.�Developmental Neurophysiology,�22,�471–479.�

Cheour,�M.,�Le��anen,�P.�H.�an���Kraus,�N.�(2000).�Mis�atch�ne��ativit���(MMN)�as�a�tool�for�investi��atin���au��itor�����iscri�ination�an���sensor����e�or���in�infants�an���chil��ren.�Clini-cal Neurophysiology,�111,�4–16.

Deouell,�L.�Y.�(2007).�The�Frontal�Generator�of�the�Mis�atch�Ne��ativit���Revisite��.�Journal of Psychophysiology,�21,�147–160.�

Fe��oren�o,�E.,�Patel,�A.�D.,�Casasanto,�D.,�Wina�er,�J.�an���Gibson,�E.�(2009).�Structural�inte-��ration�in�lan��ua��e�an����usic:�Evi��ence�for�a�share���s��ste�.�Memory and Cognition,�37,�1–9.�

Fitch,�W.�T.�an���Hauser,�M.�D.�(2004).�Co��utational�constraints�on�s��ntactic��rocessin���in�a�nonhu�an��ri�ate.�Science,�303,�377–380.�

Frie��erici,�A.�D.�(2002).�To�ar��s�a�neural�basis�of�au��itor���sentence��rocessin��.�Trends in Cog-nitive Sciences,�6,�78–84.�

Frie��erici,�A.�D.�(2005).�Neuro�h��siolo��ical��ar�ers�of�earl���lan��ua��e�acquisition:�Fro��s��l-lables�to�sentences.�Trends in Cognitive Sciences,�9,�481–488.

Frie��erici,�A.�D.,�Bahl�ann,�J.,�Hei�,�S.,�Schubotz,�R.�I.�an���An�an��er,�A.�(2006).�The�brain���ifferentiates�hu�an�an���non-hu�an���ra��ars:�Functional� localization�an���structural�connectivit��.�Proceedings of National Academy of Sciences,�103,�2458–2463.�

Frie��erici,�A.�D.,�Frie��rich,�M.�an���Weber,�C.�(2002).�Neural��anifestation�of�co��nitive�an����reco��nitive��is�atch���etection�in�earl���infanc��.�Neuroreport,�13,�1251–1254.�

Frie��rich,�R.�an���Frie��erici,�A.�D.� (2009).�Mathe�atical�Lo��ic� in� the�Hu�an�Brain:�S��ntax.�PLoS ONE,�4,�e5599.���oi:10.1371/journal.�one.0005599�

Fujio�a,�T.,�Trainor,�L.�J.,�Ross,�B.,�Ka�i��i,�R.�an���Pantev,�C.�(2004).�Musical�trainin���enhances�auto�atic�enco��in���of��elo��ic�contour�an���interval�structure.�Journal of Cognitive Neu-rosciences,�16,�1010–1021.�

Garri��o,�M.�I.,�Kilner,�J.�M.,�Ste�han,�K.�E.�an���Friston,�K.�J.�(2009).�The��is�atch�ne��ativit��:�A�revie��of�un��erl��in����echanis�s.�Clinical Neurophysiology,�120,�453–463.�

Giar��,�M.�H.,�Perrin,�F.,�Pernier,�J.�an���Bouchet,�P.�(1990).�Brain���enerators�i��licate���in�the��rocessin��� of� au��itor��� sti�ulus� ��eviance:� A� to�o��ra�hic� event-relate��� �otential� stu����.�Psychophysiology,�27,�627–640.�

Page 250: Unconscious Memory Representations in Perception

� Unconscious��e�or���re�resentations� 239

Go�es,�H.,�Molhol�,�S.,�Ritter,�W.,�Kurtzber��,�D.,�Co�an,�N.�an���Vau��han,�H.�G.,�Jr.�(2000).�Mis�atch�ne��ativit���in�chil��ren�an���a��ults,�an���effects�of�an�atten��e���tas�.�Psychophysiol-ogy,�37,�807–816.

Gri��,�S.�an���Schrö��er,�E.�(2005).�Preattentive�an���attentive��rocessin���of�te��oral�an���fre-quenc���characteristics��ithin�lon���soun��s.�Cognitive Brain Research,�25,�711–721.

Gunter,�T.�C.,�Frie��erici,�A.�D.�an���Schriefers,�H.�(2000).�S��ntactic���en��er�an���se�antic�ex�ect-anc��:�ERPs�reveal�earl���autono����an���late�interaction.�Journal of Cognitive Neuroscience,�12,�556–568.

Há��en,�G.�P.,�Stefanics,�G.,�Vester��aar��,�M.�D.,�Denha�,�S.�L.,�Sziller,�I.�an���Win�ler,�I.�(2009).�Ti�bre-in��e�en��ent�extraction�of��itch�in�ne�born�infants.�Psychophysiology,�46,�69–74.

Hein�e,�W.,�Kenntner,�R.,�Gunter,�T.�C.,�Sa��ler,�D.,�Olthoff,�D.�et�al.�(2004).�Differential�Ef-fects�of�Increasin���Pro�ofol�Se��ation�on�Frontal�an���Te��oral�Cortices:�An�ERP�stu����.�Anesthesiology,�100,�617–625.�

Hein�e,�W.�an���Koelsch,�S.�(2005).�The�effects�of�anesthetics�on�brain�activit���an���co��nitive�function.�Current Opinion in Anesthesiology,�18,�625–631.�

Herrojo,� R.� M.,� Koelsch,� S.� an��� Bhattachar��a,� J.� (2009).� Decrease� in� earl��� ri��ht� al�ha� ban����hase�s��nchronization�an���late���a��a�ban���oscillations�in��rocessin���s��ntax�in��usic.�Human Brain Mapping,�30,�1207–1225.

Janata,� P.,� Bir�,� J.,� Van� Horn,� J.� D.,� Le�an,� M.,� Till�ann,� B.� an��� Bharucha,� J.� J.� (2002b).�The� cortical� to�o��ra�h��� of� tonal� structures� un��erl��in��� Western� �usic.� Science,� 298,�2167–2170.�

Janata,�P.,�Till�ann,�B.�an���Bharucha,�J.�J.�(2002a).�Listenin���to��ol���honic��usic�recruits���o�ain-��eneral�attention�an����or�in����e�or���circuits.�Cognitive, Affective and Behav-ioral Neuroscience,�2,�121–140.�

Jentsch�e,�S.,�Koelsch,�S.�an���Frie��erici,�A.�D.�(2005).�Investi��atin���the�Relationshi��of�Music�an���Lan��ua��e�in�Chil��ren:�Influences�of�Musical�Trainin���an���Lan��ua��e�I��air�ent.�An-nals of the New York Academy of Sciences,�1060,�231–242.�

Jentsch�e,�S.,�Koelsch,�S.,�Sallat,�S.�an���Frie��erici,�A.�D.�(2008).�Chil��ren��ith�S�ecific�Lan-��ua��e�I��air�ent�also�sho��i��air�ent�of��usic-s��ntactic��rocessin��.�Journal of Cogni-tive Neuroscience,�20,�1940–1951.�

Koechlin,�E.�an���Jubault,�T.�(2006).�Broca’s�area�an���the�hierarchical�or��anization�of�hu�an�behavior.�Neuron,�50,�963–974.�

Koelsch,�S.�(2000).�Brain and Music – A contribution to the investigation of central auditory pro-cessing with a new electrophysiological approach.�PhD�thesis,�Universit���of�Lei�zi��,�Lei�zi��,�Ger�an��.

Koelsch,� S.� (2005).� Neural� substrates� of� �rocessin��� s��ntax� an��� se�antics� in� �usic.� Current Opinion in Neurobiology,�15,�1–6.�

Koelsch,�S.�(2006).�Si��nificance�of�Broca’s�area�an���ventral��re�otor�cortex�for��usic-s��ntactic��rocessin��.�Cortex,�42,�518–520.�

Koelsch,�S.�(2009).�Music-s��ntactic��rocessin���an���au��itor����e�or���–�si�ilarities�an�����iffer-ences�bet�een�ERAN�an���MMN.�Psychophysiology,�46,�179–190.�

Koelsch,� S.,�Fritz,�T.,� Schulze,�K.,�Also�,�D.� an���Schlau��,�G.� (2005b).�A��ults� an���chil��ren��rocessin����usic:�An�fMRI�stu����.�Neuroimage,�25,�1068–1076.

Koelsch,�S.,�Gross�ann,�T.,�Gunter,�T.�C.,�Hahne,�A.,�Schrö��er,�E.�an���Frie��erici,�A.�D.�(2003b).�Chil��ren��rocessin����usic:�Electric�brain�res�onses�reveal��usical�co��etence�an�����en-��er���ifferences.�Journal of Cognitive Neuroscience,�15,�683–693.�

Page 251: Unconscious Memory Representations in Perception

240� Stefan�Koelsch

Koelsch,�S.,�Gunter,�T.�C.,�Cra�ona,�D.�Y.,�Z��sseta,�S.,�Loh�anna,�G.�an���Frie��erici,�A.�D.�(2002c).�Bach�s�ea�s:�A�cortical�‘lan��ua��e-net�or�’�serves�the��rocessin���of��usic.�Neu-roimage,�17,�956–966.

Koelsch,�S.,�Gunter,�T.�C.,�Frie��erici,�A.�D.�an���Schrö��er,�E.�(2000).�Brain�In��ices�of�Music�Processin��:� ‘Non-�usicians’� are� �usical.� Journal of Cognitive Neuroscience,� 12,� 520–541.�

Koelsch,�S.,�Gunter,�T.�C.,�Schrö��er,�E.�an���Frie��erici,�A.�D.�(2003c).�Processin���tonal��o��ula-tions:�An�ERP�stu����.�Journal of Cognitive Neuroscience,�15,�1149–1159.

Koelsch,� S.,� Gunter,� T.� C.,� Schrö��er,� E.,� Tervanie�i,� M.,� Sa��ler,� D.� an��� Frie��erici,� A.� D.�(2001).�Differentiatin���ERAN�an���MMN:�An�ERP-stu����.�Neuroreport,�12,�1385–1389.�

Koelsch,�S.,�Gunter,�T.�C.,�Wittfoth,�M.�an���Sa��ler,�D.�(2005).�Interaction�bet�een�S��ntax�Processin���in�Lan��ua��e�an���in�Music:�An�ERP�Stu����.�Journal of Cognitive Neuroscience,�17,�1565–1579.�

Koelsch,�S.,�Hein�e,�W.,�Sa��ler,�D.�an���Olthoff,�D.�(2006).�Au��itor����rocessin�����urin�����ee���ro�ofol� se��ation� an��� recover��� fro�� unconsciousness.� Clinical Neurophysiology,� 117,�1746–1759.�

Koelsch,�S.,�Jentsch�e,�S.,�Sa��ler,�D.�an���Mietchen,�D.�(2007).�Untan��lin���s��ntactic�an���sen-sor����rocessin��:�An�ERP�stu�����of��usic��erce�tion.�Psychophysiology,�44,�476–490.�

Koelsch,�S.�an���Jentsch�e,�S.�(2008).�Short-ter��effects�of��rocessin����usical�s��ntax:�An�ERP�stu����.�Brain Research,�1212,�55–62.�

Koelsch,�S.,�Kilches,�S.,�Steinbeis,�N.�an���Schelin�si,�S.� (2008).�Effects�of�unex�ecte���chor��s�an���of��erfor�er’s�ex�ression�on�brain�res�onses�an���electro��er�al�activit��.�PLoS-ONE,�3(7),�e2631.

Koelsch,�S.�an���Mul��er,�J.�(2002).�Electric�brain�res�onses�to�ina��ro�riate�har�onies���urin���listenin���to�ex�ressive��usic.�Clinical Neurophysiology,�113,�862–869.�

Koelsch,�S.,�Maess,�B.,�Gross�ann,�T.�an���Frie��erici,�A.�D.�(2003a).�Electric�brain�res�onses�reveal���en��er���ifferences�in��usic��rocessin��.�Neuroreport,�14,�709–713.�

Koelsch,�S.�an���Sa��ler,�D.�(sub�itte��).�Co��nitive�co��onents�of��rocessin���au��itor���re��u-larities.�

Koelsch,� S.,� Sch�i��t,� B.� H.� an��� Kanso�,� J.� (2002a).� Influences� of� �usical� ex�ertise� on� the�ERAN:�An�ERP-stu����.�Psychophysiology,�39,�657–663.�

Koelsch,�S.,�Schrö��er,�E.�an���Gunter,�T.�C.�(2002b).�Music�Matters:�Preattentive��usicalit���of�the�hu�an�brain.�Psychophysiology,�39,�1–11.�

Koelsch,�S.,�Schrö��er,�E.�an���Tervanie�i,�M.�(1999).�Su�erior��re-attentive�au��itor����rocessin���in��usicians.�Neuroreport,�10,�1309–1313.�

Koelsch,�S.�an���Siebel,�W.�(2005).�To�ar��s�a�neural�basis�of��usic��erce�tion.�Trends in Cogni-tive Sciences,�9,�578–584.�

Korz��u�ov,�O.�A.,�Win�ler,�I.,�Gu�en��u�,�V.�I.�an���Alho,�K.�(2003).�Processin���abstract�au��i-tor���features�in�the�hu�an�au��itor���cortex.�Neuroimage,�20,�2245–2258.�

Kru�hansl,�C.�an���Kessler,�E.�(1982).�Tracin���the�����na�ic�chan��es�in��erceive���tonal�or��ani-zation�in�a�s�atial�re�resentation�of��usical��e��s.�Psychological Review,�89,�334–368.�

Kru�hansl,�C.�an���Toivainen,�P.�(2001).�Tonal�Co��nition.�In�R.�J.�Zatorre�an���I.�Peretz�(E��s.),�The�Biolo��ical�Foun��ations�of�Music.�Annals of the New York Academy of Sciences,�930,�77–91.�

Kujala,� A.,� Huotilainen,� M.,� Hota�ainen,� M.,� Lennes,� M.,� Par��onen,� L.,� Fell�an,� V.� an���Näätänen,�R.�(2004).�S�eech-soun�����iscri�ination�in�neonates�as��easure����ith�MEG.�Neuroreport,�15,�2089–2092.�

Page 252: Unconscious Memory Representations in Perception

� Unconscious��e�or���re�resentations� 241

Kushneren�o,�E.,�Win�ler,�I.,�Horváth,�J.,�Näätänen,�R.,�Pavlov,�I.,�Fell�an,�V.�an���Huotilainen,�M.�(2007).�Processin���acoustic�chan��e�an���novelt���in�ne�born�infants.�European Journal of Neurosciences,�26,�265–274.�

La�ont,�A.�an���Cross,�I.�(1994).�Chil��ren’s�co��nitive�re�resentations�of��usical��itch.�Music Perception,�12,�68–94.�

Leino,�S.,�Brattico,�E.,�Tervanie�i,�M.�an���Vuust,�P.�(2007).�Re�resentation�of�har�on���rules�in�the�hu�an�brain:�Further�evi��ence�fro��event-relate����otentials.�Brain Research,�1142,�169–177.�

Le�an,�M.�(2000).�An�au��itor����o��el�of�the�role�of�short-ter���e�or���in��robe-tone�ratin��s.�Music Perception,�17,�481–509.�

Ler��ahl,�F.�(2001).�Tonal Pitch Space.�Oxfor��:�Oxfor���Universit���Press.�Liebenthal,�E.,�Ellin��son,�M.�L.,�S�ana�i,�M.�V.,�Prieto,�T.�E.,�Ro�ella,�K.�M.�an���Bin��er,�J.�R.�

(2003).�Si�ultaneous�ERP�an���fMRI�of�the�au��itor���cortex�in�a��assive�o����ball��ara��i���.�Neuroimage,�19,�1395–1404.

Loui,�P.,�Grent-’t�Jon��,�T.,�Tor�e��,�D.�an���Wol��orff,�M.�(2005).�Effects�of�attention�on�the�neural��rocessin���of�har�onic�s��ntax�in�Western��usic.�Cognitive Brain Research,�25,�589–998.�

Ma��ne,�C.,�Schön,�D.�an���Besson,�M.�(2006).�Musician�chil��ren���etect��itch�violations�in�both��usic�an���lan��ua��e�better�than�non�usician�chil��ren:�Behavioral�an���electro�h��siolo��i-cal�a��roaches.�Journal of Cognitive Neuroscience,�18,�199–211.

Mai��hof,�C.�an���Koelsch,�S.� (2008).�Effects�of�selective�attention�on�neuro�h��siolo��ical�cor-relates�of�s��ntax��rocessin���in��usic�an���s�eech.�Sub�itte��.�

Maess,�B.,�Koelsch,�S.,�Gunter,�T.�C.�an���Frie��erici,�A.�D.�(2001).�‘Musical�S��ntax’�is��rocesse���in�the�area�of�Broca:�An�MEG-stu����.�Nature Neuroscience,�4,�540–545.�

Me��er,�L.�B.�(1956).�Emotion and Meaning in Music.�Chica��o:�Chica��o�Universit���Press.�McCarth��,�G.�an���Donchin,�E.�(1981).�A��etric�for�thou��ht:�A�co��arison�of�P300�latenc���an���

reaction�ti�e.�Science,�211,�77–80.�Miran��a,�R.�A.�an���Ull�an,�M.�T.�(2007).�Double���issociation�bet�een�rules�an����e�or���in�

�usic:�An�event-relate����otential�stu����.�Neuroimage,�38,�331–345.�Molhol�,�S.,�Martinez,�A.,�Ritter,�W.,�Javitt,�D.�C.�an���Foxe,�J.�J.�(2005).�The�neural�circuitr���of�

�re-attentive�au��itor���chan��e-��etection:�An�fMRI�stu�����of��itch�an�����uration��is�atch�ne��ativit�����enerators.�Cerebral Cortex,�15,�545–551.

Moreno,�S.�an���Besson,�M.�(2006).�Musical�trainin���an���lan��ua��e-relate���brain�electrical�activ-it���in�chil��ren.�Psychophysiology,�43,�287–291.�

Näätänen,�R.,�Jacobsen,�T.�an���Win�ler,�I.�(2005).�Me�or��-base���or�afferent��rocesses�in��is-�atch�ne��ativit���(MMN):�A�revie��of�the�evi��ence.�Psychophysiology,�42,�25–32.

Näätänen,�R.,�Lehto�os�i,�A.,�Lennes,�M.,�Cheour,�M.,�Huotilainen,�M.,�Iivonen,�A.,�Vainio,�M.,�Al�u,�P.,�Il�onie�i,�R.�J.,�Luu�,�A.,�Alli�,�J.,�Sin��onen,�J.�an���Alho,�K.�(1997).�Lan��ua��e-s�ecific��hone�e�re�resentations�reveale���b���electric�an����a��netic�brain�res�onses.�Na-ture,�385,�432–434.

Novits�i,�N.,�Huotilainen,�M.,�Tervanie�i,�M.,�Näätänen,�R.�an���Fell�an,�V.�(2007).�Neonatal�frequenc�����iscri�ination�in�250–4000-Hz�ran��e:�Electro�h��siolo��ical�evi��ence.�Clinical Neurophysiology,�118,�412–419.�

O�itz,�B.�an���Frie��erici,�A.�D.�(2007).�Neural�basis�of��rocessin���sequential�an���hierarchical�s��ntactic�structures.�Human Brain Mapping,�28,�585–592.�

O�itz,�B.,�Rinne,�T.,�Mec�lin��er,�A.,�von�Cra�on,�D.�Y.�an���Schrö��er,�E.�(2002).�Differential�contribution�of�frontal�an���te��oral�cortices�to�au��itor���chan��e���etection:�fMRI�an���ERP�Results.�Neuroimage,�15,�167–174.�

Page 253: Unconscious Memory Representations in Perception

242� Stefan�Koelsch

Paavilainen,�P.,�Arajärvi,�P.�an���Ta�e��ata,�R.�(2007).�Preattentive���etection�of�nonsalient�contin-��encies�bet�een�au��itor���features.�Neuroreport,�18,�159–163.�

Paavilainen,� P.,� De��er�an,� A.,� Ta�e��ata,� R.� an��� Win�ler,� I.� (2003).� S�ectral� an��� te��oral�sti�ulus�characteristics� in� the��rocessin���of�abstract�au��itor��� features.�Neuroreport,�14,�715–718.�

Paavilainen,�P.,�Jara�illo,�M.�an���Näätänen,�R.�(1998).�Binaural�infor�ation�can�conver��e�in�abstract��e�or���traces.�Psychophysiology,�35,�483–487.

Paavilainen,�P.,�Si�ola,�J.,�Jara�illo,�M.,�Näätänen,�R.�an���Win�ler,�I.�(2001).�Preattentive�ex-traction�of�abstract�feature�conjunctions�fro��au��itor���sti�ulation�as�reflecte���b���the��is-�atch�ne��ativit���(MMN).�Psychophysiology,�38,�359–365.

Patel,�A.�D.�(2003).�Lan��ua��e,��usic,�s��ntax�an���the�brain.�Nature Neuroscience,�6,�674–681.Patel,�A.�D.,�Gibson,�E.,�Ratner,� J.,�Besson,�M.�an���Holco�b,�P.� (1998).�Processin���s��ntactic�

relations� in� lan��ua��e� an��� �usic:� An� event-relate��� �otential� stu����.� Journal of Cognitive Neuroscience,�10,�717–733.

Patel,�A.�D.,�Iversen,�J.�R.,�Wassenaar,�M.�an���Ha��oort,�P.�(2008).�Musical�s��ntactic��rocess-in���in�a��ra��atic�Broca’s�a�hasia.�Aphasiology,�22,�776–789.�

Poulin-Charronnat,�B.,�Bi��an��,�E.�an���Koelsch,�S.�(2006).�Processin���of��usical�s��ntax�tonic�versus�sub��o�inant:�An�event-relate����otential�stu����.�Journal of Cognitive Neuroscience,�18,�1545–1554.

Pulver�üller,�F.�an���Sht��rov,�Y.�(2003).�Auto�atic��rocessin���of���ra��ar�in�the�hu�an�brain�as�reveale���b���the��is�atch�ne��ativit��.�Neuroimage,�20,�159–172.

Re��nault,�P.,�Bi��an��,�E.�an���Besson,�M.�(2001).�Different�brain��echanis�s��e��iate�sensitivit���to�sensor���consonance�an���har�onic�context:�Evi��ence�fro��au��itor���event-relate���brain��otentials.�Journal of Cognitive Neuroscience,�13,�241–255.�

Rie�ann,� H.� (1877).� Musikalische Syntaxis: Grundriss einer harmonischen Satzbildungslehre.�Nie��er�alluf:�Sän��i��.

Rinne,�T.,�De��er�an,�A.�an���Alho,�K.�(2005).�Su�erior�te��oral�an���inferior�frontal�cortices�are�activate���b���infrequent�soun�����uration���ecre�ents:�An�fMRI�stu����.�Neuroimage,�26,�66–72.�

Rohr�eier,�M.�(2005).�Towards modelling movement in music: Analysing properties and dy-namic aspects of pc set sequences in Bach’s chorales.�MPhil�Thesis.�Universit���of�Ca�-bri����e.�Dar�in�Colle��e�Research�Re�orts�04.�(htt�://���.��ar.ca�.ac.u�/��crr/��crr004.���f)

Rüsseler,� J.,�Alten�üller,�E,.�Na��er,�W.,�Kohl�etz,�C.� an���Münte,�T.�F.� (2001).�Event-relate���brain��otentials�to�soun���o�issions���iffer�in��usicians�an���non-�usicians.�Neuroscience Letters,�308,�33–36.

Saarinen,�J.,�Paavilainen,�P.,�Schö��er,�E.,�Tervanie�i,�M.�an���Näätänen,�R.�(1992).�Re�resenta-tion�of�abstract�attributes�of�au��itor���sti�uli�in�the�hu�an�brain.�Neuroreport,�3,�1149–1151.�

Sa�beth,�A.,�Huotilainen,�M.,�Kushneren�o,�E.,�Fell�an,�V.�an���Pih�o,�E.�(2006).�Ne�borns���iscri�inate�novel�fro��har�onic�soun��s:�a�stu�����usin����a��netoence�halo��ra�h��.�Clini-cal Neurophysiology,�117,�496–503.�

Sa��ler,�D.�(2008).�The Neuroanatomical Overlap of Syntax Processing in Music and Language – Evidence from Lesion and Intracranial ERP Studies.�PhD�thesis,�Universit���of�Lei�zi��.�

Schellenber��,� E.� G.,� Bi��an��,� E.,� Poulin-Charronnat,� B.,� Garnier,� C.� an��� Stevens,� C.� (2005).�Chil��ren’s� i��licit��no�le����e�of�har�on��� in�Western��usic.� Developmental Science,� 8,�551–566.�

Page 254: Unconscious Memory Representations in Perception

� Unconscious��e�or���re�resentations� 243

Schön,�D.�an���Besson,�M.�(2005).�Visuall��� in��uce���au��itor���ex�ectanc��� in��usic�rea��in��:�A� behavioural� an��� electro�h��siolo��ical� stu����.� Journal of Cognitive Neuroscience,� 17,�694–705.�

Schön,�D.,�Ma��ne,�C.�an���Besson,�M.�(2004).�The��usic�of�s�eech:�Music�trainin���facilitates��itch��rocessin���in�both��usic�an���lan��ua��e.�Psychophysiology,�41,�341–349.�

Schon�iesner,�M.,�Novits�i,�N.,�Pa�arinen,�S.,�Carlson,�S.�Tervanie�i,�M.�an���Naatanen,�R.�(2007).�Heschl’s�����rus,��osterior�su�erior�te��oral�����rus,�an����i��-ventrolateral��refron-tal�cortex�have���ifferent�roles�in�the���etection�of�acoustic�chan��es.�Journal of Neurophysiol-ogy,�97,�2075–2082.�

Schrö��er,� E.� (2007).� Mis�atch� ne��ativit��� –� a� �icro�hone� into� au��itor��� �e�or��.� Journal of Psychophysiology,�21,�138–145.�

Schrö��er,�E.,�Ben��ixen,�A.,�Trujillo-Barreto,�N.�J.�an���Roeber,�U.�(2007).�Processin���of�abstract�rule�violations�in�au��ition.�PLoS ONE,�2,�e1131.�

Schubotz,�R.�I.� (2007).�Pre��iction�of�external�events��ith�our��otor�s��ste�:�To�ar��s�a�ne��fra�e�or�.�Trends in Cognitive Sciences,�11,�211–218.

Sht��rov,�Y.,�Hau�,�O.�an���Pulver�üller,�F.�(2004).�Distribute���neuronal�net�or�s�for�enco��in���cate��or��-s�ecific�se�antic�infor�ation:�The��is�atch�ne��ativit���to�action��or��s.�Euro-pean Journal of Neuroscience,�19,�1083–1092.�

Slevc,�L.�R.,�Rosenber��,�J.�C.�an���Patel,�A.�D.�(2009).�Ma�in���Ps��cholin��uistics�Musical:�Self-�ace���rea��in���ti�e�evi��ence�for�share����rocessin���of�lin��uistic�an����usical�s��ntax.�Psy-chonomic Bulletin and Review,�16,�374–381.�

Stefanics,�G.,�Há��en,�G.,�Huotilainen,�M.,�Balázs,�L.,�Sziller,�I.,�Be�e,�A.,�Fell�an,�V.�an���Win-�ler,� I.� (2007).� Au��itor��� te��oral� ��rou�in��� in� ne�born� infants.� Psychophysiology,� 44,�697–702.�

Steinbeis,�N.�an���Koelsch,�S.�(2008).�Share���neural�resources�bet�een��usic�an���lan��ua��e�in-��icate� se�antic� �rocessin��� of� �usical� tension-resolution� �atterns.� Cerebral Cortex,� 18,�1169–1178.�

Steinbeis,�N.,�Koelsch,�S.�an���Slobo��a,�J.�(2006).�The�role�of�har�onic�ex�ectanc���violations�in��usical�e�otions:�Evi��ence�fro��subjective,��h��siolo��ical,�an���neural�res�onses.�Journal of Cognitive Neuroscience,�18,�1380–1393.�

Suss�an,�E.�S.�(2007).�A�ne��vie��on�the�MMN�an���attention���ebate:�The�role�of�context�in��rocessin���au��itor���events.�Journal of Psychophysiology,�21,�164–170.

Suss�an,�E.�S.,�Kujala,�T.,�Hal�etoja,�J.,�L����tinen,�H.,�Al�u,�P.�an���Näätänen,�R.�(2004).�Au-to�atic�an���controlle����rocessin���of�acoustic�an����honetic�contrasts.�Hearing Research,�190,�128–140.

Te��an,�H.�G.�an���Bharucha,�J.�J.�(1998).�I��licit��no�le����e�versus��s��choacoustic�si�ilarit���in��ri�in���of�chor��s.�Journal of Experimental Psychology: Human Perception and Perform-ance,�24,�252–260.�

Tervanie�i,�M.,�R��t�önen,�M.,�Schrö��er,�E.,�Il�onie�i,�R.�J.�an���Näätänen,�R.�(2001).�Su�e-rior�for�ation�of�cortical��e�or���traces�for��elo��ic��atterns�in��usicians.�Learning and Memory,�8,�295–300.

Till�ann,�B.,�Bharucha,�J.�J.,�Bi��an��,�E.�(2000).�I��licit�learnin���of�tonalit��:�A�self-or��anize���a��roach.�Psychological Review,�107,�885–913.

Till�ann,�B.,�Janata,�P.�an���Bharucha,�J.�J.�(2003).�Activation�of�the�inferior�frontal�cortex�in��usical��ri�in��.�Cognitive Brain Research,�16,�145–161.�

Page 255: Unconscious Memory Representations in Perception

244� Stefan�Koelsch

Till�ann,�B.,�Janata,�P.,�Bir�,�J.�an���Bharucha,�J.�J.�(2008).�Tonal�centers�an���ex�ectanc��:�Facili-tation�or�inhibition�of�chor��s�at�the�to��of�the�har�onic�hierarch��?�Journal of Experimental Psychology: Human Perception and Performance,�34,�1031–1043.�

Till�ann,�B.,�Koelsch,�S.,�Escoffier,�N.,�Bi��an��,�E.,�Lalitte,�P.,�Frie��erici,�A.�D.�an���von�Cra�on,�D.�Y.� (2006).�Co��nitive��ri�in��� in�sun���an��� instru�ental��usic:�Activation�of� inferior�frontal�cortex.�Neuroimage,�31,�1771–1782.�

van���en�Brin�,�D.,�Bro�n,�C.�M.�an���Ha��oort,�P.�(2001).�Electro�h��siolo��ical�evi��ence�for�earl���contextual�influences���urin���s�o�en-�or���reco��nition:�N200�versus�N400�effects.�Journal of Cognitive Neuroscience,�13,�967–985.�

Weber,�G.�(1817).�Versuch einer geordneten Theorie der Tonsetzkunst, 2 vols.�Mainz:�B.�Schott.Wi���ann,�A.,�Kujala,�T.,�Tervanie�i,�M.,�Kujala,�A.�an���Schrö��er,�E.�(2004).�Fro��s���bols�to�

soun��s:� Visual� s���bolic� infor�ation� activates� soun��� re�resentations.� Psychophysiology,�41,�709–715.�

Win�ler,� I.� (2007).� Inter�retin��� the� Mis�atch� Ne��ativit��.� Journal of Psychophysiology,� 21,�147–160.�

Win�ler,�I.,�Kujala,�T.,�Tiitinen,�H.,�Sivonen,�P.,�Al�u,�P.,�Lehto�os�i,�A.,�Czi��ler,�I.�Csé�e,�V.,�Il�onie�i,�R.�J.�an���Näätänen,�R.�(1999).�Brain�res�onses�reveal�the�learnin���of�forei��n�lan��ua��e��hone�es.�Psychophysiology,�36,�638–642.�

Win�ler,� I.,� Kushneren�o,� E.,� Horváth,� J.,� Ce�oniene,� R.,� Fell�an,� V.,� Huotilainen,� M.,�Näätänen,�R.�an���Suss�an,�E.�(2003).�Ne�born�infants�can�or��anize�the�au��itor����orl��.�Proceedings of National Academy of Sciences,�100,�11812–11815.�

Woolhouse,�M.�H.�an���Cross,�I.�(2006).�An interval cycle-based model of pitch attraction.�In�Pro-cee��in��s�of�the�9th�International�Conference�on�Music�Perce�tion�&�Co��nition,�Bolo��na,�Ital���(��.�763–771).�

Ylinen,�S.,�Shesta�ova,�A.,�Huotilainen,�M.,�Al�u,�P.�an���Näätänen,�R.�(2006).�Mis�atch�ne��a-tivit���(MMN)�elicite���b���chan��es�in��hone�e�len��th:�A�cross-lin��uistic�stu����.�Brain Re-search,�1072,�175–185.

Page 256: Unconscious Memory Representations in Perception

chapter�9

On the psychophysiology of aestheticsAuto�atic�an���controlle����rocesses��of�aesthetic�a��reciation

Tho�as�Jacobsen�Hel�ut�Sch�i��t�Universit���/�Universit���of�the�Fe��eral��Ar�e���Forces�Ha�bur���/�Universit���of�Lei�zi��,�Ger�an��

9.1 Aesthetic appreciation

Aesthetics� ��eals� �ith� as�ects� of� the� visual,� �lastic� or� fine� arts,� such� as� �aint-in��,���ra�in��,�scul�ture,�architecture,�fil�-�a�in��,��hoto��ra�h��,��rint-�a�in���an��� the� li�e,� an��� �ith� the� �erfor�in��� arts,� o�era,� �usic,� theatre� an��� ��ance.�Aesthetics�is�concerne����ith�literature�an���other�arts,�inclu��in���a��lie���arts.�It�also���eals��ith�the�beaut���of�natural�settin��s�an���ever����a���objects.�This�enor-�ous�variet���in��ental��rocesses�is�reflecte���b���the�nu�erous���ifferent,�uncon-scious�an���conscious��e�or����rocesses� involve��.�Aesthetic��rocessin��� is�also�a�uniquel���hu�an�facult���that��a�es�for�a�ver���interestin��,�albeit�co��lex�re-search� to�ic� (e.��.,� Jacobsen�2006).�Aesthetic��rocessin��,� in���eneral,� can�be���i-vi��e��� into� rece�tive,� central� an��� �ro��uctive� �rocesses� (see� Fi��ure� 1;� Jacobsen�an���Höfel�2003).�Aesthetic�a��reciation�enco��asses�rece�tive�an���central��ro-cesses.� Pro��uctive� �rocesses� relate� to� overt� behavior,� i.e.� aesthetic� ex�ressions�such� as� �aintin��,� �usic,� �oetr��,� ��ance,� theater� an��� the� li�e.� There� are� varie���qualities� of� aesthetic� a��reciation� that� recruit� ��ifferent� sub-�rocesses.� For� in-stance,� aesthetic� ju�����ent,� aesthetic� conte��lation,� an��� aesthetic� ��istraction�can�be�counte���a�on���aesthetic�a��reciation.�All�of�these�entail��erce�tion,�i.e.�rece�tive��rocesses.�Aesthetic���istraction�can�be�observe����hen�a��erson�invol-untaril���s�itches�attention�to�ar��s�the�aesthetic��rocessin���of�an�entit��,�e.��.,�an�object�that�has�ver���hi��h�aesthetic�a��eal,�or�so�eone��ho�is�stri�in��l���beauti-ful.�Aesthetic�conte��lation,�on�the�other�han��,�requires�thin�in��.�It� involves�reflection�about�a� subjective�evaluation,�but� it���oes�not�necessaril��� inclu��e�an�overt�res�onse�or�a�ju�����ent.�One��i��ht�thin��about�the�aesthetic�value�of�an�

Page 257: Unconscious Memory Representations in Perception

24�� Tho�as�Jacobsen

object��ithout�co�in���to�a�conclusion�in�the�en��.�Aesthetic�ju�����ent�also�re-quires�reflective�thin�in��.�Further�ore,�it�has�a��ro��uct,�the�ju�����ent.�This��a���be�in�the�for��of�an�evaluative�cate��orization�an���a���ecision��hether�the�entit���is�beautiful,�har�onic,�ele��ant,�etc.,�or�not.�Finall��,�there��a���be�an�overt�ex�res-sion�of�the���ecision�(ju�����ent�re�ort).

This�cha�ter��ill�focus�on�rece�tive�an���central�as�ects�of�aesthetic��rocess-in��,�that�is�on�aesthetic�a��reciation.�It��ill�hi��hli��ht�the�contributions�of�uncon-scious�an���conscious��e�or����rocesses�involve��.�When��e�ta�e�the���ifferent�art�for�s��entione���above�into�consi��eration,�it�beco�es�rea��il���obvious�that�the�nu�ber�an���co��lexities�of��ental�co��onent��rocesses�an���sta��es�of��rocessin���involve���in�an�e�iso��e�of�aesthetic�a��reciation�can�var�����ra�aticall��.�Conte�-�latin���the�beaut���of�a�na�eless�exotic�flo�er���iffers�vastl���fro��enjo��in���a�sta��-in���of�Wa��ner’s�Rin���of�the�Nibelun��.�Ex�ertise��ith�the�subject��atter,�naturall��,���eter�ines�the�levels�of��rocessin���that�can�be�reache���at�all.�Further�ore,�there�is�virtuall���no�thin��,�it�see�s,�that�cannot�be�a��reciate���aestheticall��.�The�hu�an��in���is�acco��o��atin���all�the���ifferent�entities,�objects,���esi��ns,��or�s�of�art,�an���in��ivi��ual�tastes.�Are�there�li�its�to�the�extent�of�variation�in�aesthetic�a�-�reciation,�li�its�to�the�see�in���arbitrariness?�Un��erstan��in���the��ental��o��es�of��rocessin���a��ears�to�be�crucial�for�ans�erin���this�question.�One�a��roach�to�illu�inatin���this�issue�is�loo�in���at�auto�atic�versus�controlle����rocessin��.

Figure 1. Sche�atic�re�resentation�of�sub-�rocesses�involve���in�aesthetic��rocessin��.�Aesthetic��rocessin���inclu��es�rece�tive,�central,�an����ro��uctive�sub-�rocesses.�Aesthetic�a��reciation�of�beaut���is�a�sub-for��of�aesthetic��rocessin����hich��ainl���inclu��es�rece�-tive�an���central��rocesses.�In�the��resent�ex�eri�ent,�aesthetic�ju�����ents�of�beaut����ere��erfor�e��.�Other�as�ects�of�aesthetic�a��reciation�of�beaut����a���be�aesthetic���istrac-tion�or�aesthetic�conte��lation,�for�exa��le.�(A��a�te����ith��er�ission�fro��Höfel�an���Jacobsen�2007b.)

Page 258: Unconscious Memory Representations in Perception

� On�the��s��cho�h��siolo�����of�aesthetics� 247

9.2 Automatic and controlled processing

Situations�of�aesthetic�a��reciation���iffer�vastl���in�co��lexit��.�A�quic����li��se�of� an� un�no�n,� but� flo�er� �a��� be� quic�� an��� relativel��� si��le.� It� �a��� en��a��e�onl���(a�fe�)�auto�atic��ental��rocesses.�Whereas�a�ni��ht�at�the�o�era�requires�the�o�eration�of�a��ultitu��e�of��ental��rocesses�in�concert,�auto�atic�an���con-trolle���ones�at�that.�There�are�hierarchies�an���casca��es�of��rocesses�that�loc��into�each�other.�So�e�instances�of�aesthetic�a��reciation,�li�e�aesthetic�conte��lation�an���aesthetic�ju�����ent,�entail�controlle����ental��rocessin���at�the�hi��hest�level�of��rocessin���(e.��,�Jacobsen�an���Höfel�2003;�Le��er�et�al.�2006).�The�case�is� less�clear� for�aesthetic���istraction�or��ere�aesthetic��reference� (for�beaut��).� In�an���case,�nu�erous�co��nitive��rocesses�for�in���the�infor�ational�basis�for�aesthetic�a��reciation� is�carrie���out�auto�aticall��.�The��resent�cha�ter�atte��ts� to� trace�so�e� of� these� �rocesses� b��� ��escribin��� un��erl��in��� �e�or��� s��ste�s,� �rocesses,�an���re�resentations.

9.3 Memory systems, processes and representations

In�this�section,�I�revie���e�or���s��ste�s,��rocesses�an���re�resentations�that�have�been�i��licate���in�the�literature,�or�shoul���be�i��licate���to�contribute�to�aesthet-ic�a��reciation.�These�can�be��o��alit��-s�ecific,�or�even���o�ain-s�ecific��ithin�a���iven�sensor����o��alit��,�li�e�can�be��ostulate���for��usic�an���s�eech,�for�instance.�Or�the���are�su�ra-�o��al,�central�an����a��be�also���o�ain-��eneral.�Me�or��,�obvi-ousl��,�is�affecte�����reatl���b���ex�ertise,�at���ifferent�levels�of��rocessin��.�Whether�or�not�chan��es�to��erce�tual�or��anization�base���on�trainin��,�frequent�ex�osure�or�the�li�e�shoul���be�consi��ere����e�or����ro�er�can�be�a��atter�of���ebate,�nonethe-less�the���are�inclu��e���in�this�cha�ter.�A��arentl��,��e�or���o�erates�an���contrib-utes�at�all�levels�of�co��nitive��rocessin���to�aesthetic�a��reciation.

The��resent�cha�ters�traces�several�lines�of�research�in�aesthetic�a��reciation.�These���o�not�constitute��utuall���exclusive��ostulations�of��e�or���s��ste�s,�rath-er�these�are�lines�of�research�that�set���ifferent�foci�such�that�their�objectives��a���overla��in�ter�s�of��e�or���s��ste�s.

9.3.1 Mere�ex�osure�/�fa�iliarit��

The��ere�ex�osure�accounts�of�aesthetic�a��reciation�hol��s�that�fa�iliarit���throu��h�re�eate���ex�osition��ith�an�entit��� lea��s� to� increase��� li�in��� (e.��.,�Zajonc�1968).�This�is�a�ver����ell�establishe���account�(see�Berl��ne�1971)�that�can�o�erate�outsi��e�

Page 259: Unconscious Memory Representations in Perception

24�� Tho�as�Jacobsen

the�co��nitive�fra�e�or��as��ell.�A�behaviorist�sti�ulus-res�onse�anal��sis�is�suf-ficient�in�or��er�to��a�e��re��ictions�about�an�in��ivi��uals�aesthetic��reference.�

In�a�recent�stu����,�Tinio�an���Le��er�(2009)�investi��ate���effects�of�fa�iliariza-tion�on�the�aesthetic�ju�����ent�of���ra�hic��atterns.�The���use���sti�uli�that�varie���in�s����etr���as��ell�as�in�co��lexit���(Jacobsen�an���Höfel�2002)�in���ifferent��ere�ex�osure��rotocols.�The���coul��,� in��ee��,� sho��substantial� effects�usin����assive�fa�iliarization.

Mere�ex�osure�is�one�of�the�can��i��ate�conce�ts�accountin���for�the�a��eal�an���effects�of�a��vertisin��.�Fa�iliarization�at�shallo��levels�of��rocessin���is�use���to�in-��uce��ositive�attitu��inal�effects.�These��rocesses,�in���eneral,�occur�auto�aticall���an���non�consciousl��.

9.3.2 Fluenc���theor��

Fluenc��� theor��� of� aesthetic� a��reciation� hol��s� that� �e� ��enerall��� �refer� enti-ties�that��e��rocess�fluentl���over�those�that��e��rocess�less�fluentl���(Reber�et�al.�2004).�A�art�fro��as�ects�of�the�architecture�of�the�sensor���s��ste��as�reflecte���for�instance�b���the�Gestalt�la�s�of�visual��erce�tion,�there�are�effects�of��erce�-tual�trainin��,�the�acquisition�of�ex�ertise�in�a�sensor�����o�ain,�or�si��l���sensor���conventions� that� are� culturall��� ��eter�ine��� an��� that� var��� over� ti�e.� Se�antic��e�or���an���conce�tual��no�le����e�affect��rocessin���to�-��o�n.�Ta�en�to��ether,�these� factors� affect� the� �erce�tual� fluenc��� �ith� �hich� a� sti�ulus� is� �rocesse��.�Fluenc���theor���is�a�broa���conce�t�that�overla�s��ith�al�ost�all��e�or���conce�ts��entione���in�this�cha�ter�in�one��a���or�another.�Research��ill�have�to���eter�ine�un��er��hich�con��itions�of�aesthetic�a��reciation�the�fluenc���conce�t�can�account�for��ost�of�the�variance�in�the���ata,�an���un��er��hich�con��itions�other�theoretical�notions�hol����ore�ex�lanator���value�(see�e.��.�the�subsection�on�attitu��es).

9.3.3 Proce��ural��no�le����e�/�s�ecialize����erce�tual�s��ste�s�

We�co��an���a��ealth�of��roce��ural��no�le����e,�that��e��ost�often�en��a��e�au-to�aticall��.�What���oes�this�have�to���o��ith�aesthetic�a��reciation,�one��a���as�.�Quite�a�lot,�because�our��erce�tual�strate��ies,�our��a��s�to�see,�hear,�touch,�an���taste� thin��s� are� ��e�en��ent� on� �roce��ural� �no�le����e.� Perce�tual� strate��ies� are��assivel���historicall��,�culturall���an���sociall���relative.�When��e�have�learne���to�see�the��orl�����ifferentl��,�the��orl���beco�es���ifferent.�No�,�one��a���as���hether�or�not�this�notion�has�an���bearin���on�aesthetic�a��reciation.�It���oes,�a�lot�in�fact,�because�the��roce��ural��no�le����e�controls�the�construction�of�the��ental�re�re-sentations�that�for��the�basis�of�aesthetic�a��reciation.

Page 260: Unconscious Memory Representations in Perception

� On�the��s��cho�h��siolo�����of�aesthetics� 249

In�a� series�of�ERP�stu��ies�of�aesthetic� ju�����ent,� for� instance,��e�have�ob-serve���that�the�cortical�ERP�co��onents�u��to�about�250��s�after�sti�ulus�onset�in�a�visual��rotocol��ere�not�affecte���b���the�intention�to��a�e�an�aesthetic�ju����-�ent�or�b���beaut��.�The�N170�co��onent�that� is�relate���to� face��rocessin����as�not��o��ulate���b���the�intention�to��a�e�an�aesthetic�ju�����ent�in�a�stu�����b���Ro��e�et�al.�(2008).�Co��nitive�infor�ation�construction�ta�es��lace�before�intentional�evaluation��ith�res�ect�to�beaut���sets�in.�In�a�si�ilar�vein,�ERPs�reflectin���visual��rocesses�of�novel���ra�hic��atterns��ere�not�affecte���b���evaluative�versus���escri�-tive�ju�����ent��rocessin���in�Jacobsen�an���Höfel�(2003).�

Culturall���an���historicall�����eter�ine���vie�in���strate��ies�affect�aesthetic�a�-�reciation.� Startin��� �ith� literac��,� our� �e��ia� usa��e� affects� an��� ��eter�ines� �er-ce�tual��rocesses�an���there�ith�aesthetic�a��reciation.�One�exa��le�of�cultur-all�����eter�ine���vie�in���habits�affectin���aesthetic�ju�����ent�has�been�re�orte���b���Jacobsen�an���Wolls��orf�(2007)�in�a�stu�����of�the�Bauhaus�color-for��icon.

In�au��ition,�lan��ua��e�an����usical�trainin���have�vast�effects�on�au��itor���ob-ject�for�ation�(cross-reference�to�MMN�an���Music�cha�ters).

The��erce�tual�construction�of��ental�re�resentations�in�aesthetic�a��recia-tion�uses��roce��ural��no�le����e.�In�a�stu�����of�e��e��ove�ents�an���ocular�fixations,�for�instance,�No��ine�et�al.�(1993)�observe���that�traine���art�vie�ers�e��lo��e���vi-sual�scannin����atterns�that���iffere���fro��those�of�not�traine���in��ivi��uals.�That�is,�ex�erts�constructe���infor�ation�about�the�co��ositional���esi��n�of�a���iven�visual�art�or��in�a���ifferent��a���than�non-ex�erts.

Proce��ural��no�le����e�certainl���o�erates�at�various�levels�of��rocessin�����ur-in��� aesthetic� a��reciation.� While� so�e� �rocesses� e��lo��� conventional� �roce-��ural� �no�le����e� that� is� culturall��� ��eter�ine��� an��� therefore� available� to� �ost�in��ivi��uals�of�a���iven���rou�,�ex�ert��no�le����e�requires�substantial�trainin���an���refine�ent.�Future�research��ill�hel��to��lot�the�res�ective�contributions�of�these��in��s�of��roce��ural��e�or���a��ainst�each�other.

9.3.4 Protot���es�

Protot���es�are�lon��-ter���e�or���re�resentations�of�se�antic�cate��ories�(Rosch�1975).�The�����ui��e�classification�of��erce�tual�objects�via���oo��ness�of�fit�to�a�cen-tral�exe��lar�of�a�cate��or��,� i.e.� the��rotot���e.�Hence� the���are��art�of� se�antic��no�le����e,�but���o�receive�s�ecial��ention�in�the�course�of�this�cha�ter,�because�a�bo�����of��or��on�their�role�in�aesthetic�a��reciation�has�been�co��ile��.�Whitfiel���an���Slatter�(1979)�sho�e���that��rotot���icalit���of�ite�s�of�furniture�ha���an�effect�on�the�aesthetic�choices��artici�ants��a��e.�Furniture�that�fit�a��articular��roto-t���e�better��as��referre���aestheticall���over�ite�s��ith�a��oorer�fit.

Page 261: Unconscious Memory Representations in Perception

250� Tho�as�Jacobsen

In��ro��uct���esi��n,�the�so-calle���MAYA��rinci�le,�“�ost�a��vance��,���et�ac-ce�table”�(Richar���Loe�i;�see�He��ert�et�al.�2003),�ca�tures�the�i��ea�that���oo�����esi��n�shoul���see��to���ifferentiate� itself� fro��the�avera��e��hile�still�bein���ac-co��o��ate���into�the��rotot���e�for�this���esi��n.�A�ne��Vol�s�a��en�Golf,�for�ex-a��le,�has�to�be���ifferent�fro��the��re��ecessor��hile�still�clearl���bein���a�Golf.�If�a�behol��er�can�successfull���acco��o��ate�an�object�into�the��no�le����e�s��ste�,�this�object,�on�avera��e,�is�li�el���to�have���rater�aesthetic�a��eal�than�a�ver���un-usual�object,�or�one�that�cannot�be�acco��o��ate���at�all.�He��ert�an���collea��ues�(2003)�sho�e���that�objects�are�li�el���to�receive�hi��her�a��raisal��hen�the���also�feature� a� certain� ��e��ree� of� novelt��.� Therefore,� t���icalit��� an��� novelt��� are� joint��re��ictors�of�aesthetic��reference.

In� a� stu����� on� the� a��reciation� of� �aintin��s,� He��ert� an��� van� Wierin��en�(1996)�sho�e���that��rotot���icalit���alon����ith�co��lexit���are���eter�inants�of�the�a��raisal�of�cubist��aintin��s�(see�also�Martin��ale�an���Moore�1988,�for�a�stu�����on��rotot���es�affectin����reference).

Our�ERP�stu��ies�on�aesthetic�a��reciation�(Jacobsen�an���Höfel�2003;�Höfel�an���Jacobsen�2007a,�2007b;�Ro��e�et�al.�2008)�reveale���ti�e�courses�that�su����est�that��erce�tual��rocessin���ta�es��lace��rior�to�aesthetic�evaluation.�Therefore,�it��a���be�ar��ue���that��rotot���es,�if�a�behol��er�co��an��s�the�,�are�auto�aticall���activate�����urin���the��erce�tual��rocessin���of�an�object,�that�un��erlies�aesthetic�a��reciation.�Protot���es��atter�in�the�construction�of�a��ental�re�resentation�of�an�object.�This,�of�course,���oes�not� i��l���that��rotot���iclit��� is�sufficient� in�accountin���for�aesthetic��reference.�Future�research��ill�have�to���eter�ine�un-��er��hich�con��itions��rotot���icalit���can�account�for�the�bul��of�the�variance�in�aesthetic�a��reciation,�an���un��er��hich� it�cannot�(see� for� instance�section�on�attitu��es).

9.3.5 Se�antic��e�or��

Se�antic��e�or��,�our�conce�tual��no�le����e�s��ste��in�a�broa��er�sense,��la��s�a��ajor�role�in�aesthetic�a��reciation.�In�a�stu�����of�ex�ert�an���non-ex�ert�behol��er,�He��ert�an���van�Wierin��en�(1996)�observe���that�ex�ertise� in�visual�art�affects�the�aesthetic�a��reciation�of��or�s�of�art.�In�their�stu����,�ex�erts��ei��hte����er-ceive���ori��inalit���of�an�art�or���ore�stron��l���than�non-ex�erts���i��.�Kno�le����e�about�art�histor��,�art�theor���an���the�li�e���eter�ine�the�richness,�co��lexit��,�an����recision�of� the��ental� re�resentation�of�an�art�or�� that�a�behol��er� is�able� to�construct.�Givin���vie�ers�bac���roun���infor�ation�about�an�art�or�,�for�instance�titles�of���ifferin�����e��rees�of�elaborateness,�therefore�affects�aesthetic�a��reciation�(Le��er�et�al.�2006).�In�a�si�ilar�vein,�No��ine�et�al.�(1993)�su����este���that�traine���

Page 262: Unconscious Memory Representations in Perception

� On�the��s��cho�h��siolo�����of�aesthetics� 251

art�vie�ers�constructe��� infor�ation�about�the�co��ositional���esi��n�of�a���iven�visual�art�or��in�a���ifferent��a���than�non-ex�erts.�This��a���in��art�be�consi��ere���to�be���riven�b����roce��ural��no�le����e,�but�search�for��articular��ieces�of�infor-�ation�is���ui��e���b���the�ex�ert�conce�tual��no�le����e.

Ex�ert� an��� non-ex�ert� ju����es� ali�e,� often� s�ontaneousl��� use� associations�base���on��orl����no�le����e�in�aesthetic��rocessin��,�for�instance��hen�as�e���to�for�instance��hen�as�e���to��ro��uce�color-for��assi��n�ents�(Jacobsen�2002).�Partici-�ants�also�use�s�ontaneous�associations�retrieve���fro��se�antic�(an���e�iso��ic)��e�or���as�infor�ational�cues�infor�in���their�evaluative��rocessin��,�for�instance�aesthetic�ju����e�ent�of�novel���ra�hic��atterns�(Jacobsen�2004;�Jacobsen�an���Höfel�2002,�2003;�Jacobsen�et�al.�2006).

In�a����ition,�behol��ers�co��an���conce�tual�structures�re�resentin���various���o�ains�of�aesthetic�a��reciation.�Se�antic��e�or���that�is�relevant�for�the�aes-thetics�of�objects�(Jacobsen�et�al.�2004)�or��usic�(Isto��et�al.�2009)�can�be�thou��ht�of�as�bein���re�resente���in�net�or�s�that�en��a��e��hen�aesthetic��rocessin���in�the�relevant���o�ain�occurs.�

Future�research��ill�have�to�carefull�����eter�ine��hich�levels�of�re�resenta-tion�an���t���es�of�se�antic��e�or���contribute�to���o�ain-��eneral�an���to���o�ain-s�ecific��rocesses�of�aesthetic�a��reciation.

9.3.� Co��nitive�fluenc��

Co��nitive�fluenc�����enotes�a�variant�of�the�fluenc���account�that�e��hasizes�hi��h-er-level� co��nitive� �rocesses.� It� ��eals� �ith� co��nitive� �asterin��,� st��le� classifica-tion�an���other�relate����rocesses�(e.��.,�Le��er�et�al.�2006).�For�the�a��reciation�of�so�e��or�s�of�art,�havin���a� little�bac���roun����no�le����e,�histor���or�even�just�a� title� �a�es� all� the� ��ifference.� For� a� lot� of� �o��ern� art,� ��ivin��� it� a� title� ��oes�chan��e� the� aesthetic� �rocessin��� consi��erabl��.� On� the� other� han��,� if� no� bac�-��roun����no�le����e�about�the��or��is�conve��e���to�the�behol��er�he�or�she��i��ht�not�have�a�chance�to�full���a��reciate�it.�Co��nitive�fluenc�����eals��ith�these�as-�ects.�But�also��ith�st��le�classification�an���other�ele�ents�of�hi��her�co��nition�is�aesthetic�a��reciation.�In��articular,�bein���able�to�construct��eanin���attache���to�an�art�or�,�on�the�basis�of�title�infor�ation�or�other,��a���lea���to�a��leasurable�aesthetic�ex�erience�(e.��.,�Millis�2001).�

Even�thou��h�access�to�the��no�le����e�structure�is��re��o�inantl���auto�atic,�active�search�for�the�critical�bits�of�infor�ation,�in�a��useu��for�instance,�is�hi��h-l���controlle��,�intentional��rocessin��.

Page 263: Unconscious Memory Representations in Perception

252� Tho�as�Jacobsen

9.3.7 Attitu��es

Attitu��es�are��e�or���re�resentations�that�entail�three�facets,�a��no�le����e��or-tion,�a�valence�as�ect,�an���a�behavioural�co��onent.�For�a�lot�of�entities,��e�store�our� evaluation� an��� our� behavioural� ten��enc��� alon��� �ith� our� �no�le����e.� Atti-tu��es���et�activate���auto�aticall���an���allo��to��a�e�quic��assess�ents�in�ever��-��a���situations�(for�a�revie��see�e.��.,�Pett���et�al.�1997).

Attitu��es,�not�sur�risin��l��,�also�have��o�erful�influence�on�aesthetic�a��re-ciation.�It�stan��s�to�reason�that��e�or��-store���evaluations��a���frequentl���over-ri��e��ere�ex�osure,�fa�iliarit��,�or�fluenc����hen�in��ivi��ual�aesthetic��references�are�concerne��.

Ritterfel���(2002)�re�orte���a�stu�����of�interior���esi��n��reference.�These��ere���overne���b���attitu��es�via�social�heuristic��rocessin��,�if�the�social��ar�in���of�an�object��as���eco��eable.�An�anal��sis�base���on�for�al�features�of�an�object�(�iece�of�furniture)��oul���onl���be�con��ucte��,�if�the�attitu��inal�infor�ation��as�not�avail-able�or�not�sufficient.

The�i��licit�association�test�(IAT)�is�a�behavioral�ex�eri�ental�technique�that�allo�s�to�covertl����easure�auto�atic�activation�of�attitu��es,�i.e.,�the�unconscious�use�of�the�attitu��e��e�or���s��ste��(see�e.��.,�Fazio�an���Olson�2003).�In�a�stu�����of�the�thin-bo�����i��eal��ith�fe�ale�un��er��ra��uate�stu��ents,�Ahern�an���collea��ues�(2008)�investi��ate���bo�������issatisfaction�internalization�of�a�thin-i��eal,�an���eatin�����isor��er�s����to�s�usin���the�IAT�as�a��easure�of�auto�atic�attitu��e�activation�(The�IAT�has�also�been�use���in�so�far�un�ublishe���stu��ies�of�attitu��es�to�ar��s�visual�arts).

Base���on�the�research�on�the�auto�atic�activation�of�attitu��es,� it� stan��s� to�reason� that� attitu��es� �a��� have� a� stron��,� fast� effect� on� aesthetic� a��reciation��henever�sti�uli�are�acco��o��ate���into�an�in��ivi��ual’s�attitu��e�s��ste�.�Base�on��no�le����e�an���ex�ertise,�this��ill�ver���often�be�the�case.�In�so�e�cases,�attitu��es��a���even�lea���to�full-blo�n��e�or��-base���ju�����ents�of,� for� instance,�beaut��,�rather� than� aesthetic� ones.� Partl��� for� these� reasons� so�e� research� on� aesthetic�a��reciation� of� beaut��� has� resorte��� to� usin��� novel� sti�uli� in� or��er� to� �revent�attitu��e�effects� (e.��.,� Jacobsen�2004;� Jacobsen�an���Höfel�2002,�2003;�Höfel� an���Jacobsen�2003,�2007a,�2007b).

Future�research��ill�have�to�illu�inate�the�s�ecific�contributions�of�attitu��es�to�aesthetic�a��reciation�as��ell�as�their�ti�e�courses�in�the�various���o�ains.

Page 264: Unconscious Memory Representations in Perception

� On�the��s��cho�h��siolo�����of�aesthetics� 253

9.3.� E�iso��ic��e�or��

E�iso��ic��e�or��,��ost�li�el��,��la��s�a�fun��a�ental�an���ver���i��ortant�role�in�aesthetic�a��reciation.�The�s��ste�atic�stu�����of�its�involve�ent�a��ears�to�be���e-velo�e���to�a� lesser���e��ree�than�other��e�or���s��ste�s��entione���in�the��res-ent�cha�ter.�In�our�stu��ies,��artici�ants�routinel���re�ort�to�s�ontaneousl���e�-�lo���associations�base���on�e�iso��ic��e�or���as��art�of�the�infor�ational�bases�for�their�aesthetic�ju�����ent�or�their�aesthetic��references�(e.��.,�Jacobsen�2004;�Jacobsen�an���Höfel�2002,�2003).�The�brain�net�or��subservin���aesthetic�ju����-�ent�of�novel���ra�hic��atterns,�as�contraste����ith�a�s����etr���ju�����ent�of�the�sa�e��aterial,�entails�structures�consi��ere���to�be�involve���in�the��e�or���of�life�events�(Jacobsen�et�al.�2006).�A��ain,��artici�ants�s�ontaneousl����a�e�use�of�as-sociations�for��e�iso��ic��e�or����ithout�bein���instructe���to���o�so.

These� fin��in��s� ��o� alon��� �ell� �ith� the� fact� that� e�iso��ic� �e�or��� is� often�consi��ere���to�be�activate���s�ontaneousl���or�even�auto�aticall��.�The�latter�hol��s,�es�eciall��,��hen�contents�of�stron���valence�or�hi��h�e�otional�i��ortance�are�involve��.� E�iso��ic� �e�or��� �a��� be� often� revisite��,� an��� restructure��� in� the�course.�It�can�even�be�consi��ere���constructive�(for�a�revie��see,�e.��.,�Schacter�an���A����is�2007).

Auto�atic�activation�of�the��e�or���of�life�events�can�be�stron��l���in��uctive�of�(aesthetic)�e�otions.�Konecni�(2008)�ar��ue���that�e�iso��ic��e�or���is�a��ajor,�if�not�the��ajor,�factor���overnin���the�in��uction�of�e�otion�b����usic.�Listeners��re-fer,�an���aestheticall���a��reciate,��usic�for�its�senti�ental�value.�The�choice�of�the��usic��er�se��a���be�so�e�hat�arbitrar��,�but�the�stren��th�of�its�lin��to�e�iso��ic��e�or���is�not.�This��a���in�fact�be�the���rivin���force,�as�Konecni�su����ests,�behin���the�influential�results�of�Bloo���an���Zatorre�(2001),��ho�re�orte���brain�correlates�of�intensel����leasurable��usic�rece�tion.�Partici�ants��ere�as�e���to�brin����usic�of� their�o�n�choice,� that��oul���reliabl��� in��uce�chills.�Their�selection�of��usic,�in��ee��,� co��rise��� a� �i��er� ran��e� of� �usical� st��les� an��� ��enres,� su����estin��� that�e�otion�elicitation� is���riven��ore�stron��l���b��� in��ivi��ual�e�iso��ic��e�or���as-sociations�than�a�co��on���eno�inator� in�the��usic� itself.� In�a���ifferent�vein,�Zentner�et�al.�(2008)�i��entifie���‘nostal��ia’�as�one�of�the�nine�i��ortant�aesthetic�e�otions�in��usic.�Usin����artici�ants'�self-re�ort���ata,�a��o��el�of�the�structure�of�aesthetic�e�otions�in��usic��as���erive���base���on�a�cluster�anal��sis�a��roach.

E�iso��ic��e�or����a���be�of���ifferential�i��ortance�for�various�content���o-�ains�of�aesthetics�an���the�arts�an���for���ifferent�sensor����o��alities�(recall�the�s�ecial�status�of�s�ell),�but�its�unconscious,�as��ell�as�conscious,��rocesses�clearl���ren��er��ajor�contributions�to�aesthetic�a��reciation.�Future�research��ill�illu�i-nate�these�in���reater���etail.

Page 265: Unconscious Memory Representations in Perception

254� Tho�as�Jacobsen

9.3.9 Scri�ts/sche�ata

Scri�ts,�or�sche�ata�in��ore���eneral�ter�s,�are�hi��hl���structure���often�hierar-chical�lon��-ter���e�or���re�resentations�that�allo��us�to�navi��ate�ever����a���life��ithout�bein���overtaxe���b���the��ost��un��ane�thin��s�(see�e.��.,�Abelson�1981).�Enterin���a� restaurant,��e�auto�aticall���activate�a� scri�t� that� lets�us�ex�ect� to�be���iven�a��enu�after�bein���seate��,�to��a�e�our�choice�of�foo���an�����rin�,��ait�for�our�or��er�to�be�brou��ht�to�the�table�b���a��aitress�or��aiter,�an����a���after��e�have�eaten.�When�ever��thin���runs�s�oothl���as�ex�ecte��,��e�often���o�not�even�notice�the�scri�t�that�allo�s�us�to�allocate��rocessin���resources�to�thin��s�other�than�eatin���an�����rin�in��,�the�conversation�for�instance.�When�our�scri�t�is�in-sufficient,�so�eone�challen��es�it,�or�consi��ers�it�ina��ro�riate,�that�is��hen��e�nee���to�allocate�attention�to�the�situation�at�han��.�Scri�ts�are�learne��,�the���are�historicall��,�culturall��,�an���sociall���relative,�an����e�co��an���quite�a�nu�ber�of�the��for��an���t���es�of�situations�(for�an�intro��uction�see�e.��.,�Bre�er�an���Na�a�ura�1984).�

Sche�ata�also���overn��an���situations�in��hich�aesthetic�a��reciation�ta�es��lace.�Vie�in���a�rea����-�a��e��or��of�art�li�e�Marcel�Ducha��’s�“fountain”�in�a��useu��or�an�art���aller���is�a�fun��a�entall�����ifferent�ex�erience�than�vie�in���the�sa�e�object�in�a�har���are�store,��rovi��e���that�the�behol��er�has�activate���an�art�scri�t�in�the�for�er�an���an�utilitarian�ever����a���scri�t�in�the�latter�context.�

Sche�ata�can�also�be�social�relational�sche�ata�that�hel��us�navi��ate�encoun-ters�an���interactions��ith�other�in��ivi��uals�(e.��.,�Bal���in�1992).�Also�in�aesthetic�a��reciation,��e�have�such��ental�re�resentations�of�(ritualize��)�social�interac-tion.�Artists�often�use�the��ental�scri�t�re�resentations�of�their�au��ience�b���in-tro��ucin�����eviations�an���sur�rises�(see�Bernstein�an���Rubin�for�a�stu�����on�the�cultural�relativit���of�scri�ts).

Research�on�the�role�of�these��e�or���s��ste�s�in�the�various���o�ains�of�aes-thetic�a��reciation�is�quite�scarce.�This�is�so���es�ite�the�fact�that�scri�ts�an���sche-�ata�are�central�conce�ts�in�accountin���for�the�role�of�situational�context.

Further�ore,�in��ivi��uals�have�the�ca�acit���to�en��a��e�sche�ata�activel��.�The�intention�to�conte��late�an�object�aestheticall���can�be���enerate���b���a�behol��er,�full���in��e�en��entl���of�an�(auto�atic)�activation�of�an�art�scri�t.

Future�research�in�Ex�eri�ental�Aesthetics��ill�elaborate�on�the��ivotal�role�that�sche�ata��la���in��an���aesthetic�rece�tive�situations.

Page 266: Unconscious Memory Representations in Perception

� On�the��s��cho�h��siolo�����of�aesthetics� 255

9.4 Summary and conclusion

The��resent�cha�ter�has�revie�e����e�or���s��ste�s�that�o�erate�at���ifferent�levels�of��rocessin���in�aesthetic�a��reciation.�Basic�sensor��,��erce�tual,�an���co��nitive��rocessin��� is� carrie��� out� auto�aticall��,� an��� lar��el��� unconsciousl��,� �rovi��in���the� infor�ational� basis� for� hi��her� or��er� �rocesses� of� aesthetic� conte��lation�or�ju�����ent.�Our�electro�h��siolo��ical,��re��o�inantl���ERP�results�sho��us�that�aesthetic�ju�����ent�of���ra�hic��atterns,�faces�an����usical�ca��ences�is��rece��e���b���the�construction�of�infor�ation�base���on�various��e�or���s��ste�s�o�eration�unconsciousl��.

References

Abelson,� R.� P.� (1981).� Ps��cholo��ical� status� of� the� scri�t� conce�t.� American Psychologist,� 36,�715–729.

Ahern,�A.�L.,�Bennett,�K.�M.�an���Hetherin��ton,�M.�M.�(2008).�Internalization�of�the�ultra-thin�i��eal:�Positive�i��licit�associations��ith�un��er�ei��ht�fashion��o��els�are�associate����ith���rive�for�thinness�in���oun����o�en.�Eating Disorders,�16,�294–307.

Bal���in,�M.�W.�(1992).�Relational�Sche�as�an���the�Processin���of�Social�Infor�ation.�Psycho-logical Bulletin,�112,�461–484.

Berl��ne,�D.�E.�(1971).�Aesthetics and Psychobiology.�Ne��Yor�:�A��leton-Centur��-Crofts.Bernstein,�D.�an���Rubin,�D.�C.�(2004).�Cultural�life�scri�ts�structure�recall�fro��autobio��ra�hi-

cal��e�or��.�Memory & Cognition,�32,�427–442.Bloo��,�A.�J.�an���Zatorre,�R.�J.�(2001).�Intensel����leasurable�res�onses�to��usic�correlate��ith�

activit��� in�brain�re��ions� i��licate��� in�re�ar���an���e�otion.�Proceedings of the National Academy of Sciences,�98,�11818–11823.

Brattico,�E.,�Brattico,�P.�an���Jacobsen,�T.�(2009).�The�ori��ins�of�the�aesthetic�enjo���ent�of��usic.�A�revie��of�the�literature.�Musicae Scientiae, Special Issue,�15–39.

Bre�er,�W.�F.�an���Na�a�ura,�G.�V.�(1984).�The nature and functions of schemas.�Urbana-Cha�-�ai��n:�Universit���of�Illinois�Press.

Fazio,� R.� H.� an��� Olson,� M.� A.� (2003).� I��licit��easures� in� social� co��nition� research:� Their��eanin���an���use.�Annual Review of Psychology,�54,�297–327.

Istó�,�E.,�Brattico,�E.,�Jacobsen,�T.,�Krohn,�K.�an���Tervanie�i,�M.�(2009).�Aesthetic�res�onses�to��usic:�A�questionnaire�stu����.�Musicae Scientiae,�13,�183–206.

He��ert,�P.�an���van�Wierin��en,�P.�C.�W.�(1990).�Co��lexit���an����rotot���icalit���as���eter�i-nants�of�the�a��raisal�of�cubist��aintin��s.�British Journal of Psychology,�81,�483–495.

He��ert,�P.�an���van�Wierin��en,�P.�C.�W.�(1996).�Beaut���in�the�e��e�of�ex�ert�an���nonex�ert�be-hol��ers:�A�stu�����in�the�a��raisal�of�art.�American Journal of Psychology,�109,�389–407.

He��ert,�P.,�Snel��ers,�D.�an���van�Wierin��en,�P.�C.�W.�(2003).�‘Most�a��vance��,���et�acce�table’:�T���icalit���an���novelt���as�joint��re��ictors�of�aesthetic��reference�in�in��ustrial���esi��n.�Brit-ish Journal of Psychology,�94,�111–124.

Page 267: Unconscious Memory Representations in Perception

25�� Tho�as�Jacobsen

Höfel,�L.,�Lan��e,�M.�an���Jacobsen,�T.�(2007).�Beaut���an���the�teeth:�Perce�tion�of�tooth�color�an���its�influence�on�the�overall�ju�����ent�of�facial�attractiveness.�International Journal of Periodontics & Restorative Dentistry,�27,�349–357.

Höfel,�L.�an���Jacobsen,�T.�(2003).�Te��oral�stabilit���an���consistenc���of�aesthetic�ju�����ents�of�beaut���of�for�al���ra�hic��atterns.�Perceptual and Motor Skills,�96,�30–32.

Höfel,�L.�an���Jacobsen,�T.�(2007a).�Electro�h��siolo��ical�in��ices�of��rocessin���aesthetics:�S�on-taneous�or�intentional��rocesses?�International Journal of Psychophysiology,�65,�20–31.

Höfel,�L.�an���Jacobsen,�T.�(2007b).�Electro�h��siolo��ical� in��ices�of��rocessin���s����etr���an���aesthetics:�A�result�of�ju�����ent�cate��orization�or�ju�����ent�re�ort?�Journal of Psychophysi-ology,�21,�9–21.

Jacobsen,�T.�(2002).�Kan��ins���’s�questionnaire�revisite��:�Fun��a�ental�corres�on��ence�of�basic�colors�an���for�s?�Perceptual and Motor Skills,�95,�903–913.

Jacobsen,�T.� (2004).� In��ivi��ual�an�����rou���o��elin���of�aesthetic� ju�����ent�strate��ies.�British Journal of Psychology,�95,�41–56.

Jacobsen,�T.�(2006).�Bri����in���the�arts�an���sciences:�A�fra�e�or��for�the��s��cholo�����of�aes-thetics.�Leonardo,�39,�155–162.

Jacobsen,�T.,�Buchta,�K.,�Köhler,�M.�an���Schrö��er,�E.�(2004).�The��ri�ac���of�beaut���in�ju����in���the�aesthetics�of�objects.�Psychological Reports,�94,�1253–1260.

Jacobsen,�T.�an���Höfel,�L.�(2002).�Aesthetic�ju�����ents�of�novel���ra�hic��atterns:�Anal��ses�of�in��ivi��ual�ju�����ents.�Perceptual and Motor Skills,�95,�755–766.

Jacobsen,�T.�an���Höfel,�L.�(2003).�Descri�tive�an���evaluative�ju�����ent��rocesses:�Behavioral�an���electro�h��siolo��ical�in��ices�of��rocessin���s����etr���an���aesthetics.�Cognitive, Affec-tive and Behavioral Neuroscience,�3,�289–299.

Jacobsen,�T.,�Schubotz,�R.�I.,�Höfel,�L.�an���v.�Cra�on,�D.�Y.�(2006).�Brain�correlates�of�aesthetic�ju�����ent�of�beaut��.�Neuroimage,�29,�276–285.

Jacobsen,�T.� an���Wols��orff,�C.� (2007).�Does�histor���affect� aesthetic��reference?�Kan��ins���’s�teachin���of�colour-for��corres�on��ence,�e��irical�aesthetics,�an���the�Bauhaus.�The De-sign Journal,�10,�16–27.

Konecni,�V.�J.�(2008).�Does��usic�in��uce�e�otion?�A�theoretical�an����etho��olo��ical�anal��sis.�Psychology of Aesthetics, Creativity, and the Arts,�2,�115–129.

Le��er,�H.,�Carbon,�C.�C.�an���Ri�sas,�A.�L.�(2006).�Entitlin���art:�Influence�of�title�infor�ation�on�un��erstan��in���an���a��reciation�of��aintin��s.�Acta Psychologica,�121,�176–198.

Martin��ale,�C.�an���Moore,�K.�(1988).�Pri�in����rotot���icall���an����reference.�Journal of Experi-mental Psychology: Human Perception and Performance,�14,�661–670.

Millis,�K.�(2001).�Ma�in����eanin��s�brin��s��leasure:�The�influence�of�titles�on�aesthetic�ex�eri-ences.�Emotion,�1,�320–329.

No��ine,�C.�F.,�Locher,�P.�J.�an���Kru�ins�i,�E.�A.�(1993).�The�role�of�for�al�art�trainin���on��er-ce�tion�an���aesthetic�ju�����ent�of�art�co��ositions.�Leonardo,�26,�219–227.

Pett��,�R.�E.,�We��ener,�D.�T.�an���Fabri��ar,�L.�R.�(1997).�Attitu��es�an���attitu��e�chan��e.�Annual Review of Psychology,�48,�609–647.

Reber,�R.,�Sch�arz,�N.�an���Win�iel�an,�P.�(2004).�Processin���Fluenc���an���Aesthetic�Pleasure:�Is�Beaut���in�the�Perceiver’s�Processin���Ex�erience?�Personality and Social Psychology Re-view,�8,�364–382.

Ritterfel��,�U.�(2002).�Social�heuristics�in�interior���esi��n��references.�Journal of Environmental Psychology,�22,�369–386.

Rosch,�E.�(1975).�Co��nitive�Re�resentations�of�Se�antic�Cate��ories.�Journal of Experimental Psychology: General,�104,�192–233.

Page 268: Unconscious Memory Representations in Perception

� On�the��s��cho�h��siolo�����of�aesthetics� 257

Ro��e,�A.,�Höfel,�L.�an���Jacobsen,�T.�(2008).�Aesthetics�of�faces:�Behavioural�an���electro�h��si-olo��ical�in��ices�of�evaluative�an�����escri�tive�ju�����ent��rocesses.�Journal of Psychophysiol-ogy,�22,�41–57.

Schacter,�D.�L.�an���A����is,�D.�R.�(2007).�The�co��nitive�neuroscience�of�constructive��e�or��:�Re�e�berin���the��ast�an���i�a��inin���the�future.�Philoso�hical�Transactions�of�the�Ro��al�Societ��,�Lon��on,�Biological Sciences,�362,�773–786.

Tinio,�P.�L.�an���Le��er,�H.�(2009).�Just�ho��stable�are�stable�aesthetic�features?�S����etr��,�co�-�lexit��,�an���the�ja�s�of��assive�fa�iliarization.�Acta Psychologica,�130,�241–250.

Whitfiel��,�T.�W.�A.�an���Slatter,�P.�E.�(1979).�The�effects�of�cate��orization�an����rotot���icalit���on�aesthetic�choice�in�a�furniture�selection�tas�.�British Journal of Psychology,�70,�65–75.

Zajonc,� R.� B.� (1968).� Attitu��inal� effects� of� �ere� ex�osure.� Journal of Personality and Social Psychology Monograph Supplements,�9(2�Pt�2),�1–27.

Zentner,�M.,�Gran��jean,�D.�an���Scherer,�K.�R.�(2008).�E�otions�evo�e���b���the�soun���of��usic:�Characterization,�classification,�an����easure�ent.�Emotion,�8,�494–521.

Page 269: Unconscious Memory Representations in Perception
Page 270: Unconscious Memory Representations in Perception

appendix

Using electrophysiology to study unconscious memory representations

Alexan��ra�Ben��ixen�Institute�for�Ps��cholo�����of�the�Hun��arian�Aca��e����of�Sciences,�Bu��a�est,�Hun��ar���/�Institute�for�Ps��cholo�����I,�Universit���of�Lei�zi��,�Ger�an��

10.1 Basic principles of electroencephalography (EEG)

Stu����in��� unconscious� �e�or��� essentiall��� ta�s� into� �heno�ena� �hich� cannot�be�verball���re�orte���b���the��artici�ant.�Consequentl��,� there� is�a�nee���for��ea-sure�ent� techniques� that���o�not� require�overt� res�onses.�This�a��en��ix� intro-��uces�one�such�technique,�the�electroence�halo��ra��(EEG),�as��ell�as�t�o��ain��easures�that�can�be���erive���fro��it,�na�el���event-relate����otentials�(ERPs)�an���oscillator��� activit��.� The� EEG� technique� is� onl��� briefl��� outline��� here;� for� �ore�co��rehensive�treat�ents,�the�rea��er�is�referre���to�Fabiani�et�al.�(2007),�Han�����(2005),�Luc��(2005)�as��ell�as�Ru�����an���Coles�(1995).�

Electroence�halo��ra�h���is�a�relativel���ol���technique�for��easurin���brain�ac-tivit���(Ber��er�1929)�that�has��e�t�its�a��vanta��es�in�ter�s�of�non-invasiveness�an���lo�� cost� of� the� �easure�ent.� Most� i��ortantl��,� ho�ever,� the� EEG� �rovi��es� a�te��oral�resolution�on�the�or��er�of��illisecon��s,��hich�is�su�erior�to�an���other�non-invasive�technique.�The�beaut���of�this�te��oral�accurac���lies�in�the�acquisi-tion�of�an�on-line��easure�of�brain�activit��.�This�allo�s�for���eter�inin���the�ti�e-course�of�infor�ation��rocessin��,��hich�is�i��ossible��ith�co��osite��easures�such�as�res�onse�ti�es�an���error�rates�because�the���co��rise�all��rocesses�fro��sti�ulus��erce�tion�to�res�onse�execution.�Unli�e�behavioral��easures,�the�EEG�also�allo�s�for�trac�in���the��rocessin���of�unatten��e���sti�uli.�Moreover,�the�EEG�facilitates�the�investi��ation�of�clinical���rou�s�an���infants��ho��a���not�be�able�to���ive�a�verbal�or��otor�res�onse.

In� �rinci�le,� the� EEG� �easures� the� electrical� activit��� of� the� brain� throu��h�electro��es��lace���on�the�scal�.�Metal�electro��es�are�s�ecificall����re�are���to�en-hance�con��uctivit��,�attache���to�the�scal��surface,�an���connecte���to�an�a��lifier.��

Page 271: Unconscious Memory Representations in Perception

2�0� Alexan��ra�Ben��ixen

Electrical�activit���can�then�be��easure���as�a�variation�of�volta��e�over�ti�e�be-t�een� an� electro��e� of� interest� an��� an� electricall��� neutral� reference� electro��e.�Althou��h�electric�neutralit���is�an�i��ealize���assu��tion,�so�e�feasible�reference��ositions�have�beco�e�co��on��ractice;� these� inclu��e�the�nose,�earlobes,�an����astoi���sites.�The�EEG�si��nal�at�a���iven�electro��e�site�thus�consists�in�a�volta��e���ifference�relative�to�the�reference�electro��e.�For��lacin���the�electro��es�of�inter-est,�the�so-calle���10–20�s��ste��has�been���evelo�e���as�a�stan��ar���sche�e�(Jas�er�1958).�As�illustrate���in�Fi��ure�1,�anato�ical�lan���ar�s�are���efine���b���the�nasion�(Nz),�the�inion�(Iz),�an���the�bilateral��re-auricular��oints�(A1,�A2).�The���istances�bet�een� these� �oints� are� ��ivi��e��� in� ste�s� of� 10� an��� 20%,� res�ectivel��,� to� ��iel���stan��ar���electro��e�locations�labele���b���their���eneral�re��ion��ith�C�(central,��i��-�a���bet�een�nasion�an���inion),�F�(frontal),�F��(fronto�olar),�P�(�arietal),�O�(oc-ci�ital),�an���T�(te��oral).�The�last�character�of�each�label���enotes�the�electro��e��osition�fro��left�to�ri��ht,��ith�‘z’�(‘zero’)�in��icatin���the��i��line�(50%�bet�een�the��re-auricular��oints),�o�����nu�bers���enotin���the�left�he�is�here�(increasin���to�ar��s�the�left��re-auricular��oint),�an���even�nu�bers���enotin���the�ri��ht�he�i-s�here�(increasin���to�ar��s�the�ri��ht��re-auricular��oint).�The�10–20�s��ste���as�later�ex�an��e���(Chatrian�et�al.�1985;�Oostenvel���an���Praa�stra�2001)�to�inclu��e�inter�e��iate�electro��e��ositions.�The��ajorit���of�electro�h��siolo��ical�stu��ies�use�

Figure 1. Place�ent�of�electro��es�in�the�10–20�s��ste��(Jas�er�1958).�Inter�e��iate��osi-tions�are�labele���accor��in��l��,�e.��.,�‘C1’�half�a���bet�een�Cz�an���C3,�an���‘FCz’�half�a���bet�een�Fz�an���Cz�(Chatrian�et�al.�1985;�Oostenvel���an���Praa�stra�2001).

Page 272: Unconscious Memory Representations in Perception

� A��en��ix� 2�1

�lace�ent�sche�es���erive���fro��the�10–20�s��ste�,��hich�facilitates�co��arabil-it���bet�een�stu��ies�(Nu�er�et�al.�1998;�Picton�et�al.�2000).

The�obtaine���EEG�si��nal� reflects� su��ate���electrical�activit��� in� the�brain,��hether�it��as�cause���b���neuronal�activit���(�ainl����osts��na�tic��otentials)�or�b���other�sources�such�as��uscle�activit��,�bloo���flo�,�an���e��e��ove�ents.�The�i��act�of�the�latter�contributors�can�be�re��uce���b���a��ro�riate�recor��in��,�filterin��,�an���artifact�correction��roce��ures.�The�re�ainin���si��nal�is�thou��ht�to�reflect�neuronal�activation��ainl�����enerate��� in�verticall��� ali��ne������ra�i��al� cells� (Allison�et� al.�1986).� Activit��� fro�� other� neuronal� �o�ulations� �i��ht� re�ain� un��etectable� if�their���eo�etric�confi��uration�is�such�that�their�activation���oes�not�su��ate,�or�if�the��o�ulation�is�not�lar��e�enou��h:�S��nchronous�activit���of�about�10.000�neu-rons�is�necessar���for�a�volta��e�to�be��easurable�at�the�scal�.�Therefore,�the�EEG�is�insensitive�to�ar��s��art�of�the�neuronal�activit��,��hich�shoul���be��e�t�in��in���for�its�inter�retation.

The� on��oin��� EEG� si��nal� can� be� use��� for� a� rou��h� assess�ent� of� co��nitive�states�such�as�level�of�vi��ilance.�Yet�the��ajor��otential�of�the�technique�lies�in�anal��zin��� the�chan��es� in� the�EEG�that�are� ti�e-loc�e���to�an� internal�or�exter-nal�event�in�or��er�to�trac��the��rocessin���of�this�event�(an���to�co��are�it�to�the��rocessin���of�a���ifferent�event,�or�of�the�sa�e�event�un��er���ifferent�con��itions).�Unfortunatel��,�si��nal�chan��es�relate���to�event��rocessin���are�s�all�co��are���to�bac���roun���activit���cause���b���on��oin����rocesses�in�the�brain.�Nevertheless,�it�is��ossible�to�se�arate�the�relevant�si��nal�fro��the�bac���roun���(noise)�b����resentin���the�sa�e�event�nu�erous�ti�es,�cuttin���out�EEG�e�ochs�aroun���each�event,�an���calculatin���an�avera��e�of�the�resultin���EEG�se���ents�(Fi��ure�2).�The�avera��in���technique�is�base���on�the�assu��tion�that�an���activit���that�is�uncorrelate���to�the��rocesses�un��er�stu�����shoul���fluctuate�ran��o�l���an���thus�cancel�out,��hereas�the�activit���of�interest�shoul���be�ti�e-loc�e���to�event�onset�an���thus�re�ain�as�a�characteristic��otential�elicite���b���the���iven�event�un��er�the���iven�con��itions.�This��otential�is�calle���the�event-relate����otential�(ERP).

ERP�avera��in���lea��s�to�a���ecrease�of�the�EEG�bac���roun���noise�as�a�function�of�the�square�root�of�the�nu�ber�of�e�ochs�in�the�avera��e.�A�further�i��rove�ent�of�the�si��nal-to-noise�ratio�is�obtaine���b���avera��in���across�the�ERPs�elicite���b���the�sa�e�event�in���ifferent�in��ivi��uals.�So-calle���grand averages�are�usuall���for�e���fro��the�ERPs�of�10�to�20��artici�ants.�The�un��erl��in���assu��tion�here�is�that�of���eneral��s��cholo����:�the�elicitation�of�si�ilar��rocesses�an���thus�si�ilar�brain�ac-tivit���in�all�in��ivi��uals�in�the�sa��le.�The�inter�retation�of�ERP���ata�nee��s�to�ta�e�into�account��otential���istortions�cause���b���avera��in���over�trials�(e.��.,�Atienza�et�al.�2005)�an���over��artici�ants�(e.��.,�Win�ler�et�al.�2001).�Sin��le-trial�ERP�anal��sis�re�ains�challen��in��,���et�hi��hl�����esirable�to�solve�these�inter�retation�issues�an���

Page 273: Unconscious Memory Representations in Perception

2�2� Alexan��ra�Ben��ixen

to�increase�the�flexibilit���of�ERP��rotocols�(e.��.,�Atienza�et�al.�2005;�Jon��s�a�et�al.�2006;�cf.�revie��b���S�encer�2005).

ERPs�are���is�la��e���in�volta��e-over-ti�e���ia��ra�s,��ith�‘0’�usuall�����enotin���event�onset�(Fi��ure�2).�ERP��avefor�s�are�characterize���b���a�sequence�of��osi-tive�an���ne��ative�volta��e���eflections��ith�local��axi�a�(�ea�s).�These���eflections�are�labele���b���their��olarit���(‘P’�for��ositive,�‘N’�for�ne��ative)�an���b���their�or��inal��osition�in�the��avefor��(e.��.,�‘N1’���enotin���the�first�ne��ative��ea�)�or�b���their�latenc���in��s�(e.��.,�‘N100’���enotin���a�ne��ative��ea��occurrin���100��s�after�sti�u-

Figure 2. Sche�atic�illustration�of�EEG��easure�ent�an���ERP���erivation.�A�sti�ulus�is��resente���nu�erous�ti�es.�Si��ns�of�sti�ulus��rocessin���are�har��l���noticeable�in�the�on��oin���EEG,�but�can�be�reveale���b���avera��in���over��ulti�le�EEG�e�ochs�ti�e-loc�e���to�sti�ulus��resentation.�The�si��nal-to-noise�ratio�of�the�resultin���ERP�increases��ith�the�square�root�of�the�nu�ber�of�e�ochs�in�the�avera��e.�Note�that�the�EEG�an���ERP��anels�have���ifferent�scales�an���that�ne��ative�is��lotte���u��ar��s.

Page 274: Unconscious Memory Representations in Perception

� A��en��ix� 2�3

lus�onset).�Further�sub��ivisions�are�in��icate���b���s�all�letters�after�the�na�e�(e.��.,�‘N2a’,�‘N2b’�for�t�o�subco��onents�of�the�secon���ne��ative��ea��in�the��avefor�).�Man���of�these���eflections�beca�e�associate����ith��utative�co��nitive�functions�as�the����ere�consistentl���observe���un��er�certain�ex�eri�ental�con��itions.�Ne��la-bels��ere�chosen�to�reflect�this�functional�as�ect�(e.��.,�‘�is�atch�ne��ativit��’).�Still�alternativel��,�na�es�can�be���eter�ine���to�o��ra�hicall���(e.��.,�‘left�anterior�ne��a-tivit��’)��hen�a���eflection�exhibits�a�consistent�volta��e���istribution�over�the�scal�.

Associatin���volta��e���eflections��ith�a�circu�scribe���to�o��ra�h���an���func-tionalit���lea��s�to�the�notion�of�an�ERP�component.�An�ERP�co��onent�is�a�volt-a��e���eflection��ith�a�characteristic��olarit��,�latenc��,�a��litu��e,�an���to�o��ra�h���(to��ether�calle���‘�or�holo����’)�an����ith�t���ical�con��itions�of�its�elicitation�that�allu��e�to�the�functionalit���of�the�un��erl��in����rocess.�If�an�ex�eri�ental��ani�u-lation�ta�s�into�onl���fe��sensor���or�co��nitive��rocesses�that�are��i��el�����istribute���in�ti�e,�the�relevant�ERP�co��onents�can�be�i��entifie���b����eans�of�the�observ-able�volta��e��ea�s.�Most�of� the� ti�e,�ho�ever,��rocesses��ill�overla��such�that�there�is�no�one-to-one��a��in���bet�een�ERP�co��onents�an���volta��e��ea�s�(see�Luc��2005,� for�a� thorou��h� treat�ent�of� this� issue).� In� this�case,� it��a���hel�� to�co��are�the�volta��e�course�bet�een�t�o�ex�eri�ental�con��itions�b���anal��zin���the���ifference��ave�bet�een�the�ERPs�elicite���in�the�t�o�con��itions.�Assu�in���that�the��rocessin���in�the�t�o�con��itions�is�si�ilar�in��an���res�ects,���eflections�in�the���ifference��ave�reveal�co��onents�that�are�unique�to�one�of�the�con��itions,�or�that���iffer� in�their��ara�eters�bet�een�con��itions.�The�co��arison�of�ERPs�elicite���in���ifferent�ex�eri�ental�con��itions�is�actuall���the��ajor�stren��th�of�the�ERP�technique,�as�it�allo�s�for���eter�inin���(1)��hether�the�brain���istin��uishe���bet�een�t�o�classes�of�events,�an���(2)�at��hich��oint�in�ti�e�the���iver��ence�oc-curre���at�the�latest�(bearin���in��in���that��arts�of�the�neuronal�activit���are�not�ca�ture���b���the�EEG,�it�is�i��ossible�to�exclu��e���ifferentiation�at�earlier�sta��es).�The�inference�of�a��rocessin�����ifference�is�i��ortant�in�its�o�n�ri��ht,�but�it�be-co�es�even��ore�si��nificant��hen�the���ifference�occurs�in�the�latenc���ran��e�of�a��ell-�no�n�co��onent�lin�e����ith�a�s�ecific�function.�An�overvie��of�t���icall���elicite���co��onents�in�co��nitive��ara��i���s�is��rovi��e���in�the�secon���section�of�this�a��en��ix.�

In�a����ition�to�ERPs,�there�is�a�secon����ain�a��roach�for�the�anal��sis�of�EEG�recor��in��s:�the�transfor�ation�into�frequenc���s�ace�to���erive��easures�of�oscil-lator���activit��.�Oscillations�occur�in�the�s�ontaneous�EEG�(i.e.,�uncorrelate����ith�the�ex�eri�ental�con��itions),�but�also�in�s��ste�atic�relation�to�internal�an���exter-nal�events.�Event-relate���oscillator���activit���can�be�foun����ith�or��ithout�ti�e-�an����hase-loc�in���to�the�onset�of�the�relevant�event.�The�for�er�is�calle���‘evo�e��’�oscillator���activit���an���is�visible�in�the�avera��e�ERP.�The�latter�is�calle���‘in��uce��’�activit��;� it� results� fro�� latenc��� shifts� of� oscillator��� bursts� in� sin��le� trials� an����

Page 275: Unconscious Memory Representations in Perception

2�4� Alexan��ra�Ben��ixen

avera��es�out�in�ERP�calculation.�S�ecial�anal��sis��etho��s�are�nee��e���to�reveal�os-cillator���activit��.�Wavelet�anal��sis�is�a��ro�inent�a��roach;�for�illustrative���escri�-tions,�see�Tallon-Bau��r���an���Bertran���(1999)�as��ell�as�Herr�ann�et�al.�(2005).

The� anal��sis� of� oscillator��� event-relate��� activit��� si��nificantl��� co��le�ents�the�infor�ation��rovi��e���b���ERPs.�For�instance,�oscillations�can�reflect�s��nchro-nize���activit����atterns�of�neuronal�asse�blies�co��in���features�that�belon���to�the�sa�e�object�(Ec�horn�et�al.�1988;�Gra���et�al.�1989).�The�frequenc���ban���of�this�s��nchronize���activit���lies�in�the�ran��e�of�30–100�Hz�(the�so-calle���gamma�ban��).�Be��on���feature�bin��in��,�oscillations�in���ifferent�frequenc���ban��s�are�associate����ith�broa��er�co��nitive� functions� inclu��in����erce�tion,� attention,� an����e�or���(Başar�et�al.�2001;�Jensen�et�al.�2007;�War���2003).�For�further���etails�about�the�anal��sis�an��� inter�retation�of�oscillator���EEG�activit��,� the�rea��er� is� referre��� to�En��el�et�al.�(2001),�Herr�ann�et�al.�(2005),�an���War���(2003).

Both�ERPs�an���oscillator���activit���can�be�characterize���b���their�to�o��ra�hical�scal����istribution,�inclu��in���the�i��entification�of�electro��es��ith��axi�u��a�-�litu��es.�Ho�ever,�obtainin���an�EEG�si��nal�at�a���iven�electro��e�is�not�equivalent�to�the�brain�areas�un��erneath�the�electro��e���eneratin���this�si��nal.�In��rinci�le,�an� infinite�nu�ber�of���enerator�confi��urations�can�account� for�a���iven�activa-tion�on�the�scal��(electro�a��netic�inverse��roble�;�Hel�holtz�1853).�This�a�-bi��uit���can�onl���be�overco�e�b���intro��ucin����rior�assu��tions�an���constraints.�Different�source��o��elin���a��roaches�(for�revie�,�see�Michel�et�al.�2004)�have���iel��e���encoura��in���results�fro��source�anal��ses�of�EEG�recor��in��s,�es�eciall����ith�hi��h-��ensit���electro��e��onta��es�(Lantz�et�al.�2003).�Nevertheless,�inferences�on���enerator�localizations�shoul���al�a��s�be�conceive���as�a��roxi�ations.�When�s�atial�infor�ation�is�in�the�focus�of�interest,�the�EEG�can�be�co��le�ente���b���brain�i�a��in���techniques�such�as�functional��a��netic�resonance�i�a��in���(fMRI),��ositron�e�ission�to�o��ra�h���(PET),�an���o�tical�i�a��in���(event-relate���o�tical�si��nal,� EROS;� near-infrare��� s�ectrosco���,� NIRS).� Another� �ossibilit��,� �hich� is�closest�to�the�EEG�in�the�activit���that�it�ca�tures,�is�the��a��netoence�halo��ra��(MEG;�Hä�äläinen�et�al.�1993).�MEG�ex�loits�the�fact�that�ever���current�is�sur-roun��e���b���a��a��netic�fiel��.�In�recor��in���this��a��netic�fiel��,�MEG��rovi��es�in-for�ation� that� can� easil��� be� relate��� to� EEG,� an��� can� be� anal��ze��� b��� the� sa�e��rinci�les.�Ever���ERP�co��onent�has�an�MEG�counter�art,�usuall�����enote���b���a�s�all�‘�’�after�the�na�e�of�the�co��onent�(e.��.,�‘N1�’).�MEG��rovi��es�fine�s�atial�infor�ation�unless���ee��sources�are�involve���(Hillebran���an���Barnes�2002).�Yet�the�su�erior� localization� in�MEG�co�es�at� the�cost�of�a��uch��ore�ex�ensive��easure�ent��roce��ure.

In� s�ite� of� the� �oor� s�atial� resolution� of� the� EEG� an��� so�e� other� li�ita-tions�of�the�technique�(�artl����entione���above;�for�co��rehensive�treat�ents,�see�Han�����2005;�Luc��2005),�i��ortant�conclusions�can�be���erive���fro��electro-

Page 276: Unconscious Memory Representations in Perception

� A��en��ix� 2�5

�h��siolo��ical��easures��hen�inter�retations�are��a��e��ith���ue�caution.�Since�the��ajorit���of�fin��in��s�re�orte���in�this�boo��are�base���on�ERPs,�the�re�ain��er�of�this�a��en��ix�intro��uces��ajor�ERP�co��onents�that�are�t���icall���foun���in�co��nitive��ara��i���s.�The���escri�tion�focuses�on�co��onents�elicite���b���visual�an���au��itor���sti�uli,�as�these�are��uch�better�stu��ie���than�the�other�senses.

10.2 Event-related potential components (ERPs)

ERP�co��onents�can�be���ivi��e���into�ones��rece��in���an���ones�follo�in���a�s�eci-fie��� event� (i.e.,� a� sti�ulus� or� res�onse).� Event-�rece��in��� co��onents� inclu��e�the�rea��iness��otential�(RP)��ith�its�lateralize���as�ect�(LRP),�an���the�contin��ent�ne��ative�variation�(CNV).�The�RP�(also�calle���‘Bereitschafts�otential’)�is�a�slo��ne��ative��otential��rece��in���a�res�onse;�it�in��icates�the��re�aration�of�a�voluntar����ove�ent�(Kornhuber�an���Deec�e�1965).�If�the�res�on��in���han���(left�vs.�ri��ht)�is��no�n�in�a��vance,�the�RP�is�lar��er�at�contralateral�than�at�i�silateral�electro��e�sites,��hich�can�be�ex�loite���to�calculate�the�lateralize���rea��iness��otential�(LRP)�as�a��easure�of���ifferential�res�onse�activation�(Coles�1989).�The�LRP�is�a��o�-erful�tool�for�investi��atin����hether�the�infor�ation��rovi��e���b���a�sti�ulus��as��rocesse���far�enou��h�to�reach�the��otor�s��ste�,�even��hen�a�res�onse�is�never�overtl���initiate���(e.��.,�Verle��er�in�this�volu�e).�The�CNV�li�e�ise�is�a�slo��ne��a-tive��otential��ith��re�arator���or�antici�ator���characteristics�(Walter�et�al.�1964).�The�CNV��rece��es�a�sti�ulus�rather�than�a�res�onse,�but�it��a���also�contain�a��otor�co��onent�in�case�the�sti�ulus�calls�for�a�res�onse�(Brunia�an���van�Boxtel�2001).�For�recent�revie�s�on�antici�ator���ne��ativities�inclu��in���CNV�subco��o-nents�an���their�associate���functionalities,�see�Leuthol���et�al.�(2004),�Macar�an���Vi��al�(2004),�as��ell�as�van�Boxtel�an���Böc�er�(2004).

After� the� �resentation� of� a� sti�ulus,� characteristic� ERP� co��onents� are�elicite���that�can�be�cate��orize���into�‘exo��enous’�an���‘en��o��enous’�ones.�Exo��-enous�co��onents�occur�until�ca.�200��s�after�sti�ulus�onset.�The���are��o��al-it��-s�ecific�an���are�assu�e���to�constitute�an�obli��ator���res�onse�to�the�exter-nal� sti�ulus,� reflectin��� infor�ation� trans�ission� fro�� the� �eri�heral� s��ste��to�cortical�areas�as��ell�as�sensor���cortical��rocessin��.�In�contrast,�en��o��enous�co��onents�are�less�sensitive�to��h��sical�sti�ulus��ro�erties�an����ore�affecte���b���hi��her-or��er��rocesses�resultin���fro���s��cholo��ical�states�of�the��artici�ant�such� as� the� allocation� of� attention.� The� in��e�en��ence� of� en��o��enous� co��o-nents�fro���h��sical�sti�ulus��ara�eters� is��ost�clearl��� illustrate���b���the�fact�that�so�e�can�be�elicite���even�b��� the�unex�ecte���absence�of�an�event�(Sutton�et�al.�1967).�En��o��enous�co��onents�cover�the�ti�e�ran��e�of�100–500��s�after�sti�ulus�onset�(later�res�onses�are�consi��ere���‘slo�’��otentials,��er��in���into�the��

Page 277: Unconscious Memory Representations in Perception

2��� Alexan��ra�Ben��ixen

�re�aration-� an��� antici�ation-relate��� co��onents� �entione��� above� as� event-�rece��in���co��onents).�Yet�the�clear���istinction�bet�een�en��o��enous�an���exo��-enous�co��onents���oes�not�hol��:�Al�ost�all�co��onents�share�characteristics�of�both���rou�s,�bein���influence���b���both�external�an���internal�factors.�Therefore,�the�en��o��enous-exo��enous���i�ension�can�be�seen�as�a�continuu��rather�than�a���iscrete�classification�(Coles�an���Ru�����1995),��ith�co��onents�occurrin���at�lon��er�latencies�usuall���bein���influence���to�a�hi��her���e��ree�b���co��nitive�factors.

The� earl��,� ‘sensor��’� co��onents� inclu��e� the� brainste�� res�onses� (ABR)�(e.��.,�Miller�et�al.�2008),��i��-latenc���res�onses�(MLR)� (e.��.,�Yvert�et�al.�2001),�P1� /� P50� (e.��.,� Na��a�oto� et� al.� 1991)� an��� N1� (Näätänen� an��� Picton� 1987)� in�au��ition,�as��ell�as�C1�(Jeffre��s�an���Axfor���1972a,�1972b),�P1�(e.��.,�Luc��et�al.�1990)�an���N1�(e.��.,�Vo��el�an���Luc��2000)�in�vision�(see�also�Di�Russo�et�al.�2002).�Sti�ulus��rocessin���continues��ith�the�elicitation�of�the�P2�(e.��.,�Cro�le���an���Colrain�2004)�an���N2�(e.��.,�Folstein�an���Van�Petten�2008)�co��onents�bet�een�100� an��� 200��s� �ost-sti�ulus.� At� the� sa�e� ti�e,� so�e� a����itional� ‘co��nitive’�co��onents�e�er��e�that�are�associate����ith��rocessin���s�ecifics���e�en��in���on�attentional�an���contextual� factors.�A�on��� these�are� the��rocessin���ne��ativit���(PN)� (Näätänen,� Gaillar��,� an��� Mänt��salo� 1978),� the� ne��ative� ��ifference� (Nd)�(Hill��ar���et�al.�1973),�an���the�N2pc�(Luc��an���Hill��ar���1994),�all�in��icatin���sus-taine����rocessin�����ifferences���riven�b���selective�attention�(e.��.,�Alho�et�al.�1986;�Muller-Gass�an���Ca��bell�2002).�In�this�volu�e,�both�Czi��ler�an���Verle��er�il-lustrate�ho��the�N2�c�can�be�use���to�ans�er�questions�of�unconscious��e�or��.�Ne��ativities�of�a��ore�transient�nature�are�elicite���b���sti�uli�that���eviate�fro��a���iven�context;�these�inclu��e�the��is�atch�ne��ativit���(MMN)�(Näätänen�et�al.�1978)�an���the�N2b�(Näätänen�et�al.�1982).�MMN� (so�eti�es�also�calle���N2a)�is�elicite����hen�an�au��itor���or�visual�sti�ulus�violates�a�re��ularit���that��as�set�u��b����revious�sti�uli�(Win�ler�2007).�As�MMN�elicitation�is�lar��el���in��e�en-��ent�of�the��artici�ant’s�attention�(Suss�an�2007),�the�co��onent�len��s�itself�to�the�stu�����of�unconscious��heno�ena�(see�Czi��ler;�Huotilainen�an���Teinonen;�Koelsch;�Sht��rov�an���Pulver�üller;�an���Win�ler�in�this�volu�e).�N2b�is��ore���e�en��ent�on�attention;�it�in��icates�that�a�sti�ulus�has�been�classifie���as�a���evi-ant�event�(Nova��et�al.�1990).�If�the���eviant�status�of�an�event�is�not�onl�����efine���b���a�re��ularit���violation�but�also�b���so�e��h��sical���eviation�fro��the��rece��in���sti�uli,���eviance-relate���co��onents�of�a��ore�sensor���character�e�er��e,�such�as�an�N1�enhance�ent�in�au��ition�(e.��.,�Jääs�eläinen�et�al.�2004)�an���the�chan��e-relate����ositivit���(CRP)�in�vision�(Czi��ler�in�this�volu�e;�Ki�ura�et�al.�2006).

Co��onents�associate����ith���eviance���etection�are�often�follo�e���b����osi-tivities�in�the�P3�latenc���ran��e.�These��ositive�co��onents�reflect�a�nu�ber�of���if-ferent��rocesses�an���have�been���iven���ifferent�na�es�in���ifferent�contexts�(P300,�P3a,� P3b,� novelt��� P3,� tar��et� P3,� an��� the� li�e).� P3a� an��� novelt��� P3� in��icate� an�

Page 278: Unconscious Memory Representations in Perception

� A��en��ix� 2�7

involuntar��� shift� of� attention� to�ar��s� a� ��eviatin��� or� novel� event� (Frie���an� et�al.�2001).�Recent���ata�su����est�that�the�t�o�co��onents�are�i��entical�(Polich�an���Cria��o�2006),�their��ajor�characteristic�bein���that�the�attention-ca�turin���event��as� �reviousl��� unatten��e��.� In� contrast,� P3b� is� elicite��� b��� tas�-relevant� sti�uli�(Kni��ht�an���Scabini�1998)�that�require�a���ecision�or�res�onse,�as�the�alternative�ter��‘tar��et�P3’�i��lies.�The�na�e�‘P300’�itself�(�hich�is�so�e�hat��islea��in���in�vie��of�consi��erable�latenc���variations�bet�een�ex�eri�ents)�is��ostl���use���s��n-on���ousl����ith�‘P3b’.�Partl���because�of�its���iversit��,�the���ebate�on�the�functional�si��nificance�of�the�P3�an���its�subco��onents�is�still�unsettle���(e.��.,�Barcelo�et�al.�2006;�Lin��en�2005;�Verle��er�2008).�For�a���iscussion�of�ho��P3��a���be�relate���to�conscious�sti�ulus��erce�tion,�see�Verle��er�(this�volu�e).

In��an���cases,�inter�retations�are�not�base���on�the�elicitation�of�a�co��onent��er�se,�but�on�its��o��ulation�b���ex�eri�ental��ani�ulations.�For�instance,�Sht��-rov�an���Pulver�üller�(this�volu�e)�sho��ho��MMN��o��ulations�can�in��icate�certain�as�ects�of�lan��ua��e��rocessin��.�Stu��ies��ith�lin��uistic��aterial�have�also�reveale��� a� nu�ber� of� s�ecificall��� lan��ua��e-relate��� ERP� co��onents� (the� earl���left�anterior�ne��ativit��,�ELAN;� the� left�anterior�ne��ativit��,�LAN;� the�N400;�an���the�P600),�in��icatin���the���etection�of�s��ntactic�or�se�antic�violations�an���their�re-inte��ration�into�the� lin��uistic�context�(Frie��erici�2002).�Sht��rov�an���Pulver-�üller� (this� volu�e)� �rovi��e� an� overvie�� of� these� co��onents;� Verle��er� (this�volu�e)�illustrates�ho��the�N400�can�be�a��lie���to�stu�����a�areness.�In�the���o-�ain�of��usic��rocessin��,�a� si�ilarl��� s�ecific�co��onent� is���iven�b��� the�earl���ri��ht�anterior�ne��ativit���(ERAN)�in��icatin���violations�of��usical�s��ntax�(Koelsch�in�this�volu�e).�There�are�nu�erous�other�co��onents�that�are�foun���in�s�ecific�contexts.�For�instance,�the�N170�is�a�ne��ative��otential�at�(�re��o�inantl���ri��ht)�lateral�occi�ital�electro��e�sites�elicite���b���face�relative�to�non-face�sti�uli�(cf.�its�a��lication�b���Jacobsen�in�this�volu�e).

Man����ore�co��onents�coul���be��entione��,�but�this�short�outline�ho�efull����rovi��es���ui��ance�alon���the��ain�electro�h��siolo��ical�fin��in��s�in�this�boo�.�For�further���escri�tions�of�relevant�ERP�co��onents,�the�rea��er��a���consult�Coles�an���Ru�����(1995),�Fabiani�et�al.�(2007),�Fonar��ova�Ke���et�al.�(2005),�Luc��(2005),�as��ell�as�Muller-Gass�an���Ca��bell�(2002).

Acknowledgements

This��or���as�su��orte���b���the�Ger�an�Research�Foun��ation�(DFG,�BE�4284/�1-1).

Page 279: Unconscious Memory Representations in Perception

2��� Alexan��ra�Ben��ixen

References

Alho,�K.,�Paavilainen,�P.,�Reini�ainen,�K.,�Sa�s,�M.�an���Näätänen,�R.�(1986).�Se�arabilit���of���if-ferent�ne��ative�co��onents�of�the�event-relate����otential�associate����ith�au��itor���sti�u-lus-�rocessin��.�Psychophysiology,�23,�613–623.

Allison,�T.,�Woo��,�C.�C.�an���McCarth��,�G.�M.�(1986).�The�central�nervous�s��ste�.�In�M.�G.�H.�Coles,�E.�Donchin�an���S.�W.�Por��es�(E��s.),�Psychophysiology: Systems, processes, and ap-plications�(��.�5–25).�Ne��Yor�:�Guilfor��.

Atienza,�M.,�Cantero,�J.�L.�an���Quian�Quiro��a,�R.�(2005).�Precise�ti�in���accounts�for��osttrain-in���slee�-��e�en��ent�enhance�ents�of�the�au��itor����is�atch�ne��ativit��.�Neuroimage,�26,�628–634.

Barcelo,�F.,�Escera,�C.,�Corral,�M.-J.�an���Periañez,�J.�A.�(2006).�Tas��s�itchin���an���novelt����ro-cessin���activate�a�co��on�neural�net�or��for�co��nitive�control.�Journal of Cognitive Neu-roscience,�18,�1734–1748.

Başar,�E.,�Başar-Ero��lu,�C.,�Kara�aş,�S.�an���Schür�ann,�M.�(2001).�Ga��a,�al�ha,���elta,�an���theta�oscillations���overn�co��nitive��rocesses.�International Journal of Psychophysiology,�39,�241–248.

Ber��er,�H.�(1929).�Über���as�Ele�troen�e�halo��ra�����es�Menschen�[On�the�hu�an�electroen-ce�halo��ra�].�Archiv für Psychiatrie und Nervenkrankheiten,�87,�527–570.

Brunia,�C.�H.�M.�an���van�Boxtel,�G.�J.�M.�(2001).�Wait�an���see.�International Journal of Psycho-physiology,�43,�59–75.

Chatrian,�G.�E.,�Lettich,�E.�an���Nelson,�P.�L.� (1985).�Ten��ercent�electro��e�s��ste��for� to�o-��ra�hic�stu��ies�of�s�ontaneous�an���evo�e���EEG�activities.�American Journal of EEG Tech-nology,�25,�83–92.

Coles,�M.�G.�H.�(1989).�Mo��ern��in��-brain�rea��in��:�Ps��cho�h��siolo����,��h��siolo����,�an���co��ni-tion.�Psychophysiology,�26,�251–269.

Coles,�M.�G.�H.�an���Ru����,�M.�D.�(1995).�Event-relate���brain��otentials:�An�intro��uction.�In�M.�D.�Ru�����an���M.�G.�H.�Coles�(E��s.),�Electrophysiology of mind: Event-related brain po-tentials and cognition�(��.�1–26).�Oxfor��:�Oxfor���Universit���Press.

Cro�le��,�K.�E.�an���Colrain,�I.�M.�(2004).�A�revie��of�the�evi��ence�for�P2�bein���an�in��e�en��ent�co��onent��rocess:�A��e,�slee��an����o��alit��.�Clinical Neurophysiology,�115,�732–744.

Di�Russo,�F.,�Martinez,�A.,�Sereno,�M.�I.,�Pitzalis,�S.�an���Hill��ar��,�S.�A.�(2002).�Cortical�sources�of�the�earl���co��onents�of�the�visual�evo�e����otential.�Human Brain Mapping,�15,�95–111.

Ec�horn,�R.,�Bauer,�R.,�Jor��an,�W.,�Brosch,�M.,�Kruse,�W.,�Mun�,�M.�an���Reitboec�,�H.�J.�(1988).�Coherent�oscillations:�A��echanis��of�feature�lin�in���in�the�visual�cortex?�Multi�le�elec-tro��e�an���correlation�anal��ses�in�the�cat.�Biological Cybernetics,�60,�121–130.

En��el,�A.�K.,�Fries,�P.�an���Sin��er,�W.�(2001).�D��na�ic��re��ictions:�Oscillations�an���s��nchron���in�to�-��o�n��rocessin��.�Nature Reviews Neuroscience,�2,�704–716.

Fabiani,�M.,�Gratton,�G.�an���Fe��er�eier,�K.�D.�(2007).�Event-relate���brain��otentials:�Metho��s,�theor��,� an��� a��lications.� In� J.� T.� Cacio��o,� L.� G.� Tassinar��� an��� G.� G.� Berntson� (E��s.),�Handbook of psychophysiology� (3r���e��.,���.�85–119).�Ca�bri����e:�Ca�bri����e�Universit���Press.

Folstein,�J.�R.�an���Van�Petten,�C.�(2008).�Influence�of�co��nitive�control�an����is�atch�on�the�N2�co��onent�of�the�ERP:�A�revie�.�Psychophysiology,�45,�152–170.

Page 280: Unconscious Memory Representations in Perception

� A��en��ix� 2�9

Fonar��ova�Ke��,�A.�P.,�Dove,�G.�O.�an���Ma��uire,�M.�J.�(2005).�Lin�in���brain�aves�to�the�brain:�An�ERP��ri�er.�Developmental Neuropsychology,�27,�183–215.

Frie��erici,�A.�D.�(2002).�To�ar��s�a�neural�basis�of�au��itor���sentence��rocessin��.�Trends in Cog-nitive Sciences,�6,�78–84.

Frie���an,�D.,�C��co�icz,�Y.�M.�an���Gaeta,�H.�(2001).�The�novelt���P3:�An�event-relate���brain��otential�(ERP)�si��n�of�the�brain’s�evaluation�of�novelt��.�Neuroscience and Biobehavioral Reviews,�25,�355–373.

Gra��,�C.�M.,�Köni��,�P.,�En��el,�A.�K.�an���Sin��er,�W.�(1989).�Oscillator���res�onses�in�cat�visual�cortex�exhibit�inter-colu�nar�s��nchronization��hich�reflects���lobal�sti�ulus��ro�erties.�Nature,�338,�334–337.

Hä�äläinen,�M.,�Hari,�R.,�Il�onie�i,�R.�J.,�Knuutila,�J.�an���Lounas�aa,�O.�V.�(1993).�Ma��neto-ence�halo��ra�h���–�theor��,�instru�entation,�an���a��lications�to�noninvasive�stu��ies�of�the��or�in���hu�an�brain.�Reviews of Modern Physics,�65,�413–497.

Han����,� T.� C.� (E��.).� (2005).� Event-related potentials: A methods handbook.� Ca�bri����e:� MIT�Press.

Hel�holtz,�H.�(1853).�Ueber�eini��e�Gesetze���er�Vertheilun���ele�trischer�Strö�e�in��ör�erli-chen� Leitern� �it� An�en��un��� auf� ��ie� thierisch-ele�trischen� Versuche� [So�e� la�s� con-cernin��� the� ��istribution� of� electric� currents� in� volu�e� con��uctors� �ith� a��lications� to�ex�eri�ents�on�ani�al�electricit��].�Annalen der Physik,�165,�211–233.

Herr�ann,�C.�S.,�Gri��utsch,�M.�an���Busch,�N.�A.�(2005).�EEG�oscillations�an����avelet�anal��sis.�In�T.�C.�Han�����(E��.),�Event-related potentials: A methods handbook�(��.�229–259).�Ca�-bri����e:�MIT�Press.

Hillebran��,�A.�an���Barnes,�G.�R.�(2002).�A�quantitative�assess�ent�of�the�sensitivit���of��hole-hea���MEG�to�activit���in�the�a��ult�hu�an�cortex.�Neuroimage,�16,�638–650.

Hill��ar��,�S.�A.,�Hin�,�R.�F.,�Sch�ent,�V.�L.�an���Picton,�T.�W.�(1973).�Electrical�si��ns�of�selective�attention�in�the�hu�an�brain.�Science,�182,�177–180.

Jääs�eläinen,�I.�P.,�Ahveninen,�J.,�Bon�assar,�G.,�Dale,�A.�M.,�Il�onie�i,�R.�J.,�Levänen,�S.,�Lin,�F.-H.,�Ma��,�P.,�Melcher,�J.,�Stufflebea�,�S.,�Tiitinen,�H.�an���Belliveau,�J.�W.�(2004).�Hu�an��osterior�au��itor���cortex���ates�novel�soun��s�to�consciousness.�Proceedings of the National Academy of Sciences of the United States of America,�101,�6809–6814.

Jas�er,�H.�H.�(1958).�The�ten-t�ent���electro��e�s��ste��of�the�International�Fe��eration.�Electro-encephalography and Clinical Neurophysiology,�10,�371–375.

Jeffre��s,�D.�A.�an���Axfor��,�J.�G.�(1972a).�Source�locations�of��attern-s�ecific�co��onents�of�hu-�an�visual�evo�e����otentials.�I.�Co��onent�of�striate�cortical�ori��in.�Experimental Brain Research,�16,�1–21.

Jeffre��s,�D.�A.�an���Axfor��,�J.�G.�(1972b).�Source�locations�of��attern-s�ecific�co��onents�of�hu-�an�visual�evo�e����otentials.�II.�Co��onent�of�extrastriate�cortical�ori��in.�Experimental Brain Research,�16,�22–40.

Jensen,�O.,�Kaiser,�J.�an���Lachaux,�J.-P.�(2007).�Hu�an���a��a-frequenc���oscillations�associ-ate����ith�attention�an����e�or��.�Trends in Neurosciences,�30,�317–324.

Jon��s�a,�M.�L.�A.,�Eichele,�T.,�Van�Rijn,�C.�M.,�Coenen,�A.�M.�L.,�Hu����ahl,�K.,�Nor��b��,�H.�an���Quian�Quiro��a,�R.�(2006).�Trac�in����attern�learnin����ith�sin��le-trial�event-relate����oten-tials.�Clinical Neurophysiology,�117,�1957–1973.

Ki�ura,�M.,�Kata��a�a,�J.�an���Murohashi,�H.�(2006).�Probabilit��-in��e�en��ent�an���-��e�en��ent�ERPs�reflectin���visual�chan��e���etection.�Psychophysiology,�43,�180–189.

Kni��ht,�R.�T.�an���Scabini,�D.�(1998).�Anato�ic�bases�of�event-relate����otentials�an���their�rela-tionshi��to�novelt�����etection�in�hu�ans.�Journal of Clinical Neurophysiology,�15,�3–13.

Page 281: Unconscious Memory Representations in Perception

270� Alexan��ra�Ben��ixen

Kornhuber,�H.�H.�an���Deec�e,�L.� (1965).�Hirn�otentialän��erun��en�bei�Will�ürbe�e��un��en�un����assiven�Be�e��un��en���es�Menschen:�Bereitschafts�otential�un���reafferente�Poten-tiale� [Brain��otential� chan��es��ith�voluntar���an����assive�hu�an��ove�ent:�Rea��iness��otential�an���reafferent��otentials].�Pflügers Archiv,�284,�1–17.

Lantz,�G.,�Grave���e�Peralta,�R.,�S�inelli,�L.,�Seec�,�M.�an���Michel,�C.�M.�(2003).�E�ile�tic�source�localization� �ith� hi��h� ��ensit��� EEG:� Ho�� �an��� electro��es� are� nee��e��?� Clinical Neuro-physiology,�114,�63–69.

Leuthol��,�H.,�So��er,�W.�an���Ulrich,�R.�(2004).�Pre�arin���for�action:�Inferences�fro��CNV�an���LRP.�Journal of Psychophysiology,�18,�77–88.

Lin��en,�D.�E.�J.�(2005).�The�P300:�Where�in�the�brain�is�it��ro��uce���an����hat���oes�it�tell�us?�Neuroscientist,�11,�563–576.

Luc�,� S.� J.� (2005).� An introduction to the event-related potential technique.� Ca�bri����e,� MA:�MIT�Press.

Luc�,�S.�J.,�Heinze,�H.�J.,�Man��un,�G.�R.�an���Hill��ar��,�S.�A.�(1990).�Visual�event-relate����o-tentials�in��ex�focuse���attention��ithin�bilateral�sti�ulus�arra��s.�II.�Functional���issocia-tion�of�P1�an���N1�co��onents.�Electroencephalography and Clinical Neurophysiology,�75,�528–542.

Luc�,�S.�J.�an���Hill��ar��,�S.�A.�(1994).�S�atial�filterin�����urin���visual�search:�Evi��ence�fro��hu-�an�electro�h��siolo����.�Journal of Experimental Psychology: Human Perception and Perfor-mance,�20,�1000–1014.

Macar,�F.�an���Vi��al,�F.�(2004).�Event-relate����otentials�as�in��ices�of�ti�e��rocessin��:�A�revie�.�Journal of Psychophysiology,�18,�89–104.

Michel,�C.�M.,�Murra��,�M.�M.,�Lantz,�G.,�Gonzalez,� S.,� S�inelli,�L.� an���Grave���e�Peralta,�R.�(2004).�EEG�source�i�a��in��.�Clinical Neurophysiology,�115,�2195–2222.

Miller,�C.�A.,�Bro�n,�C.�J.,�Abbas,�P.�J.�an���Chi,�S.-L.�(2008).�The�clinical�a��lication�of��oten-tials�evo�e���fro��the��eri�heral�au��itor���s��ste�.�Hearing Research,�242,�184–197.

Muller-Gass,�A.�an���Ca��bell,�K.�(2002).�Event-relate����otential��easures�of�the�inhibition�of�infor�ation��rocessin��:�I.�Selective�attention�in�the��a�in���state.�International Journal of Psychophysiology,�46,�177–195.

Näätänen,�R.,�Gaillar��,�A.�W.�K.�an���Mänt��salo,�S.�(1978).�Earl���selective-attention�effect�on�evo�e����otential�reinter�rete��.�Acta Psychologica,�42,�313–329.

Näätänen,�R.�an���Picton,�T.�(1987).�The�N1��ave�of�the�hu�an�electric�an����a��netic�res�onse�to� soun��:� A� revie�� an��� an� anal��sis� of� the� co��onent� structure.� Psychophysiology,� 24,�375–425.

Näätänen,�R.,�Si��son,�M.�an���Loveless,�N.�E.�(1982).�Sti�ulus���eviance�an���evo�e����oten-tials.�Biological Psycholog��,�14,�53–98.

Na��a�oto,�H.�T.,�A��ler,�L.�E.,�Wal��o,�M.�C.,�Griffith,� J.�an���Free���an,�R.� (1991).�Gatin���of�au��itor��� res�onse� in�Schizo�hrenics� an���nor�al� controls.�Effects�of� recor��in��� site� an���sti�ulation�interval�on�the�P50��ave.�Schizophrenia Research,�4,�31–40.

Nova�,�G.�P.,�Ritter,�W.,�Vau��han,�H.�G.�an���Wiznitzer,�M.�L.�(1990).�Differentiation�of�ne��ative�event-relate��� �otentials� in� an� au��itor��� ��iscri�ination� tas�.� Electroencephalography and Clinical Neurophysiology,�75,�255–275.

Nu�er,�M.�R.,�Co�i,�G.,�E�erson,�R.,�Fu��lsan��-Fre��eri�sen,�A.,�Guérit,� J.�M.,�Hinrichs,�H.,�I�e��a,�A.,�Luccas,�F.�J.�C.�an���Ra��elsbur��er,�P.�(1998).�IFCN�stan��ar��s�for���i��ital�recor��-in���of�clinical�EEG.�Electroencephalography and Clinical Neurophysiology,�106,�259–261.

Oostenvel��,�R.�an���Praa�stra,�P.�(2001).�The�five��ercent�electro��e�s��ste��for�hi��h-resolution�EEG�an���ERP��easure�ents.�Clinical Neurophysiology,�112,�713–719.

Page 282: Unconscious Memory Representations in Perception

� A��en��ix� 271

Picton,�T.�W.,�Bentin,�S.,�Ber��,�P.,�Donchin,�E.,�Hill��ar��,�S.�A.,�Johnson,�R.,�Jr.,�Miller,�G.�A.,�Rit-ter,�W.,�Ruch�in,�D.�S.,�Ru����,�M.�D.�an���Ta��lor,�M.�J.�(2000).�Gui��elines�for�usin���hu�an�event-relate����otentials�to�stu�����co��nition:�Recor��in���stan��ar��s�an����ublication�criteria.�Psychophysiology,�37,�127–152.

Polich,�J.�an���Cria��o,�J.�R.�(2006).�Neuro�s��cholo�����an���neuro�har�acolo�����of�P3a�an���P3b.�International Journal of Psychophysiology,�60,�172–185.

Ru����,�M.�D.�an���Coles,�M.�G.�H.�(E��s.).�(1995).�Electrophysiology of mind: Event-related brain potentials and cognition.�Oxfor��:�Oxfor���Universit���Press.

S�encer,� K.� M.� (2005).� Avera��in��,� ��etection,� an��� classification� of� sin��le-trial� ERPs.� In� T.�C.�Han����� (E��.),� Event-related potentials: A methods handbook� (��.� 209–227).� Ca�bri����e:�MIT�Press.

Suss�an,�E.�S.�(2007).�A�ne��vie��on�the�MMN�an���attention���ebate:�The�role�of�context�in��rocessin���au��itor���events.�Journal of Psychophysiology,�21,�164–175.

Sutton,�S.,�Tuetin��,�P.,�Zubin,�J.�an���John,�E.�R.�(1967).�Infor�ation���eliver���an���the�sensor���evo�e����otential.�Science,�155,�1436–1439.

Tallon-Bau��r��,�C.�an���Bertran��,�O.�(1999).�Oscillator�����a��a�activit���in�hu�ans�an���its�role�in�object�re�resentation.�Trends in Cognitive Sciences,�3,�151–162.

van�Boxtel,�G.�J.�M.�an���Böc�er,�K.�B.�E.�(2004).�Cortical��easures�of�antici�ation.�Journal of Psychophysiology,�18,�61–76.

Verle��er,�R.�(2008).�P3b:�To�ar��s�so�e���ecision�about��e�or��.�Clinical Neurophysiology,�119,�968–970.

Vo��el,�E.�K.�an���Luc�,�S.�J.�(2000).�The�visual�N1�co��onent�as�an�in��ex�of�a���iscri�ination��rocess.�Psychophysiology,�37,�190–203.

Walter,�W.�G.,�Coo�er,�R.,�Al��ri����e,�V.�J.,�McCallu�,�W.�C.�an���Winter,�A.�L.�(1964).�Contin-��ent�ne��ative�variation:�An�electric�si��n�of�sensori�otor�association�an���ex�ectanc���in�the�hu�an�brain.�Nature,�203,�380–384.

War��,�L.�M.�(2003).�S��nchronous�neural�oscillations�an���co��nitive��rocesses.�Trends in Cogni-tive Sciences,�7,�553–559.

Win�ler,�I.�(2007).�Inter�retin���the��is�atch�ne��ativit��.�Journal of Psychophysiology,�21,�147–163.

Win�ler,�I.,�Schrö��er,�E.�an���Co�an,�N.�(2001).�The�role�of�lar��e-scale��e�or���or��anization�in�the��is�atch�ne��ativit���event-relate���brain��otential.�Journal of Cognitive Neuroscience,�13,�59–71.

Yvert,�B.,�Crouzeix,�A.,�Bertran��,�O.,�Seither-Preisler,�A.�an���Pantev,�C.�(2001).�Multi�le�su�ra-te��oral�sources�of��a��netic�an���electric�au��itor���evo�e����i����le�latenc���co��onents�in�hu�ans.�Cerebral Cortex,�11,�411–423.

Page 283: Unconscious Memory Representations in Perception
Page 284: Unconscious Memory Representations in Perception

Aaesthetic��rocessin��� 245–246,�

251articulator����ove�ents� 197attentional�blin�� 37,�48–52,�54,�

57–58,�63–64,�114au��io-visual� 187au��itor���objects� 71,�73,�77,�80,�

85,�87–88,�96,�99,�223,�226,�235au��itor���strea�in��� 74,�80–81,�

83–85,�91–95,�168,�170–171

BBa��esian

belief��ro�a��ation� 155,�157,�175

inference� 148–149,�151,�153,�155–158,�171,�173–175,�177,�263

blin��si��ht� 24,�26

Cchor��-sequence� 230co��nitive�fluenc��� 251coherence�fiel��� 112contralateral�ne��lect� 24

D��o�nstrea���rocesses� 43

Eelectroence�halo��ra�h��� 153,�

214e�be����e��-�rocesses��o��el� 15event-relate����otential�(ERP)� ��

� 38–40,�43,�46–49,�51–52,��� 54,�56,�58,�60,�62,�64,��� 73–76,�84,�98,�113,�126,��� 129,�140,�184,�209,�211,�220,��

� 226–227,�229–230,�233–�� 234,�249–250,�255,�261–265Bereitschafts�otential� 265,�

270C1� 260,�266CNV�(contin��ent�ne��ative�

variation)� 265,�270–271ELAN�(earl���left�anterior�

ne��ativit��)� 181,�183,�193,�211,�229,�267

ERAN� 210–212,�215,�219–226,�228–237,�267

LAN� 226,�267LRP�(lateralize���rea��iness�

�otential)� 58,�60–62,�265MMN�(�is�atch�ne��ativit��)� �

74–80,�82–85,�87–90,�94–100,�117,�119,�121,�124,�137–140,�182–185,�209,�211,�219,�222–226,�232,�234,�249,�266–268

N1� 38,�46,�48,�51–52,�62,�183,�262,�266

N170� 68,�249,�267N2� 41,�43,�46,�50–51,�60,�

63,�266N250� 44N2b� 263,�266N2�c� 46,�48,�53–56,�60,�63,�

116,�266N400� 51–52,�67–68,�137,�

181,�183–184,�190,�198–199,�211,�267

N450� 44P1� 41–44,�46,�48,�51–52,�60,�

62,�183,�229,�266P250� 52P2� 51–52,�211,�266P300� 57,�137,�153,�204,�211,�

266–267

P3� 38–44,�49–51,�54,�57–58,�63–64,�233,�266–267

P3a� 115,�137,�266P3b� 115,�137,�233,�266–267P600� 181,�183–184,�193,�199,�

267PN�(�rocessin���ne��ativit��)� �

266novelt���P3� 266tar��et�P3� 266–267

executive�function� 3

Ffeature�bin��in��� 77–80,�120,�

141,�264feature�conjunctions� 78–80,�

148

G��enerative��o��elin��� 151��lobal��or�s�ace�theor��� 10,�23

Hhar�onic�hierarch��� 214

Iinattentional�blin��ness� 18,�113intracranial�recor��in��s� 39

Llexical��rocessin��� 180–181,�197

M�a��netic�resonance�i�a��in���

(fMRI)� 15,�116,�152,�159,�188,�197,�214,�228–229,�233,�235,�264

�a��netoence�halo��ra�h���(MEG)� 51,�57,�139–140,�180,�188–189,�191–192,�197,�217,�228–229,�264

Index

Page 285: Unconscious Memory Representations in Perception

274� Unconscious�Me�or���Re�resentations�in�Perce�tion

Mar�ov��o��els� 162�e�or��� 1–25,�27–30,�69–70,��

� 80,�82,�109–113,�123–124,��� 157,�189,�247–253�au��itor���sensor��� 75–76,�

104,�137,�182,�237–238e�iso��ic� 2,�4,�6,�31,�77,�251,�

253i��licit� 110,�117,�123se�antic� 6,�12–13,�22,�113–

114,�179–181,�186,�189–190,�195,�199,�248–251,�267

sensor��� 5,�8,�14,�76,�109,�137–138,�235,�271

short-ter��conce�tual� 113,�130

�or�in��� 1–2,�4–19,�21,�23–25,�27–28,�48,�63

�or�ho-s��ntactic�infor�ation��rocessin��� 191

�ulti-scale�anal��sis� 149–150,�171

�usic-s��ntactic� 212

Nnea�olitan�chor��s� 210,�219–

221,�231,�234–235neonatal�au��itor���s��ste�� 138–

139

Oobject�relate��� 84o����ball�sti�uli� 40,�65

P�erce�tual�object� 72–73,�77,�

98–99,�147�erce�tual�or��anisation� 147–

148�honetic�an����honolo��ical�

�is�atch� 187�honolo��ical� 6,�103,�186,�195�re��ictive�co��in��� 154–157,�

162,�176�ri�in��� 20–22,�198�ri�itive�intelli��ence� 107,�123�robabilistic� 148,�151,�153,�

158–159

�roce��ural��no�le����e� 249�seu��o�or��s� 135–136,�138,�186,�

194,�197–198

Rre��ularit���violations� 76rule-learnin��� 135

Sso�atoto�����o��el� 189statistical�learnin��� 135–136,�145strea��se��re��ation� 84–85,�92,�

96,�140–141subli�inal��erce�tion� 22,�65

Tte��orall����ersistent�

re�resentations� 147te��orar���buffer� 1

Vvisuo-s�atial�buffers� 4

Page 286: Unconscious Memory Representations in Perception

Advances in Consciousness ResearchA complete list of titles in this series can be found on the publishers’ website, www.benjamins.com

79 Perry, elaine, Daniel Collerton, Heather AsHton and Fiona e.n. leBeAu (eds.):ConsciousNonconsciousConnections.NeuroscienceinMind.Forthcoming

78 Czigler, istván and istván Winkler (eds.):UnconsciousMemoryRepresentationsinPerception.Processesandmechanismsinthebrain.2010.x, 274 pp.

77 gloBus, gordon g.:TheTransparentBecomingofWorld.Acrossingbetweenprocessphilosophyandquantumneurophilosophy.2009.xiii, 169 pp.

76 Bråten, stein:TheIntersubjectiveMirrorinInfantLearningandEvolutionofSpeech.2009.xxii, 351 pp.75 skrBinA, David (ed.):MindthatAbides.Panpsychisminthenewmillennium.2009.xiv, 401 pp.74 CAñAmero, lola and ruth Aylett (eds.):AnimatingExpressiveCharactersforSocialInteraction.

2008.xxiii, 296 pp.73 HArDCAstle, Valerie gray:ConstructingtheSelf.2008.xi, 186 pp.72 JAnzen, greg:TheReflexiveNatureofConsciousness.2008.vii, 186 pp.71 krois, John michael, mats rosengren, Angela steiDele and Dirk WesterkAmP (eds.):

EmbodimentinCognitionandCulture.2007.xxii, 304 pp.70 rAkoVer, sam s.:ToUnderstandaCat.Methodologyandphilosophy.2007.xviii, 253 pp.69 kuCzynski, John-michael:ConceptualAtomismandtheComputationalTheoryofMind.Adefenseof

content-internalismandsemanticexternalism.2007.x, 524 pp.68 Bråten, stein (ed.):OnBeingMoved.Frommirrorneuronstoempathy.2007.x, 333 pp.67 AlBertAzzi, liliana (ed.):VisualThought.Thedepictivespaceofperception.2006.xii, 380 pp.66 VeCCHi, tomaso and gabriella Bottini (eds.):ImageryandSpatialCognition.Methods,modelsand

cognitiveassessment.2006.xiv, 436 pp.65 sHAumyAn, sebastian:Signs,Mind,andReality.Atheoryoflanguageasthefolkmodeloftheworld.

2006.xxvii, 315 pp.64 HurlBurt, russell t. and Christopher l. HeAVey:ExploringInnerExperience.Thedescriptive

experiencesamplingmethod.2006.xii, 276 pp.63 BArtsCH, renate:MemoryandUnderstanding.ConceptformationinProust’sA la recherche du temps

perdu.2005.x, 160 pp.62 De Preester, Helena and Veroniek knoCkAert (eds.):BodyImageandBodySchema.

Interdisciplinaryperspectivesonthebody.2005.x, 346 pp.61 ellis, ralph D.:CuriousEmotions.Rootsofconsciousnessandpersonalityinmotivatedaction.2005.

viii, 240 pp.60 DietriCH, eric and Valerie gray HArDCAstle:Sisyphus’sBoulder.Consciousnessandthelimitsof

theknowable.2005.xii, 136 pp.59 zAHAVi, Dan, Thor grünBAum and Josef PArnAs (eds.):TheStructureandDevelopmentofSelf-

Consciousness.Interdisciplinaryperspectives.2004.xiv, 162 pp.58 gloBus, gordon g., karl H. PriBrAm and giuseppe Vitiello (eds.):BrainandBeing.Atthe

boundarybetweenscience,philosophy,languageandarts.2004.xii, 350 pp.57 WilDgen, Wolfgang:TheEvolutionofHumanLanguage.Scenarios,principles,andculturaldynamics.

2004.xii, 240 pp.56 gennAro, rocco J. (ed.):Higher-OrderTheoriesofConsciousness.AnAnthology.2004.xii, 371 pp.55 Peruzzi, Alberto (ed.):MindandCausality.2004.xiv, 235 pp.54 BeAuregArD, mario (ed.):Consciousness,EmotionalSelf-RegulationandtheBrain.2004.xii, 294 pp.53 HAtWell, yvette, Arlette streri and edouard gentAz (eds.):TouchingforKnowing.Cognitive

psychologyofhapticmanualperception.2003.x, 322 pp.52 nortHoFF, georg:PhilosophyoftheBrain.Thebrainproblem.2004.x, 433 pp.51 Droege, Paula:CagingtheBeast.Atheoryofsensoryconsciousness.2003.x, 183 pp.50 gloBus, gordon g.:QuantumClosuresandDisclosures.Thinking-togetherpostphenomenologyand

quantumbraindynamics.2003.xxii, 200 pp.49 osAkA, naoyuki (ed.):NeuralBasisofConsciousness.2003.viii, 227 pp.48 Jiménez, luis (ed.):AttentionandImplicitLearning.2003.x, 385 pp.47 Cook, norman D.:ToneofVoiceandMind.Theconnectionsbetweenintonation,emotion,cognitionand

consciousness.2002.x, 293 pp.46 mAteAs, michael and Phoebe sengers (eds.):NarrativeIntelligence.2003.viii, 342 pp.

Page 287: Unconscious Memory Representations in Perception

45 DokiC, Jérôme and Joëlle Proust (eds.):SimulationandKnowledgeofAction.2002.xxii, 271 pp.44 moore, simon C. and mike oAksForD (eds.):EmotionalCognition.Frombraintobehaviour.2002.

vi, 350 pp.43 DePrAz, nathalie, Francisco J. VArelA and Pierre VermersCH:OnBecomingAware.Apragmatics

ofexperiencing.2003.viii, 283 pp.42 stAmenoV, maxim i. and Vittorio gAllese (eds.):MirrorNeuronsandtheEvolutionofBrainand

Language.2002.viii, 392 pp.41 AlBertAzzi, liliana (ed.):UnfoldingPerceptualContinua.2002.vi, 296 pp.40 mAnDler, george:ConsciousnessRecovered.Psychologicalfunctionsandoriginsofconsciousthought.

2002.xii, 142 pp.39 BArtsCH, renate:ConsciousnessEmerging.Thedynamicsofperception,imagination,action,memory,

thought,andlanguage.2002.x, 258 pp.38 sAlzArulo, Piero and gianluca FiCCA (eds.):AwakeningandSleep–WakeCycleAcrossDevelopment.

2002.vi, 283 pp.37 Pylkkänen, Paavo and tere VADén (eds.):DimensionsofConsciousExperience.2001.xiv, 209 pp.36 Perry, elaine, Heather AsHton and Allan H. young (eds.):NeurochemistryofConsciousness.

Neurotransmittersinmind.WithaforewordbySusanGreenfield.2002.xii, 344 pp.35 mc keVitt, Paul, seán Ó nuAlláin and Conn mulViHill (eds.):Language,VisionandMusic.

Selectedpapersfromthe8thInternationalWorkshopontheCognitiveScienceofNaturalLanguageProcessing,Galway,1999.2002.xii, 433 pp.

34 Fetzer, James H. (ed.):ConsciousnessEvolving.2002.xx, 253 pp.33 yAsue, kunio, mari JiBu and tarcisio DellA sentA (eds.):NoMatter,NeverMind.Proceedingsof

TowardaScienceofConsciousness:Fundamentalapproaches,Tokyo1999.2002.xvi, 391 pp.32 Vitiello, giuseppe:MyDoubleUnveiled.Thedissipativequantummodelofbrain.2001.xvi, 163 pp.31 rAkoVer, sam s. and Baruch CAHlon:FaceRecognition.Cognitiveandcomputationalprocesses.

2001.x, 306 pp.30 Brook, Andrew and richard C. DeViDi (eds.):Self-ReferenceandSelf-Awareness.2001.viii, 277 pp.29 VAn looCke, Philip (ed.):ThePhysicalNatureofConsciousness.2001.viii, 321 pp.28 zACHAr, Peter:PsychologicalConceptsandBiologicalPsychiatry.Aphilosophicalanalysis.2000.

xx, 342 pp.27 gillett, grant r. and John mcmillAn:ConsciousnessandIntentionality.2001.x, 265 pp.26 Ó nuAlláin, seán (ed.):SpatialCognition.Foundationsandapplications.2000.xvi, 366 pp.25 BACHmAnn, talis:MicrogeneticApproachtotheConsciousMind.2000.xiv, 300 pp.24 roVee-Collier, Carolyn, Harlene HAyne and michael ColomBo:TheDevelopmentofImplicit

andExplicitMemory.2000.x, 324 pp.23 zAHAVi, Dan (ed.):ExploringtheSelf.Philosophicalandpsychopathologicalperspectivesonself-

experience.2000.viii, 301 pp.22 rossetti, yves and Antti reVonsuo (eds.):BeyondDissociation.Interactionbetweendissociated

implicitandexplicitprocessing.2000.x, 372 pp.21 Hutto, Daniel D.:BeyondPhysicalism.2000.xvi, 306 pp.20 kunzenDorF, robert g. and Benjamin WAllACe (eds.):IndividualDifferencesinConscious

Experience.2000.xii, 412 pp.19 DAutenHAHn, kerstin (ed.):HumanCognitionandSocialAgentTechnology.2000.xxiv, 448 pp.18 PAlmer, gary B. and Debra J. oCCHi (eds.):LanguagesofSentiment.Culturalconstructionsof

emotionalsubstrates.1999.vi, 272 pp.17 Hutto, Daniel D.:ThePresenceofMind.1999.xiv, 252 pp.16 ellis, ralph D. and natika neWton (eds.):TheCaldronofConsciousness.Motivation,affectandself-

organization—Ananthology.2000.xxii, 276 pp.15 CHAllis, Bradford H. and Boris m. VeliCHkoVsky (eds.):StratificationinCognitionand

Consciousness.1999.viii, 293 pp.14 sHeets-JoHnstone, maxine:ThePrimacyofMovement.1999.xxxiv, 583 pp.13 VelmAns, max (ed.):InvestigatingPhenomenalConsciousness.Newmethodologiesandmaps.2000.

xii, 381 pp.

Page 288: Unconscious Memory Representations in Perception

12 stAmenoV, maxim i. (ed.):LanguageStructure,DiscourseandtheAccesstoConsciousness.1997.xii, 364 pp.

11 Pylkkö, Pauli:TheAconceptualMind.Heideggerianthemesinholisticnaturalism.1998.xxvi, 297 pp.10 neWton, natika:FoundationsofUnderstanding.1996.x, 211 pp.9 Ó nuAlláin, seán, Paul mc keVitt and eoghan mac Aogáin (eds.):TwoSciencesofMind.

Readingsincognitivescienceandconsciousness.1997.xii, 490 pp.8 grossenBACHer, Peter g. (ed.):FindingConsciousnessintheBrain.Aneurocognitiveapproach.2001.

xvi, 326 pp.7 mAC CormAC, earl and maxim i. stAmenoV (eds.):FractalsofBrain,FractalsofMind.Insearchofa

symmetrybond.1996.x, 359 pp.6 gennAro, rocco J.:ConsciousnessandSelf-Consciousness.Adefenseofthehigher-orderthoughttheory

ofconsciousness.1996.x, 220 pp.5 stuBenBerg, leopold:ConsciousnessandQualia.1998.x, 368 pp.4 HArDCAstle, Valerie gray:LocatingConsciousness.1995.xviii, 266 pp.3 JiBu, mari and kunio yAsue:QuantumBrainDynamicsandConsciousness.Anintroduction.1995.

xvi, 244 pp.2 ellis, ralph D.:QuestioningConsciousness.Theinterplayofimagery,cognition,andemotioninthe

humanbrain.1995.viii, 262 pp.1 gloBus, gordon g.:ThePostmodernBrain.1995.xii, 188 pp.