the micro in the macro using ltra - organization for … courses... · 2017-07-05 · finding the...

FINDING THE MICRO IN THE MACRO USING ULTRA-HIGH RESOLUTION MR IMAGING

OHBM Educational Course “Why it all comes back to Anatomy”, Vancouver, Canada, June 25, 2017

Rainer Goebel

Maastricht Brain Imaging Center (M-BIC), Dept. of Cognitive Neuroscience, Maastricht University& National Institute of Neuroscience (NIN) of the Royal Netherlands Academy of Arts and

Sciences, Amsterdam, The Netherlands & CEO Brain Innovation

7T

• From Areas and Networks to Columns and Layers - Understanding visual perception and cognition at multiple scales- Towards mesoscopic-scale neuroimaging at 7 Tesla and beyond

• Cracking Coding Principles at Columnar Level- Mapping columnar-level features using sub-millimeter fMRI- The challenge ahead: Unraveling unknown feature codes- Mesoscopic correlates of visual cognition: Perceptual switches

• Extraction of Information from Laminar Profiles - Separating bottom-up from top-down information flow- Applications to attention and predictive coding tasks

• Summary and Conclusions

Overview

Brain

What are elementary mental/neural functional components?Understand categorical representations in “modules of the mind” and how they are embedded in areas of the brain (e.g. face area vs house/place area). How do basic components interact and unfold over time? Understand communication between brain areas, i.e. direction of information flow (anatomical and effective connectivity).

Sensory input Behavior

Multi-Scale Functional Organization of the Human BrainMacroscopic Level: Specialized Areas and Large-Scale Networks

Brain

What are elementary mental/neural functional components?Understand “alphabet” of basic features within specialized brain areas (e.g. face area) and how specific “words” (e.g. individual faces) are encoded as distributed patterns across mapped features. How do basic components interact and develop over time?Understand interactions between features, e.g. how complex feature codes in higher areas emerge from simpler features in lower areas.

Sensory input Behavior

Multi-Scale Functional Organization of the Human BrainMesoscopic Level: Features Coded Within Specialized Areas

Goebel et al. (2012) The Visual System. In G.Paxinos,& J.K. Mai (Eds). The Human Nervous System 2nd edition.

“What” pathway(object recognition)

“Where” pathway(spatial attention,motion, orientation

in space…)

Investigating the Brain at Macroscopic Level: Areas & Networks

V5/hMT – Motion EBA – Bodies LOC – Objects V4 – Colors FFA – Faces PPA – Places VWFA – Words

RH LH

LH RH

Selected Mid-Level Visual Areas Mapped in Individual BrainMapping specialised areas in individual brains is a prerequisite to investigate

internal functional organisation using sub-millimetre fMRI

Retinotopic Mapping of Early Visual Areas using the Population Receptive Field (pRF) Estimation Technique (based on Dumoulin & Wandell, 2008)

Goebel R (2015). Functional organization of primary visual cortex. In: A. Toga (Ed). Brain Mapping: An Encyclopedic Reference.

Mapping specialised areas in individual brains is a prerequisite to investigate internal functional organisation using sub-millimetre fMRI

Conventional (<= 3T) MRI Ultra-high field (UHF) (>= 7T) MRI Courtesy Alard Roebroeck

Investigating the Brain at Multiple Levels of Organization

Tonotopic Maps in the Inferior Colliculus @ 7 T

De Martino, Moerel, van de Moortele, Ugurbil, Goebel, Yacoub, Formisano (2013). Spatial organization of frequency preference and selectivity in the human inferior colliculus. Nature Communications, 4, 1386.

UHF fMRI reveals a feature map in a small structure that shows only unspecific response at conventional resolution

• Responses of most V1 neurons are sharply tuned for the orientation of a stimulus in a small region of the visual field

• Discovered by Hubel & Wiesel during microelectrode recordings (Nobel Prize 1981)

Prime example: Orientation selectivity in primary visual cortex

• They observed that neurons with similar response preference cluster in “cortical columns”

• This was later more clearly revealed by studies using optical imaging showing that orientation selective V1 neurons are organized into a topographic map of orientation preference

• => Resolution of columns sufficient to get (coarse) measure of features!

Features at Mesoscopic Scale: Neurons and Columns

Laye

rs

3T voxel size

Specialized brain area

7T+ voxel size

column size: ~ 0.5 - 1.5 mm

Pial surface

White/gray matter boundary

Layers

Neurons (feature detectors)

Columnar- And Laminar-Level Imaging with fMRI at 7+ TeslaWhen a Quantitative Improvement of Spatial Resolution Turns into a Qualitative Change

• If neurons cluster into functional units, we might be able to reveal fine-grained “neuron-like” repre-sentations at the columnar level.

• There is indeed substantial evidence that many areas of the cortex are organized in vertically extending columns that contain neurons with rather similar response profiles.

• Individual neurons code features but they are too small to be detected with high-resolution human fMRI.

• If neurons would be distributed randomly, ultra-high field imaging would provide no qualitative improvement.

Human - fMRI (SE, 7T) Monkey - Optical Imaging

Mapping of the (larger) ocular dominance columns had already been reported earlier (e.g. Cheng et al., 2001; Goodyear and Menon, 2001; Yacoub et al., 2007) but this spin echo (SE) EPI study was the first study revealing detailed maps of the much smaller orientation columns!

Yacoub, Harel, Ugurbil (2008) Proc Natl Acad Sci USA, 105, 10607-10612.

High-Resolution fMRI Reveals Orientation Columns in V1

High-resolution UHF SE fMRI provides the unique opportunity to investigate these basic computational units in the human brain.

Columns have been imaged non-invasively in the human primary visual cortex (V1) located within flat calcarine sulci in selected subjects.

A single thick slice with high in-plane resolution (0.5 mm) were prescribed to anatomically identified calcarine sulcus due to limitations of inner-volume SE-EPI.

3 cm

Yacoub, Harel, Ugurbil (2008) Proc Natl Acad Sci USA, 105, 10607-10612.

High-Resolution fMRI Reveals Orientation Columns in V1

Data Analysis Strategies: To Pool or Not To Pool?

Pial surface


• Lamina Profiles• Information (MVPA)• No explicit columnar features

• Mapping columnar features• No Layer Separation

• Mapping columns across depth• High signal to noise required

No pooling - Every voxel interpreted!

Kemper, De Martino, Emmerling, Yacoub, Goebel (2017). High-resolution data analysis strategies for mesoscale human functional MRI at 7 and 9.4 Tesla. Neuroimage.

• The classical model of a cortical column assumes a nearly perfect vertical penetration through the cortex.

• It has been, however, shown that in areas of monkey IT cortex columns do show strong irregularities across different layers (e.g. Keiji Tanaka, 2011)

• To reveal how feature codes change across cortical laminae, it is important to map the topography of features at different relative cortical depth levels within specialized areas.

• We developed two methods to sample topographic information at different cortical depth levels: 1) based on reconstructed cortex meshes (see also Polimeni et al., 2010), and 2) with a novel regular-grid sampling technique.

“Ideal” columnar organization More realistic columnar organizationPial surface


Columnar-Level Features At Different Relative Cortical Depth Levels

The Cortex Modelled as a 3D Structure (7+ Tesla)

Isotropic High-Resolution Scans and Grids for Advanced Analysis

High-resolution cortical depth analyses using standard mesh approach vs Cartesian grids


Layer Sampling - The Book Principle

Bok (1929). A cortical cross section depicting six cytoarchitectonic layers. The volume fraction of a segment is constant across the whole layer. This is possible because the thickness of the layer changes to compensate the curvature.


• 2D regular grid sampling using equi-volume model (Bok, 1925, Waehnert et al., 2014)

• Layers (voxels between depth grids) can be filled in voxel space


• Precisely aligned functional data is sampled at each 2D grid point

• Sampled data can be directly visualized in 2D grid space and as 3D volumes

• A 2D grid coordinate refers to the same vertical unit across levels (“column”)

• Distance, area and volume values can be easily calculated

9 depth grids

High-Resolution 2D Grid Sampling At Multiple Cortical Depth Levels

05/30/12V5

V1/V2

V5

FlowfieldStationary dots

V5

V1/V2

Can we map features inhMT with 7T fMRI?

Localizing hMT+/V5 at 3T: “Moving vs Static” Contrast

Mapping Axis-of-Motion Columns in hMT/V5 at Different Cortical Depth Levels using High-Resolution Grid Sampling

Zimmermann, Goebel, De Martino, Adriani, Van de Moortele, Feinberg, Chaimov, Shmuel, Ugurbil, Yacoub (2011). PLoS One, 6(12), e28716.

Emmerling, Frost, Goebel (2016). work in progress.

Measuring Binocular Disparity Tuning in hMT

Background


Two Joint Maps of Disparity and DoM Tuning in monkey MT

General Background

Disparity tuning has been shown in Macaque Area MT

(DeAngelis & Newsome 1999)


Mesoscopic Disparity and AoM Tuning in hMT

Design

Stimulation


Mesoscopic Disparity Tuning in hMT

Results

-2 -1 1 2-0.3

-0.2

-0.1

0

0.1

0.2

0.3

Disp -2

-2 -1 1 2-0.3

-0.2

-0.1

0

0.1

0.2

0.3

Disp -1

-2 -1 1 2-0.3

-0.2

-0.1

0

0.1

0.2

0.3

Disp 1

-2 -1 1 2-0.3

-0.2

-0.1

0

0.1

0.2

0.3

Disp 2


Mapping Results

-.15 -.05 +.05 +.15

Dis

parity

co

nditio

ns

Direc

tion

co

nditio

ns

right left up down

Simultaneous Disparity and Motion Tuning in hMT

Nasr, Polimeni, Tootell (2016). JNS, 36, 1841-1857.

Color and Disparity Selective Columns in V2 / V3

Relatively large color-selective stripes could be revealed with GE EPI with a resolution of 1 mm x 1 mm x 1 mm

• Repeating columnar stripes have been observed in histological variations of cytochrome oxidase (CO) levels.

• “Thin” and “thick” stripes of dark CO staining reportedly respond selectively to stimulus variations in color and binocular disparity.

Nasr, Polimeni, Tootell (2016). JNS, 36, 1841-1857.

Color and Disparity Selective Columns in V2 / V3

De Martino, Moerel, Ugurbil, Yacoub, Goebel, Formisano (2015). PNAS, 112, 16036-1604.

Columnar Coding in Primary Auditory Cortex

Distributed coding of shapes across columns in monkey IT (e.g. Tanaka, 1996)

In V1-V3, A1, V5 features were known! It is a much more challenging task to map features that are hitherto unknown. Cracking the columnar-level code involves not only high-end technology (7T+, GRASE) but also smart experimentation. Areas that are targeted in our lab: LOC, letter area, VWFA, OFA, FFA, and invariance transformation across areas of the ventral stream. After establishing columnar-level fMRI, applications targeting changes of columnar organisations during development and learning can be investigated.

Goal: Unraveling columnar-level feature representations in mid-level and higher-level areas of the visual hierarchy.

Columnar-level features in LOC, VWFA, FFA..

The Challenge Ahead: Unraveling Unknown Feature Codes

Being able to separate fMRI responses from different columnar-level features and cortical layers opens the possibility to relate cognitive phenomena like attention, expectation, working memory, imagery and awareness to the human mesoscopic scale for the first time providing substantially increased explanatory power for testing and creating detailed cognitive theories of the mind.

Going Beyond Feature Mapping: Using Mesoscopic 7T+ fMRI to Study Human Cognition

Towards Mesoscopic Neural Correlates of Consciousness

New model-based motion localiser for hMT, see Schneider, Marquardt, De Martino, Goebel, poster #2183 this meeting.

Ambiguous Motion Quartett Stimulus with two squares alternating same positions:Horizontal / vertical motion happens in the brain (apparent motion)

Raw Preprocessed Data - Horizontal vs Vertical Motion

Goebel R, Schneider M, Ugurbil K, De Martino F, Yacoub E.(2017). in preparation.

Mesoscopic Neural Correlates of Consciousness

Goebel R, Schneider M, Ugurbil K, De Martino F, Yacoub E.(2017). in preparation.

Sub-millimeter fMRI can reveal a coarse representation of cortical laminae allowing to investigate layer-specific interactions between brain areas. Importantly, lamina profile measurements might help to separate bottom-up from top-down information flow.

Furthermore, imaging laminar profiles opens new avenues to study spatial and feature-based attention effects with the prediction that supragranular layers exhibit increased activation during top-down attention and imagery. Laminar imaging may also help to test predictive coding theories.

Lamina-Specific Functional Analysis @ 7T+

adapted from Bastos et al. (2012)

Layer-Specific Attention Effects @ 7T in Auditory Cortex

De Martino, Moerel, Ugurbil, Yacoub, Goebel, Formisano (2015). PNAS, 112, 16036-1604.

Kanizsa Illusion — Selective Activation of Deep Layers in V1

Kok, Norris, De Lange (2016). Current Biology, 26, 371-376.

(SEM) n=6

Smith F & Muckli, L (2010). PNAS.

Context Decoding in V1 - Cross-Condition Generalization

Muckli L, De Martino F, Vizioli L, Petro LS, Smith SW, Ugurbil K, Goebel R, Yacoub E (2015). Current Biology.

Classifier)perform

ance)(%))

20)

40)

60)

80)

Average)

Cor;cal)depth)(%))

90) 74) 58) 42) 16) 10)

33%)

S1)FB)

S2)

S3)

S4)

s)<)0.05)

S1)FF)

S2)

S3)

S4)

s)<)0.05)

WM GM

Layer-Specific MVPC Performance @ 7 T

GM WM

Blood vessels excluded

Subj. 2

Subj.1

V1

Selected Research Techniques for in-vivo Human and Animal Neuroscience

Spatial resolution

Temporal resolution

Coverage Non-Invasive

Human fMRI @ 3T (GE EPI)

+ - +++ ++

Human fMRI @ 7T+ (SE EPI, GRASE)

++mesoscopic level of columns/layers

- but lamina profiles

+meso whole-brain difficult to achieve

++

Human EEG/MEG - - ++ +++ +++

Macaque optical imaging

+++ - - - - - -Macaque electrode recordings

++++ +++ - - - - - - Mesoscopic-level UHF fMRI creates a bridge between human and animal research, especially to invasive optical imaging. After establishing mesoscopic spatial resolution, UHF fMRI, research can be extended to more human-specific cortical areas. Despite laminar profiles, temporal resolution remains a severe limitation of fMRI.

Comparing field dependent human fMRI with electrophysiology and computational modeling of neural networks

De Martino et al. (2017). The impact ofultra-high field MRI on cognitive and computational neuroimaging.Neuroimage.

• To better understand brain anatomy and function, multiple levels of brain organization need to be integrated.

• Sub-millimeter ultra-high field (f)MRI is an important new tool to bridge macro- and mesoscopic scales as well as human and animal research.

• Recent fMRI experiments show that it is possible to map known columnar-level representations in specialised brain areas (V1, hMT) using ultra-high field fMRI and spin-echo based MR pulse sequences.

• It remains a challenge to “crack” the functional code for areas where the “alphabet of features” is hitherto unknown such as the face areas.

• Revealing (columnar) feature codes in specialized brain areas at mesoscopic scale has the potential to provide important new insights in the neural substrate of human perception and cognition.

• Feature codes and laminar profiles in multiple brain areas will lead to a deeper understanding of how visual perception and cognition emerge from feature representations and their interactions in the brain.

Summary and Conclusions

AcknowledgementsMarian Schneider (UM, NL)

Jan Zimmermann (NYU, US) Federico de Martino (UM, NL)

Valentin Kemper (UM, NL) Mario Senden (UM, NL)

Thomas Emmerling (UM, NL) Elia Formisano (UM, NL)

Alard Roebroeck (UM, NL) Francesco Gentile (NIN, NL)

Nienke van Atteveldt (VU, NL) Essa Yacoub (CMRR, USA) Kamil Ugurbil (CMRR, USA)

van de Moortele (CMRR, USA) Gregor Adriany (CMRR, USA)

Lars Muckli (Glasgow, UK) David Feinberg (Berkeley, USA)

Miguel Castelo-Branco (Coimbra, PT) Joel Reithler (UM, NL) Judith Peters (UM, NL)

Fabrizio Esposito (U Salerno, IT) Martin Frost (UM, NL)

the micro in the macro using ltra - organization for … courses... · 2017-07-05 · finding the...

Documents