optoneuro - newcastle university · 2021. 4. 27. · degenaar optical stimulation arrangement...
Post on 17-Aug-2021
1 Views
Preview:
TRANSCRIPT
OptoNeuro
Optogenetic visual cortical prosthesis
Dr. Patrick DegenaarReader in Neuroprosthesis
Conflicts of interest: I have filed a number of patents and CANDO may have commercial impact. But currently no commercial interests.
Neuroprosthetics @ Newcastle
Stimulate
Process
Record
http://www.optoneuro.eu
Epilepsy brain prosthesis
Visual prosthesis
Spinal cord injury
Hand/Leg bionics
Multi-EMG recording
Electrodiagnostics
http://www.cando.ac.uk
Closed loop neural
interfaces!!
Blue light is used to activate neurons expressing
channelrhodopsin. Other opsins can silence activityOptogenetic control of behaviour.
Gradinaru et al. J Neurosci (2007)
The optogenetics revolution
High radiance optoelectronics for optogenetics:Grossman N et al. IEEE TBME 2011, 58(6), 1742-1751.McGovern B et al IEEE TBCAS 2010, 4(6, part 2), 469-476.Grossman N et al, J. Neural Engineering 2010, 7(1), 016004.…
Optoelectronics for optogenetics
CMOS
µlens optics
Gallium
Nitride µLED
3D bump bond
CMOS driver
GaN MQW
p-contact
Solder bond
2010 2011 20132009 2012 2014
64x64, 64x1
passive µLED arrays16 x 16 MPW CMOS
chip driven µLED arrays90x90 Retina
wafer
2015 2016 2017 2018
CANDO 1
Optrode
CANDO 3
Wafer
Temporal
Primary visual
cortex
Optics (cornea/lens)
Sense/code (retina)
Transmission (Optic Nerve)
Sorting(Thalamus)
Interpretation(Visual Cortex)
Pulvinar nucleus
Lateral Geniculate
Nucleus
Superior
Colliculus
Optical
radiation
5
The visual system
6
PrevalenceAccording to WHO, about
180 million people worldwide
have a visual impairment
Legal blindness:
“medically diagnosed central
visual acuity of 20/200 or less
in the better eye with the best
possible correction, and/or a
visual field of 20 degrees or
less”
Cataracts
(48%)
Others
(16%)
AMD
(9%)
Corneal opacities (5%)
Diabetic retinopathy (5%)
Childhood blindness (4%)
Glaucoma
(12%)
Trauma
(?%)
Drugs and gene therapy becoming increasingly
useful.. But cannot restore lost vision
Primary visual
prosthesis target
conditions
Visual loss: statistics
From: WHO
RP (1%)
Optics (cornea/lens)
Sense/code (retina)
Transmission (Optic Nerve)
Sorting(Thalamus)
Interpretation(Visual Cortex)
Retinitis Pigmentosa
(<1% of blindness)
Glaucoma
Trauma
Other severe blindness
(~30% of blindness)
Sub/epi retinal
prosthesis
Optic nerve
prosthesis
LGN
prosthesis
Cortical
prosthesis
For invasive procedures only!
Very expensive!
Will be enormously difficult!
But can treat much larger cohort
Disorders of the visual system
Optics (cornea/lens)
Sense/code (retina)
Transmission (Optic Nerve)
Sorting(Thalamus)
Interpretation(Visual Cortex)
Implementing a visual prosthesis
Wireless
power/data
Visual
processing
Image
acquisition
Stimulator
Easy??
Brindley and Lewin did it in
1968!!!• Brindley,G.S.,Lewin,W.S.,1968.The visual sensations produced by electrical stimulation of the medial
occipital cortex. J Physiol.19454-5P.
• Brindley,G.S.,Lewin,W.S.,1968.The sensations produced by electrical stimulation of the visual
cortex.J.Physiol.196,479–493.
Challenge 1: Sheer scale of challenge
1960’s: You could build a device on a Monday
and put it in a patient on a Friday
Now: A project to create an active implantable
device requires £10-50M and 5-10 years
worth of regulatory effort to demonstrate
safety before implantation is allowed
As such, almost the entire field moved to retinal
prosthesis in the 1990’s as it was a much easier target!!
5 x 5 (25) pixels10 x 10 (100)33 x 33 (1000)70 x 70 (5000)100 x 100 (10,000)Normal vision
Asian/pictographic
scripts:
European/phonetic
scripts:
~ 1000 pixels
~ 5000 pixels
~ 100 pixels
European/phonetic
characters:
11
Challenge 2: Visual acuity
From Retina AG
1500 electrodes
– but only a few
dozen can be on
at any moment in
time.
Lessons from retinal prosthetics
Original image
Assume greyscale retina
@ 64 x 64 resolution
Add noisy retina
effect
100:1 contrast ratio 50:1 contrast ratio
With insufficient
stimulus contrast it
is like whispering
through a
typhoon!
13
Challenge 3: Effective contrast
1. Image capture
Rods/cones
2. smoothing
Horizontal cells
3. DerivativeBipolar cells
Amacrine cells
Ganglion cells
5. Spike coding
OFF ON
14
The retina
Ligtht
4. Temporal edge
The retina has 60 different cell types. If we are bypassing the natural processing, we need to replicate the key features!
Challenge 4: The eye is not a camera!
Annu. Rev. Biomed. Eng.
10:275–309, 2008
IEEE Transactions on Neural Systems and
Rehabilitation , Vol. 12, No. 2, June 2004
Electrode
insertion
point
Accelerated ageing, 1000
cycles at 1.7V/s
Journal of The Electrochemical
Society, 152 (7) J85-J92 ~2005!
Glial scarring
Ideal Real
Electrode
degradation and
adverse tissue
reactions are a major
issue for smaller
stimulating
electrodes because
of high charge
density requirements
15
Challenge 5: Biocompatibility
Where to start?
0.18 mm
0.18 mm
0.46 mm
0.41 mm
0.31 mm
0.68 mm
2.2 mm
From: Atlas of Cytoarchitectonics
- Von Economo
Developing the stimulator
Surface stimulation
(Brindley, Dobelle)
Penetrating stimulation
Can we make the communication bi-directional?
Can we treat communication as a control problem?
`
0.18 mm
0.18 mm
0.46 mm
0.41 mm
0.31 mm
0.68 mm
2.2 mm
From: Atlas of Cytoarchitectonics
- Von Economo
Penetrating stimulation
Can we make the communication bi-directional?
Can we treat communication as a control problem?
Light stimulus
Electrical recording
Optical
Electricall
Optogenetics & closed loop control
19
£10M project: Clinical trials aimed for 2021
Controlling Abnormal Network Dynamics with Optogenetics
Noise cancellation of epileptic
seizures – but adaptable to
vision
Connecting
lead
Central control unit
inc wireless comms
Brain
implant unitBrain implant unit (Actually
different position)
Control unit
External
headset
Principles:
1. Custom brain implant starting as 4 x
4 x 4 LEDs can be utilised
2. The control unit is quite different –
higher power, no battery
3. Cabling can be simpler
CANDO - Epilepsy Visual brain prosthesis
Skin Fat Muscle
Average 5 mm
2mm 1mm 4mm
TX RX
Adapting to visual brain prosthetics
Brain stimulator unit
Interface unit
Mounting
plate
Optrode
Active electronics
Communications
Recording Circuits
Stimulation Circuits
Digital control unit (FSM)
Diagnostic Circuits
Power
Recording Electrode
Photonic Stimulation
site
Optrodeshaft
Optrode head
1. Ramezani et al. submitted to IEEE TCAS-I, June 2017
2. Zhao et al. Implantable optrode GB1410886.4
3. Dekhoda et al. Temperature sensor GB1522790.3
4. Degenaar Optical stimulation arrangement GB1616725.6
5. Constandinou et al On Chip Random ID generation GB1702623.8
Vin
Vref 50C LNA
C
VIP VIN
VONVOP
C
6C Gm
C=100fF
VINVIPVOUT
Front end amplifier Programmable gain amplifier
Recordings from non-human primate brain
Recording microelectronics
LED driver circutry
DAC
LED drive power LED efficiency
• Amplitude modulation
• PWM modulation
• Diagnostic and methods
0 200 400 600 800 1000 0 200 400 600 800 1000 0 200 400 600 800 1000
0 200 400 600 800 1000
1
10
100
0.1
0.01
0.001
1
10
100
0.1
0.01
0.001
0.0001
0.00001
Tissue penetration depth (μm)
Op
tica
l P
ow
er
(mW
)
Threshold = 1mW/mm2 Threshold = 1mW/mm2 Threshold = 1mW/mm2
40μm60μm80μm100μm120μm
Op
tica
l P
ow
er
(mW
)
LED diameter
0 200 400 600 800 1000 0 200 400 600 800 1000
Tissue penetration depth (μm)
Threshold = 1mW/mm2 Threshold = 0.1mW/mm2 Threshold = 0.01mW/mm2
Lambertian emitter Shaped emitter Pseudo-collimated emitter
Pseudo-collimatedShapedLambertian
LED type
(d) (e) (f)
(g) (h) (i)
(a)
(b)
(c)
Dong et al. For submission to J. Biomedical Photonics 2017
How much light do we need?
Key take home message:
1. Size doesn’t matter to penetration depth!
2. Need to penetrate at least 100-200µm for long term implantables due to gliosis
3. More collimated light will give better penetration
Maximum allowed duty ratio (%)Maximum allowed pulse width
(e) (f)
1ms 10ms100ms 1s 10s 1ms 10ms100ms 1s 10s
1ms 10ms100ms 1s 10s 1ms 10ms100ms 1s 10s
ΔT (°C)
(a)
ΔT (°C)
ΔT (°C) ΔT (°C)
Optical stimulus Optical stimulus
Optical stimulus Optical stimulus
(b)
(c) (d)
1 LED on
1 LED on
8 LEDs on
8 LEDs on
2.5mW
5mW
10mW
10mW5mW2.5mW
10mW
5mW
2.5mW 2.5mW
5mW
10mW
0.01
0.1
1
10
0.01
0.1
1
10
0
3
0
10
2
4
6
8
1
2
10s
1s
0.1s
10ms
1ms
Thermal power (mW)
0 10 20 30 40 50 60
Thermal power (mW)
0 10 20 30 40 50 60
20
40
60
80
100
1 LED on
8 LEDs on 8 LEDs on
1 LED on
0.000
0.1000
0.2000
0.3000
0.4000
0.5000
0.6000
0.7000
0.8000
0.9000
1.000
S
0 . 0 0 0
0 . 1 0 0 0
0 . 2 0 0 0
0 . 3 0 0 0
0 . 4 0 0 0
0 . 5 0 0 0
0 . 6 0 0 0
0 . 7 0 0 0
0 . 8 0 0 0
0 . 9 0 0 0
1 . 0 0 0
0 . 0 0 0
0 . 1 0 0 0
0 . 2 0 0 0
0 . 3 0 0 0
0 . 4 0 0 0
0 . 5 0 0 0
0 . 6 0 0 0
0 . 7 0 0 0
0 . 8 0 0 0
0 . 9 0 0 0
1 . 0 0 0
0 . 0 0 0
0 . 1 0 0 0
0 . 2 0 0 0
0 . 3 0 0 0
0 . 4 0 0 0
0 . 5 0 0 0
0 . 6 0 0 0
0 . 7 0 0 0
0 . 8 0 0 0
0 . 9 0 0 0
1 . 0 0 0
EL
P
P
S
L
EL
1.00.80.60.40.20
ΔT (K)
2.01.51.00.50
2.01.51.00.50
ΔT (K)
ΔT (K)
No coating 20um polymer coating
25um in the tissue100um in the tissue
On-optrode On-optrode
S
P
ΔT(
K)
1000
20
0
200
Optrode profile
100μm
100μm
100μm
10
00μ
m
(a)
(b)
(c)
(d) I
(e) II
(f) III
0.5
0
1.0
1.5
2.0
0 1.0 2.0 3.0
0
0.5
1.0
1.5
2.0
ΔT(K)
Dimension (μm)
Dim
ensi
on
(μ
m)
ΔT(
K)
Dimension (μm)
L: LED E: Electrode P: Polymer coating S: Silicon substrate
III
II
I
Target LED characteristics:
Driving current: 1mA
Driving voltage: 4-5V
Light requirement: 1mW
Efficiency: >20%
Typical micro-LED efficiencies from
literature ~ 1-5%
Wu et al. Neuron 2015: 0.8%
Kim et al. Science 2013: 0.2%
McAlinden al: Opt Lett 2013: 5%
McGovern et al: PLOS ONE 2013: 5%
McGovern et al: IEEE TBCAS 2010:1%
Dong et al. For submission to J. Biomedical Photonics 2017
LED thermal impact
Dong et al. For submission to J. Biomedical Photonics 2017
Wu et al Neuron 2015
10µm30µmOur LED
320 x 240!100µm @ 1mA
LED efficiencies
“Official” threshold is
0.7mW/mm2 for dissociated
culture. But what about in
practice???
In NHP experiments, We saw
responses at 1mm with emission
intensities of only 0.5mW. i.e. At
that depth the irradiance <<
0.1mW/mm2
Caveat: we cannot separate
neural transmission!
Modelling indicates a lower
effective threshold of 0.1 - 0.3
mW/mm2
Luo et al. Submitted to JNE 2017Currently Unpublished work
Light requirement in-vivo
b)
Schematic design
3D assembly
CAD layout Wafer fabrication
Probe assembly
Light-
emitting
diodes
Recording
sites
Control
electronics
We are developing a fully manufacturable implant!!
Assembling the optrodes
Camera
RX
Embedded
SoC
BLE
module
Power
module
Power
trans/Amp
BLE
module
Embedded
SoC
Power RX/
AC->DC
Voltage
regulator
External control unit Internal control unit
Power and
Control unit
Optrodes
Optrode unit
To receiver
antennae
Internal
control unit
Ontogenetically
Sensitized tissue
Optrode
Implant
Base plate
Receiver
antennae Transmitter
antennae
Batt and cam
3.7V
USB
TX
Fattah & Soltan et al. in preparation for IEEE TBME submission
Developing the control system
System software - preprocessing
Song of the Machine!
Simplification Cartoonization
Visual augmentation
The vision that we willreturn to patients withvisual prosthesis will berudimentary. We aretherefore using patientswith partially degeneratevision as a simulation ofwhat can be achieved.
Camera
External control unit Internal control unit
Power and
Control unit
To receiver
antennae
Internal
control unit
Ontogenetically
Sensitized tissue
Optrode
Implant
Base plate
Receiver
antennae Transmitter
antennae
Fattah & Soltan et al. in preparation for IEEE TBME submission
Full software process
Compression
Decompression Pulse coding
Video/Image
acquisition
Simplification Retinal processing
Transmission
LED reconstruction
Visual cortical Headset
Implantable control unit
0
10
20
30
40
50
60
70
80
60 40 20
MC
Pe
rfro
man
ce (
fps)
Frequency (MHz)
44% 61% 75% 86%
20
30
40
50
60
70
80
60 40 20
Po
we
r (m
W)
Frequency (MHz)
We are here
CANDO First
in human trial
Current
funding ends
(Aug 2021)
CANDO Engineering
Visual cortical trial??
CANDO trial
CANDO
Regulatory &
ethical
submission
Timeline
OptoNeuro
Acknowledgments
CANDO project
Tim Constandinou (Imperial College), Nick Donaldson
(UCL), Andrew Sims (Newcastle RVI (NHS)), Andrew
Jackson (Newcastle IoN), Mark Cunningham
(Newcastle IoN), Anthony O’Neil (Newcastle EEE),
+ Others:
OptoNeuro FP7 project
Mark Neil (Imperial College), Ernst Bamberg (Max
Planck Institute), Christian Bamann (Max Planck
Institute), Botond Roska (FMI, Switzerland), Pleun
Maaskant (Tyndall Institute), David Rogerson
(Scientifica), Mark Johnson (Scientifica)
top related