Download - Bionic Vision Australia
![Page 1: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/1.jpg)
Bionic Vision AustraliaAugmenting intensity to enhance scene structure in prosthetic vision
Chris McCarthy, David Feng and Nick Barnes
Australian National University, College of Engineering and Computer Science
NICTA Canberra Research Lab, Computer Vision Research Group
![Page 2: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/2.jpg)
Sub HeadingLevel 1 text
HEADING
![Page 3: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/3.jpg)
Implant placement in eye
Wide-View array:supra-choroidal
High-Acuity array:epi-retinal
![Page 4: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/4.jpg)
Induced perception of light by means other than light entering the eye
Phosphene vision
![Page 5: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/5.jpg)
Expectations: move freely, avoid obstructions, and orientate
Challenges:limited bandwidth: number of electrodes, dynamic range, refresh rate
Strategy:Vision processing algorithms and alternative visual representations to improve mobility outcomes
Mobility with prosthetic vision
![Page 6: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/6.jpg)
Simplest vision processing scheme: down-sample the intensity image:
Vision processing
![Page 7: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/7.jpg)
We propose: stimulate to emphasise structurally significant features:
Vision processing
![Page 8: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/8.jpg)
Previously: augmented depth
[McCarthy et al, ARVO 2012, EMBC 2011]
Mobility with prosthetic vision
![Page 9: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/9.jpg)
Previously: augmented depth
[McCarthy et al, ARVO 2012]
Mobility with prosthetic vision
![Page 10: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/10.jpg)
Key outcomes:Augmentations that emphasise obstructions can significantly improve performanceAcceptable to participants
Limitations:Relies on accurate surface-fit and segmentationRemoves potentially useful information from the sceneDoes not easily generalise to other activities of daily living
Mobility with prosthetic vision
![Page 11: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/11.jpg)
Rationale:Stimulate conservatively and selectively, but maintain naturalityEmphasise regions of “structural interest”
New proposal: the perturbance map
![Page 12: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/12.jpg)
Algorithm inputs:A dense disparity (ie., inverse depth) imageWe use (but are not limited to!) an Asus Xtion Pro RGB-D sensor
Method
![Page 13: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/13.jpg)
Step 1: extract local disparity gradientsIf disparities are highly quantised, disparity contours may be used instead
Method
![Page 14: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/14.jpg)
Step 2: local surface orientationsHistogram orientations of local gradient vectors (or iso-disparity linear segments)
Method
![Page 15: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/15.jpg)
Step 3: fixed scale window comparison
Method
![Page 16: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/16.jpg)
Step 3: fixed scale window comparison
Method
![Page 17: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/17.jpg)
Histogram non-overlap
Step 3: fixed scale window comparison
Method
![Page 18: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/18.jpg)
Step 3: fixed scale window comparison
Method
![Page 19: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/19.jpg)
Step 4: multi-scale perturbance calculation:
Method
Combine perturbance scores across scales: weighted average based on number of iso-disparity pixels in each window
![Page 20: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/20.jpg)
Results: the perturbance map live
Perturbance mapOriginal image
![Page 21: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/21.jpg)
Scale intensity image by the perturbance map
Augmented Intensity
I(p) P(p) I*(p)
![Page 22: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/22.jpg)
Visual representation
Perturbance map Augmented Intensity
![Page 23: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/23.jpg)
Augmented Intensity
Sample results
Perturbance map Augmented intensity
![Page 24: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/24.jpg)
Augmented IntensitySample results
Perturbance map Augmented intensity
![Page 25: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/25.jpg)
AugmentationSample results
Perturbance map Augmented intensity
![Page 26: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/26.jpg)
Qualitative evaluation in simulated prosthetic vision
Visual representation
98 phosphene hexagonal grid (8 brightness levels)
Augmented Intensity Augmented phosphenes
![Page 27: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/27.jpg)
Intensity vs Augmented Intensity
intensity augmented intensity
![Page 28: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/28.jpg)
Intensity vs Augmented Intensity
intensity augmented intensity
![Page 29: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/29.jpg)
Intensity vs Augmented Intensity
intensity augmented intensity
![Page 30: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/30.jpg)
Intensity vs Augmented Intensity
intensity augmented intensity
![Page 31: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/31.jpg)
Perturbance map emphasises structureEnhances structural change w.r.t surrounding smooth surfacesPreserves natural variation providing rich visual representationBoundary preservation over surface modellingFuture work:Mobility trials: in simulation and with patientsRe-formulate augmentation in contrast space
Summary
![Page 32: Bionic Vision Australia](https://reader035.vdocument.in/reader035/viewer/2022062302/56816625550346895dd98070/html5/thumbnails/32.jpg)
HEADINGAcknowledgementsSimulation software: Paulette Lieby, Adele Scott, Ashley StaceyFunding:•BVA: Australian Research Council (ARC) Special Research Initiative (SRI) in Bionic Vision Science•NICTA: Australian Government, Dept. Broadband, Communications and the Digital Economy and the ARC ICT Centre of Excellence program