a haptic presentation of 3d objects in virtual reality for the visually disabled msc marcin...

A haptic presentation of 3D objects in virtual reality for the visually disabledA haptic presentation of 3D objects in virtual reality for the visually disabled

MSc Marcin Morański

Professor Andrzej Materka

Institute of Electronics,

Technical University of Lodz, Poland

www.eletel.eu

MSc Marcin Morański

Professor Andrzej Materka

Institute of Electronics,

Technical University of Lodz, Poland

www.eletel.eu

Plan of the presentationPlan of the presentation

• Project motivation• Background• A haptic presentation system• The system working principle• Experiments with blind users• Conclusions

ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled 2

3ICDVRAT 2012: A haptic presentation of 3D objects in virtual reality for the visually disabled

Project motivationProject motivation

• 4 out of 1000 Europeans are visually impaired or blind

• Lack of sight is a deprivation of 80%-90% perceptual abilities

• The ageing demographics increases the number of vision problems

Source: http://www.visionaustralia.org.au


Aim of the projectAim of the project

Goals:

• to construct a prototype system for the tactile presentation of real objects in a virtual reality;

• to check the opportunity of the application of the tactile presentation system for the blind and visually impaired;

• to examine the usability and potential of the force feedback device, Falcon Novint, for a 3D virtual object presentation without the usage of vision.

Assumptions:

The prototype system should integrate a depth camera and a haptic device.

BackgroundBackground


Source: http://www.niewidomi.com.pl


A haptic presentation systemA haptic presentation system

The block diagram of the prototype

The system working principleThe system working principle


Depth map acquisition

Plane detection

Obstacle detection

Calculation of geometrical parameters

of the obstacles

Mapping of the planes in virtual scene

Obstacles substitution by 3D boxes

Falcon Novint control

SCENE SEGMENTATION

VIRTUAL SCENE MODELING

Virtual scene modelingVirtual scene modeling


a) b) c) a) 2.5D depth map of the scene, b) the segmented scene, c) the reconstructed scene for the tactile presentation.

Experiments (1) Experiments (1)

Participants:

• 8 blind persons (2 women and 6 men).

Setup: • the objects in each scene were located on the ground, against a

background;• an exploration time of every scene was measured;• the task was carried out successfully when the blind person correctly

identified all obstacles in the presented scene.


a) b)a) An example of the scene, b) the experimental setup

Experiments (2) Experiments (2)

The experiments were divided into 4 stages:

1.Training phase;

2. Scene content recognition;

3. Distance estimation to each obstacle from a chosen point of observation;

4. Estimation of obstacles’ height.



Summary of experiment resultsSummary of experiment results

1 20

20

40

60

80

100

120

140

160

180

200

STAGE III: Estimation time of the obstacles location

Average exploration time Standard deviation Median

Number of scenario

Tim

e [

s]

1 2 30

1

2

3

4

5

6

7

8

9Stage IV: Estimation of objects

height

Object' height estimated correctlyObjects' height estimated incorrectly

Number of scene

Nu

mb

er

of

pa

rtic

ipa

nts

1 2 3 4 50

1

2

3

4

5

6

7

8Stage II: Scene content identification

Scene identified correctly Scene identified incorrectly

Number of scene

Nu

mb

er

of

pa

rtic

ipa

nts

Comments and suggestions of the blind participants (1)



a special focus is required when creating small objects and scene

containing many details; adding vocal information about the 3D position of the probe (the virtual

finger in the system) the sonification of the some of the scene points or objects would also be

very useful




There are a couple of potential application when such a system could help the blind in everyday activities:

new route learning in buildings (e.g. post office, bank, school); tactile interaction when using various computer applications (e.g.

web browsers); tactile interactions with objects in museums.

Experiment conclusionsExperiment conclusions


The performed experiments have shown that the system can be useful for the blind, although there are still some challenges to overcome. Many requirements need to be met, mainly:

• choosing the scale of virtual objects in the ratio to the real ones;

• solving the problem of losing oneself in VR;

• solving the problem of presenting scenes consisting of many objects/details (each of a different size), as the system’s resolution is finite.

ConclusionsConclusions

1. A prototype system for tactile presentation of real objects in a virtually reality was presented.

2. The system usability was examined by the blind participants.

3. The designed system have potential to be a very useful tool for visually disabled in many applications, such as building virtual maps and creating simulators where real objects are presented virtually.


AcknowledgementsAcknowledgements

This work has been supported by the Ministry of Science and Higher Education of Poland research grant no.

NR02008310 in years 2010 - 2013. The presented study is carried out with cooperation with Lodz Chapter of the

Polish Society for the Blind.


http://www.mnisw.gov.pl/

http://images.google.com/imgres?imgurl=http://www.pzn.hrubieszow.info/pzn.gif&imgrefurl=http://www.pzn.hrubieszow.info/biuletyn5.htm&h=142&w=142&sz=3&hl=pl&start=10&um=1&usg=__OELTgJLfjbUZ0508wuTlvyTpYQg=&tbnid=ouXRq-ZpomI2RM:&tbnh=94&tbnw=94&prev=/images?q=polski+zwi%C4%85zek+niewidomych+logo&um=1&hl=pl&rls=GGLG,GGLG:2005-33,GGLG:en&sa=N

a haptic presentation of 3d objects in virtual reality for the visually disabled msc marcin...

Documents