natural interaction haptic and training · networked teleoperation haptics in virtual environment...
TRANSCRIPT
23rd October 2008, Florence, Italy
Natural InteractionHaptic and Training
Emanuele Ruffaldi
Carlo Alberto Avizzano
PERCRO Scuola Superiore Sant’Anna
Pisa, Italy
23rd October 2008, Florence, Italy
The term haptics derives
from the Greek word
haptesthai
that means
“relative to the sense of touch”
23rd October 2008, Florence, Italy
Haptics in Human Machine InteractionClassic Robot
Teleoperated System
Networked Teleoperation
Haptics in Virtual Environment
Haptic CollaborativeVirtual Environment
Augmented Reality with Haptics
23rd October 2008, Florence, Italy
Haptic Interfaces
A haptic interface is a force reflecting device which allows a user to touch, feel, manipulate, create, and/or alter simulated three-
dimensional objects in a virtual environment.
23rd October 2008, Florence, Italy
Haptic Interfaces
A haptic interface is a force reflecting device which allows a user to touch, feel, manipulate, create, and/or alter simulated three-
dimensional objects in a virtual environment.
Kinesthetic Vibrotactile
Sensable, inc.
PERCRO
23rd October 2008, Florence, Italy
Challenge: Haptic Interfaces are not so Natural…
23rd October 2008, Florence, Italy
Research in Haptics• Haptic Perception
– Study of the perception of Haptic interaction• Haptic Devices
– Design and Construction• Computer Haptics
– Haptic Rendering computes forces based on the Virtual Environment
Robotic Haptic Human
modeling
perception
23rd October 2008, Florence, Italy
Haptics and Natural Interaction?
• Major Problem– Haptic feedback
requires to generate forces
– Forces require power– Power requires
actuators• Best use of Haptic
interfaces– Tool Mediated– Exoskeleton
• What about Natural?
HAND-Exoskeleton 3+3 DOF Hand Ex
Arm-Exoskeleton Haptic PenPERCRO PERCRO
PERCROPERCRO
23rd October 2008, Florence, Italy
Encounter InterfacesAn Encounter Interface is an interface that allows to provide haptic feedback only when the hand is in direct contact with the virtual object
Video Yokokohji98
WYSIWYF – What You See is What You Feel
Yokokohji 2000, Virtual Control Panel
23rd October 2008, Florence, Italy
Encounter Interface and Large Workspace
PERCRO
• Haptic Feedback: modified 3DOF GRAB Device• Tracking: VICON Motion Capture
0 . 2 5 0 . 3 0 . 3 5 0 . 4 0 . 4 5 00 . 1
0 . 20 . 3
- 0 . 2
- 0 . 1
0
0 . 1
0 . 2
0 . 3
Y ( m )
X ( m )
Z (
m)
h a p t i c d e v i c e
u s e r h a n d
It allows the Hands Free interaction for the PureForm Project
23rd October 2008, Florence, Italy
Sample interaction
•Track user hand with external systems•Make haptic and vision space coherent (calibration)•Model object physics and Evaluate distance from hand to objects•Determine closest surface and contact orientation•Control HI End Effector to display surface location•Control HI to slide below fingers during motion contacts
23rd October 2008, Florence, Italy
How can we use such Interfaces?
23rd October 2008, Florence, Italy
SKILLS Project
The SKILLS Integrated Project deals with the acquisition, storing and transfer of human skill by means of multimodal interfaces, Robotics and Virtual Environments technologies.
http://www.skills-ip.eu• 15 partners• 2006-2010• Coordinated by PERCRO-SSSA
23rd October 2008, Florence, Italy
• Sport and Entertainment– Juggling– Rowing
• Surgery and Rehabilitation– Maxillo Facial Surgery– Minimally Invasive Surgery– Upper Limb Rehabilitation
• Industry and Maintenance– Training of Maintenance and Operations– Programming-by-Demonstration
3 Demonstration Areas
23rd October 2008, Florence, Italy
Juggling
• What are the relative influences of haptic and visual feedback on learning to juggle?
• Can “slow motion” juggling in the virtual world improve juggling in the real world?
• Can timing cues for juggling patterns improve the time to learn the pattern? In what modality should timing cues be presented?
• Is an accurate toss more important than an accurate catch?
CEA-LIST
23rd October 2008, Florence, Italy
23rd October 2008, Florence, Italy
Workspace considerations:the ball throwing experiment
23rd October 2008, Florence, Italy
Multiple Balls
Real and Virtual Trajectories
Two Robotic Arms, Two Virtual Balls, Colocated Stereo
23rd October 2008, Florence, Italy
Vibrotactile
PERCRO Vibrotactile Glove
Tactile Jacket, TNO
PERCRO Vibrotactile Rowing
23rd October 2008, Florence, Italy
Encounter Interface with Shape Perception
Precise but Not yet Hands Free
PERCRO 2006-2008
23rd October 2008, Florence, Italy
Test conducted with TSD approach• The experiment consists in series of 100 trials. In each series two sphere Sa and Sb
with different radius were randomly presented. • So on each trial the participant have the same a priori probability of 0.5 to explore a
pair of sphere with the same radius (Sa,Sa or Sb,Sb) or with different radius (Sa,Sb or Sb,Sa).
• The observer’s task was to judge on each trial if the curvature of the two surfaces was ”different” or the ”same”.
• Three series were presented to each participant, changing the value of the difference curvature = Sb − Sa between the two possible curvatures. The values are defined in the table:
23rd October 2008, Florence, Italy
Future
• Intelligent Interfaces– Exploit Computing Power– Exploit Machine Learning
• Portable– Within Limits of Actuation
23rd October 2008, Florence, Italy
Next Event – ENACTIVE08
http://www.enactive2008.org http://www.skills-ip.eu