operation room supervision for safe robotic surgery … · operation room supervision for safe...
Post on 27-Apr-2018
215 Views
Preview:
TRANSCRIPT
KIT – University of the State of Baden-Wuerttemberg and
National Research Center of the Helmholtz Association
Institute for Process Control and Robotics (IPR)
www.kit.edu
Operation Room Supervision for Safe Robotic Surgery with a multi 3D-Camera Setup
P. Nicolai, J. Raczkowsky
Dipl.-Inform. Philip Nicolai
Institute for Process Control and Robotics (IPR)
2 Operation Room Supervision for Safe Robotic Surgery with a multi 3D-Camera
Setup – IROS 2011 Workshop on Safer Surgical Robotics
Karlsruhe Institute of Technology
The Merger of Forschungszentrum
Karlsruhe and Universität Karlsruhe
Mission of a state university with
research and teaching
Mission of a research institution of
the Helmholtz Association with
programoriented provident research
29.09.2011
Dipl.-Inform. Philip Nicolai
Institute for Process Control and Robotics (IPR)
3 Operation Room Supervision for Safe Robotic Surgery with a multi 3D-Camera
Setup – IROS 2011 Workshop on Safer Surgical Robotics
Karlsruhe Institute of Technology
29.09.2011
Employees
373 Professors
Students
20,771 8,893
Teaching
Research
Innovation
Dipl.-Inform. Philip Nicolai
Institute for Process Control and Robotics (IPR)
4 Operation Room Supervision for Safe Robotic Surgery with a multi 3D-Camera
Setup – IROS 2011 Workshop on Safer Surgical Robotics
Karlsruhe Institute of Technology
30.09.2011
Institute for Process
Control and Robotics
EU projects
AccuRobAs, RoboCast
SAFROS, ACTIVE
EuroSurge
Dipl.-Inform. Philip Nicolai
Institute for Process Control and Robotics (IPR)
5 Operation Room Supervision for Safe Robotic Surgery with a multi 3D-Camera
Setup – IROS 2011 Workshop on Safer Surgical Robotics
Outline
Introduction
OP:Sense
Multi 3D-camera setup
Outlook
28.09.2011
Dipl.-Inform. Philip Nicolai
Institute for Process Control and Robotics (IPR)
6 Operation Room Supervision for Safe Robotic Surgery with a multi 3D-Camera
Setup – IROS 2011 Workshop on Safer Surgical Robotics
Safety by Sensing
Intervention analysis
Extract information about actions
Workflow modelling and analysis
Supervision of intervention execution
Checking against surgical workflow
Alert personal of deviations from plan
Scene knowledge for active moving actuators
Guarantee safe interactions
29.09.2011
pre-operative
intra-operative
intra-operative
Dipl.-Inform. Philip Nicolai
Institute for Process Control and Robotics (IPR)
7 Operation Room Supervision for Safe Robotic Surgery with a multi 3D-Camera
Setup – IROS 2011 Workshop on Safer Surgical Robotics
Sensing parameters
Supervision area
Whole operation theatre
Single regions of interest
Robots workspace
Instrument table
…
Sensing equipment
Optical sensors 2D, Stereo, 3D
Medical devices pulse, blood pressure, oxygen levels, …
Robots sensors force-torque, skin / distance sensors
28.09.2011
Dipl.-Inform. Philip Nicolai
Institute for Process Control and Robotics (IPR)
8 Operation Room Supervision for Safe Robotic Surgery with a multi 3D-Camera
Setup – IROS 2011 Workshop on Safer Surgical Robotics
OP:Sense
Scenario
Shared workspace between surgeon and
multi-purpose robot
Robot can be handled in different modes
teleoperated
autonomous
Safety implications: 3D scene representation of robots surroundings is
needed in order to
Altogether avoid robot collisions (with humans or objects)
Lessen impact of collisions
Redundantly control pose and position of the robot
29.09.2011
Dipl.-Inform. Philip Nicolai
Institute for Process Control and Robotics (IPR)
9 Operation Room Supervision for Safe Robotic Surgery with a multi 3D-Camera
Setup – IROS 2011 Workshop on Safer Surgical Robotics
Components: robotic system
28.09.2011
Lightweight robots by KUKA
High payload, small footprint
Force-torque sensors in all joints
Seven degrees of freedom
Haptic devices by Force Dimension
Seven degrees of freedom
Three / four of the DOF active
Standard endoscopic instruments
Fitted with marker spheres
Easily changable by quick coupling
RX90 holding an endoscopic camera
28.09.2011
Dipl.-Inform. Philip Nicolai
Institute for Process Control and Robotics (IPR)
10 Operation Room Supervision for Safe Robotic Surgery with a multi 3D-Camera
Setup – IROS 2011 Workshop on Safer Surgical Robotics
Time of Flight cameras
Measurement principle
Short infrared pulses
Time measurement until reflection
Distance calculation
Accuracy
Dependens upon distance and reflectivity
Error in the cm range
Advantages
No moving mechanical parts
Detection of objects without markers
No expensive calculations
29.09.2011
Real-time sensing of
markerless objects
Dipl.-Inform. Philip Nicolai
Institute for Process Control and Robotics (IPR)
11 Operation Room Supervision for Safe Robotic Surgery with a multi 3D-Camera
Setup – IROS 2011 Workshop on Safer Surgical Robotics
Hybrid sensing I: PMD[vision] ToF cameras
PMD[vision] S3
Range: up to seven meters
Frame rate: 20 fps
Resolution: 64 x 48
Connection: Ethernet
PMD[vision] CamCube 2.0
Range: up to seven meters
Frame rate: 30 fps
Resolution: 204 x 204
Connection: USB
29.09.2011
Dipl.-Inform. Philip Nicolai
Institute for Process Control and Robotics (IPR)
12 Operation Room Supervision for Safe Robotic Surgery with a multi 3D-Camera
Setup – IROS 2011 Workshop on Safer Surgical Robotics
Hybrid sensing I: PMD[vision] ToF cameras
29.09.2011
Dipl.-Inform. Philip Nicolai
Institute for Process Control and Robotics (IPR)
13 Operation Room Supervision for Safe Robotic Surgery with a multi 3D-Camera
Setup – IROS 2011 Workshop on Safer Surgical Robotics
Hybrid Sensing II: ARTtrack2
Properties
Marker based tracking system
Very high accuracy
Update rate: 60 Hz
Advantages
Precise pose detection and tracking
Detailled object representation
ARTtrack 2
Commercially available
Range up to four meters
Used in a six camera configuration
30.09.2011
Real-time sensing of
objects with markers
Dipl.-Inform. Philip Nicolai
Institute for Process Control and Robotics (IPR)
14 Operation Room Supervision for Safe Robotic Surgery with a multi 3D-Camera
Setup – IROS 2011 Workshop on Safer Surgical Robotics
Hybrid Sensing: Microsoft Kinect
Measurement principle: Light Coding
Constant projection of a static pattern
by infrared laser
Scene capture by infrared camera
Depth calculation by triangulation
Properties
Low-cost device
RGB-D
Game console „toy“
SDK available
No further
configuration
htt
p:/
/ww
w.f
lickr.
com
/photo
s/a
udre
ypenven/5
198670323/s
izes/o
/in/s
et-
72157625454305998/
htt
p:/
/ww
w.r
os.o
rg/w
iki/kin
ect_
calib
ration/technic
al?
action=
AttachF
ile&
do=
get&
targ
et=
kin
ect_
ir_pattern
.jpg
30.09.2011
Dipl.-Inform. Philip Nicolai
Institute for Process Control and Robotics (IPR)
15 Operation Room Supervision for Safe Robotic Surgery with a multi 3D-Camera
Setup – IROS 2011 Workshop on Safer Surgical Robotics
Problem: crosstalk between IR devices?
Evaluation of PMD[vision] cameras, Microsoft Kinect and ARTtrack2
Evaluation of camera performance in different configurations
Standalone
Multi-camera with same type
Multi-camera with different types
30.09.2011
Dipl.-Inform. Philip Nicolai
Institute for Process Control and Robotics (IPR)
16 Operation Room Supervision for Safe Robotic Surgery with a multi 3D-Camera
Setup – IROS 2011 Workshop on Safer Surgical Robotics
Multiple PMD[vision] S3
30.09.2011
Dipl.-Inform. Philip Nicolai
Institute for Process Control and Robotics (IPR)
17 Operation Room Supervision for Safe Robotic Surgery with a multi 3D-Camera
Setup – IROS 2011 Workshop on Safer Surgical Robotics
PMD cameras: synchronisation
Possible solutions
Frequency multiplexing: usage of different modulation frequencies
Time multiplexing: alternative triggering of cameras
Implemented solution: split S3 cameras
into two groups
Frequency multiplexing in each group
Time multiplexing between groups
Update rates
CamCube : 20 fps
S3-groups: 10 fps each
„interlaced“ scene update rate: 20 fps
30.09.2011
Dipl.-Inform. Philip Nicolai
Institute for Process Control and Robotics (IPR)
18 Operation Room Supervision for Safe Robotic Surgery with a multi 3D-Camera
Setup – IROS 2011 Workshop on Safer Surgical Robotics
Multiple PMD[vision] S3
30.09.2011
Dipl.-Inform. Philip Nicolai
Institute for Process Control and Robotics (IPR)
19 Operation Room Supervision for Safe Robotic Surgery with a multi 3D-Camera
Setup – IROS 2011 Workshop on Safer Surgical Robotics
Results: sensing system
28.09.2011
Dipl.-Inform. Philip Nicolai
Institute for Process Control and Robotics (IPR)
20 Operation Room Supervision for Safe Robotic Surgery with a multi 3D-Camera
Setup – IROS 2011 Workshop on Safer Surgical Robotics
Outlook
Data interpretation
Detection and interpretation of humans and their gestures
Collision avoidance
Built a 3D occupancy grid
Subtract robot based on its
current pose
Use environment information
for path-planning and 7 DOF
pose optimizations
General setup
Optimization of spatial configuration of cameras
Introduce redundancy
30.09.2011
SAFROS – Patient Safety in Robot Surgery
ACTIVE – Active Constraints Technologies for ill-defined or volatile Environments
Dipl.-Inform. Philip Nicolai
Institute for Process Control and Robotics (IPR)
21 Operation Room Supervision for Safe Robotic Surgery with a multi 3D-Camera
Setup – IROS 2011 Workshop on Safer Surgical Robotics
28.09.2011
This work is supported in the seventh framework program of the European
Union in the projects „SAFROS – Patient Safety in Robotic Surgery“ under
Grant No. 248960 and „ACTIVE – Active Constraints Technologies for ill-
defined or volatile Environments“ under Grant No. 270460
top related