Detection & Identification of People & Animals by Exploration Robots

Craig Hahn

April 13, 2009

Importance

• robots must interact safely with humans and animals

Image: http://depletedcranium.com/riman

Importance

• many “exploration” applications involve close interaction with humans (such as search-and-rescue operations)

• people and animals are often the most dynamic part of a real-world environment

• humans and animals must be identified quickly in all such situations

Basic Methods

• Stationary face detection using basic pattern recognition techniques

Image: http://www.motorauthority.com/wpcontent/uploads/odds_and_ends/2008/3/2/face_pic.jpg

Facial Recognition

• Easily done but of little use to mobile robots:

– robot and target likely moving

– faces are not always visible

Image: http://www-robotics.jpl.nasa.gov/roboticImages/img840-192-browse.jpg

Basic Methods

• Tracking using stereo vision cameras• problem:

– distinguishing people from their surrounding environment, especially moving people

Basic Methods

• Use laser rangefinders to detect larger moving objects (not just looking at people’s faces)– But again you can’t always distinguish people from their

surrounding environment using only laser rangefinders– possible problems:

» people too close to other objects in environment (walls)» other objects (tables, chairs, etc.) mistaken for people

Better Idea

• Combine data from multiple sources• optical sensors• laser rangfinders• auditory systems• cameras• pattern recognition• sonar

Kyoto University

• Created a robot that combines facial recognition and auditory data

• detect people by looking for faces

• track them using stereo vision and sound source detection

Autonomous Systems Lab Swiss Federal Institute of Technology

Lausanne

• Mobile robot Robox finds (moving) objects by detecting changes in a static environment with a laser rangefinder...

Robox

• …then checks such objects using visual cues to determine if they are people

Department of Neuroinformatics and Cognitive Robotics

Ilmenau Technical University

– HOROS• laser-rangefinder • sonar system • omnidirectional

camera

sensory input from each sensing system

- assigns each set of data a Gaussian probability distribution

(,C)- mean represents the position of the detection- covariance matrix C represents the uncertainty about the

position

reasonably successful in a real-world environment

Advantages

• HOROS doesn’t rely on just one sensing system

• easy to add more sensing systems• just factor in another Gaussian distribution

Centre for Applied Autonomous Sensor Systems Örebro University

• Mobile robot identifies people in an indoor environment – laser rangefinder– camera

• Learns a color model of a person

• hair• skin• clothes• shoes

• laser rangefinder scans the environment

• detects and tracks people

• local minima correspond to the legs of a person

• image from camera is broken down into three parts

• color data is extracted from each part of the image

• hue-saturation-value color model

• color data is put into self-organizing maps (SOMs)

• color model of a person is learned based on the best fit

Assignment• Come up with another robotic system that combines

three or more different sensing methods to identify humans or animals, and tell the difference between a human and a humanoid robot– explain:

• why you chose the methods that you did• how each method can be used to identify humans or animals and

differentiate humans from humanoid robots• how the data from each can be integrated together in a

meaningful way• if applicable, identify specific equipment that could be used to

implement your system

Supply Companies• SICK laser rangefinders:

Image: http://www.pages.drexel.edu/~kws23/tutorials/sick/sickLMS200.jpg

• widely used

Omnidirectional Cameras• Nikon

• Sony

• Remote Reality

Image: http://blog.wired.com/photos/uncategorized/2008/07/01/hal9000_focus_jpg.jpg

Other Important Companies

• JAXY Optical Instrument Company (laser radar systems, sonar systems

• Opnext, Inc. (laser rangefinders)

• Instruments, Inc. (amplifiers, sonar systems, range sensors)

Cutting-Edge Researchers• Ilmenau Technical University

Department of Neuroinformatics and Cognitive Robotics

• Örebro University Dept. of Technology, Centre for Applied Autonomous Sensor Systems

• Kyoto University Department of Intelligence Science and Technology

• Swiss Federal Institute of Technology Lausanne Autonomous Systems Lab

• Albert-Ludwigs-Universität Freiburg Social Robotics Laboratory, Institut für Informatik

Future Improvements

• Improvements are expected in:– identifying more dynamic

objects (people and animals)

– differentiating between multiple objects of the same type

– differentiating between many different kinds of moving people and objects

• being worked on at Albert-Ludwigs-University in Freiburg, Germany

Other Improvements

– identifying and classifying the behaviors of humans and animals

– same accuracy of prediction and detection with:• less data

• smaller periods of observation

• can reach conclusions more quickly (less computational time)

References• Sensor Fusion Using a Probabilistic Aggregation Scheme For

People Detection and Tracking Department of Neuroinformatics and Cognitive Robotics Ilmenau Technical University <http://www.tu-ilmenau.de/fakia/fileadmin/template/startIA/neuroinformatik/publications/conferences_int/Martin-ECMR-05b.pdf>

• Person Identification by Mobile Robots in Indoor Environments Centre for Applied Autonomous Sensor Systems Dept. of Technology, O¨ rebro University <ftp://aass.oru.se/pub/tdt/rose03.pdf>

• Unsupervised Learning and Classification of Dynamic Objects Social Robotics Laboratory Institut für Informatik Albert-Ludwigs-Universität Freiburg <http://srl.informatik.uni-freiburg.de/papers/luberRSS08.pdf>

References• Auditory fovea based speech separation and its application to

dialog system Kyoto University, Department of Intelligence Science and Technology <http://thamakau.usc.edu/Proceedings/ICSLP%202002/ICSLP/PDF/AUTHOR/SL021615.PDF>

• Robox at expo.02: A large scale installation of personal robots Swiss Federal Institute of Technology Lausanne, Autonomous Systems Lab <http://infoscience.epfl.ch/record/97516>