Detection & Identification of People & Animals by Exploration Robots
Craig Hahn
April 13, 2009
Importance
• robots must interact safely with humans and animals
Image: http://depletedcranium.com/riman
Importance
• many “exploration” applications involve close interaction with humans (such as search-and-rescue operations)
• people and animals are often the most dynamic part of a real-world environment
• humans and animals must be identified quickly in all such situations
Basic Methods
• Stationary face detection using basic pattern recognition techniques
Image: http://www.motorauthority.com/wpcontent/uploads/odds_and_ends/2008/3/2/face_pic.jpg
Facial Recognition
• Easily done but of little use to mobile robots:
– robot and target likely moving
– faces are not always visible
Image: http://www-robotics.jpl.nasa.gov/roboticImages/img840-192-browse.jpg
Basic Methods
• Tracking using stereo vision cameras• problem:
– distinguishing people from their surrounding environment, especially moving people
Basic Methods
• Use laser rangefinders to detect larger moving objects (not just looking at people’s faces)– But again you can’t always distinguish people from their
surrounding environment using only laser rangefinders– possible problems:
» people too close to other objects in environment (walls)» other objects (tables, chairs, etc.) mistaken for people
Better Idea
• Combine data from multiple sources• optical sensors• laser rangfinders• auditory systems• cameras• pattern recognition• sonar
Kyoto University
• Created a robot that combines facial recognition and auditory data
• detect people by looking for faces
• track them using stereo vision and sound source detection
Autonomous Systems Lab Swiss Federal Institute of Technology
Lausanne
• Mobile robot Robox finds (moving) objects by detecting changes in a static environment with a laser rangefinder...
Robox
• …then checks such objects using visual cues to determine if they are people
Department of Neuroinformatics and Cognitive Robotics
Ilmenau Technical University
– HOROS• laser-rangefinder • sonar system • omnidirectional
camera
sensory input from each sensing system
- assigns each set of data a Gaussian probability distribution
(,C)- mean represents the position of the detection- covariance matrix C represents the uncertainty about the
position
reasonably successful in a real-world environment
Advantages
• HOROS doesn’t rely on just one sensing system
• easy to add more sensing systems• just factor in another Gaussian distribution
Centre for Applied Autonomous Sensor Systems Örebro University
• Mobile robot identifies people in an indoor environment – laser rangefinder– camera
• Learns a color model of a person
• hair• skin• clothes• shoes
• laser rangefinder scans the environment
• detects and tracks people
• local minima correspond to the legs of a person
• image from camera is broken down into three parts
• color data is extracted from each part of the image
• hue-saturation-value color model
• color data is put into self-organizing maps (SOMs)
• color model of a person is learned based on the best fit
Assignment• Come up with another robotic system that combines
three or more different sensing methods to identify humans or animals, and tell the difference between a human and a humanoid robot– explain:
• why you chose the methods that you did• how each method can be used to identify humans or animals and
differentiate humans from humanoid robots• how the data from each can be integrated together in a
meaningful way• if applicable, identify specific equipment that could be used to
implement your system
Supply Companies• SICK laser rangefinders:
Image: http://www.pages.drexel.edu/~kws23/tutorials/sick/sickLMS200.jpg
• widely used
Omnidirectional Cameras• Nikon
• Sony
• Remote Reality
Image: http://blog.wired.com/photos/uncategorized/2008/07/01/hal9000_focus_jpg.jpg
Other Important Companies
• JAXY Optical Instrument Company (laser radar systems, sonar systems
• Opnext, Inc. (laser rangefinders)
• Instruments, Inc. (amplifiers, sonar systems, range sensors)
Cutting-Edge Researchers• Ilmenau Technical University
Department of Neuroinformatics and Cognitive Robotics
• Örebro University Dept. of Technology, Centre for Applied Autonomous Sensor Systems
• Kyoto University Department of Intelligence Science and Technology
• Swiss Federal Institute of Technology Lausanne Autonomous Systems Lab
• Albert-Ludwigs-Universität Freiburg Social Robotics Laboratory, Institut für Informatik
Future Improvements
• Improvements are expected in:– identifying more dynamic
objects (people and animals)
– differentiating between multiple objects of the same type
– differentiating between many different kinds of moving people and objects
• being worked on at Albert-Ludwigs-University in Freiburg, Germany
Other Improvements
– identifying and classifying the behaviors of humans and animals
– same accuracy of prediction and detection with:• less data
• smaller periods of observation
• can reach conclusions more quickly (less computational time)
References• Sensor Fusion Using a Probabilistic Aggregation Scheme For
People Detection and Tracking Department of Neuroinformatics and Cognitive Robotics Ilmenau Technical University <http://www.tu-ilmenau.de/fakia/fileadmin/template/startIA/neuroinformatik/publications/conferences_int/Martin-ECMR-05b.pdf>
• Person Identification by Mobile Robots in Indoor Environments Centre for Applied Autonomous Sensor Systems Dept. of Technology, O¨ rebro University <ftp://aass.oru.se/pub/tdt/rose03.pdf>
• Unsupervised Learning and Classification of Dynamic Objects Social Robotics Laboratory Institut für Informatik Albert-Ludwigs-Universität Freiburg <http://srl.informatik.uni-freiburg.de/papers/luberRSS08.pdf>
References• Auditory fovea based speech separation and its application to
dialog system Kyoto University, Department of Intelligence Science and Technology <http://thamakau.usc.edu/Proceedings/ICSLP%202002/ICSLP/PDF/AUTHOR/SL021615.PDF>
• Robox at expo.02: A large scale installation of personal robots Swiss Federal Institute of Technology Lausanne, Autonomous Systems Lab <http://infoscience.epfl.ch/record/97516>