Swarm robotics

PRESENTATION OVERVIEW

Introduction To Gestures Design Of The Proposed System Hand Detection Hardware Implementation Conclusion / Future Work

Real World Insect

Examples

Natural swarmsNatural swarms

Decentralised – no-one in control Individuals are simple and autonomous Local communication and control Cooperative behaviours emerge through self-

organisatione.g. repairing damage to nest, foraging for food,

caring for brood

Bees

An In-depth Look at Real Ant Behaviour

Interrupt The Flow

The Path Thickens!

Adapting to Environment Changes

The New Shortest Path

Adapting to Environment Changes

Welcome to the Real World

Robots

• Collective task completion

• No need for overly complex algorithms

• Adaptable to changing environment

Swarm roboticsSwarm robotics Inspired by self-organisation of social insects Using local methods of control and communication

Local control: autonomous operationLocal communication: avoids bottlenecksScalable – new robots can be added, or fail without need for recalibrationSimplicity – cheap, expendable robots

Self-organisation Decentralisation

Collective RoboticsSwarm Robotics

Collective RoboticsSwarm Robotics

Introduction To Gestures Introduction To Gestures

Gestures can originate from any body motion.

Commonly from face/hand. Gesture recognition-understand human

body language. Help human to interact with machines

without any mechanical devices.

Disadvantages of centralised control and communication.

Disadvantages of centralised control and communication.

Central control: failure of controller implies failure of whole system

Robot to robot communication becomes very complex as number of robots increases.

Communication bottlenecks Adding new robots means changing the

communication and control system

Design Of The Proposed SystemDesign Of The Proposed System

Hand detection Feature extraction Gesture recognition Goal directed navigation of swarm

robots.

Hand Detection Hand Detection Hand detection

– Detection of hand in an image, background objects are avoided for feature extraction.

– Skin color is the key component.– Detecting skin and non-skin.– Detecting image pixels and regions that contains skin-tone

color.– Background is controlled.– Appearance depends on illumination conditions.

Two phases– Training phase– Detection phase

Training Phase Training Phase

Three steps– Collecting a database of skin patches from different

images

– Choosing a suitable color space

– Learning the parameters of skin classifier

Detection Phase Detection Phase

Two stepsConverting the image into some color space that was used in training phase.

Classifying each pixel using the skin classifier to either a skin or non-skin.

RGB color space Skin classifier

Variety of classification techniques

Any pixel which color falls inside the skin color class boundary is labeled as skin.

Feature Extraction Feature Extraction

Feature-An interesting part of an image. No exact definition. Depends on the problem. Transforming the input data into set of features. Result is a feature vector. Features extracted are invariant to image scaling,

rotation and less affected to changes in illumination. SIFT feature extraction.

Hand tracking and Feature extraction.

Image frame from webcam

Tracking hand by

skin detection

SIFT feature extraction

Finding match

Gesture database

Gesture1 Gesture2 Gesture3

Action1 Action2 Action3

Navigation of swarm

Hardware Implementation Hardware Implementation Foot-bot robot

Applications Of The System Applications Of The System

Three foot-bots.

Applications of swarm approachApplications of swarm approach

Some tasks are particularly suited to group of expendable simple robots e.g. - cleaning up toxic waste- exploring an unknown planet- pushing large objects

- surveillance and other military applications

ConclusionConclusion

Hand detection and feature extraction removes noise from the image.

System performance and accuracy will increase.

Swarm robots movement can be controlled through gestures.

Future WorkSpeech recognition.

Dumb parts, properly connected into a swarm, yield smart results.

Kevin Kelly

Satellite

Maintenance

The Future?Medical

Interacting Chips in

Mundane Objects

Cleaning Ship

Hulls

Pipe Inspection

Pest Eradication

Miniaturiz

ation

Engine Maintenance

Telecommunications

Self-Assembling

Robots

Job Scheduling

Vehicle Routin

g Data Clustering

Distri

buted M

ail

Syste

ms

Optim

al

Resource

Allocation

Combinatorial

Optimization

1. C.C.Wang, K.C.Wang.: Hand Posture Recognition Using Adaboost with SIFT For Human Robot Interaction, in Robotics: Viable Robotic Service to Human, Springer-2009.

2. M. Kolsch and M. Turk.: Robust hand detection, in IEEE International Conference on Automatic Face and Gesture Recognition, 2004.

3. Cristina Manresa, Javier Varona, Ramon Mas and Francisco J.Perales,.: Hand Tracking and Gesture Recognition for Human-Computer Interaction , Electronic Letters on Computer Vision and Image Analysis 5(3):96-104, 2005.

4. Alessandro Giusti, Jawad Nagi, Luca M. Gambardella, Gianni A. Di Caro : Distributed Consensus for Interaction between Humans and Mobile Robot Swarms (Demonstration).

5. Ihab Zaqout, Roziati Zainuddin, Sapian Baba,.: Pixel-Based Skin Color Detection Technique, in Machine Graphics and Vision, 2005. FLEXChip Signal Processor (MC68175/D), Motorola, 1996.

6. Qiu-yu Zhang, Mo-yi Zhang, Jian-qiang Hu,.: Hand Gesture Contour Tracking Based on Skin Color Probability and State Estimation Model, Journal of Multimedia, Vol. 4, No. 6, December 2009. A. Karnik, “Performance of TCP congestion control with rate feedback: TCP/ABR and rate adaptive TCP/IP,” M. Eng. thesis, Indian Institute of Science, Bangalore, India, Jan. 1999.

7. Lars Bretzner, Ivan Laptev, Tony Lindberg,.: Hand Gesture Recognition using Multi-Scale Colour Features, Hierarchical Models and Particle Filtering , Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition (2002). Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specification, IEEE Std. 802.11, 1997.

References

Contd…• G. Lee and N. Y. Chong, “Decentralized formation control for small scale

robot teams with anonymity,” Mechatronics, vol. 19, no. 1, pp.85–105,2009.

• G. Lee and N. Y. Chong, “A geometric approach to deploying robot swarms,” Ann. Math. Artif. Intell., vol. 52, no. 2–4, pp. 257–280, 2009.

• E. Sahin, “Swarm robotics: From sources of inspiration to domains of• application,” in Proc. 8th Int. Conf. Simulation of Adaptive

Behavior(LNCS),2005,vol.3342,pp.10–20.• H. Niwa, K. Kodaka, Y. Sakamoto, M. Otake, S. Kawaguchi, K. Fujii, Y.

Kanemori, and S. Sugano, “GPS-based indoor positioning system with multi-channel pseudolite,” in Proc. IEEE Int. Conf. Robot. Autom., 2008, pp. 905–910. Faraj Alhwarin, Chao Wang, Danijela Risti -Durrant, Axel Graser, Improved SIFT-Features Matching for Object Recognition, BCS International Academic Conference- Visions of Computer Science,2008.

Thank you