definition of high-level coordination techniques for...

EAIAEAIA’’2007, Definition of High2007, Definition of High--Level Coordination Techniques for Robotic Teams, Level Coordination Techniques for Robotic Teams, LisboaLisboa, Portugal, Portugal

1

IEEE ISIE 2007

Definition of High-Level CoordinationTechniques for Robotic Teams

Luís Paulo Reis [email protected]

LIACC – Artificial Intelligence and Computer Science Lab.Faculty of Engineering of the University of Porto

Rua Dr. Roberto Frias, s/n 4200-465 Porto, Portugalhttp://www.fe.up.pt/~lpreis

EAIA2007, Definition of HighEAIA2007, Definition of High--Level Coordination Techniques for Robotic Teams, Level Coordination Techniques for Robotic Teams, LisboaLisboa, Portugal , Portugal 22

Presentation OutlineIntroductionRoboCup InitiativeRoboCup LeaguesSimulator – Soccer ServerFC Portugal ProjectCoordination Methodologies and ResultsLegged RobotsResearch Areas and Conclusions

Introduction | RoboCup | Rob.Leagues | Simulator | FCPortugal | Coordination Meth. | Legged Robots | Conclusions


IntroductionArtificial IntelligenceIntelligent AgentsEnvironment and Its ComplexityAgent Architectures and ApplicationsMulti-Agent Systems

RoboCup InitiativeRoboCup LeaguesSimulator – Soccer ServerFC Portugal ProjectCoordination Methodologies and ResultsLegged RobotsResearch Areas and Conclusions

Presentation OutlineIntroduction | RoboCup | Rob.Leagues | Simulator | FCPortugal | Coordination Meth. | Legged Robots | Conclusions


Artificial IntelligenceIntelligence

“Capacity to solve new problems through the use of knowledge”

Artificial Intelligence“Science concerned with building intelligent machines, that is, machines that perform tasks that when performed by humans require intelligence”



Autonomous AgentsTraditional Definition:

“Computational System, situated in a given environment, that has the ability to perceivethat environment using sensors and act, in an autonomous way, in that environment using its actuators to fulfill a given function.”

?

agent

percepts

sensors

actions

effectors

environment



Autonomous AgentsInformation AgentElectronic Commerce Agent Air Traffic ControllerAutonomous PilotPersonal AssistantMeeting SchedulerGame Playing AgentAutonomous Robot



Intelligent Agents?!Are these Agents? Are they Intelligent?

Thermostat?Answering Machine?Phone?Pencil?Java or C++ Object?

Intelligence:Is this a binary (yes/no) scale?Is this a continuous scale (0-infinit)?



Agent RequisitesTraditional definition include to much or leaves “holes”!Requisites:

Perceive its environment (sensors)Decide actions to execute (“think”)Execute actions in environment using its actuatorsCommunicate?Perform a complex function?



Intelligent AgentsAgent:

Perceive its environment using sensors and executes actions using its actuatorsSensors:

Eyes, ears, nose, touch, …Actuators:

Legs, Arms, hands, vocal cords, …Robotic Agent:

Sensors: Cameras, sonar, infra-red, microphone

Actuators: Motors, wheels, manipulators, speakers

?

agent

percepts

sensors

actions

effectors

environment



Agent’s ProprietiesMain Proprieties:

AutonomyPro-ActivityReactivitySocial Ability

Other Proprieties:MobilityTruth and BenevolenceKnowledge and BelievesIntentions, ObligationsRationalityIntelligenceLearningTemporal ContinuityCharacter



Agents vs. ObjectsEssential Differences:

Agents decide what to doObject methods are called externallyAgents react to sensors and control actuators

“Objects do it for free; agents do it for money”



Agent ArchitecturesPhysical Agents vs. Software AgentsAgent’s Structure:

Simples Reflex Agents (reactive) Agents with world representationAgents Based on ObjectivesAgents Based on UtilityBDI Agent – Believes, Desires and IntentionsAgents with complex architectures(learning, decision, planning, cooperation, etc.)



Agent ArchitecturesCharacteristics:

Simplicity FunctionalityExpansibility Portability

Types:DeliberativeReactiveHybrid

Difficult to Balance:ReactivityDeliberationSociability

How to do it?



Agent Architectures - HybridRaciocínio Acções

PossíveisDecisão

Deliberativa

Actualização doEstado do

Mundo

Conhecimento /Estado do

Mundo

Objectivos

Comunicação

Percepção

Mundo Acção

Decisão Reactiva

Execução daAcção

Regras SimplesSituação/Acção

(comportamento)

Interpretação daPercepção

Fusão dasDecisões

ComponenteReactiva

ComponenteDeliberativa



Decisão(resolução deconflitos entre

objectivos)

Execução daAcção

Actualização doEstado doMundo e

Modelos dosoutros Agentes

Conhecimento /Estado do

Mundo - Próprio

Comunicação

Percepção

Mundo

Comunicação

Interpretação daPercepção e

Comunicação ConhecimentoGeral

Agenda paraCoordenação de

Actividades

Planos dosOutros Agentes

Intenções dosOutros Agentes

Meta-Regras deRaciocínio

Modelos dosoutros Agentes

Raciocínio deModelização deoutros Agentes

Modelos doAmbiente

Construção dePlanos

(resolução deinteracções)

Acção

Execução daComunicaçãoConhecimento

de Aprendizagem

PlanosPredefinidos

Preferências dosAgentes

Activação dePlanos

Agenda deObjectivos

RecursosDisponíveis

Espaço deDecisões

Espaço deAcções

Planos Activados

Novos Planos

Execução dosPlanos Activados

Resultados daSimulação

Simulação dePlanos e dasReacções de

outros Agentes

Aprendizagem

Selecção eAdaptação dePlanos Pré-Definidos



Agent ApplicationsIndustrial ApplicationsInformation SearchElectronic CommerceEntertainment Applications and GamesMedical ApplicationsIntelligent SimulationRobot ControlAutonomous DrivingIntelligent BuildingsResearch Competitions



EnvironmentAccessible vs. InaccessibleStatic vs. DynamicDiscrete vs. ContinuousDeterministic vs. Non DeterministicSingle Agent vs. Multi-Agent



Multi-Agent System (MAS)Composed by multiple agentsthat:

Exhibit autonomous behaviorInteract with the other agents in the system



MAS MotivationProblem DimensionsLegacy SystemsNatural Solution (distributed problems)Distributed knowledge or informationHuman-machine interfaceProject Clarity and simplicityEfficiencyRobustness and ScalabilityProblem divisionInformation privacy



Agents and Multi-Agent SystemsTo build individual autonomous intelligent agents is importantHowever:

Agents don’t leave alone…Necessary to work in groupMulti-Agent ApplicationsRobotic Agents: Body, complex environmentCoordination in necessary: “to work in harmony in a group”



Reasons for CoordinationDependencies in agent actionsNeed to respect global constraintsNo agent, individually has enough resources, information or capacity to execute the task or solve the complete problemEfficiency:

Information exchange or tasks divisionPrevent anarchy and chaos:

Partial vision, lack of authority, conflicts, agent’s interactions



Cooperative vs Competitive MAS

Cooperative MAS:Usually projected by a single entityGlobal utility and global performance

Competitive MAS (“self-interested agents”): Each agent has a distinct designerAgents have their own motivation and agendaAgents are interested in their own utility Usual in negotiation, electronic commerce, internet



Presentation OutlineIntroductionRoboCup Initiative

Objectives of the RoboCup InitiativeRoboSoccer - Global PerspectiveComplexity of RoboSoccerRoboCup FederationRoboCup Official Competitions

RoboCup LeaguesSimulator – Soccer ServerFC Portugal ProjectCoordination Methodologies and ResultsLegged RobotsResearch Areas and Conclusions



RoboCupObjectives of the RoboCup Initiative

Joint International Project: (Distributed) Artificial IntelligenceIntelligent Robotics

Soccer – Central Research Topic:Very complex collective gameHuge amount of technologies involved:

Autonomous Agents, Multi-Agent Systems, Cooperation, Communication, Robotics, Sensor Fusion, Real-Time Reasoning, Machine Learning, etc



RoboCup Objectives of the RoboCup InitiativeMain Goal of the RoboCup Initiative:

“By 2050, develop a team of fully autonomous humanoid robots that may win against the human world champion team in soccer!”



RoboCup RoboSoccer – Global Perspective

Soccer LeaguesSimulation – SoccerServer (2D, 3D, Coach)Simulation New: PV-League, Nanogram, Microsoft Robotics Soccer Challenge Robots Small-SizeRobots Medium-SizeLegged Robots (Aibo Dogs - Sony)Humanoid Robots

RoboCup RescueSimulation, Virtual, Robotic

RoboCup JúniorRoboCup @ Home



RoboCup The Complexity of RoboSoccer

DistributedCentralizedControl

Non SymbolicSymbolicSensor Reading

IncompleteCompleteWorld State Accessibility

Real-TimeIn TurnsState Change

DynamicStaticEnvironment

RoboSoccerChessComparison with Chess



RoboCup The RoboCup Federation

President: Minoru Asada (Japan) -> Manuela Veloso (USA)

TrusteesExecutive CommitteeTechnical Committees for each LeagueMaintenance CommitteesLocal Federations/Committees



RoboCup RoboCup Official Competitions

RoboCup 1997 – Nagoya (Japan)RoboCup 1998 – Paris (France)RoboCup 1999 – Stockholm (Sweden)RoboCup 2000 – Melbourne (Australia)RoboCup 2001 – Seattle (USA)RoboCup 2002 – Fukuoka (Japan)RoboCup 2003 – Padua (Italy)RoboCup 2004 – Lisbon (Portugal)RoboCup 2005 – Osaka (Japan)RoboCup 2006 – Bremen (Germany)RoboCup 2007 – Atlanta (USA)Local Championships:

German Open (European), Japanese Open, Australian Open, American Open, Portuguese Open, Iranian Open, China Open!



RoboCup - Participants

Awarded Countries:JapanGermanyUSAAustraliaChinaIranPortugalHolland



RoboCup 2002 – Fukuoka DomeIntroduction | RoboCup | Rob.Leagues | Simulator | FCPortugal | Coordination Meth. | Legged Robots | Conclusions


RoboCup 2002 – Fukuoka Dome1500 Researchers and over 150000 spectators



Presentation Structure RoboCup Leagues

IntroductionRoboCup InitiativeRoboCup Leagues

Simulation League (2D, 3D, Coach, PV, MRSC, Nanogram)Small-Size LeagueMiddle-Size LeagueLegged LeagueHumanoid LeagueRescue Leagues

Simulator – Soccer ServerFC Portugal ProjectCoordination Methodologies and ResultsLegged RobotsResearch Areas and Conclusions



RoboCup Leagues Simulation League 2D

Virtual Robots (software agents)105*68m Virtual FieldAgents controlled by different computers (or processes)Simulator sends perception and receives actions from agentsTeams of 11 players plus a coach



RoboCup Leagues Simulation League




Most Significant Teams:FC Portugal (Portugal) – Univ. of Porto and Aveiro

Tactics, Formations, RolesFlexible Teamwork, Configurable StrategyStrategic vs Active BehaviorSBSP - Situation Based Strategic PositioningDPRE - Dynamic Positioning and Role ExchangeADVCOM and SLM - Strategic Looking MechanismVisual DebuggerCoach UnilangRelease of FCPAgent Source Code

UVA Trilearn (Holland)Coordination GraphsRelease of UVA Source Code




Most Significant Teams:CMUnited (USA) - Carnegie Mellon University

Formations, SPAR and RolesLayered LearningRelease of the low-level Source Code

Karlsruhe Brainstormers (Germany) - Univ. KarlsruheLearning of Skills - Reinforcement Learning

Tsinghuaeolus (China) - Tsinghua UniversityUse of Windows OS, Windows MonitorIndividual Marking, Dribbling Skills

Wright Eagle (China)



RoboCup Leagues Simulation 3D League

Started in 2004Objectives of 3D Simulation:

Replace the 2D environment of previous simulator with a 3D environmentNew, more realistic, physics modelSimulation results should not be dependent on available computational power or on the quality of network resources




Complexities fromreal robotsThird dimensionadds complexityFuture evolution to legged andhumanoid robots




Humanoids startingin 2007Very realistic physics2 vs 2 games



RoboCup Leagues Microsoft Robotics Soccer Challenge



RoboCup Leagues Physical Visualization League

two step motors (1), a li-ion battery (2), a control board (8bit PIC18 processor) (3), Infrared Sensor (4) to receive its commands and aluminium body (5)

Augmented reality setupSmall real robots, Eco-Bes, play soccer on top of a virtual field with a virtual ballSeveral research challenges included:

Vision Based Self Localization, Data Fusion, Real-Time Control, Decision and Cooperation



RoboCup Leagues Physical Visualization League


Camera is calibrated to recognize the field and the robots’ markers.Each robot identified by an individual marker recognized by the vision systemServer application responsible to control socket connections with clients, monitor, camera and communication with infrared USB transmitter. Server, compiles all information and, sends relative positions of all elements in the fieldMonitor application uses the same information to draw the field and project it and shows the virtual ball


RoboCup Leagues Nanogram Competition


Microscopic robots that compete against each other in soccer-related agility challenges Robots measure a few tens of micrometers to a few hundred micrometersMasses ranging from a few nanograms to a few hundred nanogramsPlaying field is a set of insulated interdigitatedelectrodes, across which an AC waveform can be applied


RoboCup Leagues Nanogram Competition

Challenges:2 Millimeter DashSlalom DrillBall-Handling Drill



RoboCup Leagues Small-Size League

15cm RobotsField of 4*3m tableRobots controlled by radio by a single remote computerVision given by a camera placed on top of the fieldTeams of 5 Robots







Most Significant TeamsCornell (USA) - Cornell University

Omni directional driveDribbling mechanismTrajectory generationCooperation mechanisms: Roles, Passes

Fu-Fighters (Germany) - University of BerlinPowerful kicking deviceHeterogeneous robotsHierarchical generation of reactive behaviors




Lucky Star (Singapore) - Ngee Ann PolytechnicStable hardware platform, Good Basic skillsSimple cooperation mechanisms

5DPO (Portugal) - Faculty of Engineering of Univ. PortoModular Flexible SoftwareCooperation mechanisms (roles, tasks and actions)Vision (2 cameras)Tracking using circular bar codesInfrared (IR) communication system

CMU – Carnegie Mellon UniversityPowerful KickerVery good Control

Local Vision Teams!



RoboCup Leagues Middle-Size League

80 cm Robots5x10m FieldAutonomous Robots (local vision and decision)Teams of 4/5 Robots










Most Significant TeamsBrainstormers Tribots

Learning of SkillsPerfect Robot controlWorld and European Champions

CS-Freiburg (Germany) - University of FreiburgMulti-Agent Coordination: Role assignment, positioningRich set of Basic Skills, Behavior NetworksLaser range finder for Self Localization

Sharif CE (Iran) - Sharif UniversityOmnidirectional Castor Wheel, Robust platform, Simple vision




Most Significant TeamsGOLEM Team (Italy) - University of Padua

Omni directional drive and omni directional visionFlexible Roles: Homogeneous Robots

ART - Azzurra Team (Italy) - Univ. Parma, Milano, Padua, Genova and Rome

Heterogeneous robots from different universitiesCooperative behavior: Passes

Portuguese Teams: IsocRob5DPOMinho Team ISEPortoCambada



RoboCup Leagues Legged Robots League

30cm RobotsField of 5*4mAutonomous RobotsTeams of 4 Robots – Sony AIBO dogsEmphasis on Computer Vision and Legged motion










Most Significant TeamsGerman Team – Germany

4 German UniversitiesCoordinationLegged robots Simulation

UNSW (Australia) - University of New South WalesBall control and fast locomotion (supported on elbows)Low-level skillsTeammate recognition and LocalizationSimple high-level strategy

CMPack (USA) - Carnegie Mellon UniversityEmphasis on visionSensor Resetting Localization



RoboCup Leagues Humanoid League

Penalties / Challenges2 vs. 2 games (two sizes)



RoboCup Leagues Humanoid League



RoboCup Rescue is an expansion of RoboCup soccer competitions aiming at researching in socially useful areas

RoboCup RescueIntroduction | RoboCup | Rob.Leagues | Simulator | FCPortugal | Coordination Meth. | Legged Robots | Conclusions


Rescue SimulationAgents:

Police ForceAmbulancesFireFightersCenters

Objectives:Save BuildingsRescue Civilians



RoboCup RescueOne of the 3 major research lines in RoboCup initiativeMain Challenge:

Development of an heterogeneous team capable of saving people and limiting damage in a city after a major earthquake

Rescue Simulation league challenges: Urban virtual scenario, partially unknown, dynamic environmentGlobal strategy for a team, coordinationPartial hierarchical, low bandwidth communicationHeterogeneous agents in MAS systemPrediction of fire spreading and civilians health



RoboCup RescueSimulated Environment

City Virtual Model (Roads, Buildings and Agents)

Simulation of 72 hours after a major natural catastrophe

Emergency and security forces are mobilized in order to minimize life losses and material damages



Partial 3D View of Kobe City (Japan)

Partial 2D View of Foligno City (Italy)

RoboCup RescueVisualization



RoboCup RescueSimulated Environment

Agent Types:Fire Brigades, Police Forces and AmbulancesCommand Centers (superior hierarchical level)

Simulator:Perception/Decision/Communication/ActionFire Spreading and extinguishingBuilding CollapsesCivilian Movement and health evolutionTraffic jams



Division into several simulation sub-modules

Simulation in discrete steps

Permanent changes, development and growing

RoboCup Rescue Simulated Environment



RoboCup RescueSimulated World Objects

Building with increasing level of damage (1 to 4)

Roads with obstructions: (1) Partial (2) Total

Refuge

Refuge (2D and 3D view)

a) Healthy

b) hounded

c) Dead

Civilians with different health levels



RoboCup RescueAmbulances

Ambulance rescuing a civilian

Buried Civilian Ambulances

Healthy Civilian

Function:• Rescue civilians and

take them to the refuges

Associated Challenges:• Rescue scheduling

• Buried agents life time prediction

• Ambulance coordination

• Coordination with agents of different type



Buildings in fireFire Brigades

Building being extinguished

Spring of water

Function:• Extinguish fires and control its expansion

Associated Challenges:• Choose the best region to extinguish

• Choose the best building (preemptive?, direct?)

• Anticipate fire spreading (buildings and human lives)

• Collective and individual management of water in tanks

• Fire Brigades coordination

•Coordination with agents of different type

RoboCup RescueFire Brigades



Policeman approaching new

obstruction

Policeman clearing a road

Obstructed Roads

Function:• Road clearance

Associated Challenges:

• Scheduling of the roads to clear considering: trapped emergency vehicles, principal routes, paths to refuges, fires and

trapped civilians

• Coordination between police forces

• Coordination with agents of different type

RoboCup RescuePolice Forces



RoboCup RescueCenter Agents and Communication

Function:• Interface between different

agent types

• Agent Coordination

Associated Challenges:

• Use global vision to improve high-level decisions

• Coordination of different agent types

• Communication limitation management (low-bandwidth)



RoboCup RescueRobot’s Perception and Action

Robot’s Sensorial information:VisualAuralRadio reception

Robot’s Agent actions:MoveSpeak (voice and radio)Specific agent type actions

Extinguish fireClear roadUnbury civilian



RoboCup RescueExample of a Simulation



Presentation Structure Simulator – Soccer Server

IntroductionRoboCup InitiativeRoboCup LeaguesSimulator – Soccer Server

Global PerspectiveThe Soccer MonitorThe Game, Field and ObjectsPerception and Action of the AgentsAgent ConstructionSimulation Major Difficulties

FC Portugal ProjectCoordination Methodologies and ResultsLegged RobotsResearch Areas and Conclusions



How the Simulator Works?Client-Server SystemAgents (player’s brains) control a single player:

UDP sockets/LinuxServer:

Receives agent commandsSimulates the movement of objects Sends perceptions to the agents

Two teams with 11 players try to score goals!

Server Architecture

Simulator - Soccer ServerGlobal Perspective



Simulation CharacteristicsReal-Time - HumanDistributed – 24 ProcessesInaccessible (hidden), Continuous and Dynamic WorldErrors in: Perception, Movement e Action Limited Resources: Energy and Recovery

Limited Communication Multi-Objective, Cooperative and Adverse Environment

Simulator - Soccer ServerGlobal Perspective



Soccer MonitorLinux or Windows

Simulator - Soccer ServerSoccerMonitor for Linux and Windows



2D SimulationField of 105*68m

22 Players and 1 BallCircles

Marking Flags and LinesAutomatic Referee

Rules Reinforcement Online Coach

Instructions (stopped game)Human Referee

Complex situations

Simulator - Soccer ServerThe Game, Field and Objects



Visual PerceptionLimited vision (angle and distance) with errors

Aural PerceptionLimited bandwidth communication

Physical Perceptionenergy, recovery, speed, …

----------Physical

Adversaries MessagesLimited DistanceAural

ErrorsLimited Distance/AngleVisual

NoiseSpatial LimitationsType

Simulator - Soccer ServerAgent’s Perceptions – Global Perspective



Agent Vision

Simulator - Soccer ServerAgent’s Perceptions - Visual



Agent Aural Perception (communication characteristics):

Not reliable!No guaranty that the message is send or heard! 22 Agents (including opponents) use the same channel!Limited communication capacity and rangeNo information about the sender!

Simulator - Soccer ServerAgent’s Perceptions - Aural



Agent’s ActionsTurn (Angle) - RotateDash (Power) - AccelerateKick (Power, Angle) - Kick the BallCatch (Angle) - Catch the Ball (goal keeper)Turn_Neck (Angle) - Rotate NeckChangeView (VType) - Vision ControlSay (Message) - SpeakTackle (Power) - Tackle OpponentPointTo (Dist,Angle) - Point to a PointAttentionTo (Unum) - Give attention to Teammate

Simulator - Soccer ServerAgent’s Actions



How to Build Basic Agents for RoboSoccer?

PerceptionWorld State UpdateHigh-Level DecisionAction Execution

PerceptionInterpretation

ActionExecutionState of the

World

World - Soccer Server

Perception Action Soccer Monitor

AGENT

ENVIRONMENT

Simulator - Soccer ServerAgent Construction



Simulator - Soccer ServerAgent Construction

Main Questions:What Info in the World State?How to Update the World State?How to Build Complex Actions?

(dribble, pass, move, mark, intercept, etc.)How to use the Limited Resources (Energy)?How to Create Collective Actions?How to Really Create a Team?

(defending/attacking, tactics, formations, player behaviors, etc.)



Presentation Outline IntroductionRoboCup InitiativeRoboCup LeaguesThe Simulator – Soccer ServerFC Portugal Project

Global Vision of the ProjectAgent Architecture and World StateLow-Level SkillsActive Behavior with/without BallAssociated Tools

Coordination Methodologies and ResultsLegged RobotsResearch Areas and Conclusions



FC Portugal ProjectOverview

Project Leaders: Luis Paulo Reis (LIACC/FEUP) and Nuno Lau (IEETA/UA)

Approach: Multi-Agent System with Simple Coordination Techniques

CMU99 Low-level source code used as a starting pointResults:

European Champion - 2000 in Amsterdam (86-0)!World Champion – 2000 in Melbourne (94-0)!European Champion - 2001 in Paderborn (56-4)!3rd World Championship - 2001 in Seattle! (150-5)!Coach Champions 2002 – Fukuoka, JapanCoach 2nd Place – Padova, Italy and Lisbon, Portugal (2003,2004)Euro Champions (3D league) - 2006 in Eindhoven! (17-0)!World Champions (3D league) - 2006 in Bremen! (78-0)!



FC Portugal ProjectAgent’s Architecture

TeamStrategy

ADVCOMStrategies

PerceptionInterpretation

ActionPrediction Action

Tactics

World State

ActionSelection Info

World - Soccer Server

Action

Soccer Monitor

Human Coach

Player Types SLMStrategies

High-LevelDecisionModule

Opp. ModStrategy

Formations

Situations

Situation Info

Game InfoUpdateMulti-LevelWorld State

Perception

Multi-Level World State



FC Portugal ProjectLow-Level Skills – Overview

Low-Level Skills:Movement in the Field (with obstacle avoidance)Ball InterceptionStop and Hold the BallKick the BallDribble

Innovation:Optimization Kick – A kick based on online optimization techniques!

Interception

Kick



FC Portugal ProjectDecision Module Control Flux

STRATEGY – Strategic AnalysisDPRE – Positioning ExchangeADVCOM – Intelligent CommunicationSLM –Intelligent perceptionSBSP – Strategic PositioningDecisions: stopped game, with and without ballExecution and prediction of ActionsWorld State and Situation Update

Initialization

DPRE

ADVCOM

SBSP

STRATEGY

CriticalSituation?

Ball Posses-sion Decision

NoYes

YesStoppedGame?

Stopped GameDecision

ActionExecution

World StateUpdate

No

SLM

Has Ball?Yes

Ball RecoveryDecision

No



FC Portugal Project Active Behavior with Ball - Overview

Ball Possession Actions:Shoot to the GoalPass (rapidly) the BallForward the Ball (to a given point)Dribble with the BallHold the Ball

Decision Matrixes used to Select the Best Action!



FC Portugal ProjectActive Behavior with Ball - Passing

Pass Evaluation:Positional ValuePass ConeInitial ConditionsFinal CongestionOut ProbabilityDistancePosition ConfidenceReception ConfidenceReception TypeG.K. InterceptionShoot Possibility



FC Portugal Project Tools and Debugging

Debugging methodology principles:Offline debuggingVisual debuggingSuperimposed real environment and agent physical knowledgeFeature-focused debuggingInformation structured in layers of abstraction with different detail levels

Development tools implemented:Visual Debugger used to analyze the reasoning of agentsTeam Designer for graphical definition of soccer strategiesOffline client methodology3D Visualization Tools with Intelligent Camera ControlWstateMetrics that evaluates the accuracy of world states



FC Portugal Project Visual Debugger

Testing and tuning of the teamShows what the agents see, hear, feel, think and do!



Project FC Portugal3D Viewer



Presentation Outline IntroductionRoboCup InitiativeRoboCup LeaguesThe Simulator – Soccer ServerFC Portugal ProjectCoordination Methodologies and Results

Strategical CoordinationSBSP – Situation Based Strategic PositioningDPRE – Dynamic Positioning and Role ExchangeADVCOM – Advanced CommunicationSLM – Strategic Looking MechanismMM – Mutual ModelingCoaching – Coach UnilangResults in Controlled Experiments Results in Official Competitions

Legged RobotsResearch Areas and Conclusions



Coordination MethodologiesFormalization of the Team Strategy



Coordination Methodologies SBSP - Situation Based Strategic Positioning

Strategic Situation: SBSP – Strategic PositioningActive Situation (with/without Ball): Active Behavior



Coordination MethodologiesSBSP - Situation Based Strategic Positioning

















Coordination MethodologiesSBSP - Situation Based Strategic PositioningALGORITHM SituationBasedStrategicPositioning(Tactic, Situation, Player) RETURNS PositionPARAMETERS Tactic, Situation, Player{

Formation = SituationFormation(Tactic, Situation)FormHeight = SituationFormationHeight(Tactic, Situation)FormWidth = SituationFormationWidth(Tactic, Situation)Positioning = PlayerGetPositioning(Player)Position = HomeSBSPPosition(Formation, FormWidth, FormHeight, Positioning)Role = FormationGetRole(Formation, Positioning)RoleStrategic = RoleGetStrategicRole(Role)BallAdjPos = AdjustedBallPosition(BallPosition)Position = BasicSBSPPosition(Position, Formation, RoleStrategic, Positioning,

BallAdjPos)Position = RegionalAdjustSBSPPosition(Position, Formation, RoleStrategic,

Positioning, BallAdjPos)Position = LegalAdjustSBSPPosition(Position, BallAdjPos)Position = DomainAdjustSBSPPosition(Position, BallAdjPos)RETURN Position

}



Coordination Methodologies DPRE - Dynamic Positioning and Role Exchange

Dynamic Exchange of Positionings and Behaviors based on utility:

Distances from players positions to its strategic positionsPositioning importance and adequacy of agents

DPRE improves the robotic team collective performanceImportant against opponents with similar collective capabilities





Coordination Methodologies DPRE - Dynamic Positioning and Role ExchangeALGORITHM DynamicPositioningExchange(WorldState, Situation, Positionings)RETURNS Positionings(TeamSize)PARAMETERS WorldState, Positionings[TeamSize], Situation{FOR PL1 = 2 TO TeamSize-1 DO

FOR PL2 = PL1+1 TO TeamSize DO IF PositionValid(PL1) AND PositionValid(PL2) THEN {Dist11 = Distance(Position(Pl1),SBSPPosition(Pl1))Dist22 = Distance(Position(Pl2),SBSPPosition(Pl2))Dist12 = Distance(Position(Pl1),SBSPPosition(Pl2))Dist21 = Distance(Position(Pl2),SBSPPosition(Pl1))Adeq11 = PosAdequacy(Pl1, Positioning[Pl1])Adeq22 = PosAdequacy(Pl2, Positioning[Pl2]) Adeq12 = PosAdequacy(Pl1, Positioning[Pl2]) Adeq21 = PosAdequacy(Pl2, Positioning[Pl1]) Util = ExchangePositions(DPREMode, Situation, Dist11, Dist22, Dist12, Dist21,

Adeq11, Adeq22, Adeq12, Adeq21, PosImportance(Positioning[Pl1]),PosImportance(Positioning([Pl2])

IF Util > ThresUtil(Situation) THEN exchange(Positionings[Pl1], Positionings[Pl2]) }RETURN Positionings

}



Coordination Methodologies ADVCOM –Intelligent Communication

Challenges: What and When to communicate?What to Communicate?

Individual World State – to improve world state accuracyUseful Events – for coordination

When to Communicate?Communication utility is very high or is greater then communication utility (modeled) of teammates!Creation of a Communicated World State!



Coordination Methodologies SLM – Strategic Looking Mechanism

Motivation:Intelligent Sensor utilization!Different World State object’s importanceLook always to the ball? No…

What is it?Looking Direction decided based on the Expected Utility to look on that direction!Possible looking direction with greater utility is chosen!



Coordination Methodologies MM - Mutual Modeling

Motivation and principle:Use Teammate and Opponent Models!Estimate their positions and actions!Fuse this information with visual, sensorial, aural informationEnables more accurate world statesVery useful when vision and sensors is not of good quality



Coordination Methodologies Online Coach and Game Statistics

Game Statistics and Opponent Modeling InformationTime and ResultIndividual Action with/without ballBall losses and Ball recoveriesAttacks and AssistancesBall PossessionBy:

PeriodRegion (from and to)TeamPlayeretc.

Coach

Assistant Coach

Players

Gam

e S

tatis

tics

Definitions

Opp

onen

t Mod

elin

g

Field Information Field

Act

ions

Instructions

Principal Coach



Coordination MethodologiesCoach Unilang – Language to Coach a (Robo)Soccer Team

Base Concepts: Time Periods, Regions, Tactics, Formations, Situations, Player Types

Language Defined in BNFExamples:

<MESSAGE> ::= (<TIME> <ID> <MESSAGE PART> {<MESSAGE PART>})<MESSAGE PART> ::= <DEFINITION_MESS> |<STATISTICS_MESS> | <OPP_MOD_MESS> |

<INSTRUCTION_MESS>

TACTIC_DEFINITION> ::= <TEAM_MENTALITY> <GAME_PACE> <TEAM_PRESSURE> <FIELD_USE> <PLAYING_STYLE> <RISK_TAKEN> <OFFSIDE_TACTIC> <POSITIONING_EXCHANGE_USE> <FORMATIONS_USED>

<FORMATION> ::= <PREDEFINDED_FORMATION> <FORMATION_NAME> | <FORMATION_DEFINITION> <PREDEFINED_FORMATION> ::= 433 | 433att | 442 | 343 | 4123 | 352 …<FORMATION_DEFINITION>::= {(<PLAYER> <POS_NUMBER> <PLAYER_POSITIONING> <PLAYER_TYPE><PLAYER_POSITIONING> ::= <VERTICAL_POSITIONING> <HORIZONTAL_POSITIONING>



Results – Competition and Controlled Experimental Results

Competition Results

Controlled Experiments Results

Competition Local Goals Shoots Scor. Chances Atack Midle Defense

Euro 2000 Amsterdam 87 - 0 136 - 7 194 - 16 53% 35% 12%

RoboCup 2000 Melbourne 94 - 0 144 - 3 201 - 7 59% 33% 8%

German Open 2001 Paderborn 56 - 4 79 - 17 128 - 25 37% 44% 19%

RoboCup 2001 Seattle 150 - 5 195 - 8 281 - 14 60% 32% 8%

Teams GM GS RM RS OM OS Atack Middle Def. Vict. Draw Def.

Very Good 2 0,4 3,1 1 9,8 2,9 40,0% 42,0% 18,0% 70% 20% 10%

Good 8,5 0,1 12,5 0,5 17,1 1,4 56,0% 32,5% 11,5% 100% 0% 0%

Average 21,1 0 27 0 36,7 0,1 74,7% 23,6% 1,7% 100% 0% 0%

ResultsGoals Shoots Scoring Chances Ball Possession




With SBSP – Situation Based Strategic Positioning

Without SBSP (using SPAR – Atractions and Repulsions)

NormalTeams GM GS RM RS AT MF DEF V E D

Very Good 2 0,4 3,1 1 40,0% 42,0% 18,0% 70% 20% 10%Good 8,5 0,1 12,5 0,5 56,0% 32,5% 11,5% 100% 0% 0%

Average 21,1 0 27 0 74,7% 23,6% 1,7% 100% 0% 0%

Goals Shoots Ball Possession Results

SPARTeams GM GS RM RS AT MF DEF V E D

Very Good 0,1 3,0 0,7 5,2 14,0% 49,0% 37,0% 3% 13% 83%Good 0,7 2,0 1,1 4,4 20,5% 43,0% 36,5% 10% 20% 70%

Average 2,8 0,2 4,4 0,9 45,7% 38,7% 15,7% 80% 17% 3%





With DPRE – Dynamic Positioning and Role Exchange

Without DPRE – Dynamic Positioning and Role Exchange


Very Good 2 0,4 3,1 1 40,0% 42,0% 18,0% 70% 20% 10%Good 8,5 0,1 12,5 0,5 56,0% 32,5% 11,5% 100% 0% 0%

Average 21,1 0 27 0 74,7% 23,6% 1,7% 100% 0% 0%


NO DPRETeams GM GS RM RS AT MF DEF V E D

Very Good 1,4 0,8 2,3 1,4 33,7% 41,7% 24,0% 47% 30% 23%Good 7,1 0,3 9,9 0,9 50,0% 33,5% 15,5% 95% 0% 5%

Average 17,9 0,0 21,5 0,0 69,7% 27,3% 3,0% 100% 0% 0%





With ADVCOM, SLM and MM

Without ADVCOM, SLM and MM


Very Good 2 0,4 3,1 1 40,0% 42,0% 18,0% 70% 20% 10%Good 8,5 0,1 12,5 0,5 56,0% 32,5% 11,5% 100% 0% 0%

Average 21,1 0 27 0 74,7% 23,6% 1,7% 100% 0% 0%


No AdvCom

No SLM/MM GM GS RM RS AT MF DEF V E DVery Good 0,4 2,4 0,8 3,8 14,0% 43,0% 43,0% 10% 33% 57%

Good 1,8 1,0 3,7 2,0 24,0% 42,5% 33,5% 70% 20% 10%Average 11,4 0,0 14,9 1,1 38,3% 47,0% 14,7% 100% 0% 0%




Coordination Methodologies Results: European 2000 - Amsterdam

First Stage - Group AFC Portugal 3 - 0 Essex Wizards (England)FC Portugal 13 - 0 Lucky Luebeck (Germany)FC Portugal 4 - 0 Cyberoos (Australia) FC Portugal 22 - 0 Pizza Tower (Italy) FC Portugal 19 - 0 Polytech (Russia) FC Portugal 6 - 0 PSI (Russia)

Quarters and Semi-FinalsFC Portugal 13 - 0 Wroclaw (Poland) FC Portugal 5 - 0 Essex Wizards (England)

FinalFC Portugal 2 - 0 Karlsruhe Brainstormers (Germany)



Coordination Methodologies Results: RoboCup 2000 - Melbourne

First Stage - Group EFC Portugal 33 – 0 Oulu (Finland) FC Portugal 18 – 0 Zeng (Japan) FC Portugal 20 – 0 Robolog (Germany)

Second StageFC Portugal 7 – 0 Essex Wizards (England)FC Portugal 3 – 0 Karlsruhe Brainstormers (Germany)FC Portugal 6 – 0 YowAI (Japan)

Semi-FinalsFC Portugal 6 – 0 ATT-CMU (USA)

FinalFC Portugal 1 – 0 Karlsruhe Brainstormers (Germany)



Coordination Methodologies Results: European 2001 - Paderborn

First Stage - Group AFC Portugal 2 – 0 Sharif Arvand (Iran) FC Portugal 3 – 1 Aras (Iran) FC Portugal 26 – 0 Osnabruck (Germany) FC Portugal 5 – 2 UvaTrilearn (FC Portugal 3 – 1 Lucky Luebeck (Germany)

Second StageFC Portugal 6 – 0 Dr Web (Russia)FC Portugal 9 – 0 Robolog (Germany)FC Portugal 1 – 0 Karlsruhe Brainstormers (Germany)

FinalFC Portugal 1 – 0 Karlsruhe Brainstormers (Germany)



Coordination Methodologies Results: RoboCup 2001 - Seattle

First Stage – Group AFC Portugal 29 – 0 11Monkeys (Japan) FC Portugal 9 – 0 TUT-Grove (Japan) FC Portugal 32 – 0 RMIT Goannas (Australia) FC Portugal 8 – 0 Robolog (Germany)

Second Stage – Group CFC Portugal 5 – 0 Helli Respina (Iran)FC Portugal 16 – 0 UtUtd (Iran)FC Portugal 4 – 0 FC Tripletta (Japan)FC Portugal 13 – 0 AT Humboldt (Germany) FC Portugal 22 – 0 ATTUnited (USA)

Final StageFC Portugal 8 – 0 YowAI (Japan)FC Portugal 4 – 1 UvaTrilearn (Netherlands)FC Portugal 0 – 3 Tsinghuaeolos (China)FC Portugal 0 – 1 Karlsruhe Brainstormers (Germany)



ResultsFC Portugal -TsinghuAeolus (China) 5:0 FC Portugal - AllemaniACs3D (Germany) 5:0 FC Portugal - MRL (Iran) 2:0 FC Portugal - RoboLog3D (Germany) 5:0 FC Portugal - Aria (Iran) 4:0 FC Portugal - MainzRollingBrains (Germany) 8:0 FC Portugal - JU-TsubameGaeshi (Japan) 9:0 FC Portugal - WrightEagle (China) 3:0 FC Portugal - MainzRollingBrains (Germany) 8:0 FC Portugal - AmoiensisNQ (China) 5:0 FC Portugal - Arman (Iran) 1:0

FC Portugal - SEU (China) 0:0 FC Portugal - MRL (Iran) 3:0 FC Portugal - Rezvan (Iran) 1:0 FC Portugal - Brainstormers (Germany) 5:0 FC Portugal - Caspian (Iran) 1:0 FC Portugal - CZU2006 (China) 5:0 FC Portugal - Arman (Iran) 2:0 FC Portugal - Virtual Werder (Germany) 3:0 FC Portugal - Aria (Iran) 1:0

Final

FC Portugal - WrightEagle (China) 1:0

Coordination Methodologies Results: RoboCup 2006 – Bremen



Presentation OutlineIntroductionRoboCup InitiativeRoboCup LeaguesSimulator – Soccer ServerFC Portugal ProjectCoordination MethodologiesLegged Robots

AIBO Robotic PlatformMovement, Sensors and ActuatorsAIBO Programming – OPEN-R SDKSony 4Legged LeagueImage Processing and AnalysisFast Locomotion

Research Areas and Conclusions



AIBO Robotic Platform ERS110

Stereo MicrophoneInfrared SensorsProcessor MIPSCCD Color CameraLegs with 3 freedom degreesAcceleration and vibration sensors



AIBO Robotic Platform - ERS210A



Movement – Head and Tail



Movement - LegsIntroduction | RoboCup | Rob.Leagues | Simulator | FCPortugal | Coordination Meth. | Legged Robots | Conclusions


Actuators6 Leds (Face)

4 Red2 Green

Mode LedTail Led (2 colors)Speaker

Low Quality (8 KHz)8 Bits linear PCM1 Channel (mono-aural)

ERS7: Good Speaker!



Sensors – Head

Touch SensorDistance (10 - 90 cm) ERS7: 3 Distance Sensors



Sensors – Touch and Sound

BackChin4 Touch - LegsStereo Microphone

Sampling Freq: 16KHz16 bits linear PCM2 channels (stereo)Omni directional or Directional



Sensors – Color Camera

Color (YUV)176(H)x144(V) pixels25 FPS Lens

F 2.0, f=2.18mmmVision angle:

(H:57,6 x V: 47,8 graus)Shutter Speed, White Balance e Ganho



Sensors – BodyAcceleration (ax, ay e az):

-2.0G a +2.0GVibration and TemperatureClockSlots:

PC Card (Wireless)Memory Stick: Only Open-R!Battery: Lithium (7.4V / 2300mAh)



MechanicsIntroduction | RoboCup | Rob.Leagues | Simulator | FCPortugal | Coordination Meth. | Legged Robots | Conclusions


Mechanics

Be Careful with motorsMotor Overload: ShutdownDependencies between J1, J2 and J3



MechanicsIntroduction | RoboCup | Rob.Leagues | Simulator | FCPortugal | Coordination Meth. | Legged Robots | Conclusions


AIBO – ERS7

Faster ProcessorRobustCamera (352x288)3 Proximity SensorsSpeaker (16 KHz)Stronger MotorsWirelessAIBO Mind 2Charger



Examples and Applications

Obedience: Voice commandsDancesBall and Object localizationDialogue with humansDistance controlRobotic soccer



Examples and ApplicationsAIBOWare

AIBOWare: Only Sony MS

Programs:AiboLifeHello AIBOParty MascotSpringAibo Mind 2

Applications:Disco AIBOObey CatBender AIBOAIBO and CostelloScooby AIBOCartman AIBOBrainBo



AIBO ProgrammingRCode Programming

Simple language



AIBO ProgrammingOPEN-R SDK - Introduction

OPEN-R: Standard interface

OPEN-R SDK (OPEN-R Software Development Kit)gcc (C++)Open sourceHardware and Software modular architectureWireless

Aperios OSMemory Sticks (wireless?)OPEN-R Objects:

.BINMessage communication



AIBO ProgrammingOPEN-R SDK















AIBO ProgrammingOPEN-R SDK – System Objects

OVirtualRobotComm.SensorSensors: OSensorFrameVectorDataCamera: OFbkImageVectorDataSound: OSoundVectorData

OVirtualRobotComm.Effectoractuators: OCommandVectorDataSpeaker: OSoundVectorData















AIBO ProgrammingComparison RCode / OPEN-R

Software Sony: AIBOLifeEmotions, instincts, learning, growing, voice and object recognition, etc.

Simple Programs: R-CODEVoice commands DancesBall simple interactions

Serious Programming: OPEN-R SDKProgram all



Sony Legged League



Sony Legged League



Sony Legged League

Vídeos



Sony Legged League



Sony Legged LeagueSource Code:

German TeamrUNSWiftCMUPackUTS UnleashedNubotsAsuraAraiboBaby Tigers…



Color SegmentationColor Calibration:

Table associating YUV (or RGB) to each color to detectColor Segmentation:

Table Lookup for each pixel in the imageNoise filtering: Temporal and spatial

Run-length encodingBlob determination:

For each relevant colorSegment mergingComputationally expensive

Blob Analysis:Object identificationObject characteristics (direction, distance, size)



Sony Legged League - Vision

Color Segmentation




Image Analysis:DistanceDirection

3d -> World




Blob Formation



Sony Legged League - VisionIntroduction | RoboCup | Rob.Leagues | Simulator | FCPortugal | Coordination Meth. | Legged Robots | Conclusions






AIBO Fast LocomotionIntroduction | RoboCup | Rob.Leagues | Simulator | FCPortugal | Coordination Meth. | Legged Robots | Conclusions


Legged Conclusions

Legged LeagueVision is quite difficultLocalization easier than in other leaguesLocomotion and navigation very complexWireless CommunicationStrategyChallenges



Presentation OutlineIntroductionRoboCup InitiativeRoboCup LeaguesSimulator – Soccer ServerFC Portugal ProjectCoordination MethodologiesLegged RobotsResearch Areas and Conclusions

RoboCup Research TopicsApplications of RoboCup ResearchAdditional InformationConclusions



Research AreasRoboCup Research Topics

Coordination of Teams of Homogeneous / Heterogeneous Agents in Adversarial Environments: SBSP and DPREConcept of Strategy for a Competition against other Team with Opposite Goals - STRATEGYCommunication in MAS - ADVCOMIntelligent Perception – SLMOptimization Techniques – Optimization Kick, Smart DribbleLearning and Opponent Modeling in Adversarial Environments Soccer – Individual Decision



Research AreasApplications of Research

Real/Simulated Sport CompetitionsRoboCup RescueWar ScenariosMine Clearance and Land ExplorationControl of Hospital RobotsPublic Transports CoordinationSatellite Control and Nuclear Weapon ManagementCleanup of Radioactive and Toxic ContaminationImplementation of AI Opponents for Simulation Games



Research AreasAdditional Information

Web Pages:RoboCup Federation - www.robocup.orgLuis Paulo Reis – www.fe.up.pt/~lpreisFC Portugal - www.ieeta.pt/robocup ; www.fe.up.pt/~rescue5DPO - www.fe.up.pt/~robosocIsocRob - http://socrob.isr.ist.utl.ptMinho Team - www.robotica.dei.uminho.ptISEPorto Team - www.isep.ipp.pt/iseporto.html

Contacts:Luís Paulo Reis ([email protected]) A. P. Moreira ([email protected]) F. Ribeiro ([email protected]) José Almeida ([email protected])

Nuno Lau ([email protected]) Paulo Costa ([email protected]) Pedro Lima ([email protected])José L. Azevedo ([email protected])



ConclusionsUniversity of Porto - New Projects

Common framework for cooperative roboticsSmall and Middle-Size league (5DPO, Cambada) - Mixed Teams RoboCup Rescue and other domainsReal Soccer Intelligent Game Analysis SystemPresent Simulators:

General Strategy for a CompetitionGeneral Coaching – Coach UnilangOpponent Modeling and Game AnalysisLearning high-level skillsHeterogeneous PlayersExtension of SBSP, DPRE, ADVCOM and SLM3D Visualizer with Camera Intelligent control

Simulator Extensions and Changes – 3D Simulator



ConclusionsRoboCup in the Future

In the Future? Is it really possible, by 2050, to build a team of fully autonomous humanoid robots that may win against the human world champion team in soccer?

Major Difficulties?Robotic Sensors?Robotic Actuators?Artificial Brains?Social Behavior?

RoboCup is not only about Competition!Universities and Companies!Research applied in other areas


definition of high-level coordination techniques for...

Documents