desenvolvimento de aplicações de robótica inteligente em

28 Sept. 2007

1

Prof. Ph.D. Prof. Ph.D. Fernando Fernando OsórioOsório -- Applied Computing / Applied Computing / UnisinosUnisinosProfaProfa . Ph.D. . Ph.D. SoraiaSoraia MusseMusse -- Computing Science / PUCComputing Science / PUC --RSRSProf. Prof. M.ScM.Sc. . FarleiFarlei HeinenHeinen -- Computing Eng. / Computing Eng. / UnisinosUnisinosM.ScM.Sc. Milton Roberto . Milton Roberto HeinenHeinen -- Ph.D. Student at UFRGSPh.D. Student at UFRGSProf. Prof. Ph.DPh.D Christian Christian KelberKelber -- Electrical Eng. / Electrical Eng. / UnisinosUnisinosGustavo Gustavo PessinPessin -- M.ScM.Sc. at . at UnisinosUnisinos

GRAPHIT - Computer Graphics and Vision Group (Unisinos/PUC-R S)GPVA - Autonomous Vehicles Research Group (Unisinos)GIA - Artificial Intelligence Research Group (Unisinos)RBV - Rede Brasileira de Visualização [FINEP/Brazil]

IEEE / DVP - Distinguished Visitors Program Latin America

Applied Computing Applied Computing PostPost--grad. Programgrad. Program-- PIPCAPIPCA

UNISINOS University UNISINOS University -- BrazilBrazil

Desenvolvimento de Aplicações de Desenvolvimento de Aplicações de Robótica Inteligente em Robótica Inteligente em

Ambientes de Realidade VirtualAmbientes de Realidade Virtual

Research group:

Fernando Osório, Ph.D. / IEEE CS DVP

I JAC - Workshop de I.A. / UNISC 2007 - Palestra Convidada

28 Sept. 2007

2

IEEE / DVP - Distinguished Visitors Program Latin America

Prof. Fernando Santos Osório - IEEE MemberApplied Computing Research Post-Graduation Program - PIPCAUNISINOS University - Brazil (Porto Alegre - Southern Region)IEEE Computer Society DVP Program

Jesuit UniversityWith: - 30.000 Students- 900 Professors- 16 PPGs (post-grad programs)

PPG CAPES PPG CAPES NotaNota 44

28 Sept. 2007

3

Presentation Topics

Agenda:

1. Introduction: VR - Hierarchy of Models

2. VR and Simulation

Geometry, Physics, Behaviour, Knowledge and Cognition

3. Physics Simulation Tools

Opensteer, ODE, PhysX, Deformable/Dynamic

4. Intelligent Behaviour

Agents: Perception, Action, Behaviour

Autonomous Robots and Agents - Control

Multi-Agents Systems - Knowledge

5. Applications: Autonomous Robots VR Simulation Tools

6. Conclusions and New TrendsVídeo Demo Web/Java

28 Sept. 2007

4

Introduction VR - Virtual Reality

[Paul Milgram et al. 95]

From REAL to VIRTUAL3D + Immersion + Interaction

Virtual Reality

28 Sept. 2007

5

VISUALIZING 3D & VIRTUAL ENVIRONMENTS

Virtual RealityVirtual Reality...

* VRML - 3D Worlds (Geometry)

* QTVR - Panorama 3D (Images)

Introduction VR - Virtual Reality

\\VRMLVRML

Virtual Reality

28 Sept. 2007

62424

IRISA / INRIA IRISA / INRIA -- FranceFrancehttp://www.irisa.fr/lagadic/demo/demohttp://www.irisa.fr/lagadic/demo/demo--ar3/demoar3/demo--ar3ar3--eng.htmleng.html

Real World Integrated with Virtual Objects

Augmented Reality

28 Sept. 2007

7

Virtual Reality

Virtual RealityVirtual Reality...

* 3D Virtual Environment

* Interaction => Virtual Reality Devices

* Immersion => Virtual Reality Devices

* Realism => Graphical Realism (photo-realism)

Movements

Interaction Real x Virtual

"Physics Realism"

VISUALIZING 3D & VIRTUAL ENVIRONMENTS

How to do it?

28 Sept. 2007

8

1. Introduction

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

28 Sept. 2007

9

1. Introduction

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

28 Sept. 2007

10

1. Introduction

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

28 Sept. 2007

11

1. Introduction

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

28 Sept. 2007

12

1. Introduction

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

28 Sept. 2007

13

1. Introduction

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

PhantomPhantom

HaptionHaption

CyberForceCyberForce

OmegaOmega

28 Sept. 2007

14

1. Introduction

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

[Ari Chapiro - Dance]

[CromosLab]

Agents

28 Sept. 2007

15

1. Introduction

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

28 Sept. 2007

16

1. Introduction

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

28 Sept. 2007

17

1. Introduction

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

Knowledge

AutonomousBehaviour

28 Sept. 2007

18

2. VR and Simulation

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

Virtual World

xx

Real World

28 Sept. 2007

19

Presentation Topics

Agenda:

1. Introduction: VR - Hierarchy of Models

2. VR and Simulation

Geometry, Physics, Behaviour, Knowledge and Cognition

3. Physics Simulation Tools

Opensteer, ODE, PhysX, Deformable/Dynamic

4. Intelligent Behaviour

Agents: Perception, Action, Behaviour

Autonomous Robots and Agents - Control

Multi-Agents Systems - Knowledge

5. Autonomous Robots VR Simulation Tools

6. Conclusions and New TrendsVídeo Demo Web/Java

28 Sept. 2007

20

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

Virtual World

xx

Real World

Increasing Reality in VR Applications: Physical and Behavioral Simulation

2. VR and Simulation

28 Sept. 2007

21

1. Introduction

RealisticVR

Increasing Reality in VR Applications: Physical and Behavioral Simulation

Virtual World

xx

Real World

Real WorldReal WorldSimulationSimulation

28 Sept. 2007

22

Simulation Tools:

* ODE - Open Dynamics Engine

* OpenSteer

* PhysX AGEIA

* Deformable Objects and Fluids:- Finite Elements Methods- Spring-Mass Systems- CFD (Computational Fluid Dynamics)- Level Set Methods

VR Simulation: Some important questions...

3. Physics Simulation Tools

28 Sept. 2007

23

Simulation Tools:

* ODE - Open Dynamics Engine

* OpenSteer

* PhysX AGEIA

* Deformable Objects and Fluids:- Finite Elements Methods- Spring-Mass Systems- CFD (Computational Fluid Dynamics)- Level Set Methods

VR Simulation: Some important questions...

3. Physics Simulation Tools

Physics:

Physical structure: resistance, mass, density, elasticity;Position and orientation in the 3D space; Kinematics and Dynamics;Linear and angular velocities; Motion (w/ forces and torques), trajectories; Acceleration, deceleration;Attraction and repulsion;Gravity, friction, inertia; Kinetic and potential energy; Laws of energy conservation, linear and angular momentum;Collisions and reaction to collisions;Steering models (wheeled cars, aircrafts, projectiles, boats and ships); Articulated Rigid Bodies Simulation (skeleton, robotic arm);Dynamic Simulation of Deformable Objects: elastic objects; Fluid simulation and Particle Systems (fire, smoke, clouds and liquids).

28 Sept. 2007

24

* Deformable Objects and Fluids

- Finite Elements Methods

- Spring-Mass Systems

- CFD (Computational Fluid Dynamics)

- Level Set Methods

3. Physics Simulation Tools

28 Sept. 2007

25

* ODE - Open Dynamics Engine

Simulation of Articulated Rigid Body Dynamics

Open Source Library (C/C++ API)

Used with OSG, Ogre3D, CrystalSpace, ...

3. Physics Simulation Tools

Slider jointSlider jointHinge jointHinge jointBall and socket jointBall and socket joint

Universal jointUniversal joint

28 Sept. 2007

26

* ODE - Open Dynamics Engine

Simulation of Articulated Rigid Body Dynamics

3. Physics Simulation Tools

Physics Simulation:

- Gravity, friction, acceleration, deceleration; - Generation of motion: applying forces and torques (motors);- Collision avoidance and treatment (reaction, object bounce);- Kinematics models and rigid body dynamics simulation; - Different types of joints with actuators (motors)

Webbots uses ODE [Cyberbotics]

28 Sept. 2007

27

* ODE - Open Dynamics Engine

3. Physics Simulation Tools

28 Sept. 2007

28

* ODE - Open Dynamics Engine

3. Physics Simulation Tools

28 Sept. 2007

29

* ODE - Open Dynamics Engine

3. Physics Simulation Tools

WebotsCyberbotics

28 Sept. 2007

30

* ODE - Open Dynamics Engine

3. Physics Simulation Tools

Juice[Nate W.]

28 Sept. 2007

31

3. Physics Simulation Tools

Simulation Tools:

* ODE - Open Dynamics Engine

* OpenSteer

* PhysX AGEIA

* Deformable Objects and Fluids:- Finite Elements Methods- Spring-Mass Systems- CFD (Computational Fluid Dynamics)- Level Set Methods

VR Simulation: Some important questions...REAL TIME SIMULATIONREAL TIME SIMULATION

28 Sept. 2007

32

Intelligent Agents:

Agents: Perception, ActionAgent Behaviours

Control ArchitecturesAutonomous Agents Multi-Agents SystemsKnowledge / Reasoning

4. Intelligent Behaviour

Comportamento

28 Sept. 2007

33

Intelligent Agents:

Agents: Perception, ActionAgent Behaviours

Control ArchitecturesAutonomous Agents Multi-Agents SystemsKnowledge Integration

4. Intelligent Behaviour

Reactive-Deliberative Architecture

Control Architectures: Reactive, Deliberative, Hierarchical, Hybrid

Hybrid Architecture

Architecture BDI (Beliefs-Desires-Intentions)

F. Osório et al.[Virtual Concept 2005]

28 Sept. 2007

34

4. Intelligent Behaviour

RobotSensors Actuators

PositioningEstimator

(Monte Carlo)

Environment Representation Maps

Polygonal

Grid

Topologicaland

Semantic

SharedMemory

Control Layers

Path Planning

Behaviors

Sequencer

Control Modules

Robust Hybrid Control COHBRA / HyCAR

SimRob3DUnisinos

COHBRACOHBRA: : Controle Híbrido Controle Híbrido de Robôs Autônomosde Robôs Autônomos

HyCARHyCAR -- Hybrid Control Hybrid Control for Autonomous Robotsfor Autonomous Robots

Published at HIS2002Hybrid Intelligent Systems 2002, Chile

28 Sept. 2007

35

Intelligent Agents:

Agents: Perception, ActionAgent Behaviours

Control ArchitecturesAutonomous Agents Multi-Agents SystemsKnowledge / Reasoning

4. Intelligent Behaviour

Robotic

Teams, Squads, Swarms

Artificial Intelligence Tools

28 Sept. 2007

36

Presentation Topics

Agenda:

1. Introduction: VR - Hierarchy of Models

2. VR and Simulation

Geometry, Physics, Behaviour, Knowledge and Cognition

3. Physics Simulation Tools

Opensteer, ODE, PhysX, Deformable/Dynamic

4. Intelligent Behaviour

Agents: Perception, Action, Behaviour

Autonomous Robots and Agents - Control

Multi-Agents Systems - Knowledge

5. Applications: Autonomous Robots VR Simulation Tools

6. Conclusions and New Trends

28 Sept. 2007

37

5. Applications: VR Simulation Tools

Applications @ Unisinos

1. Autonomous Robots in VR Environments

SimRob3D - Mobile Robots Simulator

SEVA 3D - Autonomous Vehicle Parking

LEGGEN - Legged (articulated) Robots Simulator

Robombeiros - Multi-Robots Fire Fighting

28 Sept. 2007

38

5. Applications: VR Simulation Tools

SimRob3D

- Our Simulation Tools: SimRob2D (Khepera)SimRob3D Seva2D

Autonomous Robots in VR Environments

28 Sept. 2007

39

5. Applications: VR Simulation Tools

SimRob3D Simulator

> Sensors: Infrared, Sonar, Bumper

> Actuators / Kinematics: Differential, Ackerman

> Realistic Simulation Model: 3D World + noise / error (imprecise sensors and actuators)

Autonomous Robots in VR Environments

28 Sept. 2007

40

5. Applications: VR Simulation Tools

SimRob3D

Simulator

Autonomous Robots in VR Environments

28 Sept. 2007

41

5. Applications: VR Simulation Tools

SEVA 3D - "Sistema de Estacionamento de Veículos Autônomos"

Sources of Inspiration:

- Baja Buggy remotely controlled by a cell phoneC. Kelber - UNISINOS, Brazil

Published at:IEEE WCCIIJCNN 2006

Autonomous Robots in VR Environments

28 Sept. 2007

42

5. Applications: VR Simulation Tools

SEVA 3D

28 Sept. 2007

43

SEVA 3D Simulator

> Vehicle Simulation xx Vehicle Control

SimRob3D SEVA3D

Commands

Sensors

Control:SEVA3D-A (FSA)SEVA3D-N (Neural)

ControlSimulation

MotorActions

SensorialInformation

Kinematics:Estimation ofPosition and Orientation

Perception:Sensor Simulation

3DWorld

RobotModel

Visualization

SimRob3D SEVA3D

Commands

Sensors

Control:SEVA3D-A (FSA)SEVA3D-N (Neural)

ControlSimulation

MotorActions

SensorialInformation

Kinematics:Estimation ofPosition and Orientation

Perception:Sensor Simulation

3DWorld

RobotModel

Visualization

Autonomous Robots in VR Environments

5. Applications: SEVA 3D

28 Sept. 2007

44

SEVA: FSA - Finite State Automaton

Inputs:

- Sonar Sensors: Stochastic ray-casting / 3D cone)

Outputs:

- Steering Wheel Angle- Gas pedal (car speed + direction: fwd, back)

States:

Autonomous Robots in VR Environments

5. Applications: SEVA 3D

28 Sept. 2007

45

SEVA: NEURAL FSA - Learning the FSA...

Si(t) + Si(t-1) + ...Temporal Window

Adapted Jordan-Net using RProp Learning

5. Applications: SEVA 3D

28 Sept. 2007

46

SEVA3D - Autonomous Vehicle Parking Simulator

3D

SensorsActuatorsKinematicsFSA CtrlANN Ctrl

5. Applications: SEVA 3D

28 Sept. 2007

47

5. Applications: VR Simulation Tools

Applications @ Unisinos

1. Autonomous Robots in VR Environments

SimRob3D - Mobile Robots Simulator

SEVA 3D - Autonomous Vehicle Parking

LEGGEN - Legged (articulated) Robots Simulator

Robombeiros - Multi-Robots Fire Fighting

28 Sept. 2007

48

5. Applications: VR Simulation Tools

Legged Robots Evolution and Walking Control

Sources of Inspiration:

Autonomous Robots in VR Environments

[EPFL]

28 Sept. 2007

49

5. Applications: VR Simulation Tools

Legged Robots Evolution and Walking Control

Sources of Inspiration: LEGGEN - Published at: IEEE WCCI CEC 2006SBIA 2006

33

Autonomous Robots in VR Environments

28 Sept. 2007

50

28 Sept. 2007

51

5. Applications: VR Simulation Tools

LEGGEN - Legged Robots Evolution and Walking ControlAutonomous Robots in VR Environments

Simulation of Robots: 3D Realistic Virtual Environments- Sensors: infrared, sonar, bumpers, gyro (accelerometers),

GPS, compass, light and vision sensors, etc. - Actuators: legs and arms with angular motors (joints) - Physics: collision, kinematics, rigid body dynamics

Simulation of Legged Autonomous Robots:- Robot ControlArchitectures Implementation

Genetic Evolved Control of Articulated Robots (w/legs)

28 Sept. 2007

52

5. Applications: VR Simulation Tools

LEGGEN - Legged Robots Evolution and Walking ControlAutonomous Robots in VR Environments

Simulation of 3D Realistic Virtual Legged Robots LEGGEN Simulator - Tools:

1. OSG- Open Scene Graph (OpenGL + Extensions)[ http://www.openscenegraph.org/ ]

2. ODE - Open Dynamics EngineRigid Body Physics Simulation(gravity, inertia, friction, collision, joints, etc)[ http://www.ode.org/ ]

3. GALib - Genetic Algorithms Simulation[ http://www.lancet.mit.edu/ga/ ]

4. Robot Control FSM: Finite State Machine = Sense + Act

28 Sept. 2007

53

5. Applications: VR Simulation Tools

LEGGEN - Legged Robots Evolution and Walking ControlAutonomous Robots in VR Environments

Simulation main goals:

- Evaluate different Robot Models(hardware configurations)IEEE WCCI / CEC 2006 - Vancouver, Canadá

- Evaluate different Fitness FunctionsIBERAMIA / SBIA - Ribeirão Preto, SP Robot Models

Boston Dynamics

Evaluate different robot models in order to select a better hardware configuration

28 Sept. 2007

54

LEGGEN SIMULATOR

Simulation Results:

28 Sept. 2007

55

LEGGEN SIMULATOR

Simulation RESULTS:

Tetrapod Video - Distance, Gyro

28 Sept. 2007

56

LEGGEN SIMULATOR

Simulation RESULTS: Tetrapod Video - "bloopers"

28 Sept. 2007

57

5. Applications: VR Simulation Tools

Applications @ Unisinos

1. Autonomous Robots in VR Environments

SimRob3D - Mobile Robots Simulator

SEVA 3D - Autonomous Vehicle Parking

LEGGEN - Legged (articulated) Robots Simulator

Robombeiros - Fire Fighting

28 Sept. 2007

58

Robombeiros - Fire Fighting VR Simulation

Virtual Simulation Environment:* 2D and 3D Simulation* Simulation of fire propagation* Autonomous fire-fighting team* Define: Strategy, Mission, Execution

Fire PropagationSimulation:

- Direction and Speedof wind

- Vegetation type andcoverture density

(speed of propagation)- Terrain

Figure: 2D Simulation using SDL library => http://pessin.googlepages.com/robombeiros

Published at SVR 2007 (Symposium on Virtual and Augmented Reality) [G. Pessin, F. Osório, S. Musse, V. Nonnenmacher, S. Ferreira]

28 Sept. 2007

59

Robombeiros - Fire Fighting VR Simulation

Virtual Simulation Environment:

http://pessin.googlepages.com/robombeirosPublished at SVR 2007 (Symposium on Virtual and Augmented Reality) [G. Pessin, F. Osório, S. Musse, V. Nonnenmacher, S. Ferreira]

3D Visualization:- Vegetation, Fire- Autonomous mobile Robots- Stereo 3D - Tools: OSG, ODE, Demeter

28 Sept. 2007

60

Robombeiros - Fire Fighting VR Simulation

Virtual Simulation Environment:

3D Simulation:- Fire propagation- Physics- Robot Control

http://pessin.googlepages.com/robombeiros

28 Sept. 2007

61

Presentation Topics

Agenda:

1. Introduction: VR - Hierarchy of Models

2. VR and Simulation

Geometry, Physics, Behaviour, Knowledge and Cognition

3. Physics Simulation Tools

Opensteer, ODE, PhysX, Deformable/Dynamic

4. Intelligent Behaviour

Agents: Perception, Action, Behaviour

Autonomous Robots and Agents - Control

Multi-Agents Systems - Knowledge

5. Applications: Autonomous Robots VR Simulation Tools

6. Conclusions and New Trends

28 Sept. 2007

62

New Trends

Claytronics - Nanotechhttp://www.cs.cmu.edu/~claytronics/[Published at IROS2006]

IEEE Intelligent Robot and Systems Conference

28 Sept. 2007

63

Conclusions and New Trends

Virtual Reality Environments:

Geometric + Kinematic + Physical + Behavioural + Cognitive=

Realistic VR Environments

New Trends: VR + PhysicsArtificial IntelligenceAR - Augmented RealityHaptic Interfaces

Autonomy IVREIntelligent Virtual Reality Environments

Simulation VR PBSimVR Physical and Behavioral Simulation

More Real VR++

28 Sept. 2007

64

CONTACT INFORMATION

UNISINOS University UNISINOS University -- BrazilBrazil

Applied Computing Research PostApplied Computing Research Post --grad Programgrad Program -- PIPCAPIPCAAutonomous Vehicles Research GroupAutonomous Vehicles Research Group -- GPVAGPVA

WebWeb: : GoogleGoogle -- veiculosveiculos autonomosautonomos

GPVA GPVA WebWeb PagePage: : http://www.eletrica.unisinos.br/~autonomhttp://www.eletrica.unisinos.br/~autonom

ContactContact -- WebWeb PagePage: : http://inf.unisinos.br/~osorio/http://inf.unisinos.br/~osorio/

This conference This conference -- Web Page: Web Page: http://inf.unisinos.br/~osorio/palestras/cerma07.ht mlhttp://inf.unisinos.br/~osorio/palestras/cerma07.ht ml

Contact: Prof. Dr. Fernando OsórioE-Mail: fosorio@unisinos.br