reference of the annotated version: bossé É., jousselme a ... · artificial intelligence (e.g....

90
Éloi Bossé Anne-Laure Jousselme DRDC-Valcartier Decision Support Systems for C2 Situation Analysis and Monitoring group April 2011 The application of belief functions in a situation analysis process Reference of the annotated version: Bossé É., Jousselme A-L., Maupin P., ‘The application of belief functions in a situation analysis process’, DRDC TN-2011-xx, DRDC Valcartier, 2011.

Upload: others

Post on 11-Oct-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

Éloi BosséAnne-Laure JousselmeDRDC-Valcartier Decision Support Systems for C2Situation Analysis and Monitoring group

April 2011

The application of belief functions in a situation analysis process

Reference of the annotated version: Bossé É., Jousselme A-L., Maupin P., ‘The application of belief functions in a situation analysis process’, DRDC TN-2011-xx, DRDC Valcartier, 2011.

Page 2: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

1

Defence R&D Canada Mission

To ensure the Canadian Forces are technologically prepared and operationally relevant.

• Advise on Science & Technology

• Conduct defence research, development and analysis

• Assess technology trends, threats, and opportunities

• Engage industrial, academic and international partners in the

transition of technology

• Conduct S&T projects for non-DND clients

Page 3: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

2

Defence Research, Development and Analysis Centres

DRDC SuffieldCCMAT

CTT Military Engineering

Autonomous Intelligent SystemsChemical & Biological Defence

DRDC OttawaRadiological Analysis & DefenceNetwork Information Operations

Electronic Warfare (Radar, Comm & Nav)Future Forces Synthetic Environment

Radar Applications & Space TechRadar Systems

CORAQuantitative & Strategic Analysis

Operational EfficiencyManagement Quality

S&T Intelligence

CSSCRTIPSTP

Security Project Management

COS S&TR&D Programs

Corporate Services

DRDC ValcartierCommand & Control

Intelligence & InformationSystem of systems

Tactical Surveillance & ReconnaissanceSpectral and Geospatial Exploitation

Electro-optical WarfareEnergetic MaterialsWeapons Effects

Precision WeaponsiDV

METC (ADM MAT)

DRDC AtlanticUnderwater Sensing

Maritime Information and Combat SystemsAir Vehicles

Warship PerformanceEmerging Materials

Dockyard Lab (Pacific)

DRDC TorontoAerospace & Undersea Medical Science CentreHuman Effectiveness Experimentation Centre

Adversarial IntentCollaborative Performance & Learning

Human Systems IntegrationIndividual Readiness

Page 4: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

3

Defence Research, Development and Analysis Centres

CF/DND S&TStakeholders Other Gov’t

DepartmentsAlliedR&D

CanadianIndustry

CanadianUniversities

Page 5: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

4

Defence R&D Canada - Valcartier

Page 6: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

5

Areas of Expertise

C4ISRForce protection & Weapon systems

Page 7: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

6

C2DSS section

Modelisation & Simulation(e.g. Laboratoire d’Integration de Systemes)

Decision Theories(e.g. multi-criteria, Bayesian, games, utility)

Software & Cognitive Engineering(e.g. cognitive work/task analysis)

Artificial Intelligence(e.g. multi-agent, expert systems,

evolutionary computing)

Information Fusion(e.g. evidential, Bayesian, possibilistic,

knowledge-based systems)

4 groups covering High-Level Information Fusion for C2

Decision Support System Engineering

Future C2 Concepts & Structures

Collaborative Planning

& Logistics

Situation Analysis

& Monitoring

Trend:Consider the social

aspect

Page 8: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

7

Complex Conflict Spectrum

Globalization of Science and Technology

Asymmetric Threats

Common Defence and Security Agenda

The New Defence and Security Context

Page 9: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

8

The Future Security Environment

– The challenges of the 21st century include a variety of humanitarian disasters (earthquakes, floods, tsunamis), failed states, instability, global terrorism, intractable conflicts, pandemics, economic crises, and poverty among others.

• These problems are not one dimensional, but rather involve the consideration of effects in multiple, inter-related dimensions. These dimensions include social, political, and economic effects.

– These challenges are beyond the ability of any single actor or even a small set of very capable actors (e.g. CFs).

– Responses to these challenges, if they are to have a chance of success, must involve a large, heterogeneous collection of entities working together.

– The 21st century mission challenges described above are referred to as Complex Endeavours

Page 10: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

9 Director General Joint Force Development / Directeur Général Développement de la Force interarmées

Non-Linear

Simultaneous, non-linearoperations throughout

the battlespace

Non-Contiguous

Information Superiority

Ubiquitous Connectivity

Integrated BattlespaceSea floor to Space

Link

Where will Canada fit?

In a joint and combined, multi-purpose, combat capable capacity.

Key to participation is

interoperability.

Canada’s Allies are Transforming…Trend:

Social aspect

Page 11: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

10

Canadian Forces (CF) Mission-Critical Outcomes

1. Trusted situational awareness, intent prediction and decision making for achieving operational superiority;

2. Robust operational command and control in Canada and abroad;3. Seamless interoperability with other government departments, other

Canadian partners and allied forces in a complex environment;4. Agile, tailored force to deploy and operate in complex environments;5. Full-spectrum protection including from weapons of mass destruction;6. Sustainability, affordability, supportability of operations, assets and

people;7. Creation of asymmetric advantage by defeating terrorist groups and

tactics; and8. Defence policy and force development informed and enabled by S&T

developments.

S&T so what for C2

Page 12: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

11

Outline

1. Information systems for Situation Analysis support

– What is Situation Analysis (SA)?

– Decision Support Systems

– High-Level Information Fusion as a key enabler for SA

– Information processing: Features and challenges

2. Theoretical tools for situation analysis support

3. DRDC projects

4. Conclusions: Some critical research issues

Page 13: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

12

Information systems for situation analysis support

Our mission: To support understanding and acting in complex environments.

Our problem: Define with precision and intelligibility concepts to be understood by both a human and a machine.

Mastering knowledge to efficient action in a complex world

Page 14: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

13

WHAT IS SITUATION ANALYSIS?

Page 15: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

14

DECIDEACT

OBSERVE ORIENT

SITUATIONANALYSIS

DECISIONMAKING

Page 16: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

15

Context:

• Non-cooperative environment

• Constraints: Communication,Time, Energy, …

• Background knowledge (ontology and epistemology)

Rules of world

Entities, …

Distributed systems:

• Socio-technical systems

Set of heterogeneous agents human and artificial:

o Capturing and exchanginginformation

o Having their own view, goal, language, strategy, beliefs, knowledge, intent, interpretation, decisions, …

o Making decision and acting.

Situation

Page 17: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

16

Decision support:

• Decision makers (humans)

Decision rights

• Has a mission

• Under stress, subject to fear, panic, uncertainty,…

• Needs to have a clear and global understanding of the situation

→Reach a state of (« shared ») situation awareness.

Analysis

Page 18: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

17

Situation analysis and situation awareness

« Situation analysis is a process, the examination of a situation,its elements, and their relations, to provide and maintain a state ofsituation awareness for the decision maker » (Roy, 2001)

Situation analysis(A process)

Situation awareness(A state of mind)

Page 19: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

18

Situation awareness (Endsley’s model, 1995, 2000)

InformationProcessingMechanisms

LongTermMemoryStores Automaticity

Task/System Factors

DecisionState of theenvironment

feedback

Individual Factors

. Abilities. Experience. Training

System Capability. Interface design

. Stress and workload. Complexity, Automation

Goals & Objectives. Preconceptions

(expectations)

Perceptionof Elementsin CurrentSituation

SITUATION AWARENESS

Projectionof Future

Status

Compre-hension

of Current Situation

Performance of actions

Functional model

Page 20: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

19

These aspects should be considered all together

SITUATION AWARENESS

Projection of future status

Comprehension of current situation

Perceptionof elements in

current situation

Uncertainty-based information

Knowledge and belief

Time and non-monotonicity

Main features of situation awareness

Sensing/Measuring Reasoning Dynamics

Page 21: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

20

Multiagent Situation Awareness

SITUATION AWARENESS

Projection of future status

Comprehension of current situation

Perception of elements in current

situation

Agent 2

Agent 3

Agent 4 Agent 5

Environment

Agent 1

Page 22: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

21

Materialization of SAW

• Recognized Air Picture (RAP)

• Common Operating Picture (COP)

• Tactical Picture Compilation (TPC)

• ...

Prototype Virtual Planning Room (ViPR) as 3D planning environment

(DSTO, Adelaide, Australia)

Page 23: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

22

DECISION SUPPORT SYSTEMS

Page 24: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

23

Decision makers at all levels need...

o to rapidly develop situational awareness (e.g., understand how a situation has developed and is expected to develop);

o to rapidly develop shared understandings of the operational environment;

o to plan operations;

o to monitor the situation and the execution of the plans;

o to ensure that each individual worker is productive and concentratedin its assigned roles and tasks;

o to deal with complex crises;

o to deal with multiple, simultaneous crises (e.g., multiple operations monitoring);

o to perform usual routine tasks.

Page 25: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

24

No unified theory has yet been proposed describing how individuals make decisions in naturalistic settings:

o Real-world decisions are made in a variety of ways;

o Situation assessment is a critical element in decision making;

o Decision makers often use mental imagery;

o Understanding the context surrounding the decision process is essential;

o Decision making is dynamic, it does not consist of discrete isolated events or processes;

o Normative models of decision making must derive from an analysis of how decision makers actually function, not how they "ought" to function.

Key points

Page 26: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

25

The ideal DSS is one that:

• Provides the information needed by the human decision maker, as opposed to raw data;

• Can be controlled effortlessly by the human (transparent to the user);

• Complements the cognitive power of the human mind;

• Supports a wide variety of problem solving strategies (from instinctive reactions to knowledge based reasoning).

A computerized system that is intended to interact with and complement a human decision

maker

Decision Support System

Toward a Human-Machine synergy

Page 27: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

26

LIMITATIONS LIMITATIONS

Task

Human Technology Tradeoff

C2

“Task / Human / Technology” triad model

(R. Rousseau – Laval University)

Page 28: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

27

1. Silent/Manual 2. Informative 3. Co-operative 4. Automatic 5. Independent

System

Operator(s)Decisions

System

Operator(s)Decisions

System

Operator(s)Decisions

Support Influence

Influ

ence

System

Operator(s)

Info. Requests/Influence

DecisionsSystem

Operator(s)

DecisionsDecisions

Syne

rgy

Medium MediumHighest

None None

Wor

kD

istri

butio

n

Operator(s)Operator(s) Operator(s) System

System Operator(s)System

System

Rep

rese

ntat

ion

Override

Supp

ort

Human-Technology Tradeoff Spectrum

Page 29: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

28Time

Ope

rato

r D

eman

ds

More sensors

More data

More displays

More platforms

?

CognitiveOverload

Need for a Holistic Perspective on Design

Better OperatorInterface

Automation

Data fusion

Decision Aids

Data driven

User driven

Page 30: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

29

What is Cognitive Systems Engineering? The Cognitive Triad

• Dr. David D. Woods, OSU Cognitive Systems Engineering Lab describes Cognitive Systems Engineering as:

“working at the intersection of the problems imposed by the world, the needs of agents (both human and machine) and the interaction with the various technologies (affordances) to affect the situation”.

Note that each interacts with the other two, for example the user interface must allow the user to control the world as well as control any automation. 

Page 31: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

30

Integration: Open System Architecture e.g. SOA

Communication and Collaboration

Planning and Decision Support

Situation Awareness SupportIncluding Assets Monitoring

Execution Management (coordination and control)

Decision Support Dimensions

Experiments/Demonstration (Domestic Operations Context)

Page 32: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

31

Organizational Issues

o The sponsors, stakeholders and end-users belong to different services;o The end-users might be individuals (of various ranks), communities or groups,

from a variety of parent organizations, fulfilling different roles and responsibilities in different assignments;

o The tasks to be performed are highly specialized and specific to each end-user;o The task-related knowledge is often explicitly embedded in procedures;o The mission related knowledge remains tacit for a significant part;o The business rules are not all formulated;o The organisational culture and values are to be considered.

Page 33: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

32

Understand Business processes

(Joint Command Decision in JIMP)

Design Decision Support System

Design a SOS Integration

Architecture

Decision Support

Requirements

Systems Integration Standards

Metrics for Assessing Decision Improvements (MOEs/MOPs)

Decision Support Solutions

Integrate Human and Organization Sciences

How to Design Decision Support?

Requires multidisciplinary

Standards

Page 34: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

33

Multidisciplinary domain

Situation awareness

Reasoning about

knowledge and uncertainty

Page 35: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

34

Limitations of information systems

o Information overload: There is a significant information overload (e.g., a huge quantity of messages at a high rate of message reception).

o Distributed information:

The required data, information and knowledge and the services, applications, tools and products originate from various stovepipe systems.

There are limitations in timely fusing the information of different natures.

o Information quality:

The information is not provided or accessed in a timely fashion because of the high operational tempo.

Decisions are being made on imperfect information

o Release of information: There are information management constraints because of different security domains.

o Limited tools:

There is a lack of tools to understand how a situation has developed and is expected to develop.

There are limited visualization capabilities.

There are limited decision-aid / planning tools.

Synthesis of study for high-level requirements of IS by DRDC

Page 36: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

35

HIGH-LEVEL INFORMATION FUSION AS A KEY ENABLER FOR SITUATION ANALYSIS

Page 37: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

36

The JDL (Joint Director Laboratory) data fusion model (1998 revised framework)

Measurements

Level 1Object Assessment

Level 2Situation Assessment

Level 3Impact Assessment

Level 4Process Refinement

(Resource Management)

Level 0Sub-Object Assessment

Signal/Features

Objects

Situations

Situations/Plans

Plans

Resources

Situations

Plans

Situations

Objects

Signal/Features

Steinberg, A. N., Bowman, C., and White, F. (1998). Revisions to the JDL Data Fusion Model, Proceedings of the 3rd NATO/IRIS Conference, Québec, Canada, 1998.

Page 38: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

37

The JDL (Joint Director Laboratory) data fusion model (1998 revised framework)

Measurements

Level 1Object Assessment

Level 2Situation Assessment

Level 3Impact Assessment

Level 4Process Refinement

(Resource Management)

Level 0Sub-Object Assessment

Signal/Features

Objects

Situations

Situations/Plans

Plans

Resources

Situations

Plans

Situations

Objects

Signal/Features

Steinberg, A. N., Bowman, C., and White, F. (1998). Revisions to the JDL Data Fusion Model, Proceedings of the 3rd NATO/IRIS Conference, Québec, Canada, 1998.

Estimation and prediction of:

signal/object observable states on the basis of pixel/signal level data association and characterization

Page 39: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

38

The JDL (Joint Director Laboratory) data fusion model (1998 revised framework)

Measurements

Level 1Object Assessment

Level 2Situation Assessment

Level 3Impact Assessment

Level 4Process Refinement

(Resource Management)

Level 0Sub-Object Assessment

Signal/Features

Objects

Situations

Situations/Plans

Plans

Resources

Situations

Plans

Situations

Objects

Signal/Features

Steinberg, A. N., Bowman, C., and White, F. (1998). Revisions to the JDL Data Fusion Model, Proceedings of the 3rd NATO/IRIS Conference, Québec, Canada, 1998.

Estimation and prediction of:

1. entity states on the basis of observation-to-track association;

2. continuous and discrete state estimation.

Page 40: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

39

The JDL (Joint Director Laboratory) data fusion model (1998 revised framework)

Measurements

Level 1Object Assessment

Level 2Situation Assessment

Level 3Impact Assessment

Level 4Process Refinement

(Resource Management)

Level 0Sub-Object Assessment

Signal/Features

Objects

Situations

Situations/Plans

Plans

Resources

Situations

Plans

Situations

Objects

Signal/Features

Steinberg, A. N., Bowman, C., and White, F. (1998). Revisions to the JDL Data Fusion Model, Proceedings of the 3rd NATO/IRIS Conference, Québec, Canada, 1998.

Estimation and prediction of:

1. relations among entities, 2. force structure and cross

force relations;3. communications and

perceptual influences, physical context, etc.;

Page 41: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

40

Estimation and prediction of:

1. effects on situations of actions by the participants;

2. threat (“threat assessment”).

The JDL (Joint Director Laboratory) data fusion model (1998 revised framework)

Measurements

Level 1Object Assessment

Level 2Situation Assessment

Level 3Impact Assessment

Level 4Process Refinement

(Resource Management)

Level 0Sub-Object Assessment

Signal/Features

Objects

Situations

Situations/Plans

Plans

Resources

Situations

Plans

Situations

Objects

Signal/Features

Steinberg, A. N., Bowman, C., and White, F. (1998). Revisions to the JDL Data Fusion Model, Proceedings of the 3rd NATO/IRIS Conference, Québec, Canada, 1998.

Page 42: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

41

Adaptive fusion to support mission objectives (eg. Sensor management)

The JDL (Joint Director Laboratory) data fusion model (1998 revised framework)

Measurements

Level 1Object Assessment

Level 2Situation Assessment

Level 3Impact Assessment

Level 4Process Refinement

(Resource Management)

Level 0Sub-Object Assessment

Signal/Features

Objects

Situations

Situations/Plans

Plans

Resources

Situations

Plans

Situations

Objects

Signal/Features

SituationAnalysis

Steinberg, A. N., Bowman, C., and White, F. (1998). Revisions to the JDL Data Fusion Model, Proceedings of the 3rd NATO/IRIS Conference, Québec, Canada, 1998.

Page 43: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

42

The cognitive hierarchy

Data/Information Fusion is a key enabler for Situation Analysis that aims to support the decision maker in constantly improving his situation awareness

• High‐levels information fusion: 

– situation analysis ‐ understanding

– sense‐making

• Multi‐Sensor Data Fusion (MSDF)

Understanding

Data

Processing

InformationCognition

KnowledgeJudgment

Sem

antic

gro

wth

(rea

soni

ng, u

nder

stan

ding

, ..).

Page 44: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

43

Are techniques of lower-levels (0-1) applicable to higher-levels (2-3-4)?

Page 45: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

44

MultiSensor Data Fusion Architecture (Level 0-1)

Page 46: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

45

Detailed MultiSensor Data Fusion Architecture (Level 0-1)

Page 47: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

46

SituationElement

Acquisition

CommonReferencing

OriginUncertaintyManagement

Situation ElementPerceptionRefinement

Situation ElementContextual Analysis

SituationElement

Interpretation

SituationClassification

SituationRecognition

SituationAssessment

Situation ElementProjection

ImpactAssessment

SituationWatch

ProcessRefinement

SituationModel

SituationPerception

SituationComprehension

SituationProjection

SituationMonitoring

Situation Analysis Tasks (Levels 2-3-4)

Roy, J.

Page 48: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

47

Target-to-WeaponPairing

Analysis

Activity LevelAnalysis

(Build Up?)

Data / InfoAssociation

(Correlation?)

Existence& Size Analysis(How Many?)

Element-to-Environment

Relationship(s)

SituationElement

Acquisition

SituationElement

Interpretation

SituationClassification &

Recognition

SituationAssessment

Situation Projection & Impact AssessmentProcess Refinement Situation Watch

EnvironmentPerception

EntityPerception

GroupPerception

Group Formation& Refinement(Structure?)

KinematicsAnalysis(Where?)

IdentityAnalysis

(What? / Who?)Entity-to-EventRelationship(s) Intent

Analysis(Why?)

CommonReferencing

EventPerception

SalienceAnalysis

(Outstanding?) Capability / CapacityAnalysis

(What Could It Do?)Trends/

ExpectationsAnalysis

ThreatValue

Diagnostic(Delta From

Expectations)

Data / InfoAlignment

(When? / Where?)

Situation ElementPerceptionRefinement

TimeManagement

PerspectiveAnalysis

OriginUncertainty

Entity-to-EntityRelationship(s)

BehaviourAnalysis

(What Is It Doing?)

Event-to-EventRelationship(s)

ActivityPerception

Entity-to-ActivityRelationship(s)

Event-to-ActivityRelationship(s)

Activity-to-ActivityRelationship(s)

Situation ElementContextual

Analysis

Source/CollectionManagement

CountermeasureAssessment

GoalManagement

PerformanceEvaluation

Change(s)Analysis

ThreatPrioritization

Problem(s)Analysis

OpportunityAnalysis

RiskAssessment

SituationCharacteristics

Analysis

SituationFamiliarityAnalysisSituation

Model

EngageabilityCalculation

Decomposition of the SA tasks (Levels 2-3-4)

Roy, J.

Page 49: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

48

Five generic (unordered) tasks for SA (1)

Detection: Is there anybody?

ObjectEnumeration: How many of these?

(Objects) →NClassification: To which family does it belong?

Object → ClassTracking: Where will it be next time?

Object(t) → Object(t +1)Association: What are the links between them?

Object_i ↔ Object_j

B

AA

A

A A

Page 50: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

49

Five generic (unordered) tasks for SA (2)

Detection: Is there anybody?

Object

Page 51: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

50

Five generic (unordered) tasks for SA (2)

Detection: Is there anybody?

ObjectEnumeration: How many of these?

(Objects) →NClassification: To which family does it belong?

Object → ClassTracking: Where will it be next time?

Object(t) → Object(t +1)Association: What are the links between them?

Object_i ↔ Object_j

X XX

Y

Y

Y

...

Page 52: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

51

Five generic (unordered) tasks for SA

What are the “Objects” concerned?1. Concrete simple objects:

ContactTrackTargetGroup of targets...

2. Abstracts or complex objects:SituationWorldThreatIntentOther mental states...

MSDF(L0-1)

SA(L2-3-4)

Page 53: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

52

Analysing a situation

Situation

Hall & Linas, 2001

Building the situation model

Queries

What is the probability that the observed target is a MIG-31?

Does the MIG-31 has the intent to enter the protected area?

Will the operator know at each time step the position and allegiance of the observed target?

DetectionEnumerationClassificationTrackingAssociation

Analysing the situation

Page 54: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

53

INFORMATION PROCESSINGFEATURES AND CHALLENGES

Page 55: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

54

Two basic ingredients for information processing

OntologyFrame of discernmentUniverse of discourseSet of possible worlds

...Description of the world

Epistemic statementBelief function

Probability distributionLogical statement

...Piece of information

Expresses:1. Background knowledge2. Level of granularity

desired/required

Expresses:Epistemic disposition relatively to the world

Non-monotonicity required!

Page 56: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

55

Information overload

The economist –February-March 2010

Page 57: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

56

Information overload (2)

The challenge is to have the “right” information,at the “right” time,at the “right” place.

Information (over)load would not be an issue if:agents had unlimited capacities (reasoning, memory,...)we had no constraints (time)

Information needs to be characterized to perform an adequate filtering and ensure an efficient

processing.

Page 58: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

57

The problem at hand: socio-technical systemsNEOps = Network Enabled Operations

Agent = Artificial or Human

Page 59: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

58

Heterogeneity (1)

Each agent has its own :o View of the situation (partial)o Language

Vocabulary (ontology)Syntax and grammar (formal or not)

o Set of possible actions (competencies)o Strategy (way of functioning)o Limitations

of actions (sensing, reasoning, ...)of languageof storage (memory)...

Information is imperfect:

o Incomplete (locally),o Conflicting,o Uncertain (several types),o Erroneous,o Non-credible,o Irrelevant,o Useless, ...

Page 60: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

59

Toward an information quality ontology...

[Klir & Yuan]

[Smets]

[Smithson]

ReliabilityUtility

ProximitySupportabilityExpectability

Credibility…

Page 61: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

60

Heterogeneity (2)

Each agent has its own :o View of the situation (partial)o Language

Vocabulary (ontology)Syntax and grammar (formal or not)

o Set of possible actions (competencies)o Strategy (way of functioning)o Limitations

of actionsof languageof storage (memory)...

We need to manipulate:

o Numerous ontologieso Numerous theoretical frameworks

Page 62: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

61

InteroperabilityNumerous theoretical frameworks

Numerous ontologies

STANDARDS TRANSLATIONS

JC3IEDM

STANAG 4162 (Bayesian)

Ontology alignment / matching

Pignistic probability

Need to be general enough to encompass

the rich variety of information to be

processed

Loss of information must

be controlled

Data models(ontologies)

Theoretical framework for

information processing

Page 63: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

62

Features of information

Distributed information;

Imperfect quality;

Heterogenenous sources (hard-soft, structured-unstructured, objective-subjective, several ontologies);

Represent mental states (belief, knowledge, uncertainty, ...);

Trusted and understood by a human;

Updated and revised non-monotonically;

Possibly erroneous ontology.

Page 64: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

63

THEORETICAL TOOLS FOR SUPPORTING SA

Page 65: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

64

Sensing to Understanding to Decision Support

Measuring Reasoning

Mathematicaland logical tools

Mental state

Page 66: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

65

Why a formal model for SA?

A highly formal approach for the design of situation analysis and decision support systems is unavoidable if one is interested in

the reproducibility and traceability of the results,

verifying systems before they are fielded,

rapid and robust software implementation,

characterization of the complexity of representations and tasks.

Page 67: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

66

GIT

Generalized Information Theory (GIT)Information and knowledge modelingTheoretical aspects of uncertainty representationQuantitative and qualitative

MASMultiAgent Systems (MAS)

Multisensor Systems, NetworkMultisensor Data FusionSocial Networks

Dec.Decision Theory (Dec.)

PlanningOptimizationRisk management

MAS + Dec.Game theory

GIT + Dec.POMDP

MAS + GITInformation fusionAggregation/Combination

SA

Three domains for information processing

Page 68: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

67

In practice, three axes of thought

Algorithms anddata structures

Domain ofapplications

Theoretical aspects(GIT, MAS, Decision)

Measures of performances along the three axes:

1. Theoretical tools: Conceptually correct?

2. Programming: Able to address large problems? Computability? Tractability?

3. Domain of application: Useful to user?

Page 69: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

68

Origins of theoretical tools

Probabilistic model of the atom

Mechanistic model of the brain

Mathematical representation of the world

Logical representation of the world

Page 70: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

69

Logical vs. mathematical description of the world

Logic is about the coherence of

the world

Mathematics is about measuring

the world

(If Flying then Engage) and Flying→ Engage

•P(Flying object)=0.7•Object Flying at 872 km/h

(If Flying very fast then Engage) and (Flying at 754km/h)

→ ???SITUATION ANALYSIS

1000500

Page 71: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

70

Classification of uncertainty processing methods (Pearl, 1988)

Syntactic(Extensional)

(Quantitative, numerical, … approaches)

Semantic(Intentional)

(Qualitative, symbolic, … approaches)

Numerical values of [0,1]

Logical consequence

relations

Uncertainty Computation Semantics

Reconciliationof both aspects?

sloppy

clear

convenient

clumsy

Page 72: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

71

Convergence of two areas

Physics, EngineeringMathematics

Cognitive SciencesLogic

GeneralizationNew modalitiesKnowledge, time, action

GeneralizationNew types of uncertaintyFuzziness, non-specificity

Cross-fertilisation

Page 73: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

72

Towards a unified theory?

Page 74: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

73

Belief functions for supporting SA?Requirements Belief functions Possible extensions

Knowledge vs belief representation -(certainty=knowledge)

Link with modal logics (KD45, S5?)

Uncertainty +

Non-specificity +

Fuzziness - Fuzzy belief functions

Randomness +

Measuring the world +/-(initial BPAs)

Ontological non-monotonicity - TBM, DSmT, ...

Reasoning about the structure of the world

+/-(no clear semantics)

Incidence calculusFagin-Halpern structures

Deal with distributed information +/-(no explicit MAS

framework)

Supplemented with a MA framework

Combining sources +

Updating (changing world) + “temporal” belief functions

Revising (new information) +

Page 75: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

74

In SA we need …

o A formal ontology (description of the world);

o A structure for representing and reasoning about psychological states and semantic properties such as belief and knowledge;

o A way for representing and dealing with different aspects of uncertainty;

o Rules for combining information from different sources in order to estimate or predict states;

o A framework being generalizable to the multiagent case;

o An explicit definition of context.

Page 76: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

75

Why a logical approach to SA?

1. The logical approach brings a formal way to analyse the system in terms of completeness, consistency, decidability.

2. Logic provides a formalism and language to describe mathematical structures and their properties, as well as dynamic processes.

3. Logic gives a clear distinction between syntax and semantics.

Page 77: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

76

Modal logic and Situation Analysis?

The 2001 Workshop on Modal Logic in AI (Vienna) proposed thefollowing program:

Modal logics for multi-agent systems, Temporal reasoning based on temporal logic, Spatial reasoning and modal logic, Modal logic and logic-based knowledge representation, Description logics, Epistemic logic in AI, Deontic logic in AI, Modal logics of action, Representation of propositional attitudes, Modal logics for intelligent conceptual modeling, Modal logics for ontologies

Page 78: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

77

What is Interpreted Systems Semantics?

• Proposed in the 90s as a general model for analysing distributed systems through epistemic properties (Fagin, Halpern, Moses and Vardi).

• A logical approach based on epistemic logics notions, with a dynamic component.

• Epistemic and temporal properties of groups of agents are verified through efficient model checkingtechniques.

• The model can be enriched with probabilistic notions, and extensions (thus ... belief functions!)

Page 79: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

78

DRDC PROJECTS

Page 80: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

79

Where do we stand so far…

Measurements

Level 1Object Assessment

Level 2Situation Assessment

Level 3Impact Assessment

Level 4Process Refinement

(Resource Management)

Level 0Sub-Object Assessment

Signal/Features

Objects

Situations

Situations/Plans

Plans

Resources

Situations

Plans

Situations

Objects

Signal/Features

(Source: Steinberg et al., 1998)

Belief functions

2005

2010

2015

...

Page 81: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

80

Situation Analysis Support Systems Objectives: To analyze and design real-time, computer-based Situation Analysis Support Systems to aid the Halifax class command team in achieving the appropriate SA to conduct C2.

Apr 2003 – Mar 2006

Classification and target identification

Page 82: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

81

Goal Action Selection

ManagementConstraints

MOP

MOE

Performance Evaluation

Effectiveness Evaluation

EnvironmentFusion

Processing

Sensors

AdaptationGoal Action Selection

ManagementConstraints

MOP

MOE

Performance Evaluation

Effectiveness Evaluation

EnvironmentFusion

Processing

Sensors

AdaptationGoal Management

Airborne Information Fusion & Management for Tactical Picture

Compilation Objectives: To investigate, design and develop advanced information fusion and sensor management concepts suitable for insertion into the modernized CP-140 (P-3C).

Apr 2004 – Mar 2007

Classification and target identification

Page 83: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

82

RAP Compilation and Exploitation for Dynamic Operations Management

Objectives: To investigate: i) air situation awareness enablers, and ii) air operations management enablers for a RAP of the CANR Air Operations CentreMay 2005 – March 2008

1 CAD /CANR HQ

Set the basis for a formal model of SA based on the

interpreted systems semantics

Page 84: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

83

Situation Analysis for the Tactical Army Commander

Objective:To provide the Canadian Army the means to build an automated reasoning capability in order to support Situation Analysis, a process by which the human gains Situation Awareness.Apr 2005 – Mar 2009

Initiate a study on the use of belief functions in the

JC3IEDM

150 SAW requirements from the CSE community

Page 85: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

84

A Research Initiative to Evaluate Situation Awareness Strategies for

Harbour ProtectionObjectives: Investigate how new Situation Analysis technologies (interpreted systems) can formally characterize a complex situation for surveillance.Create a toolbox to evaluate motion strategies in a realistic surveillance context for the purpose of enhancing decision support capabilities.

Apr 2007 – Mar 2011

Develop a basic toolbox for SA

Page 86: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

85

Information Relevance in Hierarchical C2

Objectives:This project aims at investigating qualification or “RELEVANCE” of information in a system of systems (C2IS) of different levels (tactical-operational-strategic) of Command and effect based analysis enablers

March 2008 – March 2012

Study the capacity of belief functions to model relevance, reliability and credibility concepts (as

understood by II)

Page 87: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

86

Data Fusion Solutions for Monitoring CBRNE Threats (DF - Surveillance)

Objectives:

(1) Explore novel situation analysis concepts for epidemics early detection usingunconventional and conventional information sources.

(2) Investigate novel on-demand classification techniques applied to live data streams

April 2009 – Mars 2013

Use belief functions for data stream processing and

risk evaluation

Page 88: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

87

SASNet Architecture: Self-healing Autonomous Sensor Network

Objectives: To test in realistic operational situations an unattended sensor network together with its electronic and software components, with an emphasis on self-healing capacities and advanced communication protocols.

Apr 2007 – Mar 2011

Insérer une photo représentative de cette

planche

Target classification

Page 89: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

88

Information quality assessment approaches;Knowledge modeling and representation: exploitation of efficient machine

representations of relevant aspects of the world; Visualisation and human-system integration;Fusion of structured and unstructured information;Formal methods to identify and represent relevant and critical information

to support decisions;Formal methods of ontological engineering to produce defensible

representations of the world (e.g., situational constructs);Work domain models could be constructed based on cognitive

engineering techniques in order to understand and formally document the information needs of decision makers;

Distributed or social aspects: architecture SoS, open systems;General framework for representing knowledge, uncertainty, and actions;Measures of performance of support systems.

Conclusions: some critical research issues

Page 90: Reference of the annotated version: Bossé É., Jousselme A ... · Artificial Intelligence (e.g. multi-agent, expert systems, ... of decision making must derive from an analysis of

89