IS Metrics for C2 Processes

Working Group 3

Brief

Team Leaders:

Steve Soules

Dr. Mark Mandeles

Charge to the C2 Processes Working Group

• Objective: Develop metrics to measure the quality of command and control processes

• Examine historic examples of the impact information technology has had on command and control processes.

• Identify common factors that can be used to measure the quality of the command and control processes, including speed of command, correctness of decisions, and effects of dissemination of commands.

• Review the results of the other three Groups. Apply their findings to the command and control factors to begin to define command and control process changes that could better take advantage of information superiority improvements.

• Define the metrics that can be used to capture the quality of the command and control processes.

• Discuss ideas for potential futures areas of analysis.

Command & Control Process Definition

• Working Group C2 Process Definition: The integration of organizations, systems and doctrine used by commanders to direct forces to accomplish missions

• Examples of C2 Processes:– Command and Control by Direction– Command and Control by Negation– Centralized Command / Decentralized Execution– Autonomous Control (e.g., special operations or submarine

operations)– Command and Control by Self Synchronization (for future:

combat group makes its objectives)

C2 Process Metrics Users

• Acquisition Analysts• Experiment/Exercise Analysts• Operations Analysts

• M&S Developers

• Seeking Metrics to evaluate if C2 Processes are:

- Operationally Sound

- Technically Feasible

- Cost Effective

C2 Factors to Consider inDetermining Metrics

• Threats/Missions

• Risks

• Adaptive

• Visualization

• Speed of Command

• Speed of Force Actions

• Awareness

• Shared Awareness

• Synchronization

• Confidence

• Scale

• Environment

• Dissemination of Commands– Reach

– Richness

• Efficiency

• Correctness

• Completeness

• Collaboration

• Interaction

• Human Factors– Experience

– Fatigue

– Stress

– Initiative

• Errors and Types of Errors

Three Areas of C2 Process Evaluation

• Performance of the C2 Process:– Ability of the process to Monitor / Understand / Develop / Predict– Ability of the process to Decide / Direct / Collaborate

• Effectiveness of the C2 Process:– Ability to improve Force Synchronization– Ability to improve shared awareness– Ability to collaborate and interact in each of the process functions

• Impact of the C2 Process on Force / Mission Effectiveness:– Satisfy mission objectives in an efficient manner

Source: (Evidence Based Research)HEAT Analytic Structure

HEADQUARTERS

DEVELOP ALTERNATIVE ACTIONS

UNDERSTAND

MONITOR

PREDICT CONSEQUENCES

DECIDE

DIRECT

QUERYINFORM

ENVIRONMENT: - OWN & ENEMY FORCES - PHYSICAL - POLITICAL & ECONOMIC

C2 Process Functional Areas

How do you measure the performance, effectiveness and impact of

C2 Processes?

JOINT BATTLEMANAGEMENT

USAUSN

USAFUSMCJFMCC

JFLCCJFACC

CJTF

ExecutionC2

Monitor

W

e

a

p

o

ns

Command by

Negation

SynchronizedCommand and

Control AutonomousControl

Understand

Develop AlternativesPredict Consequences

Decide

Commandby

DirectionNetwork

“Power of Collaborative Interaction”

Direct

Measuring the Performance of the C2 Process

Monitor:Does the decision maker have the awareness on the situation he needs?Does he have confidence in the awareness?

Understand:Does the decision maker understand the situation?Can the decision maker retain and accumulate his understanding?

Develop Alternatives:How many possible valid alternatives did they develop?What was the degree of variety in the alternatives?Predict Consequences:Did they accurately predict the possible outcomes?

Decide:How long did it take the decision maker to make the decision?Did the decision maker make the best decision?

Direct:How long did it take to implement the decision?Was the decision implemented as intended?

Collaborate:What level of collaboration was used in each of the above functions?* Measure these throughout the organization for different C2 processes

*

Measuring the Effectiveness of the C2 Process

• Ability to Synchronize Forces• Ability to gain knowledge rapidly to enhance

awareness, build confidence and execute sound judgement

• Ability to collaborate and interact• Ability to mitigate errors, e.g., incorrect target

identification• C2 Errors• Friendly fire• Inadequate analysis, e.g., Chinese Embassy in Belgrade• Inappropriate information• System breakdowns or crashes

Measuring the Impact of the C2 Process on Mission / Force Effectiveness:

• Complete mission objectives– Yes/No

• Efficiently Complete mission objectives • Fewer Casualties• Faster Time• Fewer Leakers• Less Collateral Damage

Examples of Metrics for MOP’s in the C2 Functional Areas

Monitor Accuracy

Understand TimeAccuracy

Develop Alternative Actions NumberAccuracy

Predict Consequences Accuracy

Decide TimeAccuracy

Direct TimeAccuracy

**Degree of Collaborationused in all areas

scored valuein each area

**Note:“accuracy” also implies its opposite -- “error”ie. transmission error rate/“ground truth”

Example of a C2 Process Evaluation

Richness Reach Time

Monitor % of GroundTruth (GT)

# of Forces Hrs

Understand % of GT # of Forces Hrs

DevelopAlternatives % of GT # of Alts Hrs

PredictConsequences % of GT Hrs

Decide % of GT Hrs

Direct % of GT # of Forces Hrs

**Degree ofCollaboration used inall areas

Score Valuein Each Area

Score Valuein Each Area

Hrs

Impact of the C2 Process on Measures of Force Effectiveness

Were the mission objectives accomplished?

How long did it take to meet the missionobjectives?

What was the Loss / Exchange Ratio inaccomplishing the mission objectives?

Etc.

C2 Process Evaluation Sample Comparison Analysis

C2 Process #1 C2 Process #2

MOP: Richness ValueWith collaborationValue

% of Ground Truth % of Ground Truth

MOP: Reach ValueWith collaborationValue

# of Forces connected #' of Forces connected

MOP: Time Hrs Hrs

MOE: MissionObjectives Met

# Objectives Met # of Objectives Met

MOE: Time to MeetObjectives

Hrs Hrs

MOE: Loss/exchangeRatio ect.

Ratio Ratio

Example of Graphing MOP Aggregate Performance

Tim

e

Ric

hnes

sReach

(Hrs

)

(% o

f Gro

und

Trut

h)

C2 Poc #1

C2 Poc #2

(# Of Forces Connected)

**Note: This graph could be integrated with the the synchronization and awareness graphs to show integrated effects.

C2 Process EvaluationMOP and MOE Conversion/Graph

• Conversion factor: Score/Grade by range of performance: Value State Space

• MOPs Relationship to MOEs: Influence Diagram/ Cause and effect analysis / Multi attribute utility analysis

• Error Analysis: Tradeoff Between Type I and Type II Error

• Graph: Richness, Reach, Time, Number of Objectives, Time to Objectives, Loss Exchange Ratio values from Score/Grade

• Simulation as a Sensitivity Test

Example of GraphingMOP and MOE Aggregate Evaluation

Reach Tim

e(H

rs)

Richnes

s

Tim

e(H

rs)

Missions M

et

(#s)

Agg MOP Value Agg MOE Value

(# of Forces Connected) (% o

f Gro

und Tru

th)

(Value State Space) (Value State Space)

(Ratio)

Loss Exchange

Loss Exchange

Reach

MOE ValueMOP Value

Richnes

sTim

eT

ime

Missions M

et

Loss Exchange

Reach

MOE ValueMOP Value

Richnes

sTim

eT

ime

Missions M

et

CP #1 CP #2

Summary

• Measuring C2 processes requires an evaluation of both the performance and the effectiveness of the process in meeting military objectives

• The individual performance of interaction and collaboration enabled by future networks is difficult to measure but we should be able to capture its effects in conducting C2 functions

• The MOPs and MOEs introduced should be further evaluated before being used in a limited objective experiment in a controlled situation as a starting point

• What discussion/research needs to be accomplished next?