Status update on GM-VVKarl-Gustav Pettersson, FMV

Constantinos Giannoulis, DSV/FOI

Swedish VV&A Competence network

2011-05-06

FMV SMART-Lab

Status update on GM-VV

Agenda

GM-VV Document set

GM-VV Volume 1: Introduction and overview

Walkthrough the document

Rationale

Standardization status of GM-VV

Introduktion av SISO

SISO standardiseringsprocess

SISO Balloting-process

Pilot Case Study

Objectives

Findings

Ref:

W.F. Waite AEGIS TG

2004 JSMARTS vision 2020VV&A

Transformation

Representation

Abstraction

The GM-VV

Objectives

Provide a generic framework to develop an argument to justify why identified

models, simulations, underlying data, outcomes and capabilities are believed to

be acceptable for deployment in the target (intended) operational (use) context,

Define the required information and argumentation mechanisms allowing well-

balanced and risk-informed arguments with various levels of formality for

acceptance decision making,

Provide the community with a common language to better facilitate the

communication and co-operation between all participants,

Foundations

Well-established international standards,

knowledge and information management approach, enables formal and tool-

supported sharing, transferring, managing and reusing of verification, validation

and acceptance information at both project and enterprise levels.

The GM-VV document set

The GM-VV Volume 1: Introduction and Overview

The GM-VV Volume 2: Implementation Guide

The GM-VV Volume 3: Reference Manual

The GM-VV

The Generic Methodology for Verification & Validation

M&S/VVA

Organizations

and Personnel

GM-VV Vol 1.

Introduction

and overview

GM-VV Vol 2.

Implementation

guide

GM-VV Vol 3.

Reference

Manual

Is introduced and defined

in Provides guidance on

implementation and use of

Provides technical

and referential

background

information for

Learns the basics

of GM-VV from

the

Apply GM-VV by using the

Obtain VV&A and related background information on and for use within GM-VV

from the

GM-VV Rationale

GM-VV Vol. 1

1

Simulation Interoperability 2

Standards Organization 3

(SISO) 4

5

6

Generic Methodology for 7

Verification and Validation (GM-8

VV) to Support Acceptance of 9

Models, Simulations and Data 10

11

GM-VV Vol. 1: Introduction and 12

Overview 13

14

SISO-GUIDE-00X.2-201x-DRAFT-V1.10.6 15

16

03 - 05 - 2011 17

18

19

Prepared by: 20

SISO Generic Methodology for Verification and Validation 21

Product Development Group 22

(GM-VV PDG) 23

GM-VV standardisation

The GM-VV document set

The GM-VV Volume 1: Introduction and Overview

(soon to be balloted)

The GM-VV Volume 2: Implementation Guide

(more comment rounds, then to be balloted)

The GM-VV Volume 3: Reference Manual

(more comment rounds, not to be balloted)

What is SISO?

Simulation Interoperability Standards

Organization (SISO)

Non profit organization

Based in the US but open to the

international community

About 1200 members (20% non US)

Three annual workshops, two in the US

and one in Europe

Three ways to become a member

Annual fee ~$100

Participate in a conference ~$700

Member of a sponsoring organization

or corporate

Standardization process

http://www.sisostds.org

”SISO’s mission is to provide

an open forum that promotes

the interoperability and reuse

of models and simulations

through the exchange of

ideas, the examination of

technologies, and the

development of standards.”

Development of standard

Maturation of concept

Overview of the SISO standardization process

Standardization process

Sponsor

OrganizationStudy Group

Standing

Study Group

Product Nomination

Product

Development

Group

Balloted

Standard

Product

Support Group

Ballot Pool

SISO Balloting process

Standardization process

Final drafted

product to SAC

for approval

PDG vote to go

to balloting

SAC announce

Ballot

SAC announce

Ballot

SISO

Members

Other

interested

people

Preliminary

Ballot Group

Interme

-diate

Ballot

Group

Membership

rules

Balance

guidelines

Ballot Group

BallotingComment

resolution

Product

approval

Ballot balancing rules

Membership rules:

Apply for membership and

pay fee

Committing to conduct

thorough review and

completing in required time

Representation:

Commercial, Government,

Academic

Interest:

User, Developer, General

Interest

Balance guidelines:

No representation should

exceed 75% and each

category >10%

No organization >25%

No interest >50%

The GM-VV: Pilot Case Study

Pilot Case Study

Rationale

Technology readiness

level is 4 (Component

and/or Breadboard Validated in

Laboratory Environment

Setting)

Test the method in a

real environment

(improve, involve, strengthen)

Objectives

Correctness and utility of

the document set*

Practical applicability of

GM-VV: knowledge &

experience

Educational material

development (tutorials)

Tools (evaluation,

requirements)

Introductio

n

Pilot Case Study Scenario: FileProof

Rijkswaterstaat(2006-2008)

aimed at:

Immediate reduction of

traffic jams by means of

smart, innovative and

practical approaches

About 60 projects in 3 main

categories: Projects aimed at reduction of

regular and incidental traffic

jams

Projects tempting people to

think and act differently

Pilot Case Study Scenario: FileProof

Our Focus:

Flashing lights off On site

Why Flashing Lights

Reduction of speed

Not to draw attention to an

accident

Directive

First vehicle

(fend off car) lights ON

Other cars

(rescue workers) lights OFF

Compliance in practice?

Falls short of expectations

What are the effects of showing one or multiple

flashing lights on: The safety of rescue workers on site, and

Traffic flow

Effect of light color (blue or amber / yellow)?

Effect of time of day (daylight / evening)?

Effect of position accident (own lane or other

direction)?

Pilot Case Study Scenario: FileProof

Approach

5 workshops, 4 nations

(FR, NL, DE, SE)

Focus on Technical

Processes & Products

Spiral Development

(iterative & incremental)

Deliverables

White Paper

Authoritative & Educational

Illustration

Benefits, Limitations &

Improvements

Introductory Tutorial

Presentation

Dissemination for M&S

Community

Illustration & Guidance

Application

Introductio

nPilot Case Study Scenario: FileProof

The content of the document has a steep learning curve with a

high entry level Wording is long, fuzzy, vague and pressumes a lot

Misleading titles creating false expectations

No leading document driving the reader

Weak cross-document proof reading

There is no document that specifies THE methodology Bits & bytes scattered among the 3 documents

HB perspective not clearly communicated

Multiple objectives set

Overlapping information & inconsistencies

Incompatible to SISO product nomination

More cases

The Case

StudyPilot Case Study Scenario: Findings

Tack!

Frågor?

Status Update on GM-VV