assurance case frameworks part of high confidence software msr t. scott ankrum mitre — software...

Assurance Case FrameworksPart of

High Confidence Software MSR

T. Scott Ankrum

MITRE — Software Engineering Center

March 11, 2004

March 11, 2004 T. Scott Ankrum — MITRE 2

Credits

• Part of the “High-Confidence Software Initiative” research project

• Supported by the MITRE Sponsored Research program

• Supporting cast– Chuck Howell– Alfred Kromholz – Jim Moore

Working for almost two years.


Agenda

• What is an Assurance Case?

• Structuring an Assurance Case

• Problems With Assurance Cases

• Choosing a Tool

• Structuring Selected Standards

• Conclusions and Follow-on


What Is an Assurance Case?


History of Assurance Cases

• Originally Only Safety Cases– Aerospace– Railways, automated passenger– Nuclear power– Off-shore oil – Defense

• Security Cases– Use compliance rules more than an assurance case

• Cases for Business Critical Systems


Definition of Safety Case

• From Adelard’s ASCE manual:

“A documented body of evidence that provides a convincing and valid argument that a system is adequately safe for a given application in a given environment.”


Definition of Assurance Case

• Generalizing that definition

A documented body of evidence that provides a convincing and valid argument that a specified set of critical claims regarding a system’s properties are adequately justified for a given application in a given environment.


Where is an Assurance Case Used?

– Critical systems under regulation or acquisition constraints

– Third-party certification, approval, licensing, etc.

– Documented body of evidence required

– Need a compelling case that the system satisfies certain critical properties for specific contexts

– Examples: DO-178B, Common Criteria, MIL-STD-882D

– “safety case”, “certification evidence”, “security case”…

Collectively we’ll refer to them as “assurance cases”


Structuring an Assurance Case


Elements of an Assurance Case

• Claims

• Arguments

• Evidence

• Other elements, depending on notation


Claims in Assurance Cases

• Assertion of compliance with key requirements and properties

• Must be in a specific context– Environment

– Services or behavior

– Threats

– “Is this brick safe?” illustrates why…

• Sub-claims may be analogous to “lemmas” in a proof – separation of concerns

– workflow

– makes overall case more manageable


Arguments in Assurance Cases

• Link evidence to claims via inference rules– Deterministic: defined rules => true/false assertion

– Probabilistic: quantitative, statistical, numerical threshold (MTTF)

– Qualitative: rules with an indirect link to desired properties (standards, process guides)

• No such thing as perfection: “It is quite possible to follow a faulty analytical process and

write a clear and persuasive argument in support of an erroneous judgment.” – R. Heuer, The Psychology of Intelligence Analysis


Evidence in Assurance Cases

• Process and people used to develop the system

• Systematic testing

• Product review and analyses

• Mathematical proofs

None of these alone provides adequate evidence


Problems With Assurance Cases


Problems with Assurance Cases

• There are problems in every aspect of assurance cases– Building them– Reviewing them– Maintaining them– Reusing them

• Problems result from:– volume of material – little structuring support– ad hoc “rules of evidence”


Building the Assurance Case – 1

• Most guidance is:– strong on excruciating detail for format

– weak on gathering, merging, and reviewing evidence

• Guidance often uses the “cast a wide net” tactic– Assurance costs time and money

– “Squandered diagnostic resources”

– Some work on a “portfolio management” approach


Building the Assurance Case – 2

• With free format text and no tool support:– coordination is hard

– tracking is hard

– workflow management is hard

• Imagine building a 500 page project plan by hand, on paper


Reviewing the Assurance Case – 1

• Stacks of free-format text makes review tedious– Hard to see linkages or patterns– Hides key results in sheer volume

• Weak guidance on review of arguments and evidence often results in ad hoc criteria (be very nice to your reviewer!)

• Rarely is there explicit guidance for weighing conflicting or inconsistent evidence


Reviewing the Assurance Case – 2

“Often viewed as irrefutable, evidence is, in fact, an interpretive science, refracted through the varying perspectives of different disciplines. ... [Judging evidence requires] reasoning based on evidence that is incomplete, inconclusive, and often imprecise.”

The Evidential Foundations of Probabilistic Reasoning, David Schum


Maintaining the Assurance Case – 1

• The one thing more brittle than software is – the associated assurance case

• It is difficult to understand impact of a change on assurance structure because:– volume of information is immense

– impact of a change on assurance structure is complex


Maintaining the Assurance Case – 2

• Reasons for change– The claims and/or evidence have changed

– Arguments no longer valid or new ones needed

– Evidence is irrelevant or new evidence needed

– “Weak link effect” of discrete systems compounds problem

• Revalidation costs are a major burden

• “Breakage” of successive dependencies


Reusing the Assurance Case – 1

• Assurance case frameworks are rarely the subject of study per se

• More attention for these would be useful– tool support– idioms and templates– extracting patterns for future use


Reusing the Assurance Case – 2

• Relationship among claims, arguments, and evidence – not often explicit – hard to distinguish the reusable from the

project specific portions of assurance case

• Compare this with building a deck with the help of a project planning tool


Choosing a Tool


What Should a Tool Provide? – 1

• Simple management of complexity and volume– MS Project-like planning and tracking of complexities– Checking simple structural properties– Browsing and report generation

• Support for multiple, geographically dispersed users– with data integrity– concurrently or asynchronously

• Useable for any domain– not specific to any one industry– not specifically for safety cases or security cases


What Should a Tool Provide? – 2

• Replanning as things change: (“No plan survives contact with the enemy.”)

• Templates and tailoring to – capture lessons learned – reduce wheel reinvention

• Uses and/or exchanges consistent notation for: – claims, evidence, and arguments

• Widely executable– runs under Windows 2000 or Windows XP– or has a Windows based GUI


Notations Considered

• Toulmin Structures– Stephen Toulmin, The Uses of Argument, 1958

• Goal Structuring Notation– Described in Tim Kelly’s dissertation, York, 1998

• ASCAD (Claims-Arguments-Data): – ESPRIT SHIP project headed by Adelard

• Proprietary


Selected Tool: ASCE

• Established Notations: GSN & ASCAD

• Not Industry or Safety Specific

• Extensible through a Schema

• Case is exportable to project documents

• Stable, no failures during evaluation

ASCAD Notation

Supports Supports

Is evidence for

Is evidence for

Is a subclaim of Is a subclaim ofIs a subclaim of

Is evidence forIs evidence for

SupportsSupportsSupports

Is evidence for

CLAIMRequirements Are

Correct, Complete andConsistent

ARGUMENTA formal specificationadds rigor and exposes

defects early

ARGUMENTFagan's two famouspapers make a good

argument forinspections, and theyare a proven practice.

CLAIMSoftware Has Few

Defects

ARGUMENTRequirements areonly useful if they

are used todesign the

system.

EVIDENCETest Plan Peer

Review statistics

EVIDENCERequirements

Traced to DesignComponents

EVIDENCEFagan Inspectionof each section

EVIDENCEWritten in Z

EVIDENCEFilled in test Plan

ARGUMENTTest scenarios were

marked with thecompletion date to

document that all testswere eventually

successful.

CLAIMSystem Was Well Tested

CLAIMSystem Built toRequirements

ARGUMENTPeer reviews are a

good way to increasquality and the only

available way fortest plans.


Structuring Selected Standards


Hypotheses

• Assurance is Assurance is Assurance– All assurance cases are similar enough in structure

that a distinct tool for each domain is not required

• Assurance Standard Assurance Case– There is a relationship between the actual or

implied structure of an assurance standard and the structure of an assurance case instantiated from that standard


Mapping Standards into ASCE

• Computer Security:– Common Criteria — Evaluation Assurance Level 4

• Aviation Safety: DO-178B – Software Considerations in Airborne Systems

• Medical Device Safety:– Discussing with FDA

Center for Devices & Radiological Health

Top-levelClaim

SupportingClaim

Needs supportingclaims

ArgumentunderParent

Evidenceto support

a claim

Argumentfor Sub-claims

Argumentfor

Evidence

ParentClaim

Start

Finished


Process Mechanics

• ASCAD notation: – Claims– Arguments– Evidence

• We used arguments between claims– This is a deviation from the notation

• Tried to capture all of the standard


Advantages of the Tool

• Carries both graphic structure and text

• Hyperlinks from node to a web page or file

• Enforces structure rules– Rules can be temporarily suspended– User-supplied rules can be added

• Can export for inclusion in a document

• User views can show parts of the structure


Mapping the Common Criteria

• Most hierarchical of the standards– Classes, Families, Components, Requirements– Components are atomic and cumulative

• Nearly mechanical process of mapping• Most of the structure consists of Arguments

– No sub-claims, only a top-level claim– Requirements are place-holders for evidence

• Objectives paragraphs became arguments

Supports

Supports

Supports

Supports

Supports

Supports

Supports

Supports

Supports

Supports

SupportsSupports

Supports Supports

SupportsSupports

SupportsSupports

Supports Supports

Supports Supports

Supports

Supports Supports

SupportsSupports

Supports

Supports

Supports

CLAIMEAL4

[Confidence in Securitybecause the product has

been] methodically designed,tested, and reviewed

ARGUMENTACM

ConfigurationManagement

ARGUMENTACM_AUT

CM Automation

ARGUMENTACM_CAP

CM Capabilities

ARGUMENTACM_SCPCM Scope

ARGUMENTADO

Delivery andOperation

ARGUMENTADO_DELDelivery

ARGUMENTADO_IGS

Installation,Generation and

Start-up

ARGUMENTADV

Development

ARGUMENTADV_FSP

Development

ARGUMENTADV_HLD

High-Level Design

ARGUMENTADV_IMP

ImplementationRepresentation

ARGUMENTADV_LLD

Low-Level Design

ARGUMENTADV_RCR

RepresentationCoorespondence

ARGUMENTADV_SPM

Security PolicyModeling

ARGUMENTAGD

Guidance Documents

ARGUMENTAGD_ADM

AdministratorGuidance

ARGUMENTAGD_USR

User Guidance

ARGUMENTALC

Life Cycle Support

ARGUMENTALC_DVS

Development Security

ARGUMENTALC_LCD

Life Cycle Definition

ARGUMENTALC_TATTools and

Techniques

ARGUMENTATE

Tests

ARGUMENTATE_COVCoverage

ARGUMENTATE_DPT

Depth

ARGUMENTATE_FUN

Functional Tests

ARGUMENTATE_IND

Independent Testing

ARGUMENTAVA

VulnerabilityAssessment

ARGUMENTAVA_MSUMisuse

ARGUMENTAVA_SOF

Strength of TOESecurity Functions

ARGUMENTAVA_VLA

VulnerabilityAnalysis

Supports Supports

SupportsSupports

Supports Supports

Is evidence forIs evidence for

Supports Supports

Is evidence for Is evidence for

Supports Supports


ARGUMENTATE

Tests(50 words)

ARGUMENTATE_COVCoverage

ARGUMENTATE_COV.1Coverage(31 words)

ARGUMENTATE_COV.2Analysis ofCoverage(35 words)

EVIDENCEATE_COV.1.Coverage(53 words)

EVIDENCEATE_COV.2.2

Analysis ofCoverage(27 words)

ARGUMENTATE_DPT

Depth

ARGUMENTATE_FUN

Functional Tests(92 words)

ARGUMENTATE_DPT.1

Testing: High-LevelDesign

(43 words)

ARGUMENTATE_FUN.1

Functional Testing(27 words)

EVIDENCEATE_DPT.1.

Testing: High-LevelDesign

EVIDENCEATE_FUN.1.

Functional Testing(87 words)

ARGUMENTATE_IND

Independent Testing(53 words)

ARGUMENTATE_IND.1

IndependentTesting

(15 words)

ARGUMENTATE_IND.2

IndependentTesting

EVIDENCEATE_IND.1.Independent

Testing(51 words)

EVIDENCEATE_IND.2.Independent

Testing(41 words)


Mapping DO-178B

• Less structured, its title begins:– “Software Considerations …”

• Focused on system/software product lifecycle– Other standards are not time-structured– Claims, sub-claims, and evidence are laid out in

approximately their chronological order – No linkages between the generation of one

artifact and its later use

Supports

Is evidence for

Is evidence for

Supports Supports

Supports

Supports

Supports

Is a subclaim ofIs a subclaim ofIs a subclaim of

Supports

Supports

Is a subclaim ofIs a subclaim ofIs a subclaim of

Supports

Is evidence for

Supports SupportsSupports

Is evidence for

Supports Supports Supports

Is evidence for

Is evidence for

Is evidence for

Is a subclaim of

Is a subclaim ofIs a subclaim of

Is a subclaim of

Is a subclaim of

Is a subclaim of

Supports

CLAIM7.2.2

Baselines &traceability

are

CLAIM7.2.3 - 7.2.6Problem &Change

Management is

CLAIM7.2.8

Softwareload control

is

CLAIM7.2.9

Software lifecycle

environment

CLAIM7.2.1

Configurationitems areidentified

ARGUMENTnot explicit

Satisfactory SQAprocess requires

three characteristics(23 words)

CLAIM7.0

SCM process isproperly

established and

CLAIM8.0

SQA process isproperly

established and

ARGUMENTnot explicitSatisfactory

Certification LiaisonProcess comprises

three factors

CLAIM7.2.7

Archive,retrieval andrelease are

EVIDENCE11.17

ProblemReports

(0 words)

EVIDENCE11.15

Software LifecycleEnvironmentConfiguration

EVIDENCE11.18SCM

Records(0 words)


SCM Records containevidence for the six areas

(42 words)

CLAIM8.1a

Softwaredevelopment &

integral processescomply with

CLAIM8.1b

Transition criteriafor software

lifecycleprocesses are

CLAIM8.1c, 8.3Software

conformityreview is


SQA Records containmaterial for three areas

EVIDENCE11.19SQA

Records

CLAIM9.0

Relevantcommunication

&

CLAIM9.1

Means ofcomplianceis agreed to

CLAIM9.2

Compliancesubstantiation

is provided

EVIDENCE11.16

SoftwareConfiguration

Index(0 words)

EVIDENCE11.1

Plan for SWAspects of

Certification

EVIDENCE11.20SW

AccomplishmentSummary


Plan for SWAspects of

Certification

ARGUMENT11.20

Substantiation ofCompliance isprovided by two


satisfactory SCMprocess includes six

elements(37 words)

CLAIM9.0

CertificationLiaison Process isproperly defined


Mapping ISO 14971

• Accompanying amendment is essential for mapping into ASCE

• No structural relation between the document and the assurance case

• Claims, arguments, and evidence identified by analyzing words and phrases

• Very few arguments for evidence• For Each Identified Hazard…

Supports SupportsSupports


SupportsIs evidence for

Supports

Supports

Is a subclaim of

SupportsSupports

Is a subclaim of

Supports

Supports

Supports

Supports

Supports

Supports

Supports

Supports

Is a subclaim of

Is evidence for

Is evidence for

Is evidence for

Is evidence for

Is evidence for

Supports

Supports

Is a subclaim of

Supports

Supports

Is a subclaim of

SupportsSupports

Is evidence for

Is evidence for

Is evidence for

ARGUMENTH.4

Rationale for clause 4,Risk analysis

ARGUMENTH.4.2

Intended use/intendedpurpose and

identification of

ARGUMENTNOTE 2

EVIDENCEAnnex B

Guidance on risk analysisfor in vitro diagnostic

medical devices

ARGUMENTNOTE 3

ARGUMENTNOTE1

EVIDENCEAnnex D

Examples ofpossible

hazards andcontributing

factorsassociated with

medicaldevices

ARGUMENTNOTE 2& H.4.3

CLAIM4.2

Intended use/intendedpurpose and

identification of

ARGUMENTH.4.3

Identification of known orforeseeable hazards

CLAIM4.3

Identification of known orforeseeable hazards

(Step 2)

CLAIM4.3

Foreseeablesequences of eventsthat may result in a

CLAIM4.4

Estimation of therisk(s) for each hazard

(Step 3)

CLAIM4.1

Risk analysisprocedure

ARGUMENTH.4.4

Estimation of therisk(s) for each

hazard

ARGUMENT4.2

Compliance is checked byinspection of the risk

management file.

EVIDENCE3.6

RiskManagement

File

ARGUMENT4.3

Compliance is checkedby inspection of the risk

management file.

ARGUMENTH.4.4

Estimation of therisk(s) for each

hazard

ARGUMENTNOTE 3

EVIDENCEAnnex F

Information on riskanalysis techniques

ARGUMENT4.4

EVIDENCEAnnec C

Guidance on riskanalysis procedure fortoxicological hazards

ARGUMENT4.4

Compliance is checked byinspection of the risk

management file.

EVIDENCE3.6

Risk Management File

EVIDENCE3.6

Risk ManagementFile

CLAIM4

Risk analysis (Steps1, 2 and 3 of Figure

2)

ARGUMENTH.4.1

Risk analysisprocedure

EVIDENCEAnnex A

Questions that can beused to identify medical

device characteristics thatcould impact on safety

ARGUMENTNOTE 1

ARGUMENT4.1

Compliance ischecked by inspection

of the risk

EVIDENCE3.6

Risk ManagementFile

EVIDENCE3.6

RiskManagement File

ARGUMENTNOTE 4


Validating Our Mappings

• Domain experts reviewed our mappings– Common Criteria

• System security experts within MITRE

– DO-178B• Evaluator (FAA Designated Engineering Representative)

– ISO 14971• FDA CDRH

• Varying conclusions from validations


Conclusions and Follow-on


Ada Lovelace


Hypotheses Revisited

• Assurance Standard Assurance Case– There does not seem to be much of a

relationship between the two structures– Experience with actual assurance supports this

• Assurance is Assurance is Assurance– Negation of the above hypothesis prevents us

from coming to any conclusion on this one


Standards Templates

• Mappings might be used as templates– Could be a side benefit of the study– Without structural relation, possibility looks bad

• Advantages of consistency may help drive assurance-requirements standardization– Currently, hard to “compare apples and oranges”– Evaluation of assurance claims easier if

requirements are consistent


Extensions to Tool

• Extend ASCE features to be more helpful

• Make ACSE more generic

• Enhance possibilities for user customization


Shadow a Real Project

• Activities– Document a real process

– Identify where and how to incorporate technique

• Advantages– Learning opportunity for us– Minimal impact on the project– Not in the project’s critical path


Develop Training

• How to use the notation and notation options

• How to develop a structured assurance case

• How changes affect the assurance case– Software, hardware– Operation, environment

• How to write a structured assurance standard


Use on a Real Project

• Apply methdology within a project’s schedule

• Gain experience with maintenance of assurance cases

• Update process with lessons learned

• Propagate this knowledge to other projects


Discussion