Flight Critical Systems Software Certification Initiative

A Presentation to SAE Aerospace Control and Guidance Systems Committee, Meeting 95

2 March 05Salt Lake City, UT

Mr. David Homan, Technical Area LeaderControl Systems Development Branch

Air Vehicles DirectorateAir Force Research Laboratory

Telephone: (937) 255-4026David.homan@wpafb.af.mil

Outline

• Scope of the the Flight Critical V&V Problem

• Flight Critical System Software Initiative (FCSSI):• Philosophy• Strategy• Programmatics

Background: Flight Safety and Manned/Unmanned Functional Migration

Flight CriticalMission Critical

Manned Aircraft

Unmanned Aircraft

Flight Mgmt

Vehicle Mgmt

Mission Mgmt

Mission Mgmt

Vehicle MgmtOn-boardOff-board

On-boardOff-board

Pilot is Integrator andContingency Manager; FMS is mostly advisory.

Flight Mgmt

FMS and VMS provide

Integration andContingency

Mgmt; Operator

manages at high-level.

Situational awareness

Situational awareness?

For UAVs, “

Pilot F

unction” b

ecomes

huge design and V&V issue

Background: V&V Requirements

Flight CriticalMission Critical

System Focus is Performance/Security

Performance Metric: Throughput and Bandwidth [event driven]

Assurance Metric: Probability of Mission Success [Simplex or Back-up]

Confidence Rqmt: Performance and security are validated.

Consequence of Failure: Potential mission failure

System Focus is Performance/Assurance

Performance Metric: Sampling Rate and Latency [time triggered]

Assurance Metric: Probability of Loss of Control and N x Fail Op/Fail Safe [Triplex or Quad]

Confidence Rqmt: Performance and Assurance must be validated; [Failure Modes and Effects Testing]

Consequence of Failure: Loss of Aircraft, potential loss of life

Rule of Thumb: When you mix mission with flight criticality , the testing is held to most stringent

requirement.

Consequence of Failure: Loss of Aircraft, potential loss of life

Developmental Timeline:Flight Critical ready by First Flight!Any changes requires Total Re-test!

Flight C

ritical V

&V isn’t j

ust a softw

are issue,

it’s a system is

sue!!

Failure Modes and Effects Testing

Flight/Safety Critical System Attributes

Performance

Performance•Capability to perform the function•Quantitative Metrics: throughput, bandwidth, latency etc

Assurance

Assurance•Capability to sustain performance during mission•Quantitative Metrics: PLOC, Fail-Op rqmt, etc

Confidence

Confidence•Capability to know that performance and assurance will execute correctly during mission.•Qualitative Metric: collection of evidence that shows that the system is operating correctly

Performance

Assurance

Confidence

It’s not enough to design a flight critical system; there

must be proof that it works…

Confidence must be integrated into the system design to reduce the need for testing!

Confidence needs some quantitativeness!

If FCV&V isn’t hard enough…

New Capabilities Present New Challenges (Complexities) to V&V problem.

• Mixed Criticality Architecture: Non-obtrusive co-existence of mixed criticality

• Adaptive/Learning/Multi-Modal Functions: Indeterminate or untraceable

functionality

• Mixed Initiative/Authority Mgmt: Human/autonomy or autonomy/autonomy

interactions

• Multi-Entity Systems: Functions that encompass multiple platforms.

• Sensor Fusion/Integration: Highly confident sensor-derived information

Are these new systems/capabilities affordably provable?$ $

Mixed Criticality Challenge

How can we separate the mission and flight critical functionality as to guarantee safety?

SOA: Middleware that provides time/space partitioning (ARINC 653).

Issue:

Both Criticalities use common HW resources (i.e. processors, backplanes, busses etc); how do we determine PLOC and fault tolerance?

• Understand failure mechanisms for partitioning

• Non-critical function must not take out shared resources…Or the probability of its occurrence is predictable…

• Need guarantee on fault tolerance

A

A

A

B

B

C

ba

ckp

lan

es

Se

rial b

us

Processors

X

XX

Answer may reside in a SW/HW architecture specifically designed for mixed operation

Adaptive/Learning/Multimodal Challenge

Delta CATA

Delta A+B+C

Delta Z Dot

Delta Y dot

Delta X Dot

Delta Z

Delta Y

Delta X

Maintain a Minimum Distance

Move Towards Assigned Position

Align Flight Vector

Input Layer 1st Hidden Layer

2nd Hidden Layer

Output Layer

How can we trust functionality that we may not be able to fully test?SOA: We must try to test the complete functional envelope (till $$ runs out…)!Issue: Some new Control capabilities are untraceable and/or non-deterministic

• Adaptive systems • Huge test space• Perfect Input data

• Learning systems• Environmental stimuli• Lost memory

• Multi-modal systems• Mode transition stability• Mode synchronization• Recovery mode?

Answer may reside in bounding the function in run-time to known safe behavior.

Mixed Initiative Challenge

How can man and autonomy safely interact?

SOA: Human operator always get authority!

Issue:

Human operator may not have all the information or be able to comprehend situation in real-time:

• Situational Awareness versus Response Time

• Assessment of UAV mode/state/health

• Assessment of surrounding environment

• “Consequence of mishap” is a factor • Complete system health is a factor• Workload is a factor

Answer may reside in a authority management specification that would allow the correct party to have decision authority.

AF Poster Child:

Auto-Aerial Refueling (AAR)

Multi-Entity Challenge

How can trust systems with multiple players to safely perform cooperative functions?

SOA: Keep humans away and hope for the best…

Issue:

Entities participating in the coordinated function may not be part of individual V&V testing:

• Linked Interface Control Documents?

• Entities with different manufacturers?

• System Configuration Management?

• Mission-specific programming?

Answer may reside in a specification for contingency management, based on system degradation

High Confidence Sensing Challenge

How can we trust visual/radar systems for flight critical functions?

SOA: Brute force and analytic redundancy

Issue:

Mission-style sensors don’t have acceptable real-time methods for FDIR…

• Sensors will likely be multi-function!

• Redundant HW may not be answer,

redundant information?

• Built-in-test may not provide good real-time

coverage.

• Reliable signal processing/sensor fusion

software

Answer may reside in sensor designs that compensate for sensor degradation and plan for contingencies

Flight Critical Systems Software Initiative

• Understand the Problem• V&V for Intelligent Adaptive Control Systems

• Develop a research agenda• Establish Confidence as a research discipline

• DoD and National participation

• We have to define an Evolutionary Process• R&D Component

• S&T research feeds

V&V/Certification will never go away! Let’s plan for it!

DOD & FWV LevelWorkshop (R&D Focus)

Near-Term Program

Continuing S&T

Investment

5 yr

Continuing CRAD/IRAD R&D Investment

Team A

Team B

Near-Term Program

• • • Team N

Invite “Big Three” Airframers to discuss their ideas/approaches

Boeing

Lockheed Martin

Northrop Grumman

Develop 5 year Kick-Start R&D Program

Multiple Awards (Cooperative Agreement)

Collaborative TeamsGovt

Airframer

Vendor/Suppliers

Academia

Programmatic Strategy

This is not a “One-Shot Wonder! This feeds an evolutionary process with continuing S&T research!

DoD Participants: AFRL/AFOSR ASC DARPA NAVAIR/ONR Army AATD

Workshop Product

New Requirement New Requirement

DoD buys big weapons system from these folks…

Common Process with Proprietary Implementations

National Level Workshop (S&T Focus)

National Attention for a Nationally Crucial Issue!Rallying S&T Community to Certification Cause!

DARPA

FAA NSF

NASA

Workshop Status:

Planning for NOV/DEC 05 timeframe; DC location

Planning Meeting in August 05

“Research Needs for Flight/Safety Critical Systems”

Workshop Product

High Confidence Software and SystemsCoordinating Group

National Coordination Office for Information Tech R&D

Flight Critical System Software InitiativeCapability Focused Tech Investment

War Winning Upgrades for

Today’s Platforms

War Winning Upgrades for

Today’s Platforms

Superior Technologies for Future Aerospace

Dominance

Superior Technologies for Future Aerospace

Dominance

Flight Critical System Software Initiative

To be effective assets in the force structure and mission plans, UAV’s must …

•Be Safe & Reliable

•Be Responsive & Effective

•Be Interoperable

•Not Adversely Effect Operations Capability

CAO Background

Flight Critical System Software InitiativeCAO Technology Goals

• Mixed Manned/Unmanned Teams

• UAV In-situ Decision Making

• Transparent Airspace Ops

• Adaptive Software V&V

• Reliable Unmanned Ops

Same Base, Same Time, Same Tempo

Same Base, Same Time, Same Tempo

Based on JUCAS ICD, SAB Summer Study, Global Hawk ORD, OSD UAV Roadmap, Predator CCD

Flight Critical System Software InitiativeCooperative Airspace Ops CFTI Taxonomy

Cooperative Airspace

Ops

Operations inOperations inmanned/unmanned manned/unmanned teamsteams

Attributes Product

Safe operations Safe operations from airbases and from airbases and in airspacein airspace

J-UCAS 4-ship flight management

Multi-UAV distributed control

V&V Of flight critical intelligent software

Terminal area & ground ops

Health mgmt integ w/adaptive control

Open architecture, highly reliable VMS

HALE UAV detect and avoid

Non-GPS nav, landing, and ground ops

Multi-vehicle see and avoid

Other VA Capability’s Products

Capability’s Products

Other Org’s Products

Future Capability

Flight Critical System Software InitiativeVVIACS Objectives

Enabling Technology for Certification of Emerging Intelligent & Adaptive Vehicle/Mission Management Systems

• Establishes Emerging Control Systems and Associate Emerging Fundamental Properties

• Identifies Dominant Certification Drivers for ECS and EFP

• Develops Certification Metrics and R&D Critical Paths

Flight Critical System Software Initiative

Emerging Control System Impact by Functional Disciplines on Development and V&V Costs

0%

50%

100%

150%

200%

250%

SY

S

S&

C

CL

AW

SW

SIM

TT

D

TE

ST

HW

PA

HW

OT

HE

R

Functional Disciplines

% In

crea

se in

Co

st

Single-vehicle Development

Single-vehicle V&V

Multi-vehicle Development

Multi-vehicle V&V

Impact Analysis Results

Significant Cost Increase Projected Primarily Due to V&V SW and Test

• Single-Vehicle ECS Increase V&V Costs ~2X• Multiple-Vehicle ECS Increase V&V Costs ~3X• Software: Single-Vehicle 100% Increase, Multiple-Vehicle 200% Increase• Test: Single-Vehicle 150% Increase, Multiple-Vehicle 250% Increase

V&V Costs54% - Baseline62% - Single-Vehicle68% - Multiple-Vehicle

Flight Critical System Software Initiative

TASS SBIR Efforts

Barron Associates –

Run-time monitor to check system behavior with fail-safe controller for recovery – demonstration based on UAV flight control.

EDAptive –

Specification and Requirements languages along with “formal methods” to streamline V&V process.

Scientific Monitoring –

Model-based software development used to generate and test flight critical code.

WW Technology –

create an embedded “fault detector” and formal transformation of control system to fail-safe control system, includes partitioning using a middleware approach.

Flight Critical System Software InitiativeCertification Techniques for Advanced Flight

Control Systems – CerTA FCS

CerTA FCS Unique R&D push for Technology Breakthrough in Systems Certification

- Certification Process Paradigms- V&V Methods/Techniques Innovations

Reduced Life Cycle Costs

Higher Assurance Levels for Advanced Flight Critical Software

Enabling TechnologyIntelligent, Adaptive, ReconfigurableReal time Prognostics Health InfoAutonomous OperationsMulti-Vehicle Coordination

Cooperative Airspace Ops

Operations inOperations inmanned/unmanned teamsmanned/unmanned teams

V&V Of Intelligent Software TAD: 11

VA Initia

l Contrib

ution Towards F

light

Critica

l Sys

tems Softw

are Initia

tive

Flight Critical System Software Initiative

VVIACS SBIR

DARPA PCES, MoBIES, SEC

Flight Critical Systems Certification Initiative

Technologies for Affordable & Safe Software Development (TASS) 4 SBIR Ph I

V&V of Intell & Adapt Control Systems (VVIACS) 6.2

Projected TASS SBIR Ph II

Current Investment Roadmap to FY10

FY04 FY05 FY06 FY07 FY08 FY09Prior FY03 FY10

Runtime Monitors

Specifications & Rqmts s/ww/ formal methods

VA Buy Plan

VA Executing

Scientific Monitoring

Multi-Agency Executing

Barron

EDAptive

Model-Base s/w Development

W/W Tech Middleware Fault Detection& Isolation s/w Tech

Certification Techniques for Advanced FCS - CerTAFCS

Integrated S/W Environment

V&V for Distributed Embedded Systems (AFOSR MURI)

Planned Execution

Flight Critical System Software InitiativeSummary

• Innovations Required in Systems Certification to Enable Future Functionality of UAVs in CAO Environment

• VA has Established Technology Investment Area and VVIACS Study to Determine Near-Term R&D Investments

• SBIR Program Provides Tech Seedlings

• VA Investments Geared to Support Collaboration

• Control S&T Community “Buy In” Required NOW for Affordable and Safe Certification Practices for TOMORROW