1 19/10/2010unclassified c.a. lucas, a. lyons†, w. glover, j.s. cross* *agent oriented software...

119/10/201019/10/2010 UnclassifiedUnclassified

C.A. Lucas*, A. LyonsC.A. Lucas*, A. Lyons†, W. Glover*, J.S. Cross*†, W. Glover*, J.S. Cross**Agent Oriented Software Limited, †Aero Synergy*Agent Oriented Software Limited, †Aero Synergy

C.A. Lucas*, A. LyonsC.A. Lucas*, A. Lyons†, W. Glover*, J.S. Cross*†, W. Glover*, J.S. Cross**Agent Oriented Software Limited, †Aero Synergy*Agent Oriented Software Limited, †Aero Synergy

IET International System Safety Conference 2010IET International System Safety Conference 2010

Incremental Safety AssuranceIncremental Safety Assuranceof anof an

Autonomous Decision-making System Autonomous Decision-making System for UASfor UAS

Incremental Safety AssuranceIncremental Safety Assuranceof anof an

Autonomous Decision-making System Autonomous Decision-making System for UASfor UAS

OutlineOutline

Role of safety-critical software in autonomous UAS

The challenge of clearing incremental changes to this software

Evolving regulatory environment

Modular safety case

Icing case study

Conclusions

Role of safety-critical software in autonomous UAS

The challenge of clearing incremental changes to this software

Evolving regulatory environment

Modular safety case

Icing case study

Conclusions

2

BackgroundBackground

Modern aircraft and engines rely on software for primary air vehicle control, no manual reversion

Consequently FCS or FADEC software is safety critical

Substantial cost and timescale required to introduce software changes, once safety assured

Modifications incur re-certification costs that approach cost of original clearance

Desirable to have a modular software safety assurance process

relating software re-assurance effort to scale of change introduced

Modern aircraft and engines rely on software for primary air vehicle control, no manual reversion

Consequently FCS or FADEC software is safety critical

Substantial cost and timescale required to introduce software changes, once safety assured

Modifications incur re-certification costs that approach cost of original clearance

Desirable to have a modular software safety assurance process

relating software re-assurance effort to scale of change introduced

3

Implications for Autonomous UASImplications for Autonomous UAS

Have decision-making software that replaces many functions performed by human crew of a manned or remotely piloted vehicle

Scope extends beyond simply flying the vehicle to a wide range of other critical roles, such as:

detect and avoid

mission management and re-routing

power management

failure detection, analysis and resolution, to Prognostic Health Management

centre of gravity and fuel management

ATC and communications

ground handling

Have decision-making software that replaces many functions performed by human crew of a manned or remotely piloted vehicle

Scope extends beyond simply flying the vehicle to a wide range of other critical roles, such as:

detect and avoid

mission management and re-routing

power management

failure detection, analysis and resolution, to Prognostic Health Management

centre of gravity and fuel management

ATC and communications

ground handling

4

ChallengeChallenge

Many successful manned A/C recognised for their versatilitye.g., C130, dH Mosquito, Tornado

Breadth of capability results from inherent human versatility:

take on new missions if correctly briefed

train as necessary for the new mission

Autonomous UAS must be endowed with versatile behavioursIf adapted to new or different missions then on-board Autonomous Decision-making System (ADMS) has to be upgraded with necessary additional or revised behaviours

Challenge – how can these be safely introduced without requiring the re-generation of evidence for parts that remain unchanged?

Many successful manned A/C recognised for their versatilitye.g., C130, dH Mosquito, Tornado

Breadth of capability results from inherent human versatility:

take on new missions if correctly briefed

train as necessary for the new mission

Autonomous UAS must be endowed with versatile behavioursIf adapted to new or different missions then on-board Autonomous Decision-making System (ADMS) has to be upgraded with necessary additional or revised behaviours

Challenge – how can these be safely introduced without requiring the re-generation of evidence for parts that remain unchanged?

5

Strategy to achieve incremental approval of autonomous software systems

Start with clearance of the decision-making software engine (ADMS)

Then incrementally adding behaviours and data to reason with

Critical aspects:

Reasoning engine

Behaviours

Data

Relevant programmes:

SUAV(E) & Taranis

ASTRAEA

Ministry of Defence SSEI

Strategy to achieve incremental approval of autonomous software systems

Start with clearance of the decision-making software engine (ADMS)

Then incrementally adding behaviours and data to reason with

Critical aspects:

Reasoning engine

Behaviours

Data

Relevant programmes:

SUAV(E) & Taranis

ASTRAEA

Ministry of Defence SSEI

Approach used and relevanceApproach used and relevance

6

Evolving Regulatory Environment Evolving Regulatory Environment Civil airworthiness regulators starting to address UAS certification

Most UAS are currently operated in combat zones under war-like conditions

Not cleared to any formal set of regulations

Australia’s CASA one of first regulators to introduce civil UAS regulation

CASR, 1998, Part 101, Subpart F specific to UAS

NATO nations recently adopted STANAG 4671

However 4671 has a number of critical limitations when applied to autonomous vehicles:

Introduction: “These requirements may not be sufficient for the certification of UAV Systems with unconventional, novel or extremely complex features”, and further: “….the following areas are not covered by this airworthiness code:

Airspace integration and segregation of aircraft (including “detect and avoid”),

Non-deterministic flight, in the sense that UAV flight profiles are not pre-determined or UAV actions are not predictable to the UAV crew,…”

Civil airworthiness regulators starting to address UAS certification

Most UAS are currently operated in combat zones under war-like conditions

Not cleared to any formal set of regulations

Australia’s CASA one of first regulators to introduce civil UAS regulation

CASR, 1998, Part 101, Subpart F specific to UAS

NATO nations recently adopted STANAG 4671

However 4671 has a number of critical limitations when applied to autonomous vehicles:

Introduction: “These requirements may not be sufficient for the certification of UAV Systems with unconventional, novel or extremely complex features”, and further: “….the following areas are not covered by this airworthiness code:

Airspace integration and segregation of aircraft (including “detect and avoid”),

Non-deterministic flight, in the sense that UAV flight profiles are not pre-determined or UAV actions are not predictable to the UAV crew,…”

7

UK Regulatory InitiativesUK Regulatory Initiatives2008 CAA updated CAP722 “Unmanned Aircraft System Operations in UK Airspace – Guidance”

Chapter on autonomy introduced for the first time in a regulatory document

2009 CAA first released a draft of AMC UAS.1309: “Guidance material for UAS system safety requirements”:

(a) “This Acceptable Means of Compliance (AMC) is designed to set minimum acceptable UAS systems integrity levels in order to protect persons from collisions with UAS.

(b) This AMC is applicable to the Special Conditions defining the system requirements for the collection of systems that perform the functions usually assigned to an aircraft located pilot, in other words the ‘Synthetic Pilot’ (SP)….”

(c) This AMC, a UAS certification document, has been specifically tailored for use with UAS of all sizes …. to ensure the protection of persons in the air and on the ground from the effects of UAS….

(d) This AMC generally follows the ethos of AMC CS/FAR-25.1309.

(e) The certification basis for UAS will be similar to those for manned aircraft however there will be differences related to the absence of an aircraft-located pilot.”

2008 CAA updated CAP722 “Unmanned Aircraft System Operations in UK Airspace – Guidance”

Chapter on autonomy introduced for the first time in a regulatory document

2009 CAA first released a draft of AMC UAS.1309: “Guidance material for UAS system safety requirements”:

(a) “This Acceptable Means of Compliance (AMC) is designed to set minimum acceptable UAS systems integrity levels in order to protect persons from collisions with UAS.

(b) This AMC is applicable to the Special Conditions defining the system requirements for the collection of systems that perform the functions usually assigned to an aircraft located pilot, in other words the ‘Synthetic Pilot’ (SP)….”

(c) This AMC, a UAS certification document, has been specifically tailored for use with UAS of all sizes …. to ensure the protection of persons in the air and on the ground from the effects of UAS….

(d) This AMC generally follows the ethos of AMC CS/FAR-25.1309.

(e) The certification basis for UAS will be similar to those for manned aircraft however there will be differences related to the absence of an aircraft-located pilot.”

8

ADMS Basis of Certification –In-flight Icing

ADMS Basis of Certification –In-flight Icing

Evaluation of requirements for a UAS to fly into known icing conditions chosen as the basis of the study

This case study then used as the vehicle for the study of incrementally certifiable software system

Evaluation of requirements for a UAS to fly into known icing conditions chosen as the basis of the study

This case study then used as the vehicle for the study of incrementally certifiable software system

9

In-Flight IcingIn-Flight Icing

Comprised of two scenarios – baseline and an incremented case

Baseline is UAS not cleared for flight into known icing conditions, ADMS will include reasoning to:

Detect and identify possible icing conditions

Take necessary actions to avoid these conditions

Ensure effect of the icing conditions is minimised

Continue mission if possible

Comprised of two scenarios – baseline and an incremented case

Baseline is UAS not cleared for flight into known icing conditions, ADMS will include reasoning to:

Detect and identify possible icing conditions

Take necessary actions to avoid these conditions

Ensure effect of the icing conditions is minimised

Continue mission if possible

10

In-Flight Icing – Increment In-Flight Icing – Increment Incremented case describes the changes made so that UAS cleared for flight into known icing conditions

Autonomous software will require modification to:

Manage and maintain anti-icing and de-icing mechanisms

Assess the severity of icing conditions and determine if UAS can fly through them or if it should attempt to exit (cf Rex SAAB 340)

Continually monitor ice sensors to determine level of ice accretion

Determine preferred routes both through and out of icing conditions

Process will be developed to ensure that evidence for core autonomous system does not need to be reproduced

Only modified or new modules affected by the changed functionality

Incremented case describes the changes made so that UAS cleared for flight into known icing conditions

Autonomous software will require modification to:

Manage and maintain anti-icing and de-icing mechanisms

Assess the severity of icing conditions and determine if UAS can fly through them or if it should attempt to exit (cf Rex SAAB 340)

Continually monitor ice sensors to determine level of ice accretion

Determine preferred routes both through and out of icing conditions

Process will be developed to ensure that evidence for core autonomous system does not need to be reproduced

Only modified or new modules affected by the changed functionality

11

Software ApproachSoftware Approach

Approach for incremental certification is: Modularisation of safety argumentation, allowing re-use of base modules (e.g. compiler and Run-time Infrastructure (RTI)) for each new system

Classification of incremental changes leading to different, and potentially limited, needs for re-certification of applications whose base version is itself certified

A language is developed to:simplify modularisation of multi-agent systems and individual agents,

provide support for formal analysis of application

Additional tools help evidence collection suitable for incremental certification (non-regression of unchanged functionality)

Approach for incremental certification is: Modularisation of safety argumentation, allowing re-use of base modules (e.g. compiler and Run-time Infrastructure (RTI)) for each new system

Classification of incremental changes leading to different, and potentially limited, needs for re-certification of applications whose base version is itself certified

A language is developed to:simplify modularisation of multi-agent systems and individual agents,

provide support for formal analysis of application

Additional tools help evidence collection suitable for incremental certification (non-regression of unchanged functionality)

12

Example reasoning

plan

Example reasoning

plan

13

Proposed Safety Assurance ProcessProposed Safety Assurance Process

Establish a basis of certification, agreed with the CAA

Draft a Certification Plan, review with CAA

Functional Hazard Assessment

FMECA

Reliability assessment

Safety targets per AMC (UAS).1309

Establish a basis of certification, agreed with the CAA

Draft a Certification Plan, review with CAA

Functional Hazard Assessment

FMECA

Reliability assessment

Safety targets per AMC (UAS).1309

14

ADMS Basis of Certification – How Established?

ADMS Basis of Certification – How Established?

Existing code clauses applicable without alteration

Clauses wholly inapplicable to UAS

Clauses with a valid safety objective but which need to be re-written

Wholly new clauses unique to UAS

Existing code clauses applicable without alteration

Clauses wholly inapplicable to UAS

Clauses with a valid safety objective but which need to be re-written

Wholly new clauses unique to UAS

15

Establish Basis of Safety AssuranceEstablish Basis of Safety Assurance

Source Documents:CS-23

STANAG 4671

EU-OPS

Part Ops

Part FCL

AMC (UAS)1309

Source Documents:CS-23

STANAG 4671

EU-OPS

Part Ops

Part FCL

AMC (UAS)1309

16

Establish Basis of Safety Assurance- Airframe Codes


CS23 – EASA code for manned aeroplanes

Normal, Utility, Aerobatic, Commuter

CS23 – EASA code for manned aeroplanes

Normal, Utility, Aerobatic, Commuter

17



STANAG 4671NATO code for Unmanned Air Systems

Based upon CS23, but 2 issues:

Failure mode severities, no compatibility to manned codes

Ignores any issue that affects a pilot in the manned code

STANAG 4671NATO code for Unmanned Air Systems

Based upon CS23, but 2 issues:

Failure mode severities, no compatibility to manned codes

Ignores any issue that affects a pilot in the manned code

18

UAS Airframe Codes Issues with the STANAG – Example 1


CS23.773 Pilot’s View:Clear and undistorted view

Free from Glare

Protected against fog or frost

STANAG says: Not Applicable

CS23.773 Pilot’s View:Clear and undistorted view

Free from Glare


STANAG says: Not Applicable

19



CS(UAS)23.773 Sensor View:Clear and undistorted view of intended field

Free from Glare


CS(UAS)23.773 Sensor View:Clear and undistorted view of intended field

Free from Glare


20



CS23.775 (h)(i) Windshields and Windows(1) Windshield panes directly in front of the pilot(s) in the normal conduct of their duties, and the supporting structures for these panes must withstand, without penetration, the impact of a 0∙91 kg (2 lb) bird when the velocity of the aeroplane relative to the bird along the aeroplane’s flight path is equal to the aeroplane’s maximum approach flap speed.

CS23.775 (h)(i) Windshields and Windows(1) Windshield panes directly in front of the pilot(s) in the normal conduct of their duties, and the supporting structures for these panes must withstand, without penetration, the impact of a 0∙91 kg (2 lb) bird when the velocity of the aeroplane relative to the bird along the aeroplane’s flight path is equal to the aeroplane’s maximum approach flap speed.

21



• USAR 775 says Not applicable

– CS (UAS) 23.775 Clause devised by Aerosynergy:

– (a) Forward-facing structure, covers and lenses, which directly protect systems and equipment essential for detect and avoid and flight control functions must withstand, without penetration, the impact of a 0∙91 kg (2 lb) bird when the velocity of the aeroplane relative to the bird along the aeroplane’s flight path is equal to the aeroplane’s maximum approach flap speed.

• USAR 775 says Not applicable

– CS (UAS) 23.775 Clause devised by Aerosynergy:

– (a) Forward-facing structure, covers and lenses, which directly protect systems and equipment essential for detect and avoid and flight control functions must withstand, without penetration, the impact of a 0∙91 kg (2 lb) bird when the velocity of the aeroplane relative to the bird along the aeroplane’s flight path is equal to the aeroplane’s maximum approach flap speed.

22

What is the Problem with the Existing Codes? - Example

What is the Problem with the Existing Codes? - Example

• AMC 25.1309 7 (a) 5 Classification of Severity of failure mode:

• Catastrophic: Failure Conditions, which would result in multiple fatalities, usually with the loss of the aeroplane. (Note: A “Catastrophic” Failure Condition was defined in previous versions of the rule and the advisory material as a Failure Condition which would prevent continued safe flight and landing.)

• AMC 25.1309 7 (a) 5 Classification of Severity of failure mode:

• Catastrophic: Failure Conditions, which would result in multiple fatalities, usually with the loss of the aeroplane. (Note: A “Catastrophic” Failure Condition was defined in previous versions of the rule and the advisory material as a Failure Condition which would prevent continued safe flight and landing.)

23

HenceHence

• Discussions with CAA

• Regulatory discussions at the JARUS forum, NAAs plus FAA

• Led to:

– CAA document AMC UAS 1309

• Discussions with CAA

• Regulatory discussions at the JARUS forum, NAAs plus FAA

• Led to:

– CAA document AMC UAS 1309

24

AMC UAS 1309 Failure Mode Severities Example:

AMC UAS 1309 Failure Mode Severities Example:

Catastrophic

Failure Conditions that could result in multiple fatalities

Catastrophic

Failure Conditions that could result in multiple fatalities

25

AMC UAS 1309 Main PointsFailure Mode Severities vs. Probability:

AMC UAS 1309 Main PointsFailure Mode Severities vs. Probability:

26

Classification of failure conditions

No safety Effect

Minor Major Hazardous Catastrophic

Allowable qualitative probability

No Probability Requirement

Probable Remote Extremely Remote

Extremely Improbable

Allowable Quantitative probability per Flight Hour on the order of.

No Probability Requirement

<10-3

Note 1<10-5 <10-7 <10-9

Non Detect and Avoid Equipped. Detect and Avoid Equipped. Note 2

Software/CEH Dev Level D

Software/CEH Dev Level C

Software/CEH Dev Level B

Software/CEH Dev Level A

AMC UAS 1309 Main Points:With D & A; Integrity of Airframe and Complex

Flight Mgt System:

AMC UAS 1309 Main Points:With D & A; Integrity of Airframe and Complex

Flight Mgt System:

• AMC UAS 1309 Main Points:• With D & A; Integrity of Airframe and Complex Flight Mgt System:• AMC UAS 1309 Main Points:• With D & A; Integrity of Airframe and Complex Flight Mgt System:

27

Aircraft Category

CFMS Integrity including DA

requirement (Fig 1.0)

Airframe Integrity (analogous CS/FAR

to UAS code) requirement similar

to

Integrity delta

CS/FAR-25 AMC UAS.1309 (10-9) CS/FAR-25.1309 (10-9) No CFMS/ Airframe systems integrity delta

CS/FAR-23 AMC UAS.1309 (10-9) AC-23.1309-1D Increasing CFMS/ airframe systems integrity delta dependent on class

CS/FAR-23 AMC UAS.1309 (10-7) Mitigated case

AC-23.1309-1D Only allowed if mitigation is accepted by the authorities

CS/FAR-29 AMC UAS.1309 (10-9) CS/FAR-29.1309 Medium CFMS/ airframe systems integrity delta

CS/FAR-27 AMC UAS.1309 (10-9) CS/FAR-27.1309 Large CFMS / airframe systems integrity delta

CS/FAR-27 AMC UAS.1309 (10-7) Mitigated case

CS/FAR-27.1309 Only allowed if mitigation is accepted by the authorities

CS-VLA / VLR and others

AMC UAS.1309 (10-9) CS-VLA / VLR.1309 Very large CFMS / airframe systems integrity delta

CS-VLA / VLR and others

AMC UAS.1309 (10-7) Mitigated case

CS-VLA / VLR.1309 Only allowed if mitigation is accepted by the authorities

Other Codes:Other Codes:

• EU-OPS Part Ops

– Operational requirements

• Part FCL

– Looking at requirements for Flight Crew Licensing to understand what the human pilot is expected to do

• EU-OPS Part Ops

– Operational requirements

• Part FCL

– Looking at requirements for Flight Crew Licensing to understand what the human pilot is expected to do

28

UAS Airframe Codes Issues with the STANAG - Example 3

UAS Airframe Codes Issues with the STANAG - Example 3

• USAR1772 Part time data

• USAR1725 Powerplant data etc..

• Puts a lot of emphasis on the UAS Pilot, and this is a problem because:

– Estimated Reliability of the data link is 10-3

– Not sufficient for critical functions

• USAR1772 Part time data

• USAR1725 Powerplant data etc..

• Puts a lot of emphasis on the UAS Pilot, and this is a problem because:

– Estimated Reliability of the data link is 10-3

– Not sufficient for critical functions

29

Firescout IncidentFirescout Incident

30

“The MQ-8 Fire Scout experienced lost link and proceeded 23 miles north/northwest out of Pax River, still about 40 miles south of DC area in

northern St. Mary's county, Maryland, into National Capital Region airspace,” he said. “The operator team shifted to other Ground Control

Station, restoring link and successfully commanding vehicle to recover at Webster Field. The aircraft returned to Webster Field safely without

injuries, and without damage to the aircraft or vessel.”

According to Capt. Tim Dunigan, Fire Scout According to Capt. Tim Dunigan, Fire Scout program manager, the helicopter was at 1,700 feet program manager, the helicopter was at 1,700 feet

AGL 75 minutes into a test flight from its base, AGL 75 minutes into a test flight from its base, Naval Air Station Pax River in Southern Maryland, Naval Air Station Pax River in Southern Maryland, when operators temporarily lost communications when operators temporarily lost communications

with the unmanned rotary aircraft.with the unmanned rotary aircraft.

Firescout IncidentFirescout Incident

• He added that the MQ-8B Fire Scout has flown more than 1,000 flight hours since December 2006. i.e. 10-3

• Conclusions:

– The integrity of the UAS must be on board the UAV

– The relevance of the pilot is limited

– Autonomous logic is necessary...

– ...including ‘interrupts’ to prevent pilot error

• He added that the MQ-8B Fire Scout has flown more than 1,000 flight hours since December 2006. i.e. 10-3

• Conclusions:

– The integrity of the UAS must be on board the UAV

– The relevance of the pilot is limited

– Autonomous logic is necessary...

– ...including ‘interrupts’ to prevent pilot error

31

• Firescout incident highlights the risks

• This paper proposes a means for dealing with autonomous UAS regulation and ADMS safety assurance

• Cross fertilisation of results from SSEI Programme to ASTRAEA, with opportunity to flight test ADMS behaviours on Cranfield A/C

• Flow through to civil and military UAS programmes worldwide

• Civil regulators, CAA and FAA, developing regulations with industry cooperation

• What will MoD and the MAA do?

• Firescout incident highlights the risks

• This paper proposes a means for dealing with autonomous UAS regulation and ADMS safety assurance

• Cross fertilisation of results from SSEI Programme to ASTRAEA, with opportunity to flight test ADMS behaviours on Cranfield A/C

• Flow through to civil and military UAS programmes worldwide

• Civil regulators, CAA and FAA, developing regulations with industry cooperation

• What will MoD and the MAA do?

ConcludingConcluding

32

1 19/10/2010unclassified c.a. lucas*, a. lyons†, w. glover*, j.s. cross* *agent oriented software...

Documents

1 19/10/2010unclassified c.a. lucas, a. lyons†, w. glover, j.s. cross* *agent oriented software...