industrial practice on mixed-criticality engineering and certification in the aerospace industry...
TRANSCRIPT
Industrial practice on mixed-criticality engineering and certification in the aerospace industry
2013-03-22DATE 2013 / WICERT 2013
6 DATE 2013 / WICERT 2013
Who defines the requirements
• Customer- Typically OEM – formal and … “less formal” definition
• Civil Aviation Authority- FAA, EASA, etc., through Technical Standard Orders
(TSO, ETSO)
• Internal standards- Body performing the development
! Functional, non-functional, and process requirements
7 DATE 2013 / WICERT 2013
Who checks whether requirements are met
• Customer- Typically OEM – Reviews, Validation, Integration testing
• Civil Aviation Authority- FAA, EASA, etc. - Through Certification
• Internal standards- Body performing the development – As defined by operating procedures. Typically aligned with aerospace
development processes.
8 DATE 2013 / WICERT 2013
How is certification done
• Civil Aviation Authority does not only check the requirements testing checklist.
• CAA primarily controls the Development process and checks the evidence that it was followed- Typically, an agreement must be reached with the CAA on
acceptable means of compliance/certification. Typically, this includes: Meeting DO-254 / ED-80 for electronics Meeting DO-178B / ED-12B for software (or newly DO-178C) Meeting agreed CRI – Certification Review Item
- The following items are agreed: The means of compliance The certification team involvement in the compliance determination
process The need for test witnessing by CAA Significant decisions affecting the result of the certification process
9 DATE 2013 / WICERT 2013
Why not having one process?
• CS-22 (Sailplanes and Powered Sailplanes)• CS-23 (Normal, Utility, Aerobatic and Commuter Aeroplanes) – <5.6t,<9pax or twin-prop
<9t,<19pax
• CS-25 (Large Aeroplanes)• CS-27 (Small Rotorcraft)• CS-29 (Large Rotorcraft)• CS-31GB (Gas Balloons)• CS-31HB (Hot Air Balloons)• CS-34 (Aircraft Engine Emissions and Fuel Venting)• CS-36 (Aircraft Noise)• CS-APU (Auxiliary Power Units)• CS-AWO (All Weather Operations)• CS-E (Engines)• CS-FSTD(A) (Aeroplane Flight Simulation Training Devices)• CS-FSTD(H) (Helicopter Flight Simulation Training Devices)• CS-LSA (Light Sport Aeroplanes)• CS-P (Propellers)• CS-VLA (Very Light Aeroplanes)• CS-VLR (Very Light Rotorcraft)• AMC-20 (General Acceptable Means of Compliance for Airworthiness of Products, Parts
and Appliances)
10 DATE 2013 / WICERT 2013
Small vs. large aircraft (CS23 vs. CS25)
>10-5 10-5-10-9 <10-9
>10-5
Can happen to any aircraft
10-5-10-9
Happens to some aircraft in fleet
<10-9 Never happens
Failure examples Probability Airliner
DALAirliner
Probability for 2-seater
DAL2-seat
Reduced functions <10-5 D <10-3 D,D
Injuries <10-8 A <10-4 C,D
Damage to aircraft <10-6 B <10-5 C,D
Crash <10-9 A <10-6 C,C
Table modified from FAA AC 23.1309-1D Figure from FAA AC 25.1309-1A
11 DATE 2013 / WICERT 2013
Software-related standards – DO-178B(C)
• Development Assurance Level for software (A-D)- Similar definition also exists for Item, Function …
• Specifies - Minimum set of design assurance steps
- Requirements for the development process
- Documents required for certification
- Specific Objectives to be proven
LevelApprox. failure
categoryObjectives
(178B)178CObjectives met with
independent review (178C)
A Catastrophic (66)71 33
B Hazardous (65)69 21
C Major (57)62 8
D Minor (28)26 5
E No Safety Effect 0 0
12 DATE 2013 / WICERT 2013
More than one function
• Two situations:- Single-purpose device (e.g. NAV system, FADEC – Full Authority
Digital Engine Controller, etc.) Typically developed and certified to a single assurance level Exceptions exist
- Multi-purpose device (e.g. IMA – Integrated Modular Avionics)
13 DATE 2013 / WICERT 2013
Mixed-criticality
• Same HW runs SW of mixed criticality- For SW, more or less equals to mixed DAL
• Implicates additional requirements- Aircraft class specific
- CS-25: Very strict Time and space partitioning (e.g. ARINC 653) Hard-real-time execution determinism (Problem with multicores) Standards for inter-partition communication
- CS-23: Less strict Means to achieve safety are negotiable with certification body
14 DATE 2013 / WICERT 2013
Design assurance, verification, certification
• Design assurance + qualification does it work?- Functional verification
- Task schedulability
- Worst-case execution time analysis
- … and includes any of the verification below
• Verification does it work as specified?- Verifies requirements
Depending on the aircraft class and agreed certification baseline, requirements might include response time or other definition of meeting the temporal requirements
• On singlecore platforms, this is composable – isolation is guaranteed• On multicore platforms – currently in negotiations
(FAA, EASA, Industry, Academia, …) Requirements are typically verified by testing
• Certification is it approved as safe?- Showing the evidence of all the above
15 DATE 2013 / WICERT 2013
Software verification means (1)
• Dynamic verification- Testing – based on test cases prepared during the development
cycle. Integral part of the development phase. Unit testing – Is the implementation correct? Integration testing – Do the units work together as anticipated? System testing – Does the system perform according to (functional
and non-functional) requirements? Acceptance testing – As black-box - does the system meet user
requirements?
- Runtime analysis – Memory usage, race detection, profiling, assertions, contract checking … and sometimes (pseudo)WCET
• Domain verification – not with respect to requirements, but e.g. checking for contradictions in requirements, syntax consistency, memory leaks
16 DATE 2013 / WICERT 2013
Software verification means (2)
• Static analysis – type checking, code coverage, range checking, etc.
• Symbolic execution• Formal verification – formal definition of correct
behavior, formal proof, computer-assisted theorem proving, model checking, equivalence checking- Not much used today … just yet
17 DATE 2013 / WICERT 2013
Verification for certification
• Typical approach is to heavily rely on testing, supported by static analysis and informal verification
• DO-178C (recent update of DO-178B) defines requirements for using- Qualified tools for Verification and Development – DO-330
- Model-based development and verification – DO-331
- Object-oriented technologies – DO-332 (that’s right, 2012)
- Formal methods – DO-333 To complement (not replace) testing
• Tool qualification- Very expensive
Tools that can insert error: basically follow the same process as SW development of same DAL. Usable only for small tools for specific purposes.
Tools that replace testing: for A,B: developed and tested using similar development process. For C,D: subset needed. Can use COTS tools.
18 DATE 2013 / WICERT 2013
Near future
• Replacement of (some) testing by qualified verification tools Towards formal verification
• Adoption of DO-178C instead of DO-178B• Definition of technical and process requirements for
multicore platforms• Growth of model-based development and verification• Enablers for incremental certification (which assumes
composability of safety)