modelling of complex distributed test scenarios · 2016-10-27 · modelling of complex distributed...

20
Budapest, 26-28 October 2016 MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS Presented by Dr. Karl Ambrus & Andreas Weitl © All rights reserved

Upload: others

Post on 23-Mar-2020

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS · 2016-10-27 · MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS ... • Functional tests to ensure system operation under all use

Budapest, 26-28 October 2016

MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOSPresented by Dr. Karl Ambrus & Andreas Weitl 

© All rights reserved

Page 2: MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS · 2016-10-27 · MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS ... • Functional tests to ensure system operation under all use

CONTENT

2

• Introduction• Objectives• Virtual Integration Platform• Use Cases & Test Scenarios• Model Based Test Generation• Conclusion & Outlook

©All rights reserved

Page 3: MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS · 2016-10-27 · MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS ... • Functional tests to ensure system operation under all use

AIRBUS DEFENCE & SPACE

3 © All rights reserved

Manching

Page 4: MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS · 2016-10-27 · MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS ... • Functional tests to ensure system operation under all use

Introduction: Integration of Aircraft Systems

4

Integration of A/C Systems• Aircraft are long living products (40 years) • A lot of upgrade programs, enhancements & national adaptions during their lifetime• Growth of functionality & complexity of the aircraft subsystems• Increase and diversification of A/C variants during lifetimes• New agile development methods

Integration Environment• Complex, rare and expensive test facilities & aircraft components• Compatibility of the integration environment for all integration levels & methods• Reuse of test procedures during system lifetime (regression testing)• Create synergies between different programs & testing methods• Long Term Obsolescence handling for complex

Test environments Test procedures

© All rights reserved

Page 5: MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS · 2016-10-27 · MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS ... • Functional tests to ensure system operation under all use

Introduction: Test Levels and Test Strategies

5

Levels of system integration testing:• System specification & design model testing• S/W and unit testing• Component & subsystem testing• System testing• Virtual system integration• System integration• System qualification & certification

Test & Integration strategies:• Test & analysis of system models (design, specification, S/W)• Development tests using special development tools (agile development)• Functional tests to ensure system operation under all use case conditions• Formal qualification tests for requirement verification • Automated regression tests to ensure system functionality

© All rights reserved

Page 6: MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS · 2016-10-27 · MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS ... • Functional tests to ensure system operation under all use

Introducton: AIDASS® Test System Familyfor all steps of the Product Life Cycle

6 © All rights reserved

Page 7: MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS · 2016-10-27 · MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS ... • Functional tests to ensure system operation under all use

Introduction: TORNADO System Integration Rig

7 © All rights reserved

Page 8: MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS · 2016-10-27 · MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS ... • Functional tests to ensure system operation under all use

Introduction: Eurofighter System Integration Facility

8 © All rights reserved

Page 9: MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS · 2016-10-27 · MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS ... • Functional tests to ensure system operation under all use

Objectives for System Test & Integration

9 © All rights reserved

Reduce overall project cost for

system development, system integration and system operation

by improving product quality

It is not the primary goal to reduce the cost for system integration & test,but to increase test coverage and quality of testingto find more problems in early integration phases. 

Page 10: MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS · 2016-10-27 · MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS ... • Functional tests to ensure system operation under all use

Objectives for System Test & Integration

10 © All rights reserved

• Increase availability and reduce cost of test environments• Offline (desktop) test preparation & test execution • Agile automatic development testing• Distributed System Integration Remote operation of Test Benches & Test Tools Distributed real‐time testing of real and virtual systems Standards for interoperability & communication (Commands, Data, Time, Files, OSLC …) 

• Virtual System Integration Integrate virtual equipment on virtual integration facilities Portability of test procedures between real and virtual test environment Service Based Testing on distributed hybrid computing infrastructure

• Model Based System Integration Test variant management Modelling of Test Scenarios Automatic Test Case generation Tracing of requirements

Page 11: MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS · 2016-10-27 · MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS ... • Functional tests to ensure system operation under all use

STEVE “Virtual Integration Platform – Next Generation”Modular Distributed Test Environment & Communication Layer

11 © All rights reserved

STEVE VIP-NG Communication Layer Definition

STEVE VIP-NG Modular Concept

Distributed Test Environment

STEVE “System-Technik und Virtuelle Erprobung”VIP-NG “Virtual Integration Platform – Next Generation”

Virtual & Hybrid Integration Benches Distributed System Integration Portability of Test Procedures Enhance availability of Test Environments Replacement of obsolete Modules

Page 12: MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS · 2016-10-27 · MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS ... • Functional tests to ensure system operation under all use

Airbus D&S ‐ Virtual Integration PlatformService based system Architecture

12 © All rights reserved

AIDASS System Integration Environment & Test ToolsModules & Objects (Services) Desktop, Virtual & Target Test Environment (AIDASS) Simulation Environment (SIRIUS) Test Tools

Communication Local, LAN, WAN

Computing Host (H/W, OS) Desktop PC (Windows) & Servers Test PC (Linux, Windows) I/O Front Ends  Test Bench Modules  Desktop, Virtual, Target & hybrid 

configurations

System under Test Real & simulated equipment

Page 13: MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS · 2016-10-27 · MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS ... • Functional tests to ensure system operation under all use

Airbus D&S ‐ Virtual Integration PlatformVIP Demonstrator: AIDASS Environments & Computing Clusters

13 © All rights reserved

Desktop Integration Environmentshall run on any Windows - PC

Virtual Integration Environment

Target Integration Environment

VIP Network Clusters

Page 14: MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS · 2016-10-27 · MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS ... • Functional tests to ensure system operation under all use

Airbus D&S ‐ Virtual Integration PlatformVIP Demonstrator: Modules & Objects

14 © All rights reserved

Distributed Test Services ( Network Cluster, Modules, Objects, H/W )

Page 15: MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS · 2016-10-27 · MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS ... • Functional tests to ensure system operation under all use

15 © All rights reserved

AIDASS Module ControllerGateway for Commands & States

Unique commands & statesfor VIP Modules & Objects

Airbus D&S ‐ Virtual Integration PlatformVIP Demonstrator: AIDASS Module Controller

Local Status Monitor

Page 16: MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS · 2016-10-27 · MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS ... • Functional tests to ensure system operation under all use

Use Cases & Test ScenariosShow portability of Test Procedures

16 © All rights reserved

• Automated Generation of Test Scenarios Automated model based generation Test Cases & Test Scripts EA, MBTsuite, AIDASS Desktop

• Functional I/O Test: Test I/O interfaces and Test Rig functions AIDASS ( Desktop, Virtual, Target & Hybrid ) Test Rig ( Desktop & Target ) 

• Command Test Test of XML‐RPC Command interfaces Local & remote operation

• Quality of Service Test Commands (XML‐RPC) & Data (DDS)

• VIP Demonstration Test Show operation of VIP Test Environments Virtual Flight Test ( AC simulation & Displays )

Real & virtual data I/ODistributed test environment (Computing Clusters )

Image recording (VSS) & automated offline analysis (ADEO)

Test Variant Definition

Test Variation Parameters

Page 17: MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS · 2016-10-27 · MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS ... • Functional tests to ensure system operation under all use

Model Based Test GenerationDesign Test Scenarios (Test Model)

17 © All rights reserved

Define Test Variation Parameters Modelling  of Test SceanriosDefine Test Variants

Define Test Variant Rules

Page 18: MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS · 2016-10-27 · MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS ... • Functional tests to ensure system operation under all use

Model Based Test GenerationDesign Test Scenarios (Test Actions)

18 © All rights reserved

Modelling  of Test Actions Action Libraries for Test Model Development

Page 19: MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS · 2016-10-27 · MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS ... • Functional tests to ensure system operation under all use

Model Based Test GenerationTest Case & Test Script Generation (MBTsuite)

19 © All rights reserved

Page 20: MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS · 2016-10-27 · MODELLING OF COMPLEX DISTRIBUTED TEST SCENARIOS ... • Functional tests to ensure system operation under all use

Conclusion & Outlook

20

• Integration of Modules, AIDASS Environments & Network Clusters Desktop, Virtual, Target & Hybrid Environments Configuration & Setup of Service based Network Clusters Health monitoring ICD handling

• Use Cases and Test Variation Model for Airbus D&S ‐ VIP Demonstrator

• Generation of Test Cases and Demonstrator Test Procedures Portability of Test Procedures on Desktop, Virtual and Target Environments Distribution of Modules & Objects (Test Tasks) in different Network Clusters

• Communication Layer API & Protocols XML‐RPC (remote application control) DDS (real‐time data exchange)  Configuration & File handling protocols (http) System Time synchronisation (NTP, PTP, virtual Clock, target Clock) VISTAS – EUROCAE WG 97 Avionic Bus Interfaces for Simulation Environments Benchmarks for distributed test applications

© All rights reserved