deployment of a test management solution for a defence project using an integrated tool chain
DESCRIPTION
The presentation shows how a test management solution has been established for a defence project in compliance with a set of applicable standards using an integrated IBM Rational tool chain consisting of Rational Quality Manager for test management, IBM Rational DOORS for requirement management, IBM Rational Team Concert for defect management, IBM Rational Publishing Engine for automatic generation of project deliverables and last - but not least - IBM Rational Insight for trend and status reporting.TRANSCRIPT
![Page 1: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/1.jpg)
© 2014 IBM Corporation
Deployment of a test management solution for a defence project using an integrated tool chain
Code 1183
Dr. Einar Karlsen (IBM, Co-Speaker)
Thomas Schöning (Airbus DS)
![Page 2: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/2.jpg)
1
Introduction
Dr. Einar Karlsen (IBM)-Technical Solution Architect
Thomas Schöning (Airbus Defence and Space)- Integration and Test Manager
Thomas Südbröcker (IBM)- Technical Sales and Solutions + Community of Practices Leader Quality Management (Germany)
![Page 3: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/3.jpg)
2
• Subcontractor in a NATO Project with requirement based development, test and formal verification of a complex system entity
• Find a proper tooling solution for test and verification process using the IBM Rational tooling chain (Jazz Platform)
– Support for relevant standards ISO/IEE 15288, ISO/IEEE 12207, ISO 9001, ISO/IEC 9126
– Support for relevant process: Waterfall (V-Model) with incremental SW Development
– Integration with IBM Rational DOORS to accommodate for requirement traceability
– Automatic document generation for requirements, design and test artefacts (plans, procedures, reports)
– Insight into status and trend of the testing efforts (metrics)
Setting the Scene – Situation
![Page 4: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/4.jpg)
3
• Challenge
– Unknown effort to adequately install, integrate and customize the whole tooling chain to perform modelling, implementation, and test management
– Appropriate test management support (planning, execution, reporting, document generation, etc) according to required process?
• Improvements
– Decrease of costs, risks, inconsistencies and time-to-market through automatic generation of project deliverables
– Collection of metrics to measure test design progress, requirements coverage
– Online/Offline determination of test results while performing the test
Setting the Scene – Risk and Opportunity
![Page 5: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/5.jpg)
4
• Profound orientation for configuration and customization of IBM Rational Tools for a tool-aided test management solution (planning, execution, reporting, documentation) for similar test activities
1. Start early to implement test management prototyping to better estimate the effort to adapt the tools to the project needs
2. Invite technical professionals to have guidance and avoid pitfalls
3. Perform team training
4. Look at the deliverables before customizing your tool chain
Setting the Scene – Position and Benefits
![Page 6: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/6.jpg)
5
• Introduction
– Programme Overview
– Test and Evaluation Strategy
• Test management
– Solution Overview
– Solution Architecture
– Document Generation
– Metrics and Dashboards
• Summary
Agenda
![Page 7: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/7.jpg)
6
• Confidential, multinational NATO C4ISR Project (C4ISR=Command, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance)
– Unmanned Air, Command and Control Entities
– Transportable and Mobile Ground Stations for data exception, processing, and archiving facilities
– Communication links to other NATO and interoperable national ground stations, and to NATO and national Command and Control systems
– Exchange of standardized data via different interfaces and communication lines
• Airbus DS responsible for development, test and formal verification of a system entity consisting of several Subsystems
Programme Overview
![Page 8: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/8.jpg)
Test and Evaluation Strategy
Verification
Development
Testing
Entity
Subsystems
CSCIsHWCIs
ServicesComponents
Development
Testing
Development
TestingTest
Test
Test
Verification
Integration
![Page 9: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/9.jpg)
8
• Requirements of System Entity, Subsystems, Computer Software Configuration Items (CSCIs), and Hardware Configuration Items (HWCIs) are kept in DOORS database with traceable entries
• Testing/Verification is classified either to be
– Informal Tests
• conduct informal tests throughout the system development
– Subcontractor Development Tests
• For CSCI, HWCI, HW/CSCI groupings, Subsystem level, Entities and Integration testing
– Formal Verification
• Verification of allocated customer requirements based on proof of derived requirements (Entity, Subsystem, HW/CSCI groupings)
Test and Evaluation Strategy
![Page 10: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/10.jpg)
Test Management – Solution Overview
9
![Page 11: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/11.jpg)
10
Test Management – Solution Overview
• Test Plan
– consists of a number of Test Cases used to proof requirements
– is linked with one or more Requirements Modules (Test Plan views) � coverage and traceability (verification level)
• Test Case
– is linked to one or more particular requirements belonging to the requirements view(s) proofing requirements on different levels
• Test Script
– keeps the Test Procedure with single steps for test execution
• Test Case Execution Record/Test Case Result
– container to keep the Test Case results depending on iteration (integration, dry-run, acceptance) and test environment
• PTR (=Problem Trouble Report)
– keeps the defects occurred during Test Case execution
![Page 12: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/12.jpg)
Test Management – Solution Architecture
11
IBM Rational DOORS
IBM Rational Insight
IBM Rational Publishing Engine
REST, ETL REST, ETL
IBM Rational Quality Manager
IBM Rational Team Concert
REST, ETL
Dashboards, Status and trend reports
Test Plans, Procedures and Reports(Project Deliverables)
RESTREST REST
OSLClinks
OSLClinks
![Page 13: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/13.jpg)
12
Test Management – Test Document Generation
![Page 14: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/14.jpg)
Test Document Generation – Master Test List
![Page 15: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/15.jpg)
Test Document Generation – Test Case Details
![Page 16: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/16.jpg)
15
• Metrics: What you can measure that can be done!
• Metric types: Project, Product, Process, and Personnel
• Definition of metrics for monitoring and control test planning, design, development and execution
– test planning
• Percentage of requirements having a test case
– test design (analysis and concept)
• Percentage of test case with different status (draft, ready for review, approved)
– test development
• Percentage of test scripts with different status (draft, ready for review, approved)
– test execution
• Percentage of planned test cases being executed (passed, passed with condition, failed, blocked, error)
Test Management – Metrics and Dashboards
![Page 17: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/17.jpg)
16
Metrics and Dashboards – Test Case Design
![Page 18: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/18.jpg)
17
Metrics and Dashboards – Test Case Execution
![Page 19: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/19.jpg)
18
• Tooling Solution (Jazz platform) for test management process with combined DT&E strategy (Informal, Development Tests, Formal Verification) applied for system entity (NATO Project)
• Test Management Solution (Overview and Architecture)
– Test Plans linked with requirements view
– Test Cases linked with requirements
– Test Execution Records keeping the results for each Iteration
– Definition of timelines, iterations to support test planning andexecution
• Test Document Generation
– Tooling adaption for artefact templates (Test Plan, Test Case)
– Examples (Master Test List, Test Case Details)
• Metrics and Dashboards
– metrics for monitoring and control test planning, design, development and execution
Summary
![Page 20: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/20.jpg)
Articles on developerWorks
• RQM Template authoring articles on developer Works.
– Using IBM Rational Publishing Engine to generate compliance
documents: Report generation for Rational Quality Manager,
introducing REST service
URLs, importing schemas, and
basic template authoring
– Using IBM Rational Publishing
Engine to generate compliance
documents: Part 2. Report generation for Rational Quality
Manager, showing how to
extract information from grid sections and report on test
results and related defects
![Page 21: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/21.jpg)
Thank You!
Your Feedback is Important!
Access the Innovate agenda tool to complete your session surveys from your smartphone, laptop or
conference kiosk.
![Page 22: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/22.jpg)
21
• Test Plan:
– Artifact Templates (keep Test Cases and TCERs only)
– Artifact Categories (manufacturer, part number, type) to distinguish between different plans (system, software,etc)
• Test Case:
– Artifact Template has been customized to the documentation format needs (Test Plan, Test Procedure)
– Artifact Categories (test area, test method, test event, test type) and RQM section (description, test objective, requirement links,estimation) are used to appropriately keep Test Case data
• Timelines (Development and Production Phase)
• Iteration Types (Milestone, Software Test, System Test)
• Iteration
– directly corresponds with incremental software development (integration tests, dry run, acceptance test)
– support generation of TCER
Test Management – Tooling adaption
![Page 23: Deployment of a test management solution for a defence project using an integrated tool chain](https://reader034.vdocument.in/reader034/viewer/2022051616/557cf7e0d8b42a071b8b499e/html5/thumbnails/23.jpg)
22
• Administration of project requirements and test cases as well as the creation of bi-directional dependencies between them
• Planning test effort and milestones
• Defining test goals, acceptance and quality criteria
• Deriving test cases derived from requirement specifications
• Estimation of the effort of the test activities
• Definition of a time plan for each test iteration with finish dates of key activities
• Collection of different test environments and the generation of test configurations
• Creation and administration of test cases (manual and automated tests)
• Recording test results, creating defect and evaluating reports
• Support review and release process for the test plan and individual test cases.
IBM Rational Tools – Usage