ron kratzke, vitech corporation [email protected] mbse for system testing managing the...
TRANSCRIPT
Ron Kratzke, Vitech Corporation
MBSE for System TestingManaging the development of system testing using the principles of Model Based System Engineering
Vitech Corporation 2
Context Diagram - Verification Process(Ref: INCOSE Handbook Ver 3.2.2 Oct 2011)
Controls - Applicable Laws and Regulations - Industry Standards - Agreements - Project Procedures and Standards -Project Directives
Outputs - Verification Strategy - Verification Enabling System Requirements - Verification Constraints on Design - Verification Procedures -Final RVTM - Verified System - Verification Report
Enablers - Organization / Enterprise Policies, Practices, and Procedures - Organization / Enterprise Infrastructure - Project Infrastructure - Verification Enabling Systems
Activities - Plan Verification - Perform Verification
Inputs - System Requirements - Verification Criteria - Specification Tree - Updated RVTM - Interface Requirements - Integrated System - Interface Control Documents - Integration Report
Vitech Corporation 3
Model-Based Systems Engineering
• Model-based systems engineering (MBSE) is the formalized application of modeling to support system requirements, design, analysis, verification and validation activities beginning in the conceptual design phase and continuing throughout development and later life cycle phases.
– From “INCOSE Systems Engineering Vision” 2020 INCOSE-TP-2004-004-02 September, 2007
Vitech Corporation 4
MODELS
VITECH CORPORATION 5
Everyone has a model …
Vitech Corporation 6
System Testing has to account for a number of items…
Requirements
Functions(Behavior Architecture)
Components(Physical Architecture)
Testing
Vitech Corporation 7
MBSE Verification Process ModelMust …
• Provides insight into the problem and potential solutions
• Provide order and traceability• Possess integrity and consistency• Have ability demonstrate and explain the test
program
Vitech Corporation 8
Is The VCRM Enough???Number Name Description method
R.1.1.1 Availability The system shall be unavailable no more than a total of ten minutes per month.
Analysis
R.2.1.1.1.1 Media of Requests: Hardcopy Forms
The system shall accept requests via hardcopy forms. Demonstration
R.2.1.1.1.4 Media of Requests: Telephonic Electronic File
The system shall accept requests via telephonic electronic file. Test
R.2.1.1.1.5 Media of Requests: Web Services The system shall accept requests via via a Web service. Test R.2.2.2 Provide Products The system shall provides previously collected images/products, if
appropriate. Demonstration
R.2.3.1 Control Multiple Collectors The system shall control multiple image collectors. Test R.2.3.1 Control Multiple Collectors The system shall control multiple image collectors. Test R.2.3.1 Control Multiple Collectors The system shall control multiple image collectors. Demonstration FN.1.1 Accept And Format Request The system shall accept the requests for information, verify that the
requester is a valid customer of the system, and format the request into a form and media that the system can use.
Test
FN.1.1 Accept And Format Request -Without Certification
The system shall accept the requests for information and format the request into a form and media that the system can use.
Test
FN.2.1 Check Product Inventory The system shall check the imagery product inventory to determine if the customer's request for imagery products is available from current imagery product inventory.
Test
FN.2.2 Prioritize Request The system shall organize customer orders using a priority based approach. Priorities shall be assigned based upon delivery date, customer need, current collection assignments, and system resources.
Test
FN.2.3 Determine Collector Mix The system shall evaluate the customer's order to establish which set of collectors shall best provide imagery products answering the customer's need.
Test
FN.2.4 Notify User Of Estimated Schedule The system shall prepare and send to the customer a notification estimating the delivery date of the customer's order.
Test
Vitech Corporation 9
Additional Information Needed
• In addition to the method, shouldn’t we also know?– Objective– Environment– Special Conditions– Success Criteria
Ref: A Day in the Life of a Verification Requirement Tutorial, Northrop Grumman, Steve Scukanec, 27th Annual T&E Conference, March 14, 2011
Vitech Corporation 10
Status of Completing the Verification
• Wouldn’t we also want to track?– Method (which is in the VCRM)– Planning Status – Completion Status– Where / When the Verification Event will take
place
Vitech Corporation 11
Verification Requirement Statement
Verification Requirement Description statement provides the details on HOW the verification will be accomplished
Vitech Corporation 12
Relating Requirements to Verification
Function,Requirement
verified by(verifies)
VerificationRequirement
Verification Requirement attributes: - Description - Method - Status - Level
Add two more attributes
Vitech Corporation 13
Table Output from Model …Number Name VR No. VR Name method Status Level of Test
R.1.1.1 Availability V.R.1.1 Continuous Support Analysis Planned System R.2.1.1.1.1 Media of Requests: Hardcopy
Forms V.R.2.1.1.1 Media of Requests: Hardcopy
Forms Demonstration Planned Subsystem
R.2.1.1.1.4 Media of Requests: Telephonic Electronic File
V.R.2.1.1.4 Media of Requests: Telephone Electronic File
Test In Progress Subsystem
R.2.1.1.1.5 Media of Requests: Web Services
V.R.2.1.1.5 Media of Requests: Web-based Electronic File
Test Not Yet Planned
Subsystem
R.2.2.2 Provide Products V.R.2.2.2 Provide Products Demonstration Not Yet Planned
System
R.2.3.1 Control Multiple Collectors V.R.2.3.1.a Control Multiple Collectors - Laboratory
Test Planned Subsystem
R.2.3.1 Control Multiple Collectors V.R.2.3.1.b Control Multiple Collectors - Ground Testing
Test Completed - Unsatisfactory
Subsystem
R.2.3.1 Control Multiple Collectors V.R.2.3.1.c Control Multiple Collectors - Flight Testing
Demonstration In Progress Subsystem
FN.1.1 Accept And Format Request V.R.2.5 Provide Feedback Test Completed - Satisfactory
System
FN.1.1 Accept And Format Request -Without Certification
V.R.2.5 Provide Feedback Test Completed - Satisfactory
System
FN.2.1 Check Product Inventory V.R.2.5 Provide Feedback Test Completed - Satisfactory
System
FN.2.2 Prioritize Request V.R.2.5 Provide Feedback Test Completed - Satisfactory
System
FN.2.3 Determine Collector Mix V.R.2.5 Provide Feedback Test Completed - Satisfactory
System
FN.2.4 Notify User Of Estimated Schedule
V.R.2.5 Provide Feedback Test Completed - Satisfactory
System
Provides the data from the model to provide overall status of verification planning… But what about the test events?
Vitech Corporation 14
What about the actual Testing?
Function,Requirement
verified by(verifies)
VerificationRequirement
What test events are planned?Which Verification Requirements will be verified in which test event?Who is doing the testing? What Organization is responsible?What configuration is going to be used in testing?Which Test Procedures will be used during any particular Test Event?
Vitech Corporation 15
What about the actual Testing?
Function,Requirement
verified by(verifies)
VerificationRequirement
VerificationEventfulfilled by
(fulfills)
Add “Verification Event” to specify and describe a Test Event which will be used to verify a subset of Verification Requirements (for example, Limited User Test, Laboratory Bench Test, Operational Test, etc.)
Vitech Corporation 16
What about the actual Testing?
Function,Requirement
verified by(verifies)
VerificationRequirement
VerificationEvent
Organization
TestProcedure
assigned to(responsible for)
employs(employed by)
TestConfiguration
employs(employed by)
fulfilled by(fulfills)
Relate the Verification Event to an Organization, Test Configuration, and Test Procedure
Vitech Corporation 17
Provide a complete picture…
Vitech Corporation 18
Verification Event Information
Verification Event Test Organization Test Config/Procedure Name Limited System Functional Test Acme Test Labs TestConfiguration Test Database
TestProcedure Data Base Access TestProcedure Data Entry Procedure
Logistic Test Event Logistics Analysts, Inc TestConfiguration Forward Operation Configuration TestConfiguration Permanent Cmd Ctr Configuration TestProcedure 30 Day System Operation Model
Media Usage Testing DataBase Testers R Us TestConfiguration Forward Operation Configuration TestConfiguration Test Database TestProcedure Media Usage Test Procedure
Multiple Collector Testing Collector Image Management, LLC TestConfiguration Global Hawk UAV Test Interface TestConfiguration Micro UAV Test Interface TestProcedure Two Collector Tasking Procedure
Vitech Corporation 19
What about time and resources for the Verification Requirement?
Function,Requirement
verified by(verifies)
VerificationRequirement
VerificationEvent
Organization
TestProcedure
assigned to(responsible for)
employs(employed by)
TestConfiguration
employs(employed by)
fulfilled by(fulfills)
Test Activity
executed by(executes)
Vitech Corporation 20
Test Activity – Detailed Test Planning
Vitech Corporation 21
Produce a Test Plan from the Model
Vitech Corporation 22
Summary …• System Testing model
– Extends traditional VCRM to provide added insight– Plan Verification Events including support resources– Manage Testing Activity sequence and timing– Produce Test Plans
• Traceability• Completeness• Consistency• Document Generation
Ron Kratzke, Vitech Corporation
MBSE for System Testing
Questions ???