qualkitdo slvnv tcp
TRANSCRIPT
-
8/20/2019 Qualkitdo Slvnv Tcp
1/35
DO Qualification Kit
Simulink® Verification and Validation™ Test Cases and Procedures
R2015b, September 2015
-
8/20/2019 Qualkitdo Slvnv Tcp
2/35
How to Contact MathWorks
Latest news: www.mathworks.com
Sales and services: www.mathworks.com/sales_and_services
User community: www.mathworks.com/matlabcentral
Technical support: www.mathworks.com/support/contact_us
Phone: 508-647-7000
The MathWorks, Inc.
3 Apple Hill Drive
Natick, MA 01760-2098 DO Qualification Kit: Simulink ® Verification and Validation™ Test Cases and Procedures
© COPYRIGHT 2009 – 2015 by The MathWorks, Inc.
The software described in this document is furnished under a license agreement. The software may be used or copied only underthe terms of the license agreement. No part of this manual may be photocopied or reproduced in any form without prior written
consent from The MathWorks, Inc.
FEDERAL ACQUISITION: This provision applies to all acquisitions of the Program and Documentation by, for, or through thefederal government of the United States. By accepting delivery of the Program or Documentation, the government hereby agrees
that this software or documentation qualifies as commercial computer software or commercial computer software documentationas such terms are used or defined in FAR 12.212, DFARS Part 227.72, and DFARS 252.227-7014. Accordingly, the terms andconditions of this Agreement and only those rights specified in this Agreement, shall pertain to and govern the use, modification,
reproduction, release, performance, display, and disclosure of the Program and Documentation by the federal government (or
other entity acquiring for or through the federal government)and shall supersede any conflicting contractual terms or conditions.
If this License fails to meet the government’s needs or is inconsistent in any respect with federal procurement law, the
government agrees to return the Program and Documentation, unused, to The MathWorks, Inc.
Trademarks
MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See www.mathworks.com/trademarks f or a
list of additional trademarks. Other product or brand names may be trademarks or registered trademarks of their respectiveholders.
Patents
MathWorks products are protected by one or more U.S. patents. Please see www.mathworks.com/patents f or moreinformation.
http://www.mathworks.com/http://www.mathworks.com/http://www.mathworks.com/company/aboutus/contact_us/http://www.mathworks.com/company/aboutus/contact_us/http://www.mathworks.com/matlabcentral/http://www.mathworks.com/matlabcentral/http://www.mathworks.com/support/contact_us/http://www.mathworks.com/support/contact_us/http://www.mathworks.com/trademarkshttp://www.mathworks.com/trademarkshttp://www.mathworks.com/trademarkshttp://www.mathworks.com/patentshttp://www.mathworks.com/patentshttp://www.mathworks.com/patentshttp://www.mathworks.com/patentshttp://www.mathworks.com/trademarkshttp://www.mathworks.com/support/contact_us/http://www.mathworks.com/matlabcentral/http://www.mathworks.com/company/aboutus/contact_us/http://www.mathworks.com/
-
8/20/2019 Qualkitdo Slvnv Tcp
3/35
Revision History
March 2009 New for Version 1.0 (Applies to Release 2009a)
September 2009 Revised for Version 1.1 (Applies to Release 2009b)April 2010 Rereleased for Version 1.1.1 (Applies to Release 2009bSP1)
March 2010 Revised for Version 1.2 (Applies to Release 2010a)
September 2010 Revised for Version 1.3 (Applies to Release 2010b)April 2011 Revised for Version 1.4 (Applies to Release 2011a)
September 2011 Revised for Version 1.5 (Applies to Release 2011b)
March 2012 Revised for Version 1.6 (Applies to Release 2012a)September 2012 Revised for Version 2.0 (Applies to Release 2012b)
March 2013 Revised for Version 2.1 (Applies to Release 2013a)
September 2013 Revised for Version 2.2 (Applies to Release 2013b)
March 2014 Revised for Version 2.3 (Applies to Release 2014a)
October 2014 Revised for Version 2.4 (Applies to Release 2014b)March 2015 Revised for Version 2.5 (Applies to Release 2015a)September 2015 Revised for DO Qualification Kit Version 3.0 (Applies to Release 2015b); renamed to Simulink ® Verification
and Validation™ Test Cases and Procedures
-
8/20/2019 Qualkitdo Slvnv Tcp
4/35
-
8/20/2019 Qualkitdo Slvnv Tcp
5/35
v
Contents
1
Introduction ...................................................................................................................................... 1-1
2 Test Cases and Procedures ............................................................................................................... 2-1
2.1 DO178C/DO-331 Checks ........................................................................................................ 2-2
2.2 Model Coverage ...................................................................................................................... 2-3
3 Traceability Between Operational Requirements and Test Cases and Procedures ........................... 3-1 3.1 DO-178C/DO-331 Checks ...................................................................................................... 3-2
3.2 Model Coverage ...................................................................................................................... 3-6
4 Executing Test Cases and Procedures .............................................................................................. 4-1
5
Test Results ...................................................................................................................................... 5-1
5.1 DO-178C/DO-331 Checks ...................................................................................................... 5-2 5.1.1 Reviewing Test Results ................................................................................................. 5-2
5.2 Model Coverage ...................................................................................................................... 5-4
5.2.1 Reviewing Test Results ................................................................................................. 5-5
-
8/20/2019 Qualkitdo Slvnv Tcp
6/35
vi
-
8/20/2019 Qualkitdo Slvnv Tcp
7/35
1 Introduction
This document comprises test cases and procedures (Reference DO-330 Sections 10.3.3) for the
following capabilities of the Simulink ® Verification and Validation™ verification tool:
DO-178C/DO-331 checks
Model coverage
This document is intended for use in the DO-330 tool qualification process for TQL-5 tools.
See also the DO Qualification Kit User's Guide, R2015b.
-
8/20/2019 Qualkitdo Slvnv Tcp
8/35
1-2
-
8/20/2019 Qualkitdo Slvnv Tcp
9/35
2 Test Cases and Procedures
-
8/20/2019 Qualkitdo Slvnv Tcp
10/35
2-2
2.1 DO178C/DO-331 ChecksRequirements-based test cases and procedures have been developed from the “DO-178C/DO-
331 Checks Operational Requirements” in the:
Simulink Verification and Validation Tool Operational Requirements
To access the tool operational requirements document, on the MATLAB ® command line, type
qualkitdo to open the Artifacts Explorer. The document is in Simulink Verification andValidation.
The test cases exercise DO-178C/DO-331 checks in the Model Advisor and reside in theSimulink ® test models identified in DO-178C/DO-331 Checks — Traceability Matrix.
The test procedure applies each DO-178C/DO-331 check in the Model Advisor to a subset of the
preceding test models, and resides in the report generator file identified in DO-178C/DO-331Checks — Traceability Matrix.
The test models and report generator file, which verify the DO-178C/DO-331 checks in the
Model Advisor, comprise the Tool Qualification Data (Reference DO-330 Sections 10.3.3 and10.3.4).
The applicant will:
Review the test cases and procedures for applicability to the project under consideration.
Configure the test cases and procedures in a configuration management system.
Confirm that test reports are generated without errors or warnings.
Confirm that the actual results are the same as the expected results.
-
8/20/2019 Qualkitdo Slvnv Tcp
11/35
2-3
2.2 Model CoverageRequirements- based test cases and procedures have been developed from the “Model Coverage
Operational Requirements” in the:
Simulink Verification and Validation Tool Operational Requirements
To access the tool operational requirements document, on the MATLAB ® command line, type
qualkitdo to open the Artifacts Explorer. The document is in Simulink Verification andValidation.
The test cases exercise the model coverage capability, and reside in the Simulink test modelsidentified in Model Coverage — Traceability Matrix.
The test procedure applies each model coverage capability to a subset of the preceding test
models, and resides in the report generator file identified in Model Coverage — TraceabilityMatrix.
The test models and report generator file, which verify the model coverage capability, comprise
the Tool Qualification Data (Reference DO-330 Sections 10.3.3 and 10.3.4).
The applicant will:
Review the test cases and procedures for applicability to the project under consideration.
Configure the test cases and procedures in a configuration management system.
Confirm that test reports are generated without errors or warnings.
Confirm that the actual results are the same as the expected results.
-
8/20/2019 Qualkitdo Slvnv Tcp
12/35
2-4
-
8/20/2019 Qualkitdo Slvnv Tcp
13/35
3 Traceability Between OperationalRequirements and Test Casesand Procedures
-
8/20/2019 Qualkitdo Slvnv Tcp
14/35
3-2
3.1 DO-178C/DO-331 ChecksThe test cases and procedures cover the Tool Operational Requirements for the DO-178C/
DO-331 checks, as shown in the following table.
DO-178C/DO-331 Checks— Traceability Matrix
Check Title(Requirement ID)
Test Case and Procedure(Test Case ID)
Test Model ExpectedResult
Check safety-related
optimization settings
(DO178C_01)
qualkitdo_slvnv_tcp1.rpt
(Test Case 1)
do178c_advisor_pass_test.slxdo178c_advisor_config_test.slx
Pass
Warning
Check safety-relateddiagnostic settings for solvers
(DO178C_02)
qualkitdo_slvnv_tcp1.rpt(Test Case 2)
do178c_advisor_pass_test.slxdo178c_advisor_config_test.slx
PassWarning
Check safety-relateddiagnostic settings for sampletime
(DO178C_03)
qualkitdo_slvnv_tcp1.rpt
(Test Case 3)
do178c_advisor_pass_test.slxdo178c_advisor_config_test.slx
PassWarning
Check safety-related
diagnostic settings for signal
data(DO178C_04)
qualkitdo_slvnv_tcp1.rpt
(Test Case 4)
do178c_advisor_pass_test.slxdo178c_advisor_config_test.slx
Pass
Warning
Check safety-related
diagnostic settings for
parameters
(DO178C_05)
qualkitdo_slvnv_tcp1.rpt
(Test Case 5)
do178c_advisor_pass_test.slxdo178c_advisor_config_test.slx
Pass
Warning
Check safety-related
diagnostic settings for dataused for debugging
(DO178C_06)
qualkitdo_slvnv_tcp1.rpt
(Test Case 6)
do178c_advisor_pass_test.slxdo178c_advisor_config_test.slx
Pass
Warning
Check safety-relateddiagnostic settings for data
store memory
(DO178C_07)
qualkitdo_slvnv_tcp1.rpt
(Test Case 7)
do178c_advisor_pass_test.slxdo178c_advisor_config_test.slx
PassWarning
Check safety-relateddiagnostic settings for typeconversions
(DO178C_08)
qualkitdo_slvnv_tcp1.rpt
(Test Case 8)
do178c_advisor_pass_test.slxdo178c_advisor_config_test.slx
PassWarning
Check safety-related
diagnostic settings for signalconnectivity
(DO178C_09)
qualkitdo_slvnv_tcp1.rpt
(Test Case 9)do178c_advisor_pass_test.slxdo178c_advisor_config_test.slx
Pass
Warning
-
8/20/2019 Qualkitdo Slvnv Tcp
15/35
3-3
Check Title(Requirement ID)
Test Case and Procedure(Test Case ID)
Test Model ExpectedResult
Check safety-related
diagnostic settings for busconnectivity
(DO178C_10)
qualkitdo_slvnv_tcp1.rpt
(Test Case 10)
do178c_advisor_pass_test.slx
do178c_advisor_config_test.slx
Pass
Warning
Check safety-relateddiagnostic settings that applyto function-call connectivity
(DO178C_11)
qualkitdo_slvnv_tcp1.rpt
(Test Case 11)
do178c_advisor_pass_test.slxdo178c_advisor_config_test.slx
PassWarning
Check safety-related
diagnostic settings for
compatibility(DO178C_12)
qualkitdo_slvnv_tcp1.rpt
(Test Case 12)
do178c_advisor_pass_test.slxdo178c_advisor_config_test.slx
Pass
Warning
Check safety-related
diagnostic settings for model
referencing
(DO178C_13)
qualkitdo_slvnv_tcp1.rpt
(Test Case 13)
do178c_advisor_pass_test.slxdo178c_advisor_config_test.slx
Pass
Warning
Check safety-related model
referencing settings
(DO178C_14)
qualkitdo_slvnv_tcp1.rpt
(Test Case 14)
do178c_advisor_pass_test.slxdo178c_advisor_config_test.slxdo178c_advisor_config_test2.slx
Pass
WarningWarning
Check safety-related codegeneration settings
(DO178C_15)
qualkitdo_slvnv_tcp1.rpt(Test Case 15)
do178c_advisor_pass_test.slxdo178c_advisor_config_test.slx
PassWarning
Display model version
information
(DO178C_16)
qualkitdo_slvnv_tcp1.rpt
(Test Case 16)
do178c_advisor_blocks_test.slx Pass
Check safety-relateddiagnostic settings for saving
(DO178C_17)
qualkitdo_slvnv_tcp1.rpt
(Test Case 17)
do178c_advisor_pass_test.slx
do178c_advisor_config_test.slxPassWarning
Check for blocks that do not
link to requirements
(DO178C_18)
qualkitdo_slvnv_tcp1.rpt
(Test Case 18)
do178c_requirementinfo_pass_test.slx
do178c_advisor_blocks_test.slx
Pass
Warning
Check safety-relateddiagnostic settings for model
initialization
(DO178C_19)
qualkitdo_slvnv_tcp1.rpt
(Test Case 19)
do178c_advisor_pass_test.slx
do178c_InitDiagnosticsSet_pass_test.slx
do178c_advisor_config_test.slx
PassPass
Warning
Check state machine type ofStateflow® charts
(DO178C_20)
qualkitdo_slvnv_tcp1.rpt
(Test Case 20)
do178c_hisf_0001_pass_test.slxdo178c_hisf_0001_warn_test.slx
PassWarning
Check Stateflow charts for
ordering of states andtransitions
(DO178C_21)
qualkitdo_slvnv_tcp1.rpt
(Test Case 21)
do178c_hisf_0002_pass_test.slxdo178c_hisf_0002_warn_test.slx
Pass
Warning
-
8/20/2019 Qualkitdo Slvnv Tcp
16/35
3-4
Check Title(Requirement ID)
Test Case and Procedure(Test Case ID)
Test Model ExpectedResult
Check Stateflow debugging
options (DO178C_22)
qualkitdo_slvnv_tcp1.rpt
(Test Case 22)
do178c_hisf_0011_pass_test.slx
do178c_hisf_0011_warn_test.slx
Pass
WarningCheck usage of lookup table blocks
(DO178C_23)
qualkitdo_slvnv_tcp1.rpt
(Test Case 23)
do178c_advisor_lookup_table_
pass.slxdo178c_advisor_lookup_table_
warn.slx
Pass
Warning
Check Stateflow charts for
uniquely defined data objects
(DO178C_24)
qualkitdo_slvnv_tcp1.rpt
(Test Case 24)
do178c_advisor_sf_uniquedata.slx
do178c_advisor_sf_uniquedata_
warn.slx
Pass
Warning
Check usage of Math
Operations blocks(DO178C_25)
qualkitdo_slvnv_tcp1.rpt
(Test Case 25)
do178c_advisor_math_operations_pass.slx
do178c_advisor_math_operations_warn.slx
Pass
Warning
Check usage of Signal
Routing blocks
(DO178C_26)
qualkitdo_slvnv_tcp1.rpt
(Test Case 26)
do178c_advisor_signal_routing_pass.slxdo178c_advisor_signal_routing_
warn.slx
Pass
Warning
Check usage of Logic and BitOperations blocks
(DO178C_27)
qualkitdo_slvnv_tcp1.rpt
(Test Case 27)
do178c_advisor_logic_and_bit_operations_pass.slx
do178c_advisor_blocks_test.slx
do178c_advisor_logic_and_bit_operations_warn.slx
Pass
WarningWarning
Check usage of Ports
and Subsystems blocks
(DO178C_28)
qualkitdo_slvnv_tcp1.rpt
(Test Case 28)
do178c_advisor_ports_and_subsystems_pass.slxdo178c_advisor_blocks_test.slx
do178c_advisor_ports_and_
subsystems_warn.slx
Pass
Warning
Warning
Check for inconsistent vectorindexing methods
(DO178C_29)
qualkitdo_slvnv_tcp1.rpt
(Test Case 29)do178c_hisl_0021_pass_test.slx,do178c_hisl_0021_warn_test.slx
Pass
Warning
Check for blocks not
recommended for C/C++ production code deployment
(DO178C_30)
qualkitdo_slvnv_tcp1.rpt
(Test Case 30)
do178c_advisor_prodcode_pass_test.slx do178c_advisor_prodcode_warn_test.slx
Pass
Warning
Check for MATLAB
Function interfaces withinherited properties
(DO178C_31)
qualkitdo_slvnv_tcp1.rpt
(Test Case 31)
do178c_himl_0002_pass_test.slx do178c_himl_0002_warn_test.slx
Pass
Warning
Check MATLAB Functionmetrics
(DO178C_32)
qualkitdo_slvnv_tcp1.rpt
(Test Case 32)
do178c_himl_0003_pass_test.slx do178c_himl_0003_warn_test.slx PassWarning
Check MATLAB CodeAnalyzer messages
(DO178C_33)
qualkitdo_slvnv_tcp1.rpt
(Test Case 33)
do178c_himl_0004_pass_test.slx do178c_himl_0004_warn_test.slx
Pass
Warning
-
8/20/2019 Qualkitdo Slvnv Tcp
17/35
3-5
Check Title(Requirement ID)
Test Case and Procedure(Test Case ID)
Test Model ExpectedResult
Check MATLAB code for
global variables(DO178C_34)
qualkitdo_slvnv_tcp1.rpt
(Test Case 34)
do178c_himl_0005_pass_test.slx
do178c_himl_0005_warn_test.slx
Pass
Warning
Check for variant blocks with‘Generate preprocessor
conditionals’ active
(DO178C_35)
qualkitdo_slvnv_tcp1.rpt
(Test Case 35)
do178c_advisor_variantblock_pass_test.slx.slx do178c_advisor_variantblock_warn_test.slx
Pass
Warning
-
8/20/2019 Qualkitdo Slvnv Tcp
18/35
3-6
3.2 Model Coverage
The test cases and procedures cover the Tool Operational Requirements for the model coveragecapability, as shown in the following table.
Model Coverage— Traceability Matrix
Model CoverageCapability(Requirement ID)
Test Case and Procedure(Test Case ID)
Test Model ExpectedResult
Cumulative Coverage
(MC_SL_CUMC)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_23)
mCumulativeCoverage.slx Included in
correspondingHTML reportin expectedresults folder
Simulink CyclomaticComplexity
(MC_SL_CYC)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_1)(TC_MC_SL_2)
(TC_MC_SL_3)(TC_MC_SL_4)
(TC_MC_SL_5)(TC_MC_SL_6)(TC_MC_SL_7)(TC_MC_SL_8)
(TC_MC_SL_9)(TC_MC_SL_10)(TC_MC_SL_11)
(TC_MC_SL_14)
(TC_MC_SL_15)(TC_MC_SL_16)
(TC_MC_SL_17)(TC_MC_SL_18)
(TC_MC_SL_19)(TC_MC_SL_20)
mAbsBlock.slx mAbsOptim.slx
mcase.slx mcmblogic.slxmdintegrate.slxmdintegrate_slevel.slxmDiscIntegExternalReset.slxmsubsys.slx menable_table.slx
mfcn.slx mfor.slxmif.slx mLogicBlock.slxmLogic_largematrix.slxmLogic_manyinputs.slxmLogicBlockFxp.slxmminmax.slx mMinMaxFxp.slx
mmpswtch.slxmratelimiter_discrete.slxmratelimiter_variable.slxmrelay.slx mSaturationScalar.slxmSaturationVector.slx
mSaturationMatrix.slxmSaturationEdgeCases.slxmSaturationFixedPoint.slxmswitch.slx mwhile.slxmEMLMultipleRunMixed.slx
Included incorresponding
HTML reportin expectedresults folder
-
8/20/2019 Qualkitdo Slvnv Tcp
19/35
3-7
Model CoverageCapability(Requirement ID)
Test Case and Procedure(Test Case ID)
Test Model ExpectedResult
Decision Coverage(MC_SL_DC)
qualkitdo_slvnv_tcp2.rpt(TC_MC_SL_2)(TC_MC_SL_3)
(TC_MC_SL_4)(TC_MC_SL_5)(TC_MC_SL_6)
(TC_MC_SL_7)
(TC_MC_SL_8)
(TC_MC_SL_9)
(TC_MC_SL_10)(TC_MC_SL_11)
(TC_MC_SL_12)
(TC_MC_SL_13)
(TC_MC_SL_14)(TC_MC_SL_15)(TC_MC_SL_16)
(TC_MC_SL_17)(TC_MC_SL_18)(TC_MC_SL_19)
(TC_MC_SL_20)(TC_MC_SL_21)
mAbsBlock.slxmAbsOptim.slxmcase.slx
mcmblogic.slxmdintegrate.slxmdintegrate_slevel.slxmDiscIntegExternalReset.slxmsubsys.slx
menable_table.slxmfcn.slx mfor.slx
mif.slx
mminmax.slxmMinMaxFxp.slx
mmodelblock.slxmModelRefSubsys.slx
mmpswtch.slxmratelimiter_discrete.slxmratelimiter_variable.slxmrelay.slx
mSaturationScalar.slxmSaturationVector.slx
mSaturationMatrix.slxmSaturationEdgeCases.slxmSaturationFixedPoint.slxmswitch.slx
mwhile.slxmEMLMultipleRunMixed.slx
mDeadZoneBlock.slx
Included incorrespondingHTML reportin expected
results folder
Condition Coverage(MC_SL_CC)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_1)
(TC_MC_SL_4)
(TC_MC_SL_6)(TC_MC_SL_7)(TC_MC_SL_8)
(TC_MC_SL_10)
(TC_MC_SL_12)
(TC_MS_SL_13)
(TC_MC_SL_20)
mcmblogic.slx
msubsys.slxmenable_table.slxmfcn.slx
mif.slx
mLogicBlock.slxmLogic_largematrix.slxmLogic_manyinputs.slxmLogicBlockFxp.slx
mmodelblock.slxmModelRefSubsys.slxmEMLMultipleRunMixed.slx
Included incorresponding
HTML report
in expectedresults folder
-
8/20/2019 Qualkitdo Slvnv Tcp
20/35
3-8
Model CoverageCapability(Requirement ID)
Test Case and Procedure(Test Case ID)
Test Model ExpectedResult
ModifiedCondition/DecisionCoverage(MC_SL_MCDC)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_1)
(TC_MC_SL_6)
(TC_MC_SL_8)
(TC_MC_SL_10)
(TC_MC_SL_12)
(TC_MS_SL_13)
(TC_MC_SL_20)
msubsys.slxmfcn.slx
mif.slx
mLogicBlock.slxmLogic_largematrix.slxmLogic_manyinputs.slxmLogicBlockFxp.slx
mmodelblock.slxmModelRefSubsys.slxmEMLMultipleRunMixed.slx
Included incorrespondingHTML reportin expected
results folder
Lookup Table Coverage
(MC_SL_LUT)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_7)qualkitdo_slvnv_tcp4.rpt
(TC_MC_LUT_1)(TC_MC_LUT_2)
(TC_MC_LUT_3)(TC_MC_LUT_4)(TC_MC_LUT_5)(TC_MC_LUT_6)
menable_table.slx
minterpolation.slxminterpolation_direct.slxmlookup.slx
mnanlookup.slxmlookup2d.slx
mLookupNDas1D.slxmLookupNDas2D.slxmLookupNDasND.slxmLookupND_direct.slxmLookupNDPrelookup.slxmLookupNDPrelookupFxp.slxmLookupNDPrelookupLegacy.slx
Included in
correspondingHTML reportin expectedresults folder
Signal Range Coverage
(MC_SL_SR)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_7)qualkitdo_slvnv_tcp5.rpt
(TC_MC_SIG_1)qualkitdo_slvnv_tcp3.rpt
(TC_MC_SF_5)
menable_table.slxmrange_cont.slxmrange_discont.slxmrange_discrete.slxmrange_logic.slxmrange_lookup.slxmrange_math.slx
mrange_other.slxmrange_signals.slx
mrange_sources.slxmSignalRange.slx
MRangeTooWide.slxm_sf_data_check.slx
Included in
correspondingHTML report
in expectedresults folder
Signal Size Coverage
(MC_SL_SS)
qualkitdo_slvnv_tcp5.rpt
(TC_MC_SIG_2)
mvardims.slxmVardimsMixedBus.slxmLogic_vardims.slx
Included in
correspondingHTML reportin expected
results folder
Saturation on integeroverflow coverage
(MC_SATINT)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_22)qualkitdo_slvnv_tcp3.rpt
(TC_MC_SF_11)qualkitdo_slvnv_tcp4.rpt
(TC_MC_LUT_7)
mMathFcnOverflow.slx
mSFSatOnOverflow.slxmLookup1DOverflow.slxmLookup2DOverflow.slxmLookupNDOverflow.slx
Included incorresponding
HTML reportin expectedresults folder
-
8/20/2019 Qualkitdo Slvnv Tcp
21/35
3-9
Model CoverageCapability(Requirement ID)
Test Case and Procedure(Test Case ID)
Test Model ExpectedResult
Relational boundarycoverage(MC_SL_RELBOUND)
qualkitdo_slvnv_tcp6.rpt
(TC_MC_SLDV_2)
mAllBlocks_RelBound.slx
mAllBlocks_RelBound_Fxp.slx
mSFRelCov_Fxp.slx
mSFRelCov.slx
Included incorrespondingHTML reportin expected
results
Abs block
(MC_SL_ABS)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_2)
mAbsBlock.slx
mAbsOptim.slxIncluded in
correspondingHTML reportin expectedresults folder
Combinatorial Logic block (MC_SL_
COMBLOGIC)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_4)
mcmblogic.slx Included incorresponding
HTML reportin expectedresults folder
Dead Zone block
(MC_SL_DEAD)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_21)
mDeadZoneBlock.slx Included incorrespondingHTML report
in expected
results folderDirect Lookup Table
(n-D) block(MC_SL_DLUTND)
qualkitdo_slvnv_tcp4.rpt
(TC_MC_LUT_5)
mLookupND_direct.slx Included in
correspondingHTML reportin expected
results folder
Discrete-Time Integrator
block(MC_SL_DINT)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_5)
mdintegrate.slxmdintegrate_slevel.slxmDiscIntegExternalReset.slx
Included in
correspondingHTML report
in expectedresults folder
Enabled and TriggeredSubsystem block
(MC_SL_ENTRG)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_6)
msubsys.slx Included incorresponding
HTML report
in expectedresults folder
Enabled Subsystem
block (MC_SL_ENAB)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_6)
(TC_MC_SL_7)
msubsys.slx
menable_table.slxIncluded in
corresponding
HTML reportin expected
results folder
-
8/20/2019 Qualkitdo Slvnv Tcp
22/35
3-10
Model CoverageCapability(Requirement ID)
Test Case and Procedure(Test Case ID)
Test Model ExpectedResult
Fcn block(MC_SL_FCN)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_8)
mfcn.slxIncluded incorrespondingHTML reportin expected
results folder
For Iterator, For Iterator
Subsystem blocks(MC_SL_FOR)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_9)
mfor.slx Included in
correspondingHTML reportin expectedresults folder
If, If Action Subsystem blocks
(MC_SL_IF)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_10)
mif.slx Included incorresponding
HTML reportin expectedresults folder
Interpolation UsingPrelookup block(MC_SL_PRELUT)
qualkitdo_slvnv_tcp4.rpt
(TC_MC_LUT_1)
(TC_MC_LUT_6)
minterpolation.slxminterpolation_direct.slxmLookupNDPrelookup.slxmLookupNDPrelookupFxp.slxmLookupNDPrelookupLegacy.slx
Included incorrespondingHTML report
in expected
results folderLogical Operator block
(MC_SL_LOGIC)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_1)
mLogicBlock.slxmLogic_largematrix.slxmLogic_manyinputs.slxmLogicBlockFxp.slx
Included in
correspondingHTML reportin expected
results folder
1-D Lookup Table block
(MC_SL_LUT1D)
qualkitdo_slvnv_tcp4.rpt
(TC_MC_LUT_2)
mlookup.slx
mnanlookup.slxIncluded in
correspondingHTML report
in expectedresults folder
2-D Lookup Table block
(MC_SL_LUT2D)
qualkitdo_slvnv_tcp4.rpt
(TC_MC_LUT_3)
mlookup2d.slx Included incorresponding
HTML report
in expectedresults folder
n-D Lookup Table block
(MC_SL_LUTND)
qualkitdo_slvnv_tcp4.rpt
(TC_MC_LUT_4)
mLookupNDas1D.slxmLookupNDas2D.slxmLookupNDasND.slx
Included in
corresponding
HTML reportin expected
results folder
-
8/20/2019 Qualkitdo Slvnv Tcp
23/35
3-11
Model CoverageCapability(Requirement ID)
Test Case and Procedure(Test Case ID)
Test Model ExpectedResult
MATLAB Function block
(MC_SL_EML)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_20)qualkitdo_slvnv_tcp3.rpt
(TC_MC_SF_8)
(TC_MC_SF_9)
mEMLMultipleRunMixed.slxsf_exec_at_init.slxeml_call_gf.slxeml_call_gf_lib.slx
Included incorrespondingHTML reportin expected
results folder
MinMax block
(MC_SL_MINMAX)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_11)
mminmax.slx
mMinMaxFxp.slxIncluded in
correspondingHTML reportin expectedresults folder
Model block(MC_SL_MDLREF)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_12)(TC_MC_SL_13)
qualkitdo_slvnv_tcp3.rpt
(TC_MC_SF_10)
mmodelblock.slxmmodelblock_condition.slxmmodelblock_decision.slxmmodelblock_eml.slxmmodelblock_lookup.slxmmodelblock_mcdc.slxmmodelblock_multilevel.slxmmodelblock_multilevel2.slxmModelRefSubsys.slxmModelRefSubsys_s1.slxmModelRefSubsys_s2.slxmModelRefSubsys_s3.slxmmodelblock_topSF.slxmmodelblock_stateflow.slx
Included incorresponding
HTML reportin expectedresults folder
Multiport Switch block(MC_SL_MPSWITCH)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_14)
mmpswtch.slx Included incorrespondingHTML report
in expectedresults folder
Rate Limiter block
(MC_SL_RATLIM)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_15)
mratelimiter_discrete.slx
mratelimiter_variable.slx
Included in
correspondingHTML reportin expected
results folder
Relay block
(MC_SL_RELAY)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_16)
mrelay.slx Included in
correspondingHTML reportin expected
results folder
Saturation block(MC_SL_SATURATE)
qualkitdo_slvnv_tcp2.rpt(TC_MC_SL_17)
mSaturationScalar.slxmSaturationVector.slxmSaturationMatrix.slxmSaturationEdgeCases.slxmSaturationFixedPoint.slx
Included incorresponding
HTML reportin expectedresults folder
-
8/20/2019 Qualkitdo Slvnv Tcp
24/35
3-12
Model CoverageCapability(Requirement ID)
Test Case and Procedure(Test Case ID)
Test Model ExpectedResult
Sqrt, Signed Sqrt,Reciprocal Sqrt
(MC_SL_SQRT)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_22)
mSqrtOverflow.slx
Included incorrespondingHTML reportin expected
results folder
Switch block
(MC_SL_SWITCH)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_18)
mswitch.slx Included in
correspondingHTML reportin expectedresults folder
Switch Case, SwitchCase Action Subsystem
blocks(MC_SL_CASE)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_3)
mcase.slx Included incorresponding
HTML reportin expectedresults folder
Triggered Subsystem block(MC_SL_TRIGGER)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_6)
msubsys.slx Included incorrespondingHTML report
in expected
results folderWhile Iterator, While
Iterator Subsystem blocks(MC_SL_WHILE)
qualkitdo_slvnv_tcp2.rpt
(TC_MC_SL_19)
mwhile.slx Included in
correspondingHTML reportin expected
results folder
Stateflow Cyclomatic
Complexity(MC_SF_CYC)
qualkitdo_slvnv_tcp3.rpt
(TC_MC_SF_1)(TC_MC_SF_2)
(TC_MC_SF_3)(TC_MC_SF_4)(TC_MC_SF_6)(TC_MC_SF_7)(TC_MC_SF_8)
(TC_MC_SF_9)
mstate.slx
sfmod.slxmSFArithLogic.slxmSFTooComplexMcdc.slxlarge_mcdc.slxmSFTransitionMCDCNotExecuted.slxm_on_event_cov.slxm_sf_exit_substate.slxtruth_table_model.slxmTTNoCondition.slxtruth_table_library.slxsf_exec_at_init.slxeml_call_gf.slxeml_call_gf_lib.slx
Included in
correspondingHTML report
in expectedresults folder
Chart as triggeredSimulink block(MC_SF_TRIGGER)
qualkitdo_slvnv_tcp3.rpt(TC_MC_SF_1)
mstate.slx Included incorrespondingHTML report
in expectedresults folder
-
8/20/2019 Qualkitdo Slvnv Tcp
25/35
3-13
Model CoverageCapability(Requirement ID)
Test Case and Procedure(Test Case ID)
Test Model ExpectedResult
Chart with Exclusive ORSubstates(MC_SF_CHARTSUB)
qualkitdo_slvnv_tcp3.rpt
(TC_MC_SF_6)
m_sf_exit_substate.slx
Included incorrespondingHTML reportin expected
results folder
Superstate with
Exclusive OR Substates(MC_SF_SUPERSUB)
qualkitdo_slvnv_tcp3.rpt
(TC_MC_SF_2)
sfmod.slx
mSFArithLogic.slxmSFTooComplexMcdc.slx
Included in
correspondingHTML reportin expectedresults folder
State with OnEvent_Name Action
Statement(MC_SF_ONEVENT)
qualkitdo_slvnv_tcp3.rpt
(TC_MC_SF_4)
m_on_event_cov.slx Included incorresponding
HTML reportin expectedresults folder
Conditional transitiondecision(MC_SF_TRANDEC)
qualkitdo_slvnv_tcp3.rpt
(TC_MC_SF_1)
(TC_MC_SF_2)
(TC_MC_SF_3)
mstate.slx
sfmod.slxmSFArithLogic.slxmSFTooComplexMcdc.slxlarge_mcdc.slxmSFTransitionMCDCNotExecuted.slx
Included incorrespondingHTML report
in expected
results folder
Condition coverage of
transition decision(MC_SF_TRANCOND)
qualkitdo_slvnv_tcp3.rpt
(TC_MC_SF_2)
(TC_MC_SF_3)
sfmod.slx
mSFArithLogic.slxmSFTooComplexMcdc.slxlarge_mcdc.slxmSFTransitionMCDCNotExecuted.slx
Included in
correspondingHTML report
in expected
results folder
MC/DC coverage of
transition decision(MC_SF_TRANMCDC)
qualkitdo_slvnv_tcp3.rpt
(TC_MC_SF_2)(TC_MC_SF_3)
sfmod.slx
mSFArithLogic.slxmSFTooComplexMcdc.slx
large_mcdc.slxmSFTransitionMCDCNotExecuted.slx
Included in
correspondingHTML report
in expectedresults folder
Truth Table conditions(MC_SF_TTCOND)
qualkitdo_slvnv_tcp3.rpt
(TC_MC_SF_7)
truth_table_model.slxmTTNoCondition.slxtruth_table_library.slx
Included incorresponding
HTML report
in expectedresults folder
Truth Table decisions
(MC_SF_TTDEC)
qualkitdo_slvnv_tcp3.rpt
(TC_MC_SF_7)
truth_table_model.slxmTTNoCondition.slxtruth_table_library.slx
Included in
corresponding
HTML reportin expectedresults folder
-
8/20/2019 Qualkitdo Slvnv Tcp
26/35
3-14
Model CoverageCapability(Requirement ID)
Test Case and Procedure(Test Case ID)
Test Model ExpectedResult
Truth Table MC/DC(MC_SF_TTMCDC)
qualkitdo_slvnv_tcp3.rpt
(TC_MC_SF_7)
truth_table_model.slxmTTNoCondition.slxtruth_table_library.slx
Included incorrespondingHTML reportin expected
results folder
Simulink ® Design
Verifier ™ Coverage(MC_SLDV_SLDV)
qualkitdo_slvnv_tcp6.rpt
(TC_MC_SLDV_1)
mCombineObjectives.slxmDeadZoneCustCov.slx
Included in
correspondingHTML reportin expectedresults folder
Proof Assumption block(MC_SLDV_
PROOFASM)
qualkitdo_slvnv_tcp6.rpt
(TC_MC_SLDV_1)mCombineObjectives.slxmDeadZoneCustCov.slx
Included incorresponding
HTML reportin expectedresults folder
Proof Objective block(MC_SLDV_PROOFOBJ)
qualkitdo_slvnv_tcp6.rpt
(TC_MC_SLDV_1)
mCombineObjectives.slxmDeadZoneCustCov.slx
Included incorrespondingHTML report
in expected
results folderTest Condition block
(MC_SLDV_TESTCON)
qualkitdo_slvnv_tcp6.rpt
(TC_MC_SLDV_1)
mCombineObjectives.slxmDeadZoneCustCov.slx
Included in
correspondingHTML reportin expected
results folder
Test Objective block
(MC_SLDV_TESTOBJ)
qualkitdo_slvnv_tcp6.rpt
(TC_MC_SLDV_1)
mCombineObjectives.slxmDeadZoneCustCov.slx
Included in
correspondingHTML report
in expectedresults folder
Relational boundarycoverage for Stateflow
(MC_SF_BOUNDCOV)
qualkitdo_slvnv_tcp6.rpt
(TC_MC_SLDV_2)
mAllBlocks_RelBound.slx
mAllBlocks_RelBound_Fxp.slx
mSFRelCov_Fxp.slx
mSFRelCov.slx
Included incorresponding
HTML report
in expectedresults folder
Relational Operator
block
(MC_SL_RELOP)
qualkitdo_slvnv_tcp5.rpt
(TC_MC_SIG_3)
mRelOpEquals.slx
mRelOpGE.slx
mRelOpGT.slx
mRelOpIsFinite.slx
mRelOpIsInf.slx
mRelOpIsNaN.slx
mRelOpLE.slx
mRelOpLT.slx
mRelOpNE.slx
Included in
corresponding
HTML reportin expected
results folder
-
8/20/2019 Qualkitdo Slvnv Tcp
27/35
4 Executing Test Cases andProcedures
-
8/20/2019 Qualkitdo Slvnv Tcp
28/35
4-2
The applicant will execute the test cases and procedures for the following capabilities in the
installed environment:
DO-178C/DO-331 checks Model coverage
Note:
To run the test cases and procedures, copy the
matlabroot/toolbox/qualkits/do/slvnv folder and subfolders to a locationwhere you have write access, and then run the test cases and procedures from there.
Start a new MATLAB® session before you execute each model coverage RPT file.
Set the Java™ heap size for your MATLAB session to at leas t 512 MB before you execute
model coverage RPT files. To check the Java heap size, open the MATLAB Preferencesdialog box and select General > Java Heap Memory. If the Java Heap Size value is lessthan 512 MB, change it to 512 MB, click OK, and restart MATLAB. (If the maximumavailable heap size value is less than 512 MB, select the maximum value.) This may help
you avoid java.lang.OutOfMemoryError messages.
To execute the RPT files, you must have MATLAB® Report Generator™ and Simulink ® Report Generator™ licenses.
To execute some model coverage RPT files, you must have Fixed-Point Designer ™,Stateflow®, and Simulink ® Design Verifier ™ licenses, as noted in the Model Coverage — Test Summary.
Without an Embedded Coder ® license, some DO-178C/DO-331 checks might warn because configuration parameters are not available.
Execute the Simulink Verification and Validation test procedure:
1. Create the report:
Using the GUI.
a. Double-click the RPT file to open the Report Explorer. b. Select File > Report. The Simulink Report Generator creates the HTML report.
-
8/20/2019 Qualkitdo Slvnv Tcp
29/35
4-3
At the MATLAB® command line, enter the following command, where rpt_file isthe name of the test procedure:
report (‘rpt_file’).
2. After the report generation is complete, review the test report results. For moreinformation, see “Test Results”.
-
8/20/2019 Qualkitdo Slvnv Tcp
30/35
4-4
-
8/20/2019 Qualkitdo Slvnv Tcp
31/35
5 Test Results
-
8/20/2019 Qualkitdo Slvnv Tcp
32/35
5-2
5.1 DO-178C/DO-331 ChecksExecuting the following test procedure generates the following test result file. The test result file
provides the Tool Verification Results in a Simulink ®
Report Generator ™ report.
DO-178C/DO-331 Checks— Test Summary
Test Procedure Test Result
qualkitdo_slvnv_tcp1.rpt outputs/qualkitdo_slvnv_qualificationreport1.html
The applicant will:
Review the Tool Verification test results.
Configure the test results in a configuration management system.
Confirm that test reports are generated without errors or warnings.
Confirm that the actual results are the same as the expected results.
5.1.1 Reviewing Test Results
Each test case contains the test results for a DO-178C/DO-331 check. For each test case,compare the expected results to the actual results:
1. From the working folder, open
outputs/qualkitdo_slvnv_qualificationreport1.html in a web browser.
2. Navigate to the test case. You see:
A table that summarizes the test case. Text listing the expected results for each test model.
A figure of each test model.
A Model Advisor report for each test model.
3. For each test model, review the Model Advisor report. The report provides the expectedand actual results for the overall check and subchecks. If one of the subchecks warns, the
overall check result is a warning. For example, the following figure shows part of the report
for test case 1, providing results for the Check safety-related optimization setting checkon the do_178c_advisor_pass_test model. The Actual and Expected Resultsfigure below shows the expected overall check and subcheck results. The summary table
indicates that the subchecks passed.
-
8/20/2019 Qualkitdo Slvnv Tcp
33/35
5-3
Expected and Actual Test Results
4. Compare the actual results of the test case in the Run Summary with the expected results
listed above the figure of the test model.
-
8/20/2019 Qualkitdo Slvnv Tcp
34/35
5-4
5.2 Model CoverageExecuting the following test procedures generates the following test result files. The test result
files provide the Tool Verification Results in Simulink ®
Report Generator ™ reports.
When you use the Simulink Report Generator to execute the test procedures, the test results are
generated in outputs/qualkitdo_slvnv_qualificationreport*.html files.
Each HTML file corresponds to an .rpt file. The
outputs/qualkitdo_slvnv_qualificationreport*.html files include links tothe actual generated model coverage reports for each model, which are also located in the
outputs folder.
The model coverage expected results are in the
expected_results/qualkitdo_slvnv_qualificationreport*.html files.These HTML files include links to the expected model coverage reports for each model, which
are also located in the expected_results folder.
After you execute the test procedures, compare the actual results, located in the outputs
folder, to the expected results, located in the expected_results folder.
Model Coverage—
Test Summary
Test Procedure Test Result
qualkitdo_slvnv_tcp2.rpt1 outputs/qualkitdo_slvnv_qualificationreport2.html
qualkitdo_slvnv_tcp3.rpt2 outputs/qualkitdo_slvnv_qualificationreport3.html
qualkitdo_slvnv_tcp4.rpt1 outputs/qualkitdo_slvnv_qualificationreport4.html
qualkitdo_slvnv_tcp5.rpt2 outputs/qualkitdo_slvnv_qualificationreport5.html
qualkitdo_slvnv_tcp6.rpt3 outputs/qualkitdo_slvnv_qualificationreport6.html
Notes:1 Requires a Fixed-Point Designer ™ license.2 Requires a Stateflow® license.3 Requires a Simulink ® Design Verifier ™ license.
The applicant will:
Review the Tool Verification test results.
Configure the test results in a configuration management system.
Confirm that test reports are generated without errors or warnings.
Confirm that the actual results are the same as the expected results.
-
8/20/2019 Qualkitdo Slvnv Tcp
35/35
5-5
5.2.1 Reviewing Test Results
The test result files contain both expected results and links to actual results. For each model,
compare the expected results in the expected_results folder to the actual results in theoutputs folder:
1. From the working folder, open the actual test result HTML file in a web browser. For
example, open outputs/qualkitdo_slvnv_qualificationreport2.html .
2. In the HTML file, navigate to a specific model coverage test result. For example, thefollowing figure shows part of the model coverage report for test case TC_MC_SL_2,
providing results for the mAbsBlock.slx model.
Actual and Expected Model Coverage Results
3. Using the links provided in the actual result HTML files, compare the actual coverageresults to the expected coverage results.