verification “auditability” can be “free” james h. jones optants documented decision support...
Post on 20-Dec-2015
215 views
TRANSCRIPT
![Page 1: Verification “Auditability” Can Be “Free” James H. Jones Optants Documented Decision Support Company (ODDSCO)](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1df2c/html5/thumbnails/1.jpg)
Verification “Auditability” Can Be “Free”
James H. Jones
Optants Documented Decision Support Company (ODDSCO)
www.optants.com
![Page 2: Verification “Auditability” Can Be “Free” James H. Jones Optants Documented Decision Support Company (ODDSCO)](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1df2c/html5/thumbnails/2.jpg)
2
Verification “Auditability” Can Be “Free”1. Definitions of Key Words
• Auditability: Requirements clarity and traceability from elicitation through V&V.
• Permutation: A distinct and meaningfully separate instance of a requirement that must be part of a comprehensive verification.
• Requirement: A statement specifying a mandatory characteristic or some functional behavior to be provided by product/process.
![Page 3: Verification “Auditability” Can Be “Free” James H. Jones Optants Documented Decision Support Company (ODDSCO)](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1df2c/html5/thumbnails/3.jpg)
3
Verification “Auditability” Can Be “Free”1. Definitions of Key Words (Continued)
• Validation: Determining if “the right system was built” (for customer/user).
• Verification: Determining that “the system was built right” (according to specification).
• Verification Audit Support Standards: (VASS) Use of Requirements Management methods that automate the “auditability” of requirements verification.
![Page 4: Verification “Auditability” Can Be “Free” James H. Jones Optants Documented Decision Support Company (ODDSCO)](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1df2c/html5/thumbnails/4.jpg)
4
Verification “Auditability” Can Be “Free”2. Benefits of Audit Support Standards
• Assists proper system allocations.
• Supports % Complete project metrics, from each requirement to entire specification.
• Assists detailed technical review of plans and procedures for requirements V&V.
• Supports audit of V&V development tasks at any time during the project.
![Page 5: Verification “Auditability” Can Be “Free” James H. Jones Optants Documented Decision Support Company (ODDSCO)](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1df2c/html5/thumbnails/5.jpg)
5
Verification “Auditability” Can Be “Free”3. System Engineering 101 (FRAT)
• Functions are product “needs” from the user or customer perspective.
• Requirements specify mandatory function, performance, and qualification objectives.
• Architecture is an implementation concept for a specification fulfilling product.
• Testing (V&V) confirms (or denies) that the system fulfills all of its requirements.
![Page 6: Verification “Auditability” Can Be “Free” James H. Jones Optants Documented Decision Support Company (ODDSCO)](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1df2c/html5/thumbnails/6.jpg)
6
Verification “Auditability” Can Be “Free”4. Typical RVTM Format
[SYSTEM] REQUIREMENTS VERiCATION TRACE MATRIX
PS ID System Requirement SS ID# Allocated Requirement VP ID# Comment
3 Functional Requirements S_3 Functional Requirements N/A Header Label
3.1 Subsection 1 Title N/A N/A N/A Header Label
3.1.1 Category 1 Title N/A N/A N/A Header Label
3.1.1.1 System shall (requirement 1) E_3.4.7.3 CPU shall (provide function) E_3.1.2
3.1.1.1 S_3.3.2.1 SW shall (provide function) S_3.3.14
3.1.1.2 System shall (requirement 2) M_3.3.2.1 SW shall (provide function) S_3.2.1
... ... ... ... ... ...
4 Design Qualification S_3 Design Qualification N/A Header Label
4.1 Subsection 1 Title N/A N/A N/A Header Label
4.1.1 Category 1 Title N/A N/A N/A Header Label
... ... ... ... ... ...
![Page 7: Verification “Auditability” Can Be “Free” James H. Jones Optants Documented Decision Support Company (ODDSCO)](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1df2c/html5/thumbnails/7.jpg)
7
Verification “Auditability” Can Be “Free”5. Basic Requirements Writing Rules
• Although they may swap designation with priority change, differentiate requirements and “desirements” (non-mandatory items).
• Limit requirements to one function, unless listing closely affiliated items.
• Avoid implicit requirements.
• Avoid “shall not” wording that requires test of universe to prove a negative.
![Page 8: Verification “Auditability” Can Be “Free” James H. Jones Optants Documented Decision Support Company (ODDSCO)](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1df2c/html5/thumbnails/8.jpg)
8
Verification “Auditability” Can Be “Free”5. Requirement Writing Rules (Continued)
• Avoid redundancies of “be capable of”or “be able to” (use direct action verb).
• Until fully verifiable by instrumented test, demonstration, inspection, or analysis, a requirement is not valid. (So, be cautious with adverbs and, when possible, replace adjectives with measurable values, ranges, or limits.)
![Page 9: Verification “Auditability” Can Be “Free” James H. Jones Optants Documented Decision Support Company (ODDSCO)](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1df2c/html5/thumbnails/9.jpg)
9
Verification “Auditability” Can Be “Free”5. Requirement Writing Rules (Continued)
• A Formal Specification Language approach uses the following standard form for all the requirement statements:
[Optional qualifier,] The (system/subsys-tem/process name) shall [use “should” or “will” if a desirement] (direct action verb statement of function) [, alternate position for the optional qualifier].
![Page 10: Verification “Auditability” Can Be “Free” James H. Jones Optants Documented Decision Support Company (ODDSCO)](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1df2c/html5/thumbnails/10.jpg)
10
Verification “Auditability” Can Be “Free”5. Requirement Writing Rules (Continued)
• The foregoing basic “rules” are a small subset of many commonly listed constraints and suggestions in requirements writing training courses, but they are sufficient to provide most users a consistently effective approach.
![Page 11: Verification “Auditability” Can Be “Free” James H. Jones Optants Documented Decision Support Company (ODDSCO)](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1df2c/html5/thumbnails/11.jpg)
11
Verification “Auditability” Can Be “Free”6. SRVTM (Tracking Permutations)
SUBSIDIARY REQUIREMENTS VERIFICATION TRACE MATRIX (Electrical, Mechanical, or Software Subsystem)
SS ID SS Requirement VP ID# Permutations List Comment 3 Functional Requirements N/A Header Label 3.1 Subsection 1 Title N/A Header Label 3.1.1 Category 1 Title N/A Header Label 3.1.1.1 The [subsystem] shall
(requirement 1) Permutation 1,
Permutation 2, Permutation 3 (To do)
(Probably all in ?_3.3.2)
3.1.1.2 The [subsystem] shall (requirement 2)
?_3.3.1.1/4/7/14
Permutation 1, Permutation 2, Permutation 3 (Done) Permutation 4, Permutation 5, Permutation 6, Permutation 7 (To do)
(4 VPs necessary for coverage)
... ... ... ... ...
![Page 12: Verification “Auditability” Can Be “Free” James H. Jones Optants Documented Decision Support Company (ODDSCO)](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1df2c/html5/thumbnails/12.jpg)
12
Verification “Auditability” Can Be “Free”6. SRVTM (Hazard Abatement Trace)
• When a requirement in a subsidiary hardware or software specification has been assigned to accomplish necessary reduction of risk level for an item in system Hazard Analysis, insert a column for showing that hazard identifier. [Such a column is put in the System RVTM, because assignments in Hazard Analysis are to the System Spec.]
![Page 13: Verification “Auditability” Can Be “Free” James H. Jones Optants Documented Decision Support Company (ODDSCO)](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1df2c/html5/thumbnails/13.jpg)
13
Verification “Auditability” Can Be “Free”7. Test Procedure Suggestions (VASS)
• Identify all Test Procedures (TPs) used for Verification of the permutations of each requirement in the Subsidiary RVTM.
• To support reviewers and auditors (when more than one TP used), list in order every requirement tested therein [and number of instances, when multiple permutations are tested] in the introduction section of each.
![Page 14: Verification “Auditability” Can Be “Free” James H. Jones Optants Documented Decision Support Company (ODDSCO)](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1df2c/html5/thumbnails/14.jpg)
14
Verification “Auditability” Can Be “Free”7. Test Procedure Suggestions (Continued)
• In the Expected Results column (or in the automated test scripts or test program code comment statements [or output to the results file]), annotate each test step with both the requirement identifier and extent of testing (such as which permutation was exercised and identification of a Hazard depending on its fulfillment for risk level reduction).
![Page 15: Verification “Auditability” Can Be “Free” James H. Jones Optants Documented Decision Support Company (ODDSCO)](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1df2c/html5/thumbnails/15.jpg)
15
Verification “Auditability” Can Be “Free”7. Test Procedure Suggestions (Continued)
• To ensure test step will confirm risk level reduction for an assigned Hazard, annotate the requirement identifier with that Hazard identifier.
• Reported verification success for that test step then becomes evidence that the hazard was abated by the identified action.
![Page 16: Verification “Auditability” Can Be “Free” James H. Jones Optants Documented Decision Support Company (ODDSCO)](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1df2c/html5/thumbnails/16.jpg)
16
Verification “Auditability” Can Be “Free”8. Percent Complete Metric (PD/TP * 100)
SUBSIDIARY SPECIFICATION REQUIREMENTS VERIFICATION TRACE MATRIX SS ID SS Requirement VP ID# TP PD Permutations List % Complete 3 Functional Requirements N/A 7362 1106 15 % 3.1 Subsection 1 Title N/A 184 157 85 % 3.1.1 Category 1 Title N/A 33 12 36 % 3.1.1.1 The [subsystem] shall
(requirement 1) 3 0 Permutation 1,
Permutation 2, Permutation 3 (To do)
(Probably all in ?_3.3.2)
3.1.1.2 The [subsystem] shall (requirement 2)
?_3.3.1.1 /4/7/14
7 3 Permutation 1, Permutation 2, Permutation 3 (Done) Permutation 4, Permutation 5, Permutation 6, Permutation 7 (To do)
(4 VPs necessary for coverage)
... ... ... ... ...
![Page 17: Verification “Auditability” Can Be “Free” James H. Jones Optants Documented Decision Support Company (ODDSCO)](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1df2c/html5/thumbnails/17.jpg)
17
Verification “Auditability” Can Be “Free”9. Related Metrics Expansion
• Because each Subsidiary RVTM is separate, spreadsheets can be used to automate the results of updates computation.
• RVTMs can include a new column for each Verification development milestone. (Such as Design Done, Dry Run, Verified, and Reported.)
![Page 18: Verification “Auditability” Can Be “Free” James H. Jones Optants Documented Decision Support Company (ODDSCO)](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1df2c/html5/thumbnails/18.jpg)
18
Verification “Auditability” Can Be “Free”9. Related Metrics Expansion (Cont.)
• The verification procedure development milestone completion entry for each requirement is summed into its requirements specification subsection.
• Each subsection development milestone completion total is summed into its requirements specification section.
![Page 19: Verification “Auditability” Can Be “Free” James H. Jones Optants Documented Decision Support Company (ODDSCO)](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1df2c/html5/thumbnails/19.jpg)
19
Verification “Auditability” Can Be “Free”9. Related Metrics Expansion (Cont.)
• Each verification development milestone completion section total is summed into its requirements specification document total.
• Using requirements permutations totals for each subsection, section, and specification, compute actual verification development milestones % complete values for reporting to self, supervisor, and project management.
![Page 20: Verification “Auditability” Can Be “Free” James H. Jones Optants Documented Decision Support Company (ODDSCO)](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1df2c/html5/thumbnails/20.jpg)
20
Verification “Auditability” Can Be “Free”10. Requirements Management Tools
• For simpler projects, the RVTM and metrics computations are appropriately developed with an ordinary spreadsheet.
• For medium to large projects, requirements management tools such as DOORS support for producing the RVTMs as reports and permitting the recommended % complete metrics computation.
![Page 21: Verification “Auditability” Can Be “Free” James H. Jones Optants Documented Decision Support Company (ODDSCO)](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1df2c/html5/thumbnails/21.jpg)
21
Verification “Auditability” Can Be “Free”11. Conclusions
• VASS can provide comprehensive rigor at less cost than previous practices would, so “auditability” can be considered “free”.
• Requirements Verification development metrics that reflect reality and help you know and report your part of the project status are superior to traditionally estimated percent complete values.
![Page 22: Verification “Auditability” Can Be “Free” James H. Jones Optants Documented Decision Support Company (ODDSCO)](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1df2c/html5/thumbnails/22.jpg)
22
Optants Documented Decision Support Co.www.optants.com
• Defense/Medical Systems Engineering
• Requirements Management
• Decision Modeling (incorporating project technical/schedule/cost risk)
• Process Improvement
• Training/Consulting