Slide 1

Slide 2

CARA Process Methodology 1

Slide 3

Introduction Objectives Testing Intake Process CARA Scoring Risk Analysis and Testing Scope Report Challenges 2

Slide 4

3 Paul Shovlin Checkpoint Technologies Director of Professional Services IV&V Functional Test Manager, Department of Veterans Affairs

Slide 5

4

Slide 6

Understand the Testing Intake Assessment (TIA) process. Provide knowledge and understanding of Criticality Analysis and Risk Assessment (CARA) principles and practices. Understand the purpose and content of the Risk Analysis and Testing Scope Report (RATSR). Share our challenges. 5

Slide 7

September 2010 mandate that all application release cycles complete from development to Initial Operating Capability (IOC) within 6 months, not leaving much time if any for IV&V testing. 150+ Major application project releases each calendar year. New approach to IV&V needed. CARA was being used successfully at NASA. 6

Slide 8

7

Slide 9

8 Testing Intake Assessment Form CONOPS Requirements System Design CONOPS Requirements System Design Submit Testing Intake Assessment Read, Review Docs & Formulate Questions SMEs Answer Questions Prepare CARA Worksheet Perform CARA Create & Review RATSR Determine Services Testing Workload Form Update Project Schedule Update Project Schedule Update Testing Workload Forecast Project Management Plan Project Schedule Project Management Plan Project Schedule Assign CARA Resources CARA Worksheet Updated CARA Worksheet Updated CARA Worksheet RATSR

Slide 10

Risk is the likelihood of failure. Criticality is the measure of potential impact of failure. Risk-Based Testing is a type of software testing that prioritizes the tests of features and functions based on their criticality and likelihood of failure. Risk-based testing determines which test activities will be completed for the iteration. It is a transparent testing methodology that tells the customer exactly which test activities are executed for every feature. 9

Slide 11

Criticality Analysis and Risk Assessment Standardized risk assessment methodology. Features are consistent with PMIs Risk Management assessment philosophy, PMBOK Guide. The uniqueness is in the process of determining the impact and probability. Standardized criteria for determining risk values. CARA does not tell us how to test. 10 IMPACT Probability LowModerateHigh Catastrophic Critical Moderate Low

Slide 12

11

Slide 13

Criticality is broken down into 3 categories: Performance & Operations Safety Cost of Failure/Impact to Schedule Requirements for each category are scored on a scale of 1-4 1 = Lowest 2 = Moderate 3 = Critical 4 = Catastrophic 12

Slide 14

Risk is broken down into 5 categories: Complexity Technology Maturity Requirements Definition and Stability Testability System Characterization Requirements for each category are scored on a scale of 1-3 1 = Low 2 = Moderate 3 = High 13

Slide 15

Scores for Risk and Criticality Categories are separately averaged to determine a weighted value The weighted value scores are then multiplied to determine the IV&V Analysis Level (IAL) Threshold IALs are broken down into 4 categories: Minimal: 1CARA