Risk ManagementRisk Management

Project ManagementProject Management

Testing Effort ManagementTesting Effort Management

AdministriviaAdministrivia

►BathroomsBathrooms►LunchLunch► IntroductionsIntroductions►Downloads are available at Downloads are available at

http://turningwheel.net/ppasqhttp://turningwheel.net/ppasq

ObjectivesObjectives

► At the end of this class, you should be able to:At the end of this class, you should be able to:► Identify events (risks) that are likely to impact a project Identify events (risks) that are likely to impact a project

and document their characteristicsand document their characteristics► Assess each identified risk to determine the impact it Assess each identified risk to determine the impact it

could have to the project and the probability of it could have to the project and the probability of it occurringoccurring

► Determine a priority ranking for all the risks identifiedDetermine a priority ranking for all the risks identified► Select a mitigation approach for each risk identifiedSelect a mitigation approach for each risk identified► Develop mitigation and /or contingency plans as Develop mitigation and /or contingency plans as

necessary for the risks identifiednecessary for the risks identified► Monitor and evaluate the project for occurrence of or Monitor and evaluate the project for occurrence of or

changes to the risks identifiedchanges to the risks identified

What is Risk Management?What is Risk Management?

►What makes a project a success?What makes a project a success? 70-80% of IT projects fail!!70-80% of IT projects fail!!

►The “Law” of Project Management…The “Law” of Project Management…

What is “Risk Management”What is “Risk Management”

► Risk management is concerned with identifying risks Risk management is concerned with identifying risks and drawing up plans to minimise their effect on a and drawing up plans to minimise their effect on a project.project.

► A risk is a probability that some adverse circumstance A risk is a probability that some adverse circumstance will occur. will occur. Project risks affect schedule or resourcesProject risks affect schedule or resources Product risks affect the quality or performance of the Product risks affect the quality or performance of the

software being developedsoftware being developed Business risks affect the organisation developing or Business risks affect the organisation developing or

procuring the softwareprocuring the software► Determining how to react/avoid the impactDetermining how to react/avoid the impact► Monitoring the events throughout the life of the projectMonitoring the events throughout the life of the project

Benefits of Risk AnalysisBenefits of Risk Analysis

►Best PracticeBest Practice ““same page” for scope and prioritiessame page” for scope and priorities

►Focuses on the “most important” workFocuses on the “most important” work►Publishes “assumptions”Publishes “assumptions”►Promotes reuse of Quality materialsPromotes reuse of Quality materials►Works for both MFI and MFF projectsWorks for both MFI and MFF projects

(“Money For Information” and (“Money For Information” and …”Flexibility”)…”Flexibility”)

Risk analysisRisk analysis

►Assess probability and seriousness of Assess probability and seriousness of each riskeach risk

►Probability may be very low, low, Probability may be very low, low, moderate, high or very highmoderate, high or very high

►Risk effects might be catastrophic, Risk effects might be catastrophic, serious, tolerable or insignificantserious, tolerable or insignificant

Risk analysisRisk analysis

Risk Probability EffectsOrganisational financial problems force reductionsin the project budget.

Low Catastrophic

It is impossible to recruit staff with the skillsrequired for the project.

High Catastrophic

Key staff are ill at critical times in the project. Moderate SeriousSoftware components which should be reusedcontain defects which limit their functionality.

Moderate Serious

Changes to requirements which require majordesign rework are proposed.

Moderate Serious

The organisation is restructured so that differentmanagement are responsible for the project.

High Serious

The database used in the system cannot process asmany transactions per second as expected.

Moderate Serious

The time required to develop the software isunderestimated.

High Serious

CASE tools cannot be integrated. High TolerableCustomers fail to understand the impact ofrequirements changes.

Moderate Tolerable

Required training for staff is not available. Moderate TolerableThe rate of defect repair is underestimated. Moderate TolerableThe size of the software is underestimated. High TolerableThe code generated by CASE tools is inefficient. Moderate Insignificant

The risk management The risk management processprocess

►Risk identificationRisk identification Identify project, product and business risksIdentify project, product and business risks

►Risk analysisRisk analysis Assess the likelihood and consequences of Assess the likelihood and consequences of

these risksthese risks

►Risk planningRisk planning Draw up plans to avoid or minimise the effects Draw up plans to avoid or minimise the effects

of the riskof the risk

►Risk monitoringRisk monitoring Monitor the risks throughout the projectMonitor the risks throughout the project

The risk management The risk management processprocess

►The Process flowThe Process flow 1 Risk Identification1 Risk Identification 2 Risk Assessment2 Risk Assessment 3 Response Planning3 Response Planning 4 Planning Completion4 Planning Completion 5 Risk Monitoring5 Risk Monitoring 6 Risk Response6 Risk Response 7 Update Risk Management Plan7 Update Risk Management Plan

The risk management The risk management processprocess

Risk IdentificationRisk Identification

►The process of determining which The process of determining which events might affect the project and events might affect the project and documenting their characteristicsdocumenting their characteristics

Risk Identification StepsRisk Identification Steps

►1.1 Comparison to prior, similar 1.1 Comparison to prior, similar projects’ identified risksprojects’ identified risks Identify similar, previous projects based Identify similar, previous projects based

on subject matteron subject matter Obtain PIRs, Lessons Learned, Risk Obtain PIRs, Lessons Learned, Risk

Management Plan, Issues Logs, etcManagement Plan, Issues Logs, etc Review and document potential events for Review and document potential events for

your projectyour project SWOT analysisSWOT analysis

Risk Identification StepsRisk Identification Steps

►1.2 Identify events that are likely to 1.2 Identify events that are likely to impact the projectimpact the project What methods have you successfully used What methods have you successfully used

to identify risks?to identify risks?

Use brainstorming, affinity diagrams, Use brainstorming, affinity diagrams, interviewing, dependency modeling, interviewing, dependency modeling, questionnaires, delphi technique, questionnaires, delphi technique, prototypingprototyping

Risk Identification StepsRisk Identification Steps

►1.3 Categorize identified events1.3 Categorize identified events Grouping risks into categories will provide Grouping risks into categories will provide

valuable information later in the processvaluable information later in the process Most common categoriesMost common categories

►Business / Organizational RisksBusiness / Organizational Risks►External RisksExternal Risks►Project Management RisksProject Management Risks►Technical, Quality, or Performance RisksTechnical, Quality, or Performance Risks►Estimation RisksEstimation Risks

Risks and risk typesRisks and risk types

Risk type Possible risksTechnology The database used in the system cannot process as many

transactions per second as expected.Software components which should be reused contain defectswhich limit their functionality.

People It is impossible to recruit staff with the skills required.Key staff are ill and unavailable at critical times.Required training for staff is not available.

Organisational The organisation is restructured so that different managementare responsible for the project.Organisational financial problems force reductions in the projectbudget.

Tools The code generated by CASE tools is inefficient.CASE tools cannot be integrated.

Requirements Changes to requirements which require major design rework areproposed.Customers fail to understand the impact of requirementschanges.

Estimation The time required to develop the software is underestimated.The rate of defect repair is underestimated.The size of the software is underestimated.

Risk factorsRisk factors

Risk type Potential indicatorsTechnology Late delivery of hardware or support software, many

reported technology problemsPeople Poor staff morale, poor relationships amongst team

member, job availabilityOrganisational organisational gossip, lack of action by senior

managementTools reluctance by team members to use tools, complaints

about CASE tools, demands for higher-poweredworkstations

Requirements many requirements change requests, customercomplaints

Estimation failure to meet agreed schedule, failure to clearreported defects

Software risksSoftware risks

Risk Risk type DescriptionStaff turnover Project Experienced staff will leave the

project before it is finished.Management change Project There will be a change of

organisational management withdifferent priorities.

Hardware unavailability Project Hardware which is essential for theproject will not be delivered onschedule.

Requirements change Project andproduct

There will be a larger number ofchanges to the requirements thananticipated.

Specification delays Project andproduct

Specifications of essential interfacesare not available on schedule

Size underestimate Project andproduct

The size of the system has beenunderestimated.

CASE tool under-performance

Product CASE tools which support theproject do not perform as anticipated

Technology change Business The underlying technology on whichthe system is built is superseded bynew technology.

Product competition Business A competitive product is marketedbefore the system is completed.

Let’s Try ItLet’s Try It

Exercise 1Exercise 1

Risk Identification StepsRisk Identification Steps

►1.3 Categorize identified events1.3 Categorize identified events Along with selecting a category for each Along with selecting a category for each

task, the following should also be task, the following should also be identified and documented for each risk:identified and documented for each risk:►TriggersTriggers►AssumptionsAssumptions►Preliminary OwnerPreliminary Owner

Let’s Try ItLet’s Try It

Exercise 2Exercise 2

Risk AssessmentRisk Assessment

►The process of analyzing identified The process of analyzing identified risks in order to determine the risks in order to determine the likelihood of a risk occurring (the likelihood of a risk occurring (the probability), probability),

►the severity of the risk (the impact), the severity of the risk (the impact), ►and the potential cost to the overall and the potential cost to the overall

project (net exposure)project (net exposure)

Risk Assessment StepsRisk Assessment Steps

►2.1 Comparison to prior projects’ 2.1 Comparison to prior projects’ identified risksidentified risks Obtain PIRs, Lessons Learned, Risk Obtain PIRs, Lessons Learned, Risk

Management Plan, etcManagement Plan, etc Review assessment of similar risks for Review assessment of similar risks for

these projectsthese projects

Risk Assessment StepsRisk Assessment Steps

►2.2 Qualitative Risk Analysis of each 2.2 Qualitative Risk Analysis of each riskrisk Appendix B for small projectsAppendix B for small projects Appendix C for medium to large projectsAppendix C for medium to large projects Appendix D for programsAppendix D for programs

►Qualitative Risk Analysis Voting Qualitative Risk Analysis Voting ProcessProcess One vote for each impact and probability One vote for each impact and probability

(Agile)(Agile)

Let’s Try It!Let’s Try It!

Exercise 3Exercise 3

Risk Assessment StepsRisk Assessment Steps

►Quantitative Risk Analysis Quantitative Risk Analysis See Appendix B, C, DSee Appendix B, C, D

Risk Assessment StepsRisk Assessment Steps

►2.4 Rank Risks2.4 Rank Risks Determine the ranking of each risk, Determine the ranking of each risk,

producing a prioritized listproducing a prioritized list

Samples available atSamples available at

http://turningwheel.net/ppasqhttp://turningwheel.net/ppasq

Let’s Try ItLet’s Try It

Exercise 4Exercise 4

Risk Response PlanningRisk Response Planning

►The process of developing options and The process of developing options and determining actions and activities to determining actions and activities to reduce risk impact, probability, and reduce risk impact, probability, and exposure to the project’s objectivesexposure to the project’s objectives

Risk Response StepsRisk Response Steps

►3.1 Comparison to previous projects3.1 Comparison to previous projects►3.2 Assign a mitigation approach to 3.2 Assign a mitigation approach to

each risk (Reactive, Proactive)each risk (Reactive, Proactive)►3.3 Develop mitigation/contingency 3.3 Develop mitigation/contingency

plans for each riskplans for each risk►3.4 Finalize owner(s) of risks based on 3.4 Finalize owner(s) of risks based on

mitigation plansmitigation plans

Risk management strategiesRisk management strategies

Risk StrategyOrganisationalfinancial problems

Prepare a briefing document for senior management showinghow the project is making a very important contribution to thegoals of the business.

Recruitmentproblems

Alert customer of potential difficulties and the possibility ofdelays, investigate buying-in components.

Staff illness Reorganise team so that there is more overlap of work andpeople therefore understand each other’s jobs.

Defectivecomponents

Replace potentially defective components with bought-incomponents of known reliability.

Requirementschanges

Derive traceability information to assess requirements changeimpact, maximise information hiding in the design.

Organisationalrestructuring

Prepare a briefing document for senior management showinghow the project is making a very important contribution to thegoals of the business.

Databaseperformance

Investigate the possibility of buying a higher-performancedatabase.

Underestimateddevelopment time

Investigate buying in components, investigate use of a programgenerator.

Planning CompletionPlanning Completion

►The process of finalizing the Risk The process of finalizing the Risk Management Plan that was developed Management Plan that was developed during the risk planning processduring the risk planning process

Planning Completion StepsPlanning Completion Steps

►4.1 Obtain formal signoffs4.1 Obtain formal signoffs This signoff signifies that the stakeholders This signoff signifies that the stakeholders

agree with the content of the Risk agree with the content of the Risk Management PlanManagement Plan

►4.2 Execute mitigation plans4.2 Execute mitigation plans Begin execution of the actions or activities Begin execution of the actions or activities

defined in the mitigation plans developed defined in the mitigation plans developed during Risk Response Planningduring Risk Response Planning

Let’s Try ItLet’s Try It

Exercise 5Exercise 5

Risk MonitoringRisk Monitoring

►The process of the Risk Management Plan The process of the Risk Management Plan owner keeping track of the identified risksowner keeping track of the identified risks

►5.1 Monitor existing risks for occurrence or 5.1 Monitor existing risks for occurrence or changechange Use one or more of the following:Use one or more of the following:

►Periodic risk management plan reviewsPeriodic risk management plan reviews►Performance MeasurementsPerformance Measurements

Each key risk should be discussed at management Each key risk should be discussed at management progress meetingsprogress meetings

Risk ResponseRisk Response

►The process of executing the The process of executing the necessary contingency plan(s) once a necessary contingency plan(s) once a risk has occurredrisk has occurred

►6.1 Execute the risk contingency plan6.1 Execute the risk contingency plan Begin execution of the defined activitiesBegin execution of the defined activities

Update Risk Management Update Risk Management PlanPlan

►The process of modifying the Risk The process of modifying the Risk Management Plan with changes that Management Plan with changes that occur during the life of the projectoccur during the life of the project

Update RM Plan StepsUpdate RM Plan Steps

►7.1 Update Risk Details on RM Plan7.1 Update Risk Details on RM Plan RM Plan owner updates RM Plan based on RM Plan owner updates RM Plan based on

changes identified during monitoring and changes identified during monitoring and control phasescontrol phases

►7.2 Communicate changes to RM Plan7.2 Communicate changes to RM Plan►7.3 Obtain formal signoffs on changes 7.3 Obtain formal signoffs on changes

to RM Planto RM Plan Signoff indicates that stakeholders agree Signoff indicates that stakeholders agree

with changeswith changes

Let’s Try ItLet’s Try It

Exercise 6Exercise 6

What’re You Gonna Test?What’re You Gonna Test?

►““The Most Important Things” (MITs)The Most Important Things” (MITs)►Severity is how bad it’ll hurt if Severity is how bad it’ll hurt if

something happenssomething happens►Probability is how likely it is to happen.Probability is how likely it is to happen.►A Meteorite hitting the building is A Meteorite hitting the building is

unlikely but catastrophic if it happensunlikely but catastrophic if it happens

What’re You Gonna Test?What’re You Gonna Test?

►Risks Analysis can help:Risks Analysis can help: Define the Initial Test ScheduleDefine the Initial Test Schedule Form the Contract to TestForm the Contract to Test Analyze the Results of Tests (to prove Analyze the Results of Tests (to prove

they were useful)they were useful) Determine which tests to run (coverage)Determine which tests to run (coverage) Determine how hard to hit a testDetermine how hard to hit a test

The MITs ApproachThe MITs Approach

►More refined than the last attemptMore refined than the last attempt►Weights an index as wellWeights an index as well

Report Your FindingsReport Your Findings

►Explain your approach but also show Explain your approach but also show the danger of not testingthe danger of not testing

►Cost AnalysisCost Analysis TimeTime Potential Customer ImpactPotential Customer Impact

So, What’re You Gonna Test?So, What’re You Gonna Test?

►You can’t test everythingYou can’t test everything► Inventory RankingInventory Ranking►Test SizingTest Sizing►Risk Analysis answers:Risk Analysis answers:

What do I need to test and how?What do I need to test and how? How big is the test effort?How big is the test effort? How much will it cost?How much will it cost?

Applying Risk AnalysisApplying Risk Analysis

►Planning Phase: quick estimate of the Planning Phase: quick estimate of the number and types of testsnumber and types of tests

►Assumes there is a testing inventoryAssumes there is a testing inventory Path vs Data Testing in SoftwarePath vs Data Testing in Software Most Important Nonanalytical Tests -from SMEsMost Important Nonanalytical Tests -from SMEs Most Important Paths –Logic of the UserMost Important Paths –Logic of the User Most Important Data –Most Pass Data SetsMost Important Data –Most Pass Data Sets Most Important Environments –Most Important Environments – MITs= (MINs+MIPs+MIDs)*MIEsMITs= (MINs+MIPs+MIDs)*MIEs

The Sizing WorksheetThe Sizing Worksheet

►Like test coverage, these are relative Like test coverage, these are relative estimatesestimates

►Should contain time takenShould contain time taken►And if lucky, cost to perform (or not)And if lucky, cost to perform (or not)►Assumptions should be notedAssumptions should be noted►Estimates need to be updated as Estimates need to be updated as

actuals are learned for improving actuals are learned for improving future estimatesfuture estimates

Sizing Worksheet ContainsSizing Worksheet Contains

►MITs Test and CoverageMITs Test and Coverage►Test Units and Time to Create TestsTest Units and Time to Create Tests►Time to Run and Create Automated TestsTime to Run and Create Automated Tests►Estimate the number of errors to be Estimate the number of errors to be

foundfound►Code Turnovers/Test Cycles/IterationsCode Turnovers/Test Cycles/Iterations►Test Environments and Total TestsTest Environments and Total Tests►Planning TimePlanning Time

Sizing Worksheet Contains 2Sizing Worksheet Contains 2

►The Case for Automation (if applicable)The Case for Automation (if applicable)►Time for Administration, Time for Administration,

Documentation, and LoggingDocumentation, and Logging►Factor of SafetyFactor of Safety

50% is not unreasonable for large efforts50% is not unreasonable for large efforts

►Constraints, Assumptions, and StatusConstraints, Assumptions, and Status

Negotiating the EffortNegotiating the Effort

►Don’t forget to budget time for *fixing* Don’t forget to budget time for *fixing* bugs!bugs!

►Agile/XP MethodologiesAgile/XP Methodologies