the death of a project: can lessons from yesterday’s catastrophe prevent tomorrow’s disaster?...

The Death of a Project: Can lessons from yesterday’s catastrophe prevent tomorrow’s disaster?

Matthew GilleryProject Manager

Hewlett-Packard

Agenda

• Overview/Purpose• What is a project?• What is project management?• Introducing Risk Management• Defining Project Success• Common Causes of Project Failure• Role of Risk Management in Reducing Failure Rates• Risk Classification: ABCD Methodology

Overview/Purpose

• Problem Statement– The purpose of this presentation is to identify the

common causes of project failure, as suggested by members of the academy and industry leaders.

– Clearly, there is no “one size fits all” approach/solution to preventing project failure. However, the presentation focuses on how risk management can be used to reduce the likelihood/impact of project failure.

– Key question: Are we learning from yesterday’s mistakes?

Project Failure in the 70’s

• In 1979, IEEE published an analysis on failed “Bankrupt” projects and reported:

– “Bankrupt” projects are those that consistently missed target dates and resulted in cost over-runs, or cancellations

– IEEE argued:

• By the time the paper was written, the project “bankruptcy” phenomena hadn’t been analyzed successfully

• Occurrence of “bankruptcies” are rarely detected early in the development stage. Most are identified during testing.

• Effective methods for preventing “bankruptcies” are still being developed

An Analysis of Software Project Failure. IEEE, 1979.

Standish CHAOS Report 2009Is Project Success Really that Rare?

• Specifically, 32 percent of IT projects were considered successful, having been completed on time, on budget and with the required features and functions.

• Nearly one-in-four (24 percent) IT projects were considered failures, having been cancelled before they were completed, or having been delivered but never used.

• The rest (44 percent) were considered challenged: They were finished late, over budget, or with fewer than the required features and functions.

http://www.cio.com/article/495306/Recession_Causes_Rising_IT_Project_Failure_Rates_

The Project Management Problem

The Standish Group has reported:– IT project success rates rose steadily from 1994 until

2000, when they dipped, and then began rising again from 2002 through 2006. Success rates decreased between 2006 and 2009.

– Many projects experience governance issues: • Project take so long to complete that stakeholders lose

interest and eventually decide to cancel it, or a project that eventually gets delivered but doesn't get used because it's no longer relevant to the business. In both situations, the project is considered a failure.

http://www.cio.com/article/495306/Recession_Causes_Rising_IT_Project_Failure_Rates_

What is a project?• Temporary endeavor with a beginning and an end.

– The end is reached when the objectives have (or cannot) be achieved.– Temporary DOES NOT mean short in duration; many projects last for

several years but they do, eventually end.• Creates a unique product, service or result.

– It’s unique in that it has not been attempted before by the organization.– Creates a product or artifact, is quantifiable and can be either and end

item itself or a component of another item.– Performs a service such as business functions that support shipping

and receiving.– Delivers a result such as outcomes or documents like results from a

research project on drinking water conditions in mountainous regions of Ecuador and Chile.

• Is progressively elaborated – distinguishing characteristics of each unique project will be progressively detailed as the project is better understood (matures).

• It is comprised of interrelated activities.

Crowe. The PMP Exam. p9. Mulcahy, PMP Exam Prep, p22 PMBOK p5-7.

What is Project Management?• Project management is the application of knowledge, skills, tools

and techniques to project activities to meet project requirements. – Project management is accomplished through the application and

integration of the project management processes of initiating, planning, executing, monitoring and controlling, and closing.

– The project manager is the person responsible for accomplishing the project objectives.

• Managing a project includes: – Identifying requirements – Establishing clear and achievable objectives – Balancing the competing demands for quality, scope, time and cost – Adapting the specifications, plans, and approach to the different

concerns and expectations of the various stakeholders.

PMBOK p8.

Progressive Elaboration & Risk Management

• The term “progressive elaboration” simply means that you do not know all of the characteristics about a product when you begin the project.– Instead, they may be revisited

often and refined.– The characteristics of the

product emerge over time, or “progressively.”

• Moving forward on a project without a proactive focus on risk management increases the impact that a realized risk can have on the project and can potentially lead to project failure.

Stakeholder

Influence

Low

High

Early

PHASES Late

Conceptual ConstructPlanning Testing ClosureImplement

Risk to Project

Success

Staffing, Resources & Project Costs

Crowe. The PMP Exam. p20. PMBOK p6 & 8.

Defining the term “project”

“A project is a temporary endeavor undertaken to create a unique product, service or result.”

In answer to the question, WHAT IS A PROJECT?Temporary, Unique.

Crowe. The PMP Exam. p9. Mulcahy, PMP Exam Prep, p22 PMBOK p5-7.

Risk ManagementWhat is Risk Management?

• Risk is an uncertain event or condition, that if it occurs, may have a positive or negative effect on project objectives. Project risk is always in the future.

• Risk management is the systematic process of planning how to manage, identify, analyze, respond to, monitor and control project risks.

• The objectives of Project Risk Management are to increase the probability and impact of positive events and decrease the probability and impact of negative events on project objectives.

• A cause may be a requirement, an assumption, constraint or condition that creates the possibility of negative or positive outcomes.

– Positive risks = Opportunities– Negative risks = Threats– Known Risks = those that have been identified, analyzed making it possible to

plan responses for those risks– Unknown Risks = cannot be managed proactive, so need a contingency plan– Risk Tolerance = varying degrees of risk that organizations and stakeholders are

willing to accept

• NOTE: A project risk that has occurred can also be considered an issue

Project Risk

• Project risk is always in the future. Risk is an uncertain event or condition that, if it occurs, has an effect on at least one project objective. – Cost– Time– Scope– Quality

• A cause may be a requirement, assumption, constraint, or condition that creates the possibility of negative or positive outcomes.

Risk Management Objectives

• The objectives of Project Risk Management are to:– increase the probability and impact of

positive events– decrease the probability and impact of

negative events in the project.

Risk Example

University of Toronto http://www.cdf.toronto.edu/~csc340h/summer/lectures/w6/L6-part2-risk.pdf

What risk(s) is/are evident in this example?

Risk Semantics

• Risks are typically expressed in the form of conditional statements– If A occurs, then B…..

• Technology company example:– If project resources are not

trained in the application of technology OO prior to project start, then the project may be delayed and experience cost over-runs as a result of on-the-job training

Risk Management Problem

• There is an unwillingness to admit risks exists. • There is a lack of understanding of the benefits.• There is a natural tendency to postpone the hard parts of

a project. (i.e., Do the easy things first.)• Some believe that it costs time up front without adding

value overall.• It is difficult to prove that it’s necessary (e.g., like

insurance).

There are several reasons why risk management is sometimes not undertaken including:

Risk Management

• Risk Management includes the processes of conducting: – Risk management planning– Identification– Analysis– Response Planning– Monitoring and Control

Key Project Success Considerations

• How do we define project success?

• How do we get the project team engaged in the risk management process?

What is success?

• Answering “yes” to the following questions:– Did the project meet its objectives, requirements, budget and

schedule?– Did the project meet the customer’s needs?– Was the decision to proceed with the project correct at the time?– Would we do the project again, knowing what we know now?

• A project is a success if it meets its objectives, requirements, budget and schedule, and a failure otherwise

Society for Automotive Engineers International Conference on Environment Systems: Explaining Space Project Failures

A perspective on “Good” vs. “Bad” Projects

• A project that takes reasonable risks and fails by bad luck is not a bad project, since rational decision makes would repeat the attempt

• A project that takes an unreasonable risk and yet meets goals by chance is, conversely, not a good project

• Key question: What is a “failed” project?


Failed Project Defined

• A failed project does not successfully meet requirements/objectives

Common Causes of Project FailureStudy conducted by KPMG found several risk factors that lead to project failure. While focused on the IT industry, these risk factors are general. Project managers were surveyed. Responses were grouped, based on the experience level of the project manager.

Controlling Software Project Risks- an Empirical Study of Methods used by Experienced Project Managers. Addison & Vallabh. SAICSIT 2002, pp. 128-140

Anatomy of a Failed Project. 2008. http://blogs.zdnet.com/projectfailures/?p=1100

FBI Trilogy Project• Goal

– Upgrade FBI’s technology infrastructure to speed up data entry and analysis• Scope

– Deployment of:• An enterprise-wide upgrade if desktop hardware and software• Modern network infrastructure• An integrated suite of software for entering, finding, sharing, and analyzing

case information ($170M/$581M Trilogy project cost)

• History– In 2000, started as the UAC project to modernize the FBI’s Automated Case

System (ACS) and technological infrastructure, by “webifying” the system’s green screens

• Problem: Updating the system’s screens didn’t fix the underlying business process issues and architecture

– System was built on 1970’s standards and equipment from the late 1980’s

– In 2001, the FBI contracted with IT service providers (DynCorp and SAIC) to upgrade the FBI’s after congress approved $379.8 million for the FBI Technology Upgrade Project (FITUP)

http://www.infoworld.com/d/developer-world/anatomy-it-disaster-how-fbi-blew-it-243

FBI Trilogy Project– In 2001, scope changed form beautifying mainframe screens to replacing them

with an web application for gathering and analyzing intelligence data. Created the Virtual Case File Project (VCF)

• Requirements were difficult to “nail down,” since organizational processes were constantly changing as a result of 9/11.

• Contractors were rated based on customer satisfaction, and continuously accepted new requirements

– At least 30-31 changes were submitted each month– Endless change resulted in communications issues

(misunderstandings)

– In December 2003, SAIC deployed an incomplete system. The FBI was expecting the full version.

• Leadership at all levels was surprised. The FBI had undergone two CIO changes.

– In 2004, FBI and Homeland Security began planning an inter-agency Federal Investigative Case Management System (FICMS), which would make VCF obsolete

• network infrastructure and desktop hardware/software upgrades were completed.

– In 2005, the project was officially cancelled. The software development project was not making progress, which continuously resulted in cost and schedule over-runs


FBI Trilogy Project Issues

Issues• Lack of internal IT expertise

– Management and IT support• Scope Creep/Poor Change Management

– Key change in direction after one year: web application instead of upgraded screens

• Leadership Changes– Inconsistent IT leadership: FBI experiencd 5 CIO changes

between 2000 and 2005; each had their own vision for the project


Trilogy Replaced by Sentinel

• As of 2006, a new contract was awarded to Lockheed Martin. The Trilogy Project was renamed to become the “Sentinel Project”. – At this point, the project was projected to be complete by May

2010, with an estimated cost of $425 million. • This means that over $500 million would be spent on a

project that was estimated at $170 million in 2001.

http://www.govexec.com/dailyfed/0708/071608rb1.htm

http://www.govexec.com/dailyfed/0708/071608rb1.htm

Sentinel Status: November 2009Following is in response to the Department of Justice’s Office of the Inspector General (OIG)

report, “Sentinel Audit V: Status of the FBI’s Case Management System.” • “The FBI appreciates the Inspector General’s review of the FBI’s Sentinel Program

progress and recognition of the FBI’s efforts to resolve concerns identified in previous Sentinel audits. As the OIG notes, the FBI estimates that Sentinel is scheduled to deliver full capability within the $451 million budget in the fall of 2010. The FBI already has implemented measures to resolve all six recommendations identified in this report and has successfully closed 30 of the 31 recommendations from the four prior OIG Sentinel audits.

• “The Sentinel program has steadily improved and refined its business practices. During Phase 1 of this project, the FBI and its primary contractor, Lockheed Martin, chose to redesign and re-baseline Phase 2 to be more efficient and deliver additional capability to users earlier than originally planned. This approach reduces overall program risk. It was carefully planned and incorporated a flexible aspect to the design to further mitigate risk by shifting requirements forward in the program’s development to meet user needs.

• “The FBI is pleased the report concluded the revised Sentinel schedule is more realistic. By extending the completion of Phase 2 by three months, requirements from the latter phases will be delivered earlier, providing capabilities to users sooner than originally planned.

• “The FBI has begun user testing of Sentinel’s Administrative Case Management system, including three electronic forms and automated workflows. Following a brief pilot in the FBI’s Richmond, Tampa, and Chicago offices, the FBI will deliver these capabilities across the FBI, marking the completion of Phase 2.”

http://www.fbi.gov/pressrel/pressrel09/oigsentinel111009.htm

Sentinel: Current Status• As of April 1, 2010:

– The Associated Press reports that the FBI's $451 million Sentinel case management system is getting slower and costing more, according to a Justice Department audit.

– The project was expected to be complete by September of this year, but FBI Director Robert Mueller said it would be delayed until 2011. The bureau is renegotiating the budget with Lockheed Martin, as well as the schedule and some of the work to be performed.

• As of April 16, 2010:– FBI director reported, " When you have a project that was laid down in concrete

four or five years ago, [with] technology changes, business practice changes,

and complexity changes, one can expect some minor delays" – FBI Director

http://www.itbusinessedge.com/cm/community/news/gt/blog/audit-fbi-sentinel-system-facing-more-costs-delays/?cs=40450

http://www.informationweek.com/news/government/enterprise-apps/showArticle.jhtml?articleID=224400547

NASA Failed Missions (Projects)

• NASA has committed itself to organizational and project management improvement, by analyzing over-looked risk factors on the following space programs:

Program Physical Cause Date

Columbia

During re-entry, there was a breach in thermal protection shield resulted in the superheating of insulation, which melted the left wing and destroyed the orbiter. 2/1/2003

Challenger

O-ring failed in the right solid rocket booster and led to a flare that reached the external fuel tank, which disintegrated the shuttle. The cabin and crew hit the ocen at an unsurvivable speed 2 minuted and 45 seconds into the flight. 1/28/1986

Apollo Apollo 1: Fire. Apollo 13: systems failure. 2/21/1967


NASA Project/Organization Problems

• Engineering quality/risk management error– “None of us gave any serious consideration to a fire in the

spacecraft”» Astronaut Frank Borman, Apollo 1 Review Board

• Lack of cross-functional coordination/informal work-arounds– Process changes were made at the work group level, and not

considered during projects (Change Management/Communication)

• Failed to ask basic questions– Example: Challenger. Per NASA study, “No launch incident

escape system was provided because of the incorrect assumption that the shuttle had high reliability” (SAE)

• Management failed to ask the question: What happens if the flight crew needs to escape during a launch?

• Stated as a risk: If the flight is not able to escape during a launch, then…..


NASA: Recommended Remedies

Risk Management as remedy for NASA’s problems


Risk ManagementWhat is Risk Management?

• Risk is an uncertain event or condition, that if it occurs, may have a positive or negative effect on project objectives. Project risk is always in the future.

• Risk management is the systematic process of planning how to manage, identify, analyze, respond to, monitor and control project risks.

• The objectives of Project Risk Management are to increase the probability and impact of positive events and decrease the probability and impact of negative events on project objectives.

• A cause may be a requirement, an assumption, constraint or condition that creates the possibility of negative or positive outcomes.

– Positive risks = Opportunities– Negative risks = Threats– Known Risks = those that have been identified, analyzed making it possible to

plan responses for those risks– Unknown Risks = cannot be managed proactive, so need a contingency plan– Risk Tolerance = varying degrees of risk that organizations and stakeholders are

willing to accept

• NOTE: A project risk that has occurred can also be considered an issue

Role of Risk Management in Reducing Project Failure Rates

• Moving forward on a project without a proactive focus on risk management increases the impact that a realized risk can have on the project and can potentially lead to project failure.

• The intent of risk management practices is to promote a proactive stance towards reducing project failure

Risk Classification: ABCD

• Risk management must be a team effort!• “Research suggests that a successful project

environment may be characterized as on in which all systems professionals maintain a holistic view of organizational risk….”

» Merrill Warkentin, Mississippi State University

• The challenge for project managers is to get the entire project team actively engaged in risk management

Scope of Risk Management

TOTALCERTAINTY

TOTALUNCERTAINTY

GENERALUNCERTAINTY

SPECIFICUNCERTAINTY

(Unknownunknowns) (Knowns)

NoInformation

CompleteInformation

PartialInformation

(Knownunknowns)

SCOPE OF PROJECT RISK MANAGEMENT

Risk Management Processes

Major Inputs

1. Project Scope Statement2. Cost Mgmt, Schedule Mgmt

Communications Mgmt Plans3.Enterprise Environmental Factors4.Organizational Process Assets

Tools & Techniques

1. Planning Meetings & Analysis

Major Outputs

1. Risk Management Plan

11.1 Plan Risk Management (P)11.1 Plan Risk Management (P)

Major Inputs

1.Risk Management Plan2. Cost Mgmt, Schedule Mgmt,

Quality Mgmt Plans3.Scope Baseline4.Activity cost estimates5.Activity duration estimates6.Enterprise Environmental Factors7.Organizational Process Assets

Tools & Techniques

1. Documentation Reviews2. Info Gathering Techniques3. Checklist Analysis4. Assumption Analysis5.Expert Judgment6.SWOT Analysis7.Diagramming Techniques

Major Outputs

1. Risk Register

11.2 Identify Risks (P)11.2 Identify Risks (P)

Major Inputs

1. Risk Register2. Risk Management Plan3. Project Scope Statement4.Organizational Process Assets

Tools & Techniques

1. Risk Probability and Impact Assessment

2. Probability and Impact Matrix3.Risk Data Quality Assessment4. Risk Categorization5.Risk Urgency Assessment6.Expert Judgment

Major Outputs

1. Risk Register Updates

11.3 Perform Qualitative Risk Analysis (P)11.3 Perform Qualitative Risk Analysis (P)

Risk Management Processes

Major Inputs

1.Risk Register2.Risk Management Plan3.Schedule Mgmt Plan4.Cost Mgmt Plan5.Organizational Process Assets

Tools & Techniques

1. Data Gathering and representation techniques

2. Quantitative Risk Analysis and Modeling Techniques

3.Expert Judgment

Major Outputs

1. Risk Register Updates

11.4 Perform Quantitative Risk Analysis (P)11.4 Perform Quantitative Risk Analysis (P)

Major Inputs

1. Risk Register 2.Risk Management Plan

Tools & Techniques

1. Strategies for Negative Risks (Threats)

2. Strategies for Pos Risks (Opportunities)

3. Contingent Response Strategies4. Expert Judgment

Major Outputs

1. Risk Register Updates2.Risk-related Contract Decisions3. Project Management Plan

Updates4.Project Document Updates

11.5 Plan Risk Responses (P)11.5 Plan Risk Responses (P)

Major Inputs

1.Risk Register2.Project Management Plan3.Work Performance Information4.Performance Reports

Tools & Techniques

1. Risk Reassessment2. Risk Audits3.Variance & Trend Analysis4.Technical Performance

Measurement5.Reserve Analysis6.Status Meetings

Major Outputs

1. Risk Register Updates2. Organizational Process Asset

Updates3.Change Requests4.Project Management Plan

Updates5.Project Documentation Updates

11.6 Monitoring and Control Risks (M&C)11.6 Monitoring and Control Risks (M&C)

Risk Breakdown Structure

Architecture

Permits

MaterialsPersonnelBlueprints

HVAC (Climate Control)

Plumbing Electrical

Personnel doesn’t change

Customer Confirms Design

MaterialsElectricity at

location

PermitsContractor Selected

Permits

Materials

Risk Breakdown Structure (RBS) for Construction Project

• The RBS is a decomposition of the risk categorization and the risks within those categories that could occur on a project.

Classifying Risks

•Business risks vs. pure (insurable) risks•Classified by uncertainty (business risks)•Classified by impact on project elements•Classified by their nature•Classified by their source•Classified by their probability to occur and amount at stake

Risk Register & ABCD RatingId Description of Risk

Identify consequences [1]

Probability

Impact Grade Change

Mitigation Actions Risk Owner Cost WBS [2]

1.1 Inadequate funding to complete the project

M M B NEW

Re-scope project, focusing on time and resourcing

Project Manager NA

1.2 Lack of technical skills in Client Business Unit

H H A ↑ Develop training plan Consultant $2000

[1] In larger projects, the consequences of the threat may not be evident, and noting them under each risk, or in a separate column can be useful in identifying appropriate mitigation actions.[2] WBS = Work Breakdown Structure, this is to indicate that the identified mitigation action has been included in the WBS (workplan).

Source: PM 007 Project Risk Register Template and Guide:www.egovernment.tas.gov.au/__data/assets/word_doc/18512/pman-temp-open-risk-register.doc

ABCD Risk Management Overview

http://www.de-risk.com/page.php?7

ABCD Framework

A= =

B= =

C= =

D= =

Sensitivity

How sensitive to the project is the assumption?

Not sensitive / minimal impact

Not very sensitive / manageable impact

Fairly sensitive / significant impact

Very sensitive/ critical impact

Stability

How stable is the assumption?

Very stable / confident

Fairly stable / confident

Fairly unstable / uncomfortable

Very unstable / not confident


Assumption Analysis


Benefits of ABCD

• The key features and benefits of ABCD are: – Communication - Provides a simple, common, language for the

communication of risk up, down and sideways within the organization, while avoiding the normal problems of political sensitivity and dislike of discussing risks.

– Control - Enhances project control by an exception management approach and provides a simple overview of complex risks for senior management to prompt decision making

– Flexibility - An adaptable process which, once tailored, is applied to ensure that all significant risks to the projects are identified and controlled at the appropriate time.

– Acceptable - The non-intrusive/non-bureaucratic management process improves management discipline across the organization and is readily accepted by project teams


Discussion Question

• In your project management career, what has been the most challenging element in managing project risk?

• How do you intend to leverage risk management practices, as you manage projects now and in the future?

Recap

• Recent reports have shown that project failure rates are rising

• Risk management practices have the potential to reduce failure rates

• Research has found that there are commonalities between projects that have failed

• Risk management requires leadership and proactive thinking

Risk Management Requires Leadership

• Final thought from John C. Maxwell’s “The 360 Degree Leader: Developing Your Influence from Anywhere in the Organization”– Do More than Manage—Lead!

Thank You!

the death of a project: can lessons from yesterday’s catastrophe prevent tomorrow’s disaster?...

Documents

project success rates

failed bankrupt projects

end item

business functions

required features

unique product

yesterdays catastrophe

yesterdays mistakes