TwinSPINUniversity of Minnesota

March 2, 2006

TwinSPINTwinSPINUniversity of MinnesotaUniversity of Minnesota

March 2, 2006March 2, 2006

Project Risk ManagementSome Very Good Practices

Keith D. Hornbacher, MBAHornbacher Associates

Direct: 952.891.3579 Fax: 952.891.1099info@HornbacherAssociates.com

Project Risk ManagementProject Risk ManagementSome Very Good PracticesSome Very Good Practices

Keith D. Hornbacher, MBAKeith D. Hornbacher, MBAHornbacher AssociatesHornbacher Associates

Direct: 952.891.3579 Fax: 952.891.1099Direct: 952.891.3579 Fax: 952.891.1099info@HornbacherAssociates.com

2

Housekeeping – a list of these references (and more) available at desk after meetingHousekeeping Housekeeping –– a list of these references (and more) a list of these references (and more) available at desk after meetingavailable at desk after meeting

Software Engineering Institute (SEI)International Council on Systems Engineering (INCOSE)US Department of DefenseThe Association for Project Management (UK)Project Management Institute (PMI) Project Management Standards (ANSI/PMI 99-01-2004)

For a pdf copy of the presentation, please provide your contact info and e-mail address or send a message to:

info@HornbacherAssociates.com

Useful constructs in current dialogueUseful constructs in current dialogueUseful constructs in current dialogue

4

Boehm: Agile and Plan-driven home grounds*Boehm: Agile and PlanBoehm: Agile and Plan--driven home grounds*driven home grounds*

*See Barry Boehm, p6, “Some Future Trends and Implications for Systems and Software Engineering Processes”, Systems Engineering, Vol. 9, No. 1, 2006

Size (# personnel)Culture (% thriving on chaos vs. order)

Criticality (Loss due to impact of defects)

Personnel (% Level 1B) (% Level 2&3)

Dynamism(% Requirements – change/month)

Agile Plan-driven

10 30

0 35

40 15

20 25

30 20

1030

3.01.0

0.3

70

90

10

3050

100

300

3

10

30

cd

e

ba

a: many livesb: single lifec: essential fundsd: discretionary fundse: comfort

Home Grounds

5

Boehm: Risk-driven spiral approach to balancing Agility and Discipline*

Boehm: RiskBoehm: Risk--driven spiral approach to balancing driven spiral approach to balancing Agility and Discipline*Agility and Discipline*

*See Barry Boehm, p7, “Some Future Trends and Implications for Systems and Software Engineering Processes”, Systems Engineering, Vol. 9, No. 1, 2006

Rate the project’s environmental, agility-

oriented and plan driven risks

Uncertain about

things?

Buy information via prototyping, data collection

and analysis

Yes

No

Step 1 Risk Analysis

Architect application to encapsulate agile parts

Step 3 Architecture Analysis

Compare the agile and Plan-driven

risks

Step 2 RiskComparison

Neither dominate

Plan-driven risks dominate

Agility risks dominate

LCA

Step 5 Execute and MonitorDeliver incremental capabilities

according to strategy

Monitor progress and risks, opportunities, readjust balance

and process as appropriate

Tailor life cycle process around risk patterns and anchor point commitment

milestones

Step 4 Tailor Life Cycle

Go Risk-based Agile

Go Risk-based Plan-driven

Go Risk-based Agile in agile

parts; Go risk-based Plan-

driven elsewhere

Note: Feedback loops present but omitted

for simplicity

Concept Exploration Spirals Development Spirals

6

So, what do we mean by the word “Risk”?So, what do So, what do wewe mean by the word mean by the word ““RiskRisk””??

Merriam-Webster ONLINE (www.m-w.com)– Possibility of loss or injury : PERIL [common usage]

Security, safety, health, product use– Not our context tonight

Risks – potential problems that could arise in the project– Too limiting for some purposes, being replaced by broader view;

still in use by many who matured in threat-based military applications

– Boehm seems to be in this camp, citing risks / opportunities

7

Objectives tonightObjectives tonightObjectives tonight

Introduce evolving project risk management practices– Systems engineering - International Council on Systems

Engineering (INCOSE)– US Department of Defense (DoD)– UK Association for Project Management (APM)– Software Engineering Institute (SEI)– Project Management Institute (PMI)

Discuss PMI’s 6 processesScalability and portabilityQ & A as we proceed

8

Systems Engineering HandbookSystems Engineering HandbookSystems Engineering Handbook

Risk. A measure of the uncertainty of attaining a goal, objective, or requirement pertaining to technical performance, cost, and schedule.

[Note: Often associated with threats rather than opportunities.- kdh]

INCOSE-TP-2003-016-02 (Approved), INCOSE SE Handbook, Version 2a, June 2004

9

SE - Risk Management ProcessSE SE -- Risk Management ProcessRisk Management Process

1. PlanningEstablishment of the riskmanagement program plan andassignment of responsibilities

2. Risk IdentificationAnticipation, recognition &prioritization of potentialadverse outcomes and theassociated root causes

3. Risk AssessmentCharacterization of theMagnitude and likelihood ofrisks, to the depth warranted

4. Risk AnalysisEvaluation of costs and benefitsassociated with available riskmitigation options, if needed

5. Risk HandlingProgram intervention to reduceor eliminate risks, if justified,and tracking to assure success

ProgramManagement &

SystemsEngineering

Requirements, Functions& System Elements

Modifications to

program plan

INCOSE-TP-2003-016-02 (Approved), INCOSE SE Handbook, Version 2a, June 2004 – www.incose.org

10

US Department of Defense - GuidebookUS Department of Defense US Department of Defense -- GuidebookGuidebook

Risk Management Guide for DOD Acquisition, Fifth Ed., (Version 2), Defense Acquisition University, June 2003.

DoD seems to be moving to include opportunities with threats – that is, outcomes based

terminology

11Risk Management Guide for DOD Acquisition, Fifth Ed., (Version 2), Defense Acquisition University, June 2003.

DoD - Risk Management StructureDoD DoD -- Risk Management StructureRisk Management Structure

12

UK: The Association for Project ManagementUK: The Association for Project ManagementUK: The Association for Project Management

The PRAM Guide* originally published in 1997, . . . the second edition [November 2005] reflects many of the most recent developments in the concepts and practice of risk management, for example the inclusion of opportunity within the risk process, recognition of the importance of managing overall “project risk” as well as individual “risk events”, new insights into risk behaviour, more clear understanding of the benefits of risk management to all stakeholders, and new techniques to implement the process.

- Dr David Hillson FAPM FIRM, Director, Risk Doctor & Partners

*Project Risk Analysis and Management Guide

13

UK: APM Risk Management Cycle*UK: APM Risk Management Cycle*UK: APM Risk Management Cycle*

*Project Risk Analysis and Management Guide, APM 1997

Define project

Focus PRAM

Identify Risks

Assess Risks

Plan Risk Responses

Handle risks by change to

project

Write Reports

Achievement of project objectives

Risk Assessment

Reports

Risk Register & Risk Model

…part of broader project plans

Documented Risk

Management Procedure

14

Software Engineering Institute (SEI)Software Engineering Institute (SEI)Software Engineering Institute (SEI)

The purpose of Risk Management is to identify potential problems before they occur, so that risk-handling activities may be planned and invoked as needed across the life of the product or project to mitigate adverse impacts on achieving objectives

CMMI-SE/SW/IPPD/SS, v1.1 Continuous Representation, March 2002

(Version 1.2 of the CMMI Product Suite is planned for release in August of 2006 )

http://www.sei.cmu.edu/cmmi/

15

SEI Risk Management ParadigmSEI Risk Management ParadigmSEI Risk Management Paradigm

Source: Software Risk Evaluation Method Description, December 1999

Continuous Risk Management

16

SEI: Project Risk ManagementSEI: Project Risk ManagementSEI: Project Risk Management

Decision Analysis and Resolution

© 2001 by Carnegie Mellon University, CMMI Overview – Tutorial – V 1.0 Module 4 Part 1 – 03.27.01 – page 19

Determine Risk

Sources,Categories

DefineRisk

Parameters

EstablishRisk

ManagementStrategy Identify

Risks

Evaluate,Classify,Prioritize

RisksImplement

RiskMitigation

Plans

DevelopRisk

MitigationPlans

Risk Repository

Prepare for Risk Management Identify and Analyze Risks

Project Planning, Monitoring and Control

Mitigate Risks

17

SEI - Risk Management ProcessSEI SEI -- Risk Management ProcessRisk Management Process

Software Acquisition Planning GuidelinesGUIDELINE 11: Software Risk ManagementContributor: Ray C. Williams GUIDELINE Software risk management is only successful when organizations are trained to actively identify risk, analyze and track it, plan how to mitigate it, communicate across all levels of the organization, and use risk information actively in making key decisions.

CMU/SEI-2005-HB-006 December 2005

18

Project Management Institute (PMI)– 212K active members in 156 countries [www.pmi.org, Jan 2006)

• North America ~ 72% (US 134K, CA 19K)• Asia Pacific ~ 15%• Europe, Middle East, Africa (EMEA) ~ 10%• Latin America, Caribbean ~ 3%

– Published documents compiled and vetted by members – Guide to the Project Management Body of Knowledge (PMBOK®

Guide), Third Edition, 2004 is latest

Project Management Standards (ANSI/PMI 99-01-2004)Project Management Standards Project Management Standards (ANSI/PMI 99(ANSI/PMI 99--0101--2004)2004)

19

Risk Definition for Our PurposesRisk Definition for Our PurposesRisk Definition for Our Purposes

Project Risk. . . is an uncertain event or condition that, if it occurs, has a positive or negative effect on at least one project objective, such as time, cost, scope, or quality . . . . . . and is defined by two attributes, likelihood

of occurrence and impact if it occurs . . .PMBOK® Guide, Project Management Institute, Third ed., 2004

[Emphasis added]

20

Why practice project risk management? Why practice project risk management? Why practice project risk management?

Of Fortune 500 Companies’ Projects Implemented– Nearly Half from 100% to 400% Over Budget– 86% Late Achieving Deliverable

Risk Management – identified as one of top priority remedial strategies

“Project Management Best Practices Report,” August 2000 - Center for Business PracticesReported in Trendwatch, PM Network - November 2002

21

Project Risk Management BenefitsProject Risk Management BenefitsProject Risk Management Benefits

Increased predictabilityLower incidence of failureOverall lower project costsHigher rate of opportunity capture

22

The quality of decision making can be greatly improved by using formal decision support processes to manage associated uncertainty.

Formal Models + Associated Analysis + Synthesis

Quality Decisions

- Chapman and Ward, Managing Project Risk and Uncertainty, 2002.

Decisions are Based on Informed IntuitionDecisions are Based on Informed IntuitionDecisions are Based on Informed Intuition

23

Six Processes of Project Risk Management*Six Processes of Project Risk ManagementSix Processes of Project Risk Management**

Risk Management PlanningRisk IdentificationQualitative Risk AnalysisQuantitative Risk AnalysisRisk Response PlanningRisk Monitoring and Control

*Project Management Standards (ANSI/PMI 99-01-2004) PMBOK® Guide, Project Management Institute, Third ed., Ch 11, 2004.

24

Key to all PRM good practices: QIQOKey to all PRM good practices: Key to all PRM good practices: QIQOQIQO

Quality data must be collected, analyzed , interpreted, and communicated.

Quality In, Quality Out

25

Simple Project Risk Management ProcessSimple Project Risk Management ProcessSimple Project Risk Management Process

LearnPlan

Define

Identify

Respond

Handle

MonitorControl

Prioritize

Quant_itative

Qual_itative

Analyze

TriggerParameters?

No

Yes

Artifacts: DocumentPlan, Register, Database

26

Risk Management PlanningRisk Management PlanningRisk Management Planning

Contains the elements that are necessary to properly prepare and set the ground rules that will allow us to manage the risk of the project– There are several inputs to the risk planning– The overall project plan-- defining stakeholders, size, complexity,

and objectives of the project– Overall company strategy for managing risk

27

Project Risk Management Planning ContextProject Risk Management Planning ContextProject Risk Management Planning Context

Initial phase – align with stakeholders and policy– Objectives for project must be clear, consistent, understood, and

documented– Agreement on scope, method, process, roles, operational

implementation of risk management– Risk Management Plan (RMP) records decisions on process /

procedural details, plus the “who’s” and “how's”Clearly define what risk means (threats ~ opportunities)Risk Management effort must be explicitly funded –sends a “strong message”

Also sends clear message if not funded!

28

Risk Management Plan - Some Key Questions to Answer IncludeRisk Management Plan Risk Management Plan -- Some Key Some Key Questions to Answer IncludeQuestions to Answer Include

What is the organization's PRM policy?Who will do what tasks – roles, responsibilitiesHow are we going to manage risk on this project?How will we tailor risk identification activities to this unique project?What types of analysis, tools, techniques, frequency, prioritization, and metrics are appropriate?– Qualitative analysis: scalar criteria for probability and impact (P, I)– Quantitative analysis: aggregation of risks, complexity, visibility

29

Risk IdentificationRisk IdentificationRisk Identification

Identification of risk events calls upon the project team, subject matter experts, the stakeholders, and other project managers

30

Structuring Risk IdentificationStructuring Risk IdentificationStructuring Risk Identification

Use of a metalanguage clearly identifies the risk– As a result of <specific cause>, the <uncertain event> may

occur and lead to <impact on objective(s)>

Vague identification leads to a poor response

First principle: Understand the situation to describe it clearly

31

Risk Management Meta-LanguageRisk Management MetaRisk Management Meta--LanguageLanguage

Understand the cause(s) and impact to develop appropriate risk response strategies

Cause Risk ImpactBecause of a world-wide scarcity of

ABC materials

there is a risk that essential replacement

items cannot be found if a

problem arises during testing,

resulting in a delay to the completion of systems testing and

to the Critical Milestone (project

schedule)

32

Risk Identification InputsRisk Identification InputsRisk Identification Inputs

Among the items that will be used– Project Charter– Work Breakdown Structure (WBS)– Project Description– Project Schedule– Cost Estimates– Budgets– Resource Availability– Resource Schedules– Procurement Information, and assumptions that have been

made and recorded

33

There are many ways to discover and identify risk:– Documentation reviews– Brainstorming– Delphi technique– Expert interview– Checklists– Analogy– Risk Breakdown Structure (RBS)

Risk Identification (Cont.)Risk Identification (Cont.)

34

Risk Identification Tool –Generic Risk Breakdown StructureRisk Identification Tool Risk Identification Tool ––Generic Risk Breakdown StructureGeneric Risk Breakdown Structure

Source: PMBOK® Guide, 3rd ed.

35

Identification: Brainstorming (Adapted)Identification: Brainstorming (Adapted)Identification: Brainstorming (Adapted)Start with and augment the Risk Breakdown StructureCapture input data needed for Risk RegisterThe right people on the bus and wrong ones off the bus– Place makes a difference – eliminate distractions

Excellent team building opportunity– Facilitation by skilled non team member so all can participate– Manage dominant personalities – judging, solving NOT allowed– Time keeper helps session stay moving

Document ideas – write on sticky notes, cover walls in related groups (RBS works here); use projector in real-time to capture

36

Risk AnalysisRisk AnalysisRisk Analysis

Importance of each risk is evaluated using list of identified risksEvaluation or assessment process is necessary to itemize these risks into ranking that will place them in order of importance– Combination of probability and impact causes the risk to be an

important consideration to the project– Risk tolerance – the willingness or unwillingness of a person or an

organization to accept risk

37

Risk Impacts - those things that affect project objectives– The cost (money)– Schedule (duration)– Scope of the project (technical aspects, quality)

These impacts can manifest themselves as affects on– Effort required, labor rates, duration of tasks, technical feasibility,

material suitability, material cost, equipment availability, and more

Risk Analysis: Impact, outcome if occursRisk Analysis: Impact, outcome if occurs

38

Risk Probability – the probability of a risk occurring is essential to the assessment of the riskProbability or likelihood number that represents the chance that a particular outcome will occur Qualitative analysis uses library of terms to describeLanguage we use often imprecise

Risk Analysis: ProbabilityRisk Analysis: Probability

Exercise - Handout

39

Comparison of 507 Responses to SurveyComparison of 507 Responses to Survey

Source: Describing Probability: the limitations of a natural language, © 2005 David Hillson, Ph.D. Used with permission.

40

Expected Value – a way of combining the probability and the impact of a risk in a meaningful way– Expected Monetary Value: $100 = $1,000 * 10%

Expected value is also a good guideline for the amount of money that might be spent to eliminate the risk

Risk Analysis: ResultsRisk Analysis: Results

41

Risk Analysis - TriageRisk Analysis Risk Analysis -- TriageTriage

A way to organize the risks so that they can be dealt with in a logical way

– Risks need to be put into groups that can then be managed by individuals who are more familiar with the nature of the risks

– Risks need to be prioritizedComparative RankingGrouping the Risks (RBS hot spots)Affinity Diagramming Logic networks

42

Qualitative Risk Analysis - InputQualitative Risk Analysis Qualitative Risk Analysis -- InputInput

Risk items from Identification step recorded in Risk Register (database with search utility)Criteria for assigning event probabilities and impacts on objectives using words and phrases– Need to correspond to each branch, element, risk in RBS– Answer the questions

• What characteristics describe each term in descriptive rating labels (both event probability and impact on objectives)?

• What procedure must be used to combine probability and impact scores into risk levels?

Definition of risk level classification

43

Probability Model: Organization, Resources*Probability Model: Organization, ResourcesProbability Model: Organization, Resources**

Probability Model - OrganizationDescriptive

Rating Basis: Resources, Flexibility Obtaining Probability Rating

Very Low Common skill sets plentiful 1

Low Experienced with current effort identified and available 2

Moderate Competition among organization’s projects but informal moves possible 3

High Internal requisition process only option 4

Very High Redeployment among projects never an option, only new hires 5

* From RBS example

44

Qualitative Risk LevelsQualitative Risk LevelsQualitative Risk Levels

Probability and Impact MatrixProbabilityRating

5 5 10 15 20 25

4 4 8 12 16 20

3 3 6 9 12 15

2 2 4 6 8 10

1 1 2 3 4 51 2 3 4 5

Impact RatingP (x) I

45

Qualitative Output: Ranked RisksQualitative Output: Ranked RisksQualitative Output: Ranked Risks

Visual scorecard of risks in groups for responsePrioritize allocation of scarce resourcesDecision gate – refer some risks to quantitative analysisIndividual risks only – cannot aggregate with qualitative

Risk Ranking Scorecard Condition

High RED

Moderate YELLOW

Low GREEN

Response Action Rule

Actively manage; include in baseline plan

Actively manage; some discretion in timing using contingency plans

Team level action; put on watch list

46

Quantitative Risk Analysis FrameworkQuantitative Risk Analysis FrameworkQuantitative Risk Analysis Framework

Integrated approach to analyzing riskBuilds on data collected in way to calibrate uncertaintyLogic network of activities and tasks is model of projectIntegrated model used to test alternative response plans

47

Quantitative Risk AnalysisQuantitative Risk AnalysisQuantitative Risk Analysis

Considers all risks simultaneously – provides a measure of overall project riskData gathering tools and techniques– Risk interviews – generally for 3-point estimates – Probability distributions represent uncertainty

Analysis tools and techniques– Sensitivity analysis– Expected monetary value– Decision tree analysis– “Monte Carlo” simulation of network logic, spreadsheet models

48

Input valid Critical Path Method (CPM) essentialInterview processThree point estimates– Pessimistic (Pess)– Optimistic (Opt)– Most Likely (ML)

Not all activities (tasks) will occur as planned– Test, Fail, Fix, Re-test, Pass

Estimating Ranges and Uncertain PathsEstimating Ranges and Uncertain PathsEstimating Ranges and Uncertain Paths

PessOpt

ML

Time

49

Reality – Some paths are uncertainReality Reality –– Some paths are uncertainSome paths are uncertain

10% PROBABILITY

Probabilistic Branch

START

*

2 4

5 8

1 3 6

7

9A10

END

90%

10% 9B

6 CONDUCTINSPECTION

*

FAILINSPEC

DIAGNOSECORRECTREPEAT

PASSINSPEC

90% PROBABILITY 10

50

Multiple Paths Likely to Become Critical Multiple Paths Likely to Become Critical Multiple Paths Likely to Become Critical

Which tasks/activities are they?– Under what conditions are they likely to behave differently?– Can we identify them in advance?– Is it possible to calibrate them to show their relative chances of

appearing on a critical path?Can something be done to reduce the likelihood and/or impact of threats?Can something be done to increase the likelihood and/or impact of opportunities?Quantitative project risk analysis can answer these questions.

51

Monte Carlo Simulation ResultsMonte Carlo Simulation ResultsMonte Carlo Simulation Results

Compares results against targetsProvides calibrated likelihood of achieving datesCan be compared with alternate scenariosAggregates impacts and probabilities of all risksCan include sophisticated logic (branches, decisions, contingency plans, conditional paths)Several valid tools are commercially available, off the shelf today

52

Monte Carlo Simulation – ResultsMonte Carlo Simulation Monte Carlo Simulation –– ResultsResults

53

Quantitative Cost Risk Analysis - InputQuantitative Cost Risk Analysis Quantitative Cost Risk Analysis -- InputInput

54

Quantitative Cost Risk Analysis: OutputQuantitative Cost Risk Analysis: OutputQuantitative Cost Risk Analysis: Output Distribution for Total Pipeline / Cost

Risk/J22

Valu

es in

10

-3

Values in Thousands

012345678

Mean=1477.962

1.3 1.4 1.5 1.6 1.71.3 1.4 1.5 1.6 1.7

5% 90% 5% 1.3971 1.5569

Mean=1477.962

Distribution for Total Pipeline / CostRisk/J22

Values in Thousands

0.000

0.200

0.400

0.600

0.800

1.000

Mean=1477.962

1.3 1.4 1.5 1.6 1.71.3 1.4 1.5 1.6 1.7

5% 90% 5% 1.3971 1.5569

Mean=1477.962

55

Risk Response PlanningRisk Response PlanningRisk Response Planning

After evaluating the risks, assessing their impact and probability of occurrence, prioritizing the risks in their order of importance, we must decide what to do about them.The process of developing the procedures and techniques to enhance opportunities and reduce threatsto the project’s objectives.

56

Risk Response Strategies Applicable to Threats, Opportunities, Both

Risk Response Strategies Risk Response Strategies Applicable to Threats, Opportunities, BothApplicable to Threats, Opportunities, Both

ThreatAvoidTransferMitigate

OpportunityExploitShareEnhance

Both Threats and OpportunitiesAccept (with contingency reserve)Contingent Response

57

Avoidance - by changing the project plan or the nature of the project.Transfer - to another party to eliminate the risk from impacting the projectMitigation – actively try to reduce probability or impact of the risk to a point where the risk can be acceptedAcceptance– Active – contingency plan to deal with risk if and when it occurs.– Passive – ignore, roll dice, hope, play Powerball, update resume

Opportunities– Seek ways to enhance chances of success

Response StrategiesResponse Strategies

58

Risk Monitoring, Control: Ongoing TaskRisk Monitoring, Control: Ongoing TaskRisk Monitoring, Control: Ongoing Task

The process of keeping track– All identified risks– Identify new risks as their presence becomes known – Assess residual, secondary risks that occur when the risk

response plans implementedReassess risks periodically, focus on trigger pointsContinuously execute the risk management cycleMeasure the health of the data, cycle times, age of actions, sensitivity of alert systems

59

Remember - Risk definition for our purposesRemember Remember -- Risk definition for our purposesRisk definition for our purposes

Project Risk. . . is an uncertain event or condition that, if it occurs, has a positive or negative effect on at least one project objective, such as time, cost, scope, or quality . . . . . . and is defined by two attributes, likelihood of occurrence

and impact if it occurs . . .- PMBOK® Guide, Project Management Institute, Third ed., 2004

Probability (x) Impact or P (x) IUse this fundamental relationship often.

60

One More Time: Why Manage Risk?One More Time: Why Manage Risk?One More Time: Why Manage Risk?

Protect the project objectivesProvide early warnings while there is time to actContinuously apply risk management for successClarity of purpose – everyone on the team understands the value of their input in an environment that encourages openness and trusts their judgment

Preserve project objectives from negative impacts and enhance performance by actively searching for opportunities.

Preserve project objectives from negative impacts and enhance performance by actively searching for opportunities.

61

Thank You !!Thank You !!Thank You !!

Any Questions??

TwinSPINUniversity of Minnesota

March 2, 2006

TwinSPINTwinSPINUniversity of MinnesotaUniversity of Minnesota

March 2, 2006March 2, 2006

Project Risk ManagementSome Very Good Practices

Keith D. Hornbacher, MBAHornbacher Associates

Direct: 952.891.3579 Fax: 952.891.1099info@HornbacherAssociates.com

Project Risk ManagementProject Risk ManagementSome Very Good PracticesSome Very Good Practices

Keith D. Hornbacher, MBAKeith D. Hornbacher, MBAHornbacher AssociatesHornbacher Associates

Direct: 952.891.3579 Fax: 952.891.1099Direct: 952.891.3579 Fax: 952.891.1099info@HornbacherAssociates.com

63

SEI and DoD SourcesSEI and DoD SourcesSEI and DoD Sources

64

SEI Technical ReportSEI Technical ReportSEI Technical Report