intergrated baseline reviews as a tool in assurance
TRANSCRIPT
Presentation Agenda
Introduction
Integrated Baseline Review (IBR)
– Why? – Purpose, Objectives
– When? – What stage in a project
– Who? – Does it fit for you?
Approach to an IBR
– Overview of process
– The role of the PMO
• Before the review
• During the review
• After the review
– Benefits of the process
• For the project, PMO and customer
Link to Wider Assurance Activities
– Relationship to assurance organisation
– Maximising & sustaining benefit
Developing APM IBR Guide
– Who, what, when?
Summary
Questions
Integrated Baseline Review Experience
• Does anyone have experience of IBRs?
• Keep hand up if you were on the review team
Integrated Baseline Review – Why? Purpose
• The purpose of an Integrated Baseline Review is to achieve and/or maintain a project
and customer understanding of the content of the Performance Measurement Baseline
(PMB), the risks inherent in the PMB, and the management control processes that will
operate during it’s execution.
• It should confirm that:
– The PMB incorporates the entire scope of the project;
– The work is scheduled to meet the projects objectives;
– Risks are identified and are being managed;
– An appropriate amount and mix of resources have been assigned to accomplish all
requirements;
– Suitable management control processes are being implemented.
• This should provide both the project and its customer the assurance that valid and timely
performance data will be provided throughout the execution of the project.
Integrated Baseline Review – Why? The Performance Measurement Baseline
Time
Standard EVM Terminology
BCWS (PV) Budgeted Cost of Work Scheduled
(How much work should have been done)
BCWP (EV) Budgeted Cost of Work Performed
(How much work has actually been done)
ACWP (AC) Actual Cost of Work Performed
(How much the actual work done has cost)
SV Schedule Variance
= BCWP – BCWS
(A measure of project effectiveness)
CV Cost Variance
= BCWP – ACWP
(A measure of project efficiency)
£
ACWP
BCWP
PMB
SV
CV
BCWS
Time Now
Integrated Baseline Review – Why? Objectives
• Assumptions underlying the plan are
reasonable and documented.
• Project requirements have been
translated into appropriate breakdown
structures and authorised through
documents such as work breakdown
structure (WBS) and statement of work
(e.g. SOW).
• Project schedule key milestones are
identified and reflect a logical flow to
accomplish technical work scope
• The project organisation is identified and
a clear responsibility link to the WBS is
shown e.g. Responsibility Assignment
Matrix (RAM).
• The planned use of resources (budgets,
facilities, personnel, skills, etc.) reflects
availability and is sufficient to accomplish
the technical scope of work within
schedule constraints over the entire
performance period.
• Sub-contract effort and performance
reporting is integrated to the level that is
effective for project control.
• Earned Value Measurement techniques
applied are appropriate to the scope of
work being undertaken in order that
Project Performance data will reflect
project achievement during the entire
performance period.
Integrated Baseline Review - When? Project Control Reviews
Concept Preferred
Supplier(s) Development Delivery In Service
Down
Select
Contract
Award Acceptance
System
Review
Integrated
Baseline
Review
Demonstration
Review Provide customer
assurance that the
contractor has the
capability (process,
people, toolset) to meet
the requirements of
EVM reporting
Provide customer assurance that
the contractor’s Performance
Measurement Baseline (PMB) is
appropriate and well constructed
with sensible resources and
timescales
Provide customer assurance
and confirm that the
contractor’s Earned Value
Management System
(EVMS) is working correctly.
MoD – Acquisition Operating Framework
Integrated Baseline Review – Who? Who may benefit from an IBR?
Pro
ject
Du
rati
on
Project Cost
High
complexity
& risk Low
complexity
& risk
Low
complexity
& risk
High
complexity
& risk
Low
complexity
& risk
High
complexity
& risk
Low
complexity
& risk
Historically seen as
applicable to
projects of greater
complexity, risk, cost
and duration
BUT
The intent of the
review is applicable
to all projects
(Including where
EVM is not being
applied)
The portfolio benefits
from the improvements
delivered by the process
Integrated Baseline Review – Who? Applicability across Projects, Programmes & the Portfolio
Projects
Programmes
Can be broadened to
projects making up the
programme (with review
of programme layer)
IBRs have typically
focussed on projects
Portfolio
Benefits of IBR Process Project – PMO – Customer
Project
• Independent
confirmation that
baseline plans are
robust
• Opportunity to
identify gaps, areas
where improvements
could be achieved
PMO /
Organisation
• Early identification
of improvement
areas for projects
• Confirmation that a
project has the
requisite skills and is
in control
• Ability to share
learning with other
projects to benefit
wider programme
Customer
• Greater confidence
in delivery
• Greater
understanding of
project delivery
strategy and
organisation
Approach to an IBR Standards & Guidance
IBR Standards
• ANSI/EIA 748A – Earned Value Management Systems
• Earned Value Management: APM Guide for the UK
IBR Guidance
• ANSI/EIA-748A – Earned Value Management Systems Intent Guide
• DCMA SSOM – DoD, Defense Contract Management Agency, EVMS
Standard Surveillance Operating Manual
• A Guide to Conducting Integrated Baseline Reviews (MOD AOF guide)
Approach to an IBR Process Overview
Preparation
• Liaison with the project organisation
• IBR Handbook
• IBR Team training
• Project Self-assessment
Project In-brief
• Presentation on the project control system and PMB
•Story Board
Data Trace
• Desktop review of the project control information
Discussion preps
• IBR interview schedule
• Further data trace if required
IBR Discussions
• Interviews with CAMs and key project control stakeholders
IBR Out-brief
• IBR Team presentation on the findings of the IBR
• IBR Report
IBR Team Responsibility
• Should start 1-2 months before the review
• Should be collaborative
• Should include training of the IBR Team
• Outputs: IBR Handbook Project Team Responsibility
• Should provide the IBR Team with a good understanding of
the Project Control System
• Outputs: Presentation and Storyboard
Approach to an IBR Process
IBR Team Responsibility
• Can be 1 week to ½ day activity
• Desk top review of project control data
• Outputs: Issues to cover during discussions IBR Team Responsibility
• Staff chosen for discussions may be selected based
on data trace and In-brief findings
• Outputs: Discussion schedule and questions IBR Team Responsibility
• Can take 2 – 10 days
• The main element of the on-site review
• Outputs: Issues raised against review criteria IBR Team Responsibility
• The final element of the on-site review
• Issues are consolidated against review groupings
• Issues are characterised into CARs or Observations
• Outputs: Out-brief presentation and final report/CARs
Preparation
Project In-Brief
Data Trace
Discussion Preps
IBR Discussions
IBR Out-brief
Documentation review and data traces may start before the on-site review and
are likely to continue following the (on-site) in-brief.
Str
uctu
res
• Work Breakdown Structure
• Organisation Breakdown Structure
• Responsibility Assignment Matrix
• WBS Dictionary
Contr
ol A
ccount D
ocum
ents
• Work Authorisation Documents
• Control Account Plans
• Control Account Schedules / Supplier Schedules
• Terms of Reference
• Work Package Descriptions
Report
s
• EV Performance Reports
• Cost Performance Reports
• CPR1 – by WBS
• CPR2 – by OBS
• CPR3 – Baseline Change
• CPR4 – Resource
• CPR5 – Variance Analysis
• Trend Analysis
Pla
ns
• Project Management Plan
• EVM / Project Control Management Plan
• Plan on a Page
• Risk & Opportunity Management Plan
• Quality Management Plan
• Requirements Management Plan
• Subcontract Management Plan
• Configuration Management Plan
Oth
er
PC
S D
ocum
ents
• Project Control System Description or equivalent
• Contract (& any modifications)
• Contract Statement of Work
• Programme Master Schedule
• Baseline Change Documentation (Contractual or internal)
• Description of Product
• List of Major Subcontractors and Vendors
• Value of subcontracts/ purchase orders
Approach to an IBR Documentation Requirements
Approach to an IBR Scope
Typically against the APM EVM Guide or ANSI 748 Standard
Reciprocity agreement between them
Organisation
Scheduling/resourcing
Budget/Work Auth’n
Management Materials
Risk Management
Change Management
Managerial Analysis
Accounting
APM EVM
Guide / ANSI
748
IBR
DR
Approach to an IBR Level of Confusion
In-brief
Data traces
Project Staff
Discussions
Final Discussions
Write Up
Le
ve
l o
f C
on
fusio
n
Time
Also need to
consider the
forming, storming,
norming team
dynamic
Approach to an IBR Consolidation of Issues
Major Corrective Actions
Minor Corrective Actions
Observations
The Role of the PMO Before the review
Two
primary
roles
Supporting review
preparations
Supporting or conducting
the self-assessments
The Role of the PMO Before the review – Self Assessment & why to do it
The quality of the IBR In-
brief is highly variable
Need for greater intrusion
and investigation during
discussions
Risk of sub-optimal IBR
outcomes and CARs
Less
efficiency
Less
effectiveness
Preparation
Project In-Brief
Data Trace
Discussion Preps
IBR Discussions
IBR Out-brief
The Role of the PMO Before the review – Self Assessment requirement
A structured EVM
Assessment Tool
Enable Project Control Teams to assess their
EVM performance in
a structured manner
The EVM Compass
Need a
method to
support a
structured
approach to
the In-brief
……a shared
assessment
….. a more
collaborative
approach …
The Role of the PMO Before the review – The EVM Compass
The EVM Compass provides a mechanism to:
• Assess the current level of operational maturity
– Using a structured approach that is applicable across projects
– Provides a reference point for future improvement
• Establish a target performance level
– allowing the prioritisation of improvement actions to areas that will
provide the greatest short term return
The EVM Compass aims to measure
current performance with a view to
Improving Performance
The Role of the PMO Before the review – Benefits of PMO supported
self assessment
More collaborative activity during IBR Preparation
• Project Team Self Assessment against the Compass in
parallel with IBR Team preps
• Supporting the development of a more open In-brief
discussion
More focussed approach to data trace and IBR
discussions
• Data trace and discussion activities are focussed on
verification of Self-Assessment strengths and weaknesses
Less intrusive IBR discussions
• A need for fewer and less time consuming IBR discussions
to objectively verify Self Assessment strengths and
weaknesses against review criteria
More focussed IBR Outputs
• Self-assessment has already created the consolidation of
project control issues leading to more focussed and useful
outputs
Preparation
Project In-Brief
Data Trace
Discussion Preps
IBR Discussions
IBR Out-brief
The Role of the PMO During the review
Key question: Is the PMO within scope of
review or (supporting) conducting the review?
Review focus on People,
Process and Toolsets
Who owns them?
Independence from the
project(s) under review?
Generally PMO should be reviewed
… so where IBRs are driven by internal process
must consider how to manage this
The Role of the PMO After the review
Complete Actions
• Consider the intent of the action
• Seek sustainable solutions
• Perform root cause analysis
Share Lessons
• Ensure organisation learns from issues
• Pass the benefit to related projects
Sustain
• Embed changes in how the organisation operates
• Educate future PMs
Link to Wider Assurance Activities Relationship of IBR to assurance organisation
Within Projects-Programmes-Portfolios you equip people with the tools to do
their job and establish the environment in which they can deliver
• Assurance is the confirmation that they are using what you’ve given them
• Integrated Baseline Reviews ensure that what you’ve given them is right to deliver the
desired outcomes and can drive improvements across People, Process and Toolsets
Clearly written and
communicated processes
Access to tools to
enable delivery
Understanding of
the expected values
Understanding of the
expected behaviours
Trained and
empowered staff
Clear ownership
of assurance
Clear assurance
process Tracking
corrective actions
Consistent
enforcement
Continuous
improvement
Robust baseline(s)
Link to Wider Assurance Activities Maximising & sustaining the benefit
The IBR is part of the wider assurance suite that can be applied to a project
Following the IBR, organisations
should:
• Ensure process is updated to reflect
the change resulting from the review
findings
• Communicate process changes and
learning to other projects
• Continue to perform assurance
activities against the revised
processes
Developing APM IBR Guide
MOD IBR Guide
(DEVMIG origins, with
updates)
Experience of APM
Planning, Monitoring &
Control Specific Interest
Group
A Guide to
Integrated Baseline
Reviews (IBR)
Oct 15
Guidance has previously been Defence-centric
IBRs – establishing a common
approach
Increased confidence in successful delivery
Across Business
(via assurance)
Across Projects
(via PMO)
To IBRs (via
common guidance)
Common approach
to IBRs, across
organisations and
sectors
Common approach
across projects,
supported by a
PMO Common approach
to projects within an
organisation,
building on learning
from reviews
Presentation Agenda
Introduction
Integrated Baseline Review (IBR)
– Why? – Purpose, Objectives
– When? – What stage in a project
– Who? – Does it fit for you?
Approach to an IBR
– Overview of process
– The role of the PMO
• Before the review
• During the review
• After the review
– Benefits of the process
• For the project, PMO and customer
Link to Wider Assurance Activities
– Relationship to assurance organisation
– Maximising & sustaining benefit
Developing APM IBR Guide
– Who, what, when?
Summary
Questions
Thank you
Ewan Glen
Principal Consultant, Capability Manager
BMT Hi-Q Sigma Ltd
Mobile: 07747 011 703
Email: [email protected]