by acic under foi · the crimtrac p3m3 self-assessment process was based on the guidance provided...

Oakton Consulting Technology CrimTrac

1 EXECUTIVE SUMMARY .......................................................................................................... 2

1.1 Background ............................................................................................................................. 2

1.2 CrimTrac Self-Assessment ....................................................................................................... 2

1.3 Oakton’s Validation Results Summary .................................................................................... 4

1.4 Target Maturity Level Recommendations .......................................................................... . 5

1.5 Independent Assessment ............................................................................................... .... 5

2 THE SELF-ASSESSMENT VALIDATION PROCESS ..................................................................... 6

2.1 The Objective ............................................................................................................ ........... 6

2.2 The Plan .................................................................................................... ..... .. .... ............ 6

3 PROCESS REVIEW ..................................................................................... ...... .................... 8

4 SELF-ASSESSMENT VALIDATION ........................................................ .... ........................... 10

4.1 Portfolio Management Level 2 ................................................... .... ................................ 10

4.2 Programme Management Level 2............................................ .... .................................... 12

4.3 Project Management Level 2 ............................... .............................................................. 15

4.4 P3M3 Sub-Model Process Perspective Scores . ..... ...... .................................................. 17

4.5 P3M3 Generic Attributes Scores ............ ............... ........................................................... 17

5 AGENCY TARGET MATURITY LEVELS ............. ........ ........................................................... 19

5.1 Maturity Targets ............................ ..... ......................................................................... 19

5.2 Gap Analysis ...................................... .................................................................................. 19

6 RECOMMENDATIONS ............ ..... .................................................................................... 20

6.1 Benefits Management ...... ....... ........................................................................................ 20

6.2 Stakeholder Engagemen ..................................................................................................... 20

6.3 Resource Management ......................................................................................................... 21

6.4 Organisation l Governance .................................................................................................. 21

6.5 Prog amm Management ..................................................................................................... 21

6.6 L ssons Learned .................................................................................................................... 22

ATTACHMENT A - P3M3 MODEL AND DEFINITIONS ..................................................................... 23

ATTACHMENT B - TERMINOLOGY ............................................................................................... 26

October 2014 P3M3 Assessment Report Page i

Releas

ed by

ACIC

unde

r FOI

Oakton Consulting Technology

Government

1 EXECUTIVE SUMMARY

1.1 Background

CrimTrac was established through an Inter-Governmental Agreement in 2000 as a collaborative partnership between the Commonwealth, States and Territories to help combat crime in Austra ia. CrimTrac’s aim is to generate information sharing solutions across police services, the wider law enforcement community and international counterparts, whilst its current business prioritie include Law Enforcement Information, Background Checking, Biometrics and Capability Sustainability and Governance1.

Since its establishment, CrimTrac has adopted a structured, tailored approach to P rtfo io rogramme and Project (P3) Management, based on the Office of Government Commerce (OGC) methodologies for project (PRINCE2) and programme management (Managing Successful P ogrammes). CrimTrac has established an enterprise Portfolio Management Office (ePMO) which has developed a framework for delivery of P3 Management, known as the Portfolio, Program and Project Go ernance (PPPG) Framework.

CrimTrac is a unique agency, somewhat different to a standard non-corporate entity (as per the new PGPA Act), as it is essentially the information technology service developer and provider for shared information services for its business areas which reside w thin the Federal, State and Territory Police agencies. This creates some unique challenges for CrimTrac’s execution of Portfolio, Programme and Project Management.

1.2 CrimTrac Self-Assessment

In July 2014 the CrimTrac ePMO conducted a survey to self-assess the maturity level of CrimTrac’s P3 Management. The self-assessment was based on the OGC’s self-assessment questionnaires for Portfolio, Programme and Project Management. This self-assessment was a follow up to previous self-assessments undertaken in 2012 and 2013.

The 2014 survey was complet d by a cross section of 19 of CrimTrac’s Directors, Project and Programme Managers Oakton’s review of this assessment indicates that it was well managed and reasonably reflect the existing level of maturity for most of the process perspectives. We have identified an issue with the overall assessment for Programme Management Maturity which is discussed be ow.

The ta les below list the results of CrimTrac’s self-assessments over the period of 2012 to 2014, and Oakt n’s validations for 2013 and 2014. Green cells indicate Oakton’s assessment is in line with CrimTra ’s self-assessment. Red cells indicate Oakton’s assessment varies from CrimTrac’s self-

ssessment. Arrows indicate an improved result between Oakton’s 2013 assessment and Oakton’s 2014 assessment.

1 CrimTrac Strategic Plan 2015

October 2014 P3M3 Assessment Report

Page 2

Releas

ed by

ACIC

unde

r FOI


Government

an increase in the assessment for:

• Risk Management at Programme level from level two to level three.

CrimTrac has made significant gains in perspective maturity across all three sub-models since last year’s assessment, and particularly within the programme sub-model where gains were observed in five of the seven perspectives.

1.4 Target Maturity Level Recommendations

Oakton recommends that CrimTrac set targets for delivery maturity levels as follows:

Portfolio = Level 3: Defined Processes

Programme = Level 2: Repeatable Processes

Project = Level 3: Defined Processes

Capability delivery improvements across CrimTrac’s Portfolio, Programme and Project Management processes can lead to a more centralised and defined framework with mature investment management processes. Section 6 outlines Oakton’s recommendations for improvement.

1.5 Independent Assessment

This Independent Assessment was oversighted by , APMG Registered Consultant (PPMRCAU/026), and conducted by and


Page 5

Releas

ed by

A

unde

r FOI

s 47Fs 47F s 47F


Government

3 PROCESS REVIEW

The CrimTrac P3M3 Self-assessment process was based on the guidance provided by the UK Office of Government Commerce handbooks for P3M3 self-assessment. OGC has produced a handbook for each sub-model of Portfolio, Programme and Project Management. The figure below shows the P3M3 sub-models and process perspectives.

For each process perspective for e ch sub-model, the self-assessment questionnaire asked assessors to select the most appropriate d scrip ion of maturity level for that process perspective.

The ePMO provided the ques ionnaire to a representative sample of CrimTrac Directors, Programme and Project Managers for response. The results were assessed from 19 respondents.

Oakton has review d the resulting CrimTrac Self-assessment Report and the assessment scores appear to reflect a good understanding of the current state of affairs at CrimTrac. It is worth noting the consisten y o the answers given by the group of interviewees Oakton spoke to, which would indicate that the p oc sses and methods used by CrimTrac are systematic across the organisation.

Oakton notes that in reporting results of the self-assessment the ePMO has provided fractional scores – e g. a maturity level of 2.6 for Stakeholder Engagement for Project Management. Whilst these scores provide some indication of movement between the 2012, 2013 and 2014 self-assessments, this a proach is not supported by the Methodology. The approved approach to scoring is to determine the median score for a process perspective and round the median score to the nearest whole number. The overall maturity assessment for each sub-model is then determined from the lowest process perspective score.

Based on the 19 responses, the CrimTrac P3M3 Self-assessment rates CrimTrac’s maturity level at Level 2 – repeatable processes for each sub-model. Individual Process Perspective scores are shown in the tables below, taken from the CrimTrac Self-assessment Report.


Page 8

Releas

ed by

ACIC

unde

r FOI


Government


Page 9

Releas

ed by

ACIC

unde

r FOI


Government

4 SELF-ASSESSMENT VALIDATION

In addition to reviewing the self-assessment process, Oakton also reviewed existing CrimTrac documentation, processes and systems, and interviewed a sample of 10 people who could represent views from the PMO, Portfolio, Programme and Project Management.

The Oakton approach was to conduct the interviews and document review as we would do for a standard P3M3 assessment. We did not discuss individuals’ responses to the self-assessment questionnaire with them. We also matched documentary evidence with the expected level f evidence outlined in the APMG guidelines for P3M3 assessments.

Following are the assessment results with a description of the rationale behind the as essment.

4.1 Portfolio Management Level 2

Overall Portfolio Management maturity is assessed as Level 2.

Portfolio Management is at level 2 and not level 1 because:

• Across process perspectives there is a recognised and repeatable set of processes.

• Most processes display a considerable level f maturity n particular there is a strong alignment of change initiatives to strategic objectives and priorities. Financial management show signs of maturity and internal management ontrols display the right approach to coordination of portfolio level activiti s across the organisation.

Portfolio Management is at level 2 and not l vel 3 because:

• Benefits realisation activities are managed to level 2. Although benefits are identified in business cases, and work has commenced to track the realisation of external benefits, there is little evidence of benefits management including benefits ownership and tracking being consistently applied at the p rtfolio level.

4.1.1 Management Control Level 3 Management Control w thin Portfolio Management was found to be at level 3 and not level 2 because the assessment eve led that the portfolio management processes are centrally defined and understood, as are the roles and responsibility for delivery. The roles of the executive committees including the Portfolio Board are well understood, as is the process for approval of new initiatives and ongoing monitoring of programmes and projects. The ePMO, BE&I and BS&P teams are working towards supporting the portfolio management process with idea assessment, reporting and independent assurance. There is a clear link between the value assessment contained in the business cases for each new initiative which drives the annual plan and associated priorities that are used by the executives to develop a portfolio of projects and programmes. All new initiatives are centrally considered and approved before becoming a part of the change portfolio.

Management Control within Portfolio Management was found to be at level 3 and not level 4 because the assessment found no evidence of consistent processes or metrics used to measure the performance and success of the portfolio.

4.1.2 Benefits Management Level 2 Benefits Management within Portfolio Management was found to be at level 2 and not at level 1 as there is a reasonable level of understanding of the need for benefits management. Benefits are


Page 10

Releas

ed by

ACIC

unde

r FOI


Government

identified at project level and are contained in the business case documents, and are linked to corporate strategy.

Benefits Management within Portfolio Management was found to be level 2 and not level 3 because there is little evidence of a defined benefits realisation and management process being exercised by the Portfolio Board. Recognition of benefits happens consistently in the business case documents, however, the quality of those benefits needs improvements with focus on the measurable aspect of those benefits and assignment of ownership and accountability for realisation. It is recognised that the relationship of CrimTrac to its jurisdictional policing stakeholders makes benefit ownership and measurement a challenge for CrimTrac projects and programmes. CrimTrac has engaged the Australian Institute of Criminology (AIC) to manage external benefit definition, management and realisation.

4.1.3 Financial Management Level 3 Financial Management at the portfolio level was found to be at level 3 and not evel 2 because there are established standards for the investment management process and the p eparation of business cases. In addition, portfolio investment costs are monitored by the Portfo o Board. Annual financial planning exists. Regular reviews of the portfolio occur and corrective ac ions are put in place to rectify under or over spend situations. There is evidence of projects being stopped, due to reassessment of the business case.

Financial Management within Portfolio Management was found o be level 3 and not level 4 because there is little evidence of a proactive evidence based management of the costs. CrimTrac does not exercise the prediction of future financial performance based on the metrics gathered for programmes and projects, e.g. “estimates to complete” at the programme and project level are not assessed. External financial risks are not consi tently monitored and evaluated and there is little evidence of the active financial management of b nefits.

4.1.4 Stakeholder Management Level 3 Stakeholder Management at the po tfolio level was found to be at level 3 and not level 2 because external stakeholders are engaged th ough the Strategic Issues Group and Board of Management, project/programme based user groups and domain specific forums e.g. CFO forum. CrimTrac has a standard documented appro ch to identifying, managing and communicating with portfolio stakeholders. Projec and Programme Managers are required to comply and be consistent with a Portfolio level Commun ations Strategy and CrimTrac’s Communications Principles and Guidelines.

Stakeholder Management within Portfolio Management was found to be level 3 and not level 4 because ther is no evidence of quantitative analysis of the effectiveness of communications and stakehold r ngagement.

4.1.5 Risk Management Level 3 R sk management has improved at the portfolio level from level 2 to level 3 since last year’s assessment.

Risk Management at the portfolio level was found to be at level 3 and not level 2 because there is a standard documented approach to identifying, quantifying and managing portfolio risk.

Whilst CrimTrac’s appetite for risk and the balance between threats and opportunities is generally well know and regularly reviewed, Risk Management within Portfolio Management was found to be level 3 and not level 4 because risk analysis is relatively unsophisticated and does not include input from risk specialists or consider risk modelling or simulations.


Page 11

Releas

ed by

ACIC

unde

r FOI


Government

4.1.6 Organisational Governance Level 3 Organisational Governance at the portfolio level was found to be at level 3 and not level 2 because there is a clear alignment of initiatives, programmes and projects with the strategic objectives. Projects have been stopped as a result of the review at the portfolio level. All interviewees have a clear understanding of portfolio governance and the process for approval of an initiative.

Organisational Governance within Portfolio Management was found to be level 3 and not level 4 because, whilst it is emerging, there is little evidence of a full collection of portfolio performance information to support Portfolio Board decisions and risk mitigation. Lessons learned are only s arting to be consolidated and incorporated into decision making, for example.

4.1.7 Resource Management Level 3 Resource Management has improved at the portfolio level from level 2 to level 3 since ast year’s assessment.

Resource Management at the portfolio level was found to be at level 3 and not level 2 because there is evidence of resource management being used to develop targeted reso rce capabilities to enable delivery of longer term priorities. The ePMO tracks portfolio resource u ilisation. Commercial resource providers are engaged to supplement CrimTrac resources to meet peak demands, and there is a consistent approach to the procurement of resources within the portfolio.

Resource Management within Portfolio Management w s found o be level 3 and not level 4 because there was little evidence of performance assessment of resources based on initiative delivery rather than individual or team performance, and whilst there is a consistent approach to the procurement of resources, the current process is inefficient and often lengthy.

4.2 Programme Management Level 2

Overall Programme Management maturity s assessed as level 2, and has increased from level 1 since last year’s assessment.

Programme Management i at level 2 and not level 1 because:

• The conc pt of Programme Management has been grasped by CrimTrac and there are experienced Programme Managers (project directors) working on key (sub) programmes.

• Programme managers (and the Programme Management Office) monitor programme costs, risks, issues and dependencies in accordance with CrimTrac guidelines and procedures, with defi ed interfaces to other functions within CrimTrac.

• Across process perspectives there is a recognised and repeatable set of processes.

• There is evidence of managing change initiatives as a programme management approach to strategic change (as detailed in the ICT Strategic Blueprint), rather than simply a grouping of similar projects, as was evidences last year.

Programme Management is at level 2 and not level 3 because:

• Significant work needs to be undertaken by the Program Management Office (PMO) to develop the Programme sections of the PPPG Framework and to define and implement standard tools and templates for programme management.

• The Programme Board needs to be mandated to make decisions within agreed tolerances in order to allow the Portfolio Board to focus on strategic planning and delivery.


Page 12

Releas

ed by

ACIC

unde

r FOI


Government

4.2.1 Management Control Level 2 Management Control has improved at the programme level from level 1 to level 2 since last year’s assessment.

Management Control within Programme Management was found to be at level 2 and not level 1 because the concepts of programme management have been grasped by CrimTrac and there are experienced programme managers working on key programmes. There is still evidence of project-based controls and tools being applied, but a common vocabulary is beginning to evolve and the e is evidence of good practice in some areas. There is regular reporting to the Portfolio Board, and a Programme Board has been established and is now operating, supported by the PMO.

Management Control within Programme Management was not at level 3 because there is l ttle evidence of centrally defined and documented approach to a programme management lifecycle and controls. Version 2.0 of the PPPG Framework does not currently identify programme specif c deliverable templates, for example programme management plan, programme bl epri t project dossier, and programme business case, and programme and project management are essentially addressed as the same thing.

4.2.2 Benefits Management Level 2 Benefits Management has improved at the programme level from level 1 to level 2 since last year’s assessment.

Benefits Management within programmes was found to b at le el 2 and not at level 1 because benefits are described and defined, but not consis ently, across CrimTrac’s programme. There is some evidence of the need to track benefits beyond ‘go live’ activities, but this is not undertaken consistently. The measurement criteria for benefits ge erally exist, but are not consistently applied.

Benefits Management within Programme Management was found to be level 2 and not level 3 because there is not a centrally-managed and consistent framework for defining and tracking the realisation of benefits arising from programme outcomes. Benefits are not regularly considered at the programme board, and internal and external benefits are tracked and managed inconsistently.

4.2.3 Financial Management Level 2 Financial Management has improved at the programme level from level 1 to level 2 since last year’s assessment.

Financial Manag ment at the programme level was found to be at level 2 and not level 1 because programme business cases are produced in varying forms and contain a rationale on which to obtain CrimT ac commitment to the programme. Evidence exists of governance bodies formally committing to programmes, and of budget and cost control for programmes. Programme approval processes are integrated with CrimTrac’s portfolio approval processes and support is consistently provided by the co porate finance function.

F nancial Management at the programme level was found to be at level 2 and not level 3 because there is not yet evidence of centrally established standards for the preparation of programme business cases and processes for their management throughout the programme lifecycle. Programmes to not have clearly defined levels of authority for expenditure and sign-off. Programmes do not have their own budget and programme management costs are seen as an overhead.


Page 13

Releas

ed by

ACIC

unde

r FOI


Government

4.2.4 Stakeholder Management Level 2 Stakeholder Management within programmes was found to be at level 2 and not level 1 because there was some evidence of a structured approach to identify, manage and communicate with stakeholders. Stakeholder engagement is a planned activity.

Stakeholder Management within Programme Management was found to be level 2 and not level 3 because stakeholder management is not exercised in accordance with a centrally defined process. There is no structured centrally managed communications plan to balance communications within all programmes and projects. Local knowledge is used in place of specialist communication skills to design and assess communication activities. The costs associated with communications and stakeholder management is not considered explicitly or separated in the programme costs.

4.2.5 Risk Management Level 3 Risk Management has improved at the programme level from level 2 to level 3 ince last year’s assessment.

Risk Management at the programme level was found to be at level 3 and not level 2 because there is evidence of a centrally defined risk management framework that is deployed to all programmes. Risk management intervention points and reviews are embedded within the p ogramme lifecycle and risks are consistently categorised by type. There is some eviden e that systematic and aggregated project risks are being managed at the programme level.

Risk Management at the programme level was found to b level 3 and not level 4 because there is no evidence of the deployment of proven standards and te hniques to assist with continual improvement and no evidence of expertise be ng applied to specialist risk areas. Whilst risk workshops are undertaken at the commencement of change initiatives, there is little evidence of the proactive identification and management of risk on a regular basis.

4.2.6 Organisational Governance Level 2 Organisational Governance at the programme level was found to be at level 2 and not level 1 because there are decision making controls be ng applied in a repeatable manner through the Programme Boards. There is evidence f the inks between organisational strategy and the direction of the programme.

Organisational Governance within Programme Management was found to be level 2 and not level 3 because, while there were controls in place for the CrimTrac’s major programmes, standards for Programme governance structure or decision making were not evident in the CrimTrac PPPG Framework. The role of business change managers in programmes was not evident. There is little evidence of decision-making at the Programme Board level, with most decisions referred to the Portfolio Board.

4.2.7 Resource Management Level 3 Resource Management has improved at the programme level from level 2 to level 3 since last year’s assessment.

Resource Management at the programme level was found to be at level 3 and not level 2 because there is evidence of resource management being used to develop targeted resource capabilities to enable delivery of longer term priorities. There is a programme view of expected resource demand and effective use of commercial resource providers to supplement CrimTrac resources to meet peak demands.

Resource Management within Programme Management was found to be level 3 and not level 4 because, whilst there is evidence of resource capacity management and capacity planning, there is


Page 14

Releas

ed by

ACIC

unde

r FOI


Government

little evidence of benchmarking resource utilisation with other agencies, or an active leveraging of the supply chain to optimise resource utilisation between internal and external resource pools.

4.3 Project Management Level 2

Overall Project Management maturity is assessed as Level 2.

Project Management is at level 2 and not level 1 because:

• CrimTrac recognises projects (their language, processes and framework) and run them differently to business as usual activities based on a standard Project Framework.

• There is consistency in approach across projects with project managers, as all recognise or try to apply the current PPPG Framework.

• Key individuals have practical delivery experience in project management. New project managers are supported in their role through the ePMO.

• CrimTrac has a centrally defined set of procedures and management processes for acquiring, planning and managing project resources.

Project Management is at level 2 and not level 3 because:

• Whilst there is a project management framewo k developed and maintained by the ePMO, there are still gaps in the framework, and proje t execut on in the areas of Benefits Management and Stakeholder Engagement.

4.3.1 Management Control L vel 3 Management Control within Project Manag ment was found to be at level 3 and not level 2 as all projects have designated project managers who use the project management framework and information system. Issues and risks were monitored through a standardised process. Change control is applied where appropriate.

Management Control within Project was not at level 4 because project management metrics were not used to monitor proc s cap bility and manage project performance in quantitative terms.

4.3.2 Benefits Management Level 2 Benefits Manageme t within Projects was found to be at level 2 and not at level 1 because there is recognition of the oncept of benefits versus output from projects. Outcomes and benefits are recognised as an element within business cases and project plans.

Ben fits Management within Project Management was found to be level 2 and not level 3 because whi e t ere is some documentation of benefits, measurement, on-going and post project tracking is no defined and explicit. Business ownership of Benefits Realisation is not normally defined in project d cuments. Benefits are usually not estimated in financial terms against centrally managed assessment criteria. The Project Management Framework needs some development in terms of the Benefits Management by providing practical guidance in that area.

4.3.3 Financial Management Level 3 Financial Management at the project level was found to be at level 3 and not level 2 because project business cases documents are always developed and approved at project initiation using a standard process. A template and guide to developing a business case is centrally provided. Business cases are reviewed at appropriate points within a project’s lifecycle, and action taken to keep projects on track. Approved Project Mandates are required to start a project and these define the scope and budget


Page 15

Releas

ed by

ACIC

unde

r FOI


Government

allocation for the project. Signoff is required for any additional costs imposed by the project, including accessing contingency allowances.

Financial Management within Project Management was found to be level 3 and not level 4 as process tolerances were only monitored against total project funding. Earned value concepts for monitoring a project are not in place and risks are not evaluated in financial terms.

4.3.4 Stakeholder Management Level 2 Stakeholder Management at the project level was found to be at level 2 and not level 1 because a standard approach to identifying, managing and communicating with stakeholders exists and is used to identify and plan engagement. This was done in a basic and consistent way. Commitmen to meeting stakeholder needs and actively involving stakeholders in projects was evident among interviewees.

Stakeholder Management within Project Management was found to be level 2 and not level 3 because although there is evidence of documented Communication Plans for some projects, there does not appear to be a standard approach used by all projects. Corporate communications is not involved in development of stakeholder communication, coordinating key messages across projects or enhancing the effectiveness of communication activity.

4.3.5 Risk Management Level 3 Risk Management has improved at the project level from evel 2 to level 3 since last year’s assessment.

Risk Management at the project level was found to be t level 3 and not level 2 because project risk management is based on a centrally defined process that is cognizant of CrimTrac’s policy for the management of risk and is used consistently. There is evidence of risk workshops being undertaken during project planning, and of regular risk eviews during the project lifecycle. Risks are consistently categorised by type and there is evidence of risk escalation processes working effectively.

Risk Management within Project Management was found to be level 3 and not level 4 as there was little evidence of projects demonstra ing the budgetary and resourcing implications of risk throughout the project lifecycle, or of udits of ris management effectiveness being undertaken.

4.3.6 Organisational Governance Level 3 Organisational Governance at the project level was found to be at level 3 and not level 2 because there is clear evidence of a centrally defined set of agency level controls applied to projects. Regular consol dated reporting occurs with a single source of the truth through the CA Clarity tool. Terms of Reference and clear reporting lines to governance bodies are maintained. Decisions of Board mee ings are minuted and auditable. The Project Status Reports report progress against project ou comes. A tiered system, for classification of projects, assists in ensuring that projects are evaluated against consistent criteria prior to project approval.

Organisational Governance within Project Management was found to be level 3 and not level 4 because there are few indicators of understanding the impact of projects on business performance. Decisions to launch projects do not usually consider the impact on current initiatives.

4.3.7 Resource Management Level 3 Resource Management has improved at the project level from level 2 to level 3 since last year’s assessment.

Resource Management at the project level was found to be at level 3 and not level 2 because CrimTrac has a centrally defined set of procedures and management processes for acquiring, planning


Page 16

Releas

ed by

ACIC

unde

r FOI


Government

and managing project resources. Project resource allocation is centralised and undertaken by the Resource Allocation Committee. Resource utilisation tracking is undertaken and frameworks are in place to supplement internal resources with resources from external suppliers. There is also evidence of collaboration between projects for the sharing of critical or limited resources.

Resource Management within Project Management was found to be level 3 and not level 4 because there is a dependency on key individuals with little evidence of succession planning, and little evidence of formal induction and extraction planning for individuals joining and leaving project teams.

4.4 P3M3 Sub-Model Process Perspective Scores

The figure below indicates the scores for each sub-model against each process perspective.

4.5 P3M3 Generic Attributes Scores

The P3M3 also identifies a set of Generic Attributes that are applicable to all three sub-models. The Generic Attributes are:

• Roles and Responsibilities - how individuals working on P3 understand their roles and responsibilities?

• Experience in P3 – how experienced are the individuals charged with executing P3?

• Capability Development – what processes are in place to develop the capability of P3 managers?

• Planning and Estimating Processes – how robust are these processes supporting the development of P3 plans?


Page 17

Releas

ed by

ACIC

unde

r FOI


Government

5 AGENCY TARGET MATURITY LEVELS

5.1 Maturity Targets

Oakton recommends that CrimTrac set target capability delivery maturity levels as follows:

• Portfolio: Level 3 - Defined Processes

• Programme: Level 2 - Repeatable Processes

• Project: Level 3 - Defined Processes

5.2 Gap Analysis

The following analysis identifies the process perspectives that are respons ble fo the maturity level score i.e. the low scoring process perspectives. The Capability Improvement plan needs to focus on improving these areas and addressing the ‘gap’.

Portfolio Level 2 => 3: Def ned Processes Management Control Level 3 => 3

Benefits Management L vel 2 => 3

Financial Management Level 3 => 3

Stakeholder Management Lev l 3 => 3

Risk Management Level 3 => 3

Organisational Governance Level 3 => 3

Resource Management Level 3 => 3

Improvements in Benefits Management are required in order to achieve a level 3 maturity level.

Programme: Level 2 => 2: Repeatable Processes CrimTrac has made the necessary improvements to achieve a level 2 maturity level, and should look to mainta n this rating. Recommendations on how this might be achieved are discussed in Section 6.

Project: Level 2 => 3: Defined Processes Management Control Level 3 => 3

Benefits Management Level 2 => 3

Financial Management Level 3 => 3

Stakeholder Management Level 2 => 3

Risk Management Level 3 => 3

Organisational Governance Level 3 => 3

Resource Management Level 3 => 3

Improvements in Benefits Management and Stakeholder Management are required.


Page 19

Releas

ed by

ACIC

unde

r FOI


Government

6 RECOMMENDATIONS

Oakton recommends that CrimTrac set targets for delivery maturity levels as follows:

Portfolio = Level 3: Defined Processes

Programme = Level 2: Repeatable Processes

Project = Level 3: Defined Processes

Capability delivery improvements across CrimTrac’s Portfolio, Programme and Project Man gement processes can lead to a more centralised and defined framework with mature investme t management processes, and in order to reach these P3M maturity levels, Oakton recommends that CrimTrac focus on improvements in the following areas:

6.1 Benefits Management

• Include external benefits realisation in ongoing portfolio planning

• Improve benefits realisation guidance for internal P3 staff

• Appoint benefit owners when establishing projects and rogrammes

• Identify programme benefits in programme-level business cases

• Track internal benefits through to re lisation

CrimTrac is in a unique position in that many of th benefits realised from its portfolio of work are delivered as enhanced capabilities for th jur sdictional policing agencies. It is therefore difficult to measure the ongoing realisation of those enefits. CrimTrac has engaged the Australian Institute of Criminology (AIC) to develop a framework for benefits identification, realisation and management of these externally delivered benefits. The outputs from AIC’s work should be used to inform CrimTrac’s ongoing portfolio planning.

CrimTrac should further improve its maturity in this area by continuing to develop guidance for portfolio, programme and project staff, particularly for internal benefit realisation. Some basic steps to improve the definit on of benefits and develop practical metrics would assist in improving the tracking of internal benefits.

We also recomme d that when establishing projects and programmes, the formal appointment of appropriate be efit owners should be part of the process. This would also encourage deeper stakeh lder communication with projects/programmes and commitment to benefits realisation.

f CrimTrac wishes to improve Programme Management Maturity, then development of programme-level business cases which include identification of Programme Benefits would assist.

6.2 Stakeholder Engagement

• Implement steering committees for all projects with external stakeholders

• Include appropriate stakeholder engagement and communications budgets in project and programme plans

• Implement a programme-level approach to stakeholder engagement to ensure consistent and coordinated communications with external stakeholders

Oakton recognises the unique situation of CrimTrac as an IT service provider to jurisdictional policing agencies. That said, there is an opportunity to improve stakeholder engagement at programme and


Page 20

Releas

ed by

ACIC

unde

r FOI


Government

project level, by improving the identification of and engagement with business owners/users of the outputs of CrimTrac projects. The recent introduction of Steering Committees for externally facing projects is a positive initiative, but may require increased budgets for engagement and business change within these agencies (at the cost of doing fewer projects).

This recommendation may require significant strategic level engagement with the Strategic Issues Group and/or Board of Management. The risk of not acting is that CrimTrac will remain an IT provider rather than a business partner delivering significant value to the jurisdictions.

6.3 Resource Management

• Improve efficiencies in procuring contract-based P3 staff

• Continue to invest in portfolio-wide capacity and capability planning, re ource allocation and management

• Reintroduce P3M competency groups

Since the 2013 P3M3 assessment, CrimTrac has continued to invest in portfolio-wide resource and capacity planning, aided by the introduction of the plan-build-run model Resource capacity and capability is now well understood across CrimTrac and informs the d cision making for new project/programme initiatives. This investment has signifi antly improved resource allocation and management, and has resulted in CrimTrac lifting its lev l of maturity in resource management across portfolio, programme and project management.

Oakton recommends that this investment and focus con inues, and that CrimTrac continues to seek efficiencies in its external supply chain.

6.4 Organisational Governance

• Delegate programme and project decision authority, within appropriate tolerances, to the Programme Board

• Focus Portfolio Board a tivities on portfolio planning, strategy and enterprise risk

As noted last year, it is evide t that there is a heavy governance load currently being born by the Portfolio Board. The ime required and the level of detail provided for recent Portfolio Board meetings indicates a ne d to streamline this process and push decision making down to more appropriate levels. The Portfolio Board’s focus should be setting strategic direction, setting priorities, managing e terpr se and portfolio level risks, monitoring performance and managing exceptions. Its current focus seems to be in the detail of minor changes to projects, which should be managed at programme or project board level.

Oakton recommends that the Portfolio Board delegate authority, with appropriate tolerances, to the Programme Board so that the Programme Board can make decisions arising from programme and p oject exceptions.

6.5 Programme Management

• Structure the delivery of the ICT Blueprint as an MSP-style programme

• Clarify the roles and responsibilities of the ePMO and the PMO to avoid any duplication

• Reinstate the programme-level content in the PPPG

• Develop appropriate programme-level document and reporting templates


Page 21

Releas

ed by

ACIC

unde

r FOI


Government

• Consider excluding programme management from future P3M3 assessments if an MSP-style approach to programme management is not implemented

Last year, Oakton noted that there was a fundamental issue with the definition of a programme and how CrimTrac wished to manage programmes. There were three programmes running at the time, with one attempting to set itself up as a true programme of strategic change, and the other two simply collections of like projects. There were competing views within CrimTrac about the applicability of programme management.

The ICT Blueprint for National Police Information Sharing 2014 – 2018 (ICT Blueprint) has been a catalyst to change the focus of programme management to that of a single programme to cover the breadth of capability delivery required to meet the needs of CrimTrac’s stakeholders.

CrimTrac should structure the delivery of the ICT Blueprint as an MSP-style programme and should consider the following:

• The roles and responsibilities of the PMO and the ePMO in relation to prog amme management should be clarified in order to eliminate any duplication. For example, the ePMO should take on secretariat and reporting functions for the Programme Board, whilst the PMO should focus on programme and project delivery assurance.

• Similarly, the appropriate office, and possibly the PMO, sh uld b charged with developing the missing programme-level information in the PPPG Framew rk, including the development of appropriate programme-level document and report templates

• Depending on the level to which an MSP-style approach to programme management is adopted, CrimTrac should consider whether Programme Management be removed from future P3M3 assessment processes a together.

6.6 Lessons Learned

• Undertake post implementation reviews for all projects to capture lessons learned

• Consolidate and analyse les ons learned for systemic issues

• Incorporate lesso s lea ned i to P3 planning and the PPPG framework

Evidence of a comprehensive essons Learned Log was sighted as a component of the suite of information currently held by the ePMO, but the information contained in the log is not currently used to influence portfolio, project and programme planning.

Oakton recommends that the Lessons Learned Log be analysed to identify systemic issues and common themes, and that these be incorporated into the PPPG Framework such that they become ‘the way things are done’ for all new initiatives.


Page 22

Releas

ed by

ACIC

unde

r FOI


Government

ATTACHMENT A - P3M3 MODEL AND DEFINITIONS

P3M3 Model

The following terms are used in the assessment of portfolios, programmes and projects.

Portfolio Management

A portfolio is the totality of an organisation’s investment (or segment thereof) in the changes required to achieve its strategic object ves.

Portfolio Management is a coordinated collection of strategic processes and decisions that together enable the most effective balan e of o ganisational change and Business as Usual (BAU).

Programme M nagement

A programme is a temporary, flexible organisation created to coordinate, direct and oversee the implementation of a set of related projects and activities in order to deliver outcomes and benefits related to the organisation’s strategic objectives.

Progra me Management is the coordinated organisation, direction and implementation of a dossier of rojects and transformation activities to achieve the outcomes and realise the benefits that are of strategic

importance to the business.

Project Management

A project is a temporary organisation, usually existing for a much shorter duration (than a programme), which will deliver one or more outputs in accordance with a specific Business Case. A particular project may or may not be part of a programme.

Project Management is the planning, delegating, monitoring and control of all aspects of the project, and the motivation of those involved, to achieve the project’s objectives with the expected performance targets of time, cost, quality, scope, benefits and risks.


Page 23

Releas

ed by

ACIC

unde

r FOI


Government

The characteristics and elements of each process perspective are outlined below.

Management Control

Management Control is characterised by clear evidence of leadership and direction, scope, stages, tranches and review processes during the course of initiatives. There are regular check points and clearly defined decision making processes. There are full and clear objectives and descriptions of what the initiative will deliver with a blueprint of the target operating model.

Benefits Management

Benefits Management is the process that ensures the desired business change outcomes are clearly define are measurable and are ultimately realised through a structured approach and with full organisational ownership. Benefits are assessed and approved by the organisational areas that will deliver them. B nefit dependencies and other requirements are clearly defined and understanding gained on how outpu s of the initiatives will meet those requirements. There should be evidence of suitable classification of benef ts and holistic view of the implications being considered. All benefits should be owned, have realisation plans and be actively managed to ensure they are achieved. There is a focus on operational transition with review and follow up on actions needed to ensure benefits are owned and realised.

Financial Management

Financial Management ensures that the likely costs of the initiatives are captured and evaluated with a formal business case and that costs are categorised and managed over the investment lifecycle. There should be evidence of involvement of the agency’s financial functions, with approvals imbedded in the broader organisational hierarchy. The business case should define the value of the nitiative to the business and contain a financial appraisal of the possible options. The business case will be at the core of decision- making during the initiative’s lifecycle, and be linked to formal review stages and valuation of the costs and benefits associated with alternative actions. Financial Management will chedule the availability of funds to support the investment decisions.

Stakeholder Management

Stakeholders are the key to the success of any initia ive. Stakeholders at different levels within and outside the organisation will need to be analysed and engaged effectively in order to achieve the objectives in terms of support and engagement. Stakeholder Management includes communications planning, the effective identification and use of different comm nicat on channels, and the techniques to enable the objectives to be achieved.

Stakeholder Management is an ongoing process across all initiatives and is inherently linked to the initiative’s lifecycle and governance controls

Risk Management

Risk Management provides a view on the way opportunities and threats are presented by the initiative. Risk Management maintains a balance of focus on threats and opportunities with appropriate management actions to minimise o eliminate the likelihood of any identified threat occurring, or minimise the impact if it does occur and to maximise opportunities.

Org nisat onal Governance

This looks at how delivery of initiatives is aligned to the direction of the organisation. It considers how start-up and closure controls are applied to initiatives and how alignment is maintained during an initiative’s lifecycle. This differs from Management Control which views how control of initiatives is maintained internally. This perspective looks at how external factors that impact on the initiative are controlled if possible or mitigated if not. The management of these external factors are used to maximise the final result. Effective sponsorship is critical to this.

Resource Management

Resource Management covers all types of resources required for delivery. These include human resources, buildings, equipment, supplies, information, tools and supporting teams.


Page 24

Releas

ed by

ACIC

unde

r FOI


Government

A key element in the management of resources is the process of acquiring resources and how supply chains are utilised to maximise effective use of resources. There will be evidence of capacity planning and prioritisation to enable effective resource management. This would also include performance management and exploitation of opportunities for greater utilisation. Resources capacity considerations will be extended to the capacity of the operational groups to resource the implications of change.

Maturity Level Definitions

Maturity Level 1 - Awareness of Process

• Processes are not usually documented, actual practice is determined by events or individual preferenc s, and performance is variable.

• Successful initiatives are often based on key individuals’ competencies rather than organisation wide capability and past successes cannot be repeated consistently.

• Processes are undeveloped or incomplete. There is little or no guidance or supporting doc mentation and even terminology may not be standardised.

Maturity Level 2 - Repeatable Processes

• Basic management practices, e.g. tracking expenditure and scheduling resources, are n place and being improved. Key individuals are trained and demonstrate a successful track record and through them, the organisation is capable of repeating success.

• Initiatives are performed and managed according to their documented plans; project status and delivery is visible to management at defined points.

• There may still be inadequate measures of success; unclear re ponsibi ties; Ambiguity /inconsistency in business objectives; unintegrated Risk Management; limited Cha ge Management; and inadequacies in communications strategy.

Maturity Level 3 – Defined Processes

• Management and technical processes are documented, standardised and integrated to some extent with business processes. There is some process ownership and a group responsible for maintaining consistency and delivering process improvements.

• Senior management are engaged consist ntly, providing active and informed support. • There is an established training programme to develop individual’s skills and knowledge.

Maturity Level 4 – Managed Proce ses

• The organisation has defined processes that are quantitatively managed, i.e. controlled using metrics. There are quantitative objectives for quality and process performance, and these are being used in managing processes

• Top management are proactively seeking out innovative ways to achieve goals. • Using metrics, management can effectively control processes and identify ways to adjust and adapt them

to particular initia ives without loss of quality.

Maturity Level 5 – Optimised Processes

• There is ocus on optimisation of quantitatively managed processes to account for changing business needs. The organisation exhibits continuous process improvement, and can show strong alignment of organisational objectives with business plans.

Top managers are seen as exemplars, reinforcing the need and potential for capability and performance improvement.

• Information from process and product metrics enables the organisation to understand causes of variation and to optimise its performance.


Page 25

Releas

ed by

ACIC

unde

r FOI

by acic under foi · the crimtrac p3m3 self-assessment process was based on the guidance provided...

Documents