value based solutions to common owner frustrations · 6/26/2014 · • hiring or selecting who...

6/26/2014

1

Value-Based Procurement & Contracting…Is This the

Solution to Common Owner Frustrations?

Dr. Kenn Sullivan

Arizona State University

June 26, 2014 | 1:00 pm – 2:15 pm ET

COAA Webinar

performance based studies research group• • www.pbsrg.com

Value-Based Procurement & Contracting… Is This the

Solution to Common Owner Frustrations?

Kenneth T. Sullivan, PhD, MBA


6/26/2014

2


Goals1. Minimize Cost

2. Minimize Cost by becoming More Efficient

3. Become More Efficient in three ways:

– Hire people who know what they are doing

– Preplan before the contract is signed

– Measure for positive accountability

4. Teach the thinking, concepts, tools, and processes to organizations

3


Your Goals (typical - Owner)• Be able to get what you paid for and be able to

prove it – Demonstrate VALUE!

• Enhance preplanning and performance measurement techniques

• Add Best Value as another tool in your toolbox

• Have higher levels of accountability / breakdown organizational silos

• Become a “Client of Choice”

4

6/26/2014

3


Your Goals (typical - Vendor)• Increase competitiveness and profit

• Enhance preplanning and performance measurement techniques

• Use performance information

• Have higher levels of accountability / breakdown organizational silos

• Prove you are a High Performer

5

Arizona State University 6

PBSRG’s Research Results

• Worldwide as a leader in Best-Value Systems 19 Years 240+ Publications 650+ Presentations 1700+ Projects $6.6 Billion Services & Construction 98% Customer Satisfaction Various Awards (PMI, NIGP, IFMA, COAA, IPMA) Clients: Federal, State, Local, School Districts, Private

6/26/2014

4



InformationTechnologynetworkingdata centershardwareCOTS softwareERP systems

help desk serviceseProcurement

FacilityManagementmaintenancelandscapingsecurity servicebuilding systemsindustrial movingwaste managementenergy management

custodialconveyancepest control

Health Insurance/Medical Services

Manufacturing

Business/Municipal /University Servicesdiningmulti-media rightsfitness equipmentonline educationdocument managementproperty managementaudiovisualcommunications systemsemergency response systemslaundry

material recyclingbookstoresfurniture

Construction/Design/Engineeringlarge gcinfrastructuremunicipallaboratoryeducationhospitalfinanciallarge specialty

small gcrenovationrepairmaintenanceroofingdemolitiondevelopmentsupply chain

DBBCMARDBIDIQJOCLow BidIPD

6/26/2014

5


Strategic Partners & Efforts

Fulbright Scholarship-University of BotswanaBV tests

RMITTeaching IMTPBSRG platform

Tongji University

6+ yearsInfrastructure€1.8B plus €1B

BrunsfieldComplete Supply Chain

United States -65 clients

Univ. of Manitoba

DalhousieUniv.

Univ. of Alberta

City of Spruce Grove

SFU Alberta Infra

Ontario Consortium

WCBNS


What Percent of RFP’s Are 100% Accurate?

10

6/26/2014

6


Who Should Know More About

Performing/Delivering the Services Required?

11


It Is More Important For The Vendor To Know What To Do Than It Is For Client To Know What The Vendor Should Do

12

6/26/2014

7


Vendor 1

Vendor 2

Vendor 3

Vendor 4

High

Low High

Low High

Low High

Low

Impact of Minimum Requirements

Vendor 1Vendor 2Vendor 3Vendor 4


High

Low

Owners

“The lowest possible quality

that I want”

Vendors

“The highest possible value that you will get”

Minimum

Problem with Traditional Approach

High

Low

Maximum

6/26/2014

8



Which of these Proponents brings your organization the most risk?

16

Vendor 1

Vendor 2

Vendor 3

Vendor 4

High

Low High

Low

6/26/2014

9


What we have seen…

17

Client

Vendor

Vendor

Vendor

Vendor

Vendor



18

Client

Vendor

Vendor

Vendor

Vendor

Vendor

6/26/2014

10



19

ClientVendor



20

Client Vendor

Client PM Vendor PM

6/26/2014

11


“The Greatest Risk that I always face, is how to

accomplish all of the things that our sales team promised

we could do.”

21


What is different…

22

Client

Vendor

Vendor

Vendor

Vendor

Vendor

Plan

6/26/2014

12


What is different…

23

Client Vendor

Client PM Vendor PM

Plan


Best Value Objectives

24

Minimize cost, increase efficiency

“not just transfer risk...but minimize

risk”

Supply Chain mentality“Win-Win”

Minimize risk of non-

performance “High Client

Satisf.”Minimize the need for client management, direction, and

decision making

Vendors maximize their profits by being more efficient

Reduce Cost

6/26/2014

13


Best-Value Process


Best Value Model

6/26/2014

14


Best Value Model


Selection• Hiring or selecting who will create the plan and

execute it

• The quality of the plan and its execution is directly linked to the individuals creating it and doing the work– Quality of Plan = Minimization of Risk & Cost

28

6/26/2014

15


What are we trying to accomplish?

Scenario 1 Scenario 2

Question:

If Purchasing wants to buy a “green circle”, in which scenario is hiring the right “green circle” easiest to justify?


Filter 1Proposal

Evaluations

Filter 2Interview

Key Personnel

Filter 4Cost

ReasonablenessCheck

Filter 5Pre‐Award &Clarification

Project ExecutionRisk Reporting & Close Out Rating

Filter 3Prioritization(Identify

Best Value)

Contract Award

Evaluation Criteria‐ Price / Cost / Fee‐ Project Capability‐ Risk Assessment‐ Value Added‐ Past PerformanceInformation (PPI)

Short List prior to

Interviews (if necessary)

Pre Award Activities‐ Training‐ Kickoff Meeting‐ Plan & Clarify‐ Summary Meeting

Total Evaluation Scores are determined

Decision Matrix to confirm Selection of the potential Best Value Proponent

Project Execution‐Weekly Risk Report‐ Director Report‐ Performance Meas.‐ Close Out Ratings

Best Value Process

Copyright Arizona State Univ. 2013

1

2

3

4

1

2

3

4

6/26/2014

16


Selection – Value Based PropositionSelection dictates the maximum capacity to achieve a quality plan

Procurement methodologies to identify expertise:• Risk and Value focused RFP Process• Simple, brief, anonymous evaluation process• Evaluation of key project personnel

– Proven Past Performance Information– Ability to identify, prioritize, and minimize risk– Interviews of project delivery personnel– Focus on specific project needs– Minimize marketing information

• Cost/Financial

31


BV Model Has Multiple Filters• No single step will complete minimize all risk. Process has several

steps that act as filters.

• Filters do not limit competition, but make it difficult for non-performers to be competitive.

• Looking for dominant information

32

6/26/2014

17


Filter 1Proposal

Evaluations

Filter 2Interview

Key Personnel

Filter 4Cost

ReasonablenessCheck




Best Value)

Contract Award


Short List prior to






Best Value Process


1

2

3

4

1

2

3

4


Critical Formatting Requirements• Must use the attached “Attachments” to complete and submit the

proposal

• Proposers are NOT allowed to alter the documents (change font size, font type, boarder widths, add color, add images/graphics, re-format, re-create, etc)

• In order to minimize any bias, any Plan MUST NOT contain any names that can be used to identify who Proposer is (such as company names, personnel names, project names, letterhead, logos, or product names).

34

6/26/2014

18


Blind Submittals

Risk Plan

Value Added

Project Capability

Simple, concise, support w/ performance metrics

= risk don’t control

= capability to meetrequirements

= added scope


Project Capability Template

6/26/2014

19


• Plan 1– We will use our 20 years of experience in working with mechanical

systems to minimize the risk of the heating and cooling system design.

• Plan 2– We have identified the design of the heat/cooling system as a risk. It has

not been used before in the area. Will ensure that the system performance and installation is verified in the pre-award period.

– We have bid using best rated mechanical contractor in the area (rated at 9.8 out of 10.0, next best rated 9.1)

– Mechanical contractor identified modifications to the design to improve output and sustainability of the system with the following impacts (mechanical system cost minimized by 15% - see VA#1)

– Mechanical system will be provided by one manufacturer, and will be commissioned by the manufacturer, contractor, and general contractor, who will take full responsibility of commissioning the system

Example of SolutionsRisk: Design of Heating/Cooling SystemType: Project Capability


• Plan 1– We will work with the user to minimize the impact of noise from

demolition.

• Plan 2– We have planned to demolition during off hours and weekends. This

will have a slight impact on our cost (less than 1%), but the impact to customer satisfaction justifies this.

– We will also install rubber sheets on the floors to diminish noise and vibrations.

– Both solutions can be performed within your budget. – Both solutions have been used on multiple previous projects w/ high

levels of customer satisfaction (9.4/10).

Example of SolutionsRisk: Noise from Demolition

Type: Project Capability

6/26/2014

20


Risk Assessment• Identify major project risks (and prioritize)

• Risks that have not yet been minimized before the start of the project (i.e. risks that the Proponent does not control)– Explain why the risk is a risk– May cause cost increases, transition delays, change orders, or

cause dissatisfaction to the client– Should be non-technical– Should be a risk specific to the current project

• Identify solution to avoid / minimize the risk• Non-technical explanation• Must explain how risk is avoided or minimized from occurring• For high score solution should contain performance information &

best value practices


Risk Assessment Template

40

Typical Risk Assessment Template

6/26/2014

21


Example of Solutions Risk: Loss of Radio Flagship in Major MarketType: Risk Assessment• Plan 1

– We will work very hard to maintain excellent affiliate relationships. If we lose a radio station (e.g. it changes its format) we will move quickly to replace the lost station. If we cannot quickly replace a flagship station, we can be very creative and could even consider purchasing all local inventory from a new flagship station.

• Plan 2– In the past 10 yrs, on over 50 accounts, 7 radio stations format changes

have occurred. The following solution is optimal.– We own and will maintain two radio contracts covering the area, where

signals can be switched if required. The flagship station will be the station with the stronger signal and greater coverage.

– If a station is lost we will have a equal replacement within 2 months. If within two months a replacement is not contracted we will purchase inventory from another station or discount the cost of an inventory purchase and add it to our payments to the client.


• Plan 1– Coordination with [water company] is critical. We will

coordinate and plan with [water company] as soon as the award is made to make sure that we get water to the site to irrigate the fields.

• Plan 2– We will coordinate and schedule the water with [water

company]. However, based on past experience there is a high risk they will not meet the schedule (the water company does not meet schedule over 90% of the time).

– We will have temporary waterlines setup and ready to connect to the nearby fire hydrant to irrigate until [water company] is ready.

– We will also have water trucks on-site if there is problems with connecting the lines.

Example of Solutions Risk: Getting water to the site

Type: Risk Assessment

6/26/2014

22


Why Value Add Plan?1. Provide ways to keep project at or below budget

– Modifications to requirements to meet budget– Specific cost ($) savings– Supported by metrics (high performance)

2. Increase customer satisfaction

3. Increase performance


Value Added Template

44

Typical Value Added Template

6/26/2014

23


Example: Value Added Items

• Reroofing this building will not stop all water leaks. The majority of the leaks are caused by cracks in the parapet walls, broken/missing glass, and poor caulking. For an additional $10K and 3 weeks in schedule we can replace and repair all of these items.


Example: Value Added Items• Instead of purchasing “Named Licenses”, the Agency

may want to consider purchasing “Concurrent Licenses”. In a “Named Licensing” model, the software designates a license per user and only that particular named user can use/access the license. If that named user is in meetings, on vacation, or not using the system, the license is not utilized. In a “Concurrent Licensing” model, the server keeps track of the total number of licenses and loans the licenses to users as they log in. If a user is inactive, the server releases the license and allocates the license to the next user. The advantage is that the Agency is not required to purchase licenses that are not being used, which can result in approximately 25% savings in cost.

46

6/26/2014

24


Filter 1Proposal

Evaluations

Filter 2Interview

Key Personnel

Filter 4Cost

ReasonablenessCheck




Best Value)

Contract Award


Short List prior to






Best Value Process


1

2

3

4

1

2

3

4


What is the Objective of PPI?

• PPI should not be used as the sole factor to ensure that the vendor can perform on a current project.

• Communicate to the vendors that the owner will purchase based on value (performance and price).

• Identify if a vendor has capability to perform.

• Encourage vendor to use and partner with the best people / best subs

4848

6/26/2014

25


Survey Form

4949


Past Performance Information• PPI will be collected on the critical Entities:

– Firm/Key Firms– Project Manager (Individual)/Team Leader(s)– Key Subs or Suppliers and Other Key Individuals

• Each Entity must prepare and submit a list of references, customer surveys, and Past Performance Information Scores

50

VendorENTITY

Prepare and Send Survey Questionnaires to Past Clients Step 2

Step 3Collect/Receive Completed Surveys

Prepare Reference ListStep 1

Enter data into Reference ListStep 4

Package all material (Reference List and Surveys) and SubmitStep 5

6/26/2014

26


Filter 1Proposal

Evaluations

Filter 2Interview

Key Personnel

Filter 4Cost

ReasonablenessCheck




Best Value)

Contract Award


Short List prior to






Best Value Process


1

2

3

4

1

2

3

4


Financial Proposal/Bid Cost• Dependent on

project type

6/26/2014

27


Financial Proposal (VGS)• Vendor generated

solution


Filter 1Proposal

Evaluations

Filter 2Interview

Key Personnel

Filter 4Cost

ReasonablenessCheck




Best Value)

Contract Award


Short List prior to






Best Value Process


1

2

3

4

1

2

3

4

6/26/2014

28


Summary of Submittals• Summary of evaluation

– Past Performance Information • Sent in by past clients to the

Proponent• Proponent collects and submits

with proposal• Submit separate Excel files

– Firms and Individuals– 3 worksheets per file

» Contact Information» Past Project Info/

Reference List» PPI Analysis Table

– Project Capability (Up to 2 Pages)– Risk Assessment (Up to 2 Pages)– Value Added (Up to 2 pages)– Pricing (follow requirement) – Any additional required forms


Proposal Form & Checklist

56

6/26/2014

29


Proposal Form & Checklist

57


Proposal Package(Attachments)

• Attachment A – Proposal Form• Attachment B – Project Plan• Attachment C – Risk Assessment Plan• Attachment D – Value Assessment Plan• Attachment E – Reference List• Attachment F – Survey Questionnaires • Attachment G – Past Performance Information Scores

• Attachment H – Cost Proposal Form

58

6/26/2014

30


How The Submittal Process Works

Submittal

Evaluation Members

Proposal Form(1 page)

Proposal Form & Other Documentation

Proposal Form(1 page)

Plan

Average Score

ContractingOfficer

ContractingOfficer


Prioritization

60

NO CRITERIA POINTS FIRM A FIRM B FIRM C BESTFIRM A

POINTS

FIRM B

POINTS

FIRM C

POINTS

1 Cost 250 145,000$ 150,000$ 170,000$ 145,000$ 250 242 213

2 Interviews 350 4.5 8.1 6.2 8.1 194 350 268

3 Risk Assessment Plan 200 5.1 8.7 7.5 8.7 117 200 172

5 Value Assessment Plan 100 5.0 5.0 5.0 5 100 100 100

6 PPI – Firm (1‐10) 25 9.5 9.2 9.1 9.5 25 24 24

7 PPI – Firm (Surveys) 25 1 5 5 5 5 25 25

8 PPI – Project Manager (1‐10) 25 9.5 9.2 8.8 9.5 25 24 23

9 PPI – Project Manager (Surveys) 25 1 4 2 4 6 25 13

Total 1000 TOTAL POINTS (1,000): 723 990 838

6/26/2014

31


Shortlisting

• If necessary, depending on the number of Proponents, short listing will be conducted


Filter 1Proposal

Evaluations

Filter 2Interview

Key Personnel

Filter 4Cost

ReasonablenessCheck




Best Value)

Contract Award


Short List prior to






Best Value Process


1

2

3

4

1

2

3

4

6/26/2014

32


Are Interviews Necessary?• Not all projects require the interview filter, such as small

or simple projects (roofing, mechanical, etc).

• The interview filter should be performed on any project that has high risk (political risk)

• The interview filter should not be skipped due to time constraints or for any other factors

• Projects that will require interviews may need to shortlist vendors (if there are too many vendors competing). Shortlisting will be based on all collected information.

63


Key Personnel Interviews• The Client may interview the following individuals:

– Lead Project Manager (overall contact / will be involved on the project every day)– Site Superintendent / Technical Expert– (Key Money Decider and/or Costing Expert)

• All individuals must be available for interviews on the dates specified in the RFP. No substitutions will be allowed. If a team member is not present for the interview, they will jeopardize the team’s competitiveness.

• The client will actually “interview” each individual. This is not a “presentation”

• Goals:– Meet the critical personnel that are being assigned to the project– Identify if the personnel have experience and have thought about this project– Identify if the personnel can think ahead and minimize potential risks

6/26/2014

33


Interview Format• Individuals will be interviewed separately (not as a group).

• No other individuals can be present during interviews. The individuals cannot bring any notes or handouts.

• Interview times will be approximately 15-25 minutes per individual

• A standard set of questions will be asked to each individual. The client has the option to clarify any responses.

• Questions will be non-technical


Best Value Interviews:Identifying Expertise

1. Why were you selected for this project?

2. How many similar projects have you worked on? Individually and as a Team?

3. Describe a similar project you have developed/worked on to the current project.

4. What is different about this project from other projects that you have worked for?

5. Draw out the process for this project by major milestone activities.1. Identify, prioritize, and how you will minimize the risks of this project.2. What risks don’t you control? How will you minimize those risks?3. What do you need from the client and when do you need it?

6. How are you going to measure your performance during the project?

7. What value do you bring to the project in terms of differences based on dollars, quality, expertise, or time?

66

6/26/2014

34


Goal: Minimize Risk

“I have no idea why I am here today”

“My boss called me last night and told me to show up for this interview”

“I did not participate at all in preparing our proposal”

“I am not currently employed by this company, but if we win this project, they will then hire me”

“I have never managed a project of this size/scope”

“There is no risk on this project”

67


Client Demonstrations• Software Projects may require end user demonstrations. • This is a high level demonstration (not a traditional demo).

The detailed/technical demo will be performed at a later stage.

• The purpose of this demonstration is to:– View an actual installed and operating system (that is similar)– Identify how well the end users can use the system– Identify the end users’ satisfaction with the product, system, installation, and services

• Prior to the demonstrations, the State will provide the Proposers with a detailed timeline and script of items that are to be demonstrated.

• The end user will be asked to demonstrate basic product or system functionality.

68

6/26/2014

35


Filter 1Proposal

Evaluations

Filter 2Interview

Key Personnel

Filter 4Cost

ReasonablenessCheck




Best Value)

Contract Award


Short List prior to






Best Value Process


1

2

3

4

1

2

3

4


Prioritization

70

NO CRITERIA POINTS FIRM A FIRM B FIRM C BESTFIRM A

POINTS

FIRM B

POINTS

FIRM C

POINTS

1 Cost 250 145,000$ 150,000$ 170,000$ 145,000$ 250 242 213

2 Interviews 350 4.5 8.1 6.2 8.1 194 350 268

3 Risk Assessment Plan 200 5.1 8.7 7.5 8.7 117 200 172

5 Value Assessment Plan 100 5.0 5.0 5.0 5 100 100 100

6 PPI – Firm (1‐10) 25 9.5 9.2 9.1 9.5 25 24 24

7 PPI – Firm (Surveys) 25 1 5 5 5 5 25 25

8 PPI – Project Manager (1‐10) 25 9.5 9.2 8.8 9.5 25 24 23

9 PPI – Project Manager (Surveys) 25 1 4 2 4 6 25 13

Total 1000 TOTAL POINTS (1,000): 723 990 838

6/26/2014

36


Filter 1Proposal

Evaluations

Filter 2Interview

Key Personnel

Filter 4Cost

ReasonablenessCheck




Best Value)

Contract Award


Short List prior to






Best Value Process


1

2

3

4

1

2

3

4


Dominance Check & Cost Reasonableness

Best-Value is the lowest price

Best-Value is within [X%] of next highest ranked firm

Best-Value can be justified based on other factors

YesNo

Best ValuePrioritizationBest Value

Prioritization

YesNo

Go with AlternateProposal or Cancel

Proceed toPre-Award

YesYes

YesYes

YesYes YesNo

6/26/2014

37


Best Value Model


Filter 1Proposal

Evaluations

Filter 2Interview

Key Personnel

Filter 4Cost

ReasonablenessCheck




Best Value)

Contract Award


Short List prior to






Best Value Process


1

2

3

4

1

2

3

4

6/26/2014

38


Minimize All Surprises• Anything that may impact time, money, or satisfaction

75


How to Clarify a PlanWhat is it / Why is it important?

• Period of time allotted before work begins to the entity doing the work:

– Present their project/service plan

– Set a plan for its delivery / clarify that their plan is accurate

– Identify the risks and issues that could cause the plan to deviate• Identify what you don’t know and when you will know it and

how the plan could change based upon what you discover• Set plans to minimize those risks from occurring• Address all the concerns and risks of the client

6/26/2014

39


How to Clarify a Plan What is it / Why is it important?• Period of time allotted before work begins to the entity doing the work:

– Know how they are being successful and adding value (measurement)• What metrics you will use and how you will report them• What is the current baseline condition we are comparing against

– Identify what you need from the client and have a plan for getting it

– Have completely aligned expectations between all parties so everyone knows what is going to transpire and what they are supposed to do

– Coordinate the schedule


Pre Award Document1. Scope of Work (plan of action, detailed work plan, how technical requirements will

be met, baseline expectations, implementation plan, transitional plan, data migration plan, staffing plan, communication plan, training plan, organization change plan, what’s included, excluded, etc)

2. Financial Summary (financial details, how funding will work, etc)

3. Complete Project Schedule (a coordinated schedule showing major milestones, risky activities, client actions, client action item list, etc)

4. Project Risks/Concerns (all controllable risks/concerns, all non-controllable risks, and solutions)

5. Assumptions (all project assumptions with associated impacts, identify what you need from the client and have a plan for obtaining it, roles and responsibilities of the client, etc )

6. Performance Metrics (how the vendor will monitor performance, document success, metrics used, frequency, baseline for comparison, how will it assist the client, etc)

7. Contract (language, terms and conditions, etc)

78

6/26/2014

40


Clarification / Preplanning PeriodSt

art

Very High Level

Cost Verification

Included in Proposal

Excluded from Proposal

Major Assumptions

Major Client Risks/Concerns

High Level

Project Work Plan

Client Risks/Concerns

PA Schedule

Uncontrollable Risks

Response to all risks

Roles and Responsibilities

Value Added Ideas

Coordination

Review Functionality

Technical Level

Performance Reports / Metrics

Additional Documentation

Technical Details

Project Schedule

High level demos

PA Document

End


Clarification / Preplanning Period

6/26/2014

41


Impact of Clarification/Pre-Award (General Services Administration)

81

No CRITERIA Traditional RFP ASU‐BV

1 Number of projects analyzed 11 10

2 Total awarded cost $14,244,385 $9,994,887

3 Total awarded schedule 1,822 1,373

4 Percent awarded cost below budget 4.4% 6.0%

5 Average time RFP Release to Contract 68 days 78 days

6 Average BV‐PA duration (days) 0 7

7 Average Overall Change Order Rate 50% Decrease

8 Average Overall Project Delay Rate 38% Decrease

9 GSA Satisfaction Rating of Contractor/Job 34% Increase

For within BV projects, also tested “<1 week” PA vs “>1 week” PA

Longer PA had 33% lower change order rate (73% reduced overall)

Longer PA had 69% lower delay rate (73% reduced overall)


Importance of Pre Planning- University of Alberta• Bad news is really good news if found out during Pre-

Planning (Pre-Award / Clarification period)• Execute the project in advance of actually executing the project

• Ex: DB Balmoral Facility – Cyclotron• Highly sensitive schedule & budget• Formal pre-planning enabled the team to optimize the facility

and minimize surprises that could have driven other changes.

6/26/2014

42


Best Value Model


Best Value Model

6/26/2014

43


Filter 1Proposal

Evaluations

Filter 2Interview

Key Personnel

Filter 4Cost

ReasonablenessCheck




Best Value)

Contract Award


Short List prior to






Best Value Process


1

2

3

4

1

2

3

4


Measured Environment• Must be simple and dominant

• Must be for the purposes of positive accountability

• Transparency and openness

• Measuring against a plan (or expectation created by the individual/team doing the work)

86

6/26/2014

44


Weekly Risk Report System• Excel Spreadsheet that is setup by the Client and sent to the vendor

once Award/NTP has been issued

• Vendor must submit the report every week (Friday) until project is complete

87

Arizona State University88

Unforeseen Risks

PERFORMANCE SUMMARY• Vendor Performance• Client Performance• Individual Performance• Project Performance

PROJECT PLAN• Risk• Risk Minimization• Schedule

WEEKLY REPORT• Risk• Unforeseen Risks

METRICS• Time linked• Financial• Operational/Client Satisfac.• Environmental

Measurement of Deviation from the ExpectationManagement by Risk Minimization

6/26/2014

45


Setup Tab

89


Schedule Tab

90

6/26/2014

46


Unforeseen Risks Tab

91

RISK DETAILS:

1. What is the risk / why was it unexpected?

2. What will be done / what is plan to minimize this risk?

3. Who is responsible for resolving the issue?

4. What kind of impact will this have?

5. Any updates to this risk (if applicable)


Approved Modifications

92

6/26/2014

47


Summary Tab

93


U of MN Objectives• The UMN has a goal to be recognized as a top research institution in the

world

• In 2005, CPPM partnered with the PBSRG (ASU) to implement the PIPS Best Value Process

• CPPM’s Objectives of the Best-Value Program are to:– Contract to high performers– Respond faster to customer needs– Increase performance (on time, on budget, high quality)– Increase efficiency of procurement (spend taxpayers money more

efficient)– Create a fair and open process for all vendors

6/26/2014

48


• Award Analysis:– Number of Best-Value Procurements: 161– Awarded Cost: $50.6M (11% below average cost)– Average Number of Proposals: 4– Projects Where Best-Value was also Lowest Cost: 53%– 85% of projects were awarded to vendor with highest / second

highest RAVA Plan (7.3 vs 5.9)

• Performance Information:– Contractor Impacts: 0% Change Orders / 4% Delay– Vendor post project rating: 9.6– Average Contractor Increase in Profit: 5%

Current Results


PM 1 PM 2 PM 3 PM 4

Procurement Officer 1 Procurement Officer 2

Director

Contractor 1

Contractor 2

Contractor 3

Contractor 4

Contractor 3

Contractor 6

Contractor 1

Contractor 8

Contractor 9

Contractor 7

Contractor 7

Contractor 2

Contractor 4

Contractor 8

Contractor 9

Contractor 2

Program Report

Director 1 Director 2

PM 1 PM 2 PM 3 PM 4

Vice President

6/26/2014

49


Report – Overall Program


Directors Report

6/26/2014

50


TEAM 1

(President /

University / Admin)

TEAM 2

Academic Health

Center

TEAM 3

Provost College

1 Total Number of Projects 19 14 5

2 Percent of Projects Procured Using PIPS 79% 86% 80%

3 Total Awarded Cost: $5,359,995 $2,821,005 $2,353,761

4 Average Number of Risks per Project 3 8 12

5 Overall Owner Impacts (Time & Cost) 7.7% 41.3% 41.1%

6 Owner Change Order Rate 0.6% 3.4% 20.0%

7 Owner Delay Rate 7.2% 37.8% 21.1%

8 Percent of Projects without Owner Cost Changes 63% 36% 80%

9 Percent of Projects without Owner Delays 68% 50% 80%

10 Overall Contractor Impacts (Time & Cost) 8.1% 19.6% 14.8%

11 Contractor Change Order Rate 0.1% 0.1% ‐0.8%

12 Contractor Delay Rate 8.0% 19.6% 15.6%

13 Percent of Projects without Contractor Cost Changes 95% 93% 100%

14 Percent of Projects without Contractor Delays 79% 79% 60%

15 Total Number of Completed Projects 4 2 1

16 Total Number of Client Surveys Returned 3 2 1

17 Percent of Projects Evaluated by Client 75% 100% 100%

18 Average PM Post Project Rating of Contractor 6.75 10 10

19 Average Client Post Project Rating of Contractor 7.7 8.5 8.0

20 Average Client Post Project Rating of CPPM 10.7 8.5 7.0

Contractor Impacts

Owner Impacts

Satisfaction Ratings

General Overview

Report – End Users


Report – Internal PM’s

6/26/2014

51


No Contractor

Total

Number of

Projects

Total Awarded

Cost:

Owner

Change

Order

Rate

Owner

Delay

Rate

Vendor

Change

Order

Rate

Vendor

Delay

Rate

Percent

of Late

Reports

Vendor

Performance

1 Contractor 118 3 $ 721,965 0.3% 18.1% 0.2% 66.8% 53% 120%2 Contractor 119 3 $ 220,002 0.7% 10.4% 0.0% 0.0% 69% 69%3 Contractor 120 1 $ 269,850 9.4% 303.0% 0.0% 18.2% 47% 65%4 Contractor 104 3 $ 459,225 1.6% 2.7% 0.0% 18.8% 37% 56%5 Contractor 121 1 $ 241,575 0.0% 21.9% 2.7% 50.0% 0% 53%6 Contractor 105 8 $ 1,611,015 0.3% 32.9% 0.0% 16.3% 32% 49%7 Contractor 106 9 $ 1,280,362 2.2% 31.1% 0.7% 3.2% 35% 39%8 Contractor 122 3 $ 367,650 0.0% 79.1% 0.0% 1.4% 37% 38%9 Contractor 107 1 $ 178,440 0.0% 0.0% 0.6% 11.4% 25% 37%10 Contractor 123 2 $ 3,227,182 14.9% 0.0% ‐0.6% 5.4% 30% 35%11 Contractor 108 2 $ 327,295 0.0% 135.4% 0.0% 0.0% 32% 32%12 Contractor 124 1 $ 69,218 3.5% 0.0% 0.0% 0.0% 31% 31%13 Contractor 125 3 $ 1,150,738 1.9% 7.3% 0.0% 4.2% 26% 30%14 Contractor 109 5 $ 534,095 2.0% 23.2% 0.0% 0.0% 29% 29%15 Contractor 126 1 $ 323,000 3.3% 3.4% 0.0% 6.8% 22% 29%16 Contractor 110 1 $ 308,882 1.2% 24.8% 0.0% 0.0% 27% 27%17 Contractor 127 7 $ 1,793,355 3.8% 13.6% 0.0% 0.0% 26% 26%18 Contractor 128 4 $ 2,956,800 1.3% 1.7% 0.0% 12.2% 11% 23%19 Contractor 129 6 $ 1,319,789 2.2% 16.2% 0.0% 11.0% 9% 21%20 Contractor 111 4 $ 1,096,707 0.1% 0.0% 0.0% 9.8% 10% 19%21 Contractor 112 1 $ 446,100 0.0% 6.9% 0.0% 0.0% 15% 15%22 Contractor 113 3 $ 552,815 5.1% 29.4% 0.0% 7.0% 8% 15%23 Contractor 114 2 $ 1,841,157 13.0% 215.8% 0.0% 0.0% 13% 13%

Report – Contractors


Report – Yearly Analysis

6/26/2014

52


Report – Top 10 Riskiest Projects

No Project Awarded CostAwarded

Duration

Overall

Change

Order

Rate

Overall

Delay

Rate

Percent

of Late

Reports

Risk

Analysis

Factor

PM Director

1 Mayo Remodel Suite A652 $ 269,850 66 9% 321% 47% 377% Wycliffe Waganda Gary Summerville

2 Barn Clean Renovations $ 269,000 80 2% 166% 60% 229% Wycliffe Waganda Justin Grussing

3 WBOB Remodel Suite 150 $ 273,100 99 1% 96% 37% 134% Pete Nickel Gary Summerville

4 Vet Sciences Third Floor $ 96,930 49 3% 86% 28% 116% Pete Nickel Gary Summerville

5Weaver Densford College

of Pharmacy $ 90,862 28 2% 25% 80% 107% Pete Nickel Gary Summerville

6 PWB Remodel Suite 6‐240 $ 127,338 82 17% 23% 64% 104% Steve Bailey Gary Summerville

7 PWB Room 7‐158B $ 46,504 30 0% 0% 100% 100% Pete Nickel Gary Summerville

8Oak Street Parking

Surveillance $ 246,802 74 0% 0% 100% 100% George Mahowald Justin Grussing

9 Snyder Bldg Exterior Door $ 219,000 121 ‐4% 81% 22% 100% Wycliffe Waganda Justin Grussing

10 Heller Hall Renovation $ 1,593,561 254 29% 0% 50% 79% Matt Stringfellow Justin Grussing


Report – Analysis of Risks

Risk CategoryNumber of

Risks

Impact to

Cost

Impact to

Schedule

Percent

Impact to

Cost

Percent

Impact to

Schedule

1) Client Impacts 114 $660,369 1,200 59% 46%

Client Scope Change / Decision 111 660,369$ 976 59% 37%

Client Requested Delay 3 ‐$ 224 0% 9%

2) CPPM Impacts 135 $329,425 885 30% 34%

Design Issue 48 189,876$ 230 17% 9%

CPPM Issue (Codes / Permits) 36 46,140$ 170 4% 7%

CPPM Issue (Energy Mgmt) 2 47,533$ 30 4% 1%

CPPM Issue (Hazardous / Health & Safety) 8 35,407$ 118 3% 5%

CPPM Issue (NTS) 8 10,018$ 64 1% 2%

CPPM Issue (Contract / Payment) 11 ‐$ 132 0% 5%

CPPM Issue (Other) 22 451$ 141 0% 5%

3) Contractor Impacts 43 $21,005 411 2% 16%

Contractor Issue 11 ‐$ 101 0% 4%

Contractor Oversight of Design 9 21,005$ 38 2% 1%

Contractor Issue with Supplier / Sub 23 ‐$ 272 0% 10%

4) Unforeseen Impacts 19 $102,544 111 9% 4%

311 1,113,343$ 2,607

6/26/2014

53


University of Alberta

Since late 2010, Strategic Objectives:

1. Become a measured organization.2. Increase pre-planning and risk minimization

through accountability. 3. Procure and contract with high performing

vendors. 4. Increase the accountability and performance of

vendors throughout the contract lifetime.

105


University of Alberta – Best Value Performance

106

Project Value Cost Savings

Schedule Impacts

Satisfaction / Performance

1. Custodial Services(campus‐wide)

$18M $2M10%

5.5% performanceImprovement

10 (out of 10)

2. DB Construction (Research Facility)

$30M $8‐12M25%

14‐18 months 9.7 (out of 10)

3. Design Services(Building Redevelopment)

$4M $500k12%

0% Cost & Schedule CO’s

$190k in Value Added Options

6/26/2014

54


University of Alberta - Expansion• Butterdome Refacing – Design & Construction

• Fire Alarm Systems – Design & Construction

• Travel Management Services – Bus. Service

• Founders Hall – Design & Construction

• eProcurement Solution – IT Service

• CM under $500k Program – Construction

• CM under $2.5M Program – Construction

• New Residence Hall $37M – DB – Design & Construction

107


City of Peoria, AZ Results• Number of Best‐Value Procurements: 65

• Estimated Budget: $586 Million– (DB/CMAR/JOC)

– (Wastewater, Office Buildings, Fire Station, Parks, Roadway)

– (AE Services, Radio, Maintenance, Software)

• Average Number of Proposals per Project: 6

• Results:

– Overall C/O Rate: 0.01% (compared to 7%)

– Final Results: 100% Satisfaction

– Final Results: 9.1 Rating (10 max)

– 5 Projects where money returned

6/26/2014

55


Rio Vista ProjectNO SUMMARY OF INFORMATION FIRM A FIRM B FIRM C FIRM D FIRM E FIRM G FIRM H

1 Schedule (Days) 598 730 923 587 478 850 630

2 Risk Assessment Plan Rating 7.3 7.3 2.9 5.3 4.4 5.9 6.5

3 DB Firm Performance Ratings (1‐10) 9.9 9.8 9.4 9.2 9.6 9.4 9.5

4 DB Firm Number of Surveys (#) 25 26 5 2 25 9 17

5 Project Manager Performance Ratings (1‐10) 10.0 9.8 8.4 9.2 9.5 7.5 9.7

6 Project Manager Number of Surveys (#) 11 15 3 2 3 1 5

7 Site Superintendent Performance Ratings (1‐10) 9.9 10.0 9.9 9.1 9.5 5.0 9.4

8 Site Superintendent Number of Surveys (#) 7 1 3 1 2 1 2

11 Lead Architect Performance Ratings (1‐10) 9.5 9.5 9.4 8.1 9.8 9.4 9.6

12 Lead Architect Number of Surveys (#) 18 6 5 1 7 8 4

13 Estimator Performance Ratings (1‐10) 10.0 9.6 9.5 5.0 9.9 5.0 9.6

14 Estimator Number of Surveys (#) 15 10 3 1 2 1 5

15 Landscaping Designer Ratings (1‐10) 9.1 9.2 5.0 9.8 9.5 9.5 8.7

16 Landscaping Designer Surveys (#) 13 19 1 5 11 6 2

17 Landscaping Contractor Ratings (1‐10) 10.0 9.0 9.5 7.6 9.9 5.0 9.5

18 Landscaping Contractor Surveys (#) 11 18 6 3 3 1 6

NO SUMMARY OF INFORMATION FIRM A FIRM B FIRM C FIRM D FIRM E FIRM G FIRM H

1 Schedule (Days) 598 730 923 587 478 850 630

2 Risk Assessment Plan Rating 7.3 7.3 2.9 5.3 4.4 5.9 6.5

3 DB Firm Performance Ratings (1‐10) 9.9 9.8 9.4 9.2 9.6 9.4 9.5

4 DB Firm Number of Surveys (#) 25 26 5 2 25 9 17

5 Project Manager Performance Ratings (1‐10) 10.0 9.8 8.4 9.2 9.5 7.5 9.7

6 Project Manager Number of Surveys (#) 11 15 3 2 3 1 5

7 Site Superintendent Performance Ratings (1‐10) 9.9 10.0 9.9 9.1 9.5 5.0 9.4

8 Site Superintendent Number of Surveys (#) 7 1 3 1 2 1 2

11 Lead Architect Performance Ratings (1‐10) 9.5 9.5 9.4 8.1 9.8 9.4 9.6

12 Lead Architect Number of Surveys (#) 18 6 5 1 7 8 4

13 Estimator Performance Ratings (1‐10) 10.0 9.6 9.5 5.0 9.9 5.0 9.6

14 Estimator Number of Surveys (#) 15 10 3 1 2 1 5

15 Landscaping Designer Ratings (1‐10) 9.1 9.2 5.0 9.8 9.5 9.5 8.7

16 Landscaping Designer Surveys (#) 13 19 1 5 11 6 2

17 Landscaping Contractor Ratings (1‐10) 10.0 9.0 9.5 7.6 9.9 5.0 9.5

18 Landscaping Contractor Surveys (#) 11 18 6 3 3 1 6


Results

Top Award 2007 Gold Award

for

Project LeadershipRio Vista Recreation Center

6/26/2014

56


Model Ranking: 90% 87% 89%

NO CRITERIA FIRM C1 FIRM C2 FIRM C3

1 Interview Rating 7.9 6.4 7.4

2 RAVA Plan Rating 6.5 5.7 6.3

3 PPI ‐ GC Firm (1‐10) 9.6 9.9 9.5

4 PPI ‐ GC Firm (Surveys) 18 12 13

5 PPI ‐ Key Individuals (1‐10) 9.4 9.6 9.4

6 PPI ‐ Key Individuals (Surveys) 5 7 6

Model Ranking: 29% 76% 22% 98%

NO CRITERIA FIRM D1 FIRM D2 FIRM D3 FIRM D4

1 Interview Rating 5.9 7.0 6.7 8.2

2 RAVA Plan Rating 6.5 5.2 5.5 6.9

3 PPI ‐ Design Firm (1‐10) 9.8 9.8 9.3 9.5

4 PPI ‐ Design Firm (Surveys) 23 17 6 17

5 PPI ‐ Lead Architect (1‐10) 9.8 9.8 9.5 9.5

6 PPI ‐ Lead Architect (Surveys) 10 9 4 10

7 PPI ‐ Mechanical Engineer (1‐10) 9.8 9.6 9.0 9.8

8 PPI ‐ Mechanical Engineer (Surveys) 5 1 3 9

9 PPI ‐ Electrical Engineer (1‐10) 1.0 9.4 1.0 9.8

10 PPI ‐ Electrical Engineer (Surveys) 1 2 1 6

Fire Station 7

“Gold Medal Design Excellence” (Fire Chief Magazine – 2007)

“Design Excellence Merit Award” (Fire Rescue Magazine – 2007)

Masonry Guild Design Excellence Award ‐ 2008


112

City of Peoria AZ ResultsCriteria Low Bid Best Value Difference % Change

Number of Projects 38 9 NA NA

Awarded Cost $74,181,566 $187,935,047 NA NA

Actual Cost $79,315,696 $188,683,729 NA NA

Average % Over Budget 7% 0.4% -6.5% -94%

Average Change Order % 14% 0.5% -13.5% -96%

% Projects On Budget (Yes/No) 3% 66% 63% 2100%

Awarded Duration (Days) 6016 3792 NA NA

Actual Duration (Days) 8135 4013 NA NA

Average % Delay 35% 6% -29.4% -83%

% Projects On Time (Yes/No) 37% 44% 7% 19%

Owner Satisfaction 20% 93% 73% 365%

6/26/2014

57


What is different• Value focus, simple, transparent, risk-based, measured, expertise-

driven

• Leverage experience of industry experts to minimize risks and increase efficiency

• Select and give advantage to high performers in the procurement process, consider value (cost & ability) (the capability of the key PEOPLE you hire will correlate more to project success than any other factor)

• Proper preplanning BEFORE contract is signed

• Risk-based, value-based contracting

• Consistent measurement of risk and performance with accountability loops

113


Vision• Partner with progressive and innovative

organizations

• Move industry towards value-based, preplanning, and measured (accountable) environment

• Implement and enhance best value concepts, approach, and tools

• Provide education, research, and assistance

114

6/26/2014

58


Tools to Support Best Value

Implementation

115


Supplemental Project Support

116

Best Value Project Kit:

-Online resource for running BV projects

-Chronological roadmap of process steps

-Downloadable templates, documents, models, & training guides

-Common pitfall identification & avoidance

6/26/2014

59



117


- Concise

- Graphics / Visuals

- Agendas and critical action steps



118


- Downloadable templates, models, & training guides

6/26/2014

60



119


- Guides to help prepare for each step



120


- Short (< 5 min) interactive training videos


6/26/2014

61



121






122




- Key process deliverables training

6/26/2014

62



123


-Both Client AND Vendor

performance based studies research group• • www.pbsrg.com

Comments / Questions

124

W W W . P B S R G . C O M

value based solutions to common owner frustrations · 6/26/2014 · • hiring or selecting who...

Documents