breakout session # 801 john rush, vice president, trc inc

2

Breakout Session # 801

John Rush, Vice President, TRC Inc

Date Monday, April 23

Time 10:45 – 11:45 am

Source Selection Under Federal Acquisition Regulations Part 15

3

SOURCE SELECTION Using FAR Part 15 – Contracting by Negotiation

“With every mistake we must surely be learning.” - G. Harrison

AN OVERVIEW & LESSONS LEARNED

jrush@acqtrain.com

4

Presentation Agenda• Overview of FAR Part 15

– Competitive Negotiated Contracts• Discussion of generic source selection process

– Section C, Section M, Section L, and the Source Selection Plan• Various Evaluation Processes• Evaluation Factors• Competitive Range and Award Decisions• Debriefings

• Common Failures leading to Sustained Protests– Inconsistent Evaluation of Offers– Failure to Conduct Meaningful Discussions– Failure to award consistent with section M of RFP

• Avoiding the sustained protest• Conclusion and Q & A

5

What do we mean “trade-off” process (Best Value)

• Factors other than price play a significant role in the execution of the contract.

• Tradeoffs among cost factors and non-cost factors can be made in the source selection

• The Government is allowed to accept other than the lowest priced proposal.

• Perceived benefits to government must merit additional costs. (FAR 15.101-1)

6

Trade-off Process – FAR 15.101-1• Best Value tradeoffs

– All evaluation factors and sub-factors and their relative importance shall be clearly stated in the RFP

– RFP shall include a clear indication of the relative weight of non-cost factors/sub-factors to cost factors

– Tradeoffs and benefits to Government must be documented and consistent with RFP

7

Competitive Negotiated Acquisition Process

Define

Requirement

Determine Acquisition Strategy

Develop Acquisition Plan

Develop RFP &SSP

Issue RFP/draft RFPEvaluate Offers, Clarification Issues, Past Performance, Brief SSA

Award w/o discuss

Establish Competitive Range

Conduct Meaningful discussions with all in Competitive RangeDebrief Offerors

8

Competitive Negotiated Acquisition Process

Request Final Revised Proposals

Evaluate FRP in accordance with RFP and SSP, Further Communications & Discussions.

Brief SSA of Strengths & Weakness of Offers in Competitive Range

SSA selects Best Value offer

Award Best Value Contract

Debrief Offerors

9

Source Selection OrganizationSource Selection OrganizationSource Selection

Authority

Source Selection Advisory Council

Source Selection Evaluation Board

Price/Cost

Analysis

Past

Performance

Evaluation

Business

Management

Evaluation

Technical

Evaluation

10

Source Selection Authority ResponsibilitiesSource Selection Authority Responsibilities• Establish an evaluation team tailored for the

instant acquisition• Approve source selection plan before RFP

release• Ensure consistency between RFP, SSP, and

evaluation• Ensure evaluation only as stated in RFP• Consider recommendations of SSAC• Independently select the source(s) whose

proposal is the best value to the government

11

Adequate SSD DocumentationAdequate SSD Documentation

• SSAs have broad discretion in determining how they will make use of evaluation findings

• Tradeoffs and the extent of how the tradeoffs are made are governed only by rationality and the established evaluation criteria

12

Adequate SSD DocumentationAdequate SSD Documentation

• The propriety of tradeoffs will depend not on the difference in technical scores, but on whether the SSA’s judgment concerning the significance of the difference between proposals is reasonable and adequately justified when compared to the evaluation scheme stated in the RFP

13

If this process is so simple,

why do contractors continue to protest government’s “best value” trade-off award decisions?

14

Bid Protests Statistics for FY 2004-2006Bid Protests Statistics for FY 2004-2006FY 2006 FY 2005 FY 2004

Cases Filed 1327 1356 1485

Merit Decisions 249 306 365

Sustains 72 71 75

Sustain % 29% 23% 21%

ADR 91 103 123

ADR Success 96% 91% 91%

15

Bid Protests Statistics for FY 2001-2003Bid Protests Statistics for FY 2001-2003FY 2003 FY 2002 FY 2001

Cases Filed 1352 1204 1146

Merit Decisions 290 256 311

Sustains 50 41 66

Sustain % 17% 16% 21%

ADR 120 145 150

ADR Success 92% 84% 84%

16

Common Reasons for GAO Sustaining Protests

• Agency performs inconsistent evaluations• Agency fails to conduct meaningful

discussions• Inconsistent or inadequate documentation of

source selection decision

17

• Unreasonable evaluation of offeror’s past performance

• Unreasonable cost/price analysis, including cost realism analysis

• Inadequate documentation of evaluation findings

Common Reasons for GAO Sustaining Protests

18

Inconsistent EvaluationsInconsistent Evaluations

• The evaluation must be sufficiently documented to demonstrate a rational relationship to the announced evaluation factors

• Government must demonstrate that evaluations and decisions based on those evaluations are logical and consistent with the RFP

19

Inconsistent EvaluationsInconsistent Evaluations

• An offer that does not meet the minimum requirements can not serve as the basis for a contract

• All offerors must be treated fairly, not necessarily treated equally

• Among many see – Universal Yacht Services B-287071, B-287071.2 April 4, 2001

– Wiltex Inc B297234.2; B-297234.3 December 27, 2005

20

Universal Yacht Services B-287071, B-287071.2 April 4, 2001

• RFP for fixed price task order to provide Personal Transfer Vessel

• Evaluation factors: price, technical characteristics of vessel, past performance

• A performance requirement of the vessel: maintain 9 kts of speed in moderate weather at 80% of rated horsepower

21

Universal Yacht Services

• Petchem was the incumbent and offered to use the same vessel

• Petchem’s price offer was slightly higher than UYS’

• Petchem had outstanding past performance while UYS was a relatively new company with little past performance

22

Universal Yacht Services

• UYS’s vessel hadn’t been used since 1996, but met all of the vessel characteristic requirements

• Contracting Officer had concerns that vessel might need a ‘break-in’ period

• SSA’s decision: award to Petchem based upon Petchem’s score in non-cost factors

23

Universal Yacht Service

• Problem:– Petchem’s offer stated the vessel could maintain

9 kts of speed in moderate weather

• Omitted was the requirement to maintain the speed at 80% of rated horsepower

• Agency stated that Petchem’s vessel had proven it was capable of performing, and that the 80% requirement was a minor issue when everything is considered

24

Universal Yacht Services

• GAO’s view:– a requirement is a requirement

• Petchem’s vessel did not meet the requirements of the RFP and therefore cannot serve as the basis for award

• This agency had to back up and do it again, as well as paying UYS’s legal costs

25

“Meaningful Discussions”Meaningful Discussions”

• Contracting Officers shall discuss– significant weaknesses– Deficiencies– Adverse past performance– any other aspect of the offeror’s proposal that the Contracting

Officer believes can be revised to enhance the proposal’s potential for award (FAR 15.306)

• Among many see– TDS Inc. B-292674 November 12, 2003

– Checchi and Company B-28577 October 10 2000

– Multimax, Inc.; NCI Information Systems, Inc.; BAE Systems Information Technology LLC; Northrop Grumman Information Technology, Inc.; Pragmatics, Inc., October 24, 2006

26

TDS Inc. B-292674 November 12, 2003

• Solicitation for help desk operations and IT system administration and engineering

• Three offerors – Oral presentations– Offerors invited to revise offers based on issues

raised during oral presentations

• The evaluation of offers led to a long list of TDS weaknesses– None of which were raised during oral presentations– Award made on initial offers

27

TDS Inc.

• TDS argued– Agency had an obligation to conduct meaningful

discussions but failed to do so• None of the noted weaknesses were identified to TDS • Agency entered discussions by allowing offerors to revise offers

after oral presentation

• Agency argued– Conducted oral presentations properly– Did not open discussions– No obligation to conduct meaningful discussions

28

TDS Inc.• GAO found in oral proposals

– FAR anticipates “dialogue among parties”– Nothing wrong with agency personnel expressing

their opinions– When agency personnel begin asking questions

rather than listening discussions may take place– Acid Test: has the agency provided an opportunity for

offers to be revised– Reopen the acquisition, engage in meaningful

discussions, request new FPR and re-evaluate for new decision

29

Inconsistent Undocumented EvaluationsInconsistent Undocumented Evaluations

• Evaluations of price and any other evaluation factor must be meaningful

• The documented record must show that the agency acted reasonably and consistently with the stated evaluation factors and applicable statutes and regulations

• Source Selection decisions are required to be documented

• See– Satellite Services B-286508/ .2, January 18, 2001– York Building Services, B296948.2/ .3/ .4 November 3, 2005

30

Satellite Services, Inc. B-286508, B-286508.2 January 18,2001

• RFP for Fixed price and IDIQ facilities support services

• Evaluation Factors: – Price and four technical factors

• Past performance• Experience• Methods and procedures• Corporate resources & management)

• Methods & procedures had 8 sub factors– including Rationale to support proposed FTEs

31

Satellite Services, Inc.

• RFP stated:– Tech proposals will be evaluated to ensure the

offeror understands the requirements

• Three offerors submitted offers with dramatically different staffing levels– Resulted in dramatically differing price proposals

32

Satellite Services, Inc• The record shows that the PET evaluated the

proposals only to ensure the accuracy of the offerors’ calculations and spreadsheets

• PET did not address whether or not the offerors proposed an appropriate number of FTEs

• PET report stated it did not have an accurate or complete government estimate to determine reasonableness

33

Satellite Services, Inc.• The SSA asked an evaluator on the TET to assess

whether or not Satellites’ higher rated technical proposal warranted their higher price

• The evaluator’s assessment claimed to perform an ‘in-depth’ analysis but provided file contained no details– Evaluator concluded incumbent Satellite was too

high• When asked how the analysis was performed the

TET chair stated that they had no guidelines and used his experience with the current contract

34

Satellite Services, Inc.• For the recommended offeror (NVT) the TET used a

different standard, the Navy EPS manual• The record contained no documentation of the

Government’s assessment of the offeror’s proposed methods of accomplishing the work

• In addition, the SSA appears to have relied solely upon the recommendation of the TET in awarding to NVT

• Source Selection Decision: “TET provided a Best Value/Trade –OFF review and determined Satellite Services…”

35

Satellite Services, Inc.

• GAO sustained Satellites’ protest: documentation did not demonstrate the evaluation was reasonable or consistent with the RFP and,

• The SSA must reach an independent decision after making a comparative analysis of the proposals, and that didn’t happen here.

36

Suggestions for Survival of Acquisition Process

• If you fail to plan, then you plan to fail• Develop Acquisition Plan

– Address significant issues and considerations that may influence the acquisition

– Risk management is part of source selection• Business, management, technical, supply chain, logistics,

and any other risks identified and addressed in plan

– Overview of source selection evaluation strategy is included in Acquisition Plan

37

Suggestions for Survival of Acquisition Process

• Don’t “boilerplate” evaluation factors and sub factors– They should be tailored for the current state of nature

• Develop Source Selection Plan, Section M, Section L as SOW is defined

• Resource the Evaluation Team adequately– Right people with the right experience and the right skills

• Train the Evaluation Team– Team training– Close to evaluation time frame

38

Suggestions for Survival of Acquisition Process

• In discussions:– Identify strengths, weaknesses, and deficiencies– Give the offerors the truth and they’ll give you

what you want

• Document the record to demonstrate your logical, reasonable, and consistent findings and decisions

• BE CONSISTENT WITH THE RFP

• You don’t have to treat all offerors the same, but you do have to treat all offerors fairly

39

Questions

Answers

40

Back-up Slides

41

Summary of Acquisition Process

• Define the requirements– Market research– Funding – Draft SOO, SOW, PWS, or Specification

• Determine Acquisition Strategy– Continued market research– How will we satisfy customer’s requirement?– Continued refinement of requirement

42

Summary of Acquisition Process

• Develop Acquisition Plan– Addresses all significant issues and

considerations that may influence the acquisition

– Business, management, technical, logistics, and any other risks identified and addressed in plan

– Overview of source selection evaluation strategy is included in AP

43

Summary of Acquisition Process

• Develop RFP and SSP– RFP establishes terms and conditions of

contract, work requirements, instructions to offerors, and the basis for award.

– RFP often issued to industry in draft form soliciting comments prior to formal issuance

– SSP developed concurrently with RFP– SSP establishes Government’s plan for

evaluating proposals

44

Summary of Acquisition Process

• Develop RFP & SSP– SSP contains who, what, where, when, and how of

evaluation process– SSP and RFP both contain evaluation factors,

sub-factors, and relative order of importance of factors and sub-factors

– SSP and RFP must be consistent– SSP must be approved by SSA prior to formal

issuance of RFP

45

Summary of Acquisition Process

• Formal Issuance of RFP– Consistent with Acquisition Plan– Serves as the basis for offerors’ proposals– Must provide adequate time for offerors to prepare

proposal– Common cut-off date for all offerors– Government personnel enter “period of silence.”

All communication conducted by Contracting Officer

46

Summary of Acquisition ProcessSummary of Acquisition Process

• Evaluate offers iaw RFP & SSP– SEB must evaluate all offers completely and

consistently with RFP and SSP only– Each proposal is evaluated on its merits against

the evaluation factors– Only SSAC or SSA make comparative

evaluations– Strengths, weaknesses, and deficiencies are

documented for each proposal by SEB

47

Summary of Acquisition ProcessSummary of Acquisition Process

• Award without discussions– Generally Govt’s preference is to award w/o

discussion– Cannot award contract to offeror with

deficiencies in proposal– As long as there is 1 offer that meets

requirements and the offeror has acceptable past performance, Govt can award

48

Summary of Acquisition ProcessSummary of Acquisition Process

• Establish Competitive Range– Contracting Officer establishes competitive

range– Comprised of all of the most highly rated

proposals, unless the range is further reduced for purposes of efficiency

– Notify eliminated offerors– Conduct timely requested pre-award debriefs

49

Conduct Meaningful DiscussionsConduct Meaningful Discussions

• The intent of discussions is to allow the offeror to revise its proposal

• Discussions are tailored for each offeror

• Conducted by the Contracting Officer with each offeror in the competitive range

• The government’s objective is to maximize the ability to obtain “best value” in terms of the requirement and evaluation factors

50

Summary of Acquisition ProcessSummary of Acquisition Process

• Request Final Revised Proposals– Notification that discussions are concluded

and FRP shall be in writing– All offerors in competitive range may revise

offeror as the result of discussions– Notification that Government intends to

award without further revisions– Common cutoff date

51

Summary of Acquisition ProcessSummary of Acquisition Process

• Evaluate FRP iaw RFP and SSP– FRP are evaluated by SSEB using same

factors, sub-factors, relative weights, and SSP as the initial evaluation

– FRPs are evaluated in their entirety, not the changes only

52

Summary of Acquisition ProcessSummary of Acquisition Process

• Brief Source Selection Authority– SSEB briefs SSA and/or SSAC of evaluated

strengths and weaknesses – Offers are evaluated on own merits against

evaluation factors and sub-factors– Changes between initial and FRP are often part

of brief– Only SSA or SSAC make comparative

evaluations

53

Summary of Acquisition ProcessSummary of Acquisition Process

• SSA selects “best value” offer– Selection decision must represent independent

judgement of SSA– Shall be based upon comparative assessment of

proposals against all source selection criteria in RFP

– Shall be documented– Documentation shall explain the rationale for

business decisions and tradeoffs

54

Summary of Acquisition ProcessSummary of Acquisition Process

• Contract Award– Contracting Officer notifies the

remaining unsuccessful offerors– Conduct debriefs of timely requested

debriefings

55

Evaluation FactorsEvaluation Factors

• Tailored to fit the acquisition• Represent key areas of importance and

emphasized in source selection decision• Must support meaningful comparison and

discrimination among proposals

56

Evaluation FactorsEvaluation Factors• All factors and significant sub-factors and their relative

importance must be identifies in the RFP (section M)• Relative importance of non-cost factors to cost factors

must be stated• Cost and Quality must be evaluated in every source

selection• Past Performance must be evaluated when expected value

> $100,000• Small Disadvantaged Business Participation in unrestricted

acquisitions > $550,000 ($1,000,000 construction)• Bundling with significant subcontracting, SB participation• No unfavorable treatment of telecommuting

57

Exchanges with OfferorsExchanges with Offerors

• Clarification

• Communications with Offerors before Establishment of Competitive Range

• Exchanges with Offerors after Establishment of Competitive Range

58

ClarificationsClarifications

• May occur when award without discussions is contemplated

• Offerors may make certain aspects of proposal clearer, such as– Relevance of past performance– Rebut adverse past performance that has not

been previously rebutted

59

Communications with Offerors before Communications with Offerors before Establishment of Competitive RangeEstablishment of Competitive Range

• Conducted with offerors whose past performance information is the determining factor for exclusion from the competitive range– Communication addresses adverse pp that the

offeror has not previously rebutted

• Conducted with offerors whose exclusion or inclusion in competitive range is uncertain

60

Communications with Offerors before Communications with Offerors before Establishment of Competitive RangeEstablishment of Competitive Range

• Used to enhance Government understanding of proposals

• Used to allow a reasonable interpretation of proposals

• Used to facilitate Government’s evaluation process

• Leads to the establishment of competitive range

61

Communications with Offerors

• Exchanges with offerors after establishing Competitive Range– These “Discussions” = Negotiations– Intent to allow the offeror to revise its proposal– Tailored for each offeror– Conducted with each offeror in the competitive range

• The government’s objective – maximize the ability to obtain “best value” in terms of

the requirement and evaluation factors

62

Conduct Meaningful Discussions

• FAR does not define “meaningful discussions”

• GAO has clearly developed the concept through its bid protest decisions– Discussions must clearly address issues of FAR

15.306 (d) (3)– Failure to conduct “meaningful discussions” is

one of the more common reasons for GAO sustaining protests

63

“Meaningful Discussions”

• Contracting Officers shall discuss– significant weaknesses– deficiencies– any other aspect of the offeror’s proposal that

the Contracting Officer believes can be revised to enhance the proposal’s potential for award (FAR 15.306)

64

Limitations on Discussions

• Cannot favor one offeror over another• Cannot reveal an offeror’s technical solution• Cannot compromise an offeror’s intellectual

property• Cannot reveal another offeror’s price• Cannot reveal the names of individuals

providing past performance information

65

Two types of Best Value Source Selection evaluations.

• Lowest Price Technically Acceptable– Criteria established as GO/NO GO thresholds– Proposals are evaluated for acceptability– No tradeoffs permitted– Award to lowest evaluated price of technically

acceptable proposal.– Exchanges may occur

66

Other Factors for BundlingOther Factors for Bundling

• Past Performance evaluation must include achievement of SB goals on previous contracts

• SB subcontracting participation in subcontracting plan must be an evaluation factor