THE SCIENTIFIC REVIEW PROGRAM TRAINING PROGRAM

(SRPTP)

CONTRACT REVIEW

December 2015

Disclaimer: The National Institute of Allergy and Infectious Diseases (NIAID) Scientific Review Program (SRP) makes its training documents available to NIH staff as a service. In return for their use we request that you do not share or transmit any of the materials to anyone outside of the NIH. We do our best to keep these documents current, but we do not guarantee the content to be correct or up to date. Documents are reviewed periodically or as needed. If you have any questions or comments, please contact us at NIAIDSRPTrainingCommittee@niaid.nih.gov.

1

Table of ContentsIntroduction.................................................................................................................................................. 5Federal Regulations Governing Contracts...................................................................................................5Primary R&D Solicitation Mechanisms Used by NIAID................................................................................7

Request for Proposals (RFP).................................................................................................................. 7Broad Agency Announcement (BAA)......................................................................................................7Sole Source Solicitations (JOFOC).........................................................................................................8

Parts of a Solicitation................................................................................................................................... 9Schedule (Sections A - H)................................................................................................................... 9Contract Clauses (Section I)................................................................................................................9Lists of Documents, Exhibits, and Other Attachments (Section J).......................................................9Representations and Instructions (Sections K, L, and M)....................................................................9

Grants versus Contracts Comparison Table..............................................................................................10Basics of the Contract Award Process (at a Glance).................................................................................12NIAID Contract Solicitation Development..................................................................................................13

Initiative Planning.................................................................................................................................. 13Prior to First Draft.............................................................................................................................. 13Request for Contract (RFC)/Acquisition Plan Synopsis.....................................................................13

NIAID Contract Proposal Review Process............................................................................................15Request for Proposals Phase (“on the street” in FedBizOpps).............................................................15Proposal Receipt.................................................................................................................................. 15

Pre-Recruiting Tasks......................................................................................................................... 15Meeting set-up................................................................................................................................... 15Read the RFP/BAA Carefully!...........................................................................................................15FRN Preparation............................................................................................................................... 16Administrative Review....................................................................................................................... 16Prepare COI and Recruitment Documents for eRSS........................................................................16Prepare and Upload Review Documents in eRSS............................................................................17Meet With Program and OA Representatives....................................................................................17Recruiting Reviewers........................................................................................................................18Conflict of Interest: Special Considerations.......................................................................................18Assigning........................................................................................................................................... 19Pre-Review Teleconference..............................................................................................................19Before the Meeting............................................................................................................................ 19

Review Meeting Procedures.................................................................................................................20Order of Review................................................................................................................................ 20Initiative Evaluation Feedback...........................................................................................................23

Post-Peer Review Phase......................................................................................................................23SRO/ESA activitiesSource Selection............................................................................................................................... 23Proposal’s Competitive Status and Other Information.......................................................................23Special Review Considerations: Emergency Situations....................................................................24

Basic Contract Categories.........................................................................................................................25Fixed-Price Contracts........................................................................................................................... 25Cost Reimbursement Contracts............................................................................................................25

Other Types of Contracts........................................................................................................................... 25Indefinite-Delivery Contracts................................................................................................................. 25

Key Links for Contract Peer Review..........................................................................................................27Links to Additional Materials...................................................................................................................... 28

2

Basic Contract Review Mailout Package*..................................................................................................29

Goals for This Module Discuss basic contract terminology Discuss Federal regulations governing the acquisition (procurement) process Note main differences between peer review of grant applications and contract proposals Detail proposal peer review process: pre-review, review, post-review Describe common types of research and development (R&D) contracts

AcronymsBAA Broad Agency Announcement IND Investigational New Drug applicationBC Branch Chief IRG Integrated Review GroupCMO Committee Management Office JOFO

CJustification for Other than Full and Open Competition

CO Contracting Officer LOI Letter of IntentCOI Conflict of Interest MQC Mandatory Qualification CriteriaCOR Contracting Officer Representative ND Not DiscussedCR Competitive Range NRFC Not Recommended for Further ConsiderationCS Contract Specialist OA Office of AcquisitionsCSR Center for Scientific Review OD Office of the DirectorDAIDS Division of AIDS OFOC Other than Full and Open CompetitionDAIT Division of Allergy, Immunology, and

TransplantationOGC Office of General Counsel

DEA Division of Extramural Activities PA Program AnnouncementDEAS Division of Extramural Activities Support PI Principal Investigator DMID Division of Microbiology and Infectious Diseases PIP Pre-Initiative PlanningeCPS Electronic Contract Proposal Submission PO Project Officer eRSS Electronic Reviewer Support System R&D Research and DevelopmentER Electronic Review RFA Request for ApplicationseRA Electronic Research Administration RFC Request for ContractESA Extramural Staff Assistant RFI Request for InformationF&A Facilities and Administrative costs RFP Request for ProposalsFACA Federal Advisory Committee Act RIMS Research Initiative Management SystemFAR Federal Acquisition Regulations RSS see eRSSFDA Food and Drug Administration SEP Special Emphasis PanelFOA Funding Opportunity Announcement SOW Statement of WorkFOC Full and Open Competition SPRS Secure Payee Reimbursement SystemFPR Final Proposal Revision SRG Scientific Review GroupFTE Full Time Employee SRO Scientific Review OfficerFY Fiscal Year SSG Source Selection GroupGMP Grants Management Program TEC Technical Evaluation CriteriaGWAS Genome-Wide Association Studies TEG Technical Evaluation GroupHHS Department of Health and Human Services TEP Technical Evaluation PanelHHSAR Department of Health and Human Services

Acquisition RegulationsTESS Technical Evaluation Score Sheet

HS Human Subjects USG United States GovernmentIAR Internet Assisted Review VA Vertebrate AnimalsIDIQ Indefinite Delivery/Indefinite Quantity

3

Introduction This training module introduces the Scientific Review Officer (SRO) to contracts and the contract proposal peer review process at NIAID/SRP.

For a more complete, in-depth overview of the Federal procurement process, types of contracts, roles and responsibilities of the Project Officer vis-à-vis the SRO, and the laws and regulations applicable to US Government (USG) contract acquisitions, the Basic COR Training is a group of specialized NIH/Office of Acquisitions-sponsored courses that are currently available. Also, see the NIAID Guidance for Contract Review and Suggestions for Contract Review .

The Office of Acquisitions (OA) of the NIAID plans, solicits, negotiates, awards, and administers R&D contracts, R&D support contracts, station support contracts (non-R&D), and simplified acquisitions (non-R&D <$100K). R&D contracts address critical needs, such as the acquisition of (services for conducting) clinical trials, vaccine development, statistical and data coordinating centers, animal model development and animal resource centers, and the maintenance of reagent and specimen repositories.

The SRO manages the peer review process to ensure a fair, objective, and uniform evaluation of biomedical and behavioral R&D technical proposals submitted to NIAID. Both intramural and extramural contract proposals are reviewed by SRP.

Unlike a grant, which is an “assistance” mechanism, a contract solicitation is an “acquisition” or “procurement” mechanism used by the USG to obtain the best value for property, supplies and/or services that it needs based on specific requirements and evaluation factors stated in the solicitation.

Mechanism Type NIAID/USG Role

Grant Patron: assistance, encouragement with no substantial Programmatic involvement

Cooperative Agreement Partner: assistance with substantial Programmatic involvement

Contract Purchaser: procurement, acquisition of goods and/or services for direct use by the government with Program oversight

The contract review process is conducted using the electronic Reviewer Support System (eRSS) at http://erss.nih.gov. The SRO manual for eRSS is your step-by-step guide for its use. In some situations, the unsupported Electronic Review (ER) site may be used (see archived training modules).

4

Federal Regulations Governing ContractsThe Federal Acquisition Regulations (FAR) are the primary legal authority used by all Federal agencies when acquiring supplies or services with appropriated funds. The FAR provides for coordination, simplicity, and uniformity in the Federal acquisition process. The FAR classifies all acquisition methods in only two categories: Full and Open Competition (FOC) and Other than Full and Open Competition (OFOC). The US Air Force maintains an easy-to-use, searchable FAR website at http://farsite.hill.af.mil/VFFARa.HTM. The NIH peer review of contract proposals, like grant applications, is mandated by a statute in accordance with section 492 of the Public Health Service Act and Federal regulations governing “Scientific Peer Review of Research Grant Applications and Research and Development Contract Proposals” (Code of Federal Regulations 42CFR52h, sections 1-12).

The Department of Health and Human Services Acquisition Regulations (HHSAR) establish uniform acquisition policies and procedures that conform to the FAR for the Department of Health and Human Services (HHS). The HHSAR implements FAR policies and procedures and provides additional policies and procedures that supplement the FAR to satisfy the needs of HHS. The HHSAR contains all the formal departmental policies and procedures that govern the acquisition process or otherwise control contracting relationships between the department’s contracting offices and contractors.

The NIH Policy Manual 6315-1 for R&D Contracts presents policies and procedures for the initiation, review, evaluation, negotiation, and award of NIH biomedical and behavioral R&D contract projects. It applies to all contract projects for the conduct of R&D and the direct support of the conduct of R&D, including innovative testing, research, demonstration, and related efforts. This chapter supplements the Health and Human Services Acquisition Regulation (HHSAR Part 315).

Severable Services“Severable services” are services that are continuing and recurring in nature (such as basic research, security services, and many R&D support services). The benefit of the contract is realized at the time that services are provided, even if the contract has not been performed to completion. Severable services contracts can be separated into components that independently provide value to meet an agency’s needs.

Severable services fulfill a bona fide need for the fiscal year (FY) the service is performed. Funds are obligated in the FY the service is performed. Contract Options (e.g., to lengthen the term of the contract; to increase the level of effort [FTEs] applied to the contract; or, to change/expand the scope of the contract) are often added to a Base Contract for R&D services because they may be exercised by the USG at a later date for severable services.

Non-Severable Services“Non-severable services” refers to a single undertaking that cannot be feasibly subdivided. Non-severable services produce a single or unified outcome, product, or report.

Non-severable services fulfill a bona fide need that may require multiple FY to satisfy. However, the need is established and MUST be fully funded at the time of award.

Severable Services Versus Non-Severable Services ContractsAdapted from: http://www.hhs.gov/asfr/ogapa/acquisition/funding-of-contracts-exceeding-one-year-06282010.pdf and http://oig.hhs.gov/oas/reports/region3/31003114.asp

Prior to December 20, 2006, HHSAR permitted incremental funding to be applied to cost-reimbursement type contracts for non-severable services. This allowed non-severable cost-reimbursement type contracts, awarded for more than one year, to be funded from succeeding fiscal years. (Incremental funding is distinguished from the funding techniques that may be used for multi-year contracts.) After a review in 2006, the HHS Office of the General Council determined that this is not in compliance with the Bona Fide Needs Rule (Chapter 5), and that incremental funding can be used only for the acquisition of severable services.

See HHS Guidance Regarding Funding of Contracts Exceeding One Year of Performance under Key Links.

Primary R&D Solicitation Mechanisms Used by NIAIDRequest for Proposals (RFP)

5

The RFP is a solicitation to procure products or services, specifying both objectives and desired approaches.

It is used in negotiated acquisitions to communicate USG requirements and to obtain proposals for specific R&D projects, R&D support and non-R&D activities.

RFPs consist broadly of a request for both a “Technical” and a “Business” proposal. SRP receives only “Technical” proposals for peer review; business proposals that have cost and

price information are not reviewed and never shown to reviewers. R&D RFPs contain a common, detailed description of contract technical requirements—the “Statement of

Work” (SOW) to be performed, drafted by the government (see HHSAR sec 307.7106). Other elements of an RFP include: a due date for proposals; addresses for sending proposals;

packaging/delivery instructions; an Institute contact person (typically a Contract Specialist); required reports and delivery schedules for products or services (see SOP); a list of required documents and attachments; representations and certifications; instructions, conditions and notices to offerors, including technical proposal and business proposal instructions; and Evaluation Factors for Award (see Parts of a Solicitation, below).

Most solicitations are competitive; proposals are accepted from any available source that can meet the stated requirements.

Offerors responding to the RFP must describe their capabilities for the stated technical requirements. The review panel will evaluate each proposal against the RFP’s Technical Evaluation Criteria (TEC), which

typically reflect the RFP’s SOW. Generally, after peer review, a competitive range is established among those proposals deemed

Acceptable by the review panel (see definition, page 20), and, after negotiation, the successful Offerors are chosen at a separate Source Selection Meeting.

Primary Basis for Award: Technical Evaluation Factors, cost, past performance, Small Disadvantaged Business participation.

Broad Agency Announcement (BAA)See FAR 35.016 and NIH Policy Manual 6035

A Broad Agency Announcement is a general solicitation that identifies areas of scientific interest or aims, including products and services, in order to advance a specific field.

Offerors are encouraged to submit proposals with varying creative or innovative technical and scientific approaches to advance or increase state-of-the-art knowledge and understanding. Proposals in response to a BAA are not evaluated against a specific, common Statement of Work, as for

RFPs. Instead, the BAA provides research and technical objectives describing the areas in which the government is interested.

The offeror (not the government) develops and defines their own SOW, including specific work requirements and performance specifications. The approaches can vary, since the proposals do not adhere to a common SOW.

The BAA does include specific Technical Evaluation Criteria allowing reviewers to score each proposal submitted and evaluate them as either Acceptable or Unacceptable (see page 20).

After peer review of a BAA, an Order of Merit Ranking is established instead of a competitive range (see below), and, after negotiation, the Contracting Officer (CO), assisted by the Project Officer (PO), chooses the awardees. The BAA mechanism also provides the USG with flexibility to make awards for only those portions or

tasks of the proposal that are of high interest to the government. Award documents will be tailored based on negotiations with the Offerors selected from the Order of Merit Ranking to address identified weaknesses, questions, and areas for clarification, as well as to refine the Offeror’s proposed Statement of Work.

Primary basis for Award: Technical merit, importance to agency programs, availability of funds (and cost realism and reasonableness).

Solicitations are advertised in FedBizOpps. Also see NIAID’s R&D Requests for Proposals home page.

RFPs and BAAs, like Requests for Applications (RFAs), are developed within the Institute by Project Officers within the three Divisions (DAIDS, DAIT, DMID) after Advisory Council input regarding the R&D concepts.

6

Most solicitations are competitive, except for “sole source” contracts. Proposals responding to an RFP, for example, are accepted from any available Offeror (source) who can meet the stated requirements. At the peer review meeting, the objective of the review panel is to provide a fair and thorough evaluation of each proposal against the RFP’s Technical Evaluation Criteria, which typically reflect the RFP SOW. Because proposals in response to RFPs address the same SOW, they compete in fulfilling the RFP requirements. After peer review, OA and Program together usually establish a “competitive range” among those proposals deemed Acceptable (definition of Acceptable and Unacceptable) by the review panel, and negotiations proceed with the offerors in the competitive range. In contrast, a competitive range is not established for proposals in response to a BAA; rather, OA and the Project Officer may select among all Acceptable proposals for negotiations and funding.

Important: Proposals are always evaluated independently during the peer review process; each is judged on its own scientific and technical merits as to how well the Offeror fulfills the SOW requirements (for an RFP), or addresses the broad Statement of Objectives (for a BAA). The Technical Evaluation Criteria are the objective standards by which reviewers are to score all proposals. Proposals are NEVER compared with one another.

Special CasesSole Source Solicitations (JOFOC)Occasionally, SROs may be assigned to oversee review of a sole source contract proposal, also referred to as a JOFOC, an acronym for Justification for Other than Full and Open Competition. This solicitation, in the form of either a letter or a full RFP, contains all the information that the Offeror needs to prepare the proposal, including a detailed description of requirements—the Statement of Work—to be performed, and due dates. Sole source purchases are permitted only under specific, restricted circumstances, and when all competitive procurement methods are either unrealistic or not applicable. The burden of proof for justifying sole source procurement has been raised recently, but still falls to the requester, who must prepare a Sole Source Justification and Approval (J&A). The main justification for adopting a sole source procurement process is when there is, by definition, only one supplier capable of satisfying the requirements set forth by the SOW in the solicitation. See the links above for more details.

Indefinite Delivery/Indefinite Quantity (IDIQ)This type of contract (FAR 16.5) provides for an indefinite quantity of supplies or services during a fixed period of time. The government places task orders for services against the contract, with minimum and maximum limits specified. The government uses an IDIQ contract when the precise quantities of services to be required are not known.

Alternate ProposalsThe SRO may (rarely) encounter an Alternate (or Alternative) Proposal submitted by an Offeror. An Alternate Proposal, which must be clearly identified as such, is one that deviates from the USG-specified Statement of Work in the RFP. It may be considered by the review committee (with CO and PO concurrence), if overall performance would be improved or not compromised and if it is in the best interests of the government. However, the Offeror submitting the Alternate Proposal ALSO must respond to the RFP with a primary proposal that addresses the SOW.

7

Parts of a SolicitationSee Uniform Contract Format-FAR part 14201.1

An RFP or BAA follows a standard boilerplate template known as a Uniform Contract format. The basic organization of any NIAID RFP or BAA includes the following:

Schedule (Sections A - H)The Schedule sections (Sections A - H) specify where, when, and to whom to send the proposal, and give brief descriptions of supplies or services, reporting requirements, inspections, deliveries or performance schedule, and contract administration data (e.g., responsibilities of PO and task ordering procedures) and special contract requirements (Intellectual Property, Good Laboratory Practices, Human Subjects, Vertebrate Animal Welfare, Recombinant DNA, human Embryonic Stem Cell research, subcontracting, security, IT requirements, etc.)

Contract Clauses (Section I)The contract clauses section specifies clauses required by law or regulation, and any other clauses expected to apply to any resultant contract (Code of Business Ethics; Small Disadvantaged Business Participation Program; Privacy or Security Safeguards, Safety and Health; Care of Live Vertebrate Animals, etc.)

Lists of Documents, Exhibits, and Other Attachments (Section J)Section J is a list of the titles, dates and number of pages for each attachment document (e.g., Packaging and Delivery Instructions, Proposal Intent Response Sheet, SOW, Additional Technical Proposal Instructions, Additional Business Proposal Instructions, Additional RFP-Specific Materials, etc.)

Representations and Instructions (Sections K, L, and M)These sections contain solicitation provisions that require certifications, representations, or the submission of other information; also technical proposal instructions for the SOW, other conditions and notices to guide potential Offerors

Evaluation factors for award (Section M) (See HHSAR Sec 315. 204-1 Uniform Contract Format): Identification of factors (other than cost/price) that will be considered in evaluating contract proposals and awarding the contract. This section includes the Technical Evaluation Criteria and their point values for scoring.

Attachments Attachment 1: Packaging and Delivery of the Proposal. This attachment states the Contract Specialist or

Contracting Officer for the initiative, the guidelines for font and margins, the number of hard copies and CDs to be sent, and the page limit.

Attachment 2: Proposal Intent Response Sheet, with the deadline for submission of the Letter of Intent. Letters of Intent are neither required nor binding, but may be used to develop a preliminary conflict of interest list.

Attachment 3: RFP or BAA Description: This attachment includes the Background and Introduction of the RFP or BAA, and Statement of Work for an RFP.

Attachment 4: This attachment typically will have the Additional Technical Proposal Instructions for an RFP, or the Research and Technical Objectives for a BAA.

Attachment 5: This attachment will have the Additional Technical Proposal Instructions for a BAA.

Important: Read attachments 3, 4, and 5 carefully and instruct your reviewers to do the same.

8

Grants versus Contracts Comparison TableGrant Application Contract Proposal

Assistance Acquisition

Applicants submit grant applications Offerors submit contract proposals

Response to Funding Opportunity Announcement (FOA) (RFA/PA/PAR) posted on NIH Guide for Grants & Contracts; submitted through CSR/Grants.gov

Response to FOA as RFP or BAA posted on FedBizOpps; submitted directly to designated OA contact and electronic submission to https://ecps.niaid.nih.gov

SRP or Grants Management Program staff can communicate with Applicant

ONLY Office of Acquisitions staff can communicate with Offeror

Application listed by PI on all documents Proposal listed by Organization (Offeror) on documents

Program Officer oversight Project Officer (PO) oversight pre-award; Contracting Officer’s Representative (COR) post-award

Program Officer is SRO’s primary point-of-contact in pre-review phase

Contract Specialist or Contracting Officer is SRO’s primary point-of-contact throughout review process

Applications received electronically through eRA Commons Proposals received electronically through Electronic Contract Proposal Submission system (eCPS)

Late applications may be accepted by CSR and additional materials are permitted at discretion of SRO

*Generally NO late proposals or additional materials permitted unless it is in best interest of the government

Integrated Review Group (IRG), Special Emphasis Panel (SEP), or Scientific Review Group (SRG)

Technical Evaluation Panel (TEP) or Technical Evaluation Group (TEG)

At meeting, the NIAID SRP practice is that the GMP Specialist (or other GMP representative) sits at a side table separate from Scientific Review Group (SRG)

The Contracting Officer (or designee) must sit at the table with the Technical Evaluation Panel (TEP)

SRG may elect not to discuss a portion of the applications (not discussed [ND])

All contract proposals must be discussed and scored

Out of room conflicts permitted on SRG—conflict of interest (COI) waiver is required to allow a reviewer to be present during the review of an application the reviewer is in conflict with

Conflicts rarely permitted on a TEP, and require COI waiver to allow a conflicted reviewer to serve on TEP

Meeting set up in IMPAC II containing applications Only meeting shell without contract proposals is set up in IMPAC II upon SRO’s request

Assignment list generated by IMPAC II Assignment list created by SRO in Word or Excel

Applications may be critiqued by assigned reviewers only; entire panel scores each application

Technical Evaluation Score Sheet (TESS) forms for each Offeror must be filled out by all reviewers, either prior to (assigned reviewers) or at the review meeting (all reviewers)

IMPAC II for assignments and IAR for electronic review eRSS for recruiting, meeting materials, TESS posting, and scoring

Scoring Scale: 1 - 9 for impact and each of five core criteria; Overall Impact/Priority Score assigned by the SRG after discussion

Proposals are scored based on TEC point weights

Five core review criteria + Overall Impact + special criteria if applicable

Individual TEC point weight scores are not provided in the TER sections of individual offerors

Appl. is scored 1 - 9; not recommended for further consideration (NRFC); or ND (streamlined)

Final score is based on total TEC point weights plus any options, and determined to be either technically Acceptable or Unacceptable by the TEP

9

Final Impact/Priority scores are entered through IAR and are averaged

Final scores are entered in eRSS, checked by OA, and then exported from eRSS from the Scores Tab on the eRSS SRO Site. OA can do this if they are listed as meeting staff in eRSS.

Human Subjects, Vertebrate Animals, and Biohazards are discussed prior to scoring; other Special Issues are discussed after scoring

Special Issues are discussed after scoring and Acceptable/Unacceptable vote, for Acceptable proposals only

Budgets are reviewed by the SRG after scoring and recommendations are provided in Summary Statement (for discussed applications only)

Budget page is discussed by the TEP for Acceptable proposals only after completion of scoring and rating; comments are included in the Technical Evaluation Report (TER)

Anonymous (coded) voting sheet Electronically signed Technical Evaluation Score Sheet (TESS is a legal record)

Payment of reviewers is issued through Electronic Research Administration (eRA) Commons Secure Payee Reimbursement System (SPRS)

Payment is issued through eRA Commons SPRS**

Council Review = second level Source Selection meeting (competitive RFPs only; not BAAs); no Council involvement

Appeal by Applicant Protest from Offeror

Summary Statement: written for each application and released to each applicant via IMPAC II—composed of resume (for discussed applications only) + critiques

Technical Evaluation Report is a consolidation of reviewers’ TESS critiques: a single document containing evaluations of strengths and weaknesses of all proposals reviewed, in rank order

*Acceptance of late proposals decided by OA in conference with Project Officer. ONLY the CO or Contract Specialist (CS) can request extra material/information from an Offeror—similar inquiries to all Offerors are not required. Requested materials/information must be non-substantive and must not result in revision of the proposal.**Reviewers must have an eRA Commons account and be registered in SPRS to be paid.

10

Basics of the Contract Award Process (at a Glance) SRO administratively reviews each proposal and lists required expertise. SRO/ESA lists personnel for COI. SRO convenes the Technical Evaluation Panel (TEP). TEP critiques proposals and scores each according to the weights predefined in the TEC for that

solicitation. Reviewer evaluation of strengths and weaknesses is captured on the TESS prior to the meeting and

amended as needed as discussions proceed. TEP determines by majority vote if each Technical Proposal is Acceptable/Unacceptable (a tie is

interpreted as Acceptable). Final score rankings determine the competitive range (CR) for proposals responding to an RFP (or order of

technical merit for proposals responding to a BAA).o The Contracting Officer establishes the CR for the most highly rated RFP submissions, based, in part, on

ratings of each proposal against all evaluation criteria. The BAA technical merit ranking also is based, in part, on the final scores of each proposal, but a lower scoring proposal could be selected if it fulfills programmatic needs.

o A proposal may be technically Acceptable, but omitted from the competitive range. Unacceptable proposals are not eligible for award, are excluded from the competitive range/order or merit ranking, and do not receive further consideration.

SRO provides a Technical Evaluation Report to Contracting Officer. As the outcome of peer review, the TER summarizes technical proposal strengths, weaknesses, deficiencies, and points for clarification by the Offeror.

The Project Officer then translates weaknesses, deficiencies, and ambiguities identified in the TER into technical questions, which are then provided to Offerors in the CR during negotiations; questions outside of the TER content may also be posed. In addition to Technical Requirements, negotiations may apply to price, schedule, type of contract, or other contract terms.

Offerors respond—Final Proposal Revision (FPR). OA may convene a Source Selection Group (SSG) meeting, which is chaired by the Chief Contracting

Officer. SSG reviews Offeror responses, FPR, and recommends final source(s) (if SRO is invited and attends the

SSG, the SRO role is only as an observer and resource, and the SRO does not speak unless asked to clarify a specific point by the CO). The Offeror’s response may gain back points lost during the Technical Review, resulting in a higher overall score, or the response may cause the loss of additional points, resulting in a lower overall score.

Source selection determination and selection of participants is made by the Chief Contracting Officer with input from the Project Officer.

OA negotiates other factors as needed, and makes award.

11

NIAID Contract Solicitation Development Initiative PlanningFormerly Called Pre-Initiative Planning (PIP)

Prior to First Draft Potential contracts begin as new scientific/technical concepts, which are reviewed and approved by the

NIAID Advisory Council at meetings held around January, May, and September. After Council clearance, OA works with NIAID Program staff to create an RFP or BAA.

Initiative Planning meetings (Team Meetings) are designed to assemble key Program and DEA staff involved in the planning, preparation, quality control, peer review, and award of NIAID-solicited grant and contract initiatives to discuss a broad range of scientific, technical, policy and financial aspects regarding each approved initiative before the development of the first draft PA, RFA, RFP or BAA.

The need for Mandatory Qualification Criteria (MQC) should be addressed at the Initiative Planning meeting. The MQC establish conditions that MUST be met by each Offeror, and must be justified and documented by the PO in conjunction with the CO. MQCs must be stated such that it can be objectively determined if they are present or absent (e.g., Offeror must submit an IND and provide an FDA submission number; Offeror must have proof of patent coverage—number from patent office; Offeror must have a Materials Transfer Agreement in place and provide copy; Offeror must have proof of access to required facilities during the time frame required to carry out the SOW; NOT that the Offeror has a lead drug candidate in hand). Solicitations also must specify if all MQC should be met by the submission deadline or at the time of award. At the Initiative Planning meeting, the SRO should ensure that MQC are such that they can be judged objectively as either “present” or “absent.”

RFPs often include Options: a separate section of the contract SOW to address modifications and additions to the base SOW requirements that the USG may or may not exercise as part of the contract. The RFP can require that options be addressed by the Offeror and scored by the review panel. If required, Offerors must address their capabilities that allow them to fulfill the optional requirements as part of their proposals. As an example, options may include severable milestone completion; extending the award period; adding scientific and technical staffing for the proposed program; expanding existing facilities, equipment or production capacity; modifying or augmenting the management structure to support additional programs and projects; recruiting patients and performing clinical studies or trials; etc.

Ordinarily, once agreed to by the USG and the Offeror, the terms and conditions of a contract (e.g., SOW, period of performance, price, quantity) can be modified by either party only under very specific circumstances (see Contract Modifications SOP and the Distribution of Contract Awards, Modifications and Task Orders SOP).

Request for Contract (RFC)/Acquisition Plan Synopsis When evaluating the draft solicitation document, the SRO should look for potential pitfalls (e.g., excessive or repeated/multiple listings of items in the SOW and/or TEC in Section M “Evaluation Factors for Award”). The SRO should also ensure that:

The SOW is clear and unambiguous; it should not be extensive or highly detailed, but reasonable in length (typically fewer than 10 pages; preferably fewer than five).

The TEC reflect the principal elements of the RFP’s Statement of Work or the BAA’s requirements and guidelines (has anything been overlooked?). If an element is not in the TEC, it will not be scored.

There is no overlap in criteria among major sections of the TEC. The weights assigned to major sections of the TEC are appropriate for what is being asked. There are no more than four to five weighted criteria to be reviewed. Major criteria should not be divided

into numerous subcriteria or Sample Task Orders with low assigned point values (minimum weight = 10 points).

The criteria are broadly worded, rather than attempting to exhaustively list all requirements. The subcriteria are listed either as equally weighted (default) or as descending in weight order.

12

TEC include appropriately weighted criteria for the evaluation of Human Subjects research and Vertebrate Animal Welfare, and address Biohazards, as required by the needs of the solicitation.

If there are Sample Task Orders, be sure that these reflect the required elements. Contract options, if present in the RFP, are included in a separate section of the TEC and are weighted

appropriately. TEC do not address costs or budget-related items.

Other Administrative Considerations The RFP/BAA should specify page limits and font size. The CO, with input from the SRP Branch Chief

(BC) and the PO, will establish page limitations for technical proposals and appendices. The current “standard” page limit for a technical proposal is 150 - 200 total pages, which includes appendices. (Divider pages, table of contents and face page should be omitted from total page count. This should be stated in the RFP.)

RFP instructions should state clearly what counts toward the page limits and describe the action OA will take if a proposal exceeds the page limitations stated in the RFP (e.g., “pages in excess of this limit will be removed without review”).

The RFP should define the organization of proposals, typically through Additional Technical Proposal Instructions (Attachment 4).

13

NIAID Contract Proposal Review Process See the Administrative Review, Conflict of Interest, Recruiting and Assigning, and Preparing the Reviewers training modules for more details.

Request for Proposals Phase (“on the street” in FedBizOpps)The CO or the CS should deliver copies of all Proposal Intent Response Sheets (e.g., Letters of Intent [LOIs]) to the SRO and PO within two working days of the RFP-mandated LOI receipt deadline, and any additional late-arriving LOIs as they come in.

Proposal Receipt1. The CO must notify the SRO of the total number of proposals received by one business day following the

receipt date. The OA should notify the SRO if any late proposals are received. 2. “Responsiveness” does not apply to contract proposals in the same way it applies to grant applications. All

proposals received in response to SOW requirements in the RFP, or the technical objectives in a BAA, are generally accepted by OA. If proposals include Mandatory Qualification Criteria, and the Offeror meets the MQC, they are considered legally responsive, even though they may not address all items in the Statement of Work. All MQC should be met, preferably by the submission deadline; this must be specified clearly in the RFP language in Section M.

3. The OA will typically deliver on-time proposals and related materials to SRP via eCPS to eRSS within three business days of receipt. This may take longer if proposals must be evaluated by the PO for responsiveness and the presence or absence of Mandatory Criteria. Late proposals will be held by OA until a decision on acceptance is made, usually within three to five working days of receipt.

Pre-Recruiting TasksMeeting set-up

The SRO must request a SEP number from the SRP Referral Officer. An IMPAC II meeting “shell” is created that lacks proposals, but reviewers can be added for orientation teleconferences and for the meeting itself. This “shell” allows them to register on the eRA Secure Payee Registration System (SPRS) for reimbursement and honoraria. However, reviewer conflicts will not be viewable, because no proposal assignments can be made in IMPAC II. Assignment lists must be created by the SRO.

Set up meeting in eRSS. Include SRP and OA staff.

Read the RFP/BAA Carefully! Are Mandatory Qualification Criteria given? Are all proposals judged to meet these factors by the PO and

CO? Identify any unique requirements (e.g., location in DC area, material licenses, industry involvement,

experience of personnel, access to required population). Read the Statement of Work of an RFP or the Research and Technical Objectives of a BAA (Attachment). Read the “Additional Technical Proposal Instructions” (in an Attachment). This section contains important

information regarding the preparation of the proposal, including a suggested Table of Contents to be used by the Offerors for the organization of their proposals.

Read the RFP/BAA Section L, Technical Proposal Instructions, and Attachment 3 or 4 for relevant special issue considerations (human subjects protection, clinical trial, data & safety monitoring plan, vertebrate animal welfare, biohazards, select agents, and resource sharing, including genome data sharing).

Determine the general nature of the expertise required. Check (and recheck regularly after the RFP is issued) for amendments to the RFP and Questions and

Answers (posted on FedBizOpps). Watch for RFP-specific requirements, such as Human Subjects, Vertebrate Animal Welfare, Biohazards,

Recombinant DNA, Select Agents, and Data Sharing, where applicable. Are any included in the TEC? If so, they will be scorable.

14

FRN PreparationThe ESA will send the Federal Register Notice (FRN) (fillable form) for your approval and then send it to the Office of Committee Management (OCM) for processing. The FRN must be no later than one month prior to the review meeting, and may be sent as soon as the meeting date and location are established.

Administrative Review See Administrative Review training module

Proposal Compliance While OA determines proposal compliance, the SRO can and should check as well. All RFPs should specify page limits, and consequences for exceeding those limits. Typically, the RFP

should state that “pages in excess of (the stated limits) will be removed from the document without review” by OA. SRP needs to see the removed pages, however, to capture conflict of interest information.

Check that proposals have the sections and sample task orders as outlined in the RFP/BAA. If sections are missing, contact the CS/CO. While the proposal will still be reviewed, advance notice of such deficiencies will alert OA staff to issues that will be brought up by reviewers at the meeting.

Look for hyperlinks—especially “live” links—within the Offeror’s proposal. “Live” links are not permitted for a number of reasons: they may link to sites that can identify the viewer; they may contain information or names that are not on the conflict list; they may be an attempt to circumvent the page limits of a proposal by providing extra information. The SRO must notify the SRP BC and the CO of the presence of hyperlinks in the proposal, and instruct the reviewers not to access the websites in the Offeror’s proposal.

List of Key PersonnelNames of key personnel may be found anywhere in a given proposal, so be sure to comb through them carefully for names and institutions.

Special IssuesMake a preliminary determination of proposal compliance with regard to special issues (human subjects protection, clinical trial, data & safety monitoring plan, vertebrate animal welfare, biohazards, select agents, and resource sharing, including genome data sharing). Note that HS protection codes are used (10, 30, 44) but not VA or HS inclusion codes.

Technical Proposal Cost SummaryDirect cost information (typically one page), provided in the Technical Proposal, may describe Total Labor Hours/percent effort (labor hour base) for all personnel (or as specified in the RFP). It is appropriate for reviewers to comment on this information. However, indirect cost information (facilities and administrative [F&A] costs), or salaries or wages linked to specific personnel, should not be included in the Technical proposal. If present, it should be removed by the CO. Look for compliance with any special RFP requirements for budget presentation.

Note: Information included in the business proposal, but not included in the technical proposal, typically will not be shared with reviewers, except on rare occasions (e.g., percent effort for proposed staff is absent from the Technical Proposal, but present in the Business Proposal). If the direct cost summary is missing from a proposal, tell the panel to disregard the section for budget comments on the TESS form as OA will not pull direct cost information from the Business Proposal for the review panel. However, you will need the total direct cost (all years) for the TER, so be sure to request this figure from OA.

Prepare COI and Recruitment Documents for eRSS Upload or prepare COI list in eRSS or SRDMS. Include all subcontractors and collaborators, which may

be scattered throughout the proposals. SROs can download a completed bulk upload template from SRDMS to upload to eRSS

Prepare Expertise Check-off in eRSS or complete Expertise for the meeting in SRDMS and download a completed Bulk Upload template from SRDMS to eRSS.

Set up calendar to check reviewer availability in eRSS.15

Prepare and Upload Review Documents in eRSS Upload the RFP/BAA as a single PDF that includes all amendments. Prepare the TESS form in eRSS. Prepare the TESS according to the RFP/BAA TEC in eRSS, where a

unique header will be generated for each offeror. When submitted by the reviewer, it is a legal record of participation in the peer review of that contract proposal. Check this document scrupulously for accuracy. Ensure that it lists correct weighting for each TEC, includes any options and their weights, that totals are accurate, and that it conforms to any amendments made in the TEC since the RFP was issued.

Sync proposals from eCPS into eRSS or manually upload proposals. Prepare and upload a cover letter with instructions to reviewers. Prepare and upload the Reviewer’s Manual. Customize and upload the Pre-Review Orientation Teleconference standard slide deck (see Pre-Review

Teleconference).

Meet With Program and OA Representatives See Informing and Reporting Module

The SRO usually meets with Program and OA representatives two to three weeks after receipt. It is the SRO’s responsibility to discuss with the BC, prepare an agenda, schedule, and run the pre-review meeting. An ESA can be in attendance.

Summarize Administrative Review results; provide CO and PO with the COI list and staff pre-review COI Certification forms for original signatures by all proposed staff attendees. Retrieve (signed) prior to meeting.

Outline roles and responsibilities of SRO, Program and OA; ascertain who from Program and OA will be present at the review meeting and what will be presented. The SRO should request a script of the presentations from the PO and OA representatives to facilitate coordination of presentations, ensure adequacy of the time allotted, and assess need for audio-visual equipment at the meeting.

Discuss possible chairpersons and reviewers, and note any PO concerns with proposed reviewers or specialized reviewer expertise. The PO may suggest reviewers, though the SRO is under no obligation to include them on the panel.

Anticipate unusual scientific or technical issues that may arise during the review and discussion of proposals and discuss how they can be addressed during the review; discuss any special guidance to be given to the committee.

Discuss “responsiveness” (i.e., are MQC met by all Offerors? Are key elements of the SOW missing? Can or should OA send the proposal back?)

Provide copy of TESS to CO for review and approval, or list OA staff in eRSS so they can access documents.

Discuss administrative issues. Are any items being requested by the CO? If so, by what date? Get all amendments to the solicitation. Are there Questions & Answers?

Discuss meeting logistics, including whether by teleconference or face-to-face, and tentative review date and place (confirm with BC, CO, PO/COR, and others as needed). This is an SRP decision, but try to accommodate schedules of key staff members, especially the CO. Clarify that all schedules are tentative and flexible.

Note that representation by Program and OA is required at the meeting. The SRO should remind the OA representative that a CO (or CO designee) must arrive promptly and remain at the meeting until (daily) adjournment.

Discuss pre-review orientation teleconference scheduling. Obtain a potential sources sought list from the CO/CS (may be useful for finding reviewers if a given source

did not apply). Discuss roles at the review meeting (e.g., OA to monitor scoring, TESS completion, etc.?).

Recruiting Reviewers See Recruiting and Assigning training module

16

Heads-up to potential reviewers: Contract proposal reviewers are recruited with the admonition that they are required to attend the entire

review (see Contract Meeting Attendance Instruction SOP). Except for teleconference review meetings, teleconference reviewers are permitted only in special circumstances, and must be present for the entire review. SROs must do whatever is possible to include all reviewers in the entire meeting. In situations where reviewers arrive late or depart early use the Contract Meeting Attendance Instructions SOP for guidance. See also “Assigning”, below.

Alert potential reviewers that while you may assign a portion of the proposals to them to critique before the meeting, they will complete a critique (TESS) form for all other proposals under discussion during the meeting.

Personal Information: Instruct reviewers to fill out all information requested on the eRSS Personal Information page.

Conflict of Interest: Special Considerations See COI training module and Contract Meeting Attendance Instructions SOP

Reviewers: Including conflicted reviewers in the peer review of contract proposals is not the norm; however, it may be necessary for a complex review. The participation of conflicted reviewers in any contract proposal review always requires adequate justification and a waiver of COI, which should be done in consultation with OA. There must be a documented record with OA of the agreement to seek a waiver, and OA must be apprised of the waiver status early and often. A waiver must be requested well in advance of the mailout, and must be obtained prior to the review meeting. The reasons for the waiver must be clearly articulated. See the SOP Requesting a Waiver of Conflict of Interest Rules for NIH Peer Reviewers . See past waiver requests at R:\Administrative\Controlled Correspondence for content only, as the format has recently changed. See OER template for the current format. In all cases:

The SRO may request service by reviewers who have real conflicts of interest with one proposal through an out-of-the-room waiver, which would allow the reviewer’s service for the other proposals. In cases of apparent conflict of interest with a proposal, the SRO may request an in-room waiver for service on this and other proposals. See the NIH Policy for Managing Conflict of Interest in the Peer Review of Concepts and Proposals for Research and Development Contract Projects.

The out-of-room conflicted reviewer must not receive copies of any proposal with which he or she is in conflict, nor can he or she be present during the review of that proposal. In addition, a conflicted reviewer should not receive the complete assignment list.

The committee must be large enough so that having fewer people available to score a proposal will not significantly impact the final score. At a minimum, there must be five reviewers available to score a given proposal (FACA requirement).

Substitute or similar expertise must be present among the reviewing panel members.

NIH Staff: All OA, SRP, and Program staff in attendance and/or involved with the competition should sign COI/confidentiality documents to document that NIAID participants have no conflict of interest issues with any Offeror or key personnel (this would include spousal COIs of which the staff member is aware). For a recompeting proposal, a PO (who is now a COR) is not in conflict with proposals from Offerors whom he or she has been managing in his or her official capacity as a government representative, unless the COR publishes/collaborates with the Offeror, or the COR is a voting member of an advisory group or steering committee. These latter situations constitute conflicts and may cause recusal of the COR from the review proceedings. To facilitate early identification of COIs, SROs should provide the Key Personnel/Institution list to OA and Program staff as soon as it is available. SRP management and OA and DEA OD staff should be advised of all potential staff COIs. If there is a concern, the conflicted staff member’s supervisor may be asked to discuss the situation with him or her.

In addition, note whether Emergent Biosolutions (aka Emergent), Imgenex, or Novus Biologicals are among the offerors or subcontractors, as DEA Director Matthew Fenton is in conflict. In the event that any of these are involved, follow the COI mitigation-Dr. Fenton SOP.

Note: Staff members who are contractors may not attend review meetings without a special permission waiver (see SRP SOP Staff Attendance at Closed Review Meetings and this staff

17

waiver routing form . NIAID contractors from the Office of Acquisitions also require waivers to attend closed contract proposal reviews. Attendance by Program staff members who are contractors requires waivers issued through the office of the Director, DEA-OD.

AssigningIn contrast to grant review, all reviewers are “assigned” to all proposals, whether or not they critique a given proposal before the meeting. Thus, all reviewers must complete a TESS for every proposal under discussion with which they have no conflict, either before or during the meeting. This distinction must be made clear upon recruitment.

Ideally, every reviewer will evaluate every proposal and complete a TESS for each before the meeting. If such a workload is prohibitive (e.g., over 7 proposals received), the SRO may “assign” a subset of the proposals for each reviewer to evaluate before the meeting. “Assigned” proposals are listed on the reviewer’s site as “Required” in eRSS, while unassigned proposals are listed as “Optional”.

Pre-Review Teleconference About two weeks after the mailout, SROs should hold a 90min orientation teleconference with the entire

panel to discuss general issues (NOT specific proposals). A standard slide deck is used for the orientation teleconference, which is customized by the SRO, OA, and

the Project Officer. This customized slide deck can be saved in eRSS Meeting Documents. The PO and OA representatives introduce the scientific background of the solicitation and to describe the

acquisition process, respectively. Go over the RFP/BAA, including relevant pages they must read before beginning their critiques. Instruct the SRG on review procedures, including definitions of Weakness, Significant Weakness, and

Deficiency, and use of the eRSS website. Answer reviewer questions. Staff members are to present only factual, non-evaluative information and answer questions from the panel.

Before the Meeting Arrange Pre-Review Meeting with Chairperson: Send meeting guidance to the chairperson and answer

any questions before the meeting; maintain close communication with the chairperson. See the Preparing the Reviewers training module for more information.

Check TESS completion status a few days before the posting deadline. Send reminder notice via email or call reviewers to prompt for last-minute postings. Review completed TESS forms for any policy issues that you need to address at the meeting. Do not

share with Program Staff. ESA staff members assist with the meeting logistics, including the preparation of the meeting folders.

18

Review Meeting Procedures See also Conduct of a Meeting module

Order of Review1. Chairperson calls meeting to order and begins panel member introductions.2. SRO introduces NIH staff.

3. Contracting Officer gives opening comments.4. Project Officer gives opening comments.5. SRO provides instructions to reviewers. Notably:

Reviewers must be present for the entire review. If a reviewer leaves the table or the teleconference review for any reason, a recess must be called.

“Unassigned” reviewers must complete TESS forms for all proposals at the meeting. Reviewers enter any new review comments, or modify prepared comments on their TESS forms in eRSS as the discussions proceed. Remind the panel that they are expected to list strengths and weaknesses for every criterion.

Remind the panel not to assign points for work outside the SOW. (Reinforce in mailout materials and in the pre-review orientation teleconference.)

Weaknesses must always be accompanied by reductions in point value. If points are assigned, they must be accompanied by at least one strength.

Reviewers must not penalize the Offeror twice for the same weakness. In other words, points must not be deducted for the same weakness in multiple sections. Although multiple weaknesses can (and do) arise because of one significant weakness or deficiency (such as “no technical expertise”), the wording of that weaknesses for which points are being subtracted must reflect the TEC under which it is being evaluated, and cannot appear twice anywhere in the TESS. For example, if the deficiency is “no technical expertise” then a weakness in the Personnel section may read, “There is a lack of vaccine formulation expertise among proposed personnel,” whereas the weakness in the Technical Approach section may read, “Offeror does not demonstrate a clear understanding (good working knowledge) of the proposed vaccine formulation.”

Reviewers should identify Deficiencies on their TESS forms for proposals they deem Unacceptable:

FAR Subpart 15.001.Definitions

Weakness: a flaw in the proposal that increases the risk of unsuccessful contract performance

Significant Weakness: a flaw in the proposal that appreciably increases the risk of unsuccessful contract performance

Deficiency: a material failure of a proposal to meet a government requirement or a combination of significant weaknesses in a proposal that increase the risk of unsuccessful contract performance to an unacceptable level

SRO explains voting proposals as Acceptable or Unacceptable by referring to the Contract Review Strategy document.

The SRO should remind reviewers that they can update their critiques and scores in eRSS at any time during the meeting if they feel they have not been using uniform stringency in their critiques.

The TESS is a legal document and is the only evidence of reviewer participation. Thus, the SRO should remind all reviewers to document the strengths and weaknesses of each proposal accurately. Reviewers should be instructed initially—and periodically during the meeting—to take notes on their TESSes during the course of discussion, including major and minor strengths and weaknesses raised by others. A typical group peer review consists of individuals with differing expertise; redundancy in listing strengths and weaknesses among TESSes is preferred; the

19

statement “I agree with assigned reviewers” is insufficient.

6. Chairperson announces first proposal to be reviewed by stating the Offeror and Principal Investigator (PI), and asks Primary Reviewer to introduce the proposal.

7. Primary Reviewer gives a brief non-evaluative overview of the proposal (not the critique), including name and location of Offeror, the PI, any subcontractors involved, and briefly states what is proposed.

8. Chairperson then reads aloud the first TEC in the TESS, in its entirety (generally this is only for the first proposal, then just the general heading is read).

9. Chairperson polls assigned reviewers for preliminary scores for the first TEC, beginning with primary reviewer.

10. Primary reviewer gives critique listing major strengths and weaknesses, followed by secondary and subsequent assigned reviewers who are asked to add only new insights or differing opinions, based on their own expertise and perspectives.

11. The Chairperson opens the floor to full panel discussion of the first TEC. After the discussion has reached its productive limit, the Chairperson then briefly summarizes the main issues discussed (e.g., if there is significant disagreement on a given issue, or if major strengths or weaknesses are identified) and asks for any final discussion.

12. The Chairperson then calls for announcement of scores from all reviewers. 13. Reviewers record final scores on the appropriate TEC sections of their TESSes in eRSS. 14. The Chairperson repeats these steps for each subsequent criterion. 15. After the last criterion has been discussed and scored, reviewers tally their criterion scores on the last

page of the TESS. 16. The Chairperson will then ascertain the committee’s view as to whether the proposal is technically

Acceptable or Unacceptable (see Contract Review Strategy document). The Chairperson first asks the assigned reviewers to state whether they determine the proposal ‘Acceptable’ or ‘Unacceptable’ and provide a brief rationale with an overall proposal summary of Strength/Weakness.

NIH Policy Manual Chapter 6315-1:

Acceptable: a proposal judged to be complete in itself, to contain no major deficiencies, and to present sufficient evidence to indicate that the offeror is capable of satisfying the minimum requirements of the Request for Proposal (RFP) and thus is eligible for consideration for (a) inclusion in a competitive range for a competitive acquisition or (b) award in the case of a noncompetitive acquisition.

Unacceptable: a proposal judged to contain deficiencies, which are so material as to preclude any possibility of upgrading it to a competitive (acceptable) level except through major revisions and additions, which would be tantamount to the submission of another proposal.

17. After assigned reviewers have stated ‘Acceptable’ or ‘Unacceptable’, the Chair asks the entire panel to vote ‘A’ or ‘U’ for that proposal by a show of hands and the SRO and CO record the numbers.

18. If there is disagreement as to Acceptability, the Chairperson may ask for a show of hands or roll call vote—the majority vote determines the outcome. If the panel is evenly split, the proposal is considered Acceptable.

19. For Unacceptable votes, remind reviewers to denote in their TESS which weaknesses or group of weaknesses they deem to be Deficiencies.

20. Panel members then check either the Acceptable or Unacceptable box on their TESS in eRSS and type their name in the Reviewer Name field box in eRSS. This serves as the reviewers’ electronic signature in eRSS. The reviewer then presses the “Submit to SRO” button.

21. For Acceptable Proposals Only: Proceed to Human Subjects, Vertebrate Animals, Biohazards/Select Agents/Recombinant DNA, Resources Sharing (Data/Model Organisms/Genomic Data Sharing) Plans.

20

The chairperson may ask the Primary Reviewer to state his or her opinion and seek concurrence from the rest of the panel. The committee must render opinions, even if those subjects are not included as evaluative factors in the RFP TEC. NIH requirements for Human Subjects (HS) Protection vs. Research Risks, gender, minority and child

representation, and use of live Vertebrate Animals (VA) must be adhered to by contract Offerors. Thus, the offeror must address the four points of human subjects protection and the five points of vertebrate animal welfare. However, the extensive use of HS (e.g., clinical trials) or VA as an intrinsic part of the SOW must be referred to in the TEC so that it can be factored into scoring or in used in the committee’s determination of an Acceptable or Unacceptable proposal. The human subjects protection issues are coded by the SRO as 10 (not involved), 30 (involved, no concerns), or 44 (involved, concerns raised) after the TEP discussion. No codes are assigned for human subject inclusions or for vertebrate animal welfare (see Writing TERs).

Note: NIH can issue a restricted contract award if HS issues have been improperly or incompletely addressed by the Offeror.

If Select Agents, toxins, or Highly Pathogenic Agents are specified in an RFP/BAA, it includes special language that details specific requirements with which all Offerors must comply. For further details see: http://www.niaid.nih.gov/researchfunding/sci/biod/pages/cselectterm.aspx and http://www.niaid.nih.gov/researchfunding/sop/pages/selectagent.aspx. Research with these agents must be referred to in the TEC so it can be factored into scoring.

Resource Sharing Plans Data Sharing Plans apply to contract proposals requesting $500,000 or more in direct costs in any

single year. The reasonableness of the plan must be evaluated by the review committee, but it is not scorable unless included in the RFP TEC.

Model Organism Sharing Plan Submission of this plan is not subject to a cost threshold, and must be evaluated as applicable for every proposal involving model organisms. However, it is typically not scorable unless included in TEC.

Genomic Data Sharing Plan The GDS Policy applies to all NIH-funded research generating large-scale human or non-human genomic data and the use of these data for subsequent research. Large-scale genomic data include Genome Wide Association Studies (GWAS); single nucleotide polymorphism (SNP) arrays; and genome sequence, transcriptomic, epigenomic, and gene expression data. Examples of research within the scope of the GDS Policy may be found within the Supplemental Information to the GDS Policy. The GDS Policy applies to all funding mechanisms (grants, contracts, cooperative agreements, and intramural support) without a threshold for cost. NIH requires all contractors to comply with the NIH Genomic Data Sharing Policy requirements included in the solicitation and contract. The plan is typically not scorable unless included in a TEC.

Next, ask about the budget. Reviewers are asked to comment on the “reasonableness and appropriateness” of the budget information. For example, comments may be solicited on a proposal’s direct costs in certain areas (e.g., staffing, travel, equipment, supplies). Any concerns should be identified, discussed, and noted on the TESS and included in the TER to alert the CO to potential cost evaluation issues.

Additional official documents:

Pre-review confidentiality, lobbyist, and COI forms completed in eRSS. Post-review COI forms completed in eRSS at the end of the meeting. The Meeting Minutes can be electronically signed by the Chair and SRO, and downloaded by the SRO

from eRSS’s Completed Docs tab for archiving within SRP and provided to the Office of Committee Management. (The last page of the TER also includes the SEP Summary signed only by the SRO.)

Initiative Evaluation FeedbackThe SRO or the chairperson should lead a discussion on the Initiative Evaluation Feedback form at the conclusion of the review meeting. Alternately, the SRO may ask reviewers to fill out their forms and return them to the SRO or the ESA before leaving the meeting, or tell the committee that they will do an email post-evaluation (the SRO

21

sends the form via email with instructions to complete and email it back in a few days). The SRO will prepare a written summary of the information gathered, then submit the summary to the BC for approval, who will then forward to admin staff for posting on RIMS (see the RIMS SRP Module Guide for details), so that the DEA Office of Initiative Development and Program can improve future initiative development.

Post-Peer Review Phase SRO/ESA activities

The SRO and ESA submit the SREA report to OCM for reviewer payment processing. The SRO and ESA prepare and send out thank-you letters to reviewers. The SRO prepares the TER, ensuring that the TER accurately reflects the final opinion of the SRG at the

close of the discussion and justifies the vote of Acceptable or Unacceptable for each proposal. The TER is a final summary of the legal TESS documents. See Writing TERs training module.

The TER should clearly denote Deficiencies associated with each Unacceptable proposal. In certain circumstances, and where time is of the essence, the TER may be delivered piecemeal. That is,

a partial TER with the strengths and weaknesses of only those proposals scoring in the competitive range is submitted to the CS in advance of the final TER. The planned delivery of a draft TER or partial TER should be approved by a BC and must be clearly marked as a DRAFT document.

SRP will email the final, signed TER to OA within 120 days of receipt of proposals from OA (in most situations). In exceptional situations (e.g., a large number of proposals responding to the solicitation), reasonable delivery dates will be negotiated between OA and SRP.

Distribute a copy of the final TER electronically to the appropriate SRP BC, Contracting Officer, and Contract Specialist. (See the Informing and Reporting training module.)

The SRO or ESA will provide OA with documents listed on the Document Delivery Form. Archiving of Contract Peer Review Meeting materials within SRP includes the last page of the TER, SEP

Summary, and completed staff COI forms; other items are listed at: AI-PostMtg_ ARCHIVAL CHECKLIST.doc.

Source SelectionThe SRO may be asked to attend the Source Selection meeting, but has no role other than observer. A small subset of the original committee may participate in the Source Selection Group, which reviews Offeror responses to questions prepared from the TER. As a result of these discussions, which are led by the Chief Contracting Officer and include the Project Officer as a voting member, Offerors in the competitive range may gain or lose points assigned to their original proposals. The Source Selection process is designed to identify technically Acceptable proposals for funding that are deemed to provide “best value” to the USG (i.e., the expected outcome provides the greatest overall benefit in response to the requirements).

Proposal’s Competitive Status and Other InformationOA will not provide information on any proposal’s status until after review. At that time, OA will notify all Offerors of their status (in or out of the competitive range). Within three days after the date of contract award, the contracting officer shall provide written notification to each offeror whose proposal was in the competitive range but was not selected for an award. The notice shall include the number of offerors solicited, the number of proposals received, the name and address of each offeror receiving an award, and the items, quantities, and any stated unit prices of each award. If the number of items or other factors makes listing any stated unit prices impracticable at that time, only the total contract price need be furnished in the notice. However, the items, quantities, and any stated unit prices of each award shall be made publicly available, upon request. The notice should also include, in general terms, the reason(s) the offeror’s proposal was not accepted. Offerors may request a debriefing from OA, in writing, on their proposal’s strengths and weaknesses after the competitive range determination or after award to the successful Offeror.

Special Review Considerations: Emergency SituationsUnder US law, all R&D contract proposals must be peer reviewed, and not more than 25% of the peer review committee can be USG employees. In emergency situations, the NIAID SOP for Technical Review of R&D Proposals will apply to sole source or limited competition acquisitions, including letter contracts, resulting from such emergency situations. See the SOP for additional detail. The NIAID SRP has no role in such situations.

22

APPENDICESBasic Contract CategoriesThe basic contract categories that the SRO is likely to encounter are broadly categorized as fixed-price and cost-reimbursement. Each category has variations, but the essential characteristics within each category are the same.

Fixed-Price Contracts See FAR Subpart 16.2-Fixed-Price Contracts

Fixed-price contracts are used to acquire commercial items, supplies, or services based on definable and clear-cut specifications. The risk to the contractor is high and gives the greatest incentive to control costs and perform effectively, while imposing the smallest administrative burden on the USG; thus, fixed-price contracts are strongly encouraged for all projected awards under $150,000. Relatively few contracts of this type are reviewed by SRP.

Cost Reimbursement Contracts See FAR Subpart 16.3-Cost-Reimbursement Contracts

Cost reimbursement contracts are used when requirements impose uncertainty and cannot be translated into the definite, clear-cut specifications. NIAID primarily uses cost reimbursement contracts for research and development, demonstration projects, the conduct of surveys and studies, and related efforts. Cost contracts permit the payment of allowable incurred costs to the contractor. Contractors have less risk for performance and little incentive to control costs, and the USG has higher risk and greater administrative burden. Thus, cost contracts most often estimate total costs and establish a “ceiling” that a contractor may not exceed except at its own risk without approval by a contracting officer. The majority of contract proposals reviewed by SRP are cost reimbursement type contracts.

Other Types of ContractsIndefinite-Delivery Contracts See FAR Subpart 16.5-Indefinite-Delivery Contracts

Indefinite delivery contracts are used when exact times and/or exact quantities of future deliveries are not known at the time of contract award. Orders may be fixed-price or cost reimbursement. The contractor furnishes specified supplies or services to the USG based on the issuance of orders during the ordering period of the contract. Some examples include R&D contracts for new vaccines or other biotechnology products that involve uncertain iterative manufacturing and testing processes and/or efficacy trials that may affect delivery dates. There are three types of indefinite delivery-type contracts as follows:

Indefinite Quantity Contract An Indefinite Quantity (IDIQ) contract provides for the delivery of an indefinite quantity of supplies or services (within stated minimum and maximum limits) during a fixed period. The USG places orders for individual requirements.These are used when the USG cannot predetermine, above a specified minimum, the precise quantities of supplies or services that will be required during the contract period. Preference is given to making multiple awards. Funds for other than the stated minimum quantity are obligated on each order, not on the contract itself. The majority, if not all, of the NIAID’s Indefinite-Delivery contracts are IDIQ contracts.

Requirements Contract A Requirements contract provides for delivery of all actual requirements (usually specifying a

maximum/minimum) for specific supplies or services during a specified contract period, with deliveries scheduled upon issuance of an order. Used when the USG anticipates recurring requirements, but cannot predetermine the precise quantity of supplies or services it will need within a definite period. Funds are obligated by each delivery order, not by the contract itself. This type of Indefinite-Delivery contract is not ordinarily used for R&D requirements requiring peer review.

Definite Quantity Contract A Definite Quantity (IDDQ) contract provides for the delivery of a definite quantity of supplies or services for a

fixed period with the provision of deliveries scheduled upon the issuance of an order. Used when the USG can predetermine the exact total quantity of supplies or services needed during the period of the contract. This type of Indefinite-Delivery contract is not ordinarily used for R&D requirements requiring peer review.

23

For other types of contracts, including Time and Materials, Letter, Performance Based and Inter-/Intra-agency agreements and acquisitions, see Type of Contracts SOP.

24

Key Links for Contract Peer ReviewSite Name Hyperlink Topics

Federal Acquisition Regulations (FAR)

http://farsite.hill.af.mil/VFFARa.HTM Federal Regulations governing procurements and the competitive acquisition process; includes penalties for violations. US Air Force searchable site.

HHS Acquisition Regulations (HHSAR)

http://www.hhs.gov/grants/contracts/contract-policies-regulations/hhsar/index.html

Implements FAR and includes additional policies and procedures to supplement FAR for the needs of HHS.

HHS Guidance Regarding Contract Financing

http://www.hhs.gov/grants/contracts/contract-policies-regulations/hhsar/subpart332/index.html

Provides guidance on contract funding strategies and policies, including contract funding requirements for severable and non-severable services, and standard solicitation provisions and contract clauses to be used when specific contract types will be incrementally funded.

NIH Contract Manual Chapters

https://oma1.od.nih.gov/manualchapters/scripts/mcs/browse.asp

NIH contract manual chapters from the NIH Office of Acquisition and Logistics Management.

NIAID Policy Manual Chapter 6315-1

http://www1.od.nih.gov/oma/manualchapters/contracts/6315-1/6315-1.pdf

Policies and procedures for initiation, review, evaluation, negotiation, and award of NIH biomedical and behavioral (R&D) contracts.

FedBizOpps Site http://www.fedbizopps.gov/ A public website featuring notices of Federal business opportunities (e.g., proposed contract actions, solicitations, contract awards, amendments to contracts, and other procurement information, drafts, requests for information (RFIs), sources sought.

Funding Opportunities & Notices Site

http://grants.nih.gov/grants/guide/index.html

Public NIH website with Funding Opportunity Announcements: both current and archived RFAs and RFPs/BAAs, and PAs. The RFP itself, and any amendments, must be part of the Instructions to Reviewers Manual.

NCI’s RFP Handbook http://rcb.cancer.gov/rcb-internet/forms/OAworkforms/workforms.html

Basic RFP information followed by OA when developing an RFP. Click the RFP Handbook link (Word document).

NIAID Contract Site http://www.niaid.nih.gov/RESEARCHFUNDING/CONTRACT/Pages/about.aspx

Basic contract acquisition information for Offerors. Links to OA staff contacts, instructions, RFPs, awards, glossary, etc.

NIAID R&D Contracts Portal with Links

http://www.niaid.nih.gov/RESEARCHFUNDING/CONTRACT/Pages/default.aspx

All about Contracts, science areas supported, and FAQs. Also links to contract SOPs and to OA contacts.

NIAID Contract Procedures SOPs

http://www.niaid.nih.gov/researchfunding/sop/pages/contracts.aspx#post

SOPs for initiative development, submission, review, negotiation, award, and post-award management.

Archived NIAID RFPs FedBizOpps Archives tab Listing of archived RFPs and amendments.

Finding people in the Office of Acquisitions

http://www.niaid.nih.gov/about/findingpeople/dea/pages/oa.aspx

Office of Acquisitions personnel by branch

Electronic Review (ER) Site

https://review.niaid.nih.gov/era/ NIAID’s contract review back-up ER site used for posting reviewer TESS critiques

Reviewer Support Site (RSS)-Reviewers

http://erss.nih.gov Site for Reviewers—may be used to share contract-related documents with reviewers

Reviewer Support Site (RSS) for SRO

http://erss.nih.gov Intranet website only: May be used to share contract-related documents with reviewers

RIMS http://rims.niaid.nih.gov/ Only BCs and Admin staff currently have access to this site. The Phasing Plan for RFPs is obtained from this site.

25

Links to Additional Materials1. Non-eRSS TESS template and Instructions 2. Assignment sheet example 3. Meeting folder contents list 4. Delivery of documents (SRP Document Delivery Form)5. SOP for obtaining COI waiver for a contract review 6. Previous waiver requests and responses 7. Guide to eRSS for SROs 8. Guide to eRSS for reviewers

Basic Contract Review eRSS Meeting Documents Package*

RFP or BAA solicitation Cover Letter to include list of review documents Orientation Teleconference slide deck, customized by SRO, OA, and Project Officer Tentative Roster Tentative Agenda : Hotel or Teleconference Assignment List SRO prepares in Excel or a Word table Blank Technical Evaluation Score Sheet (TESS) (link is hard copy template), prepared in eRSS Travel Instructions: Federal or non-Federal Instructions for Reviewer Reimbursement and Honorarium payment Meeting Guidance (for CHAIR ONLY) Reviewer’s Manual (comprehensive)

* All documents uploaded to eRSS Meeting Documents must be PDFs.

++++++++++++++++++++++++++++The HHS intranet site that contains the acquisition strategy template, the acquisition plan template and other related documents useful for developing a complex Research and Development contract.

http://intranet.hhs.gov/abouthhs/contracts-grants-support/acquisition-policies-guidance/acquisition-alert-2015-01-new-aquisition-strategy-plan-icge.html

The NIH Policy guidance on the initiation, review and evaluation of Research and Development Contracts.

https://oma1.od.nih.gov/manualchapters/contracts/6315-1/  - The sequence for developing a Research and Development Contract.

26