analysis of expedited defense contracting …non-traditional contracting tools at the dod’s...
TRANSCRIPT
NAVAL POSTGRADUATE
SCHOOL
MONTEREY, CALIFORNIA
MBA PROFESSIONAL REPORT
ANALYSIS OF EXPEDITED DEFENSE CONTRACTING METHODS IN THE
ACQUISITION OF EMERGING TECHNOLOGY
December 2016 By: Jacob D. Sabin Mark K. Zakner
Advisors: Karen A. F. Landale
Robert Mortlock
Approved for public release. Distribution is unlimited.
THIS PAGE INTENTIONALLY LEFT BLANK
i
REPORT DOCUMENTATION PAGE Form Approved OMB No. 0704-0188
Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instruction, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704-0188) Washington, DC 20503. 1. AGENCY USE ONLY (Leave blank)
2. REPORT DATE December 2016
3. REPORT TYPE AND DATES COVERED MBA professional report
4. TITLE AND SUBTITLE ANALYSIS OF EXPEDITED DEFENSE CONTRACTING METHODS IN THE ACQUISITION OF EMERGING TECHNOLOGY
5. FUNDING NUMBERS
6. AUTHOR(S) Jacob D. Sabin and Mark K. Zakner
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Naval Postgraduate School Monterey, CA 93943-5000
8. PERFORMING ORGANIZATION REPORT NUMBER
9. SPONSORING /MONITORING AGENCY NAME(S) AND ADDRESS(ES)
N/A
10. SPONSORING / MONITORING AGENCY REPORT NUMBER
11. SUPPLEMENTARY NOTES The views expressed in this thesis are those of the author and do not reflect the official policy or position of the Department of Defense or the U.S. Government. IRB number 2016.0118-DD-N.
12a. DISTRIBUTION / AVAILABILITY STATEMENT Approved for public release. Distribution is unlimited.
12b. DISTRIBUTION CODE
13. ABSTRACT (maximum 200 words)
The future of national security rests on more than nuclear weapons, heavy equipment, and conventional forces. Increasingly, security depends on technological advantage, innovation, and asymmetric technology exploitation. Future conflicts will share limited semblance to historical conflicts due to the technology exploitation that characterizes modern warfare. As the U.S. government’s share of research and development (R&D) funding shrinks and defense budgets continue to decline, the Department of Defense (DOD) will increasingly depend on new innovative firms to maintain a technological advantage. Such firms inherently differ from traditional defense acquisition in process and culture. They also enjoy demand from broader domestic and international markets. R&D funding sources affect rights to intellectual property—a major concern for technology firms. The DOD has authority for applying non-traditional contracting methods to better adapt to this competitive marketplace. This project studied non-traditional contracting tools at the DOD’s disposal and their merits, with an analysis of how the DOD can effectively leverage its existing and potential authorities to be a competitive buyer in the emerging technology market. Practitioners in the field provided first-hand accounts of their awareness and experience with non-traditional contracting. Findings include the benefits and limitations of non-traditional methods with recommendations for their selective application.
14. SUBJECT TERMS emerging technology, expedited procurement, non-traditional contracting, other transactions, intellectual property
15. NUMBER OF PAGES
81 16. PRICE CODE
17. SECURITY CLASSIFICATION OF REPORT
Unclassified
18. SECURITY CLASSIFICATION OF THIS PAGE
Unclassified
19. SECURITY CLASSIFICATION OF ABSTRACT
Unclassified
20. LIMITATION OF ABSTRACT
UU
NSN 7540-01-280-5500 Standard Form 298 (Rev. 2-89) Prescribed by ANSI Std. 239-18
ii
THIS PAGE INTENTIONALLY LEFT BLANK
iii
Approved for public release. Distribution is unlimited.
ANALYSIS OF EXPEDITED DEFENSE CONTRACTING METHODS IN THE ACQUISITION OF EMERGING TECHNOLOGY
Jacob D. Sabin, Captain, United States Air Force Mark K. Zakner, Captain, United States Air Force
Submitted in partial fulfillment of the requirements for the degree of
MASTER OF BUSINESS ADMINISTRATION
from the
NAVAL POSTGRADUATE SCHOOL December 2016
Approved by: Karen A. F. Landale, Ph.D. Robert Mortlock, Ph.D. Rene Rendon, D.B.A. Academic Associate Graduate School of Business and Public Policy
iv
THIS PAGE INTENTIONALLY LEFT BLANK
v
ANALYSIS OF EXPEDITED DEFENSE CONTRACTING METHODS IN THE ACQUISITION OF EMERGING
TECHNOLOGY
ABSTRACT
The future of national security rests on more than nuclear weapons, heavy
equipment, and conventional forces. Increasingly, security depends on technological
advantage, innovation, and asymmetric technology exploitation. Future conflicts will
share limited semblance to historical conflicts due to the technology exploitation that
characterizes modern warfare. As the U.S. government’s share of research and
development (R&D) funding shrinks and defense budgets continue to decline, the
Department of Defense (DOD) will increasingly depend on new innovative firms to
maintain a technological advantage. Such firms inherently differ from traditional defense
acquisition in process and culture. They also enjoy demand from broader domestic and
international markets. R&D funding sources affect rights to intellectual property—a
major concern for technology firms. The DOD has authority for applying non-traditional
contracting methods to better adapt to this competitive marketplace. This project studied
non-traditional contracting tools at the DOD’s disposal and their merits, with an analysis
of how the DOD can effectively leverage its existing and potential authorities to be a
competitive buyer in the emerging technology market. Practitioners in the field provided
first-hand accounts of their awareness and experience with non-traditional contracting.
Findings include the benefits and limitations of non-traditional methods with
recommendations for their selective application.
vi
THIS PAGE INTENTIONALLY LEFT BLANK
vii
TABLE OF CONTENTS
I. INTRODUCTION..................................................................................................1 A. BACKGROUND ........................................................................................1 B. PURPOSE STATEMENT .........................................................................3 C. RESEARCH QUESTION .........................................................................4 D. RESEARCH BENEFITS AND LIMITATIONS ....................................4 E. ORGANIZATION OF REPORT .............................................................5 F. SUMMARY ................................................................................................5
II. LITERATURE REVIEW .....................................................................................7 A. INTRODUCTION......................................................................................7 B. TRADITIONAL CONTRACTING PROCESS ......................................7
1. Requirements Definition ...............................................................8 2. Acquisition Strategy.......................................................................8 3. Request for Proposal......................................................................9 4. Evaluation Phase ..........................................................................10 5. Contract Award ...........................................................................10
C. EXPEDITED CONTRACTING PROCESS .........................................12 1. Expedited Solicitation Methods ..................................................12 2. Expedited Contracting Methods .................................................14 3. Agreements ...................................................................................17
D. CONGRESSIONAL AND ORGANIZATIONAL INITIATIVES ......20 1. Small Business Innovation Research ..........................................20 2. Small Business Technology Transfer Research ........................21 3. Rapid Innovation Program .........................................................21
E. SUMMARY ..............................................................................................22
III. METHODOLOGY ..............................................................................................25 A. INTRODUCTION....................................................................................25 B. DATA COLLECTION ............................................................................25 C. SUMMARY ..............................................................................................26
IV. RESULTS .............................................................................................................27 A. INTRODUCTION....................................................................................27 B. MARKET ENVIRONMENT ..................................................................27 C. WARTIME RAPID ACQUISITION AUTHORITY ...........................28 D. RESULTS .................................................................................................29
viii
1. Awareness of Expedited Solicitation/Contracting Methods and Training of the Users ............................................30
2. Authority to Use ...........................................................................31 3. Flexibility of Options ...................................................................32 4. Support within the Government .................................................34 5. Intellectual Property Rights: Collection vs. Protection ............35
E. DEFENSE INNOVATION UNIT EXPERIMENTAL .........................38 F. SUMMARY ..............................................................................................40
V. DISCUSSION OF RESULTS, RECOMMENDATIONS, AND AREAS FOR FURTHER RESEARCH ...........................................................................41 A. INTRODUCTION....................................................................................41 B. STRATEGIC ANALYSIS .......................................................................41 C. DISCUSSION OF RESULTS AND RECOMMENDATIONS ............44
1. Awareness of Expedited Solicitation/Contracting Methods and Training of the Users ............................................45
2. Authority to Use ...........................................................................45 3. Flexibility of Options ...................................................................46 4. Support within the Government .................................................46 5. Intellectual Property Rights: Collection vs. Protection ............46 6. R&D Funding Sources and IP Implications ..............................47
D. SUMMARY ..............................................................................................50 E. AREAS FOR FURTHER RESEARCH .................................................52
APPENDIX A. COMPARISON OF EXPEDITED METHODS ................................55
APPENDIX B. SAMPLE VENDOR LOAN AGREEMENT .....................................57
APPENDIX C. INTERVIEW QUESTIONS ................................................................59
LIST OF REFERENCES ................................................................................................61
INITIAL DISTRIBUTION LIST ...................................................................................65
ix
LIST OF FIGURES
Figure 1. The Acquisition Process. Source: DOD (2012). ..........................................7
Figure 2. Contract Award Steps. Source: DOD (2012). ..............................................9
Figure 3. Rapid Innovation Program Project Selection Process. Source: GAO (2015). ........................................................................................................22
Figure 4. Agency Use of Other Transaction Agreements for FY 2010 through FY 2014. Source: GAO (2016). .................................................................32
Figure 5. SWOT Analysis: Government Acquisition in the Emerging Technology Market ....................................................................................43
Figure 6. SWOT Analysis: Strongest Competitive Buyers in the Emerging Technology Market ....................................................................................43
Figure 7. SWOT Analysis: Developers of Emerging Technology ............................44
Figure 8. Cumulative R&D Expenditures by Funder. Source: “Historical Trends” (2016). ..........................................................................................47
Figure 9. Allocation Ratio of R&D Expenditures by Funder. Source: “Historical Trends” (2016). .......................................................................48
x
THIS PAGE INTENTIONALLY LEFT BLANK
xi
LIST OF ACRONYMS AND ABBREVIATIONS
ACC Army Contracting Command
AFRL Air Force Research Laboratory
APDP Acquisition Professional Development Program
BBP Better Buying Power
CFR Code of Federal Regulations
CO Contracting Officer
CRADA Cooperative Research and Development Agreements
CSO Commercial Solutions Opening
DARPA Defense Advanced Research Projects Agency
DAU Defense Acquisition University
DFARS Defense Federal Acquisition Regulation Supplement
DIUx Defense Innovation Unit Experimental
DOD Department of Defense
FAR Federal Acquisition Regulation
FedBizOpps Federal Business Opportunities web portal
FFRDC federally funded research and development center
FLC Federal Laboratory Consortium
FY fiscal year
GAO Government Accountability Office
GSA Government Services Administration
IP intellectual property
JAG Judge Advocate General
MBA Master of Business Administration
MRAP mine-resistant ambush protected
NASA National Aeronautics and Space Administration
NDAA National Defense Authorization Act
OMB Office of Management and Budget
OSTP White House Office of Science and Technology Policy
OT other transaction
OTA Other Transaction Authority
xii
PALT Procurement Administrative Lead Time
PEO program executive office(r)
PIA partnership intermediary agreement
PM program manager
QDR Quadrennial Defense Review
R&D research and development
RFP Request for Proposal
RIP Rapid Innovation Program
ROM rough order of magnitude
SAM System for Award Management
SAT simplified acquisition threshold
SBA Small Business Administration
SBC small business concern
xiii
ACKNOWLEDGMENTS
We express deepest gratitude to our advisors, family, friends, and interviewees
who supported us in the fulfillment of our MBA degree.
First, a special thank-you to Nickole Sabin for her tireless support and patience
during this process as Jacob stole time away from the family to complete this project.
Next, we thank our research advisors, Lieutenant Colonel Karen Landale and
Colonel (retired) Robert Mortlock, for their enduring patience and time spent reviewing
our project for thoroughness and accuracy; their direction and guidance kept us focused
in this broad endeavor.
Third, we thank our fellow student friends we have grown closer with during our
time at the Naval Postgraduate School.
Lastly, we thank the interviewees who gave time from their busy schedules to
share their perspective in providing valuable insight on how they are tactically applying
the methods discussed in our project.
xiv
THIS PAGE INTENTIONALLY LEFT BLANK
1
I. INTRODUCTION
Over the years, much progress has been made to help Federal agencies gain greater access to the innovation and synergies generated by the commercial marketplace. Despite this progress, the standard procurement processes that agencies rely on to meet most of their needs may remain highly complex and enigmatic for companies that are not traditional government contractors. (White House Office of Science and Technology Policy [OSTP], 2014, p. 2)
A. BACKGROUND
The Department of Defense (DOD) enjoys a degree of success in expedited
procurement via the Defense Advanced Research Projects Agency (DARPA) and Special
Operations Command (SOCOM). Similar to those organizations, the Defense Innovation
Unit Experimental (DIUx) model relies on expedited contracting methods to partner with
non-traditional suppliers. The success of the DIUx venture rests on, among other things,
contracting flexibility and full utilization of expedited procurement methods. Any
shortcomings in this combination may significantly inhibit defense procurement of
emerging technology. Unlike generic commercial-off-the-shelf items, emerging
technology has no substitutes. And unlike non-commercial defense-specific items,
emerging technology enjoys high demand from private industry buyers—the DOD’s
competition in technology acquisition. In this niche sector of emerging technology, the
DOD does not have the luxury of employing a traditional, slow, tedious, and burdensome
procurement process compared to the private sector that translates into price and schedule
growth. In the area of emerging technology, the Department simply risks losing
opportunities to procure emerging technology altogether. That jeopardizes the
Department’s ability to maintain competitive military technological advantage for the
national defense strategy.
With shrinking post-wartime budgets and the shadow of long-term spending
decline hanging over the DOD, the government will need to fully leverage all technology
development sources to capture the most innovative and affordable solutions. Growing
2
technology companies not traditionally involved in defense sales offer promising
opportunities for innovative products with defense application. According to OSTP
(2014), “many of these companies can offer Federal agencies valuable new ways of
solving long-standing problems and cost-effective alternatives for meeting everyday
needs” (p. 2).
In 2015, Secretary Carter announced the formation of DIUx with a mission to
serve as a bridge between U.S. military and companies operating at the cutting edge of
technology (“Work with Us,” n.d.). The future of national defense and military
superiority rests on technological advantage over adversaries. The DOD relies on several
types of sources in technology and engineering development. They include internal
government laboratories, federally funded research and development centers (FFRDCs),
academic research institutions, traditional defense contractors, and non-traditional
suppliers of unique solutions in the areas of cybersecurity, robotics, artificial intelligence,
and machine learning. Capturing technological advantage requires leveraging these
diverse sources of innovation. This report analyzes the last, non-traditional component.
The non-traditional component has the potential to achieve great leaps in
innovation based on market incentives, particularly compared to the more measured
progress found in government laboratories and traditional defense suppliers. To capture
technology development opportunities in the commercial market for innovation, the
Department will need to evolve into being and acting as a competitive customer to the
emerging technology market sector—a sector that currently has limited experience with
government procurement philosophy, has reservations in pursuing the government
market, or has outright apprehension of government intent with regard to intellectual
property. While the notion of acting competitively in the marketplace encompasses the
overall acquisition effort, this report is focused on the available expedited contracting
tools and the DOD’s knowledge, understanding, and application thereof. To close the gap
in the Department’s competitiveness as a buyer of emerging technology, we attempt to
identify barriers to efficient and effective technology acquisition from a contracting
perspective. Possible causes that limit defense technology acquisition efforts may include
the inadequate application of existing expedited contracting tools based on a lack of
3
knowledge, training, or awareness in the defense acquisition workforce. Additionally,
contracting’s risk-averse culture and conventional thinking may also inhibit flexibility.
Finally, the existing expedited methods may actually be inadequate even when applied to
the fullest extent.
Every four years, the DOD issues a review to update strategic defense posture and
priorities.
The 2014 Quadrennial Defense Review (QDR) established innovation as a central line of effort in the national defense strategy of the United States. Asymmetric technological capabilities enabling the U.S. to maintain a decisive military advantage over its adversaries and peer competitors are steadily eroding. Globalization has contributed significantly to a renaissance in commercial innovation fueled by venture capital investment that far exceeds the research and development budget of the Department of Defense (DOD). As a result, the global technology ‘water line’ has risen faster than DOD’s ability to outpace it alone. More so, rogue nations and non-state actors have gained ready access to new technology leading to an advancement in their offensive capabilities. (DIUx, n.d., p. 1)
Consequently, Secretary Carter launched the Defense Innovation Unit Experimental, or
DIUx, “to accelerate the development, procurement, and integration of commercially
derived disruptive capabilities to maintain defense technological lead” (DIUx, n.d., p. 2).
With the premise of technology advancement and application as an item of immediate
importance, DIUx mission is to serve as a bridge between U.S. military and companies
operating at the cutting edge of technology (“Work with Us,” n.d.).
B. PURPOSE STATEMENT
The purpose of this MBA project is to analyze currently available expedited
procurement methods to quickly acquire emerging technologies from small businesses;
compare the methods to those used by DARPA, SOCOM, and Rapid Procurement Cells;
and determine barriers to the DOD’s ability to harness technology acquisitions from
prospective non-traditional suppliers. We intend to develop recommendations for DIUx
to enable expedited acquisition of emerging and innovative products from leading
engineering communities like Silicon Valley.
4
C. RESEARCH QUESTION
In this research, we answer the following question: How should DIUx leverage
expedited contracting methods to acquire emerging technological innovations? In
answering this question, we highlight the strengths and weaknesses inherent in currently
available expedited procurement methods that bolster or reduce the DOD’s ability to be a
competitive customer in this fast-paced industry.
D. RESEARCH BENEFITS AND LIMITATIONS
This project analyzes currently authorized expedited procurement methods to
assess the acquisition community’s level of awareness of and familiarity with non-
traditional procurement tools. We also seek to assess training programs for expedited
procurement methods and the actual application of expedited methods in defense
acquisition activities. We interviewed representatives from DOD contracting offices and
considered vendors’ concerns with the DOD’s acquisition processes to make findings-
based recommendations. The recommendations may lead to improve training programs,
adapt policies, and influence Better Buying Power guidance, DOD and Service
instructions, acquisition regulations, and legislative actions.
The report acknowledges broad acquisition matters such as program management
and interaction between multi-functional partners in the acquisition processes and
communities. While the overall acquisition effort involves processes with a series of
steps, this report focuses on the procurement tools and their application. Changes to DOD
5000 processes and systemic adjustments in the acquisition process are beyond the scope
of this project. Incentivizing the established traditional defense industrial companies to
become more inherently innovative is also beyond the scope of this analysis and can
serve as a promising area for further research.
The intent of this research is to determine the optimal utilization of the various
contracting methods within the boundaries of the existing acquisition framework. Our
review of contracting methods focuses on existing tools, their flexibility, and their merits.
Other factors that limit acquisition flexibility—fiscal appropriations, statutory
5
requirements, regulatory guidance, and organizational culture—can compound and
cumulatively complicate the challenges.
We rely on operational feedback from acquisition professionals to gauge the
effectiveness of non-traditional procurement tools and their adequacy in conducting
emerging technology acquisitions. The interview sample is limited to several respondents
from various DOD organizations. While we believe the respondents, their organizations,
and their responses adequately reflect the overall rapid acquisition community, they still
represent only a limited sample.
With this project, we attempt to reconcile the gap between a shrinking technology
development cycle and a widening acquisition timeline. Our expectation is that defense-
sensitive technology will continue to proliferate at accelerating speeds. We also believe
that increase in research spending will enhance innovation. Lastly, even our most
expedited procurement methods may still be too slow, too restrictive, or too burdensome
for this fast-moving technology industry.
E. ORGANIZATION OF REPORT
Chapter I presented the background, problem statement, purpose statement,
research questions, and benefits and limitations of our research. In Chapter II, we present
a review of the literature pertaining to the traditional contracting process, the expedited
contracting process, and current congressional and organizational initiatives in the
expedited procurement arena. In Chapter III, we discuss our data collection method,
which involved phone interviews with DOD contracting units that use expedited
contracting processes on a daily basis. In Chapter IV, we analyze the results of the
interviews and provide recommendations. Finally, in Chapter V, we summarize our
results and recommend further areas for research on this topic.
F. SUMMARY
The U.S. has been maintaining decades of technological edge over conventional
adversaries. As technology became cheaper and more accessible, the U.S. technological
advantage still remains but with increasing vulnerabilities to exploitation. Developments
6
in emerging technology open new fronts in unconventional warfare, allowing previous
strengths to become new vulnerabilities. Spending large amounts of money on
technology no longer assures security, superiority, or advantage. In today’s complicated
and dynamic environment, the DOD needs to take an offensive and defensive approach to
innovation: capturing leading edge defense technology before adversaries and preventing
adversaries’ access to the same. This is a daunting task because the DOD’s traditional
acquisition processes are not fully compatible with the fast-paced nature of the emerging
technology market. A growing flow of emerging technology comes from new and
innovative firms without experience in government contracting and with no need or
desire to adapt to the DOD’s traditional contracting process. This project studies the
DOD’s efforts to capture the innovative technology market through non-traditional
contracting methods.
7
II. LITERATURE REVIEW
A. INTRODUCTION
This literature review examines the differences between how the government
traditionally conducts acquisitions and how we can expedite the process to buy emerging
technology. As the DOD extends itself into Silicon Valley, a major hurdle the DOD must
overcome is how to attract non-traditional vendors1—a process that likely involves
alleviating the risk-reducing methods and processes of the traditional contracting process.
We begin with a review of the traditional contracting process, then we provide an
overview of the expedited contracting process, and conclude with multiple government
initiatives meant to provide a faster means of contracting for emerging technology.
B. TRADITIONAL CONTRACTING PROCESS
“The traditional contracting process involves all activities associated with
identifying and justifying a [traditional] mission need, formulating an acquisition strategy
to meet the need, and implementing the strategy by means of a contractual relationship
with a vendor” (DOD, 2012, p. 311). This process consists of three phases: pre-
solicitation, solicitation, and source selection (see Figure 1).
Figure 1. The Acquisition Process. Source: DOD (2012).
1 The DOD’s OT Guide for Prototype Contracts defines a non-traditional defense contractor as a
“business unit that has not, for a period of at least one year prior to the date of the OT agreement, entered into or performed on (1) any contract that is subject to full coverage under the cost accounting standards prescribed pursuant to section 26 of the Office of Federal Procurement Policy Act (41 U.S.C. 422) and the regulations implementing such section; or (2) any other contract in excess of $700,000 to carry out prototype projects or to perform basic, applied, or advanced research projects for a Federal agency that is subject to the Federal Acquisition Regulation” (Gansler, 2002, p. 7).
8
1. Requirements Definition
The requirement definition step involves capturing all customer requirements,
performing market research, and constructing a Statement of Work (SOW) or Statement
of Objectives (SOO). The most vital part of this step is to properly and thoroughly
understand what the customer needs. If the customer’s requirement is not fully
identifiable, a SOO is used to seek vendor solutions to the requirement, and a vendor-
proposed SOW is incorporated into the contract. Market research is conducted to
determine the number of capable vendors that can satisfy the agency’s requirement, to
discover whether commercially available or non-developmental items can be used, and to
determine the typical prices and practices of vendors engaged in producing the product or
service (Federal Acquisition Regulation [FAR] 10.001(a)(3), 2016). With the requirement
well defined and market research conducted, the acquisition team can begin to construct
their strategy.
2. Acquisition Strategy
Acquisition planning (or creating the strategy for the acquisition) is the “process
by which the efforts of all personnel responsible for an acquisition are coordinated and
integrated through a comprehensive plan for fulfilling the agency need in a timely
manner and at a reasonable cost. It includes developing the overall strategy for managing
the acquisition” (FAR 2.101, 2016). Planning and market research are meant to promote
the acquisition of commercial items, full and open competition, selection of the
appropriate contract type, and appropriate consideration of pre-existing contracts prior to
the award of any new contracts (FAR 7.102(a), 2016). When the plan is in place, the team
can solicit the requirement. The contracting officer, in consultation with the rest of
acquisition team, determines the appropriate contracting method and vehicle at this stage.
Policy and regulations guide decisions toward traditional procurement approach, unless
market conditions, required item or other external factors require the use of non-
traditional methods.
9
3. Request for Proposal
A Request for Proposal (RFP) is a “formal negotiated solicitation issued for buys
over $100,000 resulting in a formal contract [and] includes the contract form, contract
clauses, work statements, specifications, delivery schedule, and payment terms” (DOD,
2012, p. 314). If an RFP requires manufacturing, the solicitation could include production
cost, quality systems, manufacturing development and demonstration, production,
quality, and manufacturing efficiencies, producibility engineering, and finally, process
control and capability (DOD, 2012, p. 314). With the exception of commercial item
acquisitions and in excess of the simplified acquisition threshold (SAT), an RFP should
allow at least 30 days for proposal responses from vendors (FAR 5.203(c), 2016).
However, if a solicitation for commercial items is synopsized for less than 30 days and
receives only one proposal, the contracting officer (CO) shall resolicit for at least another
30 days (Defense Federal Acquisition Regulation Supplement [DFARS] 205.203 (S-70),
2016). The ultimate goal of issuing an RFP is to receive vendors’ proposals to fulfill the
customer’s needs. Once the proposals are received, the acquisition team evaluates them
(source selection) to determine the winning vendor and issue a contract (see Figure 2).
Figure 2. Contract Award Steps. Source: DOD (2012).
10
4. Evaluation Phase
The evaluation phase focuses on how offerors proposals met the requirements in
the solicitation. Evaluations may be conducted using various rating methods, such as
color or adjectival ratings. Required evaluation factors include “price or cost and quality,
using one or more non-cost factors such as past performance, compliance technical
excellence, management capability, and personnel qualifications” (DOD, 2012, p. 315).
Detailed lists of what the evaluation process includes are provided in RFP Section
L, Instructions to Offerors; and the basis of how the evaluation is conducted is detailed in
Section M, Evaluation Factors for Award. Best value is determined using either the
lowest price, technically acceptable or the tradeoff method of evaluation (the details of
the chosen method are provided in both Sections L and M of the RFP). “All factors and
significant sub-factors that affect the contract award and their relative importance shall be
clearly stated in the solicitation” (FAR 15.304(d), 2016).
Evaluation of proposals is based on the criteria, factors, and priority as specified
in the solicitation. If it is determined that negotiations are in order, the government must
first establish a competitive range with only the highest rated proposals, and comply with
the procedures of FAR 15.306 (2016).
5. Contract Award
The final step in the traditional contracting process is awarding the contract based
on the completion of final evaluations, conclusion of negotiations (if applicable), and
approval of the required clearance documentation. Upon completion of the evaluation
process, the successful offeror is notified by the contracting officer through an executed
contract award document (Standard Form [SF] 26, SF 33, SF 1449, or DOD Document
[DD] 1155) (DOD, 2012, pp. 315–316). Additionally, a notice is sent to all unsuccessful
offerors. Each notice contains the “number of offerors solicited; the number of proposals
received; the name and address of each offeror receiving an award; items, quantities, and
any stated unit prices of each award; and the reason(s) the offeror’s proposal was not
accepted, unless the price information reveals the reason” (FAR 15.503(b)(1), 2016).
11
Finally, the contracting officer will publish a synopsis of the contract award on
FedBizOpps as a final notice to all interested vendors.
The traditional contracting path sets a structure for the workforce to follow. It
values fairness, thoroughness, and consistency over speed or efficiency. It aligns with the
government’s socio-economic and policy objectives. The administrative burdens
accumulate for both the procuring organizations and offerors with negative effects on
efficiency, competitiveness of the government as a buyer, and competition among
vendors willing to meet government’s administrative requirements. Traditional methods
attempt to mitigate procedural risks to the government. Acquisition process can go awry
for a multitude of reasons from unethical behavior to poor contractor performance.
Traditional methods provide a degree of oversight over acquisition workforce conduct
and contractor performance. This oversight also enables procuring offices to better
withstand outside scrutiny. But in following the traditional procurement policy in an
effort to mitigate risk, the defense acquisition community largely disregards the
accompanying costs of traditional acquisition methods. This compliance-centered
approach offers little flexibility but provides a sense of process security and integrity to
the acquisition community and oversight bodies.
Expedited procurement methods, as the name suggests, offer greater speed and
efficiency in attempting to meet acquisition objectives. In employing these alternate
tools, the government frees itself from some of the administrative burdens and lowers the
administrative threshold for vendors to pursue government markets. While expedited
methods offer the potential to better capture competitive markets at a lower cost and
tighter schedule, they also carry perceived or real risk growth in oversight, compliance,
and performance. The spectrum of traditional to expedited methods involves a trade-off
between compliance and speed. Speed also affects risk levels in the acquisition process
and product performance.
12
C. EXPEDITED CONTRACTING PROCESS
This section is divided into three areas: expedited solicitation methods, expedited
contracting methods, and the agreements that are made using these two methods to buy
emerging technology.
1. Expedited Solicitation Methods
Expedited solicitation methods allow flexibility with the schedule or breadth of
requirements compared to the more rigid traditional solicitation method. Expedited
solicitations can also remain open on a continuous basis in recognition that technology
may not yet exist to satisfy the requirements or businesses may need to develop a
technological solution rather than just a price proposal in response to non-traditional
solicitation. When the requirements themselves are not defined, non-traditional
solicitations allow the government to seek for a solution in very broad terms and offer
vendors an opportunity at structuring the solution.
a. Broad Agency Announcement
According to FAR 35.016(a), “Broad Agency Announcements (BAA) may be
used by agencies to fulfill their requirements for scientific study and experimentation
directed toward advancing the state-of-the-art technology or increasing knowledge or
understanding rather than focusing on a specific system or hardware solution” (2016).
The BAA issues a set of needs (similar to the SOO) based on the agency’s future areas of
interest. While the BAA solicitation method is not typically used for specific product
acquisitions, it can be used to demonstrate a technological concept. Also similar to the
traditional solicitation, the BAA expects offerors to propose their company’s solution to
solve the need, but a lengthy full proposal is not required unless the government requests
one.
As with all DOD acquisitions, “proposals received as a result of the BAA shall be
evaluated in accordance with evaluation criteria specified therein through a peer or
scientific review process. Written evaluation reports on individual proposals will be
necessary but proposals need not be evaluated against each other since they are not
13
submitted in accordance with a common work statement” (FAR 35.016(f), 2016). This
peer or scientific review process is dramatically faster than the traditional evaluation
process directed in FAR 15.3 and described in the previous section. For BAAs, “the
primary basis for selecting proposals for acceptance shall be technical, importance to
agency programs, and fund availability. Cost realism and reasonableness shall also be
considered to the extent appropriate” (FAR 36.016(e), 2016).
There are two types of BAAs used to achieve the government’s desired outcome;
the “One-Step” method requires offerors to only submit proposed solutions, while the
“Two-Step” method requires a white paper submission followed by a proposal if the
submitter is chosen as a viable contender. A white paper is a brief summary of an
offeror’s technical approach (with a rough-order-of-magnitude [ROM] price associated)
that generally does not exceed two to three pages in length. Per the Air Force Research
Laboratory’s (2015) Guide for Industry, “The objective of a BAA is to encourage
participation by science and technology firms and educational institutions in meeting Air
Force Research and Development goals for innovative ideas and approaches for research
that is general in nature” (p. 4).
An advantage to using the BAA solicitation method is the ability to award to
multiple offerors. The BAA states the type of awards that can be made, the period of
performance, and the anticipated dollar range for each award. According to FAR Part 6,
“An award under the technique is treated as meeting the statutory requirement in the
Competition in Contracting Act for full and open competition” (FAR 6.102(d)(2), 2016).
There is no guarantee of a government contract resulting from the BAA process.
b. Commercial Solutions Opening
A Commercial Solutions Opening (CSO) is very similar to a BAA in that it is an
open-ended (typically five-year), two-step process for requesting solution briefs and
proposals to solve government questions/problems. DIUx released its own CSO in mid-
2016 when it began working with the Army Contracting Command–New Jersey. The
process begins with the CSO identifying the current capabilities of interest to the DOD.
Next, Silicon Valley vendors submit their solution briefs to solve the identified
14
technological shortcomings. After they review solution briefs, DIUx contacts the vendor
within 30 days if they are interested in the product. Some vendors may even be invited to
pitch or demonstrate their product at DIUx. If the product or solution is determined a
viable solution, the vendor is invited to provide a full proposal, including a statement of
work and a cost proposal. Similar to a BAA, an RFP does not guarantee that a contract
will be let or that funding is available to purchase the product. Finally, should a proposal
be deemed viable and funds are available, the government may choose to make an award.
Awards are fixed price and are made using Other Transaction (OT) Authority.
Unlike the BAA, the CSO results in the award of prototype projects, which
include not only commercially available technologies fueled by commercial or strategic
investment, but also concept demonstrations, pilots, and agile development activities that
can incrementally improve commercial technologies or concepts for defense application
(DIUx, n.d., p. 2). Additionally, the following are a few of the many streamlined benefits
of the CSO: minimal corporate and technical information is provided up front, the
payment terms are negotiable, there is a 30-day response to solution briefs, intellectual
property (IP) rights are negotiable (government does not plan to control IP), vendors
receive direct feedback from DOD customers in the field, and there is potential for
follow-on funding for promising technologies and sponsorship of user test cases for
prototypes and possible follow-on production (DIUx, n.d., p. 2).
2. Expedited Contracting Methods
The following section details the current expedited contracting methods available
to contracting offices. A detailed comparison of each method is included in Appendix A.
a. Rapid Technology Prototyping
Using a BAA to solicit,
rapid prototyping … shall provide for the use of innovative technologies to rapidly develop fieldable prototypes to demonstrate new capabilities and meet emerging military needs. The objective of an acquisition program under this pathway shall be to field a prototype that can be demonstrated in an operational environment and provide for a residual
15
operational capability within five years of the development of an approved requirement. (S. Rep. No. 114-49, 2015, p. 167)
The rapid technology prototyping method consists of “multiple, small, fast, and
cheap acquisitions to ‘try out’ innovative technologies. This method allows the
government rapid testing of available cutting-edge, unproven, but potentially
transformative technological advancements” (OSTP, 2014, p. 5). Success in development
and adaptation of these emerging technologies typically ranges from 10%–30% of all
submitted technologies based on substantial trial and error. Most prototyping projects are
carried out at the vendor’s facility and can last anywhere from three to six months.
Awards are made using firm fixed price contracts with an emphasis placed on non-
traditional vendor participation. The best use of this method is to test innovative solutions
presented by non-traditional vendors.
b. Staged Contracts
Also using a BAA to solicit solutions, the staged contract method is used to “scale
proven solutions and expedite the rapid and inexpensive assessment of many existing or
prototyping private sector technologies ... using a three-phase evaluation process: short
concept paper, invite-only full proposal, and a subsequent one-to-two-year pilot
evaluation period” (OSTP, 2014, p. 7). The underlying advantage of this method is that it
allows the government to test out emerging technologies prior to full commitment and/or
transferring them to programs of record. The best use of this method is the ability to scale
proven solutions by non-traditional vendors.
c. Milestone-Based Competitions
As noted in the Innovative Contracting Case Studies document, “A milestone-
based competition is an innovative contracting model that promotes competition among a
stable pool of selected offerors across a series of clear, technically feasible milestones,
with payment withheld until the associated, agreed-upon milestone is completed” (OSTP,
2014, p. 9). This method is similar to the other methods because it uses the BAA to
solicit; however, the idea of only competing among vetted vendors differentiates this
method from prototyping methods. Furthermore, small businesses in the pool are granted
16
additional access to federal financial resources and private contributions to complete the
task. The best use of this method is to fulfill an agency’s immediate technological need or
the need to discover whether novel solutions exist by non-traditional vendors.
d. Incentive Prizes
Another method for discovering innovative solutions to government problems
involves the use of incentive prizes. This method gets its authority from the America
COMPETES Reauthorization Act of 2010 that permits agencies to “carry out a program
to award prizes competitively to stimulate innovation that has the potential to advance the
mission of the respective agency” (Stevenson-Wydler Technology Innovation Act of
1980). Incentive prizes are used to attract non-traditional vendors and teams toward task
completion knowing they only get paid for successful completion of the task. Incentives
have been used by many government entities to advance R&D activities in health care,
energy, education, and military operations. Finally, incentive prizes are not governed by
the FAR, and thus are subject to fewer administrative burdens. The best use of this
method is to fulfill an agency’s immediate technological needs or to discover whether
non-traditional vendors have novel solutions to a problem.
e. Challenge-Based Acquisitions
When the government has an established requirement or need but is struggling
with actual specifications, a challenge-based acquisition may be appropriate. This method
is a great way to find solutions that may already exist in the market. The government
benefits by only paying for the solution that meets the requirements of the challenge, and
it can be quicker than other methods because it does not require the full development of a
solution. For a project to be a candidate for this method, the following conditions must be
met: (1) the ability to determine the user’s need and decomposition of complex
requirements, (2) the ability to communicate user experiences and needs to industry, (3)
the full design and execution of the challenge event, and (4) the ability to analyze the
results and provide contract or task order awards (OSTP, 2014, p. 13). This method uses
the BAA solicitation process and has been used successfully by many agencies. The best
use of this method is to test innovative solutions presented by non-traditional vendors.
17
3. Agreements
Agreements are contract vehicles used in securing a transaction between the
government and the non-traditional vendor. These agreements are less structured than
traditional contracts; however, basic requirements such as a vendor’s eligibility in the
System for Award Management (SAM) and their ability to receive a government
agreement or contract are still the same. This section details five types of agreements that
can result from expedited contracting methods.
a. Other Transactions Authority
To afford certain procuring offices and non-traditional suppliers an avenue to
bypass administrative acquisition burdens, the Congress-authorized prototype
development outside the FAR. Early users of this special authority referred to this type of
procurements as “other transactions.”
Other Transactions Authority (OTA or OT) are agreements that allow an agency to enter into agreements “other than” standard government contracts or other traditional mechanisms. Agreements under this authority are generally not subject to federal laws and regulations applicable to federal contracts or financial assistance, allowing agencies to customize their other transaction agreements to help meet project requirements and mission needs. (GAO, 2016, p. 1)
This authority is granted per 10 U.S.C. 2371 and Section 845 of the National
Defense Authorization Act (NDAA). Traditional contracting statutes that do not apply to
OTAs include the Truth in Negotiations Act, Competition in Contracting Act, Contract
Disputes Act, and even the Procurement Integrity Act. “OTA gives agencies the
flexibility necessary to develop agreements tailored to a particular transaction” (Willsey,
2016). This selective authority does have statutes of its own to follow, and in accordance
with the “Other Transactions” Authority (OT) Guide for Prototype Projects, OTAs may
only be used in the following conditions:
1. There is at least one non-traditional defense contractor participating to a significant extent in the prototype project
2. No non-traditional defense contractor is participating to a significant extent in the prototype project, but at least one of the following circumstances exists:
18
2a. At least one third of the total cost of the prototype project is to be paid out of funds provided by the parties to the transaction other than the federal Government.
2b. The senior procurement executive for the agency determines in writing that exceptional circumstances justify the use of a transaction that provides for innovative business arrangements or structures that would not be feasible or appropriate under a procurement contract. (Gansler, 2002, p. 8)
In a report to Congress by L. Elaine Halchin (2011) of the Congressional
Research Service, an OT is described as “a special vehicle used by federal agencies for
obtaining or advancing [R&D] or prototypes” (p. 4). The report covers the origins of OTs
and the congressional byproducts authorizing their use by various governmental
departments. Recent legislation included in the fiscal year (FY) 2016 NDAA added
provisions to increase small business and non-traditional offerors in the technology
sector, as well as continuing the usage of the OTA for FY 2016. A recent DARPA
presentation described OT advantages:
Under certain circumstances, OTs may be used as the acquisition instrument to carry out prototype projects that are directly relevant to enhancing the mission effectiveness of military personnel and the supporting platforms, systems, components, or materials proposed to be acquired or developed by the DOD or improvements of platforms, systems, components, or materials in use by the armed forces. (Ulrey, 2016, slide 9)
Recognizing the need to reach the emerging technology base and acquire related
technologies, Under Secretary of Defense for Acquisition, Technology, and Logistics
(USD [AT&L]) Frank Kendall (2015) emphasized the need for OTAs by directing the
development of a handbook of methods and best practices to “develop recommendations
to increase access to innovation within the national security environment through
engaging non-traditional suppliers, entrepreneurs, and inventors” (Kendall, 2015, p. 16).
b. Cooperative Research and Development Agreement
Cooperative Research and Development Agreements (CRADAs) provide federal laboratories with an extremely flexible vehicle to facilitate the transfer of commercially useful technologies from federal laboratories to the nonfederal sector. CRADAs support the broader purpose of providing
19
the means for a federal laboratory to leverage its R&D efforts consistent with the laboratory’s mission. (Federal Laboratory Consortium, 2011, pp. 33–34)
The laboratory director, in deciding what CRADA to use, gives special
consideration to small or other disadvantaged businesses on an as needed basis. “The
laboratory director also gives preference to business units located in the U.S. and to
business units that agree that products or inventions made under the CRADA, or
produced through the use of such inventions, will be manufactured substantially in the
United States” (Cooperative Research and Development Agreements, 2011).
c. Technology Investment Agreements
Formerly known as Other Transactions for Research, a Technology Investment
Agreement (TIA) is similar to a CRADA in that it “requires substantial federal
involvement in the technical or management aspects of the project” (10 C.F.R.
603.1340). This type of agreement is useful when an agency needs to stimulate research
or has a need for research support. This initiative cannot duplicate other research
conducted simultaneously with other programs, and government funds may not exceed
the total amount provided in the cost share agreement (Ulrey, 2016, slide 8).
d. Vendor Loan Agreements
A vendor loan agreement (VLA, see Appendix B for an example) allows the
government to evaluate supplier equipment in an operational environment for a specified
period. The supplier provides the equipment at no cost to the government and bears all
expenses for transportation, installation, removal, operational supplies, and repair parts.
The supplier is also responsible for scheduled and unscheduled maintenance of the
equipment. The agreement does not obligate the government to purchase the item(s) in
the future (DAU, 2011). Special Operations Command (SOCOM) contracting officers
have used VLAs, and one contracting officer (KO) notes that “a government testing
report is the payment in lieu of cash” (KO at SOCOM, personal communication,
August 3, 2016).
20
e. Partnership Intermediary Agreements
A partnership intermediary agreement (PIA) gains its authority from 15 U.S.C.
3715, and is a “contract or memorandum of understanding with an intermediary that
provides for the performance of services for a federal laboratory to increase cooperative
or joint activities with small businesses, institutions of higher education or educational
institutions” (Use of Partnership Intermediaries, 2015). SOCOM uses PIAs to
collaborate, innovate, prototype, and explore with industry, labs, and other academic
partners. For example, “the SOFWERX is a facility that allows SOCOM and their
partners to conduct hands-on experiments to develop a solution and then come back with
a white paper solution to the BAA” (KO at SOCOM, personal communication,
August 3, 2016).
D. CONGRESSIONAL AND ORGANIZATIONAL INITIATIVES
Beyond the processes detailed in Section C of this chapter, Congress has long
recognized the need for speed to acquire emerging technology in the current market and
has passed multiple initiatives as part of the annual National Defense Authorization Act
(NDAA). The DOD has initiatives to “create stronger incentives for industry prime
contractors and DOD program managers to ‘pull’ technology solutions from the DOD’s
Small Business Innovation Research (SBIR) and Small Business Technology Transfer
(STTR) investments, from non-traditional suppliers, and from entrepreneurs—and for
inventors to ‘push’ innovative ideas to program offices and other acquisition
organizations” (Kendall, 2015, p. 16).
1. Small Business Innovation Research
According to the Small Business Administration (SBA), the purpose of the SBIR
program is to “stimulate technological innovation; use small businesses to meet Federal
R&D needs; foster and encourage participation by socially and economically
disadvantaged small businesses in technological innovation; and increase private sector
commercialization of innovations ... thereby increasing competition, productivity and
economic growth” (SBA, 2014a, p. 3). The focus of SBIR is to “ensure that the DOD
makes it as easy as possible for small businesses with creative and innovative
21
technologies to work with the DOD and have their technologies included in the products
that the DOD acquires” (Kendall, 2015, p. 16). Participation in SBIR program precludes
companies from using outside venture capital funding for the same project. This
exclusionary funding policy limits the opportunity to pool various funding sources for
development of dual-use technology.
2. Small Business Technology Transfer Research
Congress established the Small Business Technology Transfer (STTR) program in
1992 in an effort to utilize industry and institutional problem solving techniques. “It is
similar in structure to SBIR and funds cooperative research and development projects
with small businesses in partnership with not-for-profit research institutions (such as
universities) to move research into the marketplace” (DOD, 2015). The STTR program’s
purpose is to “stimulate a partnership of ideas and technologies between innovative small
business concerns (SBCs) and Research Institutions through Federally-funded R&D”
(SBA, 2014b, p. 3). STTRs and SBIRs are small business enablers designed to go beyond
basic research to create government and nongovernmental products.
3. Rapid Innovation Program
The Rapid Innovation Program (RIP), known as the Rapid Innovation Fund
within the DOD, was originally established in the FY 2011 NDAA, and is “designed to
fund innovative technologies, reduce life-cycle costs, address technical risks, improve
timeliness of test and evaluation outcomes, and rapidly insert technologies needed to
meet critical national security needs” (NDAA for FY 2016, 2015, p. 51). According to
the Defense Innovation Marketplace website, some of the key requirements to utilizing
the RIF include innovative technology stimulation, the reduction of acquisition and life
cycle costs, improved timeliness and thoroughness of Test & Evaluation (T&E),
technology transfer within 24 months of award, and a cost of less than $3 million (DOD,
2016, slide 5). Of the 263 awards made in FY 2011 and FY 2012 under the RIP, over
90% were made to small businesses, and a majority of those were leveraged from
investments made under the SBIR program (NDAA for FY 2016, pp. 51–52). The RIP is
designed to “deliver new technologies to acquisition programs and operational units that
22
may not otherwise be funded or evaluated given the fiscal constraints and traditional
program management processes currently in place” (NDAA for FY 2016, p. 52). “The
[RIP] process is somewhat lengthy (see Figure 3), taking about 18 months to implement
and award contracts, but interest from contractors has been high” (GAO, 2015, p. 6).
Figure 3. Rapid Innovation Program Project Selection Process. Source: GAO (2015).
E. SUMMARY
The federal acquisition process is established in regulation that dictates how
government acquisition professionals purchase services to supplies, but in a fast-paced
and ever-changing technology environment, acquisition professionals must step outside
of day-to-day practices and apply expedited contracting methods to procure emerging
technology. If an organization is trying to scale a proven solution, the staged contract
23
method should be used. Likewise, if an organization is interested in testing innovative
solutions, the challenge-based acquisition or the rapid technology prototyping methods
are most appropriate. Use of the milestone-based contract or the incentive prize would
best fulfill an agency’s need to discover novel solutions from non-traditional vendors.
Should a solution or product be deemed viable to meet the needs of the government, an
OTA or SBIR/STTR agreement can further expedite the process. These agreements are
not governed by the FAR and provide a commercial-like means of buying products or
allowing the government to try out prototypes for further application.
24
THIS PAGE INTENTIONALLY LEFT BLANK
25
III. METHODOLOGY
A. INTRODUCTION
This project explores contracting methods available for DIUx and other DOD
organizations charged with quickly and efficiently procuring emerging technology
solutions from innovative companies. The DOD enjoys a degree of success in expedited
procurement with DARPA and SOCOM. Similar to those organizations, the DIUx model
relies on expedited contracting methods to partner with non-traditional suppliers.
Our research involves a combination of policy review, interviews, and previous
technology procurement cases to investigate available acquisition processes across
agencies and departments. Common responses may indicate persistent issues ripe for
reform. Conflicting accounts may show how offices manage the acquisition environment
differently with different outcomes. This collective feedback and analysis sheds light on
the efficacy of differing practices and philosophies.
Using interviews helps identify underlying differences between the intended
acquisition path and actual processes. Discovering the actual process details requires
personal interaction with stakeholders to glean delay-causing factors. The interviewees
did not previously know the researchers and participated in the telephone interviews
voluntarily.
B. DATA COLLECTION
We attended a DIUx small business town hall and a series of meetings with DIUx
staff to learn their processes in accomplishing market outreach mission. We studied
recent literature in the areas of rapid procurement, non-traditional contracting tools, and
innovative technology acquisition. The operational feedback was based on interviews and
written responses to our questions from defense acquisition professionals in various rapid
acquisition offices.
Protecting the identity of the interviewees permitted sincere and forthright
feedback. Our research sample includes telephone interviews with five acquisition
26
professionals and written responses from three acquisition professionals. The respondents
include civil service employees and military officers from different branches. The
respondents are assigned to various commands, services, and defense agencies. Their
acquisition experience levels vary from mid-level professionals and managers to senior
contracting leaders. Their familiarity with non-traditional tools ranges from limited
exposure to and awareness of the tools, to consistent use of the tools, to advocating policy
and authorization for existing and future rapid procurement tools. We adapted interview
questions from Senator Carper’s letter to Office of Management and Budget (OMB)
Director Donovan. Senator Carper, ranking member of the Senate Homeland Security
and Governmental Affairs Committee, asked OMB Director Donovan about the
acquisition process and the government’s ability to work around roadblocks “to better
understand what OMB is doing to shepherd the use of innovative and emerging
cybersecurity tools at Federal agencies” (Carper, 2016, p. 2). The interview questions are
listed in Appendix C.
C. SUMMARY
This study relies on multiple sources to assess the DOD’s flexibility in procuring
emerging technology. Acquisition statutes, regulations, and policies represent the
currently available expedited procurement methods. Recent DOD history offers a number
of acquisition cases in which the Department attempted to leverage the market to capture
technology. Monitoring DIUx efforts provides a current view of reaching out to a
promising technology market. Finally, interviewees provide experience-based feedback
on the merits and shortcomings of applying various procurement tools.
27
IV. RESULTS
A. INTRODUCTION
This research is focused on the DOD’s ability to competitively acquire innovative
or disruptive technology with potential offensive or defense applications. Competitive in
this instance refers not to the full and open competition in soliciting offers. To the
contrary, it refers to the government competing against other prospective buyers for
emerging technology and steering engineering talent and private capital toward the
development of innovative defense solutions.
Such disruptive technology comes with disproportionately powerful or even
exponential potential to attack or defend against a stronger opponent or a more powerful
conventional system. In this technological race, unconventional or asymmetric
technology is also used to defend against other disruptive technology. Some of the
growing areas include cyber, robotics, artificial intelligence, machine learning, and data
analytics. We refer to these kinds of technologies in this research effort.
B. MARKET ENVIRONMENT
The market dynamics behind innovation, its funding and return on investment
influence industry behavior and interaction with the government. The extent of
government R&D funding for a given product drives the level of government’s IP rights.
Companies view IP rights as a zero-sum game of their market size opportunity and ability
to charge monopolistic prices in those markets.
The defense industrial base, while historically a source of major innovations,
pursues new technology development in conjunction with sales of existing products. For
the established defense companies, protecting incumbent market share is as important, or
situationally even more important, than attempts at developing new products. The new
technology comes with high development costs, and may ultimately replace products in
the existing portfolio. Additionally, the new technology may generate lower profit
margins than existing product lines. The goals of maximizing profit and developing new
technology compete to some extent, and may even crowd each other out. That leaves less
28
than maximum incentive for traditional defense suppliers to leverage opportunities for
new technology development.
Further, as suppliers grow into mature companies, their organizational
bureaucracy tends to hinder creativity and development. Therefore, the DOD needs
continuous access to technology development companies as they emerge. New businesses
in pursuit of emerging technology operate with fewer internal conflicts. They normally
do not yet have a portfolio of products and cash cows to promote and defend in lieu of
developing new products. Rather they aggressively pursue product creation to build a
portfolio. They operate as relatively flat, agile, and focused organizations. In turn, they
seek out similarly-organized buyers for fast and efficient sales to sustain their operations.
“Flat, agile, and focused” does not closely describe the defense acquisition enterprise.
The market environment changed for the DOD when emerging technology
became cheaper and more accessible for adversaries, while, concurrently, the U.S.
government’s share of R&D funding shrank, leaving the private sector to shoulder a
growing part of the overall R&D efforts. At least in the near future, the shrinking defense
budgets will further accelerate the shift from government-funded research to privately-
funded research, leaving the government with less leverage to acquire new technology
after its privately funded development. This leaves the DOD in a less advantageous
buying position, one that relies on privately developed technology and includes increased
competition among buyers. In terms of vulnerability, the DOD’s—and indeed the U. S.—
reliance on technology exposes us to cyber-attacks and necessitates both offensive and
defensive technological advantage.
C. WARTIME RAPID ACQUISITION AUTHORITY
The DOD employs wartime rapid procurement authorization in times of large
scale conflict. Defense acquisition shifts into a frenzy during conflict. The Pentagon
outsources logistics and life support services to private companies and focuses on core
missions. Once major operations wind down, so does the flexibility to mitigate some of
the time-consuming steps in the traditional procurement process.
29
The Army’s rapid fielding office at program executive office (PEO) Soldier
serves as an example of actual and perceived procurement challenges. In wartime, PEO
Soldier utilizes rapid acquisition authority to expedite procurement. For existing
commercial solutions, PEO Soldier taps into existing contract vehicles in the DOD and
beyond (e.g., GSA vehicles) to acquire necessary products. The department has largely
viewed procurement schedule as a trade-off against cost and quality, with schedule rising
in importance during wartime and cost consciousness prevailing in peacetime.
The Mine-Resistant Ambush Protected (MRAP) program is another recent
example of a rapid system acquisition during wartime. At the height of the Iraq war,
Secretary Gates pulled the MRAP program out of the traditional acquisition path and into
a direct report program to him to expedite development and production. With improvised
explosive devices killing the highest number of troops in war zone, every day of
acquisition, production and fielding schedule mattered in the effort to avoid at least some
of the daily casualties. That pressure created a staggering schedule premium to field the
vehicles to the ground forces. Each day of lead time was of personal interest to Secretary
Gates.
D. RESULTS
Our research of DIUx and acquisition policies yielded some authorized avenues to
deviate from traditional contracting to expedite procurement and to adapt to private
procurement methods. However, non-traditional contracting methods then triggered
questions of level of awareness and acceptability of such tools among contracting offices.
Even the authority to exercise non-traditional tools such as other transactions is limited
among government agencies. To learn answers to these questions, we spoke to
acquisition staff in the field. In addition to reviewing statutes, regulations, and policies,
we conducted a series of interviews with various defense organizations engaged in rapid
procurement.
The feedback varied based on the organization, mission, size, and type of items
procured. We observed that as an organization grew in size, its contracting flexibility
diminished. Understandably, as the procured products shifted from R&D items to
30
commercial items, contracting options narrowed accordingly. On the research and
innovation-heavy end of the scale lies DARPA, whose mission is centered on developing
solutions. Its small size and inherent agility allow for unconventional contracting
methods. In fact, DARPA not only pioneered technological development within the DOD
and broader government, DARPA led the advocacy for DOD policy and legislative action
for securing the authority to conduct other transactions.
1. Awareness of Expedited Solicitation/Contracting Methods and Training of the Users
If you are not in a typically innovative organization, chance are you have never
heard of nor worked with expedited contracting methods. This lack of knowledge was
echoed during our interviews by multiple individuals; “Not a lot of people know what
OTAs are out there, or how to use them. It would be nice to know what tools are out there
for the non-traditional contracting officer” (program manager [PM] at PEO Soldier and
KO at SOCOM, personal communication, August 3, 2016). This lack of awareness of
expedited solicitation and contracting methods is a central and common theme found
across many organizations, whether they acquire innovative technology or not. Beyond
internal government awareness, many non-traditional vendors are just as unaware of the
steps taken to ease burdensome traditional contracting methods. In one interview, a PM
noted that they solicited with a BAA to seek innovative solutions, but the vendors did not
know it was even out there awaiting responses: “Success sometimes depends on training
the vendors on how to conduct business with the government” (KO at SOCOM, personal
communication, August 3, 2016). “It comes down to education and fear that someone is
going to do something wrong; the legal guys won’t let us try it” (PM at PEO Soldier,
personal communication, August 3, 2016). So, the need for awareness internally is
matched with the need to get the word out to industry.
“Defense Acquisition University doesn’t offer courses for this sort of contracting;
we had to seek out training slides from faculty at the Naval Postgraduate School and
DARPA to create a way forward, basically just trial and error” (KO at SOCOM, personal
communication, August 3, 2016). SOCOM realizes the potential benefits of these
expedited contracting methods and is establishing a special unit to spearhead their future
31
use. This unit is also tasked with educating the KOs, PMs, and requirement owners about
the proper use of these methods (KO at SOCOM, personal communication, August 3,
2016). “[The] Air Force Research Laboratory (AFRL) gave a presentation on using
‘Other Transactions’ and a vendor demonstration forum for new technology companies to
show attendees their new technology solutions” (KO at Gunter Annex, personal
communication, August 6, 2016).
2. Authority to Use
The authority for using Other Transactions is codified in 10 U.S.C. 2371 and 10
U.S.C. 2371b for prototypes. DARPA and NASA received initial authorizations, with
other government agencies following suit. However, authorization for use is only the first
step. It is indicative of the tool’s accessibility to a given agency, not the extent of use for
that agency. Therefore, pursuit of authorization and subsequent application is indicative
of a combination of the agency’s need for non-traditional acquisition (authorization) and
its cultural environment that allows (or does not allow) the agency to successfully utilize
the authorization (application). According to GAO (2016) compilation, Congress
authorized other transactions for 11 executive agencies in various permanent, temporary
or program-specific forms (p. 6). The agencies in turn vary on the extent of using OTs
and types of OT activities. Figure 4 depicts the 11 agencies with congressional authority
as of FY 2016 with types of OT activities by agency.
32
Figure 4. Agency Use of Other Transaction Agreements for FY 2010 through FY 2014. Source: GAO (2016).
3. Flexibility of Options
In the operational community, SOCOM focuses on unconventional warfare
missions. Unconventional warfare does not occur in vacuum. Operators using special
tactics are best supported with unconventional contracting tools used to acquire
customized solutions. SOCOM is small enough to avoid the bureaucracy that bogs down
larger buying organizations, yet big enough to attract vendors. SOCOM employs other
transactions and, more recently, technology investment agreements for development and
prototyping.
“As long as you are able to justify your reasoning and able to work within the
rules and regulations you are using, it’s fine. [Furthermore,] you tell me I can write an
agreement and don’t need the FAR, let’s do it” (KO at SOCOM, personal
communication, August 3, 2016). DARPA, as an early advocate and regular user of other
33
transactions, also lauds the merits and benefits of the flexibility afforded in the
authorization.
An expert in ballistics at PEO Soldier, however, expressed skepticism with regard
to other transactions. Given the magnitude of developing, procuring, and fielding
equipment for the whole Army, a large-scale acquisition requires production of millions
of items. Our research focused on the development of technology or capability and the
acquisition of prototypes, limited items, or intangibles such as software. The interview
audience discussed broader challenges across the acquisition cycle. Representatives at
PEO Soldier consider the entire acquisition process when planning and streamlining
efforts. The large scale of their procurements arose as a point of distinction from our
other interviews. Other transactions apply to prototyping and development. They allow a
degree of flexibility to make acquisitions in a wide range of research, development and
technology maturation efforts. They do not afford mass procurement of developed
products.
Like most traditional procurement offices, PEO Soldier lacks the authority to
exercise other transactions. Could the authority to issue other transactions be beneficial to
organizations such as PEO Soldier? For an affirmative response to hold true, the buying
agency must be hindered in acquisition of R&D or prototyping.
Notably, the feedback from PEO Soldier indicated that for development and
prototyping, BAAs and traditional contracting methods meets the needs of the
organization. The challenge lies in shifting from technology development to force-wide
mass procurement. That is where, despite the availability of technology and prototypes,
the Army’s acquisition of service-wide equipment slows to a crawl. Procuring
standardized gear for the force requires layers of approvals that can set the process back
by many months, or even years. Congressional fiscal appropriations and political
attempts to steer large-scale procurements also play a greater role as the procurement size
grows. While the feedback on the benefits, use and applicability of other transactions
varied among the respondents, the comparative ease of using them in smaller
organizations became apparent during our interviews.
34
In PEO Soldier’s case, the ballistic protection program office met its objectives
with developing an improved helmet without using other transactions. Their challenge lay
in launching large scale production. Other transactions were of no benefit in later stages
of the acquisition. This example shows the dynamics of a moving bottleneck in the
process. While the acquisition needs to clear the initial hurdle of developing the
necessary technology, large-scale procurements are likely to run into other hurdles as
they progress toward becoming programs of record.
Even in R&D efforts, PEO Soldier relied on the more traditional procurement
methods with BAA solicitations. The ballistic program office found available traditional
tools adequate for ballistic protection research, development, testing, and prototyping.
They did not seek other transaction authority or expect enhanced access to product
developers through non-traditional contracting tools. Given an opportunity, they would
have pursued an OT-type tool for large-scale procurement, if there were such a vehicle.
The market for ballistic defense procurements is small, comprised of relatively few
companies and few, if any, prospective new entrants. PEO Soldier can adequately reach
the ballistics R&D market with traditional tools.
But the program office acknowledged that other transactions can help the DOD
better reach other markets in emerging technology, cybersecurity, prototyping, or small
scale acquisitions that will not translate to a program of record. DARPA, for instance, as
an authority champion and avid user of other transactions, is heavily reliant on this
authority for its market access. Contracting leadership estimated that DARPA awarded
hundreds of other transactions since obtaining statutory authority.
4. Support within the Government
On the opposite end of the scale from DARPA and SOCOM lies large-scale
fielding and equipping of standardized items for the U.S. Army. Procuring millions of
rifles, pistols, or helmets requires the use of a traditional contracting process, with its
series of reviews and approvals. Needless to say, the traditional contracting process is
longer and more bureaucratic than non-traditional methods.
35
“Effectiveness of expedited acquisitions fall on the support from [Contracting
Officer’s Representatives] and the PMs to make them happen effectively” (KO at
SOCOM, personal communication, August 3, 2016). “Sometimes the commercial items
do not meet all the user’s needs; we tell them that 70 percent is all we can get, and they
say that 70 percent is better than zero percent” (PM at PEO Soldier, personal
communication, August 3, 2016). Support for non-traditional tools within an agency
depends on its mission, culture, and market access needs. When presented with a partial
solution, the customer may simply elect to rely on readily available technology and
product offerings to meet the immediate requirements. Proposals of existing products
with partial solutions afford a faster acquisition, fielding and equipping process than
prototypes or developmental products with a prospect of comprehensive solution but
uncertainty in the areas of cost, schedule or performance.
In the trade-off between accepting a quick partial solution or seeking a
comprehensive solution, procuring offices are commonly inclined to compromise on a
partial solution given immediate availability and greater price certainty. Acquisition
community philosophy leans toward optimizing use and application based on existing
solutions rather than optimizing solutions themselves. It is in this uncertain area of
improving solutions that OTs offer greater reach at the market.
For an agency to seek other transaction authority, its mission must fundamentally
hinge on advancing research or technology acquisition beyond existing commercial or
defense solutions. Much of our energy, even when constrained by procurement tools, is
focused on how to best or most closely meet acquisition objectives with existing tools,
rather than pushing procurement tools boundaries. The perception of risk mitigation
provided by conservative procurement methods serves as a deterrent from deviating
outside these predictably safe and established methods.
5. Intellectual Property Rights: Collection vs. Protection
While the private sector is taking the lead in innovation, the government trails
behind in adapting an intellectual property (IP) strategy for this new privately-funded
order of conducting research efforts. IP and data rights terms and conditions vary based
36
on funding sources and parties’ rights in using technology beyond a given acquisition.
Industry conducts comparisons of opportunity costs, profit margins, risk levels and
expected outcomes as standard business procedures. For those companies, the trade-off is
between government revenues and other potential commercial revenues (profits)
foregone at the expense of giving up property rights. The DOD carries the burden of
avoiding even the perception of unprofitable IP dilution to establish its attractiveness as a
buyer. Historic reliance on a traditional industrial base that is well-versed in defense
acquisitions, internal development, and government-funded research made the DOD
accustomed to retaining a high level of data rights. Such expectations are incompatible
with venture capital-funded research and technology development efforts. And while
domestic firms recognize the need for the DOD to maintain technological advantage, they
must also maximize their market opportunities and potential for returns on investment.
“We don’t go after the data rights; we buy the product as is” (PM at PEO Soldier,
personal communication, August 3, 2016).
According to DARPA contracts leadership, the agency historically dealt with
many Silicon Valley companies of all sizes with a notable exception of the two largest
companies—Google and Apple. While other technology companies do business with
DARPA, concerns over IP permeate the entire process and interaction. As a matter of
competition and growth strategy, large companies, in both defense and technology
industries, monitor the market for new technology development at smaller firms. That
technology, or the entire firm, then become a prime acquisition target to leverage new
technology as stand-alone item or incorporate it into other systems with commercial or
defense applications.
In discussing the rights to IP, we have identified a vast gap in perceived, or
perhaps real, interpretation of the IP rights spectrum. DARPA contracting leadership
described vendors’ perception as driven by paranoia and distrust of the government.
DIUx contracting leadership offers a cooperative approach to negotiating data rights,
leaving all options on the table as a matter of attracting the widest pool of vendors and
alleviating their concerns.
37
Justifiably, proprietary technology is the lifeblood of technology companies. So
they want a lot of flexibility with retaining control over trade secrets and patent rights.
DARPA considers its future needs in determining the extent of government IP rights to
pursue. The dilemma the government and vendors run into is that, while vendors
welcome government funding and the government is willing to fund promising
development, vendors claim the need for both government and commercial markets to
recoup their investment. One avenue DARPA considers is giving the job to a company,
but delaying government claim to data rights. Part of market research and negotiation is
having a good business sense to figure out what return on investment vendors need.
DARPA conducted hundreds of other transactions over the years, including some
with companies in Silicon Valley. And it continues to seek smaller companies for future
transactions. The hardest part is reaching them. The ballistic protection office at PEO
Soldier offered an economic perspective on the attractiveness of the government market,
including data rights: “[The government needs] to make sure the companies make money
for them to be interested.” And if the government wants a company to develop a
government-specific product, they may decline if the market is too small.
It’s really challenging. So for body armor protection and helmets, it’s a very small market compared to smartphones, automobiles, or aerospace. It’s a very small market for [force] protection. That’s why a lot of people and businesses are not interested. We don’t have large companies working in this area, only small companies. It’s not something you can help. Large companies look at that market and say: “No, we are not interested. Every month, the maximum is only a couple of million dollars in sales. We don’t need that.” (PM at PEO Soldier, personal communication, August 11, 2016)
When the government buys small lots, it does not appeal to the wider market,
“But if you are talking about new small companies, then they should be interested. I don’t
think the government is pushing any new company away.” The reservations are on the
company’s side—they are “not interested.” The ballistic protection office even expressed
skepticism of market capability:
We have all these companies coming in. Most of these companies think they have something better—they don’t. It’s the reverse—it’s not that they have something better that we don’t want to use. They don’t always
38
understand what we really want. Everybody thinks they have something better than the military. They come here to this office, we tell them the performance requirement, and ask: Can you make it? The answer is typically: “No.” (PM at PEO Soldier, personal communication, August 11, 2016)
For the relatively few companies that develop, or are capable of producing
innovative technology, they may have the option to offer it to the wider market, including
foreign governments and commercial markets. In such a competitive environment, the
DOD needs to act as a competitive buyer. But this is a difficult feat. The ballistic
protection office noted
It’s similar to trying to hire someone smart. You have to pay them more than other organizations to hire them and to retain them. If you tell a company with new technology that everyone wants: “Make this only for the military,” there is no way—unless you’re willing to pay much more. You look at their prospective target market coverage and size. Can you pay them [enough in profit to substitute other market opportunities]? You can’t. It’s taxpayers’ money. So it’s money driven. (PM at PEO Soldier, personal communication, August 11, 2016)
E. DEFENSE INNOVATION UNIT EXPERIMENTAL
As overseas conflicts, or the scale of U.S. participation in overseas conflicts,
declines, the conflict-driven or life-saving urgency for rapid acquisition also diminishes.
Extraordinary or temporary rapid acquisition authorities expire due to lack of
justification, absence of request, and/or lack of congressional support to bend traditional
processes during peacetime. Dr. Carter returned to the Pentagon as Secretary of Defense
in 2015 when the DOD was still at war on multiple fronts, but on a smaller scale than in
previous years. Secretary Carter realized that technology advancement and application
was an item of immediate importance. He displayed the commitment to acquiring the
latest technology for defense purposes with regular visits to Silicon Valley. In 2015,
Secretary Carter announced the formation of Defense Innovation Unit Experimental
(DIUx), with a mission to serve as a bridge between the U.S. military and companies
operating at the cutting edge of technology (“Work with Us,” n.d.).
Military services and other government agencies previously reached out to the
engineering community in Silicon Valley for innovative solutions. Secretary Carter made
39
a more deliberate and organized call to the technology development community by
locating DIUx in the heart of Silicon Valley, just outside Moffett Federal Airfield in
Mountain View, CA. DIUx started with a promising outreach mission to the technology
community in Silicon Valley and expected strong interest and collaboration from the
military community.
The success of the DIUx venture rests on, among other things, contracting
flexibility and full utilization of expedited procurement methods. Any shortcomings in
this combination can doom defense procurement of emerging technology and engineering
talent. Unlike generic, commercial, off-the-shelf items, emerging technology has no
substitutes. And unlike non-commercial defense-specific items, engineering talent behind
emerging technology enjoys high demand from private industry buyers—the DOD’s
competition in technology acquisition. In this niche sector of emerging technology, the
DOD simply cannot employ its traditional and burdensome procurement process. In the
area of emerging technology, the DOD risks losing opportunities to procure emerging
technology altogether, thus jeopardizing its ability to maintain the technological
advantage required to support the national defense strategy.
Importantly, DIUx opened its office without organic contracting capability.
Instead, DIUx planned for contract execution through the respective customer
organizations and requiring activities within DOD. Initially, DIUx relied on the Army’s
System of Systems Security (SOSSEC) Consortium established by the Army Contracting
Command, Picatinny Arsenal, NJ (ACC-NJ). The SOSSEC vehicle allows for OT
awards.
Over time, DIUx concluded that organic, non-traditional procurement capability
is necessary for its operations. Given the slow government hiring process and time-
consuming training required for government procurement proficiency, a government
internal transfer seemed the most reliable staffing solution. But even an internal transfer
can take months, or even a year, to accomplish, assuming availability of qualified people
willing to move to DIUx. Non-traditional defense contract writing capability, most
notably OTA expertise, resides with ACC-NJ. To quickly enable DIUx to make
purchases, they needed an immediate on-boarding of skilled OTA contracting officers. In
40
the summer of 2016, DIUx sponsored ACC-NJ contracting officers for a temporary
assignment at its Silicon Valley office in conjunction with CSO call for industry
submissions. DIUx intends to hire internal OTA contracting staff in the future.
While Silicon Valley is the leading technology innovation area in the country and
the world, it is not the only technology hub. To cast a wider net, DIUx plans to expand to
other technology hubs in Boston, Seattle, and Austin. Secretary Carter elevated DIUx
profile and expectations by reorganizing it into a direct report organization—just as
Secretary Gates did with the MRAP program years ago. As a new organization staffed
with authorizations borrowed from military department headquarters, DIUx experienced
a shortage of corporate expertise in leveraging existing non-traditional procurement
methods. As DIUx grows in staff and experience, it may encounter other constraints
similar to the more established rapid acquisition offices (i.e., DARPA and SOCOM).
The DIUx model relies on expedited contracting methods to partner with non-
traditional suppliers. Flexibility and expert-level use of expedited procurement methods
are key to the unit’s success. DIUx has been evolving since its inception, improving its
strategy, outreach, and effectiveness, to enhance the DOD’s buying position in the
competitive emerging technology market.
F. SUMMARY
This research collated feedback from various defense agencies with policy
analysis of existing traditional and non-traditional procurement tools. The feedback from
the acquisition community revealed that we need to view the market strategically, from
the lenses of profitability, market size, and market limitations related to IP rights. We
started this research with a basic premise of using the full arsenal of procurement tools to
the DOD’s advantage in the competitive market for new technology. Our research has
shown that non-traditional tools such as other transactions afford wide flexibility,
however no tool can meet all scenarios. Indeed, identifying the most appropriate tool
requires an analysis of the stage and scale of the requirement. Available procurement
authorities and flexibilities vested within the buying agency may further restrict
applicable options.
41
V. DISCUSSION OF RESULTS, RECOMMENDATIONS, AND AREAS FOR FURTHER RESEARCH
A. INTRODUCTION
The government’s current buying power still appears strong in the market;
however, capitalizing on it requires educating both industry and defense acquisition
community on the expedited contracting methods. We recommend increasing awareness
of and training on these expedited contracting the methods, seeking out tactical authority
at the DOD level, allowing more flexibility with the government, educating acquisition
partners to gain support, and finally, pursuing the minimal necessary intellectual property
rights for emerging technologies.
B. STRATEGIC ANALYSIS
The underlying conclusion of our research is that the government needs to think
strategically and empathetically in acquisition of emerging technology. The DOD is
inclined to expect that the marketplace has what it needs, or is at least willing to develop
it. Taking a step back requires considering how businesses and market segments view the
government as a buying organization. The government’s purchasing role and magnitude
varies by vendor and market segment. Some vendors rely predominantly or almost
entirely on government sales. For those vendors, absent new markets, their continued
existence critically depends on continued government orders. For other vendors,
government sales may represent only a negligible share of revenue. Low portion of
government sales offers those vendors a wider discretion in future pursuits of government
sales. The government tends to impose regulations, clauses, and its own methodology on
the market with limited regard for administrative burdens to vendors and the impact on
government’s competitiveness as a buyer. Some of the government’s methods deter firms
from actively pursuing government contracts, or may even cause them to avoid
government contracts altogether.
These market realities raise the question of how to entice technology companies
to pursue government sales and develop products for government and dual use. The long-
42
term government acquisition strategy for emerging technology should aim to shift the
market dynamics toward an environment where innovative firms view the government as
a favorably buyer and create products with the government as the intended customer.
Ideally, even business creation and venture funding should include prospective
government sales as part of technology companies’ business plans from inception.
When choosing markets to pursue and products to develop, companies consider
market size, potential, profit margins, development costs, and competition. Similarly, the
government needs to consider its size as a buyer for a given product and how it affects
the supplier’s sales portfolio. Strategically, the DOD, and particularly DIUx, is trying to
become a competitive and cost-effective buyer in the marketplace, especially in crowded
and critical market segments like emerging technology. Competing buyers—private firms
and foreign buyers—also carry advantages and disadvantages.
To better understand the participants and dynamics of this market, we gathered
characteristic elements of the government (Figure 5), other competing buyers (Figure 6)
and technology firms offering emerging technology (Figure 7). Strengths, Weaknesses,
Opportunities, and Threats (SWOT) analysis allows for scrutiny of a market participant
or homogenous groups of participants. The series of SWOT analyses then allows for
strategic comparison of market participants. In the government category, our focus lies
with the DOD. For competing buyers, we considered the strongest prospects. Due to
limited sources and offerings of emerging technology, only the most competitive and
attractive buyers will acquire the technology. For developers of emerging technology, we
narrowed the analysis to the relatively few promising and successful firms in a wide
R&D market where success is elusive. Those limited sources of emerging technology,
similar to the strongest buyers, enjoy bargaining power and leverage well above industry
average. The comparison of the DOD’s analysis against other buyers reveals some
commonality and deep differences in philosophy. After accounting for technology firms’
motivations and pursuits, the DOD needs a strategy to leverage its strengths and
opportunities to meet companies’ objectives better than competing buyers. Such strategy
demands much more than inward-looking plans for incremental streamlining of the
defense acquisition process.
43
Figure 5. SWOT Analysis: Government Acquisition in the Emerging Technology Market
Figure 6. SWOT Analysis: Strongest Competitive Buyers in the Emerging Technology Market
44
Figure 7. SWOT Analysis: Developers of Emerging Technology
This basic analysis shows that a buyer’s success depends on leveraging relative
strengths and mitigating relative weaknesses to prevail in the market. The SWOT
elements involve organizational culture, financial standing, risk tolerance and mission
objectives. Procurement method is only one of the acquisition policy tools in this
endeavor, although it is most direct and immediate in interacting with the industry. It acts
as a vehicle of exchange with vendors conveying everything from financial consideration
to organizational priorities.
C. DISCUSSION OF RESULTS AND RECOMMENDATIONS
Government agencies like DARPA and SOCOM have been successfully
executing expedited contracting methods for research and prototypes for decades. As
discussed in Chapter IV, there are five improvement areas that will enable government
agencies to meet their emerging technology needs. We conclude that the following
initiatives will aide in effective application of expedited contracting methods to further
attract non-traditional vendors in collaborating with the government.
45
1. Awareness of Expedited Solicitation/Contracting Methods and Training of the Users
If you were to ask junior contracting officers/administrators how they can quickly
acquire R&D, prototypes, or other technologies without using the FAR, you would likely
get a blank stare. Prior to conducting our own research on the topic, neither one of us
ever worked on, or had knowledge of expedited contracting procedures. To that point, if
you have never worked for DARPA, SOCOM, or another cutting edge technology user,
chances are that you have never heard of expedited contracting procedures, either. Our
research suggests the government could greatly benefit from awareness training, or
formal training, on these much quicker methods of contracting. We recommend that the
DAU create a web-based training for the multiple types of expedited contracting methods
available to the user. We also recommend that such a course be a requirement for
achieving Acquisition Professional Development Program (APDP) Level III certification.
The partnership that DIUx and the broader government seeks with the emerging
technology engineering community requires informed parties, well-versed in expedited
methods.
2. Authority to Use
Currently, only certain agencies have the authority (granted by Congress) to
execute OTAs (see Chapter IV, Table 1). This authority provides the latitude to work
outside of the FAR’s more traditional contracting methods. We recommend delegating
congressional authority for OTAs to the Assistant Secretary of Defense (Research &
Engineering) to determine which DOD agencies may execute OTAs. In the absence of
relevant authority to use OTAs, or other non-traditional methods, procurement offices
operate within the constraints of traditional methods. Those traditional methods may then
entice a limited pool of vendors with existing but partial solutions and exclude the more
innovative vendors with prototypes or developmental solutions. In this predicament,
procurement offices default to navigating the limited avenues of traditional contracting
rather than widening the available path to success. The advocacy for using non-traditional
methods should come from executive levels. Training, awareness, and cultural
acceptance of non-traditional tools hinge on existing authority to execute them.
46
3. Flexibility of Options
In the Air Force, it has long been said that flexibility is the key to airpower. While
we are not focused on actual airpower in this research, flexibility is key to successful
contracting for emerging technology. While many contracting officers will argue that
they can only work within the confines of the FAR, the FAR itself conveys that
the role of each member of the Acquisition Team is to exercise personal initiative and sound business judgment in providing the best value product or service to meet the customer’s needs [and] Government members of the Acquisition Team may assume if a specific strategy, practice, policy or procedure is in the best interests of the Government and is not addressed in the FAR, nor prohibited by law (statute or case law), Executive order or other regulation, that the strategy, practice, policy or procedure is a permissible exercise of authority. (FAR 1.102(d), 2016)
The FAR provides the flexibility for the contracting community to step out of
comfort zones, where appropriate. We recommend that no policy or other restrictive
directives be produced to limit this flexibility, particularly in acquiring emerging
technologies.
4. Support within the Government
Support for the expedited procurement processes and methods, such as other
transactions, is needed by all facets of the execution chain, from Congress to Judge
Advocate General (JAG), all the way down to the contracting officers, CORs and PMs.
Lacking direct guidance and public law direction, knowledge of how we apply the
expedited contracting solutions is key to gaining organizational support. We recommend
training for acquisition professionals and partners, including JAG, financial managers,
PMs, and system engineers so they can better understand the process of the expedited
contracting methods.
5. Intellectual Property Rights: Collection vs. Protection
In an effort to attract and gain the support of non-traditional vendors, it is
extremely important that the government only negotiate for the IP rights it absolutely
needs to execute its missions. As the IP strategy guidance directs, “Don’t make an
47
unnecessary ‘grab’ for deliverables or additional license rights for proprietary IP” (DOD
Open Systems Architecture, 2014, p. 3). We recommend agencies reduce their need for
proprietary IP rights and seek only what is needed to satisfy mission requirements.
6. R&D Funding Sources and IP Implications
While the overall R&D spending in the U.S. continues to grow (see Figure 8),
federal R&D spending has been decreasing after it peaked in 2010 (“Historical Trends,”
2016). Since then, federal R&D spending, as both a constant-dollar amount and share of
overall funding, has been on a steady decline. Industry and other funding sources have
more than offset the decrease in federal spending, leading to the overall R&D spending
increase.
Figure 8. Cumulative R&D Expenditures by Funder. Source: “Historical Trends” (2016).
This funding reliance shift from federal to other R&D funding sources goes back
several decades. As part of a long-term trend, other R&D funding sources outpaced
federal R&D spending back in the 1970s (see Figure 9). Strong growth in industry and
48
other R&D spending compared to relatively flat federal funding now leaves federal R&D
funding share at less than a quarter of the overall R&D funding—a far cry from the 1960s
when federal R&D spending share peaked at about two thirds of the overall R&D funding
(“Historical Trends,” 2016).
Figure 9. Allocation Ratio of R&D Expenditures by Funder. Source: “Historical Trends” (2016).
This is no indication that this trend of decreasing federal share of R&D spending
will change. Nor is it intrinsically disadvantageous to the U.S. economy and government.
However, this phenomenon does call for an adaptation in technology acquisition. Private
funders need to deliver attractive returns to investors both as part of the arrangement for
existing funding and to encourage future investment. R&D in the U.S. has become
predominantly dependent on private funding. The government has neither the inclination
nor the means to replace private R&D funding dominance. Private R&D funding
certainly offers the government an advantage of having sources other than appropriated
budget to fund domestic R&D. It affords the government more flexibility in how much to
fund R&D and where to target efforts that may be underfunded through private sources.
49
With federal sources taking a secondary role in overall R&D funding, the defense
acquisition posture needs to acknowledge commercial lead and dominance. Post-
development, the contractual relationship with regard to IP between vendors and the
government rests with terms, conditions, and IP clauses. The government philosophy on
IP policy and clauses lags behind, and is essentially from the era when the government
dominated R&D funding and was entitled to respectively higher levels of IP rights. And
while the government continues to heavily fund defense-specific R&D, this sector is
shrinking to a niche segment compared to the growing technology market. Thus, legacy
IP policy serves as a poor policy for the current state of privately-funded emerging
technology.
The U.S. government’s buying share in the defense market is largely inverse to its
share in the technology market. Since U.S. defense spending far exceeds other countries’
budgets, or even combinations of budgets, the DOD retains enough leverage to both
attract traditional defense suppliers and also to limit authorization to sell equipment,
systems and sensitive technologies to other countries. In emerging technology, the DOD
lacks comparable market buyer dominance. Notwithstanding the administrative burdens
of the traditional defense acquisition processes, other buying segments offer more
lucrative opportunities to technology vendors. In this environment, attempting to exclude
other markets with an us-or-them attitude is a losing proposition for the government. An
effective way to entice technology sector cooperation is not by excluding their other
lucrative markets, but by expanding them. Adding government purchases to vendors’
prospective and target markets favorably grows market opportunity and interest for
vendors. This, however, requires the government to adapt to the industry’s products,
processes, and legal terms. The government may still need product modification to meet
its needs, however these modifications should be limited, and only undertaken when the
additional value outweighs the additional costs.
The acquisition process—traditional for industry and non-traditional for the
DOD—is a non-price government lever to enhance itself as a favorable buyer. The legal
terms should still deliver sufficient security and performance assurance to the
government, without unduly hindering vendors’ market opportunities. The security and
50
protection policies, while well-intended, relevant, and necessary, do not alone account for
the economic reality of market sizes and profit margins. In a profit-maximizing
environment, restrictive IP policy excludes buying opportunities for the U.S. government,
rather than other buyers. Recent Better Buying Power (BBP) initiatives attempt to
reconcile with this reality by encouraging flexibility in IP terms. However, the
government needs to distinguish among its varying strategic positions in different
markets and adapt its IP pursuits accordingly. While the government can successfully
impose a restrictive IP policy with tank purchases where it acts as the sole buyer, such
policy will only alienate it from the emerging technology market.
The best bet for the government is to incentivize companies to enter defense
products market and pursue government solutions by growing their markets with dual
commercial and military applications. The dual market should be not an afterthought
once technology has already been developed; it should be a mindset and methodology
from the formation of businesses and the inception of technologies.
BBP does not alter the regulatory guidance, but influences government
acquisition philosophy, i.e., with the expanded use of incentive contracts. Similarly, BBP
has the potential to encourage the DOD to exercise the flexibility within existing IP
policy to better attract technology companies. Previous iterations of BBP encouraged
increased use of prototyping and experimentation, modular open systems architecture,
and other transactions. BBP has so far provided reference material on intellectual
property terms and directed internal review and assessment of trends. The next step is to
raise awareness and encourage the requiring and procuring organizations to seek lower
levels of IP rights when purchasing technology funded primarily, or entirely, at private
expense with prospective commercial application and competing buyers in the wider
market.
D. SUMMARY
Acquisition organizations in the DOD and the broader government typically
operate on the premise that they can unilaterally set the terms of market exchange with
the assumption of the market’s willingness to abide by such rules, the market’s ability to
51
provide the required products, and market’s willingness to actually attempt to provide the
products to the government instead of pursuing other opportunities. The government
continuously attempts acquisition reform and modernization initiatives. While those
attempts are well-intended, they often have fallen short of the stated goals and policy
objectives. Furthermore, acquisition reform initiatives typically offer a limited inward-
looking scope for government agencies. Such efforts to, for example, shorten the
procurement administrative lead time (PALT), consider government goals and
preferences. They seldom compare PALT with market standards and how fast
government procurement needs to be accelerated to become competitive with the broader
market.
With regard to competition, the government is understandably focused on
competition among its suppliers. The deeper question is how to broaden the existence of
suppliers in the products of government interest and how to raise the government’s
market participation among the businesses in those respective segments. Competition
works both ways. The government focuses on competition among suppliers with little
consideration for its competing buyers. Competing buyers vary by products, services,
location, and time; creating a dynamic environment. Besides buyers outside the
government, the government also falls into the trap of competition among various
government agencies and procurement offices for limited products and suppliers.
Contracting methods offer useful tools in navigating toward procurement and
policy objectives. But contracting methods are only one set of tools in the broader
acquisition environment. Flexible contracting tools have the potential to attract additional
suppliers and broader market segments than afforded through traditional methods alone.
Having a broad, market-wide acquisition reform strategy affords the government a better
chance not only of conducting procurement as intended, but also of having access to
more products, suppliers, and market segments.
Regulations, policies, and procedures may offer a degree of assurance in the
procurement process and public policy implementation. But they also come at a cost. The
costs come in the form of higher costs to suppliers that pass along the costs back to the
buyers (i.e., the government) in the form of higher prices. Higher cost to enter and
52
conduct business results in fewer suppliers, a less competitive segment, higher profit
margins to fewer suppliers, and higher prices to the government. New and small
businesses are especially sensitive to costs of entry and administrative compliance
overhead. For most businesses, establishing business systems comparable to those at the
largest defense contractors is not an option. While patents, data rights, capital intensity,
and other outside factors limit new entrants and competition, the government further
limits the supplier pool with administrative burdens already established at existing
suppliers but prohibitively costly and inconvenient to prospective suppliers. Meanwhile,
the benefits of administrative burdens are uncertain. Thus, the traditional approach carries
the risk of levying administrative costs without adequate accompanying benefits.
The existing non-traditional methods afford government buyers wide flexibility in
procurement. The challenge lies in matching available tools with relevant requirements.
Culturally, the acquisition community needs to embrace the available tools as
opportunities, while being selective with procurement methods and adaptive to the
market environment. No single procurement method, or IP approach, efficiently captures
all acquisition scenarios. Just as procuring offices attempt to apply a relevant contract
type to a specific procurement, we also need match the procurement with applicable
method and IP strategy.
E. AREAS FOR FURTHER RESEARCH
This research project highlighted the need for the U.S. government to acquire
emerging technology to retain its dominance over adversaries. We emphasized how
lengthy and burdensome processes associated with traditional contracting methods inhibit
the government from being a “good” or “sought after” customer in the eyes of emerging
technology vendors. Further, we discussed how available expedited methods could
improve this relationship, but only when we have the authority to use them. New
questions arose during the course of this research that we believe future studies should
address:
1. Do technology firms still perceive that government contracting is too burdensome? If so, in what ways?
53
2. Should DAU include expedited contracting procedures as part of APDP Level III certification? If so, how?
3. Should the FAR include expedited contracting procedures in FAR 16, or would their inclusion cause unnecessary regulation to an inherently flexible system of contracting?
4. How can we incentivize established defense industrial companies to become more innovative?
5. Could expedited contracting methods replace traditional methods for acquisitions beyond emerging technologies? If so, which ones?
54
THIS PAGE INTENTIONALLY LEFT BLANK
55
APPENDIX A. COMPARISON OF EXPEDITED METHODS
56
Source: White House Office of Science and Technology Policy (OSTP) (2014).
57
APPENDIX B. SAMPLE VENDOR LOAN AGREEMENT
1. It is known that acceptance for test, loan, or evaluation for potential use by the Government does not make a promise to pay, a recognition of novelty, originality, uniqueness, or a contractual relationship which would make the Government liable to pay for any use of information to which it would otherwise be entitled. The Government has no plan of using any article or disclosure in which the submitter has set up property rights, without compensation.
2. The Government will use care in the handling and testing of submissions. The
Government will not take responsibility of liability to submitters or others for: a. Damage to, destruction, or loss of submissions resulting from testing or otherwise. b. Damage or injuries due to negligence or otherwise, which are incurred or suffered by submitters, submitter’s employees, or invitees during any test of such article or disclosure which is under the control of the submitter, submitter’s agents or employees.
3. The submitter agrees to indemnify and hold harmless the Government, its agents,
and employees from any and all claims or causes of action whatsoever as may be incident to or come from the Government’s acceptance of and its part in the test and evaluation of any article covered by this agreement.
4. The manufacture, transportation, and maintenance of articles submitted to the
Government for demonstration or testing will be done at no cost to the Government.
5. The submitter of any articles for evaluation will give instructions to the
Government for disposal of such articles prior to completion of the test or evaluation. The disposal shall be at the cost of the submitter.
6. The testing or evaluation of such articles will in no way cause the Government to
acquire the articles submitted. 7. The articles submitted will be handled according to Government rules for
safeguarding such articles from unauthorized disclosure. The submitter agrees that any liability of unauthorized disclosure by the Government will not go past the actual loss of the submitter caused by acts of the Government.
8. The acceptance of articles for demonstration, testing, or evaluation is not to be
construed in any way as an acceptance or offer to accept such articles for Government use or as any promise implied that any contract to buy is to follow from the demonstration, test, or evaluation.
58
9. The terms of this Agreement shall be for the articles listed below, and also for all articles submitted hereafter until this agreement expires or is terminated in writing:
Please list MAKE, MODEL, and SERIAL NUMBER for each component… __________________________________________________________________
10. The time of this Agreement shall be from ____________ to ____________ unless
sooner terminated by either party. ____________________ may remove this equipment at any time after giving prior notice to the Contracting Officer so coordinated action can be taken for timely removal. In case of changed operation conditions, the Government may ask ___________________ to remove the equipment prior to expiration of the Agreement.
I,__________________________, certify that I have read this Agreement and will adhere to its terms and conditions. I further certify that I am ( ) sole owner of all articles and disclosure submitted for demonstration, testing, or evaluation; ( ) a membership of the partnership or association known as ________________________________ and have full authority to bind the corporation. The address of the corporation is: ____________________________________. ___________________________________ (Signature/Date of Submitter) ______________________________________ (Typed Name) ______________________________________ (Address) ______________________________________ (Signature/Date of Contracting Officer)
Source: Defense Acquisition University (DAU). (2011, August 30)
59
APPENDIX C. INTERVIEW QUESTIONS
Questions are adapted from Senator Carper’s letter to Office of Management and Budget (Carper, 2016).
1. What is the organization’s process or flow chart with schedule estimates for rapid acquisitions (if available)? Does the organization apply any method to measure or evaluate success in using the current process(es) for rapid acquisition?
2. What techniques is the organization using to acquire innovative technology solutions developed by start-ups and other companies that have not traditionally done business with the government? What percentage of purchases come from businesses making first sale to the government?
3. What is the organization doing to ensure that contracting officers are knowledgeable and comfortable with the relevant use of acquisition tools we discussed, as well as any other ways of rapidly acquiring emerging technology?
4. What other methods are available to acquire new technology in a rapid manner? Has the organization considered these methods?
60
THIS PAGE INTENTIONALLY LEFT BLANK
61
LIST OF REFERENCES
Air Force Research Laboratory (AFRL). (2015, March). Broad Agency Announcement (BAA) guide for industry. Retrieved from http://www.wpafb.af.mil/Portals/60/ documents/afrl/AFRL-broad-agency-announcement-guide.pdf?ver=2016-07-08-141724-390
Carper, T. (2016, April 7). [Letter to Office of Management and Budget]. Washington, DC: United States Senate.
Cooperative research and development agreements, 15 U.S.C. § 3710a (2011). Retrieved from https://www.law.cornell.edu/uscode/text/15/3710a
Defense Acquisition University (DAU). (2011, August 30). What are vendor loan agreements and when does the federal government do one with a vendor? Retrieved from https://dap.dau.mil/aap/pages/qdetails.aspx?cgiSubjectAreaID= 3&cgiQuestionID=111412
Defense Federal Acquisition Regulation Supplement [DFARS], 49 C.F.R. ch. 1 (2016)
Defense Innovation Unit Experimental (DIUx). (n.d.). Commercial solutions opening (CSO). Retrieved from http://docplayer.net/18547267-Commercial-solutions-opening-cso-office-of-the-secretary-of-defense-defense-innovation-unit-experimental.html
Department of Defense (DOD). (2012). Technology development and investments. In Defense manufacturing management guide for program managers (ch. 8). Retrieved from https://acc.dau.mil/CommunityBrowser.aspx?id=520818&lang =en-US
Department of Defense (DOD) Open Systems Architecture—Data Rights Team (2014). Intellectual property strategy guidance. Retrieved from http://bbp.dau.mil/ ocs/IP_Strategy_Brochure_FINAL_em.pdf
Department of Defense (DOD). (2015). About small business innovation research, small business technology transfer. Retrieved from http://www.acq.osd.mil/ osbp/SBIR/about/index.shtml
Department of Defense (DOD). (2016). Rapid Innovation Fund (RIF) Program [Slideshow]. Retrieved from http://www.defenseinnovationmarketplace.mil/ resources/RIF_Overview_Mar2016.pdf
Department of Defense (DOD) Open Systems Architecture—Data Rights Team. (2014). Intellectual property strategy guidance. Retrieved from http://bbp.dau.mil/ ocs/IP_Strategy_Brochure_FINAL_em.pdf
62
Federal Acquisition Regulation, 48 C.F.R. ch. 1 (2016).
Federal Laboratory Consortium (FLC). (2011, April). Technology transfer desk reference: A comprehensive guide to technology transfer. Retrieved from http://www.faa.gov/about/office_org/headquarters_offices/ang/offices/tc/initiatives/ttp/view/t2_desk_reference.pdf
Gansler, J. (2002, August). “Other Transactions” (OT) guide for prototype projects [Memorandum]. Washington, DC: Department of Defense.
Government Accountability Office (GAO). (2015, March). DOD Rapid Innovation Program: Some technologies have transitioned to military users, but steps can be taken to improve program metrics and outcomes (GAO-15-421). Retrieved from http://www.gao.gov/products/GAO-15-421
Government Accountability Office (GAO). (2016, January). Use of “other transaction” agreements limited and mostly research and development activities (GAO-16-209). Retrieved from http://gao.gov/assets/680/674534.pdf
Halchin, L. E. (2011). Other Transaction (OT) Authority (No. RL34760). Washington, DC: Congressional Research Service.
Historical trends in federal R&D. (2016). Retrieved from https://www.aaas.org/page/ historical-trends-federal-rd#National
Kendall, F. (2015, April 9). Implementation directive for Better Buying Power 3.0—Achieving capabilities through technical excellence and innovation [Memorandum]. Retrieved from http://www.acq.osd.mil/fo/docs/ betterBuyingPower3.0(9Apr15).pdf
National Defense Authorization Act (NDAA) for Fiscal Year 2001, Pub. L. No. 106-398, § 803, 114 Stat. 1654 (2000).
National Defense Authorization Act for Fiscal Year 2016 Report. S. Rept. No. 114-49, § 192 (2015).
Small Business Administration (SBA). (2014a, February 24). Small Business Innovation Research (SBIR) Program: Policy directive. Retrieved from https://www.sbir.gov/ sites/default/files/sbir_pd_with_1-8-14_amendments_2-24-14.pdf
Small Business Administration (SBA). (2014b, February 24). Small Business Technology Transfer (STTR) Program: Policy directive. Retrieved from https://www.sbir.gov/ sites/default/files/sttr_pd_with_1-8-14_amendments_2-24-14.pdf
Stevenson-Wydler Technology Innovation Act of 1980, 15 U.S.C. § 3719(b)) (1980).
63
Technology Investment Agreement, 10 C.F.R. § 603.1340. (1985) Retrieved from https://www.law.cornell.edu/cfr/text/10/603.1340
Ulrey, S. (2016, July). Other transactions 101 [Slideshow]. Personal communication, August 3, 2016.
Use of partnership intermediaries, 15 U.S.C. § 3715 (2015). Retrieved from https://www.law.cornell.edu/uscode/text/15/3715
White House Office of Science and Technology Policy (OSTP). (2014). Innovative contracting case studies. Retrieved from: https://www.whitehouse.gov/sites/ default/files/microsites/ostp/innovative_contracting_case_studies_2014_-_august.pdf
Willsey, R. (2016, February 18). A translation of the contracting alphabet: From BAAs to OTAs [Slideshow]. Retrieved from http://www.ncmbc.us/docs/TechTransition/ Contracting.pdf
Work with us. (n.d.). Retrieved September 15, 2016, from https://www.diux.mil/ workwithus/
64
THIS PAGE INTENTIONALLY LEFT BLANK
65
INITIAL DISTRIBUTION LIST
1. Defense Technical Information Center Ft. Belvoir, Virginia 2. Dudley Knox Library Naval Postgraduate School Monterey, California