i

MITRE’s mission-driven teams are dedicated to solving problems for a safer world. Through our federally funded R&D centers and public-private partnerships, we work across government to tackle challenges to the safety, stability and well-being of our nation.

ii

Table of Contents Background ................................................................................................................................................... 1

Scope ............................................................................................................................................................. 1

Problem Statement ...................................................................................................................................... 1

The Acquisition Environment and How We Got Here ................................................................................. 2

Recommendations ....................................................................................................................................... 3

1. Identify a Champion to Enable a Flexible Agile Approach: ............................................................. 5

2. Establish a Flexible, Agile Development and DevSecOps Culture: ................................................. 5

3. Put “the Right” Acquisition Team in Place: ..................................................................................... 6

4. Bring End-Users into the Fold: ......................................................................................................... 7

5. Give the Acquisition Office(s) a Prominent Seat at the Table: ....................................................... 7

6. Create and Implement Defined Processes: ..................................................................................... 8

7. Research Exemplars and Gain a Foundational Understanding of Agile Practices: ........................ 9

8. Inject Training and Experience to Support a Flexible Agile Approach: ........................................ 10

9. Institute a Change Management Process: ..................................................................................... 11

10. Enforce a Robust Requirements Elicitation and Development Process: ...................................... 12

11. Create a “Definition of Done”: ....................................................................................................... 13

12. Identify the Right Agile Development Metrics: ............................................................................. 14

13. Develop an Overall Vision: ............................................................................................................. 14

14. When in Agile, Act Like an 874: ...................................................................................................... 15

15. Maximize Use of Inherently Flexible Acquisition Vehicles: ........................................................... 16

16. Consider Using Challenge-Based Acquisition Processes:............................................................... 17

17. Utilize a Statement of Objectives (SOO) to Define Requirements: ............................................... 18

18. Tailor the Pricing Structure: ............................................................................................................ 18

19. Maximize Use of Options: .............................................................................................................. 19

20. Enforce Automation, Testing, and Quality: ................................................................................... 20

21. Frame the Solution as a Modular Service: ..................................................................................... 20

22. Clearly Articulate Intent and Requirement:................................................................................... 21

23. Carefully Craft Contract Modifications: ......................................................................................... 21

Conclusion .................................................................................................................................................. 22

References .................................................................................................................................................. 23

Acronym List ............................................................................................................................................... 24

1

Background Sluggish…Rigid…Inadaptable…Unresponsive to Evolving Mission Needs… These are some of the most commonly heard complaints about the Federal acquisition system. Mimicking and capitalizing upon the benefits of Agile Software development, the Government desires to provide programmatic and execution flexibility throughout the acquisition lifecycle; however, traditional acquisition and contracting approaches are challenged to fulfill this need. Government agencies are in search of new techniques to overcome the slowness and rigidity of traditional acquisition approaches. The contrast in flexibility between traditional and Agile acquisition processes is captured well in a Government Accountability Office (GAO) report titled, Effective Practice and Federal Challenges in Applying Agile Methods, which states, “Procurement practices may not support Agile projects: Agile projects call for flexibility in adding the staff and resources needed to meet each iteration, and to adapt to changes from one iteration to the next. One official stated that working with federal procurement practices presents a challenge where they do not support the flexibility required.”1 To further emphasize this point, the recent efforts of the Section 809 Panel on Streamlining Acquisition and the current focus on Other Transaction Authority (OTA), rapid prototyping, assessments, and fielding, Challenge-Based Acquisition (ChBA), Minimal Viable Products (MVP), and DevSecOps, make it clear that agencies are eager to adapt their acquisition processes and products to allow for greater speed and flexibility to meet evolving demands. Scope The increasing desire to leverage Agile software development processes and DevSecOps to acquire cybersecurity solutions and other information technology (IT)-software requirements, calls for solutions capable of adapting and responding to rapidly changing threats and evolving requirements. This paper analyzes the perceived limitations of current acquisition practices and suggests ways to enable Agile-inspired approaches that introduce greater flexibility into acquisition and contracting products and processes. Problem Statement It is imperative that the Government embrace processes that enable flexibility throughout the acquisition lifecycle and within contracts. These allow the Government to effectively manage the inevitable changes that occur during system or software development. Simultaneously, these processes should ensure full compliance with the letter and spirit of all acquisition laws and regulations. This raises the question: What best practices can be applied to contracts and acquisitions that will maximize flexibility and responsiveness and support a successful Agile Development and DevSecOps implementation?

1 Effective Practice and Federal Challenges in Applying Agile Methods, GAO, 2012; https://www.gao.gov/assets/600/593091.pdf

2

The Acquisition Environment and How We Got Here The environment in which the Federal acquisition community operates is changing. From 2001 to 2015, one major goal of the defense acquisition system was to adopt an overall environment of “Jointness” for acquisition efforts among the Military Services within the Department of Defense (DoD). The intent was to strengthen the role of DoD acquisition discipline and leadership at the Office of the Secretary of Defense (OSD) level, to focus on reducing the multitude of stove-piped systems, and to promote a culture of shared requirements and shared use. Unfortunately, while this effort succeeded, the GAO has reported that with those benefits came the unintended consequences of significant delays, slower decision making, and stifled flexibility. These consequences were best illustrated in 2015, when GAO reported that it took an average of twenty-four (24) months for the DoD to complete a single Capability Development Document (CDD), which captures the requirements for a militarily useful increment of capability. By contrast, employing an agile software development discipline (such as Scaled Agile Framework (SAFe)) can release a military useful increment of capability in less than 30 days. In 2016, Congress began to implement new acquisition initiatives to make the Federal Acquisition Process more efficient, effective, and responsive to the accelerated pace of adversarial threats. As a result, the focus of the Federal acquisition process is evolving to place greater emphasis on rapid capability delivery. This trend was further amplified by the 2018 National Defense Strategy which highlighted the need to deliver capabilities at the “speed of relevance” to maintain the United States (U.S.) military’s strategic advantage over its adversaries. Congress began taking steps to enable a more flexible and responsive acquisition environment through implementation of multiple National Defense Authorization Acts (NDAAs), providing the acquisition workforce (AWF) with new tools, capabilities, and opportunities to achieve greater results. The Government contracting environment continues to demand a balance of acquiring solutions with speed, with quality, and at a fair and reasonable cost. Even though existing policies and regulations encourage “out of the box” thinking and innovative contracting techniques, contracting professionals tend to follow “tried and true” techniques to put requirements on contract without considering innovative approaches. By failing to consider such approaches, contracting solutions often fail to offer requisite flexibility after award. Agility is fundamentally about discipline. Implementing work differently is challenging. Moreover, institutionalizing a process that continuously seeks and integrates stakeholder feedback into development ceremonies is particularly challenging. The best implementation programs understand Agile principles, internalize the benefits that they provide, seek to realize the benefits, and change their workflow processes to implement and maximize those benefits.

Implementing Agile techniques without changing underlying

business processes (often substantially when migrating

from waterfall techniques) does little to increase flexibility,

increase velocity, or deliver better products to the end user.

3

The contracting workforce is under pressure to deliver contracting solutions that are “fast and flexible” without fully understanding what that actually means or how to do it. Existing processes constrain and challenge the exploration of innovative contracting techniques. Local guidance often creates additional limitations within an already rigid process by adding multiple layers of unnecessary reviews and revisions before finalizing any acquisition package. These constraints contribute to protracted processes and delay contract award and modification actions. Acquisition teams need to focus on delivering solutions that deliver the benefits of agile development (i.e., continuous reprioritization of requirements throughout the development lifecycle to meet stated outcome objectives, rather than deliver a specific product) and inject the inherent flexibility of agile (i.e., ability to reprioritize requirements without requiring a modification), The National Defense Authorization Act (NDAA) Section 809 Panel—Advisory Panel on Streamlining and Codifying Acquisition Regulations—noted, “…DoD must acknowledge its acquisition system suffers from processes and procedures that are obsolete, redundant, or unnecessary and work to move quickly enough to keep pace with private-sector innovation.”2 Some of these process adjustments are explicitly and strongly advocated for, such as in Section 804 of the 2016 NDAA, which encourages the use of Middle Tier of Acquisition for Rapid Prototyping and Rapid Fielding and provides authority to the DoD to rapidly prototype and/or rapidly field capabilities under a new pathway, distinct from the traditional acquisition process. Another relevant initiative can be found in Sections 873 and 874 of the 2018 NDAA, which establish a pilot program to tailor and simplify software development requirements and processes for major software-intensive warfighting systems and defense business systems using Agile or iterative development methods. Multiple program offices, such as the Defense Information Systems Agency’s (DISA) National Background Investigation Service (NBIS), are already producing excellent results. However, implementing Agile techniques without changing underlying business processes (often substantially when migrating from waterfall techniques) does little to increase flexibility, increase velocity, or deliver better products to the end user. Considering the opportunities listed above, it is encouraging that the Government and Congress both have a clear view of the problems that currently inhibit the acquisition environment from achieving greater success in complex acquisitions, and that they continue to roll out numerous initiatives to positively shape the acquisition environment. Acquisition offices must take advantage of these new authorities, and they must evolve their underlying business processes and ways of approaching acquisition challenges. Recommendations While this evolving acquisition environment provides steps in the right direction to enable acquisitions with greater inherent flexibilities, additional actions are needed. This section offers

2 809 Panel—Advisory Panel on Streamlining and Codifying Acquisition Regulations, Volume 3. January 2019; https://section809panel.org/wp-content/uploads/2019/01/Vol3_Summary_Letter-size.pdf

4

twenty-three recommendations that Government agencies can incorporate into acquisition strategies and contracts for Agile software development in order to inject flexibility:

5

1. Identify a Champion to Enable a Flexible Agile Approach: If Agile development and flexible contracting methodologies are not championed from the top of an organization down to the program levels and below, then there is very little chance the organization or programs will be successful in Agile development.

• Observations/Challenges: Agile software development is often implemented for

the wrong reasons. Some organizations base their decision to use Agile on being told that “Agile will make things faster.” This message typically comes from leadership and is passed down the chain of command. Unfortunately, leadership forces this change without proper reasoning and without changing the status quo. That status quo is often based on a culture characterized by fear of failure. Beyond this, programs are often punished for perceived failure rather than being applauded for failing fast and often and learning from mistakes. All of these can increase resistance to implementing agility, prevent flexibility, and negatively impact the success of the program.

• Recommendations: Programs must possess an executive champion as part of the team, who removes roadblocks, eliminates cultural impediments to agility, and provides “top cover.” If Agile development and flexible contracting methodologies are not championed from the top of an organization down to the program levels and below, then there is very little chance the organization or programs will be successful in Agile development. An executive champion must be willing to be held accountable during the learning process and ensure successes and failures are shared across the organization. A good example of an executive champion is Soraya Correa, the Department of Homeland Security’s (DHS) Chief Acquisition Officer. She has created several arenas where the DHS acquisition staff can fail often and fail fast. When a large failure arises, she is the first to defend her organization and her staff. Ms. Correa has created efforts such as the Acquisition Innovations in Motion Framework, the Procurement Innovation Lab, and the Education, Development, Growth, and Excellence mentoring programs to help facilitate flexible contracting methods.3

2. Establish a Flexible, Agile Development and DevSecOps Culture:

Adopting Agile development requires dedicated Government team involvement throughout the entire lifecycle.

• Observations/Challenges: Culture is the most important enabler of Agile

development and its adoption, and leadership in every organization plays a key role in shaping and propagating the right culture. Agile practices, processes, and responsibilities are different from those of traditional development and often, they run counter to deeply ingrained cultures in Federal acquisitions – Agile success depends entirely upon an organization possessing the right culture.

3 About: Soraya Correa, DHS.gov, July 2019. https://www.dhs.gov/person/soraya-correa

6

• Recommendation: Adopting Agile development requires dedicated Government

team involvement throughout the entire lifecycle to manage evolving requirements, facilitate collaboration, and obtain active and consistent engagement across all organizations and levels. Senior leaders should be vocal champions of practices to enable Agile development; they should regularly communicate their personal commitment to making it work in the organization to include acquisition processes. As part of the culture shift, DevSecOps culture must be driven into the organization. DevSecOps should not be optional, as it offers the opportunity to significantly improve productivity, quality, speed of delivery, and responsiveness to user needs. Flexible contracting practices should help Government and contractor teams feel empowered to run their program and tailor processes, documents, and reviews as needed. Functional process owners should be supportive of tailoring their policies, processes, and oversight to enable adoption of Agile. The contracting strategies associated with Agile development should harness all available authorities, thereby maximizing flexibility, and enabling both the Government and contractors the ability to pivot if and when needed. Focus on managing the DevSecOps pipeline to successful product deliveries and define processes around using automation wherever possible. Use data from pipeline to make decisions.

3. Put “the Right” Acquisition Team in Place:

A forward-leaning and flexible acquisition team will enable Agile success.

• Observations/Challenges: “Contracting can be difficult and contracting for Agile Development is even harder. It demands adaptability, openness to change, and acceptance of trial, error, and changing course. Neither the Federal Acquisition Regulation (FAR) nor its modifications specifically address Agile software development, so it also calls for contracting and acquisition professionals to exercise creativity, alliance building and critical thinking.”4

• Recommendation: Ultimately, it is essential that agencies have the right Contracting Officer and Program Management team with not only the depth and breadth of experience needed, but also the right temperament and ability to anticipate challenges, think creatively, and be able to openly and effectively engage and collaborate with industry. While leadership focuses on enabling a culture of Agile development and DevSecOps, the acquisition team should be proactively driving individual programs to the leading-edge of the organization. Selecting the right acquisition team is particularly important for organizations that are undergoing the early stages of cultural change. Early adopters of Agile

4 McCabe, K. “Buying agile without jumping through hoops”, FCW.com, June 2014. https://fcw.com/articles/2014/06/26/buying-agile-without-jumping-through-hoops.aspx

7

techniques should be hand selecting their acquisition teams for those programs – flexibility must be driven from the ground up.

This can be accomplished through interviewing for the position; instituting formal mentorship programs; providing training; sharing and exposing success stories; providing incentives for being innovative; advocating leadership backing; and creating a culture of learning, experimentation, and risk tolerance. Ultimately, a forward-leaning and flexible acquisition team is a fundamental component of Agile success.

4. Bring End-Users into the Fold:

Program teams must understand how their users operate, and those users must be full participants in the end-to-end acquisition process.

• Observations/Challenges: Release management and deployment considerations

for users can often be overlooked when programs desire to rapidly develop and deliver capability. Additionally, even when a capability can be rapidly delivered, the user’s capacity to ingest updates may not be synchronized with the speed at which the capability is being delivered.

• Recommendation: It is critical to develop a Vision and Roadmap as part of the Agile construct, and end-users must be part of the creation process. This ensures that a clear path is articulated, enhancing a common understanding, collaboration, and engagement of stakeholders and end-users, which are all pivotal to program success. Furthermore, acquisition strategies and plans must account for how the end-user will receive and be trained for deployment of new capabilities. To inject flexibility, acquiring and gaining organizations should establish release plans to include considerations for training and supportability prior to a solicitation. This way, any specific constraints (e.g., end-users operating in austere/low-communication environments) are articulated in the requirements and addressed by industry during the proposal phase.

Program teams must understand how their users operate, and those users must be full participants in the complete end-to-end acquisition process. Program managers (PMs) should seek early buy-in from these stakeholders, establish ground rules for user engagement, and ensure that user perspectives are both heard and incorporated into any activity.

5. Give the Acquisition Office(s) a Prominent Seat at the Table:

Acquisition and acquisition strategy cannot continue to be afterthoughts; they must be directly aligned with the technical approach.

8

• Observations/Challenges: Anyone who has worked in the field of acquisition for any significant length of time has likely observed a situation where a requirements/programmatic team, either intentionally or unintentionally, started working on a requirement without fully informing their contracts and legal teams on their efforts, requirements, and expectations. When situations like this occur, it often results in friction and delays later in the acquisition process due to impacts related to unexpected changes or unknown acquisition requirements.

• Recommendation: It is imperative that the acquisition components of a project (to include business processes, program management, contracting, legal, and other functions) be welcome participants to the entire acquisition lifecycle. The skill sets, ideas, recommendations, points of contact (POCs), issues, challenges, and team for those stakeholders should be elevated in importance to ensure that the technical voice does not dominate the conversation, particularly early in the acquisition process. While the technical side of Agile implementation often consumes most of the dialogue, it is critical that agencies understand the importance of being able to implement the strategy through a flexible acquisition approach. Acquisition and acquisition strategy cannot continue to be afterthoughts; they must be directly aligned with the technical approach to achieve synergy and ultimate flexibility when acquiring Agile development solutions. As stated by KPMG, “For Procurement to achieve a place at the table, more work should be done to align to key stakeholders and understand the business operations to become a true strategic partner. This means moving up the value chain to ensure that the function is involved much earlier in the decision-making processes, and clearly demonstrating how active involvement adds tangible value to both the bottom and the top lines.”5

6. Create and Implement Defined Processes:

Establishing processes and defining roles/responsibilities upfront reduces execution risk.

• Observations/Challenges: Agile practices inherently drive flexibility into the acquisition process, enabling programs to capitalize upon new requirements and previously unknown information. However, the program management organization must be able to flexibly adapt to the changes that are generated from the outside world.

• Recommendation: Program teams go fast when the members of those teams know what they are supposed to do, when they are supposed to do it, and how to process input information. Having these defined processes allows programs to rapidly adapt. Program teams should develop processes and decision

5 The Power of Procurement: A Global Survey of Procurement functions, KPMG.com, 2012. https://assets.kpmg/content/dam/kpmg/pdf/2012/07/the-power-of-procurement-a-global-survey-of-procurement-functions.pdf

9

frameworks with clear roles and responsibilities. These processes and decision frameworks should be relatively flat and rapid; moreover, they must engender trust in decision makers at the lowest possible level. Establishing these processes and defining roles/responsibilities upfront reduces execution risk. As new and challenging situations are encountered, the program team has already defined how it will consider those situations, who will make decisions, and who will be consulted. Decision frameworks, decision processes, roles, and responsibilities should be documented by the program management team. Prior to starting any acquisition contract, the program team should run mock simulations against those documented frameworks to test their effectiveness. By conducting “what if” scenarios and analyses, the program team will identify the strengths and weaknesses of the processes and frameworks; programs should update their documentation accordingly.

7. Research Exemplars and Gain a Foundational Understanding of Agile Practices:

It is critical that the Integrated Product Team (IPT) research, gain knowledge, and understand the differences between traditional contracts and those contracts for Agile.

• Observations/Challenges: Implementing agile techniques without changing

underlying business processes (often substantially when migrating from waterfall techniques) does little to increase ability to move with velocity and rarely provides a better product to the end user. It is important that the entire acquisition team on any effort, to include the contracting team, understand the special nuances of the Agile acquisition approach as a model and applies those specialized processes to each specific program. Without this emphasis, the Government has a tendency to rely on traditional acquisition models, which are inflexible and less expedient. Moreover, implementing “half-agile” approaches exposes programs to unforeseen risks, often reducing flexibility while increasing uncertainty and cost.

• Recommendation: It is critical that the Integrated Product Team (IPT) research, gain knowledge, and understand the differences between traditional contracts and those contracts for Agile software development. While the Digital Services Playbook’s Play #5 (Structure Budgets and Contracts to Support Delivery) puts forth a checklist that is tantamount to success,6 the following are a few key sources that can be leveraged to establish a foundational understanding: • Acquisition in the Digital Age • Accelerate – Speed to Mission Impact • Defense Agile Acquisition Guide • The TechFAR Handbook • Digital Services Playbook

6 Digital Services Playbook’s Play #5 Checklist; https://playbook.cio.gov/#play5

10

• RFP Patterns and Techniques for Successful Contracting • Contracting Guidance to Support Modular Development • SAFe 4.6 • GAO’s (12-681) Software Development: Effective Practices and Federal

Challenges in Applying Agile Methods • SEI’s RFP Patterns and Techniques for Successful Agile Contracting • Defense Innovation Board (DIB) Software Acquisition and Practices (SWAP)

Study Familiarization with these publications equips the workforce with all of the foundational knowledge needed to achieve acquisition flexibility and prevent them from having to “reinvent the wheel.”

8. Inject Training and Experience to Support a Flexible Agile Approach: Agile coaches, with deep experience in managing agile programs, provide the needed expertise to guide program teams.

• Observations/Challenges: It is an understatement to say that, “applying an Agile approach to software development is not easy.” The need for an experienced, co-located Agile coach in a clearly defined visible role is made more and more apparent every day. Programs frequently encounter difficult or unique situations that require training, experience, and guidance.

• Recommendation: The Defense Acquisition University (DAU) offers some short courses that provide acquisition and source selection teams with tools to help understand the different methodologies of Agile software development. One important continuous learning course, Continuous learning module CLE076 - Introduction to Agile Software Development for Defense, includes Agile approaches, and benefits and risks of Agile development. DAU also offers a series of short videos produced by DAU’s Agile subject matter experts (SMEs), Mr. Chris Collins and Mr. Robert Thomas. Additional videos on the site include “Agile Software Development” and “Scrum 101.” These videos cover basic principles of Agile software development. By investing time in these course offerings and understanding principles of Agile development, an organization will be better positioned to write requirements or define higher level objectives for a SOO with flexible and open parameters.

Moreover, understanding the intricacies of standing up a large Agile development program, particularly one with many levels and comprising a large solution, requires a “Sherpa” who has done similar work in the past. It is not sufficient to learn on the job with so much at stake. The coach can also have bigger impact with a clearly defined role on the program.

11

Agile purists are not able to deliver success or positive change. The best Agile development implementations tailor the methods to the needs of the organization. The best coaches understand the principles and know what to tailor to help the organization progress and become more efficient, while not tailoring away the goodness that is enabled by the agile methods. Additionally, program teams should reply upon a Sherpa who can guide the program and keep it on a path to success. Agile coaches, with deep experience in managing agile programs, provide the needed expertise to guide program teams; they should be integrated into the Government’s program management team and empowered. Programs must adopt and include a co-located agile “coach” in a clearly defined, visible role. This role should not be provided by the contractor who is leading the development effort.

9. Institute a Change Management Process:

Change management needs to continue throughout the lifecycle of the program, and programs should continuously look to adopt such practices at a larger scale.

• Observations/Challenges: In many cases, a program team establishes a clear and disciplined roadmap to achieving program success but are less disciplined when an unexpected change disrupts the team’s plans. This, in turn, could result in extended delays and additional unnecessary costs. Implementing Agile Development requires change in organizations. This includes change to their planning processes, change to their deliverables, change in the way testing is accomplished, and change in how deployments are managed. It is insufficient to enable changes in one area and expect each of the other areas to change themselves. This “change” should be accomplished as part of the DevSecOps Tactics, Techniques, and Procedures (TTPs).

• Recommendation: The introduction of flexibility into acquisition processes does not mean that the processes should be undisciplined. Brown and Hegarty provide a sound recommendation and caution the Government to continuously “Anticipate the unexpected and include language to govern contract extensions.”7 This not only ensures that contractors are fully prepared to adapt to evolving needs, but also encourages them to build a plan in response to the Government request for proposal (RFP) describing how the contractor will deal with inevitable changes to avoid having to re-negotiate the contract.

According to a report by the Software Engineering Institute, Change Management is important to, “…define how changes will be handled in the RFP

7 Brown, A. & C. Hegarty, Agile Contracts: The Foundation of Successful Partnering, Scrum.inc., June 2014. https://34slpa7u66f159hfp1fhl9aur1-wpengine.netdna-ssl.com/wp-content/uploads/2014/06/Agile-Contracts.pdf

12

and how the resulting agreement will be documented in the contract.”8 The Government must adapt to expected changes and develop a plan/process to “…de-scope lower priority features and re-prioritize remaining features in response to new or changed requirements.”9 Brown and Hegarty again agree that Government and industry must work together in a transparent and collaborative fashion. They state: “There should be a regular discussion and refinement of backlog, and the contract should include a mechanism to update scope, as requirements are refined, without worrying about ‘scope creep.’”10 This needed flexibility can be built into acquisitions by allowing inspection and associated adaptation, establishing rules of engagement and processes, and implementing feedback cycles to refine and/or re-prioritize requirements as needed.11 The established change management process should include a defined decision authority (e.g., the equivalent of a product owner). This product owner would represent the “business interests” of the diverse stakeholder set and provide any requisite direction. Furthermore, the change management process should include a prioritization mechanism for agile requirements; a requirements backlog meets this need. Finally, the established change management process must include repeatable processes for evaluating and reprioritizing the backlog (or equivalent) on a regular basis. Establishing this normalized rhythm of reprioritization will minimize many of the negative effects that evolving requirements generate. Change management needs to continue throughout the lifecycle of the program, and programs should continuously look to adopt such practices at a larger scale. Furthermore, it is imperative that programs are aware and build relationships with supporting organizations to help make and support change throughout the program’s lifecycle.

10. Enforce a Robust Requirements Elicitation and Development Process:

It is imperative to time-box requirement elicitation, and most importantly, to have all of the stakeholders in the room, working collaboratively on defining these requirements.

• Observations/Challenges: It is common for agencies to initiate an acquisition

effort only to discover that not all of the stakeholder equities have been addressed in the initial plan or products, resulting in additional delays and

8 RFP Patterns and Techniques for Successful Agile Contracting, Software Engineering Institute, November 2016. https://resources.sei.cmu.edu/asset_files/SpecialReport/2016_003_001_484063.pdf 9 Ibid. 10 Brown, A. & C. Hegarty, Agile Contracts: The Foundation of Successful Partnering, Scrum.inc., June 2014. https://34slpa7u66f159hfp1fhl9aur1-wpengine.netdna-ssl.com/wp-content/uploads/2014/06/Agile-Contracts.pdf 11 Ibid.

13

resource constraints. Additionally, implementing agile techniques without changing the underlying Government business processes (often substantially when migrating from waterfall techniques) does little to increase the ability to move with velocity, and it rarely provides a better product to the end user. Executing an effective and efficient requirements elicitation and development process is and should be accomplished as part of Agile Program Increment (PI) Planning.

• Recommendation: Acquisition staff should take time to ensure that requirements are fully vetted and articulated by all stakeholders early in the procurement planning process. This does not mean to “slow roll” the requirements development process. It is imperative to time-box this step, and most importantly, to have all of the stakeholders in the room, working collaboratively on defining these requirements. It is important that the objectives (via the contractor work statement) and broad vision be fully articulated to ensure the overall needed scope can be executed for a particular acquisition. This process is difficult in many ways (particularly for IT systems and Agile development, where not only technology, but also the roles of stakeholders, interfaces, and related processes constantly evolve), and it is imperative to add flexibility into Agile development contracts to account for this evolution. Reference the Defense Agile Acquisition Guide’s Key Questions to Validate Requirements Management to get started.12 Traditionally speaking, soliciting more input early in the process reduces the number of adjustments needed later during execution.

11. Create a “Definition of Done”:

The Government should possess a clear definition of completeness that will be used to determine if individual user stories are both finished, tested and acceptable.

• Observations/Challenges: Some projects have a tendency to be endless and self-

perpetuating. Often, this is due to a lack of success metrics and a lack of a clear understanding among all stakeholders of when the work should be considered complete, or at least “good enough.” Lack of such metrics could result in unnecessary cost and effort performing extra work that is not central to mission success.

• Recommendation: Agile development calls for a definition of done that articulates the Government and contractor agreement about what the contractor must achieve in a given development cycle. The Government should possess a clear definition of completeness that will be used to determine if

12 Modigliani, P. & S. Chang, “MITRE Technical Paper- Defense Agile Acquisition Guide: Tailoring DoD IT Acquisition Program Structures and Processes to Rapidly Deliver Capabilities,” March 2014. https://www.mitre.org/publications/technical-papers/defense-agile-acquisition-guide-tailoring-dod-it-acquisition-program

14

individual user stories are both finished, tested and acceptable. This can be stated within the contracting requirements; however, if the Government does not want to create this definition upfront, it should require offerors to include that definition as part of their Agile proposal or the proposed process. Ultimately, this definition, tied with a well-defined vision, will be used to accept or reject the contractor’s output at the end of each sprint and will maintain boundaries to promote greater flexibility within.

12. Identify the Right Agile Development Metrics:

Practitioners must define metrics to determine value to the end user.

• Observations/Challenges: Pursuant to Section 872 of the 2018 NDAA, the Defense Innovation Board (DIB) conducted a study on Software Acquisition Practices (SWAP). The first of three (3) fundamental themes discussed how “Speed and cycle time are the most important metrics for managing software.”13 This aligns to the 2018 National Defense Strategy to ensure that the U.S. Government executes acquisition and development faster than its adversaries. However, finding the right metrics to use for evaluating a program can be challenging and must progress beyond estimating complexity based on Source Lines of Code and in terms of programmer productivity.14

• Recommendation: The DIB Metrics for Software Development report stipulates that different software types (e.g., commercial, custom, or blends) will drive different metrics to evaluate program success. Consequently, the report recommends deployment rate, response rate, code quality, and program management as four (4) necessary categories of metrics, while providing discrete examples that can be incorporated into both acquisition strategies and plans to maintain accountability for program success. Finally, practitioners must define metrics to determine value to the end user in relation to the aforementioned four (4) general categories to align programmatic and mission success. As with the Vision, and definition of Done, strong metrics help to communicate clear reasonable boundaries, allowing for greater variability of activities and flexibility within those limits.

13. Develop an Overall Vision:

The Product Vision, coupled with the use of a SOO, provides flexibility.

• Observations/Challenges: FAR 15.203(a)(1) requires Requests for Proposals (RFPs) to describe the Government’s requirements; however, it is significantly

13 Defense Innovation Board (DIB) Software Acquisition and Practices (SWAP) Report. https://innovation.defense.gov/software/ 14 Defense Innovation Board (DIB) Metrics for Software Development. https://media.defense.gov/2019/May/02/2002127284/-1/-1/0/DEFENSEINNOVATIONBOARDMETRICSFORSOFTWAREDEVELOPMENT.PDF

15

more difficult to define tasks and develop a work statement for an Agile-like requirement due the evolving nature of solution development and employment.

• Recommendation: In 2014, Scrum.inc hosted a presentation on Agile Contracts: The Foundation of Successful Partnering, led by Alex Brown and Christopher Hegarty, which suggested that agencies specify overall vision and context in the Statement of Objectives (SOO) versus stipulating the required process.15 This approach enables the Government to articulate a framework and award flexible contracts that meet the Government’s vision. The TechFAR also recommends the use of a Product Vision, “…which establishes a high-level definition of the scope of the project, specifies expected outcomes, and produces high level budgetary estimates.” This Product Vision, coupled with the use of a SOO, provides flexibility and allows vendors to develop innovative solutions.16

14. When in Agile, Act Like an 874:

Tailor programs to reduce contract requirements for DoD software development.

• Observations/Challenges: The role of these 874 pilots is to “write the book” on how Agile can be best integrated into the current Department of Defense (DoD) acquisition system to help streamline delivery of software intensive developments. Just because a program or project is not explicitly designated as a formal 874 project doesn’t mean that agencies cannot utilize 874 Agile best practices to build a better solution.

• Recommendation: If a program is selected as one of a handful of annual NDAA Section 874 Software Development Pilot Programs using Agile best practices, then it will innately be more flexible when acquiring Agile software development services. This program allows “DoD to leverage Agile acquisition methods to reduce certain contract requirements for DoD software development ...”17 Some of these requirements include Earned Value Management (EVM) and reporting, development of an integrated master schedule, master plan, technical requirement documents, and system requirement documents. Programs that are not selected, can still leverage Agile best practices to include defining a product vision and product roadmap; using frequent and iterative end-user validation of features; using commercial best practices to include such features as automated testing; and using Agile development metrics, to include pace of work accomplishment, completeness of scope, and goals for each iteration. All of these best practices add flexibility which typically leads to more successful outcomes.

15 Brown, A. & C. Hegarty, Agile Contracts: The Foundation of Successful Partnering, Scrum.inc. June 2014; https://34slpa7u66f159hfp1fhl9aur1-wpengine.netdna-ssl.com/wp-content/uploads/2014/06/Agile-Contracts.pdf 16 TechFar: Handbook for Procuring Digital Services Using Agile Processes, August 2014; https://techfarhub.cio.gov/assets/files/TechFAR%20Handbook_2014-08-07.pdf 17 Mahon, C. “Agile practices on the rise in the latest NDAA”, Federal Times. February 2018. https://www.federaltimes.com/opinions/2018/02/06/agile-practices-on-the-rise-in-the-latest-ndaa/

16

Furthermore, it is recommended that smaller programs network, communicate with, and tap into the lessons learned of these 874-approved Agile Pilot programs. They should continue to work with the larger Test and Evaluation community in order to share, exploit, execute, and enhance lessons learned, best practices, and practical knowledge.

15. Maximize Use of Inherently Flexible Acquisition Vehicles: Using flexible acquisition vehicles enables the Government to rapidly procure solutions that meet the Government’s needs.

• Observations/Challenges: It is common for agencies to pursue contracting

approaches or select contract types that subsequently prove to be problematic during execution. For instance, an agency may award a firm fixed-priced contract for continued maintenance, upgrades and improvements on an existing system, only to learn that unexpected development is required to comply with new security laws, guidance or regulations. The wrong contract type can prevent an agency from meeting its mission requirements.

• Recommendation: While an agency may consider numerous acquisition approaches when acquiring Agile development solutions, the procuring agencies should focus on flexible contracting arrangements that can endure an evolving acquisition landscape. Specifically, agencies should maximize the use of Indefinite Delivery, Indefinite Quantity (IDIQ) contracts, Blanket Purchase Agreements (BPAs), Basic Ordering Agreements (BOAs), and Other Transaction Agreements (OTs) within their acquisition frameworks. The Office of Management and Budget’s (OMB’s) 2012 Contracting Guidance to Support Modular Development appropriately notes that IDIQ contracts are well suited for Agile development requirements “… because they provide a high level of acquisition responsiveness, provide flexibility, and accommodate the full spectrum of the system lifecycle that provide both development and operational products and services.”18

Also, according to the Contracting Guidance to Support Modular Development, IDIQs are advantageous and provide “maximum flexibility” when the full scope cannot be “clearly defined.”19 Using this contract structure allows quick changes in direction and adjustments through the award of incremental Task Orders (TOs).

18 TechFar: Handbook for Procuring Digital Services Using Agile Processes, August 2014; https://techfarhub.cio.gov/assets/files/TechFAR%20Handbook_2014-08-07.pdf 19 Contracting Guidance to Support Modular Development, June 2012. https://obamawhitehouse.archives.gov/sites/default/files/omb/procurement/guidance/modular-approaches-for-information-technology.pdf

17

Using these types of structures enable the Government to rapidly procure solutions that meet the Government’s needs. With contracts and agreements already in place, the Government can delivery capability significantly faster than under traditional means. Procuring organizations should look for opportunities to create broad agency-wide, enterprise-wide, and/or portfolio-wide contracts to meet their future acquisition needs.

16. Consider Using Challenge-Based Acquisition Processes:

ChBA allows the vendor base to innovate and enables the Government to drive risk out of the program prior to contract award.

• Observations/Challenges: From the very beginning of the procurement process,

it is critical to solicit and inject emerging technologies, processes and other innovations into the acquisition. This is often not achieved when the Government initiates the acquisition process with a work statement that pre-defines the anticipated solution. A better solution is to enable industry to understand the problem and help build the solution based on their knowledge and experience with emerging technologies.

• Recommendation: Challenge Based Acquisition (ChBA) approaches allow a program to compete solutions through vendor demonstrations. This approach inherently drives technical risk out of the program by tackling tough situations prior to awarding a contract. Using ChBA enables a program team to define the riskiest elements of the acquisition and develop demonstration scenarios for vendors to perform. As part of any source selection, the program team can integrate these scenarios and base its selection decisions upon demonstrated success. For example, a program might want to physically participate in a mock Agile development sprint prior to awarding a contract; using ChBA enables a program to interact with a vendor on this level. ChBA challenges can be leveraged to engage vendors in demonstrating their agile development processes, technical innovations, and unique approaches.

Integrating ChBA principles allows a vendor base to innovatively respond to a program’s needs. ChBA inherently relies upon vendor participation and it empowers vendors to derive unique solutions and drive innovation into technical products.

Furthermore, ChBA, as part of a phased source selection process, brings another unique advantage. Such an approach allows a Program organization to learn and adapt as it proceeds through a challenge. Challenge events may reveal previously unknown risks or requirements, and a phased ChBA approach enables the program to learn from these challenges, adjust the acquisition solicitation accordingly, drive risk out of the program, and drive new opportunities into the program. In such situations, the program office is able to capitalize on these

18

changes without the costly need for contract modifications and without the need for changing programs after initiation.

17. Utilize a Statement of Objectives (SOO) to Define Requirements:

Don’t stifle innovation or prevent the injection of emerging technologies.

• Observations/Challenges: It is entirely possible for agencies to trip an unexpected acquisition land mine by overly defining their requirements in the contract. By establishing rigid tasks and too many metrics, a team could “over-spec” the requirement and unintentionally stifle innovation or prevent the injection of emerging technologies into the solution.

• Recommendation: To ensure the vision is properly articulated and not overly

restrictive, it is imperative to promote flexibility in defining requirements. Using a SOO to define contractor work helps promote flexibility by avoiding the rigid and prescriptive requirements of traditional acquisitions that utilize a Statement of Work (SOW). The SOO sets the overall “vision” and context of the effort and provides a broad scope for flexibility where changes and enhancements to the overall industry-proposed solution set are expected. This approach permits flexibility in industry responses to the Government’s requirements, and the approach encourages industry to be innovative in their responses. Further, the SOO construct enables the Government to clearly leverage existing Agile development frameworks (such as Scaled Agile Framework (SAFe)) without being overly prescriptive.

As long as the work (or change in work) supports the SOO and/or overall vision with little or no change to the schedule and budget, it can be deemed within the scope of the contract. By defining the Government’s needs in terms of high-level objectives rather than rigid requirements, the SOO makes it much easier to incorporate evolving needs as they are identified and as long as they fall within the intent of those initial objectives. In summary, a SOO enables flexibility and the leveraging of the Agile and DevSecOps constructs, setting the contract up for success.

18. Tailor the Pricing Structure:

Strike the right balance for both performance and cost risk. • Observations/Challenges: Contracting teams typically prefer fixed-price

contracts because they place the majority of the performance and cost risk on industry, but this approach is not always consequence free. Conversely, cost contracts tend to offer more flexibility, and T&M and labor hour contracts “… allow an agency to buy a not-to-exceed amount of services to achieve a product

19

vision.”20 However, exclusive reliance on cost-reimbursable price structure can leave the Government with a lot of costs and little to show for it.

• Recommendation: Flexibility can be achieved not only through the contract

scope, but also through the pricing structure. Agencies should consider a hybrid approach to pricing structure. The work that is “known” and can be easily priced can be structured under a fixed-price approach, while the work that is “less certain” and/or that may call for greater flexibility in scope, can be acquired utilizing Cost, Time-and-Materials (T&M), or Labor Hour structures. This approach also allows for incremental moving of items from a cost-based to a fixed-price structure as the requirements are refined and contractor risk is reduced. This hybrid approach offers full flexibility in executing requirements that may fall under the Vision, and IDIQs can use this type of hybrid pricing structure.21,22 Using a hybrid pricing structure affords the Government a better match among the “requirement,” “level of uncertainty,” and “how the work is priced.”23 By establishing a menu of various line item structures, an agency gives itself the greatest amount of flexibility to strike the right balance for both performance and cost risk.

19. Maximize Use of Options:

Account for potential additional work and scope.

• Observations/Challenges: In many cases, agencies have challenges identifying tasks and requirements that may or may not occur during the life of an acquisition. In some cases, those agencies may include qualifying statements in the work statement to relay these issues, without providing sufficient information to potential bidders.

• Recommendation: Agencies should use broadly scoped, optional, priced line items to account for potential additional work (scope) if needed (see FAR 17.202). By specifically delineating optional Contract Line Item Numbers (CLINs) in support of Agile development efforts, an agency builds a formal structure to empower such flexibility.

Some agencies include “surge” Contract Line Item Numbers (CLINs) to handle unknown future requirements. These are funded or unfunded CLINs with broad scope that enable the contractor to address emerging requirements. Agencies should consider adding Surge CLINs to future acquisitions.

20 McCabe, K. “Buying agile without jumping through hoops”, FCW.com, June 2014. https://fcw.com/articles/2014/06/26/buying-agile-without-jumping-through-hoops.aspx 21 TechFar: Handbook for Procuring Digital Services Using Agile Processes. 22 Contracting Guidance to Support Modular Development, June 2012. https://obamawhitehouse.archives.gov/sites/default/files/omb/procurement/guidance/modular-approaches-for-information-technology.pdf 23 Ibid.

20

20. Enforce Automation, Testing, and Quality:

Failure to consider automation, testing, and quality increases risk.

• Observations/Challenges: Application of Agile approaches and products alone will not necessarily lead to mission success. There are three critical ingredients to make this recipe work: Automation, Testing, and strong Quality Assurance practices.

• Recommendation: To realize the benefit of DevSecOps and Agile Practices three requirements at minimum should be added to the contract. 1) Automation – Depending on the toolchain defined for a program’s pipeline, scripts or Chef Cookbooks should be a requirement written into the vendors contract. This will mitigate the risk of vendor code not being compatible with the programs DevSecOps environment. 2) In addition to Automation, Test scripts also need to be provided by the vendor to assure the code provided by the vendor is not only compatible with the DevSecOps environment, but all works in the indented environment it is being deployed to. 3) Quality acceptance should be part of the contract; a user story is not considered “done” until the previous two items are successfully demonstrated by the DevSecOps pipeline and the product owner has concurred the user story has been completed to their satisfaction.

21. Frame the Solution as a Modular Service:

Modules increase flexibility by being incremental, building upon one another, and enabling tailoring as requirements evolve.

• Observations/Challenges: With many IT requirements, it difficult to develop a

focused end state due to the evolving technical, security and legislative environment. An IT approach or solution that is adopted this year may be rendered obsolete or problematic 6 years from now, leaving users stuck with a system that is difficult to evolve.

• Recommendation: Where possible, it is more useful to frame the acquisition as a service that builds a solution rather than a product. The end-state solution is achieved under a broad statement of need rather than a prescriptive SOW and structuring the contract this way will inherently add flexibility.24 Furthermore, according to FAR 39, Agile acquisition should be modular in nature. GSA’s 18F, an office “…that collaborates with other agencies to fix technical problems, build products, and improve how Government serves the public through technology,”25 describes modular contracting as a strategy that “break[s] up

24 McCabe, K. “Buying agile without jumping through hoops”, FCW.com, June 2014. https://fcw.com/articles/2014/06/26/buying-agile-without-jumping-through-hoops.aspx 25 About: Government Services Administration (GSA), https://18f.gsa.gov/about/

21

large, complex projects into multiple, tightly-scoped procurements to implement technology systems in successive, interoperable increments.”26 By executing a modular contract, agencies can build additional scope into the contract with the next increment (i.e., with the next piece of the modular acquisition). Modular contracting provides flexibility in that the modules are incremental, build upon one another, and can be tailored as requirements and the associated capabilities evolve.

22. Clearly Articulate Intent and Requirement:

There are a variety of Agile approaches, so identify, define, and articulate the chosen approach early.

• Observations/Challenges: One of the primary functions of the acquisition community is to promote more broad and open communication and interactions with industry to set common expectations. Historically, this amounted to posting a rigid work statement with the solicitation or request for information. Agile development approaches demand more. Industry partners need to not only understand the approach that the Government intends to follow in development acquisitions, but also the anticipated range of changes that can be expected during the life of the contract.

• Recommendation: It is critical that all interested parties be notified up-front that

requirements will shift and be adjusted within the limits of the Vision Statement and the work statement. As stated in the TechFAR, this will alert offerors that the refinement of technical requirements will occur post-award using a highly disciplined process of testing and customer feedback.27 Additionally, this will set clear expectations by industry that the requirement will evolve over the life of the contract and will reduce the risk of potential protest related to perceived “scope creep.” Furthermore, it is important to articulate what “agile” means to your particular project and what specific Agile methodology you are employing. There are a variety of Agile approaches, so identify, define, and articulate the chosen approach early. Everyone must be on board.

23. Carefully Craft Contract Modifications:

Agencies must announce the requisite flexibility as a baseline expectation in their requirements.

• Observations/Challenges: While agencies have the ability execute a simple contract modification for additional work, these modifications can get tangled up in competition and legal issues if the modification is determined to be “out of scope.”

26 18f Contracting. https://18f.gsa.gov/2019/04/09/why-we-love-modular-contracting/ 27 TechFar: Handbook for Procuring Digital Services Using Agile Processes

22

• Recommendation: Establishing the broad Vision, using a SOO, and using flexible

contract arrangements provide needed breadth when determining what is in scope vs. out of scope. If the work is determined to be “out of scope,” agencies may have to initiate and execute a new competitive acquisition.28 However, there is an art to defining a modification. The key is to frame the modification in such a way as to show that the change is of a “type that reasonably could have been anticipated” by bidders over the life of the contract and that such a change would not have “materially changed the field of competition for the requirement.”29

In order to be poised for such future modifications, agencies must announce the requisite flexibility as a baseline expectation in their requirements. Including language about potential future requirements and flexible approaches to meet unknown future needs are critical. Industry must clearly be aware that the use of flexible arrangements is a baseline expectation, and that any bidder would be reasonably aware of such future modifications.

Conclusion With the recent focus of our national adversaries to challenge the U.S. in areas such as IT and other technologies, both Government and Industry must take a harder look at our competitiveness on the world stage. In this modern era, it is not enough to discover and apply new research for practical uses. Instead, we need to be able to establish an acquisition and program management framework that allows us to rapidly award contracts and execute solutions in an incremental Agile manner that allows for speedy application of this research. Understanding the need to tailor acquisition processes for Agile development and aligning the right people, processes, and tools are essential for acquiring Agile software development solutions. Programs should leverage the best practices and recommendations suggested in this paper and incorporate innovation and flexibility into their acquisition processes to be successful with Agile development. Agencies must build these strong acquisition frameworks at the beginning of the process in order to allow for maximum flexibility throughout the execution cycle. By selectively adopting specialized techniques provided herein, agencies will be better equipped to achieve the “speed of relevance.”

28 TechFar: Handbook for Procuring Digital Services Using Agile Processes 29 Marvin J. Perry & Assocs., GAO Protest. B-277684, B-277685, November 1997, 97-2 CPD ¶ 128 at 3.

23

References 1. 18f Contracting: https://18f.gsa.gov/2019/04/09/why-we-love-modular-contracting/ 2. About: Government Services Administration (GSA.gov), July 2019.

https://18f.gsa.gov/about/ 3. About: Soraya Correa, (DHS.gov), July 2019. https://www.dhs.gov/person/soraya-correa 4. Brown, A. & C. Hegarty, Agile Contracts: The Foundation of Successful Partnering,

Scrum.inc., June 2014. https://34slpa7u66f159hfp1fhl9aur1-wpengine.netdna-ssl.com/wp-content/uploads/2014/06/Agile-Contracts.pdf

5. Contracting Guidance to Support Modular Development, June 2012. https://obamawhitehouse.archives.gov/sites/default/files/omb/procurement/guidance/modular-approaches-for-information-technology.pdf

6. Defense Innovation Board (DIB) Software Acquisition and Practices (SWAP) Report. https://innovation.defense.gov/software/

7. Defense Innovation Board (DIB) Metrics for Software Development. https://media.defense.gov/2019/May/02/2002127284/-1/-1/0/DEFENSEINNOVATIONBOARDMETRICSFORSOFTWAREDEVELOPMENT.PDF

8. Digital Services Playbook’s Play #5 Checklist, July 2019. https://playbook.cio.gov/#play5 9. Effective Practice and Federal Challenges in Applying Agile Methods, GAO, July 2012.

https://www.gao.gov/assets/600/593091.pdf 10. Mahon, C. “Agile practices on the rise in the latest NDAA”, Federal Times. February 2018. 11. Marvin J. Perry & Assocs., GAO Protest, November 1997. 12. McCabe, K. “Buying agile without jumping through hoops”, FCW.com, June 2014.

https://fcw.com/articles/2014/06/26/buying-agile-without-jumping-through-hoops.aspx 13. Modigliani, P. & S. Chang, “MITRE Technical Paper- Defense Agile Acquisition Guide:

Tailoring DoD IT Acquisition Program Structures and Processes to Rapidly Deliver Capabilities”, March 2014. https://www.mitre.org/publications/technical-papers/defense-agile-acquisition-guide-tailoring-dod-it-acquisition-program

14. The Power of Procurement: A Global Survey of Procurement functions. KPMG.com, 2012. https://assets.kpmg/content/dam/kpmg/pdf/2012/07/the-power-of-procurement-a-global-survey-of-procurement-functions.pdf

15. RFP Patterns and Techniques for Successful Agile Contracting, Software Engineering Institute, November 2016. https://resources.sei.cmu.edu/asset_files/SpecialReport/2016_003_001_484063.pdf

16. Section 809 Panel, Report of Advisory Panel on Streamlining and Codifying Acquisition Regulations, https://section809panel.org/media/updates/

17. TechFar: Handbook for Procuring Digital Services Using Agile Processes, August 2014, https://techfarhub.cio.gov/assets/files/TechFAR%20Handbook_2014-08-07.pdf

18. Veterans Administration (VA) Office of Inspector General (OIG) Report, "Inadequate Oversight of Contracted Disability Exam Cancellations.” 2019

24

Acronym List AWF Acquisition Workforce BOA Basic Ordering Agreement BPA Blanket Purchase Agreement CDD Capability Development Document ChBA Challenge-Based Acquisition CLIN Contract Line Item Number DAU Defense Acquisition University DHS Department of Homeland Security DIB Defense Innovation Board DISA Defense Information Systems Agency DoD Department of Defense DoDI Department of Defense Instruction EVM Earned Value Management FAR Federal Acquisition Regulation GAO Government Accountability Office IDIQ Indefinite Delivery/Indefinite Quantity IPT Integrated Product Team IT Information technology MVP Minimal Viable Products NDAA National Defense Authorization Acts OIG Office of Inspector General OMB Office of Management and Budget OSD Office of the Secretary of Defense OT Other Transaction OTA Other Transactional Authority PI Program Increment PM Program Manager POC Point of contact RFP Request for Proposal SAFe Scaled Agile Framework SOO Statement of Objectives SOW Statement of Work SME Subject matter expert SWAP Software Acquisition and Practices T&M Time-and-Materials TO Task Order TTP Tactics, Techniques, and Procedures U.S. United States VA Veterans Administration

25

© 2020 The MITRE Corporation..All rights reserved.. Approved for Public Release.19-3551.Distribution unlimited.