SYST/OR 699 Master’s Project – Final Report

Department of Defense Research, Development, Test and Evaluation Budget Exhibit Analysis Tool Enhancements

Prepared by Craig Jayson, Josh Singh, and Phillip Sutton

i

Table of Contents Table of Figures ................................................................................................................................................................. iii

Table of Tables ................................................................................................................................................................... iii

Problem Definition ............................................................................................................................................................. 1

Sponsorship ..................................................................................................................................................................... 1

Background ..................................................................................................................................................................... 1

Problem Statement ......................................................................................................................................................... 4

Needs Statement ............................................................................................................................................................. 4

Mission Statement .......................................................................................................................................................... 4

Project Objective ............................................................................................................................................................ 4

Stakeholders .................................................................................................................................................................... 5

Scope ..................................................................................................................................................................................... 6

Requirements ....................................................................................................................................................................... 7

Use Case Diagrams ............................................................................................................................................................. 9

Use Case 1: Load R-2 PE and Project Level Data into BEAT ............................................................................... 9

Use Case 2: Load R-2 PE and Project Level Data into BEAT ............................................................................. 11

Structural Analysis ............................................................................................................................................................ 12

Functional Decomposition .............................................................................................................................................. 13

Activity Modeling (IDEF0) ............................................................................................................................................. 14

Data Modeling (Entity Relationship Diagram) ............................................................................................................ 16

Rule Modeling (Decision Tables) ................................................................................................................................... 18

Technical Approach and Expected Results .................................................................................................................. 20

BEAT Model Changes ..................................................................................................................................................... 21

BEAT Version 1.0 (Fall 2016 Capstone Project) .................................................................................................... 21

BEAT Version 2.0 ........................................................................................................................................................ 21

Database Update ............................................................................................................................................................... 22

BEAT Visualization (Program and Project Analysis Dashboard) ............................................................................ 26

Program Analysis Dashboard ..................................................................................................................................... 27

Program Selection .................................................................................................................................................... 27

Contextual Program Information ......................................................................................................................... 28

Program Visualizations and Statistics ................................................................................................................... 28

Project Analysis Dashboard ........................................................................................................................................ 29

Project Selection....................................................................................................................................................... 30

Contextual Project Information and Visualizations ........................................................................................... 30

ii

Additional Project Visualization ............................................................................................................................ 31

Reporting Functions ......................................................................................................................................................... 32

Delta Analysis ............................................................................................................................................................... 32

New or Recently Ended Program/Project Identification ..................................................................................... 33

Third Offset Strategy ........................................................................................................................................................ 34

U.S. Army – High Energy Laser (PE: Advanced Weapons Technology) ........................................................... 35

U.S. Airforce – Cyber Technology Pattern Recognition and Autonomous Learning (PE: Aerospace

Sensors) .......................................................................................................................................................................... 35

Conclusion ......................................................................................................................................................................... 36

Way Forward Recommendations .............................................................................................................................. 36

Appendix A- References ..................................................................................................................................................... I

Appendix B- Definitions .................................................................................................................................................. II

Budget Activity 1, Basic Research .......................................................................................................................... II

Budget Activity 2, Applied Research ..................................................................................................................... II

Budget Activity 3, Advanced Technology Development (ATD)...................................................................... II

Budget Activity 4, Advanced Component Development and Prototypes (ACD&P) ................................. III

Budget Activity 5, System Development and Demonstration (SDD) ........................................................... III

Budget Activity 6, RDT&E Management Support ........................................................................................... III

Budget Activity 7, Operational System Development ...................................................................................... III

Appendix C- Acronyms ................................................................................................................................................... IV

Appendix D- Use Cases in Detail ................................................................................................................................... V

Use Case 1: Load R-2 PE and Project Level Data into BEAT .............................................................................. V

Use Case 2: Perform Visualization of Project Level Data in BEAT ................................................................. VII

Appendix E- Deliverables ...............................................................................................................................................IX

Appendix F- Program Management ............................................................................................................................... X

Appendix G- Recognition ........................................................................................................................................... XIV

iii

Table of Figures Figure 1: DoD Budget (source DoD.gov) ...................................................................................................................... 1

Figure 2: Example Project Level Data ........................................................................................................................... 20

Figure 3: BEAT v1.0 Visualizations ............................................................................................................................... 21

Figure 4: BEAT v2.0 Methodology ............................................................................................................................... 22

Figure 5: MS Access Data Tables ................................................................................................................................... 23

Figure 6: XML Program Element Tree Example ........................................................................................................ 24

Figure 7: XML Project Element Tree Example ........................................................................................................... 25

Figure 8: Program Analysis Dashboard ......................................................................................................................... 26

Figure 9: Program Selection Fields ................................................................................................................................ 27

Figure 10: Program Search User Form .......................................................................................................................... 27

Figure 11: Program Information .................................................................................................................................... 28

Figure 12: Program Budget Chart .................................................................................................................................. 28

Figure 13: Program Summary Statistics ......................................................................................................................... 29

Figure 14: Program Budget Heat Map .......................................................................................................................... 29

Figure 15: Project Selection Fields ................................................................................................................................. 30

Figure 16: Project Search User Form ............................................................................................................................ 30

Figure 17: Project Analysis Dashboard ......................................................................................................................... 31

Figure 18: Project Analysis Pie Chart ............................................................................................................................ 31

Figure 19: Delta Analysis Reporting .............................................................................................................................. 32

Figure 20: “New" Program/Project Report ................................................................................................................. 33

Figure 21: "Ended" Programs/Projects ........................................................................................................................ 33

Figure 22: BEAT Roadmap ............................................................................................................................................. 36

Figure 23: Work Breakdown Structure (WBS) .............................................................................................................. X

Figure 24: Project Schedule .............................................................................................................................................XI

Figure 26: EVM Graph ................................................................................................................................................ XIII

Table of Tables Table 1: Earned Value Management (EVM) Breakdown ......................................................................................... XII

1 | P a g e

Problem Definition

Sponsorship Systems Planning and Analysis, Inc. (SPA) is a professional services contractor located in Alexandria, VA. SPA has been in business since 1972 with the organization first working with the Department of the Navy. Since then, SPA has expanded its customer base to include supporting clients such as the services that make up the Department of Defense (DoD), Homeland Security, and the Department of Energy. This project is a continuation of a capstone project selected last year that supported categorizing and analyzing

past, present, and future budget submissions. The original problem statement had been derived from

conversations with Operations Support Analysis Group (OSAG), the original client of the project. The

proposal presented by OSAG was addressed during the fall ’16 Capstone project. Subsequently, SPA took over

as the lead for sponsorship and has presented this year’s team with guidance on the below problem statement.

The remainder of the paper will cover where the team started, what the team did, and detail recommendations

for future work.

Background The DoD budget, is an annual appropriation allocation that makes up the largest portion of the U.S.

government’s discretionary spending; in FY 2015, the DoD consumed over 50% of discretionary spending.

The U.S. defense outlay is equivalent to the next seven largest military budgets combined – India, the United

Kingdom, France, Saudi Arabia, Russia, and China. In 2016, over $580 billion was allocated to fund the DoD

seen in Figure 1 (source defense.gov).

Figure 1: DoD Budget (source DoD.gov)

Since post war WWII, the budget has fluctuated with peaks (overseas contingency operations, war-time

operations) and troughs (defense cuts, interwar periods). Over the past decade plus the DoD budget has seen

a precipitous decline in the budget as operations in Afghanistan and Iraq have slowed. This decline has been

of some concern for some pundits in the defense industry with their opinion that the U.S. has lost a decade or

more of technological advancements. While the U.S. has focused on counterinsurgency and Congressional

budget squabbles, peer competitors have continued to advance their militaries. The figure below provides a

visual of the ebbs and flows of the DoD budget over time.

2 | P a g e

Figure 2: DoD Budget from 1950 – 2025 in FY16 dollars (source: gmuseorspa.wordpress.com)

The budget is nested with the National Military Strategy and the President’s budget request and reflects the

strategic outlook and operational commitments necessary to fight and win in any future conflicts. Addressing

the multiple strategic challenges and threats, whether it is North Korean aggression or China island building,

the budget considers new and innovative ways to meet mission requirements. The U.S. must remain a

modernized, technologically superior global force with the capability to project power and respond to

contingences. As former Secretary of Defense Ash Carter stated, “even as we fight today’s fights, we must also

be prepared for the fights that might come in 10, 20, or 30 years”.

Since 2014, following Secretary of Defense Hagel’s Innovation Initiative, there has been a push to invest monies

in finding capabilities that have the potential to provide a technological offset against near peer competitors.

This offset strategy has led to the Deputy Secretary of Defense, Robert Work, creating a deterrence panel that

continues to evaluate an offset strategy. One of the major efforts that the deterrence panel is mandated to

focus on is the evaluation of technologies and capabilities that are intended to address both capability gaps and

technological opportunities that would reinvigorate the U.S. dominance in power projection. Capabilities of

interest for the panel include: multi-domain fires, electronic and cyber warfare, manned-unmanned systems,

robotics, undersea warfare, hypersonic munitions, advanced engineering, among others.

The DoD budget is divided into seven distinct budget appropriation categories. One of the categories is the

Research, Development, Test and Evaluation (RDT&E) budget; which is that portion of the DoD budget that

focuses on Science and Technology (S&T) and prototype development. Since FY00, the amount appropriated

to the DoD RDT&E budget has patterned itself to fall in line with national security concerns and overseas

military activities. Currently, this appropriation is 11.8% of the total DoD budget, or $69 billion.

Figure 3: FY16 Budget By Appropriation (source DoD.gov)

3 | P a g e

It is this RDT&E appropriation category that the previous GMU Capstone Group gathered DoD Budget data

on and developed a tool, known as the Budget Exhibit Analysis Tool (BEAT), to begin analyzing multi-year

budget data.1 The previous group drilled-down to the Program Element (PE) level for each service agency.

The specifics regarding scoping of the project and last year’s work can be found at the following website:

https://gmuseorspa.wordpress.com/. The figure below summarizes the level of detail at which the previous

project group scoped the problem to.

Figure 4: Budget Activity Scoping

The PE is the primary data element in the FYDP and normally the smallest aggregation of resources used by

the Office of the Secretary of Defense (OSD) for analysis. PEs are designed and quantified to be

comprehensive and mutually exclusive. Each PE has a distinct numbers/letter identifier. The first number,

in our scope above is 06, is the DoD Program of Research and Development. The second number, 3, is

related to the budget activity (BA). The BA is a category within each appropriation and fund account that

identifies the purposes, projects, or types of activities financed by the respective appropriation or fund. The

BAs are explained in more detail in Appendix B of this document. The BAs that were in-scope for this

project are seen highlighted in Figure 3. The next number, in this case 28, is a unique serial number assigned

by OSD for each PE. These numbers are in sequential order based on the PE. Following the serial number

is the equipment/activity type. The equipment/activity type is another way in which OSD categorized

RDT&E programs. Following the equipment type is the only letter identifier in the PE. This letter identifier

represents a code for a specific service/agency. Figure 4 below provides the listing of equipment/activity

type and code for service/agency.

Figure 5: PE identification last number and letter

1 The Budget Exhibit Analysis Tool allows users to quickly identify Program Elements within RDT&E using historic data and to

visualize the individual budget trends over time using Future Years Defense Program (FYDP) projected data.

4 | P a g e

Problem Statement Prior to the efforts of the Fall 2016 Capstone team, there was no known method or available tool to digest and

depict multiple budget cycles (the current year plus four future years known as the Program Objective

Memorandum2) to facilitate the analysis and characterization of past, current, and/or future budget information.

The Fall 2016 Capstone team was successful in aggregating budget data into a database and creating an Excel

tool that preformed such analysis at the PE level. Understanding that there is valuable information available at

the more granular project level, the BEAT analysis has the capacity to provide further information and insights,

but currently that capability has not been realized.

Needs Statement There is a need to advance the BEAT system to analyze project level budget data in order to project future

trends for better decision making.

Mission Statement The BEAT will provide a mechanism for capturing and analyzing past, present, and future DoD, Research

Development, Testing & Evaluation (RDT&E) Presidential Budget (PB) submissions. The figure below

displays the concept of operations portraying the several organizations that interact with the BEAT system.

These organizations are detailed in the stakeholders’ section of this paper.

Figure 6: Operational Concept Diagram

Project Objective The objective of this project is to enhance the BEAT to incorporate project level data to facilitate a finer analysis

of budget trends. The enhancements will allow the stakeholders and users to identify and view trends happening

to select projects that support the DoD’s strategy initiatives or other documents. The BEAT will maintain its

current ability to ingest new data as it becomes available and the new project data will be mapped to the

appropriate program data. Additionally, new visualizations and user interface will be developed to provide

additional information and ease the use of tool application.

ID projects of interest within the program elements based on DoD policy on future techs that

support an offset strategy.

Forecast future budget trends of predetermined projects over next 5 years.

Detailed notes included with each project that highlight trends.

Compare a new Program Budget (PB) ingestion to the previous year’s PB to determine if new

program/project/accomplishments have started anew or have ended.

2 The Program Objective Memorandum is the 5-year Future Year Defense Program (FYDP) and presents the Services and Defense

Agencies proposal on how they will balance their allocation of available resources. The POM includes an analysis of missions, objectives, alternative methods to accomplish objectives, and allocation of resources.

5 | P a g e

Stakeholders

Figure 7: Stakeholder Interaction

The President and Congress- The DoD submits an annual budget request to the President for review. The

President then submits a budget request to Congress. This request is reviewed and subjected to a process of

revisions before it is returned to the President, at which point it is either signed into law or vetoed. The

President and Congress have a vested interest in seeing that budget meets policy.

Office of the Secretary of Defense (OSD)- Primary defense policy advisor to the President and responsible

for formulating and executing policy through the DoD. The secretariats within OSD handle and submit the

final DoD budget to the President. The BEAT can potentially support DoD decision analysis.

Services (DoD)- This includes the Services and the S&T community. They receive the tech. and support and

collaborate with multiple stakeholders. They are the end users and the primary recipients of the RDT&E

dollars.

Communities of Interest (COIs)- A network of technical groups that integrate with experience and technical

expertise within DoD. They serve a collaboration mechanism to get interested stakeholders to communicate

on projects in future tech.

Federally Funded Research and Development Centers (FFRDCs), and University Affiliated Research

Centers (UARCs)- Research partnerships in which the DoD is the sponsor of a research facility which is

under the administration of either a corporate entity (FFRDCs) or a University. They provide the backbone of

studies as it relates to RDT&E and application to policy.

Industry- Private entities whose interests and level of involvement with the DoD vary significantly. Some

provide the product or service, prototype development, or test data of a specific or system of systems. There

are also contractors who provide consulting or advising services. This is the military industrial complex that

support the development of the specific idea, component, or end item.

6 | P a g e

Scope As previously mentioned, the Fall 2016 Capstone team led an effort to obtain DoD budgeting data from the

RDT&E appropriation level to the PE level for all services and accounting lines within DoD. The team created

the BEAT to analyze the data using charts and graphs. Additionally, the team scoped the project to include

only three of the seven budget activities: BA 2, 3 and 4. Overall, this initial effort was the first of many future

endeavors to develop a Decision Management System using the BEAT and DoD budgeting data to better

predict future RDT&E investment dollars.

The scope of this project is to continue with the work of moving toward final development of a Decision

Management System for DoD budget analysis. The project team for Spring 2017 Capstone team will focus on

taking the PEs and continuing to add to the database by including all project level data for all PEs that are in

scope. For instance, the PE associated with Advanced Aerospace Systems is a PE that OSD-Policy is interested

in potentially adding future dollars towards in meeting future deterrence requirements and is within BA 2, 3,

and 4.

The team will drill one level down into the project level of Advanced Aerospace Systems and gather all project

The team will also highlight notes and accomplishment level data in the notes which is one level down from

project level data. The figure below provides an illustration of what the scope will entail.

Figure 9: Illustration of Budget Levels

There are natural scoping mechanisms built into the budget nomenclature and taxonomy used to build the

budget documents. The team will use this scoping mechanism to continue work from previous Capstone

Team.

Figure 8: Program Element Example

7 | P a g e

There are several critical assumptions that are required to scope the problem to the needs of the sponsor.

1) The Project Data is readily accessible and does not require infeasible time for download and transfer

to the database.

2) The team will have access to DTIC website and other website for download of data.

3) The team will be able to relate OSD-Policy offset techs/capabilities to PE and follow-on project level

data.

4) The accomplishment level data is available; and is included in the notes.

Requirements Mission Requirements

MR1: The BEAT system shall be able to ingest project level information and data from DoD

documents that summarize PEs.

MR2: The BEAT system shall ingest project level description information for each project.

MR3: The BEAT system shall ingest five future years of expected project level data.

MR4: The BEAT system shall be able to ingest accomplishment level data for specific projects.

MR5: The BEAT system shall visualize RDT&E, specifically project level data, over a pre-determined

period.

MR6: The BEAT system shall be able to display PB new program/project/accomplishments have

started anew or ended.

System Requirements

SR1: The BEAT system shall generate project level data (R-2) for predetermined Program Elements.

SR2: The BEAT system shall generate project level description for Program Elements.

SR3: The BEAT system shall generate accomplishment level data based on specific project level.

SR4: The BEAT system shall aggregate budget data for each Program Element.

SR5: The BEAT system shall display project level data over time.

SR6: The BEAT system shall display new PB ingestion to the previous year’s PB to determine if new

program/project/accomplishments have started anew or have ended.

Functional Requirements

FR1: The BEAT system shall have ability to create a plot that visualizes specific project level data over

time.

FR2: The BEAT system shall have the ability to create a plot that visualizes significant “outliers” of

project level data.

FR3: The BEAT system shall have the ability to create a plot that visualizes accomplishment “outliers”.

FR4: The BEAT system shall monitor a minimum of five RDT&E budget cycles.

FR5: The BEAT system shall aggregate specific project level data.

FR6: The BEAT system shall not consider discontinued PEs in project level analysis.

FR7: The BEAT system shall consider PB data and determine which

program/project/accomplishment data has started new or have completed.

8 | P a g e

Figure 10: Project Requirements Diagram

9 | P a g e

Use Case Diagrams After the first IPR with the sponsor use cases were developed to formalize the viability of the system design

and to meet the agreed upon requirements. The use cases primary purpose is to capture the functional

requirements of the BEAT system; they are a means of specifying the usage of a system. The following two

main use cases capture the interaction of the user with the BEAT. Additionally, and part of behavioral

component of structured analysis, sequence diagrams have been included to represent how the user and actors

interact with the BEAT. The reason for the early introduction of the sequence diagrams is to show how they

are influenced and representative of the use cases. Appendix D provides greater detail for each of the main use

cases.

Use Case 1 displays the BEAT which has the capability to load R2 PE and project level data. The primary actor

of the database is the user with the trigger event being the downloading of data from DTIC website.

Use Case 1: Load R-2 PE and Project Level Data into BEAT

Figure 11: Use Case 1 User Interface Diagram

Schedule: Main Use Case; must be tested before providing to sponsor

Priority: Top, must be executed

Performance Target: All project level, unclassified budget level data is available from FY’11 through FY’20

Frequency: Every release of new Presidential Budget for corresponding FY

The sequence diagram below was developed to support Use Case 1. This diagram ensures the operations were

occurring in an orderly fashion with what made sense during the downloading process and the interactions

between the BEAT and the Access database. After the downloading of data from DTIC website the user

begins to interact with the BEAT. Subsequently, the BEAT then begins communicating with the Access

Interface. The Access Database then completes the storing of PE and project level data and communicates

back to the BEAT through the interface. This downloading process is behind the scenes; the user will only

need to wait a few minutes before all PB data is loaded into BEAT and stored in Access.

10 | P a g e

Figure 12: Use Case 1 Sequence Diagram

Use Case 2 displays the BEAT visualization tool and the interaction with the user and the queries, buttons, and

filters dispersed throughout the tool. The primary actor of the visualization tool is the user with the trigger

event being the user beginning to conduct exploratory analysis. The BEAT visualization has the capability to

interact with PE and project data and provides many unique dashboard features.

11 | P a g e

Use Case 2: Load R-2 PE and Project Level Data into BEAT

Figure 13: Use Case 2 User Interface Diagram

Schedule: Main Use Case; must be tested before providing to sponsor

Priority: Top, must be executed

Performance Target: All project level, unclassified budget level data is available from FY’11 through FY’20

Frequency: Every time a query and filter for specific project level data is executed

The sequence diagram for Use Case 2 describes the dynamics of the user interacting with the many facets of

the BEAT visualization tool. This includes buttons, graphs, displays, and dashboard functions that the user

can manipulate. The sequence diagram begins with the user receiving feedback from BEAT that all R2 PE and

project level data has been saved. The user then opens the visualization tool and begins exploration of either

PE or project level data. The program and project analysis dashboard section of this paper will describe in

detail the many user interfaces that the BEAT visualization tool provides. Below is a visual of the sequence

diagram that supports Use Case 2.

12 | P a g e

Figure 14: Use Case 2 Sequence Diagram

Structural Analysis After agreeing on the mission, operational concept, and primary use cases with the sponsor the team then

focused efforts on the structural analysis (functional decomposition and the functional architecture) of the

BEAT. The structured analysis approach was chosen because this approach best defines what the BEAT must

do. The approach provides a decomposition of the top-level functions, summarizes inputs and outputs, and

provides a road map to the actual upgraded design of the BEAT.

The functional decomposition is the first diagram of the structured analysis. It consists of all high-level

functions that must be accomplished by the BEAT.

13 | P a g e

Functional Decomposition

Figure 15: Functional Decomposition of BEAT

14 | P a g e

Activity Modeling (IDEF0) The IDEF0 model was chosen to represent the activity model due to its ability to best represent the BEAT

inputs, outputs, and controls at each functional decomposition level. The BEAT IDEF0 A-0 diagram details

the activities associated with taking PB data and importing the data into a displays/dashboard that lend itself

to streamlined follow-on analysis. The IDEF0 and subordinate diagrams are at the viewpoint of the user.

Below is the A-0, A0, and A5 (over next three pages) diagrams that supported and summarized the activities

associated with the BEAT. Each diagram builds on the other and is nested with the functional decomposition.

Figure 16: A-0 diagram of BEAT (Highest Level)

15 | P a g e

Figure 17: A0 diagram of BEAT (highlights A1-A5 activities of the BEAT)

Figure 18: A5 diagram of BEAT (highlights A51 through A53 activities of the BEAT)

16 | P a g e

Data Modeling (Entity Relationship Diagram) The ERD primary purpose is to analyze the BEAT data structures and relationships based on the activities from the BEAT IDEF0 diagrams. The BEAT ERD has five primary entities (DTIC website, User, Visualization Tool, Access Interface, and Excel Interface) that interact with the BEAT. There are multiple relationships and attributes associated with each entity. The below picture provides a depiction of the BEAT ERD.

17 | P a g e

Figure 19: BEAT Entity Relationship Diagram (Dataflow Model)

18 | P a g e

Rule Modeling (Decision Tables) The rules model is used to model what BEAT activities/processes in the IDEF0 are going to do. The rule

model for the BEAT depicts what is to be done when a set of conditions evaluates as true and what conditions

lend themselves to be false. To do this, the team decided on using a decision table. The decision table includes

all possible conditions based on the A0 diagram of the BEAT IDEF0. The decision table provided the best

way to formulate and evaluate relationships among the relevant data elements. The decision table was the

easiest and most suited way to specify what actions to take based on the complexities of the BEAT system.

Below highlights each of the Decision Tables that supported the BEAT.

Figure 22: BEAT Decision Table (Load PE and Project Level Data in BEAT – A1)

Figure 20: BEAT Decision Table (Identify and Parse Data – A2)

Figure 21: BEAT Decision Table (Store PE and Project Data – A3)

19 | P a g e

Figure 22: BEAT Decision Table (Display Query Dropdowns – A4)

Figure 23: BEAT Decision Table (Provides Visualization Tool – A5)

20 | P a g e

Technical Approach and Expected Results Because of the work done in the previous semester, there is an existing database architecture in place that stores

the DoD budget information at the PE level. The database is created using the XML files that are available

from the DoD through DTIC which are then parsed via an XML parser. Currently the database only ingests

program level data, however, the project and accomplishment data is also available in the same XML files.

The DoD budget information is available in two different formats R-1 and R-2. The R-2 data contains data

specific to each program and serves as the item justification for each program. That justification also provides

detailed information on the projects within each program. Therefore, our analysis will be most interested in

dissecting the R-2 XML files to retrieve the requisite project level information.

Figure 2: Example Project Level Data

The initial effort of this project will be to expand the current data absorption from only program level data to

include project and accomplishment information. In order to accomplish this, an XML parsing script will be

developed in Visual Basic for Applications (VBA) to absorb the program and project information and store

that data in a MS Access database. Once the data file is updated, the Access database will house all required

information in a structured format to be easily exported into the Excel BEAT.

Once the data has been properly housed, part two of the project will be to expand the capabilities within the

BEAT tool to visualize the latest information. Additional visualizations will be developed and integrated into a

single dashboard to provide a visual representation of the quantitative data, as well as display the qualitative

contextual information that is embedded within the budget data files. VBA will be used to facilitate the data

handling and visualization creation to provide a streamlined user friendly interface to automatically create and

modify the visualizations.

Upon completion of our support, the database will have an updated structure and parsing program to ingest

project level data. That new data will be housed in a structured format that can be easily exported into a csv

file for upload into additional data analysis programs, specifically the BEAT. Additionally, the BEAT tool will

contain new visualizations pertaining to the project level information to provide meaningful insights into trends

within the RDT&E community.

21 | P a g e

BEAT Model Changes

BEAT Version 1.0 (Fall 2016 Capstone Project)

Figure 3: BEAT v1.0 Visualizations

The figure above represents the BEAT system that the previous Capstone team developed based upon the

sponsor needs at that time. In version 1.0, the BEAT data was stored in a MySQL database that was populated

using a JAVA program to parse through the R1 and R2 budget XML to extract the required information. That

data was then queried and incorporated into the BEAT. The BEAT is a MS Excel based tool that allows the

user to select and visualize BAs, Services/Agencies, and PB years to visualize and gain insights. The R1 data

visualization provide many different charts and graphics that provide insights as to how programs stack against

each other in the R1 data. The R2 Data visualization allowed the user to select multiple programs in a single

PB year and compare those programs against each other. Those visualizations answer the question “How does

this program compare against others?” but doesn’t provide much insight about what’s going on within each of

those programs across all available budget submissions. In BEAT v2.0, we aim to answer that question, as well

as provide insights at the more granular project level.

BEAT Version 2.0 BEAT version 2.0 is a redesigned version of BEAT v1.0. After discussions with the sponsor, the database was

moved to MS Access and the XML parsing scripts rewritten in VBA and enhanced to ingest the project level

data in addition to the program data. Additionally, more visualizations accompanied by contextual information

provided in the data files are displayed in a consolidated dashboard, described later.

Visualization

R1 Data R2 Output

22 | P a g e

Figure 4: BEAT v2.0 Methodology

In BEAT version 2.0, the process for updating and loading information starts the same as version 1.0, by

downloading the PB data from DTIC. From there, the user can open the BEAT and select “Upload new R2

files” from the homepage menu. That process will then open the linked Access Database in the background

and launch the XML parsing algorithm. From there the user is prompted to select the folder containing the

XML files, and the parsing commences. Once completed the BEAT will then update queries and tables within

the tool that are linked to the database to ensure the most up to date information is available. The user is then

free to explore and gain insights to hundreds of programs and projects across all available budget submissions.

Database Update In BEAT v1.0, the PB data was stored in a MySQL database that was populated using a JAVA program that

parsed through the budget XML files. The new, improved database was moved to MS Access and the XML

parsing program was implemented in Visual Basic for Applications (VBA). That process moved the data

gathering process “under one roof” and allowed the BEAT (implemented in MS Excel) to more seamlessly

integrate with another MS Office product.

To extract the data from the XML files, the VBA script relies on the tree structure inherent to XML file types.

The script identifies the “root” program nodes, and then cycles through each child node passing the name of

the child node through a switch (or select) statement to identify and place the data value to the appropriate field

in Access. When the Project node is identified, it passes that sub tree structure through a similar process.

The Access database contains two tables, one for the PE data and the other for the Project Level data named

“Program Data” and “Project Data”, respectively. The composite key for the Program Data table are the

Download PB data from DTIC (Manual Process)

Open BEAT and Select “Upload new R2 files”

BEAT (Excel) Opens Access and runs the XML parsing Code

Excel Sheets Update Existing Queries and Connections

Visualization

Visualization Tool within BEAT that allows user to graphically display data based on inputs

Input Cells for scoping data

User conducts

exploratory

analysis and

informs

management Descriptive Information

23 | P a g e

Program Numbers and Budget Years. The Project Data composite key consists of the Program Number,

Project Number, and Budget Year. The tables are linked through the program number to allow for specific

SQL scripting.

The BEAT has been linked to the Access Database and two new worksheets have been created to query the

data each time the workbook is opened, or the database imports new data. Currently, the data set goes back to

2011 and contains 4366 Program records across the DoD comprising of 9388 Project Records

Figure 5: MS Access Data Tables

24 | P a g e

Figure 6: XML Program Element Tree Example

R2 Exhbit List

(Program List)

Program Element Number

Program Element Title

Budget YearService Agency

NameAppropriation

CodeAppropriation

Number

Program Element Funding

Prior Year

Current Year

Budget Year One

Budget Year Two

Budget Year Three

Budget Year Four

Budget Year Five

Project List

25 | P a g e

Figure 7: XML Project Element Tree Example

Project list

Project Number

ProjectTitle Poject Funding

Prior Year

Current Year

Budget Year One

Budget Year Two

Budget Year Three

Budget Year Four

Budget Year Five

Project NoteProject Mission

Description

26 | P a g e

BEAT Visualization (Program and Project Analysis Dashboard) The BEAT Program and Project Analysis Dashboards allows the user to visually analyze a specific program’s or project’s funding trends and relate those

variations to the notes from the budgets in a consolidated environment.

Figure 8: Program Analysis Dashboard

27 | P a g e

Program Analysis Dashboard

Program Selection The user can narrow the organizations and programs by selecting a (or all) BA of interest. From there, the user

selects among the organizations and agencies that have programs under the selected BA. The “Select Program”

options are then updated and the user can select a program belonging to the organization or agency by name

of Program Number.

Figure 9: Program Selection Fields

Additionally, the user can search for a program using the “Program Keyword Search” function. Through that

function, the user can filter all available programs by Service/Agency, BA (similar to the dashboard), or utilize

Keyword Searches that will allow the user to search by a specific word or phrase to aid in identifying the

program of interest. Once a program has been identified, the user can select that program and the Program

Analysis Dashboard will update to the selected program.

Figure 10: Program Search User Form

28 | P a g e

Contextual Program Information Once a program is selected, information about the program selected is populated with information from the

R-2 XML files that is housed in the Access database. That information includes program description,

earliest/latest PB years available, and program notes per PB year. Additionally, there is a button below the

Program Description to select the current program for analysis of its projects, described later.

Figure 11: Program Information

Program Visualizations and Statistics The top visualization included in the dashboard is a line chart that plots funding of the single program across

the various PB years available in the database. This visualization provides the user a visual representation of the

changes in funding across the budgets. The example below demonstrates a relatively steady funding and

projection stream, illustrated by the funding lines falling close, if not on top of one another. A program with

an unstable funding stream would have funding lines that do not fall relatively close to one another and have

many increases and decreases in funding that are not consistent across PBs. Included at the bottom of the

chart is the data that created the chart in a table to provide values for data to provide further information.

Figure 12: Program Budget Chart

29 | P a g e

Following the Program Budget Chart are basic summary statistics that have been generated on the available

data set. The summary statistics provide metrics describing the behavior of each FYs budget projections. The

statistics include Mean, Standard Deviation, Variance, Min, Median, Max, and Range. A stable budget will have

minimal Standard Deviations and Variances between budget submissions. Note that these statistics are not

generated across a single budget year to examine the stability or changes in a single PB’s projections. Rather,

these statistics are generated for a single FY across all the PBs that have an actual or projected value for that

FY.

Figure 13: Program Summary Statistics

The final visualization is a heat map that highlights the various deltas between PB projections for a FY and

their actual from that FY’s PB. Each PB contains the prior FY’s budget (FY-1), the current FY, and 5 out years

(FY+1-5), tied to the POM. For example, in the heat map below, the first value for column FY_2014 was

calculated as the absolute value of the difference between PB 2012 and PB 2013 projections for FY 2014.

Figure 14: Program Budget Heat Map

Given that there are only FY-1 to FY+5 years of data available in each PB, it is clearly not possible to calculate

deltas for all FY for each pair of PBs. The “#N/A” is used to indicate those FYs where there is no delta to

calculate between the selected PBs. For instance, in PB 2014, we have a prior year data point in FY 2013.

However, in PB 2015, there is no value for FY13, resulting in an #N/A for the delta between PB14 and PB 15

for FY_2013.

Project Analysis Dashboard The Project Analysis allows the user to investigate the different projects that belong to each program. These

insights provide more granular detail of how budgets are behaving within each program.

30 | P a g e

Project Selection The down selecting process follows the same as the Program Analysis Dashboard, but adds a Project Selection

by Project Name or Project Number. Similar to the Program Analysis Dashboard, there is a “Project Search”

function where the user can filter all available Projects by Service/Agency, Budget Activity, or search by

Program and/or Project key words and phrases.

Figure 15: Project Selection Fields

Figure 16: Project Search User Form

Contextual Project Information and Visualizations The Project Mission (analogous to the Program Description), Earliest and Latest PB Data Years, and Project

Notes are all imported for the select project, similar to the Program Analysis Dashboard. Additionally, the tool

provides the funding chart and heat map to highlight the deltas in funding using the same methodology outlined

for the program funding chart and heat map.

31 | P a g e

Figure 17: Project Analysis Dashboard

Additional Project Visualization

The Project Analysis Dashboard does not have the Summary Statistics that were provided at the program level.

Instead, a pie chart that breaks down the most recent budget year’s current year funding for each project across

the parent program was included to provide context as to how the project in question stacks up against the

other projects competing for funding in that program.

Figure 18: Project Analysis Pie Chart

32 | P a g e

Reporting Functions To facilitate the identification of programs or projects of interest, two reporting functions were included to

flag programs and projects that meet certain criteria, the Delta Analysis and New/Recently Ended Program

and Project Identification.

Delta Analysis The Delta Analysis examines the FY+1 projections and FY actuals of all programs and projects to identify

those that have differences in excess of a user defined budget variation percentage. In other words, any program

or project whose projection was more (or less) that X% of the actual using the formula:

|𝑃𝑟𝑜𝑗𝑒𝑐𝑡𝑖𝑜𝑛 − 𝐴𝑐𝑡𝑢𝑎𝑙|

𝐴𝑐𝑡𝑢𝑎𝑙

Those programs and projects that exceed the delta threshold are logged in a report that can be reviewed and

then explored further utilizing the Program and Project Analysis Dashboards.

Figure 19: Delta Analysis Reporting

Organization/Agency Program Number Program Name Year Projection Acutal Delta

Air Force 0101125F Nuclear Weapons Modernization 2015 67.874 33 -51.38%

Air Force 0105921F Service Support to STRATCOM - Space Activities2017 0 8.09 Inf

Air Force 0203761F Warfighter Rapid Acquisition Process (WRAP) Rapid Transition Fund2015 3.844 0 -100.00%

Air Force 0207100F LAAR Squadrons 2013 23.721 13.721 -42.16%

Air Force 0207131F A-10 Squadrons 2017 0 16.2 Inf

Air Force 0207133F F-16 Squadrons 2015 177.298 112.667 -36.45%

Air Force 0207142F F-35 Squadrons 2015 33 3 -90.91%

Air Force 0207142F F-35 Squadrons 2017 115.395 53.921 -53.27%

Air Force 0207171F F-15 EPAWSS 2016 68.944 0 -100.00%

Air Force 0207227F Combat Rescue - Pararescue 2015 0.542 0.35 -35.42%

Air Force 0207249F Precision Attack Systems Procurement2015 1.075 2 86.05%

Air Force 0207268F Aircraft Engine Component Improvement Program2015 139.369 89.369 -35.88%

Air Force 0207410F Air & Space Operations Center (AOC)2016 41.066 25.662 -37.51%

Air Force 0207412F Control and Reporting Center (CRC) 2016 0.552 0 -100.00%

Air Force 0207455F Three Dimensional Long-Range Radar (3DELRR)2017 14.939 8.139 -45.52%

Air Force 0207581F Joint Surveillance/Target Attack Radar System (JSTARS)2015 13.248 23.148 74.73%

Air Force 0208059F CYBER COMMAND ACTIVITIES 2015 68.099 38.099 -44.05%

Air Force 0303131F Minimum Essential Emergency Communications Network (MEECN)2015 36.288 18.481 -49.07%

Progam Projection vs Acutal Deltas greater than 35%

33 | P a g e

New or Recently Ended Program/Project Identification The BEAT will also parse through both program and project level data to identify programs/projects who’s

PB is new within the last three years or no longer have data in the most recent PB. Note that the “beginning”

or “ending” of a program/project can be difficult to automatically define and flag. For example, a single PB’s

projections may go to zero for consecutive years but in a subsequent PB we may see that program/project

regain a funding stream. Additionally, programs/projects may have a “Cost to Complete”, but get canceled

prior to reaching that cost. It has also been observed that programs/projects may get rolled into others, and

seemingly disappear from the data set. It is only with careful observation and human investigation that a

program/project’s start or end can be identified with certainty.

Understanding those limitations, this report is generated under the following criteria:

“New” Programs/Projects

a. The first budget year within the dataset for a program/project (identified by program/project

number) is greater than or equal 3 years prior to the current year (as defined by the user)

“Ended” Programs/Projects

a. The final budget year within the data set for a program/project (identified by its

program/project number) is less than the current year (as defined by the user)

Using these criteria, a new worksheet is created that lists all the programs/projects that violate those criteria for

further inspection.

Figure 20: “New" Program/Project Report

Figure 21: "Ended" Programs/Projects

*ASSUMPTION: A program/Project is flagged as "Started" or "New" if there is not prior year data available

Organization/Agency Program Number Program Name Organization/Agency Program Number Program Name Project Number Project Name

Air Force 0101122F Air-Launched Cruise Missile (ALCM) Air Force 0101113F B-52 SQUADRONS 675041 Bomber Tactical Data Link

Air Force 0101126F B-1B Squadrons Air Force 0101113F B-52 SQUADRONS 675048 1760 INTERNAL WEAPONS BAY UPGRADE (IWBU)

Air Force 0101127F B-2 SQUADRONS Air Force 0101113F B-52 SQUADRONS 675049 MODE S/5 IFF

Air Force 0101213F Minuteman Squadrons Air Force 0101113F B-52 SQUADRONS 675050 CONECT

Air Force 0101313F Strat War Planning System - USSTRATCOM Air Force 0101113F B-52 SQUADRONS 675051 ANTI-SKID

Air Force 0101314F Night Fist - USSTRATCOM Air Force 0101113F B-52 SQUADRONS 675055 GPS-IU

Air Force 0101316F Worldwide Joint Strategic Communications Air Force 0101113F B-52 SQUADRONS 675056 B-52 Radar Modernization Program (RMP)

Air Force 0102110F UH-1N Replacement Program Air Force 0101113F B-52 SQUADRONS 675057 B-52 Low Cost Improvement (LCI)

Air Force 0102326F Region/Sector Operation Control Center Modernization Program Air Force 0101113F B-52 SQUADRONS 675058 B-52 Weapon Sys Trainer Air Ref Training Upgrade

Air Force 0102823F Strategic Aerospace Intelligence System Activities Air Force 0101113F B-52 SQUADRONS 675160 B-52 Crypto Modernization

Air Force 0105921F Service Support to STRATCOM - Space Activities Air Force 0101122F Air-Launched Cruise Missile (ALCM)674797 Flight Testing & Navigation Enhancement

Air Force 0201184F Counter Narcotics Technology Program Office Air Force 0101126F B-1B Squadrons 675344 B-1B Modernization

Air Force 0203761F Warfighter Rapid Acquisition Process (WRAP) Rapid Transition Fund Air Force 0101127F B-2 SQUADRONS 675345 B-2 Modernization

Air Force 0205219F MQ-9 UAV Air Force 0101127F B-2 SQUADRONS 676021 BASELINE SUPPORT

Air Force 0205671F Joint Counter RCIED Electronic Warfare Air Force 0101127F B-2 SQUADRONS 676022 EHF SATCOM and Computer

Programs Started in Last 3 years Projects Started in Last 3 years

Organization/Agency Program Number Program Name Last Data Year Organization/Agency Program Number Program Name Program Number Program Name Last Data Year

Air Force 0101113F B-52 SQUADRONS 2017 Air Force 0101113F B-52 SQUADRONS 675039 B-52 System Improvements 2017

Air Force 0101122F Air-Launched Cruise Missile (ALCM)2017 Air Force 0101113F B-52 SQUADRONS 675041 Bomber Tactical Data Link 2017

Air Force 0101125F Nuclear Weapons Modernization 2017 Air Force 0101113F B-52 SQUADRONS 675048 1760 INTERNAL WEAPONS BAY UPGRADE (IWBU)2017

Air Force 0101126F B-1B Squadrons 2017 Air Force 0101113F B-52 SQUADRONS 675049 MODE S/5 IFF 2015

Air Force 0101127F B-2 SQUADRONS 2017 Air Force 0101113F B-52 SQUADRONS 675050 CONECT 2017

Air Force 0101213F Minuteman Squadrons 2017 Air Force 0101113F B-52 SQUADRONS 675051 ANTI-SKID 2015

Air Force 0101313F Strat War Planning System - USSTRATCOM2017 Air Force 0101113F B-52 SQUADRONS 675055 GPS-IU 2017

Air Force 0101314F Night Fist - USSTRATCOM 2017 Air Force 0101113F B-52 SQUADRONS 675056 B-52 Radar Modernization Program (RMP)2017

Air Force 0101316F Worldwide Joint Strategic Communications2017 Air Force 0101113F B-52 SQUADRONS 675057 B-52 Low Cost Improvement (LCI)2017

Air Force 0102110F UH-1N Replacement Program 2017 Air Force 0101113F B-52 SQUADRONS 675058 B-52 Weapon Sys Trainer Air Ref Training Upgrade2017

Air Force 0102326F Region/Sector Operation Control Center Modernization Program2017 Air Force 0101113F B-52 SQUADRONS 675160 B-52 Crypto Modernization 2017

Air Force 0102823F Strategic Aerospace Intelligence System Activities2014 Air Force 0101122F Air-Launched Cruise Missile (ALCM)674797 ALCM Upgrades 2017

Air Force 0105921F Service Support to STRATCOM - Space Activities2017 Air Force 0101125F Nuclear Weapons Modernization657007 B61 LIFE EXTENSION PROGRAM 2017

Former Programs Not Included in Current FY Former Projects Not Included in Current FY

34 | P a g e

Third Offset Strategy Technological superiority has been a staple of the DoD strategy since post WWII. This philosophy and follow-

on investment has allowed U.S. to outpace its peers and remain at the forefront of battlefield innovation.

However, since the current budget drawdown and the military mission shift to counterinsurgency (COIN), it is

no longer safe to assume technological superiority. The 3rd offset strategy remained a top DoD priority in 2016.

The 3rd offset strategy is being pursued to counter great peer competitors like China and Russia. Its focus is

on continuing U.S. military technological superiority. Released publicly, the President’s FY2017 budget

included over $18 billion over the course of the FYDP for 3rd Offset Related investments. Some of the focus

areas of this $18 billion are on anti-access technology, undersea warfare capability, human-machine

collaboration or unmanned and manned systems, and asymmetric capabilities (Cyber, Space, EW, and Laser).

Third Offset Strategy (if maintained) provides significant opportunities for future RDT&E investment- should

observe trends of increased investments in these capabilities. The picture below shows the RDT&E budget

authority by service during the counter-insurgency fight in Afghanistan and Iraq and the follow-on military

drawdown.

The below two charts provide an example of how the sponsor can utilize the BEAT to identify areas of interest

in RDT&E investment. This is the part where the sponsor or user begins to match policy with trend

development found in the BEAT. With the DII of 2014, and focus on a 3rd Offset Strategy, there has been

renewed emphasis in DoD investment in technology. The first chart, High Energy Laser, is a 3rd Offset

investment in laser energy as a means for force protection. Much emphasis has been placed on the need to

counter rocket, artillery, and mortar munitions along with interdicting UAVs. This chart shows what will

happen to an existing project (Army) that has an uptick in spending due policy changes and refocus on technical

innovation. Trends like this could be identified by the sponsor by matching 3rd Offset policy and capabilities

with invigorated spending.

35 | P a g e

U.S. Army – High Energy Laser (PE: Advanced Weapons Technology)

This second chart displays a capability that did not exist in any past PB. This capability is tied directly to Air

Force investments in asymmetric capabilities with the 3rd Offset initiative. This capability focuses heavily on

autonomous learning, which is a staple of future technology innovation.

U.S. Airforce – Cyber Technology Pattern Recognition and Autonomous Learning (PE:

Aerospace Sensors)

Overall, the BEAT provides a unique, first-of-its-kind way to visualize PE and project level data. It will allow

the user the opportunity to start matching policy changes to budgetary changes. Special emphasis should focus

on the mapping of capabilities and technologies that support the 2014 Defense Innovation Initiative and the

3rd Offset Strategy.

Existing Project with Uptick in future spending

New Project

36 | P a g e

Conclusion In conclusion, the BEAT version 2.0 provides the user with a Decision Management Support Tool that

allows for exploratory analysis of RDT&E PE and project level data. The BEAT can present data in minutes,

run surface level analytics by displaying requested query information in a user-friendly way to identify

trends/changes, and provide a visual and statistics of beginning/ending PE/project funding. The BEAT

Version 2.0 has been created in such a way that SPA and future users can import future data and immediately

start using the tool. Overall, the team met all deliverables and has provided a way forward in moving the

project from Mile Marker 2.5 to Mile Marker 5.0 (see Figure below).

Way Forward Recommendations The BEAT and the analysis it provides is an ongoing endeavor to provide further analysis to the sponsor and

other stakeholders. Future recommendations for development of the BEAT include:

Retrieve and house accomplishment data in the database

Improve the database by creating more tables and linking them appropriately

Explore and incorporate predictive analytics tools such as Linear Regressions, Support Vector

Machines, K-Nearest Neighbors, etc. to better analyze data (may require more advanced analytical

software packages/programming languages e.g. Julia, Python, R, SAS)

Explore and incorporate textual analytics on the program/project descriptions and notes to

investigate correlations between keywords/phrases and budget projections (may require more

advanced analytical software packages/programming languages e.g. Julia, Python, R, SAS)

Continue to link policy (National Military Strategy, Defense Innovation Initiative) to budgetary

trend analysis using BEAT (Significant opportunity for future RDT&E investment if current

administration and DoD continues to invest in a Third Offset Strategy)

Upgrades to BEAT:

o Heat Chart with conditional formatting based on relative changes

o Link each program to its page in the corresponding budget file

Figure 22: BEAT Roadmap

I | P a g e

Appendix A- References American Association for the Advancement of Science 2016 [3] - Govini. No Date.

DoD Financial Management Regulation Volume 2B, Chapter 5

Department of Defense, Future Years Defense Program (FYDP) Structure, April 2014 at

http://www.prim.osd.mil/Documents/DoD_704507_future_years_defense.pdf

Department of Defense (DoD) Releases Fiscal Year 2017 President’s Budget Proposal

Defense Technical Information Center (DTIC) – www.dtic.mil

GMUSEORSPA at https://gmuseorspa.wordpress.com/deliverables/

Military budget of the United States, Wikipedia

U.S. House of Representatives, Committee on the Budget at http://budget.house.gov/budgetprocess/

"US Department of Defense Third Offset Standard Market Taxonomy." Analyst Report, Washington, D.C.

II | P a g e

Appendix B- Definitions Budget Activity 1, Basic Research Basic research is systematic study directed toward greater knowledge or understanding of the fundamental

aspects of phenomena and of observable facts without specific applications towards processes or products in

mind. It includes all scientific study and experimentation directed toward increasing fundamental knowledge

and understanding in those fields of the physical, engineering, environmental, and life sciences related to long-

term national security needs. It is farsighted high payoff research that provides the basis for technological

progress. Basic research may lead to: (a) subsequent applied research and advanced technology developments

in Defense-related technologies, and (b) new and improved military functional capabilities in areas such as

communications, detection, tracking, surveillance, propulsion, mobility, guidance and control, navigation,

energy conversion, materials and structures, and personnel support. Program elements in this category involve

pre-Milestone A efforts

Budget Activity 2, Applied Research Applied research is systematic study to understand the means to meet a recognized and specific national security

requirement. It is a systematic application of knowledge to develop useful materials, devices, and systems or

methods. It may include design, development, and improvement of prototypes and new processes to meet

general mission area requirements. Applied research translates promising basic research into solutions for

broadly defined military needs, short of system development. This type of effort may vary from systematic

mission-directed research beyond that in Budget Activity 1 to sophisticated breadboard hardware, study,

programming and planning efforts that establish the initial feasibility and practicality of proposed solutions to

technological challenges. It includes studies, investigations, and non-system specific technology efforts. The

dominant characteristic is that applied research is directed toward general military needs with a view toward

developing and evaluating the feasibility and practicality of proposed solutions and determining their

parameters. Applied Research precedes system specific research. Program control of the Applied Research

program element is normally exercised by general level of effort. Program elements in this category involve pre-

Milestone B efforts, also known as Concept and Technology Development phase tasks, such as concept

exploration efforts and paper studies of alternative concepts for meeting a mission need.

Budget Activity 3, Advanced Technology Development (ATD) This budget activity includes development of subsystems and components and efforts to integrate subsystems

and components into system prototypes for field experiments and/or tests in a simulated environment. ATD

includes concept and technology demonstrations of components and subsystems or system models. The

models may be form, fit and function prototypes or scaled models that serve the same demonstration purpose.

The results of this type of effort are proof of technological feasibility and assessment of subsystem and

component operability and producibility rather than the development of hardware for service use. Projects in

this category have a direct relevance to identified military needs. Advanced Technology Development

demonstrates the general military utility or cost reduction potential of technology when applied to different

types of military equipment or techniques. Program elements in this category involve pre-Milestone B efforts,

such as system concept demonstration, joint and Service-specific experiments or Technology Demonstrations.

Projects in this category do not necessarily lead to subsequent development or procurement phases.

III | P a g e

Budget Activity 4, Advanced Component Development and Prototypes (ACD&P) Efforts necessary to evaluate integrated technologies, representative modes or prototype systems in a high

fidelity and realistic operating environment are funded in this budget activity. The ACD&P phase includes

system specific efforts that help expedite technology transition from the laboratory to operational use.

Emphasis is on proving component and subsystem maturity prior to integration in major and complex systems

and may involve risk reduction initiatives. Program elements in this category involve efforts prior to Milestone

B and are referred to as advanced component development activities and include technology demonstrations.

Completion of Technology Readiness Levels 6 and 7 should be achieved for major programs. Program control

is exercised at the program and project level. A logical progression of program phases and development and/or

production funding must be evident in the FYDP.

Budget Activity 5, System Development and Demonstration (SDD) SDD programs have passed Milestone B approval and are conducting engineering and manufacturing

development tasks aimed at meeting validated requirements prior to full-rate production. This budget activity

is characterized by major line item projects and program control is exercised by review of individual programs

and projects. Prototype performance is near or at planned operational system levels. Characteristics of this

budget activity involve mature system development, integration and demonstration to support Milestone C

decisions, and conducting live fire test and evaluation (LFT&E) and initial operational test and evaluation

(IOT&E) of production representative articles. A logical progression of program phases and development and

production funding must be evident in the FYDP consistent with the Department's full funding policy.

Budget Activity 6, RDT&E Management Support

This budget activity includes research, development, test and evaluation efforts and funds to sustain and/or

modernize the installations or operations required for general research, development, test and evaluation. Test

ranges, military construction, maintenance support of laboratories, operation and maintenance of test aircraft

and ships, and studies and analyses in support of the RDT&E program are funded in this budget activity. Costs

of laboratory personnel, either in-house or contractor operated, would be assigned to appropriate projects or

as a line item in the Basic Research, Applied Research, or Advanced Technology Development program areas,

as appropriate. Military construction costs directly related to major development programs are included.

Budget Activity 7, Operational System Development

This budget activity includes development efforts to upgrade systems that have been fielded or have received

approval for full rate production and anticipate production funding in the current or subsequent fiscal year. All

items are major line item projects that appear as RDT&E Costs of Weapon System Elements in other programs.

Program control is exercised by review of individual projects. Programs in this category involve systems that

have received Milestone C approval. A logical progression of program phases and development and production

funding must be evident in the FYDP, consistent with the Department's full funding policy.

IV | P a g e

Appendix C- Acronyms

ACD&P Advance Component Development & Prototypes

ATD Advance Technology Development

BEAT Budget Exhibit Analysis Tool

COIN Counter Insurgency

CPI Cost Performance Index

DII Defense Innovation Initiative

DoD Department of Defense

DTIC Defense Technical Information Center

ERD Entity Relationship Diagram

EVM Earned Value Management

EW Electronic Warfare

FYDP Future Year Defense Program

GMU George Mason University

IPR In Process Review

JTAAD Joint Airforce and Analysis Division

LFT&E Live Fire Test & Evaluation

MR Mission Requirement

OSAG Operations Support Analysis Group

OSD Office of Secretary of Defense

PB Program Budget

PE Program Element

POM Program Objective Memorandum

RDT&E Research, Development, Testing & Evaluation

SDD System Development and Demonstration

SOW Statement of Work

SPA Systems Planning and Analysis, Inc.

SPI Schedule Performance Index

SQL Structured Query Language

SR System Requirement

S&T Science & Technology

UAV Unmanned Aerial Vehicle

VBA Visual Basic for Applications

WBS Work Breakdown Structure

XML eXtensible Markup Language

V | P a g e

Appendix D- Use Cases in Detail This appendix provides a more descriptive nature to each Use Case.

Use Case 1: Load R-2 PE and Project Level Data into BEAT 1. BEAT Characteristic Information (Use Case 1)

Goal in Context: Project level data from R-2 files downloaded into BEAT

Scope: Access based Database, XML VBA, Excel

Level: Primary

Pre-Condition: R-2 project level data in XML available from DTIC website

Success End Condition: Project level data loaded into BEAT

Minimal Guarantees: BEAT has PE and project level data and is ready for analysis

Primary Actor: User of the database

Supplementary Actors: BEAT, Access Interface, Access Database, XML VBA Parse

Trigger Event: R-2 (PE and project level data) downloaded from DTIC website

2. BEAT Main Success Story (Use Case 1)

VI | P a g e

Include Use Case 1

3. BEAT Scenario Extensions (Use Case 1)

4. BEAT Related Information (Use Case 1)

Schedule: Main Use Case; must be tested before providing to sponsor

Priority: Top, must be executed

Performance Target: All project level, unclassified budget level data is available from FY’11

through FY’20

Frequency: Every time a download is required due to release of new Presidential Budget for

corresponding FY

Super Use Case: This is the parent use case

Sub Use Case: N/A

Channel To Primary Actor: Reliant on R-2 project level data download without error

Secondary Actors: Access Database

Load R-2 PE and Project Level Data into BEAT Use Case Diagram for Use Case 1

System Boundary

VII | P a g e

Use Case 2: Perform Visualization of Project Level Data in BEAT 1. BEAT Characteristic Information (Use Case 2)

Goal in Context: A manageable tool available for user to visualize, graph, and scope project level

data

Scope: The BEAT Visualization has the capability to interact with R-1/R-2 (down to project level)

data

Level: Primary or User-Goal

Pre-Condition: Visualization tool built with VBA (In Excel)

Success End Condition: User able to graph data using a number of visualization options.

Minimal Guarantees: The BEAT Visualization code updated, the Access database has all current

R1 and R2 data, all project level data and associations up to date

Primary Actor: User of the data

Supplementary Actors: BEAT Visualization Tool, Access Database

Trigger Event: User requests to visualize project level data

2. BEAT Main Success Story (Use Case 2)

Include Use Case 2

VIII | P a g e

3. BEAT Scenario Extensions (Use Case 2)

4. BEAT Related Information (Use Case 2)

Schedule: Main Use Case; must be tested before providing to sponsor.

Priority: Top, must be executed

Performance Target: All project level, unclassified budget level data is available from FY’11

through FY’20

Frequency: Every time a query and filter for specific project level data is executed

Super Use Case: This is the parent use case

Sub Use Case: N/A

Channel To Primary Actor: Reliant on project-level data stored in Access database and read by

Excel VBA

Secondary Actors: Access Database, BEAT Visualization Tool

Perform Visualization of Project Level Data Use Case Diagram for Use Case 2

System Boundary

IX | P a g e

Appendix E- Deliverables The team is currently scheduling a minimum of three customers+ facing meetings to review progress that is

being made. The team will provide SPA with a written report and brief the results of the research and analysis

obtained. The team will continue to refine and develop BEAT to fit the newly derived needs of SPA. The team

will also update the BEAT user manual. Each deliverable will identify future budget trends that are nested with

the Office of Secretary of Defense’s offset strategy more specifically by identifying specific accomplishments

of interest within applicable projects.

The project team provided the following deliverables:

1) BEAT continued refinement and project level data uploaded

2) BEAT user interface improvement with ability to visualize new information

3) BEAT user manual updated

4) BEAT example using policy (3rd offset examples)

5) Written report and briefing summarizing results of the analysis

6) Websites with all products https://stbudgetgmuspring17.wixsite.com/gmuproject

7) Listing of all sources

8) Visualization Tool that identifies when projects begin and end

9) Identification of future budget trends of projects that are nested with Office of Secretary of Defense’s

offset strategy (this deliverable is still on-going)

X | P a g e

Appendix F- Program Management The team used Microsoft Project as the tool when developing the Work Breakdown Structure (WBS) and

project schedule. The final WBS was broken into seven high level phases; Analysis, Design, Implementation,

Verification, Final Deliverables, Management, and Research. The project schedule was broken up into the seven

high level phases as shown in the WBS and then further defined into smaller tasks. This project was estimated

to take a total of 75 days to completion starting from January 30, 2017 to May 12, 2017. This project took into

the systems engineering lifecycle methodology as tasks ranged from problem formulation to architecture design

and tool verification and validation.

S&T Budget Project 1.0

Analysis 1.1

Define Problem 1.1.1

Deliverable: Preliminary Problem

Definition 1.1.2

Define Problem Scope 1.1.3

Define Problem Overview

1.1.4

Initial Sponsor Meeting

1.1.5

Design 1.2

Requirements Gathering

1.2.1

Data Gathering 1.2.2

Develop Tool Methodology

1.2.3

Deliverable: Project Proposal

1.2.4

Sponsor Meeting (IPR#1)

1.2.5

Develop Use Cases 1.2.6

Develop Engineering Diagrams

1.2.7

Implementation 1.3

Database Setup 1.3.1

Database Code 1.3.2

Data Analysis 1.3.3

Sponsor Meeting (IPR#2)

1.3.4

Deliverable: Progress Report

1.3.5

Verification 1.4

Test Data Integrity 1.4.1

Test User Interface 1.4.2

QA Testing 1.4.3

Sponsor Meeting (IPR#3)

1.4.4

Final Deliverables 1.5

Deliver Final Report 1.5.1

Conduct Final Presentation

1.5.2

Produce Website 1.5.3

Deliver Final Tool Product

1.5.4

Management 1.6

Research 1.7

Figure 23: Work Breakdown Structure (WBS)

XI | P a g e

Figure 24: Project Schedule

XII | P a g e

The project budget and schedule was calculated using the Earned Value Management (EVM) method. The project length was a total of 15 weeks and

each of the three-team members earned rate of $85.11 after a 2% overhead rate was added. The planned budget was calculated at a 30-hour work week.

The team decided that the best approach was to follow the 50/50 method where 50% of the budget would be taken at the start of each of the seven high

level phases derived from the WBS and once that phase was completed the other 50% was granted. Upon completion of the 15 weeks the teams remained

under budget with a cost performance index (CPI) of 1.01. The team remained on schedule meaning all tasks were completed by the May 12, 2017 deadline

resulting in a schedule performance index (SPI) of 1.00. Even though the CPI and SPI fluctuated throughout the 15 weeks the team managed to stay

under budget and remain within schedule producing a decision support that the SPA sponsor was pleased with.

Table 1: Earned Value Management (EVM) Breakdown

XIII | P a g e

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

1.60

1.80

Week 1Week 2Week 3Week 4Week 5Week 6Week 7Week 8Week 9 Week10

Week11

Week12

Week13

Week14

Week15

Rati

o

CPI and SPI

CPI

SPI

$-

$5,000.00

$10,000.00

$15,000.00

$20,000.00

$25,000.00

$30,000.00

$35,000.00

$40,000.00

$45,000.00

Do

llars

Earned Value

Earned Value

Planned Value

Actual Cost

Figure 25: CPI and SPI Graph

Figure 2625: EVM Graph

XIV | P a g e

Appendix G- Recognition The team wanted to thank the following for their support throughout this project:

Slider Griese (SPA)

Brandon Davis (SPA)

Chris Anderson (SPA)

Andy Dichter (SPA)

Matt Hooker (SPA)

Luke Piepkorn (SPA)

Alex Park (SPA)

Dr. Kathryn Laskey (GMU)

SYST 699 Classmates (GMU)