SYST/OR 699 Master’s Project – Final Report
Department of Defense Research, Development, Test and Evaluation Budget Exhibit Analysis Tool Enhancements
Prepared by Craig Jayson, Josh Singh, and Phillip Sutton
i
Table of Contents Table of Figures ................................................................................................................................................................. iii
Table of Tables ................................................................................................................................................................... iii
Problem Definition ............................................................................................................................................................. 1
Sponsorship ..................................................................................................................................................................... 1
Background ..................................................................................................................................................................... 1
Problem Statement ......................................................................................................................................................... 4
Needs Statement ............................................................................................................................................................. 4
Mission Statement .......................................................................................................................................................... 4
Project Objective ............................................................................................................................................................ 4
Stakeholders .................................................................................................................................................................... 5
Scope ..................................................................................................................................................................................... 6
Requirements ....................................................................................................................................................................... 7
Use Case Diagrams ............................................................................................................................................................. 9
Use Case 1: Load R-2 PE and Project Level Data into BEAT ............................................................................... 9
Use Case 2: Load R-2 PE and Project Level Data into BEAT ............................................................................. 11
Structural Analysis ............................................................................................................................................................ 12
Functional Decomposition .............................................................................................................................................. 13
Activity Modeling (IDEF0) ............................................................................................................................................. 14
Data Modeling (Entity Relationship Diagram) ............................................................................................................ 16
Rule Modeling (Decision Tables) ................................................................................................................................... 18
Technical Approach and Expected Results .................................................................................................................. 20
BEAT Model Changes ..................................................................................................................................................... 21
BEAT Version 1.0 (Fall 2016 Capstone Project) .................................................................................................... 21
BEAT Version 2.0 ........................................................................................................................................................ 21
Database Update ............................................................................................................................................................... 22
BEAT Visualization (Program and Project Analysis Dashboard) ............................................................................ 26
Program Analysis Dashboard ..................................................................................................................................... 27
Program Selection .................................................................................................................................................... 27
Contextual Program Information ......................................................................................................................... 28
Program Visualizations and Statistics ................................................................................................................... 28
Project Analysis Dashboard ........................................................................................................................................ 29
Project Selection....................................................................................................................................................... 30
Contextual Project Information and Visualizations ........................................................................................... 30
ii
Additional Project Visualization ............................................................................................................................ 31
Reporting Functions ......................................................................................................................................................... 32
Delta Analysis ............................................................................................................................................................... 32
New or Recently Ended Program/Project Identification ..................................................................................... 33
Third Offset Strategy ........................................................................................................................................................ 34
U.S. Army – High Energy Laser (PE: Advanced Weapons Technology) ........................................................... 35
U.S. Airforce – Cyber Technology Pattern Recognition and Autonomous Learning (PE: Aerospace
Sensors) .......................................................................................................................................................................... 35
Conclusion ......................................................................................................................................................................... 36
Way Forward Recommendations .............................................................................................................................. 36
Appendix A- References ..................................................................................................................................................... I
Appendix B- Definitions .................................................................................................................................................. II
Budget Activity 1, Basic Research .......................................................................................................................... II
Budget Activity 2, Applied Research ..................................................................................................................... II
Budget Activity 3, Advanced Technology Development (ATD)...................................................................... II
Budget Activity 4, Advanced Component Development and Prototypes (ACD&P) ................................. III
Budget Activity 5, System Development and Demonstration (SDD) ........................................................... III
Budget Activity 6, RDT&E Management Support ........................................................................................... III
Budget Activity 7, Operational System Development ...................................................................................... III
Appendix C- Acronyms ................................................................................................................................................... IV
Appendix D- Use Cases in Detail ................................................................................................................................... V
Use Case 1: Load R-2 PE and Project Level Data into BEAT .............................................................................. V
Use Case 2: Perform Visualization of Project Level Data in BEAT ................................................................. VII
Appendix E- Deliverables ...............................................................................................................................................IX
Appendix F- Program Management ............................................................................................................................... X
Appendix G- Recognition ........................................................................................................................................... XIV
iii
Table of Figures Figure 1: DoD Budget (source DoD.gov) ...................................................................................................................... 1
Figure 2: Example Project Level Data ........................................................................................................................... 20
Figure 3: BEAT v1.0 Visualizations ............................................................................................................................... 21
Figure 4: BEAT v2.0 Methodology ............................................................................................................................... 22
Figure 5: MS Access Data Tables ................................................................................................................................... 23
Figure 6: XML Program Element Tree Example ........................................................................................................ 24
Figure 7: XML Project Element Tree Example ........................................................................................................... 25
Figure 8: Program Analysis Dashboard ......................................................................................................................... 26
Figure 9: Program Selection Fields ................................................................................................................................ 27
Figure 10: Program Search User Form .......................................................................................................................... 27
Figure 11: Program Information .................................................................................................................................... 28
Figure 12: Program Budget Chart .................................................................................................................................. 28
Figure 13: Program Summary Statistics ......................................................................................................................... 29
Figure 14: Program Budget Heat Map .......................................................................................................................... 29
Figure 15: Project Selection Fields ................................................................................................................................. 30
Figure 16: Project Search User Form ............................................................................................................................ 30
Figure 17: Project Analysis Dashboard ......................................................................................................................... 31
Figure 18: Project Analysis Pie Chart ............................................................................................................................ 31
Figure 19: Delta Analysis Reporting .............................................................................................................................. 32
Figure 20: “New" Program/Project Report ................................................................................................................. 33
Figure 21: "Ended" Programs/Projects ........................................................................................................................ 33
Figure 22: BEAT Roadmap ............................................................................................................................................. 36
Figure 23: Work Breakdown Structure (WBS) .............................................................................................................. X
Figure 24: Project Schedule .............................................................................................................................................XI
Figure 26: EVM Graph ................................................................................................................................................ XIII
Table of Tables Table 1: Earned Value Management (EVM) Breakdown ......................................................................................... XII
1 | P a g e
Problem Definition
Sponsorship Systems Planning and Analysis, Inc. (SPA) is a professional services contractor located in Alexandria, VA. SPA has been in business since 1972 with the organization first working with the Department of the Navy. Since then, SPA has expanded its customer base to include supporting clients such as the services that make up the Department of Defense (DoD), Homeland Security, and the Department of Energy. This project is a continuation of a capstone project selected last year that supported categorizing and analyzing
past, present, and future budget submissions. The original problem statement had been derived from
conversations with Operations Support Analysis Group (OSAG), the original client of the project. The
proposal presented by OSAG was addressed during the fall ’16 Capstone project. Subsequently, SPA took over
as the lead for sponsorship and has presented this year’s team with guidance on the below problem statement.
The remainder of the paper will cover where the team started, what the team did, and detail recommendations
for future work.
Background The DoD budget, is an annual appropriation allocation that makes up the largest portion of the U.S.
government’s discretionary spending; in FY 2015, the DoD consumed over 50% of discretionary spending.
The U.S. defense outlay is equivalent to the next seven largest military budgets combined – India, the United
Kingdom, France, Saudi Arabia, Russia, and China. In 2016, over $580 billion was allocated to fund the DoD
seen in Figure 1 (source defense.gov).
Figure 1: DoD Budget (source DoD.gov)
Since post war WWII, the budget has fluctuated with peaks (overseas contingency operations, war-time
operations) and troughs (defense cuts, interwar periods). Over the past decade plus the DoD budget has seen
a precipitous decline in the budget as operations in Afghanistan and Iraq have slowed. This decline has been
of some concern for some pundits in the defense industry with their opinion that the U.S. has lost a decade or
more of technological advancements. While the U.S. has focused on counterinsurgency and Congressional
budget squabbles, peer competitors have continued to advance their militaries. The figure below provides a
visual of the ebbs and flows of the DoD budget over time.
2 | P a g e
Figure 2: DoD Budget from 1950 – 2025 in FY16 dollars (source: gmuseorspa.wordpress.com)
The budget is nested with the National Military Strategy and the President’s budget request and reflects the
strategic outlook and operational commitments necessary to fight and win in any future conflicts. Addressing
the multiple strategic challenges and threats, whether it is North Korean aggression or China island building,
the budget considers new and innovative ways to meet mission requirements. The U.S. must remain a
modernized, technologically superior global force with the capability to project power and respond to
contingences. As former Secretary of Defense Ash Carter stated, “even as we fight today’s fights, we must also
be prepared for the fights that might come in 10, 20, or 30 years”.
Since 2014, following Secretary of Defense Hagel’s Innovation Initiative, there has been a push to invest monies
in finding capabilities that have the potential to provide a technological offset against near peer competitors.
This offset strategy has led to the Deputy Secretary of Defense, Robert Work, creating a deterrence panel that
continues to evaluate an offset strategy. One of the major efforts that the deterrence panel is mandated to
focus on is the evaluation of technologies and capabilities that are intended to address both capability gaps and
technological opportunities that would reinvigorate the U.S. dominance in power projection. Capabilities of
interest for the panel include: multi-domain fires, electronic and cyber warfare, manned-unmanned systems,
robotics, undersea warfare, hypersonic munitions, advanced engineering, among others.
The DoD budget is divided into seven distinct budget appropriation categories. One of the categories is the
Research, Development, Test and Evaluation (RDT&E) budget; which is that portion of the DoD budget that
focuses on Science and Technology (S&T) and prototype development. Since FY00, the amount appropriated
to the DoD RDT&E budget has patterned itself to fall in line with national security concerns and overseas
military activities. Currently, this appropriation is 11.8% of the total DoD budget, or $69 billion.
Figure 3: FY16 Budget By Appropriation (source DoD.gov)
3 | P a g e
It is this RDT&E appropriation category that the previous GMU Capstone Group gathered DoD Budget data
on and developed a tool, known as the Budget Exhibit Analysis Tool (BEAT), to begin analyzing multi-year
budget data.1 The previous group drilled-down to the Program Element (PE) level for each service agency.
The specifics regarding scoping of the project and last year’s work can be found at the following website:
https://gmuseorspa.wordpress.com/. The figure below summarizes the level of detail at which the previous
project group scoped the problem to.
Figure 4: Budget Activity Scoping
The PE is the primary data element in the FYDP and normally the smallest aggregation of resources used by
the Office of the Secretary of Defense (OSD) for analysis. PEs are designed and quantified to be
comprehensive and mutually exclusive. Each PE has a distinct numbers/letter identifier. The first number,
in our scope above is 06, is the DoD Program of Research and Development. The second number, 3, is
related to the budget activity (BA). The BA is a category within each appropriation and fund account that
identifies the purposes, projects, or types of activities financed by the respective appropriation or fund. The
BAs are explained in more detail in Appendix B of this document. The BAs that were in-scope for this
project are seen highlighted in Figure 3. The next number, in this case 28, is a unique serial number assigned
by OSD for each PE. These numbers are in sequential order based on the PE. Following the serial number
is the equipment/activity type. The equipment/activity type is another way in which OSD categorized
RDT&E programs. Following the equipment type is the only letter identifier in the PE. This letter identifier
represents a code for a specific service/agency. Figure 4 below provides the listing of equipment/activity
type and code for service/agency.
Figure 5: PE identification last number and letter
1 The Budget Exhibit Analysis Tool allows users to quickly identify Program Elements within RDT&E using historic data and to
visualize the individual budget trends over time using Future Years Defense Program (FYDP) projected data.
4 | P a g e
Problem Statement Prior to the efforts of the Fall 2016 Capstone team, there was no known method or available tool to digest and
depict multiple budget cycles (the current year plus four future years known as the Program Objective
Memorandum2) to facilitate the analysis and characterization of past, current, and/or future budget information.
The Fall 2016 Capstone team was successful in aggregating budget data into a database and creating an Excel
tool that preformed such analysis at the PE level. Understanding that there is valuable information available at
the more granular project level, the BEAT analysis has the capacity to provide further information and insights,
but currently that capability has not been realized.
Needs Statement There is a need to advance the BEAT system to analyze project level budget data in order to project future
trends for better decision making.
Mission Statement The BEAT will provide a mechanism for capturing and analyzing past, present, and future DoD, Research
Development, Testing & Evaluation (RDT&E) Presidential Budget (PB) submissions. The figure below
displays the concept of operations portraying the several organizations that interact with the BEAT system.
These organizations are detailed in the stakeholders’ section of this paper.
Figure 6: Operational Concept Diagram
Project Objective The objective of this project is to enhance the BEAT to incorporate project level data to facilitate a finer analysis
of budget trends. The enhancements will allow the stakeholders and users to identify and view trends happening
to select projects that support the DoD’s strategy initiatives or other documents. The BEAT will maintain its
current ability to ingest new data as it becomes available and the new project data will be mapped to the
appropriate program data. Additionally, new visualizations and user interface will be developed to provide
additional information and ease the use of tool application.
ID projects of interest within the program elements based on DoD policy on future techs that
support an offset strategy.
Forecast future budget trends of predetermined projects over next 5 years.
Detailed notes included with each project that highlight trends.
Compare a new Program Budget (PB) ingestion to the previous year’s PB to determine if new
program/project/accomplishments have started anew or have ended.
2 The Program Objective Memorandum is the 5-year Future Year Defense Program (FYDP) and presents the Services and Defense
Agencies proposal on how they will balance their allocation of available resources. The POM includes an analysis of missions, objectives, alternative methods to accomplish objectives, and allocation of resources.
5 | P a g e
Stakeholders
Figure 7: Stakeholder Interaction
The President and Congress- The DoD submits an annual budget request to the President for review. The
President then submits a budget request to Congress. This request is reviewed and subjected to a process of
revisions before it is returned to the President, at which point it is either signed into law or vetoed. The
President and Congress have a vested interest in seeing that budget meets policy.
Office of the Secretary of Defense (OSD)- Primary defense policy advisor to the President and responsible
for formulating and executing policy through the DoD. The secretariats within OSD handle and submit the
final DoD budget to the President. The BEAT can potentially support DoD decision analysis.
Services (DoD)- This includes the Services and the S&T community. They receive the tech. and support and
collaborate with multiple stakeholders. They are the end users and the primary recipients of the RDT&E
dollars.
Communities of Interest (COIs)- A network of technical groups that integrate with experience and technical
expertise within DoD. They serve a collaboration mechanism to get interested stakeholders to communicate
on projects in future tech.
Federally Funded Research and Development Centers (FFRDCs), and University Affiliated Research
Centers (UARCs)- Research partnerships in which the DoD is the sponsor of a research facility which is
under the administration of either a corporate entity (FFRDCs) or a University. They provide the backbone of
studies as it relates to RDT&E and application to policy.
Industry- Private entities whose interests and level of involvement with the DoD vary significantly. Some
provide the product or service, prototype development, or test data of a specific or system of systems. There
are also contractors who provide consulting or advising services. This is the military industrial complex that
support the development of the specific idea, component, or end item.
6 | P a g e
Scope As previously mentioned, the Fall 2016 Capstone team led an effort to obtain DoD budgeting data from the
RDT&E appropriation level to the PE level for all services and accounting lines within DoD. The team created
the BEAT to analyze the data using charts and graphs. Additionally, the team scoped the project to include
only three of the seven budget activities: BA 2, 3 and 4. Overall, this initial effort was the first of many future
endeavors to develop a Decision Management System using the BEAT and DoD budgeting data to better
predict future RDT&E investment dollars.
The scope of this project is to continue with the work of moving toward final development of a Decision
Management System for DoD budget analysis. The project team for Spring 2017 Capstone team will focus on
taking the PEs and continuing to add to the database by including all project level data for all PEs that are in
scope. For instance, the PE associated with Advanced Aerospace Systems is a PE that OSD-Policy is interested
in potentially adding future dollars towards in meeting future deterrence requirements and is within BA 2, 3,
and 4.
The team will drill one level down into the project level of Advanced Aerospace Systems and gather all project
The team will also highlight notes and accomplishment level data in the notes which is one level down from
project level data. The figure below provides an illustration of what the scope will entail.
Figure 9: Illustration of Budget Levels
There are natural scoping mechanisms built into the budget nomenclature and taxonomy used to build the
budget documents. The team will use this scoping mechanism to continue work from previous Capstone
Team.
Figure 8: Program Element Example
7 | P a g e
There are several critical assumptions that are required to scope the problem to the needs of the sponsor.
1) The Project Data is readily accessible and does not require infeasible time for download and transfer
to the database.
2) The team will have access to DTIC website and other website for download of data.
3) The team will be able to relate OSD-Policy offset techs/capabilities to PE and follow-on project level
data.
4) The accomplishment level data is available; and is included in the notes.
Requirements Mission Requirements
MR1: The BEAT system shall be able to ingest project level information and data from DoD
documents that summarize PEs.
MR2: The BEAT system shall ingest project level description information for each project.
MR3: The BEAT system shall ingest five future years of expected project level data.
MR4: The BEAT system shall be able to ingest accomplishment level data for specific projects.
MR5: The BEAT system shall visualize RDT&E, specifically project level data, over a pre-determined
period.
MR6: The BEAT system shall be able to display PB new program/project/accomplishments have
started anew or ended.
System Requirements
SR1: The BEAT system shall generate project level data (R-2) for predetermined Program Elements.
SR2: The BEAT system shall generate project level description for Program Elements.
SR3: The BEAT system shall generate accomplishment level data based on specific project level.
SR4: The BEAT system shall aggregate budget data for each Program Element.
SR5: The BEAT system shall display project level data over time.
SR6: The BEAT system shall display new PB ingestion to the previous year’s PB to determine if new
program/project/accomplishments have started anew or have ended.
Functional Requirements
FR1: The BEAT system shall have ability to create a plot that visualizes specific project level data over
time.
FR2: The BEAT system shall have the ability to create a plot that visualizes significant “outliers” of
project level data.
FR3: The BEAT system shall have the ability to create a plot that visualizes accomplishment “outliers”.
FR4: The BEAT system shall monitor a minimum of five RDT&E budget cycles.
FR5: The BEAT system shall aggregate specific project level data.
FR6: The BEAT system shall not consider discontinued PEs in project level analysis.
FR7: The BEAT system shall consider PB data and determine which
program/project/accomplishment data has started new or have completed.
8 | P a g e
Figure 10: Project Requirements Diagram
9 | P a g e
Use Case Diagrams After the first IPR with the sponsor use cases were developed to formalize the viability of the system design
and to meet the agreed upon requirements. The use cases primary purpose is to capture the functional
requirements of the BEAT system; they are a means of specifying the usage of a system. The following two
main use cases capture the interaction of the user with the BEAT. Additionally, and part of behavioral
component of structured analysis, sequence diagrams have been included to represent how the user and actors
interact with the BEAT. The reason for the early introduction of the sequence diagrams is to show how they
are influenced and representative of the use cases. Appendix D provides greater detail for each of the main use
cases.
Use Case 1 displays the BEAT which has the capability to load R2 PE and project level data. The primary actor
of the database is the user with the trigger event being the downloading of data from DTIC website.
Use Case 1: Load R-2 PE and Project Level Data into BEAT
Figure 11: Use Case 1 User Interface Diagram
Schedule: Main Use Case; must be tested before providing to sponsor
Priority: Top, must be executed
Performance Target: All project level, unclassified budget level data is available from FY’11 through FY’20
Frequency: Every release of new Presidential Budget for corresponding FY
The sequence diagram below was developed to support Use Case 1. This diagram ensures the operations were
occurring in an orderly fashion with what made sense during the downloading process and the interactions
between the BEAT and the Access database. After the downloading of data from DTIC website the user
begins to interact with the BEAT. Subsequently, the BEAT then begins communicating with the Access
Interface. The Access Database then completes the storing of PE and project level data and communicates
back to the BEAT through the interface. This downloading process is behind the scenes; the user will only
need to wait a few minutes before all PB data is loaded into BEAT and stored in Access.
10 | P a g e
Figure 12: Use Case 1 Sequence Diagram
Use Case 2 displays the BEAT visualization tool and the interaction with the user and the queries, buttons, and
filters dispersed throughout the tool. The primary actor of the visualization tool is the user with the trigger
event being the user beginning to conduct exploratory analysis. The BEAT visualization has the capability to
interact with PE and project data and provides many unique dashboard features.
11 | P a g e
Use Case 2: Load R-2 PE and Project Level Data into BEAT
Figure 13: Use Case 2 User Interface Diagram
Schedule: Main Use Case; must be tested before providing to sponsor
Priority: Top, must be executed
Performance Target: All project level, unclassified budget level data is available from FY’11 through FY’20
Frequency: Every time a query and filter for specific project level data is executed
The sequence diagram for Use Case 2 describes the dynamics of the user interacting with the many facets of
the BEAT visualization tool. This includes buttons, graphs, displays, and dashboard functions that the user
can manipulate. The sequence diagram begins with the user receiving feedback from BEAT that all R2 PE and
project level data has been saved. The user then opens the visualization tool and begins exploration of either
PE or project level data. The program and project analysis dashboard section of this paper will describe in
detail the many user interfaces that the BEAT visualization tool provides. Below is a visual of the sequence
diagram that supports Use Case 2.
12 | P a g e
Figure 14: Use Case 2 Sequence Diagram
Structural Analysis After agreeing on the mission, operational concept, and primary use cases with the sponsor the team then
focused efforts on the structural analysis (functional decomposition and the functional architecture) of the
BEAT. The structured analysis approach was chosen because this approach best defines what the BEAT must
do. The approach provides a decomposition of the top-level functions, summarizes inputs and outputs, and
provides a road map to the actual upgraded design of the BEAT.
The functional decomposition is the first diagram of the structured analysis. It consists of all high-level
functions that must be accomplished by the BEAT.
13 | P a g e
Functional Decomposition
Figure 15: Functional Decomposition of BEAT
14 | P a g e
Activity Modeling (IDEF0) The IDEF0 model was chosen to represent the activity model due to its ability to best represent the BEAT
inputs, outputs, and controls at each functional decomposition level. The BEAT IDEF0 A-0 diagram details
the activities associated with taking PB data and importing the data into a displays/dashboard that lend itself
to streamlined follow-on analysis. The IDEF0 and subordinate diagrams are at the viewpoint of the user.
Below is the A-0, A0, and A5 (over next three pages) diagrams that supported and summarized the activities
associated with the BEAT. Each diagram builds on the other and is nested with the functional decomposition.
Figure 16: A-0 diagram of BEAT (Highest Level)
15 | P a g e
Figure 17: A0 diagram of BEAT (highlights A1-A5 activities of the BEAT)
Figure 18: A5 diagram of BEAT (highlights A51 through A53 activities of the BEAT)
16 | P a g e
Data Modeling (Entity Relationship Diagram) The ERD primary purpose is to analyze the BEAT data structures and relationships based on the activities from the BEAT IDEF0 diagrams. The BEAT ERD has five primary entities (DTIC website, User, Visualization Tool, Access Interface, and Excel Interface) that interact with the BEAT. There are multiple relationships and attributes associated with each entity. The below picture provides a depiction of the BEAT ERD.
17 | P a g e
Figure 19: BEAT Entity Relationship Diagram (Dataflow Model)
18 | P a g e
Rule Modeling (Decision Tables) The rules model is used to model what BEAT activities/processes in the IDEF0 are going to do. The rule
model for the BEAT depicts what is to be done when a set of conditions evaluates as true and what conditions
lend themselves to be false. To do this, the team decided on using a decision table. The decision table includes
all possible conditions based on the A0 diagram of the BEAT IDEF0. The decision table provided the best
way to formulate and evaluate relationships among the relevant data elements. The decision table was the
easiest and most suited way to specify what actions to take based on the complexities of the BEAT system.
Below highlights each of the Decision Tables that supported the BEAT.
Figure 22: BEAT Decision Table (Load PE and Project Level Data in BEAT – A1)
Figure 20: BEAT Decision Table (Identify and Parse Data – A2)
Figure 21: BEAT Decision Table (Store PE and Project Data – A3)
19 | P a g e
Figure 22: BEAT Decision Table (Display Query Dropdowns – A4)
Figure 23: BEAT Decision Table (Provides Visualization Tool – A5)
20 | P a g e
Technical Approach and Expected Results Because of the work done in the previous semester, there is an existing database architecture in place that stores
the DoD budget information at the PE level. The database is created using the XML files that are available
from the DoD through DTIC which are then parsed via an XML parser. Currently the database only ingests
program level data, however, the project and accomplishment data is also available in the same XML files.
The DoD budget information is available in two different formats R-1 and R-2. The R-2 data contains data
specific to each program and serves as the item justification for each program. That justification also provides
detailed information on the projects within each program. Therefore, our analysis will be most interested in
dissecting the R-2 XML files to retrieve the requisite project level information.
Figure 2: Example Project Level Data
The initial effort of this project will be to expand the current data absorption from only program level data to
include project and accomplishment information. In order to accomplish this, an XML parsing script will be
developed in Visual Basic for Applications (VBA) to absorb the program and project information and store
that data in a MS Access database. Once the data file is updated, the Access database will house all required
information in a structured format to be easily exported into the Excel BEAT.
Once the data has been properly housed, part two of the project will be to expand the capabilities within the
BEAT tool to visualize the latest information. Additional visualizations will be developed and integrated into a
single dashboard to provide a visual representation of the quantitative data, as well as display the qualitative
contextual information that is embedded within the budget data files. VBA will be used to facilitate the data
handling and visualization creation to provide a streamlined user friendly interface to automatically create and
modify the visualizations.
Upon completion of our support, the database will have an updated structure and parsing program to ingest
project level data. That new data will be housed in a structured format that can be easily exported into a csv
file for upload into additional data analysis programs, specifically the BEAT. Additionally, the BEAT tool will
contain new visualizations pertaining to the project level information to provide meaningful insights into trends
within the RDT&E community.
21 | P a g e
BEAT Model Changes
BEAT Version 1.0 (Fall 2016 Capstone Project)
Figure 3: BEAT v1.0 Visualizations
The figure above represents the BEAT system that the previous Capstone team developed based upon the
sponsor needs at that time. In version 1.0, the BEAT data was stored in a MySQL database that was populated
using a JAVA program to parse through the R1 and R2 budget XML to extract the required information. That
data was then queried and incorporated into the BEAT. The BEAT is a MS Excel based tool that allows the
user to select and visualize BAs, Services/Agencies, and PB years to visualize and gain insights. The R1 data
visualization provide many different charts and graphics that provide insights as to how programs stack against
each other in the R1 data. The R2 Data visualization allowed the user to select multiple programs in a single
PB year and compare those programs against each other. Those visualizations answer the question “How does
this program compare against others?” but doesn’t provide much insight about what’s going on within each of
those programs across all available budget submissions. In BEAT v2.0, we aim to answer that question, as well
as provide insights at the more granular project level.
BEAT Version 2.0 BEAT version 2.0 is a redesigned version of BEAT v1.0. After discussions with the sponsor, the database was
moved to MS Access and the XML parsing scripts rewritten in VBA and enhanced to ingest the project level
data in addition to the program data. Additionally, more visualizations accompanied by contextual information
provided in the data files are displayed in a consolidated dashboard, described later.
Visualization
R1 Data R2 Output
22 | P a g e
Figure 4: BEAT v2.0 Methodology
In BEAT version 2.0, the process for updating and loading information starts the same as version 1.0, by
downloading the PB data from DTIC. From there, the user can open the BEAT and select “Upload new R2
files” from the homepage menu. That process will then open the linked Access Database in the background
and launch the XML parsing algorithm. From there the user is prompted to select the folder containing the
XML files, and the parsing commences. Once completed the BEAT will then update queries and tables within
the tool that are linked to the database to ensure the most up to date information is available. The user is then
free to explore and gain insights to hundreds of programs and projects across all available budget submissions.
Database Update In BEAT v1.0, the PB data was stored in a MySQL database that was populated using a JAVA program that
parsed through the budget XML files. The new, improved database was moved to MS Access and the XML
parsing program was implemented in Visual Basic for Applications (VBA). That process moved the data
gathering process “under one roof” and allowed the BEAT (implemented in MS Excel) to more seamlessly
integrate with another MS Office product.
To extract the data from the XML files, the VBA script relies on the tree structure inherent to XML file types.
The script identifies the “root” program nodes, and then cycles through each child node passing the name of
the child node through a switch (or select) statement to identify and place the data value to the appropriate field
in Access. When the Project node is identified, it passes that sub tree structure through a similar process.
The Access database contains two tables, one for the PE data and the other for the Project Level data named
“Program Data” and “Project Data”, respectively. The composite key for the Program Data table are the
Download PB data from DTIC (Manual Process)
Open BEAT and Select “Upload new R2 files”
BEAT (Excel) Opens Access and runs the XML parsing Code
Excel Sheets Update Existing Queries and Connections
Visualization
Visualization Tool within BEAT that allows user to graphically display data based on inputs
Input Cells for scoping data
User conducts
exploratory
analysis and
informs
management Descriptive Information
23 | P a g e
Program Numbers and Budget Years. The Project Data composite key consists of the Program Number,
Project Number, and Budget Year. The tables are linked through the program number to allow for specific
SQL scripting.
The BEAT has been linked to the Access Database and two new worksheets have been created to query the
data each time the workbook is opened, or the database imports new data. Currently, the data set goes back to
2011 and contains 4366 Program records across the DoD comprising of 9388 Project Records
Figure 5: MS Access Data Tables
24 | P a g e
Figure 6: XML Program Element Tree Example
R2 Exhbit List
(Program List)
Program Element Number
Program Element Title
Budget YearService Agency
NameAppropriation
CodeAppropriation
Number
Program Element Funding
Prior Year
Current Year
Budget Year One
Budget Year Two
Budget Year Three
Budget Year Four
Budget Year Five
Project List
25 | P a g e
Figure 7: XML Project Element Tree Example
Project list
Project Number
ProjectTitle Poject Funding
Prior Year
Current Year
Budget Year One
Budget Year Two
Budget Year Three
Budget Year Four
Budget Year Five
Project NoteProject Mission
Description
26 | P a g e
BEAT Visualization (Program and Project Analysis Dashboard) The BEAT Program and Project Analysis Dashboards allows the user to visually analyze a specific program’s or project’s funding trends and relate those
variations to the notes from the budgets in a consolidated environment.
Figure 8: Program Analysis Dashboard
27 | P a g e
Program Analysis Dashboard
Program Selection The user can narrow the organizations and programs by selecting a (or all) BA of interest. From there, the user
selects among the organizations and agencies that have programs under the selected BA. The “Select Program”
options are then updated and the user can select a program belonging to the organization or agency by name
of Program Number.
Figure 9: Program Selection Fields
Additionally, the user can search for a program using the “Program Keyword Search” function. Through that
function, the user can filter all available programs by Service/Agency, BA (similar to the dashboard), or utilize
Keyword Searches that will allow the user to search by a specific word or phrase to aid in identifying the
program of interest. Once a program has been identified, the user can select that program and the Program
Analysis Dashboard will update to the selected program.
Figure 10: Program Search User Form
28 | P a g e
Contextual Program Information Once a program is selected, information about the program selected is populated with information from the
R-2 XML files that is housed in the Access database. That information includes program description,
earliest/latest PB years available, and program notes per PB year. Additionally, there is a button below the
Program Description to select the current program for analysis of its projects, described later.
Figure 11: Program Information
Program Visualizations and Statistics The top visualization included in the dashboard is a line chart that plots funding of the single program across
the various PB years available in the database. This visualization provides the user a visual representation of the
changes in funding across the budgets. The example below demonstrates a relatively steady funding and
projection stream, illustrated by the funding lines falling close, if not on top of one another. A program with
an unstable funding stream would have funding lines that do not fall relatively close to one another and have
many increases and decreases in funding that are not consistent across PBs. Included at the bottom of the
chart is the data that created the chart in a table to provide values for data to provide further information.
Figure 12: Program Budget Chart
29 | P a g e
Following the Program Budget Chart are basic summary statistics that have been generated on the available
data set. The summary statistics provide metrics describing the behavior of each FYs budget projections. The
statistics include Mean, Standard Deviation, Variance, Min, Median, Max, and Range. A stable budget will have
minimal Standard Deviations and Variances between budget submissions. Note that these statistics are not
generated across a single budget year to examine the stability or changes in a single PB’s projections. Rather,
these statistics are generated for a single FY across all the PBs that have an actual or projected value for that
FY.
Figure 13: Program Summary Statistics
The final visualization is a heat map that highlights the various deltas between PB projections for a FY and
their actual from that FY’s PB. Each PB contains the prior FY’s budget (FY-1), the current FY, and 5 out years
(FY+1-5), tied to the POM. For example, in the heat map below, the first value for column FY_2014 was
calculated as the absolute value of the difference between PB 2012 and PB 2013 projections for FY 2014.
Figure 14: Program Budget Heat Map
Given that there are only FY-1 to FY+5 years of data available in each PB, it is clearly not possible to calculate
deltas for all FY for each pair of PBs. The “#N/A” is used to indicate those FYs where there is no delta to
calculate between the selected PBs. For instance, in PB 2014, we have a prior year data point in FY 2013.
However, in PB 2015, there is no value for FY13, resulting in an #N/A for the delta between PB14 and PB 15
for FY_2013.
Project Analysis Dashboard The Project Analysis allows the user to investigate the different projects that belong to each program. These
insights provide more granular detail of how budgets are behaving within each program.
30 | P a g e
Project Selection The down selecting process follows the same as the Program Analysis Dashboard, but adds a Project Selection
by Project Name or Project Number. Similar to the Program Analysis Dashboard, there is a “Project Search”
function where the user can filter all available Projects by Service/Agency, Budget Activity, or search by
Program and/or Project key words and phrases.
Figure 15: Project Selection Fields
Figure 16: Project Search User Form
Contextual Project Information and Visualizations The Project Mission (analogous to the Program Description), Earliest and Latest PB Data Years, and Project
Notes are all imported for the select project, similar to the Program Analysis Dashboard. Additionally, the tool
provides the funding chart and heat map to highlight the deltas in funding using the same methodology outlined
for the program funding chart and heat map.
31 | P a g e
Figure 17: Project Analysis Dashboard
Additional Project Visualization
The Project Analysis Dashboard does not have the Summary Statistics that were provided at the program level.
Instead, a pie chart that breaks down the most recent budget year’s current year funding for each project across
the parent program was included to provide context as to how the project in question stacks up against the
other projects competing for funding in that program.
Figure 18: Project Analysis Pie Chart
32 | P a g e
Reporting Functions To facilitate the identification of programs or projects of interest, two reporting functions were included to
flag programs and projects that meet certain criteria, the Delta Analysis and New/Recently Ended Program
and Project Identification.
Delta Analysis The Delta Analysis examines the FY+1 projections and FY actuals of all programs and projects to identify
those that have differences in excess of a user defined budget variation percentage. In other words, any program
or project whose projection was more (or less) that X% of the actual using the formula:
|𝑃𝑟𝑜𝑗𝑒𝑐𝑡𝑖𝑜𝑛 − 𝐴𝑐𝑡𝑢𝑎𝑙|
𝐴𝑐𝑡𝑢𝑎𝑙
Those programs and projects that exceed the delta threshold are logged in a report that can be reviewed and
then explored further utilizing the Program and Project Analysis Dashboards.
Figure 19: Delta Analysis Reporting
Organization/Agency Program Number Program Name Year Projection Acutal Delta
Air Force 0101125F Nuclear Weapons Modernization 2015 67.874 33 -51.38%
Air Force 0105921F Service Support to STRATCOM - Space Activities2017 0 8.09 Inf
Air Force 0203761F Warfighter Rapid Acquisition Process (WRAP) Rapid Transition Fund2015 3.844 0 -100.00%
Air Force 0207100F LAAR Squadrons 2013 23.721 13.721 -42.16%
Air Force 0207131F A-10 Squadrons 2017 0 16.2 Inf
Air Force 0207133F F-16 Squadrons 2015 177.298 112.667 -36.45%
Air Force 0207142F F-35 Squadrons 2015 33 3 -90.91%
Air Force 0207142F F-35 Squadrons 2017 115.395 53.921 -53.27%
Air Force 0207171F F-15 EPAWSS 2016 68.944 0 -100.00%
Air Force 0207227F Combat Rescue - Pararescue 2015 0.542 0.35 -35.42%
Air Force 0207249F Precision Attack Systems Procurement2015 1.075 2 86.05%
Air Force 0207268F Aircraft Engine Component Improvement Program2015 139.369 89.369 -35.88%
Air Force 0207410F Air & Space Operations Center (AOC)2016 41.066 25.662 -37.51%
Air Force 0207412F Control and Reporting Center (CRC) 2016 0.552 0 -100.00%
Air Force 0207455F Three Dimensional Long-Range Radar (3DELRR)2017 14.939 8.139 -45.52%
Air Force 0207581F Joint Surveillance/Target Attack Radar System (JSTARS)2015 13.248 23.148 74.73%
Air Force 0208059F CYBER COMMAND ACTIVITIES 2015 68.099 38.099 -44.05%
Air Force 0303131F Minimum Essential Emergency Communications Network (MEECN)2015 36.288 18.481 -49.07%
Progam Projection vs Acutal Deltas greater than 35%
33 | P a g e
New or Recently Ended Program/Project Identification The BEAT will also parse through both program and project level data to identify programs/projects who’s
PB is new within the last three years or no longer have data in the most recent PB. Note that the “beginning”
or “ending” of a program/project can be difficult to automatically define and flag. For example, a single PB’s
projections may go to zero for consecutive years but in a subsequent PB we may see that program/project
regain a funding stream. Additionally, programs/projects may have a “Cost to Complete”, but get canceled
prior to reaching that cost. It has also been observed that programs/projects may get rolled into others, and
seemingly disappear from the data set. It is only with careful observation and human investigation that a
program/project’s start or end can be identified with certainty.
Understanding those limitations, this report is generated under the following criteria:
“New” Programs/Projects
a. The first budget year within the dataset for a program/project (identified by program/project
number) is greater than or equal 3 years prior to the current year (as defined by the user)
“Ended” Programs/Projects
a. The final budget year within the data set for a program/project (identified by its
program/project number) is less than the current year (as defined by the user)
Using these criteria, a new worksheet is created that lists all the programs/projects that violate those criteria for
further inspection.
Figure 20: “New" Program/Project Report
Figure 21: "Ended" Programs/Projects
*ASSUMPTION: A program/Project is flagged as "Started" or "New" if there is not prior year data available
Organization/Agency Program Number Program Name Organization/Agency Program Number Program Name Project Number Project Name
Air Force 0101122F Air-Launched Cruise Missile (ALCM) Air Force 0101113F B-52 SQUADRONS 675041 Bomber Tactical Data Link
Air Force 0101126F B-1B Squadrons Air Force 0101113F B-52 SQUADRONS 675048 1760 INTERNAL WEAPONS BAY UPGRADE (IWBU)
Air Force 0101127F B-2 SQUADRONS Air Force 0101113F B-52 SQUADRONS 675049 MODE S/5 IFF
Air Force 0101213F Minuteman Squadrons Air Force 0101113F B-52 SQUADRONS 675050 CONECT
Air Force 0101313F Strat War Planning System - USSTRATCOM Air Force 0101113F B-52 SQUADRONS 675051 ANTI-SKID
Air Force 0101314F Night Fist - USSTRATCOM Air Force 0101113F B-52 SQUADRONS 675055 GPS-IU
Air Force 0101316F Worldwide Joint Strategic Communications Air Force 0101113F B-52 SQUADRONS 675056 B-52 Radar Modernization Program (RMP)
Air Force 0102110F UH-1N Replacement Program Air Force 0101113F B-52 SQUADRONS 675057 B-52 Low Cost Improvement (LCI)
Air Force 0102326F Region/Sector Operation Control Center Modernization Program Air Force 0101113F B-52 SQUADRONS 675058 B-52 Weapon Sys Trainer Air Ref Training Upgrade
Air Force 0102823F Strategic Aerospace Intelligence System Activities Air Force 0101113F B-52 SQUADRONS 675160 B-52 Crypto Modernization
Air Force 0105921F Service Support to STRATCOM - Space Activities Air Force 0101122F Air-Launched Cruise Missile (ALCM)674797 Flight Testing & Navigation Enhancement
Air Force 0201184F Counter Narcotics Technology Program Office Air Force 0101126F B-1B Squadrons 675344 B-1B Modernization
Air Force 0203761F Warfighter Rapid Acquisition Process (WRAP) Rapid Transition Fund Air Force 0101127F B-2 SQUADRONS 675345 B-2 Modernization
Air Force 0205219F MQ-9 UAV Air Force 0101127F B-2 SQUADRONS 676021 BASELINE SUPPORT
Air Force 0205671F Joint Counter RCIED Electronic Warfare Air Force 0101127F B-2 SQUADRONS 676022 EHF SATCOM and Computer
Programs Started in Last 3 years Projects Started in Last 3 years
Organization/Agency Program Number Program Name Last Data Year Organization/Agency Program Number Program Name Program Number Program Name Last Data Year
Air Force 0101113F B-52 SQUADRONS 2017 Air Force 0101113F B-52 SQUADRONS 675039 B-52 System Improvements 2017
Air Force 0101122F Air-Launched Cruise Missile (ALCM)2017 Air Force 0101113F B-52 SQUADRONS 675041 Bomber Tactical Data Link 2017
Air Force 0101125F Nuclear Weapons Modernization 2017 Air Force 0101113F B-52 SQUADRONS 675048 1760 INTERNAL WEAPONS BAY UPGRADE (IWBU)2017
Air Force 0101126F B-1B Squadrons 2017 Air Force 0101113F B-52 SQUADRONS 675049 MODE S/5 IFF 2015
Air Force 0101127F B-2 SQUADRONS 2017 Air Force 0101113F B-52 SQUADRONS 675050 CONECT 2017
Air Force 0101213F Minuteman Squadrons 2017 Air Force 0101113F B-52 SQUADRONS 675051 ANTI-SKID 2015
Air Force 0101313F Strat War Planning System - USSTRATCOM2017 Air Force 0101113F B-52 SQUADRONS 675055 GPS-IU 2017
Air Force 0101314F Night Fist - USSTRATCOM 2017 Air Force 0101113F B-52 SQUADRONS 675056 B-52 Radar Modernization Program (RMP)2017
Air Force 0101316F Worldwide Joint Strategic Communications2017 Air Force 0101113F B-52 SQUADRONS 675057 B-52 Low Cost Improvement (LCI)2017
Air Force 0102110F UH-1N Replacement Program 2017 Air Force 0101113F B-52 SQUADRONS 675058 B-52 Weapon Sys Trainer Air Ref Training Upgrade2017
Air Force 0102326F Region/Sector Operation Control Center Modernization Program2017 Air Force 0101113F B-52 SQUADRONS 675160 B-52 Crypto Modernization 2017
Air Force 0102823F Strategic Aerospace Intelligence System Activities2014 Air Force 0101122F Air-Launched Cruise Missile (ALCM)674797 ALCM Upgrades 2017
Air Force 0105921F Service Support to STRATCOM - Space Activities2017 Air Force 0101125F Nuclear Weapons Modernization657007 B61 LIFE EXTENSION PROGRAM 2017
Former Programs Not Included in Current FY Former Projects Not Included in Current FY
34 | P a g e
Third Offset Strategy Technological superiority has been a staple of the DoD strategy since post WWII. This philosophy and follow-
on investment has allowed U.S. to outpace its peers and remain at the forefront of battlefield innovation.
However, since the current budget drawdown and the military mission shift to counterinsurgency (COIN), it is
no longer safe to assume technological superiority. The 3rd offset strategy remained a top DoD priority in 2016.
The 3rd offset strategy is being pursued to counter great peer competitors like China and Russia. Its focus is
on continuing U.S. military technological superiority. Released publicly, the President’s FY2017 budget
included over $18 billion over the course of the FYDP for 3rd Offset Related investments. Some of the focus
areas of this $18 billion are on anti-access technology, undersea warfare capability, human-machine
collaboration or unmanned and manned systems, and asymmetric capabilities (Cyber, Space, EW, and Laser).
Third Offset Strategy (if maintained) provides significant opportunities for future RDT&E investment- should
observe trends of increased investments in these capabilities. The picture below shows the RDT&E budget
authority by service during the counter-insurgency fight in Afghanistan and Iraq and the follow-on military
drawdown.
The below two charts provide an example of how the sponsor can utilize the BEAT to identify areas of interest
in RDT&E investment. This is the part where the sponsor or user begins to match policy with trend
development found in the BEAT. With the DII of 2014, and focus on a 3rd Offset Strategy, there has been
renewed emphasis in DoD investment in technology. The first chart, High Energy Laser, is a 3rd Offset
investment in laser energy as a means for force protection. Much emphasis has been placed on the need to
counter rocket, artillery, and mortar munitions along with interdicting UAVs. This chart shows what will
happen to an existing project (Army) that has an uptick in spending due policy changes and refocus on technical
innovation. Trends like this could be identified by the sponsor by matching 3rd Offset policy and capabilities
with invigorated spending.
35 | P a g e
U.S. Army – High Energy Laser (PE: Advanced Weapons Technology)
This second chart displays a capability that did not exist in any past PB. This capability is tied directly to Air
Force investments in asymmetric capabilities with the 3rd Offset initiative. This capability focuses heavily on
autonomous learning, which is a staple of future technology innovation.
U.S. Airforce – Cyber Technology Pattern Recognition and Autonomous Learning (PE:
Aerospace Sensors)
Overall, the BEAT provides a unique, first-of-its-kind way to visualize PE and project level data. It will allow
the user the opportunity to start matching policy changes to budgetary changes. Special emphasis should focus
on the mapping of capabilities and technologies that support the 2014 Defense Innovation Initiative and the
3rd Offset Strategy.
Existing Project with Uptick in future spending
New Project
36 | P a g e
Conclusion In conclusion, the BEAT version 2.0 provides the user with a Decision Management Support Tool that
allows for exploratory analysis of RDT&E PE and project level data. The BEAT can present data in minutes,
run surface level analytics by displaying requested query information in a user-friendly way to identify
trends/changes, and provide a visual and statistics of beginning/ending PE/project funding. The BEAT
Version 2.0 has been created in such a way that SPA and future users can import future data and immediately
start using the tool. Overall, the team met all deliverables and has provided a way forward in moving the
project from Mile Marker 2.5 to Mile Marker 5.0 (see Figure below).
Way Forward Recommendations The BEAT and the analysis it provides is an ongoing endeavor to provide further analysis to the sponsor and
other stakeholders. Future recommendations for development of the BEAT include:
Retrieve and house accomplishment data in the database
Improve the database by creating more tables and linking them appropriately
Explore and incorporate predictive analytics tools such as Linear Regressions, Support Vector
Machines, K-Nearest Neighbors, etc. to better analyze data (may require more advanced analytical
software packages/programming languages e.g. Julia, Python, R, SAS)
Explore and incorporate textual analytics on the program/project descriptions and notes to
investigate correlations between keywords/phrases and budget projections (may require more
advanced analytical software packages/programming languages e.g. Julia, Python, R, SAS)
Continue to link policy (National Military Strategy, Defense Innovation Initiative) to budgetary
trend analysis using BEAT (Significant opportunity for future RDT&E investment if current
administration and DoD continues to invest in a Third Offset Strategy)
Upgrades to BEAT:
o Heat Chart with conditional formatting based on relative changes
o Link each program to its page in the corresponding budget file
Figure 22: BEAT Roadmap
I | P a g e
Appendix A- References American Association for the Advancement of Science 2016 [3] - Govini. No Date.
DoD Financial Management Regulation Volume 2B, Chapter 5
Department of Defense, Future Years Defense Program (FYDP) Structure, April 2014 at
http://www.prim.osd.mil/Documents/DoD_704507_future_years_defense.pdf
Department of Defense (DoD) Releases Fiscal Year 2017 President’s Budget Proposal
Defense Technical Information Center (DTIC) – www.dtic.mil
GMUSEORSPA at https://gmuseorspa.wordpress.com/deliverables/
Military budget of the United States, Wikipedia
U.S. House of Representatives, Committee on the Budget at http://budget.house.gov/budgetprocess/
"US Department of Defense Third Offset Standard Market Taxonomy." Analyst Report, Washington, D.C.
II | P a g e
Appendix B- Definitions Budget Activity 1, Basic Research Basic research is systematic study directed toward greater knowledge or understanding of the fundamental
aspects of phenomena and of observable facts without specific applications towards processes or products in
mind. It includes all scientific study and experimentation directed toward increasing fundamental knowledge
and understanding in those fields of the physical, engineering, environmental, and life sciences related to long-
term national security needs. It is farsighted high payoff research that provides the basis for technological
progress. Basic research may lead to: (a) subsequent applied research and advanced technology developments
in Defense-related technologies, and (b) new and improved military functional capabilities in areas such as
communications, detection, tracking, surveillance, propulsion, mobility, guidance and control, navigation,
energy conversion, materials and structures, and personnel support. Program elements in this category involve
pre-Milestone A efforts
Budget Activity 2, Applied Research Applied research is systematic study to understand the means to meet a recognized and specific national security
requirement. It is a systematic application of knowledge to develop useful materials, devices, and systems or
methods. It may include design, development, and improvement of prototypes and new processes to meet
general mission area requirements. Applied research translates promising basic research into solutions for
broadly defined military needs, short of system development. This type of effort may vary from systematic
mission-directed research beyond that in Budget Activity 1 to sophisticated breadboard hardware, study,
programming and planning efforts that establish the initial feasibility and practicality of proposed solutions to
technological challenges. It includes studies, investigations, and non-system specific technology efforts. The
dominant characteristic is that applied research is directed toward general military needs with a view toward
developing and evaluating the feasibility and practicality of proposed solutions and determining their
parameters. Applied Research precedes system specific research. Program control of the Applied Research
program element is normally exercised by general level of effort. Program elements in this category involve pre-
Milestone B efforts, also known as Concept and Technology Development phase tasks, such as concept
exploration efforts and paper studies of alternative concepts for meeting a mission need.
Budget Activity 3, Advanced Technology Development (ATD) This budget activity includes development of subsystems and components and efforts to integrate subsystems
and components into system prototypes for field experiments and/or tests in a simulated environment. ATD
includes concept and technology demonstrations of components and subsystems or system models. The
models may be form, fit and function prototypes or scaled models that serve the same demonstration purpose.
The results of this type of effort are proof of technological feasibility and assessment of subsystem and
component operability and producibility rather than the development of hardware for service use. Projects in
this category have a direct relevance to identified military needs. Advanced Technology Development
demonstrates the general military utility or cost reduction potential of technology when applied to different
types of military equipment or techniques. Program elements in this category involve pre-Milestone B efforts,
such as system concept demonstration, joint and Service-specific experiments or Technology Demonstrations.
Projects in this category do not necessarily lead to subsequent development or procurement phases.
III | P a g e
Budget Activity 4, Advanced Component Development and Prototypes (ACD&P) Efforts necessary to evaluate integrated technologies, representative modes or prototype systems in a high
fidelity and realistic operating environment are funded in this budget activity. The ACD&P phase includes
system specific efforts that help expedite technology transition from the laboratory to operational use.
Emphasis is on proving component and subsystem maturity prior to integration in major and complex systems
and may involve risk reduction initiatives. Program elements in this category involve efforts prior to Milestone
B and are referred to as advanced component development activities and include technology demonstrations.
Completion of Technology Readiness Levels 6 and 7 should be achieved for major programs. Program control
is exercised at the program and project level. A logical progression of program phases and development and/or
production funding must be evident in the FYDP.
Budget Activity 5, System Development and Demonstration (SDD) SDD programs have passed Milestone B approval and are conducting engineering and manufacturing
development tasks aimed at meeting validated requirements prior to full-rate production. This budget activity
is characterized by major line item projects and program control is exercised by review of individual programs
and projects. Prototype performance is near or at planned operational system levels. Characteristics of this
budget activity involve mature system development, integration and demonstration to support Milestone C
decisions, and conducting live fire test and evaluation (LFT&E) and initial operational test and evaluation
(IOT&E) of production representative articles. A logical progression of program phases and development and
production funding must be evident in the FYDP consistent with the Department's full funding policy.
Budget Activity 6, RDT&E Management Support
This budget activity includes research, development, test and evaluation efforts and funds to sustain and/or
modernize the installations or operations required for general research, development, test and evaluation. Test
ranges, military construction, maintenance support of laboratories, operation and maintenance of test aircraft
and ships, and studies and analyses in support of the RDT&E program are funded in this budget activity. Costs
of laboratory personnel, either in-house or contractor operated, would be assigned to appropriate projects or
as a line item in the Basic Research, Applied Research, or Advanced Technology Development program areas,
as appropriate. Military construction costs directly related to major development programs are included.
Budget Activity 7, Operational System Development
This budget activity includes development efforts to upgrade systems that have been fielded or have received
approval for full rate production and anticipate production funding in the current or subsequent fiscal year. All
items are major line item projects that appear as RDT&E Costs of Weapon System Elements in other programs.
Program control is exercised by review of individual projects. Programs in this category involve systems that
have received Milestone C approval. A logical progression of program phases and development and production
funding must be evident in the FYDP, consistent with the Department's full funding policy.
IV | P a g e
Appendix C- Acronyms
ACD&P Advance Component Development & Prototypes
ATD Advance Technology Development
BEAT Budget Exhibit Analysis Tool
COIN Counter Insurgency
CPI Cost Performance Index
DII Defense Innovation Initiative
DoD Department of Defense
DTIC Defense Technical Information Center
ERD Entity Relationship Diagram
EVM Earned Value Management
EW Electronic Warfare
FYDP Future Year Defense Program
GMU George Mason University
IPR In Process Review
JTAAD Joint Airforce and Analysis Division
LFT&E Live Fire Test & Evaluation
MR Mission Requirement
OSAG Operations Support Analysis Group
OSD Office of Secretary of Defense
PB Program Budget
PE Program Element
POM Program Objective Memorandum
RDT&E Research, Development, Testing & Evaluation
SDD System Development and Demonstration
SOW Statement of Work
SPA Systems Planning and Analysis, Inc.
SPI Schedule Performance Index
SQL Structured Query Language
SR System Requirement
S&T Science & Technology
UAV Unmanned Aerial Vehicle
VBA Visual Basic for Applications
WBS Work Breakdown Structure
XML eXtensible Markup Language
V | P a g e
Appendix D- Use Cases in Detail This appendix provides a more descriptive nature to each Use Case.
Use Case 1: Load R-2 PE and Project Level Data into BEAT 1. BEAT Characteristic Information (Use Case 1)
Goal in Context: Project level data from R-2 files downloaded into BEAT
Scope: Access based Database, XML VBA, Excel
Level: Primary
Pre-Condition: R-2 project level data in XML available from DTIC website
Success End Condition: Project level data loaded into BEAT
Minimal Guarantees: BEAT has PE and project level data and is ready for analysis
Primary Actor: User of the database
Supplementary Actors: BEAT, Access Interface, Access Database, XML VBA Parse
Trigger Event: R-2 (PE and project level data) downloaded from DTIC website
2. BEAT Main Success Story (Use Case 1)
VI | P a g e
Include Use Case 1
3. BEAT Scenario Extensions (Use Case 1)
4. BEAT Related Information (Use Case 1)
Schedule: Main Use Case; must be tested before providing to sponsor
Priority: Top, must be executed
Performance Target: All project level, unclassified budget level data is available from FY’11
through FY’20
Frequency: Every time a download is required due to release of new Presidential Budget for
corresponding FY
Super Use Case: This is the parent use case
Sub Use Case: N/A
Channel To Primary Actor: Reliant on R-2 project level data download without error
Secondary Actors: Access Database
Load R-2 PE and Project Level Data into BEAT Use Case Diagram for Use Case 1
System Boundary
VII | P a g e
Use Case 2: Perform Visualization of Project Level Data in BEAT 1. BEAT Characteristic Information (Use Case 2)
Goal in Context: A manageable tool available for user to visualize, graph, and scope project level
data
Scope: The BEAT Visualization has the capability to interact with R-1/R-2 (down to project level)
data
Level: Primary or User-Goal
Pre-Condition: Visualization tool built with VBA (In Excel)
Success End Condition: User able to graph data using a number of visualization options.
Minimal Guarantees: The BEAT Visualization code updated, the Access database has all current
R1 and R2 data, all project level data and associations up to date
Primary Actor: User of the data
Supplementary Actors: BEAT Visualization Tool, Access Database
Trigger Event: User requests to visualize project level data
2. BEAT Main Success Story (Use Case 2)
Include Use Case 2
VIII | P a g e
3. BEAT Scenario Extensions (Use Case 2)
4. BEAT Related Information (Use Case 2)
Schedule: Main Use Case; must be tested before providing to sponsor.
Priority: Top, must be executed
Performance Target: All project level, unclassified budget level data is available from FY’11
through FY’20
Frequency: Every time a query and filter for specific project level data is executed
Super Use Case: This is the parent use case
Sub Use Case: N/A
Channel To Primary Actor: Reliant on project-level data stored in Access database and read by
Excel VBA
Secondary Actors: Access Database, BEAT Visualization Tool
Perform Visualization of Project Level Data Use Case Diagram for Use Case 2
System Boundary
IX | P a g e
Appendix E- Deliverables The team is currently scheduling a minimum of three customers+ facing meetings to review progress that is
being made. The team will provide SPA with a written report and brief the results of the research and analysis
obtained. The team will continue to refine and develop BEAT to fit the newly derived needs of SPA. The team
will also update the BEAT user manual. Each deliverable will identify future budget trends that are nested with
the Office of Secretary of Defense’s offset strategy more specifically by identifying specific accomplishments
of interest within applicable projects.
The project team provided the following deliverables:
1) BEAT continued refinement and project level data uploaded
2) BEAT user interface improvement with ability to visualize new information
3) BEAT user manual updated
4) BEAT example using policy (3rd offset examples)
5) Written report and briefing summarizing results of the analysis
6) Websites with all products https://stbudgetgmuspring17.wixsite.com/gmuproject
7) Listing of all sources
8) Visualization Tool that identifies when projects begin and end
9) Identification of future budget trends of projects that are nested with Office of Secretary of Defense’s
offset strategy (this deliverable is still on-going)
X | P a g e
Appendix F- Program Management The team used Microsoft Project as the tool when developing the Work Breakdown Structure (WBS) and
project schedule. The final WBS was broken into seven high level phases; Analysis, Design, Implementation,
Verification, Final Deliverables, Management, and Research. The project schedule was broken up into the seven
high level phases as shown in the WBS and then further defined into smaller tasks. This project was estimated
to take a total of 75 days to completion starting from January 30, 2017 to May 12, 2017. This project took into
the systems engineering lifecycle methodology as tasks ranged from problem formulation to architecture design
and tool verification and validation.
S&T Budget Project 1.0
Analysis 1.1
Define Problem 1.1.1
Deliverable: Preliminary Problem
Definition 1.1.2
Define Problem Scope 1.1.3
Define Problem Overview
1.1.4
Initial Sponsor Meeting
1.1.5
Design 1.2
Requirements Gathering
1.2.1
Data Gathering 1.2.2
Develop Tool Methodology
1.2.3
Deliverable: Project Proposal
1.2.4
Sponsor Meeting (IPR#1)
1.2.5
Develop Use Cases 1.2.6
Develop Engineering Diagrams
1.2.7
Implementation 1.3
Database Setup 1.3.1
Database Code 1.3.2
Data Analysis 1.3.3
Sponsor Meeting (IPR#2)
1.3.4
Deliverable: Progress Report
1.3.5
Verification 1.4
Test Data Integrity 1.4.1
Test User Interface 1.4.2
QA Testing 1.4.3
Sponsor Meeting (IPR#3)
1.4.4
Final Deliverables 1.5
Deliver Final Report 1.5.1
Conduct Final Presentation
1.5.2
Produce Website 1.5.3
Deliver Final Tool Product
1.5.4
Management 1.6
Research 1.7
Figure 23: Work Breakdown Structure (WBS)
XI | P a g e
Figure 24: Project Schedule
XII | P a g e
The project budget and schedule was calculated using the Earned Value Management (EVM) method. The project length was a total of 15 weeks and
each of the three-team members earned rate of $85.11 after a 2% overhead rate was added. The planned budget was calculated at a 30-hour work week.
The team decided that the best approach was to follow the 50/50 method where 50% of the budget would be taken at the start of each of the seven high
level phases derived from the WBS and once that phase was completed the other 50% was granted. Upon completion of the 15 weeks the teams remained
under budget with a cost performance index (CPI) of 1.01. The team remained on schedule meaning all tasks were completed by the May 12, 2017 deadline
resulting in a schedule performance index (SPI) of 1.00. Even though the CPI and SPI fluctuated throughout the 15 weeks the team managed to stay
under budget and remain within schedule producing a decision support that the SPA sponsor was pleased with.
Table 1: Earned Value Management (EVM) Breakdown
XIII | P a g e
0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
1.60
1.80
Week 1Week 2Week 3Week 4Week 5Week 6Week 7Week 8Week 9 Week10
Week11
Week12
Week13
Week14
Week15
Rati
o
CPI and SPI
CPI
SPI
$-
$5,000.00
$10,000.00
$15,000.00
$20,000.00
$25,000.00
$30,000.00
$35,000.00
$40,000.00
$45,000.00
Do
llars
Earned Value
Earned Value
Planned Value
Actual Cost
Figure 25: CPI and SPI Graph
Figure 2625: EVM Graph
XIV | P a g e
Appendix G- Recognition The team wanted to thank the following for their support throughout this project:
Slider Griese (SPA)
Brandon Davis (SPA)
Chris Anderson (SPA)
Andy Dichter (SPA)
Matt Hooker (SPA)
Luke Piepkorn (SPA)
Alex Park (SPA)
Dr. Kathryn Laskey (GMU)
SYST 699 Classmates (GMU)