I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Headquarters U.S. Air Force

1

Progress and Challenges in Performance Based Budgeting;

an Air Force Perspective

Mr. Mark DanburgMr. Matt McGuire

SAF/FMBMA

I n t e g r i t y - S e r v i c e - E x c e l l e n c e 2

Agenda

Emphasis on Performance Performance-Driven Organizations Evolution of the Air Force Performance Budget

Progress to Date Meeting the Challenges for Continued Progress

Examples of a Performance Budget Methodology Questions Back Up Slides

Analysis of Performance

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Emphasis on Performance

3

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Performance Budgeting Guidance/Legislation

4

1990 Chief Financial Officers Act 1993 Government Performance Results Act 1993 National Performance Review 2002 President’s Management Agenda, MID 901, 910, 913

2004 OMB Circular A11 - Performance Budget Meets GPRA Requirements

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

OMB Circular A-11 required agencies to develop and report on performance based budgets

Key terms/phrases of interest: Clearly explains Relationship Strategic goals Outcome-oriented long-term and annual performance goals

…measures that show results

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Increased Focus on Performance fromCurrent Administration

“It is time to pay far more attention to the use of Federal performance information as a powerful performance-improving tool:1) useful for communicating priorities, progress, and raising issues; 2) for illuminating what works and should be continued and what does not work that needs attention; 3) for allocating scarce resources wisely.”

Oral testimony of Jeffrey Zients, Oct 2009Deputy Director Management Office of Management and Budget

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Performance-Driven Organizations

7

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Attributes

Establish a Leadership Tone at the Top Commit to rewarding leaders who produce results

Build an Analytics-Based Mindset Reward production of relevant outcome-based performance measures,

reliable performance information, and a disciplined approach to monitoring, evaluation and reporting on the program (both internal and external)

Develop Relevant Performance Goals and Supporting Measures Determine achievement of anticipated outcomes and contribution to the

success of the mission Performance measurement systems should provide intelligence for decision-

makers, not just compile data Build an Analytics Capability

Establish best practices in the performance lifecycle processes Leverage and enhance organizational structure, policies, staff competencies,

staff motivators, and enabling technology tools Establish/enhance integrated decision support and information reporting

processes

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Elements Required forPerformance-Based Decision Making

Relevant Information (e.g., appropriate performance measures) Reliable Information (e.g., timeliness, completeness, and data

accuracy) Strong Analytic Capabilities Timely Decisions and Cost-Effective Implementation External Reporting Transparency A Spirit of Accountability

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Strategic Planning Linkage withPerformance Budgeting

10

Performance Budgets Should be Linked to each Agency’s Strategic Plan

Strategic Goals and Objectives Need to be Established with Measurable Outcomes (and supported by senior leadership)

Programs Aligned with Strategic Goals and Objectives Performance Budgets Should Link the Resources Used to Meet

Performance Targets Use of a Consistent Set of Performance Measures from Planning

through Budget Execution

The Goal is to Continually Deliver Better Quality Products and Services While Increasing Efficiency and Effectiveness

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Performance Measure Best Practices

Focus on Agency Strategic Plan – Each priority should have several objectives. The Performance Measures should track progress on meeting the objectives that support each priority

Measures should be outcome oriented. A series of measures may be necessary to address some objectives

Data should be available to provide information on measures – if it is not, focus on measures that can be supported with existing data

Keep the number of performance measures to a minimum. Having hundreds of measures is too hard to analyze and doesn’t support leadership understanding on key issues

Study how other organizations build their performance strategy – benchmarking is an excellent way to conceptualize design of a performance measurement system for your agency

11

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Evolution of the Air Force Performance Budget

12

I n t e g r i t y - S e r v i c e - E x c e l l e n c e 13

The Air Force Framework Has Evolved Over Time

FY 2008/2009 PB

Feb 2007 Feb 2008

FY 2009 PB

May 2009

FY 2010 PB

BALANCED SCORECARD

FORMAT

DoD DECISION

LANESFORMAT

AIR FORCE STRATEGIC

PLANFORMAT

Linked Appropriation

Budget Highlights and

PBB

FY 2011 PB

Feb 2010

AIR FORCE STRATEGIC

PLANFORMAT

Linked Appropriation

Budget Highlights and

PBB

Demonstrated Relationship

with Air Force Core Functions

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Current Air Force Performance Budget Framework

14

Major sections: Overview Appropriations Summaries

MILPERs, O&M, RDT&E, Acquisition, MFH, MILCON, etc Performance Based Budget

Overview and linkages with AF Core Functions Reinvigorate the Air Force Nuclear Enterprise Partner with the Joint and Coalition Team to Win Today’s Fight Develop and Care for Airmen and their Families Modernize Our Air and Space Inventories, Organizations and Training Recapture Acquisition Excellence

Overseas Contingency Operations Funding Air Force Working Capital Funds Appendices

Discusses linkage with higher level guidance documents

Shows the budget as it is presented for

Congressional action

Discusses progress and performance aligned to

AF Strategic Plan

Addresses supplemental requests, OCO, and

other key budget areas

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Progress to Date

Establishment of a Performance Measures community of stakeholders across the Air Staff

Sharing of information with Air Force Posture Statement and Air Force Budget Overview has resulted in improvements in both products and better consistency of messaging

Better understanding across the Air Staff of who is measuring what and how to leverage other products and analysis

Feedback from Congressional staffs has been very positive on the product (easy to read and understand Service priorities and progress)

15

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Meeting the Challengesfor Continued Progress

Garner agreement on Performance Measures that link to Strategic Goals and Objectives (working closely with AF Strategic Plan office)

Determine relationship between performance and funding (increased analytics)

Applying performance measures to aid decision support in all PPBE phases (evolving from a compliance item to a decision tool)

Transparency of performance measures

16

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Examples of a Performance Budget Methodology

17

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Start with Senior Leadership Priorities(Example Only)

Priority X: Train and take Care of our People and Their Families

1) Objective: Right people, right skills at the right time2) Objective: Meet the needs of our people and their

families3) Objective: Highly trained workforce

Priority Y:Priority Z:

18

Objectives supporting each Priority need to be documented and have

leadership buy-in

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Example of Construct (continued)

Priority X: Train and take Care of our People and Their FamiliesObjective 1: PM = Recruiting Goal versus Actual by Specialty CodeObjective 1: PM = Retention Goal versus Actual by Specialty CodeObjective 2: PM = Member Satisfaction Rate versus GoalObjective 2: PM = Spouse Satisfaction Rate versus GoalObjective 3: PM = Percentage meeting Training Goals versus Actual

19

Specific Goals and Thresholds Determined by Leadership

Performance Measures should focus on meeting each objective

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Example of Construct (continued)

20

Each priority should be addressed with a summary if possible. The sections that follow should discuss the overall situation, how the performance measures are calculated and provide details of each initiative funded in the budget. Explain what the Agency plans to achieve with each initiative in performance terms.

Priority Objective Performance Measures Goal Actual Status Trend Resources

Recruiting Goal vice Actual by Specialty Code 100% 101%

MILPERS: $100,000,000, O&M: $68,000,000, Cost Per Recruit: $14,000

Retention Goal vice Actual by Specialty Code 80% 77% MILPERS SRB: $600M

Member Satisfaction Rate vice Goal 80% 81%

MWR Act: $5K per member, Tuition Assistance: $12K per member

Spouse Satisfaction Rate vice Goal

80% 69%CDC: $4K per member

Highly trained workforcePercentage meeting training Goals vice Actual

98% 98%Basic Training: $25K per member, Specialty Training: $36K per member

Train and Take Care of Our People and Their Families

Have the right people, with the right skills, at the right time

Meet the Needs of our People and their Families

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Ideal use of Performance Measures Across PPBE Phases

21

Planning

What is the desired performance

outcome needed to meet plans?

Programming

What resources are required to meet the

desired performance

outcome?

Budgeting

Can we predict performance in the

budget to help justify resources?

Execution

Feedback on actual performance to each phase to improve future

decisions

Performance Measures can be used to integrate all phases of PPBE

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

How Performance Measures Assist Decision Support

Are we meeting recruiting and retention goals? Funding changes to recruiting or retention bonuses? Increases in recruiting and retention initiatives in challenging

specialty codes?

Are our People and their Families happy? Issues causing satisfaction increases or decreases? Options to increase initiatives or change processes to increase

satisfaction

Are we meeting training goals and addressing changing mission needs? What training is required to meet forecasted mission requirements?

22

Answers to questions should drive funding or process decisions in the PPBE

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Performance and Budget Integration Conclusion

Progress is Ongoing More resources is not always the solution to better performance Continued emphasis on using consistent measures for improved

decision support (planned outcomes versus actual across the PPBE)

Still working to evolve performance measures to be more “outcome” oriented

Need to share best practices across Services (especially for similar performance areas like “People” and “Modernization”

Need to include performance measures in existing information systems

23

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Questions?

24

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

BACK UP SLIDES

25

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Analysis of Performance

26

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Performance AnalysisDrivers on Performance

Historical budget execution analysis with performance measures by fiscal year may provide some insight to the level of impact funding has on performance Need an understanding of policy or process changes impacting

performance for analysis Lagging indicators are fairly common Analysis is not as easy as pulling execution data from the

accounting system. Meeting with the program managers is important to get an understanding of the baseline, key initiatives, or other changes impacting program performance over time

Correlation Analysis can be performed after an understanding of changes (both funding and process) to programs is captured

27

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Data Validation andVariable Identification

Technique: Graph Data : Picture is worth a thousand words; so convert your

data to a graphical representation. Toolsets:

Scatter Plot Diagrams, Histograms, Pareto Charts (MS Excel) What you're looking for:

Clusters of data points, Trend Indications, Potential outliers, Frequency amongst variables

28

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Application and Descriptive Statistics

Technique: Determine central tendency & Normality: Compute the Mean,

Confidence Intervals, & determining the shape of the data distribution.

Toolsets: Goodness-of-Fit feature (Crystal Ball), RI$K wizard (ACE-IT), Cost

Estimating Manuals/Guidebooks (Multiple acceptable sources)

What you're looking for: Skewness, Symmetry about the mean, Kurtosis (Peakness),

variable frequency fit within the distribution curve -- all will indicate what distribution your data tends to follow (normal, lognormal, weibull, triangular, etc.)

29

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Define InteractionBetween Variables

Technique: Correlation / Regression Analysis: Statistically quantify how one

variable effects the other. Defining these dependencies are crucial to understand impact of What-If scenarios requested by Decision Makers.

Toolset: Goodness-of-Fit feature (Crystal Ball), CO$TAT & RI$K wizard (ACE-IT),

Computations, Add-Ins, and Stats Functions (Visual Basic & Excel)

What you're looking for: Correlation Coefficients (e.g. r2, and others ) and their tendency

towards -1.0 & +1.0, indicating degree of pos. or neg. correlation.

30

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Accounting for Multiple Scenarios

Technique: Monte Carlo Simulation: Regardless of the pedigree of your data

and the diligence of your analysis, there will be inherent “error” due to the uniqueness of the circumstances generating data; nonetheless, data driven responses to this risk can be accounted for by assigning statistical distributions and correlation parameters to your data variables.

Toolset: Monte Carlo Simulation features (Crystal Ball), (ACE-IT), (SEER),

What you're looking for: Delta change in output as degree of certainty increases or

decreases.

31

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Example of Trend Analysis: Air Force Recruiting

Recruiting Metrics

95%

101% 101% 101% 101% 100% 101%102%

101% 100%

$0.00

$5.00

$10.00

$15.00

$20.00

$25.00

$30.00

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007

Fiscal Year

$ 00

0s (T

Y)

90%92%94%96%98%100%102%104%

% o

f Goa

l

Cost Per recruit (MILPERS) Cost Per recruit (R&A and Examining) GOALS

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Government Performance Results Act of 1993 led the way

GPRA 1993 found that waste and inefficiency in Federal programs undermines the confidences of the American people in Government

Purpose of GRPA 1993 (1) improve the confidence of the American people in the capability of the Federal

Government, by systematically holding Federal agencies accountable for achieving program results;

(2) initiate program performance reform with a series of pilot projects in setting program goals, measuring program performance against those goals, and reporting publicly on their progress;

(3) improve Federal program effectiveness and public accountability by promoting a new focus on results, service quality, and customer satisfaction;

(4) help Federal managers improve service delivery, by requiring that they plan for meeting program objectives and by providing them with information about program results and service quality;

(5) improve congressional decision making by providing more objective information on achieving statutory objectives, and on the relative effectiveness and efficiency of Federal programs and spending; and

(6) improve internal management of the Federal Government.