OECD 2016 Performance and Results Survey
Lisa von Trapp
12th Annual Meeting of the OECD SBO Performance and Results Network OECD Headquarters, Paris
24-25 November 2016
OECD Performance Budgeting Survey 2016
• Fourth OECD PB Survey
• Coverage: 65 questions on
– Performance budgeting
– Spending review (second iteration)
– Evaluation
• Respondents mainly from CBAs
• Response rate: 32 countries
• Some data cleaning is ongoing
Part I – Performance Budgeting
Key Observations
Performance budgeting (PB) 1. Performance frameworks are the norm
in OECD 2. Most countries set performance targets
but number and coverage varies 3. Institutions across government report
being more engaged in performance budgeting
4. Accountability and transparency: key priorities and benefits
5. Management responses to poor performance still more likely than budgetary consequences
6. Mixed signals on whether on whether countries believe performance budgeting has improved the quality of public finances
Performance Budgeting Performance frameworks are the norm in OECD
Source: 2016 OECD Performance Budgeting Survey (preliminary results, subject to revision)
17 6 2
3 4
Yes (25)
No (9)
compulsory for line ministries & agencies compulsory for line ministries only
Optional
No standard framework, Line ministries have
their own
None
Performance Budgeting Most countries set performance targets but number and coverage varies
Coverage and responsibility for setting targets
38%
27%
35%
All programmes
Most programmes
Only priorityprogrammes
0 10 20 30
Internal Audit
Other
Supreme Audit
Legislature
Chief Executive
CBA
Agencies
Line Ministries
Number of responding countries
2016
2011
2007
Performance Budgeting
Institutions across government report being more engaged in performance budgeting
Source: 2016 OECD Performance Budgeting Survey, Q20 preliminary results, subject to revision)
Which institutions play important roles in the performance budgeting process?
0
10
20
30
40
50
60
70
80
90
2016
2011
Performance Budgeting Slight increase in use of performance information in budget negotiations but…
Source: 2016 OECD Performance Budgeting Survey (preliminary results, subject to revision)
How often do the CBA and line ministries use the following kinds of performance information in their budget negotiations?
1 2 3 4 5
Independent performance information
Spending Review
Statistical information
Line Ministries/Agencies performance evaluations
Line Ministries/ Agencies operational data and performancereports
Line Ministries'/Agencies' financial data
2016
2011
NEVER ALWAYS
Performance Budgeting …accountability and transparency seen as the the key priorities and benefits
Ranking of the importance of factors behind introducing PB and effectiveness of PB system in advancing them
Index
Setting service delivery targets
Legal compliance
Coordinating government-wide strategic objectives
Facilitating evaluations/value-for-money studies
Allocation and prioritisation
Parliamentary budget scrutiny
Culture of performance
Transparency
Accountability
Effectiveness
Rationale
Low High
Source: 2016 OECD Performance Budgeting Survey, Q22-23 (preliminary results, subject to revision) Note: Index calculated as high=5, medium-high=4, medium=3, low-medium=2, low=1, N/A=0. Depicted as sum of all OECD member countries.
Performance Budgeting Management responses to poor performance still more likely than
budgetary consequences
NEVER ALWAYS
If performance targets are not met, how likely is it that the following consequences are triggered?
Source: 2016 OECD Performance Budgeting Survey, Q37 (preliminary results, subject to revision)
1 2 3 4 5
Pay cut for headProgramme eliminated
Programme transferred to othersBudget decreasesBudget increases
Budget freezesNegative consequences for leadersMore training for programme staff
More staff assigned to…New leadership brought in
Spending review or evaluationMore intense monitoring in future
No consequencesPoor performance made public
2007
2011
2016
Performance Budgeting Mixed signals on whether on whether countries believe performance
budgeting has improved the quality of public finances
Has PB contributed to improved quality of public finances in your country? Number of countries
Source: 2016 OECD Performance Budgeting Survey, Q37 (preliminary results, subject to revision)
5
1
10
13
5
-5
-2
-7
-2
-6
Strongly agree
Somewhat agree
Somewhat disagree
Strongly disagree
No benefit
Non-quantifiable benefits
Quantifiable benefits
Performance Budgeting Certain sectors stand out where performance information is used more
frequently and with higher impact on budgetary decisions
Top 4
Environment
Social affairs
Health
Education
Bottom 4
Internal operations
Trust in government
Inclusiveness
Gender equality
Source: 2016 OECD Performance Budgeting Survey, Q37 (preliminary results, subject to revision)
Performance Budgeting Greatest challenges to implementation
1. Lack of performance culture
2. Lack of resources
3. Lack of capacity/training
4. Lack of accurate/timely data
5. Lack of info on efficiency
Part II – Spending Review
Key Observations
Spending review (SR)
1. More countries than ever conduct spending reviews
2. The CBA tends to lead spending review exercises
3. Spending review outcomes are not always clear
Spending Reviews More countries than ever conduct spending reviews
Source: 2016 OECD Performance Budgeting Survey, Q56 (preliminary results, subject to revision)
0
2
4
6
8
10
12
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016
No
. of
spe
nd
ing
revi
ews
acro
ss O
ECD
re
spo
nd
en
ts e
ach
ye
ar
Comprehensive (20-100% of total government spending)
Broad (5-20% of total government spending)
Narrow (0-5% of total government spending)
Spending Reviews
The CBA tends to lead spending review exercises
Three governance models for spending review Percentage of spending review functions performed by ‘centre of government’ institutions (CBA or President/PM’s office)
Source: 2016 OECD Performance Budgeting Survey, Q54 (preliminary results, subject to revision) Note: Japan uses a committee-based review model.
0%
20%
40%
60%
80%
100%
PL DK NL DE SE US SL IT CH LX FI MX BE FR CAN GR UK PT IR NZ
Ce
ntr
alis
atio
n in
dex
Pres/PM CBA
Pres./PM centric (3)
CBA centric (10)
Line ministry centric (6)
Spending Reviews Spending review outcomes are not always clear
Source: 2016 OECD Performance Budgeting Survey, Q57 (preliminary results, subject to revision)
47%
33%
20%
All met
Some met
None met
Cannot be assessed
27%
33%
27%
13%
Were the objectives of the spending review met?
Fiscal objectives
Output and outcome objectives
Spending Reviews Major challenges around availability and quality of PI for spending reviews
Source: 2016 OECD Performance Budgeting Survey , Q58 ( preliminary results, subject to revision)
0 10 20 30 40 50 60
Lack of ICT
Gaming
Lack of political support (legislative)
Lack of capacity (e.g. available staff)
Lack of framework
Lack of support (senior civil service)
Lack of capability (e.g. technical expertise in terms of analysis or…
Lack of political support (executive)
Lack of attention on implementation/execution
Lack of time (e.g. short time frame for implementation)
Poor quality of performance information/data
Lack of performance information/data
What have been the greatest challenges to ensuring spending reviews are successful?
Part III – Evaluation
Key Observations
Evaluation and Audit
1. Evaluation is more
decentralised than PB and SR
2. Evaluations continue to have a limited budgetary impact
Evaluation Evaluation is more decentralised than PB and spending review
Source: 2016 OECD Performance Budgeting Survey, Q42 (preliminary results, subject to revision)
0%
5%
10%
15%
20%
25%
30%
35%
40%
LineMinistries /
Agencies
CBA SupremeAudit
Institution
Otherexternal
body
Governmentevaluationservice(s)
Legislature
Which organisations are responsible for conducting evaluations?
Evaluation Evaluation is more decentralised than PB and spending review
Many organisations within OECD member countries have expanded their roles in evaluation since 2011 Percentage of organisations
Source: 2016 OECD Performance Budgeting Survey, Q42 (preliminary results, subject to revision)
0
20
40
60
80
100Chief Executive
CBA
Line Ministries
AgenciesLegislature
Supreme Audit
Internal Audit
2016 2011
Evaluation Evaluations continue to have a limited budgetary impact
How impactful is information generated through the following types of ex post evaluation on budget decisions?
Source: 2016 OECD Performance Budgeting Survey, Q30 (preliminary results, subject to revision)
0
2
4
6
8
10
12
14
16
Always Usually Occasionally Rarely Never
Nu
mb
er o
f O
ECD
res
po
nd
ents
Used in CBA-Line Ministry negotiations
Used in Line Ministry-Agency negotiation
Evaluation Evaluations continue to have a limited budgetary impact
Countries indicated the following challenges:
– Evaluations have considerable delays between policy
implementation and evaluation publication, limiting their relevance to policy and budget makers.
– The evaluation process can often create added bureaucracy and drain on existing financial and human resources. The benefits may not, at times, justify the effort, weakening the legitimacy of evaluation as an information gathering tool.
– Meagre results in evaluation are partly an indication of evaluation as a technically difficult exercise, but it are also symptomatic of poorly aligned incentives.
For more information
OECD Senior Budget Officials Network on Performance and Results please visit:
www.oecd.org/gov/budgeting/seniorbudgetofficialsnetworkonperformanceandresults
Thank you!