measuring performance, benchmarking and setting objectives · measuring performance, benchmarking...
Post on 12-Apr-2018
247 Views
Preview:
TRANSCRIPT
Measuring performance, benchmarking
and setting objectives
ATC Global Session 3
Ralph RIEDLE
Chairman Performance Review Commission
17 September 2014
Measuring performance, benchmarking and setting objectives 2
Topics
ANS in the European aviation context
Performance oriented approach in ANS
ANS performance review examples
Conclusions
Measuring performance, benchmarking and setting objectives 3
ANS in the European aviation context (1/2)
≈ 6% of Airline
operating costs (Europe)
Air transport delay (2013)
All ≈ 9 min. per flight
ANS-related ≈1 min. per flight
≈ 6% of aviation related CO2
emissions (0.2% of total emissions)
Safety is the primary objective! No accident with ANS contribution since 2011 Reported incidents in 0.3% of flights
Although ANS is comparatively small in aviation context….
GDP from Air Transport
in EU ≈ $160 B Source: ATAG
Air Navigation Services
≈ $10 B
Safety Cost-efficiency
CapacityEnvironment
European IFR traffic
Measuring performance, benchmarking and setting objectives 4
ANS in the European aviation context (2/2)
ANS generates….
Value….. Safety Efficient flow of air traffic
Costs …. Total economic cost ≈ $ 14 B (SES area)
Direct ANS provision costs (user charges) Indirect service quality related costs
(delays, non-optimum flight profiles) Airborne equipment costs to be added
Environmental impact
High penalties to economy if disrupted
Scope for improvements!
Total
economic
cost
≈ $14 B
Flight
efficiency
ANS related
delays
ATCO
costs
Other costs
Direct
ANS
Cost
≈ $10 B
Support
costs
CAPEX
Indirect
service
quality
related
costs
≈ $4 B
… the stakes are high!
SES area
Measuring performance, benchmarking and setting objectives 5
Performance-oriented approach in ANS (1/5)
You can’t manage what
you don’t measure!
Measuring performance, benchmarking and setting objectives 6
Performance-oriented approach in ANS (2/5)
Objectives (What should be achieved?)
Commonly accepted framework (What should be measured, targeted: KPAs, KPIs?)
Performance targets (How much should be achieved? When?)
Performance monitoring (Are we on track?)
Corrective measures (As necessary)
Strategy & objectives
(what must be done well to implement strategy )
Framework, KPIs & PIs
(How successshould be measured)
Target setting(Quantification of what should be achieved)
Reporting, monitoring& analysis
(actual vs. targets)(understand achieved
performance)
Performance Review
Verified
Data
Measuring performance, benchmarking and setting objectives 7
Performance-oriented approach in ANS (3/5)
ECAC institutional strategy (1997)
Common ATM Performance strategy Focus on outcome, not only means (e.g. technology)
Independent Performance Review Commission (PRC) (1998) Independent Performance review Light-handed regulation: objective information, recommendations
Single European Sky (SES II: 2009)
SES Performance scheme Enforceable performance targets (EU, national/FAB levels), incentives Performance Review Body (PRB) advising the European Commission
ECAC Common ATM Performance
strategy
PRC
1997 1998 2004
SES PRB
2010 2009
SES II
2012 2015
Start RP1
Start RP2
1990
Delay crisis
Delay crisis
Measuring performance, benchmarking and setting objectives 8
Performance-oriented approach in ANS (4/5)
Safety
Cost-
effectiveness
Flexibility
ICAO
high-level expectations of the ATM community
Access &
Equity
Participation
Security
Efficiency
Predictability
Interoperability
Capacity
Environment
European focus
areas for target
setting
Safety
management
Capacity
management
Cost
management
Environmental sustainability
European focus areas in line with the 11 Key performance areas (KPAs)
defined in ICAO “Global Air Traffic Management Operational Concept”
Binding targets for Safety, Cost-efficiency, Capacity and Environment
Other KPAs measured
Performance-oriented approach in ANS (5/5)
Over Time
Benchmarking dimensions
Within region
Across regions
Different dimensions of performance benchmarking enable to:
• Understand and position own performance
• Identify performance gaps and scope for improvement
• set ambitious but realistic targets
Well established publications on ANS performance in Europe:
• Annual Performance Review Report (system view, all KPAs)
• ATM Cost Effectiveness Benchmarking (ACE) reports (Economics)
• Ad hoc reports (US/ Europe comparisons, etc.)
9
European focus
areas for target
setting
Safety
management
Capacity
management
Cost
management
Environmental
sustainability
Measuring performance, benchmarking and setting objectives
Annual Performance Review Reports (PRR)
2,2
2,9
4,5
2,9
2,5
1,4
0,9
0,8 0,9 1,0 1,2 1
,4
0,9
2,0
1,1
0,6
3
0,5
3
70
80
90
100
110
120
130
140
150
0,0
0,5
1,0
1,5
2,0
2,5
3,0
3,5
4,0
4,5
5,0
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
20
11
20
12
20
13
Tra
ffic
in
de
x (
ba
se
: 1
99
7)
En
-ro
ute
AT
FM
de
lay/ flig
ht (m
in.)
ATC Other (strike, equipment, etc.) WEATHER
OTHER (Special event, military, etc.) IFR Traffic
Average en-route ATFM delay per flight
source: Network Manager
4,9
1
4,8
7
4,8
6
3,3
1
3,2
0
3,1
4
2,0
2,5
3,0
3,5
4,0
4,5
5,0
5,5
6,0
200
9
2010
2011
2012
2013
Flight Plan (KEP) Actual trajectory (KEA)PRU analysis
ineffic
ien
cy
(%)
2009/10 KEA data based on regression analysis
• Verified facts and performance indicators
• Independent critical analysis (wide spectrum)
• Recommendations to decisions makers
SAFETY: Review of safety incident evolution
in order to identify trends (RI, SMI, etc.)
COST-EFFICIENCY: Analysis of ANS unit costs
und underlying drivers
ENVIRONMENT: Monitoring of flight
efficiency en route and in terminal areas;
CAPACITY: Evaluation of ANS related en route
and airport delays and constraining factors;
Safety Cost-efficiency
CapacityEnvironment
European IFR traffic
Measuring performance, benchmarking and setting objectives 10
ANSP benchmarking – ACE reports
ATM Cost Effectiveness Benchmarking (ACE) reports
Widely accepted performance framework (ICAO Doc 9161)
Verified data, official performance indicators and metrics
Factual analysis at European level, within region, over time
Purpose: Inform decision makers, support target setting within SES, etc
Safety Cost-efficiency
CapacityEnvironment
European IFR traffic
11 Measuring performance, benchmarking and setting objectives
Global perspective: US-Europe comparison
2012/2013 Europe USA US vs.
Europe Geographic Area (million km
2) 11.5 10.4 ≈ -10%
Nr. of civil en route Air Navigation Service Providers 37 1
Number of en route centres 63 20 -43
Number of APP units (Europe) and terminal facilities (US) 260 163 -97
Number of fully trained Air Traffic Controllers (ATCOs in Ops.) 17 200 13 400 ≈ -22%
Total staff 58 000 35 500 ≈ -39%
Controlled flights (IFR) (million) 9.6 15.1 ≈ +57%
Flight hours controlled (million) 14.3 22.4 ≈ +57%
Average length of flight (within respective airspace) 551 NM 515 NM ≈ -7%
Relative density (flight hours per km2) 1.2 2.2 ≈ x1.7
Share of General Aviation 3.9% 21%
Facts
Similar conditions (geographical area, average flight length)
US controls more IFR traffic (+57%) with fewer staff (-39%),
including ATC Controllers (-22%)
What are the underlying drivers of performance?
12 Measuring performance, benchmarking and setting objectives
Reactive policy in the 90’s: delays go up while costs go down, and vice versa
As of 1998, performance-oriented approach and improved capacity management contributed to reduction of both delays and unit costs
Enforceable SES performance targets apply from 2012 onwards
Performance in capacity and cost-efficiency
Traffic index
En route
delay
13 Measuring performance, benchmarking and setting objectives
5.42 5.385.18 5.15 5.11
3.29 3.17 3.12
0.0
1.0
2.0
3.0
4.0
5.0
6.0
2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019
rou
te e
xte
nsi
on
(%
)
Source: PRU analysis
Target RP1 Targets RP2
Environmental performance
Filed flight plan
Flight efficiency first measured in 2006: continuous improvement since then Good routing efficiency of ANS (≈3%) compared to other transport modes
• Yet significant economic impact (fuel burn, flight time) • Impossible to reach 0% with full civil-military traffic load
SES targets on Environment set for 2014 (FPL), 2019 (Actual, FPL)
Improved flight-efficiency compensates for air traffic growth
Carbon-neutral growth of aviation (due in 2020) already being met as far as European ANS is concerned!
Actual flown
trajectory
ENVIRONMENT:
Performance targets on
en route flight efficiency
within SES scheme
14 Measuring performance, benchmarking and setting objectives
Conclusions
Performance is the bottom line for ANS policy – Impact on more than $14B p.a. in EU, while ensuring safety
European Performance-driven strategy delivers, but margins for significant further improvements remain – Independent Performance review since 1998 (PRC)
– Stronger regulation under SES II (enforceable targets, regulations)
– EUROCONTROL supports both PRC and EC/PRB
Global benchmarking reveals best practices, weaknesses
– EUROCONTROL willing to engage with benchmarking partners
More details in workshop “Driving excellence in ATM performance” ( Workshop theatre, 19 Sep. , 10am)
Reports available at http://www.eurocontrol.int/prc/publications
15 Measuring performance, benchmarking and setting objectives
top related