development of a collision avoidance ... - atm seminar › seminarcontent › seminar13 ›...
TRANSCRIPT
Sybert Stroeve, Henk BlomCarlos Hernandez Medel, Carlos Garcia Daroca
Alvaro Arroyo Cebeira, Stanislaw Drozdowski
13th USA/Europe ATM R&D Seminar, Vienna, 17-21 June 2019
Development of a Collision Avoidance Validation and Evaluation Tool (CAVEAT)
Addressing the intrinsic uncertainty in TCAS II and ACAS X
2
Contents
• Context
• Agent-based sociotechnical model
• CAVEAT software tool
• Illustrative simulation results
• Conclusions
ContextUncertainty in encounter scenarios for ACAS design
3
4
TCAS II & ACAS X
Airborne Collision Avoidance System (ACAS)– Provides advisories to pilots for avoiding potential collisions– Traffic advisories (TAs): support pilot situation awareness on nearby traffic– Resolution advisories (RAs): Level off, Climb, Descend, Do not climb, etc.
Traffic Collision Avoidance System II (TCAS II)– Current commercial implementation, version 7.1 required by ICAO Annex 10– Filtering of transponder data and a fixed set of rules for alert generation
Recent development of ACAS X– Modular architecture for various surveillance sources– Optimization of logic tables for alert generation– Variants: ACAS Xa / ACAS Xo / ACAS Xp / ACAS Xu
5
Evaluation of ACAS designs
USEurope
ACAS advisories TCAS II v7.1 ACAS XaPilot response Standard 20% no response
P(NMAC) ACAS type Altitude Pilot responseP(RA types) Reversal Crossing Strengthening
1
encN
1
encN
6
Elements in an encounter scenario
7
Intrinsic uncertainty (each aircraft)
Pilot performance variability• Probability of response• Time of response• Strength of response
Sensor errors• Static errors (bias)• Variable errors (jitter)
8
Resulting uncertainty in ACAS alerting
As a result of uncertainty in ACAS surveillance, ACAS alerting is a stochastic process• No single unique RA for a scenario• Probability density for RA timing • Probability distribution RA types
P(RA)
9
Uncertainty in outcomes of an encounter scenario
10
ACAS evaluation with Monte Carlo simulation
MCN
1 MCN
1
encN
1
encN
ACAS typeSensor error models Ownship state estimation Othership measurementPilot response models Stochastic
Agent-based Sociotechnical Model
11
12
Agent-based sociotechnical model
13
Pressure altimetry system
0
0
0
,, , ,,
, , ,, , ,
( ;0, ) if { }
( ;0, ) if
N z bias biasPAS i PAS iz bias
t PAS i L z bias biasPAS i PAS i
f Normal
f Laplaceτ
ττ
σ κε
σ κ≥
⋅ =
⋅ =
Static errors (bias)• Normal or Laplace distribution• Altitude dependent variance
Dynamic errors (jitter)• First-order auto-regressive noise process
,
, , 0,,, ,
, , 0, ,
if if sample
PAS i
noiset PAS iz jitterauto z jitter noiset PAS iPAS t PAS it T PAS i
tt
ε τε α ε ε τ
−
== + >, 2
, , ,{ } (0, 1 ( ) )noise N z jitter autot PAS i PAS PAS ifε σ α−
, ,, , , , , , , ,z z z bias z jittert PAS i t AC i t PAS i t PAS is s ε ε= + +
Resulting pressure altitude
14
Slant range measurement
ACAS MOPS• RMS jitter ≤ 50 ft (mode S or C)• RMS bias ≤ 125 ft (mode S) or 250 ft (mode C)
0 ,
, ,, , , ,
{ } ( ;0, )jTS i
r bias N r biasTS i TS m i
fτε σ⋅
, , ,, , , , ,, ,sample
TS
r jitter auto r r jitter noiset TS i TS i t TS it T TS iε α ε ε
−= +
,
, , 2, , ,, ,
{ } ( ;0, 1 ( ) )jTS i
noise N r jitter auto rt TS i TS iTS m i
fε σ α⋅ −
, ,, , , , , , , , , ,j r bias r jitter
t TS i t AC i t AC j t TS i t TS ir ε ε= − + +s s
Error modelStatic and variable noise processes with mode dependent variance
• Gaussian bias
• 1st order autoregressive jitter
• Resulting slant range
15
ACAS algorithms in CAVEAT
TCAS II• Versions 7.0, 7.1, 7.2 stem from InCAS 3.3, based on MITRE
implementation (v7.0) and adjustments by Evosys (v7.1, v7.2)
ACAS Xa• Presented results: R15V2 in Julia with interface to C++ • Current implementation: R15V4 in C++ code from Honeywell
ACAS Xu• Expected in 2020/2021
16
Pilot model: response probability
ICAO standard model• Response probability: 100%• Delay
5 s (initial RA) 2.5 s (modified RA/COC)
• Acceleration 0.25g (normal) 0.35g (reverse / increase)
CAVEAT response probability model• P(initial RA | context)
altitude rate reversal parallel approach
• P(modified RA | previous response)• P(COC)• 13 conditional probabilities
Response probability measurements• Londner and Moss (2017)• P(response) <<100%• P(response | context)
rate reversal / parallel approach
17
Pilot model: delay
ICAO standard model• Response probability: 100%• Delay
5 s (initial RA) 2.5 s (modified RA/COC)
• Acceleration 0.25g (normal) 0.35g (reverse / increase)
CAVEAT delay model• Log-normal probability distribution• Advisory dependent parameters
initial / modified / COC
Delay measurements• Cloerec (2005)
18
Pilot model: acceleration
ICAO standard model• Response probability: 100%• Delay
5 s (initial RA) 2.5 s (modified RA/COC)
• Acceleration 0.25g (normal) 0.35g (reverse / increase)
CAVEAT acceleration model• Log-normal probability distribution• Linear relation between mean
acceleration and vertical speed
Acceleration measurements• Cloerec (2005)• Relation between acceleration
and vertical speed
1 1 1
, , ,, , , , , ,
a RA z RA a zPF i PF i PF iv vτ τ τµ α β= + −
CAVEAT Software Tool
19
20
High-level design of CAVEAT software
21
HMI allows setting of all agent parameters
Pilot response Ownship state estimation
22
Horizontal / vertical views of a simulation run
23
Statistics report (for MC simulation)
Illustrative Simulation Results
24
25
Encounter scenarios
Encounter Horizontal Vertical HMD (ft) VMD (ft)
E1 Crossing AC1 climbsAC2 is level 3038 200
E2 Crossing AC1 is levelAC2 is level 0 0
Scenario Sensor errors
Pilot Dynamic Variability
Pilot Response Probability
D none none (standard) 100%
S1 MOPS none (standard) 100%
S2 MOPS delay, rate, acc. 100%
S3 MOPS delay, rate, acc. 80%
Stochastic
Deterministic
26
Encounter scenario E1-D with TCAS II v7.1Vertical viewHorizontal view
HMD: 3038 ft VMD: 200 ft (original) / 522 ft (modified)
27
Encounter scenario E2-D with TCAS II v7.1Vertical viewHorizontal view
HMD: 0 ft VMD: 0 ft (original) / 975 ft (modified)
DE
CL
28
Timing of advisories (TCAS II v7.1) Encounter scenario E1-S1 Encounter scenario E2-S1
29
Number & types of advisories (TCAS II v7.1)
AC RADeterministic Stochastic (S1)
Sense P(RA) P(Sense | RA)
1
RA-1 Level off 100% Level off 97%Descend 3%
RA-2 - 6% Level off 53%Descend 47%
RA-3 - 3% Level off 100%
2
RA-1 Climb 100%Climb 94%Do not
descend 6%
RA-2 Level off 94% Level off 94%Climb 6%
RA-3 - 1% Level off 1%
Encounter E1 (climb x level)AC RA
Deterministic Stochastic (S1)
Sense P(RA) P(Sense | RA)
1RA-1 Descend 100% Climb 51%
Descend 49%
RA-2 Level off 100% Level off 100%
2RA-1 Climb 100%
Descend 49%
Climb 51%
RA-2 Level off 100% Level off 100%
Encounter E2 (same level)
30
VMD empirical PDF of Encounter E1 (climb x level)
Sensor errors only (S1) Sensor errors & pilot var. (S3)522 723
Deterministic: TCAS II v7.1 ACAS Xa V15R2
TCAS II v7.1mean: 519 ftSD: 111 ft
ACAS Xa V15R2mean: 691 ftSD: 96 ft
522 723
TCAS II v7.1mean: 466 ftSD: 143 ft
ACAS Xa V15R2mean: 639 ftSD: 162 ft
31
VMD empirical PDF of Encounter E2 (same level)
Sensor errors only (S1) Sensor errors & pilot var. (S3)
Deterministic: TCAS II v7.1 ACAS Xa V15R2
975 1025 975 1025
TCAS II v7.1mean: 845 ftSD: 85 ft
ACAS Xa V15R2mean: 906 ftSD: 78 ft
TCAS II v7.1mean: 916 ftSD: 230 ft
ACAS Xa V15R2mean: 916 ftSD: 230 ft
Conclusions
32
33
Conclusions
Evaluation of ACAS designs requires assessment of all relevant sources of variability and uncertainty
– Variability in encounter geometries / altitude layers– Variability in equipage types– Uncertainty due to pilot response– Uncertainty due to sensor errors
Development of an agent-based sociotechnical model for Monte Carlo simulation of encounter scenarios
– Various sensor error (bias & jitter) models– Pilot performance models– Supports deterministic (single run) simulations– Systematic evaluation of the impact of uncertainty in encounter scenarios– Extendible for new agents: ACAS Xu, UAV, ATC,...
34
Conclusions
Illustrative simulation results– Variability in timing and types of RAs, and dispersion in VMD
• Considerable differences with deterministic simulation results – Mean MC simulation results can differ considerably from deterministic simulation results
Use for prospective analysis: What will be the impact of changes? – Impact on RA types / VMD / P(NMAC)– MC simulation of sets of encounter scenarios – Evaluation of ACAS designs / ACAS related systems / system compatibility– Evaluation of changes in airspace, ATM, and regulations
Use for retrospective analysis: What happened / could have happened? – MC simulation of a single encounter scenario– Likelihood of observed RAs (type / timing)– Likelihood of observed CPA
35
Conclusions
CAVEAT software development for EUROCONTROL– Extensive HMI gives wide control over parameters of the agent-based model– TCAS II v7.0, v7.1, v7.2, ACAS Xa R15V4– Detailed views for incident investigation of single encounters– Powerful simulation core for analysis of large sets of encounters – Integration of ACAS Xu (when available)
Future research options– Required sizes of encounters sets and number of MC simulation runs– Sensitivity analysis for understanding the impact of the range of uncertainty sources– Detailed analysis of the impact of new ACAS designs (ACAS Xa, ACAS Xu)
36
Acknowledgements
The R&D in this paper has been conducted with support from EUROCONTROL in the scope of SESAR 2020 Project 11
We thank MITRE and Honeywell for making available their TCAS II and ACAS Xa/Xo libraries, respectively
We thank Carmelo Javier Villanueva Cañizares, Daniel Rubio Garcia, Cristina Midori Fukuda Leon, Eduardo Pablos Ruiz, Ángela Merino Pérez, Volker Huck, David Phu, Garfield Dean, Busso Gellert, Frank Bussink and Bert Bakker for their support of the CAVEAT development
NLR AmsterdamAnthony Fokkerweg 21059 CM AmsterdamThe Netherlands
p ) +31 88 511 31 13 e ) [email protected] i ) www.nlr.org
NLR MarknesseVoorsterweg 318316 PR MarknesseThe Netherlands
p ) +31 88 511 44 44 e ) [email protected] i ) www.nlr.org
Fully engagedRoyal Netherlands Aerospace Centre