interactive performance dashboards - eurocontrol · interactive performance dashboards making sense...
TRANSCRIPT
![Page 1: Interactive Performance Dashboards - Eurocontrol · Interactive Performance Dashboards Making sense of Big Human Performance Data in large-scale Real-Time Simulations EUROCONTROL-FAA](https://reader033.vdocument.in/reader033/viewer/2022052718/5f0541457e708231d4120d74/html5/thumbnails/1.jpg)
Supporting European
Aviation
Interactive Performance DashboardsMaking sense of Big Human Performance Data in large-scale Real-Time SimulationsEUROCONTROL-FAA Technical Interchange Meeting ‘Modelling and Simulation’ Atlantic City, USA, 16/17 October 2019
Bernd LorenzHuman Performance & Validation Expert
![Page 2: Interactive Performance Dashboards - Eurocontrol · Interactive Performance Dashboards Making sense of Big Human Performance Data in large-scale Real-Time Simulations EUROCONTROL-FAA](https://reader033.vdocument.in/reader033/viewer/2022052718/5f0541457e708231d4120d74/html5/thumbnails/2.jpg)
Overview
How we do Real-Time Simulations (RTS)
The Challenge: Large-scale RTS
Examples of Interactive Performance Dashboards
The way forward
EUROCONTROL-FAA Technical Interchange Meeting ‘Modelling and Simulation’: Atlantic City, USA, 16/17 October 2019 2
![Page 3: Interactive Performance Dashboards - Eurocontrol · Interactive Performance Dashboards Making sense of Big Human Performance Data in large-scale Real-Time Simulations EUROCONTROL-FAA](https://reader033.vdocument.in/reader033/viewer/2022052718/5f0541457e708231d4120d74/html5/thumbnails/3.jpg)
E-OCVM: Performance-driven Operational Concept Development
3
R&D
Full-scale RTS
PeopleSystem
Procedures
EUROCONTROL-FAA Technical Interchange Meeting ‘Modelling and Simulation’: Atlantic City, USA, 16/17 October 2019
Staff adaptation
![Page 4: Interactive Performance Dashboards - Eurocontrol · Interactive Performance Dashboards Making sense of Big Human Performance Data in large-scale Real-Time Simulations EUROCONTROL-FAA](https://reader033.vdocument.in/reader033/viewer/2022052718/5f0541457e708231d4120d74/html5/thumbnails/4.jpg)
Validation Activities in RTS Life-cycle
Kick-off MeetingState Concept and Assumptions
Validation StrategyIdentify stakeholders, issues, aims, objectives, claims, indicators, maturity criteria, cost-benefit mechanisms and expected outputs of the validation process
Validation Plan
2-stage Acceptance Testing
Simulation Conduct
Final ReportProvide a documented body of evidence in conclusion on the claimsInteractive Validation Data Repository
Core team Review
Core team Review
Core team Review
Final Review
Data Analysis&
Initial Reporting
EUROCONTROL-FAA Technical Interchange Meeting ‘Modelling and Simulation’: Atlantic City, USA, 16/17 October 2019 4
![Page 5: Interactive Performance Dashboards - Eurocontrol · Interactive Performance Dashboards Making sense of Big Human Performance Data in large-scale Real-Time Simulations EUROCONTROL-FAA](https://reader033.vdocument.in/reader033/viewer/2022052718/5f0541457e708231d4120d74/html5/thumbnails/5.jpg)
Visualisation in Data Analysis and Claim Evaluation
Quantitative DataSubjective ratings
Platform recordings
Qualitative DataQuestionnaire comments
DebriefingsInterviews
SharedSense-Making
EUROCONTROL-FAA Technical Interchange Meeting ‘Modelling and Simulation’: Atlantic City, USA, 16/17 October 2019 5
![Page 6: Interactive Performance Dashboards - Eurocontrol · Interactive Performance Dashboards Making sense of Big Human Performance Data in large-scale Real-Time Simulations EUROCONTROL-FAA](https://reader033.vdocument.in/reader033/viewer/2022052718/5f0541457e708231d4120d74/html5/thumbnails/6.jpg)
The challenge: Large-scale RTS
EUROCONTROL-FAA Technical Interchange Meeting ‘Modelling and Simulation’: Atlantic City, USA, 16/17 October 2019
165ATCOs
25Controller Working Positions
11Scenarios
88Exercises
1900Questionnaires
1335Debriefing log entries
616Interview
log entriesTMA re-organisation supporting Istanbul New Airport (2018)
6
![Page 7: Interactive Performance Dashboards - Eurocontrol · Interactive Performance Dashboards Making sense of Big Human Performance Data in large-scale Real-Time Simulations EUROCONTROL-FAA](https://reader033.vdocument.in/reader033/viewer/2022052718/5f0541457e708231d4120d74/html5/thumbnails/7.jpg)
Interactive Sector Performance Dashboard
(1) Sector SEN
7
![Page 8: Interactive Performance Dashboards - Eurocontrol · Interactive Performance Dashboards Making sense of Big Human Performance Data in large-scale Real-Time Simulations EUROCONTROL-FAA](https://reader033.vdocument.in/reader033/viewer/2022052718/5f0541457e708231d4120d74/html5/thumbnails/8.jpg)
Interactive Sector Performance Dashboard
(1) Sector SWF
8
![Page 9: Interactive Performance Dashboards - Eurocontrol · Interactive Performance Dashboards Making sense of Big Human Performance Data in large-scale Real-Time Simulations EUROCONTROL-FAA](https://reader033.vdocument.in/reader033/viewer/2022052718/5f0541457e708231d4120d74/html5/thumbnails/9.jpg)
Interactive Sector Performance Dashboard
(1) Sector SEL
9
![Page 10: Interactive Performance Dashboards - Eurocontrol · Interactive Performance Dashboards Making sense of Big Human Performance Data in large-scale Real-Time Simulations EUROCONTROL-FAA](https://reader033.vdocument.in/reader033/viewer/2022052718/5f0541457e708231d4120d74/html5/thumbnails/10.jpg)
Drilling into details … what exactly was the issue?
10
![Page 11: Interactive Performance Dashboards - Eurocontrol · Interactive Performance Dashboards Making sense of Big Human Performance Data in large-scale Real-Time Simulations EUROCONTROL-FAA](https://reader033.vdocument.in/reader033/viewer/2022052718/5f0541457e708231d4120d74/html5/thumbnails/11.jpg)
Drilling into further details … what exactly was the issue?
11
![Page 12: Interactive Performance Dashboards - Eurocontrol · Interactive Performance Dashboards Making sense of Big Human Performance Data in large-scale Real-Time Simulations EUROCONTROL-FAA](https://reader033.vdocument.in/reader033/viewer/2022052718/5f0541457e708231d4120d74/html5/thumbnails/12.jpg)
Dashboard: Targeted browsing through ATCO comments made in Post-Exercise-
Questionnaires
12
![Page 13: Interactive Performance Dashboards - Eurocontrol · Interactive Performance Dashboards Making sense of Big Human Performance Data in large-scale Real-Time Simulations EUROCONTROL-FAA](https://reader033.vdocument.in/reader033/viewer/2022052718/5f0541457e708231d4120d74/html5/thumbnails/13.jpg)
Dashboard: LOS analysis
13
13
![Page 14: Interactive Performance Dashboards - Eurocontrol · Interactive Performance Dashboards Making sense of Big Human Performance Data in large-scale Real-Time Simulations EUROCONTROL-FAA](https://reader033.vdocument.in/reader033/viewer/2022052718/5f0541457e708231d4120d74/html5/thumbnails/14.jpg)
Dashboard: LOS analysis
14
14
![Page 15: Interactive Performance Dashboards - Eurocontrol · Interactive Performance Dashboards Making sense of Big Human Performance Data in large-scale Real-Time Simulations EUROCONTROL-FAA](https://reader033.vdocument.in/reader033/viewer/2022052718/5f0541457e708231d4120d74/html5/thumbnails/15.jpg)
15
EUROCONTROL-FAA Technical Interchange Meeting ‘Modelling and Simulation’: Atlantic City, USA, 16/17 October 2019
Dashboard: Inspection of a more
complicated statistical evaluation of workload
![Page 16: Interactive Performance Dashboards - Eurocontrol · Interactive Performance Dashboards Making sense of Big Human Performance Data in large-scale Real-Time Simulations EUROCONTROL-FAA](https://reader033.vdocument.in/reader033/viewer/2022052718/5f0541457e708231d4120d74/html5/thumbnails/16.jpg)
16
![Page 17: Interactive Performance Dashboards - Eurocontrol · Interactive Performance Dashboards Making sense of Big Human Performance Data in large-scale Real-Time Simulations EUROCONTROL-FAA](https://reader033.vdocument.in/reader033/viewer/2022052718/5f0541457e708231d4120d74/html5/thumbnails/17.jpg)
17
![Page 18: Interactive Performance Dashboards - Eurocontrol · Interactive Performance Dashboards Making sense of Big Human Performance Data in large-scale Real-Time Simulations EUROCONTROL-FAA](https://reader033.vdocument.in/reader033/viewer/2022052718/5f0541457e708231d4120d74/html5/thumbnails/18.jpg)
Our experience: Interactive Performance Dashboards …
Provide an actionable insight into the collected data
Provide data visualisations at the right level of detail
Help to better grasp connections between different data-sets
Help to better communicate the data
Help to better prompt expert reasoning about operational issues
Support deeper human performance analysis
Make data integrity checks more efficient
EUROCONTROL-FAA Technical Interchange Meeting ‘Modelling and Simulation’: Atlantic City, USA, 16/17 October 2019 18
![Page 19: Interactive Performance Dashboards - Eurocontrol · Interactive Performance Dashboards Making sense of Big Human Performance Data in large-scale Real-Time Simulations EUROCONTROL-FAA](https://reader033.vdocument.in/reader033/viewer/2022052718/5f0541457e708231d4120d74/html5/thumbnails/19.jpg)
The way forward
Data-base Management
Live-connect to the SIM platform to support online monitoring dashboards
Extending the Metric Toolbox
ATCO monitoring/information acquisition/intervention Eye-tracking Mouse-position/click tracking HMI/support tool usage
EUROCONTROL-FAA Technical Interchange Meeting ‘Modelling and Simulation’: Atlantic City, USA, 16/17 October 2019 19
![Page 20: Interactive Performance Dashboards - Eurocontrol · Interactive Performance Dashboards Making sense of Big Human Performance Data in large-scale Real-Time Simulations EUROCONTROL-FAA](https://reader033.vdocument.in/reader033/viewer/2022052718/5f0541457e708231d4120d74/html5/thumbnails/20.jpg)
(2) Extending the metric tool box
20
Eye/mouse - tracking HMI input
Sector load
Fixed vs. Free Route
R/T activity
ISA –WL rating
Hilburn, 2004
![Page 21: Interactive Performance Dashboards - Eurocontrol · Interactive Performance Dashboards Making sense of Big Human Performance Data in large-scale Real-Time Simulations EUROCONTROL-FAA](https://reader033.vdocument.in/reader033/viewer/2022052718/5f0541457e708231d4120d74/html5/thumbnails/21.jpg)
EUROCONTROL-FAA Technical Interchange Meeting ‘Modelling and Simulation’: Atlantic City, USA, 16/17 October 2019 21
Thank you for your attention