Upload File

reinforcement learning for railway scheduling · reinforcement learning for railway scheduling...

  • Home
  • Documents
  • Reinforcement Learning for Railway Scheduling · Reinforcement Learning for Railway Scheduling Overcoming Data Sparseness through Simulations Dr. Erik Nygren Research and Innovation
14
Reinforcement Learning for Railway Scheduling Overcoming Data Sparseness through Simulations Dr. Erik Nygren Research and Innovation Lab Swiss Federal Railway

Upload: others

Post on 31-Jul-2020

2 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Reinforcement Learning for Railway Scheduling · Reinforcement Learning for Railway Scheduling Overcoming Data Sparseness through Simulations Dr. Erik Nygren Research and Innovation

Reinforcement Learning

for Railway Scheduling

Overcoming Data Sparseness through Simulations

Dr. Erik Nygren Research and Innovation Lab Swiss Federal Railway

Page 2: Reinforcement Learning for Railway Scheduling · Reinforcement Learning for Railway Scheduling Overcoming Data Sparseness through Simulations Dr. Erik Nygren Research and Innovation

© SBB • Solution Center Infrastructure • Research & Innovation • October 2017 2

Swiss Railway Network. A Complex Dynamical System.

Influencing Factors Facts

1

10,000

1,210,000

Weather People

Infrastructure Events

12,997

Most dense network 33,000

210,000 t

1t

31,266

3,230 km

KM

Energy

Page 3: Reinforcement Learning for Railway Scheduling · Reinforcement Learning for Railway Scheduling Overcoming Data Sparseness through Simulations Dr. Erik Nygren Research and Innovation

© SBB • Solution Center Infrastructure • Research & Innovation • October 2017 3

Train Dispatching and Scheduling. Challenges in the Worlds Densest Train Network.

RCS

Train runs

Production

Timetable

Page 4: Reinforcement Learning for Railway Scheduling · Reinforcement Learning for Railway Scheduling Overcoming Data Sparseness through Simulations Dr. Erik Nygren Research and Innovation

Evolution of Dispatching. Towards Full Automation.

Today

Future

Past

© SBB • Solution Center Infrastructure • Research & Innovation • October 2017 4

Page 5: Reinforcement Learning for Railway Scheduling · Reinforcement Learning for Railway Scheduling Overcoming Data Sparseness through Simulations Dr. Erik Nygren Research and Innovation

© SBB • Solution Center Infrastructure • Research & Innovation • October 2017 5

Automated Train Dispatching. Current Challenges.

Big Data

Big Data: Not enough relevant information

Automated

Dispatching

Learning

Measure-

ments Action

Page 6: Reinforcement Learning for Railway Scheduling · Reinforcement Learning for Railway Scheduling Overcoming Data Sparseness through Simulations Dr. Erik Nygren Research and Innovation

© SBB • Solution Center Infrastructure • Research & Innovation • October 2017 6

Reinforcement Learning for Railway Dispatching. Overcoming Data Sparseness through Simulations.

WIP

Measure- ments Action

Validation

Data generation

Learning

Learning

Action

Artificial Data

Big Data

High Performance Simulation

Page 7: Reinforcement Learning for Railway Scheduling · Reinforcement Learning for Railway Scheduling Overcoming Data Sparseness through Simulations Dr. Erik Nygren Research and Innovation

© SBB • Solution Center Infrastructure • Research & Innovation • October 2017 7

High Performance Simulations. Unleashing the Power of Parallel Computing.

DGX-1 High Performance Simulations

Time speedup Scenario variations Influencing factor analysis

Page 8: Reinforcement Learning for Railway Scheduling · Reinforcement Learning for Railway Scheduling Overcoming Data Sparseness through Simulations Dr. Erik Nygren Research and Innovation

© SBB • Solution Center Infrastructure • Research & Innovation • October 2017 8

Preliminary Results. Visualization of Simulation Results.

2h realtime

500x

5000x Simulation speed

Visualization speed

Page 9: Reinforcement Learning for Railway Scheduling · Reinforcement Learning for Railway Scheduling Overcoming Data Sparseness through Simulations Dr. Erik Nygren Research and Innovation

© SBB • Solution Center Infrastructure • Research & Innovation • October 2017 9

Reinforcement Learning. Playing the Dispatcher Game.

Action

Reward

DGX-1 High Performance Simulations

Artificial Data

DGX-1 Automated Dispatcher

Page 10: Reinforcement Learning for Railway Scheduling · Reinforcement Learning for Railway Scheduling Overcoming Data Sparseness through Simulations Dr. Erik Nygren Research and Innovation

© SBB • Solution Center Infrastructure • Research & Innovation • October 2017 10

Machine Learning on Artificial Data. Generating, Evaluating and Optimizing Train Dispatching.

Automated Dispatcher

Reinforcement Learning

Tree Search

Evolutionary Strategies

Building Blocks Variable Topologies

1

2

3

Mixed Integer Linear Programming

Genetic Algorithm

Page 11: Reinforcement Learning for Railway Scheduling · Reinforcement Learning for Railway Scheduling Overcoming Data Sparseness through Simulations Dr. Erik Nygren Research and Innovation

© SBB • Solution Center Infrastructure • Research & Innovation • October 2017 11

Current State... And Future Expected Reward.

DGX-1 High Performance Simulations

DGX-1 Automated Dispatcher

Fully Automated Process

Train runs

Production

Timetable

Page 12: Reinforcement Learning for Railway Scheduling · Reinforcement Learning for Railway Scheduling Overcoming Data Sparseness through Simulations Dr. Erik Nygren Research and Innovation

Take Home.

Big Data Big Information

Page 13: Reinforcement Learning for Railway Scheduling · Reinforcement Learning for Railway Scheduling Overcoming Data Sparseness through Simulations Dr. Erik Nygren Research and Innovation

© SBB • Solution Center Infrastructure • Research & Innovation • October 2017 13

Take Home.

AI

Model

Big Data Big Information

Dr. Erik Nygren

[email protected]

AI Researcher

Research Team

mailto:[email protected]
mailto:[email protected]
Page 14: Reinforcement Learning for Railway Scheduling · Reinforcement Learning for Railway Scheduling Overcoming Data Sparseness through Simulations Dr. Erik Nygren Research and Innovation

© SBB • Solution Center Infrastructure • Research & Innovation • October 2017 14

Reward Function. How to Reward an Artificial Dispatcher.

Reward


ADAPTIVE REACTIVE JOB-SHOP SCHEDULING WITH REINFORCEMENT LEARNING AGENTSml.informatik.uni-freiburg.de/_media/publications/gr07.pdf · concepts covered in this article. Accordingly,

Railway Rolling-Stock-Assignment-Scheduling Algorithm for

A Generic Multi-Agent Reinforcement Learning Approach … · A Generic Multi-Agent Reinforcement Learning Approach for Scheduling Problems ... Frank, Anne-Sonia, Kimberly, Eve, Marilyn,

Exploration of Reinforcement Learning in Radar Scheduling

Scheduling Virtual Machine Migration during Datacenter Upgrades … · 2020. 7. 23. · Scheduling Virtual Machine Migration during Datacenter Upgrades with Reinforcement Learning

A Reinforcement Learning Approach for Hybrid Flexible Flowline Scheduling Problems

Railway - Spare Inventory & Maintenance Scheduling model

Reinforcement Learning for Operating Systems …cs.rochester.edu/.../csc256/seminars/inaim_reinforcement_learning.pdf · Reinforcement Learning for Operating Systems Scheduling Iftekhar

Deep Reinforcement Learning for Scheduling in an Edge

Decentralised reinforcement learning for energy …...Decentralised reinforcement learning for energy-efficient scheduling in wireless sensor networks Mihail Mihaylov* and Yann-Aël

A Reinforcement Learning Variant for Control Scheduling · scheme for accelerated learning, if the system visits the same failed state successively, the negative reinforcement value

Optimal opportunistic tamping scheduling for railway track …1464836/... · 2020. 9. 8. · Optimal opportunistic tamping scheduling for railway track geometry Hamid Khajeheia, Mohammad

Energy Aware Deep Reinforcement Learning Scheduling for

LEARNET: Reinforcement Learning Based Flow Scheduling ......LEARNET: Reinforcement Learning Based Flow Scheduling for Asynchronous Deterministic Networks Jonathan Prados-Garzon∗,

Reinforcement Learning applied to Meta -scheduling in grid environments Bernardo Costa Inês Dutra

Reinforcement Learning for Wireless Network …...Multi-armed Bandits in Wireless Networks •Distributed channel access •Scheduling with limited feedback (IoT, sensor, etc.) : Markov

Cellular Network Traffic Scheduling using Deep Reinforcement … · 2019-03-11 · Cellular Network Traffic Scheduling using Deep Reinforcement Learning Sandeep Chinchali, et. al

Reinforcement of railway ballast using random fibre …Dependence of Shape on Particle Size for a Crushed Rock Railway Ballast. Granular Matter, 15 (6) 849-86. Le Pen, L., Bhandari,

BS 4466-1989 Specification for Scheduling, Dimensioning, Bending & Cutting of Steel Reinforcement for Concrete

INFORMS 2012 Shared Corridor Railway Maintenance Scheduling

On a Railway Maintenance Scheduling Problem with Customer ...scheith/ABSTRACTS/PREPRINTS/14-Heinicke.pdf · On a Railway Maintenance Scheduling Problem with Customer Costs and Multi-Depots

BS 4466-1989 Specification for Scheduling, Dimension Ing, Bending & Cutting of Steel Reinforcement for Concrete_2

ADAPTIVE REACTIVE JOB-SHOP ... - Machine Learning Labml.informatik.uni-freiburg.de/former/_media/publications/gr07.pdfADAPTIVE REACTIVE JOB-SHOP SCHEDULING WITH REINFORCEMENT LEARNING

UK Rebar Shape Codes · BS 8666:2005: Scheduling, dimensioning, bending and cutting of steel reinforcement for concrete. The revised British Standard for scheduling came into effect

Reinforcement Learning for Scheduling Wireless Powered ...193.136.60.49/docs/reinforcement_learning_for... · Reinforcement Learning for Scheduling Wireless Powered Sensor Communications

Railway, Metro and Lightrail Electrification Return Circuit APKJ... · Railway, Metro and Lightrail Electrification ... of reinforced concrete ... where corrosion of the reinforcement

Train scheduling and resource management in a railway stationpschaus/assets/master... · Railway stations have ceaselessly evolved together with the trains that use them. This growth

Dynamic Pricing and Energy Consumption Scheduling with ...medianetlab.ee.ucla.edu/papers/sg_tsg.pdf1 Dynamic Pricing and Energy Consumption Scheduling with Reinforcement Learning Byung-Gook

Hierarchical Bayesian Methods for Reinforcement Learning · Reinforcement Learning Logistics and scheduling Acrobatic helicopters Load balancing Robot soccer Bipedal locomotion Dialogue

ISPA 2008 APDCT Workshop1 Reinforcement Learning applied to Meta- scheduling in grid environments Bernardo Costa Inês Dutra Marta Mattoso

Reinforcement Learning-based Hierarchical Seed Scheduling for …csong/ndss21-afl-hier.pdf · 2021. 1. 14. · Reinforcement Learning-based Hierarchical Seed Scheduling for Greybox

Deep Reinforcement Learning-based Scheduling for ...dl.ifip.org/db/conf/wiopt/wiopt2017/1570333085.pdf1 Deep Reinforcement Learning-based Scheduling for Roadside Communication Networks

Storage Systems for High-Speed Railway Traction …...energies Article Optimized Sizing and Scheduling of Hybrid Energy Storage Systems for High-Speed Railway Traction Substations

Deep Reinforcement Learning for Delay-Oriented IoT Task ...arXiv:2010.01471v1 [cs.LG] 4 Oct 2020 1 Deep Reinforcement Learning for Delay-Oriented IoT Task Scheduling in Space-Air-Ground

Research Article Genetic Scheduling and Reinforcement Learning …downloads.hindawi.com/journals/mpe/2015/597956.pdf · 2019-07-31 · Genetic Scheduling and Reinforcement Learning