Upload File

cs885 reinforcement learning lecture 1a: may 2, 2018

  • Home
  • Documents
  • CS885 Reinforcement Learning Lecture 1a: May 2, 2018
14
CS885 Reinforcement Learning Lecture 1a: May 2, 2018 Course Introduction [SutBar] Chapter 1, [Sze] Chapter 1 CS885 Spring 2018 Pascal Poupart 1 University of Waterloo

Upload: others

Post on 13-Jan-2022

5 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: CS885 Reinforcement Learning Lecture 1a: May 2, 2018

CS885 Reinforcement LearningLecture 1a: May 2, 2018

Course Introduction[SutBar] Chapter 1, [Sze] Chapter 1

CS885 Spring 2018 Pascal Poupart 1University of Waterloo

Page 2: CS885 Reinforcement Learning Lecture 1a: May 2, 2018

CS885 Spring 2018 Pascal Poupart 2

Outline

• Introduction to Reinforcement Learning• Course website and logistics

University of Waterloo

Page 3: CS885 Reinforcement Learning Lecture 1a: May 2, 2018

CS885 Spring 2018 Pascal Poupart 3

Machine Learning

• Traditional computer science– Program computer for every task

• New paradigm– Provide examples to machine– Machine learns to accomplish a task based on the

examples

University of Waterloo

Page 4: CS885 Reinforcement Learning Lecture 1a: May 2, 2018

CS885 Spring 2018 Pascal Poupart 4

Machine Learning

• Success mostly due to supervised learning– Bottleneck: need lots of labeled data

• Alternatives– Unsupervised learning, semi-supervised learning– Reinforcement Learning

University of Waterloo

Page 5: CS885 Reinforcement Learning Lecture 1a: May 2, 2018

What is Reinforcement Learning?

• Reinforcement learning is also known as – Optimal control– Approximate dynamic programming– Neuro-dynamic programming

• Wikipedia: reinforcement learning is an area of machine learning inspired by behavioural psychology, concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward.

CS885 Spring 2018 Pascal Poupart 5University of Waterloo

Page 6: CS885 Reinforcement Learning Lecture 1a: May 2, 2018

CS885 Spring 2018 Pascal Poupart 6

Animal Psychology

• Negative reinforcements:– Pain and hunger

• Positive reinforcements:– Pleasure and food

• Reinforcements used to train animals

• Let’s do the same with computers!

University of Waterloo

Page 7: CS885 Reinforcement Learning Lecture 1a: May 2, 2018

CS885 Spring 2018 Pascal Poupart 7

Reinforcement Learning Problem

Agent

Environment

StateReward Action

Goal: Learn to choose actions that maximize rewards

University of Waterloo

Page 8: CS885 Reinforcement Learning Lecture 1a: May 2, 2018

CS885 Spring 2018 Pascal Poupart 8

RL Examples

• Game playing (go, atari, backgammon)• Operations research (pricing, vehicle routing)• Elevator scheduling• Helicopter control• Spoken dialog systems• Data center energy optimization• Self-managing network systems• Autonomous vehicles• Computational finance

University of Waterloo

Page 9: CS885 Reinforcement Learning Lecture 1a: May 2, 2018

CS885 Spring 2018 Pascal Poupart 9

Operations research

• Example: vehicle routing

• Agent: vehicle routing software• Environment: stochastic demand• State: vehicle location,

capacity and depot requests• Action: vehicle route• Reward: - travel costs

University of Waterloo

Page 10: CS885 Reinforcement Learning Lecture 1a: May 2, 2018

CS885 Spring 2018 Pascal Poupart 10

Robotic Control• Example: helicopter control

• Agent: controller• Environment: helicopter• State: position, orientation,

velocity and angular velocity• Action: collective pitch, cyclic pitch, tail rotor control• Reward: - deviation from desired trajectory

• 2008 (Andrew Ng): automated helicopter wins acrobatic competition against humans

University of Waterloo

Page 11: CS885 Reinforcement Learning Lecture 1a: May 2, 2018

CS885 Spring 2018 Pascal Poupart 11

Game Playing• Example: Go (one of the oldest

and hardest board games)

• Agent: player• Environment: opponent• State: board configuration• Action: next stone location• Reward: +1 win / -1 loose

• 2016: AlphaGo defeats top player Lee Sedol (4-1)– Game 2 move 37: AlphaGo plays unexpected move (odds 1/10,000)

University of Waterloo

Page 12: CS885 Reinforcement Learning Lecture 1a: May 2, 2018

CS885 Spring 2018 Pascal Poupart 12

Conversational agent

• Agent: virtual assistant• Environment: user• State: conversation history• Action: next utterance• Reward: points based on task

completion, user satisfaction, etc.

• Today: active area of research

University of Waterloo

Page 13: CS885 Reinforcement Learning Lecture 1a: May 2, 2018

CS885 Spring 2018 Pascal Poupart 13

Computational Finance• Automated trading

• Agent: trading software• Environment: other traders• State: price history• Action: buy/sell/hold• Reward: amount of profit

Example: how to purchase a large # of shares in a short period of time without affecting the price

University of Waterloo

Page 14: CS885 Reinforcement Learning Lecture 1a: May 2, 2018

Reinforcement Learning

• Comprehensive, but challenging form of machine learning– Stochastic environment– Incomplete model– Interdependent sequence of decisions– No supervision– Partial and delayed feedback

• Long term goal: lifelong machine learning

CS885 Spring 2018 Pascal Poupart 14University of Waterloo


Section #1A 1A · 1A-1 1A-2 1A-3 1A-4 1A-5 1A-6 1A-7 1A-8 1A-9 1A-10 1A-11 1A-12 1A-13 1A-14 1A-17 1A-18 1A-20 1A-21 1A-22 1A-23 1A-25 1A-26 This guide is to be used as a reference

Reinforcement Learning and Deep Reinforcement Learningcse.ucdenver.edu/.../Class-22-Reinforcement-learning-DL.pdf · 2018. 11. 28. · Outlines 1 Principles of Reinforcement Learning

MEMORY AUGMENTED CONTROL NETWORKSppoupart/teaching/cs885... · Comparison with other work § Cognitive Mapping and Planning for Visual Navigation (Gupta et al. 2017) § Value iteration

Generalization in Reinforcement Learning: Successful ...papers.nips.cc/paper/1109-generalization-in-reinforcement-learning... · Generalization in Reinforcement Learning: Successful

CAIE GCE A LEVEL - SNS€¦ · danyal sohail faraz mustafa hania mumtaz omair rehman rimshah gul m. shaheer ul hassan syeda zoha ahmed wajih haider 1a 1a 1a 1a 1a 1a 1a 1a m. ibrahim

Introduction to Reinforcement Learning - FUDIPOfudipo.eu/.../01/Introduction-to-Reinforcement-Learning.pdfIntroduction to Reinforcement Learning Ather Gattami SeniorScientist,RISESICS

*+ 1A 1 1A î

Trust Region Policy Optimizationppoupart/teaching/cs885-spring...Trust Region Policy Optimization John Schulman, Sergey Levine, Philipp Moritz, Michael I. Jordan, Pieter Abbeel @ICML

Control of a Quadrotor with Reinforcement Learningppoupart/teaching/cs885-spring...Control of a Quadrotor with Reinforcement Learning Jemin Hwangbo, Inkyu Sa, Roland Siegwart, and

Continuous hoops for transverse reinforcement of ... · Continuous hoops for transverse reinforcement of ... transverse reinforcement details for the ... Fig. 1 Transverse reinforcement

for high REINFORCEMENT SYSTEMS - nevoga.com€¦ · 2 REINFORCEMENT SYSTEMS REINFORCEMENT SYSTEMS REINFORCEMENT SYSTEM PLEXUS®, PYRAPLEX®, FTW The reinforcement system PLEXUS®,

Cooperative Inverse Reinforcement Learning...Cooperative Inverse Reinforcement Learning Dylan Hadfield-Menell CS237: Reinforcement Learning May 31, 2017

CS885 Reinforcement Learning Lecture 9: May 30, 2018ppoupart/teaching/cs885... · •Learn directly from real experience University of Waterloo Agent: update policy/value function

Skinner’s Behavioral Reinforcement Theory Positive Reinforcement Negative Reinforcement PunishmentExtinction Person repeats desired behaviors to gain a

Reinforcement Learning - uni-freiburg.degki.informatik.uni-freiburg.de/.../recordings/reinforcement.pdf · Reinforcement Learning 3 What is Reinforcement Learning? Learning from interaction

PC-1A & MPC-1A Popcorn Poppers

Inverse Reinforcement Learning - Peoplecbfinn/_files/bootcamp_inverserl.pdf · Apprenticeship Learning via Inverse Reinforcement Learning. Good introduction to inverse reinforcement

Hierarchical Deep Reinforcement Learning: Integrating ...papers.nips.cc/paper/6233-hierarchical-deep-reinforcement-learning... · work that integrates deep reinforcement learning

Bayesian Reinforcement Learning in Continuous …MotivationBayesian Reinforcement LearningExperiments and ApplicationsConclusion Bayesian Reinforcement Learning in Continuous POMDPs

Schedules of reinforcement. Schedules of Reinforcement Continuous reinforcement refers to reinforcement being administered to each instance of a response

Reinforcement Learning Introduction Passive Reinforcement Learning Temporal Difference Learning Active Reinforcement Learning Applications Summary

Chapter 5A reinforcement - Lane Community College Media …Chapter 5A: Reinforcement 5 4/21/2014 Examples of Negative Reinforcement Negative reinforcement: A situation in which a response

for high ReinfoRcement systems - kotaca.cz · ReinfoRcement systems rEiNforcEmENt systEm PyraPlEx ... and DBV data sheet "Reinforcement system steel and ... suant to EC2 6.2.2 and

ITD Pipe Materials Selection & Specification ProceduresStandard Specificaton 1A for reinforcement requirements and for concrete wall thickness. ITD Pipe Materials Selection and Pipe

Section #1A 1A

Efficient Off-Policy Meta- Reinforcement Learning via ...ppoupart/teaching/cs885-spring...Some Terminology Off-policy learning: Two policies, one for exploring and the other for action

CS885 Reinforcement Learning Lecture 8b: May 25, 2018 · CS885 Reinforcement Learning Lecture 8b: May 25, 2018 Bayesian and Contextual Bandits [SutBar] Sec. 2.9 University of Waterloo

Learning from Experienceppoupart/teaching/cs885-spring...Learning from Experience Author James Cagalawan Created Date 6/27/2018 1:34:22 PM

CS885 Reinforcement Learning Lecture 4b: May 11, 2018ppoupart/teaching/cs885-spring... · 2018-05-11 · CS885 Spring 2018 Pascal Poupart 14 Deep Q-network Initialize weights !and

Inverse Reinforcement Learning CS885 Reinforcement

je=JEPPESEN (c) JEPPESEN SANDERSON, INC., 2019, ALL … · amepu 1a [amep1a] fasad 1a [fasa1a] okman 1a [okma1a] ba 1a [ba1a] er 1a [er1a] sb 1a [sb1a] rwy 05 rnav arrivals.eff.24.may

Reinforcement Learning - Multi-Agent Reinforcement

Reinforcement Learning or Active Inference?karl/Reinforcement Learning or Active... · Reinforcement Learning or Active Inference? ... From the point of view of reinforcement learning

Multi-Objective Reinforcement Learning using Sets of Pareto … · 2020. 10. 19. · learning and multi-objective reinforcement learning. 2.1 Reinforcement Learning A reinforcement

Eick: Reinforcement Learning. Reinforcement Learning Introduction Passive Reinforcement Learning Temporal Difference Learning Active Reinforcement Learning