last today - university of washington...1 t tp expected reward of anyonline alg 0 ifpick actin 1...

Lasttime Today

PAC learning a online learningGradient descent SGD Must weightupdateIg

applicators includingzero sum games

Figure credits

Arvin Blum slidesEmily Fox 446 slidesbook on Boosting by SchapiredFreundNate Jensen GabrielleCohn

is 0 3

A Pa Pa PmT

online earningIn

A i action set IAt r.MX

ffkiewmardfay4FrtA sEhD Ifntfa payoffon daytif you choose action on

Afg on dayt chooses 9 CA pstay prob choose

y i algae agitefamydattt T It trephufffsman

What can youhope to achieve aeachuffsqtYFmBest possible outcome a

a ptfgThis benchmark is way too strong A41,22

if pta z r'TD lpti a pts ra o o.w.pt 2js'zrtG j

benchmark Tr'T2 I

combingexpert advicear doing as well as best

expert

T

Regatta ah at a Eartha Entfateng bestrewed aby totalifayfaffeysad reward

ondayt need tochoose AEAFollow the leader

Claimi without randomization impossible to do well

endayt Flat 0 algtotalrt a I afat reward J O

n actors t.EE a n t T

Fatin y reward atleast C l th T

Aem3IWhat is the best we can hope for 120

h 2 toss a fancoin1

t tpExpected reward of any online alg 0

if pick actin 1 H's

2 IFT's A's

F I It's T's1 Cvfyfceyhaew.tRegItisrME offs o

Thin's 1 onlinealg win expregret QTRmaaxfn.la Elm IIg e2ta

teeth 3 E F I M

1 Srta El fan 2FMaeia Iifa Fa intuarealeweightsto Efrewaddymwattnet

Jtfiatoafaifw.tnprobability proportionalto wtf pt a ntfait I EwtaTEEHE

pt sung newts at tretrfpttt.zwt.la rID

Suppose a is best action Pt q wt a Main hindsight IN

Ttr

areas a i

pTEpTeNReenatnMaD_gpe.nA

EF ik

s EenltkMaDs.e'EI ErtIaaT

GIT eee

EizM E E f

1 RT thy

www.rf.EEFEfiewad gmwdstfnaaxi.I.hr

2

Instant atfo as T

2 player Zero sum games

Penalty kicks

zero sum means

trotone player'sgain is other

Cdplayer players lossgoateeL R

rae kicker L payeptorowpiafor R l O

rows Gols are called pure strategies

mixed strategy is prob distn over purestrategiesgoaleen73 LV Is a

Kian 71 EYEamt.Eixnedqztess

L

e isS 3 re.x

asif goalie goes Rexp loss

I o5x x

if kickergoesfirst maximize lossygoallee

minimax optstratg for rowplayert d

zero sum game defined Mxn matrix Awhere aij is payoff torn player

whenhe plays purestrategyicelplayer plays purestrategy ja Mm.us c

Xm

mixtedstrategy for crow player

Xi so

Iffpagoutorenplayn EiEP ax y ai

Xi yj

Ap'Sapighestgrowplayer goesfirst

chooses crass ycd player will choose

iIt

valveg game