algorithms, game theory and risk-averse decision making*...shortest paths example algorithms, game...

EvdokiaNikolovaUniversityofTexasatAus2n

Algorithms,GameTheoryandRisk-averseDecision

Making*

*IntroductorylectureforSimonsReal-BmeDecisionMakingBootcamp,January24,2018

Real-2medecisionmakingexamples

Algorithms,GameTheory&Risk-averseDecisionMakingEvdokiaNikolova

1010

10

13

3

3

2

16

2

8

12

3

11

2

22

Real-2medecisionmakingexamples


A

B

C

D

E

F

G

PartI:Algorithms

EvdokiaNikolova Algorithms,GameTheory&Risk-averseDecisionMaking

Algorithms:thebasics

Algorithms,GameTheory&Risk-averseDecisionMaking

•  What is an algorithm? – a step-by-step

procedure to solve a problem

– every computer program is the instantiation of some algorithm

EvdokiaNikolova

Shortestpathsexample


•  Solves a general, well-specified problem –  given a graph G = (V, E), a source node s and a destination node t,

and edge costs c1,…,cn, as input, produce as output a shortest st-path, namely an st-path with smallest total edge cost

•  Problem has specific instances

•  Algorithm takes every possible instance and produces output with desired properties –  Dijkstra, Bellman-Ford, Floyd-Warshall shortest path algorithms

EvdokiaNikolova

Modelingthereal-world


•  Cast your application in terms of well-studied abstract data structures

EvdokiaNikolova

Concrete Abstract

arrangement, tour, ordering, sequence permutation

cluster, collection, committee, group, packaging, selection subsets

hierarchy, ancestor/descendants, taxonomy, radial network trees

network, circuit, web, relationship graph

sites, positions, locations points

shapes, regions, boundaries polygons

text, characters, patterns strings

Graphterminology


•  Adirectedgraph(ordigraph)Gisapair(V,E)whereVisafiniteset(of“ver2ces”)andE(the“edges”)isasubsetofV×V.

•  AnundirectedgraphGisapair(V,E)whereVisafiniteset(of

“ver2ces”)andE(the“edges”)isasetofunorderedpairsofedges{u,v},whereu≠v.

EvdokiaNikolova

1 2

36

1 2 3 41 2 3

45 1 2 3 4

5 61 2 3 4

5 64

1 2 3

u v w x y z

5 61 2 3 4

5 64

1 2 3 4

1 2

36

1 2 3 41 2 3

45 1 2 3 4

5 61 2 3 4

5 64

1 2 3

u v w x y z

5 61 2 3 4

5 64

1 2 3 4

Graph=NetworkNodes=Ver2cesEdges=Links=Arcs

•  Abstrac2onoftransporta2ongraph(Braessparadoxgraph)

B A

Graphexamples


0.5hr 1 hour

1 hour 0.5hr

1hr

1hr

0


•  Abstrac2onofelectricitygraph:

B A

Graphexamples


0.5hr 1 hour

1 hour 0.5hr

1hr

1hr

0

Bus=NodeLine=Edge?

B A

Graphexamples



•  Abstrac2onofelectricitygraph:

EvdokiaNikolova

0.5hr 1 hour

1 hour 0.5hr

1hr

1hr

0

Bus=NodeLine=Edge?


Backtoshortestpaths•  Algorithmicchallenge:exponen2allymanypaths

•  Bruteforce:enumerateallpossiblepaths;outputbestone.Bruteforcealgorithmhasexponen2alrunning2me

V1 V2 V3 Vn-1 Vn ...5

3

2

3

5

4

…

2n




V1 V2 V3 Vn-1 Vn ...5

3

2

3

5

4

…

2n1067:numberofatomsinourgalaxy




V1 V2 V3 Vn-1 Vn ...5

3

2

3

5

4

…

2n



•  Inshortestpaths,op*malsubstructureproperty(partoftheop*malsolu*onremainsop*malonthesubproblem)allowsforefficientdynamicprogrammingalgorithms(Dijkstra,Bellman-Ford,etc)

AB

C

5

6.9

5

V1 V2 V3 Vn-1 Vn ...5

3

2

3

5

4

Dijkstrashortestpathalgorithm


B A

0.5hr

0.5hr

1hr

1hr

0

inf

inf

inf



B A

0.5hr

0.5hr

1hr

1hr

0

0.5

1

inf



B A

0.5hr

0.5hr

1hr

1hr

0

0.5

0.5

1.5



B A

0.5hr

0.5hr

1hr

1hr

0

0.5

0.5

1

(Basic)AlgorithmDesignTechniques


•  Brute Force & Exhaustive Search –  follow definition / try

all possibilities •  Divide & Conquer

–  break problem into distinct subproblems

•  Transformation –  convert problem to

another one

EvdokiaNikolova

•  Dynamic Programming –  break problem into

overlapping subproblems

•  Greedy –  repeatedly do what is

best now •  Randomization

–  use random numbers •  Linear programming


Algorithmrunning2me

•  Wewouldlikeadefini2onofalgorithmefficiencythatis:–  plaiorm-independent–  instance-independent–  ofpredic2vevaluewithrespecttoincreasinginstance(input)sizes(e.g.,foragraph,inputsizeisthenumberofnodesnandedgesm)

•  AnalgorithmisefficientifithaspolynomialrunningBme.–  f(n)=aknk+ak-1nk-1+…+a1n+a0isapolynomialofdegreek.


Algorithmrunning2me•  Analgorithmisefficientifithaspolynomialrunning

Bme.–  f(n)=aknk+ak-1nk-1+…+a1n+a0isapolynomialofdegreek.–  Wecareabouthighestordertermandnocoefficients:algorithmpolynomialrun2meisO(nk),wherekisaconstant

•  Ofcourse,running2meofn100isclearlynotgreat,and

arunning2meofn1+.02(logn)isnotclearlybad.Butinprac2ce,polynomial2meisgenerallygood.

•  Inaddi2ontobeingprecise,thisdefini2onisalsonegatable.

PvsNP


•  P is the class of problems which can be solved in polynomial time

•  NP (“nondeterministic polynomial time”) is the class of problems for which a candidate solution can be verified in polynomial time –  Note: NP does not stand for “not polynomial” –  P is a subset of NP

EvdokiaNikolova

P

NP

PvsNP


Difference between solving a problem and verifying a candidate solution: •  Solving a problem: is there a path in graph G from vertex u

to vertex v with at most k edges? •  Verifying a candidate solution: is v0, v1, …, vl a path in

graph G from vertex u to vertex v with at most k edges?

EvdokiaNikolova

PvsNP


Difference between solving a problem and verifying a candidate solution: •  Solving a problem: is there a path in graph G from vertex u

to vertex v with at most k edges? •  Verifying a candidate solution: is v0, v1, …, vl a path in

graph G from vertex u to vertex v with at most k edges?

EvdokiaNikolova

Isthereapathoflengthatmostk=5?

source

des2na2on

PvsNP


•  A Hamiltonian cycle in an undirected graph is a cycle that visits every vertex exactly once.

•  Solving a problem: is there a Hamiltonian cycle in graph G?

•  Verifying a candidate solution: is v0, v1, …, vl a Hamiltonian cycle of graph G?

EvdokiaNikolova

PvsNP


•  Although poly time verifiability seems like a weaker condition than poly time solvability, no one has been able to prove that it is weaker (describes a larger class of problems)

•  So it is unknown whether P = NP.

EvdokiaNikolova

all problems

P �

NP �or

all problems

P=NP � ?

NP-Completeproblems


•  NP-complete problems is class of "hardest" problems in NP. •  They have the property that if any NP-complete problem

can be solved in poly time, then all problems in NP can be, and thus P = NP.

EvdokiaNikolova

?all problems

P �

NP �or

all problems

P=NP=NPC �NPC �

NPC = NP-complete

NP-Completeproblems


•  NP-complete problems is class of "hardest" problems in NP. •  A decision problem is NP-complete if

–  It is in NP –  It is “at least as hard as” some known NP-complete problem

EvdokiaNikolova

…

•  When given a new problem, a computer science theorist first wants to understand whether the problem is in P, or NP-complete, or another complexity class.

FirstNP-CompleteProblem(SAT) Problem2 Problem3 Problemk

OurProblem

NP-Completeproblems


•  NP-complete problems is class of "hardest" problems in NP. •  They have the property that if any NP-complete problem

can be solved in poly time, then all problems in NP can be, and thus P = NP.

•  Other complexity classes:

EvdokiaNikolova

PvsNPproblem


•  Open question since about 1970 •  One of the seven"MillenniumPrizeProblems” bytheClayMathema2csIns2tute($1millionprizeforsolvingit)–alongwiththeRiemannHypothesis,thePoincaréConjecture,etc.*

•  Great theoretical interest •  Great practical importance:

–  If your problem is NP-complete, then don't waste time looking for an efficient algorithm

–  Instead look for efficient approximations, heuristics, etc.

EvdokiaNikolova

*htp://www.claymath.org/millennium-problems

PvsNPproblem


•  Open question since about 1970 •  One of the seven"MillenniumPrizeProblems” bytheClayMathema2csIns2tute($1millionprizeforsolvingit)*

•  Great theoretical interest •  Great practical importance:

–  If your problem is NP-complete, then don't waste time looking for an efficient algorithm

–  Instead look for efficient approximations, heuristics, etc.

EvdokiaNikolova

*htp://www.claymath.org/millennium-problems

Approxima2onalgorithms*


•  If we do not know how to solve a problem in polynomial time, can we find a near-optimal solution in polynomial time?

•  An α-approximation algorithm for an optimization problem –  runs in polynomial time –  always returns a candidate solution –  cost of returned solution is at most α times the cost

of the optimal solution (for a minimization problem)

EvdokiaNikolova*Thissemester:CS294-145"Approxima2onAlgorithms"instructorDavidWilliamson.Tue/Thu3:30-5:00in310SodaHall.

TravelingSalesmanProblem


•  Given a set of cities, distances between all pairs of cities, and a bound B, does there exist a tour (sequence of cities to visit that returns to the start and visits each city exactly once) that requires at most distance B to be traveled?

•  TSP is in NP: – given a candidate solution (a tour), add up all

the distances and check if total is at most B

EvdokiaNikolova

TSPapproxima2onalgorithm


•  Input: set of cities and distances b/w them that satisfy the triangle inequality

EvdokiaNikolova


A

B C

D

E

F G


1)  Compute MST



A

B C

D

E

F G


1)  Compute MST 2)  Go around MST to

get a tour



A

B C

D

E

F G



get a tour 3)  Remove duplicate

cities



A

B C

D

E

F G


•  This is a 2-approximation algorithm


get a tour 3)  Remove duplicate

cities


PartII:GameTheory


Best route depends on others


•  Gamesarethoughtexperimentstohelpuslearnhowtopredictra)onalbehaviorinsitua)onsofconflict

•  Situa)onofconflict:Everybody'sac2onsaffectothers.Thisiscapturedbythetabulargameformalism.

•  Ra)onalBehavior:Theplayerswanttomaximizetheirownexpectedu2lity.Noaltruism,envy,masochism,orexternali2es(ifmyneighborgetsthemoney,hewillbuylouderstereo,soIwillhurtalitlemyself...).

•  Predict:Wewanttoknowwhathappensinagame.Suchpredic2onsarecalledsolu2onconcepts(e.g.,Nashequilibrium).

Gametheory


Travel time increases with congestion


• Highwayconges2oncostswere$160billionin2014(TTI)

• Avg.commutertravels100minutesaday.EvdokiaNikolova

Example:Inefficiencyofequilibria

Suppose 100 drivers leave from town A towards town B.

What is the traffic on the network?

Every driver wants to minimize her own travel time.

In any unbalanced traffic pattern, all drivers on the most loaded path have incentive to switch their path.

Delay is 1.5 hours for everybody at the unique Nash equilibrium

Town A Town B

x/100 hours

1 hour

1 hour

1/2

1/2

x/100hours

x/100hours

1hour

1hour



A benevolent mayor builds a superhighway connecting the fast highways of the network.

What is now the traffic on the network?

No matter what the other drivers are doing it is always better for me to follow the zig-zag path.

Delay is 2 hours for everybody at the unique Nash equilibrium

Town A Town B

x/100 hours

x/100 hours

1 hour

1 hour

1

x/100hours

x/100hours

1hour

1hour

0hours



A B

x/100 hours

x/100 hours

1 hour

1 hour

1

A B

x/100 hours

x/100 hours

1 hour

1 hour

1/2

1/2

vs

Adding a fast road on a road-network is not always a good idea! Braess’s paradox

In the RHS network there exists a traffic pattern where all players have delay 1.5 hours.

PoA =performance of system in worst Nash equilibrium

optimal performance if drivers did not decide on their own

4/3

Price of Anarchy:

x/100hours

x/100hours

1hour

1hour

x/100hours

x/100hours

1hour

1hour


measures the loss in system performance due to free-will

EvdokiaNikolova

Equilibrium

•  “Travel2mesonusedroutesareequalandnogreaterthantravel2mesonunusedroutes.”

•  DefinesWardropEquilibrium,also called User Equilibrium or Nash Equilibrium.


0hoursTown A Town B

x/100 hours

x/100 hours

1 hour

1 hour

1

x/100hours

x/100hours

1hour

1hour

•  “Theaverage[total]journey2meisminimum.”

•  DefinesSocialOp2mum(SO)

SocialOp2mum


Town A Town B

x/100 hours

1 hour

1 hour

1/2

1/2

x/100hours

x/100hours

1hour

1hour

0hours

•  “[Modified]Travel2mesonusedroutesareequalandnogreaterthantravel2mesonunusedroutes.”

•  DefinesSocialOp2mum(SO)

SocialOp2mum


Town A Town B

x/100 hours

1 hour

1 hour

1/2

1/2

x/100hours

x/100hours

1hour

1hour

0hours


•  SocialOp2mum(SO)isEquilibriumw.r.ttravel2mesplustolls!

SocialOp2mum


Town A Town B

x/100 hours

1 hour

1 hour

1/2

1/2

x/100hours+1/2

x/100hours+1/2

1hour

1hour

0hours


•  SocialOp2mum(SO)isEquilibriumw.r.ttravel2mesplustolls!àMechanismdesign

SocialOp2mum


Town A Town B

x/100 hours

1 hour

1 hour

1/2

1/2

x/100hours+1/2

x/100hours+1/2

1hour

1hour

0hours

Note:bothEquilibriumandSocialop2mumexistandcanbecomputedefficiently.

PriceofAnarchy •  Price of anarchy: (Koutsoupias, Papadimitriou ’99)

•  Measures the degradation of system performance due to free will (selfish behavior)

•  4/3 in general graphs, linear delays as function of traffic; 2 for quartic delays (Roughgarden, Tardos ’02; Correa, Schulz, Stier-Moses ‘04, ‘08)

Cost Optimum SocialCost mEquilibriusup

instancesproblem


Take-awaypointsonconges2ongames

•  Shortest paths (and algorithmic questions in general) can get complicated due to presence of more people

•  Equilibrium and Social Optimum in nonatomic routing games exist and can be found efficiently via convex programs.

•  Social optimum is an equilibrium with respect to modified latencies = original latencies plus toll.

•  Price of anarchy: 4/3 for linear delays, can be found

similarly for more general classes of delay functions.


PartIII:Risk-aversedecisionmaking


EvdokiaNikolova Reliablerouteplanning

•  Op2malroutemaydifferwithstart2me

Op2malroute?

30or60

40



Op2malroute?

30or60

40

10:30 am 9:50

late ontime

Departure time



Op2malroute?

30or60

40

10:30 am 9:50 10

late 50%

ontime

9:30 Departure time



Op2malroute?

30or60

40

10:30 am 9:50 10

Optimal route

Departure time

Implica2onsofriska{tudes


•  Uncertaintyandriskcanaffectourdesignandanalysisof:

•  Algorithms•  AlgorithmicGameTheory•  AlgorithmicMechanismDesign

EvdokiaNikolova

Whatisrisk?

Threemainquan2ta2veapproachestodefiningrisk:•  Economics:Expectedu2litytheoryandalterna2ves•  Finance:Markowitzmean-varianceframework•  Modernrisktheory:axioma2capproachtodefiningrisk(coherent&convexriskmeasures)

•  Systemicrisk:willnottalkaboutittoday.See“ASurveyofSystemicRiskAnalyBcs”byBisias,Flood,Lo,Valavanis,2012.

•  Dynamic/mul2-periodrisk


•  Expresspreferencesoverrandomvariables(loteries)bymappingeachtoasinglenumberthrougha“u2lityfunc2on”.[Bernoulli1738]

RiskI:ExpectedU2lityTheory



•  U2lityexistsassumingindependence,con2nuityaxioms




•  U2lityexistsassumingindependence,con2nuityaxioms•  Experiment:



$48,000$55,000w.p.33%$48,000w.p.66%0w.p.1%

$55,000w.p.33%0w.p.67%

$48,000w.p.34%0w.p.66%


•  U2lityexistsassumingindependence,con2nuityaxioms•  Experiment:

•  Convexu2lity=>risk-lovingpreference•  Concaveu2lity=>risk-aversepreference



$48,000$55,000w.p.33%$48,000w.p.66%0w.p.1%

$55,000w.p.33%0w.p.67%

$48,000w.p.34%0w.p.66%

RiskII:Mean-varianceframework•  Ininvestmentscience,Markowitz(1952)iden2fiedriskwith

thevarianceofaporiolio•  E.g.rou2ngunderuncertainty:

minimize(mean+standarddevia2on)oftravel2meRelated:

minimizevariances.t.meanisatmostathresholdminimizetripbudgets.t.Prob(late)<5%[valueatrisk]

•  Cri2cisms

–  Non-monotonicity(maypreferstochas2callydominatedsolu2ons)–  Non-convexity(conflictsrisk-diversifica2on?)

Algorithms,GameTheory&Risk-averse

DecisionMakingEvdokiaNikolova

RiskIII:Coherentriskmeasures•  Axioma2capproachtotheconstruc2onofriskmeasures

(Artzneretal.1999)–  A1[Monotonicity]:X≥0impliesR(X)≤0–  A2[Convexity]:R(βX+(1-β)Y)≤βR(X)+(1-β)R(Y)–  A3[PosiBvehomogeneity]:R(βX)=βR(X)–  A4[TranslaBoninvariance]:R(X+c)=R(X)–cforallrealc

•  Example:Condi2onalValueatRisk,CVaRα(X):thecondi2onalexpecta2onoflossesthatexceedVaRα(X);Xiscon2nuous.

•  Remark:whenXisnormallydistributed,bothVaRα(X)and

CVaRα(X)areequaltoE[X]–kStdev(X)forappropriateconstantsk.

Algorithms,GameTheory&Risk-averse






EvdokiaNikolova


Algorithmicchallenges•  E.g.,Inshortestpaths,op*malsubstructure(addi*vity)

propertyallowsforefficientdynamicprogrammingalgorithms(Dijkstra,Bellman-Ford,etc)

AB C

5

6.9

5

•  Nonaddi2vity:inrisk-averseshortestpaths,op*malsubstructurefails.

•  Non-convex(concave)objec2veEvdokiaNikolova Algorithms,GameTheory&Risk-averse

DecisionMaking

Algorithmicchallenges

AB C

Mean5St.dev.3

Mean6.9St.dev.1

Mean5St.dev.1

AB-top:mean+st.dev.=8AB-botom:mean+st.dev.=7.9

AC-top:mean+st.dev.=10+√10≈13.2AC-botom:mean+st.dev.=11.9+√2≈13.3

Op2malpath


AlgorithmicinsightsInsight1:Concaveobjec2ve

Insight2:Visualizeonmean-

varianceplaneInsight3:Solu2onisan

extremepointonmean-variancefron2er

minpathmean+r√pathvar{paths}

mean

varMean-VarianceofPaths

c

Exactalgorithmforrisk-averseshortestpathhasrun2men^O(logn).[Nikolova,Kelner,Brand,Mitzenmacher2006]OPEN:Isrisk-averseshortestpathinP?Onplanargraphs?

EvdokiaNikolova


AlgorithmicinsightsInsight1:Concaveobjec2ve

Insight2:Projectonmean-

varianceplaneInsight3:Solu2onisan

extremepoint

minrisk-aversefunc2on{feas.set}

mean

varMean-VarianceofPaths

c

There is an (1+ε)-approximation algorithm for risk-averse problem that uses poly( |input|, 1/ε) queries to exact algorithm for corresponding deterministic problem. [Nikolova2010]

EvdokiaNikolova

Algorithmicchallenges


•  Challengeisnonlinearobjec2vefunc2onovercombinatorialfeasibleset

•  Examplefromelectricitygrid

EvdokiaNikolova

Resistance

Ac2vepower

Reac2vepower

SuccessorsofedgeeinspanningtreeST





EvdokiaNikolova


Risksensi2vityofpriceofanarchy•  [Piliouras,Nikolova,Shamma2013]considerrou2nggameswith

uncertaindelaysresul2ngfrom“uniformschedulers”•  Priceofanarchyoflinearconges2ongamesunderriska{tudes:

–  Wald’sminimaxcost 2–  Savage’sminimaxregret [4/3,1]–  MinimizingExpectedcost 5/3–  Averagecaseanalysis 5/3–  Win-or-Go-Home unbounded –  Secondmomentmethod unbounded

•  Conclusion:RiskcriBcallyaffectspredicBonsofsystemperformance

•  Relatedworkonrisk-aversioninrou2nggames:Ordóñez&S2er-Moses’10,Boyles-Kockelman-Waller’10(tolls),Nikolova&S2er-Moses’11-’14,‘15,Nie’11,Angelidakis-Fotakis-Lianeas’13,Comine{-Torico’13,Meir-Parkes’15,Lianeas-Nikolova-S2er-Moses’16,etc.

EvdokiaNikolova

EffectofRiskvsSelfishBehavior?

•  CostoftrafficpaternC(x):centralplannerisrisk-neutral(althoughusersarerisk-averse),soweconsiderthesumofexpectedtravel*mes

•  PriceofRiskAversion:capturesinefficiencyintroducedbyuserrisk-aversionw.r.t.torisk-neutralusers

risk-averseequilibriumrisk-neutralequilibrium

)C(x )C(xsup 0

instancesproblem

r


•  Ingraphswithgeneralmean,variancefunc2onswhereusersminimize(mean+r*variance):

Cost(Risk-averseeq.)≤(1+ηrk)Cost(Risk-neutraleq.)•  η=1forseries-parallelgraphs,η=2forBraessgraph,

η≤|V|/2forageneralgraph

•  [Lianeas,Nikolova,S2er-Moses’16,*]•  Aboveboundis2ghtforgeneralgraphs.•  Boundcanalsobefoundw.r.tlatencyfunc2onclasses:•  Priceofriskaversion≤(1+rk)PoA

Risk-averseselfishrou2ng*


*Lianeas,Nikolova,S2erMoses.“Risk-averseselfishrouBng.”ForthcominginMathema2csofOpera2onsResearch.

•  Ingraphswithgeneralmean,variancefunc2onswhereusersminimize(mean+r*variance):

Cost(Risk-averseeq.)≤(1+ηrk)Cost(Risk-neutraleq.)•  η=1forseries-parallelgraphs,η=2forBraessgraph,

η≤|V|/2forageneralgraph

•  Aboveboundis2ghtforgeneralgraphs.•  Boundcanalsobefoundw.r.tlatencyfunc2onclasses:

Priceofriskaversion≤(1+rk)PoA

Risk-averseselfishrou2ng*


*Lianeas,Nikolova,S2erMoses.“Risk-averseselfishrouBng.”ForthcominginMathema2csofOpera2onsResearch.





EvdokiaNikolova

Effectofriskonmechanismdesign•  Peoplerespondbetertoloteries(examplesin

transporta2on,energy,recycling,etc.)•  Whataretheop2malloteries?I.e.whatistheop2mal

mechanismdesignifpeoplehavegivenrisk-averseorrisk-lovinga{tudes?

•  Workinprogressonrisk-lovingmechanismdesignwithManolisPountourakis,GerYangatUTAus2n.

•  Wanttoknowmore?TalktoManolis(SimonsFellow)Algorithms,GameTheory&Risk-averse


$48,000$55,000w.p.33%$48,000w.p.66%0w.p.1%

$55,000w.p.33%0w.p.67%

$48,000w.p.34%0w.p.66%

Conclusion

•  CStheoristsliketoprovetheorems(andhaveguaranteesonalgorithmrun2mes,etc)

•  CStheoristslike“cleanformula2ons”thatabstractawaymanyengineeringdetailssoastofindandunderstandcorealgorithmicchallenges(reducetotoypuzzles)

•  Openques2ons?–  Dynamic/Adap2vemodels?(streamingalgorithms,learningalgorithmsforreal-2medecisionmaking)


algorithms, game theory and risk-averse decision making*...shortest paths example algorithms, game...

Documents