PERTEMUAN 26

Markov Chains and Random Walks

Fundamental Theorem of Markov Chains

If Mg is an irreducible, aperiodic Markov Chain:1. All states are positive reccurent.2. Pk converges to W, where each row of W is the same (and

equal to , say)3. is the unique vector for which P =

Fundamental Theorem of Markov Chains

Let Mg be a Markov Chain with states S0…Sn

The Fundamental Theorem tells us that after a sufficiently large number of time steps, the probability of being in state Si+1 is the same as being in state Si.

This steady-state condition is known as a stationary distributionThe rate at which a Markov Chain converges to a stationary distribution is called the mixing rate.

Random Walks

A Random Walk on connected, undirected, non-bipartite Graph G can be modeled as a Markov Chain Mg, where the vertices of the Graph, V(G), are represented by the states of the the Markov Chain and the transition matrix is as follows

0

)(1udPuv

If (u,v) is a member of E

otherwise

Random WalksMg is irreducible because G is connected

Mg is aperiodic Periodicity is the GCD of the length of all closed walks on G Since G is undirected, there exist closed walks of length 2 (u,v E,

exists walk u-v-u) Since G is non-bipartite it contains odd cycles Therefore GCD of all closed walks is 1 Mg is aperiodic

Random WalksGiven that Mg is aperiodic and irreducible, we can apply the Fundamental Theorem of Markov Chains and deduce that Mg converges to a stationary distribution. Lemma:

For all v V, v = d(v) / 2 |E| ( d(v) = the degree of v)Proof

||2)(

||21

)(1*

||2)(

),(

),(

Evd

E

udEud

PP

Evu

Evu

uuuv

denote the component corresponding to vertex v in the

probability vector P

Hitting time (huv) – expected number of steps in a Random Walk that starts at u and ends upon its first visit to vCommute time (cuv) -- expected number of steps in a Random Walk that starts at u, visitsv once and returns to u. (cuv = huv + hvu)

The Lollipop Graph

Lollipop Graph consists of n vertices A clique on n/2 vertices A path on n/2 vertices Let u,v V, u is in the clique, v is at the far end

of the path. Surprisingly, huv != hvu

(huv is (n3) hvu is (n2)

Markov Chains: an Application

Link Prediction and Path Analysis using Markov Chains Use Markov Chains to perform probabilistic analysis and modeling

of weblink sequences; ie. If a user requests page n, what will be her most likely next choice

Possible Applications Web Server Request Prediction Adaptive Web Navigation Tour Generation Personalized Hub

Model can be used in adaptive mode; transition matrix can be updated as new data (example: Web Server Request) arrives

Markov Chains: an Application

Link Prediction and Path Analysis using Markov ChainsSystem Overview

Markov Chains: an Application

Experimental Results HTTP Server Request Prediction

6572 URIs (including html documents, directories, gifs, and cgi requests) 40,000 Requests Over 50% of the web server requests can be predicted to be the state with

the highest probability

ReferencesL. Lovasz. Random Walks on Graphs: A Survey. Combinatorics: Paul Erdos is Eighty (vol. 2), 1996, pp. 353-398. (http://www.cs.yale.edu/HTML/YALE/CS/HyPlans/lovasz/erdos.ps) R. Sarukkai, "Link Prediction and Path Analysis Using Markov Chains: 9th World Wide Wide Conference, May, 2000. (http://www9.org/w9cdrom/68/68.html)Introduction to Markov chains : with special emphasis on rapid mixing / Ehrhard Behrends. Germany [1990-onward] Vieweg & Sohn, GW Am Math Soc 2000.