Chapter 5 Probability Models

Introduction– Modeling situations that involve an element of chance– Either independent or state variables is probability or random

variable– Markov chain– Random variables– Statistics, system reliability, – Game theory– Casino, ……

An example

The Problem --- Consider a car rental company with branches in Orlando and Tampa. Each branch rents car to Florida tourists. The company specializes in catering to travel agents who want to arrange tourist activities in both Orlando and Tampa. Consequently, a traveler will rent a car in one city and drop off the car in either city. Travelers begin their itinerary in either city. Cars can be returned to either location, which can cause an imbalance in available cars to rent. Historical data on the percentages of cars rented and returned to these companies is collected from the previous few years as shown here.

An example – transition matrix

An example – transition matrix

The recorded data form a transition matrix and show that the probability for– Returning a car to Orlando that was rented in Orlando is 0.6;

whereas the probability it will be returned in Tampa is 0.4.– Returning a car to Orlando that was rented in Tampa is 0.3;

whereas the probability it will be returned in Tampa is 0.7.There are two states – Orlando and Tampa– The sum of the probabilities for transitioning from a present

state to next state, which is the sum of the probabilities in each row, ==1--all possible outcomes are taken into account.

An example -- model

Variables– pn --- the percentage of cars available to rent in Orlando at the

end of period n– qn --- the percentage of cars available to rent in Tampa at the

end of period n

Probabilistic model1 , 0,1,2,

0.6 0.4,

0.3 0.7

Tn n

nn

n

P Q P n

pP Q

q

1

1

0.6 0.3

0.4 0.7 , 0,1,2,n n n

n n n

p p q

q p q n

An example – model solution

An example – model solution

An example – model interpretation

If the two branches begin the year with a total of n cars, after 14 time periods or days approximately 57% of cars will be in Tampa and 43% will be in Orlando. Starting with 100 cars in each location, about 114 cars will be based out of Tampa and 86 will be based out of Orlando in the steady state (and it would take about 5 days to reach this state)

Markov chain

A Markov chain is a process in which there are the same finite number of states or outcomes that can be occupied at any given time. The states do not overlap and cover all possible outcomes. In a Markov process, the system may move from one state to another one at each time step, and there is a probability associated with this transition for each possible outcome. The sum of the probabilities for transitioning from the present state to the next state is equal to 1 for each state at each time step.

A Markov process with two states

1 transition matrix

1

p pQ

q q

Markov chain

A sequence of events with the following properties:– An event has a finite number of outcomes, called states. The

process is always in one of these states– At each stage or period of the process, a particular outcome

can transition from its present state to any other state or remain in the same state.

– The probability of going from one state to another in a single state is represented by a transition matrix for which the entries in each row lie between 0 and 1; each row sums to 1. These probabilities depend only on the present state and not on past states.

Application of Markov chain

In biology– Color of a plant root --- yellow or green– Color of pig hair --- black or white

These can be explained by Markov chainThe outside character of a species is determined by its genome which can be divided into two kinds – dominate (d) and reminder (r). For each outside character, there is two genomes and each can be either d or r.

Application of Markov chain

Three combinations– d d --- good --- denoted by D – d r --- mixture -- denoted by H– r r --- worse --- denoted by R

From the biology theory– With either d d or d r genome combination, the outside

character shows dominate (or d), e.g. green in plant.– With r r genome combination, the outside character shows

reminder (or r), e.g. yellow in plant.

Application of Markov chain

In biology theory– A child randomly picks up one genome from the two

genomes of his father and one genome from the two genomes of his mother to form its own genome.

– Three states: D, H and R• Case 1: Parents are both D, child is D• Case 2: Parents are D+H, child is either D or H with

probability ½ and ½• Case 3: Parents are H+H, child is D or H or R with

probability ¼, ½ and ¼

Application of Markov chain

Problem 1 – Marriage with mixture state– Random pick up one, marry it with a mixture H, for their

children, marry them again with mixture, and continue. What is probability of children shows outside characters d and r?

Solution: There are three states – D, H & R

For each marry, the transition matrix is

0.5 0.5 01 / 2 1 / 2 0

0.25 0.5 0.251 / 4 1 / 2 1 / 4

0 0.5 0.50 1 / 2 1 / 2

D H R

DQ

H

R

Application of Markov chain

Model

Limiting behavior

Conclusion – after many generations– The probability of children shows outside character d is

0.25+0.5 =0.75, and outside character r is 0.25.– This theoretical results agree well with experiments!!

1 2 3 1 2 3( , , ) & 1

(0.25,0.5,0.25)

T T

T

Q w w w w w w w w

w

1

1

1

0.5 0.25 0

0.5 0.5 0.5

0 0.25 0.5

n n nT

n n n

n n n

D D D

H Q H H

R R R

Application of Markov chain

Problem 2 – Marriage with relatives– The parents at the beginning can be either good,

mixture or worse, they have many children. The marriage is only taken between children and continue. What happens after many generations?

States of parents --- D, H or RStates of children – DD, RR, DH, DR, HH, HR

Transition matrix

1 0 0 0 0 0

0 1 0 0 0 0

1 / 4 0 1 / 2 0 1 / 4 0

0 0 0 0 1 0

1 /16 1 /16 1 / 4 1 / 8 1 / 4 1 / 4

0 1 / 4 0 0 1 / 4 1 / 2

DD RR DH DR HH HR

DD

RR

QDH

DR

HH

HR

Application

Model

Limiting behavior, when n is very large

Conclusion – After several generations, all children will be either only good or only worse and will keep it forever!! No biodiversity!!

1

1 2 3 4 5 6

, 0,1,2, with

( , , , , , )

Tn n

T

P Q P n

P p p p p p p

1 2 3 4 5 6& 1

(1,0,0,0,0,0) or (0,1,0,0,0,0)

T

T T

Q p p p p p p p p

p p

System reliability

Recent Toyota call home problemComputer, automobile, etc. are complicated system! If they perform well for a reasonably long period of time, we say these system are reliable!!The reliability of a component or system is the probability that it will not fail over a specific time period.Define f(t) to be the failure rate of an item, component, or system over time t, thus f(t) is a probability distribution.

System reliability

Let F(t) be the cumulative distribution function corresponding to f(t).Define the reliability of the item, component, or system by

For a complicated system, if we know the reliability of each component, then we can build simple models to examine the reliability of complex systems.

( ) 1 ( )R t F t

System reliability

A complicated system consists of many small parts via – series, parallel or combinationsConnected by series – A series system is consisted of several independent components and it becomes failure due to the failure of any one of the independent component – An example: A NASA space shuttle’s rocket propulsion system

System reliability

– System reliability

Connected by parallel – it is a system that performs as long as a single one of its components remains operational.– An example:

System reliability

– System reliability

Connected by combination – combining series and parallel relationships

System reliability

Exercise – find the system reliability