chapter 1 - introduction and review of probability theory
TRANSCRIPT
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Chapter 1:
Introduction and Review of Probability Theory
Department of Statistics and Actuarial Science
The University of Hong Kong
STAT 2803 / 3903
Stochastic Models
2015-2016 (2nd Semester)
1 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
1 Overview of The Course: Stochastic Process
Overview of the Course: Stochastic Process
Outline
2 Probability and Random Variables
3 Independence
4 Conditional Probability and Conditional Expectation
5 First Step Analysis
6 Probability Generating Function
2 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Stochastic Process
A collection of random variables, indexed by an ordered subset of real numbers:
Overview of The Course: Stochastic Process Overview of The Course: Stochastic Process
• The set S of all possible values that the random variables X(t) can take is called the state space of the process.
• The index t is often, but not necessarily, interpreted as time.
• The index set T of the process, which is an ordered subset of real numbers, can be discrete or continuous.
• For each t T, X(t) is a random variable that follows a particular distribution.
• Stochastic processes are used to model the evolution of physical processes through time. The random variable X(t) is the random state of the process at time t.
• Each realization of the stochastic process, x(t), is a function of t. Therefore a stochastic process can be viewed as a random function of time t.
3 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Examples of Stochastic Processes
• Xn = accumulated total points after rolling a dice n times, n = 1, 2, …
Overview of The Course: Stochastic Process Overview of The Course: Stochastic Process
discrete time T = {1, 2, …}, discrete state space S = {1, 2, …} stochastic process
Markov Chain
4 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Examples of Stochastic Processes
• X(t) = total number of customers visited a supermarket by time t
Overview of The Course: Stochastic Process Overview of The Course: Stochastic Process
continuous time T , discrete state space S = {0, 1, 2, …} stochastic process
Poisson Process
• X(t) = closing price of a stock at the end of day t
discrete time T = {1, 2, …}, continuous state space S (0, ) stochastic process
Time Series
• X(t) = x-coordinate of the 3D location of a gas molecule at time t
continuous time T , continuous state space S stochastic process
Brownian Motion
5 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
• Required knowledge in mathematics
Matrix algebra
Calculus – univariate and multivariable
Differential equation (better, but not a must)
Setting the Stage Overview of The Course: Stochastic Process
• Required knowledge in probability theory
Conditional probability and conditional expectation
Distributions: Binomial, Poisson, Exponential, Gamma, Normal
Moment generating function
• Pre-study suggestion
Do some revision on STAT1801/2901 course materials
Read Chapter 1-3 of the text book (Introduction to Probability Models, by Sheldon M. Ross.)
6 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Outline
Probability and Random Variables
7 / 55
1 Overview of The Course: Stochastic Process
2 Probability and Random Variables
3 Independence
4 Conditional Probability and Conditional Expectation
5 First Step Analysis
6 Probability Generating Function
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
• Probability defined on events
Probability Probability and Random Variables
For each event E of the sample space , the probability of event E is defined as a set function P(E), that satisfies the following conditions:
1
2
3 (Countable Additivity) For any sequence of events E1, E2, … that are mutually exclusive (En Em = when n m), then
8 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
• Properties of probability
Impossible event
Bounded
Non-occurrence
Implication relation
Boole’s inequality
Probability Probability and Random Variables
9 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
• Inclusion-Exclusion Principle
Probability Probability and Random Variables
• Allows us to calculate probability of union from probabilities of intersections.
10 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
• Example: An urn contains 2n balls labelled 1 to n in pairs. If we keep drawing balls two at a time without replacement. What is the probability that at the end, at least one matched pair can be drawn?
Probability Probability and Random Variables
1
1
2
2
n
n
?
?
Let Ei = i th drawn pair matches
Probability of drawing at least one pair:
11 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
• Random Variable
a real-valued function of the outcomes in performing an experiment, i.e. defined on the sample space .
X is discrete if it takes on countable number of possible values.
X is continuous if it takes on a continuum (e.g. interval) of possible values.
Random Variable Probability and Random Variables
• Cumulative distribution function (cdf)
Non-decreasing
Right continuous
Limits
12 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
• If a random variable X takes on the values x1 , x2 , …, then the probability mass function (pmf) of X is defined as
Discrete Random Variable Probability and Random Variables
• Cumulative distribution function
• Expected value
13 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
• For continuous random variable X, the probability density function (pdf) of X is a nonnegative function defined on (– , ), such that for any set B of real numbers,
Continuous Random Variable Probability and Random Variables
• Cumulative distribution function
• Expected value
14 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
• The long-term average of a random variable X is represented by its (population) mean:
Mean, Variance, and Moments Probability and Random Variables
• The spread/variation of a random variable X is represented by its (population) variance:
• Formulae under linear transformation
• Other behaviors of a random variables are described by moments:
15 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
• A useful alternative specification of a random variable X is the moment generating function (mgf):
Moment Generating Function Probability and Random Variables
• Generate moments:
• The mgf, if exists, uniquely characterizes the distribution.
• Formulae under linear transformation
16 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
• A Bernoulli random variable X takes only values 0 or 1. The pmf is simply
where p is often called the success probability.
Binomial Distribution Probability and Random Variables
Number of ‘successes’ out of n independent Bernoulli trials (with the same p).
Sum of n independent Bernoulli random variables (with the same p).
Notation:
• A binomial random variable X takes values from {0, 1, …, n}, with pmf
17 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Binomial Distribution Probability and Random Variables
• Moment generating function
• Mean and Variance
18 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Poisson Distribution Probability and Random Variables
Common model of random counts, e.g. number of accidents, defects, injuries, insurance claims, …, etc.
Notation:
Approximation to binomial distribution:
• A Poisson random variable X takes values from {0, 1, 2, …}, with pmf
where > 0 is the parameter.
19 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Poisson Distribution Probability and Random Variables
• Moment generating function
• Mean and Variance
20 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Exponential Distribution Probability and Random Variables
Common model of waiting time, e.g. lifetime, failure time, decay time of radioactive particle, interarrival times of insurance claims, …, etc.
Notation:
Important distribution for studying Poisson process and continuous time Markov chain.
Memoryless property:
• An Exponential random variable X takes values from (0, ), with pdf and cdf
where > 0 is the parameter.
21 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Exponential Distribution Probability and Random Variables
• Moment generating function
• Mean and Variance
• Moments
22 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Gamma Distribution Probability and Random Variables
Common model of nonnegative random quantities positively skewed, e.g. size of insurance claims, rainfalls, accumulative waiting times, …, etc.
Notation:
Closely related with the exponential distribution:
Exponential is a special case of gamma with = 1.
Sum of independent and identically distributed exponential random variables is distributed as gamma.
• A Gamma random variable X takes values from (0, ), with pdf
where > 0, > 0 are the parameters, and () is the Gamma function with properties , , and .
23 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Gamma Distribution Probability and Random Variables
• Moment generating function
• Mean and Variance
24 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Normal Distribution Probability and Random Variables
Common model of naturally occurring variables with symmetric random noises, e.g. heights, weights, blood pressures of adult humans, measurement errors, rates of change of logarithm of stock market indices, …, etc.
Notation:
Limiting distribution of sample mean, by central limit theorem (CLT).
Key component of the Brownian motion.
• A Normal (Gaussian) random variable X takes values from , with pdf
where – < < , > 0 are the parameters.
25 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Normal Distribution Probability and Random Variables
• Moment generating function
• Mean and Variance
• Linear transformation
• Standardization
26 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Jointly Distributed Random Variables Probability and Random Variables
• Random Vector
vector-valued function that maps the outcomes in to p-dimensional vectors in p.
X is discrete if it takes on countable number of possible values.
X is continuous if it takes on a continuum (e.g. intervals) of possible values.
• Joint cdf
• Joint pmf (discrete)
• Joint pdf (continuous)
27 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Jointly Distributed Random Variables Probability and Random Variables
• Marginal pmf (discrete)
• Marginal pdf (continuous)
• Expected value
28 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Outline
Independence
29 / 55
1 Overview of The Course: Stochastic Process
2 Probability and Random Variables
3 Independence
4 Conditional Probability and Conditional Expectation
5 First Step Analysis
6 Probability Generating Function
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Independence Independence
• Independence of events
The events E1, E2, …, En are said to be (mutually) independent if for every subset Ei1
, Ei2, …, Eir
(i1 < i2 < <ir, 2 r n) of these events,
Meaning: information (of the occurrences) on any subset of events tells us nothing about the others.
Important remark: pairwise independence
does not imply mutual independence.
30 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Independence Independence
• Independence of random variables
The random variables X1, X2, …, Xn are said to be independent if
for any real numbers x1, x2, …, xn.
Meaning: information on any subset of the random variables tells us nothing about the others.
Note that it implies the mutual independence of the events
for any sets A1, A2, …, An of real numbers.
31 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Independence Independence
• Independence of random variables
The random variables X1, X2, …, Xn are independent if and only if
or
for any real numbers x1, x2, …, xn.
(discrete case)
(continuous case)
• If the random variables X1, X2, …, Xn are independent, then X1, X2, …, Xk
are independent for any integer k such that 2 k n – 1.
• If the random variables X1, X2, …, Xn are independent, then g(X1, X2, …, Xk)
and h(Xk+1, Xk+2, …, Xn) are independent for any real-valued functions g and h, and integer k such that 2 k n – 1.
32 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Outline
Conditional Probability and Conditional Expectation
33 / 55
1 Overview of The Course: Stochastic Process
2 Probability and Random Variables
3 Independence
4 Conditional Probability and Conditional Expectation
5 First Step Analysis
6 Probability Generating Function
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Conditional Probability Conditional Probability and Conditional Expectation
• Suppose A and B are two events such that P(B)>0, the conditional probability that A occurs given that B has occurred is defined by
It allows us to access the uncertainty of event A, based on additional information from event B.
• If the events A and B are independent, then
That is, knowledge of occurrence of one event does not affect the probability of occurrence of the other event.
34 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Law of Total Probability Conditional Probability and Conditional Expectation
• Law of Total Probability
Suppose B1, B2, …, Bn are mutually exclusive ( ) and exhaustive ( ) events, then for any event A,
It relates the marginal probabilities to conditional probabilities.
It can also be stated for conditional probabilities:
35 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
• Conditional Independence
Suppose A, B, C are events such that P(C) > 0. If
then A and B are said to be conditionally independent given C.
A restrictive form of independence between A and B under the condition that C has occurred.
Conditional Independence Conditional Probability and Conditional Expectation
• An important concept to understand the Markov property, which is essential in studying stochastic processes.
• Markov property: the future state and the past history are conditionally independent, given the current state.
36 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Conditional Independence Conditional Probability and Conditional Expectation
• Example: There are two coins in a bag, one is unbiased and the other one is biased with Head probability 0.9. A coin is randomly drawn and then flipped twice independently. Consider the events
A : 1st flip results in a Head B : 2nd flip results in a Head F : the selected coin is biased
• Obviously A and B are conditionally independent, given F (or F c).
• Hence A and B are not independent when no information of F is obtained. In other words, A depends on B (or vice versa) through F.
• Conditional independence does not imply unconditional independence, and vice versa.
37 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Bayes’ Formula Conditional Probability and Conditional Expectation
• Bayes’ Formula
Suppose A and B are events such that P(A) > 0, P(B) > 0, then
It enable us to evaluate the ‘inverse probability’ (conditional probability with events in reversed order).
It suggests a way to adjust the prior probability P(B) into the posterior probability P(B|A) after observed information from A.
It is often used together with the law of total probability:
38 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Bayes’ Formula Conditional Probability and Conditional Expectation
• Example: There are two coins in a bag, one is unbiased and the other one is biased with Head probability 0.9. A coin is randomly drawn and then flipped twice independently. Consider the events
A : 1st flip results in a Head B : 2nd flip results in a Head F : the selected coin is biased
• The coin is more likely to be biased (76.4%) if both flips resulted in Heads.
39 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Conditional Expectation Conditional Probability and Conditional Expectation
• For two random variables X and Y, the conditional distribution of X given Y = y is the distribution of X for the sub-population corresponding to the constraint Y = y.
• (Discrete case) Conditional pmf
• (Discrete case) Conditional pdf
• Conditional Expectation: expected value evaluated from sub-population:
• Note that h(y) = E(g(X)|Y = y) is a function of y. Therefore h(Y) = E(g(X)|Y) is a function of Y and hence a random variable.
40 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Conditional Expectation Conditional Probability and Conditional Expectation
• Conditional Mean
• Conditional Variance
• Properties
Linear transformation
If X and Y are independent, then
41 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Conditional Expectation Conditional Probability and Conditional Expectation
• Law of Total Expectation
It enables us to calculate expectations iteratively.
The law of total probability is a special case.
42 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Conditional Expectation Conditional Probability and Conditional Expectation
• Law of Total Variance / Variance Decomposition
It enables us to calculate variances iteratively.
The expectation E(Var(X|Y)) can be regarded as a measure of the within-group variation of X in the sup-populations defined by Y.
The variance Var(E(X|Y)) can be regarded as a measure of the between-group variation of X among the sup-populations defined by Y.
The formula can be interpreted as a decomposition of variation into components that explains the within-group and between-group variations.
43 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Conditional Expectation Conditional Probability and Conditional Expectation
• Example: The number of claims, N, on a block of insurance policies in a year follows (). Given N = n, the claim sizes (in $1000) are distributed as Gamma(9, 0.06) independently. What is the mean and variance of the total claim amount in a year?
Number of claims:
Claim sizes:
conditionally independent
Total claim amount: random sum
44 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Conditional Expectation Conditional Probability and Conditional Expectation
Number of claims:
Claim sizes:
(conditionally independent)
45 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Outline
First Step Analysis
46 / 55
1 Overview of The Course: Stochastic Process
2 Probability and Random Variables
3 Independence
4 Conditional Probability and Conditional Expectation
5 First Step Analysis
6 Probability Generating Function
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
First Step Analysis First Step Analysis
• The first step analysis (FSA) is a useful technique for solving probability problems with hierarchical structures and Markov property.
Original problem
Intermediate stage
Case 1
Case 2
Case k
Result 1
Result 2
• Set up recursion equation(s) and solve.
47 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
First Step Analysis
• Example: Suppose the number of a man’s sons follows a (1.5) distribution, and the sons will independently have number of son’s following the same distribution, and so on. What is the probability that their surnames will eventually die out?
First step analysis: consider the first generation of spread
E : event that their surnames will eventually die out
Solving by Newton’s method:
• A special case of Branching Process, known as the Galton-Watson Process.
First Step Analysis
48 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
First Step Analysis
• Example: A biased coin with Head probability p is repeatedly flipped independently, until k consecutive Heads are obtained. What is the mean number of flips?
= Number of flips to obtain k consecutive Heads
First Step Analysis : consider the first time when (k – 1) consecutive Heads are obtained, i.e. the coin was flipped times, then
if the next flip is Head, k consecutive Heads are obtained;
If the next flip is Tail, we restart.
First Step Analysis
49 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Outline
Probability Generating Function
50 / 55
1 Overview of The Course: Stochastic Process
2 Probability and Random Variables
3 Independence
4 Conditional Probability and Conditional Expectation
5 First Step Analysis
6 Probability Generating Function
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Probability Generating Function
• Let X be a nonnegative and integer-valued random variable. Then the probability generating function (pgf) of X is defined by
Probability Generating Function
• Generate probabilities:
• Like mgf, the pgf uniquely characterizes the distribution.
• Relationship with the mgf:
51 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Probability Generating Function
• Example: Binomial distribution
Probability Generating Function
• Example: Poisson distribution
• Example: Sum of Independent Random Variables
52 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Probability Generating Function
• Example: Random Sum where Xi are i.i.d. given N
Probability Generating Function
Therefore the pgf of the random sum is the composite function of the pgfs of N and X, under the iid assumption.
53 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Probability Generating Function Probability Generating Function
• The probability generating function is useful in solving differential equations and recurrence equations, which are often encountered in studying stochastic processes.
• Example: Suppose a stochastic process has pmfs satisfying
What is the distribution of X(t) ?
54 / 55
2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models
Probability Generating Function Probability Generating Function
• To solve the differential equation, we can multiply both side by and obtain an equation in terms of probability generating function:
• Hence which is the Poisson Process.
55 / 55