stats 241.3 probability theory summary. the sample space, s the sample space, s, for a random...
TRANSCRIPT
![Page 1: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/1.jpg)
Stats 241.3
Probability Theory
Summary
![Page 2: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/2.jpg)
The sample Space, S
The sample space, S, for a random phenomena is the set of all possible outcomes.
![Page 3: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/3.jpg)
An Event , E
The event, E, is any subset of the sample space, S. i.e. any set of outcomes (not necessarily all outcomes) of the random phenomena
S
E
![Page 4: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/4.jpg)
Probability
![Page 5: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/5.jpg)
Suppose we are observing a random phenomena
Let S denote the sample space for the phenomena, the set of all possible outcomes.
An event E is a subset of S.
A probability measure P is defined on S by defining for each event E, P[E] with the following properties
1. P[E] ≥ 0, for each E.
2. P[S] = 1.
3. if for all ,i i i iii
P E P E E E i j
1 2 1 2P E E P E P E
![Page 6: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/6.jpg)
Finite uniform probability space
Many examples fall into this category
1. Finite number of outcomes
2. All outcomes are equally likely
3.
no. of outcomes in =
total no. of outcomes
n E n E EP E
n S N
: = no. of elements of n A ANote
To handle problems in case we have to be able to count. Count n(E) and n(S).
![Page 7: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/7.jpg)
Techniques for counting
![Page 8: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/8.jpg)
Basic Rule of countingSuppose we carry out k operations in sequence
Letn1 = the number of ways the first operation can be
performed
ni = the number of ways the ith operation can be performed once the first (i - 1) operations have been completed. i = 2, 3, … , k
Then N = n1n2 … nk = the number of ways the k operations can be performed in sequence.
![Page 9: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/9.jpg)
1n
2nDiagram: 3n
2n
2n
![Page 10: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/10.jpg)
Basic Counting Formulae1. Permutations: How many ways can you order n
objects
n!2. Permutations of size k (< n): How many ways can you
choose k objects from n objects in a specific order
!
= 1 1!n k
nP n n n k
n k
![Page 11: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/11.jpg)
3. Combinations of size k ( ≤ n): A combination of size k chosen from n objects is a subset of size k where the order of selection is irrelevant. How many ways can you choose a combination of size k objects from n objects (order of selection is irrelevant)
n k
nC
k
1 2 1!
! ! 1 2 1
n n n n kn
n k k k k k
![Page 12: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/12.jpg)
Important Notes
1. In combinations ordering is irrelevant. Different orderings result in the same combination.
2. In permutations order is relevant. Different orderings result in the different permutations.
![Page 13: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/13.jpg)
Rules of Probability
![Page 14: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/14.jpg)
The additive rule
P[A B] = P[A] + P[B] – P[A B]
and
if P[A B] = P[A B] = P[A] + P[B]
![Page 15: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/15.jpg)
The additive rule for more than two events
then
and if Ai Aj = for all i ≠ j.
11
n n
i i i ji i ji
P A P A P A A
i j ki j k
P A A A
1
1 21n
nP A A A
11
n n
i iii
P A P A
![Page 16: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/16.jpg)
The Rule for complements
for any event E
1P E P E
![Page 17: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/17.jpg)
Conditional Probability,Independence
andThe Multiplicative Rue
![Page 18: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/18.jpg)
Then the conditional probability of A given B is defined to be:
P A BP A B
P B
if 0P B
![Page 19: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/19.jpg)
if 0
if 0
P A P B A P AP A B
P B P A B P B
The multiplicative rule of probability
and
P A B P A P B
if A and B are independent.
This is the definition of independent
![Page 20: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/20.jpg)
1 2 nP A A A
The multiplicative rule for more than two events
1 2 1 3 2 1P A P A A P A A A
1 2 1n n nP A A A A
![Page 21: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/21.jpg)
Independencefor more than 2 events
![Page 22: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/22.jpg)
Definition:
The set of k events A1, A2, … , Ak are called mutually independent if:
P[Ai1 ∩ Ai2 ∩… ∩ Aim
] = P[Ai1] P[Ai2
] …P[Aim]
For every subset {i1, i2, … , im } of {1, 2, …, k }
i.e. for k = 3 A1, A2, … , Ak are mutually independent if:
P[A1 ∩ A2] = P[A1] P[A2], P[A1 ∩ A3] = P[A1] P[A3],
P[A2 ∩ A3] = P[A2] P[A3],
P[A1 ∩ A2 ∩ A3] = P[A1] P[A2] P[A3]
![Page 23: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/23.jpg)
Definition:
The set of k events A1, A2, … , Ak are called pairwise independent if:
P[Ai ∩ Aj] = P[Ai] P[Aj] for all i and j.
i.e. for k = 3 A1, A2, … , Ak are pairwise independent if:
P[A1 ∩ A2] = P[A1] P[A2], P[A1 ∩ A3] = P[A1] P[A3],
P[A2 ∩ A3] = P[A2] P[A3],
It is not necessarily true that P[A1 ∩ A2 ∩ A3] = P[A1]
P[A2] P[A3]
![Page 24: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/24.jpg)
Bayes Rule for probability
P A P B AP A B
P A P B A P A P B A
![Page 25: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/25.jpg)
Let A1, A2 , … , Ak denote a set of events such that
1 1
i ii
k k
P A P B AP A B
P A P B A P A P B A
An generalization of Bayes Rule
1 2 and k i jS A A A A A
for all i and j. Then
![Page 26: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/26.jpg)
Random Variables
an important concept in probability
![Page 27: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/27.jpg)
A random variable , X, is a numerical quantity whose value is determined be a random experiment
![Page 28: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/28.jpg)
Definition – The probability function, p(x), of a random variable, X.
For any random variable, X, and any real number, x, we define
p x P X x P X x
where {X = x} = the set of all outcomes (event) with X = x.
For continuous random variables p(x) = 0 for all values of x.
![Page 29: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/29.jpg)
Definition – The cumulative distribution function, F(x), of a random variable, X.
For any random variable, X, and any real number, x, we define
F x P X x P X x
where {X ≤ x} = the set of all outcomes (event) with X ≤ x.
![Page 30: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/30.jpg)
Discrete Random Variables
For a discrete random variable X the probability distribution is described by the probability function p(x), which has the following properties
1
2. 1ix i
p x p x
1. 0 1p x
3. a x b
P a x b p x
![Page 31: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/31.jpg)
Graph: Discrete Random Variable
p(x)
a x b
P a x b p x
a b
![Page 32: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/32.jpg)
Continuous random variables
For a continuous random variable X the probability distribution is described by the probability density function f(x), which has the following properties :
1. f(x) ≥ 0
2. 1.f x dx
3. .
b
a
P a X b f x dx
![Page 33: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/33.jpg)
Graph: Continuous Random Variableprobability density function, f(x)
1.f x dx
.b
a
P a X b f x dx
![Page 34: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/34.jpg)
The distribution function F(x)
This is defined for any random variable, X.
F(x) = P[X ≤ x]
Properties
1. F(-∞) = 0 and F(∞) = 1.
2. F(x) is non-decreasing (i. e. if x1 < x2 then F(x1) ≤ F(x2) )
3. F(b) – F(a) = P[a < X ≤ b].
![Page 35: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/35.jpg)
4. p(x) = P[X = x] =F(x) – F(x-)
5. If p(x) = 0 for all x (i.e. X is continuous) then F(x) is continuous.
Here limu x
F x F u
![Page 36: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/36.jpg)
6. For Discrete Random Variables
F(x) is a non-decreasing step function with
u x
F x P X x p u
jump in at .p x F x F x F x x
0 and 1F F
0
0.2
0.4
0.6
0.8
1
1.2
-1 0 1 2 3 4
F(x)
p(x)
![Page 37: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/37.jpg)
7. For Continuous Random Variables Variables
F(x) is a non-decreasing continuous function with
x
F x P X x f u du
.f x F x
0 and 1F F F(x)
f(x) slope
0
1
-1 0 1 2x
To find the probability density function, f(x), one first finds F(x) then .f x F x
![Page 38: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/38.jpg)
Some Important Discrete distributions
![Page 39: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/39.jpg)
The Bernoulli distribution
![Page 40: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/40.jpg)
Suppose that we have a experiment that has two outcomes
1. Success (S)2. Failure (F)
These terms are used in reliability testing.Suppose that p is the probability of success (S) and q = 1 – p is the probability of failure (F)This experiment is sometimes called a Bernoulli Trial
Let 0 if the outcome is F
1 if the outcome is SX
Then 0
1
q xp x P X x
p x
![Page 41: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/41.jpg)
The probability distribution with probability function
is called the Bernoulli distribution
0
1
q xp x P X x
p x
0
0.2
0.4
0.6
0.8
1
0 1
p
q = 1- p
![Page 42: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/42.jpg)
The Binomial distribution
![Page 43: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/43.jpg)
We observe a Bernoulli trial (S,F) n times.
0,1,2, ,x n xnp x P X x p q x n
x
where
Let X denote the number of successes in the n trials.Then X has a binomial distribution, i. e.
1. p = the probability of success (S), and2. q = 1 – p = the probability of failure (F)
![Page 44: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/44.jpg)
The Poisson distribution
• Suppose events are occurring randomly and uniformly in time.
• Let X be the number of events occuring in a fixed period of time. Then X will have a Poisson distribution with parameter .
0,1,2,3,4,!
x
p x e xx
![Page 45: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/45.jpg)
The Geometric distribution
Suppose a Bernoulli trial (S,F) is repeated until a success occurs.
X = the trial on which the first success (S) occurs.
The probability function of X is:
p(x) =P[X = x] = (1 – p)x – 1p = p qx - 1
![Page 46: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/46.jpg)
The Negative Binomial distribution
Suppose a Bernoulli trial (S,F) is repeated until k successes occur.
Let X = the trial on which the kth success (S) occurs.
The probability function of X is:
1 , 1, 2,
1k x kx
p x P X x p q x k k kk
![Page 47: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/47.jpg)
The Hypergeometric distribution
Suppose we have a population containing N objects.
Suppose the elements of the population are partitioned into two groups. Let a = the number of elements in group A and let b = the number of elements in the other group (group B). Note N = a + b.
Now suppose that n elements are selected from the population at random. Let X denote the elements from group A.
The probability distribution of X is
a b
x n xp x P X x
N
n
![Page 48: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/48.jpg)
Continuous Distributions
![Page 49: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/49.jpg)
Continuous random variables
For a continuous random variable X the probability distribution is described by the probability density function f(x), which has the following properties :
1. f(x) ≥ 0
2. 1.f x dx
3. .
b
a
P a X b f x dx
![Page 50: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/50.jpg)
Graph: Continuous Random Variableprobability density function, f(x)
1.f x dx
.b
a
P a X b f x dx
![Page 51: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/51.jpg)
0
0.1
0.2
0.3
0.4
0 5 10 15
1
b a
a b
f x
x0
0.1
0.2
0.3
0.4
0 5 10 15
1
b a
a b
f x
x
1
b a
a b
f x
x
Continuous Distributions
The Uniform distribution from a to b
1
0 otherwise
a x bf x b a
![Page 52: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/52.jpg)
The Normal distribution (mean , standard deviation )
2
221
2
x
f x e
![Page 53: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/53.jpg)
0
0.1
0.2
-2 0 2 4 6 8 10
The Exponential distribution
0
0 0
xe xf x
x
![Page 54: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/54.jpg)
The Weibull distribution
A model for the lifetime of objects that do age.
![Page 55: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/55.jpg)
The Weibull distribution with parameters and.
1 0x
f x x e x
![Page 56: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/56.jpg)
The Weibull density, f(x)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0 1 2 3 4 5
( = 0.5, = 2)
( = 0.7, = 2)
( = 0.9, = 2)
![Page 57: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/57.jpg)
The Gamma distribution
An important family of distributions
![Page 58: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/58.jpg)
The Gamma distribution
Let the continuous random variable X have density function:
1 0
0 0
xx e xf x
x
Then X is said to have a Gamma distribution with parameters and .
![Page 59: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/59.jpg)
Graph: The gamma distribution
0
0.1
0.2
0.3
0.4
0 2 4 6 8 10
( = 2, = 0.9)
( = 2, = 0.6)
( = 3, = 0.6)
![Page 60: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/60.jpg)
Comments
1. The set of gamma distributions is a family of distributions (parameterized by and ).
2. Contained within this family are other distributionsa. The Exponential distribution – in this case = 1, the
gamma distribution becomes the exponential distribution with parameter . The exponential distribution arises if we are measuring the lifetime, X, of an object that does not age. It is also used a distribution for waiting times between events occurring uniformly in time.
b. The Chi-square distribution – in the case = /2 and = ½, the gamma distribution becomes the chi- square (2) distribution with degrees of freedom. Later we will see that sum of squares of independent standard normal variates have a chi-square distribution, degrees of freedom = the number of independent terms in the sum of squares.
![Page 61: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/61.jpg)
Expectation
![Page 62: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/62.jpg)
Let X denote a discrete random variable with probability function p(x) (probability density function f(x) if X is continuous) then the expected value of X, E(X) is defined to be:
i ix i
E X xp x x p x
E X xf x dx
and if X is continuous with probability density function f(x)
![Page 63: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/63.jpg)
Expectation of functionsLet X denote a discrete random variable with probability function p(x) then the expected value of X, E[g (X)] is defined to be:
x
E g X g x p x
E X g x f x dx
and if X is continuous with probability density function f(x)
![Page 64: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/64.jpg)
Moments of a Random Variable
![Page 65: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/65.jpg)
the kth moment of X :
kk E X
-
if is discrete
if is continuous
k
x
k
x p x X
x f x dx X
• The first moment of X , = 1 = E(X) is the center of gravity of the distribution of X.
• The higher moments give different information regarding the distribution of X.
![Page 66: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/66.jpg)
the kth central moment of X
0 k
k E X
-
if is discrete
if is continuous
k
x
k
x p x X
x f x dx X
![Page 67: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/67.jpg)
Moment generating functions
![Page 68: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/68.jpg)
Definition
Let X denote a random variable, Then the moment generating function of X , mX(t) is defined by:
if is discrete
if is continuous
tx
xtX
Xtx
e p x X
m t E ee f x dx X
![Page 69: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/69.jpg)
Properties1. mX(0) = 1
0 derivative of at 0.k thX Xm k m t t 2.
kk E X
2 33211 .
2! 3! !kk
Xm t t t t tk
3.
continuous
discrete
k
kk k
x f x dx XE X
x p x X
![Page 70: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/70.jpg)
4. Let X be a random variable with moment generating function mX(t). Let Y = bX + a
Then mY(t) = mbX + a(t)
= E(e [bX + a]t) = eatE(e X[ bt ])
= eatmX (bt)
5. Let X and Y be two independent random variables with moment generating function mX(t) and mY(t) .
Then mX+Y(t) = E(e [X + Y]t) = E(e Xt e Yt)
= E(e Xt) E(e Yt)
= mX (t) mY (t)
![Page 71: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/71.jpg)
6. Let X and Y be two random variables with moment generating function mX(t) and mY(t) and two distribution functions FX(x) and FY(y) respectively.
Let mX (t) = mY (t) then FX(x) = FY(x).
This ensures that the distribution of a random variable can be identified by its moment generating function
![Page 72: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/72.jpg)
M. G. F.’s - Continuous distributions
Name
Moment generating function MX(t)
Continuous Uniform
ebt-eat
[b-a]t
Exponential t
for t <
Gamma t
for t <
2
d.f.
1
1-2t /2
for t < 1/2
Normal et+(1/2)t22
![Page 73: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/73.jpg)
M. G. F.’s - Discrete distributions
Name
Moment generating
function MX(t)
Discrete Uniform
et
N etN-1et-1
Bernoulli q + pet Binomial (q + pet)N
Geometric pet
1-qet
Negative Binomial
pet
1-qet k
Poisson e(et-1)
![Page 74: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/74.jpg)
Note:
The distribution of a random variable X can be described by:
probability function if is discrete1.
probability density function if is continuous
p x X
f x X
3. Moment generating function:
if is discrete
if is continuous
tx
xtX
Xtx
e p x X
m t E ee f x dx X
2. Distribution function:
if is discrete
if is continuous
u x
x
p u X
F xf u du X
![Page 75: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/75.jpg)
Summary of Discrete Distributions
Name
probability function p(x)
Mean
Variance
Moment generating
function MX(t)
Discrete Uniform p(x) =
1N x=1,2,...,N
N+12
N2-112
et
N etN-1et-1
Bernoulli p(x) =
p x=1q x=0
p pq q + pet
Binomial p(x) =
N
x pxqN-x Np Npq (q + pet)N
Geometric p(x) =pqx-1 x=1,2,... 1p
qp2
pet
1-qet
Negative Binomial p(x) =
x-1
k-1 pkqx-k
x=k,k+1,...
kp
kqp2
pet
1-qet k
Poisson p(x) =
x
x! e- x=1,2,... e(et-1)
Hypergeometric
p(x) =
A
x
N-A
n-x
N
n
n
A
N n
A
N
1-AN
N-n
N-1 not useful
![Page 76: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/76.jpg)
Summary of Continuous Distributions
Name
probability density function f(x)
Mean
Variance
Moment generating function MX(t)
Continuous Uniform
otherwise
bxaabxf
0
1)(
a+b2
(b-a)2
12 ebt-eat
[b-a]t
Exponential
00
0)(
x
xlexf
lx
1
12
t
for t <
Gamma
f(x) =
00
0)( f(x)
1
x
xexaG
l lxaa
2
t
for t <
2
d.f.
f(x) = (1/2)
(/2) x e-(1/2)x x ? 0
0 x < 0
1
1-2t /2
for t < 1/2
Normal f(x) =
1
2 e-(x-)2/22
2 et+(1/2)t22
Weibull
f(x) = x e-x x ? 0
0 x < 0
( )+1
( )+2 -[ ]( )+1
not avail.
![Page 77: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/77.jpg)
Jointly distributed Random variables
Multivariate distributions
![Page 78: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/78.jpg)
Discrete Random Variables
![Page 79: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/79.jpg)
The joint probability function;
p(x,y) = P[X = x, Y = y]
1. 0 , 1p x y
2. , 1x y
p x y
3. , ,P X Y A p x y ,x y A
![Page 80: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/80.jpg)
Continuous Random Variables
![Page 81: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/81.jpg)
Definition: Two random variable are said to have joint probability density function f(x,y) if
1. 0 ,f x y
2. , 1f x y dxdy
3. , ,P X Y A f x y dxdy A
![Page 82: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/82.jpg)
Marginal and conditional distributions
![Page 83: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/83.jpg)
Marginal Distributions (Discrete case):
Let X and Y denote two random variables with joint probability function p(x,y) then
the marginal density of X is
,Xy
p x p x y
the marginal density of Y is
,Yx
p y p x y
![Page 84: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/84.jpg)
Marginal Distributions (Continuous case):
Let X and Y denote two random variables with joint probability density function f(x,y) then
the marginal density of X is
,Xf x f x y dy
the marginal density of Y is
,Yf y f x y dx
![Page 85: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/85.jpg)
Conditional Distributions (Discrete Case):
Let X and Y denote two random variables with joint probability function p(x,y) and marginal probability functions pX(x), pY(y) then
the conditional density of Y given X = x
,
Y XX
p x yp y x
p x
conditional density of X given Y = y
,
X YY
p x yp x y
p y
![Page 86: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/86.jpg)
Conditional Distributions (Continuous Case):
Let X and Y denote two random variables with joint probability density function f(x,y) and marginal densities fX(x), fY(y) then
the conditional density of Y given X = x
,
Y XX
f x yf y x
f x
conditional density of X given Y = y
,
X YY
f x yf x y
f y
![Page 87: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/87.jpg)
The bivariate Normal distribution
![Page 88: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/88.jpg)
Let
2 2
1 1 1 1 2 2 2 2
1 1 2 2
1 2 2
2
,1
x x x x
Q x x
1 21
,2
1 2 21 2
1, e
2 1
Q x xf x x
where
This distribution is called the bivariate Normal distribution.
The parameters are 1, 2 , 1, 2 and
![Page 89: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/89.jpg)
Surface Plots of the bivariate Normal distribution
![Page 90: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/90.jpg)
2. The marginal distribution of x2 is Normal with mean 2 and standard deviation 2.
1. The marginal distribution of x1 is Normal with mean 1 and standard deviation 1.
Marginal distributions
![Page 91: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/91.jpg)
Conditional distributions
1. The conditional distribution of x1 given x2 is Normal with:
andmean
standard deviation
11 2 212
2
x
2112 1
2. The conditional distribution of x2 given x1 is Normal with:
andmean
standard deviation
22 1 121
1
x
2221 1
![Page 92: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/92.jpg)
Independence
![Page 93: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/93.jpg)
Two random variables X and Y are defined to be independent if
Definition:
, X Yp x y p x p y if X and Y are discrete
, X Yf x y f x f y if X and Y are continuous
![Page 94: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/94.jpg)
multivariate distributions
k ≥ 2
![Page 95: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/95.jpg)
Definition
Let X1, X2, …, Xn denote n discrete random variables, then
p(x1, x2, …, xn )
is joint probability function of X1, X2, …, Xn if
1
12. , , 1n
nx x
p x x
11. 0 , , 1np x x
1 13. , , , ,n nP X X A p x x 1, , nx x A
![Page 96: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/96.jpg)
Definition
Let X1, X2, …, Xk denote k continuous random variables, then
f(x1, x2, …, xk )
is joint density function of X1, X2, …, Xk if
1 12. , , , , 1n nf x x dx dx
11. , , 0nf x x
1 1 13. , , , , , ,n n nP X X A f x x dx dx A
![Page 97: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/97.jpg)
The Multinomial distribution
Suppose that we observe an experiment that has k possible outcomes {O1, O2, …, Ok } independently n times.
Let p1, p2, …, pk denote probabilities of O1, O2, …, Ok respectively.
Let Xi denote the number of times that outcome Oi occurs in the n repetitions of the experiment.
![Page 98: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/98.jpg)
is called the Multinomial distribution
1 21 1 2
1 2
! , ,
! ! !kxx x
n kk
np x x p p p
x x x
1 21 2
1 2
kxx xk
k
np p p
x x x
The joint probability function of:
![Page 99: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/99.jpg)
The Multivariate Normal distributionRecall the univariate normal distribution
2121
2
x
f x e
the bivariate normal distribution
221
22 12
2
1 ,
2 1
x xx x y yx xx x y y
x y
f x y e
![Page 100: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/100.jpg)
The k-variate Normal distribution
112
1 / 2 1/ 2
1 , ,
2k kf x x f e
x μ x μx
where
1
2
k
x
x
x
x
1
2
k
μ
11 12 1
12 22 2
1 2
k
k
k k kk
![Page 101: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/101.jpg)
Marginal distributions
![Page 102: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/102.jpg)
Definition
Let X1, X2, …, Xq, Xq+1 …, Xk denote k discrete random variables with joint probability function
p(x1, x2, …, xq, xq+1 …, xk )
1
12 1 1 , , , ,q n
q q nx x
p x x p x x
then the marginal joint probability function
of X1, X2, …, Xq is
![Page 103: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/103.jpg)
Definition
Let X1, X2, …, Xq, Xq+1 …, Xk denote k continuous random variables with joint probability density function
f(x1, x2, …, xq, xq+1 …, xk )
12 1 1 1 , , , ,q q n q nf x x f x x dx dx
then the marginal joint probability function
of X1, X2, …, Xq is
![Page 104: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/104.jpg)
Conditional distributions
![Page 105: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/105.jpg)
Definition
Let X1, X2, …, Xq, Xq+1 …, Xk denote k discrete random variables with joint probability function
p(x1, x2, …, xq, xq+1 …, xk )
1
1 11 11 1
, , , , , ,
, ,k
q q kq q kq k q k
p x xp x x x x
p x x
then the conditional joint probability function
of X1, X2, …, Xq given Xq+1 = xq+1 , …, Xk = xk is
![Page 106: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/106.jpg)
Definition
Let X1, X2, …, Xq, Xq+1 …, Xk denote k continuous random variables with joint probability density function
f(x1, x2, …, xq, xq+1 …, xk )
then the conditional joint probability function
of X1, X2, …, Xq given Xq+1 = xq+1 , …, Xk = xk is
Definition
11 11 1
1 1
, , , , , ,
, ,k
q q kq q kq k q k
f x xf x x x x
f x x
![Page 107: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/107.jpg)
Definition
Let X1, X2, …, Xq, Xq+1 …, Xk denote k continuous random variables with joint probability density function
f(x1, x2, …, xq, xq+1 …, xk )
then the variables X1, X2, …, Xq are independent of Xq+1, …, Xk if
Definition – Independence of sets of vectors
1 1 1 1 1 , , , , , ,k q q q k q kf x x f x x f x x
A similar definition for discrete random variables.
![Page 108: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/108.jpg)
Definition
Let X1, X2, …, Xk denote k continuous random variables with joint probability density function
f(x1, x2, …, xk )
then the variables X1, X2, …, Xk are called mutually independent if
Definition – Mutual Independence
1 1 1 2 2 , , k k kf x x f x f x f x
A similar definition for discrete random variables.
![Page 109: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/109.jpg)
Expectation
for multivariate distributions
![Page 110: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/110.jpg)
Definition
Let X1, X2, …, Xn denote n jointly distributed random variable with joint density function
f(x1, x2, …, xn )
then
1, , nE g X X
1 1 1, , , , , ,n n ng x x f x x dx dx
![Page 111: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/111.jpg)
Some Rules for Expectation
![Page 112: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/112.jpg)
1 11. , ,i i n nE X x f x x dx dx
i i i ix f x dx
Thus you can calculate E[Xi] either from the joint distribution of X1, … , Xn
or the marginal distribution of Xi.
1 1 1 12. n n n nE a X a X b a E X a E X b
The Linearity property
![Page 113: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/113.jpg)
1 1, , , ,q q kE g X X h X X
In the simple case when k = 2
3. (The Multiplicative property) Suppose X1, … , Xq
are independent of Xq+1, … , Xk then
1 1, , , ,q q kE g X X E h X X
E XY E X E Y
if X and Y are independent
![Page 114: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/114.jpg)
Some Rules for Variance
2 2 2Var X XX E X E X
![Page 115: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/115.jpg)
Ex:
2
11P X k
k
32
4P X
Tchebychev’s inequality
83
9P X
154
16P X
![Page 116: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/116.jpg)
1. Var Var Var 2Cov ,X Y X Y X Y
where Cov , = X YX Y E X Y
Cov , 0X Y
and Var Var VarX Y X Y
Note: If X and Y are independent, then
![Page 117: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/117.jpg)
The correlation coefficient XY
Cov , Cov ,=
Var Varxy
X Y
X Y X Y
X Y
:
1. If and are independent than 0.XYX Y Properties
2. 1 1XY
if there exists a and b such thatand 1XY
1P Y bX a
whereXY = +1 if b > 0 and XY = -1 if b< 0
![Page 118: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/118.jpg)
2 22. Var Var Var 2 Cov ,aX bY a X b Y ab X Y
Some other properties of variance
1 13. Var n na X a X
2 21 1Var Varn na X a X
1 2 1 2 1 12 Cov , 2 Cov ,n na a X X a a X X
2 3 2 3 2 22 Cov , 2 Cov ,n na a X X a a X X
1 12 Cov ,n n n na a X X
2
1
Var 2 Cov ,n
i i i j i ji
a X a a X X
![Page 119: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/119.jpg)
4. Variance: Multiplicative Rule for independent random variables
Suppose that X and Y are independent random variables, then:
2 2X YVar XY Var X Var Y Var Y Var X
![Page 120: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/120.jpg)
Mean and Variance of averages
Let1
1 n
ii
X Xn
Let X1, … , Xn be n mutually independent random variables each having mean and standard deviation (variance 2).
Then X E X
and2
2X Var X
n
![Page 121: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/121.jpg)
The Law of Large Numbers
Let1
1 n
ii
X Xn
Let X1, … , Xn be n mutually independent random variables each having mean
Then for any > 0 (no matter how small)
1 as P X P X n
![Page 122: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/122.jpg)
Conditional Expectation:
![Page 123: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/123.jpg)
Let X1, X2, …, Xq, Xq+1 …, Xk denote k continuous random variables with joint probability density function
f(x1, x2, …, xq, xq+1 …, xk )
then the conditional joint probability function
of X1, X2, …, Xq given Xq+1 = xq+1 , …, Xk = xk is
Definition
11 11 1
1 1
, , , , , ,
, ,k
q q kq q kq k q k
f x xf x x x x
f x x
![Page 124: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/124.jpg)
Let U = h( X1, X2, …, Xq, Xq+1 …, Xk )
then the Conditional Expectation of U
given Xq+1 = xq+1 , …, Xk = xk is
Definition
1 1 1 11 1 , , , , , ,k q q k qq q kh x x f x x x x dx dx
1 , , q kE U x x
Note this will be a function of xq+1 , …, xk.
![Page 125: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/125.jpg)
A very useful rule
E U E E U y y
Var U E Var U Var E U y yy y
Then
1 1Let , , , , , ,q mU g x x y y g x y
Let (x1, x2, … , xq, y1, y2, … , ym) = (x, y) denote q + m random variables.
![Page 126: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/126.jpg)
Functions of Random Variables
![Page 127: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/127.jpg)
Methods for determining the distribution of functions of Random Variables
1. Distribution function method
2. Moment generating function method
3. Transformation method
![Page 128: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/128.jpg)
Distribution function method
Let X, Y, Z …. have joint density f(x,y,z, …)Let W = h( X, Y, Z, …)First step
Find the distribution function of WG(w) = P[W ≤ w] = P[h( X, Y, Z, …) ≤ w]
Second stepFind the density function of Wg(w) = G'(w).
![Page 129: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/129.jpg)
Use of moment generating functions
1. Using the moment generating functions of X, Y, Z, …determine the moment generating function of W = h(X, Y, Z, …).
2. Identify the distribution of W from its moment generating function
This procedure works well for sums, linear combinations, averages etc.
![Page 130: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/130.jpg)
Let x1, x2, … denote a sequence of independent random variables
1 2 1 2
=n nS x x x x x xm t m t m t m t m t
SumsLet S = x1 + x2 + … + xn then
1 1 2 2 1 21 2=
n n nL a x a x a x x x x nm t m t m a t m a t m a t
Linear CombinationsLet L = a1x1 + a2x2 + … + anxn then
![Page 131: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/131.jpg)
Arithmetic MeansLet x1, x2, … denote a sequence of independent random variables coming from a distribution with moment generating function m(t)
1 2
1 1 1
1 1 1
nx
x x xn n n
m t m t m t m t m tn n n
1 2Let , thennx x xx
n
nt
mn
![Page 132: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/132.jpg)
The Transformation Method
Theorem
Let X denote a random variable with probability density function f(x) and U = h(X).
Assume that h(x) is either strictly increasing (or decreasing) then the probability density of U is:
1
1 ( )( )
dh u dxg u f h u f x
du du
![Page 133: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/133.jpg)
The Transfomation Method(many variables)
Theorem
Let x1, x2,…, xn denote random variables with joint probability density function
f(x1, x2,…, xn )
Let u1 = h1(x1, x2,…, xn).u2 = h2(x1, x2,…, xn).
un = hn(x1, x2,…, xn).
define an invertible transformation from the x’s to the u’s
![Page 134: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/134.jpg)
Then the joint probability density function of u1, u2,…, un is given by:
11 1
1
, ,, , , ,
, ,n
n nn
d x xg u u f x x
d u u
1, , nf x x J
where
1
1
, ,
, ,n
n
d x xJ
d u u
Jacobian of the transformation
1 1
1
1
detn
n n
n
dx dx
du du
dx dx
du du
![Page 135: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/135.jpg)
Some important results
Distribution of functions of random variables
![Page 136: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/136.jpg)
The method used to derive these results will be indicated by:
1. DF - Distribution Function Method.
2. MGF - Moment generating function method
3. TF - Transformation method
![Page 137: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/137.jpg)
Student’s t distribution
Let Z and U be two independent random variables with:
1. Z having a Standard Normal distribution
and
2. U having a 2 distribution with degrees of freedom
then the distribution of:Z
tU
12 2
( ) 1t
g t K
12
where
2
K
is:
DF
![Page 138: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/138.jpg)
The Chi-square distribution
Let Z1, Z2, … , Zv be v independent random variables having a Standard Normal distribution, then
has a 2 distribution with degrees of freedom.
2
1i
i
U Z
MGF
DF for = 1
for > 1
![Page 139: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/139.jpg)
Distribution of the sample mean
Let x1, x2, …, xn denote a sample from the normal distribution with mean and variance 2.
and standard deviation x xn
then
has a Normal distribution with:
1
n
ii
xx
n
MGF
![Page 140: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/140.jpg)
If x1, x2, …, xn is a sample from a distribution with mean , and standard deviations then if n is large the sample meanx
The Central Limit theorem
22x n
and variance
x has a normal distribution with mean
standard deviation xn
MGF
![Page 141: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/141.jpg)
Distribution of the sample variance
Let x1, x2, …, xn denote a sample from the normal distribution with mean and variance 2.
2
21
2 2
1
n
ii
x xn s
U
then
has a 2 distribution with = n - 1 degrees of freedom.
2
21 1 and 1
n n
i ii i
x x xx s
n n
Let
MGF
![Page 142: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/142.jpg)
Distribution of sums of Gamma R. V.’s
Let X1, X2, … , Xn denote n independent random variables each having a gamma distribution with parameters
(,i), i = 1, 2, …, n.
Then W = X1 + X2 + … + Xn has a gamma distribution with
parameters (, 1 + 2 +… + n).
Distribution of a multiple of a Gamma R. V.
Suppose that X is a random variable having a gamma distribution with parameters (,).
Then W = aX has a gamma distribution with parameters (/a, ).
MGF
MGF
![Page 143: Stats 241.3 Probability Theory Summary. The sample Space, S The sample space, S, for a random phenomena is the set of all possible outcomes](https://reader038.vdocument.in/reader038/viewer/2022110321/56649f4d5503460f94c6de79/html5/thumbnails/143.jpg)
Distribution of sums of Binomial R. V.’s
Let X1, X2, … , Xk denote k independent random variables each having a binomial distribution with parameters
(p,ni), i = 1, 2, …, k.
Then W = X1 + X2 + … + Xk has a binomial distribution with
parameters (p, n1 + n2 +… + nk).
Distribution of sums of Negative Binomial R. V.’s
Let X1, X2, … , Xn denote n independent random variables each having a negative binomial distribution with parameters
(p,ki), i = 1, 2, …, n.
Then W = X1 + X2 + … + Xn has a negative binomial distribution with
parameters (p, k1 + k2 +… + kn). MGF
MGF