niprl chapter 2. random variables 2.1 discrete random variables 2.2 continuous random variables 2.3...
TRANSCRIPT
NIPRL
Chapter 2. Random Variables
2.1 Discrete Random Variables2.2 Continuous Random Variables2.3 The Expectation of a Random Variable2.4 The Variance of a Random Variable2.5 Jointly Distributed Random Variables2.6 Combinations and Functions of Random Variables
NIPRL
2.1 Discrete Random Variable2.1.1 Definition of a Random Variable (1/2)
• Random variable – A numerical value to each outcome of a particular
experiment
S
0 21 3-1-2-3
R
NIPRL
2.1.1 Definition of a Random Variable (2/2)
• Example 1 : Machine Breakdowns– Sample space : – Each of these failures may be associated with a repair
cost– State space : – Cost is a random variable : 50, 200, and 350
{ , , }S electrical mechanical misuse
{50,200,350}
NIPRL
2.1.2 Probability Mass Function (1/2)
• Probability Mass Function (p.m.f.)– A set of probability value assigned to each of the
values taken by the discrete random variable– and – Probability :
ip
ix
0 1ip 1iip
( )i iP X x p
NIPRL
2.1.2 Probability Mass Function (1/2)
• Example 1 : Machine Breakdowns– P (cost=50)=0.3, P (cost=200)=0.2,
P (cost=350)=0.5– 0.3 + 0.2 + 0.5 =1 50 200 350
0.3 0.2 0.5
ix
ip( )f x
0.5
0.3
50 200 350 Cost($)
0.2
NIPRL
2.1.3 Cumulative Distribution Function (1/2)
• Cumulative Distribution Function– Function : – Abbreviation : c.d.f
( ) ( )F x P X x :
( ) ( )y y x
F x P X y
( )F x1.0
0.5
0.3
50 200 3500 ($cost)x
NIPRL
2.1.3 Cumulative Distribution Function (2/2)
• Example 1 : Machine Breakdowns
50 ( ) (cost ) 0
50 200 ( ) (cost ) 0.3
200 350 ( ) (cost ) 0.3 0.2 0.5
350 ( ) (cost ) 0.3 0.2 0.5 1.0
x F x P x
x F x P x
x F x P x
x F x P x
NIPRL
2.2 Continuous Random Variables 2.2.1 Example of Continuous Random Variables (1/1)
• Example 14 : Metal Cylinder Production– Suppose that the random variable is the diameter
of a randomly chosen cylinder manufactured by the company. Since this random variable can take any value between 49.5 and 50.5, it is a continuous random variable.
X
NIPRL
2.2.2 Probability Density Function (1/4)
• Probability Density Function (p.d.f.)– Probabilistic properties of a continuous random variable
( ) 0f x
( ) 1statespace
f x dx
NIPRL
2.2.2 Probability Density Function (2/4)
• Example 14– Suppose that the diameter of a metal cylinder has a p.d.f
2( ) 1.5 6( 50.2) for 49.5 50.5
( ) 0, elsewhere
f x x x
f x
( )f x
x49.5 50.5
NIPRL
2.2.2 Probability Density Function (3/4)
• This is a valid p.d.f.
50.5 2 3 50.549.549.5
3
3
(1.5 6( 50.0) ) [1.5 2( 50.0) ]
[1.5 50.5 2(50.5 50.0) ]
[1.5 49.5 2(49.5 50.0) ]
75.5 74.5 1.0
x dx x x
NIPRL
2.2.2 Probability Density Function (4/4)
• The probability that a metal cylinder has a diameter between 49.8 and 50.1 mm can be calculated to be
50.1 2 3 50.149.849.8
3
3
(1.5 6( 50.0) ) [1.5 2( 50.0) ]
[1.5 50.1 2(50.1 50.0) ]
[1.5 49.8 2(49.8 50.0) ]
75.148 74.716 0.432
x dx x x
( )f x
x49.5 50.549.8 50.1
NIPRL
2.2.3 Cumulative Distribution Function (1/3)
• Cumulative Distribution Function
( ) ( ) ( )
( )( )
xF x P X x f y dy
dF xf x
dx
( ) ( ) ( )
( ) ( )
( ) ( )
P a X b P X b P X a
F b F a
P a X b P a X b
NIPRL
2.2.2 Probability Density Function (2/3)
• Example 14
2
49.5
349.5
3 3
3
( ) ( ) (1.5 6( 50.0) )
[1.5 2( 50.0) ]
[1.5 2( 50.0) ] [1.5 49.5 2(49.5 50.0) ]
1.5 2( 50.0) 74.5
x
x
F x P X x y dy
y y
x x
x x
3
3
(49.7 50.0) (50.0) (49.7)
(1.5 50.0 2(50.0 50.0) 74.5)
(1.5 49.7 2(49.7 50.0) 74.5)
0.5 0.104 0.396
P X F F
NIPRL
2.2.2 Probability Density Function (3/3)
( )F x
( 49.7) 0.104P X
x49.5 50.549.7 50.0
1
( 50.0) 0.5P X
(49.7 50.0) 0.396P X
NIPRL
2.3 The Expectation of a Random Variable 2.3.1 Expectations of Discrete Random Variables (1/2)
• Expectation of a discrete random variable with p.m.f
• Expectation of a continuous random variable with p.d.f f(x)
• The expected value of a random variable is also called the mean of the random variable
( )i iP X x p
( ) i ii
E X p x
state space( ) ( )E X xf x dx
NIPRL
2.3.1 Expectations of Discrete Random Variables (2/2)
• Example 1 (discrete random variable)– The expected repair cost is
(cost) ($50 0.3) ($200 0.2) ($350 0.5) $230E
NIPRL
2.3.2 Expectations of Continuous Random Variables (1/2)
• Example 14 (continuous random variable)– The expected diameter of a metal cylinder is
– Change of variable: y=x-50
50.5 2
49.5( ) (1.5 6( 50.0) )E X x x dx
0.5 2
0.5
0.5 3 2
0.5
4 3 2 0.50.5
( ) ( 50)(1.5 6 )
( 6 300 1.5 75)
[ 3 / 2 100 0.75 75 ]
[25.09375] [ 24.90625] 50.0
E x y y dy
y y y dy
y y y y
NIPRL
2.3.2 Expectations of Continuous Random Variables (2/2)
• Symmetric Random Variables– If has a p.d.f that is
symmetric about a point so that
– Then, (why?)
– So that the expectation of the random variable is equal to the point of symmetry
( )f x
( ) ( )f x f x
x
( )E X
( )E X ( )f x
x
NIPRL
( ) ( )
( ) ( )
2
( ) ( )
E X xf x dx
xf x dx xf x dx
y x
xf x dx yf y dy
-
- -
+
+
NIPRL
2.3.3 Medians of Random Variables (1/2)
• Median– Information about the “middle” value of the random
variable
• Symmetric Random Variable– If a continuous random variable is symmetric about a
point , then both the median and the expectation of the random variable are equal to
( ) 0.5F x
NIPRL
2.3.3 Medians of Random Variables (2/2)
• Example 14
3( ) 1.5 2( 50.0) 74.5 0.5F x x x
50.0x
NIPRL
2.4 The variance of a Random Variable 2.4.1 Definition and Interpretation of Variance (1/2)
• Variance( )– A positive quantity that measures the spread of the
distribution of the random variable about its mean value– Larger values of the variance indicate that the
distribution is more spread out
– Definition:
• Standard Deviation– The positive square root of the variance– Denoted by
2
2 2
Var( ) (( ( )) )
( ) ( ( ))
X E X E X
E X E X
2
NIPRL
2.4.1 Definition and Interpretation of Variance (2/2)
2
2 2
2 2
2 2
Var( ) (( ( )) )
( 2 ( ) ( ( )) )
( ) 2 ( ) ( ) ( ( ))
( ) ( ( ))
X E X E X
E X XE X E X
E X E X E X E X
E X E X
Two distribution with identical mean values but different variances
( )f x
x
NIPRL
2.4.2 Examples of Variance Calculations (1/1)
• Example 1
2 2
2 2 2
2
Var( ) (( ( )) ) ( ( ))
0.3(50 230) 0.2(200 230) 0.5(350 230)
17,100
i ii
X E X E X p x E X
17,100 130.77
NIPRL
• Chebyshev’s Inequality– If a random variable has a mean and a variance ,
then
– For example, taking gives
2
2
1( ) 1
for 1
P c X cc
c
2c
2
1( 2 2 ) 1 0.75
2P X
2.4.3 Chebyshev’s Inequality (1/1)
NIPRL
• Proof
2 2 2 2 2
| | | |
( ) ( ) ( ) ( ) ( ) .x c x c
x f x dx x f x dx c f x dx
2(| | ) 1/P x c c
2(| | ) 1 (| | ) 1 1/P x c P x c c
NIPRL
2.4.4 Quantiles of Random Variables (1/2)
• Quantiles of Random variables– The th quantile of a random variable X
– A probability of that the random variable takes a value less than the th quantile
• Upper quartile– The 75th percentile of the distribution
• Lower quartile– The 25th percentile of the distribution
• Interquartile range– The distance between the two quartiles
p
( )F x pp
p
NIPRL
2.4.4 Quantiles of Random Variables (2/2)
• Example 14
– Upper quartile :
– Lower quartile :
– Interquartile range :
3( ) 1.5 2( 50.0) 74.5 for 49.5 50.5F x x x x ( ) 0.75F x 50.17x
( ) 0.25F x 49.83x
50.17 49.83 0.34
NIPRL
2.5 Jointly Distributed Random Variables 2.5.1 Jointly Distributed Random Variables (1/4)
• Joint Probability Distributions– Discrete
– Continuous
( , ) 0
satisfying 1
i j ij
iji j
P X x Y y p
p
state space( , ) 0 satisfying ( , ) 1f x y f x y dxdx
NIPRL
2.5.1 Jointly Distributed Random Variables (2/4)
• Joint Cumulative Distribution Function– Discrete
– Continuous
( , ) ( , )i jF x y P X x Y y
: :
( , )i j
iji x x j y y
F x y p
( , ) ( , )x y
w zF x y f w z dzdw
NIPRL
2.5.1 Jointly Distributed Random Variables (3/4)
• Example 19 : Air Conditioner Maintenance– A company that services air conditioner units in
residences and office blocks is interested in how to schedule its technicians in the most efficient manner
– The random variable X, taking the values 1,2,3 and 4, is the service time in hours
– The random variable Y, taking the values 1,2 and 3, is the number of air conditioner units
NIPRL
2.5.1 Jointly Distributed Random Variables (4/4)
• Joint p.m.f
• Joint cumulative distribution function
Y=number of units
X=service time
1 2 3 4
1 0.12 0.08 0.07 0.05
2 0.08 0.15 0.21 0.13
3 0.01 0.01 0.02 0.07
0.12 0.18
0.07 1.00
iji j
p
11 12 21 22(2,2)
0.12 0.18 0.08 0.15
0.43
F p p p p
NIPRL
2.5.2 Marginal Probability Distributions (1/2)
• Marginal probability distribution– Obtained by summing or integrating the joint probability
distribution over the values of the other random variable– Discrete
– Continuous
( ) i ijj
P X i p p
( ) ( , )Xf x f x y dy
NIPRL
2.5.2 Marginal Probability Distributions (2/2)
• Example 19– Marginal p.m.f of X
– Marginal p.m.f of Y
3
11
( 1) 0.12 0.08 0.01 0.21jj
P X p
4
11
( 1) 0.12 0.08 0.07 0.05 0.32ii
P Y p
NIPRL
• Example 20: (a jointly continuous case)
• Joint pdf:
• Marginal pdf’s of X and Y:
( , )f x y
( ) ( , )
( ) ( , )
X
Y
f x f x y dy
f y f x y dx
NIPRL
2.5.3 Conditional Probability Distributions (1/2)
• Conditional probability distributions– The probabilistic properties of the random variable X
under the knowledge provided by the value of Y– Discrete
– Continuous
– The conditional probability distribution is a probability distribution.
|
( , )( | )
( )ij
i jj
pP X i Y jp P X i Y j
P Y j p
|
( , )( )
( )X Y yY
f x yf x
f y
NIPRL
2.5.3 Conditional Probability Distributions (2/2)
• Example 19– Marginal probability distribution of Y
– Conditional distribution of X
3( 3) 0.01 0.01 0.02 0.07 0.11P Y p
131| 3
3
0.01( 1| 3) 0.091
0.11Y
pp P X Y
p
NIPRL
2.5.4 Independence and Covariance (1/5)
• Two random variables X and Y are said to be independent if– Discrete
– Continuous
– How is this independency different from the independence among events?
for all values of and ofij i jp p p i X j Y
( , ) ( ) ( )X Yf x y f x f y for all x and y
NIPRL
2.5.4 Independence and Covariance (2/5)
• Covariance
– May take any positive or negative numbers.– Independent random variables have a covariance of zero– What if the covariance is zero?
Cov( , ) (( ( ))( ( )))
( ) ( ) ( )
X Y E X E X Y E Y
E XY E X E Y
Cov( , ) (( ( ))( ( )))
( ( ) ( ) ( ) ( ))
( ) ( ) ( ) ( ) ( ) ( ) ( )
( ) ( ) ( )
X Y E X E X Y E Y
E XY XE Y E X Y E X E Y
E XY E X E Y E X E Y E X E Y
E XY E X E Y
NIPRL
2.5.4 Independence and Covariance (3/5)
• Example 19 (Air conditioner maintenance)
( ) 2.59, ( ) 1.79E X E Y 4 3
1 1
( )
(1 1 0.12) (1 2 0.08)
(4 3 0.07) 4.86
iji j
E XY ijp
Cov( , ) ( ) ( ) ( )
4.86 (2.59 1.79) 0.224
X Y E XY E X E Y
NIPRL
2.5.4 Independence and Covariance (4/5)
• Correlation:
– Values between -1 and 1, and independent random variables have a correlation of zero
Cov( , )Corr( , )
Var( )Var( )
X YX Y
X Y
NIPRL
2.5.4 Independence and Covariance (5/5)
• Example 19: (Air conditioner maintenance)
Var( ) 1.162, Var( ) 0.384X Y
Cov( , )Corr( , )
Var( )Var( )
0.2240.34
1.162 0.384
X YX Y
X Y
NIPRL
• What if random variable X and Y have linear relationship, that is,
where
That is, Corr(X,Y)=1 if a>0; -1 if a<0.
Y aX b 0a
2 2
2 2
( , ) [ ] [ ] [ ]
[ ( )] [ ] [ ]
[ ] [ ] [ ] [ ]
( [ ] [ ]) ( )
Cov X Y E XY E X E Y
E X aX b E X E aX b
aE X bE X aE X bE X
a E X E X aVar X
2
( , ) ( )( , )
( ) ( ) ( ) ( )
Cov X Y aVar XCorr X Y
Var X Var Y Var X a Var X
NIPRL
2.6 Combinations and Functions of Random Variables
2.6.1 Linear Functions of Random Variables (1/4)• Linear Functions of a Random Variable
– If X is a random variable and for some numbers thenand
• Standardization-If a random variable has an expectation of and a variance of ,
has an expectation of zero and a variance of one.
Y aX b ,a bR ( ) ( )E Y aE X b
2Var( ) Var( )Y a X
X 2 1X
Y X
NIPRL
2.6.1 Linear Functions of Random Variables (2/4)
• Example 21:Test Score Standardization– Suppose that the raw score X from a particular testing
procedure are distributed between -5 and 20 with an expected value of 10 and a variance 7. In order to standardize the scores so that they lie between 0 and 100, the linear transformation is applied to the scores.
4 20Y X
NIPRL
2.6.1 Linear Functions of Random Variables (3/4)
– For example, x=12 corresponds to a standardized score of y=(4ⅹ12)+20=68
( ) 4 ( ) 20 (4 10) 20 60E Y E X
2 2Var( ) 4 Var( ) 4 7 112Y X
112 10.58Y
NIPRL
2.6.1 Linear Functions of Random Variables (4/4)
• Sums of Random Variables– If and are two random variables, then
and
– If and are independent, so that then
1X 2X
1 2 1 2( ) ( ) ( ) ( ?)E X X E X E X why
1 2 1 2 1 2Var( ) Var( ) Var( ) 2Cov( , )X X X X X X
1X 2X 1 2Cov( , ) 0X X
1 2 1 2Var( ) Var( ) Var( )X X X X
NIPRL
• Properties of 1 2( , )Cov X X
1 2 1 2 1 2( , ) [ ] [ ] [ ]Cov X X E X X E X E X
1 2 2 1( , ) ( , )Cov X X Cov X X
1 1 1( , ) ( )Cov X X Var X 2 2 2( , ) ( )Cov X X Var X
1 2 1 1 1 2 1( , ) ( , ) ( , )Cov X X X Cov X X Cov X X
1 2 1 2 1 1 1 2
2 1 2 2
( , ) ( , ) ( , )
( , ) ( , )
Cov X X X X Cov X X Cov X X
Cov X X Cov X X
1 2 1 2 1 2( ) ( ) ( ) 2 ( , )Var X X Var X Var X Cov X X
NIPRL
2.6.2 Linear Combinations of Random Variables (1/5)
• Linear Combinations of Random Variables– If is a sequence of random variables and
and are constants, then
– If, in addition, the random variables are independent, then
1, , nX X1, , na a
b
1 1 1 1( ) ( ) ( )n n n nE a X a X b a E X a E X b
2 21 1 1 1Var( ) Var( ) Var( )n n n na X a X b a X a X
NIPRL
2.6.2 Linear Combinations of Random Variables (2/5)
• Averaging Independent Random Variables– Suppose that is a sequence of independent
random variables with an expectation and a variance .
– Let
– Then
and
– What happened to the variance?
1, , nX X 2
1 nX XX
n
( )E X
2
Var( )Xn
NIPRL
2.6.2 Linear Combinations of Random Variables (3/5)
1 1
1 1 1 1( ) ( ) ( )
1 1
n nE X E X X E X E Xn n n n
n n
2 2
1 1
2 2 22 2
1 1 1 1Var( ) Var Var( ) Var( )
1 1
n nX X X X Xn n n n
n n n
NIPRL
2.6.2 Linear Combinations of Random Variables (4/5)
• Example 21– The standardized scores of the two tests are
– The final score is
1 1 2 2
10 5 50and
3 3 3Y X Y X
1 2 1 2
2 1 20 5 50
3 3 9 9 9Z Y Y X X
NIPRL
2.6.2 Linear Combinations of Random Variables (5/5)
– The expected value of the final score is
– The variance of the final score is
1 2
20 5 50( ) ( ) ( )
9 9 920 5 50
18 309 9 9
62.22
E Z E X E X
1 2
2 2
1 2
2 2
20 5 50Var( ) Var
9 9 9
20 5Var( ) Var( )
9 9
20 524 60 137.04
9 9
Z X X
X X
NIPRL
2.6.3 Nonlinear Functions of a Random Variable (1/3)
• Nonlinear function of a Random Variable– A nonlinear function of a random variable X is another
random variable Y=g(X) for some nonlinear function g.
– There are no general results that relate the expectation and variance of the random variable Y to the expectation and variance of the random variable X
2 , , XY X Y X Y e
NIPRL
2.6.3 Nonlinear Functions of a Random Variable (2/3)
– For example,
– Consider
– What is the pdf of Y?
( ) 1 for 0 1
( ) 0 elsewhereXf x x
f x
0 1 x
f(x)=1 E(x)=0.
5
1.0 2.718 y
f(y)=1/y E(y)=1.71
8
( ) for 0 1XF x x x
where 1 2.718
X
Y
Y e
NIPRL
2.6.3 Nonlinear Functions of a Random Variable (3/3)• CDF methd
– The p.d.f of Y is obtained by differentiation to be
• Notice that
( ) ( ) ( ) ( ln( )) (ln( )) ln( )XY xF y P Y y P e y P X y F y y
for 1 2.718( ) 1
( ) YY y
dF yf y
dy y
2.718 2.718
1 1
( ) 0.5
( ) ( ) 1 2.718 1 1.718
( ) 1.649
Yz z
E X
E Y zf z dz dz
E Y e e
NIPRL
• Determining the p.d.f. of nonlinear relationship between r.v.s:
Given and ,what is ?
If are all its real roots, that is,
then where
( )Xf x ( )Y g X ( )Yf y
1 2, , , nx x x
1 2( ) ( ) ( )ny g x g x g x
1
( )( )
| '( ) |
nX i
Yi i
f xf y
g x
( )'( )
dg xg x
dx
NIPRL
• Example: determine
-> one root is possible in
( ) 1 for 0 1
( ) 0 elsewhereXf x x
f x
where 1 2.718
X
Y
Y e
( )Yf y
( )
ln
xy g x e
y x
xdge y
dx
1 1( )
| |Yf yy y
0 1x