Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 1 of 66 ECE 3800
Henry Stark and John W. Woods, Probability, Statistics, and Random Variables for Engineers, 4th ed.,
Pearson Education Inc., 2012. ISBN: 978-0-13-231123-6
Chapter 2: Introduction to Probability
Sections 2.1 Introduction 79 2.2 Definition of a Random Variable 80 2.3 Cumulative Distribution Function 83 Properties of FX(x) 84 Computation of FX(x) 85 2.4 Probability Density Function (pdf) 88 Four Other Common Density Functions 95 More Advanced Density Functions 97 2.5 Continuous, Discrete, and Mixed Random Variables 100 Some Common Discrete Random Variables 102 2.6 Conditional and Joint Distributions and Densities 107 Properties of Joint CDF FXY (x, y) 118 2.7 Failure Rates 137 Summary 141 Problems 141 References 149 Additional Reading 149
Random variable: A real function whose domain is that of the outcomes of an experiment (sample space, S) and whose actual value is unknown in advance of the experiment.
From: http://en.wikipedia.org/wiki/Random_variable
A random variable can be thought of as the numeric result of operating a non-deterministic mechanism or performing a non-deterministic experiment to generate a random result.
Unlike the common practice with other mathematical variables, a random variable cannot be assigned a value; a random variable does not describe the actual outcome of a particular experiment, but rather describes the possible, as-yet-undetermined outcomes in terms of real numbers.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 2 of 66 ECE 3800
2.2 Definition of a Random Variable
This section is strongly driven by theory and formality.
It assumes that you have heard about a “Cumulative Distribution Function” to define probability … so is a bit backwards (Sec 2.3 defines it).
So …
Cumulative Distribution Function (CDF):The probability of the event that the observed random variable X is less than or equal to the allowed value x.
xXxFX Pr
The defined function can be discrete or continuous along the x-axis. Constraints on the cumulative distribution function are:
xforxFX ,10
0XF and 1XF
XF is non-decreasing as x increases
1221Pr xFxFxXx XX
Returning to random variables …
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 3 of 66 ECE 3800
For any experiment with a sample space, we map the outcomes into a numerical space, assigning a real number (or numbers). Therefore, it is a mapping from experiments and events to numbers with values. For consistency on a number line, we define the events in terms of segments beginning at -∞ up to the value x as xX .
All sets of engineering interest can be written as countable unions or intersections of events on the interval (-∞,x]. Thus, if X is a random variable, the set of points is an event.
Under the mapping X we have generated a new probability space (R1,B,PX), where R1 is the real line, B is the Borel σ-field of all subsets of R1 generated by all unions, intersections, and complements of the semi-infinite interval (-∞,x], and PX is a set of function assigning a number PX[A]≥0 to each set BA .
Figure 2.2-1 Symbolic representations of the action of the random variable X.
Enough of the theoretical semantics ….
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 4 of 66 ECE 3800
2.3 Cumulative Distribution Function
In some text this is call the Probability Distribution Function but this is too close to another required term … the probability density function often referred to as the pdf.
The CDF is defined by
xPXXPxF XX ,:
You may also see
xXPxF XX
For discrete probability of discontinuous curves, the probability is “inclusive” of x or as the text book suggests; the CDF value immediately to the right of x.
Properties of the CDF, xFX
i. 0XF and 1XF
ii. XF is non-decreasing as x increases. For 2121 xFxFxx XX
iii. xFX is continuous from the right, that is,
0,lim0
xFxF XX
Derivable concepts
xforxFX ,10
1221 xFxFxXxP XX
Opened and close interval derivations: (note these are needed for discrete probability)
11221 xXPxFxFxXxP XX
21221 xXPxFxFxXxP XX
211221 xXPxXPxFxFxXxP XX
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 5 of 66 ECE 3800
For discrete events, the probability density function, on the x-axis, consists of discrete steps “climbing” towards 1 at the appropriate points.
For a six-sided die,
6
161,Pr intint egereger aaX
The probability density function can be defined as:
For discrete events, 061,Pr intint egereger aaX or
061,Pr intintintint egerXegerXegereger aFaFaaX
There will be a difference for continuous events … coming soon.
Examples:
6
111Pr XFX
2
133Pr XFX
6
555Pr XFX
0.177Pr XFX
6
2
6
41414Pr XFX
6
3
6
2
6
52552Pr XX FFX
From: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 6 of 66 ECE 3800
For continuous events, the CDF consists of a continuous, non-decreasing curve. For example:
10
xFX
1.0
0.0x
-10
Examples:
2
100Pr XFX
4
155Pr XFX
20
13103
20
133Pr XFX
20
3107
20
177Pr XFX
4
1
20
5105
20
11515Pr XFX
20
2101
20
1101
20
11111Pr XX FFX
What about 0Pr X ?
20
210
20
110
20
1Pr
XX FFX
020
02
20
2limPrlim
00
X
As there are an infinite number of points in any region of the x axis, the probability of any specific point for a continuous distribution is zero.
From: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 7 of 66 ECE 3800
Example 2.3-2: Waiting for a bus
A bus arrives at random in the interval T,0 . For the random variable related to the bus arrival, the bus is equally likely of coming at any time during the interval (uniformly distributed). Then
tT
TtT
t
t
tFX
,1
0,
0,0
Figure 2.3-2 Cumulative distribution function of the uniform random variable X of Example 2.3-2.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 8 of 66 ECE 3800
Example 2.3-3: Binomial Distribution Function
Flipping 4 coins and counting the number of heads, but in the case, each of the coins is unfair with a probability p=0.6 of coming up heads.
x
k
knkX pp
k
nxF
0
1
Note: The textbook is in error, use the above formula (k should be j in text).
Figure 2.3-3 Cumulative distribution function for a binomial RV with n = 4, p = 0.6.
see MATLAB: Ex_2_3_3.m Discrete probability density function 2.560% of the time get 0 heads 15.360% of the time get 1 heads 34.560% of the time get 2 heads 34.560% of the time get 3 heads 12.960% of the time get 4 heads Discrete cumulative distribution function 2.560% of the time get 0 or fewer heads 17.920% of the time get 1 or fewer heads 52.480% of the time get 2 or fewer heads 87.040% of the time get 3 or fewer heads 100.000% of the time get 4 or fewer heads
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 9 of 66 ECE 3800
Example 2.3-4: Binomial Distribution Function Computations Discrete probability density function 2.560% of the time get 0 heads 15.360% of the time get 1 heads 34.560% of the time get 2 heads 34.560% of the time get 3 heads 12.960% of the time get 4 heads Discrete cumulative distribution function 2.560% of the time get 0 or fewer heads 17.920% of the time get 1 or fewer heads 52.480% of the time get 2 or fewer heads 87.040% of the time get 3 or fewer heads 100.000% of the time get 4 or fewer heads
(a) ?35.1 XPX Note the upper less than sign instead of less than or equal to and non-integer:
1335.13335.1 XXXXXXX FPFFPFXP
3456.01792.03456.087040.035.1 XPX
(b) ?30 XPX Note the lower greater than or equal to sign:
00330 XXXX PFFXP
8704.00256.00256.08704.030 XPX
(c) ?8.12.1 XPX
Note the non-integer values:
112.18.18.12.1 XXXXX FFFFXP
01792.01792.08.12.1 XPX
(c) ?399.1 XPX Note the upper less than sign instead of less than or equal to and non-integer:
99.1133
99.199.133399.1
XXXX
XXXXX
PFPF
PFPFXP
3456.001792.03456.08704.0399.1 XPX
Note: Equation on top of p.88 is wrong … forgot the combinatorial! See p. 86.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 10 of 66 ECE 3800
2.4 Probability Density Function (the pdf)
The derivative of the cumulative distribution function
dx
xdFxFxFxf XXX
X
0lim
Assumption …the CDF is continuous and differentiable.
Properties of the pdf, if it exists, include
1. xforxf X ,0
2. 1
XXX FFdxxf
3. xXPduufxFx
XX
4. 1221
2
1
Pr xFxFdxxfxXx XX
x
x
X
From: http://en.wikipedia.org/wiki/Probability_density_function
In mathematics, a probability density function (pdf) serves to represent a probability distribution in terms of integrals. A probability density function is everywhere non-negative and its integral from −∞ to +∞ is equal to 1. If a probability distribution has density f(x), then intuitively the infinitesimal interval [x, x + dx] has probability f(x) dx.
An interpretation is
xFdxxFdxxXxPdxxf XXX
This helps with discrete functions ….
Observe that if xf X exists, meaning that it is bounded and has at most a finite number
of discontinuities, then xFX is continuous and therefore 0 xXP (except at the discontinuities).
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 11 of 66 ECE 3800
ProbabilityMassFunction(pmf)
The probability that a discrete random variable takes on an exact value is defined as the pmf.
xXxf X Pr
xFxFxf XXX
Note that for discrete random variables, the cumulative distribution function, CDF, is not continuous at the discrete inputs of interest.
Properties of the pdf include
1. xforxf X ,0
2. 1
u
X uf
3.
x
u
XX ufxF
4.
2
1
21Prx
xuX ufxXx
From: http://en.wikipedia.org/wiki/Probability_mass_function
In probability theory, a probability mass function (abbreviated pmf) gives the probability that a discrete random variable is exactly equal to some value. A probability mass function differs from a probability density function in that the values of the latter, defined only for continuous random variables, are not probabilities; rather, its integral over a set of possible values of the random variable is a probability.
Note: The textbook does not differentiate between the probability density function and probability mass function. Notice that in the definitions, the pmf represents the actual probability while the pdf is defined in terms of the derivative of the “distribution” function (CDF).
If you wish to pursue correct mathematical derivations, use pmf and pdf and CDF. If you just intend to apply this concept to engineering problems, you can do it like the textbook. …
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 12 of 66 ECE 3800
Examples:
1 2 3 4 5 6
xf X1.0
0.0x
1/6
Cumulative Distribution Function (CDF) Probability Mass Function (pmf)
10
xFX
1.0
0.0x
-10 10
xf X
1.0
0.0x
-10
1/20
Cumulative Distribution Function (CDF) Probability Density Function (pdf)
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 13 of 66 ECE 3800
Examples:
Given a CDF of
xfor
xFX ,
5tan
21
2
1 1
Define the pdf …
The derivative of the CDF is the pdf. Therefore,
xfor
xxf X ,
5
1522
Math hint: dx
du
uu
dx
d
221
1
1tan
From: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9
-100 -80 -60 -40 -20 0 20 40 60 80 1000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1Probability Distribution Function (PDF)
-100 -80 -60 -40 -20 0 20 40 60 80 1000
0.01
0.02
0.03
0.04
0.05
0.06
0.07Probability Density Function (pdf)
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 14 of 66 ECE 3800
UpdatingPreviousExamples
Experiment: Flip two Coins and count the number of heads
HHHTTHTTSPair ,,, 2,1,0S
For xXxFX Pr
xfor
xfor
xfor
xfor
xFX
2,1
21,43
10,41
0,0
And the probability mass function, xXxf X Pr , is then
else
xfor
xfor
xfor
xf X
,0
2,41
1,42
0,41
1 2 3 4
xFX
1.0
0.0x
0-1 1 2 3 4
xf X
1.0
0.0x1/4
0-1
1/2
Cumulative Distribution Function (CDF) Probability Mass Function (pmf)
Note: The pmf corresponds to Bernoulli trials of 0, 1, and 2 occurrences in 2 trials with a probability of 50%.
knkn qp
k
nkptrialsnintimeskoccuringA
Pr
4
15.0
0
20 2
2
p
4
25.0
1
21 2
2
p
4
15.0
2
22 2
2
p
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 15 of 66 ECE 3800
First Look describing “named” random variables
Note: there are documents describing specific discrete and continuous pmf, pdf and PDF or CDF on the password web site.
UniformRandomVariables
The uniform random variable arises in situations where all values in an interval of the real line are equally likely to occur. The uniform random variable U in the interval [a,b] has pdf:
bxandx
bxaabxfU
0,0
,1
bx
bxaab
ax
x
xFU
,1
,
0,0
xFX
x
xf X
x
Note: this is the “generic derivation”. It is applicable for all cases! Some classic problems: Arrival time: Uniform density on 0 to T or on –T1 to + T2 Random Phase angles: Uniform from 0 to 360 degrees or 0 to 2 pi or –pi to pi.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 16 of 66 ECE 3800
ExponentialRandomVariables
The exponential random variable arises in the modeling of the time between occurrence of events (e.g., the time between customer demands for call connections), and in the modeling of the lifetime of devices and systems. The exponential random variable X with parameter l has pdf
0,exp
0,0
xx
x
dz
xdFxf X
X
0,exp1
0,0
xx
xxFX
Expected waiting times for “services” (computer, traffic, stores, etc.) The modeling the lifetimes of devices and systems. (probability of failure increases exponentially in time). Interesting note … this distribution has a “memoryless” property. As time or values passes, the probability remains exponential …
Fx(x
)
f x(x)
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 17 of 66 ECE 3800
TheGaussianprobabilitydensityfunction(pdf)
The Gaussian or Normal probability density function is defined as:
xforx
xf X ,2
exp2
12
2
where is the mean and is the variance
The Gaussian Cumulative Distribution Function (CDF)
dv
vxF
x
v
X
2
2
2exp
2
1
The CDF can not be represented in a closed form solution!
Not yet proven reasons for importance:
1. It provides a good mathematical model for a great many different physically observed random phenomena that can be justified theoretically in many ways.
2. It is one of the few density functions that can be extended to handle an arbitrarily large number of random variables conveniently.
3. Linear combinations of Gaussian random variables lead to new random variables that are also Gaussian. This is not true for most other density functions.
4. The random process from which Gaussian random variables are derived can be completely specified, in a statistical sense, from a knowledge of the first and second moments. This is not true for other processes. All higher level moments are sums, products and/or powers of the mean and variance.
5. In system analysis, the Gaussian process is often the only one for which a complete statistical analysis can be carried through in either the linear or nonlinear situation.
6. The function is infinitely differentiable (all the derivatives exist).
-8 -6 -4 -2 0 2 4 6 80
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1Gaussian PDF and pdf
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 18 of 66 ECE 3800
Gaussian or Normal Distribution
http://en.wikipedia.org/wiki/Normal_distribution http://en.wikipedia.org/wiki/Normal_distribution#Occurrence
To summarize, here is a list of situations where approximate normality is sometimes assumed. For a fuller discussion, see below.
In counting problems (so the central limit theorem includes a discrete-to-continuum approximation) where reproductive random variables are involved, such as
Binomial random variables, associated to yes/no questions; Poisson random variables, associated to rare events;
In physiological measurements of biological specimens: The logarithm of measures of size of living tissue (length, height, skin
area, weight); The length of inert appendages (hair, claws, nails, teeth) of biological
specimens, in the direction of growth; presumably the thickness of tree bark also falls under this category;
Other physiological measures may be normally distributed, but there is no reason to expect that a priori;
Measurement errors are assumed to be normally distributed, and any deviation from normality must be explained;
Financial variables The logarithm of interest rates, exchange rates, and inflation; these
variables behave like compound interest, not like simple interest, and so are multiplicative;
Stock-market indices are supposed to be multiplicative too, but some researchers claim that they are Levy-distributed variables instead of lognormal;
Other financial variables may be normally distributed, but there is no reason to expect that a priori;
Light intensity The intensity of laser light is normally distributed; Thermal light has a Bose-Einstein distribution on very short time scales,
and a normal distribution on longer timescales due to the central limit theorem.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 19 of 66 ECE 3800
TheGaussianCumulativeDistributionFunctionis
dv
vxF
x
v
X
2
2
2exp
2
1
The CDF can not be represented in a closed form solution!
xforx
xf X ,2
exp2
12
2
Important notes on the curve:
1. There is only one maximum and it occurs at the mean value.
2. The density function is symmetric about the mean value.
3. The width of the density function is directly proportional to the standard deviation, . The width of 2 occurs at the points where the height is 0.607 of the maximum value. These are also the points of the maximum slope. Also note that:
683.0Pr X
955.022Pr X
4. The maximum value of the density function is inversely proportional to the standard deviation, .
2
1Xf
5. Since the density function has an area of unity, it can be used as a representation of the impulse or delta function by letting approach zero. That is
2
2
0 2exp
2
1lim
xx
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 20 of 66 ECE 3800
ConversionoftheGaussiantothestandardnormal
The CDF is tabulated for a zero mean, unit variance in Table 1 page G-3 of the appendix. For these values, it is often described as “normalized” and is defined as
dyy
2exp
2
1 2
The distribution function is then converted based on the normalizing relationship
x
y
x
yxFX
When using Appendix D, the negative values of u are derived from the positive as
uu 1
AnotherwaytofindvaluesfortheGaussian
The error function, defined as (Note: this is not your textbook definition of erf !!)
duuxerf
x
u
0
2exp2
22
1
2
1
21
2
11
xerf
xerfxFX
For multiple bounds
22
1
2
1
22
1
2
11
aerf
berfFbFbXaP XXX
22
1
22
11
aerf
berfFbFbXaP XXX
For some reason, the textbook has defined erf() differently than MATLAB and EXCEL and WIKIPEDIA. Other sources do it the author’s way …
This may be a problem … to be determined!!
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 21 of 66 ECE 3800
MeanandVarianceofarandomvariable(ImportantNotEmphasizedinText)
In chapter 4 we will be doing this a lot (like for every density function used), but in the meantime …
The mean value of a random variable is defined as
dxxfxXEX X
For a discrete random variable, integration is replaced by summation and
i
iXii
iXi xPxxfxXEX
The variance of a random variable is defined as
dxxfxXEXX X2222
For a discrete random variable, integration is replaced by summation and
i
iXii
iXi xPxxfxXEXX 22222
Often when we talk about values we say
That is to say that we expect the result x to be
x
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 22 of 66 ECE 3800
2.5 Continuous, Discrete and Mixed R.V.
see “Continuous RV” and Discrete RV” on the solution web site ….
derived from the ECE 5820 textbook: Alberto Leon-Garcia, “Probability, Statistics, and Random Processes For Electrical Engineering, 3rd ed., Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
A Continuous R.V. as a continuous density (pdf) and distribution (CDF).
integrals and derivatives are used
A Discrete R.V. as a CDF composed of steps and a pdf composed of delta functions with magnitude.
summations or differences are used Dirac delta functions (another area where engineers drive mathematicians crazy)
A Mixed R.V. has both constructs present. There will be cases where a continuous R.V. becomes a Mixed R.V. due to conditionals or other “knowledge gained” about an event.
mixed math is likely required (integrals and sums along with derivatives and differences)
Some Continuous RV Densities
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 23 of 66 ECE 3800
Defining proper CDF/pdf functions
Example: xxFX1tan1
Determine the values of and .
Known values at 0XF and 1XF and monotonically increasing, meaning that the derivative is positive.
0tan1 1 XF
02
1
XF
02
1
02
1
2
1tan1 1 XF
12
1
XF
12
21
2
1
Therefore,
xxFX
1tan2
12
1
and
21
11
xxf X
As a check, it is positive for all selections of x!
This is a Cauchy Random Variable density and distribution! (It has really odd properties.)
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 24 of 66 ECE 3800
Exercise 2-3.1Cooper and McGillem: A probability density function
xuexf xKX 5
Determine the value of K
dxxf X1
0
51 dxe xK
0
151 xKe
K
5111
51 0
Ke
Ke
KKK
5K
Therefore xuexf xX 55
As an extension or generalization, note that xueKxf xKX
Probability X>1
1
11Pr dxxfX X
1
0
5511Pr dxeX x
1
0
5
5
511Pr xeX
0067.01111Pr 55 eeX
Probability X≤0.5
5.0
0
555.0Pr dxeX x
5.0
0
5
5
55.0Pr xeX
9179.01115.0Pr 5.25.2 eeX
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 25 of 66 ECE 3800
Example: Archery target shooting with
A Rayleigh distribution - two dimensional Gaussian
Archer capability described by 4
125.0 YX in feet
0,0
0,2
exp2
2
2
rfor
rforrr
rfR
0,0
0,2
exp12
2
rfor
rforr
rFR
0,0
0,8exp16 2
rfor
rforrrrfR
0,0
0,8exp1 2
rfor
rforrrFR
Assume a 1 foot radius target with a 1 inch radius Bulls-eye
The archers expected performance can be described by ….
Probability of a Bulls-eye (1 inch radius)
0540.0144
8exp1
12
18exp1
12
1 2
RF
Probability of missing the target (1 foot radius)
42 1035.38exp18exp1111 RF
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 26 of 66 ECE 3800
Some Common Continuous Random Variables
1.Uniform
otherwise
bxaabxfU
,0
,1
xb
bxaab
ax
ax
xFU
,1
,
,0
2. Triangle
xfor
xforx
xforx
xfor
xf X
1,0
10,1
01,1
1,0
xfor
xforxx
xforxx
xfor
xFX
1,1
10,2
1
2
01,2
1
2
1,0
2
2
3. Exponential
0,exp
1
0,0
0,exp
0,0
xx
x
xx
xxf X
0,exp1
0,0
0,exp1
0,0
xx
x
xx
xxFX
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 27 of 66 ECE 3800
4. Laplacian
0,2
exp2
1
xxf X
5. Rayleigh
0,0
0,2
exp2
2
2
rfor
rforrr
rfR
0,0
0,2
exp12
2
rfor
rforr
rFR
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 28 of 66 ECE 3800
Triangular density function example
Assume that a random variables probability density function is triangular and can be described as
xfor
xforx
xforx
xfor
xf X
1,0
10,1
01,1
1,0
Find the cumulative distribution function.
The definition
x
v
XX dvvfxF
For 1 x 0xFX
For 01 x
x
v
X dvvxF
1
1
x
Xv
vxF
1
2
2
2
1
22
11
2
22
x
xxxxFX
For 10 x
x
v
X dvvxF
0
12
1
x
Xv
vxF
0
2
22
1
2
1
222
1 22
x
xxxxFX
For x1 1xFX
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 29 of 66 ECE 3800
Therefore,
xfor
xforxx
xforxx
xfor
xFX
1,1
10,2
1
2
01,2
1
2
1,0
2
2
There will be numerous times that problems must be broken into multiple subproblems to perform solutions as segments … like here for -1<x<0 and 0<x<1.
-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-0.2
0
0.2
0.4
0.6
0.8
1
1.2
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 30 of 66 ECE 3800
Some Common Discrete Random Variables
1. Bernoulli 1,0XS
qpp 10 and pp 1 , for 10 p
else
kp
kq
kpmfkPB
,0
1,
0,
1 kpkqkpmfkPB
k
kq
k
kFB
1,1
10,
0,0
1 kupkuqkpmfkFB
2. Binomial nS X ,,2,1,0
knkk pp
k
np
1 , for nk ,,2,1,0
else
nkppk
n
kpmfkPknk
B
,0
,,1,0,1
kn
nkppj
n
k
kFk
j
jnjB
,1
0,1
0,0
0
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 31 of 66 ECE 3800
4. Geometric
First Version
,2,1,0XS
else
kppkpmfkP
k
B,0
,,1,0,1
kpp
k
kF k
j
jB 0,1
0,0
0
Math Tricks ….
1,1
11
1
00
qfor
q
qpqppp
kk
j
jk
j
j
1
1
0
111
11
k
kk
j
j qp
qppp
Therefore, it is commonly stated as
kq
qp
k
kF kB 0,
1
1
0,01
Second Version
,2,1XS
else
kppkpmfkP
k
B,0
,,2,1,1 1
kq
qp
k
kF kB 1,
1
1
1,0
3. Poisson
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 32 of 66 ECE 3800
,2,1,0XS
ek
pk
k !, for ,2,1,0k
else
nkekkpmfkP
k
B
,0
,,1,0,!
kk
e
k
kF k
j
kB 0,
!
0,0
0
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 33 of 66 ECE 3800
Example 2.4-5 Cell phone received signal power model.
The power can be described as a Rayleigh distribution.
rforrr
rf R
0,2
exp2
2
2
rforr
rFR
0,2
exp12
2
Assume mW1 for the r power radius.
What is the probability that the power W is less than 0.8 mW?
2
2
12
8.0exp18.0RF
or
8.0
02
2
2 12exp
18.0 dr
rrPR
Hint:
2
2
2
2
2 2exp
2exp
2
2
xdx
xx
2
0exp
2
8.0exp
2exp8.0
228.0
0
2rPR
29.032.0exp18.0 RP
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 34 of 66 ECE 3800
2.6 Conditional and Joint Distributions and Densities
Using the Cumulative Distribution Function (CDF), define
BxXBxFX |Pr|
B
BxXBxXBxFX Pr
,Pr|Pr|
, for 0Pr B
where BxX , is the event of all outcomes such that
xX and B
Note: X is the value of the random variable when the experimental outcome is .
The event B that conditions the probability has several possibilities:
1. Every B may be an event that can be expressed in terms of the random variable X. Which means we have a simple conditional probability.
2. Every B may be an event that depends upon some other random variable, which may be either continuous or discrete. A joint probability or independent Prob.
3. Every B may be an event that depends upon both the random variable X and some other random variable. Even more complicated.
For our purposes (#1 above) envision that
mXB or mXB
Then, it can be shown that BxF | is a valid cumulative distribution function with all the expected characteristics:
1. xforBxF ,1|0
2. 0| BF and 1| BF
3. BxF | is non-decreasing as x increases
4. BxFBxFBxXx |||Pr 1221
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 35 of 66 ECE 3800
Example 2.6-1:
Given 10 XB , compute the conditional CDF for values xX .
For any continuous function …
1|10 BxFX
We know that event B has occurred … therefore any resulting CDF for x>10 is the same as 1| BF .
The new function only has non-unity values where 10x . But the CDF must be rescaled by BFX
10,,
10,1
|x
BF
BxF
x
BxF
X
XX
Figure 2.6-1 Conditional and unconditional CDFs of X.
Based on the condition B, is known that 10 XB . Therefore, the conditional CDF
must go to 1.0 for BC or for 10 XBC . Also as a result, the density function in this region becomes 0.0 (the derivative of a constant). It can be seen that …
10,
10,0
|x
BF
xf
x
Bxf
X
XX
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 36 of 66 ECE 3800
For the new “subspace based on the condition B
If we define the density using the derivative,
dx
BxFdBxf X
X
||
In general, envision that
BF
xfBxf
X
XX | , for Bx
and
0| Bxf X , for Bx
Again, the density function is a scaled version of the original density function in the range where the event B exists and is 0 outside of B.
Properties of the new pdf must include
1. xforMxf ,0|
2. 1|
dxMxf
3. duMufMxF
x
||
4. dxMxfxXx
x
x
2
1
|Pr 21
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 37 of 66 ECE 3800
Example p. 100 Cooper and McGillem: Conditional Gaussian density function given that the event, M, is less than or equal to the mean value.
xforx
xf X ,2
exp2
12
2
where is the mean and is the variance
For the event M,
21 MxF
Then, constructing a conditional cumulative distribution, MXxF | , the density function is defined as
2
2
2exp
2
2
21
|
xxf
XF
xfMxf , for x
and 0| Mxf , for x
The previous and new pdf and CDF can be “sketched” as: (see CondGaussianRandn.m)
-4 -3 -2 -1 0 1 2 3 40
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1Cond. gaussian Dist. and Density Fundtions
PDFc-pdf
C-PDF
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 38 of 66 ECE 3800
Distribution function as a weighted sum of conditional distribution functions.
Define a mutually exclusive, exhaustive set of events: niAi ,,1, .
Note that: 11
n
iiAP
Then total probability requires that
n
iiiXX APAxFxF
1
|
Example 2.6-3: good and defective memory chips with different CDFs
xux
defectxFX
2exp1|
xux
goodxFX
10exp1|
Two different exponential failure rates, one much faster than the other!
Also assume we are given
61 qdefectP and 6
5 pgoodP
Then a total probability for an IC is
goodPgoodxFdefectPdefectxFxF XXX ||
6
5
10exp1
6
1
2exp1
xxxFX
10exp
6
5
2exp
6
11
xxxFX
What is the probability that a chip will fail before 6 months?
534.010
6exp
6
5
2
6exp
6
116
XF
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 39 of 66 ECE 3800
Joint Cumulative Distribution Functions
Cumulative Distribution Function:The probability of the event that the observed random variable X is less than or equal to the allowed value x and that the observed random variable Y is less than or equal to the allowed value y.
yYxXyxFXY ,Pr,
The defined function can be discrete or continuous along the x- and y-axis. Constraints on the cumulative distribution function are:
1. yandxforyxFXY ,1,0
2. 0,,, XYXYXY FxFyF
3. 1, XYF
4. yxFXY , is non-decreasing as either x or y increases
5. xFxF XXY , and yFyF YXY ,
Analogies:
a 2-dimensional probability moving from scalars to vectors (2 or more elements) Calc 3 as compared to Calc 1 & 2?
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 40 of 66 ECE 3800
2-D probability computations: yYyxXxP 121 ,
Think in terms of unions and intersections of 2-D boxes in the x,y plane …
Figure 2.6-4 Point set associated with the event {X ≤ x, Y ≤ y}.
Figure 2.6-5 Point set for the event {x1 < X ≤ x2, y1 < Y ≤ y2}.
Then by inspection we can arrive at …
11211222121 ,,,,, yxFyxFyxFyxFyYyxXxP XYXYXYXY
Read the textbook p. 118-120 and see if you prefer the mathematical way … … math works, geometry provides some intuition …
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 41 of 66 ECE 3800
JointProbabilityDensityFunction(pdf)
The derivative of the cumulative distribution function is the density function
yx
xFyxf X
2
,
Properties of the pdf include
1. yandxforyxf ,0,
2. 1,
dydxyxf
Note: the “volume” of the 2-D density function is one.
3.
y x
dvduvufyxF ,,
4. dyyxfxf X
, and dxyxfyfY
,
5. 2
1
2
1
,,Pr 2121
y
y
x
x
dydxyxfyYyxXx
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 42 of 66 ECE 3800
UniformDensityExample
The uniform density function in two dimensions can be defined as:
else
yyyandxxxforyyxx
yxf YX
,0
,1
, 21211212
,
Determine the density function in y
2
1
,,
x
x
YXY dxyxfyf
2
1
2
1 12121212
1x
x
x
x
Y yyxx
xdx
yyxxyf
21
121212
12 ,1
yyyforyyyyxx
xxyfY
Similarly, the density function in x is
2112
,1
xyxforxx
xf X
Note: this is a characteristic of random variables that are independent …
yfxfyxf YXYX ,,
and
yFxFyxF YXYX ,,
The inverse is also true … if the joint pdf and CDF have this property, then the random variables X and Y are independent!
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 43 of 66 ECE 3800
Exercise 3-1.2 Cooper and McGillem
else
yandxforyxAyxf YX
,0
00,32exp,,
Determine A
0 0
32exp,1 dydxyxAdydxyxf
0 00 0 2
2exp3exp2exp3exp1 dy
xyAdydxxyA
3
1
2
1
3
3exp
2
13exp
2
11
00
Ay
AdyyA
6A
Determine the Distribution Function
y x
dydxyxyxF0 0
32exp6,
y x
dydxxyyxF0 0
2exp3exp6,
y
dyx
yyxF0 2
2exp
2
13exp6,
yxyx
yxF
3exp12exp13
3exp
3
1
2
2exp
2
16,
Then, for
4
1,
2
1Pr yx
4
13exp1
2
12exp1
4
1,
2
1
4
1,
2
1Pr Fyx
4
3exp11exp1
4
1,
2
1
4
1,
2
1Pr Fyx
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 44 of 66 ECE 3800
Conditional Probability (Again, with multiple r.v.)
Using the Cumulative Distribution Function (CDF), define
yF
yxF
M
MxXyYxF
YX
,
Pr
|Pr|
Another way.
12
1221
,,|
yFyF
yxFyxFyYyxF
YYX
Leading to (for the Y interval going to zero),
yf
yxfyYxf
YX
,|
and
xf
yxfxXyf
XY
,|
These are different from the probability of a continuous distribution taking on a single value in X and Y…
0 xXFX or 0 yYFY
An engineering derivation follows:
y
yFyyFy
yxFyyxF
yFyyF
yxFyyxFyYxF
YYyYYyX
,,
lim,,
lim|00
yf
duyuf
yyF
yyxF
yYxFY
x
YX
,,
|
Then taking the partial with respect to x
yf
yxf
yyF
xyyxF
yYxfxyYxF
YYX
X,
,
||
2
yf
yxfyYxf
YX
,|
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 45 of 66 ECE 3800
The corresponding conditional density function is
yf
yxfyYxf
Y
,|
and similarly it can be shown that
xf
yxfxXyf
X
,|
From these equations, it can also be seen that
xfxXyfyfyYxfyxf XY ||,
This provides a way to compute the joint density function based on a conditional density function.
JointDensitytomarginaldensitycomputations.
The joint density total probability concepts can define the x and y marginal densities.
dyyxfxf X
, and dxyxfyfY
,
Then from the conditional density relationship with the joint density
xfxXyfyfyYxfyxf XY ||,
We can replace the joint density functions in the total probability equations to define the pdf densities of x and y based on the conditional densities as
dxxfxXyfdxyxfyf XY
|,
or
dyyfyYxfdyyxfxf YX
|,
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 46 of 66 ECE 3800
To derive the multiple variables Bayes Theorem, we return to
xfxXyfyfyYxfyxf XY ||,
Equating the right two elements result in ….
yf
xfxXyfyYxf
Y
X
||
or
xf
yfyYxfxXyf
X
Y
||
Important Note: the joint probability density function completely specifies:
both marginal density functions and
both conditional density functions.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 47 of 66 ECE 3800
Independence Random Variables
Property
yfxfyxf YX ,
and
yFxFyxF YX ,
Then
xFyF
yFxF
yF
yxFyYxF X
Y
YX
Y
XYXY
,|
yFxF
yFxF
xF
yxFxXyF Y
X
YX
X
XYXY
,|
and xfyYxf XXY |
yfxXyf YXY |
Independence simplifies required computations, so pay attention to problems statements!
Usefulresultsofindependenceforjointdensityfunctions
yfxfxfxXyfyfyYxfyxf YXXY ||,
Therefore
yfxfxfxXyf YXX |
yfxXyf Y|
and
xfyYxf X|
Note: if they are independent, knowing one does not help with the other!
It only matters if x and y are correlated in some way … and not independent.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 48 of 66 ECE 3800
Example 2.6-9 Mapping to create a joint density function
The experiment involves rolling one die, having equally likely outcomes from 1 to 6.
6,5,4,3,2,1 and 6,,2,1,61Pr ifori
We define two new random variables,
6,5,4,2,0
1,2
3,4
for
for
for
X
6,5,4,1,0
2,1
3,2
for
for
for
Y
Then, the pmfs become
0,64
2,61
4,61
xfor
xfor
xfor
xpmf
0,64
1,61
2,61
yfor
yfor
yfor
ypmf
2,4,,61
1,4,,0
0,4,,0
2,2,,0
1,2,,0
0,2,,61
2,0,,0
1,0,,61
0,0,,63
,
yxfor
yxfor
yxfor
yxfor
yxfor
yxfor
yxfor
yxfor
yxfor
yxpmf
Note that: ypmfxpmfyxpmf ,
A 2-D plot “describing” the location of the 2D-pmf is shown in the textbook.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 49 of 66 ECE 3800
Example p. 126 Cooper and McGillem for independence.
1010,15
6, 2
, yandxforyxyxf YX
Computing the marginal densities:
1
0
31
0
2
35
61
5
6
y
xxdxyxyfY
31
5
6 yyfY
and
1
0
22
1
0
2
25
61
5
6
yxydyyxxf X
21
5
6 2xxf X
Note that yfxfyxf YX , , Therefore the variables are not independent!
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 50 of 66 ECE 3800
Exercise 3-3.2 from Cooper and McGillem
Assume X and Y independent
xforxxf X ,1exp5.0
yforyyfY ,1exp5.0
Find 0Pr YX ? That is, the product of the random variables is positive.
0Pr0Pr0Pr0Pr0Pr YXYXYX
0001010Pr YxYx FFFFYX
Find the distribution for X and Y based on the ranges defined for the absolute value
x
X dxxxF 1exp5.0
For 1 xfor and xfor1
1 xfor xfor1
x
X dxxxF 1exp5.0
x
X
xxF
1
1exp5.0
1exp5.0 xxFX
x
X dxxxF1
1exp5.05.0
x
X
xxF
11
1exp5.05.0
xxFX 1exp15.05.0
1839.01exp5.000 YX FF
0338.01839.01839.000 YX FF
6660.01839.011839.010101 Yx FF
6998.00001010Pr YxYx FFFFYX
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 51 of 66 ECE 3800
Gaussian
The Gaussian or Normal probability density function is defined as:
xforx
xf X ,2
exp2
12
2
where is the mean and is the variance
The Gaussian Cumulative Distribution Function (CDF)
dv
vxF
x
v
X
2
2
2exp
2
1
The CDF can not be represented in a closed form solution!
Normal – Gaussian with zero mean and unit variance.
The Normal probability density function is defined as:
xfor
xxN ,
2exp
2
1 2
The Normal Cumulative Distribution Function (CDF)
dvv
xx
v
N
2exp
2
1 2
Note the relationship between the Gaussian and Gaussian-Normal
x
xF XX
see the MATLAB: GaussianDemo.m
-8 -6 -4 -2 0 2 4 6 80
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1Gaussian PDF and pdf
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 52 of 66 ECE 3800
Joint Gaussian: Independent X and Y
For X and Y independent:
2
2
2
2
2exp
2
1
2exp
2
1,
X
X
XY
Y
Y
XY
xyyxf
2
2
2
2
22exp
2
1,
X
X
Y
Y
XYXY
xyyxf
If both functions have a zero mean and identical variances
2
22
2 2exp
2
1,
xy
yxf XY
see Example 2.6-10
Figure 2.6-10 Graph of the joint Gaussian density.
This has been referred to as a hat function ….
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 53 of 66 ECE 3800
Example 2.6-11 independent Gaussians, zero mean unit variance
Rectangular to circular conversion …
22 xyr and
x
yatan
Note that for an infinitesimal area ddrrdydx
Then, for cumulative distribution function
y x
XY ddyxF 2
exp2
1,
22
We could consider a change to circular area as
r
R ddrF0
2
0
2
2exp
2
1,
r
R ddrF0
2
0
2
2
1
2exp,
r
R drF0
2
2exp
12
exp2
exp2
0
2
rrF
r
R
2exp1
2rrFR
And the probability density function is
2exp
2rrrf R
with
20,2
1f
Also they are they are independent …
randrr
rf R
020,
2exp
2,
2
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 54 of 66 ECE 3800
JointGaussian:NotIndependent
Assume that X and Y are not independent random variables and have equal variances. :
22
22
22 12
2exp
12
1,
xyxyyxf XY
Note: is a correlation coefficient between the two random variables. The description of correlation coefficients is coming in Chap. 4. They are rather important.
Note that if = 0, X and Y are uncorrelated.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 55 of 66 ECE 3800
Example 2.6-12
Consider the joint pdf:
.
otherwise
yxyxAyxf XY
,0
10,10,,
(i) Compute A.
Performing the double integral
1
0
1
0
1
0
1
0
,1 dydxyxAdydxyxf XY
1
0
21
0
1
0
2
12
1
21 dyyAdyyx
xA
AAy
yA
2
11
2
1
22
11
21
0
2
Therefore
otherwise
yxyxyxf XY
,0
10,10,1,
(ii) Compute the marginal density functions for x and y
10,1,1
0
1
0
ydxyxdxyxfyf XYY
10,2
1
2
1
0
2
yyyx
xyfY
Similarly
10,2
1 xxxf X
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 56 of 66 ECE 3800
(iii) Compute the Joint CDF
y xy x
XYXY dvduvudvduvufyxF0 00 0
1,,
yy x
XY dvvxx
dvvuu
yxF0
2
0 0
2
22,
2222,
22
0
22 yxy
xvxv
xyxF
y
XY
22
2
1, yxyxyxFXY
Now we are computing the “regions” of Figure 2.6-9
Figure 2.6-9 Shaded region in (a) to (e) is the intersection of supp(fXY) with the point set associated with the event {−∞ < X ≤ x,−∞ < Y ≤ y}. In (f), the shaded region is the intersection of supp(fXY) with {X + Y ≤ 1}.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 57 of 66 ECE 3800
(a) Compute yx 1,1Pr
1,11,1Pr XYFyx
111112
11,1 22 XYF
(b) Compute yx 1,10Pr
1,1,10Pr xFyx XY
xxxxxFXY 222
2
111
2
11,
(c) Compute 10,1Pr yx
yFyx XY ,1,10,1Pr
222
2
111
2
1,1 yyyyyFXY
(d) Compute 10,10Pr yx
22
2
1,10,10Pr yxyxyxFyx XY
(e) Compute 10,0Pr yx
0002
1
,010,0Pr
22
yy
yFyx XY
Similarly Compute 0,10Pr yx
0002
1
0,0,10Pr
22
xx
xFyx XY
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 58 of 66 ECE 3800
(e) Compute 1Pr YX
1
,1Pryx
XY dydxyxfYX
If we let x drive the function,
x goes from 0 to 1
Meanwhile, y goes from 0 to 1-x for each value of x
This allows us to perform the double integral as shown below!
1
0
1
0
1
0
1
0
1,1Pr dxdyyxdxdyyxfYXxx
XY
1
0
21
0
1
0
2
2
11
21Pr dx
xxxdx
yyxYX
x
1
0
21
0
22
22
1
2
211Pr dx
xdx
xxxxYX
3
1
6
2
6
1
2
1
621Pr
31
0
3
xxYX
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 59 of 66 ECE 3800
Failure Rates
Simple approximation for failure rates: Use the exponential density function!
0,exp
0,0
tt
ttfT
0,exp1
0,0
tt
ttFT
In these equations, lambda, , is the failure rate and t is time. The failure rate is assumed to be the same constant for the lifetime of the product!
Homework Problem 2.38. A laser used to scan bar codes in a store is assumed to have a constant failure rate lambda, . What is the maximum value of lambda that will yield a probability of a first breakdown in 100 hours of operation less than or equal to 0.05?
Solution: Using the exponential CDF and pdf, the probability of a failure in time t can be defined using the CDF as …
ttFT exp1 We are given
05.01000 tP with
TTFTtP T exp10 Therefore,
05.0100exp1100 TF
100exp05.01
10005.01ln
100
95.0ln
-4105.1293
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 60 of 66 ECE 3800
Author’s Solution
Backtoamorecompletediscussionoffailurerates
Following the author’s detailed derivations …
Let X denote the time of failure. Then by Bayes’ theorem, the probability that failures occur in the time interval {t, t+dt} given that the object has survived up to time t can be written as
tXP
XtdttXtPtXdttXtP
,
|
The joint probability function contains two events, where the first event is completed contained in the second event. Therefore, the intersection of the two events produces the first event! That is
tXP
dttXtPtXdttXtP
|
If there is a CDF function that described the probability, FX, the right hand functions can be easily defined in terms of the CDF as …
tF
tFdttFtXdttXtP
x
xx
1
|
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 61 of 66 ECE 3800
Using the “engineering derivative” definition
tF
dttf
tF
dtdttFdttF
tXdttXtPx
x
x
xx
11
|
We now want to call the resulting function “the conditional failure rate”, (t).
dtttF
dttftXdttXtP
x
x
1|
Now, the conditional failure rate described the failures expected in a time interval based on the length of time the product has been in service:
This rate can …
be constant be time varying be linear or non-linear be defined any way it has too!
In many cases for failure analysis, there is a common shape for the failure rate.
“The Bathtub curve” – often product failures rates follow a curve that looks like a bathtub … a lot of initial failures (infant mortality),
… very few during the “lifetime” (constant random failures) and
… an increasing number once past the expected lifetime (things eventually wear out).
see: https://en.wikipedia.org/wiki/Bathtub_curve
also see Failure Rate at https://en.wikipedia.org/wiki/Failure_rate
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 62 of 66 ECE 3800
Determining classes of solutions ….
Based on the previous derivation
dtttF
dttftXdttXtP
x
x
1|
We can described the conditional failure rate to CDF as (again “engineering math”
dtttF
tdF
tF
dttf
x
x
x
x
11
If we assume that the CDF is a function, y, … and “integrate”
1
0
1
01
t
t
y
y
dtty
dy
We can integrate using
0
101 lnlnln
1
0y
yyy
y
dyy
y
1
0
1
0
1
0
1ln1
t
t
y
y
y
y
dttyy
dy
1
0
01 1ln1lnt
t
dttyy
1
01
0
1
1ln
t
t
dtty
y
If we relate the function, y, to the CDF and start at time t=0 …
t
X
X dtttF
F
01
01ln
We know that for a failure CDF, 00 XF and 1XF . Therefore,
t
X
dtttF 01
1ln or we have
t
X dtttF0
1ln
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 63 of 66 ECE 3800
Leading to
t
X dtttF0
1ln
t
X dtttF0
exp1
and finally
t
X dtttF0
exp1
Finding the pdf ….
t
XX dttttf
dt
tFd
0
exp
Notice for a constant conditional failure rate
t
tdttFt
X
exp1exp1
0
ttf X exp
The exponential CDF!
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 64 of 66 ECE 3800
Matlab – Examples and Concepts
rand.m
randn.m
Histogram of a probability density function
Generating random numbers from an arbitrary cumulative distribution function.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 65 of 66 ECE 3800
Uniform Density Example
The uniform density function in two dimensions can be defined as:
else
yandxforyxf YX
,02
1
2
1
2
1
2
1,1
11
1,,
Determine the density in y
2
1
,,
x
x
YXY dxyxfyf
121
211 2
1
21
21
21
xdxyfY
22
1
2
1,1 yforyfY
Similarly
2
1
2
1,1 xforxf X
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 66 of 66 ECE 3800
Distribution
y x
YX dydxyxfyxF ,,,
2
1
2
11,
21
21
, yxdydxyxFy x
YX
Detailed distribution in the entire x,y-plane
yandxfor
yandxfory
yandxforx
yandxforyx
yandxfor
yxF YX
2
1
2
1,1
2
1
2
1
2
1,
2
1
2
1
2
1
2
1,
2
1
2
1
2
1
2
1
2
1,
2
1
2
12
1
2
1,0
,,