cmpe 252a: computer networks review set:
DESCRIPTION
CMPE 252A: Computer Networks Review Set:. Quick Review of Probability Theory and Random Processes. Why Probability Theory?. Information is exchanged in a computer network in a random way, and events that modify the behavior of links and nodes in the network are also random - PowerPoint PPT PresentationTRANSCRIPT
1
CMPE 252A: Computer Networks
Review Set:Quick Review of Probability Theory and Random Processes
2
Why Probability Theory? Information is exchanged in a computer network in
a random way, and events that modify the behavior of links and nodes in the network are also random
We need a way to reason in quantitative ways about the likelihood of events in a network, and to predict the behavior of network components.
Example 1: Measure the time between two packet arrivals
into the cable of a local area network. Determine how likely it is that the interarrival
time between any two packets is less than T sec.
3
Probability Theory A mathematical model used to quantify
the likelihood of events taking place in an experiment in which events are random.
It consists of: A sample space: The set of all possible
outcomes of a random experiment. The set of events: Subsets of the sample
space. The probability measure: Defined according
to a probability law for all the events of the sample space.
4
Random Experiment A random experiment is specified by
stating an experimental procedure and a set of measurements and observations.
Example 2: We count the number of packets received
correctly at a base station in a wireless LAN during a period of time of T sec.We want to know how likely it is that the next packet received after T sec is correct.
5
Our Modeling Problem ... We want to reason in quantitative
terms about the likelihood of events that can be observed.
This requires: Describing all the events in which we are
interested for the experiment. Combining events into sets of events that
are interesting (e.g., all packets arriving with 5 sec. latency)
Assigning a number to each of those events reflecting the likelihood that they occur.
6
Probability Law Probability of an event: A number
assigned to the event reflecting the likelihood with which it can occur or has been observed to occur.
Probability Law: A rule that assigns probabilities to events in a way that reflects our intuition of how likely the events are.
What we need then is a formal way of assigning these numbers to events and any combination of events that makes sense in our experiments!
7
Probability Law Let E be a random experiment (r.e.) Let A be an event in E The probability of A is denoted by P(A) A probability law for E is a rule that assigns
P(A) to A in a way that the following conditions, taken from our daily experience, are satisfied: A may or may not take place; it has some
likelihood (which may be 0 if it never occurs). Something must occur in our experiment. If one event negates another, then the
likelihood that either occurs is the likelihood that one occurs plus the likelihood that the other occurs.
8
Probability Law More formally, we state the same as
follows. A probability law for E is a rule that
assigns a number P(A) to each event A in E satisfying the following axioms: AI: AII: AIII:
P(A) 0 ≤1 )( =SP
P(B)P(A) BAPBA +=⇒= ) ( UI φ
Everything else is derived from these axioms!
9
Important Corollaries
1.
2.
3.
4.
5.
∑=⇒≠∀Φ=k
kk
kjin APAPjiAAAAA )()( ... , , 21 UI
)(1)( APAP c −=
1)( ≤AP
0)( =ΦP
)()()()( BAPBPAPBAP IU −+=
10
Probability Law
All probabilities must be in [0, 1] The sum of probabilities must be at most 1
A
S
x
)(1 SP=
)(0 Φ=PΦ
)(AP
)(xP
11
Conditional ProbabilityEvents of interest occur within some context, and that context changes their likelihood.
time
packet collisions
timeinterarrival times
We are interested in events occurring given that others take place!
12
Conditional Probability The likelihood of an event A occurring given
that another event B occurs is smaller than the likelihood that A occurs at all.
We define the conditional probability of A given B as follows:
0)(for )(
)()|( >
∩= BP
BPBAP
BAP
We require P(B) > 0 because we know B occurs!
13
Theorem of Total Probability Purpose is to divide and conquer We can describe how likely an event is by
partitioning it into mutually exclusive pieces.
timesuccess failure
busy period idle period busy period
1B 2B 3B
14
Theorem of Total Probability1B 2B
nB...
A
)()()|()( ∑∑ ∩==i
ii
ii BAPBPBAPAP
5BA∩
Intersections of A with B’s are mutually exclusive
15
Independence of Events In many cases, an event does not depend on prior
events, or we want that to be the case. Example: Our probability model should not have to
account for the entire history of a LAN. Independence of an even with respect to another
means that its likelihood does not depend on that other event.
A is independent of B if P(A | B) = P(A) B is independent of A if P(B | A) = P(B) So the likelihood of A does not change
by knowing about B and viceversa! This also means: )()()( BPAPBAP =∩
16
Random Variables We are not interested in describing the likelihood
of every outcome of an experiment explicitly. We are interested in quantitative properties
associated with the outcomes of the experiment. Example:
What is the probability with which each packet sent in an experiment is received correctly?We don’t really care!
What is the probability of receiving no packets correctly within a period of T sec.?We care! This may make a router delete a neighbor!
17
Random Variables We implicitly use a measurement that assigns a
numerical value to each outcome of the experiment. The measurement is [in theory] deterministic; based
on deterministic rules. The randomness of the observed values of the
measurement is completely determined by the randomness of the experiment itself.
A random variable X is a rule that assigns a numerical value to each outcome of a random experimentYes, it is really a function!
18
Random Variables Definition: A random variable X on a
sample space S is a function X: S ->R that assigns a real number X(s) to each sample point s in S.
∞− ∞+
S
5s3s
2s1s4s
0)( 2sX )( 1sX )()( 54 sXsX =
ℜ⊂∈= }|)({ SssXS iiX which is called the event space
19
Random Variables Purpose is to simplify the description of the problem. We will not have to define the sample space!
∞+∞−
0
1S
s
xsX =)(
))(( xsXP =
Possible values of X
20
Types of Random Variables Discrete and continuous: Typically
used for counting packets or measuring time intervals.
time0 t
time
Busy period next packet
time?
1 2 3 4
21
Discrete Random Variables We are interested in the probability that a discrete
random variable X (our measurement!) equals a certain value or range of values.
Example:We measure the delay experienced by each packet sent from one host to another over the Internet; say we sent 1M packets (we have 1M delay measurements). We want to know the likelihood with which any one packet experiences a delay of 5 ms or less.
Probability Mass Function (pmf) of a random variable X :
The probability that X assumes a given value x
)()( xpxXP X==
22
Discrete Random VariablesCumulative Distribution Function (cdf) of a random variable X: The probability that X takes on any value in the interval ],( x−∞
)()( xXPxFX ≤=The pdf and pmf of a random variable are just probabilities and obey the same axioms AI to AIII. Therefore,
)()( b)XP(a
for )()(
0)(lim ;1)(lim ;1)(0
aFbF
babFaF
xFxFxF
XX
XX
XXxX x
−=≤<<≤
==≤≤−∞→+∞→
23
Continuous Random Variables The probability that a continuous r.v. X assumes a
given value is 0. Therefore, we use the probability that X assumes a
range of values and make that length of that range tend to 0.
The probability density function (pdf) of X, if it exists, is defined in terms of the cdf of X as
dx
xdFxf X
X
)()( =
∫
∫∫∞+
∞−
∞−
=+∞→
≤≤∞−===≤≤
yprobabilit a is pdf because ,1)( then if
)()()( ;(x)dxfb)XP(ab
a
X
dttfx
xXPdttfxF
X
x
XX
24
What We Will Use We are interested in:
Using the definitions of well-known r.v.s to compute probabilities
Computing average values and deviations from those values for well-known r.v.s
25
Mean and Variance
What is the average queue length at each router?
What is our worst case queue?
BA
D
C
3
5
4
6
1
2
7
For VC1 use 3For VC2 use 2…..For VCn use 3
VC1
VC2
26
Mean and VarianceExpected value or mean of X:
us!for defined always isMean
c.r.vfor )()(
d.r.v.for )()(
∫
∑∞+
∞−
=
=
dtttfXE
kpxXE
X
Xk
k
time
queue size
mean
too much?
27
Variance
Describes how much a r.v. deviates from its average value, i.e., D = X - E(X)We are only interested in the magnitude of the deviation, so we use 22 ))(( XEXD −=
The variance of a r.v. is defined as the mean squared variation )( 2DE
))](([)( 22 XEXEXVar −==σ
Important relation: )()()( 22 XEXEXVar −=
28
Properties of Mean and Variance
Useful when we discuss amplifying random variables or adding constant biases.
)()(
)()(
0)(
)( ;)()(
2 XVarccXVar
XVarcXVar
cVar
bbEbXaEbaXE
=
=+=
=+=+
29
Examples of Random Variables We are interested in those r.v. that permit us
to model system behavior based on the present state alone.
We need to count arrivals in a time interval count the number of times we need to repeat
something to succeed count the number of successes and failures measure the time between consecutive
arrivals The trick is to map our performance questions
into the above four types of experiments
30
Bernoulli Random Variable Let A be an event related to the
outcomes of a random experiment. X = 1 if outcome occurs and 0
otherwise This is the Bernoulli r.v. and has two
possible outcomes: success (1) or failure (0)
qpXPPP
pqXPPP
X
X
−=====−=====
1)1()1(1)0()0(
1
0
We use it as a building block for other types of counting
31
Geometric Random Variable Used to count the number of attempts
needed to succeed doing something. Example: How many times do we have to
transmit a packet over a broadcast radio channel before it is sent w/o interference?
time
Assume that each attempt is independent of any prior attempt!
Assume each attempt has the same probability of success (p)
failure failure failure success!
32
Geometric Random Variable We want to count the number k of trials needed for
the first success in a sequence of Bernoulli trials!
,...2,1 ;)1()()(
trialain success ofy probabilit1 =−===
=− kppkXPkp
pk
X
Why this is the case is a direct consequence of assuming independent Bernoulli trials, each with the same probability of success: k-1 failures needed before the last successful trial
Memoryless property: The probability of having a success in k additional trials having experienced n failures is the same as the probability of success in k trials before the n failed trails
)()|( kXPnXknXP ==>+=
33
Binomial Random Variable X denotes the number of times success
occurs in n independent Bernoulli trials.
success 1 where0); , ... 0, 0, 1, , ... 1, 1, ,1(
such that outcome specific heConsider t
==∈
sSs
knkcn
ckk ppAPAPAPAPAPsP −+ −== )1()()...()()....()()(
:tindependen are trialsBecause
121
s0' and s1' have we; 1 1 n-kk nkk +
qpAPAPpAP
AAAAAs
iA
ci
cn
ckk
i
=−=−==
∩∩∩∩∩∩=
=
+
1)(1)( ;)(
......
then, in trial success Let
121
34
Binomial Random Variableknkc
nckk ppAPAPAPAPAPsP −+ −== )1()()...()()....()()( 121
!)!(
!
k
n and slots, in the s1'for
positions choose towaysdifferent k
n are There
kkn
nn
k
−=⎟⎟
⎠
⎞⎜⎜⎝
⎛
⎟⎟⎠
⎞⎜⎜⎝
⎛
Because each outcome is mutually exclusive of the others:
nk
ppkkn
npp
k
nkP knkknk
n
≤≤
−⎟⎟⎠
⎞⎜⎜⎝
⎛−
=−⎟⎟⎠
⎞⎜⎜⎝
⎛= −−
0
)1(!)!(
!)1()(
35
Poisson Random Variable We use this r.v. in cases where we need to count
event occurrences in a time period. An event occurrence will typically be a packet
arrival. Arrivals are assumed to occur at random over a
time interval. The time interval is (0, t] We divide the time interval into n small subintervals
of length tnt Δ=/
• The probability of a new arrival in a given subinterval is defined to be tΔλ
• is constant, independent of the subinterval.λ
36
Poisson Random Variable
A sequence of n independent Bernoulli trials; with X being the number of arrivals in (0, t]
arrivals. 1or 0 having ofy probabilit the tocompared
negligible is in arrival one than more ofy probabilit The
:0 and Make
t
tn
Δ→Δ∞→
By assumption, whether or not an event occurs in a subinterval is independent of the outcomes in other subintervals.We have:
tΔ
…. time
t0
arrival
1 2 3 4 n
1 2 3 k
37
Poisson Random Variablearrivals ofnumber theis ;)1()( kpp
k
nkP knk
n−−⎟⎟
⎠
⎞⎜⎜⎝
⎛=
nttnttPp /}/in arrival{ λλ =Δ==Δ=kkn
k n
t
n
t
kkn
nPkXPtkP ⎥⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡ −−
====− λλ
1 !)!(
!)(]}[0, in arrivals {
: then,0 and Make →Δ∞→ tn
knk
kk n
t
n
t
k
t
knn
nP
−
⎥⎦
⎤⎢⎣
⎡ −⎥⎦
⎤⎢⎣
⎡ −⎥⎦
⎤⎢⎣
⎡−
=λλλ
11!)(
)!(!
:get toRearrange
[ ] knk
k n
t
k
t
n
knnnn −
⎥⎦
⎤⎢⎣
⎡−⎥⎦
⎤⎢⎣
⎡ +−−−11
!
)()1)..(2)(1( λλ
€
Pk =nk
n
t
k
t⎥⎦
⎤⎢⎣
⎡ −λλ
1!)(
)1(
nxx
ex
x −=⎟⎠
⎞⎜⎝
⎛ +≡ ∞→ with;1
1lim also and
38
Poisson Random Variable
We see that the Poisson r.v. is the result of an approximation of i.i.d. arrivals in a time interval.
We will say “arrivals are Poisson” meaning we can use the above formulas to describe packet arrivals
The probability of 0 arrivals in [0, t] plays a key role in our treatment of interarrival times.
€
Pk = P{k arrivals in (0,t]} =λ t( )
k
k!e−λt
tetPP λ−== ]}(0, in arrivals 0{0
tePtP λ−−=−= 11]}(0, in arrivals some{ 0
39
Important Properties of Poisson Arrivals
The aggregation of Poisson sources is a Poisson source.
If packets from a Poisson source are routed such that a path is chosen independently with probability p, that stream is also Poisson, with rate p times the original rate.
It turns out that the time of a given Poisson arrival is uniformly distributed in a time interval
• The parameter λ is called the arrival rate
40
Exponential Random Variable Consider the Poisson r.v., then teP λ−=0• Let Y be the time interval from a time origin (chosen arbitrarily) to the first arival after that origin• Let ] (0,in arrivals ofNumber )( ττ =X then
λτττ −===> eXPYP )0)(()(• We can obtain now a c.d.f. for Y as follows:
0 ;1)(1)()( ≥−=>−=≤= − ττττ λτeYPYPFY
and the p.d.f. of Y is then
0 ;)()( ≥=≤= − τλττ λτeYPdtd
fY
41
Memoryless Property of Exponential Random Variable
After waiting h sec. for the first arrival, the probability that it occurs after t sec. equals the prob. that the first arrival occurs after t sec.
Knowing that we have waited any amount of time does not improve our knowledge of how much longer we’ll have to wait! )()|( hYPtYhtYP >=>+>
time
time > t?time? time >t ?
42
Mean and Variance Results
You have to memorize these!You should be able to derive any of the above
Exponential: 22 /1 ;/1)( λσλ ==YE
Poisson: tXE λσ == 2)(
Geometric:22 / ;/1)( pqpXE == σ
Binomial: npqnpXE == 2 ;)( σ
43
Random Processes
Definition: A random process is a family of random variables {X(t) | t is in T } defined on a given sample space, and indexed by a parameter t, where t varies over an index set T.
t
xx
x
xx
xx
x
x
xx
1t 2t 3t 4t 5t
a state of X(t)
X(t5) is a R.V.statespace ofrandomprocess
44
Poisson Process We assume that events occur at random instants of time at
an average rate of lambda events per second. Let N(t) be the number of event occurrences (e.g., packet
arrivals) in (0,t]. N(t) is a non-decreasing, integer-valued, continuous time
random process.
t1a 2a 3a
4a 5a
54321
a r.v.a r.v.
4)( 2 =tN4)( 1 =tN
45
Poisson Process As we did for the case of the Poisson r.v., we divide the time axis into very
small subintervals, such that The probability of more than one arrival in a subinterval is much smaller than the
probability of 0 or 1 arrivals in the subinterval. Whether or not an arrival occurs in a subinterval is independent of what takes place in
any other subinterval. We end up with the same Binomial counting process and with the length of
the subintervals going to 0 we can approximate:
( ),...1,0 ,
!]}(0,in arrivals {])([ ==== − ke
k
ttkPktNP t
kλλ
Again, inter arrival times are exponentially distributed with parameter lambda
46
Poisson Process Why do we say that arrivals occur “at random” with a Poisson process? Suppose that we know that only one arrival occurs in (0, t] and x is the
arrival time of the single arrival. Let N(x) be the number of arrivals up to time x, 0 < x ≤ t Then:
N(t) -N(x) = increment of arrivals in (x, t]
}1)({
}0)()( and 1)({
}1)({
}1)()({
}1)(|1)({][
==−=
==
===
===≤
tNPxNtNxNP
tNPtNxNPtNxNPxXP
47
Poisson Process Arrivals in different intervals are independent.
}1)({
}0)()(P{ }1)({][
==−=
=≤tNP
xNtNxNPxXP
t
x
te
exxXP
t
x
==≤ −
−
λ
λλ
λλ ] ][e)[(
][)x-(t-
The above probability corresponds to the uniform distribution!
Hence, in the average case, a packet arrives in the middle of a fixed time interval with Poisson arrivals