stochastic systems analysis and simulationssystems analysis introduction 16 modeling i let t be a...
TRANSCRIPT
![Page 1: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/1.jpg)
Stochastic Systems Analysis and Simulations
Alejandro RibeiroDept. of Electrical and Systems Engineering
University of [email protected]
http://www.seas.upenn.edu/users/~aribeiro/
September 7, 2011
Stoch. Systems Analysis Introduction 1
![Page 2: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/2.jpg)
Presentations
Presentations
Class description and contents
Gambling
Stoch. Systems Analysis Introduction 2
![Page 3: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/3.jpg)
Who are us, where to find me, lecture times
I Alejandro Ribeiro
I Assistant Professor, Dept. of Electrical and Systems Engineering
I GRW 276, [email protected],
I http://alliance.seas.upenn.edu/~aribeiro/wiki
I Felicia Lin
I Teaching assistant, [email protected]
I We also have a separate grader
I We meet on Moore 216
I Mondays, Wednesdays, Fridays 10 am to 11 am
I My office hours, Fridays at 2 pm
I Anytime, as long as you have something interesting to tell me
I http://alliance.seas.upenn.edu/~ese303/wiki
Stoch. Systems Analysis Introduction 3
![Page 4: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/4.jpg)
Prerequisites
I Probability theory
I Stochastic processes are time-varying random entities
I If unknown, need to learn as we go
I Will cover in first seven lectures
I Linear algebra
I Vector matrix notation, systems of linear equations, eigenvalues
I Programming in Matlab
I Needed for homework.
I If you know programming you can learn Matlab in one afternoon
I But it has to be this afternoon
I Differential equations, Fourier transforms
I Appear here and there. Should not be a problem
Stoch. Systems Analysis Introduction 4
![Page 5: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/5.jpg)
Homework and grading
I 14 homework sets in 14 weeks
I Collaboration accepted, welcomed, and encouraged
I Sets graded as 0 (bad), 1 (good), 2 (very good) and 3 (outstanding)
I We’ll use the 3 sparingly. Goal is to earn 28 homework points
I Midterm examination starts on Friday October 21 worth 36 points
I In class-piece + take home piece due on Monday October 24
I Work independently. No collaboration, no discussion
I If things are going well, no in-class piece
I Final examination on December 14-21 worth 36 points
I At least 60 points are required for passing.
I C requires at least 70 points. B at least 80. A at least 90
I Goal is for everyone to earn an A
Stoch. Systems Analysis Introduction 5
![Page 6: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/6.jpg)
Textbooks
I Textbook for the class is (older or newer editions acceptable)
I Sheldon M. Ross ”Introduction to Probability Models”,Academic Press, 10th ed.
I Same topics at advanced level (more rigor, includes proofs)
I Sheldon Ross ”Stochastic Processes”, John Wiley & sons, 2nd ed.
I Stohastic processes in systems biology
I Darren J. Wilkinson ”Stochastic Modelling for Systems Biology”,Chapman & Hall/CRC, 1st ed.
I Part on simulation of chemical reactions taken from here
I Use of stochastic processes in finance
I Masaaki Kijima ”Stochastic Processes with Applications toFinance”, Chapman & Hall/CRC, 1st ed.
Stoch. Systems Analysis Introduction 6
![Page 7: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/7.jpg)
Be nice
I I am a very emotional person. I will love most of you, despise a few
I Seriously though, I work hard for this course, please do the same
I Come to class, be on time, pay attention
I Do all of your homework
I Do not hand in as yours my own solution
I Do not collaborate in the take-home midterm
I A little bit of probability ...
I Probability of getting an F in this class is 0.04
I Probability of getting an F given you skip 4 homework sets is 0.7
I I’ll give you three notices, afterwards, I’ll give up on you
I Come and learn. Useful. Very good student ratings
Stoch. Systems Analysis Introduction 7
![Page 8: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/8.jpg)
Class contents
Presentations
Class description and contents
Gambling
Stoch. Systems Analysis Introduction 8
![Page 9: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/9.jpg)
Stochastic systems
I Anything random that evolves in time
I Time can be discrete (0, 1, . . . ) or continuous
I More formally, assign a function to a random event
I Compare with “random variable assigns a value to a random event”
I Generalizes concept of random vector to functions
I Or generalizes the concept of function to random settings
I Can interpret a stochastic process as a set of random variables
I Not always the most appropriate way of thinking
Stoch. Systems Analysis Introduction 9
![Page 10: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/10.jpg)
A voice recognition system
I Random event ∼ word spoken. Stochastic process ∼ the waveformI Try the file speech signals.m
“Hi” “Good”
0 0.02 0.04 0.06 0.08 0.1 0.12 0.14−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
Time [sec]
Ampl
itude
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
Time [sec]
Ampl
itude
“Bye” ‘S”
0 0.05 0.1 0.15 0.2 0.25−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
Time [sec]
Ampl
itude
0 0.02 0.04 0.06 0.08 0.1 0.12 0.14−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
Time [sec]
Ampl
itude
Stoch. Systems Analysis Introduction 10
![Page 11: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/11.jpg)
Five blocks
I Probability theory review (6 lectures)I Probability spacesI Conditional probability: time n + 1 given time n, future given past ...I Limits in probability, almost sure limits: behavior as t →∞ ...I Common probability distributions (binomial, exponential, Poisson, Gaussian)
I Stochastic processes are complicated entities
I Restrict attention to particular classes that are somewhat tractable
I Markov chains (9 lectures)
I Continuous time Markov chains (12 lectures)
I Stationary random processes (9 lectures)
I Midterm covers up to Markov chains
Stoch. Systems Analysis Introduction 11
![Page 12: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/12.jpg)
Markov chains
I A set of states 1, 2, . . .. At time n, state is Xn
I Memoryless property
⇒ Probability of next state Xn+1 depends on current state Xn
⇒ But not on past states Xn−1, Xn−2, . . .
I Can be happy (Xn = 0) or sad (Xn = 1)
I Happiness tomorrow affected byhappiness today only
I Whether happy or sad today, likely tobe happy tomorrow
I But when sad, a little less likely so
H S
0.8
0.2
0.3
0.7
I Classification of states, ergodicity, limiting distributions
I Google’s page rank, machine learning, virus propagation, queues ...
Stoch. Systems Analysis Introduction 12
![Page 13: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/13.jpg)
Continuous time Markov chains
I A set of states 1, 2, . . .. Continuous time index t
I Transition between states can happen at any time
I Future depends on present but is independent of the past
I Probability of changing state inan infinitesimal time dt
H S
0.2dt
0.7dt
I Poisson processes, exponential distributions, transition probabilities,Kolmogorov equations, limit distributions
I Chemical reactions, queues, communication networks, weatherforecasting ...
Stoch. Systems Analysis Introduction 13
![Page 14: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/14.jpg)
Stationary random processes
I Continuous time t, continuous state x(t), not necessarily memoryless
I System has a steady state in a random sense
I Prob. distribution of x(t) constant or becomes constant as t grows
I Brownian motion, white noise, Gaussian processes, autocorrelation,power spectral density.
I Black Scholes model for option pricing, speech, noise in electriccircuits, filtering and equalization ...
Stoch. Systems Analysis Introduction 14
![Page 15: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/15.jpg)
Gambling
Presentations
Class description and contents
Gambling
Stoch. Systems Analysis Introduction 15
![Page 16: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/16.jpg)
An interesting betting game
I There is a certain game in a certain casino in which ...
⇒ your chances of winning are p > 1/2
I You place $b bets
(a) With probability p you gain $b and(b) With probability (1− p) you loose your $b bet
I The catch is that you either
(a) Play until you go broke (loose all your money)(b) Keep playing forever
I You start with an initial wealth of $w0
I Shall you play this game?
Stoch. Systems Analysis Introduction 16
![Page 17: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/17.jpg)
Modeling
I Let t be a time index (number of bets placed)
I Denote as x(t) the outcome of the bet at time tI x(t) = 1 if bet is won (with probability p)I x(t) = 0 if bet is lost (probability (1− p))
I x(t) is called a Bernoulli random varible with parameter p
I Denote as w(t) the player’s wealth at time t
I At time t = 0, w(0) = w0
I At times t > 0 wealth w(t) depends on past wins and losses
I More specifically we haveI When bet is won w(t) = w(t − 1) + bI When bet is lost w(t) = w(t − 1)− b
Stoch. Systems Analysis Introduction 17
![Page 18: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/18.jpg)
Coding
t = 0; w(t) = w0; maxt = 103; // Initialize variables
% repeat while not broke up to time maxtwhile (w(t) > 0) & (t < maxt) do
x(t) = random(’bino’,1,p); % Draw Bernoulli random variableif x(t) == 1 then
w(t + 1) = w(t) + b; % If x = 1 wealth increases by belse
x(t + 1) = w(t)− b; % If x = 0 wealth decreases by bendt = t + 1;
end
I Initial wealth w0 = 20, bet b = 1, win probability p = 0.55
I Shall we play?
Stoch. Systems Analysis Introduction 18
![Page 19: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/19.jpg)
One lucky player
I She didn’t go broke. After t = 1000 bets, her wealth is w(t) = 109
I Less likely to go broke now because wealth increased
0 100 200 300 400 500 600 700 800 900 10000
20
40
60
80
100
120
140
160
180
200
bet index
weal
th (i
n $)
Stoch. Systems Analysis Introduction 19
![Page 20: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/20.jpg)
Two lucky players
I Wealths are w1(t) = 109 and w2(t) = 139
I Increasing wealth seems to be a pattern
0 100 200 300 400 500 600 700 800 900 10000
20
40
60
80
100
120
140
160
180
200
bet index
weal
th (i
n $)
Stoch. Systems Analysis Introduction 20
![Page 21: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/21.jpg)
Ten lucky players
I Wealths wj(t) between 78 and 139
I Increasing wealth is definitely a pattern
0 100 200 300 400 500 600 700 800 900 10000
20
40
60
80
100
120
140
160
180
200
bet index
weal
th (i
n $)
Stoch. Systems Analysis Introduction 21
![Page 22: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/22.jpg)
One unlucky player
I But this does not mean that all players will turn out as winners
I The twelfth player j = 12 goes broke
0 100 200 300 400 500 600 700 800 900 10000
20
40
60
80
100
120
140
160
180
200
bet index
weal
th (i
n $)
Stoch. Systems Analysis Introduction 22
![Page 23: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/23.jpg)
One unlucky player
I But this does not mean that all players will turn out as winners
I The twelfth player j = 12 goes broke
0 50 100 150 200 2500
5
10
15
20
25
30
35
40
bet index
weal
th (i
n $)
Stoch. Systems Analysis Introduction 23
![Page 24: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/24.jpg)
One hundred players
I Only one player (j = 12) goes broke
I All other players end up with substantially more money
0 100 200 300 400 500 600 700 800 900 10000
20
40
60
80
100
120
140
160
180
200
bet index
weal
th (i
n $)
Stoch. Systems Analysis Introduction 24
![Page 25: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/25.jpg)
Average tendency
I It is not difficult to find a line estimating the average of w(t)
I w̄(t) ≈ w0 + (2p − 1)t ≈ w0 + 0.1t
0 100 200 300 400 500 600 700 800 900 10000
20
40
60
80
100
120
140
160
180
200
bet index
weal
th (i
n $)
Stoch. Systems Analysis Introduction 25
![Page 26: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/26.jpg)
Where does the average tendency comes from?
I To discover average tendency w̄(t) assume w(t − 1) > 0 and note
E[w(t)
∣∣w(t − 1)]
= w(t − 1) + bP [x(t) = 1]− bP [x(t) = 0]
= w(t − 1) + bp − b(1− p)
= w(t − 1) + (2p − 1)b
I Now, condition on w(t − 2) and use the above expression once more
E[w(t)
∣∣w(t − 2)]
= E[w(t − 1)
∣∣w(t − 2)]
+ (2p − 1)b
= w(t − 2) + (2p − 1)b + (2p − 1)b
I Proceeding recursively t times, yields
E[w(t)
∣∣w(0)]
= w0 + t(2p − 1)b
I This analysis is not entirely correct because w(t) might be zero
Stoch. Systems Analysis Introduction 26
![Page 27: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/27.jpg)
Analysis of outcomes: mean
I For a more accurate analysis analyze simulation’s outcome
I Consider J experiments
I For each experiment, there is a wealth history wj(t)
I We can estimate the average outcome as
w̄J(t) =1
J
J∑j=1
wj(t)
I w̄J(t) is called the sample average
I Do not confuse w̄J(t) with E [w(t)]I w̄J(t) is computed from experiments, it is a random quantity in itselfI E [w(t)] is a property of the random variable w(t)I We will see later that for large J, w̄J(t)→ E [w(t)]
Stoch. Systems Analysis Introduction 27
![Page 28: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/28.jpg)
Analysis of outcomes: mean
I Expected value E [w(t)] in black (approximation)
I Sample average for J = 10 (blue), J = 20 (red), and J = 100(magenta)
0 50 100 150 200 250 300 350 40020
25
30
35
40
45
50
55
60
65
bet index
weal
th (i
n $)
Stoch. Systems Analysis Introduction 28
![Page 29: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/29.jpg)
Histogram
I There is more information in the simulation’s output
I Estimate the probability distribution function (pdf) ⇒ Histogram
I Consider a set of points w (1), . . . ,w (N)
I Indicator function of the event w (n) ≤ wj < w (n+1)
I I[w (n) ≤ wj < w (n+1)
]= 1 when w (n) ≤ wj < w (n+1)
I I[w (n) ≤ wj < w (n+1)
]= 0 else
I Histogram is then defined as
H[t;w (n),w (n+1)
]=
1
J
J∑j=1
I[w (n) ≤ wj(t) < w (n+1)
]I Fraction of experiments with wealth wj(t) between w (n) and w (n+1)
Stoch. Systems Analysis Introduction 29
![Page 30: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/30.jpg)
Histogram
I The pdf broadens and shifts to the right (t = 10, 50, 100, 200)
0 10 20 30 40 50 60 70 80 900
0.05
0.1
0.15
0.2
0.25
wealth (in $)
frequ
ency
0 10 20 30 40 50 60 70 80 900
0.05
0.1
0.15
0.2
0.25
wealth (in $)
frequ
ency
0 10 20 30 40 50 60 70 80 900
0.05
0.1
0.15
0.2
0.25
wealth (in $)
frequ
ency
0 10 20 30 40 50 60 70 80 900
0.05
0.1
0.15
0.2
0.25
wealth (in $)
frequ
ency
Stoch. Systems Analysis Introduction 30
![Page 31: Stochastic Systems Analysis and SimulationsSystems Analysis Introduction 16 Modeling I Let t be a time index (number of bets placed) I Denote as x(t)the outcome of the bet at time](https://reader033.vdocument.in/reader033/viewer/2022050109/5f46f32a6b9d890a2d2985f9/html5/thumbnails/31.jpg)
What is this class about
I Analysis and simulation of stochastic systems
⇒ A system that evolves in time with some randomness
I They are usually quite complex ⇒ Simulations
I We will learn how to model stochastic systems, e.g.,I x(t) Bernoulli with parameter pI w(t) = w(t − 1) + b when x(t) = 1I w(t) = w(t − 1)− b when x(t) = 0
I ... how to analyze, e.g., E[w(t)
∣∣w(0)]
= w0 + t(2p − 1)b
I ... and how to interpret simulations and experiments, e.g,I Average tendency through sample average
Stoch. Systems Analysis Introduction 31