ENMA 420/520Statistical ProcessesSpring 2007

Michael F. Cochrane, Ph.D.

Dept. of Engineering Management

Old Dominion University

Class FiveReadings & Problems

Reading assignment

M & S Chapter 6 Sections 6.1 - 6.8

Recommended problems

M & S Chapter 6

Chapter 6: 5, 13, 18, 20, 29, 38, 40

Chapter 6Bivariate Distributions

What do we mean by univariate probability distributions?

What do we mean by multivariate probability distributions?

Special case of multivariate probability distributions:

Bivariate distributions

Bivariate Distributions

Looking for p(2 things happening)

p(XY)Alternative way of writing p(xy)

p(X,Y)What does p(x, y) really mean?

p(X = x0, Y = y0)

)()(),(,

)()(),(

)(

)()(

xpxypyxpor

ypyxpyxp

yp

yxpyxp

Recall

Example Bivariate DistributionToss 2 Dice

1 … 6

1 1/36 1/36

…

6 1/36 1/36

y

x

What is p(1, 1)? ?),( i j

ji yxp ?)1()1,( ypyxp xi

i

P(x,y)

GeneralizingMarginal Probabilities

Generalize from previous example:

dxyxfyypyxpyyp

dyyxfxxpyxpxxp

iiyiiy

iixiix

),()(or ),()(

),()(or ),()(

xall

y all

Discrete rv Continuous rv

How do you interpret?

Over y

Bivariate DistributionsExtending the Univariate Case

dxdyyxfyxgyxgE

yxgE

dxdyyxfdycbxapd

c

b

a

),(),()),((

)y,p(x )y,g(x)),((

),(),(

i jjiji

Note the extensions from the univariate special case!

Bivariate DistributionsWhat if x & y Are Independent?

)()()(

)()(),(

)()(),(

)()(),(),(

),()(

)()()()()(

yExExyE

yFxFyxF

yfxfyxf

ypxpyxpByAxp

ByAxpBAp

BpApBpBApBAp

yx

Do not forget: above only valid if x & y are independent!Note the convenience of assuming independence!

Recall: E(.) more generallya linear operator

Very useful relationships

CovarianceLooking for Relationship Between x & y

Is There a Relationship Between x & y?

0

5

10

15

0 5 10 15

x

y

Does knowingx tells us anythingabout y?

Point (xi, yj)

CovarianceLooking for Relationship Between x & y

Is There a Relationship?

0

5

10

15

20

25

0 5 10 15

x

y

Does knowingx tell you something aboutcorresponding y?

Covariance a measure of strengthof relationship!

CovarianceHow Do x & y Vary?

Is There a Relationship?

0

5

10

15

20

25

0 5 10 15

x

y

Recall VAR(y):VAR(y)=E(y - y)2

= y2

Covariance COV(x,y)= E[(x - x) (y - y)]=xy

2

What would youexpect COV here to be?

CovarianceHow Do x & y Vary?

Is There a Relationship?

0

5

10

15

20

25

0 5 10 15

x

y

x

y

Covariance COV(x,y)= E[(x - x) (y - y)]=xy

2

What is (x - x) (y - y)

for these points?

What can you say about COV for a set of points like these?

What is (x - x) (y - y)for these points?

Covariance of Independent Variables

Recall formula for COV(x,y)

COV(x,y) = E[(x - x) (y - y)]

= E(xy) - xy

Make sure you can derive the above

What happens if x & y are independent?COV(x,y) = 0 Why??

Suppose COV(x,y) = 0, are x and y independent??

Covariance of Independent VariablesA One Way Street

If x & y are independent

Then COV(x,y)=0

If COV(x,y) = 0

Then cannot conclude x & y are independent

CovarianceA Unit Dependent Measure

Suppose

x = height in feet and y = weight in pounds

What are the units of COV(x,y)?

Same observations: x now cm & y now g, what happens to COV(x,y)?

How did we address same issues in univariate case

CV = /

Bivariate counterpart: Coefficient of Correlation

yx

yxCOV

),(

What are the units?What is range of values?

Coefficient of CorrelationPossibilities

Range of possible values: -1 1

Here = 1All observations on a line

But here also = 1What does = 1 imply?What about the value of slope?

What does = -1 mean?How about = 0?

Bivariate RelationshipsLinear Functions of RVs

Recall simple example from MBA 101:

P = S - C

If S & C are both rv P is also a rv Is P a linear function of S & C? What is E(P)? What is VAR(P)? What is the distribution of P?

Easy to tell only for special cases

For example if S ~ N AND C ~ N

Lets generalize for linear functions of rv

Mean & VarianceLinear Functions of RVs

Define l l = a1y1 + … + anyn

ai = constants

yj = random variables l =linear function of n random variables

What is the E( l )?

VAR(l) = VAR (a1y1 + … + anyn)

= a121

2 + … + an2n

2 + 2a1a2COV(y1,y2)

+ 2a1a3COV(y1,y3) + … + 2a1anCOV(y1,yn)

+ … + 2an-1anCOV(yn-1,yn)

Note the pattern

What happens when all yi are independent random variables?

Short ExerciseSimple Example

Consider a nut & bolt

r = bolt radius, a rv

r = 0.5 inches; r = 0.01 inches

w = nut width, a rv

w = 0.51 inches; w = 0.001 inches

What are r & r2 in centimeters?

Nut is placed around bolt

Let gap = g = w - r

What are g & g ?

w

r

Functions of Random VariablesSetting the Stage for Inferential Statistics

Random variables are just a type of variable

w = x + y

What type of variable is w? & What is its distribution? IF:

x=12 and y=3 (i.e., both constants)

x~N(12,4) and y=3 (i.e., x =rv & y=constant)

x~N(12,4) and y~N(3,1) (i.e., x & y both rv)

x~U(0,1) and y~U(0,1)

Will verify using @Risk

@RISKUseful Add-In to Excel

Basic concepts of @RISK software

Add-in to Excel Builds on basic spreadsheet capabilities

Allows definition of random variables within cells Excel only allows constants Results determined using simulation

techniques

Will review simple concepts of

simulation

Spreadsheet Simulation

Build spreadsheetIncorporate rv’s

Define number of iterationsin simulation

Run simulationGather data of interest

Output results

Performing a SimulationDoing 1 Iteration

Sample each rv in spreadsheet model

Perform calculationsin the model

Save output valuesof interest

Do these stepsn times during a single run of a simulation.

After simulationhave n values(observations) for each output valueof interest.What do these noutput values represent?

Illustratebased onpreviouslyquestionsregardingw = x + y

Sampling Issues

Sampling key to inferential statistics

Random sampling from Population (actual members of set) A distribution representing a population

PopulationRandomsample

Set of nobservations

withinsample

RANDOM is key word!

What is thedifference?

Sampling From a Population

Methods to be discussed

Random number table

Excel sampling data analysis tool Statistical analysis sw all have

capability

Population Extract n observations Sample

Random Number TableSimple Approach

Associate random integerswith

members of population

Choose appropriate observations

Generate n random integersin range [1, N]

N - # in populationn - # in sample

Could userandom #table or sw

Using Table 1 of Appendix B(page 897 in M&S)

Problem:

Have population of 100 members want random sample of 5 observations

Approach

Number observations 1 ==> 100

Go to any page in table & select 5 adjacent numbers

Use last 3 digits to designate selected observations

Question

What if there were 130 members in population?

How would you adapt the method in this case?

More Practical Approaches

Systematic sample

Select every kth element of population Useful for very large populations, why? What is implicit assumption of this method?

Software more practical approach

Excel

MiniTab or equivalent statistical analysis sw

What if you want random sample from a rv

Recall rv represents a “population”

Rv described by a probability distribution

Inferential StatisticsLooking for Insight into Population

Population y y

2 1 Sample

of ns

y

Sample

What type of variables are these

How big a sample would you need in order for these to be equal?

Sampling Distribution

)yf(

yf(s)

s

Both of thesedistributionshave a mean anda standard deviation.

Standard error of a statistic is standarddeviation of itssampling distribution.

Do you recall seeingit in Excel output?

Class FiveReadings & Problems

Reading assignment

M & S Chapter 6 Sections 6.1 - 6.8 Chapter 7 Sections 7.1 - 7.2

Recommended problems

M & S Chapter 6 Chapter 6: 5, 13, 18, 20, 29, 38, 40