cs1512 jhunter/teaching/cs1512/lectures/ 1 cs1512 foundations of computing science 2 lecture 23...

www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/

CS1512

CS1512

1

CS1512Foundations of

Computing Science 2

Lecture 23

Probability and statistics (4)

© J R W Hunter, 2006


CS1512

CS1512

2

a priori and Experimental ProbabilitiesRanges and Odds

• In both cases the minimum probability is 0• a priori – event cannot occur

a blue ball is drawn throwing 7 on an ordinary dice

• experimental – event has not occurred

• In both cases the maximum probability is 1• a priori – event must occur

sun will rise tomorrow (knowledge of physics)• experimental – the event always has occurred

the sun has risen every day in my experience

• For two events, express as odds:• odds on a red ball are 3/10 to 7/10 i.e. 3 to 7


CS1512

CS1512

3

Question

I think there’s a better than evens chance that English Jim will win the 2:50 at Exeter today.

How likely is it to snow tomorrow?

What kind of probability are we talking about here?• a priori?• experimental?

Neither• future event so no experimental evidence (and only happens once)• reasoning from first principles – what principles?

Subjective• best guess• most valuable when the guesser is an expert


CS1512

CS1512

4

Probability of two independent events

Experiment

• trial 1: pick a ball – event E1 = the ball is red

• replace the ball in the bag


What is the probability of the event E: E = picking two red balls on successive trials (with replacement).

P( E ) = P( E1 and E2 ) or P( E1∩ E2 )

If the two trials are independent (the outcome of the first trial does not affect the outcome of the second) then:

P( E1 and E2 ) = P( E1 )* P( E2 ) - the multiplication law


CS1512

CS1512

5

Probability of two red balls

R R R W W W W W W WR Y Y Y N N N N N N NR Y Y Y N N N N N N NR Y Y Y N N N N N N NW N N N N N N N N N NW N N N N N N N N N NW N N N N N N N N N NW N N N N N N N N N NW N N N N N N N N N NW N N N N N N N N N NW N N N N N N N N N N

• 9 favourable outcomes out of 100 possibilities• all equally likely• probability of two reds = 9/100 ( = 3/10 * 3/10)


CS1512

CS1512

6

Probability of two non-independent events

Experiment


• do not replace the ball in the bag


What is the probability of the event E:

E = picking two red balls on successive trials (without replacement).

P( E1 and E2 ) = P( E1 )* P( E2 | E1 )

• the general form of the multiplication law


CS1512

CS1512

7

Probability of two red balls

R R W W W W W W WR Y Y N N N N N N NR Y Y N N N N N N NR Y Y N N N N N N NW N N N N N N N N NW N N N N N N N N NW N N N N N N N N NW N N N N N N N N NW N N N N N N N N NW N N N N N N N N NW N N N N N N N N N

firstdraw

second draw – assuming red chosen first time

• 6 favourable outcomes out of 90 possibilities• all equally likely• probability of two reds = 6/90 ( = 3/10 * 2/9)


CS1512

CS1512

8

Conditional probabilities

P( E2 | E1 ) = • the probability that E2 will happen given that E1 has

already happened.

If E2 and E1 are (conditionally, statistically) independent then

P( E2 | E1 ) = P( E2 )

because the occurrence of E1 has no effect on the occurrence of E2

so P( E1 and E2 ) = P( E1 )* P( E2 | E1 ) = P( E1 )* P( E2 )

Be careful about assuming independence.


CS1512

CS1512

9

(Revd. Thomas) Bayes Theorem

P( E1 and E2 ) = P( E1 )* P( E2 | E1 )

order of E1 and E2 is not important (probability of two red balls without replacement)

P( E2 and E1 ) = P( E2 )* P( E1 | E2 )

P( E2 )* P( E1 | E2 ) = P( E1 )* P( E2 | E1 )

P( E1 )* P( E2 | E1 )

P( E1 | E2 ) =

P( E2 )


CS1512

CS1512

10

Bayes Theorem

Why bother?

• Suppose E1 is a particular disease (say measles)

• Suppose E2 is a particular disease (say spots)

• P( E1 ) is the probability of a given patient having measles (before you see them)

• P( E2 ) is the probability of a given patient having spots (for whatever reason)

• P( E2 | E1 ) is the probability that someone who has measles has spots

Bayes theorem allows us to calculate the probability:

P( E1 | E2 ) = P (patient has measles given that the patient has spots)


CS1512

CS1512

11

Probability of two mutually exclusive events

Experiment• bag contains 3 red balls, 7 white balls and 5 blue balls• trial: pick a ball• event E1 = the ball is red• event E2 = the ball is blue

What is the probability of the event E = picking either a red ball or a blue ball

P( E ) = P( E1 or E2 ) or P( E1 ⋃ E2 )

If the two events are mutually exclusive (they can’t both happen in the same trial) then:

P( E1 or E2 ) = P( E1 ) + P( E2 ) - the addition law


CS1512

CS1512

12

Additive probabilities

1 2 3 4 5 6 1 2 3 4 5 6 7

2 3 4 5 6 7 8

3 4 5 6 7 8 9

4 5 6 7 8 9 10

5 6 7 8 9 10 11

6 7 8 9 10 11 12

• 9 outcomes correspond to event E = E1 or E2 • 36 possible outcomes• probability = 9/36 (= ¼ ) = 5/36 + 4/36 = P(8) + P(9)

Probability of throwing 8 (E1) or 9 (E2) with two dice:


CS1512

CS1512

13

Probability of two non-mutually exclusive events

1 2 3 4 5 6

1 1,1 1,2 1,3 1,4 1,5 1,6

2 2,1 2,2 2,3 2,4 2,5 2,6

3 3,1 3,2 3,3 3,4 3,5 3,6

4 4,1 4,2 4,3 4,4 4,5 4,6

5 5,1 5,2 5,3 5,4 5,5 5,6

6 6,1 6,2 6,3 6,4 6,5 6,6

• 11 outcomes correspond to event E E = E1 or E2 or [implicitly] (E1 and E2)

• 36 possible outcomes• probability = 11/36 = 6/36 + 6/36 – 1/36

E = Probability of throwing at least one 5 with two diceE1 = 5 on first dice; E2 = 5 on second dice


CS1512

CS1512

14

Discrete random variables

Variable

• an attribute or property of an element which has a value

Discrete variable

• a variable where the possible values are countable

Discrete random variable

• a discrete variable where the values taken are associated with a probability

Discrete probability distribution

• formed by the possible values and their probabilities

• P(X = <some value>) = ...


CS1512

CS1512

15

Example – fair coin

Let variable (X) be number of heads in one trial (toss).

Values are 0 and 1

Probability distribution:

Value of X ( i ) 0 1

P(X = i) 0.5 0.5

0

0.1

0.2

0.3

0.4

0.5

0.6

0 1


CS1512

CS1512

16

Example – total of two dice

Let variable (X) be total when two dice are tossed.Values are 2, 3, 4, ... 11, 12


P(X = 2) = 1/36 P(X = 3) = 2/36

...

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

2 3 4 5 6 7 8 9 10 11 12

1 2 3 4 5 6

1 2 3 4 5 6 7

2 3 4 5 6 7 8

3 4 5 6 7 8 9

4 5 6 7 8 9 10

5 6 7 8 9 10 11

6 7 8 9 10 11 12


CS1512

CS1512

17

Bernouilli probability distribution

Bernoulli trial:

• one in which there are only two possible and exclusive outcomes;

• call the outcomes ‘success’ and ‘failure’;

• consider the variable X which is the number of successes in one trial


Value of X ( i ) 0 1

P(X = i) (1-p) p

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0 1


CS1512

CS1512

18

Binomial distribution

Carry out n Bernoulli trials; in each trial the probability of success is p

X is the number of successes in n trials?

Possible values are: 0, 1, 2, ... i, ... n-1, n

We state (without proving) that the distribution of X is:

P(X=r) = nCr pr (1-p)n-r

n!nCr = n! = n (n-1) (n-2) ... 2 1 r! (n-r)!


CS1512

CS1512

19

Example of binomial distribution

0.00E+00

2.00E-02

4.00E-02

6.00E-02

8.00E-02

1.00E-01

1.20E-01

1.40E-01

1.60E-01

1.80E-01

2.00E-01

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Probability of r heads in n trials (n = 20)

P(X = r)


CS1512

CS1512

20

Example of binomial distribution

0.00E+00

1.00E-02

2.00E-02

3.00E-02

4.00E-02

5.00E-02

6.00E-02

7.00E-02

8.00E-02

9.00E-02


P(X = r)


CS1512

CS1512

21

Probability of being in a given range


P(0 ≥ X < 5)P(5 ≥ X < 10)...

Moving towards the notionof a continuous distribution

0.00E+00

5.00E-02

1.00E-01

1.50E-01

2.00E-01

2.50E-01

3.00E-01

3.50E-01

4.00E-01


CS1512

CS1512

22

Probability of being in a given range

P(55 ≥ X < 65) = P(55 ≥ X < 60) + P(60 ≥ X < 65) (mutually exclusive events)

Add the heights of the columns

0.00E+00

5.00E-02

1.00E-01

1.50E-01

2.00E-01

2.50E-01

3.00E-01

3.50E-01

4.00E-01

cs1512 jhunter/teaching/cs1512/lectures/ 1 cs1512 foundations of computing science 2 lecture 23...

Documents

occurrence of e

order of e

spots p e

replacement p e

reason p e

ball event e

red event e

blue ball p e