probability

Probability By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer

Shakeel NoumanM.Phil Statistics

Probability

Using Statistics Basic Definitions: Events, Sample Space,

and Probabilities Basic Rules for Probability Conditional Probability Independence of Events Combinatorial Concepts The Law of Total Probability and Bayes’

Theorem Summary and Review of Terms

Probability2


1 Probability is:

A quantitative measure of uncertainty A measure of the strength of belief in the

occurrence of an uncertain event A measure of the degree of chance or likelihood

of occurrence of an uncertain event Measured by a number between 0 and 1 (or

between 0% and 100%)


Types of Probability

Objective or Classical Probabilitybased on equally-likely eventsbased on long-run relative frequency of eventsnot based on personal beliefs is the same for all observers (objective) examples: toss a coin, throw a die, pick a card


Types of Probability (Continued)

Subjective Probabilitybased on personal beliefs, experiences, prejudices, intuition -

personal judgmentdifferent for all observers (subjective) examples: Super Bowl, elections, new product introduction,

snowfall


2 Basic Definitions

Set - a collection of elements or objects of interestEmpty set (denoted by )

a set containing no elementsUniversal set (denoted by S)

a set containing all possible elementsComplement (Not). The complement of A is

a set containing all elements of S not in A

A


Complement of a Set

A

A

S


Basic Definitions (Continued)

Intersection (And)– a set containing all elements in both

A and BUnion (Or)

– a set containing all elements in A or B or both

A B

A B


A B

Sets: A Intersecting with B

AB

S


Sets: A Union B

A B

AB

S


Basic Definitions (Continued)

•Mutually exclusive or disjoint sets–sets having no elements in common,

having no intersection, whose intersection is the empty set

•Partition–a collection of mutually exclusive sets

which together include all possible elements, whose union is the universal set


Mutually Exclusive or Disjoint

Sets

A B

S

Sets have nothing in common


Sets: Partition

1A

2A

3A

4A

5A

S


Experiment

• Process that leads to one of several possible outcomes *, e.g.:Coin toss

» Heads,TailsThrow die

» 1, 2, 3, 4, 5, 6Pick a card

» AH, KH, QH, ...Introduce a new product

• Each trial of an experiment has a single observed outcome.

• The precise outcome of a random experiment is unknown before a trial.

* , , Also called a basic outcome elementary event or simple event


Events : Definition

Sample Space or Event SetSet of all possible outcomes (universal set) for a

given experiment E.g.: Throw die

• S = {1,2,3,4,5,6} Event

Collection of outcomes having a common characteristic

E.g.: Even number • A = {2,4,6}

– Event A occurs if an outcome in the set A occurs Probability of an event

Sum of the probabilities of the outcomes of which it consists

P(A) = P(2) + P(4) + P(6)


Equally-likely Probabilities

(Hypothetical or Ideal Experiments)

• For example:Throw a die

» Six possible outcomes {1,2,3,4,5,6}» If each is equally-likely, the probability of each is 1/6

= .1667 = 16.67%

» » Probability of each equally-likely outcome is 1 over

the number of possible outcomesEvent A (even number)

» P(A) = P(2) + P(4) + P(6) = 1/6 + 1/6 + 1/6 = 1/2» for e in AP A P e

n A

n S

( ) ( )

( )

( )

3

6

1

2

P en S

( )( )

1


Hearts Diamonds Clubs Spades

A A A AK K K KQ Q Q QJ J J J

10 10 10 109 9 9 98 8 8 87 7 7 76 6 6 65 5 5 54 4 4 43 3 3 32 2 2 2

‘ ’Event Ace Union of ‘ ’Events Heart ‘ ’and Ace

‘ ’Event Heart

The intersection of the ‘ ’ ‘ ’ events Heart and Ace

comprises the single point : circled twice the ace of hearts

P Heart Ace

n Heart Ace

n S

( )

( )

( )

16

52

4

13

P Heartn Heart

n S

( )( )

( )

13

52

1

4

P Acen Ace

n S

( )( )

( )

4

52

1

13

P Heart Acen Heart Ace

n S

( )( )

( )

1

52

Pick a Card: Sample Space


3 Basic Rules for Probability

Range of Values

Complements - Probability of not A

Intersection - Probability of both A and B

Mutually exclusive events (A and C) :

Range of Values

Complements - Probability of not A

Intersection - Probability of both A and B

Mutually exclusive events (A and C) :

0 1 P A( )

P A P A( ) ( ) 1

P A B n A Bn S

( ) ( )( )

P A C( ) 0


Basic Rules for Probability

(Continued)

• Union - Probability of A or B or both (rule of unions)

Mutually exclusive events: If A and B are mutually exclusive, then

• Union - Probability of A or B or both (rule of unions)

Mutually exclusive events: If A and B are mutually exclusive, then

P A B n A Bn S

P A P B P A B( ) ( )( )

( ) ( ) ( )

)()()( 0)( BPAPBAPsoBAP


Sets: P(A Union B)

)( BAP

AB

S


Basic Rules for Probability

(Continued)

• Conditional Probability - Probability of A given B

Independent events:

• Conditional Probability - Probability of A given B

Independent events:

0)( ,)(

)()( BPwhereBP

BAPBAP

P A B P A

P B A P B

( ) ( )

( ) ( )


Rules of conditional probability :Rules of conditional probability

If events A and D are statistically independent:

so

so

P A B P A BP B

( ) ( )( )

P A B P A B P B

P B A P A

( ) ( ) ( )

( ) ( )

P AD P A

P D A P D

( ) ( )

( ) ( )

P A D P A P D( ) ( ) ( )

2-4 Conditional Probability


AT& T IBM Total

Telecommunication 40 10 50

Computers 20 30 50

Total 60 40 100

Counts

AT& T IBM Total

Telecommunication .40 .10 .50

Computers .20 .30 .50

Total .60 .40 1.00

Probabilities

P IBM TP IBM T

P T( )

( )

( )

.

..

10

502

Probability that a project is undertaken by IBM

given it is a telecommunications

project:

Contingency Table - Example 2-2


P A B P A

P B A P B

and

P A B P A P B

( ) ( )

( ) ( )

( ) ( ) ( )

Conditions for the statistical independence of events A and B:

P Ace HeartP Ace Heart

P Heart

P Ace

( )( )

( )

( )

1521352

113

P Heart AceP Heart Ace

P Ace

P Heart

( )( )

( )

( )

1524

52

14

P Ace Heart P Ace P Heart( ) ( ) ( ) 4

521352

152

2-5 Independence of Events


a P T B P T P B

b P T B P T P B P T B

) ( ) ( ) ( )

. * . .

) ( ) ( ) ( ) ( )

. . . .

0 04 0 06 0 0024

0 04 0 06 0 0024 0 0976

Events Television (T) and Billboard (B) are assumed to be independent.

Independence of Events - Example 2-5


The probability of the union of several independent events is 1 minus the product of probabilities of their complements:

P A A A An P A P A P A P An( ) ( ) ( ) ( ) ( )1 2 3

11 2 3

Example 2-7:( ) ( ) ( ) ( ) ( )

. . .

Q Q Q Q P Q P Q P Q P Q1 2 3 10

11 2 3 10

1 9010 1 3487 6513

The probability of the intersection of several independent events is the product of their separate individual probabilities:

P A A A An P A P A P A P An( ) ( ) ( ) ( ) ( )1 2 3 1 2 3

Product Rules for Independent Events


Consider a pair of six-sided dice. There are six possible outcomes from throwing the first die {1,2,3,4,5,6} and six possible outcomes from throwing the second die {1,2,3,4,5,6}. Altogether, there are

6*6=36 possible outcomes from throwing the two dice.

In general, if there are n events and the event i can happen in Ni possible ways, then the number of ways in which the

sequence of n events may occur is N1N2...Nn.

2-6 Combinatorial Concepts

Pick 5 cards from a deck of 52 - with replacement

52*52*52*52*52=525 380,204,032 different possible outcomes

Pick 5 cards from a deck of 52 - without replacement

52*51*50*49*48 = 311,875,200 different possible outcomes


..

. .Order the letters: A, B, and C

A

B

C

B

C

A

B

A

C A

C

B

C

B

A

. ....

.

..

..

. ABC

ACB

BAC

BCA

CAB

CBA

More on Combinatorial Concepts(Tree Diagram)


How many ways can you order the 3 letters A, B, and C?

There are 3 choices for the first letter, 2 for the second, and 1 for the last, so there are 3*2*1 = 6 possible ways to order the three

letters A, B, and C.

How many ways are there to order the 6 letters A, B, C, D, E, and F? (6*5*4*3*2*1 = 720)

Factorial: For any positive integer n, we define n factorial as:n(n-1)(n-2)...(1). We denote n factorial as n!.

The number n! is the number of ways in which n objects can be ordered. By definition 1! = 1 and 0! = 1.

Factorial


Permutations are the possible ordered selections of r objects out of a total of n objects. The number of permutations of n objects

taken r at a time is denoted by nPr, where

What if we chose only 3 out of the 6 letters A, B, C, D, E, and F?There are 6 ways to choose the first letter, 5 ways to choose the

second letter, and 4 ways to choose the third letter (leaving 3letters unchosen). That makes 6*5*4=120 possible orderings or

permutations.

n Prn

n r

For example

P

!( )!

:

!

( )!

!

!

* * * * *

* ** *

6 3

6

6 3

6

3

6 5 4 3 2 1

3 2 16 5 4 120

Permutations (Order is important)


Combinations are the possible selections of r items from a group of n itemsregardless of the order of selection. The number of combinations is denoted

and is read as n choose r. An alternative notation is nCr. We define the numberof combinations of r out of n elements as:

Suppose that when we pick 3 letters out of the 6 letters A, B, C, D, E, and F we chose BCD, or BDC, or CBD, or CDB, or DBC, or DCB. (These are the6 (3!) permutations or orderings of the 3 letters B, C, and D.) But these are

orderings of the same combination of 3 letters. How many combinations of 6different letters, taking 3 at a time, are there?

n

rC

n!

r!(n r)!

n

r

n r

For example

C

:

!

!( )!

!

! !

* * * * *

( * * )( * * )

* *

* *6 3

6

3 6 3

6

3 3

6 5 4 3 2 1

3 2 1 3 2 1

6 5 4

3 2 1

120

620

n

r

Combinations (Order is not Important)


Example: Template for Calculating

Permutations & Combinations


P A P A B P A B( ) ( ) ( )

In terms of conditional probabilities:

More generally (where Bi make up a partition):

P A P A B P A BP A B P B P A B P B

( ) ( ) ( )( ) ( ) ( ) ( )

P A P A Bi

P ABi

P Bi

( ) ( )

( ) ( )

2-7 The Law of Total Probability and Bayes’ Theorem

The law of total probability:


Event U: Stock market will go up in the next yearEvent W: Economy will do well in the next year

P UW

P U W

P W P W

P U P U W P U WP U W P W P U W P W

( ) .

( )

( ) . ( ) . .

( ) ( ) ( )( ) ( ) ( ) ( )

(. )(. ) (. )(. ). . .

75

30

80 1 8 2

75 80 30 2060 06 66

The Law of Total Probability-Example 2-9


Bayes’ Theorem

• Bayes’ theorem enables you, knowing just a little more than the probability of A given B, to find the probability of B given A.

• Based on the definition of conditional probability and the law of total probability.

P B AP A B

P A

P A B

P A B P A B

P AB P B

P AB P B P AB P B

( )( )

( )

( )

( ) ( )

( ) ( )

( ) ( ) ( ) ( )

Applying the law of total probability to the denominator

Applying the definition of conditional probability throughout


Bayes’ Theorem - Example 2-10

• A medical test for a rare disease (affecting 0.1% of the population [ ]) is imperfect:When administered to an ill person, the test will indicate

so with probability 0.92 [ ]» The event is a false negative

When administered to a person who is not ill, the test will erroneously give a positive result (false positive) with probability 0.04 [ ] » The event is a false positive. .

P I( ) .0 001

08.)(92.)( IZPIZP

)( IZ

)( IZ

96.0)(04.0)( IZPIZP


P I

P I

P Z I

P Z I

( ) .

( ) .

( ) .

( ) .

0001

0999

092

004

P I ZP I Z

P Z

P I Z

P I Z P I Z

P Z I P I

P Z I P I P Z I P I

( )( )

( )

( )

( ) ( )

( ) ( )

( ) ( ) ( ) ( )

(. )( . )

(. )( . ) ( . )(. )

.

. .

.

..

92 0001

92 0001 004 999

000092

000092 003996

000092

040880225

Example 2-10 (continued)


P I( ) .0001

P I( ) .0999 P Z I( ) .004

P Z I( ) .096

P Z I( ) .008

P Z I( ) .092 P Z I( ) ( . )( . ) . 0 001 0 92 00092

P Z I( ) ( . )( . ) . 0 001 0 08 00008

P Z I( ) ( . )( . ) . 0 999 0 04 03996

P Z I( ) ( . )( . ) . 0 999 0 96 95904

Prior Probabilities

Conditional Probabilities

JointProbabilities

Example 2-10 (Tree Diagram)


• Given a partition of events B1,B2 ,...,Bn:

P B AP A B

P A

P A B

P A B

P A B P B

P A B P B

i

i i

( )( )

( )

( )

( )

( ) ( )

( ) ( )

1

1

1

1 1

Applying the law of total probability to the denominator

Applying the definition of conditional probability throughout

Bayes’ Theorem Extended


An economist believes that during periods of high economic growth, the U.S. dollar appreciates with probability 0.70; in periods of moderate economic growth, the dollar appreciates with probability 0.40; and during periods of

low economic growth, the dollar appreciates with probability 0.20. During any period of time, the probability of high economic growth is 0.30,

the probability of moderate economic growth is 0.50, and the probability of low economic growth is 0.50.

Suppose the dollar has been appreciating during the present period. What is the probability we are experiencing a period of high economic growth?

Partition:H - High growth P(H) = 0.30

M - Moderate growth P(M) = 0.50L - Low growth P(L) = 0.20

Event A Appreciation

P A HP A MP A L

( ) .( ) .( ) .

0 700 40

0 20

Bayes’ Theorem Extended -Example 2-11


P H AP H A

P AP H A

P H A P M A P L AP A H P H

P A H P H P A M P M P A L P L

( )( )

( )( )

( ) ( ) ( )( ) ( )

( ) ( ) ( ) ( ) ( ) ( )( . )( . )

( . )( . ) ( . )( . ) ( . )( . ).

. . ...

.

0 70 0 300 70 0 30 0 40 050 0 20 0 20

0 210 21 0 20 0 04

0 210 45

0 467

Example 2-11 (continued)


Prior Probabilities

Conditional Probabilities

JointProbabilities

P H( ) . 0 30

P M( ) . 0 50

P L( ) . 0 20

P A H( ) . 0 70

P A H( ) . 0 30

P A M( ) .0 40

P A M( ) . 0 60

P A L( ) . 0 20

P A L( ) . 0 80

P A H( ) ( . )( . ) . 0 30 0 70 0 21

P A H( ) ( . )( . ) . 0 30 0 30 0 09

P A M( ) ( . )( . ) . 0 50 0 40 0 20

P A M( ) ( . )( . ) . 0 50 0 60 0 30

P A L( ) ( . )( . ) . 0 20 0 20 0 04

P A L( ) ( . )( . ) . 0 20 0 80 0 16

Example 2-11 (Tree Diagram)


2-8 Using Computer: Template for Calculating the Probability

of at least one success


M.Phil (Statistics)

GC University, . (Degree awarded by GC University)

M.Sc (Statistics) GC University, . (Degree awarded by GC University)

Statitical Officer(BS-17)(Economics & Marketing Division)

Livestock Production Research Institute Bahadurnagar (Okara), Livestock & Dairy Development

Department, Govt. of Punjab

Name Shakeel NoumanReligion ChristianDomicile Punjab (Lahore)Contact # 0332-4462527. 0321-9898767E.Mail [email protected] [email protected]


mailto:[email protected]

mailto:[email protected]

probability

Education

college university

phil statistics

high economic

low economic

moderate economic

prior probabilitiesconditional

mutually exclusive

mutually exclusive