the basic practice of statistics chapter 6: introducing probability

The Basic Practice of StatisticsChapter 6:

Introducing Probability

A random phenomenon is a situation in which we know what outcomes could happen, but we don’t know which particular outcome did or will happen.

The probability of an event is its long-run relative frequency.While we may not be able to predict a particular individual outcome, we can talk about what will happen in the long run.

For any random phenomenon, each attempt, or trial, generates an outcome.

Something happens on each trial, and we call whatever happens the outcome.These outcomes are individual possibilities, like the number we see on top when we roll a die.

This is the sample space of event A. Event A is rolling a total of five on a pair of dice. Always express sample space in brackets.

An event is any set or collection of outcomes.

Sample Spaces• Collection of all possible outcomes

– e.g.: All six faces of a die:

– e.g.: All 52 cards in a deck:

All the possible outcomes of rolling two dice

Events• Simple event

– Outcome from a sample space with one characteristic

– e.g.: A red card from a deck of cards

• Joint event– Involves two outcomes simultaneously– e.g.: An ace that is also red from a deck of cards

Simple EventsThe Event of a Triangle

There are 55 triangles in this collection of 18 objects

The event of a triangle AND blue in color

Joint Events

Two triangles that are blue

Visualizing Events

• Contingency tables

• Tree diagrams

Red 2 24 26

Black 2 24 26

Total 4 48 52

Ace Not Ace Total

Full Deck of Cards

Red CardsBlack Cards

Not an AceAce

Ace

Not an Ace

Contingency TableA Deck of 52 Cards

Ace Not anAce

Total

Red

Black

Total

2 24

2 24

26

26

4 48 52

Sample Space

Red Ace

P(A1 and B2) P(A1)

TotalEvent

Joint Probability Using Contingency Table

P(A2 and B1)

P(A1 and B1)

Event

Total 1

Joint Probability Marginal (Simple) Probability

A1

A2

B1 B2

P(B1) P(B2)

P(A2 and B2) P(A2)

P(A1)

P(B2)

P(A1 and B1)

Compound Probability (Addition Rule)

P(A1 or B1 ) = P(A1) + P(B1) - P(A1 and B1)

P(A1 and B2)

TotalEvent

P(A2 and B1)

Event

Total 1

A1

A2

B1 B2

P(B1)

P(A2 and B2) P(A2)

For Mutually Exclusive Events: P(A or B) = P(A) + P(B)

Conditional Probability Using Contingency Table

BlackColor

Type Red Total

Ace 2 2 4Non-Ace 24 24 48Total 26 26 52

Revised Sample Space

(Ace and Red) 2 / 52 2(Ace | Red) (Red) 26 / 52 26

PPP

Full Deck of Cards

Tree Diagram

Event Possibilities

Red Cards

Black Cards

Ace

Not an Ace

Ace

Not an Ace

Tree DiagramsA tree diagram helps us think through conditional probabilities by showing sequences of events as paths that look like branches of a tree.Making a tree diagram for situations with conditional probabilities is consistent with our “make a picture” mantra.

Probability Trees• A method for solving probability problems

– Given probabilities for some events (perhaps union, intersection, or conditional)

• Find probabilities for other events– Record the basic information on the tree

• Usually three probability numbers are given– Perhaps two probability numbers if events are independent

• The tree helps guide your calculations– Each column of circled probabilities adds up to 1– Circled prob times conditional prob gives next probability– For each group of branches

• Conditional probabilities add up to 1• Circled probabilities at end add up to probability at start

Probability Tree (continued)• Shows probabilities and conditional

probabilitiesP(A and B)

P(A and “not B”)

P(“not A” and B)

P(“not A” and “not B”)

P(A)

P(not A)Yes

Yes

No

P(“not B” given A)

P(B given A)

P(B given “not A”)

P(“not B” given “not A”)

No

Event B

Yes

No

Event A

Example: Appliance Purchases• First, record the basic information

• Prob(Washer) = 0.20, Prob(Dryer) = 0.25• Prob(Washer and Dryer) = 0.15

0.15

0.20

Yes

Yes

Yes

No

No

No

Dryer?Washer?

These add

up to P(Dryer)

= 0.25

Example (continued)

• Next, subtract: 1–0.20 = 0.80, 0.25–0.15 = 0.10

0.15

0.10

0.20

0.80

Yes

Yes

Yes

No

No

No

Dryer?Washer?

These add

up to P(Dryer)

= 0.25

Example (continued)

• Now subtract: 0.20–0.15 = 0.05, 0.80–0.10 = 0.70

0.15

0.05

0.10

0.70

0.20

0.80

Yes

Yes

Yes

No

No

No

Dryer?Washer?

Example (completed tree)• Now divide to find conditional probabilities

0.15/0.20 = 0.75, 0.05/0.20 = 0.25 0.10/0.80 = 0.125, 0.70/0.80 = 0.875

0.15

0.05

0.10

0.70

0.20

0.80

Yes

Yes

Yes

No

0.25

No

0.75

0.125

0.875

No

Dryer?Washer?

Example (finding probabilities)• Finding probabilities from the completed tree

P(Washer) = 0.20

P(Dryer) = 0.15+0.10 = 0.25

P(Washer and Dryer) = 0.15

P(Washer or Dryer) = 0.15+0.05+0.10 = 0.30

P(Washer and not Dryer) = 0.05

P(Dryer given Washer) = 0.75

P(Dryer given not Washer) = 0.125

P(Washer given Dryer) = 0.15/0.25 = 0.60(using the conditional probability formula)

0.15

0.05

0.10

0.70

0.20

0.80

Yes

Yes

Yes

No

0.25

No

0.75

0.125

0.875

No

Dryer?Washer?

Tree Diagrams Figure is a nice example of a tree diagram and shows how we multiply the probabilities of the branches together:

Probability tree

• What is the probability that a car has a CD player, given that it has AC ?

i.e., we want to find P(CD | AC)

Conditional Probability Example

• Of the cars on a used car lot, 70% have air conditioning (AC) and 40% have a CD player (CD). 20% of the cars have both.

Using Decision Trees

Has CD

Does not have CD

Has AC

Does not have AC

Has AC

Does not have AC

P(CD)= 0.4

P(CD’)= 0.6

P(CD and AC) = 0.2

P(CD and AC’) = 0.2

P(CD’ and AC’) = 0.1

P(CD’ and AC) = 0.5

4.2.

6.5.

6.1.

AllCars

4.2.

Given CD or no CD:

(continued)

Using Decision Trees

Has AC

Does not have AC

Has CD

Does not have CD

Has CD

Does not have CD

P(AC)= 0.7

P(AC’)= 0.3

P(AC and CD) = 0.2

P(AC and CD’) = 0.5

P(AC’ and CD’) = 0.1

P(AC’ and CD) = 0.2

7.5.

3.2.

3.1.

AllCars

7.2.

Given AC or no AC:

Probability trees

Conditional probabilities can get complex and it is often a good strategy to build a probability tree that represents all possible outcomes graphically and assigns conditional probabilities to subsets of events.

0.47Internet user

Tree diagram for chat room habits for three adult age groups.

P(chatting) = 0.136 + 0.099 + 0.017 = 0.252

About 25% of all adult Internet users visit chat rooms.

Breast cancer screening

Cancer

No cancer

Mammography

Positive

Negative

Positive

Negative

Disease incidence

Diagnosis sensitivity

Diagnosis specificity

False negative

False positive

0.0004

0.9996

0.8

0.20.1

0.9Incidence of breast cancer among women ages 20–30 Mammography

performance

If a woman in her 20s gets screened for breast cancer and receives a positive test result, what is the probability that she has breast cancer?

She could either have a positive test and have breast cancer, or have a positive test but not have cancer (false positive).

Cancer

No cancer

Mammography

Positive

Negative

Positive

Negative

Disease incidence

Diagnosis sensitivity

Diagnosis specificity

False negative

False positive

( )( | )( ) ( )

0.0004*0.8 0.3%0.0004*0.8 0.9996*0.1

P cancer and posP cancer posP cancer and pos P nocancer and pos

0.0004

0.9996

0.8

0.2

0.1

0.9Incidence of breast cancer among women ages 20–30 Mammography

performance

Possible outcomes given the positive diagnosis: positive test and breast cancer,

or positive test but not cancer (false positive).

This value is called the positive predictive value, or PV+. It is an important piece

of information but, unfortunately, is rarely communicated to patients.

EventsWhen outcomes are equally likely, probabilities for events are easy to find just by counting.When the k possible outcomes are equally likely, each has a probability of 1/k.For any event A that is made up of equally likely outcomes

count of outcomes inPcount of all possible outcomes

AA

(There are 2 ways to get one 6 and the other 4)e.g. P( ) = 2/36

Computing Probabilities

• The probability of an event E:

• Each of the outcomes in the sample space is equally likely to occur

number of event outcomes( )total number of possible outcomes in the sample space

P E

XT

We can see this in the following example. If we flip a coin 500 times and it lands on heads 248 times, then the empirical probability is given by:

248( ) 0.5500

P heads

# of times event occurred( )total # of times experiment performed

EP E

Remember

The probability of drawing a king is given by:

# of ways can occur( )total # of possible outcomes

EP E

Remember

# of kings in a deck 4 1( )total # of cards in a deck 52 13

P king

The probability of tossing a die and rolling a 7 is given by:

The probability of tossing a die and rolling a number less than 7 is given by:

# faces with 7 0(7) 0total # of faces 6

P

# faces with less than 7 6(less than 7) 1total # of faces 6

P

The probability of drawing a heart is given by:


EP E

Remember

# of hearts in a deck 13 1( )total # of cards in a deck 52 4

P heart

The probability of drawing a king is given by:


EP E

Remember

# of kings in a deck 4 1( )total # of cards in a deck 52 13

P king

Formal ProbabilityTwo requirements for a probability:

A probability is a number between 0 and 1.

For any event A, 0 ≤ P(A) ≤ 1.

Probability• Probability is the numerical

measure of the likelihood that an event will occur

• Value is between 0 and 1• Sum of the probabilities of

all mutually exclusive and collectively exhaustive events is 1

Certain

Impossible

.5

1

0

Formal Probability “Something has to happen rule”:

The probability of the set of all possible outcomes of a trial must be 1.

P(S) = 1 (S represents the set of all possible outcomes.)

In this example, the trials are independent only when you put

the coin back (“sampling with replacement”) each time.

Two events are independent if the probability that one event occurs

on any given trial of an experiment is not influenced in any way by

the occurrence of the other event.

Imagine coins spread out so that half were heads up, and half were tails up. Pick

a coin at random. The probability that is was head-up is 0.5. But, if you don’t put

it back, the probability of picking up another head-up coin is now less than 0.5.

Without replacement, successive trials are not independent.

Independent versus disjoint events

Two events are disjoint if they

have no outcomes in common

and can never happen together.

If you flip a coin once, it may turn out head or tail. However, you cannot

obtain both head and tail on the same flip. Head and tail are disjoint events.

If a couple gets a child, the child could be a boy or a girl. The child cannot

be both boy and girl. Boy and girl are disjoint events.

Events A and B are disjoint.

Events A and B are NOT disjoint.

General addition rule General addition rule for any two events A and B:

The probability that A occurs,

or B occurs, or both events occur is:

P(A or B) = P(A) + P(B) – P(A and B)

What is the probability of randomly drawing either an ace or a heart from a pack of

52 playing cards? There are 4 aces in the pack and 13 hearts. However, one card

is both an ace and a heart. Thus:

P(ace or heart) = P(ace) + P(heart) – P(ace and heart)

= 4/52 + 13/52 - 1/52 = 16/52 ≈ 0.3

Mutually Exclusive Events• Two events are Mutually Exclusive if they cannot both

happen, that is, ifProb(A and B) = 0

• No overlapin Venn diagram

• Examples– Profit and Loss (for a selected business division)– Green and Purple (for a manufactured product)– Country Squire and Urban Poor (marketing segments)

• Mutually exclusive events are dependent events

A B

Formal ProbabilityComplement Rule:

The set of outcomes that are not in the event A is called the complement of A, denoted AC.

The probability of an event occurring is 1 minus the probability that it doesn’t occur: P(A) = 1 – P(AC)

Since any event will either occur or it will not occur, by rule 4 previously discussed, we get:

RememberRule 4: the sum of the probabilities of all possible outcomes of an experiment is 1.

( ) ( ) 1P E P not E

can also be stated as:

So the probability of tossing a die and not rolling a 4 is:

( ) ( ) 1P E P not E

1 5( 4) 1 (4) 16 6

P not P

( ) 1 ( )P not E P E

Special Events

• Impossible evente.g.: Club & diamond on one card

draw• Complement of event

– For event A, all events not in A– Denoted as A’– e.g.: A: queen of diamonds

A’: all cards in a deck that are not queen of diamonds

Null Event

Formal Probability Addition Rule:

Events that have no outcomes in common (and, thus, cannot occur together) are called disjoint (or mutually exclusive).

Special Events• Mutually exclusive events

– Two events cannot occur together– e.g. -- A: queen of diamonds; B: queen of clubs

• Events A and B are mutually exclusive• Collectively exhaustive events

– One of the events must occur– The set of events covers the whole sample space– e.g. -- A: all the aces; B: all the black cards; C: all the diamonds;

D: all the hearts• Events A, B, C and D are collectively exhaustive• Events B, C and D are also collectively exhaustive

Formal Probability

Addition Rule:– For two disjoint events A and B, the

probability that one or the other occurs is the sum of the probabilities of the two events.

– P(A or B) = P(A) + P(B), provided that A and B are disjoint.

So, out of the six numbers that can show up on top, we have four ways that we can roll either a 5 or an even number. The probability is given by:

1 3 4 2(5 or even)6 6 6 3

P

Probability of rolling a 5 Probability of rolling an even number

Example: Venn Diagram• Venn diagram probabilities correspond to

right-hand endpoints of probability tree

Washer Dryer

0.050.15

0.10

0.70

P(Washer and Dryer)

P(Washer and “not Dryer”)

P(“not Washer” and Dryer)

P(“not Washer” and “not Dryer”)

Notice, if we want the probability of rolling a 5 or rolling a number greater than 3. There are three numbers greater than 3 on a die and one of them is the 5. We cannot count the 5 twice. The probability is given by:

Probability of rolling a 5

Probability of rolling a number greater than 3

1 3 1 3 1(5 or greater than 3)6 6 6 6 2

P

Probability of rolling the same 5

Stated mathematically the rule is given by:

Thus, the probability of drawing a 3 or a club from a standard deck of cards is:

4 13 1 16 4(3 or club)52 52 52 52 13

P

( or ) ( ) ( ) ( and )P A B P A P B P A B

Cards with a 3

Cards with clubs Card that is a 3 and a club

Computing Compound Probability

• Probability of a compound event, A or B:( or ) ( )number of outcomes from either A or B or both

total number of outcomes in sample space

P A B P A B

E.g. (Red Card or Ace)4 Aces + 26 Red Cards - 2 Red Aces

52 total number of cards28 7 52 13

P

In order to calculate the probability that we roll a Five given that we roll a pair of dice. You must add up all of the possible combinations of that occurring do not forget ORDER!!!!! Red die versus green die

What Can Go Wrong?Beware of probabilities that don’t add up to 1.

To be a legitimate probability distribution, the sum of the probabilities for all possible outcomes must total 1.

Don’t add probabilities of events if they’re not disjoint.Events must be disjoint to use the Addition Rule.

Sometimes we are interested in a combination of outcomes (e.g., a die is rolled and comes up even).

A combination of outcomes is called an event.When thinking about what happens with combinations of outcomes, things are simplified if the individual trials are independent.

Roughly speaking, this means that the outcome of one trial doesn’t influence or change the outcome of another.For example, coin flips are independent.

General multiplication rule • The probability that any two events, A and B, both occur is:

P(A and B) = P(A)P(B|A)

This is the general multiplication rule.

• If A and B are independent then P(A and B) = P(A)P(B)

(A and B are independent when they have no influence on each other’s occurrence.) What is the probability of randomly drawing either an ace or heart from a pack of

52 playing cards? There are four aces in the pack and thirteen hearts.

P(heart|ace) = 1/4 P(ace) = 4/52

P(ace and heart) = P(ace)* P(heart|ace) = (4/52)*(1/4) = 1/52

Notice that heart and ace are independent events.

Computing Joint ProbabilityThe probability of a joint event, A and B:

( and ) = ( )number of outcomes from both A and B

total number of possible outcomes in sample space

P A B P A B

E.g. (Red Card and Ace)2 Red Aces 1

52 Total Number of Cards 26

P

If the set of crayons consists only of red, yellow, and blue, the probability of picking red is . The probability of tossing a die and rolling a 5 is . Butthe probability of picking red and rolling a 5 is given by:

13 1

6

(red and 5) (red) (5)P P P 1 1 13 6 18

This can be illustrated using a “tree” diagram.

Since there are three choices for the color and six choices for the die, there are eighteen different results. Out of these, only one gives a combination of red and 5. Therefore, the probability of picking a red crayon and rolling a 5 is given by:

(red and 5) (red) (5)P P P 1 1 13 6 18

The multiplication rule for independent events can be stated as:

This rule can be extended for more than two independent events:

( and ) ( ) ( )P A B P A P B

( and and , .) ( ) ( ) ( ), .P A B C etc P A P B P C etc

The Law of Large Numbers The Law of Large Numbers (LLN) says that the long-run relative frequency of repeated independent events gets closer and closer to the true relative frequency as the number of trials increases.

For example, consider flipping a fair coin many, many times. The overall percentage of heads should settle down to about 50% as the number of outcomes increases.

You can flip a coin ten times and have heads come up seven times, but this does not mean that the probability is 0.7. The more times a coin is flipped, the more certainty we have to determine the probability of coming up heads.

Thanks to the LLN, we know that relative frequencies settle down in the long run, so we can officially give the name probability to that value.Probabilities must be between 0 and 1, inclusive.

A probability of 0 indicates impossibility.A probability of 1 indicates certainty.

Formal Probability Multiplication Rule:

For two independent events A and B, the probability that both A and B occur is the product of the probabilities of the two events.

P(A and B) = P(A) x P(B), provided that A and B are independent.

Formal Probability Multiplication Rule:

Two independent events A and B are not disjoint, provided the two events have probabilities greater than zero:

Formal Probability - NotationNotation alert: We use the notation P(A or B) and P(A and B). Or you might see the following:

P(A B) instead of P(A or B) P(A B) instead of P(A and B)

What Can Go Wrong? (cont.)Don’t multiply probabilities of events if they’re not independent.

The multiplication of probabilities of events that are not independent is one of the most common errors people make in dealing with probabilities.

Don’t confuse disjoint and independent—disjoint events can’t be independent.

Multiplication Rule for Dependent Events

Dependent events are events that are not independent. The occurrence of one event affects the probability of the occurrence of other events. An example of dependent events is picking a card from a standard deck then picking another card from the remaining cards in the deck.

For instance, what is the probability of picking two kings from a standard deck of cards? The probability of the first card being a king is . However, the probability of the second card depends on whether or not the the first card was a king.

4 152 13

If the first card was a king then the probability of the second card being a king is .

If the first card was not a king, the probability of the second card being a king is .

Therefore, the selection of the first card affects the probability of the second card.

3 151 17

451

When we are looking at probability for two dependent events we need to have notation to express the probability for an event to occur given that another event has already occurred.

If A and B are the two events, we can express the probability that B will occur if A has already occurred by using the notation:

This notation is generally read as “the probability of B, given A.

( | )P B A

The multiplication rule can now be expanded to include dependent events. The rule now reads:

Of course, if A and B are independent, then:

( and ) ( ) ( | )P A B P A P B A

( | ) ( )P B A P B

As an example, in a group of 25 people 16 of them are married and 9 are single. What is the probability that if two people are randomly selected from the group, they are both married?

If A represents the first person chosen is married and B represents the second person chosen is married then:

Here, is now the event of picking another married person from the remaining 15 married persons. The probability for the selection made in B is affected by the selection in A.

16 15 2( and )25 24 5

P A B

( | )P B A

( and )( | )( )

P A BP A BP B

E.g. (Red Card given that it is an Ace)2 Red Aces 1

4 Aces 2

P

Conditional probability

Conditional probabilities reflect how the probability of an event can change if we know that

some other event has occurred/is occurring.

– Example: The probability that a cloudy day will result in rain is different if you live in Los Angeles than if you live in Seattle.

– Our brains effortlessly calculate conditional probabilities, updating our “degree of belief” with each new piece of evidence.

The conditional probability of event B given event A is:

(provided that P(A) ≠ 0) )(

)()|(AP

BandAPABP

If A and B are independent, P(B | A) P(B).

Conditional Probability (continued)• Example: appliance store purchases

• Prob(Washer) = 0.20• Prob(Dryer) = 0.25• Prob(Washer and Dryer) = 0.15

– Conditional probability of buying a Dryer given that they bought a Washer• Prob(Dryer given Washer) • = Prob(Washer and Dryer)/Prob(Washer) = 0.15/0.20 = 0.75• 75% of those buying a washer also bought a dryer

– Conditional probability of Washer given Dryer• = Prob(Washer and Dryer)/Prob(Dryer) = 0.15/0.25 = 0.60• 60% of those buying a dryer also bought a washer

Watch the denominator!

Independent Events• Two events are Independent if information about one

does not change the likelihood of the other– Three equivalent ways to check independence

• Prob (A given B) = Prob (A)• Prob (B given A) = Prob (B)• Prob (A and B) = Prob (A) Prob (B)

• Two events are Dependent if not independent– e.g., Prob(Washer and Dryer) = 0.15

• Prob (Washer) Prob (Dryer) = 0.20 0.25 = 0.05– Washer and Dryer are not independent

• They are dependent

If independent,

all three are true.

Use any one to check.

Not equal

Permutations A permutation is an arrangement of objects where order is important. For instance the digits 1,2, and 3 can be arranged in six different orders --- 123, 132, 213, 231, 312, and 321. Hence, there are six permutations of the three digits. In fact there are six permutations of any three objects when all three objects are used.

In general the number of permutations can be derived from the Multiplication Principal. For three objects, there are three choices for selecting the first object. Then there are two choices for selecting the second object, and finally there is only one choice for the final object. This gives the number of permutations for three objects as 3 2 1=6.

Now suppose that we have 10 objects and wish to make arrangements by selecting only 3 of those objects. For the first object we have 10 choices. For the second we have 9 choices, and for the third we have 8 choices. So the number of permutations when using 3 objects out of a group of 10 objects is 10 9 8=720.

We can use this example to help derive the formula for computing the number of permutations of r objects chosen from n distinct objects r n. The notation for these permutations is and the formula is:

( , )P n r

( , ) ( 1) ( 2) ... [ ( 1)]P n r n n n n r

We often use factorial notation to rewrite this formula. Recall that:

And

Using this notation we can rewrite the Permutation Formula for as

! ( 1) ( 2) ( 3) ... 3 2 1n n n n n

0! 1

( , )P n r

!( , )( )!

nP n rn r

It is important to remember that in using this formula to determine the number of permutations:

1. The n objects must be distinct2. That once an object is used it

cannot be repeated 3. That the order of objects is

important.

Combinations A combination is an arrangement of objects in which order is not important. We arrange r objects from among n distinct objects where r n. We use the notation C(n, r) to represent this combination. The formula for C(n, r) is given by: !( , )

( )! !nC n r

n r r

The Combination Formula is derived from the Permutation Formula in that for a permutation every different order of the objects is counted even when the same objects are involved. This means that for r objects, there will be r! different order arrangements.

Permutation Combination

So in order to get the number of different combinations, we must divide the number of permutations by r!. The result is the value we get for C(n, r) in the previous formula.

!( , )( )!

nP n rn r

!( , )( )! !

nC n rn r r

Permutations of Repeated Objects

It is possible that in a group of objects some of the objects may be the same. In taking the permutation of this group of objects, different orders of the objects that are the same will not be different from one another.

In other words if we look at the group of letters in the word ADD and use D1 to represent the first D, and D2 to represent the second, we can then write the different permutations as AD1D2, AD2D1,

D1AD2, D2AD1, D1D2A, and D2D1A.

But if we substitute the Ds back for the D1 and D2, then AD1D2 and AD2D1 both appear as ADD, and the six permutations become only three distinct permutations.

Therefore we will need to divide the number of permutations by 2 to get the number of distinct permutations.

In permutations of larger groups of objects, the division becomes a little more complicated.

To explain the process, let us look at the word WALLAWALLA. This word has 4 A’s, 4 L’s, and 2 W’s.

Consider that there are 10 locations for each of these letters. These 10 locations will be filled with 4 A’s, and since the A’s are all the same, the order in which we place the A’s will not matter. So if we are filling 10 locations with 4A’s the number of ways we can do this is C(10, 4).

!( , )( )! !

nC n rn r r

Remember

Once these 4 locations have been filled, there remain 6 locations to fill with the 4 L’s. These can be filled in C(6,4) ways, and the last 2 locations are filled with the W’s in C(2,2) ways.

Finally, we multiply these together to get10! 6! 2!(10,4) (6,4) (2,2)4! 6! 4! 2! 2! 0!

C C C 10!

4! 4! 2! 1

10!

4! 4! 2!

This leads to the general formula for permutations involving n objects with n1 of one kind, n2 of a second kind, …and nk of a kth kind.

The number of permutations in this case is:

where n=n1+n2+…nk.

1 2

!! ! ... !k

nn n n

Counting other choices sometimes requires a bit more reasoning to determine how many possibilities there are.

Suppose there are three cards that are each marked with a different letter, A, B, or C. If the cards are face down, and a person can pick one, two or all three of the cards, what is the possibility that the person will pick up the card with the letter A on it?

?

In this case there are three ways that one card can be picked. Out of these there is only one possibility of picking the A.

First way

Second way

Third way A is picked!

There are three ways of picking two cards. Out of these three pairs, there are two that will include the A.

First way

Second way

Third way A is picked!

A is picked!

There is only one way to pick all three cards, and of course, if all three cards are picked, the A will always be included.

A is picked!

So there are a total of seven ways the cards can be picked if the person can pick one, two, or all three cards. Of these choices, four of them will include the A, so the probability that the A will be picked is:

47 Total # of ways to pick the three cards

Possibilities of picking the A card

When we want the probability of an event from a conditional distribution, we write P(B|A) and pronounce it “the probability of B given A has already occurred.”A probability that takes into account a given condition is called a conditional probability.

It Depends… To find the probability of the event B given the event A, we restrict our attention to the outcomes in A. We then find the fraction of those outcomes B that also occurred.

Note: P(A) cannot equal 0, since we know that A has occurred.

(A and B)( | )( )

PPP

B AA

The General Multiplication RuleWhen two events A and B are independent, we can use the multiplication rule for independent events:

P(A and B) = P(A) x P(B)However, when our events are not independent, this earlier multiplication rule does not work. Thus, we need the General Multiplication Rule.

The General Multiplication Rule (cont.)We encountered the general multiplication rule in the form of conditional probability. Rearranging the equation in the definition for conditional probability, we get the General Multiplication Rule:

For any two events A and B,

P(A and B) = P(A) x P(B|A) or

P(A and B) = P(B) x P(A|B)

IndependenceIndependence of two events means that the outcome of one event does not influence the probability of the other.With our new notation for conditional probabilities, we can now formalize this definition:

Events A and B are independent whenever P(B|A) = P(B). (Equivalently, events A and B are independent whenever P(A|B) = P(A).)

Independent ≠ DisjointDisjoint events cannot be independent! Well, why not?

Since we know that disjoint events have no outcomes in common, knowing that one occurred means the other didn’t. Thus, the probability of the second occurring changed based on our knowledge that the first occurred. It follows, then, that the two events are not independent.

A common error is to treat disjoint events as if they were independent, and apply the Multiplication Rule for independent events—don’t make that mistake.

Depending on IndependenceIt’s much easier to think about independent events than to deal with conditional probabilities.

It seems that most people’s natural intuition for probabilities breaks down when it comes to conditional probabilities.

Don’t fall into this trap: whenever you see probabilities multiplied together, stop and ask whether you think they are really independent.

Drawing Without ReplacementSampling without replacement means that once one individual is drawn it doesn’t go back into the pool.

We often sample without replacement, which doesn’t matter too much when we are dealing with a large population. However, when drawing from a small population, we need to take note and adjust probabilities accordingly.

Drawing without replacement is just another instance of working with conditional probabilities.

Reversing the ConditioningReversing the conditioning of two events is rarely intuitive. Suppose we want to know P(A|B), but we know only P(A), P(B), and P(B|A).We also know P(A and B), since

P(A and B) = P(A) x P(B|A) From this information, we can find P(A|B):

Reversing the ConditioningReversing the conditioning of two events is rarely intuitive. Suppose we want to know P(A|B), but we know only P(A), P(B), and P(B|A).We also know P(A and B), since

P(A and B) = P(A) x P(B|A) From this information, we can find P(A|B):

(A and B)(A|B)(B)

PPP

Reversing the Conditioning (cont.)When we reverse the probability from the conditional probability that you’re originally given, you are actually using Bayes’s Rule.

Bayes’s Theorem

1 1

||

| |

and

i ii

k k

i

P A B P BP B A

P A B P B P A B P B

P B AP A

Adding up the parts of A in all the B’s

Same Event

Bayes’s Theorem Using Contingency Table

Fifty percent of borrowers repaid their loans. Out of those who repaid, 40% had a college degree. Ten percent of those who defaulted had a college degree. What is the probability that a randomly selected borrower who has a college degree will repay the loan?

.50 | .4 | .10P R P C R P C R

| ?P R C

Bayes’s Theorem Using Contingency Table

||

| |

.4 .5 .2 .8.4 .5 .1 .5 .25

P C R P RP R C

P C R P R P C R P R

Repay

Repay

CollegeCollege 1.0.5 .5

.2

.3.05.45

.25.75

Total

Total

Bayes’ Theorem Example• A drilling company has estimated a 40%

chance of striking oil for their new well.

• A detailed test has been scheduled for more information. Historically, 60% of successful wells have had detailed tests, and 20% of unsuccessful wells have had detailed tests.

• Given that this well has been scheduled for a detailed test, what is the probability

that the well will be successful?

• Let S = successful well U = unsuccessful well

• P(S) = 0.4 , P(U) = 0.6 (prior probabilities)

• Define the detailed test event as D• Conditional probabilities:

P(D|S) = 0.6 P(D|U) = 0.2• Goal is to find P(S|D)

Bayes’ Theorem Example(continued)

0.6670.120.24

0.24

(0.2)(0.6)(0.6)(0.4)(0.6)(0.4)

U)P(U)|P(DS)P(S)|P(DS)P(S)|P(DD)|P(S

Bayes’ Theorem Example(continued)

Apply Bayes’ Theorem:

So the revised probability of success, given that this well has been scheduled for a detailed test, is 0.667

• Given the detailed test, the revised probability of a successful well has risen to 0.667 from the original estimate of 0.4

Bayes’ Theorem Example

EventPriorProb.

Conditional Prob.

JointProb.

RevisedProb.

S (successful) 0.4 0.6 (0.4)(0.6) = 0.24

0.24/0.36 = 0.667

U (unsuccessful) 0.6 0.2 (0.6)(0.2) = 0.12

0.12/0.36 = 0.333Sum = 0.36

(continued)

Counting PrinciplesSometimes determining probability depends on being able to count the number of possible events that can occur, for instance, suppose that a person at a dinner can choose from two different salads, five entrees, three drinks, and three desserts. How many different choices does this person have for choosing a complete dinner?

The Multiplication Principal for counting (which is similar to the Multiplication Principle for Probability) says that if an event consists of a sequence of choices, then the total number of choices is equal to the product of the numbers for each individual choice.

If c1,c2, c3, …,cn, represent the number of choices that can be made for each option then the total number of choices is:

c1 c2 c3 … cn

For our person at the dinner, the total number of choices would then be 2 5 3 3=90 different choices for combining salad, entrée, drink, and dessert.

Counting Rules

• Rules for counting the number of possible outcomes

• Counting Rule 1:– If any one of k different mutually exclusive

and collectively exhaustive events can occur on each of n trials, the number of possible outcomes is equal to kn

Counting Rules• Counting Rule 2:

– If there are k1 events on the first trial, k2 events on the second trial, … and kn events on the nth trial, the number of possible outcomes is

– Example:• You want to go to a park, eat at a restaurant, and see a movie.

There are 3 parks, 4 restaurants, and 6 movie choices. How many different possible combinations are there?

• Answer: (3)(4)(6) = 72 different possibilities

(k1)(k2)…(kn)

(continued)

Counting Rules

• Counting Rule 3:– The number of ways that n items can be arranged in

order is

– Example:• Your restaurant has five menu choices for lunch. How many

ways can you order them on your menu?• Answer: 5! = (5)(4)(3)(2)(1) = 120 different possibilities

n! = (n)(n – 1)…(1)

(continued)

Permutations A permutation is an arrangement of objects where order is important. For instance the digits 1,2, and 3 can be arranged in six different orders --- 123, 132, 213, 231, 312, and 321. Hence, there are six permutations of the three digits. In fact there are six permutations of any three objects when all three objects are used.

In general the number of permutations can be derived from the Multiplication Principal. For three objects, there are three choices for selecting the first object. Then there are two choices for selecting the second object, and finally there is only one choice for the final object. This gives the number of permutations for three objects as 3 2 1=6.

Now suppose that we have 10 objects and wish to make arrangements by selecting only 3 of those objects. For the first object we have 10 choices. For the second we have 9 choices, and for the third we have 8 choices. So the number of permutations when using 3 objects out of a group of 10 objects is 10 9 8=720.

We can use this example to help derive the formula for computing the number of permutations of r objects chosen from n distinct objects r n. The notation for these permutations is and the formula is:

( , )P n r

( , ) ( 1) ( 2) ... [ ( 1)]P n r n n n n r

We often use factorial notation to rewrite this formula. Recall that:

And

Using this notation we can rewrite the Permutation Formula for as

! ( 1) ( 2) ( 3) ... 3 2 1n n n n n

0! 1

( , )P n r

!( , )( )!

nP n rn r

Counting Rules• Counting Rule 4:

– Permutations: The number of ways of arranging X objects selected from n objects in order is

– Example:• Your restaurant has five menu choices, and three are selected for

daily specials. How many different ways can the specials menu be ordered?

• Answer: • different possibilities

n! 5! 120nPr 60(n )! (5 3)! 2r

nn!P

(n )!r r