Probability Theory

1. Basic concepts

2. Probability

3. Permutations

4. Combinations

Randomness

Experiments» Trial: execution of one experiment» Outcome: result of one experiment» Sample space (S): set of all possible outcomes» Sample size (n): number of trials» Not all outcomes the same due to randomness not predictable in a

deterministic sense

Events» Sample space divided into events (A1, A2, A3, …)» Union and intersection of events

» Disjoint and complement events

321

21

21

setsboth in points allonIntersecti

seteither in points allUnion

AAAS

AA

AA

CC AASAA

AA

1111

21

events Complement

eventsDisjoint

Mean and Variance

Data: multiple measurements of same quantity

Represent data graphically using histogram Definitions

» Range: » Median: middle value when values are ordered

according to magnitude» Properties

» Outlier: data value that falls outside a certain number of standard deviations

nn xxxx 121

n

iin xx

1

1Mean

n

jjn

n

jjn xxsxxs

1

21

1

1

21

12 )(deviation Standard)(Variance

minmax xxR

Matlab: Histograms

>> y = [1 3 5 8 2 4 6 7 8 3 2 9 4 3 6 7 4 1 5 3 5 8 9 6 2 4 6 1 5 6 9 8 7 5 3 4 5 2 9 6 5 9 4 1 6 7 8 5 4 2 9 6 7 9 2 5 3 1 9 6 8 4 3 6 7 9 1 3 4 7 5 2 9 8 5 7 4 5 4 3 6 7 9 3 1 6 9 5 6 7 3 2 1 5 7 8 5 3 1 9 7 5 3 4 7 9 1]’;

>> hist(y,9) histogram plot with 9 bins

>> n = hist(y,9) store result in vector n

>> x = [2 4 6 8]’

>> n = hist(y,x) create histogram with bin centers specified by vector x

>> mean(y) ans = 5.1589

>> var(y) ans = 6.1726

>> std(y) ans = 2.4845 2 4 6 80

5

10

15

20

25

30

35

1 2 3 4 5 6 7 8 90

2

4

6

8

10

12

14

16

Definition of Probability

Simple definition for finitely many equally likely outcomes

Relative frequency

General definition: P(Aj) satisfies the following axioms of probability:

1)(in points ofnumber

in points ofnumber )( SP

S

AAP jj

0)()()()(1)(

1)(0occurs timesofnumber

)(

212121

AAfAfAfAAfSf

Afn

AAf

relrelrelrelrel

jrelj

jrel

)()()()()()(

1)(1)(0

21212121 APAPAAPAPAPAAP

SPAP j

Basic Theorems of Probability

Complementation: Addition rule for mutually exclusive events

Addition rule for arbitrary events

Conditional probability of A2 given A1

Independent events

)(1)( jCj APAP

)()()()( 2121 mm APAPAPAAAP

)()()()( 212121 AAPAPAPAAP

)|()()|()()()(

)()|(

21212121

1

2112

AAPAPAAPAPAAPAP

AAPAAP

)()|()()|()()()( 1212122121 APAAPAPAAPAPAPAAP

Probability Examples

Probability that at least one coin will turn heads up from five tossed coins» Number of outcomes: 25 = 32» Probability of each outcome: 1/32» Probability of no heads: P(AC) = 1/32» Probability at least one head: P(A) = 1-P(AC) = 31/32

Probability of getting an odd number or a number less than 4 from a single dice toss» Probability of odd number: P(A) = 3/6» Probability of number less than 4: P(B) = 3/6» Probability of both:» Probability of either:

3/26/26/36/3)()()()( BAPBPAPBAP

6/2)( BAP

Permutations

Permutation – arrangement of objects in a particular order

The number of permutations of n different objects taken all at a time is: n! = 1.2.3. . .n

The number of permutations of n objects divided into c different classes taken all at a time is:

Number of permutations of n different objects taken k at a time is:

nnnnnnn

nc

c

21

21 !!!

!

knkn

nknnnn

srepetitionWith )!(

!)1()2)(1(srepetitionWithout

Permutation Examples

Box containing 6 red and 4 blue balls» Compute probability that all red balls and then all blue

balls will be removed

» n1 = 6, n2 = 4

» Probability

Coded telegram» Letters arranged in five-letter words: n = 26, k = 5

» Total number of different words: nk = 265 = 11,881,376

» Total number of different words containing each letter no more than once:

%5.0005.0!10

!4!6

!!!

1

21

nnn

P

600,893,7)!526(

!26

)!(

!

kn

n

Combinations

Combination – selection of objects without regard to order Binomial coefficients

Stirling formula:

Number of combinations of n different objects taken k at a time is:

)!1(!

)!1(1srepetitionWith

)!(!

!srepetitionWithout

nk

kn

k

knknk

n

k

n

11

1Recursion

)!(!

!

!

)1()2)(1(0Integer

!

)1()2)(1(number Real

k

a

k

a

k

aknk

n

k

knnnn

k

nnk

k

kaaaa

k

aa

!2lim2! nennennn n

n

n

Combination Examples

Effect of repetitions» Three letters a, b, c taken two at a time (n = 3, k = 2)

» Combinations without repetition

» Combinations with repetitions

500 light bulbs taken 5 at a time» Repetitions not possible

» Combinations

bcacabk

n3

)!23(!2

!3

2

3

ccbbaa

bcacab

k

kn6

)!24(!2

!4

2

41

600,686,244,255)!5500(!5

!500

5

500

k

n

Matlab: Permutations & Combinations

>> perms([2 4 6]) all possible permutations of 2, 4, 6

6 4 26 2 44 6 24 2 62 4 62 6 4

>> randperm(6) returns one possible permutation of 1-65 1 2 3 4 6

>> nchoosek(5,4) number of combinations of 5 things taken 4 at a time without repetitions

ans = 5

>> nchoosek(2:2:10,4) all possible combinations of 2, 4, 6, 8, 10 taken 4 at a time without repetitions

2 4 6 82 4 6 102 4 8 102 6 8 104 6 8 10

Matlab: Categorical Arrays

Label data in numerical arrays by category (e.g. color, size, species)

Example: flipping 10 coins

>> toss = ceil(2*rand(10,1)) randomly generate an integer = 1 or 210 times

2 1 2 1 1 1 2 1 1 1>> toss = nominal(toss,{'heads','tails'}) form a categorical array

that replaces each value of 1 with ‘heads’ and 2 with ‘tailstails heads tails heads heads heads tails heads heads heads

>> summary(toss) return the number of occurrences of each data label in the categorical array

heads tails

7 3

Repeat 100 times and plot data in histogram

>> tosses = nominal(ceil(2*rand(10,100)),{'heads','tails'})

>> T = summary(tosses)

>> hist(T’,9)

0 2 4 6 8 100

5

10

15

20

25

headstails