course 003: basic...

Course 003: Basic Econometrics, 2017

Course 003: Basic Econometrics

Rohini Somanathan- Part 1

Delhi School of Economics, 2017

Page 0 Rohini Somanathan

Course 003: Basic Econometrics, 2017'

&

$

%

Outline of the Part 1

Texts: Morris H. DeGroot and Mark J. Schervish, Probability and Statistics, fourth edition.

Joseph Blitzstein and Jessica Hwang, Introduction to Probability.

Topics

Probability Theory:

• Probability basics: axioms, counting methods, independence, conditional probability.

• Random variables: distribution and density functions, marginal and conditional

distributions, distributions of functions of random variables, moments of a random

variable, properties of expectations.

• Special distributions : binomial, poisson, uniform, normal, gamma..

• Convergence of random variables: laws of large numbers (LLN), central limit

theorems (CLT)

Statistical Inference:

• Estimation: maximum likelihood estimation, sufficient statistics, sampling

distributions of estimators.

• Hypotheses Testing: simple and composite hypotheses, tests for differences in means,

test size and power, uniformly most powerful tests.



&

$

%

Administrative Information

• Internal Assessment: 25% for Part 1

Two midterm exams: 10% each, August 14, September 11.

Problem Sets and attendance : 5% must hand in during tutorials (not accepted later).

• Tutorials: Each group will have one tutorial per week and a lab session every fortnight.

• Punctuality is critical - coming in late disturbs the rest of the class



&

$

%

Why is this course useful?

• We (as economists, citizens, consumers, juries, exam-takers) are often faced with

uncertainty. This may be caused by:

– randomness in the world -rain, sickness, future output

– incomplete information about a realized state of the world -Is a politician’s promise

sincere? Is a firm telling us the truth about a product? Has our opponent been dealt a

better hand of cards? Is a prisoner guilty or innocent? Is a topic going to be on the

exam?

• By putting structure on this uncertainty, we can arrive at

– decision rules: on technology, voting, conviction, study plans

– estimates of empirical relationships (wages and education, drugs and health, money

supply and GDP )

– tests of hypothesis: do we agree or disagree with the statement: smoking increase the

chance of lung cancer, speed limits lower accident rates, NREGS reduces poverty.

• Probability theory puts structure on uncertain events. Statistics is about collecting and

using data to make inferences about the population.



&

$

%

A motivating example: gender ratios

• Does the gender ratio in a population reflect discrimination?

• We visit a village and count the number of children below the age of 1.

• With no discrimination, the number of girls X ∼ Bin(n, .5).

• How should we decide that there is discrimination?



&

$

%

Gender ratios ...

Perhaps when the sample proportion, p̂ is below some threshold.

But what threshold? The following table shows P(Xn ≤ x) when p = .5

p̂ = .1 p̂ = .2 p̂ = .3 p̂ = .4 p̂ = .5

sample size

20 .0002 .0059 .0577 .2517 .5881

40 0 .0001 .0083 .1341 .5627

60 0 0 .0013 .0775 .5513

80 0 0 .0002 .0465 .5445

100 0 0 0 .0284 .5398



&

$

%

Gender ratios and sample size

0.0

5.1

.15

.2fre

quen

cy

0.0

2.0

4.0

6.0

8fre

quen

cy

Binomial probabilities for n=20 and 100, p =.5

As the sample size increases, so does the threshold below which the null hypothesis is falsely

rejected with a probability of less than .05

binomial(20,5,.5)=.021, binomial(20,5,.5)=.058, binomial(100,42,.5)=.067 and binomial(100,41,.5)=.044



&

$

%

Basic definitions

An experiment is any process whose outcome is not known in advance with certainty. These

outcomes may be random or non-random, but we should be able to specify all of them and

attach probabilities to them.

A sample space S of an experiment is the set of all possible outcomes of an experiment.

An event A is a subset of the sample space S. An event A has occurred if the outcome is in

A

Let A,B⊂ S be events. Then A∪B is the even that occurs if and only if at least one of A

and B occurs, A∩B occurs of both A and B occur and Ac (A complement) occurs if and

only if A does not occur.

When A1,A2,A3 . . . . . . of sample space S are disjoint sets the events A1,A2,A3 . . . . . . are said

to be mutually exclusive .



&

$

%

Example: 3 tosses of a coin

The experiment has 23 possible outcomes and we can define the sample space S = {s1, . . . ,s8}

where

s1 =HHH, s2 =HHT s3 =HTH, s4 =HTT , s5 = THH, s6 = THT , s7 = TTH, s8 = TTT

Any subset of this sample space is an event.

If we have a fair coin, each of the listed events are equally likely and we attach probability 18

to each of them.

The events exactly one head and exactly two heads are mutually exclusive events.



&

$

%

The concept of probability

A probability is a number attached to an event which expresses the likelihood of the event

occurring.

How are probabilities assigned to events?

– By thinking about all possible outcomes. If there are n of these, all equally likely, we

can attach numbers 1n to each of them. If an event contains k of these outcomes, we

attach a probability kn to the event. This is the classical interpretation of probability.

– Alternatively, imagine the event as a possible outcome of an experiment. Its probability

is the fraction of times it occurs when the experiment is repeated a large number of

times. This is the frequency interpretation of probability

– In many cases events cannot be thought of in terms of repeated experiments or equally

likely outcomes. We could base likelihoods in this case on what we believe about the

world subjective probabilities. The subjective probability of an event A is a real

number in the interval [0, 1] which reflects a subjective belief in the validity or

occurrence of event A. Different people might attach different probabilities to the same

events. Examples?

We formalize this subjective interpretation by imposing certain consistency conditions on

combinations of events.



&

$

%

Finite and infinite sample spaces

• Our goal is to assign probabilities to events in S.

• If S is finite, we can consider all possible subsets in S. If S has n elements, there are 2n

possible subsets. The set of all these subsets is called the power set of S.

• When S is infinite (such as the time we have to wait for a letter to arrive after an

interview), it is not obvious how we should do this.

• Carefully defining which subsets can be assigned probabilities leads to the concept of a

σ-algebra.



&

$

%

σ-algebras and Borel sets

Definition: A σ-algebra on S is a collection F of subsets of S such that

1. ∅ ∈ F

2. If A ∈ F, then Ac ∈ F

3. If A1,A2, · · · ∈ F, then⋃∞j=1Aj ∈ F

In words: F contains ∅ and is closed under complements and countable unions.

Intuition: If it makes sense to talk about the probability of an event happening, it makes sense to

talk about it not happening, and if a set of events can occur, then at least one them can occur!

Definition: A Borel σ-algebra B on R is defined to be the σ-algebra generated by all open intervals (a,b)

with a,b ∈ R. A Borel set is a set in the Borel σ-algebra.

Analogously, we define the Borel σ-algebra on Rn to be the σ-algebra in Rn generated by open

boxes or rectangles in Rn



&

$

%

Probability spaces and axioms

Definition: A probability space is a triple (S,F,P), with S a sample space, F a σ-algebra on S and P, a

probability measure defined on F

Definition: A probability measure is a function on F, taking values between 0 and 1 such that:

1. P(∅) = 0, P(S) = 1

2. P(⋃)∞j=1Aj) =

∞∑j=1

P(Aj) if the Aj are disjoint events. (countable additivity)

We will usually use P(Aj) rather than P(Aj)



&

$

%

Probability measures... some useful results

Use the probability axioms to derive the following useful results:

1. For each A⊂ S, P(A) = 1−P(Ac)

2. For A1,A2 ∈ S such that A1 ⊂A2, P(A1)≤ P(A2)

3. For each A⊂ S, 0≤ P(A)≤ 1

4. If A1 and A2 are subsets of S then P(A1 ∪A2) = P(A1)+P(A2)−P(A1 ∩A2)

5. For a finite number of events, we have:

P(

n⋃i=1

Ai) =

n∑i=1

P(Ai)−∑i<j

P(AiAj)+∑i<j<k

Pr(AiAjAk)− ...(−1)n+1P(A1A2 . . .An)



&

$

%

Examples

1. Consider two events A and B such that Pr(A) = 13 and Pr(B) = 1

2 . Determine the value of

P(BAc) in each of the following cases: (a) A and B are disjoint (b) A⊂ B (c) Pr(AB) = 18

2. Consider two events A and B, where P(A) = .4 and P(B) = .7. Determine the minimum and

maximum values of Pr(AB) and the conditions under which they are obtained?

3. A point (x,y) is to be selected from the square containing all points (x,y), such that

0≤ x≤ 1 and 0≤ y≤ 1. Suppose that the probability that the point will belong to any

specified subset of S is equal to the area of that subset. Find the following probabilities:

(a) (x− 12)

2 +(y− 12)

2 ≥ 14

(b) 12 < x+y < 3

2

(c) y < 1− x2

(d) x = y

answers: (1) 1/2, 1/6, 3/8 (2) .1, .4 (3) 1-π/4, 3/4, 2/3, 0



&

$

%

Counting methods: the multiplication rule

• A sample space containing n outcomes is called a simple sample space if the probability

assigned to each of the outcomes s1 . . . ,sn is 1n . Probability measures are easy to define in

such spaces. If the event A contains exactly m outcomes, then P(A) = mn

• Counting the number of elements in an event and in the sample space can be laborious and

sometimes complicated - we’ll look at ways to make our job easier

• The multiplication rule: Sometimes we can think of an experiment being performed in

stages, where the first stage has m possible outcomes and the second n outcomes. The total

number of possible outcomes is then mn (e.g. a sandwich can have brown or white bread

and then a tomato or cheese filling)

W

B

t

c



&

$

%

Samples, arrangements and combinations

• Suppose we are making k choices from n objects with replacement. There are nk possible

outcomes

• How many arrangements of k objects from a total of n distinct objects can be had if we are

sampling without replacement? The first object can be chosen in n different ways, leaving

(n− 1) objects so the second one can be picked in (n− 1) different ways....

• The total number of permutations of n objects taken k at a time is then

Pn,k = n(n− 1) . . . (n− k+ 1)

and Pn,n = n!. Pn,k can alternatively be written as:

Pn,k = n(n− 1).. . . . (n− k+ 1) = n(n− 1).. . . . (n− k+ 1)(n− k)!

(n− k)!=

n!

(n− k)!

• How many different subsets of k elements can be chosen from a set of n distinct elements?

Think of permutations as arising by first picking k elements and then organizing them in k!

ways. Then the number of permutations is given by Pn,k = k!Cn,k, or

Cn,k =Pn,k

k!=

n!

k!(n− k)!

This is called the binomial coefficient.



&

$

%

An application: the birthday problem

You go to watch a cricket match with a friend.

He would like to bet Rs. 100 that among the group of 23 people on the field (2 teams plus a

referee) at least two people share a birthday

Should you take the bet?

What is the probability that out of a group of k, at least two share a birthday?

– the total number of possible birthdays is 365k

– the number of different ways in which each of them has different birthdays is 365!(365−k)!

(because the second person has only 364 days to choose from, etc.). The required

probability is therefore p = 1− 365!(365−k)!365k

It turns out that for k = 23 this number is .507, so you have a small expected monetary gain

from the bet - if you don’t like risk you probably should not take it



&

$

%

The multinomial coefficient

Suppose we have k choices (jobs, modes of transport, methods of water filtration..) and are

interested in number of ways that these can be chosen by n people such that for j = 1, 2, . . . ,k

jth group contains exactly nj elements.

The n1 elements for the first group can be chosen in(nn1

)ways, the second group is chosen

out of (n−n1) elements and this can be done in(n−n1n2

)ways...The total number of ways of

dividing the n elements into k groups is therefore(nn1

)(n−n1n2

)(n−n1−n2

n3

). . .(nk−1+nknk−1

)This can be simplified to n!

n1!n2!...nk! This expression is known as the multinomial coefficient.

Example

– An student organization of 1000 people is picking 4 office-bearers and 8 members for its

managing council. The total number of ways of picking this groups is given by 1000!4!8!988!



&

$

%

Independent Events

Definition: Let A and B be two events in a sample space S. Then A and B are independent

iff P(A∩B) = P(A)P(B). If A and B are not independent, A and B are said to be dependent.

If events are independent, the occurrence of one provides no information on the likelihood

of occurrence of the other. This may be because they are physically unrelated -tossing a

coin and rolling a die, two different people falling sick with some non-infectious disease, but

not necessarily:

Example:

– The event A is getting an even number on a roll of a die .

– The event B is getting one of the first four numbers.

– The intersection of these two events is the event of rolling the number 2 or 4, which we

know has probability 13 .

– Are A and B independent? Yes because P(A)P(B) = 1223 = 1

3 But why?

If A and B are independent, then A and Bc are also independent as are Ac and Bc and Ac

and B. Why do you think this should be true?



&

$

%

Independent Events..examples and special cases

1. The experiment involves flipping two coins. A is the event that the coins match and B is

the event that the first coins is heads. Are these events independent?

In this case P(B) = P(A) = 12 ( {H,H} or {T,T}) and P(A∩B) = 1

4 , so yes, the events are

independent.

2. Suppose A and B are disjoint sets in S. Does it tell us anything about the independence of

events A and B?

3. Remember that disjointness is a property of sets whereas independence is a property of the

associated probability measure and the dependence of events will depend on the probability

measure that is being used.



&

$

%

Independence of many events

Definition: For n events, A1,A2,A3 . . . . . . be independent, every finite subset of the events must be

independent.

So pairwise independence is necessary but not sufficient.

Examples:

1. One ticket is chosen at random from a box containing 4 lottery tickets with numbers

112, 121, 211, 222.

The event Ai is that a 1 occurs in the ith place of the chosen number.

P(Ai) = 12 and P(Ai ∩Aj) = 1

4 so these 3 events are pairwise independent.

But they are not independent since P(A1 ∩A2 ∩A3) 6= P(A1)P(A2)P(A3)

2. Toss two fair dice. The sample space consists of all ordered pairs (i, j) i, j = 1, 2 . . . 6. Define

the following events :

A1 : first die = {1, 2 or 3}

A2 : first die = {3, 4 or 5}

A3 : the sum of the faces equals 9

So P(A1) = P(A2) = 12 and P(A3) = 1

9 .

P(A1 ∩A2 ∩A3) = P(3, 6) = 136 = ( 1

2)(12)(

19) = P(A1)P(A2)P(A3) but....

P(A1 ∩A3) = P(3, 6) = 136 6= P(A1)P(A3) = 1

18

So the events are not pairwise independent and so not independent.



&

$

%

Conditional probability

Probabilities reflect our beliefs about the likelihood of uncertain events. Conditional

probabilities reflect updated beliefs in the light of new evidence.

All probabilities are conditional in that they reflect accumulated knowledge.

Definition: If A and B are events with P(B) > 0, then the conditional probability of A, given B is

defined as:

P(A|B) =P(A∩B)P(B)

A is the event whose uncertainty we wish to update and B is the evidence we observe or the event we

want to take as given. P(A) is called the prior probability of A and P(A|B) is the posterior or revised

probability of A

If B occurs, we eliminate Bc and renormalize in the restricted sample space ( Fig 2.1-BH).44 Introduction to Probability

B

A

B

A

B

A

FIGURE 2.1

Pebble World intuition for P (A|B). From left to right: (a) Events A and B are sub-sets of the sample space. (b) Because we know B occurred, get rid of the outcomesin Bc. (c) In the restricted sample space, renormalize so the total mass is still 1.

all the masses by a constant so that the new total mass of the remaining pebblesis 1. This is achieved by dividing by P (B), the total mass of the pebbles in B.The updated mass of the outcomes corresponding to event A is the conditionalprobability P (A|B) = P (A � B)/P (B).

In this way, our probabilities have been updated in accordance with the observedevidence. Outcomes that contradict the evidence are discarded, and their mass isredistributed among the remaining outcomes, preserving the relative masses of theremaining outcomes. For example, if pebble 2 weighs twice as much as pebble 1initially, and both are contained in B, then after conditioning on B it is still truethat pebble 2 weighs twice as much as pebble 1. But if pebble 2 is not contained inB, then after conditioning on B its mass is updated to 0. ⇤

Intuition 2.2.4 (Frequentist interpretation). Recall that the frequentist interpre-tation of probability is based on relative frequency over a large number of repeatedtrials. Imagine repeating our experiment many times, generating a long list of ob-served outcomes. The conditional probability of A given B can then be thought ofin a natural way: it is the fraction of times that A occurs, restricting attention tothe trials where B occurs. In Figure 2.2, our experiment has outcomes which canbe written as a string of 0’s and 1’s; B is the event that the first digit is 1 and A isthe event that the second digit is 1. Conditioning on B, we circle all the repetitionswhere B occurred, and then we look at the fraction of circled repetitions in whichevent A also occurred.

In symbols, let nA, nB, nAB be the number of occurrences of A, B, A�B respectivelyin a large number n of repetitions of the experiment. The frequentist interpretationis that

P (A) � nA

n, P (B) � nB

n, P (A � B) � nAB

n.

Then P (A|B) is interpreted as nAB/nB, which equals (nAB/n)/(nB/n). This inter-pretation again translates to P (A|B) = P (A � B)/P (B). ⇤

For practice with applying the definition of conditional probability, let’s do a fewmore examples. The next three examples all start with the same basic scenario of



&

$

%

Conditioning on multiple events

For any 3 events, A1, A2 and A3 with positive probabilities,

P(A1A2A3) = P(A1)P(A2|A1)P(A3|A1A2)

We can generalize this to as many events as we like. For A1,A2, . . .An:

P(A1A2 . . .An) = P(A1)P(A2|A1)P(A3|A1A2) . . .P(An|A1 . . . ,An−1)

Notice there are multiple forms of these expressions depending and which is most convenient to

use depends on the problem at hand.



&

$

%

Bayes’ Rule

Notice from the definition of conditional probability that the following expressions are equivalent:

P(A∩B) = P(A|B)P(B) = P(B|A)P(A)

We can therefore write

P(A|B) =P(B|A)P(A)

P(B)

This expression for conditional probability of event A, given B is known as Bayes’ Rule.



&

$

%

The law of total probability

Let A1,A2, . . .Ak be a partition of the sample space S, with P(Ai) > 0 for all i. Then

P(B) =

k∑i=1

P(Ai)P(B|Ai)

This is the law of total probability. Notice that the RHS is just P(BA1 ∪BA2 ∪ · · · ∪BAk)

Using conditional probabilities sometimes makes it easy to solve a complicated problem.

Example: You play a game in which your score takes integer values between 1 and 50 with equal

probability. If your score the first time you play is equal to X, and you play until you score

Y ≥ X, what is the probability that Y = 50?

Solution: Let Ai be the event X = xi and B is getting a 50 to end the game. P(X = xi) = 150 . The

probability of getting xi in the first round and 50 to end the game is given by the product,

P(B|Ai)P(Ai). The required probability is the sum of these products over all possible values of i:

P(Y = 50) =

50∑x=1

1

51− x.1

50=

1

50(1+

1

2+

1

3+ · · ·+ 1

50) = .09



&

$

%

Bayes Rule ...examples

• C1, C2 and C3 are plants producing 10, 50 and 40 per cent of a company’s output. The

percentage of defective pieces produced by each of these is 1, 3 and 4 respectively. Given

that a randomly selected piece is defective, what is the probability that it is from the first

plant?

P(C1|C) =P(C|C1))(P(C1)

P(C)=

(.01)(.1)

(.01)(.1)+ (.03)(.5)+ (.04)(.4)=

1

32= .03

How do the prior and posterior probabilities of the event C1 compare? What does this tell

you about the difference between the priors and posteriors for the other events?

• Suppose that there is a new blood test to detect a virus. Only 1 in every thousand people

in the population has the virus. The test is 98 per cent effective in detecting a disease in

people who have it and gives a false positive for one per cent of disease free persons tested.

What is the probability that the person actually has the disease given a positive test result?

P(Disease|Positive) =P(Positive|Disease)P(Disease)

P(Positive)=

(.98)(.001)

(.98)(.001)+ (.01)(.999)= .089

So in spite of the test being very effective in catching the disease, we have a large number of

false positives. Why?



&

$

%

Bayes Rule ... priors, posteriors and politics

To understand the relationship between prior and posterior probabilities a little better, consider

the following example:

• A politician, on entering parliament, has a fairly good reputation. A citizen attaches a prior

probability of 34 to his being honest.

• At the end of his tenure, there are many potholes on roads in the politician’s constituency.

While these do not leave the citizen with a favorable impression of the incumbent, it is

possible that the unusually heavy rainfall over these years was responsible.

• Elections are coming up. How does the citizen update his prior on the moral standing of

the politician? Let us compute the posterior probability of the politician’s being honest,

given the event that the roads are in bad condition:

– Suppose that the probability of bad roads is 13 if the politician is honest and is 2

3 if

he/she is dishonest.

– The posterior probability of the politician being honest is now given by

P(honest|bad roads) =P(bad roads|honest)P(honest)

P(bad roads)=

( 13)(

34)

( 13)(

34)+ ( 2

3)(14)

=3

5

• What would the posterior be if the prior is equal to 1? What if it the prior is zero? What if

the probability of bad roads was equal to 12 for both types of politicians? When are

differences between priors and posteriors going to be large?



&

$

%

Conditioning matters: The Sally Clark case

• Sally Clark was a British solicitor who became the victim of a “one of the great miscarriages

of justice in modern British legal history”

• Her first son died within a few weeks of his birth in 1996 and her second one died in

similarly in 1998 after which she was arrested and tried for their murder.

• A well-known paediatrician Professor Sir Roy Meadow, who testified that the chance of two

children from an affluent family suffering sudden infant death syndrome was 1 in 73 million,

which was arrived by squaring 1 in 8500 for likelihood of a cot death in similar circumstance.

• Clark was convicted in November 1999. In 2001 the Royal Statistical Society issued a public

statement expressing its concern at the “misuse of statistics in the courts” and arguing that

there was “no statistical basis” for Meadow’s claim

• In January 2003, she was released from prison having served more than three years of her

sentence after it emerged that the prosecutor’s pathologist had failed to disclose

microbiological reports that suggested one of her sons had died of natural causes.

• Mistake: assumption of independence of outcomes, and confusing P(I|E) 6= P(E|I)(I=innocent, E=evidence). If the prosecution had looked at P(I|E), then the very large

prior on innocence P(I) would have played a role.


course 003: basic...

Documents