quantitative methods for lawyers - class #11 - power laws, hypothesis testing & statistical...

Quantitative Methods

for Lawyers

Power Laws, Hypothesis Testing & Statistical

Significance Class #11

@ computational

computationallegalstudies.com

professor daniel martin katz danielmartinkatz.com

lexpredict.com slideshare.net/DanielKatz

http://lexpredict.com

http://slideshare.net/Danielkatz

Power Law Distribution (Scale Free)

This is a Classic and Very Important Distribution

A power law is a special kind of mathematical relationship between two quantities. When the frequency of an event varies as a power of some attribute of that event (e.g. its size), the frequency is said to follow a power law.

Power Law Distribution (Scale Free)

Pareto distribution ( Wealth Distribution )

Zipf's law ( Natural Language Frequency )

Links on the Internet

Citations

Richardson's Law for the severity of violent conflicts (wars and terrorism)

Population of cities

Etc.

Examples:

http://en.wikipedia.org/wiki/Pareto_distribution

http://en.wikipedia.org/wiki/Zipf%27s_law

Power Laws Appear to be a Common Feature of Legal Systems

Katz, et al (2011) American Legal Academy

Katz & Stafford (2010) American Federal Judges

Geist (2009) Austrian Supreme Court

Smith (2007) U.S. Supreme Court

Smith (2007) U.S. Law Reviews

Post & Eisen (2000) NY Ct of Appeals

Rare Events, Criticality

Power Laws

Rare Events

Criticality

Disorder

Induction

“ [T]here are known knowns; there are things we know we know.

We also know there are known unknowns; that is to say we know there are some things we do not know.

But there are also unknown unknowns – there are things we do not know we don't know. ”

United States Secretary of Defense Donald Rumsfeld

Unknown, Unknowns and Inductivist Reasoning

Philosophy of Science = How do we Know What We Know?

Black Swan Problem Even If We Observe White Swan after White Swan cannot induce that all swans are white

Learning by Falsification

Science Advances Incrementally as Hypotheses are Falsified

Popperian Perspective

Karl Popper Rejected Inductivist Reasoning


the sun has risen every day for as long as anyone can remember.

what is the rational proof that it will rise tomorrow?

How can one rationally prove that past events will continue to repeat in the future, just because they have repeated in the past?

Of Course, Certain Hypothesis cannot likely be falsified on a Reasonable Time Scale

The problem of induction:


No Need to Reject the Hypothesis of Sun Rising

Popper Solution to the Question:

Cannot Really Formulate a Theory that Can Prove that the Sun Will Always Rise

Can Develop a Theory that It Rise which will be falsified if the sun fails to rise

Hypothesis Testing & Statistical Significance

The Null and Alternative Hypothesis

Criminal Trial Burden of Proof

Example from Criminal Law:

Must Be Overruled Beyond a Reasonable Doubt

Presumption of Innocence

Not Possible to Conclusively Prove a Lack of Innocence (with zero doubt)

The Null and Alternative Hypothesis

Study is Typically Designed to Determine Whether a Particular Hypothesis is Supported

Switch Now To a Scientific Inquiry:

Start with Presumption that Hypothesis is Not True (Null Hypothesis)

Researcher Must Demonstrate That The Presumption is Unlikely to Be True given the Population

Example: Coin Flip Nostradamus

Predicting Coin Flips - Does you Friend Have the General Ability to Actually Predict Coin Flips?

How Would You Evaluate This Proposition?

How Many Predictions Would Your Friend Have to Get Right For You To Believe They Actually Have Real Ability?

Ho: Cannot Actually Predict Coin Flips

Example: Coin Flip Nostradamus

H1: Can Actually Predict Coin Flip (i.e. do so at a rate greater than chance)

Ho is the Null Hypothesis

H1 is the Alternative Hypothesis

Reject the Null versus Failing to Reject the Null

If We Fail to Reject the Null, we are left with the assumption of no relationship

In the Coin Flip Example, We might have enough evidence to reject the null

Remember the default (null) is that there is no relationship

Although a Relationship might actually exist

Coin Flip Nostradamus: Binomial Distribution

Here is the Formula for a binomial experiment consisting of n trials and results in x successes. If the probability of success on an individual trial is P, then the binomial probability is:

b(x; n, P) = nCx * Px * (1 - P)n - x

What is the Probability Coin Flip Nostradamus Predicts at least 3 of 4 Coin Tosses ?

4!


( 3! (4-3)! )

(.53) (.54-3)

(.125) (.5)

24 ( 6(1) ) = .25

Here is the Prob of Getting Exactly 3 of

4 correct

4!


( 3! (4-3)! )

(.53) (.54-3)

(.125) (.5)

24 ( 6(1) ) = .25

Here is the Prob of Getting Exactly 3 of

4 correct

= .3125We Want “At Least” Which Implies BOTH 3 and 4

.25 + .0625Exactly 3 Exactly 4 at least

3 of 4 Coin Tosses

Namely, there is a 31.25% Probability that by Chance he/she would be able to predict at least 3 out of 4

If Our Would Be Coin Flip Nostradamus were able to get 3 out 4 Correct - we would not generally be prepared to give him/her credit just yet


How Much Do We Need to Be Convinced that Our Friend is Actually Coin Flip Nostradamus?

Now We Can Calculate Probability Associated of Prediction across some arbitrary number of trials


This is a Question of Type I and Type II Error

Type I v. Type II Error

Type I v.

Type II Error

Type I v. Type II Error

It is Depends Upon the Application

Typical Convention is that a 5% Chance of Error is Acceptable for Purposes of Statistical Significance

Social Science = 5%

Medicine with Serious Side Effects might Require Greater Level of Significance 1% or even less

Back To Coin Flip Nostradamus

Predicts 43 out of 75 Correct

Okay let say Our Coin Flip Nostradamus agrees to run 75 coins flips in order to demonstrate his/her true powers

Is this Sufficient to Label Our Friend the Coin Flip Nostradamus?

Binomial Probability Calculator

http://stattrek.com/tables/binomial.aspx




Enter These Three Values

+ Hit Calculate




And These are the Results

Our P value

Here is 12.4%


Coin Flip Nostradamus

In this Case, the P Value is

Our P Value is the Probability of Observing this Data Given the Null (i.e. that our friend does not have psychic powers)

Our Pvalue > 5% Statistical Significance Threshold

“Fail to Reject” Our Null of No Psychic Powers (We Do not Say Accept -- see the induction problem)

One Tailed -or- Two Tailed Tests

In the Coin Flip Nostradamus Example it would be amazing if our friend could actually fail to predict 75 consecutive events

There is a Difference Between a Directional and a Non-Directional Hypothesis

Note:

These are

Symmetric


Stricter Crime Law and the Crime Rate

We are Often Interested in a Non-Directional Hypothesis

We are Interested in Whether there is Deterrence and if there were to be higher crime rates

New Drug and Health We Want to Both if It Makes the Patient Better and if the Patient’s condition get worse


Two Tailed TestOne Tailed Test

(negative direction)

One Tailed Test (Positive direction)

https://onlinecourses.science.psu.edu/stat500/book/export/html/43

An Example of a Hypothesis Test

Note: π is Prob α is the Significance Level




Want to Make Sure Sample is Large

Enough

Note: π is Prob α is the Significance Level




Want to Make Sure Sample is Large

Enough

If you Do Equal vs. Does Not Equal --

Two Tail

Note: π is Probα is the Significance Level




z = (p - P) / σ where p is our sample prov

P is theorized population prob σ is our Standard Deviation


I roll a single die 1,000 times and obtain a "6" on 204 rolls.

Is there significant evidence to suggest that the die is not fair?

Another Example Question

Another Example Question

Daniel Martin Katz

@ computational

computationallegalstudies.com

lexpredict.com

danielmartinkatz.com

illinois tech - chicago kent college of law@

http://computationallegalstudies.com

http://lexpredict.com

http://about.me/daniel.martin.katz