stat 155, section 2, last time

Stat 155, Section 2, Last Time

• Normal Distribution:– Interpretation: 68%-95%-99.7% rule– Computation of areas (frequencies)– Inverse Normal area computation

• Diagnostics (for Normal approximation)– Normal Quantile plot (linear?)

• Relations between variables– Scatterplots – useful visualization

Reading In Textbook

Approximate Reading for Today’s Material:

Pages 102-112, 123-127, 132-145

Approximate Reading for Next Class:

Pages 151-163, 173-179, 192-196

E-mail Special Request

Date: Mon, 29 Jan 2007 19:37:30 -0500

Subject: special request

Professor Marron,

I was wondering if you could do a problem like C5 in class tomorrow? I went through the notes, and I went through the workbook for Excel but it seems as though I just can't seem to get how to make a normal standard distribution. Thank you.

An Earlier EmailDate: Thu, 25 Jan 2007 08:34:34 -0500 (EST)

> I don't understand how you draw a density curve on excel without having data> points. In the NORMDIST function, you need data to go along with u, s, and> False. How do you draw it without, or where is the data?

Right, in the class example that we considered (Stor155Eg8Done.xls), we thought about fitting a normal curve to a data set. We did this by taking the mean, and s.d. of the data set, and then using that to generate the appropriate memmber of the family of normal curves.

Problem C5 is in some sense easier, since bascically the work fo calculationg the mean and s.d. is already done (note they are given as 63.1 and 4.8). So you only need to go through the other steps of generating the graphics input.

I guess that one question that will come up is "what to use for endpoints of the x grid?" You could experiment a bit, but usually

mean +- 3 s.d.gives a nice looking curve. We will see why in today's class meeting.

Another EmailDate: Sun, 28 Jan 2007 21:32:36 -0500 (EST)

> Hey Professor Marron, I'm having some problems with C5. Ok, so I opened > excel spreadsheet, I typed in the mean and median, I did mean+ 3sd mean-

3sd> For the X-value under NOrdmdist I put in the two x-values and then put in the> sd and mean, and put in FALSE for the cumulative since we want a height> distribution, but it gave me back a number. Am I doing the wrong function?

Hmm, sounds like you may not be computing enough points to generate the plot. Basically you should generate a whole column of X-values, and then plug all of those into NORMDIST.

An example of this available in Class Eg 8, which is linked to page 19 of the notes for 1/23/07.

In that spreadsheet, this grid is in cells E78-E178. The corresponding calls of NORMDIST appear inthe range: J78 - J178.

Then you plot those against each other.

Decision Problem:When should I do additional things in class?

When should I send people to Open Tutorial Sessions?

Depends on # benefitted

• 1 or 2: send to Open Tutorials

• Majority: should do in Class

Your Opinion?

1. Raise hand if you think this is worth class time right now.

2. Raise hand if you find this prospect boring, and want to move on instead.

If we do this, go to Class Example 8:http://stat-or.unc.edu/webspace/postscript/marron/Teaching/stor155-2007/Stor155Eg8Done.xls

Variable Relationships

Chapter 2 in Text

Idea: Look beyond single quantities, to how quantities relate to each other.

E.g. How do HW scores “relate”

to Exam scores?

Section 2.1: Useful graphical device:

Scatterplot

Plotting Bivariate Data

Toy Example:

(1,2)

(3,1)

(-1,0)

(2,-1)

Toy Scatterplot, Separate Points

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

-2 -1 0 1 2 3 4

x

y

Plotting Bivariate Data

Common Name: “Scatterplot”

A look under the hood:

EXCEL: Chart Wizard (colored bar icon)

• Chart Type: XY (scatter)

• Subtype controls points only, or lines

• Later steps similar to above

(can massage the pic!)

Important Aspects of Relations

I. Form of Relationship

II. Direction of Relationship

III. Strength of Relationship

I. Form of Relationship• Linear: Data approximately follow a line

Previous Class Scores Examplehttp://stat-or.unc.edu/webspace/postscript/marron/Teaching/stor155-2007/Stor155Eg10.xls

Final vs. High values of HW is “best”

• Nonlinear: Data follows different pattern

Nice Example: Bralower’s Fossil Data

http://stat-or.unc.edu/webspace/postscript/marron/Teaching/stor155-2007/Stor155Eg11.xls



Bralower’s Fossil Datahttp://stat-or.unc.edu/webspace/postscript/marron/Teaching/stor155-2007/Stor155Eg11.xls

From T. Bralower, formerly of Geological Sci.

Studies Global Climate, millions of years ago:

• Ratios of Isotopes of Strontium

• Reflects Ice Ages, via Sea Level

(50 meter difference!)

• As function of time

• Clearly nonlinear relationship

http://www.geosc.psu.edu/people/faculty/personalpages/tbralower/index.html

http://www.geosc.psu.edu/people/faculty/personalpages/tbralower/index.html

http://www.geosci.unc.edu/



II. Direction of Relationship

• Positive Association (slopes upwards)

X bigger Y bigger

• Negative Association (slopes down)

X bigger Y smaller

E.g. X = alcohol consumption, Y = Driving Ability

Clear negative association

III. Strength of Relationship

Idea: How close are points to lying on a line?

Revisit Class Scores Example:http://stat-or.unc.edu/webspace/postscript/marron/Teaching/stor155-2007/Stor155Eg10.xls

• Final Exam is “closely related to HW”

• Midterm 1 less closely related to HW

• Midterm 2 even related to Midterm 1

Linear Relationship HW

2.3, 2.5, 2.7, 2.11

Comparing Scatterplots

Additional Useful Visual Tool:

• Overlaying multiple data sets

• Allows comparison

• Use different colors or symbols

• Easy in EXCEL (colors are automatic)

Already done in HW scores example:http://stat-or.unc.edu/webspace/postscript/marron/Teaching/stor155-2007/Stor155Eg12.xls

Comparing Scatterplots HW

2.17

And Now for Something Completely Different

Remember it takes a college degree to fly a plane, but only a high school diploma to fix one.

After every flight, Qantas pilots fill out a form, called a gripe sheet which tells mechanics about problems with the aircraft. The mechanics correct the problems, document their repairs on the form, and then pilots review the gripe sheets before the next flight.


Never let it be said that ground crews lack a sense of humor. Here are some actual maintenance complaints submitted by Qantas' pilots (marked with a P) and the solutions recorded (marked with an S) by maintenance engineers.


Never let it be said that ground crews lack a sense of humor. Here are some actual maintenance complaints submitted by Qantas' pilots (marked with a P) and the solutions recorded (marked with an S) by maintenance engineers.

By the way, Qantas is the only major airline that has never, ever, had an accident.


P: Left inside main tire almost needs replacement.

S: Almost replaced left inside main tire.


P: Test flight OK, except auto-land very rough.

S: Auto-land not installed on this aircraft.


P: Dead bugs on windshield.

S: Live bugs on back-order.


P: Evidence of leak on right main landing gear.

S: Evidence removed.


P: IFF inoperative in OFF mode.

S: IFF always inoperative in OFF mode.


P: Number 3 engine missing.

S: Engine found on right wing after brief search.


P: Noise coming from under instrument panel. Sounds like a midget pounding on something with a hammer.

S: Took hammer away from midget.

Section 2.2: Correlation

Main Idea: Quantify Strength of Relationship

Context:

– A numerical summary

– In spirit of mean and standard deviation

– But now applies to pairs of variables

Section 2.2: Correlation

Main Idea: Quantify Strength of Relationship

Specific Goals:

– Near 1: for positive relat’ship & nearly linear

– > 0: for positive relationship (slopes up)

– = 0: for no relationship

– < 0: for negative relationship (slopes down)

– Near -1: for negative relat’ship & nearly linear

Correlation - Approach

Numerical Approach:

for symmetric around

has similar properties

Worked out Example :http://stat-or.unc.edu/webspace/postscript/marron/Teaching/stor155-2007/Stor155Eg13.xls

)0,0(),( ii yx

n

iii yx

1

Correlation – Graphical View

Plots (a) & (b), illustrating :

• > 0 for positive relationship

• < 0 for negative relationship

• Bigger for data closer to line

Problem 1: Not between -1 & 1

Problem 2: Feels “Scale”, see plot (c)

Problem 3: Feels “Shift” even more, see (d)

(even gets sign wrong!)

n

iii yx

1

Correlation - Approach

Solution to above problems:

Standardize!

Define Correlation

n

i y

i

x

i

syy

sxx

r1

Correlation - Example

Revisit above examplehttp://stat-or.unc.edu/webspace/postscript/marron/Teaching/stor155-2007/Stor155Eg13.xls

• r is always same, and ~1, for (a), (c), (d)

• r < 0, and not so close to -1, for (b)


A look under the hoodhttp://stat-or.unc.edu/webspace/postscript/marron/Teaching/stor155-2007/Stor155Eg13.xls

• Cols A&B: generated random numbers

(will study later)• Product versions used SUMPRODUCT• r computed with CORREL (important)• r’s same for (a) & (c), since Y’s are “just shifted”• r’s also same for (d), since x’s and Y’s shifted

(standardization cancels shifts & scales)


Revisit Class Scores Example:http://stat-or.unc.edu/webspace/postscript/marron/Teaching/stor155-2007/Stor155Eg10.xls

• r is always > 0

• r is biggest for Final vs. HW

• r is smallest for MT2 vs. MT1


Fun Example from Publisher’s Website:

http://bcs.whfreeman.com/ips5e/

Choose

• Statistical Applets

• Correlation and Regression

Gives feeling for how correlation is affected by changing data.


Fun Example from Publisher’s Website:

http://bcs.whfreeman.com/ips5e/

Interesting Exercise:

• Choose points to give correlation r = 0.95

(within 0.01)

• Destroy with a few outliers

Correlation - HW

HW:

2.23

2.25

2.27a

Correlation - Outliers

Caution:

Outliers can strongly affect correlation, r

HW:

2.27b

2.30 (big outlier reduces correlation)

Also: recompute correlation with outlier removed

And now for something completely different

Recall

Distribution

of majors of

students in

this course:

Stat 155, Section 2, Majors

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Busine

ss /

Man

.

Biolog

y

Public

Poli

cy /

Health

Pharm

/ Nur

sing

Jour

nalis

m /

Comm

.

Env. S

ci.

Other

Undec

ided

Fre

qu

ency


Tried to Google “Public Policy Jokes”

But couldn’t find anything decent.

Next tried “Public Health Jokes”

And came up with…


Regular Consumption of Guinness

Well now, you see it's like this....


A herd of buffalo can only move as fast as the slowest buffalo. And when the herd is hunted, it is the slowest and weakest ones at the rear that are killed. This natural selection is good for the herd as a whole because only the fittest survive thus improving the general health and speed of the entire herd.


In much the same way the human brain

only operates as quickly as the slowest

of it's brain cells. Excessive intake of

alcohol kills brain cells, as we all know,

and naturally the alcohol attacks the

slowest/weakest cells first....


So it is as plain as the nose on your face

that regular consumption of Guinness

will eliminate the weaker, slower brain

cells thus leaving the remaining cells

the best in the brain.


The end result, of course, is a faster more efficient brain.

If you doubt this at all, tell me, isn't it true that we always feel a bit smarter after a few pints?

Section 2.3: Linear Regression

Idea:

Fit a line to data in a scatterplot

• To learn about “basic structure”

• To “model data”

• To provide “prediction of new values”

Linear Regression

Recall some basic geometry:A line is described by an equation:

y = mx + b

m = slope m

b = y intercept b

Varying m & b gives a “family of lines”,Indexed by “parameters” m & b

Basics of Lines

Textbook’s notation:

Y = bx + a

b = m (above) = slope

a = b (above) = y-intercept

Basics of Lines

HW (to review line ideas):

C6: Fred keeps his savings in his mattress. He begins with $500 from his mother, and adds $100 each year. His total savings y, after x years are given by the equation:

y = 500 + 100 x

(a) Draw a graph of this equation.

Basics of LinesC6: (cont.)

(b) After 20 years, how much will Fred have?

($2500)

(c) If Fred adds $200 instead of $100 each year to his initial $500, what is the equation that describes his savings after x years? (y = 500 + 200 x)

Linear Regression

Approach:

Given a scatterplot of data:

Find a & b (i.e. choose a line)

to “best fit the data”

),(),...,,( 11 nn yxyx

Linear Regression - Approach

Given a line, , “indexed” by

Define “residuals” = “data Y” – “Y on line”

=

Now choose to make these “small”

),( 11 yx

abxy

)( abxy ii

),( 22 yx

),( 33 yx

ab&

ab&


Excellent Demo, by Charles Stanton, CSUSBhttp://www.math.csusb.edu/faculty/stanton/m262/regress/regress.html

• Try choosing points near a line

• Then throw in outlier

• Clear and put points on curve

• Use “Residual Plot” to diagnose that line is not a good fit to data.


JAVA Demo, by David Lane at Rice U.http://www.ruf.rice.edu/~lane/stat_sim/reg_by_eye/index.html

• Try drawing lines (to min MSE)

• Experiment with slopes

• And intercepts

• Guess r?


Make Residuals > 0, by squaring

Least Squares: adjust to

Minimize the “Sum of Squared Errors”

ab&

21

)(

n

iii abxySSE

stat 155, section 2, last time

Documents