multiple linear regression

What is a Multiple Linear Regression?

Welcome to this learning module onMultiple Linear Regression

In this presentation we will cover the following aspects of Multiple Regression:

In this presentation we will cover the following aspects of Multiple Regression:- Connection to Partial Correlation and ANCOVA

In this presentation we will cover the following aspects of Multiple Regression:- Connection to Partial Correlation and ANCOVA- Unique contribution of each variable

In this presentation we will cover the following aspects of Multiple Regression:- Connection to Partial Correlation and ANCOVA- Unique contribution of each variable- Contribution of all variables at the same time

In this presentation we will cover the following aspects of Multiple Regression:- Connection to Partial Correlation and ANCOVA- Unique contribution of each variable- Contribution of all variables at the same time- Type of data multiple regression can handle

In this presentation we will cover the following aspects of Multiple Regression:- Connection to Partial Correlation and ANCOVA- Unique contribution of each variable- Contribution of all variables at the same time- Type of data multiple regression can handle- Types of relationships multiple regression

describes


describes

In this presentation we will cover the concept of Partial Correlation.


describes

After going through this presentation look at the presentation on Analysis of Covariance and consider what multiple regression and ANCOVA have in common.


describes

What is a Partial Correlation?

Partial correlation estimates the relationship between two variables while removing the influence of a third variable from the relationship.

Like in the example that follows,

Like in the example that follows, a Pearson Correlation between height and weight would yield a .825 correlation.

Like in the example that follows, a Pearson Correlation between height and weight would yield a .825 correlation. We might then control for gender (because we think being female or male has an effect on the relationship between height and weight).

However, when controlling for gender the correlation between height and weight drops to .770.


Individual Height (inches) Weight (pounds) Sex (1 – male, 2 – female)A 73 240 1B 70 210 1C 69 180 1D 68 160 1E 70 150 2F 68 140 2G 67 135 2H 62 120 2



&



& controlling for



& controlling for = .770

This is very helpful because we may think two variables (height and weight) are highly correlated but we can determine if that correlation holds when we take out the effect of a third variable (gender).

While in partial correlation, only two variables are correlated while holding a third variable constant, in multiple regression several variables are grouped together to predict an outcome variable.

While in partial correlation, only two variables are correlated while holding a third variable constant, in multiple regression several variables are grouped together to predict an outcome variable

Independent or Predictor Variables


Height



Height

Gender



Height

Gender

Age



Height

Gender

Age

Soda Drinking



Height

Gender

Age

Soda Drinking

Exercise



Height

Gender

Age

Soda Drinking

Exercise


Dependent, Response or Outcome Variable


Height

Gender

Age

Soda Drinking

Exercise



Weight


Height

Gender

Age

Soda Drinking

Exercise



Weight

all have an influence on . . .


Height

Weight

Gender

Age

Soda Drinking

Exercise





describes

Essentially, the group of predictors are all covariates to each other.

Height

Weight

Gender

Age

Soda Drinking

Exercise




Meaning,

Height

Weight

Gender

Age

Soda Drinking

Exercise




Meaning, for example,

Height

Weight

Gender

Age

Soda Drinking

Exercise




Meaning, for example, that it is possible to identify the unique prediction power of height on weight

Height

Weight

Gender

Age

Soda Drinking

Exercise




Meaning, for example, that it is possible to identify the unique prediction power of height on weight

Gender

Age

Soda Drinking

Exercise



all have an influence on . . . Height

Weight

Meaning, for example, that it is possible to identify the unique prediction power of height on weight after you’ve taken out the influence of all of the other predictors.

Gender

Age

Soda Drinking

Exercise




Weight

For example,

Height

Weight

Gender

Age

Soda Drinking

Exercise




For example, here is the correlation between Height and Weight without controlling for all of the other variables.

Gender

Age

Soda Drinking

Exercise




Weight

For example, here is the correlation between Height and Weight without controlling for all of the other variables.

Gender

Age

Soda Drinking

Exercise




Weight

Correlation = .825

However,

Height

Weight

Gender

Age

Soda Drinking

Exercise




However, here is the correlation between Height and Weight after taking out the effect of all of the other variables.

Height

Weight

Gender

Age

Soda Drinking

Exercise





Gender

Age

Soda Drinking

Exercise




Weight

Height


Gender

Age

Soda Drinking

Exercise




Weight

Correlation = .601


Gender

Age

Soda Drinking

Exercise




Weight

Correlation = .601

So, after eliminating the effect of gender, age, soda, and exercise on weight, the

unique correlation that height shares with weight is .601.

Height

Even though we were only correlating height and weight when we computed a correlation of .825,

Even though we were only correlating height and weight when we computed a correlation of .825, the other four variables still had an influence on weight.

Even though we were only correlating height and weight when we computed a correlation of .825, the other four variables still had an influence on weight. However, that influence was not accounted for and remained hidden.

With multiple regression we can control for these four variables and account for their influence

With multiple regression we can control for these four variables and account for their influence thus calculating the unique contribution height makes on weight without their influence being present.

We can do the same for any of these other variables. Like the relationship between Gender and Weight.


Height

Age

Soda Drinking

Exercise




Weight

Gender


Height

Age

Soda Drinking

Exercise




Weight

Gender Correlation = .701

But when you take out the influence of the other variables the correlation drops from .701 to .582.

BEFORE

Height

Age

Soda Drinking

Exercise




Weight


AFTER

Height

Age

Soda Drinking

Exercise




Weight

Gender

AFTER

Height

Age

Soda Drinking

Exercise




Weight


Here is the correlation between age and weight before you take out the effect of the other variables:


Height

Gender

Soda Drinking

Exercise




Weight Age


Height

Gender

Soda Drinking

Exercise




Weight Age Correlation = .435

The correlation drops from .435 to .385 after taking out the influence of the other variables:


Height

Gender

Soda Drinking

Exercise




Weight Age


Height

Gender

Soda Drinking

Exercise




Weight Age Correlation = .385


describes

Beyond estimating the unique power of each predictor,

Beyond estimating the unique power of each predictor, multiple regression also estimates the combined power of the group of predictors.


Height

Weight

Gender

Age

Soda Drinking

Exercise




Height

Weight

Gender

Age

Soda Drinking

Exercise



Combined Correlation

= .982


describes

Multiple regression can estimate the effects of continuous and categorical variables in the same model.


Height



Age

Soda Drinking

Exercise

Gender

Weight


Height



Age

Soda Drinking

Exercise

Gender

Weight

Height is represented by continuous data – because height can take on any value between two points in inches or centimeters.


Height

Age

Soda Drinking

Exercise




Weight

Gender Gender is a represented by categorical data – because gender can take on two values (female or male)


describes

Multiple regression can describe or estimate linear relationships like average monthly temperature and ice cream sales:


0 20 40 60 80 100 1200

100

200

300

400

500

600

700

Average Monthly Ice Cream Sales

Ave

Mon

thly

Tem

pera

ture


0 20 40 60 80 100 1200

100

200

300

400

500

600

700


Ave

Mon

thly

Tem

pera

ture

Linea

r Rela

tionsh

ip

It can also describe or estimate curvilinear relationships.

For example,

For example, what if in our fantasy world the temperature reached 100 degrees and then 120 degrees. Let’s say with such extreme temperatures ice cream sales actually dip as consumers seek out products like electrolyte-enhanced drinks or slushies.

Then the relationship might look like this:


0 20 40 60 80 100 1200

100

200

300

400

500

600

700


Ave

Mon

thly

Tem

pera

ture


0 20 40 60 80 100 1200

100

200

300

400

500

600

700


Ave

Mon

thly

Tem

pera

ture

This is an example of a Curvilinear

Relationship

In summary,

In summary, Multiple Regression is like single linear regression but instead of determining the predictive power of one variable (temperature) on another variable (ice cream sales) we consider the predictive power of other variables (such as socio-economic status or age).

With multiple regression you can estimate the predictive power of many variables on a certain outcome,

With multiple regression you can estimate the predictive power of many variables on a certain outcome, as well as the unique influence each single variable makes on that outcome after taking out the influence of all of the other variables.

multiple linear regression

Education

data multiple regression

partial correlation

variable contribution

multiple linearregression

ancova unique contribution

variables presentation

regression concept

willtime type