school of distance education · school of distance education mathematical economics page 6 economic...

MATHEMATICAL ECONOMICS

IV Semester

COMPLEMENTARY COURSE

B Sc MATHEMATICS

(2011 Admission)

UNIVERSITY OF CALICUTSCHOOL OF DISTANCE EDUCATIONCalicut University P.O. Malappuram, Kerala, India 673 635

422


IV Semester


B Sc MATHEMATICS

(2011 Admission)


422


IV Semester


B Sc MATHEMATICS

(2011 Admission)


422

School of Distance Education

Mathematical Economics

UNIVERSITY OF CALICUT

SCHOOL OF DISTANCE EDUCATION

STUDY MATERIALCOMPLEMENTARY COURSEB Sc Mathematics

IV Semester


Prepared by &Scrutinised by:

Sri. Shabeer K P,Assistant Professor,Dept. of Economics,Govt College Kodanchery.

Layout: Computer Section, SDE

©Reserved



CONTENTS PAGE No.

MODULE I INTRODUCTION TO ECONOMETRICS 5

MODULE II TWO VARIABLE REGRESSION MODEL 17

MODULE III THE CLASSICAL NORMAL LINEARREGRESSION MODEL 37

MODULE IV EXTENSION OF TWO VARIABLEREGRESSION MODEL 52



MODULE I

INTRODUCTION TO ECONOMETRICS

1.1 Definition and Scope of Econometrics

Literally interpreted, econometrics means economic measurement. Econometricsdeals with the measurement of economic relationships. It is a science whichcombines economic theory with economic statistics and tries by mathematical andstatistical methods to investigate the empirical support of general economic lawestablished by economic theory. Econometrics, therefore, makes concrete certaineconomic laws by utilising economics, mathematics and statistics. The termeconometrics is formed from two words of Greek origin, ‘oukovouia’ meaningeconomy and ‘uetpov’ meaning measure.

Although measurement is an important part of econometrics, the scope ofeconometrics is much broader, as can be seen from the following quotations. In thewords of Arthur S Goldberger “econometrics may be defined as the social science inwhich the tools of economic theory, mathematics and statistical inference areapplied to the analysis of economic phenomena”. Gerhard Tintner points out that“econometrics, as a result of certain outlook on the role of economics, consists ofapplication of mathematical statistics to economic data to lend empirical support tothe models constructed by mathematical economics and to obtain numericalresults”. For H Theil “econometrics is concerned with the empirical determination ofeconomic laws”. In the words of Ragnar Frisch “the mutual penetration ofquantitative econometric theory and statistical observation is the essence ofeconometrics”.

Thus, econometrics may be considered as the integration of economics,mathematics and statistics for the purpose of providing numerical values for theparameters of economic relationships and verifying economic theories. It is a specialtype of economic analysis and research in which the general economic theory,formulated in mathematical terms, is combined with empirical measurement ofeconomic phenomena. Econometrics is the art and science of using statisticalmethods for the measurement of economic relations. In the practice ofeconometrics, economic theory, institutional information and other assumptions arerelied upon to formulate a statistical model, or a set of statistical hypotheses toexplain the phenomena in question.

Economic theory makes statements or hypotheses that are mostly qualitative innature. Econometrics gives empirical content to most economic theory.Econometrics differs from mathematical economics. The main concern of themathematical economics is to express economic theory in mathematical form(equations) without regard to measurability or empirical verification of the theory. Asnoted above, econometrics is mainly interested in the empirical verification of



economic theory. The econometrician often uses the mathematical equationsproposed by mathematical economist but put these equations in such a form thatthey lend themselves to empirical testing. Further, although econometricspresupposes the expression of economic relationships in mathematical form, likemathematical economics it does not assume that economic relationships that areexact. On the contrary, econometrics assumes that economic relationships are notexact but stochastic. Econometric methods are designed to take into accountrandom disturbances which create deviations from exact behavioural patternssuggested by economic theory and mathematical economics. Econometric methodsare designed in such a way that they take into account the random disturbances.

Econometrics differs both from mathematical statistics and economic statistics.An economic statistician gathers empirical data, records them or charts them, andthen attempts to describe the pattern in their development over time and detectssome relationship between various economic magnitudes. Economic statistics ismainly descriptive aspect of economics. It does not provide explanations of thedevelopment of the various variables and measurement of the parameters ofeconomic relationships. On the contrary, mathematical statistics deals withmethods of measurement which are developed on the basis of controlledexperiments in laboratories. Statistical methods of measurement are notappropriate for economic relationships, which cannot be measured on the basis ofevidence provided by controlled experiments, because such experiments cannot bedesigned for economic phenomena. For instance, in studying the economicbehaviour of human beings one cannot change only one factor while keeping allother factors constant. In real world, all variables change continuously andsimultaneously. So the controlled experiments are not possible. Econometrics usesstatistical methods for adapting them to the problems of economic life. Theseadapted statistical methods are called econometric methods. In particular,econometric methods are adjusted so that they become appropriate for themeasurement of economic relationships which are stochastic, that is, they includerandom elements.

1.2 Methodology of Econometrics

Broadly speaking, traditional or classical econometric methodology consists ofthe following steps.

1) Statement of the theory or hypothesis2) Specification of the mathematical model of the theory3) Specification of the econometric model of the theory4) Obtaining the data5) Estimation of the parameters of the econometric model6) Hypothesis testing7) Forecasting or prediction8) Using the model for control or policy purposes.

To illustrate the preceding steps, let us consider the well known psychological law ofconsumption.



1) Statement of theory or hypothesis

Keynes stated “the fundamental psychological law......is that men (women) aredisposed, as a rule and on average, to increase their consumption as their incomeincreases, but not as much as the increase in their income”. In short, Keynespostulated that the marginal propensity to consume (MPC), that is, the rate ofchange in consumption as a result of change in income, is greater than zero, butless than one. That is 0<MPC<1.

2) Specification on the mathematical model of consumption

Economic theory may or may not indicate the precise mathematical form of therelationship or the number of equations to be included in the economic model.Mathematical model is specifying mathematical equations that describe therelationships between economic variables as proposed by the economic theory.Although Keynes postulated a positive relationship between consumption andincome, he did not specify the precise form of functional relationship between thetwo. For simplicity, a mathematical economist might suggest the following form ofthe Keynesian consumption function:

Y =β1+β2X 0< β2 <1 (1.1)

Where Y = consumption expenditure, X= income and β1and β2, known asparameters of the model are intercept and slope coefficients respectively. The slopecoefficient β2 measures the MPC. This equation, which states that consumption islinearly related to income, is an example of mathematical model of relationshipbetween consumption and income that is called the consumption function ineconomics. A model is simply a set of mathematical equations. If the model has onlyone equation, it is called single equation model. If the model has more than oneequation it is called a multiple equation model.

In the above equation (1.1), the variable appearing on the left side of the equalitysign is called the dependent variable and the variables on the right side are calledthe independent or explanatory variables. Thus, in the Keynesian consumptionfunction, consumption expenditure is the dependent variable and income is theexplanatory variable.

(3) Specification of the econometric model of consumption

The purely mathematical model of the consumption function as in equation(1.1) is of limited interest to the econometrician because it assumes that there is anexact or deterministic relationship between consumption and income. Butrelationships between economic variables are generally inexact. Thus, if we obtaindata on consumption and income, we could not expect all the observations to lie onthe straight line. This is because of the fact that in addition to income othervariables affect consumption expenditure. For example, size of family, ages of themembers in the family, family religion etc are likely to exert some influence onconsumption.



To allow for the inexact relationship between economic variables, theeconometrician would modify the deterministic consumption function as follows

Y =β1+β2X+u (1.2)Where u, known as the disturbance or error term, is a random or stochastic

variable. The disturbance term u represents all those factors that affectconsumption but are not taken into account explicitly. This equation is an exampleof an econometric model. More technically, it is an example of linear regressionmodel. The econometric consumption function hypotheses that the dependentvariable Y (consumption) is linearly related to the explanatory variable X (income)but that the relationship between the two is not exact; it is subject to individualvariation. The econometric model of consumption function is shown below

(4) Obtaining DataTo estimate the econometric model given in equation (1.2), that is, to obtain

the numerical values of β1and β2, we need data. Three types of data may beavailable for empirical analysis. They are time series, cross-sectional and pooleddata. A time series is a set of observations on the values that a variable takes atdifferent times. That is, time series data give information about the numericalvalues of variables from period to period. Such data may be collected at regulartime intervals such as daily, weekly, monthly, quarterly, annually, quinquennially ordecennially. The data thus collected may be quantitative or qualitative. Qualitativevariables also called “dummy variables or categorical variables” can be every bit asimportant as the quantitative variables. Thus, data on one or more variablescollected over a period of time is called time series data. That is, values of one ormore variables for several time periods pertaining to a single economic entity aregiven such data set is called time series data. Cross-sectional data are data on oneor more variables collected at the same point of time. These data give information onthe variables concerning individual agents at a given point of time. Pooled data is acombination of time series and cross sectional data. That is, in the pooled data areelements of both time series and cross-sectional data.

u

0

Con

sum

ptio

n ex

pend

itur

e

Income



(5) Estimation of the econometric model

After the model has been specified and data has been collected, theeconometrician must proceed with its estimation. In other words, he must obtainthe numerical estimates of the coefficients of the model. In our case, the task is toestimate the parameters of the consumption function, that is, β1 and β2. Thenumerical estimates of the parameters gives empirical content to the consumptionfunction. The statistical tool of regression analysis is the main tool used to obtainthe estimates. Choice of the appropriate econometric technique for the estimation ofthe function and critical examination of the assumptions of the chosen technique isa crucial step.

(6) Hypothesis Testing

A hypothesis is a theoretical proposition that is capable of empiricalverification or disproof. It may be viewed as an explanation of some event or events,and which may be true or false explanation. Assuming that the fitted model is areasonably good approximation of reality, we have to develop suitable criteria to findout whether the estimates obtained are in accordance with the expectation of thetheory that is being tested. According to Milton Friedman, a theory or hypothesisthat is not verifiable by appeal to empirical evidence may not be admissible as a partof scientific theory. Confirmation or refutation of economic theories on the basis ofsample evidence is based on a branch of statistical theory known as statisticalinference or hypothesis testing.

(7) Forecasting or PredictionThe objective of any econometric research is to obtain good numerical

estimates of the coefficients of economic relationships and use them for theprediction of the values of economic variables. Forecasting is one of the prime aimsof econometric research. Econometric methods are used to estimate the parametersof the model, to test the hypothesis concerning them and to generate forecasts fromthe model. If the model confirms the hypotheses or theory under consideration, wemay use it to predict the future values of the dependent or forecast variable Y, onthe basis of known value or expected values of the explanatory, or predictor,variable X. It is conceivably possible that the model is economically meaningful andstatistically and econometrically correct for the sample period for which the modelhas been estimated, yet it may well not be suitable for forecasting. The reasons maybe either (a) the values of the explanatory variables used in the forecast may beinaccurate or (b) the estimates of the coefficients may be incorrect due todeficiencies of the sample data.

(8) Use the model for control or policy purposes

Suppose that we have estimated the Keynesian consumption function, thenthe government can use it for control or policy purposes such as to determine thelevel of income that will guarantee the target amount of consumption expenditure.In other words, an estimated model may be used for control or policy purposes. Byappropriate fiscal and monetary policy mix, the government can manipulate thecontrol variable X to produce the desired lavel the target variable Y.



1.3 The Nature of Regression Analysis

The term regression was introduced by Francis Galton. In a famous paper“Family Likeness in Stature”, Galton found that, although there was a tendency fortall parents to have tall children and for short parents to have short children, theaverage height of children born of parents of a given height tended to move or“regress” toward the average height of population as a whole. In other words, theheight of the children of unusually tall or unusually short parents tends to movetoward the average height of the population. Galton’s “law of universal regression”was confirmed by Karl Pearson, who collected more than a thousand records ofheights of members of family groups.

However, the modern interpretation of regression is quite different. Broadlyspeaking, we may say that regression analysis is concerned with the study of thedependence of one variable, the dependent variable, on one or more other variables,the explanatory variables, with a view to estimating and/or predicting thepopulation mean or average value of the former in terms of the known or fixedvalues of the latter.

In regression analysis, we are concerned with what is known as the statistical,not functional or deterministic, dependence among variables. In statisticalrelationships among variables we essentially deal with ‘random’ or ‘stochastic’variables. That is, variables that have probability distributions. A random orstochastic variable is a variable that can take on any set of values, positive ornegative, with a given probability. On the other hand, in functional or deterministicdependency, we also deal with variables, but these variables are not random orstochastic.

In other words, a relation between X and Y characterised as Y = f(X) is said tobe deterministic or non-stochastic if for each value of independent variable X, thereis one and only one corresponding value of dependant variable Y. On the otherhand, a relation between X and Y is said to be stochastic if for a particular value ofX, there is whole probability distribution of values of Y.

Although regression analysis deals with the dependence of one variable onother variables, it does not necessary imply causation. In the words of Kendall andStuart, “a statistical relationship, however strong and however suggestive, can neverestablish casual connection: our ideas of causation must come from outsidestatistics, ultimately from theory or other”. That is, a statistical relationship per secannot logically imply causation. To ascribe causality, one must appeal to a priori ortheoretical consideration.

1.4 Regression and Correlation

Closely related to but conceptually very much different from regressionanalysis is correlation analysis, where the primary objective is to measure thestrength or degree of linear association between two variables. The correlationcoefficient measures this strength of linear association. For example, we may beinterested in finding the correlation coefficient between marks of mathematics and



statistics examination, between smoking and lung cancer and so on. In regressionanalysis, we are not primarily interested in such a measure. Instead, we try toestimate or predict the average value of one variable on the basis of the fixed valuesof other variables. Thus, we want to know whether predict the average mark on amathematics examination by knowing a student’s marks on a statisticsexamination.

In regression analysis, there is an asymmetry in the way the dependent andexplanatory variables are treated. The dependent variable is assumed to bestatistical, random or stochastic, that is, to have a probability distribution. On theother hand, the explanatory variables are assumed to have fixed values (in repeatedsampling). But in correlation analysis we treat any two variables symmetrically;there is no distinction between the dependent and explanatory variables. Thecorrelation between marks of mathematics and statistics examinations is the sameas that between marks of statistics and mathematics examinations. Moreover, bothvariables are assumed to be random. While most of the correlation theory is basedon the assumption of randomness of variables, whereas most of the regressiontheory is based on the assumption that the dependent variable is stochastic butexplanatory variables are fixed or non-stochastic.

1.5 Two-Variable Regression Analysis

As noted above, regression analysis is largely concerned with estimatingand/or predicting the population mean or average value of the dependant variableon the basis of the known value of the explanatory variable. We start by a simplelinear regression model, that is, by the relationship between two variables, onedependent and one explanatory, related with a linear function. If we are studyingthe dependence of a variable on only a single explanatory variable, such asconsumption expenditure on income, such study is known as the simple or two-variable regression analysis.

Suppose we are interested in studying the relationship between weeklyconsumption expenditure Y and weekly after-tax or disposable family income X.More specifically, we want to predict the population mean level of weeklyconsumption expenditure knowing the family weekly income. For each of theconditional probability distributions of Y we can compute its mean average value,known as conditional mean or conditional expectation, denoted as E (Y/X=Xi) and isread as “the expected value of Y given that X takes the specific value X i, which forsimplicity written as E(Y/Xi). An expected value is simply a population mean oraverage value.

Each conditional mean E(Y/Xi) will be a function of Xi. Symbolically,

E(Y/Xi) = f (Xi) (1.3)

Where f (Xi) denotes some function of the explanatory variable Xi. We assumethat E(Y/Xi) is a linear function of Xi. Equation (1.3) is known as the two variablePopulation Regression Function (PRF) or Population Regression (PR) for short. Itstates merely that the population mean of the distribution of Y given X i is



functionally related to Xi. In other words, it tells how the mean or average responseof Y varies with X. The form of the function f (Xi) is important because in realsituations we do not have the entire population available for examination. Therefore,the functional of the PRF is an empirical question. For example, an economist mighthypothesize that consumption expenditure is linearly related to income. Thus, weassume that the PRF E(Y/Xi) is a linear function of Xi, say of the type,

E(Y/Xi) = β1+β2Xi (1.4)

Where β1and β2 are unknown but fixed parameters known as the regressioncoefficients. β1and β2 are also known as the intercept and slope coefficientsrespectively. Equation (1.4) is known as the linear population regression function orsimply the linear population regression. Some alternative expressions used arelinear population regression model or linear population regression equation. Inregression analysis our interest is in estimating the PRF like equation (1.4) that isestimating the values of unknowns β1and β2 on the basis of observations on Y andX.

1.6 The Meaning of the term “Linear”

The term linear can be interpreted in two different ways, namely, linearity invariables and linearity in parameters. The first and perhaps more natural meaningof linearity is that the conditional expectation of Y is a linear function of X i, such asequation (1.4). Geometrically, the regression curve in this case is a straight line. Inthis interpretation, a regression function such as E(Y/Xi) = β1+β2Xi2 is not a linearfunction because the variable X appears with a power of 2.

The second interpretation of linearity, that is, linearity in parameters is thatthe conditional expectation of Y, E(Y/Xi), is a linear function of the parameters thatis β’s. It may or may not be linear in the variable X. In this interpretation E(Y/X i) =β1+β2Xi2 is a linear regression model but E(Y/Xi) = β1+ β Xi is not. The later is anexample of a nonlinear in parameters regression model.

Of the two interpretations of linearity, linearity in the parameters is relevantfor the development of regression theory. Thus, for our analysis, the term linearregression will always mean a regression that is linear in the parameters, that is β’s.In other words, parameters are raised to the first power only. It may or may not belinear in the explanatory variable, the X’s. It can be noted that E(Y/X i) = β1+β2Xi islinear in both parameters and variable.

1.7 Stochastic Specification of PRF

The stochastic nature of the regression model implies that for every value of Xthere is a whole probability distribution of values of Y. In other words, the value of Ycan never be predicted exactly. The form of the equation (1.4) implies that therelationship between consumption expenditure and income is exact, that is, all thevariations in Y are is due solely to changes in X, and there is no other factorsaffecting the dependent variable. Given the income level of X i, an individual family’sconsumption expenditure will cluster around the average consumption of allfamilies at that Xi, that is around the conditional expectation. Therefore, we canexpress the deviation of an individual Yi around the expected value as follows.



ui = Yi ̶ E(Y/Xi)

or

Yi= E(Y/Xi) + ui (1.5)

Where the deviation ui is an unobservable random variable taking positive ornegative values. Technically ui is known as the stochastic disturbance or stochasticerror term. This is because ui is supposed to ‘disturb’ the exact linear relationshipwhich is assumes to exist between X and Y. The equation (1.5) means that theexpenditure of an individual family, given its income level, can be expressed as thesum of two components. Firstly, E(Y/Xi), which is simply the mean consumptionexpenditure of all families with the same level of income. This component is knownas the systematic or deterministic component. Secondly, ui, which is the random ornon-systematic component. In other words, [variations in Yi] = [Systematic variation]+ [random variation] or [variations in Yi] = [explained variations] + [unexplainedvariation].

Since E(Y/Xi) is assumed to be linear in Xi, equation (1.5) may be written as

Yi= E(Y/Xi) + ui

Yi = β1+β2Xi + ui (1.6)

Equation (1.6) posits that the consumption expenditure of a family is linearlyrelated to its income plus the disturbance term. Now if we take the conditionalexpectation, conditional upon the given X’s on both sides of the equation (1.5), weobtain

E(Yi/Xi) = E[E(Y/Xi)]+ E(ui/Xi)

E(Yi/Xi) = E(Y/Xi)+ E(ui/Xi) (1.7)

Since the expected value of a constant is that constant itself. Again, E(Y i/Xi) isthe same thing as E(Y/Xi), equation (1.7) implies that

E(ui/Xi) = 0 (1.8)

Thus, the assumption that the regression line passes through the conditionalmean of Y implies that the conditional mean values of ui (conditional upon the givenX’s) are zero. The stochastic specification clearly shows that there are othervariables besides income that affect the consumption expenditure and that anindividual’s family consumption expenditure cannot be fully explained only by thevariables included in the regression model.

1.8 The Significance of the Stochastic Disturbance Term

As noted above, the disturbance tern ui is the substitute for all those variables thatare omitted from the model but that collectively affect Y. The reasons for notintroducing those variables into the model and significance of stochasticdisturbance term ui are explained below.



a) Vagueness of theory

The theory determining the behaviour of Y may be incomplete. We might knowfor certain that weekly income X influences weekly consumption expenditure Y, butwe might be ignorant or unsure about other variables affecting Y. Therefore, u i maybe used as a substitute for all the excluded or omitted variables from the model.

b) Unavailability of Data

Even if we know some of the excluded variables and therefore consider a multipleregression rather than a simple regression, we may not have quantitativeinformation about these variables. For example, we could introduce family wealth asan explanatory variable in addition to the income variable to explain familyconsumption expenditure. But unfortunately, information about family wealth is notgenerally available.

c) Core variables vs. Peripheral variables

Assume the in the consumption-income example that besides income X1, thenumber of children per family X2, sex X3, religion X4, and education X5 also affectconsumption expenditure. But it is quite possible that the joint influence of all orsome of these variables may be so small ad non-systematic or random. Thus, as apractical matter and for cost consideration we do not introduce them into the modelexplicitly. Their combined effect can be treated as a random variable u i.

d) Intrinsic Randomness in Human Behaviour

Human behaviour is not predictable. Even if we succeed in introducing all therelevant variables into the model, there is bound to be some intrinsic randomness inindividual Y that cannot be explained no matter how hard we try. The disturbances,the u’s may very well reflect this intrinsic randomness.

e) Poor Proxy Variables

Although regression model assumes that the variables Y and X are measuredaccurately, in practice the data may be plagued by errors of measurement. Thedeviations of the points from the true regression line may be due to errors ofmeasurement of the variables, which are inevitable due to the methods of collectingand processing statistical information. The disturbance term u i also represents theerrors of measurement.

f) Principle of Parsimony

Following Occam’s razor which states that descriptions be kept as simple aspossible until proved inadequate, we would like to keep our regression model assimple as possible. If we can explain the behaviour of Y substantially with two orthree explanatory variables and if our theory is not strong enough to suggest thatother variables might be included, there is no need to introduce other variables. Letui represent all other variables.



g) Wrong Functional Form

Even if we are theoretically correct variables explaining a phenomenon and evenif we can obtain data on these variables, very often we do not know the form of thefunctional relationship between the regressand and the regressors. We may havelinearised a possibly nonlinear relationship. Or we may have left out of the modelsome equations. The economic phenomena are much more complex than a singleequation may reveal, no matter how many explanatory variables it contains. Thus,the disturbance term ui represents such errors which may be due to wrongspecification of the functional relationship.

Thus, for all these reasons, the stochastic disturbances ui assume an extremelycritical role in regression analysis.

1.9 The Sample Regression Function (SRF)

The linear relationship Yi = β1+β2Xi + ui holds for the population of the values ofX and Y so that we could obtain the numerical values of β1 and β2 only if we have allthe conceivably possible values of X, Y and u which form the population of thesevariables. But in practice one rarely has access to entire population of interest.Thus, in most practical situations we have only sample Y values corresponding tosome fixed X’s. Therefore our task is to estimate the PRF on the basis of the sampleinformation.

Analogous to the PRF, we can develop the concept of Sample RegressionFunction (SRF) to represent the sample regression line. The sample counter part ofequation (4) may be written asY = β + β X (1.9)

Where Y is read as “Y-hat” or “Y-cap”. In the above equation,Y =Estimator of E(Y/Xi)

β = Estimator of β1

β = Estimator of β2

An estimator also known as a sample statistic is simply a rule or formula ormethod that tells us how to estimate the population parameter from the informationprovided by the sample at hand. The particular numerical value obtained by theestimator in an application is known as an estimate. Now we can express the SRF inits stochastic form as Y = β + β X + u (1.10)

Where , u denotes the sample residual term. Conceptually, u is analogous to ui

and can be regarded as an estimate of ui. It is introduced in the SRF for the samereason as ui was introduced in the PRF.



Thus, to sum up, the primary objective in the regression analysis is to estimate thePRF

Yi = β1+β2Xi + ui

On the basis of the SRFY = β + β X + uBut because of the sampling fluctuations our estimate of PRF based on the SRF

is at best an approximate one. This approximation is shown the figure below.

For X = Xi, we have one sample observation Y=Yi. In terms of the SRF, the observedYi can be expressed as

Yi = Y + u (1.11)

And in terms of PRF, it can be expressed as

Yi = E (Y/Xi) + ui (1.12)In the figure, Y overestimates the true E(Y/Xi). But at the same time, for any Xi

to the left of point A, SRF will underestimate the true PRF. Such over andunderestimation is inevitable because of sampling fluctuations. The important taskis to device a rule or method that will make the approximation as close as possible.That is, SRF should be constructed such that β is as close as possible to true β1and β is as close possible to the true β2, even though we never know the true β1and β2.

YYi

Yui

A

E(Y/Xi)

SRF: = +PRF: E(Y/Xi) = β1+β2Xi

u

0

Wee

kly

Cons

umpt

ion

expe

nditu

re

Yi

Xi

Weekly Income



MODULE II

TWO VARIABLE REGRESSION MODEL

2.1 The Method of Ordinary Least Squares

Our task is to estimate the population regression function (PRF) on the basisof the sample regression function (SRF) as accurately as possible. Though there areseveral methods of constructing the SRF, the most popular method to estimate thePRF from SRF is the method of Ordinary Least Squares (OLS). The method ofordinary least squares is attributed to German mathematician Carl Friedrich Gauss.Under certain assumptions, the method of least squares has some very attractivestatistical properties that have made it one of the most popular methods ofregression analysis.

The relationship between X and Y in the PRF is

Yi = β1+β2Xi + ui

Since PRF is not directly observable, we estimate it from the SRF,Y = β + β X + uYi = Y + u

Where Y is the estimated conditional mean value of Yi. Now to determine the SRF,we have u = Y − Yu = Y − ( β + β X )u = Y − β − β X (2.1)

Which shows that the residuals, u are simply the difference between the actualand estimated Y values. Given n pairs of observations on Y and X, we would like todetermine the SRF in such a manner that it is as close as possible to the actual Y.To this objective, we may adopt the following criterion: Choose the SRF in such away that the sum of residuals ∑u = ∑(Y − Y ) is as small as possible. Therationale of this criterion is easy to understand. It is intuitively obvious that thesmaller the deviations from the population regression line, the better the fit of theline to the scatter of observations. Although intuitively appealing this is not a goodcriterion as can be seen from the following scatter diagram.



If we adopt the criterion of minimising ∑u , then all the residuals will receivethe same weight in the sum, although u and u are more widely scattered aroundthe SRF than u and u . In other words, all the residuals receive equal importanceno matter how close or how widely scattered the individual observations are fromthe SRF. A consequence of this is that it is quite possible that the algebraic sum ofthe u is small or even zero although u are widely scattered around the SRF. That is,while summing these deviations the positive values will offset the negative values, sothat the final algebraic sum of these residuals might equal to zero.

For avoiding this problem the best solution is to square the deviations andminimise the sum of squares. That is we can adopt the least squares criterion,which states that the SRF can be fixed in such a way thatu = (Y − Y )

That is, ∑u = ∑(Y − β − β X ) (2.2)

Thus, in the least squares criterion, we try to make equation (2.2) as small aspossible, where u are the squared residuals. The reason for calling this method asthe least squares method is that it seeks the minimisation of the sum of squares ofthe deviations of actual observations from the line. By squaring u , this methodgives more weight to residuals such as u and u in the above than the residuals uand u As noted previously, under the minimum∑u criterion, the sum can be smalleven though the u are widely spread around the SRF. But this is not possible underthe least squares method because the larger the absolute value of u , the larger willbe∑u . A further justification of the least squares method lies in the fact that theestimators obtained by it have some very desirable statistical properties.

SRF: = +u

X

u

Y

X1 X2 X3 X4

u u



It is obvious from the equation (2.2) that∑u = f (β ,β ) (2.3)

That is, the sum of squared residuals is some function of theestimatorsβ and β . For any given set of data, choosing different values for β and βwill give different u′s and hence different values of ∑u . The method of leastsquares chooses β and β in such a manner that for a given sample or set of data∑u is as small as possible. In other words, for a given sample, the method of leastsquares provides us with unique estimators of β and β that give smallest possiblevalue of ∑u . This can be accomplished through the application of differentialcalculus. The process of differentiation yields the following equations for estimatingβ1 and β2.∑ = nβ + β ∑X (2.4)∑Y = β ∑X + β ∑X (2.5)

Where n is the sample size. These simultaneous equations are known as the“normal equations”.

2.1.1 Formal Derivation of the Normal Equations

We have to minimise the functionu = (Y − β − β X )With respect to β and β . The necessary condition for minimum is that the

first derivatives of the function be equal to zero. That is,

β= 0 and

β= 0

To obtain the above derivatives we apply the function of the function rule ofdifferentiation. Then, the partial derivatives with respect to β will be,∑(Y − β − β X )

β= 0

2 (Y − β − β X ) (−1) = 0Y − β − β X ) = 0

This can be written as∑Y = nβ + β ∑X (2.4)



The partial derivatives with respect to β will be,∑(Y − β − β X )β

= 02 (Y − β − β X ) (−X ) = 0(YX − β X − β X ) = 0

This can be written as∑Y = β ∑X + β ∑X (2.5)

2.1.2 Derivation of Least Squares Estimators

We have the normal equations as,∑ = nβ + β ∑X (2.4)∑Y = β ∑X + β ∑X (2.5)

Applying the Crammer’s rule for solving we have

|A| =n ∑ X∑ X ∑ X = n∑ X − (∑ X )

For solving β , forming the special matrix , |A2|

|A2| =n ∑ Y∑ X ∑ X Y = n ∑ X Y − ∑ X ∑ Y

β = n ∑ X Y − ∑ X ∑ Yn ∑ X − (∑ X )Multiplying all the elements with n n we have

β = ∑ X Y − n ∑ Xn ∑ Yn∑ X − n ∑ Xn ∑ Xnβ = ∑ X Y − n X Y∑ X − nX



β = ∑(X − X ) (Y − Y)∑(X − X )Where X and Y are the sample means of X and Y. By defining xi = (X − X )

and yi =(Y − Y), that is letting the lower case letters to denote deviations from theirmean values, we have= ∑∑ (2.6)

To derive β , let us reproduce the first normal equation, that is equation (2.4), wehave ∑ = nβ + β ∑ XDividing both sides by n, we have

∑ = β + β ∑That is, Y = β + β X = − (2.7)

The estimators β and β obtained are known as ordinary least squares estimatorsbecause they are derived from the least squares principle.

Example:

To illustrate the use of the above formulae we will estimate the supply function ofcommodity ‘z’. We have 12 pairs of observations as shown in the following table.

Number of Observations Quantity Price1 69 92 76 123 52 64 56 105 57 96 77 107 58 78 55 89 67 1210 53 611 72 1112 64 8



The following is the worksheet for the estimation of the supply function ofcommodity ‘z’.

n Quantity(Yi)

Price(Xi)

yi =( − ) xi = ( − ) xi yi xi2

1 69 9 6 0 0 0

2 76 12 13 3 39 9

3 52 6 -11 -3 33 9

4 56 10 -7 1 -7 1

5 57 9 -6 0 0 0

6 77 10 14 1 14 1

7 58 7 -5 -2 10 4

8 55 8 -8 -1 8 1

9 67 12 4 3 12 9

10 53 6 -10 -3 30 9

11 72 11 9 2 18 4

12 64 8 1 -1 -1 1

n=12 Y = 756 X = 108 ∑ y =0 x = 0 ∑ x y =156 x = 48Y = ∑n = 75612 =X = ∑n = 10812 =

β = ∑ x y∑ x = 15648 = .β = − β X = 63-(3.25)9 = 33.75

Thus, the estimated supply function is = . + .2.2 Numerical Properties of OLS Estimators

Numerical properties are those that hold as a consequence of the use of ordinaryleast squares, regardless of how the data were generated. The following arenumerical properties of OLS estimators

I. The OLS estimators are expressed solely in terms of the observable (that is,sample ) quantities of X and Y. Therefore it can be easily computed.



II. OLS estimators are point estimators. That is, given the sample, eachestimator will provide only a single (point) value of the relevant populationparameter.

III. Once the OLS estimates are obtained from the sample data, the sampleregression line can be easily obtained. The regression line thus obtained hasthe following properties:

1) It passes through the sample means of Y and X. This fact is obviousfrom equation (2.7) which can also be written as Y = β + β X. This isshown in the following figure.

2) The mean value of the estimated Y that is, Y is equal to the mean value of theactual Y. That is Y = YProof: Y = β + β XSince, β = Y − β XWe will get,Y = (Y − β X) + β XY = Y + β (X − X)Applying summation Y = nY + β (X − X)Dividing by ‘n’, we have ∑ Yn = nYn + ∑ β (X − X)nsince (X − X) = 0we will get, Y = Y

3) The mean value residuals, u is zero.

Y

X

SRF: = +

X

Y



Proof:From our earlier equation, we have2 (Y − β − β X ) (−1) = 0Or (Y − β − β X ) = 0

But since u = Y − β − β X , the preceding equation reduces to u =0.

As a result of the proceeding property, the sample regression,Y = β + β X + u (1.10)

Can be expressed in an alternative form where both Y and X are expressed asdeviations from their mean values. For thus, taking the sum on both sides of (1.10),we will get Y = β + β X + uSince, ∑ u = 0, we have∑ Y = β + β ∑ X (2.8)

Dividing equation (2.8) through by n, we obtain= β + β X (2.9)

Note that equation (2.9) is the same as equation (2.7). Subtracting (2.9) from (1.10)we obtainY − = β (X − X) + uor y = β x + u (2.10)

Where yi and xi are deviations from their respective sample mean values. Equation(2.10) is known as the deviation form. Note that the intercept term, β is absent. Yetwe can find it from our earlier equation (2.7). In the deviation form, the SRF can bewritten as y = β x (2.11)

4) The residuals are uncorrelated with the predicted Yi. That is ∑ Yu = 0.



Proof: ∑ Y u = β ∑ x u as per equation (2.11)= β ∑ x (y − β x )= β x y − β xSince β = ∑∑

We have ∑ Y u = β ∑ x − β ∑ x∑ Y u = 05) The residuals u are uncorrelated with Xi, that is ∑ u X = 0

Proof:

From, our earlier normal equations, we have2 (Y − β − β X ) (−X ) = 0That is−2 ∑ u X = 0 oru X = 0

2.3 The Classical Linear Regression Model: The Assumptions Underlying theMethod of Least Squares

If our objective is to estimate β1 and β2 only, then the method of OLS will besufficient. But our aim is not only to obtain β and β but also to draw inferencesabout the true β1 and β2. We would like to know how close β and β are to theircounterparts in the population or how close is Y to the true E(Y/Xi).The PRF: Yi =β1+β2Xi + ui shows that Yi depends on both Xi and ui. Therefore unless we arespecific about how Xi and ui are generated, there is no way we can make anystatistical inference about Yi, β1 and β2. The classical linear regression model, whichis the cornerstone of the most econometric theory, makes 10 assumptions, whichare explained below.

1. Linear Regression Model

The regression model is linear in the parameters as shown in the PRF, Yi = β1+β2Xi +ui.

2. X Values are fixed in repeated sampling

Values taken by the regressor X are considered fixed in repeated samples. Moretechnically, X is assumed to be nonstochastic. Our regression analysis isconditional regression analysis, that is, condition upon the given values of the



regressor X. The Xi’s are a set of fixed values in the hypothetical process of repeatedsampling which underlines the linear regression model. This means that, in taking alarge number of samples on Y and X, the Xi values are the same in all the samples,but the ui and Yi do differ from sample to sample.

3. Zero mean value of the disturbance ui

Given the value of X, the mean or expected value of the random disturbance termui is zero. Technically, the conditional mean value of ui is zero. Symbolically we have

E (ui/Xi) = 0 (2.12)

The equation states that the mean value of ui conditional upon the given Xi iszero. This means that for each X, u may assume various values, some greater thanzero and some smaller than zero, but if we consider all the possible value of u, forany given value of X, they would have an average value of zero. This assumptionimplies that E(Yi/Xi) = β1+β2Xi.

4. Homoscedasticity or equal variance of ui

Given the value X, the variance of ui is the same for all observations. That is, theconditional variances of ui are identical. Symbolically, we have

var(ui/Xi) =E [u − E ]var(ui/Xi)= E(ui2/Xi) because of assumption 3

var(ui/Xi)=σ2 (2.13)

var stands for variance. Equation (2.13) states that the variance of ui for each X i issome positive constant number equal to σ2. Technically it represents theassumption of Homoscedasticity or equal spread or equal variance. Stateddifferently, the equation means that the Y populations corresponding to X valueshave the same variance. The assumption also implies that the conditional variancesof Yi are also homoscedastic. That is,

var(Yi/Xi)=σ2 (2.14)

5. No autocorrelation between the disturbances

Given any two X values, Xi and Xj (i≠j), the correlation between any two ui and uj(i≠j) is zero. Symbolically,

cov (ui,uj / Xi, Xj) = E [ui-E(ui/Xi)][ uj-E(uj/Xj)]

cov (ui,uj / Xi, Xj) = E(ui/Xi)E(uj/Xj) since E(ui)=E(uj)=0

cov (ui,uj / Xi, Xj) =0 (2.15)



Where i and j are two different observations and where cov means covariance.Equation (2.15) postulates that the disturbances ui and uj are uncorrelated.Technically, this is the assumption of no serial correlation or no auto correlation.That is the covariances of any ui with any other uj are equal to zero. The valuewhich the random term assumed in one period does not depend on the valuewhich it assumed in any other period.

6. Zero covariance between ui and Xi or E(uiXi)=0

Formally, cov (ui, Xi) = E [ui-E(ui)][ Xi-E(Xi)]

cov (ui, Xi) = E [ui ( Xi-E(Xi))] since E(ui) = 0

cov (ui, Xi) = E (uiXi)-E(Xi)E(ui)

cov (ui, Xi) = E (uiXi) since E(ui)=0

cov (ui, Xi) =0, by assumption (2.16)

The above assumption states that the disturbance u and explanatory variable Xare uncorrelated. The rationale for this assumption is as follows: when we expressthe PRF as Yi = β1+β2Xi+ui, we assumed that explanatory variable X and u whichrepresent the influence of all omitted variables have separate and additive influenceon Y. But if X and u are correlated, it is impossible to assess their individual effectson Y.

7. The number of observations n must be greater than the number ofparameters to be estimated.

Alternatively, the number of observations n must be greater than the number ofexplanatory variables. For instance, if we had only one pair of observations on Y andX, there is no way to estimate the two unknowns, namely β1and β2. We need at leasttwo pairs of observations to estimate the two unknowns.

8. Variability in X values

The X values in a given sample must not all be the same. Technically, var(X)must be a finite positive number. If all the X values are identical, then X = X andthe denominator of the equation β = ∑∑ will be zero. Thus, it is impossible toestimate β2 and therefore β1. Variation in both Y and X is essential for regressionanalysis. In short, variables must vary.

9. The regression model is correctly specified

Alternatively, there is no specification bias or error in the model used inempirical analysis. An econometric investigation begins with the specification of theeconometric model underlying the phenomenon of interest. Some importantquestions that arise in the specification of the model include the following: (a) whatvariables should be included in the model? (b) What is the functional form of the



model? Is it linear in the parameters, the variables or both? (c) What are theprobabilistic assumptions made about the Yi, the Xi and the ui entering the model?

10. There is no perfect multicollenearity

That is, there is no perfect linear relationship among the explanatory variables. Ifthere is more than one explanatory variable in the relationship it is assumed thatthey are not perfectly correlated with each other. Indeed, the regressors should noteven be strongly correlated; they should not be highly multicollinear.

2.4 Properties of Least Squares Estimators: The Gauss Markov Theorem

As noted earlier, given the assumptions of classical linear regression model,the least squares estimates possess some ideal or optimum properties. Theseproperties are contained in the well known Gauss Markov theorem. To understandthis theorem, first we need to consider the best linear unbiasedness properties of anestimator, which is explained below.

An estimator is the best when it has the smallest variance as compared to anyother estimators obtained from other econometric methods. Symbolically assumedthat θ has two estimates, namely θ and θ . θ is the best if,E [θ − E θ ] < E [θ − E θ ]Or var(θ ) < var θ )

An estimator is linear if it is linear function of sample observations. That is, itis determined by linear combination of sample data. Given the sample observations,Y1, Y2, Y3....Yn, a linear estimator will have the form

k1Y1+ k2Y2+ k3Y3+...+kn Yn

Where ki’s are some constants. The bias of an estimator is defined as thedifference between its expected value and the true parameter.

Bias = E (θ) − θ

The estimator is unbiased if its bias is zero, that is, E (θ) = θ this means thatthe unbiased estimator converges to the true value of the parameter as the numberof samples increases. An unbiased estimator gives ‘on the average’ the true value ofthe parameter.

An estimator θ is best linear unbiased estimator (BLUE) if it is linear,unbiased and has the smallest variance as compared to all other linear unbiasedestimators of the true θ. The BLU estimate has the minimum variance within a classof linear unbiased estimators of the true θ.

An estimator, say the ordinary least squares estimator β is said to be BLUEof β2 if the following hold.



1) It is linear, that is, a linear function of a random variable, such as thedependent variable Y in the regression model.

2) It is unbiased, that is, its average or expected value, E(β ) is equal to thetrue value β2

3) It has the minimum variance in the class of all such linear unbiasedestimators; an unbiased estimator with the least variance is known as anefficient estimator.

In the regression context it can be proved that the OLS estimators are BLUE.This is the essence of the Gauss Markov Theorem, which can be stated as follows:Given the assumptions of the classical linear regression model, the least squaresestimators, in the class of unbiased linear estimators, have the minimum variance,that is, they are BLUE.

2.4.1 Proof of the Gauss Markov Theorem

1) Property of Linearity

The least squares estimates β and β are linear functions of the observedsample values of Yi. Given that Xi’s appear always in the same values inhypothetical repeated sampling process, it can be shown that the least squaresestimates depend on the values of Yi only, that is, β = f(Y )and β = f (Y )

We have β = ∑∑β = ∑ x (Y − Y)∑ xβ = ∑ x Y − Y ∑ x )∑ x

β = ∑∑ since ∑ x = 0Or

β = ∑ k Y where ki = ∑The values of X’s are fixed in hypothetical repeated sampling. Hence the ki’s arefixed constants from sample to sample and may be regarded as constant weightsassigned to the individual values of Y. We write,

β = k Y = k Y + k Y +⋯+ k Y = f (Y)The estimate β is a linear function of Y’s, that is, a linear combination of the valuesof dependent variable.



Similarly,β = Y − β Xβ = Y − X k Y

β = ∑ Yn − X k Yβ = 1n − Xk Y

Since X values are constants and X and k are fixed constants from sample tosample. Thus, β depends only on the values of Yi, that is, β is a linear function ofthe sample values of Y.

2) Property of Unbiasedness

The property of unbiasedness of β and β can be proved if we can establishE(β ) = β and E(β ) = β . The meaning of this property is that the estimatesconverge to the true value of the parameters as we increase the number ofhypothetical sample.

We have β = ∑ k Yβ = k (β + β X + u )

β = β k +β k X + k u )We can prove that ∑ k = 0 and ∑ k X = 1

That is, ∑ k = ∑∑ki = ∑( )∑ =0

similarly, ∑ k X = ∑∑ Xk X = ∑(X − X)X∑ x

k X = ∑ X − X ∑ X∑ xSince ∑ x = ∑ X − X ∑ X we have



k X = ∑ X − X ∑ X∑ X − X ∑ X = 1Therefore we have β = β + ∑ k u

β = β + ∑ x u∑ xTaking the expected values, we haveE β = E β + ∑∑Since E β = β and E(u ) = 0 we will get E β = β . Therefore, β is unbiasedestimator of β .

Similarly we can prove that β is an unbiased estimator of β .

We have

β = 1n − Xk YTaking the expected values we have

E (β ) = 1n − Xk E(Y )E (β ) = 1n − Xk (β + β X )

E (β ) = βn − Xk β + β Xn − Xk β XE (β ) = nβn + X β k + β

Xn − X β k XSince ∑ k = 0 and ∑ k X = 1 , we getE (β ) = β + β X − β XE (β ) = β

Thus, β is an unbiased estimator of β .

3) Property of Minimum Variance



In this section we will prove the Gauss Markov theorem which states that theleast squares estimates have the smallest variance as compared with any otherlinear unbiased estimators.

It can be proved that var (β ) = E[β − E(β )]Since β = ∑ k Y we get var β = var ( k Y )

var β = k var (Y )since k is constantwe have var (Y ) = σ and k = ∑

so, var (β ) = σ ∑(∑ ) ( ) = ∑Similarly, it can be proved thatvar (β ) = E[β − E(β )]

var (β ) = var 1n − Xk Yvar β = 1n − Xk var( Y )since var (Y ) = σ we havevar β = σ ∑ − 2 + X k

var β = σ − ∑ + X ∑ ksince ∑ k = 0 we have var β = σ + X ∑ kThat is, var β = σ + X ∑Now taking LCM, we have



var β = σ ∑ ∑since ∑ xi

2 = ∑ X2 − nX2,

var β1 = σ2 ∑ X2 nX2 nX2

n ∑ xi2

that is, ()̂ = ∑∑Thus, we obtained the variance of both ordinary least squares estimators.

Now we need to prove that any other linear unbiased estimate of the true parameterfor example,β2, obtained from any other econometric method has a bigger variancethan the OLS estimator β2.

That is, var(β2) < var(β2)

The new estimator β2 is by assumption a linear combination of Yi’s. As aweighted sum of the sample values of Yi, the weights being different from the weightki (= xi∑ xi2

) of the OLS estimates. For example, let us assume

β2 = ciYi

Where ci = ki + di

‘ki’ being the weights defined earlier for OLS estimates and ‘di’, an arbitrary set ofweights.

The new estimator β2is also assumed to be unbiased estimator of β2. That is,

E(β2) = β2

We have

β2 = ciYi

β2 = ci (β1 + β2Xi + ui)

β2 = β1 ci+β2 ci Xi + ci ui

Taking expected value on both sides,(β2) = β1 ci+β2 ci Xi + ci ui



If we assume, ∑ ci = o, ∑ ci Xi = 1 and ∑ ci ui = 0, then(β ) = βNow, the variance of the new estimator β isvar β = var ( c Y )

var β = c var (Y )since c is constantvar β = σ c

var β = σ k + dvar β = σ k + σ d

But σ ∑ k = variance of β and therefore,var β = var β + σ dGiven that di’s are defined as an arbitrary constant weights not all of them are zero,the second term is positive, that is, σ ∑ d > 0.

Therefore, var β > βThus, in the group of linear unbiased estimates of true β , the least squares

estimate has the minimum variance. In the similar way, we can prove that the leastsquares intercept coefficientβ has the minimum variance.

2.5 The Coefficient of Determination: A Measure of Goodness of Fit

After the estimation of the parameters and the determination of the leastsquares regression line, we need to know how ‘good’ is the fit of this line to thesample observations of Y and X. That is, we need to measure the dispersion ofobservations around the regression line. This knowledge is essential, the closer theobservations to the line, the better the goodness of fit and the better is theexplanation of the variations of Y by the changes in the explanatory variables.

If all the observations lie on the sample regression line, we would obtain aperfect fit. But this is rarely the case. Generally, there will be some positive u andnegative u . What we hope for is that these residuals around the regression line areas small as possible. The square of correlation coefficient known as coefficient ofdetermination r2, in two variable case, and R2 in multiple regression is a summary



measure that tells how well the sample regression line fits the data. As a measure ofgoodness of fit, r2 shows the percentage of the total variation of the dependentvariable Y that is explained by the independent variable X.

To compute the coefficient of determination (r2) we proceed as follows. Let∑ y = ∑(Y − Y) → Total variation of the actual Y values about their sample mean,which may be called Total Sum of Squares (TSS). We compute total variation of thedependent variable by comparing each value of Y to the mean value Y and adding allthe resulting deviations. Note that in order to find the TSS, we square the simpledeviations, since by definition the sum of simple deviations of any variable aroundits means is identically equal to zero.∑ y = ∑ Y − Y → Variations of the estimated Y values about their means, whichmay be called the sum of squares due to regression, or explained by regression orsimply the Explained Sum of Squares (ESS). This is the part of the total variationsof Yi which is explained by the regression line. Thus, ESS is the total explained bythe regression line variation of the dependent variable.∑ u = ∑ Y − Y →Variations of the dependent variable which is not explained bythe dependent variable and is not explained by the regression line and is attributedto the existence of the disturbance term. The Residual Sum of Squares (RSS) is thesum of the squared residuals that gives the total unexplained variation of thedependent variable Y around its mean.

In summaryy = Y − Y → deviations of Y from its meany = Y − Y → deviations of regressed value Y from the meanu = Y − Y → deviations of Y from the regression lineThus we have Y = Y + uTaking the sum and squaring, ∑ Y = ∑ Y + uY = Y + 2 Y u + uSince 2 ∑ Y ∑ u = 0, we get Y = Y + uThat is TSS = ESS + RSS (2.17)

or



[ ] = [ ] + [ ( ) ]The above equation shows that the total variation in the observed Y values

about their mean value can be partitioned into two parts; one attributable to theregression line and the other to random forces because not all actual Y observationlie on the fitted line.

Now dividing equation (2.17) by TSS on both sides, we obtain,

1 = ESSTSS + RSSTSS1 = ∑ Y − Y∑(Y − Y) + ∑ Y − Y∑(Y − Y)

We now define r2 as = ∑∑( ) =Or alternatively

= − ∑ −∑( − ) = −The quantity r2 thus defined is known as the coefficient of determination and is

the most commonly used measure of the goodness of fit of a regression line.Verbally, r2 measures the proportion or percentage of the total variation in Yexplained by the regression model. For example, if r . = 0.90, this means that theregression line explains 90 per cent of the total variation of Y values around theirmean. The remaining 10 per cent of the total variation in Y is unaccounted for bythe regression line and is attributed to the factors included in the disturbancevariable u. The following are two properties of the r2.

1) r2 is a non negative quantity.

2) The limits of r2 are 0 ≤ r ≤ 1 . An r2 of 1 means perfect fit, that is, Y =Y for each i. On the other hand, an r2 of zero means that there is no relationshipbetween the regressand and the regressor whatsoever (β = 0). In such caseY = β = Y. That is, the best prediction of any Y value is simply its mean value.Therefore, in this situation, the regression line will be horizontal to the X axis.That is, as noted above, ∑ u = ∑ Y − Y is the proportion of the unexplainedvariation of the Y around their mean. In other words, the total variation of Y isexplained completely by the estimated regression line. Consequently, there willbe no unexplained variation (RSS=0) and hence r2=1. On the other hand, if theregression line explains only part of the variation in Y, there will be someunexplained variation, (RSS>0) and therefore r2will be smaller than 1. Finally, ifthe regression line does not explain any part of the variation in Y, ∑ Y − Y =∑(Y − Y) and hence r2=0.



MODULE III

THE CLASSICAL NORMAL LINEAR REGRESSION MODEL

3.1 The Probability Distribution of Disturbances

For the application of the method of ordinary least squares (OLS) to theclassical linear regression model, we did not make any assumptions about theprobability distribution of the disturbances ui. The only assumption made about ui

were that they had zero expectations, were uncorrelated and had constant variance.With these assumptions we saw that the OLS estimators satisfy several desirablestatistical properties, such as unbiasedness and minimum variance. If our objectiveis point estimation only, the OLS method will be sufficient. But point estimation isonly one aspect of statistical inference, the other being hypothesis testing.

Thus, our interest is not only in obtaining, say β but also using it to makestatements or inferences about true β . That is, the goal is not merely to obtain theSample Regression Function (SRF) but to use it to draw inferences about thePopulation Regression Function (PRF). Since our objective is estimation as well ashypothesis testing, we need to specify the probability distribution of disturbances u i.In the module II we proved that the OLS estimators of β and β are both linearfunctions of ui, which is random by assumption. Therefore, the sampling orprobability distribution of OLS estimators will depend upon the assumptions madeabout the probability distribution of ui. Since the probability distribution of theseestimators are necessary to draw inferences about their population values, thenature of probability distribution of ui assumes an extremely important role inhypothesis testing.

But since the method of OLS does not make any assumptions about theprobabilistic nature of ui, it is of little help for the purpose of drawing inferencesabout the PRF from SRF. But this can solved if we assume that the u’s follow someprobability distribution. In the regression context, it is usually assumed that the u’sfollow the normal distribution.

3.2 The Normality Assumption

The classical normal linear regression model assumes that each u i isdistributed normally withMean: E(u ) = 0 (3.1)Variance: E(u ) = σ (3.2)cov u , u = 0 i ≠ j (3.3)

These assumptions may also be compactly stated asu ~N(0, σ ) (3.4)



Where ~ means “distributed as” and where N stands for the “normal distribution”.The terms in the parentheses represents the two parameters of the normaldistribution, namely, the mean and the variance. u is normally distributed aroundzero mean and a constant finite variance σ . For each ui, there is a distribution ofthe type of (3.4).The meaning is that small values of u have a higher probability tobe observed than large values. Extreme values of u are more and more unlikely themore extreme we get.

For two normally distributed variables zero covariance or correlation meansindependence of the two variables. Therefore, with the normality assumption,equation (3.3) means that u and u are not only uncorrelated but also independentlydistributed. Therefore, we can write equation (3.4) as,u ~NID(0, σ ) (3.5)

Where NID stands for normally and independently distributed. There are severalreasons for the use of normality assumption, which are summarised below.

1) As noted earlier, ui represents the combined influence of a large number ofindependent variables that are not explicitly introduced in the regressionmodel. We hope that the influence of these omitted or neglected variables issmall or at best random. By the central limit theorem of statistics it can beshown that if there are large number of independent and identicallydistributed random variables, then, with few exceptions, the distribution oftheir sum tends to a normal distribution as the number of such variablesincreases indefinitely. It is this central limit theorem that provided atheoretical justification for the assumption of normality of u i.

2) A variant of central limit theorem states that even if the number of variables isnot very large or if these variables are not strictly independent, their sum maystill be normally distributed.

3) With the normality assumption, the probability distribution of the OLSestimators can be easily derived because one property of the normaldistribution is that any linear function of normally distributed variables isitself normally distributed.

4) The normal distribution is a comparatively simple distribution involving onlytwo parameters, namely mean and variance.

5) The assumption of normality is necessary for conducting the statistical testsof significance of the parameter estimates and for constructing confidenceintervals. If this assumption is violated, the estimates of β and β are stillunbiased and best, but we cannot assess their statistical reliability by theclassical test of significance, because the latter are based on normaldistribution.

3.3 Properties of OLS Estimators under the Normality Assumption

With the assumptions of normality the OLS estimators have the following properties

1. They are unbiased.



2. They have the minimum variance. Combined with property 1, this means thatthey are minimum-variance unbiased or efficient estimators.

3. As the sample size increases indefinitely, the estimators converge to theirpopulation values. That is, they are consistent.

4. β is normally distributed with E β = βvar β = σ = ∑ Xn ∑ x σor more compactly,β ~N(β , σ ) (3.6)Then by the properties of normal distribution, the variable Z, which is defined

as Z = follows the standardised normal distribution, that is normal

distribution with zero mean and unit variance or Z~N(0,1)5. β is normally distributed with E β = βvar β = σ = σ∑ xor more compactly,β ~N(β , σ ) (3.7)

And Z = follows the standardised normal distribution.

6. ( ) = is distributed as X2 (chi- square) distribution with n-2 degrees offreedom.

7. β , β are distributed independently of .8. β and β have the minimum variance in the entire class of unbiased

estimators, whether linear or not. Therefore, we can say that the least squaresestimates are best unbiased estimators (BUE).

3.4 The Method of Maximum Likelihood

Like the OLS, the method of Maximum Likelihood (ML) is a method forobtaining estimates of the parameters of population from the random sample. Themethod was developed by R A Fisher and is an important procedure of estimation ineconometrics. In the ML, we take a fixed random sample. This sample might havebeen generated by many different normal populations, each having its ownparameters of mean and variance. Which of these possible alternative populations ismost probable to have given rise to the observed n sample values? To answer thisquestion we must estimate the joint probability of obtaining all the n values for eachpossible normal population. Then choose the population whose parametersmaximise the joint probability of observed sample values.

The ML method chooses among all possible estimates of the parameters, thosevalues, which make the probability of obtaining the observed sample as large as



possible. The function which defines the joint (total) probability of any sample beingobserved is called the likelihood function of the variable X.

The general expression of the likelihood function isL(X , X , … , X ; θ , θ , … θ )Where θ , θ , … θ denote the parameters of the function which we want to estimate.In the case of normal distribution of X, the likelihood function in its general form isL(X , X , … , X ; μ, σ )The ML method consists of maximisation of the likelihood function. Following thegeneral condition of maximisation, the maximum value of the function is that valuewhere the first derivative of the function with respect to its parameters is equal tozero. The estimated value of the parameters are the maximum likelihood estimatesof population parameters. The various stages of ML method are outlined below.

1) Form the likelihood function, which gives the total probability of theparticular sample values being observed.

2) Takes the partial derivatives of the likelihood function with respect to theparameters which we want to estimate and set them equal to zero.

3) Solve the equations of the partial derivatives for the unknown parameters toobtain their maximum likelihood estimates.

3.5 Maximum Likelihood Estimation of Two Variable Regression Model

We already established that in the two variable regression model,

Yi = β1+β2Xi + ui

The Yi are normally distributed with mean= β1+β2Xi and the variance σ2. As aresult, the joint probability density function, given the man and the variance, canbe written as

f (Y1, Y2,...Yn/ β1+β2Xi, σ2)

as Y’s are independent, this probability density function can be written as aproduct of n individual P D F’s as

f (Y1, Y2,...Yn/ β1+β2Xi, σ2) = f (Y1 / β1+β2Xi, σ2) f (Y2 / β1+β2Xi, σ2)..... f (Yn /β1+β2Xi, σ2) (3.8)

where ,

f (Yi) = f(Y ) = √ e (3.9)

which is the density function of a normally distributed variance with given meanand variance. Substituting equation (3.9) in (3.8), we get



f (Y1, Y2,...Yn/ β1+β2Xi, σ2) = ( ) e ∑(3.10)

if Y1, Y2...Yn are known or given, but β1,β2and σ2 are not known, the function in(3.10) is called a likelihood function denoted by LF (β1,β2, σ2) and written as,

LF (β1,β2, σ2)= ( ) e ∑(3.11)

In the method of maximum likelihood, we estimate the unknown parameters insuch a manner that the probability of observing the given Y’s is as high aspossible. Therefore, we have to find the maximum of equation (3.11). This is astraight forward exercise of differential calculus as shown below.

For differentiation, it is easier to express equation (3.11) in log form as,

Log LF = −nlogσ − n2 log(2π) − 12 Y − β − β XσLog LF = − n2 logσ − n2 log(2π) − 12 Y − β − β Xσ

Or

Log LF = − n2 logσ − n2 log(2π) − 12σ Y − β − β XDifferentiating with respect to β and setting equal to zero,dLog LFdβ → − 12σ 2 Y − β − β X (−1) = 0

Y − β − β X = 0Y = nβ + β Xwhich is the same as the normal equation of the least square theory.

Similarly, dLog LFdβ → − 12σ 2 Y − β − β X (−X ) = 0Y X = β X + β X



Which is same as the second normal equation of the least squares theory.Therefore, the ML estimators, β′s are the same as OLS estimators β′s.

3.6 Two variable Regression Model: Interval Estimation and HypothesisTesting

3.6.1 Interval Estimation: Some Basic Ideas

In statistics, the reliability of point estimator is measured by its standard error.Therefore, instead of relying on the point estimate alone, we may construct aninterval around the point estimator, say within two or more standard errors oneither side of the point estimator, such that this interval has, say 95 % probabilityof including the true parameter value. This is roughly the idea behind intervalestimator.

Assume that we want to know how close is β to β . For this purpose, we try tofind out two positive numbers, θ and α, the latter lying between 0 and 1, such thatthe probability that the random interval (β − θ, β + θ ) contain the true β is 1-α.Symbolically,Pr (β − θ ≤ β ≤ β + θ = 1 − α (3.12)

Such interval is known as a confidence interval. 1-α is known as confidencecoefficient and α (0< α<1) is known as the significance level. It is also probability ofcommitting Type I error. A Type I error consist in rejecting the true hypotheses,where as Type II error consist in accepting the false hypotheses. In other words,whenever we happen to incorrectly reject the null hypothesis, we make error of TypeI and whenever we happen to incorrectly accept the null hypothesis, we make errorof Type II. As a simile, Type I error is ‘convicting an innocent’, while that of Type IIerror is ‘letting go a guilty’. Thus, the significance level (α) is the probability ofmaking the wrong decision. That is, the probability of rejecting the hypothesis whenit is actually true or the probability of committing Type I error. We choose a level ofsignificance for deciding whether to accept or reject our hypothesis.

The probability of Type II error may be denoted as β. Then Pr(1-β)= P, isknown as the power of the test, that is probability of rejecting a false hypothesis.“Significance level minimum and power maximum” is the procedure usuallyfollowed. The end point of the confidence interval are known as confidence limits.

The equation (3.12) shows that an interval estimator, in contrast to the pointestimator, is an interval constructed in such a manner that it has a specifiedprobability 1 − α of including within its limits the true value of the parameter. Forexample, if α= 0.05 or 5%, then the probability that the random interval includesthe true β is 0.95 or 95%. Thus, the interval estimator gives a range of valueswithin which the true β may lie. Generally in practice a level of significance (α) of0.05 or 0.01 is customary.



3.6.2 Confidence Interval for Regression Coefficients

(1) If is known β ~N(β , σ )Z = (3.13)

P Z > Z∝/ = ∝(3.14)P Z < −Z∝/ = ∝(3.15)

That is, P −Z∝ < < Z∝ = 1 − α (3.16)

Substituting equation (3.13) in (3.16), we getP −Z∝ < < Z∝ = 1 − α (3.17)

To remove σ from the denominator, we multiply all the elements by σ andsubtracting β from each element, we will get− ∝σ − β < −β < ∝σ − β = 1 − (3.18)

Rearranging, we will have[β − ∝σ < β < β + ∝σ ] = 1 − (3.19)

Therefore, [β − ∝σ , β + ∝σ ] is the 100(1-α) % confidence interval for β .

Substituting σ as it is known, we get,

β − ∝ ∑∑ , β + ∝ ∑∑ (3.20)

(2) If is unknown

If ~ (0, 1) and Z ~X with n degrees of freedom, thenZZn ~t distribution with (n − 2)degree of freedom



= ∑∑(3.21)

That is, = ∑∑(3.22)

or t = (3.23)

We proceed on the similar way as we did earlier, we have− ∝ < < ∝ = 1 − (3.24)

Substituting equation (3.23) in (3.24), we getP −t∝ < < t∝ = 1 − α (3.25)

Multiplying all the elements with seβ and subtracting β from (3.25), we haveP −t∝seβ − β < −β < t∝seβ − β = 1 − α (3.26)

Rearranging, we will haveP[β − t∝seβ < β < β + t∝seβ ] = 1 − α (3.27)

Therefore, β − ∝seβ , β + ∝seβ or β ± ∝seβ is the 100(1-α) % confidence

interval for β .

By following the same procedure, we can get the confidence interval for β also.P[β − t∝seβ < β < β + t∝seβ ] = 1 − α (3.28)

Therefore, β − ∝seβ , β + ∝seβ or β ± ∝seβ is the 100(1-α) % confidence

interval for β .

An important feature of the confidence interval given in (3.27) and (3.28) maybe noted. In both the cases, the width of the confidence interval is proportional tothe standard error of the estimator. That is, the larger the standard error, the larger



is the width of the confidence interval. Put differently, the larger is the standarderror of the estimator, the greater is the uncertainty of estimating the true value ofthe unknown parameter. Thus, the standard error of an estimator is often describedas a measure of the precision of the estimator. That is, how precisely the estimatormeasures the true population value.

3.6.3 Confidence Interval for σ2

As pointed out in the properties of OLS estimators under the normalityassumption (Property 6), the variable

X = (n − 2)σσ ~X with n − 2 degrees of freedomTherefore, we can use X2 distribution to establish confidence interval σ .Pr X ∝/ ≤ X ≤ X ∝/ = 1−∝ (3.29)

Substituting for X2, we havePr X ∝/ ≤ ( ) ≤ X ∝/ = 1−∝ (3.30)

Taking the reciprocal of equation (3.30) and rearranging we havePr ∝/ ≤ ( ) ≤ ∝/ = 1−∝ (3.31)

Multiplying all the elements of equation (3.31) by (n − 2)σ we will get,Pr ( )∝/ ≤ σ ≤ ( )∝/ = 1−∝ (3.31)

Therefore,( )∝/ , ( )∝/ gives the 100(1-∝) % confidence interval for σ

3.6.4 Hypothesis Testing

Very often in practice, we have to make decisions about population on thebasis of sample information. In attempting to arrive at decisions, we makeassumptions about population. Such assumptions about populations are calledstatistical hypothesis and in general, are assumptions about the form of distributionof the population or the values of the parameter involved.

If a hypothesis determines the population completely, that is, if it specifies theform of the distribution of the population and the values of all parameters involved,it is called simple hypothesis; otherwise it is called composite hypotheses. Testing ofhypothesis is a procedure by which we accept or reject a hypothesis on the basis ofsample taken from population.



The hypothesis which is tested for possible rejection under the assumptionthat it is true is called null hypothesis and is denoted by Ho. Rejection of Honaturally results in acceptance of some other hypothesis which is called alternativehypothesis and is denoted by H1. The testable hypothesis is called the nullhypothesis. The term ‘null’ refers to the idea that there is no difference between thetrue value and the value we hypothesise. Since null hypothesis is a testablehypothesis there must also exists a counter proposition to it in order to test thehypothesised proposition. This counter proposition is called alternative hypothesis.In other words, the stated hypothesis is known null hypothesis. The null hypothesisis usually tested against alternative hypothesis which is also known as maintainedhypothesis.

The theory of hypothesis testing is concerned with developing rules orprocedures for deciding whether to reject or not reject the null hypothesis. There aretwo mutually complementary approaches for devising such rules, namely confidenceinterval and test of significance. Both these approaches predicate that the variable(statistic or estimator) under consideration has some probability distribution andthat hypothesis tasting involves making statement or assertions about the values ofthe parameters of such distribution.

In the confidence interval approach of hypothesis testing we construct a100(1-α) % confidence interval for the estimator, say β2. If β2 under H0 falls withinthis confidence interval, we accept H0. But if it falls outside this interval we rejectH0. When we reject the null hypothesis, we say that our finding is statisticallysignificant. On the other hand, when we do not reject the null hypothesis, we saythat our finding is not statistically significant.

An alternative but complementary approach to the confidence interval methodof testing statistical hypothesis is the test of significance approach developed by R AFisher, Neyman and Pearson. Broadly speaking, a test of significance is a procedureby which sample results are used to verify the truth or falsity of a null hypothesis.The key idea of test of significance is that of a test statistic (estimator) and thesampling distribution of such a statistic under the null hypothesis. The decision toaccept or reject H0 is made on the basis of the value of the test statistic obtainedfrom the data at hand. In the language of significance tests, a statistic is said to bestatistically significant if the value of the test statistic lies in the regions of rejection(H0) or the critical region. In this case, the null hypothesis is rejected. By the sametoken, a test is said to be statistically insignificant if the value of the test statisticlies in the region of acceptance (of the null hypothesis). In this situation, the nullhypothesis is not rejected.

Thus, the first step in hypothesis testing is that of formulation of the nullhypothesis and its alternative. The next step consists of devising a criterion of testthat would enable us to decide whether the null hypothesis is to be rejected or not.For this purpose the whole set of values of the population is divided into tworegions, namely the acceptance region and the rejection region. The acceptanceregion includes the values of the population which have a high probability of beingobserved and the rejection region or critical region includes those values which arehighly unlikely to be observed. Then the test is performed with reference to test



statistic. The empirical tests that are used for testing the hypothesis are called testsof significance. If the value of the test statistic falls in the critical region, the nullhypothesis is rejected; while if the value of test statistic falls in the acceptanceregion, the null hypothesis is not rejected.

The following tables summarises the test of significance approach tohypothesis testing.

(1) Normal Test of Significance- σ is known

Suppose β ~N(β , σ )Then test statistic used isZ =Decision Rules – Z Test

Type ofHypothesis

H0: The NullHypothesis

H1: The AlternativeHypothesis

Critical Region

Two Tail

Right Tail

Left Tail

β = β ∗β ≤ β ∗β ≥ β ∗β ≠ β ∗β > β ∗β < β ∗

|Z| > Z∝/Z > Z∝Z < −Z∝Here, β ∗ is the hypothesised numerical value of β . |Z| means absolute

value of Z. Z∝ or Z∝/ means the critical Z value at the α or α/2 level of significance.The same procedure holds to test hypothesis about β .

(2) t-Test of Significance- σ is unknown

When σ is unknown, we use t- distribution we use t distribution as the teststatistic. Under Normality assumption the variable,

t = β − βseβt = β − β ∑ xσ



Decision Rules – t Test

Type ofHypothesis

H0: The NullHypothesis


Critical Region

Two Tail

Right Tail

Left Tail

β = β ∗β ≤ β ∗β ≥ β ∗β ≠ β ∗β > β ∗β < β ∗

|t| > t∝/t > t∝t < −t∝The same procedure holds to test hypothesis about β .

(3) Testing the Significance of σ2: the Chi- Square test

For testing the significance of σ2, as noted earlier, we use the chi square test.

X = (n − 2)σσ ~X with n − 2 degrees of freedomH0: The NullHypothesis


Critical Region

σ = σσ ≤ σσ ≥ σσ ≠ σσ > σσ < σ

X > X ∝/X > X ∝X < X ∝σ is the value of σ under the null hypothesis.

3.7 Regression Analysis and Analysis of Variance

In this section, we analyse regression analysis from the point of view of theanalysis of variance and try to develop a complementary way of looking at thestatistical inference problem. We have developed in the previous module,y = y + u = β x + u

That is, TSS= ESS+RSS. In other words, the total sum of squares composed ofexplained sum of squares and the residual sum squares. A study of thesecomponents of TSS is known as the analysis of variance (ANOVA) from theregression view point. ANOVA is a statistical method developed by R A Fisher for theanalysis of experimental data.



Associated with any sum of squares is its degree of freedom (df), that is, thenumber of independent observations on which it s based. TSS has n-1 df becausewe lose 1 df in computing the sample mean . RSS has n-2 df and ESS has 1 dfwhich follows from the fact that = β ∑ x is a function of β only as∑ x is known . Both case is true only in two variable regression model. The followingtable presents the various sum of squares and their associated df which is thestandard form of the AOV table, sometimes also called the ANOVA table.

Source ofvariation

Sum of Squares(SS)

Degree ofFreedom

Mean Sum ofSquares (MSS)

Due t0 regression(ESS)

Due to residuals(RSS)

y = β xu

1

n-2

β x∑ u− 2 =

TSS y n-1

In table the MSS obtained by dividing SS by their df. From the table let us consider,F == β ∑ x∑ u− 2= ∑

(3.32)

If we assume that the disturbances ui are normally distributed and H0:β2=0, it canbe shown that the F of equation (3.32) follows the F distribution with 1 and n-2 df.

3.8 Application of Regression Analysis: The Problem of Prediction

On the basis of sample data, we obtained the following sample regressionY = β + β XWhere Y is the estimator of true E(Yi). We want use it to predict or forecast Ycorresponding to some given level of X. There are two kinds of predictions, namely

1) Prediction of the conditional mean value of Y corresponding to chosen X, sayX0. That is, the point on the population regression line itself. This predictionis known as mean prediction.

2) Prediction of an individual Y value corresponding to X0, which is known asindividual prediction.



Mean Prediction

Given Xi= X0, the mean prediction E(Y0/X0) is given by

E(Y0/ X0) = β1+β2Xi (3.33)

We estimate from equation (3.33)Y = β + β X (3.34)

Taking the expectation of equation (3.34) given X0, we getY = (β ) + ( β )X (3.35)E Y = β + β X because β and β are unbiased estimators.

That is, E Y = E(Y / X ) = β + β X (3.36)

That is Y is an unbiased predictor of E(Y / X )var Y = var(β + β X ) (3.37)

Now using the property that var (a+b)=var (a) +var (b)+ 2 cov (a,b), we obtainvar Y = var β + var β X + 2 cov(β , β )X (3.38)

Using the formulas for variances and covariances of β and β we get,

var Y = σ 1n + X∑ x + X σ∑ x + 2X Xσ∑ xRearranging and manipulating the terms we obtain,var Y = σ + ( )∑ (3.39)

by replacing the unknown σ by its estimator σ , it follows that the variable,t = ( )( ) (3.40)

Follows t distribution with n-2 df. Therefore t distribution can be used to deriveconfidence intervals for the true E(Y / X ) and test hypotheses about it in the usualmanner. That is,P[β + β X − t∝se(Y ) < β + β X < β + β X + t∝se(Y )] = 1 − α (3.41)

Where se Y is obtained from equation (3.39)



Individual Prediction

We want to predict an individual Y corresponding to a given X value, say X . That iswe want to obtain,

Y0 = β1+β2Xi+ u0 (3.42)

We predict this as Y = β + β X . The prediction error Y − Y isY − Y = β + β X + u − β + β XThat is, Y − Y = (β − β ) + β − β X + u (3.43)

Taking the expectations on both sides of equation (3.43), we haveE(Y − Y ) = E(β − β ) + E β − β X − E (u )E Y − Y = 0since β and β are unbiased, X is a number and (u ) = 0 by assumption.var Y = E Y − Y = E (β − β ) + β − β X + u (3.44)var Y = E (β − β ) + X β − β + u + 2(β − β )X β − β + 2(β − β )u +2β2−β2u0 (3.45)var Y = var β + X var (β )+ var (u )+2X cov(β , β ) + var(u ) (3.46)

Using the variance and covariance formula for β and β and noting that var(u ) = σ ,and slightly rearranging the equation (3.46), we havevar Y = σ 1 + + ( )∑ (3.47)

further , it can be shown that Y follows the normal distribution. Substituting σ forthe unknown σ , it follows that,t = ( ) (3.48)

Also follows the t distribution. Therefore, t distribution can be used to drawinferences about the true Y .



MODULE IV

EXTENSION OF TWO VARIABLE REGRESSION MODEL

4.1 Introduction

Some aspects of linear regression analysis can be easily introduced within theframe work of the two variable linear regression models that we have beendiscussing so far. First we consider the case of regression through the origin, ie, asituation where the intercept term, β1, is absent from the model. Then we considerthe question of the functional form of the linear regression model. Here we considerthe models that are linear in parameters but not in variables. Finally we considerthe question of unit of measurement, i.e, how the X and Y variables are measuredand whether a change in the units of measurement affects the regression results.

4.2 Regression through origin

There are occasions when two variables PRF assume the following form:

` Yi = β2Xi + ui (4.1)

In this model the intercept term is absent or zero, hence regression throughorigin. How do we estimate models like (4.1) and what special problems do theypose? To answer these questions, let us first write SRF of (4.1) namely:Y = β X + u (4.2)

Now applying the ordinary least square (OLS) method to (4.2), we obtain thefollowing formulae for the β , and its variance. We want to minimize∑ u = ∑(Y − β Xi) (4.3)

With respect to β .

Differentiating (4.5) with respect to β , we obtain∑= 2∑ Y − β Xi (−Xi) (4.4)

Setting (4.4) equal to zero and simplifying, we getβ = ∑∑ (4.5)

Now substituting the PRF: Yi = β2Xi + ui in to this equation, we obtainβ = ∑ ( )∑ (4.6)



= β2 + ∑∑Note: E (β ) = β2. Therefore,

E ( β -β2)2 = E ∑∑ (4.7)

Expand the right hand side of (4.7) and noting that the Xi is nonstochastic and theui are homoscedastic and uncorrelated, we obtain:

Var (β ) = E (β -β2)2 = ∑ (4.8)

Where σ is estimated by

2 =∑

(4.9)

It is interesting to compare these formulas with those obtained when theintercept term is included in the model.β = ∑∑ (4.10)

Var (β ) = ∑ (4.11)

2 =∑

(4.12)

The difference between two sets of formulae should be obvious. In the modelwith intercept term is absent, we use raw sums of squares and cross product but inthe intercept present model, we use adjusted (from mean) sum of squares and crossproducts. Second, the degrees of freedom for computing 2 is (n-1) in the first caseand (n-2) in the second case.

Although the zero intercept models may be appropriate on occasions, thereare some features of this model that need to be noted. First, ∑ which is alwayszero for the model with the intercept term need not be zero when that term isabsent. In short ∑ need not be zero for the regression through the origin.

Suppose we want to impose conditions that ∑ = 0. In that case we have

∑Y = β ∑X + ∑u= β ∑X (4.13)

This expression then gives



β = ∑∑ = (4.14)

But this estimator is not the same as equation (4.5). And since β of (4.5) isunbiased, the β of (4.14) is unbiased. Incidentally note from (4.4), we get afterequating it to zero.

∑ =0 (4.15)

The upshot is that, in regression through origin, we can’t have both ∑ u Xiand ∑ equal to zero. The only condition that is satisfied is that ∑ u Xi = 0. Recall

Yi = Y + u (4.16)

Summing this equation on both sides and dividing by n, we get= Y + u (4.17)

Since for the zero intercept model, ∑ and u need not be zero then it follows thatY = Y (4.18)

That is the mean of actual Y values need not be equal to the mean of theestimated Y values; the two mean values are identical for the intercept presentmodel.

Second, r2, the coefficient of determination which is always non negative forthe conventional model, can on occasions turn out to be negative for the interceptless model. Therefore, conventionally, computed r2 may not be appropriate forregression through origin model.

r2 =1− = 1− ∑∑ (4.19)

Note, for conventional or intercept present model,

RSS = ∑ = ∑ − ∑ ≤ ∑

Unless is zero. That is for conventional model, RSS ≤ TSS or r2 can neverbe negative.

For the zero intercept models, it can be shown analogously that,

RSS= ∑ u = ∑ Y − β ∑ X (4.20)

Now there is no guarantee that this RSS will always be less than TSS whichsuggests that RSS can be greater than TSS, implying that r2 as conventionallydefined can be negative. The conventional r2 is not appropriate for regression



through origin model. But we can compute what is known as the raw r2 for suchmodels which is defined as

Raw r2 =(∑ )∑ ∑ (4.21)

Although the r2 satisfies the relation 0< r2< 1, it is not directly comparable tothe conventional r2 value.

Because of these special features of this model, one needs to exercise greatcaution in using the zero intercept models. Unless there is strong aprioriexpectation, one would be well advised to seek to the conventional intercept presentmodel.

4.3 Functional forms of regression models

So far we have considered models that are linear in parameters as well as inthe variables. Here we consider some commonly used regression models that may benonlinear in the variables but are linear in the parameters or that can be made soby suitable transformation of the variables. In particular we discuss the followingregression models

1. Log – linear model2. Semi log models3. Reciprocal models

4.4 How to measure elasticity: the log linear model

Consider the following model, known as exponential regression model:

Yi = β1 Xiβ2 eui (4.23)

Which may be expressed alternatively as

ln Yi = ln β1 + β2 lnXi +ui (4.24)

Where ln = natural log, ie, log to the base e, and where e= 2.718. If we writeequation (4.24) as;

ln Yi = ∝ + β2 ln Xi +ui (4.25)

Where ∝ = ln β1, this model is linear in parameters ∝ and β, linear in thelogarithms of the variables Y and X and can be estimated by OLS regression.Because of this linearity, such models are called log-log, double log or log linearmodels.

If assumptions of the classical linear regression models are fulfilled, the parametersof equation (4.25) can be estimated by OLS method by letting

Yi* = ∝ + β2 Xi* +ui (4.26)



Where Yi* = ln Yi, Xi* = ln Xi. The OLS estimator α and β obtained will be best linearunbiased estimator of ∝ and β2 respectively.

One attractive feature of the log- log model, which has made it popular inapplied work, is that the slope co-efficient β2 measures the elasticity of Y withrespect to X, that is the percentage change in Y for given small percentage change inX. Thus if Y represents the quantity of a commodity demanded and X its unit price,β2 measures the price elasticity of demand.

In the two variable models, the simplest way to decide whether the log linearmodel fit the data is to plot the scatter diagram of ln Yi against Xi and see thescatter plots lie approximately on a straight line.

4.5 Semi log models: Log Lin and Lin Log models:

4.5.1 How to measure the growth rate: the Log Lin model

Economists, business people and governments are often interested in findingout the rate of growth of certain economic variables such as population GDP, moneysupply, employment etc.

Suppose we want to find out the growth rate of personal consumptionexpenditure on services. Let Yt denote real expenditure on services at time t and Y0

the initial value of the expenditure on services. We may recall the following well-known compound interest formula given as

Yt = Y0 ( 1+r)t (4.27)

Where r is the compound that is overtime rate of growth of Y. taking the naturallogarithm of equation (4.27), we can write

ln Yi = ln Y0 + t ln(1+r) (4.28)

Now letting

β1 = ln Y0 (4.29)

β2 = ln (1+r) (4.30)

We can write equation (4.28) as

ln Yi = β1 + β2t (4.31)

Adding the disturbance term to equation (4.31), we obtain

ln Yt = β1 + β2t +ut (4.32)

This model is like any other regression model in that the parametersβ1 and β2 are linear. The only difference is that the regressand is the logarithm of Yand the regressor is ‘time’ which will take values of 1, 2, 3 etc.



Models like (4.31) are called semi log models because only one variable (in thecase of regressand) appears in the logarithmic form. For descriptive purposes amodel in which the regressand is logarithmic will be called a log lin model. A modelin which the regressand is linear but the regressor is logarithmic is called a lin-logmodel.

Let us briefly examine the properties of the model. In this model the slope co-efficient measures the constant proportional or relative change in Y for a givenabsolute change in the value of the regressor (in the case of variable t) that is,

β2 = (4.33)

If we multiply the relative change in Y by 100, equation (4.33) will then givethe percentage change or the growth rate, in Y for an absolute change in x, theregressor. That is, 100 times β2 give the growth rate in Y; 100 times β2 is known inthe literature as semi elasticity of Y with repeat of X.

The slope coefficient of the growth model, β2 gives the instantaneous (at apoint in time) rate of growth and not the compound (over a period of time) rate ofgrowth. But the latter can be easily found from (4.32) by taking the antilog theestimated β2 and subtracting 1 from it and multiplying the difference by 100.

Linear trend model: instead of estimating model (4.32), researchers sometimesestimate the following model:

Yt = β1 + β2t +ut (4.34)

That is instead of regressing the log of Y on time, they regress Y on time,where Y is the regressand under consideration. Such a model is called a linear trendmodel and the time variable t is known as the trend variable. If the slope coefficientis positive, there is an upward trend in Y, whereas if, it is negative there is adownward trend in.

4.5.2. The Lin – Log model:

Unlike the growth model just discussed, in which we were interested infinding the per cent growth in Y for an absolute change in X, suppose we now wantto find the absolute change in Y for the present change in X. A model that canaccomplish this purpose can be written as

Yi = β1 + β2ln Xi + ui (4.35)

For descriptive purposes we call such a model as a lin log model. Let us interpretthe slope of the coefficient. As usual

β2 =



=

The relative step follows from the fact that a change the log of a number is a relativechange. Symbolically we have,

β2 =∆∆ /

So that ∆Y = β 2 (∆X/X) (4.36)

This equation states that absolute change in Y (= ∆Y) is equal to slope timesthe relative change in X. If the latter is multiplied by 100, then (4.36) gives theabsolute change in Y for a percentage change in X. Thus if (∆X/X) changes by 0.01unit (or 1%), the absolute change in Y is 0.01 (β2): if in an application one finds thatβ2 = 500, the absolute change in Y is (0.01) (500) = 5.0. Therefor when regressionslike (1) is estimated by OLS, do not forget to multiply by the value of estimated slopecoefficient by 0.01.

4.6 Reciprocal models

Models of the following types are known as the reciprocal models.

Yi = β1 + β2( )+ui (4.37)

Although this model is non-linear in the variable X, because it entersinversely or reciprocally, the model is linear in β1 and β2 and therefore a linearregression model.

This model has three features: As X increases indefinitely; the term β2 ( )approaches zero (note β2 is a constant) and Y approaches the limiting or asymptotevalue β1. Therefore the models like (4.37) have built in term an asymptote or limitvalue that the dependent variable will take when the value of the X variableincreases indefinitely.

We conclude our discussion of reciprocal models by considering thelogarithmic reciprocal model, which takes the following form;

Yi = β1 − β2( )+ui (4.38)

Such a model may therefore be appropriate for shot run production functions.

4.7 Scaling and unit of measurement

Here we consider, how the Y and X variables are measured and whether achange in the unit of measurement affects the regression results. Let,

Y = β + β X + u (4.39)



Define

Yi* = w1Yi (4.40)

Xi* = w2Xi (4.41)

Where w1 and w2 are constants, called the scale factors; w1 may be equal to w2

or may be different. From (4.40) and (4.41) it is clear that Y i* and Xi* are rescaled Yiand Xi. Thus if, Yi and Xi measured in billions of dollars and one want to expressthem in millions of dollars, we will have Yi* = 1000Yi and Xi*=1000 Xi; here w1 = w2 =1000.

Now consider the egression using Yi* and Xi* variables:

Yi= β *+ β *Xi + u * (4.42)

Where Yi* = w1Yi, Xi* = w1 Xi, and u * = wi u (why?)

We want to find out the relationship between the following pairs:

1. β * and β *

2. β * and β *

3. Var( β *) and Var(β *)4. Var( β *) and Var(β *)5. 2 and σ*2

6. r2xy and r2x*y*

From least square theory, we know thatβ = Y − β Xβ = ∑ x y∑ x

var β = σ ∑∑var (β ) = σ∑ x

2 =∑

Applying OLS to (4.42), we obtain similarly,

β * = − β Xβ * = ∑ ∗ ∑ ∗∑ ∗



Var β * = σ ∗ ∑ ∗∑ ∗Var β * = ∗∑ ∗

*2 =∑ ∗

From these results it is easy to establish relationships between the two sets ofparameter estimates. All that one has to do is recall these definitional relationships:Yi* = w1Yi (or yi* = w1yi); Xi* = w2Xi (xi* = w2xi); u * = wi u ; *=w1 and * = w2 .Making use of these definitions, the reader can easily verify thatβ * = β (4.43)β * = w1 β (4.44)

*2 = w12 2 (4.45)

Var β * = w12 Var β (4.46)

Var β * = Var β (4.47)

2xy = 2x*y* (4.48)

From the preceding results, it should be clear that, given the regressionresults based on one scale of measurement, one scale of measurement, one canderive the results based on another scale of measurement once the scaling factors,the w’s are known. In practice, though, one should choose the units ofmeasurement sensibly; there is little point in carrying all these zeros in expressingnumbers in millions or billions of dollars.

From the results given in (4.43) though (4.48) one can easily derive somespecial cases. For instance, if w1= w2, that is, the scaling factors are identical, theslope coefficient and its standard error remain unaffected in going from the (X i Yi) tothe (Xi*, Yi*) scale, which should be intuitively clear. However the intercept and itsstandard error are both multiplied by w1. But if X scale is not changed, (i.e. w2 = 1),and Y scale is changed by the factor w1, the slope as well as the interceptcoefficients and their respective standard errors are all multiplied by the same w1factor. Finally if Y scale remains unchanged, (i.e. w1=1), but the X scale is changedby the factor w2, the slope coefficient and its standard error are multiplied by thefactor (1/w2) but the intercept coefficient and its standard error remain unaffected.

It should be noted that the transformation from (Xi Yi) to the ( Xi*, Yi* ) scale does notaffect the properties of OLS estimators.



4.7 Regression on standardised variable

A variable is said to be standardised if we subtract mean value of the variable fromits individual values and divide the difference by the standard deviation of thatvariable. Thus in the regression Y and X, if we redefine these variables as

Yi* = (4.49)

Xi* = (4.50)

Where = sample mean of Y, = standard deviation of Y, = sample mean of X,= standard deviation of X. the variables Xi* and Yi* are called standardised variable.An interesting property of a standardised variable is that its mean value is alwayszero and its standard deviation is always one.

As a result it is not matter in what unit the regressand and regressor aremeasured. Therefore instead of running the standard bivariate regression:

` Yi = β1 + β2Xi + ui (4.51)

We would run regression on the standardised variable as

Yi* = β *+ β *Xi + * (4.52)

= β *Xi + u * (4.53)

Since it is easy to show that in the regression involving standardised regressandand regressor, the intercept term is always zero. The regression coefficient of thestandardised variables denoted by β * and β *, are known as the beta coefficients.Incidentally, notice that (4.53) is a regression through origin model.

How do we interpret beta coefficients? The interpretation is that if the standardisedregressor increases by one standard deviation, on average, the standardisedregressand increases by β * standardised units. Thus unlike the traditional modelwe measure the effect not in terms of the original units in which Y and X areexpressed, but in standard deviation units.

References

1) Gujarati D M, Sangeetha : “Basic Econometrics”. Fourth Edition. TataMcGraw Hill Education Pvt Ltd, New Delhi.

2) Koutsoyiannis A : “ Theory of Econometrics”. Second Edition McMillan PressLtd. London.

3) Shyamala S, Navadeep Kaur and Arul Pragasam : “A text Book onEconometrics- Theory and Applications”. Vishal Publishing Co. Delhi.

4) Gregory C Chow: “Econometrics”. McGraw Hill Book Co. Singapore5) Madnani G M K : “Introduction to Econometrics- Principles and Applications”.

Oxford and IBH Publishing Co. New Delhi.

school of distance education · school of distance education mathematical economics page 6 economic...

Documents