An economic model – which may be formally developed (e.g. derivation of consumer demand equations from a model of utility maximisation) or based on intuitive reasoning
An econometric model – which requires specifying the nature of the relationship between variables

The Nature of Econometrics

In econometric analysis the special features of the datasets need to be taken into account

The most important data structures are:

Cross-sections
Time series
Pooled Cross-sections
Panels

Sample of individuals, households, firms, countries or other units taken at a point in time (i.e. “snapshot”)

Usually obtained by random sampling from the population

Cross-sectional data are widely used in economics and other social sciences

Data set on wages and other personal characteristics

obs.	wage	educ.	exper.	female	married
1	3.10	11	2	1	0
2	3.24	12	22	1	1
3	3.00	11	2	0	0
4	6.00	8	40	0	1
5	5.30	12	7	0	1
.	.	.	.	.	.
.	.	.	.	.	.
525	11.56	16	5	0	1
526	3.50	14	5	1	0

Note: The order of the observations (obs. number) is not important

Observations on a variable (or set of variables) over time For example: stock prices, CPI, GDP, crime rates

The chronological ordering of observations is important

Observations cannot be assumed to be independent over time, most economic time series are (strongly) related to their recent histories

Econometric model needs to take this into account

Data frequency is also important, due to seasonal patterns (e.g. daily, weekly, monthly, quarterly, annual)

Data set on minimum wages in Puerto Rico

obs.	year	minwg	cover	unemp	gnp
1	1950	0.20	20.1	15.4	879
2	1951	0.21	20.7	16.0	925
3	1952	0.23	22.6	14.8	1016
.	.	.	.	.	.
.	.	.	.	.	.
38	1987	3.35	58.2	16.8	4497

Some data sets have both cross-sectional and time series characteristics

For example ABS Household Expenditure Surveys; 1975, 1984, 1988, 1993, 1998,

2003, 2009, 2014

Record family income, expenditure, family size,…
New random sample each year

Pool them to increase sample size – no family is in the sample for the 2 years

Two years of house prices

obs.	year	hprice	proptax m2 bedrms
1	1992	85000	42	530	2
2	1992	67800	35	512	2
3	1992	134690	34	940	2.5
.	.	.	.	.	.
250	1992	65780	41	310	1
251	1994	187999	24	820	3
.	.	.	.	.	.
520	1994	57200	16	500	1.5

Consists of a time series for each cross-sectional unit

Follow the same individuals / families / firms etc. over time

Example: data on wages, union membership and other individual characteristic

obs.	person	year	wage	educ	union
1	1	1990	21.5	16	0
2	1	1991	22.1	16	0
3	2	1990	15.6	12	0
4	2	1991	17.9	12	1
.	.	.	.	.	.
499	250	1990	15.0	8	1
500	250	1991	16.1	8	0

Panel data has some important advantages over other data structures:

Can control for certain types of unobserved characteristics, and can study lags in behaviour

Some important questions can only be answered with panel data:

e.g. studying dynamic behaviour of individual units

Has become an important area for policy evaluation

Focus on panel data methods toward the end of the semester

In most tests of theory, and policy evaluation, the goal is to infer a causal effect of one variable on another

Most propositions in economics are ‘ceteris paribus’ (or conditional) by nature

Example: What is the effect of an extra year of education on individual’s wages, holding all other factors constant (such as experience, hours of work, occupation, innate ability)? If these other factors are not constant, we cannot determine the casual effect of education on wages:

Not feasible to literally hold ‘all else equal’
Have enough other factors been held constant to infer causality?

Properly applied, econometric methods can simulate a ceteris paribus experiment

Economic theory and econometrics together can help us uncover causal effects

The Multiple Regression Model – Estimation

Motivation

Multiple regression analysis allows us to explicitly control for many factors which affect the dependent variable

better able to draw ‘ceteris paribus’ conclusions
better explanation of the variation in the dependent variable
flexible functional forms

Multiple regression allows many observed factors to affect y. The general multiple regression model is:

y = β₀ + β₁ x₁ + β₂ x₂ + β₃ x₃ + · · · + β_k x_k + u (1)

There are k independent variables plus an intercept, hence k + 1 (unknown) population parameters

No matter how many explanatory variables we include in the model, there will always be factors we cannot include and these are all contained in u – the regression error term

The key assumption for the multiple regression model is:

E (u | x₁, x₂, . . . , x_k) = 0 (2)

This is called the Zero Conditional Mean (ZCM) assumption: factors in the unobserved error term are independent of all the explanatory variables

Implies that u is uncorrelated with x₁, . . . , x_k

Critical for OLS to be unbiased

The Mechanics of OLS

Graphically, it is the ‘line of best fit’

Algebraically, the OLS estimates βˆ0, βˆ1, . . . , βˆk are chosen to minimise the sum of squared residuals:

yi βˆ0 βˆ1 xi1 βˆ2 xi2 βˆ3 xi3 βˆk xik (3)

i=1

Can solve using calculus (e.g. there are k + 1 first order conditions)

But for moderate n or k, this is very hard to do by hand, but is easy with modern computers!

The equation:

ˆy = βˆ0 + βˆ1 x₁ + βˆ2 x₂ + · · · + βˆk x_k (4)

is the OLS regression line (or sample regression function)

βˆ0 = OLS intercept estimate

βˆ1, . . . , βˆk = OLS slope estimates

(Also equivalent “Method of Moments” derivation for OLS)

The OLS regression line can be written in the form of changes:

∆ˆy = βˆ1 ∆x₁ + βˆ2 ∆x₂ + · · · + βˆk ∆x_k (5)

The coeﬀicient on x₁ measures the change in ˆy due to a one-unit increase in x₁ holding all other independent variables constant

Can use estimated model to gauge the effect of changing more than one independent variable at the same time

Once we have the OLS regression line, we can calculate the fitted or predicted value for each observation:

ˆy_i = βˆ0 + βˆ1 x_i₁ + βˆ2 x_i₂ + · · · + βˆk x_ik (6)

we just plug the values of the independent variables into the OLS regression model to get the predicted values

The OLS residual for observation i is:

uˆ_i = y_i − ˆy_i (7)

The OLS fitted values and residuals have some important properties:

The sample average of the residuals is 0: ∑n uˆ_i = 0
The sample covariance between each independent variable and the OLS residuals is 0. As a result, the sample covariance between the OLS fitted values and the

residuals is 0. ∑n x_i_juˆ_i = 0 for all j

The point (¯x₁, ¯x₂, . . . , ¯x_k, ¯y) is always on the regression line

Define:

total sum of squares (SST)

SST = (y_i ¯y)² (8)

i=1

explained sum of squares (SSE)

SSE = (ˆy_i ¯y)² (9)

i=1

sum of squared residuals (SSR)

2 (10)

Note: SST = SSE + SSR

i=1

The R-squared is defined as:

R2 = SSE = 1 − SSR

(11)

SST SST

which is the ratio of the explained variation to the total variation in y R2 also equals the squared correlation coeﬀicient between y_i and ˆy_i

The R2 never decreases (and generally increases) when additional explanatory variables are added to the model

Measurement and Functional Form

The effects of changing the units of measurement:

If y is multiplied by a constant c, then the OLS intercept and slope estimates are also multiplied by c

Multiplying x by a constant c leads to the OLS slope estimate changing by 1/c (and the intercept estimate is unchanged)

Example: CEO Salary and Returns on Equity

salary = β₀ + β₁ roe + u

where salary is measured in $1000 and roe is measured in per cent

R2 not change when we change the units of measurement

Can allow for nonlinearities by transforming the dependent and/or independent variables

It is common in applied research for the dependent variable to be in log form

Example 1: The wage-education regression model Model 1:

wage = β₀ + β₁ educ + u

where, for a particular sample, βˆ1 = 12.9

An extra year of education is predicted to increase a persons wage by $12.90 (this applies for whether the person has 1 year or 16 years of education)

Model 2:

log(wage) = β₀ + β₁ educ + u

where, for a particular sample, βˆ1 = 0.083

An extra year of education is predicted to increase a persons wage by 8.3%

The percentage change in the wage is the same for each additional year of education (so the absolute change in the wage level for an extra year of education increases as education increases)

Table: Summary of Functional Forms Involving Logarithms

Model	Dependent variable	Independent variable	Interpretation of β₁
level-level	y	x	∆y = β₁ ∆x

level-log y log (x) ∆y = (β₁/100) %∆x log-level log (y) x %∆y = (100β₁) ∆x log-log log (y) log (x) %∆y = β₁%∆x

Why is this linear regression?

The model y = β₀ + β₁ x₁ + . . . + β_k x_k + u is linear in the parameters

The mechanics of OLS do not depend on how y and x are defined, however the

interpretation of the coeﬀicients does depend on their definitions

Expected Values of the OLS Estimators

Unbiasedness of OLS: Does E (βˆj) = β_j? Multiple Linear Regression (MLR) Assumptions:

Linear in Parameters: Population model:

y = β₀ + β₁ x + β₂ x₂ + · · · + β_k x_k + u (12)

Random Sampling: We have a random sample of size n:

{(x_i₁, x_i₂, . . . , x_ik, y_i) : i = 1, . . . , n} from the population, for which:

y_i = β₀ + β₁ x_i₁ + β₂ x_i₂ + · · · + β_k x_ik + u_i, i = 1, . . . , n (13)

No Perfect Collinearity: No independent variable is constant and there is no exact linear relationships among the independent variables:
1. independent variables may be correlated, but they cannot be perfectly correlated
1. if the variables are perfectly correlated, the solution is to drop one of the variables
1. this has nothing to do with the error term u
1. Zero Conditional Mean: The error u has an expected value of 0 given any value of the independent variables

E (u | x₁, x₂, . . . , x_k) = 0 (14)

With Assumptions A-1 to A-4:

E (βˆj) = β_j, j = 0, 1, . . . , k (15)

The OLS estimators are unbiased estimators for the population parameters

the sampling distributions of βˆj is centred around β_j

if we collected repeated samples of data the OLS procedure would, on average, give us the population parameter values

if the sample is ‘typical’ then our estimates will be ‘near’ the population values

Including Irrelevant Variables in the Regression Model:

if an explanatory variable is irrelevant, then the population parameter, β, is equal to 0

OLS is still unbiased

but, including irrelevant variables is not completely harmless as it has undesirable effects on the variances of the OLS estimators

True model:

y = β₀ + β₁ x₁ + β₂ x₂ + u (16)

Estimate the regression model:

ˆy = βˆ0 + βˆ1 x₁ + βˆ2 x₂ + βˆ3 x₃ (17)

Since E (βˆj) = β_j, this implies: E (βˆ0) = β₀, E (βˆ1) = β₁, E (βˆ2) = β₂, and

E (βˆ3) = β₃ = 0

Suppose we omit a variable that belongs in the population model

causes the OLS estimators to be biased
can consider the direction and size of the bias

Example:

True model:

y = β₀ + β₁ x₁ + β₂ x₂ + u (18)

Estimate the regression model:

˜y = β˜0 + β˜1 x₁ (19)

Suppose:

wage = β₀ + β₁ educ + β₂ ability + u (20)

but we only have data on wage and education and so estimate the alternative model:

wage = β₀ + β₁ educ + v (21)

and hence v = β₂ ability + u

The simple regression estimate of β₁ from (21) corresponds to β˜1 in (19)

Find that:

E (β˜1) = β₁ + β₂δ˜

̸= β₁

(22)

where δ˜ is the slope coeﬀicient from the simple regression of x₂ on x₁ (i.e. δ˜ is the sample covariance between x₂ and x₁)

Note that:

E β˜1 − β₁ = β₂δ˜

is known as the omitted variable bias

(23)

There are two (2) special cases where the omitted variable bias will equal zero: 1. β₂ = 0

2. δ˜ = 0 Even if x₂ does belong in the population model, β˜1 will be an unbiased estimator of β₁ if x₁ and x₂ are uncorrelated

This is equivalent to E (x₂ | x₁) = 0

leaving x₂ in the error term does not violate the ZCM assumption for the error term

Summary of bias in β˜1 when x₂ is omitted in estimating equation (23):

Cov (x₁, x₂) > 0 Cov (x₁, x₂) < 0

β₂ > 0 positive bias negative bias

β₂ < 0 negative bias positive bias

In practice, we do not know β₂ but we may have a good idea based on theory or intuition

Suppose the model:

log (wage) = β₀ + β₁ educ + β₂ ability + u (24)

satisfies Assumptions 1–4, but we estimated the following:

log^(wage) = 0.584 + 0.083 educ (25)

Since these are the results from just one sample, we cannot say whether the true β₁ is greater or less than 0.083, but the average of estimates across many random samples would be too large (assuming education and ability are positively correlated and

β₂ > 0)

the size of the bias is also important (e.g. a small bias in either direction may not be a concern)

Variances of the OLS Estimators

Concerned with the spread of the sampling distribution of the OLS estimators Need to add an extra assumption:

Homoskedasticity (i.e. constant variance):

Var (u | x₁, . . . , x_k) = σ²

variance of the error term is unrelated to the explanatory variables
if this assumption fails then the model exhibits heteroskedasticity
Very different to Assumption 4 (ZCM)

Note: Assumptions 1–5 together are known as the Gauss-Markov Assumptions

Under Assumptions A-1 to A-5:

SST_j (1 − R2)

for j = 1, . . . , k (26)

where SST_j = ∑n (x_ij − ¯x_j) is the total sample variation in x_j and R2 is the R-squared

from regressing x_j on all the other independent variables (including an intercept)

not prove this result
useful to see the components that make up Var (βˆj)

the size of Var βˆj is important in practice – it tells us how precise the OLS estimates are

The Error Variance: σ²
1. the larger is σ² the larger the variance of the OLS estimators

i.e. more ‘noise’ in the population equation, the harder it is to estimate the partial effect of any independent variable on y

The Total Sample Variation in x_j: SST_j
- the larger the total variation in x_j the smaller is Var βˆj
- we can increase the total sample variation in the x_j by increasing the sample size

The Linear Relationship among the Independent Variables: R2
- Notice this is NOT the R2 in the regression of y on x₁, . . . , x_k
- R2 is obtained from the regression of x_j on the other independent (x) variables

R2 is the proportion of the sample variation in x_j that can be explained by the other independent variables in the model

High (but not perfect) correlation between 2 or more independent variables is known as multicollinearity

Multicollinearity does not violate any of our Assumptions 1–5

A high degree of correlation between some independent variables may be irrelevant in practice if it does not involve the parameters/variables we are most interested in

Example:

y = β₀ + β₁ x₁ + β₂ x₂ + β₃ x₃ + u

x₂ and x₃ may be highly correlated, but if mostly interested in x₁ then Var βˆ1 may be small if x₁ not highly correlated with x₂ and x₃

An unbiased estimator of σ² (under Assumptions 1–5) is:

σˆ2 = (n 1 1) ∑ uˆ2

(27)

= (n

− k −

SSR

− k − 1)

i=1

where n is the number of observations and k is the number of independent variables Note that the degrees of freedom (df) for the OLS model is given by:

df = (number of obs) − (number of estimated parameters)

= n − (k + 1) = n − k − 1

Eﬀiciency of OLS

This result is important because it justifies the use of OLS above any other alternative method for estimating the population parameters!

Under Assumptions A-1 to A-5, βˆ0, βˆ1, . . . , βˆk are the best linear unbiased estimators (BLUEs) of β₀, β₁, . . . , β_k, respectively

Assumptions 1–4 imply that the OLS estimators are unbiased

Linear estimators are estimators that can be expressed as a linear function of the data for the dependent variable

OLS is one of many linear, unbiased estimators of the population parameters ‘Best’ means that the estimator has the lowest variance

Given Assumptions 1–5, OLS are BLUE: the most precise, unbiased, linear estimator – justifying our use of OLS

Note: No need to prove Gauss-Markov result – important to be aware of the eﬀiciency of OLS (under Assumptions 1–5) → reason for focus on OLS

Summary

Multiple regression allows us to hold other factors fixed when we examine the effect of a particular independent variable on the dependent variable
The model is linear in the parameters (β) – and nonlinear relationships can be modelled by choosing the form of the dependent and independent variables
The method of OLS is easy to apply to estimate the multiple regression model
R2 is the proportion of the sample variation in the dependent variable explained by the independent variables. It is one measure of goodness-of-fit of the model

Under the first 4 GM assumptions, the OLS estimators are unbiased
Under the 5 GM assumptions, the variance of the OLS slope estimator depends on the variance of the error term (σ²) and the total, independent variation in the explanatory variable
Under the 5 GM assumptions, the OLS estimators are BLUE