Outline

Specification Issues

Functional Form Misspecification

Using Proxy Variables for Unobserved Explanatory Variables Properties of OLS under Measurement Error

Missing Data, Nonrandom Samples and Outliers Summary

Regression Analysis with Time Series Data The Nature of Time Series Data Examples of Time Series Regressions

Properties of OLS for Time Series Regressions Functional Form and Indicator Variables Trends and Seasonality

Summary

Specification Issues

Functional Form Misspecification

Previously dealt with the failure of one of the GM Assumptions: homoskedasticity

Now deal with more serious violations of the assumptions:

correlation between the error, u, and one or more explanatory variables x
measurement problems and nonrandom samples

The regression model suffers from functional form misspecification when it does not properly account for the relationship between the dependent variable and the (observed) explanatory variables

Functional form misspecification occurs when a key variable has been omitted and that variable is a function of the other variable(s) in the model

Example 1:

log (wage) = β₀ + β₁ educ + β₂ exper + β₃ exper2 + u

but we omit the exper2 term and estimate the following:

log (wage) = β₀ + β₁ educ + β₂ exper + u

this leads to biased estimators of β₀, β₁, and β₂

Misspecifying how exper affects log (wage) results in a biased estimator for the returns to education as well as the returns to experience

Example 2:

log (wage) = β₀ + β₁ educ + β₂ exper + β₃ female + β₄ female × educ + u

but we omit the interaction term female educ then we are misspecifying the functional form and in general we will not have unbiased estimators for any of the parameters

Since the omitted variable is a function of the observed explanatory variables, we can use the data to test for this form of misspecification

A Test for neglected non-linearities Basic idea: If the original model:

y = β₀ + β₁ x₁ + β₂ x₂ + · · · + β_k x_k + u (1)

satisfies the ZCM assumption then no nonlinear functions of the independent variables should be significant when added to (1):

could add quadratic terms ( x2) for all the significant explanatory variables in (1), but this may use up many degrees of freedom (e.g. like in the original White Test for HET)

better to add polynomials of the OLS fitted values ˆy: usually add ˆy2 and ˆy3

Note: the ˆy2, ˆy3 are simply nonlinear functions of the x_j

The RESET test is based on estimating the following model:

y = β₀ + β₁ x₁ + · · · + β_k x_k + δ₀ ˆy2 + δ₁ ˆy3 + u (2) The null hypothesis is that (1) is the correct model. The RESET test is simply an

F-test of H₀ : δ₀ = 0, δ₁ = 0 in the expanded model (2). The F-statistic is distributed approximately as F with df = 2, n − k − 3

Application: Housing Price Model

Two models were estimated for housing prices. The first in level form:

price = β₀ + β₁ lotsize + β₂ sqrft + β₃ bdrms + u (3) and the second using logs for all variables except bdrms:

log (price) = β₀ + β₁ log (lotsize) + β₂ log (sqrft) + β₃ bdrms + u (4)

The RESET statistic for model (3) turns out to be 4.67. The test statistic is distributed as F₂_,₈₂ (i.e. n = 88, k = 3). The critical value for 5% significance level is

3.10. The test statistic > critical value, and hence there is evidence that model (3) is misspecified

The RESET statistic for model (4) is 2.56 (which is less than the 5% critical value of 3.10) and so we do not reject (4) as being correctly specified

On the basis of the RESET test, the log-log specification of (4) is preferred

Note the RESET test may have rejected both models (3) and (4) (or neither). A limitation of the RESET test is that it does not provide direction on how to proceed if a model is rejected

There are a range of tests where the alternative model is explicitly specified:

‘nested tests’ → F-tests for exclusion restrictions
‘nonnested tests’ where the alternative model has different explanatory variables

Tests Against Nonnested Alternatives

Method 1: Construct a comprehensive model that contains each model as a special case and then test the restrictions that led to each of the models

e.g. interested in testing whether explanatory variables should be in level or log form. That is, want to test the model:

y = β₀ + β₁ x₁ + β₂ x₂ + u (5)

against the alternative:

y = β₀ + β₁ log (x₁) + β₂ log (x₂) + u (6)

Estimate the comprehensive model:

y = γ₀ + γ₁ x₁ + γ₂ x₂ + γ₃ log (x₁) + γ₄ log (x₂) + u (7)

and test H₀ : γ₃ = 0, γ₄ = 0 as a test of (5). Then test H₀ : γ₁ = 0, γ₂ = 0 as a test of (6)

Method 2: Obtain the fitted values from the alternative model, and include as one of the explanatory variables in the null model. If the null model is correct, the coeﬀicient on the fitted value from the other model should be insignificant; if not, reject the null model

Example: (same problem as for Method 1)

To test model (5):

Estimate (6) and calculate the fitted values (˜y)
- Then do a (two-sided) t-test of the coeﬀicient on ˜y in the following regression:

y = β₀ + β₁ x₁ + β₂ x₂ + θ˜y + u (8)

A significant t-statistic is rejection of model (5) Follow a similar procedure to test model (6) Note:

no clear winner may emerge

if a model is rejected with this test, it does not imply that the alternative model used is the correct model

Using Proxy Variables for Unobserved Explanatory Variables

A more diﬀicult problem is when a key explanatory variable is omitted because the data is not available

Example: Wage model and the returns to education

Consider a model where ability (ability) affects wages:

log (wage) = β₀ + β₁ educ + β₂ exper + β₃ ability + u (9) If we do not have data on ability, and educ is correlated with ability, then ability

becomes part of the error term and the OLS estimator of β₁ (and β₂) will be biased

We can solve (or reduce) the problem of omitted variable bias if we can use a proxy variable for ability – e.g. we have data on individuals’ IQ and plug that into the regression in place of ability

Requirements for a variable (IQ) to be a valid proxy for an important explanatory variable (ability):

The error u is uncorrelated with educ, exper, ability and IQ (ZCM assumption for all variables used in the model)
1. IQ is irrelevant in the population model once educ, exper, ability have been included
E (ability | educ, exper, IQ) = E (ability | IQ) = δ₀ + δ₃IQ
1. ability is not correlated with educ or exper once we control for IQ
1. the average level of ability only changes with IQ, not with educ or exper

If we run the following regression:

log (wage) = α₀ + β₁ educ + β₂ exper + α₃ IQ + e (10) and the 2 conditions are satisfied, we will get unbiased and consistent estimators of β₁

and β₂ (and α₀ = (β₀ + β₃δ₀), α₃ = β₃δ₃)

We will not get unbiased estimators of β₀ or β₃

Note: can use binary variables as proxies; can include many potential proxies for one (unobserved) explanatory variable

Proxy Variables

Using Lagged Dependent Variables as Proxy Variables

If we suspect that one or more of the independent variables is correlated with an omitted variable, and we cannot obtain a proxy for the omitted variable, then we may be able to use the value of the dependent variable from an earlier time period

Use a lagged dependent variable

Example

Example 9.4. City Crime Rates

Interested in the effects of expenditures on law enforcement (lawexpc₈₇) and the unemployment rate (unem₈₇) on the crime rate in 1987 (crmrte₈₇) using a sample of 46 observations for cities in the US

Dependent Variable: log (crmrte₈₇)

Independent Variables	(1)	(2)
unem₈₇log (lawexpc₈₇) log (crmrte₈₂) intercept	-0.029(0.032)0.203(0.173)–3.34(1.25)	0.009(0.020)-0.140(0.109)1.194(0.132)0.076(0.821)
Obs	46	46

R2 0.057 0.680

Proxy Variables

Model (1), explanatory variables not significant and counter-intuitive

Add log (crmrte₈₂) to Model (2) to control for unobservable characteristics of cities which affect crime and may be correlated with current law enforcement expenditures

See current crime levels strongly related to past crime levels; if the crime rate in a city in 1982 was 1% higher, then the crime rate in 1987 was predicted to be 1.19% higher; the coeﬀicient on log (lawexpc₈₇) is now negative (though marginally insignificant)

Note: Using lagged dependent variables as a proxy is especially useful in policy analysis

Properties of OLS under Measurement Error

Measurement Error in the Dependent Variable

We have an imprecise measure of the dependent variable

e.g. we are interested in analysing the determinants of family savings (y∗) but have ‘noisy’ data on savings (y) due to, e.g., some people misreporting their savings because they forgot about some categories:

e = y − y∗

To estimate the model for y∗, plug in y∗ = y e into the regression equation: leads to a new composite error term (u + e)

If the measurement error (e) is independent of the explanatory variables then OLS estimators will be unbiased and consistent and the usual inference procedures are valid (the measurement error generally causes the std. errors to be greater than if there was no measurement error)

If e is correlated with an independent variable then this causes bias in the OLS estimators

Measurement Error in an Explanatory Variable

This is a much more important problem, with:

e₁ = x₁ − x∗1

If e₁ is uncorrelated with the unobserved variable of interest x∗1 then e₁ will be correlated with the observed x₁, and OLS will give a biased and inconsistent estimator of β₁

The OLS estimator of β₁ will be biased toward 0; this is known as attenuation bias

Missing Data, Nonrandom Samples and Outliers

Missing Data

Most statistical packages will simply ignore obs where information on a variable is missing

If the missing data are simply random, then this causes no problem for OLS (apart from having smaller sample sizes)

Nonrandom Samples

If data are missing for systematic reasons then this results in a nonrandom sample

This will not cause a problem if the sample is chosen on the basis of the independent variables: this is known as exogenous sample selection

Example: Consider the population model for annual family savings:

saving = β₀ + β₁ income + β₂ age + β₃ size + u (11) and suppose our dataset is based on a survey of people aged 35 years and older

Still get unbiased and consistent estimators of the parameters since the regression function E (saving income, age, size) is the same for any subset of the population described by income, age and size

Situation is much worse when there is selection based on the dependent variable y, which is an example of endogenous sample selection

If the sample is based on whether the dependent variable is above/below a certain value then OLS estimators will be biased

e.g. the sample is from a survey of families who are ‘high savers’, with saving > $5000, then:

E (saving | income, age, size) ̸= E (saving | income, age, size, savings > 5000)

Outlying Observations

In some applications, especially with small datasets, the OLS estimates may be strongly influenced by one or several observations – these are known as outliers or influential observations

OLS is susceptible to outliers because it minimises the sum of squared residuals, and hence large residuals receive a lot of weight in calculating the OLS estimates

Outliers can result from simple data entry mistakes (always a good idea to look at the summary stats for your data to check that the min, max, averages look reasonable) or because a small population is being sampled

As a practical matter, best to report the OLS results with and without the outliers included in the data (where the outliers substantially change the results)

Summary

Testing for misspecification of the model using the RESET test, or testing against non-nested alternatives
1. One possible solution to the omitted variable problem is the use of proxy variables. Using proxies for the omitted variable may reduce or eliminate the bias. Often the diﬀiculty is finding a good proxy variable
1. Measurement error in the dependent variable – posed no problem for the properties of OLS; however measurement error in an independent variable causes bias (toward 0)
1. Non-random samples from the population can lead to biases in OLS when the selection of the sample is correlated with the error term. Outliers in the data can also have a large impact on the OLS estimates – should check for outliers and

re-estimate models with them excluded

Regression Analysis with Time Series Data

The Nature of Time Series Data

Time Series Data

The key feature of time series data is that it has a temporal ordering

recognise that the past can affect the future (and not vice versa)

a random variable indexed by time is known as a stochastic process or time series process

Year	Inflation	Unemployment
1948	8.1	3.8
1949	-1.2	5.9
1950	1.3	5.3
.	.	.
1994	2.6	6.1
1995	2.8	5.6
1996	3.0	5.4

Examples of Time Series Regressions

Suppose we have time series data on 2 variables, say y and z, where y_t and z_t

correspond to the same time period (i.e. they are contemporaneous) A static model relating y to z is:

y_t = β₀ + β₁ z_t + ε_t, t = 1, 2, . . . , T (12)

It is a static model since it models a contemporaneous relationships

A change in z at time t has an immediate effect on y

Static Phillips Curve:

π_t = β₀ + β₁ u_t + ε_t (13)

Explaining the Murder Rate:

mrdrte_t = β₀ + β₁ convrte_t + β₂ unemp_t + β₃ yngmle_t + ε_t (14)

where mrdrte_t is the no. of murders per 10,000 people during year t, convrte_t is the murder conviction rate, unemp_t is the unemployment rate and yngmle_t is the fraction of the population who are males aged 18-25 years

Can include many independent variables

FDL models allow one or more explanatory variables to affect y with a lag:

Example:

gfr_t = α₀ + δ₀ pe_t + δ₁ pe_t₋₁ + δ₂ pe_t₋₂ + ε_t (15)

where gfr_t is the general fertility rate (children born per 1000 women) and pe_t is the real value of the personal income tax exemption (per child)

This model recognises that the decision to have children may not immediately change when pe_t changes (but there may be a lagged response)

The more general form of a FDL model is given by:

y_t = α₀ + δ₀ z_t + δ₁ z_t₋₁ + δ₂ z_t₋₂ + ε_t (16) This is a FDL model of order two

To interpret the ceteris paribus effect of z on y, define 2 multipliers:

The impact propensity or impact multiplier is given by δ₀

this measures the immediate impact of a one-unit increase in z at time t on y. It is the coeﬀicient on the contemporaneous z

The long-run propensity (LRP) or long-run multiplier is given by δ₀ + δ₁ + δ₂
- this measures the long-run change in y due to a permanent one-unit increase in z

More generally, in a FDL model of order q, the LRP is given by the sum of all the coeﬀicients on the variables z_t₋_j that is, for:

y_t = α₀ + δ₀ z_t + δ₁ z_t₋₁ + · · · + δ_q z_t₋_q + ε_t (17)

then:

LRP = δ₀ + δ₁ + · · · + δ_q (18)

The FDL model contains the static model as a special case. We can do an F-test to see if z has a lagged effect on y (this is often the primary reason for estimating a FDL model)

Often there is substantial correlation in z at different lags, and due to multicollinearity (revealed by large standard errors), it can be diﬀicult to get precise estimates of the individual δ_j. However, often we still get good estimates of the LRP

Note: we can add more than one explanatory variable with lags, or additional contemporaneous variables, to the FDL model

Properties of OLS for Time Series Regressions

Focus on how the previous assumptions must be altered for the special features of time series data

Time Series (TS) Assumptions:

TS-1 Linear in Parameters:

The stochastic process follows the linear model:

y_t = β₀ + β₁ x_t₁ + · · · + β_k x_tk + ε_t (19)

TS-2 No Perfect Collinearity:

In the sample no independent variable is constant or a perfect linear combination of the others

TS-3 Zero Conditional Mean:

For each t, the expected value of the error ε_t, given the explanatory variables for all time periods, is zero. That is:

E (ε_t | x) = 0, t = 1, 2, . . . , T (20)

This is a critical assumption

The error at every time t is uncorrelated with each explanatory variable in every

time period

In cross-sectional analysis, we require ε_t to be uncorrelated with the explanatory variables also dated at time t. This condition is equivalent to:

E (ε_t | x_t₁, x_t₂, . . . , x_tk) = E (ε_t | x_t) = 0 (21) When condition (21) is true, the x_tj are contemporaneously exogenous (i.e. the ε_t

and x_tj are contemporaneously uncorrelated)

The ZCM assumption here requires more than contemporaneous exogeneity: the ε_t must be uncorrelated with the x’s from other time periods. That is, Corr (ε_t, x_s) = 0 even when s ̸= t

When condition (20) is satisfied, the explanatory variables are strictly exogenous

We need this stronger assumption for OLS to be unbiased

For cross-sectional data, we did not need to state how the error term for unit i, u_i, related to the explanatory variables for other units because random sampling ensures that u_i is independent of the value of the explanatory variables for other units

For time series data, random sampling is almost never appropriate so we need to explicitly assume strict exogeneity (20)

Result: Unbiasedness of OLS

With Assumptions TS-1 to TS-3:

E (βˆj) = β_j, for all j = 0, 1, . . . , k (22) Proof same as the case for cross-section data

Consequences of omitted variables (leading to bias) same as for cross-section data

TS-4 Homoskedasticity:

Var (ε_t | x) = Var (ε_t) = σ², for all t = 1, 2, . . . , T (23)

The violation of TS-4 causes heteroskedasticity (we can use the tests for HET we previously covered for time series models as well)

TS-5 No Serial Correlation:

Conditional on x, the errors in two different time periods are uncorrelated; or more simply:

Corr (ε_t, ε_s) = 0, for all t ̸= s (24)

The violation of TS-5 means the errors term have serial correlation or

autocorrelation

An example of serial correlation, is when say studying the interest rate, it is found when the interest rate is unexpectedly high in one period (ε_t > 0), it is also likely to be above average in the next period (ε_t₊₁ > 0) and hence Corr (ε_t, ε_t₊₁) > 0

Note: this condition says nothing about the correlation among the independent variables

Result: OLS Sampling Variances

With the Assumptions TS-1 to TS-5, the variance of βˆj is the same as for cross-section data

Result: Unbiased Estimator of σ²

With the Assumptions TS-1 to TS-5, an unbiased estimator of σ² is:

Result: Gauss-Markov Theorem

σˆ2 = (n

SSR

− k − 1)

(25)

With the Assumptions TS-1 to TS-5, the OLS estimators are BLUE

TS-6 Normality

The error terms ε_t are independent of x and are independently and identically distributed as N 0, σ²

Note: Assumption TS-6 implies TS-3, TS-4 and TS-5

Result: Normal Sampling

With the Assumptions TS-1 to TS-6, the OLS estimators are normally distributed. Further, under the null hypothesis, each t-statistic has a t-distribution, the F-statistics have F-distributions and the usual confidence intervals are valid

When TS-1 to TS-6 hold, everything we have covered regarding estimation and inference for cross-section data applies to the time series regression model

Functional Form and Indicator Variables

All the functional forms we have studies so far in the subject can be used in time series regressions

The most important specification is the log-log model – as it is very popular in empirical studies (recall these models estimate constant elasticities or constant percentage effects)

Example 1: Employment Effects of the Minimum Wage

Using data for the years for 1950–1987 for Puerto Rico, the following model estimates were obtained:

log (^emprte_t) = −10.05 − 0.154 log (mincov_t) − 0.012 log (USGNP_t) (0.77) (0.065) (0.089)

T = 38, R2 = 0.661, R¯2 = 0.641

where emprte_t is the employment rate, mincov_t measures the importance of the minimum wage relative to the average wage and USGNP_t is US GNP (which is a proxy for aggregate demand)

The estimated elasticity of emprte_t w.r.t. mincov_t is -0.154 (t-statistic = -2.37) which is statistically significant. Therefore a higher minimum wage lowers the employment rate

Example 2: Money Demand

Note we can use log form in FDL models

e.g. suppose, for quarterly data, a model relating the demand for money, M_t, and

GDP_t is given by:

log (M) = α₀ + δ₀ log (GDP_t) + δ₁ log (GDP_t₋₁) (26)

+ δ₂ log (GDP_t₋₂) + δ₃ log (GDP_t₋₃) + δ₄ log (GDP_t₋₄) + ε_t

The impact propensity, δ₀, is also the short-run elasticity

The LRP, δ₀ + δ₁ + δ₂ + δ₃ + δ₄, is also the long-run elasticity

this measures the percentage increase in money demand after 4 quarters given a permanent 1% increase in GDP

Estimating, and testing, the LRP

Question: How can we estimate the LRP and its standard error?

Answer: For the model (26), let LRP = θ. Then:

θ = δ₀ + δ₁ + δ₂ + δ₃ + δ₄ δ₀ = θ − δ₁ − δ₂ − δ₃ − δ₄

Substitute this expression for δ₀ into the model (26)

Transforming the model to obtain the LRP:

log (M) = α₀ + θ log (GDP_t)

+ δ₁ [log (GDP_t₋₁) − log (GDP_t)]

+ δ₂ [log (GDP_t₋₂) − log (GDP_t)]

+ δ₃ [log (GDP_t₋₃) − log (GDP_t)]

+ δ₄ [log (GDP_t₋₄) − log (GDP_t)]

+ ε_t

By transforming the variables and running OLS on the above equation, the coeﬀicient and associated standard error on log (GDP_t) will be for the LRP

Note:

Can use dummy variables in time series analysis:

useful for studying the impact of a specific event in time
also useful to control for particular / unusual time periods

Likewise, can use interactions between dummy and quantitative variables

Trends and Seasonality

Many economic time series have a common tendency of growing over time, and we need to take into account the time trend of some time series in order to draw causal inferences; otherwise if we do not take account that 2 series are trending together we may falsely conclude that changes in one variable is caused by changes in another

Many time series may appear to be correlated only because they are both trending over time for reasons related to other unobserved factors

Need to incorporate time trends into the model

Linear time trend:

y_t = α₀ + α₁ t + ε_t (27)

α₁ measures the change in y_t from one period to the next due to the passage of time (holding other factors constant)

∆y_t = y_t − y_t₋₁ = α₁

Exponential trend

log (y_t) = β₀ + β₁ t + ε_t (28)

The average growth rate is constant (i.e. the percentage change in y is roughly constant through time)

Model the log of the series as a linear trend, hence:

y_t = exp (β₀ + β₁ t + ε_t)

has an exponential trend

The average growth rate is constant (i.e. the percentage change in y is roughly constant through time)

e.g. if βˆ1 = 0.027 then y_t grows approximately 2.7% per year on average

Finding a relationship between 2 or more trending variables simply because each is growing over time is an example of spurious regression; adding a time trend to the model eliminates this problem

Example:

y_t = β₀ + β₁ x_t₁ + β₂ x_t₂ + β₃ t + ε_t (29)

and the time trend captures unobserved factors that are simply growing (or shrinking) over time

If (29) satisfies assumptions TS-1 to TS-3 then omitting t from (29) will generally lead to biased estimators of β₁ and β₂

Note: R2s in time series regressions are often very high, especially compared to typical R2 statistics for cross-sectional data; however, this does not necessarily mean we learn more about y from time series data. The R2 may be artificially high because the dependent variable is trending

Seasonality

A time series observed at monthly or quarterly intervals (or even weekly or daily) may exhibit seasonality (a recurring pattern over the course of a year)

Sometime data have been ‘seasonally adjusted’, which means that the seasonal pattern has been removed from the series

If we work with seasonally unadjusted data, it is simple to take the seasonal pattern of the data into account in the regressions model

Include a set of seasonal dummy variables (e.g. indicators variables for specific months or quarters of the year) if the dependent variable, an independent variable or both exhibit seasonality

Seasonal data may also be trending; in which case we should include both a time trend and seasonal dummy variables

Summary

Under assumptions parallel to those for cross-section data, OLS is unbiased and BLUE, and the usual tests can be used
1. The temporal correlation in time series data requires us to make assumptions about how the errors are correlated with the explanatory variables in all time periods, and the correlation across time in the errors themselves
1. The CLM assumptions for time series are very strong (though a useful starting point)
1. We considered both static and dynamic FDL models
1. We dealt with issues of trends and seasonality, and how these can be easily included in regression models