Regression Analysis
Least Square Regression Analysis
The aim of a least squares regression is to minimize the distance between the regression line and error terms (e).
5 The Constant
6 The Slope Coefficient (β)
7 T-testWhen conducting a t-test, we can use either a 1 or 2 tailed test, depending on the hypothesisWe usually use a 2 tailed test, in this case our alternative hypothesis is that our variable does not equal 0. In a one tailed test we would stipulate whether it was greater than or less than 0.Thus the critical value for a 2 tailed test at the 5% level of significance is the same as the critical value for a 1 tailed test at the 2.5% level of significance.
8 T-testWe can also test whether our coefficient equals 1.
9 Gauss-Markov Assumptions
There are 4 assumptions relating to the error term.The first is that the expected value of the error term is zeroThe second is that the error terms are not correlatedThe third is that the error term has a constant varianceThe fourth is that the error term and explanatory variable are not correlated.
There are 4 assumptions relating to the error term.The first is that the expected value of the error term is zeroThe second is that the error terms are not correlatedThe third is that the error term has a constant varianceThe fourth is that the error term and explanatory variable are not correlated.
11 Additional Assumptions
There are a number of additional assumptions such as normality of the error term and n (number of observations) exceeding k (the number of parameters).If these assumptions hold, we say the estimator is BLUE
There are a number of additional assumptions such as normality of the error term and n (number of observations) exceeding k (the number of parameters).If these assumptions hold, we say the estimator is BLUE
12 BLUE Best or minimum variance Linear or straight line
Unbiased or the estimator is accurate on average over a large number of samples.Estimator
Unbiased or the estimator is accurate on average over a large number of samples.Estimator
13 Consequences of BLUEIf the estimator is not BLUE, there are serious implications for the regression, in particular we can not rely on the t-tests.In this case we need to find a remedy for the problem.
14 AutocorrelationAutocorrelation occurs when the second Gauss-Markov assumption fails.It is often caused by an omitted variableIn the presence of autocorrelation the estimator is not longer Best, although it is still unbiased. Therefore the estimator is not BLUE.
15 Durbin-Watson Test This tests for 1st order autocorrelation only
In this case the autocorrelation follows the first-order autoregressive process
In this case the autocorrelation follows the first-order autoregressive process
17 DW StatisticThe DW test statistic lies between 0 and 4, if it lies below the dl point, we have positive autocorrelation. If it lies between du and 4-du, we have no autocorrelation and if above 4-dl we have negative autocorrelation.The dl and du value can be found in the DW d-statistic tables (at the back of most text books)
18 Lagrange Multiplier (LM) Statistic
Tests for higher order autocorrelationThe test involves estimating the model and obtaining the error term .Then run a second regression of the error term on lags of itself and the explanatory variable: (the number of lags depends on the order of the autocorrelation, i.e. second order)
Tests for higher order autocorrelationThe test involves estimating the model and obtaining the error term .Then run a second regression of the error term on lags of itself and the explanatory variable: (the number of lags depends on the order of the autocorrelation, i.e. second order)
19 LM TestThe test statistic is the number of observations multiplied by the R-squared statistic.It follows a chi-squared distribution, the degrees of freedom are equal to the order of autocorrelation tested for (2 in this case)The null hypothesis is no autocorrelation, if the test statistic exceeds the critical value, reject the null and therefore we have autocorrelation.
20 Remedies for Autocorrelation
There are 2 main remedies:The Cochrane-Orcutt iterative processAn unrestricted version of the above process
There are 2 main remedies:The Cochrane-Orcutt iterative processAn unrestricted version of the above process
21 HeteroskedasticityThis occurs when the variance of the error term is not constantAgain the estimator is not BLUE, although it is still unbisased it is no longer BestIt often occurs when the values of the variables vary substantially in different observations, i.e. GDP in Cuba and the USA.
22 ConclusionThe residual or error term is the difference between the fitted value and actual value of the dependent variable.There are 4 Gauss-Markov assumptions, which must be satisfied if the estimator is to be BLUEAutocorrelation is a serious problem and needs to be remediedThe DW statistic can be used to test for the presence of 1st order autocorrelation, the LM statistic for higher order autocorrelation.
Multiple Regression
1 Multiple Regression
2 IntroductionDescribe some of the differences between the multiple regression and bi-variate regressionAssess the importance of the R-squared statistic.Examine the F-test and distributionShow how we can use the F-test to determine joint significance.
3 Multiple RegressionIn general the regression estimates are more reliable if:i) n is large (large dataset)ii) The sample variance of the explanatory variable is high.iii) the variance of the error term is smalliv) The less closely related are the explanatory variables.
4 Multiple RegressionThe constant and parameters are derived in the same way as with the bi-variate model. It involves minimising the sum of the error terms. The equation for the slope parameters (α) contains an expression for the covariance between the explanatory variables.When a new variable is added it affects the coefficients of the existing variables
5 Regression
6 RegressionIn the previous slide, a unit rise in x produces 0.4 of a unit rise in y, with z held constant.Interpretation of the t-statistics remains the same, i.e /0.4=1 (critical value is 2.02), so we fail to reject the null and x is not significant.The R-squared statistic indicates 30% of the variance of y is explainedDW statistic indicates we are not sure if there is autocorrelation, as the DW statistic lies in the zone of indecision (Dl=1.43, Du=1.62)
7 Adjusted R-squared Statistic
This statistic is used in a multiple regression analysis, because it does not automatically rise when an extra explanatory variable is added.Its value depends on the number of explanatory variablesIt is usually written as (R-bar squared):
This statistic is used in a multiple regression analysis, because it does not automatically rise when an extra explanatory variable is added.Its value depends on the number of explanatory variablesIt is usually written as (R-bar squared):
8 Adjusted R-squaredIn generally rises when the t-statistic of an extra variable exceeds unity (1),so does not necessarily imply the extra variable is significant.It has the following formula (n-number of observations, k-number of parameters):
9 The F-test The F-test is an analysis of the variance of a regression
It can be used to test for the significance of a group of variables or for a restrictionIt has a different distribution to the t-test, but can be used to test at different levels of significanceWhen determining the F-statistic we need to collect either the residual sum of squares (RSS) or the R-squared statisticThe formula for the F-test of a group of variables can be expressed in terms of either the residual sum of squares (RSS) or explained sum of squares (ESS)
It can be used to test for the significance of a group of variables or for a restrictionIt has a different distribution to the t-test, but can be used to test at different levels of significanceWhen determining the F-statistic we need to collect either the residual sum of squares (RSS) or the R-squared statisticThe formula for the F-test of a group of variables can be expressed in terms of either the residual sum of squares (RSS) or explained sum of squares (ESS)
10 F-test of explanatory power
This is the F-test for the goodness of fit of a regression and in effect tests for the joint significance of the explanatory variables.It is based on the R-squared statisticIt is routinely produced by most computer software packagesIt follows the F-distribution, which is quite different to the t-test
This is the F-test for the goodness of fit of a regression and in effect tests for the joint significance of the explanatory variables.It is based on the R-squared statisticIt is routinely produced by most computer software packagesIt follows the F-distribution, which is quite different to the t-test
11 F-test formulaThe formula for the F-test of the goodness of fit is:
12 F-distributionTo find the critical value of the F-distribution, in general you need to know the number of parameters and the degrees of freedomThe number of parameters is then read across the top of the table, the d of f. from the side. Where these two values intersect, we find the critical value.
13 F-test critical value
14 F-distribution Both go up to infinity
If we wanted to find the critical value for F(3,4), it would be 6.6The first value (3) is often termed the numerator, whilst the second (4) the denominator.It is often written as:
If we wanted to find the critical value for F(3,4), it would be 6.6The first value (3) is often termed the numerator, whilst the second (4) the denominator.It is often written as:
15 F-statisticWhen testing for the significance of the goodness of fit, our null hypothesis is that the explanatory variables jointly equal 0.If our F-statistic is below the critical value we fail to reject the null and therefore we say the goodness of fit is not significant.
16 Joint SignificanceThe F-test is useful for testing a number of hypotheses and is often used to test for the joint significance of a group of variablesIn this type of test, we often refer to ‘testing a restriction’This restriction is that a group of explanatory variables are jointly equal to 0
18 F-testsThe test for joint significance has its own formula, which takes the following form:
19 Joint Significance of a group of variables
To carry out this test you need to conduct two separate OLS regression, one with all the explanatory variables in (unrestricted equation), the other with the variables whose joint significance is being tested, removed.Then collect the RSS from both equations.Put the values in the formulaFind the critical value and compare with the test statistic. The null hypothesis is that the variables jointly equal 0.
To carry out this test you need to conduct two separate OLS regression, one with all the explanatory variables in (unrestricted equation), the other with the variables whose joint significance is being tested, removed.Then collect the RSS from both equations.Put the values in the formulaFind the critical value and compare with the test statistic. The null hypothesis is that the variables jointly equal 0.
20 Joint SignificanceIf we have a 3 explanatory variable model and wish to test for the joint significance of 2 of the variables (x and z), we need to run the following restricted and unrestricted models:
21 Example of the F-test for joint significance
Given the following model, we wish to test the joint significance of w and z. Having estimated them, we collect their respective RSSs (n=60).
Given the following model, we wish to test the joint significance of w and z. Having estimated them, we collect their respective RSSs (n=60).
22 Joint significance
23 Joint significanceHaving obtained the RSSs, we need to input the values into the earlier formula (slide 18):
24 Joint significanceAs the F statistic is greater than the critical value (28>3.15), we reject the null hypothesis and conclude that the variables w and z are jointly significant and should remain in the model.
25 ConclusionMultiple regression analysis is similar to bi-variate analysis, however correlation between the x variables needs to be taken into accountThe adjusted R-squared statistic tends to be used in this caseThe F-test is used to test for joint explanatory power of the whole regression or a sub-set of the variablesWe often use the F-test when testing for things like seasonal effects in the data.
TIme Series
1 Time-series analysis
2 Basic time seriesData on the outcome of a variable or variables in different time periods are known as time-series data.Time-series data are prevalent in finance and can be particularly challenging because they are likely to violate the underlying assumptions of linear regression.Residual errors are correlated instead of being uncorrelated, leading to inconsistent coefficient estimates.The mean and/or variance of the explanatory variables may change over time, leading to invalid regression results.Example of a basic time series known as an autoregressive process:This slide introduces time series as a concept, and the basic autoregressive process makes it easy to see where the correlation of the error terms can be a problem. If we are using time-series observations on a given variable, x, then observations in two or more periods are likely to be related to observations from the prior period purely by construction (they may not be so but are likely to be so) as seen in the autoregressive process in the slide.𝑥 𝑡 = 𝑏 0 + 𝑏 1 𝑥 𝑡−1 + ε 𝑡
3 Trend analysisThe most basic form of time-series analysis examines trends that are sustained movements in the variable of interest in a specific direction.Trend analysis often takes one of two forms:Linear trend analysis, in which the dependent variable changes at a constant rate over time.Ex: if b0=3 and b1=2.3, then the predicted value of y after three periods is2. Log-linear trend analysis, in which the dependent variable changes at an exponential rate over time or constant growth at a particular rateEx: if b0=2.8 and b1=1.4, then the predicted value of y after three periods is𝑦 𝑡 = 𝑏 0 + 𝑏 1 𝑡+ ε 𝑡LOS: Compute the predicted trend value for a time series modeled as either a linear trend or log-linear trend, given the estimated trend coefficients.Pages 377–381Recall that the inverse process of ln() is raising e to the () power. Because it is the slope of the trend line, b1 is referred to as the trend coefficient. Log-linear growth should be familiar already, it is the foundation for continuous compounding from earlier chapters.ln ( 𝑦 𝑡 ) = 𝑏 0 + 𝑏 1 𝑡+ ε 𝑡
4 Linear or log-linear?How do we decide between linear and log-linear trend models?Is the estimated relationship persistently above or below the trend line?Are the error terms correlated?We can diagnose these by examining plots of the trend line, the observed data, and the residuals over time.LOS: Discuss the factors affecting the choice between a linear trend and a log-linear trend model for a time series incorporating a trend.Pages 381–384The first plot has a linear trend, and the second, an exponential trend. Point out the curvature in the second plot and the fact that the error terms frequently lie above or below the line for consistent periods of time—both signs that your trend (here shown, again, as a linear trend) isn’t the right model.
5 Trend models and serial correlation
Are the results of our trend model estimation valid?Trend models, by their very construction, are likely to exhibit serial correlation.In the presence of serial correlation, our linear regression estimates are inconsistent and potentially invalid.Use the Durbin–Watson test to establish whether there is serial correlation in the estimated model.If so, it may be necessary to transform our data or use other estimation techniques.LOS: Discuss the factors affecting the choice between a linear trend and a log-linear trend model for a time series incorporating a trend.Page 385
Are the results of our trend model estimation valid?Trend models, by their very construction, are likely to exhibit serial correlation.In the presence of serial correlation, our linear regression estimates are inconsistent and potentially invalid.Use the Durbin–Watson test to establish whether there is serial correlation in the estimated model.If so, it may be necessary to transform our data or use other estimation techniques.LOS: Discuss the factors affecting the choice between a linear trend and a log-linear trend model for a time series incorporating a trend.Page 385
6 Autoregressive time-series models
Abbreviated as AR(p) models, the p indicates how many lagged values of the dependent variable are used and is known as the “order” of the model.Current values are a function of prior values.The “order” of the AR(p) models is the number of prior values used in the model.AR(1) 𝑥 𝑡 = 𝑏 0 + 𝑏 1 𝑥 𝑡−1 + ε 𝑡AR(2) 𝑥 𝑡 = 𝑏 0 + 𝑏 1 𝑥 𝑡−1 + 𝑏 2 𝑥 𝑡−2 + ε 𝑡We can use longer interval differences to account for seasonality.LOS: Discuss the structure of an autoregressive model of order p.Page 386Notice the notation of x and y, which we have used for independent and dependent, are now gone because the distinction is a moot point. In a time-series model, we are only focused on the underlying statistical process.𝑥 𝑡 = 𝑏 0 + 𝑏 1 𝑥 𝑡−1 + 𝑏 2 𝑥 𝑡−4 + ε 𝑡
Abbreviated as AR(p) models, the p indicates how many lagged values of the dependent variable are used and is known as the “order” of the model.Current values are a function of prior values.The “order” of the AR(p) models is the number of prior values used in the model.AR(1) 𝑥 𝑡 = 𝑏 0 + 𝑏 1 𝑥 𝑡−1 + ε 𝑡AR(2) 𝑥 𝑡 = 𝑏 0 + 𝑏 1 𝑥 𝑡−1 + 𝑏 2 𝑥 𝑡−2 + ε 𝑡We can use longer interval differences to account for seasonality.LOS: Discuss the structure of an autoregressive model of order p.Page 386Notice the notation of x and y, which we have used for independent and dependent, are now gone because the distinction is a moot point. In a time-series model, we are only focused on the underlying statistical process.𝑥 𝑡 = 𝑏 0 + 𝑏 1 𝑥 𝑡−1 + 𝑏 2 𝑥 𝑡−4 + ε 𝑡
7 Covariance-stationary series
A time series is said to be covariance stationary if its mean and variance do not change over time.Time series that are not covariance stationary have linear regression estimates that are invalid and have no economic meaning.For a time series to be stationary,The expected value of the series must be finite and constant across time.The variance of the series must be finite and constant across time.The covariance of the time series with itself must be finite and constant for all intervals over all periods across time.Visually, we can inspect the time-series model for a mean and variance that appear stationary as an initial screen for likely stationarity.LOS: Explain the requirements for a time series to be covariance stationary, differentiate between stationary and nonstationary time series by visual inspection of time-series plots, and explain the impact of nonstationarity in the context of autoregressive time-series models.Pages 386–387“Weakly stationary” is a synonym for covariance stationary. Most financial time series are unlikely to be stationary, and visual stationarity of the mean and variance do NOT guarantee covariance stationarity.
A time series is said to be covariance stationary if its mean and variance do not change over time.Time series that are not covariance stationary have linear regression estimates that are invalid and have no economic meaning.For a time series to be stationary,The expected value of the series must be finite and constant across time.The variance of the series must be finite and constant across time.The covariance of the time series with itself must be finite and constant for all intervals over all periods across time.Visually, we can inspect the time-series model for a mean and variance that appear stationary as an initial screen for likely stationarity.LOS: Explain the requirements for a time series to be covariance stationary, differentiate between stationary and nonstationary time series by visual inspection of time-series plots, and explain the impact of nonstationarity in the context of autoregressive time-series models.Pages 386–387“Weakly stationary” is a synonym for covariance stationary. Most financial time series are unlikely to be stationary, and visual stationarity of the mean and variance do NOT guarantee covariance stationarity.
8 Residual autocorrelation
We can use the autocorrelation of the residuals from our estimated time-series model to assess model fit.The autocorrelation between one time-series observation and another one at distance k in time is known as the kth order autocorrelation.A correctly specified autoregressive model will have residual autocorrelations that do not differ significantly from zero.Testing procedure:Estimate the AR model and calculate the error terms (residuals).Estimate the autocorrelations for the error terms (residuals).Test to see whether the autocorrelations are statistically different from zero.This is a t-test, which, if the null hypothesis of no correlation is rejected, mandates modification of the model or data.A failure to reject the null indicates that the model is statistically valid.LOS: Explain how autocorrelations of the residuals from an autoregressive model can be used to test whether the model fits the time series.Pages 387–389The Durbin–Watson test is invalid for AR(p) series; hence, we need another test to assess the presence of serial correlation and model fit.
We can use the autocorrelation of the residuals from our estimated time-series model to assess model fit.The autocorrelation between one time-series observation and another one at distance k in time is known as the kth order autocorrelation.A correctly specified autoregressive model will have residual autocorrelations that do not differ significantly from zero.Testing procedure:Estimate the AR model and calculate the error terms (residuals).Estimate the autocorrelations for the error terms (residuals).Test to see whether the autocorrelations are statistically different from zero.This is a t-test, which, if the null hypothesis of no correlation is rejected, mandates modification of the model or data.A failure to reject the null indicates that the model is statistically valid.LOS: Explain how autocorrelations of the residuals from an autoregressive model can be used to test whether the model fits the time series.Pages 387–389The Durbin–Watson test is invalid for AR(p) series; hence, we need another test to assess the presence of serial correlation and model fit.
9 Mean reversion For an AR(1) the values will Stay constant when
A series is mean reverting if its values tend to fall when they are above the mean and rise when they are below the mean.For an AR(1) the values willStay constant whenRise whenFall when𝑥 𝑡 = 𝑏 0 + 𝑏 1 𝑥 𝑡−1 + ε 𝑡LOS: Explain mean reversion and determine whether particular time series are mean reverting.Page 391Mean reversion is extremely common, particularly for a long-run macro-economic series. However, it is often misapplied. It is worth noting that all covariance-stationary time series will have finite mean-reverting levels.
A series is mean reverting if its values tend to fall when they are above the mean and rise when they are below the mean.For an AR(1) the values willStay constant whenRise whenFall when𝑥 𝑡 = 𝑏 0 + 𝑏 1 𝑥 𝑡−1 + ε 𝑡LOS: Explain mean reversion and determine whether particular time series are mean reverting.Page 391Mean reversion is extremely common, particularly for a long-run macro-economic series. However, it is often misapplied. It is worth noting that all covariance-stationary time series will have finite mean-reverting levels.
10 Multiperiod forecasts
We can use the chain rule of forecasting to gain multiperiod forecasts with an AR(p) model.Consider an AR(1) model wherein the estimated 𝑏 0 =3 and 𝑏 1 =2.3.What is the one-step ahead forecast of x1 when x0 = 3?What is the two-step ahead forecast of x2?This is the chain rule of forecasting. We are using the forecast for x1 to then forecast for x2.LOS: Compute the one- and two-period-ahead forecasts of a time series using an autoregressive model.Pages 391–394We can do this indefinitely into the future, but recall that each forecast includes the parameter estimation error. So the forecast for x2 includes the error in the forecast of x1, which itself includes the parameter estimation error plus any other model or data error, and then x2 also “re-includes” the parameter estimation error again because we are using the same parameters we used to get x1 to get x2. It should be easy to see that as we move progressively further into the future, the error of the forecast gets larger as a multiplicative function of the parameter estimation errors. A confidence interval on the forecast must then get progressively larger, quickly making forecasts further out less reliable even if the underlying parameters don’t change over time.
We can use the chain rule of forecasting to gain multiperiod forecasts with an AR(p) model.Consider an AR(1) model wherein the estimated 𝑏 0 =3 and 𝑏 1 =2.3.What is the one-step ahead forecast of x1 when x0 = 3?What is the two-step ahead forecast of x2?This is the chain rule of forecasting. We are using the forecast for x1 to then forecast for x2.LOS: Compute the one- and two-period-ahead forecasts of a time series using an autoregressive model.Pages 391–394We can do this indefinitely into the future, but recall that each forecast includes the parameter estimation error. So the forecast for x2 includes the error in the forecast of x1, which itself includes the parameter estimation error plus any other model or data error, and then x2 also “re-includes” the parameter estimation error again because we are using the same parameters we used to get x1 to get x2. It should be easy to see that as we move progressively further into the future, the error of the forecast gets larger as a multiplicative function of the parameter estimation errors. A confidence interval on the forecast must then get progressively larger, quickly making forecasts further out less reliable even if the underlying parameters don’t change over time.
11 In- and out-of-sample forecasting
In-sample forecast errors are simply the residuals from a fitted time series, whereas out-of-sample forecast errors are the difference between predicted values from outside the sample period and the actual values once realized.An in-sample forecast uses the fitted model to obtain predicted values within the time period used to estimate model parameters.An out-of-sample forecast uses the estimated model parameters to forecast values outside of the time period covered by the sample.In both cases, the forecast error is the difference between the forecast and the realized value of the variable.Ideally, we will select models based on out-of-sample forecasting error.Model accuracy is generally assessed by using the root mean squared error criterion.Calculate all the errors, square them, calculate the average, and then take the square root of that average.The model with the lowest mean-squared error is judged the most accurate.LOS: Explain the difference between in-sample forecasts and out-of-sample forecasts, and contrast the forecasting accuracy of different time-series models based on the root mean squared error criterion.Pages 394–395Refer to the prior forecasting example. Both of the forecasts in the prior example would be out-of-sample forecasts, and the forecast errors (uncalculated) would be the difference between the forecasts and the actual values.
In-sample forecast errors are simply the residuals from a fitted time series, whereas out-of-sample forecast errors are the difference between predicted values from outside the sample period and the actual values once realized.An in-sample forecast uses the fitted model to obtain predicted values within the time period used to estimate model parameters.An out-of-sample forecast uses the estimated model parameters to forecast values outside of the time period covered by the sample.In both cases, the forecast error is the difference between the forecast and the realized value of the variable.Ideally, we will select models based on out-of-sample forecasting error.Model accuracy is generally assessed by using the root mean squared error criterion.Calculate all the errors, square them, calculate the average, and then take the square root of that average.The model with the lowest mean-squared error is judged the most accurate.LOS: Explain the difference between in-sample forecasts and out-of-sample forecasts, and contrast the forecasting accuracy of different time-series models based on the root mean squared error criterion.Pages 394–395Refer to the prior forecasting example. Both of the forecasts in the prior example would be out-of-sample forecasts, and the forecast errors (uncalculated) would be the difference between the forecasts and the actual values.
12 Coefficient instability
Time-series coefficient estimates can be unstable across time. Accordingly, sample period selection becomes critical to estimating valuable models.This instability can also affect model estimation because changes in the underlying time-series process can mean that different time-series models work better over different time periods.Ex. A basic AR(1) model may work well in one period, but an AR(2) may fit better in another period. If we combine the two periods, we are likely to select either the AR(1) or AR(2) model for the combined time span, thereby poorly fitting at least one time span of data.There are no clear-cut rules for selecting an appropriate time frame for a particular analysis.Rely on basic sampling theory Don’t use two clearly different populations.Rely on basic time-series properties Don’t mix stationary and nonstationary series or series with different mean or variance terms.The longer the sample period The more likely the samples come from different populations.LOS: Discuss the instability of coefficients of time-series models.Page 397Selection of appropriate sample periods for time-series estimation is notoriously difficult, but users should follow, at a minimum, basic statistical common sense. For example, don’t mix regimes (different populations).
Time-series coefficient estimates can be unstable across time. Accordingly, sample period selection becomes critical to estimating valuable models.This instability can also affect model estimation because changes in the underlying time-series process can mean that different time-series models work better over different time periods.Ex. A basic AR(1) model may work well in one period, but an AR(2) may fit better in another period. If we combine the two periods, we are likely to select either the AR(1) or AR(2) model for the combined time span, thereby poorly fitting at least one time span of data.There are no clear-cut rules for selecting an appropriate time frame for a particular analysis.Rely on basic sampling theory Don’t use two clearly different populations.Rely on basic time-series properties Don’t mix stationary and nonstationary series or series with different mean or variance terms.The longer the sample period The more likely the samples come from different populations.LOS: Discuss the instability of coefficients of time-series models.Page 397Selection of appropriate sample periods for time-series estimation is notoriously difficult, but users should follow, at a minimum, basic statistical common sense. For example, don’t mix regimes (different populations).
13 Random walks 𝑥 𝑡 = 𝑥 𝑡−1 + ε 𝑡
An AR(1) series where b0=0 and b1=1 is known as a random walk because the best prediction for tomorrow is the value today plus a random error term.Very prevalent in financeUndefined mean-reversion level because b0/(1 – b1) = 0/0 undefinedNot covariance stationaryThere is another common variation, known as a random walk with a drift, where b0 is a constant number that is not zero.LOS: Define a random walk.Page 400Burton Malkiel’s, A Random Walk Down Wall Street, is a reference to this type of series. Demonstrate the first bullet point by pointing out that E(error) = zero; hence, xt = 0 +(1)xt = xt-1 .
An AR(1) series where b0=0 and b1=1 is known as a random walk because the best prediction for tomorrow is the value today plus a random error term.Very prevalent in financeUndefined mean-reversion level because b0/(1 – b1) = 0/0 undefinedNot covariance stationaryThere is another common variation, known as a random walk with a drift, where b0 is a constant number that is not zero.LOS: Define a random walk.Page 400Burton Malkiel’s, A Random Walk Down Wall Street, is a reference to this type of series. Demonstrate the first bullet point by pointing out that E(error) = zero; hence, xt = 0 +(1)xt = xt-1 .
14 Unit rootsFor an AR(1) time series to be covariance stationary, the absolute value of the b1 coefficient must be less than 1.When the absolute value of b1 is 1, the time series is said to have a unit root.Because a random walk is defined as having b1 = 1, all random walks have a unit root.We cannot estimate a linear regression and then test for b1 = 1 because the estimation itself is invalid.Instead, we conduct a Dickey–Fuller test, which is available in most common statistics packages, to determine if we have a unit root.LOS: Explain the relationship between a random walk and unit roots, and discuss the unit root test for nonstationarity.Page 404If the presenter has sufficient time, covering the derivation of the Dickey–Fuller test is a nice transition to first differencing on the next slide
15 Unit roots and estimation
We cannot use linear regression to estimate parameters for a series containing a unit root without transforming the data.“Differencing” is the process we use to transform data with a unit root; it is performed by subtracting one value in the time series from another.Differencing also has an “order,” which is the number of time units the two differenced variables lie apart in time.For a random walk, we first-difference the time series.𝑦 𝑡 = 𝑥 𝑡 − 𝑥 𝑡−1 = ε 𝑡A properly differenced random walk time series will be covariance stationary with a mean-reversion level of zero.LOS: Discuss how a time series with a unit root can be transformed so that it can be analyzed with an autoregressive model.Pages 400–403The differencing process can be used with all time series that exhibit nonstationarity, including those in which the nonstationarity occurs at greater than order 1, although most financial time series will become nonstationary with first-order differencing. A first-differenced variable is often denoted d(1).
We cannot use linear regression to estimate parameters for a series containing a unit root without transforming the data.“Differencing” is the process we use to transform data with a unit root; it is performed by subtracting one value in the time series from another.Differencing also has an “order,” which is the number of time units the two differenced variables lie apart in time.For a random walk, we first-difference the time series.𝑦 𝑡 = 𝑥 𝑡 − 𝑥 𝑡−1 = ε 𝑡A properly differenced random walk time series will be covariance stationary with a mean-reversion level of zero.LOS: Discuss how a time series with a unit root can be transformed so that it can be analyzed with an autoregressive model.Pages 400–403The differencing process can be used with all time series that exhibit nonstationarity, including those in which the nonstationarity occurs at greater than order 1, although most financial time series will become nonstationary with first-order differencing. A first-differenced variable is often denoted d(1).
16 Smoothing modelsThese models remove short-term fluctuations by smoothing out a time series.An n-period moving average is calculated asConsider the returns on a given bond index as x0 = 0.12 , x-1 = 0.14, x-2 = 0.13, x-3 = 0.2.What is the three-period moving-average return for one period ago (t = –1)?What is the three-period moving-average return for this period (t = 0)?LOS: Compute an n-period moving average of a time series.Pages 407–409One weakness of these models is that they will always “turn late” when the actual data turns. Accordingly, there is a class of n-period average models that may perform better.It is worth noting that this is an equally weighted n-period moving average. We can also use unequally weighted n-period moving averages, in which the weight of each past observation is different and all weights sum to 1. By doing so, we can, for example, place more weight on more recent observations (i.e., wt = 0.5,wt-1 = 0.3, wt-2 = 0.2).We often use root mean squared error to assess the quality of an n-period moving-average model.
17 Moving-average time-series models
Called MA(q) modelsNot commonly usedHave order just like AR modelsMA(1) 𝑥 𝑡 = ε 𝑡 +θ ε 𝑡−1Not the same as smoothing models on the prior slide.LOS: Discuss the structure of a moving-average model of order q.Pages 407–409Moving-average models are much less common in financial applications than AR models except when we use more advanced models that include both MA and AR terms.
Called MA(q) modelsNot commonly usedHave order just like AR modelsMA(1) 𝑥 𝑡 = ε 𝑡 +θ ε 𝑡−1Not the same as smoothing models on the prior slide.LOS: Discuss the structure of a moving-average model of order q.Pages 407–409Moving-average models are much less common in financial applications than AR models except when we use more advanced models that include both MA and AR terms.
18 Determining the order of a MA(q)
For a MA(q) model, the first q autocorrelations will be significantly different from 0, and all autocorrelations beyond that will be equal to 0.For a MA(1) such as, 𝑥 𝑡 = ε 𝑡 +θ ε 𝑡−1 , we would expect all autocorrelations beyond the first to be statistically zero.What is the likely MA order of this process?LagAutocorrelationt-Statistic11.4609–6.891221.43845.458931.45896.120440.9875–0.234550.03560.0132LOS: Determine the moving-average order of a time series from the autocorrelations of that series.Pages 409–411This is likely a MA(3) because the size of the autocorrelations and the statistical significance of the autocorrelations drop off rapidly after the third lag.
For a MA(q) model, the first q autocorrelations will be significantly different from 0, and all autocorrelations beyond that will be equal to 0.For a MA(1) such as, 𝑥 𝑡 = ε 𝑡 +θ ε 𝑡−1 , we would expect all autocorrelations beyond the first to be statistically zero.What is the likely MA order of this process?LagAutocorrelationt-Statistic11.4609–6.891221.43845.458931.45896.120440.9875–0.234550.03560.0132LOS: Determine the moving-average order of a time series from the autocorrelations of that series.Pages 409–411This is likely a MA(3) because the size of the autocorrelations and the statistical significance of the autocorrelations drop off rapidly after the third lag.
19 AR(p) vs. MA(q)To determine whether a time series is an AR(p) or a MA(q), examine the autocorrelations.The autocorrelations for an AR model will generally begin as large values and gradually decline.The autocorrelations for a MA model will drop dramatically after q lags are reached, identifying both the MA process and its order.LOS: Distinguish an autoregressive time series from a moving-average time series.Page 410Distinguishing between these time series is more of an art than a science, and has led to a great number of tests designed to assess the quality of the “fit” for time-series models. Such tests are beyond the scope of this book, but those interested are recommended to further research time-series models before moving much beyond a basic AR or MA process.
20 seasonalityTime series that show regular patterns of movement within a year across years.Seasonal lags are most often included as a lagged value one year before the prior value.We detect such patterns through the autocorrelations in the data.For quarterly data, the fourth autocorrelation will not be statistically zero if there is quarterly seasonality.For monthly, the 12th, and so on.To correct for seasonality, we can include an additional lagged term to capture the seasonality.For quarterly data, we would include a prior year quarterly seasonal lag asLOS: Discuss how to test and correct for seasonality in a time-series model.Pages 412–414Balance sheet and income statement data, in particular, will often have seasonality related to the underlying nature of the business (the fourth-quarter retail bump in sales, third-quarter wholesale bump in sales, etc.).𝑥 𝑡 = 𝑏 0 + 𝑏 1 𝑥 𝑡−1 + 𝑏 2 𝑥 𝑡−4 + ε 𝑡
21 Forecasting with seasonal lags
Recall our AR(1) model, wherein the estimated 𝑏 0 =3 and 𝑏 1 =2.3. We have determined that the model, which uses monthly data, has a one-year seasonal component with 𝑏 12 =0.4.What is the one-step ahead forecast of x1 when x0 = 3 and x-12 = 1.1?LOS: Compute a forecast using an autoregressive model with a seasonal lag.Pages 412–414
Recall our AR(1) model, wherein the estimated 𝑏 0 =3 and 𝑏 1 =2.3. We have determined that the model, which uses monthly data, has a one-year seasonal component with 𝑏 12 =0.4.What is the one-step ahead forecast of x1 when x0 = 3 and x-12 = 1.1?LOS: Compute a forecast using an autoregressive model with a seasonal lag.Pages 412–414
22 Autoregressive Moving-Average models
It is possible for a time series to have both AR and MA processes in it, leading to a class of models known as ARMA (p,q) models (and beyond).𝑥 𝑡+1 = 𝑏 0 + 𝑏 1 𝑥 𝑡 …+ ε 𝑡 +θ ε 𝑡−1Although it is an attractive proposition, using an ARMA (p,q) model gains the flexibility of both AR and MA models, but comes at the cost of increased likelihood of parameter instability.Selecting the correct order (p and q) is more art than science.ARMA models generally do not forecast well.LOS: Discuss the limitations of autoregressive moving-average models.Pages 416– 417
It is possible for a time series to have both AR and MA processes in it, leading to a class of models known as ARMA (p,q) models (and beyond).𝑥 𝑡+1 = 𝑏 0 + 𝑏 1 𝑥 𝑡 …+ ε 𝑡 +θ ε 𝑡−1Although it is an attractive proposition, using an ARMA (p,q) model gains the flexibility of both AR and MA models, but comes at the cost of increased likelihood of parameter instability.Selecting the correct order (p and q) is more art than science.ARMA models generally do not forecast well.LOS: Discuss the limitations of autoregressive moving-average models.Pages 416– 417
23 Autoregressive conditional heteroskedasticity
Heteroskedasticity is the dependence of the error term variance on the independent variable.When heteroskedasticity is present, the variance of the error terms will vary with a varying independent variable, thereby violating the underlying assumptions of linear regression.AR models with conditional heteroskedasticity are known as ARCH models.An AR(1) model with conditional heteroskedasticity is therefore, an ARCH(1) model.To test for ARCH(1) conditional heteroskedasticity:Regress the squared residuals from each period on the prior period squared residuals.Estimate: ε 𝑡 2 = 𝑎 0 + 𝑎 1 ε 𝑡−1 2 + ω 𝑡If the estimated slope coefficient, 𝑎 1 , is statistically different from zero, the series exhibits an ARCH(1) effect.LOS: Discuss how to test for autoregressive conditional heteroskedasticity.Pages 417–418
Heteroskedasticity is the dependence of the error term variance on the independent variable.When heteroskedasticity is present, the variance of the error terms will vary with a varying independent variable, thereby violating the underlying assumptions of linear regression.AR models with conditional heteroskedasticity are known as ARCH models.An AR(1) model with conditional heteroskedasticity is therefore, an ARCH(1) model.To test for ARCH(1) conditional heteroskedasticity:Regress the squared residuals from each period on the prior period squared residuals.Estimate: ε 𝑡 2 = 𝑎 0 + 𝑎 1 ε 𝑡−1 2 + ω 𝑡If the estimated slope coefficient, 𝑎 1 , is statistically different from zero, the series exhibits an ARCH(1) effect.LOS: Discuss how to test for autoregressive conditional heteroskedasticity.Pages 417–418
24 Predicting varianceIf a series is an ARCH(1) process, then we can use the parameter estimates from our test for conditional heteroskedasticity to predict next period variance as σ 𝑡 2 = 𝑎 0 + 𝑎 1 ε 𝑡−1 2LOS: Discuss how to predict the variance of a time series using an autoregressive conditional heteroskedasticity model.Pages 417–420
25 CointegrationTwo time series are cointegrated when they have a financial or economic relationship that prevents them from diverging without bound in the long run.We will often formulate models that include more than one time series.If any time series in a regression contains a unit root, the ordinary least squares estimates may be invalid.If both time series have a unit root and they are cointegrated, the error term will be stationary and we can proceed with caution to estimate the relationship via ordinary least squares and conduct valid hypothesis tests.The caution arises because the regression coefficients represent the long- term relationship between the variables and may not be useful for short- term forecasts.We can test for cointegration using either an Engle–Granger or Dickey– Fuller test.LOS: Discuss the effects of cointegration on regression results.Pages 420–424Cointegration is a “good thing” from the standpoint of our ability to obtain valid estimates, but it is a “bad thing” from the standpoint of our ability to interpret those estimates as being economically meaningful and/or to exploit them.
26 Selecting an appropriate time-series model
Focus On: Regression OutputYou are modeling the rate of growth in the money supply of a developing country using 100 years of annual data. You have estimated an AR(1) model and used the residuals to form the table to the right.Is the AR(1) model sufficient?If not, how would you modify it?LagAutocorrelationt-Statistic10.06991.301920.10070.198530.09641.637040.05568.055350.03770.11056–0.0933–0.9724LOS: Select and justify the choice of a particular time-series model from a group of models, given regression output and other information for those models.Pages throughout the chapterThis topic is huge in time series. As the careful reader will note, we generally do not select an appropriate time-series model from the regression output per se, but from the autocorrelations and their statistics.This model looks like it needs a four-year “seasonal” term. It might be worth asking what could explain a four-year seasonal component to the money supply. One possible answer is a political business cycle in which politicians run up the money supply to get themselves reelected. The likelihood of this explanation could be examined by seeing if the country has a four-year term cycle.
Focus On: Regression OutputYou are modeling the rate of growth in the money supply of a developing country using 100 years of annual data. You have estimated an AR(1) model and used the residuals to form the table to the right.Is the AR(1) model sufficient?If not, how would you modify it?LagAutocorrelationt-Statistic10.06991.301920.10070.198530.09641.637040.05568.055350.03770.11056–0.0933–0.9724LOS: Select and justify the choice of a particular time-series model from a group of models, given regression output and other information for those models.Pages throughout the chapterThis topic is huge in time series. As the careful reader will note, we generally do not select an appropriate time-series model from the regression output per se, but from the autocorrelations and their statistics.This model looks like it needs a four-year “seasonal” term. It might be worth asking what could explain a four-year seasonal component to the money supply. One possible answer is a political business cycle in which politicians run up the money supply to get themselves reelected. The likelihood of this explanation could be examined by seeing if the country has a four-year term cycle.
27 SummaryMost financial data are sampled over time and, accordingly, can be modeled using a special class of estimations known as time-series models.Time-series models in which the value in a given period depends on values in prior periods, are known as autoregressive, or AR models.Time-series models in which the value in a given period depends on the error values from prior periods, are known as moving average, or MA models.Models whose error variance changes as a function of the independent variable are known as conditional heteroskedastic models.For an AR dependency, these are known as ARCH models.