Let
be a strictly stationary and weakly dependent time series with mean of zero. In practice,
may represent the demeaned, differenced, detrended or deseasonalized series. At first, it is instructive to emphasize a fact: there are multiple ways to model a time series. For instance, suppose the data generating process (DGP) is an AR(2) with serially uncorrelated errors:
where
can be white noise or martingale difference. Then, we can always rewrite Equation (
1) as an AR(1) with new error
and the new error follows an AR(1) process, so is serially correlated:
where
and
by construction. The point is, the exact form of the DGP does not matter. In this example, it can be AR(1) or AR(2). What matters is the serial correlation of
which can be captured by Equation (
1), or Equation (
2) along with Equation (3) equally well. This example indicates that it is plausible to obtain forecasts based on the parsimonious AR(1) model, as long as the serial correlation in
has been accounted for, even if the “true” DGP is a general AR(
p).
Using the parsimonious model (
2) has two benefits that are overlooked in the forecasting literature. First, notice that
is correlated with
As a result, there is the issue of multicollinearity (correlated regressors) for Equation (
1), but not Equation (
2). The absence of multicollinearity can reduce the variance and improve the efficiency of
which explains why a simple model can outperform a complicated model in terms of out-of-sample forecasting. Second, it is well known that the autoregressive coefficient estimated by OLS can be biased—see Shaman and Stine [
15], for instance. As more coefficients need to be estimated in a complex AR model, its forecast can be less accurate than that of a parsimonious model.
2.1. Iterated Block Bootstrap Prediction Intervals
The goal is to find the prediction intervals for future values where h is the maximum forecast horizon, after observing This paper focuses on the bootstrap prediction intervals because (i) they do not assume the distribution of conditional on is normal, and (ii) the bootstrap intervals can automatically take into account the sampling variability of the estimated coefficients.
The TS intervals of Thombs and Schucany [
2] are based on a “long”
p-th order autoregression:
The TS intervals assume that the error
is serially uncorrelated, because the standard or iid bootstrap only works in the independent setting. This assumption of independent errors requires that the model (
4) be dynamically adequate, i.e., a sufficient number of lagged values should be included. It is not uncommon that the chosen model can be complicated (e.g., for series with a long memory), which contradicts the principle of parsimony.
Actually, the model (
4) is just a finite-order approximation if the true DGP is an ARMA process with AR
representation. In that case, the error
is always serially correlated no matter how large
p is. This extreme case implies that the assumption of serially uncorrelated errors can be too restrictive in practice.
This paper relaxes that independence assumption, and proposes the block bootstrap prediction intervals (BBPI) based on a “short” autoregression. Consider the AR(1), the simplest one:
Most often, the error
is serially correlated, so model (
5) is inadequate. Nevertheless, the serial correlation in
can be utilized to improve the forecast. Toward that end, the block bootstrap will later be applied to the residual
where
is the coefficient estimated by OLS.
But first, any bootstrap prediction intervals should account for the sampling variability of
This is accomplished by running repeatedly the regression (
5) using the bootstrap replicate, a pseudo time series. Following Thombs and Schucany [
2] we generate the bootstrap replicate using the
backward representation of the AR(1) model
Note that the regressor is lead not lag. Denote the OLS estimate by
and the residual by
then one series of the bootstrap replicate
is computed in a backward fashion as (starting with the last observation, then moving backward)
By using the backward representation we can ensure the conditionality of AR forecasts on the last observed value
Put differently, all the bootstrap replicate series have the same last observation,
See Figure 1 of Thombs and Schucany [
2] for an illustration of this conditionality.
In Equation (
9), the randomness of the bootstrap replicate comes from the pseudo error term
which is obtained by the block bootstrap as follows:
Save the residual of the backward regression
given in Equation (
8).
Let
b denote the block size (length). The first (random) block of residuals is
where the index number
is a random draw from the discrete uniform distribution between 1 and
For instance, let
and suppose a random draw produces
then
In this example the first block contains three consecutive residuals starting from the 20th observation. By redrawing the index number with replacement we can obtain the second block
the third block
and so on. We stack up these blocks until the length of the stacked series becomes
denotes the
t-th observation of the stacked series.
Resampling blocks of residuals is intended to preserve the serial correlation of the error term in the parsimonious model. Generally speaking, the block bootstrap can be applied to any weakly dependent stationary series. Here it is applied to the residual of the short autoregression.
After generating the bootstrap replicate series using Equation (
9), next, we refit the model (
5) using the bootstrap replicate
Denote the newly estimated coefficient (called bootstrap coefficient) by
Then, we can compute the iterated block bootstrap
l-step forecast
as
where the pseudo error
is obtained by block bootstrapping the residual (
6). For example, let
Then two blocks of residuals (
6) are randomly drawn, and they are
Notice that
in Equation (
11) represents the
l-th observation of the stacked series
The ordering of
and
in the stacked series (
12) does not matter. It is the ordering of the observations within each block that matters. That within-block ordering preserves the temporal structure.
Notice that the block bootstrap has been invoked twice: first it is applied to
(
8), then it is applied to
(
6). The first application adds randomness to the bootstrap replicate
whereas the second application randomizes the predicted value
To get the BBPI, we need to generate
C series of the bootstrap replicate (
9), use them to fit the model (
5), and use Equation (
11) to obtain a series of the iterated block bootstrap
l-step forecasts
where
i is the index. The
l-step iterated BBPI at the
nominal level are given by
where
and
are the
-th and
-th percentiles of the empirical distribution of
Throughout this paper, we let
To avoid the discreteness problem, one may let
see Booth and Hall [
16]. In this paper we use
and find no qualitative difference.
Basically, we apply the percentile method of Efron and Tibshirani [
17] to construct the BBPI. De Gooijer and Kumar [
18] emphasize the percentile method performs well when the conditional distribution of the predicted values is unimodal. In preliminary simulation, we conduct the DIP test of Hartigan and Hartigan [
19] and find that the distribution is indeed unimodal.
2.2. Direct Block Bootstrap Prediction Intervals
We call the BBPI (
14) iterated because the forecast is computed in an iterative fashion: in Equation (
11), the previous step forecast
is used to compute the next step
Alternatively, we can use the bootstrap replicate
to run a set of
direct regressions using only one regressor. In total there are
h direct regressions. More explicitly, the
l-th direct regression uses
as the dependent variable and
as the independent variable. Denote the estimated direct coefficient by
The residual is computed as
Then, the direct bootstrap forecast is computed as
where
is a random draw with replacement from the empirical distribution of
The
l-step direct BBPI at the
nominal level is given by
where
and
are the
-th and
-th percentiles of the empirical distribution of
There are other ways to obtain the direct prediction intervals. For example, the bootstrap replicate
can be generated based on the backward form of direct regression. Ing [
20] compares the mean-squared prediction errors of the iterated and direct point forecasts. In the next section, we will compare the iterated and direct BBPIs.