**4. Results**

The analysis procedure was organized inductively. First, the results of the forecasts and how they and the smoothing parameters evolve over time were observed for both double and triple seasonal Holt-Winters models, and then, the influence of the size of the adjusting set was analyzed.

#### *4.1. Double Seasonal Models*

Table 2 shows the results of the forecasts made by double seasonal models. In particular, the average of the MAPE when predicting the 250 random samples of two weeks. Only models with the correction of the first order autocorrelation error setting are shown, since they offer better results. Removing the year 2009, the year in which the crisis was emphasized in the Spanish industrial sector until that uncertain moment, the rest of the years there is a clear stability in the forecasts. Holt-Winters models are very robust to small variations. The forecast accuracy values are around 2.6% in terms of MAPE for 24 hours-ahead forecasts. In the triple seasonal case these values are around 7.9% of MAPE.


**Table 2.** Summary of the 24-hour ahead forecasting MAPE of double seasonal models, split by years.

The *NMC*24,168 is selected as it offers the best forecast accuracy. As the MAPE provided is an average each year, a split including the months is graphed in Figure 2 as weather conditions and months may influence on the results. Clearly the winter and autumn months have higher levels of MAPE, this time the winter months when worse forecasts occur.

**Figure 2.** MAPE radar diagram of the forecasts for the *NMC*24,168 model using the demand time series from 2008 to 2017.

Table 3 shows the distribution of the parameter values according to the year and seasonal period for the model *NMC*24,168. These values were obtained using the sets for adjustment composed of the 250 random samples of 8 weeks. The total row is the mean of the parameters if they are calculated without dividing into seasons. It can be seen that there was a shift in the parameter values from autumn to winter. The values for the daily seasonality are in the order of 0.2 to 0.3 with the exception of winter, where it increases to the value of 0.4. For the intra-weekly seasonality it has a value of 0.2 and it becomes 0.5 for the winter season.


**Table 3.** Distribution of the parameters for the *NMC*24,168 model.

Figure 3 shows a representation of the parameters over time. It can be observed how the values follow a pattern according to the period of the year. The level and AR(1) adjustment values are slightly different from those of the previous analysis. The level values are higher while the AR(1) adjustment is lower. This is because when the trend is removed, possible long-term variations are supplied by the level. The MAPE in winter season worsens while the smoothing parameters increase their variability in autumn. The randomness of the method chosen to analyse variability makes many forecasts done in winter to use a model optimised with data from autumn season. The special events during autumn impact greatly on the winter forecasts.

**Figure 3.** Radar diagram of the average of the parameters for the model *NMC*24,168 depending on the time.

This analysis was completed with a study on the influence of the size of the training dataset on prediction accuracy. The date of 11 July 2016 was set for the 24-hours-ahead forecasts during two weeks, and the size of the training dataset increased from 8 weeks to several years. In this way, it is intended to observe the behaviour of the model with respect to the sample size, following the indications of [41].

Figure **??** presents the MAPE of the 24-hour forecasts according to the size of the sample for the *NMC*24,168 and *AMC*24,168 models in order to compare their behaviour. It can be observed that the *AMC*24,168 model presents higher variability and the model starts to stabilize around the average value 1.7% when a sufficiently large set is used, in particular from 20,000 h. However, the *NMC*24,168 model is much more stable, and with relatively small training sets, in particular, with a time series composed of 5000 h, it maintains accuracy values around 1.6%. The *AMC*24,168 model is more unstable due to the trend equation (Equation (6)) as the electricity demand time series has no clear trend as shown in Figure 1. Using a larger dataset helps the *NMC*24,168 model to stabilize while *AMC*24,168 model depends clearly on the season the dataset starts. Although *NMC*24,168 model shows better stability than *AMC*24,168, the smoothing parameter associated with the trend does not tend to 0 in order to converge to the same model. This is due to the parameters are local optimal, but not global optimal. Thus, the need for an analysis of these parameters and their stability is supported.

**Figure 4.** MAPE of the 24-hours ahead forecasts according to the size of the datasets used to obtain the model.

Figure 5 shows the evolution of the value of each parameter according to the size of the data set used for the adjustment of the *NMC*24,168 model. It can be seen how there is a stabilization of the values in an asymptotic way. It is surprising how the values of the parameters associated with the seasonality exchange the weights as the data set grows. It can be noticed some peaks where the values go up. These peaks coincide with the beginning of the series on holidays. In fact, the most intense peaks coincide with the dates of the Immaculate Conception or Christmas. Once again, a stability in the predictions is observed demonstrating a grea<sup>t</sup> predictability. This characteristic does not have the same pattern for all models, and depends largely on the values of the seasonal component, always keeping low values for the *α* smoothing parameter.

**Figure5.**Evolutionofparametersversussizeofdataset.

#### *4.2. Triple Seasonal Models*

In this section, an analysis of the parameters of the triple seasonal Holt-Winters models is carried out. Data sets of size 3 years (53 × 24 × 7 × 3 h) were used to adjust the models. As in the case of double seasonal models, 24-hour forecasts were made for two weeks. Although all triple seasonal methods depending on the seasonality and trend were analyzed, there are no major differences between the models, and the *AMC*24,168,8766 model turns out to be the easiest to study, as it has 6 parameters, including level, trend, seasonality and fit.

The results obtained are shown in Table 4, where the values of the parameters are organized according to the season and year. In particular, the mean and standard deviation of the smoothing parameters obtained using the 80 sets for adjustment composed of 8 random weeks are shown. It can be noticed that the standard deviation for Autumn of the year 2016 is not defined because it was only a random set of 8 weeks into the Autumn. Contrary to what happens in the double seasonal models, and as seen in the previous section, when using sets of size greater than 20,000 h, the values of the parameters are stabilized.


**Table 4.** Distribution of smoothing parameters of the *AMC*24,168,8766 model.

The value of the *δ*(24) parameter stabilizes around 0.3 for any period and with a minimum variability. The value of the *δ*(168) parameter stabilizes around 0.2 and the new parameter *δ*(8766) around 0.08 and 0.1. These values follow the pattern shown in Figure 5, although there is a transfer between the *δ*(168) parameter and the *δ*(8766). The introduction of the third seasonality has forced the model to update the intra-weekly seasonality with more recent data over time. On the other hand, values of *α* stabilize around practically 0, and values of *ϕAR* around 0.95. The existence of so much data makes the AR(1) adjustment practically responsible for adapting the level and makes the level equation redundant. This would imply the elimination of a parameter to be optimized.

In addition, an analysis of the size of the data set and its consequences on the forecasts was carried out in the same way as in the previous case. It was used on the same day, 11 July 2016, where the size of the observed data set was gradually increased and 24-hour forecasts were made over two weeks. The results are shown in Figure 6. An asymptotic evolution towards a MAPE of 2% can be seen. Although the MAPE is triggered and reduced again periodically. The shark fin form responds to an adjusting dataset beginning in the autumn, and finishing in the end of autumn—remember that in autumn the calendar effect was much more important than at other times. One of the main characteristics observed in the triple seasonal models is the variability in predictability, mainly produced by the time of year with more holidays, i.e., autumn.

I  **Figure 6.** MAPE of the forecasts for 24-hours ahead according to the size of the dataset used to obtain the *AMC*24,168,8766 model.

The selection of 11 July 2016 for the forecasts responds to the necessity to avoid many special events nearby. The nearest one is the 1st May, and the series has enough time to react. As the model has low values for alpha, gamma and *δ*8766, near to zero, only the intraday and intraweek seasonal components can deal and react against the irregularities of the series, smoothing the model to newer values. The closer the start of the autumn period is, the less reaction time the model has to smooth the irregularities. These irregularities affect forecasts more intensely. When forecasting in other dates, the same analysis provides a similar fin-shaped graph. The only difference is the minimum and maximum of the MAPE, that depends on the date chosen.

#### **5. Discussion of the Results**

The objective of this section is the empirical analysis of multiple-seasonal Holt-Winters models applied to hourly electricity demand in Spain.

The literature found that analyses the parameters of the models tries to give a solution to the theoretical stability analysis, with the determination of the concept of predictability. However, its application to Holt-Winters models is not direct. It is necessary to carry out an empirical analysis of the models and their forecasts, and from there to draw conclusions about predictability.

A framework was established consisting of the usual process of adjustment and forecasting using a Spanish hourly electricity demand data set provided by REE. The forecasts obtained are then analyzed.

The double seasonal models with an 8 week adjustment period are shown to be robust with respect to predictions. Two different periods, with different characteristics, were used and 24-hour prediction MAPEs of around 2% to 2.6% were obtained. In the models with the best behaviour, the parameters were analyzed, and a direct relationship was found between variability and high values of the smoothing parameters associated with seasonality, and in the periods when a greater number of holidays occur.

The size of the observed data set influences the stability. The MAPE of the *AMC*24,168 model has a variability of 0.2%, which from about 20,000 h is reduced to 0.1%. The *NMC*24,168 model is much more stable—the series did not show a trend either—achieving this stability in sets longer than 5000 h. As the number of observations increases, the smoothing parameters show a stabilization.

In the case of triple seasonal models, a large number of observations are necessary to adjust the model. Therefore, parameter values are stabilized from the beginning. It can be seen how the predictions ge<sup>t</sup> worse when the set of values starts at significant dates in the autumn, a season with many public holidays. Forecasters need to use a dataset avoiding to start in autumn. If no other solution is possible, it is interesting to reduce the dataset size as it is large enough, but not including autumn.

In short, it can be seen that the parameters need a large data set to stabilize, and that the triple seasonal models are not able to improve the double seasonal forecasts due to the calendar effect. As a consequence, it is necessary to develop models that are able to reduce this variability by including the calendar effect in the model.
