Long-Term Natural Gas Consumption Forecasting Based on Analog Method and Fuzzy Decision Tree

Round 1

Reviewer 1 Report

This manuscript is fine. No suggested comments or changes.

Author Response

We would like to thank the Reviewer for the positive review, which confirms the quality of our months of work. The review did not suggest any changes.

Reviewer 2 Report

This paper presents a NG consumption prediction using decision tree. My comments are below.
Major:
1. The explanation of notations and equations are not clear enough. Please try to explain every equation right after, and also I suggest adding a notation table if necessary.
2. The compared methods seems pretty old and conventional, is it possible adding some newer methods.
3. In Fig.3, 4, 5, seems like the curve being predicted is pretty obvious, which is monotonically increasing, so it's hard to justify the performance of proposed methods.  

4.  regarding fig 3,4,5, I would recommend having plots of all methods being compared for a clearer demonstration.

5. In Table 6, please double-check the numbers, does 1,80% means 180%? which seems like a typo.

Minor:
Quality of flowchart and figures needs to be improved. e.g. it's sometimes hard to read the numbers in the figure. So please try to adjust the font, and also use vector format instead of compressed images in the paper.

Author Response

We would like to thank the Reviewer for the review and comments. It have improved the quality of our paper. Below our answer to Reviewer’s comments.

Major:

1. The explanation of notations and equations are not clear enough. Please try to explain every equation right after, and also I suggest adding a notation table if necessary.

We included additional explanation of notation under some equations.

 

2. The compared methods seems pretty old and conventional, is it possible adding some newer methods.

 

We described the results of “ML Benchmark” in section 7.2 and added a corresponding entry in table 5. We also added a comment to the results.

 

We also want to mention that: The main problem with comparing our methods with other is that in long-term forecasts authors usually focus on one, two countries or even specific segment of consumption in one country. Those approaches will give better estimates but only for this country. If we want to apply those methods in other countries it simply won’t work. Our goal was different. We want to show approach how to deal with many territorial units on the example of countries in the world. So our approach should be compare with typical methods with automatic selection of parameters, because aim of our methods is “expect to be better than the automatic approach”. The goal is achieved – MAPE in our approach is 37% better then best benchmark. Our methods can be easy applied to any territorial units such as municipality in Poland or county in United Kingdom. We used you all over the world as an example to show the universality of the method.

 

After reading your comment we decided to add one more benchmark - "ML benchmark". We decide to train popular Neural Network. As expected, it achieved slightly better results than the classic approach, but not enough to compete with our approach. The characteristics of gas consumption curves are such that the advantages of more advanced forecasting methods do not apply here (please look at the answer for next question).

 

Our observations confirm researchers who publish “Natural gas consumption forecasting: A discussion on forecasting history and future challenges”. We would like to cite excerpt from this publication. The researches observe that we are now in era of “revival of conventional methods”. “(…) As a data-driven model, AI-based models pay more attention to the data itself, and optimize the model parameters through training to obtain the results. During this process, it is difficult for users to observe the relationship between different variables and the specific form of the model, and hard to explain or modify the results. However, traditional statistical models have strong interpretability. In recent years, researches have shown that when the parameters are selected reasonably, the performance of traditional statistical models is comparable to AI-based models. Therefore, despite the rapid development of various AI-based models, traditional statistical models can still have its effect in the field of natural gas consumption forecasting”

 

3. In Fig.3, 4, 5, seems like the curve being predicted is pretty obvious, which is monotonically increasing, so it's hard to justify the performance of proposed methods.  

In long-term, the consumption of gas are usually monotonically increasing. This is because natural gas is becoming a key fossil fuel which is replacing coal and oil. Contrary to short-term and medium-term forecasting, the key here is to capture the trend and potential changes that cause the system to start behaving different (tipping point). In article we tried to select representative countries and show them as examples. And here is strength of our method, because based on history we can identify tipping points.

 

We also think comparisons of MAPE which are computed for whole countries in the world shows potential of our approach (table 5). MAPE of our approach is about 37% better than best benchmark.

 

4. regarding fig 3,4,5, I would recommend having plots of all methods being compared for a clearer demonstration.

Originally, we put comparison on chars but it make them unreadable. We decide to show MAPE instead in table 6.

 

5. In Table 6, please double-check the numbers, does 1,80% means 180%? which seems like a typo.

It’s mistake – commas ware converted into dots.

Minor:

Quality of flowchart and figures needs to be improved. e.g. it's sometimes hard to read the numbers in the figure. So please try to adjust the font, and also use vector format instead of compressed images in the paper.

We did our best and improved quality of flowchart.

Reviewer 3 Report

Section 1:

-Section 1 introduces forecasting methods which are reviewed in Section 2. The introduction could therefore gain in conciseness as it initially gives the impression of documenting a literature review but at a somewhat vague level.

-Is the approach used of predicting future natural gas consumption based on previous history for groups of countries, able to capture future changes in the behavior of a country that would lead a given country to stop belonging to the group? For example, rate and type of alternative (e.g., renewable) energy sources integration including synthetic fuels (e.g. renewable methane), supported by policies.

Section 2:

-Overall in this section, it is recommended to define key terminologies employed throughout the text to the reader (for example, “gray models”, “scenario analysis”, etc), and to review and analyze the state of the art documented in recent publications cited in greater depth with more detail to lead to demonstrated research gaps.

-In an introductory part of this section the concept of peak oil and peak gas would be valuable to briefly summarize.

-Please document Hubbert model principles, and refer more specifically to successful / unsuccessful applications of Hubbert model (countries, timelines).

-Several models are stated to lack applicability to small countries “due to their size and methods”– please quantify size, describe methods and analyze/summarize in greater depth why their applicability is considered limited.

-Please name typical “macroeconomic variables”.

-What are the methods referred to in “by aggregating forecasts made with different methods”.

Section 5-6

-The list of countries considered and clusters and timelines for historical data should be documented at least in Section 5 or 6.

-Are the countries listed in Table 5 (forecasting errors) the complete list of countries analyzed? On what basis were these countries selected?

-None of the variables in Table 1 directly relate to alternative energy sources, such as renewables, nuclear. Could it be explained why such variables are not considered required, and has this been verified.

-Could the fact that number of heating days was not found to be important be related / justified based on the share of gas use for heating (buildings).

-Should the effect of climate change be considered if “long-term forecasting”is the goal based on the manuscript title “Long-term natural gas consumption forecasting”…?

-How is “long-term” defined in “long term natural gas consumption forecasting”…?

 

Section 8:

-Are the timelines in Figures 3-6 adequate for model validation, i.e. to evaluate confidence in the model to predict longer term evolution of natural gas consumption in the future, particularly considering the manuscript title "Long-term natural gas consumption forecasting”…? Please consider whether the manuscript title is appropriate.

-Over what future timelines could the model be applied with confidence?

-Does the model apply to both conventional and unconventional gas, with no distinction between these categories of resources? Discussion should be included on this aspect in the manuscript.

 

References:

-It is recommended to include a higher number of recent references covering the state of the art.

General:

-The manuscript would gain to be more concise overall.

-Abbreviations should be defined on first use (e.g., GDP in the abstract, MAPE, ARIMA in introduction, etc).

-Please avoid consecutive section headings without text in between.

-Inclusion of line numbers of the manuscript would facilitate peer review.

Author Response

Thank you for positive review and comments. It have improved the quality of our paper. Below we answer for your comments.

Section 1:

-Section 1 introduces forecasting methods which are reviewed in Section 2. The introduction could therefore gain in conciseness as it initially gives the impression of documenting a literature review but at a somewhat vague level.

We move paragraph that reviews literature about analog forecasting from section 1 to section 2. We agree that It improves quality of reading.

-Is the approach used of predicting future natural gas consumption based on previous history for groups of countries, able to capture future changes in the behavior of a country that would lead a given country to stop belonging to the group? For example, rate and type of alternative (e.g., renewable) energy sources integration including synthetic fuels (e.g. renewable methane), supported by policies.

We add explanatory sentence at the end of section 6.1:

“It is visible that analogue approach can capture changes in the economic behavior of a country by changes in a group membership. Synthetic indicators of Energy GDP and Energy_mix partially considering the technological development of the country. The combination of these two variables also includes the impact of increased use of alternative energy sources.”

 

Our comments to alternative/renewable – please look at the answer for comment “None of the variables in Table 1 directly relate to alternative energy sources, such as renewables, nuclear. Could it be explained why such variables are not considered required, and has this been verified.”

 

Section 2:

-Overall in this section, it is recommended to define key terminologies employed throughout the text to the reader (for example, “gray models”, “scenario analysis”, etc), and to review and analyze the state of the art documented in recent publications cited in greater depth with more detail to lead to demonstrated research gaps.

 

We define some key terminology and once more check literature about long-term forecasting. Unfortunately nowadays researchers focus on short and middle term forecasting. Mainly because of development of big data and AI. Today we observe the opposite phenomenon. The growing interest of the business community in long-term gas consumption forecasts. We add some new references.

 

-In an introductory part of this section the concept of peak oil and peak gas would be valuable to briefly summarize.

We add paragraph in section two which summarized concept of gas peak.

-Please document Hubbert model principles, and refer more specifically to successful / unsuccessful applications of Hubbert model (countries, timelines).

We add paragraphs in section two.

 

-Several models are stated to lack applicability to small countries “due to their size and methods”– please quantify size, describe methods and analyze/summarize in greater depth why their applicability is considered limited.

Thank you for this comment. We rewrite the paragraph to better presents our opinion. We change it to:

“There are many forecasts of natural gas demand in the US, Canada, China (e.g. [21],[22]) and Australia, but their accuracy is worse for smaller countries then countries specific-models. The trend of building country-specific forecasts is very visible in the literature. Policymakers are interested in the relationship between consumption and country-specific variables. Unfortunately, the literature on forecasting for middle markets like e.g. Polish one is quite scarce and includes: [23] (logistic model), [24] (neural networks, medium-term forecasting for a single urban area).”

-Please name typical “macroeconomic variables”.

We included explanation of macroeconomic variables (GDP, population, unemployment rate etc.) on page 3.

Typical variables used in forecasting are explained on page 10 (section 5.1)

-What are the methods referred to in “by aggregating forecasts made with different methods”.

We added explanatory paragraph in text

Section 5-6

-The list of countries considered and clusters and timelines for historical data should be documented at least in Section 5 or 6.

Due to readability of article we decided to put only characteristic of the cluster without detailed data. We put additional comment to this paragraph:

“The study used publicly available data from several sources. The annual natural gas consumption and total annual energy consumption has been taken from the BP Statistical Review of World Energy [62]. Our dataset covers all countries from this dataset. The data covers 79 countries around the world for the period 1965-2019.”

-Are the countries listed in Table 5 (forecasting errors) the complete list of countries analyzed? On what basis were these countries selected?

Yes, errors was computed for all countries. We changed title of Table 5

-None of the variables in Table 1 directly relate to alternative energy sources, such as renewables, nuclear. Could it be explained why such variables are not considered required, and has this been verified.

We add explanation after table 1

“None of the above-mentioned variables is directly related to renewable energy sources. But it is indirectly represented in NG_mix. Natural gas with lower emissions per unit of energy replaces coal in power plants due to its low emissivity during combustion. So this variable describes the relationship between fossil fuels and re-newables as gas which will become the dominant fossil fuel in more and more coun-tries. This approach is consistent with the literature [41]. While alternative energy sources and nuclear power are listed in this paper as potential variables for long-term forecasting, in fact these variables were only used in one study [61]. In [61] goal was to build a global gas demand model. In this model, the percent of consumption of alter-native energy sources in energy mix was adopted as a variable. So this use of energy mix was similar to presented in this article.”

-Could the fact that number of heating days was not found to be important be related / justified based on the share of gas use for heating (buildings).

Probably statistical insignificance is due to much higher industrial natural gas consumption in relation to communal one. Also please consider answer to comment above.

-Should the effect of climate change be considered if “long-term forecasting”is the goal based on the manuscript title “Long-term natural gas consumption forecasting”

Please see answer for question “How is “long-term” defined in “long term natural gas consumption forecasting”…?”

-How is “long-term” defined in “long term natural gas consumption forecasting”…?

We added two paragraphs – one in section 1.

(…) Long-term forecasting is an essential part of the strategic decision-making process. In the case of the natural gas market, such decisions usually concern the situation that gas sellers wants to know the demand on territorial area in order to organize their supply chain and plan the infrastructure. It is assumed that the minimum period for this type of decisions is five years. (…)

We also add paragraph in section 8.1.

In literature, building a forecast is three step process: define a model, check per-formance and produce forecast. In literature, "check performance" means comparison of fitting forecast and measured data. For long-term forecast it's usually a five to ten years period (e.g. [30]). Goal of this publication is to show the methodology for long-term, territorial gas consumptions. To do this we should build "ex ante" measures that estimate expected forecasting quality. This process for all countries will be time-consuming and exceed the scope of the publication. So technically we end up with checking performance but it is sufficient for our goal. Our model can be used in forecast longer than ten years but it associated with checking "ex-ante" measures every time. 

Section 8:

-Are the timelines in Figures 3-6 adequate for model validation, i.e. to evaluate confidence in the model to predict longer term evolution of natural gas consumption in the future, particularly considering the manuscript title "Long-term natural gas consumption forecasting”…? Please consider whether the manuscript title is appropriate.

In our opinion title is adequate. Please see answer to: How is “long-term” defined in “long term natural gas consumption forecasting”…? We hope it dispel your doubts.

 

-Over what future timelines could the model be applied with confidence?

Our internal test for our countries show that it can be applied with confidence for even 15 years. We work for article that show how we build and estimate “ex-ante” measures for Poland.

 

-Does the model apply to both conventional and unconventional gas, with no distinction between these categories of resources? Discussion should be included on this aspect in the manuscript.

Our model applies only to natural gas consumption.

References:

-It is recommended to include a higher number of recent references covering the state of the art.

We try to show at least all important long-term forecasting methods. We check again and add some more references

General:

-The manuscript would gain to be more concise overall.

-Abbreviations should be defined on first use (e.g., GDP in the abstract, MAPE, ARIMA in introduction, etc).

Abbreviations are explained now in the body of the paper.

-Please avoid consecutive section headings without text in between.

We are with accordance with Energies template in which there is no text between main titles and subtitles

-Inclusion of line numbers of the manuscript would facilitate peer review.

We turned on line numbers in manuscript

Reviewer 4 Report

In an overall review, the manuscript complies with the following:

Is the writing style concise with no repetition or padding?
Yes, the writing style is concise with no repetition or padding.

Is interest kept throughout and do any sections tend to drag?
Yes, the manuscript kept me interested throughout the sections.

Can the manuscript be better structured?
The manuscript is well structured.

The manuscript is very well detailed and explained, just tell the authors to fix the following problems in tables and graphs:

Table 2, 3 and 4: Use a fixed number of decimal places, in some cases the authors used one decimal place, in some cases two, in some cases three. Please unify.

Table 3: The coefficients in italics are hard to see, please use a letter with another colour or mark the cell in another colour.

Figure 1: In figure 1, the authors use a comma as a decimal separator, while in the text they use points. Please check.

Figures 3, 4 and 5: The authors use different scales on the y-axis, in order to be comparable all graphs must have the same scale on the y-axis.

Author Response

Table 2, 3 and 4: Use a fixed number of decimal places, in some cases the authors used one decimal place, in some cases two, in some cases three. Please unify.

We unified table 3 and 4 and in Table 2 we set one decimal points. In our opinion this setup gives tables good readability

 

Table 3: The coefficients in italics are hard to see, please use a letter with another colour or mark the cell in another colour.

We made this and it improved readability – thank you for comment.

 

Figure 1: In figure 1, the authors use a comma as a decimal separator, while in the text they use points. Please check.

We converted commas into dots.

 

Figures 3, 4 and 5: The authors use different scales on the y-axis, in order to be comparable all graphs must have the same scale on the y-axis.

Unfortunately, we cannot unify them because of different levels of gas consumption. If we made this it would cause figures to be unreadable.

 

Round 2

Reviewer 2 Report

I can see that the authors have addressed most of my comments and concerns, and the manuscript has been improved. However I might still have one more question, since the authors mentioned that they are able to predict tipping points, but in Fig. 5 (a) for example it seems like tipping points have not been correctly reflected. Please help address this concern.

Author Response

Our method of analogues is able to predict changes in membership to a particular cluster due to the level of development of a country measured by energy mix and energy intensity of economy. Tipping points can arise when a country moves from one group of analogies to another one. In our approach the source of tipping points is history of other objects. We cannot  predict tipping points caused by external factors. After investigation we found that in case of Poland the drop in natural gas consumption in years 2012-2014 resulted from very warm winters that years. It is impossible to predict such changes without long-term weather forecasts, which are not credible in reality.

Changes in membership to the clusters of energy consumption are pretty good visible for Algeria (Figure 3a) and Germany (Figure 4a) at the end of forecasted period. Unfortunately that changes have not reflected actual consumption of natural gas for those countries.

It also worth mentioning that our approach is much better than neural networks which are usually used to determine tipping points in e.g. combining forecast.

In broader perspective our method can predict changes in behavior of territorial units, but some changes come from outside and it my cause potential errors.

We added sentence in line 772:

Our method of analogues is able to cover changes in membership to particular clusters of natural gas consumption due to the level of development of a country measured by energy mix and energy intensity of economy. Our forecasting method, like other long-term ones, is unable to predict political, social or weather changes.