Delphi Method Supported by Forecasting Software

Abstract

The Delphi method is one of the basic tools for forecasting values in various types of issues. It uses the knowledge of experts, which is properly aggregated (e.g., in the form of descriptive statistics measures) and returns to the previous group of experts again, thus starting the next round of forecasting. The multi-stage prediction under the Delphi method allows for better stabilization of the results, which is extremely important in the process of forecasting. Experts in the forecasting process often have access to time series forecasting software but do not necessarily use it. Therefore, it seems advisable to add to the aggregate the value obtained using forecasting software. The advantage of this approach is in saving the time and costs of obtaining a forecast. That should be understood as a smaller burden on data analysts and the value of their work. According to the above mentioned key factors, the main contribution of the article is the use of a virtual expert in the form of a computer-enhanced mathematical tool, i.e., a programming library for a forecasting time series. The chosen software tool is the Prophet library—a Facebook tool that can be used in Python or R programming languages.

1. Introduction

The prediction of phenomena and processes accompanies humanity almost from the very beginning. In ancient times, even priests were responsible for this process, e.g., the priestess Pythia in the Temple of Apollo. In modern times, this process is implemented using mathematical methods and expert knowledge. Mathematical methods often are based on advanced statistical apparatuses [1] or artificial intelligence algorithms [2,3]. In turn, the experts’ methods are based on the knowledge and experience of an expert in a specific field [4], and in those cases, the advanced mathematical tools are not emphasized. In both areas, there is interest among scientists and forecasting practitioners in what is shown in the numbers of the scientific works in that field.

One method of predicting a value in a particular phenomenon is the Delphi method [5]. It relies on using the knowledge of experts in a given area of consideration. Experts predict the value of a given event through surveys. This knowledge is then aggregated and, in the form of measures of descriptive statistics (e.g., arithmetic average), returns to the assessment by the previous group of experts. This process is repeated until a satisfactory (stable) descriptive measure of the data is achieved. This method is widely used in long-term forecasting issues.

One of the variants of the Delphi method is the fuzzy Delphi method [6]. The difference between these methods lies in the way the experts express the assessment. In the fuzzy variant, experts reveal their assessment using a fuzzy value in the form of a triangular fuzzy number, i.e., they define three numerical values, which are successively the smallest, most likely, and the largest value that a given process can achieve.

The Delphi method is not without flaws, as it is quite expensive in its implementation process [7,8]. These include the cost and time of getting answers from experts. Each of the experts will demand adequate remuneration for their work, which can be quite large in the case of a multi-stage repetition of surveys. Another critical issue is the time it takes to collect the answers.

The main motivation of this work was the idea of implementing new features to the traditional and fuzzy Delphi methods. According to the previous paragraph, both Delphi methods face their own problems. To address these issues, the authors of this article propose using a so-called virtual expert that predicts values in a given process. Such a tool could be, for example, the Prophet [9] library created by Facebook, designed for issues related to predicting values in given phenomena. The experts may support themselves by different tools, among other programming tools; however, the direct use of values from the Prophet library, or any other library, is a new approach. To our best knowledge, there was no other proposal of that approach in the Delphi method.

This article is divided into the following parts: the first section discusses the Delphi method and the Prophet library. The second part presents two variants of the Delphi method supported by the Prophet library. The next part is an example illustrating the process of fuzzy forecasting using the Delphi method.

1.1. Delphi Method and Related Work

The Delphi method, in its classic form, is described by following steps [5,10,11]:

• Experts fill out a questionnaire, in which they formulate forecasts on a specific topic, referring to a long-time perspective.
• In the next round of surveys, respondents complete the same questionnaire, with information in the form of descriptive statistics on the aggregate results of the previous survey.
• We repeat the surveys until the statistical results obtained stabilize.

The Delphi method is characterized by, among others, the following properties [4]:

• anonymity of opinions and experts;
• multi-stage;
• controlled feedback; and
• statistical data presentation.

The classic Delphi method is used in many issues, including: factors related to the quality assessment of healthcare [12], transport [13], social sciences [14], qualitative research [15] and nursing [16].

One of the variants of the Delphi method is the fuzzy Delphi method. It uses the concept of fuzzy set A in the space X [17]:

$$A=\{(x,{\mu }_A(x)):x\in X\}.$$

(1)

The fuzzy sets are defined by the function ${\mu }_A:X\to [0,1]$, which is called the membership function of the set A and it determines the membership of the element x to the set A.

Fuzzy sets are widely used in common engineering problems, e.g., [18,19].

One of the most common membership functions is the triangular membership function defined by the formula:

$$(a,b,c)=\left\{\begin{array}{cc}0,& x⩽a\\ \frac{x-a}{b-a},& a<x⩽b\\ \frac{c-x}{c-b},& b<x⩽c\\ 0,& x>c\end{array}\right..$$

(2)

The values $(a,b,c)$ can be interpreted as the smallest, most probable, and highest.

Sometimes in order to use the fuzzy Delphi method, other membership functions are used, e.g., the trapezoidal membership function [20] or functions, which are appropriately matched to the issue [21]. The use of the triangular membership function affects the simplicity and transparency of the discussed modification of the Delphi method.

Using the above dependencies, the fuzzy Delphi method can be determined using the steps:

• Experts fill out a questionnaire, in which they formulate forecasts on a specific topic in the form of fuzzy triangular numbers (2) referring to a long-time perspective.
• In the next round of surveys, the respondents complete the same questionnaire, while they have information in the form of statistics on the aggregate results of the previous survey. An example of such statistics is the fuzzy average $\overline{A}$, which for triangular fuzzy sets $A_i=(a_i,b_i,c_i)$, is defined by the formula [22]:

  $$\overline{A}=\left(\frac{1}{n}\sum _{i=1}^n{a}_i,\frac{1}{n}\sum _{i=1}^n{b}_i,\frac{1}{n}\sum _{i=1}^n{c}_i\right).$$

  (3)
• We repeat the surveys until the statistical results obtained stabilize.
• The forecast value is the value from the last fuzzy average expressed as [22]:

  $$\overline{x}=\frac{a+2b+c}{4}.$$

  (4)

Applications of the fuzzy Delphi method can be observed (among others), for example in: choosing a player [23], fashion styles [24], evaluation of hydrogen production [25], lubricant regenerative technology selection [6], teaching methods [26], or in the creation of teaching applications [27].

A variant of the fuzzy Delphi method is described in [28]. It uses the concept of a Z-number, i.e., an ordered pair of fuzzy numbers [29]. In this variant, experts express their assessment in the form of a fuzzy triangular number, adding the reliability of their forecast. This approach seems very promising, and the Z-numbers themselves are gaining popularity in various applications, e.g., decision support [30,31], or the selection of a user group for user experience (UX) research on applications [32].

Apart from the use in the research variant of the Delphi method, its "quality" is based on the quality of the expert’s opinion [33]. That approach does not mean that all experts invited to the research should have scientific degrees and rich branch experience. The empirical examples show that differentiation of their opinions and the experts themselves may have a positive influence on the research and the final result [34]. Furthermore, in the literature, one can also find articles devoted to other technical aspects of the Delphi method, such as who should be considered an expert [35], how many experts should be invited to study [4], and other general schemes on the subject [36]. In addition, in the case of the fuzzy Delphi method, modifications often occur taking into account other methods, such as the analytic hierarchy process (AHP) [37] or fuzzy version of the technique for order of preference by similarity to ideal solution (TOPSIS) [38].

1.2. The Prophet library

The Prophet library [39] is a Python and R language library for time series prediction developed by Facebook under an open-source license. A data set is the basis for calculations, and the data must be in numerical form.

There are two reasons for creating software for forecasting time series [39]:

• A completely automated approach is often insensitive to taking into account useful assumptions or heuristics;
• Analysts who can prepare high-quality forecasts are a scarce resource that requires knowledge of data science and considerable experience.

The above reasons clearly show that the requirements for high-quality analyses go beyond the capacity of data analysts. This approach was the motivation to create the Prophet library [39].

Not all predictive problems can be solved in the same way. As the Prophet library is associated with Facebook and developed for the needs of business forecasts, the issues must include one of the following characteristics [39]:

• hourly, daily, or weekly observations of at least several months;
• seasonality regarding human behavior;
• prediction disorders that occur within predefined intervals;
• reasonable number of missing observations or extreme observations;
• historical trend changes; and
• situations when the trend increases based on non-linear curves when it reaches its natural limits or saturates.

The Prophet library is not entirely automatic. If the forecast is not satisfactory, then the person responsible for creating it can improve the obtained results by changing the available parameters [39].

The critical success factor for proper use of the Prophet library is data preparation. The data for analysis must be daily or monthly. An important aspect is to ensure a significant number of observations. In the absence of large amounts of data, it is necessary to perform a resampling technique or change the forecast time horizon [40].

The Prophet library is based on an additive model in which [41]:

• long-term trends are included;
• the seasonality in Fourier ranks is taken into account;
• holidays and business situations that affect the forecast are included; and
• a random value is included that specifies a random error.

Among others, it is worth mentioning that the effort for successful use of the Prophet library is relatively small to knowledge and experience gained by experts for years. Of course, the user should know the basics of R or Python programming languages, and the programming environment for work should be prepared. Being familiar with those issues is a much easier task than, for example, understanding and using statistics for forecasting methods. The conclusion is being that intuitional interface and proper documentation of the library make it useful with little effort for non-experts in the field of forecasting of time series.

2. Method

The Delphi method supported by forecasting tools can be used in the traditional and fuzzy variant.

2.1. Delphi Method Supported by the Prophet Library

• Experts $E_i(i=1,2,\dots ,n-1)$ present a forecast about the given process in the form of values:

  $$A_i=x_i.$$

  (5)
• From the expert’s opinions, the $A_n$ forecast from the suitable software system expressed as a value:

  $$A_n=x_n$$

  (6)

  is appended.
• Then statistical measures are calculated, e.g., the average forecast value:

  $$\overline{A}=\left(\frac{1}{n}\sum _{i=1}^n{x}_i\right).$$

  (7)
• The value of $\overline{A}$ goes to experts, thus starting the next round of forecasting.
• Repeat steps 1–4 until the value of (7) reaches the appropriate level of stability or for a certain number of turns.
• The value of the arithmetic mean $\overline{A}$ from the last round is the forecast value.

2.2. Fuzzy Delphi Method Supported by the Prophet Library

This version of the Delphi method assumes that the forecasting tools allow specifying not only the forecast value but also its limits, i.e., the lowest and highest possible forecast values. The fuzzy Delphi method supported by the expert forecast in the form of forecasting software can be described in the following steps:

• Experts $E_i(i=1,2,\dots ,n-1)$ present a forecast about the given process in the form of fuzzy triangular numbers:

  $$A_i=(a_i,b_i,c_i).$$

  (8)
• The $A_n$ forecast from the suitable software system that is expressed as three values:

  $$A_n=(a_n,b_n,c_n)$$

  (9)

  is appended to the expert’s opinions .
• Then the fuzzy average is calculated according to the relationship (3):

  $$\overline{A}=\left(\overline{a},\overline{b},\overline{c}\right)=\left(\frac{1}{n}\sum _{i=1}^n{a}_i,\frac{1}{n}\sum _{i=1}^n{b}_i,\frac{1}{n}\sum _{i=1}^n{c}_i\right).$$

  (10)
• Using the relationship (4), the crisp value from the fuzzy average $\overline{A}$ is calculated:

  $$\overline{x}=\frac{\overline{a}+2\overline{b}+\overline{c}}{4}.$$

  (11)
• The value of $\overline{x}$ goes to experts, thus starting the next round of forecasting.
• Repeat steps 1–5 until the value of (11) reaches the appropriate level of stability or for a certain number of turns.
• The value of crisp $\overline{x}$ from the last round is the predicted value in the process.

2.3. Discussion and Method Analysis

Advantages

Regarding the Delphi method, the advantage of the proposed approach is the time and cost of obtaining the forecast. In the case of a cost-saving variant, no further expert is required. In turn, they are saving time because there is no need to wait for an expert’s opinion, which can appear at any time of the assumed time horizon. This forecast can be made by the person who orders the expert opinions after receiving feedback from experts.

Modifications

The presented Delphi methods can be modified by adding a virtual expert opinion (the Prophet library) to the subsequent forecasting rounds. The influence of the Prophet library on the final grade will then be more significant. In this case, however, there will be some difficulties that result from updating the Prophet library forecast—further fresh observations are needed to be able to correct the estimates. One solution to this problem is to tune the library model (using various available parameters) in such a way that the new forecast is corrected accordingly. Another solution that requires no further action is to use the Prophet prediction without updating it.

Other forecasting software

It is also worth noting that the Prophet library is not the only tool used to predict time series. Not including paid computing platforms like Matlab or Statistica, Python can specify at least several other libraries for the same task. A list of selected Python libraries can be found below:

• TensorFlow [42]: “time series forecasting using Recurrent Neural Networks (RNNs)”,
• PyFlux [43]: “offers a probabilistic approach to time-series modeling”,
• Statsmodels [44]: “provides classes and functions for the estimation of many different statistical models, as well as for conducting statistical tests, and statistical data exploration”.

Compared to the libraries mentioned above, the advantage of Prophet is that it has a reasonably intuitive interface that allows it to be used even by non-specialists in the field of time series analysis. Additionally, it is one of the few libraries that is developed by a vast entity, which is Facebook. This ensures that the Prophet should be more resistant to all kinds of errors in the software development process compared to other forecasting tools.

3. Example

The Delphi method is often used for the forecasting of different processes and phenomena over long periods, where a high level of uncertainty appears. That is the main reason to present below the fuzzy variant of the method. The example is made for showing the possibility of the appearance of a virtual expert in the group of human experts.

The fuzzy Delphi method supported by the Prophet library presents the EUR/USD exchange rate in two months. The data for the model creation were collected from the [45] website. Their graph with the Prophet forecast can be seen in Figure 1.

Expert forecasts, together with Prophet forecast, are shown in Figure 2 and

Table 1. Round 1 of the fuzzy Delphi method.

Expert	a	b	c
1	1.112	1.167	1.235
2	1.077	1.146	1.222
3	1.056	1.138	1.139
4	1.085	1.142	1.184
5	1.112	1.168	1.211
Prophet	1.11	1.155	1.2

. The calculated fuzzy mean value and crisp value can be found in Figure 3 and

Table 2. Fuzzy mean and defuzzified value for rounds 1 and 2.

Round	Fuzzy Average	Crisp Value
1	(1.092, 1.153, 1.199)	1.148
2	(1.112, 1.152, 1.185)	1.150

. Then the second round of forecasts begins, in which experts view the value of $\overline{x}=1.148$. The results are shown in

Table 3. Round 2 of the fuzzy Delphi method.

Expert	a	b	c
1	1.106	1.153	1.192
2	1.131	1.157	1.197
3	1.102	1.147	1.18
4	1.101	1.149	1.154
5	1.123	1.153	1.183
Prophet	1.11	1.155	1.2

. This round ends with a recalculation of the fuzzy average and crisp value (Table 2). The final forecast for the euro-dollar exchange rate is $\overline{x}=1.150$.

4. Conclusions

The article presents the Delphi method as supported by forecasting software. For this purpose, the Prophet library was used. The Prophet is a programming library for forecasting, which may be used by Python or R programmers. In the discussed approach, the Prophet library was treated as a kind of virtual expert—its forecasts are attached to the aggregated knowledge that goes to the group of experts in the following rounds. In this way, the classic and fuzzy Delphi methods were discussed. The most crucial feature of the Prophet library can be used in both traditional and fuzzy methods. The advantage of using such an expert is in the time and cost of obtaining the forecast. The Delphi method with the Prophet library is illustrated by the example of a euro to dollar forecast. The mentioned example contains the fuzzy Delphi method, which allows coping with a level of uncertainty in the data. The future work for this research is to compare other forecasting softwares to the Delphi method. They will have to contain a UX (user experience) approach, the mathematical background of each software and the simplicity of use.