1. Introduction
Decision-making is a common, if not daily, issue in a variety of situations, e.g., in most companies where decisions often require inclusion and thus analyses of a variety of parameters. One example that is used as an exemplary case in the present study comes from a study on food sustainability. Here, sugar, meat, fat, and salt consumption appear as crucial parameters, but the relative importance of these factors may be different from stakeholder to stakeholder. In the present case, the stakeholders are experts and politicians. However, such decision-making based on a group of parameters/elements, each being characterized by several indicators, often involves the aggregation of data into a single composite indicator by assigning weights to the single indicators followed by a simple arithmetic summation, despite the obvious problems associated with such a method, like compensation effects [
1]. Nevertheless, using such composite indicators has obvious advantages, e.g., a subsequent ranking of the single elements based on these indicator values will lead to a strict, complete order. However, it is worthwhile to remember that when applying composite indicators, the role of the individual indicators is masked and no longer distinguishable [
1,
2].
A crucial point in such an aggregation process is the establishment of the weight regime, i.e., assigning weights to single indicators based on assumptions of the importance of the single indicators. The process of assigning weights typically involves a group of stakeholders [
3,
4,
5,
6] and can, in principle, be conducted in two different ways: (a) either a group of stakeholders mutually agrees on the weight regime, which may be a troublesome, time-consuming, and thus expensive process, or (b) each stakeholder comes up with their individual weight regime. The advantage of the first method is that eventually, only one single ranking is based on the agreed weight regime, whereas the disadvantage is that the weight regime is based on consensus, which may well be subject to controversy. The advantage of the second method is that each ranking is based on the single stakeholder’s weight regime, whereas the disadvantage is that the result would be several rankings equal to the number of stakeholders, which eventually need to be combined to give a final ranking. In two recent papers [
7,
8], a method, generalized linear aggregation (GLA), was introduced to circumvent this step. The GLA method is based on partial ordering [
7,
8] and results in a weak order based on average ranking, where all stakeholders’ opinions simultaneously are considered. The involvement of several stakeholders is foreseen with various associated consequences, not least from an administrative point of view. On the other hand, the potentially troublesome agreement on one single weight regime is avoided. The objective of the present study is to elucidate the possibility that weight regimes that are constructed simply through a random generation of single weights, eventually saving both time and money, can, and possibly to what extent, substitute the involvement of ‘real’ stakeholders.
It should be mentioned in this connection that such a process is a purely top-down approach, in contrast to a bottom-up one, where employees, experts, or possibly even various political systems promoting specific interests may participate in the decision process as stakeholders.
In the following, the proposed methodology [
7,
8] will be illustrated using the abovementioned example from studies on food sustainability [
8,
9]. In the methodology section, a short introduction to partial ordering and GLA is given.
2. Methodology
2.1. Random Generation of Weight Regimes
Weight regimes were simply generated through random numbers within selected intervals, like [0, 1] or [1, 3], applying the appropriate function in Excel-2021. In all cases described below, three to five different weight regimes were generated.
2.2. Data
The data applied in this study were adopted from the Food Sustainability Index 2021 [
9], which has also been used in a recent paper [
4] describing the influence of stakeholders on the ranking of the 78 countries based on four indicators (
Table 1 and
Table 2).
It should be emphasized that “all indicator scores are normalized to a 0 to 100 scale, where 100 indicates the highest sustainability and greatest progress towards meeting environmental, social, and economic key performance indicators (KPI), and 0 represents the lowest” [
9] (cf. Excel Workbook: Methodology), i.e., for all 4 indicators, the higher the indicator value, the better. Hence, no further treatment of the indicator values was necessary (cf. discussion about normalization in [
7,
8]).
2.3. Partial Ordering
Partial ordering is a relation between objects (here, the 78 countries, cf.
Table 2). The method allows the analysis of the data without any pretreatments, e.g., aggregation of the data into one single indicator. In mathematical terms, partial ordering is based on the “≤” relation (cf. e.g., [
10,
11]). Considering two objects, x and y, where object x is characterized by a set of indicators
rs(x),
s = 1, …, m (in the present case m = 4, cf.
Table 1) is comparable to the object y, which is characterized by an identical set of indicators
rs(y), if and only if the relation x ≤ y holds (see below, Equation (1)):
The application of Equation (1) needs a convention about the orientation of the single indicators, i.e., the larger the value of an indicator, the better. In cases where indicators do not have the same orientation, this will initially be remedied by multiplying these indicator values by −1 to secure the common orientation. Since the single indicator values are not numerically aggregated, the method excludes, by default, possible compensation problems, i.e., a “good” value of an indicator may compensate a “bad” one of another indicator. [
1,
10]. A graphical representation of Equation (1) is the so-called Hasse diagram [
2,
11], visually displaying the partial ordering of the objects.
The Hasse Diagram
In the Hasse diagram, comparable objects are connected through a sequence of lines [
2,
11]. If Equation (1) is not fulfilled for some objects x, y, then x is incomparable with y, denoted by x ‖ y. Such incomparabilities point to the data leading to conflicts between the objects, e.g., x < y for some indicator(s) and y < x for other(s). If, for a subset of the data x, y, the ≤ - relation (Equation (1)) holds for all elements, this set will be denoted as a chain. On the other hand, if for a given subset, x ‖ y for all x, y, this set is called an antichain.
2.4. Generalized Linear Aggregation (GLA)
Although the GLA procedure has been explained in detail previously [
7,
8], a brief explanation is given here.
When the original multi-indicator system (MIS) (denoted as “MIS(old)” to emphasize the role of the aggregation process) is written in the form where r
r are the indicators and e
i the studied elements:
If only the opinion of one single stakeholder is introduced, i.e., applying one single weight scheme, only the aggregation to a single scalar, CI (composite indicator), that subsequently may serve as a ranking index can be formulated using (Equation (3)):
where the selection of weights, g
i, is responsible for the composite indicator CI but is based on a system of indicator values r
iold, where r
iold refers to the MIS(old).
The result of the matrix multiplication, where a row of m entries is acting on each column of matrix MIS, leads to the traditional weighted sum expressing the aggregation process. The difficulty in Equation (3) is not its mathematics, but the way how the weights can be found cf. [
3,
4,
5,
6].
If several stakeholders, s
1, s
2, …, s
t, are considered, the corresponding weight scheme can be summarized in a matrix G (Equation (4)):
The weights bear important information concerning the roles played through the single indicators of an original MIS. The aggregation to a set of single scalars can be formulated as follows:
Equation (5) describes the calculation of a new MIS through matrix multiplication of the weight matrix G (Equation (4)), where each row of G corresponds to a weight regime. In Equation (5), the role of G as an operator Ĝ is stressed; Ĝ denotes the transformed G matrix. Application of Equation (5) is more convenient as it accepts any number of weight regimes/stakeholder opinions.
Following the GLA procedure, a highly enriched Hasse diagram is obtained, i.e., a diagram with a much higher number of comparisons and, simultaneously, a significantly reduced number of incomparisons, U, as a consequence of including all stakeholder opinions, i.e., weight regimes, simultaneously. In general, the procedure will not lead to a strict linear order, as would the result of taking only one weight regime/stakeholder into account.
2.5. Average Ranks
The level structure of the Hasse diagram offers a first approximation to an order. However, as all objects on a level automatically will be assigned identical orders, such an order will cause many tied orders. It is desirable that the degree of tiedness is as low as possible, i.e., a ranking with a low number of incomparabilities, ideally a linear ordering of the single objects. However, since a certain portion of incomparable objects are present, this is not immediately obtainable. Partial order methodology provides a weak order, i.e., where tied orders are not excluded—an average ranking—as described by Bruggemann and Carlsen [
12] and Bruggemann and Annoni [
13]. The above-described GLA procedure significantly enriches the Hasse diagram, thus decreasing U and eventually leading to an average ranking closer to the single linear order.
2.6. Software
All partial-order analyses were carried out using the PyHasse software [
14]. PyHasse is programmed using the interpreter language Python (version 2.6). Today, the software package contains around 140 more or less specialized modules. Selected modules are available from the author.
3. Results and Discussion
In a recent study [
4], the advantageous effect of applying the GLA method was demonstrated. This study included four ”stakeholders” denoted as Expert, Political, Outcome, and Uniform (cf.
Table 3), each giving rise to a weighting scheme, with the single weights ranging from 0 to 1. Subsequently, this weighting scheme (
Table 3) was applied to give an average ranking of the countries shown in
Table 2 [
8]. The results of the GLA, i.e., the resulting Hasse diagram and the average ranking are shown and discussed in detail previously [
8].
The question now arises: if a random generation of weighting schemes would lead to an identical ranking of the 78 countries, thereby making a significant reduction in the use of resources required to achieve the weighting schemes from the single stakeholders.
A ranking based on a single randomly generated weighting scheme may well result in a quite different ranking than the one shown in
Table 3 [
8]. To circumvent this, a series of GLA calculations were performed, with the eventual ranking generated as an average of rankings generated through the individual GLAs. In the present case, five randomly generated weighting schemes were applied. The resulting average rankings are shown in
Table 4, together with the average and the standard deviation (std).
It is immediately noted that the single average ranking from the five GLAs does not appear identical but still quite similar, which itself appears somewhat surprising. This is further verified through the relatively small standard deviations obtained by averaging the five rankings. In
Table 5, the randomly generated weight regime leading to rank a (
Table 4) is shown. In
Figure 1, the original Hasse diagram based on the data in
Table 2 (A) is shown together with the diagram based on the GLA a (B), and for comparison, the diagram following the GLA applying the original weights [
4], as given in
Table 3 (C).
Figure 1A,C is adopted from [
8].
The enrichment of the diagram (
Figure 1B,C) compared to the original (
Figure 1A) is immediately apparent. Thus, the original Hasse diagram has only 1048 comparisons and 1955 incomparisons, whereas the diagram in
Figure 1B displays 2602 comparisons and only 401 incomparisons. The diagram in
Figure 1C displays 2718 comparisons and 285 incomparisons, which for the two diagrams in
Figure 1B,C are further visualized through much slimmer and higher diagrams. The similarity between the diagrams in
Figure 1B,C is striking, although there are differences. However, it must be remembered that this is only an exemplary case. Hence, the average rankings obtained through the five GLAs based on randomly generated weighting schemes remain to be discussed, compared to the rankings obtained by applying the original fixed ranking (cf.
Table 3). In
Table 5, the top 10 and bottom 10 ranked countries are summarized and compared to the ranking previously obtained [
8] (
Table 6).
Looking at the data given in
Table 5, it is obvious that some variations between the original raking and that based on the randomly generated weighting schemes prevail. However, taking into account the standard deviations, a surprisingly good agreement between the two sets of data is immediately noted.
An obvious question now arises: how many randomly generated weighting schemes and subsequent GLAs are necessary to obtain reliable results? To answer this question, a series of weighting schemes were randomly generated, allowing integer values of 1, 2, and 3 for four ‘stakeholders’ (cf.
Table 1). The original dataset (
Table 2) was applied, and the combined average rankings for the GLAs were generated following three, five, and seven GLAs. It should be noted that all weighing schemes, i.e., in total fifteen, were different, meaning that, e.g., for the series of five weighting schemes, none of the weighting schemes for the series of three were reused.
In
Figure 2, the comparison of three series, SH3:
●, SH5:
♦, and SH7:
▲, is visualized. It is immediately clear that a good agreement between the three series prevails. However, a somewhat better agreement between SH5 and SH7 than the agreement with SH3 is noted—although not spectacular. Hence, it is suggested that SH5 appears sufficient and should be the preferred choice in general.
A further obvious question to ask is to what extent the proposed method can be used and what the possible limitations are. Here, it should initially be noted that a satisfactory agreement between the ranking based on random data and those from real stakeholder opinions is striking. However, despite the fact that the method a priori does not suffer from specific limitations, in cases where real stakeholders’ opinions are available, they obviously should be applied.
4. Conclusions and Outlook
The present paper answers the question: Are stakeholders’ opinions redundant? To answer the question, partial order-based methods, generalized linear aggregation (GLA) combined with average ranking, were applied. To simulate—here four—stakeholders’ opinions, randomly generated weight regimes were brought into play, and using GLA, the combined effect of the stakeholders was calculated. The overall effect was retrieved by averaging the outcome of the individual GLAs. The result is compared to that of a previous study applying real stakeholders’ weight regimes, demonstrating close agreement.
In a further study, the effect of the number of simulations was disclosed using three, five, and seven sets of weight regimes for the ranking of the data. Based on these calculations, it is suggested that applying five sets of weight regimes satisfactorily leads to a result that closely mimics what was obtained based on real stakeholders’ opinions.
Application of the suggested method discloses that stakeholders’ opinions may well be redundant and that a combined inclusion of a series of randomly generated weight regimes to substitute opinions of real stakeholders is assumed to be a highly resource-reducing decision-supporting process. It must, however, be stressed that this obviously top-down method excludes advantages possibly obtained through the involvement of employees in the decision process through a bottom-up process. However, obviously, employees may well be stakeholders. In such a situation, obviously ‘real’ data, e.g., the weighting based on employee opinions, will be used.