Group Decision-Making Based on Artificial Intelligence: A Bibliometric Analysis

Abstract

Decisions concerning crucial and complicated problems are seldom made by a single person. Instead, they require the cooperation of a group of experts in which each participant has their own individual opinions, motivations, background, and interests regarding the existing alternatives. In the last 30 years, much research has been undertaken to provide automated assistance to reach a consensual solution supported by most of the group members. Artificial intelligence techniques are commonly applied to tackle critical group decision-making difficulties. For instance, experts’ preferences are often vague and imprecise; hence, their opinions are combined using fuzzy linguistic approaches. This paper reports a bibliometric analysis of the ample literature published in this regard. In particular, our analysis: (i) shows the impact and upswing publication trend on this topic; (ii) identifies the most productive authors, institutions, and countries; (iii) discusses authors’ and journals’ productivity patterns; and (iv) recognizes the most relevant research topics and how the interest on them has evolved over the years.

1. Introduction

Making decisions under complex and uncertain situations frequently requires the cooperation of a team of experts, each one with their own background, opinions, motivations, etc. As Huber [1] already noticed in 1984, in these circumstances, experts usually need to spend considerable time in meetings to reach a collective agreement. For more than 30 years, research on Group Decision-Making (GDM) systems have pursued saving much of this time by providing automated support to accomplish consensual decisions [2,3].

Figure 1 sketches the general GDM framework, where a group of experts desire to make a collective decision among a set of alternatives. First, they express their individual preferences on the alternatives. Then, those preferences are combined using an aggregation function. As typically the resulting collective preference does not achieve experts’ consensus, a feedback mechanism assists experts in changing their preferences for augmenting the consensus level.

Experts’ preferences are often vague and imprecise [4,5]. In these situations, Artificial Intelligence (AI) techniques are applied. For example, experts’ opinions are sometimes expressed as fuzzy preference relations that for each pair of alternatives $A_i$ and $A_j$, indicate the expert’s preference degree of $A_i$ over $A_j$. In these cases, individual preferences are aggregated using Ordered Weighted Averaging (OWA) operators [6], and fuzzy consensus models look for a solution supported by all or, at least, the majority of the experts [7]. Other times, experts’ preferences are conveyed linguistically, combined with Linguistic Ordered Weighted Average (LOWA) operators [8], and linguistic consensus models are used [9]. There are many other real-world GDM problems that use AI techniques as well, such as dealing with heterogeneous preference representation structures [10], detecting and managing non-cooperative experts’ behaviors [11], etc.

For instance, Ertugrul [12] reports the AI-GDM application for achieving a consensual decision concerning the facility location of a Turkish textile company, which has experienced a demand growth and thus needs to decide among three new alternative locations. The decision committee is composed of three experts that evaluate the locations according to five criteria: (i) favorable labor climate, (ii) proximity to markets, (iii) community considerations, (iv) quality of life, and (v) proximity to suppliers and resources. Experts express their preferences and criterion importance using linguistic variables (e.g., “I consider that the importance of proximity to markets is Very High, and that the second location Poorly satisfies this criterion”). Then, experts’ agreement is accomplished using Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) [13].

Given the great applicability of AI-GDM to a variety of domains (e.g., energy policies [14], green economy [15], web 2.0 communities [16], construction industry [17], animal behavior [18], water allocation management [19], supply chain coordination [20], library quality evaluation [21], construction project management [22], offshore wind farm siting [23], airport ground access [24], etc.), much research has been published on this topic.

This paper main contribution is to provide a systematic analysis of the vast literature published for the last 30 years on AI-GDM. To do so, a sample of 2862 articles gathered from Clarivate’s Web of Science (WoS) is examined. Analyzing manually such a large sample would be difficult and error-prone, and thus automated procedures are preferable [25,26,27,28]. The science of science provides a collection of techniques to analyze scientific documents to identify patterns and trends [29,30,31,32]. In particular, this paper adopts two approaches: performance analysis [33] and science mapping [34]. Performance analysis estimates the productivity and impact of the scientific actors (researchers, organizations, etc.) measuring how frequently articles are cited. This paper uses the h-index [35], which is one of the most widespread indicators for citation analysis. Science mapping uncovers the structural and dynamic aspects of scientific research, quantifying and visualizing its thematic subfields. This paper uses the co-word analysis method [36], which accounts for the association strengths of the papers’ keywords.

Using both science mapping and performance analysis, this paper answers the following Research Questions (RQs):

• RQ1: How is the number of publications on AI-GDM evolving over the years?
• RQ2: What is the impact of the research literature on AI-GDM?
• RQ3: Who are the most productive authors?
• RQ4: Is there any authors’ productivity pattern?
• RQ5: How do the most productive authors collaborate?
• RQ6: What countries and institutions are leading research?
• RQ7: What journals are publishing most articles?
• RQ8: Is there any journal productivity pattern?
• RQ9: What are the most relevant themes of research?
• RQ10: How has the interest in those themes evolved over time?
• RQ11: What are the main application domains?

The remaining of this paper is arranged as follows: Section 2 introduces the materials and methods used to undertake our bibliometric analysis; Section 3 reports the analysis results and provides some discussion regarding the research questions above; finally, Section 4 summarizes the conclusions of our work.

2. Materials and Methods

This section describes the systematic procedure we have followed to analyze the literature on AI-GDM.

2.1. Bibliometric Workflow

The workflow suggested by Cobo et al. [29] has been adopted to undertake our analysis systematically, which is similar to others proposed in the literature, such as PRISMA [37] or Börner et al. [38].

Figure 2 shows the followed workflow, which is organized in three stages:

• Data retrieval. As many experts have stated [39,40,41], obtaining all the articles relevant for a literature review is unrealistic. The objective is then to achieve an unbiased publication sample that represents the population satisfactorily.
  A sample of 2862 bibliometric records was gathered from the Clarivate WoS database using the following query:
  The first line sets the topic of the analysis; the NEAR/0 operator forces that (Making OR Support) follows immediately Group Decision, but tolerates spaces and the ‘-’ character (e.g., the query catches articles with Group Decision-Making and Group Decision-Making). As this paper focuses on the application of AI techniques to GDM, Line 2 limits the scope to the WoS category Computer Science Artificial Intelligence. Line 3 sets the time period of the records: every article published until 2019. Finally, Line 4 specifies the WoS indexes against the query is thrown.
  As a final remark, the criterion to select WoS instead of other databases, such as Google Scholar or Dimensions, is its outstanding data quality prestige [42].
• Data normalization. Bibliographic data are sometimes not normalized enough [29,30]: an author may appear differently in several records, the same concept may correspond to distinct keywords, etc. These problems can bias the subsequent analysis. For this reason, we preprocessed the data to guaranty its normalization.
• Data analysis. The normalized data were examined using two widespread bibliometric procedures [43]: performance analysis and science mapping. Both techniques have been successfully applied in recent studies (e.g., [25,26,28]) because they complement each other very well: performance analysis determines the importance of the bibliometric elements, and science mapping models how those elements are interrelated.

2.2. Performance Analysis

The primary method to assess research performance is citation analysis [44]. The Hirsch index [35], typically known as h-index, is probably the most commonly accepted citation analysis indicator [45]. If the index is used to quantify the author’s productivity, then it is defined as follows:

An author has index h whenever h of her n papers have at least h citations each, and the remaining $n-h$ papers have less than or equal to h citations each.

Furthermore, the h-index concept can be adapted to account for the performance of any bibliographic element: articles [46], journals [45], research organizations, etc.

2.3. Science Mapping

Three complementary techniques [47] were applied to identify the key research topics, the significance and role that those topics play, and how the interest in the topics has evolved over time. The following sections introduce these techniques.

2.3.1. Thematic Network Identification

A method called co-word analysis [36] was used to recognize the most relevant topics in AI-GDM research. Co-word analysis works by measuring the co-occurrence frequency of pairs of article’s keywords. Co-occurrences are first normalized [48], using the equivalence index [34] typically. Then, a clustering algorithm groups the keywords in function of the computed equivalence indexes [49], corresponding each group to a thematic network, i.e., to a key topic. In particular, the clustering algorithm we applied was simple centers [34].

2.3.2. Strategic Diagrams

The role that each thematic network plays in AI-GDM research was modeled using the density and centrality measures. Density [34] accounts for the thematic network internal coherence by examining the links between keywords inside the network. Centrality [50] estimates the interaction degree of the network with others by analyzing the links between keywords inside and outside the network.

Strategic diagrams are then used to provide a global representation of all topics’ role. In these diagrams, the x-axis and y-axis denote the network’s centrality and density, respectively. Thus networks are classified according to the quadrants where they are placed [51,52]; see Figure 3.

2.3.3. Maps of Conceptual Evolution

As the years go passing by, the vocabulary authors employ evolves: whereas some new words appear, others fall into disuse. Hence, the keyword set used in each period provides information concerning if the number of researched topics increases (new terms are included in the set), decreases (old words are erased from the set), or remains stable. Following the indications given in [47], we used the Inclusion index to track the vocabulary evolution in AI-GDM.

3. Results and Discussion

The following sections summarize the results of our analysis and answer the research questions this paper targets.

3.1. How Has the Number of Publications on AI-GDM Evolved over the Years? (RQ1)

Figure 4 shows the number of published papers per year. Colors blue and yellow denote periods of stability and growth, respectively. In particular, four stages can be distinguished:

• During the first ten years (from 1991 to 2000), the fundamental ideas were proposed and developed in 82 articles.
• The subsequent nine years (from 2001 to 2009) correspond to a growth period, where 540 articles where published.
• A short period of three years (from 2010 to 2012) with a stable publication rate (121.33 articles per year on average, accumulating a total of 364 papers).
• A rapid growth period that lasts up to present days (from 2013 to 2019), where 1856 articles have been published.

3.2. RQ2: What Is the Impact of the Research Literature on AI-GDM? (RQ2)

Citations to the published literature on AI-GDM have also followed an upswing trend. Figure 5 represents the evolution over time of the total number of citations to all the articles in the sample. The articles’ h-index is 113.

3.3. Who Are the Most Productive Authors? (RQ3)

Table 1. Most prolific authors.

Author	#Papers	#Citations	h-Index
HERRERA-VIEDMA E	135	7182	44
XU ZS	122	7778	40
CHICLANA F	72	4081	31
LIU PD	64	1773	22
MARTINEZ L	64	2337	19
CHEN HY	53	951	16
CABRERIZO FJ	51	1905	15
HERRERA F	48	3269	23
DONG YC	46	2438	24
ZHOU LG	46	871	15

summarizes the authors who have published the highest number of papers, including also the total number of citations that those papers have received, and the authors’ h-index (limited to the sample).

3.4. Is There Any Authors’ Productivity Pattern? (RQ4)

Figure 6 represents the number of authors per year. As the number of articles increases over time, the number of authors rises as well. There is a total of 3514 authors. Although most of them have published a pretty reduced number of papers (67.92% of the authors have written only one paper in 29 years), a small group of authors have contributed with a much bigger number of articles (8.22% of the authors have published at least five articles). This fact is not surprising, as it is consistent with one of the fundamental laws in bibliometrics: Lotka’s law [53] (also known as the inverse square law).

In 1926, after analyzing authors’ productivity in different domains, Lotka found that the number of authors with n papers is usually inversely proportional to $n^2$. In our case, 2387 authors have written one article; hence, Lotka’s law predicts that the number of authors that have published n papers should be $\frac{2387}{n^2}$. Figure 7 compares the empirical distribution found in the sample with the distribution predicted by Lotka’s law, showing that both distributions fit much.

3.5. How Do the Most Productive Authors Collaborate? (RQ5)

Pretty much as industrial production relies on teamwork, academic literature is increasingly the result of the collective work of several researchers [54]. Figure 8 shows that research on AI-GDM follows this trend too.

Accordingly, studying the collaboration between authors has a remarkable interest [45]. The graph in Figure 9 represents how the most productive authors collaborate. Each node accounts for one of the top 1% most prolific authors. The size of each node is proportional to its Eigenvector centrality in the collaboration network. This centrality models the importance of a node considering not only the number and weights of its connections to other nodes but also the influence of those nodes in the network [55]. There is an edge between two nodes whenever the corresponding authors have published some paper together. Edge thickness is proportional to its weight, i.e., to the number of papers that both researchers have coauthored.

3.6. What Countries and Institutions Are Leading Research? (RQ6)

Figure 10 shows the number of papers that each country’s researchers have published. The most prolific countries are China (39.25% of the papers), Spain (9.64%), United States of America (5.15%), United Kingdom (4.29%), Taiwan (4.10%), Turkey (4.07%), and India (3.25%).

Table 2. Most prolific organizations.

Organization	#Papers	#Citations	h-Index
University of Granada	182	8383	49
Sichuan University	154	4216	37
Central South University	107	2291	28
Southeast University China	81	5530	35
De Montfort University	77	4189	32
Universidad de Jaen	76	2872	24
King Abdulaziz University	72	2750	27
Nanjing University of Information Science Technology	71	732	14
Shandong University of Finance Economics	69	1788	22
Hohai University	60	1445	21

summarizes the most prolific organizations, showing the number of papers that their researchers have published, how many times those articles have been cited, and the organizations’ h-index (limited to the sample).

3.7. What Journals Are Publishing Most Articles? (RQ7)

Table 3. Most prolific journals.

Journal	#Papers	#Citations	h-Index	Bradford’s
				Zone ($\mathit{n}=3$)
Journal of Intelligent Fuzzy Systems	302	2520	24	1
Expert Systems with Applications	210	13,137	67	1
Applied Soft Computing	176	6268	42	2
Knowledge Based Systems	150	4891	36	2
International Journal of Intelligent	145	4891	36	2
Systems
Soft Computing	112	1700	23	2
International Journal of Fuzzy Systems	94	1106	17	2
Decision Support Systems	77	4174	29	3
IEEE Transactions on Fuzzy Systems	77	6762	36	3
International Journal of Uncertainty	69	1677	20	3
Fuzziness and Knowledge Based Systems

shows the journals that have published most articles. Again, the table includes the total number of papers, the citations per journal, and the journal’s h-index. The last column will be described in Section 3.8.

3.8. Is There Any Journal Productivity Pattern? (RQ8)

Analogous to Lotka’s law for authors’ productivity (see Section 3.4), there is another bibliometric law for journal productivity called Bradford’s law [56]. It predicts an inverse relationship between the number of papers published in an area and the number of journals where the articles appear. In other words, a few journals usually account for a high portion of the total publications, while a high number of journals publish fewer articles in the area.

In our case, there are 2862 papers in the sample; 1016 published in conferences and 1846 published in journals. Although a total of 32 journals have published the 1856 articles, 9 of them have published 66% of the articles, i.e., journal productivity concentration is even higher than the one predicted by Bradford’s law. Figure 11 compares the cumulative distributions of the empirical data and the data predicted by Bradford’s law, according to the procedure proposed by Egghe and Rousseau [57]. Roughly speaking, suppose that the journals in the sample are sorted according to the number of articles into 3 groups, each one including $\frac{1}{3}$ of all articles approximately. Those groups are named Bradford’s zones. They are registered in Table 3’s last column, and highlighted with different colors in Figure 11 (Zone 1 in blue, Zone 2 in pink, and Zone 3 in yellow). Zone 1 comprises 2 core journals. Zone 2 includes 5 journals, thus Bradford’s constant n is $\frac{5}{2}=2.5$. Although Bradford’s law predicts that the number of journals in Zone 3 should be $2\times 2.5^2=12.5$, the empirical number of journals is much bigger: 25. Trying with different numbers of zones produces even more distant results.

3.9. What Are the Most Relevant Themes of Research? (RQ9)

The complete results of the science mapping analysis we performed to determine the most prominent themes of the AI-GDM research field (RQ9) and their longitudinal evolution (RQ10) are available in the following public repository: https://github.com/rheradio/AI-GDM-BibAnalysis.

A strategic diagram is shown for each of the periods 1991–2009, 2010–2014 and 2015–2019. In each period, the volume of the spheres is proportional to the number of published documents associated with each research theme.

In the first period, 1991–2009, according to the strategic diagram shown in Figure 12, the GDM research field was focused on 17 themes. Nine of them stand out since they are motor, basic, and transversal: fuzzy-sets, public-investment-decision, multi-attribute-group-decision-making, trapezoid-fuzzy-numbers, consistency (i.e., approaches to measure the level of consistency of the information provided by the experts), information-retrieval, OWA-operators, TOPSIS, and decision-making.

Taking into account the performance measures shown in

Table 4. Performance of the themes in the 1991–2009 period.

Name	Number of Documents	Number of Citations	h-Index
OWA-operators	176	10,260	46
Decision-making	109	3230	29
Consistency	100	6752	42
TOPSIS	83	6361	35
Fuzzy-sets	60	2824	25
Alternatives	40	945	16
Information-retrieval	35	2573	19
Majority	27	1628	13
Multi-attribute-group-decision-making	22	567	8
Computer-mediated-communication	17	336	12
Expert-system	17	148	6
Extent-analysis-method	12	823	9
Customer-requirements	7	369	7
Public-investment-decisions	6	8	2
Fuzzy-majority	4	57	2
Trapezoid-fuzzy-numbers	3	38	1
Group-consensus-opinion	2	0	0

, the themes OWA-operators, decision-making, and consistency got more than 100 documents. Considering the citations achieved, OWA-operators is the most cited theme, reaching more than 10,000 citations. Moreover, consistency and TOPSIS, with more than 6000 citations, achieved a significant impact.

In the next period, 2010–2014, as it is shown in Figure 13, the GDM research field delved into the following ten themes (motor plus basic and transversal): OWA-operators, majority (i.e., the soft-computing approach that relaxes the total consensus, seeking the alternative supported by most experts), analytical-network-process, consistency, additive-consistency, vague-set-theory, TOPSIS, linguistic-variables, fuzzy-sets, and decision-making.

Bear in mind that according to the performance measures in

Table 5. Performance of the themes in the 2010–2014 period.

Name	Number of Documents	Number of Citations	h-Index
Consistency	376	15,073	70
OWA	188	8671	50
Fuzzy-sets	173	7027	49
OWA-operators	168	6936	47
Vague-set-theory	157	7584	49
Linguistic-variables	100	4755	36
Decision-making	81	2841	30
Analytic-network-process	76	3980	32
Additive-consistency	74	2519	30
Majority	65	2531	27
Consistency-measures	28	1486	19
Choquet-integral	27	1136	15
Group-members	17	1463	15
Recommender-system	17	902	10
Personality	12	138	5
Neural-networks	11	390	7
Fuzzy-game-theory	8	223	8
Multidimensional-analysis	7	191	4

, the themes consistency, TOPSIS, fuzzy-sets, OWA-operators, vague-set-theory, and lingustic-variables got more than 100 documents. Furthermore, the theme consistency, with more than 15,000 citations, almost double the impact of the second more cited theme. Moreover, the themes TOPSIS, fuzzy-sets, and vague-set-theory stand out with more than 6000 citations.

As Figure 14 shows, the primary main research fields turned around 12 main themes in the last period, 2015–2019: terms-sets, AHP, Vikor-method, similarity-measures, consensus, consensus-reaching-process, multi-attribute-group-decision-making, supplier-selection, multi-criteria-group-decision-making, uncertainty, fuzzy-sets, and linguistic-term-sets.

Moreover, according to the performance measures shown in

Table 6. Performance of the themes in the 2015–2019 period.

Name	Number of Documents	Number of Citations	h-Index
Term-sets	757	11,079	54
AHP	499	7213	44
Similarity-measures	352	4968	40
Multi-attribute-group-decision-making	337	4871	35
Consensus	300	4944	39
Uncertainty	206	2274	25
Multi-criteria-group-decision-making	189	3169	31
Fuzzy-sets	187	2284	24
Vikor-method	167	2326	26
Consensus-reaching-process	153	2828	27
Supplier-selection	134	1735	22
Information-aggregation	80	941	15
Genetic-algorithm	76	564	13
Linguistic-term-sets	71	1413	21
Weighted-averaging-operators	58	1005	16
Linguistic-information	53	799	16
Recommender-system	51	720	14
Priority-weights	46	523	13
Membership-grades	46	1027	18
Ranking-method	37	420	10
Geometric-operators	36	489	11
Priority	35	361	13
Multi-agent-systems	27	109	6

, except for linguistic-terms-sets, the main themes pointed above got a great number of documents (more than 100). Taking into account the achieved citations, Term-sets was the most cited theme, with more than 11,000 citations. In comparison with the previous periods, themes have had high impact considering this period’s small citation window. In addition, themes AHP, similarity-measures, multi-attribute-group-decision-making, and consensus achieved more than 4000 citations.

3.10. How Has the Interest in Those Themes Evolved over Time? (RQ10)

This section discusses the thematic network evolution, describing how these themes evolved through the years, and how the topics emerged and changed. For that reason, an evolution map [47] is provided, in which each column represents a period. There is a link between the themes of two consecutive periods if both themes have keywords in common. Indeed, the link strength is proportional to the Inclusion index (the more words they have in common, the thicker the link).

Therefore, analyzing the themes across three consecutive periods, we can summarize the conceptual evolution of AI-GDM in seven thematic areas (Figure 15): (i) multi-attribute/criteria in GDM, (ii) analytical network process, (iii) decision-making and uncertainty, (iv) fuzzy sets, (v) recommender systems, (vi) consensus and majority, and (vii) agent systems.

Furthermore, for each thematic area, a set of bibliometric indicators were calculated to show the performance and impact score. In that way,

Table 7. Performance of thematic areas.

Name	Number of Documents	Number of Citations	h-Index
Multi-attribute/criteria in GDM	2066	54,048	110
Analytical network process	250	6660	43
Decision-making and uncertainty	458	8828	52
Fuzzy sets	724	15,740	64
Recommender systems	201	8673	51
Consensus and majority	355	9500	51
Agent systems	39	247	8

shows for each thematic area, the total number of documents, the number of citations achieved, and the h-index. It is worth noting that the documents were associated with each thematic area using the algebraic union of the documents belonging to each theme, so it could be possible that the same documents count in different research areas. That is, the sum of the documents could be different from the total number of documents analyzed in this study.

Considering the thematic areas shown in Figure 15, and their performance measures, we should point out that AI-GDM has been mainly focused on the research area of multi-attribute criteria, as it is the largest one (it has the biggest number of documents). Also, it achieves the highest number of citations count. The thematic network fuzzy-sets also has a significant number of documents, which have been highly cited.

3.11. What Are the Main Application Domains? (RQ11)

WoS provides a classification system called, research areas that organizes publications according to their subjects into 252 areas. Research literature on AI-GDM spreads over a variety of application domains: 19.67% of the papers fall into the Engineering area, 15.79% into Operations Research Management Science, 7.97% into Automation Control Systems, etc. The word cloud in Figure 16 shows the foremost application domains; words have been abbreviated, and their size is proportional to the number of articles classified in the corresponding areas.

4. Conclusions and Future Challenges

In this paper, a systematic and highly automated bibliometric workflow has been followed to analyze the literature on group decision-making based on artificial intelligence. Our longitudinal analysis shows that:

• Research on AI-GDM is increasing as the number of papers and citations to those papers is growing substantially.
• Most research has been carried out by Chinese universities. Nevertheless, a few Spanish investigators lead research in terms of productivity and collaboration network centrality.
• Two basic bibliometric laws hold to a great extent, Lotka’s law and Bradford’s law, which model authors’ and journal productivity concentrations, respectively.
• AI-GDM is being applied to a variety of domains, including engineering, operations research management science, automation control systems, robotics, economic, telecommunications, imaging science, etc.
• Currently, themes such as terms-sets, analytical-hierarchical-process, Vikor-method, similarity-measure, consensus, consensus-reaching-process, and multi-attribute- group-decision-making are motor in AI-GDM research.
• In summary, the conceptual evolution of the AI-GDM research fields delved into seven thematic areas: multi-attribute/criteria in GDM, analytical network process, decision-making and uncertainty, fuzzy sets, recommender systems, consensus and majority, and agent systems.

Finally, recent literature on AI-GDM reveals the following trends and challenges:

• There is an increased need to support the consensus of huge groups of decision-makers. This need arises in several contexts, such as social networks, e-democracy platforms, crowd-funding systems, group recommender systems, etc. Those large groups are typically decomposed into smaller ones by applying different clustering algorithms, such as hierarchical clustering [58], discriminant analysis [59], etc.
• In classical GDM, a reduced group of experts needs to make a consensual decision. Presently, the experts’ group is often replaced by internet users’ opinions. As a result, natural language processing techniques have started to be applied for mining linguistic information that is subsequently processed by GDM systems [60].
• As AI-GDM problems become more complex, advanced models and simulations are required to support the experts’ group dynamics [61], e.g., for identifying the most influential experts, detecting manipulative and non-cooperative behaviors, etc.
• Deep learning has started to be used [62] for (i) estimating the importance (or weight) of the experts, their preferences, and their relationships, and (ii) learning the optimal settings of parameterized aggregation operators.