A Systematic Literature Review and Meta-Analysis of Studies on Online Fake News Detection

Abstract

The ubiquitous access and exponential growth of information available on social media networks have facilitated the spread of fake news, complicating the task of distinguishing between this and real news. Fake news is a significant social barrier that has a profoundly negative impact on society. Despite the large number of studies on fake news detection, they have not yet been combined to offer coherent insight on trends and advancements in this domain. Hence, the primary objective of this study was to fill this knowledge gap. The method for selecting the pertinent articles for extraction was created using the preferred reporting items for systematic reviews and meta-analyses (PRISMA). This study reviewed deep learning, machine learning, and ensemble-based fake news detection methods by a meta-analysis of 125 studies to aggregate their results quantitatively. The meta-analysis primarily focused on statistics and the quantitative analysis of data from numerous separate primary investigations to identify overall trends. The results of the meta-analysis were reported by the spatial distribution, the approaches adopted, the sample size, and the performance of methods in terms of accuracy. According to the statistics of between-study variance high heterogeneity was found with τ² = 3.441; the ratio of true heterogeneity to total observed variation was I² = 75.27% with the heterogeneity chi-square (Q) = 501.34, the degree of freedom = 124, and p ≤ 0.001. A p-value of 0.912 from the Egger statistical test confirmed the absence of a publication bias. The findings of the meta-analysis demonstrated satisfaction with the effectiveness of the recommended approaches from the primary studies on fake news detection that were included. Furthermore, the findings can inform researchers about various approaches they can use to detect online fake news.

1. Introduction

The rapid adoption of social media has significantly altered the manner in which people live their lives, resulting in newspapers and other traditional news sources becoming less relevant. “Social media” refers to platforms such as Twitter and Facebook that help individuals from around the globe build networks and share information and/or sentiments in real-time [1]. Due to the broad usage of social media and the pervasive availability of the internet, social media is the ideal location to propagate disinformation or fake news due to little to no oversight of social media platforms.

The term “fake news” can be described as claims or stories that are purposefully and verifiably untrue and attempt to pass themself off as news or journalistic reports [2,3,4]. However, it can be challenging for average people to distinguish this type of news from the plethora of information publicly available because of restrictions in knowledge and experience. Researchers have examined fake news from various viewpoints and produced a basic classification of several categories of fake news [5]. According to Shu et al. [6], the classification of fake news detection includes knowledge-based, style-based, source-based, and propagation-based methods. Knowledge-based methods are used to check for the truthfulness of claims by verifying if the knowledge contained in the news content is accurate. This method is seen as a superior option for scalable fact checking [6,7]. Style-based methods aim to detect the distinctions between the writing styles of fake and real news. The text, images, and/or videos contained in to-be-verified content can be used to extract journalistic style, allowing one to infer the goal of the news content [6,8]. Propagation based methods detect fake news based on news propagated on social networks [6,8]. Source-based methods demonstrate that fake news can be detected by examining the credibility of its source, where credibility is frequently characterized in terms of quality and believability, offering plausible grounds for belief [6,8].

Fake news has been spreading for many years and is not a new problem [9]. Although this incorrect or misleading news is an intentional propagation that causes society to trust misleading information, identifying fake news from authentic news based on shared content has become increasingly challenging. Because social media actively fosters the flow of information from user to user, it is difficult to spot bogus news content. As a result of its global diffusion and the inability of humans to deal with the quick spread of news on the internet, dealing with misinformation is a challenging undertaking.

By broadcasting misleading and biased information, fake news has the potential to damage people’s trust in authorities, experts, and the government. Furthermore, this type of news has serious consequences for society, politics, information technology, and financial issues, as well as for everyone who lives in a cyber environment where there is a lack of trust [10]. The rise of fake news on social media has compelled the progress of research for accurately detecting these news instances. As a result, researchers have developed a variety of approaches, with some claiming to be superior to others. Therefore, a well-established, accurate-focused approach to detecting online fake news is urgently needed to mitigate its significant influence and harm to society.

Therefore, researchers have developed a variety of detection methods that rely on artificial intelligence (AI) techniques, including deep learning [11,12,13,14], machine learning [15,16,17,18], and ensemble [19,20,21] approaches. Numerous review and survey papers in this domain have been published as a result of the extensive collection of studies on the subject. The vast majority of review studies already published, such as Collins et al. [22], Choraś et al. [23], Varlamis et al. [24], Shahid et al. [25], Khan et al. [26], and Lozano et al. [27], are descriptive rather than providing a quantitative evaluation of techniques for detecting fake news. Hence, a meta-analysis is necessary as it allows for a credible analysis of the findings from the published literature to uncover varied perspectives [28]. Furthermore, meta-analysis often improves reliability and assures previous study results on the detection of online fake news. Due to a lack of studies on meta-analysis to understand the proper detection approach, the purpose of this study was to utilize meta-analysis as a statistical tool to assess the efficacy of different proposed approaches from primary studies on fake news detection conducted independently in the literature. The following are the unique contributions of this study:

• The discovery of a variety of sources in research on the detection of online fake news can help researchers make better decisions by identifying appropriate AI approaches for detecting fake news online.
• The examination of publication bias in establishing the reliability of the main conclusions of research on detection methods.
• The identification of studies that contribute most to the heterogeneity of the detection studies.

The remainder of the paper is structured as follows: the related work to review various methods suggested in the literature for spotting fake news is covered in Section 2. The material and methods are presented in Section 3 in both theoretical and applied forms. Section 4 presents the results and discussion, while Section 5 presents the study conclusion.

2. Related Works

Traditional news media generally rely on news content for the identification of fake news, as opposed to social media, where additional social context auxiliary information can be used as supplementary information to help detect fake news. The use of supervised fake news detection models based on machine learning (ML) and deep learning (DL) techniques has significantly expanded in recent years due to their excellent detection accuracy. These methods extract the distinguishing characteristics of fake news using feature representation based on linguistic and visual data [6]. Linguistic-based characteristics are derived from many levels of textual content organization, such as characters, words, phrases, and documents. Visual-based features are derived from visual resources such as images and videos in order to recognize the numerous characteristics of fake news. With the reported increase in online fake news [29,30], automated methods for its detection on social media have attracted the attention of researchers worldwide [31,32,33]. COVID-19 and the numerous related hoaxes, rumors, and misinformation surrounding the cures, treatment, and prevention have further fueled the interest of researchers in improved methods for detection [34]. Even with this increased attention, the task of detecting fake news is still reported as challenging [35].

Through the analysis of the literature relating to this area, it is evident that a diverse range of ML and DL approaches as well as hybrid and ensemble versions of these have been employed. This section presents the literature relating to the approaches mentioned above.

Several researchers have developed ML methods for the detection of fake news. Vicario et al. [36] built a logistic regression (LR) classifier to predict this type of news using a massive Italian dataset consisting of actual news and hoaxes published on Facebook, achieving an accuracy of 91%. The LR method also achieved the highest accuracy (96%) in the study by Stitini et al. [37] where Bidirectional Encoder Representations from Transformers (BERT) transformed the dataset text into vectors. Random forest (RF) often emerges as the method achieving the most accurate results, with an accuracy of 97.3% reported by Fayaz et al. [38]. The study used data from the ISOT fake news dataset and compared results with other state-of-the-art machine learning techniques such as gradient boosting machines (GBM), extreme gradient boosting machines (Boost), and the adaptive boost regression model. Support vector machine (SVM) models have also shown promising results, with an accuracy of 93.15% being achieved when applying the data from the fake news dataset extracted from Kaggle, outperforming the LR approach applied to the same data by 6.82% [39].

While many researchers investigate the performance of individual ML methods, some researchers chose to investigate the effect of applying an ensemble of ML methods on the data to achieve improved accuracy results. A blended ensemble machine learning method that applies the LR, SVM, linear discriminant analysis, stochastic gradient descent, and ridge regression techniques achieved 79.9% accuracy when data from the ISOT and LIAR datasets were used [40]. Accuracies over 95% have been achieved by many studies that have applied voting ensemble methods to the datasets, including Elsaeed et al. [41], Verma et al. [42], Biradar et al. [43], Kanagavalli and Priya [44], and Elhadad, Li, and Gebali [21], who achieved accuracy measures of 95.6%, 96.7%, 97%, 98.6%, and 99.7%, respectively. These works based their results on data from different datasets, including ISOT, WELFake, COVID19 Fake, LIAR, and researcher-created datasets.

DL methods such as convolutional neural networks (CNN), long-short term memory (LSTM), and bi-directional long-short term memory (BiLSTM) have attracted much interest in the area of fake news detection. Galli et al. [45] applied both ML and DL methods to datasets, comparing the results obtained. It was established that, by applying the CNN technique to the limited PoliFact dataset, an accuracy of 75.6% was achieved. The study reported that the CNN method outperformed the other approaches investigated, which include, among others, naive Bayes (NB); RF; LR; nearest neighbor (NN); decision tree; gradient boost; and BiLSTM. BiLSTM has also been investigated for its value in detecting fake news by many other researchers [46,47,48,49]. With most studies focusing on the English language, both Fouad, Sabbeh, and Medhat [47] and Nassif, Elnagar, Elgendy, and Afadar [48] investigated the accuracy of state-of-the-art classification methods for the identification of fake news in the Arabic language. Fouad, Sabbeh, and Medhat [47] evaluated the performance of eight machine learning algorithms and also experimented with five different combinations of deep learning algorithms, including CNN and LSTM, with the results indicating that the BiLSTM method outperformed the other methods, achieving an accuracy of 75% on the dataset of size 4561. Nassif, Elnagar, Elgendy, and Afadar [48] created a customized dataset based on tweets that consisted of 5000 fake and 5000 true news instances. Their Arabic Bi-directional Encoder Representations from the Transformers model (ARBERT) achieved 98.8% accuracy on the data.

Ensemble deep learning approaches have also been investigated for their value as detection methods, with the novel MisRoBÆRTa technique proposed by Truică and Apostol [20]. The technique combines CNN and many BiLSTM, achieving an accuracy of 92.5% when tested on a dataset with a sample size of 100,000. Jang et al. [50] collected data from Twitter and classified tweets as fake news by using the temporal propagation pattern of the retweeted quotes. The authors applied a two-phase deep learning model based on CNN and LSTM for training and testing, achieving an accuracy measure of 85.7%. An ensemble-based deep learning technique for classifying news as real or fake achieved a significant accuracy of 89.8% using data from the LIAR dataset [51]. The approach used two deep learning models, with a Bi-LSTM-gated recurrent unit (GRU) being used for the textual “statement” attribute, while the deep dense learning model was used on the remaining nine attributes.

While these studies all reported on the accuracy of the employed methods, the literature also includes studies that survey and review current approaches. In the paper by Collins, Hoang, Nguyen, and Hwang [22], a synthesis of methods for combating misinformation and fake news on social media is presented, while possible solutions, methodological gaps, and challenges relating to current detection methods were presented in a systematic review by Choraś, Demestichas, Giełczyk, Herrero, Ksieniewicz, Remoundou, Urda, and Woźniak [23]. Similarly, Shahid, Jamshidi, Hakak, Isah, Khan, Khan, and Choo [25], through a survey of novel AI approaches, uncovered key challenges in the area while also highlighting potential future research to be considered. An approach-specific survey by Varlamis, Michail, Glykou, and Tsantilas [24] investigated and reported on the studies that apply graph convolutional networks (GCNs) for detecting rumors, fake content, and fake accounts, with the aim of the paper being to provide a starting point for those researchers wanting to further investigate GCNs for the detection of fake news. Both Khan, Hakak, Deepa, Dev, and Trelova [26] and Lozano, Brynielsson, Franke, Rosell, Tjörnhammar, Varga, and Vlassov [27] chose to rather review ML models, providing a set of advantages and disadvantages associated with the datasets used in the reviewed studies. Additionally, Shu, Sliva, Wang, Tang, and Liu [6] provided a thorough analysis from a data mining perspective and emphasized the future research prospects according to four categories: data-oriented, feature-oriented, model-oriented, and application-oriented. One of the potential study areas for fake news detection that Shu, Sliva, Wang, Tang, and Liu [6] suggested is model-oriented fake-news research, which opens the path for the development of more effective and useful models based on supervised and unsupervised approaches to fake news detection.

While the current literature provides insight into the latest methods being employed and highlights reviews that have been performed, there appears to be no single study that quantitatively analyzes the current methods proposed for fake news detection. Furthermore, no systematic, comprehensive study of model-oriented fake news detection based on supervised learning techniques such as ML, DL, and ensemble methods has been conducted. Lozano, Brynielsson, Franke, Rosell, Tjörnhammar, Varga, and Vlassov [27] also highlight the lack of literature that considers multiple datasets and multiple approaches for detection. With the increasing number of publications in this research area and the reported proliferation of fake news, a systematic analysis is required so that an objective and comprehensive understanding of current supervised approaches can be obtained. The results provide valuable insight to researchers in the field regarding the DL, ML, and ensemble methods that were applied. This study, therefore, aimed to identify current trends, approaches, and methods for online fake news detection. Through meta-analysis, the patterns and correlations that exist in the area of ML, DL, and ensemble methods were unveiled and reported on.

3. Materials and Methods

This section provides a detailed description of the data extraction method applied as well as the criteria applied for the selection of relevant works. The meta-analysis is also presented and the measures are described.

3.1. Literature Search Strategy

A search of the literature was conducted to identify all published studies reporting on fake news detection. Following the recommendations of preferred reporting items for systematic reviews and meta-analyses (PRISMA) [52], the literature search strategy, screening and selection of publications, identification of parameters to be extracted, quality assessment, data extraction into tabular format, and reporting results were carried out [53]. The researchers searched the academic databases of the Web of Science to find pertinent published articles for this meta-analysis study. Previous research revealed that searching only one database is sufficient, as the checking of additional databases shows a minimal effect on the meta-analysis outcome [54,55]. On 17 August 2022, The Web of Science database was scoured for English-language papers published between 2014 and 2022.

The search terms used during a comprehensive literature search were: (“Fake news detection” OR “online fake news” OR “false news” OR (“fake news” AND “social media”) OR (“fake news” AND (“internet” OR “online”))). Between 2014 and 2022, 2159 published articles in total were found before applying exclusion criteria relating to publication years, document types, open access, and languages. This resulted in 945 studies being identified for screening and thereafter imported into Excel. Furthermore, reference lists from relevant papers were manually checked to identify any citations that the electronic database search may have missed.

3.2. Inclusion and Exclusion Criteria

The 945 studies identified for screening were subjected to inclusion and exclusion criteria as shown in

Table 1. Exclusion and inclusion criteria.

Criterion
	Exclusion Criteria
EC1	Papers in which only the abstract is available
EC2	Review and survey papers
EC3	Duplicate records
EC4	Papers not written in the English language
EC5	Papers not relevant to fake news detection
EC6	Papers not applying the DL, ML, or ensemble approaches
EC7	Papers not reporting sample size
EC8	Papers not reporting fake news detection results in terms of accuracy
	Inclusion criteria
IC1	Articles published in English
IC2	Papers stating the fake news detection method using DL, ML, or ensemble approaches on linguistic or visual based data
IC3	Papers providing clear information about the datasets and sample size
IC4	Papers providing the detection results in terms of accuracy

.

3.3. Quality Assessment and Data Extraction

The authors of this study assessed the merits and relevance of each article. From the chosen studies, data that met the inclusion criteria were taken for further analysis. The studies that used an ensemble approach such as voting and stacking were manually labeled as an ensemble approach. Some articles did not explicitly state the method for detecting fake news, but the authors were still able to categorize them using the approach they presented. The systematic review and meta-analysis were appropriate for the study, which had 100% of the information and met all the inclusion criteria [56].

The Excel spreadsheet was populated with article data extracted according to variables listed in

Table 2. Fields created to extract the relevant information for the meta-analysis.

Extraction Element	Contents	Type
1	Title	Title of the article
2	Author	The authors of the article
3	Country	The country of the research institute
4	Year	The year of publication
5	Approach	DL, ML, Ensemble DL, Ensemble ML, Hybrid, and Sentiment analysis
6	Method	For instance, BiLSTM, CNN, LSTM, RF, LR, SVM, and NB
7	Dataset	List of the datasets used for evaluation
8	Sample size	The number of samples used for detection
9	Accuracy	The average accuracy of the results

. The resulting database consisted of nine variables, which were populated with both qualitative and quantitative data that were retrieved through the review of the selected studies.

The data for the meta-analysis, therefore comprised of a matrix with nine fields and 125 rows and consisted of information for fake news detection approaches. The PRISMA flowchart detailing the extraction of relevant studies is presented in Figure 1.

3.4. Data Synthesis and Statistical Analysis

In order to prepare for the statistical analysis, the information was entered into an Excel spreadsheet. These data were then imported into the statistical analysis software, STATA version 17. The effect sizes of each included primary study and the total pooled effect size of all primary studies were calculated using the data extracted. The random-effects model served as the groundwork for our analysis. Due to data collected from published studies authored by several authors who worked independently on various fake news detection datasets, dataset sample sizes, fake news detection approaches, and fake news detection methods, the randomization hypothesis is plausible. Hence, the distinct underlying effect sizes of the included studies were presupposed in a random-effects model [57]. Using the Cochrane Q statistic, the study heterogeneity was determined; consequently, τ² and I² were employed to measure study heterogeneity [56]. I² values of 25%, 50%, and 75% respectively reflect low, medium, and high heterogeneity. The effect sizes were calculated using the Forest plot [58] as a preamble to assessing heterogeneity and biases in the results of the included studies. For the purpose of assessing the efficacy of different fake news detection approaches, a pooled estimate was produced using a DerSimonian and Laird random-effects model.

Furthermore, when conducting a moderator analysis in a systematic review with meta-analysis, subgroup analysis and meta-regression are frequently utilized [57]. To compare a sample of data, subgroup analysis divides participant data into smaller groups. Hence, to identify the source of study heterogeneity, this study performed a subgroup analysis focused on research performance evaluation metrics, i.e., the accuracy, of the included studies. The subgroups were based on the approach used (machine learning, deep learning, ensemble deep learning, ensemble machine learning, hybrid, and sentiment analysis for fake news detection). In addition, to determine if any subsets of the included studies captured the pooled effect size, meta-regression analyses were conducted [59].

In systematic reviews and meta-analyses, publication bias is a possible problem that cannot be avoided. Additionally, it poses one of the biggest risks to the reliability of meta-analysis. The goal of the investigation was to determine to what extent publication bias affects a study’s outcome when judging the reliability of its main findings. In order to report publication bias among the included studies, this study used a funnel plot [60]. Since visual interpretation is subjective, in this investigation, the statistical Egger’s test as well as the visual examination of the funnel plot were used to determine publication bias. p < 0.05 was chosen to denote the statistical significance of publication bias for Egger’s regression test [56].

4. Results

One of the 945 studies identified by the database search was removed because it was a duplicate paper, and four others were eliminated because they were not in English. During the screening process, eight papers were rejected, as only the abstract was available. After analyzing the full text of the remaining 932 papers, a further 807 were rejected for a list of reasons: a review article, not related to fake news detection, not based on DL, ML, or ensemble, no reported sample size, or no reported accuracy. The final meta-analysis was performed on 125 studies.

4.1. Meta-Analysis Summary

To estimate the accuracy of fake news detection methods, random-effects model meta-analyses were performed using the sample size and accuracy based on effect size and standard error of effect size.

Table 3. Meta-analysis summary: random-effects model (DerSimonian–Laird).

Meta-Analysis Summary: Random-Effects Model: DerSimonian–Laird						Heterogeneity		τ2	3.440	I2	75.27
Study (n = 125)		Effect Size	[95% CI]		Weight	Study		Effect Size	[95% CI]		Weight
(Sadeghi, Bidgoly, and Amirkhani 2022)	[46]	−10.230	−12.692	−7.769	0.730	(Khan et al. 2022)	[26]	−7.181	−9.265	−5.097	0.800
(Jarrahi and Safari 2022)	[62]	−9.698	−11.669	−7.727	0.820	(Stitini, Kaloun, and Bencharef 2022)	[37]	−6.132	−8.135	−4.130	0.820
(Ni, Li and Kao 2021)	[4]	−7.129	−9.170	−5.088	0.810	(Wang et al. 2021)	[63]	−10.617	−12.699	−8.534	0.800
(Fouad, Sabbeh, and Medhat 2022)	[47]	−8.713	−10.976	−6.449	0.770	(Abdelminaam et al. 2021)	[64]	−12.803	−14.901	−10.705	0.800
(Seddari et al. 2022)	[65]	−4.362	−6.393	−2.332	0.810	(Madani, Erritali, and Bouikhalene 2021)	[66]	−7.836	−10.042	−5.631	0.780
(Tembhurne, Almin, and Diwan 2022)	[29]	−11.199	−13.189	−9.209	0.820	(Endo et al. 2022)	[67]	−9.402	−11.423	−7.380	0.810
(Ying et al. 2021b)	[68]	−9.620	−11.714	−7.526	0.800	(Ke et al. 2020)	[69]	−8.930	−10.970	−6.889	0.810
(Do et al. 2021)	[33]	−9.558	−11.737	−7.379	0.780	(Wu et al. 2020)	[70]	−11.147	−13.797	−8.497	0.700
(Abonizio et al. 2020)	[71]	−9.362	−11.484	−7.240	0.800	(Singh and Sharma 2021)	[72]	−9.488	−11.636	−7.341	0.790
(Amer, Kwak, and El-Sappagh 2022)	[73]	−10.722	−12.692	−8.752	0.820	(Thaher et al. 2021)	[74]	−7.734	−9.905	−5.562	0.790
(Ying et al. 2021a)	[35]	−9.760	−11.843	−7.676	0.800	(Gereme et al. 2021)	[75]	−8.258	−10.226	−6.291	0.820
(Jang et al. 2021)	[50]	−11.400	−13.517	−9.283	0.800	(Galende et al. 2022)	[11]	−8.610	−10.775	−6.446	0.790
(Elsaeed et al. 2021)	[41]	−11.156	−13.160	−9.151	0.820	(Raza and Ding 2022)	[12]	−8.023	−10.297	−5.748	0.770
(Galli et al. 2022)	[45]	−6.483	−8.726	−4.240	0.770	(Kiruthika and Thailambal 2022)	[76]	−8.740	−10.932	−6.549	0.780
(Vicario et al. 2019)	[36]	−15.764	−17.819	−13.710	0.810	(Ma et al. 2022)	[77]	−10.665	−12.661	−8.670	0.820
(Verma et al. 2021)	[42]	−11.220	−13.213	−9.227	0.820	(Choi et al. 2021)	[13]	−8.993	−11.100	−6.887	0.800
(Ahmed et al. 2021)	[78]	−10.231	−12.213	−8.249	0.820	(Bangyal et al. 2021)	[14]	−9.261	−11.251	−7.271	0.820
(Tashtoush et al. 2022)	[79]	−10.03	−12.049	−8.010	0.810	(Tang et al. 2022)	[80]	−10.593	−12.589	−8.598	0.820
(Wang et al. 2021)	[63]	−9.090	−11.174	−7.007	0.800	(Upadhyay, Pasi, and Viviani 2022)	[81]	−9.546	−11.602	−7.490	0.810
(Rohera et al. 2022)	[82]	−8.834	−10.874	−6.794	0.810	(Al-Yahya et al. 2021)	[83]	−12.435	−14.712	−10.158	0.770
(Mertoğlu and Genç 2020)	[84]	−11.385	−13.377	−9.393	0.820	(Xing et al. 2021)	[85]	−10.180	−12.988	−7.371	0.670
(Jiang et al. 2021)	[86]	−10.864	−12.844	−8.884	0.820	(Varshney and Vishwakarma 2022)	[87]	−9.406	−11.376	−7.436	0.820
(Paka et al. 2021)	[88]	−10.767	−12.774	−8.761	0.820	(Ilie et al. 2021)	[89]	−11.645	−13.739	−9.551	0.800
(Kaliyar, Goswami, and Narang 2021a)	[90]	−5.279	−7.324	−3.234	0.810	(Upadhyay, Pasi, and Viviani 2022)	[81]	−9.246	−11.271	−7.221	0.810
(Akhter et al. 2021)	[91]	−8.236	−10.473	−5.999	0.770	(Kausar, Tahir, and Mehmood 2020)	[16]	−9.451	−11.505	−7.396	0.810
(Sharma and Garg 2021)	[31]	−11.010	−13.032	−8.989	0.801	(Ilias and Roussaki 2021)	[92]	−15.732	−17.888	−13.575	0.790
(Awan et al. 2021)	[30]	−10.140	−12.105	−8.175	0.830	(Waheeb, Khan, and Shang 2022)	[93]	−13.927	−16.171	−11.683	0.770
(Ghayoomi and Mousavian 2022)	[32]	−9.671	−11.690	−7.653	0.820	(Salem et al. 2021)	[15]	−6.805	−8.884	−4.726	0.800
(Fang et al. 2019)	[94]	−10.043	−12.048	−8.039	0.820	(Amoudi et al. 2022)	[95]	−8.589	−10.781	−6.398	0.780
(Karnyoto et al. 2022)	[96]	−7.449	−9.449	−5.449	0.820	(Dixit, Bhagat, and Dangi 2022a)	[97]	−11.095	−13.069	−9.120	0.820
(Kaliyar, Goswami, and Narang 2021b)	[98]	−9.954	−11.925	−7.983	0.820	(Umer et al. 2020)	[99]	−11.253	−13.234	−9.271	0.820
(Dixit, Bhagat, and Dangi 2022b)	[100]	−11.407	−13.622	−9.192	0.780	(Olaleye et al. 2022)	[101]	−12.240	−14.360	−10.120	0.800
(Islam et al. 2021)	[39]	−10.009	−12.039	−7.979	0.810	(Kasnesis, Toumanidis, and Patrikakis 2021)	[102]	−9.847	−11.823	−7.870	0.820
(Kapusta and Obonya 2020)	[103]	−5.358	−7.627	−3.090	0.770	(Althubiti, Alenezi, and Mansour 2022)	[104]	−10.83	−12.795	−8.865	0.830
(Fayaz et al. 2022)	[38]	−10.740	−12.727	−8.753	0.820	(Qasem, Al-Sarem, and Saeed 2021)	[19]	−8.269	−10.306	−6.232	0.810
(Lai et al. 2022)	[105]	−10.646	−12.626	−8.666	0.820	(Khan and Michalas 2021)	[17]	−10.861	−12.861	−8.860	0.820
(Karande et al. 2021)	[106]	−8.802	−10.810	−6.794	0.820	(Truică and Apostol 2022)	[20]	−11.591	−13.629	−9.553	0.810
(Nassif et al. 2022)	[48]	−9.222	−11.194	−7.250	0.820	(Panagiotou, Saravanou, and Gunopulos 2021)	[18]	−6.296	−8.341	−4.251	0.810
(Himdi et al. 2022)	[107]	−7.236	−9.442	−5.030	0.780	(Lee 2019)	[108]	−13.993	−16.034	−11.952	0.810
(Palani, Elango, and Viswanathan K 2022)	[109]	−10.025	−12.062	−7.987	0.810	(Cheng et al. 2021)	[110]	−9.770	−12.118	−7.423	0.750
(Biradar, Saumya, and Chauhan 2022)	[43]	−9.308	−11.299	−7.318	0.820	(Elhadad, Li, and Gebali 2020)	[21]	−8.924	−10.887	−6.961	0.830
(Dong, Victor and Qian 2020)	[111]	−11.869	−14.179	−9.559	0.760	(Ayoub, Yang, and Zhou 2021)	[112]	−9.372	−11.338	−7.406	0.830
(Buzea, Trausan-Matu, and Rebedea 2022)	[113]	−10.190	−12.169	−8.211	0.820	(Alouffi et al. 2021)	[114]	−7.004	−8.967	−5.041	0.830
(Hansrajh, Adeliyi, and Wing 2021)	[40]	−11.191	−13.386	−8.995	0.780	(Rajapaksha, Farahbakhsh, and Crespi 2021)	[115]	−10.780	−12.902	−8.658	0.800
(Saleh, Alharbi, and Alsamhi 2021)	[116]	−11.585	−13.689	−9.481	0.800	(Kumari et al. 2022)	[34]	−13.171	−15.226	−11.116	0.810
(Goldani, Momtazi, and Safabakhsh 2021)	[117]	−11.302	−13.650	−8.954	0.750	(Kula, Kozik, and Choraś 2021)	[118]	−9.779	−11.750	−7.808	0.820
(Das, Basak, and Dutta 2022)	[119]	−10.433	−12.446	−8.420	0.820	(Malla and Alphonse 2022)	[120]	−9.289	−11.260	−7.318	0.820
(Ghanem, Rosso, and Rangel 2020)	[121]	−12.442	−14.744	−10.140	0.760	(Apolinario-Arzube et al. 2020)	[122]	−9.328	−11.431	−7.225	0.800
(Qureshi et al. 2021)	[123]	−10.928	−12.952	−8.904	0.810	(Hayawi et al. 2022)	[124]	−11.366	−13.409	−9.322	0.810
(Rafique et al. 2022)	[9]	−6.853	−8.865	−4.841	0.820	(Rahman et al. 2021)	[125]	−10.303	−12.286	−8.320	0.820
(Aslam et al. 2021)	[51]	−8.532	−10.600	−6.463	0.810	(Ghaleb et al. 2022)	[126]	−13.414	−15.401	−11.427	0.820
(Jain et al. 2022)	[127]	−10.002	−12.879	−7.124	0.660	(Bezerra and Fabio 2021)	[128]	−10.909	−13.040	−8.778	0.790
(Agarwal et al. 2022)	[129]	−7.998	−9.985	−6.012	0.820	(Toivanen, Nelimarkka, and Valaskivi 2022)	[130]	−9.105	−11.231	−6.979	0.790
(Mahabub 2020)	[131]	−8.836	−10.852	−6.820	0.820	(Chintalapudi, Battineni, and Amenta 2021)	[132]	−8.152	−10.230	−6.074	0.800
(Al-Ahmad et al. 2021)	[133]	−8.594	−10.851	−6.337	0.770	(Pujahari and Sisodia 2021)	[134]	−9.711	−11.701	−7.721	0.820
(Albahar 2021)	[135]	−14.655	−16.772	−12.537	0.800	(Gayakwad et al. 2022)	[136]	−16.686	−18.684	−14.689	0.820
(Lee and Kim 2022)	[49]	−11.200	−13.408	−8.992	0.780	(Rastogi and Bansal 2022)	[137]	−8.016	−9.986	−6.046	0.820
(Jang, Park, and Seo 2019)	[138]	−8.034	−10.216	−5.852	0.780	(Shao and Chen 2022)	[139]	−7.815	−9.997	−5.633	0.780
(Sansonetti et al. 2020)	[140]	−13.442	−15.483	−11.401	0.810	(Mohammed et al. 2022)	[141]	−6.858	−9.363	−4.354	0.720
(Coste and Bufnea 2021)	[142]	−7.380	−9.661	−5.100	0.770	(Khan et al. 2022)	[26]	−7.181	−9.265	−5.097	0.800
(Alonso-Bartolome and Segura-Bedmar 2021)	[143]	−13.573	−15.674	−11.472	0.800	(Stitini, Kaloun, and Bencharef 2022)	[37]	−6.132	−8.135	−4.130	0.820
(Ozbay and Alatas 2019)	[144]	−9.561	−11.602	−7.520	0.810	(Wang et al. 2021)	[63]	−10.617	−12.699	−8.534	0.800
(Kanagavalli and Priya 2022)	[44]	−9.474	−11.448	−7.500	0.820	(Abdelminaam et al. 2021)	[64]	−12.803	−14.901	−10.705	0.800
(Ahmad et al. 2020)	[145]	−11.303	−13.357	−9.248	0.810	(Madani, Erritali, and Bouikhalene 2021)	[66]	−7.836	−10.042	−5.631	0.780
(Shang et al. 2020)	[146]	−15.803	−18.094	−13.512	0.760	(Endo et al. 2022)	[67]	−9.402	−11.423	−7.380	0.810
(Mazzeo, Rapisarda, and Giuffrida 2021)	[147]	−8.157	−10.158	−6.157	0.820
Theta		−9.942	−10.317	−9.567		Test of homogeneity: Q = chi2(124) = 501.340
Test of theta = 0		z = −51.910				Prob > |z| = 0.000 Prob > Q = 0.000

reveals that the between-study variability was high when considering the statistics of τ² = 3.4401. True heterogeneity to total observed variation I² = 75.270% (p < 0.001) was significantly high (25%, 50%, and 75% are respectively considered low, moderate, and high levels [61]), indicating that the variability is due to heterogeneity and not by chance. The high heterogeneity chi-square test result, (Q) = 501.340, is further evidence of the heterogeneity in effect sizes. An overall random pooled effect size of −9.942 within a 95% confidence interval (CI) of −10.317 to −9.567 was observed. The Forest plot (Figure 2) provided a graphical representation of the meta-analysis summary.

The Galbraith plot (Figure 3) shows a strong relationship between the sample size and the accuracy achieved by the model used, with the negative slope of the regression line further indicating that the accuracy reduces as the sample size increases. With only eight studies falling outside the 95% CI, this further supports high heterogeneity.

4.2. Subgroup Analysis

While the heterogeneity measures do provide an overall measure, they do not provide an indication of the source of heterogeneity. Subgroup analysis was therefore performed so that the source and level of heterogeneity between each group (each approach) could be determined. Creating subgroups allows comparisons to be made between data groups, and the interpretation can lead to informative insights into different approaches.

Table 4. Subgroup analysis for the comparison of different approaches.

Group	Number of Studies	ES 95% CI	Q	I2	Test for Heterogeneity
					df	p-Value
Deep learning	60	−10.08 [−10.60, −9.58]	202.90	70.92	59	0.000 *
Ensemble deep learning	11	−10.03 [−10.65, −9.40]	8.70	0.00	10	0.561
Ensemble machine learning	16	−10.23 [−11.00, 9.46]	34.29	56.25	15	0.003
Hybrid	10	−11.13 [12.36, −9.90]	38.22	76.45	9	0.000 *
Machine learning	27	−8.98 [−10.04, −7.91]	181.83	85.70	26	0.000 *
Sentiment analysis	1	−8.15 [−10.23, −6.07]	0.00	0.00	0	-
Overall	125	−9.94 [−10.32, −9.57]	501.34	75.27	124	0.000 *

* p < 0.001.

, which provides a summary of the subgroup analysis per approach, shows notable differences. The machine learning approach featured as the primary contributor to the high level of heterogeneity (I² = 85.70%, heterogeneity chi-square = 181.83, a degree of freedom = 26, and p < 0.001). Hybrid models also had high heterogeneity (I² = 76.45%, heterogeneity chi-square = 38.22, a degree of freedom = 9, and p < 0.001). Moderate to high heterogeneity was evident for deep learning models (I² = 70.92%, heterogeneity chi-square = 202.90, a degree of freedom = 59, and p < 0.001). Ensemble machine learning approaches presented with significantly moderate heterogeneity (I² = 56.25%, heterogeneity chi-square = 34.29, a degree of freedom = 15, and p = 0.003), while ensemble deep learning was not significantly heterogenous and was, therefore, rather homogenous (I² = 0.00%, heterogeneity chi-square = 8.70, a degree of freedom = 10, and p = 0.561). The heterogeneity of the sentiment analysis approach could not be determined and was not relevant as it has a degree of freedom = 0.

4.3. Meta-Regression

Due to the difference in sample sizes, the year of study, and the approach used, the parameters responsible for heterogeneity need to be examined further. Meta-regression was used to investigate the sources of heterogeneity, with parameters of sample size, year, and approach being used as moderators. The results in

Table 5. Meta-regression model to assess the source of heterogeneity.

Sources of Heterogeneity	Estimates	Std. Error	95% CI	p-Value
Year	0.361	0.201	[−0.037, 0.756]	0.075
Approach	−0.070	0.111	[−0.290, 0.149]	0.526
Sample size	−0.000	0.000	[−0.000, −0.000]	0.000 *
Constant	−734.199	406.931	[−1539.826, −71.429]	0.074
Year	0.361	0.201	[−0.037, 0.756]	0.075

* p < 0.001; Test of residual homogeneity: Q_res = chi²(121) = 353.71 Prob > Q_res = 0.0000.

indicate that sample size was the only significant cause of heterogeneity, with p < 0.001. This is further supported by the Bubble plot on year (Figure 4) and on sample size (Figure 5), where the distribution of studies on sample size was far more widespread.

4.4. Publication Bias

Publication bias cannot be avoided in systematic reviews and meta-analyses [148], with the literature suggesting that this bias be evaluated so that sound conclusions surrounding the extent to which bias may influence the generalizability of findings are addressed. The publication bias for this study was visually evaluated using a funnel plot. The symmetrical distribution of studies within the triangular region of Figure 6 indicates that this bias is not relevant. Due to the interpretation of the funnel plot relying on a visual evaluation, it can be subjective. For this reason, Egger’s test was performed as a quantitative measure of publication bias. From

Table 6. Egger’s test for examining publication bias.

Parameter	Estimate	Std. Error	t	p	95% Conf. Interval
Slope	−9.641	2.723	−3.54	0.001	−15.031	−4.250
Bias	−0.282	2.545	−0.11	0.912	−5.319	4.755

Test of residual homogeneity: Q_res = chi²(121) = 353.71 Prob > Q_res = 0.0000.

, it is evident that no significant bias exists, as the p-value obtained from the Egger’s test was 0.912.

4.5. Descriptive Statistics of Primary Studies

The publication trends for the years 2019 to 2022 are illustrated in Figure 7, where it is evident that the interest in methods for fake news detection increased significantly from five in 2019 to 15 in 2020, with 2021 seeing 51 publications. The year 2022 reports the highest number of studies, with 54 publications in the timeframe of January to August. The reported plethora of fake news on social media surrounding the COVID-19 pandemic could have contributed to this increased interest in detection methods [79,119]. From Figure 8, it is evident that the methods used most often are those based on a deep learning approach, with Figure 7 showing that this approach has seen an increase in popularity over the years investigated. While the number of studies based on machine learning approaches in 2021 and 2022 was almost the same (Figure 7), it must be noted that the analysis was based on data collected up to August 2022; therefore, there is a high possibility that additional publications may occur in the journals being published in later months of 2022. Ensemble approaches also have the potential to see an increasing number of publications, but, with this being a newer area, their value is yet to be established.

With DL and ML emerging as the approaches relied upon most often for fake news detection, and the existence of a number of methods within each of these approaches, Figure 9 represents the DL and ML methods with the highest frequency of use in the 125 articles analyzed. The most common method applied in DL approaches is CNN (10), with this being followed by LSTM (7) and BiLSTM (6). ML approaches are most reliant on RF (13), with SVM (3) and LG (3) being used but to a far lesser extent.

5. Conclusions

This study employed a systematic review and meta-analysis methodology to quantitatively evaluate the fake news detection methods based on DL, ML, and ensemble. A database, created with nine variables related to these methods using data from 125 scientific articles, was the basis for the meta-analysis. For the included studies, effect sizes, heterogeneity, subgroup analysis, meta-regression analysis, and publication bias were all addressed. This was due to the different sample sizes and approaches that were previously used in the methods. The main approaches used in the literature were deep learning, ensemble deep learning, ensemble machine learning, hybrid, machine learning, and sentiment analysis.

The results led to the following deductions.

• Deep learning was the most widely used approach, with the CNN method most commonly employed due to its most effective architecture for accurate and efficient detection.
• The most used method in machine learning is RF. It is capable of handling hundreds of input variables and performs well on large datasets. Additionally, RF calculates the relative value of every feature and creates an incredibly accurate classifier.
• The sample sizes used by each study to establish detection accuracy varied significantly. The sample size and the accuracy of the fake news detection method are strongly negatively correlated. This underscores how crucial it is to use a large number of samples when testing fake news detection methods. Further, the sample size utilized to determine the detection accuracy was a major contributor to heterogeneity.
• The findings of the study revealed the existence of heterogeneity and revealed a trivial publication bias, demonstrating the effectiveness of the inclusion and exclusion criteria in reducing bias.
• Finally, the meta-analysis results revealed that the efficacy of the various proposed approaches from the included primary studies was sufficient for the detection of online fake news.

The meta-analysis carried out in this work contributed to highlighting the improvements in detection techniques, with a particular focus on deep learning, machine learning, and ensemble approaches. Review findings also support the importance of deep learning and machine learning techniques. The meta-analysis enables the presentation of transparent, unbiased, and repeatable summaries of the fake news detection techniques. The study acknowledges the important relationship between sample size and detection accuracy in light of the findings. This insightful meta-analysis helps comprehend the recent developments in the research area. It is believed that this meta-analysis highlights current state-of-the-art methods and, more importantly, provides direction for the further investigation of novel methods for spotting fake news.

The review was performed on methods that rely on supervised learning models, which may be a limitation to the study; therefore, further studies that consider semi-supervised and unsupervised models may reveal additional results. Moreover, with this study relying on literature in only the Web Of Science database, further research that includes numerous databases and additional performance evaluations other than accuracy is advised.