Computer-Aided Diagnosis Systems for Automatic Malaria Parasite Detection and Classification: A Systematic Review

Abstract

Malaria is a disease that affects millions of people worldwide with a consistent mortality rate. The light microscope examination is the gold standard for detecting infection by malaria parasites. Still, it is limited by long timescales and requires a high level of expertise from pathologists. Early diagnosis of this disease is necessary to achieve timely and effective treatment, which avoids tragic consequences, thus leading to the development of computer-aided diagnosis systems based on artificial intelligence (AI) for the detection and classification of blood cells infected with the malaria parasite in blood smear images. Such systems involve an articulated pipeline, culminating in the use of machine learning and deep learning approaches, the main branches of AI. Here, we present a systematic literature review of recent research on the use of automated algorithms to identify and classify malaria parasites in blood smear images. Based on the PRISMA 2020 criteria, a search was conducted using several electronic databases including PubMed, Scopus, and arXiv by applying inclusion/exclusion filters. From the 606 initial records identified, 135 eligible studies were selected and analyzed. Many promising results were achieved, and some mobile and web applications were developed to address resource and expertise limitations in developing countries.

1. Introduction

Malaria is a deadly disease worldwide and is known as a major cause of infant mortality. Statistics from the World Malaria Report 2023 [1] compiled by the World Health Organization (WHO) show that 249 million cases of malaria were detected globally in 2022, 5 million more than the previous year. The main countries contributing to the increase in cases between 2021 and 2022 were Pakistan, Ethiopia, Nigeria, Uganda, and Papua New Guinea. Between 2000 and 2022, the mortality rate for malaria halved from 28.8 per 100,000 population at risk to 14.3, and the proportion of total deaths from malaria among children under the age of 5 decreased from 86.8% in 2000 to 76.0% in 2022.

1.1. Malaria Parasite Transmission and Life Cycle

Malaria transmission occurs through the bite of an infected female Anopheles mosquito, which transfers the parasite to humans by attacking their red blood cells (RBCs). The parasitic protozoan that causes malaria infection is of the genus Plasmodium [2], which has four types of species, namely P. falciparum, P. vivax, P. ovale, and P. malariae, of which the former is the most dangerous. Plasmodium matures and reproduces through a complex cycle [3,4] that varies in relation to the temperature of the environment [5], slowing down in cold periods, and during which it undergoes a change in its shape, size, morphology, color and so on, depending on which phase it is in, i.e., ring, trophozoite, schizont, or gametocyte [6]. The different stages in the life cycle of the Plasmodium parasite are shown in Figure 1.

1.2. Malaria Diagnosis and Treatment

To date, there has been no effective vaccine for the treatment of malaria due to the high diversity and adaptability of Plasmodium antigens, which would, therefore, require the development of specific solutions [7]. However, it is clear from recent WHO analyses that there has been a decrease in malaria deaths in recent years, despite an increase in cases worldwide. Malaria is a curable disease if diagnosed early. Currently, the gold standard for diagnosis and detection of different types of malaria is a visual inspection of Giemsa-stained blood smears using hand-held light microscopy [8], which requires an interpretation of the results by an experienced pathologist. Microscopic examination can be conducted on both thin and thick blood smears. The thick blood smear is 6 to 20 times thicker than a single layer of RBCs, making it 20 to 40 times more sensitive in detecting malaria parasites than a thin smear. Indeed, the blood smear method is often suitable for detecting parasites with an 11-fold higher sensitivity rate [9], but not for counting infected RBCs; it is, therefore, a method to be avoided in conditions of high parasitemia. In addition, further image artifacts in thick smears interfere with reading and bring more image processing challenges. On the other hand, the thin blood smear preserves the shape of the RBCs, allowing for them to be counted, and is used to visualize a parasite within a cell. Additional techniques for malaria detection include the antigen test, a fast but less precise process, and the polymerase chain reaction process, precise but expensive. Microscopic analysis is, therefore, the one generally used, but it requires laboratory instruments, a great deal of experience and time. Since people can be infected with more than one malaria parasite at the same time, it is necessary to examine the entire blood smear sample, a very time-consuming procedure for pathologists who examine voluminous samples. The manual evaluation and cell count in a thin blood smear sample takes an expert about 15 min [10], and the accuracy of the diagnosis directly depends on their experience. Due to the lack of equipment and human resources, malaria is difficult to detect in rural areas and developing countries, where only the patient’s history and symptoms are considered to reach a diagnosis [11], resulting in a delay in the diagnosis of the disease. To formulate an effective and timely treatment for malaria, in addition to early diagnosis, it is essential to recognize the degree of spread, quantify the number of parasites, and identify their life stages.

1.3. Automatic Malaria Detection and Diagnosis

To overcome the obstacles just discussed and to ensure rapid and effective malaria diagnosis, as well as a low cost and equal accuracies worldwide, automated methods must be used. Conventional image processing techniques, whose simplicity and low computational cost make them potentially suitable for contexts where it is impossible to acquire large-scale marked data, have shown promising results in various scenarios including the determination of blood film parasitization [12]. However, computer-aided diagnosis (CAD) systems based on artificial intelligence (AI) have gained much more popularity in recent decades [13] and can also be adapted for malaria parasite detection. The traditional pipeline of CAD systems for malaria diagnosis follows the following steps: data acquisition and pre-processing, cell segmentation, feature extraction, and finally object detection and classification of infected cells, parasite type, and life stage. The first step is to acquire images of the blood smears using the cameras already mounted on the light microscopy device or using external devices such as smartphones. Samples are generally stained with Giemsa, the recommended and most reliable procedure for staining thick and thin blood films to detect and identify blood parasites. The Giemsa solution consists of eosin, which colors the parasite nucleus in red, and methylene blue (light blue), which colors the cytoplasm in blue. However, automated malaria diagnosis works have been published in the literature that make use of the combination of the Giemsa reagent with other types of dyes, e.g., Wright-Giemsa, May Grünwald–Giemsa, and Leishman–Giemsa. Once the data have been acquired, pre-processing methods aim to improve the quality of blood smear images by removing impurities that may affect the accuracy of subsequent processing steps, especially cell segmentation. To this end, Gaussian, median, and geometric filtering techniques and morphological operators are commonly used for noise reduction [14,15,16], while adaptive thresholding and histogram equalization are used to improve image resolution and contrast [12]. In addition, normalization techniques are performed to reduce illumination variation in images. To increase the size of image datasets and, consequently, the performance of AI models, some transformations of data augmentation (DA) are applied to the initial data to create new artificial examples [17]. Among the classic operations used for data augmentation are rotation, zooming, cropping, noise addition, scaling, and translation [18]. In addition to these standard operations, more advanced transformations such as changing contrast or brightness can be applied. DA techniques are also applied to balance the data in the different classes. The next step of cell segmentation is significant in automatic malaria detection systems and aims to identify RBCs, white blood cells (WBCs), parasites, and other artifacts, and then separate them from each other and the background. This can be performed manually by experts, requiring very long time frames, or by image processing techniques. In the latter case, thresholding or watershed methods are the most frequently used, along with machine learning (ML) and deep learning (DL) techniques such as clustering algorithms and deep artificial neural networks (ANNs) for segmentation. Features relevant for their subsequent classification are then extracted from the segmented cells and parasites. This can be carried out manually, pointing to extract shape, color, and texture information preferring the HSV color space and the green channel of the RGB color space [19], or automatically using AI algorithms. The last step in the pipeline of CAD systems for automatic malaria diagnosis is to detect and classify parasites. At this stage, once Plasmodium-infected cells have been identified, a cell count can be performed to determine the degree of parasitemia, but also to determine the infecting type of parasite and its life cycle stage. Classic ML methods make use of the features extracted in the previous processing step, while DL methods automatically extract the information most relevant to the purpose of the analysis. Although these two approaches are generally used separately, some works propose hybrid ML/DL models or even ensemble learning systems to increase the diagnostic performance of CAD systems. In particular, ensemble learning reduces model variance by optimally combining predictions from multiple models and decreasing sensitivity to the specification of training data and selection of training algorithms. The CAD systems pipeline for the malaria just discussed is briefly summarized in Figure 2.

CAD systems for malaria diagnosis are proving to be useful tools for increasing diagnostic accuracy, limiting intra- and inter-operator variability, and reducing time and human resources. Moreover, some of these systems have been integrated into mobile applications that can be used by anyone anywhere in the world, even without an internet connection. This systematic review aims to provide a comprehensive overview of the last ten years of the literature on the applications of AI to the automatic diagnosis of malaria, and to analyze its performance.

2. Methods

The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines [20] were followed in the present study. More details on the search, screening, and article selection strategies are given in the following subsections.

2.1. Data Sources and Search Strategy

To identify the recently published articles concerning the use of ML and DL for malaria diagnosis from tissue or cell images, the electronic databases PubMed, Scopus, and arXiv were considered, imposing the timespan between 1 January 2014 and 13 February 2024. The keywords Artificial Intelligence, Machine Learning, Deep Learning, Malaria, Images, Classification, and Diagnosis were used and combined with the logical operators “AND” and “OR” as illustrated in Figure 3.

In addition, we set filters to limit the publication language to English and to select only original articles in the Scopus database. After collecting all the articles resulting from the automatic search, duplicates were removed.

2.2. Inclusion and Exclusion Criteria

Since this systematic review aims to collect and analyze studies on the applications of ML and DL to automatic diagnosis of malaria on images, we established the following exclusion criteria for the articles selected: lack of application of automatic diagnosis techniques, implementation of segmentation tasks not followed by the automatic classification, focus on diseases other than from malaria, and analysis of data other than images. In addition, we excluded reviews and systematic reviews, non-publicly accessible articles, and work with unclear or non-explicit results. Moreover, the previous set of search criteria excludes papers not written in English and those published before 2014. The inclusion criteria were the negation of the exclusion criteria.

2.3. Quality Assessment

Quality assessment of primary research is a key step in systematic reviews. It is necessary for a complete and transparent review process, the results of which are not influenced by poorly conducted data collection. Therefore, in the present work, the standard questions of the quality checklist (SCQ) designed in [21] were applied, and, following [22], articles that answered “yes” to at least 7 of the 10 questions in

Table 1. Quality checklist.

No	Quality Question
SCQ1	Is the report clear and coherent?
SCQ2	Is the aim of the research clearly specified?
SCQ3	Is the data collection method clearly described?
SCQ4	Have the diversity contexts been well explored?
SCQ5	Are the findings of the study reliable?
SCQ6	Are there links between the data, interpretation, and conclusion?
SCQ7	Are the methodology and experimentation process clear?
SCQ8	Are the research procedures documented adequately?
SCQ9	Are they important, if credible?
SCQ10	Could the research findings be replicated?

were deemed eligible.

2.4. Data Extraction

Data extraction was performed following the PRISMA guidelines [20] (Figure 4).

The examination and screening of all the search results were conducted by two independent authors (F.G. and A.A.). The initial search of electronic databases generated 606 articles (257 PubMed, 314 Scopus, and 35 arXiv). Among these articles, 398 were eliminated because they were duplicated records. After reading the title and abstract, 45 articles were discarded because they were outside the topic of the present review, following the exclusion criteria described above. In the next step, articles were carefully selected by reading the remaining 163 articles. Of these, 22 articles were excluded after reapplying the previously established inclusion/exclusion criteria, and a further 6 failed the quality analysis test. Finally, a total of 135 articles were deemed suitable for a full evaluation of the reported results. Data were collected and extracted from the selected articles by highlighting the following key aspects: malaria dataset used, pre-processing technique applied, task performed, AI tools used, results obtained (quantified by evaluation metrics), cross-validation and external validation conducted, and eXplainable AI (XAI) method used, where present. This information has been sorted and set out in summary table following the manuscript organization.

3. Artificial Intelligence Background

The purpose of this systematic review is to summarize and analyze recent applications of AI and CAD systems to malaria diagnosis. In this section, we briefly summarize the main information behind these concepts to provide a general overview of what is discussed in the following sections.

3.1. ML and DL

AI, which tries to perform some of the tasks characteristic of human intelligence like learning, has now become part of our daily lives, including medical applications. Two main branches of AI are ML and DL, a series of models and algorithms that autonomously extract useful information from data and learn from them with the aim of make predictions on new data never seen before. The difference between these two types of approaches is that DL, the most successful ML solution, exploits complex architectures, characterized by many layers and many parameters. The principal ML models include Decision Trees (DT) [23], Support Vector Machines (SVM) [24], K-Nearest Neighbors (KNNs) [25], and simply ANNs [26]. For DL instead, the most effective and widely used algorithms in computer vision (CV) are the convolutional neural networks (CNNs or ConvNets), networks composed of several layers, such as convolutional, max-pooling, normalization, dropout, fully connected, and output layers, each with its own specific function [27]; in fact, some of them have trainable parameters, while others only have the task of implementing an established function. Through these layers, each input is transformed down to the last layer where a class-based probability of that image is produced. Other commonly used DL approaches in CV are autoencoders (AEs) [28], widely applied in dimensionality reduction and feature learning, and Generative Adversarial Networks (GANs) [29], a class of deep networks for generative modeling. Training DL models with randomly initialized parameters requires a large amount of labeled data not readily available. Transfer learning (TL) represents the best solution by allowing the reuse of knowledge (the weights) extracted from a pre-trained CNN model on large labeled datasets and achieving good results in the source domain. This operation can be used as a feature extraction model but also to refine hyperparameters by freezing or unfreezing various layers. Several CNN architectures are available as pre-trained models [30,31,32], and the most commonly used ones are summarized in

Table 2. Overview of common pre-trained models.

Model	Variants	Parameters	Layers	Input Size
ResNet	ResNet-18	11.17 M	18	224 × 224
	ResNet-50	25.60 M	50	224 × 224
	ResNet-101	44 M	101	224 × 224
DenseNet	DenseNet-121	17 M	121	224 × 224
	DenseNet-169	29 M	169	224 × 224
	DenseNet-201	20.2 M	201	224 × 224
Inception-ResNet	Inception-ResNetV1	24 M	60	299 × 299
	Inception-ResNetV2	56 M	164	299 × 299
MobileNet	MobileNet	13 M	30	224 × 224
	MobileNetV2	3.5 M	53	224 × 224
VGG	VGG-16	138 M	16	224 × 224
	VGG-19	143 M	19	224 × 224

.

3.2. Evaluation Metrics

To evaluate the learning algorithms in a classification task, it is possible to use some synthetic indices [33], summarized in

Table 3. Summary of the evaluation metrics.

Metric	Formula	Description
Recall (REC) or Sensitivity	${\displaystyle \frac{TP}{TP+FN}}$	A measure of an algorithm’s ability to correctly predict TP from the total number of positive observations
Specificity (SP)	${\displaystyle \frac{TN}{TN+FP}}$	Calculation of the percentage of TN out of the total number of negative observations predicted by an algorithm
Precision (PR)	${\displaystyle \frac{TP}{TP+FP}}$	The extent to which the algorithm is correct in terms of positive results. It is calculated as the TP over the total number of positive observations
Accuracy (ACC)	${\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$	General description of the performance of the model on all classes. It is calculated as the ratio of correct predictions to the total number of predictions
F1-score (F1)	$2\times {\displaystyle \frac{TP}{P+P^{\prime }}}$	The harmonic mean of precision and sensitivity
Matthews correlation coefficient (MCC)	${\displaystyle \frac{(TP\times TN)-(FP\times FN)}{\sqrt{\left(P^{\prime }\right)(TP+FN)(TN+FP)\left(N^{\prime }\right)}}}$	Evaluation of a binary classification model performance. Value in range [−1, 1], where the minimum value indicates a poor classifier while the maximum value indicates a correct classifier

, from the scores of the true positive (TP), true negative (TN), false positive (FP), and false negative (FN) predictions. By P and N we mean the positive and negative instances ($P=TP+FN$; $N=TN+FP$), and by P′ and N′ those for which the output of the classification (correct or incorrect) is the positive and negative class ($P^{\prime }=TP+FP$; $N^{\prime }=TN+FN$).

In medical diagnosis, the goal is to have a high sensitivity, indicating the correct classification of the disease, but also a low false positive rate (FPR) to reduce patient and physician concerns. In addition, it is crucial to achieve a high diagnostic accuracy, that is, a high frequency with which a cancer diagnosis is actually related to a sick patient. If the dataset is balanced, the accuracy metric is valid for evaluating the model; otherwise, the other metrics are more reliable. For example, Receiver Operating Characteristic (ROC) curves [34] are used to select classification models based on their performance against true positive rate (TPR) and FPR values, calculated by varying the decision threshold of the classifier. From the ROC graph, it is possible to derive the Area Under Curve (AUC), representing the area enclosed under the ROC curve. The closer the AUC value is to 1, the better the classifier.

3.3. Cross-Validation

After training, the ML and DL models are tested on new data to assess their performance. In common practice, the dataset is divided into two parts, one of which is used to train the model and the other to test it. This method, called Holdout Cross-Validation, is easy to implement and computationally inexpensive. However, not all of the dataset is used for training and the split can lead to estimates that are highly dependent on the way the data were split. To obtain a reliable estimate of the model’s ability to generalize previously unseen data, it is best to use other statistical cross-validation techniques. The most commonly used method is the k-fold one [35], in which the dataset is divided into k subgroups (or “folds”) of approximately equal sizes; the model is trained k times, using each time k − 1 fold for training and the remaining fold for validation; and finally, the performance of the model is evaluated as the average of the performance on the k folds. Usually, k is set as 5 or 10. The extreme case of k-fold cross-validation is the so-called Leave-One-Out Cross-Validation, where k is equal to the number of instances in the dataset. In this variant, every instance is used as a validation fold once, and all others are used for training.

3.4. External Validation

External validation refers to the validation of ML and DL models on sources external to those used during training and testing. In the medical field, such models tend to perform well on data from the same cohort but do not perform as well on new data (of new patients, perhaps from other centers), often due to overfitting or co-variate shifts [36]. Therefore, to evaluate the application of medical AI, it is necessary to perform such validation [37] on datasets that are either public or private.

4. Public Malaria Datasets

The growing interest in AI applications in the medical field, and in particular for the diagnosis of malaria, has led researchers around the world to collect images of thick and thin blood smears of malaria patients. Many of the data collected are then made public to offer the possibility of using them to develop ML and DL approaches and to compare their performance with other works. The most used public datasets for malaria classification are listed below.

• The National Institutes of Health (NIH) dataset consists of a set of thick and thin smears’ images of infected (P. falciparum and P. vivax) and uninfected patients [38] and a set of already segmented data [39]. The latter is the most widely used and contains 27,558 cells segmented from images of thin Giemsa-stained blood films captured with a mobile phone mounted on a microscope. The cells, manually annotated by an expert, are perfectly balanced between healthy and infected from P. falciparum.
• The MaMic Image Database [40] contains images of blood slides that are used to identify P. falciparum. These images are captured with a microscopic camera that scans the entire slide, obtaining approximately 549 images with a resolution of 1280 × 1024 pixels. A total of 16,991 cells are segmented and manually annotated as definite or uncertain parasites by a trained expert.
• The MP-IDB dataset [41] contains four different genera of malaria parasites including P. vivax, P. falciparum, P. ovale and P. malaria, each with four distinct life cycle stages (ring, trophozoite, schizont, and gametocyte). In total, 48,000 blood cells are extracted from the 229 images of thin blood smears stained with Giemsa, which are then labeled by experienced pathologists.
• The BBBC041 dataset [42] contains a total of 1364 images manually annotated by three world-class experts. The 80,000 cells contained in the images belong to six distinct classes, two of which are uninfected (RBCs and leucocytes) and four parasitized (gametocytes, rings, trophozoites, and schizonts). In addition to the phase labels, the coordinates of the bounding boxes for each cell are also provided.

5. Results

This section summarizes articles on the use of AI for malaria detection and diagnosis collected from the literature. Of the 606 articles resulting from the initial search, 135 were considered eligible (see Section 2.4 and Figure 4). The organization of these studies is as follows: first, all articles using the public NIH malaria dataset are described; then, the remaining ones are analyzed. Special subsections are then dedicated to articles proposing generalizable approaches also tested on the automatic malaria diagnosis task and articles proposing methods for multiple diagnoses, including malaria.

5.1. Articles Using the NIH Malaria Dataset

For the recognition and classification of malaria on the NIH dataset, several ML and DL approaches (based on custom CNNs or pre-trained networks) have been proposed. In this section, we summarize the work resulting from the research conducted, dividing it according to the AI method used. The extracted information is summarized in dedicated tables, reporting only ACC and AUC performance due to the balance of the data. We report other metrics only if ACC and AUC are not present.

5.1.1. Pre-Trained Networks

For the task of automatic diagnosis of malaria in RBC images, among the most frequently adopted strategies is TL in pre-trained neural networks, which are considered an excellent compromise between performance effectiveness and computational efficiency. In this scenario, Zhao et al. [43] used SSD300 [44] for object detection in blood smears, developed a resolution upscaling model using a Fast Super-Resolution CNN (FSRCNN) network for all those cases where the resolution of RBCs images is insufficient, and classified them using the modified VGG-16 network. Sriporn et al. [45] used a pre-trained Xception with Mish activation function [46] and Nadam optimizer [47]. Kassim et al. [48] proposed the PlasmodiumVF-Net framework for the analysis of thick smear microscopy images for malaria diagnosis at the image and patient levels. They used two Mask R-CNNs to detect candidate RBCs for the presence of the parasite, and then two pre-trained ResNet-50s (one after each previous pattern) to filter FPs. Finally, they classified the patient based on the number and score of parasites detected in the first step so that if less than two RBCs are found to be parasitic, the patient is labeled as healthy. Sangameswaran et al. [49] developed Mass-AI-Scope (MAIScope), a novel, low-cost, portable device that automatically detects malaria parasites with two subsystems, the first of which is a pre-trained EfficientNet-lite4 network that detects RBCs, followed by a pre-trained MobileNetV2 that recognizes malaria parasites, while the second is the hardware, consisting of components such as Raspberry Pi, a camera, a touchscreen display, and an innovative low-cost microscope. Alnussairi et al. [50] classified segmented patches from microscopic images of RBC smears using pre-trained networks such as VGG-19, ResNet-50, and MobileNetV2 trained for a few epochs. Sinha et al. [51] report an automated CAD method to classify malarial thin smear blood cell images as parasitized and uninfected by using the pre-trained ResNet-50. Kundu et al. [52] proposed an intelligent hyper-parameter tuned DL-enabled malaria parasite detection and classification (HPTDL-MPDC) technique for blood smear images, which incorporates data pre-processing, VGG-19-based feature extraction, both deep ANN with Long Short-Term Memory (LSTM) and custom CNN for classification, and Adagrad optimization-based hyperparameter tuning.

Extra details on data manipulation, pre-trained network used, and results are given in

Table 4. Overview of cited works using pre-trained networks to classify NIH datasets (results have been rounded). The symbol “-” indicates that no information is provided on a particular operation. The abbreviation “Av” indicates the average results over several repetitions of the experiment.

Pre-Processing	Data Augmentation	Network	Results	Cross-Validation	External Validation	Author and Year
Rescaling $128\times 128\times 3$	-	SSD300 + modified pre-trained VGG-16	Original images: ACC 96.00 ± 0.3%; AUC 99.2 ± 0.2%. Images after FSRCNN: ACC 96.53 ± 0.43%; AUC 99.40 ± 0.10%	5-fold	BBBC041: AUC 94.50 ± 2.5%	Zhao et al., 2020 [43]
Rescaling $299\times 299\times 3$; normalization [0, 1]	Random rotation between 0° and 270°	Pre-trained Xception	ACC 98.86%	Hold-Out	-	Sriporn et al., 2020 [45]
-	-	Mask R-CNN + 2 pre-trained ResNet-50	Image level: Av ACC 83.9%. Patient level: Av ACC 92.3%	5-fold	-	Kassim et al., 2021 [48]
Rescaling $224\times 224\times 3$	Rotation, tone, etc.	Pre-trained EfficientDet-lite4 + MobileNetv2 (MAIScope)	Av ACC 89.90%	Hold-Out	Images of malaria field microscope slides provided by Cancyte Labs, India	Sangameswaran et al., 2022 [49]
Rescaling $50\times 50\times 3$; normalization [0, 1]; color standardization	Rotation, flipping, horizontal/vertical shift, lightning or darkening, zooming in and out	Pre-trained VGG-19, ResNet-50, MobileNetV2	ACC 100.00%	Hold-Out	-	Alnussairi et al., 2022 [50]
-	MixUp [53] and CutMix [54] algorithms	Pre-trained ResNet-50	ACC 98.75%	Hold-Out	-	Sinha et al., 2023 [51]
Rescaling $120\times 120\times 3$	Rotation (${20}^{\circ }$), zooming (0.05), width/height shifting ($\pm 0.05$), shearing (0.05), and flipping	Pre-trained VGG-19 + Custom CNNs (HPTDL-MPDC)	ACC 89.00%	5-fold	-	Kundu et al., 2023 [52]

.

Staying on the topic of using TL for automatic malaria diagnosis, several papers report the results of comparing different networks, both customized and pre-trained, which are generally superior using the latter. Rajaraman et al. [55] proposed a system for cell segmentation and classification of infected and non-infected cells for which they tested custom CNNs and several pre-trained networks, the best of which turned out to be ResNet-50 after the optimization of hyperparameters with the randomized grid search method. Rahman et al. [56] focused on improving malaria detection from segmented patches from microscopic images of RBC smears by introducing a deep CNN. They tested a custom CNN, the hybrid VGG-16 + SVM solution, an ensemble learning approach, and the pre-trained VGG-16, of which the latter proved to be the best. Jameela et al. [57] developed a Malaria Diagnosis System for automatic diagnosis based on the intensity features of Plasmodium parasites and erythrocytes using ResNet-50, ResNet-34, VGG-16, and VGG-19 pre-trained, the last being the best. Loddo et al. [58] compared 11 existing architectures for multiclass classification on parasite life stages and evaluated the robustness of the models on two different public datasets. Turuk et al. [59] tested three pre-trained networks, demonstrating the superiority of the modified VGG-19 over all of them. Furthermore, Qadir et al. [60] tested different TL approaches and reported the best in DenseNet in its variants DenseNet-121, DenseNet-169, and DenseNet-201. Hassan et al. [61] developed a malaria CNN by fine-tuning InceptionV3, ResNet-50, and VGG-19, the last of which had the best classification result. Shewajo et al. [62] proposed an efficient and robust image processing method based on tiles containing the malaria parasite and tested three variants of YOLOV4-based object detectors: a modified version (YOLOV4-MOD) and two versions with a lightweight architecture: YOLOV4-tiny with two detector heads and YOLOV4-tiny-3L on three detector heads. To conduct a performance analysis of DL algorithms in the diagnosis of malaria disease, Hemachandran et al. [63] compared a custom CNN with MobileNetV2 and ResNet-50, with the latter being the best. Dath et al. [64], after applying noise reduction techniques, image segmentation, and GANs as a DA strategy, implemented a custom CNN, a pre-trained ResNet-50, and a pre-trained VGG-19, of which the latter was the best.

Further information on the work just discussed is given in

Table 5. Overview of cited works comparing some custom and pre-trained CNN approaches to classify NIH datasets (results have been rounded). The symbol “-” indicates that no information is provided on a particular operation. The abbreviation “Av” indicates the average results over several repetitions of the experiment.

Pre-Processing	Data Augmentation	Best Model	Results	Cross-Validation	External Validation	Author and Year
Rescaling $224\times 224\times 3$	-	ResNet-50	Cellular level: ACC 98.60%; AUC 99.90%. Patient level: ACC 95.9 ± 0.8%; AUC 99.10 ± 0.5%	5-fold	-	Rajaraman et al., 2018 [55]
Rescaling $200\times 200\times 3$; Gaussian blurring; darkening and lightening; ZCA whitening; feature-wise standardization	Vertical/horizontal flipping and shifting, rotation ($\pm {25}^{\circ }$), clipping 0-20% of height/width, translation ($\pm 0.2$), and cutting ($\pm 25$)	Pre-trained VGG-16	Image level: ACC 97.00 ± 0.5%. Patient level: ACC 93.00%; AUC 86.50%.	5-fold and Hold-Out	-	Rahman et al., 2019 [56]
Rescaling $224\times 224\times 3$; normalization	-	Pre-trained VGG-19	ACC 97.21%	Hold-Out	-	Jameela et al., 2022 [57]
Color adjustment (multiplication of each channel by the ratio of the average of the three channels to the average of the channel under consideration)	Random rotations ($\pm {90}^{\circ }$), random translations ($\pm 10$ pixels) on the X-axis and Y-axis, and 50% reflection around the X-axis and Y-axis.	Pre-trained ResNet-18 and DenseNet-201	ResNet-18 binary classification on NIH: 97.68%. DenseNet-201 multiclass classification on MP-IDB-FC: ACC 99.40%	5-fold	DenseNet-201 trained on NIH and tuned to MP-IDB-FC: ACC 97.45 ± 1.40%. DenseNet-201 trained on MP-IDB-FC and tuned to MP-IDB-VC (P. vivax): ACC 87.10 ± 1.15%	Loddo et al., 2022 [58]
-	-	Pre-trained VGG-19	ACC 93.89%	Hold-Out	-	Turuk et al., 2022 [59]
-	-	Pre-trained DenseNet	Densenet-121: REC 94.90%. DenseNet-169: PR 95.97%. DenseNet-201: ACC 93.39%	Hold-Out	-	Qadir et al., 2022 [60]
Rescaling $224\times 224\times 3$; normalization [0, 1]	Rotation (range of ${20}^{\circ }$), width/height shift (0.2), shear (0.2), horizontal/vertical flip	Pre-trained VGG-19	ACC 98.90%	Hold-Out	-	Hassan et al., 2022 [61]
Tile cropping	-	Modified YOLO-v4 (YOLO-v4-tiny)	REC 95.30%; Av PR 87.10%	Hold-Out	On [65]: Av PR 83.40%; REC 94.70%). On [66]: Av PR 73.10%; REC 96.30%	Shewajo et al., 2023 [62]
-	-	Pre-trained ResNet-50	ACC 98.49%	Hold-Out	-	Hemachandran et al., 2023 [63]
-	GAN	Pre-trained VGG-19	ACC 98.01%	Hold-Out	-	Dath et al., 2023 [64]

to complete the discussion.

5.1.2. Custom CNN

Besides pre-trained networks, the most widely explored solutions for the task of malaria parasite detection are custom CNNs. Many authors such as Sarkar et al. [67], Masud et al. [68], Umer et al. [69], Alqudah et al. [70], Malhotra et al. [71], Delgado-Ortet et al. [72], Irmak [73], Maqsood et al. [74], Magotra et al. [75], Uzun Ozsahin et al. [76], Hcini et al. [77], Cho et al. [78], Yebasse et al. [79], and Shambhu et al. [80] have proposed custom CNNs involving the types of layers commonly used in CV and optimized architectures and hyperparameters to achieve the best performance. Some authors have conferred a name to the developed CNN networks, for example, Optimized Step-Increase CNN [81], DilationNet [82], Falcon [83], DACNN [84], Mosquito-Net [85], and MozzieNet [86]. In [87], different CNN architectures were generated using an open-source DL framework, Autokeras, which identifies the optimal neural network to perform the classification task; the optimal choice is based on performance analysis. Custom CNNs with unconventional blocks are also proposed for malaria parasite classification. For example, Khan et al. [88] proposed an STM-SB-RENet block-based CNN that consists of three split–transform–merge (STM)-based convolutional blocks with the same topology, in each of which modified channel squeezing-boosting (SB) and TL are integrated; finally, fully connected and dropout layers are added. Islam et al. [89] proposed a multiheaded attention-based transformer model with a compact convolutional transformer and visualization of heat maps highlighting the most significant regions for the network. Alonso-Ramírez et al. [90] used a cascade of two networks: CNN-LSTM, which integrates the LSTM algorithm into CNN, and CNN-BiLSTM, similar to the former but with Bidirectional LSTM (BiLSTM). Amin et al. [91] developed a CAD system by combining a semi-supervised GAN and TL, called TL-SGAN, in which the model discriminator is replaced by a modified pre-trained VGG-16, while the generator is a custom CNN.

Details of the operations performed on the data, the custom network developed, and how the results were evaluated are given in

Table 6. Overview of cited works using custom CNN approaches to classify NIH datasets (results have been rounded). The symbol “-” indicates that no information is provided on a particular operation. The abbreviation “Av” indicates the average results over several repetitions of the experiment.

Pre-Processing	Data Augmentation	CNN Details	Results	Cross-Validation	External Validation	Author and Year
Rescaling; median filtering	-	3 convolutional layers	Binary: ACC 98.93%. Multi-class: ACC 99.06%	10-fold	-	Sarkar et al., 2020 [67]
Rescaling 64 × 64 × 3; grayscale conversion; flattening	-	4 convolutional blocks	ACC 92.00%	Hold-Out	-	Masud et al., 2020 [68]
Rescaling $120\times 120\times 3$; YUV conversion; intensity equalization; BGR conversion; stain normalization	-	22 layers	ACC 99.96 ± 0.001%	5-fold	-	Umer et al., 2020 [69]
Rescaling $64\times 64\times 3$	-	18 layers	ACC 98.85%	Hold-Out	-	Alqudah et al., 2020 [70]
Rescaling $64\times 64\times 3$	-	5 convolutional layers	ACC 94.56%	4-fold	-	Malhotra et al., 2020 [71]
Rescaling $181\times 181\times 3$	-	13 layers	ACC 95.00%	Hold-Out	Thin and thick blood films from Core Laboratory at the Hospital Clínic of Barcelona: ACC 75.39%	Delgado-Ortet et al., 2020 [72]
Rescaling $44\times 44\times 3$	-	20 layers	ACC. 95.28%; AUC 98.86%	5-fold	-	Irmak, 2021 [73]
Rescaling $125\times 125\times 3$; bilateral filtering	Zooming (0.1), rotation (${25}^{\circ }$), shearing (0.05), horizontal flipping, width/height shifting (0.1)	5 convolutional layers	ACC 96.82%	5-fold	-	Maqsood et al., 2021 [74]
Rescaling $134\times 134\times 3$	Rotation, shearing and zooming, width/height shifting, and horizontal flipping	6 convolutional layers	ACC 96.23%	Hold-Out	-	Magotra et al., 2022 [75]
Rescaling $64\times 64\times 3$	-	4 convolutional layers	Thin blood smears: ACC 96.06%. Thick blood smears: ACC 96.97%	Hold-Out	-	Uzun Ozsahin et al., 2022 [76]
Rescaling $64\times 64\times 3$	-	-	ACC 99.70%	Hold-Out	-	Hcini et al., 2022 [77]
Rescaling; Gaussian blurring	-	2 convolutional layers	ACC 97.81%	4-fold	-	Cho et al., 2023 [78]
Adaptive thresholding; increased intensity of the red channel; reassembly of the 3 RGB channels	-	1 convolutional layer	ACC 97.21 ± 0.3%	5-fold	-	Yebasse et al., 2023 [79]
Rescaling $64\times 64\times 3$	-	2 convolutional layers	ACC 96.02%	Hold-Out	-	Shambhu et al., 2023 [80]
Rescaling $100\times 100\times 3$	Rotations (${90}^{\circ }$, ${180}^{\circ }$, and ${270}^{\circ }$), and reflections along X-axis and Y-axis	4 convolutional layers	ACC 98.3%	5-fold	-	Kashtriya et al., 2019 [81]
Rescaling $32\times 32\times 3$, $64\times 64\times 3$, $128\times 128\times 3$, $256\times 256\times 3$; normalization	-	4 parallel CNNs with 2 convolutional layers each	ACC 99.50%; AUC 99.90%	Hold-Out	-	Mahmud et al., 2020 [82]
Rescaling $128\times 128\times 3$; normalization [0, 1]	-	9 layers	ACC 92.94%	Hold-Out	-	Banerjee et al., 2022 [83]
-	Rotation ($\pm {20}^{\circ }$, to the right or left), translation ($\pm 3$ pixels), horizontal/vertical random scaling (between $\pm 1$)	-	ACC 94.79%	Hold-Out	-	Oyewola et al., 2022 [84]
Rescaling $120\times 120\times 3$; normalization with mean of 0.5 and standard deviation of 0.5	Random rotation, random horizontal/vertical flip, and color jitter of 0.05%	3 convolutional layers	ACC 96.6%; AUC 99.01%	Hold-Out	-	Kumar et al., 2022 [85]
Rescaling $50\times 50\times 3$	Horizontal flipping, rotation (${20}^{\circ }$), width/height shifting (0.1), zooming (0.1), shearing (0.1), and filling mode “nearest”	4 blocks	ACC 96.73%; AUC 99.35%	Hold-Out	-	Asif et al., 2024 [86]
Rescaling $32\times 32\times 3$	-	2 convolutional layers	ACC 95.60%	Hold-Out	-	Alaiad et al., 2023 [87]
Rescaling $124\times 124\times 3$; discrete wavelet transform (DWT); merging the inverse DWTs at low–low and high–high resolution, moving to a size $82\times 82\times 1$	Rotation (range $\pm 5^{\circ }$), shearing (±0.05), translation, reflection (±1)	3 convolutional blocks	ACC 97.98%; AUC 99.6%	Hold-Out	-	Khan et al., 2022 [88]
Rescaling $96\times 96\times 3$	-	-	Original dataset: ACC 96.41%; AUC 99.11%. Modified dataset: ACC 99.25%; AUC 99.99%	Hold-Out	-	Islam et al., 2022 [89]
Rescaling $96\times 96\times 3$	-	3 convolutional layers	CNN-LSTM: ACC 99.86%. CNN-BiLSTM: ACC 99.89%	Hold-Out	-	Alonso-Ramírez et al., 2022 [90]
Rescaling $128\times 128\times 3$; normalization [0, 1]	-	VGG-16 + CNN	ACC 96.6%; AUC 98.00%	5-fold	-	Amin et al., 2023 [91]

.

5.1.3. Autoencoders

AE models, generally used as image generators or dimensionality reducers, have also been employed as blood cell classifiers. In particular, Fuhad et al. [92] trained a complete AE model, in which they then replaced the decoder with a flattened layer followed by two fully connected layers, the last of which is dedicated to binary image classification. Huq et al. [93] proposed an image classification technique based on outlier detection, which exploits an AE model called AnoMalNet; this model is trained entirely on non-infected images, and in the inference phase images are labeled as parasitic if their reconstruction loss value is greater than the mean plus three times the standard deviation of the loss obtained on the training dataset.

Table 7. Overview of cited works using AE to classify NIH datasets (results have been rounded). The symbol “-” indicates that no information is provided on a particular operation. The abbreviation “Av” indicates the average results over several repetitions of the experiment.

Pre-Processing	Data Augmentation	Model	Results	Cross-Validation	External Validation	Author and Year
Rescaling $32\times 32\times 3$; normalization [0, 1]	Random rotation (${20}^{\circ }$) and zooming (0.05), width/height shift ($\pm 0.05$), shearing (0.05), and horizontal flipping	Custom CNN based on AE	ACC 99.23%	Hold-Out	On [94]: ACC 98.79%	Fuhad et al., 2020 [92]
Rescaling $32\times 32\times 3$; grayscale conversion	-	Custom AE	ACC 98.49%	Hold-Out	-	Huq et al., 2023 [93]

provides information on the works just mentioned.

5.1.4. Hybrid Models

Hybrid models combining ML and DL have also been proposed for the classification of malaria on the NIH dataset, exploiting CNN networks as feature extractors and classical ML models as classifiers. Diker [95] optimized the parameters of a deep residual network with the Bayesian method, performed neighborhood component analysis to select automatic features and increase the classifiers’ performance, and finally used SVM and KNN to perform the final classification. Raihan et al. [96] used a custom CNN to extract features from images transformed using the 2d wavelet package, select the effective features by using the Whale Optimization Algorithm (WOA), and finally classify the remaining features with XGBoost. Li et al. [97] presented a new hybrid model dubbed RAL-CNN-SVM, which incorporates residual attention learning into a ResNet-50-backbone CNN, and performs the classification with an SVM with a radial basis function (RBF) kernel. Jones et al. [98] extracted and selected features using a custom CNN method and an improved Grey–Wolf optimization method, respectively. An SVM with RBF kernel was used as a classifier. Goni et al. [99] combined a custom CNN-based feature extractor with a double hidden layer extreme learning machine (ELM) classifier, called DELM. Abubakar et al. [100] tested the combination of six different feature extractor pre-trained networks and four ML models, from which VGG-16 + SVM was the best solution. Dutta et al. [101] introduced a new Barnacles Mating Optimizer (BMO) with Deep TL (DTL) Enabled Biomedical Malaria Parasite Detection and Classification (BMODTL-BMPC) model, which uses the ELM model for the identification and classification of malaria parasites. Alassaf et al. [102] developed an intelligent deep-TL-based malaria parasite detection and classification (IDTL-MPDC) model which performs feature vector extraction with Res2Net, whose hyperparameters were optimally adjusted with the differential evolution algorithm, and uses KNN for classification. Imran et al. [103] merged the extracted features with pre-trained DarkNet-53 and DenseNet-201 networks, performed vector dimensionality reduction using the WOA, and applied several ML classifiers among which Linear Regression (LR) was the best. Madhu et al. [104] developed an innovative approach utilizing an urgent, inception-based capsule network, a diagnostic model incorporating the pre-trained InceptionV3 network to extract features from images, and Imperative Capsule to detect the malaria parasite. Qadri et al. [105] proposed the use of the pre-trained Neural Search Architecture Network (NASNet) model to extract spatial features from images, the Random Forest (RF) algorithm to derive class prediction probability features from spatial ones, and a final SVM classifier. Amin et al. [106] performed feature extraction both manually and automatically using pre-trained ResNet-50 and ResNet-18, selected a portion of these features with the generalized normal distribution optimization method, and performed classification with SVM (linear, quadratic, and cubic) and ANN classifiers (narrow, wide, and medium). Finally, Murmu et al. [107] developed a hybrid Deep-CNN-RF model for malaria parasite detection, combining pre-trained networks with RF.

Table 8. Overview of cited works using hybrid models to classify NIH datasets (results have been rounded). The symbol “-” indicates that no information is provided on a particular operation. The abbreviation “Av” indicates the average results over several repetitions of the experiment.

Pre-Processing	Data Augmentation	Model	Results	Cross-Validation	External Validation	Author and Year
Rescaling $224\times 224\times 3$; color constancy method	-	Custom Deep Residual CNN + SVM	ACC 99.90%	Hold-Out	-	Diker, 2022 [95]
Rescaling $120\times 120\times 3$; grayscale conversion	-	Custom CNN + XGBoost	ACC 94.78%	Hold-Out	-	Raihan et al., 2022 [96]
Rescaling $224\times 224\times 3$; normalization [0, 1]; median filtering; data cleaning	-	Custom RAL-CNN + SVM (RAL-CNN-SVM)	ACC 99.70%; AUC 99.90%	Hold-Out	-	Li et al., 2022 [97]
Bilateral filtering	-	Custom CNN + SVM	ACC 94.00%	Hold-Out	-	Jones et al., 2023 [98]
Rescaling $32\times 32\times 3$; normalization [0, 1]	-	Custom CNN + DELM (CNN-DELM)	Original dataset: ACC 97.79%; AUC 99.52%. Modified dataset: ACC 99.66%; AUC 99.96%	Hold-Out	-	Goni et al., 2023 [99]
Rescaling $224\times 224\times 3$	-	Pre-trained VGG-16 + SVM	ACC 94.88%; AUC 98.50%	10-fold	-	Abubakar et al., 2021 [100]
Gaussian filtering	-	NasNetLarge + ELM (BMODTL-BMPC)	ACC 99.04%	Hold-Out	-	Dutta et al., 2022 [101]
Median filtering	-	Pre-trained Res2Net + KNN (IDTL-MPDC)	ACC 95.86%; AUC 98.63%	Hold-Out	-	Alassaf et al., 2022 [102]
L*a*b* conversion; intensity adjustment; histogram equalization and leveling	-	Pre-trained DarkNet-53 and DenseNet-201 + LR	ACC 99.67%	5-fold	-	Imran et al., 2022 [103]
Rescaling $128\times 128\times 3$	-	Pre-trained Inception-v3 + Capsule ANN	ACC 99.35%; AUC 99.73%	Hold-Out	-	Madhu et al., 2023 [104]
-	-	Pre-trained NASNet + RF + SVM	ACC 98.60 ± 0.18%	10-fold	-	Qadri et al., 2023 [105]
Bilateral filtering	-	Pre-trained ResNet-18 + Custom ANN	Wide ANN: ACC 99.48%; AUC 98.00%	10-fold	-	Amin et al., 2024 [106]
Rescaling $224\times 224\times 3$; normalization [0, 1]; thresholding	Rotation	Pre-trained VGG-16 and VGG-19 + RF	ACC 93.35%	Hold-Out	-	Murmu et al., 2024 [107]

summarizes the hybrid models used to detect and classify malaria parasites on the NIH dataset.

5.1.5. Ensemble Learning

The technique of ensemble learning was also explored for the task of malaria parasite detection and classification. In particular, Marques et al. [108] report an automatic malaria detection model based on the ensemble of 10 modified pre-trained EfficientNetB0 models. Zhu et al. [109] proposed a pre-trained ResNet-18 backbone network used as a feature extractor, and the random vector functional link (RVFL), Schmidt neural network (SNN), and ELM as classifiers. The final result is the ensemble output of the three networks obtained by majority voting. Rajaraman et al. [110] combined the predictions of the VGG-19 and SqueezeNet models to reduce the variance of the model, thus improving its robustness and generalization. A combination of pre-trained VGG-16, VGG-19, and DenseNet-201 was run two-by-two with the adaptive weighted average method in [111]. In addition, a maximum vote ensemble technique was applied in combination with adaptive weighted average ensemble models to reduce the dispersion of the predictions. Finally, Nayak et al. [112] proposed an ensemble AI-enabled IoMT automated diagnosis model to classify malaria parasitized from microscopic images. This model involves two parallel phases, the first of which is a CBRM Renealing Snapshot lightweight ensemble learning model created from a combination of three distinct CNN layers, while the second is an ensemble of the three transfer-learning models DenseNet-121, ResNet-101, and VGG-19 whose results are averaged. The final result of the two phases in parallel is a majority vote.

An overview of the ensemble learning models used for malaria parasite classification performed on the NIH dataset is reported in

Table 9. Overview of cited works using ensemble learning approaches to classify NIH datasets (results have been rounded). The symbol “-” indicates that no information is provided on a particular operation. The abbreviation “Av” indicates the average results over several repetitions of the experiment.

Pre-Processing	Data Augmentation	Model	Results	Cross-Validation	External Validation	Author and Year
Normalization [0, 1]	-	Ensemble of 10 pre-trained EfficientNetB0	ACC 96.23%	10-fold	-	Marques et al., 2022 [108]
Rescaling $224\times 224\times 3$	-	Pre-trained ResNet-18 + Ensemble RVFL/SNN/ELM	ACC 95.73 ± 2.63%	5-fold	-	Zhu et al., 2022 [109]
Rescaling $100\times 100\times 3$; mean normalization	Rotation, translation, cutting, zooming, and flipping	Ensemble pre-trained VGG-19/SqueezeNet	ACC 99.51 ± 0.1%; AUC 99.92 ± 0.1%	5-fold	-	Rajaraman et al., 2019 [110]
Rescaling $64\times 64\times 3$; normalization [0, 1]	Rotation (0.2), zooming (0.2), horizontal/vertical flipping	Ensemble of pre-trained VGG-16, VGG-19, and DenseNet-201	ACC 97.92%	Hold-Out	-	Bhuiyan et al., 2023 [111]
Rescaling $224\times 224\times 3$; normalization [0, 1]	Horizontal flipping, rotation (${15}^{\circ }$), zooming (10%), lighting (20%)	Ensemble CNN/pre-trained DenseNet-121, ResNet-101, VGG-19	Av ensemble: ACC 99.13%; AUC 99.60%	Hold-Out	-	Nayak et al., 2022 [112]

.

5.1.6. Classic ML Models

Although DL and hybrid approaches are in vogue both for CV tasks in general and for the malaria recognition and classification on the NIH dataset, classical ML approaches have also been proposed. In [113], a system is proposed that integrates Hossain et al. [113], a variational quantum circuit (VQC) approach to parasite detection, and a rule-based expert system to detect malaria types. The optimal ML-based automated malaria parasite detection and classification (OML-AMPDC) model is proposed in [114] for pre-processing, feature extraction using Local Derivative Radial Patterns, and classification of blood smear images with an RF classifier, whose optimal parameters are chosen using the particle swarm optimization algorithm. Phumkuea et al. [115] applied the Information Gain filter to select the most significant features from the pre-processed images and analyzed the histogram of the HSV images using a DT classifier. To improve the accuracy of malaria detection and classification, Telang et al. [116] computed the first four statistical intensity moments for pixels in each image RGB color channel, performed feature selection to reduce the dimensionality of the previously obtained feature vectors, and classified the data using the RF, KNN, Linear SVM + LR, DT, Kernel SVM, and Naïve Bayes (NB) approaches.

More details on the results just discussed are given in

Table 10. Overview of cited works using classic ML models to classify NIH datasets (results have been rounded). The symbol “-” indicates that no information is provided on a particular operation. The abbreviation “Av” indicates the average results over several repetitions of the experiment.

Pre-Processing	Data Augmentation	Model	Results	Cross-Validation	External Validation	Author and Year
Rescaling; median filtering	-	VQC	Binary: ACC 98.93%. Multiclass: ACC 99.06%	10-fold	-	Hossain et al., 2021 [113]
Noise removal and contrast enhancement using CLAHE technique	-	RF (OML-AMPDC)	ACC 90.32%	Hold-Out	-	Kundu et al., 2023 [114]
Rescaling $100\times 100\times 3$; 24-bit color code conversion; HSV conversion	-	DT	ACC 95.60%	10-fold	-	Phumkuea et al., 2023 [115]
-	-	RF, KNN, Linear SVM + LR, DT, Kernel SVM and NB	ACC between 95.00% and 99.00%	10-fold	-	Telang et al., 2023 [116]

.

5.1.7. Mobile Applications

The aim of some of the work was to develop mobile applications to support rapid, efficient, and cost-effective malaria diagnosis as a support tool in less developed areas. In this sense, Yu et al. [117] designed an Android mobile application called Malaria Screener that consists of a slide screening module with three sequential sub-modules for acquisition, parasite detection, and display of results; a data management module that stores images and corresponding metadata acquired during slide screening sessions; and finally, a data upload module that transfers local data to an online repository for record keeping and further training of the system. Yang et al. [118] developed an image processing method for parasite candidates search using an iterative global minimum screening based on intensity and their classification using a custom CNN.

Table 11. Overview of cited works developing mobile applications for malaria diagnosis on NIH datasets (results have been rounded). The symbol “-” indicates that no information is provided on a particular operation. The abbreviation “Av” indicates the average results over several repetitions of the experiment.

Pre-Processing	Data Augmentation	Model	Results	Cross-Validation	External Validation	Author and Year
-	-	Pre-trained CNN	Thick smears: patch level ACC 96.89%, AUC 98.48%; patient level ACC 78.00%, AUC 84.90%. Thin smears: patch level ACC 98.60%, AUC 99.90%; patient level ACC 95.90%, AUC 99.10%	5-fold	-	Yu et al., 2020 [117]
Rescaling $44\times 44\times 3$	-	Custom CNN	ACC 93.46 ± 0.32%); AUC 98.39 ± 0.18%	5-fold	Part of the original dataset: ACC 97.26%; AUC 97.34%	Yang et al., 2019 [118]

summarizes the results of mobile applications for malaria diagnosis on the NIH dataset.

5.2. Articles Using Other Public and Private Datasets

In this section, we review works proposing AI methods for the automatic diagnosis of malaria using private or public but non-NIH datasets. We summarize the results by distinguishing between works developing DL approaches, classical ML approaches, and those developing web or mobile applications. The information extracted from the various articles is reported in apposite tables that also highlight the specific task under consideration.

5.2.1. DL Approaches

This review considers recent research on the development of automatic image-based malaria diagnosis applications. As this is a CV task, DL approaches are the most commonly used. Hung et al. [119] proposed the identification of RBCs and the recognition of their stages in bright-field microscopic images of malaria-infected blood by using a pre-trained Faster Region-based Convolutional Neural Network (Faster R-CNN) followed by an AlexNet. A focus-stacking approach to the automated quantitative detection of the malaria parasite Plasmodium falciparum was proposed by Gopakumar et al. [120]. Once the best-focused images are chosen for each field of view (FoV), an SVM trained on the statistical and structural features extracted from the positions of suspected parasites (A) is compared with a CNN trained on patches surrounding the suspected positions (B), and with a CNN trained on the patch stack surrounding the suspected positions (C). Jagtap D et al. proposed a Cuckoo Search-Based Ensemble Classifier (CSEC) [121] to overcome the limitations of the monoclassifier strategy in the detection of malaria-infected erythrocytes. For the identification of the malaria parasites in thick blood smears, Pattanaik et al. proposed a CAD system that includes an ANN with two stacked models for malaria parasite detection, followed by a softmax classifier for binary classification [122]. The same authors also conducted different experiments. For example, in [123], they present a deep filter bridge combining multi-rolling stacked denoising AE, fisher vector, and ELM to automatically classify the different types of single cells in microscopic blood smear images as either infected or uninfected. In addition, for the automatic classification of microscopic blood smear images at multiple magnifications, they [124] proposed an effective multi-enhanced deep residual neural network (MM-ResNet) consisting of ResNets with different inputs, the results of which are concatenated and then classified using a dense layer. For a rapid automated diagnosis of malaria, Manescu et al. [125] applied the Extended Depth-of-Field (EDoF) in thick film microscopy and developed an EDoF-CNN that can rapidly compute EDoF images from z-stacks while improving the spatial resolution of the resulting image. The same authors also proposed the Deep Malaria Convolutional Neural Network (DeepMCNN) classifier [126], which performs object detection with a RetinaNet, and binary classification of stacks at the patient level using a pre-trained VGG-19. Total malaria parasite (MP) and WBC counts are also performed, allowing for the estimation of parasitemia in MP/$\mathsf{\mu }$L, as recommended by the WHO. A low-cost, portable digital microscope scanner capable of both bright-field and fluorescence imaging is being developed by Holmström et al. [127]. Two supervised DL algorithms called DLS1 and DLS2 are trained to detect P. falciparum parasites. The first network detects all RBCs, and in a second step, the two networks in parallel tell whether the RBCs are healthy or diseased. Vijayalakshmi [128] proposed a pre-trained VGG replacing the final layers with an SVM classifier with RBF kernel. Pardede et al. [129] implemented the RetinaNet network with ResNet-101 as a backend to detect normal and infected erythrocytes on a thin blood smear image. The label counting of the detected objects and calculation of the malaria index values were assessed to perform a diagnosis of malaria. Nakasi et al. [130] tested three networks, identifying Fastest R-CNN as the best network for classification, and SSD as a possible best solution for mobile implementation. The main objective of the project by Chowdhury et al. [131] was to create a low-cost CNN-based automated system to identify and count blood cells (RBCs, platelets or thrombocytes, and different types of WBCs), and also detect malaria infection from a digitized blood smear image. The proposed Modified Yolo network is a modified version of the YOLO network made smaller and optimized without compromising performance too much but providing faster automatic detection in hardware implementation. Lebel et al. [132] demonstrated that DL allows for both the life-stage classification of malaria parasites and accurate quantification of parasitemia in unlabeled images obtained by bright-field microscopy of live, unstained RBCs. To do this, a standard optical microscope with visible and near-UV illumination and a microscope with deep UV illumination were used, exhibiting superior performance compared to Giemsa’s manually scored smears performed by experts. The workflow involved semantic cell segmentation with ResNet-50, post-processing of the resulting masks, and parasite stage classification using GoogLeNet. To overcome the strong imbalance between classes, they used a weighted cross-entropy loss function that introduces a normalization term for the weights based on the fractional representation of each class. To automatically detect and quantify blood cells at different life stages of the malaria parasite, Loh et al. [133] used a pre-trained Mask R-CNN that returns binary masks and bounding coordinates. In addition, they performed segmentation and cell counting, and then calculated the ratio between the infected and healthy. Li et al. [134] developed a malaria stage recognition approach in blood smear images using a deep transfer graph convolutional network (DTGCN) consisting of a feature extractor based on the pre-trained ResNet-50, a source transfer graph construction component, and an unsupervised GCN. Molina et al. [135] proposed the use of the pre-trained VGG-16 model for the differentiation of RBCs, both normal erythrocytes and erythrocytes, with other types of inclusions that have a parasite-like morphology. Davidson et al. [136] performed RBC detection by pre-trained Faster R-CNN with ResNeto-50 as backbone, cell differentiation into infected/uninfected by pre-trained ResNet-50, and parasite life stage categorization by pre-trained ResNet-34. For the automated diagnosis of malaria, Ufuktepe et al. [137] proposed the Channel-wise Feature Pyramid Network for Medicine (CFPNet-M)—Detection, Extraction, and Counting (CFPNet-M-DEC), in which the cell detection module includes a modified CFPNet-M network that takes two input images and returns the truncated distance transformation for each, whereas the cell extraction and counting module is based on a modified CFPNet with two-channel input, one for each output of the previous module. To perform the P. falciparum automatic screening on thick blood smears, Abdurahman et al. [138] modified the pre-trained YOLOv3 (YOLOv3-MOD1 and YOLOv3-MOD2) and YOLOv4 (YOLOv4-MOD) to improve their ability to detect small objects by extending the feature scales and adding more detection levels. The YOLOv4-MOD turned out to be the best. Yang et al. [139] proposed a system for segmentation and classification of malaria cells. The parasite segmentation is performed using a modified U-Net (U-Net∖VGG-19) with a VGG-19 encoder and a customized loss function based on the Jaccard index. Next, the largest parasite detected in each image is classified by a Light-Net. Jabbar et al. [140] compared three different CNN networks, with/without DA and with/without noise reduction pre-processing, the best of which has five layers and uses connected convolution architecture. For a quick classification of RBCs, Preißinger et al. [141] proposed a system that, after choosing the microscopy technique for acquisition and performing some pre-processing operations, selects the RBC cross-sections that best captures the features associated with the presence/absence of the parasites and their life stage and classifies them using a custom CNN. The localization, detection, and counting of parasites and WBCs were performed by Nakasi et al. [66] using the two pre-trained models Faster R-CNN with ResNet-101 backbone and Single Shot Multibox Detector (SSD) MobileNetV2. While the former model had a better performance, the latter was more suitable for the mobile application. For the life cycle classification of P. vivax, Sengar et al. [142] designed a method that combines four modules, namely, image acquisition, data augmentation, parasitic data generation with GAN, and classification, with the vision transformer (ViT) model. The generation of additional training samples increased the robustness of the resulting model. Arshad et al. [143] proposed a method to locate Plasmodium parasites by segmentation with U-Net, on which the binary masks produced are run by the watershed method, and to classify them in two stages: binary classification and subsequent multiclass classification with ResNet50V2. For the pathogen detection on microscope images of thick blood smears, Koirala et al. [144] developed a customized YOLO-mp architecture by modifying the pre-trained YOLO architecture with three and four levels (YOLO-mp-3l and YOLO-mp-4l). Acherar et al. [145] constructed an image dataset of P. falciparum-infected and non-infected blood components, thus evaluating different networks for their classification. After segmenting the cells using the watershed algorithm, a custom CNN and the pre-trained VGG-19, ResNet-50, and EfficientNetB7 networks were compared, with the latter being the best. Meng et al. [146] studied a multi-stage malaria parasite recognition problem, proposing a novel Neighbor Correlated Graph Convolutional Network (NCGCN) that extracts image representations using a ResNet-50, uses them to formulate the correlated relationships between the samples by combining the KNN and the EPSILON-radius graph construction methods, and finally performs the classification with a two-layer Graph Convolutional Network (GCN). To determine infected cells in thin whole blood smear images, Sukumarran et al. [147] implemented and compared the three object detectors YoloV4, modified Faster R-CNN, and modified SSD-300, of which the first was the best for end-to-end identification and localization of malaria-infected cells. Katharina et al. [148] proposed the Malaria Stage Classifier, which performs the identification of individual RBCs, the one-dimensional cross-section feature extraction, and the classification of malaria blood stages with a custom CNN. Liu et al. [149] developed an AI-based object detection system for malaria diagnosis, called AIDMAN, that combines YOLOV5 for cell recognition and a CNN with an attentional alignment model for cell classification. This model was optimized by changing the number of heads, achieving the best performance with a value of 2. For the automatic identification of Plasmodium species and their developmental stages in thin blood smears, Wang et al. [150] used the YOLOV7 network with the Cross Stage Partial Network as a backbone, and the PAN algorithm for multiscale detection and feature fusion. Fasihfar et al. [151] proposed a cell segmentation approach and subsequent classification using custom six-layer CNNs. Fu et al. [152] used the ResNetSt-50 attention mechanism, applying the BYOL self-supervised learning to pre-train the network with positive samples and alleviate the problem of insufficient labeled data. To detect malaria parasites with a high efficiency and accuracy, Zedda et al. [153] proposed a modified version of the pre-trained YOLOV8m architecture, called YOLO-PAM. With a trial-and-error approach, Saxena et al. [154] developed five DL models, of which the best was a custom CNN with four convolutional layers without dropout layers and categorical output. Guemas et al. [155] proposed a real-time detection transformer (RT-DETR) object detection algorithm for discriminating plasmodium species. The proposed architecture consists of a backbone, an efficient hybrid encoder, and a transformer decoder with auxiliary prediction heads. Ilyas et al. [156] simultaneously addressed domain variation, life cycle, and magnification with a new encoder–decoder architecture with a multiclass focus for robust malaria diagnosis. In particular, they used the Fourier Adaptive Recognition System (FARS) with the integration of adversarial training and color Domain Aware Fourier Domain Adaptation for normalizing staining in histopathology, and the Segment Anything Model as a binary segmentation model. This integrated approach offers a robust and reliable solution for cross-domain, multi-stage, and multi-magnification malaria parasite recognition.

An overview of the articles just summarized is given in

Table 12. Overview of cited works using DL approaches to malaria parasite diagnosis (results have been rounded). The symbol “-” indicates that no information is provided on a particular operation. The abbreviation “Av” indicates the average results over several repetitions of the experiment.

Dataset	Task	Model	Results	Cross-Validation	External Validation	Author and Year
1300 Giemsa-stained images infected with P. vivax, containing approximately 100,000 cells	Binary classification	Faster R-CNN + AlexNet	ACC 98%	Hold-Out	-	Hung et al., 2017 [119]
Images of slides colored by Leishman, from which 1400 positive patches and 326,934 negative patches are cut out	Binary classification and cell counting	SVM and custom CNN	Binary classification: (A) SE 96.38 ± 0.88%; SP 95.43 ± 0.85%; MCC 91.81 ± 1.50%; AUC 99.10 ± 0.37%. (B) SE 98.91 ± 0.36%; SP 99.39 ± 0.31%; MCC 98.31 ± 0.39%; AUC 99.87 ± 9.9871 $\times {10}^{-2}$%. (C) SE 99.14 ± 0.37%; SP 99.62 ± 0.18%; MCC 98.77 ± 0.32%; AUC 99.92 ± 7.5764 $\times {10}^{-2}$%. Cell counting: (A) SE 92.87%; SP 93.84%; MCC 44.35%. (B) SE 96.56%; SP 98.27%; MCC 70.33%. (C) SE 96.98%; SP 98.50%; MCC 73.02%	10-fold	-	Gopakumar et al., 2018 [120]
2867 samples of which 1244 normal and 1623 infected	Binary classification	Ensemble learning (CSEC)	PR 95.64%; SE 92.80%; SP 94.96%; ACC 93.78%;	3-fold	-	Jagtap D et al., 2019 [121]
1182 microscopic images of thick blood smears collected by the Makerere University AI research group	Binary classification	ANN with stacked models	ACC 89.10%; SE 93.90%; SP 83.10%; F1 94.50%	10-fold	-	Pattanaik et al., 2020 [122]
DPDx dataset: 39,000 cells extracted from 108 patch-cut images [157]	Binary and multiclass classification	Deep filter bridge	Binary: Av F1 98.36%; Av ACC 98.12%. Multiclass: ACC P. falciparum 90.00%, P. ovale 98.80%, P. vivax 87.47%, P. malariae 98.60%, and P. knowlesi 90.00%	10-fold	-	Pattanaik et al., 2021 [123]
1182 images with 3 different magnifications and frames of 7245 parasites	Binary classification	MM-ResNet	Av ACC 98.08%	5-fold	-	Pattanaik et al., 2022 [124]
Z-stacks of thick blood smears stained with Giemsa	Malaria parasite detection	EDoF-CNN	REC 74.00%; PR 79.00%	Hold-Out	-	Manescu et al., 2019 [125]
Z-stacks of Giemsa-stained thick and thin blood smears from 239 patients. The 4168 extracted cells are annotated by pathologists	Binary classification	RetinaNet + VGG-19 (DeepMCNN)	SE 92.00%; SP 90.00%; PR 91.00%; PPV 92.00%; NPV 90.00%	Hold-Out	-	Manescu et al., 2020 [126]
125 thin blood smears stained with DAPI collected from 100 patients	Malaria detection	DLS 1 e DLS 2	r = 0.99, p < 0.01	Hold-Out	-	Holmström et al., 2020 [127]
50 microscopic images obtained from Giemsa-stained blood smears + MaMic Database	Binary classification	VGG19 - SVM	ACC 93.13%; SE 93.44%; SP 92.92%; PR 89.95%; F1 91.66%	5-fold	-	Vijayalakshmi, 2020 [128]
27 images from [119] and 79 images from Dr. Yani Triyani repository.	Object detection	RetinaNet (ResNet-101 backbone)	Object detection: Av PR 94.00%; Av REC 74.00%; Av ACC 73.00%. Diagnosis: SP 98.00%; SE 99.00%	Hold-Out	-	Pardede et al., 2020 [129]
643 images of thick blood smears with parasite delineation boxes annotated by experienced laboratory technicians	Parasite detection	Faster R-CNN (ResNet-101 backbone)	Av PR 94.48%; PR 77.91%; REC 89.81%	Hold-Out	-	Nakasi et al., 2020 [130]
(1) Leukocyte Images for Segmentation and Classification (LISC) database [158] (five types of WBCs on a background of RBCs); (2) Isfahan University of Medical Science (IUMC) database [159] (individual WBCs with binary masks); (3) MAMIC Database; (4) segmented NIH	Multiclass classification + object detection	ModifiedYolo network	Cell classification: Av PR 95.56%. Malaria detection: SE 100.00%; SP 97.47%	Hold-Out	Combination of LISC and MAMIC images not contained in the training set: Av PR 93.66%	Chowdhury et al., 2020 [131]
64,966 cells from the first system and 14,219 from the second. The images are annotated by hand in 4 classes	Multiclass classification	ResNet50 + GoogLeNet	First system: ACC 96.80%. Second system: ACC 98.80%	Hold-Out	-	Lebel, et al., 2021 [132]
297 images acquired from Giemsa-stained samples and annotated by hand	Multiclass classification	Pre-trained Mask R-CNN	Detection: ACC 94.57%; Av PR 73.10%; PR 82.00%. Cell counting: ACC 89.56%	Hold-Out	-	Loh et al., 2021 [133]
BBBC041	Multi-stage classification + binary adaptation	DTGCN	Staging: ACC 98.30 ± 0.03%; PR 98.50 ± 0.02%; REC 98.30 ± 0.02%; F1 98.30 ± 0.03%	-	Binary classification on NIH: ACC 95.4 ± 0.05%; PR 95.4 ± 0.07%; REC 95.4 ± 0.06%; F1 95.4 ± 0.07%	Li et al., 2021 [134]
6415 images of RBCs segmented from 53 thin peripheral blood smears stained with May Grünwald-Giemsa.	Multiclass classification at the cellular level, and binary at the patient level	Pre-trained VGG-16	ACC 99.75%; PR 97.00%; SE 100.00%	Hold-Out	23 independent blood smears: ACC 99.50%; PR 95.70%; F1 97.40%; SE 99.20%, and SP 99.90% on RBCs. SE 100.00% and SP 91.70% at patient level	Molina et al., 2021 [135]
38,000 RBCs from Giemsa-stained blood smears with P. falciparum and P. vivax	Parasite detection + binary classification	Faster R-CNN + ResNet-50 and ResNet-34	Object detection: Av PR 98.00% on P. vivax; Av PR 99.00% on P. falciparum. Binary classification: ACC 99.80%; AUC 97.90%	Hold-Out	-	Davidson et al., 2021 [136]
955 microscopy images from 193 patients (148 P. falciparum and 45 non-infected)	Cell counting	CFPNet-M-DEC	Image level: ACC 92.28%. Patient level: ACC: 92.20%	Hold-Out	-	Ufuktepe et al., 2021 [137]
1182 images of which 948 were malaria-infected (with 7628 P. falciparum parasites) and 234 normal with artifacts due to impurities [65]	Object detection	YOLOv3-MOD1, YOLOv3-MOD2, and YOLOv4-MOD	YOLOv4-MOD: Av PR 96.32%; PR 95.00%; REC 94.00%; F1 94.00%. YOLOV3-MOD2: Av PR 96.14%. YOLOV3-MOD1: Av PR 95.46%	Hold-Out	-	Abdurahman et al., 2021 [138]
Giemsa-stained images from MP-IDB dataset (1) and [160] (2).	Multiclass classification	U-Net_VGG-19 + Custom CNN (Light-Net)	Cell classification: Av PR 88.00% (1); ACC 99.50% (2). Patient classification: ACC 100.00% (1) (2).	Hold-Out	Detection rate (dt) 99.00% on [161]. Species classification (SpC) ACC 63.00% on [55]. dt 99.00% and SpC ACC 84.00% on [162]. dt 85.00% and SpC ACC 31.00% on [163]	Yang et al., 2021 [139]
2565 RBC images belonging to two classes	Binary classification	Custom CNN	ACC 99.22%; REC 98.37%; PR 100.00%; F1 99.17%	Hold-Out	-	Jabbar et al., 2022 [140]
792 Giemsa-stained microscopy images of 17 malaria-infected patients, totaling 26,223 cells [160] manually labeled by three experts	Multiclass classification	Custom CNN	ACC 97.90%	Hold-Out	Same training data acquired by fluorescence microscopy (FM) (1232 images, 45,726 RBC): ACC 96.30%; and acquired by atomic force microscopy (AFM) (173 images, 7386 RBCs): ACC 98.90%	Preißinger et al., 2022 [141]
903 field stain images of blood smears often manually annotated by experienced laboratory technicians	Multiclass detection	Faster R-CNN and SSD MobileNetV2	Faster R-CNN: Av PR 66.09%. SSD MobileNetV2: Av PR 62.69%	Hold-Out	-	Nakasi et al., 2021 [66]
345 images from thin Giemsa-stained blood smears, each containing approximately 111 cells. The labels refer to the stage and location of the parasite	Parasite life stage classification	ViT	ACC: 90.03%	Hold-Out	-	Sengar et al., 2022 [142]
Dataset–IML-Malaria: 345 images of Giemsa-stained blood samples containing P. vivax in four phases. A total of 38,000 cells	Binary + multiclass classification	ResNet50V2	Av ACC 79.61%; F1 82.04%	Hold-Out	NIH: Av ACC 96.26%; F1 96.18%. BBBC041: Av ACC 99.62%; F1 93.07%	Arshad et al., 2022 [143]
1182 images of thick blood smears treated with field stain (948 P. falciparum infected and 234 negative) [164]	Binary classification	YOLO-mp	YOLO-mp-3l: Av PR 93.99% (IoU = 0.5). YOLO-mp-4l: Av PR 94.07% (IoU = 0.5)	5-fold	2703 images from 133 thick smears treated with field stain. YOLO-mp-3l: Av PR 79.32%. YOLO-mp-4l: Av PR 80.92%.	Koirala et al., 2022 [144]
Images of Giemsa-stained thin blood smears of 202 patients (half healthy, half malaria)	Binary classification	Pre-trained networks	Cells level ResNet50: ACC 99.40 ± 0.3%; PR 99.10 ± 0.4%; SE 98.50 ± 0.5%; SP 99.70 ± 0.1%; F1 98.80 ± 0.5%. Patient level VGG-19: SE 98.00%; SP 56.00%	5-fold	NIH. Cells level: ACC 98.50 ± 0.1%; PR 98.60 ± 1.2%; SE 98.40 ± 0.6%; SP 98.6 ± 1%; F1 98.60 ± 1.2%. Patient level: SE 100.00%; SP 0.02%	Acherar et al., 2022 [145]
1364 images of cells captured from Giemsa-stained blood smears and labeled in six classes	Multiclass classification	NCGCN	ACC 94.17%; PR 94.84%; REC 94.17%; F1 94.20%; AUC 99.00%	Hold-Out	Binary classification on 2149 infected and 5302 healthy cells. ACC 95.46%; REC 95.21%; PR 95.46%; F1 95.12%	Meng et al., 2022 [146]
MP-IDB Dataset + 236 Giemsa-stained thin blood films and 3038 cells (all parasite types in various stages of infection)	Object detection	YOLOV4	Av PR 93.87%; PR 83.00%; REC 95.00%; F1 89.00%	Hold-Out	-	Sukumarran et al., 2023 [147]
See [141]	Multiclass classification	Malaria Stage Classifier	ACC 98.70%	Hold-Out	AFM images: ACC 99.30%. FM images: ACC 98.00%	Katharina et al., 2023 [148]
1822 Giemsa-stained thin-blood-smear images (5654 patch) from 140 patients	Binary classification	AIDMAN	Patch level: ACC, PR, SE, SP, F1 98.62%; AUC 99.92%. Whole image level: ACC 97.00%; PR 97.13%; SE 97.00%; SP 94.18%; F1 96.96%; AUC 98.84%.	Hold-Out	-	Liu et al., 2023 [149]
12,708 Wright-Giemsa thin blood smear images of 380 patients. 12,546 malaria RBCs in various stages	Multiclass classification	YOLOV7	Av PR 90.20%; REC 96.00%; PR 94.90%; SE 96.80%; SP 99.30%; AUC > 99.90%	Hold-Out	-	Wang et al., 2023 [150]
624 images of P. falciparum, 548 images of P. vivax, 588 images of P. ovale and 160 images of suspected healthy individuals	Multiclass classification	Custom CNN	ACC 99.59%; SE 99.58%; SP 99.59%; PR 95.62%; REC 99.58%; F1 99.58%; MMC 98.68%;	5-fold	-	Fasihfar et al., 2023 [151]
Blood smear images	Binary classification	ResNetSt50 + BYOL	ACC 97.80%; SE 96.50%; SP 98.90%	Hold-Out	-	Fu et al., 2023 [152]
MP-IDB + Dataset–IML-Malaria [143]	Multiclass classification	YOLO-PAM	MP-IDB Av PR 83.60%. IML Av PR 59.90%	Hold-Out	-	Zedda et al., 2023 [153]
352 images of Leishman-Giemsa-stained peripheral blood smears containing normal and infected RBCs	Multiclass classification	Custom CNN	SE 85.00%; SP 94.00%	Hold-Out	-	Saxena et al., 2023 [154]
24,720 images from 475 thin blood smears stained by May Grunwald–Giemsa corresponding to 2,002,597 labels	Object detection + binary classification	RT-DETR	-	-	4508 images taken from 170 thin blood smears of 54 healthy and 116 infected patients. Patient level: 6-classes ACC 68.20%; 5-classes ACC 79.40%; binary ACC 91.80%. Label level: PR 68.60%; REC 66.90%; Av PR 63.80%; F1 67.70%; MCC 63.30%	Guemas et al., 2024 [155]
M5 Dataset [165]. 7543 large-scale multi-microscope images, for a total of 20,331 labeled nucleus	Object detection	FARS	Highest Av PR 67.13% (HCM at 1000× magnification)	Hold-Out	-	Ilyas et al., 2024 [156]

.

5.2.2. Classic ML Approaches

A minority of works have proposed classical ML approaches to automatic malaria diagnosis, staging, and parasite counting. For example, Molina et al. [166] developed an ML system that is able to discriminate parasitized RBCs not only from normal but also from other erythrocyte inclusions. After performing an initial pre-processing and segmentation phase, they extracted color and texture features from each component in different color spaces. The proposed system is composed of three sequential modules, of which the first recognizes normal RBCs against RBCs containing inclusions (including malaria), the second identifies whether the inclusion corresponds to malaria or not, and the third discriminates between Howell–Jolly bodies, Pappenheimer bodies, basophilic stippling, and platelets. The result of the overall system is the automatic classification of RBC images into one of six classes. SVM, KNN, Linear Discriminant Analysis (LDA), RF, and Gaussian NB models are tested at each stage, and the best models together with the most appropriate number of features are chosen on the basis of the ACC: SVM + the first 7 most relevant features for the first module; LDA + the first 610 most relevant features for the second module; LDA + the first 700 most relevant features for the third module. Abbas et al. [167] developed an automated approach to the detection, counting, and classification of erythrocytes infected with the malaria parasite at different life cycle stages. From the original images, cell segmentation is performed, followed by the detection of clumps and overlapping RBCs, which are then cleaved. Once the individual RBCs are obtained, the infected ones are detected by assessing the presence of “child” borders within their outer edges, and automatic counting is then easily implemented. The results of the automatic count are compared with those performed manually by the experts using Pearson’s correlation coefficient. For the classification of the malaria parasite’s life cycle (ring stage, trophozoite, schizont, gametocytes) on the basis of histograms of oriented gradients (HOGs) and LBP, the KNN, NB, and SVM multiclass models are used, of which SVM is the best. The automation of the process of malaria diagnosis through ML techniques was executed by Sadiq et al. [168], who followed a pipeline consisting of several steps: image pre-processing to improve their illumination and reduce noise; cell segmentation and categorization using the K-means clustering algorithm; extraction of texture and morphology features with subsequent selection of those with a high covariance between the two classes; and finally, classification using DT. Kudisthalert et al. [169] proposed an automated malaria parasite counting and classification system for Giemsa-stained thin-film images. The morphological image processing technique was used to clean up noise and reconstruct object structures, and the Hough transformation technique was used to segment RBC objects. Color features are concatenated with those extracted using a pre-trained AlexNet network and then classified with an improved version of ELM called Weighted Similarity ELM (WELM). WELM is implemented with Euclidean kernel and replaced at the last level of the Multi-Layer Perceptron (MLP). Poostchi et al. [170] developed an end-to-end automatic detection system to identify and quantify P. faciparum malaria parasites in thin Wright–Giemsa-stained blood smears from both humans and mice. The image processing pipeline comprises an initial phase of cell segmentation, followed by a phase of extraction, selection, and dimensionality reduction of color and texture features, and finally, a phase of cell classification using the linear SVM and ANN classifiers. Das et al. [171] proposed an automated system for the characterization and classification of P. vivax- and P. falciparum-infected stages using Leishman-stained microscopic images of thin blood smears. After segmenting the images with the watershed algorithm, the authors extracted texture and morphological features, selected only the most relevant ones, and finally classified them with different models among which MLP was the best. To recognize the parasite species between P. falciparum and P. vivax and assess their staging, Aris et al. [172] performed a standardized segmentation framework using thresholding and grouping; extracted the features of size, shape, texture, and color from the thick smear, including those of size, shape, and color from the thin smear; and finally designed a multistage RF classifier with 32 trees. The work of RK et al. [173] focused on the design of a new statistical image feature processing model that aims to negate the impact of brightness and illumination variation caused by the lack of standardization in microscopic settings, leading to increased performance in malaria classification. After segmenting the erythrocytes, texture features are extracted using GLCM, and a non-linear regression model is used to identify the optimum brightness factor (OBF) specific to a selected statistical feature. The feature correction factor (CF) is estimated based on the OBF, and classification is performed using SVM. The significant increase in performance with the use of CF is evidenced by the p-value analysis.

Table 13. Overview of cited works using ML approaches to malaria parasite diagnosis (results have been rounded). The symbol “-” indicates that no information is provided on a particular operation. The abbreviation “Av” indicates the average results over several repetitions of the experiment.

Dataset	Task	Model	Results	Cross-Validation	External Validation	Author and Year
15,660 RBC images from 87 smears stained with May Grünwald–Giemsa labeled by experts	Multiclass classification (6)	SVM and LDA	Image level: ACC 97.7%. Patient level: SE 100.00%; SP 90.00%	5-fold	-	Molina et al., 2020 [166]
DPDx dataset [157]	Cell counting + multiclass classification (4)	SVM	Cell counting: PR 96.18%; REC 98.03%; F1 97.04%. Object detection: SE 83.79%; SP 95.67%. Classification: SE 80.54%; SP 91.99%	Hold-Out		Abbas et al., 2019 [167]
MaMic dataset and bio-Sig data [174]	Binary classification	DT	ACC 92.60%; PR 96.64%; SE 93.12%; SP 91.21%	Hold-Out	-	Sadiq et al., 2017 [168]
23,248 RBC images from Giemsa-stained thin blood smears: 22,290 healthy WBCs, 437 ring shapes, 327 trophozoites and 194 schizonts	Binary classification + multiclass classification of infected RBC (ring, trophozoite, and schizont).	WELM	Cell counting: ACC 97.94%. Classification ACC: 95.99%	10-fold	SE 94.64% and SP 93.50% on NIH.	Kudisthalert et al., 2020 [169]
Human-NIAID dataset	Binary classification	SVM and ANN	ANN: ACC 88.00%; REC 91.00%; PR 99.00%; F1 90.00%. SVM: ACC 83.00%; REC 91.00%; PR 98.00%; F1 97.00%	Hold-Out	-	Poostchi et al., 2018 [170]
Leishman-stained thin blood smear images of 100 healthy subjects and 50 with malaria. 750 images obtained for P. falciparum- and P. vivax-infected blood smears labeled by a pathologist, with a total of 888 RBCs	Multiclass classification (6 classes: 5 infected and 1 healthy)	MLP	SP 98.64%; SE 100.00%; PR 96.84%	3-fold	-	Das et al., 2015 [171]
Private dataset: 50 Giemsa-stained blood samples. Public dataset: MP-IDB	Binary classification	SVM	Private dataset: ACC 86.11%; F1 90.90%. Public dataset: ACC 95.38%; F1 97.11%.	Hold-Out	-	RK et al., 2021 [173]
500 images from 4 thick blood smears and 4035 images from 34 thin blood smears, both under different illumination conditions	Binary + multiclass classification	RF	Parasites detection: ACC 86.89% on thick smears; Species recognition: ACC 98.82%. Parasite staging: ACC 90.78%	Hold-Out	-	Aris et al., 2023 [172]

provides an overview of the work using classic ML approaches to malaria automatic diagnosis.

5.2.3. Mobile and Web Applications

Several web and smartphone applications have been proposed for the rapid and effective diagnosis of malaria. The design and development of a web-based distributed health management system for the automatic diagnosis of malaria was proposed by Maity et al. [175]. The proposed methodology includes image pre-processing, erythrocyte segmentation, extraction, and selection of color, morphology and texture features, and automatic classification using NB, Instance-Based Learning (IB1), and C4.5 DT individually, of which the last one coupled with Correlation-based Feature Selection was the best in terms of performance. The primary entities of the proposed system are the central server, local servers (indirectly connected to each other through the central server), database (two separate modules, i.e., the local database that stores local health data for patients visited in the local hospital, and the central database that periodically retrieves the local database from all connected local centers and combines them all together for one centralized database development), and network connectivity. Chibuta et al. [176] developed an image processing pipeline to run in real-time on low-cost devices (mobile phones and desktop computers with basic specifications). By mounting a mobile phone or camera on an optical microscope and using the modified YOLOV3 detection algorithm, they proposed a malaria diagnosis system in low-resource settings. The network was trained using images resized to $224\times 224$, $544\times 544$, and $800\times 800$ pixels, achieving the best results with the largest size. Aiming to develop a new automated, low-cost, mobile-based diagnostic system to identify P. falciparum species at the ring development stage, Oliveira et al. [177] combined digital image processing techniques and a learning process based on AI algorithms. The pre-processing phase is aimed at identifying RBCs and limiting the algorithm’s search space, and then classification is performed by creating DTs called classifier cascades, with a number of stages of 10, 15, and 20. The result of the training was a degenerative DT with two classes that worked best using the 20-stage cascade. To use both local and remote resources, Chen et al. [178] designed a diagnostic pipeline that is implemented in a cloud-edge paradigm. The idea is that, after collecting the datasets sent by the hospitals, pre-processing with automatic color equalization, and classification using supervised U-Net are performed, and then the generated coarse probability maps and the normalized images are uploaded to the cloud server. Maps are used to weakly supervise the stacked dilated U-Net (SDU-Net), which performs segmentation by replacing the standard second convolution module in each encoding/decoding operation with multiple dilated convolutions, and concatenating all convolution outputs instead of a single second convolution output as input at the next level. Finally, the classification of parasite species and stage is performed with a pre-trained MobileNetV1, and the result is sent to the original local hospital immediately through the cloud. Blockchain technology is adopted during the data transmission process. To automate the malaria diagnostic process, Maturana et al. [179] developed a system in which a prototype of 3D-printed parts is designed for the robotization of conventional light microscopy, capable of automatically focusing the sample and tracing the entire slide, and the pre-trained YOLOV5x is used for the Plasmodium parasites’ detection. The whole system, which also performs the calculation of parasite density in thick blood smear samples, was integrated into the iMAGING smartphone application. More details about the mobile and web applications for malaria detection are summarized in

Table 14. Overview of cited works developing mobile and web applications for malaria parasite diagnosis (results have been rounded). The symbol “-” indicates that no information is provided on a particular operation. The abbreviation “Av” indicates the average results over several repetitions of the experiment.

Dataset	Task	Model	Results	Cross-Validation	External Validation	Author and Year
500 images of Leishman-stained blood smears from different FoVs. The ratio between P. falcuparum and P. vivax is 2:3	Multiclass classification (6 classes: 5 infected and 1 healthy)	C4.5 DT	ACC 99.24%, SE 99.20%, SP 99.60%, F1 99.00%, AUC 100.00%	Hold-Out	10 images acquired from two different microscopes with different settings. The developed model correctly classifies each image	Maity et al., 2017 [175]
Dataset A: coordinates of 49,900 P. falciparum parasites in 2703 images. Dataset B: coordinates of 7628 P. falciparum in 1182 images	Object (parasite) detection at cell and blood smear image levels	Modified pre-trained YOLOV3	Parasite level: Av PR 88.70% (A) and 90.20% (B). Whole image level: ACC 99.07% (A) and 97.46% (B).	Hold-Out	Parasite level: Av PR 80.10% for the network trained on dataset A and tested on dataset B; Av PR 83.20% for the network trained on dataset B and tested on dataset A	Chibuta et al., 2020 [176]
500 images of Giemsa-stained samples, annotated by expert parasitologists	Binary classification	DT	ACC 91.62 ± 2.26%; SP 96.11 ± 2.72%; PR 66.34 ± 10.68%; SE 54.91 ± 24.46%	10-fold	-	Oliveira et al., 2017 [177]
MP-IDB	Multiclass classification	U-Net + SDU-Net + MobileNetV1	Parasite type: ACC 98.47%; SP 96.80%; SE 97.99%. Parasite stage: ACC 98.83%; SP 97.91%; SE 98.54%.	Hold-Out	-	Chen et al., 2023 [178]
2571 images of Giemsa-stained thick blood smears infected with parasites.	Object detection and multiclass classification	iMAGING	YOLOv5x: PR 92.10%; REC 93.50%; F1 92.79%; Av PR 94.40%; mAP 50.00%	Hold-Out	116 images of negative samples: ACC 96.98%. 50 images of Plasmodium samples: ACC 94.00%	Maturana et al., 2023 [179]

.

5.3. Articles on Generalizing Malaria Diagnosis

This section is dedicated to summarizing all those studies that show the generalizability of the approaches proposed for the malaria dataset. First of all, we review studies that were not designed for a specific dataset or for the automatic diagnosis of a specific disease, but rather for an overall view, and that show their ability to generalize to different datasets including those of malaria. For example, Horstmeyer et al. [180] presented a framework to jointly optimize the physical parameters of a microscope and the CNN weights used to classify the images previously generated. Muthumbi et al. [181] proposed a method to improve the image classification performance of a standard microscope by adding a simple LED array and optimizing its illumination pattern within an improved deep CNN. Hung et al. [182] created a programming interface called Keras R-CNN that implements DL techniques for object detection, showing its potential for the identification and classification of a large number of cells. Batch and online variational learning was studied by Manouchehri et al. [183] by comparing four algorithms, the best of which was the online variational multivariate Beta mixture model (OVMBMM). In their work, Yao et al. [184] demonstrated a multi-lens microscopic imaging system that superimposes multiple independent FoVs on a single sensor for automated sample analysis with a high efficiency. Schwarz Schuler et al. [185] proposed an improved scheme to reduce the complexity of point convolutions in deep CNNs for image classification based on interleaved grouped filters without divisibility constraints. Gupta et al. [186] presented an innovative framework, called SimSearch, for quick and easy user-guided training of a DL model for rapid ROI detection in large-scale microscopy experiments. A new DL approach called IMNet, which combines multiple incremental modular CNNs, was proposed by Ali et al. [187]; this approach is computationally efficient for exploiting small data for learning generalizable, domain-invariant representations in various medical imaging applications. Musaev et al. [188] proposed a CNN ensemble called ICNN-Ensemble based on the representation of high-resolution image channels and a systematic model dropout ensemble that allows for the selection of possible members. In their research, El-Bana et al. [189] evaluated the efficiency of wavelet pooling (WP) in lightweight MobileNets architectures showing that the use of WP achieves average data savings of more than 30% compared to traditional pooling techniques.

Additional information on the methods just discussed can be found in

Table 15. Overview of cited works proposing general approaches tested on malaria diagnosis (results have been rounded). The abbreviation “Av” indicates the average results over several repetitions of the experiment.

Aim of the Study	Malaria Dataset	Method Used	Results	Author and Year
Optimize the image acquisition and classification methods	Hema-stained single-layer smears. 563 uninfected and 695 infected data	Discretized physical CNN (DP-CNN) network	Av ACC 94.90 ± 2.0%	Horstmeyer et al., 2017 [180]
Co-opting image illumination and automated classification	15 images of Hema, 3 stained thin smears, and 2126 samples from thick smears	Custom CNN with physical layers	Thin smears: Av ACC 88.20 ± 1.9%; PR 93.90%; REC 81.80%. Thick smears: Av ACC 99.00 ± 0.1%; PR 99.90%; REC 98.10%	Muthumbi et al., 2019 [181]
Identification and classification of a large number of cells	BBBC041	Keras R-CNN	Av PR 78.00%	Hung et al., 2020 [182]
Study of batch and online variational learning for multiple tasks	NIH	OVMBMM	ACC 92.50%; PR 90.47%; REC 95.11%; F1 92.68%	Manouchehri et al., 2021 [183]
Study of multi-lens microscopic imaging system	SimuData [118]: 1800 large FOV images from 150 individual patients	10-layers VGG	2 lenses: AUC 98.50%. 7 lenses: AUC 95.30%	Yao et al., 2022 [184]
Simplify the point convolutions in deep CNNs	NIH	Modified EfficientNet-B0	With 16 channels for groups: ACC 97.61%	Schwarz Schuler et al., 2022 [185]
Making training of a DL model quick and easy	BBBC041	Pre-trained ResNet-18	AUC > 70.00%	Gupta et al., 2022 [186]
Provide an efficient approach to medical imaging applications	NIH	IMNet (incremental modular network)	ACC 97.00 ± 0.36%; SP 97.90 ± 0.39%; SE 96.10 ± 0.63%; AUC 99.50 ± 0.1%	Ali et al., 2022 [187]
Improve model accuracy using EL	NIH	ICNN-Ensemble	ACC 99.67%; PR 96.60%; REC 90.00%; F1 96.60%	Musaev et al., 2023 [188]
Demonstrating the efficiency of WP in medical diagnostics	NIH	Lightweight MobileNet	ACC 97.18%; PR 97.43 ± 1.20; REC 97.44 ± 1.28; F1 97.4 ± 1.30	El-Bana et al., 2023 [189]

.

In addition to these results, some of the articles that emerged from the research conducted for the present review are focused on the diagnosis of diseases other than malaria, but they report tests of the proposed models on the detection of parasitized cells. Ferreira et al. [190] aimed to distinguish five different cancer types through the RNA-Seq datasets of the thyroid, skin, stomach, breast, and lung using a pre-trained AE bound to the upper layers of a CNN. To demonstrate the generalizability of the proposed approach, they performed tests for the classification of malaria and breast cancer. The effective and early detection of acute lymphoblastic leukaemia (ALL) using CAD was proposed by Abd El-Ghany et al. [191], who developed a classification model based on EfficientNet-B3. This model was also used to examine microscopic blood images to identify the malaria parasite.

See

Table 16. Overview of cited works tested on parasite detection (results have been rounded). The abbreviation “Av” indicates the average results over several repetitions of the experiment.

Aim of the Study	Generalization Task	Malaria Dataset	Method Used	Results	Author and Year
Distinguish five different types of cancer through RNA sequences	Malaria and breast cancer diagnosis	26,839 samples and 1058 features, collected by the Fraunhofer AICOS institution, through the MalariaScope project [192]	Pre-trained AE bound to the final layers of a custom CNN	ACC 92.36 ± 0.46%; MCC 84.00 ± 1%; PR 88.91 ± 2.14%; REC 91.11 ± 2.49%; F1 89.95 ± 0.61%	Ferreira et al., 2020 [190]
Automatic early detection of ALL	Malaria parasite identification	NIH	EfficientNet-B3	ACC 97.68%; Av PR 97.69%; REC 97.68%; SP 97.67%	Abd El-Ghany et al., 2023 [191]

for further details.

5.4. Articles on Multiple Diagnoses

The results of the research conducted for this review show that some articles are not only focused on the identification and classification of malaria, but also propose valid approaches to the automatic analysis of different diseases/conditions, including malaria.

Table 17. Overview of cited works focusing on multiple diagnoses (results have been rounded). The abbreviation “Av” indicates the average results over several repetitions of the experiment.

Aim of the Study	Task	Malaria Dataset	Method Used	Results	Author and Year
Development of point-of-care diagnostics using microscopes and smartphones	Automatic diagnosis of malaria, tuberculosis, and intestinal parasites	Thick blood smears stained with Giemsa. 7245 Plasmodium objects in 1182 annotated images	Custom CNN	AUC 100.00%; Av PR 97.00%	Quinn et al., 2016 [193]
Make microscopes able to indicate the presence of an epidemic pathogen in a sample	Identification of the malaria and vibrio cholerae pathogen	Images downloaded from the web	Custom CNN	ACC 94.64%	Traore et al., 2018 [194]
Show that adversarial learning can be efficiently used in the medical field	Human embryos evaluation, human sperm morphology quantification, and malaria diagnosis	8 patients’ thin blood smears stained with Giemsa captured using 3 different microscopes [195]	Unsupervised adversarial neural network using ResNet-50 trained on NIH dataset as backbone (MD-net-ResNet-50)	AUC 95.2% (IC 95% = 93.5–96.6); AUC 95.4% (IC 95% = 92.6–97.4); AUC 92.00% (IC 95% = 88.5–94.7)	Kanakasabapathy et al., 2021 [196]

summarizes the details of this work, including the objective of the study, the analysis conducted, the proposed approach, the dataset used for malaria classification, and the results obtained.

6. Discussion

Malaria is a disease that affects millions of people worldwide, with a consistent mortality rate. The early identification and diagnosis of malaria is essential to determine the best treatment for the patient and to increase the survival rate. The light microscope examination of blood smears is the gold standard for detecting infection by malaria parasites, but it proves to be time-consuming and difficult. By using CAD systems, this procedure can become much easier, faster, and more accurate. This systematic literature review aimed to provide an overview of the development of CAD systems for the automatic identification and diagnosis of malaria. Scientific publications published between 2014 and 2024 related to ML and DL approaches to the detection and classification of malaria parasite were selected. The searches, conducted in the PubMed, Scopus, and arXiv databases, resulted in the selection of 135 research articles that focused on malaria detection and classification using histological images and reported the results obtained in terms of model performance. Based on the data collected from the selected articles, we conducted a qualitative and quantitative analysis and highlighted the main critical issues found.

6.1. Qualitative and Quantitative Analysis

Although the search included publications since 2014, the first study that reported AI approaches applied to malaria’s automatic diagnosis was published in 2015, and more than half of the selected articles were published between 2022 and 2024. Interestingly, almost 86% of the selected articles were published within the past five years. In addition, over half of the ML articles were published by 2021, and over half of the articles using DL, including hybrid and ensemble learning models, were published from January 2022 to February 2024. The trend of recent publications on the use of CAD systems for the automatic diagnosis of malaria is shown in Figure 5.

To provide as complete an overview as possible of the applications of AI to malaria diagnosis, we have included both articles that used public datasets and those that exploited private datasets in this systematic review. Slightly more than 20% of the total number of articles analyzed used non-publicly available datasets for training automated models; of the remaining articles, the NIH dataset was used in more than 50% of the total (69.16% of public datasets). A graphical representation of this analysis is provided in Figure 6.

It is evident that the NIH dataset is the most considered among the public datasets and as a whole; hence, we provide a more detailed analysis of the AI approaches involving it in the training phase. Concerning the articles summarized in Section 5.1, the following statistics are derived: DL techniques were used 94% of cases, including pre-trained networks (28.13%), custom CNN (43.75%), hybrid models (20.31%), and ensemble learning (7.81%). In general, the most commonly used pre-trained network is VGG, and in particular VGG-19. In hybrid approaches, SVM is the most popular ML model. Solutions based entirely on ML models are proposed with an overall percentage of 6%, indicating that their simplicity is not particularly suited to the task of malaria classification. Figure 7 shows the results of the analysis conducted.

As previously introduced, for the NIH dataset, which is balanced in the classes of infected and non-infected RBCs, we report in the summary Tables of Section 5.1 only the performance in terms of ACC and AUC. However, we also want to point out in

Table 18. Evaluation metrics used by articles based on the NIH dataset.

Evaluation Metrics	References
PR	[49,50,51,52,59,60,61,62,63,64,67,68,69,70,73,74,80,86,87,88,89,90,91,92,93,96,97,98,99,100,101,102,103,107,108,110,111,112,113,114,115,117,118]
REC	[43,50,51,52,55,57,59,60,61,62,63,64,67,68,69,70,73,74,75,80,82,83,85,86,87,88,89,90,91,92,93,95,96,97,98,99,100,101,102,103,104,106,107,108,109,111,112,113,114,115,117,118]
SP	[43,55,57,59,61,67,70,73,74,75,80,82,85,86,88,91,92,95,98,101,102,103,104,106,109,112,113,117,118]
F1	[43,50,52,55,57,59,60,61,63,64,67,68,69,74,80,83,85,86,87,88,89,90,91,92,93,95,96,97,98,99,100,101,102,103,104,107,108,109,110,111,112,114,115,117]
MCC	[43,55,61,67,68,74,85,86,95,98,103,110]

the other performance metrics used by the authors of the cited articles to provide a possible comparison for those interested.

Although the NIH dataset has been used in many applications, especially in recent years, some authors have shown that segmented data contain annotation errors. In particular, Fuhad et al. [92] had the images evaluated by an expert, who eliminated 647 falsely labeled parasites and 750 non-infected suspects. This modified dataset has been publicly shared, and several authors have already used it in their studies. If these errors were confirmed, most of the results published so far in the field of automatic malaria diagnosis would be called into question. Therefore, further investigations should be carried out.

6.2. Validation Analysis

One of the critical aspects to consider when evaluating automated ML and DL systems is their ability to generalize to data never seen before, and coming from sources other than training data. Only in this way is it possible to estimate the extent to which these models are applicable in the real world. Therefore, the so-called external validation is crucial, especially in the medical field. What emerges from the analysis of the articles selected for this paper is that only 22 of the 120 articles focusing solely on the diagnosis of malaria (18.33%) perform this validation. Furthermore, to enhance the generalizability of the proposed models, it would be appropriate to test them on public datasets so that performance can be compared with that obtained by other researchers, a key point of scientific research.

6.3. Explainability

To introduce CAD systems into clinical and diagnostic practice, it is necessary to be able to explain the decisions made by AI models. While ML algorithms are generally transparent due to their simplicity, DL algorithms, and in particular CNNs, involve opacity in decision-making due to the high complexity of the networks [197]. For this reason, such models are also called “black boxes”. In an attempt to explain and interpret the reasoning behind the decisions of DL models, reference can be made to the emerging XAI techniques, which in CV aim to generate heat maps that highlight the regions and pixels of the image that contribute the most to the final prediction. The most commonly used XAI techniques are the Class Activation Mapping (CAM), with its gradient-weighted version (Grad-CAM) [198], and the SHapley Additive exPlanations (SHAP) [199]. These techniques help to increase the confidence of medical experts in CAD systems by providing a visual explanation of automatic diagnosis models. The analysis of the papers selected for this systematic review shows that only a small percentage of the published articles, i.e., 10%, used XAI methods. These papers are shown in

Table 19. XAI methods implemented in malaria CAD systems.

XAI Method	Dataset Used	References
CAM	NIH	[70,73,107]
	Other	[146]
Grad-CAM	NIH	[67,85,86,87,89,97,106]
	Other	-
SHAP	NIH	[96]
	Other	-

, distinguishing between those that used the NIH dataset and those that exploited other data.

XAI methods are still poorly integrated into CAD systems for malaria diagnosis, but it is worth remembering that their use is crucial in all fields where important decisions are required, such as medicine.

6.4. Mobile and Web Applications

Malaria is a widespread disease throughout the world, but its mortality rate is higher in less developed areas. This fact is linked to the reduced availability of instrumental and human resources. Therefore, mobile applications using smartphones and web applications with simple graphical interfaces are a useful tool in these disadvantaged situations. In Section 5.1.7 and Section 5.2.3, mobile and web applications supporting the rapid and cost-effective diagnosis of malaria are discussed. Although the primary focus of other works is not the same, many have developed solutions for the online and offline diagnoses of malaria.

Table 20. Overview of articles on developing mobile and web applications for automatic malaria diagnosis.

Applications
Mobile	Web
[57,59,85,92,110,136]	[43,66,68,86,92,143,179]

shows the works that contribute to this, distinguishing between mobile and web applications.

To introduce these mobile and web-based systems in real-world settings such as underdeveloped countries where malaria is a serious health problem, some considerations have to be made regarding their feasibility, which is highly dependent on the type of device required (type of smartphone/computer or specific equipment), the possibility of working without an internet connection, and the availability of intelligent instruments that perform well with heterogeneous, uncleaned data, with different acquisition conditions (type of instrumentation, setting, and lighting), and for different sample staining. In addition, CAD systems, in general, and those designed for digital pathology as in this case, must achieve a performance that is not inferior to that of experts but, ideally, superior to be considered suitable for diagnostic routines [200].

6.5. Future Directions of AI Approaches to Malaria Diagnosis

The histological evaluation of tissue samples remains a key tool for the assessment of different types of diseases, including malaria infection. However, this examination is time-consuming and tiring for operators, some of the limitations that prompted researchers to investigate the effectiveness of AI techniques in this field. Although ML and DL techniques have shown exciting results in histopathological diagnosis by reducing diagnosis time and achieving high performance, uncertainties remain as to the current feasibility of their application in clinical practice due to several gaps to be filled, which include the absence of standardization of image acquisition/storage/analysis protocols, validation of AI models in real-world settings, and verification by experienced pathologists [201]. These and other weaknesses were also found in the articles examined in this review. A lack of homogeneity was found in the choice of datasets used for training and testing of the proposed approaches, as well as the lack of the validation phase with datasets other than the training datasets and the expert validation conducted by pathologists. Furthermore, the NIH dataset has been questioned by some authors. Since this dataset is the most frequently used public data source, it is worthwhile to check its reliability and, consequently, the credibility of the results obtained so far. Another limitation found in some of the articles analyzed is the lack of detailed information on the models developed and the pre-processing and segmentation techniques applied, a problem easily solved by making public the source code of the algorithms, which would increase the transparency of the processes and provide a starting point for further modifications in the future. Finally, a low rate of application of the model explainability analysis, necessary to provide a clear and transparent picture to medical users, was found to increase their confidence in these new intelligent systems. Despite these weaknesses, overall, AI techniques to support medical diagnosis, and in particular malaria, have attracted the interest of many researchers and have shown promising results. However, to introduce AI in the medical field, the ethical aspects of cybersecurity, protecting patients’ privacy, and ensuring fair access to AI-based healthcare must be considered in the future [202], in addition to filling the above-mentioned gaps.

6.6. General Considerations on This Review

Like any systematic review, this work may suffer from biases related to search strategies, inclusion and exclusion criteria, and the limited number of electronic databases consulted (see Section 2). For the purpose of this work, the search strategies and in particular the keywords chosen and their combinations using the “AND” and “OR” operators led to very satisfactory search results from both a quantitative and qualitative point of view. With regard to the inclusion and exclusion criteria, language and time restrictions do not seem to hinder the identification of relevant articles as we note that the number of publications on automated applications for malaria diagnosis showed an increase after 2016 (2 years later than the lower end of the time range considered), and in addition, no articles written in languages other than English were identified. Furthermore, although we excluded articles focused on the diagnosis of diseases other than malaria, we still included all those articles of generic applications or focused on different diseases but tested on malaria datasets. The decision not to include articles that use AI approaches for tasks other than classification and object detection (i.e., only for segmentation) left some work out but did not affect the objective of reviewing recent research on CAD systems. Another aspect to be assessed when conducting a systematic literature review is the quality of the databases from which articles are selected. In particular, we wondered whether the arXiv database, which is not peer-reviewed, could undermine the credibility and reliability of the review results. To this end, we consulted the research of Gusenbauer et al. [203] who aimed to outline guidance on which search systems are suitable for systematic searches. Their study analyzed 28 academic search systems widely used by researchers, including those used in this systematic review (PubMed, Scopus, and arXiv), making their qualities and limitations transparent. As a result of this analysis, the electronic databases PubMed and Scopus turned out to be principal resources, while arXiv was labeled as a supplementary resource, i.e., a resource to be used “in addition to a principal resource for its specific qualities that could retrieve additional records and to further improve the evidence base”. This database is also classified as gray literature, which the authors consider useful to maximize the completeness of systematic reviews and mitigate publication bias. Based on these findings, we decided to include the arXiv database in this systematic review, also bearing in mind that the number of articles extracted from it is 8, a small amount compared to the total number of papers reviewed. Therefore, the only real limitation of this systematic review might be the limited number of databases considered (PubMed, Scopus, and arXiv). However, we believe that these limitations may weigh less heavily than the highlighted strengths.

7. Conclusions

The current systematic review analyzes the literature evidence on the application of AI to the improvement of malaria diagnosis. Although the results of our review may be limited by the inclusion/exclusion criteria and the choice of electronic databases, its strength can be attributed to the transparency of the methodology used for the search, which makes the results reproducible and exhaustive in summarizing the evidence of AI approaches in malaria diagnosis. Our analysis showed that since 2014, there has been a progressive increase in the application of ML and DL algorithms to the diagnosis of malaria. In particular, most of the studies analyzed chose to rely on DL for the automatic analysis of blood smear images. Although these algorithms have demonstrated excellent performance in CV tasks, the implications regarding the large amount of data and high computational costs required to train the models must be considered. However, once the model is trained, it can be deployed on mobile or web applications and used quickly and easily even in underdeveloped areas. Interestingly, one promising AI approach emerged as potentially translatable shortly. Indeed, according to what was highlighted by a very recent systematic review [204], our analysis showed that DL approaches have notably evolved in this field. In particular, we observed a passage from single CNNs (custom or pre-trained) to hybrid models combining DL models and classical ML approaches, up to more complex ensemble architectures that train multiple models and combine their predictions for the specific task. These latter models often outperform the classical approaches, but their performance strongly depends on several factors including the ensemble system design, the choice of fusion method, and the high computational costs. The implementation of these approaches could be of great relevance in the future to enable early differential diagnosis of the malaria strain and its life cycle, thus providing effective guidance for treatments. Notably, despite significant research progress in this field, many challenges still need to be addressed before CAD systems can be safely and reliably introduced into clinical and diagnostic practice. Future research should focus on overcoming the problems outlined above, as well as on increased collaboration with medical experts in the field to also try to close the gaps that limit the diagnostic accuracy of intelligent systems.