Automatic Inspection of Photovoltaic Power Plants Using Aerial Infrared Thermography: A Review

Abstract

In recent years, aerial infrared thermography (aIRT), as a cost-efficient inspection method, has been demonstrated to be a reliable technique for failure detection in photovoltaic (PV) systems. This method aims to quickly perform a comprehensive monitoring of PV power plants, from the commissioning phase through its entire operational lifetime. This paper provides a review of reported methods in the literature for automating different tasks of the aIRT framework for PV system inspection. The related studies were reviewed for digital image processing (DIP), classification and deep learning techniques. Most of these studies were focused on autonomous fault detection and classification of PV plants using visual, IRT and aIRT images with accuracies up to 90%. On the other hand, only a few studies explored the automation of other parts of the procedure of aIRT, such as the optimal path planning, the orthomosaicking of the acquired images and the detection of soiling over the modules. Algorithms for the detection and segmentation of PV modules achieved a maximum F1 score (harmonic mean of precision and recall) of 98.4%. The accuracy, robustness and generalization of the developed algorithms are still the main issues of these studies, especially when dealing with more classes of faults and the inspection of large-scale PV plants. Therefore, the autonomous procedure and classification task must still be explored to enhance the performance and applicability of the aIRT method.

1. Introduction

As the world experiences a continuously growing share of photovoltaics (PVs) in the energy mix, increasing the performance and reliability of PV installations is of utmost importance. In this context, infrared thermography (IRT) has become a well-established and competitive fault detection method for the condition monitoring and maintenance of PV systems [1]. It provides reliability and accuracy in the detection of typical PV module faults such as bypassed or disconnected substrings, microcracks, soldering problems, shunted cells and disconnected modules. Another feature of this technique is the possible large-scale applicability, through the combination of IRT cameras with an unmanned aerial vehicle (UAV), configured for aerial infrared thermography (aIRT) [1,2].

The first description of the potential of using aIRT in the literature was given in 2012 by Denio [3]. This was followed by the publication of the results of an experimental setup that inspected 60 different PV plants of up to 1 MWp, based on a remote-controlled drone [4]. Since then, several publications have demonstrated the technique’s capability to detect failures in photovoltaic systems quickly and efficiently from the commissioning phase of the power plant through its expected 25 years of operation [5,6,7,8,9].

To further improve the time and cost efficiency of the method, the automation of the entire procedure of the aIRT technique has been assessed in recent years by several research groups worldwide. However, this is a complex task, since it includes not only the automation of the inspection itself (flight path planning and autonomous flight), but also the analysis of a large amount of data for the detection of PV modules, detection of faults, classification and localization of these faults in the field. A significant amount of progress has been made recently in this area, using either simple digital image processing (DIP) techniques or more complex algorithms such as deep learning (DL).

This paper aims at reviewing the reported studies in the literature on the automation of the inspection procedure of PV plants using aIRT.

2. Method of Review

The literature search for conference and journal papers was carried out in the scientific databases IEEE Xplore and ScienceDirect, and on the web scientific indexing services Web of Science and Google Scholar. The keywords used included PV systems, inspections and thermography synonyms. Keywords related to aerial and UAVs were not included to not limit results and exclude the automatic assessment of ground-based thermographic images, besides reducing false positives related to the development of UAVs powered by PV modules. The string used for the search was “(photovoltaic OR PV OR (PV AND modules)) AND (faults OR detection OR classification) AND (automatic OR (artificial AND intelligence) OR processing) AND (thermography OR thermal OR infrared)” and initially returned 183 papers. The papers were filtered to fit the theme and classified according to application and automation algorithm. Papers that focused on the automatic assessment of visual images were not excluded since, normally, aIRT-aimed UAVs also have a visual camera coupled to them, which is also used in the inspection to better classify faults.

The list of papers was expanded using references of the original papers and the references already known by the authors. When overlapping work was found in multiple publications (e.g., preprint in a conference and full paper in a journal), only the publication deemed most important was included.

3. Infrared Thermography (IRT)

Infrared thermography (IRT) is a technique that assesses the radiation emitted by the surface of any body in the infrared wavelength spectrum between 1.4 and 15 μm. IRT cameras used in PV inspections normally have the capability of measuring wavelengths in the mid-infrared wavelength range of 7–14 μm. This is a good compromise between costs and product availability, according to measuring conditions in the field [10].

In a PV plant, PV modules that are close to each other receive almost the same amount of irradiance. Some 20% of this irradiance will be converted to electricity; however, most of the photons which are not converted into electricity will produce heat in the cell. This will increase the temperature of these damaged cells and modules, and the faults will appear in the acquired IRT images as temperature differences [11], like a “temperature signature”.

IRT has been shown to be a reliable and non-destructive tool for detection of different types of faults in PV cells, modules and strings, as they have an effect on the PV module thermal behavior. Table 1 presents some thermal signatures which have been identified and classified in previous studies [2,12] and are standardized according to the international standard IEC TS 62446-3 Edition 1.0 2017-06 [13].

Table 1. Examples of IRT images of typical faults in PV systems.

IRT Image	Description	IRT Image	Description
	Suspicious conductor strip [13,14]		Disconnected substring [13,15]
	Overheated cell [14]		Disconnected string [13]
	Module with broken front glass [13]		String in short circuit (patchwork) [13,14]
	Heated module junction box [13,14,15]		Partly shaded cells (not a defect) [13,14]
	Substring in short circuit [13,15]

Table 1. Examples of IRT images of typical faults in PV systems.

IRT Image	Description	IRT Image	Description
	Suspicious conductor strip [13,14]		Disconnected substring [13,15]
	Overheated cell [14]		Disconnected string [13]
	Module with broken front glass [13]		String in short circuit (patchwork) [13,14]
	Heated module junction box [13,14,15]		Partly shaded cells (not a defect) [13,14]
	Substring in short circuit [13,15]

4. Unmanned Aerial Vehicles (UAVs)

Unmanned aerial vehicles (UAVs) are aircrafts that are capable of remote or autonomous operation. They were initially developed for military applications, but due to recent developments, they are now available for civil activities and commonly used in applications such as rescuing and disaster relief, energy power line monitoring and environmental and forest control [16]. UAVs are becoming more popular especially in the energy and agriculture sectors, due to their fast large-area coverage, precise imagery, high flexibility, light weight, low cost and ability to operate in hostile environments [17,18]. Several countries have established legal guidelines for the use of UAVs. The rules can include limitations of flight areas, license requirements and insurance [19].

UAVs can carry various cameras and sensors to collect data and can be classified by size, range or endurance [20]. The most appropriate UAV equipment type for thermographic inspections is multicopters, which use rotary blades to generate lift, because of their stability and easy usability. They can be classified by the number of motors (tricopters, quadcopters, hexacopters and octocopters), with quadcopters being the most popular on the market [19]. They can also be classified by autonomy levels, being manually operated, semi-autonomous (i.e., need of a human operator for mission planning and for taking some of the movement decisions) or fully autonomous (i.e., all decisions for a delegated mission are made by the UAV based on sensor observations) [21].

5. Aerial Infrared Thermography

In the past, IRT inspections of PV systems were performed on the ground or on lifting platforms with handheld IRT cameras. This procedure cannot provide fast and accurate coverage of the power plant and is dependent on human labor and competence, besides being very time-consuming and labor-intensive. As a result, the inspection accuracy is susceptible to human error. A possible solution to the problem is to combine the IRT sensor with aerial technologies such as UAVs. This procedure is known as aIRT and increases the cost effectiveness of IRT inspection, allowing the technique to be employed for large-scale PV plants or roof-mounted PV systems with limited access [8,22,23,24].

This method has shown its potential in recent years, but its use can still be expanded to increase and simplify routine inspections of PV power plants and revolutionize the future of PV plant monitoring procedures [16]. For this method to achieve its full potential, however, it should be combined with automation algorithms and technologies, such as automated route planning and defect identification.

Aerial IRT uses visual and IRT cameras that are mounted on the UAV. The equipment provides real-time and precise imagery, allowing a time-efficient inspection. The procedure can detect faults of different types, such as cell cracks, corrosion spots, broken cells, hot spots, snail trails, discoloration, soiling, disconnected modules or strings and potential-induced degradation (PID) [18,25]. According to the inspection goal and the PV plant layout, aIRT can be performed from different altitudes and directions to identify specific defects or faults, also depending on the time available for the inspection [26].

The aIRT process is carried out in three stages: acquisition of imagery, assessment of data and remediation actions. For the acquisition step, the UAV must fly a designated route acquiring successive photos or videos over the site to build an imagery database covering all modules in the system. As the equipment has a limited battery capacity, each flight has a duration of around 20 min [6].

During the flight, the weather conditions, wind speed and sunlight reflection must be monitored, as they can affect the measurements and consequently the quality of the aIRT images. Additionally, the velocity of the UAV and the orientation and angle of the IRT sensor must be controlled to avoid self-shading, blurred images and other reflections. aIRT should be performed on cloudless, sunny and clear days, with minimum irradiance of 600 W/m² on the plane of the PV array under inspection. The flight path and velocity should be planned beforehand in order to optimize the coverage and to avoid any drift during the flight [25,26,27,28].

The acquired data are then analyzed, and an action report is produced. With a detailed site mapping, it is possible to obtain an accurate location of the system faults, and the remediation can be planned based on full knowledge of the site state. The report is passed to the stakeholders for remediation actions such as assessment of connections or replacement of modules or fuses.

6. Aerial Inspection Algorithms

6.1. Digital Image Processing

The field of digital image processing (DIP) has been an object of increasing interest because it allows many applications such as: remote sensing, component defect detection, biomedical engineering, geoprocessing, metallography and industrial automation applications. DIP aims to transform and analyze images by extracting relevant information, highlighting and identifying patterns and objects [29,30].

A digital image consists of a set of a finite number of elements (i.e., pixel), each one with a specific location and intensity. DIP techniques apply several operations to these pixels to transform the images, allowing the automatic interpretation of them by systems or machines. These operations include image rotation, thresholding, binary image analysis, brightness and contrast adjustment, filtering, resizing and interpolation [27,29].

The methods of DIP are normally simple but can often solve problems in a more time- and computing-efficient way than DL techniques. Pixel thresholding or algorithms such as the scale-invariant feature transform (SIFT) are normally very general, do not require a large dataset to be trained and can be replicated in other images. DIP can even be combined with DL to take the best from both methods to automate image processing and recognition [31,32].

6.2. Deep Learning (DL)

Artificial neural networks (ANNs) are the most popular technique in machine learning (ML) and were first developed based on the structure and operation of the human brain. They are commonly used because they can deal with highly nonlinear systems and allow constant updates in the model [33]. They are composed of a set of simple, connected processors called neurons that produce a sequence of activation calculations. The fundamental property of an ANN is its ability to learn from the environment (read a set of examples), through an iterative process of adjustments applied to synaptic weights and bias levels. Learning a task consists in finding weights that make the ANN exhibit a desired output when processing an input. Depending on the problem and how the neurons are connected, this process may require chains of consecutive computational stages, where each one modifies the aggregate activation of the network. DL is about accurately assigning credit across many of these stages, using convolutional neural networks (CNNs) [34,35].

CNNs were inspired by the human visual system’s structure and are the state-of-the-art method for image classification [36,37,38,39]. They are easy to train if there is a substantial number of labeled images that represent distinct categories.

They are layered sequences, and each has a specific function in the propagation of the input signal. There are three main types of neural layers: (i) convolutional layers, (ii) pooling layers and (iii) fully connected layers. Figure 1 illustrates the CNN architecture and its three main layers for a task of object detection in an image:

• Convolutional layers: responsible for extracting attributes from the input volumes.
• Pooling layers: responsible for reducing the spatial dimensions of the input volume after the convolutional layers, reducing the computational work of the network.
• Fully connected layers: perform the final classification of the images, in the same way as a conventional neural network.

In the end, the CNN output is the probability that the input image belongs to one of the classes for which the network was trained [40].

6.3. Other Machine Learning Techniques

Other ML techniques are also used in the automation of aIRT inspection, especially for classification in combination with other techniques. Among them, support vector machines (SVMs) are supervised ML algorithms that are usually used for classification and regression analysis of data, as they are based on a decision function that divides the classes by decision planes. SVMs work relatively well when there is a clear separation between classes, and they are more effective in high-dimensional spaces [41,42].

Another commonly used classifier is the random forest (RF), which is an algorithm that builds several decision trees on multiple random portions of the data for training. In these trees, different binary classifications concatenated in a tree structure are performed, for different input features, and a combination of the nodes is used to calculate the result. The most common result among the trees is “elected” as the output of the classifier [43,44].

On the other hand, k-nearest neighbors (KNN) is a classification supervised algorithm that does not require training, as the samples are classified by a metric of distance, where the k-nearest points to the test sample are defined as a class [43].

For clustering, the most popular method is k-means clustering, which is an unsupervised ML algorithm. This method consists of dividing data in k clusters that will be grouped by the mean distance between points [45,46].

6.4. Algorithm Evaluation Metrics

Performance indices quantify the capacity of an algorithm to identify the events of interest. The accuracy (Ac) of an algorithm is the percentage of correct predictions over the test dataset. The precision index (Pr) provides the proportion of all segments that were identified in one class that in fact belong to this class (results of true positives over all the positives). The recall (Re) measures how well the algorithm can identify each class among the dataset (results of true positives over all images in a dataset that were originally labeled as positives). The F1 score is calculated by the harmonic mean of Pr and Re. A good algorithm should have all of these metrics close to 100% for all the classes [47].

Other less common metrics are also used in the papers covered by this work. The Matthews correlation coefficient (MCC) is a coefficient related to accuracy, which does not penalize classes of different sizes [48]. The Dice coefficient and the Jaccard index are often used for evaluating segmentation tasks. The first represents the overlapping between the predicted result with the ground truth, and the second measures the similarity of the predicted result to the ground truth [37,49]. The Jaccard index is also known as Intersection over Union (IOU). The area under the receiver operating characteristic (AUROC) is an index obtained by integrating the curve of the true positives of a task over the false positives at various decision thresholds [50].

In this paper, when available, the preferred metrics are F1 score and precision indices, in order to provide a comparison between different works, whenever possible.

7. Applications of Automatization Algorithms

7.1. Automatic Path Planning

In an automatic aIRT mission, the UAV flies over a set of waypoints that cover all modules of the PV plant. Therefore, an optimized path-planning algorithm aiming at an optimal path for time and battery efficiency is essential [51]. Figure 2 depicts the coverage area by a UAV based on the field of view (FoV) and resolution of the camera on board as well as the essential parameters used for path planning [51].

Available market software packages already provide an automatic flight based on a so-called “lawn mower” flight pattern. However, they do not always provide the most efficient flight and do not guarantee a centralized view to the PV arrays, especially in power plants installed over complex topography.

Studies aiming at optimizing path planning include different approaches to the problem. In the study developed by Salahat et al. [52], the traveling salesman shortest path algorithm was used to generate a path that includes a randomly selected set of modules that represent the entire PV plant, allowing an optimization of the battery use. Ding et al. [53] based their method on density clustering, boustrophedon path planning and Bezier curves. Luo et al. [54] also based their algorithm for path planning optimization on Bezier curves in a joint approach with particle swarm optimization (PSO), taking into consideration the flight attitude, gimbal limitation and path length.

Image stitching and DIP techniques were used by Henry et al. [55] to find contours of the power plant and generate a “lawn mower” path over it. A similar approach, using DL, was adopted by Moradi Sizkouhi et al. [51,56], which was also complemented with a dynamic path planning, which deviates the previous flight plan to take closer photos when faults are detected. Pérez-González et al. [57] also used DL to detect the area of the PV plant and then used different algorithms to determine the best flight path, wherein exact cellular decomposition boustrophedon and grid-based wavefront coverage algorithms produced the best results.

Other real-time algorithms have been proposed that calculate the optimized path of the UAV during the flight. In Roggi et al. [58], the UAV corrected the pre-planned “lawn mower” path according to the images that it acquired and the image processing techniques that are applied. A vision-based flight control was also proposed by Xi et al. [59], which performs a real-time direction and velocity correction.

7.2. Detection of PV Systems

Although not always related to aerial inspections, the detection of PV systems in aerial imagery (UAV or satellite imagery) has been proposed by many authors and even used for path planning before the aIRT flight [56,57]. Table 2 presents a summary of methods used in the literature for detecting PV systems in aerial imagery. The table presents the best metrics obtained in each study, the type of image data used as input and the type of detection output obtained with each method. The detection output was classified into three categories:

(a)

Boxes: the output is given by the coordinates of a box or polygon placed around the PV system;

(b)

Mask: the output is a binary image where the pixels corresponding to the segment of the PV system are represented by the value 1 and the rest of the image is represented by 0;

(c)

Binary: for each image, the presence or absence of a PV system is the result of the algorithm (1 or 0).

Figure 3 shows two examples of two types of detections, by the coordinates of boxes around the detected PV panels (left) and masks of the segment of the PV system (right). Besides developing an algorithm for the detection of PV arrays, Wu et al. [60] also matched them to their string identifiers.

7.3. Detection of PV Modules

The detection of the area of the modules is an important step in the image processing task, which is required for both detecting modules with defects and locating them in the power plant.

Table 2. Summary of methods for detecting PV systems using aerial imagery.

[Ref]/Year	Algorithm	Best Results	Output Type	Images
[38] 2016	RF and DL	Pr: 90%	Mask	Aerial imagery
[60] 2017	Feature description vector according to PV modules’ different colors	-	Boxes	UAV
[62] 2017	Adaptive clustering method based on k- means	Loss rate is lower than 5%	Mask	Aerial imagery
[63] 2017	GLCM algorithm	Pr: 93.16%	Mask	aIRT
		F1: 77.8%
[64] 2018	DIP and k-means classifier	Pr > 99%	Boxes	Aerial imagery
[65] 2018	DL (Segnet)	Pr: 90%	Mask	Aerial imagery
[66] 2018	DL (PolyCNN)	IoU: 79.5%	Mask	Google Earth
[67] 2019	DL (Faster R-CNN, based on the classifier ResNet-50)	Pr: 92.9%	Boxes	Google Earth
[49] 2019	DL (Res-UNet)	Ac: 97.11%	Mask	System IRT images
[68] 2020	DL (Mask R-CNN and VGG16)	Ac: 96.99%	Mask	UAV
[47] 2020	DL (U-net)	F1: 82%	Mask	Google Earth
[69] 2020	DL	F1: 92.2%	Binary	Satellite imagery
[70] 2020	DL	Pr: 92.66%	Mask	Google Earth
		Re: 97.43%
[71] 2020	DL (CNN for semantic segmentation)	Average error of 5.75%	Mask	UAV
[72] 2020	k-means, SVM and CNN	MCC: 0.17	Mask	Identify solar on rooftops
[61] 2020	DIP (edge detection) and DL (R-CNN)	Pr: 92.25%	Mask	Panels in aIRT images
[73] 2021	DL algorithms	F1: 95.38%	Mask	Aerial imagery
[74] 2021	DIP (transform invariant low-rank textures (TILT) algorithm for orthographic view and Otsu’s method for segmentation)	-	Mask	Panels in aIRT images
[75] 2021	Unsupervised segmentation parameter optimization (USPO) and RF classifier	F1: 98.7%	Mask	UAV
[57] 2021	DL server	-	Mask	UAV
[51] 2022	Mask R-CNN structure	Ac: 96.99%	Mask	UAV

Table 2. Summary of methods for detecting PV systems using aerial imagery.

[Ref]/Year	Algorithm	Best Results	Output Type	Images
[38] 2016	RF and DL	Pr: 90%	Mask	Aerial imagery
[60] 2017	Feature description vector according to PV modules’ different colors	-	Boxes	UAV
[62] 2017	Adaptive clustering method based on k- means	Loss rate is lower than 5%	Mask	Aerial imagery
[63] 2017	GLCM algorithm	Pr: 93.16%	Mask	aIRT
		F1: 77.8%
[64] 2018	DIP and k-means classifier	Pr > 99%	Boxes	Aerial imagery
[65] 2018	DL (Segnet)	Pr: 90%	Mask	Aerial imagery
[66] 2018	DL (PolyCNN)	IoU: 79.5%	Mask	Google Earth
[67] 2019	DL (Faster R-CNN, based on the classifier ResNet-50)	Pr: 92.9%	Boxes	Google Earth
[49] 2019	DL (Res-UNet)	Ac: 97.11%	Mask	System IRT images
[68] 2020	DL (Mask R-CNN and VGG16)	Ac: 96.99%	Mask	UAV
[47] 2020	DL (U-net)	F1: 82%	Mask	Google Earth
[69] 2020	DL	F1: 92.2%	Binary	Satellite imagery
[70] 2020	DL	Pr: 92.66%	Mask	Google Earth
		Re: 97.43%
[71] 2020	DL (CNN for semantic segmentation)	Average error of 5.75%	Mask	UAV
[72] 2020	k-means, SVM and CNN	MCC: 0.17	Mask	Identify solar on rooftops
[61] 2020	DIP (edge detection) and DL (R-CNN)	Pr: 92.25%	Mask	Panels in aIRT images
[73] 2021	DL algorithms	F1: 95.38%	Mask	Aerial imagery
[74] 2021	DIP (transform invariant low-rank textures (TILT) algorithm for orthographic view and Otsu’s method for segmentation)	-	Mask	Panels in aIRT images
[75] 2021	Unsupervised segmentation parameter optimization (USPO) and RF classifier	F1: 98.7%	Mask	UAV
[57] 2021	DL server	-	Mask	UAV
[51] 2022	Mask R-CNN structure	Ac: 96.99%	Mask	UAV

The first attempts to automatize the segmentation of the PV modules in IRT images appear to have been made in 2015, with images obtained with a moving cart, using simple DIP methods [76]. Other attempts were described in Menéndez et al. [77], Montanez et al. [78] and Wu et al. [79]. Uma et al. [80] did the same using a k-means clustering algorithm. In 2021, Xie et al. [81] used a Segnet, a CNN used for segmentation.

For aerial images, Tribak and Zaz [82], Salamanca et al. [83] and Perez et al. [84] published results on detecting PV modules in aerial visual images, and many studies used different techniques to detect and segment PV module boundaries in aIRT data. Table 3 shows the studies related to DIP and DL algorithms. In this case, the detection result of the algorithm can be given by a box or polygon around the PV module, a mask of the segment of the module or lines that mark the borders of the modules in a PV panel.

The best metric among the studies was obtained with a combination of many algorithms (DIP, SVM and DL) [85]. On the other hand, the worst metrics were obtained with simple DIP filters [86]. It is important to note that not all studies are comparable, because not all have presented metrics for their performance, and they have different dataset sizes, which make the comparison difficult. The studies described by Carletti et al. [87] and Bommes et al. [88], besides detecting the PV module, also developed ways to track the modules in subsequent frames of a video. A tracking system was also developed by Xie et al. [89] using an AlexNet CNN.

Table 3. Summary of methods for detecting PV modules in aIRT using DIP and DL algorithms.

[Ref]/Year	Algorithm	Best Results	Output Type
[48] 2016	DIP (normalization and thresholding)	F1: 92.76%	Boxes
[90] 2017	DIP (edge extraction by Hough transform)	-	Boxes
[91] 2017	DIP (thresholding)	Pr: 96.9%	Mask
[92] 2017	RANSAC (random sample consensus) algorithm	-	Boxes
[93] 2017	DIP (not detailed)	Pr: 82%	Boxes
[94] 2017	DIP (thresholding in HSV color space)	-	Mask
[95] 2018	DIP (template matching algorithm)	F1: 83.0%	Boxes
[87] 2020	DIP (canny edge and morphological filters)	F1: 87%	Boxes
[86] 2020	DIP (ACM LS and filtering by area, Hough transform)	Re: 70%	Boxes
[96] 2020	DIP (thresholding in HSV color space and MSER algorithm)	Ac: 98%	Boxes
[97] 2020	DL (YOLOv3)	F1: 95%	Boxes
[85] 2020	DIP + support vector machine (SVM) and DL (Mask R-CNN)	F1: 98.9%	Boxes
[98] 2020	DIP (Hough line detection, Sobel operator)	-	Lines
[99] 2020	DIP (Sobel and canny operator, HoughPLine)	Pr: 90.91%	Lines
[100] 2020	DIP (LSD algorithm and k-means clustering)	Pr: 77.3%	Mask
		F1: 86.3%
[101] 2020	DIP (k-means clustering and thresholding)	Ac: 98.46%	Mask
[88] 2021	DL (Mask R-CNN)	Pr: 90.01%	Mask
		F1: 90.51%
[102] 2022	DIP (geometry coercion, clustering and angularity-based segment filtering)	-	Mask

Table 3. Summary of methods for detecting PV modules in aIRT using DIP and DL algorithms.

[Ref]/Year	Algorithm	Best Results	Output Type
[48] 2016	DIP (normalization and thresholding)	F1: 92.76%	Boxes
[90] 2017	DIP (edge extraction by Hough transform)	-	Boxes
[91] 2017	DIP (thresholding)	Pr: 96.9%	Mask
[92] 2017	RANSAC (random sample consensus) algorithm	-	Boxes
[93] 2017	DIP (not detailed)	Pr: 82%	Boxes
[94] 2017	DIP (thresholding in HSV color space)	-	Mask
[95] 2018	DIP (template matching algorithm)	F1: 83.0%	Boxes
[87] 2020	DIP (canny edge and morphological filters)	F1: 87%	Boxes
[86] 2020	DIP (ACM LS and filtering by area, Hough transform)	Re: 70%	Boxes
[96] 2020	DIP (thresholding in HSV color space and MSER algorithm)	Ac: 98%	Boxes
[97] 2020	DL (YOLOv3)	F1: 95%	Boxes
[85] 2020	DIP + support vector machine (SVM) and DL (Mask R-CNN)	F1: 98.9%	Boxes
[98] 2020	DIP (Hough line detection, Sobel operator)	-	Lines
[99] 2020	DIP (Sobel and canny operator, HoughPLine)	Pr: 90.91%	Lines
[100] 2020	DIP (LSD algorithm and k-means clustering)	Pr: 77.3%	Mask
		F1: 86.3%
[101] 2020	DIP (k-means clustering and thresholding)	Ac: 98.46%	Mask
[88] 2021	DL (Mask R-CNN)	Pr: 90.01%	Mask
		F1: 90.51%
[102] 2022	DIP (geometry coercion, clustering and angularity-based segment filtering)	-	Mask

7.4. Orthomosaicking

The localization of faults within a power plant is a challenging issue that can be addressed by creating an orthomosaic of the PV power plant [103]. Image mosaicking, also known as image stitching, is a computational technique that detects overlapping key points in spatially subsequent photos and uses them to create a so-called panorama picture [94].

To improve the sight perspective and enable an expanded view of the localization of faults in both visual and thermal images of PV power plants, some researchers used commercially available software packages to create orthomaps with aerial imagery. Lee and Park [104] and Zefri et al. [105] used the software Pix4D capture to process thermal and visual images and create orthographic images with temperature information. Oliveira et al. [103] compared the use of two software packages (DroneDeploy and ContextCapture) to create an orthomosaic of a 1 MW PV power plant. Higuchi and Babasaki [106] used the software OpenDroneMap to generate the orthographic image of a 2 MW PV power plant.

Grimaccia et al. [94], Aghaei et al. [27] and Ismail et al. [107] have proposed methods for the orthomosaicking of visual images of PV power plants using DIP techniques. Tsanakas et al. [108] used the method of aerial triangulation, and Lafkih and Zaz [109] and Zefri [102,110] used the SIFT technique to perform the task. To optimize the mosaicking of visual PV images, Qi et al. [111] used a Faster R-CNN to detect key points in aerial sequence images in the world coordinate system, so it avoids redundant information generated by traditional methods. López-Fernández et al. [92] developed a tool that creates a 5D point cloud of the power plant, where each coordinate point has a temperature and an intensity value associated with it. After segmenting the modules in a dataset, Costa et al. [73] used a sliding window algorithm with overlapping pixels, combining frames side by side to reconstruct orthomosaics of power plants.

7.5. Soiling

A common cause of hot spots in PV power plants is soiling and shadow over the modules, which hinders the evaluation of results since they are not considered real defects of the PV modules [6]. Cipriani et al. [112] approached this issue by using a CNN to differentiate hot spots caused by faults from soiling, obtaining an accuracy of up to 98%.

Another solution to the problem is the analysis of the visual images that are normally taken together with the IRT images in the UAV, which enables the operator to discard hot spots caused by soiling. Automation of the task of detecting soiling in individual modules was proposed by Yang et al. [113], Pivem et al. [114] and Qasem et al. [115] using DIP techniques. Similar techniques were employed by Wen et al. [116], and by Karaköse and Firildak [117] to detect shadows over PV systems. Hanafy et al. [43] compared different ML algorithms (KNN, NN, RF and SVM) to classify modules in different categories of cleanliness and obtained an accuracy of over 90% using an SVM algorithm. Mehta et al. [37] proposed a method that uses a weakly supervised CNN-based classification network to predict power loss, detect soiling and categorize it given a PV module image. This method obtained an accuracy of about 87% on a test dataset of about 50 images.

7.6. Detection and Classification of Faults

The manual assessment of aIRT imagery is a time- and computing-consuming task; therefore, its automation is the most explored part of the aIRT framework in the literature. This detection can either be processed on board, during the UAV flight, as shown in the example in Figure 4, or in a computer software, after the acquisition of images has been carried out by the UAV (Figure 5). Both Figure 4 and Figure 5 show the procedure of the inspections for the two different approaches, including all tasks being automatized in each case.

In 2002, Pilla et al. [118] used the Sobel and canny edge operators to detect cracks in IRT PV cell images. In 2003, Wang et al. [119] used thresholding and a fuzzy classifier to detect faults in IRT images. After that, in 2011, Vergura and Falcone [120] used DIP techniques to analyze IRT images for faults. Since then, many other studies have used mostly DIP techniques to segment faults in IRT images. Table 4 presents a summary of the main algorithms for the detection and classification of faults in IRT images. In this case, the results can be in the form of a segmentation of the faults (mask), the detection of modules with faults and the classification of these faults in categories. Some methods presented high metrics when used in association with a classification algorithm, such as an SVM [121]. Many papers do not present a metric for the performance evaluation of the algorithms proposed, especially for the case of fault segmentation (masks). It is important to note that a comparison between metrics is not always possible, because the dataset size and number of classes differ among studies.

Table 4. Summary of methods for detecting and classifying faults in IRT images of PV modules.

[Ref]/Year	Algorithm	Best Results	Output Type
[76] 2015	DIP filters	Pr: 97.40%	Detection of modules with faults
[122] 2015	Histogram analysis, thresholding and canny edge	-	Fault mask
[123] 2016	DIP and ANN	Ac: 80%	Fault mask
[124] 2016	Filter by temperature values	-	Detection of hot spots, cracks and soldering issues
[125] 2016	k-means color quantization and density-based spatial clustering of application with noise (DBSCAN)	-	Fault mask
[126] 2017	DIP (thresholding and clustering)	-	Detection of modules with faults
[127] 2017	DIP and fuzzy rule on temperature data	-	Classification into 6 anomaly types
[77] 2018	Filtering by temperature values	Pr: 96.52%	Detection of modules with hot spots
[128] 2019	Thresholding in HSV color space	Ac: 100% (only 3 images)	Detection of modules with faults
[129,130] 2019	n-Bayes classifier	Ac: 94.1%	Classification into healthy, shaded and defective modules
[80] 2019	DIP filters	-	Detection of delamination, snail trails and bubbled faults
[131] 2019	ANN and discrete wavelet transform	Ac: 100%	Combination of IRT with electrical data—classification into 5 anomaly types
[132] 2019	Fuzzy inference system (FIS) using Mamdani-type fuzzy controller	Ac: 96.7%	Combination of IRT with electrical data—detection and classification of hot spots
[133] 2020	GoogleNet vs. LeNet-5 and VGG-16	Ac: 97.9%	Classification into 6 cell anomaly types
[134] 2020	DL and transfer learning	Ac: 99.23%	Detection of modules with faults
[121] 2020	DIP and SVM	Ac: 97%	Detection of modules with faults
[78] 2020	Histogram analysis of segmented module	-	Detection of modules with faults
[135] 2020	Fuzzy classifier based on temperatures of the module	Ac: 94%	Detection of EVA discoloration and delamination faults
[136] 2021	Multilevel Otsu thresholding	Ac: 91.81%	Fault mask
[137] 2021	DL and SVM	Ac: 86%	Classification into 12 anomaly types
[138] 2021	DIP (adaptative histogram equalization), DL and SVM	Ac: 92.96%	Detection of modules with faults
[139] 2021	DIP filters and SVM classifier	Ac: 94.4%	Classification into 10 anomaly types

Table 4. Summary of methods for detecting and classifying faults in IRT images of PV modules.

[Ref]/Year	Algorithm	Best Results	Output Type
[76] 2015	DIP filters	Pr: 97.40%	Detection of modules with faults
[122] 2015	Histogram analysis, thresholding and canny edge	-	Fault mask
[123] 2016	DIP and ANN	Ac: 80%	Fault mask
[124] 2016	Filter by temperature values	-	Detection of hot spots, cracks and soldering issues
[125] 2016	k-means color quantization and density-based spatial clustering of application with noise (DBSCAN)	-	Fault mask
[126] 2017	DIP (thresholding and clustering)	-	Detection of modules with faults
[127] 2017	DIP and fuzzy rule on temperature data	-	Classification into 6 anomaly types
[77] 2018	Filtering by temperature values	Pr: 96.52%	Detection of modules with hot spots
[128] 2019	Thresholding in HSV color space	Ac: 100% (only 3 images)	Detection of modules with faults
[129,130] 2019	n-Bayes classifier	Ac: 94.1%	Classification into healthy, shaded and defective modules
[80] 2019	DIP filters	-	Detection of delamination, snail trails and bubbled faults
[131] 2019	ANN and discrete wavelet transform	Ac: 100%	Combination of IRT with electrical data—classification into 5 anomaly types
[132] 2019	Fuzzy inference system (FIS) using Mamdani-type fuzzy controller	Ac: 96.7%	Combination of IRT with electrical data—detection and classification of hot spots
[133] 2020	GoogleNet vs. LeNet-5 and VGG-16	Ac: 97.9%	Classification into 6 cell anomaly types
[134] 2020	DL and transfer learning	Ac: 99.23%	Detection of modules with faults
[121] 2020	DIP and SVM	Ac: 97%	Detection of modules with faults
[78] 2020	Histogram analysis of segmented module	-	Detection of modules with faults
[135] 2020	Fuzzy classifier based on temperatures of the module	Ac: 94%	Detection of EVA discoloration and delamination faults
[136] 2021	Multilevel Otsu thresholding	Ac: 91.81%	Fault mask
[137] 2021	DL and SVM	Ac: 86%	Classification into 12 anomaly types
[138] 2021	DIP (adaptative histogram equalization), DL and SVM	Ac: 92.96%	Detection of modules with faults
[139] 2021	DIP filters and SVM classifier	Ac: 94.4%	Classification into 10 anomaly types

Table 5. Summary of methods for detecting and classifying faults in visual images of PV modules.

[Ref]/Year	Algorithm	Best Results	Detection Type
[142] 2014	Color segmentation based on k-means clustering	-	Detection of cracks, interconnect problems and discolored areas
[140] 2018	Luminance filters	-	Cell fault detection
[143] 2018	DL	Ac: 98.9625%	Classification into 8 anomaly types of faults in aerial images
[144] 2020	Different CNNs	Ac: 93.3%	Detection of faults
[71] 2020	CNN for semantic segmentation	Ac: 75%	Detection of glass breakages, shadows and dust
[145,146,147] 2020	DL and SVM	Ac: 99.8%	Classification into 6 anomaly types of faults
[148] 2020	Kirsch edge detection	-	Detection of glass breakages
[149] 2020	Yolo and MobileNet-SSD network	Pr: 89.2%	Classification into 3 anomaly types of size of hot spot (not carried out in real PV system)
[150] 2021	DL	Ac: 95.07%	Detection of burn marks, delamination, discoloration, glass breakages and snail trails
[151] 2021	k-means, SVM and CNN	MCC: 1.0	Detection of damaged modules on rooftops
[141] 2021	Naïve Bayes, SVM, k-nearest neighbors, decision tree, RF and pretrained DL models	Ac: 100%	Detection of burn marks, delamination, discoloration, glass breakages and snail trails
[152] 2021	DL	Ac: 93%	Masks of bird droppings

shows the summary of studies of the detection and classification of faults in visual images. The faults detected in this case are mostly related to visible problems, such as delamination, burn marks and glass breakages. An important issue that makes the comparison between studies difficult is the difference between the data resolution used as input for each one of them, as images vary from PV cells [140] to aerial images [141].

Table 6 and Table 7 present a summary of the methods for detecting and classifying faults in aIRT images of PV systems, using DIP and DL algorithms, respectively. In general, the algorithms with the highest results are the ones dedicated to the detection of faults or the classification of a few types of faults, with the classification of many classes of faults being a much more complex task. It is noticeable that the DIP algorithms have comparable results to DL techniques, even though most of them use smaller datasets, and therefore their generalization capabilities can be jeopardized.

Some of the challenges for the development of a robust automatic classification of faults include the reflections and shadows from surroundings and the lack of a standardized image database (with standard flight directions and weather conditions). When using DL, the computation requirements, the need for a large dataset with annotated data and the processing time must also be overcome.

Table 6. Summary of methods for detecting PV modules in aIRT using DIP and classification algorithms.

[Ref]/Year	Algorithm	Best Results	Detection Type
[16] 2015	Thresholding and temperature filtering	-	Detection of faults
[153] 2015	DIP filters	F1: 99.4%	Detection of faults
[154] 2015	Canny edge and thresholding		Fault mask
[48] 2016	Statistical classification of faults	F1: 93.88%	Classification into 3 classes of faults
[155] 2016	DIP and k-means clustering	-	Detection of faults
[156] 2016	Histogram filtering	-	Detection of hot spots
[157] 2017	Thresholding module segment	-	Detection of faults
[94] 2017	Thresholding by module luminance distribution	-	Classification into 3 classes of faults
[91,158] 2017	Mean and std of luminance of area of module	Ac: 97%	Detection of faults
[90] 2017	DIP filters	-	Fault mask
[92] 2017	Temperature segmentation	Ac: 100%	Fault mask
[159] 2017	Non-uniform illumination (NUI) boundary detection	-	Hot spot detection and analysis of visual images for soiling or shadowing in laboratory setup
[160] 2018	Thresholding, pixel seed and canny edge	-	Fault mask
[95] 2018	Normalized cross-correlation as a similarity measure for template matching	F1: 75%	Detection of faults
[79] 2018	DIP filters	-	Fault mask
[161] 2019	Thresholding	-	Fault mask
[162] 2019	Gaussian filter and Hough line	-	Detection of hot spots
[163] 2019	Hog features and cascade object detector	-	Detection of hot spots
[164] 2020	Thresholding	-	Fault masks
[99] 2020	Statistics of the luminance	Pr: 92.71%	Classification into 3 classes of faults
[165] 2020	DIP for feature extraction + different algorithms for classification (SVM, n-Bayes, KNN, etc.)	Ac: 92%	Detection of faults
[45] 2020	k-means clustering	-	Fault mask
[96] 2020	Temperature-based thresholding	Ac: 97%	Detection of faults
[101] 2020	Temperature-based thresholding	Pr: 97.6%	Classification of size and severity of faults
[87] 2020	Water filling and temporal tracking algorithms	F1: 72%	Detection of hot spots
[166] 2021	Statistical analysis of temperature of modules	Ac: 96%	Classification into 6 classes of faults
[74] 2021	Robust PCA decomposition and thresholding	F1: 78.23%	Fault mask
[167] 2021	Filtering and probability density functions	-	Fault detection using both visual and aIRT images

Table 6. Summary of methods for detecting PV modules in aIRT using DIP and classification algorithms.

[Ref]/Year	Algorithm	Best Results	Detection Type
[16] 2015	Thresholding and temperature filtering	-	Detection of faults
[153] 2015	DIP filters	F1: 99.4%	Detection of faults
[154] 2015	Canny edge and thresholding		Fault mask
[48] 2016	Statistical classification of faults	F1: 93.88%	Classification into 3 classes of faults
[155] 2016	DIP and k-means clustering	-	Detection of faults
[156] 2016	Histogram filtering	-	Detection of hot spots
[157] 2017	Thresholding module segment	-	Detection of faults
[94] 2017	Thresholding by module luminance distribution	-	Classification into 3 classes of faults
[91,158] 2017	Mean and std of luminance of area of module	Ac: 97%	Detection of faults
[90] 2017	DIP filters	-	Fault mask
[92] 2017	Temperature segmentation	Ac: 100%	Fault mask
[159] 2017	Non-uniform illumination (NUI) boundary detection	-	Hot spot detection and analysis of visual images for soiling or shadowing in laboratory setup
[160] 2018	Thresholding, pixel seed and canny edge	-	Fault mask
[95] 2018	Normalized cross-correlation as a similarity measure for template matching	F1: 75%	Detection of faults
[79] 2018	DIP filters	-	Fault mask
[161] 2019	Thresholding	-	Fault mask
[162] 2019	Gaussian filter and Hough line	-	Detection of hot spots
[163] 2019	Hog features and cascade object detector	-	Detection of hot spots
[164] 2020	Thresholding	-	Fault masks
[99] 2020	Statistics of the luminance	Pr: 92.71%	Classification into 3 classes of faults
[165] 2020	DIP for feature extraction + different algorithms for classification (SVM, n-Bayes, KNN, etc.)	Ac: 92%	Detection of faults
[45] 2020	k-means clustering	-	Fault mask
[96] 2020	Temperature-based thresholding	Ac: 97%	Detection of faults
[101] 2020	Temperature-based thresholding	Pr: 97.6%	Classification of size and severity of faults
[87] 2020	Water filling and temporal tracking algorithms	F1: 72%	Detection of hot spots
[166] 2021	Statistical analysis of temperature of modules	Ac: 96%	Classification into 6 classes of faults
[74] 2021	Robust PCA decomposition and thresholding	F1: 78.23%	Fault mask
[167] 2021	Filtering and probability density functions	-	Fault detection using both visual and aIRT images

Table 7. Summary of methods for detecting PV modules in aIRT using DL and classification algorithms.

[Ref]/Year	Algorithm	Best Results	Detection Type
[168] 2018	U-Net, LinkNet, FPN and Mask R-CNN	Dice: 0.841	Fault mask
		IOU: 0.741
[169] 2019	Hough line transformation, canny operator and Faster R-CNN	F1: 95.15%	Detection of reflections and hot spots
[106] 2018	VGG	Pr: 49.11%	Detection of substring, module and string failures
[32] 2019	DIP and DL	-	Fault mask
[98] 2020	DL	Pr: 95%	Detection of hot spots
[170] 2020	RF, SVM, VGG-16 and MobileNet	Ac: 91.2%	Classification into disconnected substring, patchwork, hot spot, soiling and string problems
[61] 2020	R-CNN	Pr: 91%	Detection of hot spots
[171] 2021	Thresholding, CNN and multi-layer perceptron	Ac: 100%	Detection of hot spots
[50] 2021	DL (ResNet-34) and k-nearest neighbors classifier	AUROC from 73.3% to 96.6%	Fault masks
[172] 2021	DL using a Nadam optimizer	Ac: 66.43%	Classification into 11 anomaly types
[88] 2021	ResNet-50 with ImageNet	Ac: 90%.	Classification into 10 anomaly types
[173] 2021	ICNM and transfer learning	Ac: 97.62%	Detection of bird drops, hot spots, patchwork, disconnected strings and disconnected substrings
[174] 2021	DIP and XGBoosz (algorithm for statistical characteristics of temperatures) as input preparation for a CNN	Ac: 93.8%	Classification into hot spots, PID and disconnected modules
[175] 2021	YOLOv3	Ac: 75%	Classification into 5 fault classes using composites (aIRT and visual images)
[102] 2022	DL	F1: 94.52%	Classification into 5 fault classes using composites

Table 7. Summary of methods for detecting PV modules in aIRT using DL and classification algorithms.

[Ref]/Year	Algorithm	Best Results	Detection Type
[168] 2018	U-Net, LinkNet, FPN and Mask R-CNN	Dice: 0.841	Fault mask
		IOU: 0.741
[169] 2019	Hough line transformation, canny operator and Faster R-CNN	F1: 95.15%	Detection of reflections and hot spots
[106] 2018	VGG	Pr: 49.11%	Detection of substring, module and string failures
[32] 2019	DIP and DL	-	Fault mask
[98] 2020	DL	Pr: 95%	Detection of hot spots
[170] 2020	RF, SVM, VGG-16 and MobileNet	Ac: 91.2%	Classification into disconnected substring, patchwork, hot spot, soiling and string problems
[61] 2020	R-CNN	Pr: 91%	Detection of hot spots
[171] 2021	Thresholding, CNN and multi-layer perceptron	Ac: 100%	Detection of hot spots
[50] 2021	DL (ResNet-34) and k-nearest neighbors classifier	AUROC from 73.3% to 96.6%	Fault masks
[172] 2021	DL using a Nadam optimizer	Ac: 66.43%	Classification into 11 anomaly types
[88] 2021	ResNet-50 with ImageNet	Ac: 90%.	Classification into 10 anomaly types
[173] 2021	ICNM and transfer learning	Ac: 97.62%	Detection of bird drops, hot spots, patchwork, disconnected strings and disconnected substrings
[174] 2021	DIP and XGBoosz (algorithm for statistical characteristics of temperatures) as input preparation for a CNN	Ac: 93.8%	Classification into hot spots, PID and disconnected modules
[175] 2021	YOLOv3	Ac: 75%	Classification into 5 fault classes using composites (aIRT and visual images)
[102] 2022	DL	F1: 94.52%	Classification into 5 fault classes using composites

7.7. Other Applications

Imaging techniques have been employed in some other applications to facilitate the analysis of PV modules. An example is the detection of blurred images that was addressed by Tribak and Zaz [176] with image processing techniques in order to filter frames of videos before employing mosaicking techniques. Similar techniques were used by Shen et al. [177] to correct the angle distortion of IRT images.

8. Discussion

This review has shown that different automatization algorithms, including DIP, DL and classification techniques, have been employed for automating different tasks of the aIRT procedure for inspecting PV power plants. Among the conclusions, this review showed that only a few among the selected studies have assessed two important aspects of the autonomous inspection procedure, namely, the optimization of the flight path (nine papers), and the detection of soiling (eight papers). These two topics are of great importance to increase time efficiency in aIRT and therefore should be further investigated. The latter goal of detecting soiling over PV modules and differentiating it from actual faults of the modules was investigated by some authors, e.g., Dunderdale et al. [170] and Arosh et al. [159], together with the detection and classification of other faults.

For the task of performing the orthomosaicking of aIRT images to facilitate the localization of the faults in the field, four papers employed existing software to perform the task, while ten studies approached the development of algorithms to create the orthomosaic of the PV plant. However, most of the proposed methods are based on DIP techniques; therefore, the resulting mosaic consists of a simple image, without additional GPS information. The correlation of orthomosaic images with GPS coordinates and the identification of modules and strings according to the site nomenclature are areas that require further investigation.

Another approach to the challenge was developed by Wu et al. [60], with the development of an algorithm that detects PV arrays in power plants and performs the automatic correlation with their string identifiers. This is a promising strategy that could also be used to facilitate the localization of detected faults in the field through aIRT. Besides the study carried out by Wu et al. [60], another 20 studies among the selected literature focused on the development of algorithms to detect PV systems and panels in aerial imagery. However, only three of these studies focused on aerial IRT images of the PV plants, obtaining up to 93.16% precision in the results [63]. On the one hand, 18 papers presented the results of developed algorithms for the detection of individual PV modules in aIRT images, of which three of them applied DL techniques. Although the methods are hardly comparable given their different structures for results (i.e., mask, box or line), their different dataset sizes and the different evaluation metrics used, a method that combined many algorithms (DIP, SVM and DL) for detecting PV modules in aIRT images and obtained an F1 score of 98.4% can be highlighted [85]. On the other hand, the worst metrics were obtained with simple DIP filters [86], which although providing fast results with small datasets required for training, are characterized by a lack in generalization. This is important for the replication of the algorithm in images acquired in different conditions and with a different quality. The algorithms proposed by Carletti et al. [87], Xie et al. [89] and Bommes et al. [88] also performed the tracking of the modules in subsequent frames of an aIRT video. This task is of utter importance for the cross-correlation of detected modules and faults, as well as their location in PV plants.

Most of the selected studies have assessed autonomous fault detection and classification in PV plants through visual (12 papers), IRT (22 papers) and aIRT images (43 papers). Among these studies, 35% used DL techniques for the detection or classification of PV faults, with an increase in developed algorithms using CNNs in recent years. Still, DIP-based algorithms also presented high accuracy results, even though most of them use smaller datasets, and therefore their replication in other sets of data is possibly not feasible. The combination of DL or DIP techniques with classifier algorithms was a promising approach in recent studies. In the field, fault detection can either be processed on board, during the UAV flight, or subsequently through a post-processing procedure after the flight. For the first case, the high computational requirements and the processing time of DL are still a challenge, as even in high-performing computers, the processing of a set of images of a large-scale PV power plant (that consists of some gigabytes of data) can take hours when using a DL algorithm. In the same way as in the detection of PV systems and modules, many types of outputs for the algorithms are possible, namely the segmentation of the faults, the detection of damaged modules or even the classification of faults in separate classes. The classes also differ among authors, and these differences represent a great challenge not only for the comparison between studies, but also for the exchange of data, experiences and algorithms among researchers in PV community, which hinders the advancements in this area. The exchange of data to enable the development of larger and more generalized datasets that consider different environmental conditions is also deaccelerated by data protection clauses.

Besides the different result types, the different evaluation metrics (or the lack of them), dataset sizes and image resolutions of the inputs also make the comparison between studies difficult. However, in general, the algorithms with the highest metrics are the ones dedicated to detecting and classifying a few types of faults compared to those that carry out the classification of many classes of faults. This proves that detection and classification of multiple faults is a complex task and further investigation is required. On this subject, the algorithm developed by Bommes et al. [88] can be highlighted for its encouraging results, with an accuracy of 90% in the detection and classification of faults in ten different anomaly types. In summary, to achieve the goal of an entirely autonomous aIRT procedure, advances in some of the tasks related to the technique must be achieved. Even tasks that were already the focus of many research studies, such as the detection and classification of faults, should be further explored to contemplate different types of datasets and conditions. The exchange of data and academic collaborations are fundamental to allow for a fully automatic procedure that not only detects modules and faults on PV modules but also provides information about the type and location of the faults, in a simple and accessible manner, to enable quick remediation measures.

9. Conclusions

This paper has conducted a comprehensive review of the literature for methods of automating different tasks of the aIRT framework of PV power plants, since it is a subject that has been intensely investigated by researchers in recent years. Most of these studies (77 studies) focused on the autonomous fault detection and classification of PV plants in visual, IRT and aIRT images. Among these studies, the use of DL algorithms has provided good results with an accuracy of up to 90% in the detection and classification of faults in 10 different anomaly types detected in module segments extracted from aIRT images. On the other hand, only a few studies have explored the automation of other parts of the procedure of aIRT, such as the optimization of the path planning (nine papers) for the inspection flight, the orthomosaicking of the PV plant (14 studies) that is performed to facilitate the localization of the faults in the field and the detection of soiling, and its differentiation from actual faults on PV modules (eight studies). Algorithms for the detection and segmentation of PV modules were presented in 38 papers and achieved a maximum F1 score of 98.4%.

For the automation of these procedures, different algorithms have been investigated, including DIP filters and methods such as canny edge detection and thresholding; DL algorithms such as Fast R-CNN, ImageNET and VGG16; and other ML-based algorithms used for classification tasks such as SVMs, KNNs and RFs. However, the accuracy, robustness and generalization of the developed algorithms are still the main challenges of these studies, especially when dealing with more classes of faults and the inspection of large-scale PV plants. With the ever-increasing capacity and size of utility-scale PV power plants, reaching the scales of gigawatts and hundreds of hectares, automation is increasingly becoming a matter not only of scientific interest, but also of economic importance. Therefore, the autonomous procedure and classification task must still be explored to enhance the accuracy and applicability of the aIRT method.