A Comparison and Introduction of Novel Solar Panel’s Fault Diagnosis Technique Using Deep-Features Shallow-Classifier through Infrared Thermography

Abstract

Solar photovoltaics (PV) are susceptible to environmental and operational stresses due to their operation in an open atmosphere. Early detection and treatment of stress prevents hotspots and the total failure of solar panels. In response, the literature has proposed several approaches, each with its own limitations, such as high processing system requirements, large amounts of memory, long execution times, fewer types of faults diagnosed, failure to extract relevant features, and so on. Therefore, this research proposes a fast framework with the least memory and computing system requirements for the six different faults of a solar panel. Infrared thermographs from solar panels are fed into intense and architecturally complex deep convolutional networks capable of differentiating one million images into 1000 classes. Features without backpropagation are calculated to reduce execution time. Afterward, deep features are fed to shallow classifiers due to their fast training time. The proposed approach trains the shallow classifier in approximately 13 s with 95.5% testing accuracy. The approach is validated by manually extracting thermograph features and through the transfer of learned deep neural network approaches in terms of accuracy and speed. The proposed method is also compared with other existing methods.

1. Introduction

With the exponential increase in energy demand due to population growth, and the United Nations’ sustainable development goal to facilitate the world with affordable and clean energy in the context of global climate change under the Paris climate accord, much research has been required, since 73% of greenhouse gas emissions are due to the energy sector [1,2,3]. However, the photovoltaic (PV) system’s energy output without stack emissions reduces the dependence on conventional technologies, effectively mitigating greenhouse emissions [4]. This has resulted in higher penetration of PV systems into the main grid globally, as PV, along with wind, is estimated to meet 88% of the global energy demand by 2050 [5]. The global penetration of PV systems and the increasing volume of solar PV installed capacity from 2011 to 2021 are depicted in Figure 1.

In the presence of environmental stresses experienced during its operation, the PV system faces issues like hotspots and faults due to defects arising from natural parameters such as bird droppings, shading, snow, dust, etc. This decreases the PV system’s performance, creates reliability issues, and increases the payback time [4,5,7,8,9,10,11,12,13,14,15]. These pollutants’ accumulation on the PV panel surface reduces photon transmission, in turn reducing the activation energy for valence electrons. The PV cells beneath pollutants operate in the reverse region (higher resistance), restrict the current coming from healthy PV cells due to their series connection, and create localized high-temperature areas known as hotspots [4,5,16]. The existence of hotspots for a longer period may lead to irreversible damage to PV panels, such as wire meltdown, burning, cracks, etc. Therefore, it requires replacing the PV panel with a new one of a similar rating [16]. Around 18% and 23% of defect sources are hotspots and cell cracks, respectively [17].

Therefore, PV system health monitoring and identifying defects are of utmost importance to maintain the system’s performance, reliability, cash flows, and greenhouse gas mitigation potential [4,5,9,18]. The PV system faces issues at the module level (protection breakdown, bypass diode failure, hotspots, wiring, shade, mismatches, etc.) and array level (electrical mismatch, faults, etc.) [18]. Figure 2 shows the protection breakdown of the PV module. However, all the PV system issues are not identifiable by the naked eye, and automation is highly desirable, especially in megawatt projects [4,5]. Therefore, multiple approaches are widely used in the literature to identify and classify based on PV panel health, such as electrical signals (current and voltage characteristics [I–V]), infrared thermographs, electroluminescence, fluorescence, photoluminescence, etc. [4,5,11,19,20,21,22,23]. Monitoring PV systems has its advantages and limitations. For instance, I–V-based measurements are convenient with instant feedback (sensors) capability for PV condition monitoring [17].

However, infrared thermographs are widely used due to their cost-effectiveness, speed, accuracy, low storage requirements, non-invasive approach, etc. [4].

Relevant Previous Studies—PV System Monitoring

Until recently, electrical signal analysis and PV imaging were widely used to identify and classify PV panels based on their faults and health. Since research on fault diagnosis has focused on factors such as accuracy, speed, precision of identified faults and so on, artificial intelligence models (AIM) are widely used, such as backpropagation AIM, generalized regression AIM, probabilistic AIM, radial basis function AIM, etc. [24]. Lin et al. [25] investigated faults caused by line-to-line, open-circuit, and partial-shading conditions, as measured by a decrease in the instantaneous maximum power point current under various conditions. Garoudja et al. [26] utilized probabilistic neural networks on the DC parameters of the PV system to detect and diagnose three different faults. Aziz et al. [11] used irradiance, temperature, short circuit current, open-circuit voltage, PV current, maximum power point current, maximum power point voltage, and maximum power point power and boost converter output as data points to evaluate no fault, line-to-line, open circuit, partial shading, fault in partial shading, and series arc fault cases using fine-tuned AlexNet as a feature extractor and classifier, and pre-trained AlexNet to extract features and classify through support vector machine or random forest.

In contrast, Natsheh et al. [16] used tree hierarchy to model PV systems and utilized fuzzy nonlinear autoregressive networks with exogenous inputs for diagnosis, classification, and source of fault identification. Moreover, Belaout et al. [27] proposed a multi-class adaptive neuro-fuzzy classifier for five PV fault classifications and reduced the dimensions of features to speed up the process. Chen et al. [21] used a random forest ensemble algorithm to diagnose early faults (line-to-line, degradation, open circuit, and partial shading) of PV arrays using the voltage and current of PV strings. Chen et al. [22] proposed using voltage, current, irradiance, and temperature to detect and diagnose faults using a two-dimensional ResNet structure trained by an adaptive moment estimation algorithm.

Building on this, Tuupke et al. [19] presented photoluminescence imaging advantages for photovoltaic applications. Through current-voltage and electroluminescence, Eder et al. [20] investigated the detection proportion for mechanically-induced failures such as glass cracks, solar cells, microcracks, and defects in cell connection systems, as well as their propagation in artificially-created stresses and environmental conditions. Moreover, the failure detection proportion was investigated through ultraviolet fluorescence imaging. Ahmed et al. [4] used infrared thermographs to monitor the health of the PV system through an isolated neural network and used a transfer learning approach to detect five different faults in the PV system. Ali et al. [5] extracted features such as texture, the mean histogram of an oriented gradient, and the local binary pattern of infrared thermographs to classify PV panels into three classes based on their health using shallow classifiers. In addition, Niazi et al. [10] used infrared thermographs to classify PV panels based on health (hotspot, defective, healthy) using texture features, histograms of oriented gradients, and principal component analysis to reduce feature dimensionality and feed the features to a naive Bayes classifier. Niazi et al. [9] used the mean of texture features to classify PV panels as either healthy or defective using a naive Bayes classifier using infrared thermographs.

Moreover, Pierdicca et al. [18] used augmented infrared data to classify PV panels as damaged or healthy using a deep convolutional neural network. Ali et al. [28] used a color image scale-invariant feature transform descriptor to train a shallow classifier k-nearest neighbors (k-NN) to predict the health of PV panels as healthy, faulty, or hotspots. Ahmed et al. [29] converted IR thermographs into feature histograms by extracting SURF features, clustering using k-means, and classifying PV panels into three health-based states. Li et al. [30] proposed the identification of concurrent faults quantitatively through fault parameter extraction from the measured voltage/current curve of a PV array. Alves et al. [23] used CNN to classify anomalies in an unbalanced dataset through data augmentation techniques, which were able to differentiate between 11 different PV panel states.

Concisely, deep and shallow classifiers have their respective applications and limitations, such as operating system requirements, memory, complexity, fast and accurate convergence in multi-dimensional problems, etc. [4,22]. Moreover, relevant feature extraction from images is the most important, which can be done manually or using convolution neural network layers [5,31]. Given the limitations, this study employs a deep pre-trained convolutional neural network’s robust architecture, which is trained to classify images into 1000 different classes to extract deep features of infrared thermographs without backpropagation to reduce the high operating system requirements, execution time, memory, and so on, and fed to shallow classifiers because they require the least training time. Moreover, the proposed approach is compared with classical machine learning and pre-trained transfer-learned deep convolutional neural networks in terms of accuracy and execution time requirements.

The rest of the paper is structured as follows. Section II discusses the design approach, followed by Section III, which represents the features of infrared thermograph visualization. Section IV presents the results, while Section V discusses the approach in light of the results and the conclusion. Finally, the future work is presented in Section VI.

2. Design Approach

The infrared thermographs were obtained on 8 PV strings with 22 PV modules per string, a 42.24 kW PV system in Lahore, Pakistan. On a clear day with 32–40 °C ambient temperature, 6.9 m/s wind speed, and 700 W/m² irradiance level, a horizontally aligned FLIR VUE-Pro 640 thermal camera was used. The thermograph has an 8-bit depth with a spatial resolution of 640 × 512 pixels. The detailed experimental setup is provided by Niazi et al. [10]. The IR dataset was segregated into six classes based on healthy, single-cell, patchwork, bird-dropping, block, and string conditions. Details of conditions are provided at [4,32] and shown in

Table 1. Faults of PV panels.

Condition	Healthy	Bird Drop	Single	Patchwork	String	Block
Thermograph
Grayscale
Schematic (Probable)
Possible Cause	_	Bird drop	Shadow due to vicinal objects, dust, etc.			Undetected hotspot leading to complete failure
Surface Temperature (Approx.)	50–60° Celsius	65 to 80° Celsius				Above 80° Celsius

. Thermographs are converted into grayscale for PV condition visualization by averaging the RGB channels.

However, it is crucial to consider that each condition on the PV panel is due to a specific reason, and their occurrence is due to operational (i.e., environmental) stresses. Thus, the dataset can be imbalanced. For this purpose, a data augmentation approach was utilized to balance the dataset using the shifting and scaling (0°: 5°: 20°) approach [4]. A 330-image balanced dataset was constructed, with each class having 55 images. Afterward, each class dataset was randomly split into training and testing datasets in 80/20 ratios, to ensure proper training and testing of each class with equal visibility. Finally, features of IR datasets (training and testing) were extracted from the 22nd layer of pre-trained GoogleNet, 18th layer of ResNet18, and 18th layer of SqueezeNet neural networks, and shallow classifiers such as trees, ensemble, k-NN, support vector machine (SVM), and naive Bayes (nB) were used. Additionally, to validate the concept, two common and widely-used approaches were used: manual extraction of IR dataset features (such as energy, correlation, homogeneity, histogram of oriented gradient (HOG), RGB, local binary pattern, etc.); training and testing with shallow classifiers; and utilizing pre-trained 22-layer GoogleNet, 18-layer ResNet18, and 18-layer SqueezeNet through transfer learning for comparison.

2.1. Manual Features Extraction

The variables that differ from one class image to the next are referred to as features. The selection of the most prominent, i.e., highly varying characteristics of images, increases the classification accuracy. Infrared training and testing datasets’ feature vectors were calculated separately. To begin, images were pre-processed by removing their backgrounds, and images were transformed into grayscale, contrast improved, resized for uniformity, and an averaging filter was applied to remove image noise. Features, i.e., the mean of HOG and texture features, were calculated (using a grey-level co-occurrence matrix) to form the feature vector. Energy, contrast, and homogeneity calculations are provided in Equations (1)–(3). Afterward, feature vectors were utilized for training with a 5-fold cross-validation approach, and shallow classifiers such as SVM, k-NN, nB, trees, and ensemble were trained. The approach is simplified in Figure 3.

$$\mathrm{Energy}={\sum }_{x,y}^{i-1}{W}_{x,y}^2$$

(1)

$$\mathrm{Contrast}={\sum }_{x,y}^{i-1}{\left|x-y\right|}_{W_{x,y}}^2$$

(2)

$$\mathrm{Homogeneity}={\sum }_{x,y}^{i-1}\frac{W_{x,y}}{1+\left|x-y\right| }$$

(3)

w_x,_y is the pixel value at the pixel’s x and y coordinates, and i is the gray level number.

2.2. Transfer Learning on Pre-Trained Networks

Transfer-learning in neural networks is the re-utilization of trained neural networks for something other than the base problem. Therefore, avoiding isolated neural networks during creation, adjustment, iterative training, tuning, and validation [4]. Moreover, neural networks can be divided into three stages: image input, feature extraction, and classification based on features by adjusting the neurons’ weight iteratively through backpropagation, as shown in Figure 4. Neural networks have a strong architecture that helps them extract the relevant features.

Therefore, pre-trained neural networks with transfer learning are utilized for classification by introducing a new classification layer and a fully connected layer to replace the old classification layer and a fully connected layer to change the expected classes for the neural network.

2.3. Proposed Approach—Neural Network Features and Shallow Classifiers

In this work, neural networks extract features on their own, and for shallow classifiers, features are extracted manually. Iterative training of neural networks has a high execution time and higher accuracy [4], while shallow classifiers’ accuracy depends on extracted features and output classes and has a fast response [4,5,22]. Considering the advantages of both distinct approaches, a new approach based on the fusion of neural network features with shallow classifiers is proposed. In this approach, a pre-trained deep convolutional neural network such as SqueezeNet, GoogleNet, or ResNet18 that is trained on almost one million images and able to differentiate into 1000 different classes is used to extract the features of IR thermograph datasets (training and testing separately) without backpropagation to reduce the execution time, along with the requirement of memory and a strong operating system. Due to their fast-training time, feature vectors are then used to train and test shallow classifiers for multi-dimensional issues, hence utilizing the strong architecture of neural networks and the fast response of shallow classifiers. The proposed approach is presented in Figure 5.

3. Features of Infrared Thermographs—Visualization

Most relevant features are important since they vary significantly from class to class and help achieve higher training and testing accuracy. Moreover, features greatly differ from one dataset to another. In Figure 6, features extracted from deep neural networks and manually (texture and HOG features) are visually represented for ease of understanding. All features are mapped between 0 and 1, where 0 represents the lowest value and is denoted as black, and 1 represents the highest value and is denoted as white.

Overall, there are six classes, which are represented by each figure, and the higher the color contrast vertically, the better the selection of the most varied and appropriate features between images and classes. All images are resized to 500 × 500 for uniformity. ResNet18 has higher color contrast vertically, and manual extraction has the least contrast vertically. The last column of manual features represents six different classes encapsulated in a red rectangle (see Figure 6).

4. Results

In this section, the results are presented, which were obtained on MATLAB 2021a with 7th-generation Core i7, 16 GB of RAM, an NVIDIA GeForce GTX 1060, a 1.5 TB SSD, and 64-bit operating system specifications. Shallow classifiers were trained and validated using 5-fold cross-validation approaches.

4.1. Manual Features and Shallow Classifiers

Different combinations of manual features extracted from IR training and testing datasets were created. Finally, the mean of HOG and texture features, i.e., 12 energy, 12 contrast, and 12 homogeneities, were used. A total of 37 feature-based vectors are used, while the 38th column stores the six conditions of PV panels. Among five shallow classifiers, naive Bayes resulted in the least accurate trained model with 50.8% accuracy, while multi-class SVM resulted in 83% accuracy with a manual feature vector. The true identification rate (TPR) of bird drops was the lowest of all classes, owing to the small area covered by each drop. In contrast, a block’s positive predicted value (PPV) was highest due to a third area highlighted in IrT since texture features were strong and easily separable. Furthermore, the k-NN model was trained quickly and had the highest online testing capacity of 5000 observations per second. The results provided in

Table 2. Manual features-based shallow classification.

Classifier	Tree				Ensemble				nB				KNN				SVM
Class	TPR	FNR	PPV	FDR	TPR	FNR	PPV	FDR	TPR	FNR	PPV	FDR	TPR	FNR	PPV	FDR	TPR	FNR	PPV	FDR
Bird Drop	54.5	45.5	46.2	53.8	36.4	63.6	50	50	36.4	63.6	44.4	55.6	81.8	18.2	75	25	63.6	36.4	77.8	22.2
Block	81.8	18.2	100	0	100	0	100	0	54.5	45.5	60	40	100	0	100	0	81.8	18.2	100	0
Healthy	72.7	27.3	88.9	11.1	72.7	27.3	100	0	72.7	27.3	44.4	55.6	81.8	18.2	90	10	81.8	18.2	100	0
Patchwork	63.6	36.4	58.3	41.7	81.8	18.2	69.2	30.8	9.1	90.9	100	0	90.9	9.1	83.3	16.7	100	0	84.6	15.4
Single	90.9	9.1	76.9	23.1	100	0	78.6	21.4	90.9	9.1	52.6	47.4	100	0	91.7	8.3	100	0	68.8	31.2
String	63.6	36.4	70	30	72.7	27.3	66.7	33.3	45.5	54.5	55.6	44.4	81.8	18.2	100	0	72.7	27.3	80	20
Training Accuracy (%)	64				70.1				50.8				79.5				83.0
Training Time (s)	5.78				5.84				13.4				1.55				3.49
Prediction Speed (obs/s)	2100				1300				350				5000				2400
Testing Accuracy (%)	71.2				77.3				51.5				89.4				83.3

, incorporating TPR, false-negative rate (FNR), PPV, and false discovery rate (FDR), are based on testing accuracy.

4.2. Transfer Learning-Based Classification

Pre-trained deep convolutional neural networks such as GoogleNet, SqueezeNet and ResNet18 were utilized using the transfer-learning approach to monitor the six PV system conditions. Models were trained using GPUs to reduce the execution time for neural network training and validation.

Table 3. Training options.

Solver	Stochastic Gradient Descent with Momentum
Initial Learn Rate	0.001
Epochs	150
Execution Environment	Multi-GPU
Shuffle	Every epoch
Momentum	0.9

shows the training options used for transferring learning-based classifications. The initial learner’s rate was kept slow for better learning. Neuronal weights of pre-trained neural models trained on different classification problems were re-adjusted after backpropagation. Because neural networks are designed to mimic the human brain, they are learned on the PV dataset with high training accuracy. Moreover, in the testing phase of trained neural networks, the healthy class TPR was lowest, resulting in a considerable FNR. Moreover, in the presence of GPUs, SqueezeNet was the fastest to converge. Results are provided in

Table 4. Transfer-learned deep neural networks for PV panels condition monitoring.

Classifier	GoogleNet				SqueezeNet				ResNet18
Class	TPR	FNR	PPV	FDR	TPR	FNR	PPV	FDR	TPR	FNR	PPV	FDR
Bird Drop	100	0	100	0	100	0	100	0	100	0	100	0
Block	100	0	100	0	90.9	9.1	100	0	100	0	100	0
Healthy	81.8	18.2	100	0	90.9	9.1	100	0	81.8	18.2	100	0
Patchwork	100	0	100	0	100	0	91.7	8.3	100	0	100	0
Single	100	0	84.6	15.4	100	0	91.7	8.3	100	0	84.6	15.4
String	100	0	100	0	100	0	100	0	100	0	100	0
Training Accuracy (%)	100				100				100
Training Loss	0.0038				0.0053				0.0038
Validation Accuracy (%)	96.3				96.3				100
Validation Loss	0.099				0.36				0.036
Testing Accuracy (%)	96.97				96.97				96.97
Execution Time (s)	985				196				229

, while TPR, FNR, PPV, and FDR are based on testing accuracy.

4.3. Fusing Neural Networks with Shallow Classifiers—A Proposed Approach

In this section, features were extracted from a pre-trained network such as GoogleNet, SqueezeNet, and ResNet18 because of their strong feature extraction approaches. Afterward, shallow classifiers were trained and tested on feature vectors. TPR, FNR, PPV, and FDR are based on testing accuracy.

4.3.1. Features through ResNet18 and Shallow Classifier

ResNet18 is an 18-layered deep neural network that has been pre-trained. ResNet18 was used to extract the features of IR thermographs without backpropagation. A total of 512 features were extracted from each thermograph, and the 513th column was added, which stored the condition of the PV panel. Following that, training thermographic image-based features vectors were used to train five shallow classifiers using a 5-fold cross-validation approach to avoid over-fitting, and a test thermographic image-based features vector was used to test the accuracy of the trained classifier.

Results were analyzed based on TPR, FNR, PPV, FDR, training time, prediction speed (specifically for online testing of images), training accuracy, and testing accuracy. Multi-class SVM converged in approximately 13 s, resulting in 97% training accuracy. TPR of the healthy class was the lowest, same as transfer-learned ResNet18. However, testing the new dataset resulted in 95.5% accuracy. The detailed results are provided in

Table 5. ResNet18-based features calculation and shallow classification.

Classifier	Tree				Ensemble				nB				KNN				SVM
Class	TPR	FNR	PPV	FDR	TPR	FNR	PPV	FDR	TPR	FNR	PPV	FDR	TPR	FNR	PPV	FDR	TPR	FNR	PPV	FDR
Bird Drop	81.8	18.2	81.8	18.2	90.0	9.1	100	0	90.9	9.1	100	0	100	0	100	0	100	0	100	0
Block	100	0	100	0	100	0	100	0	81.8	18.2	100	0	100	0	100	0	90.9	9.1	100	0
Healthy	81.8	18.2	81.8	18.2	90.9	9.1	100	0	100	0	73.3	26.7	90.9	9.1	100	0	81.8	18.2	100	0
Patchwork	81.8	18.2	90	10	100	0	100	0	100	0	91.7	8.3	100	0	100	0	100	0	100	0
Single	81.8	18.2	69.2	30.8	100	0	84.6	15.4	90.9	9.1	90.9	9.1	100	0	91.7	8.3	100	0	78.6	21.4
String	81.8	18.2	90	10	100	0	100	0	81.8	18.2	100	0	100	0	100	0	100	0	100	0
Training Accuracy (%)	72.3				35.6				80.3				95.5				97
Training Time (s)	2.32				11.47				111.56				4.97				12.89
Prediction Speed (obs/s)	1700				1100				46				430				480
Testing Accuracy (%)	84.8				97				90.9				98.5				95.5

.

4.3.2. Features through GoogleNet and Shallow Classifier

GoogleNet, a pre-trained deep neural network, has a 22-layered structure and extracts the features of IR images without iterative adjustment of neuron weights, resulting in a reduction in training time and higher operating system requirements. A total of 1024 features were extracted from each thermograph, and the 1025th column was added, storing the PV panel condition among six health states. Afterward, features vectors based on training thermographic images were used to train five shallow classifiers, and a tested thermographic-based features vector was used to test the accuracy of the trained classifier. Results were analysed based on TPR, FNR, PPV, FDR, training time, prediction speed (specifically for online testing of images), training accuracy, and testing accuracy. As with transfer-learned, pre-trained GoogleNet, activations of GoogleNet suffered heavily from healthy class differentiation. Shallow classifier utilization versus deep training results in less accurate models when compared to transfer-learned models. Multi-class SVM resulted in 97% and 95.5% training and testing accuracy, respectively. The healthy class was mainly misclassified as having a single-cell defect. The detailed results are provided in

Table 6. GoogleNet-based features calculation and shallow classification.

Classifier	Tree				Ensemble				nB				KNN				SVM
Class	TPR	FNR	PPV	FDR	TPR	FNR	PPV	FDR	TPR	FNR	PPV	FDR	TPR	FNR	PPV	FDR	TPR	FNR	PPV	FDR
Bird Drop	90.9	9.1	90.0	9.1	100	0	91.7	8.3	100	0	19	81.8	100	0	100	0	100	0	100	0
Block	90.9	9.1	100	0	90.9	9.1	90.9	9.1	18.2	81.8	100	0	90.9	9.1	100	0	90.9	9.1	100	0
Healthy	45.5	54.5	71.4	28.6	81.8	18.2	100	0	0	100	0	100	81.8	18.2	90	10	81.8	18.2	100	0
Patchwork	90.9	9.1	71.4	28.6	100	0	100	0	0	100	0	0	100	0	100	0	100	0	100	0
Single	72.7	27.3	61.5	38.5	100	0	84.6	15.4	0	100	0	0	100	0	78.6	21.4	100	0	73.3	26.7
String	81.8	18.2	81.8	18.2	90.9	9.1	100	0	36.4	63.6	100	0	90.9	9.1	100	0	90.9	9.1	100	0
Training Accuracy (%)	73.9				77.3				35.2				96.2				96.2
Training Time (s)	4.1				24.39				201.04				6.82				7.5
Prediction Speed (obs/s)	620				430				24				230				380
Testing Accuracy (%)	78.8				93.9				25.8				93.9				93.9

.

4.3.3. Features through SqueezeNet and Shallow Classifier

SqueezeNet, a pre-trained deep neural network, has an 18-layered structure and extracts the features of IR images without iterative adjustment of neuron weights, resulting in a reduction in training time and lower operating system requirements. A total of 1000 features were extracted from each thermograph, and the 1001st column was added, which stored the condition of the PV panel among six fault states. Afterward, feature vectors based on training thermographs were used to train five shallow classifiers, and a test thermograph-based feature vector was used to test the accuracy of the trained classifier. Results were analyzed based on TPR, FNR, PPV, FDR, training time, prediction speed (specifically for online testing of images), training accuracy, and testing accuracy. TPR revealed that healthy was frequently misclassified as a primarily single cell. However, multi-class SVM performed better compared to the rest of the classifiers and achieved 95.8% and 93.9% accuracy in training and testing, respectively, with a 7.38 s training time. The detailed results are provided in

Table 7. SqueezeNet-based features calculation and shallow classification.

Classifier	Tree				Ensemble				nB				KNN				SVM
Class	TPR	FNR	PPV	FDR	TPR	FNR	PPV	FDR	TPR	FNR	PPV	FDR	TPR	FNR	PPV	FDR	TPR	FNR	PPV	FDR
Bird Drop	100	0	100	0	100	0	84.6	15.4	81.8	18.2	100	0	90.9	9.1	100	0	100	0	100	0
Block	100	0	100	0	100	0	100	0	90.9	9.1	100	0	100	0	100	0	90.9	9.1	100	0
Healthy	81.8	18.2	69.2	30.8	72.7	27.3	100	0	54.5	45.5	100	0	63.6	36.4	100	0	81.8	18.2	100	0
Patchwork	100	0	73.3	26.7	81.8	18.2	90	10	90.9	9.1	83.3	16.7	100	0	91.7	8.3	100	0	100	0
Single	63.6	36.4	77.8	22.2	100	0	84.6	15.4	100	0	64.7	35.3	100	0	64.7	35.3	100	0	78.6	21.4
String	63.6	36.4	100	0	90.9	9.1	90.9	9.1	90.9	9.1	83.8	16.7	81.8	18.2	100	0	90.9	9.1	90.9	9.1
Training Accuracy (%)	76.9				26.5				84.1				95.8				95.8
Training Time (s)	4.45				18.45				171.83				6.15				7.38
Prediction Speed (obs/s)	640				400				28				240				370
Testing Accuracy (%)	84.8				90.9				84.8				89.4				93.9

.

5. Discussion and Comparison with Already Existing Techniques

In the literature or previously introduced techniques, PV system faults are widely monitored using electrical signals and non-invasive thermographs [10,16].

Moreover, shallow classifiers, such as SVM, nB, KNN, etc., are trained on manually-extracted input features to identify the PV system’s fault [5,9,10]. Moreover, deep neural network classifiers, such as convolutional neural networks, etc., were utilized to monitor PV system faults by extracting input features through different layers and iteratively adjusting neurons’ weights through backpropagation to assign probabilities and weights for each fault identification [4,11,18].

However, shallow classifiers’ accuracy is greatly limited by the extraction of the most relevant features of the input and the dimensionality of condition monitoring. While their training and validation times are short, i.e., approximately seconds, they have fewer memory and operating system requirements, as witnessed in the literature [5]. On the other hand, because of their non-linearity, deep neural networks work very well with multi-dimensional classification problems and achieve higher accuracy. They extract the input features through their architectural layers and adjust the neurons’ weight through backpropagation. Therefore, neural networks have comparatively higher memory requirements and execution times for training and validation, and work well on advanced operating systems (GPUs). Considering the advantages and limitations of shallow classifiers and neural network-based classifiers, a fusion of neural networks, i.e., extraction of input features without backpropagation (in approximately a couple of seconds) and training of shallow classifiers such as KNN, SVM, nB, etc., is proposed.

In comparison to the least accurate manual feature extraction approach and training through shallow classifiers, and the high memory, high execution time, and strong operating system requirements of deep neural network classifiers, the proposed approach extracts, trains, and validates shallow classifiers in seconds while achieving good testing accuracy and is capable of processing hundreds of observations per second. For instance, 330 infrared thermographs with six PV systems monitored through manual feature extraction (texture and HOG) and using SVM resulted in 83.3% testing accuracy after training and validation in approximately 4 s. In contrast, neural networks trained with transfer learning achieved 96.97% testing accuracy after 3 min and 49 s of GPU training and validation. In contrast, the proposed approach resulted in 95.5% testing accuracy through SVM, while training and validation took approximately 13 s using the CPU.

Table 8. SVM testing accuracy and training time using different features extraction approach.

Features Extraction Approach	Testing Accuracy (%)	Training Time (Sec)
Manual	83.3	3.5
ResNet18	95.5	12.9
GoogleNet	93.9	7.5
SqueezeNet	93.9	7.4

displays the testing accuracy as well as the training and validation times of SVM on various feature extraction approaches.

6. Conclusions

To maintain PV system output, monitoring the PV system is of utmost importance to keep the performance, emissions mitigation potential, and payback time at an optimal level. PV panel condition is monitored by keeping track of parameters such as current, voltage, irradiance, surface temperature, etc., of PV panels and strings through electrical signal monitoring or non-invasive thermographs and identifying conditions using shallow and deep classifiers.

However, considering the training and validation time, the method of most relevant input feature extraction, accuracy, low operating system requirements, multi-dimensionality issues, etc., a new fused approach was proposed to identify the six different faults of PV panels by extracting input features through deep neural networks and the utilization of shallow classifiers. The proposed approach, trained on 264 balanced IR thermographs and tested on 66 balanced IR thermographs, proved that the proposed approach can be trained 15 times faster using CPUs than deep neural networks using GPUs, and classifies PV panel faults with 95.5% accuracy on a new dataset that it has not experienced during training. This results in good accuracy compared to manual extraction-based shallow classifiers with 83.3% accuracy.