Stain Detection Based on Unmanned Aerial Vehicle Hyperspectral Photovoltaic Module

Abstract

Solar power generation has great development potential as an abundant and clean energy source. However, many factors affect the efficiency of the photovoltaic (PV) module; among these factors, outdoor PV modules are inevitably affected by stains, thus reducing the power generation efficiency of the PV panel. This paper proposes a framework for PV module stain detection based on UAV hyperspectral images (HSIs). The framework consists of two stain detection methods: constrained energy minimization (CEM)-based and orthogonal subspace projection (OSP)-based stain detection methods. Firstly, the contaminated PV modules are analyzed and processed to enhance the data’s analytical capability. Secondly, based on the known spectral signature of the PV module, stain detection methods are proposed, including CEM-based stain detection and OSP-based stain detection for PV modules. The experimental results on real data illustrate that, in comparison with contrasting methods, the proposed method achieves stain detection results that closely align with known stain percentages. Additionally, it exhibits a fitting curve similar to the more maturely developed electroluminescence-based methods currently in use.

1. Introduction

At present, people’s demand for energy is increasing, and the problem of energy shortage follows. Seeking green and new energy for sustainable development has become a key issue for all countries. Solar energy, as a clean energy source, has garnered significant research attention [1]. In 2022, the energy prices surged due to the impact of the Ukrainian–Russian war, leading countries to pay greater attention to renewable energy. Consequently, the photovoltaic market is experiencing a boom [2]. Global photovoltaic (PV) module demand reached 280 W in 2022, exhibiting a staggering growth rate of 56.5% compared with 2021 [3]. Distributed PV power generation has become a crucial technology actively promoted by developed countries, and China is also actively promoting the development of distributed PV construction. Since June 2021, the National Energy Administration has organized nationwide initiatives to promote the development of rooftop distributed photovoltaics on a county-wide scale, addressing electricity consumption issues in both rural and industrial areas [4]. It is anticipated that by 2050, the total installed capacity of PVs in China will reach 1300 GW [5]. Concurrently, InfoLink predicts that global demand will further increase by 21.6% to reach 338 GW in 2023, driven by the ongoing energy transition efforts of various countries. PV plans in more and more countries also indicate that solar PV power generation is increasingly assuming a pivotal role in the energy sector. However, factors such as production, daily activities, environmental conditions, and weather can lead to the accumulation of dust on the PV panel, which is inevitable. Therefore, the study of dust and other factors that cause the reduction of PV power generation efficiency has received more and more attention [6]. According to the research findings in [7], the output power of a clean PV module is at least 15% higher than that of a PV module with ash accumulation in the same environment. The output power of a PV module decreases accordingly with the increase in ash accumulation. Additionally, dust on a PV module can result in leakage current and power attenuation, significantly decreasing the solar energy conversion efficiency of the PV module. Wang et al. [8] studied the effect of ash accumulation on the leakage current and attenuation life of the PV module. Jiang et al. [9] conducted laboratory simulations and experiments using solar simulators and test chambers to study the efficiency degradation of a PV module caused by dust deposition under different conditions. This work aims to study the dust accumulation on different types of solar PV modules and the corresponding efficiency degradation. The influence of dust on solar panel efficiency and factors affecting dust deposition on PV panels are summarized in [10]. In [11], three factors affecting the efficiency of PV panels are considered simultaneously: dust, humidity, and air speed; moreover, it is shown that they also interact with each other. In addition, in [12,13], the influence of different types of dust on the efficiency of PV power generation is studied. The above literature, from the perspectives of power generation efficiency and PV lifespan, demonstrates the impact of stains on the efficiency of PV power generation. Furthermore, airborne dust deposition on outdoor PV modules can reduce the transmittance of solar cell glass [14,15]. For example, the study in [16] investigated the fitting formula of dust deposition and the reduction in light transmittance. Kaldellis and Kokala [17] pointed out that the solar radiation intensity significantly influences PV panel output. However, the presence of soil and dirt can lead to a decrease in efficiency of potentially up to 15%. Conversely, the presence of dust, which leads to attenuated solar exposure due to reflection, reduces the efficiency of PV panels. Some studies suggest that the high concentration of airborne dust deposition may reduce the transmittance of battery glass, leading to a significant decrease in solar energy conversion efficiency [18]. These studies aim to investigate the impact of different dust concentrations on power generation efficiency. Nurjanah et al. [19] studied the effect of dust characteristics and deposition density on PV efficiency. The results showed that fine particulate dust has a greater impact on the power generation performance of PV panels than coarse ones. Amr [20] studied the influence of a dust layer on the light beam transmittance of a PV module glass surface in experimental and mathematical ways. The results revealed that as the number of dust particles on the glass sample increased, the reflection at longer wavelengths was much greater than at shorter wavelengths. Similarly, a dust concentration and energy conversion efficiency model was proposed in [21] to evaluate the impact of dust accumulation on PV power generation performance. The impact of dust on the efficiency of a PV module can be analyzed from two perspectives: the diffusion of dust in the atmosphere and the deposition of dust on the surface of the panel. Additionally, electrical characteristics, such as its current–voltage curve and power–voltage curve, vary under different radiation levels. With increased insolation on the PV module surface, more power can be generated, thus improving efficiency [22]. Additionally, the type and particle size of stains are also related to their impact on PV panel power generation. In [23], a comprehensive physical model was proposed to predict the influence of dust deposition on the transmittance of PV modules. The results showed that the effect of deposition increases significantly with the increase in particle concentration and particle size, but decreases with the change in particle transparency. These studies collectively contribute to a comprehensive understanding of how dust affects PV module performance and highlight various factors that can influence efficiency.

The above studies show that the existence of dust accumulation affects the output efficiency of PV modules. It is necessary to study the relationship between dust accumulation and the power generation efficiency of PV modules. On the one hand, it differs from the method of modeling by exploring the relationship between dust accumulation density and module efficiency through simulation experiments. Dust density is often unknown in real scenarios. Additionally, the dust accumulation situation of an outdoor PV module is affected by a variety of natural factors, which may differ from the simulated experimental scenario. On the other hand, existing EL-based stain detection technologies, such as image entropy-based or variance-based methods, achieve satisfactory detection results. However, an inherent limitation is that EL requires a power supply for luminescence, demanding high imaging requirements and offering a limited imaging range. In contrast, UAV remote sensing does not necessitate specific imaging conditions and provides a broader imaging range. Furthermore, hyperspectral imaging technology combines imaging and spectral technologies to capture both the two-dimensional geometric space and one-dimensional spectral information of the target. It is based on narrow-band image data technology and results in continuous and narrow-band data with high spectral resolution, and it finds numerous applications in various detection fields. Several hyperspectral image (HSI)-based detection techniques have been successively proposed [24,25,26], among which the method in [25,26] has been widely reported due to its simplicity and effective detection capability [25,26,27]. Therefore, this study aims to leverage the advantages of hyperspectral remote sensing images, which include a wide imaging range, no constraints on imaging conditions, and high spectral resolution. Simultaneously, it aims to investigate stain detection in PV images under conditions where stain accumulation density and types are unknown, utilizing the effective spectral feature detection capabilities of OSP and CEM. This paper is dedicated to the development of a stain detection method based on UAV remote sensing images, aiming to achieve detection capabilities comparable with EL-based technologies. At the same time, it strives to overcome the limitations of EL-based stain detection methods, facilitating unrestricted detection in large-scale scenarios.

The contributions of this paper are as follows:

(1)

A stain detection framework based on an HSI PV module is proposed to address the challenges posed by various type of stains, large stains, and unknown spectral signatures. The framework consists of two detection methods: constrained energy minimization-based and orthogonal subspace projection-based methods.

(2)

The relationship between the amount of stains accumulated by the spectral module and the power generation efficiency of the PV module is modeled. The results demonstrate that the developed method achieves comparable performance with the electroluminescence (EL) image-based stain detection technology.

2. Methods

The overall process of the proposed method is illustrated in Figure 1. It can be summarized into two main modules: A. removing interference and B. stain detection. In the A. removing interference stage, considerations are given to factors such as gridlines and illumination that may cause spectral distortion affecting the detection of the target spectrum. In the B. stain detection stage, two methods for PV module stain detection are introduced, namely the CEM-based method (B₁) and the OSP-based method (B₂). Specifically, after acquiring the UAV HSI of the PV, irrelevant interference is removed from the data to be detected. Then, the PV module stain detection method based on HSIs is executed to obtain the detection value and detection result image. Finally, the detection value and percentage of stains are modeled.

2.1. Remove Irrelevant Interference

For outdoor UAV HSIs, the quality of the collected images are affected by environmental illumination, image distortion, sensor instability, and the structure of the PV module itself. As a result, the precision and accuracy of subsequent processing are compromised. Therefore, it is necessary to preprocess the data before processing.

2.1.1. Remove Gridlines

Typically, the UAV-acquired data of PVs will contain a solar panel and gridlines, where the solar panel is usually in the shape of a square or rectangle and is typically made of semiconductor materials (e.g., silicon) that are highly absorbent of solar light, and its surface is dark in color. The gridlines are the conductive lines located on the surface of the solar panels; these lines form a grid-like structure [28] that connects the solar panels and is used to collect and conduct the current generated by the solar panels, and it is usually white or silver. Figure 2 is the experimental setup during the data collection process and the panoramic view of the captured data. The orange boxes indicate the polluted areas, while the green boxes represent the cleaned areas. Due to the structural characteristics of PV modules, gridlines are inevitably present on the surface, as shown by the white lines in Figure 2a. They are made of materials that are different from the PV panel, typically exhibiting brighter pixel values. As a result, direct detection of the PV module can significantly interfere with the effectiveness of the detection algorithm, leading to unsatisfactory stain detection results. Therefore, it is necessary to mitigate this effect before performing the detection.

In this stage, two ways are employed to remove the gridlines of the acquired PV module: (1) a threshold-based cutting method and fabrication mask and (2) physical-based cutting. Firstly, a target image for detection is randomly chosen, and a single-band image is extracted from it. By comparing the pixel intensity values of the gridlines with the PV module in the single-band image, the threshold-based segmentation method is applied to remove the gridlines. A single-band PV module image without gridlines is obtained. Because each band of an HSI corresponds to the reflectance of the same surface within a specific wavelength interval, the spatial structure remains consistent across all bands. However, due to the varying reflectivity in different bands, the pixel intensity differs among bands. Therefore, a fixed threshold cannot be applied uniformly to remove the gridlines in different bands. To achieve a more accurate and efficient gridlines removal, a mask is generated with the single-band PV module obtained after removing the gridlines. Then, the mask image is used to remove the gridlines from the remaining bands. Figure 3 illustrates the visual results before and after gridlines removal.

Although the above method can remove most of the gridlines, it also removes stains with high-intensity values. To address this issue, the consideration of physical-based cutting is further explored for removing the gridlines. Figure 4 illustrates the visualization before and after the physical removal process.

2.1.2. Data Correction

In addition to the effects of gridlines, the spectral quality of outdoor-acquired UAV HSIs can also be affected by factors such as ambient illumination, which in turn affects the precision and accuracy of subsequent processing. Therefore, after removing the gridlines from the HSI PV module, the spectral correction of the PV module is required in the next step. In this paper, the PV module is spectrally corrected based on the reference plate acquired in the HSI PV module acquisition. Figure 5 shows the visualization results of the PV modules with and without correction.

2.2. Method Introduction

Because there are many types of stains (generally including dust (ceramic/mud), metal, oil, and miscellaneous) and the degree of pollution varies, it is difficult to determine the spectrum of pollutants. However, the spectrum of clean PV modules can be obtained in advance. Therefore, two stain detection methods are designed based on the HSI PV module using the spectrum of the clean PV module.

2.2.1. Stain Detection of HSI PV Module Based on CEM

Given a set of observation samples $O=\{r_1,r_2,\cdot \cdot \cdot ,r_N\}$, $r_i$ = ${\{r_{i1},r_{i2},\cdot \cdot \cdot ,r_{iL}\}}^T$ is the sample vector of $1\times L$,$L$ is the number of bands of the HSI, $1\le i\le N$, and N is the total number of pixels in each band. Further assume the target endmembers $d={\{d_1,d_2,\cdot \cdot \cdot ,d_L\}}^T$ and that the size of d is the same as r. The goal of CEM is to design a filter w obeying the constraint $d^{T}w=w^{T}d=1$. More specifically, $y_i$ represents the output of the finite impulse response (FIR) filter generated by the input $r_i$. Thus, $y_i$ can be expressed as

$$y_i={\displaystyle \sum _{l=1}^L{w}_l{r}_{il}={(w)}^T{r}_i=r_i{}^{T}w}$$

(1)

The average energy output of the filter is given by the following equation:

$$\begin{array}{l}\left(\frac{1}{N}\right){\displaystyle \sum _{i=1}^N{y}_i^2=\left(\frac{1}{N}\right)}{{\displaystyle \sum _{i=1}^N\left(r_i^{T}w\right)}}^2\\ =w^T\left[\left(\frac{1}{N}\right){\displaystyle \sum _{i=1}^N{r}_i{r}_i^T}\right]w\\ =w^{T}Rw\end{array}$$

(2)

where $R=\left(1/N\right)\left[{\displaystyle \sum _{i=1}^N{r}_i{r}_i^T}\right]$ is the autocorrelation sample matrix of the image. CEM can solve the following linearly constrained optimization problem:

$$\underset{w}{\min }\left\{w^{T}Rw\right\}\mathrm{subject}\mathrm{to}{d}^{T}w=w^{T}d=1$$

(3)

The optimal solution for (3) is given as follows:

$$w^{CEM}=\frac{R^{-1}d}{d^T{R}^{-1}d}$$

(4)

Use the optimal weights $w^{CEM}$ specified by (4); the filter called CEM was derived in [25], represented by ${\delta }^{CEM}\left(r\right)$.

$${\delta }^{CEM}\left(r\right)={\left(w^{CEM}\right)}^{T}r={\left(\frac{R^{-1}d}{d^T{R}^{-1}d}\right)}^{T}r=\frac{d^T{R}^{-1}r}{d^T{R}^{-1}d}$$

(5)

Because the spectrum of the stains to be detected is unknown, direct application of CEM for detecting stains in PV modules is not feasible. Fortunately, the spectral curve of the region that is not of interest in the HSI PV module can be obtained. Therefore, to address the issue of the unknown target spectrum d, a stain detection method for the HSI PV module based on CEM is designed in this paper.

After eliminating factors such as gridlines and spectral distortion that affect the detection, it is assumed that the HSI only contains PV modules and stains. Then, by taking the clean PV module spectrum as d, execute ${\delta }^{CEM}\left(r\right)$. Then, the following formula can be utilized:

$${\zeta }^{cem}(r)=1-\frac{d^T{R}^{-1}r}{d^T{R}^{-1}d}$$

(6)

Then, the region of interest can be calculated using the above equation ${\zeta }^{cem}(r)$.

Although the above methods effectively suppress the background, due to R counting a part of the stain, the above methods have a suppression effect on the stain. To achieve a more accurate detection of unknown stains, this paper further proposes a second scheme by combining OSP on the basis of the above section.

2.2.2. Stain Detection of HSI PV Module Based on OSP

Because the target spectrum d is unknown, after removing the interfering factors in the image, all materials except for the PV module are considered as stains. However, this method is not ideal, and a further improvement of the method is considered. Another way is to treat the PV module as the signal that needs removal, and a space is constructed based on it. The second method is then performed on the orthogonal space of this constructed space.

Given a three-dimensional HSI $X={\left\{r_i\right\}}_{i=1}^n$, $r_i={\left(r_{i1},r_{i2},\dots ,r_{iL}\right)}^T$, $n$ is the total pixels and $L$ is the total spectral channels. The objective function of RAD [26] is given by

$${\delta }^{\mathrm{RAD}}(r)=r_n{}^T{R}^{-1}{r}_n$$

(7)

where $r_n$ is spectral signature of the current pixel to be detected, $R=\frac{1}{n}{\displaystyle \sum _{i=1}^n{r}_i}{r}_i{}^T$ is autocorrelation matrix of the measured image pixels, and $r_i$ is the spectral vector of $i$ pixel. The correlation matrix is obtained according to the spectral correlation of all pixels preceding the current pixel. By considering the PV module as a signal that needs to be removed, it is used to construct a space in which the stains are detected in the orthogonal space. Part of the stain is counted in (7), which is equivalent to suppressing a part of the stains. Therefore, the consideration involves combining OSP to design the second method. Let U = $\left[m_1{m}_2,\dots ,m_{p-1}\right]$ be the unwanted material spectral signature matrix composed of $m_1$,$m_2$,…,$m_{p-1}$, then:

$${\widehat{\alpha }}_p^{ROSP}\left(r\right)=r^{T}P\stackrel{\perp }{U}r$$

(8)

Here,

$$P\stackrel{\perp }{U}=I-U{U}^{\#}=I-U{\left(U^{T}U\right)}^{-1}{U}^T$$

(9)

where $U^{\#}$ is the pseudoinverse of U, and it is obtained by $U{\left(U^{T}U\right)}^{-1}{U}^T$.

3. Experimental Results and Analysis

In order to validate the effectiveness of the developed method in detecting stains in PV images, it was applied to multiple UAV aerial HSI PV images, and the detection value (DV) and corresponding percentage of stain (PS) were modeled. At the same time, the proposed detection framework was compared with the unmixing-based method (FCLS) [29], the deep learning-based (DL) method BLTSC [30], and anomaly detection methods (FEBPAD [31] and CRDBPSW [32]) to demonstrate the superiority of the proposed framework. In addition, a comparison was performed with an EL image-based method, which has been maturely developed at present, to further verify the performance of the presented method. For a more intuitive quantitative comparison of the results among different methods, we use bold font and underline to mark the best and second-best values, respectively.

The dataset used in this paper was acquired by the HEADWALL Hyperspec III sensor at heights of 15, 30, and 50 m, with a vertically downward shooting angle. The data size for the first batch is 1640 $\times$ 1251 $\times$ 270, and for the second batch, it is 8172 $\times$ 2684 $\times$ 270. As depicted in Figure 2, the focus was on processing the PV panel area during the experiment to minimize interference factors in the data processing.

3.1. Relationship between the Detection Results of Different Methods and Percentage of Stain

Figure 6 and Figure 7 displays the detection results obtained using different methods. The detection colors corresponding to different values are shown in the color bar in Figure 6. In this bar, colors closer to red indicate detection values closer to one, representing a higher amount of detected stain. Conversely, as colors approach blue, the corresponding detection values get closer to zero, indicating a lower amount of detected stain. Visually, the proposed method exhibits distinguishable visual effects compared with FCLS, BLTSC, EFBPAD, and CRDBPSW. The presented method can effectively detect the areas with and without cleaning the PV module, where the distribution of these areas is visible. Figure 8 presents a curve comparison between the values obtained from different detection results and the corresponding percentage of stains. Additionally, to analyze the relationship between the detection values of different methods and the percentage of stains more effectively, a quantitative approach was adopted using the spectral angle measure (SAM) in

Table 1. The SAM between the DV and PS of different methods.

	CEM	OSP	FCLS	DL	FEBPAD	CRDBPSW
SAM	0.6439	0.6446	0.8986	0.9521	1.2172	NaN

to calculate the distance between them. Among them, bold values are used to indicate the best results, and underlined values represent the second-best results. A smaller SAM value indicates that the detection results are closer to the stain percentage, indicating better stain detection performance for the corresponding algorithm. From Table 1, it can be observed that CEM exhibits the lowest SAM value, which indicates that it detects best, followed by OSP. This indicates that the stain detection results of the proposed framework demonstrate optimal performance.

3.2. Comparison of EL Image-Based and HSI-Based Experimental Results

In order to validate the superiority of the developed method, the results of the proposed methods and EL image-based methods, which have been developed maturely, are presented in

Table 2. DVs of different methods.

PS (%)	3.9	3.1	4.6	3.3	3.7	2.8
CEM-based	3.0709	3.1594	3.4536	2.2714	2.4803	2.9618
OSP-based	0.0602	0.0595	0.0562	0.0410	0.0383	0.0355
EL1	2.9500	3.5500	7.8500	2.5300	4.3300	5.5500
EL2	0.5600	0.5700	0.6400	0.0500	0.2400	0.2400

. EL1 and EL2 are the mean and entropy of the grayscale for the EL image, respectively. Additionally, Figure 9 compares the fitting curves between the stain detection results using the proposed HSI-based method and the EL image-based method. It can be observed that the fitting curve of the detection results using the proposed HSI method follows a similar trend to the EL image-based method, indicating the effectiveness of the proposed HSI-based stain detection model to a certain extent.

4. Discussion

Based on the above experimental results and analysis, it is evident that the proposed method demonstrates a competitive performance. This achievement can be attributed to the proposed method’s comprehensive considerations. Specifically, in addressing the issue of detection being influenced by various interfering factors, the proposed method designs a series of preprocessing steps aimed at mitigating external interferences in detection results. Furthermore, to tackle the challenge posed by the unknown spectral characteristics of the target, two detection methods were designed specifically for unknown spectral d. However, upon closer examination of the SAM curves in Figure 8, it becomes apparent that the proposed method does not consistently align with the trend of stain percentage across all images. For example, in the proposed CEM-based stain detection method illustrated in Figure 8a, the detection trend in the sixth image of Figure 6 deviates from the expected stain percentage trend. Conversely, the proposed OSP-based stain detection method shown in Figure 8b yields improved results, although OSP-based detection falls short in the third and fifth images. This discrepancy suggests that there is still room for enhancement in the proposed method.

A comprehensive analysis of the influencing factors at each step of the proposed method reveals crucial elements contributing to the detection result’s instability. These factors include variations in gridline width caused by different drone flight altitudes during data collection and the instability of light sources due to morning or afternoon data capture. Additionally, during the stain detection stage, the unknown target spectral characteristics of the stain may lead to their suppression. All of these factors could contribute to the instability of detection results.

Finally, the instability observed in the detection results highlights the imperative for future refinements in future work. This will be a focal point for future improvements.

5. Conclusions

In this paper, two algorithms for detecting stains in HSI PV modules based on a UAV acquisition method were designed. Firstly, this paper took into account factors that influence the detection results and designed preprocessing steps such as removing gridlines and performing spectral correction. Secondly, as the pollutants to be detected may consist of a mixture of various types of stains, their spectral curves are also unknown. Therefore, the detection methods were designed to tackle the challenge posed by an unknown target spectrum d. These methods employed a reverse search strategy to identify the targets that require detection. Furthermore, in the proposed OSP-based method, we regarded the PV component as a signal to be removed and searched for the targets in the orthogonal space of the space it constructs, which is also advantageous for the detection task under the unknown target spectrum d. Finally, we modeled the percentage of stains and detected values to evaluate the effect of stain accumulation on power generation performance. The experiment conducted on real outdoor PV module data adds realism and credibility to our findings. One limitation of our proposed methods is that the detection results are susceptible to the influence of gratings with high grayscale values, necessitating the removal of gridlines in advance during the preprocessing step. Therefore, in future work, further exploration will be conducted on stain detection methods that are robust to the impact of gridlines.