Compressed Sensing Super-Resolution Method for Improving the Accuracy of Infrared Diagnosis of Power Equipment

Abstract

The infrared image of power equipment plays a crucial role in identifying faults, monitoring equipment condition, and so on. The low resolution and low definition of infrared images in applications contribute to the low accuracy of infrared diagnosis. A super-resolution reconstruction method of infrared image, based on compressed sensing theory, is proposed. Firstly, by analyzing the variation of high-frequency information in infrared images with different blurring degrees, the image gradient norm ratio is introduced to estimate the blur kernel matrix in the degradation model a priori. Then, in the process of image reconstruction, we add the full variational regularization term to the traditional compressed sensing model, and design a two-step full variational sparse reconstruction algorithm. Experimental results verify the effectiveness of the method. Compared with the existing classical super-resolution methods, this method offers improvement in subjective visual effect and objective evaluation index. In addition, the final image recognition and infrared diagnosis experiments show that this method is helpful to improve the accuracy of infrared diagnosis of power equipment.

1. Introduction

With the proposal of the concept of the Internet of Things and the continuous promotion of the construction of the smart grid, the use of infrared diagnosis technology for real-time monitoring and fault diagnosis of power equipment can effectively improve the operation reliability of the power grid [1,2,3]. However, there is a wide range of electrical equipment in the power grid, and the large-scale deployment of expensive traditional high-resolution infrared sensors will cause enterprises to incur unbearable costs [4,5]. Therefore, how to use low-cost, low-resolution infrared sensors to achieve the effect of high-resolution infrared sensors is the key to promote the application of infrared diagnosis technology in the Internet of Things power system [6].

The main purpose of super resolution (SR) imaging technology is to overcome the limitations of low-cost and low-precision image acquisition devices. Through single- frame or multi-frame low resolution (LR) image input, it is necessary to complete image reconstruction according to different prior knowledge, obtain high resolution (HR) images, and recover high-frequency information lost in the process of image acquisition [7]. According to the number of LR images inputted during reconstruction, super-resolution can be divided into single-frame and multi-frame image super-resolution. Among these, multi-frame image super-resolution methods include the interpolation method [8,9], iterative back projection method [10], maximum likelihood estimation method [11], sparse coding method [12] and learning-based method [13,14]. The super-resolution method of multiple images uses complementary information contained in different images to improve the resolution by registering multiple images obtained at similar time positions in the same scene. The single-frame super-resolution method only uses a single image in a scene, which is more difficult. For the specific objective of infrared imaging of electrical equipment, the focus of super-resolution image reconstruction is not only clear structure and easy recognition, but also accurate response to equipment temperature. Multi-image super-resolution can easily cause the reflection of the reconstructed image to the temperature to lose accuracy, because the input low-resolution infrared images do not strictly come at the same time. This problem is particularly serious in the case of rapid change in equipment temperature, which is more concerning in the power industry. Therefore, the super-resolution method based on a single image can ensure accuracy of reconstruction.

The super-resolution methods based on a single image are mainly divided into three categories: interpolation-based, reconstruction-based and learning-based super-resolution methods. Among them, the super-resolution method based on interpolation is the simplest, but due to its inherent smoothing benefits, it is easy to lose details and blur the edges of the reconstructed image. Researchers have put forward various improvement measures for this problem [15,16,17]. The super-resolution method based on reconstruction [11,18,19] completes the inverse iterative solution, based on the image degradation model, by introducing a priori knowledge as the limiting condition. Reference [18] proposed a super-resolution algorithm based on nonlocal information. Shao et al. [19] calculated the blur kernel and high-resolution image at the same time. In the process of alternating iteration, a normalized sparsity measure is adopted, which is a generalized integral of relative total variation. It is used to represent the heavy tailed a priori characteristics of the image. For the accurate processing of a local image, Li et al. [11] adopted local structure self-similarity in the natural image to put forward the notion of turning kernel regression total variational, which is supplemented by the non-local total variational regular term to complete the establishment of the SR model. The main goal of the learning-based method is to find the corresponding relationship between HR image and LR image, usually by training an isomorphic dictionary or establishing both mapping relationships [20]. Kim et al. [21] proposed a low complexity adaptive single image super-resolution method. This method analyzes the input image and the corresponding low-resolution image to determine the intensity of the high-frequency component, while establishing the relationship between them. The effective addition of high-frequency information and high-quality image reconstruction are realized. Wang et al. [22] proposed the SFT-GAN model, which takes the image segmentation mask as the prior knowledge of single image super-resolutions to guide the texture reconstruction of different regions in single-image super-resolutions. Literature [23] offers a kind of latest method. It uses reinforcement learning to find the optimal network, which can reduce the influence of the manual on model design. This kind of method first defines a search space, uses the corresponding controller to process the search, outputs the reconstruction results after evaluation, and automatically adjusts them through network training until the network converges to get good reconstruction results.

The learning-based super-resolution method requires a large number of high-resolution infrared image samples. In super-resolution, the up-sampling rate is limited by the constructed model or isomorphic dictionary. The flexibility of the algorithm is low. In addition, the current super-resolution methods pay more attention to the accurate estimation of image degradation process. Efrat et al. [24] pointed out that in the super-resolution problem, the accurate estimation of the blur kernel representing the degradation process has a greater impact than the selection of the model. Therefore, in order to reconstruct infrared images with higher quality and accuracy, before establishing an accurate reconstruction model, we need to estimate the blur kernel.

Aiming at the problem of low resolution of infrared images obtained by infrared sensors of power equipment, this paper adopts the method based on reconstruction under the framework of compressed sensing. We analyze the change law of high-frequency information in the stage of blur degradation. For blur kernel estimation, an image gradient norm ratio is introduced as a priori information to realize the accurate estimation of the image degradation process. After obtaining the blur kernel, we use the compressed sensing model, with full variational regularization term, as the reconstruction objective function. It is optimized by sparse total variational quadratic iteration. The final experimental results show that our method can reconstruct the infrared image of power equipment with high quality.

2. SR Basic Model of Compressed Sensing

Compressed sensing theory gives a method to reconstruct the down-sampled signal through nonlinear optimization technology [25]. Suppose that the desired high-resolution image is represented as an n-dimensional column vector $x\in R^n$, where $n$ is a larger value. Theoretically, $x$ can represent any one-dimensional signal. For this problem, $x$ is an n-pixel gray image (or an n-pixel image of a channel of RGB image) that has been converted into an $n\times 1$ dimensional vector. SR is to obtain the HR image $x$ from the LR image $y\in R^m$, where $m\ll n$. The goal of this paper is to recover the high-resolution signal $x$ using only $y$ as the input.

This problem is ill posed. Due to the reduction of resolution and the loss of information, an LR image can be mapped to multiple potential high-resolution images. In this case, the key assumption of compressed sensing theory needs to be applied, that is, the transformation shape $\tilde{x}$ of signal $x$ is sparse under a certain base $D$, and there are $k$ non-zero factors in $\tilde{x}$. It is a reasonable assumption that high-resolution images can be compressed in different transform domains, such as the wavelet domain, Fourier domain or artificially constructed dictionary. At this time, according to the rationality of the above assumptions, combined with the image degradation process, we can write the compressed sensing super-resolution model thus:

$$y=CHD\tilde{x}$$

(1)

where $C$ is the subsample matrix corresponding to the degradation process; $H$ is the matrix multiplication form of blur kernel $h$; and $D$ is the sparse basis.

3. Blur Kernel Estimation

According to Formula (1) in Section 2, if we want to obtain the blur matrix $H$, you first need to model according to the image degradation principle to estimate the blur kernel $h$. To improve the accuracy of blur kernel estimation, the most commonly used method is the maximum a posteriori probability estimation method. The accuracy of estimation is improved by adding reasonable prior information to the known degradation model. Therefore, we introduce the gradient norm ratio a priori into the blur kernel estimation under the super-resolution problem.

3.1. Gradient Norm Ratio Priori

The essence of image degradation is the loss and aliasing of pixel information. This process will result in the loss of high-frequency information of the image, which is reflected in the intuitive visual effect, that is, the edge texture of the low-resolution image is not clear and the contrast is low. Therefore, it is necessary to find an index as a constraint. When the reconstructed image tends to blur latent image, the index increases, and when it tends to clear latent image, the index decreases. As a result, the restored image in the iterative process is close to the latent HR image, achieving an accurate estimation of blur kernel.

In optimization problems, the most common constraints are $L_1$ norm, $L_2$ norm and $L_p$ norm. Generally, $0<p<1$. Taking $L_1$ norm as an example, this is mostly used to punish the high frequency band in image processing. When the image contains noise, the noise can be removed by minimizing the $L_1$ norm. When the image is degraded, the high-frequency band itself will decay, and minimizing the $L_1$ norm of the high-frequency component of the image will lead to further blur of the reconstructed image. For the other norms, because its mechanism is similar to the $L_1$ norm, if the norm constraint is directly introduced as the regular term, it cannot improve the quality of image reconstruction. However, it is found that although the cost decreases with the decrease of the resolution of the input image, that is, the blur of the reconstructed image, the cost reduction speed of different norms is different, and the cost reduction speed of $L_2$ norm is significantly higher than that of $L_1$ norm. Therefore, the introduction of high-frequency information norm ratio as a priori information can ensure that the penalty cost increases with the reduction of reconstructed image quality. In other words, the purpose of improving reconstructed image quality can be achieved by reducing the cost. The image high-frequency information can be obtained in many ways. In this paper, the transverse and longitudinal gradient is selected as an example to constrain the reconstructed image in the form of norm ratio. Figure 1 is a schematic diagram of the change of the high-frequency information cost of reconstructed images when different constraints are adopted at twice the down-sampling ratio.

High-frequency information is obtained by convoluting the reconstructed image with the transverse difference operator $f_1=\left[1-1\right]$ and the longitudinal difference operator $f_2={\left[1-1\right]}^T$, respectively. The reconstructed image is a series of images obtained from the infrared image of typical power equipment, which is reconstructed back to the original size through cubic interpolation after Gaussian kernel action with different width and 2 times down sampling. It can be clearly seen from Figure 1 that with the increase of ambiguity, the cost of high-frequency information under each norm constraint is decreasing, while the cost of the ratio of $L_1$ norm to $L_2$ norm is increasing. Define $\mathrm{GNR}\left(x\right)=\parallel x_1\parallel /\parallel x_2\parallel$. $\mathrm{GNR}\left(x\right)$ is introduced into the objective function as a constraint term. During the process of minimizing an objective function, the closeness of the intermediate latent image to the clear image can help estimate blur kernels more accurately.

The estimation model of the blur kernel is:

$$\begin{array}{c}\left(\widehat{X},\widehat{h}\right)=\arg \underset{X,h}{\min }\parallel Y-S_l\left(h\otimes X\right)S_r\parallel {}_2^2+\lambda \left(GNR\left(f_1\otimes X\right)+GNR\left(f_2\otimes X\right)\right)\\ +\gamma \parallel h{\parallel }_2^2+\delta \parallel {\mathsf{\Psi }}^{T}X{\parallel }_1\end{array}$$

(2)

where $\lambda ,\delta ,$ and $\gamma$ are weighting coefficients; $S_l$ is a row sampling matrix; $S_r$ is a column sampling matrix; $X$ and $Y$ is the two-dimensional form corresponding to $x$ and $y$ respectively; $\mathsf{\Psi } \mathrm{is} \mathrm{sparse} \mathrm{basis}$. The first to fourth terms in the equation are the data fidelity term, gradient norm ratio prior information term and constraint term to ensure the sparsity of blur kernel and image sparsity coefficient. The addition of the last two items is determined by the sparsity of blur kernel and the application requirements of compressed sensing theory.

3.2. Blure Kernel Estimation Algorithm

3.2.1. Intermediate Latent Image X Estimation

In order to solve Equation (2), we first need to use the semi-quadratic splitting algorithm [26] to introduce auxiliary variables. Then it is solved by alternating minimization. After introducing auxiliary variables, the equation is shown in Equation (3).

$$\begin{array}{l}\left(X^{\prime },h^{\prime }\right)=\arg \underset{X,h}{\min }\parallel Y-S_{l}G{S}_r\parallel {}_2^2+\lambda \left(GNR\left(f_1\otimes X\right)+GNR\left(f_2\otimes X\right)\right)\\ +\gamma \parallel h{\parallel }_2^2+\delta \parallel {\mathsf{\Psi }}^{T}X{\parallel }_1+\epsilon \parallel h\otimes X-G{\parallel }_2^2\end{array}$$

(3)

When solving the intermediate latent image x, we only need to pay attention to the following parts of the Equation (3).

$$\begin{gathered} \left(X^{\prime },h^{\prime }\right)=\arg \underset{X,h}{\min }\lambda \left(GNR\left(f_1\otimes X\right)+GNR\left(f_2\otimes X\right)\right) \\ +\delta \parallel {\mathsf{\Psi }}^{T}X{\parallel }_1+\epsilon \parallel h\otimes X-G{\parallel }_2^2 \end{gathered}$$

(4)

Equation (4) contains the GNR function and the $L_1$ norm. We cannot solve it directly with fast Fourier transform, so auxiliary variables $W_i$ and $\tilde{X}$ are introduced. Where $W$ corresponds to $f_i\otimes X$ and $\tilde{X}$ corresponds to ${\mathsf{\Psi }}^{T}X$. Then, (4) can be transformed into (5):

$$\begin{array}{l}\left(X^{\prime },W_i^{\prime },{\tilde{X}}^{\prime }\right)=\arg \underset{X,W_i,\tilde{X}}{\min }{\lambda }^{\prime }\parallel f_i\otimes X-W_i\parallel {}_2^2+\lambda GNR\left(w_i\right)\\ +{\delta }^{\prime }\parallel \tilde{X}-{\mathsf{\Psi }}^{T}X{\parallel }_2^2+\delta \parallel \tilde{X}{\parallel }_1+\epsilon \parallel h\otimes X-G{\parallel }_2^2\end{array}$$

(5)

where ${\lambda }^{\prime }$ and ${\delta }^{\prime }$ are the regularization parameters; $i=1, 2$. The above optimization problem can be solved by minimizing $X$ and $W_i$ and $\tilde{X}$, respectively. First, fix the other variables, and $X$ solves the equation as:

$$X^{\prime }=\arg \underset{X}{\min }{\lambda }^{\prime }\parallel f_i\otimes X-W_i{\parallel }_2^2+{\delta }^{\prime }\parallel \tilde{X}-{\mathrm{\Psi }}^{T}X{\parallel }_2^2+\epsilon \parallel h\otimes X-G{\parallel }_2^2$$

(6)

where $\parallel f_i\otimes X-W_i{\parallel }_2^2=\parallel f_1\otimes X-W_1{\parallel }_2^2+\parallel f_2\otimes X-W_2{\parallel }_2^2$. We use fast Fourier transform to solve Equation (6):

$$X=F^{-1}\left(\frac{{\lambda }^{\prime }{F}_W+{\delta }^{\prime }F\left(\mathrm{\Psi }\tilde{X}\right)+\epsilon \overline{F\left(h\right)}\circ F\left(G\right)}{{\lambda }^{\prime }{F}_f+{\delta }^{\prime }+\epsilon \overline{F\left(h\right)}\circ F\left(h\right)}\right)$$

(7)

where $F_W$ is $\overline{F\left(f_1\right)}\circ F\left(W_1\right)+\overline{F\left(f_2\right)}\circ F\left(W_2\right)$; $F_f$ is $\overline{F\left(f_1\right)}\circ F\left(f_1\right)+\overline{F\left(f_2\right)}\circ F\left(f_2\right)$; $F\left(\cdot \right)$ and $F^{-1}\left(\cdot \right)$ denote the fast Fourier transform and inverse fast Fourier transform, respectively; $\overline{F\left(\cdot \right)}$ is the complex conjugate operator; $\circ$ denotes component multiplication, and the division in Equation (7) is component division. After the calculation of $X$, the $\tilde{X}$ can be obtained by:

$${\tilde{X}}^{\prime }=\arg \underset{\tilde{X}}{\min }{\delta }^{\prime }\parallel \tilde{X}-{\mathrm{\Psi }}^{T}X{\parallel }_2^2+\delta \parallel \tilde{X}{\parallel }_1$$

(8)

We solve for $\tilde{X}$ by soft-threshold contraction:

$$\tilde{X}=\max \left\{\left|{\mathsf{\Psi }}^{T}X\right|-\frac{\delta }{2{\delta }^{\prime }},0\right\}\circ \mathrm{sign}\left({\mathsf{\Psi }}^{T}X\right)$$

(9)

where $\mathrm{sign}\left(\cdot \right)$ is a sign function. Next, the solution to the subproblem of $W_i$ can be expressed as:

$$W_i^{\prime }=\arg \underset{W_i}{\min }{\lambda }^{\prime }\parallel f_i\otimes X-W_i{\parallel }_2^2+\lambda GNR\left(W_i\right)$$

(10)

Expansion function $GNR \left(W_i\right)$:

$$W_i^{\prime }=\arg \underset{W_i}{\min }{\lambda }^{\prime }\parallel f_i\otimes X-W_i{\parallel }_2^2+\lambda \frac{\parallel W_i{\parallel }_1}{\parallel W_i{\parallel }_2}$$

(11)

Due to the existence of $GNR \left(W_i\right)$ term, the $W_i$ subproblem is nonconvex. However, Equation (11) can be changed into a convex optimization problem by fixing $W_i$ in the denominator $\parallel W_i{\parallel }_2$ as the value of the last iteration. At this time, the problem shown in Equation (11) can be solved by soft threshold like Equation (8):

$$W_i=\max \left\{\left|f_i\otimes X\right|-\frac{\lambda }{2{\lambda }^{\prime }},0\right\}\circ \mathrm{sign}\left(f_i\otimes X\right)$$

(12)

After solving the above variables, the variable $G$ is solved before solving the blur kernel $h$. When solving $h$ later, $G$ is substituted as a known quantity. The solution to $G$ is:

$$G^{\prime }=\arg \underset{G}{\min }\parallel Y-S_{l}G{S}_r{\parallel }_2^2+\epsilon \parallel h\otimes X-G{\parallel }_2^2$$

(13)

For Equation (13), it can be solved using gradient descent. The iterative step size is determined by a non-monotonic linear search method [27].

3.2.2. Blur Kernel h Estimation

When solving variable $h$. It has been pointed out in the literature [28] that directly using the intermediate latent image estimated above for calculation will reduce the accuracy of the results. Using gradient image for calculation can more effectively ensure the accuracy of estimation. So, we can get $h$ by solving the following:

$$h^{\prime }=\arg \underset{h}{\min }\epsilon \parallel h\otimes \nabla X-\nabla G{\parallel }_2^2+\gamma \parallel h{\parallel }_2^2$$

(14)

Using fast Fourier transform solution (14):

$$h=F^{-1}\left(\frac{\overline{F\left(\nabla X\right)}F\left(\nabla G\right)}{\overline{F\left(\nabla X\right)}F\left(\nabla X\right)+\gamma }\right)$$

(15)

Since the blur kernel element is non-zero, and the sum is 1, for the negative elements produced by each iteration of $h$, we set them to zero and normalize $h$ at last. As suggested by [28], we decrease $\lambda$ gradually to make more information available for kernel estimation.

To sum up, the flow of the blur kernel estimation algorithm is shown in Figure 2.

4. Image SR Reconstruction

We will build a super-resolution image reconstruction model based on the relationship between high resolution image and low-resolution image put forward by combining compressed sensing theory and the image SR model. An optimization method named Two-step Total Variation Sparse Iteration (TwTVSI) is proposed. We lead a regular term of Total Variation (TV) into the objective function of the $L_1$ norm, so that the finite variation feature of the image can be used to remove boundary effects from the BCS reconstruction method. In this way, we do not use a single minimum total variation model. Our model can reflect the sparseness of the image gradient domain and make full use of the sparseness of the image, with short time consumption and good reconstruction effect.

4.1. Objective Function

On the basis of the principle of image degradation and compressed sensing, the objective function of SR can be written as follows [29].

$$\begin{array}{c}\arg \underset{\tilde{x}}{\min }\parallel \tilde{x}{\parallel }_1\\ s.t. y=CHD\tilde{x}\end{array}$$

(16)

Since the SR reconstruction of the whole image requires a large storage space and is very complicated to calculate, we choose the BCS method to reconstruct the image in blocks. We segment an image as follows:

$$Y=\left(\begin{array}{cccc}{Y}_{b\left(1,1\right)}& Y_{b\left(1,2\right)}& \cdots & Y_{b\left(1,J\right)}\\ Y_{b\left(2,1\right)}& Y_{b\left(2,2\right)}& \cdots & Y_{b\left(2,J\right)}\\ ⋮& ⋮& \ddots & ⋮\\ Y_{b\left(I,1\right)}& Y_{b\left(I,2\right)}& \cdots & Y_{b\left(I,J\right)}\end{array}\right)$$

(17)

where $Y$ is an LR image. $Y_{b\left(i,j\right)}$ is an LR image block; $Y$ is divided into $I\times J$ image blocks. $X_{b\left(i,j\right)}$ is the HR image block at the corresponding position of $Y_{b\left(i,j\right)}$, and the size of $X_{b\left(i,j\right)}$ is $S\times S$.

The reconstruction of blocked compressed sensing approach will bring about the problem of “block effect”, so we introduce a TV regular term into the objective function. In this way, the minimum variation constraint can be applied to the whole image, thereby eliminating the reconstructed image block effect. The image total variation is calculated as follows:

$$\begin{array}{l}\parallel X{\parallel }_{TV}=\\ {\displaystyle \sum }_{i,j}\sqrt{{\left(X\left(i,j\right)-X\left(i-1,j\right)\right)}^2+{\left(X\left(i,j\right)-X\left(i,j-1\right)\right)}^2}\end{array}$$

(18)

where $X\left(i,j\right)$ represents the value contained in row $i$ and column $j$ of the image matrix $X$. The image is segmented and a TV regular term is introduced. So, the objective function becomes:

$$\begin{array}{c}\arg \underset{\tilde{X}}{\min \mathsf{\lambda }}\parallel X{\parallel }_{TV}+{\displaystyle \sum }_{i=1}^I{\displaystyle \sum }_{j=1}^J\parallel {\tilde{X}}_{\left(i,j\right)}{\parallel }_1\\ s.t. y_{\left(i,j\right)}=CHD{\tilde{X}}_{\left(i,j\right)}, i=1,2,\dots I, j=1,2,\dots ,J\end{array}$$

(19)

where ${\tilde{X}}_{\left(i,j\right)}$ is the sparse representation of the column vector obtained by expanding $X_{b\left(i,j\right)}$, and $y_{\left(i,j\right)}$ is the row-expanded column vector corresponding to $Y_{b\left(i,j\right)}$.

In the solution process of Equation (19), it is a difficult problem to obtain ${\tilde{X}}_{\left(i,j\right)}$ from $X$ when we perform sparse constraints on the updated $X$ under the constraint of $\parallel X{\parallel }_{TV}$. $D$ as an overcomplete dictionary makes the sparse coefficient dimension of the image much larger than its own dimension. Therefore, we use the sparse coefficient matrix $\tilde{X}$ for the image matrix $X$ with the expression $X=Ar\left(D\tilde{X}\right)$. The objective function is improved and expressed as:

$$\begin{array}{c}\arg \underset{\tilde{X}}{\min \mathsf{\lambda }}\parallel Ar\left(D\tilde{X}\right){\parallel }_{TV}+{\displaystyle \sum }_{i=1}^I{\displaystyle \sum }_{j=1}^J\parallel {\tilde{X}}_{\left(i,j\right)}{\parallel }_1\\ s.t. y_{\left(i,j\right)}=CHD{\tilde{X}}_{\left(i,j\right)}, i=1,2,\dots I, j=1,2,\dots ,J\end{array}$$

(20)

where $\tilde{X}=\left({\tilde{X}}_{\left(1,1\right)},{\tilde{X}}_{\left(1,2\right)},\cdots ,{\tilde{X}}_{\left(1,J\right)},{\tilde{X}}_{\left(2,1\right)},\cdots {\tilde{X}}_{\left(I,J\right)}\right)$ is the sparse coefficient matrix. The function $Ar\left(z\right)$ indicates that the column vectors in $z$ are sequentially stitched into corresponding images.

4.2. Optimization Solution

We adopt the Lagrange multiplier method to transform Equation (20) into:

$$\begin{array}{l}\arg \underset{\tilde{X}}{ \underset{\tilde{X}}{\min \mathsf{\lambda }}\parallel Ar\left(D\tilde{X}\right){\parallel }_{TV}}\hfill \\ +{\displaystyle \sum }_{i=1}^I{\displaystyle \sum }_{j=1}^j\frac{{\beta }_{\left(i,j\right)}}{2}\parallel y_{\left(i,j\right)}-CHD{\tilde{X}}_{\left(i,j\right)}{\parallel }_2^2+\parallel {\tilde{X}}_{\left(i,j\right)}{\parallel }_1\hfill \end{array}$$

(21)

where ${\beta }_{\left(i,j\right)}$ is the Lagrangian multiplier. We put forward the TwTVSI optimization method to solve Equation (21). Iterative solutions are expressed as follows:

$$\begin{array}{l}{\tilde{X}}^{\left(K+1\right)}={\tilde{X}}^{\left(K\right)}-{\mu }^{\left(K\right)}\left\{\lambda \frac{\partial \parallel Ar\left(D\tilde{X}\right){\parallel }_{TV}}{\partial {\tilde{X}}^{\left(K\right)}}\right.+\hfill \\ \left. \frac{1}{2}\frac{\partial {{\displaystyle \sum }}_{i=1}^I{{\displaystyle \sum }}_{j=1}^J\frac{{\beta }_{\left(i,j\right)}}{2}\parallel y_{\left(i,j\right)}-CHD{\tilde{X}}_{\left(i,j\right)}{\parallel }_2^2+2\parallel {\tilde{X}}_{\left(i,j\right)}{\parallel }_1}{\partial {\tilde{X}}^{\left(K\right)}}\right\}\hfill \\ ={\tilde{X}}^{\left(K\right)}-{\mu }^{\left(K\right)}\left.\left\{\lambda g_1\left({\tilde{X}}^{\left(K\right)}\right)\right.+\frac{1}{2}{g}_2\left({\tilde{X}}^{\left(K\right)}\right)\right\}\hfill \end{array}$$

(22)

where ${\mu }^{\left(K\right)}$ is the iteration step size; and $g_1\left({\tilde{X}}^{\left(K\right)}\right)$ is the gradient derivative of $Ar{\left(D\tilde{X}\right)}_{TV}$ to the sparse coefficient matrix ${\tilde{X}}^{\left(K\right)}$, which can be solved by the gradient descent method directly.

$$g_1\left({\tilde{X}}^{\left(K\right)}\right)=\frac{\partial \parallel Ar\left(D\tilde{X}\right){\parallel }_{TV}}{\partial {\tilde{X}}^{\left(K\right)}}=\left.\left\{\frac{\partial \parallel Ar\left(D\tilde{X}\right){\parallel }_{TV}}{\partial {\tilde{X}}_{\left(1,1\right)}^{\left(K\right)}}\right.,\dots ,\frac{\partial \parallel Ar\left(D\tilde{X}\right){\parallel }_{TV}}{\partial {\tilde{X}}_{\left(i,j\right)}^{\left(K\right)}},\dots ,\frac{\partial \parallel Ar\left(D\tilde{X}\right){\parallel }_{TV}}{\partial {\tilde{X}}_{\left(I,J\right)}^{\left(K\right)}}\right\}$$

(23)

$$\begin{array}{l}\frac{\partial \parallel Ar\left(D\tilde{X}\right){\parallel }_{TV}}{\partial {\tilde{X}}_{\left(i,j\right)}^{\left(K\right)}}={\displaystyle \sum }_{m=1}^S{\displaystyle \sum }_{n=1}^S\mathrm{Pd}\left(m,n,{\tilde{X}}_{\left(i,j\right)}^{\left(K\right)},{\tilde{X}}_{\left(i,j\right)}^{\left(K\right)}\right)\hfill \\ +{\displaystyle \sum }_{m=1}^S{\displaystyle \sum }_{n=1}^1\mathrm{Pd}\left(m,n,{\tilde{X}}_{\left(i,j+1\right)}^{\left(K\right)},{\tilde{X}}_{\left(i,j\right)}^{\left(K\right)}\right)\hfill \\ +{\displaystyle \sum }_{m=1}^1{\displaystyle \sum }_{n=1}^S\mathrm{Pd}\left(m,n,{\tilde{X}}_{\left(i+1,j\right)}^{\left(K\right)},{\tilde{X}}_{\left(i,j\right)}^{\left(K\right)}\right)\hfill \end{array}$$

(24)

$$\mathrm{Pd}\left(m,n,{\tilde{X}}_a,{\tilde{X}}_b\right)=\frac{\frac{\partial \mathrm{Dr}\left(m,n,{\tilde{X}}_a\right)}{\partial {\tilde{X}}_b}\times \mathrm{Dr}\left(m,n,{\tilde{X}}_a\right)+\frac{\partial \mathrm{Dc}\left(m,n,{\tilde{X}}_a\right)}{\partial {\tilde{X}}_b}\times \mathrm{Dc}\left(m,n,{\tilde{X}}_a\right)}{\sqrt{\left\{{\left.\mathrm{Dr}\left(m,n,{\tilde{X}}_a\right)\right\}}^2+{\left.\mathrm{Dr}\left(m,n,{\tilde{X}}_a\right)\right\}}^2+{10}^{-6}\right.}}$$

(25)

where $\mathrm{Dr}\left(m,n,{\tilde{X}}_{\left(i,j\right)}^{\left(K\right)}\right)$ and $\mathrm{Dc}\left(m,n,{\tilde{X}}_{\left(i,j\right)}^{\left(K\right)}\right)$ are, respectively, image row and column difference operation functions.

Since we use the block SR model for the image, the cross operation of edge elements between adjacent image blocks is involved in the row–column direction difference operation. By derivation, we can use a periodically recurring sparse dictionary matrix to represent the difference operation between adjacent image patches. Let $\theta$ be a cyclic matrix with a period of $S$, i.e., $\theta \left(m,n\right)=\theta \left(m+S,n\right)=\theta \left(m,n+S\right)$, and its matrix structure is as follows:

$$\theta =\left(\begin{array}{cccc}1& 2& \cdots & S\\ S+1& S+2& \cdots & 2S\\ ⋮& ⋮& \ddots & ⋮\\ S^2-S+1& S^2-S+2& \cdots & S^2\end{array}\right)$$

(26)

where $k$ is the value corresponding to the $m$-th row and $n$-th column in the matrix $\theta$, denoted as $\theta \left(m,n\right)$; $D_{\theta \left(m,n\right)}$ is the $k$-th row vector of the sparse dictionary $D$. Therefore, the high-resolution image block $X_{b\left(i,j\right)}$ can be represented by the sparse coefficients as:

$$X_{b\left(i,j\right)}=\left(\begin{array}{cccc}{D}_{\theta \left(1,1\right)}{\tilde{X}}_{\left(i,j\right)}& D_{\theta \left(1,2\right)}{\tilde{X}}_{\left(i,j\right)}& \cdots & D_{\theta \left(1,S\right)}{\tilde{X}}_{\left(i,j\right)}\\ D_{\theta \left(2,1\right)}{\tilde{X}}_{\left(i,j\right)}& D_{\theta \left(2,2\right)}{\tilde{X}}_{\left(i,j\right)}& \cdots & D_{\theta \left(2,S\right)}{\tilde{X}}_{\left(i,j\right)}\\ ⋮& ⋮& \ddots & ⋮\\ D_{\theta \left(S,1\right)}{\tilde{X}}_{\left(i,j\right)}& D_{\theta \left(S,2\right)}{\tilde{X}}_{\left(i,j\right)}& \cdots & D_{\theta \left(S,S\right)}{\tilde{X}}_{\left(i,j\right)}\end{array}\right)$$

Then, $\mathrm{Dr}\left(m,n,{\tilde{X}}_{\left(i,j\right)}^{\left(K\right)}\right)$ and $\mathrm{Dr}\left(m,n,{\tilde{X}}_{\left(i,j\right)}^{\left(K\right)}\right)$ can be calculated by the following method:

$$\left\{\begin{array}{c}Dr\left(m,n,{\tilde{X}}_{\left(i,j\right)}^{\left(K\right)}\right)=D_{\theta \left(m,n\right)}{\tilde{X}}_{\left(i,j\right)}^{\left(K\right)}-D_{\theta \left(m-1,n\right)}{\tilde{X}}_{\left(i-\neg \left(m-1\right),j\right)}^{\left(K\right)}\\ Dc\left(m,n,{\tilde{X}}_{\left(i,j\right)}^{\left(K\right)}\right)=D_{\theta \left(m,n\right)}{\tilde{X}}_{\left(i,j\right)}^{\left(K\right)}-D_{\theta \left(m,n-1\right)}{\tilde{X}}_{\left(i,j-\neg \left(n-1\right)\right)}^{\left(K\right)}\end{array}\right.$$

where $\neg$ is the negation operator, and the operation method is $\neg \left(z\right)=\left\{\begin{array}{c}0 z\ne 0\\ 1 z=0\end{array}\right.$.

As for $g_2\left({\tilde{X}}^{\left(K\right)}\right)$, $\parallel {\tilde{X}}_{\left(i,j\right)}^{\left(K\right)}{\parallel }_1$ is a nondifferentiable convex function about ${\tilde{X}}_{\left(i,j\right)}^{\left(K\right)}$, and $\parallel y_{\left(i,j\right)}-CHD{\tilde{X}}_{\left(i,j\right)}^{\left(K\right)}{\parallel }_2^2$ is a differentiable convex function about ${\tilde{X}}_{\left(i,j\right)}^{\left(K\right)}$.Therefore, $g_2\left({\tilde{X}}^{\left(K\right)}\right)$ cannot be directly calculated by the gradient descent method, so it is solved by the proximal gradient (PG) method:

$$\left\{\begin{array}{l}{Z}_{\left(i,j\right)}^{\left(K\right)}={\tilde{X}}_{\left(i,j\right)}^{\left(K\right)}-{\beta }_{\left(i,j\right)}^{\left(K\right)}{D}^T{H}^T{C}^T\left(y_{\left(i,j\right)}-CHD{\tilde{X}}_{\left(i,j\right)}^{\left(K\right)}\right)\hfill \\ {\tilde{X}}_{\left(i,j\right)}^{\left(K+1\right)}=\mathrm{Shrink}\left(Z_{\left(i,j\right)}^{\left(K\right)},{\beta }_{\left(i,j\right)}^{\left(K\right)}\right)\hfill \\ =\max \left\{\left.\left|Z_{\left(i,j\right)}^{\left(K\right)}\right|-{\beta }_{\left(i,j\right)}^{\left(K\right)},0\right\}\circ \mathrm{sign}\left(Z_{\left(i,j\right)}^{\left(K\right)}\right)\right.\hfill \end{array}\right.$$

(27)

where $\mathrm{Shrink}\left(·\right)$ represents the soft threshold shrink function; ${\beta }_{\left(i,j\right)}^{\left(K\right)}$ is actually the step size factor, and we can use the backtracking line search method to calculate its size.

According to the above elaboration, the TwTVSI algorithm flow is summarized as shown in Figure 3.

The specific calculation steps of the TwTVSI algorithm are as follows:

• Initialize $K$ and $\tilde{X}$ separately.
• The first iteration: Use gradient descent to solve the TV regular term to get ${\tilde{X}}^{\left(k,1\right)}$, ${\tilde{X}}^{\left(k,1\right)}={\tilde{X}}^{\left(K,0\right)}-\lambda {\mu }^{\left(K\right)}{g}_1\left({\tilde{X}}^{\left(K,0\right)}\right)$.
• The second iteration: ${\tilde{X}}^{\left(K+1,0\right)}$ is obtained by using the proximal gradient method from ${\tilde{X}}^{\left(k,1\right)}$ for the sparse regular term according to Equation (27). Perform the
• Judgment: when $\parallel Ar\left(D{\tilde{X}}^{\left(K+1,0\right)}\right)-Ar\left(D{\tilde{X}}^{\left(K,0\right)}\right){\parallel }_2^2$ is less than the error constraint $\epsilon$, or $K$ is greater than the maximum number of iterations $K_{max}$, stop the iteration, when it is otherwise, we set $K=K+1$ and go back to step 2.
• Output: high resolution image $Ar\left(D{\tilde{X}}^{\left(K+1,0\right)}\right)$.

5. Experiment and Result Analysis

5.1. Experimental Data and Evaluation Paramenters

All the environments are carried out on a computer with Intel(R) Core (TM)i5-9300H CPU @2.40 GHz and 16.00 GB RAM. We debugged the following parameters on MATLAB: $\lambda =0.004$; ${\epsilon }_0=0.008$; $\gamma =2$. The HR image is composed of 32 × 32 blocks. A sparse dictionary, consisting of 3096 atoms of length 1024, is trained by the high-dimensional dictionary training method introduced in [26].

We compared our method with the methods proposed by Keys [30], Shao [19], Li [11], and Kim [21] and adopted the average gradient (AG) and information entropy (IE) as objective evaluation indicators.

In order to verify the actual application effect of the infrared image reconstructed by our proposed algorithm, we used the image recognition algorithm proposed in literature [31] to design a comparative experiment. Through the recognition of infrared images reconstructed by different SR methods, the performance and practical application value of our method was verified. Since the basis of the image recognition algorithm lies in the effective extraction and correct matching of image feature points, the number of feature points obtained by image extraction and the correct matching rate of feature points were used as evaluation indicators. In order to more clearly reflect the difference between the performance of different algorithms, we randomly selected 80 sets of feature point matching results for visual display.

In addition, we performed the same infrared image diagnosis process on the high-resolution images reconstructed by different SR algorithms. We sequentially performed super pixel segmentation, image recognition, super pixel merging, image segmentation, equipment area extraction, and equipment operating status diagnosis on the reconstructed infrared image. The super pixel segmentation and merging algorithm adopted the methods in the literature [32], and the image recognition still used the algorithm proposed in the literature [31]. The diagnostic process of infrared images is shown in Figure 4.

5.2. Infrared Image Reconstruction Experiment

We reconstructed low-resolution infrared images, thereby validating the effectiveness of our method in practical applications. We used 9 infrared images with a resolution of 128 × 128 taken on site for super-resolution reconstruction. The reconstruction results of image No. 9 using different methods are shown in Figure 5.

According to Figure 5, the reconstruction result of the method proposed by Keys et al. are the most ambiguous. The image details and textures reconstructed by the methods proposed by Shao and Li are similar. Although they have a significant improvement in visual quality compared to the original low-resolution blurred images, they are still blurry. The method proposed by Kim et al. reconstructed some significant textures of the image, but the textures have obvious defects. In addition, the image reconstructed by Kim et al. has poor visual quality, due to the ringing effect, but the AG indexes of the image are relatively high. However, the images reconstructed by our method are the sharpest. In summary, our method has certain advantages over current methods in handling blurred LR images. In addition, the blur kernel estimated by our method in the reconstruction process allows us to discover that image blur is very complex. It is inaccurate to use the standard Gaussian blur kernel to represent the image blurring process; that is to say, it is necessary to study the blind SR algorithm. The SR reconstruction results of the other eight infrared images by different methods are listed in

Table 1. Comparison of AG indexes of images reconstructed using different methods.

Image Number	Keys	Shao	Li	Kim	Ours
1	19.775	27.041	30.884	29.349	32.264
2	22.088	29.501	34.080	32.242	35.862
3	21.673	29.864	34.454	35.397	37.037
4	18.224	20.516	23.267	25.403	24.967
5	18.663	26.544	29.113	28.802	30.924
6	17.382	25.653	26.671	26.239	28.594
7	24.436	34.371	35.125	37.538	42.723
8	20.335	26.411	34.929	27.216	32.857

and

Table 2. Comparison of IE indexes of images reconstructed using different methods.

Image Number	Keys	Shao	Li	Kim	Ours
1	5.504	6.203	6.111	6.048	6.276
2	6.597	6.419	6.653	6.706	6.948
3	6.318	6.581	6.738	6.442	6.281
4	5.530	5.603	5.762	5.891	5.806
5	6.103	6.143	6.249	6.543	6.698
6	5.609	5.581	5.609	5.191	5.792
7	6.742	6.764	6.865	6.782	6.876
8	5.713	5.862	5.790	5.983	5.316

in the form of AG and IE metrics, respectively. It can be seen from the data in the tables that the reconstruction results of our method have certain advantages over these two indicators.

5.3. Comparative Experiment of Image Recognition

It can be seen from Figure 6 that the HR infrared image reconstructed by our proposed method has clearer and richer details. Therefore, the correct matching rate is significantly improved compared with the contrast method when performing feature point matching and recognition with the visible light sample image.

Table 3. The number of feature points detected in images reconstructed by different methods.

Image Number	LR	Keys	Shao	Li	Kim	Ours
1	257	278	351	377	385	405
2	219	248	309	328	334	365
3	153	164	210	223	221	252
4	256	276	351	375	379	416
5	185	209	267	282	292	326

and

Table 4. The correct matching rate of feature points in images reconstructed by different methods.

Reconstruction Algorithm	Correct Matching Rate of Feature Points
	Image 1	Image 2	Image 3	Image 4	Image 5
LR	52.38	48.78	51.15	50.22	53.79
Keys’	55.95	50.70	52.37	53.84	54.34
Shao’s	61.52	59.56	64.56	61.88	64.75
Li’s	71.03	71.76	71.79	69.75	70.91
Kim’s	74.23	72.62	76.65	76.36	74.48
Ours	82.13	83.40	81.04	83.25	84.21

respectively show the number of feature points and the correct recognition rate of the infrared images reconstructed by different methods. The experimental results are mutually corroborated with Figure 6, which reflects the superiority of our method.

5.4. Comparative Experiment of Image Recognition

The intermediate results produced during the image diagnosis process are shown in Figure 7, Figure 8, Figure 9, Figure 10, Figure 11 and Figure 12, and the final diagnosis results are shown in Figure 13. It can be seen from Figure 13 that the detection area obtained by our method has the highest agreement with the actual device label image, and does not include the base of the transformer bushing. Compared with the actual device label image, the detection area obtained by other methods contains different degrees of the base of the transformer bushing. As a result, Figure 13b–e misdiagnosed the base of transformer bushing as a defective area of transformer bushing. According to the “Application rules of infrared diagnosis for live electrical equipment”, the diagnostic standards for different equipment are different. Since the bushing base is connected to the transformer tank, the internal oil temperature is relatively high, and the temperature of the bushing base is often higher than the surface temperature of the bushing. Therefore, the criteria for judging these two defects, i.e., “hot spot temperature or temperature difference” should be different. The image reconstructed by our method has clear edges, high contrast, and more obvious contours. Therefore, the image quality improvement of our method can improve the accuracy of equipment recognition, improve the accuracy of equipment area extraction, and improve the accuracy of defect diagnosis in the area to be detected in the infrared image diagnosis of equipment, thereby realizing accurate judgment of equipment operation.

6. Conclusions

Based on the background of the power Internet of Things, this paper presents a compression sensing super resolution method to solve the problem that the accuracy of infrared sensors is not enough to affect the accuracy of infrared diagnosis. A gradient norm ratio is used to improve the accuracy of blur kernel estimation, and a TwTVSI method is designed to achieve high-quality image reconstruction. The experimental part proves the effectiveness of the super-resolution method proposed in this paper.

In the era of the Internet of Things, the processing of infrared image data will be carried out automatically by computer. Our method can provide a good data base for the application of various algorithms, and has a broad application space. Firstly, the improvement of image quality by this method is helpful to the accurate operation of the image recognition algorithm. In the reconstructed image, the algorithm can identify more effective feature points and match them accurately, which is helpful to the classification and storage of infrared image data. Second, good image quality helps to improve the recognition accuracy of power equipment in the image. It can be seen from the experiment carried out in Section 5.4 that when image definition is insufficient, some transformer bushing cannot be accurately identified. Thirdly, the infrared image reconstructed by the super-resolution algorithm is more accurate in fault diagnosis, because the reconstruction results provide a good foundation for the image segmentation algorithm and can locate the equipment area more accurately. To sum up, the improvement of image quality by this method is helpful to the accurate realization of various practical applications based on infrared images of power equipment.

In addition, we believe that in the near future, when 5G communication, cloud computing and other technologies are further applied, the image data collected by infrared sensors will be analyzed on the cloud platform through cloud computing technology. After the operation and maintenance personnel obtain the access port of the cloud platform, they can accurately grasp the operation status of the equipment through mobile phones and computers anytime, anywhere, and then complete the operation and maintenance work according to the platform prompts. Our algorithm will have greater significance for image preprocessing, which could not only improve the accuracy of various algorithms based on equipment infrared images, but also provide clearer equipment observation conditions for operation and maintenance personnel.