Forest Single-Frame Remote Sensing Image Super-Resolution Using GANs

Abstract

Generative Adversarial Networks (GANs) possess remarkable fitting capabilities and play a crucial role in the field of computer vision. Super-resolution restoration is the process of converting low-resolution images into high-resolution ones, providing more detail and information. This is of paramount importance for monitoring and managing forest resources, enabling the surveillance of vegetation, wildlife, and potential disruptive factors in forest ecosystems. In this study, we propose an image super-resolution model based on Generative Adversarial Networks. We incorporate Multi-Scale Residual Blocks (MSRB) as the core feature extraction component to obtain image features at different scales, enhancing feature extraction capabilities. We introduce a novel attention mechanism, GAM Attention, which is added to the VGG network to capture more accurate feature dependencies in both spatial and channel domains. We also employ the adaptive activation function Meta ACONC and Ghost convolution to optimize training efficiency and reduce network parameters. Our model is trained on the DIV2K and LOVEDA datasets, and experimental results indicate improvements in evaluation metrics compared to SRGAN, with a PSNR increase of 0.709/2.213 dB, SSIM increase of 0.032/0.142, and LPIPS reduction of 0.03/0.013. The model performs on par with Real-ESRGAN but offers significantly improved speed. Our model efficiently restores single-frame remote sensing images of forests while achieving results comparable to state-of-the-art methods. It overcomes issues related to image distortion and texture details, producing forest remote sensing images that closely resemble high-resolution real images and align more closely with human perception. This research has significant implications on a global scale for ecological conservation, resource management, climate change research, risk management, and decision-making processes.

1. Introduction

Super-resolution restoration can convert low-resolution images into high-resolution images, providing more detail and information. This is critical for the monitoring and management of forest resources, as it allows more accurate detection and identification of different characteristics in forests, such as tree species, forest health status and forest cover type, which can help monitor forest vegetation, wildlife and potential destructive factors such as forest fires, illegal logging and tree diseases. This helps to identify problems early and take the necessary conservation measures to reduce the loss of forest resources. High-quality image data are the basis for scientific research to better understand the complexity of forest ecosystems, reduce forest destruction, protect ecosystems and improve the sustainable use of forest resources.

Image super-resolution technology refers to the technique of enlarging the resolution of low-resolution degraded images through technical means, thereby obtaining high-resolution, clearer images. It is one of the important technologies in the fields of computer vision and image processing. This technology not only enhances the visual perception of images but also plays a significant role in medical imaging and single-frame remote sensing imaging. Moreover, image super-resolution technology contributes to various computer vision tasks such as object detection and semantic segmentation.

With the continuous evolution of deep learning technology, the prevailing image super-resolution models now heavily rely on deep learning. The development of deep learning-based image super-resolution models has roughly progressed in tandem with the architecture of mainstream classification models. The key milestones in this progression are as follows: In 2014, Dong et al. pioneered the integration of deep learning methods into the field of image super-resolution by introducing the SRCNN model [1], based on AlexNet. This marked the initial application of convolutional neural networks to image super-resolution. Subsequently, in 2016, they further introduced the FSRCNN model [2], gradually deepening the network layers. In 2017, KIM et al. introduced VDSR [3], which was the first model to address image super-resolution reconstruction problems using residual deep networks. The core idea was that by learning the high-frequency residual information between high-resolution and low-resolution images, super-resolution could be achieved. This is because the low-frequency information present in low-resolution images is almost identical to the low-frequency information conveyed by high-resolution images. To address the issue of insufficient utilization of hierarchical features of LR (low-resolution) images during the reconstruction process, ZHANG et al. proposed the RDN (residual dense network) [4], which combined multiple residual dense blocks in a cascading manner. Subsequently, Dong et al. also built upon the principles of residual networks and introduced the EDSR [5] model based on VDSR. Following this, Ledig et al. applied Generative Adversarial Networks (GANs) [6] to the field of super-resolution, combining them with residual networks to introduce SRGAN [7]; this marks the initial exploration of Generative Adversarial Networks in the field of image super-resolution.

GANs exhibit exceptional fitting capabilities and play a pivotal role in the field of computer vision. They consist of a generator and a discriminator. In the context of image super-resolution, the generator’s primary function is to extract image features from the input low-resolution image, uncover higher-frequency details, and, at the network’s end, generate a high-resolution image. The generator is continuously optimized by adjusting gradients based on the output from the discriminator, with the aim of minimizing the loss function to enhance its generation performance. The discriminator is responsible for assessing the authenticity of the input high-resolution images, essentially serving as a binary classifier that outputs corresponding classification probabilities. The optimization objective for the generator is to render the discriminator incapable of distinguishing between the original high-resolution images and those generated by the generator. In contrast, the optimization goal for the discriminator is to accurately differentiate between the original high-resolution images and the generator’s output. These two components engage in a strategic game where the performance of both the generator and the discriminator improves during training, ultimately reaching a Nash equilibrium [8]. In recent years, a series of image super-resolution models based on Generative Adversarial Networks have emerged, such as ESRGAN, RankSRGAN, leading up to Real-ESRGAN, introduced in 2021.

In order to better serve the restoration of single-frame remote sensing images for forests and, on a global scale, enhance their role in ecological conservation, resource management, climate change research, risk management, and decision making, we have designed a single-frame remote sensing image super-resolution model based on Generative Adversarial Networks.

Our key contributions are summarized as follows:

• We introduce MSRB as a feature extraction component to enhance feature extraction capabilities by obtaining image features at different scales. MSRB constructs a dual-branch network, where different branches use distinct convolution kernels. These branches share information with each other, enabling the adaptive detection of image features at different scales.
• We propose GAM Attention and incorporate it into the VGG network to capture more precise feature dependencies in both spatial and channel domains.
• We apply Meta ACONC as the activation function within the VGG network. By dynamically learning the parameters of the Meta ACONC activation function for each neuron, it is designed to enhance network feature representation. Additionally, Ghost convolution is employed to optimize the convolution layers in the network, reducing network parameters and computational complexity.
• We conduct a multitude of comparative experiments with mature and advanced models. When tested on the high-resolution DIV2K dataset and the single-frame forest remote sensing image dataset LOVEDA, our proposed model outperforms some mainstream models in terms of perceived image quality. It exhibits higher image realism, with an improvement of 0.709/2.213 dB in PSNR and an increase of 0.032/0.142 in SSIM; LPIPS shows a decrease of 0.03/0.013 compared to SRGAN, while performing on par with Real-ESRGAN in terms of metrics. Significantly, it achieves faster processing speeds. These results demonstrate that our proposed model efficiently attains competitive performance in the context of single-frame forest remote sensing images.
• This model contributes to improving the quality and information content of remote sensing images, providing a powerful tool for better understanding and preserving Earth’s forest ecosystems.

2. Related Work

Traditional image super-resolution algorithms can be categorized into three main types: 1—Interpolation-based super-resolution algorithms, such as bicubic interpolation [9] and nearest-neighbor interpolation [10]. These methods rely on image interpolation techniques, where interpolation functions or kernel methods are used to estimate unknown pixel values from known pixels. Each known data point is used for interpolation, and the values of the interpolated pixels are calculated to minimize image noise [11] and eliminate blurriness through image restoration techniques. 2—Degradation model-based super-resolution algorithms [12,13], such as iterative back-projection, convex-set projection, and maximum a posteriori estimation. These algorithms are based on specific degradation models and involve techniques like image deblurring and reconstruction. 3—Learning-based super-resolution algorithms [14], including manifold learning and sparse coding methods. Modern image super-resolution algorithms often leverage deep learning techniques, given the continuous development of deep learning technology.

Currently, most single-frame image super-resolution reconstruction techniques are researched using deep learning methods and have achieved significant results. There are three common approaches: 1—Early Convolutional Neural Network (CNN)-based Super-Resolution Methods. These methods include shallow CNNs, residual networks, recursive neural networks, and dense convolutions. Examples include SRCNN, ESDR, and SEDenseNet [15]. Early shallow convolutional neural network models typically had a network depth of fewer than five layers, and their network structures were relatively simple. These models did not incorporate an extensive range of network design strategies. However, in comparison to traditional image super-resolution methods, they managed to achieve a noticeable improvement in reconstruction quality. As such, they played a pioneering role in the development of deep learning-based image super-resolution. 2—Prominent and Promising GAN-based Super-Resolution Methods. Notably, the SRGAN (Super-Resolution Generative Adversarial Network) belongs to this category. GAN-based methods have been widely applied and have shown great potential in achieving high-quality super-resolution results. 3—Recent Transformer-based Super-Resolution Methods. Transformer-based methods have gained popularity in low-level computer vision tasks, including super-resolution, with models like IPT [16]. Transformer-based methods have significantly improved image reconstruction quality and generally outperform CNN-based methods. For instance, the IPT network comprises three main components: a head for feature extraction from input degraded images, an encoder-decoder, and a Transformer, all working together to reconstruct the output image. To maximize the potential of Transformers, IPT uses a large dataset of degraded image pairs, constructed from the ImageNet dataset, for model training. However, Transformer-based methods are still in the development phase and primarily focus on enhancing the quality of network reconstruction, with high computational costs, making them less practical efor some real-world applications.

GAN-based methods leverage the adversarial structure of GANs to enhance the realism of network reconstruction, making them a preferred solution in the current state of image super-resolution. They perform well for overall image enhancement when fine details are not the primary concern. However, this method still has some drawbacks [17], including unstable model training, significant parameter fluctuations, slow model convergence, and the presence of artefacts in the generated images. Therefore, when using GAN-based methods, it is crucial to focus on reconstructing image details and employ appropriate strategies to build lightweight networks that ensure stable training.

3. Method

3.1. Method Overview

This paper introduces an image super-resolution model based on Generative Adversarial Networks. For the generator, we incorporate Multi-Scale Residual Blocks (MSRB) as a critical feature extraction component. Further details can be found in Section 3.2. Subsequently, we provide a detailed introduction to the loss functions used by the generator, as described in Section 3.3. Regarding the discriminator, we propose a novel attention mechanism known as GAM Attention, which is integrated into the VGG network. A comprehensive explanation can be found in Section 3.4.1. Additionally, we employ the adaptive activation function, Meta ACONC, as the activation function within the VGG network. Furthermore, we use Ghost convolution to optimize the convolution layers within the network, thereby reducing network parameters and computational complexity. For more in-depth insights, please refer to Section 3.4.2.

3.2. Multi-Scale Residual Block (MSRB)

Removing Batch Normalization (BN) layers has been shown to improve network performance and reduce computational complexity when dealing with PSNR-oriented tasks [18]. BN normalizes input data at different layers of the model during training, but when the statistical characteristics of the training and test datasets differ significantly, it can introduce artefacts that affect the visual quality of SR model-generated images. For very deep networks trained within a GAN framework, removing BN layers can improve robustness, reduce computation, and save memory.

In this paper, it is proposed to apply multi-scale residual block (MSRB) [19] to replace the common residual block commonly used in a feature extraction network to obtain image features of different scales to improve feature extraction capability. Based on residual blocks, we introduced convolution kerbs of different sizes to detect image features of different scales adaptively. Simple series of features of different scales would result in insufficient utilization of local features, so MSRB chose to make these features interact in parallel with each other to obtain the most effective image information. We constructed a double-bypass network, with different convolution kerbs used for different bypasses. In this way, the information between these bypasses can be shared with each other, making it possible to detect image features at different scales. Figure 1 shows the basic architecture of multi-scale residual blocks. The input is the 64 layers feature matrix after convolution, and the convolution kernel of 3 × 3 and 5 × 5 is used to generate feature graphs of different sizes. After the PreLU activation layer, the feature matrix of 128 layers is concat, and the same convolution operation is performed. After the concat is formed into a 256 layers feature matrix, 1 × 1 convolution is used to reduce the dimension, which can improve the feature abstraction ability and improve the expression ability of the network. The output of each MSRB is used in the generator as a hierarchical feature for global feature fusion. Finally, all these features are sent to the up-sampling module to generate a high-quality image, as shown in Figure 2 for the generator using MSRB.

Figure 2 illustrates the generator design that incorporates Multi-Scale Residual Blocks (MSRB). The generator consists of four key components: input, feature extraction, output, and up-sampling. In the input component, low-resolution images are up-sampled from 3 channels to 64 using a 9 × 9 convolution layer. A Parametric Rectified Linear Unit (PreLU) activation function introduces non-linearity. The feature extraction component comprises a MSRB, responsible for feature extraction. In the output component, a 3 × 3 convolution is applied for preprocessing the features without changing the number of channels. This preprocessing enriches feature representation, aiding in the better restoration of high-resolution information. In the up-sampling component, the channel count is initially increased through convolution to facilitate the learning of more features, enhancing feature richness. PixelShuffle is then used for up-sampling, employing a layer-by-layer incremental approach to upscale by a factor of two each time. This incremental approach helps the network gradually learn complex feature representations without introducing very high resolutions immediately, which could make the network more challenging to train due to the handling of larger feature maps within a single layer.

3.3. Generator Loss Function

The definition of the perceptual loss function is crucial for the generator network. Traditional super-resolution networks use pixel-wise loss functions such as Mean Squared Error (MSE) loss. However, images generated using MSE-based loss functions tend to be overly smooth in texture details. GAN-based image super-resolution algorithms typically use perceptual loss, which combines content loss (VGG loss) and adversarial loss (GAN loss). The complete loss function is as follows:

$$L^{SR}=L_{MSE}^{SR}+{\lambda }_{Gen}{L}_{Gen}^{SR}+{\lambda }_{Vgg}{L}_{Vgg}^{SR}$$

(1)

The MSE loss, which has been traditionally emphasized, can be expressed as:

$$L_{MSE}^{SR}=\frac{1}{r^{2}WH}{\sum }_{x=1}^{rW}{\sum }_{y=1}^{rH}{{[I}_{x,y}^{HR}-G_{{\theta }_G}{\left(I^{LR}\right)}_{x,y}]}^2$$

(2)

where, $I^{LR}$ is the low-resolution image corresponding to the original high-resolution image, W and H are the width and height of the low-resolution image, respectively, r is the subsampling multiple from the original high-resolution image to the low-resolution image, $G_{{\theta }_G}$ is the generator network constructed by the parameter ${\theta }_G$, and $G_{{\theta }_G}\left(I^{LR}\right)$ is the reconstructed image.

GEN loss can make the generated high-resolution image as much as possible visually similar to the ground truth value, where the function of the discriminator is the probability distribution to judge whether the image generated by generator is real, and its formula is expressed as Equation (3):

$$L_{GEN}^{SR}={\sum }_{n=1}^N-log{D}_{{\theta }_D}[G_{{\theta }_G}\left(I^{LR}\right)]$$

(3)

VGG loss represents the per-pixel loss between the high-resolution original image and the deep features of the generated image. It is a regularized loss function based on the total variational norm, and this regularized loss tends to preserve the smoothness of the image, and its formula is expressed as Equation (4):

$$L_{VGG}^{SR}=\frac{1}{W_{i,j}{H}_{i,j}}{\sum }_{x=1}^{W_{i,j}}{\sum }_{y=1}^{H_{i,j}}{\{{{\Phi }_{i,j}\left(I^{HR}\right)}_{x,y}-{\Phi }_{i,j}{\left[G_{{\theta }_G}\left(I^{LR}\right)\right]}_{x,y}\}}^2$$

(4)

${\Phi }_{i,j}$ represents the feature graph obtained from the $i$th maximum pooled layer of the $j$th layer convolution of the VGG19 network. Ultimately, its essence is the Euclidean distance between the feature of the reconstructed image representing $G_{{\theta }_G}\left(I^{LR}\right)$ and the reference image $I^{HR}$.

The weight $\lambda$ of GEN loss and VGG loss affects the balance between the deep perception quality of the reconstructed image and the PSNR index. The larger the weight of the two, the more realistic the reconstructed image will be, but the overall distortion will be aggravated. A too-small weight of the two will make the generated image too smooth.

3.4. Discriminator Design

3.4.1. Novel Attention Mechanism (GAM Attention)

Attention mechanisms assign varying weights to different parts of the network input, extracting essential information to assist the model in making accurate judgments. They achieve this without significantly increasing computational or storage costs, which is why attention mechanisms are widely used. Attention mechanisms can generally be categorized into three domains: spatial, channel, and mixed. Commonly used channel domain attention mechanisms include SE-NET (Squeeze-and-Excitation Network) [20], ECA-NET (Efficient Channel Attention Network) [21], while mixed domain attention mechanisms include CBAM (Convolutional Block Attention Module) [22] and DA-Net (Dual Attention Network) [23]. In this paper, we introduce an attention mechanism module based on the VGG 19 architecture, commonly used in the discriminator, to fully exploit feature correlations and overcome the limitation of the original model in global information extraction. Due to the unique nature of the generator network, controlling the number of feature extraction layers can help strike a favorable balance between training speed and the quality of generated results.

We propose a novel attention mechanism called GAM Attention, which enhances neural network performance by reducing information diffusion and amplifying global interactive representations. GAM Attention includes both channel and spatial domain attention, providing more comprehensive and reliable attention information to guide a more reasonable allocation of computational resources. It is added to the discriminator at the end of the VGG network’s convolutional layers, enabling it to obtain more precise feature dependencies in both spatial and channel domains. Its structure is illustrated in Figure 3.

The channel attention module takes the feature matrix F₁ obtained after VGG convolution as its input feature map, where the input feature map size is set to C × W × H, with H and W representing the input feature’s height and width, and C representing the number of channels. It undergoes a dimension transformation (C × W × H to W × H × C) and then feeds the dimension-transformed feature map into a two-layer Multi-Layer Perceptron (MLP) to amplify cross-dimensional channel-space dependencies. After processing, it is transformed back to dimension C × H × W and subjected to sigmoid processing to obtain the attention weight feature map M₁, which is then multiplied with the input feature map to obtain the new feature map F₂. The channel attention structure is depicted in Figure 4.

The spatial attention module takes the feature map F₂ obtained from the channel attention module as its input feature map. It reduces its channel count to C/r (where r is set to 4) using a 7 × 7 convolution to reduce computational load. It then increases the channel count again using another 7 × 7 convolution to match the original channel count. Finally, it employs sigmoid processing to obtain the attention-weight feature map M₂. After obtaining the weights, they are multiplied with the output feature layer F₂ from the channel attention module to obtain the final dependency-enhanced feature matrix F₃. The spatial attention structure is shown in Figure 5.

3.4.2. Network Performance and Training Enhancement

Activation functions in neural networks serve to introduce non-linearity and combine features effectively, thereby enhancing the model’s representational power. Traditional activation functions assume the same activation form for all neurons, limiting feature expressiveness. To address this limitation, we introduce the Meta ACONC (Adaptive Activation Function) to design different activation forms for each neuron, allowing each neuron to learn whether to activate or not, thereby enhancing the network’s feature representation capability. In this paper, we replace the PreLU and Leaky ReLU activation functions in the VGG network with the Meta ACONC activation function.

The ReLU activation function has been commonly used due to its excellent properties such as non-saturation and sparsity for a long time. However, it can lead to neuron death to some extent. Inspired by the Swish activation function obtained through NAS (Neural Architecture Search) [24], we propose a new activation function called the ACON (ACtivation functiON) series by exploring the smooth approximation principle between Swish and ReLU. Since ReLU is a special form of Maxout [25], we use the Maxout family of activation functions to derive a new type of activation function. The basic principle is as follows:

Our commonly used ReLU activation function is essentially a MAX function, and a smooth differentiable variant of the MAX function is called the Smooth Maximum. Its formula is shown in Equation (5):

$$S_{\beta }\left(x_1{,\dots ,x}_n\right)=\frac{\sum _{i=1}^n{x}_i{e}^{\beta x_i}}{\sum _{i=1}^n{e}^{\beta x_i}}$$

(5)

where β represents the coupling coefficient. When $\beta \to \mathrm{\infty },S_{\beta }\to max$, and the Smooth Maximum becomes the standard MAX function, which is non-linear (activating). When $\beta \to 0,S_{\beta }\to mean$, making the Smooth Maximum an arithmetic average operation, which is linear (non-activating). When $n=2$, the maximum function is $\mathrm{m}\mathrm{a}\mathrm{x}\left({\eta }_a\right(x),{\eta }_b(x\left)\right)$, and it can be expressed as a function in Sigmoid as follows:

$$S_{\beta }\left({\eta }_a\left(x\right),{\eta }_b\left(x\right)\right)=\left({\eta }_a\left(x\right)–{\eta }_b\left(x\right)\right)\times \sigma \left[\beta \left({\eta }_a\left(x\right)–{\eta }_b\left(x\right)\right)\right]+{\eta }_b\left(x\right)$$

(6)

When ${\eta }_a\left(x\right)$ = x, ${\eta }_b=0$ we obtain the Swish function as a smooth approximation of ReLU. By selecting different values for ${\eta }_a\left(x\right),{\eta }_b\left(x\right)$, we define different ACON activation functions, as shown in

Table 1. Various scenarios for the ACON (Adaptive Contextual Normalization) activation function.

		$\mathbf{M}\mathbf{a}\mathbf{x}\mathbf{o}\mathbf{u}\mathbf{t}\mathbf{ }\mathbf{F}\mathbf{a}\mathbf{m}\mathbf{i}\mathbf{l}\mathbf{y}$	$\mathbf{A}\mathbf{C}\mathbf{O}\mathbf{N}\mathbf{ }\mathbf{F}\mathbf{a}\mathbf{m}\mathbf{i}\mathbf{l}\mathbf{y}$
${\eta }_a\left(x\right)$	${\eta }_b\left(x\right)$	$\mathrm{m}\mathrm{a}\mathrm{x}\left({\eta }_a\right(x),{\eta }_b(x\left)\right)$	$S_{\beta }\left({\eta }_a\left(x\right),{\eta }_b\left(x\right)\right)$
$x$	$0$	$\max \left(x,0\right):ReLU$	$ACON-A\left(Swish\right):$ $x\sigma \left(\beta x\right)$
$x$	$px$	$\max \left(x,px\right):PReLU$	$ACON-B:$ $\left(1-p\right)x·\sigma \left(\beta \left(1-p\right)x\right)+px$
$p_{1}x$	$p_{2}x$	$\max \left(p_{1}x,p_{2}x\right)$	$ACON-C:$ $\left(p_1-p_2\right)x·\sigma \left(\beta \left(p_1-p_2\right)x\right)+p_{2}x$

:

Subsequently, based on the ACONC activation function, we learn to activate (non-linear) or not activate (linear) by simply maintaining the switching factor β. This led to Meta ACONC, which explicitly optimizes the factor and demonstrates significant improvements. The design space of the adaptive function includes layer-wise, channel-wise, and pixel-wise spaces. Here, we choose the channel-wise space, where we first calculate the means separately for the H and W dimensions and then use two convolution layers to ensure that every channel shares a weight for all pixels. The formula is shown in Equation (7):

$${\beta }_C=\sigma W_1{W}_2{\sum }_{h=1}^H{\sum }_{w=1}^W{x}_{c,h,w}$$

(7)

To maintain a balance between the generator and the discriminator and avoid increasing the model complexity of the generator G, which is crucial for practical applications where the prediction process is the most commonly used scenario, we conducted an analysis of the generator G structure. As a result, we did not incorporate attention mechanisms into the generator G, nor did we alter the activation functions within the generator.

Due to the VGG19 network’s high frequency of convolutions with fixed parameters such as kernel size, stride, padding, etc., we applied optimization using Ghost convolutions [26] to replace the original convolutional layers in the VGG19 network. This approach effectively reduces the number of parameters and computational costs while ensuring that the model’s overall stability and results are minimally impacted. Figure 6 illustrates the working principle of Ghost convolutions.

Ghost convolution operates in three steps:

Initially, it convolves the input feature map. However, unlike regular convolutions with an output channel size of N, Ghost convolution first obtains a preliminary feature map with an output channel size of N/2. Next, it individually convolves each channel of the N/2 feature map in groups of size N/2. Finally, it concatenates the results of the grouped convolutions with the N/2 feature map.

Ghost convolution achieves the reduction in computational load by employing fewer convolutional operations during the process. Specifically, it reduces the number of convolutional kernels involved by half, effectively halving the computational workload.

Figure 7 shows the improved discriminator model in this paper. The model is designed in a concatenated manner, with Ghost convolution layers and Meta ACONC activation functions appearing alternately. As the depth of the convolution layers gradually increases, hierarchical feature extraction is achieved, and an attention mechanism is applied at the end to fully explore feature relevance.

3.5. Evaluation Metrics

Three super-resolution image quality evaluation metrics were used in this study: Peak Signal-to-Noise Ratio (PSNR) [27], Structural Similarity Index (SSIM) [28], and Learned Perceptual Image Patch Similarity (LPIPS) [29].

PSNR objectively evaluates the level of distortion in the reconstructed image by calculating the error between the generated image (I_SR) and the original high-resolution image (I_HR). PSNR primarily depends on Mean Squared Error (MSE) and is expressed as follows Equation (8):

$$PSNR=10\times {log}_{10}\left(\frac{{MAX}^2}{MSE}\right)$$

(8)

where MAX represents the maximum pixel value in the I_HR (i, j) image. SSIM measures the structural similarity between the reference image (I_HR) and the generated image (I_SR) in terms of brightness, contrast, and structure. These three aspects are independent and do not affect each other. SSIM is typically expressed as follows, in Equation (9):

$$SSIM={\left[l\right(I_{HR},I_{SR}\left)\right]}^{\alpha }{\left[c\right(I_{HR},I_{SR}\left)\right]}^{\beta }{\left[s\right(I_{HR},I_{SR}\left)\right]}^{\gamma }$$

(9)

where α, β, and γ are weighting parameters used to adjust the importance of brightness, contrast, and structure, with commonly used values being α = β = γ = 1.

$$SSIM=\frac{(2{\mu }_{I_{HR}}{\mu }_{I_{SR}}+C_1)(2{\sigma }_{I_{HR}{I}_{SR}}+C_2)}{({\mu }_{I_{HR}}^2+{\mu }_{I_{SR}}^2+C_1)({\sigma }_{I_{HR}}^2+{\sigma }_{I_{SR}}^2+C_2)}$$

(10)

Among them, the ${\mu }_{I_{HR}}{,\mu }_{I_{SR}}$, respectively said $I_{HR,}{I}_{SR}$ average, ${\sigma }_{I_{HR}},{\sigma }_{I_{SR}}$, respectively said $I_{HR,}{I}_{SR}$ standard deviation, ${\sigma }_{I_{HR}{I}_{SR}}$ said $I_{HR},I_{SR}$ covariance, $C_1$ and $C_2$ are constants added to avoid instability in the calculation.

LPIPS is a perceptual metric based on human perception and is thus well-suited to reflect subjective image quality. Smaller LPIPS values indicate better perceived image quality. The metric uses a pre-trained deep convolutional neural network to extract features from reference and distorted images, calculates the L2 distance in the deep feature space, and assesses perceptual similarity. The expression for LPIPS is shown as follows Equation (11):

$$LPIPS=d\left(I_{HR,}{I}_{SR}\right)={\sum }_l\frac{1}{H_l{W}_l}{\sum }_{h,w}{\Vert w_l\odot (y_{I_{HR}hw}^l-y_{I_{SR}hw}^l)\Vert }_2^2$$

(11)

where $l$ represents the $l$ layer of the convolutional neural network, and $y_{I_{HR}}^l,y_{I_{SR}}^l$ represent the features extracted from $I_{HR,}{I}_{SR}$ normalized along the channel dimension.

4. Experimental Comparison and Analysis

4.1. Experimental Environment and Training Data

This experiment was conducted on a Linux server equipped with an NVIDIA 2080 Ti GPU with 11 GB of VRAM, 16 GB of RAM, and the following software stack: PyTorch 1.8, Python 3.8.5, CUDA 11.0, and cuDNN 8.0.4. The training data consisted of two parts: the publicly available DIV2K dataset [30] and the LoveDA dataset of single-frame remote sensing images in hilly terrain [31]. The DIV2K dataset contains 900 high-definition images (2K resolution), with 800 images used for training and 100 for testing. The LoveDA dataset, created by the RSIDEA team at Wuhan University, comprises a significant collection of single-frame remote sensing images. This dataset is constructed from 0.3-m imagery sourced from Google Earth, captured in July 2016, covering a total geographical area of 536.15 square kilometers in the regions of Nanjing, Changzhou, and Wuhan. All images are selected from undeveloped hilly areas and include abundant wild vegetation, water bodies, and sporadic buildings. After undergoing geometric registration and preprocessing, each region is covered by non-overlapping 1024 × 1024 images, as illustrated in Figure 8. For this study, 1000 single-frame remote sensing images were selected from LoveDA and used as a dataset for training the forest single-frame remote sensing images. Another 100 images were chosen as a test set.

4.2. Training Process

In the training process, the adaptive moment estimation (Adam) optimizer [32] was chosen. The number of residual blocks in the generator G was set to N = 16, ${\lambda }_{Gen}=0.001, {\lambda }_{Vgg}=0.000002$, and a 4× upscaling factor was used. The batch size for training was set to 4, and the initial learning rate for both the generator and discriminator was set to 0.0002. The maximum number of training epochs was set to 100. During training, the low-resolution images required for training were obtained by applying a series of operations to the original high-resolution images, including scaling, distortion, flipping, and adding gray bars, all resulting in a unified size of 96 × 96.

The training process was divided into two parts. The first part involved training the discriminator. First, the original high-resolution image $I_{HR}$ was fed into the discriminator to obtain an adversarial loss based on $I_{HR}$’s authenticity. Then, the generated high-resolution image $I_{SR}$ was input to the discriminator to obtain a second adversarial loss. Both losses were optimized using Adam gradient descent. The second part focused on training the generator G. The generator aimed to produce a high-resolution image $I_{SR}$. The loss consisted of the mean squared error (MSE) loss between $I_{SR}$ and $I_{HR}$, an adversarial loss between $I_{SR}$ and the ground truth, and a content loss between $I_{SR}$ and $I_{HR}$. These losses were summed, and the sum was optimized using Adam gradient descent.

4.3. Comparative Experiments

In this section, we conduct comparative experiments between our model and established models, namely VDSR, EDSR, SRGAN, and Real-ESRGAN. We use the DIV2K and LoveDA training datasets to train our model for 4× super-resolution, and the corresponding weights are obtained. Subsequently, we downscale the resolution of 100 test images from each dataset by a factor of four and evaluate the performance of each model in predicting the super-resolved images. We then combine the predicted results with the original test dataset to obtain the three evaluation metrics required in this study. The average evaluation metrics from this experiment are presented in

Table 2. Comparative experimental results of image metrics for the VDSR, EDSR, SRGAN, and Real-ESRGAN models at 4× image super-resolution.

Dataset	Algorithm	VDSR	EDSR	SRGAN	Real-ESRGAN	Ours
DIV2K	PSNR/dB	24.843	25.141	27.353	28.374	28.062
	SSIM	0.542	0.597	0.757	0.803	0.789
	LPIPS	0.519	0.492	0.343	0.315	0.313
LOVEDA	PSNR/dB	21.349	21.947	23.685	25.907	25.898
	SSIM	0.351	0.383	0.522	0.673	0.664
	LPIPS	0.593	0.560	0.407	0.399	0.394

. The experiments demonstrate that, in terms of evaluation metrics, the model proposed in this paper outperforms SRGAN with an increase in PSNR values of 0.709/2.213 dB, an improvement in SSIM of 0.032/0.142, and a decrease in LPIPS of 0.03/0.013. These results are on par with those achieved by Real-ESRGAN.

Additionally, we have collected FPS (Frames Per Second) data for each model. We conducted experiments in which 100 test images from both datasets were processed by each model in a loop to generate 4× super-resolved images. The data were measured after the models stabilized. It is worth noting that this data may vary depending on the experimental environment. The data are presented in

Table 3. Test results for the average frames per second (FPS) of the four models VDSR, EDSR, SRGAN, and Real-ESRGAN.

Dataset	VDSR	EDSR	SRGAN	Real-ESRGAN	Ours
DIV2K	118.4	111.19	81	22.7	44
LOVEDA	130.61	121	90.43	31.4	50.4

. The results from both sets of experiments demonstrate that the model we propose achieves results in single-frame forest remote sensing images that are comparable to state-of-the-art methods while maintaining efficiency.

Selecting one image from the test dataset and enlarging it for comparison, the image reconstruction results of our proposed algorithm and the comparative algorithms are shown in Figure 9. Through the comparative experiments and relevant metrics, it is evident from the image results that VDSR and EDSR reconstructions lack many fine details, resulting in overly blurry and unrealistic images. SRGAN enhances the realism of the images but exhibits noticeable distortion. Based on LPIPS values and visual perception, it can be concluded that our proposed algorithm generates images that are more in line with human perception, making single-frame remote sensing image reconstructions closer to real high-resolution images in terms of perception and content. Although slightly lower in some parameters compared to Real-ESRGAN, our algorithm provides higher FPS and a more stable training process. In some scenarios, Real-ESRGAN can exhibit distorted lines and excessive alterations compared to the original image, possibly due to aliasing issues. The super-resolution processing makes the separation between different trees more apparent, allowing observers to easily distinguish each tree, thus facilitating the identification of tree species and distribution. Tree crown shapes are clearly distinguishable, and one can clearly see the size and shape of each tree’s canopy, which aids in the study and monitoring of forest ecosystems. Analyzing the evaluation metrics, it is evident that our algorithm optimizes PSNR, SSIM, and LPIPS differently based on the training dataset used. Additional dataset test results are depicted in Figure 10.

4.4. Ablation Experiments

To thoroughly evaluate the performance enhancement brought by our proposed attention mechanism, GAM Attention, we conducted experiments by incorporating this attention mechanism into algorithms like VDSR, EDSR, and SRGAN, which do not inherently possess attention mechanisms. We trained these modified models for 4× super-resolution using the DIV2K and LoveDA training datasets to obtain their respective weights. We then downscaled the resolution of 100 test images from each dataset by a factor of four and evaluated the performance of each model in predicting the super-resolved images. We combined the predicted results with the original test dataset to obtain the three evaluation metrics required in this study. The average evaluation metrics are presented in

Table 4. Results of ablation experiments on the addition of the GAM Attention mechanism to the VDSR, EDSR, and SRGAN models. The results before the addition can be referred to in Table 2.

Dataset	Algorithm	VDSR + GAM	EDSR + GAM	SRGAN + GAM
DIV2K	PSNR/dB	24.955	25.268	27.361
	SSIM	0.560	0.611	0.764
	LPIPS	0.508	0.480	0.334
LOVEDA	PSNR/dB	21.226	22.101	23.775
	SSIM	0.339	0.392	0.513
	LPIPS	0.583	0.553	0.394

. The results before adding the attention mechanism can be referred to in Table 2.

To validate the consistently positive effects brought by the attention mechanism and activation functions, ablation experiments were conducted. The experiments involved gradually adding three components, namely Basic G, GAM, and Meta ACONC, to the generator based on the MSRB backbone network and the conventional VGG network for the discriminator. These experiments were conducted by training the models for 4× super-resolution using the DIV2K and LoveDA training datasets to obtain their respective weights. The resolution of 100 test images from each dataset was downscaled by a factor of four, and the performance of each model in predicting super-resolved images was evaluated. The predicted results were then combined with the original test dataset to obtain the three evaluation metrics required for this study. The average evaluation metrics are presented in

Table 5. Test results for the generator obtained through ablation training on the basic discriminator. “BasicD” represents the basic discriminator using a conventional VGG network.

Dataset	Algorithm	Basic D	Basic D + GAM	Basic D + GAM + Meta ACONC
DIV2K	PSNR/dB	27.829	27.924	28.062
	SSIM	0.777	0.785	0.789
	LPIPS	0.322	0.316	0.313
LOVEDA	PSNR/dB	25.169	25.467	25.898
	SSIM	0.597	0.629	0.664
	LPIPS	0.382	0.380	0.394

.

The results from the ablation experiments show that with the continuous addition of improvement methods, all evaluation metrics on both datasets exhibit ongoing optimization, indicating a significant improvement in reconstruction. Selecting one image from the test dataset and enlarging it for comparison, the image reconstruction results of our improved algorithm and the original algorithm are shown in Figure 11. It can be observed that as the algorithm components are progressively added, the generated images exhibit clearer texture details, smoother transitions, and a substantial increase in realism.

5. Conclusions

This paper addresses the restoration of single-frame forest remote sensing images and introduces a novel image super-resolution model based on Generative Adversarial Networks (GANs). By avoiding the use of Batch Normalization layers (BN), improving residual blocks, enhancing activation functions, and incorporating attention mechanisms, this model reduces the computational load of convolutional layers. It effectively mitigates issues related to image distortion and texture details. Moreover, the generated images closely align with human perception and exhibit content that is more similar to real high-resolution images.

When tested on high-resolution DIV2K and single-frame forest remote sensing images from the LOVEDA dataset, the model proposed in this paper outperforms some mainstream models in terms of image perceptual quality, exhibiting higher realism. The model’s performance metrics are on par with Real-ESRGAN, but it offers significantly improved speed. Enhancing forest remote sensing image super-resolution is pivotal for improving the efficiency and sustainability of forest resource management. This approach provides more information and data, facilitating better protection and management of our forest resources. Our future endeavors will focus on enhancing model training and testing efficiency while ensuring reconstruction quality.