SpaceLight: A Framework for Enhanced On-Orbit Navigation Imagery

Abstract

In the domain of space rendezvous and docking, visual navigation plays a crucial role. However, practical applications frequently encounter issues with poor image quality. Factors such as lighting uncertainties, spacecraft motion, uneven illumination, and excessively dark environments collectively pose significant challenges, rendering recognition and measurement tasks during visual navigation nearly infeasible. The existing image enhancement methods, while visually appealing, compromise the authenticity of the original images. In the specific context of visual navigation, space image enhancement’s primary aim is the faithful restoration of the spacecraft’s mechanical structure with high-quality outcomes. To address these issues, our study introduces, for the first time, a dedicated unsupervised framework named SpaceLight for enhancing on-orbit navigation images. The framework integrates a spacecraft semantic parsing network, utilizing it to generate attention maps that pinpoint structural elements of spacecraft in poorly illuminated regions for subsequent enhancement. To more effectively recover fine structural details within these dark areas, we propose the definition of a global structure loss and the incorporation of a pre-enhancement module. The proposed SpaceLight framework adeptly restores structural details in extremely dark areas while distinguishing spacecraft structures from the deep-space background, demonstrating practical viability when applied to visual navigation. This paper is grounded in space on-orbit servicing engineering projects, aiming to address visual navigation practical issues. It pioneers the utilization of authentic on-orbit navigation images in the research, resulting in highly promising and unprecedented outcomes. Comprehensive experiments demonstrate SpaceLight’s superiority over state-of-the-art low-light enhancement algorithms, facilitating enhanced on-orbit navigation image quality. This advancement offers robust support for subsequent visual navigation.

1. Introduction

Space rendezvous and docking stands as a critical focal point of on-orbit services, enabling tasks such as spacecraft assembly and maintenance. These activities significantly contribute to prolonging the operational lifespan of space vehicles and enhancing their mission capabilities [1]. Visual navigation plays a pivotal role in on-orbit services, facilitating a range of tasks including attitude measurement [2], component identification [3], and visual multi-sensor fusion [4]. It is also critical in non-cooperative target detection [5], shape reconstruction of spacecraft [6], and formation flying [7,8], among various other domains. However, throughout the data acquisition process, on-orbit navigation images are unavoidably influenced by varying shooting conditions, giving rise to challenges such as extreme darkness, uneven illumination, and blurriness. These conditions pose obstacles for subsequent visual navigation tasks. Therefore, the imperative need for the development of an image enhancement framework for visual navigation is evident.

In the domain of image enhancement, numerous methods leveraging deep convolutional neural networks have been proposed and demonstrated efficacy. However, these methods predominantly find application in medical image enhancement [9,10], underwater image enhancement [11,12], remote sensing image enhancement [13,14], natural image enhancement [15], and other domains [16,17]. There is currently a lack of a dedicated method specifically tailored for enhancing on-orbit navigation images. Despite the visually appealing outcomes produced by the current methods, they lack originality and struggle with structural recovery, presenting a noteworthy drawback. In the context of visual navigation image enhancement, the primary objective is to fully restore the structural integrity of the spacecraft. Additionally, a majority of deep convolutional neural network algorithms necessitate paired image data. However, on-orbit images cannot simultaneously capture dark and bright images of the same visual scene. Consequently, we adopted non-paired learning. Under high-contrast conditions, the restoration of structure and texture details in extremely dark visual navigation images, especially in the absence of paired datasets and prior information, poses a formidable challenge. Furthermore, the predominant black deep-space background in visual navigation images exacerbates the difficulty of recovering image details in spacecraft structures within extremely dark areas. Given the characteristics of spacecraft with similar components, it is necessary to utilize image semantic information as a strong, key prior [18,19,20]. To this end, we employ a spacecraft semantic parsing network to generate prior attention maps. Upon further investigation, we observed that the Laplacian pyramid of gamma-corrected pre-enhanced images [21] encompasses numerous intricate structural features, which are challenging to restore in dark areas. This inspired us to define the global structural loss of the network.

The engineering project on which this study is based was launched into orbit in August 2022. This project comprises two satellites, named Innovation-16. The slave satellite captured images of the master satellite for visual navigation validation. Moreover, the project originated from the Shenzhou VII accompanying small satellite mission, which was designed to capture close-range and multi-angle images of the Shenzhou VII spacecraft. Importantly, it marked a significant achievement by capturing the inaugural real-life images of China’s orbiting spacecraft. In this study, we present a groundbreaking visual navigation image enhancement framework that incorporates a spacecraft semantic parsing attention network and a structural recovery enhancement network. This framework demonstrates the capability to effectively enhance navigation images, robustly restoring spacecraft structures. It addresses the inherent challenges of space navigation image degradation. Our contributions in this endeavor encompass the following key aspects:

• Proposing the first-ever framework for enhancing space navigation images. This paper pioneers the integration of on-orbit navigation images into image enhancement research and solves important issues in visual navigation engineering. It introduces an innovative framework that combines a spacecraft semantic parsing attention network with a structural recovery enhancement network to effectively restore detailed and structured navigation images
• Pioneering Application of Spacecraft Semantic Parsing Network. The introduction of a novel approach using a spacecraft semantic parsing network to generate attention-guided maps. These maps serve as inputs to the enhancement network, improving the utilization of prior information for more effective enhancement.
• Define the global structural loss in navigation image enhancement. It involves initially pre-enhancing the original image using gamma correction, followed by the extraction of its Laplacian pyramid to construct the global structure loss function. This approach enables the network to effectively recover the structure of extremely dark images.

Extensive experimentation substantiates the advantages of our approach in image quality generation, highlighting the practical value of the framework. Notably, our algorithm has successfully been applied to real visual navigation images. The subsequent sections of this paper follow this structure: Section 2 provides an exhaustive review of the pertinent literature within the realm of image enhancement. Section 3 commences with the introduction of the proposed spacecraft semantic segmentation network, followed by the delineation of the visual navigation image enhancement network, and concludes with an overview of the entire SpaceLight process. Section 4 is dedicated to the presentation of experiments and the analysis of results. In Section 5, we delve into a comprehensive study on component ablation and application analysis. Ultimately, Section 6 summarizes the research presented in this paper and the future directions for visual navigation advancements.

2. Related Work

Image enhancement is a key direction in image processing, and significant achievements have been made to date [22]. The current research on image enhancement primarily focuses on applications in underwater, remote sensing, medical imaging, and autonomous driving. The range of research methods includes both traditional approaches and methods based on deep learning [17].

The traditional methods for image enhancement predominantly rely on image and signal processing techniques, including histogram equalization [16], genetic algorithms, wavelet transforms, dark channel prior, and the Retinex model [23]. For example, histogram equalization enhances contrast and brightness by redistributing pixel values but may amplify noise and cause overexposure. Contrast-limited adaptive histogram equalization (CLAHE), introduced by SM Pizer [24], is widely adopted for its ability to distribute pixels beyond a threshold evenly, achieving gradual enhancement. LIME [25], an algorithm estimating illumination maps, stands as a classical method in image enhancement. Wavelet transforms [26] decompose images into different scale frequency bands, capturing fine structural details effectively. The Retinex model [27], inspired by human visual perception, enhances images by separating illumination and reflection components, frequently employed in underwater images to mitigate color distortion. Traditional algorithms, despite their straightforward implementation and process clarity, often necessitate manual parameter tuning, lack robustness, and involve complex processing steps. In uncertain imaging environments characterized by variable lighting and diverse spacecraft surface materials in visual navigation images, these methods frequently fail to deliver satisfactory results.

In recent years, learning-based methods have made significant strides in image enhancement, propelled by increased computational power and expanded datasets [28]. Learning-based techniques have demonstrated success in a range of image enhancement tasks including deblurring, deraining, dehazing, super-resolution, and beautification [29]. Notably, generative adversarial networks (GANs), introduced by Ian Goodfellow in 2014 [30], represent one of the most promising methodologies in this area. GANs effectively address the challenge of data generation through an adversarial training process between a generator and a discriminator. This method enables the production of highly realistic images that conform to predefined conditions [31]. Convolutional neural networks (CNNs) are extensively applied in image enhancement, where a single convolutional layer captures basic image features like textures, edges, and local characteristics. Multiple layers collaborate to learn high-dimensional abstract features, facilitating a hierarchical learning process that significantly enhances the understanding of image content. Learning-based image enhancement methods can be categorized into five types based on their learning strategies: supervised learning, semi-supervised learning, unsupervised learning, zero-shot learning, and reinforcement learning. Since 2017, advancements in computational power and algorithmic innovations have driven substantial growth in these methods. LLNet, proposed by KG Lore, represents a pioneering effort in adapting convolutional neural networks (CNNs) specifically for low-light image enhancement [32], utilizing a deep autoencoder for enhancement and denoising. KinD [33] decomposes images into illumination and reflectance maps based on the Retinex theory. It requires training on paired images captured under different exposure conditions for weak light image enhancement. Alongside LLNet [34], LightenNet [35], and Retinex-Net [36], KinD operates under a supervised learning framework, requiring extensive paired datasets. In contrast, DRBN [37] is a semi-supervised method that initially focuses on fidelity using paired datasets. In its subsequent stage, DRBN prioritizes perceptual quality, employing non-paired datasets to refine low-light image details. While supervised learning requires a significant amount of paired training data, unsupervised learning extracts features from unlabeled data. EnlightenGAN [38], a pioneering method in low-light image enhancement, utilizes non-paired images for feature learning. This approach incorporates self-regularization techniques to guide the training process while preserving the texture and structure of the images. ZeroDCE [39] and RUAS [40] exemplify zero-shot learning methods in image enhancement. ZeroDCE enhances images by training a non-linear curve mapping across multiple exposures, whereas RUAS combines a neural network structure search with Retinex theory-based prior constraints for low-light enhancement. It further employs an optimization process to solve for the overall network architecture and introduces a reference-free cooperative learning strategy. DeepExposure [41] leverages reinforcement learning to adjust local exposure variations in sub-images while designing global learning rewards. This method effectively balances local exposures, achieving a comprehensive exposure equilibrium across the image.

3. The Proposed Method

Unlike the traditional scenario of enhancing low-light images, visual navigation images predominantly consist of exceedingly dark backgrounds and feature high-contrast illumination. The backdrop predominantly features deep black space. In this challenging imaging context, restoring spacecraft structures within these dark regions poses a significant challenge. To address this, we introduce the SpaceLight model, specifically tailored for this unique application. The model seamlessly integrates two distinct sub-networks: the spacecraft semantic parsing network and the visual navigation image enhancement network. Detailed descriptions of each sub-network’s functionalities and their integration are meticulously expounded upon in the following sections.

3.1. Spacecraft Semantic Parsing Network

We believe that integrating semantic priors can produce more realistic structures. Thus, for the first time, we introduce a semantic segmentation network into the visual navigation enhancement domain to segment on-orbit images. Unlike conventional application scenarios, visual navigation images typically feature a predominantly black background. The use of a segmentation network facilitates rapid identification of target areas and precise enhancement. Given the characteristic of spacecraft having similar components [42,43], we propose a spacecraft semantic parsing network. This network processes images to generate attention maps, thereby enhancing target areas more effectively.

The spacecraft semantic parsing network employs the U-Net architecture, originally introduced by Olaf Ronneberger [44]. Initially designed for cell segmentation in medical images, U-Net has proven successful across various fields such as medical imaging, industrial segmentation, and autonomous driving. Recognized for its efficiency and adaptability, U-Net is particularly beneficial for scenarios requiring model training with limited datasets. Given the challenges of obtaining large on-orbit navigation datasets, we chose U-Net for its lightweight and straightforward structure. The architecture of U-Net, depicted in Figure 1, is U-shaped and includes both a contracting path and an expansive path. The contracting path performs convolution and downsampling to extract features effectively, while the expansive path uses upsampling and skip connections to merge detailed low-level information with high-level semantic information from earlier layers. This integration is crucial for enhancing the resolution restoration of the original image.

During training, we utilize the stochastic gradient descent to optimize the model parameters. For the computation of the energy function, a pixel-wise softmax operation is applied to the final feature map. This operation converts the raw pixel values into corresponding probability distributions. The deviation between these probability distributions and the actual labels is quantified using the cross-entropy loss function. This metric assists in adjusting the model parameters, gradually optimizing them to better align with the training data. The softmax function is defined as follows:

$$q_m\left(\mathbf{x}\right)=exp\left(a_m\left(\mathbf{x}\right)\right)/\left(\sum _{m^{\prime }=1}^{M}exp\left(a_{m^{\prime }}\left(\mathbf{x}\right)\right)\right)$$

(1)

$a_m\left(\mathbf{x}\right)$ represents the activation value of $\mathbf{x}$ in feature channel m, M stands for the number of classes, and $q_m\left(\mathbf{x}\right)$ indicates the confidence level associated with the class predicted for $\mathbf{x}$. The cross-entropy loss is defined as follows:

$$E=-\sum _{\mathbf{x}\in Q}w\left(\mathbf{x}\right)log\left(q_{r\left(\mathbf{x}\right)}\left(\mathbf{x}\right)\right)$$

(2)

$w\left(\mathbf{x}\right)$ denotes the weight assigned to a specific position $\mathbf{x}$, $r:Q\to 1,\dots ,M$ represents the true label function that assigns labels to each position, and $q_{r\left(\mathbf{x}\right)}\left(\mathbf{x}\right)$ represents the predicted probability of the true class at position $\mathbf{x}$.

In our methodology, it is crucial to not only focus on the semantic parsing outcomes of navigation images but also to emphasize the dark regions within areas adjacent to the identified spacecraft components. We achieve this by assigning higher attention weights to these areas in our computational model, thereby improving the accuracy and detail of our analysis. This approach highlights significant but typically underrepresented features, ensuring enhanced image quality and reducing the risk of overexposure. Given the inherent darkness of the input images, the results produced by the spacecraft semantic parsing network are likely to capture only a portion of the actual semantic information. To address this limitation, we introduce a `distance factors module’ that performs secondary adjustments based on the outputs from the spacecraft semantic parsing. This module first calculates the Euclidean distance from each point to the nearest target point, then applies a nonlinear adjustment, and finally, adds this to the inverted grayscale image to generate the attention map, as depicted in Figure 2. These maps utilize cooler hues to denote lower attention weights and warmer hues for higher weights. We tailor the size of the attention map to match each feature map, subsequently multiplying it with all intermediate feature maps and the output image.

Calculation of distance factors:

$$D(x_i,y_j)=|x_i-x_b|+|y_j-y_b|$$

(3)

Nonlinear mapping adjustment:

$$D_{i,j}=u·{\left(D(x_i,y_j)\right)}^v$$

(4)

The target point coordinate within the image coordinate system is denoted as $(x_b,y_b)$, where ’target points’ refer to the points in semantic segmentation that are the nearest target points based on the Euclidean distance calculation, points that belong to the spacecraft class rather than the background. The point coordinates that need to be calculated are represented as $(x_i,y_j)$. Additionally, the coefficients u = 1 and v = 1.3 represent the mapping function, which can be manually adjusted based on the dataset.

3.2. On-Orbit Navigation Image Enhancement Network

As depicted in Figure 3, our network enhancement is inspired by the EnlightGAN architecture [38], featuring a lightweight unidirectional GAN structure, with Unet acting as the generator. We introduce a global–local–global triple discriminator that strategically emphasizes both global and local information. The preservation of structural integrity is critical in enhancing on-orbit navigation images, as previously discussed. The prior studies have employed perceptual loss in pixel space to maintain the structure and color of input images. However, in the context of extremely dark images and those with highly uneven lighting typical of navigation scenarios, these methods often fail to restore structural details adequately. To overcome this challenge and ensure our model retains the structural integrity of the input while generating a more natural visual effect, we have integrated a pre-enhancement module and devised a structure preservation loss. This specific loss is designed to maintain the structure of spacecraft in areas of extreme darkness during the training phase.

In order to enhance both the overall image and adaptively improve local regions, we utilize a global–local–global discriminator structure within the navigation image enhancement network. The global discriminator incorporates a relativistic approach, employing the least squares GAN (LSGAN) model instead of the traditional Sigmoid activation. This modification is designed to improve the discriminator’s ability to assess the relative realism between real and fake samples more accurately.

The loss functions for the global discriminator (D) and the generator (G) are as follows:

$${\mathcal{L}}_{\mathcal{D}}^{Global}={\mathbb{E}}_{z_r\sim P_{\mathrm{real}}}\left[{(D_{Ra}(z_r,z_f)-1)}^2\right]+{\mathbb{E}}_{z_f\sim P_{\mathrm{fake}}}\left[D_{Ra}{(z_f,z_r)}^2\right]$$

(5)

$${\mathcal{L}}_{\mathcal{G}}^{Global}={\mathbb{E}}_{z_f\sim P_{\mathrm{fake}}}\left[{(D_{Ra}(z_f,z_r)-1)}^2\right]+{\mathbb{E}}_{z_r\sim P_{\mathrm{real}}}\left[D_{Ra}{(z_r,z_f)}^2\right]$$

(6)

$z_r$ and $z_f$ represent samples drawn from the real and fake distributions, respectively. Real samples refer to high-quality images of spacecraft obtained in laboratory and virtual environments with clear and evenly distributed lighting, while fake samples are those generated by the generative network during training. The operations $D_{Ra}(z_f,z_r)$ and $D_{Ra}(z_r,z_f)$ indicate the standard functions of the relativistic discriminator, which compare a fake sample $z_f$ against a real sample $z_r$, and vice versa.

$$D_{Ra}(z_f,z_r)=\sigma \left(C\left(z_f\right)-\mathbb{E}\left[C\left(z_r\right)\right]\right)-\sigma \left(C\left(z_r\right)-\mathbb{E}\left[C\left(z_f\right)\right]\right)$$

(7)

$$D_{Ra}(z_r,z_f)=\sigma \left(C\left(z_r\right)-\mathbb{E}\left[C\left(z_f\right)\right]\right)-\sigma \left(C\left(z_f\right)-\mathbb{E}\left[C\left(z_r\right)\right]\right)$$

(8)

$C\left(z_f\right)$ and $C\left(z_r\right)$ are preliminary, raw outputs from the discriminator. $\mathbb{E}\left[C\left(z_r\right)\right]$ and $\mathbb{E}\left[C\left(z_f\right)\right]$ indicate that these are mean values calculated over batches or datasets of real and fake samples, respectively. $D_{Ra}\left(z_f\right)$ and $D_{Ra}\left(z_r\right)$ are specified as the final processed outputs, which are used for calculating the probability of a sample being real. The adversarial loss for the local discriminator is expressed as follows:

$${\mathcal{L}}_D^{Local}=E_{z_r\sim P_{\mathrm{real}-\mathrm{patches}}}\left[{(D\left(z_r\right)-1)}^2\right]+E_{z_f\sim P_{\mathrm{fake}-\mathrm{patches}}}\left[{\left(D\left(z_f\right)\right)}^2\right]$$

(9)

$${\mathcal{L}}_G^{Local}=E_{z_f\sim P_{\mathrm{fake}-\mathrm{patches}}}\left[{(D\left(z_f\right)-1)}^2\right]$$

(10)

To ensure consistency, many methodologies employ a pre-trained VGG model to measure the feature space distance between images. However, our experiments have revealed that, in scenarios involving non-paired training, reliance solely on self-regulation fails to restore certain fine structural details in extremely dark navigation images. Moreover, our observations indicate that although the use of a dehazing algorithm [45] results in distortion of spacecraft structure and color due to adjustments for overexposure, it simultaneously significantly enhances the visibility of structural details in the dark regions of the image. As a result, we introduce a global structural feature loss by employing the Laplacian pyramid representations of both the pre-enhanced and generated images. Specifically, we leverage a pre-trained VGG16 model on ImageNet to extract features from these Laplacian pyramids. The pre-enhancement module employs a dehazing algorithm, which treats the image as if it were hazy by utilizing pixel inversion, followed by enhancing visibility through gamma correction. Although the visual quality of the pre-enhanced images is substandard, this process significantly improves the capture of spacecraft structural details in extremely dark regions. The formula can be expressed as follows:

$$I_{\mathrm{out}}=255-{(255-I_{\mathrm{in}})}^{\gamma }$$

(11)

$I_{\mathrm{in}}$ represents the input image, and $\gamma$ represents the gamma correction factor. The Gaussian pyramid is constructed by sequentially downsampling $I_{\mathrm{out}}$, with each level $I_{\mathrm{out}}^i$ obtained from the preceding level $I_{\mathrm{out}}^{i-1}$. The Laplacian pyramid at each level is defined as follows:

$$L_i=I_{\mathrm{out}}^i-upsample\left(I_{\mathrm{out}}^{i+1}\right)$$

(12)

The upsample function denotes a bilinear interpolation method. The Laplacian pyramid is used to capture multiscale fine details from the pre-enhanced image, as illustrated in Figure 4.

As elucidated earlier, the definition of the global structural loss is as follows:

$${\mathcal{L}}_S\left(I\right)=\alpha {\mathcal{L}}_{LS}\left(I\right)+\beta {\mathcal{L}}_C\left(I\right)$$

(13)

$${\mathcal{L}}_{LS}\left(I\right)=\frac{1}{W_{i,j}\times H_{i,j}}\sum _{x=1}^{W_{i,j}}\sum _{y=1}^{H_{i,j}}{\left({\psi }_{i,j}(I_{out}^k-upsample\left(I_{out}^{k+1}\right))-{\psi }_{i,j}\left(G\left(I\right)\right)\right)}^2$$

(14)

$${\mathcal{L}}_C\left(I\right)=\frac{1}{W_{i,j}\times H_{i,j}}\sum _{x=1}^{W_{i,j}}\sum _{y=1}^{H_{i,j}}{\left({\psi }_{i,j}\left(I\right)-{\psi }_{i,j}\left(G\left(I\right)\right)\right)}^2$$

(15)

$\alpha$ and $\beta$ serve as weight factors that adjust the significance of the components of the global structural loss. $W_{i,j}\times H_{i,j}$ represents the dimensions of the feature map. I represents the input image, $G\left(I\right)$ is the output of the generation network, $I_{out}^k$ denotes the k-th level output of the Laplacian pyramid, and ${\psi }_{i,j}$ is the feature extracted using a pre-trained VGG model, where i refers to the i-th max-pooling layer, and j is the j-th convolutional layer following the i-th max-pooling layer. The specific layers used are $i=5$ and $j=1$. The overall loss function is defined as follows:

$${\mathcal{L}}_{LOSS}={\mathcal{L}}_S^{Global}+{\mathcal{L}}_S^{Local}+{\mathcal{L}}_G^{Global}+{\mathcal{L}}_G^{Local}$$

(16)

4. Experiment

4.1. Datasets

(1)

Spacecraft Semantic Segmentation Datasets:

(a)

The dataset includes a total of 3307 images, with a ratio of 9:1 between virtual images and those captured in a laboratory setting.

(b)

The dataset is randomly partitioned into a validation set (10%) and a training set (90%).

(2)

Navigation Image Enhancement Datasets:

(a)

Training Datasets:

Comprising unpaired images that are extremely dark or extremely bright.

• Train A consists of 1427 images, with laboratory-captured images and Unity-generated virtual images combined in a 3:1 ratio.
• Train B: Comprising 1020 images.

(b)

Test Datasets:

The test dataset combines 7741 laboratory-captured images and 73 real on-orbit navigation images, totaling 5 datasets, all with a resolution of 2248 × 2048.

• Test A: Laboratory-Captured Dataset
  Comprises 627 images captured under low-light conditions simulating an approach towards satellites.
• Test B: Laboratory-Captured Dataset
  Comprises 3231 images capturing a rotating satellite under oblique illumination from a faint light source.
• Test C: Laboratory-Captured Dataset
  Comprises 449 images capturing the proximity approach of satellites under changing illumination conditions.
• Test D: Laboratory-Captured Dataset
  Comprises 3434 images capturing the proximity approach of rotating satellites under varying illumination conditions.
• Test E: On-Orbit Navigation Imaging Dataset
  Comprises 73 images captured by a spacecraft in low Earth orbit, featuring various imaging environments including Earth’s illuminated regions, shadowed regions, and the transitional zone between illuminated and shadowed areas.

4.2. Implementation Details

This paper utilizes PyTorch to construct the network within a Python 3.8 environment, executed on a 12th Gen Intel(R) Core(TM) i7-12700H processor(Intel Corporation, Santa Clara, CA, USA). at 2.30 GHz and an NVIDIA GeForce RTX 3090 GPU. The spacecraft semantic parsing network was trained for 200 epochs using the Adam optimizer. Training commenced with an initial learning rate of 0.0001, which was held constant for the first 120 epochs and then reduced linearly over the remaining 80 epochs. To expedite the training pace, the network input was set to [512, 512], and all convolutional layers were configured with a kernel size of [3, 3]. The training process lasted approximately 10 h. The spacecraft semantic parsing network contains approximately 24.89 million trainable parameters and occupies 2253 MB of memory during training.

The navigation image enhancement network was similarly trained for 100 epochs, with the entire network optimized using the Adam optimizer at a learning rate of 0.0001. The learning rate remained constant for the initial 60 epochs, then linearly decayed to zero over the subsequent 40 epochs. The training process lasted approximately 20 h. To enhance the diversity of the generated images, data augmentation techniques such as random cropping and image pooling were employed. The navigation image enhancement network contains approximately 6.96 million trainable parameters and occupies approximately 3540 MB of memory during training.

For comparative evaluation, eight enhancement models—including SpaceLight (ours), LIME [25], KinD [33], EnlightGAN [38], PSENet [46], SCI [47], Night [48], and HEP [49]—were rigorously tested, and their outcomes were meticulously documented for analysis.

4.3. Evaluation Metrics

The approaches for assessing image quality are broadly classified into subjective and objective methods. Subjective methods rely on human observers to evaluate the fidelity of an image compared to real-world visuals. Objective methods, in contrast, use mathematical algorithms to measure image quality based on image characteristics and statistical information. This paper employs four distinct quantitative metrics specifically chosen for evaluating image enhancement and restoration effects.

The peak signal-to-noise ratio (PSNR) evaluates the structural resemblance between two images. A higher PSNR value indicates better image quality. $I_{\mathrm{original}}\left(i\right)$ and $I_{\mathrm{processed}}\left(i\right)$ denote the pixel values at corresponding locations in the original and enhanced images, respectively, while MAX represents the maximum possible pixel value. PSNR is calculated using the following equations:

$$MSE=\frac{1}{N}\sum _{i=1}^N{(I_{\mathrm{original}}\left(i\right)-I_{\mathrm{processed}}\left(i\right))}^2$$

(17)

$$PSNR=10{log}_{10}\left(\frac{MA{X}^2}{MSE}\right)$$

(18)

The structural similarity (SSIM) metric evaluates the similarity between an original image and its enhanced version by analyzing luminance, contrast, and structural elements [50,51]. SSIM values range from [−1, 1], with values near 1 indicating higher image fidelity. In the SSIM formula, x and y are the images being compared. The local means, standard deviations, and covariance of pixel values in the corresponding image regions are represented by ${\mu }_x, {\mu }_y, {\sigma }_x, {\sigma }_y,$ and ${\sigma }_{xy}$, respectively. Constants $c_1$ and $c_2$ stabilize the calculation in cases of weak denominators. I, C, and S denote the luminance, contrast, and structure comparison functions.

The SSIM index is defined by

$$SSIM(x,y)={\left[I(x,y)\right]}^{\alpha }{\left[C(x,y)\right]}^{\beta }{\left[S(x,y)\right]}^{\gamma }$$

(19)

$\alpha$, $\beta$, and $\gamma$ are weights used to adjust the relative importance of luminance similarity, contrast similarity, and structural similarity, respectively. In certain situations, it may be desirable to give more weight to contrast similarity or structural similarity, which can be achieved by adjusting $\alpha$, $\beta$, and $\gamma$. Simplifying with $\alpha =\beta =\gamma =1$:

$$SSIM(x,y)=\frac{(2{\mu }_x{\mu }_y+c_1)(2{\sigma }_{xy}+c_2)}{({\mu }_x^2+{\mu }_y^2+c_1)({\sigma }_x^2+{\sigma }_y^2+c_2)}$$

(20)

Multi-scale structural similarity (MS-SSIM) is an enhancement of the original SSIM metric, extending the evaluation of structural similarity across multiple scales. This method calculates weighted averages of SSIM measurements at different scales, providing a more comprehensive evaluation of image quality that captures both localized and overall image structures. The MS-SSIM values range from 0 to 1, where higher values indicate better image fidelity. Here, x and y are the two images being compared, M is the total number of scales used in the evaluation, $SSI{M}_i(x,y)$ is the SSIM at the i-th scale, and $w_i$ is the weighting factor for each scale.

The MS-SSIM for images x and y is calculated as

$$MS-SSI{M}_{(x,y)}=\prod _{i=1}^M{\left(SSI{M}_i(x,y)\right)}^{w_i}$$

(21)

$$MS-SSI{M}_{(x,y)}={\left[l(x,y)\right]}^{{\alpha }_M}·\prod _{i=1}^M{\left[c(x,y)\right]}^{{\beta }_i}·{\left[S(x,y)\right]}^{{\gamma }_i}$$

(22)

Learned perceptual image patch similarity (LPIPS) is a deep-learning-based metric that evaluates perceptual differences between images by analyzing attributes such as color, texture, and structure. The metric produces values between 0 and 1, where values closer to 0 signify higher image quality. In this metric, $I_{original}$ and $I_{processed}$ are the images under comparison. Features extracted from these images at layer i by a pre-trained network are denoted as $f_i\left(I_{original}\right)$ and $f_i\left(I_{processed}\right)$.

The LPIPS index is calculated using

$$LPIPS(I_{original},I_{processed})=\sum _{i=1}^n{w}_i·{\parallel f_i\left(I_{original}\right)-f_i\left(I_{processed}\right)\parallel }^2$$

(23)

5. Results Analysis

5.1. Quantitative Comparison

This paper presents a quantitative comparison of our algorithm, SpaceLight, against the established methods. These include LIME [25], a classical supervised method, KinD [33], and five unsupervised algorithms: EnlightGAN [38], Night [48], HEP [49], SCI [47], and PSENet [46]. The implementations for these algorithms were obtained from code repositories specified by their respective authors. The quantitative evaluations were conducted using five distinct datasets: Datasets 1 through 4 consist of laboratory-captured images, while Dataset 5 features authentic on-orbit navigation images. As demonstrated in

Table 1. Performance Metrics Comparison for TestA.

	SpaceLight	LIME	KinD	EnlightGAN	Night	HEP	SCI	PSENet
PSNR	25.8375 2	25.7300	5.3682	10.9198	13.3153	12.7971	26.6539 1	22.4496
SSIM	0.4842 1	0.2830	0.0147	0.0434	0.0377	0.0529	0.4267 2	0.1971
MS-SSIM	0.7165 1	0.2808	0.0237	0.0469	0.0545	0.05988	0.5697 2	0.2964
LPIPS	0.0393 1	0.2217	0.4183	0.1644	0.1721	0.0651 2	0.0967	0.1822

¹ the superior outcomes; ² the next-best results.

,

Table 2. Performance Metrics Comparison for TestB.

	SpaceLight	LIME	KinD	EnlightGAN	Night	HEP	SCI	PSENet
PSNR	26.4139 1	24.5643	5.3266	10.9286	12.9719	12.8326	26.2267 2	21.9694
SSIM	0.5575 1	0.2156	0.0122	0.03950	0.0171	0.0540	0.3550 2	0.1607
MS-SSIM	0.7174 1	0.2686	0.0249	0.05020	0.0579	0.0665	0.5562 2	0.2920
LPIPS	0.0435 1	0.2857	0.4471	0.2097	0.2319	0.0641 2	0.1544	0.2382

¹ the superior outcomes; ² the next-best results.

,

Table 3. Performance Metrics Comparison for TestC.

	SpaceLight	LIME	KinD	EnlightGAN	Night	HEP	SCI	PSENet
PSNR	26.8031 1	23.3959	5.6495	10.9257	12.5737	12.8928	24.3550 2	20.1984
SSIM	0.5582 1	0.2041	0.0153	0.0395	0.0292	0.0384	0.3039 2	0.1318
MS-SSIM	0.7295 1	0.2497	0.0285	0.0545	0.0579	0.0825	0.5693 2	0.2978
LPIPS	0.05831	0.2529	0.4057	0.1924	0.1972	0.0669 2	0.1445	0.2099

¹ the superior outcomes; ² the next-best results.

and

Table 4. Performance Metrics Comparison for TestD.

	SpaceLight	LIME	KinD	EnlightGAN	Night	HEP	SCI	PSENet
PSNR	28.5612 1	24.5100	4.5959	10.5278	12.0562	12.6895	26.7876 2	22.5661
SSIM	0.5074 1	0.1570	0.0063	0.0229	0.0092	0.0425	0.2937 2	0.1220
MS-SSIM	0.7066 1	0.2700	0.0200	0.0430	0.0489	0.0611	0.5562 2	0.2874
LPIPS	0.0370 1	0.2116	0.4173	0.1751	0.2189	0.0642 2	0.1168	0.1806

¹ the superior outcomes; ² the next-best results.

, the SpaceLight algorithm exhibits significant enhancement effects, showing a clear advantage across various metrics. However, a decline in these metrics is observed in

Table 5. Performance Metrics Comparison for TestE.

	SpaceLight	LIME	KinD	EnlightGAN	Night	HEP	SCI	PSENet
PSNR	25.9703 1	23.9686 2	7.9310	14.3434	11.8444	14.1392	23.9627	22.7644
SSIM	0.4978	0.4737	0.0103	0.2174	0.1724	0.2658	0.6510 1	0.5273 2
MS-SSIM	0.6961	0.8221	0.1710	0.1778	0.1211	0.2383	0.7881 2	0.8268 1
LPIPS	0.2234 2	0.3180	0.6690	0.2761	0.3107	0.1318 1	0.2568	0.2581

¹ the superior outcomes; ² the next-best results.

during tests with on-orbit navigation images. This decline predominantly occurs in images featuring Earth or cloud backgrounds. The primary objective of our algorithm is to minimize the enhancement of these backgrounds, focusing instead on the robust restoration of satellite structures. As a result, the performance metrics are lower in Experiment 5.

To address this challenge, we annotated satellite regions within the images and computed specific metrics for these areas. The results, presented in

Table 6. Performance Metrics Comparison for TestF.

	SpaceLight	LIME	KinD	EnlightGAN	Night	HEP	SCI	PSENet
PSNR	36.6030 1	34.2695 2	22.0033	29.5256	26.8166	25.5238	33.4020	32.9006
SSIM	0.9080 1	0.8629	0.5740	0.5478	0.5138	0.4605	0.8869 2	0.8781
MS-SSIM	0.9899 1	0.9867	0.9646	0.9665	0.9669	0.9640	0.9891 2	0.9887
LPIPS	1.23 × 10−6 1	1.71 × 10−6 2	1.22 × 10−5	1.84 × 10−6	3.24 × 10−6	3.47 × 10−6	2.19 × 10−6	2.31 × 10−6

¹ the superior outcomes; ² the next-best results.

, indicate that our metrics have maintained optimal values, highlighting the algorithm’s advanced capabilities. Moreover, our framework operates on the NVIDIA GeForce RTX 3090(NVIDIA Corporation, Santa Clara, CA, USA), processing 256 × 256 images in just 0.01766227 s. This processing speed significantly exceeds the operational requirements for on-orbit navigation spacecraft, demonstrating exceptional efficiency.

In tandem, the data presented in Table 1, Table 2, Table 3 and Table 4 outline various experimental scenarios under extreme low-light conditions. Table 1 features satellite navigation images captured under consistent, direct lighting, whereas Table 2 displays images under tilted and low-intensity lighting. Table 3 and Table 4 include images of rotating and approaching satellites under dynamic lighting conditions. The numerical assessments across these tables demonstrate the superior adaptability of our algorithm to varying lighting conditions and angles, demonstrating its potential for on-orbit applications.

The analysis across Table 1 through Table 6 reveals significant insights into the performance of our framework. Notably, SpaceLight consistently outperforms other methods, delivering exceptional metrics and producing clear visual results. It effectively distinguishes spacecraft from deep space backgrounds, focusing on and restoring the structure of the spacecraft with high fidelity. Figure 5 illustrates these distinctions, displaying results processed by SpaceLight alongside those from other state-of-the-art methods. The red bounding boxes highlight areas with significant discrepancies within the images, aiding in visual comparison. The top three images in the examples are captured from laboratory settings, while the last two depict real on-orbit navigation images captured in both shadowed and illuminated areas.

As shown in Figure 6, SpaceLight performs exceptionally well across all four metrics. In terms of PSNR, SpaceLight significantly improves the pixel-level differences between the enhanced and original images, indicating its superior performance in reducing noise and distortion. For the SSIM metric, SpaceLight demonstrates high levels of brightness, contrast, and structural similarity, suggesting its strong ability to maintain image structural integrity and prevent the loss of important details. The MS-SSIM results further confirm SpaceLight’s capacity to preserve image structural similarity across multiple scales, which is crucial for handling complex images and scenes. On the LPIPS metric, SpaceLight maintains high visual consistency, making the enhanced images appear more natural and realistic. To ensure consistency in the observation of all four metrics, the LPIPS values have been inverted. Compared to other algorithms, SpaceLight occupies the largest area in the radar chart, indicating its overall superior performance across all metrics. This comprehensive advantage showcases its effectiveness in enhancing pixel differences, maintaining image structure, and improving perceptual quality. In summary, SpaceLight excels in enhancing pixel differences, maintaining image structure, and improving perceptual quality. This indicates that the algorithm effectively balances various metrics during the image enhancement process, enhancing both objective image quality (such as PSNR and SSIM) and subjective visual effects (such as LPIPS). This visual comparison vividly demonstrates the exceptional performance of the SpaceLight algorithm across a range of scenarios, both in laboratory and on-orbit settings.

5.2. Qualitative Comparison

As depicted in Figure 5, the KinD, Night, and HEP algorithms significantly brighten images but fail to preserve image originality, resulting in oversaturation and texture loss. This often leads to excessive noise and color distortion, obscuring or completely losing the structure of the spacecraft. Conversely, EnlightGAN tends to overemphasize exceptionally dark backgrounds, causing unnecessary alterations and some color distortion in spacecraft structure restoration. LIME aggressively enhances images, amplifying irrelevant background elements, reducing spacecraft visibility, and introducing significant noise. PSENet and SCI maintain inherent image characteristics better, resulting in relatively satisfactory enhancement effects. However, they still struggle to effectively augment the spacecraft structure in darker areas. In contrast, our proposed method, SpaceLight, achieves optimal quality metrics by enhancing spacecraft structures clearly and accurately. SpaceLight avoids pseudoshadows caused by overexposure, minimizes interference from background elements, and consistently outperforms other methods. The favorable outcomes achieved with SpaceLight confirm its reliability as a framework for on-orbit navigation processing. Furthermore, these results demonstrate SpaceLight’s capability to effectively support subsequent vision-related tasks.

5.3. Ablation Study

In this section, we conduct a series of experiments to evaluate the stability of SpaceLight and assess the influence of its key components. Specifically, we examine the impacts of the spacecraft semantic parsing network (SSP-Network) and the structural preservation loss(SP Loss). Figure 7 illustrates the individual augmentation effects of each module when operated independently.

It is evident that employing either the SSP-Network or the SP Loss alone provides a certain level of effectiveness in enhancing structures within darker regions. However, the combined integration of the SP Loss and SSP-Network significantly increases the overall enhancement intensity. Consequently, we propose using a semantic parsing network to produce self-attention maps and applying a structure-preserving loss in the generative network. This approach aims to generate more competitively enhanced images for on-orbit navigation.

5.4. Applications

In this section, we evaluate the efficacy of the SpaceLight algorithm for semantic segmentation tasks on navigation images, particularly those under extremely dark conditions and with extreme lighting variations. We utilized navigation images enhanced by various algorithms, as illustrated in Figure 5, as inputs for a trained semantic segmentation network. This allowed us to assess the accuracy of the segmentation maps produced. The first three images used in the assessment were captured under diverse lighting conditions in a laboratory setting, while the last two images were obtained from satellites in low Earth orbit.

Figure 8 demonstrates the effectiveness of SpaceLight in restoring structural details of spacecraft in extremely dark, deep-space environments. This capability significantly enhances the accuracy of semantic segmentation in such navigation contexts, preserving the integrity of spacecraft components substantially. In the segmentation maps, red indicates solar panels, green denotes the satellite body, and yellow marks the antenna. By comparing the original images with those enhanced by SpaceLight, a marked improvement in the segmentation of spacecraft components is observed. This enhancement is primarily due to SpaceLight’s ability to minimize background interference, facilitating the on-orbit identification of spacecraft elements. Notably, the region highlighted by the blue box showcases the exceptional effectiveness of our algorithm.

5.5. Limitations

While SpaceLight’s non-supervised image enhancement methodology is effective, it has inherent limitations. The incorporation of structural preservation loss within the algorithm boosts structural recovery performance but may impact color fidelity in images. Furthermore, a key component of our framework, the semantic parsing network, is specifically designed for entities such as satellites and spacecraft. Consequently, its application to other types of objects would necessitate retraining the network.

6. Conclusions

This article has introduced a novel navigation image enhancement framework, SpaceLight, which comprises two primary components: the spacecraft semantic parsing network and the on-orbit navigation image enhancement network. The principal objective of this framework is to enhance on-orbit navigation images by specifically restoring spacecraft structures within environments characterized by extreme darkness, which is crucial for subsequent tasks such as visual navigation and on-orbit servicing. The key findings from this study are summarized as follows:

In contrast to ground-based low-light images, on-orbit navigation images often exhibit extreme darkness or high-contrast lighting conditions, making it challenging to discern structures within dark areas against the predominantly dark deep space background. Moreover, the acquisition of paired on-orbit navigation image datasets, unlike those for ground images, is impractical. To address these challenges, we developed a navigation image enhancement framework based on spacecraft semantic priors. The semantic parsing network, which generates attention maps, employs a lightweight U-Net structure. The enhancement network utilizes a combination of GANs and structural preservation loss, along with attention maps, to restore the intricate structures of spacecraft. Notably, this study includes experiments conducted for the first time with navigation images captured by on-orbit spacecraft, further validating the natural visual effects and structural preservation capabilities of our method. Comprehensive experimental results demonstrate the exceptional performance of SpaceLight in enhancing navigation images. Additionally, the experiments detailed in the application section of the paper further demonstrate the effectiveness of SpaceLight in visual tasks, underscoring its practical utility in navigation scenarios.

Looking forward, we plan to focus on extracting prior information from on-orbit navigation images to further enhance visual navigation systems. Our aim is to integrate image enhancement techniques with advanced tasks such as on-orbit target tracking, attitude estimation [2], component identification [3], and visual multi-sensor fusion [4]. This integration will provide robust visual support for future space on-orbit servicing missions. The continued research and innovation in this field are expected to significantly improve the robustness and accuracy of visual navigation systems, thereby enhancing the efficacy and safety of space on-orbit service missions.