1. Introduction
Marine resources are rich and have not been fully exploited. Compared with rivers, lakes, and other waters, the underwater environment of the ocean is more complex and hazardous, and the risk coefficient of artificial exploration and development is too high. Autonomous underwater vehicles (AUV) have become essential tools for human beings to explore the ocean [
1,
2,
3], and visual images play an important role in exploring and perceiving the surrounding environment of an AUV. Due to the absorption and scattering of the water’s body, however, underwater images have some problems, such as low contrast, image blur and color deviation, which affect the follow-up vision task of the underwater vehicle. Therefore, the acquisition of high-quality underwater images is of great significance for human exploration and the development of our understanding of the ocean. Existing underwater image enhancement algorithms are mainly divided into the physical model-based algorithm [
4,
5], the non-physical model-based algorithm [
6,
7], and the neural network-based algorithm [
8,
9].
The physical model-based algorithm is mainly represented by dark channel prior algorithm. In 2009, He proposed the dark channel prior (DCP) algorithm [
10]. The DCP algorithm builds the mathematical model of the degradation process of the foggy image, and inverts the degradation process of the image by estimating the unknown parameters and combining then with the known parameters, so as to enhance the image quality. Because the attenuation process of the underwater image is similar to that of an outdoor image, a large number of underwater image enhancement algorithms based on DCP have been proposed. The non-physical model-based algorithm is mainly represented by Retinex algorithm. In 1963, Edwin H. Land proposed the Retinex theory [
6]. The Retinex algorithm can balance the three aspects of edge enhancement, dynamic range compression and color constancy in the process of image processing, but the problem of exposure will occur when the illumination intensity is high. In order to solve the problems existing in the Retinex algorithm, many scholars have improved it based on the Retinex algorithm. The Multi-Scale Retinex algorithm (MSR) [
11] and Multi-Scale Retinex with Color Restoration (MSRCR) [
12] have been successively proposed. The neural network-based algorithm is mainly represented by generative adversarial network. In 2014, Goodfellow proposed the concept of generative adversarial network (GAN) [
13]. GAN uses the network to directly learn the mapping relationship between degraded underwater images and clear images, trains the model, and then restores images. After GAN theory was proposed, many scholars improved the GAN and applied it to different fields, and the deep learning theory began to be applied in the field of underwater image enhancement. Among them, the GAN based on the neural network algorithm [
14,
15,
16,
17,
18] is the most widely used in underwater image enhancement.
Many scholars apply the GAN model to the field of underwater image enhancement, among which the representative algorithms are UWGAN [
19], WaterGAN [
20], UGAN [
21], FUnIE-GAN [
22], etc. In 2017, Li [
20] proposed an underwater image generative adversarial network (WaterGAN). WaterGAN first uses atmospheric images and depth maps to synthesize underwater images. It then takes the synthesized underwater images as datasets and constructs an end-to-end color correction network to realize the real-time color correction of underwater images. In 2018, Fabbri [
21] proposed an underwater GAN (UGAN), UGAN, which first uses unpaired clear underwater images and degraded underwater images to train CycIeGAN, and then inputs clear underwater images into CycleGAN to generate corresponding degraded underwater images. Then, it uses pairs of underwater images as datasets for subsequent network training. Finally, L1 loss and gradient loss are added to the original loss of Wasserstein GAN to restore a degraded underwater image. In 2019, Guo [
19] proposed a new underwater GAN (UWGAN), UWGAN, which adds a residual multi-scale dense block (RMSDB) to the generator; this is used to correct image color and restore image details. In 2020, fast underwater image enhancement GAN (FUnIE-GAN) [
22] was proposed, which has good robustness and efficiency and can be applied to underwater vehicles. Generally speaking, in the underwater image enhancement algorithm, the GAN learns the mapping relationship from underwater degraded images to true underwater images. GAN needs a lot of training data for underwater image enhancement, including underwater degraded images and their corresponding truth images. However, existing truth images are limited by their unique imaging environment, and it is difficult to obtain an underwater truth image in the learning sample. At present, a truth sample depends on a variety of traditional algorithms. In selecting the best image from the captured images, or synthesizing an image by estimating the random parameters of an underwater image as the true image, there is a certain gap between a true image and a real underwater image. As a result, the image quality generated by the GAN is not ideal.
In the FUnIE-GAN algorithm, the contrast effect of the generated image is not very good. We analyzed that the low contrast may be caused by the following reasons:
- (1)
There are some low contrast underwater images in the truth samples of the training set, so the training effect of the model is not good.
- (2)
The function that can improve the image contrast is not added in the FUnIE-GAN algorithm, so the contrast of the generated image is not high.
Considering the above problems, a fast underwater image enhancement algorithm based on the NIQE index (FUnIE-GAN-NIQE) is proposed in this paper. The main contributions of this paper are as follows:
- (1)
To solve the problem of low contrast images in truth datasets, this paper filters the images into truth images based on EUVP datasets to screen out truth images that meet the requirements.
- (2)
To solve the problem of the low contrast of the generated image, this paper takes the NIQE as a part of the loss function of the generator in FUnIE-GAN and becomes its enhancement index.
- (3)
To make the discriminant factors more diversified, this paper adds NIQE as FUnIE-GAN to the structure of the discriminator as part of the discriminator, which makes the resulting image more uniform in the color histogram distribution and more consistent with the perception of the human eye; this makes the generated image exceed the effect of the truth image set in the existing dataset.
- (4)
In FUnIE-GAN-NIQE, there are four loss items in the loss function of the generator, including the adversarial loss function of the standard conditional GAN, L1 loss, content loss, and image quality loss. The weight loss of each part will affect the training result of the generator in the whole network; thus, this paper proposes to train 10 generators and 10 discriminators, traverse the weights of three parts (L1 loss, content loss, and image quality loss), and select the best generator among the 10 generators to generate the image. This method not only enhances the underwater image, but it can be applied to the enhancement of non-underwater images.
Section 1 provides a brief introduction to the development of underwater image enhancement and the idea of this paper.
Section 2 briefly introduces relevant background knowledge.
Section 3 introduces the main work of this paper.
Section 4 describes experiments on the algorithm proposed in this paper and objectively analyzes its performance.
Section 5 is a summary and conclusion of the findings of this paper.
5. Conclusions
This paper aimed to solve the difficulty of obtaining underwater truth images when a supervised generative adversarial network, which works in underwater image enhancement, leads to low contrast for generated images. First of all, this paper filtered the dataset of EUVP truth images. Secondly, this paper proposed to add the NIQE index to the loss of the generator to provide the generated image with higher contrast and make it more in line with the perception of the human eye. At the same time, we attempted to make a generated image exceed the effects of the truth image set by the existing dataset. Then, this paper proposed to add the NIQE index to the discriminator to diversify its discriminant factors in FUnIE-GAN. Finally, this paper proposed a new structure of GAN, and selected the generated image suitable for this paper by training 10 generators. In this paper, several groups of experiments were compared. Through subjective evaluation and objective evaluation indicators, it was verified that the enhanced image contrast of this algorithm was better than the truth image contrast set by the existing dataset theory. At the end of this paper, the real-time performance of the algorithm was analyzed to verify that the algorithm could be used in engineering.
In this paper, it was verified that the FUnIE-GAN-NIQE algorithm proposed in this paper can improve the contrast of the enhanced image through subjective evaluation and objective index evaluation. This paper does not prove the superiority of this algorithm through specific engineering examples, such as underwater target detection, underwater image segmentation, etc. In future work, we will continue to study the task in this direction, and verify the superiority of the underwater image enhancement algorithm through more abundant engineering examples.
The contrast of the image generated by the FUnIE-GAN-NIQE algorithm proposed in this paper is better than that of the truth image, but the clarity of the image is far lower than that of the truth image. This is because the underwater image enhancement algorithm has high engineering requirements, and the running speed can only be improved by reducing the amount of computation, which is a common problem in the existing underwater image enhancement algorithms. Therefore, it is hoped that the future work can achieve a balance between clarity and speed, and ensure the running speed of the algorithm in the case of greatly reducing the clarity.