Underwater Optical Image Restoration Method for Natural/Artificial Light

Abstract

This paper investigates the underwater optical image restoration method under the background of underwater target detection based on optical vision in AUVs. The light source used for AUV detection is different when the AUV operates in different depths. The natural light source is used in shallow water and the artificial light source is used in deep water. This paper investigates underwater optical image restoration in these two light conditions. Aiming at the problem of image blurring in underwater optical images, the traditional underwater image restoration method based on scattering model can obtain satisfactory image restoration performance in natural light conditions. However, it cannot obtain the same image restoration result in artificial light conditions. To solve this problem, this paper presents an improved underwater optical image restoration method based on the scattering model. The scattering model and power spectrum are used to solve the initial parameters of the filter, and the parameters are optimized based on an evaluation index. The index of image definition is introduced to evaluate the restoration performance and to achieve the satisfactory image restoration result in both natural light and artificial light conditions. The effectiveness of the presented method is verified by experiments.

1. Introduction

Autonomous underwater vehicles (AUV) have been widely used in underwater target detection and other tasks because of their good maneuverability and wide operation range [1,2]. Optical vision is the main means for AUVs to obtain information about targets being close to themselves [3,4] and has been widely used in AUV-based pipeline monitoring, underwater salvage operations [5,6,7], etc. In this paper, AUV target detection based on optical image is taken as the background to investigate the underwater optical image restoration method. At the same time, it is necessary to consider the use of natural light sources in shallow water and artificial light in deep water. The modeling, detection, and compensation of artificial light sources is a relatively new topic compared with underwater image processing [8,9].

Compared with land images [10,11], underwater optical images have two main problems [12,13]. One problem is the image blurring caused by light scattering in water. The other is the image color deviation caused by the different wavelengths of light attenuation in water. Therefore, it is necessary to process underwater optical images to obtain a clearer and more realistic image which lays a good technical foundation for the subsequent target recognition.

There are two main solutions for underwater optical image processing: image enhancement and image restoration [14]. Image enhancement mainly starts from the image pixel itself. It can improve the definition and highlight the details of the target through pixel transformation, and the performance is good in general. However, when the image degradation is serious, this method has poor performance [15,16]. The physical principle of underwater image quality degradation is considered in image restoration. The image quality degradation process is inversely deduced based on the underwater imaging model, and the degraded image is restored and reconstructed along the inverse process of degradation. When the image degradation is serious, it is better than the image enhancement methods [17,18]. Since the underwater optical image considered in this paper is seriously degraded, the underwater optical image restoration method is investigated in this paper.

There are two models used in underwater optical image restoration: scattering model and underwater imaging model. The scattering model focuses on image blurring and neglects color deviation [19], while the underwater imaging model pays attention on both image blurring and color deviation. However, its high computational complexity results in failure to meet the real-time requirement of underwater detection, and it is difficult to widely apply online [20]. In this paper, underwater target recognition is the research background. Relatively speaking, image blurring has a much greater impact on target recognition than image color deviation. Therefore, this paper investigates the image restoration method based on the scattering model.

Image restoration based on scattering model usually uses experimental method or mathematical model method to obtain the degradation function and then estimates and solves the parameters in the restoration process to make degraded images clearer. Ref. [21] measured the optical transfer function of seawater and deduced relevant parameters with an experimental method to achieve good restoration results. However, the difference of water quality, flow, and other factors in different waters led to different optical transfer functions. Ref. [22] measured the point spread function of seawater with an experimental method in the laboratory environment and restored the underwater image with a Wiener filter. How to apply the point spread function of seawater measured in laboratory environments to the actual marine environment needs to be further investigated. A self-calibrated recovery filter method was proposed in [23]. The basic idea is to assume that the forward scattering part of the light is the main attenuation component when light travels in water and to ignore the light’s backscattering. However, in the actual underwater environment, the light in water is greatly affected by the backscattering, so the restoration result of this method is not satisfactory. A method was proposed in [24] to calculate the medium scattering light with a polarization filter to remove the medium scattering light in the original degraded image. However, the realization of this method requires multiple images of the same scene as the known conditions. Methods based on a mathematical model deduces the image degradation model and degradation function through scattering theory and ocean optics theory, which is simple and practical. Although it cannot accurately describe all the influencing factors of image degradation, it has a good effect on solving the problem of blurred image restoration caused by light scattering. At the same time, the calculation time of this method is lower, which is conducive to meeting the real-time requirement of underwater target detection for AUVs.

Based on the abovementioned analysis, under the background of underwater target detection, it is necessary to restore the blurred underwater image under two lighting conditions comprising natural lighting and artificial lighting. Therefore, in the underwater image restoration based on the scattering model, a mathematical model method is used to establish the regression model. Then the scattering attenuation parameters are calculated from the scattering energy power spectrum, and image restoration is achieved through Wiener filtering [25]. In this paper, it is found that the method works well in a natural lighting environment, i.e., the theoretical value of the parameter corresponds to the best image restoration result. However, in the artificial lighting environment, the theoretical value of the parameter does not correspond to the best image restoration result. Therefore, the main contributions of this paper are as follows.

• This paper provides a choice for underwater image restoration from the technical view for natural lighting and artificial lighting conditions. To obtain satisfactory restoration results, this paper developed an improved underwater image restoration method based on the scattering model by adding a loop of parameter adjustment.
• In this method, the scattering model and power spectrum are used to solve the initial parameter of the filter, and then the parameter is adjusted in a certain range based on the index of image definition to achieve the best image restoration effect in both natural lighting and artificial lighting conditions.
• The effectiveness of the developed underwater image restoration method is verified by pool-experiment results.

The organization of this paper is as follows: Section 2 briefly describes the underwater image restoration method based on the scattering model. In Section 3, the problems and the corresponding reason analysis are presented. Section 4 describes the idea and implementation process of the improved method. Section 5 presents experimental verification. Finally, conclusions are drawn in Section 6.

2. Introduction of Underwater Image Restoration Method Based on the Scattering Model

In this paper, image restoration is investigated for underwater target recognition. Image blurring has a much greater impact on target recognition than image color deviation in underwater optical images. Since the scattering model focuses on image blurring, it takes less time in general. The image restoration method based on the scattering model mainly involves the scattering model and filtering restoration. Here, we introduce the scattering model and filtering restoration.

(1)

The scattering model

In this model, the relationship between the image received by the camera and the original image can be expressed as in ref. [26]:

$$g\left(x,y\right)=f\left(x,y\right)\ast h\left(x,y\right)+n_i\left(x,y\right)$$

(1)

where ∗ represents convolution operation. Due to the complexity of convolution operation, the expression of Equation (1) in frequency domain is

$$G\left(u,v\right)=F\left(u,v\right)H\left(u,v\right)+n\left(u,v\right)$$

(2)

where $g\left(x,y\right)$ represents the image received by the camera; $f\left(x,y\right)$ represents the original image; $h\left(x,y\right)$ represents the water degradation function; $n_i\left(x,y\right)$ represents the noise caused by scattering in each layer of water body; $G\left(u,v\right)$, $H\left(u,v\right)$ and $n\left(u,v\right)$ represent their frequency domain representation, respectively.

The key of image restoration is to find out the relationship between the water degradation function $h\left(x,y\right)$ and the noise caused by light scattering in water $n_i\left(x,y\right)$. The frequency domain expression of the total scattering power spectrum of water obtained from the scattering layered transmission model is as in ref. [26]:

$${\left|G\left(u,v\right)\right|}^2=\frac{{\left|K\left(u,v\right)\right|}^2{I}_0^2}{{\left[2c+\rho {\pi }^2\left(u^2+v^2\right)\right]}^2}$$

(3)

where $K\left(u,v\right)$ represents the frequency domain of the random variable related to the light scattering; I₀ represents the light intensity at the initial position of the light source; c represents the attenuation coefficient of the light intensity when propagating in water; and $\rho$ represents the parameters corresponding to the Gaussian transfer function of the unit water body thickness.

(2)

Filter restoration

Wiener filtering is a typical filtering method [25]. The restoration expression of Wiener filtering in the frequency domain is

$$\widehat{F}(u,v)=[\frac{1}{H(u,v)}\frac{{\left|H(u,v)\right|}^2}{{\left|H(u,v)\right|}^2+P_n(u,v)/P_f(u,v)}]G(u,v)$$

(4)

where $P_n(u,v)$ represents the power spectrum of the noise, and $P_f(u,v)$ represents the power spectrum of the undegraded image.

When the power spectrum of the undegraded image is unknown, the frequency domain expression of the Wiener filter transfer function is as in ref. [21]:

$$P\left(u,v\right)\approx \frac{H{\left(u,v\right)}^{\ast }}{{\left|H\left(u,v\right)\right|}^2+k}$$

(5)

where the values of $H{\left(u,v\right)}^{\ast }={\left|H(u,v)\right|}^2/H(u,v)$ and k are obtained from the actual scattered noise power spectrum and the power spectrum of the undegraded image, that is, k = $P_n(u,v)/P_f(u,v)$, and the value of k has great impact on the image restoration.

3. Problem Discovery and Cause Analysis

The original images in this paper are obtained by a camera in an UVIC AUV operating in 2 m depth in a pool with the size of 8 m × 4 m × 4 m. The camera is a UWC-575 produced by the Outland company. An LED underwater lamp (GLF-SX-2) is installed on both sides of the vehicle, and its power is 100 W. In addition, an image acquisition card (MPEG4-HD-N) is used. Then, two lighting conditions are involved in this paper. The one is the natural lighting conditions, i.e., the images are obtained in day with good lighting. The other is the lighting conditions, where the shading cloth is used to cover windows, and images are obtained at night to minimize the impact of natural light; at the same time the LED underwater lamps are opened to simulate the deep-sea environment.

3.1. Problem Discovery

In this subsection, image restoration experiments are carried out on underwater degraded images under natural lighting and artificial lighting conditions, and problems are found by analyzing the experimental results

(1)

Underwater image restoration under natural lighting conditions

Under natural lighting conditions, the underwater color image and image preprocessing results are shown in Figure 1, where Figure 1a is the original color image, which is obtained by a camera in the pool environment; Figure 1b is the grayscale image after grayscale processing, and Figure 1c is the background image of the target-free area. It can be seen from Figure 1b that the details in the target are almost integrated with the background noise due to the external noise, and it is difficult to identify the target. It can be seen from Figure 1c that the image degradation is caused by background noise, and the goal of image restoration is to remove the influence of background noise.

In order to objectively evaluate the quality of image restoration, this paper introduces the evaluation index of “image definition” [27], shown as follows:

$$f\left(I\right)=\frac{1}{m\times n}{\displaystyle \sum _x{\displaystyle \sum _y\left({\left[I\left(x,y\right)-I\left(x+1,y\right)\right]}^2+{\left[I\left(x,y\right)-I\left(x,y+1\right)\right]}^2\right)}}$$

(6)

where $f\left(I\right)$ represents the definition of the image; m and n represent the image size; an d $I\left(x,y\right)$ represents the pixels at each point in the image.

A theoretical value of the parameter k under the natural lighting conditions can be obtained by the power spectrum, i.e., k = 0.00125. Then, on the both sides of the theoretical value, 5 values of k are selected by a step with 0.00002. The k values corresponding to the maximum definition on the both sides of 0.00125, and the theoretical value (k = 0.00125) are selected into reason analysis. The restored images under the three values of the parameter k are shown in Figure 2, and the corresponding definition is shown in

Table 1. Definition results under different k values under natural light conditions.

Definition of the Original Image	Value of k	Definition after Image Restoration
0.39516	k = 0.00123	0.80549
	k = 0.00125	0.87452
	k = 0.00127	0.84439

.

It can be seen from Table 1 that the image definition after restoration is significantly increased compared with that without restoration, which indicates the effectiveness of underwater image restoration based on the scattering model method under natural lighting conditions. At the same time, the maximum definition is obtained when k = 0.00125, indicating that the k value calculated by this method is also correct. In this paper, experiments with multi-group images were carried out, and we also concluded that the theoretical value of k has the maximum definition.

(2)

Underwater image restoration under artificial lighting conditions

The underwater color image and preprocessing results under artificial lighting conditions are shown in Figure 3, where Figure 3a is the original image obtained by a camera in the pool.

A theoretical value of the parameter k under the artificial lighting conditions can be obtained by the power spectrum, i.e., k = 0.00134. Then, on the both sides of the theoretical value, five values of k are selected by a step with 0.00002. The k values corresponding to the maximum definition on the both sides of 0.00134, and the theoretical value 0.00134 are selected into reason analysis. The restored images under the three values of the parameter k are shown in Figure 4, and the corresponding definition is shown in

Table 2. Definition results under different k values under artificial lighting conditions.

Definition of the Original Image	Value of k	Definition after Image Restoration
0.43213	k = 0.00132	0.86323
	k = 0.00134	0.82125
	k = 0.00136	0.85936

.

It can be seen from Table 2 that the image definition after restoration is also significantly increased under artificial lighting conditions. However, the theoretical value of the parameter k, i.e., k = 0.00134, does not correspond to the optimal definition. Experiments of multi-group images under artificial lighting conditions are also conducted, and the result is the same as the above case.

From the abovementioned results, the theoretical value parameter k calculated by the power spectrum can obtain the best image restoration result under natural lighting conditions. Under the artificial lighting conditions, the theoretical value of the parameter k does not have the best image restoration, but the optimal value of the parameter k is near the theoretical value.

3.2. Cause Analysis

The natural light direction is vertically downward and the lighting direction is horizontally forward. The direction of the two light sources is different. The theoretical value of the parameter k for image restoration is calculated based on the background noise. The main factor affecting the background noise is the backscattering of light. Specifically, the scattering of light by suspended particles in water in the non-target area produces background noise. The scattering direction of light by suspended particles in water is random. Although the direction of natural light and lighting light is different, the rules of background noise under the two lighting conditions are similar after the scattering of suspended particles in the random direction of light. Thus, the underwater image restoration method based on the scattering model can be used for the artificial lighting conditions in theory, but the theoretical value of k is optimal for natural lighting conditions. The theoretical value of k calculated from the power spectrum is not the optimal one for artificial lighting conditions, and the optimal parameter should be near the theoretical value.

4. Improved Method for Underwater Image Restoration

Aiming at the problems of underwater image restoration method based on the scattering model, this paper developed an improved method based on the reason analysis in the previous section. This section describes the basic idea and implementation process of this improved method.

(1)

Basic idea of this improved method

Based on the above problem and cause analysis, the fundamental reason is that the backscattering of natural light and lighting is different, and there are many reasons and parameters involved in this difference. The problems of modeling, detection, and compensation of artificial light sources are a relatively new topic in underwater image processing. At present, there are few references about underwater artificial light sources, and it is difficult to accurately grasp the backscattering law of artificial light sources in a short time. From the above analysis, the underwater image restoration method based on the scattering model can be used for artificial lighting conditions in theory. The theoretical value of the parameter k obtained by this method is optimal for natural lighting conditions, and the best restoration result does not result from the theoretical value but from a value near the theoretical value.

Therefore, the idea of this improved method is shown as follows. Regardless of natural lighting and artificial lighting, the theoretical value of the parameter k is calculated through the power spectrum from the original image, grayscale image, and background noise image, and then the parameter k is adjusted near the theoretical value. Different values of k are respectively used to restore the image, and the image restoration effects are evaluated through the index definition. The image with the highest definition is the final restoration result.

(2)

Implementation process of this method

The basic process of the method in this paper is to obtain the theoretical/initial value of the parameter k through the power spectrum, denoted as k₀. According to the previous experimental results, it is determined that the adjusting range of k is [−0.01, 001], i.e., the value range of k is k₀ ± 0.001. The definition of the restored image is calculated when the value of k is changed. The final restored image is the image with the highest definition, and the flow chart of the improved image restoration method is shown as Figure 5.

According to Figure 5, the specific steps of the improved method are as follows:

(1)

Image acquisition

The underwater degradation images are collected through a camera in the pool. The scattering model is established and the degradation function is obtained.

(2)

Calculate the initial value k₀

The initial parameter k₀ of the natural light environment of the Wiener filter is obtained by solving the power spectrum.

(3)

Determine the adjustment step and value range

The step size is 0.00001 and the value range is k₀ ± 0.001 in the adjustment of the parameter k after examining some underwater images.

(4)

Perform image restoration and calculate image definition

Image restoration is carried out according to different values of the parameter k, and the definition of the corresponding restored images is calculated.

(5)

Determine the final restored image

From the definitions calculated from Step 4, the image with the highest definition is determined as the final restored image.

5. Experimental Results

In order to verify the performance of the improved image restoration method, the underwater degraded images under natural light and artificial light are collected through the pool experiment to verify the image restoration.

The relevant conditions of the experimental platform in the experiments are shown as follows. The host is a PC104 with Intel Pentium M, 2 GHz and 512 MB memory. The operating system is VxWorks5.0, while the programming environment is Tornado 2.2. The underwater image size is 352 × 288.

5.1. Underwater Image Restoration Results under Natural Light Conditions

Under natural light conditions, the underwater color image and image preprocessing results are shown in Figure 6. The restoration results of three different values of the parameter k are shown in Figure 7, and the corresponding definitions are shown in

Table 3. Statistics of restoration results under natural light conditions.

Definition of Original Image	Value of Parameter k	Definition of the Restored Image
0.48475	k = 0.00094	0.87152
	k = 0.00096	0.88645
	k = 0.00098	0.81964

.

The theoretical value of the parameter k obtained by the power spectrum is k = 0.00096 under natural light conditions. It can be seen from Figure 7 that the definition of the restored image is significantly improved compared with the original degraded image, and the restoration result when k = 0.00096 is best from the intuitive feeling. It can be also seen from Table 3 that the definition of the restored image is largest when k = 0.00096, up to 0.88645. The result verifies the correctness of the previous analysis.

5.2. Underwater Image Restoration Effect under Artificial Lighting Conditions

In the pool experiment, the shading cloth is used to block the natural light, and the underwater lamp is turned on. In this case, the color images of the underwater sphere target are collected.

The underwater color image and image preprocessing results are shown in Figure 8.

The theoretical value of the parameter k under the artificial lighting conditions can be obtained by the power spectrum, i.e., k = 0.00113. Then, according to the flowchart in Figure 5, different k values are used to restore images by the step size of 0.00001 near the theoretical value. The k values corresponding to the maximum definition on the both sides of 0.00113 are recorded. The restored images under the recorded values and the theoretical value of the parameter k are shown in Figure 9, and the corresponding definition is shown in

Table 4. Statistics of restoration results under artificial lighting conditions.

Definition of the Original Image	Value of Parameter k	Definition of the Restored Image
0.54357	k = 0.00111	0.87265
	k = 0.00113	0.83467
	k = 0.00115	0.88936

.

The theoretical value of the parameter k obtained by the power spectrum is k = 0.00113 under artificial lighting conditions. From Figure 9, the definition of the restored image is significantly improved, and it is intuitively felt that the restoration result is best when k = 0.00115. It can be seen from Table 4 that the improved method obtains the maximum definition 0.88936 when k = 0.00115, and the theoretical value of the parameter k does not correspond to the optimal definition of the restored image. Therefore, it is verified that adjusting the parameter k near the theoretical value can obtain better image restoration result compared with the traditional method.

From the time consumption of the algorithm, the time consumption of the improved method is within 10 ms, which can fully meet the real-time requirements of AUV underwater detection (<500 ms).

The above experimental results verify the effectiveness of the improved method. This method can obtain better image restoration results under natural/artificial lighting conditions. The improved method provides a solution to the problem in the traditional method where it can not get the best restoration result under artificial lighting conditions.

6. Conclusions

An improved underwater image restoration method is developed based on the scattering model in this paper. This paper provides a solution for underwater image restoration under natural/artificial lighting conditions, which satisfies the real-time requirement of AUV target detection based on optical images. The experimental results show that the developed method can achieve the best underwater image restoration results in both natural lighting and artificial lighting conditions, but the traditional method cannot obtain the best restoration results under the artificial lighting conditions. The experimental results verify the effectiveness of the developed image restoration method, which has laid a good technical foundation for subsequent target recognition. In future research, the binary search is combined into the improved image restoration method to further shorten the image processing time. In addition, the object edge extraction method will be investigated based on the image restoration results.