Sequential Two-Mode Fusion Underwater Single-Photon Lidar Imaging Algorithm

Abstract

Aiming at the demand for long-range and high-resolution imaging detection of small targets such as submerged submarine markers in shallow coastal waters, research on single-photon lidar imaging technology is carried out. This paper reports the sequential two-mode fusion imaging algorithm, which has a strong information extraction capability and can reconstruct scene target depth and reflection intensity images from complex signal photon counts. The algorithm consists of four steps: data preprocessing, extremely large group value estimation, noise sieving, and total variation smoothing constraints to image the target with high quality. Simulation and test results show that the imaging performance and imaging characteristics of the method are better than the current high-performance first-photon group imaging algorithm, indicating that the method has a great advantage in sparse photon counting imaging, and the method proposed in this paper constructs a clear depth and reflectance intensity image of the target scene, even in the 50,828 Lux ambient strong light and strong interference, the 0.1 Lux low-light environment, or the underwater high-attenuation environment.

1. Introduction

In recent years, high-quality imaging technology in low-light and high-light-intensity attenuation environments has received great attention. However, traditional optical detection, such as traditional laser radar, visible light detection, and fluorescence imaging, is affected by the absorption and scattering of water molecules, dissolved substances, and suspended particles, which not only reduces the intensity of light, but also expands the pulse laser in the time domain and increases the spot radius in space, which affects the detection accuracy [1,2,3,4]; thus, the practical application of detection is greatly restricted. At present, single-photon lidar detection technology has aroused extensive research interest [5,6,7,8,9,10,11,12,13], and has become an effective way to solve the problem of detection in low-light and high-light-intensity attenuation environments.

As a key factor affecting the performance of single-photon lidar systems, the iteration of imaging algorithms goes hand in hand with the progress of the system. At present, in the field of single-photon lidar imaging, many researchers and scholars have made quite good results [14,15,16,17,18,19,20,21]. Based on the imaging principle, data processing method, and technical realization, the existing single-photon lidar imaging methods can be divided into two major categories; one is based on the neural network [22,23], which extracts the data information by designing various neural networks and then reconstructs the target scene. However, the lack of large-scale pairwise single-photon lidar image training datasets, the long training time, the easy loss of high-frequency information at the edge, the high cost of computational resources, and the high cost of hardware deployment in the field deployment [24,25,26,27] make the single-photon imaging algorithms based on neural networks suffer great limitations, and their actual deployment faces great difficulties [28,29]. In contrast, modeling-based imaging algorithms have demonstrated their efficiency and environmental adaptability in many practical applications [30,31,32,33,34,35,36,37]. However, although 3D point cloud imaging based on the modeling approach can enhance the perception of the shape, size, and spatial location of objects, it has higher technical complexity, increased cost, and higher processing power requirements [38], so 2D imaging technology is more suitable for a wide range of practical application scenarios due to its lower technical requirements and cost. In the existing single-photon 2D imaging algorithms, the fixed-pixel acquisition time reconstruction method does not fully take into account the variability of the pixel echo signals, which may lead to redundancy or insufficient acquisition time for overly bright or dark scenes [39,40,41,42], and thus the imaging method is not optimal. Another 2D imaging algorithm, based on the photon echo intensity, determines the pixel depth information, among which the classic first-photon method [43] and first-photon group method [40] use the “first” relevant photon information to reconstruct the target scene; however, according to the description of the paper [43,44], it was found that the first-photon algorithm is not suitable for complex environments, especially for environments with low signal-to-noise ratios, while the first-photon group algorithm has better anti-noise performance, but the “group” information needed for imaging needs to be manually tuned according to the environment, which makes it difficult to adapt to the changing environment.

An efficient single-photon lidar imaging algorithm that can effectively extract the target information is urgently needed for the high-resolution detection requirements in variable and extreme environments, especially underwater environments. In this thesis, we report a sequential two-mode fusion (STMF) imaging algorithm, which utilizes the fundamental property that the scene information photons in the echo signal are differently distributed from the noise photons in the time histogram to extract effective information. Compared with the current high-performance first-photon group algorithm, this method can quickly adapt to the current scene and perform high-quality depth imaging and reflection intensity imaging without any parameter tuning. Finally, we verify the convenience and effectiveness of the method through land and underwater experiments.

2. STMF Imaging Algorithm Modeling Approach

The signal photon counts have a small variance and show highly concentrated Gaussian distribution characteristics, while the noise photon counts have a large variance and the signal tends to cluster while the noise photons are scattered throughout the whole period. Therefore, based on this characteristic, we propose the STMF imaging algorithm, which is schematically shown in Figure 1.

In Figure 1, the strategy of the STMF algorithm to reconstruct the target scene is divided into four steps, which are (1) data preprocessing, (2) maximal group value estimation, (3) pixel value filtering and replacement, and (4) total variation smoothing constraints.

The data preprocessing step includes data reconstruction and data cleaning steps, which are used to construct the returned array of the system into an array of image pixels, remove noise initially, and locate the approximate range of the target. The principle is shown in Figure 2.

To find a “specific target”, the target time range window can be set, and the window can be used to scan near the superimposed peak point, which can not only filter out most of the noise outside the window but also locate the target range, making the imaging more targeted. This method is one of the advantages that other algorithms do not have.

In the subsequent information extraction process, firstly, the group interval in which the signal echo photons are located is determined according to the aggregation characteristics of the echo signal to initially filter the clutter interference and reduce the dependence on the subsequent photons. Secondly, the sum of the number of group photons and the corresponding time index are taken in the photon group interval, and the depth and reflectivity maps are then obtained after the screening and replacement steps and the total variational smoothing constraint steps. The algorithm interpretation is shown in Algorithm 1.

In Algorithm 1, num_of_bin is the number of bins of each pixel histogram, which can be given instead by $\chi$. num_of_pixesl is the number of pixels in each image datum. time_of_per_bin is the time duration of each bin. location_of_data is the storage location after the echo data are acquired. $n_{x,y}^{indices,num}$ indicates the num bin of the indices group of the current pixel. $T_{indices,max}^{\chi }$ indicates the bin corresponding to the maximum photon number of a bin in an indices group among all bin data of the current pixel. $Diff(T_{indices,max}^{\chi })$ and $Diff(N_{sum,grouprange}^{\chi })$ are the differences between the meantime, the mean number of photons in the scanning window of the current pixel time value, and the photon value. $\psi$ is the parameter used to adjust the pixel excess. ${\chi }_{x,y}^{new}$ and $n_{x,y}^{\chi ,new}$ are the reflection intensity and time matrices adjusted by the parameter $\psi$.

Algorithm 1. STMF algorithm interpretation

2.1. Maximum Group Estimation

Assuming that $\chi$ is the number of bin cells of each pixel histogram in the test, the pixel time and photon number obtained by the test are as in Equation (1), the corresponding time of the maximum group data is given by Equation (2), and the number of echo photons is given by Equation (3).

$$\left\{t_{x,y}^1,t_{x,y}^2,t_{x,y}^3,\dots \dots ,t_{x,y}^{\chi }\right\}\begin{array}{cc} & \left\{n_{x,y}^1,n_{x,y}^2,n_{x,y}^3,\dots \dots ,n_{x,y}^{\chi }\right\}\end{array}$$

(1)

$$\left\{t_{x,y}^{\gamma ,1},t_{x,y}^{\gamma ,2},t_{x,y}^{\gamma ,3},\dots \dots ,t_{x,y}^{\gamma ,\zeta }\right\}$$

(2)

$$\left\{n_{x,y}^{\gamma ,1},n_{x,y}^{\gamma ,2},n_{x,y}^{\gamma ,3},\dots \dots ,n_{x,y}^{\gamma ,\zeta }\right\}$$

(3)

In the above formula, $\gamma$ is the ordinal number of the $(x,y)$ position pixel group, $\zeta$ is the ordinal number of the bin within the group, $t_{x,y}^{\chi }$ is the moment of the $\chi$-th bin of $(x,y)$ position pixel, $t_{x,y}^{\gamma ,\zeta }$ is the corresponding moment of the bin within the group, and $n_{x,y}^{\gamma ,\chi }$ is the number of photons arriving at the corresponding moment of the bin within the group. $x,y$ in Equations (1)–(3) denotes the coordinates of the pixel at $(x,y)$. After obtaining the photon data from Equation (3), the index in Equation (2) corresponding to the maximum value in Equation (3) can be obtained, and the photon number and time index are shown in Equation (4) and Equation (5), respectively.

$$\underset{indices}{N_{x,y}^{\gamma }}={\displaystyle \sum _1^{\zeta =\zeta }{n}_{x,y}^{\gamma ,\zeta }}$$

(4)

$$\underset{indices}{T_{x,y}^{\gamma }}=\mathrm{indices}[\arg \max (\left\{n_{x,y}^{\gamma ,1},n_{x,y}^{\gamma ,2},n_{x,y}^{\gamma ,3},\dots \dots ,n_{x,y}^{\gamma ,\zeta }\right\})]$$

(5)

In the above equation, $n_{x,y}^{\gamma ,\zeta }$ is the number of photons arriving at the corresponding moment in the bin cell of the group, $\underset{indices}{N_{x,y}^{\gamma }}$ is the total number of photons within the pixel group at $(x,y)$, $\underset{indices}{T_{x,y}^{\gamma }}$ is the moment index corresponding to the bin maximum photon value within the position pixel group at the $(x,y)$ location, and $\gamma$ is the pixel group serial number at $(x,y)$ location. To reduce the problem of pixel value loss when the number of echo photons is low, we sum the photons in the group when determining the current number of pixel photons.

The initial reflection intensity map and depth map of the target scene can be obtained based on the sum of the maximum number of photons per pixel $\underset{indices}{N_{x,y}^{\gamma }}$ and the corresponding moment index $\underset{indices}{T_{x,y}^{\gamma }}$. The diagram of pixel value selection is shown in Figure 3.

2.2. Pixel Value Filtering and Replacement

Because the depth and reflection intensity of the current pixel and the adjacent pixel are approximately equal, this relationship can be used to replace the anomalous pixel [44]. Assuming that the root-mean-square width of the system response function (the distribution function of the number of single pulse-echo photons on the time axis) is $\eta$, the median replacement strategy is shown in Figure 4.

(1) To realize the scanning replacement of the edge pixels of the image, one layer of the 0-value pixel is added to the edge of the original image, i.e., assuming that the original image is 64 × 64 pixels, after adding the edge pixels, it becomes an image of 66 × 66 pixels in size; the schematic of the 0-value expansion and scanning is shown in Figure 5.

(2) Utilizing the correlation of spatial pixels, the current pixel moment value $T_{x,y}$ is compared with the average value of spatial pixels $T_{x,y}^{average}$, and whether to change the current pixel value is determined by Equation (6).

$$\underset{indices}{T_{x,y}^{\gamma ,new}}=\left\{\begin{array}{cc}\underset{indices}{T_{x,y}^{\gamma }}& \left|\underset{indices}{T_{x,y}^{\gamma }}-T_{x,y}^{average}\right|\le 2\cdot \eta \\ T_{x,y}^{average}& \left|\underset{indices}{T_{x,y}^{\gamma }}-T_{x,y}^{average}\right|>2\cdot \eta \end{array}\right.$$

(6)

In Equation (6), $\underset{indices}{T_{x,y}^{\gamma ,new}}$ is the photon arrival time after the replacement strategy, $T_{x,y}^{average}$ is the time flight value averaged in the spatial pixel, $\underset{indices}{T_{x,y}}$ is the current pixel time-of-flight value, and $\eta$ is the root-mean-square width of the system response function. We assume that the time flight values $T_{x,y}^{average}$ represented by all pixels in the entire spatial pixel are averaged instead of excluding the current pixel value and then averaging it. This is because it is not possible to determine which of the current pixel and the domain pixel are noisy. When the absolute value of the difference between the current pixel time-of-flight value $\underset{indices}{T_{x,y}^{\gamma ,new}}$ and the mean value $T_{x,y}^{average}$ of the 3 × 3 spatial pixels is not greater than 2$\eta$, then the current pixel time $\underset{indices}{T_{x,y}^{\gamma ,new}}$ remains unchanged. When the absolute value of the difference between the current pixel time-of-flight value $\underset{indices}{T_{x,y}}$ and the mean value $T_{x,y}^{average}$ of the 3 × 3 spatial pixels is greater than 2$\eta$, then the current pixel value $\underset{indices}{T_{x,y}^{\gamma ,new}}$ needs to be replaced with the mean value $T_{x,y}^{average}$.

After replacement, the difference between the abnormal value (time value/distance value) and the surrounding pixels can be observed. If the abnormal value is too much, the spatial pixel group range needs to be reset or the secondary replacement can be considered.

Compared with the first-photon group imaging algorithm, a post-feedback step is introduced for pixel value screening and replacement in the STMF imaging algorithm. When the evaluation index after replacement is below the threshold, resetting the spatial pixel group range or secondary replacement can be considered.

2.3. Total Variation Smoothing Constraints

After obtaining the replaced pixels data, the image is smoothed using the optimal image total variation smoothing constraint method [44,45,46]. The smoothing rule is defined by Equation (7), and its flow diagram is shown in Figure 6.

$$\widetilde{\underset{indices}{T_{x,y}^{\gamma ,new}}}=\underset{\widetilde{\underset{indices}{T_{x,y}^{\gamma ,new}}}}{\arg \min }\left[(\widetilde{\underset{indices}{T_{x,y}^{\gamma ,new}}}-\underset{indices}{T_{x,y}^{\gamma ,new}})+\psi \cdot f(\widetilde{\underset{indices}{T_{x,y}^{\gamma ,new}}})\right]$$

(7)

In Equation (7), $\widetilde{\underset{indices}{T_{x,y}^{\gamma ,new}}}$ is the time matrix of $x\ast y$ pixel size after smoothing constraints, and its value range is $0\le \widetilde{\underset{indices}{T_{x,y}^{\gamma ,new}}}\le T_{x,y}^{new}$. $\underset{indices}{T_{x,y}^{\gamma ,new}}$ is the time matrix after spatial pixel replacement, and $\psi$ is the parameter to adjust the bias of $\widetilde{\underset{indices}{T_{x,y}^{\gamma ,new}}}$. If $\psi$ is small, then $\widetilde{\underset{indices}{T_{x,y}^{\gamma ,new}}}$ and $\underset{indices}{T_{x,y}^{\gamma ,new}}$ will be closer and if $\psi$ is large, then $\widetilde{\underset{indices}{T_{x,y}^{\gamma ,new}}}$ will be smoother. In the experiment, to ensure that the flight time is as accurate as possible and the image is smoothed, we take $\psi =0.1$. In Equation (7), $f(\widetilde{\underset{indices}{T_{x,y}^{\gamma ,new}}})$ can be defined by Equation (8).

$$f(\widetilde{\underset{indices}{T_{x,y}^{\gamma ,new}}})={\displaystyle \sum _{x=1}^{X-1}{\displaystyle \sum _{y=1}^{Y-1}\sqrt{{(\widetilde{\underset{indices}{T_{x,y}^{\gamma ,new}}}-\widetilde{\underset{indices}{T_{x+1,y}^{\gamma ,new}}})}^2+{(\widetilde{\underset{indices}{T_{x,y}^{\gamma ,new}}}-\widetilde{\underset{indices}{T_{x,y+1}^{\gamma ,new}}})}^2}}}+{\displaystyle \sum _{x=1}^{X-1}\left|\widetilde{\underset{indices}{T_{x,Y}^{\gamma ,new}}}-\widetilde{\underset{indices}{T_{x+1,Y}^{\gamma ,new}}}\right|}+{\displaystyle \sum _{y=1}^{Y-1}\left|\widetilde{\underset{indices}{T_{X,y}^{\gamma ,new}}}-\widetilde{\underset{indices}{T_{X,y+1}^{\gamma ,new}}}\right|}$$

(8)

In Equation (8), $f(\widetilde{\underset{indices}{T_{x,y}^{\gamma ,new}}})$ is the defining function of the difference size between the current pixel and the neighboring pixels, $\widetilde{\underset{indices}{T_{x,y}^{\gamma ,new}}}$ is the time matrix after smoothing constraints of $x\ast y$ pixel sizes, $\widetilde{\underset{indices}{T_{x+1,y}^{\gamma ,new}}}$ is the neighboring pixel value of the current pixel horizontally, and $\widetilde{\underset{indices}{T_{x,y+1}^{\gamma ,new}}}$ is the neighboring pixel value of the current pixel vertically.

After the maximum group estimation, pixel value filtering and replacement, and the total variation smoothing constraint step, the final depth image can be obtained.

3. Terrestrial Imaging Test Analysis

To verify the superiority of the STMF imaging algorithm in performance compared to the current high-performance first-photon group imaging algorithm, multiple sets of land experiments were performed. Secondly, since underwater experiments are more complicated and costly than land experiments, the effects of external light intensity, the number of laser pulses, and the distance from the target scene on the imaging results were analyzed in land experiments to provide experimental references for further underwater experiments.

3.1. Comparative Analysis of Imaging Effect

The first-photon group imaging algorithm shows great superiority relative to previous imaging algorithms. To verify that the STMF imaging algorithm reported in this paper is more convenient and has a better imaging effect relative to the first-photon group algorithm, we first launched single-photon imaging tests on land. We built a single-photon lidar; its working principle is in Figure 7.

The MCC-532-5 laser (wavelength 532 nm, repetition frequency 5 kHz, single pulse energy 30 μJ) is used to emit pulsed laser light, while the photodetector (SPAD) outputs an electrical pulse synchronization signal after the pulsed laser light is detected, which is captured by the time-dependent photon counting module (TCSPC), and starts timing. The pulsed laser is injected through the shaping optical path to increase the beam diameter and reduce the divergence angle, and it scans the target after changing the direction by the two-axis galvanometer. When the photons return to the system in the form of diffuse reflection from the target point of the target, they are coupled into the echo receiving module after changing the propagation direction by the two-axis galvanometer again. The SPAD detector generates an electric pulse signal immediately after receiving the echo photons, and the time-correlation photon counting (TCSPC) module subtracts the time of the electrical signal and the time of the synchronization signal when the pulse laser is emitted, obtains the total flight time of the outgoing photons, and finally calculates the numerical distance of the target in the computer. The input and output signals relied on by each module in the system are managed by the signal control module, and the data flow of the time-correlation photon counting (TCSPC) module is read directly by the host computer and then processed by the software algorithm for imaging. In this system, FPGA is responsible for the key tasks of data synchronization and system control, to ensure the accurate synchronization of the laser emission and detector through timing control, and to control the working mode of the galvanometer and other modules. The function of the heat exchanger is mainly to carry out thermal management and thermal control of the laser, galvanometer, TCSPC, and other modules to maintain the stable operation of the system.

In the experiment, the imaging scene is about 40 m, the number of scanned pixels is 64 × 64, and the imaging targets are three sets of targets, which are the regular target plate (Figure 8), complex geometry target (Figure 9), and multi-depth parameters targets (Figure 10). We use the first-photon group algorithm and STMF imaging algorithm to image the three sets of scenes, respectively, and the imaging effects are comprehensively analyzed using Mean Squared Error (MSE), Root Mean Squared Error (RMSE), and the Structural Similarity Index (SSIM) to comprehensively analyze the imaging effect. MSE is a measure of the difference between the predicted value of the model and the actual observed value, calculated by summing the square of the difference between the predicted value and the actual value and averaging it. RMSE is the square root of MSE and provides a standardized measure of prediction error that is more intuitive. The SSIM is a measure of the similarity of two images, which takes into account brightness, contrast, and structural information. The selection of MSE, RMSE, and SSIM parameters can improve the evaluation of single-photon lidar imaging results because they provide evaluation information from different perspectives. MSE and RMSE are directly concerned with the difference between the predicted value and the actual value, and are suitable for quantifying the prediction accuracy of the model. The SSIM focuses on the visual quality of the image and can reflect the human eye’s perception of the change in the image structure, which is suitable for evaluating the visual similarity of the imaging results. These three parameters can be used to evaluate the accuracy and image quality of single-photon lidar.

The thing to note here is that the first-photon group algorithm currently shows great advantages in single-photon imaging algorithms (Maximum likelihood estimation, Peak method, Cross-correlation method, Shin algorithm, Unmixing algorithm, first-photon imaging algorithm, etc.), and the simple logic also makes it easier to deploy [46,47,48,49]. So, our proposed algorithm will be directly compared with the first-photon group algorithm.

In Figure 8, the part of the imaging target facing the single-photon lidar system is a regular 10 cm × 10 cm square, the rest of the part is a base; the base is not the main imaging scene, it mainly plays a role in supporting and adjusting the height. In Figure 9, the part of the imaging target facing the single-photon lidar system has a circle, rectangles of different sizes, and so on. Similarly, the remaining part is a base, which is not the main imaging scene and mainly supports and adjusts the height of the imaging target. In Figure 10, the part of the imaging target facing the single-photon lidar system has six square squares with different depths (the middle square is the 0-depth datum to view the entire target, and the rest of the numbers are the depth from the datum), with a length and width of 10 cm, and a minimum depth resolution of up to 5 cm. In addition, as in Figure 10, three squares in the right column have three kinds of reflectivity, and the experiment will also test if the STMF algorithm can distinguish between targets with the same scene depth and different reflectivity.

It is worth mentioning that, to ensure the randomness and fairness of the test, the data of the three targets are randomly selected from the results of multiple data acquisition, which are tested and evaluated using the first-photon group and the STMF imaging algorithm, respectively. In addition, the key to whether effective information can be collected from the echo signal is the second step of the algorithm, and the subsequent step is the noise reduction step after the information extraction, so the imaging effect is compared between the first-photon group and the STMF imaging algorithm through the information extraction step.

Figure 11, Figure 12 and Figure 13 are the echo depth maps and reflectivity maps formed at low pulse numbers, so the number of photons reflected (noise photons + signal photons) is relatively small, and the 5000-pulse maps imaged with high fractional detailed display as the reference maps are shown in Figure 14.

Analyze the first-photon group and STMF, and evaluate the imaging algorithms in terms of MSE, RMSE, and SSIM as shown in

Table 1. Comparison of MSE, RMSE, and SSIM evaluation values of depth maps of first-photon group and STMF imaging algorithms.

Index		MSE	RMSE	SSIM
Methods
First-photon group	I	1944.474829	44.096200	0.719074
	II	2597.687134	50.967511	0.677240
	III	4475.418699	66.898570	0.672388
STMF	I	1877.190681	43.326559	0.799389
	II	3072.724840	55.432164	0.760533
	III	1508.434584	38.838571	0.786565

and

Table 2. Comparison of MSE, RMSE, and SSIM evaluation values of reflected intensity maps for first-photon group and STMF imaging algorithms.

Index		MSE	RMSE	SSIM
Methods
First-photon group	I	4291.454902	65.509197	0.550790
	II	3478.640814	58.980004	0.677963
	III	4152.372382	64.438904	0.627105
STMF	I	3064.493265	55.357865	0.577002
	II	2620.730375	51.193070	0.625744
	III	4482.540446	66.951777	0.567506

(I represents the regular target plate, II represents the complex geometry target, III represents multi-depth parameters target).

By comprehensive comparison of the data in Table 1 and Table 2, the performance of the two algorithms is different in different scenarios with low pulse numbers, and the integrated MSE, RMSE, and SSIM evaluation values, although the first-photon composition imaging algorithm shows a slight advantage in some scenarios. For example, in the second type of target imaging experiment, the RMSE index of the STMF algorithm is reduced by 8.7% compared with the first-photon group algorithm, which reflects less degree error reduction. However, the SSIM is increased by 12.3%, which reflects better similarity with the original scene. In other targets, STMF shows better restoration of the original scene compared with the first-photon group algorithm. The algorithm displays the image when the display range of the impact will greatly affect the evaluation value, and through locating the range, the first step of the STMF algorithm is preprocessed. This is another advantage that the first-photon set algorithm does not have. The STMF imaging results can be seen in Figure 15. Therefore, on the whole, the STMF imaging algorithm performs better.

From the multi-depth parametric target analysis, it can be seen that the depth resolution of the system equipped with STMF can be up to 2.5 cm (middle column) in the case of very low echo photons. To further demonstrate the practical applicability of the STMF algorithm relative to the first-photon group algorithm, the following is a continuation of the comparison of the two algorithms from the perspective of real-world applications.

The μ value is an important parameter of the first-photon group single-photon imaging technique. According to the discussion in reference [45] and the parameter simulation and test of the first-photon group algorithm above, a small μ value indicates that the definition of the first signal photon group is very tight, and a large number of very concentrated photon counts must occur to obtain the first signal photon group. The result is high robustness to noise, because the noise just accumulates, but the probability of satisfying the condition is smaller, and the efficiency of the whole method will be reduced, that is, more accumulation time will be needed to obtain the first signal photon group aggregation condition. On the contrary, if the μ value is large, the conditions of the first signal photon group are easily satisfied, and the imaging efficiency of the whole system will increase, but the probability that the noise just meets the aggregation conditions will increase, and the robustness to the noise will decrease. Therefore, the determination of the μ value is very important. However, because of the above characteristics, specific parameters need to be adjusted in different scenarios and targets with different reflectance, and the practical application is more complicated. This is exactly what the STMF imaging algorithm avoids.

From Figure 16, Figure 17, Figure 18 and Figure 19, it can be seen that the choice of the threshold value has a direct impact on the imaging results, and the results of parameters μ = 1, μ = 2, μ = 3, μ = 4, μ = 5 need to be viewed in combination with the MSE, RMSE, and SSIM values rather than a single value because the context also affects the metric values. In practical application scenarios, since the μ values cannot be tuned autonomously, the actual imaging results will be greatly reduced, and even there will be no target, as shown in Figure 16 and Figure 17. Therefore, the determination of the μ value based on prior experience is obviously not conducive to the actual situation in the changeable environment, so the application of the first-photon group algorithm in the actual environment has certain limitations. The STMF imaging algorithm reported in this paper employs a maximum group selection strategy, which maintains the ability of the first-photon group algorithm to select signal clusters and avoids the shortcomings of parameter settings.

3.2. STMF Imaging Algorithm Noise Reduction Results

The advantages of the STMF imaging algorithm can be seen through the above analysis, and the next test was conducted to further observe the imaging results using the STMF imaging algorithm. At 0.1 Lux light intensity, 64 × 64 scanning pixel number, 5 pulses at 5 kHz repetition frequency, 30 μJ single-pulse power, and 532 nm laser wavelength at 40 m, the three sets of target imaging results and the imaging reference map (maximum group estimation + pixel value filtering and substitution + total variance smoothing constraints) are shown in Figure 20, Figure 21, and Figure 22, respectively.

In

Table 3. Evaluation index values of three types of targets before and after the STMF noise reduction step.

Index		MSE	RMSE	SSIM
Methods
STMF	I (before noise reduction)	1236.794840	35.168094	0.789318
	I (after noise reduction)	937.125737	30.612509	0.912257
	II (before noise reduction)	878.321448	29.636488	0.749681
	II (after noise reduction)	601.322533	24.521879	0.897178
	III (before noise reduction)	1448.678334	38.061507	0.743624
	III (after noise reduction)	402.982322	20.074420	0.910434

, Ⅰ represents the regular target plate, Ⅱ represents the complex geometry target, Ⅲ represents the multi-depth parameters target. Through observing Figure 20, Figure 21 and Figure 22, the three types of targets, MSE decreased (24.3%, 13.0%), (31.5%, 17.3%), and (72.2%, 47.3%), respectively, while RMSE increased by 15.6%, 19.7%, and 22.4%, respectively. It was found that the STMF imaging algorithm effectively extracts the target information after the maximum group estimation in the case of very few echo photons (the number of echo photons per pixel of the scene is less than 5) (Figure 22 left column), but the echo signal also contains more pixel-missing values and noise signals (Figure 22 left column). After the pixel value screening replacement + total variance smoothing constraint, and benchmarking against Table 3, it was found that the pixel anomalies obviously disappear after the noise reduction step of the algorithm, which shows that the subsequent two steps of the STMF imaging algorithm have very good noise reduction and compensation characteristics.

In addition, from Figure 22, we can observe that the algorithm can clearly distinguish the three targets with the same depth and different reflectivity of 10 cm side length on the rightmost side of the target and 2.5 cm depth difference (middle column) with very low echo photon count, which shows the excellent performance of the algorithm.

3.3. Analysis of Imaging Influences

In the process of applying single-photon lidar imaging in the actual environment, the external lighting conditions, the scene distance, and the number of pulses may have an impact on the imaging effect. The increase in the number of pulses can significantly improve the temporal resolution and enhance the signal-to-noise ratio through signal accumulation, thus improving the clarity and accuracy of the imaging. The distance of the scene is directly related to the intensity and depth resolution of the signal. The farther the distance, the greater the signal attenuation, which puts forward higher requirements for the design of the imaging system. The level of background noise is determined by the light intensity, and too much light will increase the noise and reduce the image quality. Taking these key factors into consideration, the imaging performance of single-photon lidar can be effectively improved through fine adjustment and algorithm optimization to ensure that high detection accuracy can be maintained under different environmental conditions. Thus, the influencing factors are analyzed, which can be used as a reference for the practical application.

3.3.1. Light Intensity Impact Analysis

The MSE, RMSE, and SSIM values of the STMF imaging algorithm to analyze the direct imaging effect of the multi-depth parametric target scene are shown in

Table 4. Depth imaging MSE, RMSE, and SSIM values with external light intensity (some example points are shown).

Index	MSE	RMSE	SSIM
Intensity/Lux
0.1	1805.239107	42.488105	0.862387
16146	2094.045478	45.760742	0.865294
50828	3542.234712	59.516676	0.747120

(some example points are shown). To visualize the imaging effect, the analysis data are shown in Figure 23.

It can be seen that with the enhancement of the external light intensity, the MSE and RMSE are enhanced and the SSIM is reduced, which shows that the light intensity reduces the imaging effect. The external light intensity mainly manifests itself as noise photons that enhance the received photons. As far as this is concerned, the single-photon lidar is more applicable in the dark ocean when performing practical detection.

3.3.2. Line Art Figures

The MSE, RMSE, and SSIM values of the STMF imaging algorithm to analyze the direct imaging effect of the multi-depth parameters target scene are shown in

Table 5. Depth imaging MSE, RMSE, and SSIM values a with scene distance (some example points are shown).

Index	MSE	RMSE	SSIM
Distance/m
25	1895.817939	43.540991	0.869116
40	2094.045478	45.760742	0.865294
55	2873.154976	53.601819	0.837796

(some example points are shown). To visualize the imaging effect, the analyzed data are shown in Figure 24.

In Figure 24, the anomalous value of the 10 m scene is mainly due to the strong noise inside the system, and after ignoring these data, it can be seen that with the increase in the scene distance, the MSE and RMSE are enhanced, and the SSIM is reduced, which explains that the increase in the scene distance reduces the imaging effect. Concerning this, the optimum detection distance exists for single-photon lidar when performing actual detection, and future tests can be performed to determine the optimum detection distance for this system to guide system optimization.

3.3.3. Pulse Number Impact Analysis

The MSE, RMSE, and SSIM values of the STMF imaging algorithm to analyze the direct imaging effect of the multi-depth parameter target scene are shown in

Table 6. Depth imaging MSE, RMSE, and SSIM values with number of pulses (some example points are shown).

Index	MSE	RMSE	SSIM
Pulse Number
5	1888.402562	43.455754	0.874237
20	1350.848840	36.753896	0.903139
50	1059.433603	32.548942	0.926321

(some example points are shown). To visualize the imaging effect, the analyzed data are shown in Figure 25.

In Figure 25, it can be seen that with the enhancement of the number of pulses, the MSE and RMSE decreased and the SSIM was enhanced, which indicates that the increase in the number of pulses enhances the imaging effect. For that matter, there is a cumulative time advantage of the single-photon lidar when performing actual detection, and the number of detection pulses can be appropriately increased in subsequent tests to guide the system optimization. To further remove the noise and enhance the imaging effect, we employed contrast enhancement. High-frequency information enhancement was completed with a PnP-ADMM algorithm iteration. We could have employed wavelet transform and ESRGAN enhancement, but the index evaluation is better than this method. In future research, we will try non-local spatial pixel replacement, superpixel, and other methods.

Overall, the single-photon lidar we built and equipped with the STMF imaging algorithm reported in this paper demonstrated good imaging performance as well as practicality, and its key imaging factors were analyzed in detail, which will be taken into account appropriately to optimize the system in future experiments.

4. Analysis of Underwater Imaging Tests

In the underwater environment, due to the strong absorption and scattering of light, traditional imaging technology faces many challenges, and it is difficult to obtain high-contrast and high-resolution images; so, the practical application of single-photon lidar underwater will be of great significance. Subject to the test conditions, the test was carried out in a pool constructed with a length of 12 m, a water depth of 1 m, and a water attenuation coefficient of 0.457 m⁻¹. The test environment is photographed in Figure 26.

The lateral resolution, distance resolution, and color resolution were tested with a multi-depth parameters target. To further test the resolution ability of the algorithm, the top square in the middle column of the target was decomposed into four small square blocks with side lengths of 2.5 cm and depth difference of 2.5 cm, as shown in Figure 26c. The middle square is the 0-depth datum to view the entire target, and the rest of the numbers are the depth from the datum. The test results are shown in Figure 27, Figure 28, Figure 29 and Figure 30. It should be noted that the pool is not wrapped with light-absorbing material to simulate the actual environment.

The comparison before and after noise reduction using the first-photon group method in the same environment is shown in Figure 31, Figure 32, Figure 33 and Figure 34.

The test scene was at a distance of about 12 m, and the imaging under 5000 pulses obtained by the independent night test was also used as a reference (Figure 22 right column) to evaluate the detail retention effect by MSE, RMSE, and the SSIM; the index of underwater test depth map is shown in

Table 7. MSE, RMSE, and SSIM evaluation metrics of underwater depth map (STMF).

Index		MSE	RMSE	SSIM
Pulse Number
50	before noise reduction	4967.051578	70.477313	0.656152
	after noise reduction	5162.811476	71.852707	0.689871
500	before noise reduction	3974.910685	63.046893	0.685066
	after noise reduction	3978.879936	63.078363	0.716883

and

Table 8. MSE, RMSE, and SSIM evaluation metrics of underwater depth map (first-photon group).

Index		MSE	RMSE	SSIM
Pulse Number
50	before noise reduction	9343.743232	96.663040	0.611259
	after noise reduction	10,459.830375	102.273312	0.615554
500	before noise reduction	10,699.728375	103.439491	0.631418
	after noise reduction	10,820.452469	104.021404	0.617580

.

According to the results, the imaging map retains more detailed information, and the increase in the number of pulses enhances the effectiveness. For the STMF algorithm, for both 50 pulses or 500 pulses, SME and RMSE have a relatively obvious reduction, and the SSIM has a relatively obvious improvement. However, in the imaging of the first-photon group algorithm, only the rough outline of the target can be obtained; even if the display range is adjusted, the details cannot be obtained.

In addition, in Figure 27 and Figure 29, the imaging results in the right column show that the STMF algorithm can distinguish targets with different resolutions of 10 cm in the above environment, and the lateral resolution is up to 2.5 cm. From the first square in the middle column, it can be seen that the distance resolution is also up to 2.5 cm, which shows the algorithm’s excellent underwater detection performance. In contrast, although the first-photon group algorithm has also achieved good imaging effects on land, it is a great challenge to image complex targets under a loaded underwater environment. Both depth maps and reflection intensity maps can only see the approximate shape of the target, but the resolution is far from the effect of the STMF algorithm.

5. Conclusions

In this paper, we report a sequential two-mode fusion imaging algorithm, which achieves excellent performance of 2.5 cm horizontal and 2.5 cm vertical resolution on an autonomous single-photon lidar with a water attenuation coefficient of 0.457 m⁻¹ at a scene depth of 12 m through three steps. After the description of the method and all the experimental analysis, it was found that the STMF algorithm not only solves the problem of manual frequent parameter adjustment for the first-photon set algorithm with an excellent imaging effect at present, but also has the ability to effectively extract information from small echo photon numbers and in challenging underwater environments. The excellent performance is a firm step forward for underwater high-resolution optical imaging. This algorithm and the matching single-photon lidar achieve fine imaging of complex targets underwater, and with the upgrading of test conditions, the underwater detection range is expected to be further improved.

In future research, we will further optimize the algorithm performance and reduce the algorithm imaging time in the expectation of real-time underwater high-resolution imaging. In addition, we will also focus on studying the transmission characteristics of a pulsed laser underwater, including its attenuation law in different water environments, broadening effect, and how to reduce the impact of scattering and absorption through technical means, to optimize the design of an underwater lidar system.