A Novel Method for CSAR Multi-Focus Image Fusion

Abstract

Circular synthetic aperture radar (CSAR) has attracted a lot of interest, recently, for its excellent performance in civilian and military applications. However, in CSAR imaging, the result is to be defocused when the height of an object deviates from a reference height. Existing approaches for this problem rely on digital elevation models (DEMs) for error compensation. It is difficult and costly to collect DEM using specific equipment, while the inversion of DEM based on echo is computationally intensive, and the accuracy of results is unsatisfactory. Inspired by multi-focus image fusion in optical images, a spatial-domain fusion method is proposed based on the sum of modified Laplacian (SML) and guided filter. After obtaining CSAR images in a stack of different reference heights, an all-in-focus image can be computed by the proposed method. First, the SMLs of all source images are calculated. Second, take the rule of selecting the maximum value of SML pixel by pixel to acquire initial decision maps. Secondly, a guided filter is utilized to correct the initial decision maps. Finally, fuse the source images and decision maps to obtain the result. A comparative experiment has been processed to verify the exceptional performance of the proposed method. The final processing result of real-measured CSAR data demonstrated that the proposed method is effective and practical.

1. Introduction

Circular synthetic aperture radar (CSAR) is one of the special modes of synthetic aperture radar (SAR) that can provide a long-time illumination of a region of interest (ROI) with a wide accumulation angle [1]. Unlike linear SAR, this geometry provides wide-angle information about the reflectivity of the scattering centers in ROI and a method to obtain high-resolution images. Furthermore, this geometry makes it possible for 3D imaging by single-baseline and single-channel CSAR [2]. For these excellent advantages, CSAR has attracted a lot of attention and has been widely applied in civilian and military applications.

However, CSAR is sensitive to terrain undulation due to the curve synthetic aperture. Traditional CSAR imaging algorithms rely on the flat-Earth assumption, assuming that objects are in the same height plane, such as the back projection (BP) algorithm [3,4] and the polar format algorithm (PFA) [1,5]. Practically, the terrain of ROI is rugged, especially in urban areas. The imaging results will be defocused when an object’s height deviates from a reference height, which seriously affects the quality of the CSAR images.

A similar phenomenon is observed in applications of digital cameras. Limited by the depth of field (DOF), when a lens focuses on an object at a certain distance, only objects within the DOF are focused, while objects out of the DOF are defocused [6]. It indicates that obtaining an image with all objects focused by digital cameras directly is impossible. Therefore, multi-focus image fusion is proposed to solve this problem, in which one can acquire a series of images focusing on objects of different distances and fuse them to produce an all-in-focus image. Presently, many multi-focus image fusion methods have been applied to handle this challenge, which can be roughly classified into three main groups [7]: spatial domain methods [8,9,10,11,12,13,14], transform domain methods [15,16,17,18,19,20,21,22,23], and deep learning methods [24,25,26,27,28].

Spatial domain methods estimate decision maps pixel-wise [8,9], block-wise [10,11], and region-wise [12,13], according to focus measures. These methods are simple, efficient, and can retain more spatial information from source images. Pixel-wise methods usually produce inaccurate decision maps, especially near focused/defocused boundaries (FDBs). The effectiveness of block-wise methods relies on the size of the blocks. However, they are susceptible to blocking effects [14]. Region-wise methods discriminate focused and defocused regions by image segmentation methods, which can alleviate the block effect.

Transform domain methods transform source images to feature domain, then select feature coefficients based on fusion strategy. The fused image is reconstructed by the selected coefficients in the feature domain. The discrete wavelet transform (DWT) [15,16] is one of the classical methods, which has good time–frequency localization properties, while ineffectively distinguishing FDB. Nonsubsampled contourlet transform (NSCT) [17,18] can make up for the defects of DWT, but it has a finite number of directions. Easkey et al. [19] proposed the nonsubsampled shearlet transform (NSST), which is capable of extracting features changing in arbitrary directions. Sparse representation (SR) was also used for multi-focus image fusion [20]. In addition, multi-scale transform methods modified based on the methods above have been applied to extract multi-scale features of source images [21,22].

It is inevitable to introduce artifacts in spatial domain methods and transform domain methods. Recently, since deep learning methods are excellent in feature selection capability, they have become the mainstream methods for multi-focus image fusion. A series of convolutional neural networks (CNNs) have been proposed to meet the requirements of image fusion with satisfactory results [24,25,26,27,28]. The main shortcoming is a large number of samples are needed to train the CNNs, while CSAR data collection is difficult and costly. Therefore, these methods are not applicable in CSAR image fusion.

Based on the analysis above and considering the differences between CSAR images and optical images, which are detailed in Section 2, a spatial-domain CSAR multi-focus image fusion method is proposed in this article. First, multi-layer images that are full-aperture images in different referent heights are obtained by a high-resolution imaging algorithm. Second, the sum of modified Laplacian (SML) is utilized as a focus measure to estimate initial decision maps. Thirdly, the initial decision maps are corrected by a guided filter [29] since it is articulated discontinuously near FDB. Finally, source images and decision maps are fused to obtain an all-in-focus image. The proposed method does not rely on a priori information, such as DEM, so it is very adaptable in practical applications. We validate the advantage of the proposed method through a comparative experiment. The correctness and effectiveness of the proposed method are verified by the processing of real-measured, single-baseline, and single-channel CSAR data.

The rest of this article is organized as follows. Section 2 details the complete framework for the proposed method. The experiments and conclusions are presented in Section 3 and Section 4, respectively.

2. Method and Application

2.1. Characteristics of CSAR Images

Figure 1 shows the geometric model of the optical lens. Suppose there are three point-like objects whose distances from the film plane of the lens are, from far to near, points B, A, and C. When the lens is focused on A, conclude from Figure 1 that B is inside the DOF, and C is outside the DOF. In the imaging result, A is focused to be a point, while B and C are presented as circles that are called diffuse circles. However, the radius of the diffuse circle of B is less than one systematic resolution cell, while the opposite is true for C. Then, B is focused, while C is defocused.

Similar to Figure 1, energy mapping modeling in CSAR imaging is established, as shown in Figure 2. Unlike Figure 1, the solid lines in Figure 2 represent the mapping of the objects’ scattered echo energy to image energy. Points D, E, and F are point-like objects with different heights of 0, $\Delta h_1$, and $\Delta h_2$, where $\Delta h_1>0$, $\Delta h_2<0$, and ${‖\Delta h_1‖}_1>{‖\Delta h_2‖}_1$. Assuming that the height of the imaging plane is 0, which leads to, in the imaging result, D focused as a point and E and F defocused as rings that are different from the diffused circle in the optical images because the defocused rings have clear edges like focused regions. If the size of the rings of E and F is smaller than the smallest resolving unit of the system, E and F are focused, while defocused in contrast. It can also be noticed from Figure 2 that E and F focus as points when the reference heights are $\Delta h_1$ and $\Delta h_2$, respectively. Therefore, if E and F are defocused when the reference height is 0, they can be made to be focused by adjusting the image plane. Imaging results of the same CSAR real-measured data in reference heights of 0 m and 4.5 m are shown in Figure 3. To have a clear comparison, regions labeled by red rectangles in Figure 3a are zoomed in, as shown in Figure 4. Objects in Region 1, 2, and the upper part of 3 are focused at a reference height of 4.5 m, while Region 4 and the lower part of 3 are focused at a reference height of 0 m. It indicates that multi-layer imaging makes objects focused on the image whose reference height is close to its height.

As stated above, the characteristics of the defocused regions are different on the optical and CSAR images, which can be summarized as the defocused regions on the optical images are blurred so their spatial information is poor, such as their edge information. In contrast, the energy of the defocused regions in the CSAR images is concentrated, which behaves similarly to the focused regions, especially in the high-frequency coefficients in the transform domain. A comparison is shown in Figure 5 and Figure 6. Figure 5 shows the high-frequency coefficients in the vertical direction of Figure 3, which is produced by one-layer NSST in eight directions. Two images of different reference heights both have rich and clear detailed information in the focused and defocused regions. Therefore, for the transform domain methods, defocused regions may be extracted as focused regions and fused into the result, which seriously affects the quality of the image fusion. In Figure 6, only the focused regions have clear edges, which is suitable for the transform domain methods to detect focused and defocused regions. To draw a conclusion, the transform domain methods are not able to meet the requirements of CSAR image fusion.

Spatial domain methods recognize the focused and defocused regions directly in the image domain, which is less likely to lose information during processing and can retain more original information from source images. Moreover, they are adaptable to different types of images, including CSAR images, and do not require complex transformations or training, which results in a small amount of computation and a fast-processing speed. Based on the analysis above, it is suitable to utilize spatial domain methods in this article.

2.2. Spatial Domain Method Based on SML

Spatial frequency (SF), average gradient (AG), and SML are usually used in spatial domain methods as focus measures. The SML is locally adaptable and can maintain image edge information better, which makes the fusion result clearer and richer in detailed information. At the same time, the SML is highly interpretable because the weight information that is closely related to the pixel position and gradient of each pixel has a clear physical meaning. The definition of the SML is detailed as follows.

Compared with the traditional sum of Laplacian, the SML considers the contribution of pixels in eight directions that are horizontal, vertical, and diagonal to the modified Laplacian (ML) of the center pixel. The ML is strongly directional and effective in preserving detailed information such as edges and textures. The SML is defined as

$$SML\left(i,j\right)={\displaystyle \sum _{m=-M}^M{\displaystyle \sum _{n=-N,}^{N}ML\left[\left(i+m\right),\left(j+n\right)\right]}},$$

(1)

with $\left(i,j\right)$ as the coordinates of pixels. $M$ and $N$ are the size of the accumulating window. $ML\left(i,j\right)$ denotes the ML of pixel $\left(i,j\right)$, which is described as

$$\begin{array}{ll}ML\left(i,j\right)& =\left|2I\left(i,j\right)-I\left(i-step,j\right)-I\left(i+step,j\right)\right|\\ & +\left|2I\left(i,j\right)-I\left(i,j-step\right)-I\left(i,j+step\right)\right|\\ & +\left|1.4I\left(i,j\right)-0.7I\left(i+step,j-step\right)-0.7I\left(i-step,j+step\right)\right|\\ & +\left|1.4I\left(i,j\right)-0.7I\left(i-step,j-step\right)-0.7I\left(i+step,j+step\right)\right|,\end{array}$$

(2)

while the pixel value is denoted by $I\left(i,j\right)$, $step$ denotes variable pitch. The effect of variable-step pixels in eight directions is considered in the ML. For the center pixel’s ML, the closer pixel has a greater contribution. Therefore, the weight of the contribution is set as 1 for horizontal and vertical pixels and 0.7 for diagonal pixels. The computational process above involves redundant matrix summation operations, so we adopt the modified computation method for SML proposed in [30] to improve the computational efficiency.

2.3. Concept of Guided Filter

However, on the one hand, the focused and defocused regions will be selected in the initial decision maps, of which the valid values are 1. If fusing the source images and the initial decision maps directly, the defocused regions will be entirely fused into the fused image. We experience that the energy of the defocused regions is lower than that of the focused regions. According to [29], the guided filter can smooth the input images by using the guided images. Therefore, we use the source images as guided images to smooth the initial decision maps, which can minimize the valid values of the defocused regions based on the energy of the pixels. It is said that, in the fused image, the energy of the defocused regions is weakened, but the focused regions are wholly fused.

On the other hand, the initial decision maps are blurred near the FDB, as shown in Figure 7, which may lead to faults in the fusion result. The guided filter is a local linear filter with edge-preserving and smoothing properties while subtracting noise. Guided by source images, the outputs of the guided filter are more organized than the input images.

According to [29], the definition of the guided filter is based on the assumption that the relationship between the guided image $G$ and output image $O$ is locally linear. The local linear variation defined within a window ${\omega }_k$ centered at a pixel $k$ with a radius $r$ can be expressed as

$$O_i=a_k{G}_i+b_k,\forall i\in {\omega }_k,$$

(3)

where $a_k$ and $b_k$ are linear coefficients that are constant in the window ${\omega }_k$. The determination of $a_k$ and $b_k$ is limited by constraints from the input image $I$, which is to minimize the difference between $O$ and input image $I$. The cost function in the window ${\omega }_k$ can be expressed as

$$E\left(a_k,b_k\right)={\displaystyle \sum \left({\left(a_k{G}_i+b_k-I_i\right)}^2+\epsilon a_k^2\right)},$$

(4)

with $\epsilon$ as a regularization parameter penalizing large $a_k$. $a_k$ and $b_k$ are given by the linear ridge regression model.

$$a_k={\displaystyle \frac{\frac{1}{\left|\omega \right|}{\displaystyle {\sum }_{i\in {\omega }_k}{G}_i{I}_i}-{\mu }_k{\overline{I}}_k}{{\sigma }_k^2+\epsilon }},$$

(5)

$$b_k={\overline{I}}_k-a_k{\overline{I}}_k,$$

(6)

where ${\mu }_k$ and ${\sigma }_k^2$ are the mean and variance of $G$ in window ${\omega }_k$. $\left|\omega \right|$ is the number of pixels in ${\omega }_k$. ${\overline{I}}_k={\displaystyle \frac{1}{\left|\omega \right|}}{\displaystyle {\sum }_{i\in {\omega }_k}{I}_i}$ is the mean of $I$ in ${\omega }_k$. Having obtained $a_k$ and $b_k$, $O$ can be solved by (3). However, the results $O_i$ may be different because it will be covered by different windows. The average of all the possible values of $O_i$ is used in this article, so the filtering output can be computed by

$$O_i=\overline{a}{G}_i+\overline{b},$$

(7)

with $\overline{a}={\displaystyle \frac{1}{\left|\omega \right|}}{\displaystyle {\sum }_{k\in {\omega }_i}{a}_k}$, $\overline{b}={\displaystyle \frac{1}{\left|\omega \right|}}{\displaystyle {\sum }_{k\in {\omega }_i}{b}_k}$ is the average coefficients of all windows overlapping pixels $O_i$.

The performance of the filter is mainly affected by parameters that are the radius $r$ of ${\omega }_k$ and the regularization parameter $\epsilon$. Therefore, when utilizing the guided filter, adjust these two parameters as needed.

2.4. Data Process

When the object’s height differs from a reference height, the imaging result will be defocused, of which the degree is positively correlated with the height difference between the object’s height and reference heights. Inspired by multi-focus image fusion in the optical images, we utilize a spatial-domain fusion method based on the SML and a guided filter to obtain an all-in-focus image of ROI with terrain undulation. The real-measured data processing diagram is shown in Figure 8, which has been detailed in [31]. This article focuses on introducing the proposed image fusion method, and the framework of the proposed method briefly includes four steps, as shown in Figure 9. First, a high-resolution imaging algorithm is used to obtain full-aperture CSAR images in different reference heights. Second, take all of the images obtained in the first step as source images and calculate the SMLs of them. Then, initial decision maps are estimated based on the rule of taking the largest SML pixel by pixel. Fourth, correct the initial decision maps using the guided filter. Finally, the decision maps and the source images are fused to obtain the final image.

2.4.1. Multi-Layer Imaging

The focused quality of objects is closely related to the selection of reference heights. The closer the object’s height is to the reference height, the better the focused quality of the imaging result is. Since the heights of the objects within the same region are different, multi-layer imaging is necessary, whose schematic is shown in Figure 10a. The range of reference heights $H$ is determined by roughly estimating the range of the objects’ heights within ROI. Then, the height interval $\Delta H$ is determined according to the resolution of the CSAR system. The displacement of the imaging result is related to the height mismatch. When the displacement is less than the CSAR system’s resolution, it can be ignored. Therefore, the $\Delta H$ can be calculated as [31]

$$\Delta H<{\displaystyle \frac{2\rho }{\tan {\theta }_0}},$$

(8)

where $\rho$ is the resolution of the CSAR system, and ${\theta }_0$ is the pitch angle of the center point of the scene, as shown in Figure 10b.

Finally, the BP algorithm [4] is utilized to process the echo.

2.4.2. Focus Region Detection

The focused and defocused regions of CSAR images both have clear edge information and appear sharply, so it is a challenge to distinguish them. Unlike the transform domain methods, the spatial domain methods are sensitive to spatial information. In this article, an SML is selected to be the focus measure. Theoretically, a larger SML denotes the pixel with higher focused quality. Focused and defocused regions are distinguished relying on the rule of choosing the largest SML pixel by pixel, which is implemented after calculating the source images’ SMLs. The process of obtaining initial decision maps can be represented as

$$bd{d}_x\left(i,j\right)=\left\{\begin{array}{l}1,ifsm{l}_x\left(i,j\right)=\max SML\\ 0,otherwise\end{array}\right.,$$

(9)

where $SML=\left\{sm{l}_1,sm{l}_2,\cdots sm{l}_N\right\}$, $sm{l}_x$ is the SML of the source image $I_x$.

2.4.3. Guided Filter

Since the initial decision maps are blurred near the FDB, the guided filter introduced in Section 2 is utilized to optimize the initial decision maps. The process can be described as

$$BD{D}_x=guidedfilter\left(I_x,bd{d}_x,r,\epsilon \right),$$

(10)

with $BD{D}_x$ as the final decision map of source image $I_x$. $r$ denotes the radius of the window ${\omega }_k$, and $\epsilon$ is a regularization parameter, which has been detailed in Section 2.

2.4.4. Fused Result

After obtaining the final decision maps, the fused image $I_F$ is computed by fusing source images and final decision maps, which is represented as

$$I_F={\displaystyle \sum _{x=1}^N{I}_x·BD{D}_x}.$$

(11)

3. Experiments and Results

Data were collected by an L-band CSAR system in Shanxi Province, China. Further system parameters are listed in

Table 1. Basic system parameters.

Parameters	Values
Carrier frequency $f_c$	L band
The velocity of the platform $v_a$	52 m/s
The flight radius of the platform $R$	2000 m
The altitude of the platform $H$	2000 m

. The aircraft flew along a circular trajectory with a radius of 2000 m at a height of 2000 m, spotlighting at a circular ROI whose radius is 300 m. As shown in Figure 11, the ROI is centered on an island with objects of different heights, such as buildings, and marinas, whose optical images are shown in Figure 12. There is a square in the northeast and streetlamps along a road in the north. The size of the imaging result is 4000 pixels × 4000 pixels with a 0.15 m × 0.15 m sampling interval and a theoretical resolution of 1 m by incoherent imaging.

By estimating the height range of objects in ROI, we set 24 reference heights in a range of −1.8–2.8 m at a 0.2 m interval. In total, 24 full-aperture images with different reference heights were obtained using the BP algorithm, where the images with reference heights of −1.6 m and 1.4 m are shown in Figure 13. Focused regions are different on images of different reference heights, which is obvious, especially in regions marked with red rectangles in Figure 13a whose zoomed-in results are shown in Figure 14. Objects in the lower part of Regions 1 and 2 are focused when the reference height is −1.6 m because there are below-grade marinas whose heights are closer to −1.6 m compared with 1.4 m. Meanwhile, Region 3 and the upper part of Regions 1 and 2 are focused when the reference height is 1.4 m. Moreover, streetlamps along the road marked with blue rectangles are focused as points in Figure 13b because they are higher than the ground. The conclusion is adaptive to other images. It indicates that multi-layer imaging enables objects with different heights to focus on images whose reference height is close to its height.

To verify the excellent performance of the spatial domain methods in CSAR image fusion, the proposed method is compared with three representative methods: the AG-based method, the NSST [23], and the pulse-coupled neural network (PCNN) [32]. The AG-based method is a spatial domain method whose focus measure is AG, which is a classical focus measure in image fusion. The NSST is a classical method of the transform domain. The PCNN can be categorized as a deep learning method, but it does not depend on a training data set for training and prediction, so it can be used in CSAR image fusion.

3.1. Evaluation Metrics

A well-performing image fusion method needs to fulfill the following requirements. First, it should be able to extract complementary information between fused images accurately. Second, a high degree of jerkiness and reliability is necessary. Third, introducing information and errors that are incompatible with the human visual system should be avoided.

Subjective evaluation relies on human subjective judgment, and the reliability of results is closely related to the experience of the observer, which makes the evaluation process time consuming and the results unrepeatable. In contrast, supported by a complete theoretical derivation process, objective evaluation is highly interpretable and can objectively assess the similarity between fused images and source images. Therefore, both subjective and objective evaluation should be included in the image fusion effect evaluation system [33]. The definition of objective evaluation metrics used in this article is as follows.

• All Cross-Entropy (ACE)

Cross-entropy (CE) can describe the difference between two images by calculating the ratio of the distribution probability of gray values. $C{E}^{AF}$ is the value of CE between the source image $A$ and the fused image $F$, which can be described as

$$C{E}^{AF}={\displaystyle \sum _k{P}_A\left(k\right)\cdot {\log }_2\frac{P_A\left(k\right)}{P_F\left(k\right)}},$$

(12)

with $P_A\left(k\right)$ and $P_F\left(k\right)$ as the distribution probability of the gray values of $A$ and $F$, respectively. $k\in \left[0,255\right]$ is the range of gray values.

ACE is defined as the average of CEs between source images $A$ and $B$,

$$ACE={\displaystyle \frac{C{E}^{AF}+C{E}^{BF}}{2}}.$$

(13)

A smaller $ACE$ indicates that the overall difference between the source images and the fused image is smaller.

2.

Structural similarity (SSIM)

SSIM [34] reflects the similarity of the two images. $SSI{M}^{AF}$ is the value of SSIM between $A$ and $F$

$$SSI{M}^{AF}={\displaystyle \frac{\left(2{\mu }_A{\mu }_F+c_1\right)\left(2{\sigma }_{AF}+c_1\right)}{\left({\mu }_A^2+{\mu }_F^2+c_1\right)\left({\sigma }_A^2+{\sigma }_F^2+c_2\right)}},$$

(14)

where ${\mu }_A$ and ${\mu }_F$ are the average of pixels of $A$ and $F$, respectively, whose variances are ${\sigma }_A^2$ and ${\sigma }_F^2$. The covariance of $A$ and $F$ is ${\sigma }_{AF}$. $c_1={\left(k_{1}L\right)}^2$ and $c_2={\left(k_{2}L\right)}^2$ are constants used to maintain stability, where $L$ is the dynamic range of pixel values $k_1=0.01$ and $k_2=0.03$. The definition of $SSI{M}^{BF}$ is the same as $SSI{M}^{AF}$. $SSI{M}_{avg}$ can be calculated as

$$SSI{M}_{avg}={\displaystyle \frac{SSI{M}^{AF}+SSI{M}^{BF}}{2}}.$$

(15)

A smaller $SSIM$ value means better performance.

3.

Sum mutual information (SMI)

The evaluation metric SMI [35] calculates the image feature information between the source images and the fused image, such as gradient and edge. The SMI between $A$, $B$, and $F$ is defined as follows.

$$SMI=M{I}^{AF}+M{I}^{BF},$$

(16)

$$M{I}^{AF}={\displaystyle \sum _{a,f}{p}_{AF}\left(a,f\right)\log \frac{p_{AF}\left(a,f\right)}{p_A\left(a\right)p_F\left(f\right)}},$$

(17)

where $a$ and $f$ are the normalized gray values of $A$ and $F$, respectively. $p_A\left(a\right)$ and $p_F\left(f\right)$ denote the normalized grayscale histogram probability distribution function. $p_{AF}\left(a,f\right)$ is the probability distribution function of the joint grayscale histogram between $A$ and $F$. A larger value of $SMI$ indicates that the information of the fused image is similar to that of the source images.

4.

Edge retention (ER)

The ER is called $Q^{AB/F}$, which evaluates the amount of edge information transferred from the source images to the fused image [36]. $Q^{AB/F}$ is defined as

$$Q^{AB/F}={\displaystyle \frac{{\displaystyle \sum _{i=1}^R{\displaystyle \sum _{j=1}^C\left[Q^{AF}\left(i,j\right)w^A\left(i,j\right)+Q^{BF}\left(i,j\right)w^B\left(i,j\right)\right]}}}{{\displaystyle \sum _{i=1}^R{\displaystyle \sum _{j=1}^C\left[w^A\left(i,j\right)+w^B\left(i,j\right)\right]}}}},$$

(18)

where $w^A\left(i,j\right)$ is the weights of $Q^{AF}$ in $Q^{AB/F}$.

$Q^{AF}=Q_g^{AF}\left(i,j\right)\ast Q_{\alpha }^{AF}\left(i,j\right)$ is the edge information hold value, with $Q_g^{AF}\left(i,j\right)$ and $Q_{\alpha }^{AF}\left(i,j\right)$ as the edge intensity and the direction of the edge gradient of $A$ and $F$, respectively. The calculation process is as follows.

Firstly, the edge strengths $g_A$ and $g_F$ and the edge directions ${\alpha }_A$ and ${\alpha }_F$ of $A$ and $F$ are calculated as

$$\left\{\begin{array}{l}{g}_A\left(i,j\right)=\sqrt{s_A^x{\left(i,j\right)}^2+s_A^y{\left(i,j\right)}^2}\\ g_F\left(i,j\right)=\sqrt{s_F^x{\left(i,j\right)}^2+s_F^y{\left(i,j\right)}^2}\end{array}\right.,$$

(19)

$$\left\{\begin{array}{l}{\alpha }_A\left(i,j\right)={\tan }^{-1}\left({\displaystyle \frac{s_A^y\left(i,j\right)}{s_A^x\left(i,j\right)}}\right)\\ {\alpha }_F\left(i,j\right)={\tan }^{-1}\left({\displaystyle \frac{s_F^y\left(i,j\right)}{s_F^x\left(i,j\right)}}\right)\end{array}\right.,$$

(20)

with $s_A^x\left(i,j\right)$ and $s_A^y\left(i,j\right)$ as the edge intensity of $A$ in directions of $x$ and $y$, respectively. $s_F^x\left(i,j\right)$ and $s_F^y\left(i,j\right)$ are similarly defined.

Secondly, relevant edge intensity $Q_g^{AF}\left(i,j\right)$ and relevant edge direction $Q_{\alpha }^{AF}\left(i,j\right)$ can be represented as

$$g^{AF}\left(i,j\right)=\left\{\begin{array}{l}{\displaystyle \frac{g_F\left(i,j\right)}{g_A\left(i,j\right)}},if\left(g_A\left(i,j\right)>g_F\left(i,j\right)\right)\\ {\displaystyle \frac{g_A\left(i,j\right)}{g_F\left(i,j\right)}},otherwise\end{array}\right.,$$

(21)

$${\alpha }^{AF}=\left|1-{\displaystyle \frac{\left|{\alpha }_A\left(i,j\right)-{\alpha }_F\left(i,j\right)\right|}{\pi /2}}\right|,$$

(22)

$$Q_g^{AF}={\displaystyle \frac{{\Gamma }_g}{1+\exp \left(K_g\left({\alpha }^{AF}\left(i,j\right)-{\sigma }_g\right)\right)}},$$

(23)

$$Q_{\alpha }^{AF}={\displaystyle \frac{{\Gamma }_{\alpha }}{1+\exp \left(K_{\alpha }\left({\alpha }^{AF}\left(i,j\right)-{\sigma }_{\alpha }\right)\right)}},$$

(24)

$L$, ${\Gamma }_g$, $K_g$, ${\sigma }_g$, ${\Gamma }_{\alpha }$, $K_{\alpha }$, and ${\sigma }_{\alpha }$ in the derivation of the equation above are hyper parameterizations that are set as $L=1$, ${\Gamma }_g=0.9879$, $K_g=-15$, ${\sigma }_g=0.5$, ${\Gamma }_{\alpha }=0.9994$, $K_{\alpha }=-15$, and ${\sigma }_{\alpha }=0.8$ according to [36].

Finally, $w^A\left(i,j\right)$ is calculated as

$$w^A\left(i,j\right)={\left[g_A\left(i,j\right)\right]}^L.$$

(25)

$Q^{AB/F}$ can describe the edge strength and orientation information obtained by the Sobel operator in each pixel of $A$, $B$, and $F$. The larger the value of $Q^{AB/F}$ is, the more edge information will be transferred from the source images to the fused image. It can also be used as a metric for evaluating image sharpness in this article.

5.

Equivalent Number of Looks (ENL)

ENL reflects the intensity of the noise contained in the SAR image. It is the square of the ratio of the mean to the standard deviation of the SAR magnitude plot, the ENL of image $F$ is defined as:

$$EN{L}_F={\left({\displaystyle \frac{\mu }{\sigma }}\right)}^2,$$

(26)

where $\mu$ and $\sigma$ are the mean and the standard deviation of image $F$. The smaller the ENL, the smaller the noise contained in the image, and the better the image quality.

3.2. Parameter Setting

In the process of the proposed method, the parameters for calculating SML are set as $step=15$ and $N=M=30$. For the guided filter, the radius of the window ${\omega }_k$ is $r=8$, and the regularization parameter is $\epsilon ={0.4}^2$. The fusion rule for the low-frequency coefficients of NSST is averaged pixel by pixel, which is the only difference from [23]. The high-frequency coefficients are decomposed in four layers, each of which is decomposed in eight directions, and the fusion rule is SML taking the largest one pixel by pixel. The processing of PCNN is consistent with [32]. However, the computation of PCNN is time-consuming. To operate an effective and fair comparative experiment, only two images shown in Figure 13 are used as source images to test the performance of the proposed method and comparative methods.

3.3. Experimental Results and Discussion

Experimental results are shown in Figure 15; there are differences in the details. To have a clear comparison, we zoomed in on the same regions that are labeled with red rectangles in Figure 15a, as shown in Figure 16, where Figure 16a–d correspond to that in Figure 15, and images in each group are the zoomed-in results of Regions 1–3 from top to bottom, respectively. The results of the AG-based method, NSST, and PCNN are blurred near FDB because the defocused regions are fused. In contrast, the proposed method can distinguish the focused and defocused regions correctly and suppress the energy of defocused regions. With the help of the guided filter, the fused result of the proposed method is continuous and smooth near FDR, which is more compatible with the human visual system.

The results of the objective evaluation metrics are listed in

Table 2. Objective evaluation metrics of different methods.

Method	${\mathit{Q}}^{\mathit{A}\mathit{B}/\mathit{F}}$	ACE	SMI	SSIM	ENL	Time (s)
Our	0.4979	0.0064	4.8886	0.8748	2.7061	51
AG	0.5243	0.0050	5.5499	0.8894	2.7686	647
NSST	0.3652	-	-	-	-	1726
PCNN	0.3291	-	-	-	-	9186

Note: The best results are shown in bold.

, where the best results are shown in bold. The $Q^{AB/F}$ describes the edge strength and orientation information. The AG-based method distinguishes the focused and defocused regions by analyzing the AG value of each pixel, so it performs better in keeping the edge strength and orientation information than the proposed. Meanwhile, a small $Q^{AB/F}$ value indicates the grayscale value of the image changes smoothly, indicating that the image is relatively blurry and bad quality. Therefore, we can know that the NSST and PCNN methods are worse than other methods. In the subsequent discussion, we focus on comparing the proposed method and the AG-based method.

$ACE$ is used to measure the similarity between the probability distributions of grayscale values in two images. $SMI$ describes the total amount of information obtained from the source images by the fused image. $SSIM$ is the evaluation metric to measure the degree of structural similarity between the fused image and the source images. In summary, these metrics evaluate the similarity between several images from different aspects, which are simultaneously used to comprehensively measure the performance of the proposed method. In the field of image fusion, the more the defocused regions fused into the fusion result, the higher the similarity between the source images and the fusion result, and the worse the effect of the image fusion. Therefore, a large value of $ACE$ and small values of $SMI$ and $SSIM$ illustrate that the fusion method performs well. Combining the evaluation results of $ACE$, $Q^{AB/F}$, and $SMI$, we can know that the proposed method is better than the AG-based method.

$ENL$ can measure the intensity of noise contained in a single SAR image. It is listed that the proposed method achieves the smallest value of $ENL$, which indicates that the proposed method has the best suppression effect on noise. In terms of computational time, the AG-based method, NSST, and PCNN are 13, 34, and 180 times more than the proposed method, so the proposed method has the highest computational efficiency among these four methods. In summary, the results of objective evaluation metrics show that the proposed method has the best performance.

To have a more intuitive comparison, the marked regions’ zoomed-in results are shown in Figure 16. The proposed method accurately recognizes and extracts the focused regions on source images, and the outline of each object on the fused image is clear and readable. The other methods yielded poorly readable results because they blended the defocused regions, resulting in blurring around the objects.

Considering the results of the subjective and objective evaluations, the proposed method performs the best. It can distinguish the focused and defocused regions exactly to produce a highly readable all-in-focus image.

Finally, 24 full-aperture images with different reference heights were processed using the proposed method. An all-in-focus image is shown in Figure 17. To assess the imaging results’ quality, Figure 18 shows the image slices of the marked target, where the target is best focused when the imaging reference height is 1.4 m. The imaging quality of the fused result is similar to that of Figure 18b, subjectively. To have an objective analysis, the X and Y slices of the marked target in these four images are compared in Figure 19. The spike of the blue slice in Figure 19b is the peak energy of the defocused region. It is clear that the best-focused result is fused into the fusion image, which fully verifies the correctness and effectiveness of the proposed method.

However, when the energy of the defocused region is similar to that of the focused regions, the defocused region will be fused into the final result. This raises the question of how to set parameters to achieve better fusion results. We will do further research on this.

4. Conclusions

Multi-focus image fusion provides a new approach to solving the problem of CSAR imaging in areas with terrain undulation. This article focused on finding an image fusion method that is suitable for CSAR images. The proposed method is a spatial domain method based on the SML and the guided filter. The SML is a focus measure, and the guided filter is utilized to optimize the decision maps. An all-in-focus CSAR image of ROI with terrain undulation can be obtained by the proposed method. A comparative experiment was taken, which consisted of the AG-based method, NSST, and PCNN, in addition to the proposed method. The conclusions of the subjective and objective evaluations illustrated the superior performance of the proposed method compared to the other methods. Furthermore, the processing result of the real-measured data verified the correctness and effectiveness of the proposed method. However, it should be noted that the settings of the reference heights and fusion parameters directly affect the quality of image fusion, so how to carry out appropriate parameter settings deserves further research.