Fractal Dimension-Based Multi-Focus Image Fusion via Coupled Neural P Systems in NSCT Domain

Abstract

In this paper, we introduce an innovative approach to multi-focus image fusion by leveraging the concepts of fractal dimension and coupled neural P (CNP) systems in nonsubsampled contourlet transform (NSCT) domain. This method is designed to overcome the challenges posed by the limitations of camera lenses and depth-of-field effects, which often prevent all parts of a scene from being simultaneously in focus. Our proposed fusion technique employs CNP systems with a local topology-based fusion model to merge the low-frequency components effectively. Meanwhile, for the high-frequency components, we utilize the spatial frequency and fractal dimension-based focus measure (FDFM) to achieve superior fusion performance. The effectiveness of the method is validated through extensive experiments conducted on three benchmark datasets: Lytro, MFI-WHU, and MFFW. The results demonstrate the superiority of our proposed multi-focus image fusion method, showcasing its potential to significantly enhance image clarity across the entire scene. Our algorithm has achieved advantageous values on metrics $Q_{AB/F}$, $Q_{CB}$, $Q_{CV}$, $Q_E$, $Q_{FMI}$, $Q_G$, $Q_{MI}$, and $Q_{NCIE}$.

1. Introduction

Multi-focus image fusion (MFIF) is a sophisticated process used in image processing that involves combining multiple images taken with different focus settings into a single image where all objects are in focus [1]. This technique is particularly useful in scenarios where the depth of field is limited and parts of the scene are out of focus in each image. By integrating these images, multi-focus image fusion produces a composite that is entirely sharp and clear [2].

The process of multi-focus image fusion involves several key techniques and steps: (1) Image registration: aligning the multiple images accurately is crucial, as even slight misalignments can lead to poor fusion results; (2) Focus measurement: this involves assessing the focus level of different parts of each image, often using clarity or sharpness metrics; (3) Fusion algorithm: the core of the process, where algorithms decide how to combine the sharp portions of each input image into the final composite. Techniques vary from simple averaging to complex wavelet-based methods; (4) Post-processing: enhancing the fused image to improve visual quality or to prepare it for analysis, including tasks like contrast adjustment or noise reduction [3,4].

MFIF is not without its challenges. These include handling misalignments, reducing artifacts that can arise during fusion, and dealing with variations in exposure and color balance among the source images. Advances in computational photography, machine learning, and deep learning have led to more sophisticated fusion algorithms that can more effectively address these challenges, resulting in higher-quality fused images. Various algorithms and techniques, including multi-resolution analysis, image decomposition, and feature-based methods, have been developed to effectively fuse multi-focus images. One of the key challenges in image fusion is to ensure that important information from both input images is preserved and enhanced in the fused image, without introducing artifacts or losing critical details [5,6,7].

In traditional image fusion algorithms, the multi-scale transforms such as nonsubsampled contourlet transform (NSCT) [8] and nonsubsampled shearlet transform (NSST) [9] are commonly employed. These are mathematical tools that are used for image analysis and processing. They offer an enhanced representation of local features and multi-scale analysis capabilities, making them particularly suitable for tasks like image fusion. Li et al. [10] introduced the medical image fusion approach using NSST. Lv et al. [11] proposed MFIF via parameter-adaptive pulse-coupled neural network and fractal dimension in NSST domain. Li et al. [12] introduced the image fusion algorithm based on spatial frequency and improved sum-modified-Laplacian in NSST domain. Coupled neural P (CNP) systems were proposed by Peng et al. [13] in 2019, and Li et al. [14] proposed the medical image fusion method based on coupled neural P systems (CNP) in NSST domain; the experiments show that the CNP systems achieve excellent results in image fusion. Although the algorithm achieves good image fusion results, it is only suitable for images of size $m\times m$. For images of size $m\times n$, due to the necessity of preprocessing the image with NSST to obtain an $m\times m$ or $n\times n$ image, followed by decomposition, applying different fusion rules to obtain the fused image, and finally resizing the image back to $m\times n$, information loss and distortion occur to varying degrees during this process. Therefore, NSST is more suitable for the image fusion processing of $m\times m$-sized images. NSCT can directly process images with size $m\times n$, which can reduce image distortion and information loss. Li et al. [15] proposed the MFIF method via NSCT and achieved an excellent fusion effect.

To improve the clarity and information complementarity of the fused images, we propose a novel MFIF method based on fractal dimension and coupled neural P systems (CNP) in NSCT domain. The main contributions of our paper are as follows:

(1)

The coupled neural P systems (CNP) are used to process low-frequency components in order to obtain better background information;

(2)

A fractal dimension-based focus measure (FDFM) combined with spatial frequency (SF) is used to process high-frequency components, thereby obtaining more detailed image information;

(3)

Through extensive qualitative and quantitative experiments conducted on three datasets, our method consistently outperforms state-of-the-art (SOTA) techniques, demonstrating superior performance.

The remainder of the article is structured into six sections. Section 2 provides an overview of related works. In Section 3, the nonsubsampled contourlet transform is introduced. Section 4 presents the introduction of CNP systems. The proposed method is detailed in Section 5. Section 6 covers experimental results and discussions, while Section 7 delves into further discussions.

2. Related Works

In this section, we provide a concise overview of the existing literature on MFIF, categorizing it into three main approaches: spatial domain-based methods, transform domain-based methods, and deep learning-based methods [16].

2.1. Spatial Domain-Based Image Fusion

Spatial domain-based fusion methods involve the direct manipulation of pixel values using specific algorithms designed for image fusion. Among them, image fusion algorithms based on edge-preserving filtering are the most commonly used. These filters typically include guided image filtering [17], rolling guidance filtering [18], Gaussian curvature filter [19], etc. Fiza et al. [20] proposed a technique for MFIF specifically tailored for satellite images. It addresses issues such as visual distortion and spatial inconsistencies at sharp edges by introducing the edge discriminative diffusion filter (EDDF). EDDF combines anisotropic diffusion (AD) with guided filter (GF) to discriminate between local and global features, preserving edges while maintaining spatial consistency. The technique involves transforming source images into detail and base layers to extract low-pass and high-pass information, processing saliency maps through EDDF to generate weight maps, and, finally, combining fused detail and base layers to produce the desired fused image. Quantitative and qualitative tests demonstrate that the proposed technique outperforms some SOTA alternatives.

Yan et al. [21] introduced a novel MFIF approach, utilizing dictionary learning alongside a rolling guidance filter to accommodate both registered and mis-registered input images. Initially, a dictionary is learned from classical multi-focus images blurred by the rolling guidance filter. Then, a model is proposed for identifying focus regions by applying the learned dictionary to input images, generating focus feature maps. These maps are compared to derive an initial decision map, which is optimized and applied to the input images to produce fused images. Experimental results demonstrate the competitiveness of the proposed algorithm with the current state of the art, particularly excelling when handling both well-registered and mis-registered input images.

Adeel et al. [22] introduced a two-stage spatial domain framework for MFIF, which finds applications in computer vision. Initially, the salient features of focused regions are detected using Gaussian curvature filter (GCF) and range filtering. Subsequently, morphological filters refine the initial focus detection map. Experimental results demonstrate the method’s speed and robustness, outperforming recent multi-focus fusion schemes.

Tang et al. [23] introduced the image fusion technique that utilizes a simple weighted least squares filter. In their approach, source images are first decomposed into base and detail layers using this filter. The detail layers are then fused through a sub-window variance filter. For the base layer, they developed a fusion strategy that integrates visual saliency mapping with adaptive weight assignment techniques. Although the algorithm enhances the details, the fusion image as a whole suffers from some distortion.

2.2. Transform Domain-Based Image Fusion

Image fusion approaches via transform domains involve processing the transformed coefficients of source images using various transforms such as contourlet transform [24,25], shearlet transform [26,27], gradient domain [28], sparse representation [29,30], etc. Subsequently, these coefficients are inversely transformed back into the spatial domain. This approach typically comprises three primary stages: image transform, coefficient fusion, and inverse transform.

Jie et al. [31] proposed a method for enhancing full-field optical angiography (FFOA) images to achieve full focus, thereby improving its clinical utility. Existing FFOA techniques suffer from a limited depth of focus, resulting in partially unclear images. The method utilizes NSCT and contrast spatial frequency to fuse FFOA images effectively. Firstly, source images are decomposed into low-pass and bandpass images. Then, a sparse representation-based rule fuses low-pass images, while a contrast spatial frequency rule fuses bandpass images, considering pixel correlation and gradient relationships. Finally, a fully focused image is reconstructed. Experimental results demonstrate the superiority of the proposed method over SOTA approaches in both qualitative and quantitative evaluations, offering promising prospects for clinical applications in disease prevention and diagnosis.

Lu et al. [32] presented a novel focus measure called the sum of Gaussian-based fractional order differentiation (SGFD) to improve the accuracy of detecting low-frequency regions in MFIF. SGFD outperforms traditional focus measures by retaining more low-frequency information. The fusion process involves initial fusion using the NSST and SGFD, followed by refinement of the initial decision map through quadtree decomposition. Residual regions are identified and removed to generate the final fused image. Comparative experiments with SOTA approaches demonstrate that the SGFD-based approach excels in both subjective visual quality and objective metrics, offering promising advancements in MFIF.

Paul et al. [28] introduced a method for fusing color images that addresses both multi-exposure and multi-focus scenarios. It achieves this by blending the gradients of the luminance components of input images, prioritizing the maximum gradient magnitude at each pixel location. The fused luminance is then obtained through a Haar wavelet-based image reconstruction approach.

Tang et al. [33] proposed a sparse representation-based fusion approach for image fusion. Traditional methods often suffer from including redundant information during dictionary learning, leading to artifacts and increased computational time. To address this, the proposed method introduces a novel dictionary construction method based on joint patch grouping and informative sampling. Nonlocal similarity is utilized for joint patch grouping across all source images, simplifying the calculation by selecting only one class of informative image patches. Sparse coefficients are obtained using the orthogonal matching pursuit (OMP) algorithm, and a max-L1 fusion rule is employed for image reconstruction. Experimental results demonstrate the superiority of the proposed approach in effectively fusing multi-focus images.

Chen et al. [34] introduced an image fusion approach via a complex sparse representation (CSR) model. This model employs hypercomplex signal properties to derive directional information from real-valued signals by extending them into the complex domain. Following this, the directional aspects of the input signal are separated into sparse coefficients using corresponding directional dictionaries.

2.3. Deep Learning-Based Image Fusion

Deep learning has been widely applied in the field of image processing, including tasks such as image classification [35], image segmentation [36], object detection [37], image enhancement [38], image fusion [39], etc. The commonly used deep learning models have CNNs [40], GANs [41], autoencoders [42], etc. There has been extensive research into leveraging deep learning techniques for image fusion [43]. This has led to the emergence of numerous methods that utilize deep learning for MFIF. Zhang et al. [44] introduced the IFCNN fusion framework, a convolutional neural network (CNN)-based approach that offers a comprehensive fusion solution. Notably, it can undergo end-to-end training without necessitating preprocessing steps. Hu et al. [45] proposed a ZMFF method based on a deep prior network. Zhang et al. [46] introduced a fast unified image fusion network called PMGI. This network addresses various image fusion tasks like medical image fusion and MFIF. The key idea is to maintain the proportional relationship between texture and intensity information from source images. Xu et al. [47] introduced an innovative unified and unsupervised end-to-end image fusion network (U2Fusion).

Traditional algorithms have the following two advantages compared to deep learning: (1) Traditional algorithms typically operate based on mathematical models and specific rules, making their results easier to interpret and control; (2) Researchers can clearly understand how each step influences the final fusion outcome and can adjust parameters and methods according to their needs.

3. Nonsubsampled Contourlet Transform

The nonsubsampled contourlet transform (NSCT) is an advanced mathematical tool used for signal and image analysis, extending the principles of the contourlet transform but with significant improvements in handling images [48,49,50]. Developed to overcome some of the limitations of previous multi-scale and multi-directional transforms, the NSCT offers a flexible, multi-resolution, multi-direction, and shift-invariant framework for image decomposition. The NSCT decomposes images into components at multiple scales and orientations. It performs this through two main stages: a nonsubsampled pyramid (NSP) structure for capturing point discontinuities and a series of nonsubsampled directional filter banks (NSDFB) for linking point discontinuities into linear structures. This approach is adept at capturing edges and textures in an image, which are essential features in many image processing tasks. Figure 1a displays an overview of the proposed NSCT. The structure consists of a bank of filters that splits the 2-D frequency plane in the sub-bands illustrated in Figure 1b.

The NSCT requires selecting appropriate scale and directional parameters, which can significantly impact the final processing results and necessitate experience and experimentation to optimize. Due to its involvement with multi-scale and multidirectional data representations, NSCT requires high memory demands, especially when processing large-scale data.

4. Coupled Neural P Systems

For the fusion of multi-focus images, coupled neural P (CNP) systems are designed as an array of neurons with local topology, i.e., CNP systems with local topology [13,14].

A CNP system with local topology, of degree $m\times n$, is delineated as follows:

$$\Pi =\left(O,{\sigma }_{11},{\sigma }_{12},\cdots ,{\sigma }_{1n},\cdots ,{\sigma }_{m1},{\sigma }_{m2},\cdots ,{\sigma }_{mn},\mathrm{syn}\right)$$

(1)

where

(1)

$O=\left\{a\right\}$ is an alphabet (the objective a is known as the spike);

(2)

${\sigma }_{11},{\sigma }_{12},\cdots ,{\sigma }_{mn}$ are an array of $m\times n$ coupled neurons of the form

$${\sigma }_{ij}=\left(x_{ij},y_{ij},z_{ij},R_{ij}\right), 1\le i\le h, 1\le j\le w$$

where

(a)

$x_{ij}\in R$ is the value of spikes in feeding input unit in neuron ${\sigma }_{ij}$;

(b)

$y_{ij}\in R$ is the value of spikes in linking input unit in neuron ${\sigma }_{ij}$;

(c)

$z_{ij}\in R$ is the value of spikes in dynamic threshold unit in neuron ${\sigma }_{ij}$;

(d)

$R_{ij}$ denotes the finite set of spiking rules, of the form $E/\left(a^x,a^y,a^z\right)\to a^p$, where E is the firing condition, $p\ge 0$, $\tau \ge 0$, and $p\le u\left(1+v\right)$.

(3)

$\mathrm{syn}=\left\{\left(ij,kl\right)|1\le i\le h,1\le j\le w,\left|k-i\right|\le r,\left|l-j\right|\le r,i\ne k,j\ne l\right\}$, where r is the neighborhood radius.

Suppose that I is an input image with size $h\times w$ and $I_{BC}$ is the matrix containing the base components (BC) obtained from the image decomposition method. Figure 2 shows the relationship between CNP system $\Pi$ and base components matrix $I_{BC}$.

In CNP system $\Pi$, each coupled neuron only communicates with its neighboring neurons, i.e., local topology. An r-neighborhood of neurons ${\sigma }_{ij}$ is defined as follows, as shown in Figure 2:

$${\delta }_r\left({\sigma }_{ij}\right)=\left\{{\sigma }_{kl}\left|\left|k-i\right|\le r,\left|l-j\le r\right|\right.\right\}$$

(2)

The spiking rule of neuron ${\sigma }_{ij}$ is defined as follows [14]:

$$E/\left(a^x,a^y,a^z\right)\to a^p$$

(3)

where E is a firing condition denoted as follows [14]:

$$E\equiv \left(n_i\left(t\right)\ge z_i\left(t\right)\right)\wedge \left(x_i\left(t\right)\ge x\right)\wedge \left(y_i\left(t\right)\ge y\right)\wedge \left(z_i\left(t\right)\ge z\right)$$

(4)

where $n_{ij}\left(t\right)=x_{ij}\left(t\right)\left(1+y_{ij}\left(t\right)\right)$, which represents a nonlinear modulation mechanism. According to the spiking mechanism, the state equation for neuron ${\sigma }_{ij}$ can be given by

$$x_{ij}\left(t+1\right)=\left\{\begin{array}{l}{x}_{ij}\left(t\right)-x+C_{ij}+{\displaystyle \sum _{{\sigma }_{kl}\in {\delta }_r}{\omega }_{kl}{p}_{kl}\left(t\right)}, \mathrm{if} {\sigma }_{ij} \mathrm{fires}\\ x_{ij}\left(t\right)+C_{ij}+{\displaystyle \sum _{{\sigma }_{kl}\in {\delta }_r}{\omega }_{kl}{p}_{kl}\left(t\right)}, \mathrm{otherwise}\end{array}\right.$$

(5)

$$y_{ij}\left(t+1\right)=\left\{\begin{array}{l}{y}_{ij}\left(t\right)-y+{\displaystyle \sum _{{\sigma }_{kl}\in {\delta }_r}{\omega }_{kl}{p}_{kl}}\left(t\right), \mathrm{if} {\sigma }_{ij} \mathrm{fires}\\ y_{ij}\left(t\right)+{\displaystyle \sum _{{\sigma }_{kl}\in {\delta }_r}{\omega }_{kl}{p}_{kl}\left(t\right), \mathrm{otherwise}}\end{array}\right.$$

(6)

$$z_{ij}\left(t+1\right)=\left\{\begin{array}{l}{z}_{ij}\left(t\right)-z+p, \mathrm{if} {\sigma }_{ij} \mathrm{fires}\\ z_{ij}\left(t\right), \mathrm{otherwise}\end{array}\right.$$

(7)

where $p_{kl}\left(t\right)$ is the value of the spikes received by neuron ${\sigma }_{ij}$ from neighboring neuron ${\sigma }_{kl}$ and ${\omega }_{kl}\left(t\right)$ is the corresponding local weight, and $C_{ij}$ is an external stimulus. $p$ is the value of spikes generated by neuron ${\sigma }_{ij}$ when it fires. Peng et al. [51] introduced the MFIF method based on CNP systems in NSCT domain, and the algorithm has achieved good fusion results.

5. The Proposed Method

We present a novel approach for MFIF utilizing fractal dimension and coupled neural P systems in NSCT domain. The main steps can be concluded as follows: NSCT decomposition, low-frequency coefficient fusion, high-frequency coefficient fusion, and inverse NSCT transform. Figure 3 illustrates the architecture of the proposed method.

5.1. NSCT Decomposition

Assume that A and B are the input images, the NSCT is used to decompose the two images, and the low- and high-frequency components are generated, named as $\left\{L_A,H_A^{l,k}\right\}$ and $\left\{L_B,H_B^{l,k}\right\}$, respectively.

5.2. Low-Frequency Coefficient Fusion

The low-frequency components have the most brightness and energy information; the fusion strategy for low-frequency components has a significant impact on the final fusion quality. In this section, the CNP systems-based low-frequency fusion rule is designed. Suppose that ${\prod }_A$ and ${\prod }_B$ are two CNP systems with local topology; the low-frequency coefficients of two multi-focus images are regarded as the external inputs of ${\prod }_A$ and ${\prod }_B$. Starting from the initial state, the two CNP systems work constantly until iteration number $t_{max}$ is reached. Then, they halt. Denoted by $T_A$ and $T_B$, the excitation number matrixes are associated with ${\prod }_A$ and ${\prod }_B$, i.e., $T_A={\left(t_{ij}^A\right)}_{h\times w}$ and $T_B={\left(t_{ij}^B\right)}_{h\times w}$, where $t_{ij}^A$ (or $t_{ij}^B$) is the number of times that ${\sigma }_{ij}$ fires in ${\prod }_A$ (or ${\prod }_B$). The fusion rules for low-frequency NSCT coefficients are formulated based on the information provided by the two excitation number matrices as follows [14]:

$$L_F\left(i,j\right)=\left\{\begin{array}{c}{L}_A\left(i,j\right)\mathrm{if} t_{ij}^A\ge t_{ij}^B\\ L_B\left(i,j\right)\mathrm{i}\mathrm{f} t_{ij}^A<t_{ij}^B\end{array}\right.$$

(8)

where $L_A\left(i,j\right)$ and $L_B\left(i,j\right)$ are the low-frequency coefficients of two source images at position $\left(i,j\right)$, respectively, and $L_F\left(i,j\right)$ is the fused low-frequency coefficients.

5.3. High-Frequency Coefficient Fusion

The high-frequency coefficients contain richer texture and detailed information, as well as some of the image’s noise. Fractal and fractional have extensive applications in image processing [52,53,54,55,56,57,58,59,60,61,62,63]. In this section, the spatial frequency (SF) [64] and fractal dimension-based focus measure (FDFM) [52] are defined as follows:

$$S{F}^{H_A^{l,k}}\left(i,j\right)={{\displaystyle \sum _{i\in M,j\in N}\left(H_A^{l,k}\left(i,j\right)-H_A^{l,k}\left(i-1,j\right)\right)}}^2+{\left(H_A^{l,k}\left(i,j\right)-H_A^{l,k}\left(i,j-1\right)\right)}^2$$

(9)

$$S{F}^{H_B^{l,k}}\left(i,j\right)={{\displaystyle \sum _{i\in M,j\in N}\left(H_B^{l,k}\left(i,j\right)-H_B^{l,k}\left(i-1,j\right)\right)}}^2+{\left(H_B^{l,k}\left(i,j\right)-H_B^{l,k}\left(i,j-1\right)\right)}^2$$

(10)

$$FDF{M}^{H_A^{l,k}}\left(i,j\right)=g_{\max }^{H_A^{l,k}}\left(i,j\right)-g_{\min }^{H_A^{l,k}}\left(i,j\right)$$

(11)

$$FDF{M}^{H_B^{l,k}}\left(i,j\right)=g_{\max }^{H_B^{l,k}}\left(i,j\right)-g_{\min }^{H_B^{l,k}}\left(i,j\right)$$

(12)

where $g_{max}^{X\left(i,j\right)}$ and $g_{\min }^X\left(i,j\right)$ are the maximum and minimum intensities, respectively, over a $3\times 3$ window centered at the ${\left(i,j\right)}^{th}$ pixel of $X\in \left\{H_A^{l,k},H_B^{l,k}\right\}$. FDFM can measure such small activity levels, which also affect the fusion results.

The fused high-frequency coefficients are generated by

$$H_F^{l,k}\left(i,j\right)=\left\{\begin{array}{ll}{H}_A^{l,k}\left(i,j\right)& \mathrm{if} S{F}^{H_A^{l,k}}\left(i,j\right)\times FDF{M}^{H_A^{l,k}}\left(i,j\right)\\ & \ge S{F}^{H_B^{l,k}}\left(i,j\right)\times FDF{M}^{H_B^{l,k}}\left(i,j\right)\\ H_B^{l,k}\left(i,j\right)& \mathrm{e}\mathrm{l}\mathrm{s}\mathrm{e}\end{array}\right.$$

(13)

where $H_F^{l,k}\left(i,j\right)$ shows the fused high-frequency coefficients.

5.4. Inverse NSCT Transform

The final fused image F can be generated by using inverse NSCT transform performed on the fused low- and high-frequency coefficients $\left\{L_F,H_F^{l,k}\right\}$.

The main steps of the proposed method can be summarized as in Algorithm 1.

Algorithm 1 Proposed MFIF method

Input: the source images: A and B Parameters: The number of NSCT decomposition levels: $L$, the number of directions at each decomposition level: $K\left(l\right)$, $l\in \left[1,L\right]$ Main step: Step 1: NSCT decomposition For each source image $X\in \left\{A,B\right\}$    Perform NSCT decomposition on $X$ to generate $\left\{L_X,H_X^{l,k}\right\}$, $l\in \left[1,L\right]$, $k\in \left[1,K\left(l\right)\right]$; End Step 2: Low-frequency components fusion For each source image $X\in \left\{A,B\right\}$    Calculate the CNP for $L_X$ using Equations (1)–(7); End Merge $L_A$ and $L_B$ using Equation (8) to generate $L_F$; Step 3: High-frequency components fusion For each level $l=1:L$    For each direction $k=1:K\left(l\right)$    For each source image $X\in \left\{A,B\right\}$    Calculate the SF for $H_X^{l,k}\left(i,j\right)$ using Equations (9) and (10);    Calculate the FDFM for $H_X^{l,k}\left(i,j\right)$ using Equations (11) and (12);      End      Merge $H_A^{l,k}$ and $H_B^{l,k}$ using Equation (13);    End End Step 4: Inverse NSCT Perform inverse NSCT on $\left\{L_F,H_F^{l,k}\right\}$ to generate $F$; Output: the fused image $F$.

6. Experimental Results and Discussion

6.1. Experimental Setup

In this section, the classical Lytro [65], MFI-WHU [66], and MFFW [67] datasets are used in the experiments, as shown in Figure 4. Eight image fusion methods, named GD [28], PMGI [46], MFFGAN [66], LEGFF [68], U2Fusion [47], CBFM [69], FUFusion [70], and EgeFusion [23], are used to compare. The metrics $Q_{AB/F}$ [64,71], $Q_{CB}$ [72], $Q_{CV}$ [72], $Q_E$ [72], $Q_{FMI}$ [73], $Q_G$ [72], $Q_{MI}$ [64], and $Q_{NCIE}$ [72] are used to evaluate the fusion results. The source code of these compared methods is available or shared by the authors, and the relevant parameter settings are set according to the original papers. In our method, the NSCT decomposition levels is 4, and the corresponding directions are 2, 4, 8, and 8. The parameters in CNP systems are set to $i_{max}$, $z_0=0.3$, $r=7$, and $p=1$. $W_{7\times 7}={\left\{{\omega }_{ij}\right\}}_{7\times 7}$ is determined as follows: (i) ${\omega }_{44}=0$; (ii) ${\omega }_{ij}=1/\mathrm{sqrt}\left({\left(i-4\right)}^2+{\left(j-4\right)}^2\right)$ for $i\ne 4$, $j\ne 4$, $1\le i\le 7$, $1\le j\le 7$.

6.2. Fusion Results and Discussion

(1)

Results on the Lytro Dataset

We conduct qualitative and quantitative evaluations to evaluate the performance of different image fusion methods. Figure 5 displays the fused images of Data 1 from the Lytro dataset. The fused image generated by GD exhibits over-brightness in certain areas, leading to the loss of some fine details and the presence of artifacts. The PMGI algorithm results in severe distortion, with the visual effect of the image appearing dim and blurry, and a significant loss of information. The brightness of the image generated by MFFGAN is insufficient, especially in areas such as the hair. The LEGFF algorithm achieved a relatively good fusion result, but there is slight darkness observed in the brightness of the hair section. The U2Fusion algorithm generated an image with some dim areas, particularly noticeable in regions like the hat and hair, as well as the watch section, where usable information is not observable. The CBFM generates a fused image that darkens the area around the arm. The FUFusion produces a blurry fused image, making it difficult to obtain some image information. The EgeFusion results in a significant distortion and loss of image information in the fused image. Compared to other algorithms, our method demonstrates a superior fusion result, featuring higher clarity and moderate brightness. Information pertaining to arm, watch, clothing, hat, golf club, and grass in the image is distinctly observable.

Table 1. Quantitative comparative analysis of different methods for Data 1 in the Lytro dataset.

	Year	${\mathit{Q}}_{\mathit{A}\mathit{B}/\mathit{F}}$	${\mathit{Q}}_{\mathit{C}\mathit{B}}$	${\mathit{Q}}_{\mathit{C}\mathit{V}}$	${\mathit{Q}}_{\mathit{E}}$	${\mathit{Q}}_{\mathit{F}\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{G}}$	${\mathit{Q}}_{\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{N}\mathit{C}\mathit{I}\mathit{E}}$
GD	2016	0.7220	0.6684	63.5814	0.8144	0.9222	0.6985	3.1161	0.8096
PMGI	2020	0.5466	0.6070	70.2785	0.6316	0.9169	0.5156	5.1347	0.8169
MFFGAN	2021	0.6860	0.7026	23.3439	0.8451	0.9296	0.6599	5.5783	0.8190
LEGFF	2022	0.6923	0.6857	38.6156	0.8205	0.9306	0.6658	4.8919	0.8158
U2Fusion	2022	0.6575	0.6164	56.5810	0.7952	0.9206	0.6338	5.2894	0.8176
CBFM	2023	0.7201	0.7403	13.3863	0.8720	0.9334	0.6974	5.4462	0.8184
FUFusion	2024	0.7202	0.6652	47.2751	0.8146	0.9259	0.6967	5.6856	0.8197
EgeFusion	2024	0.3120	0.3356	468.3896	0.4318	0.8892	0.3080	2.6294	0.8084
Proposed		0.7524	0.7745	6.5508	0.8862	0.9380	0.7382	6.5466	0.8245

presents the numerical values of various indicators corresponding to different algorithms depicted in Figure 5. These indicators serve as quantitative metrics for evaluating the performance of each algorithm in the context of the depicted data. By examining these values, we gain insights into the effectiveness of each algorithm in achieving the desired objectives. From Table 1, it is evident that our algorithm has achieved optimal values for the eight indicators, with respective values of $Q_{AB/F}$ (0.7524), $Q_{CB}$ (0.7745), $Q_{CV}$ (6.5508), $Q_E$ (0.8862), $Q_{FMI}$ (0.9380), $Q_G$ (0.7382), $Q_{MI}$ (6.5466), and $Q_{NCIE}$ (0.8245).

Figure 6 displays the fused images of Data 2 from the Lytro dataset. The image clarity obtained by the GD and PMGI algorithms is not high. The MFFGAN, LEGFF, and U2Fusion methods produce unevenly fused images, in which some regions are clear while others are dark, such as the neck area being too dim, obscuring detailed information. The fusion image generated by CBFM is dark and exhibits low brightness and clarity, resulting in a significant loss of information. The fused image produced by the FUFusion method is overall blurry, with a severe loss of detailed information. The EgeFusion algorithm produces an image with significant distortion, although some sharpening is applied to enhance detailed information; this sharpening also results in varying degrees of block effects in the image. Compared to other algorithms, our algorithm achieved the best fusion result, including image brightness, contrast, etc., allowing for a genuine observation of people and objects in the image.

Table 2. Quantitative comparative analysis of different methods for Data 2 in the Lytro dataset.

	Year	${\mathit{Q}}_{\mathit{A}\mathit{B}/\mathit{F}}$	${\mathit{Q}}_{\mathit{C}\mathit{B}}$	${\mathit{Q}}_{\mathit{C}\mathit{V}}$	${\mathit{Q}}_{\mathit{E}}$	${\mathit{Q}}_{\mathit{F}\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{G}}$	${\mathit{Q}}_{\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{N}\mathit{C}\mathit{I}\mathit{E}}$
GD	2016	0.6823	0.6135	85.4217	0.7559	0.8645	0.6660	4.2116	0.8156
PMGI	2020	0.4798	0.5977	53.3298	0.5816	0.8573	0.4592	6.3071	0.8251
MFFGAN	2021	0.6609	0.6291	28.2393	0.7931	0.8651	0.6440	6.4491	0.8260
LEGFF	2022	0.6770	0.6466	22.7596	0.7920	0.8680	0.6603	5.8173	0.8225
U2Fusion	2022	0.5951	0.4969	168.9820	0.6838	0.8619	0.5786	6.1325	0.8242
CBFM	2023	0.7116	0.6634	31.8620	0.8348	0.8690	0.7019	5.9728	0.8234
FUFusion	2024	0.7226	0.6400	16.7184	0.8511	0.8671	0.7088	7.0218	0.8294
EgeFusion	2024	0.2492	0.3688	490.2211	0.3210	0.8429	0.2419	3.2940	0.8127
Proposed		0.7445	0.6870	5.3153	0.8672	0.8732	0.7369	7.6750	0.8339

presents the numerical values of various indicators corresponding to different algorithms depicted in Figure 6. By analyzing the data in Table 2, we can conclude that our algorithm has achieved the optimal values in all eight indicators, with values as follows: $Q_{AB/F}$ (0.7445), $Q_{CB}$ (0.6870), $Q_{CV}$ (5.3153), $Q_E$ (0.8672), $Q_{FMI}$ (0.8732), $Q_G$ (0.7369), $Q_{MI}$ (7.6750), and $Q_{NCIE}$ (0.8339).

Figure 7 depicts a line chart of the metrics for different data in the Lytro dataset (20 sets of data were used in this experiment), allowing for an observation of fluctuations in the indicator data. Additionally, we have computed the average indicators, as shown in both Figure 7 and

Table 3. Quantitative average comparative analysis of different methods on the Lytro dataset.

	Year	${\mathit{Q}}_{\mathit{A}\mathit{B}/\mathit{F}}$	${\mathit{Q}}_{\mathit{C}\mathit{B}}$	${\mathit{Q}}_{\mathit{C}\mathit{V}}$	${\mathit{Q}}_{\mathit{E}}$	${\mathit{Q}}_{\mathit{F}\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{G}}$	${\mathit{Q}}_{\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{N}\mathit{C}\mathit{I}\mathit{E}}$
GD	2016	0.7034	0.6115	123.5691	0.7874	0.8887	0.6987	3.8521	0.8139
PMGI	2020	0.3901	0.5656	98.3494	0.4736	0.8815	0.3857	5.8641	0.8225
MFFGAN	2021	0.6642	0.6457	42.5655	0.8409	0.8915	0.6592	6.0604	0.8237
LEGFF	2022	0.6810	0.6751	53.0073	0.8195	0.8937	0.6754	5.6138	0.8214
U2Fusion	2022	0.6143	0.5682	97.5910	0.7835	0.8844	0.6093	5.7765	0.8221
CBFM	2023	0.7137	0.7018	41.5959	0.8626	0.8956	0.7101	5.7967	0.8223
FUFusion	2024	0.6906	0.6346	128.5882	0.7972	0.8877	0.6862	6.4774	0.8263
EgeFusion	2024	0.3576	0.4034	340.4188	0.5032	0.8472	0.3541	3.2191	0.8120
Proposed		0.7390	0.7388	31.4085	0.8772	0.8989	0.7362	6.9683	0.8296

. Considering that a lower value of $Q_{CV}$ indicates better performance, we take its negative value (i.e., $-Q_{CV}$) to illustrate the sub-figure of $Q_{CV}$. The horizontal axis represents the number of image groups in the dataset, and the vertical axis represents the metric value. From the Figure 7 and Table 3, we can observe that our method has achieved the optimal average values for all the indicators, with values as follows: $Q_{AB/F}$ (0.7390), $Q_{CB}$ (0.7388), $Q_{CV}$ (31.4085), $Q_E$ (0.8772), $Q_{FMI}$ (0.8989), $Q_G$ (0.7362), $Q_{MI}$ (6.9683), and $Q_{NCIE}$ (0.8296).

(2)

Results on the MFI-WHU Dataset

Figure 8 depicts the fused images of Data 1 from the MFI-WHU dataset. The images produced by GD exhibit some slight pseudo-shadow artifacts. The PMGI algorithm generates blurry fused images; for instance, the details of the roof and mural cannot be accurately captured. The MFFGAN and U2Fusion algorithms achieve higher brightness fusion results in certain regions, such as the glass display area. However, there are also areas with lower brightness. For instance, MFFGAN exhibits shadows around the base of the display cabinet, while U2Fusion produces significant shadowing along the edges of the roof and the bench, hindering the retrieval of complete information. The fused image produced by LEGFF has a higher clarity. The fused images produced by the CBFM and FUFusion algorithms have some areas with shadows, such as on the side of a bench. Although the EgeFusion algorithm enhances the texture information in the image, it also introduces some degree of distortion. Upon comprehensive comparison, our algorithm outperforms others in the fusion experiment, enhancing the overall fusion effect by reducing pseudo-shadows and dark areas, while simultaneously improving the brightness and clarity of the fused image.

Table 4. Quantitative comparative analysis of different methods for Data 1 in the MFI-WHU dataset.

	Year	${\mathit{Q}}_{\mathit{A}\mathit{B}/\mathit{F}}$	${\mathit{Q}}_{\mathit{C}\mathit{B}}$	${\mathit{Q}}_{\mathit{C}\mathit{V}}$	${\mathit{Q}}_{\mathit{E}}$	${\mathit{Q}}_{\mathit{F}\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{G}}$	${\mathit{Q}}_{\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{N}\mathit{C}\mathit{I}\mathit{E}}$
GD	2016	0.6739	0.6571	82.3636	0.7953	0.8906	0.6829	4.1802	0.8180
PMGI	2020	0.3769	0.6528	25.6514	0.4425	0.8883	0.3772	7.0843	0.8320
MFFGAN	2021	0.6371	0.6336	37.0591	0.7936	0.8950	0.6337	6.9928	0.8315
LEGFF	2022	0.6253	0.6348	37.4732	0.7451	0.8959	0.6345	6.2296	0.8271
U2Fusion	2022	0.5571	0.4815	140.7225	0.7173	0.8867	0.5574	6.2829	0.8273
CBFM	2023	0.6846	0.7014	28.2899	0.8218	0.8963	0.6835	6.5658	0.8290
FUFusion	2024	0.7024	0.7546	24.4700	0.8452	0.8970	0.7039	8.0555	0.8390
EgeFusion	2024	0.2691	0.3308	453.2295	0.3956	0.8557	0.2720	3.1227	0.8148
Proposed		0.7281	0.8256	6.4076	0.8534	0.9000	0.7288	9.0964	0.8476

presents the numerical values of various indicators corresponding to different algorithms depicted in Figure 8. From Table 4, it is evident that our algorithm has achieved optimal values for the eight indicators, with respective values of $Q_{AB/F}$ (0.7281), $Q_{CB}$ (0.8256), $Q_{CV}$ (6.4076), $Q_E$ (0.8534), $Q_{FMI}$ (0.9000), $Q_G$ (0.7288), $Q_{MI}$ (9.0964), and $Q_{NCIE}$ (0.8476).

Figure 9 depicts the fused images of Data 2 from the MFI-WHU dataset. The GD method produced a fused image with low clarity. The PMGI method generated a low-quality fused image with varying degrees of shadowing in objects such as the vehicle, windowsill, sky, and trees. The images produced by the MFFGAN and LEGFF algorithms cause the windowsill area to become darker. The U2Fusion method generated fused image with uneven brightness distribution, such as a darker area in the windowsill region and a brighter area in the vehicle region. The fused images produced by the CBFM and FUFusion algorithms have relatively low brightness. The EgeFusion algorithm causes distortion in the fused image. Through comprehensive comparison, our algorithm produces the optimal fusion result, with balanced brightness and high clarity, which are beneficial for information retrieval.

Table 5. Quantitative comparative analysis of different methods for Data 2 in the MFI-WHU dataset.

	Year	${\mathit{Q}}_{\mathit{A}\mathit{B}/\mathit{F}}$	${\mathit{Q}}_{\mathit{C}\mathit{B}}$	${\mathit{Q}}_{\mathit{C}\mathit{V}}$	${\mathit{Q}}_{\mathit{E}}$	${\mathit{Q}}_{\mathit{F}\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{G}}$	${\mathit{Q}}_{\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{N}\mathit{C}\mathit{I}\mathit{E}}$
GD	2016	0.6876	0.5795	137.2797	0.7862	0.8655	0.6842	3.9284	0.8140
PMGI	2020	0.5906	0.4640	97.4159	0.6549	0.8559	0.5880	5.1975	0.8190
MFFGAN	2021	0.6536	0.5945	64.3859	0.7661	0.8695	0.6497	5.3730	0.8198
LEGFF	2022	0.6546	0.5709	58.2280	0.7660	0.8744	0.6462	4.8070	0.8173
U2Fusion	2022	0.6053	0.5280	86.2756	0.7221	0.8578	0.6035	5.1287	0.8187
CBFM	2023	0.7031	0.6458	59.4391	0.8212	0.8757	0.7015	5.3351	0.8196
FUFusion	2024	0.7070	0.7063	30.6077	0.8411	0.8784	0.7061	5.9021	0.8224
EgeFusion	2024	0.3031	0.2703	653.9751	0.3296	0.8315	0.2953	2.8712	0.8110
Proposed		0.7408	0.8250	38.0683	0.8550	0.8795	0.7409	7.8104	0.8348

presents the numerical values of various indicators corresponding to different algorithms depicted in Figure 9. From Table 5, it is evident that our algorithm has achieved optimal values for seven indicators, with respective values of $Q_{AB/F}$ (0.7408), $Q_{CB}$ (0.8250), $Q_E$ (0.8550), $Q_{FMI}$ (0.8795), $Q_G$ (0.7409), $Q_{MI}$ (7.8104), and $Q_{NCIE}$ (0.8348). The FUFusion algorithm achieved the optimal value on the $Q_{CV}$ metric, with a value of 30.6077. Our algorithm ranked second in the $Q_{CV}$ metric, with a value of 38.0683.

Figure 10 depicts a line chart of the metrics of different data in the MFI-WHU dataset (30 sets of data were used in this experiment), allowing for the observation of fluctuations in the indicator data. Additionally, we have computed the average indicators, as shown in both Figure 10 and

Table 6. Quantitative average comparative analysis of different methods on the MFI-WHU dataset.

	Year	${\mathit{Q}}_{\mathit{A}\mathit{B}/\mathit{F}}$	${\mathit{Q}}_{\mathit{C}\mathit{B}}$	${\mathit{Q}}_{\mathit{C}\mathit{V}}$	${\mathit{Q}}_{\mathit{E}}$	${\mathit{Q}}_{\mathit{F}\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{G}}$	${\mathit{Q}}_{\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{N}\mathit{C}\mathit{I}\mathit{E}}$
GD	2016	0.6752	0.6301	105.0418	0.7754	0.8648	0.6686	3.6940	0.8136
PMGI	2020	0.4237	0.5933	62.8111	0.5061	0.8558	0.4177	5.4884	0.8210
MFFGAN	2021	0.6427	0.6329	45.6960	0.7826	0.8684	0.6367	5.6832	0.8222
LEGFF	2022	0.6190	0.6060	71.1462	0.7067	0.8692	0.6106	4.8291	0.8183
U2Fusion	2022	0.5502	0.5156	119.8639	0.6970	0.8565	0.5447	5.1498	0.8194
CBFM	2023	0.6974	0.7064	59.4521	0.8174	0.8707	0.6922	5.6757	0.8224
FUFusion	2024	0.7038	0.7443	23.7140	0.8337	0.8753	0.6992	6.6901	0.8284
EgeFusion	2024	0.2874	0.3277	537.7216	0.3757	0.8255	0.2841	2.8055	0.8111
Proposed		0.7296	0.8072	36.4954	0.8453	0.8772	0.7254	7.8107	0.8371

. We can observe that our algorithm has achieved optimal values for seven indicators, with respective values of $Q_{AB/F}$ (0.7296), $Q_{CB}$ (0.8072), $Q_E$ (0.8453), $Q_{FMI}$ (0.8772), $Q_G$ (0.7254), $Q_{MI}$ (7.8107), and $Q_{NCIE}$ (0.8371). The FUFusion algorithm achieved the optimal value on the $Q_{CV}$ metric, with a value of 23.7140. Our algorithm ranked second in the $Q_{CV}$ metric, with a value of 36.4954.

(3)

Results on the MFFW Dataset

Figure 11 depicts the fused images of Data 1 from the MFFW dataset. The fused images produced by the GD, PMGI, and FUFusion algorithms are blurry and have lower clarity. The fused images produced by the MFFGAN and LEGFF methods cause the center of the rose to appear darker, with shadowing present. The U2Fusion causes the piano keys and the rose in the fused image to appear darker. The fused images produced by the CBFM and FUFusion algorithms exhibit varying degrees of artifacts. Although the EgeFusion algorithm enhances the texture information in the fused image, it also introduces a certain degree of distortion. By comparison with other algorithms, our method achieves a superior fusion result, attaining effective information complementarity.

Table 7. Quantitative comparative analysis of different methods for Data 1 in the MFFW dataset.

	Year	${\mathit{Q}}_{\mathit{A}\mathit{B}/\mathit{F}}$	${\mathit{Q}}_{\mathit{C}\mathit{B}}$	${\mathit{Q}}_{\mathit{C}\mathit{V}}$	${\mathit{Q}}_{\mathit{E}}$	${\mathit{Q}}_{\mathit{F}\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{G}}$	${\mathit{Q}}_{\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{N}\mathit{C}\mathit{I}\mathit{E}}$
GD	2016	0.7297	0.4875	110.5392	0.6970	0.8882	0.6829	4.7958	0.8195
PMGI	2020	0.4084	0.4709	61.0451	0.3863	0.8842	0.3766	6.5896	0.8282
MFFGAN	2021	0.6626	0.5677	57.0983	0.7777	0.8923	0.6165	6.3993	0.8272
LEGFF	2022	0.7001	0.5947	60.1930	0.7886	0.8968	0.6457	6.1499	0.8258
U2Fusion	2022	0.5914	0.5325	101.9227	0.6879	0.8790	0.5539	6.2297	0.8262
CBFM	2023	0.7037	0.6146	51.8537	0.8237	0.8933	0.6588	6.0571	0.8254
FUFusion	2024	0.6992	0.5522	76.2735	0.7652	0.8897	0.6479	6.6305	0.8285
EgeFusion	2024	0.3877	0.4132	365.6116	0.4142	0.8406	0.3611	4.2145	0.8173
Proposed		0.7013	0.6155	33.3908	0.8290	0.8987	0.6571	6.6801	0.8290

presents the numerical values of various indicators corresponding to different algorithms depicted in Figure 11. From Table 7, it is evident that our algorithm has achieved optimal values for six indicators, with respective values of $Q_{CB}$ (0.6155), $Q_{CV}$ (33.3908), $Q_E$ (0.8290), $Q_{FMI}$ (0.8987), $Q_{MI}$ (6.6801), and $Q_{NCIE}$ (0.8290). The GD algorithm achieved optimal values for the $Q_{AB/F}$ and $Q_G$ metrics, with values of 0.7297 and 0.6829, respectively.

Figure 12 depicts a line chart of the metrics of different data in the MFFW dataset (13 sets of data were used in this experiment), allowing for the observation of fluctuations in the indicator data. Additionally, we have computed the average indicators, as shown in both Figure 12 and

Table 8. Quantitative average comparative analysis of different methods on the MFFW dataset.

	Year	${\mathit{Q}}_{\mathit{A}\mathit{B}/\mathit{F}}$	${\mathit{Q}}_{\mathit{C}\mathit{B}}$	${\mathit{Q}}_{\mathit{C}\mathit{V}}$	${\mathit{Q}}_{\mathit{E}}$	${\mathit{Q}}_{\mathit{F}\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{G}}$	${\mathit{Q}}_{\mathit{M}\mathit{I}}$	${\mathit{Q}}_{\mathit{N}\mathit{C}\mathit{I}\mathit{E}}$
GD	2016	0.6279	0.5557	217.9965	0.7011	0.8730	0.6114	3.6107	0.8122
PMGI	2020	0.3807	0.5057	275.9272	0.4245	0.8675	0.3675	5.0472	0.8178
MFFGAN	2021	0.5905	0.5851	138.1153	0.7557	0.8742	0.5744	5.0498	0.8179
LEGFF	2022	0.6294	0.6032	172.4173	0.7386	0.8775	0.6165	4.8088	0.8169
U2Fusion	2022	0.5537	0.5499	228.0064	0.7076	0.8690	0.5388	4.8894	0.8171
CBFM	2023	0.6352	0.6408	126.1058	0.7912	0.8748	0.6203	4.8905	0.8171
FUFusion	2024	0.5987	0.5779	226.8354	0.6974	0.8718	0.5817	5.3456	0.8196
EgeFusion	2024	0.3517	0.4213	443.4456	0.4581	0.8380	0.3439	3.3785	0.8115
Proposed		0.6377	0.6362	116.0756	0.8048	0.8799	0.6224	5.0505	0.8180

. We can observe that our algorithm has achieved optimal values for five indicators, with respective values of $Q_{AB/F}$ (0.6377), $Q_{CV}$ (116.0756) $Q_E$ (0.8048), $Q_{FMI}$ (0.8799), and $Q_G$ (0.6224). The CBFM algorithm achieved the optimal value for the $Q_{CB}$ metric, with a value of 0.6408. Our algorithm ranked second in the $Q_{CB}$ metric, with a value of 0.6362. The FUFusion algorithm achieved the optimal value on the $Q_{MI}$ and $Q_{NCIE}$ metrics, with values of 5.3456 and 0.8196, respectively. Our method ranked second in these two metrics, with values of 5.0505 and 0.8180, respectively.

6.3. Application Extension

In this section, we extend the proposed algorithm to the application of multi-modal medical image fusion. We utilized two medical datasets from the Whole Brain Atlas [74]. When processing color medical images (such as magnetic resonance (MR) and positron emission tomography (PET)), the PET images need to be converted between the RGB and YUV color spaces. The Y channel of the PET image is fused with the MR image, and then the fused Y channel is converted back to the RGB space along with the U and V channels of the PET image to obtain the final color fusion image. The corresponding schematic diagram is shown in Figure 13. From the experimental results shown in Figure 14, it is evident that our algorithm performs exceptionally well for medical image fusion, achieving significantly enhanced information complementarity in the fused images.

7. Conclusions

This paper presents a novel approach to multi-focus image fusion that integrates fractal dimension and CNP systems in NSCT domain. Our fusion technique utilizes CNP systems with a local topology-based fusion model to effectively merge low-frequency components. Additionally, for high-frequency components, we employ a focus measure based on spatial frequency and FDFM to achieve superior fusion performance. Extensive experiments conducted on three classic datasets (Lytro, MFI-WHU, and MFFW), validate the effectiveness of the proposed method. In security surveillance, fusing images with different focal points can enhance the detail and clarity of the monitoring footage, improving target recognition capabilities. Furthermore, we have extended and applied this method to the field of medical image fusion, achieving the complementary integration of multi-modal medical image information. The number of decomposition levels in NSCT affects the effectiveness of image fusion. However, too many levels increase algorithm complexity. Therefore, choosing an optimal number of decomposition levels is our research focus. In future work, we will attempt to improve this algorithm and expand its application to the fusion of SAR (synthetic aperture radar) and optical images [48]. Additionally, the application of image fusion in multimodal finger knuckle print identification and image change detection are also worthwhile directions to explore [75,76,77,78,79].