A Novel Multi-Focus Image Fusion Method Based on Stochastic Coordinate Coding and Local Density Peaks Clustering

Abstract

The multi-focus image fusion method is used in image processing to generate all-focus images that have large depth of field (DOF) based on original multi-focus images. Different approaches have been used in the spatial and transform domain to fuse multi-focus images. As one of the most popular image processing methods, dictionary-learning-based spare representation achieves great performance in multi-focus image fusion. Most of the existing dictionary-learning-based multi-focus image fusion methods directly use the whole source images for dictionary learning. However, it incurs a high error rate and high computation cost in dictionary learning process by using the whole source images. This paper proposes a novel stochastic coordinate coding-based image fusion framework integrated with local density peaks. The proposed multi-focus image fusion method consists of three steps. First, source images are split into small image patches, then the split image patches are classified into a few groups by local density peaks clustering. Next, the grouped image patches are used for sub-dictionary learning by stochastic coordinate coding. The trained sub-dictionaries are combined into a dictionary for sparse representation. Finally, the simultaneous orthogonal matching pursuit (SOMP) algorithm is used to carry out sparse representation. After the three steps, the obtained sparse coefficients are fused following the max L1-norm rule. The fused coefficients are inversely transformed to an image by using the learned dictionary. The results and analyses of comparison experiments demonstrate that fused images of the proposed method have higher qualities than existing state-of-the-art methods.

1. Introduction

High-quality images are widely used in different areas of a highly developed society. Following the development of cloud computing, more and more images are processed in a cloud [1,2]. High-quality images can increase the accuracy and efficiency of image processing. Due to the limitation of field depth in most of optical lenses, only the objects a certain distance away from the camera can be captured in focus and sharply, and other objects are out of focus and blurred. It usually takes multiple images of the same scene to enhance the robustness of image processing. However, viewing and analyzing a series of images separately is neither convenient nor efficient [3]. The multi-focus image fusion method is an effective way to resolve this problem by combining complementary information from multiple images into a fused image, which is useful for human or machine perception [4,5].

During the past few years, many image fusion algorithms have been developed to integrate multi-focus images. In general, multi-focus fusion algorithms can be classified into two groups: spatial-domain fusion and transform-domain fusion [3,6,7,8,9,10,11]. The spatial-domain methods only need spatial information of images to carry out image fusion, without doing any type of transformation. The main principle of spatial-domain methods is to select those pixels or regions with higher clarity to construct the fused image, according to the image clarity measurement. Energy of Laplacian [8,12] and spatial frequency [3,6,11] are two typical focus measures used to decide the clarity of pixels or regions. The main limitations of spatial-domain fusion methods in generating desirable fused images are the misalignment of decision map within the boundary of focused objects, and the incorrect decision in locating sub-regions of focused or out-of-focused regions. To reduce these limitations, some spatial-domain techniques use the average weight of pixel values to fuse source images, instead of using binary decision [7]. Due to the weight construction method, these spatial-domain methods may lead to blurring edges, contrast decrease, and reduction of sharpness [6].

In contrast to spatial-domain fusion methods, transform-domain fusion methods first convert source images into a transform domain to obtain the corresponding transform coefficients. Then, the transformed coefficients are merged, according to the pre-defined fusion rule. Finally, the fused image is constructed by carrying out inverse transform of the fused coefficients. The most commonly used transform-domain fusion methods are based on multi-scale transform (MST). MST algorithms use the following methods, including discrete wavelet transform, gradient pyramid, dual tree complex wavelet transform, and so on. Recently, some novel transform-domain analysis methods have been proposed, such as curvelet transform [13], and nonsubsampled contourlet transform [9]. Although multi-scale transform coefficients can reasonably represent important features of an image, each transform has its own merits and limitations corresponding to the context of input images. Thus, it is difficult to select an optimal transform basis without apriori knowledge [14,15].

In recent years, sparse-representation based methods, as a subset of transform-domain fusion methods, have been applied to image fusion. Different from other MST methods, sparse-representation based methods usually use learned bases, which can adaptively change according to the input images without apriori knowledge. Due to the adaptively learning feature, sparse representation is an effective way to describe and reconstruct images and signals. It is widely applied to image denoising [16], image deblurring [17], image inpainting [18], super-resolution [19] and image fusion [20]. Yang and Li [21] first applied the sparse representation theory to image fusion field and also proposed a multi-focus image fusion method with an MST dictionary. Li and Zhang applied the morphologically filtering sparse feature to the matrix decomposition method to improve the accuracy of sparse representation in multi-focus image fusion [20]. Wang and Liu proposed an approximate K-SVD based sparse representation method for multi-focus fusion and exposure fusion to reduce the computation costs of sparse-representation based image fusion [22]. Nejati and Samavi proposed K-SVD dictionary-learning based sparse representation for the decision map construction of multi-focus fusion [6]. However, these aforementioned sparse-representation based methods do not take the high computation costs into account as K-SVD, and online dictionary learning. In recent years, many researchers have been devoted to speeding up dictionary learning for image fusion. Zhang and Fu [23] proposed a joint sparse-representation-based image fusion method. Their method had lower complexity than K-SVD. However, it still required a substantial amount of computations. Kim and Han [14] proposed a joint-clustering-based dictionary construction method for image fusion. The proposed method used K-means clustering to group the image patches before dictionary learning. The K-means clustering needed a number of cluster centers before clustering. However, in most cases, the number of cluster centers is difficult to estimate accurately before clustering.

This paper proposes a novel stochastic coordinate coding (SCC)-based image fusion framework integrated with local density peaks clustering. The proposed multi-focus image fusion framework consists of three steps. First, a local density peaks clustering method is applied in order to cluster image patches. The local density peaks based algorithm increases the accuracy of clustering, and does not need any presetting value for the input image data. Second, an SCC-based dictionary construction approach is proposed. The constructed dictionary not only obtains accurate descriptions of input images, but also dramatically decreases the costs of dictionary learning. Finally, the trained dictionary is used for the sparse representation of image patches, and max-L1 theory is implemented in the image fusion process. The key contributions of this paper can be elaborated as follows:

• An integrated sparse representation framework for multi-focus image fusion is proposed that combines the local density peaks based image-patch clustering and stochastic coordinate coding.
• An SCC-based dictionary construction method is proposed and applied to sparse representation process, which can obtain a more accurate dictionary and decrease the computation cost of dictionary learning.

The rest of this paper is structured as follows: Section 2 presents and specifies the proposed framework; Section 3 simulates the proposed solutions and analyzes experiment results; and Section 4 concludes this paper.

2. Framework

2.1. Introduction of Framework

The proposed framework for image fusion shown in Figure 1 has three main steps. All image patches are clustered into different groups in the first step. Then each image patch group is learned by a sub-dictionary using the SCC algorithm [24] and these sub-dictionaries are combined into an integrated dictionary. Finally, the learned dictionary is used for image fusion. The details of each algorithm and method will be explained in the following paragraphs.

2.2. Local Density Peaks Clustering

An image usually consists of different types of image patches. It is efficient to describe the underlying structure of each image patch by using specific sub-dictionary that describes different types of image patches. This paper uses the local density peaks clustering method to classify image patches into a specific group by the similarity of structure [25,26]. Compared with other existing clustering methods, the local density peaks clustering method has two advantages. First, the method is insensitive to the start point (or initialization). Second, it does not need to know the number of clusters before clustering. Moreover, the basis of local density peaks clustering can be easily expressed by Euclidean distance between two different patches.

In Figure 2a, the local density ${\rho }_i$ of each image patch i is calculated by using Equation (1).

$${\rho }_i=\sum _j\chi (d_{ij}-d_c),$$

(1)

when $x\le 0$, $\chi \left(x\right)=0$, otherwise $\chi \left(x\right)=1$. $d_{ij}$ is the Euclidean distance between image patch i and j. $d_c$ is a cutoff distance and usually set to the median value of $d_{ij}$. Basically, ${\rho }_i$ equals the number of patches that are closer than $d_c$ to patch i. The clustering algorithm is only sensitive to the relative magnitude of ${\rho }_i$ from a different image patch, and robust with respect to the choice of $d_c$.

A distance ${\delta }_i$ of each image patch is measured to find the cluster centers. The calculation equation is shown in Equation (2). ${\delta }_i$ is the minimum distance between the image patch i and any other patch j with higher density. For the point with highest density, ${\delta }_i=max\left(d_{ij}\right)$. A local density map can be constructed by using ${\rho }_i$ (x-axis) and the normalized ${\delta }_i$ (y-axis, $0\le {\delta }_i\le 1$), which is shown in Figure 2b. The cluster centers are recognized that are boxed by dotted squares in Figure 2b, when the value of ${\delta }_i$ is anomalously large. When the cluster centers are identified, the remaining image patches are clustered into the relatively nearest identified center.

$${\delta }_i=\underset{j:{\rho }_j>{\rho }_i}{min}\left(d_{ij}\right),$$

(2)

2.3. Dictionary Construction

In the clustering step, image patches with similar structure are classified into a few groups. To construct a more discriminative and compact dictionary, the SCC online dictionary learning algorithm [24] is used to learn a sub-dictionary for each cluster. Subsequently, the learned sub-dictionaries are combined to a new dictionary for the image sparse representation and restoration. The dictionary construction process is illustrated in Figure 1 as the dictionary learning step.

2.3.1. Sub-Dictionary Learning Approach

The SCC online dictionary learning algorithm [24] shown in Algorithm 1 extracts eigenvalues from each cluster and builds the corresponding sub-dictionary. The dictionary and sparse code are initialized and denoted as ${\mathbf{D}}_1^1$, ${\mathbf{z}}_i^0$ = 0, i = 1, 2, ... n respectively. The general expression of the sparse code is ${\mathbf{z}}_i^k$ = 0, i = 1, 2, ... n, k = 1, 2, ... n and learning rate ${\eta }_1^1$ = 1. The number of epochs and the index of data points are represented as superscript k and subscript i respectively. It acquires an image patch ${\mathbf{x}}_i$, when k = 1 and i = 1. The sparse code $z_i^k$ is updated by a few steps of coordinate descent (CD):

$$z_i^k=CD({\mathbf{D}}_i^k,{\mathbf{x}}_i,{\mathbf{z}}_1^{k-1}),$$

(3)

The j-th coordinate $z_{i,j}^{k-1}$ of ${\mathbf{z}}_i^{k-1}$ ($0\le j\le m$) is updated as follows:

$$b_j\leftarrow {\left(d_i{,}_j^k\right)}^T(x_i-{\mathbf{D}}_i^k{\mathbf{z}}_i^{k-1})+z_{i,j}^{k-1},$$

(4)

$$z_{i,j}^{k-1}\leftarrow h_{\lambda }\left(b_j\right),$$

(5)

where $h_{\lambda }$ is a soft threshold shrinkage function [27,28] and $b_j$ is a descent parameter that can be calculated by Equation (5). An updating cycle is equivalent to one step of coordinate descent. Dictionary D is updated by stochastic gradient decent (SGD):

$${\mathbf{D}}_{i+1}^k=P_{B_m}({\mathbf{D}}_i^k-{\eta }_i^k{\nabla }_{{\mathbf{D}}_i^k}{f}_i({\mathbf{D}}_i^k,{\mathbf{z}}_i^k)),$$

(6)

where P denotes the projection operator, and $B_m$ is the feasible set of D that is defined as follows: $B_m=\{D\in {\mathbb{R}}^{m\times n}:\forall j=1,\dots ,m,\left|\right|d_j{\left|\right|}_2\le 1\}$. The learning rate is an approximation of the inverse of the Hessian matrix $\mathbf{H}={\sum }_{k,i}{z}_i^k{\left(z_i^k\right)}^T$. The gradient of ${\mathbf{D}}_i^k$ can be obtained as follows:

$${\nabla }_{{\mathbf{D}}_i^k}{f}_i({\mathbf{D}}_i^k,{\mathbf{z}}_i^k)=({\mathbf{D}}_i^k{\mathbf{z}}_i^k-x_i){\left({\mathbf{z}}_i^k\right)}^T,$$

(7)

This allows $i=i+1$. If $i>n$, set ${\mathbf{D}}_1^{k+1}={\mathbf{D}}_{n+1}^k$, $k=k+1$ and $i=1$. Then it repeats the previous calculating process. When $k>m$, the calculation stops, m is preset value and usually $10\le m\le 15$. The SCC only runs a few steps of CD to update the sparse codes and SGD algorithm is conducted to update the dictionary.

All sub-dictionaries ${\mathbf{D}}_1$, ${\mathbf{D}}_2$, ..., ${\mathbf{D}}_n$ are learned by using SCC. These sub-dictionaries are used for describing the underlying structure of each image patches cluster.

2.3.2. Sub-Dictionary Combination

As a sub-dictionary for each cluster is learned, all sub-dictionaries are combined into a new dictionary Φ.

$$\mathsf{\Phi }=[{\mathbf{D}}_1,{\mathbf{D}}_2\dots ,{\mathbf{D}}_n],$$

(8)

2.4. Fusion Scheme

The fusion scheme is shown in Figure 1 and the image fusion algorithm is shown in Algorithm 2. The learned dictionary is used for the estimation of coefficient vectors. For each image patch ${\mathbf{p}}_i$, a coefficient vector ${\mathbf{z}}_i$ is estimated by the SOMP algorithm using the learned dictionary. Max-L1 rule [21] is conducted for coefficient fusion shown in Equation (17),

$${\mathbf{z}}^i={\displaystyle \sum _{k=1}^m}{\mathbf{z}}_k*O^k,where\left\{\begin{array}{c}{O}^k=1,max({∥{\mathbf{z}}_1∥}_1,{∥{\mathbf{z}}_2∥}_1\dots ,{∥{\mathbf{z}}_m∥}_1)={∥{\mathbf{z}}_k∥}_1\hfill \\ O^k=0,otherwise\hfill \end{array}\right.$$

(9)

where ${\mathbf{z}}^i$ is the fused coefficient vector, ${∥•∥}_1$ is the $l_1$ norm, and * is an elementary multiplication operation.

Algorithm 1 Online Sub-dictionary Learning Algorithm

Input:

image patch ${\mathbf{x}}_i$, sparse code ${\mathbf{z}}_i^k$, learning rate ${\eta }_1^1$, running time m

Output:

learned sub-dictionaries ${\mathbf{D}}_1$, ${\mathbf{D}}_2$, ..., ${\mathbf{D}}_n$

1: Initialize ${\mathbf{D}}_1^1$, ${\mathbf{z}}_i^0$=0 (i=1,2,...n), ${\eta }_1^1$= 1 2: Acquire an image patch ${\mathbf{x}}_i$, when k=1 and i=1 3: while $k\le m$ do 4:   Update sparse code $z_i^k=CD({\mathbf{D}}_i^k,{\mathbf{x}}_i,{\mathbf{z}}_1^{k-1})$ 5:   Update j-th coordinate $z_{i,j}^{k-1}$ of ${\mathbf{z}}_i^{k-1}$ according to Equations (4) and (5) 6:   Update dictionary ${\mathbf{D}}_{i+1}^k=P_{B_m}({\mathbf{D}}_i^k-{\eta }_i^k{\nabla }_{{\mathbf{D}}_i^k}{f}_i({\mathbf{D}}_i^k,{\mathbf{z}}_i^k))$ 7:   Update $i=i+1$ 8:   if $i>n$ then 9:    ${\mathbf{D}}_1^{k+1}={\mathbf{D}}_{n+1}^k$, $k=k+1$, and $i=1$ 10:   end if 11: end while

Algorithm 2 Image Fusion Algorithm

Input:

image patch ${\mathbf{p}}_i$, coefficient vector ${\mathbf{z}}_i$

Output:

fused image I

1: for $i=1$; $i\le n$; $i++$ do 2:  ${\mathbf{z}}^i={\sum }_{k=1}^m{\mathbf{z}}_k*O^k$ according to Equation 9 3: end for 4: for $i=1$; $i\le n$; $i++$ do 5:  ${\mathbf{I}}^i=\mathbf{D}{\mathbf{Z}}^i$ 6: end for

The fused coefficient vectors are restored to an image. The restoring process is based on Equation (10),

$${\mathbf{I}}^i=\mathbf{D}{\mathbf{Z}}^i$$

(10)

where the ${\mathbf{Z}}^i=\{{\mathbf{z}}_1^i,{\mathbf{z}}_2^i\dots ,{\mathbf{z}}_m^i\}$ is corresponding to image patches of the fused image and D is the learned dictionary.

3. Experiments and Analyses

The proposed multi-focus image fusion method is applied to standard multi-focus images from public website [29]. All standard multi-focus images used in this paper are free to use for research purposes. The images from the image fusion library have the size of 256 × 256 pixels or 320 × 240 pixels. The fused images are evaluated by comparing them to the fused images of other existing methods. In this paper, four pairs of images are used as a case study to simulate the proposed multi-focus image fusion method. To simulate a real world environment, four pairs of images have two scenes. One is an outdoor scene, such as a hot-air balloon and leopard shown in Figure 3a,b respectively. The other is an indoor scene, such as a lab and bottle shown in Figure 3c,d respectively. These four pairs of original images are from the same sensor modality. Since each image focuses on a different object, there are two images for each scene. The out-of-focus regions in the original images are blurred.

3.1. Experiment Setup

The quality of the proposed image fusion scheme is evaluated against the other seven popular multi-focus fusion methods. These multi-focus fusion methods consist of popular spatial-based image fusion methods, like Laplacian energy (LE) [8]; four states of art MST methods, including disperse wavelet transform (DWT) [30], dual-tree complex wavelet transform (DT-CWT) [31], curvelet (CVT) [13], non-subsampled contourlet (NSCT) [9]; two sparse representation methods, including sparse representation with a fixed DCT dictionary (SR-DCT) [21] and sparse representation with a trained dictionary by K-SVD (SR-KSVD) [32]. The objective evaluation of fused image includes edge intensity (EI) [33], edge retention $Q^{AB/F}$ [34], mutual information (MI) [35,36], and visual information fidelity (VIF) [37]. Then it compares the dictionary construction time of the proposed method with K-SVD [32], which is the most popular dictionary construction method. All experiments are implemented using Matlab, version 2014a; MathWorks: Natick, MA, 2014. and Visual Studio, version Community 2013; Microsoft: Redmond, WA, 2013. mixed programming on an Intel(R) Core(TM), version i7-4720HQ; Intel: Santa Clara, CA, 2015. CPU @ 2.60GHz Laptop with 12.00 GB RAM.

3.1.1. Edge Intensity

The quality of the fused image is measured by the local edge intensity L in image I [38]. It folds a Gaussian kernel G with the image I to get a smoothed image. Then it obtains the edge intensity image by subtracting the smoothed image from the original image. The spectrum of the edge intensities depends on the width of the Gaussian kernel G.

$$\mathbf{L}=max(0,\mathbf{I}-\mathbf{G}\ast \mathbf{I})$$

(11)

The fused image H is calculated by image $L_j,j=1,\dots ,n$ and the weighted average of local edge intensities.

$$\mathbf{H}(x,y)={\displaystyle \sum _{j=1}^N}{w}_j(x,y){\mathbf{L}}_j(x,y),$$

(12)

$$w_j(x,y)=\frac{L_j(x,y)}{{\sum }_{k=1}^N{L}_k(x,y)}$$

(13)

3.1.2. Mutual Information

MI for images can be formalized as Equation (14) [35].

$$MI={\displaystyle \sum _{i=1}^L}{\displaystyle \sum _{j=1}^L}{h}_{A,F}(i,j)lo{g}_2\frac{h_{A,F}(i,j)}{h_A\left(i\right)h_{F\left(j\right)}},$$

(14)

where L is the number of gray-level, $h_{R,F}(i,j)$ is the gray histogram of image A and F. The $h_A\left(i\right)$ and $h_F\left(j\right)$ are edge histogram of image A and F. For a fused image, the MI of the fused image can be calculated by Equation (15).

$$MI(A,B,F)=MI(A,F)+MI(B,F),$$

(15)

where $MI(A,F)$ represents the MI value between input image A and fused image F; $MI(B,F)$ represents the MI value of input image B and fused image F.

3.1.3. $Q^{AB/F}$

The $Q^{AB/F}$ metric is a gradient-based quality index to measure how well the edge information of source images conducted to the fused image [34]. It is calculated by:

$$Q^{AB/F}=\frac{{\sum }_{i,j}(Q^{AF}(i,j)w^A(i,j)+Q^{BF}(i,j)w^B(i,j))}{{\sum }_{i,j}(w^A(i,j)+w^B(i,j))},$$

(16)

where $Q^{AF}=Q_g^{AF}{Q}_0^{AF}$, $Q_g^{AF}$ and $Q_0^{AF}$ are the edge strength and orientation preservation values at location (i,j). $Q^{BF}$ can be computed similarly to $Q^{AF}$ . $w_A(i,j)$ and $w_B(i,j)$ are the importance weights of $Q^{AF}$ and $Q^{BF}$ respectively.

3.1.4. Visual Information Fidelity

VIF is the novel full reference image quality metric. VIF quantifies the mutual information between the reference and test images based on natural scene statistics (NSS) theory and human visual system (HVS) model. It can be expressed as the ratio between the distorted test image information and the reference image information, the calculation equation of VIF is shown in Equation (17).

$$VIF=\frac{{\sum }_{i\in subbands}I(\overrightarrow{C^{N,i}};\overrightarrow{F^{N,i}})}{{\sum }_{i\in subbands}I(\overrightarrow{C^{N,i}};\overrightarrow{E^{N,i}})},$$

(17)

where $I(\overrightarrow{C^{N,i}};\overrightarrow{F^{N,i}})$ and $I(\overrightarrow{C^{N,i}};\overrightarrow{E^{N,i}})$ represent the mutual information, which are extracted from a particular subband in the reference and the test images respectively. $\overrightarrow{C^N}$ denotes N elements from a random field, $\overrightarrow{E^N}$ and $\overrightarrow{F^N}$ are visual signals at the output of HVS model from the reference and the test images respectively.

An average VIF value of each input image and integrated image is used to evaluate the fused image. The evaluation function of VIF for image fusion is shown in Equation (18) [37].

$$VIF(A,B,F)=\frac{VIF(A,F)+VIF(B,F)}{2},$$

(18)

where $VIF(A,F)$ is the VIF value between input image A and fused image F; $VIF(B,F)$ is the VIF value between input image B and fused image F.

3.2. Image Quality Comparison

To show the efficiency of the proposed method, the quality comparison of fused images is demonstrated. Four pairs of multi-focused images of a hot-air balloon, leopard, lab, and bottle are employed for quality comparison. It compares the quality of fused image based on visual effect, the accuracy of focused region detection, and the objective evaluations. The different images are used to show the differences between fused images and corresponding source images. This paper increases the contrast and brightness of difference images for printable purposes. All difference images are adjusted by using the same parameters.

In the first comparison experiment, the "hot-air balloon" images are a pair of multi-focused images. The source multi-focused images are shown in Figure 4a,b. In Figure 4a, the biggest hot-air balloon on the bottom left is out of focus, the rest of the hot-air balloons are in focus. In contrast, in Figure 4b, the biggest hot-air balloon is in focus, but the rest of balloons are out of focus. LE, DWT, DT-CWT, CVT, NSCT, SR-DCT, SR-KSVD and the proposed method are employed to merge two multi-focused images into a clear one, respectively. The corresponding fusion results are shown in Figure 4c–j respectively. The difference images of LE, DWT, DT-CWT, CVT, NSCT, SR-DCT, SR-KSVD and the proposed method do the matrix subtraction with the source images shown in Figure 4a,b. The corresponding subtracted results are shown in Figure 5a–h,i–p respectively.

Figure 5a–h are the difference images between Figure 4a and Figure 4c–j, and Figure 5i–p are the difference images between Figure 4b and Figure 4c–j. The difference images of LE, DWT, DT-CWT, CVT, NSCT, SR-DCT, SR-KSVD and the proposed method are the matrix subtraction results of the corresponding fused images and source images shown in Figure 4a,b.

There are a lot of noises in Figure 4h, which are acquired by SR-DCT. The rest of integrated images in Figure 4 are similar. Difference images, that show hot-air balloons of LE, DWT, and DT-CWT on the left side respectively, do not get all the edge information in Figure 5a–c.

Similarly, Figure 5i–k shows that the biggest hot-air balloons of LE, DWT, and DT-CWT on the bottom left, respectively, are not totally focused. Due to the misjudgement of focused areas, the fused "hot-air balloon" images of LE, DWT, DT-CWT, and SR-DCT have shortcomings. Compared with source images, the rest of the methods do great job in identifying the focused area. To further compare the quality of fused images, objective metrics are used.

Table 1. Objective Evaluations of Multi-focus "Hot-air Balloon" Fusion Experiments.

Method	EI	${\mathit{Q}}^{\mathit{AB}/\mathit{F}}$	MI	VIF
LE	70.5069	0.9843	7.1359	0.7829
DWT	64.2923	0.9465	6.8532	0.7463
DT-CWT	63.5688	0.9878	6.8828	0.7771
CVT	69.6241	0.9812	7.1579	0.8113
NSCT	69.4987	0.9850	7.6092	0.8187
SR-DCT	65.9602	0.7488	3.9220	0.4695
SR-KSVD	69.5190	0.9854	7.8824	0.8234
SR-SCC	69.5625	0.9852	7.9117	0.8242

shows the objective evaluations. Compared with the rest of the image fusion methods, the proposed method SR-SCC gets the largest value in MI and VIF. LE and DT-CWT get the largest value in EI and $Q^{AB/F}$ respectively, but they provide an inaccurate decision in detecting the focused region. The proposed method has the best overall performance of multi-focus image fusion in the "hot-air balloon" scene among all eight methods, according to the quality of fused image, accuracy of locating focused region, and objective evaluations.

Similarly, the source images of the other three comparison experiments, as "leopard", "lab" and "bottle", are shown in Figure 6, Figure 7 and Figure 8a,b respectively. In a set of source images, two images (a) and (b) focus on different items. The source images are fused by LE, DWT, DT-CWT, CVT, NSCT, SR-DCT, SR-KSVD, and the proposed method to get a totally focused image, and the corresponding fusion results are shown in Figure 6, Figure 7 and Figure 8c–j respectively. The difference between fused images and their corresponding source images are shown in Figure 9, Figure 10 and Figure 11a–h,i–p respectively.

Objective metrics of multi-focus "leopard", "lab", and "bottle" fusion experiments are shown in

Table 2. Objective Evaluations of Multi-focus "Leopard" Fusion Experiments.

Method	EI	${\mathit{Q}}^{\mathit{AB}/\mathit{F}}$	MI	VIF
LE	94.1343	0.9443	6.7677	0.7920
DWT	94.8093	0.9430	6.1765	0.7629
DT-CWT	80.2643	0.9470	6.2083	0.7636
CVT	93.5736	0.9359	6.4109	0.8068
NSCT	93.6767	0.9447	7.1940	0.8245
SR-DCT	76.9935	0.7343	3.3002	0.7291
SR-KSVD	93.5276	0.9450	7.4607	0.8269
SR-SCC	93.6448	0.9448	7.5420	0.8283

,

Table 3. Objective Evaluations of Multi-focus "Lab" Fusion Experiments.

Method	EI	${\mathit{Q}}^{\mathit{AB}/\mathit{F}}$	MI	VIF
LE	55.0926	0.7964	5.0724	0.6299
DWT	56.7262	0.8795	4.6665	0.5885
DT-CWT	44.6470	0.8196	5.0999	0.6025
CVT	53.5711	0.8438	4.7937	0.6367
NSCT	53.5958	0.8055	5.3111	0.6625
SR-DCT	48.8765	0.6332	3.1122	0.3385
SR-KSVD	53.0461	0.8791	5.7345	0.6766
SR-SCC	53.0304	0.8826	5.6656	0.6893

and

Table 4. Objective Evaluations of Multi-focus "Bottle" Fusion Experiments.

Method	EI	${\mathit{Q}}^{\mathit{AB}/\mathit{F}}$	MI	VIF
LE	128.0512	0.8478	4.1059	0.4113
DWT	131.5033	0.8355	3.5696	0.3779
DT-CWT	96.3171	0.8548	3.8049	0.4035
CVT	122.5616	0.8448	3.8799	0.4678
NSCT	69.4987	0.9850	7.6092	0.8187
SR-DCT	122.4806	0.8518	4.1766	0.4784
SR-KSVD	128.4125	0.8625	4.8410	0.5118
SR-SCC	128.5106	0.8629	4.8901	0.5291

respectively to evaluate the quality of fused images.

• Multi-focus "leopard" fusion: The proposed method SR-SCC achieves the largest value in MI and VIF. LE obtains the largest value in EI index, but it makes inaccurate decision in detecting the focused region. SR-KSVD shows great performance in $Q^{AB/F}$, and the result of proposed method is only 0.0002 smaller than SR-KSVD. According to the quality of visual image, the accuracy of focused region, and objective evaluations, the proposed method does a better job than the rest of the methods.
• Multi-focus "lab" fusion: The proposed method SR-SCC achieves the largest value in $Q^{AB/F}$ and VIF. DWT obtains the largest value of EI index, but it cannot distinguish the correct focused areas. SR-KSVD has the best performance in MI. The proposed method and SR-KSVD show great performance of visual effect in focused area, distinguishing focused area, and objective evaluation. Compared with SR-KSVD, the proposed method dramatically reduces computation costs in dictionary construction. So the proposed method has the best overall performance among all comparison methods.
• Multi-focus "bottle" fusion: DWT obtains the largest value in EI, but it does not get an accurate focused area. The proposed method achieves SR-SCC with the largest values of the rest of the objective evaluations. So the proposed method has the best overall performance compared with other methods in the "bottle" scene.

3.3. Dictionary Construction Time Comparison

As shown in the previous subsection, the fused images of different multi-focus fusion methods are compared by objective evaluations. Dictionary-learning based image fusion methods, including SR-KSVD and the proposed SCC method, achieve the best performance. However, the dictionary construction process usually takes a very long time. The efficiency of dictionary construction is an important feature of image fusion method. Both K-SVD [39] and the proposed SCC are sparse-representation based dictionary learning methods. So it compares the dictionary construction time of K-SVD and the proposed SCC. K-SVD is one of the most popular dictionary learning methods in recent years. It uses an iterative algorithm to reduce dictionary learning errors and can describe the underlying structure of the image perfectly. To verify the low computation cost of the proposed method, four pairs of images are used for testing computation time. The time consumption of K-SVD and SCC are shown in Figure 12 and

Table 5. Time Consumption Comparison.

Method	Hot-Air Balloon	Leopard	Lab	Bottle
K-SVD	46.49	45.02	47.72	42.72
SCC	3.56	3.60	3.72	3.12

. SCC uses low computation times, that are marked in bold, in four group experiments. The experimental results demonstrate that SCC has a much better performance of computation time than K-SVD.

4. Conclusions

This paper proposed an integrated image fusion framework based on online dictionary learning. Compared with traditional image fusion methods, the integrated framework had two major improvements. First, it introduced a local density-based clustering method in sparse representation, which had high performance in clustering without any apriori knowledge. Second, an online dictionary learning algorithm was used to extract discriminative features that enhanced the efficiency of image fusion. The proposed method was compared with seven existing algorithms LE, DWT, DT-CWT, CVT, NSCT, SR-DCT, SR-KSVD, and SR-SCC using four source image pairs and objective metrics. Experimental results demonstrated that the proposed method was significantly superior to other methods in terms of subjective and objective evaluation. This means that the fused images of the proposed method had better quality than other methods. Compared with other sparse-representation based methods, the proposed method had high efficiency in generating fused images.

Although the proposed solution had a good performance in image fusion, many optimizations are still worth doing in the follow-up research. The parallel processing and the use of multiple graphics processing units (GPUs) will be considered to improve the efficiency of proposed solution. Denoising techniques will also be applied to the proposed solution to enhance the quality of the fused image.