An Improved Recognition Approach for Noisy Multispectral Palmprint by Robust L2 Sparse Representation with a Tensor-Based Extreme Learning Machine

Abstract

For the past decades, recognition technologies of multispectral palmprint have attracted more and more attention due to their abundant spatial and spectral characteristics compared with the single spectral case. Enlightened by this, an innovative robust L2 sparse representation with tensor-based extreme learning machine (RL2SR-TELM) algorithm is put forward by using an adaptive image level fusion strategy to accomplish the multispectral palmprint recognition. Firstly, we construct a robust L2 sparse representation (RL2SR) optimization model to calculate the linear representation coefficients. To suppress the affection caused by noise contamination, we introduce a logistic function into RL2SR model to evaluate the representation residual. Secondly, we propose a novel weighted sparse and collaborative concentration index (WSCCI) to calculate the fusion weight adaptively. Finally, we put forward a TELM approach to carry out the classification task. It can deal with the high dimension data directly and reserve the image spatial information well. Extensive experiments are implemented on the benchmark multispectral palmprint database provided by PolyU. The experiment results validate that our RL2SR-TELM algorithm overmatches a number of state-of-the-art multispectral palmprint recognition algorithms both when the images are noise-free and contaminated by different noises.

1. Introduction

Palmprint recognition technologies have become a novel biometric approach and have attracted increasingly attention in recent years. In comparison with some other biological features (i.e., the iris and fingerprints, etc.), palmprints have a larger collection area with more abundant information. Besides, palmprints possess the characteristics of uniqueness, stability, scalability and non-contact acquisition, etc. As a consequence, they have strong anti-noise capability and efficient discrimination performance.

The current palmprint recognition algorithms can be mainly categorized into various sorts, such as subspace-based methods, feature-based methods and sparse representation-based classification (SRC) methods, etc. The subspace-based methods [1,2,3,4,5,6,7,8,9,10] adopt dimension reduction theory to accomplish the feature space transformation. This can reduce the data complexity and efficiently improve the discrimination of image characteristics. The conventional subspace transformation methods mainly includes the principal component analysis (PCA) [1], linear discriminant analysis (LDA) [4] and independent component analysis (ICA) [7], etc. However, due to their sensitivity to lighting, noise and other contaminations, the conventional linear discriminant methods already don’t meet the requirements of actual palmprint recognition problems. To address these issues, a nonlinear spatial structure transformation technique, namely the kernel PCA method [9,10], was introduced into the palmprint recognition field. In addition, lots of feature-based methods were presented to implement palmprint recognition tasks. For instance, the coding-based method [11,12,13,14,15,16,17] has been extensively researched in the past decades. In these studies, the palmprint features were extracted by using the coding of some filtering results. The common coding methods include binarized statistical image features (BSIF) [13], double-orientation code (DOC) [14] and block dominate orientation code [17], etc. Minaee et al. [18] proposed a palmprint recognition algorithm by using the deep scattering network which achieved fine recognition performance. Some other feature- based methods [19,20,21,22,23,24] mainly take advantage of the statistic characteristics, such as mean, variance, and covariance and so on, to implement palmprint recognition. In recent years, the linear representation methods based on sparse theory [25] were proposed and popularly applied to the palmprint recognition problem [26,27,28,29,30,31]. These methods consider a testing sample as a linear representation of the training set. That is, a given testing sample was anticipated to be approximately expressed by the training samples lied in a unitary class. This can be effectively accomplished by imposing the sparseness constraint on the approximate representation with the training samples.

For the sake of higher recognition accuracy, some multispectral palmprint recognition methods [32,33,34,35,36,37,38,39,40,41,42,43,44] have been studied. Because the collected images under different spectra contain more plentiful feature information, the recognition rate can be effectively improved. In these studies, different fusion strategies were utilized to increase the recognition accuracy. The conventional multispectral palmprint recognition methods can be mainly categorized into image level fusion strategies and matching score level fusion strategies. The basic idea of image level fusion is to decompose the images under different spectra at the start, then integrate these separated decompositions for a compound approximation and reconstruct the fusion image through the inverse transformation to implement the recognition task. Based on this, Han et al. [32] used the discrete wavelet transform (DWT) method to decompose palmprint images acquired under different spectra, and then reconstructed the fused palmprint image to accomplish the multispectral palmprint recognition. Xu et al. [37] introduced the quaternion matrix to represent the palmprint images under different spectra, and then extended PCA and DWT into the quaternion domain to implement feature extraction. Finally, the Euclidean distance was used to perform the recognition task. Gumaei et al. [38] employed an autoencoder with the regularized extreme learning machine (AE-RELM) to accomplish the multispectral palmprint recognition and effectively improve the accuracy. Xu et al. [39] presented a novel multispectral palmprint recognition algorithm. They used the digital shearlet transform (DST) to implement the image fusion and proposed a multiclass projection ELM (MPELM) to accomplish the classification task. For the score level fusion method, the matching scores are obtained separately by a comparator for different spectral bands firstly, then the obtained matching scores are fused by utilizing some rules and accomplish the classification based on the fusion score. Zhang et al. [41] presented a novel algorithm named line orientation-based coding (LOC) to extract the featurew of the palmprint images with different spectrq, and then carried out the recognition task with a matching level fusion rule. Minaee et al. [42] used the co-occurrence matrix to extract the texture features, then employed the minimum distance classifier (MDC) and weighted majority voting system (WMV) to accomplish the multispectral palmprint recognition. Minaee et al. [43] presented a set of wavelet-DCT features for multispectral palmprint recognition. Although many achievements have been made in the study of multispectral palmprint recognition, there are still many open questions that need to be further studied. For example, how to increase the recognition accuracy when the collected images are contaminated by different noises.

Inspired by the these studes, in this article, we present a novel robust L2 sparse representation with a tensor-based extreme learning machine (RL2SR-TELM) algorithm by using an adaptive image level fusion strategy to accomplish the multispectral palmprint recognition. The key contributions of our algorithm can be summarized as follows: Firstly, a robust L2 norm-based sparse representation model is constructed to calculate the linear representation coefficients. It overcomes the defects of high computational complexity of the L1 norm regularization and the lack of robustness to noise contamination. Secondly, an adaptive weighted method is presented to accomplish the fusion of multispectral palmprint images at the image level. In this method, a weighted sparse and collaborative concentration index (WSCCI) is proposed that can quantify the multispectral palmprint image discrimination efficiently. By using the robust sparse coefficients and WSCCI, an adaptive weighted fusion strategy is proposed to reconstruct the fused palmprint image. Finally, aiming at the high order signal classification problem, we extend the conventional ELM [45] into the tensor space, then put forward a novel TELM method. It inherits the advantages of the conventional ELM (i.e., excellent learning speed and generalization performance) which achieves an outstanding recognition efficiency.

The rest of this paper is organized as follows: in Section 2, we introduce the principle of multispectral palmprint acquisition device. Then we discuss our proposed RL2SR-TELM algorithm in Section 3. In Section 4, simulation experiments and the result analysis of our proposed algorithm are illustrated in detail. Section 5 concludes this paper.

2. Acquisition Device of Multispectral Palmprint Images

The Biometrics Research Centre (BRC) of Hong Kong Polytechnic University (PolyU) has developed an acquisition device [46] for multispectral palmprints. It can collect the palmprint images using the Blue, Green, Red and Near Infrared (NIR) spectra, respectively. Figure 1 illustrates the principle of the acquisition device. It mainly includes a multispectral light source module, a light source control module, a CCD imaging sensor, an image acquisition module (A/D conversion module) and an image display module, etc. The multispectral light source module locates at the bottom of the device and consists of four monochromatic light sources. The light controller module controls the multispectral light and enables CCD imaging module to acquire palmprint images under different spectrums. The image acquisition module captures the multispectral palmprint images and converts analog image into a digital one by an A/D conversion. Figure 2 shows the acquired palmprint images with different spectrums.

3. Proposed Algorithm

Figure 3 illustrates the flowchart of the presented RL2SR-TELM algorithm. It can be mainly separated into the following steps: Firstly, the acquired multispectral palmprint image is preprocessed to obtain the region of interest (ROI) of the image. Then, we calculate the sparse representation coefficients of sample images under different spectra by utilizing the proposed robust L2 sparse representation method. After that, an adaptive weighted fusion strategy is presented to obtain the fused images. Finally, by integrating the tensor theory with ELM, we propose a TELM method to complete the recognition task.

3.1. Robust L2 Sparse Representation Method

3.1.1. SRC Model

The sparse representation idea was introduced into the biometric recognition for the first time in 2009 by Wright et al. Given the training set matrix denoted as $X=[x_1,x_2,\dots ,x_n]\in R^{d\times n}$, where $x_i$ is a training sample, $d$ and $n$ denote the training sample dimension and number, respectively. For any given testing sample $y\in R^d$, we suppose that it can be coded over the training matrix $X$ approximately, then the SRC model can be described as:

$$\alpha =\arg \underset{\alpha }{\min }{\Vert \alpha \Vert }_0,s.t.y=X\alpha ,$$

(1)

where $\alpha \in R^n$ is the representation coefficient, ${\Vert \alpha \Vert }_0$ denotes the L0 norm and it counts the nonzero element number of the vector. The objective of SRC is to find as fast as possible as sparse coefficient $\alpha$ which can represent the testing sample over the training set. Model (1) is a NP hard problem and theoretically intractable. Reference [47] has proved that when the representation coefficient is sparse enough, the L0 norm can be approximately represented by using the L1 norm. On the basis of this theory, Wright et al. proposed the following model:

$$\alpha =\arg \underset{\alpha }{\min }{\Vert \alpha \Vert }_1,s.t.y=X\alpha .$$

(2)

This is a classical model and it has been extensively used in various areas including the image reconstruction, image de-noising, compressive sensing and machine learning, and so on. Although many scholars have devoted themselves to this algorithm and proposed largely improvements, the drawback of inefficiency is still not completely resolved.

3.1.2. Robust L2 Sparse Representation Method

A SRC model actually supposes that the coding residual obeys a Gaussian or Laplacian probability density function distribution. However, this hypothetical description is not always accurate enough in practice. In addition, the SRC needs to solve the L1 regularization problem and its calculation speed is very slow. To address these drawbacks, many researchers have proposed a lot of improved SRC algorithms. For examples, Yang et al. [48] proposed a novel sparse representation method that solved the sparse representation problem by using the maximum likelihood estimation (MLE) method. It can deal with the occlusion and outliers more robustly. Xu et al. [49] made use of the L2 regularization to acquire the sparse coefficient and proposed a new discriminative sparse representation method (DSRM). Inspired by these ideas, we propose a novel robust L2 regularization based sparse representation method, namely RL2SR.

Suppose that there are $s$ different spectral bands, the class number of each spectral palmprint is $C$ and each class has $m$ training samples. Thus, there are $N=mC$ training samples for each spectrum. Vectorize the training sample into the $d-$ dimensional column vector, then the training sample matrix can be denoted as $X=[X_1,\dots ,X_i,\dots ,X_C]$, where $X_i=[x_{m(i-1)+1}^1,x_{m(i-1)+1}^2,\dots ,x_{m(i-1)+1}^s,\dots ,x_{mi}^1,x_{mi}^2,\dots ,x_{mi}^s]$ is the training sample sub-matrix of the i-th class, $x_{mi}^1,x_{mi}^2,\dots ,x_{mi}^s$ are the (m×i)-th training samples under different spectra. Then, given any testing sample $y^l$, where $l=1,2,\dots ,s$ denotes the spectral bands, we can construct the following optimization problem:

$$\arg \underset{A^l}{\min }\rho (y^l-X{A}^l)+\lambda \varphi (A^l),(l=1,2,\dots ,s),$$

(3)

where $\lambda >0$ is a constant namely regularization parameter which can balance the representation residual term and the regularization term. Here, $A^l=[A_1^l;A_2^l;\cdots ;A_C^l]$ is the linear representation coefficient with respect to the testing sample $y^l$ over the training set.

For the first term of the optimization function (3), it can be denoted as $\rho (e^l)=\rho (y^l-X{A}^l)$, where $\rho (\cdot ):R^d\to R$. Then:

$$\rho (e^l)={\displaystyle \sum _{k=1}^d\rho (e_k^l)},$$

(4)

where $e_k^l=\left|y_k^l-X_k{A}^l\right|$ denotes the residual term with respect to the $k\mathrm{th}$ element between $y^l$ and its approximate linear representation $X{A}^l$. $y_k^l$ and $X_k$ are the k-th element of the testing sample and the $k\mathrm{th}$ row of the training set matrix, respectively. In general, the residual function $\rho (\cdot )$ is designed to minimize the effect generated by the occlusion and outliers. Huber, Cauchy and Welsch functions can be used to express the residual function. In reference [48], Yang et al. utilized the logistic function to describe the residual information and got satisfactory performance. The logistic function can be expressed as follows:

$$\rho (e_k^l)=-\frac{1}{2\mu }(\ln (1+\exp (-\mu {(e_k^l)}^2+\mu \delta ))-\ln (1+\exp (\mu \delta )))$$

(5)

where $\mu$ and $\delta$ are the positive parameters. The selection of parameters $\mu$ and $\delta$ will be discussed in Section 4.2. In order to solve question (3), we derivative $\rho (y^l-X{A}^l)$ with respect to Aⁱ, then we have:

$$\frac{d\rho (y^l-X{A}^l)}{d{A}^l}=\frac{{\displaystyle \sum _{k=1}^{d}d\rho (e_k^l)}}{d{A}^l}={\displaystyle \sum _{k=1}^d\frac{d\rho (e_k^l)}{d{e}_k^l}\frac{d{e}_k^l}{d{A}^l}}=\frac{1}{2}{\displaystyle \sum _{k=1}^d\frac{d\rho (e_k^l)}{d{e}_k^l}\frac{1}{e_k^l}\frac{d{(e_k^l)}^2}{d{A}^l}}=\frac{1}{2}{\displaystyle \sum _{k=1}^d\omega (e_k^l)\frac{d{(e_k^l)}^2}{d{A}^l}}.$$

(6)

Furthermore, since $\frac{d}{d{A}^l}({\Vert {(W^l)}^{1/2}{e}^l\Vert }_2^2)=W^l\frac{d}{d{A}^l}({\Vert y^l-X{A}^l\Vert }_2^2)$, where $W^l=diag(\omega (e_1^l),\omega (e_2^l),\dots ,\omega (e_d^l))$ denotes the residual function, Equation (6) can be regarded as the derivative of $\frac{1}{2}{\Vert {(W^l)}^{1/2}{e}^l\Vert }_2^2$.

By using Equation (5), the residual function can be calculated as follows:

$$\omega (e_k^l)=\frac{d\rho (e_k^l)}{d{e}_k^l}\frac{1}{e_k^l}=\frac{\exp (-\mu {(e_k^l)}^2+\mu \delta )}{1+\exp (-\mu {(e_k^l)}^2+\mu \delta )}.$$

(7)

For the residual matrix $W^l$, the following method is proposed to calculate it:

Step 1: Initiate $W^{l,1}=diag(1,1,\dots ,1)$ and calculate the collaborative code ${\gamma }^l$ of each testing sample by using the collaborative representation model

$${\gamma }^l=\arg \underset{{\gamma }^l}{\min }{\Vert W^l(y^l-X{\gamma }^l)\Vert }_2^2+\xi {\Vert {\gamma }^l\Vert }_2^2,(l=1,2,\dots ,s).$$

Step 2: Substitute the collaborative residual $e_k^l=\left|y_k^l-X_k{\gamma }^l\right|,(k=1,\dots ,d)$ into Equation (7) and obtain the residual matrix $W^l$.

Step 3: If $W^l$ is not convergent, repeat step 1 and step 2, otherwise output $W^l$.

With the residual matrix $W^l$ calculated, Equation (3) can be rewritten as follows

$$\arg \underset{A^l}{\min }\frac{1}{2}{\Vert {(W^l)}^{1/2}{e}^l\Vert }_2^2+\lambda \varphi (A^l),(l=1,2,\dots ,s).$$

(8)

Due to the existence of parameter $\lambda$, omit the coefficient in front of the first term and the Equation (8) becomes:

$$\arg \underset{A^l}{\min }{\Vert {(W^l)}^{1/2}{e}^l\Vert }_2^2+\lambda \varphi (A^l),(l=1,2,\dots ,s).$$

(9)

For the second term $\varphi (A^l)$ of the optimal objective function, SRC [25] adopted the L1 norm to realize the sparseness of linear representation coefficient. In general, an iterative algorithm is employed to solve the L1 norm regularization based sparse representation problem. There are many famous algorithms [50] to implement the iteration, such as L1 regularized least squares (L1LS), homotopy method, augmented Lagrangian method (ALM), orthogonal matching pursuit method (OMP) [51] and fast iterative shrinkage thresholding algorithm (FISTA), etc. However, these methods still suffer from the issue of low efficiency. To address this issue, Zhang et al. [52] introduced the collaborative representation-based classification (CRC) into the method and utilized the L2 regularization to obtain the representation coefficient. Although CRC provided an efficient algorithm, it failed to give full consideration to the sparseness of linear representation. Reference [49] employed L2 regularization to implement the face recognition by utilizing a discriminative sparse representation method. Inspired by this, the L2 regularization item is introduced into our model and a novel RL2SR model is proposed as follows:

$$\arg \underset{A^l}{\min }{\Vert {(W^l)}^{1/2}{e}^l\Vert }_2^2+\lambda {\displaystyle \sum _{i=1}^C{\displaystyle \sum _{j=1}^C{\Vert X_i{A}_i^l+X_j{A}_j^l\Vert }_2^2}},(l=1,2,\dots ,s).$$

(10)

Since:

$${\displaystyle \sum _{i=1}^C{\displaystyle \sum _{j=1}^C{\Vert X_i{A}_i^l+X_j{A}_j^l\Vert }_2^2}}=2{\displaystyle \sum _{i=1}^C{\Vert X_i{A}_i^l\Vert }^2}+2{\displaystyle \sum _{i=1}^C{\displaystyle \sum _{j=1}^C({(X_i{A}_i^l)}^T(X_j{A}_j^l))}},(l=1,2,\dots ,s),$$

(11)

Equation (11) can be separated into two parts. Minimizing ${(X_i{A}_i^l)}^T(X_j{A}_j^l)$ implies that the correlation between the i-th class and j-th class is also minimal with respect to the linear representation. This makes the linear approximation combination have the best discrimination ability. Thus, the second term of Equation (11) has the capability of decorrelating the linear representation combination with different classes. Correspondingly, minimization of the sum ${(X_i{A}_i^l)}^T(X_j{A}_j^l)$, instead of any individual terms, can accomplish the decorrelation affection for different classes. In consequence, this approach can discriminate the testing sample to the really nearest class. Minimization of ${\Vert X_i{A}_i^l\Vert }^2,(i=1,2,\dots ,C)$ means that the norm of the linear representation combination with each class is also small. Similar to the presented linear representation approaches, such as SRC and CRC, there is a competitive relationship between different classes of training samples. In other word, the testing sample can be denoted by the weight sum of the training samples from all of the classes. Obviously, that is a linear representation which means every class makes its impact to represent the testing sample. Competition in representation implies that when a class makes an important impact to the linear representation, the remainder classes make considerably less impact.

The objective function shown in Equation (10) can be rewritten as:

$$L(A^l)=\arg \underset{A^l}{\min }{\Vert {(W^l)}^{1/2}(y^l-X{A}^l)\Vert }_2^2+\lambda {\displaystyle \sum _{i=1}^C{\displaystyle \sum _{j=1}^C{\Vert X_i{A}_i^l+X_j{A}_j^l\Vert }_2^2}},(l=1,2,\dots ,s).$$

(12)

For the first term of objective function (12), using $\arg \underset{A^l}{\min }{\Vert {(W^l)}^{1/2}(y^l-X{A}^l)\Vert }_2^2$ instead of $y^l=X{A}^l$ implies that $X{A}^l$ is a linear approximation of the test image. That is to say, this model can tolerate considerable noise contamination. In the meantime, the residual function can measure the linear representation residual well and enhance the noise robustness of the proposed model. In order to optimize the presented model, we introduce the following theorem:

Theorem 1. 

The proposed RL2SR model (12) is convex and differentiable w.r.t. coefficient$A^l$, and it has a closed form solution.

Proof. 

Firstly, the objective function (12) can be considered as a combination of two L2 regularization terms, i.e., ${\Vert {(W^l)}^{1/2}(y^l-X{A}^l)\Vert }_2^2$ and ${\displaystyle \sum _{i=1}^C{\displaystyle \sum _{j=1}^C{\Vert X_i{A}_i^l+X_j{A}_j^l\Vert }_2^2}}$. By adopting the properties of L2 norm, the convexity and derivative of the proposed model (12) can be easily proved.

Secondly, the derivative of function ${\Vert {(W^l)}^{1/2}(y^l-X{A}^l)\Vert }_2^2$ can be computed as follows:

$$\frac{d}{d{A}^l}{\Vert {(W^l)}^{1/2}(y^l-X{A}^l)\Vert }_2^2=-2X^T{W}^l(y^l-X{A}^l).$$

On the other hand, for the second term ${\displaystyle \sum _{i=1}^C{\displaystyle \sum _{j=1}^C{\Vert X_i{A}_i^l+X_j{A}_j^l\Vert }_2^2}}$, since it does not contain the coefficient $A^l$ explicitly, we could not compute the derivative directly. To address this issue, we compute the partial derivatives of ${\displaystyle \sum _{i=1}^C{\displaystyle \sum _{j=1}^C{\Vert X_i{A}_i^l+X_j{A}_j^l\Vert }_2^2}}$ w.r.t. $A_k^l(k=1,2,\dots ,C)$. Denote $\phi (A^l)={\displaystyle \sum _{i=1}^C{\displaystyle \sum _{j=1}^C{\Vert X_i{A}_i^l+X_j{A}_j^l\Vert }_2^2}}$, we have:

$$\begin{array}{ll}\frac{\partial \phi }{\partial A_k^l}=& \frac{\partial }{\partial A_k^l}\left({\displaystyle \sum _{i=1}^C{\displaystyle \sum _{j=1}^C{\Vert X_i{A}_i^l+X_j{A}_j^l\Vert }_2^2}}\right)\\ & =\frac{\partial }{\partial A_k^l}\left({\displaystyle \sum _{\begin{array}{l}j=1\\ j\ne k\end{array}}^C{\Vert X_k{A}_k^l+X_j{A}_j^l\Vert }_2^2}+{\displaystyle \sum _{\begin{array}{l}i=1\\ i\ne k\end{array}}^C{\Vert X_i{A}_i^l+X_k{A}_k^l\Vert }_2^2}+{\displaystyle \sum _{\begin{array}{l}i=1\\ i\ne k\end{array}}^C{\displaystyle \sum _{\begin{array}{l}j=1\\ j\ne k\end{array}}^C{\Vert X_i{A}_i^l+X_j{A}_j^l\Vert }_2^2}}+{\Vert X_k{A}_k^l+X_k{A}_k^l\Vert }_2^2\right)\\ & =\frac{\partial }{\partial A_k^l}\left(2{\displaystyle \sum _{\begin{array}{l}j=1\\ j\ne k\end{array}}^C{\Vert X_k{A}_k^l+X_j{A}_j^l\Vert }_2^2}+{\displaystyle \sum _{\begin{array}{l}i=1\\ i\ne k\end{array}}^C{\displaystyle \sum _{\begin{array}{l}j=1\\ j\ne k\end{array}}^C{\Vert X_i{A}_i^l+X_j{A}_j^l\Vert }_2^2}}+{\Vert X_k{A}_k^l+X_k{A}_k^l\Vert }_2^2\right)\\ & =4X_k^T\left((C+1)X_k{A}_k^l+{\displaystyle \sum _{\begin{array}{l}j=1\\ j\ne k\end{array}}^C{X}_j{A}_j^l})\right)\\ & =4X_k^T\left(C{X}_k{A}_k^l+{\displaystyle \sum _{j=1}^C{X}_j{A}_j^l})\right)\\ & =4X_k^T(C{X}_k{A}_k^l+X{A}^l)\end{array}$$

Then, we can obtain the derivative as follows:

$$\frac{d\phi }{d{A}^l}=\left(\begin{array}{c}\frac{\partial \phi }{\partial A_1^l}\\ ⋮\\ \frac{\partial \phi }{\partial A_C^l}\end{array}\right)=\left(\begin{array}{c}4X_1^T(C{X}_1{A}_1^l+X{A}^l)\\ ⋮\\ 4X_C^T(C{X}_C{A}_C^l+X{A}^l)\end{array}\right)=4C\left(\begin{array}{ccc}{X}_1^T{X}_1& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & X_C^T{X}_C\end{array}\right)A^l+4X^{T}X{A}^l.$$

By denoting:

$$M=\left(\begin{array}{ccc}{X}_1^T{X}_1& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & X_C^T{X}_C\end{array}\right),$$

we have:

$$\frac{d\phi }{d{A}^l}=4(CM+X^{T}X)A^l.$$

As a consequence, the derivative of objective function (12) with respect to $A^l$ is:

$$\frac{dL}{d{A}^l}=-2X^T{W}^l(y^l-X{A}^l)+4\lambda (CM{A}^l+X^{T}X{A}^l),(l=1,2,\dots ,s).$$

By employing the property of optimal solution, and setting is as zero, the closed solution of objective function (12) is obtained as follows:

$$A^l={(2\lambda CM+2\lambda X^{T}X+X^T{W}^{l}X)}^{-1}{X}^T{W}^l{y}^l,(l=1,2,\dots ,s).$$

(13)

The proof of Theorem 1 is thus completed. □

The proposed RL2SR method is summarized in

Table 1. Robust L2 sparse representation algorithm.

Input: testing sample $y^l,(l=1,2,\dots ,s)$, training sample matrix $X=[X_1,\dots ,X_i,\dots ,X_C]$, initiate the residual function matrix $W^{l,1}=diag(1,1,\dots ,1)$. Output: linear representation coefficient $A^l$$,(l=1,2,\dots ,s)$.

While $error$ not convergent, do 1. Calculate the collaborative representation code ${\gamma }^l$ by solving $${\gamma }^{l,t+1}=\arg \underset{{\gamma }^l}{\min }{\Vert W^{l,t}(y^l-X{\gamma }^l)\Vert }_2^2+\xi {\Vert {\gamma }^l\Vert }_2^2.$$ 2. Calculate the residual by employing $$e_k^{l,t+1}=\left|y_k^l-X_k{\gamma }^{l,t+1}\right|,(k=1,\dots ,d).$$ 3. Calculate the residual function by using $$\omega (e_k^{l,t+1})=\frac{\exp (-\mu {(e_k^{l,t+1})}^2+\mu \delta )}{1+\exp (-\mu {(e_k^{l,t+1})}^2+\mu \delta )},(k=1,\dots ,d).$$ 4. Update $W^l$ by utilizing $$W^{l,t+1}=diag(\omega (e_1^{l,t+1}),\omega (e_2^{l,t+1}),\dots ,\omega (e_d^{l,t+1})).$$ 5. Calculate $error={\Vert W^{l,t+1}-W^{l,t}\Vert }_F/{\Vert W^{l,t}\Vert }_F.$ End while 6. For each spectral testing sample $y^l,(l=1,2,\dots ,s)$, calculate $A^l,(l=1,2,\dots ,s)$ by using $$A^l={(2\lambda CM+2\lambda X^{T}X+X^T{W}^{l}X)}^{-1}{X}^T{W}^l{y}^l$$

.

3.2. Image Fusion Based on Adaptive Weighted Method

In this section, a weighted sparse and collaborative concentration index is introduced to quantify the discrimination of each spectral testing sample and an adaptive weighted fusion method is proposed to construct the fused palmprint image.

Definition 1. 

[25] (sparse concentration index (SCI)) The SCI of a coefficient vector$\alpha \in R^n$is defined as:

$$SCI(\alpha )=\frac{C\cdot \underset{i}{\max }{\Vert {\delta }_i(\alpha )\Vert }_1/{\Vert \alpha \Vert }_1-1}{C-1},$$

(14)

where$C$is the class number,${\delta }_i(\alpha )$is an indicator function defined on$R^n$which keeps the coefficients affiliated to the$i\mathrm{th}$class and sets all the other coefficients to be zero.

Obviously, $SCI(\alpha )=1$ implies that the training samples from a unitary class can express the testing sample well. On the contrary, $SCI(\alpha )=0$ means that all of the training samples have an average impact to represent the testing sample. Therefore, SCI can measure the sparseness of the linear representation coefficient and the discrimination ability of the testing sample efficiently. If $SCI(\alpha )=1$, the testing sample has the strongest discrimination ability and it can be easily classified into the correct class. If $SCI(\alpha )=0$, the testing sample has the weakest discrimination ability and we cannot determine the actual class that the testing sample should belong to.

The SCI uses the L1 norm to evaluate the sparseness of the linear representation coefficient and it can’t efficiently evaluate the coefficient obtained by our RL2SR method since the L2 norm regularization is utilized. It considers not only the sparseness, but also the collaborative representation information of the representation coefficient. To address this issue, the definition of SCI is extended and a weighted sparse and collaborative concentration index, namely WSCCI, is proposed to evaluate the representation coefficient obtained by our RL2SR model.

Definition 2. 

(weighted sparse and collaborative concentration index (WSCCI)) The WSCCI of a coefficient vector$\alpha \in R^n$is defined as:

$$WSCCI(\alpha )=\frac{{\mu }_1\left(C\cdot \underset{i}{\max }{\Vert {\delta }_i(\alpha )\Vert }_1/{\Vert \alpha \Vert }_1-1\right)+{\mu }_2\left(C\cdot \underset{i}{\max }{\Vert {\delta }_i(\alpha )\Vert }_2/{\Vert \alpha \Vert }_2-1\right)}{({\mu }_1+{\mu }_2)(C-1)},$$

(15)

where$C$denotes the class number,${\mu }_1$and${\mu }_2$are nonnegative parameters.

In WSCCI, the weighted fusion of the sparse and collaborative concentration index defined by the L1 norm and L2 norm is utilized to evaluate the discriminative performance of the given sample. As a consequence, it can be regarded as the weighted sum of SCI and CCI (i.e., collaborative concentration index). From the above analysis, the proposed WSCCI can be utilized to model our adaptive weighted fusion method.

The proposed adaptive weighted image fusion method can be summarized as follows:

(1) For the linear representation coefficients $A^l,(l=1,2,\dots ,s)$ obtained by Equation (13), calculate the $WSCCI(A^l),(l=1,2,\dots ,s)$ by using Equation (15).

(2) Normalize $WSCCI(A^l),(l=1,2,\dots ,s)$ by using:

$${\eta }^l=\frac{WSCCI(A^l)}{WSCCI(A^1)+WSCCI(A^2)+\dots +WSCCI(A^s)}=\frac{WSCCI(A^l)}{{\displaystyle \sum _{i=1}^{s}WSCCI(A^i)}},(l=1,2,\dots ,s).$$

(16)

(3) Reconstruct the fused multispectral palmprint image y by using:

$$y=X({\eta }^1{A}^1+{\eta }^2{A}^2+\dots +{\eta }^s{A}^s)=X{\displaystyle \sum _{l=1}^s{\eta }^l{A}^l}.$$

(17)

With the fused multispectral palmprint image obtained, TELM is proposed to implement the recognition task.

3.3. Principle of Tensor Based ELM

ELM can be considered as a generalized single hidden layer feedforward neural network (SLFN). Since ELM randomly chooses the initial values of the hidden nodes and analytically calculates the output weights, the learning speed is extremely fast compared to the conventional supervised learning algorithms (i.e., support vector machine (SVM) [53] and k-nearest neighbor (KNN) algorithm, etc.). In addition, its generalization ability is better than many back propagation neural networks algorithms. In consequence, ELM has been extensively studied and widely applied in lots of areas (such as pattern classification, clustering analysis and regression etc.) and plenty of research achievements have been acquired. Inspired by this idea, we present a novel TELM by extend the conventional ELM to the tensor space, and it can regard the image as a tensor to execute the recognition task.

3.3.1. ELM

Given a training set with $N$ different training samples $(x_j,t_j)\in R^d\times R^m,(j=1,2,\dots ,N)$, where $x_j={[x_{j1},x_{j2},\dots ,x_{jd}]}^T\in R^d$ denotes the $j\mathrm{th}$ training sample, $t_j={[t_{j1},t_{j2},\dots ,t_{jm}]}^T\in R^m$ represents the target of sample $x_j$. A classical SLFNs can be theoretically defined by:

$${\displaystyle \sum _{i=1}^L{\beta }_i{f}_i(x_j)}={\displaystyle \sum _{i=1}^L{\beta }_{i}f(a_i\cdot x_j+b_i)}=t_j,j=1,2,\dots ,N.$$

(18)

In this model, the hidden node number is $L$ and activation function is $f(x)$. $a_i={[a_{i1},a_{i2},\dots ,a_{id}]}^T$ denotes the input weight value which connects the input nodes with the $i\mathrm{th}$ hidden node. ${\beta }_i={[{\beta }_{i1},{\beta }_{i2},\dots ,{\beta }_{im}]}^T$ denotes the output weight value which connects the output nodes with the $i\mathrm{th}$ hidden node. $b_i$ denotes the bias for the $i\mathrm{th}$ hidden node. $a_i\cdot x_j$ means a dot product between $a_i$ and $x_j$. The classical SLFNs can approximate the given training samples set with the minimum residual.

Obviously, Equation (18) is a system of linear equations. By introducing the concept of matrix, we can rewrite it as follows:

$$H\beta =T,$$

(19)

where:

$$H={\left(\begin{array}{ccc}f(a_1\cdot x_1+b_1)& \cdots & f(a_L\cdot x_1+b_L)\\ ⋮& \ddots & ⋮\\ f(a_1\cdot x_N+b_1)& \cdots & f(a_L\cdot x_N+b_L)\end{array}\right)}_{N\times L},\beta ={\left(\begin{array}{c}{\beta }_1^T\\ ⋮\\ {\beta }_L^T\end{array}\right)}_{L\times m},T={\left(\begin{array}{c}{t}_1^T\\ ⋮\\ t_N^T\end{array}\right)}_{N\times m}.$$

Theorem 2. 

For a given normative SLFNs which possesses$L$hidden nodes and an activation function$f$, where$f:R\to R$is an infinitely differentiable function on the definition interval. Given a training set with$N$different samples$(x_j,t_j)$, where$x_j\in R^n$denotes the sample data and$t_j\in R^m$represents the target of$x_j$. For any randomly assigned weight$a_i$and bias$b_i$, the output matrix$H$of the hidden layer can be obtained by the pseudo-inverse and satisfies$\Vert H\beta -T\Vert =0$for probability one with respect to any continuous probability distribution.

For the proof of the Theorem 2 readers can refer to [45]. Based on this theory, ELM can be descripted as follows: With the initial weight vector and the biases of hidden layer nodes determined by random assignment, we can obtain the output matrix $H$ for the hidden layer based on the input samples. Therefore, we can transform the training procedure of ELM to a classical least squares problem of linear equations, i.e.,

$$\underset{\beta }{\min }{\Vert H\beta -T\Vert }^2.$$

(20)

We can obtain the least square solution of Equation (20) as follows:

$$\widehat{\beta }=H^{+}T.$$

(21)

where $H^{+}$ refers to the Moore-Penrose pseudo-inverse for matrix $H$.

3.3.2. Tensor Based ELM

Although the conventional ELM can deal well with one-dimensional signals, for two-dimensional images, it needs to be vectorized and solved in the one-dimensional space. However, in this transformation it is easy to lose the spatial structure information of the image. In order to solve this problem, we extend the conventional ELM to the tensor space and put forward a novel tensor- based ELM to deal with the high-dimensional signals.

In view of the high-dimensional characteristics of the palmprint image, we regard the fused image as a second-order tensor and classify it by the proposed TELM. In our method, the high order singular value decomposition (HOSVD) algorithm [54] is utilized to decompose the fused palmprint image and construct the input weight values of the TELM model.

Given an $M$ order tensor $F\in R^{I_1\times I_2\times \cdots \times I_M}$ and a matrix $U\in R^{J_m\times I_m}$, we define $B\in R^{I_1\times \cdots \times I_{m-1}\times J_m\times I_{m+1}\times \cdots \times I_M}$ as the $m\mathrm{th}$ modal product of $F$ and $U$, the elements of $B$ can be calculated by:

$${(F\times U_m)}_{i_1\cdots i_{m-1}{j}_m{i}_{m+1}\cdots i_M}={\displaystyle \sum _{i_n}{f}_{i_1\cdots i_{m-1}{i}_m{i}_{m+1}\cdots i_M}{u}_{j_m{i}_m}}.$$

(22)

so the m-th modal tensor product can be simply denoted by:

$$B=F\times U_m.$$

(23)

The HOSVD algorithm can be implemented by using the tensor product. Given an $M$ order tensor $F\in R^{I_1\times I_2\times \cdots \times I_M}$, we can use the tensor product to decompose $F$ in the following:

$$F=S\times U^{(1)}\times U^{(2)}\times \cdots \times U^{(M)},$$

(24)

where $S$ denotes an $M$ order tensor which is called as a core tensor, $U^{(1)}\in R^{I_1\times I_1},U^{(2)}\in R^{I_2\times I_2},\dots ,$ $U^{(M)}\in R^{I_M\times I_M}$ are unitary matrices and each column is corresponding to the orthogonal basis of unfolded matrices $F^{(1)},F^{(2)},\dots ,F^{(M)}$.

The low rank approximation of tensor $F$ can be calculated by HOSVD, i.e.,

$$F\approx S_{(q_1,q_2,\cdots ,q_M)}\times U_{q_1}^{(1)}\times U_{q_2}^{(2)}\times \cdots \times U_{q_M}^{(M)},$$

(25)

where $S\in R^{q_1\times q_2\times \cdots \times q_M}$ represents the principal component core tensor, $U_{q_i}^{(i)}$ represents the truncation matrix composed by the first $q_i$ columns of $U^{(i)}$, $i=1,2,\dots ,M$.

According to the above discussion, we summarize the detailed process of the tensor based ELM as follows: let $G_i\in R^{s\times t},(i=1,2,\dots ,N)$ be the $i\mathrm{th}$ fused training palmprint image, $t_i\in R^m,(i=1,2,\dots ,N)$ be the target of sample $G_i$. Denoted the training sample set as $G\in R^{s\times t\times N}$. Then the HOSVD algorithm utilized to decompose $G_i\in R^{s\times t},(i=1,2,\dots ,N)$ can be formulated as:

$$G_i\approx S_{(L_1,L_2)}\times U_{L_1}\times V_{L_2},$$

(26)

where $U_{L_1}=[u_1,u_2,\dots ,u_{L_1}]$ and $V_{L_2}=[v_1,v_2,\dots ,v_{L_2}]$ represent the truncation matrix with $L_1$ and $L_2$ columns, respectively. Then TELM can be defined as:

$${\displaystyle \sum _{l_1=1}^{L_1}{\displaystyle \sum _{l_2=1}^{L_2}g(G_j\times u_{l_1}\times v_{l_2}){\beta }_{l_2+(l_1-1)L_1}=t_j}},(j=1,2,\dots ,N),$$

(27)

where $L_1$ and $L_2$ denote the hidden layer node numbers along the tensor directions. In consequence, there are in total $L=L_1\times L_2$ hidden layer nodes. $u_{l_1}$ and $v_{l_2}$ denote the input weight vectors of the hidden layer along the tensor directions, respectively. ${\beta }_{l_2+(l_1-1)L_1}$ denotes the weight value between the output nodes and the $(l_2+(l_1-1)L_1)\mathrm{th}$ node in the hidden layer. Similar to ELM algorithm, $g(\cdot )$ denotes the activation function. Finally, the output weight $\beta$ can be obtained from Equation (27) by utilizing the least squares method.

4. Experiments

In this section, we evaluate the presented multispectral palmprint recognition algorithm on the benchmark available database offered by PolyU. Extensive experiments are implemented to demonstrate the effectiveness of the presented RL2SR method, adaptive fusion strategy and TELM. In the experiment of this paper, we use the fused palmprint image as the input of TELM classifier. In this section, we accomplish the experiments on a PC equipped with Windows 7, Intel Core i5-2320 CPU (3.0 GHz), and 6 GB RAM, and the algorithm is programmed using MATLAB 2017a.

4.1. The PolyU Multispectral Palmprint Database

The PolyU multispectral palmprint database was taken from 250 persons where the males are 195 and females are 55. The age of volunteers was mainly between 20 and 60 years old. In order to embody the differences of the acquired palmprint and make the palmprint images be various, the palmprint images were acquired in two separate phases. The time interval between the two phases was 5–15 days and each phase lasted about 9 days. In each phase, both hands of the volunteers were acquired six times respectively under the condition of four different spectra: Blue (470 nm), Green (525 nm), Red (660 nm) and NIR (880 nm). For each spectrum, 500 different palmprints were acquired from the 250 volunteers in the two phases. Therefore, the database contains 6000 palmprint images under each spectrum. That is, the multispectral palmprint database contains 6000 × 4 = 24,000 images in total. Reference [46] provided the ROI extraction process from the acquired multispectral palmprint images and established the database namely PolyU multispectral palmprint database (see Figure 4).

Figure 5 illustrates some images in the multispectral palmprint database. The images in the rows 1–4 are acquired under the Blue, Green, Red and NIR spectra, respectively. Every column is from the same class. In practice, the acquirement process is easily contaminated by various noises. To simulate this, the white Gaussian noise and salt & pepper noise are added into the images and the recognition experiments are implemented, respectively.

Figure 6 displays some multispectral palmprint images contaminated by different noises. Figure 6a shows the images contaminated by white Gaussian noise. Here, the mean is 0 and the standard deviation is 25. Meanwhile, Figure 6b shows the images contaminated by 50% salt & pepper noise. Rows 1–4 of Figure 6 exhibit the noisy palmprint images under the Blue, Green, Red and NIR spectra, respectively.

4.2. Parameter Selection

4.2.1. Selection of $\mu$ and $\delta$ for Residual Function

Now, let’s discuss the selection of parameters $\mu$ and $\delta$ for the residual function in Equation (7). It can be seen from Equation (7) that $\omega (e_k^l)\to \exp (\mu \delta )/(1+\exp (\mu \delta ))$ when $e_k^l\to 0$. Similarly, when $e_k^l\to \infty$, $\omega (e_k^l)=\exp (\mu \delta )/(\exp (\mu {(e_k^l)}^2)+\exp (\mu \delta ))\to 0$. In order to make $\omega$ belong to (0, 1), set the product $\mu \delta$ to be large enough, then $\omega (e_k^l)\approx \exp (\mu \delta )/\exp (\mu \delta )=1$. For simplicity, we denote $T=\mu \delta$. Since $e^7>1000$, in order to meet $\omega (e_k^l)\to 1$ when $e_k^l\to 0$, set $T=\mu \delta >7$. From Equation (7), $\omega (e_k^l)=1/2$, when $\delta ={(e_k^l)}^2$, so the parameter $\delta$ determines the boundary point position of the residual function value. That is to say, $\delta$ is determined when the weight $\omega$ will pass through 0.5. For the sake of enhancing the robustness of the model for the outlier or noise contamination efficiently, a novel method of selecting the parameter $\delta$ is presented as follows. Firstly, vectorize the square of the error ${(e_k^l)}^2$ and denote it as ${\overline{e}}^l={[{(e_1^l)}^2,{(e_2^l)}^2,\dots ,{(e_d^l)}^2]}^T$, then arrange this vector’s elements in descending order and denote the new vector by ${\tilde{e}}^l$. By denoting its maximum element as $M$ and the minimum element as $m$, set ${\tau }_1=(1-\theta )m+\theta M$, where $\theta$ is a constant and $\theta \in [0.6,0.8]$. Since the dimension of ${\tilde{e}}^l$ is $d$, suppose that $s$ is the nearest integer to $\theta d$ and the $s\mathrm{th}$ biggest element of ${\tilde{e}}^l$ is selected as ${\tau }_2$. Finally, let $\delta =({\tau }_1+{\tau }_2)/2$. Once $\delta$ is selected, parameter $\mu$ can be calculated by $\mu =T/\delta$. In our experiments, select the constant T = 8.

4.2.2. Selection of the Hidden Node Numbers Along the Directions of TELM

To evaluate the effect of the hidden node numbers along the directions of TELM, the experiments are implemented by setting the hidden node numbers varying from 1 to 20 under the cases of noise-free and different noise contaminations. The recognition performance is illustrated in Figure 7, Figure 8 and Figure 9. At the same time, Figure 7 and Figure 8 illustrate that our algorithm could converge rapidly with the increase of hidden node numbers. Obviously, when the hidden node numbers are both greater than 7, our algorithm achieves a perfect performance. From Figure 9, although the convergence performance is inferior to the noise-free case, our algorithm can still obtain better convergence speed. As a consequence, the appropriate hidden node numbers can be selected according to the above analysis. For simplicity, the hidden node numbers in our experiments are set as $L_1=L_2=10$.

4.3. Experiment Results and Analysis

In this subsection, the experiments are implemented to validate the efficiency of our presented algorithm from the aspects of sparse representation, fusion strategy, classification approach and the overall algorithm. For the sake of demonstrating the robustness of the presented RL2SR model, we accomplish the experiments compared with several different models, such as SRC, CRC and DSRM. The recognition rates are shown in

Table 2. Recognition rates for different representation methods.

Representation Method	Recognition Rate (%)
	Noise-Free	White Gaussian Noise	Salt & Pepper Noise
SRC	99.64	97.84	94.28
CRC	99.44	98.76	96.68
DSRM	97.96	96.68	96.28
RL2SR	99.68	99.20	97.24

.

From Table 2, it is easy to discover that each algorithm achieves the highest and the lowest recognition rates under the cases of noise-free and salt & pepper noise contamination, respectively. Since our proposed adaptive weighted fusion process approximates a spatial smoothing filtering, the decrease of recognition rate under the white Gaussian noise contamination is not obvious. Furthermore, by using our RL2SR coefficient for fusion, the recognition rates achieve 99.68%, 99.20% and 97.24%, which are 1.72%, 2.52 and 2.96% higher than DSRM under the cases of noise free, white Gaussian noise and SRC in the case of salt & pepper noise contamination, respectively. This indicates that our RL2SR is robust to different noises, which can improve the discriminant competency and increase the recognition rate of the fusion image.

To evaluate the efficiency of the presented adaptive fusion strategy, some comparison fusion experiments (i.e., the sum and min-max fusion strategy) are simulated and the recognition performance is listed in

Table 3. Recognition rates for different fusion methods when the training sample number per class varies from 2 to 4 under the cases of noise-free and different noise contaminations.

Fusion Method	Noise Contamination Case	Recognition Rate (%)
		2	3	4
Sum fusion	Noise-free	97.50	99.56	99.90
	White Gaussian noise	96.70	99.44	99.65
	Salt & pepper noise	89.63	96.56	98.55
Min-max fusion	Noise-free	92.83	97.68	99.25
	White Gaussian noise	92.67	97.44	99.20
	Salt & pepper noise	72.53	82.16	85.85
Our adaptive fusion	Noise-free	97.73	99.68	100.00
	White Gaussian noise	97.47	99.20	99.95
	Salt & pepper noise	92.27	97.24	99.05

. In this experiment, the training sample number of each class varies from 2 to 4.

Table 3 illustrates that the recognition accuracies under different fusion strategies increases with the training sample number. In particularly, our presented fusion strategy achieves the highest recognition accuracy of 100%, 99.95% and 99.05% when we set the number of training samples as 4. Even when the training sample number declines to 2, our approach achieves an accuracy of 92.27% which is 19.74% higher than the min-max fusion strategy in the case of salt & pepper noise contamination (72.53%). This implies that our fusion strategy has the strongest robustness compared with the sum and min-max fusion methods.

To demonstrate the classification efficiency of the presented TELM, we accomplish the experiments compared with some other classifiers, such as NN, KNN, ELM, MPELM and RELM. For these comparison classifiers, we vectorize the fused image and take this vector as the input. For each classifier, 3–6 training samples are selected to complete the recognition experiments and the classification accuracy curves are plotted in Figure 10.

The curves in Figure 10 indicate that when the training sample number is greater than or equal to 4, the recognition rates of all the algorithms achieve excellent performance. The experimental results also show that, in the case of noise-free, the recognition rate of our proposed TELM algorithm gradually increases with the number of training samples. On the other hand, our TELM achieves higher recognition rates than the other algorithms. Although the improvement is not significant because the recognition rate is much approximate or even reaches to 100%. From the above analysis, it is easy to observe that the presented TELM algorithm can achieve efficient recognition performance and has strong stability compared with the other classifiers. Furthermore, more simulation experiments are implemented with the multispectral palmprint database when it is contaminated by the aforementioned noise. The recognition performances are illustrated in

Table 4. Recognition rates for different classifiers under the cases of noise-free and noise contaminations.

Classifiers	Recognition Rate (%)
	Noise-Free	White Gaussian Noise	Salt & Pepper Noise
NN	99.24	96.48	44.24
KNN	97.12	93.32	38.92
ELM	99.18	99.16	95.55
MPELM	99.00	98.80	95.60
RELM	99.41	98.96	96.07
TELM	99.68	99.20	97.24

.

It is observed from Table 4 that, in the case of white Gaussian noise contamination, the recognition rate of TELM outperforms the other classifiers. Meanwhile, the recognition accuracy of the presented TELM is remarkably higher than the other methods under the case of salt & pepper noise contamination. In consideration of the pulse characteristic of the salt & pepper noise, it impacts remarkably on the distance measurement between different samples. When the testing samples are contaminated by salt & pepper noise, the recognition accuracy of KNN method achieves 38.92%, which is significantly lower than our TELM algorithms (97.24%). Since the proposed TELM abandons the eigenvectors corresponding to the smaller eigenvalues which have the higher correlation to the noise contamination, and retains the principal components corresponding to the major eigenvalues, TELM has the ability of noise reduction and the better discrimination ability. The experimental results in Table 4 also validate that our algorithm can achieve the higher recognition rate and possess the stronger robustness to noise contamination compared with the other classifiers.

To further validate the robustness of the proposed TELM algorithm, we add different degrees of salt & pepper noise to the testing sample and implement the recognition experiment. Figure 11 shows some noisy multispectral palmprint images contaminated by salt & pepper noise with 10% to 80% percentages. Figure 11a is the original images under different spectra. Figure 11b–i are the noisy contaminated images under different spectra when the degree of salt & pepper noise varies from 10% to 80%.

Figure 12 illustrates the recognition rate curves of our TELM algorithm and some of the aforementioned comparison classifiers. It is easy to find that the recognition rate curves of ELM MPELM, RELM and our algorithm drop significantly when the percentage of noise contamination is greater than 60%. Particularly, the recognition rate curves of NN and KNN methods are obviously lower than the other algorithms when the palmprint image is contaminated by more than 20% salt & pepper noise. That is to say, the accuracy curves of NN and KNN have the fast decline. The experiment result curves mean that our proposed TELM algorithm outperforms the comparison classifiers with different percentages of noise contamination and possesses stronger robustness.

Table 5. Classification time for different classifiers.

Classifiers	Classify Time (s)
NN	7.76
KNN	5.17
ELM	1.51
MPELM	1.82
RELM	1.67
TELM	1.59

illustrates the average classification times of the aforementioned classifiers on the whole database. Although our TELM classifier is slower than the ELM method, the difference (i.e., 0.08 s) is very small. Moreover, it is distinctly faster than NN, KNN, MPELM and RELM classifiers. Especially, the classification time of NN is about five times that of our TELM. In additional, the above experiment results demonstrate that the recognition performance of our classifier significantly exceeds the NN, KNN, ELM, MPELM and RELM classifiers. This validates the recognition ability and efficiency of our algorithm.

Table 6. Recognition rates for our RL2SR-TELM with different spectral combinations under the cases of noise-free and different noise contaminations.

Spectral Combination	Recognition Rate (%)
	Noise-Free	White Gaussian Noise	Salt & Pepper Noise
Blue	99.55	98.65	80.90
Green	99.50	99.25	87.65
Red	99.45	99.15	83.10
NIR	98.65	94.50	76.75
Blue, Green	100.00	99.80	95.80
Blue, Red	99.95	99.80	93.30
Blue, NIR	100.00	99.85	90.70
Green, Red	99.75	99.50	96.15
Green, NIR	100.00	99.80	95.80
Red, NIR	99.90	99.90	96.60
Blue, Green, Red	100.00	99.85	98.65
Blue, Green, NIR	100.00	99.90	97.60
Blue, Red, NIR	100.00	99.90	97.15
Green, Red, NIR	99.95	99.85	98.35
Blue, Green, Red, NIR	100.00	99.95	99.05

lists the recognition rates of our RL2SR-TELM algorithm with different spectral combinations. This experiment is implemented under the cases of noise-free, white Gaussian noise and 50% salt & pepper noise contamination and the training sample number per class is 4.

Table 6 summarizes the excellent performance of our presented algorithm in the cases of noise-free and white Gaussian noise contamination. In the noise-free case, the recognition accuracy achieves 100% for most of the spectral combinations. Even when the sample is contaminated by white Gaussian noise, our algorithm achieves the accuracy of more than 99.50% for all of the spectral combinations and 99.95% under the combination of Blue, Green, Red and NIR spectra. When the testing sample is contaminated by salt & pepper noise, the recognition rate declines significantly and achieves the lowest recognition rate 76.75% under the NIR spectrum. At the meantime, our RL2SR-TELM algorithm achieves an recognition performance under the combination of Blue, Green, Red and NIR spectra in the noise-free , white Gaussian noise and salt & pepper noise contamination cases, i.e., 100%, 99.95% and 99.05%, respectively. This indicates that our proposed RL2SR-TELM algorithm has excellent robustness to noise pollution.

Table 7. Recognition rates for different multispectral palmprint recognition algorithms in the case of noise-free.

Algorithm	Recognition Rate (%) for Different Training Sample Number
	3	4	5	6
Deep scattering network method [18]	100	100	100	100
Texture feature based method [42]	-	99.96	99.99	100
DCT-based features method [43]	-	99.97	100	100
Our proposed RL2SR-TELM	99.68	100	100	100

illustrates the recognition rates of our RL2SR-TELM algorithm compared with some state-of-the art palmprint recognition methods, such as deep scattering network method [18], texture feature-based method [42], and DCT-based features method [43] etc. It is easy to find that in the case of different training samples, our algorithm achieves an excellent recognition performance. Although the recognition accuracy of our algorithm is 0.32% lower than the deep scattering network method when the training sample number is three, and it is higher than the texture feature based method and DCT-based features method when the training sample number is four. Particularly, the recognition rate of our proposed algorithm reaches 100% when the number of training samples is greater than four.

Table 8. Recognition rates for our RL2SR-TELM and some other multispectral palmprint recognition algorithms.

Algorithm	Recognition Rate (%)
	Noise-Free	White Gaussian Noise	Salt & Pepper Noise
Matching score-level fusion by LOC [41]	99.43	99.23	96.48
DST-MPELM [39]	99.47	98.30	89.98
AE-RELM [38]	99.16	98.48	95.76
QPCA + QDWT [37]	98.83	93.33	90.16
Image-level fusion by DWT [32]	99.03	96.23	82.75
Our proposed RL2SR-TELM	99.68	99.20	97.24

lists the recognition rates of our RL2SR-TELM algorithm comparing with some state-of-the-art multispectral palmprint recognition algorithms, such as matching score-level fusion by LOC method, DST-MPELM method, AE-RELM method, quaternion PCA using quaternion DWT method and image-level fusion by DWT method. In this experiment, we choose three samples per class to constitute the training set. The experimental results in Table 8 illustrate that our proposed algorithm can achieve an excellent recognition accuracy in the cases of both the noise-free (99.68%) and various noise contaminations (99.20% and 97.24%, respectively).

When the sample is contaminated by salt & pepper noise, the presented algorithm has more obvious advantages, which an accuracy that is respectively 0.76%, 7.26%, 1.48%, 7.08% and 14.49% higher than that of the other comparison algorithms.

Table 9. Time cost of our RL2SR-TELM algorithm.

Procedure	RL2SR and Adaptive Fusion	TELM	Total Time
Average time (s)	0.10892	0.00053	0.10945

demonstrates the time cost of our proposed RL2SR-TELM multispectral recognition algorithm for each test sample. It is easy to find that our RL2SR-TELM algorithm takes about 0.10945 s for a test sample recognition task.

To further demonstrate the performance of our presented RL2SR-TELM method, in the case of salt & pepper noise contamination, we plot the cumulative match characteristic (CMC) curves generated by our RL2SR-TELM method and the aforementioned comparison methods. Figure 13 shows the CMC curves.

From Figure 13, it is easy to find that our presented RL2SR-TELM method has the highest rank-1 recognition accuracy. Meanwhile, the cumulative match characteristic curve of our algorithm is mostly close to the upper left corner of the coordinate system comparing with the comparison multispectral palmprint recognition approaches which means that it has the rapidest convergence speed. This implies that our algorithm outperforms the others in recognition accuracy and noise robustness, and it is quite consistent with the aforementioned experiment results and analysis.

5. Conclusions

In this paper, a novel RL2SR-TELM algorithm is presented to implement multispectral palmprint recognition. Since the L2 regularization term is employed, the regularization optimal objective function is convex and a closed solution can be efficiently obtained. In addition, a new measurement, namely WSCCI, and an adaptive fusion framework are proposed to construct the fused multispectral palmprint images. For the classification task, we extend the conventional extreme leaning machine to the tensor domain and present a TELM algorithm. It deals with the palmprint image in two-dimensional space directly and makes the best use of its spatial structure to enhance the classification ability. Extensive experiments on PolyU multispectral palmprint database confirm the strong robustness, excellent recognition accuracy and high efficiency of our proposed algorithm.