Discriminative Local Feature for Hyperspectral Hand Biometrics by Adjusting Image Acutance

Featured Application

Hyperspectral hand biometrics, which combines hyperspectral techniques and hand traits is a potential application in high-security scenarios. Based on adjusting image acutance, the discriminative local features were extracted from hyperspectral dorsal hand vein images and palm vein images. In addition, this work lays the foundation for continuing studies in hyperspectral hand biometrics.

Abstract

Image acutance or edge contrast in an image plays a crucial role in hyperspectral hand biometrics, especially in the local feature representation phase. However, the study of acutance in this application has not received a lot of attention. Therefore, in this paper we propose that there is an optimal range of image acutance in hyperspectral hand biometrics. To locate this optimal range, a thresholded pixel-wise acutance value (TPAV) is firstly proposed to assess image acutance. Then, through convolving with Gaussian filters, a hyperspectral hand image was preprocessed to obtain different TPAVs. Afterwards, based on local feature representation, the nearest neighbor method was used for matching. The experiments were conducted on hyperspectral dorsal hand vein (HDHV) and hyperspectral palm vein (HPV) databases containing 53 bands. The results that achieved the best performance were those where image acutance was adjusted to the optimal range. On average, the samples with adjusted acutance compared to the original improved by a recognition rate (RR) of 29.5% and 45.7% for the HDHV and HPV datasets, respectively. Furthermore, our method was validated on the PolyU multispectral palm print database producing similar results to that of the hyperspectral. From this we can conclude that image acutance plays an important role in hyperspectral hand biometrics.

1. Introduction

Hand biometrics has been largely studied in the last few decades [1,2,3,4,5] because of its effectiveness in personal authentication. One of the most universal traits of a hand is the palmprint [6,7,8,9]. Palmprint biometrics, with the center of the palm not touching the capture device for identification, is less likely to be copied by others and is more hygienic [9]. Contrary to palmprint biometrics, hand vein biometrics, such as dorsal hand veins [10,11,12] and palm veins [13,14], are studied in security with advantages in liveness detection and anti-spoofing [12]. In the meantime, hyperspectral technology known from remote sensing [15,16,17,18], has been introduced in biometrics, where it is applied in high-security scenarios [18]. Therefore, hand biometrics when combined with hyperspectral technology, is a potential solution achieving better personal authentication [19,20].

Hyperspectral hand biometrics utilizes spectral information in a hand image. The epidermal and dermal layers of the skin on a hand constitute a scattering medium that contains various combinations of water, melanosomes, hemoglobin, bilirubin, beta-carotene, etc., which provide different absorption coefficients for an irradiated spectrum [19]. Little changes in the distribution of these layers and pigments of the skin induce significant changes in the skin’s spectral reflectance, thus generating a unique response for each person that is difficult to modify and counterfeit [20]. For hyperspectral imagery in hand biometrics in particular, each spectrum penetrates a different depth of the hand, indicating the spectral property and enabling the imaging of different surface characteristics of an individual’s hand. For example, near-infrared (NIR) can penetrate deeper than thermal infrared (8–12 um), making it useful for hyperspectral hand imagery [21]. Considering spectral information as a complement in hyperspectral hand biometrics, security issues can reach a higher level, which is more secure and better at anti-spoofing.

Hyperspectral hand biometrics is a new trend derived from single spectral and multispectral biometrics. Fei et al. [22] performed palmprint recognition on extensive single spectrum images, such as the PolyU, IITD, GPDS, and CASIA databases. Zhang et al. [23] first applied the combinations of different bands containing Blue, Green, Red, and NIR for multispectral palmprint verification, where its result was recently improved by Hong et al. [24]. Hyperspectral palm was studied by Guo et al. [25] to propose an anti-spoofing recognition system (prototype). The dorsal hand mainly concentrates on the vein structures, first discovered by Joe Rice, a senior Kodak engineer when he was designing an infrared barcode system [26]. Huang et al. [12] performed dorsal hand vein recognition based on an 850 nm NIR dorsal hand database, while other databases with the same band can be found in [27,28,29]. Chen et al. [30] applied hyperspectral techniques to select the best spectrum for the improvement of dorsal hand vein recognition, which showed potential in hyperspectral hand biometrics.

In hand biometrics, the local features play a crucial role for texture analysis [31,32]. As palmprint lines are the most significant features [7], Wu et al. [33] used the local line features for palmprint recognition. To explore more elaborate features, Zhang et al. [34] proposed a local CompCode (competitive code) method for online palmprint recognition, which was based on Gabor features. Later, this was developed into a discriminative and robust CompCode by Xu et al. [35]. Most recently, a novel double-layer local direction pattern extraction method for personal authentication was proposed by Fei et al. [6]. For dorsal hand vein recognition, Wang et al. [36] applied local SIFT (scale-invariant feature transform) features. Wang et al. [11] extracted LBP (local binary patterns) from a dorsal hand vein to build an automatic access control system. Based on the Gabor filter, Lee et al. [37] designed directional filter banks to draw local patterns for dorsal hand vein recognition.

For local feature extraction in hand biometrics, image quality is a make or break factor [38,39,40]. High-quality images collected from calibrated cameras contain much more detailed information for human perception [41]. However, Zhang et al. [42] found that the presentation of a local feature was more effective in biometrics after filtering the high-quality images to low quality. Image acutance, which can be referred to as the contrast of the edge and the background in an image evaluates the image quality. Zhang et al. [42] improved the recognition performance by converting high-quality, single-spectral images to low acutance images. That being said, to the best of our knowledge there is little to nothing in the literature that studies the properties of low-resolution hyperspectral hand biometric images. The low-resolution image mostly considered as a low-quality image can be easily imaged by a low-cost device, which can further promote the use of biometrics. Hyperspectral biometrics possess the properties of uniqueness, liveness detection, and anti-spoofing that are difficult to achieve by single-band spectral images.

Inspired by [42], in this paper we explore the performance of hyperspectral hand biometrics by filtering these images to have different acutance. The hypothesis we propose is that there exists an optimal range in image acutance for a set of hyperspectral hand images, and when this set is filtered in this range, the recognition performance will be improved. For this reason, thresholded pixel-wise acutance value (TPAV) is proposed to evaluate image acutance. By convolving with Gaussian filters, TPAV of the hyperspectral hand image changed. Local features were extracted from the changed TPAV’s corresponding database and used to implement identification. In particular, extensive experiments were conducted on HDHV (hyperspectral dorsal hand vein), HPV (hyperspectral palm vein), and MPP (multispectral palmprint) images. We found that the recognition performance can be improved by filtering the images to have different acutance, where the local features extracted are more discriminative after adjusting for image acutance.

In the rest of this paper, the image acutance adjustment is proposed in Section 2. Section 3 designs extensive experiments with an analysis of the results. Finally, the conclusions are drawn in Section 4.

2. Adjusting Image Acutance

Even though a camera can be calibrated to capture clear images based on human perception, the collected images may not achieve the best performance in digital image processing. In order to improve the effectiveness of hyperspectral hand biometrics, this section first proposes a method to evaluate image quality based on image acutance. The image acutance can be changed to different levels by having the image convolved with different Gaussian filters. Through the specific task of hyperspectral hand biometrics, the optimal range of image acutance can be found to achieve the best performance in identification. The distinction from general hyperspectral hand recognition and our proposed method is the image acutance adjusting phase (see Figure 1). Here, TPAV is applied to the extracted ROI (region of interest) image before feature extraction.

2.1. Assessing Image Acutance

Motivated by the thresholded gradient magnitude maximization denoted Tenengrad [43] and edge acutance value (EAV) in [44], a new method named thresholded pixel-wise acutance value (TPAV) is proposed to assess the image acutance in hyperspectral hand biometrics.

$$TPAV\left(I\right)=\frac{{\displaystyle \sum }g\left(x,y\right)}{C},T_1<g\left(x,y\right)<T_2$$

(1)

where

$$g\left(x,y\right)=\frac{{{\displaystyle \sum }}_i{{\displaystyle \sum }}_j\left|I\left(x,y\right)-I\left(x+i,y+j\right)\right|}{\sqrt{i^2+j^2}}$$

(2)

$$T_1=Min\left(g\left(x,y\right)\right)+a\ast \left(Max\left(g\left(x,y\right)\right)-Min\left(g\left(x,y\right)\right)\right)$$

(3)

$$T_2=Max\left(g\left(x,y\right)\right)-a\ast \left(Max\left(g\left(x,y\right)\right)-Min\left(g\left(x,y\right)\right)\right)$$

(4)

$C$ is the count of $g\left(x,y\right)$ whose value is in the range of $T_1\sim T_2$. $I\left(x,y\right)$ is the value of pixel $\left(x,y\right)$. $\left|i\right|\le 1,\left|j\right|\le 1,\left|i\right|+\left|j\right|>0$. The parameter $a$ is a small positive constant, which is set as 0.05 in the following experiments empirically. This means outliers in the image will not be counted when measuring the image acutance. TPAV both takes advantage of Tenengrad [43] and EAV [44] into consideration. Tenengrad sets a threshold for reducing the sensitivity of the algorithm to noise when it assesses the acutance of an image. However, only horizonal and vertical edges in an image are considered to count as acutance in Tenengrad. EAV applied an additional 45° and 145° directional edges in 8 neighbors of a pixel, which is more reasonable for acutance assessment [44]. To this end, TPAV combines the threshold and more directional edges to effectively assess the acutance of an image for local feature analysis. In the following Figure 2 the comparison of acutance evaluation values was demonstrated for a ROI hand image (from a single band (in a hyperspectral dorsal hand vein database) with both rotation and noise). When the image rotates, the acutance value of Tenengrad was dramatically changed (+40.4% from 6.992 to 9.816) due to the direction limitation (see Figure 2b). Also, the acutance value of EAV increased significantly (+90.5% from 12.214 to 23.265) due to noise (refer to Figure 2c). However, the TPAV did not change a great deal under rotation (−7.1% from 15.226 to 14.140) or with noise (+9.6% from 15.226 to 16.685), which validates its robustness in both rotation and noise.

2.2. Modified Image Acutance

Under normal circumstances the image directly captured by a device has clear acutance for human perception, while it may not be effective for a computer to process it for a specific vision task. To improve the task’s effectiveness, the captured image first needs to be processed. By convolving with 2-D (two-dimensional) Gaussian filters, the image obtains a modified acutance, which can be regarded as a pre-processing stage in local pattern analysis for hyperspectral hand biometrics. The 2-D Gaussian filter has two main parameters for image pre-processing: window size and variance. In the following experiments of this paper (refer to Section 3), the window size of the Gaussian filter is set as 5 × 5 empirically. By changing the variance with respect to δ, different acutance of the same image can be obtained.

$$G\left(x,y\right)=\frac{1}{2\pi {\delta }^2}{e}^{-\frac{x^2+y^2}{2{\delta }^2}}$$

(5)

After the image is filtered, the TPAV can be obtained again for each image with different δ. As an example, Figure 3 and Figure 4 show a decreasing acutance for a dorsal hand vein and a palm vein after filtering, respectively.

2.3. Determining an Optimal Range of Image Acutance

The average TPAV of all images in a database is defined to measure the acutance of the database,

$$\overline{TPAV}=\frac{{{\displaystyle \sum }}_{i=1}^{N}TPVA\left(I_i\right)}{N}$$

(6)

where $I_i$ is the $i\mathrm{th}$ image in the database which contains $N$ images.

A hypothesis can be made that there is an optimal range $\left[t_1,t_2\right]$ on the TPAV axis (see Figure 5). When the $\overline{TPAV}$ of a hand database is adjusted to this range, the performance of local pattern analysis will be improved. In order to find this optimal range, each image in the database will be filtered by a group of Gaussian filters with different δ in order to calculate the $\overline{TPAV}$ of each generated database and its recognition performance. The optimal range can be found when the recognition performance reaches its first three highest results with the corresponding Gaussian filters. The experiments validated that after each image in the hyperspectral hand database is filtered with a Gaussian filter, the recognition performance improves compared with the original database.

3. Experiments

In order to validate our proposed method, experiments were conducted on our HDHV and HPV datasets as well as the publicly available PolyU multispectral palmprint datasets [24] to demonstrate its generality. This section first introduces the details of the databases used. Then, according to the experimental settings identification was implemented to find an optimal acutance range. Based on the local features extracted from the modified acutance images, the mechanism of improved performance was analyzed. Finally, the computation time of every phase in the proposed method was evaluated.

3.1. Databases

The HDHV database consists of 120 individuals ranging from the ages of 20 to 50, where images were captured of their left hand. The HPV database in contrast involved 209 volunteers with respect to their left hand. All subjects gave their informed consent for inclusion before they participated in the study. The study was conducted in accordance with the Declaration of Helsinki and the protocol was approved by the Research Services and Knowledge Transfer Office of the University of Macau. The light source used in both databases including the visible and NIR covers a spectral wavelength from 520 nm to 1040 nm with an interval 10 nm, which means a total of 53 bands were captured. Each hand was imaged five times with a size of 501 × 501 at 96 dpi. These images were stored in a single channel bitmap with 8 bits per pixel format. Altogether, there were 31,800 images in the HDHV database (120 individuals × 5 images × 53 bands), and 55,385 images from the HPV database (209 volunteers × 5 samples × 53 bands). Figure 6 and Figure 7 depict some samples from the two hyperspectral hand databases.

To further confirm the effectiveness of the proposed method, the PolyU MPP (The Hong Kong Polytechnic University Multispectral Palm Print) database [24] was also experimented. This multispectral database consists of 500 different palms, where each palm was shot 6 times including four bands (Red, Green, Blue, and NIR) with the size of each ROI being 128 × 128. Only the first session of the database was involved in this paper. Therefore, there are 12,000 (500 × 6 × 4) images from the multispectral database. Figure 8 demonstrates some samples from the PolyU MPP database.

3.2. Experimental Settings

Identification experiments were conducted on every band of the hyperspectral/multispectral hand databases. The regional LBP [11] method was applied to the HDHV database for local feature extraction, while HPV and MPP used CompCode [35] for local feature representation. n samples from a total of M samples belonging to the same person’s hand in the database were selected for training with the others assigned as testing (n < M). Every band from the hyperspectral or multispectral databases were convolved with a Gaussian filter, obtaining a mean recognition rate (RR) from 100 combinations of random training-testing samples. This guarantees statistically significant results [45]. Each RR can be obtained from the following equation:

$$RR=\frac{NCTS}{NATS}\times 100\%$$

(7)

where NCTS and NATS are the number of correctly classified test samples and the number of all test samples, respectively. All experiments were completed on MATLAB R2018b running on a PC with 64-bit Windows 7, i7-6700 CPU (3.40 GHz), and 16 GB RAM. To fully test the hypothesis, deep features in some respects regarded as local feature representation were also experimented under the same settings mentioned above. In particular, pre-trained CNN models, which were well trained on the ImageNet dataset [46], were used to extract local deep features from the hand databases. The pre-trained VGG-16 having performed the best in hand recognition [47], was applied in our experiments. The first fully connected layer in VGG-16 was set as the feature representation with a 4096-dimensional vector.

3.3. Experimental Results

For the HDHV database to demonstrate the identification results, three samples from each hand were chosen for training, while the remainder were used for testing. The curve of each band in Figure 9 shows that the RR first increased and then decreased after being convolved with different Gaussian filters through different local feature representation methods. To demonstrate the connection between acutance and the performance, one single band was presented in Figure 10. The RR of the single band (810 nm) from the HDHV database steadily increases before reaching a peak and then starts to sharply decrease. Therefore, there exists an optimal range based on image acutance to obtain better performances than the original image. Here, the optimal acutance range is 4.103–6.755 for different feature extractors. For example, regional LBP where the highest RR can reach 0.9667 (see Figure 10a), achieves an improvement of 7.9% compared with the original image (RR = 0.8958).

For the HPV database, RR had the same tendency as the above (HDHV), which confirmed the hypothesis that an optimal range based on image acutance does exist (see Figure 11). With different feature descriptors, the optimal acutance range of a single band (840 nm for example in Figure 12) is 4.927–6.856, using CompCode where the highest RR can reach 0.9880 (see Figure 12a), which is an improvement of 21.8% compared with the original image (RR = 0.8110).

For the MPP database, the RR followed the same trends as the previous two databases. This shows that there is an optimal range based on image acutance in every spectrum with different feature representation methods (see Figure 13). For CompCode as an example (see Figure 14a), the optimal acutance range of the green spectrum is 11.549–14.257, where the highest RR reached 0.9993, which is an improvement of 7.2% compared with the original image (RR = 0.9320).

To draw conclusions more easily, the representative results of the best performing bands, in addition to the average results for all bands in each hyperspectral database are depicted in the

Table 1. Results on representative spectral from the hyperspectral databases.

Dataset	Local Pattern	Band	$\mathbf{Optimal}\mathbf{Range}({\mathit{t}}_1-{\mathit{t}}_2)$	RR 1 (In Optimal Range)	RR (Original Image)	Improvement
HDHV	Reginal LBP	890 nm	3.515–6.412	0.9833	0.9583	2.6%
		All	-	0.9540	0.7365	29.5%
	Deep feature	890 nm	3.515–6.412	0.9750	0.9542	2.2%
		All	-	0.8426	0.6154	36.9%
HPV	CompCode	900 nm	4.865–6.237	0.9928	0.7919	25.3%
		All	-	0.8964	0.6152	45.7%
	Deep feature	900 nm	4.865–6.237	0.9809	0.7895	24.2%
		All	-	0.7317	0.4414	65.7%

¹ The best result in the optimal range.

. From this table it can be seen that for every case, after adjusting image acutance (from the optimal range) the RR increases when compared to the original ROI image, no matter which local feature extractor was used.

3.4. Experimental Analysis

To investigate why hyperspectral hand databases performed better after acutance adjustment. We assumed that the distance of the local features for an individual’s same hand is decreased, while for different people their hand distance is increased when the acutance of the database is adjusted to the optimal range. To this end, one evaluation method which is inspired by the Fisher criterion [48], was used to measure the discrimination of the features extracted from the database (after acutance adjustment). The $r$ with respect to the ratio of within-class variance to between-class variance is introduced as follows:

$$r=\frac{1}{D}{\displaystyle \sum }_{d=1}^D{\left(\frac{v_w^d}{v_b^d}\right)}^2$$

(8)

where

$$v_w^d={{\displaystyle \sum }}_{k=1}^K{{\displaystyle \sum }}_{nϵC_k}{\left(x_n^d-m_k^d\right)}^2{m}_k^d=\frac{1}{N_k}{{\displaystyle \sum }}_{nϵC_k}{x}_n^d$$

(9)

$$v_b^d={{\displaystyle \sum }}_{k=1}^K{N}_k{\left(m_k^d-m^d\right)}^2{m}^d=\frac{1}{N}{{\displaystyle \sum }}_{n=1}^N{x}_n^d$$

(10)

The notations are detailed below:

$v_w^d$: the within-class variance for the $d_{th}$ feature of the $D$ features.

$v_b^d$: the between-class variance for the $d_{th}$ feature of the $D$ features.

$x_n^d$: the $d_{th}$ feature of the $n_{th}$ sample in the database containing $N$ samples.

$C_k$: the $k_{th}$ class in the database consisting of $K$ classes.

$N_k$: the number of samples belonging to the $k_{th}$ class.

The best performing band in the identification phase from the hyperspectral (HDHV—890 nm and HPV—900 nm) hand databases was chosen for analysis. Based on the local features extracted from these databases, the $r$ values were calculated via Equation (8), where a smaller $r$ value signifies a more discriminative local feature. Figure 15 depicts that after being convolved with different Gaussian filters, the $r$ value first decreased and then increased, which indicates that the local features have the most discrimination in the optimal acutance range obtained from the previous subsection (see

Table 2. $r$ values of single spectrum in optimal acutance range.

Spectrum	$\mathbf{Optimal}\mathbf{Range}({\mathit{t}}_1-{\mathit{t}}_2)$	$\mathit{r}{}^1\left(\mathbf{In}\mathbf{Optimal}\mathbf{Range}\right)$	$\mathit{r}$ (Original Image)
890 nm (HDHV)	3.515–6.412	0.1686	0.3052
900 nm (HPV)	4.865–6.237	0.2615	0.4344

¹ The lowest value in the optimal range.

). Therefore, by adjusting image acutance, the local feature from the hyperspectral and multispectral hand databases become discriminative resulting in a better performance.

3.5. Computation Time

The average computation time of each stage (in general hyperspectral/multispectral hand recognition) for a single image is shown in

Table 3. Computation time (in seconds) of every stage in general hyperspectral/multispectral hand recognition (sans ROI extraction).

Database	Acutance Adjusting	Feature Extraction	Featuring Matching	Total Time
HDHV	0.0241	0.0153 1	0.0108	0.0502
HPV	0.0225	0.0356 2	0.0135	0.0716
MPP	0.0221	0.0345 2	0.0166	0.0732

¹ Regional LBP method. ² CompCode method.

. Here, a ROI extraction time is not calculated since the proposed acutance adjustment is performed on an extracted ROI, which means the proposed method is performed after ROI extraction. As the proposed method of acutance adjustment adds only around 0.0229 s on average for the three databases, it is considered acceptable in real-world applications.

4. Conclusions

This paper presents an approach to improve hyperspectral hand recognition by adjusting image acutance. First, a thresholded pixel-wise acutance value (TPAV) for evaluating the image acutance was proposed. Next, Gaussian filters were applied to change the image acutance via image convolution, which acts as a preprocessing phase for discriminative local feature extraction. Finally, for each band in a hyperspectral hand database, the optimal acutance range can be determined based on TPAV. Experiments were extensively conducted on HDHV and HPV databases. The results validated the hypothesis that there exists an optimal acutance range for hyperspectral hand biometrics to reach its peak performance. To assess the generalization ability of our method, the PolyU multispectral palmprint database was experimented and confirmed the hypothesis as well. Even though on average it takes 0.0229 s for acutance adjustment (for a single image from the three datasets), the final result afterwards is significantly improved compared to the original (with no acutance adjustment).

In the future, we will investigate the properties of each band in hyperspectral hand biometrics, in order to design a more powerful local feature extraction method to achieve more discriminative feature representation. Besides this, we will study how best to combine deep learning or local features from different bands to achieve a better biometrics system.