1. Introduction
Biometric identification systems are widely used in security-critical systems [
1,
2,
3] and man-machine interfaces (MMI) [
4]. They are used for personnel control and criminal monitoring/detection in areas such as military, hospital, airport, education [
3,
5,
6,
7]. In systems used for personnel control, persons are registered, and recognition is performed with the perception of biometric data with certain standards. However, people tend to hide themselves with criminal identification. This tendency complicates the task of criminal identification [
5,
6,
7]. In such cases, recognition can be achieved with camera images taken in the outdoors, where the targeted person is unaware. Thus, face recognition techniques are widely used in criminal identification applications [
8]. In facial recognition systems, users must adhere to the rules as they enter the system with face data. However, in some public areas, such as metros and airports, people tend to conceal themselves in recognition systems. Thus, the performance of facial recognition applications could be decreased due to various factors such as hairstyle, make-up, accessories, and esthetics [
9,
10,
11]. For these reasons, face recognition applications based on salient features have become widespread in the literature [
12,
13].
There are many studies in the literature on facial recognition [
14,
15]. A descriptor called Local Gradient Hexa Pattern (LGHP) was proposed by Chakraborty et al. [
16]. This descriptor provided effective facial features with binary micro-patterns. The performance of this study was evaluated using the widely used Cropped Extended Yale B, CMU-PIE, color-FERET, LFW, and Ghallager databases. The performance of the descriptor defined for use in face recognition applications by Zhang et al. [
17]. Their proposed descriptor was the local derivative pattern. The proposed descriptor generated features based on local derivative variations. This descriptor initially used the Local Binary Pattern. In this study, FERET, CAS-PEAL, Carnegie Mellon University-Pose, Illumination, and Expression (PIE) database (CMU-PIE), Extended Yale B, and FRGC databases were used for experimental results. The results were evaluated on gray-level and Gabor feature images. A new pattern encoding the direction information of the face image was proposed by Pillai et al. [
18], to generate a face feature code. In their study, the face images were separated, and the facial features were considered with respect to their diagonal neighbors. Then, all features were concatenated to obtain the final features. The proposed descriptor was evaluated using FERET, Extended YALE B, ORL, and LFW-a databases. The study was compared with methods commonly used in the literature, such as PCA, local binary pattern (LBP), and local directional number (LDN). A method based on the Chain Code-Based Local Descriptor (CCBLD) for face recognition was presented by Karczmareka et al. [
19]. In this study, the feature set was increased by dividing the pixels of the image into blocks. The experimental results were evaluated in AT&T, FERET, CAS-PEAL, the face and gesture recognition network (FG-NET), Essex Collection of Facial Images, and Yale face database. The experimental results were presented in terms of recognition rate, computing time, noise/occlusion presence, and similarity/dissimilarity measures.
An extended LBP-based method was proposed by Li et al. [
20]. Pixel differences based on angular and radial differences were used in this study. Yale B, FERET, CAS-PEAL-R1 databases were used by Li et al. [
20] for experimental results. Recognition rate was selected as a performance criterion. Gradient-based local descriptors were presented by Su and Yang [
21]. The histogram of the gradient phases (HGP) was durable to local geometric and photometric errors. In the study, the success of descriptors was assessed using Yale B and FERET databases. A new descriptor named local edge direction and texture descriptor (LEDTD) was proposed by Li et al. [
22]. In the study, the durability of the proposed descriptor was evaluated. The performance of the descriptor was evaluated when four well-known databases (AR, Yale B, CMU PIE, FERET) were used. The results show that the proposed descriptor was more successful than the other descriptors. A new descriptor for the localized ternary pattern for facial expression recognition was presented by Ryu et al. [
23]. In this study, a two-level grid approach was used to extract the face structure. The obtained outcomes by Ryu et al. [
23] showed that the proposed method was sensitive to facial expressions. Recognition rate and accuracy parameters were used to measure performance in selected databases (Extended Cohn-Kanade, JAFFE, MMI, CMU-PIE, GEMEP-FERA, and BU-3DFE).
In this study, two novel image descriptors and a novel domain are presented. The main aim of this study is to achieve a high classification ability for facial images. Two descriptors are defined in this study with a novel image transformation. The proposed descriptors are compared to 10 commonly used descriptors in the literature. Moreover, a Discrete Wavelet Transform (DWT)-fuzzy set-based domain (FWD) is proposed to increase the performance of 12 descriptors examined in the model. The performance of the proposed method is analyzed based on the accuracy rate. The abbreviations of the used methods are listed in
Table 1.
Major Contribution
In this paper, a new Fuzzy wavelet-based face recognition architecture is proposed. This architecture is applied to widely used image descriptors. Moreover, two novel graph structures, Vertical Local Cross Pattern (VLCP) and Horizontal Local Cross Pattern (HLCP), are presented. This method generally consists of two main phases: feature extraction and classification. In the feature extraction phase, fuzzy wavelet domain (FWD) and image descriptors were used to extract salient features. Four classifiers were used in the classification phase. The technical contributions of the current study are given below.
Graph theory is a mathematical structure that has been used in many disciplines to solve different real-world problems. In the current study, using graphs, patterns were created for textural feature extraction, and these features were used for texture and face recognition. In this method, two novel graph-based image descriptors are presented to extract salient features for face recognition.
FWD is created using DWT and fuzzy set theory. The main aim of the FWD method is to gain a high face recognition capability.
In the feature extraction phase, 12 image descriptors, 4 classifiers (Linear Discriminant Analysis (LDA), QDA, SVM, and KNN), and 2 domains (pixel and fuzzy wavelet) were utilized in the classification phase. Therefore, 96 novel face recognition settings (12 × 4 × 2) are proposed by using the proposed architecture, and these methods are compared in this paper.
A large benchmark set is given in the experimental results to understand the effects of the image descriptors on facial image recognition with variable domains and classifiers.
The rest of this study is structured as follows. Our proposed method for face recognition is elaborated in
Section 2. The obtained results are presented in
Section 3. In
Section 4, discussions are presented. Finally, we conclude the study in
Section 5.
2. The Proposed Method
In this study, a novel face recognition architecture-based on wavelet [
24,
25,
26], fuzzy logic method [
12,
27], and Local Cross Pattern (LCP) is proposed. There are two major phases: feature extraction and classification phases. In the feature extraction phase, LL, LH, HL, and HH bands were obtained by applying 2D-DWT onto the image. The low-low (LL) is an approximate band that is robust against compression attacks because many methods have used DWT to achieve robustness against JPEG compression attacks. Then, A, B, and C clusters were obtained by using the proposed triangular-based fuzzy method. It should be expressed that the triangle-based fuzzy method is a modified version of Neutrosophy theory [
28]. Then, the proposed LCPs, LBP, and the other 9 graph structures were applied to the B cluster. The histogram of the B cluster utilizes descriptors as a feature set with a dimension of 256. In the classification phase, experimental results are presented using SVM [
29], KNN [
30,
31], LDA [
32,
33], and QDA [
34,
35,
36]. The block diagram of the proposed method is given in
Figure 1.
In the following subsections, the components of the proposed architecture are described in more detail.
2.1. The Proposed Fuzzy Set
In this study, a fuzzy set like Neutrosophy is proposed. In the proposed method, three clusters are defined. Hence, membership degrees of the A, B, and C clusters are calculated using the median filter and triangular clusters [
12,
27]. The proposed fuzzy set is given in
Figure 2.
The main aim of the FWD is to propose a novel basic and effective domain using both fuzzy and wavelet methods. According to the literature, Neutrosophy and fuzzy-based Neutrosophy-like transformations are very effective methods for image and signal processing. In this article, a simple and effective fuzzy-based transformation is proposed. A novel Neutrosophy like transformation is also proposed. Firstly, A, B, and C membership degrees of the images are calculated. To calculate these membership degrees, triangular fuzzy sets are. These sets are shown in
Figure 2. The mathematical notations of the membership calculations of the proposed triangular fuzzy-based transformation are given in Equations (1)–(4).
where
w represents the size of overlapping blocks and
represents the 2D median filter function.
2.2. Local Cross Pattern (LCP)
In this article, a novel image descriptor is presented for face recognition. This descriptor is called LCP. The main motivation of the LCP is to use a novel graph-based pattern for distinctive feature extraction. The widely used image descriptor is the LBP that extracts features using neighbor pixels. However, the proposed LBP uses cross pixels and extracts features both vertically and horizontally. The proposed LCP is also extendable, because the users can use any block size. We can also define only the minimum sized block for LCP. In the study, the LCP descriptor is analyzed vertically and horizontally. The representation of these two different forms are shown in
Figure 3a,b.
To apply the LCP descriptor, in the first step, the image is divided into 4 × 2 and 2 × 4 areas in the vertical and horizontal versions, respectively. The feature sets of the VLCP and HLCP descriptors are obtained by using the following equations:
where
is the signum function, the
set represents extracted bits,
,
, and
are the
ith and
jth pixels of the original HLCP and VLCP images, respectively.
The histogram and images obtained from the VLCP and HLCP descriptors are given in
Figure 4.
2.3. Steps of the Proposed Face Recognition Architecture
This study provides a novel face recognition method based on fuzzy wavelets and two new local descriptors. The proposed local descriptors are used in the fuzzy wavelet method. The steps of the proposed method are given as follows.
Step 1: Load the face image.
Step 2: Convert the RGB face image to gray.
where
is a gray image,
,
and
are the red, green, and blue channels of the face image, respectively. It should be noted if a face image is gray, this step is skipped.
Step 3: Apply 2D DWT to the gray image and obtain the LL sub-bands.
where
is a 2DDWT function with Haar filter,
are Low-Low, Low-High, High-Low and High-High subbands, respectively.
Step 4: Apply triangle fuzzy transform to the LL sub-band by using Equations (1)–(4) and obtain the B image.
Step 5: Use VLCP, HLCP, or other local descriptors.
where
is a function of VLCP and this function is defined in Equations (5)–(14),
is a function of HLCP, and this function is defined in Equations (15)–(24),
and
are the VLCP and HLCP applied images.
Step 6: Calculate histogram of these images. The histogram of these images is utilized as a feature.
Step 7: Classify face images using this feature. In this paper, four well-known algorithms, including LDA, QDA, SVM, and KNN classifiers, are used.
2.4. The Used Definitions for the Proposed Method
In recent years, machine learning methods have been very popular and effective in solving problems in various fields [
37,
38,
39,
40,
41,
42,
43,
44]. Thus, to classify the extracted features, the MATLAB 2016a classification learner toolbox was used. This study applies LDA, QDA, SVM, and KNN classifiers. The parameters of these classifiers are given in
Table 2. We used 10-fold cross validation to obtain the experimental results.
2.5. Databases
In this study, AT&T, CIE, Face 94, and FERET databases were used. These databases are widely used in many studies. The properties of the databases are given in
Table 3.
The performance is analyzed for the LDA, QDA, SVM, and KNN classifiers using (10) widely used descriptors (LBP [
50], local graph structure (LGS) [
51], symmetric local structure (SLGS) [
52], vertical local structure (VLGS) [
53], vertical symmetric local structure VSLGS [
53], zigzag horizontal local structure (ZHLGS) [
53], zigzag horizontal middle local structure (ZHMLGS) [
53], zigzag vertical local structure ZVLGS [
53], zigzag vertical middle local structure (ZVMLGS) [
53], LELGS (logically extended local graph structure) [
53]) and the proposed VLGS, and HLGS image descriptors. Here, these descriptors are enumerated using classical descriptors (from 1 to 10) and the proposed descriptors (11,12). Then, fuzzy wavelet FWD-LBP, FWD-LGS, FWD-SLGS, FWD-VLGS, FWD-VSLGS, FWD-ZHLGS, FWD-ZHMLGS, FWD-ZVLGS, FWD-ZHMLGS, FWD-LELGS (logically extended local graph structure), FWD-VLGS, and FWD-HLGS descriptors are presented and enumerated using (13)–(24) in this paper.
4. Discussion
In the experiments, 12 (descriptor) × 4 (classifier) × 2 (domain) = 96 methods have been compared and a large benchmark set has been obtained. The comprehensive results are given in this section.
According to the obtained results, the SVM classifier performed better than other classifiers applied in this study.
In
Figure 5, the average success of classifiers for databases is presented for each descriptor and domain using boxplot analysis. The bold blue lines and blue boxes represent results of the fuzzy wavelet and pixel domains, respectively. The boxplot analysis gives the minimum, maximum, and average values. To better explain the results, boxplot analysis is used in this section.
The proposed architecture can be programmed and as a result can be used as a practical model. The results show that the proposed method can be successfully used in face recognition applications. Therefore, it can be used to solve real world face recognition problems.
Table 7 clearly illustrated that the proposed descriptors achieved satisfactory results. For more clarity on the effectiveness of the proposed methods, the obtained confusion matrixes are given in
Figure 6.
As can be seen from confusion matrix of the proposed method, a classification accuracy of 100.0% is achieved for the four datasets used in this study. According to the confusion matrix of the AT&T data set, higher errors occurred in the first and ninth classes. As shown in the confusion matrix of the FERET, the worst classification accuracy rate was 66.67% for each class.
Briefly, two novel descriptors are proposed to extract features from the facial images. In addition, we proposed FWD as a novel domain. FWD is an alternative transformation to Neutrosophy. The first application of the FWD approach is applied to four widely used face images datasets. To show the impact of the FWD approach, our proposed descriptors and several widely used descriptors are used. The positive effects of the FWD for face recognition are demonstrated in
Table 4,
Table 5,
Table 6 and
Table 7. Also, these tables proved the success of the proposed VLCP and HLCP. Especially, we used classification accuracy to test the proposed methods. This is an effective performance measuring parameter because the used datasets are homogenous. The best classifier is SVM, because the best results are generally obtained using SVM. After SVM, KNN gained the highest performance rates. These results clearly illustrated that the proposed descriptors and FWD can gain high success rates in larger facial image datasets. The advantages and novelties of this paper are given below.
Two effective novel graph-based patterns called VLCP and HLCP are presented.
A novel transformation (FWD) is presented, and higher classification accuracies are obtained by using this domain.
96 facial image recognition methods are presented, and significant benchmark results are given.
Lightweight methods are presented, because the computational complexity of the proposed methods is calculated as , where n is size of the image.
Cognitive methods are proposed.
The presented methods can be applied to texture and texture images. As can be seen from the literature, many texture recognition problems have been solved using descriptors.
It should be noted that the demerit of this paper is its use of small face image datasets.
5. Conclusions
In this study, a new descriptor is defined for face recognition applications. The proposed descriptor is analyzed in vertical and horizontal forms. This descriptor is called LCP. By using LBP, graph structures, and the proposed LCPs, 96 methods were applied. Moreover, a novel image domain, which was created by using DWT and triangle fuzzy sets, is proposed. The proposed domain is called Fuzzy Wavelet (FW). In the classification phase, LDA, QDA, SVM, and KNN were utilized as classifiers. By using 12 descriptors, 4 classifiers, and 2 domains, 96 facial image recognition methods were obtained. To evaluate this method and obtain benchmarks, widely used face image datasets (AT&T, CIE, Face94 and FERET) were used. The classification accuracy results of these methods are given in the experiments. Furthermore, the presented descriptors and fuzzy wavelet domain were successful in face recognition.
In our future works, novel lightweight face recognition applications and devices can be developed using the proposed method. Moreover, novel deep neural networks are proposed using FWD and LCP descriptors. Our future intension is to propose one- and two-dimensional VHLCP and ternary VHLCP to classify textures and signals.