*3.3. Summary of Local Approaches*

Table 1 summarizes the local approaches that we presented in this section. Various techniques are introduced to locate and to identify the human faces based on some regions of the face, geometric features, and facial expressions. These techniques provide robust recognition under different illumination conditions and facial expressions. Furthermore, they are sensitive to noise, and invariant to translations and rotations.




## **4. Holistic Approach**

Holistic or subspace approaches are supposed to process the whole face, that is, they do not require extracting face regions or features points (eyes, mouth, noses, and so on). The main function of these approaches is to represent the face image by a matrix of pixels, and this matrix is often converted into feature vectors to facilitate their treatment. After that, these feature vectors are implemented in low dimensional space. However, holistic or subspace techniques are sensitive to variations (facial expressions, illumination, and poses), and these advantages make these approaches widely used. Moreover, these approaches can be divided into categories, including linear and non-linear techniques, based on the method used to represent the subspace.

#### *4.1. Linear Techniques*

The most popular linear techniques used for face recognition systems are Eigenfaces (principal component analysis; PCA) technique, Fisherfaces (linear discriminative analysis; LDA) technique, and independent component analysis (ICA).

• Eigenface [34] and principal component analysis (PCA) [27,62]: Eigenfaces is one of the popular methods of holistic approaches used to extract features points of the face image. This approach is based on the principal component analysis (PCA) technique. The principal components created by the PCA technique are used as Eigenfaces or face templates. The PCA technique transforms a number of possibly correlated variables into a small number of incorrect variables called "principal components". The purpose of PCA is to reduce the large dimensionality of the data space (observed variables) to the smaller intrinsic dimensionality of feature space (independent variables), which are needed to describe the data economically. Figure 9 shows how the face can be represented by a small number of features. PCA calculates the Eigenvectors of the covariance matrix, and projects the original data onto a lower dimensional feature space, which are defined by Eigenvectors with large Eigenvalues. PCA has been used in face representation and recognition, where the Eigenvectors calculated are referred to as Eigenfaces (as shown in Figure 10).

An image may also be considering the vector of dimension *M* × *N*, so that a typical image of size 4 × 4 becomes a vector of dimension 16. Let the training set of images be {*X*1, *X*2, *X*<sup>3</sup> ... *XN*}. The average face of the set is defined by the following:

$$X = \frac{1}{N} \sum\_{i=1}^{N} X\_i. \tag{9}$$

Calculate the estimate covariance matrix to represent the scatter degree of all feature vectors related to the average vector. The covariance matrix *Q* is defined by the following:

$$Q = \frac{1}{N} \sum\_{i=1}^{N} \left(\mathbf{X} - \mathbf{X}\_i\right) \left(\mathbf{X} - \mathbf{X}\_i\right)^{\mathrm{T}}.\tag{10}$$

The Eigenvectors and corresponding Eigen-values are computed using

$$
\mathcal{L}V = \lambda V, \quad \begin{pmatrix} V \epsilon \mathbb{R}\_{\text{tr}} \ V \neq 0 \end{pmatrix}, \tag{11}
$$

where *V* is the set of eigenvectors matrix *Q* associated with its eigenvalue λ. Project all the training images of *ith* person to the corresponding Eigen-subspace:

$$y\_k^i = w^T \ (\mathbf{x}\_i), \quad (i = 1, \ 2, \ 3 \ \dots \ N), \tag{12}$$

where the *yi <sup>k</sup>* are the projections of *x* and are called the principal components, also known as eigenfaces. The face images are represented as a linear combination of these vectors' "principal components". In order to extract facial features, PCA and LDA are two different feature extraction algorithms that are used. Wavelet fusion and neural networks are applied to classify facial features. The ORL database is used for evaluation. Figure 10 shows the first five Eigenfaces constructed from the ORL database [63].

• Fisherface and linear discriminative analysis (LDA) [64,65]: The Fisherface method is based on the same principle of similarity as the Eigenfaces method. The objective of this method is to reduce the high dimensional image space based on the linear discriminant analysis (LDA) technique instead of the PCA technique. The LDA technique is commonly used for dimensionality reduction and face recognition [66]. PCA is an unsupervised technique, while LDA is a supervised learning technique and uses the data information. For all samples of all classes, the within-class scatter matrix *SW* and the between-class scatter matrix *SB* are defined as follows:

$$S\_B = \sum\_{I=1}^{C} M\_i (\mathbf{x}\_i - \boldsymbol{\mu})(\mathbf{x}\_i - \boldsymbol{\mu})^T,\tag{13}$$

$$S\_w = \sum\_{l=1}^{C} \sum\_{\mathbf{x}\_k \in \mathcal{X}\_i} M\_i (\mathbf{x}\_k - \mu) (\mathbf{x}\_k - \mu)^T,\tag{14}$$

where μ is the mean vector of samples belonging to class *i*, *Xi* represents the set of samples belonging to class *i* with *xk* being the number image of that class, *c* is the number of distinct classes, and *Mi* is the number of training samples in class *i*. *SB* describes the scatter of features around the overall mean for all face classes and *Sw* describes the scatter of features around the mean of each face class. The goal is to maximize the ratio *det*|*SB*|/*det*|*Sw* |, in other words, minimizing *Sw* while maximiz ing *SB*. Figure 11 shows the first five Eigenfaces and Fisherfaces obtained from the ORL database [63].


the redundancies and to get the best face images description. Finally, the cosine metric is used to evaluate the similarity.


**Figure 9.** Example of dimensional reduction when applying principal component analysis (PCA) [62].

**Figure 10.** The first five Eigenfaces built from the ORL database [63].

**Figure 11.** The first five Fisherfaces obtained from the ORL database [63].

Owing to their limitations in managing the linearity in face recognition, the subspace or holistic techniques are not appropriate to represent the exact details of geometric varieties of the face images. Linear techniques offer a faithful description of face images when the data structures are linear. However, when the face images data structures are non-linear, many types of research use a function named "kernel" to construct a large space where the problem becomes linear. The required steps to implement the DCT technique are presented as Algorithm 1.

#### **Algorithm 1.** DCT Algorithm

*1. The input image is N by M;*

*2. f(i,j) is the intensity of the pixel in row i and column j;*

*3. F(u,v) is the DCT coe*ffi*cient in row u and column v of the DCT matrix:*

$$F(u,v) = \frac{2\mathbb{C}(u)\mathbb{C}(v)}{N} \sum\_{i=1}^{N} \sum\_{j=1}^{N} f(i,j) \cos\left(\frac{(2i-1)(u-1)\pi}{2N}\right) \cos\left(\frac{(2j-1)(v-1)\pi}{2N}\right)$$

$$= \frac{2\mathbb{C}(u)\mathbb{C}(v)}{N} \sum\_{i=0}^{N-1} \sum\_{j=0}^{N-1} f(i,j) \cos\left(\frac{(2i+1)u\pi}{2N}\right) \cos\left(\frac{(2j+1)v\pi}{2N}\right)$$
 $where \ 0 \le i, v \le N-1 \text{ and } \mathbb{C}(n) = \begin{cases} \frac{1}{\sqrt{2}} & (n=0) \\ 1 & (n \ge 0) \end{cases}$ 

⎪⎪⎪⎩ *4. For most images, much of the signal energy lies at low frequencies; these appear in the upper left corner of the DCT.*


1 (*n* -0)

*7. 8-bit pixels have levels from 0 to 255.*
