A Review of Three-Dimensional Facial Asymmetry Analysis Methods

Abstract

Three-dimensional symmetry and coordination are important factors in facial aesthetics, and analysis of facial asymmetry is the basis for clinical diagnosis, treatment, and doctor–patient communication. With the development of three-dimensional measurement and data analysis technology, facial asymmetry analysis methods are mainly based on facial anatomic landmarks, original-mirror alignment algorithm, facial anthropometric mask, and artificial intelligence. This review summarizes the methods of three-dimensional facial asymmetry analysis, and current research progress in the field. The advantages and limitations of various methods are analyzed and discussed to provide a reference for oral clinical application.

1. Introduction

Facial symmetry and coordination are important factors in aesthetics and attractiveness [1,2,3]. Symmetry is represented in size, structure, and balanced distribution on both sides of the face; however, due to congenital development, environment, disease, and other factors, there is no perfectly symmetrical human face [4,5]. In dental clinics, lower degrees of asymmetry are not easily detected, but severe facial asymmetry or facial defects can affect the physical and mental health of patients and often require treatment by orthognathic surgery and orthodontics [6]. Asymmetry analysis is critical to surgical planning and preoperative evaluation in orthodontics, oral and maxillofacial surgery, and aesthetic prosthodontics, also important in plastic and reconstructive surgery, facial nerve paralysis and anthropologic evaluation [7,8,9]. The development of digital technology has allowed the acquisition of facial data through three-dimensional facial scanning, computed tomographic (CT), cone-beam computed tomography (CBCT) and other three-dimensional imaging technologies generating clinically valuable reference data. Analysis of facial symmetry is also important in clinical diagnosis and doctor–patient communication when evaluating the efficacy of preoperative and postoperative orthognathic surgery [10,11]. Three-dimensional facial asymmetry analysis has become the focus of clinical research, and intelligent and automated evaluation methods are emerging.

The methods for facial asymmetry analysis are mainly based on anatomical landmarks, original-mirror alignment algorithms, template-mapping strategy, and artificial intelligence. Three-dimensional facial asymmetry analysis methods were shown in

Table 1. Three-dimensional facial asymmetry analysis methods.

Three-Dimensional Facial Asymmetry Analysis Methods		Asymmetry Evaluation Index	Application Conditions	Advantages	Limitation
facial anatomic landmarks	distance to reference plane method	$\mathrm{asymmetry} \mathrm{index} (\mathrm{AI}=\sqrt{{\left(\mathrm{Ldx}-\mathrm{Rdx}\right)}^2+{\left(\mathrm{Ldy}-\mathrm{Rdy}\right)}^2+{\left(\mathrm{Ldz}-\mathrm{Rdz}\right)}^2}$$), \mathrm{asymmetry} \mathrm{rate} (\mathrm{Q}=\frac{G-K}{G}$100%)	determine three-dimensional anatomical landmark	mature, simple, and intuitive	selection of landmarks is mostly based on manual annotation; reference plane affect the evaluation result
	Euclidean distance matrix analysis	FDM (ratio of the total number of asymmetrical lines to the total lines)	determine three-dimensional anatomical landmark	comprehensive, does not depend on the facial reference planes	selection of landmarks is mostly based on manual annotation
original-mirror alignment algorithm	Landmark-based method	RMS$(\sqrt{\frac{x_1^2+x_2^2+\dots +x_n^2}{n}}$$), \mathit{MAD} (\frac{\left|x_1\right|+\left|x_2\right|+\dots +\left|x_n\right|}{n})$, MSD$\left(\frac{x_1+x_2+\dots +x_n}{n}\right)$	original-mirror alignment algorithm (PA)	good repeatability and accuracy	manually selected landmarks
	Landmark-independent method		original-mirror alignment algorithm (ICP)	good repeatability and accuracy	manually selected stable regions
template-mapping strategy	dense quasi-landmarks mapping to target	RMS$\left(\sqrt{\frac{x_1^2+x_2^2+\dots +x_n^2}{n}}\right)$	original-mirror alignment algorithm (template-mapping + PA)	automation, structured data, corresponding quasi-landmarks	construct representative anthropometric mask
artificial intelligence	convolutional neural network (CNN)	facial symmetry score (MinClass + (MaxClass − MinClass) × PPredit)	N/A	efficient and intelligent	the training cost is high

.

2. Three-Dimensional Facial Asymmetry Analysis Method Based on Facial Anatomic Landmarks

Dental clinicians should first pay attention to important anatomical landmark features, such as the pronasale, pupil, and cheilion, when performing facial asymmetry analysis [12]. The quantitative evaluation of facial asymmetry based on three-dimensional facial landmarks mainly includes the distance to reference plane method and Euclidean distance matrix analysis (EDMA).

In the distance to reference plane method, there are two indexes, asymmetry index (AI) and asymmetry rate. Both are commonly used indexes in dental clinics, and the expert chooses the index according to personal preference. There is a slight difference in the calculation formula, but two indexes both belong to the calculation of the distance to the reference plane method.

The asymmetry index (AI) can be used to evaluate overall facial asymmetry and asymmetry of individual landmarks. The degree of landmark asymmetry is indicated by the difference in the linear distance between the landmarks and the reference plane. In 2013, Huang et al. used AI to quantify the degree of asymmetry of 16 facial landmarks in 60 Chinese adults, and left–right asymmetry (x-direction) was crucial in clinical treatment and was most easily detected by doctors and patients [13]. By calculating the distance from each landmark to the median sagittal plane and drawing a line chart, the left and right landmark deviations could be clearly described. The following formula was extended to evaluate the asymmetrical representation of the face data in three-dimensional space: $\mathrm{AI}=\sqrt{{\left(\mathrm{Ldx}-\mathrm{Rdx}\right)}^2+{\left(\mathrm{Ldy}-\mathrm{Rdy}\right)}^2+{\left(\mathrm{Ldz}-\mathrm{Rdz}\right)}^2}$, where Ldx, Rdx, Ldy, Rdy, Ldz, and Rdz represent the distances from the homonymous left and right anatomical landmarks to the sagittal, coronal, and horizontal planes, respectively; reference planes are shown in Figure 1. Faces that are more symmetrical have a smaller difference from the left and right homonymous landmarks to the reference plane. The smaller difference between two landmarks’ distances, the smaller the AI value while larger differences are associated with a greater AI value. In 2015, Xiong et al. using the asymmetry index by the average distance from the midpoint of bilateral landmarks and midline landmarks to the median sagittal plane and evaluated the facial asymmetry of 60 young people and 30 edentulous patients [14]. In 2016, Tian Kaiyue et al. used AI to evaluate the preoperative and postoperative facial symmetry of patients with skeletal Class III deformities and evaluated the digital design efficacy to correct the asymmetry deformity in patients with skeletal Class III deformities [15]. As a brief summary, the asymmetry index is relatively intuitive for evaluating the asymmetry of facial data and landmarks. However, the premise of calculating this AI value is to determine the facial reference planes; therefore, the accuracy of the facial reference planes is a key factor affecting AI.

The asymmetry rate is an established method for evaluating facial asymmetry, and is widely used [16,17,18]. It is expressed by the formula Q = $\frac{G-K}{G}$100%, where Q is the asymmetry rate, G is the larger linear distance from the measurement landmark on the left and right sides to the median sagittal plane, and K is the smaller one. A smaller Q value indicates that the two homonymous landmarks are more symmetrical, and a value of zero indicates complete symmetry. In 2014, Wang et al. applied the asymmetry rate to study the distance between the bilateral exocanthion and the cheilion line, and the distance between the bilateral gonion and the median sagittal plane. By calculating the asymmetry rate, a value of less than 10% was regarded as relatively symmetric [19]. The asymmetry rate evaluates the degree of asymmetry from the landmark to the sagittal plane or the reference plane, ignoring the facial morphological asymmetry of the face in the depth dimension; thus, it does not evaluate facial asymmetry in three-dimensional space. Nonetheless, the asymmetry rate is affected by the accuracy of the facial reference planes.

The Euclidean distance matrix analysis is a widely used morphological measurement method. In recent years, this method has been applied to the evaluation of facial asymmetry [20]. Based entirely on the three-dimensional spatial distribution characteristics of the anatomical facial landmarks, the EDMA method constructed the form difference matrix of the linear distance ratio between each landmark of the “left face” and “right face” and extracted the relevant ratios to construct the evaluation of the facial asymmetry index. The principle is to represent the shape and size of individuals through a form matrix (FM) generated by the Euclidean distances between landmarks in the geometric morphology of individuals. The form difference matrix (FDM) represents the morphological differences among individuals through the ratio of the corresponding matrix elements. Furthermore, the asymmetry of the three-dimensional facial region was evaluated by calculating the percentage of the total number of too large or too small matrix ratio elements. In 2008, Ercan et al. examined facial asymmetry among Turkish youths based on FDM ratio analysis. This population-level study found that there was asymmetry in the female face and that the shape of the left face had considerable impact on facial asymmetry [7], The results from the asymmetry analysis are shown in Figure 2. In 2014, Shuang et al. studied facial asymmetry among young women and men in Liaoning province of China using the EDMA method. The investigators recorded an FDM ratio lower than 0.95 and higher than 1.05 as asymmetry, and calculated the percentage of the total number of asymmetrical lines to the total lines [21]. The algorithm refers to the association between the landmarks; it is objective and comprehensive and does not depend on the facial reference planes.

Methods for evaluating facial asymmetry based on facial anatomical landmarks have been widely used and are relatively mature. The AI, asymmetry rate, and EDMA methods are used to evaluate the overall asymmetry of a face or individual landmark, which is simple and intuitive, and is conducive to the adoption in relevant research. The method described above has some advantages. However, limitations have been identified. First, the selection of landmarks is mostly based on manual annotation, and the accuracy of the selection of landmarks directly affects the evaluation of its asymmetry, which is prone to interference by human factors and the reproducibility is poor. Second, the accuracy of the reference plane is also an important factor affecting the evaluation of landmark asymmetry because it is necessary to calculate the AI and asymmetry rate. However, there is no unified standard method for defining a reference plane. It is difficult to fully express the morphological characteristics of the face by manually selecting a limited number of landmarks, and a detailed evaluation of facial asymmetry cannot be performed. Therefore, a more automatic, intelligent, and comprehensive evaluation method for facial asymmetry has emerged to solve the above problems to a certain extent.

3. Three-Dimensional Facial Asymmetry Analysis Method Based on Original-Mirror Alignment Algorithm

There is no consensus on a method for the evaluation of asymmetry in all types of facial deformities, and human factor is inevitable in the process of manual landmark selection. Therefore, in recent years, there has been increasing interest in algorithms that overlap the original and mirror facial models and evaluate facial asymmetry based on the overlap effect, which is referred to as the original-mirror alignment algorithm [22]. Broadly, the method involves the following steps: first, acquiring three-dimensional facial model; second, constructing a mirror image of the original facial model by reflecting left and right side of the face; lastly, calculating the distance between the two after alignment of the original and mirror models.

The core part of the original-mirror alignment algorithm is the optimal overlapping algorithm between the three-dimensional facial original and its mirror model. Current research on the three-dimensional overlapping algorithm of the original-mirror alignment method has developed in different directions. In dental clinics, if anatomical landmarks have been determined, the Procrustes analysis (PA) algorithm is more suitable to finish the alignment process. The Procrustes transformation is fundamentally the least-squares estimate of the rotation, translation and scaling required to align original-mirror corresponding landmarks. The core idea of the PA algorithm is to transform the coordinates of the representative one-to-one correspondence landmarks on the original and mirror models, so that the optimal matching is achieved between the corresponding landmarks on the original and the mirror model [23]. The landmarks used by the PA algorithm in medical image analysis are mainly anatomical landmarks with clinical significance, such as the pronasale, endocanthion, exocanthion, and pupil. Overlapping of the original and mirror models was achieved under the guidance of these landmarks with clear corresponding relations. In 2016, Xiong et al. reported clinical evaluation research on the application of the “regional PA original-mirror algorithm” to construct the facial median sagittal plane of patients with mandibular deviation. This improves the accuracy of the PA original-mirror algorithm by artificially screening and removing the landmarks related to facial asymmetry, which could meet the needs of symmetry analysis of clinical facial deformities in dental clinics [24]. In 2020, Zhu et al. conducted a weighted Procrustes analysis (WPA) algorithm for complex facial deformities to determine a three-dimensional facial median sagittal plane [25]. The feasibility and effectiveness of the WPA algorithm strategy in constructing a three-dimensional facial SRP were confirmed by applying this method to 40 patients with three-dimensional facial deformities. For patients with asymmetry deformity in the lower face, the clinical adaptability of the SRP constructed by the six WPA algorithms has been improved compared with the classical PA algorithm, achieving a result close to the truth plane of experts. The WPA algorithm is important and beneficial for oral clinical diagnosis [26]. In another clinical scenario when landmarks are not determined, the iterative closest point (ICP) algorithm is more suitable to finish the original-mirror alignment. The ICP algorithm is currently one of the most common used algorithms for overlapping 3D medical models. The ICP algorithm has two steps that are iterated until convergence or for a set number of iterations: (1) a corresponding point on the stationary surface is estimated for each point on the other floating surface; (2) like the PA algorithm, the Procrustes transformation translation to align the floating surface to its corresponding points on the targe surface is estimated and applied. During the iterative process, the corresponding point (the closest point) of stationary and floating surface is changed. Its principle is the iterative matching calculation of the nearest spatial distance between the point clouds of the two models based on the least-squares method. The algorithm obtains the optimal matching position between the two models through multiple iterations of the spatial matrix transformation, which is called registration. Overlapping regions of the original and mirror models, such as the frontal, upper, middle, and lower parts of the face, are manually selected to achieve the optimal matching effect in a clinical sense [12,27,28].

Different evaluation indicators are used to quantify facial asymmetry based on the original and mirror model overlapping methods. In 2016, Ozsoy et al. compared root mean square (RMS), mean absolute deviation (MAD), and mean signed distance (MSD). The calculation formulae for each indicator are as follows: $RMS=\sqrt{\frac{x_1^2+x_2^2+\dots +x_n^2}{n}}; MAD=\frac{\left|x_1\right|+{\dots }_{\dots }+\left|x_n\right|}{n}; MSD=\frac{x_1+x_2+\dots +x_n}{n}$ [29], where x represents the nearest distance from a point on the original model to the mirror model after overlapping, and n represents the total number of distances. The study found that RMS and MAD are more suitable indicators for the evaluation of the degree of facial asymmetry, while MSD does not contribute to the real morphological differences between the two models because it includes positive and negative distance values. In 2017, Codari et al. adopted RMS as a quantitative indicator of facial asymmetry to assess facial asymmetry in patients with facial paralysis by region and found that this method had high repeatability and accuracy [8]. In 2019, Cassi et al. applied RMS to evaluate facial asymmetry in patients with hemifacial microsomia and noted that this method could provide richer information for facial analysis, treatment planning, and follow-up [30].

RMS is widely used in the quantitative evaluation indicator of three-dimensional facial asymmetry based on the original-mirror alignment algorithm. Owing to its good repeatability and accuracy, this method can quantify the asymmetry of the entire or part of the face, which is more comprehensive for the evaluation of facial asymmetry. When original-mirror alignment is conducted using the PA or ICP algorithm, landmarks or relatively stable regions need to be manually selected by the operator. This inevitably introduces a degree of subjectivity and potential for error. In addition, there is no corresponding relationship between the point clouds of the original and mirror models in ICP algorithm, according to the nearest principle to overlap the original and mirror models may underestimate the facial asymmetry.

4. Three-Dimensional Facial Asymmetry Analysis Method Based on Template-Mapping Strategy

In recent years, facial asymmetry analysis method based on the template-mapping strategy has received considerable attention and application [31,32,33]. The anthropometric mask (AM) that contains dense quasi-landmarks is deformed into the shape of the three-dimensional target facial point clouds. This imposes the vertices and topology of the AM onto the target. The AM consists of thousands or tens of thousands of spatially dense quasi-landmarks that cover the entire facial region, which can be used as an extension of the traditional facial anatomical landmarks. Dense quasi-landmarks can express facial geometry in more detail and can be used for three-dimensional facial morphology analysis and symmetry analysis by mathematical and statistical methods. In 2011, Claes et al. constructed an AM with a dense sequence of point sets with approximately 10,000 points, and proposed the concept of weighted overlap of original and mirror masks. A three-dimensional facial median sagittal plane was constructed based on the PA alignment algorithm, and the RMSE was used to evaluate the three-dimensional facial symmetry. In 2012, Claes et al. created a distance map showing the amount and direction of facial soft tissue changes in three patients based on AM, and the results showed that they could meet clinical expectations [34]. In 2013, Walters et al. applied AM to patients with hemi-mandibular hyperplasia and hemi-mandibular elongation anomalies and observed the mandibular rotation direction and asymmetric magnitude in these patients, providing a basis for quantitative evaluation of facial asymmetry and design of orthognathic treatment plans [35]. In 2019, the AM template and the related Meshmonk program were made available to researchers for in-depth research in areas such as three-dimensional facial asymmetry evaluation and facial landmark determination [36,37].

RMS indices are used in both the template-mapping strategy and ICP algorithms when evaluating three-dimensional facial asymmetry, although these are essentially different methods for calculating the RMS value. The RMS value of ICP algorithm calculate the nearest distance between original and mirror models; however, in the template-mapping strategy, the distance between original and mirror corresponding quasi-landmarks on AM is calculated. Compared with other algorithms, the asymmetry analysis method based on a facial anthropometric mask has the advantages of more points, a corresponding relationship, and higher degree of automation, which compensate for the shortage of areas that lack anatomic landmarks, and is helpful in the study of subtle changes in the face and in determining the advantages of the differences in the facial parts.

5. Three-Dimensional Facial Asymmetry Analysis Method Based on Artificial Intelligence

Artificial intelligence technology has a strong ability for feature extraction and learning. With the deepening application of this technology in face recognition, there has been an increased interest in research in related medical fields, including facial asymmetry evaluation and automatic facial landmark detection. In 2019, Lin et al. established a deep convolutional neural network to extract three-dimensional (3D) contour features and assess the degree of facial symmetry in patients treated with orthognathic surgery. This included a deep convolutional neural network (CNN) for feature extraction and a regression network for prediction. The results showed that the 3D contour line-based features deep learning system serves as a general, useful, automated, and human-like efficient decision tool for objective assessment of facial symmetry before and after orthognathic surgery for the improvement of treatment in clinical practice. In 2021, Lin et al. applied a transfer learning model with a convolutional neural network based on three-dimensional (3D) contour line features to evaluate facial symmetry before and after orthognathic surgery. The results showed that the mean postoperative facial symmetry score was 3.52, which was significantly better than the preoperative score (mean, 2.74). The mean degree of improvement in facial symmetry after surgery was 21% [38]. In 2022, Zhu Yujia et al. proposed a method for automatically constructing a facial median sagittal plane based on a deep learning algorithm, a multi-view stacked hourglass convolutional neural network, and a mathematical model, which was the basis of three-dimensional facial symmetry evaluation [39]. Facial asymmetry analysis methods based on artificial intelligence have great potential by reducing dependency on doctors and improving clinical diagnosis and treatment efficiency.

6. Discussion

6.1. Automatic Determination of Facial Anatomical Landmarks Is Helpful to Improve the Traditional Method of 3D Facial Asymmetry Analysis Based on Landmarks

Facial landmarks play a vital role in the evaluation of asymmetry and are the basis for the asymmetry analysis method of PA and WPA algorithms, and calculation of indicators such as AI, asymmetry rate, and EDMA method. Following the traditional method, anatomical landmarks are defined by experts and human factors may affect this process. With the increasing development of digital technology, there is an urgent need for rapid and automatic landmark-determination algorithms, facial asymmetry analysis method based on PA alignment of dense facial points and artificial intelligence to gradually become a research hotspot. Wen et al. applied an AM template with 32 landmarks to the three-dimensional facial data of normal people and patients with mandibular deviation [40]. In general, the impact on the overall facial asymmetry is minimal in the case of the maxilla because the maxilla is connected to the skull and is difficult to separate from the facial structures, so the mandibular deviation patients were mostly evaluated. This approach obtained the facial landmarks of the target model and achieved efficient and automatic landmark extraction. Zhu et al. applied a deep learning algorithm to study the position error automatically determined 21 landmarks of three-dimensional facial data from 30 patients, and the results indicated that the average error was less than 2 mm, which could meet the needs of diagnosis and analysis of a large amount of oral clinical data [39]. The automatic determination of landmarks represents a promising future direction of research, and it can make up for the shortcomings of traditional methods affected by human factors and improve the efficiency of clinical diagnosis and treatment.

6.2. Automatic Construction Method of the Median Sagittal Plane to Provide a More Reliable Baseline for 3D Facial Asymmetry Analysis

Median sagittal plane is the basis of 3D facial asymmetry analysis, especially the calculation of facial asymmetry index based on landmarks and mirror the original model in original-mirror alignment algorithm. The automatic construction methods of the median sagittal plane based on 3D facial data include ICP original-mirror alignment algorithm and artificial intelligence. Among the original mirror-alignment algorithms, the ICP algorithm is a highly automated and repeatable method for analyzing facial asymmetry [8,28]. The global or regional ICP registration algorithm makes full use of the rich morphological information of the three-dimensional face model, which does not require the manual definition of landmarks and treats all point clouds participating in the operation equally. The alignment between the original and mirror point clouds was completely automatic, and no matching relation was required manually. The method is efficient, automatic, and has a certain universality, and can be used for facial analysis in patients with unilateral cleft lip and palate, orthodontic patients, orthognathic patients, and teenagers [41,42,43]. Considering the experts’ habits and experience in clinical diagnosis and treatment, this method should be used cautiously in specific cases. Important facial anatomic landmarks often receive more attention and sometimes even play a decisive role. Therefore, an asymmetry analysis method with reference to facial landmarks, such as the PA algorithm, has substantial strengths. With the advent of the era of big data, deep learning algorithms have great potential in the automatic segmentation and recognition of 3D features, but the research on the automatic construction of a median sagittal plane algorithm based on deep learning has not been reported. In 2019, Gao Lin et al. built a PRS-Net model based on the ShapeNet dataset to realize the automatic construction of the symmetry plane of the 3D point cloud data, which provided an important reference for the automatic construction of the facial median sagittal plane [44]. In 2022, Zhu et al. construct a three-dimensional facial median sagittal plane based on a deep learning algorithm in 30 patients with mandibular deviation. The angle error between the median sagittal plane constructed by the WPA algorithm and the expert plane was less than 1.5° [45].

6.3. Future Direction

Digital and intelligent diagnosis and treatment will become increasingly available, and the automatic evaluation of three-dimensional facial asymmetry will provide the basis for facial aesthetic analysis and design, allowing automated, accurate, and efficient determination of three-dimensional facial anatomical landmarks. Based on the non-rigid registration principle, the anthropometric mask realizes deformation matching from the template to the target model and constructs the corresponding average template for people of different ages, sex, race, and type of disease, improving the accuracy and robustness of the non-rigid registration. Artificial intelligence methods can automatically learn features from the three-dimensional facial data to obtain quantitative and objective indicators to assist doctors in clinical evaluations. This function can be integrated with diagnostic evaluation and surgical design software to evaluate the degree of improvement in patient asymmetry deformity after orthodontic and orthognathic treatment. It will also become an efficient doctor–patient communication tool.

7. Conclusions

Over the years, with the continuous improvement of facial data acquisition means and people’s aesthetic needs, symmetry analysis has provided a basis for oral clinical diagnosis and analysis. Facial asymmetry analysis methods reviewed in the paper mainly included the following: facial anatomic landmarks, original-mirror alignment algorithm, facial anthropometric mask, and artificial intelligence. Future studies are needed to develop more automatic, intelligent, and comprehensive evaluation method for three-dimensional facial asymmetry analysis.