To evaluate the computer-aided early diagnosis of facial paralysis based on IRT, the following three key issues were discussed: (1) the difference of the facial temperature feature symmetry between facial paralysis patients and healthy population, (2) the difference of the texture feature symmetry between facial paralysis patients and healthy population, and the influence of different feature symmetry combinations on the early diagnosis of facial paralysis, especially whether the application of texture features contributes to the diagnosis of facial paralysis.
3.1. Symmetry Measurements of Temperature Features
The degree of thermal symmetry on the left and right sides of the face can be observed intuitively through the pseudo-color of the facial temperature distribution. The facial thermal images of a healthy person and a patient with facial paralysis were compared using different pseudo-color methods presented in
Figure 5. Due to the distribution of blood vessels on the human face, both patients with facial paralysis and normal people have the facial thermal characteristics of high temperature in the middle and low temperature on both sides, which is consistent with the research findings of Guan [
47] and Liu et al. [
14,
36]. In addition, facial paralysis destroys the symmetry of temperature distribution between the left and right sides of the face. The quantitative evaluation is shown in
Table 5.
Temperature difference is one of the most commonly used methods to measure the symmetry of temperature features on both sides of the face [
14].
Table 5 presents the statistics for the facial average temperature difference (Δ
Tmean) and maximum temperature difference (Δ
Tmax) between facial paralysis patients and normal people. It can be seen from
Table 5 that in the forehead, orbital, and infraorbital ROIs, the Δ
Tmean of facial paralysis patients were greater than that of the normal population (control group), and there were statistical differences between the two groups (
Figure 6). Additionally, the Δ
Tmax of patients with facial paralysis in the forehead and infraorbital ROIs were greater than that of the normal population, and there were statistical differences between the two groups (
Figure 7). These results show that the symmetry of temperature features measured by Δ
Tmean and Δ
Tmax can preliminarily distinguish patients with facial paralysis from normal people, and the symmetry of temperature features with statistical differences can be used as the selected features for early diagnosis of facial paralysis.
Drawing comparison to previous studies [
14], the results of Δ
Tmean are seen to be consistent, but the results of Δ
Tmax are not completely identical. The reasons for the difference are (1) the calculation method of Δ
Tmax is different, i.e., the previous study [
14] only considered the difference of the maximum temperature between the two ROIs, but this study highlights the difference between the temperature features of the two ROIs through combining the maximum and minimum temperature; (2) the size of ROIs, especially in the orbital region, was not entirely consistent; and (3) the sample size was enlarged in this study.
Figure 6 and
Figure 7 further show the difference in temperature feature symmetry between patients with facial paralysis and healthy people. According to the interquartile ranges and the probability density of data distribution, it can be found that the inter-subject variability of patients with facial paralysis was greater than that of normal people; this is mainly caused by the inconsistent severity of symptoms in different parts of different facial paralysis patients. In addition, Δ
Tmean and Δ
Tmax cannot be used in the diagnosis of facial paralysis in nasal and mouth regions, mainly due to the large variation of temperature features in these regions in the normal population.
3.3. Performance Measurement of Different Feature Symmetry Combinations for Diagnosis of Facial Paralysis
Different feature symmetry combinations have great influence for the diagnostic performance of classifiers. In this study, an easy-to-use feature symmetry selection method was used, i.e., we selected the temperature and texture feature symmetry with statistical differences between patients with facial paralysis and normal population. Therefore, the performance of different feature symmetry combinations was compared: (1) all temperature feature symmetries and the temperature feature symmetries with statistical differences (
Table 8), (2) all texture feature symmetries and the texture feature symmetries with statistical differences (
Table 9), (3) the combinations of all temperature and texture feature symmetries and the combinations of the feature symmetries with statistical differences (
Table 10).
The simplest feature symmetry selection method involves inputting all temperature and texture feature symmetries. When only temperature feature symmetry was selected, the accuracy of SVM in the diagnosis of facial paralysis was 0.833 (
Table 8). When only texture feature symmetry was input, the diagnostic accuracy of facial paralysis was 0.767 (
Table 9). When all temperature and texture feature symmetries were combined, the diagnostic accuracy of facial paralysis was 0.8 (
Table 10). With the increase of feature dimensionality and use of texture features, there was no improvement in the classification performance after using only temperature feature symmetry. This attributed to the lack of feature selection. More features are not necessarily better, but features that are more complementary and differentiated should be selected.
For the results in
Table 5,
Table 6 and
Table 7, t-test was used to select temperature and texture feature symmetries with significant differences between the two groups of subjects from all features. When only temperature feature symmetry with significant differences was used, the diagnostic accuracy of facial paralysis was 0.767 (
Table 8). When only texture feature symmetry with significant differences was used, the accuracy was 0.833 (
Table 9). After combining the above two feature symmetries, the diagnostic accuracy of facial paralysis was 0.933 (
Table 10). In conclusion, texture feature symmetry is helpful for diagnosis of facial paralysis, and the combination of temperature and texture feature symmetry with significant differences can improve the diagnostic accuracy of facial paralysis compared to using a single type of feature symmetry.
Moreover, SVM was selected as a diagnostic classifier for facial paralysis. The AUC was used for comparing the generalization performance of SVM with other two typical classifiers, as shown in
Figure 8. SVM has good adaptability to the diagnosis of facial paralysis, and the AUC is 0.947, i.e., higher than k-NN and LDA. In addition, this study proposed an SVM diagnosis system for facial paralysis based on temperature and texture features, which have higher sensitivity, specificity, and AUC than those of previous similar studies [
14,
34] (
Table 11). The reasons for these results are: (1) SVM is good at small sample machine learning; (2) the previous studies only used single type features and threshold method; (3) two types of complementary thermal features were combined, and supervised learning was used to train the classification parameters in this study.
HB score is a conventional method to evaluate facial paralysis. Based on the asymmetry of facial expressions of the subjects, the facial nerve function is evaluated as normal (grade I) and facial paralysis (grade II-VI) by physicians. The drawback is that it relies on a subjective judgment with significant inter-rater variation. In order to solve the problem, the computer-aided diagnosis methods for facial paralysis based on HB Score are developed, and the highest accuracy of distinguishing facial paralysis from normal is 0.923 [
1,
9,
48]. In this study, the accuracy of the proposed method based on temperature asymmetry is 0.933. We consider this accuracy excellent because relevant studies have shown that the reliability of HB Score itself ranges from 0.8 to 0.93 [
49]. Because the data set used for training the classifier is calibrated by the conventional method, the accuracy has reached the upper limit of the conventional method. In addition to the comparison with the above literatures [
1,
9,
48,
49], the proposed method and the conventional method will be used to diagnose the same patients for a clearer comparison in the future study. The proposed method verifies the feasibility of infrared thermal imaging in the diagnosis of facial paralysis, which is beneficial compared with traditional methods. This is mainly caused by two reasons: (1) the automatic thermal asymmetry analysis algorithm can avoid the inter-rater variation, (2) subjects do not need to make a series of facial expressions and it leads to higher coordination.
The patients with very mild symptoms may be missed diagnosed through HB scores. In the proposed thermal imaging diagnosis, no patients with facial paralysis was missed diagnosed, but 13.3% of healthy subjects were misdiagnosed as facial paralysis due to the variation of the facial temperature asymmetry. This method is a useful supplement to the conventional method. The asymmetry of facial temperature distribution in patients with facial paralysis is not completely consistent with the HB scores. Patients with mild symptoms may have significant facial thermal asymmetry, which needs to be verified by expanding the data set in future studies.
In order to assist physicians in clinical decision-making, our final goal is to develop a facial paralysis evaluation system based on infrared thermal imaging. This process is divided into four steps. Firstly, the facial temperature distribution between patients with facial paralysis and healthy people is compared to find out the feature symmetry combination with significant difference. Secondly, an automatic method which distinguishes between facial paralysis and normal is developed using thermal asymmetry analysis. Thirdly, the differences in thermal images between facial paralysis and other diseases that change the facial temperature distribution are analyzed, and a thermal image analysis method to distinguish facial paralysis from other diseases will be further explored. Finally, the correlation between thermal asymmetry and HB scores is analyzed, and a computerized facial paralysis grading system is explored based on thermal asymmetry. This study focuses on the first two steps. In order to complete the distinction between facial paralysis and normal on thermal images, the exclusion criteria are used to eliminate the interference of certain diseases on the diagnosis of facial paralysis. The exclusion criteria can enhance the sensitivity and specificity of thermal asymmetry analysis and simplify the complexity of this research.
Certain diseases may have similar facial temperature manifestations, such as facial paralysis, stroke, Parkinson, temporomandibular arthritis, etc. These diseases may lead to the facial temperature asymmetry, but the degree and site of temperature asymmetry may be different. However, this study only analyzes the difference in facial thermal images between facial paralysis and healthy people. In addition, a general framework for analyzing facial thermal asymmetry is proposed in this study. In the future research, it is necessary to distinguish the diseases with similar temperature features by changing the facial ROIs.
The automatic output results of the developed computer-aided thermal image analysis system are divided into two categories: facial paralysis and normal. In future studies, the output of the extended system will be the severity of facial paralysis (six classes, normal to complete paralysis [
48,
49]). Accordingly, the following methods could be adopted: (1) enlarging the sample dataset and increasing the number of subjects with different facial paralysis severity; (2) analyzing the correlation between temperature and texture features and the severity of facial paralysis; (3) evaluating other temperature and texture features, such as Histogram of oriented gradients and Gabor filters; (4) extending the application to the curative effect evaluation of facial paralysis.