Next Article in Journal
Hyperacusis Assessment Questionnaire—A New Tool Assessing Hyperacusis in Subjects with Tinnitus
Next Article in Special Issue
Accuracy of Palatal Orthodontic Mini-Implants Placed Using Fully Digital Planned Insertion Guides: A Cadaver Study
Previous Article in Journal
Unveiling the Accuracy of Ultrasonographic Assessment of Thyroid Volume: A Comparative Analysis of Ultrasonographic Measurements and Specimen Volumes
Previous Article in Special Issue
Scattered Radiation Distribution Utilizing Three Different Cone-Beam Computed Tomography Devices for Maxillofacial Diagnostics: A Research Study
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
JCM
Volume 12
Issue 20
10.3390/jcm12206621
jcm-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Correlation Analysis of Nasal Septum Deviation and Results of AI-Driven Automated 3D Cephalometric Analysis

by
Natalia Kazimierczak
1,
Wojciech Kazimierczak
1,2,*,
Zbigniew Serafin
2,
Paweł Nowicki
1,
Adam Lemanowicz
2,
Katarzyna Nadolska
2 and
Joanna Janiszewska-Olszowska
3
1
Kazimierczak Private Dental Practice, Dworcowa 13/u6a, 85-009 Bydgoszcz, Poland
2
Collegium Medicum, Nicolaus Copernicus University in Torun, Jagiellońska 13-15, 85-067 Bydgoszcz, Poland
3
Department of Interdisciplinary Dentistry, Pomeranian Medical University in Szczecin, 70-111 Szczecin, Poland
*
Author to whom correspondence should be addressed.
J. Clin. Med. 2023, 12(20), 6621; https://doi.org/10.3390/jcm12206621
Submission received: 31 August 2023 / Revised: 4 October 2023 / Accepted: 13 October 2023 / Published: 19 October 2023
(This article belongs to the Special Issue Clinical Advances in Head and Neck Imaging including Dentistry)
Browse Figures
Versions Notes

Abstract

:
The nasal septum is believed to play a crucial role in the development of the craniofacial skeleton. Nasal septum deviation (NSD) is a common condition, affecting 18–65% of individuals. This study aimed to assess the prevalence of NSD and its potential association with abnormalities detected through cephalometric analysis using artificial intelligence (AI) algorithms. The study included CT scans of 120 consecutive, post-traumatic patients aged 18–30. Cephalometric analysis was performed using an AI web-based software, CephX. The automatic analysis comprised all the available cephalometric analyses. NSD was assessed using two methods: maximum deviation from an ideal non-deviated septum and septal deviation angle (SDA). The concordance of repeated manual measurements and automatic analyses was assessed. Of the 120 cases, 90 met the inclusion criteria. The AI-based cephalometric analysis provided comprehensive reports with over 100 measurements. Only the hinge axis angle (HAA) and SDA showed significant (p = 0.039) negative correlations. The rest of the cephalometric analyses showed no correlation with the NSD indicators. The analysis of the agreement between repeated manual measurements and automatic analyses showed good-to-excellent concordance, except in the case of two angular measurements: LI-N-B and Pr-N-A. The CephX AI platform showed high repeatability in automatic cephalometric analyses, demonstrating the reliability of the AI model for most cephalometric analyses.

1. Introduction

The nasal septum is a key factor in the development of the craniofacial skeleton during ontogeny [1,2,3,4,5,6]. Research conducted on animal models has shown that the mechanical forces generated by the growing nasal septum have a crucial impact on the surrounding sutural growth sites [7,8,9]. It is believed that the development of the nasal septum and the forces it exerts on surrounding tissues are responsible for the development of midface, sagittal, and vertical maxillary growth [8,10,11].
Numerous studies have demonstrated that, depending on the criteria applied and the populations examined, 18–65% of individuals exhibit nasal septum deviation (NSD), with a significantly higher prevalence observed in the European population [12,13,14,15,16]. The impact of NSD on the development of facial and cranial asymmetry remains a subject of ongoing research. Apart from the obvious correlations, such as nasal asymmetry and unilateral nasal turbinate hypertrophy [17,18,19], more distant correlations, such as facial and palatal asymmetries, have been demonstrated [20,21,22,23]. NSD and subsequent altered respiratory functions are linked to disrupted growth of the maxilla and mandible [24,25,26]. This suggests that NSD may have far-reaching effects on craniofacial development and highlights the importance of further investigation into the potential consequences of this condition. Considering the well-documented influence of the growing nasal septum on the anatomical proportions of the developing craniofacial structure, it is therefore pertinent to explore potential correlations between nasal septum abnormalities and the anatomical proportions of the craniofacial region.
Since its introduction in 1931, cephalometric analysis has been a fundamental technique used in orthodontics and craniofacial research to assess skeletal and dental relationships in the craniofacial complex [27]. It involves the use of X-ray lateral cephalograms of the head and face to obtain precise linear and angular measurements between predefined lanmarks. These measurements are then compared to established norms, allowing us to evaluate growth patterns, diagnose malocclusions, and plan orthodontic treatment. The obtained results provide valuable insights into the relationship between the maxilla, mandible, and cranial base, contributing to the diagnosis and treatment planning process. The advancement of technology has enabled the replacement of manual measurements with digital cephalometric analysis software, facilitating quicker measurements and the automatic presentation of analysis results. In recent years, there has been a growing interest in the application of artificial intelligence (AI) in the medical sciences, particularly in the field of medical imaging. This technology has seen rapid implementation in orthodontics, specifically in the analysis of X-ray images for pre-orthodontic treatment and cephalometric analysis. Manual analysis of cephalometric X-ray images is highly operator-dependent and prone to significant variability in landmark identification [28,29,30,31]. However, the results of automatic cephalometric analysis using AI have been shown to be relatively stable and repeatable compared to manual analysis. Several studies have reported the accuracy and reliability of AI software in cephalometric analysis, demonstrating potential for improving diagnostic accuracy and reducing variability and analysis time [32,33,34,35,36,37,38]. In addition to automated cephalometric analysis, AI has already proven its effectiveness in other orthodontic pre-treatment assessments. Lo Giudice et al. have demonstrated that automated segmentation of the upper airway and hard tissues (mandible) in CBCT scans, based on convolutional neural networks (CNNs), is as accurate as an experienced reader [39,40]. This efficiency can lead to improved workflow and productivity in dental practices.
The integration of AI and cone-beam computed tomography (CBCT) in dental diagnostics has paved the way for the development of AI-based programs such as WebCeph, WeDoCeph, and CephX. These programs automate the identification of anatomical measurement points, evaluate landmarks, calculate angles and distances, and generate automated analysis reports with significant findings. The primary advantage of such software is the ability to automatically perform measurements and analyses based on craniofacial CT scans, potentially reducing the need for additional cephalometric images for patients prior to orthodontic treatment. Studies conducted thus far have demonstrated a high degree of agreement between measurements and analyses performed with CephX software and those obtained through digital cephalometric analysis [35,36,41].
The first objective of our study was to perform cephalometric analysis on computed tomography (CT) scans of our sample population using AI-based algorithms. The second aim of our research was to evaluate the correlation between NSD and the results of the cephalometric analyses of our subjects.

2. Materials and Methods

2.1. Patient Selection

This study has been approved by the bioethical committee of our institution (decision reference No. KB 227/2023).
The study material comprised CT scans performed on 120 consecutive patients aged 18 to 30 admitted to the Emergency Department of our institution between 1 January 2020, and 31 December 2022. All the CT scans were performed on the same 64-slice CT scanner (Discovery 750HD; GE Health Care; Waukesha, WI, USA) using 64 × 0.625 mm collimation, 32 cm scan FOV, 260-mA tube current, 120 kVp tube voltage, 0.625 mm slice thickness, 0.8 s per gantry rotation, and a pitch of 0.531. CT scans were acquired in the range from the vertex to the lower levels of the cervical spine, covering the whole craniofacial area.
The indications for CT scans included post-traumatic assessments in patients who experienced generalized trauma or trauma to the craniofacial area.
The inclusion criteria were as follows:
  • CT scan covering the region from the chin to the vertex;
  • Age 18–30 years, to exclude multiple missing teeth and acquired craniofacial deformations from the measurements conducted;
  • Centric occlusion of the patient’s teeth.
The exclusion criteria were as follows:
  • Fractures of the craniofacial bones;
  • Severe motion artifacts;
  • >4 teeth missing per dental arch;
  • Tumors in the craniofacial area;
  • Severe metal artifacts.

2.2. Cephalometric Analysis

CT scans were manually uploaded into a cloud-based AI software database—CephX (ORCA Dental AI, Las Vegas, NV, USA). The software automatically performed all the available cephalometric analyses and provided a report for each patient. Supplementary Table S1 summarizes all the automatically performed analyses. The major reference points are summarized in Table 1 and presented in Figure 1.

2.3. NSD Analysis

The NSD was assessed using a method that measures the maximum deviation from an “ideal” non-deviated septum. This hypothetical, non-deviated septum was determined based on multiplanar reconstructions (MPR), drawing a straight line between the perpendicular plate of the ethmoid bone and the midline palatine suture. The deviation measurement was obtained by calculating the maximum horizontal distance from the line representing the ideal, non-deviated septum to the most outer, bony, deviated septal contour. Previous studies have demonstrated the usefulness of the perpendicular plate–vomer suture (PPV) in assessing the maximum deviation of the nasal septum in computed tomography (CT) images [5,42]. The septal deviation angle (SDA) was measured in coronal CT sections using the criteria presented in papers by Orhan and Kajan [19,43]. Figure 2 presents both types of measurements.
Due to high variability, the thickness of the nasal septum mucous membrane was not considered in the measurements. All manual measurements were performed using the OsiriX MD v. 13.1 software (Pixmeo SARL; Geneva, Switzerland).

2.4. Error Study

Twenty randomly selected subjects were re-examined by the same author 1 month after initial tracings. PPV and SDA measurements were repeated 1 month later by the same investigator in 20 randomly selected CT scans to calculate the intraobserver repeatability. The repeatability of measurements was assessed using a one-sided Wilcoxon test. The intraclass correlation coefficient (ICC) regarding AR calculations was calculated to assess the agreement between examinations.
Twenty randomly selected subjects were re-uploaded to the AI-software database in order to assess the repeatability of automatic measurements. The ICC regarding the results of repeated automatic analyses was calculated to assess the agreement between examinations.

2.5. Statistical Evaluation

Sample size was verified using an online sample size calculator (https://clincalc.com (accessed on 10 January 2023)). Clinical significance was set at the level of 1 mm in linear measurements and 1° in angular measurements. The correlations between quantitative variables were analyzed using Spearman’s correlation coefficient. The agreement between manual measurements and automatic analysis for quantitative variables was assessed using the intraclass correlation coefficient (ICC) of type 2, according to the classification by Shrout and Fleiss. The significance level for all statistical tests was set to 0.05. R 4.2.3. statistical software was used for computations.

3. Results

3.1. Population

The authors reviewed the CT scans of 120 cases. Thirty cases were excluded as they failed to meet our inclusion criteria. Measurements from 90 patients were included. The mean age of all participants was 23.9 years (SD 3.81; median 24; range 18–30). This constituted 66 males with a mean age of 23.7 (SD 3.18; range 18–30) and 24 females with a mean age of 21.15 (SD 4.03; range 18–29).

3.2. Automatic Cephalometric Analysis

The results of automated cephalometric analyses conducted by AI were primarily provided in the form of reports containing >100 measurements (linear and angular), with the established range, normal values, and comments on potential clinical implications. Part of the sample report is shown in Figure 3.

3.3. NSD Analysis

The findings from the two methods employed for assessing NSD, namely SSD and PPV, demonstrated a significant (p < 0.001) and positive correlation, with Spearman’s correlation coefficient equating to 0.967. This suggests that as the deviation in millimeters increases, so does the angle of deviation. A graphical representation of the results from this analysis can be found in Figure 4.
Out of multiple correlations between NSD measurements and the results of the cephalometric analyses, only the hinge axis angle (HAA) and SDA showed a significant negative correlation (p ˂ 0.05, r ˂ 0). However, no significant correlation was found between HAA and PPV. See Figure 5 for a visual representation and Table 2 for the sample results of the correlation analysis.

3.4. Error Study

The analysis of the repeatability of PPV and SDA measurements carried out by the reader demonstrated excellent concordance (data summarized in Table 3).
The results of the repeatability of automatic cephalometric measurements showed excellent concordance in the majority of measurements. A list of parameters with poor concordance of repeated measurements is shown in Table 4. A full list of the conducted repeatability analyses is provided in the Supplementary Material—Table S2.

4. Discussion

Our study revealed no significant correlations between NSD parameters (SDA and PPV) and the majority of cephalometric measurements, with the exception of a weak correlation between SDA and HAA (p = 0.039). No significant correlation was found between PPV and HAA. The concordance of repeated manual reader measurements demonstrated excellent repeatability, while the majority of parameters in the repeated automatic analysis displayed good-to-excellent concordance, excluding two angular measurements: Lower Incisor–Nasion–B-Point (LI-N-B) and Prosthion–Nasion–A-Point (Pr-N-A). The latter indicates the need for the correction of algorithms determining single cephalometric points. Good, and in most cases excellent, concordance of the repeated analysis results indicates the high effectiveness of the tested AI algorithms in determining cephalometric landmarks.
The results of our study align with earlier published research, demonstrating the potential of AI cephalometric analysis, with varying degrees of success. A study by Lee et al. (2018) developed an AI system for automatic cephalometric landmark detection. The system was found to be highly accurate, with a mean landmark error of 1.53 ± 1.74 mm and an error of less than 2 mm for 82% of landmarks. However, the study also highlighted that the performance of the AI system varied depending on the specific landmark, suggesting the need for further refinement and training of the AI algorithm [44]. Similar results were achieved by other research teams, demonstrating a superior AI success classification rate compared to humans in some cephalometric analysis measures [30,31,45]. An interesting study by Bao et al. (2023) evaluated the accuracy of AI in the automated cephalometric analysis of reconstructed lateral cephalograms from CBCTs for 85 patients. The mean radial error for 19 chosen landmarks was 2.07 ± 1.35 mm and an error of less than 3 mm in 71.7% with the automatic program. The authors concluded that automatic analysis is almost effective enough to be acceptable in clinical work, but is not currently capable of completely replacing manual tracing [46]. Some minor inaccuracies have also been found in previously published papers regarding the reliability of CephX cephalometric analysis. Despite these issues, the authors unanimously concluded that the software is reliable for cephalometric analysis [35,36,41].
The hinge axis (HA) remains an integral part of dental assessments, especially orthognathic and orthodontic assessments, regardless of disputes over the existence of pure rotational movement in temporo-mandibular joints [47]. The HAA has emerged as a critical parameter in cephalometric analysis, providing invaluable insights into craniofacial morphology and growth patterns. The HAA refers to the angle formed by the intersection of the Frankfort horizontal (FH) plane and a line passing through the anatomical HA of the mandible [48]. The HAA in our study was defined as the angle between three cephalometric landmarks, Dc–Go–LI (distobuccal cusp of the first permanent upper molar–gonion–lower lip). Studies have shown that a larger HAA indicates a more vertical growth pattern, while a smaller HAA suggests a more horizontal growth pattern [48]. However, it is crucial to recognize that the HAA is influenced by several factors, including cranial base flexure, facial height, and the position of the mandible [48,49,50]. Our analysis revealed only a slight, negative correlation between HAA and SDA, and no correlation with PPV. This means that as the deviation angle of the nasal septum increases, the HAA decreases. Contrary to expectations, no significant correlation between HAA and PPV was found. A review of the scientific literature did not reveal any articles convincingly demonstrating a correlation between HAA and NSD. Therefore, we assume that the correlation between HAA and SDA might be coincidental. Further studies with a larger patient sample are necessary to clarify this issue.
NSD is associated with a range of direct disorders, such as headaches, rhinosinusitis, gastroesophageal reflux, and sleep apnea [51,52,53,54]. Understanding the full extent of NSD’s impact on facial and cranial asymmetry will be crucial in developing effective treatment strategies and improving patient outcomes. Among the factors influencing NSD, midfacial trauma, septal abscess, and craniofacial anomalies (e.g., cleft lip and palate) have been identified [11,23,55,56]. In some cases, a more subtle influence of the anatomical relationships of the surrounding nasofacial skeleton, including developmental constraints and morphological details of the nasal septum, has been suggested to contribute to the development of NSD [57]. A number of studies have demonstrated that a smaller volume of the nasal cavity is associated with a higher percentage of NSD, indicating a mutual influence between these phenomena [12,43,57,58]. More distant correlations between NSD and facial structures have also been found. A study published by Kim et al. [22] revealed a relationship between NSD and horizontal facial asymmetries. Additionally, Gray et al. found a high correlation between septal and palatal asymmetries and dental malocclusion [20,21,23]. A study conducted in 2015, involving 55 patients, demonstrated a correlation between NSD and asymmetries in the nasal and palatal regions [4]. However, it did not establish a correlation between these asymmetries and those of the lateral facial region.
Doubts about the generally accepted norms of cephalometric analysis results and prevailing beauty standards have been present for some time [59,60]. Additionally, the issues of beauty and attractiveness can be perceived differently by orthodontists and patients. Furthermore, apart from skeletal scaffolding, the assessment of soft facial tissues, which greatly influence the matter, is significantly limited in 2D cephalometric assessment, leading to an increasing trend of including 3D scans of the face and intraoral tissues (such as 3D dental models, 2D or 3D X-ray images, and photographs) in the standard set of examinations [60,61,62]. Such a multimodal, diverse dataset could later be used to create a more complete representation, display, and perception of the relevant structures [63]. It could also be used to create a “virtual patient” for discussing the expected treatment outcomes with the patient [64]. The issue of treatment planning, visualization of its results, and the integration of large-scale, multimodal datasets is a problem that hinders the implementation of these concepts. Therefore, there is an increasing focus on the use of AI in the analysis of 3D faces and in further treatment outcome planning [65,66,67]. Studies that assessed the accuracy of three-dimensional soft tissue prediction for Le Fort I osteotomy and orthognathic cases using Dolphin 3D software showed a limited reliability of the software [66,67]. However, in a 2022 study, Tanikawa attempted to develop AI systems that predict the 3D facial morphology after orthodontic treatment and orthognathic surgery [65]. The authors utilized lateral cephalograms, 3D facial images, and two AI systems to predict facial morphology after dental treatment. The AI systems proved to sufficiently predict facial morphology after treatment and were considered clinically acceptable. Such integration of AI, multimodal facial morphology assessment, treatment planning, and advanced visualization appears to be the future of orthodontics.
Our study evaluated lateral cephalograms, assessing the correlation of NSD mainly using vertical facial morphology. Considering the results of our work and existing data from the literature, the strongest potential connection appears to be between NSD and horizontal facial asymmetries. Taking into account the previously cited studies based on animal models, narrowing down the study group to patients with nasal septum damage at an early stage of development would likely reveal correlations not present in our group of consecutive, post-traumatic patients admitted to the Emergency Department. Furthermore, although we consider the AI-driven cephalometric analysis to be highly efficient and cost-effective in modern orthodontic practices, we believe that further studies regarding its performance must be conducted. Issues related to the influence of study quality on cephalometric analysis results, the repeatability of analysis results, and refinements and reliability of the algorithms are certainly areas that should be further explored. It should also be noted that our study is one of the initial analyses of selected morphological parameters using AI. An exciting future prospect is the ability of AI to automatically analyze large databases of imaging studies available on PACS servers at large institutions. This will certainly enable the discovery of subtle, previously unnoticed correlations between distant morphological parameters. It will undoubtedly have a significant impact on the development of fields as “hoary” and seemingly explored as anatomy.
The potential limitations of this study could be attributed to the relatively small sample size of patients included, potentially limiting the ability to detect subtle correlations between the studied parameters, which might only become apparent when analyzing larger groups. A geographic limitation was also present, as all radiographs were obtained from the same research center. Attention should also be given to the potential influence of errors in AI algorithms when determining cephalometric landmarks in the analysis results. It should also be noted that our study analyzed one of the commercially available solutions, and the obtained results should not be generalized to other AI software.

5. Conclusions

In conclusion, the results of the multiparametric cephalometric analysis were not correlated with the degree of nasal septum deviation in patients in our study group, except for a weak, negative correlation between HAA and SDA. The results of automatic cephalometric analyses performed by the CephX AI platform showed excellent repeatability, except for two types of angular measurements, LI-N-B and Pr-N-A, indicating the high reproductivity of the tested AI model in most cephalometric analyses.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/jcm12206621/s1; Table S1: List of performed automatic cephalometric analyses; Table S2: Results of repeatability analysis of automatic AI cephalometric analyses.

Author Contributions

Conceptualization, N.K., W.K. and J.J.-O.; methodology, N.K. and W.K.; software, W.K.; validation, Z.S. and J.J.-O.; formal analysis, P.N.; investigation, N.K. and P.N.; resources, W.K.; data curation, P.N.; writing—original draft preparation, N.K. and W.K.; writing—review and editing, N.K., W.K., A.L. and K.N.; visualization, W.K.; supervision, Z.S. and J.J.-O.; project administration, N.K. and W.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding. The APC was funded by the authors.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by the Ethics Committee of Collegium Medicum, Nicolaus Copernicus University in Torun, Poland (protocol No. KB 227/2023, 10.04.20223) for studies involving humans.

Informed Consent Statement

Patient consent was waived due to the retrospective nature of the study and patients’ data anonymization.

Data Availability Statement

Data are available on request.

Acknowledgments

LD, a statistician, was involved in data analysis.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Latham, R.A. Maxillary Development and Growth: The Septo-Premaxillary Ligament. J. Anat. 1970, 107, 471–478. [Google Scholar]
  2. Goergen, M.J.; Holton, N.E.; Grünheid, T. Morphological Interaction between the Nasal Septum and Nasofacial Skeleton during Human Ontogeny. J. Anat. 2017, 230, 689–700. [Google Scholar] [CrossRef]
  3. Pirsig, W. Growth of the Deviated Septum and Its Influence on Midfacial Development. Facial Plast. Surg. 1992, 8, 224–232. [Google Scholar] [CrossRef]
  4. Hartman, C.; Holton, N.; Miller, S.; Yokley, T.; Marshall, S.; Srinivasan, S.; Southard, T. Nasal Septal Deviation and Facial Skeletal Asymmetries. Anat. Rec. 2015, 299, 295–306. [Google Scholar] [CrossRef] [PubMed]
  5. Denour, E.; Roussel, L.O.; Woo, A.S.; Boyajian, M.; Crozier, J. Quantification of Nasal Septal Deviation with Computed Tomography Data. J. Craniofacial Surg. 2020, 31, 1659–1663. [Google Scholar] [CrossRef]
  6. Rönning, O.; Kantomaa, T. Experimental Nasal Septum Deviation in the Rat. Eur. J. Orthod. 1985, 7, 248–254. [Google Scholar] [CrossRef] [PubMed]
  7. Lieberman, D.E.; Hallgrímsson, B.; Liu, W.; Parsons, T.E.; Jamniczky, H.A. Spatial Packing, Cranial Base Angulation, and Craniofacial Shape Variation in the Mammalian Skull: Testing a New Model Using Mice. J. Anat. 2008, 212, 720–735. [Google Scholar] [CrossRef] [PubMed]
  8. Baddam, P.; Kung, T.; Adesida, A.B.; Graf, D. Histological and Molecular Characterization of the Growing Nasal Septum in Mice. J. Anat. 2020, 238, 751–764. [Google Scholar] [CrossRef] [PubMed]
  9. Parsons, T.E.; Downey, C.M.; Jirik, F.R.; Hallgrimsson, B.; Jamniczky, H.A. Mind the Gap: Genetic Manipulation of Basicranial Growth within Synchondroses Modulates Calvarial and Facial Shape in Mice through Epigenetic Interactions. PLoS ONE 2015, 10, e0118355. [Google Scholar] [CrossRef]
  10. Babula, W.J.; Smiley, G.R.; Dixon, A.D. The Role of the Cartilaginous Nasal Septum in Midfacial Growth. Am. J. Orthod. 1970, 58, 250–263. [Google Scholar] [CrossRef]
  11. Hall, B.K.; Precious, D.S. Cleft Lip, Nose, and Palate: The Nasal Septum as the Pacemaker for Midfacial Growth. Oral Surg. Oral Med. Oral Pathol. Oral Radiol. 2013, 115, 442–447. [Google Scholar] [CrossRef] [PubMed]
  12. Holton, N.E.; Yokley, T.R.; Figueroa, A. Nasal Septal and Craniofacial Form in European- and African-Derived Populations. J. Anat. 2012, 221, 263–274. [Google Scholar] [CrossRef] [PubMed]
  13. Stallman, J.S.; Lobo, J.N.; Som, P.M. The Incidence of Concha Bullosa and Its Relationship to Nasal Septal Deviation and Paranasal Sinus Disease. Am. J. Neuroradiol. 2004, 25, 1613–1618. [Google Scholar]
  14. Smith, K.D.; Edwards, P.C.; Saini, T.S.; Norton, N.S. The Prevalence of Concha Bullosa and Nasal Septal Deviation and Their Relationship to Maxillary Sinusitis by Volumetric Tomography. Int. J. Dent. 2010, 2010, 404982. [Google Scholar] [CrossRef]
  15. Zielnik-Jurkiewicz, B.; Olszewska-Sosińska, O. The Nasal Septum Deformities in Children and Adolescents from Warsaw, Poland. Int. J. Pediatr. Otorhinolaryngol. 2006, 70, 731–736. [Google Scholar] [CrossRef] [PubMed]
  16. Šubarić, M.; Mladina, R. Nasal Septum Deformities in Children and Adolescents: A Cross Sectional Study of Children from Zagreb, Croatia. Int. J. Pediatr. Otorhinolaryngol. 2002, 63, 41–48. [Google Scholar] [CrossRef]
  17. Egeli, E.; Demirci, L.; Yazýcý, B.; Harputluoglu, U. Evaluation of the Inferior Turbinate in Patients with Deviated Nasal Septum by Using Computed Tomography. Laryngoscope 2004, 114, 113–117. [Google Scholar] [CrossRef]
  18. Estomba, C.C.; Schmitz, T.R.; Echeverri, C.C.O.; Reinoso, F.A.B.R.; Velasquez, A.O.; Hidalgo, C.S. Compensatory Hypertrophy of the Contralateral Inferior Turbinate in Patients with Unilateral Nasal Septal Deviation. A Computed Tomography Study. Otolaryngol. Polska 2015, 69, 14–20. [Google Scholar] [CrossRef]
  19. Orhan, I.; Aydin, S.; Ormeci, T.; Yilmaz, F. A Radiological Analysis of Inferior Turbinate in Patients with Deviated Nasal Septum by Using Computed Tomography. Am. J. Rhinol. Allergy 2014, 28, e68–e72. [Google Scholar] [CrossRef]
  20. Gray, L.P.; Brogan, W.F. Septal Deformity Malocclusion and Rapid Maxillary Expansion. Orthodontist 1972, 4, 2–14. [Google Scholar]
  21. Gray, L.P. The Development and Significance of Septal and Dental Deformity from Birth to Eight Years. Int. J. Pediatr. Otorhinolaryngol. 1984, 6, 265–277. [Google Scholar] [CrossRef]
  22. Kim, Y.M.; Rha, K.S.; Weissman, J.D.; Hwang, P.H.; Most, S.P. Correlation of Asymmetric Facial Growth with Deviated Nasal Septum. Laryngoscope 2011, 121, 1144–1148. [Google Scholar] [CrossRef]
  23. Gray, L.P.; Dillon, P.I.; Brogan, W.F.; Henry, P.J. The Development of Septal and Dental Deformity from Birth. Angle Orthod. 1982, 52, 265–278. [Google Scholar]
  24. Vig, K.W. Nasal Obstruction and Facial Growth: The Strength of Evidence for Clinical Assumptions. Am. J. Orthod. Dentofac. Orthop. 1998, 113, 603–611. [Google Scholar] [CrossRef]
  25. D’Ascanio, L.; Lancione, C.; Pompa, G.; Rebuffini, E.; Mansi, N.; Manzini, M. Craniofacial Growth in Children with Nasal Septum Deviation: A Cephalometric Comparative Study. Int. J. Pediatr. Otorhinolaryngol. 2010, 74, 1180–1183. [Google Scholar] [CrossRef]
  26. Freng, A.; Kvam, E.; Kramer, J. Facial Skeletal Dimensions in Patients with Nasal Septal Deviation. Scand. J. Plast. Reconstr. Surg. 1988, 22, 77–81. [Google Scholar] [CrossRef]
  27. Leonardi, R.; Giordano, D.; Maiorana, F.; Spampinato, C. Automatic Cephalometric Analysis: A Systematic Review. Angle Orthod. 2008, 78, 145–151. [Google Scholar] [CrossRef] [PubMed]
  28. Chen, Y.J.; Chen, S.K.; Yao, J.C.C.; Chang, H.F. The Effects of Differences in Landmark Identification on the Cephalometric Measurements in Traditional versus Digitized Cephalometry. Angle Orthod. 2004, 74, 155–161. [Google Scholar] [PubMed]
  29. Dias Da Silveira, H.L.; Dias Silveira, H.E. Reproducibility of Cephalometric Measurements Made by Three Radiology Clinics. Angle Orthod. 2006, 76, 394–399. [Google Scholar]
  30. Hwang, H.-W.; Moon, J.-H.; Kim, M.-G.; Donatelli, R.E.; Lee, S.-J. Evaluation of Automated Cephalometric Analysis Based on the Latest Deep Learning Method. Angle Orthod. 2021, 91, 329–335. [Google Scholar] [CrossRef]
  31. Hwang, H.W.; Park, J.H.; Moon, J.H.; Yu, Y.; Kim, H.; Her, S.B.; Srinivasan, G.; Aljanabi, M.N.A.; Donatelli, R.E.; Lee, S.J. Automated Identification of Cephalometric Landmarks: Part 2-Might It Be Better than Human? Angle Orthod. 2020, 90, 69–76. [Google Scholar] [CrossRef]
  32. Nishimoto, S. Locating Cephalometric Landmarks with Multi-Phase Deep Learning. J. Dent. Health Oral Res. 2023, 4, 1–13. [Google Scholar] [CrossRef]
  33. Kang, S.H.; Jeon, K.; Kang, S.H.; Lee, S.H. 3D Cephalometric Landmark Detection by Multiple Stage Deep Reinforcement Learning. Sci. Rep. 2021, 11, 17509. [Google Scholar] [CrossRef]
  34. Chung, E.J.; Yang, B.E.; Park, I.Y.; Yi, S.; On, S.W.; Kim, Y.H.; Kang, S.H.; Byun, S.H. Effectiveness of Cone-Beam Computed Tomography-Generated Cephalograms Using Artificial Intelligence Cephalometric Analysis. Sci. Rep. 2022, 12, 20585. [Google Scholar] [CrossRef]
  35. Meric, P.; Naoumova, J. Web-Based Fully Automated Cephalometric Analysis: Comparisons between App-Aided, Computerized, and Manual Tracings. Turk. J. Orthod. 2020, 33, 142–149. [Google Scholar] [CrossRef] [PubMed]
  36. Alqahtani, H. Evaluation of an Online Website-Based Platform for Cephalometric Analysis. J. Stomatol. Oral Maxillofac. Surg. 2020, 121, 53–57. [Google Scholar] [CrossRef] [PubMed]
  37. Bulatova, G.; Kusnoto, B.; Grace, V.; Tsay, T.P.; Avenetti, D.M.; Sanchez, F.J.C. Assessment of Automatic Cephalometric Landmark Identification Using Artificial Intelligence. Orthod. Craniofac. Res. 2021, 24, 37–42. [Google Scholar] [CrossRef] [PubMed]
  38. Tsolakis, I.A.; Tsolakis, A.I.; Elshebiny, T.; Matthaios, S.; Palomo, J.M. Comparing a Fully Automated Cephalometric Tracing Method to a Manual Tracing Method for Orthodontic Diagnosis. J. Clin. Med. 2022, 11, 6854. [Google Scholar] [CrossRef]
  39. Lo Giudice, A.; Ronsivalle, V.; Spampinato, C.; Leonardi, R. Fully Automatic Segmentation of the Mandible Based on Convolutional Neural Networks (CNNs). Orthod. Craniofac. Res. 2021, 24, 100–107. [Google Scholar] [CrossRef]
  40. Lo Giudice, A.; Ronsivalle, V.; Gastaldi, G.; Leonardi, R. Assessment of the Accuracy of Imaging Software for 3D Rendering of the Upper Airway, Usable in Orthodontic and Craniofacial Clinical Settings. Prog. Orthod. 2022, 23, 22. [Google Scholar] [CrossRef] [PubMed]
  41. Mosleh, M.A.A.; Baba, M.S.; Malek, S.; Almaktari, R.A. Ceph-X: Development and Evaluation of 2D Cephalometric System. BMC Bioinform. 2016, 17, 499 . [Google Scholar] [CrossRef]
  42. Lin, J.K.; Wheatley, F.C.; Handwerker, J.; Harris, N.J.; Wong, B.J.F. Analyzing Nasal Septal Deviations to Develop a New Classification System: A Computed Tomography Study Using MATLAB and OsiriX. JAMA Facial Plast. Surg. 2014, 16, 183–187. [Google Scholar] [CrossRef]
  43. Dalili, K.Z.; Khademi, J.; Nemati, S.; Niksolat, E. The Effects of Septal Deviation, Concha Bullosa, and Their Combination on the Depth of Posterior Palatal Arch in Cone-Beam Computed Tomography. J. Dent. 2016, 17, 26–31. [Google Scholar]
  44. Lee, J.H.; Yu, H.J.; Kim, M.J.; Kim, J.W.; Choi, J. Automated Cephalometric Landmark Detection with Confidence Regions Using Bayesian Convolutional Neural Networks. BMC Oral Health 2020, 20, 270. [Google Scholar] [CrossRef]
  45. Song, Y.; Qiao, X.; Iwamoto, Y.; Chen, Y.W. Automatic Cephalometric Landmark Detection on X-Ray Images Using a Deep-Learning Method. Appl. Sci. 2020, 10, 2547. [Google Scholar] [CrossRef]
  46. Bao, H.; Zhang, K.; Yu, C.; Li, H.; Cao, D.; Shu, H.; Liu, L.; Yan, B. Evaluating the Accuracy of Automated Cephalometric Analysis Based on Artificial Intelligence. BMC Oral Health 2023, 23, 191. [Google Scholar] [CrossRef] [PubMed]
  47. Barretto, M.D.A.; Melhem-Elias, F.; Deboni, M.C.Z. Methods of Mandibular Condyle Position and Rotation Center Used for Orthognathic Surgery Planning: A Systematic Review. J. Stomatol. Oral Maxillofac. Surg. 2022, 123, 345–352. [Google Scholar]
  48. Ricketts, R.M. Perspectives in the Clinical Application of Cephalometrics. The First Fifty Years. Angle Orthod. 1981, 51, 115–150. [Google Scholar] [PubMed]
  49. Solow, B.; Tallgren, A. Natural Head Position in Standing Subjects. Acta Odontol. Scand. 1971, 29, 591–607. [Google Scholar] [CrossRef]
  50. Billiaert, K.; Al-Yassary, M.; Antonarakis, G.S.; Kiliaridis, S. Measuring the Difference in Natural Head Position between the Standing and Sitting Positions Using an Inertial Measurement Unit. J. Oral Rehabil. 2021, 48, 1144–1149. [Google Scholar] [CrossRef] [PubMed]
  51. Torre, C.; Capasso, R.; Zaghi, S.; Williams, R.; Liu, S.Y.-C. High Incidence of Posterior Nasal Cavity Obstruction in Obstructive Sleep Apnea Patients. Sleep Sci. Pract. 2017, 1, 286. [Google Scholar] [CrossRef]
  52. Wong, E.; Deboever, N.; Sritharan, N.; Singh, N. Laryngopharyngeal Reflux Is Associated with Nasal Septal Deviation. Eur. J. Rhinol. Allergy 2020, 3, 1–3. [Google Scholar] [CrossRef]
  53. Kwon, S.H.; Lee, E.J.; Yeo, C.D.; Kim, M.G.; Kim, J.S.; Noh, S.J.; Kim, E.J.; Kim, S.G.; Lee, J.-H.; Yoo, J.S.; et al. Is Septal Deviation Associated with Headache? Medicine 2020, 99, e20337. [Google Scholar] [CrossRef]
  54. Orlandi, R.R. A Systematic Analysis of Septal Deviation Associated with Rhinosinusitis. Laryngoscope 2010, 120, 1687–1695. [Google Scholar] [CrossRef]
  55. Grymer, L.F.; Bosch, C. The Nasal Septum and the Development of the Midface. A Longitudinal Study of a Pair of Monozygotic Twins. Rhinology 1997, 35, 6–10. [Google Scholar]
  56. Pirsig, W. Historical Notes and Actual Observations on the Nasal Septal Abscess Especially in Children. Int. J. Pediatr. Otorhinolaryngol. 1984, 8, 43–54. [Google Scholar] [CrossRef] [PubMed]
  57. Kim, J.; Kim, S.W.; Kim, S.W.; Cho, J.H.; Park, Y.J. Role of the Sphenoidal Process of the Septal Cartilage in the Development of Septal Deviation. Otolaryngol. Head Neck Surg. 2012, 146, 151–155. [Google Scholar] [CrossRef] [PubMed]
  58. Mays, S. Nasal Septal Deviation in a Mediaeval Population. Am. J. Phys. Anthropol. 2012, 148, 319–326. [Google Scholar] [CrossRef] [PubMed]
  59. Peck, H.; Peck, S. A Concept of Facial Esthetics. Angle Orthod. 1970, 40, 284–317. [Google Scholar]
  60. Masoud, M.I.; Bansal, N.; Castillo, J.C.; Manosudprasit, A.; Allareddy, V.; Haghi, A.; Hawkins, H.C.; Otárola-Castillo, E. 3D Dentofacial Photogrammetry Reference Values: A Novel Approach to Orthodontic Diagnosis. Eur. J. Orthod. 2017, 39, 215–225. [Google Scholar] [CrossRef]
  61. Tanikawa, C. Facial Morphospace: A Clinical Quantitative Analysis of the Three-Dimensional Face in Patients with Cleft Lip and Palate. Plast. Aesthet. Res. 2020, 7, 48. [Google Scholar] [CrossRef]
  62. Tanikawa, C.; Akcam, M.O.; Takada, K. Quantifying Faces Three-Dimensionally in Orthodontic Practice. J. Cranio Maxillofac. Surg. 2019, 47, 867–875. [Google Scholar] [CrossRef]
  63. Tomaka, A.A.; Luchowski, L.; Pojda, D.; Tarnawski, M.; Domino, K. The Dynamics of the Stomatognathic System from 4D Multimodal Data. In Multiscale Locomotion: Its Active-Matter Addressing Physical Principles; Gadomski, A., Ed.; Publishing Department of the UTP University of Science & Technology: Bydgoszcz, Poland, 2019. [Google Scholar]
  64. Joda, T.; Gallucci, G.O. The Virtual Patient in Dental Medicine. Clin. Oral Implants. Res. 2015, 26, 725–726. [Google Scholar] [CrossRef] [PubMed]
  65. Tanikawa, C.; Yamashiro, T. Development of Novel Artificial Intelligence Systems to Predict Facial Morphology after Orthognathic Surgery and Orthodontic Treatment in Japanese Patients. Sci. Rep. 2021, 11, 15853. [Google Scholar] [CrossRef] [PubMed]
  66. Resnick, C.M.; Dang, R.R.; Glick, S.J.; Padwa, B.L. Accuracy of Three-Dimensional Soft Tissue Prediction for Le Fort I Osteotomy Using Dolphin 3D Software: A Pilot Study. Int. J. Oral Maxillofac. Surg. 2017, 46, 289–295. [Google Scholar] [CrossRef] [PubMed]
  67. Elshebiny, T.; Morcos, S.; Mohammad, A.; Quereshy, F.; Valiathan, M. Accuracy of Three-Dimensional Soft Tissue Prediction in Orthognathic Cases Using Dolphin Three-Dimensional Software. J. Craniofacial Surg. 2019, 30, 525–528. [Google Scholar] [CrossRef]
Jcm 12 06621 g001
Figure 1. Cephalogram annotation example (A), showing major landmarks used in this study (B).
Figure 1. Cephalogram annotation example (A), showing major landmarks used in this study (B).
Jcm 12 06621 g001
Jcm 12 06621 g002
Figure 2. Sample of NSD measurements conducted in one patient: (A) perpendicular plate–vomer (PPV) measurement; (B) septal deviation angle (SDA) measurement.
Figure 2. Sample of NSD measurements conducted in one patient: (A) perpendicular plate–vomer (PPV) measurement; (B) septal deviation angle (SDA) measurement.
Jcm 12 06621 g002
Jcm 12 06621 g003
Figure 3. Part of sample, automatic, AI-derived cephalometric report. Patient: 18-year-old male with class III malocclusion, with graphical representations of the results regarding to norm range (column “Graph”) and clinical comments.
Figure 3. Part of sample, automatic, AI-derived cephalometric report. Patient: 18-year-old male with class III malocclusion, with graphical representations of the results regarding to norm range (column “Graph”) and clinical comments.
Jcm 12 06621 g003
Jcm 12 06621 g004
Figure 4. Plot displaying the correlation between SDA and PPV measurements.
Figure 4. Plot displaying the correlation between SDA and PPV measurements.
Jcm 12 06621 g004
Jcm 12 06621 g005
Figure 5. Plot displaying the correlations between HAA and NSD parameters: (A) SDA (r = −0.218, p = 0.039); (B) PPV (r = −0.186, p = 0.079).
Figure 5. Plot displaying the correlations between HAA and NSD parameters: (A) SDA (r = −0.218, p = 0.039); (B) PPV (r = −0.186, p = 0.079).
Jcm 12 06621 g005
Table 1. Cephalometric landmarks used in analyses (part of).
Table 1. Cephalometric landmarks used in analyses (part of).
LandmarkDefinition
SSellaMidpoint of the sella turcica
CoCondylionThe extreme superior point on the condylar head
ANSAnterior Nasal SpineTip of the bony anterior nasal spine in the midline
NNasionThe most anterior point of the frontonasal suture
APoint AThe innermost point on the contour of the maxilla between the anterior nasal spine and the alveolar crest
BPoint BThe most posterior point in the concavity along the anterior border of the symphysis
GoGonionThe most prominent point on the angle of the mandible formed by the junction of the ramus and the body of the mandible
GnGnathionThe most inferior bony point of the mandible
LlLower LipThe most anterior point of the lower lip
MeMentonThe most inferior point of the mandibular symphysis in the midline
NNasionThe most anterior point of the frontonasal suture
OrOrbitaleThe lowest point on the inferior margin of the orbit
PPorionThe central point on the upper margin of the external auditory meatus
PrProsthionThe point of alveolar contact with the upper central incisor
PgPogonionThe most anterior point on the contour of the bony chin
Table 2. Sample of correlation analysis performed: NSD/HAA, and NSD/results of Björk–Jarabak cephalometric analysis.
Table 2. Sample of correlation analysis performed: NSD/HAA, and NSD/results of Björk–Jarabak cephalometric analysis.
ParameterSDA (°)PPV (mm)
Spearman’s Rank Correlation CoefficientSpearman’s Rank Correlation Coefficient
HINGE AXIS ANGLE (°)r= −0.218, p = 0.039 *r = −0.186, p = 0.079
Björk–Jarabak cephalometric analysis
SADDLE ANGLE (°)r = 0.017, p = 0.875r = −0.02, p = 0.853
ARTICULAR ANGLE (°)r = 0.103, p = 0.335r = 0.104, p = 0.329
GONIAL ANGLE (°)r = −0.154, p = 0.148r = −0.129, p = 0.226
SUM OF ANGLES (°)r = 0.044, p = 0.679r = 0.03. p = 0.782
UPPER GONIAL ANGLE (°)r = −0.175, p = 0,099r = −0.183, p = 0.084
LOWER GONIAL ANGLE (°)r = −0.002, p = 0.984r = 0.023, p = 0.826
ANT. CRANIAL BASE (mm)r = −0.011, p = 0.915r = 0.018, p = 0.67
POST. CRANIAL BASE (mm)r = 0.018, p = 0.866r = 0.052, p = 0.626
RAMUS HEIGHT (mm)r = 0.024, p = 0.825r = 0.099, p = 0.353
MANDIBULAR BODY (mm)r = −0.02, p = 0.851r = −0.004, p = 0.972
POST. FACE HEIGHT (mm)r = 0.031, p = 0.77r = 0.096, p = 0.368
ANT. FACE HEIGHT (mm)r = −0.016, p = 0.879r = 0.043, p = 0.69
PFH:AFH (%)r = 0.023, p = 0.832r = 0.053, p = 0.623
ACB:MAND.BODY (%)r = 0.004, p = 0.97r = −0.023, p = 0.832
UI to SN (°)r = −0.096, p = 0.37r = −0.094, p = 0.376
UI to FH (°)r = −0.073, p = 0.497r = −0.074, p = 0.486
UPPER FACE HEIGHT (%)r = −0.127, p = 0.233r = −0.057, p = 0.596
LOWER FACE HEIGHT (%)r = 0.125, p = 0.24r = 0.065, p = 0.544
*—statistically significant relationship (p < 0.05). Abbreviations: SDA—septal deviation angle; PPV—perpendicular plate–vomer measurement.
Table 3. Results of repeatability of manual measurements.
Table 3. Results of repeatability of manual measurements.
ParameterMeasurement I
(Mean ± SD)		Measurement II
(Mean ± SD)		ICC95% CIAgreement (Cicchetti)Agreement (Koo and Li)
PPV (mm)5.19 ± 2.65.28 ± 2.490.9740.9370.990ExcellentPPV (mm)
SDA (°)10.78 ± 5.6910.85 ± 5.730.9720.9310.989ExcellentSDA (°)
Abbreviations: PPV—perpendicular plate–vomer measurement; SDA—septal deviation angle.
Table 4. List of parameters with low agreement in repeated measurements.
Table 4. List of parameters with low agreement in repeated measurements.
Parameter Measurement 1
(Mean ± SD)		Measurement 2
(Mean ± SD)		ICC95% CIAgreement (Cicchetti)Agreement (Koo and Li)
LI-N-B 21.73 ± 8.4136.67 ± 45.540.000−0.5360.548PoorPoor
Pr-N-A 1.76 ± 0.661.85 ± 0.540.302−0.2870.730PoorPoor
Abbreviations: SD—Standard Deviation; I—Incisor; NB—Nasion–B-Point; Pr—Prosthion; A—A-Point; ICC—Interclass Correlation Coefficient; CI—Confidence Interval.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Kazimierczak, N.; Kazimierczak, W.; Serafin, Z.; Nowicki, P.; Lemanowicz, A.; Nadolska, K.; Janiszewska-Olszowska, J. Correlation Analysis of Nasal Septum Deviation and Results of AI-Driven Automated 3D Cephalometric Analysis. J. Clin. Med. 2023, 12, 6621. https://doi.org/10.3390/jcm12206621

AMA Style

Kazimierczak N, Kazimierczak W, Serafin Z, Nowicki P, Lemanowicz A, Nadolska K, Janiszewska-Olszowska J. Correlation Analysis of Nasal Septum Deviation and Results of AI-Driven Automated 3D Cephalometric Analysis. Journal of Clinical Medicine. 2023; 12(20):6621. https://doi.org/10.3390/jcm12206621

Chicago/Turabian Style

Kazimierczak, Natalia, Wojciech Kazimierczak, Zbigniew Serafin, Paweł Nowicki, Adam Lemanowicz, Katarzyna Nadolska, and Joanna Janiszewska-Olszowska. 2023. "Correlation Analysis of Nasal Septum Deviation and Results of AI-Driven Automated 3D Cephalometric Analysis" Journal of Clinical Medicine 12, no. 20: 6621. https://doi.org/10.3390/jcm12206621

APA Style

Kazimierczak, N., Kazimierczak, W., Serafin, Z., Nowicki, P., Lemanowicz, A., Nadolska, K., & Janiszewska-Olszowska, J. (2023). Correlation Analysis of Nasal Septum Deviation and Results of AI-Driven Automated 3D Cephalometric Analysis. Journal of Clinical Medicine, 12(20), 6621. https://doi.org/10.3390/jcm12206621

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop