Next Article in Journal
Advanced Nanotechnological Approaches for Biofilm Prevention and Control
Previous Article in Journal
The Ballpark Effect: Spatial-Data-Driven Insights into Baseball’s Local Economic Impact
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Applied Sciences
Volume 14
Issue 18
10.3390/app14188135
applsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Comparison of Automatic Bone Age Assessments between the Left and Right Hands: A Tool for Filtering Measurement Errors

by
Kyu-Chong Lee
1,
Chang Ho Kang
1,
Kyung-Sik Ahn
1,
Kee-Hyoung Lee
2,
Jae Joon Lee
3,
Kyu Ran Cho
1 and
Saelin Oh
1,*
1
Department of Radiology, Anam Hospital, Korea University College of Medicine, Seoul 02841, Republic of Korea
2
Department of Pediatrics, Anam Hospital, Korea University College of Medicine, Seoul 02841, Republic of Korea
3
Crescom, Seongnam 13493, Republic of Korea
*
Author to whom correspondence should be addressed.
Appl. Sci. 2024, 14(18), 8135; https://doi.org/10.3390/app14188135
Submission received: 2 July 2024 / Revised: 2 August 2024 / Accepted: 7 August 2024 / Published: 10 September 2024
(This article belongs to the Special Issue Artificial Intelligence for Pediatric Monitoring, Diagnosis, and Treatment)
Browse Figures
Versions Notes

Abstract

:
This study aimed to investigate whether the left and right hands yield the same bone age using the automated bone age assessment (BAA) system and proposed the right-hand BAA as a tool for filtering out measurement errors. The Bland–Altman, Passing–Bablok, and Spearman correlation coefficients were analyzed to compare the automated BAA results for each hand. The absolute difference between each hand obtained by the model (ADBH model) was calculated. The mean absolute difference (MAD) was estimated between the automatic BAA results for each hand and the reference standard. The mean of the ADBH model was 0.23 ± 0.19 years; 92.2% of the participants showed an ADBH model result of <0.5 years. The Passing–Bablok regression analysis revealed an excellent overall correlation between the BAAs of both hands. Of the total cases, 59 participants showed an ADBH model result >0.5 years, with a MAD between the model and the reference standard of 0.409 years for the left hand and 0.424 years for the right hand; both MADs were higher than those of previous studies using the same model. Given the excellent overall correlation of the BAA between both hands using the model, the high ADBH model value may indicate BAA measurement errors and serve as a cue for manual supervision.

1. Introduction

A bone age assessment (BAA) is a clinical procedure used to evaluate skeletal maturity in pediatric patients [1]. Discrepancies between bone and chronological ages over two standard deviations (SDs) indicate growth disorders [2,3]. In clinical practice, left-hand and wrist radiography-based BAAs are widely used, including the Greulich–Pyle (GP) method [4] and the Tanner–Whitehouse 3 (TW3) approach [5]. The GP method is an atlas-based method that compares a patient’s radiograph with an atlas of representative age, whereas TW3 is a scoring system based on the osseous stages and events at each level. To overcome some issues with these methods, including inter-/intra-observer variability and dependency on accuracy [6,7,8], researchers have attempted to develop an automated BAA approach that demonstrates high accuracy, reproducibility, and time efficiency [9,10,11,12].
Most artificial intelligence (AI)-based automated BAA approaches are derived from the whole hand [13,14,15,16,17] or region-based convolutional neural networks (CNNs) that extract the region of interest (ROI) [18] from left-hand radiographs. These automated BAAs are validated as reference data against manual ones by experts analyzing left-hand radiographs using the GP or TW method. Although AI performance has improved and the mean absolute difference (MAD) values have gradually decreased to an acceptable range, the risk of measurement errors persists. These errors can potentially be mitigated through consistent results from repeated measurements [19]. Given the ethical considerations against obtaining a second radiograph from each child, we propose using the right-hand BAA as a validation tool during the initial evaluation.
Left-handed radiographs are preferred over right-handed ones for detecting bone age [20]. One rationale is that since most people are right-handed, the right hand is more prone to injuries than the left hand [4], making the bone ossification levels of the non-dominant hand a more accurate reflection of true bone maturity [21]. Another reason is that in the early 1900s, in a conference involving physical anthropologists, it was decided that physical measurements should be taken on the left rather than on the right side of the body [4]. Roche [22] observed that the left hand was more advanced than the right hand, whereas Baer and Djrkatz [23] found no such effect.
Current developments of artificial intelligence have come to the level of recognizing human activity and their physical, motor, and mental statuses while moving and acting in physical spaces [24]. It is time to focus on how to use it efficiently in real practice without missing inevitable errors. As a simple tool for error filtering, we propose right-hand BAA as a validation tool. The aim of this study was to introduce an upgraded automatic hand bone alignment technique and to propose right-hand BAA as a tool for filtering out measurement errors.

2. Materials and Methods

2.1. Model Development

Our study included 757 children (321 boys and 436 girls) who visited the pediatric department and underwent bilateral radiography at our institution between January and December 2023. All acceptable left- and right-hand images were scanned, yielding 1514 images. Patients with congenital anomalies (including Down syndrome, Noonan syndrome, congenital adrenal hyperplasia, or methylmalonic acidemia) or those with poor imaging quality were excluded from the study.
The digital images were processed using an automated MediAI-BA method (Crescom, Seongnam-Si, Republic of Korea). The methodology was previously described by Lee et al. [11]; briefly, the model relies on hybrid TW3 and GP AI-based automatic bone age measurements. The CNN algorithm automatically classified multiple ROIs and holistic hand images based on maturity (Figure 1A). First, the algorithm automatically identifies multiple regions based on TW3. Next, it automatically categorizes each ROI and holistic hand image based on maturity. The final bone age estimation involves integrating features from both ROIs and holistic hand images. The proposed method used eight regions (the radius, ulna, distal phalanges, middle phalanges, proximal phalanges, metacarpal of the third digit, metacarpal of the first digit, and carpal region) and was classified into 34 stages with 6-month intervals from 1.5 to 18 years to assess bone maturity. The model was upgraded with data augmentation techniques and the automatic hand bone alignment technique. The automatic hand bone alignment technique automatically indicates three regions (distal phalanges, the metacarpal of the third digit, and the metacarpal of the first digit) and rotates, aligns, and flips the image to match the regions’ orientation (Figure 1B). The model additionally applied the data augmentation technique, artificially increasing the training set by creating modified copies of a dataset using existing data. The model achieved human expert-level accuracy, with an upper 95% confidence interval (CI) of <0.5 years for the MAD between the BAA models and the reference standard [11,25].
Cases with absolute differences over 6 months between the left- and right-handed BAAs were analyzed separately. The reference standard was the average independent bone age estimated by the two reviewers. Reviewer 1 was a pediatric radiologist with 11 years of clinical experience; reviewer 2 was a musculoskeletal radiologist with 9 years of clinical experience. The two reviewers independently estimated the bone age based on the GP atlas, reporting their estimates to the nearest year with a decimal place. They intended to adhere to the GP standard for the bone age assessment whenever possible; however, they were allowed to use the median age according to their experience level. If there was a discrepancy of >2 years, the image was re-evaluated until a consensus was reached.

2.2. Statistical Analyses

All statistical analyses were performed using either SPSS 20.0 (International Business Machines Corporation, Armonk, NY, USA) or MedCalc 11.3 (Med Calc Software bvba, Mariakerke, Belgium) statistical software packages. Values for continuous variables were represented as mean and standard deviations (SDs), while those for categorical variables as number and percentage. The absolute difference between each hand BAA by the model (ADBH model) was calculated with a 95% CI. Bland–Altman, Passing–Bablok, and Spearman correlation coefficients were analyzed to compare the automatic BAA results for each hand [26]. To compare the patients with an ADBH model result of ≤0.5 years and patients with an ADBH model result of >0.5 years, differences in continuous variables were analyzed using Student’s t-test; differences in categorical variables were tested using the Chi-squared and Fisher’s exact tests. To validate the reference standard, the intraclass correlation coefficient (ICC) was measured to assess the inter-observer agreement among the three reviewers by considering the following values and levels of agreement: 0–0.20, poor; 0.21–0.40, fair; 0.41–0.60, moderate; 0.61–0.80, substantial; and 0.80–1, almost perfect [27]. To measure the predictive accuracy of each hand’s BAA, the MAD between each hand’s BAA and the reference standard was measured with a 95% CI. The reference standard, BAA, and MAD of each hand were compared using paired t-tests. The ADBH model results and absolute difference between each hand’s BAA using the reference standard (ADBH-RF) were compared using a paired t-test. Statistical significance was set at p = 0.05.

3. Results

The mean chronological age ± SD of the participants was 8.75 ± 2.66 years (male: 9.49 ± 3.06; female: 8.21 ± 2.18) (Table 1). According to the age distribution, the largest group included 498 participants (65.8%) aged ≥5 years but <10 years, which refers to most ages within the early and mid-puberty stages according to the major developmental stages of the skeletal system [pre-puberty (males ≤9 years; females ≤7 years); early and mid-puberty (9 years < males ≤ 14 years; 7 years < females ≤ 13 years); late puberty (14 years < males ≤ 16 years; 13 years < females ≤ 15 years); and post-puberty (males > 16 years; females > 15 years)] [13]. The mean bone ages ± SDs assessed by the model were 9.15 ± 3.21 for the left hand and 9.20 ± 3.23 for the right hand (p < 0.001). The mean of the ADBH model was 0.23 ± 0.19 years (95% CI: 0.22–0.25); 92.2% of participants showed an ADBH model result of <0.5 years.
The results of the Bland–Altman analysis, which was used to assess the agreement between each hand’s BAA, are shown in Figure 2. The mean difference between both hands’ BAAs was 0.05 years (95% CI, 0.63–0.53); 4.6% (35/767) of the data were outside the limits of agreement. The Passing–Bablok regression analysis showed that the overall correlation of the BAA between both hands was excellent, having no significant deviation from linearity in this association (p = 0.15) (Figure 3). According to the model, the bone age of the right hand was approximately 1 month more advanced than that of the left. The Pearson’s correlation coefficient was r = 0.995 (p  <  0.001) for the BAA between both hands.
Among all of the participants, 59 (7.8%) showed an ADBH model result of >0.5 years (Table 2). There was no statistical difference in the chorological age, sex, and hand BAA results between patients with an ADBH model result of ≤0.5 years and those with an ADBH model result of >0.5 years. There was a statistical difference according to developmental stage; the majority of patients in the pre-puberty and early/mid-puberty groups showed an ADBH model result of <0.5 years.
With an ICC (95% CI) of 0.997 (95% CI, 0.996–0.998), the agreement between the two reviewers was sufficiently high to justify using the average BAA value as a reference standard. The reference standard showed no statistical differences between the BAAs of each hand (Table 3). The MAD between the model and the reference standard was 0.409 years for the left hand (95% CI, 0.322–0.467; range, 0–1.75) and 0.424 years for the right hand (95% CI, 0.339–0.510; range, 0–1.33); both were higher than those reported in previous studies using the same model [11,24]. The mean ADBH model score (0.667; 95% CI, 0.638–0.695) was significantly higher than that of the ADBH-RF (0.102; 95% CI, 0.059–0.144) (p < 0.001).

4. Discussion

Our study showed an excellent overall correlation of BAA using the model between both hands, refuting the findings of Roche [23]. Considering that approximately 10% of the population is assumed to be left-handed [28], it can be concluded that the dominant hand appears to mature at the same rate as the non-dominant hand, and that the right-hand BAA can be used as a validation tool. Among the total participants, 59 (7.8%) showed an ADBH model score of >0.5 years, with higher MAD values (left: 0.409, right: 0.424) compared to previous studies using the same model (0.39 and 0.37, respectively) [11,24]; their highest values were 1.75 years for the left hand and 1.33 years for the right hand. Although this result does not mean that a higher ADBH model score indicates a measurement error, it could serve as a cue for manual supervision.
Since the early 1990s, researchers have attempted to automate BAA using computers. In 1992, the computer-assisted skeletal age score (CASAS) was presented, a semi-automatic method that required a significant amount of manual labor [29]. In 1997, the automated computer-aided skeletal maturation assessment system (CASMAS) was also presented, which was reported to automatically analyze 90% of films but required a manual rating of 10% [30]. Fully automated BAA has been routinely used in clinical practice since 2009, when BoneXpert (Visiana Aps, Holte, Denmark; http://www.boneexpert.com, accesses on 15 June 2024) was introduced [31]. Since then, various automatic and deep learning methods for BAA have been proposed, which have demonstrated high accuracy, reproducibility, and time efficiency [9,10,11,12]. Despite these advantages, AI-based methods are still not suitable for clinical use due to the risk of large measurement errors. These errors are usually rare under ideal conditions and may not have a significant impact on the overall error measure, especially if error measures such as the MAD are used. Although the percentages were low, the figures and tables of some studies showed a certain percentage of measurement errors [9,10,17,32].
Radiologists using automated BAA, which can completely replace experts, tend to change the bone age value very rarely: 40% of users never change it, while 43% change less than 5% of the cases [33]. If bone age assessment is taken over by automated methods, there is a risk for large measurement errors, even when used under supervision. These types of measurement errors can be filtered out by obtaining identical results from repeated measurements [19], overcoming the effects of slightly different hand positions and the effects of using different X-ray equipment with different settings. Since it is unethical to repeatedly expose the same children to X-rays, previous studies have used retrospective longitudinal X-ray series taken, for example, at 1-year intervals, to establish a level of precision [19,31]. Another study by Martin et al. used differences between the left and right hands of normal children [34]. They found the upper limit of its between-image precision error to be 0.18 years, which was in good agreement with previous studies concerned with the BAA rating precision [19,31]. To the best of our knowledge, our study is the first to compare the automated BAA results of both hands in various groups of patients visiting a tertiary hospital.
Recently upgraded with a self-validation mechanism, BoneXpert rejects the images if the bone age deviates from the average bone age determined by tubular bones [32]. Even when using a self-validation mechanism, the method failed to reject cases where wrong bone ages were assigned, with 1.5% of patients showing an absolute difference between the automated method and manual assessment exceeding 1.8 years. Considering the self-validation mechanism proposed by BoneXpert, an automated BAA method approaching only the holistic hand image may have limitations in filtering out large measurement errors.
Our hybrid model addressed the limitations of GP and TW3 by focusing on regions highly pertinent to changes in bone maturity and employing more refined maturity stages than TW3, leading to a robust and precise bone age estimation [11]. Our model integrated bone age assessments using detailed ROIs alongside holistic images. In comparison, the current commercial automated BAA system, BoneXpert version 3.0 (launched September 2019), evaluates bone age using both GP and alternative TW2 methods [31], similar to our approach. BoneXpert utilizes a feature extraction technique that reconstructs the border of 15 bones, including metacarpals, phalangeal bones, distal radius, and ulna [35], distinguishing it from our model. The model has been upgraded to improve user convenience and accuracy, including data augmentation techniques and the automatic hand bone alignment technique.
Our study had a few limitations. First, the reference standard was based on the GP atlas, which was inherently limited to children. Differences in the nutritional and ethnic makeup of children in the 1930s and 1940s were used to generate standards. Additionally, although previous studies have reported racial differences among growth patterns in certain age groups [36,37], cases were limited to patients of a single ethnicity. Finally, it would have been better to compare each ROI bone age of both hands, considering that skeletal maturation can vary within the same age.
Our study showed an excellent overall correlation of BAAs between both hands using the model and provided the possibility of using the right-hand BAA as a validation tool. Perceptible differences between each hand may indicate a large measurement error and thus may be a signal for manual supervision.

Author Contributions

Conceptualization, S.O. and C.H.K.; formal analysis, S.O. and K.-C.L.; investigation, S.O. and K.-C.L.; methodology, S.O. and J.J.L.; supervision, C.H.K., K.-H.L., and K.R.C.; writing—original draft, S.O. and K.-C.L.; writing—review and editing, S.O., C.H.K., and K.-S.A. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by a grant of Korea University Anam Hospital, Seoul, Repulic of Korea (grant number: K2409051).

Institutional Review Board Statement

This study was approved by the Institutional Review Board and ethics committee of our institution (IRB No. 2024AN0159).

Informed Consent Statement

Written informed consent was waived by the Institutional Review Board.

Data Availability Statement

The data that support the findings of this study are available on request from the corresponding author (S.O.).

Acknowledgments

Kyung Sook Yang (Department of Biostatistics, Korea University College of Medicine) kindly provided statistical advice for this manuscript.

Conflicts of Interest

C.H.K. is a stockholder and J.J.L. is a founder of Crescom Inc. (Seongnam, Repulic of Korea), a startup company, the products and services of which are related to the topic of this article. The other authors declare no conflicts of interest or organizational support for this study.

References

  1. Zerin, J.M.; Hernandez, R.J. Approach to skeletal maturation. Hand Clin. 1991, 7, 53–62. [Google Scholar] [CrossRef] [PubMed]
  2. Oh, M.S.; Kim, S.; Lee, J.; Lee, M.S.; Kim, Y.J.; Kang, K.S. Factors associated with Advanced Bone Age in Overweight and Obese Children. Pediatr. Gastroenterol. Hepatol. Nutr. 2020, 23, 89–97. [Google Scholar] [CrossRef] [PubMed]
  3. Kim, D.; Cho, S.Y.; Maeng, S.H.; Yi, E.S.; Jung, Y.J.; Park, S.W.; Sohn, Y.B.; Jin, D.K. Diagnosis and constitutional and laboratory features of Korean girls referred for precocious puberty. Korean J. Pediatr. 2012, 55, 481–486. [Google Scholar] [CrossRef] [PubMed]
  4. Greulich, W.W.; Pyle, S.I. Radiographic Atlas of Skeletal Development of the Hand and Wrist, 2nd ed.; Stanford University Press: Stanford, CA, USA, 1959. [Google Scholar]
  5. Malina, R.M.; Beunen, G.P. Assessment of skeletal maturity and prediction of adult height (TW3 method). Third edition. Am. J. Hum. Biol. 2002, 14, 788–789. [Google Scholar] [CrossRef]
  6. Berst, M.J.; Dolan, L.; Bogdanowicz, M.M.; Stevens, M.A.; Chow, S.; Brandser, E.A. Effect of knowledge of chronologic age on the variability of pediatric bone age determined using the Greulich and Pyle standards. AJR Am. J. Roentgenol. 2001, 176, 507–510. [Google Scholar] [CrossRef] [PubMed]
  7. Bull, R.K.; Edwards, P.D.; Kemp, P.M.; Fry, S.; Hughes, I.A. Bone age assessment: A large scale comparison of the Greulich and Pyle, and Tanner and Whitehouse (TW2) methods. Arch. Dis. Child. 1999, 81, 172–173. [Google Scholar] [CrossRef] [PubMed]
  8. Lee, B.D.; Lee, M.S. Automated Bone Age Assessment Using Artificial Intelligence: The Future of Bone Age Assessment. Korean J. Radiol. 2021, 22, 792–800. [Google Scholar] [CrossRef] [PubMed]
  9. Kim, J.R.; Shim, W.H.; Yoon, H.M.; Hong, S.H.; Lee, J.S.; Cho, Y.A.; Kim, S. Computerized Bone Age Estimation Using Deep Learning Based Program: Evaluation of the Accuracy and Efficiency. AJR Am. J. Roentgenol. 2017, 209, 1374–1380. [Google Scholar] [CrossRef] [PubMed]
  10. Eng, D.K.; Khandwala, N.B.; Long, J.; Fefferman, N.R.; Lala, S.V.; Strubel, N.A.; Milla, S.S.; Filice, R.W.; Sharp, S.E.; Towbin, A.J.; et al. Artificial Intelligence Algorithm Improves Radiologist Performance in Skeletal Age Assessment: A Prospective Multicenter Randomized Controlled Trial. Radiology 2021, 301, 692–699. [Google Scholar] [CrossRef]
  11. Lee, K.C.; Lee, K.H.; Kang, C.H.; Ahn, K.S.; Chung, L.Y.; Lee, J.J.; Hong, S.J.; Kim, B.H.; Shim, E. Clinical Validation of a Deep Learning-Based Hybrid (Greulich-Pyle and Modified Tanner-Whitehouse) Method for Bone Age Assessment. Korean J. Radiol. 2021, 22, 2017–2025. [Google Scholar] [CrossRef]
  12. Booz, C.; Yel, I.; Wichmann, J.L.; Boettger, S.; Al Kamali, A.; Albrecht, M.H.; Martin, S.S.; Lenga, L.; Huizinga, N.A.; D’Angelo, T.; et al. Artificial intelligence in bone age assessment: Accuracy and efficiency of a novel fully automated algorithm compared to the Greulich-Pyle method. Eur. Radiol. Exp. 2020, 4, 6. [Google Scholar] [CrossRef] [PubMed]
  13. Lee, H.; Tajmir, S.; Lee, J.; Zissen, M.; Yeshiwas, B.A.; Alkasab, T.K.; Choy, G.; Do, S. Fully Automated Deep Learning System for Bone Age Assessment. J. Digit. Imaging 2017, 30, 427–441. [Google Scholar] [CrossRef] [PubMed]
  14. Spampinato, C.; Palazzo, S.; Giordano, D.; Aldinucci, M.; Leonardi, R. Deep learning for automated skeletal bone age assessment in X-ray images. Med. Image Anal. 2017, 36, 41–51. [Google Scholar] [CrossRef] [PubMed]
  15. Ren, X.; Li, T.; Yang, X.; Wang, S.; Ahmad, S.; Xiang, L.; Stone, S.R.; Li, L.; Zhan, Y.; Shen, D.; et al. Regression Convolutional Neural Network for Automated Pediatric Bone Age Assessment from Hand Radiograph. IEEE J. Biomed. Health Inform. 2019, 23, 2030–2038. [Google Scholar] [CrossRef] [PubMed]
  16. Mutasa, S.; Chang, P.D.; Ruzal-Shapiro, C.; Ayyala, R. MABAL: A Novel Deep-Learning Architecture for Machine-Assisted Bone Age Labeling. J. Digit. Imaging 2018, 31, 513–519. [Google Scholar] [CrossRef] [PubMed]
  17. Tajmir, S.H.; Lee, H.; Shailam, R.; Gale, H.I.; Nguyen, J.C.; Westra, S.J.; Lim, R.; Yune, S.; Gee, M.S.; Do, S. Artificial intelligence-assisted interpretation of bone age radiographs improves accuracy and decreases variability. Skelet. Radiol. 2019, 48, 275–283. [Google Scholar] [CrossRef] [PubMed]
  18. Shin, N.Y.; Lee, B.D.; Kang, J.H.; Kim, H.R.; Oh, D.H.; Lee, B.I.; Kim, S.H.; Lee, M.S.; Heo, M.S. Evaluation of the clinical efficacy of a TW3-based fully automated bone age assessment system using deep neural networks. Imaging Sci. Dent. 2020, 50, 237–243. [Google Scholar] [CrossRef]
  19. Martin, D.D.; Sato, K.; Sato, M.; Thodberg, H.H.; Tanaka, T. Validation of a new method for automated determination of bone age in Japanese children. Horm. Res. Paediatr. 2010, 73, 398–404. [Google Scholar] [CrossRef] [PubMed]
  20. Wittschieber, D.; Vieth, V.; Wierer, T.; Pfeiffer, H.; Schmeling, A. Cameriere’s approach modified for pelvic radiographs: A novel method to assess apophyseal iliac crest ossification for the purpose of forensic age diagnostics. Int. J. Leg. Med. 2013, 127, 825–829. [Google Scholar] [CrossRef]
  21. Li, Z.; Chen, W.; Ju, Y.; Chen, Y.; Hou, Z.; Li, X.; Jiang, Y. Bone age assessment based on deep neural networks with annotation-free cascaded critical bone region extraction. Front. Artif. Intell. 2023, 6, 1142895. [Google Scholar] [CrossRef]
  22. Roche, A.F.; Chumlea, W.; Thissen, D. Assessing the Skeletal Maturity of the Hand-Wrist: Fels Method; Thomas: Springfield, IL, USA, 1988. [Google Scholar]
  23. Baer, M.J.; Djrkatz, J. Bilateral asymmetry in skeletal maturation of the hand and wrist: A roentgenographic analysis. Am. J. Phys. Anthropol. 1957, 15, 181–196. [Google Scholar] [CrossRef] [PubMed]
  24. Germanese, D.; Colantonio, S.; Del Coco, M.; Carcagnì, P.; Leo, M. Computer Vision Tasks for Ambient Intelligence in Children’s Health. Information 2023, 14, 548. [Google Scholar] [CrossRef]
  25. Kim, S.U.; Oh, S.; Lee, K.H.; Kang, C.H.; Ahn, K.S. Improvement of Bone Age Assessment Using a Deep Learning Model in Young Children: Significance of Carpal Bone Analysis. Iran. J. Radiol. 2023, 20, e136311. [Google Scholar] [CrossRef]
  26. Gerke, O. Reporting Standards for a Bland-Altman Agreement Analysis: A Review of Methodological Reviews. Diagnostics 2020, 10, 334. [Google Scholar] [CrossRef] [PubMed]
  27. Benchoufi, M.; Matzner-Lober, E.; Molinari, N.; Jannot, A.S.; Soyer, P. Interobserver agreement issues in radiology. Diagn. Interv. Imaging 2020, 101, 639–641. [Google Scholar] [CrossRef]
  28. Papadatou-Pastou, M.; Ntolka, E.; Schmitz, J.; Martin, M.; Munafò, M.R.; Ocklenburg, S.; Paracchini, S. Human handedness: A meta-analysis. Psychol. Bull. 2020, 146, 481–524. [Google Scholar] [CrossRef] [PubMed]
  29. Tanner, J.M.; Gibbons, R.D. A computerized image analysis system for estimating Tanner-Whitehouse 2 bone age. Horm. Res. 1994, 42, 282–287. [Google Scholar] [CrossRef] [PubMed]
  30. Sato, K.; Ashizawa, K.; Anzo, M.; Otsuki, F.; Kaneko, S.; Tanaka, T.; Tsukagoshi, K.; Nimura, A.; Matsuoka, H.; Matsuo, N.; et al. Setting up an automated system for evaluation of bone age. Endocr. J. 1999, 46, S97–S100. [Google Scholar] [CrossRef] [PubMed]
  31. Thodberg, H.H.; Kreiborg, S.; Juul, A.; Pedersen, K.D. The BoneXpert Method for Automated Determination of Skeletal Maturity. IEEE Trans. Med. Imaging 2009, 28, 52–66. [Google Scholar] [CrossRef]
  32. Martin, D.D.; Calder, A.D.; Ranke, M.B.; Binder, G.; Thodberg, H.H. Accuracy and self-validation of automated bone age determination. Sci. Rep. 2022, 12, 6388. [Google Scholar] [CrossRef]
  33. Thodberg, H.H.; Thodberg, B.; Ahlkvist, J.; Offiah, A.C. Autonomous artificial intelligence in pediatric radiology: The use and perception of BoneXpert for bone age assessment. Pediatr. Radiol. 2022, 52, 1338–1346. [Google Scholar] [CrossRef] [PubMed]
  34. Martin, D.D.; Neuhof, J.; Jenni, O.G.; Ranke, M.B.; Thodberg, H.H. Automatic determination of left- and right-hand bone age in the First Zurich Longitudinal Study. Horm. Res. Paediatr. 2010, 74, 50–55. [Google Scholar] [CrossRef] [PubMed]
  35. van Rijn, R.R.; Thodberg, H.H. Bone age assessment: Automated techniques coming of age? Acta Radiol. 2013, 54, 1024–1029. [Google Scholar] [CrossRef] [PubMed]
  36. Ontell, F.K.; Ivanovic, M.; Ablin, D.S.; Barlow, T.W. Bone age in children of diverse ethnicity. AJR Am. J. Roentgenol. 1996, 167, 1395–1398. [Google Scholar] [CrossRef] [PubMed]
  37. Zhang, A.; Sayre, J.W.; Vachon, L.; Liu, B.J.; Huang, H.K. Racial differences in growth patterns of children assessed on the basis of bone age. Radiology 2009, 250, 228–235. [Google Scholar] [CrossRef]
Applsci 14 08135 g001
Figure 1. (A) An overview of the hybrid model of Greulich–Pyle (GP) and modified Tanner–Whitehouse 3 (TW3) methods for bone age assessment in this study. (B) Upgraded automatic hand bone alignment technique.
Figure 1. (A) An overview of the hybrid model of Greulich–Pyle (GP) and modified Tanner–Whitehouse 3 (TW3) methods for bone age assessment in this study. (B) Upgraded automatic hand bone alignment technique.
Applsci 14 08135 g001
Applsci 14 08135 g002
Figure 2. The Bland–Altman plots indicate an agreement between each hand’s bone age assessment. The dotted horizontal lines represent a standard deviation of ±2.
Figure 2. The Bland–Altman plots indicate an agreement between each hand’s bone age assessment. The dotted horizontal lines represent a standard deviation of ±2.
Applsci 14 08135 g002
Applsci 14 08135 g003
Figure 3. The Passing–Bablok regression analysis showed that the overall correlation of the bone age assessment between both hands was excellent, with no significant deviation from linearity in this association (p = 0.15).
Figure 3. The Passing–Bablok regression analysis showed that the overall correlation of the bone age assessment between both hands was excellent, with no significant deviation from linearity in this association (p = 0.15).
Applsci 14 08135 g003
Table 1. The baseline characteristics of the subjects.
Table 1. The baseline characteristics of the subjects.
VariablesTotal (n = 757)Male (n = 321)Female (n = 436)
Age, years a8.75 ± 2.669.49 ± 3.068.21 ± 2.18
Age distribution, years
 <556/757 (7.4%)23/321 (7.2%)33/436 (7.6%)
 ≥5 and <10495/757 (65.8%)161/321 (50.2%)337/436 (77.3%)
 ≥10 and < 15190/757 (25.1%)126/321 (39.3%)64/436 (14.7%)
 ≥1513/757 (1.7%)11/321 (3.4%)2/436 (0.5%)
Automatic bone age assessment by the model 0.775
 Left hand a9.15 ± 3.219.85 ± 3.728.63 ± 2.67
 Right hand a9.20 ± 3.239.90 ± 3.728.69 ± 2.72
ADBH model b0.23 ± 0.19 (0.22, 0.25)0.24 ± 0.19 (0.22, 0.26)0.23 ± 0.19 (0.21, 0.25)
 <0.25510/757 (67.4%)215/321 (67.0%)295/436 (67.7%)
 ≥0.25 and <0.5188/757 (24.8%)80/321 (24.9%)108/436 (24.8%)
 ≥0.5 and <0.7548/757 (6.3%)20/321 (6.2%)28/436 (6.4%)
 ≥0.75 and <111/757 (1.5%)6/321 (1.9%)5/436 (1.1%)
Unless otherwise indicated, the data are presented as the number of patients, with percentages in parentheses. ADBH model, the absolute difference between each hand’s bone age assessment. a The data are presented as means ± standard deviation. b The data are shown as the means ± standard deviation, with the 95% confidence interval in parentheses.
Table 2. A comparison of the characteristics of the study population.
Table 2. A comparison of the characteristics of the study population.
VariablesADBH Model ≤ 0.5 Year
(n = 698)		ADBH Model > 0.5 Year
(n = 59)		p Value
Age, years a8.77 ± 2.668.55 ± 2.730.555
Male-to-female ratio295:40326:330.788
Left-hand BAA a9.18 ± 3.218.78 ± 2.270.356
Right-hand BAA a9.22 ± 3.229.00 ± 3.390.620
ADBH model b0.198 ± 0.145 (0.187, 0.209)0.667 ± 0.109 (0.638, 0.695)<0.001
Developmental stage 0.025
 Pre-puberty201/698 (28.2%)26/59 (44.1%)0.018
 Early and mid-puberty462/698 (66.2%)29/59 (49.2%)0.010
 Late puberty28/698 (4.0%)2/59 (3.4%)1.000
 Post puberty7/698 (1.0%)2/59 (3.4%)0.151
Unless otherwise indicated, the data are presented as the number of patients, with percentages in parentheses. ADBH model, the absolute difference between each hand’s bone age assessment. BAA, bone age assessment. a The data are presented as the means ± standard deviation. b The data are shown as the means ± standard deviation, with the 95% confidence interval in parentheses.
Table 3. A comparison of the bone age assessment by the model and the reference standard.
Table 3. A comparison of the bone age assessment by the model and the reference standard.
Left HandRight Handp Value
Automatic bone age assessment by the model8.78 ± 3.279.00 ± 3.390.011
Reference standard bone age reference by two reviewers8.93 ± 3.298.89 ± 3.330.157
Mean absolute difference a0.409 ± 0.335 (0.322, 0.467)0.424 ± 0.329 (0.339, 0.510)0.809
 Median0.3540.330
 Range0–1.750–1.33
Unless otherwise indicated, the data are shown as the means ± standard deviation. a The data are shown as the means ± standard deviation, with the 95% confidence intervals in parentheses.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Lee, K.-C.; Kang, C.H.; Ahn, K.-S.; Lee, K.-H.; Lee, J.J.; Cho, K.R.; Oh, S. A Comparison of Automatic Bone Age Assessments between the Left and Right Hands: A Tool for Filtering Measurement Errors. Appl. Sci. 2024, 14, 8135. https://doi.org/10.3390/app14188135

AMA Style

Lee K-C, Kang CH, Ahn K-S, Lee K-H, Lee JJ, Cho KR, Oh S. A Comparison of Automatic Bone Age Assessments between the Left and Right Hands: A Tool for Filtering Measurement Errors. Applied Sciences. 2024; 14(18):8135. https://doi.org/10.3390/app14188135

Chicago/Turabian Style

Lee, Kyu-Chong, Chang Ho Kang, Kyung-Sik Ahn, Kee-Hyoung Lee, Jae Joon Lee, Kyu Ran Cho, and Saelin Oh. 2024. "A Comparison of Automatic Bone Age Assessments between the Left and Right Hands: A Tool for Filtering Measurement Errors" Applied Sciences 14, no. 18: 8135. https://doi.org/10.3390/app14188135

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop