Machine Learning Approach for Prediction of the Test Results of Gonadotropin-Releasing Hormone Stimulation: Model Building and Implementation
Abstract
:1. Introduction
2. Methods
2.1. Study Design, Setting, and Samples
2.2. Features and Outcome Variables
2.3. Statistical Analysis and Model Building
2.4. Model Evaluation and Practical Implementation
3. Results
3.1. Enrollment and Baseline Statistical Tests
3.2. Characteristics and Features
3.3. Model Building and Evaluation
3.4. Prediction System Development and User Evaluation
3.5. Case Scenarios
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Kota, A.S.; Ejaz, S. Precocious Puberty. In StatPearls [Internet]; StatPearls Publishing: Treasure Island, FL, USA, 2022. Available online: https://www.ncbi.nlm.nih.gov/books/NBK544313/ (accessed on 19 March 2022).
- Copeland, W.; Shanahan, L.; Miller, S.; Costello, E.J.; Angold, A.; Maughan, B. Outcomes of early pubertal timing in young women: A prospective population-based study. Am. J. Psychiatry 2010, 167, 1218–1225. [Google Scholar] [CrossRef] [PubMed]
- Tung, Y.C. Central precocious puberty in girls: Increasing with time. Pediatr. Neonatol. 2021, 62, 127–128. [Google Scholar] [CrossRef]
- Ma, H.M.; Du, M.L.; Luo, X.P.; Chen, S.K.; Liu, L.; Chen, R.M.; Zhu, C.; Xiong, F.; Li, T.; Wang, W.; et al. Onset of breast and pubic hair development and menses in urban chinese girls. Pediatrics 2009, 124, e269–e277. [Google Scholar] [CrossRef] [PubMed]
- Howard, S.R.; de Roux, N.; Leger, J. Puberty and its disorders. In Brook’s Clinical Pediatric Endocrinology, 7th ed.; Dattani, M.T., Brook, C.G.D., Eds.; John Wiley & Sons: Oxford, UK, 2020; pp. 256–264. [Google Scholar]
- Neely, E.K.; Hintz, R.L.; Wilson, D.M.; Lee, P.A.; Gautier, T.; Argente, J.; Stene, M. Normal ranges for immunochemiluminometric gonadotropin assays. J. Pediatr. 1995, 127, 40–46. [Google Scholar] [CrossRef] [PubMed]
- Latronico, A.C.; Brito, V.N.; Carel, J.C. Causes, diagnosis, and treatment of central precocious puberty. Lancet Diabetes Endocrinol. 2016, 4, 265–274. [Google Scholar] [CrossRef] [PubMed]
- Chen, W.; Chang, M.H. New growth charts for Taiwanese children and adolescents based on World Health Organization standards and health-related physical fitness. Pediatr. Neonatol. 2010, 51, 69–79. [Google Scholar] [CrossRef]
- Greulich, W.W.; Pyle, S.I. Radiographic Atlas of Skeletal Development of the Hand and Wrist; Stanford University Press: Stanford, CA, USA, 1959. [Google Scholar]
- Chawla, N.V.; Bowyer, K.W.; Hall, L.W.; Kegelmeyer, W.P. SMOTE: Synthetic Minority Over-sampling Technique. J. Artif. Intell. Res. 2002, 16, 321–357. [Google Scholar] [CrossRef]
- Yeh, S.N.; Ting, W.H.; Huang, C.Y.; Huang, S.K.; Lee, Y.C.; Chua, W.K.; Lin, C.H.; Cheng, B.W.; Lee, Y.J. Diagnostic evaluation of central precocious puberty in girls. Pediatr. Neonatol. 2021, 62, 187–194. [Google Scholar] [CrossRef]
- Houk, C.P.; Kunselman, A.R.; Lee, P.A. Adequacy of a single unstimulated luteinizing hormone level to diagnose central precocious puberty in girls. Pediatrics 2009, 123, e1059–e1063. [Google Scholar] [CrossRef]
- Lee, D.S.; Ryoo, N.Y.; Lee, S.H.; Kim, S.; Kim, J.H. Basal luteinizing hormone and follicular stimulating hormone: Is it sufficient for the diagnosis of precocious puberty in girls? Ann. Pediatr. Endocrinol. Metab. 2013, 18, 196–201. [Google Scholar] [CrossRef]
- Lee, H.S.; Park, H.K.; Ko, J.H.; Kim, Y.J.; Hwang, J.S. Utility of Basal luteinizing hormone levels for detecting central precocious puberty in girls. Horm. Metab. Res. 2012, 44, 851–854. [Google Scholar] [CrossRef]
- Neely, E.K.; Wilson, D.M.; Lee, P.A.; Stene, M.; Hintz, R.L. Spontaneous serum gonadotropin concentrations in the evaluation of precocious puberty. J. Pediatr. 1995, 127, 47–52. [Google Scholar] [CrossRef] [PubMed]
- Pasternak, Y.; Friger, M.; Loewenthal, N.; Haim, A.; Hershkovitz, E. The utility of basal serum LH in prediction of central precocious puberty in girls. Eur. J. Endocrinol. 2012, 166, 295–299. [Google Scholar] [CrossRef] [PubMed]
- Cavallo, A.; Richards, G.E.; Busey, S.; Michaels, S.E. A simplified gonadotrophin-releasing hormone test for precocious puberty. Clin. Endocrinol. 1995, 42, 641–646. [Google Scholar] [CrossRef]
- Kandemir, N.; Demirbilek, H.; Özön, Z.A.; Gönç, N.; Alikaşifoğlu, A. GnRH stimulation test in precocious puberty: Single sample is adequate for diagnosis and dose adjustment. J. Clin. Res. Pediatr. Endocrinol. 2011, 3, 12–17. [Google Scholar] [CrossRef]
- Eckert, K.L.; Wilson, D.M.; Bachrach, L.K.; Anhalt, H.; Habiby, R.L.; Olney, R.C.; Hintz, R.L.; Neely, E.K. A single-sample, subcutaneous gonadotropin-releasing hormone test for central precocious puberty. Pediatrics 1996, 97, 517–519. [Google Scholar] [CrossRef]
- You, J.; Cheng, X.; Li, X.; Li, M.; Yao, L.; Luo, F.; Cheng, R.; Xi, L.; Ye, J. Clinical risk score for central precocious puberty among girls with precocious pubertal development: A cross sectional study. BMC Endocr. Disord. 2021, 21, 75. [Google Scholar] [CrossRef]
- Pan, L.; Liu, G.; Mao, X.; Li, H.; Zhang, J.; Liang, H.; Li, X. Development of Prediction Models Using Machine Learning Algorithms for Girls with Suspected Central Precocious Puberty: Retrospective Study. JMIR Med. Inf. 2019, 7, e11728. [Google Scholar] [CrossRef] [PubMed]
- Ab Rahim, S.N.; Omar, J.; Tuan Ismail, T.S. Gonadotropin-releasing hormone stimulation test and diagnostic cutoff in precocious puberty: A mini review. Ann. Pediatr. Endocrinol. Metab. 2020, 25, 152–155. [Google Scholar] [CrossRef]
- Marshall, W.A.; Tanner, J.M. Variations in pattern of pubertal changes in girls. Arch. Dis. Child. 1969, 44, 291–303. [Google Scholar] [CrossRef]
- Parent, A.S.; Teilmann, G.; Juul, A.; Skakkebaek, N.E.; Toppari, J.; Bourguignon, J.P. The timing of normal puberty and the age limits of sexual precocity: Variations around the world, secular trends, and changes after migration. Endocr. Rev. 2003, 24, 668–693. [Google Scholar] [CrossRef] [PubMed]
- Williams, R.M.; Ward, C.E.; Hughes, I.A. Premature adrenarche. Arch. Dis. Child. 2012, 97, 250–254. [Google Scholar] [CrossRef] [PubMed]
- Nam, H.K.; Rhie, Y.J.; Son, C.S.; Park, S.H.; Lee, K.H. Factors to predict positive results of gonadotropin releasing hormone stimulation test in girls with suspected precocious puberty. J. Korean Med. Sci. 2012, 27, 194–199. [Google Scholar] [CrossRef] [PubMed]
- Cavallo, F.; Mohn, A.; Chiarelli, F.; Giannini, C. Evaluation of Bone Age in Children: A Mini-Review. Front. Pediatr. 2021, 9, 580314. [Google Scholar] [CrossRef]
Variable | Negative Test Result (Peak LH < 5 IU/L) (n = 24) | Positive Test Result (Peak LH ≥ 5 IU/L) (n = 137) | p-Value |
---|---|---|---|
Age at test (yr) | 8.55 (0.63) | 8.51 (1.17) | 0.389 |
Age of breast development (yr) | 6.55 (1.14) | 6.50 (1.00) | 0.777 |
Ht (cm) | 133.52 (4.53) | 135.04 (9.16) | 0.120 |
Ht SD | 0.79 (0.69) | 1.07 (1.15) | 0.137 |
Wt (kg) | 34.74 (8.18) | 32.57 (6.98) | 0.443 |
Wt SD | 1.08 (1.05) | 0.82 (0.95) | 0.340 |
BMI | 19.35 (3.67) | 17.69 (2.46) | 0.057 |
BMI classification | 0.046 * | ||
underweight | 0 | 1 (0.73) | |
NR | 15 (62.50) | 101 (73.72) | |
overweight | 2 (8.33) | 23 (16.79) | |
obese | 7 (29.17) | 12 (8.76) | |
Father Ht (cm) | 172.79 (6.76) | 171.42 (5.94) | 0.304 |
Mother Ht (cm) | 157.06 (5.69) | 157.47 (4.71) | 0.673 |
Mid-parental height (MPH, cm) | 158.43 (5.00) | 157.94 (4.15) | 0.818 |
Bone age (yr) | 10.36 (1.22) | 10.65 (1.46) | 0.205 |
BA advancement (yr) | 1.81 (1.10) | 2.14 (0.98) | 0.297 |
ΔBA/ΔCA | 1.67 (1.46) | 1.98 (1.55) | 0.337 |
Predicted adult height (PAH, cm) | 154.40 (4.78) | 153.89 (4.98) | 0.885 |
PAH-MPH (cm) | −4.02 (5.92) | −4.05 (4.96) | 0.911 |
Annual growth rate (cm/yr) | 7.10 (1.55) | 8.35 (2.19) | 0.007 * |
Tanner stage for breast development | 0.001 * | ||
2 | 17 (70.83) | 38 (27.74) | |
3 | 6 (25.00) | 70 (51.09) | |
4 | 1 (4.17) | 27 (19.71) | |
5 | 0 | 2 (1.46) | |
Tanner stage for pubic hair development | 0.344 | ||
1 | 23 (95.83) | 109 (79.56) | |
2 | 1 (4.17) | 19 (13.87) | |
3 | 0 | 8 (5.84) | |
4 | 0 | 1 (0.73) | |
Axillary hair | 0.742 | ||
None | 22 (91.67) | 118 (86.13) | |
Present | 2 (8.33) | 19 (13.87) | |
Baseline FSH (mIU/mL) | 2.19 (1.00) | 3.66 (1.84) | <0.001 * |
Baseline LH (mIU/mL) | 0.21 (0.20) | 1.42 (1.72) | <0.001 * |
Baseline E2 (pg/mL) | 21.57 (16.46) | 26.77 (20.00) | 0.146 |
LH/FSH ratio | 0.09 (0.05) | 0.34 (0.30) | <0.001 * |
GA (wk) | 38.23 (1.89) | 38.64 (1.31) | 0.544 |
Premature birth | 0.213 | ||
No | 21 (87.50) | 129 (94.16) | |
Yes | 3 (12.50) | 8 (5.84) | |
BBW (g) | 2792.83 (493.94) | 2964.34 (368.74) | 0.093 |
Size for gestational age | 0.113 | ||
SGA | 15 (62.50) | 110 (80.29) | |
AGA | 8 (33.33) | 23 (16.79) | |
LGA | 1 (4.17) | 4 (2.92) | |
LH peak (mIU/mL) | 3.75 (1.02) | 22.39 (20.09) | <0.001 * |
Variable | Negative Test Results (Peak LH < 10 IU/L) (n = 65) | Positive Test Results (Peak LH ≥ 10 IU/L) (n = 96) | p-Value |
---|---|---|---|
Age at test (yr) | 8.48 (0.85) | 8.54 (1.25) | 0.144 |
Age of breast development (yr) | 6.44 (0.95) | 6.55 (1.07) | 0.216 |
Ht (cm) | 133.88 (6.07) | 135.44 (9.99) | 0.040 * |
Ht SD | 0.93 (0.87) | 1.10 (1.23) | 0.292 |
Wt (cm) | 33.17 (6.87) | 32.71 (7.43) | 0.996 |
Wt SD | 0.93 (0.99) | 0.80 (0.95) | 0.420 |
BMI | 18.40 (3.04) | 17.62 (2.45) | 0.150 |
BMI classification | 0.542 | ||
underweight | 0 | 1 (1.04) | |
NR | 44 (67.69) | 72 (75.00) | |
overweight | 11 (16.92) | 14 (14.58) | |
obese | 10 (15.38) | 9 (9.38) | |
Father Ht (cm) | 172.12 (6.21) | 171.29 (5.97) | 0.182 |
Mother Ht (cm) | 157.79 (5.31) | 157.15 (4.53) | 0.562 |
Mid-parental height (MPH, cm) | 158.46 (4.38) | 157.72 (4.19) | 0.225 |
Bone age (yr) | 10.47 (1.09) | 10.71 (1.62) | 0.047 * |
BA advancement (yr) | 1.99 (0.90) | 2.16 (1.06) | 0.473 |
ΔBA/ΔCA | 1.79 (1.41) | 2.04 (1.61) | 0.347 |
Predicted adult height (PAH, cm) | 154.10 (5.02) | 153.88 (4.90) | 0.918 |
PAH-MPH (cm) | −4.36 (5.17) | −3.83 (5.06) | 0.391 |
Annual growth rate (cm/yr) | 7.22 (1.75) | 8.80 (2.17) | <0.001 * |
Tanner stage for breast development | 0.001 * | ||
2 | 32 (49.23) | 23 (23.96) | |
3 | 28 (43.08) | 48 (50.00) | |
4 | 5 (7.69) | 23 (23.96) | |
5 | 0 | 2 (2.08) | |
Tanner stage for pubic hair development | 0.078 | ||
1 | 59 (90.77) | 73 (76.04) | |
2 | 5 (7.69) | 15 (15.62) | |
3 | 1 (1.54) | 7 (7.29) | |
4 | 0 | 1 (1.04) | |
Axillary hair | 0.803 | ||
None | 56 (86.15) | 84 (87.50) | |
Present | 9 (13.85) | 12 (12.50) | |
Baseline FSH (mIU/mL) | 2.80 (1.44) | 3.87 (1.91) | <0.001 * |
Baseline LH (mIU/mL) | 0.37 (0.38) | 1.83 (1.89) | <0.001 * |
Baseline E2 (pg/mL) | 22.83 (15.59) | 28.13 (21.65) | 0.114 |
LH/FSH ratio | 0.12 (0.07) | 0.43 (0.32) | <0.001 * |
GA (wk) | 38.64 (1.50) | 38.53 (1.35) | 0.466 |
Premature birth | 1.000 | ||
no | 61 (93.85) | 89 (92.71) | |
yes | 4 (6.15) | 7 (7.29) | |
BBW(g) | 2935.89 (393.59) | 2940.73 (394.54) | 0.989 |
Size for gestational age | 0.889 | ||
SGA | 52 (80.00) | 73 (76.04) | |
AGA | 11 (16.92) | 20 (20.83) | |
LGA | 2 (3.08) | 3 (3.12) | |
LH peak (mIU/mL) | 6.07 (2.23) | 28.78 (20.95) | <0.001 * |
Predictive Models with Each ML Algorithm | Accuracy | Sensitivity | Specificity | AUC | 95% CI (AUC) |
---|---|---|---|---|---|
Peak LH cutoff of 5 IU/L | |||||
Logistic regression | 0.939 | 0.952 | 0.857 | 0.884 | 0.686–0.999 |
Random forest | 0.857 | 0.881 | 0.714 | 0.912 | 0.825–0.998 |
LightGBM | 0.898 | 0.905 | 0.857 | 0.942 | 0.877–0.999 |
XGBoost | 0.878 | 0.905 | 0.714 | 0.942 | 0.876–0.999 |
Peak LH cutoff of 10 IU/L | |||||
Logistic regression | 0.837 | 0.862 | 0.800 | 0.909 | 0.830–0.988 |
Random forest | 0.878 | 0.897 | 0.850 | 0.941 | 0.876–0.999 |
LightGBM | 0.857 | 0.862 | 0.850 | 0.934 | 0.849–0.999 |
XGBoost | 0.837 | 0.931 | 0.700 | 0.881 | 0.786–0.976 |
Algorithms | Accuracy | Sensitivity | Specificity | AUC | AUC 95% CI | p-Value |
---|---|---|---|---|---|---|
Scoring system | 0.735 | 0.966 | 0.400 | 0.683 | 0.568–0.798 | - |
Logistic regression | 0.796 | 0.966 | 0.550 | 0.909 | 0.830–0.988 | 0.001 |
Random forest | 0.878 | 0.966 | 0.750 | 0.941 | 0.876–0.999 | <0.001 |
LightGBM | 0.918 | 0.966 | 0.850 | 0.934 | 0.849–0.999 | <0.001 |
XGBoost | 0.837 | 0.966 | 0.650 | 0.881 | 0.786–0.976 | 0.003 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Chen, Y.-S.; Liu, C.-F.; Sung, M.-I.; Lin, S.-J.; Tsai, W.-H. Machine Learning Approach for Prediction of the Test Results of Gonadotropin-Releasing Hormone Stimulation: Model Building and Implementation. Diagnostics 2023, 13, 1550. https://doi.org/10.3390/diagnostics13091550
Chen Y-S, Liu C-F, Sung M-I, Lin S-J, Tsai W-H. Machine Learning Approach for Prediction of the Test Results of Gonadotropin-Releasing Hormone Stimulation: Model Building and Implementation. Diagnostics. 2023; 13(9):1550. https://doi.org/10.3390/diagnostics13091550
Chicago/Turabian StyleChen, Yu-Shao, Chung-Feng Liu, Mei-I Sung, Shio-Jean Lin, and Wen-Hui Tsai. 2023. "Machine Learning Approach for Prediction of the Test Results of Gonadotropin-Releasing Hormone Stimulation: Model Building and Implementation" Diagnostics 13, no. 9: 1550. https://doi.org/10.3390/diagnostics13091550