Artificial Intelligence (AI)-Based Systems for Automatic Skeletal Maturity Assessment through Bone and Teeth Analysis: A Revolution in the Radiological Workflow?

Round 1

Reviewer 1 Report

The authors put together a comprehensive review of available bone age calculation methods range from traditional to concurrent ML-based method. This is very important for the fields, as it will help researchers/doctors to understand the status quo and inspire researchers for new ideas.

The manuscripts are well written. However, I do have one comment:

Based on the manuscript, I don't see any good summary of the pros and cons across all the various methods. I am wondering if it's worth adding a summary tables that compares the major differences, pros and cons across the mentioned methods in the manuscript. This can help the researchers/doctors to select the most fitted methods for their studies.

Author Response

The manuscripts are well written. However, I do have one comment:

Based on the manuscript, I don't see any good summary of the pros and cons across all the various methods. I am wondering if it's worth adding a summary tables that compares the major differences, pros and cons across the mentioned methods in the manuscript. This can help the researchers/doctors to select the most fitted methods for their studies.

 

The authors thank the reviewer for the time and effort spent revising our manuscript and the precious suggestions.

As recommended, we added two new tables to sum up the limitations and advantages of the different available approaches to help the readers in a better comprehension of these techniques.

Thank you

Best regards

The authors

Reviewer 2 Report

Note: Unfortunately, the term artificial intelligence is also overused in the scientific community. From the point of view of computer science, the described methods are called machine learning (ML) and this is the term I will use in the review.

According to the title of the work, it is to summarize and show the prospects of using ML in the analysis of images of the skeletal system. In my opinion, the work is not suitable for publication. The main objections (see below) relate to the lack of insight in current research regarding the use of ML in the analysis of radiological images.

11.  The chapter "2.2 AI-based method" should contain key information regarding the use of ML in the described applications. Unfortunately, this chapter contains 14 references, of which three references [23,54,55] refer to semi-automatic systems that do not use machine learning, two references [17,55] refer to the comparison of these systems with manual systems. References 56 refer to the use of neural networks in the analysis of radiological images, however, this is a very old work from 1995. The remaining references concern the application of commercial BoneExpert software for various types of analysis. The authors provide in Table 1 software packages other than BoneExpert, however, they do not provide any information on how the reader is to find information about this type of software (manufacturer, literature reference, etc.). There is no reference to the BoneExpert software, even to the existing manufacturer's website. The presented literature references show that the practical application of ML to solve the problems of radiological image analysis is dominated by the commercial BoneExpert software, which is used in scientific and clinical practice. However, BoneExpert is practically the only software that uses machine learning methods, is the title of the publication not adequate to the content? Quite valuable is reference 62, which refers to the original work using the ML method for the analysis of radiological images of bones. The text above shows that the use of ML in this field is very limited and there are no original works applying different methods to different types of applications. It is very strange that the authors do not refer to The Radiological Society of North America (RSNA) Pediatric Bone Age Machine Learning Challenge (Halabi SS, Prevedello LM, Kalpathy-Cramer J et al (2018) The RSNA pediatric bone age machine learning challenge Radiology 290:498–503. Siegel EL (2018) What can we learn from the RSNA pediatric bone age machine learning challenge? Radiology 290:504–505). (Halabi et al's work appears as reference 94). It seems that the authors, preparing to write the publication, did not conduct an in-depth review of the literature on the subject. A quick search of the Pubmed database with the keywords "bone age learning" yields a large number of literature items from recent years concerning the presented issue, and not mentioned or discussed in the reviewed work.

2. The part concerning the study of teeth seems to be much better developed than the part concerning the skeletal system (but it does not mean that it is perfect). However, this chapter does not quite correspond to the title of the work, which clearly points to the analysis of bones, and teeth are part of the digestive system.

3. The cited literature contains a number of references that are very poorly related to the topic of the work, or simply poorly selected. As an example, I will use the authors' self-citations:

Item 22 - review paper on the use of ML in oncology

Item 23 - original work on oncology ??

Item 32 - review of ML in radiology

Item 36 - review paper on the use of ML in oncology

Frequent references in a review paper to other review papers alone do little for the reader and are simply bad practice

Author Response

The authors thank the reviewer for the work and precious suggestions.

We modified the manuscript according to the requests, as follows:

 

Note: Unfortunately, the term artificial intelligence is also overused in the scientific community. From the point of view of computer science, the described methods are called machine learning (ML) and this is the term I will use in the review.

 

Thank you for this pertinent comment. We inserted new studies based on DL to complete our review

 

According to the title of the work, it is to summarize and show the prospects of using ML in the analysis of images of the skeletal system. In my opinion, the work is not suitable for publication. The main objections (see below) relate to the lack of insight in current research regarding the use of ML in the analysis of radiological images.

11. The chapter "2.2 AI-based method" should contain key information regarding the use of ML in the described applications. Unfortunately, this chapter contains 14 references, of which three references [23,54,55] refer to semi-automatic systems that do not use machine learning, two references [17,55] refer to the comparison of these systems with manual systems. References 56 refer to the use of neural networks in the analysis of radiological images, however, this is a very old work from 1995. The remaining references concern the application of commercial BoneExpert software for various types of analysis. The authors provide in Table 1 software packages other than BoneExpert, however, they do not provide any information on how the reader is to find information about this type of software (manufacturer, literature reference, etc.). There is no reference to the BoneExpert software, even to the existing manufacturer's website. The presented literature references show that the practical application of ML to solve the problems of radiological image analysis is dominated by the commercial BoneExpert software, which is used in scientific and clinical practice. However, BoneExpert is practically the only software that uses machine learning methods, is the title of the publication not adequate to the content? Quite valuable is reference 62, which refers to the original work using the ML method for the analysis of radiological images of bones. The text above shows that the use of ML in this field is very limited and there are no original works applying different methods to different types of applications. It is very strange that the authors do not refer to The Radiological Society of North America (RSNA) Pediatric Bone Age Machine Learning Challenge (Halabi SS, Prevedello LM, Kalpathy-Cramer J et al (2018) The RSNA pediatric bone age machine learning challenge Radiology 290:498–503. Siegel EL (2018) What can we learn from the RSNA pediatric bone age machine learning challenge? Radiology 290:504–505). (Halabi et al's work appears as reference 94). It seems that the authors, preparing to write the publication, did not conduct an in-depth review of the literature on the subject. A quick search of the Pubmed database with the keywords "bone age learning" yields a large number of literature items from recent years concerning the presented issue, and not mentioned or discussed in the reviewed work.

Thank you for the comment.

As suggested, we extensively rewrote the manuscript, modified the highlighted references, and further extended the analysis of the existing literature, adding studies for all the possible approaches to perform bone age assessment.

We also deepened the topic of the Radiological Society of North America (RSNA) Pediatric Bone Age Machine Learning Challenge.

11. The part concerning the study of teeth seems to be much better developed than the part concerning the skeletal system (but it does not mean that it is perfect). However, this chapter does not quite correspond to the title of the work, which clearly points to the analysis of bones, and teeth are part of the digestive system.

As suggested, we revised all sections of our manuscript, including bone age calculation based on teeth, to offer a more comprehensive description of the existing literature.

We added many available studies in all sections for a better overview of the available applications

 

11. The cited literature contains a number of references that are very poorly related to the topic of the work, or simply poorly selected. As an example, I will use the authors' self-citations:

Item 22 - review paper on the use of ML in oncology

Item 23 - original work on oncology ??

Item 32 - review of ML in radiology

Item 36 - review paper on the use of ML in oncology

Frequent references in a review paper to other review papers alone do little for the reader and are simply bad practice

 

Thank you for the comment.

As suggested, we extensively modified the references, by replacing the suggested ones, and adding new references in the whole manuscript

 

The authors thank again the reviewer

 

Best regards

 

The authors

Reviewer 3 Report

Why only wrist and dental age assessment is given importance?

The section - Other methods and under that subsections - traditional and AI based methods - divert the topics. what is the significance of the term 'methods'?

Comments for author File: Comments.pdf

Author Response

The authors thank the reviewer for the work and precious suggestions.

 

Why only wrist and dental age assessment is given importance?

 

Thank you very much for the suggestions.

The available literature is focused on hand-wrist and dental analysis due to the presence of consolidated traditional methods and availability of X-Rays.

 

According to the suggestion, we extended the section about other possible approaches, including different studies on knee MRI, clavicle medial extremity, cervical vertebrae and elbow.

As you can see, we added a lot of new studies on anatomical district different form hand and teeth.

The section - Other methods and under that subsections - traditional and AI based methods - divert the topics. what is the significance of the term 'methods'?

 

Thank you for the suggestion.

 

Best regards

 

The authors

 

We replace the term method with approach in order to avoid confusion

Round 2

Reviewer 2 Report

Thanks to the authors for their corrections. The chapter on age assessment based on radiographs of bones looks much better. Table 1 clearly shows the current state of research in the discussed area.

Anatomically, bones and teeth belong to two different systems. So the title should be changed to indicate the use of bone and tooth images for age assessment. The current title does not reflect the content of the article (this was a comment made in the first review). Similarly, in the Abstract and Conclusions, the authors refer only to skeletal research. It is not clear to me why the authors devote a significant part of their work to dental examinations, and on the other hand, in important parts of the manuscript, such as the title, abstract, conclusions, they completely omit the presented material.

The structure of Chapter 3.1 is very strange, mostly single sentences in the form of separate paragraphs with many sentences starting with the word "Anyway". The chapter should be thoroughly revised in terms of writing style. A similar strange writing style is found in chapter 3.2, 4.1 (4.1. Traditional approaches) and chapter numbered 4.1 but with a name (4.1. AI-based approaches). It's probably a numbering error. It seems easiest to organize the material as in Table 1. I would recommend that the material in chapters 3 and 4 be organized in the same way. In its current form, the readability of the text is practically nil.

Figures 4 and 5 add absolutely nothing and I would suggest removing them.

Author Response

The authors thank the reviewers for the work and precious suggestions

We performed all required changes as follows:

Anatomically, bones and teeth belong to two different systems. So the title should be changed to indicate the use of bone and tooth images for age assessment. The current title does not reflect the content of the article (this was a comment made in the first review). Similarly, in the Abstract and Conclusions, the authors refer only to skeletal research. It is not clear to me why the authors devote a significant part of their work to dental examinations, and on the other hand, in important parts of the manuscript, such as the title, abstract, conclusions, they completely omit the presented material.

 

Thank you for the suggestions.

We included a referral to the teeth in all requested section (abstract, introduction, and conclusion), as suggested, and in the title as well, to underline the inclusion of teeth analysis for bone age estimation in our review

The structure of Chapter 3.1 is very strange, mostly single sentences in the form of separate paragraphs with many sentences starting with the word "Anyway". The chapter should be thoroughly revised in terms of writing style. A similar strange writing style is found in chapter 3.2, 4.1 (4.1. Traditional approaches) and chapter numbered 4.1 but with a name (4.1. AI-based approaches). It's probably a numbering error. It seems easiest to organize the material as in Table 1. I would recommend that the material in chapters 3 and 4 be organized in the same way. In its current form, the readability of the text is practically nil.

 

Thank you for the comment.

We have completely rewritten the above section to make the article more readable and fluent and we also added appropriate tables, to sum up the results of the reported study for a better comprehension of the cited articles

 

Figures 4 and 5 add absolutely nothing and I would suggest removing them.

 

Thank you for the suggestion. We have removed them from the text.

 

Thank you again

 

Best regards

 

The authors

 

Round 3

Reviewer 2 Report

I have no more comments. 

Author Response

Thank you for the suggestions and your precious time and work

 

Best regards

 

The authors