Detection of Periapical Lesions on Panoramic Radiographs Using Deep Learning

Round 1

Reviewer 1 Report

Dear Authors, 

It is nice effort. Please read and address the comments on the reviewed manuscript (Introduction and methods sections). 

Comments for author File: Comments.pdf

Author Response

Reviewer 1:

We thank the reivewer for the valuable comments that have helped us improve the quality of the papper. Underneath we address each of the reviewers comments.

Comment 1: Consider revising to '' Thus, this condition should be diagnosed and treated without delay. Failure to treat might lead to the spread of disease to the surrounding tissues, resulting in serious complication for the patient.

Response:  revised as suggested
Change in the text: Thus, this condition should be diagnosed and treated without delay. Failure to treat might lead to the spread of disease to the surrounding tissues, resulting in serious complication for the patient

Comment 2: In the reference 4 it is written in the following words ''While a provisional diagnosis of acute AP may be confidently diagnosed from its clinical presentation, the diagnosis of chronic AP is usually dependent on radiological signs.''
please revise the first sentence as in the reference authors have noted ''may be'' and in your manuscript the word "is" is giving an impression that AAP can always be diagnosed clinically (definitive diagnosis rather than provisional).
Moreover, please replace the word performed with the more suitable word, as diagnostic procedures are performed and

Response: revised as suggested
Change in the text: While an initial diagnosis of acute apical periodontitis may be made clinically, the detection of chronic apical periodontitis is made by radiographs used to reveal characteristic periapical radiolucencies that are usually called apical lesion.
Lines: ….

Comment 3: radiographs are usually as the gold standard
Response: change as suggested
Change in the text: Radiographs are usually considered as the gold standard imaging technique for diagnosis of apical lesions. (Lines ……..)
New reference: 
Geibel MA, Schreiber ES, Bracher AK, Hell E, Ulrici J, Sailer LK, Ozpeynirci Y, Rasche V. Assessment of apical periodontitis by MRI: a feasibility study. Rofo. 2015 Apr;187(4):269-75. doi: 10.1055/s-0034-1385808. Epub 2015 Jan 16. PMID: 25594373).
Neto R., Reibel J., Wenzel A., Kirkevang L.-L. Diagnostic validity of periapical radiography and CBCT for assessing periapical lesions that persist after endodontic surgery. Dentomaxillofac. Radiol. 2017;46:20170210. doi: 10.1259/dmfr.20170210.
Li CW, Lin SY, Chou HS, Chen TY, Chen YA, Liu SY, Liu YL, Chen CA, Huang YC, Chen SL, Mao YC, Abu PAR, Chiang WY, Lo WS. Detection of Dental Apical Lesions Using CNNs on Periapical Radiograph. Sensors (Basel). 2021 Oct 24;21(21):7049. doi: 10.3390/s21217049. PMID: 34770356; PMCID: PMC8588190
Karamifar K, Tondari A, Saghiri MA. Endodontic Periapical Lesion: An Overview on the Etiology, Diagnosis and Current Treatment Modalities. Eur Endod J. 2020 Jul 14;5(2):54-67. doi: 10.14744/eej.2020.42714. PMID: 32766513; PMCID: PMC7398993).

Comment 4: Oral
Response: corrected
Change in the text: line…

Comment 5: Please provide the specifications and brand of the machines by which the radiographs were taken.
Response: added
Change in the text: Cranex D (Soredex, Tuusula, Finland) panoramic imaging system with the following parameters: 85 kVp, 10mA, exposure time 17.6 seconds, CCD sensor size 48 microme-ter, and focal spot size 0.5 mm. (Lines……)

Comment 6: 1) Please mention the experience/ the initials of the names of the first two examiners and third examiner as you have highlighted the issue of ''low sensitivity'' in the introduction section.
2) Was evaluators' calibration done before starting the process of radiographic evaluation and annotation? If yes how it was achieved.

Response: added to the text.
Change in the text: Three examiners (A. D., E.A., and R. BH) independently annotated the Periapical Root Areas (PRAs) as having Periapical Lesion (PL) or not having periapical lesion (Healthy (H)) in duplicate; a fourth examiner (S.O.) settled discrepancies between examiners. At the time of the conduction of the study all above-mentioned examiners had more than 15 years of clinical experience. The examiners were calibrated on 10 OPGs to address discrepancies between them, and then, recalibrated using 20 OPGs and a Kappa index for inter examiner agreement in detecting PLs of 90% was achieved. Upon labeling only 5% of the cases need the consultation of the fourth examiner to settle discrepancies.

 

Reviewer 2 Report

Detection of Periapical Lesions on Panoramic Radiographs Using Deep Learning

It is a very interesting work that addresses a topic of interest in the endodontic area.

The manuscript can be enriched if it includes more precisely in the introduction the problem to be solved or the existing controversy with currently available diagnostic methods and not only focus on cost.

Include study design.

It is also important to explicitly reflect the hypothesis of the study. In the case of a diagnostic test, it is important to show data of inter- and intra-observer consistency in the location of the lesions.

A better description of the results and a more extensive description of the figures are required.

The discussion should better highlight the clinical value of the proposition and how a specialist or even a general practice dentist can use the tool in their dental office.

The conclusion must be consistent with the objective of the study.

Author Response

Reviewer 2

 We thank the reivewer for the valuable comments that have helped us improve the quality of the papper. Underneath we address each of the reviewers comments.

Comment 1: The manuscript can be enriched if it includes more precisely in the introduction the problem to be solved or the existing controversy with currently available diagnostic methods and not only focus on cost.

 

Response more elaboration of the diagnostic methods used for diagnosis of periapical lesions and their limitation has been added to the introduction. We have also added new references for support.

Change in the text:  A previous study showed that interpreters were only able to reach a 50% level of agreement on the assessment peri-apical lesions on peri-apical radiographs. Also, re-evaluation of radiographs by the same clinician showed different interpretation of their own original diagnosis (Neto et al). Thus, image interpretation by dentists could sometimes be inconsistent (Li et al).

……..

Other techniques such as CBCT , MRI and Echography can also be useful (Geibel et al ) (Karamifar et al). However, these methods

cannot be used for routine screening because CBCT requires too much radiation, MRI is very expensive and time consum-ing, and

Echography is ineffective in lesions not affecting the cortical bone. In this context, OPG are better suited for screening. However,

even though dentists are sup-posed to make accurate screenings of periapical lesions on OPGs human errors occur and dentists

can often miss obvious periapical lesions. A tool to automate detection can help minimize these errors.

New references:

1. - Neto R., Reibel J., Wenzel A., Kirkevang L.-L. Diagnostic validity of periapical radiography and CBCT for assessing periapical lesions that persist after endodontic surgery. Dentomaxillofac. Radiol. 2017;46:20170210. doi: 10.1259/dmfr.20170210.
2. - Li CW, Lin SY, Chou HS, Chen TY, Chen YA, Liu SY, Liu YL, Chen CA, Huang YC, Chen SL, Mao YC, Abu PAR, Chiang WY, Lo WS. Detection of Dental Apical Lesions Using CNNs on Periapical Radiograph. Sensors (Basel). 2021 Oct 24;21(21):7049. doi: 10.3390/s21217049. PMID: 34770356; PMCID: PMC8588190
3. - (Geibel MA, Schreiber ES, Bracher AK, Hell E, Ulrici J, Sailer LK, Ozpeynirci Y, Rasche V. Assessment of apical periodontitis by MRI: a feasibility study. Rofo. 2015 Apr;187(4):269-75. doi: 10.1055/s-0034-1385808. Epub 2015 Jan 16. PMID: 25594373.)
4. Karamifar K, Tondari A, Saghiri MA. Endodontic Periapical Lesion: An Overview on the Etiology, Diagnosis and Current Treatment Modalities. Eur Endod J. 2020 Jul 14;5(2):54-67. doi: 10.14744/eej.2020.42714. PMID: 32766513; PMCID: PMC7398993).

 

 

Comment 2: Include study design.

 

Response: added as suggested-

Change in the text: This retrospective diagnostic cohort study was conducted in accordance….

Comment 3: It is also important to explicitly reflect the hypothesis of the study.

Response: We now describe the hypothesis more clearly

Change in the text: We hypothesized that an AI tool trained on healthy and non-health periapical areas in panoramic radiographs could detect the periapical regions of the teeth and classify them into healthy apices and apices with periapical lesions.

 

Comment 3: In the case of a diagnostic test, it is important to show data of

Response: We now report more clearly the inter and intra observer consistency of the labelling clinicians

The inter and intra observer consistency of the AI, is not relevant because for the AI it is was 100% because the computer algorithm always assessed the radiographs the same way

 

Change in the text: The examiners were calibrated on 10 OPGs to address discrepancies between them, and then, recalibrated using 20 OPGs and a Kappa index for inter examiner agreement in detecting PLs of 90% was achieved. Upon labelling only 5% of the cases needed the consultation of the fourth examiner to settle discrepancies.

 

Comment 3: A better description of the results and a more extensive description of the figures are required

Response: We now provide a better and more extensive description of the figures and the results

Change in the text:

We rewrote the entire result section to make it clearer. We also rewrote the captions as follow

 

Figure 1. Proposed System Architecture and Workflow. Input: Panoramic radiograph. The system first performs ROI extraction on the input images using the “Faster RCNN” algorithm. Then, the system classifies the extracted PRAs into two possible categories: peri apical lesion (PL) or Healthy (H); this was done using the “Inception v3” algorithm. The final output: the detected periapical lesions depicted by red bounding boxes with the system confidence score. Lines (124-128)

Figure 2 Detection of periapical areas(PRA) on Panoramic radiograph by clinicians and by the AI. First image: OPG showing the PRA annotated by the expert clinician. Second image: OPG showing the proposed PRA generated by the Detection Model for the same image. All PRAs were detected by the proposed model (Faster RCNN). Lines(211-213)

Figure 3. Graph depicting the Learning Curves and Error Rates of the Inception v3 model we used to classify the periapical areas. The graph shows the Training accuracy (Train Acc.), the Training loss (Train Loss), the Validation accuracy (Val. Acc.) and the Validation loss (Val. Loss) as a function of epochs. Training and validation loss decreased over time and stabilized at epoch 100. Training and validation accuracy on the other hand increased over time and stabilized at epoch 100. This indicated the convergence of the model after 100 epochs. Lines(230-234)

Figure 4. . Performance of the proposed AI system for Periapical Lesion detection. This figure shows OPGs of three different cases (a, b and c) labeled by the clinician (left) and the AI (right). For each row, the OPG on the left shows the boxes annotated by the expert including PL and Healthy PRAs, while on the right, we show the same OPG labeled by the AI with red boxes indicating the PL detected by the AI. The confidence score calculated by the AI system can be found on top of each red boxes. Case “a” shows a periapical lesion detected in a multirooted tooth (#36). Case “b” shows two periapical lesions in teeth with root canal treatments and crowns in a patient with dental implants. Case “c” shows a case with a single PL in a multirooted tooth (#26). Lines(247-250)

 

Table 2. Evaluation of the performance of the different Classification Models tested

Table 3. Confusion Matrix and Classification Metrics assessing the performance of Inception v3

Comment 4: The discussion should better highlight the clinical value of the proposition and how a specialist or even a general practice dentist can use the tool in their dental office.

Response: The discussion has now been expanded to better highlight the value of the proposition and how dentists could use the tool (Lines ….)

Change in the text:

Diagnosing and documenting pathologies on dental radiographs is time consuming and eventhough general and specialist dentists are well trained to do this, they are not exempt of human error. In fact  most complications in dental practice stem from missdiagnosis which often involves missing out on noticing periapical lessions. In this sense , the proposed technolgy could help clinicians fill dental charts, and minimize diagnosis errors in the detection of periapcial lessions. In fact , a previous study has already demonstrated that AI could outperfrom dental specialists in detection of apical lesions (32).

 

 

 

Comment 5: The conclusion must be consistent with the objective of the study.

Response: adjusted as suggested

Change in the text: In the present study, the proposed AI tool based on “Faster-RCNN“ and Inception-v3 was able to detect the periapical region of the teeth on panoramic radiographs and classify them into healthy and periapical lessions achieving an accuracy of 84.6%.

 

 

Reviewer 3 Report

Interested to read this well-written paper about AI detection of peri apical lesions from OPG foto's. Nice software and good try , however , i do have some major concerns:

1. As i said, it is a good try, but in general, if a dentist would like to make a diagnose of apical lesion , he/she would take an extra apical foto instead of OPG or using extra software.  

2. It is not clear for me how this software define the ROI size , according to the different size of roots (big, small, one root , two roots and more..)? If the ROI can not change accordingly, then results are not trustable. 

3. I do not see if authors considered the implant case as well in the selection criteria. How does the software distinguish the nature teeth and implant?

4. In discussion , 'this study showed the effectiveness of the proposed model to detect periapical lesions on panoramic radiographs. This model could be useful in clinical application for quick and easy detection of periapical lesions.'. I do not see any control group , so how would authors  prove its quick and easy detection? 

 

Author Response

-Reviewer 3:

We thank the reivewer for the valuable comments that have helped us improve the quality of the papper. Underneath we address each of the reviewers comments.

Comment 1. As i said, it is a good try, but in general, if a dentist would like to make a diagnose of apical lesion , he/she would take an extra apical foto instead of OPG or using extra software. 

Response

The main use of the porposed technolgy is screening rather than diangosis. Eventhough perioapical radiographs are the gold standard for diagnosis of periapical lessions, due to radiation concerns, they cannot be routinly used for screening the entire dentition. OPGs are better suited for screening. However, even though dentists are supposed to make accurate screenings of periapical lesions on OPGs there is plenty of literature indicating that human errors occur and often dentists can miss obvious periapical lesions. A tool to automate detection  can help minimize these errors. We now elaborate on this issue in the introduction and in the dicusssion of the article. We have also added new references for support.

Change in the text: Introduction

Periapical radiographs are usually considered as the gold standard imaging techniques for diagnosis of apical lesions (Geibel et al). However, there could be inconsistency across dentists in their interpretation of such radiographs, and due to radiation concerns, they cannot be routinely used for screening the entire dentition.

Other techniques such as CBCT , MRI and Echography can also be useful (Geibel et al ) (Karamifar et al). However, these methods cannot be used for routine screening because CBCT requires too much radiation, MRI is very expensive and time consuming, and Echography is ineffective in lesions not affecting the cortical bone. In this context, OPG are better suited for screening. However, even though dentists are sup-posed to make accurate screenings of periapical lesions on OPGs human errors occur and dentists can often miss obvious periapical lesions. A tool to automate detection can help minimize these errors.

New references:

Neto R., Reibel J., Wenzel A., Kirkevang L.-L. Diagnostic validity of periapical radiography and CBCT for assessing periapical lesions that persist after endodontic surgery. Dentomaxillofac. Radiol. 2017;46:20170210. doi: 10.1259/dmfr.20170210.

Li CW, Lin SY, Chou HS, Chen TY, Chen YA, Liu SY, Liu YL, Chen CA, Huang YC, Chen SL, Mao YC, Abu PAR, Chiang WY, Lo WS. Detection of Dental Apical Lesions Using CNNs on Periapical Radiograph. Sensors (Basel). 2021 Oct 24;21(21):7049. doi: 10.3390/s21217049. PMID: 34770356; PMCID: PMC8588190

Karamifar K, Tondari A, Saghiri MA. Endodontic Periapical Lesion: An Overview on the Etiology, Diagnosis and Current Treatment Modalities. Eur Endod J. 2020 Jul 14;5(2):54-67. doi: 10.14744/eej.2020.42714. PMID: 32766513; PMCID: PMC7398993).

 

 

Change in the text: Discussion

Diagnosing and documenting pathologies on dental radiographs is time consuming and eventhough general and specialist dentists are well trained to do this, they are not exempt of human error. In fact  most complications in dental practice stem from missdiagnosis which often involves missing out on noticing periapical lessions. In this sense , the proposed technolgy could help clinicians fill dental charts, and minimize diagnosis errors in the detection of periapcial lessions. In fact , a previous study has already demonstrated that AI could outperfrom dental specialists in detection of apical lesions (32).

 

Comment 2. It is not clear for me how this software define the ROI size , according to the different size of roots (big, small, one root , two roots and more..)? If the ROI can not change accordingly, then results are not trustable.

Response: As explained in the methodology section the tool that detects the ROI is an AI that is able to learn the to identify the periapical area of the roots of the teeth. The tool adjusts the size of the ROI automatically according to the size and extnesion of the periapical area and the lession ( see figures).

 Change in text:

Resuts section:

The resulting AI tool desing to identify the peripical area was able to adapt to the size and extension of the periapical region atuomaticlaly depending on the size and extension of the rooths and the lesions. This was demostratd by the high AP50 of the algorythm. Each periapical area was labled independently regardless of wether it was found on single rooted or multi rooted teeth.

 

Comment 3. I do not see if authors considered the implant case as well in the selection criteria. How does the software distinguish the nature teeth and implant?

Response: yes, it distingueshed teeth from implants. The deep learning neural network was designed to detect periapical areas and ignore anything else in the radiograph. For this reason it was not affected at all by the presence of implants or any other artifact.we now clarify this in the reuslts section. An example is shown in figure 4.

Change in text:

Results:

Figure 4 shows representative examples of the final output generated by the proposed method. This final tool was designed to detect periapical areas and ignore anything else in the radiograph, thus it was not affected by the presence of distractors and arti-facts such as implants and crowns, also, the size of the detection box was able to adapt to the actual PRA, and it discriminated between different apices on the same tooth.

 

Comment 4. In discussion , 'this study showed the effectiveness of the proposed model to detect periapical lesions on panoramic radiographs. This model could be useful in clinical application for quick and easy detection of periapical lesions.'. I do not see any control group , so how would authors  prove its quick and easy detection?

Response: We now report in the discussion the speed of the model. The algorythm was able to detect all periapical lessions in a radiograph in a fraction of a second.

Change in text:  The experimental results of this study showed the effectiveness of the proposed model to detect periapical lesions on panoramic radiographs. This model could be useful in clinical application for quick and easy detection of periapical lesions. On average, the proposed AI system takes 2.3 seconds to detect and classify all PRAs as H or PL on a panoramic radiograph.

 

 

Round 2

Reviewer 3 Report

Thanks a lot for sending the revision which indeed answers my questions.  

Also big congratulations for the work!