Knowledge of Diabetic Retinopathy among Primary Care Nurses Performing Fundus Photography and Agreement with Ophthalmologists on Screening
, Retisalud-Tenerife Primary Healthcare Research Group †, Alfonso Miguel García-Hernández 1,* and Pedro Ruymán Brito-Brito 1,2
Round 1
Reviewer 1 Report
Thank you for inviting me to review this manuscript. I have the following comments for the author(s) in this manuscript:
Materials and Methods.
1. Suggest having a power analysis on each phase.
2. Need to describe clearly how the author(s) randomly select fundus photographs from the EHRS.
3. Why the information is selected in 2020 or 2021?
4. In the EHRS, is any missing data on each selected record? If yes, will you replace them by choosing another record? Need to write it down in the data collection section.
Results
1. In 263-264, the author(s) need to provide statistics and p-value on each comparison.
2. In 267, the author(s) used correlation to determine matches. The rho value is not high (p=0.021). I can't see any discussion on this part in your discussion section.
3. Fo the agreement between both professionals on fundus imaging screening in patients with diabetes (n=100), Kappa is 0.477 (p<0.001). Does it reflect good or low? Same for other results in Table 3, and 4, the Kappa is not high BUT sig., so does it refer to something need to discuss? According to McHugh (2012), Kappa, if range 0.2-0.6, refers to fair to moderate agreement only. Perhaps, need to discuss more in the discussion section.
McHugh ML (2012) Interrater reliability: The Kappa statistics. Biochem Med (Zagreb) 22(3), 276-282.
4. In the aim, the author(s) wrote to assess the degree of agreement on screening for DR between nurses who received that training and ophthalmologists. Did all receive training? If no, which Table shows the difference in the agreement?
Discussion
1. Need to discuss results between with vs. without training.
2. Need to discuss the value of Kappa and correlation on each analysis.
Author Response
1. Suggest having a power analysis on each phase.
-Thank you very much for your suggestion. We have followed the reviewer’s suggestiong and added the following paragraph to sub-section 2.1. in Materials and Methods, Design and sampling method:
“Considering that the aims of the study are to calculate correlations and agreements between nurses and ophthalmologists, as well as measuring their level of knowledge on the topic rather than calculating prevalence, the necessary sample sizes were esti-mated to be at least 65 nurses in phase 1 and approximately 100 fundus photographs of 100 different patients in phase 2. This would facilitate the calculation of non-parametric correlation coefficients with a significance of at least 0.28, thereby giving the study a 90% power in two-tailed hypothesis testing at an alpha significance level of 0.05 and 95% confidence intervals.”
2. Need to describe clearly how the author(s) randomly select fundus photographs from the EHRS.
- Thank you very much for pointing this out. We have tried to clarify this aspect by adding the following paragraph:
“To randomise image selection in both phases, an anonymous list is made by as-signing a unique number to each participating nurse in order of inclusion in the study. A fundus image from each nurses’ latest diabetes patients is then retrieved sequential-ly from each EHR in the Retisalud consultation module until the total number required is reached, i.e., 20 for phase 1 and 100 for phase 2. The fundus photographs selected for phase 1 were taken between May and June 2022, and those selected for phase 2 were taken between August and September 2022.”
3. Why the information is selected in 2020 or 2021?
The allusion to the years 2020 and 2021 was what was initially planned in the methodological section of this research project. The images included in the study all correspond to fundus photography tests carried out in 2022. Thanks to the recommendations of the reviewer, we have noticed this mistake and corrected it immediately, including the relevant modifications in the text and in accordance with the way the study was carried out.
4. In the EHRS, is any missing data on each selected record? If yes, will you replace them by choosing another record? Need to write it down in the data collection section.
- In view of the relevance of the reviewer’s observation, we have added a sentence clarifying this aspect to the penultimate paragraph in subsection 2.3, Data collection procedure: “Should the information on a given EHR not be complete, the case will not be included in the study and new data will be collected from a different patient.”
Results.
1. In 263-264, the author(s) need to provide statistics and p-value on each comparison.
- The wording of that paragraph has been improved and expanded to include that information:
“No significant differences were identified between sex and level of knowledge: overall mean total score between women and men was 39.5(4.0) matching answers versus 37.4(4.4) matching answers respectively (p=0.054). No significant associations were identified between the overall mean total score and age (Spearman’s rho correla-tion coefficient=0.164; p=0.176); years of experience (Spearman’s rho=0.100; p=0.410); or length of time taking fundus photographs (Spearman’s rho=0.226; p=0.060).”
2. In 267, the author(s) used correlation to determine matches. The rho value is not high (p=0.021). I can't see any discussion on this part in your discussion section.
- The wording on this matter has been revised and clarified in the last paragraph under Level of Knowledgein the Discussion section.
3. For the agreement between both professionals on fundus imaging screening in patients with diabetes (n=100), Kappa is 0.477 (p<0.001). Does it reflect good or low? Same for other results in Table 3, and 4, the Kappa is not high BUT sig., so does it refer to something need to discuss? According to McHugh (2012), Kappa, if range 0.2-0.6, refers to fair to moderate agreement only. Perhaps, need to discuss more in the discussion section.
McHugh ML (2012) Interrater reliability: The Kappa statistics. Biochem Med (Zagreb) 22(3), 276-282.
-The Discussion sub-section Agreement between nurses and ophthalmologists has been modified to better reflect the aspects resulting from the levels of agreement obtained, which, although statistically significant, are not optimal from the perspective of good agreement other than for normal screening versus non-normal screening.
4. In the aim, the author(s) wrote to assess the degree of agreement on screening for DR between nurses who received that training and ophthalmologists. Did all receive training? If no, which Table shows the difference in the agreement?
- That is right indeed, the nurses participating in this research, both in phase 1 and phase 2, have received the aforementioned training. This is explained at the beginning of subsection 2.2. Study setting, where it is stated that training is a compulsory condition for performing fundus photography. However, in section 2.1. Design and sampling method, to further clarify this issue, we have added that the nurses selected have received training and usually perform outpatient fundus photography in primary care.
Discussion
1. Need to discuss results between with vs. without training.
- Thank you for your comment. However, we believe it is not applicable, as explained in the previous section when we discussed the fact that all the nurses included in our study had received training. Such training is also a prerequisite for being able to perform fundus photography tests in the health area where the study was carried out.
2. Need to discuss the value of Kappa and correlation on each analysis.
- This recommendation has been addressed by amending the Discussion sub-section Agreement between nurses and ophthalmologists, as discussed above.
Reviewer 2 Report
The Limitations paragraph needs to be considerably expanded. First, the investigators are measuring the quality of nurse assessment made by the 42% of nurses who agreed to participate in the study. These nurses may be more confident of their skills than those who did not participate, and in fact their skills may have been higher. This limitation comes into play even more sharply for the 9 nurses who agreed to participate in the second part of the study.
Author Response
- The limitations section was certainly too short. We appreciate the suggestion and have incorporated a paragraph addressing the reviewer’s recommendation:
“This study features several limitations. Firstly, our results are not comparable with other healthcare areas where there are no fundus photography training programmes for nurses with similar characteristics to the one implemented in our setting. Secondly, the participant selection method may involve a notable bias, given that, among the 165 nurses trained in our health area to perform fundus photography, the 70 who finally participated did so voluntarily. This may be indicative of their motivation to do well in an area of expertise that they feel more comfortable performing than the nurses who chose not to participate. Finally, the nine nurses participating in the second phase could have been even better trained and more willing to participate. Nevertheless, the results point to a need for further training and competence development in the identi-fication and specification of fundus lesions in patients with diabetes.”
Reviewer 3 Report
This is a very interesting article that showcases the knowledge and diagnostic ability of diabetic retinopathy in PHC nurses. The writing is clear and easy to read, the introduction provides the necessary background, and the methods are adequately described. However, I have some suggestions for the results, discussion, and conclusion that could improve the study.
Introduction.
Line 110-111: I think the verb should be revised. I understand that the project has already been implemented.
Materials and methods.
No comments.
Results.
Line: 310-317: Please cite table 4.
Table 4: The subheading "are displaced" needs to be corrected to "5 years or less since onset" and "Values below 7%."
I understand that the sample size is limited, but considering the characteristics of the research, why was the diagnostic ability and Cohen's kappa index not evaluated in relation to experience in PHC and fundus photography training? Since 50% of the participants have less than 2 years of experience, this could be a cutoff point for this analysis.
Line 360-364: It attempts to answer, but no values are shown to corroborate such a statement.
Discussion.
Line 351-360: I don't understand the argument about the competencies of nurses, since the study is about trained nurses. If they receive training, then it would be reasonable for them to have these competencies. Could you clarify this point?
It seems to me that the discussion could be enriched by commenting on the clinical and epidemiological implications of the results.
Conclusion.
The conclusion does not refer to the main conclusions of the variables studied. It is more descriptive than critical.
Author Response
Introduction.
Line 110-111: I think the verb should be revised. I understand that the project has already been implemented.
- Thank you very much for your comments. We have modified the verb tenses in both sentences.
Results.
Line: 310-317: Please cite table 4.
- We have cited Table 4.
Table 4: The subheading "are displaced" needs to be corrected to "5 years or less since onset" and "Values below 7%."
- Those sub-headings have been relocated to centre them in the table based on their results.
I understand that the sample size is limited, but considering the characteristics of the research, why was the diagnostic ability and Cohen's kappa index not evaluated in relation to experience in PHC and fundus photography training? Since 50% of the participants have less than 2 years of experience, this could be a cutoff point for this analysis.
- The agreement tests were carried out using a sample of 100 patients that had recently been seen by the 9 nurses who participated in the previous phase. The data describing that 50% of the nurses had less than two years of experience performing fundus photography corresponds to the sample participating in phase 1, i.e., all 70 professionals. The database used in phase 2 contained patient information, and the nurses involved in this second phase all had at least two years of experience.
Line 360-364: It attempts to answer, but no values are shown to corroborate such a statement.
- That information has already been included in the results in the last paragraph of subsection 3.1 at the suggestion of another reviewer:
“No significant associations were identified between the overall mean total score and age (Spearman’s rho correla-tion coefficient=0.164; p=0.176); years of experience (Spearman’s rho=0.100; p=0.410); or length of time taking fundus photographs (Spearman’s rho=0.226; p=0.060).”
Discussion.
Line 351-360: I don't understand the argument about the competencies of nurses, since the study is about trained nurses. If they receive training, then it would be reasonable for them to have these competencies. Could you clarify this point?
- Thank you very much for your suggestion. We have reworded part of that sub-section for clarity. The idea is that the training that nurses receive is mainly focused on the use of technology to obtain high-quality fundus images, as well as on the identification (screening) of the result as normal or non-normal, which should speed up the process of patient referral to the ophthalmologist.
It seems to me that the discussion could be enriched by commenting on the clinical and epidemiological implications of the results.
- In line with that indication and the recommendations made by other reviewers, the Discussion section has been modified in an effort to enrich its content and to clarify and expand on a number of relevant aspects.
Conclusion.
The conclusion does not refer to the main conclusions of the variables studied. It is more descriptive than critical.
- Thank you very much for pointing this out. The wording of the Conclusion section has been completely reworded.
Round 2
Reviewer 1 Report
The authors addressed all the comments, and I am satisfied with all their corrections. Well done!
