Evaluation of Smiles for Life: A Caregiver Focused Oral Health Education Programme

Round 1

Reviewer 1 Report

Disabilities

Evaluation of Smiles for Life: A Caregiver Focused Oral Health Education Program

June 5, 2022

This manuscript describes a program targeted to reach caregivers of persons who have an intellectual or developmental disability (IDD) and in need of routine oral health care in an effort to improve the knowledge, attitude, and skills of caregivers. The authors describe this as a pilot study and it reads as such so much so that I wonder if this should have been conducted as a pilot and then changes made to the education program and trialed after those edits. I don’t think we learn a lot here within this pilot – more so that what one might expect – such that having done this pilot, it would have been more interesting to read about a ‘study’ that was developed post-pilot having addressed the issues that arose herein. What seems to be the take-home message here is what one could easily have predicted; thus, it doesn’t really seem to add to the literature and our knowledge base.

Abstract:

·         I think the authors addressed a very important topic and took on the difficult task of trying to increase the knowledge of caregivers such that the person’s oral health care could be improved. The difficulties of assisting any health care needs of persons with an intellectual or developmental disability (IDD) should not go unstated. The Abstract does a reasonably good job of highlighting the different components of the manuscript.

·         It was unclear to me why a factor analysis was conducted to explore the results – when, in fact, there was no discussion of the factor analysis methodology and results within the manuscript proper.

·         There was little connection between the use of the Smiles for Life program and those with intellectual or developmental disabilities – such that this should be viewed as the ideal program to use.

Introduction:

·         The authors clearly outline the importance of known health care disparities in persons with an IDD and specifically focus on the need to address these disparities within the context of oral health. The review of the literature seemed very complete.

·         There was no specific definition used for what ‘qualifies’ as an intellectual or developmental disability. There is clearly a wide range of definitions whether they are related to specific diagnoses or the attainment of specific skills. The authors note, much later in the manuscript, that one of the limitations might be the heterogeneity of the persons being cared for as well as those who are the caregivers. This is quite true.

Methods:

·         There was little clarity in the use of the term ‘anonymity’ of the data from the study participants; it seems that the data could not have been anonymous if there was to be pairing between the pre- and post-survey scores. When one uses a paired methodology, then a paired analysis should be conducted – but it’s unclear how the pre- and post-survey data were paired while maintaining anonymity of the data. This needs a better description.

·         While the educational intervention was reasonably well described, the data collection was not as well done. It’s not clear what the 13 questions were that constituted assessing the knowledge base of the caregivers participating in this study and how those questions were scored. This would be important in terms of looking at the results and assessing the use of the various statistical tests conducted.

·         The data analysis section within the Methods was very, very brief and didn’t appear to truly mimic the tests that were conducted. It was a surprise to see within the Abstract the use of factor analyses when these were not mentioned in the Methods (nor presented in the Results). It was equally surprising to see in the Results (and tables) the use of logistic regression techniques when these were not mentioned in the Methods as one of the statistical tests utilized to assess the data.

Results:

·         The authors did a good job walking the reader through the sample size – the numbers at the beginning, how various steps reduced that sample, and the ultimate group whose data was used in the analyses assessing the SFL program.

·         In the section on ‘oral health literacy’, data is presented as means and SDs; however, there is no indication of the direction of these scores and how they were collected pre- and post-training; i.e., is a higher score ‘good’ or ‘bad’? It’s hard in Table 2 to read the results when it’s not clear the direction of the data. There are many, many increases and decreases from pre- to post-test.

·         It was very hard to believe (in Table 2) that an extremely small change in score changes from pre- to post-intervention assessment would have been statistically significant. The sample size is likely not to have contributed significantly to this – but the results do not seem plausible.

·         The data presented in Table 4: predictors of higher post-training knowledge scores among the SFL participants also seems contrary to what’s described in the text written just before the presentation of Table 4. The authors report using a binary logistic regression analysis, yet the typical ORs are presented alongside beta coefficients and standard errors – that one would have seen more in line with linear regressions. And while the authors note the OR and 95% CIs were ‘adjusted’, it’s not clear what covariates were used to adjust the outcome variable(s).

·         As noted earlier, there is no presentation of factor analysis results though the Abstract notes that this type of analysis was conducted.

·         Toward the end of the Results section, there is an analysis of the open-ended questions; however, it’s not clear what those questions were and how they were collected and analyzed.

 Discussion:

·         Ultimately, I believe that the work presented herein is truly a pilot (as the authors note at the beginning of the Discussion). However, I believe that this pilot doesn’t really warrant publication – but rather the authors should use the lessons learned herein and then conducted a more complete and thorough study that would have been improved based on that learned through the conduct of this (pilot) study.

·         While I do believe that we don’t do enough reporting of ‘negative’ findings from studies such that we miss the opportunity to showcase what’s working and what’s not (the latter being new knowledge in and of itself), it seems that the results from this study are ‘all over the place’ – there are so many unexpected ‘negative’ findings in terms of no change or a decrease in knowledge from pre- to post-intervention assessment, I wonder whether this ‘new data’ truly adds to the knowledge base of what we know on this topic and with this population. It seems that what’s added to the literature here isn’t at a level sufficiently high to warrant publication.

·         One of the key findings (or an interpretation of the results) is the thought that the heterogeneity of participants in the program resulted in findings that were not expected. Both the heterogeneity of the participants as well as that of those persons with IDD would have made it difficult for anyone to truly assess the educational intervention presented. Perhaps future studies could assess this type of change in knowledge in sub-populations of persons (both subjects of the study – the caregivers, as well as those persons they are caring for – those with an IDD).

·         The Limitations section within the Discussion had a number of ‘misses’ in not identifying more. For example, there is a lack of generalizability, some possible information bias in the collection of self-reported data (and the social desirability which may have influenced how one would report their pre- and post-knowledge), and a potential issue with matching pre- and post-intervention data in order to do a paired analysis. 

Conclusion:

·         No comments.

Author Response

Please see the attachment which contains response to comments from all reviewers

Author Response File: Author Response.docx

Reviewer 2 Report

Thank you for sharing your research project. Although the oral health literacy of the participants did not improve after the education intervention, the authors have thoroughly examined and analysed the problems from the study. The share information made valuable and valid suggestions to all the health educators for further research and education. Congratulations to the research team.

Please see minor comments in the manuscript. 

Comments for author File: Comments.pdf

Author Response

Please see the attachment which contains response to comments from all reviewers

Author Response File: Author Response.docx

Reviewer 3 Report

Methods:

·       Data analysis- I would like to suggest for authors to add type of statistical test involved in this study for each outcome measure) (e.g. Mann Whitney, binary logistic regresssion, etc) in methods section.

·       For oral health literacy and knowledge outcome- any cut off point for good or poor oral health literacy/ knowledge score?

Results:

·       Page 7- authors reported results on oral health literacy score. However, not clear how you measure oral health literacy and which instrument was used for oral health literacy items? Are you using the same items for knowledge outcome?

Discussion

·       Perhaps authors can add/propose strategies for tailored-made oral health education activities for caregivers in this study population·       Please check consistency of references formatting. In particular on authors’ name and journal’s name. Majority of the references did not have journal name. Please check this and comply to authors’ guideline.  

Author Response

Please see the attachment which contains response to comments from all reviewers

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

I think the authors did a reasonable job in responding to my initial concerns in describing the Methods and Results. However, they appear to recognize that perhaps the training and the knowledge-based questions were pitched too high for the audience, plus the heterogeneity of those attending and those with a varying number/type of intellectual disability - made it difficult to see change over time given that there was little change. The regression results that pre-workshop knowledge was the only significant predictor of post-knowledge was no surprise, though it was still unclear what additional variables were adjusted for (it's unclear if it's the other variables on the table). I think this is a good pilot study from which to build on.

Author Response

1.

"I think the authors did a reasonable job in responding to my initial concerns in describing the Methods and Results."

response:
Thankyou for acknowledging this, we made a number of changes that, we felt, improved the clarity of these sections.

2.
"However, they appear to recognize that perhaps the training and the knowledge-based questions were pitched too high for the audience, plus the heterogeneity of those attending and those with a varying number/type of intellectual disability - made it difficult to see change over time given that there was little change."

response:
Thanks for this comment, yes this is one of the core outcomes from this study that we, and others, will need to consider in future oral health education programs. In the previous revision, we made a number of suggestions for how future programs could be developed and presented.

3.
"The regression results that pre-workshop knowledge was the only significant predictor of post-knowledge was no surprise, though it was still unclear what additional variables were adjusted for (it's unclear if it's the other variables on the table)."

response:
The variables in the table were the variables controlled for. Although this information has already been included in the data analysis section, the associations with knowledge regression section has been reworded for clarity.

4.

"I think this is a good pilot study from which to build on."

response:
Thank you, we agree, this study has highlighted a number of interconnected issues that need to be considered into the future.