Clinical Trials Gone Missing—A Potential Source for Publication Bias in Dentistry

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

I congratulate the authors for their work.

The chosen topic is interesting and valuable for clinicians and researchers as well. I have some recommendations:

1) Introduction

- I would suggest moving the paragraph from lines 88-91 ("Likewise...failure rates") to the Discussion chapter

-maybe a hypothesis would be useful to be added here

2) Materials and methods

-it is unclear from lines 134-140 and Table 1 and Figure 1 where those 15 articles are included. It's about 663+15? Or the 15 is included in 663?

Thank you

Author Response

• I would suggest moving the paragraph from lines 88-91 ("Likewise...failure rates") to the Discussion chapter - The review of industry sponsorship as a source of publication bias was presented as part of the introduction to publication bias at large. I opted to move it to the discussion, as the study itself did not include such a group.
• maybe a hypothesis would be useful to be added here -  A hypothesis was added (95-96)
• Materials and methods -it is unclear from lines 134-140 and Table 1 and Figure 1 where those 15 articles are included. It's about 663+15? Or the 15 is included in 663? - The 15 were part of the original group of 744. I added a sentence to clarify this point (144).  

Reviewer 2 Report

Comments and Suggestions for Authors

Dear Authors,

I haven't found any serious issue, therefore I recommend a minor revision. Below these are some suggestions for You:

1. Title: As the research uses dental related topics, I suggest to indicate it in the title

2. Line 24: It should be 7 duplicates, according to Figure 1

3. Line 26: There is different value in line 183, but it should be the same

4. References 6 (line 67), 7 (line 74) 23 (line 285) need to be corrected

5. Line 183: There is different value in line 26

Best regards and good luck

Author Response

Title: As the research uses dental related topics, I suggest to indicate it in the title  - The title was modified (2).

Line 24: It should be 7 duplicates, according to Figure 1 - Corrected: 7 replaced 4 (24-25)

Line 26: There is different value in line 183, but it should be the same -  The inconsistency was corrected (27)

References 6 (line 67), 7 (line 74) 23 (line 285) need to be corrected - Reference 6 was removed. the newly numbered 6 & 7 are now in accordance with the text (77, 78). As for reference 23 (now #21)  I checked again for the accuracy of the reported results and the citation in the reference list, but could not find an error.

 

Reviewer 3 Report

Comments and Suggestions for Authors

Thank you for your research and submission regarding the paper “Clinical Trials Gone Missing – A Potential Source for 2 Publication Bias”.  The aim of this study was to examine publication bias associated with failure to report clinical trial results of studies in the dental field, that were initially posted on www.ClinicalTrials.gov registry and further explore variables that might have affected these publication rates.

I found this study interesting because of the thematic. I have a CT registered and was able to publish the results. It was quite an adventure! Well let’s go and do the job requested.

 

The clinical field was limited to “limit our search to topics related periodontal and implant topics. These included: toothbrushes, dental lasers, dental implants, periimplantitis, regenerative procedures (GTR, GBR, bone grafts) and root coverage”. Therefore, in the limitations, you must remember the readers for this important detail.

 

Material and methods.

Data extraction was made jointly done by the two primary authors for entries updated until May 26, 2021. But then, Inclusion criteria states “Clinical trials listed on www.ClinicalTrials.gov associated with above keywords and listed as completed on or before December 31, 2019”. And the Exclusion criteria: Studies listed as not completed or completed after December 31, 2019.

-          Why did you use a time lapse of almost 1 and half years between the last “updated” and the “completed” according to the inclusion criteria? Include a small phrase explaining why for this time lapse.

-          I do think you must include a phrase explaining that you retried studies ranging from 2015 – 2019, as the information is only available in table 2.

Results:

Table 2. A suggestion for using a more usual form of conveying p-values in the case of the “p=5.4x10^-6”: write it as “p<0.001” (both in the table, text, and abstract)

It’s interesting that commencement of the CT prior to registration is the only significant factor associated with an increase of the chance for publishing the study (has an OR=2.2). Today this would be a huge “no-no” regarding Ethical commissions and so forward…

Table 3- please change the background colour of the first line of the table. Its almost imperceptible to read the information.

Regarding all p-values, you must use always the same number of decimal places. The usual is to use 3. But if you want you might use 4. But it is very strange to see some with 2, other with 3 and other with 4. And do not use scientific notation (.. x10^…).

Abstract:

The phrase “Multi-center studies and those done in commercial facilities had much higher publication rates (56.5% and 58.3% respectively)” is a bit inaccurate although the numbers are correct. Please rewrite as “Multi-center studies and those done in commercial facilities had much higher, though non-significant, publication rates (56.5% and 58.3% respectively)”.

Do not forget to correct the p-values.

Author Response

The clinical field was limited to “limit our search to topics related periodontal and implant topics. These included: toothbrushes, dental lasers, dental implants, periimplantitis, regenerative procedures (GTR, GBR, bone grafts) and root coverage”. Therefore, in the limitations, you must remember the readers for this important detail - The title was modified to better represent the study population (2)

Material and methods: Data extraction was made jointly done by the two primary authors for entries updated until May 26, 2021. But then, Inclusion criteria states “Clinical trials listed on www. ClinicalTrials.gov associated with above keywords and listed as completed on or before December 31, 2019”. And the Exclusion criteria: Studies listed as not completed or completed after December 31, 2019.

Why did you use a time lapse of almost 1 and half years between the last “updated” and the “completed” according to the inclusion criteria? Include a small phrase explaining why for this time lapse - The reason being that we allowed enough time (~18 months or longer) for a completed study to get published (It would have been an error to include a study that was completed in late 2020 as unpublished by May 2021, as it might very well be in the pipeline).  

I do think you must include a phrase explaining that you retried studies ranging from 2015 – 2019, as the information is only available in table 2 - Studies that commenced after 31 December 2016 and onwards, could have not yet been completed; thus we could not deem them "terminated".

Results: Table 2. A suggestion for using a more usual form of conveying p-values in the case of the “p=5.4x10^-6”: write it as “p<0.001” (both in the table, text, and abstract) - Agree.  these changes were made throughout the text and tables.  

It’s interesting that commencement of the CT prior to registration is the only significant factor associated with an increase of the chance for publishing the study (has an OR=2.2). Today this would be a huge “no-no” regarding Ethical commissions and so forward… - Agree!

Table 3- please change the background colour of the first line of the table. Its almost imperceptible to read the information- Background of table 3 was changed

Regarding all p-values, you must use always the same number of decimal places. The usual is to use 3. But if you want you might use 4. But it is very strange to see some with 2, other with 3 and other with 4. And do not use scientific notation (.. x10^…)- In accordance with this comment I revised all tables and text with a unified 3 decimal points.  

Abstract: The phrase “Multi-center studies and those done in commercial facilities had much higher publication rates (56.5% and 58.3% respectively)” is a bit inaccurate although the numbers are correct. Please rewrite as “Multi-center studies and those done in commercial facilities had much higher, though non-significant, publication rates (56.5% and 58.3% respectively)” - This phrase was re-written as suggested

 

Reviewer 4 Report

Comments and Suggestions for Authors

The manuscript "Clinical Trials Gone Missing – A Potential Source for Publication Bias" addresses a relevant and critical topic in medical research, the under-reporting of clinical trials. The comprehensive analysis of clinical trials registered in the clinicaltrials.gov database, particularly in the fields of periodontology and implantology, provides interesting insights. I congratulate the authors on their work.

Major Comments:

1. Please check the reported numbers in the flowchart (Figure 1), the methods section, and the abstract regarding duplicated studies, and revise as necessary.

2. Consider moving the flowchart (Figure 1) and the first table to the results section, as they pertain to the number of trials screened and efforts to detect missing manuscripts from the initial search rather than methods themselves.

3. The first paragraph of the discussion is confusing. Consider revising to avoid arbitrary referencing (e.g., "these topics" - which topics?; "these numbers" - which numbers?).

Minor Details:

4. While describing the univariate results, avoid specifying the percentage of two-state variables twice; it is redundant, as they are expected to sum to 100%.

5. Revise tables and figures to improve readability.

6. Please describe why correction for multiple testing was not applied, or revise the results to incorporate it.

7. Avoid phrases like "closer to statistical significance" (e.g., line 216). 

8. Keep the reporting of p-values consistent in regard to the number of significant figures.

Comments on the Quality of English Language

Some mistakes detected should be revised (this list is not comprehensive):

1. Use of lowercase at the start of sentences (e.g., in the abstract).

2. Consistency for the clinicaltrials.gov name, which is provided in various forms such as: "ClinicalTrials.gov", "Clinicaltrials.gov", "clinicaltrials.gov", "www.ClinicalTrials.gov", "www. ClincialTrials.gov"

3. Scientific notation should be standardized; it is currently provided as: 3.3x10^-6, 5.4x10^-6

Author Response

1. Please check the reported numbers in the flowchart (Figure 1), the methods section, and the abstract regarding duplicated studies, and revise as necessary - Adjustments were made. figure 1 and the  abstract (24-25) and M&M (143-152) are consistent.  

2. Consider moving the flowchart (Figure 1) and the first table to the results section, as they pertain to the number of trials screened and efforts to detect missing manuscripts from the initial search rather than methods themselves - This is a boarder-line case.  Both figure 1 and table 1 describes how the methodology worked and how we have arrived with the final patients base.  I would rather keep it in the methodology.  

3. The first paragraph of the discussion is confusing. Consider revising to avoid arbitrary referencing (e.g., "these topics" - which topics?; "these numbers" - which numbers?) - I have eliminated these vague wording and made it read better (229, 230, 266, 290-291)

Minor Details:

4. While describing the univariate results, avoid specifying the percentage of two-state variables twice; it is redundant, as they are expected to sum to 100% - Done (192-195)

5. Revise tables and figures to improve readability - Done

6. Please describe why correction for multiple testing was not applied, or revise the results to incorporate it - We opted to do these further test:  (1) We added an ANOVA test p-value for the research setting and topic analyses to describe an overall association of the variable with publication. The original tables only compare odds ratios between categories, but not an overall result across all categories. (2) We modified the “reduced model” analysis that corresponds to table 6. Instead of simply using ~0.05 as the p-value cutoff, I used 0.0045 (0.05 / 6) as a more stringent threshold since we tested 6 variables. The M&M statistical section (170, 172-174, 178-180) and tables 2-6 were modified to reflect this. Where necessary, we amended the results in the results section. A paragraph was added to the results section ( 229-232).

    

7. Avoid phrases like "closer to statistical significance" (e.g., line 216).  -Done (217)

8. Keep the reporting of p-values consistent in regard to the number of significant figures - This was unified to three decimal points (with anything under 0.001 presented as <0.001).  

Comments on the Quality of English Language

Some mistakes detected should be revised (this list is not comprehensive):

1. Use of lowercase at the start of sentences (e.g., in the abstract). - Done (15,32)

2. Consistency for the clinicaltrials.gov name, which is provided in various forms such as: "ClinicalTrials.gov", "Clinicaltrials.gov", "clinicaltrials.gov", "www.ClinicalTrials.gov", "www. ClincialTrials.gov" - Spelling was modified according to the web's name: "ClinicalTrials.gov".    

3. Scientific notation should be standardized; it is currently provided as: 3.3x10^-6, 5.4x10^-6 

Done.  all p values are now decimal to the 3rd decimal number only.  

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Thank you for the opportunity to review this work.

Once again, I congratulate the authors for their efforts.

I have a minor recommendation:

* References

- please insert all the references according to the Journal's indications

Reviewer 4 Report

Comments and Suggestions for Authors

My comments were adequatelly adressed and I have no further comments.