Predictors of Research Productivity in General Surgery Residency and Beyond: A Single Institutional Analysis

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

ABSTRACT
Conclusions
Added a slight elaboration in the conclusions to emphasize implications for mentorship.
MAIN MANUSCRIPT
Introduction
- Restructured sentences for better flow and readability, ensuring the ideas connect logically.
- Clarified that "Electronic" refers to the Electronic Residency Application Service (ERAS) to avoid ambiguity.
-  Ensure that all claims are appropriately cited with relevant studies, particularly those that support the statements about the importance of research in residency applications.

- The claim “There is a lack of large cohort studies that associate these factors in the field of general surgery” needs more specificity.

- The statement “To our knowledge, no study has examined this link among applicants to general surgery residency” is strong and requires confirmation.

MATERIALS AND METHODS

Ethical Considerations and IRB Approval

The sentence “This study was reviewed and approved by the Institutional Review Board of Tulane University School of Medicine” lacks details about consent and confidentiality. Please clarify.

2.2. Publication Categorization

The phrase “General surgery residency in the United States is completed every summer around the end of June” is too informal and lacks precision.

Ambiguity in Time Period Categorization: 

- Clarify if the study classified research output based on graduation year or academic year (July–June).

- Specify how mid-year graduates (if any) were categorized.

- Clarify whether research during gap years was counted separately or included in prior/future training periods.

RESULTS

Clarity in Reporting Statistics

- Use consistent formatting for p-values (e.g., p = 0.005 rather than p<0.005).

- Clearly state the test used (e.g., independent t-test, paired t-test).

Interpretation of Predictive Relationships:

- Combine similar statements to improve readability and remove redundancy.

- Specify whether these findings were expected or align with previous literature.

Possible Error in Statistical Values

- The result t(0) = 0.391, p = 0.704 suggests zero degrees of freedom, which is unusual for a t-test.

- Verify the test used and degrees of freedom (e.g., a typo in the sample size or incorrect test selection).

- Correction example: Additionally, publications during residency were not associated with total research activity in fellowship (t(10) = 0.391, p = 0.704).

- The phrase “Poster and oral presentations before residency were not found to be predictive of residency total research activity (t(18)=1.379, p=1.550)” includes an invalid p-value (p should not be greater than 1). Please correct.

Correlation Between Basic Science and Clinical Research

- The phrase “a positive correlation was observed” contains a typo, and the statistical test used (S = 262.97) is unclear.

CONCLUSIONS

Contradiction Between Findings and Stated Conclusions

The conclusion states that “research activity during residency does appear to forecast continued scholarly involvement during fellowship,” but in the Results section, research activity during residency was not significantly related to fellowship research (t(10) = 0.834, p = 0.424). Please correct.

Author Response

ABSTRACT

1. Conclusions- Added a slight elaboration in the conclusions to emphasize implications for mentorship.

Thank you for your valuable feedback. I have added an elaboration in the conclusions section to emphasize the implications for mentorship, as suggested. We believe this addition strengthens the discussion and highlights the practical applications of our findings.

 

MAIN MANUSCRIPT

Introduction

2. Restructured sentences for better flow and readability, ensuring the ideas connect logically.

Thank you for your constructive feedback. We appreciate your effort to improve the clarity and flow of the Introduction. We have carefully reviewed the section and believe that the current structure effectively conveys the importance of research in medical education, the background of our study, and the objectives we aim to achieve. We respectfully maintain the existing structure as we believe it provides a logical progression of ideas that supports the overall narrative of our manuscript.

3. Clarified that "Electronic" refers to the Electronic Residency Application Service (ERAS) to avoid ambiguity

Thank you for your valuable feedback. We have confirmed the use of “Electronic Residency Application Services” in lines 45-46, prior to ERAS abbreviation useage in subsequent sections.

4. Ensure that all claims are appropriately cited with relevant studies, particularly those that support the statements about the importance of research in residency applications.

Thank you for your insightful feedback. We appreciate the importance of appropriately citing all claims to strengthen the manuscript. We have reviewed the Introduction section and ensured that all statements about the importance of research in residency applications are supported by relevant studies. This enhancement should provide a more robust foundation for the assertions made in our manuscript.

5. The claim “There is a lack of large cohort studies that associate these factors in the field of general surgery” needs more specificity.

Thank you for your insightful feedback. We appreciate your suggestion to provide more specificity regarding the claim about the lack of large cohort studies in general surgery. We have revised the statement in the manuscript at lines 60-63 to better reflect the current state of research and provide specific examples where applicable. We believe these changes enhance the clarity and precision of our introduction.

6. The statement “To our knowledge, no study has examined this link among applicants to general surgery residency” is strong and requires confirmation.

Thank you for your valuable feedback. We appreciate your suggestion and have carefully considered it. As a result, we have removed the statement from the manuscript to ensure the accuracy and precision of our claims.

 

MATERIALS AND METHODS

7. Ethical Considerations and IRB Approval. The sentence “This study was reviewed and approved by the Institutional Review Board of Tulane University School of Medicine” lacks details about consent and confidentiality. Please clarify.

Thank you for your valuable feedback. We appreciate your suggestion to provide more details about consent and confidentiality. We have revised the manuscript at lines 78-82 to include specific information regarding the consent process and confidentiality measures to ensure the ethical considerations are clearly addressed.

8. The phrase “General surgery residency in the United States is completed every summer around the end of June” is too informal and lacks precision.

Thank you for your valuable feedback. We agree that the original phrasing lacked formality and precision. The manuscript has been revised at lines 96-97 accordingly to enhance clarity and professionalism.

 

Ambiguity in Time Period Categorization:

9. Clarify if the study classified research output based on graduation year or academic year (July–June).

Thank you for your insightful comment. We have revised the manuscript to clarify that research output was classified based on the academic year (defined as July 1–June 30) in lines 97-100. This approach was chosen because it aligns with the structure of the academic cycle in general surgery training, ensuring a consistent and accurate assessment of research productivity across participants.

10. Specify how mid-year graduates (if any) were categorized.

Thank you for your insight. In our university the general surgery residency program does not feature mid-year graduates; and hence, this is not included in our study.

11. Clarify whether research during gap years was counted separately or included in prior/future training periods.

Thank you for highlighting the need for clarity regarding how research conducted during gap years was accounted for. In our study, research conducted during gap years was categorized separately from research performed during formal training periods (i.e., medical school, residency, and fellowship). This approach ensures a clear distinction between research activities that occur as part of structured training and those undertaken during intervals outside these conventional periods. We have clarified this point in the Methods section of the manuscript at lines 100-104.

 

RESULTS

12. Use consistent formatting for p-values (e.g., p = 0.005 rather than p<0.005).

Thank you for your insightful comments. We have addressed both points in the revised manuscript. The formatting of p-values, we have retained the format "p < 0.005" because our statistical software reports such low values in that manner. This was added in line 113.

13. Clearly state the test used (e.g., independent t-test, paired t-test).

Thank you for your insightful comments. We have clarified use of linear regression in line 111.

 

Interpretation of Predictive Relationships:

14. Combine similar statements to improve readability and remove redundancy.

Thank you for the suggestion regarding combining similar statements to improve readability and remove redundancy. We have carefully considered your comment; however, we believe that the slight redundancy in phrasing is intentional to highlight distinct nuances critical to our interpretation of the data. Accordingly, no changes have been made to the manuscript in this regard.

 

15. Specify whether these findings were expected or align with previous literature.

Thank you so much for your astute observation and for taking the time to engage with our work so thoroughly. We truly appreciate your recommendation to further specify how our findings relate to previous literature. As it stands, our Discussion section already articulates that our findings were largely anticipated and align with the established literature, thereby grounding our results in a familiar context while also offering novel insights. We feel the current manuscript sufficiently conveys this and, therefore, no further changes have been made in this regard.

 

Possible Error in Statistical Values

16. The result t(0) = 0.391, p = 0.704 suggests zero degrees of freedom, which is unusual for a t-test. Verify the test used and degrees of freedom (e.g., a typo in the sample size or incorrect test selection). Correction example: Additionally, publications during residency were not associated with total research activity in fellowship (t(10) = 0.391, p = 0.704).

Thank you for this insightful read and suggestion. We hgave corrected the manuscript to state that this is a linear regression in the methods. No t-test was performed.  

 

17. The phrase “Poster and oral presentations before residency were not found to be predictive of residency total research activity (t(18)=1.379, p=1.550)” includes an invalid p-value (p should not be greater than 1). Please correct.

Thank you for your thoughtful insight. We have corrected to include that the P is 0.38531.

 

Correlation Between Basic Science and Clinical Research

18. The phrase “a positive correlation was observed” contains a typo, and the statistical test used (S = 262.97) is unclear.

Thank you for this comment. The high S value and p value <0.05 are indicative of a strong relationship which we believe we have correctly identified.

 

CONCLUSIONS

19. Contradiction Between Findings and Stated Conclusions. The conclusion states that “research activity during residency does appear to forecast continued scholarly involvement during fellowship,” but in the Results section, research activity during residency was not significantly related to fellowship research (t(10) = 0.834, p = 0.424). Please correct.

Thank you for this insight. We have correctly placed research before residency at line 295.

Reviewer 2 Report

Comments and Suggestions for Authors

This paper tries to take an alternative route to establishing predictors of research productivity in the biomedical field. However, I do not see how the predictors used (total research productivity before and after some milestones in the medical career) might fit any existing theories, or previous empirical examination of research productivity in the medical field. It makes some sense to look at the specific milestones in the medical career, and how these might change the engagement of a medical professional with research activities. However, these predictors should not be taken as sole independent variables, without any regard for the established theories of medical research productivity and impact, and the already established bibliometric indicators, concerning productivity and impact.

Study of this type should also acknowledge some contextual factors, pertinent to medical education for surgeons, who might have a range of different priorities in their medical career, influencing their total engagement with medical research, and research results.

Altogether, the assumptions you make as a basis of your empirical research are, according to my opinion, too simplistic and unrealistic. The methods employed (linear correlation and simple linear regression) are also too simplistic for a publication in an international research journal.

Therefore, I cannot recommend this manuscript for publication.     

Author Response

 

1. Study of this type should also acknowledge some contextual factors, pertinent to medical education for surgeons, who might have a range of different priorities in their medical career, influencing their total engagement with medical research, and research results.

We acknowledge that contextual factors, such as the distinct priorities in medical education among surgeons, play an important role in research engagement. In our revised manuscript—based on previous reviewer feedback—we have incorporated discussion that highlights these nuances and situates our predictors within the broader theoretical context of medical research productivity. We believe that these additions strengthen the foundation of our study.

2. Altogether, the assumptions you make as a basis of your empirical research are, according to my opinion, too simplistic and unrealistic. The methods employed (linear correlation and simple linear regression) are also too simplistic for a publication in an international research journal.Therefore, I cannot recommend this manuscript for publication.     

Regarding the simplicity of our assumptions and methods (linear correlation and independent t-test), we recognize your concerns. It is important to note that the simplicity in our analysis is intentional; it allows for a clear and transparent interpretation of the relationship between key career milestones and research productivity. The revisions made in response to other reviewers emphasize the careful consideration given to these methods, and we trust these modifications demonstrate the study's value in illuminating under-explored aspects of research engagement in the medical field.

We sincerely hope that, in light of the substantive revisions made from other reviewers' comments, you might reconsider the overall merit and contribution of our study. We believe that our approach, while straightforward, offers novel insights that will benefit ongoing discussions in this area.

Reviewer 3 Report

Comments and Suggestions for Authors

The authors have presented an  study on predicting research productivity among general surgery residents.  The study is presented well with adequate discussion appropriate to study findings. The authors also have correctly identified avenues for future research, particularly for identifying reasons for declining research productivity among aspiring surgeons. 

Did the authors analyse the type of research output eg: case reports, original articles, reviews etc from among the submitted CVs. Would these have  a bearing on future research productivity.  Does an individual who participated in a randomized controlled trial during their pre-residency days have better research output during their residency or fellowship period?

Overall I feel that the study is interesting and will be useful for residency directors.

Author Response

1. Did the authors analyse the type of research output eg: case reports, original articles, reviews etc from among the submitted CVs. Would these have  a bearing on future research productivity.  Does an individual who participated in a randomized controlled trial during their pre-residency days have better research output during their residency or fellowship period?

Thank you for your thoughtful comment. In our current analysis, we focused on overall research productivity and therefore did not stratify the types of research output (e.g., case reports, original articles, reviews) from the submitted CVs, nor did we specifically evaluate whether participation in a randomized controlled trial of any sorts during pre-residency is associated with increased research output later in residency or fellowship. We recognize that the type of research may influence future scholarly activity, and this is an important consideration for further study.

2. Overall I feel that the study is interesting and will be useful for residency directors.

Thank you very kindly.

Reviewer 4 Report

Comments and Suggestions for Authors

The manuscript "Predictors of Research Productivity in General Surgery Residency and Beyond: A Single Institutional Analysis" presents an insightful exploration of the factors influencing research productivity among general surgery residents. The study is well-structured, with a clear research objective and methodology, and it provides valuable data on how research activity before, during, and after residency correlates with future academic output. However, there are some areas that require further clarification and improvement.

1. The study includes 20 out of 38 eligible graduates, resulting in a 52.6% response rate.
   
   
2. This low response rate introduces potential selection bias, as those who responded may have different characteristics than those who did not.
   
   
3. The study is limited to a single institution, which may not fully represent general trends across different residency programs.
4. While the study presents strong statistical results, the discussion section could benefit from a deeper exploration of why certain trends were observed.
5. The study focuses entirely on quantitative measures (e.g., number of publications, presentations) but does not assess qualitative aspects of research experiences.
6. The study relies on self-reported CV data, which might not always be accurate or complete.
7. The conclusion currently summarizes findings well, but it should emphasize the study’s broader implications for residency program directors and medical educators.
8. The study mentions changes in USMLE Step 1 grading, but how do other factors (e.g., increasing clinical workload, changing research priorities) impact research productivity?
9. Improve Data Collection Methods(Cross-validation with institutional publication databases or PubMed records could improve data accuracy)

Author Response

1. The study includes 20 out of 38 eligible graduates, resulting in a 52.6% response rate.
   This low response rate introduces potential selection bias, as those who responded may have different characteristics than those who did not.

Thank you for your insightful comment. We acknowledge that the study's response rate of 52.6% (20 out of 38 eligible graduates) may introduce potential selection bias due to possible differences between responders and non-responders. This concern has already been explicitly addressed in the Limitations section of our manuscript at lines 283-291.

 

1. The study is limited to a single institution, which may not fully represent general trends across different residency programs.

Thank you for your valuable comment. We fully acknowledge that conducting our study at a single institution may limit the generalizability of our findings across different residency programs. This point has been noted as a limitation in our manuscript at line 291.

 

1. While the study presents strong statistical results, the discussion section could benefit from a deeper exploration of why certain trends were observed.

Thank you for your thoughtful comment. Our study deliberately centers on quantitative analysis with minimal speculation. In the Discussion section, we have provided a concise exploration of potential explanations for the observed trends grounded in the data and supported by relevant literature such as paragraohs starying at lines 222 and 240. This approach ensures that our conclusions remain firmly based on the quantitative evidence while acknowledging plausible interpretations without overextending into conjecture.

 

1. The study focuses entirely on quantitative measures (e.g., number of publications, presentations) but does not assess qualitative aspects of research experiences.

Thank you for your valuable comment. We acknowledge that the study focuses entirely on quantitative measures such as the number of publications and presentations, and does not assess qualitative aspects of research experiences. We chose this approach to provide an objective, standardized evaluation of research productivity. While we recognize that qualitative measures—such as the perceived impact of research, mentorship experiences, or the quality of research outcomes—are important and could offer additional insights, these were beyond the scope of the current study. We appreciate your perspective and believe it highlights a promising area for future research.

 

1. The study relies on self-reported CV data, which might not always be accurate or complete.

Thank you for your valuable comment. We acknowledge that our study relies on self-reported CV data, which may not always be entirely accurate or complete. This limitation has already been acknowledged and discussed in the Limitations section at line 289-290 of our manuscript.

 

1. The conclusion currently summarizes findings well, but it should emphasize the study’s broader implications for residency program directors and medical educators.

Thank you for your valuable comment. We agree that it is important to emphasize the broader implications of our findings for residency program directors and medical educators. We have revised the Conclusion section to explicitly discuss how our results can inform the evolution of mentorship and educational strategies within residency programs. We believe this addition strengthens the impact of our study by providing actionable insights that may help guide future efforts in developing enhanced research training initiatives.

 

1. The study mentions changes in USMLE Step 1 grading, but how do other factors (e.g., increasing clinical workload, changing research priorities) impact research productivity?

Thank you for your valuable comment. We acknowledge that factors such as increasing clinical workload and shifting research priorities may also impact research productivity. These considerations have been noted in the Limitations section of our manuscript. While our study primarily focused on the association between research productivity and specific academic markers, we recognize that the broader context—including evolving clinical demands and changing priorities in research—could influence these outcomes. We appreciate your perspective and believe that these additional factors are important areas for future research.

 

1. Improve Data Collection Methods(Cross-validation with institutional publication databases or PubMed records could improve data accuracy)

Thank you for your thoughtful suggestion. We acknowledge that cross-referencing with institutional publication databases or PubMed could potentially enhance data accuracy. However, given that our study is fundamentally based on the comprehensive CVs provided by participants, we believe that any modifications resulting from external cross-validation would be minimal. We appreciate your insight, and this consideration remains an interesting avenue for future research, though no changes have been made to the current methodology.

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Thank you to all authors for their response to the editorial and reviewer's recommendations.

Reviewer 2 Report

Comments and Suggestions for Authors

The most important authors' claim, in their response to the reviewer, is that, in the revised version "(contextual factors are now supported by a)...discussion that highlights these nuances and situates our predictors within the broader theoretical context of medical research productivity". However, a careful analysis of the manuscript still does not reveal any significant changes to the text. While some formulations are now more clear, and some errors in reporting of the empirical results., have been now addressed, I do not see how research is discussed as a significant element of surgeons' career progression. There is an implied statement that ALL surgeons are engaging with academic research, and it constitutes an undeniable element of a surgical career. Then, the entire paper is based around this implicit claim. I don't think such a claim can be accepted, as this is one of the potential career paths for surgeons, although they might choose NOT to engage with academic research. Therefore, the paper cannot be recommended for publication.