The Influence of the Sherman STEM Teacher Scholars Program on Persistence in Science, Technology, Engineering, and Mathematics: A Mixed-Methods Study

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Thank you for the opportunity to review this manuscript. It holds interest for me, and is an important topic.  

However, I found the topic may be best suited as a pilot.  I did not have confidence in the reporting, and the assumptions were broad. 

One issue that downgrades the trustworthiness/power of the manuscript is the interchange of using the terms "exploratory" and "explanatory". This confusion could be mine as a reader, but what exactly is the study? 

In the introduction, with the claim of "...the culture of STEM creates a hostle and racist environment for students of color..."  it would strengthen the argument to expand/contextualize this statement.  I do not disagree, but the premise of the study, I believe is to show how participation in the program combats these issues, yet they are not expanded upon. 

Considering the research questions and subsequent limitations, but it serve the study to clarify and reduce the number and aspect of the questions? 

Citing one person's experience as an example of faculty role seems like it could be built out.  I am most curious about the Belonging and STEM Persistence, yet there was only one paragraph, as with the other contextual literature.  I found the several paragraphs with various topics to not go deep enough and spread the work too thin.  In addition, on page 2, it would bolster the argument you are proposing to be more formal than "...researchers have unpacked how there must be discussion..."  For this reviewer, I needed to have more claims and evidence in the introduction and literature review to see how your proposed study adds to the conversation.  I don't disagree with what you are attempting, but this reviewer failed to see a contextual depth prior to presenting the findings of your study.  

For those not familiar with CPQ, as on page 5, perhaps writing it out as a reminder and restating some familiarity on what it is would be beneficial. This section seemed confusing and read teadious.  Perhaps a chart or table with the categories? 

In the methodology and methods, perhaps the the last few sentences in the first paragraph could be moved up in the manuscript?  This would help contextualize the project, in this reader's opinion. 

In research design, again is it exploratory or explanatory?  and what "list" is referred to of the 1500 non-STEP STEM majors? 

In the discussion, is it really the results of the quantitative analysis that suggests college persistence is impacted by race and gender?  We know this across many contexts. How is your study coming to this conclusion different than what is known? I would think belonging and mattering is where there is something to be added.  

Overall, I don't think this study, as written, adds anything to the literature, and could be more specific on what it is trying to say and do.  Perhaps a pilot and clarity on what the program did, rather than the limited literature review and claims being made that are not necessarily outside of what the literature already claims and shows.  I like the idea, but it seems to be a stretch with the claims.  

Comments on the Quality of English Language

Overall, the quality of English seems ok. There are a couple of informal statements, and the lack of consistency between exploratory and explanatory. 

Author Response

Reviewer #1--please see the attached document outlining the suggests raised by the reviewers and how we addressed each of your concerns.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Dear Authors,

I have read your article with great interest and can see the merit of your work being published. I can see that you are trying to cover a huge territory and many different aspects of the STEP Program. I was left to wonder if maybe you are trying to do too much ( gender, race, belonging/persistence, etc.) in the one paper? Maybe you could split this paper and give each of these aspects the attention they deserve? Food for thought….

I will now provide you with feedback that is designed to help you improve the article and add to its quality.

Comments:

Introduction and Review of Literature Sections

1) please explain what exactly you mean by STEM education in the context of your study and this article. There are two different ways of understanding STEM: a) as an umbrella term describing the separate academic disciplines comprising Science, Technology, Engineering and Mathematics courses - this is often used in public parlance and university jargon alike to simply refer to the field of science-related disciplines; or b) an integrated view of STEM (see curriculum integration) meaning, starting with Science, the other disciplines of Technology, Engineering and Mathematics are integrated into a coherent and co-taught curriculum which is very rarely seen in university environments as a course in its own right. When we speak of STEM or STEAM in school environments, we almost always refer to the latter i.e. b)

2) please make sure that all acronyms are explicitly explained so that non-US readers can make good sense of the acronyms. 

Methodology

3) please note that every study should be stating whether or not Ethics Clearance was obtained for the study from the University. This is particularly essential when investigating differences in race, gender, educational opportunities, performance and persistence, etc..

Especially since the University’s name has been stated - you may reconsider that! - we need to know what efforts have been undertaken to ensure confidentiality of the research participants especially in terms of privacy/confidentiality. We, the readers, don’t even hear whether or not the names listed are pseudonyms or real names. In short, protecting participants’ privacy is of paramount importance.

4) the research questions would benefit from re-writing - clear and concise would be good.

5) you chose mixed-methods methodology which is fine for your research purpose. Because of that, it is appropriate to use validity and reliability and triangulation for research quality assurance purposes. Please note that Crystallisation which you mentioned in your paper is better aligned with Constructivist/Interpretivist research methodologies since they allow you to bring in other research strategies in addition to questionnaires and interviews. Since you only had those two approaches plus focus groups, I believe, triangulation is appropriate and sufficient. I suggest deleting Crystallisation.

Results and Discussion:

This is where the article needs the most work.

6) whilst the quantitative results are presented separately, the qualitative are a mix of quant and qual and of STEP and non-STEP which gets confusing. There were several occasions where you lost me as to who exactly you were referring to …please check and improve readability! 

7) make good use of the interview data and include as many verbatim quotes  as evidence for your claims to know. You mentioned you have good interview data.

8) currently you present your results using a ‘language of certainty’ that tells us, “ We know for sure that this is the case!” Fact is, unless you did experimental research with a randomised design you are unlikely to have data that can give you certainty about causality, and therefore all knowledge claims are ‘best guesses’ based on the data presented. This applies to qualitative data in particular but also to correlational results. This means that a language of probability (could, would, might, may, etc….) is more appropriate than a language of certainty.

Suggestions:

I feel strongly that each one of the aspects you are trying to cover, e.g., gender and color, bullying and stereotyping in university courses, etc. deserves your full attention, and deserves to be covered perhaps in a separate article. Maybe separating STEP and non-STEP might also help.  At the moment, each only gets a very superficial mention without going into any depth yet you say you collected good data from your interviews… please consider how best to use them.

You may wish to adjust strong knowledge  claims (e.g., students believe…) in the results using a language of probability.

Make sure you provide evidence that links directly to the research questions/problem you mentioned at the outset. Please check.

I wish you all the best with the revisions and look forward to reading your revised work.

Comments on the Quality of English Language

Just a few typos and minor grammar issues.

Author Response

Reviewer #2--please see the attached document outlining the suggests raised by the reviewers and how we addressed each of your concerns.

Author Response File: Author Response.pdf