Gender and STEM Education: An Analysis of Interest and Experience Outcomes for Black Girls within a Summer Engineering Program

Round 1

Reviewer 1 Report (Previous Reviewer 3)

Comments and Suggestions for Authors

The authors made great adjustments based on the reviewers suggestions. The added paragraphs throughout the beginning of the manuscript greatly strengthen the context and position this work as a needed addition to the literature. The additional references added greater rationale and depth to the literature review and conceptual framework. The revisions addressed the reviewer comments very well! 

Some of the references were not listed for the in-text citations and should be reviewed. The citation read "Error! Reference source not found." 

Author Response

Thank you for the note. Regarding in-text citations, they were addressed appropriately.

Reviewer 2 Report (New Reviewer)

Comments and Suggestions for Authors

This manuscript has been completely re-worked and is much improved. The content will be of interest to the STEM community. It is also an issue that is highly debated (single sex education).   

Author Response

Thank you so much for taking the time to review our article and for giving this feedback. Much appreciated. 

To Editor: Reivewer did not have any requests, only positive feedback on the changes we made. 

Reviewer 3 Report (New Reviewer)

Comments and Suggestions for Authors

This study investigated the differences in learning environments for Black girls in single-gender versus coed settings using data from as part of the Search for Education, Elevation and Knowledge (SEEK) program at different sites.

The conceptual framework was described effectively and Intersectionality  Social Cognitive Theory well-defined and supported with literature.

There was significant data supplied but at times the purpose of the data in relation to the focus of the paper, ie. single versus co-ed educational settings, was not always clear. I think the readability of the paper would be improved if the relevance of the data was more explicitly explained.

Overall, this paper provides a spotlight on an area that should be seen as a priority for research. As such, it is recommended the paper be published but it's impact would be improved by tightening the focus.

Author Response

Thank you for the comments. 

We addressed your issue by clearly conceptualizing the main question of our study to inform the reader of what to expect. We also added a section on the survey to provide more details (lines 329-341).

Reviewer 4 Report (New Reviewer)

Comments and Suggestions for Authors

The article discusses the experiences of black girls in a summer engineering studies program.

The introduction is very interesting and clearly describes the complexity of the situation to be studied. The sections on STEM Education and coeducation contextualize the problem in a satisfactory way. In fact, the section on theoretical framework lays the groundwork for the study. After all this introduction, I miss the explicit formulation of questions or research objectives to concretize and inform the reader of the direction the study is going to take.

The materials and methods section is correct and allows to clarify the setting of the research, allowing replicability of the study. In my view, it would be necessary to provide more details on the questionnaires used. The statistical methods are adequate and allow adequate results to be generated.

The results are reliable and properly reported.

The discussion and conclusions are interesting, they contribute to the knowledge about the phenomenon studied, but they would benefit from having specific research objectives to direct the discussion. In any case, changes in this direction would be small.

Author Response

Thank you for taking time to review our document. We addressed the lack of focus in relation to the purpose of the data to enhance the readability of the paper.  We also utilized the research objective to direct the discussion by adding sentences that helped to highlight the relevant results.

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Thank you for the opportunity to review this interesting and timely manuscript. The topic is relevant and the large n across multiple sites obtained by collaborating with evaluators allows for testing of research questions that are usually quite difficult to answer because of insufficient quantitative data.

 

The literature is well organized and well written.

 

Section 2.3: I am a little unclear about the rankings and outputs of higher ed institutions and how they link to the focus on 3-5^th graders. A sentence linking the information about Spellman college to the focus of the literature review would be helpful to ground that section.

 

Section 2.4 details the various summer programs, and read more like a description of the programs considered for the sampling frame than a lit review focused on the findings of research on these programs. The authors may consider breaking this section into 2 parts – discussing the findings of research on co-ed/all girl camps (for lit review) and a discussion of available sites for data collection for the present study (perhaps called “sampling” or “program selection”).

 

I was glad to see intersectionality in the conceptual framing, but I would note that the authors didn’t really pick apart aspects of identity in the analysis or discussion, beyond gender. Though the study focuses on Black girls, the influence of race on the students’ experiences at camp wasn’t really unpacked. I suspect that some of this has to do with the data that were available, and that other marginalized identities (e.g., SES, disability status, language) were not available to the researchers to explore in regression analyses. This is the nature of secondary datasets, so I completely understand. But if intersectionality is used as a frame, I would encourage the researchers to note that these identities are also marginalized in STEM education, and that the analysis wasn’t able to fully explore them in the analysis that was limited to available variables. Though these variables may not be available for the analysis, the discussion could really pull in aspects of intersectionality, especially because the authors had some mixed results.

 

I was not clear about why the authors set exclusion criteria for programs serving fewer than 50 female students. This is probably a simple explanation (statistical power?) but as it is presented now, it reads as if it was arbitrary, and I am sure that it was not.

 

I wanted to know where the survey questions came from (table 3). Did any of them come from validated instruments? It looks like they were written by the evaluators, so any information the authors can provide about the processes for ensuring construct validity or age-appropriateness would be great too include, if it is available.

 

The section 4.6 is well-written, but I don’t think that the authors need to justify the use of evaluation data. I suspect that this may be in response to previous criticisms, so I would just add that as a researcher who sometimes does program evaluation, some of my best data come from evaluations. Data from some student and community populations are especially difficult to come by reliably and in numbers sufficient for statistical analysis. Particularly when working with under-represented groups and populations that have been historically marginalized and harmed by research agendas, the community-provider partnerships that underlie programming like SEEK are sometimes the only way to obtain data en masse. If the authors keep this section in the manuscript, I would encourage them to include citations so the section reads as a methodological strength, rather than an apology. On a related note, I would also note that on line 684 in the discussion, the authors make specific recommendations for the SEEK program, and this sounds like the type of recommendation to be made by an evaluator. As researchers, I encourage the authors to think about the generalizable empirical contributions that this study makes, and to leave specific program-level recommendations to evaluators.

 

Line 451 says that table 5 highlights gains scores between pre- and post-assessments, but the table only includes the p values associated with t-tests of those scores. I would really like to see the means and SDs that are presented in paragraph form in the tables. They are well-presented in the paragraph, but I prefer (and I think other readers would too) to look at tables for the “big picture” and summary stats.

 

I would suggest not presenting figures for the findings that are not statistically significant (e.g., figure 2) and using the figures rhetorically to underscore the significant differences between the two groups.

 

I was a little unclear about why the authors combined “maybe” and “yes” answers, as well as “no” or “not really” in figures 4-9. I think there’s a big difference between “maybe” and “yes” answers. Since the Likert scale only has 5 choices, I don’t think it would be too much to include the additional information in the histograms. Or, alternatively, if the authors have a reason for making this combination, I would just ask that they explain the rationale somewhere around the presentation of the first figure.

 

I’m a little unclear about how engaging with parents (5.6) ties into the RQ; it seems that this answer may have more to do with family dynamics than the program itself. If the authors keep this outcome in the analysis, I’d just encourage them to add a sentence or two to the lit review and discussion about family engagement and how it ties in. Of course it is important for youth career aspirations and school outcomes. But I am just not sure I understand how it links to the SEEK program’s activities.

 

I found outcomes 5.8 and 5.9 to be some of the most interesting findings. The discussion doesn’t draw in much additional literature, and I think this is an area of opportunity. There is a big literature around collaboration and skills for working on teams, as well as how these experiences exclude women and people from non-dominant backgrounds from STEM fields. That this presents as early as elementary school is an interesting finding, and area of opportunity for people running informal STEM programs. I was also very interested to see that bullying/teasing was the one area where co-ed programs seemed to do better than all-girl camps. I would love for the discussion to engage literature that sheds some light on why the researchers identified this dynamic. Overall, I think there is an opportunity to include more literature in the discussion.

 

In the discussion, I strongly encourage the authors to engage more literature to contextualize and explain their findings, and tie them back to prior research, teasing out the new findings or understandings or contributions that this analysis offers.

 

In the discussion, and linking back to the intersectionality framing, I would note that the findings the researchers identified are nuanced. I would caution against making a blanket recommendation for all-girl camps over co-ed, just because all-girl camps performed better on some of the measured outcomes. For starters, there are lots of different ways that these camps could be structured, and these aren’t the only options. But more than that, feeling bullied or picked on is a big deal. Though it was just one survey item and all of the outcomes are important, I’d argue that all items are not qualitatively equal when it comes to evaluating the effectiveness or value of a learning experience. I think there is a lot of nuance here, and I would be careful about extrapolating too many absolute recommendations from finings that are, ultimately, mixed.

 

I am a little confused about why statistically significant differences in pre-assessment scores are presented with outcomes, since the interest is on the effect of the camp as a treatment. Differences in pre-assessments suggest to me that there may be selection bias among who chooses to participate in mixed gender/all-girl camps, and that this bias would need to be accommodated in the inferential stats.

 

Small typos throughout – just needs a light edit. In noted several places where “data” was followed with “is” rather than “are.”

 

Overall, I think the manuscript makes a valuable contribution, and with small changes, would be a valuable piece for the journal.

Reviewer 2 Report

Comments and Suggestions for Authors

Theoretical framework should be embedded more within the discussion. Figure 2 and 3 should be adjusted

Reviewer 3 Report

Comments and Suggestions for Authors

"Single-gender versus coeducation: A comparison of interest and experience outcomes for Black girls within a summer engineering program" research presented a secondary analysis of program evaluation data. Overall, the authors used appropriate citations throughout the introduction, literature review and conceptual framework sections. The relevance and importance of the research to the larger STEM education context is highlighted very well throughout the paper. 

Sec. 1: The Introduction presented clearly articulated arguments regarding Black girls in informal STEM learning spaces with adequate focus on engineering education.  

Sec. 2: The Materials and Methods section presents the relevance of this research to the field and situates the need for such investigations aimed at broadening participation for Black girls in STEM. A reference to the racialized and gendered experiences of Black girls within STEM is lacking overall in the literature review. This connecting literature is missing largely from the research presented thusly requiring the improvement on theoretical background and empirical research on the topic. Although intersectionality is provided as a part of the conceptual framework of this research, the impacts of intersectionality on STEM identity and learning experiences for Black girls could be better supported throughout the literature review. Specifically, there are "three overarching challenges that inhibit Black girls from participating..." (lines 58-67) followed by concerns with discipline that are not woven through the paper such that the discussion/conclusion sections address these challenges. How is coed and single-gender education presented broadly in the literature regardless of race? Additionally, is there a larger connection to disciplinary concerns included in the findings of this research that better supports justifying the reference to the over-discipline of Black girls?

In section 2.3, it is unclear how "education spaces" is being defined. "The team could not find...single gender education spaces" then later the  "I Am STEM" camp which is a single gender informal education program is referenced. Clearly identifying what is understood as a "single gender education space" would benefit the reader.

More information is needed on the NSBE SEEK program for overall context. How might the mission, vision, and values of the program already be aligned with broadening participation in STEM for Black girls? What was the program evaluation outcomes of the SEEK program during the time presented? How could this information inform the single-gender vs coed analysis? Was a purpose of the SEEK program to shift STEM identity development or college-interest in STEM fields? 

Sec 3: The authors presented adequate reasoning and justification for the conceptual framing presented. However, the conceptual framework does not fit the research design or data analysis. It is unclear how intersectionality and social cognitive theory inform the findings of this project.

Sec 4: The research design was very detailed that was clear and appropriate to answer the research question. The data sources identified were analyzed adequately with a very detailed analysis plan. The findings were clearly presented. It is unclear how the conceptual framework informed the data analysis. More justification on why the 3rd-5th grade range was the target population is needed. There is literature around this age group of girls, particularly girls of color losing interest in STEM that would support reasons for highlighting this particular population. The justification for 50 female students per site is unclear. How might analyzing data from sites that were less than 50 female students or less that 30% alongside the data presented here further expand the argument for single-gender education? Since this project uses a secondary data analysis structure, the data source of evaluation data should be presented sooner in the section for clarity since there is no raw, newly collected data.

The overall purpose/goal of the research study to understand the difference between Black female participant interest and experience outcomes at single-gender sites versus coed sites was not answered with the research design or question presented. Data sources that include qualitative data for these sites would better address the research goal presented. 

Sec 5: For empirical research, are the results clearly presented. The results flow sequentially with appropriate supporting tables and graphs. 

Sec 6: The arguments and discussion of findings are not coherent, balanced and compelling and must be improved. Correlation does not equal causation; therefore the arguments made that the increase with certain constructs in a single-gender education space meant more positive outcomes than a coed learning space must be improved. Factors such as impact on mentors at each site, demographics of small groups, differentiated grouping strategies, choice of activities are not broadly discussed in the findings that could have significantly impacted students' experiences regardless of single gender or coed site. Language of "more positive outcomes" and "favorable" should be supported appropriately with strong quantitative and qualitative data. The quantitative data is being connected to broad outcomes such as "more inclusive, positive, self-efficacy- oriented" (line 699) that are not only reliable on single-gender versus coed learning sites. As such, the arguments presented should be strengthened through qualitative data to present a more nuanced approach inclusive of intersectionality and SCT. The conclusions are not thoroughly supported by the results presented in the article.

Grammar concerns:

Lines 29-32 and Lines 37-39 are duplicates.

Line 111- "found ad utilized" - should be "and"