Active Learning in STEM Education with Regard to the Development of Inquiry Skills

Round 1

Reviewer 1 Report

Thank you for submitting this paper. The central premise of inquiry based approaches is well written and logically presented. A few points to be attended to.

line 24,25 claim needs to be substantiated-this is one of the main cruxes of the paper-IBSE and inquiry skills.

line 43 Satchwell &Lopp (2002) does not support the claim of the origins of Inquiry based learning in science and other evidence is required. 

line 57 the calls for improvement to STEM has been since the 1980's refer to STEMmania artciel no. (18) or Bybee's work article no. (17).

lines 54-78 The definition of STEM remains contested, this needs to be explicitly stated as the reader is left thinking these are the only defintions.

line 278-283 An elaborated explanation on what quasi-experimental model means in this context and how the schools were selected-randomly, pre-determined?

line 290 why were these outliers excluded? it is obvious but needs to be explained.

line 325-335 needs re-writing. The authors assumptions on homogeneity are correct due to the possible false rejecting the null hypothesis but a simpler and more concise explanation can be presented ie. a Welch ANOVA was used to compare means . I understand why the Wald test measure was used (assuming the parametric statistical testing was required) to confirm whether the independent variables were collectively 'significant' but an explanation as to why is required or selected in preference to other test measures 

line  381 and 392 to be unclear and needs to be re-written and presented in more conventional manner. It would also be useful to have the effect size in a or one of the tables. 

line 211 raises the spectre of gender but this is not addressed pg 1-5 but is a significant part of the analysis ie. table 2 and results and is referred to throughout the remaining part of the paper particularly from line 503 in the conclusion. 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

The manuscript is well organized. 

Authors should review the referencing style along the manuscript.

Please consider the following aspects to improve the paper:

- The abstract should be completed with the identification of the methodology and method used.

- it would be enlightening for the reader to understand, through the Introduction, the pertinence of the study and why the authors decided to do it. Also, it should include a previous discussion where active learning, IBSE, STEM Education, digital technologies and formative assessment are articulated presenting a more extensive discussion, so the sections of the theoretical framework are better perceived.

- line 23, the statement is very strong and reductive in which concerns active learning. If we conduct a thorough literature review, we may conclude that active learning is not only associated with IBSE.

- the main concepts of the study should be clearly delimited and grounded on the literature. For instance: what are the principles of active learning? What entails IBSE? 

- The aim of the study should be clearly delimited and defined always in the same manner along the different sections of the paper. In what sense does the 1^st research question differ from the research problem? They look exactly the same.

- The methodological options should be theoretically sustained and the method should be identified; What criteria were used to select the participating schools?; Which students were integrated (grade levels) and why? Criteria for the sample selection?

Its important to clarify the reader about the nature of the activities implemented during the intervention.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

Thank you for the opportunity to review this manuscript. In general, the manuscript is organised and presents information systematically. Nonetheless, the authors might consider proof-reading it once more to correct minor errors (e.g., Cohen should be Cohen's on p.8 l.323). To enhance readability and value add the potential inclusion of this manuscript in Ed Sciences, I'd suggest to the authors to consider the following:

1. The IBSE was presented to be associated closely with "active learning". However, it would be useful to draw up a table that compares similarities/differences of contrasting frameworks (e.g., 5E instructional model) that are also touted to support inquiry in Science. This table could be accompanied with a paragraph discussing why was the IBSE (and not other models) was used in this study. It is noteworthy that "active learning" is a large body of study on its own.

2. Please update Figure 2 (and with a short paragraph) to illustrate what is the time between the pre- and post-test. Pre- and post-testing methods generally are accompanied by confounds, and the authors should consider explaining how some of the common confounds were managed. For example, in this study, it remains unknown whether students had "practice effect"; if there was really "practice effect", the results based on paired t-test might not be valid. Further, the authors could discuss to what extent was the experiment controlled. For example, it was noted that teachers with knowledge if IBSE used formative assessment and digital technologies as part of the intervention. Were there controls to ascertain that the pedagogical approaches were similar (or within reasonable expectations) across schools/teachers?

3. Adding to point 2, it was noted that IBSE was implemented across various disciplines from 2016-2022. How might the authors discuss/suggest that the results were due to the IBSE intervention, and that subject difficulty didn't play a statistically significant role in this?

3. It might be useful to use Kraft's effect size for educational interventions (https://doi.org/10.3102%2F0013189X20912798) rather than Cohen's. Kraft's is less conservative, and might suggest a stronger effect size. 

4. Please include the ethics approval under "methods". How were the participants approached, and were participants given any option to withdraw etc?

Overall, this manuscript would be a contribution to scholarship in particular if points 2 and 3 are addressed adequately. The descriptive stats presented and t-test methods applied would be strengthened if there is sufficient discussion on how confounds to the study were managed.

 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

There are some minor grammatical issues, I think my only concern now is the use of a control and the experimental/empirical results/discussions. Once this is resolved I am happy for it to be published.

Author Response

Thank you very much for your review:

• language proofreading was done by a professional
• discussion part was complemented based also on the comments of other reviewers

Reviewer 2 Report

The authors took into consideration the majority of the suggestions and comments made in the 1^st review. However: the referencing style still needs to be corrected along the text; and looking at the research problem and associated questions, I continue to not see a substantial difference between the 1^st research question and the research problem.

Author Response

Thank you very much for your review:

• The referencing style was checked again and modified using Zotero taking into account the template of the Educational Sciences Journal.
• The research problem was reformulated ,
  “…the main research problem of this study concerns the efficacy of IBSE with regard to inquiry skills development.”
  This is a central problem investigated in this study – IBSE and inquiry skills. The research questions are focusing on specific issues (1^st – development of inquiry skill group – the group of six skills as a whole is evaluated with the help of test of inquiry skills, 2^nd-individual inquiry skills – particular skills are evaluated and analyzed, 3^rd –parameters influencing the effect). So that the research questions, namely the 1^st question  was edited, in this sense:
  1. What is the effect of IBSE, if implemented consistently across several STEM related disciplines, on the level of selected inquiry skill group development?
  2. What is the effect of IBSE, if implemented consistently across several STEM related disciplines, on the development of selected inquiry skills?
  3. How do different factors (gender and number of implemented inquiry activities) influence the level of inquiry skills development?

• Some minor changes were made in the following text to be consistent with the research questions (in the 4.1, 4.2 and 4.3 paragraph with regard to the terms: inquiry skill group and selected inquiry skills)

Reviewer 3 Report

Thanks for addressing the earlier comments. It is noteworthy that that lines 487-490 specifies that the findings of this study are consistent with those earlier with the exception that the earlier studies focussed on one STEM related course. I believe the foci on particular STEM subjects are not unfounded, given the confounds that subject difficulty brings to a study on inquiry where researchers would like to have as much control as possible over variables. Including a detailed discussion of your response to 3.3 (of the previous comments) within this discussion section would make overt the contribution of your manuscript to scholarship, since you shared that there is a dearth of research related to studies in this field. An effort should be made to clarify how subject difficulty might confound the study outcomes; an admission of this could also be possible.

I am also concerned with some of the validity issues presented, though these could have been attributed by typos. For example, lines 325-330 mentioned "geography" but I was not able to observe this in Table 2. The claim made about inquiry development influenced by STEM subjects could well be attributable to geography but how is geography a STEM related subject? A para on this should be included else the outcomes are confounding for the reader. 

Author Response

Thank you very much for your comments:

• We would like to explain our approach to this issue. We were looking into the effect of IBSE on inquiry skills development. The inquiry activities for teaching STEM subjects were designed to target the development of inquiry skills, whether for physics, chemistry or any other subject. And STEM subjects have the largest potential for developing these skills. We believe that when implementing IBSE across the STEM related subjects, it is not the subject content but the approach to teaching and learning that is important in this case. The reason why we conducted this research was to see the synergetic effect – if students are exposed to this method regularly and frequently across more subjects, the effect might be better than if they experience this approach from time to time just within a single subject. The subject difficulty is a parameter not clearly defined in the literature, usually is based on students´ perception of “difficulty”. This perception can change with age. Nevertheless, mathematics, chemistry and physics, in particular, is usually at the end of the list. Some researches put forward the mathematical requirements of the subject as the main reason. (Physics is seen as mathematical subject whereas biology not). There is a lack of relevant studies in the field of subject difficulty. This is a rather complicated issue and it is very difficult to connect with our research. 

On the other hand, what is important and common for all STEM related subjects is that the lesson plans followed the same structure. In each plan it was explicitly stated which inquiry skills the lesson focuses on since the activities were designed to target the development of inquiry skills, in particular. It is explained in the text in more details – line 284 in the edited version. “Each lesson plan was developed based on a pre-agreed structure common for all subjects. It started with an overview of the learning objectives with explicitly specified inquiry skills developed in the lesson using the framework in Table 1, teaching aids and materials needed for the lesson. Subsequently, the expected misconceptions and designed level of inquiry were addressed. This overview was followed by a detailed description of the teaching and learning scenario. The concept described served as the common framework for all subjects” .

In addition, we also added a sentence in line 500 in the edited version to explain how we perceive this issue: “Unlike the aforementioned studies, this research investigates implementation of IBSE across several STEM related subjects. The question is whether the subject difficulty might affect the gained results. It was assumed that the inquiry approach to teaching and learning was crucial and the subject content did not play a determining role in this case. The inquiry activities in all subjects were designed based on the agreed framework to target the development of inquiry skills in all subjects concerned.“

• Concerning the issue of geography: Geography studies Earth´s surface - its physical and human-related features and relationships between them. It was included in the research as a large part of its curriculum covers physical geographical knowledge with the emphasize on students´ active learning. 

The sentence explaining why geography was also included, is added to the text (line 332 in the edited version): “Even though geography is considered a bridge between natural and social science, it was included into the STEM group as a large part of geography curriculum covers physical geography (branch of natural science). The IBSE approach has a large potential in this field as well.”

The table 2 represents skills and subject matter covered in specific test items. With regard to the previous comment, the test item assessing the skill of formulating hypothesis concerns the earth and oceans (as can be seen in Table 6), however unfortunately geography was not explicitly mentioned there. The table was completed with this information. 

 

Round 3

Reviewer 3 Report

Thanks for addressing the comments. It's noteworthy that effort has been taken to proofread the manuscript. There remains a few typos possibly due to the formatting and conversion process. For example:

1. Line 316 "error reference not found"

2. Line 426 - consider "Cohen's d for this improvement was 0.48, which suggested a moderate effect size [116]"

 

Author Response

Thank you for pointing to these errors. The paper was checked again and the references were corrected and updated and some other mistakes were fixed.