Influence of the Algorithmization Process on the Mathematical Competence: A Case Study of Trainee Teachers Assessing ABN- and CBC-Instructed Schoolchildren by Gamification

Round 1

Reviewer 1 Report

The manuscript has 20 pages, 44 citation references, 8 figures. The manuscript is about the education with an arithmetic-related knowledge for the children aged 12-13. These are educational systems, one of such innovative methodologies is the ABN method (Open Number-Based, by its Spanish acronym); which has been widely faced to the more conventional and spread CBC method (Closed Digit-Based, by its Spanish acronym). The highlights worrisome data: on-service teachers specific lack on mathematical and didactical knowledge. So the authors introduce an Educational Escape Room (EER) as a tool to “learn” from the experience. It helps students in the world of gaming, get a live-evaluation as the very essence of the method. The educational escape room content (in terms of joy, difficulties, and knowledge). That is why I suggest adding “educational escape rooms” to the keywords. Also if the authors use this words in the search (Figure 5. Published documents by year (Scopus database) with the “mathematics” and “escape room” words on their title, abstract or keywords).

In other words, a competency focuses on aspects of actively doing mathematics.

Figure 3 helps us to see the mentality process - Basic operations solved by the traditional (CBC) method. Figure 4 shows the grid (ABN) method (“Open, based in numbers” acronym). It shows single-digit calculations and an ability to memorize a set of procedures. I, personally, also have some issues with mind calculations. So, I see the ABN method easier for me. Also in the conclusions, authors declare “ABN students show similar results with respect to mental calculations (they prefer using paper and pencil), but they also show a higher ability to carry out such mental calculations when mandatory”.

the playgroups

L. 191 “schoolteachers”, better to write separately “school teachers”,

L. 215 “system”. Point at the end of the sentence.

L. 280 citation is wrong. Authors write Freudenthal [32]. However, [32] is B. Zolkower, A. M. Bressan, S. Pérez, and M. F. Gallego

 

L. 742 delete “The APC was funded by XXX”.

L. 90-98 the objectives of this study can be summarized as follow O1-O4. However, I don’t see any citation in the text or conclusions, that O1 was done, O2 was reached with …tool. So, do we actually need to number them? Or it is here just for the info? Maybe we could observe simply pointed list?

 

Check references citation style: surnames firstly, then names.

Research Involving Human Subjects: as I understand, the students were under video surveillance (l. 368, 408-409). It means that this point should be mention in the end of the manuscript. “Research Ethics” authors must declare that the investigations were carried out following the rules of the Declaration of Helsinki of 1975 (https://www.wma.net/what-we-do/medical-ethics/declaration-of-helsinki/), revised in 2013.

Author Response

This manuscript has been reviewed by 3 referees. Ammendments to the manuscript have been included and highglighted.

Reviewer#1 pondering have been considered and the original manuscript has been modified. A detailed letter to R#1, providing an answer to each question, is attached to this reply.

Author Response File: Author Response.pdf

Reviewer 2 Report

This paper presents a didactic experiment using an “educational escape room” learning process, aimed at determining the effectiveness of selected teaching methods (ABN and CBC) in the mathematical competence of elementary students.

The experiment is adequately described, and the roles of the stakeholders are analytically explained. To my knowledge, the theoretical part of the paper is fine.  

However, the actual research results of the study i.e. the comparison of the students’ performance trained by the two methodologies, are given narratively (just reporting the conversations of the students). With the present analysis, neither a reader nor a researcher is capable of deciding if the educational goals are achieved or making the interpretation in the didactical context. Moreover, there is no inferential statistics part. In fact, there is no statistical analysis! This makes the paper a case study report!

Some crucial questions that need to be answered are:

How do the authors quantify the mathematical skills?

Which are the variables under study?

In which metrics is the analysis based?

A robust and reliable “Results and Discussion” section should include the variables used to quantify the mathematical skills tested, with the associated sample data per playgroup and student reported and given in a comparative representation (in a table or in a graph) for the ABN and CBC methods. The outcomes of the two methods should be compared using statistical tests (e.g. independent sample T-test, chi-square test of independence) with specific statistical significance.

The same holds for the “4.3 Profits for student teachers” section. The authors do not substantiate their claims scientifically, in an evidence-based way. In lines 690-694 of the manuscript, they claim that

“On the other hand, by the end of the experience, the involved student teachers were interviewed and asked about the experience in terms of professional didactic-mathematic knowledge. They all stated that they felt much more confident in their capabilities to instruct mathematics, showing enthusiasm to mathematics learning, and improving the self-belief for being able to adapt educational environments; finally, they all confess to be more passionate to engage innovative practices to the class.”

 

The results should be based on quantitative research methods (i.e. questionnaire survey) or qualitative research methods (i.e. in-depth interviews). None of these practices is applied here. The manuscript lacks crucial information, such as the sample used for the above statement, the number of trainee teachers involved, some words on their scientific profile, teaching experience, e.t.c. And most important of all, were there any text analytics or statistical study to come to this conclusion?

Author Response

This manuscript has been reviewed by 3 referees. Ammendments to the manuscript have been included and highglighted.

Reviewer#2 pondering have been considered and the original manuscript has been modified. A detailed letter to R#2, providing an answer to each question, is attached to this reply.

Author Response File: Author Response.pdf

Reviewer 3 Report

The paper investigates educational gamification methodologies in Mathematics, especially an Escape Room involving K-6 and K-7 students with diverse backgrounds and using different methodologies for algorithmic calculations (ABN and CBC).

 

Major remarks

The paper is well written and organized in its various sections, the content is very interesting.

However the scientific soundness of the results can be objected: the spotted reductions and improvements are not collected in a significant quantitative way, but only qualitatively. Recorded speeches should be coded to detect those elements that were highlighted in the discussion section as main findings.

 

Minor remarks

• 3 references in Spanish. It is allowed to cite references in other languages beyond English, but this influences the readability and repeatability of the experimentation

• Url of 12 is a google url, not the direct one

• Some parts of section 2 are maybe too wordy: is it necessary to detail CBC algorithm? Is it necessary to fully detail the diverse kind of assessment (diagnostic, formative, summative) quite known in literature?

• Materials and Methods should describe how you carry on research, I did not find it

 

Author Response

This manuscript has been reviewed by 3 referees. Ammendments to the manuscript have been included and highglighted.

Reviewer#3 pondering have been considered and the original manuscript has been modified. A detailed letter to R#3, providing an answer to each question, is attached to this reply.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

The authors have not adequately addressed my comments.

The authors make clear that their manuscript describes an educational experiment, with no evidence-based analysis. The trainee teachers involved stated their experience, without further in-depth analysis (lines 699-715 of the revised manuscript). 

I consider the whole study as a pilot study for an experimental design, that could use statistical reasoning to assess the results.  I would be happy to see this work moving to its next stage, involving a sufficient number of participants and a robust analysis.

 

Author Response

We've updated the debate with R#2 by including statistical data. A chi-sqr test has been delivered and we propose to update the manuscript with an additional discussion page. Attached, R#2 can find our reply and the proposed text. Any conflictive statement, considered by R#2, will be removed if needed (indeed, section "profits for student teachers" could be entirely removed if required).

We'd also like to thank to R#2 for the time spent on this review.

Author Response File: Author Response.pdf